2024-02-17T11:37:03.035 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-17T11:37:03.035 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi --owner scheduled_yuriw@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} --verbose --archive /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122 -- /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122/orig.config.yaml 2024-02-17T11:37:03.095 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-17T11:37:03.184 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122 branch: main description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} email: yweinste@redhat.com first_in_suite: false job_id: '7563122' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 6822c7b54e949006be7a0802650d01723d834d1d subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 6822c7b54e949006be7a0802650d01723d834d1d kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=secure - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 6822c7b54e949006be7a0802650d01723d834d1d owner: scheduled_yuriw@teuthology priority: 51 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2794 sha1: 6822c7b54e949006be7a0802650d01723d834d1d sleep_before_teardown: 0 subset: 111/120000 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 6822c7b54e949006be7a0802650d01723d834d1d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG6smY1OgeolRnBZxzg0cPkWaGtYzRTvBjeOAvk+Tg7hEfmIaXh6F9yZ0rlGIOCxRA/TDXBEJdoarl7ytQsTcKU= smithi125.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDlXkWaPWNtFn8/H+ch7nJ/biNFnxA6pCB6Cb7U4zEV6p7vu9ABQzXPtZbFiyNcU8jchLsjEuYjG+5N3aMp6Jzc= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNYrbB3c5Fp9foewg7kcNImMuQ/hpiCvDnGIYCbqCjSVXftEuoy3ssLVmWZr7yzeMAZllzl5M9UupWehnnHwTSA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-16_20:13:25 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-17T11:37:03.184 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa; will attempt to use it 2024-02-17T11:37:03.185 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks 2024-02-17T11:37:03.185 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-17T11:37:03.187 INFO:teuthology.task.internal:Checking packages... 2024-02-17T11:37:03.210 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6822c7b54e949006be7a0802650d01723d834d1d' 2024-02-17T11:37:03.210 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-17T11:37:03.210 INFO:teuthology.packaging:ref: None 2024-02-17T11:37:03.210 INFO:teuthology.packaging:tag: None 2024-02-17T11:37:03.210 INFO:teuthology.packaging:branch: main 2024-02-17T11:37:03.210 INFO:teuthology.packaging:sha1: 6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T11:37:03.211 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-02-17T11:37:03.459 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1543.g0ed764e6 2024-02-17T11:37:03.460 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-17T11:37:03.468 INFO:teuthology.task.internal:no buildpackages task found 2024-02-17T11:37:03.468 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-17T11:37:03.477 INFO:teuthology.task.internal:Saving configuration 2024-02-17T11:37:03.499 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-17T11:37:03.506 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-17T11:37:03.528 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 11:31:48.489394', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG6smY1OgeolRnBZxzg0cPkWaGtYzRTvBjeOAvk+Tg7hEfmIaXh6F9yZ0rlGIOCxRA/TDXBEJdoarl7ytQsTcKU='} 2024-02-17T11:37:03.544 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 11:31:48.487684', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDlXkWaPWNtFn8/H+ch7nJ/biNFnxA6pCB6Cb7U4zEV6p7vu9ABQzXPtZbFiyNcU8jchLsjEuYjG+5N3aMp6Jzc='} 2024-02-17T11:37:03.563 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 11:31:48.490480', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNYrbB3c5Fp9foewg7kcNImMuQ/hpiCvDnGIYCbqCjSVXftEuoy3ssLVmWZr7yzeMAZllzl5M9UupWehnnHwTSA='} 2024-02-17T11:37:03.564 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-17T11:37:03.571 INFO:teuthology.task.internal:roles: ubuntu@smithi094.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-02-17T11:37:03.571 INFO:teuthology.task.internal:roles: ubuntu@smithi125.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-02-17T11:37:03.571 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-02-17T11:37:03.571 INFO:teuthology.run_tasks:Running task console_log... 2024-02-17T11:37:03.665 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f22c3bf7670>, signals=[15]) 2024-02-17T11:37:03.665 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-17T11:37:03.672 INFO:teuthology.task.internal:Opening connections... 2024-02-17T11:37:03.672 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi094.front.sepia.ceph.com 2024-02-17T11:37:03.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T11:37:03.810 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi125.front.sepia.ceph.com 2024-02-17T11:37:03.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T11:37:03.913 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-02-17T11:37:03.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T11:37:04.005 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-17T11:37:04.012 DEBUG:teuthology.orchestra.run.smithi094:> uname -m 2024-02-17T11:37:04.030 INFO:teuthology.orchestra.run.smithi094.stdout:x86_64 2024-02-17T11:37:04.031 DEBUG:teuthology.orchestra.run.smithi094:> cat /etc/os-release 2024-02-17T11:37:04.087 INFO:teuthology.orchestra.run.smithi094.stdout:NAME="CentOS Stream" 2024-02-17T11:37:04.087 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION="9" 2024-02-17T11:37:04.088 INFO:teuthology.orchestra.run.smithi094.stdout:ID="centos" 2024-02-17T11:37:04.088 INFO:teuthology.orchestra.run.smithi094.stdout:ID_LIKE="rhel fedora" 2024-02-17T11:37:04.088 INFO:teuthology.orchestra.run.smithi094.stdout:VERSION_ID="9" 2024-02-17T11:37:04.088 INFO:teuthology.orchestra.run.smithi094.stdout:PLATFORM_ID="platform:el9" 2024-02-17T11:37:04.088 INFO:teuthology.orchestra.run.smithi094.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-17T11:37:04.088 INFO:teuthology.orchestra.run.smithi094.stdout:ANSI_COLOR="0;31" 2024-02-17T11:37:04.088 INFO:teuthology.orchestra.run.smithi094.stdout:LOGO="fedora-logo-icon" 2024-02-17T11:37:04.088 INFO:teuthology.orchestra.run.smithi094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-17T11:37:04.088 INFO:teuthology.orchestra.run.smithi094.stdout:HOME_URL="https://centos.org/" 2024-02-17T11:37:04.088 INFO:teuthology.orchestra.run.smithi094.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-17T11:37:04.088 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-17T11:37:04.088 INFO:teuthology.orchestra.run.smithi094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-17T11:37:04.089 INFO:teuthology.lock.ops:Updating smithi094.front.sepia.ceph.com on lock server 2024-02-17T11:37:04.109 DEBUG:teuthology.orchestra.run.smithi125:> uname -m 2024-02-17T11:37:04.126 INFO:teuthology.orchestra.run.smithi125.stdout:x86_64 2024-02-17T11:37:04.126 DEBUG:teuthology.orchestra.run.smithi125:> cat /etc/os-release 2024-02-17T11:37:04.182 INFO:teuthology.orchestra.run.smithi125.stdout:NAME="CentOS Stream" 2024-02-17T11:37:04.182 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION="9" 2024-02-17T11:37:04.183 INFO:teuthology.orchestra.run.smithi125.stdout:ID="centos" 2024-02-17T11:37:04.183 INFO:teuthology.orchestra.run.smithi125.stdout:ID_LIKE="rhel fedora" 2024-02-17T11:37:04.183 INFO:teuthology.orchestra.run.smithi125.stdout:VERSION_ID="9" 2024-02-17T11:37:04.183 INFO:teuthology.orchestra.run.smithi125.stdout:PLATFORM_ID="platform:el9" 2024-02-17T11:37:04.183 INFO:teuthology.orchestra.run.smithi125.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-17T11:37:04.183 INFO:teuthology.orchestra.run.smithi125.stdout:ANSI_COLOR="0;31" 2024-02-17T11:37:04.183 INFO:teuthology.orchestra.run.smithi125.stdout:LOGO="fedora-logo-icon" 2024-02-17T11:37:04.183 INFO:teuthology.orchestra.run.smithi125.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-17T11:37:04.183 INFO:teuthology.orchestra.run.smithi125.stdout:HOME_URL="https://centos.org/" 2024-02-17T11:37:04.183 INFO:teuthology.orchestra.run.smithi125.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-17T11:37:04.183 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-17T11:37:04.183 INFO:teuthology.orchestra.run.smithi125.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-17T11:37:04.184 INFO:teuthology.lock.ops:Updating smithi125.front.sepia.ceph.com on lock server 2024-02-17T11:37:04.202 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-02-17T11:37:04.220 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-02-17T11:37:04.220 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-02-17T11:37:04.277 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="CentOS Stream" 2024-02-17T11:37:04.277 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="9" 2024-02-17T11:37:04.277 INFO:teuthology.orchestra.run.smithi189.stdout:ID="centos" 2024-02-17T11:37:04.277 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE="rhel fedora" 2024-02-17T11:37:04.277 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="9" 2024-02-17T11:37:04.278 INFO:teuthology.orchestra.run.smithi189.stdout:PLATFORM_ID="platform:el9" 2024-02-17T11:37:04.278 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-17T11:37:04.278 INFO:teuthology.orchestra.run.smithi189.stdout:ANSI_COLOR="0;31" 2024-02-17T11:37:04.278 INFO:teuthology.orchestra.run.smithi189.stdout:LOGO="fedora-logo-icon" 2024-02-17T11:37:04.278 INFO:teuthology.orchestra.run.smithi189.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-17T11:37:04.278 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://centos.org/" 2024-02-17T11:37:04.278 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-17T11:37:04.278 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-17T11:37:04.278 INFO:teuthology.orchestra.run.smithi189.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-17T11:37:04.278 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-02-17T11:37:04.301 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-17T11:37:04.313 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-17T11:37:04.319 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-17T11:37:04.319 DEBUG:teuthology.orchestra.run.smithi094:> test '!' -e /home/ubuntu/cephtest 2024-02-17T11:37:04.322 DEBUG:teuthology.orchestra.run.smithi125:> test '!' -e /home/ubuntu/cephtest 2024-02-17T11:37:04.324 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-02-17T11:37:04.340 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-17T11:37:04.348 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-17T11:37:04.348 DEBUG:teuthology.orchestra.run.smithi094:> test -z $(ls -A /var/lib/ceph) 2024-02-17T11:37:04.379 DEBUG:teuthology.orchestra.run.smithi125:> test -z $(ls -A /var/lib/ceph) 2024-02-17T11:37:04.381 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-02-17T11:37:04.704 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-17T11:37:04.817 INFO:teuthology.run_tasks:Running task kernel... 2024-02-17T11:37:04.832 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-02-17T11:37:04.833 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-02-17T11:37:04.833 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-02-17T11:37:04.833 DEBUG:teuthology.orchestra.run.smithi094:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-17T11:37:04.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T11:37:04.850 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-02-17T11:37:04.907 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-361.el9.x86_64 2024-02-17T11:37:04.908 INFO:teuthology.task.kernel:Running kernel on smithi094: 5.14.0-361.el9.x86_64 2024-02-17T11:37:04.908 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum install -y kernel 2024-02-17T11:37:07.365 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - AppStream 8.8 MB/s | 6.7 MB 00:00 2024-02-17T11:37:09.132 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2024-02-17T11:37:11.998 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.0 MB 00:02 2024-02-17T11:37:13.958 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-02-17T11:37:19.675 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.1 MB 00:02 2024-02-17T11:37:21.451 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 15 kB 00:00 2024-02-17T11:37:23.770 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 9.3 MB/s | 20 MB 00:02 2024-02-17T11:37:28.586 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-02-17T11:37:30.860 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-17T11:37:30.860 INFO:teuthology.orchestra.run.smithi094.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-17T11:37:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-02-17T11:37:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-02-17T11:37:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-02-17T11:37:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-02-17T11:37:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-02-17T11:37:30.984 INFO:teuthology.orchestra.run.smithi094.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-17T11:37:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-02-17T11:37:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-17T11:37:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-17T11:37:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-17T11:37:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:37:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-02-17T11:37:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:================================================================================ 2024-02-17T11:37:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:Install 4 Packages 2024-02-17T11:37:30.985 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:37:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 100 M 2024-02-17T11:37:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:Installed size: 123 M 2024-02-17T11:37:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-02-17T11:37:31.513 INFO:teuthology.orchestra.run.smithi094.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 11 MB/s | 6.1 MB 00:00 2024-02-17T11:37:32.065 INFO:teuthology.orchestra.run.smithi094.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 19 MB/s | 21 MB 00:01 2024-02-17T11:37:32.958 INFO:teuthology.orchestra.run.smithi094.stdout:(3/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 20 MB/s | 39 MB 00:01 2024-02-17T11:37:33.374 INFO:teuthology.orchestra.run.smithi094.stdout:(4/4): kernel-modules-core-5.14.0-421.el9.x86_6 18 MB/s | 33 MB 00:01 2024-02-17T11:37:33.377 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-02-17T11:37:33.378 INFO:teuthology.orchestra.run.smithi094.stdout:Total 42 MB/s | 100 MB 00:02 2024-02-17T11:37:33.378 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-02-17T11:37:33.743 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-02-17T11:37:33.744 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-02-17T11:37:34.764 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-02-17T11:37:34.764 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-02-17T11:37:37.173 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-02-17T11:37:37.878 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-17T11:37:38.003 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-17T11:37:40.730 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-17T11:37:40.911 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-17T11:37:45.642 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-17T11:37:45.765 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-17T11:37:50.567 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-17T11:38:19.823 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-17T11:38:19.823 INFO:teuthology.orchestra.run.smithi094.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-17T11:38:19.824 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:38:19.891 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-17T11:38:20.356 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-17T11:38:20.356 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-17T11:38:20.356 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-17T11:38:20.356 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-17T11:38:20.804 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-17T11:38:20.804 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:38:20.804 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-02-17T11:38:20.804 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-17T11:38:20.804 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-17T11:38:20.804 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-17T11:38:20.804 INFO:teuthology.orchestra.run.smithi094.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-17T11:38:20.804 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:38:20.804 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-02-17T11:38:21.311 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-02-17T11:38:21.440 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-17T11:38:21.440 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-17T11:38:21.440 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-17T11:38:21.440 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-17T11:38:21.440 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-17T11:38:21.441 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-02-17T11:38:21.441 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi094.front.sepia.ceph.com, path=None, version=distro) 2024-02-17T11:38:21.441 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q kernel | sort -rV | head -n 1 2024-02-17T11:38:21.548 INFO:teuthology.orchestra.run.smithi094.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-17T11:38:21.549 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-17T11:38:21.549 DEBUG:teuthology.orchestra.run.smithi094:> sudo rpm -qi grub2-tools 2024-02-17T11:38:21.582 INFO:teuthology.orchestra.run.smithi094.stdout:Name : grub2-tools 2024-02-17T11:38:21.582 INFO:teuthology.orchestra.run.smithi094.stdout:Epoch : 1 2024-02-17T11:38:21.582 INFO:teuthology.orchestra.run.smithi094.stdout:Version : 2.06 2024-02-17T11:38:21.582 INFO:teuthology.orchestra.run.smithi094.stdout:Release : 68.el9 2024-02-17T11:38:21.582 INFO:teuthology.orchestra.run.smithi094.stdout:Architecture: x86_64 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout:Group : Unspecified 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout:Size : 8264254 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout:License : GPLv3+ 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout:Packager : builder@centos.org 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout:Vendor : CentOS 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout:Summary : Support tools for GRUB. 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout:Description : 2024-02-17T11:38:21.583 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:38:21.584 INFO:teuthology.orchestra.run.smithi094.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-17T11:38:21.584 INFO:teuthology.orchestra.run.smithi094.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-17T11:38:21.584 INFO:teuthology.orchestra.run.smithi094.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-17T11:38:21.584 INFO:teuthology.orchestra.run.smithi094.stdout:hardware devices. 2024-02-17T11:38:21.584 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:38:21.584 INFO:teuthology.orchestra.run.smithi094.stdout:This subpackage provides tools for support of all platforms. 2024-02-17T11:38:21.585 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-17T11:38:21.585 INFO:teuthology.task.kernel:Updating grub on smithi094 to boot 5.14.0-421.el9.x86_64 2024-02-17T11:38:21.586 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-17T11:38:22.223 INFO:teuthology.orchestra.run.smithi094.stderr:Generating grub configuration file ... 2024-02-17T11:38:23.615 INFO:teuthology.orchestra.run.smithi094.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-17T11:38:23.641 INFO:teuthology.orchestra.run.smithi094.stderr:done 2024-02-17T11:38:23.644 DEBUG:teuthology.orchestra.run.smithi094:> mktemp 2024-02-17T11:38:23.659 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/tmp.ANLvMvVCGh 2024-02-17T11:38:23.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ANLvMvVCGh 2024-02-17T11:38:23.727 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0666 /tmp/tmp.ANLvMvVCGh 2024-02-17T11:38:23.870 DEBUG:teuthology.orchestra.remote:smithi094:/tmp/tmp.ANLvMvVCGh is 6KB 2024-02-17T11:38:23.920 DEBUG:teuthology.orchestra.run.smithi094:> rm -fr /tmp/tmp.ANLvMvVCGh 2024-02-17T11:38:23.934 DEBUG:teuthology.orchestra.run.smithi094:> sudo /bin/ls /boot/loader/entries || true 2024-02-17T11:38:23.999 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-17T11:38:23.999 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-17T11:38:23.999 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-17T11:38:23.999 INFO:teuthology.orchestra.run.smithi094.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-17T11:38:24.001 DEBUG:teuthology.orchestra.run.smithi094:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-17T11:38:24.159 DEBUG:teuthology.orchestra.run.smithi094:> sudo shutdown -r now 2024-02-17T11:38:54.162 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-02-17T11:38:54.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-17T11:38:54.164 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T11:39:12.572 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-02-17T11:39:21.573 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-17T11:39:21.574 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T11:39:24.636 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.94 2024-02-17T11:39:36.637 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-17T11:39:36.638 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T11:40:08.456 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-02-17T11:40:08.906 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-17T11:40:08.906 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-421.el9.x86_64"... 2024-02-17T11:40:08.906 DEBUG:teuthology.orchestra.run.smithi094:> uname -r 2024-02-17T11:40:08.920 INFO:teuthology.orchestra.run.smithi094.stdout:5.14.0-421.el9.x86_64 2024-02-17T11:40:08.920 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-17T11:40:08.920 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-17T11:40:08.920 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-17T11:40:09.921 DEBUG:teuthology.parallel:result is None 2024-02-17T11:40:09.922 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-17T11:40:09.932 INFO:teuthology.task.internal:Creating test directory... 2024-02-17T11:40:09.933 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-17T11:40:09.935 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-17T11:40:09.937 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-17T11:40:09.955 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-17T11:40:09.994 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-17T11:40:10.003 INFO:teuthology.task.internal:Creating archive directory... 2024-02-17T11:40:10.004 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-17T11:40:10.006 DEBUG:teuthology.orchestra.run.smithi125:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-17T11:40:10.009 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-17T11:40:10.308 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-17T11:40:10.315 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-17T11:40:10.315 DEBUG:teuthology.orchestra.run.smithi125:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-17T11:40:10.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T11:40:10.332 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-17T11:40:10.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T11:40:10.349 DEBUG:teuthology.orchestra.run.smithi094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-17T11:40:10.351 DEBUG:teuthology.orchestra.run.smithi125:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-17T11:40:10.375 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-17T11:40:10.418 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-17T11:40:10.421 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-17T11:40:10.438 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-17T11:40:10.441 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-17T11:40:10.468 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-17T11:40:10.478 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-17T11:40:10.480 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-17T11:40:10.487 INFO:teuthology.task.internal:Configuring sudo... 2024-02-17T11:40:10.488 DEBUG:teuthology.orchestra.run.smithi094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-17T11:40:10.490 DEBUG:teuthology.orchestra.run.smithi125:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-17T11:40:10.492 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-17T11:40:10.522 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-17T11:40:10.531 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-17T11:40:10.531 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-17T11:40:10.561 DEBUG:teuthology.orchestra.run.smithi125:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-17T11:40:10.564 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-17T11:40:10.580 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-17T11:40:10.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-17T11:40:10.710 DEBUG:teuthology.orchestra.run.smithi094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-17T11:40:10.793 DEBUG:teuthology.orchestra.run.smithi094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-17T11:40:10.840 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:40:10.840 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-17T11:40:10.909 DEBUG:teuthology.orchestra.run.smithi125:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-17T11:40:10.955 DEBUG:teuthology.orchestra.run.smithi125:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-17T11:40:11.017 DEBUG:teuthology.orchestra.run.smithi125:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-17T11:40:11.060 DEBUG:teuthology.orchestra.run.smithi125:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-17T11:40:11.106 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-17T11:40:11.106 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-17T11:40:11.175 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-17T11:40:11.218 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-17T11:40:11.268 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-17T11:40:11.352 DEBUG:teuthology.orchestra.run.smithi189:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-17T11:40:11.397 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-17T11:40:11.397 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-17T11:40:11.467 DEBUG:teuthology.orchestra.run.smithi094:> sudo service rsyslog restart 2024-02-17T11:40:11.469 DEBUG:teuthology.orchestra.run.smithi125:> sudo service rsyslog restart 2024-02-17T11:40:11.471 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-02-17T11:40:11.509 INFO:teuthology.orchestra.run.smithi125.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-17T11:40:11.538 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-17T11:40:11.545 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-17T11:40:12.017 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-17T11:40:12.025 INFO:teuthology.task.internal:Starting timer... 2024-02-17T11:40:12.025 INFO:teuthology.run_tasks:Running task pcp... 2024-02-17T11:40:12.035 INFO:teuthology.run_tasks:Running task selinux... 2024-02-17T11:40:12.042 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-17T11:40:12.043 DEBUG:teuthology.orchestra.run.smithi094:> sudo service auditd rotate 2024-02-17T11:40:12.128 INFO:teuthology.orchestra.run.smithi094.stdout:Rotating logs: 2024-02-17T11:40:12.133 DEBUG:teuthology.orchestra.run.smithi125:> sudo service auditd rotate 2024-02-17T11:40:12.214 INFO:teuthology.orchestra.run.smithi125.stdout:Rotating logs: 2024-02-17T11:40:12.216 DEBUG:teuthology.orchestra.run.smithi189:> sudo service auditd rotate 2024-02-17T11:40:12.280 INFO:teuthology.orchestra.run.smithi189.stdout:Rotating logs: 2024-02-17T11:40:12.281 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-17T11:40:12.282 DEBUG:teuthology.orchestra.run.smithi094:> /usr/sbin/getenforce 2024-02-17T11:40:12.306 INFO:teuthology.orchestra.run.smithi094.stdout:Permissive 2024-02-17T11:40:12.306 DEBUG:teuthology.orchestra.run.smithi125:> /usr/sbin/getenforce 2024-02-17T11:40:12.329 INFO:teuthology.orchestra.run.smithi125.stdout:Permissive 2024-02-17T11:40:12.330 DEBUG:teuthology.orchestra.run.smithi189:> /usr/sbin/getenforce 2024-02-17T11:40:12.353 INFO:teuthology.orchestra.run.smithi189.stdout:Permissive 2024-02-17T11:40:12.354 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi094.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi125.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi189.front.sepia.ceph.com': 'permissive'} 2024-02-17T11:40:12.354 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-17T11:40:12.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T11:40:12.383 DEBUG:teuthology.orchestra.run.smithi125:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-17T11:40:12.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T11:40:12.413 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-17T11:40:12.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T11:40:12.442 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-17T11:40:12.442 DEBUG:teuthology.orchestra.run.smithi094:> sudo /usr/sbin/setenforce permissive 2024-02-17T11:40:12.474 DEBUG:teuthology.orchestra.run.smithi125:> sudo /usr/sbin/setenforce permissive 2024-02-17T11:40:12.506 DEBUG:teuthology.orchestra.run.smithi189:> sudo /usr/sbin/setenforce permissive 2024-02-17T11:40:12.538 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-17T11:40:12.548 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-17T11:40:12.553 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-17T11:40:20.420 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-17T11:40:20.436 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-17T11:40:20.437 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi094.front.sepia.ceph.com,smithi125.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-17T11:45:00.685 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi094.front.sepia.ceph.com'), Remote(name='ubuntu@smithi125.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-02-17T11:45:00.687 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-17T11:45:00.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T11:45:00.775 DEBUG:teuthology.orchestra.run.smithi094:> true 2024-02-17T11:45:00.863 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi094.front.sepia.ceph.com' 2024-02-17T11:45:00.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi125.front.sepia.ceph.com' 2024-02-17T11:45:00.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi125.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T11:45:00.951 DEBUG:teuthology.orchestra.run.smithi125:> true 2024-02-17T11:45:01.018 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi125.front.sepia.ceph.com' 2024-02-17T11:45:01.019 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-02-17T11:45:01.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-17T11:45:01.105 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-02-17T11:45:01.179 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-02-17T11:45:01.179 INFO:teuthology.run_tasks:Running task clock... 2024-02-17T11:45:01.193 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-17T11:45:01.193 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-17T11:45:01.194 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-17T11:45:01.196 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-17T11:45:01.196 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-17T11:45:01.199 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-17T11:45:01.199 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-17T11:45:01.227 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-17T11:45:01.228 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-17T11:45:01.244 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-17T11:45:01.245 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-17T11:45:01.251 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-17T11:45:01.268 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-17T11:45:01.273 INFO:teuthology.orchestra.run.smithi125.stderr:sudo: ntpd: command not found 2024-02-17T11:45:01.275 INFO:teuthology.orchestra.run.smithi094.stderr:sudo: ntpd: command not found 2024-02-17T11:45:01.287 INFO:teuthology.orchestra.run.smithi125.stdout:506 Cannot talk to daemon 2024-02-17T11:45:01.289 INFO:teuthology.orchestra.run.smithi094.stdout:506 Cannot talk to daemon 2024-02-17T11:45:01.297 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: ntpd: command not found 2024-02-17T11:45:01.303 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-17T11:45:01.305 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-17T11:45:01.311 INFO:teuthology.orchestra.run.smithi189.stdout:506 Cannot talk to daemon 2024-02-17T11:45:01.319 INFO:teuthology.orchestra.run.smithi125.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-17T11:45:01.322 INFO:teuthology.orchestra.run.smithi094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-17T11:45:01.328 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-17T11:45:01.344 INFO:teuthology.orchestra.run.smithi189.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-17T11:45:01.362 INFO:teuthology.orchestra.run.smithi125.stderr:bash: line 1: ntpq: command not found 2024-02-17T11:45:01.366 INFO:teuthology.orchestra.run.smithi125.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-17T11:45:01.366 INFO:teuthology.orchestra.run.smithi125.stdout:=============================================================================== 2024-02-17T11:45:01.366 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T11:45:01.366 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T11:45:01.366 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T11:45:01.366 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T11:45:01.383 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-02-17T11:45:01.387 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-17T11:45:01.387 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-02-17T11:45:01.387 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T11:45:01.387 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T11:45:01.387 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T11:45:01.387 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T11:45:01.388 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-02-17T11:45:01.391 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-17T11:45:01.391 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-02-17T11:45:01.392 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T11:45:01.392 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T11:45:01.392 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T11:45:01.392 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T11:45:01.393 INFO:teuthology.run_tasks:Running task install... 2024-02-17T11:45:01.404 DEBUG:teuthology.task.install:project ceph 2024-02-17T11:45:01.405 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d'}} 2024-02-17T11:45:01.405 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d'} 2024-02-17T11:45:01.405 INFO:teuthology.task.install:Using flavor: default 2024-02-17T11:45:01.413 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-17T11:45:01.413 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-02-17T11:45:01.413 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d', 'tag': None, 'wait_for_package': False} 2024-02-17T11:45:01.414 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T11:45:01.415 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d', 'tag': None, 'wait_for_package': False} 2024-02-17T11:45:01.416 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T11:45:01.417 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d', 'tag': None, 'wait_for_package': False} 2024-02-17T11:45:01.417 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T11:45:01.566 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/ 2024-02-17T11:45:01.566 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1515.g6822c7b5 2024-02-17T11:45:01.569 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/ 2024-02-17T11:45:01.569 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1515.g6822c7b5 2024-02-17T11:45:01.648 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/ 2024-02-17T11:45:01.648 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1515.g6822c7b5 2024-02-17T11:45:01.682 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-17T11:45:01.682 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-17T11:45:01.682 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-17T11:45:01.686 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-17T11:45:01.686 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-17T11:45:01.686 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-17T11:45:01.709 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-17T11:45:01.710 DEBUG:teuthology.orchestra.run.smithi125:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6822c7b54e949006be7a0802650d01723d834d1d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-17T11:45:01.714 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-17T11:45:01.714 DEBUG:teuthology.orchestra.run.smithi189:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6822c7b54e949006be7a0802650d01723d834d1d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-17T11:45:01.759 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-17T11:45:01.759 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:45:01.759 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-17T11:45:01.777 DEBUG:teuthology.orchestra.run.smithi125:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-17T11:45:01.782 DEBUG:teuthology.orchestra.run.smithi189:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-17T11:45:01.787 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-17T11:45:01.787 DEBUG:teuthology.orchestra.run.smithi094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6822c7b54e949006be7a0802650d01723d834d1d/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-17T11:45:01.856 DEBUG:teuthology.orchestra.run.smithi094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-17T11:45:01.857 DEBUG:teuthology.orchestra.run.smithi125:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-17T11:45:01.861 DEBUG:teuthology.orchestra.run.smithi189:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-17T11:45:01.923 INFO:teuthology.orchestra.run.smithi125.stdout:check_obsoletes = 1 2024-02-17T11:45:01.924 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum clean all 2024-02-17T11:45:01.927 INFO:teuthology.orchestra.run.smithi189.stdout:check_obsoletes = 1 2024-02-17T11:45:01.928 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum clean all 2024-02-17T11:45:01.935 DEBUG:teuthology.orchestra.run.smithi094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-17T11:45:02.001 INFO:teuthology.orchestra.run.smithi094.stdout:check_obsoletes = 1 2024-02-17T11:45:02.003 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum clean all 2024-02-17T11:45:02.156 INFO:teuthology.orchestra.run.smithi125.stdout:74 files removed 2024-02-17T11:45:02.167 INFO:teuthology.orchestra.run.smithi189.stdout:74 files removed 2024-02-17T11:45:02.183 DEBUG:teuthology.orchestra.run.smithi125:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-17T11:45:02.194 DEBUG:teuthology.orchestra.run.smithi189:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-17T11:45:02.253 INFO:teuthology.orchestra.run.smithi094.stdout:74 files removed 2024-02-17T11:45:02.283 DEBUG:teuthology.orchestra.run.smithi094:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-17T11:45:02.872 INFO:teuthology.orchestra.run.smithi125.stdout:ceph packages for x86_64 205 kB/s | 82 kB 00:00 2024-02-17T11:45:02.880 INFO:teuthology.orchestra.run.smithi189.stdout:ceph packages for x86_64 208 kB/s | 82 kB 00:00 2024-02-17T11:45:03.025 INFO:teuthology.orchestra.run.smithi094.stdout:ceph packages for x86_64 193 kB/s | 82 kB 00:00 2024-02-17T11:45:03.198 INFO:teuthology.orchestra.run.smithi189.stdout:ceph noarch packages 36 kB/s | 10 kB 00:00 2024-02-17T11:45:03.289 INFO:teuthology.orchestra.run.smithi125.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-02-17T11:45:03.384 INFO:teuthology.orchestra.run.smithi094.stdout:ceph noarch packages 32 kB/s | 10 kB 00:00 2024-02-17T11:45:03.632 INFO:teuthology.orchestra.run.smithi125.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-02-17T11:45:03.632 INFO:teuthology.orchestra.run.smithi189.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-02-17T11:45:03.726 INFO:teuthology.orchestra.run.smithi094.stdout:ceph source packages 6.3 kB/s | 1.9 kB 00:00 2024-02-17T11:45:04.100 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-02-17T11:45:04.159 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-02-17T11:45:04.270 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-02-17T11:45:05.626 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.0 MB 00:00 2024-02-17T11:45:05.693 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2024-02-17T11:45:06.038 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2024-02-17T11:45:07.369 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.0 MB 00:01 2024-02-17T11:45:07.424 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.0 MB 00:01 2024-02-17T11:45:09.260 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-02-17T11:45:09.384 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-02-17T11:45:09.935 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.0 MB 00:03 2024-02-17T11:45:13.152 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - CRB 18 MB/s | 6.1 MB 00:00 2024-02-17T11:45:13.343 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-02-17T11:45:13.537 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 19 MB 00:02 2024-02-17T11:45:14.761 INFO:teuthology.orchestra.run.smithi189.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2024-02-17T11:45:14.961 INFO:teuthology.orchestra.run.smithi125.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2024-02-17T11:45:16.754 INFO:teuthology.orchestra.run.smithi125.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2024-02-17T11:45:17.564 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.1 MB 00:00 2024-02-17T11:45:19.097 INFO:teuthology.orchestra.run.smithi094.stdout:CentOS Stream 9 - Extras packages 88 kB/s | 15 kB 00:00 2024-02-17T11:45:20.225 INFO:teuthology.orchestra.run.smithi094.stdout:Extra Packages for Enterprise Linux 20 MB/s | 20 MB 00:00 2024-02-17T11:45:21.837 INFO:teuthology.orchestra.run.smithi125.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-02-17T11:45:23.090 INFO:teuthology.orchestra.run.smithi189.stdout:Extra Packages for Enterprise Linux 2.5 MB/s | 20 MB 00:08 2024-02-17T11:45:24.356 INFO:teuthology.orchestra.run.smithi125.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-17T11:45:24.356 INFO:teuthology.orchestra.run.smithi125.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-17T11:45:24.371 INFO:teuthology.orchestra.run.smithi125.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-17T11:45:24.372 INFO:teuthology.orchestra.run.smithi125.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-17T11:45:24.373 INFO:teuthology.orchestra.run.smithi125.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-17T11:45:24.374 INFO:teuthology.orchestra.run.smithi125.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-17T11:45:24.376 INFO:teuthology.orchestra.run.smithi125.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-17T11:45:24.379 INFO:teuthology.orchestra.run.smithi125.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-17T11:45:24.380 INFO:teuthology.orchestra.run.smithi125.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-17T11:45:24.461 INFO:teuthology.orchestra.run.smithi125.stdout:Dependencies resolved. 2024-02-17T11:45:24.473 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================ 2024-02-17T11:45:24.474 INFO:teuthology.orchestra.run.smithi125.stdout: Package Arch Version Repository Size 2024-02-17T11:45:24.474 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================ 2024-02-17T11:45:24.474 INFO:teuthology.orchestra.run.smithi125.stdout:Installing: 2024-02-17T11:45:24.474 INFO:teuthology.orchestra.run.smithi125.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-17T11:45:24.474 INFO:teuthology.orchestra.run.smithi125.stdout: ceph x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 6.6 k 2024-02-17T11:45:24.474 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-base x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 5.3 M 2024-02-17T11:45:24.474 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-fuse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 885 k 2024-02-17T11:45:24.474 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 146 k 2024-02-17T11:45:24.474 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 1.5 M 2024-02-17T11:45:24.474 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 138 k 2024-02-17T11:45:24.474 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 1.8 M 2024-02-17T11:45:24.474 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 7.4 M 2024-02-17T11:45:24.474 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-rook noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 52 k 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-radosgw x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 9.8 M 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-test x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 47 M 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-volume noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 264 k 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: cephadm noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 748 k 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-mirror x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 217 k 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-shell x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 27 k 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-top noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 18 k 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs-devel x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 31 k 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 743 k 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: librados-devel x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 127 k 2024-02-17T11:45:24.475 INFO:teuthology.orchestra.run.smithi125.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-17T11:45:24.476 INFO:teuthology.orchestra.run.smithi125.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-17T11:45:24.476 INFO:teuthology.orchestra.run.smithi125.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-17T11:45:24.476 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-17T11:45:24.476 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-17T11:45:24.476 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-17T11:45:24.476 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cephfs x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 162 k 2024-02-17T11:45:24.476 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rados x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 321 k 2024-02-17T11:45:24.476 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rbd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 298 k 2024-02-17T11:45:24.476 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rgw x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 99 k 2024-02-17T11:45:24.476 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-fuse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 85 k 2024-02-17T11:45:24.476 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-mirror x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.1 M 2024-02-17T11:45:24.476 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-nbd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 171 k 2024-02-17T11:45:24.477 INFO:teuthology.orchestra.run.smithi125.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-17T11:45:24.477 INFO:teuthology.orchestra.run.smithi125.stdout:Upgrading: 2024-02-17T11:45:24.477 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-17T11:45:24.477 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-17T11:45:24.477 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-17T11:45:24.477 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-17T11:45:24.477 INFO:teuthology.orchestra.run.smithi125.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-17T11:45:24.477 INFO:teuthology.orchestra.run.smithi125.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-17T11:45:24.477 INFO:teuthology.orchestra.run.smithi125.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-17T11:45:24.477 INFO:teuthology.orchestra.run.smithi125.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-17T11:45:24.477 INFO:teuthology.orchestra.run.smithi125.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-17T11:45:24.477 INFO:teuthology.orchestra.run.smithi125.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-17T11:45:24.477 INFO:teuthology.orchestra.run.smithi125.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-17T11:45:24.478 INFO:teuthology.orchestra.run.smithi125.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-17T11:45:24.478 INFO:teuthology.orchestra.run.smithi125.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-17T11:45:24.478 INFO:teuthology.orchestra.run.smithi125.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-17T11:45:24.478 INFO:teuthology.orchestra.run.smithi125.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-17T11:45:24.478 INFO:teuthology.orchestra.run.smithi125.stdout: librados2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.4 M 2024-02-17T11:45:24.478 INFO:teuthology.orchestra.run.smithi125.stdout: librbd1 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.1 M 2024-02-17T11:45:24.478 INFO:teuthology.orchestra.run.smithi125.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-17T11:45:24.478 INFO:teuthology.orchestra.run.smithi125.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-17T11:45:24.478 INFO:teuthology.orchestra.run.smithi125.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-17T11:45:24.478 INFO:teuthology.orchestra.run.smithi125.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-17T11:45:24.478 INFO:teuthology.orchestra.run.smithi125.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-17T11:45:24.478 INFO:teuthology.orchestra.run.smithi125.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-17T11:45:24.478 INFO:teuthology.orchestra.run.smithi125.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout:Installing dependencies: 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-common x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 20 M 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 31 k 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mds x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 2.1 M 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 247 k 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mon x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 4.7 M 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-osd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 17 M 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 15 k 2024-02-17T11:45:24.479 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-selinux x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 25 k 2024-02-17T11:45:24.480 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-17T11:45:24.480 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-17T11:45:24.480 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-17T11:45:24.480 INFO:teuthology.orchestra.run.smithi125.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-17T11:45:24.480 INFO:teuthology.orchestra.run.smithi125.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-17T11:45:24.480 INFO:teuthology.orchestra.run.smithi125.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-17T11:45:24.480 INFO:teuthology.orchestra.run.smithi125.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-17T11:45:24.480 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-17T11:45:24.480 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-17T11:45:24.480 INFO:teuthology.orchestra.run.smithi125.stdout: libcephsqlite x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 163 k 2024-02-17T11:45:24.480 INFO:teuthology.orchestra.run.smithi125.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: libradosstriper1 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 499 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: librgw2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 5.1 M 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-17T11:45:24.481 INFO:teuthology.orchestra.run.smithi125.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-argparse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 45 k 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-common x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 131 k 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-17T11:45:24.482 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-17T11:45:24.483 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-17T11:45:24.484 INFO:teuthology.orchestra.run.smithi125.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-17T11:45:24.484 INFO:teuthology.orchestra.run.smithi125.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-17T11:45:24.484 INFO:teuthology.orchestra.run.smithi125.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-17T11:45:24.484 INFO:teuthology.orchestra.run.smithi125.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-17T11:45:24.484 INFO:teuthology.orchestra.run.smithi125.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-17T11:45:24.484 INFO:teuthology.orchestra.run.smithi125.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-17T11:45:24.484 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-17T11:45:24.484 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-17T11:45:24.484 INFO:teuthology.orchestra.run.smithi125.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-17T11:45:24.484 INFO:teuthology.orchestra.run.smithi125.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-17T11:45:24.484 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-17T11:45:24.484 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-17T11:45:24.484 INFO:teuthology.orchestra.run.smithi125.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-17T11:45:24.485 INFO:teuthology.orchestra.run.smithi125.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-17T11:45:24.485 INFO:teuthology.orchestra.run.smithi125.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-17T11:45:24.485 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-17T11:45:24.485 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-17T11:45:24.485 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-17T11:45:24.485 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-17T11:45:24.485 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-17T11:45:24.485 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-17T11:45:24.485 INFO:teuthology.orchestra.run.smithi125.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-17T11:45:24.485 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-17T11:45:24.485 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-17T11:45:24.485 INFO:teuthology.orchestra.run.smithi125.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-17T11:45:24.485 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-17T11:45:24.486 INFO:teuthology.orchestra.run.smithi125.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-17T11:45:24.486 INFO:teuthology.orchestra.run.smithi125.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-17T11:45:24.486 INFO:teuthology.orchestra.run.smithi125.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-17T11:45:24.486 INFO:teuthology.orchestra.run.smithi125.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-17T11:45:24.486 INFO:teuthology.orchestra.run.smithi125.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-17T11:45:24.486 INFO:teuthology.orchestra.run.smithi125.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-17T11:45:24.486 INFO:teuthology.orchestra.run.smithi125.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-17T11:45:24.486 INFO:teuthology.orchestra.run.smithi125.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-17T11:45:24.486 INFO:teuthology.orchestra.run.smithi125.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-17T11:45:24.486 INFO:teuthology.orchestra.run.smithi125.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-17T11:45:24.486 INFO:teuthology.orchestra.run.smithi125.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-17T11:45:24.486 INFO:teuthology.orchestra.run.smithi125.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-17T11:45:24.486 INFO:teuthology.orchestra.run.smithi125.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-17T11:45:24.487 INFO:teuthology.orchestra.run.smithi125.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-17T11:45:24.487 INFO:teuthology.orchestra.run.smithi125.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-17T11:45:24.487 INFO:teuthology.orchestra.run.smithi125.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-17T11:45:24.487 INFO:teuthology.orchestra.run.smithi125.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-17T11:45:24.487 INFO:teuthology.orchestra.run.smithi125.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-17T11:45:24.487 INFO:teuthology.orchestra.run.smithi125.stdout:Installing weak dependencies: 2024-02-17T11:45:24.487 INFO:teuthology.orchestra.run.smithi125.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-17T11:45:24.487 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:45:24.487 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction Summary 2024-02-17T11:45:24.487 INFO:teuthology.orchestra.run.smithi125.stdout:============================================================================================ 2024-02-17T11:45:24.487 INFO:teuthology.orchestra.run.smithi125.stdout:Install 150 Packages 2024-02-17T11:45:24.487 INFO:teuthology.orchestra.run.smithi125.stdout:Upgrade 26 Packages 2024-02-17T11:45:24.487 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:45:24.488 INFO:teuthology.orchestra.run.smithi125.stdout:Total download size: 223 M 2024-02-17T11:45:24.488 INFO:teuthology.orchestra.run.smithi125.stdout:Downloading Packages: 2024-02-17T11:45:25.082 INFO:teuthology.orchestra.run.smithi094.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-02-17T11:45:25.196 INFO:teuthology.orchestra.run.smithi125.stdout:(1/176): ceph-19.0.0-1515.g6822c7b5.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-02-17T11:45:25.571 INFO:teuthology.orchestra.run.smithi125.stdout:(2/176): ceph-base-19.0.0-1515.g6822c7b5.el9.x8 8.3 MB/s | 5.3 MB 00:00 2024-02-17T11:45:25.638 INFO:teuthology.orchestra.run.smithi125.stdout:(3/176): ceph-immutable-object-cache-19.0.0-151 2.1 MB/s | 146 kB 00:00 2024-02-17T11:45:25.922 INFO:teuthology.orchestra.run.smithi125.stdout:(4/176): ceph-mds-19.0.0-1515.g6822c7b5.el9.x86 7.5 MB/s | 2.1 MB 00:00 2024-02-17T11:45:26.114 INFO:teuthology.orchestra.run.smithi125.stdout:(5/176): ceph-fuse-19.0.0-1515.g6822c7b5.el9.x8 965 kB/s | 885 kB 00:00 2024-02-17T11:45:26.147 INFO:teuthology.orchestra.run.smithi125.stdout:(6/176): ceph-mgr-19.0.0-1515.g6822c7b5.el9.x86 6.6 MB/s | 1.5 MB 00:00 2024-02-17T11:45:26.406 INFO:teuthology.orchestra.run.smithi125.stdout:(7/176): ceph-common-19.0.0-1515.g6822c7b5.el9. 13 MB/s | 20 MB 00:01 2024-02-17T11:45:27.123 INFO:teuthology.orchestra.run.smithi125.stdout:(8/176): ceph-radosgw-19.0.0-1515.g6822c7b5.el9 14 MB/s | 9.8 MB 00:00 2024-02-17T11:45:27.164 INFO:teuthology.orchestra.run.smithi125.stdout:(9/176): ceph-selinux-19.0.0-1515.g6822c7b5.el9 607 kB/s | 25 kB 00:00 2024-02-17T11:45:27.672 INFO:teuthology.orchestra.run.smithi094.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-17T11:45:27.673 INFO:teuthology.orchestra.run.smithi094.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-17T11:45:27.688 INFO:teuthology.orchestra.run.smithi094.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-17T11:45:27.689 INFO:teuthology.orchestra.run.smithi094.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-17T11:45:27.690 INFO:teuthology.orchestra.run.smithi094.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-17T11:45:27.691 INFO:teuthology.orchestra.run.smithi094.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-17T11:45:27.693 INFO:teuthology.orchestra.run.smithi094.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-17T11:45:27.696 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-17T11:45:27.697 INFO:teuthology.orchestra.run.smithi094.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-17T11:45:27.777 INFO:teuthology.orchestra.run.smithi094.stdout:Dependencies resolved. 2024-02-17T11:45:27.796 INFO:teuthology.orchestra.run.smithi094.stdout:============================================================================================ 2024-02-17T11:45:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: Package Arch Version Repository Size 2024-02-17T11:45:27.796 INFO:teuthology.orchestra.run.smithi094.stdout:============================================================================================ 2024-02-17T11:45:27.796 INFO:teuthology.orchestra.run.smithi094.stdout:Installing: 2024-02-17T11:45:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-17T11:45:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: ceph x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 6.6 k 2024-02-17T11:45:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 5.3 M 2024-02-17T11:45:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 885 k 2024-02-17T11:45:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 146 k 2024-02-17T11:45:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 1.5 M 2024-02-17T11:45:27.796 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 138 k 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 1.8 M 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 7.4 M 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 52 k 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 9.8 M 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 47 M 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 264 k 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 748 k 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 217 k 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 27 k 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 18 k 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-17T11:45:27.797 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 31 k 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 743 k 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 127 k 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 162 k 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 321 k 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 298 k 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 99 k 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 85 k 2024-02-17T11:45:27.798 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.1 M 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 171 k 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrading: 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-17T11:45:27.799 INFO:teuthology.orchestra.run.smithi094.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-17T11:45:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-17T11:45:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-17T11:45:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-17T11:45:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-17T11:45:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: librados2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.4 M 2024-02-17T11:45:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.1 M 2024-02-17T11:45:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-17T11:45:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-17T11:45:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-17T11:45:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-17T11:45:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-17T11:45:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-17T11:45:27.800 INFO:teuthology.orchestra.run.smithi094.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-17T11:45:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-17T11:45:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-17T11:45:27.801 INFO:teuthology.orchestra.run.smithi094.stdout:Installing dependencies: 2024-02-17T11:45:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-17T11:45:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-17T11:45:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-17T11:45:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 20 M 2024-02-17T11:45:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 31 k 2024-02-17T11:45:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 2.1 M 2024-02-17T11:45:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 247 k 2024-02-17T11:45:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 4.7 M 2024-02-17T11:45:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 17 M 2024-02-17T11:45:27.801 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 15 k 2024-02-17T11:45:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 25 k 2024-02-17T11:45:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-17T11:45:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-17T11:45:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-17T11:45:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-17T11:45:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-17T11:45:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-17T11:45:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-17T11:45:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-17T11:45:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-17T11:45:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 163 k 2024-02-17T11:45:27.802 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 499 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 5.1 M 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-17T11:45:27.803 INFO:teuthology.orchestra.run.smithi094.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 45 k 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 131 k 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-17T11:45:27.804 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-17T11:45:27.805 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-17T11:45:27.806 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-17T11:45:27.807 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-17T11:45:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-17T11:45:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-17T11:45:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-17T11:45:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-17T11:45:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-17T11:45:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-17T11:45:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-17T11:45:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-17T11:45:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-17T11:45:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-17T11:45:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-17T11:45:27.808 INFO:teuthology.orchestra.run.smithi094.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-17T11:45:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-17T11:45:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-17T11:45:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-17T11:45:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-17T11:45:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-17T11:45:27.809 INFO:teuthology.orchestra.run.smithi094.stdout:Installing weak dependencies: 2024-02-17T11:45:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-17T11:45:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:45:27.809 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction Summary 2024-02-17T11:45:27.809 INFO:teuthology.orchestra.run.smithi094.stdout:============================================================================================ 2024-02-17T11:45:27.809 INFO:teuthology.orchestra.run.smithi094.stdout:Install 150 Packages 2024-02-17T11:45:27.809 INFO:teuthology.orchestra.run.smithi094.stdout:Upgrade 26 Packages 2024-02-17T11:45:27.809 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:45:27.810 INFO:teuthology.orchestra.run.smithi094.stdout:Total download size: 223 M 2024-02-17T11:45:27.810 INFO:teuthology.orchestra.run.smithi094.stdout:Downloading Packages: 2024-02-17T11:45:27.890 INFO:teuthology.orchestra.run.smithi125.stdout:(10/176): ceph-osd-19.0.0-1515.g6822c7b5.el9.x8 9.6 MB/s | 17 MB 00:01 2024-02-17T11:45:27.957 INFO:teuthology.orchestra.run.smithi125.stdout:(11/176): ceph-mon-19.0.0-1515.g6822c7b5.el9.x8 2.5 MB/s | 4.7 MB 00:01 2024-02-17T11:45:27.982 INFO:teuthology.orchestra.run.smithi125.stdout:(12/176): cephfs-mirror-19.0.0-1515.g6822c7b5.e 2.3 MB/s | 217 kB 00:00 2024-02-17T11:45:28.007 INFO:teuthology.orchestra.run.smithi125.stdout:(13/176): cephfs-shell-19.0.0-1515.g6822c7b5.el 539 kB/s | 27 kB 00:00 2024-02-17T11:45:28.020 INFO:teuthology.orchestra.run.smithi189.stdout:lab-extras 10 kB/s | 1.7 kB 00:00 2024-02-17T11:45:28.024 INFO:teuthology.orchestra.run.smithi125.stdout:(14/176): libcephfs-devel-19.0.0-1515.g6822c7b5 743 kB/s | 31 kB 00:00 2024-02-17T11:45:28.108 INFO:teuthology.orchestra.run.smithi125.stdout:(15/176): libcephsqlite-19.0.0-1515.g6822c7b5.e 1.9 MB/s | 163 kB 00:00 2024-02-17T11:45:28.183 INFO:teuthology.orchestra.run.smithi125.stdout:(16/176): librados-devel-19.0.0-1515.g6822c7b5. 1.7 MB/s | 127 kB 00:00 2024-02-17T11:45:28.217 INFO:teuthology.orchestra.run.smithi125.stdout:(17/176): libcephfs2-19.0.0-1515.g6822c7b5.el9. 3.5 MB/s | 743 kB 00:00 2024-02-17T11:45:28.266 INFO:teuthology.orchestra.run.smithi094.stdout:(1/176): ceph-19.0.0-1515.g6822c7b5.el9.x86_64. 30 kB/s | 6.6 kB 00:00 2024-02-17T11:45:28.267 INFO:teuthology.orchestra.run.smithi125.stdout:(18/176): libradosstriper1-19.0.0-1515.g6822c7b 5.8 MB/s | 499 kB 00:00 2024-02-17T11:45:28.342 INFO:teuthology.orchestra.run.smithi125.stdout:(19/176): python3-ceph-argparse-19.0.0-1515.g68 600 kB/s | 45 kB 00:00 2024-02-17T11:45:28.426 INFO:teuthology.orchestra.run.smithi125.stdout:(20/176): python3-ceph-common-19.0.0-1515.g6822 1.5 MB/s | 131 kB 00:00 2024-02-17T11:45:28.518 INFO:teuthology.orchestra.run.smithi125.stdout:(21/176): python3-cephfs-19.0.0-1515.g6822c7b5. 1.7 MB/s | 162 kB 00:00 2024-02-17T11:45:28.618 INFO:teuthology.orchestra.run.smithi125.stdout:(22/176): python3-rados-19.0.0-1515.g6822c7b5.e 3.1 MB/s | 321 kB 00:00 2024-02-17T11:45:28.718 INFO:teuthology.orchestra.run.smithi125.stdout:(23/176): python3-rbd-19.0.0-1515.g6822c7b5.el9 2.9 MB/s | 298 kB 00:00 2024-02-17T11:45:28.819 INFO:teuthology.orchestra.run.smithi125.stdout:(24/176): python3-rgw-19.0.0-1515.g6822c7b5.el9 994 kB/s | 99 kB 00:00 2024-02-17T11:45:28.919 INFO:teuthology.orchestra.run.smithi125.stdout:(25/176): rbd-fuse-19.0.0-1515.g6822c7b5.el9.x8 854 kB/s | 85 kB 00:00 2024-02-17T11:45:29.150 INFO:teuthology.orchestra.run.smithi094.stdout:(2/176): ceph-fuse-19.0.0-1515.g6822c7b5.el9.x8 1.0 MB/s | 885 kB 00:00 2024-02-17T11:45:29.278 INFO:teuthology.orchestra.run.smithi125.stdout:(26/176): librgw2-19.0.0-1515.g6822c7b5.el9.x86 4.8 MB/s | 5.1 MB 00:01 2024-02-17T11:45:29.300 INFO:teuthology.orchestra.run.smithi094.stdout:(3/176): ceph-immutable-object-cache-19.0.0-151 973 kB/s | 146 kB 00:00 2024-02-17T11:45:29.378 INFO:teuthology.orchestra.run.smithi125.stdout:(27/176): rbd-nbd-19.0.0-1515.g6822c7b5.el9.x86 1.7 MB/s | 171 kB 00:00 2024-02-17T11:45:29.436 INFO:teuthology.orchestra.run.smithi125.stdout:(28/176): rbd-mirror-19.0.0-1515.g6822c7b5.el9. 6.0 MB/s | 3.1 MB 00:00 2024-02-17T11:45:29.462 INFO:teuthology.orchestra.run.smithi125.stdout:(29/176): ceph-grafana-dashboards-19.0.0-1515.g 376 kB/s | 31 kB 00:00 2024-02-17T11:45:29.512 INFO:teuthology.orchestra.run.smithi125.stdout:(30/176): ceph-mgr-cephadm-19.0.0-1515.g6822c7b 1.8 MB/s | 138 kB 00:00 2024-02-17T11:45:29.904 INFO:teuthology.orchestra.run.smithi125.stdout:(31/176): ceph-mgr-dashboard-19.0.0-1515.g6822c 4.0 MB/s | 1.8 MB 00:00 2024-02-17T11:45:30.117 INFO:teuthology.orchestra.run.smithi094.stdout:(4/176): ceph-mds-19.0.0-1515.g6822c7b5.el9.x86 2.6 MB/s | 2.1 MB 00:00 2024-02-17T11:45:30.284 INFO:teuthology.orchestra.run.smithi094.stdout:(5/176): ceph-base-19.0.0-1515.g6822c7b5.el9.x8 2.4 MB/s | 5.3 MB 00:02 2024-02-17T11:45:30.359 INFO:teuthology.orchestra.run.smithi094.stdout:(6/176): ceph-mgr-19.0.0-1515.g6822c7b5.el9.x86 6.2 MB/s | 1.5 MB 00:00 2024-02-17T11:45:30.379 INFO:teuthology.orchestra.run.smithi125.stdout:(32/176): ceph-test-19.0.0-1515.g6822c7b5.el9.x 15 MB/s | 47 MB 00:03 2024-02-17T11:45:30.404 INFO:teuthology.orchestra.run.smithi125.stdout:(33/176): ceph-mgr-modules-core-19.0.0-1515.g68 495 kB/s | 247 kB 00:00 2024-02-17T11:45:30.438 INFO:teuthology.orchestra.run.smithi125.stdout:(34/176): ceph-mgr-rook-19.0.0-1515.g6822c7b5.e 898 kB/s | 52 kB 00:00 2024-02-17T11:45:30.496 INFO:teuthology.orchestra.run.smithi125.stdout:(35/176): ceph-prometheus-alerts-19.0.0-1515.g6 167 kB/s | 15 kB 00:00 2024-02-17T11:45:30.522 INFO:teuthology.orchestra.run.smithi125.stdout:(36/176): ceph-volume-19.0.0-1515.g6822c7b5.el9 3.1 MB/s | 264 kB 00:00 2024-02-17T11:45:30.582 INFO:teuthology.orchestra.run.smithi189.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-17T11:45:30.583 INFO:teuthology.orchestra.run.smithi189.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-02-17T11:45:30.597 INFO:teuthology.orchestra.run.smithi189.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-02-17T11:45:30.598 INFO:teuthology.orchestra.run.smithi189.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-02-17T11:45:30.599 INFO:teuthology.orchestra.run.smithi189.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-02-17T11:45:30.600 INFO:teuthology.orchestra.run.smithi189.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-02-17T11:45:30.602 INFO:teuthology.orchestra.run.smithi189.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-02-17T11:45:30.605 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-02-17T11:45:30.606 INFO:teuthology.orchestra.run.smithi189.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-02-17T11:45:30.655 INFO:teuthology.orchestra.run.smithi125.stdout:(37/176): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-02-17T11:45:30.680 INFO:teuthology.orchestra.run.smithi125.stdout:(38/176): cephfs-top-19.0.0-1515.g6822c7b5.el9. 115 kB/s | 18 kB 00:00 2024-02-17T11:45:30.686 INFO:teuthology.orchestra.run.smithi189.stdout:Dependencies resolved. 2024-02-17T11:45:30.706 INFO:teuthology.orchestra.run.smithi189.stdout:============================================================================================ 2024-02-17T11:45:30.706 INFO:teuthology.orchestra.run.smithi189.stdout: Package Arch Version Repository Size 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout:============================================================================================ 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout:Installing: 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: ceph x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 6.6 k 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 5.3 M 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 885 k 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 146 k 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 1.5 M 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 138 k 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 1.8 M 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 7.4 M 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 52 k 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 9.8 M 2024-02-17T11:45:30.707 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 47 M 2024-02-17T11:45:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 264 k 2024-02-17T11:45:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 748 k 2024-02-17T11:45:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 217 k 2024-02-17T11:45:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 27 k 2024-02-17T11:45:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 18 k 2024-02-17T11:45:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-02-17T11:45:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-02-17T11:45:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-02-17T11:45:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 31 k 2024-02-17T11:45:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 743 k 2024-02-17T11:45:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 127 k 2024-02-17T11:45:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-02-17T11:45:30.708 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel x86_64 2.37.4-18.el9 CentOS-AppStream 24 k 2024-02-17T11:45:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-02-17T11:45:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-02-17T11:45:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-02-17T11:45:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-02-17T11:45:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 162 k 2024-02-17T11:45:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 321 k 2024-02-17T11:45:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 298 k 2024-02-17T11:45:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 99 k 2024-02-17T11:45:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 85 k 2024-02-17T11:45:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.1 M 2024-02-17T11:45:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 171 k 2024-02-17T11:45:30.709 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-02-17T11:45:30.709 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrading: 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager x86_64 1:1.45.91-1.el9 CentOS-BaseOS 2.3 M 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm x86_64 1:1.45.91-1.el9 CentOS-BaseOS 1.8 M 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team x86_64 1:1.45.91-1.el9 CentOS-BaseOS 41 k 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui x86_64 1:1.45.91-1.el9 CentOS-BaseOS 246 k 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: iproute x86_64 6.7.0-2.el9 CentOS-BaseOS 802 k 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: iproute-tc x86_64 6.7.0-2.el9 CentOS-BaseOS 433 k 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: libblkid x86_64 2.37.4-18.el9 CentOS-BaseOS 107 k 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: libfdisk x86_64 2.37.4-18.el9 CentOS-BaseOS 154 k 2024-02-17T11:45:30.710 INFO:teuthology.orchestra.run.smithi189.stdout: libmount x86_64 2.37.4-18.el9 CentOS-BaseOS 135 k 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: librados2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.4 M 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 3.1 M 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: libsmartcols x86_64 2.37.4-18.el9 CentOS-BaseOS 63 k 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid x86_64 2.37.4-18.el9 CentOS-BaseOS 28 k 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux x86_64 2.37.4-18.el9 CentOS-BaseOS 2.2 M 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-core x86_64 2.37.4-18.el9 CentOS-BaseOS 438 k 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-user x86_64 2.37.4-18.el9 CentOS-BaseOS 30 k 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout:Installing dependencies: 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-17T11:45:30.711 INFO:teuthology.orchestra.run.smithi189.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 20 M 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 31 k 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 2.1 M 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 247 k 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 4.7 M 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 17 M 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1515.g6822c7b5.el9 ceph-noarch 15 k 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 25 k 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-17T11:45:30.712 INFO:teuthology.orchestra.run.smithi189.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 163 k 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 499 k 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2 x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 5.1 M 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-02-17T11:45:30.713 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-17T11:45:30.714 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 45 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common x86_64 2:19.0.0-1515.g6822c7b5.el9 ceph 131 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-17T11:45:30.715 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-17T11:45:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-17T11:45:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-17T11:45:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-17T11:45:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-17T11:45:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-17T11:45:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-17T11:45:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-17T11:45:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-17T11:45:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-17T11:45:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-17T11:45:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-17T11:45:30.716 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-17T11:45:30.717 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib x86_64 23.11.0-1.el9 CentOS-AppStream 215 k 2024-02-17T11:45:30.718 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs x86_64 1.1.2-2.el9 CentOS-AppStream 43 k 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout:Installing weak dependencies: 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service x86_64 23.11.0-1.el9 CentOS-AppStream 34 k 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction Summary 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout:============================================================================================ 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout:Install 150 Packages 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout:Upgrade 26 Packages 2024-02-17T11:45:30.719 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:45:30.720 INFO:teuthology.orchestra.run.smithi189.stdout:Total download size: 223 M 2024-02-17T11:45:30.720 INFO:teuthology.orchestra.run.smithi189.stdout:Downloading Packages: 2024-02-17T11:45:30.772 INFO:teuthology.orchestra.run.smithi125.stdout:(39/176): cephadm-19.0.0-1515.g6822c7b5.el9.noa 2.7 MB/s | 748 kB 00:00 2024-02-17T11:45:30.851 INFO:teuthology.orchestra.run.smithi094.stdout:(7/176): ceph-mon-19.0.0-1515.g6822c7b5.el9.x86 8.3 MB/s | 4.7 MB 00:00 2024-02-17T11:45:30.922 INFO:teuthology.orchestra.run.smithi125.stdout:(40/176): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2024-02-17T11:45:30.948 INFO:teuthology.orchestra.run.smithi125.stdout:(41/176): boost-program-options-1.75.0-8.el9.x8 608 kB/s | 106 kB 00:00 2024-02-17T11:45:30.973 INFO:teuthology.orchestra.run.smithi125.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 410 kB/s | 20 kB 00:00 2024-02-17T11:45:30.998 INFO:teuthology.orchestra.run.smithi125.stdout:(43/176): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 920 kB 00:00 2024-02-17T11:45:31.023 INFO:teuthology.orchestra.run.smithi125.stdout:(44/176): flex-2.6.4-9.el9.x86_64.rpm 4.0 MB/s | 307 kB 00:00 2024-02-17T11:45:31.049 INFO:teuthology.orchestra.run.smithi125.stdout:(45/176): flexiblas-3.0.4-8.el9.x86_64.rpm 421 kB/s | 32 kB 00:00 2024-02-17T11:45:31.266 INFO:teuthology.orchestra.run.smithi125.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 71 kB/s | 17 kB 00:00 2024-02-17T11:45:31.291 INFO:teuthology.orchestra.run.smithi125.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 42 kB/s | 10 kB 00:00 2024-02-17T11:45:31.408 INFO:teuthology.orchestra.run.smithi125.stdout:(48/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.3 MB/s | 3.0 MB 00:00 2024-02-17T11:45:31.433 INFO:teuthology.orchestra.run.smithi125.stdout:(49/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 159 kB 00:00 2024-02-17T11:45:31.459 INFO:teuthology.orchestra.run.smithi125.stdout:(50/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 266 kB/s | 44 kB 00:00 2024-02-17T11:45:31.567 INFO:teuthology.orchestra.run.smithi125.stdout:(51/176): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 661 kB 00:00 2024-02-17T11:45:31.592 INFO:teuthology.orchestra.run.smithi125.stdout:(52/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 182 kB/s | 24 kB 00:00 2024-02-17T11:45:31.626 INFO:teuthology.orchestra.run.smithi125.stdout:(53/176): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 578 kB 00:00 2024-02-17T11:45:31.651 INFO:teuthology.orchestra.run.smithi125.stdout:(54/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 554 kB/s | 46 kB 00:00 2024-02-17T11:45:31.693 INFO:teuthology.orchestra.run.smithi125.stdout:(55/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 282 kB 00:00 2024-02-17T11:45:31.726 INFO:teuthology.orchestra.run.smithi094.stdout:(8/176): ceph-radosgw-19.0.0-1515.g6822c7b5.el9 11 MB/s | 9.8 MB 00:00 2024-02-17T11:45:31.777 INFO:teuthology.orchestra.run.smithi125.stdout:(56/176): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-02-17T11:45:31.793 INFO:teuthology.orchestra.run.smithi094.stdout:(9/176): ceph-selinux-19.0.0-1515.g6822c7b5.el9 377 kB/s | 25 kB 00:00 2024-02-17T11:45:31.802 INFO:teuthology.orchestra.run.smithi125.stdout:(57/176): openblas-0.3.21-2.el9.x86_64.rpm 228 kB/s | 34 kB 00:00 2024-02-17T11:45:31.951 INFO:teuthology.orchestra.run.smithi094.stdout:(10/176): ceph-osd-19.0.0-1515.g6822c7b5.el9.x8 11 MB/s | 17 MB 00:01 2024-02-17T11:45:32.002 INFO:teuthology.orchestra.run.smithi094.stdout:(11/176): cephfs-mirror-19.0.0-1515.g6822c7b5.e 4.3 MB/s | 217 kB 00:00 2024-02-17T11:45:32.077 INFO:teuthology.orchestra.run.smithi094.stdout:(12/176): cephfs-shell-19.0.0-1515.g6822c7b5.el 360 kB/s | 27 kB 00:00 2024-02-17T11:45:32.079 INFO:teuthology.orchestra.run.smithi189.stdout:(1/176): ceph-19.0.0-1515.g6822c7b5.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-02-17T11:45:32.161 INFO:teuthology.orchestra.run.smithi094.stdout:(13/176): libcephfs-devel-19.0.0-1515.g6822c7b5 374 kB/s | 31 kB 00:00 2024-02-17T11:45:32.319 INFO:teuthology.orchestra.run.smithi125.stdout:(58/176): openblas-openmp-0.3.21-2.el9.x86_64.r 7.6 MB/s | 4.7 MB 00:00 2024-02-17T11:45:32.394 INFO:teuthology.orchestra.run.smithi125.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 2.6 MB/s | 1.5 MB 00:00 2024-02-17T11:45:32.502 INFO:teuthology.orchestra.run.smithi125.stdout:(60/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 4.3 MB/s | 3.0 MB 00:00 2024-02-17T11:45:32.603 INFO:teuthology.orchestra.run.smithi125.stdout:(61/176): postgresql-contrib-13.11-1.el9.x86_64 2.8 MB/s | 811 kB 00:00 2024-02-17T11:45:32.628 INFO:teuthology.orchestra.run.smithi125.stdout:(62/176): postgresql-private-libs-13.11-1.el9.x 575 kB/s | 134 kB 00:00 2024-02-17T11:45:32.711 INFO:teuthology.orchestra.run.smithi094.stdout:(14/176): libcephfs2-19.0.0-1515.g6822c7b5.el9. 1.3 MB/s | 743 kB 00:00 2024-02-17T11:45:32.786 INFO:teuthology.orchestra.run.smithi094.stdout:(15/176): libcephsqlite-19.0.0-1515.g6822c7b5.e 2.1 MB/s | 163 kB 00:00 2024-02-17T11:45:32.886 INFO:teuthology.orchestra.run.smithi094.stdout:(16/176): librados-devel-19.0.0-1515.g6822c7b5. 1.2 MB/s | 127 kB 00:00 2024-02-17T11:45:32.945 INFO:teuthology.orchestra.run.smithi125.stdout:(63/176): python3-attrs-20.3.0-7.el9.noarch.rpm 247 kB/s | 78 kB 00:00 2024-02-17T11:45:32.995 INFO:teuthology.orchestra.run.smithi094.stdout:(17/176): libradosstriper1-19.0.0-1515.g6822c7b 4.5 MB/s | 499 kB 00:00 2024-02-17T11:45:33.045 INFO:teuthology.orchestra.run.smithi125.stdout:(64/176): postgresql-server-13.11-1.el9.x86_64. 10 MB/s | 5.7 MB 00:00 2024-02-17T11:45:33.046 INFO:teuthology.orchestra.run.smithi189.stdout:(2/176): ceph-fuse-19.0.0-1515.g6822c7b5.el9.x8 916 kB/s | 885 kB 00:00 2024-02-17T11:45:33.078 INFO:teuthology.orchestra.run.smithi125.stdout:(65/176): protobuf-3.14.0-13.el9.x86_64.rpm 2.2 MB/s | 1.0 MB 00:00 2024-02-17T11:45:33.305 INFO:teuthology.orchestra.run.smithi189.stdout:(3/176): ceph-immutable-object-cache-19.0.0-151 564 kB/s | 146 kB 00:00 2024-02-17T11:45:33.371 INFO:teuthology.orchestra.run.smithi125.stdout:(66/176): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 5.8 MB 00:00 2024-02-17T11:45:33.396 INFO:teuthology.orchestra.run.smithi125.stdout:(67/176): python3-jinja2-2.11.3-5.el9.noarch.rp 653 kB/s | 227 kB 00:00 2024-02-17T11:45:33.421 INFO:teuthology.orchestra.run.smithi125.stdout:(68/176): python3-jsonpatch-1.21-16.el9.noarch. 72 kB/s | 24 kB 00:00 2024-02-17T11:45:33.446 INFO:teuthology.orchestra.run.smithi125.stdout:(69/176): python3-jsonpointer-2.0-4.el9.noarch. 242 kB/s | 18 kB 00:00 2024-02-17T11:45:33.471 INFO:teuthology.orchestra.run.smithi125.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-02-17T11:45:33.513 INFO:teuthology.orchestra.run.smithi125.stdout:(71/176): python3-mako-1.1.4-6.el9.noarch.rpm 1.5 MB/s | 144 kB 00:00 2024-02-17T11:45:33.587 INFO:teuthology.orchestra.run.smithi094.stdout:(18/176): librgw2-19.0.0-1515.g6822c7b5.el9.x86 8.6 MB/s | 5.1 MB 00:00 2024-02-17T11:45:33.654 INFO:teuthology.orchestra.run.smithi094.stdout:(19/176): python3-ceph-argparse-19.0.0-1515.g68 679 kB/s | 45 kB 00:00 2024-02-17T11:45:33.729 INFO:teuthology.orchestra.run.smithi094.stdout:(20/176): python3-ceph-common-19.0.0-1515.g6822 1.7 MB/s | 131 kB 00:00 2024-02-17T11:45:33.730 INFO:teuthology.orchestra.run.smithi125.stdout:(72/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-02-17T11:45:33.756 INFO:teuthology.orchestra.run.smithi125.stdout:(73/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 768 kB/s | 217 kB 00:00 2024-02-17T11:45:33.781 INFO:teuthology.orchestra.run.smithi125.stdout:(74/176): python3-oauthlib-3.1.1-5.el9.noarch.r 657 kB/s | 174 kB 00:00 2024-02-17T11:45:33.806 INFO:teuthology.orchestra.run.smithi125.stdout:(75/176): python3-packaging-20.9-5.el9.noarch.r 921 kB/s | 69 kB 00:00 2024-02-17T11:45:33.813 INFO:teuthology.orchestra.run.smithi094.stdout:(21/176): python3-cephfs-19.0.0-1515.g6822c7b5. 1.9 MB/s | 162 kB 00:00 2024-02-17T11:45:33.831 INFO:teuthology.orchestra.run.smithi125.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 4.7 MB/s | 238 kB 00:00 2024-02-17T11:45:33.848 INFO:teuthology.orchestra.run.smithi125.stdout:(77/176): python3-prettytable-0.7.2-27.el9.noar 616 kB/s | 41 kB 00:00 2024-02-17T11:45:33.882 INFO:teuthology.orchestra.run.smithi125.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-02-17T11:45:33.905 INFO:teuthology.orchestra.run.smithi094.stdout:(22/176): python3-rados-19.0.0-1515.g6822c7b5.e 3.4 MB/s | 321 kB 00:00 2024-02-17T11:45:33.907 INFO:teuthology.orchestra.run.smithi125.stdout:(79/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 132 kB 00:00 2024-02-17T11:45:33.924 INFO:teuthology.orchestra.run.smithi125.stdout:(80/176): python3-pytz-2021.1-5.el9.noarch.rpm 629 kB/s | 47 kB 00:00 2024-02-17T11:45:33.949 INFO:teuthology.orchestra.run.smithi125.stdout:(81/176): python3-requests-oauthlib-1.3.0-12.el 642 kB/s | 43 kB 00:00 2024-02-17T11:45:33.974 INFO:teuthology.orchestra.run.smithi125.stdout:(82/176): python3-toml-0.10.2-6.el9.noarch.rpm 772 kB/s | 39 kB 00:00 2024-02-17T11:45:33.997 INFO:teuthology.orchestra.run.smithi094.stdout:(23/176): python3-rbd-19.0.0-1515.g6822c7b5.el9 3.2 MB/s | 298 kB 00:00 2024-02-17T11:45:34.080 INFO:teuthology.orchestra.run.smithi094.stdout:(24/176): python3-rgw-19.0.0-1515.g6822c7b5.el9 1.2 MB/s | 99 kB 00:00 2024-02-17T11:45:34.164 INFO:teuthology.orchestra.run.smithi094.stdout:(25/176): rbd-fuse-19.0.0-1515.g6822c7b5.el9.x8 1.0 MB/s | 85 kB 00:00 2024-02-17T11:45:34.191 INFO:teuthology.orchestra.run.smithi125.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 169 kB/s | 41 kB 00:00 2024-02-17T11:45:34.217 INFO:teuthology.orchestra.run.smithi125.stdout:(84/176): qatlib-23.11.0-1.el9.x86_64.rpm 897 kB/s | 215 kB 00:00 2024-02-17T11:45:34.409 INFO:teuthology.orchestra.run.smithi125.stdout:(85/176): qatlib-service-23.11.0-1.el9.x86_64.r 156 kB/s | 34 kB 00:00 2024-02-17T11:45:34.451 INFO:teuthology.orchestra.run.smithi125.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 187 kB/s | 43 kB 00:00 2024-02-17T11:45:34.610 INFO:teuthology.orchestra.run.smithi125.stdout:(87/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-02-17T11:45:34.631 INFO:teuthology.orchestra.run.smithi094.stdout:(26/176): ceph-test-19.0.0-1515.g6822c7b5.el9.x 17 MB/s | 47 MB 00:02 2024-02-17T11:45:34.635 INFO:teuthology.orchestra.run.smithi125.stdout:(88/176): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 300 kB 00:00 2024-02-17T11:45:34.647 INFO:teuthology.orchestra.run.smithi189.stdout:(4/176): ceph-base-19.0.0-1515.g6822c7b5.el9.x8 1.8 MB/s | 5.3 MB 00:02 2024-02-17T11:45:34.660 INFO:teuthology.orchestra.run.smithi125.stdout:(89/176): uuid-1.6.2-55.el9.x86_64.rpm 273 kB/s | 56 kB 00:00 2024-02-17T11:45:34.686 INFO:teuthology.orchestra.run.smithi125.stdout:(90/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2024-02-17T11:45:34.688 INFO:teuthology.orchestra.run.smithi189.stdout:(5/176): ceph-mds-19.0.0-1515.g6822c7b5.el9.x86 1.5 MB/s | 2.1 MB 00:01 2024-02-17T11:45:34.698 INFO:teuthology.orchestra.run.smithi094.stdout:(27/176): rbd-mirror-19.0.0-1515.g6822c7b5.el9. 5.8 MB/s | 3.1 MB 00:00 2024-02-17T11:45:34.711 INFO:teuthology.orchestra.run.smithi125.stdout:(91/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-02-17T11:45:34.723 INFO:teuthology.orchestra.run.smithi094.stdout:(28/176): rbd-nbd-19.0.0-1515.g6822c7b5.el9.x86 1.8 MB/s | 171 kB 00:00 2024-02-17T11:45:34.737 INFO:teuthology.orchestra.run.smithi125.stdout:(92/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 582 kB/s | 44 kB 00:00 2024-02-17T11:45:34.748 INFO:teuthology.orchestra.run.smithi094.stdout:(29/176): ceph-grafana-dashboards-19.0.0-1515.g 629 kB/s | 31 kB 00:00 2024-02-17T11:45:34.762 INFO:teuthology.orchestra.run.smithi125.stdout:(93/176): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-02-17T11:45:34.773 INFO:teuthology.orchestra.run.smithi094.stdout:(30/176): ceph-mgr-cephadm-19.0.0-1515.g6822c7b 2.7 MB/s | 138 kB 00:00 2024-02-17T11:45:34.787 INFO:teuthology.orchestra.run.smithi125.stdout:(94/176): jq-1.6-16.el9.x86_64.rpm 3.7 MB/s | 186 kB 00:00 2024-02-17T11:45:34.829 INFO:teuthology.orchestra.run.smithi125.stdout:(95/176): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 802 kB 00:00 2024-02-17T11:45:34.854 INFO:teuthology.orchestra.run.smithi125.stdout:(96/176): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 194 kB 00:00 2024-02-17T11:45:34.879 INFO:teuthology.orchestra.run.smithi125.stdout:(97/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.3 MB/s | 217 kB 00:00 2024-02-17T11:45:34.904 INFO:teuthology.orchestra.run.smithi125.stdout:(98/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-17T11:45:34.915 INFO:teuthology.orchestra.run.smithi094.stdout:(31/176): ceph-mgr-dashboard-19.0.0-1515.g6822c 11 MB/s | 1.8 MB 00:00 2024-02-17T11:45:34.955 INFO:teuthology.orchestra.run.smithi125.stdout:(99/176): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-02-17T11:45:34.980 INFO:teuthology.orchestra.run.smithi125.stdout:(100/176): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-02-17T11:45:35.005 INFO:teuthology.orchestra.run.smithi125.stdout:(101/176): python3-pycparser-2.20-6.el9.noarch. 1.2 MB/s | 124 kB 00:00 2024-02-17T11:45:35.007 INFO:teuthology.orchestra.run.smithi094.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1515.g68 2.6 MB/s | 247 kB 00:00 2024-02-17T11:45:35.031 INFO:teuthology.orchestra.run.smithi125.stdout:(102/176): python3-requests-2.25.1-8.el9.noarch 1.5 MB/s | 113 kB 00:00 2024-02-17T11:45:35.056 INFO:teuthology.orchestra.run.smithi125.stdout:(103/176): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-02-17T11:45:35.091 INFO:teuthology.orchestra.run.smithi094.stdout:(33/176): ceph-mgr-rook-19.0.0-1515.g6822c7b5.e 628 kB/s | 52 kB 00:00 2024-02-17T11:45:35.174 INFO:teuthology.orchestra.run.smithi094.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1515.g6 184 kB/s | 15 kB 00:00 2024-02-17T11:45:35.255 INFO:teuthology.orchestra.run.smithi189.stdout:(6/176): ceph-mgr-19.0.0-1515.g6822c7b5.el9.x86 2.4 MB/s | 1.5 MB 00:00 2024-02-17T11:45:35.291 INFO:teuthology.orchestra.run.smithi094.stdout:(35/176): ceph-mgr-diskprediction-local-19.0.0- 14 MB/s | 7.4 MB 00:00 2024-02-17T11:45:35.317 INFO:teuthology.orchestra.run.smithi094.stdout:(36/176): ceph-volume-19.0.0-1515.g6822c7b5.el9 1.8 MB/s | 264 kB 00:00 2024-02-17T11:45:35.339 INFO:teuthology.orchestra.run.smithi125.stdout:(104/176): lua-devel-5.4.4-4.el9.x86_64.rpm 72 kB/s | 22 kB 00:00 2024-02-17T11:45:35.358 INFO:teuthology.orchestra.run.smithi094.stdout:(37/176): cephadm-19.0.0-1515.g6822c7b5.el9.noa 11 MB/s | 748 kB 00:00 2024-02-17T11:45:35.384 INFO:teuthology.orchestra.run.smithi094.stdout:(38/176): cephfs-top-19.0.0-1515.g6822c7b5.el9. 273 kB/s | 18 kB 00:00 2024-02-17T11:45:35.592 INFO:teuthology.orchestra.run.smithi094.stdout:(39/176): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 662 kB 00:00 2024-02-17T11:45:35.598 INFO:teuthology.orchestra.run.smithi125.stdout:(105/176): python3-devel-3.9.17-2.el9.x86_64.rp 414 kB/s | 245 kB 00:00 2024-02-17T11:45:35.626 INFO:teuthology.orchestra.run.smithi094.stdout:(40/176): bison-3.7.4-5.el9.x86_64.rpm 3.7 MB/s | 920 kB 00:00 2024-02-17T11:45:35.651 INFO:teuthology.orchestra.run.smithi094.stdout:(41/176): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 106 kB 00:00 2024-02-17T11:45:35.676 INFO:teuthology.orchestra.run.smithi094.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 409 kB/s | 20 kB 00:00 2024-02-17T11:45:35.710 INFO:teuthology.orchestra.run.smithi094.stdout:(43/176): flex-2.6.4-9.el9.x86_64.rpm 5.1 MB/s | 307 kB 00:00 2024-02-17T11:45:35.736 INFO:teuthology.orchestra.run.smithi094.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 542 kB/s | 32 kB 00:00 2024-02-17T11:45:35.853 INFO:teuthology.orchestra.run.smithi094.stdout:(45/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-02-17T11:45:35.865 INFO:teuthology.orchestra.run.smithi125.stdout:(106/176): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-02-17T11:45:35.878 INFO:teuthology.orchestra.run.smithi094.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 121 kB/s | 17 kB 00:00 2024-02-17T11:45:35.890 INFO:teuthology.orchestra.run.smithi125.stdout:(107/176): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-02-17T11:45:35.903 INFO:teuthology.orchestra.run.smithi094.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 201 kB/s | 10 kB 00:00 2024-02-17T11:45:35.915 INFO:teuthology.orchestra.run.smithi125.stdout:(108/176): abseil-cpp-20211102.0-3.el9.x86_64.r 952 kB/s | 548 kB 00:00 2024-02-17T11:45:35.929 INFO:teuthology.orchestra.run.smithi094.stdout:(48/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-02-17T11:45:35.941 INFO:teuthology.orchestra.run.smithi125.stdout:(109/176): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-02-17T11:45:35.954 INFO:teuthology.orchestra.run.smithi094.stdout:(49/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-02-17T11:45:35.979 INFO:teuthology.orchestra.run.smithi094.stdout:(50/176): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-02-17T11:45:35.983 INFO:teuthology.orchestra.run.smithi125.stdout:(110/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 388 kB/s | 26 kB 00:00 2024-02-17T11:45:36.008 INFO:teuthology.orchestra.run.smithi125.stdout:(111/176): liboath-2.6.7-2.el9.x86_64.rpm 726 kB/s | 49 kB 00:00 2024-02-17T11:45:36.012 INFO:teuthology.orchestra.run.smithi094.stdout:(51/176): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2024-02-17T11:45:36.038 INFO:teuthology.orchestra.run.smithi094.stdout:(52/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 423 kB/s | 24 kB 00:00 2024-02-17T11:45:36.064 INFO:teuthology.orchestra.run.smithi094.stdout:(53/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 920 kB/s | 46 kB 00:00 2024-02-17T11:45:36.088 INFO:teuthology.orchestra.run.smithi094.stdout:(54/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-02-17T11:45:36.089 INFO:teuthology.orchestra.run.smithi189.stdout:(7/176): ceph-mon-19.0.0-1515.g6822c7b5.el9.x86 3.3 MB/s | 4.7 MB 00:01 2024-02-17T11:45:36.100 INFO:teuthology.orchestra.run.smithi125.stdout:(112/176): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-02-17T11:45:36.130 INFO:teuthology.orchestra.run.smithi094.stdout:(55/176): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2024-02-17T11:45:36.147 INFO:teuthology.orchestra.run.smithi094.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 588 kB/s | 34 kB 00:00 2024-02-17T11:45:36.242 INFO:teuthology.orchestra.run.smithi125.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 647 kB/s | 151 kB 00:00 2024-02-17T11:45:36.314 INFO:teuthology.orchestra.run.smithi094.stdout:(57/176): openblas-openmp-0.3.21-2.el9.x86_64.r 26 MB/s | 4.7 MB 00:00 2024-02-17T11:45:36.373 INFO:teuthology.orchestra.run.smithi094.stdout:(58/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-02-17T11:45:36.406 INFO:teuthology.orchestra.run.smithi094.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 17 MB/s | 1.5 MB 00:00 2024-02-17T11:45:36.440 INFO:teuthology.orchestra.run.smithi094.stdout:(60/176): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-02-17T11:45:36.465 INFO:teuthology.orchestra.run.smithi094.stdout:(61/176): postgresql-private-libs-13.11-1.el9.x 2.3 MB/s | 134 kB 00:00 2024-02-17T11:45:36.483 INFO:teuthology.orchestra.run.smithi125.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.1 MB/s | 838 kB 00:00 2024-02-17T11:45:36.609 INFO:teuthology.orchestra.run.smithi125.stdout:(115/176): python3-asyncssh-2.13.2-4.el9.noarch 1.5 MB/s | 548 kB 00:00 2024-02-17T11:45:36.640 INFO:teuthology.orchestra.run.smithi094.stdout:(62/176): postgresql-server-13.11-1.el9.x86_64. 28 MB/s | 5.7 MB 00:00 2024-02-17T11:45:36.650 INFO:teuthology.orchestra.run.smithi125.stdout:(116/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 260 kB/s | 43 kB 00:00 2024-02-17T11:45:36.674 INFO:teuthology.orchestra.run.smithi094.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-02-17T11:45:36.684 INFO:teuthology.orchestra.run.smithi125.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-02-17T11:45:36.699 INFO:teuthology.orchestra.run.smithi094.stdout:(64/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-02-17T11:45:36.709 INFO:teuthology.orchestra.run.smithi125.stdout:(118/176): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-02-17T11:45:36.851 INFO:teuthology.orchestra.run.smithi125.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-02-17T11:45:36.875 INFO:teuthology.orchestra.run.smithi094.stdout:(65/176): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 5.8 MB 00:00 2024-02-17T11:45:36.900 INFO:teuthology.orchestra.run.smithi094.stdout:(66/176): python3-jinja2-2.11.3-5.el9.noarch.rp 1.1 MB/s | 227 kB 00:00 2024-02-17T11:45:36.925 INFO:teuthology.orchestra.run.smithi094.stdout:(67/176): python3-jsonpatch-1.21-16.el9.noarch. 490 kB/s | 24 kB 00:00 2024-02-17T11:45:36.950 INFO:teuthology.orchestra.run.smithi094.stdout:(68/176): python3-jsonpointer-2.0-4.el9.noarch. 359 kB/s | 18 kB 00:00 2024-02-17T11:45:36.960 INFO:teuthology.orchestra.run.smithi125.stdout:(120/176): python3-cherrypy-18.6.1-2.el9.noarch 1.4 MB/s | 358 kB 00:00 2024-02-17T11:45:36.976 INFO:teuthology.orchestra.run.smithi094.stdout:(69/176): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-02-17T11:45:37.001 INFO:teuthology.orchestra.run.smithi094.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 635 kB/s | 32 kB 00:00 2024-02-17T11:45:37.085 INFO:teuthology.orchestra.run.smithi125.stdout:(121/176): python3-colorama-0.4.4-12.el9.noarch 282 kB/s | 35 kB 00:00 2024-02-17T11:45:37.114 INFO:teuthology.orchestra.run.smithi189.stdout:(8/176): ceph-common-19.0.0-1515.g6822c7b5.el9. 3.7 MB/s | 20 MB 00:05 2024-02-17T11:45:37.119 INFO:teuthology.orchestra.run.smithi125.stdout:(122/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.1 MB/s | 303 kB 00:00 2024-02-17T11:45:37.151 INFO:teuthology.orchestra.run.smithi094.stdout:(71/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2024-02-17T11:45:37.164 INFO:teuthology.orchestra.run.smithi189.stdout:(9/176): ceph-selinux-19.0.0-1515.g6822c7b5.el9 507 kB/s | 25 kB 00:00 2024-02-17T11:45:37.176 INFO:teuthology.orchestra.run.smithi094.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2024-02-17T11:45:37.202 INFO:teuthology.orchestra.run.smithi094.stdout:(73/176): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-02-17T11:45:37.227 INFO:teuthology.orchestra.run.smithi094.stdout:(74/176): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-02-17T11:45:37.236 INFO:teuthology.orchestra.run.smithi125.stdout:(123/176): python3-google-auth-2.27.0-1.el9.noa 1.3 MB/s | 203 kB 00:00 2024-02-17T11:45:37.252 INFO:teuthology.orchestra.run.smithi094.stdout:(75/176): python3-prettytable-0.7.2-27.el9.noar 821 kB/s | 41 kB 00:00 2024-02-17T11:45:37.278 INFO:teuthology.orchestra.run.smithi094.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-02-17T11:45:37.303 INFO:teuthology.orchestra.run.smithi125.stdout:(124/176): libarrow-9.0.0-10.el9.x86_64.rpm 3.1 MB/s | 4.4 MB 00:01 2024-02-17T11:45:37.303 INFO:teuthology.orchestra.run.smithi094.stdout:(77/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-02-17T11:45:37.328 INFO:teuthology.orchestra.run.smithi094.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-02-17T11:45:37.345 INFO:teuthology.orchestra.run.smithi125.stdout:(125/176): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-02-17T11:45:37.354 INFO:teuthology.orchestra.run.smithi094.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 945 kB/s | 47 kB 00:00 2024-02-17T11:45:37.379 INFO:teuthology.orchestra.run.smithi094.stdout:(80/176): python3-requests-oauthlib-1.3.0-12.el 849 kB/s | 43 kB 00:00 2024-02-17T11:45:37.387 INFO:teuthology.orchestra.run.smithi125.stdout:(126/176): python3-jaraco-classes-3.2.1-5.el9.n 424 kB/s | 18 kB 00:00 2024-02-17T11:45:37.412 INFO:teuthology.orchestra.run.smithi125.stdout:(127/176): python3-grpcio-tools-1.46.7-10.el9.x 823 kB/s | 144 kB 00:00 2024-02-17T11:45:37.437 INFO:teuthology.orchestra.run.smithi125.stdout:(128/176): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-02-17T11:45:37.463 INFO:teuthology.orchestra.run.smithi125.stdout:(129/176): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-02-17T11:45:37.496 INFO:teuthology.orchestra.run.smithi125.stdout:(130/176): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-02-17T11:45:37.555 INFO:teuthology.orchestra.run.smithi125.stdout:(131/176): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-02-17T11:45:37.614 INFO:teuthology.orchestra.run.smithi125.stdout:(132/176): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-02-17T11:45:37.643 INFO:teuthology.orchestra.run.smithi094.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 147 kB/s | 39 kB 00:00 2024-02-17T11:45:37.655 INFO:teuthology.orchestra.run.smithi125.stdout:(133/176): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2024-02-17T11:45:37.756 INFO:teuthology.orchestra.run.smithi125.stdout:(134/176): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-02-17T11:45:37.806 INFO:teuthology.orchestra.run.smithi189.stdout:(10/176): ceph-osd-19.0.0-1515.g6822c7b5.el9.x8 6.6 MB/s | 17 MB 00:02 2024-02-17T11:45:37.838 INFO:teuthology.orchestra.run.smithi094.stdout:(82/176): ceph-common-19.0.0-1515.g6822c7b5.el9 2.0 MB/s | 20 MB 00:09 2024-02-17T11:45:37.881 INFO:teuthology.orchestra.run.smithi125.stdout:(135/176): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2024-02-17T11:45:37.906 INFO:teuthology.orchestra.run.smithi189.stdout:(11/176): ceph-radosgw-19.0.0-1515.g6822c7b5.el 5.4 MB/s | 9.8 MB 00:01 2024-02-17T11:45:37.931 INFO:teuthology.orchestra.run.smithi125.stdout:(136/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-02-17T11:45:37.932 INFO:teuthology.orchestra.run.smithi189.stdout:(12/176): cephfs-mirror-19.0.0-1515.g6822c7b5.e 1.7 MB/s | 217 kB 00:00 2024-02-17T11:45:37.957 INFO:teuthology.orchestra.run.smithi189.stdout:(13/176): cephfs-shell-19.0.0-1515.g6822c7b5.el 541 kB/s | 27 kB 00:00 2024-02-17T11:45:37.973 INFO:teuthology.orchestra.run.smithi125.stdout:(137/176): python3-pyperclip-1.8.2-2.el9.noarch 676 kB/s | 28 kB 00:00 2024-02-17T11:45:37.982 INFO:teuthology.orchestra.run.smithi189.stdout:(14/176): libcephfs-devel-19.0.0-1515.g6822c7b5 623 kB/s | 31 kB 00:00 2024-02-17T11:45:37.996 INFO:teuthology.orchestra.run.smithi094.stdout:(83/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-02-17T11:45:38.015 INFO:teuthology.orchestra.run.smithi125.stdout:(138/176): python3-repoze-lru-0.7-16.el9.noarch 739 kB/s | 31 kB 00:00 2024-02-17T11:45:38.021 INFO:teuthology.orchestra.run.smithi094.stdout:(84/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 109 kB/s | 41 kB 00:00 2024-02-17T11:45:38.033 INFO:teuthology.orchestra.run.smithi189.stdout:(15/176): libcephsqlite-19.0.0-1515.g6822c7b5.e 3.2 MB/s | 163 kB 00:00 2024-02-17T11:45:38.047 INFO:teuthology.orchestra.run.smithi094.stdout:(85/176): qatlib-23.11.0-1.el9.x86_64.rpm 1.0 MB/s | 215 kB 00:00 2024-02-17T11:45:38.072 INFO:teuthology.orchestra.run.smithi094.stdout:(86/176): qatlib-service-23.11.0-1.el9.x86_64.r 679 kB/s | 34 kB 00:00 2024-02-17T11:45:38.082 INFO:teuthology.orchestra.run.smithi125.stdout:(139/176): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-02-17T11:45:38.083 INFO:teuthology.orchestra.run.smithi189.stdout:(16/176): librados-devel-19.0.0-1515.g6822c7b5. 2.5 MB/s | 127 kB 00:00 2024-02-17T11:45:38.097 INFO:teuthology.orchestra.run.smithi094.stdout:(87/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 581 kB/s | 43 kB 00:00 2024-02-17T11:45:38.116 INFO:teuthology.orchestra.run.smithi189.stdout:(17/176): libcephfs2-19.0.0-1515.g6822c7b5.el9. 4.6 MB/s | 743 kB 00:00 2024-02-17T11:45:38.131 INFO:teuthology.orchestra.run.smithi094.stdout:(88/176): socat-1.7.4.1-5.el9.x86_64.rpm 3.5 MB/s | 300 kB 00:00 2024-02-17T11:45:38.132 INFO:teuthology.orchestra.run.smithi125.stdout:(140/176): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-02-17T11:45:38.156 INFO:teuthology.orchestra.run.smithi094.stdout:(89/176): uuid-1.6.2-55.el9.x86_64.rpm 972 kB/s | 56 kB 00:00 2024-02-17T11:45:38.174 INFO:teuthology.orchestra.run.smithi125.stdout:(141/176): python3-tempora-5.0.0-2.el9.noarch.r 859 kB/s | 36 kB 00:00 2024-02-17T11:45:38.181 INFO:teuthology.orchestra.run.smithi094.stdout:(90/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 87 kB 00:00 2024-02-17T11:45:38.183 INFO:teuthology.orchestra.run.smithi189.stdout:(18/176): libradosstriper1-19.0.0-1515.g6822c7b 4.9 MB/s | 499 kB 00:00 2024-02-17T11:45:38.206 INFO:teuthology.orchestra.run.smithi094.stdout:(91/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 834 kB/s | 63 kB 00:00 2024-02-17T11:45:38.216 INFO:teuthology.orchestra.run.smithi125.stdout:(142/176): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-02-17T11:45:38.232 INFO:teuthology.orchestra.run.smithi094.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-02-17T11:45:38.234 INFO:teuthology.orchestra.run.smithi189.stdout:(19/176): python3-ceph-argparse-19.0.0-1515.g68 904 kB/s | 45 kB 00:00 2024-02-17T11:45:38.257 INFO:teuthology.orchestra.run.smithi094.stdout:(93/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 586 kB/s | 44 kB 00:00 2024-02-17T11:45:38.282 INFO:teuthology.orchestra.run.smithi094.stdout:(94/176): jq-1.6-16.el9.x86_64.rpm 2.4 MB/s | 186 kB 00:00 2024-02-17T11:45:38.291 INFO:teuthology.orchestra.run.smithi125.stdout:(143/176): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-02-17T11:45:38.292 INFO:teuthology.orchestra.run.smithi189.stdout:(20/176): python3-ceph-common-19.0.0-1515.g6822 2.2 MB/s | 131 kB 00:00 2024-02-17T11:45:38.316 INFO:teuthology.orchestra.run.smithi094.stdout:(95/176): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 802 kB 00:00 2024-02-17T11:45:38.341 INFO:teuthology.orchestra.run.smithi094.stdout:(96/176): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 194 kB 00:00 2024-02-17T11:45:38.342 INFO:teuthology.orchestra.run.smithi125.stdout:(144/176): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-02-17T11:45:38.351 INFO:teuthology.orchestra.run.smithi189.stdout:(21/176): python3-cephfs-19.0.0-1515.g6822c7b5. 2.7 MB/s | 162 kB 00:00 2024-02-17T11:45:38.366 INFO:teuthology.orchestra.run.smithi094.stdout:(97/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.6 MB/s | 217 kB 00:00 2024-02-17T11:45:38.391 INFO:teuthology.orchestra.run.smithi094.stdout:(98/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.2 MB/s | 241 kB 00:00 2024-02-17T11:45:38.417 INFO:teuthology.orchestra.run.smithi094.stdout:(99/176): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 103 kB 00:00 2024-02-17T11:45:38.426 INFO:teuthology.orchestra.run.smithi189.stdout:(22/176): python3-rados-19.0.0-1515.g6822c7b5.e 4.2 MB/s | 321 kB 00:00 2024-02-17T11:45:38.451 INFO:teuthology.orchestra.run.smithi094.stdout:(100/176): python3-cryptography-36.0.1-4.el9.x8 14 MB/s | 1.1 MB 00:00 2024-02-17T11:45:38.459 INFO:teuthology.orchestra.run.smithi125.stdout:(145/176): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-02-17T11:45:38.475 INFO:teuthology.orchestra.run.smithi094.stdout:(101/176): python3-pycparser-2.20-6.el9.noarch. 1.5 MB/s | 124 kB 00:00 2024-02-17T11:45:38.492 INFO:teuthology.orchestra.run.smithi125.stdout:(146/176): python3-zc-lockfile-2.0-10.el9.noarc 599 kB/s | 20 kB 00:00 2024-02-17T11:45:38.501 INFO:teuthology.orchestra.run.smithi094.stdout:(102/176): python3-requests-2.25.1-8.el9.noarch 1.4 MB/s | 113 kB 00:00 2024-02-17T11:45:38.502 INFO:teuthology.orchestra.run.smithi189.stdout:(23/176): python3-rbd-19.0.0-1515.g6822c7b5.el9 3.9 MB/s | 298 kB 00:00 2024-02-17T11:45:38.526 INFO:teuthology.orchestra.run.smithi094.stdout:(103/176): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-02-17T11:45:38.552 INFO:teuthology.orchestra.run.smithi189.stdout:(24/176): python3-rgw-19.0.0-1515.g6822c7b5.el9 1.9 MB/s | 99 kB 00:00 2024-02-17T11:45:38.559 INFO:teuthology.orchestra.run.smithi125.stdout:(147/176): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-02-17T11:45:38.602 INFO:teuthology.orchestra.run.smithi189.stdout:(25/176): rbd-fuse-19.0.0-1515.g6822c7b5.el9.x8 1.7 MB/s | 85 kB 00:00 2024-02-17T11:45:38.743 INFO:teuthology.orchestra.run.smithi094.stdout:(104/176): lua-devel-5.4.4-4.el9.x86_64.rpm 92 kB/s | 22 kB 00:00 2024-02-17T11:45:38.810 INFO:teuthology.orchestra.run.smithi094.stdout:(105/176): protobuf-compiler-3.14.0-13.el9.x86_ 3.0 MB/s | 863 kB 00:00 2024-02-17T11:45:38.868 INFO:teuthology.orchestra.run.smithi125.stdout:(148/176): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-02-17T11:45:38.877 INFO:teuthology.orchestra.run.smithi094.stdout:(106/176): python3-devel-3.9.17-2.el9.x86_64.rp 613 kB/s | 245 kB 00:00 2024-02-17T11:45:38.919 INFO:teuthology.orchestra.run.smithi189.stdout:(26/176): librgw2-19.0.0-1515.g6822c7b5.el9.x86 6.4 MB/s | 5.1 MB 00:00 2024-02-17T11:45:38.934 INFO:teuthology.orchestra.run.smithi125.stdout:(149/176): python3-kubernetes-26.1.0-2.el9.noar 710 kB/s | 1.0 MB 00:01 2024-02-17T11:45:38.978 INFO:teuthology.orchestra.run.smithi189.stdout:(27/176): rbd-nbd-19.0.0-1515.g6822c7b5.el9.x86 2.9 MB/s | 171 kB 00:00 2024-02-17T11:45:39.010 INFO:teuthology.orchestra.run.smithi094.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-02-17T11:45:39.044 INFO:teuthology.orchestra.run.smithi094.stdout:(108/176): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-02-17T11:45:39.053 INFO:teuthology.orchestra.run.smithi189.stdout:(28/176): rbd-mirror-19.0.0-1515.g6822c7b5.el9. 6.9 MB/s | 3.1 MB 00:00 2024-02-17T11:45:39.078 INFO:teuthology.orchestra.run.smithi189.stdout:(29/176): ceph-grafana-dashboards-19.0.0-1515.g 313 kB/s | 31 kB 00:00 2024-02-17T11:45:39.086 INFO:teuthology.orchestra.run.smithi094.stdout:(109/176): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-02-17T11:45:39.111 INFO:teuthology.orchestra.run.smithi094.stdout:(110/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-02-17T11:45:39.112 INFO:teuthology.orchestra.run.smithi189.stdout:(30/176): ceph-mgr-cephadm-19.0.0-1515.g6822c7b 2.3 MB/s | 138 kB 00:00 2024-02-17T11:45:39.145 INFO:teuthology.orchestra.run.smithi094.stdout:(111/176): liboath-2.6.7-2.el9.x86_64.rpm 833 kB/s | 49 kB 00:00 2024-02-17T11:45:39.170 INFO:teuthology.orchestra.run.smithi094.stdout:(112/176): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-02-17T11:45:39.237 INFO:teuthology.orchestra.run.smithi094.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-02-17T11:45:39.271 INFO:teuthology.orchestra.run.smithi094.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 838 kB 00:00 2024-02-17T11:45:39.354 INFO:teuthology.orchestra.run.smithi094.stdout:(115/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 522 kB/s | 43 kB 00:00 2024-02-17T11:45:39.380 INFO:teuthology.orchestra.run.smithi094.stdout:(116/176): python3-asyncssh-2.13.2-4.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-02-17T11:45:39.385 INFO:teuthology.orchestra.run.smithi125.stdout:(150/176): librbd1-19.0.0-1515.g6822c7b5.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-02-17T11:45:39.405 INFO:teuthology.orchestra.run.smithi094.stdout:(117/176): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-02-17T11:45:39.418 INFO:teuthology.orchestra.run.smithi125.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-02-17T11:45:39.472 INFO:teuthology.orchestra.run.smithi094.stdout:(118/176): libarrow-9.0.0-10.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-02-17T11:45:39.497 INFO:teuthology.orchestra.run.smithi094.stdout:(119/176): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-02-17T11:45:39.519 INFO:teuthology.orchestra.run.smithi125.stdout:(152/176): librados2-19.0.0-1515.g6822c7b5.el9. 5.2 MB/s | 3.4 MB 00:00 2024-02-17T11:45:39.521 INFO:teuthology.orchestra.run.smithi189.stdout:(31/176): ceph-mgr-dashboard-19.0.0-1515.g6822c 4.0 MB/s | 1.8 MB 00:00 2024-02-17T11:45:39.522 INFO:teuthology.orchestra.run.smithi094.stdout:(120/176): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-02-17T11:45:39.556 INFO:teuthology.orchestra.run.smithi094.stdout:(121/176): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-02-17T11:45:39.560 INFO:teuthology.orchestra.run.smithi125.stdout:(153/176): NetworkManager-1.45.91-1.el9.x86_64. 16 MB/s | 2.3 MB 00:00 2024-02-17T11:45:39.581 INFO:teuthology.orchestra.run.smithi094.stdout:(122/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-02-17T11:45:39.587 INFO:teuthology.orchestra.run.smithi189.stdout:(32/176): ceph-mgr-modules-core-19.0.0-1515.g68 3.6 MB/s | 247 kB 00:00 2024-02-17T11:45:39.606 INFO:teuthology.orchestra.run.smithi094.stdout:(123/176): python3-colorama-0.4.4-12.el9.noarch 422 kB/s | 35 kB 00:00 2024-02-17T11:45:39.619 INFO:teuthology.orchestra.run.smithi125.stdout:(154/176): NetworkManager-libnm-1.45.91-1.el9.x 18 MB/s | 1.8 MB 00:00 2024-02-17T11:45:39.629 INFO:teuthology.orchestra.run.smithi189.stdout:(33/176): ceph-mgr-rook-19.0.0-1515.g6822c7b5.e 1.2 MB/s | 52 kB 00:00 2024-02-17T11:45:39.631 INFO:teuthology.orchestra.run.smithi094.stdout:(124/176): python3-google-auth-2.27.0-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-02-17T11:45:39.636 INFO:teuthology.orchestra.run.smithi125.stdout:(155/176): NetworkManager-team-1.45.91-1.el9.x8 564 kB/s | 41 kB 00:00 2024-02-17T11:45:39.663 INFO:teuthology.orchestra.run.smithi189.stdout:(34/176): ceph-prometheus-alerts-19.0.0-1515.g6 459 kB/s | 15 kB 00:00 2024-02-17T11:45:39.670 INFO:teuthology.orchestra.run.smithi125.stdout:(156/176): NetworkManager-tui-1.45.91-1.el9.x86 4.8 MB/s | 246 kB 00:00 2024-02-17T11:45:39.673 INFO:teuthology.orchestra.run.smithi094.stdout:(125/176): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-02-17T11:45:39.695 INFO:teuthology.orchestra.run.smithi125.stdout:(157/176): acl-2.3.1-4.el9.x86_64.rpm 1.1 MB/s | 69 kB 00:00 2024-02-17T11:45:39.699 INFO:teuthology.orchestra.run.smithi094.stdout:(126/176): python3-jaraco-8.2.1-3.el9.noarch.rp 160 kB/s | 11 kB 00:00 2024-02-17T11:45:39.712 INFO:teuthology.orchestra.run.smithi125.stdout:(158/176): elfutils-debuginfod-client-0.190-2.e 841 kB/s | 35 kB 00:00 2024-02-17T11:45:39.730 INFO:teuthology.orchestra.run.smithi189.stdout:(35/176): ceph-volume-19.0.0-1515.g6822c7b5.el9 3.9 MB/s | 264 kB 00:00 2024-02-17T11:45:39.737 INFO:teuthology.orchestra.run.smithi125.stdout:(159/176): elfutils-libelf-0.190-2.el9.x86_64.r 4.4 MB/s | 191 kB 00:00 2024-02-17T11:45:39.740 INFO:teuthology.orchestra.run.smithi094.stdout:(127/176): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-02-17T11:45:39.766 INFO:teuthology.orchestra.run.smithi094.stdout:(128/176): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-02-17T11:45:39.788 INFO:teuthology.orchestra.run.smithi125.stdout:(160/176): elfutils-libs-0.190-2.el9.x86_64.rpm 3.3 MB/s | 254 kB 00:00 2024-02-17T11:45:39.791 INFO:teuthology.orchestra.run.smithi094.stdout:(129/176): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-02-17T11:45:39.816 INFO:teuthology.orchestra.run.smithi094.stdout:(130/176): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-02-17T11:45:39.850 INFO:teuthology.orchestra.run.smithi094.stdout:(131/176): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-02-17T11:45:39.863 INFO:teuthology.orchestra.run.smithi189.stdout:(36/176): cephadm-19.0.0-1515.g6822c7b5.el9.noa 5.5 MB/s | 748 kB 00:00 2024-02-17T11:45:39.881 INFO:teuthology.orchestra.run.smithi125.stdout:(161/176): iproute-6.7.0-2.el9.x86_64.rpm 5.5 MB/s | 802 kB 00:00 2024-02-17T11:45:39.883 INFO:teuthology.orchestra.run.smithi094.stdout:(132/176): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-02-17T11:45:39.909 INFO:teuthology.orchestra.run.smithi094.stdout:(133/176): python3-logutils-0.3.5-21.el9.noarch 504 kB/s | 46 kB 00:00 2024-02-17T11:45:39.914 INFO:teuthology.orchestra.run.smithi189.stdout:(37/176): cephfs-top-19.0.0-1515.g6822c7b5.el9. 365 kB/s | 18 kB 00:00 2024-02-17T11:45:39.934 INFO:teuthology.orchestra.run.smithi094.stdout:(134/176): python3-more-itertools-8.12.0-2.el9. 940 kB/s | 79 kB 00:00 2024-02-17T11:45:39.946 INFO:teuthology.orchestra.run.smithi125.stdout:(162/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 2.7 MB/s | 433 kB 00:00 2024-02-17T11:45:39.959 INFO:teuthology.orchestra.run.smithi094.stdout:(135/176): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-02-17T11:45:39.972 INFO:teuthology.orchestra.run.smithi125.stdout:(163/176): libacl-2.3.1-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-02-17T11:45:39.984 INFO:teuthology.orchestra.run.smithi094.stdout:(136/176): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-02-17T11:45:39.997 INFO:teuthology.orchestra.run.smithi125.stdout:(164/176): libblkid-2.37.4-18.el9.x86_64.rpm 2.1 MB/s | 107 kB 00:00 2024-02-17T11:45:40.009 INFO:teuthology.orchestra.run.smithi094.stdout:(137/176): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-02-17T11:45:40.022 INFO:teuthology.orchestra.run.smithi125.stdout:(165/176): libfdisk-2.37.4-18.el9.x86_64.rpm 3.0 MB/s | 154 kB 00:00 2024-02-17T11:45:40.035 INFO:teuthology.orchestra.run.smithi094.stdout:(138/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-02-17T11:45:40.047 INFO:teuthology.orchestra.run.smithi125.stdout:(166/176): libmount-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 135 kB 00:00 2024-02-17T11:45:40.048 INFO:teuthology.orchestra.run.smithi189.stdout:(38/176): automake-1.16.2-8.el9.noarch.rpm 4.9 MB/s | 662 kB 00:00 2024-02-17T11:45:40.052 INFO:teuthology.orchestra.run.smithi094.stdout:(139/176): python3-pyperclip-1.8.2-2.el9.noarch 421 kB/s | 28 kB 00:00 2024-02-17T11:45:40.073 INFO:teuthology.orchestra.run.smithi125.stdout:(167/176): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 63 kB 00:00 2024-02-17T11:45:40.077 INFO:teuthology.orchestra.run.smithi094.stdout:(140/176): python3-repoze-lru-0.7-16.el9.noarch 456 kB/s | 31 kB 00:00 2024-02-17T11:45:40.098 INFO:teuthology.orchestra.run.smithi125.stdout:(168/176): libuuid-2.37.4-18.el9.x86_64.rpm 550 kB/s | 28 kB 00:00 2024-02-17T11:45:40.102 INFO:teuthology.orchestra.run.smithi094.stdout:(141/176): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-02-17T11:45:40.106 INFO:teuthology.orchestra.run.smithi189.stdout:(39/176): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 920 kB 00:00 2024-02-17T11:45:40.119 INFO:teuthology.orchestra.run.smithi094.stdout:(142/176): python3-rsa-4.9-2.el9.noarch.rpm 880 kB/s | 59 kB 00:00 2024-02-17T11:45:40.137 INFO:teuthology.orchestra.run.smithi094.stdout:(143/176): python3-tempora-5.0.0-2.el9.noarch.r 608 kB/s | 36 kB 00:00 2024-02-17T11:45:40.148 INFO:teuthology.orchestra.run.smithi189.stdout:(40/176): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2024-02-17T11:45:40.157 INFO:teuthology.orchestra.run.smithi125.stdout:(169/176): openssl-3.0.7-25.el9.x86_64.rpm 14 MB/s | 1.2 MB 00:00 2024-02-17T11:45:40.161 INFO:teuthology.orchestra.run.smithi094.stdout:(144/176): python3-typing-extensions-4.4.0-2.el 868 kB/s | 51 kB 00:00 2024-02-17T11:45:40.187 INFO:teuthology.orchestra.run.smithi094.stdout:(145/176): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-02-17T11:45:40.212 INFO:teuthology.orchestra.run.smithi094.stdout:(146/176): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-02-17T11:45:40.223 INFO:teuthology.orchestra.run.smithi125.stdout:(170/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 17 MB/s | 2.1 MB 00:00 2024-02-17T11:45:40.240 INFO:teuthology.orchestra.run.smithi189.stdout:(41/176): ceph-mgr-diskprediction-local-19.0.0- 6.5 MB/s | 7.4 MB 00:01 2024-02-17T11:45:40.265 INFO:teuthology.orchestra.run.smithi189.stdout:(42/176): elfutils-libelf-devel-0.190-2.el9.x86 174 kB/s | 20 kB 00:00 2024-02-17T11:45:40.269 INFO:teuthology.orchestra.run.smithi094.stdout:(147/176): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-02-17T11:45:40.291 INFO:teuthology.orchestra.run.smithi125.stdout:(171/176): util-linux-2.37.4-18.el9.x86_64.rpm 17 MB/s | 2.2 MB 00:00 2024-02-17T11:45:40.291 INFO:teuthology.orchestra.run.smithi189.stdout:(43/176): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-02-17T11:45:40.296 INFO:teuthology.orchestra.run.smithi094.stdout:(148/176): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-02-17T11:45:40.316 INFO:teuthology.orchestra.run.smithi125.stdout:(172/176): util-linux-core-2.37.4-18.el9.x86_64 4.8 MB/s | 438 kB 00:00 2024-02-17T11:45:40.316 INFO:teuthology.orchestra.run.smithi189.stdout:(44/176): flexiblas-3.0.4-8.el9.x86_64.rpm 636 kB/s | 32 kB 00:00 2024-02-17T11:45:40.321 INFO:teuthology.orchestra.run.smithi094.stdout:(149/176): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-02-17T11:45:40.341 INFO:teuthology.orchestra.run.smithi125.stdout:(173/176): util-linux-user-2.37.4-18.el9.x86_64 607 kB/s | 30 kB 00:00 2024-02-17T11:45:40.371 INFO:teuthology.orchestra.run.smithi094.stdout:(150/176): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-02-17T11:45:40.383 INFO:teuthology.orchestra.run.smithi125.stdout:(174/176): xfsdump-3.1.12-4.el9.x86_64.rpm 4.7 MB/s | 321 kB 00:00 2024-02-17T11:45:40.413 INFO:teuthology.orchestra.run.smithi094.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2024-02-17T11:45:40.433 INFO:teuthology.orchestra.run.smithi189.stdout:(45/176): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-02-17T11:45:40.433 INFO:teuthology.orchestra.run.smithi125.stdout:(175/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-02-17T11:45:40.458 INFO:teuthology.orchestra.run.smithi189.stdout:(46/176): flexiblas-openblas-openmp-3.0.4-8.el9 121 kB/s | 17 kB 00:00 2024-02-17T11:45:40.483 INFO:teuthology.orchestra.run.smithi189.stdout:(47/176): libaio-devel-0.3.111-13.el9.x86_64.rp 200 kB/s | 10 kB 00:00 2024-02-17T11:45:40.509 INFO:teuthology.orchestra.run.smithi189.stdout:(48/176): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-02-17T11:45:40.522 INFO:teuthology.orchestra.run.smithi094.stdout:(152/176): NetworkManager-1.45.91-1.el9.x86_64. 21 MB/s | 2.3 MB 00:00 2024-02-17T11:45:40.534 INFO:teuthology.orchestra.run.smithi189.stdout:(49/176): librabbitmq-0.11.0-7.el9.x86_64.rpm 882 kB/s | 44 kB 00:00 2024-02-17T11:45:40.559 INFO:teuthology.orchestra.run.smithi189.stdout:(50/176): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-02-17T11:45:40.593 INFO:teuthology.orchestra.run.smithi189.stdout:(51/176): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2024-02-17T11:45:40.619 INFO:teuthology.orchestra.run.smithi189.stdout:(52/176): libuuid-devel-2.37.4-18.el9.x86_64.rp 421 kB/s | 24 kB 00:00 2024-02-17T11:45:40.622 INFO:teuthology.orchestra.run.smithi094.stdout:(153/176): librados2-19.0.0-1515.g6822c7b5.el9. 10 MB/s | 3.4 MB 00:00 2024-02-17T11:45:40.644 INFO:teuthology.orchestra.run.smithi189.stdout:(53/176): libzstd-devel-1.5.1-2.el9.x86_64.rpm 922 kB/s | 46 kB 00:00 2024-02-17T11:45:40.664 INFO:teuthology.orchestra.run.smithi094.stdout:(154/176): NetworkManager-libnm-1.45.91-1.el9.x 13 MB/s | 1.8 MB 00:00 2024-02-17T11:45:40.669 INFO:teuthology.orchestra.run.smithi189.stdout:(54/176): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-02-17T11:45:40.681 INFO:teuthology.orchestra.run.smithi094.stdout:(155/176): NetworkManager-team-1.45.91-1.el9.x8 719 kB/s | 41 kB 00:00 2024-02-17T11:45:40.694 INFO:teuthology.orchestra.run.smithi189.stdout:(55/176): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-02-17T11:45:40.706 INFO:teuthology.orchestra.run.smithi094.stdout:(156/176): NetworkManager-tui-1.45.91-1.el9.x86 5.8 MB/s | 246 kB 00:00 2024-02-17T11:45:40.720 INFO:teuthology.orchestra.run.smithi189.stdout:(56/176): openblas-0.3.21-2.el9.x86_64.rpm 686 kB/s | 34 kB 00:00 2024-02-17T11:45:40.723 INFO:teuthology.orchestra.run.smithi094.stdout:(157/176): acl-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 69 kB 00:00 2024-02-17T11:45:40.757 INFO:teuthology.orchestra.run.smithi094.stdout:(158/176): elfutils-debuginfod-client-0.190-2.e 708 kB/s | 35 kB 00:00 2024-02-17T11:45:40.782 INFO:teuthology.orchestra.run.smithi094.stdout:(159/176): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 191 kB 00:00 2024-02-17T11:45:40.832 INFO:teuthology.orchestra.run.smithi094.stdout:(160/176): librbd1-19.0.0-1515.g6822c7b5.el9.x8 6.1 MB/s | 3.1 MB 00:00 2024-02-17T11:45:40.853 INFO:teuthology.orchestra.run.smithi189.stdout:(57/176): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-02-17T11:45:40.858 INFO:teuthology.orchestra.run.smithi094.stdout:(161/176): elfutils-libs-0.190-2.el9.x86_64.rpm 2.5 MB/s | 254 kB 00:00 2024-02-17T11:45:40.891 INFO:teuthology.orchestra.run.smithi094.stdout:(162/176): iproute-6.7.0-2.el9.x86_64.rpm 7.2 MB/s | 802 kB 00:00 2024-02-17T11:45:40.912 INFO:teuthology.orchestra.run.smithi189.stdout:(58/176): openssl-devel-3.0.7-25.el9.x86_64.rpm 16 MB/s | 3.0 MB 00:00 2024-02-17T11:45:40.916 INFO:teuthology.orchestra.run.smithi094.stdout:(163/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 5.2 MB/s | 433 kB 00:00 2024-02-17T11:45:40.942 INFO:teuthology.orchestra.run.smithi094.stdout:(164/176): libacl-2.3.1-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-02-17T11:45:40.945 INFO:teuthology.orchestra.run.smithi189.stdout:(59/176): postgresql-13.11-1.el9.x86_64.rpm 17 MB/s | 1.5 MB 00:00 2024-02-17T11:45:40.967 INFO:teuthology.orchestra.run.smithi094.stdout:(165/176): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 107 kB 00:00 2024-02-17T11:45:40.987 INFO:teuthology.orchestra.run.smithi189.stdout:(60/176): postgresql-contrib-13.11-1.el9.x86_64 11 MB/s | 811 kB 00:00 2024-02-17T11:45:40.992 INFO:teuthology.orchestra.run.smithi094.stdout:(166/176): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 154 kB 00:00 2024-02-17T11:45:41.009 INFO:teuthology.orchestra.run.smithi094.stdout:(167/176): libmount-2.37.4-18.el9.x86_64.rpm 3.2 MB/s | 135 kB 00:00 2024-02-17T11:45:41.013 INFO:teuthology.orchestra.run.smithi189.stdout:(61/176): postgresql-private-libs-13.11-1.el9.x 2.0 MB/s | 134 kB 00:00 2024-02-17T11:45:41.034 INFO:teuthology.orchestra.run.smithi094.stdout:(168/176): libsmartcols-2.37.4-18.el9.x86_64.rp 942 kB/s | 63 kB 00:00 2024-02-17T11:45:41.059 INFO:teuthology.orchestra.run.smithi094.stdout:(169/176): libuuid-2.37.4-18.el9.x86_64.rpm 412 kB/s | 28 kB 00:00 2024-02-17T11:45:41.151 INFO:teuthology.orchestra.run.smithi094.stdout:(170/176): openssl-3.0.7-25.el9.x86_64.rpm 9.9 MB/s | 1.2 MB 00:00 2024-02-17T11:45:41.188 INFO:teuthology.orchestra.run.smithi189.stdout:(62/176): postgresql-server-13.11-1.el9.x86_64. 28 MB/s | 5.7 MB 00:00 2024-02-17T11:45:41.193 INFO:teuthology.orchestra.run.smithi094.stdout:(171/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 14 MB/s | 2.1 MB 00:00 2024-02-17T11:45:41.221 INFO:teuthology.orchestra.run.smithi189.stdout:(63/176): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-02-17T11:45:41.247 INFO:teuthology.orchestra.run.smithi189.stdout:(64/176): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 78 kB 00:00 2024-02-17T11:45:41.252 INFO:teuthology.orchestra.run.smithi094.stdout:(172/176): util-linux-2.37.4-18.el9.x86_64.rpm 11 MB/s | 2.2 MB 00:00 2024-02-17T11:45:41.277 INFO:teuthology.orchestra.run.smithi094.stdout:(173/176): util-linux-core-2.37.4-18.el9.x86_64 3.5 MB/s | 438 kB 00:00 2024-02-17T11:45:41.302 INFO:teuthology.orchestra.run.smithi094.stdout:(174/176): util-linux-user-2.37.4-18.el9.x86_64 281 kB/s | 30 kB 00:00 2024-02-17T11:45:41.328 INFO:teuthology.orchestra.run.smithi094.stdout:(175/176): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-02-17T11:45:41.395 INFO:teuthology.orchestra.run.smithi094.stdout:(176/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-02-17T11:45:41.400 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-02-17T11:45:41.400 INFO:teuthology.orchestra.run.smithi094.stdout:Total 16 MB/s | 223 MB 00:13 2024-02-17T11:45:41.414 INFO:teuthology.orchestra.run.smithi189.stdout:(65/176): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 5.8 MB 00:00 2024-02-17T11:45:41.439 INFO:teuthology.orchestra.run.smithi189.stdout:(66/176): python3-jinja2-2.11.3-5.el9.noarch.rp 1.2 MB/s | 227 kB 00:00 2024-02-17T11:45:41.447 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction check 2024-02-17T11:45:41.456 INFO:teuthology.orchestra.run.smithi189.stdout:(67/176): python3-jsonpatch-1.21-16.el9.noarch. 590 kB/s | 24 kB 00:00 2024-02-17T11:45:41.473 INFO:teuthology.orchestra.run.smithi189.stdout:(68/176): python3-jsonpointer-2.0-4.el9.noarch. 537 kB/s | 18 kB 00:00 2024-02-17T11:45:41.498 INFO:teuthology.orchestra.run.smithi189.stdout:(69/176): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-02-17T11:45:41.524 INFO:teuthology.orchestra.run.smithi189.stdout:(70/176): python3-markupsafe-1.1.1-12.el9.x86_6 631 kB/s | 32 kB 00:00 2024-02-17T11:45:41.666 INFO:teuthology.orchestra.run.smithi189.stdout:(71/176): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2024-02-17T11:45:41.691 INFO:teuthology.orchestra.run.smithi189.stdout:(72/176): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-02-17T11:45:41.716 INFO:teuthology.orchestra.run.smithi189.stdout:(73/176): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-02-17T11:45:41.741 INFO:teuthology.orchestra.run.smithi189.stdout:(74/176): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-02-17T11:45:41.748 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction check succeeded. 2024-02-17T11:45:41.748 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction test 2024-02-17T11:45:41.766 INFO:teuthology.orchestra.run.smithi189.stdout:(75/176): python3-prettytable-0.7.2-27.el9.noar 820 kB/s | 41 kB 00:00 2024-02-17T11:45:41.792 INFO:teuthology.orchestra.run.smithi189.stdout:(76/176): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-02-17T11:45:41.817 INFO:teuthology.orchestra.run.smithi189.stdout:(77/176): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-02-17T11:45:41.843 INFO:teuthology.orchestra.run.smithi189.stdout:(78/176): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-02-17T11:45:41.868 INFO:teuthology.orchestra.run.smithi189.stdout:(79/176): python3-pytz-2021.1-5.el9.noarch.rpm 943 kB/s | 47 kB 00:00 2024-02-17T11:45:41.902 INFO:teuthology.orchestra.run.smithi189.stdout:(80/176): python3-requests-oauthlib-1.3.0-12.el 730 kB/s | 43 kB 00:00 2024-02-17T11:45:42.269 INFO:teuthology.orchestra.run.smithi189.stdout:(81/176): python3-toml-0.10.2-6.el9.noarch.rpm 105 kB/s | 39 kB 00:00 2024-02-17T11:45:42.478 INFO:teuthology.orchestra.run.smithi189.stdout:(82/176): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-02-17T11:45:42.502 INFO:teuthology.orchestra.run.smithi189.stdout:(83/176): python3-wcwidth-0.2.5-8.el9.noarch.rp 177 kB/s | 41 kB 00:00 2024-02-17T11:45:42.544 INFO:teuthology.orchestra.run.smithi189.stdout:(84/176): qatlib-23.11.0-1.el9.x86_64.rpm 3.2 MB/s | 215 kB 00:00 2024-02-17T11:45:42.569 INFO:teuthology.orchestra.run.smithi189.stdout:(85/176): qatlib-service-23.11.0-1.el9.x86_64.r 507 kB/s | 34 kB 00:00 2024-02-17T11:45:42.595 INFO:teuthology.orchestra.run.smithi189.stdout:(86/176): qatzip-libs-1.1.2-2.el9.x86_64.rpm 864 kB/s | 43 kB 00:00 2024-02-17T11:45:42.705 INFO:teuthology.orchestra.run.smithi189.stdout:(87/176): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 300 kB 00:00 2024-02-17T11:45:42.854 INFO:teuthology.orchestra.run.smithi189.stdout:(88/176): uuid-1.6.2-55.el9.x86_64.rpm 218 kB/s | 56 kB 00:00 2024-02-17T11:45:42.880 INFO:teuthology.orchestra.run.smithi189.stdout:(89/176): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 498 kB/s | 87 kB 00:00 2024-02-17T11:45:42.905 INFO:teuthology.orchestra.run.smithi189.stdout:(90/176): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-02-17T11:45:42.930 INFO:teuthology.orchestra.run.smithi189.stdout:(91/176): zlib-devel-1.2.11-40.el9.x86_64.rpm 879 kB/s | 44 kB 00:00 2024-02-17T11:45:42.956 INFO:teuthology.orchestra.run.smithi189.stdout:(92/176): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-02-17T11:45:42.981 INFO:teuthology.orchestra.run.smithi189.stdout:(93/176): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-02-17T11:45:43.407 INFO:teuthology.orchestra.run.smithi189.stdout:(94/176): ceph-test-19.0.0-1515.g6822c7b5.el9.x 7.6 MB/s | 47 MB 00:06 2024-02-17T11:45:43.440 INFO:teuthology.orchestra.run.smithi189.stdout:(95/176): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 802 kB 00:00 2024-02-17T11:45:43.465 INFO:teuthology.orchestra.run.smithi189.stdout:(96/176): libquadmath-11.4.1-3.el9.x86_64.rpm 402 kB/s | 194 kB 00:00 2024-02-17T11:45:43.490 INFO:teuthology.orchestra.run.smithi189.stdout:(97/176): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.6 MB/s | 217 kB 00:00 2024-02-17T11:45:43.574 INFO:teuthology.orchestra.run.smithi189.stdout:(98/176): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.2 MB/s | 241 kB 00:00 2024-02-17T11:45:43.711 INFO:teuthology.orchestra.run.smithi094.stdout:Transaction test succeeded. 2024-02-17T11:45:43.712 INFO:teuthology.orchestra.run.smithi094.stdout:Running transaction 2024-02-17T11:45:43.741 INFO:teuthology.orchestra.run.smithi189.stdout:(99/176): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.1 MB 00:00 2024-02-17T11:45:43.766 INFO:teuthology.orchestra.run.smithi189.stdout:(100/176): python3-ply-3.11-14.el9.noarch.rpm 373 kB/s | 103 kB 00:00 2024-02-17T11:45:43.791 INFO:teuthology.orchestra.run.smithi189.stdout:(101/176): python3-pycparser-2.20-6.el9.noarch. 572 kB/s | 124 kB 00:00 2024-02-17T11:45:43.808 INFO:teuthology.orchestra.run.smithi189.stdout:(102/176): python3-requests-2.25.1-8.el9.noarch 1.7 MB/s | 113 kB 00:00 2024-02-17T11:45:43.842 INFO:teuthology.orchestra.run.smithi189.stdout:(103/176): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-02-17T11:45:43.967 INFO:teuthology.orchestra.run.smithi189.stdout:(104/176): python3-devel-3.9.17-2.el9.x86_64.rp 1.4 MB/s | 245 kB 00:00 2024-02-17T11:45:44.076 INFO:teuthology.orchestra.run.smithi189.stdout:(105/176): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-02-17T11:45:44.176 INFO:teuthology.orchestra.run.smithi189.stdout:(106/176): abseil-cpp-20211102.0-3.el9.x86_64.r 2.6 MB/s | 548 kB 00:00 2024-02-17T11:45:44.218 INFO:teuthology.orchestra.run.smithi189.stdout:(107/176): grpc-data-1.46.7-10.el9.noarch.rpm 466 kB/s | 19 kB 00:00 2024-02-17T11:45:44.285 INFO:teuthology.orchestra.run.smithi189.stdout:(108/176): gperftools-libs-2.9.1-2.el9.x86_64.r 1.4 MB/s | 309 kB 00:00 2024-02-17T11:45:44.343 INFO:teuthology.orchestra.run.smithi189.stdout:(109/176): libarrow-doc-9.0.0-10.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-02-17T11:45:44.402 INFO:teuthology.orchestra.run.smithi189.stdout:(110/176): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2024-02-17T11:45:44.469 INFO:teuthology.orchestra.run.smithi189.stdout:(111/176): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-02-17T11:45:44.528 INFO:teuthology.orchestra.run.smithi189.stdout:(112/176): libarrow-9.0.0-10.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-02-17T11:45:44.553 INFO:teuthology.orchestra.run.smithi189.stdout:(113/176): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-02-17T11:45:44.586 INFO:teuthology.orchestra.run.smithi189.stdout:(114/176): parquet-libs-9.0.0-10.el9.x86_64.rpm 14 MB/s | 838 kB 00:00 2024-02-17T11:45:44.654 INFO:teuthology.orchestra.run.smithi189.stdout:(115/176): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-02-17T11:45:44.679 INFO:teuthology.orchestra.run.smithi189.stdout:(116/176): python3-asyncssh-2.13.2-4.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-02-17T11:45:44.729 INFO:teuthology.orchestra.run.smithi189.stdout:(117/176): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-02-17T11:45:44.755 INFO:teuthology.orchestra.run.smithi189.stdout:(118/176): python3-cachetools-4.2.4-1.el9.noarc 319 kB/s | 32 kB 00:00 2024-02-17T11:45:44.780 INFO:teuthology.orchestra.run.smithi189.stdout:(119/176): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-02-17T11:45:44.838 INFO:teuthology.orchestra.run.smithi189.stdout:(120/176): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-02-17T11:45:44.864 INFO:teuthology.orchestra.run.smithi189.stdout:(121/176): python3-bcrypt-3.2.2-1.el9.x86_64.rp 157 kB/s | 43 kB 00:00 2024-02-17T11:45:44.889 INFO:teuthology.orchestra.run.smithi189.stdout:(122/176): python3-colorama-0.4.4-12.el9.noarch 703 kB/s | 35 kB 00:00 2024-02-17T11:45:44.914 INFO:teuthology.orchestra.run.smithi189.stdout:(123/176): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-02-17T11:45:44.940 INFO:teuthology.orchestra.run.smithi189.stdout:(124/176): python3-google-auth-2.27.0-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-02-17T11:45:44.998 INFO:teuthology.orchestra.run.smithi189.stdout:(125/176): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-02-17T11:45:45.198 INFO:teuthology.orchestra.run.smithi189.stdout:(126/176): python3-grpcio-tools-1.46.7-10.el9.x 509 kB/s | 144 kB 00:00 2024-02-17T11:45:45.232 INFO:teuthology.orchestra.run.smithi189.stdout:(127/176): python3-jaraco-collections-3.0.0-8.e 695 kB/s | 23 kB 00:00 2024-02-17T11:45:45.241 INFO:teuthology.orchestra.run.smithi125.stdout:(176/176): python3-grpcio-1.46.7-10.el9.x86_64. 257 kB/s | 2.0 MB 00:08 2024-02-17T11:45:45.245 INFO:teuthology.orchestra.run.smithi125.stdout:-------------------------------------------------------------------------------- 2024-02-17T11:45:45.245 INFO:teuthology.orchestra.run.smithi125.stdout:Total 11 MB/s | 223 MB 00:20 2024-02-17T11:45:45.265 INFO:teuthology.orchestra.run.smithi189.stdout:(128/176): python3-jaraco-functools-3.5.0-2.el9 586 kB/s | 19 kB 00:00 2024-02-17T11:45:45.282 INFO:teuthology.orchestra.run.smithi189.stdout:(129/176): python3-jaraco-classes-3.2.1-5.el9.n 62 kB/s | 18 kB 00:00 2024-02-17T11:45:45.298 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction check 2024-02-17T11:45:45.299 INFO:teuthology.orchestra.run.smithi189.stdout:(130/176): python3-jaraco-text-3.2.0-6.el9.noar 586 kB/s | 20 kB 00:00 2024-02-17T11:45:45.341 INFO:teuthology.orchestra.run.smithi189.stdout:(131/176): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-02-17T11:45:45.398 INFO:teuthology.orchestra.run.smithi189.stdout:(132/176): python3-kubernetes-26.1.0-2.el9.noar 8.9 MB/s | 1.0 MB 00:00 2024-02-17T11:45:45.425 INFO:teuthology.orchestra.run.smithi189.stdout:(133/176): python3-logutils-0.3.5-21.el9.noarch 369 kB/s | 46 kB 00:00 2024-02-17T11:45:45.451 INFO:teuthology.orchestra.run.smithi189.stdout:(134/176): python3-more-itertools-8.12.0-2.el9. 723 kB/s | 79 kB 00:00 2024-02-17T11:45:45.476 INFO:teuthology.orchestra.run.smithi189.stdout:(135/176): python3-natsort-7.1.1-5.el9.noarch.r 746 kB/s | 58 kB 00:00 2024-02-17T11:45:45.501 INFO:teuthology.orchestra.run.smithi189.stdout:(136/176): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-02-17T11:45:45.526 INFO:teuthology.orchestra.run.smithi189.stdout:(137/176): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-02-17T11:45:45.571 INFO:teuthology.orchestra.run.smithi189.stdout:(138/176): python3-pyperclip-1.8.2-2.el9.noarch 407 kB/s | 28 kB 00:00 2024-02-17T11:45:45.574 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction check succeeded. 2024-02-17T11:45:45.574 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction test 2024-02-17T11:45:45.638 INFO:teuthology.orchestra.run.smithi189.stdout:(139/176): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-02-17T11:45:45.680 INFO:teuthology.orchestra.run.smithi189.stdout:(140/176): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-02-17T11:45:45.788 INFO:teuthology.orchestra.run.smithi189.stdout:(141/176): python3-pyOpenSSL-21.0.0-1.el9.noarc 288 kB/s | 90 kB 00:00 2024-02-17T11:45:45.814 INFO:teuthology.orchestra.run.smithi189.stdout:(142/176): python3-repoze-lru-0.7-16.el9.noarch 107 kB/s | 31 kB 00:00 2024-02-17T11:45:45.864 INFO:teuthology.orchestra.run.smithi189.stdout:(143/176): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-02-17T11:45:46.131 INFO:teuthology.orchestra.run.smithi189.stdout:(144/176): python3-webob-1.8.7-6.el9.noarch.rpm 724 kB/s | 230 kB 00:00 2024-02-17T11:45:46.215 INFO:teuthology.orchestra.run.smithi189.stdout:(145/176): python3-tempora-5.0.0-2.el9.noarch.r 67 kB/s | 36 kB 00:00 2024-02-17T11:45:46.240 INFO:teuthology.orchestra.run.smithi189.stdout:(146/176): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-02-17T11:45:46.307 INFO:teuthology.orchestra.run.smithi189.stdout:(147/176): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-02-17T11:45:46.499 INFO:teuthology.orchestra.run.smithi189.stdout:(148/176): python3-zc-lockfile-2.0-10.el9.noarc 70 kB/s | 20 kB 00:00 2024-02-17T11:45:46.591 INFO:teuthology.orchestra.run.smithi189.stdout:(149/176): thrift-0.15.0-2.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-02-17T11:45:46.883 INFO:teuthology.orchestra.run.smithi189.stdout:(150/176): python3-websocket-client-1.2.3-2.el9 88 kB/s | 90 kB 00:01 2024-02-17T11:45:46.958 INFO:teuthology.orchestra.run.smithi189.stdout:(151/176): libacl-devel-2.3.1-4.el9.x86_64.rpm 905 kB/s | 68 kB 00:00 2024-02-17T11:45:47.033 INFO:teuthology.orchestra.run.smithi189.stdout:(152/176): librados2-19.0.0-1515.g6822c7b5.el9. 6.3 MB/s | 3.4 MB 00:00 2024-02-17T11:45:47.092 INFO:teuthology.orchestra.run.smithi189.stdout:(153/176): librbd1-19.0.0-1515.g6822c7b5.el9.x8 6.2 MB/s | 3.1 MB 00:00 2024-02-17T11:45:47.284 INFO:teuthology.orchestra.run.smithi189.stdout:(154/176): NetworkManager-1.45.91-1.el9.x86_64. 6.9 MB/s | 2.3 MB 00:00 2024-02-17T11:45:47.474 INFO:teuthology.orchestra.run.smithi125.stdout:Transaction test succeeded. 2024-02-17T11:45:47.474 INFO:teuthology.orchestra.run.smithi125.stdout:Running transaction 2024-02-17T11:45:47.492 INFO:teuthology.orchestra.run.smithi189.stdout:(155/176): NetworkManager-libnm-1.45.91-1.el9.x 3.9 MB/s | 1.8 MB 00:00 2024-02-17T11:45:47.517 INFO:teuthology.orchestra.run.smithi189.stdout:(156/176): NetworkManager-team-1.45.91-1.el9.x8 97 kB/s | 41 kB 00:00 2024-02-17T11:45:47.542 INFO:teuthology.orchestra.run.smithi189.stdout:(157/176): NetworkManager-tui-1.45.91-1.el9.x86 955 kB/s | 246 kB 00:00 2024-02-17T11:45:47.585 INFO:teuthology.orchestra.run.smithi189.stdout:(158/176): elfutils-debuginfod-client-0.190-2.e 855 kB/s | 35 kB 00:00 2024-02-17T11:45:47.610 INFO:teuthology.orchestra.run.smithi189.stdout:(159/176): acl-2.3.1-4.el9.x86_64.rpm 1.0 MB/s | 69 kB 00:00 2024-02-17T11:45:47.643 INFO:teuthology.orchestra.run.smithi189.stdout:(160/176): elfutils-libelf-0.190-2.el9.x86_64.r 1.9 MB/s | 191 kB 00:00 2024-02-17T11:45:47.669 INFO:teuthology.orchestra.run.smithi189.stdout:(161/176): elfutils-libs-0.190-2.el9.x86_64.rpm 3.0 MB/s | 254 kB 00:00 2024-02-17T11:45:47.727 INFO:teuthology.orchestra.run.smithi189.stdout:(162/176): iproute-6.7.0-2.el9.x86_64.rpm 6.7 MB/s | 802 kB 00:00 2024-02-17T11:45:47.770 INFO:teuthology.orchestra.run.smithi189.stdout:(163/176): iproute-tc-6.7.0-2.el9.x86_64.rpm 3.4 MB/s | 433 kB 00:00 2024-02-17T11:45:47.786 INFO:teuthology.orchestra.run.smithi189.stdout:(164/176): libacl-2.3.1-4.el9.x86_64.rpm 188 kB/s | 22 kB 00:00 2024-02-17T11:45:47.803 INFO:teuthology.orchestra.run.smithi189.stdout:(165/176): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 107 kB 00:00 2024-02-17T11:45:47.828 INFO:teuthology.orchestra.run.smithi189.stdout:(166/176): libfdisk-2.37.4-18.el9.x86_64.rpm 2.6 MB/s | 154 kB 00:00 2024-02-17T11:45:47.870 INFO:teuthology.orchestra.run.smithi189.stdout:(167/176): libmount-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 135 kB 00:00 2024-02-17T11:45:47.887 INFO:teuthology.orchestra.run.smithi189.stdout:(168/176): libsmartcols-2.37.4-18.el9.x86_64.rp 753 kB/s | 63 kB 00:00 2024-02-17T11:45:47.904 INFO:teuthology.orchestra.run.smithi189.stdout:(169/176): libuuid-2.37.4-18.el9.x86_64.rpm 365 kB/s | 28 kB 00:00 2024-02-17T11:45:48.021 INFO:teuthology.orchestra.run.smithi189.stdout:(170/176): openssl-3.0.7-25.el9.x86_64.rpm 7.7 MB/s | 1.2 MB 00:00 2024-02-17T11:45:48.338 INFO:teuthology.orchestra.run.smithi189.stdout:(171/176): openssl-libs-3.0.7-25.el9.x86_64.rpm 5.0 MB/s | 2.1 MB 00:00 2024-02-17T11:45:48.380 INFO:teuthology.orchestra.run.smithi189.stdout:(172/176): util-linux-2.37.4-18.el9.x86_64.rpm 4.6 MB/s | 2.2 MB 00:00 2024-02-17T11:45:48.406 INFO:teuthology.orchestra.run.smithi189.stdout:(173/176): util-linux-core-2.37.4-18.el9.x86_64 1.1 MB/s | 438 kB 00:00 2024-02-17T11:45:48.422 INFO:teuthology.orchestra.run.smithi189.stdout:(174/176): util-linux-user-2.37.4-18.el9.x86_64 362 kB/s | 30 kB 00:00 2024-02-17T11:45:48.456 INFO:teuthology.orchestra.run.smithi189.stdout:(175/176): xfsdump-3.1.12-4.el9.x86_64.rpm 4.2 MB/s | 321 kB 00:00 2024-02-17T11:45:48.531 INFO:teuthology.orchestra.run.smithi189.stdout:(176/176): xfsprogs-6.3.0-1.el9.x86_64.rpm 9.8 MB/s | 1.1 MB 00:00 2024-02-17T11:45:48.535 INFO:teuthology.orchestra.run.smithi189.stdout:-------------------------------------------------------------------------------- 2024-02-17T11:45:48.535 INFO:teuthology.orchestra.run.smithi189.stdout:Total 12 MB/s | 223 MB 00:17 2024-02-17T11:45:48.583 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction check 2024-02-17T11:45:48.860 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction check succeeded. 2024-02-17T11:45:48.861 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction test 2024-02-17T11:45:50.530 INFO:teuthology.orchestra.run.smithi189.stdout:Transaction test succeeded. 2024-02-17T11:45:50.530 INFO:teuthology.orchestra.run.smithi189.stdout:Running transaction 2024-02-17T11:45:50.710 INFO:teuthology.orchestra.run.smithi094.stdout: Preparing : 1/1 2024-02-17T11:45:50.812 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-17T11:45:50.842 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-17T11:45:50.914 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-17T11:45:51.124 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-17T11:45:51.218 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-17T11:45:51.258 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-17T11:45:51.314 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-17T11:45:51.353 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-17T11:45:51.377 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-17T11:45:51.429 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-17T11:45:51.757 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-17T11:45:51.811 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T11:45:51.915 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T11:45:51.953 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T11:45:52.045 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T11:45:52.087 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 13/202 2024-02-17T11:45:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-17T11:45:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-17T11:45:52.332 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T11:45:52.443 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T11:45:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 17/202 2024-02-17T11:45:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 17/202 2024-02-17T11:45:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-17T11:45:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-17T11:45:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-17T11:45:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-17T11:45:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-17T11:45:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-17T11:45:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-17T11:45:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-17T11:45:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-17T11:45:53.704 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T11:45:53.757 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T11:45:53.959 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T11:45:54.065 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-17T11:45:54.150 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-17T11:45:54.178 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-17T11:45:54.248 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-17T11:45:54.301 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-17T11:45:54.332 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-17T11:45:54.376 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-17T11:45:54.415 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-17T11:45:54.472 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-17T11:45:54.506 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-17T11:45:54.549 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-17T11:45:54.559 INFO:teuthology.orchestra.run.smithi125.stdout: Preparing : 1/1 2024-02-17T11:45:54.582 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-17T11:45:54.610 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-17T11:45:54.679 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-17T11:45:54.694 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-17T11:45:54.715 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9. 39/202 2024-02-17T11:45:54.745 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-17T11:45:54.767 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el 40/202 2024-02-17T11:45:54.776 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-17T11:45:54.795 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_6 41/202 2024-02-17T11:45:54.801 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-17T11:45:54.854 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-17T11:45:54.864 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-17T11:45:54.893 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-17T11:45:54.919 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-17T11:45:54.946 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-17T11:45:54.980 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-17T11:45:54.992 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-17T11:45:54.998 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-17T11:45:55.039 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-17T11:45:55.126 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-17T11:45:55.179 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-17T11:45:55.356 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-17T11:45:55.415 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T11:45:55.561 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T11:45:55.591 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T11:45:55.674 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T11:45:55.700 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-17T11:45:55.703 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 13/202 2024-02-17T11:45:55.783 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-17T11:45:55.848 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-17T11:45:55.865 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-17T11:45:55.892 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-17T11:45:55.911 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-17T11:45:55.920 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-17T11:45:55.920 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T11:45:55.958 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-17T11:45:55.996 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_6 52/202 2024-02-17T11:45:56.006 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T11:45:56.028 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-17T11:45:56.029 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 17/202 2024-02-17T11:45:56.093 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 17/202 2024-02-17T11:45:56.126 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-17T11:45:56.183 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-17T11:45:56.196 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-17T11:45:56.206 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-17T11:45:56.221 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-17T11:45:56.388 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-17T11:45:56.419 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-17T11:45:56.451 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-17T11:45:56.473 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-17T11:45:56.547 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-17T11:45:56.637 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-17T11:45:56.706 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-17T11:45:56.782 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T11:45:56.832 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T11:45:56.865 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-17T11:45:56.949 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-17T11:45:56.994 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-17T11:45:57.053 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T11:45:57.136 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-17T11:45:57.161 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-17T11:45:57.187 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-17T11:45:57.242 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-17T11:45:57.258 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-17T11:45:57.303 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-17T11:45:57.315 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-17T11:45:57.346 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-17T11:45:57.371 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-17T11:45:57.382 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-17T11:45:57.407 INFO:teuthology.orchestra.run.smithi189.stdout: Preparing : 1/1 2024-02-17T11:45:57.425 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-17T11:45:57.430 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-17T11:45:57.457 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-17T11:45:57.465 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-17T11:45:57.492 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-17T11:45:57.508 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-17T11:45:57.518 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/202 2024-02-17T11:45:57.523 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-17T11:45:57.543 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/202 2024-02-17T11:45:57.550 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-17T11:45:57.587 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-17T11:45:57.599 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-17T11:45:57.622 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-17T11:45:57.623 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/202 2024-02-17T11:45:57.666 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-17T11:45:57.678 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 4/202 2024-02-17T11:45:57.683 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-17T11:45:57.699 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-17T11:45:57.716 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 5/202 2024-02-17T11:45:57.727 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-17T11:45:57.731 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-17T11:45:57.761 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-17T11:45:57.770 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/202 2024-02-17T11:45:57.803 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-17T11:45:57.804 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : thrift-0.15.0-2.el9.x86_64 7/202 2024-02-17T11:45:57.805 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-17T11:45:57.821 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 8/202 2024-02-17T11:45:57.840 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9. 39/202 2024-02-17T11:45:57.863 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 9/202 2024-02-17T11:45:57.876 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el 40/202 2024-02-17T11:45:57.898 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-17T11:45:57.904 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_6 41/202 2024-02-17T11:45:57.945 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-17T11:45:57.973 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-17T11:45:57.977 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-17T11:45:58.002 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-17T11:45:58.019 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-17T11:45:58.027 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-17T11:45:58.093 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-17T11:45:58.126 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-17T11:45:58.191 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 10/202 2024-02-17T11:45:58.227 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-17T11:45:58.247 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T11:45:58.254 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-17T11:45:58.280 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-17T11:45:58.361 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-17T11:45:58.368 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T11:45:58.402 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-17T11:45:58.406 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T11:45:58.451 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-17T11:45:58.490 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T11:45:58.516 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-17T11:45:58.519 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 13/202 2024-02-17T11:45:58.669 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 14/202 2024-02-17T11:45:58.716 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-17T11:45:58.716 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/202 2024-02-17T11:45:58.744 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T11:45:58.765 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-17T11:45:58.811 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-17T11:45:58.847 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T11:45:58.870 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 17/202 2024-02-17T11:45:58.891 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-17T11:45:58.893 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-17T11:45:58.939 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-17T11:45:58.942 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 17/202 2024-02-17T11:45:58.971 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-17T11:45:58.977 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-17T11:45:58.984 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/202 2024-02-17T11:45:59.004 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-17T11:45:59.007 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-17T11:45:59.011 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-17T11:45:59.020 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-17T11:45:59.022 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/202 2024-02-17T11:45:59.038 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-17T11:45:59.058 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-17T11:45:59.070 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-17T11:45:59.096 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_6 52/202 2024-02-17T11:45:59.098 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-17T11:45:59.127 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-17T11:45:59.132 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-17T11:45:59.171 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-17T11:45:59.197 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-17T11:45:59.203 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-17T11:45:59.219 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-17T11:45:59.229 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/202 2024-02-17T11:45:59.233 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-17T11:45:59.261 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/202 2024-02-17T11:45:59.278 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-17T11:45:59.300 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-17T11:45:59.335 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-17T11:45:59.365 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/202 2024-02-17T11:45:59.368 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-17T11:45:59.410 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-17T11:45:59.445 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-17T11:45:59.461 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-17T11:45:59.514 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-17T11:45:59.523 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 23/202 2024-02-17T11:45:59.533 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-17T11:45:59.595 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T11:45:59.624 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-17T11:45:59.641 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T11:45:59.667 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-17T11:45:59.671 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-17T11:45:59.699 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-17T11:45:59.738 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-17T11:45:59.746 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-17T11:45:59.782 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-17T11:45:59.844 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-17T11:45:59.848 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.45.91-1.el9.x86_64 24/202 2024-02-17T11:45:59.892 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-17T11:45:59.921 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-17T11:45:59.954 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/202 2024-02-17T11:45:59.971 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-17T11:46:00.025 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-17T11:46:00.040 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/202 2024-02-17T11:46:00.079 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/202 2024-02-17T11:46:00.090 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-17T11:46:00.122 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-17T11:46:00.147 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/202 2024-02-17T11:46:00.158 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-17T11:46:00.200 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/202 2024-02-17T11:46:00.204 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-17T11:46:00.232 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/202 2024-02-17T11:46:00.249 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-17T11:46:00.267 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/202 2024-02-17T11:46:00.292 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-17T11:46:00.307 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/202 2024-02-17T11:46:00.371 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/202 2024-02-17T11:46:00.407 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/202 2024-02-17T11:46:00.460 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/202 2024-02-17T11:46:00.502 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-17T11:46:00.516 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/202 2024-02-17T11:46:00.545 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/202 2024-02-17T11:46:00.613 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-17T11:46:00.614 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/202 2024-02-17T11:46:00.634 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-17T11:46:00.649 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9. 39/202 2024-02-17T11:46:00.657 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-17T11:46:00.674 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-17T11:46:00.690 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-17T11:46:00.704 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-17T11:46:00.704 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el 40/202 2024-02-17T11:46:00.730 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_6 41/202 2024-02-17T11:46:00.748 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-17T11:46:00.754 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-17T11:46:00.800 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/202 2024-02-17T11:46:00.836 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-17T11:46:00.845 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/202 2024-02-17T11:46:00.847 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-17T11:46:00.875 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-17T11:46:00.918 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/202 2024-02-17T11:46:00.921 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-17T11:46:00.960 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-17T11:46:00.987 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-17T11:46:01.011 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-17T11:46:01.044 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-17T11:46:01.054 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/202 2024-02-17T11:46:01.073 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-17T11:46:01.082 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-17T11:46:01.107 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/202 2024-02-17T11:46:01.155 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-17T11:46:01.202 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-17T11:46:01.311 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-17T11:46:01.353 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-17T11:46:01.400 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-17T11:46:01.455 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-17T11:46:01.638 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-17T11:46:01.638 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/202 2024-02-17T11:46:01.689 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-17T11:46:01.716 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/202 2024-02-17T11:46:01.788 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/202 2024-02-17T11:46:01.805 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-17T11:46:01.829 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-17T11:46:01.837 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/202 2024-02-17T11:46:01.845 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-17T11:46:01.875 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/202 2024-02-17T11:46:01.888 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-17T11:46:01.913 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_6 52/202 2024-02-17T11:46:01.918 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-17T11:46:01.920 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-17T11:46:01.952 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-17T11:46:01.953 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 53/202 2024-02-17T11:46:01.987 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-17T11:46:02.018 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-17T11:46:02.046 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-17T11:46:02.054 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 54/202 2024-02-17T11:46:02.080 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-17T11:46:02.104 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : iproute-6.7.0-2.el9.x86_64 55/202 2024-02-17T11:46:02.120 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-17T11:46:02.152 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-17T11:46:02.181 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-17T11:46:02.257 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-17T11:46:02.300 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-17T11:46:02.334 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-17T11:46:02.334 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 56/202 2024-02-17T11:46:02.383 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-17T11:46:02.418 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-17T11:46:02.503 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 57/202 2024-02-17T11:46:02.506 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-17T11:46:02.564 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 58/202 2024-02-17T11:46:02.588 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-17T11:46:02.608 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 59/202 2024-02-17T11:46:02.632 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-17T11:46:02.663 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-17T11:46:02.702 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-17T11:46:02.735 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 60/202 2024-02-17T11:46:02.778 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 61/202 2024-02-17T11:46:02.799 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-17T11:46:02.833 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 62/202 2024-02-17T11:46:02.848 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-17T11:46:02.904 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 63/202 2024-02-17T11:46:02.937 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 64/202 2024-02-17T11:46:02.973 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 65/202 2024-02-17T11:46:03.019 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 66/202 2024-02-17T11:46:03.056 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 67/202 2024-02-17T11:46:03.099 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 68/202 2024-02-17T11:46:03.134 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-17T11:46:03.208 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-17T11:46:03.234 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-17T11:46:03.361 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 69/202 2024-02-17T11:46:03.499 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 70/202 2024-02-17T11:46:03.540 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-17T11:46:03.555 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 71/202 2024-02-17T11:46:03.563 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-17T11:46:03.573 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 125/202 2024-02-17T11:46:03.585 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 72/202 2024-02-17T11:46:03.611 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-17T11:46:03.634 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 73/202 2024-02-17T11:46:03.645 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-17T11:46:03.663 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 125/202 2024-02-17T11:46:03.694 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-17T11:46:03.694 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 126/202 2024-02-17T11:46:03.729 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 74/202 2024-02-17T11:46:03.731 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-17T11:46:03.773 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-17T11:46:03.775 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 75/202 2024-02-17T11:46:03.815 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/202 2024-02-17T11:46:03.841 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 77/202 2024-02-17T11:46:03.846 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-17T11:46:03.866 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 78/202 2024-02-17T11:46:03.937 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/202 2024-02-17T11:46:04.027 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-17T11:46:04.053 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-17T11:46:04.057 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 80/202 2024-02-17T11:46:04.130 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-17T11:46:04.157 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/202 2024-02-17T11:46:04.181 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-17T11:46:04.206 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 82/202 2024-02-17T11:46:04.209 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 129/202 2024-02-17T11:46:04.255 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-ply-3.11-14.el9.noarch 83/202 2024-02-17T11:46:04.272 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 129/202 2024-02-17T11:46:04.294 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 130/202 2024-02-17T11:46:04.318 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 84/202 2024-02-17T11:46:04.501 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 85/202 2024-02-17T11:46:04.544 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 86/202 2024-02-17T11:46:04.671 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 87/202 2024-02-17T11:46:04.708 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 88/202 2024-02-17T11:46:04.742 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 89/202 2024-02-17T11:46:04.798 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-17T11:46:04.798 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 90/202 2024-02-17T11:46:04.808 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : jq-1.6-16.el9.x86_64 91/202 2024-02-17T11:46:04.839 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : fuse-2.9.9-15.el9.x86_64 92/202 2024-02-17T11:46:04.865 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-17T11:46:04.867 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 93/202 2024-02-17T11:46:04.901 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 94/202 2024-02-17T11:46:04.949 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : uuid-1.6.2-55.el9.x86_64 95/202 2024-02-17T11:46:04.981 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 96/202 2024-02-17T11:46:05.011 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 97/202 2024-02-17T11:46:05.087 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 98/202 2024-02-17T11:46:05.121 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 99/202 2024-02-17T11:46:05.154 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 100/202 2024-02-17T11:46:05.195 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 101/202 2024-02-17T11:46:05.230 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 102/202 2024-02-17T11:46:05.318 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 103/202 2024-02-17T11:46:05.400 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 104/202 2024-02-17T11:46:05.444 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 105/202 2024-02-17T11:46:05.476 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 106/202 2024-02-17T11:46:05.523 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 107/202 2024-02-17T11:46:05.619 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-packaging-20.9-5.el9.noarch 108/202 2024-02-17T11:46:05.668 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 109/202 2024-02-17T11:46:06.063 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-17T11:46:06.131 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-17T11:46:06.158 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-17T11:46:06.401 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/202 2024-02-17T11:46:06.449 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 111/202 2024-02-17T11:46:06.465 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-17T11:46:06.482 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T11:46:06.483 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 112/202 2024-02-17T11:46:06.496 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 125/202 2024-02-17T11:46:06.517 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T11:46:06.530 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 113/202 2024-02-17T11:46:06.569 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 125/202 2024-02-17T11:46:06.600 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 126/202 2024-02-17T11:46:06.613 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 114/202 2024-02-17T11:46:06.637 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-17T11:46:06.659 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 115/202 2024-02-17T11:46:06.830 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-0.3.21-2.el9.x86_64 116/202 2024-02-17T11:46:06.865 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 117/202 2024-02-17T11:46:06.942 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 118/202 2024-02-17T11:46:07.068 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T11:46:07.093 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-17T11:46:07.112 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 132/202 2024-02-17T11:46:07.124 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 129/202 2024-02-17T11:46:07.187 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 129/202 2024-02-17T11:46:07.207 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 132/202 2024-02-17T11:46:07.207 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-17T11:46:07.207 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-17T11:46:07.207 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:46:07.217 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 130/202 2024-02-17T11:46:07.238 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 133/202 2024-02-17T11:46:07.575 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 119/202 2024-02-17T11:46:07.685 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 120/202 2024-02-17T11:46:08.890 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 121/202 2024-02-17T11:46:08.967 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 122/202 2024-02-17T11:46:08.994 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 123/202 2024-02-17T11:46:09.304 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 124/202 2024-02-17T11:46:09.332 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 125/202 2024-02-17T11:46:09.334 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T11:46:09.363 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T11:46:09.423 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 125/202 2024-02-17T11:46:09.454 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 126/202 2024-02-17T11:46:09.490 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 127/202 2024-02-17T11:46:09.900 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T11:46:09.934 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 132/202 2024-02-17T11:46:09.947 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 128/202 2024-02-17T11:46:09.977 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 129/202 2024-02-17T11:46:09.998 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 132/202 2024-02-17T11:46:09.998 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-17T11:46:09.998 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-17T11:46:09.998 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:46:10.027 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 133/202 2024-02-17T11:46:10.048 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 129/202 2024-02-17T11:46:10.079 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 130/202 2024-02-17T11:46:12.177 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T11:46:12.217 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T11:46:12.757 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 131/202 2024-02-17T11:46:12.795 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 132/202 2024-02-17T11:46:12.859 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 132/202 2024-02-17T11:46:12.860 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-17T11:46:12.860 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-17T11:46:12.860 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:46:12.888 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 133/202 2024-02-17T11:46:19.312 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 133/202 2024-02-17T11:46:19.312 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /sys 2024-02-17T11:46:19.313 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /proc 2024-02-17T11:46:19.313 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /mnt 2024-02-17T11:46:19.313 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /var/tmp 2024-02-17T11:46:19.313 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /home 2024-02-17T11:46:19.313 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /root 2024-02-17T11:46:19.313 INFO:teuthology.orchestra.run.smithi094.stdout:skipping the directory /tmp 2024-02-17T11:46:19.313 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:46:19.480 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 134/202 2024-02-17T11:46:19.494 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 134/202 2024-02-17T11:46:19.494 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:19.495 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-17T11:46:19.495 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:46:20.975 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 135/202 2024-02-17T11:46:20.998 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 135/202 2024-02-17T11:46:20.998 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:20.999 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-17T11:46:20.999 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-17T11:46:20.999 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-17T11:46:20.999 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:46:21.216 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 136/202 2024-02-17T11:46:21.238 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 136/202 2024-02-17T11:46:21.238 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:21.238 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-17T11:46:21.238 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-17T11:46:21.239 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-17T11:46:21.239 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:46:21.685 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 137/202 2024-02-17T11:46:21.689 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 133/202 2024-02-17T11:46:21.689 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /sys 2024-02-17T11:46:21.690 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /proc 2024-02-17T11:46:21.690 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /mnt 2024-02-17T11:46:21.690 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /var/tmp 2024-02-17T11:46:21.690 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /home 2024-02-17T11:46:21.690 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /root 2024-02-17T11:46:21.690 INFO:teuthology.orchestra.run.smithi125.stdout:skipping the directory /tmp 2024-02-17T11:46:21.690 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:46:21.707 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 137/202 2024-02-17T11:46:21.707 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:21.707 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-17T11:46:21.707 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-17T11:46:21.707 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-17T11:46:21.707 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:46:21.795 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-17T11:46:21.848 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 134/202 2024-02-17T11:46:21.863 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 134/202 2024-02-17T11:46:21.864 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:21.864 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-17T11:46:21.864 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:46:22.221 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 139/202 2024-02-17T11:46:22.252 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 139/202 2024-02-17T11:46:22.283 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.e 140/202 2024-02-17T11:46:22.343 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5. 141/202 2024-02-17T11:46:22.367 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 142/202 2024-02-17T11:46:22.597 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 142/202 2024-02-17T11:46:22.626 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 143/202 2024-02-17T11:46:23.351 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 135/202 2024-02-17T11:46:23.376 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 135/202 2024-02-17T11:46:23.377 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:23.377 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-17T11:46:23.377 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-17T11:46:23.377 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-17T11:46:23.377 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:46:23.585 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 136/202 2024-02-17T11:46:23.592 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 143/202 2024-02-17T11:46:23.608 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 136/202 2024-02-17T11:46:23.608 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:23.608 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-17T11:46:23.608 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-17T11:46:23.608 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-17T11:46:23.609 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:46:23.618 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 144/202 2024-02-17T11:46:23.719 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 144/202 2024-02-17T11:46:23.888 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el 145/202 2024-02-17T11:46:23.922 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 146/202 2024-02-17T11:46:23.945 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 146/202 2024-02-17T11:46:23.945 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:23.945 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-17T11:46:23.945 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-17T11:46:23.945 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-17T11:46:23.945 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:46:23.998 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 147/202 2024-02-17T11:46:24.008 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 147/202 2024-02-17T11:46:24.047 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 137/202 2024-02-17T11:46:24.061 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 148/202 2024-02-17T11:46:24.067 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 137/202 2024-02-17T11:46:24.067 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:24.067 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-17T11:46:24.067 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-17T11:46:24.067 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-17T11:46:24.067 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:46:24.154 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-17T11:46:24.483 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 133/202 2024-02-17T11:46:24.483 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /sys 2024-02-17T11:46:24.483 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /proc 2024-02-17T11:46:24.483 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /mnt 2024-02-17T11:46:24.483 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /var/tmp 2024-02-17T11:46:24.483 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /home 2024-02-17T11:46:24.483 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /root 2024-02-17T11:46:24.483 INFO:teuthology.orchestra.run.smithi189.stdout:skipping the directory /tmp 2024-02-17T11:46:24.483 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:46:24.554 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 139/202 2024-02-17T11:46:24.585 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 139/202 2024-02-17T11:46:24.617 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.e 140/202 2024-02-17T11:46:24.641 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 134/202 2024-02-17T11:46:24.656 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 134/202 2024-02-17T11:46:24.656 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:24.656 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-17T11:46:24.656 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:46:24.677 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5. 141/202 2024-02-17T11:46:24.700 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 142/202 2024-02-17T11:46:24.929 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 142/202 2024-02-17T11:46:24.942 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-17T11:46:24.959 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 143/202 2024-02-17T11:46:24.975 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 150/202 2024-02-17T11:46:24.997 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 150/202 2024-02-17T11:46:24.998 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:24.998 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-17T11:46:24.998 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-17T11:46:24.998 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-17T11:46:24.998 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:46:25.051 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1515.g6822c 151/202 2024-02-17T11:46:25.075 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1515.g6822c 151/202 2024-02-17T11:46:25.076 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:25.076 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-17T11:46:25.076 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:46:25.118 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 152/202 2024-02-17T11:46:25.140 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 152/202 2024-02-17T11:46:25.141 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:25.141 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-17T11:46:25.141 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-17T11:46:25.141 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-17T11:46:25.141 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:46:25.444 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T11:46:25.466 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T11:46:25.466 INFO:teuthology.orchestra.run.smithi094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:25.466 INFO:teuthology.orchestra.run.smithi094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-17T11:46:25.466 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-17T11:46:25.467 INFO:teuthology.orchestra.run.smithi094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-17T11:46:25.467 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:46:25.928 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 143/202 2024-02-17T11:46:25.960 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 144/202 2024-02-17T11:46:26.060 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 144/202 2024-02-17T11:46:26.153 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 135/202 2024-02-17T11:46:26.178 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 135/202 2024-02-17T11:46:26.178 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:26.178 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-17T11:46:26.178 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-17T11:46:26.178 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-17T11:46:26.179 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:46:26.215 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el 145/202 2024-02-17T11:46:26.245 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 146/202 2024-02-17T11:46:26.268 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 146/202 2024-02-17T11:46:26.269 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:26.269 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-17T11:46:26.269 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-17T11:46:26.269 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-17T11:46:26.269 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:46:26.312 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 147/202 2024-02-17T11:46:26.323 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 147/202 2024-02-17T11:46:26.368 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 148/202 2024-02-17T11:46:26.386 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 136/202 2024-02-17T11:46:26.409 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 136/202 2024-02-17T11:46:26.409 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:26.409 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-17T11:46:26.409 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-17T11:46:26.409 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-17T11:46:26.409 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:46:26.854 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 137/202 2024-02-17T11:46:26.876 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 137/202 2024-02-17T11:46:26.876 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:26.876 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-17T11:46:26.876 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-17T11:46:26.877 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-17T11:46:26.877 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:46:26.964 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : automake-1.16.2-8.el9.noarch 138/202 2024-02-17T11:46:27.228 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-17T11:46:27.280 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 150/202 2024-02-17T11:46:27.303 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 150/202 2024-02-17T11:46:27.304 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:27.304 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-17T11:46:27.304 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-17T11:46:27.304 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-17T11:46:27.304 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:46:27.349 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1515.g6822c 151/202 2024-02-17T11:46:27.371 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1515.g6822c 151/202 2024-02-17T11:46:27.371 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:27.371 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-17T11:46:27.371 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:46:27.414 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 139/202 2024-02-17T11:46:27.416 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 152/202 2024-02-17T11:46:27.438 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 152/202 2024-02-17T11:46:27.438 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:27.438 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-17T11:46:27.438 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-17T11:46:27.439 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-17T11:46:27.439 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:46:27.445 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 139/202 2024-02-17T11:46:27.477 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.e 140/202 2024-02-17T11:46:27.537 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5. 141/202 2024-02-17T11:46:27.560 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 142/202 2024-02-17T11:46:27.750 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T11:46:27.773 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T11:46:27.773 INFO:teuthology.orchestra.run.smithi125.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:27.773 INFO:teuthology.orchestra.run.smithi125.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-17T11:46:27.773 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-17T11:46:27.773 INFO:teuthology.orchestra.run.smithi125.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-17T11:46:27.773 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:46:27.789 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 142/202 2024-02-17T11:46:27.819 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 143/202 2024-02-17T11:46:28.797 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 143/202 2024-02-17T11:46:28.819 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 144/202 2024-02-17T11:46:28.921 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 144/202 2024-02-17T11:46:29.082 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el 145/202 2024-02-17T11:46:29.104 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 146/202 2024-02-17T11:46:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 146/202 2024-02-17T11:46:29.128 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:29.128 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-17T11:46:29.128 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-17T11:46:29.128 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-17T11:46:29.128 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:46:29.172 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 147/202 2024-02-17T11:46:29.183 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 147/202 2024-02-17T11:46:29.229 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 148/202 2024-02-17T11:46:29.907 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 154/202 2024-02-17T11:46:29.956 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 155/202 2024-02-17T11:46:29.995 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 156/202 2024-02-17T11:46:30.031 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-17T11:46:30.087 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libtool-2.4.6-45.el9.x86_64 149/202 2024-02-17T11:46:30.115 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 150/202 2024-02-17T11:46:30.138 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 150/202 2024-02-17T11:46:30.138 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:30.138 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-17T11:46:30.138 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-17T11:46:30.138 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-17T11:46:30.139 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:46:30.183 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1515.g6822c 151/202 2024-02-17T11:46:30.204 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1515.g6822c 151/202 2024-02-17T11:46:30.205 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:30.205 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-17T11:46:30.205 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:46:30.218 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 158/202 2024-02-17T11:46:30.250 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 152/202 2024-02-17T11:46:30.272 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 152/202 2024-02-17T11:46:30.272 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:30.273 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-02-17T11:46:30.273 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-17T11:46:30.273 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-02-17T11:46:30.273 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:46:30.330 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-17T11:46:30.371 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 160/202 2024-02-17T11:46:30.427 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-17T11:46:30.479 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-17T11:46:30.558 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-17T11:46:30.592 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T11:46:30.606 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-17T11:46:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T11:46:30.615 INFO:teuthology.orchestra.run.smithi189.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-17T11:46:30.615 INFO:teuthology.orchestra.run.smithi189.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-17T11:46:30.615 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-17T11:46:30.615 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-17T11:46:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:46:30.658 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-17T11:46:30.689 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_ 166/202 2024-02-17T11:46:30.719 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-17T11:46:30.802 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-17T11:46:30.857 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-17T11:46:30.893 INFO:teuthology.orchestra.run.smithi094.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-17T11:46:31.527 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T11:46:31.575 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T11:46:31.594 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T11:46:32.163 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 154/202 2024-02-17T11:46:32.203 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 155/202 2024-02-17T11:46:32.242 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 156/202 2024-02-17T11:46:32.270 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-17T11:46:32.397 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 172/202 2024-02-17T11:46:32.447 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 158/202 2024-02-17T11:46:32.471 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-17T11:46:32.512 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-17T11:46:32.559 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-17T11:46:32.601 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 160/202 2024-02-17T11:46:32.603 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-17T11:46:32.634 INFO:teuthology.orchestra.run.smithi094.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-17T11:46:32.640 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-17T11:46:32.658 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-17T11:46:32.692 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-17T11:46:32.700 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-17T11:46:32.718 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-17T11:46:32.718 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-17T11:46:32.775 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-17T11:46:32.803 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-17T11:46:32.819 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-17T11:46:32.831 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-17T11:46:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-17T11:46:32.861 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-17T11:46:32.863 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-17T11:46:32.893 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_ 166/202 2024-02-17T11:46:32.923 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-17T11:46:32.946 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-17T11:46:32.983 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-17T11:46:32.987 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-17T11:46:33.023 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-17T11:46:33.028 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-17T11:46:33.048 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-17T11:46:33.074 INFO:teuthology.orchestra.run.smithi125.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-17T11:46:33.075 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-17T11:46:33.108 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-17T11:46:33.143 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-17T11:46:33.189 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-17T11:46:33.221 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-17T11:46:33.246 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-17T11:46:33.272 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-17T11:46:33.306 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-17T11:46:33.331 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-17T11:46:33.337 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T11:46:33.337 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T11:46:33.517 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T11:46:33.517 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-17T11:46:33.600 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-17T11:46:33.625 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-17T11:46:33.638 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T11:46:33.650 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-17T11:46:33.677 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T11:46:33.684 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-17T11:46:33.697 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T11:46:33.716 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-17T11:46:33.784 INFO:teuthology.orchestra.run.smithi094.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-17T11:46:34.509 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 172/202 2024-02-17T11:46:34.581 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-17T11:46:34.613 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-17T11:46:34.694 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-17T11:46:34.743 INFO:teuthology.orchestra.run.smithi125.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-17T11:46:34.768 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-17T11:46:34.793 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-17T11:46:34.820 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-17T11:46:34.820 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-17T11:46:34.904 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-17T11:46:34.932 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-17T11:46:34.962 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-17T11:46:34.962 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-17T11:46:35.047 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-17T11:46:35.063 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 154/202 2024-02-17T11:46:35.073 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-17T11:46:35.104 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 155/202 2024-02-17T11:46:35.107 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-17T11:46:35.131 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-17T11:46:35.134 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 156/202 2024-02-17T11:46:35.159 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-17T11:46:35.162 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 157/202 2024-02-17T11:46:35.191 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-17T11:46:35.224 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-17T11:46:35.264 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-17T11:46:35.296 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-17T11:46:35.321 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-17T11:46:35.341 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 158/202 2024-02-17T11:46:35.346 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-17T11:46:35.372 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-17T11:46:35.397 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-17T11:46:35.404 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T11:46:35.404 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T11:46:35.461 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 159/202 2024-02-17T11:46:35.510 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 160/202 2024-02-17T11:46:35.549 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 161/202 2024-02-17T11:46:35.583 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T11:46:35.583 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-17T11:46:35.601 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 162/202 2024-02-17T11:46:35.658 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-17T11:46:35.669 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 163/202 2024-02-17T11:46:35.683 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-17T11:46:35.708 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-17T11:46:35.711 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : iproute-tc-6.7.0-2.el9.x86_64 164/202 2024-02-17T11:46:35.733 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-17T11:46:35.748 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 165/202 2024-02-17T11:46:35.766 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-17T11:46:35.779 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_ 166/202 2024-02-17T11:46:35.808 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-team-1:1.45.91-1.el9.x86_64 167/202 2024-02-17T11:46:35.831 INFO:teuthology.orchestra.run.smithi125.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-17T11:46:35.863 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : NetworkManager-tui-1:1.45.91-1.el9.x86_64 168/202 2024-02-17T11:46:35.907 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 169/202 2024-02-17T11:46:35.949 INFO:teuthology.orchestra.run.smithi189.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 170/202 2024-02-17T11:46:36.503 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T11:46:36.555 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T11:46:36.574 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 171/202 2024-02-17T11:46:37.394 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 172/202 2024-02-17T11:46:37.458 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 173/202 2024-02-17T11:46:37.499 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 174/202 2024-02-17T11:46:37.579 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : flex-2.6.4-9.el9.x86_64 175/202 2024-02-17T11:46:37.628 INFO:teuthology.orchestra.run.smithi189.stdout: Installing : bison-3.7.4-5.el9.x86_64 176/202 2024-02-17T11:46:37.653 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 177/202 2024-02-17T11:46:37.678 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 178/202 2024-02-17T11:46:37.696 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 179/202 2024-02-17T11:46:37.696 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-17T11:46:37.772 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 180/202 2024-02-17T11:46:37.800 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 181/202 2024-02-17T11:46:37.830 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 182/202 2024-02-17T11:46:37.830 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-17T11:46:37.907 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 183/202 2024-02-17T11:46:37.933 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/202 2024-02-17T11:46:37.967 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 185/202 2024-02-17T11:46:37.991 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : iproute-tc-6.2.0-5.el9.x86_64 186/202 2024-02-17T11:46:38.019 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 187/202 2024-02-17T11:46:38.051 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : iproute-6.2.0-5.el9.x86_64 188/202 2024-02-17T11:46:38.084 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 189/202 2024-02-17T11:46:38.123 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 190/202 2024-02-17T11:46:38.156 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 191/202 2024-02-17T11:46:38.181 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 192/202 2024-02-17T11:46:38.206 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 193/202 2024-02-17T11:46:38.232 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 194/202 2024-02-17T11:46:38.257 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 195/202 2024-02-17T11:46:38.263 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T11:46:38.263 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T11:46:38.460 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 196/202 2024-02-17T11:46:38.460 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-17T11:46:38.526 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 197/202 2024-02-17T11:46:38.551 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 198/202 2024-02-17T11:46:38.576 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 199/202 2024-02-17T11:46:38.601 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 200/202 2024-02-17T11:46:38.634 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 201/202 2024-02-17T11:46:38.699 INFO:teuthology.orchestra.run.smithi189.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-17T11:46:38.804 INFO:teuthology.orchestra.run.smithi094.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-17T11:46:38.804 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 1/202 2024-02-17T11:46:38.804 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 2/202 2024-02-17T11:46:38.804 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 3/202 2024-02-17T11:46:38.804 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 4/202 2024-02-17T11:46:38.804 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1515.g6822c 5/202 2024-02-17T11:46:38.804 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 6/202 2024-02-17T11:46:38.804 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 7/202 2024-02-17T11:46:38.804 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 8/202 2024-02-17T11:46:38.804 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 9/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 10/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 13/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 14/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_ 15/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 17/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_6 18/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 19/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 20/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el 21/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9. 22/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_6 23/202 2024-02-17T11:46:38.805 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 24/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 25/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 26/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 27/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 28/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 29/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5. 30/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 31/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 32/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 33/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el 34/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 35/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.e 36/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 37/202 2024-02-17T11:46:38.806 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 38/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 39/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-17T11:46:38.807 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-17T11:46:38.808 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-17T11:46:38.808 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-17T11:46:38.808 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-17T11:46:38.808 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-17T11:46:38.808 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-17T11:46:38.808 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-17T11:46:38.808 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-17T11:46:38.808 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-17T11:46:38.808 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-17T11:46:38.808 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-17T11:46:38.808 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-17T11:46:38.808 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-17T11:46:38.808 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-17T11:46:38.809 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-17T11:46:38.810 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-17T11:46:38.810 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-17T11:46:38.810 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-17T11:46:38.811 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-17T11:46:38.811 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-17T11:46:38.811 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-17T11:46:38.811 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-17T11:46:38.811 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-17T11:46:38.811 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-17T11:46:38.811 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-17T11:46:38.811 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-17T11:46:38.811 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-17T11:46:38.811 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-17T11:46:38.811 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-17T11:46:38.811 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-17T11:46:38.812 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-17T11:46:38.812 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-17T11:46:38.812 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-17T11:46:38.812 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-17T11:46:38.812 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-17T11:46:38.812 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-17T11:46:38.812 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-17T11:46:38.812 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-17T11:46:38.812 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-17T11:46:38.812 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-17T11:46:38.812 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-17T11:46:38.812 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-17T11:46:38.812 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-17T11:46:38.813 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-17T11:46:38.814 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-17T11:46:38.815 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-17T11:46:38.815 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-17T11:46:38.815 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-17T11:46:38.815 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-17T11:46:38.815 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-17T11:46:38.815 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-17T11:46:38.815 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-17T11:46:38.815 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-17T11:46:38.815 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-17T11:46:38.815 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-17T11:46:38.815 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-17T11:46:38.815 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-17T11:46:38.815 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 151/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-17T11:46:38.816 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-17T11:46:38.817 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-17T11:46:38.818 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-17T11:46:38.819 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-17T11:46:38.819 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-17T11:46:38.819 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-17T11:46:38.819 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-17T11:46:38.819 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-17T11:46:38.819 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-17T11:46:38.819 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-17T11:46:38.819 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-17T11:46:40.435 INFO:teuthology.orchestra.run.smithi125.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-17T11:46:40.435 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 1/202 2024-02-17T11:46:40.435 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 2/202 2024-02-17T11:46:40.435 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 3/202 2024-02-17T11:46:40.435 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 4/202 2024-02-17T11:46:40.435 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1515.g6822c 5/202 2024-02-17T11:46:40.435 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 6/202 2024-02-17T11:46:40.436 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 7/202 2024-02-17T11:46:40.436 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 8/202 2024-02-17T11:46:40.436 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 9/202 2024-02-17T11:46:40.436 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 10/202 2024-02-17T11:46:40.436 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T11:46:40.436 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T11:46:40.436 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 13/202 2024-02-17T11:46:40.436 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 14/202 2024-02-17T11:46:40.436 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_ 15/202 2024-02-17T11:46:40.436 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T11:46:40.437 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 17/202 2024-02-17T11:46:40.437 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_6 18/202 2024-02-17T11:46:40.437 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 19/202 2024-02-17T11:46:40.437 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 20/202 2024-02-17T11:46:40.437 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el 21/202 2024-02-17T11:46:40.437 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9. 22/202 2024-02-17T11:46:40.437 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_6 23/202 2024-02-17T11:46:40.437 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 24/202 2024-02-17T11:46:40.437 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 25/202 2024-02-17T11:46:40.437 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 26/202 2024-02-17T11:46:40.437 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 27/202 2024-02-17T11:46:40.437 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 28/202 2024-02-17T11:46:40.437 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 29/202 2024-02-17T11:46:40.438 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5. 30/202 2024-02-17T11:46:40.438 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 31/202 2024-02-17T11:46:40.438 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 32/202 2024-02-17T11:46:40.438 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 33/202 2024-02-17T11:46:40.438 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el 34/202 2024-02-17T11:46:40.438 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 35/202 2024-02-17T11:46:40.438 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.e 36/202 2024-02-17T11:46:40.438 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 37/202 2024-02-17T11:46:40.438 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 38/202 2024-02-17T11:46:40.438 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 39/202 2024-02-17T11:46:40.438 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-17T11:46:40.438 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-17T11:46:40.438 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-17T11:46:40.439 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-17T11:46:40.439 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-17T11:46:40.439 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-17T11:46:40.439 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-17T11:46:40.439 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-17T11:46:40.439 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-17T11:46:40.439 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-17T11:46:40.439 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-17T11:46:40.439 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-17T11:46:40.439 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-17T11:46:40.439 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-17T11:46:40.439 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-17T11:46:40.439 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-17T11:46:40.440 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-17T11:46:40.440 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-17T11:46:40.440 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-17T11:46:40.440 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-17T11:46:40.440 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-17T11:46:40.440 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-17T11:46:40.440 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-17T11:46:40.440 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-17T11:46:40.440 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-17T11:46:40.440 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-17T11:46:40.441 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-17T11:46:40.441 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-17T11:46:40.441 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-17T11:46:40.441 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-17T11:46:40.441 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-17T11:46:40.441 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-17T11:46:40.441 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-17T11:46:40.441 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-17T11:46:40.441 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-17T11:46:40.441 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-17T11:46:40.441 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-17T11:46:40.441 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-17T11:46:40.442 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-17T11:46:40.443 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-17T11:46:40.444 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-17T11:46:40.444 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-17T11:46:40.444 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-17T11:46:40.444 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-17T11:46:40.444 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-17T11:46:40.444 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-17T11:46:40.444 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-17T11:46:40.444 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-17T11:46:40.444 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-17T11:46:40.444 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-17T11:46:40.444 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-17T11:46:40.444 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-17T11:46:40.444 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-17T11:46:40.445 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-17T11:46:40.445 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-17T11:46:40.445 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-17T11:46:40.445 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-17T11:46:40.445 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-17T11:46:40.445 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-17T11:46:40.445 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-17T11:46:40.445 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-17T11:46:40.445 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-17T11:46:40.445 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-17T11:46:40.445 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-17T11:46:40.445 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-17T11:46:40.446 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-17T11:46:40.446 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-17T11:46:40.446 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-17T11:46:40.446 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-17T11:46:40.446 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-17T11:46:40.446 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-17T11:46:40.446 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-17T11:46:40.446 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-17T11:46:40.446 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-17T11:46:40.446 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-17T11:46:40.446 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 151/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-17T11:46:40.447 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-17T11:46:40.448 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-17T11:46:40.448 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-17T11:46:40.448 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-17T11:46:40.448 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-17T11:46:40.448 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-17T11:46:40.448 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-17T11:46:40.448 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-17T11:46:40.448 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-17T11:46:40.448 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-17T11:46:40.448 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-17T11:46:40.449 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-17T11:46:40.449 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-17T11:46:40.449 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-17T11:46:40.449 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-17T11:46:40.449 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-17T11:46:40.449 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-17T11:46:40.449 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-17T11:46:40.449 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-17T11:46:40.449 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-17T11:46:40.449 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-17T11:46:40.450 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-17T11:46:40.450 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-17T11:46:40.450 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-17T11:46:40.450 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-17T11:46:40.450 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-17T11:46:40.450 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-17T11:46:40.450 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-17T11:46:40.450 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-17T11:46:40.450 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-17T11:46:40.450 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-17T11:46:40.450 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-17T11:46:40.451 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-17T11:46:40.451 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-17T11:46:40.451 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-17T11:46:40.452 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-17T11:46:40.452 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-17T11:46:40.452 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-17T11:46:40.452 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-17T11:46:40.452 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-17T11:46:40.453 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-17T11:46:40.453 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-17T11:46:40.453 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-17T11:46:40.453 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-17T11:46:40.453 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-17T11:46:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-17T11:46:41.480 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:46:41.481 INFO:teuthology.orchestra.run.smithi094.stdout:Upgraded: 2024-02-17T11:46:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-17T11:46:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-17T11:46:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-17T11:46:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-17T11:46:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: acl-2.3.1-4.el9.x86_64 2024-02-17T11:46:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-17T11:46:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-17T11:46:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-17T11:46:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-17T11:46:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-17T11:46:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-17T11:46:41.481 INFO:teuthology.orchestra.run.smithi094.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-17T11:46:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-17T11:46:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-17T11:46:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-17T11:46:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-17T11:46:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-17T11:46:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-17T11:46:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-17T11:46:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-17T11:46:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-17T11:46:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-17T11:46:41.482 INFO:teuthology.orchestra.run.smithi094.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout:Installed: 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: automake-1.16.2-8.el9.noarch 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: bison-3.7.4-5.el9.x86_64 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-immutable-object-cache-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.483 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.484 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: flex-2.6.4-9.el9.x86_64 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: jq-1.6-16.el9.x86_64 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-17T11:46:41.485 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-17T11:46:41.486 INFO:teuthology.orchestra.run.smithi094.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: lua-5.4.4-4.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-13.11-1.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-17T11:46:41.487 INFO:teuthology.orchestra.run.smithi094.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-17T11:46:41.488 INFO:teuthology.orchestra.run.smithi094.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-17T11:46:41.489 INFO:teuthology.orchestra.run.smithi094.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-17T11:46:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-17T11:46:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-17T11:46:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-17T11:46:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-17T11:46:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-17T11:46:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-17T11:46:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-17T11:46:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-17T11:46:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: python3-ply-3.11-14.el9.noarch 2024-02-17T11:46:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-17T11:46:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-17T11:46:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-17T11:46:41.490 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-17T11:46:41.491 INFO:teuthology.orchestra.run.smithi094.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-17T11:46:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-17T11:46:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-17T11:46:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-17T11:46:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-17T11:46:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-17T11:46:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-17T11:46:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-17T11:46:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-17T11:46:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-17T11:46:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:41.492 INFO:teuthology.orchestra.run.smithi094.stdout: re2-1:20211101-3.el9.x86_64 2024-02-17T11:46:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-17T11:46:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-17T11:46:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-17T11:46:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-17T11:46:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-17T11:46:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-17T11:46:41.493 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:46:41.493 INFO:teuthology.orchestra.run.smithi094.stdout:Complete! 2024-02-17T11:46:42.018 DEBUG:teuthology.parallel:result is None 2024-02-17T11:46:43.312 INFO:teuthology.orchestra.run.smithi125.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-17T11:46:43.312 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:46:43.312 INFO:teuthology.orchestra.run.smithi125.stdout:Upgraded: 2024-02-17T11:46:43.312 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-17T11:46:43.312 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-17T11:46:43.312 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-17T11:46:43.312 INFO:teuthology.orchestra.run.smithi125.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-17T11:46:43.312 INFO:teuthology.orchestra.run.smithi125.stdout: acl-2.3.1-4.el9.x86_64 2024-02-17T11:46:43.312 INFO:teuthology.orchestra.run.smithi125.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-17T11:46:43.312 INFO:teuthology.orchestra.run.smithi125.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-17T11:46:43.313 INFO:teuthology.orchestra.run.smithi125.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout:Installed: 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout: automake-1.16.2-8.el9.noarch 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout: bison-3.7.4-5.el9.x86_64 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.314 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-immutable-object-cache-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.315 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.316 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.316 INFO:teuthology.orchestra.run.smithi125.stdout: ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:43.316 INFO:teuthology.orchestra.run.smithi125.stdout: cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:43.316 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.316 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.316 INFO:teuthology.orchestra.run.smithi125.stdout: cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:43.316 INFO:teuthology.orchestra.run.smithi125.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-17T11:46:43.316 INFO:teuthology.orchestra.run.smithi125.stdout: flex-2.6.4-9.el9.x86_64 2024-02-17T11:46:43.316 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-17T11:46:43.316 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-17T11:46:43.316 INFO:teuthology.orchestra.run.smithi125.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-17T11:46:43.316 INFO:teuthology.orchestra.run.smithi125.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-17T11:46:43.316 INFO:teuthology.orchestra.run.smithi125.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: jq-1.6-16.el9.x86_64 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.317 INFO:teuthology.orchestra.run.smithi125.stdout: libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: lua-5.4.4-4.el9.x86_64 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-17T11:46:43.318 INFO:teuthology.orchestra.run.smithi125.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-13.11-1.el9.x86_64 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.319 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.320 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.320 INFO:teuthology.orchestra.run.smithi125.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-17T11:46:43.320 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-17T11:46:43.320 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-17T11:46:43.320 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-17T11:46:43.320 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-17T11:46:43.320 INFO:teuthology.orchestra.run.smithi125.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-17T11:46:43.320 INFO:teuthology.orchestra.run.smithi125.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-17T11:46:43.320 INFO:teuthology.orchestra.run.smithi125.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-17T11:46:43.320 INFO:teuthology.orchestra.run.smithi125.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-17T11:46:43.320 INFO:teuthology.orchestra.run.smithi125.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-17T11:46:43.320 INFO:teuthology.orchestra.run.smithi125.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-17T11:46:43.320 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-17T11:46:43.321 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-ply-3.11-14.el9.noarch 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-17T11:46:43.322 INFO:teuthology.orchestra.run.smithi125.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-17T11:46:43.323 INFO:teuthology.orchestra.run.smithi125.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-17T11:46:43.324 INFO:teuthology.orchestra.run.smithi125.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-17T11:46:43.324 INFO:teuthology.orchestra.run.smithi125.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-17T11:46:43.324 INFO:teuthology.orchestra.run.smithi125.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-17T11:46:43.324 INFO:teuthology.orchestra.run.smithi125.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-17T11:46:43.324 INFO:teuthology.orchestra.run.smithi125.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-17T11:46:43.324 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.324 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.324 INFO:teuthology.orchestra.run.smithi125.stdout: rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:43.324 INFO:teuthology.orchestra.run.smithi125.stdout: re2-1:20211101-3.el9.x86_64 2024-02-17T11:46:43.324 INFO:teuthology.orchestra.run.smithi125.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-17T11:46:43.324 INFO:teuthology.orchestra.run.smithi125.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-17T11:46:43.324 INFO:teuthology.orchestra.run.smithi125.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-17T11:46:43.324 INFO:teuthology.orchestra.run.smithi125.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-17T11:46:43.325 INFO:teuthology.orchestra.run.smithi125.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-17T11:46:43.325 INFO:teuthology.orchestra.run.smithi125.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-17T11:46:43.325 INFO:teuthology.orchestra.run.smithi125.stdout: 2024-02-17T11:46:43.325 INFO:teuthology.orchestra.run.smithi125.stdout:Complete! 2024-02-17T11:46:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: Running scriptlet: openssl-libs-1:3.0.7-24.el9.x86_64 202/202 2024-02-17T11:46:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 1/202 2024-02-17T11:46:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 2/202 2024-02-17T11:46:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 3/202 2024-02-17T11:46:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 4/202 2024-02-17T11:46:43.467 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1515.g6822c 5/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 6/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 7/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 8/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 9/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 10/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 11/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 12/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 13/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 14/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_ 15/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 16/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 17/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_6 18/202 2024-02-17T11:46:43.468 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86 19/202 2024-02-17T11:46:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 20/202 2024-02-17T11:46:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el 21/202 2024-02-17T11:46:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9. 22/202 2024-02-17T11:46:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_6 23/202 2024-02-17T11:46:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 24/202 2024-02-17T11:46:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 25/202 2024-02-17T11:46:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 26/202 2024-02-17T11:46:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 27/202 2024-02-17T11:46:43.469 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 28/202 2024-02-17T11:46:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 29/202 2024-02-17T11:46:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5. 30/202 2024-02-17T11:46:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noa 31/202 2024-02-17T11:46:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.n 32/202 2024-02-17T11:46:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1515.g682 33/202 2024-02-17T11:46:43.487 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el 34/202 2024-02-17T11:46:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 35/202 2024-02-17T11:46:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.e 36/202 2024-02-17T11:46:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 37/202 2024-02-17T11:46:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 38/202 2024-02-17T11:46:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 39/202 2024-02-17T11:46:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/202 2024-02-17T11:46:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/202 2024-02-17T11:46:43.488 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/202 2024-02-17T11:46:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 43/202 2024-02-17T11:46:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/202 2024-02-17T11:46:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/202 2024-02-17T11:46:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/202 2024-02-17T11:46:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/202 2024-02-17T11:46:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/202 2024-02-17T11:46:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/202 2024-02-17T11:46:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/202 2024-02-17T11:46:43.489 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/202 2024-02-17T11:46:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/202 2024-02-17T11:46:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 53/202 2024-02-17T11:46:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/202 2024-02-17T11:46:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/202 2024-02-17T11:46:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-5.4.4-4.el9.x86_64 56/202 2024-02-17T11:46:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/202 2024-02-17T11:46:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/202 2024-02-17T11:46:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 59/202 2024-02-17T11:46:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/202 2024-02-17T11:46:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/202 2024-02-17T11:46:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/202 2024-02-17T11:46:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/202 2024-02-17T11:46:43.490 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 64/202 2024-02-17T11:46:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 65/202 2024-02-17T11:46:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 66/202 2024-02-17T11:46:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 67/202 2024-02-17T11:46:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 68/202 2024-02-17T11:46:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 69/202 2024-02-17T11:46:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 70/202 2024-02-17T11:46:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 71/202 2024-02-17T11:46:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 72/202 2024-02-17T11:46:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 73/202 2024-02-17T11:46:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 74/202 2024-02-17T11:46:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 75/202 2024-02-17T11:46:43.491 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 76/202 2024-02-17T11:46:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 77/202 2024-02-17T11:46:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 78/202 2024-02-17T11:46:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 79/202 2024-02-17T11:46:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 80/202 2024-02-17T11:46:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/202 2024-02-17T11:46:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 82/202 2024-02-17T11:46:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 83/202 2024-02-17T11:46:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 84/202 2024-02-17T11:46:43.492 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 85/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 86/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 87/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 88/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 89/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 90/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 91/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 92/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 93/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : jq-1.6-16.el9.x86_64 94/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 95/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 96/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 97/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 98/202 2024-02-17T11:46:43.493 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 99/202 2024-02-17T11:46:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-ply-3.11-14.el9.noarch 100/202 2024-02-17T11:46:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 101/202 2024-02-17T11:46:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 102/202 2024-02-17T11:46:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 103/202 2024-02-17T11:46:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 104/202 2024-02-17T11:46:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/202 2024-02-17T11:46:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/202 2024-02-17T11:46:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/202 2024-02-17T11:46:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/202 2024-02-17T11:46:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/202 2024-02-17T11:46:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/202 2024-02-17T11:46:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/202 2024-02-17T11:46:43.494 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/202 2024-02-17T11:46:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/202 2024-02-17T11:46:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/202 2024-02-17T11:46:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/202 2024-02-17T11:46:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 116/202 2024-02-17T11:46:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/202 2024-02-17T11:46:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/202 2024-02-17T11:46:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/202 2024-02-17T11:46:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/202 2024-02-17T11:46:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/202 2024-02-17T11:46:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/202 2024-02-17T11:46:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/202 2024-02-17T11:46:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 124/202 2024-02-17T11:46:43.495 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/202 2024-02-17T11:46:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/202 2024-02-17T11:46:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/202 2024-02-17T11:46:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/202 2024-02-17T11:46:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/202 2024-02-17T11:46:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/202 2024-02-17T11:46:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/202 2024-02-17T11:46:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/202 2024-02-17T11:46:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/202 2024-02-17T11:46:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/202 2024-02-17T11:46:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/202 2024-02-17T11:46:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/202 2024-02-17T11:46:43.496 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/202 2024-02-17T11:46:43.497 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 151/202 2024-02-17T11:46:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/202 2024-02-17T11:46:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 153/202 2024-02-17T11:46:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/202 2024-02-17T11:46:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 155/202 2024-02-17T11:46:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 156/202 2024-02-17T11:46:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.45.91-1.el9.x86_64 157/202 2024-02-17T11:46:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/202 2024-02-17T11:46:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.45.91-1.el9.x86_64 159/202 2024-02-17T11:46:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/202 2024-02-17T11:46:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.45.91-1.el9.x86_64 161/202 2024-02-17T11:46:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/202 2024-02-17T11:46:43.498 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.45.91-1.el9.x86_64 163/202 2024-02-17T11:46:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/202 2024-02-17T11:46:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/202 2024-02-17T11:46:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/202 2024-02-17T11:46:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/202 2024-02-17T11:46:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/202 2024-02-17T11:46:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/202 2024-02-17T11:46:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/202 2024-02-17T11:46:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/202 2024-02-17T11:46:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/202 2024-02-17T11:46:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : iproute-6.7.0-2.el9.x86_64 173/202 2024-02-17T11:46:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : iproute-6.2.0-5.el9.x86_64 174/202 2024-02-17T11:46:43.499 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : iproute-tc-6.7.0-2.el9.x86_64 175/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : iproute-tc-6.2.0-5.el9.x86_64 176/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 177/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 178/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 179/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 180/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 181/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 182/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 183/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 184/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 185/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 186/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 187/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 188/202 2024-02-17T11:46:43.500 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 189/202 2024-02-17T11:46:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 190/202 2024-02-17T11:46:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 191/202 2024-02-17T11:46:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 192/202 2024-02-17T11:46:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 193/202 2024-02-17T11:46:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 194/202 2024-02-17T11:46:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 195/202 2024-02-17T11:46:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 196/202 2024-02-17T11:46:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 197/202 2024-02-17T11:46:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 198/202 2024-02-17T11:46:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 199/202 2024-02-17T11:46:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 200/202 2024-02-17T11:46:43.501 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 201/202 2024-02-17T11:46:43.934 DEBUG:teuthology.parallel:result is None 2024-02-17T11:46:46.339 INFO:teuthology.orchestra.run.smithi189.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 202/202 2024-02-17T11:46:46.339 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:46:46.339 INFO:teuthology.orchestra.run.smithi189.stdout:Upgraded: 2024-02-17T11:46:46.339 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-1:1.45.91-1.el9.x86_64 2024-02-17T11:46:46.339 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-libnm-1:1.45.91-1.el9.x86_64 2024-02-17T11:46:46.339 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-team-1:1.45.91-1.el9.x86_64 2024-02-17T11:46:46.339 INFO:teuthology.orchestra.run.smithi189.stdout: NetworkManager-tui-1:1.45.91-1.el9.x86_64 2024-02-17T11:46:46.340 INFO:teuthology.orchestra.run.smithi189.stdout: acl-2.3.1-4.el9.x86_64 2024-02-17T11:46:46.340 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-02-17T11:46:46.340 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-02-17T11:46:46.340 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-02-17T11:46:46.340 INFO:teuthology.orchestra.run.smithi189.stdout: iproute-6.7.0-2.el9.x86_64 2024-02-17T11:46:46.340 INFO:teuthology.orchestra.run.smithi189.stdout: iproute-tc-6.7.0-2.el9.x86_64 2024-02-17T11:46:46.340 INFO:teuthology.orchestra.run.smithi189.stdout: libacl-2.3.1-4.el9.x86_64 2024-02-17T11:46:46.340 INFO:teuthology.orchestra.run.smithi189.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-02-17T11:46:46.340 INFO:teuthology.orchestra.run.smithi189.stdout: libblkid-2.37.4-18.el9.x86_64 2024-02-17T11:46:46.340 INFO:teuthology.orchestra.run.smithi189.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-02-17T11:46:46.340 INFO:teuthology.orchestra.run.smithi189.stdout: libmount-2.37.4-18.el9.x86_64 2024-02-17T11:46:46.340 INFO:teuthology.orchestra.run.smithi189.stdout: librados2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.340 INFO:teuthology.orchestra.run.smithi189.stdout: librbd1-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-2.37.4-18.el9.x86_64 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-2.37.4-18.el9.x86_64 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout:Installed: 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: automake-1.16.2-8.el9.noarch 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: bison-3.7.4-5.el9.x86_64 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-17T11:46:46.341 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-grafana-dashboards-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-immutable-object-cache-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mds-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-rook-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mon-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.342 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-osd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-prometheus-alerts-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-radosgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-selinux-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-test-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-volume-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: cephadm-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-shell-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-top-2:19.0.0-1515.g6822c7b5.el9.noarch 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: flex-2.6.4-9.el9.x86_64 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-17T11:46:46.343 INFO:teuthology.orchestra.run.smithi189.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-17T11:46:46.344 INFO:teuthology.orchestra.run.smithi189.stdout: fuse-2.9.9-15.el9.x86_64 2024-02-17T11:46:46.344 INFO:teuthology.orchestra.run.smithi189.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-17T11:46:46.344 INFO:teuthology.orchestra.run.smithi189.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-17T11:46:46.344 INFO:teuthology.orchestra.run.smithi189.stdout: jq-1.6-16.el9.x86_64 2024-02-17T11:46:46.344 INFO:teuthology.orchestra.run.smithi189.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-02-17T11:46:46.344 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-17T11:46:46.344 INFO:teuthology.orchestra.run.smithi189.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-17T11:46:46.344 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs-devel-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.344 INFO:teuthology.orchestra.run.smithi189.stdout: libcephfs2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.344 INFO:teuthology.orchestra.run.smithi189.stdout: libcephsqlite-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.344 INFO:teuthology.orchestra.run.smithi189.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-17T11:46:46.344 INFO:teuthology.orchestra.run.smithi189.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-17T11:46:46.344 INFO:teuthology.orchestra.run.smithi189.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-17T11:46:46.345 INFO:teuthology.orchestra.run.smithi189.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-17T11:46:46.345 INFO:teuthology.orchestra.run.smithi189.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-17T11:46:46.345 INFO:teuthology.orchestra.run.smithi189.stdout: librados-devel-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.345 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.345 INFO:teuthology.orchestra.run.smithi189.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-17T11:46:46.345 INFO:teuthology.orchestra.run.smithi189.stdout: librgw2-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.345 INFO:teuthology.orchestra.run.smithi189.stdout: libtool-2.4.6-45.el9.x86_64 2024-02-17T11:46:46.345 INFO:teuthology.orchestra.run.smithi189.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-17T11:46:46.345 INFO:teuthology.orchestra.run.smithi189.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-02-17T11:46:46.345 INFO:teuthology.orchestra.run.smithi189.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-02-17T11:46:46.345 INFO:teuthology.orchestra.run.smithi189.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-17T11:46:46.345 INFO:teuthology.orchestra.run.smithi189.stdout: lua-5.4.4-4.el9.x86_64 2024-02-17T11:46:46.345 INFO:teuthology.orchestra.run.smithi189.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-13.11-1.el9.x86_64 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-server-13.11-1.el9.x86_64 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-17T11:46:46.346 INFO:teuthology.orchestra.run.smithi189.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-02-17T11:46:46.347 INFO:teuthology.orchestra.run.smithi189.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-17T11:46:46.347 INFO:teuthology.orchestra.run.smithi189.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-17T11:46:46.347 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-17T11:46:46.347 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-argparse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.347 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ceph-common-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.347 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cephfs-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.347 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-17T11:46:46.347 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-17T11:46:46.347 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-17T11:46:46.347 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-17T11:46:46.347 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-02-17T11:46:46.347 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-02-17T11:46:46.348 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-17T11:46:46.348 INFO:teuthology.orchestra.run.smithi189.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-17T11:46:46.348 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-17T11:46:46.348 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-17T11:46:46.348 INFO:teuthology.orchestra.run.smithi189.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-17T11:46:46.348 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-17T11:46:46.348 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-17T11:46:46.348 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-17T11:46:46.348 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-17T11:46:46.348 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-17T11:46:46.348 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-17T11:46:46.348 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-17T11:46:46.348 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-17T11:46:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-17T11:46:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-17T11:46:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-17T11:46:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-17T11:46:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-17T11:46:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-17T11:46:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-17T11:46:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-17T11:46:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-17T11:46:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-17T11:46:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-17T11:46:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: python3-ply-3.11-14.el9.noarch 2024-02-17T11:46:46.349 INFO:teuthology.orchestra.run.smithi189.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-17T11:46:46.350 INFO:teuthology.orchestra.run.smithi189.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-17T11:46:46.350 INFO:teuthology.orchestra.run.smithi189.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-17T11:46:46.350 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-17T11:46:46.350 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-17T11:46:46.350 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-17T11:46:46.350 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-17T11:46:46.350 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-02-17T11:46:46.350 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-17T11:46:46.350 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.350 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.350 INFO:teuthology.orchestra.run.smithi189.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-17T11:46:46.350 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-17T11:46:46.350 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-17T11:46:46.351 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rgw-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.351 INFO:teuthology.orchestra.run.smithi189.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-17T11:46:46.351 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-17T11:46:46.351 INFO:teuthology.orchestra.run.smithi189.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-17T11:46:46.351 INFO:teuthology.orchestra.run.smithi189.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-17T11:46:46.351 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-17T11:46:46.351 INFO:teuthology.orchestra.run.smithi189.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-17T11:46:46.351 INFO:teuthology.orchestra.run.smithi189.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-17T11:46:46.352 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-02-17T11:46:46.352 INFO:teuthology.orchestra.run.smithi189.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-17T11:46:46.352 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-17T11:46:46.352 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-17T11:46:46.352 INFO:teuthology.orchestra.run.smithi189.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-17T11:46:46.352 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-23.11.0-1.el9.x86_64 2024-02-17T11:46:46.352 INFO:teuthology.orchestra.run.smithi189.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-02-17T11:46:46.352 INFO:teuthology.orchestra.run.smithi189.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-02-17T11:46:46.352 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-fuse-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.352 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-mirror-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.352 INFO:teuthology.orchestra.run.smithi189.stdout: rbd-nbd-2:19.0.0-1515.g6822c7b5.el9.x86_64 2024-02-17T11:46:46.352 INFO:teuthology.orchestra.run.smithi189.stdout: re2-1:20211101-3.el9.x86_64 2024-02-17T11:46:46.352 INFO:teuthology.orchestra.run.smithi189.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-17T11:46:46.353 INFO:teuthology.orchestra.run.smithi189.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-17T11:46:46.353 INFO:teuthology.orchestra.run.smithi189.stdout: uuid-1.6.2-55.el9.x86_64 2024-02-17T11:46:46.353 INFO:teuthology.orchestra.run.smithi189.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-02-17T11:46:46.353 INFO:teuthology.orchestra.run.smithi189.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-17T11:46:46.353 INFO:teuthology.orchestra.run.smithi189.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-02-17T11:46:46.353 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:46:46.353 INFO:teuthology.orchestra.run.smithi189.stdout:Complete! 2024-02-17T11:46:46.937 DEBUG:teuthology.parallel:result is None 2024-02-17T11:46:46.938 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T11:46:47.100 DEBUG:teuthology.orchestra.run.smithi094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-17T11:46:47.126 INFO:teuthology.orchestra.run.smithi094.stdout:19.0.0-1515.g6822c7b5.el9 2024-02-17T11:46:47.126 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1515.g6822c7b5.el9 2024-02-17T11:46:47.126 INFO:teuthology.task.install:The correct ceph version 19.0.0-1515.g6822c7b5 is installed. 2024-02-17T11:46:47.128 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T11:46:47.346 DEBUG:teuthology.orchestra.run.smithi125:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-17T11:46:47.372 INFO:teuthology.orchestra.run.smithi125.stdout:19.0.0-1515.g6822c7b5.el9 2024-02-17T11:46:47.372 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1515.g6822c7b5.el9 2024-02-17T11:46:47.372 INFO:teuthology.task.install:The correct ceph version 19.0.0-1515.g6822c7b5 is installed. 2024-02-17T11:46:47.374 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T11:46:47.594 DEBUG:teuthology.orchestra.run.smithi189:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-17T11:46:47.617 INFO:teuthology.orchestra.run.smithi189.stdout:19.0.0-1515.g6822c7b5.el9 2024-02-17T11:46:47.617 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1515.g6822c7b5.el9 2024-02-17T11:46:47.617 INFO:teuthology.task.install:The correct ceph version 19.0.0-1515.g6822c7b5 is installed. 2024-02-17T11:46:47.619 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-17T11:46:47.619 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:46:47.620 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-17T11:46:47.654 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-17T11:46:47.654 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-17T11:46:47.690 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-17T11:46:47.690 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-17T11:46:47.720 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-17T11:46:47.721 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:46:47.721 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/daemon-helper 2024-02-17T11:46:47.751 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-17T11:46:47.820 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-17T11:46:47.820 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/daemon-helper 2024-02-17T11:46:47.852 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-17T11:46:47.924 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-17T11:46:47.924 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-02-17T11:46:47.952 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-17T11:46:48.020 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-17T11:46:48.021 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:46:48.021 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-17T11:46:48.051 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-17T11:46:48.120 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-17T11:46:48.120 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-17T11:46:48.150 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-17T11:46:48.219 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-17T11:46:48.219 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-17T11:46:48.247 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-17T11:46:48.314 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-17T11:46:48.314 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:46:48.315 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/usr/bin/stdin-killer 2024-02-17T11:46:48.345 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-17T11:46:48.415 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-17T11:46:48.415 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/usr/bin/stdin-killer 2024-02-17T11:46:48.445 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-17T11:46:48.514 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-17T11:46:48.514 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-02-17T11:46:48.542 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-17T11:46:48.610 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-17T11:46:48.705 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d', 'subvols': {'create': 2}} 2024-02-17T11:46:48.706 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T11:46:48.706 INFO:tasks.cephadm:Cluster fsid is 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:46:48.706 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-17T11:46:48.706 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.94', 'mon.b': '172.21.15.125', 'mon.c': '172.21.15.189'} 2024-02-17T11:46:48.706 INFO:tasks.cephadm:First mon is mon.a on smithi094 2024-02-17T11:46:48.707 INFO:tasks.cephadm:First mgr is x 2024-02-17T11:46:48.707 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-17T11:46:48.707 DEBUG:teuthology.orchestra.run.smithi094:> sudo hostname $(hostname -s) 2024-02-17T11:46:48.738 DEBUG:teuthology.orchestra.run.smithi125:> sudo hostname $(hostname -s) 2024-02-17T11:46:48.768 DEBUG:teuthology.orchestra.run.smithi189:> sudo hostname $(hostname -s) 2024-02-17T11:46:48.796 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-17T11:46:48.796 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T11:46:49.024 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '6822c7b54e949006be7a0802650d01723d834d1d', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76836/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1515-g6822c7b5', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1515.g6822c7b5'}, 'url': 'https://3.chacra.ceph.com/r/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-16 19:04:48.315261', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'main', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-02-17T11:46:49.166 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref main, sha1 6822c7b54e949006be7a0802650d01723d834d1d from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T11:46:49.167 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/x86_64/flavors/default/cephadm 2024-02-17T11:46:49.167 DEBUG:teuthology.orchestra.run.smithi094:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-17T11:46:49.722 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-r--r--. 1 ubuntu ubuntu 766938 Feb 17 11:46 /home/ubuntu/cephtest/cephadm 2024-02-17T11:46:49.722 DEBUG:teuthology.orchestra.run.smithi125:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-17T11:46:50.342 INFO:teuthology.orchestra.run.smithi125.stdout:-rw-r--r--. 1 ubuntu ubuntu 766938 Feb 17 11:46 /home/ubuntu/cephtest/cephadm 2024-02-17T11:46:50.342 DEBUG:teuthology.orchestra.run.smithi189:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/main/6822c7b54e949006be7a0802650d01723d834d1d/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-17T11:46:50.962 INFO:teuthology.orchestra.run.smithi189.stdout:-rw-r--r--. 1 ubuntu ubuntu 766938 Feb 17 11:46 /home/ubuntu/cephtest/cephadm 2024-02-17T11:46:50.962 DEBUG:teuthology.orchestra.run.smithi094:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-17T11:46:50.981 DEBUG:teuthology.orchestra.run.smithi125:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-17T11:46:50.999 DEBUG:teuthology.orchestra.run.smithi189:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-17T11:46:51.033 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d on all hosts... 2024-02-17T11:46:51.033 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d pull 2024-02-17T11:46:51.035 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d pull 2024-02-17T11:46:51.042 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d pull 2024-02-17T11:46:51.880 INFO:teuthology.orchestra.run.smithi189.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d... 2024-02-17T11:46:51.914 INFO:teuthology.orchestra.run.smithi125.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d... 2024-02-17T11:46:52.039 INFO:teuthology.orchestra.run.smithi094.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d... 2024-02-17T11:47:21.783 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:47:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_version": "ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev)", 2024-02-17T11:47:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: "image_id": "32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd", 2024-02-17T11:47:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: "repo_digests": [ 2024-02-17T11:47:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d" 2024-02-17T11:47:21.783 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-02-17T11:47:21.783 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:47:39.341 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-02-17T11:47:39.341 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev)", 2024-02-17T11:47:39.341 INFO:teuthology.orchestra.run.smithi189.stdout: "image_id": "32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd", 2024-02-17T11:47:39.341 INFO:teuthology.orchestra.run.smithi189.stdout: "repo_digests": [ 2024-02-17T11:47:39.341 INFO:teuthology.orchestra.run.smithi189.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d" 2024-02-17T11:47:39.341 INFO:teuthology.orchestra.run.smithi189.stdout: ] 2024-02-17T11:47:39.341 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-02-17T11:47:45.020 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2024-02-17T11:47:45.020 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_version": "ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev)", 2024-02-17T11:47:45.020 INFO:teuthology.orchestra.run.smithi125.stdout: "image_id": "32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd", 2024-02-17T11:47:45.020 INFO:teuthology.orchestra.run.smithi125.stdout: "repo_digests": [ 2024-02-17T11:47:45.021 INFO:teuthology.orchestra.run.smithi125.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d" 2024-02-17T11:47:45.021 INFO:teuthology.orchestra.run.smithi125.stdout: ] 2024-02-17T11:47:45.021 INFO:teuthology.orchestra.run.smithi125.stdout:} 2024-02-17T11:47:45.045 DEBUG:teuthology.orchestra.run.smithi094:> sudo mkdir -p /etc/ceph 2024-02-17T11:47:45.081 DEBUG:teuthology.orchestra.run.smithi125:> sudo mkdir -p /etc/ceph 2024-02-17T11:47:45.110 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph 2024-02-17T11:47:45.146 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 777 /etc/ceph 2024-02-17T11:47:45.176 DEBUG:teuthology.orchestra.run.smithi125:> sudo chmod 777 /etc/ceph 2024-02-17T11:47:45.203 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /etc/ceph 2024-02-17T11:47:45.235 INFO:tasks.cephadm:Writing seed config... 2024-02-17T11:47:45.236 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-17T11:47:45.236 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-17T11:47:45.236 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-17T11:47:45.236 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-17T11:47:45.237 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-17T11:47:45.237 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-02-17T11:47:45.237 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-17T11:47:45.237 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-17T11:47:45.237 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-17T11:47:45.237 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-02-17T11:47:45.237 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-02-17T11:47:45.237 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-02-17T11:47:45.237 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-02-17T11:47:45.237 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-17T11:47:45.237 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-17T11:47:45.237 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-17T11:47:45.238 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-17T11:47:45.238 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-02-17T11:47:45.238 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-17T11:47:45.238 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-17T11:47:45.238 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-17T11:47:45.238 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-02-17T11:47:45.238 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-02-17T11:47:45.238 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-17T11:47:45.238 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-17T11:47:45.238 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-17T11:47:45.238 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-17T11:47:45.238 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-17T11:47:45.239 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-17T11:47:45.239 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-17T11:47:45.239 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-02-17T11:47:45.239 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-17T11:47:45.239 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-02-17T11:47:45.239 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-02-17T11:47:45.239 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-17T11:47:45.239 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-17T11:47:45.239 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-02-17T11:47:45.239 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-17T11:47:45.240 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-17T11:47:45.240 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-17T11:47:45.240 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-17T11:47:45.240 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-17T11:47:45.240 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-17T11:47:45.240 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-17T11:47:45.240 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-02-17T11:47:45.241 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:47:45.241 DEBUG:teuthology.orchestra.run.smithi094:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-17T11:47:45.258 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-17T11:47:45.258 DEBUG:teuthology.orchestra.run.smithi094:mon.a> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.a.service 2024-02-17T11:47:45.300 DEBUG:teuthology.orchestra.run.smithi094:mgr.x> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mgr.x.service 2024-02-17T11:47:45.343 INFO:tasks.cephadm:Bootstrapping... 2024-02-17T11:47:45.343 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d -v bootstrap --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.94 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-17T11:47:45.543 INFO:teuthology.orchestra.run.smithi094.stdout:-------------------------------------------------------------------------------- 2024-02-17T11:47:45.543 INFO:teuthology.orchestra.run.smithi094.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d', '-v', 'bootstrap', '--fsid', '3bc5cfc8-cd8a-11ee-95ba-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.94', '--skip-admin-label'] 2024-02-17T11:47:45.568 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-17T11:47:45.569 INFO:teuthology.orchestra.run.smithi094.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-17T11:47:45.569 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying podman|docker is present... 2024-02-17T11:47:45.593 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-17T11:47:45.593 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying lvm2 is present... 2024-02-17T11:47:45.593 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying time synchronization is in place... 2024-02-17T11:47:45.603 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-17T11:47:45.604 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-17T11:47:45.607 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-17T11:47:45.607 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-02-17T11:47:45.614 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-02-17T11:47:45.620 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-02-17T11:47:45.620 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-02-17T11:47:45.621 INFO:teuthology.orchestra.run.smithi094.stdout:Repeating the final host check... 2024-02-17T11:47:45.644 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-17T11:47:45.644 INFO:teuthology.orchestra.run.smithi094.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-17T11:47:45.644 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl is present 2024-02-17T11:47:45.644 INFO:teuthology.orchestra.run.smithi094.stdout:lvcreate is present 2024-02-17T11:47:45.652 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-17T11:47:45.652 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-17T11:47:45.659 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-17T11:47:45.660 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-02-17T11:47:45.667 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout enabled 2024-02-17T11:47:45.675 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout active 2024-02-17T11:47:45.675 INFO:teuthology.orchestra.run.smithi094.stdout:Unit chronyd.service is enabled and running 2024-02-17T11:47:45.675 INFO:teuthology.orchestra.run.smithi094.stdout:Host looks OK 2024-02-17T11:47:45.675 INFO:teuthology.orchestra.run.smithi094.stdout:Cluster fsid: 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:47:45.675 INFO:teuthology.orchestra.run.smithi094.stdout:Acquiring lock 139804357178992 on /run/cephadm/3bc5cfc8-cd8a-11ee-95ba-87774f69a715.lock 2024-02-17T11:47:45.675 INFO:teuthology.orchestra.run.smithi094.stdout:Lock 139804357178992 acquired on /run/cephadm/3bc5cfc8-cd8a-11ee-95ba-87774f69a715.lock 2024-02-17T11:47:45.676 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 3300 ... 2024-02-17T11:47:45.676 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying IP 172.21.15.94 port 6789 ... 2024-02-17T11:47:45.676 INFO:teuthology.orchestra.run.smithi094.stdout:Base mon IP(s) is [172.21.15.94:3300, 172.21.15.94:6789], mon addrv is [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-02-17T11:47:45.679 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.94 metric 100 2024-02-17T11:47:45.679 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.94 metric 100 2024-02-17T11:47:45.682 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-17T11:47:45.682 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-17T11:47:45.682 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1773sec hoplimit 64 pref medium 2024-02-17T11:47:45.684 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-17T11:47:45.684 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-17T11:47:45.685 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-17T11:47:45.685 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-17T11:47:45.685 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-02-17T11:47:45.685 INFO:teuthology.orchestra.run.smithi094.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-17T11:47:45.685 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-02-17T11:47:45.685 INFO:teuthology.orchestra.run.smithi094.stdout:Mon IP `172.21.15.94` is in CIDR network `172.21.0.0/20` 2024-02-17T11:47:45.685 INFO:teuthology.orchestra.run.smithi094.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-17T11:47:45.686 INFO:teuthology.orchestra.run.smithi094.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-17T11:47:45.687 INFO:teuthology.orchestra.run.smithi094.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d... 2024-02-17T11:47:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stdout 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd 2024-02-17T11:47:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d... 2024-02-17T11:47:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Getting image source signatures 2024-02-17T11:47:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:f0670f204a477bfeca6489528d255a0313ae7bf2a9ae6454558a9981c6019cc7 2024-02-17T11:47:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying blob sha256:0f642e61a11afb42c0ccb5b05f5809250ad643ed143780d6ff49a5a4c492fc38 2024-02-17T11:47:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Copying config sha256:32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd 2024-02-17T11:47:46.165 INFO:teuthology.orchestra.run.smithi094.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-17T11:47:47.112 INFO:teuthology.orchestra.run.smithi094.stdout:ceph: stdout ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T11:47:47.112 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph version: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T11:47:47.112 INFO:teuthology.orchestra.run.smithi094.stdout:Extracting ceph user uid/gid from container image... 2024-02-17T11:47:48.960 INFO:teuthology.orchestra.run.smithi094.stdout:stat: stdout 167 167 2024-02-17T11:47:48.960 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial keys... 2024-02-17T11:47:58.702 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQDonNBlqyIjAxAA3yOXL/wmCvexwEewpK4dgw== 2024-02-17T11:48:00.170 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQDvnNBlObfJIhAAxc+PaULtK/bHW/uQpJHguA== 2024-02-17T11:48:01.050 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph-authtool: stdout AQDwnNBlC3kqHxAAircJ83gUZMzyLmZDo3hwcA== 2024-02-17T11:48:01.051 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial monmap... 2024-02-17T11:48:02.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-17T11:48:02.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-17T11:48:02.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:48:02.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-17T11:48:02.016 INFO:teuthology.orchestra.run.smithi094.stdout:monmaptool for a [v2:172.21.15.94:3300,v1:172.21.15.94:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-17T11:48:02.016 INFO:teuthology.orchestra.run.smithi094.stdout:setting min_mon_release = quincy 2024-02-17T11:48:02.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: set fsid to 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:48:02.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-17T11:48:02.016 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:48:02.016 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mon... 2024-02-17T11:48:02.904 INFO:teuthology.orchestra.run.smithi094.stdout:create mon.a on 2024-02-17T11:48:03.098 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-17T11:48:03.260 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-17T11:48:03.455 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715.target. 2024-02-17T11:48:03.456 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715.target → /etc/systemd/system/ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715.target. 2024-02-17T11:48:03.721 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.a 2024-02-17T11:48:03.721 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.a.service: Unit ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.a.service not loaded. 2024-02-17T11:48:03.892 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715.target.wants/ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.a.service → /etc/systemd/system/ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@.service. 2024-02-17T11:48:04.416 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:04 smithi094 podman[32113]: 2024-02-17 11:48:04.046087355 +0000 UTC m=+0.021946869 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T11:48:04.993 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-17T11:48:04.994 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-02-17T11:48:05.000 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-17T11:48:05.000 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-02-17T11:48:05.000 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-02-17T11:48:05.001 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-02-17T11:48:05.001 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon to start... 2024-02-17T11:48:05.001 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mon... 2024-02-17T11:48:05.125 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:04 smithi094 podman[32113]: 2024-02-17 11:48:04.738068089 +0000 UTC m=+0.713927589 container create 771d9a120b47aab3b1babbe8f48fc8dafe957c3a531186c5cd122e4605f157c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, org.label-schema.license=GPLv2) 2024-02-17T11:48:05.125 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:04 smithi094 podman[32113]: 2024-02-17 11:48:04.915137412 +0000 UTC m=+0.890996912 container init 771d9a120b47aab3b1babbe8f48fc8dafe957c3a531186c5cd122e4605f157c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T11:48:05.125 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:04 smithi094 podman[32113]: 2024-02-17 11:48:04.918015082 +0000 UTC m=+0.893874583 container start 771d9a120b47aab3b1babbe8f48fc8dafe957c3a531186c5cd122e4605f157c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS) 2024-02-17T11:48:05.125 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:04 smithi094 bash[32113]: 771d9a120b47aab3b1babbe8f48fc8dafe957c3a531186c5cd122e4605f157c3 2024-02-17T11:48:05.126 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:04 smithi094 ceph-mon[32133]: mkfs 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:48:05.126 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:04 smithi094 ceph-mon[32133]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-17T11:48:05.126 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:04 smithi094 systemd[1]: Started Ceph mon.a for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:48:06.131 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout cluster: 2024-02-17T11:48:06.131 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout id: 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:48:06.131 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-17T11:48:06.131 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-17T11:48:06.131 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout services: 2024-02-17T11:48:06.131 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.529334s) 2024-02-17T11:48:06.131 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-17T11:48:06.131 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-17T11:48:06.131 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-17T11:48:06.132 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout data: 2024-02-17T11:48:06.132 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-17T11:48:06.132 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-17T11:48:06.132 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-17T11:48:06.132 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout pgs: 2024-02-17T11:48:06.132 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-17T11:48:06.132 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.431+0000 7f7b69a7d700 1 Processor -- start 2024-02-17T11:48:06.132 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.431+0000 7f7b69a7d700 1 -- start start 2024-02-17T11:48:06.133 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.431+0000 7f7b69a7d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 0x7f7b6410a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:06.133 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.431+0000 7f7b69a7d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6407adf0 con 0x7f7b64108120 2024-02-17T11:48:06.133 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.432+0000 7f7b637fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 0x7f7b6410a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:06.133 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.432+0000 7f7b637fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 0x7f7b6410a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39544/0 (socket says 172.21.15.94:39544) 2024-02-17T11:48:06.133 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.432+0000 7f7b637fe700 1 -- 172.21.15.94:0/341776633 learned_addr learned my addr 172.21.15.94:0/341776633 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:06.133 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.432+0000 7f7b637fe700 1 -- 172.21.15.94:0/341776633 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6407af30 con 0x7f7b64108120 2024-02-17T11:48:06.133 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.432+0000 7f7b637fe700 1 --2- 172.21.15.94:0/341776633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 0x7f7b6410a510 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009e80 tx=0x7f7b5400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad96b5d8cca15afa server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:06.133 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.432+0000 7f7b627fc700 1 -- 172.21.15.94:0/341776633 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b5400e070 con 0x7f7b64108120 2024-02-17T11:48:06.133 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.432+0000 7f7b627fc700 1 -- 172.21.15.94:0/341776633 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b54003a20 con 0x7f7b64108120 2024-02-17T11:48:06.134 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.432+0000 7f7b627fc700 1 -- 172.21.15.94:0/341776633 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b54003ce0 con 0x7f7b64108120 2024-02-17T11:48:06.134 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.433+0000 7f7b69a7d700 1 -- 172.21.15.94:0/341776633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 msgr2=0x7f7b6410a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:06.134 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.433+0000 7f7b69a7d700 1 --2- 172.21.15.94:0/341776633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 0x7f7b6410a510 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009e80 tx=0x7f7b5400b330 comp rx=0 tx=0).stop 2024-02-17T11:48:06.134 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.433+0000 7f7b69a7d700 1 -- 172.21.15.94:0/341776633 shutdown_connections 2024-02-17T11:48:06.134 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.433+0000 7f7b69a7d700 1 --2- 172.21.15.94:0/341776633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 0x7f7b6410a510 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:06.134 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.433+0000 7f7b69a7d700 1 -- 172.21.15.94:0/341776633 >> 172.21.15.94:0/341776633 conn(0x7f7b64101ad0 msgr2=0x7f7b64103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:06.134 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.433+0000 7f7b69a7d700 1 -- 172.21.15.94:0/341776633 shutdown_connections 2024-02-17T11:48:06.134 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.433+0000 7f7b69a7d700 1 -- 172.21.15.94:0/341776633 wait complete. 2024-02-17T11:48:06.134 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.433+0000 7f7b69a7d700 1 Processor -- start 2024-02-17T11:48:06.134 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.434+0000 7f7b69a7d700 1 -- start start 2024-02-17T11:48:06.134 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.434+0000 7f7b69a7d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 0x7f7b641079a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:06.135 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.434+0000 7f7b69a7d700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64107ee0 con 0x7f7b64108120 2024-02-17T11:48:06.135 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.434+0000 7f7b637fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 0x7f7b641079a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:06.135 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.434+0000 7f7b637fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 0x7f7b641079a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39548/0 (socket says 172.21.15.94:39548) 2024-02-17T11:48:06.135 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.434+0000 7f7b637fe700 1 -- 172.21.15.94:0/683381096 learned_addr learned my addr 172.21.15.94:0/683381096 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:06.135 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.434+0000 7f7b637fe700 1 -- 172.21.15.94:0/683381096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b54009b30 con 0x7f7b64108120 2024-02-17T11:48:06.135 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.434+0000 7f7b637fe700 1 --2- 172.21.15.94:0/683381096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 0x7f7b641079a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b6407a230 tx=0x7f7b54004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:06.135 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.434+0000 7f7b60ff9700 1 -- 172.21.15.94:0/683381096 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b54004370 con 0x7f7b64108120 2024-02-17T11:48:06.135 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.434+0000 7f7b60ff9700 1 -- 172.21.15.94:0/683381096 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b54004510 con 0x7f7b64108120 2024-02-17T11:48:06.135 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.434+0000 7f7b69a7d700 1 -- 172.21.15.94:0/683381096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b641060d0 con 0x7f7b64108120 2024-02-17T11:48:06.135 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.435+0000 7f7b60ff9700 1 -- 172.21.15.94:0/683381096 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b540047f0 con 0x7f7b64108120 2024-02-17T11:48:06.135 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.435+0000 7f7b69a7d700 1 -- 172.21.15.94:0/683381096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b64106570 con 0x7f7b64108120 2024-02-17T11:48:06.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.435+0000 7f7b60ff9700 1 -- 172.21.15.94:0/683381096 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7b540127c0 con 0x7f7b64108120 2024-02-17T11:48:06.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.435+0000 7f7b60ff9700 1 -- 172.21.15.94:0/683381096 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7b5401de00 con 0x7f7b64108120 2024-02-17T11:48:06.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.435+0000 7f7b69a7d700 1 -- 172.21.15.94:0/683381096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b640632a0 con 0x7f7b64108120 2024-02-17T11:48:06.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.438+0000 7f7b60ff9700 1 -- 172.21.15.94:0/683381096 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7b5402b060 con 0x7f7b64108120 2024-02-17T11:48:06.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.507+0000 7f7b69a7d700 1 -- 172.21.15.94:0/683381096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f7b6410ac30 con 0x7f7b64108120 2024-02-17T11:48:06.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.507+0000 7f7b60ff9700 1 -- 172.21.15.94:0/683381096 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f7b54021030 con 0x7f7b64108120 2024-02-17T11:48:06.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.508+0000 7f7b69a7d700 1 -- 172.21.15.94:0/683381096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 msgr2=0x7f7b641079a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:06.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.508+0000 7f7b69a7d700 1 --2- 172.21.15.94:0/683381096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 0x7f7b641079a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b6407a230 tx=0x7f7b54004ad0 comp rx=0 tx=0).stop 2024-02-17T11:48:06.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.508+0000 7f7b69a7d700 1 -- 172.21.15.94:0/683381096 shutdown_connections 2024-02-17T11:48:06.136 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.508+0000 7f7b69a7d700 1 --2- 172.21.15.94:0/683381096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b64108120 0x7f7b641079a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:06.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.508+0000 7f7b69a7d700 1 -- 172.21.15.94:0/683381096 >> 172.21.15.94:0/683381096 conn(0x7f7b64101ad0 msgr2=0x7f7b64103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:06.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.508+0000 7f7b69a7d700 1 -- 172.21.15.94:0/683381096 shutdown_connections 2024-02-17T11:48:06.137 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:05.508+0000 7f7b69a7d700 1 -- 172.21.15.94:0/683381096 wait complete. 2024-02-17T11:48:06.137 INFO:teuthology.orchestra.run.smithi094.stdout:mon is available 2024-02-17T11:48:06.137 INFO:teuthology.orchestra.run.smithi094.stdout:Assimilating anything we can from ceph.conf... 2024-02-17T11:48:06.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:05 smithi094 ceph-mon[32133]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-17T11:48:06.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:05 smithi094 ceph-mon[32133]: monmap e1: 1 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T11:48:06.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:05 smithi094 ceph-mon[32133]: fsmap 2024-02-17T11:48:06.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:05 smithi094 ceph-mon[32133]: osdmap e1: 0 total, 0 up, 0 in 2024-02-17T11:48:06.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:05 smithi094 ceph-mon[32133]: mgrmap e1: no daemons active 2024-02-17T11:48:06.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:05 smithi094 ceph-mon[32133]: from='client.? 172.21.15.94:0/683381096' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-17T11:48:07.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:06 smithi094 ceph-mon[32133]: from='client.? 172.21.15.94:0/4119870421' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-17T11:48:07.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:06 smithi094 ceph-mon[32133]: from='client.? 172.21.15.94:0/4119870421' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-17T11:48:07.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-17T11:48:07.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-02-17T11:48:07.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:48:07.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.94:3300,v1:172.21.15.94:6789] 2024-02-17T11:48:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-17T11:48:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-17T11:48:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-17T11:48:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-17T11:48:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-17T11:48:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-17T11:48:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-17T11:48:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-17T11:48:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-02-17T11:48:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-17T11:48:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-17T11:48:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-17T11:48:07.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-17T11:48:07.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-17T11:48:07.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.554+0000 7f2b5e8fa700 1 Processor -- start 2024-02-17T11:48:07.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.554+0000 7f2b5e8fa700 1 -- start start 2024-02-17T11:48:07.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.554+0000 7f2b5e8fa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 0x7f2b5810a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:07.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.554+0000 7f2b5e8fa700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5807ae70 con 0x7f2b581080a0 2024-02-17T11:48:07.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.555+0000 7f2b57fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 0x7f2b5810a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:07.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.555+0000 7f2b57fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 0x7f2b5810a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39556/0 (socket says 172.21.15.94:39556) 2024-02-17T11:48:07.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.555+0000 7f2b57fff700 1 -- 172.21.15.94:0/471113956 learned_addr learned my addr 172.21.15.94:0/471113956 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:07.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.555+0000 7f2b57fff700 1 -- 172.21.15.94:0/471113956 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5810a9d0 con 0x7f2b581080a0 2024-02-17T11:48:07.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.555+0000 7f2b57fff700 1 --2- 172.21.15.94:0/471113956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 0x7f2b5810a490 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2b480098a0 tx=0x7f2b48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8307ab22cccd781 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:07.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.555+0000 7f2b56ffd700 1 -- 172.21.15.94:0/471113956 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b48004160 con 0x7f2b581080a0 2024-02-17T11:48:07.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.555+0000 7f2b56ffd700 1 -- 172.21.15.94:0/471113956 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b480042c0 con 0x7f2b581080a0 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.555+0000 7f2b56ffd700 1 -- 172.21.15.94:0/471113956 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b480045a0 con 0x7f2b581080a0 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.556+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/471113956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 msgr2=0x7f2b5810a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.556+0000 7f2b5e8fa700 1 --2- 172.21.15.94:0/471113956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 0x7f2b5810a490 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2b480098a0 tx=0x7f2b48009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.556+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/471113956 shutdown_connections 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.556+0000 7f2b5e8fa700 1 --2- 172.21.15.94:0/471113956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 0x7f2b5810a490 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.556+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/471113956 >> 172.21.15.94:0/471113956 conn(0x7f2b58101ad0 msgr2=0x7f2b58103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.556+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/471113956 shutdown_connections 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.556+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/471113956 wait complete. 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.556+0000 7f2b5e8fa700 1 Processor -- start 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.556+0000 7f2b5e8fa700 1 -- start start 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.557+0000 7f2b5e8fa700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 0x7f2b5819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.557+0000 7f2b5e8fa700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5819b0c0 con 0x7f2b581080a0 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.557+0000 7f2b57fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 0x7f2b5819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:07.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.557+0000 7f2b57fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 0x7f2b5819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39562/0 (socket says 172.21.15.94:39562) 2024-02-17T11:48:07.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.557+0000 7f2b57fff700 1 -- 172.21.15.94:0/4119870421 learned_addr learned my addr 172.21.15.94:0/4119870421 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:07.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.557+0000 7f2b57fff700 1 -- 172.21.15.94:0/4119870421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b48009550 con 0x7f2b581080a0 2024-02-17T11:48:07.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.557+0000 7f2b57fff700 1 --2- 172.21.15.94:0/4119870421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 0x7f2b5819ab80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2b5807a2b0 tx=0x7f2b48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:07.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.557+0000 7f2b557fa700 1 -- 172.21.15.94:0/4119870421 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b48011630 con 0x7f2b581080a0 2024-02-17T11:48:07.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.557+0000 7f2b557fa700 1 -- 172.21.15.94:0/4119870421 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b48011790 con 0x7f2b581080a0 2024-02-17T11:48:07.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.557+0000 7f2b557fa700 1 -- 172.21.15.94:0/4119870421 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b48011a30 con 0x7f2b581080a0 2024-02-17T11:48:07.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.557+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/4119870421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5819b2c0 con 0x7f2b581080a0 2024-02-17T11:48:07.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.557+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/4119870421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5819b760 con 0x7f2b581080a0 2024-02-17T11:48:07.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.558+0000 7f2b557fa700 1 -- 172.21.15.94:0/4119870421 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2b48021070 con 0x7f2b581080a0 2024-02-17T11:48:07.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.558+0000 7f2b557fa700 1 -- 172.21.15.94:0/4119870421 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2b4801b7d0 con 0x7f2b581080a0 2024-02-17T11:48:07.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.558+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/4119870421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b581948d0 con 0x7f2b581080a0 2024-02-17T11:48:07.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.561+0000 7f2b557fa700 1 -- 172.21.15.94:0/4119870421 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2b4801ba30 con 0x7f2b581080a0 2024-02-17T11:48:07.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.627+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/4119870421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f2b5810ac20 con 0x7f2b581080a0 2024-02-17T11:48:07.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.630+0000 7f2b557fa700 1 -- 172.21.15.94:0/4119870421 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f2b4801bce0 con 0x7f2b581080a0 2024-02-17T11:48:07.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.631+0000 7f2b557fa700 1 -- 172.21.15.94:0/4119870421 <== mon.0 v2:172.21.15.94:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b4801be80 con 0x7f2b581080a0 2024-02-17T11:48:07.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.632+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/4119870421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 msgr2=0x7f2b5819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:07.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.632+0000 7f2b5e8fa700 1 --2- 172.21.15.94:0/4119870421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 0x7f2b5819ab80 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2b5807a2b0 tx=0x7f2b48004750 comp rx=0 tx=0).stop 2024-02-17T11:48:07.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.632+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/4119870421 shutdown_connections 2024-02-17T11:48:07.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.632+0000 7f2b5e8fa700 1 --2- 172.21.15.94:0/4119870421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b581080a0 0x7f2b5819ab80 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:07.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.632+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/4119870421 >> 172.21.15.94:0/4119870421 conn(0x7f2b58101ad0 msgr2=0x7f2b58103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:07.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.632+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/4119870421 shutdown_connections 2024-02-17T11:48:07.440 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:06.632+0000 7f2b5e8fa700 1 -- 172.21.15.94:0/4119870421 wait complete. 2024-02-17T11:48:07.440 INFO:teuthology.orchestra.run.smithi094.stdout:Generating new minimal ceph.conf... 2024-02-17T11:48:08.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:07 smithi094 ceph-mon[32133]: from='client.? 172.21.15.94:0/1089735222' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:48:08.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.880+0000 7f9dc43c5700 1 Processor -- start 2024-02-17T11:48:08.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.880+0000 7f9dc43c5700 1 -- start start 2024-02-17T11:48:08.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.881+0000 7f9dc43c5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 0x7f9dbc107a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:08.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.881+0000 7f9dc43c5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc108030 con 0x7f9dbc107640 2024-02-17T11:48:08.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.881+0000 7f9dc2161700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 0x7f9dbc107a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:08.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.881+0000 7f9dc2161700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 0x7f9dbc107a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39574/0 (socket says 172.21.15.94:39574) 2024-02-17T11:48:08.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.881+0000 7f9dc2161700 1 -- 172.21.15.94:0/1130328629 learned_addr learned my addr 172.21.15.94:0/1130328629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:08.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.881+0000 7f9dc2161700 1 -- 172.21.15.94:0/1130328629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dbc1087f0 con 0x7f9dbc107640 2024-02-17T11:48:08.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.881+0000 7f9dc2161700 1 --2- 172.21.15.94:0/1130328629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 0x7f9dbc107a60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0098a0 tx=0x7f9dac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cdbd2cb748118b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:08.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.882+0000 7f9dc115f700 1 -- 172.21.15.94:0/1130328629 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dac004160 con 0x7f9dbc107640 2024-02-17T11:48:08.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.882+0000 7f9dc115f700 1 -- 172.21.15.94:0/1130328629 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9dac0042c0 con 0x7f9dbc107640 2024-02-17T11:48:08.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.882+0000 7f9dc115f700 1 -- 172.21.15.94:0/1130328629 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dac011ac0 con 0x7f9dbc107640 2024-02-17T11:48:08.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.882+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1130328629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 msgr2=0x7f9dbc107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:08.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.882+0000 7f9dc43c5700 1 --2- 172.21.15.94:0/1130328629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 0x7f9dbc107a60 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9dac0098a0 tx=0x7f9dac009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:08.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.882+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1130328629 shutdown_connections 2024-02-17T11:48:08.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.882+0000 7f9dc43c5700 1 --2- 172.21.15.94:0/1130328629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 0x7f9dbc107a60 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:08.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.882+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1130328629 >> 172.21.15.94:0/1130328629 conn(0x7f9dbc102ca0 msgr2=0x7f9dbc1050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:08.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.882+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1130328629 shutdown_connections 2024-02-17T11:48:08.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.882+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1130328629 wait complete. 2024-02-17T11:48:08.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.883+0000 7f9dc43c5700 1 Processor -- start 2024-02-17T11:48:08.482 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.883+0000 7f9dc43c5700 1 -- start start 2024-02-17T11:48:08.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.883+0000 7f9dc43c5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 0x7f9dbc19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:08.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.883+0000 7f9dc43c5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dbc19b970 con 0x7f9dbc107640 2024-02-17T11:48:08.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.883+0000 7f9dc2161700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 0x7f9dbc19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:08.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.883+0000 7f9dc2161700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 0x7f9dbc19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39590/0 (socket says 172.21.15.94:39590) 2024-02-17T11:48:08.483 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.883+0000 7f9dc2161700 1 -- 172.21.15.94:0/1089735222 learned_addr learned my addr 172.21.15.94:0/1089735222 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:08.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.883+0000 7f9dc2161700 1 -- 172.21.15.94:0/1089735222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac009550 con 0x7f9dbc107640 2024-02-17T11:48:08.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.883+0000 7f9dc2161700 1 --2- 172.21.15.94:0/1089735222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 0x7f9dbc19b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9dac011780 tx=0x7f9dac011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:08.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.884+0000 7f9db37fe700 1 -- 172.21.15.94:0/1089735222 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dac003af0 con 0x7f9dbc107640 2024-02-17T11:48:08.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.884+0000 7f9db37fe700 1 -- 172.21.15.94:0/1089735222 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9dac003c50 con 0x7f9dbc107640 2024-02-17T11:48:08.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.884+0000 7f9db37fe700 1 -- 172.21.15.94:0/1089735222 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dac0197e0 con 0x7f9dbc107640 2024-02-17T11:48:08.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.884+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1089735222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dbc19bb70 con 0x7f9dbc107640 2024-02-17T11:48:08.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.884+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1089735222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dbc19c010 con 0x7f9dbc107640 2024-02-17T11:48:08.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.884+0000 7f9db37fe700 1 -- 172.21.15.94:0/1089735222 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9dac025460 con 0x7f9dbc107640 2024-02-17T11:48:08.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.884+0000 7f9db37fe700 1 -- 172.21.15.94:0/1089735222 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9dac00bb70 con 0x7f9dbc107640 2024-02-17T11:48:08.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.885+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1089735222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dbc10ba20 con 0x7f9dbc107640 2024-02-17T11:48:08.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.887+0000 7f9db37fe700 1 -- 172.21.15.94:0/1089735222 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9dac02a060 con 0x7f9dbc107640 2024-02-17T11:48:08.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.952+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1089735222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f9dbc10bc30 con 0x7f9dbc107640 2024-02-17T11:48:08.484 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.953+0000 7f9db37fe700 1 -- 172.21.15.94:0/1089735222 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f9dac019db0 con 0x7f9dbc107640 2024-02-17T11:48:08.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.954+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1089735222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 msgr2=0x7f9dbc19b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:08.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.954+0000 7f9dc43c5700 1 --2- 172.21.15.94:0/1089735222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 0x7f9dbc19b430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9dac011780 tx=0x7f9dac011b40 comp rx=0 tx=0).stop 2024-02-17T11:48:08.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.954+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1089735222 shutdown_connections 2024-02-17T11:48:08.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.954+0000 7f9dc43c5700 1 --2- 172.21.15.94:0/1089735222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9dbc107640 0x7f9dbc19b430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:08.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.954+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1089735222 >> 172.21.15.94:0/1089735222 conn(0x7f9dbc102ca0 msgr2=0x7f9dbc1037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:08.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.954+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1089735222 shutdown_connections 2024-02-17T11:48:08.485 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:07.954+0000 7f9dc43c5700 1 -- 172.21.15.94:0/1089735222 wait complete. 2024-02-17T11:48:08.485 INFO:teuthology.orchestra.run.smithi094.stdout:Restarting the monitor... 2024-02-17T11:48:08.761 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:08 smithi094 systemd[1]: Stopping Ceph mon.a for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:48:09.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:08 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a[32129]: 2024-02-17T11:48:08.761+0000 7f0bb0764700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T11:48:09.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:08 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a[32129]: 2024-02-17T11:48:08.761+0000 7f0bb0764700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-17T11:48:09.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:08 smithi094 podman[32375]: 2024-02-17 11:48:08.986584429 +0000 UTC m=+0.339045125 container died 771d9a120b47aab3b1babbe8f48fc8dafe957c3a531186c5cd122e4605f157c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T11:48:09.576 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:09 smithi094 podman[32375]: 2024-02-17 11:48:09.277738536 +0000 UTC m=+0.630199231 container cleanup 771d9a120b47aab3b1babbe8f48fc8dafe957c3a531186c5cd122e4605f157c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-17T11:48:09.576 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:09 smithi094 bash[32375]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a 2024-02-17T11:48:09.576 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:09 smithi094 podman[32389]: 2024-02-17 11:48:09.466018142 +0000 UTC m=+0.483058711 container remove 771d9a120b47aab3b1babbe8f48fc8dafe957c3a531186c5cd122e4605f157c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-17T11:48:09.908 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:09 smithi094 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-17T11:48:09.908 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:09 smithi094 systemd[1]: Stopped Ceph mon.a for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:48:09.909 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:09 smithi094 systemd[1]: Starting Ceph mon.a for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:48:09.909 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:09 smithi094 podman[32481]: 2024-02-17 11:48:09.704036108 +0000 UTC m=+0.017161055 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T11:48:09.909 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:09 smithi094 podman[32481]: 2024-02-17 11:48:09.80420736 +0000 UTC m=+0.117332334 container create fb2eb803b897b54c521527681de37aa41d30c103a5357cf24c5a3722d0440580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a, ceph=True, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-17T11:48:10.076 INFO:teuthology.orchestra.run.smithi094.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-17T11:48:10.199 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 podman[32481]: 2024-02-17 11:48:10.006248862 +0000 UTC m=+0.319373792 container init fb2eb803b897b54c521527681de37aa41d30c103a5357cf24c5a3722d0440580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a, RELEASE=main-6822c7b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-17T11:48:10.200 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 podman[32481]: 2024-02-17 11:48:10.009090472 +0000 UTC m=+0.322215406 container start fb2eb803b897b54c521527681de37aa41d30c103a5357cf24c5a3722d0440580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=main-6822c7b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T11:48:10.200 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: set uid:gid to 167:167 (ceph:ceph) 2024-02-17T11:48:10.200 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev), process ceph-mon, pid 2 2024-02-17T11:48:10.200 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: pidfile_write: ignore empty --pid-file 2024-02-17T11:48:10.200 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: load: jerasure load: lrc 2024-02-17T11:48:10.200 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: RocksDB version: 7.9.2 2024-02-17T11:48:10.200 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Git sha 0 2024-02-17T11:48:10.200 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Compile date 2024-02-16 17:52:22 2024-02-17T11:48:10.200 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: DB SUMMARY 2024-02-17T11:48:10.201 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: DB Session ID: 6GL4WWBKP9L307YIK4BR 2024-02-17T11:48:10.201 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: CURRENT file: CURRENT 2024-02-17T11:48:10.201 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: IDENTITY file: IDENTITY 2024-02-17T11:48:10.201 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-17T11:48:10.201 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-17T11:48:10.201 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100326 ; 2024-02-17T11:48:10.201 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.error_if_exists: 0 2024-02-17T11:48:10.201 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.create_if_missing: 0 2024-02-17T11:48:10.202 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.paranoid_checks: 1 2024-02-17T11:48:10.202 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-17T11:48:10.202 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-17T11:48:10.202 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-17T11:48:10.202 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.env: 0x557e724c0ae0 2024-02-17T11:48:10.202 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.fs: PosixFileSystem 2024-02-17T11:48:10.202 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.info_log: 0x557e736498c0 2024-02-17T11:48:10.202 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_file_opening_threads: 16 2024-02-17T11:48:10.202 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.statistics: (nil) 2024-02-17T11:48:10.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.use_fsync: 0 2024-02-17T11:48:10.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_log_file_size: 0 2024-02-17T11:48:10.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-17T11:48:10.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-17T11:48:10.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.keep_log_file_num: 1000 2024-02-17T11:48:10.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.recycle_log_file_num: 0 2024-02-17T11:48:10.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.allow_fallocate: 1 2024-02-17T11:48:10.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.allow_mmap_reads: 0 2024-02-17T11:48:10.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.allow_mmap_writes: 0 2024-02-17T11:48:10.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.use_direct_reads: 0 2024-02-17T11:48:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-17T11:48:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.create_missing_column_families: 0 2024-02-17T11:48:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.db_log_dir: 2024-02-17T11:48:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.wal_dir: 2024-02-17T11:48:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-17T11:48:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-17T11:48:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-17T11:48:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-17T11:48:10.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-17T11:48:10.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-17T11:48:10.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.advise_random_on_open: 1 2024-02-17T11:48:10.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.db_write_buffer_size: 0 2024-02-17T11:48:10.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.write_buffer_manager: 0x557e7361d400 2024-02-17T11:48:10.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-17T11:48:10.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-17T11:48:10.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-17T11:48:10.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.rate_limiter: (nil) 2024-02-17T11:48:10.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-17T11:48:10.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.wal_recovery_mode: 2 2024-02-17T11:48:10.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.enable_thread_tracking: 0 2024-02-17T11:48:10.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.enable_pipelined_write: 0 2024-02-17T11:48:10.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.unordered_write: 0 2024-02-17T11:48:10.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-17T11:48:10.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-17T11:48:10.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-17T11:48:10.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-17T11:48:10.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.row_cache: None 2024-02-17T11:48:10.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.wal_filter: None 2024-02-17T11:48:10.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-17T11:48:10.210 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.allow_ingest_behind: 0 2024-02-17T11:48:10.210 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.two_write_queues: 0 2024-02-17T11:48:10.210 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.manual_wal_flush: 0 2024-02-17T11:48:10.210 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.wal_compression: 0 2024-02-17T11:48:10.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.atomic_flush: 0 2024-02-17T11:48:10.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-17T11:48:10.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-17T11:48:10.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-17T11:48:10.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.log_readahead_size: 0 2024-02-17T11:48:10.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-17T11:48:10.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.best_efforts_recovery: 0 2024-02-17T11:48:10.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-17T11:48:10.211 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-17T11:48:10.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.allow_data_in_errors: 0 2024-02-17T11:48:10.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.db_host_id: __hostname__ 2024-02-17T11:48:10.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-17T11:48:10.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_background_jobs: 2 2024-02-17T11:48:10.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_background_compactions: -1 2024-02-17T11:48:10.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_subcompactions: 1 2024-02-17T11:48:10.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-17T11:48:10.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-17T11:48:10.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-17T11:48:10.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_total_wal_size: 0 2024-02-17T11:48:10.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-17T11:48:10.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-17T11:48:10.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-17T11:48:10.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-17T11:48:10.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_open_files: -1 2024-02-17T11:48:10.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.bytes_per_sync: 0 2024-02-17T11:48:10.213 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-17T11:48:10.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-17T11:48:10.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compaction_readahead_size: 0 2024-02-17T11:48:10.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_background_flushes: -1 2024-02-17T11:48:10.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Compression algorithms supported: 2024-02-17T11:48:10.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-17T11:48:10.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: kZSTD supported: 0 2024-02-17T11:48:10.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: kXpressCompression supported: 0 2024-02-17T11:48:10.214 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: kLZ4HCCompression supported: 1 2024-02-17T11:48:10.215 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: kZlibCompression supported: 1 2024-02-17T11:48:10.215 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: kSnappyCompression supported: 1 2024-02-17T11:48:10.215 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: kLZ4Compression supported: 1 2024-02-17T11:48:10.215 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: kBZip2Compression supported: 0 2024-02-17T11:48:10.215 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-17T11:48:10.215 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-17T11:48:10.215 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-17T11:48:10.215 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-17T11:48:10.215 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-17T11:48:10.216 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.merge_operator: 2024-02-17T11:48:10.217 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compaction_filter: None 2024-02-17T11:48:10.217 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compaction_filter_factory: None 2024-02-17T11:48:10.217 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.sst_partitioner_factory: None 2024-02-17T11:48:10.217 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-17T11:48:10.217 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-17T11:48:10.217 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557e736499c0) 2024-02-17T11:48:10.217 INFO:journalctl@ceph.mon.a.smithi094.stdout: cache_index_and_filter_blocks: 1 2024-02-17T11:48:10.217 INFO:journalctl@ceph.mon.a.smithi094.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-17T11:48:10.217 INFO:journalctl@ceph.mon.a.smithi094.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-17T11:48:10.218 INFO:journalctl@ceph.mon.a.smithi094.stdout: pin_top_level_index_and_filter: 1 2024-02-17T11:48:10.218 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_type: 0 2024-02-17T11:48:10.218 INFO:journalctl@ceph.mon.a.smithi094.stdout: data_block_index_type: 0 2024-02-17T11:48:10.218 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_shortening: 1 2024-02-17T11:48:10.218 INFO:journalctl@ceph.mon.a.smithi094.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-17T11:48:10.218 INFO:journalctl@ceph.mon.a.smithi094.stdout: checksum: 4 2024-02-17T11:48:10.218 INFO:journalctl@ceph.mon.a.smithi094.stdout: no_block_cache: 0 2024-02-17T11:48:10.218 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache: 0x557e7364d090 2024-02-17T11:48:10.218 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_name: BinnedLRUCache 2024-02-17T11:48:10.218 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_options: 2024-02-17T11:48:10.218 INFO:journalctl@ceph.mon.a.smithi094.stdout: capacity : 536870912 2024-02-17T11:48:10.219 INFO:journalctl@ceph.mon.a.smithi094.stdout: num_shard_bits : 4 2024-02-17T11:48:10.219 INFO:journalctl@ceph.mon.a.smithi094.stdout: strict_capacity_limit : 0 2024-02-17T11:48:10.219 INFO:journalctl@ceph.mon.a.smithi094.stdout: high_pri_pool_ratio: 0.000 2024-02-17T11:48:10.219 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_cache_compressed: (nil) 2024-02-17T11:48:10.219 INFO:journalctl@ceph.mon.a.smithi094.stdout: persistent_cache: (nil) 2024-02-17T11:48:10.219 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_size: 4096 2024-02-17T11:48:10.219 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_size_deviation: 10 2024-02-17T11:48:10.219 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_restart_interval: 16 2024-02-17T11:48:10.219 INFO:journalctl@ceph.mon.a.smithi094.stdout: index_block_restart_interval: 1 2024-02-17T11:48:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: metadata_block_size: 4096 2024-02-17T11:48:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: partition_filters: 0 2024-02-17T11:48:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: use_delta_encoding: 1 2024-02-17T11:48:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: filter_policy: bloomfilter 2024-02-17T11:48:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: whole_key_filtering: 1 2024-02-17T11:48:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: verify_compression: 0 2024-02-17T11:48:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: read_amp_bytes_per_bit: 0 2024-02-17T11:48:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: format_version: 5 2024-02-17T11:48:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: enable_index_compression: 1 2024-02-17T11:48:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: block_align: 0 2024-02-17T11:48:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: max_auto_readahead_size: 262144 2024-02-17T11:48:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: prepopulate_block_cache: 0 2024-02-17T11:48:10.220 INFO:journalctl@ceph.mon.a.smithi094.stdout: initial_auto_readahead_size: 8192 2024-02-17T11:48:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout: num_file_reads_for_auto_readahead: 2 2024-02-17T11:48:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.write_buffer_size: 33554432 2024-02-17T11:48:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_write_buffer_number: 2 2024-02-17T11:48:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compression: NoCompression 2024-02-17T11:48:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.bottommost_compression: Disabled 2024-02-17T11:48:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.prefix_extractor: nullptr 2024-02-17T11:48:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-17T11:48:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.num_levels: 7 2024-02-17T11:48:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-17T11:48:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-17T11:48:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-17T11:48:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-17T11:48:10.221 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-17T11:48:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-17T11:48:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-17T11:48:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-17T11:48:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-17T11:48:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-17T11:48:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-17T11:48:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-17T11:48:10.222 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compression_opts.level: 32767 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compression_opts.strategy: 0 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compression_opts.enabled: false 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.target_file_size_base: 67108864 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-17T11:48:10.223 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-17T11:48:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-17T11:48:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-17T11:48:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-17T11:48:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-17T11:48:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-17T11:48:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-17T11:48:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-17T11:48:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-17T11:48:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-17T11:48:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-17T11:48:10.224 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-17T11:48:10.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.arena_block_size: 1048576 2024-02-17T11:48:10.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-17T11:48:10.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-17T11:48:10.225 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.disable_auto_compactions: 0 2024-02-17T11:48:10.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-17T11:48:10.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-17T11:48:10.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-17T11:48:10.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-17T11:48:10.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-17T11:48:10.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-17T11:48:10.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-17T11:48:10.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-17T11:48:10.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-17T11:48:10.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-17T11:48:10.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.table_properties_collectors: 2024-02-17T11:48:10.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.inplace_update_support: 0 2024-02-17T11:48:10.226 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-17T11:48:10.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-17T11:48:10.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-17T11:48:10.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-17T11:48:10.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.bloom_locality: 0 2024-02-17T11:48:10.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.max_successive_merges: 0 2024-02-17T11:48:10.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-17T11:48:10.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.paranoid_file_checks: 0 2024-02-17T11:48:10.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.force_consistency_checks: 1 2024-02-17T11:48:10.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.report_bg_io_stats: 0 2024-02-17T11:48:10.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.ttl: 2592000 2024-02-17T11:48:10.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-17T11:48:10.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-17T11:48:10.227 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-17T11:48:10.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.enable_blob_files: false 2024-02-17T11:48:10.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.min_blob_size: 0 2024-02-17T11:48:10.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.blob_file_size: 268435456 2024-02-17T11:48:10.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-17T11:48:10.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-17T11:48:10.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-17T11:48:10.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-17T11:48:10.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-17T11:48:10.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.blob_file_starting_level: 0 2024-02-17T11:48:10.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-17T11:48:10.228 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-17T11:48:10.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-17T11:48:10.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 03675e8d-1aa6-4c68-bfba-d79ec9367355 2024-02-17T11:48:10.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708170490038232, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-17T11:48:10.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-17T11:48:10.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708170490039606, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95455, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 318, "table_properties": {"data_size": 93441, "index_size": 296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13912, "raw_average_key_size": 50, "raw_value_size": 85885, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708170490, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "03675e8d-1aa6-4c68-bfba-d79ec9367355", "db_session_id": "6GL4WWBKP9L307YIK4BR", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-17T11:48:10.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708170490039714, "job": 1, "event": "recovery_finished"} 2024-02-17T11:48:10.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-17T11:48:10.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-17T11:48:10.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557e7373e000 2024-02-17T11:48:10.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: DB pointer 0x557e73728000 2024-02-17T11:48:10.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-17T11:48:10.229 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-17T11:48:10.230 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** DB Stats ** 2024-02-17T11:48:10.230 INFO:journalctl@ceph.mon.a.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-17T11:48:10.230 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-17T11:48:10.230 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-17T11:48:10.230 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-17T11:48:10.230 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-17T11:48:10.231 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-17T11:48:10.231 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-17T11:48:10.231 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-02-17T11:48:10.231 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** Compaction Stats [default] ** 2024-02-17T11:48:10.231 INFO:journalctl@ceph.mon.a.smithi094.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-17T11:48:10.231 INFO:journalctl@ceph.mon.a.smithi094.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-17T11:48:10.231 INFO:journalctl@ceph.mon.a.smithi094.stdout: L0 2/0 95.05 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-17T11:48:10.231 INFO:journalctl@ceph.mon.a.smithi094.stdout: Sum 2/0 95.05 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-17T11:48:10.231 INFO:journalctl@ceph.mon.a.smithi094.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 112.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-17T11:48:10.231 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-02-17T11:48:10.231 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** Compaction Stats [default] ** 2024-02-17T11:48:10.232 INFO:journalctl@ceph.mon.a.smithi094.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-17T11:48:10.232 INFO:journalctl@ceph.mon.a.smithi094.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-17T11:48:10.232 INFO:journalctl@ceph.mon.a.smithi094.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 112.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-17T11:48:10.232 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-02-17T11:48:10.232 INFO:journalctl@ceph.mon.a.smithi094.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-17T11:48:10.232 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-02-17T11:48:10.232 INFO:journalctl@ceph.mon.a.smithi094.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-17T11:48:10.232 INFO:journalctl@ceph.mon.a.smithi094.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-17T11:48:10.232 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-17T11:48:10.232 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-17T11:48:10.232 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-17T11:48:10.233 INFO:journalctl@ceph.mon.a.smithi094.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-17T11:48:10.233 INFO:journalctl@ceph.mon.a.smithi094.stdout: Cumulative compaction: 0.00 GB write, 22.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-17T11:48:10.233 INFO:journalctl@ceph.mon.a.smithi094.stdout: Interval compaction: 0.00 GB write, 22.27 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-17T11:48:10.233 INFO:journalctl@ceph.mon.a.smithi094.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-17T11:48:10.234 INFO:journalctl@ceph.mon.a.smithi094.stdout: Block cache BinnedLRUCache@0x557e7364d090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-17T11:48:10.234 INFO:journalctl@ceph.mon.a.smithi094.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-02-17T11:48:10.234 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-02-17T11:48:10.234 INFO:journalctl@ceph.mon.a.smithi094.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-17T11:48:10.234 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: starting mon.a rank 0 at public addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] at bind addrs [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:48:10.234 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: mon.a@-1(???) e1 preinit fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:48:10.234 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: mon.a@-1(???).mds e1 new map 2024-02-17T11:48:10.234 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: mon.a@-1(???).mds e1 print_map 2024-02-17T11:48:10.234 INFO:journalctl@ceph.mon.a.smithi094.stdout: e1 2024-02-17T11:48:10.234 INFO:journalctl@ceph.mon.a.smithi094.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-17T11:48:10.234 INFO:journalctl@ceph.mon.a.smithi094.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-17T11:48:10.234 INFO:journalctl@ceph.mon.a.smithi094.stdout: legacy client fscid: -1 2024-02-17T11:48:10.235 INFO:journalctl@ceph.mon.a.smithi094.stdout: 2024-02-17T11:48:10.235 INFO:journalctl@ceph.mon.a.smithi094.stdout: No filesystems configured 2024-02-17T11:48:10.235 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-17T11:48:10.235 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T11:48:10.235 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T11:48:10.235 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T11:48:10.235 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-17T11:48:10.235 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 bash[32481]: fb2eb803b897b54c521527681de37aa41d30c103a5357cf24c5a3722d0440580 2024-02-17T11:48:10.235 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 systemd[1]: Started Ceph mon.a for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:48:10.235 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-17T11:48:10.235 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: monmap e1: 1 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T11:48:10.235 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: fsmap 2024-02-17T11:48:10.236 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: osdmap e1: 0 total, 0 up, 0 in 2024-02-17T11:48:10.236 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:10 smithi094 ceph-mon[32499]: mgrmap e1: no daemons active 2024-02-17T11:48:11.209 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.543+0000 7f33d93f3700 1 Processor -- start 2024-02-17T11:48:11.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.544+0000 7f33d93f3700 1 -- start start 2024-02-17T11:48:11.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.544+0000 7f33d93f3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 0x7f33d4106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:11.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.544+0000 7f33d93f3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d4106ee0 con 0x7f33d41064f0 2024-02-17T11:48:11.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.544+0000 7f33d2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 0x7f33d4106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:11.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.544+0000 7f33d2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 0x7f33d4106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39604/0 (socket says 172.21.15.94:39604) 2024-02-17T11:48:11.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.544+0000 7f33d2ffd700 1 -- 172.21.15.94:0/3225315196 learned_addr learned my addr 172.21.15.94:0/3225315196 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:11.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.544+0000 7f33d2ffd700 1 -- 172.21.15.94:0/3225315196 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d41076f0 con 0x7f33d41064f0 2024-02-17T11:48:11.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.544+0000 7f33d2ffd700 1 --2- 172.21.15.94:0/3225315196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 0x7f33d4106910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f33c80098a0 tx=0x7f33c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416b551e55963e5f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:11.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.545+0000 7f33d1ffb700 1 -- 172.21.15.94:0/3225315196 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33c8004160 con 0x7f33d41064f0 2024-02-17T11:48:11.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.545+0000 7f33d1ffb700 1 -- 172.21.15.94:0/3225315196 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33c80042c0 con 0x7f33d41064f0 2024-02-17T11:48:11.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.545+0000 7f33d1ffb700 1 -- 172.21.15.94:0/3225315196 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33c8011ac0 con 0x7f33d41064f0 2024-02-17T11:48:11.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.545+0000 7f33d93f3700 1 -- 172.21.15.94:0/3225315196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 msgr2=0x7f33d4106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:11.210 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.545+0000 7f33d93f3700 1 --2- 172.21.15.94:0/3225315196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 0x7f33d4106910 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f33c80098a0 tx=0x7f33c8009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.545+0000 7f33d93f3700 1 -- 172.21.15.94:0/3225315196 shutdown_connections 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.545+0000 7f33d93f3700 1 --2- 172.21.15.94:0/3225315196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 0x7f33d4106910 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.545+0000 7f33d93f3700 1 -- 172.21.15.94:0/3225315196 >> 172.21.15.94:0/3225315196 conn(0x7f33d4101ad0 msgr2=0x7f33d4103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.545+0000 7f33d93f3700 1 -- 172.21.15.94:0/3225315196 shutdown_connections 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.545+0000 7f33d93f3700 1 -- 172.21.15.94:0/3225315196 wait complete. 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.546+0000 7f33d93f3700 1 Processor -- start 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.546+0000 7f33d93f3700 1 -- start start 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.546+0000 7f33d93f3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 0x7f33d407aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.546+0000 7f33d93f3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d407afb0 con 0x7f33d41064f0 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.546+0000 7f33d2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 0x7f33d407aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.546+0000 7f33d2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 0x7f33d407aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39612/0 (socket says 172.21.15.94:39612) 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.546+0000 7f33d2ffd700 1 -- 172.21.15.94:0/640086541 learned_addr learned my addr 172.21.15.94:0/640086541 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.546+0000 7f33d2ffd700 1 -- 172.21.15.94:0/640086541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c8009550 con 0x7f33d41064f0 2024-02-17T11:48:11.211 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.547+0000 7f33d2ffd700 1 --2- 172.21.15.94:0/640086541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 0x7f33d407aa70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f33c8011780 tx=0x7f33c8011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.547+0000 7f33bbfff700 1 -- 172.21.15.94:0/640086541 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33c8003af0 con 0x7f33d41064f0 2024-02-17T11:48:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.547+0000 7f33bbfff700 1 -- 172.21.15.94:0/640086541 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33c8003c50 con 0x7f33d41064f0 2024-02-17T11:48:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.547+0000 7f33d93f3700 1 -- 172.21.15.94:0/640086541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d419e1b0 con 0x7f33d41064f0 2024-02-17T11:48:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.547+0000 7f33bbfff700 1 -- 172.21.15.94:0/640086541 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33c80197e0 con 0x7f33d41064f0 2024-02-17T11:48:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.547+0000 7f33d93f3700 1 -- 172.21.15.94:0/640086541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d419e650 con 0x7f33d41064f0 2024-02-17T11:48:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.547+0000 7f33bbfff700 1 -- 172.21.15.94:0/640086541 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f33c8025460 con 0x7f33d41064f0 2024-02-17T11:48:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.547+0000 7f33bbfff700 1 -- 172.21.15.94:0/640086541 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f33c800bb70 con 0x7f33d41064f0 2024-02-17T11:48:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.548+0000 7f33d93f3700 1 -- 172.21.15.94:0/640086541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d410a920 con 0x7f33d41064f0 2024-02-17T11:48:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.550+0000 7f33bbfff700 1 -- 172.21.15.94:0/640086541 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f33c802a060 con 0x7f33d41064f0 2024-02-17T11:48:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.617+0000 7f33d93f3700 1 -- 172.21.15.94:0/640086541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f33d419e890 con 0x7f33d41064f0 2024-02-17T11:48:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.619+0000 7f33bbfff700 1 -- 172.21.15.94:0/640086541 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f33c8019db0 con 0x7f33d41064f0 2024-02-17T11:48:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.619+0000 7f33d93f3700 1 -- 172.21.15.94:0/640086541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 msgr2=0x7f33d407aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:11.212 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.619+0000 7f33d93f3700 1 --2- 172.21.15.94:0/640086541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 0x7f33d407aa70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f33c8011780 tx=0x7f33c8011b40 comp rx=0 tx=0).stop 2024-02-17T11:48:11.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.619+0000 7f33d93f3700 1 -- 172.21.15.94:0/640086541 shutdown_connections 2024-02-17T11:48:11.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.619+0000 7f33d93f3700 1 --2- 172.21.15.94:0/640086541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d41064f0 0x7f33d407aa70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:11.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.619+0000 7f33d93f3700 1 -- 172.21.15.94:0/640086541 >> 172.21.15.94:0/640086541 conn(0x7f33d4101ad0 msgr2=0x7f33d4102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:11.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.620+0000 7f33d93f3700 1 -- 172.21.15.94:0/640086541 shutdown_connections 2024-02-17T11:48:11.213 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:10.620+0000 7f33d93f3700 1 -- 172.21.15.94:0/640086541 wait complete. 2024-02-17T11:48:11.213 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-17T11:48:11.228 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-17T11:48:11.228 INFO:teuthology.orchestra.run.smithi094.stdout:Creating mgr... 2024-02-17T11:48:11.228 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-17T11:48:11.228 INFO:teuthology.orchestra.run.smithi094.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-17T11:48:11.474 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mgr.x 2024-02-17T11:48:11.474 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mgr.x.service: Unit ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mgr.x.service not loaded. 2024-02-17T11:48:11.617 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715.target.wants/ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@.service. 2024-02-17T11:48:11.855 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:11 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/640086541' entity='client.admin' 2024-02-17T11:48:11.855 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:11 smithi094 systemd[1]: Starting Ceph mgr.x for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:48:12.140 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:11 smithi094 podman[32714]: 2024-02-17 11:48:11.856263274 +0000 UTC m=+0.110691375 container create 452c06f58c50e0d24beb2dfd45a3b94108942deeeaf99056fe52494bdff76a3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True) 2024-02-17T11:48:12.140 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:11 smithi094 podman[32714]: 2024-02-17 11:48:11.763298786 +0000 UTC m=+0.017726824 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d 2024-02-17T11:48:12.140 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:12 smithi094 podman[32714]: 2024-02-17 11:48:12.058337686 +0000 UTC m=+0.312765724 container init 452c06f58c50e0d24beb2dfd45a3b94108942deeeaf99056fe52494bdff76a3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-02-17T11:48:12.140 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:12 smithi094 podman[32714]: 2024-02-17 11:48:12.060774542 +0000 UTC m=+0.315202576 container start 452c06f58c50e0d24beb2dfd45a3b94108942deeeaf99056fe52494bdff76a3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-17T11:48:12.140 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:12 smithi094 bash[32714]: 452c06f58c50e0d24beb2dfd45a3b94108942deeeaf99056fe52494bdff76a3c 2024-02-17T11:48:12.153 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-17T11:48:12.153 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-02-17T11:48:12.160 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-17T11:48:12.160 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-02-17T11:48:12.161 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-02-17T11:48:12.161 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to enable service . firewalld.service is not available 2024-02-17T11:48:12.169 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-17T11:48:12.169 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-02-17T11:48:12.176 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-17T11:48:12.176 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-02-17T11:48:12.176 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-02-17T11:48:12.177 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-17T11:48:12.177 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr to start... 2024-02-17T11:48:12.177 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr... 2024-02-17T11:48:12.447 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:12 smithi094 systemd[1]: Started Ceph mgr.x for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:48:12.705 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:12 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:12.609+0000 7f43315aa200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T11:48:13.024 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:12 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1035400887' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-17T11:48:13.024 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:12 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:12.706+0000 7f43315aa200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T11:48:13.335 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-17T11:48:13.335 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-17T11:48:13.335 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "3bc5cfc8-cd8a-11ee-95ba-87774f69a715", 2024-02-17T11:48:13.335 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-02-17T11:48:13.335 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-17T11:48:13.335 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-17T11:48:13.335 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-17T11:48:13.336 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:13.336 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-17T11:48:13.336 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-17T11:48:13.336 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-02-17T11:48:13.336 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-17T11:48:13.336 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-17T11:48:13.336 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-02-17T11:48:13.336 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-17T11:48:13.336 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-17T11:48:13.336 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-17T11:48:13.336 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:13.337 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-17T11:48:13.337 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-17T11:48:13.337 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:13.337 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-17T11:48:13.337 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:13.337 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-17T11:48:13.337 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-17T11:48:13.337 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-17T11:48:13.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-17T11:48:13.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-17T11:48:13.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-17T11:48:13.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:13.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-17T11:48:13.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-17T11:48:13.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-17T11:48:13.346 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-17T11:48:13.347 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-17T11:48:13.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-17T11:48:13.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-17T11:48:13.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-17T11:48:13.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-02-17T11:48:13.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-17T11:48:13.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T11:48:13.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:13.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-17T11:48:13.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:13.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-02-17T11:48:04.980699+0000", 2024-02-17T11:48:13.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T11:48:13.348 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-17T11:48:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-17T11:48:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.645+0000 7f5a5537b700 1 Processor -- start 2024-02-17T11:48:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.645+0000 7f5a5537b700 1 -- start start 2024-02-17T11:48:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.645+0000 7f5a5537b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 0x7f5a5010a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.645+0000 7f5a5537b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5007adf0 con 0x7f5a50108170 2024-02-17T11:48:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.645+0000 7f5a4effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 0x7f5a5010a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.645+0000 7f5a4effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 0x7f5a5010a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39634/0 (socket says 172.21.15.94:39634) 2024-02-17T11:48:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.645+0000 7f5a4effd700 1 -- 172.21.15.94:0/4270137949 learned_addr learned my addr 172.21.15.94:0/4270137949 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.646+0000 7f5a4effd700 1 -- 172.21.15.94:0/4270137949 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5007af30 con 0x7f5a50108170 2024-02-17T11:48:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.646+0000 7f5a4effd700 1 --2- 172.21.15.94:0/4270137949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 0x7f5a5010a560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5a400098a0 tx=0x7f5a40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd618f9caab4b06d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.646+0000 7f5a4dffb700 1 -- 172.21.15.94:0/4270137949 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a40004160 con 0x7f5a50108170 2024-02-17T11:48:13.349 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.646+0000 7f5a4dffb700 1 -- 172.21.15.94:0/4270137949 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a400042c0 con 0x7f5a50108170 2024-02-17T11:48:13.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.646+0000 7f5a4dffb700 1 -- 172.21.15.94:0/4270137949 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a40011ac0 con 0x7f5a50108170 2024-02-17T11:48:13.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.646+0000 7f5a5537b700 1 -- 172.21.15.94:0/4270137949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 msgr2=0x7f5a5010a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:13.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.646+0000 7f5a5537b700 1 --2- 172.21.15.94:0/4270137949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 0x7f5a5010a560 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5a400098a0 tx=0x7f5a40009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:13.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.647+0000 7f5a5537b700 1 -- 172.21.15.94:0/4270137949 shutdown_connections 2024-02-17T11:48:13.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.647+0000 7f5a5537b700 1 --2- 172.21.15.94:0/4270137949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 0x7f5a5010a560 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:13.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.647+0000 7f5a5537b700 1 -- 172.21.15.94:0/4270137949 >> 172.21.15.94:0/4270137949 conn(0x7f5a50101b20 msgr2=0x7f5a50103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:13.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.647+0000 7f5a5537b700 1 -- 172.21.15.94:0/4270137949 shutdown_connections 2024-02-17T11:48:13.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.647+0000 7f5a5537b700 1 -- 172.21.15.94:0/4270137949 wait complete. 2024-02-17T11:48:13.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.647+0000 7f5a5537b700 1 Processor -- start 2024-02-17T11:48:13.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.647+0000 7f5a5537b700 1 -- start start 2024-02-17T11:48:13.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.648+0000 7f5a5537b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 0x7f5a501079f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:13.350 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.648+0000 7f5a5537b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a50107f30 con 0x7f5a50108170 2024-02-17T11:48:13.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.648+0000 7f5a4effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 0x7f5a501079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:13.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.648+0000 7f5a4effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 0x7f5a501079f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39636/0 (socket says 172.21.15.94:39636) 2024-02-17T11:48:13.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.648+0000 7f5a4effd700 1 -- 172.21.15.94:0/1035400887 learned_addr learned my addr 172.21.15.94:0/1035400887 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:13.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.648+0000 7f5a4effd700 1 -- 172.21.15.94:0/1035400887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a40009550 con 0x7f5a50108170 2024-02-17T11:48:13.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.648+0000 7f5a4effd700 1 --2- 172.21.15.94:0/1035400887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 0x7f5a501079f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5a40011a70 tx=0x7f5a40011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:13.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.648+0000 7f5a37fff700 1 -- 172.21.15.94:0/1035400887 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a400039c0 con 0x7f5a50108170 2024-02-17T11:48:13.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.648+0000 7f5a5537b700 1 -- 172.21.15.94:0/1035400887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a50106120 con 0x7f5a50108170 2024-02-17T11:48:13.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.648+0000 7f5a5537b700 1 -- 172.21.15.94:0/1035400887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a501065c0 con 0x7f5a50108170 2024-02-17T11:48:13.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.648+0000 7f5a37fff700 1 -- 172.21.15.94:0/1035400887 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a40003b20 con 0x7f5a50108170 2024-02-17T11:48:13.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.648+0000 7f5a37fff700 1 -- 172.21.15.94:0/1035400887 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a40024610 con 0x7f5a50108170 2024-02-17T11:48:13.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.649+0000 7f5a37fff700 1 -- 172.21.15.94:0/1035400887 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5a40024c30 con 0x7f5a50108170 2024-02-17T11:48:13.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.649+0000 7f5a37fff700 1 -- 172.21.15.94:0/1035400887 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5a40019930 con 0x7f5a50108170 2024-02-17T11:48:13.351 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.649+0000 7f5a5537b700 1 -- 172.21.15.94:0/1035400887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3c005360 con 0x7f5a50108170 2024-02-17T11:48:13.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.651+0000 7f5a37fff700 1 -- 172.21.15.94:0/1035400887 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5a4001aa10 con 0x7f5a50108170 2024-02-17T11:48:13.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.725+0000 7f5a5537b700 1 -- 172.21.15.94:0/1035400887 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5a3c0058e0 con 0x7f5a50108170 2024-02-17T11:48:13.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.725+0000 7f5a37fff700 1 -- 172.21.15.94:0/1035400887 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f5a4001abb0 con 0x7f5a50108170 2024-02-17T11:48:13.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.726+0000 7f5a5537b700 1 -- 172.21.15.94:0/1035400887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 msgr2=0x7f5a501079f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:13.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.726+0000 7f5a5537b700 1 --2- 172.21.15.94:0/1035400887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 0x7f5a501079f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5a40011a70 tx=0x7f5a40011b50 comp rx=0 tx=0).stop 2024-02-17T11:48:13.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.726+0000 7f5a5537b700 1 -- 172.21.15.94:0/1035400887 shutdown_connections 2024-02-17T11:48:13.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.726+0000 7f5a5537b700 1 --2- 172.21.15.94:0/1035400887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5a50108170 0x7f5a501079f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:13.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.726+0000 7f5a5537b700 1 -- 172.21.15.94:0/1035400887 >> 172.21.15.94:0/1035400887 conn(0x7f5a50101b20 msgr2=0x7f5a501026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:13.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.727+0000 7f5a5537b700 1 -- 172.21.15.94:0/1035400887 shutdown_connections 2024-02-17T11:48:13.352 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:12.727+0000 7f5a5537b700 1 -- 172.21.15.94:0/1035400887 wait complete. 2024-02-17T11:48:13.352 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (1/15)... 2024-02-17T11:48:13.592 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:13 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:13.362+0000 7f43315aa200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T11:48:13.592 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:13 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:13.453+0000 7f43315aa200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T11:48:13.592 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:13 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:13.592+0000 7f43315aa200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T11:48:13.847 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:13 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:13.688+0000 7f43315aa200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T11:48:13.847 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:13 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:13.769+0000 7f43315aa200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T11:48:14.142 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:14 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:14.033+0000 7f43315aa200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T11:48:14.142 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:14 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:14.142+0000 7f43315aa200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T11:48:14.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:14 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:14.227+0000 7f43315aa200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T11:48:14.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:14 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:14.320+0000 7f43315aa200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T11:48:14.955 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:14 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:14.499+0000 7f43315aa200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T11:48:15.800 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:15 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:15.469+0000 7f43315aa200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T11:48:15.800 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:15 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:15.602+0000 7f43315aa200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T11:48:16.183 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:15 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2627765474' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-17T11:48:16.183 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:15 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:15.946+0000 7f43315aa200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T11:48:16.184 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:16 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:16.041+0000 7f43315aa200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T11:48:16.441 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:16 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:16.247+0000 7f43315aa200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:48:16.442 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:16 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:16.442+0000 7f43315aa200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T11:48:16.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-17T11:48:16.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-17T11:48:16.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "3bc5cfc8-cd8a-11ee-95ba-87774f69a715", 2024-02-17T11:48:16.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-02-17T11:48:16.466 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:16.467 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-17T11:48:16.468 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-17T11:48:16.468 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:16.468 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-17T11:48:16.468 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:16.468 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-17T11:48:16.468 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-17T11:48:16.468 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-17T11:48:16.475 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-17T11:48:16.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-17T11:48:16.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-17T11:48:16.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:16.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-17T11:48:16.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-17T11:48:16.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-17T11:48:16.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-17T11:48:16.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-17T11:48:16.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-17T11:48:16.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-17T11:48:16.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-17T11:48:16.476 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-17T11:48:16.477 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-02-17T11:48:04.980699+0000", 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.800+0000 7ff58569a700 1 Processor -- start 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.800+0000 7ff58569a700 1 -- start start 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.800+0000 7ff58569a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580108080 0x7ff58010a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.800+0000 7ff58569a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58007ae90 con 0x7ff580108080 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.800+0000 7ff57effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580108080 0x7ff58010a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:16.478 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.800+0000 7ff57effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580108080 0x7ff58010a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40270/0 (socket says 172.21.15.94:40270) 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.801+0000 7ff57effd700 1 -- 172.21.15.94:0/3601695684 learned_addr learned my addr 172.21.15.94:0/3601695684 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.801+0000 7ff57effd700 1 -- 172.21.15.94:0/3601695684 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58010a9b0 con 0x7ff580108080 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.801+0000 7ff57effd700 1 --2- 172.21.15.94:0/3601695684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580108080 0x7ff58010a470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5740098a0 tx=0x7ff574009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdaee787550c136d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.801+0000 7ff57dffb700 1 -- 172.21.15.94:0/3601695684 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff574004160 con 0x7ff580108080 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.801+0000 7ff57dffb700 1 -- 172.21.15.94:0/3601695684 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5740042c0 con 0x7ff580108080 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.801+0000 7ff57dffb700 1 -- 172.21.15.94:0/3601695684 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff574011ac0 con 0x7ff580108080 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.802+0000 7ff58569a700 1 -- 172.21.15.94:0/3601695684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580108080 msgr2=0x7ff58010a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.802+0000 7ff58569a700 1 --2- 172.21.15.94:0/3601695684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580108080 0x7ff58010a470 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff5740098a0 tx=0x7ff574009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.802+0000 7ff58569a700 1 -- 172.21.15.94:0/3601695684 shutdown_connections 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.802+0000 7ff58569a700 1 --2- 172.21.15.94:0/3601695684 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff580108080 0x7ff58010a470 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.802+0000 7ff58569a700 1 -- 172.21.15.94:0/3601695684 >> 172.21.15.94:0/3601695684 conn(0x7ff580101ad0 msgr2=0x7ff580103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.802+0000 7ff58569a700 1 -- 172.21.15.94:0/3601695684 shutdown_connections 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.802+0000 7ff58569a700 1 -- 172.21.15.94:0/3601695684 wait complete. 2024-02-17T11:48:16.479 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.802+0000 7ff58569a700 1 Processor -- start 2024-02-17T11:48:16.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.803+0000 7ff58569a700 1 -- start start 2024-02-17T11:48:16.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.803+0000 7ff58569a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58019b620 0x7ff58019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:16.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.803+0000 7ff58569a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58019bf80 con 0x7ff58019b620 2024-02-17T11:48:16.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.803+0000 7ff57effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58019b620 0x7ff58019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:16.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.803+0000 7ff57effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58019b620 0x7ff58019ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40280/0 (socket says 172.21.15.94:40280) 2024-02-17T11:48:16.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.803+0000 7ff57effd700 1 -- 172.21.15.94:0/2627765474 learned_addr learned my addr 172.21.15.94:0/2627765474 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:16.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.803+0000 7ff57effd700 1 -- 172.21.15.94:0/2627765474 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff574009550 con 0x7ff58019b620 2024-02-17T11:48:16.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.803+0000 7ff57effd700 1 --2- 172.21.15.94:0/2627765474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58019b620 0x7ff58019ba40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff574009fb0 tx=0x7ff574004240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:16.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.803+0000 7ff567fff700 1 -- 172.21.15.94:0/2627765474 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff574011b40 con 0x7ff58019b620 2024-02-17T11:48:16.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.803+0000 7ff567fff700 1 -- 172.21.15.94:0/2627765474 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff57401f070 con 0x7ff58019b620 2024-02-17T11:48:16.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.803+0000 7ff58569a700 1 -- 172.21.15.94:0/2627765474 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58019c180 con 0x7ff58019b620 2024-02-17T11:48:16.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.804+0000 7ff567fff700 1 -- 172.21.15.94:0/2627765474 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff57401ac90 con 0x7ff58019b620 2024-02-17T11:48:16.480 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.804+0000 7ff58569a700 1 -- 172.21.15.94:0/2627765474 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58019ecf0 con 0x7ff58019b620 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.804+0000 7ff567fff700 1 -- 172.21.15.94:0/2627765474 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff574021030 con 0x7ff58019b620 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.804+0000 7ff567fff700 1 -- 172.21.15.94:0/2627765474 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff57401be00 con 0x7ff58019b620 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.805+0000 7ff58569a700 1 -- 172.21.15.94:0/2627765474 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58010b7b0 con 0x7ff58019b620 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.807+0000 7ff567fff700 1 -- 172.21.15.94:0/2627765474 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff57401a4f0 con 0x7ff58019b620 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.880+0000 7ff58569a700 1 -- 172.21.15.94:0/2627765474 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff58010b9c0 con 0x7ff58019b620 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.880+0000 7ff567fff700 1 -- 172.21.15.94:0/2627765474 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7ff57401a690 con 0x7ff58019b620 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.881+0000 7ff58569a700 1 -- 172.21.15.94:0/2627765474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58019b620 msgr2=0x7ff58019ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.881+0000 7ff58569a700 1 --2- 172.21.15.94:0/2627765474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58019b620 0x7ff58019ba40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff574009fb0 tx=0x7ff574004240 comp rx=0 tx=0).stop 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.881+0000 7ff58569a700 1 -- 172.21.15.94:0/2627765474 shutdown_connections 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.881+0000 7ff58569a700 1 --2- 172.21.15.94:0/2627765474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff58019b620 0x7ff58019ba40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.881+0000 7ff58569a700 1 -- 172.21.15.94:0/2627765474 >> 172.21.15.94:0/2627765474 conn(0x7ff580101ad0 msgr2=0x7ff580102780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.881+0000 7ff58569a700 1 -- 172.21.15.94:0/2627765474 shutdown_connections 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:15.881+0000 7ff58569a700 1 -- 172.21.15.94:0/2627765474 wait complete. 2024-02-17T11:48:16.481 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (2/15)... 2024-02-17T11:48:17.008 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:16 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:16.743+0000 7f43315aa200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:48:17.008 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:17 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:17.009+0000 7f43315aa200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T11:48:17.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:17 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:17.248+0000 7f43315aa200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T11:48:18.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:18 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:18.043+0000 7f43315aa200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T11:48:18.914 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:18 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:18.611+0000 7f43315aa200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T11:48:18.914 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:18 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:18.711+0000 7f43315aa200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T11:48:19.205 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:18 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:18.938+0000 7f43315aa200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T11:48:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:19 smithi094 ceph-mon[32499]: Activating manager daemon x 2024-02-17T11:48:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:19 smithi094 ceph-mon[32499]: mgrmap e2: x(active, starting, since 0.00421735s) 2024-02-17T11:48:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:19 smithi094 ceph-mon[32499]: from='mgr.14100 172.21.15.94:0/46600955' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-17T11:48:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:19 smithi094 ceph-mon[32499]: from='mgr.14100 172.21.15.94:0/46600955' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-17T11:48:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:19 smithi094 ceph-mon[32499]: from='mgr.14100 172.21.15.94:0/46600955' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-17T11:48:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:19 smithi094 ceph-mon[32499]: from='mgr.14100 172.21.15.94:0/46600955' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:48:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:19 smithi094 ceph-mon[32499]: from='mgr.14100 172.21.15.94:0/46600955' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-17T11:48:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:19 smithi094 ceph-mon[32499]: Manager daemon x is now available 2024-02-17T11:48:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:19 smithi094 ceph-mon[32499]: from='mgr.14100 172.21.15.94:0/46600955' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-17T11:48:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:19 smithi094 ceph-mon[32499]: from='mgr.14100 172.21.15.94:0/46600955' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-17T11:48:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:19 smithi094 ceph-mon[32499]: from='mgr.14100 172.21.15.94:0/46600955' entity='mgr.x' 2024-02-17T11:48:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:19 smithi094 ceph-mon[32499]: from='mgr.14100 172.21.15.94:0/46600955' entity='mgr.x' 2024-02-17T11:48:19.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:19 smithi094 ceph-mon[32499]: from='mgr.14100 172.21.15.94:0/46600955' entity='mgr.x' 2024-02-17T11:48:19.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-17T11:48:19.638 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-17T11:48:19.639 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "3bc5cfc8-cd8a-11ee-95ba-87774f69a715", 2024-02-17T11:48:19.639 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-02-17T11:48:19.639 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-17T11:48:19.639 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-17T11:48:19.639 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-17T11:48:19.639 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:19.639 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-17T11:48:19.639 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-17T11:48:19.639 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-02-17T11:48:19.639 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-17T11:48:19.639 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-17T11:48:19.639 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-02-17T11:48:19.639 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-17T11:48:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-17T11:48:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-17T11:48:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-17T11:48:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-17T11:48:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-17T11:48:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-17T11:48:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-17T11:48:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-17T11:48:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-17T11:48:19.640 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-17T11:48:19.641 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:19.642 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-02-17T11:48:04.980699+0000", 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.933+0000 7fcfcdd83700 1 Processor -- start 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.933+0000 7fcfcdd83700 1 -- start start 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.934+0000 7fcfcdd83700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 0x7fcfc810a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.934+0000 7fcfcdd83700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc807ae90 con 0x7fcfc81080b0 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.934+0000 7fcfc77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 0x7fcfc810a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.934+0000 7fcfc77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 0x7fcfc810a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40284/0 (socket says 172.21.15.94:40284) 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.934+0000 7fcfc77fe700 1 -- 172.21.15.94:0/3812327165 learned_addr learned my addr 172.21.15.94:0/3812327165 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:19.643 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.934+0000 7fcfc77fe700 1 -- 172.21.15.94:0/3812327165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc810a9e0 con 0x7fcfc81080b0 2024-02-17T11:48:19.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.934+0000 7fcfc77fe700 1 --2- 172.21.15.94:0/3812327165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 0x7fcfc810a4a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80098a0 tx=0x7fcfb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdf31ce4d02282 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:19.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.934+0000 7fcfc67fc700 1 -- 172.21.15.94:0/3812327165 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfb8004160 con 0x7fcfc81080b0 2024-02-17T11:48:19.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.935+0000 7fcfc67fc700 1 -- 172.21.15.94:0/3812327165 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfb80042c0 con 0x7fcfc81080b0 2024-02-17T11:48:19.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.935+0000 7fcfc67fc700 1 -- 172.21.15.94:0/3812327165 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfb8011ac0 con 0x7fcfc81080b0 2024-02-17T11:48:19.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.935+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3812327165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 msgr2=0x7fcfc810a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:19.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.935+0000 7fcfcdd83700 1 --2- 172.21.15.94:0/3812327165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 0x7fcfc810a4a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcfb80098a0 tx=0x7fcfb8009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:19.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.935+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3812327165 shutdown_connections 2024-02-17T11:48:19.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.935+0000 7fcfcdd83700 1 --2- 172.21.15.94:0/3812327165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 0x7fcfc810a4a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:19.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.935+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3812327165 >> 172.21.15.94:0/3812327165 conn(0x7fcfc8101a80 msgr2=0x7fcfc8103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:19.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.935+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3812327165 shutdown_connections 2024-02-17T11:48:19.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.935+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3812327165 wait complete. 2024-02-17T11:48:19.644 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.936+0000 7fcfcdd83700 1 Processor -- start 2024-02-17T11:48:19.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.936+0000 7fcfcdd83700 1 -- start start 2024-02-17T11:48:19.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.936+0000 7fcfcdd83700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 0x7fcfc81991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:19.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.936+0000 7fcfcdd83700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc8199730 con 0x7fcfc81080b0 2024-02-17T11:48:19.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.936+0000 7fcfc77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 0x7fcfc81991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:19.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.936+0000 7fcfc77fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 0x7fcfc81991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40298/0 (socket says 172.21.15.94:40298) 2024-02-17T11:48:19.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.936+0000 7fcfc77fe700 1 -- 172.21.15.94:0/3341241113 learned_addr learned my addr 172.21.15.94:0/3341241113 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:19.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.937+0000 7fcfc77fe700 1 -- 172.21.15.94:0/3341241113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb8009550 con 0x7fcfc81080b0 2024-02-17T11:48:19.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.937+0000 7fcfc77fe700 1 --2- 172.21.15.94:0/3341241113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 0x7fcfc81991f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009fb0 tx=0x7fcfb8011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:19.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.937+0000 7fcfc4ff9700 1 -- 172.21.15.94:0/3341241113 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfb80038b0 con 0x7fcfc81080b0 2024-02-17T11:48:19.645 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.937+0000 7fcfc4ff9700 1 -- 172.21.15.94:0/3341241113 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfb8003a10 con 0x7fcfc81080b0 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.937+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3341241113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc8199930 con 0x7fcfc81080b0 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.937+0000 7fcfc4ff9700 1 -- 172.21.15.94:0/3341241113 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfb80195e0 con 0x7fcfc81080b0 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.937+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3341241113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc8199dd0 con 0x7fcfc81080b0 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.937+0000 7fcfc4ff9700 1 -- 172.21.15.94:0/3341241113 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fcfb8019c00 con 0x7fcfc81080b0 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.937+0000 7fcfc4ff9700 1 -- 172.21.15.94:0/3341241113 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fcfb8024990 con 0x7fcfc81080b0 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.938+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3341241113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc8192b10 con 0x7fcfc81080b0 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.940+0000 7fcfc4ff9700 1 -- 172.21.15.94:0/3341241113 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fcfb8003b80 con 0x7fcfc81080b0 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:18.953+0000 7fcfc4ff9700 1 -- 172.21.15.94:0/3341241113 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 2) v1 ==== 45688+0+0 (secure 0 0 0) 0x7fcfb801a9f0 con 0x7fcfc81080b0 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:19.024+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3341241113 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcfc8106970 con 0x7fcfc81080b0 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:19.025+0000 7fcfc4ff9700 1 -- 172.21.15.94:0/3341241113 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fcfb802b340 con 0x7fcfc81080b0 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:19.025+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3341241113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 msgr2=0x7fcfc81991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:19.025+0000 7fcfcdd83700 1 --2- 172.21.15.94:0/3341241113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 0x7fcfc81991f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8009fb0 tx=0x7fcfb8011b20 comp rx=0 tx=0).stop 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:19.026+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3341241113 shutdown_connections 2024-02-17T11:48:19.646 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:19.026+0000 7fcfcdd83700 1 --2- 172.21.15.94:0/3341241113 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcfc81080b0 0x7fcfc81991f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:19.647 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:19.026+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3341241113 >> 172.21.15.94:0/3341241113 conn(0x7fcfc8101a80 msgr2=0x7fcfc818fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:19.647 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:19.026+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3341241113 shutdown_connections 2024-02-17T11:48:19.647 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:19.026+0000 7fcfcdd83700 1 -- 172.21.15.94:0/3341241113 wait complete. 2024-02-17T11:48:19.647 INFO:teuthology.orchestra.run.smithi094.stdout:mgr not available, waiting (3/15)... 2024-02-17T11:48:20.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:20 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3341241113' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-17T11:48:20.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:20 smithi094 ceph-mon[32499]: mgrmap e3: x(active, since 1.00752s) 2024-02-17T11:48:22.045 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:22 smithi094 ceph-mon[32499]: mgrmap e4: x(active, since 2s) 2024-02-17T11:48:22.943 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-17T11:48:22.943 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-17T11:48:22.943 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsid": "3bc5cfc8-cd8a-11ee-95ba-87774f69a715", 2024-02-17T11:48:22.944 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "health": { 2024-02-17T11:48:22.944 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-17T11:48:22.944 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-17T11:48:22.944 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-17T11:48:22.944 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:22.944 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-17T11:48:22.944 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-17T11:48:22.944 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 0 2024-02-17T11:48:22.944 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-17T11:48:22.944 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-17T11:48:22.944 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "a" 2024-02-17T11:48:22.944 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-17T11:48:22.944 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-02-17T11:48:22.945 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-17T11:48:22.945 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:22.945 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-17T11:48:22.945 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-17T11:48:22.945 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:22.945 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-17T11:48:22.945 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:22.945 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-17T11:48:22.945 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-17T11:48:22.945 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-17T11:48:22.945 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-17T11:48:22.946 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-17T11:48:22.946 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-17T11:48:22.946 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:22.946 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-17T11:48:22.946 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-17T11:48:22.946 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-17T11:48:22.946 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-17T11:48:22.946 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-17T11:48:22.946 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-17T11:48:22.946 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-17T11:48:22.946 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-17T11:48:22.946 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-17T11:48:22.946 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "restful" 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ], 2024-02-17T11:48:22.947 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "modified": "2024-02-17T11:48:04.980699+0000", 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout }, 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.083+0000 7f5edc8e6700 1 Processor -- start 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.083+0000 7f5edc8e6700 1 -- start start 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.083+0000 7f5edc8e6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 0x7f5ed807b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.083+0000 7f5edc8e6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed807b930 con 0x7f5ed807ceb0 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.083+0000 7f5ed659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 0x7f5ed807b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:22.948 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.083+0000 7f5ed659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 0x7f5ed807b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40368/0 (socket says 172.21.15.94:40368) 2024-02-17T11:48:22.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.083+0000 7f5ed659c700 1 -- 172.21.15.94:0/200007837 learned_addr learned my addr 172.21.15.94:0/200007837 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:22.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.083+0000 7f5ed659c700 1 -- 172.21.15.94:0/200007837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed807ba70 con 0x7f5ed807ceb0 2024-02-17T11:48:22.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.084+0000 7f5ed659c700 1 --2- 172.21.15.94:0/200007837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 0x7f5ed807b360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80098a0 tx=0x7f5ec8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fa1f05f02fcd7c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:22.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.084+0000 7f5ed559a700 1 -- 172.21.15.94:0/200007837 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ec8004160 con 0x7f5ed807ceb0 2024-02-17T11:48:22.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.084+0000 7f5ed559a700 1 -- 172.21.15.94:0/200007837 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ec80042c0 con 0x7f5ed807ceb0 2024-02-17T11:48:22.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.084+0000 7f5ed559a700 1 -- 172.21.15.94:0/200007837 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ec8011ac0 con 0x7f5ed807ceb0 2024-02-17T11:48:22.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.084+0000 7f5edc8e6700 1 -- 172.21.15.94:0/200007837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 msgr2=0x7f5ed807b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:22.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.084+0000 7f5edc8e6700 1 --2- 172.21.15.94:0/200007837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 0x7f5ed807b360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80098a0 tx=0x7f5ec8009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:22.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.085+0000 7f5edc8e6700 1 -- 172.21.15.94:0/200007837 shutdown_connections 2024-02-17T11:48:22.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.085+0000 7f5edc8e6700 1 --2- 172.21.15.94:0/200007837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 0x7f5ed807b360 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:22.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.085+0000 7f5edc8e6700 1 -- 172.21.15.94:0/200007837 >> 172.21.15.94:0/200007837 conn(0x7f5ed8102ca0 msgr2=0x7f5ed81050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:22.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.085+0000 7f5edc8e6700 1 -- 172.21.15.94:0/200007837 shutdown_connections 2024-02-17T11:48:22.949 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.085+0000 7f5edc8e6700 1 -- 172.21.15.94:0/200007837 wait complete. 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.085+0000 7f5edc8e6700 1 Processor -- start 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.085+0000 7f5edc8e6700 1 -- start start 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.085+0000 7f5edc8e6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 0x7f5ed819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.085+0000 7f5edc8e6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed819db00 con 0x7f5ed807ceb0 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.086+0000 7f5ed659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 0x7f5ed819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.086+0000 7f5ed659c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 0x7f5ed819d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40370/0 (socket says 172.21.15.94:40370) 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.086+0000 7f5ed659c700 1 -- 172.21.15.94:0/729908392 learned_addr learned my addr 172.21.15.94:0/729908392 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.086+0000 7f5ed659c700 1 -- 172.21.15.94:0/729908392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec8009550 con 0x7f5ed807ceb0 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.086+0000 7f5ed659c700 1 --2- 172.21.15.94:0/729908392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 0x7f5ed819d5c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8011a70 tx=0x7f5ec8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.086+0000 7f5ec77fe700 1 -- 172.21.15.94:0/729908392 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ec80039c0 con 0x7f5ed807ceb0 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.086+0000 7f5edc8e6700 1 -- 172.21.15.94:0/729908392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed819dd00 con 0x7f5ed807ceb0 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.086+0000 7f5ec77fe700 1 -- 172.21.15.94:0/729908392 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ec8003b20 con 0x7f5ed807ceb0 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.086+0000 7f5ec77fe700 1 -- 172.21.15.94:0/729908392 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ec8024610 con 0x7f5ed807ceb0 2024-02-17T11:48:22.950 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.086+0000 7f5edc8e6700 1 -- 172.21.15.94:0/729908392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed819e1a0 con 0x7f5ed807ceb0 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.087+0000 7f5ec77fe700 1 -- 172.21.15.94:0/729908392 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f5ec802d3f0 con 0x7f5ed807ceb0 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.087+0000 7f5edc8e6700 1 -- 172.21.15.94:0/729908392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed810da30 con 0x7f5ed807ceb0 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.087+0000 7f5ec77fe700 1 --2- 172.21.15.94:0/729908392 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f5ec0038e90 0x7f5ec003b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.087+0000 7f5ec77fe700 1 -- 172.21.15.94:0/729908392 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5ec804cf10 con 0x7f5ed807ceb0 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.087+0000 7f5ed5d9b700 1 --2- 172.21.15.94:0/729908392 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f5ec0038e90 0x7f5ec003b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.088+0000 7f5ed5d9b700 1 --2- 172.21.15.94:0/729908392 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f5ec0038e90 0x7f5ec003b350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc006fd0 tx=0x7f5ecc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.092+0000 7f5ec77fe700 1 -- 172.21.15.94:0/729908392 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ec802d6a0 con 0x7f5ed807ceb0 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.366+0000 7f5edc8e6700 1 -- 172.21.15.94:0/729908392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5ed81a0f70 con 0x7f5ed807ceb0 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.366+0000 7f5ec77fe700 1 -- 172.21.15.94:0/729908392 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f5ec8010380 con 0x7f5ed807ceb0 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.368+0000 7f5edc8e6700 1 -- 172.21.15.94:0/729908392 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f5ec0038e90 msgr2=0x7f5ec003b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.368+0000 7f5edc8e6700 1 --2- 172.21.15.94:0/729908392 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f5ec0038e90 0x7f5ec003b350 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc006fd0 tx=0x7f5ecc006e40 comp rx=0 tx=0).stop 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.368+0000 7f5edc8e6700 1 -- 172.21.15.94:0/729908392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 msgr2=0x7f5ed819d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.368+0000 7f5edc8e6700 1 --2- 172.21.15.94:0/729908392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 0x7f5ed819d5c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8011a70 tx=0x7f5ec8011b50 comp rx=0 tx=0).stop 2024-02-17T11:48:22.951 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.368+0000 7f5edc8e6700 1 -- 172.21.15.94:0/729908392 shutdown_connections 2024-02-17T11:48:22.952 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.368+0000 7f5edc8e6700 1 --2- 172.21.15.94:0/729908392 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f5ec0038e90 0x7f5ec003b350 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:22.952 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.368+0000 7f5edc8e6700 1 --2- 172.21.15.94:0/729908392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5ed807ceb0 0x7f5ed819d5c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:22.952 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.368+0000 7f5edc8e6700 1 -- 172.21.15.94:0/729908392 >> 172.21.15.94:0/729908392 conn(0x7f5ed8102ca0 msgr2=0x7f5ed81037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:22.952 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.369+0000 7f5edc8e6700 1 -- 172.21.15.94:0/729908392 shutdown_connections 2024-02-17T11:48:22.952 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:22.369+0000 7f5edc8e6700 1 -- 172.21.15.94:0/729908392 wait complete. 2024-02-17T11:48:22.952 INFO:teuthology.orchestra.run.smithi094.stdout:mgr is available 2024-02-17T11:48:23.061 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:23 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/729908392' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-17T11:48:24.245 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:24 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3125309512' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-17T11:48:24.254 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-17T11:48:24.254 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [global] 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout fsid = 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout [osd] 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-17T11:48:24.255 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-17T11:48:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-17T11:48:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.403+0000 7f9383cfc700 1 Processor -- start 2024-02-17T11:48:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.403+0000 7f9383cfc700 1 -- start start 2024-02-17T11:48:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.403+0000 7f9383cfc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 0x7f937c10a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.403+0000 7f9383cfc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c07ae70 con 0x7f937c1080e0 2024-02-17T11:48:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.403+0000 7f9381a98700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 0x7f937c10a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.403+0000 7f9381a98700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 0x7f937c10a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40376/0 (socket says 172.21.15.94:40376) 2024-02-17T11:48:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.403+0000 7f9381a98700 1 -- 172.21.15.94:0/2088509524 learned_addr learned my addr 172.21.15.94:0/2088509524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.404+0000 7f9381a98700 1 -- 172.21.15.94:0/2088509524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c10aa10 con 0x7f937c1080e0 2024-02-17T11:48:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.404+0000 7f9381a98700 1 --2- 172.21.15.94:0/2088509524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 0x7f937c10a4d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f93780098a0 tx=0x7f9378009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc128b6d1acc041 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.404+0000 7f9380a96700 1 -- 172.21.15.94:0/2088509524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9378004160 con 0x7f937c1080e0 2024-02-17T11:48:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.404+0000 7f9380a96700 1 -- 172.21.15.94:0/2088509524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93780042c0 con 0x7f937c1080e0 2024-02-17T11:48:24.256 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.404+0000 7f9380a96700 1 -- 172.21.15.94:0/2088509524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9378011ac0 con 0x7f937c1080e0 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.404+0000 7f9383cfc700 1 -- 172.21.15.94:0/2088509524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 msgr2=0x7f937c10a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.404+0000 7f9383cfc700 1 --2- 172.21.15.94:0/2088509524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 0x7f937c10a4d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f93780098a0 tx=0x7f9378009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.405+0000 7f9383cfc700 1 -- 172.21.15.94:0/2088509524 shutdown_connections 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.405+0000 7f9383cfc700 1 --2- 172.21.15.94:0/2088509524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 0x7f937c10a4d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.405+0000 7f9383cfc700 1 -- 172.21.15.94:0/2088509524 >> 172.21.15.94:0/2088509524 conn(0x7f937c101ad0 msgr2=0x7f937c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.405+0000 7f9383cfc700 1 -- 172.21.15.94:0/2088509524 shutdown_connections 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.405+0000 7f9383cfc700 1 -- 172.21.15.94:0/2088509524 wait complete. 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.405+0000 7f9383cfc700 1 Processor -- start 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.405+0000 7f9383cfc700 1 -- start start 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.405+0000 7f9383cfc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 0x7f937c19b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.405+0000 7f9383cfc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c19b8d0 con 0x7f937c1080e0 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.406+0000 7f9381a98700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 0x7f937c19b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.406+0000 7f9381a98700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 0x7f937c19b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40382/0 (socket says 172.21.15.94:40382) 2024-02-17T11:48:24.257 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.406+0000 7f9381a98700 1 -- 172.21.15.94:0/3125309512 learned_addr learned my addr 172.21.15.94:0/3125309512 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.406+0000 7f9381a98700 1 -- 172.21.15.94:0/3125309512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9378009550 con 0x7f937c1080e0 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.406+0000 7f9381a98700 1 --2- 172.21.15.94:0/3125309512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 0x7f937c19b390 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9378003f70 tx=0x7f9378011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.406+0000 7f9372ffd700 1 -- 172.21.15.94:0/3125309512 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93780039f0 con 0x7f937c1080e0 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.406+0000 7f9372ffd700 1 -- 172.21.15.94:0/3125309512 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9378003b50 con 0x7f937c1080e0 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.406+0000 7f9383cfc700 1 -- 172.21.15.94:0/3125309512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937c19bad0 con 0x7f937c1080e0 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.406+0000 7f9372ffd700 1 -- 172.21.15.94:0/3125309512 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f937801a760 con 0x7f937c1080e0 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.406+0000 7f9383cfc700 1 -- 172.21.15.94:0/3125309512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937c19bf70 con 0x7f937c1080e0 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.407+0000 7f9372ffd700 1 -- 172.21.15.94:0/3125309512 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f9378024430 con 0x7f937c1080e0 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.407+0000 7f9372ffd700 1 --2- 172.21.15.94:0/3125309512 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f9368038e40 0x7f936803b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.407+0000 7f9372ffd700 1 -- 172.21.15.94:0/3125309512 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f937804d0c0 con 0x7f937c1080e0 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.407+0000 7f9383cfc700 1 -- 172.21.15.94:0/3125309512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f937c10b810 con 0x7f937c1080e0 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.407+0000 7f9381297700 1 --2- 172.21.15.94:0/3125309512 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f9368038e40 0x7f936803b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.408+0000 7f9381297700 1 --2- 172.21.15.94:0/3125309512 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f9368038e40 0x7f936803b300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f936c006fd0 tx=0x7f936c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:24.258 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.412+0000 7f9372ffd700 1 -- 172.21.15.94:0/3125309512 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9378018a40 con 0x7f937c1080e0 2024-02-17T11:48:24.259 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.601+0000 7f9383cfc700 1 -- 172.21.15.94:0/3125309512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f937c19e850 con 0x7f937c1080e0 2024-02-17T11:48:24.259 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.602+0000 7f9372ffd700 1 -- 172.21.15.94:0/3125309512 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f9378018350 con 0x7f937c1080e0 2024-02-17T11:48:24.259 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.603+0000 7f9383cfc700 1 -- 172.21.15.94:0/3125309512 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f9368038e40 msgr2=0x7f936803b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:24.259 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.603+0000 7f9383cfc700 1 --2- 172.21.15.94:0/3125309512 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f9368038e40 0x7f936803b300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f936c006fd0 tx=0x7f936c006e40 comp rx=0 tx=0).stop 2024-02-17T11:48:24.259 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.604+0000 7f9383cfc700 1 -- 172.21.15.94:0/3125309512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 msgr2=0x7f937c19b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:24.259 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.604+0000 7f9383cfc700 1 --2- 172.21.15.94:0/3125309512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 0x7f937c19b390 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9378003f70 tx=0x7f9378011b40 comp rx=0 tx=0).stop 2024-02-17T11:48:24.259 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.604+0000 7f9383cfc700 1 -- 172.21.15.94:0/3125309512 shutdown_connections 2024-02-17T11:48:24.259 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.604+0000 7f9383cfc700 1 --2- 172.21.15.94:0/3125309512 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f9368038e40 0x7f936803b300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:24.259 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.604+0000 7f9383cfc700 1 --2- 172.21.15.94:0/3125309512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f937c1080e0 0x7f937c19b390 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:24.259 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.604+0000 7f9383cfc700 1 -- 172.21.15.94:0/3125309512 >> 172.21.15.94:0/3125309512 conn(0x7f937c101ad0 msgr2=0x7f937c102660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:24.259 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.604+0000 7f9383cfc700 1 -- 172.21.15.94:0/3125309512 shutdown_connections 2024-02-17T11:48:24.259 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:23.604+0000 7f9383cfc700 1 -- 172.21.15.94:0/3125309512 wait complete. 2024-02-17T11:48:24.259 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling cephadm module... 2024-02-17T11:48:25.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:25 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2749047388' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-17T11:48:25.206 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:25 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: ignoring --setuser ceph since I am not root 2024-02-17T11:48:25.206 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:25 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: ignoring --setgroup ceph since I am not root 2024-02-17T11:48:25.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.666+0000 7f7e77b32700 1 Processor -- start 2024-02-17T11:48:25.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.666+0000 7f7e77b32700 1 -- start start 2024-02-17T11:48:25.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.666+0000 7f7e77b32700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 0x7f7e7010a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:25.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.666+0000 7f7e77b32700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7007ae90 con 0x7f7e701080d0 2024-02-17T11:48:25.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.666+0000 7f7e758ce700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 0x7f7e7010a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:25.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.666+0000 7f7e758ce700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 0x7f7e7010a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58906/0 (socket says 172.21.15.94:58906) 2024-02-17T11:48:25.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.666+0000 7f7e758ce700 1 -- 172.21.15.94:0/2141896968 learned_addr learned my addr 172.21.15.94:0/2141896968 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:25.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.667+0000 7f7e758ce700 1 -- 172.21.15.94:0/2141896968 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7010aa00 con 0x7f7e701080d0 2024-02-17T11:48:25.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.667+0000 7f7e758ce700 1 --2- 172.21.15.94:0/2141896968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 0x7f7e7010a4c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0098a0 tx=0x7f7e6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be8d501964eab65 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:25.576 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.667+0000 7f7e748cc700 1 -- 172.21.15.94:0/2141896968 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e6c004160 con 0x7f7e701080d0 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.667+0000 7f7e748cc700 1 -- 172.21.15.94:0/2141896968 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e6c0042c0 con 0x7f7e701080d0 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.667+0000 7f7e748cc700 1 -- 172.21.15.94:0/2141896968 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e6c011ac0 con 0x7f7e701080d0 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.667+0000 7f7e77b32700 1 -- 172.21.15.94:0/2141896968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 msgr2=0x7f7e7010a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.667+0000 7f7e77b32700 1 --2- 172.21.15.94:0/2141896968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 0x7f7e7010a4c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0098a0 tx=0x7f7e6c009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.667+0000 7f7e77b32700 1 -- 172.21.15.94:0/2141896968 shutdown_connections 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.667+0000 7f7e77b32700 1 --2- 172.21.15.94:0/2141896968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 0x7f7e7010a4c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.667+0000 7f7e77b32700 1 -- 172.21.15.94:0/2141896968 >> 172.21.15.94:0/2141896968 conn(0x7f7e70101b20 msgr2=0x7f7e70103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.668+0000 7f7e77b32700 1 -- 172.21.15.94:0/2141896968 shutdown_connections 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.668+0000 7f7e77b32700 1 -- 172.21.15.94:0/2141896968 wait complete. 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.668+0000 7f7e77b32700 1 Processor -- start 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.668+0000 7f7e77b32700 1 -- start start 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.668+0000 7f7e77b32700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 0x7f7e7019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.668+0000 7f7e77b32700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7019b950 con 0x7f7e701080d0 2024-02-17T11:48:25.577 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.668+0000 7f7e758ce700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 0x7f7e7019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.668+0000 7f7e758ce700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 0x7f7e7019b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58914/0 (socket says 172.21.15.94:58914) 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.668+0000 7f7e758ce700 1 -- 172.21.15.94:0/2749047388 learned_addr learned my addr 172.21.15.94:0/2749047388 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.669+0000 7f7e758ce700 1 -- 172.21.15.94:0/2749047388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c009550 con 0x7f7e701080d0 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.669+0000 7f7e758ce700 1 --2- 172.21.15.94:0/2749047388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 0x7f7e7019b410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c011a70 tx=0x7f7e6c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.669+0000 7f7e66ffd700 1 -- 172.21.15.94:0/2749047388 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e6c0039c0 con 0x7f7e701080d0 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.669+0000 7f7e66ffd700 1 -- 172.21.15.94:0/2749047388 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e6c003b20 con 0x7f7e701080d0 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.669+0000 7f7e77b32700 1 -- 172.21.15.94:0/2749047388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7019bb50 con 0x7f7e701080d0 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.669+0000 7f7e66ffd700 1 -- 172.21.15.94:0/2749047388 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e6c024610 con 0x7f7e701080d0 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.669+0000 7f7e77b32700 1 -- 172.21.15.94:0/2749047388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7019bff0 con 0x7f7e701080d0 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.670+0000 7f7e66ffd700 1 -- 172.21.15.94:0/2749047388 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f7e6c02d3f0 con 0x7f7e701080d0 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.670+0000 7f7e77b32700 1 -- 172.21.15.94:0/2749047388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7010b800 con 0x7f7e701080d0 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.670+0000 7f7e66ffd700 1 --2- 172.21.15.94:0/2749047388 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f7e5c038e90 0x7f7e5c03b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.670+0000 7f7e66ffd700 1 -- 172.21.15.94:0/2749047388 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7e6c04d0e0 con 0x7f7e701080d0 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.670+0000 7f7e750cd700 1 --2- 172.21.15.94:0/2749047388 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f7e5c038e90 0x7f7e5c03b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:25.578 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.670+0000 7f7e750cd700 1 --2- 172.21.15.94:0/2749047388 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f7e5c038e90 0x7f7e5c03b350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006fd0 tx=0x7f7e60006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.675+0000 7f7e66ffd700 1 -- 172.21.15.94:0/2749047388 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7e6c010740 con 0x7f7e701080d0 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:24.919+0000 7f7e77b32700 1 -- 172.21.15.94:0/2749047388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f7e7019edc0 con 0x7f7e701080d0 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:25.008+0000 7f7e66ffd700 1 -- 172.21.15.94:0/2749047388 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f7e6c00b3e0 con 0x7f7e701080d0 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:25.009+0000 7f7e66ffd700 1 -- 172.21.15.94:0/2749047388 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f7e6c021d60 con 0x7f7e701080d0 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:25.011+0000 7f7e77b32700 1 -- 172.21.15.94:0/2749047388 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f7e5c038e90 msgr2=0x7f7e5c03b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:25.011+0000 7f7e77b32700 1 --2- 172.21.15.94:0/2749047388 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f7e5c038e90 0x7f7e5c03b350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006fd0 tx=0x7f7e60006e40 comp rx=0 tx=0).stop 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:25.011+0000 7f7e77b32700 1 -- 172.21.15.94:0/2749047388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 msgr2=0x7f7e7019b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:25.011+0000 7f7e77b32700 1 --2- 172.21.15.94:0/2749047388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 0x7f7e7019b410 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c011a70 tx=0x7f7e6c011b50 comp rx=0 tx=0).stop 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:25.011+0000 7f7e77b32700 1 -- 172.21.15.94:0/2749047388 shutdown_connections 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:25.011+0000 7f7e77b32700 1 --2- 172.21.15.94:0/2749047388 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7f7e5c038e90 0x7f7e5c03b350 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:25.011+0000 7f7e77b32700 1 --2- 172.21.15.94:0/2749047388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e701080d0 0x7f7e7019b410 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:25.011+0000 7f7e77b32700 1 -- 172.21.15.94:0/2749047388 >> 172.21.15.94:0/2749047388 conn(0x7f7e70101b20 msgr2=0x7f7e701026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:25.011+0000 7f7e77b32700 1 -- 172.21.15.94:0/2749047388 shutdown_connections 2024-02-17T11:48:25.579 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:25.011+0000 7f7e77b32700 1 -- 172.21.15.94:0/2749047388 wait complete. 2024-02-17T11:48:25.667 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:25 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:25.562+0000 7fe5398e9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T11:48:25.667 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:25 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:25.667+0000 7fe5398e9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T11:48:26.275 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:26 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2749047388' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-17T11:48:26.275 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:26 smithi094 ceph-mon[32499]: mgrmap e5: x(active, since 6s) 2024-02-17T11:48:26.564 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:26 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:26.331+0000 7fe5398e9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T11:48:26.564 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:26 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:26.423+0000 7fe5398e9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T11:48:26.564 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:26 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:26.565+0000 7fe5398e9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T11:48:26.865 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:26 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:26.665+0000 7fe5398e9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T11:48:26.866 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:26 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:26.749+0000 7fe5398e9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T11:48:26.873 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-17T11:48:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-17T11:48:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-17T11:48:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-17T11:48:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-17T11:48:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-17T11:48:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.011+0000 7ff4a91e0700 1 Processor -- start 2024-02-17T11:48:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.011+0000 7ff4a91e0700 1 -- start start 2024-02-17T11:48:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.011+0000 7ff4a91e0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 0x7ff4a4107ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.011+0000 7ff4a91e0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a41084a0 con 0x7ff4a4107ab0 2024-02-17T11:48:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.011+0000 7ff4a2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 0x7ff4a4107ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.011+0000 7ff4a2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 0x7ff4a4107ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58932/0 (socket says 172.21.15.94:58932) 2024-02-17T11:48:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.011+0000 7ff4a2d9d700 1 -- 172.21.15.94:0/2478530224 learned_addr learned my addr 172.21.15.94:0/2478530224 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:26.874 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.012+0000 7ff4a2d9d700 1 -- 172.21.15.94:0/2478530224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a41085e0 con 0x7ff4a4107ab0 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.012+0000 7ff4a2d9d700 1 --2- 172.21.15.94:0/2478530224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 0x7ff4a4107ed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff4940098a0 tx=0x7ff494009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cbeb76466fbdce0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.012+0000 7ff4a1d9b700 1 -- 172.21.15.94:0/2478530224 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff494004160 con 0x7ff4a4107ab0 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.012+0000 7ff4a1d9b700 1 -- 172.21.15.94:0/2478530224 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4940042c0 con 0x7ff4a4107ab0 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.012+0000 7ff4a1d9b700 1 -- 172.21.15.94:0/2478530224 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff494011ac0 con 0x7ff4a4107ab0 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.012+0000 7ff4a91e0700 1 -- 172.21.15.94:0/2478530224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 msgr2=0x7ff4a4107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.012+0000 7ff4a91e0700 1 --2- 172.21.15.94:0/2478530224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 0x7ff4a4107ed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff4940098a0 tx=0x7ff494009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.013+0000 7ff4a91e0700 1 -- 172.21.15.94:0/2478530224 shutdown_connections 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.013+0000 7ff4a91e0700 1 --2- 172.21.15.94:0/2478530224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 0x7ff4a4107ed0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.013+0000 7ff4a91e0700 1 -- 172.21.15.94:0/2478530224 >> 172.21.15.94:0/2478530224 conn(0x7ff4a4101250 msgr2=0x7ff4a4103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.013+0000 7ff4a91e0700 1 -- 172.21.15.94:0/2478530224 shutdown_connections 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.013+0000 7ff4a91e0700 1 -- 172.21.15.94:0/2478530224 wait complete. 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.013+0000 7ff4a91e0700 1 Processor -- start 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.013+0000 7ff4a91e0700 1 -- start start 2024-02-17T11:48:26.875 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.014+0000 7ff4a91e0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 0x7ff4a419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.014+0000 7ff4a91e0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a419db60 con 0x7ff4a4107ab0 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.014+0000 7ff4a2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 0x7ff4a419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.014+0000 7ff4a2d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 0x7ff4a419d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58948/0 (socket says 172.21.15.94:58948) 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.014+0000 7ff4a2d9d700 1 -- 172.21.15.94:0/352164410 learned_addr learned my addr 172.21.15.94:0/352164410 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.014+0000 7ff4a2d9d700 1 -- 172.21.15.94:0/352164410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff494009550 con 0x7ff4a4107ab0 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.014+0000 7ff4a2d9d700 1 --2- 172.21.15.94:0/352164410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 0x7ff4a419d620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff494003f70 tx=0x7ff494011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.014+0000 7ff48bfff700 1 -- 172.21.15.94:0/352164410 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff4940039f0 con 0x7ff4a4107ab0 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.014+0000 7ff48bfff700 1 -- 172.21.15.94:0/352164410 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff494003b50 con 0x7ff4a4107ab0 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.014+0000 7ff4a91e0700 1 -- 172.21.15.94:0/352164410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a419dd60 con 0x7ff4a4107ab0 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.014+0000 7ff48bfff700 1 -- 172.21.15.94:0/352164410 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff49401a690 con 0x7ff4a4107ab0 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.015+0000 7ff4a91e0700 1 -- 172.21.15.94:0/352164410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a419e200 con 0x7ff4a4107ab0 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.015+0000 7ff48bfff700 1 -- 172.21.15.94:0/352164410 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7ff49401a950 con 0x7ff4a4107ab0 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.016+0000 7ff4a91e0700 1 -- 172.21.15.94:0/352164410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a4197920 con 0x7ff4a4107ab0 2024-02-17T11:48:26.876 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.016+0000 7ff48bfff700 1 --2- 172.21.15.94:0/352164410 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7ff48c038e90 0x7ff48c03b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.016+0000 7ff48bfff700 1 -- 172.21.15.94:0/352164410 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff49404cfd0 con 0x7ff4a4107ab0 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.016+0000 7ff4a259c700 1 -- 172.21.15.94:0/352164410 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7ff48c038e90 msgr2=0x7ff48c03b350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/281060900 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.016+0000 7ff4a259c700 1 --2- 172.21.15.94:0/352164410 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7ff48c038e90 0x7ff48c03b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.020+0000 7ff48bfff700 1 -- 172.21.15.94:0/352164410 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff49402d910 con 0x7ff4a4107ab0 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.217+0000 7ff4a259c700 1 -- 172.21.15.94:0/352164410 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7ff48c038e90 msgr2=0x7ff48c03b350 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/281060900 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.217+0000 7ff4a259c700 1 --2- 172.21.15.94:0/352164410 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7ff48c038e90 0x7ff48c03b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.275+0000 7ff4a91e0700 1 -- 172.21.15.94:0/352164410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ff4a41085e0 con 0x7ff4a4107ab0 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.276+0000 7ff48bfff700 1 -- 172.21.15.94:0/352164410 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7ff4a41085e0 con 0x7ff4a4107ab0 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.278+0000 7ff4a91e0700 1 -- 172.21.15.94:0/352164410 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7ff48c038e90 msgr2=0x7ff48c03b350 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.278+0000 7ff4a91e0700 1 --2- 172.21.15.94:0/352164410 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7ff48c038e90 0x7ff48c03b350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.278+0000 7ff4a91e0700 1 -- 172.21.15.94:0/352164410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 msgr2=0x7ff4a419d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.278+0000 7ff4a91e0700 1 --2- 172.21.15.94:0/352164410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 0x7ff4a419d620 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff494003f70 tx=0x7ff494011b40 comp rx=0 tx=0).stop 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.278+0000 7ff4a91e0700 1 -- 172.21.15.94:0/352164410 shutdown_connections 2024-02-17T11:48:26.877 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.278+0000 7ff4a91e0700 1 --2- 172.21.15.94:0/352164410 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7ff48c038e90 0x7ff48c03b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:26.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.278+0000 7ff4a91e0700 1 --2- 172.21.15.94:0/352164410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff4a4107ab0 0x7ff4a419d620 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:26.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.278+0000 7ff4a91e0700 1 -- 172.21.15.94:0/352164410 >> 172.21.15.94:0/352164410 conn(0x7ff4a4101250 msgr2=0x7ff4a4102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:26.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.279+0000 7ff4a91e0700 1 -- 172.21.15.94:0/352164410 shutdown_connections 2024-02-17T11:48:26.878 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:26.279+0000 7ff4a91e0700 1 -- 172.21.15.94:0/352164410 wait complete. 2024-02-17T11:48:26.878 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-02-17T11:48:26.878 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 5... 2024-02-17T11:48:27.130 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:27 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/352164410' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-17T11:48:27.130 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:27 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:27.020+0000 7fe5398e9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T11:48:27.130 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:27 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:27.130+0000 7fe5398e9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T11:48:27.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:27 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:27.223+0000 7fe5398e9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T11:48:27.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:27 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:27.324+0000 7fe5398e9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T11:48:27.902 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:27 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:27.500+0000 7fe5398e9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T11:48:28.705 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:28 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:28.454+0000 7fe5398e9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T11:48:28.705 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:28 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:28.584+0000 7fe5398e9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T11:48:29.204 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:28 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:28.917+0000 7fe5398e9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T11:48:29.204 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:29 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:29.008+0000 7fe5398e9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T11:48:29.204 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:29 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:29.203+0000 7fe5398e9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:48:29.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:29 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:29.387+0000 7fe5398e9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T11:48:29.928 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:29 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:29.672+0000 7fe5398e9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:48:29.928 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:29 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:29.927+0000 7fe5398e9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T11:48:30.205 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:30 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:30.162+0000 7fe5398e9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T11:48:31.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:30 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:30.969+0000 7fe5398e9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T11:48:31.888 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:31.570+0000 7fe5398e9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T11:48:31.889 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:31.668+0000 7fe5398e9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T11:48:31.889 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:31.889+0000 7fe5398e9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T11:48:32.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: Active manager daemon x restarted 2024-02-17T11:48:32.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: Activating manager daemon x 2024-02-17T11:48:32.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: osdmap e2: 0 total, 0 up, 0 in 2024-02-17T11:48:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: mgrmap e6: x(active, starting, since 0.00324856s) 2024-02-17T11:48:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:48:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-17T11:48:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-17T11:48:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-17T11:48:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-17T11:48:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: Manager daemon x is now available 2024-02-17T11:48:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:48:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:31 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:48:33.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:32 smithi094 ceph-mon[32499]: Found migration_current of "None". Setting to last migration. 2024-02-17T11:48:33.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:32 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:48:33.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:32 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-17T11:48:33.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:32 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-17T11:48:33.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:32 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:33.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:32 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:33.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:32 smithi094 ceph-mon[32499]: mgrmap e7: x(active, since 1.0055s) 2024-02-17T11:48:33.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-17T11:48:33.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-17T11:48:33.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-17T11:48:33.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-17T11:48:33.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.296+0000 7fc686525700 1 Processor -- start 2024-02-17T11:48:33.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.296+0000 7fc686525700 1 -- start start 2024-02-17T11:48:33.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.296+0000 7fc686525700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 0x7fc680106320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:33.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.296+0000 7fc686525700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68007ae30 con 0x7fc680105f00 2024-02-17T11:48:33.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.296+0000 7fc67ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 0x7fc680106320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:33.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.296+0000 7fc67ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 0x7fc680106320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58962/0 (socket says 172.21.15.94:58962) 2024-02-17T11:48:33.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.296+0000 7fc67ffff700 1 -- 172.21.15.94:0/1879115680 learned_addr learned my addr 172.21.15.94:0/1879115680 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:33.512 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.297+0000 7fc67ffff700 1 -- 172.21.15.94:0/1879115680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68007af70 con 0x7fc680105f00 2024-02-17T11:48:33.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.297+0000 7fc67ffff700 1 --2- 172.21.15.94:0/1879115680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 0x7fc680106320 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc6740098a0 tx=0x7fc674009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c85acb7a630c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:33.513 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.297+0000 7fc67effd700 1 -- 172.21.15.94:0/1879115680 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc674004160 con 0x7fc680105f00 2024-02-17T11:48:33.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.297+0000 7fc67effd700 1 -- 172.21.15.94:0/1879115680 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc6740042c0 con 0x7fc680105f00 2024-02-17T11:48:33.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.297+0000 7fc67effd700 1 -- 172.21.15.94:0/1879115680 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc674011ac0 con 0x7fc680105f00 2024-02-17T11:48:33.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.298+0000 7fc686525700 1 -- 172.21.15.94:0/1879115680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 msgr2=0x7fc680106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:33.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.298+0000 7fc686525700 1 --2- 172.21.15.94:0/1879115680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 0x7fc680106320 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc6740098a0 tx=0x7fc674009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:33.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.298+0000 7fc686525700 1 -- 172.21.15.94:0/1879115680 shutdown_connections 2024-02-17T11:48:33.519 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.298+0000 7fc686525700 1 --2- 172.21.15.94:0/1879115680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 0x7fc680106320 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:33.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.298+0000 7fc686525700 1 -- 172.21.15.94:0/1879115680 >> 172.21.15.94:0/1879115680 conn(0x7fc680101a80 msgr2=0x7fc680103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:33.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.298+0000 7fc686525700 1 -- 172.21.15.94:0/1879115680 shutdown_connections 2024-02-17T11:48:33.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.298+0000 7fc686525700 1 -- 172.21.15.94:0/1879115680 wait complete. 2024-02-17T11:48:33.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.299+0000 7fc686525700 1 Processor -- start 2024-02-17T11:48:33.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.299+0000 7fc686525700 1 -- start start 2024-02-17T11:48:33.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.299+0000 7fc686525700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 0x7fc68019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:33.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.299+0000 7fc686525700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68019db20 con 0x7fc680105f00 2024-02-17T11:48:33.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.299+0000 7fc67ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 0x7fc68019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:33.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.299+0000 7fc67ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 0x7fc68019d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58978/0 (socket says 172.21.15.94:58978) 2024-02-17T11:48:33.520 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.299+0000 7fc67ffff700 1 -- 172.21.15.94:0/3208467536 learned_addr learned my addr 172.21.15.94:0/3208467536 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:33.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.299+0000 7fc67ffff700 1 -- 172.21.15.94:0/3208467536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc674009550 con 0x7fc680105f00 2024-02-17T11:48:33.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.300+0000 7fc67ffff700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 0x7fc68019d5e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc674009fb0 tx=0x7fc674011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:33.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.300+0000 7fc67d7fa700 1 -- 172.21.15.94:0/3208467536 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6740038b0 con 0x7fc680105f00 2024-02-17T11:48:33.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.300+0000 7fc67d7fa700 1 -- 172.21.15.94:0/3208467536 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc674003a10 con 0x7fc680105f00 2024-02-17T11:48:33.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.300+0000 7fc67d7fa700 1 -- 172.21.15.94:0/3208467536 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6740195e0 con 0x7fc680105f00 2024-02-17T11:48:33.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.300+0000 7fc686525700 1 -- 172.21.15.94:0/3208467536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68019dd20 con 0x7fc680105f00 2024-02-17T11:48:33.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.300+0000 7fc686525700 1 -- 172.21.15.94:0/3208467536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68019e1c0 con 0x7fc680105f00 2024-02-17T11:48:33.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.301+0000 7fc67d7fa700 1 -- 172.21.15.94:0/3208467536 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fc674019850 con 0x7fc680105f00 2024-02-17T11:48:33.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.301+0000 7fc67d7fa700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 0x7fc66803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:33.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.301+0000 7fc67d7fa700 1 -- 172.21.15.94:0/3208467536 --> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc66803bab0 con 0x7fc668038ee0 2024-02-17T11:48:33.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.301+0000 7fc67d7fa700 1 -- 172.21.15.94:0/3208467536 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc67404d1e0 con 0x7fc680105f00 2024-02-17T11:48:33.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.301+0000 7fc67f7fe700 1 -- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 msgr2=0x7fc66803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/281060900 2024-02-17T11:48:33.521 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.301+0000 7fc67f7fe700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 0x7fc66803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.502+0000 7fc67f7fe700 1 -- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 msgr2=0x7fc66803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/281060900 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.502+0000 7fc67f7fe700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 0x7fc66803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.903+0000 7fc67f7fe700 1 -- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 msgr2=0x7fc66803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/281060900 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:27.903+0000 7fc67f7fe700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 0x7fc66803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:28.703+0000 7fc67f7fe700 1 -- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 msgr2=0x7fc66803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/281060900 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:28.703+0000 7fc67f7fe700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 0x7fc66803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:30.304+0000 7fc67f7fe700 1 -- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 msgr2=0x7fc66803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/281060900 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:30.304+0000 7fc67f7fe700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 0x7fc66803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:31.904+0000 7fc67d7fa700 1 -- 172.21.15.94:0/3208467536 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 6) v1 ==== 45699+0+0 (secure 0 0 0) 0x7fc67402d960 con 0x7fc680105f00 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:31.904+0000 7fc67d7fa700 1 -- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 msgr2=0x7fc66803b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:31.904+0000 7fc67d7fa700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 0x7fc66803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.905+0000 7fc67d7fa700 1 -- 172.21.15.94:0/3208467536 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7fc67404e7a0 con 0x7fc680105f00 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.905+0000 7fc67d7fa700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fc66803c7d0 0x7fc66803ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.905+0000 7fc67d7fa700 1 -- 172.21.15.94:0/3208467536 --> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc66803bab0 con 0x7fc66803c7d0 2024-02-17T11:48:33.522 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.906+0000 7fc67f7fe700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fc66803c7d0 0x7fc66803ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.907+0000 7fc67f7fe700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fc66803c7d0 0x7fc66803ebc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc670003a10 tx=0x7fc6700092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.908+0000 7fc67d7fa700 1 -- 172.21.15.94:0/3208467536 <== mgr.14120 v2:172.21.15.94:6800/2384585889 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fc66803bab0 con 0x7fc66803c7d0 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.914+0000 7fc686525700 1 -- 172.21.15.94:0/3208467536 --> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc6801075a0 con 0x7fc66803c7d0 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.914+0000 7fc67d7fa700 1 -- 172.21.15.94:0/3208467536 <== mgr.14120 v2:172.21.15.94:6800/2384585889 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc6801075a0 con 0x7fc66803c7d0 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.914+0000 7fc686525700 1 -- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fc66803c7d0 msgr2=0x7fc66803ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.914+0000 7fc686525700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fc66803c7d0 0x7fc66803ebc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc670003a10 tx=0x7fc6700092b0 comp rx=0 tx=0).stop 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.914+0000 7fc686525700 1 -- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 msgr2=0x7fc68019d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.914+0000 7fc686525700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 0x7fc68019d5e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc674009fb0 tx=0x7fc674011b20 comp rx=0 tx=0).stop 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.914+0000 7fc686525700 1 -- 172.21.15.94:0/3208467536 shutdown_connections 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.914+0000 7fc686525700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc680105f00 0x7fc68019d5e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.914+0000 7fc686525700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fc66803c7d0 0x7fc66803ebc0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.914+0000 7fc686525700 1 --2- 172.21.15.94:0/3208467536 >> [v2:172.21.15.94:6800/281060900,v1:172.21.15.94:6801/281060900] conn(0x7fc668038ee0 0x7fc66803b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.914+0000 7fc686525700 1 -- 172.21.15.94:0/3208467536 >> 172.21.15.94:0/3208467536 conn(0x7fc680101a80 msgr2=0x7fc680194410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:33.523 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.914+0000 7fc686525700 1 -- 172.21.15.94:0/3208467536 shutdown_connections 2024-02-17T11:48:33.524 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:32.914+0000 7fc686525700 1 -- 172.21.15.94:0/3208467536 wait complete. 2024-02-17T11:48:33.524 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 5 is available 2024-02-17T11:48:33.524 INFO:teuthology.orchestra.run.smithi094.stdout:Setting orchestrator backend to cephadm... 2024-02-17T11:48:34.168 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:33 smithi094 ceph-mon[32499]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-17T11:48:34.169 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:33 smithi094 ceph-mon[32499]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-17T11:48:34.169 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:33 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:48:34.743 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.949+0000 7f2d90f4e700 1 Processor -- start 2024-02-17T11:48:34.743 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.949+0000 7f2d90f4e700 1 -- start start 2024-02-17T11:48:34.743 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.949+0000 7f2d90f4e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 0x7f2d8c107f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:34.743 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.949+0000 7f2d90f4e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c1084f0 con 0x7f2d8c107b00 2024-02-17T11:48:34.743 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.949+0000 7f2d8a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 0x7f2d8c107f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.949+0000 7f2d8a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 0x7f2d8c107f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59036/0 (socket says 172.21.15.94:59036) 2024-02-17T11:48:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.949+0000 7f2d8a59c700 1 -- 172.21.15.94:0/4043347769 learned_addr learned my addr 172.21.15.94:0/4043347769 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.950+0000 7f2d8a59c700 1 -- 172.21.15.94:0/4043347769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c108630 con 0x7f2d8c107b00 2024-02-17T11:48:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.950+0000 7f2d8a59c700 1 --2- 172.21.15.94:0/4043347769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 0x7f2d8c107f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c0098a0 tx=0x7f2d7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb839323e857b81f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.950+0000 7f2d8959a700 1 -- 172.21.15.94:0/4043347769 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d7c004160 con 0x7f2d8c107b00 2024-02-17T11:48:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.950+0000 7f2d8959a700 1 -- 172.21.15.94:0/4043347769 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d7c0042c0 con 0x7f2d8c107b00 2024-02-17T11:48:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.950+0000 7f2d8959a700 1 -- 172.21.15.94:0/4043347769 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d7c011ac0 con 0x7f2d8c107b00 2024-02-17T11:48:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.950+0000 7f2d90f4e700 1 -- 172.21.15.94:0/4043347769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 msgr2=0x7f2d8c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.950+0000 7f2d90f4e700 1 --2- 172.21.15.94:0/4043347769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 0x7f2d8c107f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c0098a0 tx=0x7f2d7c009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.951+0000 7f2d90f4e700 1 -- 172.21.15.94:0/4043347769 shutdown_connections 2024-02-17T11:48:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.951+0000 7f2d90f4e700 1 --2- 172.21.15.94:0/4043347769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 0x7f2d8c107f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.951+0000 7f2d90f4e700 1 -- 172.21.15.94:0/4043347769 >> 172.21.15.94:0/4043347769 conn(0x7f2d8c1012d0 msgr2=0x7f2d8c1036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:34.744 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.951+0000 7f2d90f4e700 1 -- 172.21.15.94:0/4043347769 shutdown_connections 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.951+0000 7f2d90f4e700 1 -- 172.21.15.94:0/4043347769 wait complete. 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.951+0000 7f2d90f4e700 1 Processor -- start 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.951+0000 7f2d90f4e700 1 -- start start 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.951+0000 7f2d90f4e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 0x7f2d8c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.951+0000 7f2d90f4e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8c19db90 con 0x7f2d8c107b00 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.952+0000 7f2d8a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 0x7f2d8c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.952+0000 7f2d8a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 0x7f2d8c19d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59046/0 (socket says 172.21.15.94:59046) 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.952+0000 7f2d8a59c700 1 -- 172.21.15.94:0/3555629063 learned_addr learned my addr 172.21.15.94:0/3555629063 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.952+0000 7f2d8a59c700 1 -- 172.21.15.94:0/3555629063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c009550 con 0x7f2d8c107b00 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.952+0000 7f2d8a59c700 1 --2- 172.21.15.94:0/3555629063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 0x7f2d8c19d650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009fb0 tx=0x7f2d7c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.952+0000 7f2d7b7fe700 1 -- 172.21.15.94:0/3555629063 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d7c0038b0 con 0x7f2d8c107b00 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.952+0000 7f2d7b7fe700 1 -- 172.21.15.94:0/3555629063 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d7c003a10 con 0x7f2d8c107b00 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.952+0000 7f2d90f4e700 1 -- 172.21.15.94:0/3555629063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d8c19dd90 con 0x7f2d8c107b00 2024-02-17T11:48:34.745 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.952+0000 7f2d7b7fe700 1 -- 172.21.15.94:0/3555629063 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2d7c0195e0 con 0x7f2d8c107b00 2024-02-17T11:48:34.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.952+0000 7f2d90f4e700 1 -- 172.21.15.94:0/3555629063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8c19e230 con 0x7f2d8c107b00 2024-02-17T11:48:34.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.953+0000 7f2d7b7fe700 1 -- 172.21.15.94:0/3555629063 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f2d7c019850 con 0x7f2d8c107b00 2024-02-17T11:48:34.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.953+0000 7f2d7b7fe700 1 --2- 172.21.15.94:0/3555629063 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f2d74038dd0 0x7f2d7403b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:34.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.953+0000 7f2d7b7fe700 1 -- 172.21.15.94:0/3555629063 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2d7c04cf80 con 0x7f2d8c107b00 2024-02-17T11:48:34.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.953+0000 7f2d90f4e700 1 -- 172.21.15.94:0/3555629063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8c106300 con 0x7f2d8c107b00 2024-02-17T11:48:34.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.953+0000 7f2d89d9b700 1 --2- 172.21.15.94:0/3555629063 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f2d74038dd0 0x7f2d7403b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:34.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.954+0000 7f2d89d9b700 1 --2- 172.21.15.94:0/3555629063 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f2d74038dd0 0x7f2d7403b290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d80006fd0 tx=0x7f2d80006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:34.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.959+0000 7f2d7b7fe700 1 -- 172.21.15.94:0/3555629063 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d7c019b50 con 0x7f2d8c107b00 2024-02-17T11:48:34.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:33.960+0000 7f2d7b7fe700 1 -- 172.21.15.94:0/3555629063 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f2d7c019d30 con 0x7f2d8c107b00 2024-02-17T11:48:34.746 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:34.169+0000 7f2d90f4e700 1 -- 172.21.15.94:0/3555629063 --> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d8c1a0b10 con 0x7f2d74038dd0 2024-02-17T11:48:34.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:34.174+0000 7f2d7b7fe700 1 -- 172.21.15.94:0/3555629063 <== mgr.14120 v2:172.21.15.94:6800/2384585889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2d8c1a0b10 con 0x7f2d74038dd0 2024-02-17T11:48:34.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:34.176+0000 7f2d90f4e700 1 -- 172.21.15.94:0/3555629063 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f2d74038dd0 msgr2=0x7f2d7403b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:34.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:34.176+0000 7f2d90f4e700 1 --2- 172.21.15.94:0/3555629063 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f2d74038dd0 0x7f2d7403b290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2d80006fd0 tx=0x7f2d80006e40 comp rx=0 tx=0).stop 2024-02-17T11:48:34.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:34.176+0000 7f2d90f4e700 1 -- 172.21.15.94:0/3555629063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 msgr2=0x7f2d8c19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:34.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:34.176+0000 7f2d90f4e700 1 --2- 172.21.15.94:0/3555629063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 0x7f2d8c19d650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009fb0 tx=0x7f2d7c011b20 comp rx=0 tx=0).stop 2024-02-17T11:48:34.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:34.176+0000 7f2d90f4e700 1 -- 172.21.15.94:0/3555629063 shutdown_connections 2024-02-17T11:48:34.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:34.176+0000 7f2d90f4e700 1 --2- 172.21.15.94:0/3555629063 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f2d74038dd0 0x7f2d7403b290 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:34.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:34.176+0000 7f2d90f4e700 1 --2- 172.21.15.94:0/3555629063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8c107b00 0x7f2d8c19d650 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:34.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:34.176+0000 7f2d90f4e700 1 -- 172.21.15.94:0/3555629063 >> 172.21.15.94:0/3555629063 conn(0x7f2d8c1012d0 msgr2=0x7f2d8c193de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:34.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:34.176+0000 7f2d90f4e700 1 -- 172.21.15.94:0/3555629063 shutdown_connections 2024-02-17T11:48:34.747 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:34.176+0000 7f2d90f4e700 1 -- 172.21.15.94:0/3555629063 wait complete. 2024-02-17T11:48:35.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:34 smithi094 ceph-mon[32499]: [17/Feb/2024:11:48:33] ENGINE Bus STARTING 2024-02-17T11:48:35.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:34 smithi094 ceph-mon[32499]: [17/Feb/2024:11:48:33] ENGINE Serving on https://172.21.15.94:7150 2024-02-17T11:48:35.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:34 smithi094 ceph-mon[32499]: [17/Feb/2024:11:48:33] ENGINE Serving on http://172.21.15.94:8765 2024-02-17T11:48:35.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:34 smithi094 ceph-mon[32499]: [17/Feb/2024:11:48:33] ENGINE Bus STARTED 2024-02-17T11:48:35.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:34 smithi094 ceph-mon[32499]: mgrmap e8: x(active, since 2s) 2024-02-17T11:48:35.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:34 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:35.064 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:34 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:48:35.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-17T11:48:35.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.163+0000 7f0a955af700 1 Processor -- start 2024-02-17T11:48:35.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.163+0000 7f0a955af700 1 -- start start 2024-02-17T11:48:35.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.163+0000 7f0a955af700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 0x7f0a9007b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:35.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.163+0000 7f0a955af700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9007b8f0 con 0x7f0a9007ce70 2024-02-17T11:48:35.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.163+0000 7f0a8effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 0x7f0a9007b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.163+0000 7f0a8effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 0x7f0a9007b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54610/0 (socket says 172.21.15.94:54610) 2024-02-17T11:48:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.163+0000 7f0a8effd700 1 -- 172.21.15.94:0/2386778073 learned_addr learned my addr 172.21.15.94:0/2386778073 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.164+0000 7f0a8effd700 1 -- 172.21.15.94:0/2386778073 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9007ba30 con 0x7f0a9007ce70 2024-02-17T11:48:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.164+0000 7f0a8effd700 1 --2- 172.21.15.94:0/2386778073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 0x7f0a9007b320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0a800098a0 tx=0x7f0a80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa042afb475c107 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.164+0000 7f0a8e7fc700 1 -- 172.21.15.94:0/2386778073 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a80004160 con 0x7f0a9007ce70 2024-02-17T11:48:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.164+0000 7f0a8e7fc700 1 -- 172.21.15.94:0/2386778073 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a800042c0 con 0x7f0a9007ce70 2024-02-17T11:48:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.164+0000 7f0a8e7fc700 1 -- 172.21.15.94:0/2386778073 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a80011ac0 con 0x7f0a9007ce70 2024-02-17T11:48:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.165+0000 7f0a955af700 1 -- 172.21.15.94:0/2386778073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 msgr2=0x7f0a9007b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.165+0000 7f0a955af700 1 --2- 172.21.15.94:0/2386778073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 0x7f0a9007b320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0a800098a0 tx=0x7f0a80009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.165+0000 7f0a955af700 1 -- 172.21.15.94:0/2386778073 shutdown_connections 2024-02-17T11:48:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.165+0000 7f0a955af700 1 --2- 172.21.15.94:0/2386778073 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 0x7f0a9007b320 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.165+0000 7f0a955af700 1 -- 172.21.15.94:0/2386778073 >> 172.21.15.94:0/2386778073 conn(0x7f0a90102c80 msgr2=0x7f0a901050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:35.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.165+0000 7f0a955af700 1 -- 172.21.15.94:0/2386778073 shutdown_connections 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.165+0000 7f0a955af700 1 -- 172.21.15.94:0/2386778073 wait complete. 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.165+0000 7f0a955af700 1 Processor -- start 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.165+0000 7f0a955af700 1 -- start start 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.166+0000 7f0a955af700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 0x7f0a9019d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.166+0000 7f0a955af700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9019daa0 con 0x7f0a9007ce70 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.166+0000 7f0a8effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 0x7f0a9019d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.166+0000 7f0a8effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 0x7f0a9019d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54620/0 (socket says 172.21.15.94:54620) 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.166+0000 7f0a8effd700 1 -- 172.21.15.94:0/1171553258 learned_addr learned my addr 172.21.15.94:0/1171553258 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.166+0000 7f0a8effd700 1 -- 172.21.15.94:0/1171553258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a80009550 con 0x7f0a9007ce70 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.166+0000 7f0a8effd700 1 --2- 172.21.15.94:0/1171553258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 0x7f0a9019d560 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009130 tx=0x7f0a80011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.166+0000 7f0a8cff9700 1 -- 172.21.15.94:0/1171553258 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a80003aa0 con 0x7f0a9007ce70 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.166+0000 7f0a8cff9700 1 -- 172.21.15.94:0/1171553258 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a80003c00 con 0x7f0a9007ce70 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.166+0000 7f0a955af700 1 -- 172.21.15.94:0/1171553258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9019dca0 con 0x7f0a9007ce70 2024-02-17T11:48:35.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.166+0000 7f0a8cff9700 1 -- 172.21.15.94:0/1171553258 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a80024800 con 0x7f0a9007ce70 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.166+0000 7f0a955af700 1 -- 172.21.15.94:0/1171553258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9019e140 con 0x7f0a9007ce70 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.167+0000 7f0a8cff9700 1 -- 172.21.15.94:0/1171553258 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f0a8001a430 con 0x7f0a9007ce70 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.167+0000 7f0a8cff9700 1 --2- 172.21.15.94:0/1171553258 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f0a7c038ee0 0x7f0a7c03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.167+0000 7f0a8cff9700 1 -- 172.21.15.94:0/1171553258 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0a8004d1a0 con 0x7f0a9007ce70 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.167+0000 7f0a955af700 1 -- 172.21.15.94:0/1171553258 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9010d9d0 con 0x7f0a9007ce70 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.167+0000 7f0a867fc700 1 --2- 172.21.15.94:0/1171553258 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f0a7c038ee0 0x7f0a7c03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.168+0000 7f0a867fc700 1 --2- 172.21.15.94:0/1171553258 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f0a7c038ee0 0x7f0a7c03b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a78006fd0 tx=0x7f0a78006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.172+0000 7f0a8cff9700 1 -- 172.21.15.94:0/1171553258 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0a8001adb0 con 0x7f0a9007ce70 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.366+0000 7f0a955af700 1 -- 172.21.15.94:0/1171553258 --> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a90103ea0 con 0x7f0a7c038ee0 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.367+0000 7f0a8cff9700 1 -- 172.21.15.94:0/1171553258 <== mgr.14120 v2:172.21.15.94:6800/2384585889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f0a90103ea0 con 0x7f0a7c038ee0 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.369+0000 7f0a955af700 1 -- 172.21.15.94:0/1171553258 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f0a7c038ee0 msgr2=0x7f0a7c03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.369+0000 7f0a955af700 1 --2- 172.21.15.94:0/1171553258 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f0a7c038ee0 0x7f0a7c03b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a78006fd0 tx=0x7f0a78006e40 comp rx=0 tx=0).stop 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.369+0000 7f0a955af700 1 -- 172.21.15.94:0/1171553258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 msgr2=0x7f0a9019d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:35.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.369+0000 7f0a955af700 1 --2- 172.21.15.94:0/1171553258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 0x7f0a9019d560 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009130 tx=0x7f0a80011b40 comp rx=0 tx=0).stop 2024-02-17T11:48:35.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.369+0000 7f0a955af700 1 -- 172.21.15.94:0/1171553258 shutdown_connections 2024-02-17T11:48:35.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.369+0000 7f0a955af700 1 --2- 172.21.15.94:0/1171553258 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f0a7c038ee0 0x7f0a7c03b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:35.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.369+0000 7f0a955af700 1 --2- 172.21.15.94:0/1171553258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0a9007ce70 0x7f0a9019d560 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:35.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.369+0000 7f0a955af700 1 -- 172.21.15.94:0/1171553258 >> 172.21.15.94:0/1171553258 conn(0x7f0a90102c80 msgr2=0x7f0a90103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:35.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.369+0000 7f0a955af700 1 -- 172.21.15.94:0/1171553258 shutdown_connections 2024-02-17T11:48:35.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:35.369+0000 7f0a955af700 1 -- 172.21.15.94:0/1171553258 wait complete. 2024-02-17T11:48:35.912 INFO:teuthology.orchestra.run.smithi094.stdout:Generating ssh key... 2024-02-17T11:48:36.253 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:35 smithi094 ceph-mon[32499]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:48:36.955 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: Generating public/private rsa key pair. 2024-02-17T11:48:36.956 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: Your identification has been saved in /tmp/tmpp3d26_dq/key. 2024-02-17T11:48:36.956 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: Your public key has been saved in /tmp/tmpp3d26_dq/key.pub. 2024-02-17T11:48:36.956 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: The key fingerprint is: 2024-02-17T11:48:36.956 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: SHA256:aE8VeJkLxMeRPARdN0FRsxVJq+vssweSKKAReAoE4BQ ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:48:36.956 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: The key's randomart image is: 2024-02-17T11:48:36.956 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: +---[RSA 3072]----+ 2024-02-17T11:48:36.956 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: |*Eo ooB+*..BB=| 2024-02-17T11:48:36.956 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: |+. o + @. . o=| 2024-02-17T11:48:36.956 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: |..o . +.o o | 2024-02-17T11:48:36.956 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: | . . . . .. . | 2024-02-17T11:48:36.956 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: | o + S . .. | 2024-02-17T11:48:36.956 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: | . . + . o .. | 2024-02-17T11:48:36.956 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: | o ... | 2024-02-17T11:48:36.957 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: | o. . | 2024-02-17T11:48:36.957 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: | .=+ | 2024-02-17T11:48:36.957 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: +----[SHA256]-----+ 2024-02-17T11:48:37.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-mon[32499]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:48:37.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:37.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:36 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:37.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.359+0000 7f1b88510700 1 Processor -- start 2024-02-17T11:48:37.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.359+0000 7f1b88510700 1 -- start start 2024-02-17T11:48:37.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.359+0000 7f1b88510700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 0x7f1b8010a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:37.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.359+0000 7f1b88510700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8007ae70 con 0x7f1b801080f0 2024-02-17T11:48:37.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.359+0000 7f1b862ac700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 0x7f1b8010a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:37.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.359+0000 7f1b862ac700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 0x7f1b8010a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54624/0 (socket says 172.21.15.94:54624) 2024-02-17T11:48:37.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.359+0000 7f1b862ac700 1 -- 172.21.15.94:0/3485163717 learned_addr learned my addr 172.21.15.94:0/3485163717 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:37.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.360+0000 7f1b862ac700 1 -- 172.21.15.94:0/3485163717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8010aa20 con 0x7f1b801080f0 2024-02-17T11:48:37.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.360+0000 7f1b862ac700 1 --2- 172.21.15.94:0/3485163717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 0x7f1b8010a4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0098a0 tx=0x7f1b7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b8bf1c85358cf8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:37.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.360+0000 7f1b852aa700 1 -- 172.21.15.94:0/3485163717 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b7c004160 con 0x7f1b801080f0 2024-02-17T11:48:37.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.360+0000 7f1b852aa700 1 -- 172.21.15.94:0/3485163717 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b7c0042c0 con 0x7f1b801080f0 2024-02-17T11:48:37.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.360+0000 7f1b852aa700 1 -- 172.21.15.94:0/3485163717 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b7c011ac0 con 0x7f1b801080f0 2024-02-17T11:48:37.298 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.360+0000 7f1b88510700 1 -- 172.21.15.94:0/3485163717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 msgr2=0x7f1b8010a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:37.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.360+0000 7f1b88510700 1 --2- 172.21.15.94:0/3485163717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 0x7f1b8010a4e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c0098a0 tx=0x7f1b7c009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:37.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.361+0000 7f1b88510700 1 -- 172.21.15.94:0/3485163717 shutdown_connections 2024-02-17T11:48:37.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.361+0000 7f1b88510700 1 --2- 172.21.15.94:0/3485163717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 0x7f1b8010a4e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:37.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.361+0000 7f1b88510700 1 -- 172.21.15.94:0/3485163717 >> 172.21.15.94:0/3485163717 conn(0x7f1b80101b20 msgr2=0x7f1b80103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:37.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.361+0000 7f1b88510700 1 -- 172.21.15.94:0/3485163717 shutdown_connections 2024-02-17T11:48:37.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.361+0000 7f1b88510700 1 -- 172.21.15.94:0/3485163717 wait complete. 2024-02-17T11:48:37.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.361+0000 7f1b88510700 1 Processor -- start 2024-02-17T11:48:37.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.361+0000 7f1b88510700 1 -- start start 2024-02-17T11:48:37.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.361+0000 7f1b88510700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 0x7f1b8019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:37.299 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.361+0000 7f1b88510700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8019b940 con 0x7f1b801080f0 2024-02-17T11:48:37.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.361+0000 7f1b862ac700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 0x7f1b8019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:37.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.361+0000 7f1b862ac700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 0x7f1b8019b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54636/0 (socket says 172.21.15.94:54636) 2024-02-17T11:48:37.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.361+0000 7f1b862ac700 1 -- 172.21.15.94:0/4288497680 learned_addr learned my addr 172.21.15.94:0/4288497680 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:37.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.362+0000 7f1b862ac700 1 -- 172.21.15.94:0/4288497680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c009550 con 0x7f1b801080f0 2024-02-17T11:48:37.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.362+0000 7f1b862ac700 1 --2- 172.21.15.94:0/4288497680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 0x7f1b8019b400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009fb0 tx=0x7f1b7c011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:37.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.362+0000 7f1b777fe700 1 -- 172.21.15.94:0/4288497680 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b7c0038b0 con 0x7f1b801080f0 2024-02-17T11:48:37.301 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.362+0000 7f1b777fe700 1 -- 172.21.15.94:0/4288497680 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b7c003a10 con 0x7f1b801080f0 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.362+0000 7f1b777fe700 1 -- 172.21.15.94:0/4288497680 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b7c0195e0 con 0x7f1b801080f0 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.362+0000 7f1b88510700 1 -- 172.21.15.94:0/4288497680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8019bb40 con 0x7f1b801080f0 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.362+0000 7f1b88510700 1 -- 172.21.15.94:0/4288497680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8019bfe0 con 0x7f1b801080f0 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.363+0000 7f1b777fe700 1 -- 172.21.15.94:0/4288497680 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f1b7c01a9f0 con 0x7f1b801080f0 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.363+0000 7f1b88510700 1 -- 172.21.15.94:0/4288497680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b8010b820 con 0x7f1b801080f0 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.363+0000 7f1b777fe700 1 --2- 172.21.15.94:0/4288497680 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f1b6c038ee0 0x7f1b6c03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.363+0000 7f1b777fe700 1 -- 172.21.15.94:0/4288497680 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1b7c04d140 con 0x7f1b801080f0 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.363+0000 7f1b85aab700 1 --2- 172.21.15.94:0/4288497680 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f1b6c038ee0 0x7f1b6c03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.364+0000 7f1b85aab700 1 --2- 172.21.15.94:0/4288497680 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f1b6c038ee0 0x7f1b6c03b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1b70006fd0 tx=0x7f1b70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.368+0000 7f1b777fe700 1 -- 172.21.15.94:0/4288497680 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1b7c003b80 con 0x7f1b801080f0 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.565+0000 7f1b88510700 1 -- 172.21.15.94:0/4288497680 --> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b80102de0 con 0x7f1b6c038ee0 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.760+0000 7f1b777fe700 1 -- 172.21.15.94:0/4288497680 <== mgr.14120 v2:172.21.15.94:6800/2384585889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1b80102de0 con 0x7f1b6c038ee0 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.762+0000 7f1b88510700 1 -- 172.21.15.94:0/4288497680 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f1b6c038ee0 msgr2=0x7f1b6c03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:37.302 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.762+0000 7f1b88510700 1 --2- 172.21.15.94:0/4288497680 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f1b6c038ee0 0x7f1b6c03b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1b70006fd0 tx=0x7f1b70006e40 comp rx=0 tx=0).stop 2024-02-17T11:48:37.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.762+0000 7f1b88510700 1 -- 172.21.15.94:0/4288497680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 msgr2=0x7f1b8019b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:37.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.762+0000 7f1b88510700 1 --2- 172.21.15.94:0/4288497680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 0x7f1b8019b400 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009fb0 tx=0x7f1b7c011b20 comp rx=0 tx=0).stop 2024-02-17T11:48:37.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.762+0000 7f1b88510700 1 -- 172.21.15.94:0/4288497680 shutdown_connections 2024-02-17T11:48:37.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.762+0000 7f1b88510700 1 --2- 172.21.15.94:0/4288497680 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f1b6c038ee0 0x7f1b6c03b3a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:37.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.762+0000 7f1b88510700 1 --2- 172.21.15.94:0/4288497680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1b801080f0 0x7f1b8019b400 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:37.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.762+0000 7f1b88510700 1 -- 172.21.15.94:0/4288497680 >> 172.21.15.94:0/4288497680 conn(0x7f1b80101b20 msgr2=0x7f1b801026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:37.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.763+0000 7f1b88510700 1 -- 172.21.15.94:0/4288497680 shutdown_connections 2024-02-17T11:48:37.303 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:36.763+0000 7f1b88510700 1 -- 172.21.15.94:0/4288497680 wait complete. 2024-02-17T11:48:38.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:37 smithi094 ceph-mon[32499]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:48:38.203 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:37 smithi094 ceph-mon[32499]: Generating ssh key... 2024-02-17T11:48:38.496 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMgNM5FW33h6Al09k3y8SyiiGgkOnK3CxNC3Z0JgyBhRzXLQGcm51ODYu2IyozwChiAl0lJ/cDrPyC8fcDkHw2/949919qppPqY+UbFrNsAg5F1jAhtJuCS4843DIPRSS8PnINudVKXoWlsAoSkZgcUf1BJCviz73FnKKyZuWqr9HwpRagc9Z0s7+P19Fc7zp8suUCQ1DHOL/6F3oZlU9Dcv0QPHPq3kwn3/kp/hqIz8mwIxUJYzOLcXVRIeftcJc2FUna52UIo/kYnIqvmvqbaHJPfYwZxlBrz7lCpqutPg1gSXpRfB7pGtimVArqWFUX0QGYOwGE8AXl9RTdkaTm6R6B7x3HoEzoRkwlVJ1klZoj6YA/Pr1j3QWkS54PrvpK2jNgwAcQ9sg/BtOeYB9Hnhw0e8fk4w0i68kVTl1h1nVbHxylBJpv27Yjiro4PU0kUnQU09ydogrYLsx6mVGhH8L/fOQEH+DgJIUMSmpVbh1o5mA0F9Yq0OwEy6eGTgk= ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:48:38.496 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.699+0000 7fb86ea25700 1 Processor -- start 2024-02-17T11:48:38.496 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.699+0000 7fb86ea25700 1 -- start start 2024-02-17T11:48:38.496 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.699+0000 7fb86ea25700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 0x7fb86810a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.699+0000 7fb86ea25700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86807ae70 con 0x7fb8681080f0 2024-02-17T11:48:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.699+0000 7fb867fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 0x7fb86810a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.699+0000 7fb867fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 0x7fb86810a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54640/0 (socket says 172.21.15.94:54640) 2024-02-17T11:48:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.699+0000 7fb867fff700 1 -- 172.21.15.94:0/179222382 learned_addr learned my addr 172.21.15.94:0/179222382 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.699+0000 7fb867fff700 1 -- 172.21.15.94:0/179222382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86810aa20 con 0x7fb8681080f0 2024-02-17T11:48:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.699+0000 7fb867fff700 1 --2- 172.21.15.94:0/179222382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 0x7fb86810a4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb8580098a0 tx=0x7fb858009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adc91a88a4b84468 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.699+0000 7fb866ffd700 1 -- 172.21.15.94:0/179222382 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb858004160 con 0x7fb8681080f0 2024-02-17T11:48:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.699+0000 7fb866ffd700 1 -- 172.21.15.94:0/179222382 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8580042c0 con 0x7fb8681080f0 2024-02-17T11:48:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.699+0000 7fb866ffd700 1 -- 172.21.15.94:0/179222382 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb858011ac0 con 0x7fb8681080f0 2024-02-17T11:48:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.700+0000 7fb86ea25700 1 -- 172.21.15.94:0/179222382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 msgr2=0x7fb86810a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.700+0000 7fb86ea25700 1 --2- 172.21.15.94:0/179222382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 0x7fb86810a4e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb8580098a0 tx=0x7fb858009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:38.497 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.700+0000 7fb86ea25700 1 -- 172.21.15.94:0/179222382 shutdown_connections 2024-02-17T11:48:38.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.700+0000 7fb86ea25700 1 --2- 172.21.15.94:0/179222382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 0x7fb86810a4e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:38.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.700+0000 7fb86ea25700 1 -- 172.21.15.94:0/179222382 >> 172.21.15.94:0/179222382 conn(0x7fb868101b20 msgr2=0x7fb868103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:38.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.700+0000 7fb86ea25700 1 -- 172.21.15.94:0/179222382 shutdown_connections 2024-02-17T11:48:38.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.700+0000 7fb86ea25700 1 -- 172.21.15.94:0/179222382 wait complete. 2024-02-17T11:48:38.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.701+0000 7fb86ea25700 1 Processor -- start 2024-02-17T11:48:38.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.701+0000 7fb86ea25700 1 -- start start 2024-02-17T11:48:38.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.701+0000 7fb86ea25700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 0x7fb86819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:38.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.701+0000 7fb86ea25700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86819b940 con 0x7fb8681080f0 2024-02-17T11:48:38.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.701+0000 7fb867fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 0x7fb86819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:38.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.701+0000 7fb867fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 0x7fb86819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54656/0 (socket says 172.21.15.94:54656) 2024-02-17T11:48:38.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.701+0000 7fb867fff700 1 -- 172.21.15.94:0/2447311252 learned_addr learned my addr 172.21.15.94:0/2447311252 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:38.498 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.701+0000 7fb867fff700 1 -- 172.21.15.94:0/2447311252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb858009550 con 0x7fb8681080f0 2024-02-17T11:48:38.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.701+0000 7fb867fff700 1 --2- 172.21.15.94:0/2447311252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 0x7fb86819b400 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb858009fb0 tx=0x7fb858011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:38.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.702+0000 7fb8657fa700 1 -- 172.21.15.94:0/2447311252 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8580038b0 con 0x7fb8681080f0 2024-02-17T11:48:38.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.702+0000 7fb8657fa700 1 -- 172.21.15.94:0/2447311252 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb858003a10 con 0x7fb8681080f0 2024-02-17T11:48:38.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.702+0000 7fb86ea25700 1 -- 172.21.15.94:0/2447311252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86819bb40 con 0x7fb8681080f0 2024-02-17T11:48:38.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.702+0000 7fb8657fa700 1 -- 172.21.15.94:0/2447311252 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8580195e0 con 0x7fb8681080f0 2024-02-17T11:48:38.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.702+0000 7fb86ea25700 1 -- 172.21.15.94:0/2447311252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86819bfe0 con 0x7fb8681080f0 2024-02-17T11:48:38.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.703+0000 7fb8657fa700 1 -- 172.21.15.94:0/2447311252 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fb858019850 con 0x7fb8681080f0 2024-02-17T11:48:38.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.703+0000 7fb86ea25700 1 -- 172.21.15.94:0/2447311252 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb86810b820 con 0x7fb8681080f0 2024-02-17T11:48:38.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.703+0000 7fb8657fa700 1 --2- 172.21.15.94:0/2447311252 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fb850038ee0 0x7fb85003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:38.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.703+0000 7fb8657fa700 1 -- 172.21.15.94:0/2447311252 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb85804d0e0 con 0x7fb8681080f0 2024-02-17T11:48:38.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.703+0000 7fb8677fe700 1 --2- 172.21.15.94:0/2447311252 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fb850038ee0 0x7fb85003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:38.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.703+0000 7fb8677fe700 1 --2- 172.21.15.94:0/2447311252 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fb850038ee0 0x7fb85003b3a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb85c006fd0 tx=0x7fb85c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:38.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.708+0000 7fb8657fa700 1 -- 172.21.15.94:0/2447311252 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb85802b340 con 0x7fb8681080f0 2024-02-17T11:48:38.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.900+0000 7fb86ea25700 1 -- 172.21.15.94:0/2447311252 --> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb868102de0 con 0x7fb850038ee0 2024-02-17T11:48:38.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.901+0000 7fb8657fa700 1 -- 172.21.15.94:0/2447311252 <== mgr.14120 v2:172.21.15.94:6800/2384585889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fb868102de0 con 0x7fb850038ee0 2024-02-17T11:48:38.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.903+0000 7fb86ea25700 1 -- 172.21.15.94:0/2447311252 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fb850038ee0 msgr2=0x7fb85003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:38.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.903+0000 7fb86ea25700 1 --2- 172.21.15.94:0/2447311252 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fb850038ee0 0x7fb85003b3a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb85c006fd0 tx=0x7fb85c006e40 comp rx=0 tx=0).stop 2024-02-17T11:48:38.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.903+0000 7fb86ea25700 1 -- 172.21.15.94:0/2447311252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 msgr2=0x7fb86819b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:38.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.903+0000 7fb86ea25700 1 --2- 172.21.15.94:0/2447311252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 0x7fb86819b400 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb858009fb0 tx=0x7fb858011b20 comp rx=0 tx=0).stop 2024-02-17T11:48:38.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.903+0000 7fb86ea25700 1 -- 172.21.15.94:0/2447311252 shutdown_connections 2024-02-17T11:48:38.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.903+0000 7fb86ea25700 1 --2- 172.21.15.94:0/2447311252 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fb850038ee0 0x7fb85003b3a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:38.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.903+0000 7fb86ea25700 1 --2- 172.21.15.94:0/2447311252 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb8681080f0 0x7fb86819b400 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:38.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.903+0000 7fb86ea25700 1 -- 172.21.15.94:0/2447311252 >> 172.21.15.94:0/2447311252 conn(0x7fb868101b20 msgr2=0x7fb8681026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:38.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.903+0000 7fb86ea25700 1 -- 172.21.15.94:0/2447311252 shutdown_connections 2024-02-17T11:48:38.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:37.903+0000 7fb86ea25700 1 -- 172.21.15.94:0/2447311252 wait complete. 2024-02-17T11:48:38.501 INFO:teuthology.orchestra.run.smithi094.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-17T11:48:38.501 INFO:teuthology.orchestra.run.smithi094.stdout:Adding key to root@localhost authorized_keys... 2024-02-17T11:48:38.501 INFO:teuthology.orchestra.run.smithi094.stdout:Adding host smithi094... 2024-02-17T11:48:39.122 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:38 smithi094 ceph-mon[32499]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:48:40.281 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:40 smithi094 ceph-mon[32499]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:48:40.281 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:40 smithi094 ceph-mon[32499]: Deploying cephadm binary to smithi094 2024-02-17T11:48:43.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Added host 'smithi094' with addr '172.21.15.94' 2024-02-17T11:48:43.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.917+0000 7f4f0b0eb700 1 Processor -- start 2024-02-17T11:48:43.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.917+0000 7f4f0b0eb700 1 -- start start 2024-02-17T11:48:43.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.917+0000 7f4f0b0eb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 0x7f4f04106e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:43.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.917+0000 7f4f0b0eb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f04107420 con 0x7f4f04106a30 2024-02-17T11:48:43.906 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.917+0000 7f4f08e87700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 0x7f4f04106e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.917+0000 7f4f08e87700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 0x7f4f04106e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54658/0 (socket says 172.21.15.94:54658) 2024-02-17T11:48:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.917+0000 7f4f08e87700 1 -- 172.21.15.94:0/3772889019 learned_addr learned my addr 172.21.15.94:0/3772889019 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.917+0000 7f4f08e87700 1 -- 172.21.15.94:0/3772889019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f04107750 con 0x7f4f04106a30 2024-02-17T11:48:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.918+0000 7f4f08e87700 1 --2- 172.21.15.94:0/3772889019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 0x7f4f04106e50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80098a0 tx=0x7f4ef8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f5ddc0158cb638 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.918+0000 7f4f037fe700 1 -- 172.21.15.94:0/3772889019 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ef8004160 con 0x7f4f04106a30 2024-02-17T11:48:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.918+0000 7f4f037fe700 1 -- 172.21.15.94:0/3772889019 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ef80042c0 con 0x7f4f04106a30 2024-02-17T11:48:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.918+0000 7f4f037fe700 1 -- 172.21.15.94:0/3772889019 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ef8011ac0 con 0x7f4f04106a30 2024-02-17T11:48:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.918+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3772889019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 msgr2=0x7f4f04106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.918+0000 7f4f0b0eb700 1 --2- 172.21.15.94:0/3772889019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 0x7f4f04106e50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80098a0 tx=0x7f4ef8009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.918+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3772889019 shutdown_connections 2024-02-17T11:48:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.918+0000 7f4f0b0eb700 1 --2- 172.21.15.94:0/3772889019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 0x7f4f04106e50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.918+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3772889019 >> 172.21.15.94:0/3772889019 conn(0x7f4f041012d0 msgr2=0x7f4f041036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:43.907 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.918+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3772889019 shutdown_connections 2024-02-17T11:48:43.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.918+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3772889019 wait complete. 2024-02-17T11:48:43.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.919+0000 7f4f0b0eb700 1 Processor -- start 2024-02-17T11:48:43.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.919+0000 7f4f0b0eb700 1 -- start start 2024-02-17T11:48:43.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.919+0000 7f4f0b0eb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 0x7f4f0419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:43.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.919+0000 7f4f0b0eb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0419a890 con 0x7f4f04106a30 2024-02-17T11:48:43.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.919+0000 7f4f08e87700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 0x7f4f0419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:43.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.919+0000 7f4f08e87700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 0x7f4f0419a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54672/0 (socket says 172.21.15.94:54672) 2024-02-17T11:48:43.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.919+0000 7f4f08e87700 1 -- 172.21.15.94:0/3833637456 learned_addr learned my addr 172.21.15.94:0/3833637456 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:43.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.919+0000 7f4f08e87700 1 -- 172.21.15.94:0/3833637456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef8009550 con 0x7f4f04106a30 2024-02-17T11:48:43.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.920+0000 7f4f08e87700 1 --2- 172.21.15.94:0/3833637456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 0x7f4f0419a350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009fb0 tx=0x7f4ef8011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:43.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.920+0000 7f4f01ffb700 1 -- 172.21.15.94:0/3833637456 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ef80038b0 con 0x7f4f04106a30 2024-02-17T11:48:43.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.920+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3833637456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f0419aa90 con 0x7f4f04106a30 2024-02-17T11:48:43.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.920+0000 7f4f01ffb700 1 -- 172.21.15.94:0/3833637456 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ef8003a10 con 0x7f4f04106a30 2024-02-17T11:48:43.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.920+0000 7f4f01ffb700 1 -- 172.21.15.94:0/3833637456 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ef80195e0 con 0x7f4f04106a30 2024-02-17T11:48:43.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.920+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3833637456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0419af30 con 0x7f4f04106a30 2024-02-17T11:48:43.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.921+0000 7f4f01ffb700 1 -- 172.21.15.94:0/3833637456 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f4ef80198a0 con 0x7f4f04106a30 2024-02-17T11:48:43.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.921+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3833637456 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0410a980 con 0x7f4f04106a30 2024-02-17T11:48:43.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.921+0000 7f4f01ffb700 1 --2- 172.21.15.94:0/3833637456 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f4ef4038e90 0x7f4ef403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:43.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.921+0000 7f4f01ffb700 1 -- 172.21.15.94:0/3833637456 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4ef804d120 con 0x7f4f04106a30 2024-02-17T11:48:43.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.921+0000 7f4f03fff700 1 --2- 172.21.15.94:0/3833637456 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f4ef4038e90 0x7f4ef403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:43.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.921+0000 7f4f03fff700 1 --2- 172.21.15.94:0/3833637456 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f4ef4038e90 0x7f4ef403b350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0006fd0 tx=0x7f4ef0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:43.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:38.926+0000 7f4f01ffb700 1 -- 172.21.15.94:0/3833637456 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ef8018dc0 con 0x7f4f04106a30 2024-02-17T11:48:43.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:39.122+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3833637456 --> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi094", "addr": "172.21.15.94", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f04102590 con 0x7f4ef4038e90 2024-02-17T11:48:43.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:43.344+0000 7f4f01ffb700 1 -- 172.21.15.94:0/3833637456 <== mgr.14120 v2:172.21.15.94:6800/2384585889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f4f04102590 con 0x7f4ef4038e90 2024-02-17T11:48:43.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:43.347+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3833637456 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f4ef4038e90 msgr2=0x7f4ef403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:43.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:43.347+0000 7f4f0b0eb700 1 --2- 172.21.15.94:0/3833637456 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f4ef4038e90 0x7f4ef403b350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0006fd0 tx=0x7f4ef0006e40 comp rx=0 tx=0).stop 2024-02-17T11:48:43.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:43.347+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3833637456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 msgr2=0x7f4f0419a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:43.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:43.347+0000 7f4f0b0eb700 1 --2- 172.21.15.94:0/3833637456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 0x7f4f0419a350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8009fb0 tx=0x7f4ef8011b20 comp rx=0 tx=0).stop 2024-02-17T11:48:43.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:43.347+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3833637456 shutdown_connections 2024-02-17T11:48:43.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:43.347+0000 7f4f0b0eb700 1 --2- 172.21.15.94:0/3833637456 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f4ef4038e90 0x7f4ef403b350 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:43.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:43.347+0000 7f4f0b0eb700 1 --2- 172.21.15.94:0/3833637456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4f04106a30 0x7f4f0419a350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:43.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:43.347+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3833637456 >> 172.21.15.94:0/3833637456 conn(0x7f4f041012d0 msgr2=0x7f4f04101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:43.913 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:43.347+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3833637456 shutdown_connections 2024-02-17T11:48:43.914 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:43.347+0000 7f4f0b0eb700 1 -- 172.21.15.94:0/3833637456 wait complete. 2024-02-17T11:48:43.914 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying unmanaged mon service... 2024-02-17T11:48:44.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:44 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:44.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:44 smithi094 ceph-mon[32499]: Added host smithi094 2024-02-17T11:48:44.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:44 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:48:45.899 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:45 smithi094 ceph-mon[32499]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:48:45.899 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:45 smithi094 ceph-mon[32499]: Saving service mon spec with placement count:5 2024-02-17T11:48:45.899 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:45 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:45.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-17T11:48:45.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.566+0000 7fa3cf4f3700 1 Processor -- start 2024-02-17T11:48:45.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.567+0000 7fa3cf4f3700 1 -- start start 2024-02-17T11:48:45.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.567+0000 7fa3cf4f3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 0x7fa3c8106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:45.908 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.567+0000 7fa3cf4f3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8106ee0 con 0x7fa3c81064f0 2024-02-17T11:48:45.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.567+0000 7fa3cd28f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 0x7fa3c8106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:45.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.567+0000 7fa3cd28f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 0x7fa3c8106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34942/0 (socket says 172.21.15.94:34942) 2024-02-17T11:48:45.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.567+0000 7fa3cd28f700 1 -- 172.21.15.94:0/3094293359 learned_addr learned my addr 172.21.15.94:0/3094293359 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:45.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.567+0000 7fa3cd28f700 1 -- 172.21.15.94:0/3094293359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c81076f0 con 0x7fa3c81064f0 2024-02-17T11:48:45.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.567+0000 7fa3cd28f700 1 --2- 172.21.15.94:0/3094293359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 0x7fa3c8106910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009e80 tx=0x7fa3c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bb042cf133fbfd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:45.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.568+0000 7fa3bffff700 1 -- 172.21.15.94:0/3094293359 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3c000e070 con 0x7fa3c81064f0 2024-02-17T11:48:45.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.568+0000 7fa3bffff700 1 -- 172.21.15.94:0/3094293359 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3c000b8c0 con 0x7fa3c81064f0 2024-02-17T11:48:45.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.568+0000 7fa3bffff700 1 -- 172.21.15.94:0/3094293359 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3c0004550 con 0x7fa3c81064f0 2024-02-17T11:48:45.909 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.568+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3094293359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 msgr2=0x7fa3c8106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.568+0000 7fa3cf4f3700 1 --2- 172.21.15.94:0/3094293359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 0x7fa3c8106910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009e80 tx=0x7fa3c000b330 comp rx=0 tx=0).stop 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.568+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3094293359 shutdown_connections 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.568+0000 7fa3cf4f3700 1 --2- 172.21.15.94:0/3094293359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 0x7fa3c8106910 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.568+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3094293359 >> 172.21.15.94:0/3094293359 conn(0x7fa3c8101ad0 msgr2=0x7fa3c8103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.568+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3094293359 shutdown_connections 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.568+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3094293359 wait complete. 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.569+0000 7fa3cf4f3700 1 Processor -- start 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.569+0000 7fa3cf4f3700 1 -- start start 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.569+0000 7fa3cf4f3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 0x7fa3c819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.569+0000 7fa3cf4f3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c819a7b0 con 0x7fa3c81064f0 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.569+0000 7fa3cd28f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 0x7fa3c819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.569+0000 7fa3cd28f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 0x7fa3c819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34956/0 (socket says 172.21.15.94:34956) 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.569+0000 7fa3cd28f700 1 -- 172.21.15.94:0/3491256526 learned_addr learned my addr 172.21.15.94:0/3491256526 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:45.910 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.569+0000 7fa3cd28f700 1 -- 172.21.15.94:0/3491256526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c0009b30 con 0x7fa3c81064f0 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.569+0000 7fa3cd28f700 1 --2- 172.21.15.94:0/3491256526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 0x7fa3c819a270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000b7b0 tx=0x7fa3c0003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.569+0000 7fa3be7fc700 1 -- 172.21.15.94:0/3491256526 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3c000e040 con 0x7fa3c81064f0 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.570+0000 7fa3be7fc700 1 -- 172.21.15.94:0/3491256526 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3c000bd70 con 0x7fa3c81064f0 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.570+0000 7fa3be7fc700 1 -- 172.21.15.94:0/3491256526 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3c001c770 con 0x7fa3c81064f0 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.570+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3491256526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c819a9b0 con 0x7fa3c81064f0 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.570+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3491256526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c819ae30 con 0x7fa3c81064f0 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.570+0000 7fa3be7fc700 1 -- 172.21.15.94:0/3491256526 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fa3c001c920 con 0x7fa3c81064f0 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.570+0000 7fa3be7fc700 1 --2- 172.21.15.94:0/3491256526 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fa3b4038ee0 0x7fa3b403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.571+0000 7fa3be7fc700 1 -- 172.21.15.94:0/3491256526 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa3c004cf00 con 0x7fa3c81064f0 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.571+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3491256526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c810a920 con 0x7fa3c81064f0 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.571+0000 7fa3cca8e700 1 --2- 172.21.15.94:0/3491256526 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fa3b4038ee0 0x7fa3b403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.571+0000 7fa3cca8e700 1 --2- 172.21.15.94:0/3491256526 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fa3b4038ee0 0x7fa3b403b3a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8006fd0 tx=0x7fa3b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.576+0000 7fa3be7fc700 1 -- 172.21.15.94:0/3491256526 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa3c0017070 con 0x7fa3c81064f0 2024-02-17T11:48:45.911 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.811+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3491256526 --> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa3c8102c60 con 0x7fa3b4038ee0 2024-02-17T11:48:45.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.815+0000 7fa3be7fc700 1 -- 172.21.15.94:0/3491256526 <== mgr.14120 v2:172.21.15.94:6800/2384585889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa3c8102c60 con 0x7fa3b4038ee0 2024-02-17T11:48:45.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.817+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3491256526 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fa3b4038ee0 msgr2=0x7fa3b403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:45.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.817+0000 7fa3cf4f3700 1 --2- 172.21.15.94:0/3491256526 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fa3b4038ee0 0x7fa3b403b3a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8006fd0 tx=0x7fa3b8006e40 comp rx=0 tx=0).stop 2024-02-17T11:48:45.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.817+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3491256526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 msgr2=0x7fa3c819a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:45.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.817+0000 7fa3cf4f3700 1 --2- 172.21.15.94:0/3491256526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 0x7fa3c819a270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000b7b0 tx=0x7fa3c0003d00 comp rx=0 tx=0).stop 2024-02-17T11:48:45.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.817+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3491256526 shutdown_connections 2024-02-17T11:48:45.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.817+0000 7fa3cf4f3700 1 --2- 172.21.15.94:0/3491256526 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fa3b4038ee0 0x7fa3b403b3a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:45.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.817+0000 7fa3cf4f3700 1 --2- 172.21.15.94:0/3491256526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3c81064f0 0x7fa3c819a270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:45.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.817+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3491256526 >> 172.21.15.94:0/3491256526 conn(0x7fa3c8101ad0 msgr2=0x7fa3c8102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:45.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.818+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3491256526 shutdown_connections 2024-02-17T11:48:45.912 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:44.818+0000 7fa3cf4f3700 1 -- 172.21.15.94:0/3491256526 wait complete. 2024-02-17T11:48:45.912 INFO:teuthology.orchestra.run.smithi094.stdout:Deploying unmanaged mgr service... 2024-02-17T11:48:47.499 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-17T11:48:47.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.701+0000 7f638b4c8700 1 Processor -- start 2024-02-17T11:48:47.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.702+0000 7f638b4c8700 1 -- start start 2024-02-17T11:48:47.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.702+0000 7f638b4c8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 0x7f6384106940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:47.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.702+0000 7f638b4c8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6384106f10 con 0x7f6384106520 2024-02-17T11:48:47.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.702+0000 7f6389264700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 0x7f6384106940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:47.500 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.702+0000 7f6389264700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 0x7f6384106940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34962/0 (socket says 172.21.15.94:34962) 2024-02-17T11:48:47.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.702+0000 7f6389264700 1 -- 172.21.15.94:0/1971339053 learned_addr learned my addr 172.21.15.94:0/1971339053 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:47.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.702+0000 7f6389264700 1 -- 172.21.15.94:0/1971339053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6384107720 con 0x7f6384106520 2024-02-17T11:48:47.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.703+0000 7f6389264700 1 --2- 172.21.15.94:0/1971339053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 0x7f6384106940 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f638000bb40 tx=0x7f638000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2472a7a234ed0383 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:47.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.703+0000 7f637bfff700 1 -- 172.21.15.94:0/1971339053 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f638000dbe0 con 0x7f6384106520 2024-02-17T11:48:47.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.703+0000 7f637bfff700 1 -- 172.21.15.94:0/1971339053 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6380004510 con 0x7f6384106520 2024-02-17T11:48:47.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.703+0000 7f637bfff700 1 -- 172.21.15.94:0/1971339053 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6380010470 con 0x7f6384106520 2024-02-17T11:48:47.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.704+0000 7f638b4c8700 1 -- 172.21.15.94:0/1971339053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 msgr2=0x7f6384106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:47.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.704+0000 7f638b4c8700 1 --2- 172.21.15.94:0/1971339053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 0x7f6384106940 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f638000bb40 tx=0x7f638000be50 comp rx=0 tx=0).stop 2024-02-17T11:48:47.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.704+0000 7f638b4c8700 1 -- 172.21.15.94:0/1971339053 shutdown_connections 2024-02-17T11:48:47.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.704+0000 7f638b4c8700 1 --2- 172.21.15.94:0/1971339053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 0x7f6384106940 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:47.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.704+0000 7f638b4c8700 1 -- 172.21.15.94:0/1971339053 >> 172.21.15.94:0/1971339053 conn(0x7f6384101b20 msgr2=0x7f6384103f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:47.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.704+0000 7f638b4c8700 1 -- 172.21.15.94:0/1971339053 shutdown_connections 2024-02-17T11:48:47.501 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.704+0000 7f638b4c8700 1 -- 172.21.15.94:0/1971339053 wait complete. 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.705+0000 7f638b4c8700 1 Processor -- start 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.705+0000 7f638b4c8700 1 -- start start 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.705+0000 7f638b4c8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 0x7f638419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.705+0000 7f638b4c8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638419dc00 con 0x7f6384106520 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.705+0000 7f6389264700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 0x7f638419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.705+0000 7f6389264700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 0x7f638419d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34976/0 (socket says 172.21.15.94:34976) 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.705+0000 7f6389264700 1 -- 172.21.15.94:0/1544560813 learned_addr learned my addr 172.21.15.94:0/1544560813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.706+0000 7f6389264700 1 -- 172.21.15.94:0/1544560813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638000b7f0 con 0x7f6384106520 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.706+0000 7f6389264700 1 --2- 172.21.15.94:0/1544560813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 0x7f638419d6c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6380003c60 tx=0x7f6380003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.706+0000 7f637a7fc700 1 -- 172.21.15.94:0/1544560813 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6380004290 con 0x7f6384106520 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.706+0000 7f638b4c8700 1 -- 172.21.15.94:0/1544560813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638419de00 con 0x7f6384106520 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.706+0000 7f638b4c8700 1 -- 172.21.15.94:0/1544560813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638419e240 con 0x7f6384106520 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.706+0000 7f637a7fc700 1 -- 172.21.15.94:0/1544560813 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6380010bc0 con 0x7f6384106520 2024-02-17T11:48:47.502 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.706+0000 7f637a7fc700 1 -- 172.21.15.94:0/1544560813 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6380019b30 con 0x7f6384106520 2024-02-17T11:48:47.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.707+0000 7f638b4c8700 1 -- 172.21.15.94:0/1544560813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6368005360 con 0x7f6384106520 2024-02-17T11:48:47.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.708+0000 7f637a7fc700 1 -- 172.21.15.94:0/1544560813 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f6380019d40 con 0x7f6384106520 2024-02-17T11:48:47.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.708+0000 7f637a7fc700 1 --2- 172.21.15.94:0/1544560813 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f6370038f30 0x7f637003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:47.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.708+0000 7f637a7fc700 1 -- 172.21.15.94:0/1544560813 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f638004d0f0 con 0x7f6384106520 2024-02-17T11:48:47.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.708+0000 7f6388a63700 1 --2- 172.21.15.94:0/1544560813 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f6370038f30 0x7f637003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:47.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.708+0000 7f6388a63700 1 --2- 172.21.15.94:0/1544560813 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f6370038f30 0x7f637003b3f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6374006fd0 tx=0x7f6374006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:47.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.713+0000 7f637a7fc700 1 -- 172.21.15.94:0/1544560813 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f638000f5b0 con 0x7f6384106520 2024-02-17T11:48:47.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.946+0000 7f638b4c8700 1 -- 172.21.15.94:0/1544560813 --> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6368000bf0 con 0x7f6370038f30 2024-02-17T11:48:47.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.951+0000 7f637a7fc700 1 -- 172.21.15.94:0/1544560813 <== mgr.14120 v2:172.21.15.94:6800/2384585889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6368000bf0 con 0x7f6370038f30 2024-02-17T11:48:47.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.954+0000 7f638b4c8700 1 -- 172.21.15.94:0/1544560813 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f6370038f30 msgr2=0x7f637003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:47.503 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.954+0000 7f638b4c8700 1 --2- 172.21.15.94:0/1544560813 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f6370038f30 0x7f637003b3f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6374006fd0 tx=0x7f6374006e40 comp rx=0 tx=0).stop 2024-02-17T11:48:47.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.954+0000 7f638b4c8700 1 -- 172.21.15.94:0/1544560813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 msgr2=0x7f638419d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:47.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.954+0000 7f638b4c8700 1 --2- 172.21.15.94:0/1544560813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 0x7f638419d6c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6380003c60 tx=0x7f6380003980 comp rx=0 tx=0).stop 2024-02-17T11:48:47.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.954+0000 7f638b4c8700 1 -- 172.21.15.94:0/1544560813 shutdown_connections 2024-02-17T11:48:47.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.954+0000 7f638b4c8700 1 --2- 172.21.15.94:0/1544560813 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f6370038f30 0x7f637003b3f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:47.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.954+0000 7f638b4c8700 1 --2- 172.21.15.94:0/1544560813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6384106520 0x7f638419d6c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:47.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.954+0000 7f638b4c8700 1 -- 172.21.15.94:0/1544560813 >> 172.21.15.94:0/1544560813 conn(0x7f6384101b20 msgr2=0x7f638418fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:47.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.954+0000 7f638b4c8700 1 -- 172.21.15.94:0/1544560813 shutdown_connections 2024-02-17T11:48:47.504 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:46.954+0000 7f638b4c8700 1 -- 172.21.15.94:0/1544560813 wait complete. 2024-02-17T11:48:47.567 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:47 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:47.567 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:47 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:47.567 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:47 smithi094 ceph-mon[32499]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:48:47.567 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:47 smithi094 ceph-mon[32499]: Saving service mgr spec with placement count:2 2024-02-17T11:48:47.567 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:47 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:48.965 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.946+0000 7f31abe54700 1 Processor -- start 2024-02-17T11:48:48.965 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.946+0000 7f31abe54700 1 -- start start 2024-02-17T11:48:48.965 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.946+0000 7f31abe54700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 0x7f31a4107a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:48.965 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.946+0000 7f31abe54700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a4108030 con 0x7f31a4107640 2024-02-17T11:48:48.965 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.946+0000 7f31a9bf0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 0x7f31a4107a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:48.965 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.946+0000 7f31a9bf0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 0x7f31a4107a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34992/0 (socket says 172.21.15.94:34992) 2024-02-17T11:48:48.966 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.946+0000 7f31a9bf0700 1 -- 172.21.15.94:0/3368692931 learned_addr learned my addr 172.21.15.94:0/3368692931 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:48.966 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.947+0000 7f31a9bf0700 1 -- 172.21.15.94:0/3368692931 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a41087f0 con 0x7f31a4107640 2024-02-17T11:48:48.966 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.947+0000 7f31a9bf0700 1 --2- 172.21.15.94:0/3368692931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 0x7f31a4107a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f31a00098a0 tx=0x7f31a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c1ace7fd7a7d2e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:48.966 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.947+0000 7f31a8bee700 1 -- 172.21.15.94:0/3368692931 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31a0004160 con 0x7f31a4107640 2024-02-17T11:48:48.966 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.947+0000 7f31a8bee700 1 -- 172.21.15.94:0/3368692931 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31a00042c0 con 0x7f31a4107640 2024-02-17T11:48:48.966 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.947+0000 7f31a8bee700 1 -- 172.21.15.94:0/3368692931 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31a0011a80 con 0x7f31a4107640 2024-02-17T11:48:48.966 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.948+0000 7f31abe54700 1 -- 172.21.15.94:0/3368692931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 msgr2=0x7f31a4107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:48.966 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.948+0000 7f31abe54700 1 --2- 172.21.15.94:0/3368692931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 0x7f31a4107a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f31a00098a0 tx=0x7f31a0009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:48.966 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.948+0000 7f31abe54700 1 -- 172.21.15.94:0/3368692931 shutdown_connections 2024-02-17T11:48:48.966 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.948+0000 7f31abe54700 1 --2- 172.21.15.94:0/3368692931 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 0x7f31a4107a60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:48.966 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.948+0000 7f31abe54700 1 -- 172.21.15.94:0/3368692931 >> 172.21.15.94:0/3368692931 conn(0x7f31a4102ca0 msgr2=0x7f31a41050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.948+0000 7f31abe54700 1 -- 172.21.15.94:0/3368692931 shutdown_connections 2024-02-17T11:48:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.948+0000 7f31abe54700 1 -- 172.21.15.94:0/3368692931 wait complete. 2024-02-17T11:48:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.948+0000 7f31abe54700 1 Processor -- start 2024-02-17T11:48:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.948+0000 7f31abe54700 1 -- start start 2024-02-17T11:48:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.948+0000 7f31abe54700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 0x7f31a419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.948+0000 7f31abe54700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a419b910 con 0x7f31a4107640 2024-02-17T11:48:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.949+0000 7f31a9bf0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 0x7f31a419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.949+0000 7f31a9bf0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 0x7f31a419b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35000/0 (socket says 172.21.15.94:35000) 2024-02-17T11:48:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.949+0000 7f31a9bf0700 1 -- 172.21.15.94:0/2657661063 learned_addr learned my addr 172.21.15.94:0/2657661063 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.949+0000 7f31a9bf0700 1 -- 172.21.15.94:0/2657661063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a0009550 con 0x7f31a4107640 2024-02-17T11:48:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.949+0000 7f31a9bf0700 1 --2- 172.21.15.94:0/2657661063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 0x7f31a419b3d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f31a0000c00 tx=0x7f31a0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.949+0000 7f319affd700 1 -- 172.21.15.94:0/2657661063 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31a00039e0 con 0x7f31a4107640 2024-02-17T11:48:48.967 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.949+0000 7f319affd700 1 -- 172.21.15.94:0/2657661063 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31a0003b40 con 0x7f31a4107640 2024-02-17T11:48:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.949+0000 7f31abe54700 1 -- 172.21.15.94:0/2657661063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a419bb10 con 0x7f31a4107640 2024-02-17T11:48:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.949+0000 7f319affd700 1 -- 172.21.15.94:0/2657661063 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31a0024820 con 0x7f31a4107640 2024-02-17T11:48:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.949+0000 7f31abe54700 1 -- 172.21.15.94:0/2657661063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a419bfb0 con 0x7f31a4107640 2024-02-17T11:48:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.950+0000 7f319affd700 1 -- 172.21.15.94:0/2657661063 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f31a0024980 con 0x7f31a4107640 2024-02-17T11:48:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.950+0000 7f319affd700 1 --2- 172.21.15.94:0/2657661063 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f3190038f30 0x7f319003b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.950+0000 7f31abe54700 1 -- 172.21.15.94:0/2657661063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a410ba20 con 0x7f31a4107640 2024-02-17T11:48:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.950+0000 7f319affd700 1 -- 172.21.15.94:0/2657661063 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f31a004d1c0 con 0x7f31a4107640 2024-02-17T11:48:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.950+0000 7f31a93ef700 1 --2- 172.21.15.94:0/2657661063 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f3190038f30 0x7f319003b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.951+0000 7f31a93ef700 1 --2- 172.21.15.94:0/2657661063 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f3190038f30 0x7f319003b3f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3194006fd0 tx=0x7f3194006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:47.955+0000 7f319affd700 1 -- 172.21.15.94:0/2657661063 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31a0024c30 con 0x7f31a4107640 2024-02-17T11:48:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:48.160+0000 7f31abe54700 1 -- 172.21.15.94:0/2657661063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f31a410bc30 con 0x7f31a4107640 2024-02-17T11:48:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:48.162+0000 7f319affd700 1 -- 172.21.15.94:0/2657661063 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f31a0018350 con 0x7f31a4107640 2024-02-17T11:48:48.968 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:48.165+0000 7f31abe54700 1 -- 172.21.15.94:0/2657661063 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f3190038f30 msgr2=0x7f319003b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:48.165+0000 7f31abe54700 1 --2- 172.21.15.94:0/2657661063 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f3190038f30 0x7f319003b3f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3194006fd0 tx=0x7f3194006e40 comp rx=0 tx=0).stop 2024-02-17T11:48:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:48.165+0000 7f31abe54700 1 -- 172.21.15.94:0/2657661063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 msgr2=0x7f31a419b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:48.165+0000 7f31abe54700 1 --2- 172.21.15.94:0/2657661063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 0x7f31a419b3d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f31a0000c00 tx=0x7f31a0011b00 comp rx=0 tx=0).stop 2024-02-17T11:48:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:48.165+0000 7f31abe54700 1 -- 172.21.15.94:0/2657661063 shutdown_connections 2024-02-17T11:48:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:48.165+0000 7f31abe54700 1 --2- 172.21.15.94:0/2657661063 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f3190038f30 0x7f319003b3f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:48.165+0000 7f31abe54700 1 --2- 172.21.15.94:0/2657661063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31a4107640 0x7f31a419b3d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:48.165+0000 7f31abe54700 1 -- 172.21.15.94:0/2657661063 >> 172.21.15.94:0/2657661063 conn(0x7f31a4102ca0 msgr2=0x7f31a4103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:48.165+0000 7f31abe54700 1 -- 172.21.15.94:0/2657661063 shutdown_connections 2024-02-17T11:48:48.969 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:48.165+0000 7f31abe54700 1 -- 172.21.15.94:0/2657661063 wait complete. 2024-02-17T11:48:49.415 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:49 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2657661063' entity='client.admin' 2024-02-17T11:48:50.572 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:50 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:50.572 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:50 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/658644035' entity='client.admin' 2024-02-17T11:48:50.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.750+0000 7f7e3a867700 1 Processor -- start 2024-02-17T11:48:50.580 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.750+0000 7f7e3a867700 1 -- start start 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.750+0000 7f7e3a867700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 0x7f7e34106df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.750+0000 7f7e3a867700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e341073c0 con 0x7f7e341069d0 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.750+0000 7f7e33fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 0x7f7e34106df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.750+0000 7f7e33fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 0x7f7e34106df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35008/0 (socket says 172.21.15.94:35008) 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.750+0000 7f7e33fff700 1 -- 172.21.15.94:0/544663755 learned_addr learned my addr 172.21.15.94:0/544663755 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.751+0000 7f7e33fff700 1 -- 172.21.15.94:0/544663755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e341076f0 con 0x7f7e341069d0 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.751+0000 7f7e33fff700 1 --2- 172.21.15.94:0/544663755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 0x7f7e34106df0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7e280098a0 tx=0x7f7e28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e17705f114975c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.751+0000 7f7e32ffd700 1 -- 172.21.15.94:0/544663755 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e28004160 con 0x7f7e341069d0 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.751+0000 7f7e32ffd700 1 -- 172.21.15.94:0/544663755 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e280042c0 con 0x7f7e341069d0 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.751+0000 7f7e32ffd700 1 -- 172.21.15.94:0/544663755 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e28011a80 con 0x7f7e341069d0 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.752+0000 7f7e3a867700 1 -- 172.21.15.94:0/544663755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 msgr2=0x7f7e34106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.752+0000 7f7e3a867700 1 --2- 172.21.15.94:0/544663755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 0x7f7e34106df0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7e280098a0 tx=0x7f7e28009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.752+0000 7f7e3a867700 1 -- 172.21.15.94:0/544663755 shutdown_connections 2024-02-17T11:48:50.581 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.752+0000 7f7e3a867700 1 --2- 172.21.15.94:0/544663755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 0x7f7e34106df0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:50.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.752+0000 7f7e3a867700 1 -- 172.21.15.94:0/544663755 >> 172.21.15.94:0/544663755 conn(0x7f7e34101250 msgr2=0x7f7e34103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:50.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.752+0000 7f7e3a867700 1 -- 172.21.15.94:0/544663755 shutdown_connections 2024-02-17T11:48:50.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.752+0000 7f7e3a867700 1 -- 172.21.15.94:0/544663755 wait complete. 2024-02-17T11:48:50.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.752+0000 7f7e3a867700 1 Processor -- start 2024-02-17T11:48:50.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.752+0000 7f7e3a867700 1 -- start start 2024-02-17T11:48:50.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.753+0000 7f7e3a867700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 0x7f7e34105fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:50.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.753+0000 7f7e3a867700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e341064f0 con 0x7f7e341069d0 2024-02-17T11:48:50.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.753+0000 7f7e33fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 0x7f7e34105fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:50.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.753+0000 7f7e33fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 0x7f7e34105fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35020/0 (socket says 172.21.15.94:35020) 2024-02-17T11:48:50.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.753+0000 7f7e33fff700 1 -- 172.21.15.94:0/658644035 learned_addr learned my addr 172.21.15.94:0/658644035 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:50.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.753+0000 7f7e33fff700 1 -- 172.21.15.94:0/658644035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e28009550 con 0x7f7e341069d0 2024-02-17T11:48:50.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.753+0000 7f7e33fff700 1 --2- 172.21.15.94:0/658644035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 0x7f7e34105fb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e28011740 tx=0x7f7e28011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:50.582 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.754+0000 7f7e317fa700 1 -- 172.21.15.94:0/658644035 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e28003b20 con 0x7f7e341069d0 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.754+0000 7f7e3a867700 1 -- 172.21.15.94:0/658644035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3419be70 con 0x7f7e341069d0 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.754+0000 7f7e317fa700 1 -- 172.21.15.94:0/658644035 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e2801a910 con 0x7f7e341069d0 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.754+0000 7f7e317fa700 1 -- 172.21.15.94:0/658644035 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e28024900 con 0x7f7e341069d0 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.754+0000 7f7e3a867700 1 -- 172.21.15.94:0/658644035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e341a0730 con 0x7f7e341069d0 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.755+0000 7f7e317fa700 1 -- 172.21.15.94:0/658644035 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f7e28024b70 con 0x7f7e341069d0 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.755+0000 7f7e317fa700 1 --2- 172.21.15.94:0/658644035 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f7e1c038e90 0x7f7e1c03b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.755+0000 7f7e3a867700 1 -- 172.21.15.94:0/658644035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3410a950 con 0x7f7e341069d0 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.755+0000 7f7e317fa700 1 -- 172.21.15.94:0/658644035 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7e2804d220 con 0x7f7e341069d0 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.755+0000 7f7e337fe700 1 --2- 172.21.15.94:0/658644035 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f7e1c038e90 0x7f7e1c03b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.755+0000 7f7e337fe700 1 --2- 172.21.15.94:0/658644035 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f7e1c038e90 0x7f7e1c03b350 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7e24006fd0 tx=0x7f7e24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.760+0000 7f7e317fa700 1 -- 172.21.15.94:0/658644035 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7e28024e20 con 0x7f7e341069d0 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.962+0000 7f7e3a867700 1 -- 172.21.15.94:0/658644035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f7e3410ab60 con 0x7f7e341069d0 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.965+0000 7f7e317fa700 1 -- 172.21.15.94:0/658644035 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f7e2802c350 con 0x7f7e341069d0 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.968+0000 7f7e3a867700 1 -- 172.21.15.94:0/658644035 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f7e1c038e90 msgr2=0x7f7e1c03b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:50.583 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.968+0000 7f7e3a867700 1 --2- 172.21.15.94:0/658644035 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f7e1c038e90 0x7f7e1c03b350 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7e24006fd0 tx=0x7f7e24006e40 comp rx=0 tx=0).stop 2024-02-17T11:48:50.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.968+0000 7f7e3a867700 1 -- 172.21.15.94:0/658644035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 msgr2=0x7f7e34105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:50.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.968+0000 7f7e3a867700 1 --2- 172.21.15.94:0/658644035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 0x7f7e34105fb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7e28011740 tx=0x7f7e28011a50 comp rx=0 tx=0).stop 2024-02-17T11:48:50.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.968+0000 7f7e3a867700 1 -- 172.21.15.94:0/658644035 shutdown_connections 2024-02-17T11:48:50.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.968+0000 7f7e3a867700 1 --2- 172.21.15.94:0/658644035 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7f7e1c038e90 0x7f7e1c03b350 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:50.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.968+0000 7f7e3a867700 1 --2- 172.21.15.94:0/658644035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e341069d0 0x7f7e34105fb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:50.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.968+0000 7f7e3a867700 1 -- 172.21.15.94:0/658644035 >> 172.21.15.94:0/658644035 conn(0x7f7e34101250 msgr2=0x7f7e34101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:50.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.968+0000 7f7e3a867700 1 -- 172.21.15.94:0/658644035 shutdown_connections 2024-02-17T11:48:50.584 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:49.968+0000 7f7e3a867700 1 -- 172.21.15.94:0/658644035 wait complete. 2024-02-17T11:48:50.584 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling the dashboard module... 2024-02-17T11:48:51.859 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:51 smithi094 ceph-mon[32499]: from='mgr.14120 172.21.15.94:0/574748499' entity='mgr.x' 2024-02-17T11:48:51.859 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:51 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/674223339' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-17T11:48:51.859 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:51 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: ignoring --setuser ceph since I am not root 2024-02-17T11:48:51.859 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:51 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: ignoring --setgroup ceph since I am not root 2024-02-17T11:48:52.427 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:52 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:52.231+0000 7f1c212b4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T11:48:52.428 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:52 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:52.324+0000 7f1c212b4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T11:48:52.430 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.037+0000 7fd16d055700 1 Processor -- start 2024-02-17T11:48:52.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.037+0000 7fd16d055700 1 -- start start 2024-02-17T11:48:52.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.037+0000 7fd16d055700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 0x7fd16810a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:52.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.038+0000 7fd16d055700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16807ae10 con 0x7fd168108130 2024-02-17T11:48:52.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.038+0000 7fd166d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 0x7fd16810a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:52.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.038+0000 7fd166d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 0x7fd16810a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35028/0 (socket says 172.21.15.94:35028) 2024-02-17T11:48:52.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.038+0000 7fd166d9d700 1 -- 172.21.15.94:0/580280804 learned_addr learned my addr 172.21.15.94:0/580280804 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:52.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.038+0000 7fd166d9d700 1 -- 172.21.15.94:0/580280804 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16807af50 con 0x7fd168108130 2024-02-17T11:48:52.435 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.038+0000 7fd166d9d700 1 --2- 172.21.15.94:0/580280804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 0x7fd16810a520 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd1580098a0 tx=0x7fd158009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1d5e035b9cbde5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:52.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.038+0000 7fd165d9b700 1 -- 172.21.15.94:0/580280804 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd158004160 con 0x7fd168108130 2024-02-17T11:48:52.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.038+0000 7fd165d9b700 1 -- 172.21.15.94:0/580280804 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1580042c0 con 0x7fd168108130 2024-02-17T11:48:52.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.038+0000 7fd165d9b700 1 -- 172.21.15.94:0/580280804 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd158011a80 con 0x7fd168108130 2024-02-17T11:48:52.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.039+0000 7fd16d055700 1 -- 172.21.15.94:0/580280804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 msgr2=0x7fd16810a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:52.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.039+0000 7fd16d055700 1 --2- 172.21.15.94:0/580280804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 0x7fd16810a520 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd1580098a0 tx=0x7fd158009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:52.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.039+0000 7fd16d055700 1 -- 172.21.15.94:0/580280804 shutdown_connections 2024-02-17T11:48:52.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.039+0000 7fd16d055700 1 --2- 172.21.15.94:0/580280804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 0x7fd16810a520 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:52.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.039+0000 7fd16d055700 1 -- 172.21.15.94:0/580280804 >> 172.21.15.94:0/580280804 conn(0x7fd168101a80 msgr2=0x7fd168103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:52.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.039+0000 7fd16d055700 1 -- 172.21.15.94:0/580280804 shutdown_connections 2024-02-17T11:48:52.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.039+0000 7fd16d055700 1 -- 172.21.15.94:0/580280804 wait complete. 2024-02-17T11:48:52.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.040+0000 7fd16d055700 1 Processor -- start 2024-02-17T11:48:52.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.040+0000 7fd16d055700 1 -- start start 2024-02-17T11:48:52.436 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.040+0000 7fd16d055700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 0x7fd16819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.040+0000 7fd16d055700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16819daf0 con 0x7fd168108130 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.040+0000 7fd166d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 0x7fd16819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.040+0000 7fd166d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 0x7fd16819d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35036/0 (socket says 172.21.15.94:35036) 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.040+0000 7fd166d9d700 1 -- 172.21.15.94:0/674223339 learned_addr learned my addr 172.21.15.94:0/674223339 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.041+0000 7fd166d9d700 1 -- 172.21.15.94:0/674223339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd158009550 con 0x7fd168108130 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.041+0000 7fd166d9d700 1 --2- 172.21.15.94:0/674223339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 0x7fd16819d5b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd158003f20 tx=0x7fd158011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.041+0000 7fd14ffff700 1 -- 172.21.15.94:0/674223339 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1580039f0 con 0x7fd168108130 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.041+0000 7fd14ffff700 1 -- 172.21.15.94:0/674223339 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd158003b50 con 0x7fd168108130 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.041+0000 7fd16d055700 1 -- 172.21.15.94:0/674223339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16819dcf0 con 0x7fd168108130 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.041+0000 7fd14ffff700 1 -- 172.21.15.94:0/674223339 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd15801a6d0 con 0x7fd168108130 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.041+0000 7fd16d055700 1 -- 172.21.15.94:0/674223339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16819e190 con 0x7fd168108130 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.042+0000 7fd14ffff700 1 -- 172.21.15.94:0/674223339 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fd15801a940 con 0x7fd168108130 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.042+0000 7fd16d055700 1 -- 172.21.15.94:0/674223339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16810b830 con 0x7fd168108130 2024-02-17T11:48:52.437 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.042+0000 7fd14ffff700 1 --2- 172.21.15.94:0/674223339 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fd150038ee0 0x7fd15003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.042+0000 7fd14ffff700 1 -- 172.21.15.94:0/674223339 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd15804d510 con 0x7fd168108130 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.042+0000 7fd16659c700 1 --2- 172.21.15.94:0/674223339 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fd150038ee0 0x7fd15003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.043+0000 7fd16659c700 1 --2- 172.21.15.94:0/674223339 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fd150038ee0 0x7fd15003b3a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd15c006fd0 tx=0x7fd15c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.047+0000 7fd14ffff700 1 -- 172.21.15.94:0/674223339 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd158018dc0 con 0x7fd168108130 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.312+0000 7fd16d055700 1 -- 172.21.15.94:0/674223339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fd16810ba70 con 0x7fd168108130 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.666+0000 7fd14ffff700 1 -- 172.21.15.94:0/674223339 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fd158018be0 con 0x7fd168108130 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.667+0000 7fd14ffff700 1 -- 172.21.15.94:0/674223339 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fd15802d9b0 con 0x7fd168108130 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.668+0000 7fd16d055700 1 -- 172.21.15.94:0/674223339 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fd150038ee0 msgr2=0x7fd15003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.668+0000 7fd16d055700 1 --2- 172.21.15.94:0/674223339 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fd150038ee0 0x7fd15003b3a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd15c006fd0 tx=0x7fd15c006e40 comp rx=0 tx=0).stop 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.668+0000 7fd16d055700 1 -- 172.21.15.94:0/674223339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 msgr2=0x7fd16819d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.668+0000 7fd16d055700 1 --2- 172.21.15.94:0/674223339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 0x7fd16819d5b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd158003f20 tx=0x7fd158011b00 comp rx=0 tx=0).stop 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.668+0000 7fd16d055700 1 -- 172.21.15.94:0/674223339 shutdown_connections 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.668+0000 7fd16d055700 1 --2- 172.21.15.94:0/674223339 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fd150038ee0 0x7fd15003b3a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:52.438 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.668+0000 7fd16d055700 1 --2- 172.21.15.94:0/674223339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd168108130 0x7fd16819d5b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:52.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.668+0000 7fd16d055700 1 -- 172.21.15.94:0/674223339 >> 172.21.15.94:0/674223339 conn(0x7fd168101a80 msgr2=0x7fd168102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:52.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.669+0000 7fd16d055700 1 -- 172.21.15.94:0/674223339 shutdown_connections 2024-02-17T11:48:52.439 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:51.669+0000 7fd16d055700 1 -- 172.21.15.94:0/674223339 wait complete. 2024-02-17T11:48:52.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:52 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/674223339' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-17T11:48:52.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:52 smithi094 ceph-mon[32499]: mgrmap e9: x(active, since 19s) 2024-02-17T11:48:53.061 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:52 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:52.983+0000 7f1c212b4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T11:48:53.317 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:53.079+0000 7f1c212b4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T11:48:53.317 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:53.225+0000 7f1c212b4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T11:48:53.596 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:53.321+0000 7f1c212b4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T11:48:53.596 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:53.403+0000 7f1c212b4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T11:48:53.862 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:53 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/534786361' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-17T11:48:53.863 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:53.667+0000 7f1c212b4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T11:48:53.863 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:53.777+0000 7f1c212b4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T11:48:53.863 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:53.863+0000 7f1c212b4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T11:48:53.880 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-17T11:48:53.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-17T11:48:53.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-17T11:48:53.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-17T11:48:53.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-17T11:48:53.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-17T11:48:53.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.856+0000 7fdd70949700 1 Processor -- start 2024-02-17T11:48:53.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.856+0000 7fdd70949700 1 -- start start 2024-02-17T11:48:53.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.856+0000 7fdd70949700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 0x7fdd6c10a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:53.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.856+0000 7fdd70949700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c07ae90 con 0x7fdd6c1080a0 2024-02-17T11:48:53.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.857+0000 7fdd6a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 0x7fdd6c10a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:53.881 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.857+0000 7fdd6a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 0x7fdd6c10a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35066/0 (socket says 172.21.15.94:35066) 2024-02-17T11:48:53.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.857+0000 7fdd6a59c700 1 -- 172.21.15.94:0/4290601562 learned_addr learned my addr 172.21.15.94:0/4290601562 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:53.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.857+0000 7fdd6a59c700 1 -- 172.21.15.94:0/4290601562 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c10a9d0 con 0x7fdd6c1080a0 2024-02-17T11:48:53.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.857+0000 7fdd6a59c700 1 --2- 172.21.15.94:0/4290601562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 0x7fdd6c10a490 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0098a0 tx=0x7fdd5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bda1a04885e2dcc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:53.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.857+0000 7fdd6959a700 1 -- 172.21.15.94:0/4290601562 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd5c004160 con 0x7fdd6c1080a0 2024-02-17T11:48:53.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.857+0000 7fdd6959a700 1 -- 172.21.15.94:0/4290601562 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd5c0042c0 con 0x7fdd6c1080a0 2024-02-17T11:48:53.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.857+0000 7fdd6959a700 1 -- 172.21.15.94:0/4290601562 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd5c011a80 con 0x7fdd6c1080a0 2024-02-17T11:48:53.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.858+0000 7fdd70949700 1 -- 172.21.15.94:0/4290601562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 msgr2=0x7fdd6c10a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:53.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.858+0000 7fdd70949700 1 --2- 172.21.15.94:0/4290601562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 0x7fdd6c10a490 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0098a0 tx=0x7fdd5c009bb0 comp rx=0 tx=0).stop 2024-02-17T11:48:53.882 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.858+0000 7fdd70949700 1 -- 172.21.15.94:0/4290601562 shutdown_connections 2024-02-17T11:48:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.858+0000 7fdd70949700 1 --2- 172.21.15.94:0/4290601562 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 0x7fdd6c10a490 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.858+0000 7fdd70949700 1 -- 172.21.15.94:0/4290601562 >> 172.21.15.94:0/4290601562 conn(0x7fdd6c101ad0 msgr2=0x7fdd6c103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.858+0000 7fdd70949700 1 -- 172.21.15.94:0/4290601562 shutdown_connections 2024-02-17T11:48:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.858+0000 7fdd70949700 1 -- 172.21.15.94:0/4290601562 wait complete. 2024-02-17T11:48:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.859+0000 7fdd70949700 1 Processor -- start 2024-02-17T11:48:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.859+0000 7fdd70949700 1 -- start start 2024-02-17T11:48:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.859+0000 7fdd70949700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 0x7fdd6c19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.859+0000 7fdd70949700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c19b8e0 con 0x7fdd6c1080a0 2024-02-17T11:48:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.859+0000 7fdd6a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 0x7fdd6c19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:48:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.859+0000 7fdd6a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 0x7fdd6c19b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35070/0 (socket says 172.21.15.94:35070) 2024-02-17T11:48:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.859+0000 7fdd6a59c700 1 -- 172.21.15.94:0/534786361 learned_addr learned my addr 172.21.15.94:0/534786361 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:48:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.860+0000 7fdd6a59c700 1 -- 172.21.15.94:0/534786361 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c009550 con 0x7fdd6c1080a0 2024-02-17T11:48:53.883 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.860+0000 7fdd6a59c700 1 --2- 172.21.15.94:0/534786361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 0x7fdd6c19b3a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c003f20 tx=0x7fdd5c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:48:53.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.860+0000 7fdd5b7fe700 1 -- 172.21.15.94:0/534786361 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd5c0039f0 con 0x7fdd6c1080a0 2024-02-17T11:48:53.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.860+0000 7fdd5b7fe700 1 -- 172.21.15.94:0/534786361 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdd5c003b50 con 0x7fdd6c1080a0 2024-02-17T11:48:53.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.860+0000 7fdd70949700 1 -- 172.21.15.94:0/534786361 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c19bae0 con 0x7fdd6c1080a0 2024-02-17T11:48:53.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.860+0000 7fdd5b7fe700 1 -- 172.21.15.94:0/534786361 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd5c01a6d0 con 0x7fdd6c1080a0 2024-02-17T11:48:53.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.860+0000 7fdd70949700 1 -- 172.21.15.94:0/534786361 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c19bf80 con 0x7fdd6c1080a0 2024-02-17T11:48:53.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.861+0000 7fdd5b7fe700 1 -- 172.21.15.94:0/534786361 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fdd5c01a9c0 con 0x7fdd6c1080a0 2024-02-17T11:48:53.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.861+0000 7fdd70949700 1 -- 172.21.15.94:0/534786361 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c10b810 con 0x7fdd6c1080a0 2024-02-17T11:48:53.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.861+0000 7fdd5b7fe700 1 --2- 172.21.15.94:0/534786361 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fdd54038f40 0x7fdd5403b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:48:53.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.861+0000 7fdd5b7fe700 1 -- 172.21.15.94:0/534786361 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdd5c04d310 con 0x7fdd6c1080a0 2024-02-17T11:48:53.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.861+0000 7fdd69d9b700 1 -- 172.21.15.94:0/534786361 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fdd54038f40 msgr2=0x7fdd5403b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2384585889 2024-02-17T11:48:53.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.861+0000 7fdd69d9b700 1 --2- 172.21.15.94:0/534786361 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fdd54038f40 0x7fdd5403b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T11:48:53.884 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:52.866+0000 7fdd5b7fe700 1 -- 172.21.15.94:0/534786361 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd5c022740 con 0x7fdd6c1080a0 2024-02-17T11:48:53.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.062+0000 7fdd69d9b700 1 -- 172.21.15.94:0/534786361 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fdd54038f40 msgr2=0x7fdd5403b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2384585889 2024-02-17T11:48:53.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.062+0000 7fdd69d9b700 1 --2- 172.21.15.94:0/534786361 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fdd54038f40 0x7fdd5403b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-17T11:48:53.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.125+0000 7fdd70949700 1 -- 172.21.15.94:0/534786361 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fdd6c19e860 con 0x7fdd6c1080a0 2024-02-17T11:48:53.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.126+0000 7fdd5b7fe700 1 -- 172.21.15.94:0/534786361 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fdd5c02c350 con 0x7fdd6c1080a0 2024-02-17T11:48:53.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.128+0000 7fdd70949700 1 -- 172.21.15.94:0/534786361 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fdd54038f40 msgr2=0x7fdd5403b400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T11:48:53.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.128+0000 7fdd70949700 1 --2- 172.21.15.94:0/534786361 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fdd54038f40 0x7fdd5403b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:53.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.128+0000 7fdd70949700 1 -- 172.21.15.94:0/534786361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 msgr2=0x7fdd6c19b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:48:53.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.128+0000 7fdd70949700 1 --2- 172.21.15.94:0/534786361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 0x7fdd6c19b3a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c003f20 tx=0x7fdd5c011b00 comp rx=0 tx=0).stop 2024-02-17T11:48:53.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.128+0000 7fdd70949700 1 -- 172.21.15.94:0/534786361 shutdown_connections 2024-02-17T11:48:53.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.128+0000 7fdd70949700 1 --2- 172.21.15.94:0/534786361 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7fdd54038f40 0x7fdd5403b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:53.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.128+0000 7fdd70949700 1 --2- 172.21.15.94:0/534786361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdd6c1080a0 0x7fdd6c19b3a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:48:53.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.128+0000 7fdd70949700 1 -- 172.21.15.94:0/534786361 >> 172.21.15.94:0/534786361 conn(0x7fdd6c101ad0 msgr2=0x7fdd6c1025a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:48:53.885 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.128+0000 7fdd70949700 1 -- 172.21.15.94:0/534786361 shutdown_connections 2024-02-17T11:48:53.886 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:53.128+0000 7fdd70949700 1 -- 172.21.15.94:0/534786361 wait complete. 2024-02-17T11:48:53.886 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for the mgr to restart... 2024-02-17T11:48:53.886 INFO:teuthology.orchestra.run.smithi094.stdout:Waiting for mgr epoch 9... 2024-02-17T11:48:54.146 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:53.967+0000 7f1c212b4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T11:48:54.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:54 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:54.147+0000 7f1c212b4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T11:48:55.205 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:55 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:55.113+0000 7f1c212b4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T11:48:55.582 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:55 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:55.242+0000 7f1c212b4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T11:48:55.582 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:55 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:55.582+0000 7f1c212b4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T11:48:55.871 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:55 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:55.673+0000 7f1c212b4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T11:48:55.871 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:55 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:55.872+0000 7f1c212b4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:48:56.205 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:56 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:56.058+0000 7f1c212b4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T11:48:56.601 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:56 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:56.345+0000 7f1c212b4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:48:56.601 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:56 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:56.602+0000 7f1c212b4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T11:48:56.955 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:56 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:56.839+0000 7f1c212b4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T11:48:57.955 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:57 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:57.650+0000 7f1c212b4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T11:48:58.540 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:58.220+0000 7f1c212b4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T11:48:58.540 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:58.320+0000 7f1c212b4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T11:48:58.540 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:48:58.540+0000 7f1c212b4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T11:48:58.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-mon[32499]: Active manager daemon x restarted 2024-02-17T11:48:58.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-mon[32499]: Activating manager daemon x 2024-02-17T11:48:58.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-mon[32499]: osdmap e3: 0 total, 0 up, 0 in 2024-02-17T11:48:58.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-mon[32499]: mgrmap e10: x(active, starting, since 0.00327s) 2024-02-17T11:48:58.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:48:58.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-17T11:48:58.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-17T11:48:58.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-17T11:48:58.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-17T11:48:58.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-mon[32499]: Manager daemon x is now available 2024-02-17T11:48:58.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:48:58.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:48:59.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-17T11:48:59.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-17T11:48:59.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:48:59 smithi094 ceph-mon[32499]: mgrmap e11: x(active, since 1.0058s) 2024-02-17T11:49:00.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout { 2024-02-17T11:49:00.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-17T11:49:00.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-17T11:49:00.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout } 2024-02-17T11:49:00.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.303+0000 7ff1f5d00700 1 Processor -- start 2024-02-17T11:49:00.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.303+0000 7ff1f5d00700 1 -- start start 2024-02-17T11:49:00.150 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.304+0000 7ff1f5d00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 0x7ff1f01062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:00.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.304+0000 7ff1f5d00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f007ae30 con 0x7ff1f0105ed0 2024-02-17T11:49:00.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.304+0000 7ff1ef7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 0x7ff1f01062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:00.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.304+0000 7ff1ef7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 0x7ff1f01062f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35084/0 (socket says 172.21.15.94:35084) 2024-02-17T11:49:00.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.304+0000 7ff1ef7fe700 1 -- 172.21.15.94:0/3116215835 learned_addr learned my addr 172.21.15.94:0/3116215835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:00.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.304+0000 7ff1ef7fe700 1 -- 172.21.15.94:0/3116215835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f007af70 con 0x7ff1f0105ed0 2024-02-17T11:49:00.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.304+0000 7ff1ef7fe700 1 --2- 172.21.15.94:0/3116215835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 0x7ff1f01062f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00098a0 tx=0x7ff1e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4bf3623f8139b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:00.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.304+0000 7ff1ee7fc700 1 -- 172.21.15.94:0/3116215835 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1e0004160 con 0x7ff1f0105ed0 2024-02-17T11:49:00.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.304+0000 7ff1ee7fc700 1 -- 172.21.15.94:0/3116215835 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1e00042c0 con 0x7ff1f0105ed0 2024-02-17T11:49:00.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.304+0000 7ff1ee7fc700 1 -- 172.21.15.94:0/3116215835 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1e0011a80 con 0x7ff1f0105ed0 2024-02-17T11:49:00.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.305+0000 7ff1f5d00700 1 -- 172.21.15.94:0/3116215835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 msgr2=0x7ff1f01062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:00.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.305+0000 7ff1f5d00700 1 --2- 172.21.15.94:0/3116215835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 0x7ff1f01062f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff1e00098a0 tx=0x7ff1e0009bb0 comp rx=0 tx=0).stop 2024-02-17T11:49:00.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.305+0000 7ff1f5d00700 1 -- 172.21.15.94:0/3116215835 shutdown_connections 2024-02-17T11:49:00.151 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.305+0000 7ff1f5d00700 1 --2- 172.21.15.94:0/3116215835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 0x7ff1f01062f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:00.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.305+0000 7ff1f5d00700 1 -- 172.21.15.94:0/3116215835 >> 172.21.15.94:0/3116215835 conn(0x7ff1f0101ad0 msgr2=0x7ff1f0103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:00.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.305+0000 7ff1f5d00700 1 -- 172.21.15.94:0/3116215835 shutdown_connections 2024-02-17T11:49:00.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.305+0000 7ff1f5d00700 1 -- 172.21.15.94:0/3116215835 wait complete. 2024-02-17T11:49:00.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.306+0000 7ff1f5d00700 1 Processor -- start 2024-02-17T11:49:00.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.306+0000 7ff1f5d00700 1 -- start start 2024-02-17T11:49:00.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.306+0000 7ff1f5d00700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 0x7ff1f019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:00.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.306+0000 7ff1f5d00700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f019dae0 con 0x7ff1f0105ed0 2024-02-17T11:49:00.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.306+0000 7ff1ef7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 0x7ff1f019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:00.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.306+0000 7ff1ef7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 0x7ff1f019d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35098/0 (socket says 172.21.15.94:35098) 2024-02-17T11:49:00.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.306+0000 7ff1ef7fe700 1 -- 172.21.15.94:0/1504010949 learned_addr learned my addr 172.21.15.94:0/1504010949 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:00.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.306+0000 7ff1ef7fe700 1 -- 172.21.15.94:0/1504010949 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e0009550 con 0x7ff1f0105ed0 2024-02-17T11:49:00.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.306+0000 7ff1ef7fe700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 0x7ff1f019d5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0003f20 tx=0x7ff1e0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:00.152 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.307+0000 7ff1ecff9700 1 -- 172.21.15.94:0/1504010949 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1e00039f0 con 0x7ff1f0105ed0 2024-02-17T11:49:00.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.307+0000 7ff1ecff9700 1 -- 172.21.15.94:0/1504010949 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1e0003b50 con 0x7ff1f0105ed0 2024-02-17T11:49:00.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.307+0000 7ff1ecff9700 1 -- 172.21.15.94:0/1504010949 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1e001a6d0 con 0x7ff1f0105ed0 2024-02-17T11:49:00.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.307+0000 7ff1f5d00700 1 -- 172.21.15.94:0/1504010949 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f019dce0 con 0x7ff1f0105ed0 2024-02-17T11:49:00.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.307+0000 7ff1f5d00700 1 -- 172.21.15.94:0/1504010949 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f019e180 con 0x7ff1f0105ed0 2024-02-17T11:49:00.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.308+0000 7ff1ecff9700 1 -- 172.21.15.94:0/1504010949 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7ff1e001a940 con 0x7ff1f0105ed0 2024-02-17T11:49:00.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.308+0000 7ff1ecff9700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 0x7ff1d803b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:00.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.308+0000 7ff1ecff9700 1 -- 172.21.15.94:0/1504010949 --> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1d803bb10 con 0x7ff1d8038f40 2024-02-17T11:49:00.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.308+0000 7ff1ecff9700 1 -- 172.21.15.94:0/1504010949 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff1e004d2a0 con 0x7ff1f0105ed0 2024-02-17T11:49:00.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.308+0000 7ff1eeffd700 1 -- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 msgr2=0x7ff1d803b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2384585889 2024-02-17T11:49:00.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.308+0000 7ff1eeffd700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 0x7ff1d803b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T11:49:00.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.509+0000 7ff1eeffd700 1 -- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 msgr2=0x7ff1d803b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2384585889 2024-02-17T11:49:00.153 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.509+0000 7ff1eeffd700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 0x7ff1d803b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-17T11:49:00.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.910+0000 7ff1eeffd700 1 -- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 msgr2=0x7ff1d803b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2384585889 2024-02-17T11:49:00.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:54.910+0000 7ff1eeffd700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 0x7ff1d803b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-17T11:49:00.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:55.711+0000 7ff1eeffd700 1 -- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 msgr2=0x7ff1d803b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2384585889 2024-02-17T11:49:00.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:55.711+0000 7ff1eeffd700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 0x7ff1d803b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-17T11:49:00.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:57.312+0000 7ff1eeffd700 1 -- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 msgr2=0x7ff1d803b400 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/2384585889 2024-02-17T11:49:00.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:57.312+0000 7ff1eeffd700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 0x7ff1d803b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-17T11:49:00.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:58.556+0000 7ff1ecff9700 1 -- 172.21.15.94:0/1504010949 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mgrmap(e 10) v1 ==== 45712+0+0 (secure 0 0 0) 0x7ff1e002da50 con 0x7ff1f0105ed0 2024-02-17T11:49:00.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:58.556+0000 7ff1ecff9700 1 -- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 msgr2=0x7ff1d803b400 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T11:49:00.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:58.556+0000 7ff1ecff9700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 0x7ff1d803b400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:00.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.557+0000 7ff1ecff9700 1 -- 172.21.15.94:0/1504010949 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 11) v1 ==== 45839+0+0 (secure 0 0 0) 0x7ff1e004e6c0 con 0x7ff1f0105ed0 2024-02-17T11:49:00.154 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.557+0000 7ff1ecff9700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff1d803c9f0 0x7ff1d803ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:00.155 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.557+0000 7ff1ecff9700 1 -- 172.21.15.94:0/1504010949 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1d803bb10 con 0x7ff1d803c9f0 2024-02-17T11:49:00.155 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.558+0000 7ff1eeffd700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff1d803c9f0 0x7ff1d803ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:00.155 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.558+0000 7ff1eeffd700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff1d803c9f0 0x7ff1d803ede0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4003a10 tx=0x7ff1e4009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:00.155 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.559+0000 7ff1ecff9700 1 -- 172.21.15.94:0/1504010949 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7ff1d803bb10 con 0x7ff1d803c9f0 2024-02-17T11:49:00.155 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.566+0000 7ff1f5d00700 1 -- 172.21.15.94:0/1504010949 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff1f01075b0 con 0x7ff1d803c9f0 2024-02-17T11:49:00.155 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.566+0000 7ff1ecff9700 1 -- 172.21.15.94:0/1504010949 <== mgr.14152 v2:172.21.15.94:6800/1808883811 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7ff1f01075b0 con 0x7ff1d803c9f0 2024-02-17T11:49:00.155 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.566+0000 7ff1f5d00700 1 -- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff1d803c9f0 msgr2=0x7ff1d803ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:00.155 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.566+0000 7ff1f5d00700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff1d803c9f0 0x7ff1d803ede0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4003a10 tx=0x7ff1e4009270 comp rx=0 tx=0).stop 2024-02-17T11:49:00.155 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.566+0000 7ff1f5d00700 1 -- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 msgr2=0x7ff1f019d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:00.155 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.566+0000 7ff1f5d00700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 0x7ff1f019d5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0003f20 tx=0x7ff1e0011b00 comp rx=0 tx=0).stop 2024-02-17T11:49:00.155 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.566+0000 7ff1f5d00700 1 -- 172.21.15.94:0/1504010949 shutdown_connections 2024-02-17T11:49:00.155 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.566+0000 7ff1f5d00700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff1d803c9f0 0x7ff1d803ede0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:00.155 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.566+0000 7ff1f5d00700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff1f0105ed0 0x7ff1f019d5a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:00.156 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.566+0000 7ff1f5d00700 1 --2- 172.21.15.94:0/1504010949 >> [v2:172.21.15.94:6800/2384585889,v1:172.21.15.94:6801/2384585889] conn(0x7ff1d8038f40 0x7ff1d803b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:00.156 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.566+0000 7ff1f5d00700 1 -- 172.21.15.94:0/1504010949 >> 172.21.15.94:0/1504010949 conn(0x7ff1f0101ad0 msgr2=0x7ff1f0194330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:00.156 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.566+0000 7ff1f5d00700 1 -- 172.21.15.94:0/1504010949 shutdown_connections 2024-02-17T11:49:00.156 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:48:59.566+0000 7ff1f5d00700 1 -- 172.21.15.94:0/1504010949 wait complete. 2024-02-17T11:49:00.156 INFO:teuthology.orchestra.run.smithi094.stdout:mgr epoch 9 is available 2024-02-17T11:49:00.156 INFO:teuthology.orchestra.run.smithi094.stdout:Generating a dashboard self-signed certificate... 2024-02-17T11:49:00.928 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:00 smithi094 ceph-mon[32499]: [17/Feb/2024:11:48:59] ENGINE Bus STARTING 2024-02-17T11:49:00.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:00 smithi094 ceph-mon[32499]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-17T11:49:00.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:00 smithi094 ceph-mon[32499]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-17T11:49:00.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:00 smithi094 ceph-mon[32499]: [17/Feb/2024:11:48:59] ENGINE Serving on http://172.21.15.94:8765 2024-02-17T11:49:00.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:00 smithi094 ceph-mon[32499]: [17/Feb/2024:11:48:59] ENGINE Serving on https://172.21.15.94:7150 2024-02-17T11:49:00.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:00 smithi094 ceph-mon[32499]: [17/Feb/2024:11:48:59] ENGINE Bus STARTED 2024-02-17T11:49:00.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:00 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:01.781 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-17T11:49:01.781 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.963+0000 7f9c7db39700 1 Processor -- start 2024-02-17T11:49:01.783 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.963+0000 7f9c7db39700 1 -- start start 2024-02-17T11:49:01.783 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.964+0000 7f9c7db39700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 0x7f9c78107a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:01.783 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.964+0000 7f9c7db39700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c78108030 con 0x7f9c78107640 2024-02-17T11:49:01.783 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.964+0000 7f9c777fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 0x7f9c78107a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:01.783 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.964+0000 7f9c777fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 0x7f9c78107a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54502/0 (socket says 172.21.15.94:54502) 2024-02-17T11:49:01.783 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.964+0000 7f9c777fe700 1 -- 172.21.15.94:0/3727566989 learned_addr learned my addr 172.21.15.94:0/3727566989 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:01.783 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.964+0000 7f9c777fe700 1 -- 172.21.15.94:0/3727566989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c781087f0 con 0x7f9c78107640 2024-02-17T11:49:01.783 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.964+0000 7f9c777fe700 1 --2- 172.21.15.94:0/3727566989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 0x7f9c78107a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c680098a0 tx=0x7f9c68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b369bfc6f9bfc1d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:01.783 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.965+0000 7f9c767fc700 1 -- 172.21.15.94:0/3727566989 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c68004160 con 0x7f9c78107640 2024-02-17T11:49:01.783 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.965+0000 7f9c767fc700 1 -- 172.21.15.94:0/3727566989 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c680042c0 con 0x7f9c78107640 2024-02-17T11:49:01.783 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.965+0000 7f9c767fc700 1 -- 172.21.15.94:0/3727566989 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c68011a80 con 0x7f9c78107640 2024-02-17T11:49:01.784 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.965+0000 7f9c7db39700 1 -- 172.21.15.94:0/3727566989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 msgr2=0x7f9c78107a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:01.784 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.965+0000 7f9c7db39700 1 --2- 172.21.15.94:0/3727566989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 0x7f9c78107a60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9c680098a0 tx=0x7f9c68009bb0 comp rx=0 tx=0).stop 2024-02-17T11:49:01.784 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.965+0000 7f9c7db39700 1 -- 172.21.15.94:0/3727566989 shutdown_connections 2024-02-17T11:49:01.784 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.965+0000 7f9c7db39700 1 --2- 172.21.15.94:0/3727566989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 0x7f9c78107a60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.965+0000 7f9c7db39700 1 -- 172.21.15.94:0/3727566989 >> 172.21.15.94:0/3727566989 conn(0x7f9c78102ca0 msgr2=0x7f9c781050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.965+0000 7f9c7db39700 1 -- 172.21.15.94:0/3727566989 shutdown_connections 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.966+0000 7f9c7db39700 1 -- 172.21.15.94:0/3727566989 wait complete. 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.966+0000 7f9c7db39700 1 Processor -- start 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.967+0000 7f9c7db39700 1 -- start start 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.967+0000 7f9c7db39700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 0x7f9c7819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.967+0000 7f9c7db39700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7819b910 con 0x7f9c78107640 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.967+0000 7f9c777fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 0x7f9c7819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.967+0000 7f9c777fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 0x7f9c7819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54518/0 (socket says 172.21.15.94:54518) 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.967+0000 7f9c777fe700 1 -- 172.21.15.94:0/843606835 learned_addr learned my addr 172.21.15.94:0/843606835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.967+0000 7f9c777fe700 1 -- 172.21.15.94:0/843606835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c68009550 con 0x7f9c78107640 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.967+0000 7f9c777fe700 1 --2- 172.21.15.94:0/843606835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 0x7f9c7819b3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c68000c00 tx=0x7f9c68011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.968+0000 7f9c74ff9700 1 -- 172.21.15.94:0/843606835 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c680039e0 con 0x7f9c78107640 2024-02-17T11:49:01.785 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.968+0000 7f9c7db39700 1 -- 172.21.15.94:0/843606835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7819bb10 con 0x7f9c78107640 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.968+0000 7f9c74ff9700 1 -- 172.21.15.94:0/843606835 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9c68003b40 con 0x7f9c78107640 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.968+0000 7f9c74ff9700 1 -- 172.21.15.94:0/843606835 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c68024820 con 0x7f9c78107640 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.968+0000 7f9c7db39700 1 -- 172.21.15.94:0/843606835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7819bfb0 con 0x7f9c78107640 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.969+0000 7f9c74ff9700 1 -- 172.21.15.94:0/843606835 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 11) v1 ==== 45839+0+0 (secure 0 0 0) 0x7f9c68024980 con 0x7f9c78107640 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.969+0000 7f9c7db39700 1 -- 172.21.15.94:0/843606835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7810ba20 con 0x7f9c78107640 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.969+0000 7f9c74ff9700 1 --2- 172.21.15.94:0/843606835 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f9c60038e10 0x7f9c6003b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.969+0000 7f9c74ff9700 1 -- 172.21.15.94:0/843606835 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9c6804cf70 con 0x7f9c78107640 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.969+0000 7f9c76ffd700 1 --2- 172.21.15.94:0/843606835 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f9c60038e10 0x7f9c6003b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.970+0000 7f9c76ffd700 1 --2- 172.21.15.94:0/843606835 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f9c60038e10 0x7f9c6003b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c006fd0 tx=0x7f9c6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:00.974+0000 7f9c74ff9700 1 -- 172.21.15.94:0/843606835 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c68080d20 con 0x7f9c78107640 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:01.180+0000 7f9c7db39700 1 -- 172.21.15.94:0/843606835 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c78103e50 con 0x7f9c60038e10 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:01.258+0000 7f9c74ff9700 1 -- 172.21.15.94:0/843606835 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f9c78103e50 con 0x7f9c60038e10 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:01.260+0000 7f9c7db39700 1 -- 172.21.15.94:0/843606835 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f9c60038e10 msgr2=0x7f9c6003b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:01.786 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:01.260+0000 7f9c7db39700 1 --2- 172.21.15.94:0/843606835 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f9c60038e10 0x7f9c6003b2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c006fd0 tx=0x7f9c6c006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:01.787 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:01.260+0000 7f9c7db39700 1 -- 172.21.15.94:0/843606835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 msgr2=0x7f9c7819b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:01.787 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:01.260+0000 7f9c7db39700 1 --2- 172.21.15.94:0/843606835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 0x7f9c7819b3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9c68000c00 tx=0x7f9c68011b00 comp rx=0 tx=0).stop 2024-02-17T11:49:01.787 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:01.260+0000 7f9c7db39700 1 -- 172.21.15.94:0/843606835 shutdown_connections 2024-02-17T11:49:01.787 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:01.260+0000 7f9c7db39700 1 --2- 172.21.15.94:0/843606835 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f9c60038e10 0x7f9c6003b2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:01.787 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:01.260+0000 7f9c7db39700 1 --2- 172.21.15.94:0/843606835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9c78107640 0x7f9c7819b3d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:01.787 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:01.260+0000 7f9c7db39700 1 -- 172.21.15.94:0/843606835 >> 172.21.15.94:0/843606835 conn(0x7f9c78102ca0 msgr2=0x7f9c78103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:01.787 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:01.260+0000 7f9c7db39700 1 -- 172.21.15.94:0/843606835 shutdown_connections 2024-02-17T11:49:01.787 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:01.260+0000 7f9c7db39700 1 -- 172.21.15.94:0/843606835 wait complete. 2024-02-17T11:49:01.787 INFO:teuthology.orchestra.run.smithi094.stdout:Creating initial admin user... 2024-02-17T11:49:02.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:02 smithi094 ceph-mon[32499]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:49:02.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:02.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:02.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:02 smithi094 ceph-mon[32499]: mgrmap e12: x(active, since 2s) 2024-02-17T11:49:02.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:03.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:03 smithi094 ceph-mon[32499]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:49:03.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:03 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:04.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$qVOkxx45YXXe9DQIb9vHJOEq9UadumBUmxYCgbsl6OgmgNCla/aEa", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708170542, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-17T11:49:04.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.225+0000 7f2446ff3700 1 Processor -- start 2024-02-17T11:49:04.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.225+0000 7f2446ff3700 1 -- start start 2024-02-17T11:49:04.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.225+0000 7f2446ff3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 0x7f244010a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:04.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.225+0000 7f2446ff3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244007aed0 con 0x7f2440108060 2024-02-17T11:49:04.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.226+0000 7f2444d8f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 0x7f244010a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:04.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.226+0000 7f2444d8f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 0x7f244010a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54522/0 (socket says 172.21.15.94:54522) 2024-02-17T11:49:04.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.226+0000 7f2444d8f700 1 -- 172.21.15.94:0/3697758626 learned_addr learned my addr 172.21.15.94:0/3697758626 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:04.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.226+0000 7f2444d8f700 1 -- 172.21.15.94:0/3697758626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244010a990 con 0x7f2440108060 2024-02-17T11:49:04.010 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.226+0000 7f2444d8f700 1 --2- 172.21.15.94:0/3697758626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 0x7f244010a450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f24340098a0 tx=0x7f2434009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b161f0aae479d592 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:04.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.226+0000 7f243f7fe700 1 -- 172.21.15.94:0/3697758626 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2434004160 con 0x7f2440108060 2024-02-17T11:49:04.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.226+0000 7f243f7fe700 1 -- 172.21.15.94:0/3697758626 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24340042c0 con 0x7f2440108060 2024-02-17T11:49:04.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.226+0000 7f243f7fe700 1 -- 172.21.15.94:0/3697758626 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2434011a80 con 0x7f2440108060 2024-02-17T11:49:04.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.227+0000 7f2446ff3700 1 -- 172.21.15.94:0/3697758626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 msgr2=0x7f244010a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:04.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.227+0000 7f2446ff3700 1 --2- 172.21.15.94:0/3697758626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 0x7f244010a450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f24340098a0 tx=0x7f2434009bb0 comp rx=0 tx=0).stop 2024-02-17T11:49:04.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.227+0000 7f2446ff3700 1 -- 172.21.15.94:0/3697758626 shutdown_connections 2024-02-17T11:49:04.011 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.227+0000 7f2446ff3700 1 --2- 172.21.15.94:0/3697758626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 0x7f244010a450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:04.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.227+0000 7f2446ff3700 1 -- 172.21.15.94:0/3697758626 >> 172.21.15.94:0/3697758626 conn(0x7f2440101ad0 msgr2=0x7f2440103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:04.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.227+0000 7f2446ff3700 1 -- 172.21.15.94:0/3697758626 shutdown_connections 2024-02-17T11:49:04.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.227+0000 7f2446ff3700 1 -- 172.21.15.94:0/3697758626 wait complete. 2024-02-17T11:49:04.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.228+0000 7f2446ff3700 1 Processor -- start 2024-02-17T11:49:04.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.228+0000 7f2446ff3700 1 -- start start 2024-02-17T11:49:04.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.228+0000 7f2446ff3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 0x7f24401078e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:04.012 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.228+0000 7f2446ff3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2440107e20 con 0x7f2440108060 2024-02-17T11:49:04.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.228+0000 7f2444d8f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 0x7f24401078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:04.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.228+0000 7f2444d8f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 0x7f24401078e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54536/0 (socket says 172.21.15.94:54536) 2024-02-17T11:49:04.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.228+0000 7f2444d8f700 1 -- 172.21.15.94:0/3424023583 learned_addr learned my addr 172.21.15.94:0/3424023583 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:04.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.228+0000 7f2444d8f700 1 -- 172.21.15.94:0/3424023583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2434009550 con 0x7f2440108060 2024-02-17T11:49:04.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.228+0000 7f2444d8f700 1 --2- 172.21.15.94:0/3424023583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 0x7f24401078e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2434003f20 tx=0x7f2434011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:04.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.228+0000 7f243dffb700 1 -- 172.21.15.94:0/3424023583 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24340039f0 con 0x7f2440108060 2024-02-17T11:49:04.013 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.229+0000 7f243dffb700 1 -- 172.21.15.94:0/3424023583 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2434003b50 con 0x7f2440108060 2024-02-17T11:49:04.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.229+0000 7f243dffb700 1 -- 172.21.15.94:0/3424023583 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f243401a6d0 con 0x7f2440108060 2024-02-17T11:49:04.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.229+0000 7f2446ff3700 1 -- 172.21.15.94:0/3424023583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2440106010 con 0x7f2440108060 2024-02-17T11:49:04.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.229+0000 7f2446ff3700 1 -- 172.21.15.94:0/3424023583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24401064b0 con 0x7f2440108060 2024-02-17T11:49:04.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.230+0000 7f243dffb700 1 -- 172.21.15.94:0/3424023583 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f2434024430 con 0x7f2440108060 2024-02-17T11:49:04.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.230+0000 7f243dffb700 1 --2- 172.21.15.94:0/3424023583 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2430038ee0 0x7f243003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:04.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.230+0000 7f243dffb700 1 -- 172.21.15.94:0/3424023583 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f243404d490 con 0x7f2440108060 2024-02-17T11:49:04.014 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.230+0000 7f243ffff700 1 --2- 172.21.15.94:0/3424023583 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2430038ee0 0x7f243003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:04.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.230+0000 7f243ffff700 1 --2- 172.21.15.94:0/3424023583 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2430038ee0 0x7f243003b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f242c006fd0 tx=0x7f242c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:04.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.230+0000 7f2446ff3700 1 -- 172.21.15.94:0/3424023583 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24400632a0 con 0x7f2440108060 2024-02-17T11:49:04.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.235+0000 7f243dffb700 1 -- 172.21.15.94:0/3424023583 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f24340249a0 con 0x7f2440108060 2024-02-17T11:49:04.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.459+0000 7f2446ff3700 1 -- 172.21.15.94:0/3424023583 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f24400630c0 con 0x7f2430038ee0 2024-02-17T11:49:04.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.683+0000 7f243dffb700 1 -- 172.21.15.94:0/3424023583 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f24400630c0 con 0x7f2430038ee0 2024-02-17T11:49:04.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.686+0000 7f2446ff3700 1 -- 172.21.15.94:0/3424023583 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2430038ee0 msgr2=0x7f243003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:04.015 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.686+0000 7f2446ff3700 1 --2- 172.21.15.94:0/3424023583 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2430038ee0 0x7f243003b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f242c006fd0 tx=0x7f242c006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:04.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.686+0000 7f2446ff3700 1 -- 172.21.15.94:0/3424023583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 msgr2=0x7f24401078e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:04.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.686+0000 7f2446ff3700 1 --2- 172.21.15.94:0/3424023583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 0x7f24401078e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2434003f20 tx=0x7f2434011b00 comp rx=0 tx=0).stop 2024-02-17T11:49:04.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.686+0000 7f2446ff3700 1 -- 172.21.15.94:0/3424023583 shutdown_connections 2024-02-17T11:49:04.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.686+0000 7f2446ff3700 1 --2- 172.21.15.94:0/3424023583 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2430038ee0 0x7f243003b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:04.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.686+0000 7f2446ff3700 1 --2- 172.21.15.94:0/3424023583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2440108060 0x7f24401078e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:04.016 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.686+0000 7f2446ff3700 1 -- 172.21.15.94:0/3424023583 >> 172.21.15.94:0/3424023583 conn(0x7f2440101ad0 msgr2=0x7f24401025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:04.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.686+0000 7f2446ff3700 1 -- 172.21.15.94:0/3424023583 shutdown_connections 2024-02-17T11:49:04.017 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:02.686+0000 7f2446ff3700 1 -- 172.21.15.94:0/3424023583 wait complete. 2024-02-17T11:49:04.017 INFO:teuthology.orchestra.run.smithi094.stdout:Fetching dashboard port number... 2024-02-17T11:49:04.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:04 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/396020527' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-17T11:49:05.172 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stdout 8443 2024-02-17T11:49:05.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.442+0000 7f5b6f67a700 1 Processor -- start 2024-02-17T11:49:05.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.442+0000 7f5b6f67a700 1 -- start start 2024-02-17T11:49:05.173 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.443+0000 7f5b6f67a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 0x7f5b681068f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:05.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.443+0000 7f5b6f67a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b68106ec0 con 0x7f5b681064d0 2024-02-17T11:49:05.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.443+0000 7f5b6d416700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 0x7f5b681068f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:05.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.443+0000 7f5b6d416700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 0x7f5b681068f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44358/0 (socket says 172.21.15.94:44358) 2024-02-17T11:49:05.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.443+0000 7f5b6d416700 1 -- 172.21.15.94:0/2944249576 learned_addr learned my addr 172.21.15.94:0/2944249576 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:05.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.443+0000 7f5b6d416700 1 -- 172.21.15.94:0/2944249576 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b681076d0 con 0x7f5b681064d0 2024-02-17T11:49:05.174 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.443+0000 7f5b6d416700 1 --2- 172.21.15.94:0/2944249576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 0x7f5b681068f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5b640098a0 tx=0x7f5b64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910597d348517df0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:05.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.443+0000 7f5b5ffff700 1 -- 172.21.15.94:0/2944249576 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b64004160 con 0x7f5b681064d0 2024-02-17T11:49:05.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.443+0000 7f5b5ffff700 1 -- 172.21.15.94:0/2944249576 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b640042c0 con 0x7f5b681064d0 2024-02-17T11:49:05.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.443+0000 7f5b5ffff700 1 -- 172.21.15.94:0/2944249576 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b64011a80 con 0x7f5b681064d0 2024-02-17T11:49:05.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.444+0000 7f5b6f67a700 1 -- 172.21.15.94:0/2944249576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 msgr2=0x7f5b681068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:05.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.444+0000 7f5b6f67a700 1 --2- 172.21.15.94:0/2944249576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 0x7f5b681068f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5b640098a0 tx=0x7f5b64009bb0 comp rx=0 tx=0).stop 2024-02-17T11:49:05.175 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.444+0000 7f5b6f67a700 1 -- 172.21.15.94:0/2944249576 shutdown_connections 2024-02-17T11:49:05.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.444+0000 7f5b6f67a700 1 --2- 172.21.15.94:0/2944249576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 0x7f5b681068f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:05.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.444+0000 7f5b6f67a700 1 -- 172.21.15.94:0/2944249576 >> 172.21.15.94:0/2944249576 conn(0x7f5b68101ad0 msgr2=0x7f5b68103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:05.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.444+0000 7f5b6f67a700 1 -- 172.21.15.94:0/2944249576 shutdown_connections 2024-02-17T11:49:05.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.444+0000 7f5b6f67a700 1 -- 172.21.15.94:0/2944249576 wait complete. 2024-02-17T11:49:05.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b6f67a700 1 Processor -- start 2024-02-17T11:49:05.176 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b6f67a700 1 -- start start 2024-02-17T11:49:05.177 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b6f67a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 0x7f5b6819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:05.177 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b6f67a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6819b9c0 con 0x7f5b681064d0 2024-02-17T11:49:05.177 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b6d416700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 0x7f5b6819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:05.177 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b6d416700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 0x7f5b6819b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44374/0 (socket says 172.21.15.94:44374) 2024-02-17T11:49:05.177 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b6d416700 1 -- 172.21.15.94:0/396020527 learned_addr learned my addr 172.21.15.94:0/396020527 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:05.177 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b6d416700 1 -- 172.21.15.94:0/396020527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b64009550 con 0x7f5b681064d0 2024-02-17T11:49:05.177 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b6d416700 1 --2- 172.21.15.94:0/396020527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 0x7f5b6819b480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5b64003f20 tx=0x7f5b64011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:05.178 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b5e7fc700 1 -- 172.21.15.94:0/396020527 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b640038b0 con 0x7f5b681064d0 2024-02-17T11:49:05.178 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b5e7fc700 1 -- 172.21.15.94:0/396020527 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5b64003a10 con 0x7f5b681064d0 2024-02-17T11:49:05.178 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b6f67a700 1 -- 172.21.15.94:0/396020527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6819bbc0 con 0x7f5b681064d0 2024-02-17T11:49:05.178 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b5e7fc700 1 -- 172.21.15.94:0/396020527 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b6401a5c0 con 0x7f5b681064d0 2024-02-17T11:49:05.178 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.445+0000 7f5b6f67a700 1 -- 172.21.15.94:0/396020527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6819c060 con 0x7f5b681064d0 2024-02-17T11:49:05.178 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.446+0000 7f5b5e7fc700 1 -- 172.21.15.94:0/396020527 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f5b6401a800 con 0x7f5b681064d0 2024-02-17T11:49:05.179 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.446+0000 7f5b5e7fc700 1 --2- 172.21.15.94:0/396020527 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5b54038f40 0x7f5b5403b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:05.179 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.446+0000 7f5b6f67a700 1 -- 172.21.15.94:0/396020527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b68195570 con 0x7f5b681064d0 2024-02-17T11:49:05.179 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.446+0000 7f5b5e7fc700 1 -- 172.21.15.94:0/396020527 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5b6404d320 con 0x7f5b681064d0 2024-02-17T11:49:05.179 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.446+0000 7f5b6cc15700 1 --2- 172.21.15.94:0/396020527 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5b54038f40 0x7f5b5403b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:05.179 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.447+0000 7f5b6cc15700 1 --2- 172.21.15.94:0/396020527 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5b54038f40 0x7f5b5403b400 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5b58006fd0 tx=0x7f5b58006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:05.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.451+0000 7f5b5e7fc700 1 -- 172.21.15.94:0/396020527 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b64018530 con 0x7f5b681064d0 2024-02-17T11:49:05.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.644+0000 7f5b6f67a700 1 -- 172.21.15.94:0/396020527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f5b6810aac0 con 0x7f5b681064d0 2024-02-17T11:49:05.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.644+0000 7f5b5e7fc700 1 -- 172.21.15.94:0/396020527 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f5b6400f2e0 con 0x7f5b681064d0 2024-02-17T11:49:05.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.646+0000 7f5b6f67a700 1 -- 172.21.15.94:0/396020527 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5b54038f40 msgr2=0x7f5b5403b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:05.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.646+0000 7f5b6f67a700 1 --2- 172.21.15.94:0/396020527 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5b54038f40 0x7f5b5403b400 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5b58006fd0 tx=0x7f5b58006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:05.180 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.646+0000 7f5b6f67a700 1 -- 172.21.15.94:0/396020527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 msgr2=0x7f5b6819b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:05.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.646+0000 7f5b6f67a700 1 --2- 172.21.15.94:0/396020527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 0x7f5b6819b480 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5b64003f20 tx=0x7f5b64011b00 comp rx=0 tx=0).stop 2024-02-17T11:49:05.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.646+0000 7f5b6f67a700 1 -- 172.21.15.94:0/396020527 shutdown_connections 2024-02-17T11:49:05.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.646+0000 7f5b6f67a700 1 --2- 172.21.15.94:0/396020527 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5b54038f40 0x7f5b5403b400 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:05.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.646+0000 7f5b6f67a700 1 --2- 172.21.15.94:0/396020527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5b681064d0 0x7f5b6819b480 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:05.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.646+0000 7f5b6f67a700 1 -- 172.21.15.94:0/396020527 >> 172.21.15.94:0/396020527 conn(0x7f5b68101ad0 msgr2=0x7f5b681029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:05.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.647+0000 7f5b6f67a700 1 -- 172.21.15.94:0/396020527 shutdown_connections 2024-02-17T11:49:05.181 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:04.647+0000 7f5b6f67a700 1 -- 172.21.15.94:0/396020527 wait complete. 2024-02-17T11:49:05.182 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-17T11:49:05.182 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout disabled 2024-02-17T11:49:05.185 INFO:teuthology.orchestra.run.smithi094.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-17T11:49:05.186 INFO:teuthology.orchestra.run.smithi094.stdout:systemctl: stdout inactive 2024-02-17T11:49:05.186 INFO:teuthology.orchestra.run.smithi094.stdout:firewalld.service is not enabled 2024-02-17T11:49:05.186 INFO:teuthology.orchestra.run.smithi094.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-17T11:49:05.187 INFO:teuthology.orchestra.run.smithi094.stdout:Ceph Dashboard is now available at: 2024-02-17T11:49:05.187 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:49:05.187 INFO:teuthology.orchestra.run.smithi094.stdout: URL: https://smithi094.front.sepia.ceph.com:8443/ 2024-02-17T11:49:05.187 INFO:teuthology.orchestra.run.smithi094.stdout: User: admin 2024-02-17T11:49:05.188 INFO:teuthology.orchestra.run.smithi094.stdout: Password: jgjeh9adys 2024-02-17T11:49:05.188 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:49:05.188 INFO:teuthology.orchestra.run.smithi094.stdout:Saving cluster configuration to /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config directory 2024-02-17T11:49:05.189 INFO:teuthology.orchestra.run.smithi094.stdout:Enabling autotune for osd_memory_target 2024-02-17T11:49:05.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:05 smithi094 ceph-mon[32499]: mgrmap e13: x(active, since 6s) 2024-02-17T11:49:06.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.610+0000 7f4246183700 1 Processor -- start 2024-02-17T11:49:06.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.610+0000 7f4246183700 1 -- start start 2024-02-17T11:49:06.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.610+0000 7f4246183700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 0x7f424007b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:06.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.610+0000 7f4246183700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424007b840 con 0x7f424007ce50 2024-02-17T11:49:06.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.611+0000 7f423f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 0x7f424007b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:06.326 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.611+0000 7f423f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 0x7f424007b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44384/0 (socket says 172.21.15.94:44384) 2024-02-17T11:49:06.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.611+0000 7f423f7fe700 1 -- 172.21.15.94:0/3864566101 learned_addr learned my addr 172.21.15.94:0/3864566101 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:06.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.611+0000 7f423f7fe700 1 -- 172.21.15.94:0/3864566101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424007b980 con 0x7f424007ce50 2024-02-17T11:49:06.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.611+0000 7f423f7fe700 1 --2- 172.21.15.94:0/3864566101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 0x7f424007b300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42300098a0 tx=0x7f4230009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3fcd4dda820b44 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:06.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.611+0000 7f423effd700 1 -- 172.21.15.94:0/3864566101 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4230004160 con 0x7f424007ce50 2024-02-17T11:49:06.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.611+0000 7f423effd700 1 -- 172.21.15.94:0/3864566101 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42300042c0 con 0x7f424007ce50 2024-02-17T11:49:06.327 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.611+0000 7f423effd700 1 -- 172.21.15.94:0/3864566101 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4230011a80 con 0x7f424007ce50 2024-02-17T11:49:06.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.612+0000 7f4246183700 1 -- 172.21.15.94:0/3864566101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 msgr2=0x7f424007b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:06.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.612+0000 7f4246183700 1 --2- 172.21.15.94:0/3864566101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 0x7f424007b300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f42300098a0 tx=0x7f4230009bb0 comp rx=0 tx=0).stop 2024-02-17T11:49:06.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.612+0000 7f4246183700 1 -- 172.21.15.94:0/3864566101 shutdown_connections 2024-02-17T11:49:06.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.612+0000 7f4246183700 1 --2- 172.21.15.94:0/3864566101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 0x7f424007b300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:06.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.612+0000 7f4246183700 1 -- 172.21.15.94:0/3864566101 >> 172.21.15.94:0/3864566101 conn(0x7f4240102c40 msgr2=0x7f4240105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:06.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.612+0000 7f4246183700 1 -- 172.21.15.94:0/3864566101 shutdown_connections 2024-02-17T11:49:06.328 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.612+0000 7f4246183700 1 -- 172.21.15.94:0/3864566101 wait complete. 2024-02-17T11:49:06.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.612+0000 7f4246183700 1 Processor -- start 2024-02-17T11:49:06.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.612+0000 7f4246183700 1 -- start start 2024-02-17T11:49:06.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.613+0000 7f4246183700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 0x7f424019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:06.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.613+0000 7f4246183700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424019dab0 con 0x7f424007ce50 2024-02-17T11:49:06.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.613+0000 7f423f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 0x7f424019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:06.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.613+0000 7f423f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 0x7f424019d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44388/0 (socket says 172.21.15.94:44388) 2024-02-17T11:49:06.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.613+0000 7f423f7fe700 1 -- 172.21.15.94:0/2864861519 learned_addr learned my addr 172.21.15.94:0/2864861519 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:06.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.613+0000 7f423f7fe700 1 -- 172.21.15.94:0/2864861519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4230009550 con 0x7f424007ce50 2024-02-17T11:49:06.329 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.613+0000 7f423f7fe700 1 --2- 172.21.15.94:0/2864861519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 0x7f424019d570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4230009130 tx=0x7f4230011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:06.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.613+0000 7f423d7fa700 1 -- 172.21.15.94:0/2864861519 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4230003b20 con 0x7f424007ce50 2024-02-17T11:49:06.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.613+0000 7f423d7fa700 1 -- 172.21.15.94:0/2864861519 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f423001a910 con 0x7f424007ce50 2024-02-17T11:49:06.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.613+0000 7f423d7fa700 1 -- 172.21.15.94:0/2864861519 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4230024930 con 0x7f424007ce50 2024-02-17T11:49:06.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.613+0000 7f4246183700 1 -- 172.21.15.94:0/2864861519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424019dcb0 con 0x7f424007ce50 2024-02-17T11:49:06.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.613+0000 7f4246183700 1 -- 172.21.15.94:0/2864861519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424019e150 con 0x7f424007ce50 2024-02-17T11:49:06.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.614+0000 7f423d7fa700 1 -- 172.21.15.94:0/2864861519 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f4230024af0 con 0x7f424007ce50 2024-02-17T11:49:06.330 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.614+0000 7f423d7fa700 1 --2- 172.21.15.94:0/2864861519 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f422c039030 0x7f422c03b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:06.331 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.614+0000 7f423d7fa700 1 -- 172.21.15.94:0/2864861519 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f423004d470 con 0x7f424007ce50 2024-02-17T11:49:06.331 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.614+0000 7f4236ffd700 1 --2- 172.21.15.94:0/2864861519 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f422c039030 0x7f422c03b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:06.331 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.614+0000 7f4246183700 1 -- 172.21.15.94:0/2864861519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424010d9c0 con 0x7f424007ce50 2024-02-17T11:49:06.331 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.615+0000 7f4236ffd700 1 --2- 172.21.15.94:0/2864861519 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f422c039030 0x7f422c03b4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4228006fd0 tx=0x7f4228006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:06.331 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.619+0000 7f423d7fa700 1 -- 172.21.15.94:0/2864861519 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4230024da0 con 0x7f424007ce50 2024-02-17T11:49:06.331 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.812+0000 7f4246183700 1 -- 172.21.15.94:0/2864861519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f424010dba0 con 0x7f424007ce50 2024-02-17T11:49:06.332 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.813+0000 7f423d7fa700 1 -- 172.21.15.94:0/2864861519 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f423001adf0 con 0x7f424007ce50 2024-02-17T11:49:06.332 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.815+0000 7f4246183700 1 -- 172.21.15.94:0/2864861519 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f422c039030 msgr2=0x7f422c03b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:06.332 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.815+0000 7f4246183700 1 --2- 172.21.15.94:0/2864861519 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f422c039030 0x7f422c03b4f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4228006fd0 tx=0x7f4228006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:06.332 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.815+0000 7f4246183700 1 -- 172.21.15.94:0/2864861519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 msgr2=0x7f424019d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:06.332 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.815+0000 7f4246183700 1 --2- 172.21.15.94:0/2864861519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 0x7f424019d570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4230009130 tx=0x7f4230011b00 comp rx=0 tx=0).stop 2024-02-17T11:49:06.332 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.815+0000 7f4246183700 1 -- 172.21.15.94:0/2864861519 shutdown_connections 2024-02-17T11:49:06.333 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.815+0000 7f4246183700 1 --2- 172.21.15.94:0/2864861519 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f422c039030 0x7f422c03b4f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:06.333 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.815+0000 7f4246183700 1 --2- 172.21.15.94:0/2864861519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f424007ce50 0x7f424019d570 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:06.333 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.815+0000 7f4246183700 1 -- 172.21.15.94:0/2864861519 >> 172.21.15.94:0/2864861519 conn(0x7f4240102c40 msgr2=0x7f4240103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:06.333 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.815+0000 7f4246183700 1 -- 172.21.15.94:0/2864861519 shutdown_connections 2024-02-17T11:49:06.333 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:05.815+0000 7f4246183700 1 -- 172.21.15.94:0/2864861519 wait complete. 2024-02-17T11:49:07.620 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.749+0000 7f97c1654700 1 Processor -- start 2024-02-17T11:49:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.749+0000 7f97c1654700 1 -- start start 2024-02-17T11:49:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.750+0000 7f97c1654700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 0x7f97bc106910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.750+0000 7f97c1654700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc106ee0 con 0x7f97bc1064f0 2024-02-17T11:49:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.750+0000 7f97baffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 0x7f97bc106910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.750+0000 7f97baffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 0x7f97bc106910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44390/0 (socket says 172.21.15.94:44390) 2024-02-17T11:49:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.750+0000 7f97baffd700 1 -- 172.21.15.94:0/1491352688 learned_addr learned my addr 172.21.15.94:0/1491352688 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.750+0000 7f97baffd700 1 -- 172.21.15.94:0/1491352688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc1076f0 con 0x7f97bc1064f0 2024-02-17T11:49:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.750+0000 7f97baffd700 1 --2- 172.21.15.94:0/1491352688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 0x7f97bc106910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0098a0 tx=0x7f97ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7959618550bf45cf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.750+0000 7f97b9ffb700 1 -- 172.21.15.94:0/1491352688 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97ac004160 con 0x7f97bc1064f0 2024-02-17T11:49:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.750+0000 7f97b9ffb700 1 -- 172.21.15.94:0/1491352688 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97ac0042c0 con 0x7f97bc1064f0 2024-02-17T11:49:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.750+0000 7f97b9ffb700 1 -- 172.21.15.94:0/1491352688 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97ac011a80 con 0x7f97bc1064f0 2024-02-17T11:49:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.751+0000 7f97c1654700 1 -- 172.21.15.94:0/1491352688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 msgr2=0x7f97bc106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:07.621 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.751+0000 7f97c1654700 1 --2- 172.21.15.94:0/1491352688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 0x7f97bc106910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0098a0 tx=0x7f97ac009bb0 comp rx=0 tx=0).stop 2024-02-17T11:49:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.751+0000 7f97c1654700 1 -- 172.21.15.94:0/1491352688 shutdown_connections 2024-02-17T11:49:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.751+0000 7f97c1654700 1 --2- 172.21.15.94:0/1491352688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 0x7f97bc106910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.751+0000 7f97c1654700 1 -- 172.21.15.94:0/1491352688 >> 172.21.15.94:0/1491352688 conn(0x7f97bc101ad0 msgr2=0x7f97bc103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.751+0000 7f97c1654700 1 -- 172.21.15.94:0/1491352688 shutdown_connections 2024-02-17T11:49:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.751+0000 7f97c1654700 1 -- 172.21.15.94:0/1491352688 wait complete. 2024-02-17T11:49:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.752+0000 7f97c1654700 1 Processor -- start 2024-02-17T11:49:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.752+0000 7f97c1654700 1 -- start start 2024-02-17T11:49:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.752+0000 7f97c1654700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 0x7f97bc19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.752+0000 7f97c1654700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc19b960 con 0x7f97bc1064f0 2024-02-17T11:49:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.752+0000 7f97baffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 0x7f97bc19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.752+0000 7f97baffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 0x7f97bc19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44398/0 (socket says 172.21.15.94:44398) 2024-02-17T11:49:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.752+0000 7f97baffd700 1 -- 172.21.15.94:0/2267426841 learned_addr learned my addr 172.21.15.94:0/2267426841 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:07.622 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.752+0000 7f97baffd700 1 -- 172.21.15.94:0/2267426841 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac009550 con 0x7f97bc1064f0 2024-02-17T11:49:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.752+0000 7f97baffd700 1 --2- 172.21.15.94:0/2267426841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 0x7f97bc19b420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f97ac003f20 tx=0x7f97ac011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.753+0000 7f97a3fff700 1 -- 172.21.15.94:0/2267426841 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97ac0038b0 con 0x7f97bc1064f0 2024-02-17T11:49:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.753+0000 7f97a3fff700 1 -- 172.21.15.94:0/2267426841 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97ac003a10 con 0x7f97bc1064f0 2024-02-17T11:49:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.753+0000 7f97c1654700 1 -- 172.21.15.94:0/2267426841 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97bc19bb60 con 0x7f97bc1064f0 2024-02-17T11:49:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.753+0000 7f97a3fff700 1 -- 172.21.15.94:0/2267426841 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97ac01a5c0 con 0x7f97bc1064f0 2024-02-17T11:49:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.753+0000 7f97c1654700 1 -- 172.21.15.94:0/2267426841 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97bc19c000 con 0x7f97bc1064f0 2024-02-17T11:49:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.754+0000 7f97a3fff700 1 -- 172.21.15.94:0/2267426841 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f97ac01a780 con 0x7f97bc1064f0 2024-02-17T11:49:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.754+0000 7f97a3fff700 1 --2- 172.21.15.94:0/2267426841 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f97a4039030 0x7f97a403b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.754+0000 7f97c1654700 1 -- 172.21.15.94:0/2267426841 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97bc195590 con 0x7f97bc1064f0 2024-02-17T11:49:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.754+0000 7f97a3fff700 1 -- 172.21.15.94:0/2267426841 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f97ac04d250 con 0x7f97bc1064f0 2024-02-17T11:49:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.754+0000 7f97ba7fc700 1 --2- 172.21.15.94:0/2267426841 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f97a4039030 0x7f97a403b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.754+0000 7f97ba7fc700 1 --2- 172.21.15.94:0/2267426841 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f97a4039030 0x7f97a403b4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f97b0006fd0 tx=0x7f97b0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:07.623 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:06.759+0000 7f97a3fff700 1 -- 172.21.15.94:0/2267426841 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f97ac018c20 con 0x7f97bc1064f0 2024-02-17T11:49:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:07.056+0000 7f97c1654700 1 -- 172.21.15.94:0/2267426841 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f97bc10aae0 con 0x7f97bc1064f0 2024-02-17T11:49:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:07.058+0000 7f97a3fff700 1 -- 172.21.15.94:0/2267426841 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f97ac018350 con 0x7f97bc1064f0 2024-02-17T11:49:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:07.060+0000 7f97c1654700 1 -- 172.21.15.94:0/2267426841 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f97a4039030 msgr2=0x7f97a403b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:07.060+0000 7f97c1654700 1 --2- 172.21.15.94:0/2267426841 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f97a4039030 0x7f97a403b4f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f97b0006fd0 tx=0x7f97b0006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:07.060+0000 7f97c1654700 1 -- 172.21.15.94:0/2267426841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 msgr2=0x7f97bc19b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:07.060+0000 7f97c1654700 1 --2- 172.21.15.94:0/2267426841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 0x7f97bc19b420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f97ac003f20 tx=0x7f97ac011b00 comp rx=0 tx=0).stop 2024-02-17T11:49:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:07.060+0000 7f97c1654700 1 -- 172.21.15.94:0/2267426841 shutdown_connections 2024-02-17T11:49:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:07.060+0000 7f97c1654700 1 --2- 172.21.15.94:0/2267426841 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f97a4039030 0x7f97a403b4f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:07.060+0000 7f97c1654700 1 --2- 172.21.15.94:0/2267426841 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f97bc1064f0 0x7f97bc19b420 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:07.060+0000 7f97c1654700 1 -- 172.21.15.94:0/2267426841 >> 172.21.15.94:0/2267426841 conn(0x7f97bc101ad0 msgr2=0x7f97bc102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:07.060+0000 7f97c1654700 1 -- 172.21.15.94:0/2267426841 shutdown_connections 2024-02-17T11:49:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr 2024-02-17T11:49:07.060+0000 7f97c1654700 1 -- 172.21.15.94:0/2267426841 wait complete. 2024-02-17T11:49:07.624 INFO:teuthology.orchestra.run.smithi094.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:Or, if you are only running a single cluster on this host: 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout: ceph telemetry on 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout:For more information see: 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:49:07.625 INFO:teuthology.orchestra.run.smithi094.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-17T11:49:07.626 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:49:07.626 INFO:teuthology.orchestra.run.smithi094.stdout:Bootstrap complete. 2024-02-17T11:49:07.657 INFO:tasks.cephadm:Fetching config... 2024-02-17T11:49:07.657 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:49:07.657 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-17T11:49:07.672 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-17T11:49:07.672 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:49:07.672 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-17T11:49:07.728 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-17T11:49:07.728 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:49:07.728 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-17T11:49:07.794 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-17T11:49:07.794 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:49:07.794 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-17T11:49:07.849 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-17T11:49:07.850 DEBUG:teuthology.orchestra.run.smithi094:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMgNM5FW33h6Al09k3y8SyiiGgkOnK3CxNC3Z0JgyBhRzXLQGcm51ODYu2IyozwChiAl0lJ/cDrPyC8fcDkHw2/949919qppPqY+UbFrNsAg5F1jAhtJuCS4843DIPRSS8PnINudVKXoWlsAoSkZgcUf1BJCviz73FnKKyZuWqr9HwpRagc9Z0s7+P19Fc7zp8suUCQ1DHOL/6F3oZlU9Dcv0QPHPq3kwn3/kp/hqIz8mwIxUJYzOLcXVRIeftcJc2FUna52UIo/kYnIqvmvqbaHJPfYwZxlBrz7lCpqutPg1gSXpRfB7pGtimVArqWFUX0QGYOwGE8AXl9RTdkaTm6R6B7x3HoEzoRkwlVJ1klZoj6YA/Pr1j3QWkS54PrvpK2jNgwAcQ9sg/BtOeYB9Hnhw0e8fk4w0i68kVTl1h1nVbHxylBJpv27Yjiro4PU0kUnQU09ydogrYLsx6mVGhH8L/fOQEH+DgJIUMSmpVbh1o5mA0F9Yq0OwEy6eGTgk= ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-17T11:49:07.940 INFO:teuthology.orchestra.run.smithi094.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMgNM5FW33h6Al09k3y8SyiiGgkOnK3CxNC3Z0JgyBhRzXLQGcm51ODYu2IyozwChiAl0lJ/cDrPyC8fcDkHw2/949919qppPqY+UbFrNsAg5F1jAhtJuCS4843DIPRSS8PnINudVKXoWlsAoSkZgcUf1BJCviz73FnKKyZuWqr9HwpRagc9Z0s7+P19Fc7zp8suUCQ1DHOL/6F3oZlU9Dcv0QPHPq3kwn3/kp/hqIz8mwIxUJYzOLcXVRIeftcJc2FUna52UIo/kYnIqvmvqbaHJPfYwZxlBrz7lCpqutPg1gSXpRfB7pGtimVArqWFUX0QGYOwGE8AXl9RTdkaTm6R6B7x3HoEzoRkwlVJ1klZoj6YA/Pr1j3QWkS54PrvpK2jNgwAcQ9sg/BtOeYB9Hnhw0e8fk4w0i68kVTl1h1nVbHxylBJpv27Yjiro4PU0kUnQU09ydogrYLsx6mVGhH8L/fOQEH+DgJIUMSmpVbh1o5mA0F9Yq0OwEy6eGTgk= ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:49:07.955 DEBUG:teuthology.orchestra.run.smithi125:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMgNM5FW33h6Al09k3y8SyiiGgkOnK3CxNC3Z0JgyBhRzXLQGcm51ODYu2IyozwChiAl0lJ/cDrPyC8fcDkHw2/949919qppPqY+UbFrNsAg5F1jAhtJuCS4843DIPRSS8PnINudVKXoWlsAoSkZgcUf1BJCviz73FnKKyZuWqr9HwpRagc9Z0s7+P19Fc7zp8suUCQ1DHOL/6F3oZlU9Dcv0QPHPq3kwn3/kp/hqIz8mwIxUJYzOLcXVRIeftcJc2FUna52UIo/kYnIqvmvqbaHJPfYwZxlBrz7lCpqutPg1gSXpRfB7pGtimVArqWFUX0QGYOwGE8AXl9RTdkaTm6R6B7x3HoEzoRkwlVJ1klZoj6YA/Pr1j3QWkS54PrvpK2jNgwAcQ9sg/BtOeYB9Hnhw0e8fk4w0i68kVTl1h1nVbHxylBJpv27Yjiro4PU0kUnQU09ydogrYLsx6mVGhH8L/fOQEH+DgJIUMSmpVbh1o5mA0F9Yq0OwEy6eGTgk= ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-17T11:49:08.002 INFO:teuthology.orchestra.run.smithi125.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMgNM5FW33h6Al09k3y8SyiiGgkOnK3CxNC3Z0JgyBhRzXLQGcm51ODYu2IyozwChiAl0lJ/cDrPyC8fcDkHw2/949919qppPqY+UbFrNsAg5F1jAhtJuCS4843DIPRSS8PnINudVKXoWlsAoSkZgcUf1BJCviz73FnKKyZuWqr9HwpRagc9Z0s7+P19Fc7zp8suUCQ1DHOL/6F3oZlU9Dcv0QPHPq3kwn3/kp/hqIz8mwIxUJYzOLcXVRIeftcJc2FUna52UIo/kYnIqvmvqbaHJPfYwZxlBrz7lCpqutPg1gSXpRfB7pGtimVArqWFUX0QGYOwGE8AXl9RTdkaTm6R6B7x3HoEzoRkwlVJ1klZoj6YA/Pr1j3QWkS54PrvpK2jNgwAcQ9sg/BtOeYB9Hnhw0e8fk4w0i68kVTl1h1nVbHxylBJpv27Yjiro4PU0kUnQU09ydogrYLsx6mVGhH8L/fOQEH+DgJIUMSmpVbh1o5mA0F9Yq0OwEy6eGTgk= ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:49:08.018 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMgNM5FW33h6Al09k3y8SyiiGgkOnK3CxNC3Z0JgyBhRzXLQGcm51ODYu2IyozwChiAl0lJ/cDrPyC8fcDkHw2/949919qppPqY+UbFrNsAg5F1jAhtJuCS4843DIPRSS8PnINudVKXoWlsAoSkZgcUf1BJCviz73FnKKyZuWqr9HwpRagc9Z0s7+P19Fc7zp8suUCQ1DHOL/6F3oZlU9Dcv0QPHPq3kwn3/kp/hqIz8mwIxUJYzOLcXVRIeftcJc2FUna52UIo/kYnIqvmvqbaHJPfYwZxlBrz7lCpqutPg1gSXpRfB7pGtimVArqWFUX0QGYOwGE8AXl9RTdkaTm6R6B7x3HoEzoRkwlVJ1klZoj6YA/Pr1j3QWkS54PrvpK2jNgwAcQ9sg/BtOeYB9Hnhw0e8fk4w0i68kVTl1h1nVbHxylBJpv27Yjiro4PU0kUnQU09ydogrYLsx6mVGhH8L/fOQEH+DgJIUMSmpVbh1o5mA0F9Yq0OwEy6eGTgk= ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-17T11:49:08.063 INFO:teuthology.orchestra.run.smithi189.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMgNM5FW33h6Al09k3y8SyiiGgkOnK3CxNC3Z0JgyBhRzXLQGcm51ODYu2IyozwChiAl0lJ/cDrPyC8fcDkHw2/949919qppPqY+UbFrNsAg5F1jAhtJuCS4843DIPRSS8PnINudVKXoWlsAoSkZgcUf1BJCviz73FnKKyZuWqr9HwpRagc9Z0s7+P19Fc7zp8suUCQ1DHOL/6F3oZlU9Dcv0QPHPq3kwn3/kp/hqIz8mwIxUJYzOLcXVRIeftcJc2FUna52UIo/kYnIqvmvqbaHJPfYwZxlBrz7lCpqutPg1gSXpRfB7pGtimVArqWFUX0QGYOwGE8AXl9RTdkaTm6R6B7x3HoEzoRkwlVJ1klZoj6YA/Pr1j3QWkS54PrvpK2jNgwAcQ9sg/BtOeYB9Hnhw0e8fk4w0i68kVTl1h1nVbHxylBJpv27Yjiro4PU0kUnQU09ydogrYLsx6mVGhH8L/fOQEH+DgJIUMSmpVbh1o5mA0F9Yq0OwEy6eGTgk= ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:49:08.079 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-17T11:49:08.184 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:08 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2267426841' entity='client.admin' 2024-02-17T11:49:08.314 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:49:10.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.761+0000 7f273bfbd700 1 -- 172.21.15.94:0/3574468836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27341005a0 msgr2=0x7f27341009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:10.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.761+0000 7f273bfbd700 1 --2- 172.21.15.94:0/3574468836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27341005a0 0x7f27341009c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2730009910 tx=0x7f2730009c20 comp rx=0 tx=0).stop 2024-02-17T11:49:10.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.761+0000 7f273bfbd700 1 -- 172.21.15.94:0/3574468836 shutdown_connections 2024-02-17T11:49:10.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.761+0000 7f273bfbd700 1 --2- 172.21.15.94:0/3574468836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27341005a0 0x7f27341009c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:10.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.761+0000 7f273bfbd700 1 -- 172.21.15.94:0/3574468836 >> 172.21.15.94:0/3574468836 conn(0x7f27340fbb40 msgr2=0x7f27340fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:10.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.761+0000 7f273bfbd700 1 -- 172.21.15.94:0/3574468836 shutdown_connections 2024-02-17T11:49:10.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.761+0000 7f273bfbd700 1 -- 172.21.15.94:0/3574468836 wait complete. 2024-02-17T11:49:10.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.761+0000 7f273bfbd700 1 Processor -- start 2024-02-17T11:49:10.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.761+0000 7f273bfbd700 1 -- start start 2024-02-17T11:49:10.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.762+0000 7f273bfbd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27341005a0 0x7f27340fda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:10.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.762+0000 7f273bfbd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27340fdf40 con 0x7f27341005a0 2024-02-17T11:49:10.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.762+0000 7f2739d59700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27341005a0 0x7f27340fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:10.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.762+0000 7f2739d59700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27341005a0 0x7f27340fda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44408/0 (socket says 172.21.15.94:44408) 2024-02-17T11:49:10.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.762+0000 7f2739d59700 1 -- 172.21.15.94:0/360359685 learned_addr learned my addr 172.21.15.94:0/360359685 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:10.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.762+0000 7f2739d59700 1 -- 172.21.15.94:0/360359685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27300095f0 con 0x7f27341005a0 2024-02-17T11:49:10.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.762+0000 7f2739d59700 1 --2- 172.21.15.94:0/360359685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27341005a0 0x7f27340fda00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2730005850 tx=0x7f2730004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:10.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.762+0000 7f272affd700 1 -- 172.21.15.94:0/360359685 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f273001d070 con 0x7f27341005a0 2024-02-17T11:49:10.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.762+0000 7f272affd700 1 -- 172.21.15.94:0/360359685 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f273000be40 con 0x7f27341005a0 2024-02-17T11:49:10.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.762+0000 7f273bfbd700 1 -- 172.21.15.94:0/360359685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27340fc130 con 0x7f27341005a0 2024-02-17T11:49:10.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.762+0000 7f272affd700 1 -- 172.21.15.94:0/360359685 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2730022ea0 con 0x7f27341005a0 2024-02-17T11:49:10.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.762+0000 7f273bfbd700 1 -- 172.21.15.94:0/360359685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27340fc5d0 con 0x7f27341005a0 2024-02-17T11:49:10.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.763+0000 7f272affd700 1 -- 172.21.15.94:0/360359685 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f273000f4d0 con 0x7f27341005a0 2024-02-17T11:49:10.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.763+0000 7f272affd700 1 --2- 172.21.15.94:0/360359685 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2720038fe0 0x7f272003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:10.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.763+0000 7f272affd700 1 -- 172.21.15.94:0/360359685 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f273004d200 con 0x7f27341005a0 2024-02-17T11:49:10.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.763+0000 7f2739558700 1 --2- 172.21.15.94:0/360359685 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2720038fe0 0x7f272003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:10.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.764+0000 7f273bfbd700 1 -- 172.21.15.94:0/360359685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2734106d30 con 0x7f27341005a0 2024-02-17T11:49:10.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.764+0000 7f2739558700 1 --2- 172.21.15.94:0/360359685 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2720038fe0 0x7f272003b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2724006fd0 tx=0x7f2724006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:10.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.770+0000 7f272affd700 1 -- 172.21.15.94:0/360359685 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2730027020 con 0x7f27341005a0 2024-02-17T11:49:10.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.977+0000 7f273bfbd700 1 -- 172.21.15.94:0/360359685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f27341066f0 con 0x7f27341005a0 2024-02-17T11:49:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.979+0000 7f272affd700 1 -- 172.21.15.94:0/360359685 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2730017440 con 0x7f27341005a0 2024-02-17T11:49:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.982+0000 7f273bfbd700 1 -- 172.21.15.94:0/360359685 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2720038fe0 msgr2=0x7f272003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.982+0000 7f273bfbd700 1 --2- 172.21.15.94:0/360359685 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2720038fe0 0x7f272003b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2724006fd0 tx=0x7f2724006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.982+0000 7f273bfbd700 1 -- 172.21.15.94:0/360359685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27341005a0 msgr2=0x7f27340fda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.982+0000 7f273bfbd700 1 --2- 172.21.15.94:0/360359685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27341005a0 0x7f27340fda00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2730005850 tx=0x7f2730004ff0 comp rx=0 tx=0).stop 2024-02-17T11:49:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.982+0000 7f273bfbd700 1 -- 172.21.15.94:0/360359685 shutdown_connections 2024-02-17T11:49:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.982+0000 7f273bfbd700 1 --2- 172.21.15.94:0/360359685 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2720038fe0 0x7f272003b4a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.982+0000 7f273bfbd700 1 --2- 172.21.15.94:0/360359685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f27341005a0 0x7f27340fda00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.982+0000 7f273bfbd700 1 -- 172.21.15.94:0/360359685 >> 172.21.15.94:0/360359685 conn(0x7f27340fbb40 msgr2=0x7f2734069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.982+0000 7f273bfbd700 1 -- 172.21.15.94:0/360359685 shutdown_connections 2024-02-17T11:49:10.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:10.982+0000 7f273bfbd700 1 -- 172.21.15.94:0/360359685 wait complete. 2024-02-17T11:49:12.154 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-17T11:49:12.154 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-17T11:49:12.389 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:49:12.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:11 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/360359685' entity='client.admin' 2024-02-17T11:49:12.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:12.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:12.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi094", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:49:12.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:12.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:49:12.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:13.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.795+0000 7f696af4f700 1 -- 172.21.15.94:0/2568419369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69641016c0 msgr2=0x7f6964101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:13.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.795+0000 7f696af4f700 1 --2- 172.21.15.94:0/2568419369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69641016c0 0x7f6964101ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6958009910 tx=0x7f6958009c20 comp rx=0 tx=0).stop 2024-02-17T11:49:13.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.795+0000 7f696af4f700 1 -- 172.21.15.94:0/2568419369 shutdown_connections 2024-02-17T11:49:13.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.795+0000 7f696af4f700 1 --2- 172.21.15.94:0/2568419369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69641016c0 0x7f6964101ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:13.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.795+0000 7f696af4f700 1 -- 172.21.15.94:0/2568419369 >> 172.21.15.94:0/2568419369 conn(0x7f69640fcc60 msgr2=0x7f69640ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:13.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.795+0000 7f696af4f700 1 -- 172.21.15.94:0/2568419369 shutdown_connections 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.795+0000 7f696af4f700 1 -- 172.21.15.94:0/2568419369 wait complete. 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.796+0000 7f696af4f700 1 Processor -- start 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.796+0000 7f696af4f700 1 -- start start 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.796+0000 7f696af4f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69641016c0 0x7f69640758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.796+0000 7f696af4f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6964075e20 con 0x7f69641016c0 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.796+0000 7f6968ceb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69641016c0 0x7f69640758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.796+0000 7f6968ceb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69641016c0 0x7f69640758e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44416/0 (socket says 172.21.15.94:44416) 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.796+0000 7f6968ceb700 1 -- 172.21.15.94:0/2720291479 learned_addr learned my addr 172.21.15.94:0/2720291479 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.796+0000 7f6968ceb700 1 -- 172.21.15.94:0/2720291479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69580095f0 con 0x7f69641016c0 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.797+0000 7f6968ceb700 1 --2- 172.21.15.94:0/2720291479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69641016c0 0x7f69640758e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6958005850 tx=0x7f6958004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.797+0000 7f6961ffb700 1 -- 172.21.15.94:0/2720291479 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f695801d070 con 0x7f69641016c0 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.797+0000 7f6961ffb700 1 -- 172.21.15.94:0/2720291479 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f695800be40 con 0x7f69641016c0 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.797+0000 7f696af4f700 1 -- 172.21.15.94:0/2720291479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6964074060 con 0x7f69641016c0 2024-02-17T11:49:13.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.797+0000 7f6961ffb700 1 -- 172.21.15.94:0/2720291479 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6958022ea0 con 0x7f69641016c0 2024-02-17T11:49:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.797+0000 7f696af4f700 1 -- 172.21.15.94:0/2720291479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6964074500 con 0x7f69641016c0 2024-02-17T11:49:13.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.798+0000 7f6961ffb700 1 -- 172.21.15.94:0/2720291479 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f6958022670 con 0x7f69641016c0 2024-02-17T11:49:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.798+0000 7f696af4f700 1 -- 172.21.15.94:0/2720291479 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6964107e50 con 0x7f69641016c0 2024-02-17T11:49:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.798+0000 7f6961ffb700 1 --2- 172.21.15.94:0/2720291479 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f6954038fe0 0x7f695403b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.798+0000 7f6961ffb700 1 -- 172.21.15.94:0/2720291479 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f695804d080 con 0x7f69641016c0 2024-02-17T11:49:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.798+0000 7f6963fff700 1 --2- 172.21.15.94:0/2720291479 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f6954038fe0 0x7f695403b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:13.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.798+0000 7f6963fff700 1 --2- 172.21.15.94:0/2720291479 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f6954038fe0 0x7f695403b4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6950006fd0 tx=0x7f6950006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:13.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.803+0000 7f6961ffb700 1 -- 172.21.15.94:0/2720291479 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f695800b960 con 0x7f69641016c0 2024-02-17T11:49:13.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:13.998+0000 7f696af4f700 1 -- 172.21.15.94:0/2720291479 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f6964074740 con 0x7f6954038fe0 2024-02-17T11:49:14.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:14.002+0000 7f6961ffb700 1 -- 172.21.15.94:0/2720291479 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6964074740 con 0x7f6954038fe0 2024-02-17T11:49:14.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:14.004+0000 7f696af4f700 1 -- 172.21.15.94:0/2720291479 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f6954038fe0 msgr2=0x7f695403b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:14.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:14.004+0000 7f696af4f700 1 --2- 172.21.15.94:0/2720291479 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f6954038fe0 0x7f695403b4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6950006fd0 tx=0x7f6950006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:14.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:14.004+0000 7f696af4f700 1 -- 172.21.15.94:0/2720291479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69641016c0 msgr2=0x7f69640758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:14.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:14.004+0000 7f696af4f700 1 --2- 172.21.15.94:0/2720291479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69641016c0 0x7f69640758e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6958005850 tx=0x7f6958004ff0 comp rx=0 tx=0).stop 2024-02-17T11:49:14.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:14.004+0000 7f696af4f700 1 -- 172.21.15.94:0/2720291479 shutdown_connections 2024-02-17T11:49:14.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:14.004+0000 7f696af4f700 1 --2- 172.21.15.94:0/2720291479 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f6954038fe0 0x7f695403b4a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:14.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:14.004+0000 7f696af4f700 1 --2- 172.21.15.94:0/2720291479 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69641016c0 0x7f69640758e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:14.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:14.004+0000 7f696af4f700 1 -- 172.21.15.94:0/2720291479 >> 172.21.15.94:0/2720291479 conn(0x7f69640fcc60 msgr2=0x7f69640fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:14.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:14.004+0000 7f696af4f700 1 -- 172.21.15.94:0/2720291479 shutdown_connections 2024-02-17T11:49:14.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:14.004+0000 7f696af4f700 1 -- 172.21.15.94:0/2720291479 wait complete. 2024-02-17T11:49:14.776 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi125 2024-02-17T11:49:14.776 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-17T11:49:14.776 DEBUG:teuthology.orchestra.run.smithi125:> dd of=/etc/ceph/ceph.conf 2024-02-17T11:49:14.795 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-17T11:49:14.795 DEBUG:teuthology.orchestra.run.smithi125:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:49:14.851 INFO:tasks.cephadm:Adding host smithi125 to orchestrator... 2024-02-17T11:49:14.852 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch host add smithi125 2024-02-17T11:49:15.089 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:49:15.309 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:15 smithi094 ceph-mon[32499]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:49:15.310 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:15 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:15.310 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:15 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:49:15.310 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:15 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:15.310 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:15 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:49:15.310 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:15 smithi094 ceph-mon[32499]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-17T11:49:15.310 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:15 smithi094 ceph-mon[32499]: Updating smithi094:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:49:15.310 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:15 smithi094 ceph-mon[32499]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:49:15.310 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:15 smithi094 ceph-mon[32499]: Updating smithi094:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:49:15.310 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:15 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:15.310 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:15 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:15.310 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:15 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.504+0000 7fa7a741b700 1 -- 172.21.15.94:0/2779342822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0107700 msgr2=0x7fa7a0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.504+0000 7fa7a741b700 1 --2- 172.21.15.94:0/2779342822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0107700 0x7fa7a0107ae0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa79c009910 tx=0x7fa79c009c20 comp rx=0 tx=0).stop 2024-02-17T11:49:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.504+0000 7fa7a741b700 1 -- 172.21.15.94:0/2779342822 shutdown_connections 2024-02-17T11:49:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.504+0000 7fa7a741b700 1 --2- 172.21.15.94:0/2779342822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0107700 0x7fa7a0107ae0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.504+0000 7fa7a741b700 1 -- 172.21.15.94:0/2779342822 >> 172.21.15.94:0/2779342822 conn(0x7fa7a00fb7f0 msgr2=0x7fa7a00fdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.504+0000 7fa7a741b700 1 -- 172.21.15.94:0/2779342822 shutdown_connections 2024-02-17T11:49:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.504+0000 7fa7a741b700 1 -- 172.21.15.94:0/2779342822 wait complete. 2024-02-17T11:49:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.505+0000 7fa7a741b700 1 Processor -- start 2024-02-17T11:49:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.505+0000 7fa7a741b700 1 -- start start 2024-02-17T11:49:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.505+0000 7fa7a741b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0107700 0x7fa7a0196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.505+0000 7fa7a741b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0197400 con 0x7fa7a0107700 2024-02-17T11:49:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.505+0000 7fa7a51b7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0107700 0x7fa7a0196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.505+0000 7fa7a51b7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0107700 0x7fa7a0196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39766/0 (socket says 172.21.15.94:39766) 2024-02-17T11:49:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.505+0000 7fa7a51b7700 1 -- 172.21.15.94:0/2616763864 learned_addr learned my addr 172.21.15.94:0/2616763864 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.505+0000 7fa7a51b7700 1 -- 172.21.15.94:0/2616763864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c0095f0 con 0x7fa7a0107700 2024-02-17T11:49:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.505+0000 7fa7a51b7700 1 --2- 172.21.15.94:0/2616763864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0107700 0x7fa7a0196ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa79c004f40 tx=0x7fa79c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.506+0000 7fa7967fc700 1 -- 172.21.15.94:0/2616763864 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa79c01d070 con 0x7fa7a0107700 2024-02-17T11:49:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.506+0000 7fa7967fc700 1 -- 172.21.15.94:0/2616763864 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa79c00bbd0 con 0x7fa7a0107700 2024-02-17T11:49:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.506+0000 7fa7a741b700 1 -- 172.21.15.94:0/2616763864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a0197660 con 0x7fa7a0107700 2024-02-17T11:49:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.506+0000 7fa7967fc700 1 -- 172.21.15.94:0/2616763864 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa79c003900 con 0x7fa7a0107700 2024-02-17T11:49:16.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.506+0000 7fa7a741b700 1 -- 172.21.15.94:0/2616763864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a0197b60 con 0x7fa7a0107700 2024-02-17T11:49:16.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.506+0000 7fa7a741b700 1 -- 172.21.15.94:0/2616763864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a00632a0 con 0x7fa7a0107700 2024-02-17T11:49:16.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.506+0000 7fa7967fc700 1 -- 172.21.15.94:0/2616763864 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fa79c003bc0 con 0x7fa7a0107700 2024-02-17T11:49:16.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.507+0000 7fa7967fc700 1 --2- 172.21.15.94:0/2616763864 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa78c038f80 0x7fa78c03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:16.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.507+0000 7fa7967fc700 1 -- 172.21.15.94:0/2616763864 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa79c04cd50 con 0x7fa7a0107700 2024-02-17T11:49:16.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.507+0000 7fa7a49b6700 1 --2- 172.21.15.94:0/2616763864 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa78c038f80 0x7fa78c03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:16.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.507+0000 7fa7a49b6700 1 --2- 172.21.15.94:0/2616763864 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa78c038f80 0x7fa78c03b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa790006fd0 tx=0x7fa790006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:16.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.511+0000 7fa7967fc700 1 -- 172.21.15.94:0/2616763864 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa79c050030 con 0x7fa7a0107700 2024-02-17T11:49:16.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:16.711+0000 7fa7a741b700 1 -- 172.21.15.94:0/2616763864 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi125", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7a0197b60 con 0x7fa78c038f80 2024-02-17T11:49:17.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:16 smithi094 ceph-mon[32499]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi125", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:49:18.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:17 smithi094 ceph-mon[32499]: Deploying cephadm binary to smithi125 2024-02-17T11:49:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:19 smithi094 ceph-mon[32499]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:20.935 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi125' with addr '172.21.15.125' 2024-02-17T11:49:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:20.935+0000 7fa7967fc700 1 -- 172.21.15.94:0/2616763864 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa7a0197b60 con 0x7fa78c038f80 2024-02-17T11:49:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:20.938+0000 7fa7a741b700 1 -- 172.21.15.94:0/2616763864 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa78c038f80 msgr2=0x7fa78c03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:20.938+0000 7fa7a741b700 1 --2- 172.21.15.94:0/2616763864 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa78c038f80 0x7fa78c03b440 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa790006fd0 tx=0x7fa790006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:20.938+0000 7fa7a741b700 1 -- 172.21.15.94:0/2616763864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0107700 msgr2=0x7fa7a0196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:20.938+0000 7fa7a741b700 1 --2- 172.21.15.94:0/2616763864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0107700 0x7fa7a0196ec0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa79c004f40 tx=0x7fa79c005020 comp rx=0 tx=0).stop 2024-02-17T11:49:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:20.938+0000 7fa7a741b700 1 -- 172.21.15.94:0/2616763864 shutdown_connections 2024-02-17T11:49:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:20.938+0000 7fa7a741b700 1 --2- 172.21.15.94:0/2616763864 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa78c038f80 0x7fa78c03b440 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:20.938+0000 7fa7a741b700 1 --2- 172.21.15.94:0/2616763864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7a0107700 0x7fa7a0196ec0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:20.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:20.938+0000 7fa7a741b700 1 -- 172.21.15.94:0/2616763864 >> 172.21.15.94:0/2616763864 conn(0x7fa7a00fb7f0 msgr2=0x7fa7a00fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:20.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:20.938+0000 7fa7a741b700 1 -- 172.21.15.94:0/2616763864 shutdown_connections 2024-02-17T11:49:20.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:20.938+0000 7fa7a741b700 1 -- 172.21.15.94:0/2616763864 wait complete. 2024-02-17T11:49:21.508 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-17T11:49:21.744 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:49:21.866 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:21 smithi094 ceph-mon[32499]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:21.866 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:21.866 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:49:21.866 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:22.778 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:22 smithi094 ceph-mon[32499]: Added host smithi125 2024-02-17T11:49:22.778 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:22 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:23.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.257+0000 7faf24907700 1 -- 172.21.15.94:0/280158372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf20107660 msgr2=0x7faf20107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.257+0000 7faf24907700 1 --2- 172.21.15.94:0/280158372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf20107660 0x7faf20107a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7faf10009910 tx=0x7faf10009c20 comp rx=0 tx=0).stop 2024-02-17T11:49:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.258+0000 7faf24907700 1 -- 172.21.15.94:0/280158372 shutdown_connections 2024-02-17T11:49:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.258+0000 7faf24907700 1 --2- 172.21.15.94:0/280158372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf20107660 0x7faf20107a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.258+0000 7faf24907700 1 -- 172.21.15.94:0/280158372 >> 172.21.15.94:0/280158372 conn(0x7faf200fd1b0 msgr2=0x7faf200ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.258+0000 7faf24907700 1 -- 172.21.15.94:0/280158372 shutdown_connections 2024-02-17T11:49:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.258+0000 7faf24907700 1 -- 172.21.15.94:0/280158372 wait complete. 2024-02-17T11:49:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.258+0000 7faf24907700 1 Processor -- start 2024-02-17T11:49:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.258+0000 7faf24907700 1 -- start start 2024-02-17T11:49:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.258+0000 7faf24907700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf20107660 0x7faf20196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.259+0000 7faf24907700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf201974e0 con 0x7faf20107660 2024-02-17T11:49:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.259+0000 7faf1e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf20107660 0x7faf20196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.259+0000 7faf1e59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf20107660 0x7faf20196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39784/0 (socket says 172.21.15.94:39784) 2024-02-17T11:49:23.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.259+0000 7faf1e59c700 1 -- 172.21.15.94:0/253031104 learned_addr learned my addr 172.21.15.94:0/253031104 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:23.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.259+0000 7faf1e59c700 1 -- 172.21.15.94:0/253031104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf100095f0 con 0x7faf20107660 2024-02-17T11:49:23.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.259+0000 7faf1e59c700 1 --2- 172.21.15.94:0/253031104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf20107660 0x7faf20196fa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf10004f40 tx=0x7faf10005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:23.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.259+0000 7faf0f7fe700 1 -- 172.21.15.94:0/253031104 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf1001d070 con 0x7faf20107660 2024-02-17T11:49:23.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.259+0000 7faf0f7fe700 1 -- 172.21.15.94:0/253031104 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf1000bbd0 con 0x7faf20107660 2024-02-17T11:49:23.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.259+0000 7faf0f7fe700 1 -- 172.21.15.94:0/253031104 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf10003900 con 0x7faf20107660 2024-02-17T11:49:23.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.259+0000 7faf24907700 1 -- 172.21.15.94:0/253031104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf20197740 con 0x7faf20107660 2024-02-17T11:49:23.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.259+0000 7faf24907700 1 -- 172.21.15.94:0/253031104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf20197bc0 con 0x7faf20107660 2024-02-17T11:49:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.260+0000 7faf0f7fe700 1 -- 172.21.15.94:0/253031104 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7faf10003bc0 con 0x7faf20107660 2024-02-17T11:49:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.260+0000 7faf24907700 1 -- 172.21.15.94:0/253031104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf20102da0 con 0x7faf20107660 2024-02-17T11:49:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.260+0000 7faf0f7fe700 1 --2- 172.21.15.94:0/253031104 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faf08038f80 0x7faf0803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.260+0000 7faf0f7fe700 1 -- 172.21.15.94:0/253031104 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7faf1004cde0 con 0x7faf20107660 2024-02-17T11:49:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.260+0000 7faf1dd9b700 1 --2- 172.21.15.94:0/253031104 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faf08038f80 0x7faf0803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:23.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.261+0000 7faf1dd9b700 1 --2- 172.21.15.94:0/253031104 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faf08038f80 0x7faf0803b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faf14006fd0 tx=0x7faf14006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:23.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.265+0000 7faf0f7fe700 1 -- 172.21.15.94:0/253031104 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faf10029910 con 0x7faf20107660 2024-02-17T11:49:23.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.462+0000 7faf24907700 1 -- 172.21.15.94:0/253031104 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf2010bbb0 con 0x7faf08038f80 2024-02-17T11:49:23.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.464+0000 7faf0f7fe700 1 -- 172.21.15.94:0/253031104 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7faf2010bbb0 con 0x7faf08038f80 2024-02-17T11:49:23.463 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:49:23.463 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.125", "hostname": "smithi125", "labels": [], "status": ""}] 2024-02-17T11:49:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.466+0000 7faf24907700 1 -- 172.21.15.94:0/253031104 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faf08038f80 msgr2=0x7faf0803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.466+0000 7faf24907700 1 --2- 172.21.15.94:0/253031104 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faf08038f80 0x7faf0803b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faf14006fd0 tx=0x7faf14006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.466+0000 7faf24907700 1 -- 172.21.15.94:0/253031104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf20107660 msgr2=0x7faf20196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.466+0000 7faf24907700 1 --2- 172.21.15.94:0/253031104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf20107660 0x7faf20196fa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf10004f40 tx=0x7faf10005020 comp rx=0 tx=0).stop 2024-02-17T11:49:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.466+0000 7faf24907700 1 -- 172.21.15.94:0/253031104 shutdown_connections 2024-02-17T11:49:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.466+0000 7faf24907700 1 --2- 172.21.15.94:0/253031104 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faf08038f80 0x7faf0803b440 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.466+0000 7faf24907700 1 --2- 172.21.15.94:0/253031104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faf20107660 0x7faf20196fa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.466+0000 7faf24907700 1 -- 172.21.15.94:0/253031104 >> 172.21.15.94:0/253031104 conn(0x7faf200fd1b0 msgr2=0x7faf20109150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:23.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.466+0000 7faf24907700 1 -- 172.21.15.94:0/253031104 shutdown_connections 2024-02-17T11:49:23.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:23.466+0000 7faf24907700 1 -- 172.21.15.94:0/253031104 wait complete. 2024-02-17T11:49:23.777 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:23 smithi094 ceph-mon[32499]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:23.777 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:23 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:24.065 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi189 2024-02-17T11:49:24.065 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-17T11:49:24.065 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.conf 2024-02-17T11:49:24.083 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-17T11:49:24.084 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:49:24.141 INFO:tasks.cephadm:Adding host smithi189 to orchestrator... 2024-02-17T11:49:24.141 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch host add smithi189 2024-02-17T11:49:24.384 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:49:24.789 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:24 smithi094 ceph-mon[32499]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:49:25.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.783+0000 7fafe97a6700 1 -- 172.21.15.94:0/2305034231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4106600 msgr2=0x7fafe41069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:25.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.783+0000 7fafe97a6700 1 --2- 172.21.15.94:0/2305034231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4106600 0x7fafe41069e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009910 tx=0x7fafd4009c20 comp rx=0 tx=0).stop 2024-02-17T11:49:25.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.783+0000 7fafe97a6700 1 -- 172.21.15.94:0/2305034231 shutdown_connections 2024-02-17T11:49:25.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.783+0000 7fafe97a6700 1 --2- 172.21.15.94:0/2305034231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4106600 0x7fafe41069e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:25.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.783+0000 7fafe97a6700 1 -- 172.21.15.94:0/2305034231 >> 172.21.15.94:0/2305034231 conn(0x7fafe40fc020 msgr2=0x7fafe40fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:25.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.783+0000 7fafe97a6700 1 -- 172.21.15.94:0/2305034231 shutdown_connections 2024-02-17T11:49:25.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.783+0000 7fafe97a6700 1 -- 172.21.15.94:0/2305034231 wait complete. 2024-02-17T11:49:25.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.783+0000 7fafe97a6700 1 Processor -- start 2024-02-17T11:49:25.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.783+0000 7fafe97a6700 1 -- start start 2024-02-17T11:49:25.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.784+0000 7fafe97a6700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4106600 0x7fafe4195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:25.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.784+0000 7fafe97a6700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe406a6a0 con 0x7fafe4106600 2024-02-17T11:49:25.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.784+0000 7fafe2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4106600 0x7fafe4195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:25.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.784+0000 7fafe2ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4106600 0x7fafe4195b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40036/0 (socket says 172.21.15.94:40036) 2024-02-17T11:49:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.784+0000 7fafe2ffd700 1 -- 172.21.15.94:0/492380264 learned_addr learned my addr 172.21.15.94:0/492380264 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.784+0000 7fafe2ffd700 1 -- 172.21.15.94:0/492380264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd40095f0 con 0x7fafe4106600 2024-02-17T11:49:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.784+0000 7fafe2ffd700 1 --2- 172.21.15.94:0/492380264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4106600 0x7fafe4195b30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fafd4004f40 tx=0x7fafd4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.784+0000 7fafe0ff9700 1 -- 172.21.15.94:0/492380264 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafd401d070 con 0x7fafe4106600 2024-02-17T11:49:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.784+0000 7fafe0ff9700 1 -- 172.21.15.94:0/492380264 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafd400bc30 con 0x7fafe4106600 2024-02-17T11:49:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.784+0000 7fafe97a6700 1 -- 172.21.15.94:0/492380264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe4196070 con 0x7fafe4106600 2024-02-17T11:49:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.784+0000 7fafe0ff9700 1 -- 172.21.15.94:0/492380264 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafd4003970 con 0x7fafe4106600 2024-02-17T11:49:25.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.784+0000 7fafe97a6700 1 -- 172.21.15.94:0/492380264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe4196570 con 0x7fafe4106600 2024-02-17T11:49:25.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.785+0000 7fafe0ff9700 1 -- 172.21.15.94:0/492380264 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fafd4003be0 con 0x7fafe4106600 2024-02-17T11:49:25.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.785+0000 7fafe97a6700 1 -- 172.21.15.94:0/492380264 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe4101c70 con 0x7fafe4106600 2024-02-17T11:49:25.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.785+0000 7fafe0ff9700 1 --2- 172.21.15.94:0/492380264 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fafd0038fe0 0x7fafd003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:25.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.785+0000 7fafe0ff9700 1 -- 172.21.15.94:0/492380264 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fafd404ce90 con 0x7fafe4106600 2024-02-17T11:49:25.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.785+0000 7fafdbfff700 1 --2- 172.21.15.94:0/492380264 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fafd0038fe0 0x7fafd003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:25.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.786+0000 7fafdbfff700 1 --2- 172.21.15.94:0/492380264 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fafd0038fe0 0x7fafd003b4a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fafcc006fd0 tx=0x7fafcc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:25.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.790+0000 7fafe0ff9700 1 -- 172.21.15.94:0/492380264 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fafd402ae20 con 0x7fafe4106600 2024-02-17T11:49:25.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:25 smithi094 ceph-mon[32499]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:25.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:25.988+0000 7fafe97a6700 1 -- 172.21.15.94:0/492380264 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}) v1 -- 0x7fafe41967c0 con 0x7fafd0038fe0 2024-02-17T11:49:27.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:49:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi125", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:49:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:49:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: Updating smithi125:/etc/ceph/ceph.conf 2024-02-17T11:49:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:49:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: Updating smithi125:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:49:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: Deploying cephadm binary to smithi189 2024-02-17T11:49:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:49:27.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:27.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:27.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:29.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:29 smithi094 ceph-mon[32499]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:30.122+0000 7fafe0ff9700 1 -- 172.21.15.94:0/492380264 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fafe41967c0 con 0x7fafd0038fe0 2024-02-17T11:49:30.123 INFO:teuthology.orchestra.run.smithi094.stdout:Added host 'smithi189' with addr '172.21.15.189' 2024-02-17T11:49:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:30.125+0000 7fafe97a6700 1 -- 172.21.15.94:0/492380264 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fafd0038fe0 msgr2=0x7fafd003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:30.125+0000 7fafe97a6700 1 --2- 172.21.15.94:0/492380264 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fafd0038fe0 0x7fafd003b4a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fafcc006fd0 tx=0x7fafcc006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:30.125+0000 7fafe97a6700 1 -- 172.21.15.94:0/492380264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4106600 msgr2=0x7fafe4195b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:30.125+0000 7fafe97a6700 1 --2- 172.21.15.94:0/492380264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4106600 0x7fafe4195b30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fafd4004f40 tx=0x7fafd4005020 comp rx=0 tx=0).stop 2024-02-17T11:49:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:30.125+0000 7fafe97a6700 1 -- 172.21.15.94:0/492380264 shutdown_connections 2024-02-17T11:49:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:30.125+0000 7fafe97a6700 1 --2- 172.21.15.94:0/492380264 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fafd0038fe0 0x7fafd003b4a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:30.125+0000 7fafe97a6700 1 --2- 172.21.15.94:0/492380264 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafe4106600 0x7fafe4195b30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:30.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:30.125+0000 7fafe97a6700 1 -- 172.21.15.94:0/492380264 >> 172.21.15.94:0/492380264 conn(0x7fafe40fc020 msgr2=0x7fafe40fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:30.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:30.125+0000 7fafe97a6700 1 -- 172.21.15.94:0/492380264 shutdown_connections 2024-02-17T11:49:30.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:30.125+0000 7fafe97a6700 1 -- 172.21.15.94:0/492380264 wait complete. 2024-02-17T11:49:30.883 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-17T11:49:31.124 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:49:31.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:31 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:31.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:31 smithi094 ceph-mon[32499]: Added host smithi189 2024-02-17T11:49:31.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:31 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:49:31.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:31 smithi094 ceph-mon[32499]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:31.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:31 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:32.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:32 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:32.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.694+0000 7fa0cbf77700 1 -- 172.21.15.94:0/3549178973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0c4106640 msgr2=0x7fa0c4106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:32.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.694+0000 7fa0cbf77700 1 --2- 172.21.15.94:0/3549178973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0c4106640 0x7fa0c4106a20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0009910 tx=0x7fa0c0009c20 comp rx=0 tx=0).stop 2024-02-17T11:49:32.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.694+0000 7fa0cbf77700 1 -- 172.21.15.94:0/3549178973 shutdown_connections 2024-02-17T11:49:32.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.694+0000 7fa0cbf77700 1 --2- 172.21.15.94:0/3549178973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0c4106640 0x7fa0c4106a20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:32.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.694+0000 7fa0cbf77700 1 -- 172.21.15.94:0/3549178973 >> 172.21.15.94:0/3549178973 conn(0x7fa0c40fc0c0 msgr2=0x7fa0c40fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:32.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.694+0000 7fa0cbf77700 1 -- 172.21.15.94:0/3549178973 shutdown_connections 2024-02-17T11:49:32.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.695+0000 7fa0cbf77700 1 -- 172.21.15.94:0/3549178973 wait complete. 2024-02-17T11:49:32.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.695+0000 7fa0cbf77700 1 Processor -- start 2024-02-17T11:49:32.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.695+0000 7fa0cbf77700 1 -- start start 2024-02-17T11:49:32.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.695+0000 7fa0cbf77700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0c4106640 0x7fa0c40fc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:32.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.695+0000 7fa0cbf77700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40fe460 con 0x7fa0c4106640 2024-02-17T11:49:32.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.695+0000 7fa0c9d13700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0c4106640 0x7fa0c40fc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:32.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.695+0000 7fa0c9d13700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0c4106640 0x7fa0c40fc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40054/0 (socket says 172.21.15.94:40054) 2024-02-17T11:49:32.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.695+0000 7fa0c9d13700 1 -- 172.21.15.94:0/1954836104 learned_addr learned my addr 172.21.15.94:0/1954836104 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:32.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.696+0000 7fa0c9d13700 1 -- 172.21.15.94:0/1954836104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c00095f0 con 0x7fa0c4106640 2024-02-17T11:49:32.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.696+0000 7fa0c9d13700 1 --2- 172.21.15.94:0/1954836104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0c4106640 0x7fa0c40fc570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0004f40 tx=0x7fa0c0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:32.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.696+0000 7fa0baffd700 1 -- 172.21.15.94:0/1954836104 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0c001d070 con 0x7fa0c4106640 2024-02-17T11:49:32.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.696+0000 7fa0baffd700 1 -- 172.21.15.94:0/1954836104 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0c000bc30 con 0x7fa0c4106640 2024-02-17T11:49:32.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.696+0000 7fa0cbf77700 1 -- 172.21.15.94:0/1954836104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c40fcab0 con 0x7fa0c4106640 2024-02-17T11:49:32.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.696+0000 7fa0cbf77700 1 -- 172.21.15.94:0/1954836104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c40fcf30 con 0x7fa0c4106640 2024-02-17T11:49:32.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.696+0000 7fa0baffd700 1 -- 172.21.15.94:0/1954836104 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0c0003970 con 0x7fa0c4106640 2024-02-17T11:49:32.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.697+0000 7fa0baffd700 1 -- 172.21.15.94:0/1954836104 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fa0c0003be0 con 0x7fa0c4106640 2024-02-17T11:49:32.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.697+0000 7fa0cbf77700 1 -- 172.21.15.94:0/1954836104 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c4101cb0 con 0x7fa0c4106640 2024-02-17T11:49:32.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.697+0000 7fa0baffd700 1 --2- 172.21.15.94:0/1954836104 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa0b0038fe0 0x7fa0b003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:32.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.697+0000 7fa0baffd700 1 -- 172.21.15.94:0/1954836104 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa0c004cd80 con 0x7fa0c4106640 2024-02-17T11:49:32.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.697+0000 7fa0c9512700 1 --2- 172.21.15.94:0/1954836104 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa0b0038fe0 0x7fa0b003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:32.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.698+0000 7fa0c9512700 1 --2- 172.21.15.94:0/1954836104 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa0b0038fe0 0x7fa0b003b4a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006fd0 tx=0x7fa0b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:32.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.702+0000 7fa0baffd700 1 -- 172.21.15.94:0/1954836104 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa0c0005c00 con 0x7fa0c4106640 2024-02-17T11:49:32.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.901+0000 7fa0cbf77700 1 -- 172.21.15.94:0/1954836104 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0c40fd280 con 0x7fa0b0038fe0 2024-02-17T11:49:32.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.902+0000 7fa0baffd700 1 -- 172.21.15.94:0/1954836104 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7fa0c40fd280 con 0x7fa0b0038fe0 2024-02-17T11:49:32.901 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:49:32.901 INFO:teuthology.orchestra.run.smithi094.stdout:[{"addr": "172.21.15.94", "hostname": "smithi094", "labels": [], "status": ""}, {"addr": "172.21.15.125", "hostname": "smithi125", "labels": [], "status": ""}, {"addr": "172.21.15.189", "hostname": "smithi189", "labels": [], "status": ""}] 2024-02-17T11:49:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.904+0000 7fa0cbf77700 1 -- 172.21.15.94:0/1954836104 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa0b0038fe0 msgr2=0x7fa0b003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.904+0000 7fa0cbf77700 1 --2- 172.21.15.94:0/1954836104 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa0b0038fe0 0x7fa0b003b4a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006fd0 tx=0x7fa0b4006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.904+0000 7fa0cbf77700 1 -- 172.21.15.94:0/1954836104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0c4106640 msgr2=0x7fa0c40fc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:32.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.904+0000 7fa0cbf77700 1 --2- 172.21.15.94:0/1954836104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0c4106640 0x7fa0c40fc570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0004f40 tx=0x7fa0c0005020 comp rx=0 tx=0).stop 2024-02-17T11:49:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.904+0000 7fa0cbf77700 1 -- 172.21.15.94:0/1954836104 shutdown_connections 2024-02-17T11:49:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.904+0000 7fa0cbf77700 1 --2- 172.21.15.94:0/1954836104 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa0b0038fe0 0x7fa0b003b4a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.904+0000 7fa0cbf77700 1 --2- 172.21.15.94:0/1954836104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0c4106640 0x7fa0c40fc570 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.904+0000 7fa0cbf77700 1 -- 172.21.15.94:0/1954836104 >> 172.21.15.94:0/1954836104 conn(0x7fa0c40fc0c0 msgr2=0x7fa0c4069ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.904+0000 7fa0cbf77700 1 -- 172.21.15.94:0/1954836104 shutdown_connections 2024-02-17T11:49:32.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:32.904+0000 7fa0cbf77700 1 -- 172.21.15.94:0/1954836104 wait complete. 2024-02-17T11:49:33.455 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-17T11:49:33.455 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd crush tunables default 2024-02-17T11:49:33.695 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:49:33.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:33 smithi094 ceph-mon[32499]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:33.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:33 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:34.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:34 smithi094 ceph-mon[32499]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:49:35.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.137+0000 7fafc19a1700 1 -- 172.21.15.94:0/138112226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc107560 msgr2=0x7fafbc107940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:35.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.137+0000 7fafc19a1700 1 --2- 172.21.15.94:0/138112226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc107560 0x7fafbc107940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009910 tx=0x7fafb0009c20 comp rx=0 tx=0).stop 2024-02-17T11:49:35.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.138+0000 7fafc19a1700 1 -- 172.21.15.94:0/138112226 shutdown_connections 2024-02-17T11:49:35.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.138+0000 7fafc19a1700 1 --2- 172.21.15.94:0/138112226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc107560 0x7fafbc107940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.138+0000 7fafc19a1700 1 -- 172.21.15.94:0/138112226 >> 172.21.15.94:0/138112226 conn(0x7fafbc0fcfa0 msgr2=0x7fafbc0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.138+0000 7fafc19a1700 1 -- 172.21.15.94:0/138112226 shutdown_connections 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.138+0000 7fafc19a1700 1 -- 172.21.15.94:0/138112226 wait complete. 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.138+0000 7fafc19a1700 1 Processor -- start 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.138+0000 7fafc19a1700 1 -- start start 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.138+0000 7fafc19a1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc107560 0x7fafbc073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.138+0000 7fafc19a1700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc075e60 con 0x7fafbc107560 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.138+0000 7fafc099f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc107560 0x7fafbc073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.138+0000 7fafc099f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc107560 0x7fafbc073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36190/0 (socket says 172.21.15.94:36190) 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.139+0000 7fafc099f700 1 -- 172.21.15.94:0/1374655495 learned_addr learned my addr 172.21.15.94:0/1374655495 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.139+0000 7fafc099f700 1 -- 172.21.15.94:0/1374655495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb00095f0 con 0x7fafbc107560 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.139+0000 7fafc099f700 1 --2- 172.21.15.94:0/1374655495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc107560 0x7fafbc073f70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fafb0004f40 tx=0x7fafb0005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.139+0000 7fafb9ffb700 1 -- 172.21.15.94:0/1374655495 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafb001d070 con 0x7fafbc107560 2024-02-17T11:49:35.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.139+0000 7fafb9ffb700 1 -- 172.21.15.94:0/1374655495 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fafb000bc30 con 0x7fafbc107560 2024-02-17T11:49:35.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.139+0000 7fafc19a1700 1 -- 172.21.15.94:0/1374655495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafbc0744b0 con 0x7fafbc107560 2024-02-17T11:49:35.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.139+0000 7fafb9ffb700 1 -- 172.21.15.94:0/1374655495 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fafb0003970 con 0x7fafbc107560 2024-02-17T11:49:35.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.139+0000 7fafc19a1700 1 -- 172.21.15.94:0/1374655495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafbc074930 con 0x7fafbc107560 2024-02-17T11:49:35.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.140+0000 7fafb9ffb700 1 -- 172.21.15.94:0/1374655495 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fafb0003c30 con 0x7fafbc107560 2024-02-17T11:49:35.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.140+0000 7fafb9ffb700 1 --2- 172.21.15.94:0/1374655495 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fafac038f80 0x7fafac03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:35.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.140+0000 7fafb9ffb700 1 -- 172.21.15.94:0/1374655495 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fafb004cf30 con 0x7fafbc107560 2024-02-17T11:49:35.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.140+0000 7fafc19a1700 1 -- 172.21.15.94:0/1374655495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafbc102bd0 con 0x7fafbc107560 2024-02-17T11:49:35.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.140+0000 7fafbbfff700 1 --2- 172.21.15.94:0/1374655495 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fafac038f80 0x7fafac03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:35.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.141+0000 7fafbbfff700 1 --2- 172.21.15.94:0/1374655495 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fafac038f80 0x7fafac03b440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fafa8006fd0 tx=0x7fafa8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:35.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.145+0000 7fafb9ffb700 1 -- 172.21.15.94:0/1374655495 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fafb0026070 con 0x7fafbc107560 2024-02-17T11:49:35.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.342+0000 7fafc19a1700 1 -- 172.21.15.94:0/1374655495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fafbc0632a0 con 0x7fafbc107560 2024-02-17T11:49:35.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.680+0000 7fafb9ffb700 1 -- 172.21.15.94:0/1374655495 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fafb000fd00 con 0x7fafbc107560 2024-02-17T11:49:35.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.683+0000 7fafc19a1700 1 -- 172.21.15.94:0/1374655495 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fafac038f80 msgr2=0x7fafac03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:35.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.683+0000 7fafc19a1700 1 --2- 172.21.15.94:0/1374655495 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fafac038f80 0x7fafac03b440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fafa8006fd0 tx=0x7fafa8006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:35.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.683+0000 7fafc19a1700 1 -- 172.21.15.94:0/1374655495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc107560 msgr2=0x7fafbc073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:35.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.683+0000 7fafc19a1700 1 --2- 172.21.15.94:0/1374655495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc107560 0x7fafbc073f70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fafb0004f40 tx=0x7fafb0005020 comp rx=0 tx=0).stop 2024-02-17T11:49:35.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.683+0000 7fafc19a1700 1 -- 172.21.15.94:0/1374655495 shutdown_connections 2024-02-17T11:49:35.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.683+0000 7fafc19a1700 1 --2- 172.21.15.94:0/1374655495 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fafac038f80 0x7fafac03b440 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:35.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.683+0000 7fafc19a1700 1 --2- 172.21.15.94:0/1374655495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fafbc107560 0x7fafbc073f70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:35.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.683+0000 7fafc19a1700 1 -- 172.21.15.94:0/1374655495 >> 172.21.15.94:0/1374655495 conn(0x7fafbc0fcfa0 msgr2=0x7fafbc0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:35.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.683+0000 7fafc19a1700 1 -- 172.21.15.94:0/1374655495 shutdown_connections 2024-02-17T11:49:35.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:49:35.683+0000 7fafc19a1700 1 -- 172.21.15.94:0/1374655495 wait complete. 2024-02-17T11:49:35.683 INFO:teuthology.orchestra.run.smithi094.stderr:adjusted tunables profile to default 2024-02-17T11:49:35.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:35 smithi094 ceph-mon[32499]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:35.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:35 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1374655495' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-17T11:49:36.399 INFO:tasks.cephadm:Adding mon.a on smithi094 2024-02-17T11:49:36.399 INFO:tasks.cephadm:Adding mon.b on smithi125 2024-02-17T11:49:36.399 INFO:tasks.cephadm:Adding mon.c on smithi189 2024-02-17T11:49:36.399 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch apply mon '3;smithi094:172.21.15.94=a;smithi125:172.21.15.125=b;smithi189:172.21.15.189=c' 2024-02-17T11:49:36.635 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:49:36.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:36 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1374655495' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-17T11:49:36.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:36 smithi094 ceph-mon[32499]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T11:49:36.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:36.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:36.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:36.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:36.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:49:36.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:36.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:49:36.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:36.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:36.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:37.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:37 smithi094 ceph-mon[32499]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-17T11:49:37.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:37 smithi094 ceph-mon[32499]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:49:37.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:37 smithi094 ceph-mon[32499]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:49:37.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:37 smithi094 ceph-mon[32499]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:49:37.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:37 smithi094 ceph-mon[32499]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:38.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.105+0000 7fb170fed700 1 -- 172.21.15.189:0/1597942249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16c102810 msgr2=0x7fb16c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:38.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.105+0000 7fb170fed700 1 --2- 172.21.15.189:0/1597942249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16c102810 0x7fb16c102bf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009910 tx=0x7fb15c009c20 comp rx=0 tx=0).stop 2024-02-17T11:49:38.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.105+0000 7fb170fed700 1 -- 172.21.15.189:0/1597942249 shutdown_connections 2024-02-17T11:49:38.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.105+0000 7fb170fed700 1 --2- 172.21.15.189:0/1597942249 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16c102810 0x7fb16c102bf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:38.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.105+0000 7fb170fed700 1 -- 172.21.15.189:0/1597942249 >> 172.21.15.189:0/1597942249 conn(0x7fb16c0fc040 msgr2=0x7fb16c0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:38.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.106+0000 7fb170fed700 1 -- 172.21.15.189:0/1597942249 shutdown_connections 2024-02-17T11:49:38.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.106+0000 7fb170fed700 1 -- 172.21.15.189:0/1597942249 wait complete. 2024-02-17T11:49:38.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.106+0000 7fb170fed700 1 Processor -- start 2024-02-17T11:49:38.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.106+0000 7fb170fed700 1 -- start start 2024-02-17T11:49:38.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.106+0000 7fb170fed700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16c102810 0x7fb16c196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:38.106 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.106+0000 7fb170fed700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c1974e0 con 0x7fb16c102810 2024-02-17T11:49:38.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.107+0000 7fb16a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16c102810 0x7fb16c196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:38.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.107+0000 7fb16a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16c102810 0x7fb16c196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.189:57084/0 (socket says 172.21.15.189:57084) 2024-02-17T11:49:38.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.107+0000 7fb16a59c700 1 -- 172.21.15.189:0/4037446292 learned_addr learned my addr 172.21.15.189:0/4037446292 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-02-17T11:49:38.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.107+0000 7fb16a59c700 1 -- 172.21.15.189:0/4037446292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c0095f0 con 0x7fb16c102810 2024-02-17T11:49:38.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.107+0000 7fb16a59c700 1 --2- 172.21.15.189:0/4037446292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16c102810 0x7fb16c196fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb15c006010 tx=0x7fb15c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:38.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.107+0000 7fb15b7fe700 1 -- 172.21.15.189:0/4037446292 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb15c01d070 con 0x7fb16c102810 2024-02-17T11:49:38.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.107+0000 7fb170fed700 1 -- 172.21.15.189:0/4037446292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c1976e0 con 0x7fb16c102810 2024-02-17T11:49:38.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.107+0000 7fb170fed700 1 -- 172.21.15.189:0/4037446292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c197b00 con 0x7fb16c102810 2024-02-17T11:49:38.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.108+0000 7fb15b7fe700 1 -- 172.21.15.189:0/4037446292 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb15c00bc30 con 0x7fb16c102810 2024-02-17T11:49:38.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.108+0000 7fb15b7fe700 1 -- 172.21.15.189:0/4037446292 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb15c003950 con 0x7fb16c102810 2024-02-17T11:49:38.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.108+0000 7fb170fed700 1 -- 172.21.15.189:0/4037446292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16c0632a0 con 0x7fb16c102810 2024-02-17T11:49:38.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.108+0000 7fb15b7fe700 1 -- 172.21.15.189:0/4037446292 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fb15c00b750 con 0x7fb16c102810 2024-02-17T11:49:38.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.109+0000 7fb15b7fe700 1 --2- 172.21.15.189:0/4037446292 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb154038f80 0x7fb15403b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:38.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.109+0000 7fb15b7fe700 1 -- 172.21.15.189:0/4037446292 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb15c04cdb0 con 0x7fb16c102810 2024-02-17T11:49:38.109 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.109+0000 7fb169d9b700 1 --2- 172.21.15.189:0/4037446292 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb154038f80 0x7fb15403b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:38.110 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.109+0000 7fb169d9b700 1 --2- 172.21.15.189:0/4037446292 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb154038f80 0x7fb15403b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb160006fd0 tx=0x7fb160006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:38.113 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.114+0000 7fb15b7fe700 1 -- 172.21.15.189:0/4037446292 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb15c004390 con 0x7fb16c102810 2024-02-17T11:49:38.317 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.318+0000 7fb170fed700 1 -- 172.21.15.189:0/4037446292 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi125:172.21.15.125=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fb16c10bb90 con 0x7fb154038f80 2024-02-17T11:49:38.323 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.324+0000 7fb15b7fe700 1 -- 172.21.15.189:0/4037446292 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb16c10bb90 con 0x7fb154038f80 2024-02-17T11:49:38.324 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mon update... 2024-02-17T11:49:38.326 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.326+0000 7fb170fed700 1 -- 172.21.15.189:0/4037446292 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb154038f80 msgr2=0x7fb15403b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:38.326 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.326+0000 7fb170fed700 1 --2- 172.21.15.189:0/4037446292 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb154038f80 0x7fb15403b440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb160006fd0 tx=0x7fb160006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:38.326 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.326+0000 7fb170fed700 1 -- 172.21.15.189:0/4037446292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16c102810 msgr2=0x7fb16c196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:38.326 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.326+0000 7fb170fed700 1 --2- 172.21.15.189:0/4037446292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16c102810 0x7fb16c196fa0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb15c006010 tx=0x7fb15c004ff0 comp rx=0 tx=0).stop 2024-02-17T11:49:38.326 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.326+0000 7fb170fed700 1 -- 172.21.15.189:0/4037446292 shutdown_connections 2024-02-17T11:49:38.326 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.326+0000 7fb170fed700 1 --2- 172.21.15.189:0/4037446292 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb154038f80 0x7fb15403b440 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:38.326 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.326+0000 7fb170fed700 1 --2- 172.21.15.189:0/4037446292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb16c102810 0x7fb16c196fa0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:38.326 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.326+0000 7fb170fed700 1 -- 172.21.15.189:0/4037446292 >> 172.21.15.189:0/4037446292 conn(0x7fb16c0fc040 msgr2=0x7fb16c1035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:38.326 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.326+0000 7fb170fed700 1 -- 172.21.15.189:0/4037446292 shutdown_connections 2024-02-17T11:49:38.326 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:38.326+0000 7fb170fed700 1 -- 172.21.15.189:0/4037446292 wait complete. 2024-02-17T11:49:38.924 DEBUG:teuthology.orchestra.run.smithi125:mon.b> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.b.service 2024-02-17T11:49:38.927 DEBUG:teuthology.orchestra.run.smithi189:mon.c> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.c.service 2024-02-17T11:49:38.930 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-17T11:49:38.930 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-17T11:49:39.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:39 smithi094 ceph-mon[32499]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi125:172.21.15.125=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:49:39.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:39 smithi094 ceph-mon[32499]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi125:172.21.15.125=b;smithi189:172.21.15.189=c;count:3 2024-02-17T11:49:39.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:49:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:49:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:49:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:39 smithi094 ceph-mon[32499]: Deploying daemon mon.c on smithi189 2024-02-17T11:49:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:39 smithi094 ceph-mon[32499]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:40.016 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:49:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:41 smithi094 ceph-mon[32499]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:42.619 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.618+0000 7f8a760a5700 1 -- 172.21.15.189:0/546429883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a70106600 msgr2=0x7f8a701069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:42.619 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.618+0000 7f8a760a5700 1 --2- 172.21.15.189:0/546429883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a70106600 0x7f8a701069e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8a60009910 tx=0x7f8a60009c20 comp rx=0 tx=0).stop 2024-02-17T11:49:42.619 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.618+0000 7f8a760a5700 1 -- 172.21.15.189:0/546429883 shutdown_connections 2024-02-17T11:49:42.619 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.618+0000 7f8a760a5700 1 --2- 172.21.15.189:0/546429883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a70106600 0x7f8a701069e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:42.619 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.618+0000 7f8a760a5700 1 -- 172.21.15.189:0/546429883 >> 172.21.15.189:0/546429883 conn(0x7f8a700fc020 msgr2=0x7f8a700fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:42.619 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.619+0000 7f8a760a5700 1 -- 172.21.15.189:0/546429883 shutdown_connections 2024-02-17T11:49:42.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.619+0000 7f8a760a5700 1 -- 172.21.15.189:0/546429883 wait complete. 2024-02-17T11:49:42.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.619+0000 7f8a760a5700 1 Processor -- start 2024-02-17T11:49:42.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.620+0000 7f8a760a5700 1 -- start start 2024-02-17T11:49:42.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.620+0000 7f8a760a5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a70106600 0x7f8a70197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:42.620 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.620+0000 7f8a760a5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a701981a0 con 0x7f8a70106600 2024-02-17T11:49:42.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.620+0000 7f8a6f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a70106600 0x7f8a70197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:42.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.620+0000 7f8a6f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a70106600 0x7f8a70197c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.189:57096/0 (socket says 172.21.15.189:57096) 2024-02-17T11:49:42.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.620+0000 7f8a6f7fe700 1 -- 172.21.15.189:0/881455528 learned_addr learned my addr 172.21.15.189:0/881455528 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-02-17T11:49:42.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.621+0000 7f8a6f7fe700 1 -- 172.21.15.189:0/881455528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a600095f0 con 0x7f8a70106600 2024-02-17T11:49:42.621 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.621+0000 7f8a6f7fe700 1 --2- 172.21.15.189:0/881455528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a70106600 0x7f8a70197c60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8a60006010 tx=0x7f8a60004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:42.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.621+0000 7f8a6cff9700 1 -- 172.21.15.189:0/881455528 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a6001d070 con 0x7f8a70106600 2024-02-17T11:49:42.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.621+0000 7f8a6cff9700 1 -- 172.21.15.189:0/881455528 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8a6000bc30 con 0x7f8a70106600 2024-02-17T11:49:42.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.621+0000 7f8a760a5700 1 -- 172.21.15.189:0/881455528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a70198400 con 0x7f8a70106600 2024-02-17T11:49:42.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.621+0000 7f8a6cff9700 1 -- 172.21.15.189:0/881455528 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a60003930 con 0x7f8a70106600 2024-02-17T11:49:42.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.621+0000 7f8a760a5700 1 -- 172.21.15.189:0/881455528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a70198820 con 0x7f8a70106600 2024-02-17T11:49:42.622 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.622+0000 7f8a760a5700 1 -- 172.21.15.189:0/881455528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a70101c70 con 0x7f8a70106600 2024-02-17T11:49:42.623 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.622+0000 7f8a6cff9700 1 -- 172.21.15.189:0/881455528 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f8a60003b60 con 0x7f8a70106600 2024-02-17T11:49:42.623 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.623+0000 7f8a6cff9700 1 --2- 172.21.15.189:0/881455528 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8a58038fe0 0x7f8a5803b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:42.623 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.623+0000 7f8a6cff9700 1 -- 172.21.15.189:0/881455528 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8a6004cc90 con 0x7f8a70106600 2024-02-17T11:49:42.623 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.623+0000 7f8a6effd700 1 --2- 172.21.15.189:0/881455528 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8a58038fe0 0x7f8a5803b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:42.623 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.623+0000 7f8a6effd700 1 --2- 172.21.15.189:0/881455528 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8a58038fe0 0x7f8a5803b4a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8a64006fd0 tx=0x7f8a64006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:42.627 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.628+0000 7f8a6cff9700 1 -- 172.21.15.189:0/881455528 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8a60029740 con 0x7f8a70106600 2024-02-17T11:49:42.769 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:42 smithi189 podman[36556]: 2024-02-17 11:49:42.672360704 +0000 UTC m=+0.017910104 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T11:49:42.910 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.910+0000 7f8a760a5700 1 -- 172.21.15.189:0/881455528 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8a70069f40 con 0x7f8a70106600 2024-02-17T11:49:42.910 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.911+0000 7f8a6cff9700 1 -- 172.21.15.189:0/881455528 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7f8a60026070 con 0x7f8a70106600 2024-02-17T11:49:42.911 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:49:42.911 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":1,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","modified":"2024-02-17T11:48:01.408577Z","created":"2024-02-17T11:48:01.408577Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-17T11:49:42.913 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.913+0000 7f8a760a5700 1 -- 172.21.15.189:0/881455528 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8a58038fe0 msgr2=0x7f8a5803b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:42.913 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.913+0000 7f8a760a5700 1 --2- 172.21.15.189:0/881455528 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8a58038fe0 0x7f8a5803b4a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8a64006fd0 tx=0x7f8a64006e40 comp rx=0 tx=0).stop 2024-02-17T11:49:42.913 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.913+0000 7f8a760a5700 1 -- 172.21.15.189:0/881455528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a70106600 msgr2=0x7f8a70197c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:42.913 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.913+0000 7f8a760a5700 1 --2- 172.21.15.189:0/881455528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a70106600 0x7f8a70197c60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8a60006010 tx=0x7f8a60004ff0 comp rx=0 tx=0).stop 2024-02-17T11:49:42.913 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.913+0000 7f8a760a5700 1 -- 172.21.15.189:0/881455528 shutdown_connections 2024-02-17T11:49:42.913 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.913+0000 7f8a760a5700 1 --2- 172.21.15.189:0/881455528 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8a58038fe0 0x7f8a5803b4a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:42.913 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.913+0000 7f8a760a5700 1 --2- 172.21.15.189:0/881455528 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8a70106600 0x7f8a70197c60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:42.913 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.913+0000 7f8a760a5700 1 -- 172.21.15.189:0/881455528 >> 172.21.15.189:0/881455528 conn(0x7f8a700fc020 msgr2=0x7f8a700fd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:42.913 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.913+0000 7f8a760a5700 1 -- 172.21.15.189:0/881455528 shutdown_connections 2024-02-17T11:49:42.913 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:42.913+0000 7f8a760a5700 1 -- 172.21.15.189:0/881455528 wait complete. 2024-02-17T11:49:42.914 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 1 2024-02-17T11:49:43.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 podman[36556]: 2024-02-17 11:49:43.049558332 +0000 UTC m=+0.395107722 container init a90006e43b15a690a4cb86daee7382196cda083bfd32953829ee745da343d565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-c, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-17T11:49:43.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 podman[36556]: 2024-02-17 11:49:43.052924962 +0000 UTC m=+0.398474340 container start a90006e43b15a690a4cb86daee7382196cda083bfd32953829ee745da343d565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-17T11:49:43.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: set uid:gid to 167:167 (ceph:ceph) 2024-02-17T11:49:43.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev), process ceph-mon, pid 2 2024-02-17T11:49:43.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: pidfile_write: ignore empty --pid-file 2024-02-17T11:49:43.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: load: jerasure load: lrc 2024-02-17T11:49:43.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: RocksDB version: 7.9.2 2024-02-17T11:49:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Git sha 0 2024-02-17T11:49:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Compile date 2024-02-16 17:52:22 2024-02-17T11:49:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: DB SUMMARY 2024-02-17T11:49:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: DB Session ID: 63LW2QW7BHQB5ANMHQAV 2024-02-17T11:49:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: CURRENT file: CURRENT 2024-02-17T11:49:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: IDENTITY file: IDENTITY 2024-02-17T11:49:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-17T11:49:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-17T11:49:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-02-17T11:49:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.error_if_exists: 0 2024-02-17T11:49:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.create_if_missing: 0 2024-02-17T11:49:43.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.paranoid_checks: 1 2024-02-17T11:49:43.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-17T11:49:43.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-17T11:49:43.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-17T11:49:43.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.env: 0x5560c82b1ae0 2024-02-17T11:49:43.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.fs: PosixFileSystem 2024-02-17T11:49:43.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.info_log: 0x5560ca9618c0 2024-02-17T11:49:43.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_file_opening_threads: 16 2024-02-17T11:49:43.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.statistics: (nil) 2024-02-17T11:49:43.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.use_fsync: 0 2024-02-17T11:49:43.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_log_file_size: 0 2024-02-17T11:49:43.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.keep_log_file_num: 1000 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.recycle_log_file_num: 0 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.allow_fallocate: 1 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.allow_mmap_reads: 0 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.allow_mmap_writes: 0 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.use_direct_reads: 0 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.create_missing_column_families: 0 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.db_log_dir: 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.wal_dir: 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-17T11:49:43.405 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.advise_random_on_open: 1 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.db_write_buffer_size: 0 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.write_buffer_manager: 0x5560ca935400 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.rate_limiter: (nil) 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.wal_recovery_mode: 2 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.enable_thread_tracking: 0 2024-02-17T11:49:43.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.enable_pipelined_write: 0 2024-02-17T11:49:43.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.unordered_write: 0 2024-02-17T11:49:43.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-17T11:49:43.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-17T11:49:43.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-17T11:49:43.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-17T11:49:43.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.row_cache: None 2024-02-17T11:49:43.407 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.wal_filter: None 2024-02-17T11:49:43.408 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.allow_ingest_behind: 0 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.two_write_queues: 0 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.manual_wal_flush: 0 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.wal_compression: 0 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.atomic_flush: 0 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.log_readahead_size: 0 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.best_efforts_recovery: 0 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-17T11:49:43.409 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.allow_data_in_errors: 0 2024-02-17T11:49:43.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.db_host_id: __hostname__ 2024-02-17T11:49:43.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-17T11:49:43.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_background_jobs: 2 2024-02-17T11:49:43.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_background_compactions: -1 2024-02-17T11:49:43.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_subcompactions: 1 2024-02-17T11:49:43.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-17T11:49:43.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-17T11:49:43.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-17T11:49:43.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_total_wal_size: 0 2024-02-17T11:49:43.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-17T11:49:43.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-17T11:49:43.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-17T11:49:43.410 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_open_files: -1 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.bytes_per_sync: 0 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compaction_readahead_size: 0 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_background_flushes: -1 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Compression algorithms supported: 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: kZSTD supported: 0 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: kXpressCompression supported: 0 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: kLZ4HCCompression supported: 1 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: kZlibCompression supported: 1 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: kSnappyCompression supported: 1 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: kLZ4Compression supported: 1 2024-02-17T11:49:43.411 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: kBZip2Compression supported: 0 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.merge_operator: 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compaction_filter: None 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compaction_filter_factory: None 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.sst_partitioner_factory: None 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5560ca9619c0) 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout: cache_index_and_filter_blocks: 1 2024-02-17T11:49:43.412 INFO:journalctl@ceph.mon.c.smithi189.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: pin_top_level_index_and_filter: 1 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_type: 0 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: data_block_index_type: 0 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_shortening: 1 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: checksum: 4 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: no_block_cache: 0 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache: 0x5560ca965090 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_name: BinnedLRUCache 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_options: 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: capacity : 536870912 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: num_shard_bits : 4 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: strict_capacity_limit : 0 2024-02-17T11:49:43.413 INFO:journalctl@ceph.mon.c.smithi189.stdout: high_pri_pool_ratio: 0.000 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_cache_compressed: (nil) 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: persistent_cache: (nil) 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_size: 4096 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_size_deviation: 10 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_restart_interval: 16 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: index_block_restart_interval: 1 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: metadata_block_size: 4096 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: partition_filters: 0 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: use_delta_encoding: 1 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: filter_policy: bloomfilter 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: whole_key_filtering: 1 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: verify_compression: 0 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: read_amp_bytes_per_bit: 0 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: format_version: 5 2024-02-17T11:49:43.414 INFO:journalctl@ceph.mon.c.smithi189.stdout: enable_index_compression: 1 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout: block_align: 0 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout: max_auto_readahead_size: 262144 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout: prepopulate_block_cache: 0 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout: initial_auto_readahead_size: 8192 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout: num_file_reads_for_auto_readahead: 2 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.write_buffer_size: 33554432 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_write_buffer_number: 2 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compression: NoCompression 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.bottommost_compression: Disabled 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.prefix_extractor: nullptr 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.num_levels: 7 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-17T11:49:43.415 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-17T11:49:43.416 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-17T11:49:43.416 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-17T11:49:43.416 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-17T11:49:43.416 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-17T11:49:43.416 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-17T11:49:43.416 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-17T11:49:43.416 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-17T11:49:43.416 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-17T11:49:43.416 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-17T11:49:43.416 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-17T11:49:43.416 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-17T11:49:43.416 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compression_opts.level: 32767 2024-02-17T11:49:43.416 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compression_opts.strategy: 0 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compression_opts.enabled: false 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.target_file_size_base: 67108864 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-17T11:49:43.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.arena_block_size: 1048576 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.disable_auto_compactions: 0 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-17T11:49:43.418 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.table_properties_collectors: 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.inplace_update_support: 0 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-17T11:49:43.419 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-17T11:49:43.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.bloom_locality: 0 2024-02-17T11:49:43.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.max_successive_merges: 0 2024-02-17T11:49:43.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-17T11:49:43.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.paranoid_file_checks: 0 2024-02-17T11:49:43.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.force_consistency_checks: 1 2024-02-17T11:49:43.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.report_bg_io_stats: 0 2024-02-17T11:49:43.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.ttl: 2592000 2024-02-17T11:49:43.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-17T11:49:43.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-17T11:49:43.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-17T11:49:43.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.enable_blob_files: false 2024-02-17T11:49:43.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.min_blob_size: 0 2024-02-17T11:49:43.420 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.blob_file_size: 268435456 2024-02-17T11:49:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-17T11:49:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-17T11:49:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-17T11:49:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-17T11:49:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-17T11:49:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.blob_file_starting_level: 0 2024-02-17T11:49:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-17T11:49:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-17T11:49:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-17T11:49:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a4321466-b6a4-4f93-9d0c-978c2c67f9f0 2024-02-17T11:49:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708170583082279, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-17T11:49:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-17T11:49:43.421 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708170583082707, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708170583, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a4321466-b6a4-4f93-9d0c-978c2c67f9f0", "db_session_id": "63LW2QW7BHQB5ANMHQAV", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708170583082790, "job": 1, "event": "recovery_finished"} 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5560caa56000 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: DB pointer 0x5560caa40000 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** DB Stats ** 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-17T11:49:43.422 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-17T11:49:43.423 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-17T11:49:43.423 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-02-17T11:49:43.423 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** Compaction Stats [default] ** 2024-02-17T11:49:43.423 INFO:journalctl@ceph.mon.c.smithi189.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-17T11:49:43.423 INFO:journalctl@ceph.mon.c.smithi189.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-17T11:49:43.423 INFO:journalctl@ceph.mon.c.smithi189.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-17T11:49:43.423 INFO:journalctl@ceph.mon.c.smithi189.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-17T11:49:43.423 INFO:journalctl@ceph.mon.c.smithi189.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-17T11:49:43.423 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-02-17T11:49:43.423 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** Compaction Stats [default] ** 2024-02-17T11:49:43.423 INFO:journalctl@ceph.mon.c.smithi189.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-17T11:49:43.423 INFO:journalctl@ceph.mon.c.smithi189.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-17T11:49:43.424 INFO:journalctl@ceph.mon.c.smithi189.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-17T11:49:43.424 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-02-17T11:49:43.424 INFO:journalctl@ceph.mon.c.smithi189.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-17T11:49:43.424 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-02-17T11:49:43.424 INFO:journalctl@ceph.mon.c.smithi189.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-17T11:49:43.424 INFO:journalctl@ceph.mon.c.smithi189.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-17T11:49:43.424 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-17T11:49:43.424 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-17T11:49:43.424 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-17T11:49:43.424 INFO:journalctl@ceph.mon.c.smithi189.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-17T11:49:43.424 INFO:journalctl@ceph.mon.c.smithi189.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-17T11:49:43.425 INFO:journalctl@ceph.mon.c.smithi189.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-17T11:49:43.425 INFO:journalctl@ceph.mon.c.smithi189.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-17T11:49:43.425 INFO:journalctl@ceph.mon.c.smithi189.stdout: Block cache BinnedLRUCache@0x5560ca965090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-17T11:49:43.425 INFO:journalctl@ceph.mon.c.smithi189.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-17T11:49:43.425 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-02-17T11:49:43.425 INFO:journalctl@ceph.mon.c.smithi189.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-17T11:49:43.425 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-17T11:49:43.425 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: using public_addr v2:172.21.15.189:0/0 -> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-02-17T11:49:43.425 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: starting mon.c rank -1 at public addrs [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] at bind addrs [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:49:43.425 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(???) e0 preinit fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:49:43.425 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(synchronizing).mds e1 new map 2024-02-17T11:49:43.426 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-17T11:49:43.426 INFO:journalctl@ceph.mon.c.smithi189.stdout: e1 2024-02-17T11:49:43.426 INFO:journalctl@ceph.mon.c.smithi189.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-17T11:49:43.426 INFO:journalctl@ceph.mon.c.smithi189.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-17T11:49:43.426 INFO:journalctl@ceph.mon.c.smithi189.stdout: legacy client fscid: -1 2024-02-17T11:49:43.426 INFO:journalctl@ceph.mon.c.smithi189.stdout: 2024-02-17T11:49:43.426 INFO:journalctl@ceph.mon.c.smithi189.stdout: No filesystems configured 2024-02-17T11:49:43.426 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-17T11:49:43.426 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-17T11:49:43.426 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-17T11:49:43.426 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-17T11:49:43.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-17T11:49:43.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-17T11:49:43.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-17T11:49:43.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T11:49:43.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T11:49:43.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T11:49:43.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:43.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:49:43.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.427 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi125", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:49:43.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:43.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:49:43.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: Updating smithi125:/etc/ceph/ceph.conf 2024-02-17T11:49:43.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:49:43.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: Updating smithi125:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:49:43.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:43.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: Deploying cephadm binary to smithi189 2024-02-17T11:49:43.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:49:43.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.428 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:43.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: Added host smithi189 2024-02-17T11:49:43.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:49:43.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:43.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:43.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:49:43.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:43.429 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1374655495' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-17T11:49:43.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1374655495' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-17T11:49:43.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T11:49:43.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:49:43.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:43.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:49:43.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.430 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-17T11:49:43.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:49:43.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:49:43.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:49:43.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:43.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi125:172.21.15.125=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:49:43.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi125:172.21.15.125=b;smithi189:172.21.15.189=c;count:3 2024-02-17T11:49:43.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:49:43.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:43.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:49:43.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:43.431 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:49:43.432 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:43.432 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: Deploying daemon mon.c on smithi189 2024-02-17T11:49:43.432 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:43.432 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:43.432 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 ceph-mon[36588]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-17T11:49:43.432 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 bash[36556]: a90006e43b15a690a4cb86daee7382196cda083bfd32953829ee745da343d565 2024-02-17T11:49:43.432 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:43 smithi189 systemd[1]: Started Ceph mon.c for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:49:44.847 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-17T11:49:44.847 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-17T11:49:45.094 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T11:49:48.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:49:48.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: mon.a calling monitor election 2024-02-17T11:49:48.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:48.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:48.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:48.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:48.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: mon.c calling monitor election 2024-02-17T11:49:48.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:48.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:48.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-17T11:49:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: monmap e2: 2 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T11:49:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: fsmap 2024-02-17T11:49:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T11:49:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: mgrmap e13: x(active, since 49s) 2024-02-17T11:49:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: overall HEALTH_OK 2024-02-17T11:49:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:49:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:48.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:49:48.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: mon.a calling monitor election 2024-02-17T11:49:48.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:48.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: mon.c calling monitor election 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: monmap e2: 2 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: fsmap 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: mgrmap e13: x(active, since 49s) 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: overall HEALTH_OK 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:48.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:48.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:48.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:49:48.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:49.467 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:49 smithi094 ceph-mon[32499]: Deploying daemon mon.b on smithi125 2024-02-17T11:49:49.467 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:49 smithi094 ceph-mon[32499]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:49.467 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:49 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:49.553 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.553+0000 7f74959f3700 1 -- 172.21.15.189:0/1795416832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74780035c0 msgr2=0x7f7478005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:49.553 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.553+0000 7f74959f3700 1 --2- 172.21.15.189:0/1795416832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74780035c0 0x7f7478005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7490069e90 tx=0x7f748000feb0 comp rx=0 tx=0).stop 2024-02-17T11:49:49.554 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.553+0000 7f74959f3700 1 -- 172.21.15.189:0/1795416832 shutdown_connections 2024-02-17T11:49:49.554 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.553+0000 7f74959f3700 1 --2- 172.21.15.189:0/1795416832 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74780035c0 0x7f7478005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:49.554 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.553+0000 7f74959f3700 1 -- 172.21.15.189:0/1795416832 >> 172.21.15.189:0/1795416832 conn(0x7f74900fc190 msgr2=0x7f74900fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:49.554 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.554+0000 7f74959f3700 1 -- 172.21.15.189:0/1795416832 shutdown_connections 2024-02-17T11:49:49.554 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.554+0000 7f74959f3700 1 -- 172.21.15.189:0/1795416832 wait complete. 2024-02-17T11:49:49.554 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:49 smithi189 ceph-mon[36588]: Deploying daemon mon.b on smithi125 2024-02-17T11:49:49.554 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:49 smithi189 ceph-mon[36588]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:49.554 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:49 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:49.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.555+0000 7f74959f3700 1 Processor -- start 2024-02-17T11:49:49.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.555+0000 7f74959f3700 1 -- start start 2024-02-17T11:49:49.556 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.555+0000 7f74959f3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74780035c0 0x7f7490199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:49.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.555+0000 7f74959f3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490199750 0x7f749019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:49.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.555+0000 7f74959f3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7490199ce0 con 0x7f74780035c0 2024-02-17T11:49:49.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.555+0000 7f74959f3700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7490199e50 con 0x7f7490199750 2024-02-17T11:49:49.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.555+0000 7f748e7fc700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490199750 0x7f749019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:49.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.556+0000 7f748e7fc700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490199750 0x7f749019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:39434/0 (socket says 172.21.15.189:39434) 2024-02-17T11:49:49.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.556+0000 7f748e7fc700 1 -- 172.21.15.189:0/1843782393 learned_addr learned my addr 172.21.15.189:0/1843782393 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-02-17T11:49:49.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.556+0000 7f748e7fc700 1 -- 172.21.15.189:0/1843782393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490199750 msgr2=0x7f749019dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-17T11:49:49.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.556+0000 7f748e7fc700 1 -- 172.21.15.189:0/1843782393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490199750 msgr2=0x7f749019dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-17T11:49:49.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.556+0000 7f748effd700 1 --2- 172.21.15.189:0/1843782393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74780035c0 0x7f7490199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:49.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.556+0000 7f748e7fc700 1 --2- 172.21.15.189:0/1843782393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490199750 0x7f749019dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-17T11:49:49.557 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.556+0000 7f748e7fc700 1 --2- 172.21.15.189:0/1843782393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490199750 0x7f749019dae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T11:49:49.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.556+0000 7f748effd700 1 -- 172.21.15.189:0/1843782393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490199750 msgr2=0x7f749019dae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T11:49:49.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.556+0000 7f748effd700 1 --2- 172.21.15.189:0/1843782393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490199750 0x7f749019dae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:49.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.556+0000 7f748effd700 1 -- 172.21.15.189:0/1843782393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748000f900 con 0x7f74780035c0 2024-02-17T11:49:49.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.556+0000 7f748effd700 1 --2- 172.21.15.189:0/1843782393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74780035c0 0x7f7490199210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7480010010 tx=0x7f74800056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:49.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.557+0000 7f74949f1700 1 -- 172.21.15.189:0/1843782393 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f7480004aa0 con 0x7f74780035c0 2024-02-17T11:49:49.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.557+0000 7f74949f1700 1 -- 172.21.15.189:0/1843782393 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7480013e40 con 0x7f74780035c0 2024-02-17T11:49:49.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.557+0000 7f74959f3700 1 -- 172.21.15.189:0/1843782393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749019e080 con 0x7f74780035c0 2024-02-17T11:49:49.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.557+0000 7f74949f1700 1 -- 172.21.15.189:0/1843782393 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f7480011a80 con 0x7f74780035c0 2024-02-17T11:49:49.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.557+0000 7f74959f3700 1 -- 172.21.15.189:0/1843782393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749019e5d0 con 0x7f74780035c0 2024-02-17T11:49:49.559 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.558+0000 7f74949f1700 1 -- 172.21.15.189:0/1843782393 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f7480011c60 con 0x7f74780035c0 2024-02-17T11:49:49.559 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.558+0000 7f74959f3700 1 -- 172.21.15.189:0/1843782393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7490106f40 con 0x7f74780035c0 2024-02-17T11:49:49.559 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.558+0000 7f74949f1700 1 --2- 172.21.15.189:0/1843782393 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f747c038d70 0x7f747c03b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:49.559 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.558+0000 7f74949f1700 1 -- 172.21.15.189:0/1843782393 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f74800041c0 con 0x7f74780035c0 2024-02-17T11:49:49.559 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.558+0000 7f748e7fc700 1 --2- 172.21.15.189:0/1843782393 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f747c038d70 0x7f747c03b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:49.559 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.559+0000 7f748e7fc700 1 --2- 172.21.15.189:0/1843782393 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f747c038d70 0x7f747c03b230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7484009180 tx=0x7f7484008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:49.563 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.563+0000 7f74949f1700 1 -- 172.21.15.189:0/1843782393 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f74800109c0 con 0x7f74780035c0 2024-02-17T11:49:49.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.837+0000 7f74959f3700 1 -- 172.21.15.189:0/1843782393 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f749019e8a0 con 0x7f74780035c0 2024-02-17T11:49:49.837 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.837+0000 7f74949f1700 1 -- 172.21.15.189:0/1843782393 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f74800103d0 con 0x7f74780035c0 2024-02-17T11:49:49.837 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:49:49.837 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":2,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","modified":"2024-02-17T11:49:43.105517Z","created":"2024-02-17T11:48:01.408577Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-17T11:49:49.839 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.839+0000 7f74959f3700 1 -- 172.21.15.189:0/1843782393 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f747c038d70 msgr2=0x7f747c03b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:49.839 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.839+0000 7f74959f3700 1 --2- 172.21.15.189:0/1843782393 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f747c038d70 0x7f747c03b230 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7484009180 tx=0x7f7484008040 comp rx=0 tx=0).stop 2024-02-17T11:49:49.839 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.839+0000 7f74959f3700 1 -- 172.21.15.189:0/1843782393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74780035c0 msgr2=0x7f7490199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:49.839 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.839+0000 7f74959f3700 1 --2- 172.21.15.189:0/1843782393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74780035c0 0x7f7490199210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7480010010 tx=0x7f74800056d0 comp rx=0 tx=0).stop 2024-02-17T11:49:49.839 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.839+0000 7f74959f3700 1 -- 172.21.15.189:0/1843782393 shutdown_connections 2024-02-17T11:49:49.839 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.839+0000 7f74959f3700 1 --2- 172.21.15.189:0/1843782393 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f747c038d70 0x7f747c03b230 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:49.840 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.839+0000 7f74959f3700 1 --2- 172.21.15.189:0/1843782393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f74780035c0 0x7f7490199210 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:49.840 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.839+0000 7f74959f3700 1 --2- 172.21.15.189:0/1843782393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7490199750 0x7f749019dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:49.840 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.839+0000 7f74959f3700 1 -- 172.21.15.189:0/1843782393 >> 172.21.15.189:0/1843782393 conn(0x7f74900fc190 msgr2=0x7f74900fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:49.840 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.840+0000 7f74959f3700 1 -- 172.21.15.189:0/1843782393 shutdown_connections 2024-02-17T11:49:49.840 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:49.840+0000 7f74959f3700 1 -- 172.21.15.189:0/1843782393 wait complete. 2024-02-17T11:49:49.841 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 2 2024-02-17T11:49:50.382 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:50 smithi189 ceph-mon[36588]: from='client.? 172.21.15.189:0/1843782393' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T11:49:50.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:50 smithi094 ceph-mon[32499]: from='client.? 172.21.15.189:0/1843782393' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T11:49:51.407 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-17T11:49:51.408 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-17T11:49:51.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:51 smithi189 ceph-mon[36588]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:51.655 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T11:49:51.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:51 smithi094 ceph-mon[32499]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:51.795 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(???) e0 preinit fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:49:51.796 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(synchronizing).mds e1 new map 2024-02-17T11:49:51.796 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-17T11:49:51.796 INFO:journalctl@ceph.mon.b.smithi125.stdout: e1 2024-02-17T11:49:51.796 INFO:journalctl@ceph.mon.b.smithi125.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-17T11:49:51.796 INFO:journalctl@ceph.mon.b.smithi125.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-17T11:49:51.796 INFO:journalctl@ceph.mon.b.smithi125.stdout: legacy client fscid: -1 2024-02-17T11:49:51.796 INFO:journalctl@ceph.mon.b.smithi125.stdout: 2024-02-17T11:49:51.796 INFO:journalctl@ceph.mon.b.smithi125.stdout: No filesystems configured 2024-02-17T11:49:51.796 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-17T11:49:51.796 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-17T11:49:51.796 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-17T11:49:51.796 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-17T11:49:51.796 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-17T11:49:52.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-17T11:49:52.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-17T11:49:52.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T11:49:52.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T11:49:52.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-17T11:49:52.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:52.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi189", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:49:52.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi125", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: Updating smithi125:/etc/ceph/ceph.conf 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: Updating smithi125:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: Deploying cephadm binary to smithi189 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:52.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: Added host smithi189 2024-02-17T11:49:52.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:49:52.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:52.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:52.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:49:52.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:52.188 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1374655495' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-17T11:49:52.188 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1374655495' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-17T11:49:52.188 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T11:49:52.188 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.188 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.188 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.188 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.191 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi189", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:49:52.191 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:52.191 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:49:52.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-17T11:49:52.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:49:52.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:49:52.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:49:52.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:52.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi094:172.21.15.94=a;smithi125:172.21.15.125=b;smithi189:172.21.15.189=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:49:52.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: Saving service mon spec with placement smithi094:172.21.15.94=a;smithi125:172.21.15.125=b;smithi189:172.21.15.189=c;count:3 2024-02-17T11:49:52.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:49:52.192 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: Deploying daemon mon.c on smithi189 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.a calling monitor election 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:52.193 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.c calling monitor election 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: monmap e2: 2 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: fsmap 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mgrmap e13: x(active, since 49s) 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: overall HEALTH_OK 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:52.194 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:49:52.195 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:52.195 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: Deploying daemon mon.b on smithi125 2024-02-17T11:49:52.195 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:52.195 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:52.195 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: from='client.? 172.21.15.189:0/1843782393' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T11:49:52.195 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:52.195 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 ceph-mon[36819]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-17T11:49:52.195 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 bash[36800]: 232c18f9347d9b84a1560773cfc20747b17a1bf6c7f8a9f0302f9728ebecfc14 2024-02-17T11:49:52.195 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:51 smithi125 systemd[1]: Started Ceph mon.b for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:49:57.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:49:57.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: mon.c calling monitor election 2024-02-17T11:49:57.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: mon.a calling monitor election 2024-02-17T11:49:57.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:57.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:57.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:57.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:57.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:57.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:57.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:57.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:57.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:57.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:57.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-17T11:49:57.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: monmap e3: 3 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],b=[v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T11:49:57.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: fsmap 2024-02-17T11:49:57.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T11:49:57.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: mgrmap e13: x(active, since 58s) 2024-02-17T11:49:57.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-17T11:49:57.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-17T11:49:57.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-17T11:49:57.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: mon.b (rank 2) addr [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] is down (out of quorum) 2024-02-17T11:49:57.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:57.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:57.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:57.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:57.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: mon.c calling monitor election 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: mon.a calling monitor election 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-17T11:49:57.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: monmap e3: 3 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],b=[v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T11:49:57.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: fsmap 2024-02-17T11:49:57.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T11:49:57.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: mgrmap e13: x(active, since 58s) 2024-02-17T11:49:57.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-17T11:49:57.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-17T11:49:57.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-17T11:49:57.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: mon.b (rank 2) addr [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] is down (out of quorum) 2024-02-17T11:49:57.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:57.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:57.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:57.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:57.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.443+0000 7f08b063e700 1 -- 172.21.15.189:0/3626955954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08940035c0 msgr2=0x7f0894005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:57.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.443+0000 7f08b063e700 1 --2- 172.21.15.189:0/3626955954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08940035c0 0x7f0894005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f08a8073ec0 tx=0x7f08a400fe60 comp rx=0 tx=0).stop 2024-02-17T11:49:57.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.443+0000 7f08b063e700 1 -- 172.21.15.189:0/3626955954 shutdown_connections 2024-02-17T11:49:57.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.443+0000 7f08b063e700 1 --2- 172.21.15.189:0/3626955954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08940035c0 0x7f0894005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:57.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.443+0000 7f08b063e700 1 -- 172.21.15.189:0/3626955954 >> 172.21.15.189:0/3626955954 conn(0x7f08a80fd210 msgr2=0x7f08a80ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:57.446 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.443+0000 7f08b063e700 1 -- 172.21.15.189:0/3626955954 shutdown_connections 2024-02-17T11:49:57.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.443+0000 7f08b063e700 1 -- 172.21.15.189:0/3626955954 wait complete. 2024-02-17T11:49:57.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.443+0000 7f08b063e700 1 Processor -- start 2024-02-17T11:49:57.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08b063e700 1 -- start start 2024-02-17T11:49:57.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08b063e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08940035c0 0x7f08a81993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:57.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08b063e700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8199900 0x7f08a819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:57.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08b063e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a819e1d0 0x7f08a819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:57.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08b063e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a8199ff0 con 0x7f08a819e1d0 2024-02-17T11:49:57.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08b063e700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a81a5f40 con 0x7f08a8199900 2024-02-17T11:49:57.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08b063e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a81a6080 con 0x7f08940035c0 2024-02-17T11:49:57.447 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08ae3da700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08940035c0 0x7f08a81993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:57.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08ae3da700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08940035c0 0x7f08a81993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:36842/0 (socket says 172.21.15.189:36842) 2024-02-17T11:49:57.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08ae3da700 1 -- 172.21.15.189:0/305157961 learned_addr learned my addr 172.21.15.189:0/305157961 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-02-17T11:49:57.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08adbd9700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8199900 0x7f08a819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:57.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08aebdb700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a819e1d0 0x7f08a819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:57.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08ae3da700 1 -- 172.21.15.189:0/305157961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08940035c0 msgr2=0x7f08a81993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-17T11:49:57.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08ae3da700 1 -- 172.21.15.189:0/305157961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08940035c0 msgr2=0x7f08a81993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-17T11:49:57.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08ae3da700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08940035c0 0x7f08a81993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-17T11:49:57.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08ae3da700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08940035c0 0x7f08a81993c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T11:49:57.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08adbd9700 1 -- 172.21.15.189:0/305157961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8199900 msgr2=0x7f08a819dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-17T11:49:57.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08adbd9700 1 -- 172.21.15.189:0/305157961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8199900 msgr2=0x7f08a819dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-17T11:49:57.448 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08adbd9700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8199900 0x7f08a819dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-17T11:49:57.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08adbd9700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8199900 0x7f08a819dc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T11:49:57.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08aebdb700 1 -- 172.21.15.189:0/305157961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08940035c0 msgr2=0x7f08a81993c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T11:49:57.449 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08aebdb700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08940035c0 0x7f08a81993c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:57.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08aebdb700 1 -- 172.21.15.189:0/305157961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8199900 msgr2=0x7f08a819dc90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T11:49:57.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08aebdb700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8199900 0x7f08a819dc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:57.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.444+0000 7f08aebdb700 1 -- 172.21.15.189:0/305157961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a400f8b0 con 0x7f08a819e1d0 2024-02-17T11:49:57.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.445+0000 7f08aebdb700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a819e1d0 0x7f08a819e680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f08a000bd50 tx=0x7f08a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:57.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.445+0000 7f089f7fe700 1 -- 172.21.15.189:0/305157961 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a000cd10 con 0x7f08a819e1d0 2024-02-17T11:49:57.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.445+0000 7f08b063e700 1 -- 172.21.15.189:0/305157961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a81a6360 con 0x7f08a819e1d0 2024-02-17T11:49:57.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.445+0000 7f089f7fe700 1 -- 172.21.15.189:0/305157961 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08a000ce70 con 0x7f08a819e1d0 2024-02-17T11:49:57.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.445+0000 7f089f7fe700 1 -- 172.21.15.189:0/305157961 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a00128e0 con 0x7f08a819e1d0 2024-02-17T11:49:57.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.445+0000 7f08b063e700 1 -- 172.21.15.189:0/305157961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a81a6860 con 0x7f08a819e1d0 2024-02-17T11:49:57.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.446+0000 7f08b063e700 1 -- 172.21.15.189:0/305157961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a80632a0 con 0x7f08a819e1d0 2024-02-17T11:49:57.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.446+0000 7f089f7fe700 1 -- 172.21.15.189:0/305157961 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f08a0012a40 con 0x7f08a819e1d0 2024-02-17T11:49:57.450 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.446+0000 7f089f7fe700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f088c039410 0x7f088c03b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:49:57.451 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.446+0000 7f089f7fe700 1 -- 172.21.15.189:0/305157961 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f08a004b730 con 0x7f08a819e1d0 2024-02-17T11:49:57.451 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.447+0000 7f08ae3da700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f088c039410 0x7f088c03b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:49:57.452 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.447+0000 7f08ae3da700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f088c039410 0x7f088c03b8d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08a4005850 tx=0x7f08a4001e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:49:57.452 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.452+0000 7f089f7fe700 1 -- 172.21.15.189:0/305157961 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f08a0011800 con 0x7f08a819e1d0 2024-02-17T11:49:57.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.745+0000 7f08b063e700 1 -- 172.21.15.189:0/305157961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f08a810b7e0 con 0x7f08a819e1d0 2024-02-17T11:49:57.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.745+0000 7f089f7fe700 1 -- 172.21.15.189:0/305157961 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1283 (secure 0 0 0) 0x7f08a0019070 con 0x7f08a819e1d0 2024-02-17T11:49:57.745 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-02-17T11:49:57.745 INFO:teuthology.orchestra.run.smithi189.stdout:{"epoch":3,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","modified":"2024-02-17T11:49:51.802220Z","created":"2024-02-17T11:48:01.408577Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:3300","nonce":0},{"type":"v1","addr":"172.21.15.94:6789","nonce":0}]},"addr":"172.21.15.94:6789/0","public_addr":"172.21.15.94:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:3300","nonce":0},{"type":"v1","addr":"172.21.15.189:6789","nonce":0}]},"addr":"172.21.15.189:6789/0","public_addr":"172.21.15.189:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:3300","nonce":0},{"type":"v1","addr":"172.21.15.125:6789","nonce":0}]},"addr":"172.21.15.125:6789/0","public_addr":"172.21.15.125:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-17T11:49:57.747 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.747+0000 7f08b063e700 1 -- 172.21.15.189:0/305157961 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f088c039410 msgr2=0x7f088c03b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:57.747 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.748+0000 7f08b063e700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f088c039410 0x7f088c03b8d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08a4005850 tx=0x7f08a4001e50 comp rx=0 tx=0).stop 2024-02-17T11:49:57.747 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.748+0000 7f08b063e700 1 -- 172.21.15.189:0/305157961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a819e1d0 msgr2=0x7f08a819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:49:57.747 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.748+0000 7f08b063e700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a819e1d0 0x7f08a819e680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f08a000bd50 tx=0x7f08a000be30 comp rx=0 tx=0).stop 2024-02-17T11:49:57.747 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.748+0000 7f08b063e700 1 -- 172.21.15.189:0/305157961 shutdown_connections 2024-02-17T11:49:57.747 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.748+0000 7f08b063e700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f088c039410 0x7f088c03b8d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:57.747 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.748+0000 7f08b063e700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a819e1d0 0x7f08a819e680 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:57.748 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.748+0000 7f08b063e700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08940035c0 0x7f08a81993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:57.748 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.748+0000 7f08b063e700 1 --2- 172.21.15.189:0/305157961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8199900 0x7f08a819dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:49:57.748 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.748+0000 7f08b063e700 1 -- 172.21.15.189:0/305157961 >> 172.21.15.189:0/305157961 conn(0x7f08a80fd210 msgr2=0x7f08a80ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:49:57.748 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.748+0000 7f08b063e700 1 -- 172.21.15.189:0/305157961 shutdown_connections 2024-02-17T11:49:57.748 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:49:57.748+0000 7f08b063e700 1 -- 172.21.15.189:0/305157961 wait complete. 2024-02-17T11:49:57.748 INFO:teuthology.orchestra.run.smithi189.stderr:dumped monmap epoch 3 2024-02-17T11:49:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:49:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:57 smithi189 ceph-mon[36588]: from='client.? 172.21.15.189:0/305157961' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T11:49:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:58.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:49:58.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:57 smithi094 ceph-mon[32499]: from='client.? 172.21.15.189:0/305157961' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T11:49:58.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:58.412 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-17T11:49:58.412 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph config generate-minimal-conf 2024-02-17T11:49:58.658 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:49:59.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:49:59.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: mon.c calling monitor election 2024-02-17T11:49:59.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: mon.a calling monitor election 2024-02-17T11:49:59.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:59.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:49:59.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:59.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:59.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:59.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:59.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: monmap e3: 3 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],b=[v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: fsmap 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: mgrmap e13: x(active, since 58s) 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: mon.b (rank 2) addr [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] is down (out of quorum) 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:49:59.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='client.? 172.21.15.189:0/305157961' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-17T11:49:59.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:49:59.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: mon.b calling monitor election 2024-02-17T11:49:59.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: mon.b calling monitor election 2024-02-17T11:49:59.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: mon.c calling monitor election 2024-02-17T11:49:59.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: mon.a calling monitor election 2024-02-17T11:49:59.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-17T11:49:59.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: monmap e3: 3 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],b=[v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T11:49:59.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: fsmap 2024-02-17T11:49:59.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: mgrmap e13: x(active, since 60s) 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: Cluster is now healthy 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: overall HEALTH_OK 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:49:59.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T11:49:59.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:49:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: mon.b calling monitor election 2024-02-17T11:49:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: mon.b calling monitor election 2024-02-17T11:49:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: mon.c calling monitor election 2024-02-17T11:49:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: mon.a calling monitor election 2024-02-17T11:49:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-17T11:49:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: monmap e3: 3 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],b=[v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T11:49:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: fsmap 2024-02-17T11:49:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T11:49:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: mgrmap e13: x(active, since 60s) 2024-02-17T11:49:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-17T11:49:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: Cluster is now healthy 2024-02-17T11:49:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: overall HEALTH_OK 2024-02-17T11:49:59.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:59.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:49:59.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:49:59.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T11:49:59.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:49:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:59.937 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: mon.b calling monitor election 2024-02-17T11:49:59.937 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: mon.b calling monitor election 2024-02-17T11:49:59.937 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: mon.c calling monitor election 2024-02-17T11:49:59.937 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: mon.a calling monitor election 2024-02-17T11:49:59.937 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: monmap e3: 3 mons at {a=[v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0],b=[v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0],c=[v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: fsmap 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: osdmap e4: 0 total, 0 up, 0 in 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: mgrmap e13: x(active, since 60s) 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: Cluster is now healthy 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: overall HEALTH_OK 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:49:59.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:49:59.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T11:49:59.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:49:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:00.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.254+0000 7fea5d3a8700 1 -- 172.21.15.94:0/4237774745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea58107650 msgr2=0x7fea58107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.254+0000 7fea5d3a8700 1 --2- 172.21.15.94:0/4237774745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea58107650 0x7fea58107a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fea48009910 tx=0x7fea48009c20 comp rx=0 tx=0).stop 2024-02-17T11:50:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.254+0000 7fea5d3a8700 1 -- 172.21.15.94:0/4237774745 shutdown_connections 2024-02-17T11:50:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.254+0000 7fea5d3a8700 1 --2- 172.21.15.94:0/4237774745 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea58107650 0x7fea58107a30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.254+0000 7fea5d3a8700 1 -- 172.21.15.94:0/4237774745 >> 172.21.15.94:0/4237774745 conn(0x7fea580fd180 msgr2=0x7fea580ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:50:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.254+0000 7fea5d3a8700 1 -- 172.21.15.94:0/4237774745 shutdown_connections 2024-02-17T11:50:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.254+0000 7fea5d3a8700 1 -- 172.21.15.94:0/4237774745 wait complete. 2024-02-17T11:50:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.255+0000 7fea5d3a8700 1 Processor -- start 2024-02-17T11:50:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.255+0000 7fea5d3a8700 1 -- start start 2024-02-17T11:50:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.255+0000 7fea5d3a8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea58107650 0x7fea581971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.255+0000 7fea5d3a8700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea58197730 0x7fea5819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.255+0000 7fea5d3a8700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea5819c000 0x7fea5819c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.255+0000 7fea5d3a8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea58197e20 con 0x7fea58107650 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.255+0000 7fea5d3a8700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea581a3d70 con 0x7fea5819c000 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.255+0000 7fea5d3a8700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea581a3eb0 con 0x7fea58197730 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.255+0000 7fea56ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea58107650 0x7fea581971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.255+0000 7fea56ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea58107650 0x7fea581971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40256/0 (socket says 172.21.15.94:40256) 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.255+0000 7fea56ffd700 1 -- 172.21.15.94:0/1209631348 learned_addr learned my addr 172.21.15.94:0/1209631348 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.255+0000 7fea567fc700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea58197730 0x7fea5819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.255+0000 7fea577fe700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea5819c000 0x7fea5819c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.256+0000 7fea56ffd700 1 -- 172.21.15.94:0/1209631348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea58197730 msgr2=0x7fea5819bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.256+0000 7fea56ffd700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea58197730 0x7fea5819bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.256+0000 7fea56ffd700 1 -- 172.21.15.94:0/1209631348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea5819c000 msgr2=0x7fea5819c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.256+0000 7fea56ffd700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea5819c000 0x7fea5819c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.256+0000 7fea56ffd700 1 -- 172.21.15.94:0/1209631348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea480095f0 con 0x7fea58107650 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.256+0000 7fea577fe700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea5819c000 0x7fea5819c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.256+0000 7fea56ffd700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea58107650 0x7fea581971f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fea48005b40 tx=0x7fea4800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:50:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.256+0000 7fea3ffff700 1 -- 172.21.15.94:0/1209631348 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4801d070 con 0x7fea58107650 2024-02-17T11:50:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.256+0000 7fea3ffff700 1 -- 172.21.15.94:0/1209631348 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea4800fb50 con 0x7fea58107650 2024-02-17T11:50:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.256+0000 7fea5d3a8700 1 -- 172.21.15.94:0/1209631348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea581a4190 con 0x7fea58107650 2024-02-17T11:50:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.256+0000 7fea3ffff700 1 -- 172.21.15.94:0/1209631348 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea48022c90 con 0x7fea58107650 2024-02-17T11:50:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.256+0000 7fea5d3a8700 1 -- 172.21.15.94:0/1209631348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea581a4690 con 0x7fea58107650 2024-02-17T11:50:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.257+0000 7fea3ffff700 1 -- 172.21.15.94:0/1209631348 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fea48017440 con 0x7fea58107650 2024-02-17T11:50:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.257+0000 7fea5d3a8700 1 -- 172.21.15.94:0/1209631348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea58102d90 con 0x7fea58107650 2024-02-17T11:50:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.257+0000 7fea3ffff700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fea40039410 0x7fea4003b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.257+0000 7fea3ffff700 1 -- 172.21.15.94:0/1209631348 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fea4804d1f0 con 0x7fea58107650 2024-02-17T11:50:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.257+0000 7fea567fc700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fea40039410 0x7fea4003b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.258+0000 7fea567fc700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fea40039410 0x7fea4003b8d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009ad0 tx=0x7fea4c009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:50:00.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.262+0000 7fea3ffff700 1 -- 172.21.15.94:0/1209631348 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fea48020960 con 0x7fea58107650 2024-02-17T11:50:00.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.457+0000 7fea5d3a8700 1 -- 172.21.15.94:0/1209631348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fea5810ba40 con 0x7fea58107650 2024-02-17T11:50:00.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.457+0000 7fea3ffff700 1 -- 172.21.15.94:0/1209631348 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7fea48027070 con 0x7fea58107650 2024-02-17T11:50:00.457 INFO:teuthology.orchestra.run.smithi094.stdout:# minimal ceph.conf for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:50:00.457 INFO:teuthology.orchestra.run.smithi094.stdout:[global] 2024-02-17T11:50:00.457 INFO:teuthology.orchestra.run.smithi094.stdout: fsid = 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:50:00.457 INFO:teuthology.orchestra.run.smithi094.stdout: mon_host = [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] 2024-02-17T11:50:00.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.459+0000 7fea5d3a8700 1 -- 172.21.15.94:0/1209631348 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fea40039410 msgr2=0x7fea4003b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:00.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.459+0000 7fea5d3a8700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fea40039410 0x7fea4003b8d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fea4c009ad0 tx=0x7fea4c009210 comp rx=0 tx=0).stop 2024-02-17T11:50:00.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.459+0000 7fea5d3a8700 1 -- 172.21.15.94:0/1209631348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea58107650 msgr2=0x7fea581971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:00.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.459+0000 7fea5d3a8700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea58107650 0x7fea581971f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fea48005b40 tx=0x7fea4800bcf0 comp rx=0 tx=0).stop 2024-02-17T11:50:00.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.460+0000 7fea5d3a8700 1 -- 172.21.15.94:0/1209631348 shutdown_connections 2024-02-17T11:50:00.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.460+0000 7fea5d3a8700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fea40039410 0x7fea4003b8d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:00.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.460+0000 7fea5d3a8700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea5819c000 0x7fea5819c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:00.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.460+0000 7fea5d3a8700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea58107650 0x7fea581971f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:00.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.460+0000 7fea5d3a8700 1 --2- 172.21.15.94:0/1209631348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea58197730 0x7fea5819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:00.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.460+0000 7fea5d3a8700 1 -- 172.21.15.94:0/1209631348 >> 172.21.15.94:0/1209631348 conn(0x7fea580fd180 msgr2=0x7fea58103150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:50:00.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.460+0000 7fea5d3a8700 1 -- 172.21.15.94:0/1209631348 shutdown_connections 2024-02-17T11:50:00.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:00.460+0000 7fea5d3a8700 1 -- 172.21.15.94:0/1209631348 wait complete. 2024-02-17T11:50:00.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:00 smithi125 ceph-mon[36819]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-17T11:50:00.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:00 smithi125 ceph-mon[36819]: Updating smithi125:/etc/ceph/ceph.conf 2024-02-17T11:50:00.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:00 smithi125 ceph-mon[36819]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-17T11:50:00.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:00 smithi125 ceph-mon[36819]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:50:00.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:00 smithi125 ceph-mon[36819]: Updating smithi094:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:50:00.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:00 smithi125 ceph-mon[36819]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:50:00.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:00 smithi125 ceph-mon[36819]: Reconfiguring mon.a (unknown last config time)... 2024-02-17T11:50:00.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:00 smithi125 ceph-mon[36819]: Reconfiguring daemon mon.a on smithi094 2024-02-17T11:50:00.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:00 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:50:00.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:00 smithi125 ceph-mon[36819]: overall HEALTH_OK 2024-02-17T11:50:00.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:00 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1209631348' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:00.749 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:00 smithi094 ceph-mon[32499]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-17T11:50:00.749 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:00 smithi094 ceph-mon[32499]: Updating smithi125:/etc/ceph/ceph.conf 2024-02-17T11:50:00.749 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:00 smithi094 ceph-mon[32499]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-17T11:50:00.750 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:00 smithi094 ceph-mon[32499]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:50:00.750 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:00 smithi094 ceph-mon[32499]: Updating smithi094:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:50:00.750 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:00 smithi094 ceph-mon[32499]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:50:00.750 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:00 smithi094 ceph-mon[32499]: Reconfiguring mon.a (unknown last config time)... 2024-02-17T11:50:00.750 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:00 smithi094 ceph-mon[32499]: Reconfiguring daemon mon.a on smithi094 2024-02-17T11:50:00.750 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:00 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:50:00.750 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:00 smithi094 ceph-mon[32499]: overall HEALTH_OK 2024-02-17T11:50:00.750 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:00 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1209631348' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:00 smithi189 ceph-mon[36588]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:00 smithi189 ceph-mon[36588]: Updating smithi125:/etc/ceph/ceph.conf 2024-02-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:00 smithi189 ceph-mon[36588]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:00 smithi189 ceph-mon[36588]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:00 smithi189 ceph-mon[36588]: Updating smithi094:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:00 smithi189 ceph-mon[36588]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:50:00.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:00 smithi189 ceph-mon[36588]: Reconfiguring mon.a (unknown last config time)... 2024-02-17T11:50:00.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:00 smithi189 ceph-mon[36588]: Reconfiguring daemon mon.a on smithi094 2024-02-17T11:50:00.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:00 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:50:00.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:00 smithi189 ceph-mon[36588]: overall HEALTH_OK 2024-02-17T11:50:00.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:00 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1209631348' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:01.433 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-17T11:50:01.433 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:50:01.433 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.conf 2024-02-17T11:50:01.467 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:50:01.467 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:50:01.536 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-17T11:50:01.536 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/ceph/ceph.conf 2024-02-17T11:50:01.571 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-17T11:50:01.571 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:50:01.645 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-17T11:50:01.645 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.conf 2024-02-17T11:50:01.689 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-17T11:50:01.690 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:50:01.703 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:01 smithi094 ceph-mon[32499]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:01.761 INFO:tasks.cephadm:Adding mgr.x on smithi094 2024-02-17T11:50:01.761 INFO:tasks.cephadm:Adding mgr.y on smithi125 2024-02-17T11:50:01.761 INFO:tasks.cephadm:Adding mgr.z on smithi189 2024-02-17T11:50:01.761 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch apply mgr '3;smithi094=x;smithi125=y;smithi189=z' 2024-02-17T11:50:01.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:01 smithi189 ceph-mon[36588]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:01.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:01 smithi125 ceph-mon[36819]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:02.036 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T11:50:03.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:03.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:03.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:02 smithi125 ceph-mon[36819]: Reconfiguring mon.b (monmap changed)... 2024-02-17T11:50:03.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:50:03.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T11:50:03.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:03.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:02 smithi125 ceph-mon[36819]: Reconfiguring daemon mon.b on smithi125 2024-02-17T11:50:03.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:02 smithi125 ceph-mon[36819]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:03.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:03.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:03.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:02 smithi094 ceph-mon[32499]: Reconfiguring mon.b (monmap changed)... 2024-02-17T11:50:03.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:50:03.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T11:50:03.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:03.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:02 smithi094 ceph-mon[32499]: Reconfiguring daemon mon.b on smithi125 2024-02-17T11:50:03.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:02 smithi094 ceph-mon[32499]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:03.223 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:03.223 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:03.223 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:02 smithi189 ceph-mon[36588]: Reconfiguring mon.b (monmap changed)... 2024-02-17T11:50:03.223 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:50:03.223 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T11:50:03.223 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:03.223 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:02 smithi189 ceph-mon[36588]: Reconfiguring daemon mon.b on smithi125 2024-02-17T11:50:03.223 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:02 smithi189 ceph-mon[36588]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:03.422 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.422+0000 7fa3dce53700 1 -- 172.21.15.189:0/3487088728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d81006d0 msgr2=0x7fa3d8100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:03.422 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.422+0000 7fa3dce53700 1 --2- 172.21.15.189:0/3487088728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d81006d0 0x7fa3d8100af0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009910 tx=0x7fa3c8009c20 comp rx=0 tx=0).stop 2024-02-17T11:50:03.422 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.422+0000 7fa3dce53700 1 -- 172.21.15.189:0/3487088728 shutdown_connections 2024-02-17T11:50:03.422 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.422+0000 7fa3dce53700 1 --2- 172.21.15.189:0/3487088728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d81006d0 0x7fa3d8100af0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:03.422 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.422+0000 7fa3dce53700 1 -- 172.21.15.189:0/3487088728 >> 172.21.15.189:0/3487088728 conn(0x7fa3d80fbc50 msgr2=0x7fa3d80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:50:03.423 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.422+0000 7fa3dce53700 1 -- 172.21.15.189:0/3487088728 shutdown_connections 2024-02-17T11:50:03.423 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.422+0000 7fa3dce53700 1 -- 172.21.15.189:0/3487088728 wait complete. 2024-02-17T11:50:03.423 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.423+0000 7fa3dce53700 1 Processor -- start 2024-02-17T11:50:03.423 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.423+0000 7fa3dce53700 1 -- start start 2024-02-17T11:50:03.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.423+0000 7fa3dce53700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3d81006d0 0x7fa3d8196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:03.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.423+0000 7fa3dce53700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3d8197340 0x7fa3d819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:03.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.423+0000 7fa3dce53700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d81977c0 0x7fa3d819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:03.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.423+0000 7fa3dce53700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d819f0e0 con 0x7fa3d81977c0 2024-02-17T11:50:03.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.423+0000 7fa3dce53700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d819f250 con 0x7fa3d8197340 2024-02-17T11:50:03.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.423+0000 7fa3dce53700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d819f410 con 0x7fa3d81006d0 2024-02-17T11:50:03.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.423+0000 7fa3d659c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3d81006d0 0x7fa3d8196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:03.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.423+0000 7fa3d659c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3d81006d0 0x7fa3d8196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:36850/0 (socket says 172.21.15.189:36850) 2024-02-17T11:50:03.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.423+0000 7fa3d659c700 1 -- 172.21.15.189:0/3909896208 learned_addr learned my addr 172.21.15.189:0/3909896208 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-02-17T11:50:03.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.424+0000 7fa3d6d9d700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d81977c0 0x7fa3d819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:03.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.424+0000 7fa3d5d9b700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3d8197340 0x7fa3d819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:03.425 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.424+0000 7fa3d659c700 1 -- 172.21.15.189:0/3909896208 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3d8197340 msgr2=0x7fa3d819c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:03.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.424+0000 7fa3d659c700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3d8197340 0x7fa3d819c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:03.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.424+0000 7fa3d659c700 1 -- 172.21.15.189:0/3909896208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d81977c0 msgr2=0x7fa3d819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:03.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.424+0000 7fa3d659c700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d81977c0 0x7fa3d819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:03.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.424+0000 7fa3d659c700 1 -- 172.21.15.189:0/3909896208 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c80095f0 con 0x7fa3d81006d0 2024-02-17T11:50:03.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.424+0000 7fa3d659c700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3d81006d0 0x7fa3d8196e00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800bcf0 tx=0x7fa3c800bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:50:03.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.424+0000 7fa3c77fe700 1 -- 172.21.15.189:0/3909896208 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c801d070 con 0x7fa3d81006d0 2024-02-17T11:50:03.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.424+0000 7fa3c77fe700 1 -- 172.21.15.189:0/3909896208 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa3c8004ec0 con 0x7fa3d81006d0 2024-02-17T11:50:03.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.424+0000 7fa3dce53700 1 -- 172.21.15.189:0/3909896208 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d819f690 con 0x7fa3d81006d0 2024-02-17T11:50:03.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.424+0000 7fa3c77fe700 1 -- 172.21.15.189:0/3909896208 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c8022e30 con 0x7fa3d81006d0 2024-02-17T11:50:03.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.424+0000 7fa3dce53700 1 -- 172.21.15.189:0/3909896208 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d819fc00 con 0x7fa3d81006d0 2024-02-17T11:50:03.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.425+0000 7fa3c77fe700 1 -- 172.21.15.189:0/3909896208 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fa3c800f460 con 0x7fa3d81006d0 2024-02-17T11:50:03.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.425+0000 7fa3dce53700 1 -- 172.21.15.189:0/3909896208 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d81908a0 con 0x7fa3d81006d0 2024-02-17T11:50:03.426 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.425+0000 7fa3c77fe700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa3c0039410 0x7fa3c003b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:03.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.425+0000 7fa3c77fe700 1 -- 172.21.15.189:0/3909896208 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa3c804cfe0 con 0x7fa3d81006d0 2024-02-17T11:50:03.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.426+0000 7fa3d5d9b700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa3c0039410 0x7fa3c003b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:03.427 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.426+0000 7fa3d5d9b700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa3c0039410 0x7fa3c003b8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009850 tx=0x7fa3cc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:50:03.429 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.430+0000 7fa3c77fe700 1 -- 172.21.15.189:0/3909896208 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa3c8020960 con 0x7fa3d81006d0 2024-02-17T11:50:03.640 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.640+0000 7fa3dce53700 1 -- 172.21.15.189:0/3909896208 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi125=y;smithi189=z", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3d802d7d0 con 0x7fa3c0039410 2024-02-17T11:50:03.646 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.647+0000 7fa3c77fe700 1 -- 172.21.15.189:0/3909896208 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa3d802d7d0 con 0x7fa3c0039410 2024-02-17T11:50:03.646 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mgr update... 2024-02-17T11:50:03.649 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.649+0000 7fa3dce53700 1 -- 172.21.15.189:0/3909896208 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa3c0039410 msgr2=0x7fa3c003b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:03.649 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.649+0000 7fa3dce53700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa3c0039410 0x7fa3c003b8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009850 tx=0x7fa3cc008040 comp rx=0 tx=0).stop 2024-02-17T11:50:03.650 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.649+0000 7fa3dce53700 1 -- 172.21.15.189:0/3909896208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3d81006d0 msgr2=0x7fa3d8196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:03.650 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.649+0000 7fa3dce53700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3d81006d0 0x7fa3d8196e00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800bcf0 tx=0x7fa3c800bd20 comp rx=0 tx=0).stop 2024-02-17T11:50:03.650 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.650+0000 7fa3dce53700 1 -- 172.21.15.189:0/3909896208 shutdown_connections 2024-02-17T11:50:03.650 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.650+0000 7fa3dce53700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa3c0039410 0x7fa3c003b8d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:03.650 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.650+0000 7fa3dce53700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3d81977c0 0x7fa3d819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:03.650 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.650+0000 7fa3dce53700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3d81006d0 0x7fa3d8196e00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:03.650 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.650+0000 7fa3dce53700 1 --2- 172.21.15.189:0/3909896208 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3d8197340 0x7fa3d819c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:03.650 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.650+0000 7fa3dce53700 1 -- 172.21.15.189:0/3909896208 >> 172.21.15.189:0/3909896208 conn(0x7fa3d80fbc50 msgr2=0x7fa3d806a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:50:03.651 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.650+0000 7fa3dce53700 1 -- 172.21.15.189:0/3909896208 shutdown_connections 2024-02-17T11:50:03.651 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:03.650+0000 7fa3dce53700 1 -- 172.21.15.189:0/3909896208 wait complete. 2024-02-17T11:50:04.225 DEBUG:teuthology.orchestra.run.smithi125:mgr.y> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mgr.y.service 2024-02-17T11:50:04.228 DEBUG:teuthology.orchestra.run.smithi189:mgr.z> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mgr.z.service 2024-02-17T11:50:04.231 INFO:tasks.cephadm:Deploying OSDs... 2024-02-17T11:50:04.231 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:50:04.231 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/scratch_devs of=/dev/stdout 2024-02-17T11:50:04.250 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-17T11:50:04.250 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_1 2024-02-17T11:50:04.308 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-17T11:50:04.309 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T11:50:04.309 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-17T11:50:04.309 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T11:50:04.309 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T11:50:04.309 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-02-17 11:49:10.857301526 +0000 2024-02-17T11:50:04.309 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-02-17 11:44:08.232340240 +0000 2024-02-17T11:50:04.309 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-02-17 11:44:08.232340240 +0000 2024-02-17T11:50:04.309 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-02-17 11:44:08.232340240 +0000 2024-02-17T11:50:04.309 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-17T11:50:04.379 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-02-17T11:50:04.380 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-02-17T11:50:04.380 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 0.000128279 s, 4.0 MB/s 2024-02-17T11:50:04.381 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-17T11:50:04.438 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_2 2024-02-17T11:50:04.496 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-17T11:50:04.496 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T11:50:04.496 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-02-17T11:50:04.496 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T11:50:04.496 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T11:50:04.496 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-02-17 11:49:10.890300986 +0000 2024-02-17T11:50:04.496 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-02-17 11:44:08.562334803 +0000 2024-02-17T11:50:04.496 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-02-17 11:44:08.562334803 +0000 2024-02-17T11:50:04.496 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-02-17 11:44:08.562334803 +0000 2024-02-17T11:50:04.497 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-17T11:50:04.564 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-02-17T11:50:04.564 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-02-17T11:50:04.564 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 9.4946e-05 s, 5.4 MB/s 2024-02-17T11:50:04.565 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-17T11:50:04.623 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_3 2024-02-17T11:50:04.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:04 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:04.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:04 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:04.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:04 smithi189 ceph-mon[36588]: Reconfiguring mon.c (monmap changed)... 2024-02-17T11:50:04.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:04 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:50:04.668 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:04 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T11:50:04.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:04 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:04.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:04 smithi189 ceph-mon[36588]: Reconfiguring daemon mon.c on smithi189 2024-02-17T11:50:04.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:04 smithi189 ceph-mon[36588]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi125=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:50:04.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:04 smithi189 ceph-mon[36588]: Saving service mgr spec with placement smithi094=x;smithi125=y;smithi189=z;count:3 2024-02-17T11:50:04.669 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:04 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-17T11:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T11:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-17T11:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T11:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T11:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-02-17 11:49:10.928300364 +0000 2024-02-17T11:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-02-17 11:44:08.893329349 +0000 2024-02-17T11:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-02-17 11:44:08.893329349 +0000 2024-02-17T11:50:04.679 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-02-17 11:44:08.893329349 +0000 2024-02-17T11:50:04.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-17T11:50:04.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:04 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:04.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:04 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:04.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:04 smithi125 ceph-mon[36819]: Reconfiguring mon.c (monmap changed)... 2024-02-17T11:50:04.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:04 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:50:04.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:04 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T11:50:04.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:04 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:04.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:04 smithi125 ceph-mon[36819]: Reconfiguring daemon mon.c on smithi189 2024-02-17T11:50:04.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:04 smithi125 ceph-mon[36819]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi125=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:50:04.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:04 smithi125 ceph-mon[36819]: Saving service mgr spec with placement smithi094=x;smithi125=y;smithi189=z;count:3 2024-02-17T11:50:04.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:04 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:04.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:04 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:04.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:04 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:04.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:04 smithi094 ceph-mon[32499]: Reconfiguring mon.c (monmap changed)... 2024-02-17T11:50:04.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:04 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-17T11:50:04.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:04 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-17T11:50:04.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:04 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:04.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:04 smithi094 ceph-mon[32499]: Reconfiguring daemon mon.c on smithi189 2024-02-17T11:50:04.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:04 smithi094 ceph-mon[32499]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi094=x;smithi125=y;smithi189=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:50:04.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:04 smithi094 ceph-mon[32499]: Saving service mgr spec with placement smithi094=x;smithi125=y;smithi189=z;count:3 2024-02-17T11:50:04.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:04 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:04.732 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-02-17T11:50:04.732 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-02-17T11:50:04.732 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 9.6254e-05 s, 5.3 MB/s 2024-02-17T11:50:04.733 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-17T11:50:04.790 DEBUG:teuthology.orchestra.run.smithi094:> stat /dev/vg_nvme/lv_4 2024-02-17T11:50:04.846 INFO:teuthology.orchestra.run.smithi094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-17T11:50:04.847 INFO:teuthology.orchestra.run.smithi094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T11:50:04.847 INFO:teuthology.orchestra.run.smithi094.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-17T11:50:04.847 INFO:teuthology.orchestra.run.smithi094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T11:50:04.847 INFO:teuthology.orchestra.run.smithi094.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T11:50:04.847 INFO:teuthology.orchestra.run.smithi094.stdout:Access: 2024-02-17 11:49:10.965299759 +0000 2024-02-17T11:50:04.847 INFO:teuthology.orchestra.run.smithi094.stdout:Modify: 2024-02-17 11:44:09.222323928 +0000 2024-02-17T11:50:04.847 INFO:teuthology.orchestra.run.smithi094.stdout:Change: 2024-02-17 11:44:09.222323928 +0000 2024-02-17T11:50:04.847 INFO:teuthology.orchestra.run.smithi094.stdout: Birth: 2024-02-17 11:44:09.222323928 +0000 2024-02-17T11:50:04.847 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-17T11:50:04.914 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records in 2024-02-17T11:50:04.914 INFO:teuthology.orchestra.run.smithi094.stderr:1+0 records out 2024-02-17T11:50:04.914 INFO:teuthology.orchestra.run.smithi094.stderr:512 bytes copied, 9.4944e-05 s, 5.4 MB/s 2024-02-17T11:50:04.915 DEBUG:teuthology.orchestra.run.smithi094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-17T11:50:04.972 DEBUG:teuthology.orchestra.run.smithi125:> set -ex 2024-02-17T11:50:04.972 DEBUG:teuthology.orchestra.run.smithi125:> dd if=/scratch_devs of=/dev/stdout 2024-02-17T11:50:04.991 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-17T11:50:04.991 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_1 2024-02-17T11:50:05.048 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-17T11:50:05.049 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T11:50:05.049 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-02-17T11:50:05.049 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T11:50:05.049 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T11:50:05.049 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2024-02-17 11:49:51.250350543 +0000 2024-02-17T11:50:05.049 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2024-02-17 11:44:37.956757619 +0000 2024-02-17T11:50:05.049 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2024-02-17 11:44:37.956757619 +0000 2024-02-17T11:50:05.049 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2024-02-17 11:44:37.956757619 +0000 2024-02-17T11:50:05.049 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-17T11:50:05.118 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2024-02-17T11:50:05.119 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2024-02-17T11:50:05.119 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000289687 s, 1.8 MB/s 2024-02-17T11:50:05.120 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-17T11:50:05.179 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_2 2024-02-17T11:50:05.236 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-17T11:50:05.236 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T11:50:05.236 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-02-17T11:50:05.237 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T11:50:05.237 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T11:50:05.237 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2024-02-17 11:49:51.250350543 +0000 2024-02-17T11:50:05.237 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2024-02-17 11:44:38.297751538 +0000 2024-02-17T11:50:05.237 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2024-02-17 11:44:38.297751538 +0000 2024-02-17T11:50:05.237 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2024-02-17 11:44:38.297751538 +0000 2024-02-17T11:50:05.237 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-17T11:50:05.306 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2024-02-17T11:50:05.306 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2024-02-17T11:50:05.306 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000247045 s, 2.1 MB/s 2024-02-17T11:50:05.307 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-17T11:50:05.365 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_3 2024-02-17T11:50:05.421 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-17T11:50:05.421 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T11:50:05.421 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-02-17T11:50:05.421 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T11:50:05.421 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T11:50:05.422 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2024-02-17 11:49:51.250350543 +0000 2024-02-17T11:50:05.422 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2024-02-17 11:44:38.628745635 +0000 2024-02-17T11:50:05.422 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2024-02-17 11:44:38.628745635 +0000 2024-02-17T11:50:05.422 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2024-02-17 11:44:38.628745635 +0000 2024-02-17T11:50:05.422 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-17T11:50:05.488 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2024-02-17T11:50:05.489 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2024-02-17T11:50:05.489 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000299571 s, 1.7 MB/s 2024-02-17T11:50:05.490 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-17T11:50:05.547 DEBUG:teuthology.orchestra.run.smithi125:> stat /dev/vg_nvme/lv_4 2024-02-17T11:50:05.603 INFO:teuthology.orchestra.run.smithi125.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-17T11:50:05.603 INFO:teuthology.orchestra.run.smithi125.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T11:50:05.603 INFO:teuthology.orchestra.run.smithi125.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-02-17T11:50:05.603 INFO:teuthology.orchestra.run.smithi125.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T11:50:05.603 INFO:teuthology.orchestra.run.smithi125.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T11:50:05.603 INFO:teuthology.orchestra.run.smithi125.stdout:Access: 2024-02-17 11:49:51.251350527 +0000 2024-02-17T11:50:05.603 INFO:teuthology.orchestra.run.smithi125.stdout:Modify: 2024-02-17 11:44:38.973739483 +0000 2024-02-17T11:50:05.604 INFO:teuthology.orchestra.run.smithi125.stdout:Change: 2024-02-17 11:44:38.973739483 +0000 2024-02-17T11:50:05.604 INFO:teuthology.orchestra.run.smithi125.stdout: Birth: 2024-02-17 11:44:38.973739483 +0000 2024-02-17T11:50:05.604 DEBUG:teuthology.orchestra.run.smithi125:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-17T11:50:05.670 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records in 2024-02-17T11:50:05.671 INFO:teuthology.orchestra.run.smithi125.stderr:1+0 records out 2024-02-17T11:50:05.671 INFO:teuthology.orchestra.run.smithi125.stderr:512 bytes copied, 0.000256029 s, 2.0 MB/s 2024-02-17T11:50:05.672 DEBUG:teuthology.orchestra.run.smithi125:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-17T11:50:05.728 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-02-17T11:50:05.729 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-02-17T11:50:05.746 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-17T11:50:05.746 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-02-17T11:50:05.802 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-17T11:50:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T11:50:05.803 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-02-17T11:50:05.803 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T11:50:05.803 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T11:50:05.803 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-02-17 11:49:42.341671722 +0000 2024-02-17T11:50:05.803 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-02-17 11:44:38.197626744 +0000 2024-02-17T11:50:05.803 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-02-17 11:44:38.197626744 +0000 2024-02-17T11:50:05.803 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-02-17 11:44:38.197626744 +0000 2024-02-17T11:50:05.803 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-17T11:50:05.869 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:05 smithi189 ceph-mon[36588]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:05.869 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:05 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:05.869 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:05 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:05.869 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:05 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:05.872 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-02-17T11:50:05.872 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-02-17T11:50:05.872 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000269085 s, 1.9 MB/s 2024-02-17T11:50:05.873 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-17T11:50:05.931 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-02-17T11:50:05.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:05 smithi125 ceph-mon[36819]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:05.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:05 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:05.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:05 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:05.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:05 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:05.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:05 smithi094 ceph-mon[32499]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:05.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:05 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:05.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:05 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:05.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:05 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-17T11:50:05.986 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T11:50:05.986 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-02-17T11:50:05.986 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T11:50:05.986 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T11:50:05.986 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-02-17 11:49:42.341671722 +0000 2024-02-17T11:50:05.986 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-02-17 11:44:38.545619422 +0000 2024-02-17T11:50:05.986 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-02-17 11:44:38.545619422 +0000 2024-02-17T11:50:05.987 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-02-17 11:44:38.545619422 +0000 2024-02-17T11:50:05.987 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-17T11:50:06.052 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-02-17T11:50:06.052 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-02-17T11:50:06.052 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.00020473 s, 2.5 MB/s 2024-02-17T11:50:06.053 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-17T11:50:06.108 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-02-17T11:50:06.163 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-17T11:50:06.164 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T11:50:06.164 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-02-17T11:50:06.164 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T11:50:06.164 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T11:50:06.164 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-02-17 11:49:42.341671722 +0000 2024-02-17T11:50:06.164 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-02-17 11:44:38.876612458 +0000 2024-02-17T11:50:06.164 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-02-17 11:44:38.876612458 +0000 2024-02-17T11:50:06.164 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-02-17 11:44:38.876612458 +0000 2024-02-17T11:50:06.164 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-17T11:50:06.230 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-02-17T11:50:06.230 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-02-17T11:50:06.230 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000227298 s, 2.3 MB/s 2024-02-17T11:50:06.231 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-17T11:50:06.286 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-02-17T11:50:06.342 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-17T11:50:06.342 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-17T11:50:06.342 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-02-17T11:50:06.342 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-17T11:50:06.342 INFO:teuthology.orchestra.run.smithi189.stdout:Context: system_u:object_r:device_t:s0 2024-02-17T11:50:06.342 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-02-17 11:49:42.342671702 +0000 2024-02-17T11:50:06.342 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-02-17 11:44:39.231604989 +0000 2024-02-17T11:50:06.342 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-02-17 11:44:39.231604989 +0000 2024-02-17T11:50:06.342 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: 2024-02-17 11:44:39.231604989 +0000 2024-02-17T11:50:06.343 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-17T11:50:06.409 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-02-17T11:50:06.443 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-02-17T11:50:06.443 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000225632 s, 2.3 MB/s 2024-02-17T11:50:06.444 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-17T11:50:06.499 INFO:tasks.cephadm:Deploying osd.0 on smithi094 with /dev/vg_nvme/lv_4... 2024-02-17T11:50:06.499 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-17T11:50:06.574 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:06 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:06.575 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:06 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:06.575 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:06 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:06.575 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:06 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T11:50:06.575 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:06 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-17T11:50:06.575 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:06 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T11:50:06.575 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:06 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:06.735 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:50:06.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:06 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:06.858 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:06 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:06.858 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:06 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:06.858 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:06 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T11:50:06.858 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:06 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-17T11:50:06.858 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:06 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T11:50:06.858 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:06 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:06.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:06 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:06.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:06 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:06.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:06 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:06.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:06 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T11:50:06.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:06 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-17T11:50:06.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:06 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T11:50:06.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:06 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:07.691 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:07 smithi189 ceph-mon[36588]: Deploying daemon mgr.z on smithi189 2024-02-17T11:50:07.691 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:07 smithi189 ceph-mon[36588]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:07.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:07 smithi094 ceph-mon[32499]: Deploying daemon mgr.z on smithi189 2024-02-17T11:50:07.764 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:07 smithi094 ceph-mon[32499]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:07.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:07 smithi125 ceph-mon[36819]: Deploying daemon mgr.z on smithi189 2024-02-17T11:50:07.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:07 smithi125 ceph-mon[36819]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:08.063 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:07 smithi189 systemd[1]: Starting Ceph mgr.z for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:50:08.333 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:08 smithi189 podman[38436]: 2024-02-17 11:50:08.06429617 +0000 UTC m=+0.109132678 container create 907011b69dcd9fd9bad578f3a60863c27376ebb6526de702df265092aac13b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-17T11:50:08.333 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:08 smithi189 podman[38436]: 2024-02-17 11:50:07.972573528 +0000 UTC m=+0.017410050 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T11:50:08.333 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:08 smithi189 podman[38436]: 2024-02-17 11:50:08.284166354 +0000 UTC m=+0.329002868 container init 907011b69dcd9fd9bad578f3a60863c27376ebb6526de702df265092aac13b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-17T11:50:08.333 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:08 smithi189 podman[38436]: 2024-02-17 11:50:08.286581999 +0000 UTC m=+0.331418508 container start 907011b69dcd9fd9bad578f3a60863c27376ebb6526de702df265092aac13b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-17T11:50:08.333 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:08 smithi189 ceph-mgr[38455]: -- 172.21.15.189:0/840793957 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b9975201a0 con 0x55b9974f6800 2024-02-17T11:50:08.333 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:08 smithi189 bash[38436]: 907011b69dcd9fd9bad578f3a60863c27376ebb6526de702df265092aac13b32 2024-02-17T11:50:08.603 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-17T11:50:08.603 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-17T11:50:08.603 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-02-17T11:50:08.603 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-02-17T11:50:08.603 INFO:teuthology.orchestra.run.smithi094.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0085019 s, 1.2 GB/s 2024-02-17T11:50:08.603 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-02-17T11:50:08.650 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:08 smithi189 systemd[1]: Started Ceph mgr.z for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:50:09.151 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:08 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:08.828+0000 7f809f9ec200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T11:50:09.151 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:08 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:08.922+0000 7f809f9ec200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T11:50:09.198 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi094:vg_nvme/lv_4 2024-02-17T11:50:09.434 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:50:09.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:09 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:09.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:09 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:09.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:09 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:09.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:09 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T11:50:09.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:09 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-17T11:50:09.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:09 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T11:50:09.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:09 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:09.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:09 smithi094 ceph-mon[32499]: Deploying daemon mgr.y on smithi125 2024-02-17T11:50:09.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:09 smithi094 ceph-mon[32499]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:09.651 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:09 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:09.570+0000 7f809f9ec200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T11:50:09.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:09 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:09.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:09 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:09.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:09 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:09.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:09 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T11:50:09.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:09 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-17T11:50:09.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:09 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T11:50:09.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:09 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:09.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:09 smithi189 ceph-mon[36588]: Deploying daemon mgr.y on smithi125 2024-02-17T11:50:09.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:09 smithi189 ceph-mon[36588]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:09.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:09 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:09.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:09 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:09.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:09 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:09.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:09 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T11:50:09.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:09 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-17T11:50:09.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:09 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T11:50:09.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:09 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:09.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:09 smithi125 ceph-mon[36819]: Deploying daemon mgr.y on smithi125 2024-02-17T11:50:09.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:09 smithi125 ceph-mon[36819]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:10.001 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:09 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:09.668+0000 7f809f9ec200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T11:50:10.001 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:09 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:09.816+0000 7f809f9ec200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T11:50:10.002 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:09 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:09.916+0000 7f809f9ec200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T11:50:10.002 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:10 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:10.002+0000 7f809f9ec200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T11:50:10.569 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:10 smithi125 systemd[1]: Starting Ceph mgr.y for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:50:10.569 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:10 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:10.274+0000 7f809f9ec200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T11:50:10.570 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:10 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:10.386+0000 7f809f9ec200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T11:50:10.570 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:10 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:10.474+0000 7f809f9ec200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T11:50:10.570 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:10 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:10.570+0000 7f809f9ec200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T11:50:10.829 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:10 smithi125 podman[38191]: 2024-02-17 11:50:10.570053496 +0000 UTC m=+0.092622924 container create bb76ce3979008a37b54d1708008b9d54bb6c82e4eb375293ccf834bd80d28bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-17T11:50:10.829 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:10 smithi125 podman[38191]: 2024-02-17 11:50:10.494604384 +0000 UTC m=+0.017173827 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T11:50:10.829 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:10 smithi125 podman[38191]: 2024-02-17 11:50:10.76370993 +0000 UTC m=+0.286279382 container init bb76ce3979008a37b54d1708008b9d54bb6c82e4eb375293ccf834bd80d28bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, io.buildah.version=1.33.5) 2024-02-17T11:50:10.829 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:10 smithi125 podman[38191]: 2024-02-17 11:50:10.766143424 +0000 UTC m=+0.288712850 container start bb76ce3979008a37b54d1708008b9d54bb6c82e4eb375293ccf834bd80d28bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-17T11:50:10.829 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:10 smithi125 ceph-mgr[38210]: -- 172.21.15.125:0/1026069294 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5641a88c01a0 con 0x5641a8897000 2024-02-17T11:50:10.829 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:10 smithi125 bash[38191]: bb76ce3979008a37b54d1708008b9d54bb6c82e4eb375293ccf834bd80d28bf0 2024-02-17T11:50:10.901 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:10 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:10.749+0000 7f809f9ec200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T11:50:11.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.007+0000 7f7655b59700 1 -- 172.21.15.94:0/819005061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7650074650 msgr2=0x7f7650074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:11.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.007+0000 7f7655b59700 1 --2- 172.21.15.94:0/819005061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7650074650 0x7f7650074ad0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7644009990 tx=0x7f7644009ca0 comp rx=0 tx=0).stop 2024-02-17T11:50:11.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.007+0000 7f7655b59700 1 -- 172.21.15.94:0/819005061 shutdown_connections 2024-02-17T11:50:11.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.007+0000 7f7655b59700 1 --2- 172.21.15.94:0/819005061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7650109ba0 0x7f765010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:11.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.007+0000 7f7655b59700 1 --2- 172.21.15.94:0/819005061 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7650075c20 0x7f7650074080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:11.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.007+0000 7f7655b59700 1 --2- 172.21.15.94:0/819005061 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7650074650 0x7f7650074ad0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:11.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.007+0000 7f7655b59700 1 -- 172.21.15.94:0/819005061 >> 172.21.15.94:0/819005061 conn(0x7f76500fd280 msgr2=0x7f76500ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:50:11.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.007+0000 7f7655b59700 1 -- 172.21.15.94:0/819005061 shutdown_connections 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.007+0000 7f7655b59700 1 -- 172.21.15.94:0/819005061 wait complete. 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.007+0000 7f7655b59700 1 Processor -- start 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f7655b59700 1 -- start start 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f7655b59700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7650074650 0x7f7650199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f7655b59700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7650075c20 0x7f765019a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f7655b59700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7650109ba0 0x7f76501a13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f7655b59700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76501a19b0 con 0x7f7650075c20 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f7655b59700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76501a1af0 con 0x7f7650074650 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f7655b59700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76501a1c80 con 0x7f7650109ba0 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f764effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7650075c20 0x7f765019a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f764effd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7650075c20 0x7f765019a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37126/0 (socket says 172.21.15.94:37126) 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f764effd700 1 -- 172.21.15.94:0/993503054 learned_addr learned my addr 172.21.15.94:0/993503054 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f764f7fe700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7650074650 0x7f7650199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:11.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f764ffff700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7650109ba0 0x7f76501a13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:11.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f764effd700 1 -- 172.21.15.94:0/993503054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7650109ba0 msgr2=0x7f76501a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:11.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f764effd700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7650109ba0 0x7f76501a13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:11.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f764effd700 1 -- 172.21.15.94:0/993503054 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7650074650 msgr2=0x7f7650199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:11.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f764effd700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7650074650 0x7f7650199e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:11.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f764effd700 1 -- 172.21.15.94:0/993503054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76440095f0 con 0x7f7650075c20 2024-02-17T11:50:11.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f764f7fe700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7650074650 0x7f7650199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:50:11.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.008+0000 7f764effd700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7650075c20 0x7f765019a360 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7644000c00 tx=0x7f764400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:50:11.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.009+0000 7f764cff9700 1 -- 172.21.15.94:0/993503054 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764401d070 con 0x7f7650075c20 2024-02-17T11:50:11.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.009+0000 7f764cff9700 1 -- 172.21.15.94:0/993503054 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7644022470 con 0x7f7650075c20 2024-02-17T11:50:11.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.009+0000 7f7655b59700 1 -- 172.21.15.94:0/993503054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76501a1f00 con 0x7f7650075c20 2024-02-17T11:50:11.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.009+0000 7f764cff9700 1 -- 172.21.15.94:0/993503054 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644017650 con 0x7f7650075c20 2024-02-17T11:50:11.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.009+0000 7f7655b59700 1 -- 172.21.15.94:0/993503054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76501a22b0 con 0x7f7650075c20 2024-02-17T11:50:11.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.009+0000 7f764cff9700 1 -- 172.21.15.94:0/993503054 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f76440177b0 con 0x7f7650075c20 2024-02-17T11:50:11.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.010+0000 7f764cff9700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f76380391c0 0x7f763803b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:11.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.010+0000 7f764cff9700 1 -- 172.21.15.94:0/993503054 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f764404df10 con 0x7f7650075c20 2024-02-17T11:50:11.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.010+0000 7f764f7fe700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f76380391c0 0x7f763803b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:11.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.010+0000 7f7655b59700 1 -- 172.21.15.94:0/993503054 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76501089b0 con 0x7f7650075c20 2024-02-17T11:50:11.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.010+0000 7f764f7fe700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f76380391c0 0x7f763803b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7640010420 tx=0x7f76400073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:50:11.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.015+0000 7f764cff9700 1 -- 172.21.15.94:0/993503054 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7644027020 con 0x7f7650075c20 2024-02-17T11:50:11.185 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:10 smithi125 systemd[1]: Started Ceph mgr.y for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:50:11.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:11.221+0000 7f7655b59700 1 -- 172.21.15.94:0/993503054 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f76500620b0 con 0x7f76380391c0 2024-02-17T11:50:11.621 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:11 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:11.332+0000 7fb32b45d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T11:50:11.621 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:11 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:11.431+0000 7fb32b45d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T11:50:11.880 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:11 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:11.772+0000 7f809f9ec200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T11:50:11.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:11 smithi189 ceph-mon[36588]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:11.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:11.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:11.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:11.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:11.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:11.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:50:11.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:50:11.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:11.930 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:11 smithi094 ceph-mon[32499]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:11.930 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:11.931 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:11.931 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:11.931 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:11.931 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:11.931 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:50:11.931 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:50:11.931 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:11.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:11 smithi125 ceph-mon[36819]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:11.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:11.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:11.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:11.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:11.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:11.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:50:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:50:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:12.150 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:11 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:11.909+0000 7f809f9ec200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T11:50:12.307 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:12.130+0000 7fb32b45d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T11:50:12.307 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:12.224+0000 7fb32b45d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T11:50:12.546 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:12.257+0000 7f809f9ec200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T11:50:12.546 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:12.349+0000 7f809f9ec200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T11:50:12.622 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:12.376+0000 7fb32b45d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T11:50:12.622 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:12.473+0000 7fb32b45d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T11:50:12.622 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:12.554+0000 7fb32b45d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T11:50:12.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:12 smithi094 ceph-mon[32499]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:50:12.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:12 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:12 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:12 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:12 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:12 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:12.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:12 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:12.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:12 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:12 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T11:50:12.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:12 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T11:50:12.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:12 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:12.901 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:12.546+0000 7f809f9ec200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:50:12.901 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:12.728+0000 7f809f9ec200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T11:50:12.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-mon[36588]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:50:12.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:12.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:12.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T11:50:12.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T11:50:12.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:12 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:12.924 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:12.815+0000 7fb32b45d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T11:50:12.924 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-mon[36819]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:50:12.924 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.924 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.925 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.925 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.925 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:12.925 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:12.925 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:12.925 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-17T11:50:12.925 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-17T11:50:12.925 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:13.184 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:12.922+0000 7fb32b45d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T11:50:13.184 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:13 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:13.008+0000 7fb32b45d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T11:50:13.184 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:13 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:13.100+0000 7fb32b45d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T11:50:13.269 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:13 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:13.009+0000 7f809f9ec200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:50:13.269 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:13 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:13.270+0000 7f809f9ec200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T11:50:13.623 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:13 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:13.506+0000 7f809f9ec200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T11:50:13.623 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:13 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:13.274+0000 7fb32b45d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T11:50:13.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:13 smithi189 ceph-mon[36588]: Reconfiguring mgr.x (unknown last config time)... 2024-02-17T11:50:13.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:13 smithi189 ceph-mon[36588]: Reconfiguring daemon mgr.x on smithi094 2024-02-17T11:50:13.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:13 smithi189 ceph-mon[36588]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:13 smithi125 ceph-mon[36819]: Reconfiguring mgr.x (unknown last config time)... 2024-02-17T11:50:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:13 smithi125 ceph-mon[36819]: Reconfiguring daemon mgr.x on smithi094 2024-02-17T11:50:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:13 smithi125 ceph-mon[36819]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:13 smithi094 ceph-mon[32499]: Reconfiguring mgr.x (unknown last config time)... 2024-02-17T11:50:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:13 smithi094 ceph-mon[32499]: Reconfiguring daemon mgr.x on smithi094 2024-02-17T11:50:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:13 smithi094 ceph-mon[32499]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:14.624 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:14 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:14.302+0000 7f809f9ec200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T11:50:14.625 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:14.237+0000 7fb32b45d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T11:50:14.625 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:14.367+0000 7fb32b45d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T11:50:14.901 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:14 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:14.873+0000 7f809f9ec200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T11:50:14.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:14 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:14.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:14 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/613599744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7ed7cb6-3fc6-4264-8da2-fb1c340fe308"}]: dispatch 2024-02-17T11:50:14.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:14 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/613599744' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7ed7cb6-3fc6-4264-8da2-fb1c340fe308"}]': finished 2024-02-17T11:50:14.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:14 smithi189 ceph-mon[36588]: osdmap e5: 1 total, 0 up, 1 in 2024-02-17T11:50:14.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:14 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:14.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:14 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3551439617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:50:14.934 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:14.698+0000 7fb32b45d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T11:50:14.934 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:14.787+0000 7fb32b45d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T11:50:14.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:14 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:14.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:14 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/613599744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7ed7cb6-3fc6-4264-8da2-fb1c340fe308"}]: dispatch 2024-02-17T11:50:14.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:14 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/613599744' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7ed7cb6-3fc6-4264-8da2-fb1c340fe308"}]': finished 2024-02-17T11:50:14.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:14 smithi125 ceph-mon[36819]: osdmap e5: 1 total, 0 up, 1 in 2024-02-17T11:50:14.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:14 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:14.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:14 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3551439617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:50:14.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:14 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:14.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:14 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/613599744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e7ed7cb6-3fc6-4264-8da2-fb1c340fe308"}]: dispatch 2024-02-17T11:50:14.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:14 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/613599744' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7ed7cb6-3fc6-4264-8da2-fb1c340fe308"}]': finished 2024-02-17T11:50:14.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:14 smithi094 ceph-mon[32499]: osdmap e5: 1 total, 0 up, 1 in 2024-02-17T11:50:14.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:14 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:14.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:14 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3551439617' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:50:15.401 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:14 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:14.972+0000 7f809f9ec200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T11:50:15.401 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:50:15 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:50:15.187+0000 7f809f9ec200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T11:50:15.434 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:14.980+0000 7fb32b45d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:50:15.434 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:15 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:15.163+0000 7fb32b45d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T11:50:15.701 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:15 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:15.444+0000 7fb32b45d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:50:15.701 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:15 smithi125 ceph-mon[36819]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:15.701 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:15 smithi125 ceph-mon[36819]: Standby manager daemon z started 2024-02-17T11:50:15.701 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:15 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.189:0/954646861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-17T11:50:15.701 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:15 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.189:0/954646861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T11:50:15.701 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:15 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.189:0/954646861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-17T11:50:15.701 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:15 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.189:0/954646861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T11:50:15.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:15.847+0000 7f764cff9700 1 -- 172.21.15.94:0/993503054 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 14) v1 ==== 91350+0+0 (secure 0 0 0) 0x7f764402c960 con 0x7f7650075c20 2024-02-17T11:50:15.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:15 smithi189 ceph-mon[36588]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:15.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:15 smithi189 ceph-mon[36588]: Standby manager daemon z started 2024-02-17T11:50:15.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:15 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.189:0/954646861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-17T11:50:15.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:15 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.189:0/954646861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T11:50:15.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:15 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.189:0/954646861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-17T11:50:15.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:15 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.189:0/954646861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T11:50:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:15 smithi094 ceph-mon[32499]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:15 smithi094 ceph-mon[32499]: Standby manager daemon z started 2024-02-17T11:50:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:15 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.189:0/954646861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-17T11:50:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:15 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.189:0/954646861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T11:50:15.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:15 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.189:0/954646861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-17T11:50:15.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:15 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.189:0/954646861' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T11:50:16.184 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:15 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:15.701+0000 7fb32b45d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T11:50:16.184 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:15 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:15.935+0000 7fb32b45d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T11:50:17.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:16 smithi189 ceph-mon[36588]: mgrmap e14: x(active, since 77s), standbys: z 2024-02-17T11:50:17.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:16 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-17T11:50:17.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:16 smithi189 ceph-mon[36588]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:17.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:16 smithi125 ceph-mon[36819]: mgrmap e14: x(active, since 77s), standbys: z 2024-02-17T11:50:17.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:16 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-17T11:50:17.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:16 smithi125 ceph-mon[36819]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:17.185 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:16 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:16.732+0000 7fb32b45d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T11:50:17.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:16 smithi094 ceph-mon[32499]: mgrmap e14: x(active, since 77s), standbys: z 2024-02-17T11:50:17.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:16 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-17T11:50:17.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:16 smithi094 ceph-mon[32499]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:17.606 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:17 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:17.297+0000 7fb32b45d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T11:50:17.607 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:17 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:17.394+0000 7fb32b45d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T11:50:17.607 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:50:17 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:50:17.607+0000 7fb32b45d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T11:50:17.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:17.860+0000 7f764cff9700 1 -- 172.21.15.94:0/993503054 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f764402ac20 con 0x7f7650075c20 2024-02-17T11:50:17.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:17 smithi125 ceph-mon[36819]: Standby manager daemon y started 2024-02-17T11:50:17.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:17 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.125:0/3648775589' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-17T11:50:17.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:17 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.125:0/3648775589' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T11:50:17.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:17 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.125:0/3648775589' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-17T11:50:17.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:17 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.125:0/3648775589' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T11:50:18.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:17 smithi189 ceph-mon[36588]: Standby manager daemon y started 2024-02-17T11:50:18.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:17 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.125:0/3648775589' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-17T11:50:18.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:17 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.125:0/3648775589' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T11:50:18.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:17 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.125:0/3648775589' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-17T11:50:18.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:17 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.125:0/3648775589' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T11:50:18.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:17 smithi094 ceph-mon[32499]: Standby manager daemon y started 2024-02-17T11:50:18.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:17 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.125:0/3648775589' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-17T11:50:18.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:17 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.125:0/3648775589' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T11:50:18.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:17 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.125:0/3648775589' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-17T11:50:18.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:17 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.125:0/3648775589' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T11:50:19.041 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:18 smithi094 ceph-mon[32499]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-02-17T11:50:19.041 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-17T11:50:19.041 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:18 smithi094 ceph-mon[32499]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:19.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:18 smithi189 ceph-mon[36588]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-02-17T11:50:19.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-17T11:50:19.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:18 smithi189 ceph-mon[36588]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:19.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:18 smithi125 ceph-mon[36819]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-02-17T11:50:19.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-17T11:50:19.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:18 smithi125 ceph-mon[36819]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:20.543 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:20 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:20.543 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:20 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:20.543 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:20 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:20.543 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:20 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:20.543 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:20 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:20.543 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:20 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:20.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:20 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:20.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:20 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:20.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:20 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:20.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:20 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:20.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:20 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:20.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:20 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:20.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:20 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:20.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:20 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:20.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:20 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:20.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:20 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:20.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:20 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:20.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:20 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:21.566 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:21 smithi094 ceph-mon[32499]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:21.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:21 smithi189 ceph-mon[36588]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:21.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:21 smithi125 ceph-mon[36819]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:22.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:22 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-17T11:50:22.662 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:22 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:22.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:22 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-17T11:50:22.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:22 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:22.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:22 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-17T11:50:22.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:22 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:23.660 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:23 smithi094 ceph-mon[32499]: Deploying daemon osd.0 on smithi094 2024-02-17T11:50:23.660 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:23 smithi094 ceph-mon[32499]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:23.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:23 smithi125 ceph-mon[36819]: Deploying daemon osd.0 on smithi094 2024-02-17T11:50:23.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:23 smithi125 ceph-mon[36819]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:23.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:23 smithi189 ceph-mon[36588]: Deploying daemon osd.0 on smithi094 2024-02-17T11:50:23.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:23 smithi189 ceph-mon[36588]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:25.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:25 smithi094 ceph-mon[32499]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:25.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:25 smithi189 ceph-mon[36588]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:25.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:25 smithi125 ceph-mon[36819]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:27.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:27 smithi189 ceph-mon[36588]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:27.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:27.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:27.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:27.922 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:27 smithi094 ceph-mon[32499]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:27.923 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:27.923 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:27.923 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:27.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:27 smithi125 ceph-mon[36819]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:27.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:27.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:27.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:29.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:29 smithi094 ceph-mon[32499]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:29.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:29 smithi189 ceph-mon[36588]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:29.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:29 smithi125 ceph-mon[36819]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:30.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:30.590+0000 7f764cff9700 1 -- 172.21.15.94:0/993503054 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f76500620b0 con 0x7f76380391c0 2024-02-17T11:50:30.590 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 0 on host 'smithi094' 2024-02-17T11:50:30.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:30.593+0000 7f7655b59700 1 -- 172.21.15.94:0/993503054 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f76380391c0 msgr2=0x7f763803b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:30.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:30.593+0000 7f7655b59700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f76380391c0 0x7f763803b680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7640010420 tx=0x7f76400073d0 comp rx=0 tx=0).stop 2024-02-17T11:50:30.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:30.593+0000 7f7655b59700 1 -- 172.21.15.94:0/993503054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7650075c20 msgr2=0x7f765019a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:30.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:30.593+0000 7f7655b59700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7650075c20 0x7f765019a360 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7644000c00 tx=0x7f764400f690 comp rx=0 tx=0).stop 2024-02-17T11:50:30.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:30.593+0000 7f7655b59700 1 -- 172.21.15.94:0/993503054 shutdown_connections 2024-02-17T11:50:30.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:30.593+0000 7f7655b59700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f76380391c0 0x7f763803b680 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:30.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:30.593+0000 7f7655b59700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7650109ba0 0x7f76501a13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:30.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:30.593+0000 7f7655b59700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7650074650 0x7f7650199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:30.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:30.593+0000 7f7655b59700 1 --2- 172.21.15.94:0/993503054 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7650075c20 0x7f765019a360 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:30.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:30.593+0000 7f7655b59700 1 -- 172.21.15.94:0/993503054 >> 172.21.15.94:0/993503054 conn(0x7f76500fd280 msgr2=0x7f7650105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:50:30.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:30.593+0000 7f7655b59700 1 -- 172.21.15.94:0/993503054 shutdown_connections 2024-02-17T11:50:30.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:50:30.593+0000 7f7655b59700 1 -- 172.21.15.94:0/993503054 wait complete. 2024-02-17T11:50:31.272 DEBUG:teuthology.orchestra.run.smithi094:osd.0> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.0.service 2024-02-17T11:50:31.275 INFO:tasks.cephadm:Deploying osd.1 on smithi125 with /dev/vg_nvme/lv_4... 2024-02-17T11:50:31.276 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-17T11:50:31.517 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T11:50:31.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:31 smithi189 ceph-mon[36588]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:31.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:31 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:31.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:31 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:31.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:31 smithi189 ceph-mon[36588]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-17T11:50:31.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:31 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:31.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:31 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:31.907 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:31 smithi125 ceph-mon[36819]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:31.907 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:31 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:31.907 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:31 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:31.907 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:31 smithi125 ceph-mon[36819]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-17T11:50:31.907 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:31 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:31.907 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:31 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:31 smithi094 ceph-mon[32499]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:31 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:31 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:31 smithi094 ceph-mon[32499]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-17T11:50:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:31 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:31 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:32.883 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:32 smithi125 ceph-mon[36819]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-17T11:50:32.883 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:32 smithi125 ceph-mon[36819]: osdmap e6: 1 total, 0 up, 1 in 2024-02-17T11:50:32.883 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:32 smithi125 ceph-mon[36819]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-17T11:50:32.883 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:32 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:32.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:32 smithi189 ceph-mon[36588]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-17T11:50:32.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:32 smithi189 ceph-mon[36588]: osdmap e6: 1 total, 0 up, 1 in 2024-02-17T11:50:32.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:32 smithi189 ceph-mon[36588]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-17T11:50:32.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:32 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:32.919 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:32 smithi094 ceph-mon[32499]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-17T11:50:32.919 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:32 smithi094 ceph-mon[32499]: osdmap e6: 1 total, 0 up, 1 in 2024-02-17T11:50:32.919 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:32 smithi094 ceph-mon[32499]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-17T11:50:32.919 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:32 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:33.205 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 11:50:33 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0[47699]: 2024-02-17T11:50:33.122+0000 7f680116b700 -1 osd.0 0 waiting for initial osdmap 2024-02-17T11:50:33.205 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 11:50:33 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0[47699]: 2024-02-17T11:50:33.125+0000 7f67f8f56700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T11:50:33.472 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-17T11:50:33.472 INFO:teuthology.orchestra.run.smithi125.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-17T11:50:33.472 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 10+0 records in 2024-02-17T11:50:33.472 INFO:teuthology.orchestra.run.smithi125.stderr:10+0 records out 2024-02-17T11:50:33.472 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117482 s, 893 MB/s 2024-02-17T11:50:33.472 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping successful for: 2024-02-17T11:50:33.783 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:33 smithi125 ceph-mon[36819]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:33.783 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:33 smithi125 ceph-mon[36819]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-17T11:50:33.783 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:33 smithi125 ceph-mon[36819]: osdmap e7: 1 total, 0 up, 1 in 2024-02-17T11:50:33.783 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:33 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:33.783 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:33 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:33.783 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:33 smithi125 ceph-mon[36819]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' 2024-02-17T11:50:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:33 smithi189 ceph-mon[36588]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:33 smithi189 ceph-mon[36588]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-17T11:50:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:33 smithi189 ceph-mon[36588]: osdmap e7: 1 total, 0 up, 1 in 2024-02-17T11:50:33.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:33 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:33.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:33 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:33.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:33 smithi189 ceph-mon[36588]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' 2024-02-17T11:50:33.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:33 smithi094 ceph-mon[32499]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:33.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:33 smithi094 ceph-mon[32499]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-17T11:50:33.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:33 smithi094 ceph-mon[32499]: osdmap e7: 1 total, 0 up, 1 in 2024-02-17T11:50:33.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:33 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:33.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:33 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:33.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:33 smithi094 ceph-mon[32499]: from='osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800]' entity='osd.0' 2024-02-17T11:50:34.111 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi125:vg_nvme/lv_4 2024-02-17T11:50:34.351 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T11:50:34.752 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:34 smithi125 ceph-mon[36819]: purged_snaps scrub starts 2024-02-17T11:50:34.752 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:34 smithi125 ceph-mon[36819]: purged_snaps scrub ok 2024-02-17T11:50:34.752 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:34 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:34.752 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:34 smithi125 ceph-mon[36819]: osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800] boot 2024-02-17T11:50:34.752 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:34 smithi125 ceph-mon[36819]: osdmap e8: 1 total, 1 up, 1 in 2024-02-17T11:50:34.752 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:34 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:34.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:34 smithi189 ceph-mon[36588]: purged_snaps scrub starts 2024-02-17T11:50:34.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:34 smithi189 ceph-mon[36588]: purged_snaps scrub ok 2024-02-17T11:50:34.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:34 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:34.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:34 smithi189 ceph-mon[36588]: osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800] boot 2024-02-17T11:50:34.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:34 smithi189 ceph-mon[36588]: osdmap e8: 1 total, 1 up, 1 in 2024-02-17T11:50:34.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:34 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:34.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:34 smithi094 ceph-mon[32499]: purged_snaps scrub starts 2024-02-17T11:50:34.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:34 smithi094 ceph-mon[32499]: purged_snaps scrub ok 2024-02-17T11:50:34.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:34 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:34.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:34 smithi094 ceph-mon[32499]: osd.0 [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800] boot 2024-02-17T11:50:34.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:34 smithi094 ceph-mon[32499]: osdmap e8: 1 total, 1 up, 1 in 2024-02-17T11:50:34.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:34 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:50:35.759 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:35.759 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: Detected new or changed devices on smithi094 2024-02-17T11:50:35.759 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:35.759 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:35.759 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:50:35.759 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi094 to 17088M 2024-02-17T11:50:35.760 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:35.760 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:35.760 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:35.760 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:35.760 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:35.760 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:35.760 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:35.760 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:35.861 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.862+0000 7efe1a0ad700 1 -- 172.21.15.125:0/2371921451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe14102730 msgr2=0x7efe14104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:35.862 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.862+0000 7efe1a0ad700 1 --2- 172.21.15.125:0/2371921451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe14102730 0x7efe14104b20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efe04009990 tx=0x7efe04009ca0 comp rx=0 tx=0).stop 2024-02-17T11:50:35.862 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.862+0000 7efe1a0ad700 1 -- 172.21.15.125:0/2371921451 shutdown_connections 2024-02-17T11:50:35.862 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.862+0000 7efe1a0ad700 1 --2- 172.21.15.125:0/2371921451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe14107990 0x7efe14109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:35.862 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.862+0000 7efe1a0ad700 1 --2- 172.21.15.125:0/2371921451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe14102730 0x7efe14104b20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:35.862 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.862+0000 7efe1a0ad700 1 --2- 172.21.15.125:0/2371921451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe14105060 0x7efe14107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:35.862 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.862+0000 7efe1a0ad700 1 -- 172.21.15.125:0/2371921451 >> 172.21.15.125:0/2371921451 conn(0x7efe140fc160 msgr2=0x7efe140fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:50:35.862 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.862+0000 7efe1a0ad700 1 -- 172.21.15.125:0/2371921451 shutdown_connections 2024-02-17T11:50:35.862 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.862+0000 7efe1a0ad700 1 -- 172.21.15.125:0/2371921451 wait complete. 2024-02-17T11:50:35.862 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.862+0000 7efe1a0ad700 1 Processor -- start 2024-02-17T11:50:35.864 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe1a0ad700 1 -- start start 2024-02-17T11:50:35.864 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe1a0ad700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe14102730 0x7efe14197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:35.864 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe1a0ad700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe14105060 0x7efe141980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:35.864 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe1a0ad700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe14107990 0x7efe1419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:35.864 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe1a0ad700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1419f740 con 0x7efe14107990 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe1a0ad700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1419f880 con 0x7efe14105060 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe1a0ad700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1419fa10 con 0x7efe14102730 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe12ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe14105060 0x7efe141980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe12ffd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe14105060 0x7efe141980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:36490/0 (socket says 172.21.15.125:36490) 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe12ffd700 1 -- 172.21.15.125:0/1868495211 learned_addr learned my addr 172.21.15.125:0/1868495211 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe137fe700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe14102730 0x7efe14197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe13fff700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe14107990 0x7efe1419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe12ffd700 1 -- 172.21.15.125:0/1868495211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe14102730 msgr2=0x7efe14197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe12ffd700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe14102730 0x7efe14197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe12ffd700 1 -- 172.21.15.125:0/1868495211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe14107990 msgr2=0x7efe1419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe12ffd700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe14107990 0x7efe1419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe12ffd700 1 -- 172.21.15.125:0/1868495211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe040095f0 con 0x7efe14105060 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe13fff700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe14107990 0x7efe1419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:50:35.865 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe12ffd700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe14105060 0x7efe141980f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe0800d980 tx=0x7efe0800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:50:35.866 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.863+0000 7efe10ff9700 1 -- 172.21.15.125:0/1868495211 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08004220 con 0x7efe14105060 2024-02-17T11:50:35.866 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.864+0000 7efe10ff9700 1 -- 172.21.15.125:0/1868495211 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efe08007640 con 0x7efe14105060 2024-02-17T11:50:35.866 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.864+0000 7efe10ff9700 1 -- 172.21.15.125:0/1868495211 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe080056f0 con 0x7efe14105060 2024-02-17T11:50:35.866 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.864+0000 7efe1a0ad700 1 -- 172.21.15.125:0/1868495211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1419fcf0 con 0x7efe14105060 2024-02-17T11:50:35.866 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.864+0000 7efe1a0ad700 1 -- 172.21.15.125:0/1868495211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe141a01e0 con 0x7efe14105060 2024-02-17T11:50:35.866 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.864+0000 7efe1a0ad700 1 -- 172.21.15.125:0/1868495211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe14068930 con 0x7efe14105060 2024-02-17T11:50:35.867 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.866+0000 7efe10ff9700 1 -- 172.21.15.125:0/1868495211 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7efe0800b750 con 0x7efe14105060 2024-02-17T11:50:35.867 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.866+0000 7efe10ff9700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7efdfc0a2140 0x7efdfc0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:35.867 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.866+0000 7efe10ff9700 1 -- 172.21.15.125:0/1868495211 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7efe080cc440 con 0x7efe14105060 2024-02-17T11:50:35.867 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.867+0000 7efe137fe700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7efdfc0a2140 0x7efdfc0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:35.867 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.867+0000 7efe137fe700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7efdfc0a2140 0x7efdfc0a4600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efe04003790 tx=0x7efe04003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:50:35.872 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:35.870+0000 7efe10ff9700 1 -- 172.21.15.125:0/1868495211 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe08096e80 con 0x7efe14105060 2024-02-17T11:50:35.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:35.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: Detected new or changed devices on smithi094 2024-02-17T11:50:35.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:35.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:35.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:50:35.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi094 to 17088M 2024-02-17T11:50:35.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:35.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:35.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:35.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:35.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:35.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:35.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:35.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:36.068 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:36.068+0000 7efe1a0ad700 1 -- 172.21.15.125:0/1868495211 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7efe1402d960 con 0x7efdfc0a2140 2024-02-17T11:50:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-17T11:50:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: Detected new or changed devices on smithi094 2024-02-17T11:50:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:50:36.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi094 to 17088M 2024-02-17T11:50:36.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:36.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:36.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:36.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:36.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:36.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:36.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:36.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:36.919 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:36 smithi125 ceph-mon[36819]: osdmap e9: 1 total, 1 up, 1 in 2024-02-17T11:50:36.919 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:36 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:50:36.919 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:36 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:50:36.919 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:36 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:37.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:36 smithi189 ceph-mon[36588]: osdmap e9: 1 total, 1 up, 1 in 2024-02-17T11:50:37.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:36 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:50:37.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:36 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:50:37.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:36 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:37.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:36 smithi094 ceph-mon[32499]: osdmap e9: 1 total, 1 up, 1 in 2024-02-17T11:50:37.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:50:37.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:50:37.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:37.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:37 smithi125 ceph-mon[36819]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:50:37.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:37 smithi125 ceph-mon[36819]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:38.052 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:37 smithi189 ceph-mon[36588]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:50:38.052 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:37 smithi189 ceph-mon[36588]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:38.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:37 smithi094 ceph-mon[32499]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:50:38.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:37 smithi094 ceph-mon[32499]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:39.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:38 smithi189 ceph-mon[36588]: from='client.? 172.21.15.125:0/3624734446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2a8735-cd15-4041-9013-178ba85852c2"}]: dispatch 2024-02-17T11:50:39.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:38 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2a8735-cd15-4041-9013-178ba85852c2"}]: dispatch 2024-02-17T11:50:39.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:38 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf2a8735-cd15-4041-9013-178ba85852c2"}]': finished 2024-02-17T11:50:39.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:38 smithi189 ceph-mon[36588]: osdmap e10: 2 total, 1 up, 2 in 2024-02-17T11:50:39.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:38 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:39.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:38 smithi125 ceph-mon[36819]: from='client.? 172.21.15.125:0/3624734446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2a8735-cd15-4041-9013-178ba85852c2"}]: dispatch 2024-02-17T11:50:39.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:38 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2a8735-cd15-4041-9013-178ba85852c2"}]: dispatch 2024-02-17T11:50:39.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:38 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf2a8735-cd15-4041-9013-178ba85852c2"}]': finished 2024-02-17T11:50:39.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:38 smithi125 ceph-mon[36819]: osdmap e10: 2 total, 1 up, 2 in 2024-02-17T11:50:39.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:38 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:39.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:38 smithi094 ceph-mon[32499]: from='client.? 172.21.15.125:0/3624734446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2a8735-cd15-4041-9013-178ba85852c2"}]: dispatch 2024-02-17T11:50:39.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:38 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2a8735-cd15-4041-9013-178ba85852c2"}]: dispatch 2024-02-17T11:50:39.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:38 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf2a8735-cd15-4041-9013-178ba85852c2"}]': finished 2024-02-17T11:50:39.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:38 smithi094 ceph-mon[32499]: osdmap e10: 2 total, 1 up, 2 in 2024-02-17T11:50:39.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:38 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:40.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:39 smithi189 ceph-mon[36588]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:40.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:39 smithi189 ceph-mon[36588]: from='client.? 172.21.15.125:0/1532084069' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:50:40.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:39 smithi125 ceph-mon[36819]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:40.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:39 smithi125 ceph-mon[36819]: from='client.? 172.21.15.125:0/1532084069' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:50:40.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:39 smithi094 ceph-mon[32499]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:40.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:39 smithi094 ceph-mon[32499]: from='client.? 172.21.15.125:0/1532084069' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:50:42.136 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:41 smithi125 ceph-mon[36819]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:42.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:41 smithi189 ceph-mon[36588]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:42.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:41 smithi094 ceph-mon[32499]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:43.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:43 smithi125 ceph-mon[36819]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:44.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:43 smithi189 ceph-mon[36588]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:44.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:43 smithi094 ceph-mon[32499]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:46.091 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:45 smithi125 ceph-mon[36819]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:46.092 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-17T11:50:46.092 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:45 smithi189 ceph-mon[36588]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-17T11:50:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:45 smithi094 ceph-mon[32499]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-17T11:50:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:47.012 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:46 smithi125 ceph-mon[36819]: Deploying daemon osd.1 on smithi125 2024-02-17T11:50:47.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:46 smithi189 ceph-mon[36588]: Deploying daemon osd.1 on smithi125 2024-02-17T11:50:47.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:46 smithi094 ceph-mon[32499]: Deploying daemon osd.1 on smithi125 2024-02-17T11:50:47.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:47 smithi125 ceph-mon[36819]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:48.106 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:47 smithi094 ceph-mon[32499]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:48.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:47 smithi189 ceph-mon[36588]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:49.883 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:49 smithi125 ceph-mon[36819]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:50.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:49 smithi189 ceph-mon[36588]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:50.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:49 smithi094 ceph-mon[32499]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:52.002 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:51 smithi125 ceph-mon[36819]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:52.002 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:52.002 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:52.002 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:52.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:51 smithi189 ceph-mon[36588]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:52.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:51 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:52.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:51 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:52.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:51 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:52.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:51 smithi094 ceph-mon[32499]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:52.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:51 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:52.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:51 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:52.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:51 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:53.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:53 smithi125 ceph-mon[36819]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:53.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:53.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:53.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:53.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:53.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:54.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:53 smithi189 ceph-mon[36588]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:54.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:54.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:54.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:54.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:54.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:54.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:53 smithi094 ceph-mon[32499]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:54.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:54.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:54.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:54.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:54.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:54.279 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:54.278+0000 7efe10ff9700 1 -- 172.21.15.125:0/1868495211 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7efe1402d960 con 0x7efdfc0a2140 2024-02-17T11:50:54.279 INFO:teuthology.orchestra.run.smithi125.stdout:Created osd(s) 1 on host 'smithi125' 2024-02-17T11:50:54.280 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:54.281+0000 7efe1a0ad700 1 -- 172.21.15.125:0/1868495211 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7efdfc0a2140 msgr2=0x7efdfc0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:54.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:54.281+0000 7efe1a0ad700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7efdfc0a2140 0x7efdfc0a4600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efe04003790 tx=0x7efe04003860 comp rx=0 tx=0).stop 2024-02-17T11:50:54.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:54.281+0000 7efe1a0ad700 1 -- 172.21.15.125:0/1868495211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe14105060 msgr2=0x7efe141980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:54.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:54.281+0000 7efe1a0ad700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe14105060 0x7efe141980f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe0800d980 tx=0x7efe0800dc90 comp rx=0 tx=0).stop 2024-02-17T11:50:54.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:54.281+0000 7efe1a0ad700 1 -- 172.21.15.125:0/1868495211 shutdown_connections 2024-02-17T11:50:54.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:54.281+0000 7efe1a0ad700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7efdfc0a2140 0x7efdfc0a4600 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:54.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:54.281+0000 7efe1a0ad700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efe14107990 0x7efe1419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:54.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:54.281+0000 7efe1a0ad700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efe14102730 0x7efe14197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:54.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:54.281+0000 7efe1a0ad700 1 --2- 172.21.15.125:0/1868495211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efe14105060 0x7efe141980f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:54.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:54.281+0000 7efe1a0ad700 1 -- 172.21.15.125:0/1868495211 >> 172.21.15.125:0/1868495211 conn(0x7efe140fc160 msgr2=0x7efe14103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:50:54.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:54.281+0000 7efe1a0ad700 1 -- 172.21.15.125:0/1868495211 shutdown_connections 2024-02-17T11:50:54.281 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:50:54.281+0000 7efe1a0ad700 1 -- 172.21.15.125:0/1868495211 wait complete. 2024-02-17T11:50:54.886 DEBUG:teuthology.orchestra.run.smithi125:osd.1> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.1.service 2024-02-17T11:50:54.888 INFO:tasks.cephadm:Deploying osd.2 on smithi189 with /dev/vg_nvme/lv_4... 2024-02-17T11:50:54.888 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-17T11:50:55.134 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T11:50:55.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:55 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:55.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:55 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:55.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:55 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:55.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:55 smithi125 ceph-mon[36819]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:55.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:55 smithi125 ceph-mon[36819]: from='osd.1 [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-17T11:50:55.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:55 smithi125 ceph-mon[36819]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-17T11:50:55.496 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:55 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:55.496 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:55 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:55.496 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:55 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:55.496 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:55 smithi189 ceph-mon[36588]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:55.497 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:55 smithi189 ceph-mon[36588]: from='osd.1 [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-17T11:50:55.497 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:55 smithi189 ceph-mon[36588]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-17T11:50:55.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:55 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:55.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:55 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:50:55.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:55 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:55.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:55 smithi094 ceph-mon[32499]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:55.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:55 smithi094 ceph-mon[32499]: from='osd.1 [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-17T11:50:55.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:55 smithi094 ceph-mon[32499]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-17T11:50:56.375 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:56 smithi189 ceph-mon[36588]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-17T11:50:56.375 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:56 smithi189 ceph-mon[36588]: osdmap e11: 2 total, 1 up, 2 in 2024-02-17T11:50:56.375 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:56 smithi189 ceph-mon[36588]: from='osd.1 [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:50:56.375 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:56 smithi189 ceph-mon[36588]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:50:56.375 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:56.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:56 smithi125 ceph-mon[36819]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-17T11:50:56.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:56 smithi125 ceph-mon[36819]: osdmap e11: 2 total, 1 up, 2 in 2024-02-17T11:50:56.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:56 smithi125 ceph-mon[36819]: from='osd.1 [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:50:56.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:56 smithi125 ceph-mon[36819]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:50:56.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:56 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:56.685 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 11:50:56 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1[41431]: 2024-02-17T11:50:56.260+0000 7fa2c7599700 -1 osd.1 0 waiting for initial osdmap 2024-02-17T11:50:56.685 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 11:50:56 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1[41431]: 2024-02-17T11:50:56.263+0000 7fa2c338c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T11:50:56.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:56 smithi094 ceph-mon[32499]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-17T11:50:56.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:56 smithi094 ceph-mon[32499]: osdmap e11: 2 total, 1 up, 2 in 2024-02-17T11:50:56.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:56 smithi094 ceph-mon[32499]: from='osd.1 [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:50:56.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:56 smithi094 ceph-mon[32499]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:50:56.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:56.921 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-17T11:50:56.921 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-17T11:50:56.921 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-02-17T11:50:56.921 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-02-17T11:50:56.921 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125368 s, 836 MB/s 2024-02-17T11:50:56.922 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-02-17T11:50:57.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:57 smithi125 ceph-mon[36819]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2024-02-17T11:50:57.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:57 smithi125 ceph-mon[36819]: osdmap e12: 2 total, 1 up, 2 in 2024-02-17T11:50:57.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:57.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:57.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:57 smithi125 ceph-mon[36819]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:57.480 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi189:vg_nvme/lv_4 2024-02-17T11:50:57.506 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:57 smithi189 ceph-mon[36588]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2024-02-17T11:50:57.506 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:57 smithi189 ceph-mon[36588]: osdmap e12: 2 total, 1 up, 2 in 2024-02-17T11:50:57.506 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:57.506 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:57.506 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:57 smithi189 ceph-mon[36588]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:57.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:57 smithi094 ceph-mon[32499]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2024-02-17T11:50:57.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:57 smithi094 ceph-mon[32499]: osdmap e12: 2 total, 1 up, 2 in 2024-02-17T11:50:57.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:57.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:57.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:57 smithi094 ceph-mon[32499]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-02-17T11:50:57.728 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T11:50:58.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:58 smithi189 ceph-mon[36588]: purged_snaps scrub starts 2024-02-17T11:50:58.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:58 smithi189 ceph-mon[36588]: purged_snaps scrub ok 2024-02-17T11:50:58.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:58 smithi189 ceph-mon[36588]: osd.1 [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339] boot 2024-02-17T11:50:58.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:58 smithi189 ceph-mon[36588]: osdmap e13: 2 total, 2 up, 2 in 2024-02-17T11:50:58.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:58.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:58.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:58.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:50:58.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:58.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:58.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:58.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:58.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:58 smithi125 ceph-mon[36819]: purged_snaps scrub starts 2024-02-17T11:50:58.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:58 smithi125 ceph-mon[36819]: purged_snaps scrub ok 2024-02-17T11:50:58.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:58 smithi125 ceph-mon[36819]: osd.1 [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339] boot 2024-02-17T11:50:58.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:58 smithi125 ceph-mon[36819]: osdmap e13: 2 total, 2 up, 2 in 2024-02-17T11:50:58.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:58.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:58.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:58.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:50:58.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:58.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:58.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:58.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:58.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:58 smithi094 ceph-mon[32499]: purged_snaps scrub starts 2024-02-17T11:50:58.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:58 smithi094 ceph-mon[32499]: purged_snaps scrub ok 2024-02-17T11:50:58.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:58 smithi094 ceph-mon[32499]: osd.1 [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339] boot 2024-02-17T11:50:58.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:58 smithi094 ceph-mon[32499]: osdmap e13: 2 total, 2 up, 2 in 2024-02-17T11:50:58.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:50:58.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:58.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:58.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:50:58.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:58.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:50:58.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:50:58.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:50:59.327 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.325+0000 7f24a93da700 1 -- 172.21.15.189:0/3550739255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a4102740 msgr2=0x7f24a4108ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:59.328 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.325+0000 7f24a93da700 1 --2- 172.21.15.189:0/3550739255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a4102740 0x7f24a4108ff0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f249c00b0a0 tx=0x7f249c00b3b0 comp rx=0 tx=0).stop 2024-02-17T11:50:59.328 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.325+0000 7f24a93da700 1 -- 172.21.15.189:0/3550739255 shutdown_connections 2024-02-17T11:50:59.328 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.325+0000 7f24a93da700 1 --2- 172.21.15.189:0/3550739255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a4102740 0x7f24a4108ff0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:59.328 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.325+0000 7f24a93da700 1 --2- 172.21.15.189:0/3550739255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a4100ba0 0x7f24a4100fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:59.328 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.325+0000 7f24a93da700 1 --2- 172.21.15.189:0/3550739255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24a4101da0 0x7f24a4102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:59.328 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.325+0000 7f24a93da700 1 -- 172.21.15.189:0/3550739255 >> 172.21.15.189:0/3550739255 conn(0x7f24a40fc160 msgr2=0x7f24a40fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:50:59.328 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.325+0000 7f24a93da700 1 -- 172.21.15.189:0/3550739255 shutdown_connections 2024-02-17T11:50:59.328 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.325+0000 7f24a93da700 1 -- 172.21.15.189:0/3550739255 wait complete. 2024-02-17T11:50:59.328 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a93da700 1 Processor -- start 2024-02-17T11:50:59.329 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a93da700 1 -- start start 2024-02-17T11:50:59.329 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a93da700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a4100ba0 0x7f24a410b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:59.329 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a93da700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24a4101da0 0x7f24a4109b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:59.329 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a93da700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a4102740 0x7f24a410a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:59.329 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a93da700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a410a7c0 con 0x7f24a4102740 2024-02-17T11:50:59.329 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a93da700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a410a930 con 0x7f24a4101da0 2024-02-17T11:50:59.329 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a93da700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a41a1d10 con 0x7f24a4100ba0 2024-02-17T11:50:59.329 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a2ffd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a4100ba0 0x7f24a410b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:59.329 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a2ffd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a4100ba0 0x7f24a410b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:53868/0 (socket says 172.21.15.189:53868) 2024-02-17T11:50:59.330 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a2ffd700 1 -- 172.21.15.189:0/2980817983 learned_addr learned my addr 172.21.15.189:0/2980817983 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-02-17T11:50:59.330 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a37fe700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a4102740 0x7f24a410a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:59.330 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a2ffd700 1 -- 172.21.15.189:0/2980817983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24a4101da0 msgr2=0x7f24a4109b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:59.330 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a27fc700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24a4101da0 0x7f24a4109b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:59.330 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a2ffd700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24a4101da0 0x7f24a4109b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:59.330 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a2ffd700 1 -- 172.21.15.189:0/2980817983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a4102740 msgr2=0x7f24a410a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:50:59.330 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.326+0000 7f24a2ffd700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a4102740 0x7f24a410a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:50:59.330 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.327+0000 7f24a2ffd700 1 -- 172.21.15.189:0/2980817983 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249c009d00 con 0x7f24a4100ba0 2024-02-17T11:50:59.330 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.327+0000 7f24a37fe700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a4102740 0x7f24a410a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:50:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.327+0000 7f24a2ffd700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a4100ba0 0x7f24a410b4b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f249400c970 tx=0x7f249400cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:50:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.327+0000 7f248bfff700 1 -- 172.21.15.189:0/2980817983 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494004220 con 0x7f24a4100ba0 2024-02-17T11:50:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.327+0000 7f248bfff700 1 -- 172.21.15.189:0/2980817983 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2494007d80 con 0x7f24a4100ba0 2024-02-17T11:50:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.327+0000 7f24a93da700 1 -- 172.21.15.189:0/2980817983 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a41a1ff0 con 0x7f24a4100ba0 2024-02-17T11:50:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.327+0000 7f248bfff700 1 -- 172.21.15.189:0/2980817983 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2494003da0 con 0x7f24a4100ba0 2024-02-17T11:50:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.327+0000 7f24a93da700 1 -- 172.21.15.189:0/2980817983 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a41a2400 con 0x7f24a4100ba0 2024-02-17T11:50:59.331 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.328+0000 7f24a93da700 1 -- 172.21.15.189:0/2980817983 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a4107e90 con 0x7f24a4100ba0 2024-02-17T11:50:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.329+0000 7f248bfff700 1 -- 172.21.15.189:0/2980817983 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7f24940078a0 con 0x7f24a4100ba0 2024-02-17T11:50:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.329+0000 7f248bfff700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f248c0a20c0 0x7f248c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:50:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.330+0000 7f248bfff700 1 -- 172.21.15.189:0/2980817983 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f24940cc060 con 0x7f24a4100ba0 2024-02-17T11:50:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.330+0000 7f24a27fc700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f248c0a20c0 0x7f248c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:50:59.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.330+0000 7f24a27fc700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f248c0a20c0 0x7f248c0a4580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f249800ba60 tx=0x7f2498005d50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:50:59.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.334+0000 7f248bfff700 1 -- 172.21.15.189:0/2980817983 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2494096880 con 0x7f24a4100ba0 2024-02-17T11:50:59.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:59 smithi189 ceph-mon[36588]: Detected new or changed devices on smithi125 2024-02-17T11:50:59.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:59 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi125 to 17080M 2024-02-17T11:50:59.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:59 smithi189 ceph-mon[36588]: osdmap e14: 2 total, 2 up, 2 in 2024-02-17T11:50:59.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:50:59 smithi189 ceph-mon[36588]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:50:59.539 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:50:59.539+0000 7f24a93da700 1 -- 172.21.15.189:0/2980817983 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f24a402d950 con 0x7f248c0a20c0 2024-02-17T11:50:59.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:59 smithi125 ceph-mon[36819]: Detected new or changed devices on smithi125 2024-02-17T11:50:59.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:59 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi125 to 17080M 2024-02-17T11:50:59.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:59 smithi125 ceph-mon[36819]: osdmap e14: 2 total, 2 up, 2 in 2024-02-17T11:50:59.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:50:59 smithi125 ceph-mon[36819]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:50:59.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:59 smithi094 ceph-mon[32499]: Detected new or changed devices on smithi125 2024-02-17T11:50:59.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:59 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi125 to 17080M 2024-02-17T11:50:59.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:59 smithi094 ceph-mon[32499]: osdmap e14: 2 total, 2 up, 2 in 2024-02-17T11:50:59.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:50:59 smithi094 ceph-mon[32499]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:00.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:00 smithi189 ceph-mon[36588]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:51:00.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:00 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:51:00.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:00 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:51:00.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:00 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:00.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:00 smithi125 ceph-mon[36819]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:51:00.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:00 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:51:00.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:00 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:51:00.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:00 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:00.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:00 smithi094 ceph-mon[32499]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:51:00.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:00 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:51:00.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:00 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:51:00.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:00 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:01.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:01 smithi189 ceph-mon[36588]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:01.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:01 smithi125 ceph-mon[36819]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:01.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:01 smithi094 ceph-mon[32499]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:02 smithi189 ceph-mon[36588]: from='client.? 172.21.15.189:0/1089900006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65b51d55-af5c-411c-988a-05331aad4a52"}]: dispatch 2024-02-17T11:51:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:02 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65b51d55-af5c-411c-988a-05331aad4a52"}]: dispatch 2024-02-17T11:51:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:02 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65b51d55-af5c-411c-988a-05331aad4a52"}]': finished 2024-02-17T11:51:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:02 smithi189 ceph-mon[36588]: osdmap e15: 3 total, 2 up, 3 in 2024-02-17T11:51:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:02.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:02 smithi125 ceph-mon[36819]: from='client.? 172.21.15.189:0/1089900006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65b51d55-af5c-411c-988a-05331aad4a52"}]: dispatch 2024-02-17T11:51:02.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:02 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65b51d55-af5c-411c-988a-05331aad4a52"}]: dispatch 2024-02-17T11:51:02.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:02 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65b51d55-af5c-411c-988a-05331aad4a52"}]': finished 2024-02-17T11:51:02.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:02 smithi125 ceph-mon[36819]: osdmap e15: 3 total, 2 up, 3 in 2024-02-17T11:51:02.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:02.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:02 smithi094 ceph-mon[32499]: from='client.? 172.21.15.189:0/1089900006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65b51d55-af5c-411c-988a-05331aad4a52"}]: dispatch 2024-02-17T11:51:02.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:02 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65b51d55-af5c-411c-988a-05331aad4a52"}]: dispatch 2024-02-17T11:51:02.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:02 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65b51d55-af5c-411c-988a-05331aad4a52"}]': finished 2024-02-17T11:51:02.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:02 smithi094 ceph-mon[32499]: osdmap e15: 3 total, 2 up, 3 in 2024-02-17T11:51:02.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:03.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:03 smithi189 ceph-mon[36588]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:03.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:03 smithi189 ceph-mon[36588]: from='client.? 172.21.15.189:0/1761181020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:51:03.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:03 smithi125 ceph-mon[36819]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:03.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:03 smithi125 ceph-mon[36819]: from='client.? 172.21.15.189:0/1761181020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:51:03.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:03 smithi094 ceph-mon[32499]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:03.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:03 smithi094 ceph-mon[32499]: from='client.? 172.21.15.189:0/1761181020' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:51:05.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:05 smithi189 ceph-mon[36588]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:05.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:05 smithi125 ceph-mon[36819]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:05.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:05 smithi094 ceph-mon[32499]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:07.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:07 smithi189 ceph-mon[36588]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:07.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:07 smithi125 ceph-mon[36819]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:07 smithi094 ceph-mon[32499]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:09.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:09 smithi189 ceph-mon[36588]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:09.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:09 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-17T11:51:09.824 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:09 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:09.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:09 smithi125 ceph-mon[36819]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:09.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:09 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-17T11:51:09.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:09 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:09.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:09 smithi094 ceph-mon[32499]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:09.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:09 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-17T11:51:09.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:09 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:10.872 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:10 smithi189 ceph-mon[36588]: Deploying daemon osd.2 on smithi189 2024-02-17T11:51:10.929 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:10 smithi125 ceph-mon[36819]: Deploying daemon osd.2 on smithi189 2024-02-17T11:51:10.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:10 smithi094 ceph-mon[32499]: Deploying daemon osd.2 on smithi189 2024-02-17T11:51:11.895 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:11 smithi189 ceph-mon[36588]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:11.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:11 smithi125 ceph-mon[36819]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:11.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:11 smithi094 ceph-mon[32499]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:13.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:13 smithi189 ceph-mon[36588]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:13 smithi125 ceph-mon[36819]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:13 smithi094 ceph-mon[32499]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:15.343 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:14 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:15.343 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:14 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:15.343 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:14 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:51:15.344 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:14 smithi189 ceph-mon[36588]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:15.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:14 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:15.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:14 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:15.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:14 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:51:15.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:14 smithi125 ceph-mon[36819]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:15.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:14 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:15.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:14 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:15.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:14 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:51:15.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:14 smithi094 ceph-mon[32499]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:17.528 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:17 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:17.528 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:17 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:17.528 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:17 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:17.528 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:17 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:51:17.528 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:17 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:17.528 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:17 smithi189 ceph-mon[36588]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:17.607 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:51:17.607+0000 7f248bfff700 1 -- 172.21.15.189:0/2980817983 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f24a402d950 con 0x7f248c0a20c0 2024-02-17T11:51:17.608 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 2 on host 'smithi189' 2024-02-17T11:51:17.609 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:51:17.610+0000 7f24a93da700 1 -- 172.21.15.189:0/2980817983 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f248c0a20c0 msgr2=0x7f248c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:51:17.609 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:51:17.610+0000 7f24a93da700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f248c0a20c0 0x7f248c0a4580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f249800ba60 tx=0x7f2498005d50 comp rx=0 tx=0).stop 2024-02-17T11:51:17.609 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:51:17.610+0000 7f24a93da700 1 -- 172.21.15.189:0/2980817983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a4100ba0 msgr2=0x7f24a410b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:51:17.609 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:51:17.610+0000 7f24a93da700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a4100ba0 0x7f24a410b4b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f249400c970 tx=0x7f249400cd30 comp rx=0 tx=0).stop 2024-02-17T11:51:17.609 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:51:17.610+0000 7f24a93da700 1 -- 172.21.15.189:0/2980817983 shutdown_connections 2024-02-17T11:51:17.609 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:51:17.610+0000 7f24a93da700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f248c0a20c0 0x7f248c0a4580 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:17.609 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:51:17.610+0000 7f24a93da700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24a4102740 0x7f24a410a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:17.609 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:51:17.610+0000 7f24a93da700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24a4100ba0 0x7f24a410b4b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:17.610 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:51:17.610+0000 7f24a93da700 1 --2- 172.21.15.189:0/2980817983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24a4101da0 0x7f24a4109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:17.610 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:51:17.610+0000 7f24a93da700 1 -- 172.21.15.189:0/2980817983 >> 172.21.15.189:0/2980817983 conn(0x7f24a40fc160 msgr2=0x7f24a4104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:51:17.610 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:51:17.610+0000 7f24a93da700 1 -- 172.21.15.189:0/2980817983 shutdown_connections 2024-02-17T11:51:17.610 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:51:17.610+0000 7f24a93da700 1 -- 172.21.15.189:0/2980817983 wait complete. 2024-02-17T11:51:17.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:17 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:17.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:17 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:17.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:17 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:17.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:17 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:51:17.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:17 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:17.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:17 smithi125 ceph-mon[36819]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:17.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:17 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:17.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:17 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:17.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:17 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:17.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:17 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:51:17.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:17 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:17.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:17 smithi094 ceph-mon[32499]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:18.273 DEBUG:teuthology.orchestra.run.smithi189:osd.2> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.2.service 2024-02-17T11:51:18.315 INFO:tasks.cephadm:Deploying osd.3 on smithi094 with /dev/vg_nvme/lv_3... 2024-02-17T11:51:18.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-17T11:51:18.562 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:51:18.606 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:18.606 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:18.607 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:51:18.607 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:18 smithi094 ceph-mon[32499]: from='osd.2 [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-17T11:51:18.607 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:18 smithi094 ceph-mon[32499]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-17T11:51:18.825 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:18.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:18.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:51:18.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:18 smithi189 ceph-mon[36588]: from='osd.2 [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-17T11:51:18.826 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:18 smithi189 ceph-mon[36588]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-17T11:51:18.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:18.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:18.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:51:18.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:18 smithi125 ceph-mon[36819]: from='osd.2 [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-17T11:51:18.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:18 smithi125 ceph-mon[36819]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-17T11:51:19.757 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:19 smithi189 ceph-mon[36588]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-17T11:51:19.757 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:19 smithi189 ceph-mon[36588]: osdmap e16: 3 total, 2 up, 3 in 2024-02-17T11:51:19.758 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:19 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:19.758 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:19 smithi189 ceph-mon[36588]: from='osd.2 [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:51:19.758 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:19 smithi189 ceph-mon[36588]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:51:19.758 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:19 smithi189 ceph-mon[36588]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:19.758 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 11:51:19 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2[41765]: 2024-02-17T11:51:19.578+0000 7fd52ecde700 -1 osd.2 0 waiting for initial osdmap 2024-02-17T11:51:19.758 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 11:51:19 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2[41765]: 2024-02-17T11:51:19.582+0000 7fd52aad1700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T11:51:19.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:19 smithi125 ceph-mon[36819]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-17T11:51:19.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:19 smithi125 ceph-mon[36819]: osdmap e16: 3 total, 2 up, 3 in 2024-02-17T11:51:19.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:19 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:19.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:19 smithi125 ceph-mon[36819]: from='osd.2 [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:51:19.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:19 smithi125 ceph-mon[36819]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:51:19.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:19 smithi125 ceph-mon[36819]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:19.945 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:19 smithi094 ceph-mon[32499]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-17T11:51:19.945 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:19 smithi094 ceph-mon[32499]: osdmap e16: 3 total, 2 up, 3 in 2024-02-17T11:51:19.945 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:19 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:19.945 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:19 smithi094 ceph-mon[32499]: from='osd.2 [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:51:19.946 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:19 smithi094 ceph-mon[32499]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:51:19.946 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:19 smithi094 ceph-mon[32499]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:20.548 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-17T11:51:20.548 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-17T11:51:20.548 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-02-17T11:51:20.548 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-02-17T11:51:20.548 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00851749 s, 1.2 GB/s 2024-02-17T11:51:20.548 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-02-17T11:51:20.893 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:20 smithi094 ceph-mon[32499]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-17T11:51:20.893 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:20 smithi094 ceph-mon[32499]: osdmap e17: 3 total, 2 up, 3 in 2024-02-17T11:51:20.893 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:20 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:20.893 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:20 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:20.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:20 smithi189 ceph-mon[36588]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-17T11:51:20.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:20 smithi189 ceph-mon[36588]: osdmap e17: 3 total, 2 up, 3 in 2024-02-17T11:51:20.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:20 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:20.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:20 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:20.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:20 smithi125 ceph-mon[36819]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-17T11:51:20.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:20 smithi125 ceph-mon[36819]: osdmap e17: 3 total, 2 up, 3 in 2024-02-17T11:51:20.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:20 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:20.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:20 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:21.246 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi094:vg_nvme/lv_3 2024-02-17T11:51:21.496 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:51:21.856 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: purged_snaps scrub starts 2024-02-17T11:51:21.856 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: purged_snaps scrub ok 2024-02-17T11:51:21.856 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: osd.2 [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236] boot 2024-02-17T11:51:21.856 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: osdmap e18: 3 total, 3 up, 3 in 2024-02-17T11:51:21.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:21.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:21.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-17T11:51:21.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:21.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:21.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:51:21.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:21.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:21.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:51:21.857 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:21.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: purged_snaps scrub starts 2024-02-17T11:51:21.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: purged_snaps scrub ok 2024-02-17T11:51:21.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: osd.2 [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236] boot 2024-02-17T11:51:21.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: osdmap e18: 3 total, 3 up, 3 in 2024-02-17T11:51:21.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:21.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:21.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-17T11:51:21.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:21.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:21.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:51:21.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:21.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:21.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:51:21.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:21.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: purged_snaps scrub starts 2024-02-17T11:51:21.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: purged_snaps scrub ok 2024-02-17T11:51:21.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: osd.2 [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236] boot 2024-02-17T11:51:21.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: osdmap e18: 3 total, 3 up, 3 in 2024-02-17T11:51:21.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:51:21.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-02-17T11:51:21.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-17T11:51:21.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:21.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:21.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:51:21.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:21.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:21.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:51:21.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:22.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:22 smithi189 ceph-mon[36588]: Detected new or changed devices on smithi189 2024-02-17T11:51:22.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:22 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi189 to 17080M 2024-02-17T11:51:22.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:22 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-17T11:51:22.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:22 smithi189 ceph-mon[36588]: osdmap e19: 3 total, 3 up, 3 in 2024-02-17T11:51:22.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:22 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-17T11:51:22.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:22 smithi094 ceph-mon[32499]: Detected new or changed devices on smithi189 2024-02-17T11:51:22.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:22 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi189 to 17080M 2024-02-17T11:51:22.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:22 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-17T11:51:22.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:22 smithi094 ceph-mon[32499]: osdmap e19: 3 total, 3 up, 3 in 2024-02-17T11:51:22.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:22 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-17T11:51:22.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:22 smithi125 ceph-mon[36819]: Detected new or changed devices on smithi189 2024-02-17T11:51:22.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:22 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi189 to 17080M 2024-02-17T11:51:22.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:22 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-17T11:51:22.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:22 smithi125 ceph-mon[36819]: osdmap e19: 3 total, 3 up, 3 in 2024-02-17T11:51:22.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:22 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-17T11:51:23.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.162+0000 7ff2a5684700 1 -- 172.21.15.94:0/641202977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2a0101cd0 msgr2=0x7ff2a01020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:51:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.162+0000 7ff2a5684700 1 --2- 172.21.15.94:0/641202977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2a0101cd0 0x7ff2a01020f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff2940099c0 tx=0x7ff294009cd0 comp rx=0 tx=0).stop 2024-02-17T11:51:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.162+0000 7ff2a5684700 1 -- 172.21.15.94:0/641202977 shutdown_connections 2024-02-17T11:51:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.162+0000 7ff2a5684700 1 --2- 172.21.15.94:0/641202977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2a0103870 0x7ff2a010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.162+0000 7ff2a5684700 1 --2- 172.21.15.94:0/641202977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2a0101cd0 0x7ff2a01020f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.162+0000 7ff2a5684700 1 --2- 172.21.15.94:0/641202977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a0102ed0 0x7ff2a0103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.162+0000 7ff2a5684700 1 -- 172.21.15.94:0/641202977 >> 172.21.15.94:0/641202977 conn(0x7ff2a00fd250 msgr2=0x7ff2a00ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:51:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.162+0000 7ff2a5684700 1 -- 172.21.15.94:0/641202977 shutdown_connections 2024-02-17T11:51:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.162+0000 7ff2a5684700 1 -- 172.21.15.94:0/641202977 wait complete. 2024-02-17T11:51:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.162+0000 7ff2a5684700 1 Processor -- start 2024-02-17T11:51:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.162+0000 7ff2a5684700 1 -- start start 2024-02-17T11:51:23.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff2a5684700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2a0101cd0 0x7ff2a0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff2a5684700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a0102ed0 0x7ff2a0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff2a5684700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2a0103870 0x7ff2a019f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff2a5684700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a019f690 con 0x7ff2a0103870 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff2a5684700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a019f7d0 con 0x7ff2a0101cd0 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff2a5684700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a019f960 con 0x7ff2a0102ed0 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff29f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2a0103870 0x7ff2a019f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff29f7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2a0103870 0x7ff2a019f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34246/0 (socket says 172.21.15.94:34246) 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff29f7fe700 1 -- 172.21.15.94:0/1062719974 learned_addr learned my addr 172.21.15.94:0/1062719974 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff29e7fc700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a0102ed0 0x7ff2a0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff29effd700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2a0101cd0 0x7ff2a0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff29f7fe700 1 -- 172.21.15.94:0/1062719974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a0102ed0 msgr2=0x7ff2a0198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff29f7fe700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a0102ed0 0x7ff2a0198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff29f7fe700 1 -- 172.21.15.94:0/1062719974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2a0101cd0 msgr2=0x7ff2a0197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:51:23.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff29f7fe700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2a0101cd0 0x7ff2a0197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff29f7fe700 1 -- 172.21.15.94:0/1062719974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2940095f0 con 0x7ff2a0103870 2024-02-17T11:51:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff29f7fe700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2a0103870 0x7ff2a019f0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff298007da0 tx=0x7ff29800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:51:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff287fff700 1 -- 172.21.15.94:0/1062719974 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29800ac50 con 0x7ff2a0103870 2024-02-17T11:51:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff287fff700 1 -- 172.21.15.94:0/1062719974 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff29800adb0 con 0x7ff2a0103870 2024-02-17T11:51:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff2a5684700 1 -- 172.21.15.94:0/1062719974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a019fc40 con 0x7ff2a0103870 2024-02-17T11:51:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff287fff700 1 -- 172.21.15.94:0/1062719974 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298011680 con 0x7ff2a0103870 2024-02-17T11:51:23.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.163+0000 7ff2a5684700 1 -- 172.21.15.94:0/1062719974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a01a0140 con 0x7ff2a0103870 2024-02-17T11:51:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.164+0000 7ff285ffb700 1 -- 172.21.15.94:0/1062719974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff280005330 con 0x7ff2a0103870 2024-02-17T11:51:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.166+0000 7ff287fff700 1 -- 172.21.15.94:0/1062719974 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 15) v1 ==== 136712+0+0 (secure 0 0 0) 0x7ff298004ad0 con 0x7ff2a0103870 2024-02-17T11:51:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.167+0000 7ff287fff700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff2880a2140 0x7ff2880a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:51:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.167+0000 7ff29effd700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff2880a2140 0x7ff2880a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:51:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.167+0000 7ff287fff700 1 -- 172.21.15.94:0/1062719974 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7ff2980cce20 con 0x7ff2a0103870 2024-02-17T11:51:23.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.167+0000 7ff29effd700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff2880a2140 0x7ff2880a4600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff294006010 tx=0x7ff294000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:51:23.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.170+0000 7ff287fff700 1 -- 172.21.15.94:0/1062719974 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff2980972a0 con 0x7ff2a0103870 2024-02-17T11:51:23.205 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 11:51:22 smithi094 sudo[52366]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-17T11:51:23.205 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 11:51:22 smithi094 sudo[52366]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T11:51:23.205 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 11:51:22 smithi094 sudo[52366]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T11:51:23.205 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 11:51:22 smithi094 sudo[52366]: pam_unix(sudo:session): session closed for user root 2024-02-17T11:51:23.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:23.379+0000 7ff285ffb700 1 -- 172.21.15.94:0/1062719974 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff280000bc0 con 0x7ff2880a2140 2024-02-17T11:51:23.594 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 11:51:23 smithi094 sudo[52392]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-17T11:51:23.594 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 11:51:23 smithi094 sudo[52392]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T11:51:23.595 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 11:51:23 smithi094 sudo[52392]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T11:51:23.595 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 11:51:23 smithi094 sudo[52392]: pam_unix(sudo:session): session closed for user root 2024-02-17T11:51:23.733 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 11:51:23 smithi125 sudo[45274]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-17T11:51:23.733 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 11:51:23 smithi125 sudo[45274]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T11:51:23.733 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 11:51:23 smithi125 sudo[45274]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T11:51:23.733 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 11:51:23 smithi125 sudo[45274]: pam_unix(sudo:session): session closed for user root 2024-02-17T11:51:23.733 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:23 smithi125 ceph-mon[36819]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:23.733 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:23 smithi125 ceph-mon[36819]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-17T11:51:23.733 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:23 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-17T11:51:23.733 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:23 smithi125 ceph-mon[36819]: osdmap e20: 3 total, 3 up, 3 in 2024-02-17T11:51:23.734 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:23 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:23.734 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:23 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:51:23.734 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:23 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:51:23.734 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:23 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:23.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:23 smithi189 ceph-mon[36588]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:23.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:23 smithi189 ceph-mon[36588]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-17T11:51:23.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:23 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-17T11:51:23.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:23 smithi189 ceph-mon[36588]: osdmap e20: 3 total, 3 up, 3 in 2024-02-17T11:51:23.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:23 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:23.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:23 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:51:23.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:23 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:51:23.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:23 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:23.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:23 smithi094 ceph-mon[32499]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:23.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:23 smithi094 ceph-mon[32499]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-17T11:51:23.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:23 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-17T11:51:23.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:23 smithi094 ceph-mon[32499]: osdmap e20: 3 total, 3 up, 3 in 2024-02-17T11:51:23.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:23 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:23.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:23 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:51:23.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:23 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:51:23.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:23 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:24.184 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 11:51:23 smithi125 sudo[45277]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-17T11:51:24.184 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 11:51:23 smithi125 sudo[45277]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T11:51:24.184 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 11:51:23 smithi125 sudo[45277]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T11:51:24.184 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 11:51:23 smithi125 sudo[45277]: pam_unix(sudo:session): session closed for user root 2024-02-17T11:51:24.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:24.249+0000 7ff287fff700 1 -- 172.21.15.94:0/1062719974 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff298097670 con 0x7ff2a0103870 2024-02-17T11:51:24.251 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 11:51:24 smithi189 sudo[45808]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-17T11:51:24.251 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 11:51:24 smithi189 sudo[45808]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T11:51:24.251 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 11:51:24 smithi189 sudo[45808]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T11:51:24.251 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 11:51:24 smithi189 sudo[45808]: pam_unix(sudo:session): session closed for user root 2024-02-17T11:51:24.651 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 11:51:24 smithi189 sudo[45811]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-17T11:51:24.651 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 11:51:24 smithi189 sudo[45811]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T11:51:24.651 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 11:51:24 smithi189 sudo[45811]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T11:51:24.651 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 11:51:24 smithi189 sudo[45811]: pam_unix(sudo:session): session closed for user root 2024-02-17T11:51:24.776 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 sudo[52536]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-17T11:51:24.776 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 sudo[52536]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T11:51:24.776 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 sudo[52536]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T11:51:24.777 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 sudo[52536]: pam_unix(sudo:session): session closed for user root 2024-02-17T11:51:25.035 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:24 smithi189 ceph-mon[36588]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:51:25.035 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:24 smithi189 ceph-mon[36588]: osdmap e21: 3 total, 3 up, 3 in 2024-02-17T11:51:25.035 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:24 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:25.035 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:24 smithi189 ceph-mon[36588]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-17T11:51:25.035 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:24 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:25.035 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:24 smithi189 ceph-mon[36588]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T11:51:25.035 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 sudo[45814]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-17T11:51:25.036 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 sudo[45814]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T11:51:25.036 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 sudo[45814]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T11:51:25.036 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 sudo[45814]: pam_unix(sudo:session): session closed for user root 2024-02-17T11:51:25.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:24 smithi125 ceph-mon[36819]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:51:25.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:24 smithi125 ceph-mon[36819]: osdmap e21: 3 total, 3 up, 3 in 2024-02-17T11:51:25.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:24 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:25.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:24 smithi125 ceph-mon[36819]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-17T11:51:25.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:24 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:25.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:24 smithi125 ceph-mon[36819]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T11:51:25.196 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 sudo[52543]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-17T11:51:25.197 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 sudo[52543]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T11:51:25.197 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 sudo[52543]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T11:51:25.197 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 sudo[52543]: pam_unix(sudo:session): session closed for user root 2024-02-17T11:51:25.197 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 ceph-mon[32499]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:51:25.197 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 ceph-mon[32499]: osdmap e21: 3 total, 3 up, 3 in 2024-02-17T11:51:25.197 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:25.197 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 ceph-mon[32499]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-17T11:51:25.197 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:25.197 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:24 smithi094 ceph-mon[32499]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T11:51:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 sudo[45817]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-17T11:51:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 sudo[45817]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T11:51:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 sudo[45817]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T11:51:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 sudo[45817]: pam_unix(sudo:session): session closed for user root 2024-02-17T11:51:25.766 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 sudo[45280]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-17T11:51:25.767 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 sudo[45280]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T11:51:25.767 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 sudo[45280]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T11:51:25.767 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 sudo[45280]: pam_unix(sudo:session): session closed for user root 2024-02-17T11:51:26.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:26.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T11:51:26.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:51:26.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:51:26.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:51:26.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T11:51:26.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:51:26.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:51:26.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:51:26.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-17T11:51:26.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: Cluster is now healthy 2024-02-17T11:51:26.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T11:51:26.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T11:51:26.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:51:26.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:51:26.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:51:26.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:25 smithi189 ceph-mon[36588]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T11:51:26.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 sudo[45283]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-17T11:51:26.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 sudo[45283]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-17T11:51:26.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 sudo[45283]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 sudo[45283]: pam_unix(sudo:session): session closed for user root 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: Cluster is now healthy 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T11:51:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:51:26.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:51:26.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:51:26.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:25 smithi125 ceph-mon[36819]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T11:51:26.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:26.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T11:51:26.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:51:26.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:51:26.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:51:26.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T11:51:26.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:51:26.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:51:26.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:51:26.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-17T11:51:26.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: Cluster is now healthy 2024-02-17T11:51:26.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T11:51:26.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-17T11:51:26.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:51:26.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:51:26.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:51:26.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:25 smithi094 ceph-mon[32499]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-17T11:51:27.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:26 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3375122417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36e54a46-a0de-4a53-8bb4-6a49a457c91e"}]: dispatch 2024-02-17T11:51:27.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:26 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3375122417' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36e54a46-a0de-4a53-8bb4-6a49a457c91e"}]': finished 2024-02-17T11:51:27.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:26 smithi189 ceph-mon[36588]: osdmap e22: 4 total, 3 up, 4 in 2024-02-17T11:51:27.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:26 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:27.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:26 smithi189 ceph-mon[36588]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:27.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:26 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3324877328' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:51:27.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:26 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3375122417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36e54a46-a0de-4a53-8bb4-6a49a457c91e"}]: dispatch 2024-02-17T11:51:27.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:26 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3375122417' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36e54a46-a0de-4a53-8bb4-6a49a457c91e"}]': finished 2024-02-17T11:51:27.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:26 smithi125 ceph-mon[36819]: osdmap e22: 4 total, 3 up, 4 in 2024-02-17T11:51:27.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:26 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:27.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:26 smithi125 ceph-mon[36819]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:27.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:26 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3324877328' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:51:27.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:26 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3375122417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36e54a46-a0de-4a53-8bb4-6a49a457c91e"}]: dispatch 2024-02-17T11:51:27.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:26 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3375122417' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36e54a46-a0de-4a53-8bb4-6a49a457c91e"}]': finished 2024-02-17T11:51:27.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:26 smithi094 ceph-mon[32499]: osdmap e22: 4 total, 3 up, 4 in 2024-02-17T11:51:27.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:26 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:27.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:26 smithi094 ceph-mon[32499]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:27.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:26 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3324877328' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:51:29.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:29 smithi189 ceph-mon[36588]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 262 KiB/s wr, 18 op/s 2024-02-17T11:51:29.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:29 smithi125 ceph-mon[36819]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 262 KiB/s wr, 18 op/s 2024-02-17T11:51:29.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:29 smithi094 ceph-mon[32499]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 262 KiB/s wr, 18 op/s 2024-02-17T11:51:31.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:31 smithi189 ceph-mon[36588]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.3 KiB/s rd, 187 KiB/s wr, 13 op/s 2024-02-17T11:51:31.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:31 smithi125 ceph-mon[36819]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.3 KiB/s rd, 187 KiB/s wr, 13 op/s 2024-02-17T11:51:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:31 smithi094 ceph-mon[32499]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.3 KiB/s rd, 187 KiB/s wr, 13 op/s 2024-02-17T11:51:33.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:33 smithi094 ceph-mon[32499]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 145 KiB/s wr, 10 op/s 2024-02-17T11:51:33.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:33 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-17T11:51:33.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:33 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:33.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:33 smithi189 ceph-mon[36588]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 145 KiB/s wr, 10 op/s 2024-02-17T11:51:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:33 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-17T11:51:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:33 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:33.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:33 smithi125 ceph-mon[36819]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.3 KiB/s rd, 145 KiB/s wr, 10 op/s 2024-02-17T11:51:33.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:33 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-17T11:51:33.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:33 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:34.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:34 smithi094 ceph-mon[32499]: Deploying daemon osd.3 on smithi094 2024-02-17T11:51:34.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:34 smithi189 ceph-mon[36588]: Deploying daemon osd.3 on smithi094 2024-02-17T11:51:34.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:34 smithi125 ceph-mon[36819]: Deploying daemon osd.3 on smithi094 2024-02-17T11:51:35.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:35 smithi189 ceph-mon[36588]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-17T11:51:35.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:35 smithi125 ceph-mon[36819]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-17T11:51:35.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:35 smithi094 ceph-mon[32499]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-02-17T11:51:37.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:37 smithi189 ceph-mon[36588]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 125 KiB/s wr, 8 op/s 2024-02-17T11:51:37.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:37 smithi125 ceph-mon[36819]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 125 KiB/s wr, 8 op/s 2024-02-17T11:51:37.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:37 smithi094 ceph-mon[32499]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 125 KiB/s wr, 8 op/s 2024-02-17T11:51:39.741 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:39 smithi094 ceph-mon[32499]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-02-17T11:51:39.741 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:51:39.741 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:39.741 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:39.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:39 smithi125 ceph-mon[36819]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-02-17T11:51:39.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:39 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:51:39.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:39 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:39.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:39 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:39.962 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:39 smithi189 ceph-mon[36588]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2024-02-17T11:51:39.962 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:39 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:51:39.962 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:39 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:39.962 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:39 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:41.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:41 smithi125 ceph-mon[36819]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:41 smithi094 ceph-mon[32499]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:42.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:41 smithi189 ceph-mon[36588]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:42.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:42.751+0000 7ff287fff700 1 -- 172.21.15.94:0/1062719974 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff280000bc0 con 0x7ff2880a2140 2024-02-17T11:51:42.750 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 3 on host 'smithi094' 2024-02-17T11:51:42.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:42.753+0000 7ff285ffb700 1 -- 172.21.15.94:0/1062719974 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff2880a2140 msgr2=0x7ff2880a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:51:42.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:42.753+0000 7ff285ffb700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff2880a2140 0x7ff2880a4600 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff294006010 tx=0x7ff294000bc0 comp rx=0 tx=0).stop 2024-02-17T11:51:42.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:42.753+0000 7ff285ffb700 1 -- 172.21.15.94:0/1062719974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2a0103870 msgr2=0x7ff2a019f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:51:42.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:42.753+0000 7ff285ffb700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2a0103870 0x7ff2a019f0c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff298007da0 tx=0x7ff29800a340 comp rx=0 tx=0).stop 2024-02-17T11:51:42.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:42.753+0000 7ff285ffb700 1 -- 172.21.15.94:0/1062719974 shutdown_connections 2024-02-17T11:51:42.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:42.753+0000 7ff285ffb700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff2880a2140 0x7ff2880a4600 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:42.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:42.753+0000 7ff285ffb700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff2a0103870 0x7ff2a019f0c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:42.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:42.753+0000 7ff285ffb700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff2a0101cd0 0x7ff2a0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:42.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:42.753+0000 7ff285ffb700 1 --2- 172.21.15.94:0/1062719974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff2a0102ed0 0x7ff2a0198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:42.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:42.753+0000 7ff285ffb700 1 -- 172.21.15.94:0/1062719974 >> 172.21.15.94:0/1062719974 conn(0x7ff2a00fd250 msgr2=0x7ff2a0106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:51:42.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:42.753+0000 7ff285ffb700 1 -- 172.21.15.94:0/1062719974 shutdown_connections 2024-02-17T11:51:42.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:51:42.753+0000 7ff285ffb700 1 -- 172.21.15.94:0/1062719974 wait complete. 2024-02-17T11:51:42.950 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:42 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:42.950 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:42 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:42.950 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:42 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:42.950 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:42 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:51:42.950 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:42 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:42.951 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:42 smithi094 ceph-mon[32499]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:42.951 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:42 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:42.951 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:42 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:42.951 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:42 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:51:43.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:42 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:43.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:42 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:43.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:42 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:43.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:42 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:51:43.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:42 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:43.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:42 smithi189 ceph-mon[36588]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:43.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:42 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:43.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:42 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:43.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:42 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:51:43.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:42 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:43.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:42 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:43.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:42 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:43.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:42 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:51:43.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:42 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:43.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:42 smithi125 ceph-mon[36819]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:43.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:42 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:43.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:42 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:43.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:42 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:51:43.476 DEBUG:teuthology.orchestra.run.smithi094:osd.3> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.3.service 2024-02-17T11:51:43.479 INFO:tasks.cephadm:Deploying osd.4 on smithi125 with /dev/vg_nvme/lv_3... 2024-02-17T11:51:43.479 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-17T11:51:43.728 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T11:51:44.125 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:43 smithi125 ceph-mon[36819]: from='osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-17T11:51:44.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:43 smithi189 ceph-mon[36588]: from='osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-17T11:51:44.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:43 smithi094 ceph-mon[32499]: from='osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-17T11:51:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:44 smithi094 ceph-mon[32499]: from='osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-17T11:51:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:44 smithi094 ceph-mon[32499]: osdmap e23: 4 total, 3 up, 4 in 2024-02-17T11:51:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:44 smithi094 ceph-mon[32499]: from='osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-17T11:51:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:44 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:44.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:44 smithi094 ceph-mon[32499]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:44.956 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 11:51:44 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3[54664]: 2024-02-17T11:51:44.826+0000 7fab7b570700 -1 osd.3 0 waiting for initial osdmap 2024-02-17T11:51:44.956 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 11:51:44 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3[54664]: 2024-02-17T11:51:44.831+0000 7fab77363700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T11:51:45.095 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:44 smithi125 ceph-mon[36819]: from='osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-17T11:51:45.095 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:44 smithi125 ceph-mon[36819]: osdmap e23: 4 total, 3 up, 4 in 2024-02-17T11:51:45.095 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:44 smithi125 ceph-mon[36819]: from='osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-17T11:51:45.095 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:44 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:45.095 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:44 smithi125 ceph-mon[36819]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:45.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:44 smithi189 ceph-mon[36588]: from='osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-17T11:51:45.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:44 smithi189 ceph-mon[36588]: osdmap e23: 4 total, 3 up, 4 in 2024-02-17T11:51:45.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:44 smithi189 ceph-mon[36588]: from='osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-17T11:51:45.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:44 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:45.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:44 smithi189 ceph-mon[36588]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-02-17T11:51:45.669 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-17T11:51:45.669 INFO:teuthology.orchestra.run.smithi125.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-17T11:51:45.669 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 10+0 records in 2024-02-17T11:51:45.670 INFO:teuthology.orchestra.run.smithi125.stderr:10+0 records out 2024-02-17T11:51:45.670 INFO:teuthology.orchestra.run.smithi125.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115362 s, 909 MB/s 2024-02-17T11:51:45.670 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping successful for: 2024-02-17T11:51:46.017 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:45 smithi125 ceph-mon[36819]: from='osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-17T11:51:46.017 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:45 smithi125 ceph-mon[36819]: osdmap e24: 4 total, 3 up, 4 in 2024-02-17T11:51:46.017 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:46.017 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:46.017 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:45 smithi125 ceph-mon[36819]: osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621] boot 2024-02-17T11:51:46.017 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:45 smithi125 ceph-mon[36819]: osdmap e25: 4 total, 4 up, 4 in 2024-02-17T11:51:46.018 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:46.109 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:45 smithi094 ceph-mon[32499]: from='osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-17T11:51:46.109 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:45 smithi094 ceph-mon[32499]: osdmap e24: 4 total, 3 up, 4 in 2024-02-17T11:51:46.109 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:46.109 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:46.109 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:45 smithi094 ceph-mon[32499]: osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621] boot 2024-02-17T11:51:46.109 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:45 smithi094 ceph-mon[32499]: osdmap e25: 4 total, 4 up, 4 in 2024-02-17T11:51:46.109 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:46.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:45 smithi189 ceph-mon[36588]: from='osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-17T11:51:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:45 smithi189 ceph-mon[36588]: osdmap e24: 4 total, 3 up, 4 in 2024-02-17T11:51:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:45 smithi189 ceph-mon[36588]: osd.3 [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621] boot 2024-02-17T11:51:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:45 smithi189 ceph-mon[36588]: osdmap e25: 4 total, 4 up, 4 in 2024-02-17T11:51:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:51:46.362 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi125:vg_nvme/lv_3 2024-02-17T11:51:46.604 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T11:51:46.905 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: purged_snaps scrub starts 2024-02-17T11:51:46.905 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: purged_snaps scrub ok 2024-02-17T11:51:46.905 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: osdmap e26: 4 total, 4 up, 4 in 2024-02-17T11:51:46.905 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: Detected new or changed devices on smithi094 2024-02-17T11:51:46.905 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:46.905 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:46.905 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:51:46.905 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:51:46.906 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi094 to 8544M 2024-02-17T11:51:46.906 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:46.906 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:46.906 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:51:46.906 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:46.906 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:46 smithi125 ceph-mon[36819]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:51:47.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: purged_snaps scrub starts 2024-02-17T11:51:47.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: purged_snaps scrub ok 2024-02-17T11:51:47.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: osdmap e26: 4 total, 4 up, 4 in 2024-02-17T11:51:47.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: Detected new or changed devices on smithi094 2024-02-17T11:51:47.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:47.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:47.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:51:47.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:51:47.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi094 to 8544M 2024-02-17T11:51:47.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:47.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:47.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:51:47.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:47.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:46 smithi189 ceph-mon[36588]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:51:47.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: purged_snaps scrub starts 2024-02-17T11:51:47.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: purged_snaps scrub ok 2024-02-17T11:51:47.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: osdmap e26: 4 total, 4 up, 4 in 2024-02-17T11:51:47.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: Detected new or changed devices on smithi094 2024-02-17T11:51:47.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:47.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:47.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:51:47.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:51:47.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi094 to 8544M 2024-02-17T11:51:47.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:47.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:47.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:51:47.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:51:47.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:46 smithi094 ceph-mon[32499]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:51:48.040 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.039+0000 7fc093c69700 1 -- 172.21.15.125:0/3618934498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08c107990 msgr2=0x7fc08c109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:51:48.040 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.039+0000 7fc093c69700 1 --2- 172.21.15.125:0/3618934498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08c107990 0x7fc08c109e00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc08400b0a0 tx=0x7fc08400b3b0 comp rx=0 tx=0).stop 2024-02-17T11:51:48.040 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.039+0000 7fc093c69700 1 -- 172.21.15.125:0/3618934498 shutdown_connections 2024-02-17T11:51:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.039+0000 7fc093c69700 1 --2- 172.21.15.125:0/3618934498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08c107990 0x7fc08c109e00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.039+0000 7fc093c69700 1 --2- 172.21.15.125:0/3618934498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c102730 0x7fc08c104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.039+0000 7fc093c69700 1 --2- 172.21.15.125:0/3618934498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc08c105060 0x7fc08c107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.039+0000 7fc093c69700 1 -- 172.21.15.125:0/3618934498 >> 172.21.15.125:0/3618934498 conn(0x7fc08c0fc160 msgr2=0x7fc08c0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:51:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.039+0000 7fc093c69700 1 -- 172.21.15.125:0/3618934498 shutdown_connections 2024-02-17T11:51:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.039+0000 7fc093c69700 1 -- 172.21.15.125:0/3618934498 wait complete. 2024-02-17T11:51:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.039+0000 7fc093c69700 1 Processor -- start 2024-02-17T11:51:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc093c69700 1 -- start start 2024-02-17T11:51:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc093c69700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc08c102730 0x7fc08c197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:51:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc093c69700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08c105060 0x7fc08c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:51:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc093c69700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c107990 0x7fc08c19f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:51:48.041 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc093c69700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c19f8c0 con 0x7fc08c107990 2024-02-17T11:51:48.042 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc093c69700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c19fa00 con 0x7fc08c105060 2024-02-17T11:51:48.042 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc093c69700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c19fb90 con 0x7fc08c102730 2024-02-17T11:51:48.042 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc091204700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08c105060 0x7fc08c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:51:48.042 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc091204700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08c105060 0x7fc08c198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:41232/0 (socket says 172.21.15.125:41232) 2024-02-17T11:51:48.042 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc091204700 1 -- 172.21.15.125:0/2259909336 learned_addr learned my addr 172.21.15.125:0/2259909336 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-17T11:51:48.042 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc091a05700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc08c102730 0x7fc08c197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:51:48.042 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc091204700 1 -- 172.21.15.125:0/2259909336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc08c102730 msgr2=0x7fc08c197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:51:48.042 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc091204700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc08c102730 0x7fc08c197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:48.042 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc091204700 1 -- 172.21.15.125:0/2259909336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c107990 msgr2=0x7fc08c19f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:51:48.042 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc092206700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c107990 0x7fc08c19f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:51:48.042 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc091204700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c107990 0x7fc08c19f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:51:48.042 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc091204700 1 -- 172.21.15.125:0/2259909336 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc084009d00 con 0x7fc08c105060 2024-02-17T11:51:48.042 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc091204700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08c105060 0x7fc08c198270 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00d950 tx=0x7fc07c00dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:51:48.043 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc082ffd700 1 -- 172.21.15.125:0/2259909336 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c009b40 con 0x7fc08c105060 2024-02-17T11:51:48.043 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc082ffd700 1 -- 172.21.15.125:0/2259909336 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc07c009ca0 con 0x7fc08c105060 2024-02-17T11:51:48.043 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc082ffd700 1 -- 172.21.15.125:0/2259909336 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c00f690 con 0x7fc08c105060 2024-02-17T11:51:48.043 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.040+0000 7fc093c69700 1 -- 172.21.15.125:0/2259909336 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c19fe70 con 0x7fc08c105060 2024-02-17T11:51:48.043 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.041+0000 7fc093c69700 1 -- 172.21.15.125:0/2259909336 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c1a02e0 con 0x7fc08c105060 2024-02-17T11:51:48.043 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.041+0000 7fc093c69700 1 -- 172.21.15.125:0/2259909336 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08c068930 con 0x7fc08c105060 2024-02-17T11:51:48.044 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.042+0000 7fc082ffd700 1 -- 172.21.15.125:0/2259909336 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc07c010460 con 0x7fc08c105060 2024-02-17T11:51:48.044 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.043+0000 7fc082ffd700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc0780a2150 0x7fc0780a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:51:48.044 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.043+0000 7fc082ffd700 1 -- 172.21.15.125:0/2259909336 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fc07c0cca10 con 0x7fc08c105060 2024-02-17T11:51:48.044 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.044+0000 7fc091a05700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc0780a2150 0x7fc0780a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:51:48.044 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.044+0000 7fc091a05700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc0780a2150 0x7fc0780a4610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc088005950 tx=0x7fc0880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:51:48.046 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.047+0000 7fc082ffd700 1 -- 172.21.15.125:0/2259909336 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc07c096bc0 con 0x7fc08c105060 2024-02-17T11:51:48.251 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:51:48.251+0000 7fc093c69700 1 -- 172.21.15.125:0/2259909336 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc08c02d960 con 0x7fc0780a2150 2024-02-17T11:51:48.380 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:48 smithi094 ceph-mon[32499]: osdmap e27: 4 total, 4 up, 4 in 2024-02-17T11:51:48.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:48 smithi189 ceph-mon[36588]: osdmap e27: 4 total, 4 up, 4 in 2024-02-17T11:51:48.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:48 smithi125 ceph-mon[36819]: osdmap e27: 4 total, 4 up, 4 in 2024-02-17T11:51:49.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:49 smithi189 ceph-mon[36588]: osdmap e28: 4 total, 4 up, 4 in 2024-02-17T11:51:49.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:49 smithi189 ceph-mon[36588]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:51:49.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:49 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:51:49.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:49 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:51:49.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:49 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:49.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:49 smithi189 ceph-mon[36588]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 153 KiB/s, 0 objects/s recovering 2024-02-17T11:51:49.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:49 smithi125 ceph-mon[36819]: osdmap e28: 4 total, 4 up, 4 in 2024-02-17T11:51:49.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:49 smithi125 ceph-mon[36819]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:51:49.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:49 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:51:49.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:49 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:51:49.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:49 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:49.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:49 smithi125 ceph-mon[36819]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 153 KiB/s, 0 objects/s recovering 2024-02-17T11:51:49.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:49 smithi094 ceph-mon[32499]: osdmap e28: 4 total, 4 up, 4 in 2024-02-17T11:51:49.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:49 smithi094 ceph-mon[32499]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:51:49.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:49 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:51:49.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:49 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:51:49.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:49 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:49.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:49 smithi094 ceph-mon[32499]: pgmap v103: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 153 KiB/s, 0 objects/s recovering 2024-02-17T11:51:50.094 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:50 smithi125 ceph-mon[36819]: osdmap e29: 4 total, 4 up, 4 in 2024-02-17T11:51:50.387 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:50 smithi189 ceph-mon[36588]: osdmap e29: 4 total, 4 up, 4 in 2024-02-17T11:51:50.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:50 smithi094 ceph-mon[32499]: osdmap e29: 4 total, 4 up, 4 in 2024-02-17T11:51:51.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:51 smithi189 ceph-mon[36588]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-02-17T11:51:51.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:51 smithi189 ceph-mon[36588]: from='client.? 172.21.15.125:0/61001672' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbe91382-c694-4eb0-8e3a-1f9cdbd45356"}]: dispatch 2024-02-17T11:51:51.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:51 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbe91382-c694-4eb0-8e3a-1f9cdbd45356"}]: dispatch 2024-02-17T11:51:51.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:51 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbe91382-c694-4eb0-8e3a-1f9cdbd45356"}]': finished 2024-02-17T11:51:51.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:51 smithi189 ceph-mon[36588]: osdmap e30: 5 total, 4 up, 5 in 2024-02-17T11:51:51.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:51 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:51:51.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:51 smithi125 ceph-mon[36819]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-02-17T11:51:51.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:51 smithi125 ceph-mon[36819]: from='client.? 172.21.15.125:0/61001672' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbe91382-c694-4eb0-8e3a-1f9cdbd45356"}]: dispatch 2024-02-17T11:51:51.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:51 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbe91382-c694-4eb0-8e3a-1f9cdbd45356"}]: dispatch 2024-02-17T11:51:51.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:51 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbe91382-c694-4eb0-8e3a-1f9cdbd45356"}]': finished 2024-02-17T11:51:51.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:51 smithi125 ceph-mon[36819]: osdmap e30: 5 total, 4 up, 5 in 2024-02-17T11:51:51.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:51:51.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:51 smithi094 ceph-mon[32499]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-02-17T11:51:51.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:51 smithi094 ceph-mon[32499]: from='client.? 172.21.15.125:0/61001672' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbe91382-c694-4eb0-8e3a-1f9cdbd45356"}]: dispatch 2024-02-17T11:51:51.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:51 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dbe91382-c694-4eb0-8e3a-1f9cdbd45356"}]: dispatch 2024-02-17T11:51:51.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:51 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbe91382-c694-4eb0-8e3a-1f9cdbd45356"}]': finished 2024-02-17T11:51:51.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:51 smithi094 ceph-mon[32499]: osdmap e30: 5 total, 4 up, 5 in 2024-02-17T11:51:51.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:51 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:51:52.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:52 smithi189 ceph-mon[36588]: from='client.? 172.21.15.125:0/1743344071' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:51:52.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:52 smithi125 ceph-mon[36819]: from='client.? 172.21.15.125:0/1743344071' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:51:52.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:52 smithi094 ceph-mon[32499]: from='client.? 172.21.15.125:0/1743344071' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:51:53.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:53 smithi189 ceph-mon[36588]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-02-17T11:51:53.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:53 smithi125 ceph-mon[36819]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-02-17T11:51:53.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:53 smithi094 ceph-mon[32499]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-02-17T11:51:55.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:55 smithi189 ceph-mon[36588]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-02-17T11:51:55.908 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:55 smithi125 ceph-mon[36819]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-02-17T11:51:55.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:55 smithi094 ceph-mon[32499]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-02-17T11:51:57.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:57 smithi189 ceph-mon[36588]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:51:57.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:57 smithi125 ceph-mon[36819]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:51:57.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:57 smithi094 ceph-mon[32499]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:51:58.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-17T11:51:58.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:58.919 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-17T11:51:58.920 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:58.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-17T11:51:58.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:51:59.760 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:59 smithi125 ceph-mon[36819]: Deploying daemon osd.4 on smithi125 2024-02-17T11:51:59.760 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:51:59 smithi125 ceph-mon[36819]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:51:59.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:59 smithi189 ceph-mon[36588]: Deploying daemon osd.4 on smithi125 2024-02-17T11:51:59.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:51:59 smithi189 ceph-mon[36588]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:51:59.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:59 smithi094 ceph-mon[32499]: Deploying daemon osd.4 on smithi125 2024-02-17T11:51:59.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:51:59 smithi094 ceph-mon[32499]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:01.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:01 smithi125 ceph-mon[36819]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:01.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:01 smithi094 ceph-mon[32499]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:02.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:01 smithi189 ceph-mon[36588]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:03.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:03 smithi125 ceph-mon[36819]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:03.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:03 smithi094 ceph-mon[32499]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:04.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:03 smithi189 ceph-mon[36588]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:05.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:04 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:05.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:04 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:05.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:04 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:05.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:04 smithi189 ceph-mon[36588]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:05.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:04 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:05.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:04 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:05.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:04 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:05.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:04 smithi125 ceph-mon[36819]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:05.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:04 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:05.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:04 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:05.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:04 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:05.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:04 smithi094 ceph-mon[32499]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:07.264 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:52:07.265+0000 7fc082ffd700 1 -- 172.21.15.125:0/2259909336 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc08c02d960 con 0x7fc0780a2150 2024-02-17T11:52:07.265 INFO:teuthology.orchestra.run.smithi125.stdout:Created osd(s) 4 on host 'smithi125' 2024-02-17T11:52:07.267 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:52:07.268+0000 7fc093c69700 1 -- 172.21.15.125:0/2259909336 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc0780a2150 msgr2=0x7fc0780a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:52:07.267 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:52:07.268+0000 7fc093c69700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc0780a2150 0x7fc0780a4610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc088005950 tx=0x7fc0880058e0 comp rx=0 tx=0).stop 2024-02-17T11:52:07.267 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:52:07.268+0000 7fc093c69700 1 -- 172.21.15.125:0/2259909336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08c105060 msgr2=0x7fc08c198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:52:07.267 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:52:07.268+0000 7fc093c69700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08c105060 0x7fc08c198270 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc07c00d950 tx=0x7fc07c00dd10 comp rx=0 tx=0).stop 2024-02-17T11:52:07.267 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:52:07.268+0000 7fc093c69700 1 -- 172.21.15.125:0/2259909336 shutdown_connections 2024-02-17T11:52:07.267 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:52:07.268+0000 7fc093c69700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc0780a2150 0x7fc0780a4610 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:07.267 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:52:07.268+0000 7fc093c69700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc08c107990 0x7fc08c19f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:07.267 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:52:07.268+0000 7fc093c69700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc08c102730 0x7fc08c197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:07.267 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:52:07.268+0000 7fc093c69700 1 --2- 172.21.15.125:0/2259909336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc08c105060 0x7fc08c198270 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:07.267 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:52:07.268+0000 7fc093c69700 1 -- 172.21.15.125:0/2259909336 >> 172.21.15.125:0/2259909336 conn(0x7fc08c0fc160 msgr2=0x7fc08c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:52:07.268 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:52:07.268+0000 7fc093c69700 1 -- 172.21.15.125:0/2259909336 shutdown_connections 2024-02-17T11:52:07.268 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:52:07.268+0000 7fc093c69700 1 -- 172.21.15.125:0/2259909336 wait complete. 2024-02-17T11:52:07.873 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:07 smithi125 ceph-mon[36819]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:07.873 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:07 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:07.873 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:07 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:07.901 DEBUG:teuthology.orchestra.run.smithi125:osd.4> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.4.service 2024-02-17T11:52:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:07 smithi189 ceph-mon[36588]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:07 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:07 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:07.904 INFO:tasks.cephadm:Deploying osd.5 on smithi189 with /dev/vg_nvme/lv_3... 2024-02-17T11:52:07.904 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-17T11:52:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:07 smithi094 ceph-mon[32499]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:07 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:07 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:08.153 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T11:52:08.810 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:08 smithi189 ceph-mon[36588]: from='osd.4 [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-17T11:52:08.810 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:08 smithi189 ceph-mon[36588]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-17T11:52:08.810 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:08 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:08.810 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:08 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:08.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:08 smithi125 ceph-mon[36819]: from='osd.4 [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-17T11:52:08.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:08 smithi125 ceph-mon[36819]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-17T11:52:08.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:08 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:08.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:08 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:08.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:08 smithi094 ceph-mon[32499]: from='osd.4 [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-17T11:52:08.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:08 smithi094 ceph-mon[32499]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-17T11:52:08.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:08 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:08.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:08 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:09.756 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:09 smithi189 ceph-mon[36588]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:09.756 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:09 smithi189 ceph-mon[36588]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-17T11:52:09.756 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:09 smithi189 ceph-mon[36588]: osdmap e31: 5 total, 4 up, 5 in 2024-02-17T11:52:09.756 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:09 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:09.756 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:09 smithi189 ceph-mon[36588]: from='osd.4 [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:52:09.756 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:09 smithi189 ceph-mon[36588]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:52:09.770 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:09 smithi125 ceph-mon[36819]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:09.770 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:09 smithi125 ceph-mon[36819]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-17T11:52:09.771 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:09 smithi125 ceph-mon[36819]: osdmap e31: 5 total, 4 up, 5 in 2024-02-17T11:52:09.771 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:09 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:09.771 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:09 smithi125 ceph-mon[36819]: from='osd.4 [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:52:09.771 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:09 smithi125 ceph-mon[36819]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:52:09.771 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 11:52:09 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4[48098]: 2024-02-17T11:52:09.659+0000 7fc8da616700 -1 osd.4 0 waiting for initial osdmap 2024-02-17T11:52:09.771 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 11:52:09 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4[48098]: 2024-02-17T11:52:09.666+0000 7fc8d6409700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T11:52:09.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:09 smithi094 ceph-mon[32499]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:09.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:09 smithi094 ceph-mon[32499]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-17T11:52:09.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:09 smithi094 ceph-mon[32499]: osdmap e31: 5 total, 4 up, 5 in 2024-02-17T11:52:09.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:09 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:09.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:09 smithi094 ceph-mon[32499]: from='osd.4 [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:52:09.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:09 smithi094 ceph-mon[32499]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:52:10.334 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-17T11:52:10.334 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-17T11:52:10.334 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-02-17T11:52:10.334 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-02-17T11:52:10.334 INFO:teuthology.orchestra.run.smithi189.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111984 s, 936 MB/s 2024-02-17T11:52:10.334 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-02-17T11:52:10.880 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:10 smithi189 ceph-mon[36588]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2024-02-17T11:52:10.880 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:10 smithi189 ceph-mon[36588]: osdmap e32: 5 total, 4 up, 5 in 2024-02-17T11:52:10.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:10.881 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:10.882 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi189:vg_nvme/lv_3 2024-02-17T11:52:10.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:10 smithi125 ceph-mon[36819]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2024-02-17T11:52:10.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:10 smithi125 ceph-mon[36819]: osdmap e32: 5 total, 4 up, 5 in 2024-02-17T11:52:10.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:10.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:10.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:10 smithi094 ceph-mon[32499]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2024-02-17T11:52:10.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:10 smithi094 ceph-mon[32499]: osdmap e32: 5 total, 4 up, 5 in 2024-02-17T11:52:10.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:10.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:11.121 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T11:52:11.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: purged_snaps scrub starts 2024-02-17T11:52:11.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: purged_snaps scrub ok 2024-02-17T11:52:11.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:11.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:11.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: osd.4 [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927] boot 2024-02-17T11:52:11.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: osdmap e33: 5 total, 5 up, 5 in 2024-02-17T11:52:11.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:11.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:52:11.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:52:11.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:11.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:52:11.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:11.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:11.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:52:11.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:11 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: purged_snaps scrub starts 2024-02-17T11:52:11.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: purged_snaps scrub ok 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: osd.4 [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927] boot 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: osdmap e33: 5 total, 5 up, 5 in 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:11.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:11.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:52:11.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:11 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: purged_snaps scrub starts 2024-02-17T11:52:11.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: purged_snaps scrub ok 2024-02-17T11:52:11.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-02-17T11:52:11.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:11.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: osd.4 [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927] boot 2024-02-17T11:52:11.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: osdmap e33: 5 total, 5 up, 5 in 2024-02-17T11:52:11.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:52:11.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:52:11.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:52:11.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:11.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:52:11.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:11.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:11.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:11.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:52:11.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:11 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:12.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.703+0000 7f8f92a21700 1 -- 172.21.15.189:0/3597323646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c0fff80 msgr2=0x7f8f8c1003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:52:12.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.703+0000 7f8f92a21700 1 --2- 172.21.15.189:0/3597323646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c0fff80 0x7f8f8c1003a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c009990 tx=0x7f8f7c009ca0 comp rx=0 tx=0).stop 2024-02-17T11:52:12.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.703+0000 7f8f92a21700 1 -- 172.21.15.189:0/3597323646 shutdown_connections 2024-02-17T11:52:12.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.703+0000 7f8f92a21700 1 --2- 172.21.15.189:0/3597323646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f8c103900 0x7f8f8c10a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:12.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.703+0000 7f8f92a21700 1 --2- 172.21.15.189:0/3597323646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c0fff80 0x7f8f8c1003a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:12.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.703+0000 7f8f92a21700 1 --2- 172.21.15.189:0/3597323646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f8c102f60 0x7f8f8c1033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:12.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.703+0000 7f8f92a21700 1 -- 172.21.15.189:0/3597323646 >> 172.21.15.189:0/3597323646 conn(0x7f8f8c0fb900 msgr2=0x7f8f8c0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:52:12.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.703+0000 7f8f92a21700 1 -- 172.21.15.189:0/3597323646 shutdown_connections 2024-02-17T11:52:12.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.703+0000 7f8f92a21700 1 -- 172.21.15.189:0/3597323646 wait complete. 2024-02-17T11:52:12.705 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.703+0000 7f8f92a21700 1 Processor -- start 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.703+0000 7f8f92a21700 1 -- start start 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f92a21700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c0fff80 0x7f8f8c1028c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f92a21700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f8c102f60 0x7f8f8c100f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f92a21700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f8c103900 0x7f8f8c101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f92a21700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c101bd0 con 0x7f8f8c102f60 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f92a21700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c101d40 con 0x7f8f8c103900 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f92a21700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c1a1d80 con 0x7f8f8c0fff80 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f8bfff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c0fff80 0x7f8f8c1028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f8bfff700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c0fff80 0x7f8f8c1028c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:54240/0 (socket says 172.21.15.189:54240) 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f8bfff700 1 -- 172.21.15.189:0/2714383064 learned_addr learned my addr 172.21.15.189:0/2714383064 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f8b7fe700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f8c102f60 0x7f8f8c100f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f90fbe700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f8c103900 0x7f8f8c101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f8bfff700 1 -- 172.21.15.189:0/2714383064 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f8c103900 msgr2=0x7f8f8c101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:52:12.706 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f8bfff700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f8c103900 0x7f8f8c101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:12.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f8bfff700 1 -- 172.21.15.189:0/2714383064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f8c102f60 msgr2=0x7f8f8c100f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:52:12.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f8bfff700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f8c102f60 0x7f8f8c100f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:12.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f8bfff700 1 -- 172.21.15.189:0/2714383064 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c0095f0 con 0x7f8f8c0fff80 2024-02-17T11:52:12.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f8bfff700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c0fff80 0x7f8f8c1028c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0098e0 tx=0x7f8f7c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:52:12.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f897fa700 1 -- 172.21.15.189:0/2714383064 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c01d070 con 0x7f8f8c0fff80 2024-02-17T11:52:12.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f92a21700 1 -- 172.21.15.189:0/2714383064 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c1a2000 con 0x7f8f8c0fff80 2024-02-17T11:52:12.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.704+0000 7f8f897fa700 1 -- 172.21.15.189:0/2714383064 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f7c005530 con 0x7f8f8c0fff80 2024-02-17T11:52:12.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.705+0000 7f8f897fa700 1 -- 172.21.15.189:0/2714383064 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c004cb0 con 0x7f8f8c0fff80 2024-02-17T11:52:12.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.705+0000 7f8f92a21700 1 -- 172.21.15.189:0/2714383064 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c1a24a0 con 0x7f8f8c0fff80 2024-02-17T11:52:12.707 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.705+0000 7f8f92a21700 1 -- 172.21.15.189:0/2714383064 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8c109120 con 0x7f8f8c0fff80 2024-02-17T11:52:12.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.706+0000 7f8f897fa700 1 -- 172.21.15.189:0/2714383064 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8f7c00fcd0 con 0x7f8f8c0fff80 2024-02-17T11:52:12.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.707+0000 7f8f897fa700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8f740a2150 0x7f8f740a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:52:12.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.707+0000 7f8f897fa700 1 -- 172.21.15.189:0/2714383064 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f8f7c0cdcf0 con 0x7f8f8c0fff80 2024-02-17T11:52:12.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.707+0000 7f8f8b7fe700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8f740a2150 0x7f8f740a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:52:12.708 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.708+0000 7f8f8b7fe700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8f740a2150 0x7f8f740a4610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009a10 tx=0x7f8f80009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:52:12.711 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.712+0000 7f8f897fa700 1 -- 172.21.15.189:0/2714383064 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8f7c097dc0 con 0x7f8f8c0fff80 2024-02-17T11:52:12.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:12 smithi189 ceph-mon[36588]: Detected new or changed devices on smithi125 2024-02-17T11:52:12.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:12 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi125 to 8540M 2024-02-17T11:52:12.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:12 smithi189 ceph-mon[36588]: osdmap e34: 5 total, 5 up, 5 in 2024-02-17T11:52:12.917 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:12.918+0000 7f8f92a21700 1 -- 172.21.15.189:0/2714383064 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f8c02d7d0 con 0x7f8f740a2150 2024-02-17T11:52:12.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:12 smithi125 ceph-mon[36819]: Detected new or changed devices on smithi125 2024-02-17T11:52:12.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:12 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi125 to 8540M 2024-02-17T11:52:12.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:12 smithi125 ceph-mon[36819]: osdmap e34: 5 total, 5 up, 5 in 2024-02-17T11:52:12.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:12 smithi094 ceph-mon[32499]: Detected new or changed devices on smithi125 2024-02-17T11:52:12.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:12 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi125 to 8540M 2024-02-17T11:52:12.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:12 smithi094 ceph-mon[32499]: osdmap e34: 5 total, 5 up, 5 in 2024-02-17T11:52:13.802 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:13 smithi189 ceph-mon[36588]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:13.802 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:52:13.802 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:52:13.802 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:13 smithi125 ceph-mon[36819]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:52:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:52:13.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:13 smithi094 ceph-mon[32499]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:52:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:52:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:14.787 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:14 smithi189 ceph-mon[36588]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:52:14.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:14 smithi125 ceph-mon[36819]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:52:14.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:14 smithi094 ceph-mon[32499]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:52:15.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:15 smithi125 ceph-mon[36819]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:15.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:15 smithi125 ceph-mon[36819]: from='client.? 172.21.15.189:0/3920417820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa318c31-a0a3-4375-9980-5786a855c103"}]: dispatch 2024-02-17T11:52:15.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:15 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa318c31-a0a3-4375-9980-5786a855c103"}]: dispatch 2024-02-17T11:52:15.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:15 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa318c31-a0a3-4375-9980-5786a855c103"}]': finished 2024-02-17T11:52:15.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:15 smithi125 ceph-mon[36819]: osdmap e35: 6 total, 5 up, 6 in 2024-02-17T11:52:15.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:15 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:15.953 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:15 smithi189 ceph-mon[36588]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:15.953 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:15 smithi189 ceph-mon[36588]: from='client.? 172.21.15.189:0/3920417820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa318c31-a0a3-4375-9980-5786a855c103"}]: dispatch 2024-02-17T11:52:15.953 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:15 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa318c31-a0a3-4375-9980-5786a855c103"}]: dispatch 2024-02-17T11:52:15.953 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:15 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa318c31-a0a3-4375-9980-5786a855c103"}]': finished 2024-02-17T11:52:15.953 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:15 smithi189 ceph-mon[36588]: osdmap e35: 6 total, 5 up, 6 in 2024-02-17T11:52:15.953 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:15 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:15 smithi094 ceph-mon[32499]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:15 smithi094 ceph-mon[32499]: from='client.? 172.21.15.189:0/3920417820' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa318c31-a0a3-4375-9980-5786a855c103"}]: dispatch 2024-02-17T11:52:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:15 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aa318c31-a0a3-4375-9980-5786a855c103"}]: dispatch 2024-02-17T11:52:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:15 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aa318c31-a0a3-4375-9980-5786a855c103"}]': finished 2024-02-17T11:52:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:15 smithi094 ceph-mon[32499]: osdmap e35: 6 total, 5 up, 6 in 2024-02-17T11:52:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:15 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:16.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:16 smithi125 ceph-mon[36819]: from='client.? 172.21.15.189:0/1327772933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:52:16.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:16 smithi094 ceph-mon[32499]: from='client.? 172.21.15.189:0/1327772933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:52:17.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:16 smithi189 ceph-mon[36588]: from='client.? 172.21.15.189:0/1327772933' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:52:17.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:17 smithi125 ceph-mon[36819]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:17.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:17 smithi094 ceph-mon[32499]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:18.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:17 smithi189 ceph-mon[36588]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:19.856 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:19 smithi189 ceph-mon[36588]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:19.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:19 smithi125 ceph-mon[36819]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:19 smithi094 ceph-mon[32499]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:21.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:21 smithi125 ceph-mon[36819]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:21.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:21 smithi094 ceph-mon[32499]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:22.147 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:21 smithi189 ceph-mon[36588]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:23.764 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:23 smithi189 ceph-mon[36588]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:23.764 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:23 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-17T11:52:23.764 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:23 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:23.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:23 smithi094 ceph-mon[32499]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:23.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:23 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-17T11:52:23.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:23 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:24.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:23 smithi125 ceph-mon[36819]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:24.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:23 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-17T11:52:24.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:23 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:24.887 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:24 smithi189 ceph-mon[36588]: Deploying daemon osd.5 on smithi189 2024-02-17T11:52:24.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:24 smithi094 ceph-mon[32499]: Deploying daemon osd.5 on smithi189 2024-02-17T11:52:25.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:24 smithi125 ceph-mon[36819]: Deploying daemon osd.5 on smithi189 2024-02-17T11:52:25.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:25 smithi094 ceph-mon[32499]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:25.965 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:25 smithi189 ceph-mon[36588]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:26.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:25 smithi125 ceph-mon[36819]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:27.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:27 smithi189 ceph-mon[36588]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:27.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:27 smithi094 ceph-mon[32499]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:28.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:27 smithi125 ceph-mon[36819]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:29.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:29 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:29.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:29 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:29.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:29 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:29.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:29 smithi125 ceph-mon[36819]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:29.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:29 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:29.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:29 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:29.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:29 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:29.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:29 smithi094 ceph-mon[32499]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:29.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:29 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:29.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:29 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:29.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:29 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:29.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:29 smithi189 ceph-mon[36588]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:31.803 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:31 smithi189 ceph-mon[36588]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:31.803 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:31 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:31.803 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:31 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:31.803 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:31 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:31.803 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:31 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:52:31.803 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:31 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:31.881 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:31.882+0000 7f8f897fa700 1 -- 172.21.15.189:0/2714383064 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8f8c02d7d0 con 0x7f8f740a2150 2024-02-17T11:52:31.881 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 5 on host 'smithi189' 2024-02-17T11:52:31.883 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:31.884+0000 7f8f92a21700 1 -- 172.21.15.189:0/2714383064 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8f740a2150 msgr2=0x7f8f740a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:52:31.883 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:31.884+0000 7f8f92a21700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8f740a2150 0x7f8f740a4610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009a10 tx=0x7f8f80009190 comp rx=0 tx=0).stop 2024-02-17T11:52:31.883 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:31.884+0000 7f8f92a21700 1 -- 172.21.15.189:0/2714383064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c0fff80 msgr2=0x7f8f8c1028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:52:31.884 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:31.884+0000 7f8f92a21700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c0fff80 0x7f8f8c1028c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0098e0 tx=0x7f8f7c00f690 comp rx=0 tx=0).stop 2024-02-17T11:52:31.884 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:31.884+0000 7f8f92a21700 1 -- 172.21.15.189:0/2714383064 shutdown_connections 2024-02-17T11:52:31.884 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:31.884+0000 7f8f92a21700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8f740a2150 0x7f8f740a4610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:31.884 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:31.884+0000 7f8f92a21700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f8c103900 0x7f8f8c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:31.884 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:31.884+0000 7f8f92a21700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f8c0fff80 0x7f8f8c1028c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:31.884 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:31.884+0000 7f8f92a21700 1 --2- 172.21.15.189:0/2714383064 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f8c102f60 0x7f8f8c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:31.884 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:31.884+0000 7f8f92a21700 1 -- 172.21.15.189:0/2714383064 >> 172.21.15.189:0/2714383064 conn(0x7f8f8c0fb900 msgr2=0x7f8f8c106260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:52:31.884 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:31.884+0000 7f8f92a21700 1 -- 172.21.15.189:0/2714383064 shutdown_connections 2024-02-17T11:52:31.884 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:52:31.884+0000 7f8f92a21700 1 -- 172.21.15.189:0/2714383064 wait complete. 2024-02-17T11:52:31.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:31 smithi125 ceph-mon[36819]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:31.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:31 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:31.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:31 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:31.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:31 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:31.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:31 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:52:31.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:31 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:31 smithi094 ceph-mon[32499]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:31 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:31 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:31 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:31 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:52:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:31 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:32.507 DEBUG:teuthology.orchestra.run.smithi189:osd.5> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.5.service 2024-02-17T11:52:32.510 INFO:tasks.cephadm:Deploying osd.6 on smithi094 with /dev/vg_nvme/lv_2... 2024-02-17T11:52:32.510 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-17T11:52:32.758 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:52:33.113 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:32 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:33.113 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:32 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:33.113 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:32 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:33.113 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:32 smithi094 ceph-mon[32499]: from='osd.5 [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-17T11:52:33.113 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:32 smithi094 ceph-mon[32499]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-17T11:52:33.113 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:32 smithi094 ceph-mon[32499]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:33.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:32 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:33.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:32 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:33.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:32 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:33.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:32 smithi189 ceph-mon[36588]: from='osd.5 [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-17T11:52:33.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:32 smithi189 ceph-mon[36588]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-17T11:52:33.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:32 smithi189 ceph-mon[36588]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:33.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:32 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:33.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:32 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:33.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:32 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:33.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:32 smithi125 ceph-mon[36819]: from='osd.5 [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-17T11:52:33.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:32 smithi125 ceph-mon[36819]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-17T11:52:33.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:32 smithi125 ceph-mon[36819]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:33.950 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:33 smithi189 ceph-mon[36588]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-17T11:52:33.950 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:33 smithi189 ceph-mon[36588]: osdmap e36: 6 total, 5 up, 6 in 2024-02-17T11:52:33.950 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:33 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:33.950 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:33 smithi189 ceph-mon[36588]: from='osd.5 [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:52:33.950 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:33 smithi189 ceph-mon[36588]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:52:34.137 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:33 smithi094 ceph-mon[32499]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-17T11:52:34.137 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:33 smithi094 ceph-mon[32499]: osdmap e36: 6 total, 5 up, 6 in 2024-02-17T11:52:34.137 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:33 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:34.137 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:33 smithi094 ceph-mon[32499]: from='osd.5 [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:52:34.137 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:33 smithi094 ceph-mon[32499]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:52:34.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:33 smithi125 ceph-mon[36819]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-17T11:52:34.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:33 smithi125 ceph-mon[36819]: osdmap e36: 6 total, 5 up, 6 in 2024-02-17T11:52:34.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:33 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:34.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:33 smithi125 ceph-mon[36819]: from='osd.5 [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:52:34.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:33 smithi125 ceph-mon[36819]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:52:34.651 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 11:52:34 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T11:52:34.297+0000 7fc4cf35e700 -1 osd.5 0 waiting for initial osdmap 2024-02-17T11:52:34.651 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 11:52:34 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T11:52:34.308+0000 7fc4c7149700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T11:52:34.832 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-17T11:52:34.833 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-17T11:52:34.833 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-02-17T11:52:34.833 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-02-17T11:52:34.833 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00851952 s, 1.2 GB/s 2024-02-17T11:52:34.833 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-02-17T11:52:35.129 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:34 smithi189 ceph-mon[36588]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-17T11:52:35.129 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:34 smithi189 ceph-mon[36588]: osdmap e37: 6 total, 5 up, 6 in 2024-02-17T11:52:35.129 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:34 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:35.129 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:34 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:35.129 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:34 smithi189 ceph-mon[36588]: from='osd.5 ' entity='osd.5' 2024-02-17T11:52:35.129 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:34 smithi189 ceph-mon[36588]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:35.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:34 smithi125 ceph-mon[36819]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-17T11:52:35.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:34 smithi125 ceph-mon[36819]: osdmap e37: 6 total, 5 up, 6 in 2024-02-17T11:52:35.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:34 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:35.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:34 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:35.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:34 smithi125 ceph-mon[36819]: from='osd.5 ' entity='osd.5' 2024-02-17T11:52:35.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:34 smithi125 ceph-mon[36819]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:35.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:34 smithi094 ceph-mon[32499]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-17T11:52:35.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:34 smithi094 ceph-mon[32499]: osdmap e37: 6 total, 5 up, 6 in 2024-02-17T11:52:35.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:34 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:35.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:34 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:35.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:34 smithi094 ceph-mon[32499]: from='osd.5 ' entity='osd.5' 2024-02-17T11:52:35.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:34 smithi094 ceph-mon[32499]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-02-17T11:52:35.548 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi094:vg_nvme/lv_2 2024-02-17T11:52:35.783 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:52:36.140 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: purged_snaps scrub starts 2024-02-17T11:52:36.140 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: purged_snaps scrub ok 2024-02-17T11:52:36.140 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:36.140 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: osd.5 [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244] boot 2024-02-17T11:52:36.140 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: osdmap e38: 6 total, 6 up, 6 in 2024-02-17T11:52:36.141 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:36.141 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: osdmap e39: 6 total, 6 up, 6 in 2024-02-17T11:52:36.141 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: Detected new or changed devices on smithi189 2024-02-17T11:52:36.141 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:36.141 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:36.141 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:52:36.141 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:52:36.141 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi189 to 8540M 2024-02-17T11:52:36.141 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:36.141 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:36.141 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:52:36.141 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:36.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: purged_snaps scrub starts 2024-02-17T11:52:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: purged_snaps scrub ok 2024-02-17T11:52:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: osd.5 [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244] boot 2024-02-17T11:52:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: osdmap e38: 6 total, 6 up, 6 in 2024-02-17T11:52:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: osdmap e39: 6 total, 6 up, 6 in 2024-02-17T11:52:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: Detected new or changed devices on smithi189 2024-02-17T11:52:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:52:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:52:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi189 to 8540M 2024-02-17T11:52:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:36.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:36.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:52:36.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:36.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: purged_snaps scrub starts 2024-02-17T11:52:36.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: purged_snaps scrub ok 2024-02-17T11:52:36.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: osd.5 [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244] boot 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: osdmap e38: 6 total, 6 up, 6 in 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: osdmap e39: 6 total, 6 up, 6 in 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: Detected new or changed devices on smithi189 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi189 to 8540M 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:52:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:37.197 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:37 smithi094 ceph-mon[32499]: osdmap e40: 6 total, 6 up, 6 in 2024-02-17T11:52:37.197 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:37 smithi094 ceph-mon[32499]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:37.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:37 smithi189 ceph-mon[36588]: osdmap e40: 6 total, 6 up, 6 in 2024-02-17T11:52:37.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:37 smithi189 ceph-mon[36588]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:37.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.410+0000 7fcd3c408700 1 -- 172.21.15.94:0/357666294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34102ea0 msgr2=0x7fcd34103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:52:37.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.410+0000 7fcd3c408700 1 --2- 172.21.15.94:0/357666294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34102ea0 0x7fcd34103300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcd240099c0 tx=0x7fcd24009cd0 comp rx=0 tx=0).stop 2024-02-17T11:52:37.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.410+0000 7fcd3c408700 1 -- 172.21.15.94:0/357666294 shutdown_connections 2024-02-17T11:52:37.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.410+0000 7fcd3c408700 1 --2- 172.21.15.94:0/357666294 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd34103840 0x7fcd3410a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:37.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.410+0000 7fcd3c408700 1 --2- 172.21.15.94:0/357666294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd34101ca0 0x7fcd341020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:37.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.410+0000 7fcd3c408700 1 --2- 172.21.15.94:0/357666294 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34102ea0 0x7fcd34103300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:37.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.410+0000 7fcd3c408700 1 -- 172.21.15.94:0/357666294 >> 172.21.15.94:0/357666294 conn(0x7fcd340fd280 msgr2=0x7fcd340ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:52:37.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.410+0000 7fcd3c408700 1 -- 172.21.15.94:0/357666294 shutdown_connections 2024-02-17T11:52:37.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.410+0000 7fcd3c408700 1 -- 172.21.15.94:0/357666294 wait complete. 2024-02-17T11:52:37.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.411+0000 7fcd3c408700 1 Processor -- start 2024-02-17T11:52:37.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.411+0000 7fcd3c408700 1 -- start start 2024-02-17T11:52:37.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.411+0000 7fcd3c408700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd34101ca0 0x7fcd34197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:52:37.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.411+0000 7fcd3c408700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd34102ea0 0x7fcd34198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:52:37.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.411+0000 7fcd3c408700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34103840 0x7fcd3419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:52:37.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.411+0000 7fcd3c408700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3419f8d0 con 0x7fcd34103840 2024-02-17T11:52:37.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.411+0000 7fcd3c408700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3419fa10 con 0x7fcd34101ca0 2024-02-17T11:52:37.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.411+0000 7fcd3c408700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3419fba0 con 0x7fcd34102ea0 2024-02-17T11:52:37.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.411+0000 7fcd3a9a5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34103840 0x7fcd3419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:52:37.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd3a9a5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34103840 0x7fcd3419f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38988/0 (socket says 172.21.15.94:38988) 2024-02-17T11:52:37.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd3a9a5700 1 -- 172.21.15.94:0/1632345829 learned_addr learned my addr 172.21.15.94:0/1632345829 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:52:37.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd3a1a4700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd34101ca0 0x7fcd34197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:52:37.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd399a3700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd34102ea0 0x7fcd34198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:52:37.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd3a9a5700 1 -- 172.21.15.94:0/1632345829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd34102ea0 msgr2=0x7fcd34198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:52:37.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd3a9a5700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd34102ea0 0x7fcd34198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:37.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd3a9a5700 1 -- 172.21.15.94:0/1632345829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd34101ca0 msgr2=0x7fcd34197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:52:37.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd3a9a5700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd34101ca0 0x7fcd34197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:37.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd3a9a5700 1 -- 172.21.15.94:0/1632345829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd240095f0 con 0x7fcd34103840 2024-02-17T11:52:37.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd3a9a5700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34103840 0x7fcd3419f300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00b9a0 tx=0x7fcd2c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:52:37.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd2b7fe700 1 -- 172.21.15.94:0/1632345829 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c00c9b0 con 0x7fcd34103840 2024-02-17T11:52:37.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd2b7fe700 1 -- 172.21.15.94:0/1632345829 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcd2c014440 con 0x7fcd34103840 2024-02-17T11:52:37.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd3c408700 1 -- 172.21.15.94:0/1632345829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3419fe80 con 0x7fcd34103840 2024-02-17T11:52:37.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd2b7fe700 1 -- 172.21.15.94:0/1632345829 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0075d0 con 0x7fcd34103840 2024-02-17T11:52:37.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.412+0000 7fcd3c408700 1 -- 172.21.15.94:0/1632345829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd341a0320 con 0x7fcd34103840 2024-02-17T11:52:37.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.413+0000 7fcd3c408700 1 -- 172.21.15.94:0/1632345829 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd34109060 con 0x7fcd34103840 2024-02-17T11:52:37.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.414+0000 7fcd2b7fe700 1 -- 172.21.15.94:0/1632345829 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fcd2c014920 con 0x7fcd34103840 2024-02-17T11:52:37.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.415+0000 7fcd2b7fe700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fcd200a2150 0x7fcd200a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:52:37.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.415+0000 7fcd3a1a4700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fcd200a2150 0x7fcd200a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:52:37.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.415+0000 7fcd2b7fe700 1 -- 172.21.15.94:0/1632345829 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4533+0+0 (secure 0 0 0) 0x7fcd2c006e40 con 0x7fcd34103840 2024-02-17T11:52:37.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.415+0000 7fcd3a1a4700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fcd200a2150 0x7fcd200a4610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd30006fd0 tx=0x7fcd30008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:52:37.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.419+0000 7fcd2b7fe700 1 -- 172.21.15.94:0/1632345829 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd2c096880 con 0x7fcd34103840 2024-02-17T11:52:37.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:37 smithi125 ceph-mon[36819]: osdmap e40: 6 total, 6 up, 6 in 2024-02-17T11:52:37.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:37 smithi125 ceph-mon[36819]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:37.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:37.622+0000 7fcd3c408700 1 -- 172.21.15.94:0/1632345829 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd3402d8e0 con 0x7fcd200a2150 2024-02-17T11:52:38.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:38 smithi094 ceph-mon[32499]: osdmap e41: 6 total, 6 up, 6 in 2024-02-17T11:52:38.351 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:38 smithi094 ceph-mon[32499]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:52:38.352 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:38 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:52:38.352 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:38 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:52:38.352 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:38 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:38.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:38 smithi189 ceph-mon[36588]: osdmap e41: 6 total, 6 up, 6 in 2024-02-17T11:52:38.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:38 smithi189 ceph-mon[36588]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:52:38.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:38 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:52:38.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:38 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:52:38.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:38 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:38.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:38 smithi125 ceph-mon[36819]: osdmap e41: 6 total, 6 up, 6 in 2024-02-17T11:52:38.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:38 smithi125 ceph-mon[36819]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:52:38.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:38 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:52:38.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:38 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:52:38.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:38 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:39.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:39 smithi189 ceph-mon[36588]: osdmap e42: 6 total, 6 up, 6 in 2024-02-17T11:52:39.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:39 smithi189 ceph-mon[36588]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-02-17T11:52:39.411 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:39 smithi094 ceph-mon[32499]: osdmap e42: 6 total, 6 up, 6 in 2024-02-17T11:52:39.411 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:39 smithi094 ceph-mon[32499]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-02-17T11:52:39.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:39 smithi125 ceph-mon[36819]: osdmap e42: 6 total, 6 up, 6 in 2024-02-17T11:52:39.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:39 smithi125 ceph-mon[36819]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-02-17T11:52:40.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:40 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/495818417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fbef085-0c9b-4eec-9516-74e139980355"}]: dispatch 2024-02-17T11:52:40.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:40 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/495818417' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fbef085-0c9b-4eec-9516-74e139980355"}]': finished 2024-02-17T11:52:40.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:40 smithi125 ceph-mon[36819]: osdmap e43: 7 total, 6 up, 7 in 2024-02-17T11:52:40.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:40 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:52:40.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:40 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/495818417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fbef085-0c9b-4eec-9516-74e139980355"}]: dispatch 2024-02-17T11:52:40.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:40 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/495818417' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fbef085-0c9b-4eec-9516-74e139980355"}]': finished 2024-02-17T11:52:40.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:40 smithi094 ceph-mon[32499]: osdmap e43: 7 total, 6 up, 7 in 2024-02-17T11:52:40.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:40 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:52:40.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:40 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/495818417' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fbef085-0c9b-4eec-9516-74e139980355"}]: dispatch 2024-02-17T11:52:40.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:40 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/495818417' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fbef085-0c9b-4eec-9516-74e139980355"}]': finished 2024-02-17T11:52:40.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:40 smithi189 ceph-mon[36588]: osdmap e43: 7 total, 6 up, 7 in 2024-02-17T11:52:40.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:40 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:52:41.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:41 smithi094 ceph-mon[32499]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-02-17T11:52:41.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:41 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/415294462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:52:41.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:41 smithi189 ceph-mon[36588]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-02-17T11:52:41.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:41 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/415294462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:52:41.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:41 smithi125 ceph-mon[36819]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-02-17T11:52:41.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:41 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/415294462' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:52:43.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:43 smithi189 ceph-mon[36588]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-17T11:52:43.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:43 smithi125 ceph-mon[36819]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-17T11:52:43.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:43 smithi094 ceph-mon[32499]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-17T11:52:45.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:45 smithi189 ceph-mon[36588]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-17T11:52:45.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:45 smithi125 ceph-mon[36819]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-17T11:52:45.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:45 smithi094 ceph-mon[32499]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-17T11:52:47.779 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:47 smithi094 ceph-mon[32499]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-17T11:52:47.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:47 smithi125 ceph-mon[36819]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-17T11:52:48.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:47 smithi189 ceph-mon[36588]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-17T11:52:48.678 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-17T11:52:48.678 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:48.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:48 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-17T11:52:48.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:48 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:49.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-17T11:52:49.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:52:49.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:49 smithi125 ceph-mon[36819]: Deploying daemon osd.6 on smithi094 2024-02-17T11:52:49.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:49 smithi125 ceph-mon[36819]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:49.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:49 smithi094 ceph-mon[32499]: Deploying daemon osd.6 on smithi094 2024-02-17T11:52:49.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:49 smithi094 ceph-mon[32499]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:49.998 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:49 smithi189 ceph-mon[36588]: Deploying daemon osd.6 on smithi094 2024-02-17T11:52:49.998 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:49 smithi189 ceph-mon[36588]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:51.854 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:51 smithi094 ceph-mon[32499]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:51.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:51 smithi125 ceph-mon[36819]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:52.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:51 smithi189 ceph-mon[36588]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:53.928 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:53 smithi094 ceph-mon[32499]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:53.928 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:53.928 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:53.928 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:53.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:53 smithi125 ceph-mon[36819]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:53.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:53.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:53.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:54.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:53 smithi189 ceph-mon[36588]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:54.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:54.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:54.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:52:55.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:55 smithi125 ceph-mon[36819]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:55.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:55 smithi094 ceph-mon[32499]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:56.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:55 smithi189 ceph-mon[36588]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:57.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:57.280+0000 7fcd2b7fe700 1 -- 172.21.15.94:0/1632345829 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcd3402d8e0 con 0x7fcd200a2150 2024-02-17T11:52:57.280 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 6 on host 'smithi094' 2024-02-17T11:52:57.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:57.282+0000 7fcd3c408700 1 -- 172.21.15.94:0/1632345829 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fcd200a2150 msgr2=0x7fcd200a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:52:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:57.282+0000 7fcd3c408700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fcd200a2150 0x7fcd200a4610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcd30006fd0 tx=0x7fcd30008040 comp rx=0 tx=0).stop 2024-02-17T11:52:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:57.282+0000 7fcd3c408700 1 -- 172.21.15.94:0/1632345829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34103840 msgr2=0x7fcd3419f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:52:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:57.282+0000 7fcd3c408700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34103840 0x7fcd3419f300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00b9a0 tx=0x7fcd2c00bd60 comp rx=0 tx=0).stop 2024-02-17T11:52:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:57.282+0000 7fcd3c408700 1 -- 172.21.15.94:0/1632345829 shutdown_connections 2024-02-17T11:52:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:57.282+0000 7fcd3c408700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fcd200a2150 0x7fcd200a4610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:57.282+0000 7fcd3c408700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcd34103840 0x7fcd3419f300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:57.282+0000 7fcd3c408700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcd34101ca0 0x7fcd34197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:57.282+0000 7fcd3c408700 1 --2- 172.21.15.94:0/1632345829 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd34102ea0 0x7fcd34198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:52:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:57.282+0000 7fcd3c408700 1 -- 172.21.15.94:0/1632345829 >> 172.21.15.94:0/1632345829 conn(0x7fcd340fd280 msgr2=0x7fcd341061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:52:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:57.283+0000 7fcd3c408700 1 -- 172.21.15.94:0/1632345829 shutdown_connections 2024-02-17T11:52:57.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:52:57.283+0000 7fcd3c408700 1 -- 172.21.15.94:0/1632345829 wait complete. 2024-02-17T11:52:57.910 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:57 smithi094 ceph-mon[32499]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:57.910 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:57.910 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:57.910 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:57 smithi094 ceph-mon[32499]: from='osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-17T11:52:57.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:57 smithi125 ceph-mon[36819]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:57.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:57.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:57.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:57 smithi125 ceph-mon[36819]: from='osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-17T11:52:57.938 DEBUG:teuthology.orchestra.run.smithi094:osd.6> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.6.service 2024-02-17T11:52:57.941 INFO:tasks.cephadm:Deploying osd.7 on smithi125 with /dev/vg_nvme/lv_2... 2024-02-17T11:52:57.941 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-17T11:52:58.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:57 smithi189 ceph-mon[36588]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:58.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:57 smithi189 ceph-mon[36588]: from='osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-17T11:52:58.190 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T11:52:59.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:59.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:59.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:59 smithi094 ceph-mon[32499]: from='osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-17T11:52:59.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:59 smithi094 ceph-mon[32499]: osdmap e44: 7 total, 6 up, 7 in 2024-02-17T11:52:59.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:59 smithi094 ceph-mon[32499]: from='osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-17T11:52:59.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:52:59.289 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:52:59 smithi094 ceph-mon[32499]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:59.289 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 11:52:59 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T11:52:59.281+0000 7f084e9a9700 -1 osd.6 0 waiting for initial osdmap 2024-02-17T11:52:59.289 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 11:52:59 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T11:52:59.289+0000 7f084a79c700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T11:52:59.447 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:59.447 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:59.447 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:59 smithi125 ceph-mon[36819]: from='osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-17T11:52:59.447 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:59 smithi125 ceph-mon[36819]: osdmap e44: 7 total, 6 up, 7 in 2024-02-17T11:52:59.448 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:59 smithi125 ceph-mon[36819]: from='osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-17T11:52:59.448 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:52:59.448 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:52:59 smithi125 ceph-mon[36819]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:52:59.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:59.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:52:59.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:59 smithi189 ceph-mon[36588]: from='osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-17T11:52:59.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:59 smithi189 ceph-mon[36588]: osdmap e44: 7 total, 6 up, 7 in 2024-02-17T11:52:59.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:59 smithi189 ceph-mon[36588]: from='osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-17T11:52:59.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:52:59.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:52:59 smithi189 ceph-mon[36588]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:53:00.359 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-17T11:53:00.359 INFO:teuthology.orchestra.run.smithi125.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-17T11:53:00.359 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 10+0 records in 2024-02-17T11:53:00.359 INFO:teuthology.orchestra.run.smithi125.stderr:10+0 records out 2024-02-17T11:53:00.360 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128006 s, 819 MB/s 2024-02-17T11:53:00.360 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping successful for: 2024-02-17T11:53:00.363 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:00 smithi094 ceph-mon[32499]: from='osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-17T11:53:00.363 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:00 smithi094 ceph-mon[32499]: osdmap e45: 7 total, 6 up, 7 in 2024-02-17T11:53:00.363 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:00 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:53:00.363 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:00 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:53:00.464 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:00 smithi125 ceph-mon[36819]: from='osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-17T11:53:00.465 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:00 smithi125 ceph-mon[36819]: osdmap e45: 7 total, 6 up, 7 in 2024-02-17T11:53:00.465 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:00 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:53:00.465 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:00 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:53:00.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:00 smithi189 ceph-mon[36588]: from='osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-17T11:53:00.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:00 smithi189 ceph-mon[36588]: osdmap e45: 7 total, 6 up, 7 in 2024-02-17T11:53:00.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:00 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:53:00.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:00 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:53:01.081 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi125:vg_nvme/lv_2 2024-02-17T11:53:01.320 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T11:53:01.417 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:01 smithi125 ceph-mon[36819]: purged_snaps scrub starts 2024-02-17T11:53:01.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:01 smithi125 ceph-mon[36819]: purged_snaps scrub ok 2024-02-17T11:53:01.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:01 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:53:01.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:01 smithi125 ceph-mon[36819]: osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558] boot 2024-02-17T11:53:01.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:01 smithi125 ceph-mon[36819]: osdmap e46: 7 total, 7 up, 7 in 2024-02-17T11:53:01.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:01 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:53:01.418 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:01 smithi125 ceph-mon[36819]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:53:01.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:01 smithi189 ceph-mon[36588]: purged_snaps scrub starts 2024-02-17T11:53:01.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:01 smithi189 ceph-mon[36588]: purged_snaps scrub ok 2024-02-17T11:53:01.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:01 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:53:01.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:01 smithi189 ceph-mon[36588]: osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558] boot 2024-02-17T11:53:01.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:01 smithi189 ceph-mon[36588]: osdmap e46: 7 total, 7 up, 7 in 2024-02-17T11:53:01.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:01 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:53:01.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:01 smithi189 ceph-mon[36588]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:53:01.684 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:01 smithi094 ceph-mon[32499]: purged_snaps scrub starts 2024-02-17T11:53:01.684 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:01 smithi094 ceph-mon[32499]: purged_snaps scrub ok 2024-02-17T11:53:01.684 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:01 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:53:01.685 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:01 smithi094 ceph-mon[32499]: osd.6 [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558] boot 2024-02-17T11:53:01.685 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:01 smithi094 ceph-mon[32499]: osdmap e46: 7 total, 7 up, 7 in 2024-02-17T11:53:01.685 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:01 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:53:01.685 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:01 smithi094 ceph-mon[32499]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-02-17T11:53:02.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: osdmap e47: 7 total, 7 up, 7 in 2024-02-17T11:53:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: osdmap e47: 7 total, 7 up, 7 in 2024-02-17T11:53:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:02.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:02.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:02.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:02.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: osdmap e47: 7 total, 7 up, 7 in 2024-02-17T11:53:02.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:02.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:02.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:02.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:02.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:02.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:02.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:02.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:02.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:02.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:03.008 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.008+0000 7f26e21f3700 1 -- 172.21.15.125:0/850677700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26dc102ea0 msgr2=0x7f26dc103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:53:03.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.008+0000 7f26e21f3700 1 --2- 172.21.15.125:0/850677700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26dc102ea0 0x7f26dc103300 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f26d00099c0 tx=0x7f26d0009cd0 comp rx=0 tx=0).stop 2024-02-17T11:53:03.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.008+0000 7f26e21f3700 1 -- 172.21.15.125:0/850677700 shutdown_connections 2024-02-17T11:53:03.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.008+0000 7f26e21f3700 1 --2- 172.21.15.125:0/850677700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26dc103840 0x7f26dc10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:03.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.008+0000 7f26e21f3700 1 --2- 172.21.15.125:0/850677700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26dc101ca0 0x7f26dc1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:03.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.008+0000 7f26e21f3700 1 --2- 172.21.15.125:0/850677700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26dc102ea0 0x7f26dc103300 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:03.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.008+0000 7f26e21f3700 1 -- 172.21.15.125:0/850677700 >> 172.21.15.125:0/850677700 conn(0x7f26dc0fd280 msgr2=0x7f26dc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:53:03.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.008+0000 7f26e21f3700 1 -- 172.21.15.125:0/850677700 shutdown_connections 2024-02-17T11:53:03.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.008+0000 7f26e21f3700 1 -- 172.21.15.125:0/850677700 wait complete. 2024-02-17T11:53:03.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.009+0000 7f26e21f3700 1 Processor -- start 2024-02-17T11:53:03.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.009+0000 7f26e21f3700 1 -- start start 2024-02-17T11:53:03.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.009+0000 7f26e21f3700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26dc101ca0 0x7f26dc197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:53:03.009 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.009+0000 7f26e21f3700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26dc102ea0 0x7f26dc198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:53:03.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.009+0000 7f26e21f3700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26dc103840 0x7f26dc19f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:53:03.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.009+0000 7f26e21f3700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc19f6e0 con 0x7f26dc103840 2024-02-17T11:53:03.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.009+0000 7f26e21f3700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc19f820 con 0x7f26dc101ca0 2024-02-17T11:53:03.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.009+0000 7f26e21f3700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26dc19f9b0 con 0x7f26dc102ea0 2024-02-17T11:53:03.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26daffd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26dc102ea0 0x7f26dc198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:53:03.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26daffd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26dc102ea0 0x7f26dc198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.125:56252/0 (socket says 172.21.15.125:56252) 2024-02-17T11:53:03.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26daffd700 1 -- 172.21.15.125:0/2226031293 learned_addr learned my addr 172.21.15.125:0/2226031293 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-17T11:53:03.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26dbfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26dc103840 0x7f26dc19f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:53:03.010 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26db7fe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26dc101ca0 0x7f26dc197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:53:03.011 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26db7fe700 1 -- 172.21.15.125:0/2226031293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26dc102ea0 msgr2=0x7f26dc198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:53:03.011 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26db7fe700 1 --2- 172.21.15.125:0/2226031293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26dc102ea0 0x7f26dc198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:03.011 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26db7fe700 1 -- 172.21.15.125:0/2226031293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26dc103840 msgr2=0x7f26dc19f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:53:03.011 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26db7fe700 1 --2- 172.21.15.125:0/2226031293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26dc103840 0x7f26dc19f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:03.011 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26db7fe700 1 -- 172.21.15.125:0/2226031293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d00095f0 con 0x7f26dc101ca0 2024-02-17T11:53:03.011 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26db7fe700 1 --2- 172.21.15.125:0/2226031293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26dc101ca0 0x7f26dc197b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00c940 tx=0x7f26cc00cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:53:03.011 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26d8ff9700 1 -- 172.21.15.125:0/2226031293 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc004230 con 0x7f26dc101ca0 2024-02-17T11:53:03.012 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26d8ff9700 1 -- 172.21.15.125:0/2226031293 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f26cc004390 con 0x7f26dc101ca0 2024-02-17T11:53:03.012 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26e21f3700 1 -- 172.21.15.125:0/2226031293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26dc19fc90 con 0x7f26dc101ca0 2024-02-17T11:53:03.012 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26d8ff9700 1 -- 172.21.15.125:0/2226031293 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc003bc0 con 0x7f26dc101ca0 2024-02-17T11:53:03.012 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.010+0000 7f26e21f3700 1 -- 172.21.15.125:0/2226031293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26dc1a0190 con 0x7f26dc101ca0 2024-02-17T11:53:03.012 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.011+0000 7f26e21f3700 1 -- 172.21.15.125:0/2226031293 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26dc109000 con 0x7f26dc101ca0 2024-02-17T11:53:03.012 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.013+0000 7f26d8ff9700 1 -- 172.21.15.125:0/2226031293 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f26cc003d40 con 0x7f26dc101ca0 2024-02-17T11:53:03.013 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.014+0000 7f26d8ff9700 1 --2- 172.21.15.125:0/2226031293 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f26c40a21d0 0x7f26c40a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:53:03.013 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.014+0000 7f26d8ff9700 1 -- 172.21.15.125:0/2226031293 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f26cc013070 con 0x7f26dc101ca0 2024-02-17T11:53:03.013 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.014+0000 7f26daffd700 1 --2- 172.21.15.125:0/2226031293 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f26c40a21d0 0x7f26c40a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:53:03.013 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.015+0000 7f26daffd700 1 --2- 172.21.15.125:0/2226031293 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f26c40a21d0 0x7f26c40a4690 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f26d00038c0 tx=0x7f26d0005320 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:53:03.016 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.017+0000 7f26d8ff9700 1 -- 172.21.15.125:0/2226031293 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f26cc096c10 con 0x7f26dc101ca0 2024-02-17T11:53:03.214 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:03.215+0000 7f26e21f3700 1 -- 172.21.15.125:0/2226031293 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f26dc02d8e0 con 0x7f26c40a21d0 2024-02-17T11:53:03.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:03 smithi189 ceph-mon[36588]: Detected new or changed devices on smithi094 2024-02-17T11:53:03.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:03 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi094 to 5696M 2024-02-17T11:53:03.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:03 smithi189 ceph-mon[36588]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:03.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:03 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:53:03.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:03 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:53:03.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:03 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:03.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:03 smithi125 ceph-mon[36819]: Detected new or changed devices on smithi094 2024-02-17T11:53:03.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:03 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi094 to 5696M 2024-02-17T11:53:03.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:03 smithi125 ceph-mon[36819]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:03.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:03 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:53:03.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:03 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:53:03.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:03 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:03.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:03 smithi094 ceph-mon[32499]: Detected new or changed devices on smithi094 2024-02-17T11:53:03.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:03 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi094 to 5696M 2024-02-17T11:53:03.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:03 smithi094 ceph-mon[32499]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:03.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:03 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:53:03.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:03 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:53:03.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:03 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:04.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:04 smithi125 ceph-mon[36819]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:53:04.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:04 smithi189 ceph-mon[36588]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:53:04.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:04 smithi094 ceph-mon[32499]: from='client.24260 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:53:05.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:05 smithi125 ceph-mon[36819]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:05.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:05 smithi189 ceph-mon[36588]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:05.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:05 smithi094 ceph-mon[32499]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:06.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:06 smithi189 ceph-mon[36588]: from='client.? 172.21.15.125:0/2041779201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c76060b-420b-47f4-b746-75064957a855"}]: dispatch 2024-02-17T11:53:06.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:06 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c76060b-420b-47f4-b746-75064957a855"}]: dispatch 2024-02-17T11:53:06.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:06 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c76060b-420b-47f4-b746-75064957a855"}]': finished 2024-02-17T11:53:06.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:06 smithi189 ceph-mon[36588]: osdmap e48: 8 total, 7 up, 8 in 2024-02-17T11:53:06.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:06 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:06.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:06 smithi125 ceph-mon[36819]: from='client.? 172.21.15.125:0/2041779201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c76060b-420b-47f4-b746-75064957a855"}]: dispatch 2024-02-17T11:53:06.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:06 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c76060b-420b-47f4-b746-75064957a855"}]: dispatch 2024-02-17T11:53:06.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:06 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c76060b-420b-47f4-b746-75064957a855"}]': finished 2024-02-17T11:53:06.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:06 smithi125 ceph-mon[36819]: osdmap e48: 8 total, 7 up, 8 in 2024-02-17T11:53:06.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:06 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:06.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:06 smithi094 ceph-mon[32499]: from='client.? 172.21.15.125:0/2041779201' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c76060b-420b-47f4-b746-75064957a855"}]: dispatch 2024-02-17T11:53:06.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:06 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c76060b-420b-47f4-b746-75064957a855"}]: dispatch 2024-02-17T11:53:06.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:06 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c76060b-420b-47f4-b746-75064957a855"}]': finished 2024-02-17T11:53:06.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:06 smithi094 ceph-mon[32499]: osdmap e48: 8 total, 7 up, 8 in 2024-02-17T11:53:06.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:06 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:07.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:07 smithi189 ceph-mon[36588]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:07.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:07 smithi189 ceph-mon[36588]: from='client.? 172.21.15.125:0/2086128290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:53:07.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:07 smithi125 ceph-mon[36819]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:07.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:07 smithi125 ceph-mon[36819]: from='client.? 172.21.15.125:0/2086128290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:53:07.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:07 smithi094 ceph-mon[32499]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:07.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:07 smithi094 ceph-mon[32499]: from='client.? 172.21.15.125:0/2086128290' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:53:09.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:09 smithi125 ceph-mon[36819]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:09.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:09 smithi094 ceph-mon[32499]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:10.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:09 smithi189 ceph-mon[36588]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:11.912 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:11 smithi125 ceph-mon[36819]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:11.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:11 smithi094 ceph-mon[32499]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:12.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:11 smithi189 ceph-mon[36588]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:13 smithi125 ceph-mon[36819]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-17T11:53:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:13 smithi094 ceph-mon[32499]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-17T11:53:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:14.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:13 smithi189 ceph-mon[36588]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:14.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-17T11:53:14.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:14.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:14 smithi125 ceph-mon[36819]: Deploying daemon osd.7 on smithi125 2024-02-17T11:53:14.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:14 smithi094 ceph-mon[32499]: Deploying daemon osd.7 on smithi125 2024-02-17T11:53:15.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:14 smithi189 ceph-mon[36588]: Deploying daemon osd.7 on smithi125 2024-02-17T11:53:15.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:15 smithi125 ceph-mon[36819]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:15 smithi094 ceph-mon[32499]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:16.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:15 smithi189 ceph-mon[36588]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:17.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:17 smithi125 ceph-mon[36819]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:17.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:17 smithi094 ceph-mon[32499]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:18.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:17 smithi189 ceph-mon[36588]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:19.790 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:19 smithi125 ceph-mon[36819]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:19.790 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:19 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:19.790 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:19 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:19.790 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:19 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:19 smithi094 ceph-mon[32499]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:19 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:19 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:19 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:20.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:19 smithi189 ceph-mon[36588]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:20.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:19 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:20.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:19 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:20.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:19 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:21.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:21 smithi125 ceph-mon[36819]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:21.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:21 smithi094 ceph-mon[32499]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:22.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:21 smithi189 ceph-mon[36588]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:22.327 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:22.328+0000 7f26d8ff9700 1 -- 172.21.15.125:0/2226031293 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f26dc02d8e0 con 0x7f26c40a21d0 2024-02-17T11:53:22.328 INFO:teuthology.orchestra.run.smithi125.stdout:Created osd(s) 7 on host 'smithi125' 2024-02-17T11:53:22.330 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:22.330+0000 7f26e21f3700 1 -- 172.21.15.125:0/2226031293 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f26c40a21d0 msgr2=0x7f26c40a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:53:22.330 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:22.330+0000 7f26e21f3700 1 --2- 172.21.15.125:0/2226031293 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f26c40a21d0 0x7f26c40a4690 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f26d00038c0 tx=0x7f26d0005320 comp rx=0 tx=0).stop 2024-02-17T11:53:22.330 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:22.330+0000 7f26e21f3700 1 -- 172.21.15.125:0/2226031293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26dc101ca0 msgr2=0x7f26dc197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:53:22.330 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:22.330+0000 7f26e21f3700 1 --2- 172.21.15.125:0/2226031293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26dc101ca0 0x7f26dc197b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00c940 tx=0x7f26cc00cc50 comp rx=0 tx=0).stop 2024-02-17T11:53:22.330 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:22.331+0000 7f26e21f3700 1 -- 172.21.15.125:0/2226031293 shutdown_connections 2024-02-17T11:53:22.330 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:22.331+0000 7f26e21f3700 1 --2- 172.21.15.125:0/2226031293 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f26c40a21d0 0x7f26c40a4690 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:22.330 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:22.331+0000 7f26e21f3700 1 --2- 172.21.15.125:0/2226031293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f26dc103840 0x7f26dc19f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:22.330 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:22.331+0000 7f26e21f3700 1 --2- 172.21.15.125:0/2226031293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f26dc101ca0 0x7f26dc197b50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:22.330 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:22.331+0000 7f26e21f3700 1 --2- 172.21.15.125:0/2226031293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26dc102ea0 0x7f26dc198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:22.331 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:22.331+0000 7f26e21f3700 1 -- 172.21.15.125:0/2226031293 >> 172.21.15.125:0/2226031293 conn(0x7f26dc0fd280 msgr2=0x7f26dc106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:53:22.331 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:22.331+0000 7f26e21f3700 1 -- 172.21.15.125:0/2226031293 shutdown_connections 2024-02-17T11:53:22.331 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:53:22.331+0000 7f26e21f3700 1 -- 172.21.15.125:0/2226031293 wait complete. 2024-02-17T11:53:23.203 DEBUG:teuthology.orchestra.run.smithi125:osd.7> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.7.service 2024-02-17T11:53:23.205 INFO:tasks.cephadm:Deploying osd.8 on smithi189 with /dev/vg_nvme/lv_2... 2024-02-17T11:53:23.205 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-17T11:53:23.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:23 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:23.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:23 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:23.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:23 smithi125 ceph-mon[36819]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:23.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:23 smithi125 ceph-mon[36819]: from='osd.7 [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-17T11:53:23.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:23 smithi125 ceph-mon[36819]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-17T11:53:23.453 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T11:53:23.575 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:23 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:23.575 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:23 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:23.575 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:23 smithi189 ceph-mon[36588]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:23.575 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:23 smithi189 ceph-mon[36588]: from='osd.7 [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-17T11:53:23.575 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:23 smithi189 ceph-mon[36588]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-17T11:53:23.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:23 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:23.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:23 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:23.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:23 smithi094 ceph-mon[32499]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:23.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:23 smithi094 ceph-mon[32499]: from='osd.7 [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-17T11:53:23.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:23 smithi094 ceph-mon[32499]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-17T11:53:24.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:24 smithi189 ceph-mon[36588]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-17T11:53:24.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:24 smithi189 ceph-mon[36588]: osdmap e49: 8 total, 7 up, 8 in 2024-02-17T11:53:24.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:24 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:24.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:24 smithi189 ceph-mon[36588]: from='osd.7 [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:53:24.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:24 smithi189 ceph-mon[36588]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:53:24.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:24 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:24.600 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:24 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:24.685 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 11:53:24 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T11:53:24.330+0000 7f378391d700 -1 osd.7 0 waiting for initial osdmap 2024-02-17T11:53:24.685 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 11:53:24 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T11:53:24.338+0000 7f377f710700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T11:53:24.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:24 smithi125 ceph-mon[36819]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-17T11:53:24.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:24 smithi125 ceph-mon[36819]: osdmap e49: 8 total, 7 up, 8 in 2024-02-17T11:53:24.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:24 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:24.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:24 smithi125 ceph-mon[36819]: from='osd.7 [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:53:24.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:24 smithi125 ceph-mon[36819]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:53:24.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:24 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:24.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:24 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:24.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:24 smithi094 ceph-mon[32499]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-17T11:53:24.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:24 smithi094 ceph-mon[32499]: osdmap e49: 8 total, 7 up, 8 in 2024-02-17T11:53:24.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:24 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:24.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:24 smithi094 ceph-mon[32499]: from='osd.7 [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:53:24.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:24 smithi094 ceph-mon[32499]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:53:24.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:24 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:24.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:24 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:25.495 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-17T11:53:25.495 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-17T11:53:25.495 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-02-17T11:53:25.495 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-02-17T11:53:25.495 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0148594 s, 706 MB/s 2024-02-17T11:53:25.496 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-02-17T11:53:25.596 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:25 smithi125 ceph-mon[36819]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2024-02-17T11:53:25.596 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:25 smithi125 ceph-mon[36819]: osdmap e50: 8 total, 7 up, 8 in 2024-02-17T11:53:25.596 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:25.597 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:25.597 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:25 smithi125 ceph-mon[36819]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:25.597 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:25 smithi125 ceph-mon[36819]: osd.7 [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517] boot 2024-02-17T11:53:25.597 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:25 smithi125 ceph-mon[36819]: osdmap e51: 8 total, 8 up, 8 in 2024-02-17T11:53:25.597 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:25.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:25 smithi189 ceph-mon[36588]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2024-02-17T11:53:25.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:25 smithi189 ceph-mon[36588]: osdmap e50: 8 total, 7 up, 8 in 2024-02-17T11:53:25.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:25.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:25.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:25 smithi189 ceph-mon[36588]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:25.608 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:25 smithi189 ceph-mon[36588]: osd.7 [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517] boot 2024-02-17T11:53:25.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:25 smithi189 ceph-mon[36588]: osdmap e51: 8 total, 8 up, 8 in 2024-02-17T11:53:25.609 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:25.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:25 smithi094 ceph-mon[32499]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2024-02-17T11:53:25.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:25 smithi094 ceph-mon[32499]: osdmap e50: 8 total, 7 up, 8 in 2024-02-17T11:53:25.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:25.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:25.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:25 smithi094 ceph-mon[32499]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-02-17T11:53:25.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:25 smithi094 ceph-mon[32499]: osd.7 [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517] boot 2024-02-17T11:53:25.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:25 smithi094 ceph-mon[32499]: osdmap e51: 8 total, 8 up, 8 in 2024-02-17T11:53:25.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:53:26.052 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi189:vg_nvme/lv_2 2024-02-17T11:53:26.294 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T11:53:26.624 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:26 smithi125 ceph-mon[36819]: purged_snaps scrub starts 2024-02-17T11:53:26.624 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:26 smithi125 ceph-mon[36819]: purged_snaps scrub ok 2024-02-17T11:53:26.624 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:26 smithi125 ceph-mon[36819]: osdmap e52: 8 total, 8 up, 8 in 2024-02-17T11:53:26.634 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:26 smithi189 ceph-mon[36588]: purged_snaps scrub starts 2024-02-17T11:53:26.634 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:26 smithi189 ceph-mon[36588]: purged_snaps scrub ok 2024-02-17T11:53:26.634 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:26 smithi189 ceph-mon[36588]: osdmap e52: 8 total, 8 up, 8 in 2024-02-17T11:53:26.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:26 smithi094 ceph-mon[32499]: purged_snaps scrub starts 2024-02-17T11:53:26.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:26 smithi094 ceph-mon[32499]: purged_snaps scrub ok 2024-02-17T11:53:26.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:26 smithi094 ceph-mon[32499]: osdmap e52: 8 total, 8 up, 8 in 2024-02-17T11:53:27.498 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: pgmap v176: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:27.498 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.498 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.498 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:27.498 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:27.498 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:27.498 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.498 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:27.498 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:27.498 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.498 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: osdmap e53: 8 total, 8 up, 8 in 2024-02-17T11:53:27.499 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:27.499 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:27.499 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:27.499 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: pgmap v176: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:27.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:27.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:27.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:27.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:27.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:27.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: osdmap e53: 8 total, 8 up, 8 in 2024-02-17T11:53:27.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:27.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:27.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:27.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: pgmap v176: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:27.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:27.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:27.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:27.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:27.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:27.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: osdmap e53: 8 total, 8 up, 8 in 2024-02-17T11:53:27.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:27.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:27.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:27.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:27.965 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.963+0000 7f0787897700 1 -- 172.21.15.189:0/1260827852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0780101e10 msgr2=0x7f0780102270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:53:27.965 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.964+0000 7f0787897700 1 --2- 172.21.15.189:0/1260827852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0780101e10 0x7f0780102270 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0770009990 tx=0x7f0770009ca0 comp rx=0 tx=0).stop 2024-02-17T11:53:27.965 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.964+0000 7f0787897700 1 -- 172.21.15.189:0/1260827852 shutdown_connections 2024-02-17T11:53:27.965 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.964+0000 7f0787897700 1 --2- 172.21.15.189:0/1260827852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07801027b0 0x7f0780109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:27.965 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.964+0000 7f0787897700 1 --2- 172.21.15.189:0/1260827852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0780100c10 0x7f0780101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:27.965 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.964+0000 7f0787897700 1 --2- 172.21.15.189:0/1260827852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0780101e10 0x7f0780102270 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:27.965 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.964+0000 7f0787897700 1 -- 172.21.15.189:0/1260827852 >> 172.21.15.189:0/1260827852 conn(0x7f07800fc1d0 msgr2=0x7f07800fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:53:27.965 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.964+0000 7f0787897700 1 -- 172.21.15.189:0/1260827852 shutdown_connections 2024-02-17T11:53:27.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.964+0000 7f0787897700 1 -- 172.21.15.189:0/1260827852 wait complete. 2024-02-17T11:53:27.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.964+0000 7f0787897700 1 Processor -- start 2024-02-17T11:53:27.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.964+0000 7f0787897700 1 -- start start 2024-02-17T11:53:27.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0787897700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0780100c10 0x7f0780196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:53:27.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0787897700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0780101e10 0x7f0780197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:53:27.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0787897700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07801027b0 0x7f078019e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:53:27.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0787897700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078019e690 con 0x7f0780101e10 2024-02-17T11:53:27.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0787897700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078019e7d0 con 0x7f07801027b0 2024-02-17T11:53:27.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0787897700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078019e960 con 0x7f0780100c10 2024-02-17T11:53:27.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0785633700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0780100c10 0x7f0780196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:53:27.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0785633700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0780100c10 0x7f0780196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:48652/0 (socket says 172.21.15.189:48652) 2024-02-17T11:53:27.966 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0785633700 1 -- 172.21.15.189:0/1261273731 learned_addr learned my addr 172.21.15.189:0/1261273731 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-02-17T11:53:27.967 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0784e32700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0780101e10 0x7f0780197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:53:27.967 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0785e34700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07801027b0 0x7f078019e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:53:27.967 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0785633700 1 -- 172.21.15.189:0/1261273731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07801027b0 msgr2=0x7f078019e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:53:27.967 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0785633700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07801027b0 0x7f078019e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:27.967 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0785633700 1 -- 172.21.15.189:0/1261273731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0780101e10 msgr2=0x7f0780197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:53:27.967 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0785633700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0780101e10 0x7f0780197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:27.967 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0785633700 1 -- 172.21.15.189:0/1261273731 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07700095f0 con 0x7f0780100c10 2024-02-17T11:53:27.967 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0785633700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0780100c10 0x7f0780196b00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f077c009920 tx=0x7f077c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:53:27.967 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f07767fc700 1 -- 172.21.15.189:0/1261273731 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c012070 con 0x7f0780100c10 2024-02-17T11:53:27.967 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0787897700 1 -- 172.21.15.189:0/1261273731 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078006a3d0 con 0x7f0780100c10 2024-02-17T11:53:27.967 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f07767fc700 1 -- 172.21.15.189:0/1261273731 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f077c004500 con 0x7f0780100c10 2024-02-17T11:53:27.967 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f07767fc700 1 -- 172.21.15.189:0/1261273731 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c0074c0 con 0x7f0780100c10 2024-02-17T11:53:27.967 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.965+0000 7f0787897700 1 -- 172.21.15.189:0/1261273731 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078006a8d0 con 0x7f0780100c10 2024-02-17T11:53:27.968 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.966+0000 7f0787897700 1 -- 172.21.15.189:0/1261273731 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0780068930 con 0x7f0780100c10 2024-02-17T11:53:27.968 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.967+0000 7f07767fc700 1 -- 172.21.15.189:0/1261273731 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f077c016030 con 0x7f0780100c10 2024-02-17T11:53:27.968 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.968+0000 7f07767fc700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f076c0a2150 0x7f076c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:53:27.969 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.968+0000 7f07767fc700 1 -- 172.21.15.189:0/1261273731 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f077c0ce5f0 con 0x7f0780100c10 2024-02-17T11:53:27.969 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.968+0000 7f0784e32700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f076c0a2150 0x7f076c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:53:27.969 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.969+0000 7f0784e32700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f076c0a2150 0x7f076c0a4610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f077000b5c0 tx=0x7f0770005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:53:27.970 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:27.971+0000 7f07767fc700 1 -- 172.21.15.189:0/1261273731 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f077c098100 con 0x7f0780100c10 2024-02-17T11:53:28.175 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:28.176+0000 7f0787897700 1 -- 172.21.15.189:0/1261273731 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0780107c00 con 0x7f076c0a2150 2024-02-17T11:53:28.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:28 smithi189 ceph-mon[36588]: Detected new or changed devices on smithi125 2024-02-17T11:53:28.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:28 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi125 to 5693M 2024-02-17T11:53:28.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:28 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:53:28.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:28 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:53:28.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:28 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:28.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:28 smithi125 ceph-mon[36819]: Detected new or changed devices on smithi125 2024-02-17T11:53:28.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:28 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi125 to 5693M 2024-02-17T11:53:28.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:28 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:53:28.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:28 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:53:28.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:28 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:28.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:28 smithi094 ceph-mon[32499]: Detected new or changed devices on smithi125 2024-02-17T11:53:28.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:28 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi125 to 5693M 2024-02-17T11:53:28.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:28 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:53:28.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:28 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:53:28.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:28 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:29.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:29 smithi189 ceph-mon[36588]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:53:29.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:29 smithi189 ceph-mon[36588]: osdmap e54: 8 total, 8 up, 8 in 2024-02-17T11:53:29.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:29 smithi189 ceph-mon[36588]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:29.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:29 smithi125 ceph-mon[36819]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:53:29.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:29 smithi125 ceph-mon[36819]: osdmap e54: 8 total, 8 up, 8 in 2024-02-17T11:53:29.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:29 smithi125 ceph-mon[36819]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:29.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:29 smithi094 ceph-mon[32499]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:53:29.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:29 smithi094 ceph-mon[32499]: osdmap e54: 8 total, 8 up, 8 in 2024-02-17T11:53:29.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:29 smithi094 ceph-mon[32499]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:30.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:30 smithi189 ceph-mon[36588]: osdmap e55: 8 total, 8 up, 8 in 2024-02-17T11:53:30.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:30 smithi125 ceph-mon[36819]: osdmap e55: 8 total, 8 up, 8 in 2024-02-17T11:53:30.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:30 smithi094 ceph-mon[32499]: osdmap e55: 8 total, 8 up, 8 in 2024-02-17T11:53:31.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:31 smithi189 ceph-mon[36588]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:31.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:31 smithi189 ceph-mon[36588]: from='client.? 172.21.15.189:0/2271961948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f6c6b96-ee0a-48c7-abfd-e73733791afc"}]: dispatch 2024-02-17T11:53:31.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:31 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f6c6b96-ee0a-48c7-abfd-e73733791afc"}]: dispatch 2024-02-17T11:53:31.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:31 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f6c6b96-ee0a-48c7-abfd-e73733791afc"}]': finished 2024-02-17T11:53:31.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:31 smithi189 ceph-mon[36588]: osdmap e56: 9 total, 8 up, 9 in 2024-02-17T11:53:31.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:31 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:31.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:31 smithi125 ceph-mon[36819]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:31.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:31 smithi125 ceph-mon[36819]: from='client.? 172.21.15.189:0/2271961948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f6c6b96-ee0a-48c7-abfd-e73733791afc"}]: dispatch 2024-02-17T11:53:31.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:31 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f6c6b96-ee0a-48c7-abfd-e73733791afc"}]: dispatch 2024-02-17T11:53:31.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:31 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f6c6b96-ee0a-48c7-abfd-e73733791afc"}]': finished 2024-02-17T11:53:31.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:31 smithi125 ceph-mon[36819]: osdmap e56: 9 total, 8 up, 9 in 2024-02-17T11:53:31.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:31 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:31.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:31 smithi094 ceph-mon[32499]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:31.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:31 smithi094 ceph-mon[32499]: from='client.? 172.21.15.189:0/2271961948' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f6c6b96-ee0a-48c7-abfd-e73733791afc"}]: dispatch 2024-02-17T11:53:31.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:31 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f6c6b96-ee0a-48c7-abfd-e73733791afc"}]: dispatch 2024-02-17T11:53:31.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:31 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f6c6b96-ee0a-48c7-abfd-e73733791afc"}]': finished 2024-02-17T11:53:31.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:31 smithi094 ceph-mon[32499]: osdmap e56: 9 total, 8 up, 9 in 2024-02-17T11:53:31.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:31 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:32.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:32 smithi189 ceph-mon[36588]: from='client.? 172.21.15.189:0/2230158410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:53:32.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:32 smithi125 ceph-mon[36819]: from='client.? 172.21.15.189:0/2230158410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:53:32.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:32 smithi094 ceph-mon[32499]: from='client.? 172.21.15.189:0/2230158410' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:53:33.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:33 smithi189 ceph-mon[36588]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:33.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:33 smithi125 ceph-mon[36819]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:33.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:33 smithi094 ceph-mon[32499]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:35.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:35 smithi125 ceph-mon[36819]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:35.953 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:35 smithi189 ceph-mon[36588]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:35.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:35 smithi094 ceph-mon[32499]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:37.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:37 smithi125 ceph-mon[36819]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:37.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:37 smithi094 ceph-mon[32499]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:37.989 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:37 smithi189 ceph-mon[36588]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:38.892 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:38 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-17T11:53:38.892 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:38 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:38.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:38 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-17T11:53:38.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:38 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:38.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:38 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-17T11:53:38.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:38 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:39.866 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:39 smithi189 ceph-mon[36588]: Deploying daemon osd.8 on smithi189 2024-02-17T11:53:39.866 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:39 smithi189 ceph-mon[36588]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:39.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:39 smithi125 ceph-mon[36819]: Deploying daemon osd.8 on smithi189 2024-02-17T11:53:39.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:39 smithi125 ceph-mon[36819]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:39.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:39 smithi094 ceph-mon[32499]: Deploying daemon osd.8 on smithi189 2024-02-17T11:53:39.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:39 smithi094 ceph-mon[32499]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:41.874 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:41 smithi189 ceph-mon[36588]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:41.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:41 smithi125 ceph-mon[36819]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:41 smithi094 ceph-mon[32499]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:43.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:43 smithi125 ceph-mon[36819]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:43.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:43 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:43.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:43 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:43.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:43 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:43.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:43 smithi094 ceph-mon[32499]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:43.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:43 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:43.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:43 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:43.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:43 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:44.103 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:43 smithi189 ceph-mon[36588]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:44.104 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:44.104 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:44.104 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:45.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:45 smithi125 ceph-mon[36819]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:45.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:45 smithi094 ceph-mon[32499]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:45.992 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:45 smithi189 ceph-mon[36588]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:47.194 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 8 on host 'smithi189' 2024-02-17T11:53:47.194 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:47.194+0000 7f07767fc700 1 -- 172.21.15.189:0/1261273731 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0780107c00 con 0x7f076c0a2150 2024-02-17T11:53:47.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:47.196+0000 7f0787897700 1 -- 172.21.15.189:0/1261273731 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f076c0a2150 msgr2=0x7f076c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:53:47.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:47.196+0000 7f0787897700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f076c0a2150 0x7f076c0a4610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f077000b5c0 tx=0x7f0770005fd0 comp rx=0 tx=0).stop 2024-02-17T11:53:47.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:47.197+0000 7f0787897700 1 -- 172.21.15.189:0/1261273731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0780100c10 msgr2=0x7f0780196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:53:47.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:47.197+0000 7f0787897700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0780100c10 0x7f0780196b00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f077c009920 tx=0x7f077c009c30 comp rx=0 tx=0).stop 2024-02-17T11:53:47.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:47.197+0000 7f0787897700 1 -- 172.21.15.189:0/1261273731 shutdown_connections 2024-02-17T11:53:47.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:47.197+0000 7f0787897700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f076c0a2150 0x7f076c0a4610 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:47.196 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:47.197+0000 7f0787897700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07801027b0 0x7f078019e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:47.197 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:47.197+0000 7f0787897700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0780100c10 0x7f0780196b00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:47.197 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:47.197+0000 7f0787897700 1 --2- 172.21.15.189:0/1261273731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0780101e10 0x7f0780197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:47.197 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:47.197+0000 7f0787897700 1 -- 172.21.15.189:0/1261273731 >> 172.21.15.189:0/1261273731 conn(0x7f07800fc1d0 msgr2=0x7f07801051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:53:47.197 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:47.197+0000 7f0787897700 1 -- 172.21.15.189:0/1261273731 shutdown_connections 2024-02-17T11:53:47.197 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:53:47.197+0000 7f0787897700 1 -- 172.21.15.189:0/1261273731 wait complete. 2024-02-17T11:53:47.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:47.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:47.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:47 smithi189 ceph-mon[36588]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:47.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:47.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:47.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:47.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:47 smithi125 ceph-mon[36819]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:47.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:47.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:47.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:47.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:47 smithi094 ceph-mon[32499]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:47.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:47.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:53:47.828 DEBUG:teuthology.orchestra.run.smithi189:osd.8> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.8.service 2024-02-17T11:53:47.831 INFO:tasks.cephadm:Deploying osd.9 on smithi094 with /dev/vg_nvme/lv_1... 2024-02-17T11:53:47.831 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-17T11:53:48.079 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:53:48.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:48 smithi094 ceph-mon[32499]: from='osd.8 [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-17T11:53:48.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:48 smithi094 ceph-mon[32499]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-17T11:53:48.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:48 smithi189 ceph-mon[36588]: from='osd.8 [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-17T11:53:48.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:48 smithi189 ceph-mon[36588]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-17T11:53:48.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:48 smithi125 ceph-mon[36819]: from='osd.8 [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-17T11:53:48.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:48 smithi125 ceph-mon[36819]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-17T11:53:49.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:49 smithi094 ceph-mon[32499]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-17T11:53:49.514 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:49 smithi094 ceph-mon[32499]: osdmap e57: 9 total, 8 up, 9 in 2024-02-17T11:53:49.515 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:49 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:49.515 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:49 smithi094 ceph-mon[32499]: from='osd.8 [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:53:49.515 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:49 smithi094 ceph-mon[32499]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:53:49.515 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:49 smithi094 ceph-mon[32499]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:49.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:49 smithi189 ceph-mon[36588]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-17T11:53:49.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:49 smithi189 ceph-mon[36588]: osdmap e57: 9 total, 8 up, 9 in 2024-02-17T11:53:49.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:49 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:49.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:49 smithi189 ceph-mon[36588]: from='osd.8 [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:53:49.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:49 smithi189 ceph-mon[36588]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:53:49.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:49 smithi189 ceph-mon[36588]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:49.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:49 smithi125 ceph-mon[36819]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-17T11:53:49.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:49 smithi125 ceph-mon[36819]: osdmap e57: 9 total, 8 up, 9 in 2024-02-17T11:53:49.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:49 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:49.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:49 smithi125 ceph-mon[36819]: from='osd.8 [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:53:49.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:49 smithi125 ceph-mon[36819]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:53:49.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:49 smithi125 ceph-mon[36819]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-02-17T11:53:50.151 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 11:53:49 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T11:53:49.882+0000 7fe8f54fd700 -1 osd.8 0 waiting for initial osdmap 2024-02-17T11:53:50.151 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 11:53:49 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T11:53:49.892+0000 7fe8ed2e8700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T11:53:50.273 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-17T11:53:50.273 INFO:teuthology.orchestra.run.smithi094.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-17T11:53:50.273 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10+0 records in 2024-02-17T11:53:50.273 INFO:teuthology.orchestra.run.smithi094.stderr:10+0 records out 2024-02-17T11:53:50.273 INFO:teuthology.orchestra.run.smithi094.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00858199 s, 1.2 GB/s 2024-02-17T11:53:50.273 INFO:teuthology.orchestra.run.smithi094.stderr:--> Zapping successful for: 2024-02-17T11:53:50.503 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:50 smithi189 ceph-mon[36588]: purged_snaps scrub starts 2024-02-17T11:53:50.503 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:50 smithi189 ceph-mon[36588]: purged_snaps scrub ok 2024-02-17T11:53:50.504 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:50 smithi189 ceph-mon[36588]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-17T11:53:50.504 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:50 smithi189 ceph-mon[36588]: osdmap e58: 9 total, 8 up, 9 in 2024-02-17T11:53:50.504 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:50 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:50.504 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:50 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:50.504 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:50 smithi189 ceph-mon[36588]: from='osd.8 ' entity='osd.8' 2024-02-17T11:53:50.504 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:50 smithi189 ceph-mon[36588]: osd.8 [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253] boot 2024-02-17T11:53:50.504 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:50 smithi189 ceph-mon[36588]: osdmap e59: 9 total, 9 up, 9 in 2024-02-17T11:53:50.504 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:50 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:50.575 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:50 smithi094 ceph-mon[32499]: purged_snaps scrub starts 2024-02-17T11:53:50.575 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:50 smithi094 ceph-mon[32499]: purged_snaps scrub ok 2024-02-17T11:53:50.576 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:50 smithi094 ceph-mon[32499]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-17T11:53:50.576 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:50 smithi094 ceph-mon[32499]: osdmap e58: 9 total, 8 up, 9 in 2024-02-17T11:53:50.576 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:50 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:50.576 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:50 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:50.576 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:50 smithi094 ceph-mon[32499]: from='osd.8 ' entity='osd.8' 2024-02-17T11:53:50.576 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:50 smithi094 ceph-mon[32499]: osd.8 [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253] boot 2024-02-17T11:53:50.576 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:50 smithi094 ceph-mon[32499]: osdmap e59: 9 total, 9 up, 9 in 2024-02-17T11:53:50.576 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:50 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:50.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:50 smithi125 ceph-mon[36819]: purged_snaps scrub starts 2024-02-17T11:53:50.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:50 smithi125 ceph-mon[36819]: purged_snaps scrub ok 2024-02-17T11:53:50.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:50 smithi125 ceph-mon[36819]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-17T11:53:50.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:50 smithi125 ceph-mon[36819]: osdmap e58: 9 total, 8 up, 9 in 2024-02-17T11:53:50.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:50 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:50.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:50 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:50.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:50 smithi125 ceph-mon[36819]: from='osd.8 ' entity='osd.8' 2024-02-17T11:53:50.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:50 smithi125 ceph-mon[36819]: osd.8 [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253] boot 2024-02-17T11:53:50.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:50 smithi125 ceph-mon[36819]: osdmap e59: 9 total, 9 up, 9 in 2024-02-17T11:53:50.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:50 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:53:51.086 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi094:vg_nvme/lv_1 2024-02-17T11:53:51.336 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:53:51.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:51 smithi189 ceph-mon[36588]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:53:51.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:51 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:51.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:51 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:51.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:51 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:51.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:51 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:51.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:51 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:51.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:51 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:51.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:51 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:51.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:51 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:51.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:51 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:51.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:51 smithi189 ceph-mon[36588]: osdmap e60: 9 total, 9 up, 9 in 2024-02-17T11:53:51.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:51 smithi125 ceph-mon[36819]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:53:51.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:51.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:51.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:51.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:51.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:51.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:51.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:51.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:51.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:51 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:51.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:51 smithi125 ceph-mon[36819]: osdmap e60: 9 total, 9 up, 9 in 2024-02-17T11:53:51.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:51 smithi094 ceph-mon[32499]: pgmap v195: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:53:51.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:51 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:51.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:51 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:51.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:51 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:51.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:51 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:51.695 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:51 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:53:51.696 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:51 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:51.696 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:51 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:51.696 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:51 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:53:51.696 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:51 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:53:51.696 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:51 smithi094 ceph-mon[32499]: osdmap e60: 9 total, 9 up, 9 in 2024-02-17T11:53:52.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:52 smithi189 ceph-mon[36588]: Detected new or changed devices on smithi189 2024-02-17T11:53:52.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:52 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi189 to 5693M 2024-02-17T11:53:52.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:52 smithi189 ceph-mon[36588]: osdmap e61: 9 total, 9 up, 9 in 2024-02-17T11:53:52.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:52 smithi125 ceph-mon[36819]: Detected new or changed devices on smithi189 2024-02-17T11:53:52.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:52 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi189 to 5693M 2024-02-17T11:53:52.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:52 smithi125 ceph-mon[36819]: osdmap e61: 9 total, 9 up, 9 in 2024-02-17T11:53:52.700 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:52 smithi094 ceph-mon[32499]: Detected new or changed devices on smithi189 2024-02-17T11:53:52.700 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:52 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi189 to 5693M 2024-02-17T11:53:52.700 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:52 smithi094 ceph-mon[32499]: osdmap e61: 9 total, 9 up, 9 in 2024-02-17T11:53:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.227+0000 7f08d7e14700 1 -- 172.21.15.94:0/2621425227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d0101cd0 msgr2=0x7f08d01020f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:53:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.227+0000 7f08d7e14700 1 --2- 172.21.15.94:0/2621425227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d0101cd0 0x7f08d01020f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f08c00099c0 tx=0x7f08c0009cd0 comp rx=0 tx=0).stop 2024-02-17T11:53:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.227+0000 7f08d7e14700 1 -- 172.21.15.94:0/2621425227 shutdown_connections 2024-02-17T11:53:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.227+0000 7f08d7e14700 1 --2- 172.21.15.94:0/2621425227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d0103870 0x7f08d010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.227+0000 7f08d7e14700 1 --2- 172.21.15.94:0/2621425227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d0101cd0 0x7f08d01020f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.227+0000 7f08d7e14700 1 --2- 172.21.15.94:0/2621425227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08d0102ed0 0x7f08d0103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.227+0000 7f08d7e14700 1 -- 172.21.15.94:0/2621425227 >> 172.21.15.94:0/2621425227 conn(0x7f08d00fd270 msgr2=0x7f08d00ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:53:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.227+0000 7f08d7e14700 1 -- 172.21.15.94:0/2621425227 shutdown_connections 2024-02-17T11:53:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.227+0000 7f08d7e14700 1 -- 172.21.15.94:0/2621425227 wait complete. 2024-02-17T11:53:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d7e14700 1 Processor -- start 2024-02-17T11:53:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d7e14700 1 -- start start 2024-02-17T11:53:53.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d7e14700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d0101cd0 0x7f08d0197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:53:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d7e14700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08d0102ed0 0x7f08d0198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:53:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d7e14700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d0103870 0x7f08d019f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:53:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d7e14700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d019f860 con 0x7f08d0103870 2024-02-17T11:53:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d7e14700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d019f9a0 con 0x7f08d0102ed0 2024-02-17T11:53:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d7e14700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d019fb60 con 0x7f08d0101cd0 2024-02-17T11:53:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d63b1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d0103870 0x7f08d019f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:53:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d63b1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d0103870 0x7f08d019f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44660/0 (socket says 172.21.15.94:44660) 2024-02-17T11:53:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d63b1700 1 -- 172.21.15.94:0/1745082297 learned_addr learned my addr 172.21.15.94:0/1745082297 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:53:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d5bb0700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d0101cd0 0x7f08d0197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:53:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d53af700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08d0102ed0 0x7f08d0198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:53:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d63b1700 1 -- 172.21.15.94:0/1745082297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d0101cd0 msgr2=0x7f08d0197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:53:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d63b1700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d0101cd0 0x7f08d0197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:53.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d63b1700 1 -- 172.21.15.94:0/1745082297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08d0102ed0 msgr2=0x7f08d0198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:53:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d63b1700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08d0102ed0 0x7f08d0198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:53:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d63b1700 1 -- 172.21.15.94:0/1745082297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00095f0 con 0x7f08d0103870 2024-02-17T11:53:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.228+0000 7f08d53af700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08d0102ed0 0x7f08d0198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:53:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.229+0000 7f08d63b1700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d0103870 0x7f08d019f290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f08c800b970 tx=0x7f08c800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:53:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.229+0000 7f08c6ffd700 1 -- 172.21.15.94:0/1745082297 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c800c930 con 0x7f08d0103870 2024-02-17T11:53:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.229+0000 7f08c6ffd700 1 -- 172.21.15.94:0/1745082297 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08c8014440 con 0x7f08d0103870 2024-02-17T11:53:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.229+0000 7f08d7e14700 1 -- 172.21.15.94:0/1745082297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d019fe40 con 0x7f08d0103870 2024-02-17T11:53:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.229+0000 7f08c6ffd700 1 -- 172.21.15.94:0/1745082297 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c80075d0 con 0x7f08d0103870 2024-02-17T11:53:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.229+0000 7f08d7e14700 1 -- 172.21.15.94:0/1745082297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d01a02d0 con 0x7f08d0103870 2024-02-17T11:53:53.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.230+0000 7f08d7e14700 1 -- 172.21.15.94:0/1745082297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d0109030 con 0x7f08d0103870 2024-02-17T11:53:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.231+0000 7f08c6ffd700 1 -- 172.21.15.94:0/1745082297 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f08c80145b0 con 0x7f08d0103870 2024-02-17T11:53:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.231+0000 7f08c6ffd700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08bc0a21d0 0x7f08bc0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:53:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.231+0000 7f08d5bb0700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08bc0a21d0 0x7f08bc0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:53:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.231+0000 7f08c6ffd700 1 -- 172.21.15.94:0/1745082297 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5829+0+0 (secure 0 0 0) 0x7f08c8006e40 con 0x7f08d0103870 2024-02-17T11:53:53.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.232+0000 7f08d5bb0700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08bc0a21d0 0x7f08bc0a4690 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f08c00038c0 tx=0x7f08c000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:53:53.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.235+0000 7f08c6ffd700 1 -- 172.21.15.94:0/1745082297 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f08c8096e10 con 0x7f08d0103870 2024-02-17T11:53:53.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:53:53.439+0000 7f08d7e14700 1 -- 172.21.15.94:0/1745082297 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f08d002d870 con 0x7f08bc0a21d0 2024-02-17T11:53:53.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:53 smithi125 ceph-mon[36819]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:53:53.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:53 smithi125 ceph-mon[36819]: osdmap e62: 9 total, 9 up, 9 in 2024-02-17T11:53:53.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:53 smithi094 ceph-mon[32499]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:53:53.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:53 smithi094 ceph-mon[32499]: osdmap e62: 9 total, 9 up, 9 in 2024-02-17T11:53:53.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:53 smithi189 ceph-mon[36588]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:53:53.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:53 smithi189 ceph-mon[36588]: osdmap e62: 9 total, 9 up, 9 in 2024-02-17T11:53:54.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:54 smithi125 ceph-mon[36819]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:53:54.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:54 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:53:54.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:54 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:53:54.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:54 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:54.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:54 smithi125 ceph-mon[36819]: osdmap e63: 9 total, 9 up, 9 in 2024-02-17T11:53:54.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:54 smithi094 ceph-mon[32499]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:53:54.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:54 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:53:54.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:54 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:53:54.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:54 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:54.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:54 smithi094 ceph-mon[32499]: osdmap e63: 9 total, 9 up, 9 in 2024-02-17T11:53:54.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:54 smithi189 ceph-mon[36588]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi094:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:53:54.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:54 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:53:54.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:54 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:53:54.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:54 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:53:54.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:54 smithi189 ceph-mon[36588]: osdmap e63: 9 total, 9 up, 9 in 2024-02-17T11:53:55.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:55 smithi125 ceph-mon[36819]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:53:55.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:55 smithi094 ceph-mon[32499]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:53:55.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:55 smithi189 ceph-mon[36588]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:53:56.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:56 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3397229412' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bad201d9-439f-4236-bbf7-e736d0b12a87"}]: dispatch 2024-02-17T11:53:56.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:56 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3397229412' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bad201d9-439f-4236-bbf7-e736d0b12a87"}]': finished 2024-02-17T11:53:56.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:56 smithi125 ceph-mon[36819]: osdmap e64: 10 total, 9 up, 10 in 2024-02-17T11:53:56.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:56 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:53:56.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:56 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3397229412' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bad201d9-439f-4236-bbf7-e736d0b12a87"}]: dispatch 2024-02-17T11:53:56.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:56 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3397229412' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bad201d9-439f-4236-bbf7-e736d0b12a87"}]': finished 2024-02-17T11:53:56.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:56 smithi094 ceph-mon[32499]: osdmap e64: 10 total, 9 up, 10 in 2024-02-17T11:53:56.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:53:56.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:56 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3397229412' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bad201d9-439f-4236-bbf7-e736d0b12a87"}]: dispatch 2024-02-17T11:53:56.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:56 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3397229412' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bad201d9-439f-4236-bbf7-e736d0b12a87"}]': finished 2024-02-17T11:53:56.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:56 smithi189 ceph-mon[36588]: osdmap e64: 10 total, 9 up, 10 in 2024-02-17T11:53:56.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:53:57.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:57 smithi125 ceph-mon[36819]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-02-17T11:53:57.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:57 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1830426648' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:53:57.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:57 smithi094 ceph-mon[32499]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-02-17T11:53:57.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:57 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1830426648' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:53:57.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:57 smithi189 ceph-mon[36588]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-02-17T11:53:57.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:57 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1830426648' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:53:59.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:53:59 smithi094 ceph-mon[32499]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-17T11:53:59.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:53:59 smithi125 ceph-mon[36819]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-17T11:54:00.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:53:59 smithi189 ceph-mon[36588]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-17T11:54:01.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:01 smithi125 ceph-mon[36819]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-17T11:54:01.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:01 smithi094 ceph-mon[32499]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-17T11:54:02.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:01 smithi189 ceph-mon[36588]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-17T11:54:03.913 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:03 smithi094 ceph-mon[32499]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-17T11:54:03.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:03 smithi125 ceph-mon[36819]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-17T11:54:04.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:03 smithi189 ceph-mon[36588]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-02-17T11:54:04.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:04 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-17T11:54:04.816 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:04 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:04.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:04 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-17T11:54:04.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:04 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:05.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:04 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-17T11:54:05.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:04 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:05.909 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:05 smithi094 ceph-mon[32499]: Deploying daemon osd.9 on smithi094 2024-02-17T11:54:05.909 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:05 smithi094 ceph-mon[32499]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-17T11:54:05.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:05 smithi125 ceph-mon[36819]: Deploying daemon osd.9 on smithi094 2024-02-17T11:54:05.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:05 smithi125 ceph-mon[36819]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-17T11:54:06.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:05 smithi189 ceph-mon[36588]: Deploying daemon osd.9 on smithi094 2024-02-17T11:54:06.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:05 smithi189 ceph-mon[36588]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-17T11:54:07.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:07 smithi125 ceph-mon[36819]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:07.941 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:07 smithi094 ceph-mon[32499]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:08.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:07 smithi189 ceph-mon[36588]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:09.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:09 smithi125 ceph-mon[36819]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:09.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:09 smithi094 ceph-mon[32499]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:10.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:09 smithi189 ceph-mon[36588]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:10.765 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:54:10.765 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:10.765 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:10.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:54:10.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:10.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:11.052 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:54:11.052 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:11.052 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:11.853 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:11 smithi094 ceph-mon[32499]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:11.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:11 smithi125 ceph-mon[36819]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:12.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:11 smithi189 ceph-mon[36588]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:13.947 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:13 smithi094 ceph-mon[32499]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:13.947 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:13.947 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:13.947 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:13.947 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:54:13.947 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:14.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:54:14.004+0000 7f08c6ffd700 1 -- 172.21.15.94:0/1745082297 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f08d002d870 con 0x7f08bc0a21d0 2024-02-17T11:54:14.004 INFO:teuthology.orchestra.run.smithi094.stdout:Created osd(s) 9 on host 'smithi094' 2024-02-17T11:54:14.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:54:14.006+0000 7f08d7e14700 1 -- 172.21.15.94:0/1745082297 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08bc0a21d0 msgr2=0x7f08bc0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:54:14.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:54:14.006+0000 7f08d7e14700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08bc0a21d0 0x7f08bc0a4690 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f08c00038c0 tx=0x7f08c000b540 comp rx=0 tx=0).stop 2024-02-17T11:54:14.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:54:14.006+0000 7f08d7e14700 1 -- 172.21.15.94:0/1745082297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d0103870 msgr2=0x7f08d019f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:54:14.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:54:14.006+0000 7f08d7e14700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d0103870 0x7f08d019f290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f08c800b970 tx=0x7f08c800bc80 comp rx=0 tx=0).stop 2024-02-17T11:54:14.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:54:14.006+0000 7f08d7e14700 1 -- 172.21.15.94:0/1745082297 shutdown_connections 2024-02-17T11:54:14.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:54:14.006+0000 7f08d7e14700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08bc0a21d0 0x7f08bc0a4690 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:14.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:54:14.006+0000 7f08d7e14700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d0103870 0x7f08d019f290 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:14.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:54:14.006+0000 7f08d7e14700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d0101cd0 0x7f08d0197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:14.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:54:14.006+0000 7f08d7e14700 1 --2- 172.21.15.94:0/1745082297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08d0102ed0 0x7f08d0198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:14.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:54:14.006+0000 7f08d7e14700 1 -- 172.21.15.94:0/1745082297 >> 172.21.15.94:0/1745082297 conn(0x7f08d00fd270 msgr2=0x7f08d0106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:54:14.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:54:14.006+0000 7f08d7e14700 1 -- 172.21.15.94:0/1745082297 shutdown_connections 2024-02-17T11:54:14.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:54:14.006+0000 7f08d7e14700 1 -- 172.21.15.94:0/1745082297 wait complete. 2024-02-17T11:54:14.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:13 smithi189 ceph-mon[36588]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:14.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:14.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:14.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:14.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:54:14.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:14.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:13 smithi125 ceph-mon[36819]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:14.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:14.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:14.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:14.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:54:14.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:14.794 DEBUG:teuthology.orchestra.run.smithi094:osd.9> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.9.service 2024-02-17T11:54:14.797 INFO:tasks.cephadm:Deploying osd.10 on smithi125 with /dev/vg_nvme/lv_1... 2024-02-17T11:54:14.797 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-17T11:54:14.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:14 smithi094 ceph-mon[32499]: from='osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-17T11:54:14.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:14 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:14.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:14 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:14.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:14 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:54:15.047 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T11:54:15.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:14 smithi189 ceph-mon[36588]: from='osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-17T11:54:15.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:14 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:15.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:14 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:15.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:14 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:54:15.168 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:14 smithi125 ceph-mon[36819]: from='osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-17T11:54:15.168 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:14 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:15.168 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:14 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:15.168 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:14 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:54:15.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:15 smithi125 ceph-mon[36819]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:15.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:15 smithi125 ceph-mon[36819]: from='osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-17T11:54:15.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:15 smithi125 ceph-mon[36819]: osdmap e65: 10 total, 9 up, 10 in 2024-02-17T11:54:15.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:15 smithi125 ceph-mon[36819]: from='osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-17T11:54:15.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:15 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:54:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:15 smithi094 ceph-mon[32499]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:15 smithi094 ceph-mon[32499]: from='osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-17T11:54:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:15 smithi094 ceph-mon[32499]: osdmap e65: 10 total, 9 up, 10 in 2024-02-17T11:54:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:15 smithi094 ceph-mon[32499]: from='osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-17T11:54:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:15 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:54:15.956 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 11:54:15 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T11:54:15.701+0000 7f4f66368700 -1 osd.9 0 waiting for initial osdmap 2024-02-17T11:54:15.956 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 11:54:15 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T11:54:15.716+0000 7f4f6215b700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T11:54:16.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:15 smithi189 ceph-mon[36588]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:16.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:15 smithi189 ceph-mon[36588]: from='osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-17T11:54:16.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:15 smithi189 ceph-mon[36588]: osdmap e65: 10 total, 9 up, 10 in 2024-02-17T11:54:16.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:15 smithi189 ceph-mon[36588]: from='osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]: dispatch 2024-02-17T11:54:16.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:15 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:54:16.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:16 smithi125 ceph-mon[36819]: from='osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-17T11:54:16.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:16 smithi125 ceph-mon[36819]: osdmap e66: 10 total, 9 up, 10 in 2024-02-17T11:54:16.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:16 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:54:16.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:16 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:54:16.936 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-17T11:54:16.936 INFO:teuthology.orchestra.run.smithi125.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-17T11:54:16.936 INFO:teuthology.orchestra.run.smithi125.stderr: stderr: 10+0 records in 2024-02-17T11:54:16.937 INFO:teuthology.orchestra.run.smithi125.stderr:10+0 records out 2024-02-17T11:54:16.937 INFO:teuthology.orchestra.run.smithi125.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0121662 s, 862 MB/s 2024-02-17T11:54:16.937 INFO:teuthology.orchestra.run.smithi125.stderr:--> Zapping successful for: 2024-02-17T11:54:16.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:16 smithi094 ceph-mon[32499]: from='osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-17T11:54:16.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:16 smithi094 ceph-mon[32499]: osdmap e66: 10 total, 9 up, 10 in 2024-02-17T11:54:16.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:16 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:54:16.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:16 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:54:17.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:16 smithi189 ceph-mon[36588]: from='osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi094", "root=default"]}]': finished 2024-02-17T11:54:17.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:16 smithi189 ceph-mon[36588]: osdmap e66: 10 total, 9 up, 10 in 2024-02-17T11:54:17.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:16 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:54:17.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:16 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:54:17.623 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi125:vg_nvme/lv_1 2024-02-17T11:54:17.866 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T11:54:17.907 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:17 smithi125 ceph-mon[36819]: purged_snaps scrub starts 2024-02-17T11:54:17.908 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:17 smithi125 ceph-mon[36819]: purged_snaps scrub ok 2024-02-17T11:54:17.908 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:17 smithi125 ceph-mon[36819]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:17.908 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:17 smithi125 ceph-mon[36819]: osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455] boot 2024-02-17T11:54:17.908 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:17 smithi125 ceph-mon[36819]: osdmap e67: 10 total, 10 up, 10 in 2024-02-17T11:54:17.908 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:17 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:54:17.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:17 smithi094 ceph-mon[32499]: purged_snaps scrub starts 2024-02-17T11:54:17.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:17 smithi094 ceph-mon[32499]: purged_snaps scrub ok 2024-02-17T11:54:17.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:17 smithi094 ceph-mon[32499]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:17.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:17 smithi094 ceph-mon[32499]: osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455] boot 2024-02-17T11:54:17.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:17 smithi094 ceph-mon[32499]: osdmap e67: 10 total, 10 up, 10 in 2024-02-17T11:54:17.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:17 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:54:18.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:17 smithi189 ceph-mon[36588]: purged_snaps scrub starts 2024-02-17T11:54:18.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:17 smithi189 ceph-mon[36588]: purged_snaps scrub ok 2024-02-17T11:54:18.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:17 smithi189 ceph-mon[36588]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-02-17T11:54:18.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:17 smithi189 ceph-mon[36588]: osd.9 [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455] boot 2024-02-17T11:54:18.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:17 smithi189 ceph-mon[36588]: osdmap e67: 10 total, 10 up, 10 in 2024-02-17T11:54:18.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:17 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:54:19.026 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:18 smithi125 ceph-mon[36819]: osdmap e68: 10 total, 10 up, 10 in 2024-02-17T11:54:19.026 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:19.026 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:19.027 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:19.027 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:19.027 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:19.027 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:19.027 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:19.027 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:19.027 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:54:19.027 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:19.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:18 smithi189 ceph-mon[36588]: osdmap e68: 10 total, 10 up, 10 in 2024-02-17T11:54:19.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:19.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:19.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:19.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:19.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:19.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:19.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:19.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:19.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:54:19.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:19.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:18 smithi094 ceph-mon[32499]: osdmap e68: 10 total, 10 up, 10 in 2024-02-17T11:54:19.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:19.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:19.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:19.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:54:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:19.566 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.565+0000 7f15f2ebe700 1 -- 172.21.15.125:0/2772160362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15ec101ca0 msgr2=0x7f15ec1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.565+0000 7f15f2ebe700 1 --2- 172.21.15.125:0/2772160362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15ec101ca0 0x7f15ec1020c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f15e00099c0 tx=0x7f15e0009cd0 comp rx=0 tx=0).stop 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.565+0000 7f15f2ebe700 1 -- 172.21.15.125:0/2772160362 shutdown_connections 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.565+0000 7f15f2ebe700 1 --2- 172.21.15.125:0/2772160362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ec103840 0x7f15ec10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.565+0000 7f15f2ebe700 1 --2- 172.21.15.125:0/2772160362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15ec101ca0 0x7f15ec1020c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.565+0000 7f15f2ebe700 1 --2- 172.21.15.125:0/2772160362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15ec102ea0 0x7f15ec103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.565+0000 7f15f2ebe700 1 -- 172.21.15.125:0/2772160362 >> 172.21.15.125:0/2772160362 conn(0x7f15ec0fd280 msgr2=0x7f15ec0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.565+0000 7f15f2ebe700 1 -- 172.21.15.125:0/2772160362 shutdown_connections 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.565+0000 7f15f2ebe700 1 -- 172.21.15.125:0/2772160362 wait complete. 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f2ebe700 1 Processor -- start 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f2ebe700 1 -- start start 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f2ebe700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15ec101ca0 0x7f15ec075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f2ebe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15ec102ea0 0x7f15ec074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f2ebe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ec103840 0x7f15ec0745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f2ebe700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec074d20 con 0x7f15ec102ea0 2024-02-17T11:54:19.567 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f2ebe700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec074e90 con 0x7f15ec101ca0 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f2ebe700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ec1a1d40 con 0x7f15ec103840 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f0c5a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15ec101ca0 0x7f15ec075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f0c5a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15ec101ca0 0x7f15ec075a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.125:49266/0 (socket says 172.21.15.125:49266) 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f0c5a700 1 -- 172.21.15.125:0/1577864501 learned_addr learned my addr 172.21.15.125:0/1577864501 (peer_addr_for_me v2:172.21.15.125:0/0) 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15ebfff700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15ec102ea0 0x7f15ec074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f145b700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ec103840 0x7f15ec0745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f0c5a700 1 -- 172.21.15.125:0/1577864501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ec103840 msgr2=0x7f15ec0745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f0c5a700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ec103840 0x7f15ec0745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f0c5a700 1 -- 172.21.15.125:0/1577864501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15ec102ea0 msgr2=0x7f15ec074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.566+0000 7f15f0c5a700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15ec102ea0 0x7f15ec074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.567+0000 7f15f0c5a700 1 -- 172.21.15.125:0/1577864501 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e00095f0 con 0x7f15ec101ca0 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.567+0000 7f15f145b700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ec103840 0x7f15ec0745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.567+0000 7f15f0c5a700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15ec101ca0 0x7f15ec075a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f15e00038c0 tx=0x7f15e0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:54:19.568 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.567+0000 7f15e9ffb700 1 -- 172.21.15.125:0/1577864501 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e001d070 con 0x7f15ec101ca0 2024-02-17T11:54:19.569 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.567+0000 7f15f2ebe700 1 -- 172.21.15.125:0/1577864501 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ec1a1fc0 con 0x7f15ec101ca0 2024-02-17T11:54:19.569 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.567+0000 7f15e9ffb700 1 -- 172.21.15.125:0/1577864501 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15e0004110 con 0x7f15ec101ca0 2024-02-17T11:54:19.569 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.567+0000 7f15e9ffb700 1 -- 172.21.15.125:0/1577864501 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0004a20 con 0x7f15ec101ca0 2024-02-17T11:54:19.569 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.567+0000 7f15f2ebe700 1 -- 172.21.15.125:0/1577864501 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ec1a2370 con 0x7f15ec101ca0 2024-02-17T11:54:19.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.568+0000 7f15f2ebe700 1 -- 172.21.15.125:0/1577864501 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ec109100 con 0x7f15ec101ca0 2024-02-17T11:54:19.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.569+0000 7f15e9ffb700 1 -- 172.21.15.125:0/1577864501 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f15e0005150 con 0x7f15ec101ca0 2024-02-17T11:54:19.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.570+0000 7f15e9ffb700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f15dc0a2150 0x7f15dc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:54:19.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.570+0000 7f15e9ffb700 1 -- 172.21.15.125:0/1577864501 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f15e00ce460 con 0x7f15ec101ca0 2024-02-17T11:54:19.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.570+0000 7f15ebfff700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f15dc0a2150 0x7f15dc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:54:19.570 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.570+0000 7f15ebfff700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f15dc0a2150 0x7f15dc0a4610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f15d8005950 tx=0x7f15d800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:54:19.572 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.573+0000 7f15e9ffb700 1 -- 172.21.15.125:0/1577864501 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f15e0097cc0 con 0x7f15ec101ca0 2024-02-17T11:54:19.775 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:19.776+0000 7f15f2ebe700 1 -- 172.21.15.125:0/1577864501 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f15ec02d990 con 0x7f15dc0a2150 2024-02-17T11:54:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:19 smithi094 ceph-mon[32499]: Detected new or changed devices on smithi094 2024-02-17T11:54:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:19 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi094 to 4272M 2024-02-17T11:54:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:19 smithi094 ceph-mon[32499]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:20.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:19 smithi189 ceph-mon[36588]: Detected new or changed devices on smithi094 2024-02-17T11:54:20.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:19 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi094 to 4272M 2024-02-17T11:54:20.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:19 smithi189 ceph-mon[36588]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:20.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:19 smithi125 ceph-mon[36819]: Detected new or changed devices on smithi094 2024-02-17T11:54:20.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:19 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi094 to 4272M 2024-02-17T11:54:20.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:19 smithi125 ceph-mon[36819]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:20.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:20 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:54:20.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:20 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:54:20.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:20 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:21.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:20 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:54:21.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:20 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:54:21.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:20 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:21.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:20 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:54:21.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:20 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:54:21.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:20 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:21.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:21 smithi125 ceph-mon[36819]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:54:21.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:21 smithi125 ceph-mon[36819]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:22.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:21 smithi189 ceph-mon[36588]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:54:22.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:21 smithi189 ceph-mon[36588]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:22.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:21 smithi094 ceph-mon[32499]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi125:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:54:22.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:21 smithi094 ceph-mon[32499]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:23.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:22 smithi189 ceph-mon[36588]: from='client.? 172.21.15.125:0/1857199532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61683c46-c206-4556-b68d-d69457f6fd61"}]: dispatch 2024-02-17T11:54:23.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:22 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61683c46-c206-4556-b68d-d69457f6fd61"}]: dispatch 2024-02-17T11:54:23.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:22 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61683c46-c206-4556-b68d-d69457f6fd61"}]': finished 2024-02-17T11:54:23.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:22 smithi189 ceph-mon[36588]: osdmap e69: 11 total, 10 up, 11 in 2024-02-17T11:54:23.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:22 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:23.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:22 smithi125 ceph-mon[36819]: from='client.? 172.21.15.125:0/1857199532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61683c46-c206-4556-b68d-d69457f6fd61"}]: dispatch 2024-02-17T11:54:23.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:22 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61683c46-c206-4556-b68d-d69457f6fd61"}]: dispatch 2024-02-17T11:54:23.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:22 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61683c46-c206-4556-b68d-d69457f6fd61"}]': finished 2024-02-17T11:54:23.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:22 smithi125 ceph-mon[36819]: osdmap e69: 11 total, 10 up, 11 in 2024-02-17T11:54:23.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:22 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:23.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:22 smithi094 ceph-mon[32499]: from='client.? 172.21.15.125:0/1857199532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61683c46-c206-4556-b68d-d69457f6fd61"}]: dispatch 2024-02-17T11:54:23.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:22 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61683c46-c206-4556-b68d-d69457f6fd61"}]: dispatch 2024-02-17T11:54:23.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:22 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61683c46-c206-4556-b68d-d69457f6fd61"}]': finished 2024-02-17T11:54:23.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:22 smithi094 ceph-mon[32499]: osdmap e69: 11 total, 10 up, 11 in 2024-02-17T11:54:23.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:22 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:24.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:23 smithi189 ceph-mon[36588]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:24.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:23 smithi189 ceph-mon[36588]: from='client.? 172.21.15.125:0/4188709347' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:54:24.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:23 smithi125 ceph-mon[36819]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:24.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:23 smithi125 ceph-mon[36819]: from='client.? 172.21.15.125:0/4188709347' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:54:24.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:23 smithi094 ceph-mon[32499]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:24.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:23 smithi094 ceph-mon[32499]: from='client.? 172.21.15.125:0/4188709347' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:54:26.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:25 smithi189 ceph-mon[36588]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:26.173 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:25 smithi125 ceph-mon[36819]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:26.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:25 smithi094 ceph-mon[32499]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:27.858 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:27 smithi125 ceph-mon[36819]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:28.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:27 smithi189 ceph-mon[36588]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:28.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:27 smithi094 ceph-mon[32499]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:29.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:29 smithi125 ceph-mon[36819]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:30.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:29 smithi189 ceph-mon[36588]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:30.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:29 smithi094 ceph-mon[32499]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:30.864 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:30 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-17T11:54:30.864 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:30 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:31.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:30 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-17T11:54:31.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:30 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:31.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:30 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-17T11:54:31.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:30 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:32.031 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:31 smithi125 ceph-mon[36819]: Deploying daemon osd.10 on smithi125 2024-02-17T11:54:32.031 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:31 smithi125 ceph-mon[36819]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:32.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:31 smithi189 ceph-mon[36588]: Deploying daemon osd.10 on smithi125 2024-02-17T11:54:32.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:31 smithi189 ceph-mon[36588]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:32.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:31 smithi094 ceph-mon[32499]: Deploying daemon osd.10 on smithi125 2024-02-17T11:54:32.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:31 smithi094 ceph-mon[32499]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:33.878 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:33 smithi125 ceph-mon[36819]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:34.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:33 smithi189 ceph-mon[36588]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:34.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:33 smithi094 ceph-mon[32499]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:36.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:35 smithi189 ceph-mon[36588]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:36.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:35 smithi125 ceph-mon[36819]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:36.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:35 smithi094 ceph-mon[32499]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:37.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:36 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:37.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:36 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:54:37.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:36 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:37.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:36 smithi189 ceph-mon[36588]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:37.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:36 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:37.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:36 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:54:37.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:36 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:37.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:36 smithi125 ceph-mon[36819]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:37.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:37.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:54:37.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:37.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:36 smithi094 ceph-mon[32499]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:39.641 INFO:teuthology.orchestra.run.smithi125.stdout:Created osd(s) 10 on host 'smithi125' 2024-02-17T11:54:39.642 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:39.642+0000 7f15e9ffb700 1 -- 172.21.15.125:0/1577864501 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f15ec02d990 con 0x7f15dc0a2150 2024-02-17T11:54:39.644 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:39.644+0000 7f15f2ebe700 1 -- 172.21.15.125:0/1577864501 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f15dc0a2150 msgr2=0x7f15dc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:54:39.644 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:39.644+0000 7f15f2ebe700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f15dc0a2150 0x7f15dc0a4610 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f15d8005950 tx=0x7f15d800b410 comp rx=0 tx=0).stop 2024-02-17T11:54:39.644 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:39.644+0000 7f15f2ebe700 1 -- 172.21.15.125:0/1577864501 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15ec101ca0 msgr2=0x7f15ec075a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:54:39.644 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:39.645+0000 7f15f2ebe700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15ec101ca0 0x7f15ec075a10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f15e00038c0 tx=0x7f15e0004730 comp rx=0 tx=0).stop 2024-02-17T11:54:39.644 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:39.645+0000 7f15f2ebe700 1 -- 172.21.15.125:0/1577864501 shutdown_connections 2024-02-17T11:54:39.644 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:39.645+0000 7f15f2ebe700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f15dc0a2150 0x7f15dc0a4610 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:39.644 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:39.645+0000 7f15f2ebe700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15ec103840 0x7f15ec0745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:39.644 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:39.645+0000 7f15f2ebe700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f15ec101ca0 0x7f15ec075a10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:39.645 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:39.645+0000 7f15f2ebe700 1 --2- 172.21.15.125:0/1577864501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f15ec102ea0 0x7f15ec074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:39.645 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:39.645+0000 7f15f2ebe700 1 -- 172.21.15.125:0/1577864501 >> 172.21.15.125:0/1577864501 conn(0x7f15ec0fd280 msgr2=0x7f15ec106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:54:39.645 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:39.645+0000 7f15f2ebe700 1 -- 172.21.15.125:0/1577864501 shutdown_connections 2024-02-17T11:54:39.645 INFO:teuthology.orchestra.run.smithi125.stderr:2024-02-17T11:54:39.645+0000 7f15f2ebe700 1 -- 172.21.15.125:0/1577864501 wait complete. 2024-02-17T11:54:39.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:39 smithi125 ceph-mon[36819]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:39.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:39 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:39.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:39 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:39.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:39 smithi094 ceph-mon[32499]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:39.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:39.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:40.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:39 smithi189 ceph-mon[36588]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:40.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:39 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:40.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:39 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:40.363 DEBUG:teuthology.orchestra.run.smithi125:osd.10> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.10.service 2024-02-17T11:54:40.366 INFO:tasks.cephadm:Deploying osd.11 on smithi189 with /dev/vg_nvme/lv_1... 2024-02-17T11:54:40.366 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-17T11:54:40.618 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T11:54:40.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:40 smithi125 ceph-mon[36819]: from='osd.10 [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-17T11:54:40.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:40 smithi125 ceph-mon[36819]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-17T11:54:40.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:40 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:40.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:40 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:40.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:40 smithi094 ceph-mon[32499]: from='osd.10 [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-17T11:54:40.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:40 smithi094 ceph-mon[32499]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-17T11:54:40.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:40 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:40.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:40 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:40.962 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:40 smithi189 ceph-mon[36588]: from='osd.10 [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-17T11:54:40.962 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:40 smithi189 ceph-mon[36588]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-17T11:54:40.962 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:40 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:40.962 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:40 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:41 smithi094 ceph-mon[32499]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:41 smithi094 ceph-mon[32499]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-17T11:54:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:41 smithi094 ceph-mon[32499]: osdmap e70: 11 total, 10 up, 11 in 2024-02-17T11:54:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:41 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:41 smithi094 ceph-mon[32499]: from='osd.10 [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:54:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:41 smithi094 ceph-mon[32499]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:54:41.973 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:41 smithi189 ceph-mon[36588]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:41.974 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:41 smithi189 ceph-mon[36588]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-17T11:54:41.974 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:41 smithi189 ceph-mon[36588]: osdmap e70: 11 total, 10 up, 11 in 2024-02-17T11:54:41.974 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:41 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:41.974 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:41 smithi189 ceph-mon[36588]: from='osd.10 [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:54:41.974 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:41 smithi189 ceph-mon[36588]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:54:42.161 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:41 smithi125 ceph-mon[36819]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:42.161 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:41 smithi125 ceph-mon[36819]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-17T11:54:42.161 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:41 smithi125 ceph-mon[36819]: osdmap e70: 11 total, 10 up, 11 in 2024-02-17T11:54:42.161 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:41 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:42.162 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:41 smithi125 ceph-mon[36819]: from='osd.10 [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:54:42.162 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:41 smithi125 ceph-mon[36819]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]: dispatch 2024-02-17T11:54:42.435 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 11:54:42 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T11:54:42.161+0000 7ff724dde700 -1 osd.10 0 waiting for initial osdmap 2024-02-17T11:54:42.435 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 11:54:42 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T11:54:42.174+0000 7ff71cbc9700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T11:54:42.718 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-17T11:54:42.718 INFO:teuthology.orchestra.run.smithi189.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-17T11:54:42.718 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10+0 records in 2024-02-17T11:54:42.718 INFO:teuthology.orchestra.run.smithi189.stderr:10+0 records out 2024-02-17T11:54:42.718 INFO:teuthology.orchestra.run.smithi189.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116136 s, 903 MB/s 2024-02-17T11:54:42.718 INFO:teuthology.orchestra.run.smithi189.stderr:--> Zapping successful for: 2024-02-17T11:54:42.750 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:42 smithi125 ceph-mon[36819]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2024-02-17T11:54:42.750 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:42 smithi125 ceph-mon[36819]: osdmap e71: 11 total, 10 up, 11 in 2024-02-17T11:54:42.750 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:42 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:42.751 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:42 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:42.751 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:42 smithi125 ceph-mon[36819]: from='osd.10 ' entity='osd.10' 2024-02-17T11:54:42.751 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:42 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:42.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:42 smithi189 ceph-mon[36588]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2024-02-17T11:54:42.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:42 smithi189 ceph-mon[36588]: osdmap e71: 11 total, 10 up, 11 in 2024-02-17T11:54:42.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:42 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:42.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:42 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:42.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:42 smithi189 ceph-mon[36588]: from='osd.10 ' entity='osd.10' 2024-02-17T11:54:42.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:42 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:42.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:42 smithi094 ceph-mon[32499]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi125", "root=default"]}]': finished 2024-02-17T11:54:42.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:42 smithi094 ceph-mon[32499]: osdmap e71: 11 total, 10 up, 11 in 2024-02-17T11:54:42.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:42 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:42.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:42 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:42.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:42 smithi094 ceph-mon[32499]: from='osd.10 ' entity='osd.10' 2024-02-17T11:54:42.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:42 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:43.467 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi189:vg_nvme/lv_1 2024-02-17T11:54:43.709 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T11:54:43.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:43 smithi094 ceph-mon[32499]: purged_snaps scrub starts 2024-02-17T11:54:43.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:43 smithi094 ceph-mon[32499]: purged_snaps scrub ok 2024-02-17T11:54:43.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:43 smithi094 ceph-mon[32499]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:43.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:43 smithi094 ceph-mon[32499]: osd.10 [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115] boot 2024-02-17T11:54:43.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:43 smithi094 ceph-mon[32499]: osdmap e72: 11 total, 11 up, 11 in 2024-02-17T11:54:43.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:43 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:44.048 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:43 smithi189 ceph-mon[36588]: purged_snaps scrub starts 2024-02-17T11:54:44.048 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:43 smithi189 ceph-mon[36588]: purged_snaps scrub ok 2024-02-17T11:54:44.048 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:43 smithi189 ceph-mon[36588]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:44.048 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:43 smithi189 ceph-mon[36588]: osd.10 [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115] boot 2024-02-17T11:54:44.048 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:43 smithi189 ceph-mon[36588]: osdmap e72: 11 total, 11 up, 11 in 2024-02-17T11:54:44.048 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:43 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:44.121 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:43 smithi125 ceph-mon[36819]: purged_snaps scrub starts 2024-02-17T11:54:44.121 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:43 smithi125 ceph-mon[36819]: purged_snaps scrub ok 2024-02-17T11:54:44.121 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:43 smithi125 ceph-mon[36819]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-02-17T11:54:44.121 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:43 smithi125 ceph-mon[36819]: osd.10 [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115] boot 2024-02-17T11:54:44.121 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:43 smithi125 ceph-mon[36819]: osdmap e72: 11 total, 11 up, 11 in 2024-02-17T11:54:44.121 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:43 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:44 smithi189 ceph-mon[36588]: osdmap e73: 11 total, 11 up, 11 in 2024-02-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:44 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:44 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:44 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:44 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:44 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:44 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:44.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:44 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:44.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:44 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:44.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:44 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:54:44.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:44 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:44.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:44 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:54:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:44 smithi094 ceph-mon[32499]: osdmap e73: 11 total, 11 up, 11 in 2024-02-17T11:54:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:44 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:44 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:44 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:44 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:44 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:44 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:44 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:44 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:44.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:44 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:54:44.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:44 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:44.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:44 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:54:45.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:44 smithi125 ceph-mon[36819]: osdmap e73: 11 total, 11 up, 11 in 2024-02-17T11:54:45.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:44 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:45.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:44 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:45.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:44 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:45.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:44 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:45.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:44 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:45.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:44 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:54:45.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:44 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:45.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:44 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:45.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:44 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:54:45.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:44 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:45.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:44 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:54:45.452 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.453+0000 7f8ae1859700 1 -- 172.21.15.189:0/489817405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8adc102ea0 msgr2=0x7f8adc103300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:54:45.452 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.453+0000 7f8ae1859700 1 --2- 172.21.15.189:0/489817405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8adc102ea0 0x7f8adc103300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009990 tx=0x7f8ac4009ca0 comp rx=0 tx=0).stop 2024-02-17T11:54:45.452 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.453+0000 7f8ae1859700 1 -- 172.21.15.189:0/489817405 shutdown_connections 2024-02-17T11:54:45.452 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.453+0000 7f8ae1859700 1 --2- 172.21.15.189:0/489817405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8adc103840 0x7f8adc10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:45.453 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.453+0000 7f8ae1859700 1 --2- 172.21.15.189:0/489817405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8adc101ca0 0x7f8adc1020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:45.453 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.453+0000 7f8ae1859700 1 --2- 172.21.15.189:0/489817405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8adc102ea0 0x7f8adc103300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:45.453 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.453+0000 7f8ae1859700 1 -- 172.21.15.189:0/489817405 >> 172.21.15.189:0/489817405 conn(0x7f8adc0fd280 msgr2=0x7f8adc0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:54:45.453 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.453+0000 7f8ae1859700 1 -- 172.21.15.189:0/489817405 shutdown_connections 2024-02-17T11:54:45.453 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.453+0000 7f8ae1859700 1 -- 172.21.15.189:0/489817405 wait complete. 2024-02-17T11:54:45.455 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8ae1859700 1 Processor -- start 2024-02-17T11:54:45.455 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8ae1859700 1 -- start start 2024-02-17T11:54:45.455 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8ae1859700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8adc101ca0 0x7f8adc075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:54:45.455 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8ae1859700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8adc102ea0 0x7f8adc074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:54:45.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8ae1859700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8adc103840 0x7f8adc0745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:54:45.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8ae1859700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc074d20 con 0x7f8adc102ea0 2024-02-17T11:54:45.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8ae1859700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc074e90 con 0x7f8adc101ca0 2024-02-17T11:54:45.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8ae1859700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc1a1c40 con 0x7f8adc103840 2024-02-17T11:54:45.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8adb7fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8adc103840 0x7f8adc0745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:54:45.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8adb7fe700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8adc103840 0x7f8adc0745a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:59966/0 (socket says 172.21.15.189:59966) 2024-02-17T11:54:45.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8adb7fe700 1 -- 172.21.15.189:0/1214846231 learned_addr learned my addr 172.21.15.189:0/1214846231 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-02-17T11:54:45.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8adaffd700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8adc101ca0 0x7f8adc075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:54:45.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8ada7fc700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8adc102ea0 0x7f8adc074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:54:45.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8adb7fe700 1 -- 172.21.15.189:0/1214846231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8adc101ca0 msgr2=0x7f8adc075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:54:45.456 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8adb7fe700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8adc101ca0 0x7f8adc075a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:45.457 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8adb7fe700 1 -- 172.21.15.189:0/1214846231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8adc102ea0 msgr2=0x7f8adc074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:54:45.457 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8adb7fe700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8adc102ea0 0x7f8adc074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:54:45.457 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8adb7fe700 1 -- 172.21.15.189:0/1214846231 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac40095f0 con 0x7f8adc103840 2024-02-17T11:54:45.457 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8adaffd700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8adc101ca0 0x7f8adc075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:54:45.457 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8ada7fc700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8adc102ea0 0x7f8adc074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:54:45.457 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.454+0000 7f8adb7fe700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8adc103840 0x7f8adc0745a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c970 tx=0x7f8ad400cd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:54:45.457 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.455+0000 7f8ae0857700 1 -- 172.21.15.189:0/1214846231 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4007b00 con 0x7f8adc103840 2024-02-17T11:54:45.457 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.455+0000 7f8ae0857700 1 -- 172.21.15.189:0/1214846231 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ad4004d10 con 0x7f8adc103840 2024-02-17T11:54:45.457 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.455+0000 7f8ae1859700 1 -- 172.21.15.189:0/1214846231 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc1a1f20 con 0x7f8adc103840 2024-02-17T11:54:45.457 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.455+0000 7f8ae0857700 1 -- 172.21.15.189:0/1214846231 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4005590 con 0x7f8adc103840 2024-02-17T11:54:45.457 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.455+0000 7f8ae1859700 1 -- 172.21.15.189:0/1214846231 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc1a23e0 con 0x7f8adc103840 2024-02-17T11:54:45.457 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.456+0000 7f8ae1859700 1 -- 172.21.15.189:0/1214846231 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc109060 con 0x7f8adc103840 2024-02-17T11:54:45.458 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.457+0000 7f8ae0857700 1 -- 172.21.15.189:0/1214846231 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8ad4004750 con 0x7f8adc103840 2024-02-17T11:54:45.458 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.457+0000 7f8ae0857700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ac80a21d0 0x7f8ac80a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:54:45.458 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.458+0000 7f8ae0857700 1 -- 172.21.15.189:0/1214846231 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f8ad40cd210 con 0x7f8adc103840 2024-02-17T11:54:45.458 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.458+0000 7f8adaffd700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ac80a21d0 0x7f8ac80a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:54:45.459 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.458+0000 7f8adaffd700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ac80a21d0 0x7f8ac80a4690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8adc102d00 tx=0x7f8acc009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:54:45.460 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.462+0000 7f8ae0857700 1 -- 172.21.15.189:0/1214846231 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ad4096830 con 0x7f8adc103840 2024-02-17T11:54:45.663 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:54:45.664+0000 7f8ae1859700 1 -- 172.21.15.189:0/1214846231 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8adc02d910 con 0x7f8ac80a21d0 2024-02-17T11:54:46.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:45 smithi189 ceph-mon[36588]: Detected new or changed devices on smithi125 2024-02-17T11:54:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:45 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi125 to 4270M 2024-02-17T11:54:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:45 smithi189 ceph-mon[36588]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:54:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:45 smithi189 ceph-mon[36588]: osdmap e74: 11 total, 11 up, 11 in 2024-02-17T11:54:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:45 smithi189 ceph-mon[36588]: osdmap e75: 11 total, 11 up, 11 in 2024-02-17T11:54:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:54:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:54:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:54:46.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:45 smithi125 ceph-mon[36819]: Detected new or changed devices on smithi125 2024-02-17T11:54:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:45 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi125 to 4270M 2024-02-17T11:54:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:45 smithi125 ceph-mon[36819]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:54:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:45 smithi125 ceph-mon[36819]: osdmap e74: 11 total, 11 up, 11 in 2024-02-17T11:54:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:45 smithi125 ceph-mon[36819]: osdmap e75: 11 total, 11 up, 11 in 2024-02-17T11:54:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:54:46.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:46.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:54:46.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:54:46.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:45 smithi094 ceph-mon[32499]: Detected new or changed devices on smithi125 2024-02-17T11:54:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:45 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi125 to 4270M 2024-02-17T11:54:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:45 smithi094 ceph-mon[32499]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:54:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:45 smithi094 ceph-mon[32499]: osdmap e74: 11 total, 11 up, 11 in 2024-02-17T11:54:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:45 smithi094 ceph-mon[32499]: osdmap e75: 11 total, 11 up, 11 in 2024-02-17T11:54:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:54:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:54:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-17T11:54:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-17T11:54:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:46.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:46 smithi189 ceph-mon[36588]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:54:46.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:46 smithi189 ceph-mon[36588]: osdmap e76: 11 total, 11 up, 11 in 2024-02-17T11:54:47.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:46 smithi125 ceph-mon[36819]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:54:47.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:46 smithi125 ceph-mon[36819]: osdmap e76: 11 total, 11 up, 11 in 2024-02-17T11:54:47.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:46 smithi094 ceph-mon[32499]: from='client.24358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi189:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:54:47.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:46 smithi094 ceph-mon[32499]: osdmap e76: 11 total, 11 up, 11 in 2024-02-17T11:54:48.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:47 smithi189 ceph-mon[36588]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-02-17T11:54:48.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:47 smithi125 ceph-mon[36819]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-02-17T11:54:48.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:47 smithi094 ceph-mon[32499]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 147 KiB/s, 0 objects/s recovering 2024-02-17T11:54:49.106 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:48 smithi094 ceph-mon[32499]: from='client.? 172.21.15.189:0/1183004348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52468b82-565a-4bc9-a784-43acd747d719"}]: dispatch 2024-02-17T11:54:49.106 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:48 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52468b82-565a-4bc9-a784-43acd747d719"}]: dispatch 2024-02-17T11:54:49.106 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:48 smithi094 ceph-mon[32499]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52468b82-565a-4bc9-a784-43acd747d719"}]': finished 2024-02-17T11:54:49.106 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:48 smithi094 ceph-mon[32499]: osdmap e77: 12 total, 11 up, 12 in 2024-02-17T11:54:49.106 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:54:49.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:48 smithi189 ceph-mon[36588]: from='client.? 172.21.15.189:0/1183004348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52468b82-565a-4bc9-a784-43acd747d719"}]: dispatch 2024-02-17T11:54:49.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:48 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52468b82-565a-4bc9-a784-43acd747d719"}]: dispatch 2024-02-17T11:54:49.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:48 smithi189 ceph-mon[36588]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52468b82-565a-4bc9-a784-43acd747d719"}]': finished 2024-02-17T11:54:49.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:48 smithi189 ceph-mon[36588]: osdmap e77: 12 total, 11 up, 12 in 2024-02-17T11:54:49.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:54:49.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:48 smithi125 ceph-mon[36819]: from='client.? 172.21.15.189:0/1183004348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52468b82-565a-4bc9-a784-43acd747d719"}]: dispatch 2024-02-17T11:54:49.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:48 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52468b82-565a-4bc9-a784-43acd747d719"}]: dispatch 2024-02-17T11:54:49.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:48 smithi125 ceph-mon[36819]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52468b82-565a-4bc9-a784-43acd747d719"}]': finished 2024-02-17T11:54:49.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:48 smithi125 ceph-mon[36819]: osdmap e77: 12 total, 11 up, 12 in 2024-02-17T11:54:49.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:48 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:54:50.124 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:49 smithi094 ceph-mon[32499]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-17T11:54:50.124 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:49 smithi094 ceph-mon[32499]: from='client.? 172.21.15.189:0/471616450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:54:50.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:49 smithi189 ceph-mon[36588]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-17T11:54:50.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:49 smithi189 ceph-mon[36588]: from='client.? 172.21.15.189:0/471616450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:54:50.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:49 smithi125 ceph-mon[36819]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-17T11:54:50.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:49 smithi125 ceph-mon[36819]: from='client.? 172.21.15.189:0/471616450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-17T11:54:51.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:51 smithi125 ceph-mon[36819]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-02-17T11:54:52.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:51 smithi189 ceph-mon[36588]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-02-17T11:54:52.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:51 smithi094 ceph-mon[32499]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-02-17T11:54:54.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:53 smithi189 ceph-mon[36588]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-17T11:54:54.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:53 smithi125 ceph-mon[36819]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-17T11:54:54.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:53 smithi094 ceph-mon[32499]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-02-17T11:54:55.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:55 smithi189 ceph-mon[36588]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:54:56.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:55 smithi125 ceph-mon[36819]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:54:56.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:55 smithi094 ceph-mon[32499]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:54:56.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-17T11:54:56.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:56 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:57.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:56 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-17T11:54:57.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:56 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:57.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-17T11:54:57.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:56 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:54:57.884 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:57 smithi189 ceph-mon[36588]: Deploying daemon osd.11 on smithi189 2024-02-17T11:54:57.884 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:57 smithi189 ceph-mon[36588]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:54:58.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:57 smithi125 ceph-mon[36819]: Deploying daemon osd.11 on smithi189 2024-02-17T11:54:58.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:57 smithi125 ceph-mon[36819]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:54:58.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:57 smithi094 ceph-mon[32499]: Deploying daemon osd.11 on smithi189 2024-02-17T11:54:58.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:57 smithi094 ceph-mon[32499]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:00.041 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:54:59 smithi189 ceph-mon[36588]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:54:59 smithi125 ceph-mon[36819]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:00.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:54:59 smithi094 ceph-mon[32499]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:01.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:01 smithi189 ceph-mon[36588]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:02.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:01 smithi125 ceph-mon[36819]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:02.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:01 smithi094 ceph-mon[32499]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:02.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:02.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:02.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:02 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:03.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:03.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:03.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:02 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:03.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:03.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:03.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:02 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:04.065 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:03 smithi189 ceph-mon[36588]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:04.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:03 smithi125 ceph-mon[36819]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:04.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:03 smithi094 ceph-mon[32499]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:05.829 INFO:teuthology.orchestra.run.smithi189.stdout:Created osd(s) 11 on host 'smithi189' 2024-02-17T11:55:05.830 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:05.830+0000 7f8ae0857700 1 -- 172.21.15.189:0/1214846231 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f8adc02d910 con 0x7f8ac80a21d0 2024-02-17T11:55:05.831 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:05.832+0000 7f8ae1859700 1 -- 172.21.15.189:0/1214846231 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ac80a21d0 msgr2=0x7f8ac80a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:05.831 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:05.832+0000 7f8ae1859700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ac80a21d0 0x7f8ac80a4690 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8adc102d00 tx=0x7f8acc009040 comp rx=0 tx=0).stop 2024-02-17T11:55:05.831 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:05.832+0000 7f8ae1859700 1 -- 172.21.15.189:0/1214846231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8adc103840 msgr2=0x7f8adc0745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:05.832 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:05.832+0000 7f8ae1859700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8adc103840 0x7f8adc0745a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400c970 tx=0x7f8ad400cd30 comp rx=0 tx=0).stop 2024-02-17T11:55:05.832 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:05.832+0000 7f8ae1859700 1 -- 172.21.15.189:0/1214846231 shutdown_connections 2024-02-17T11:55:05.832 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:05.832+0000 7f8ae1859700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ac80a21d0 0x7f8ac80a4690 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:05.832 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:05.832+0000 7f8ae1859700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8adc103840 0x7f8adc0745a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:05.832 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:05.832+0000 7f8ae1859700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8adc101ca0 0x7f8adc075a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:05.832 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:05.832+0000 7f8ae1859700 1 --2- 172.21.15.189:0/1214846231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8adc102ea0 0x7f8adc074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:05.832 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:05.832+0000 7f8ae1859700 1 -- 172.21.15.189:0/1214846231 >> 172.21.15.189:0/1214846231 conn(0x7f8adc0fd280 msgr2=0x7f8adc1061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:05.832 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:05.832+0000 7f8ae1859700 1 -- 172.21.15.189:0/1214846231 shutdown_connections 2024-02-17T11:55:05.832 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:05.832+0000 7f8ae1859700 1 -- 172.21.15.189:0/1214846231 wait complete. 2024-02-17T11:55:06.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:05 smithi189 ceph-mon[36588]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:06.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:05 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:06.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:05 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:06.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:05 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:06.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:05 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:06.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:05 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:06.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:05 smithi189 ceph-mon[36588]: from='osd.11 [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-17T11:55:06.141 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:05 smithi189 ceph-mon[36588]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-17T11:55:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:05 smithi125 ceph-mon[36819]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:05 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:05 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:05 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:05 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:05 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:05 smithi125 ceph-mon[36819]: from='osd.11 [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-17T11:55:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:05 smithi125 ceph-mon[36819]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-17T11:55:06.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:05 smithi094 ceph-mon[32499]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:06.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:05 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:06.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:05 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:06.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:05 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:06.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:05 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:06.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:05 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:06.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:05 smithi094 ceph-mon[32499]: from='osd.11 [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-17T11:55:06.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:05 smithi094 ceph-mon[32499]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-17T11:55:06.560 DEBUG:teuthology.orchestra.run.smithi189:osd.11> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.11.service 2024-02-17T11:55:06.562 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-02-17T11:55:06.562 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-17T11:55:06.816 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:55:06.938 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:06 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:06.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:06 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:06.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:06 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:06.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:06 smithi094 ceph-mon[32499]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-17T11:55:06.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:06 smithi094 ceph-mon[32499]: osdmap e78: 12 total, 11 up, 12 in 2024-02-17T11:55:06.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:06 smithi094 ceph-mon[32499]: from='osd.11 [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:55:06.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:06 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:06.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:06 smithi094 ceph-mon[32499]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:55:06.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:06 smithi094 ceph-mon[32499]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:07.113 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:06 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:07.113 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:06 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:07.113 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:06 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:07.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:06 smithi189 ceph-mon[36588]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-17T11:55:07.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:06 smithi189 ceph-mon[36588]: osdmap e78: 12 total, 11 up, 12 in 2024-02-17T11:55:07.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:06 smithi189 ceph-mon[36588]: from='osd.11 [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:55:07.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:06 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:07.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:06 smithi189 ceph-mon[36588]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:55:07.114 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:06 smithi189 ceph-mon[36588]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:07.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:06 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:07.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:06 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:07.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:06 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:07.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:06 smithi125 ceph-mon[36819]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-17T11:55:07.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:06 smithi125 ceph-mon[36819]: osdmap e78: 12 total, 11 up, 12 in 2024-02-17T11:55:07.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:06 smithi125 ceph-mon[36819]: from='osd.11 [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:55:07.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:06 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:07.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:06 smithi125 ceph-mon[36819]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]: dispatch 2024-02-17T11:55:07.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:06 smithi125 ceph-mon[36819]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:07.400 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 11:55:07 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T11:55:07.337+0000 7fd31c457700 -1 osd.11 0 waiting for initial osdmap 2024-02-17T11:55:07.401 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 11:55:07 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T11:55:07.349+0000 7fd314242700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-17T11:55:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:07 smithi094 ceph-mon[32499]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-17T11:55:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:07 smithi094 ceph-mon[32499]: osdmap e79: 12 total, 11 up, 12 in 2024-02-17T11:55:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:07 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:07 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:07 smithi094 ceph-mon[32499]: from='osd.11 ' entity='osd.11' 2024-02-17T11:55:08.062 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:07 smithi189 ceph-mon[36588]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-17T11:55:08.062 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:07 smithi189 ceph-mon[36588]: osdmap e79: 12 total, 11 up, 12 in 2024-02-17T11:55:08.062 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:07 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:08.062 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:07 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:08.062 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:07 smithi189 ceph-mon[36588]: from='osd.11 ' entity='osd.11' 2024-02-17T11:55:08.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:07 smithi125 ceph-mon[36819]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi189", "root=default"]}]': finished 2024-02-17T11:55:08.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:07 smithi125 ceph-mon[36819]: osdmap e79: 12 total, 11 up, 12 in 2024-02-17T11:55:08.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:07 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:08.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:07 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:08.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:07 smithi125 ceph-mon[36819]: from='osd.11 ' entity='osd.11' 2024-02-17T11:55:08.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.593+0000 7faef238c700 1 -- 172.21.15.94:0/1797539942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec101ca0 msgr2=0x7faeec1020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:08.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.593+0000 7faef238c700 1 --2- 172.21.15.94:0/1797539942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec101ca0 0x7faeec1020c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faedc0099c0 tx=0x7faedc009cd0 comp rx=0 tx=0).stop 2024-02-17T11:55:08.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.593+0000 7faef238c700 1 -- 172.21.15.94:0/1797539942 shutdown_connections 2024-02-17T11:55:08.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.593+0000 7faef238c700 1 --2- 172.21.15.94:0/1797539942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeec103840 0x7faeec10a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:08.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.593+0000 7faef238c700 1 --2- 172.21.15.94:0/1797539942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec101ca0 0x7faeec1020c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:08.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.593+0000 7faef238c700 1 --2- 172.21.15.94:0/1797539942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeec102ea0 0x7faeec103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:08.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.593+0000 7faef238c700 1 -- 172.21.15.94:0/1797539942 >> 172.21.15.94:0/1797539942 conn(0x7faeec0fd280 msgr2=0x7faeec0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:08.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.593+0000 7faef238c700 1 -- 172.21.15.94:0/1797539942 shutdown_connections 2024-02-17T11:55:08.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.593+0000 7faef238c700 1 -- 172.21.15.94:0/1797539942 wait complete. 2024-02-17T11:55:08.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.593+0000 7faef238c700 1 Processor -- start 2024-02-17T11:55:08.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.593+0000 7faef238c700 1 -- start start 2024-02-17T11:55:08.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faef238c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeec101ca0 0x7faeec197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:08.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faef238c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec102ea0 0x7faeec1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:08.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faef238c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeec103840 0x7faeec19f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:08.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faef238c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec19f830 con 0x7faeec102ea0 2024-02-17T11:55:08.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faef238c700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec19f970 con 0x7faeec101ca0 2024-02-17T11:55:08.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faef238c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeec19fb30 con 0x7faeec103840 2024-02-17T11:55:08.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faeeb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec102ea0 0x7faeec1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:08.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faeeb7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec102ea0 0x7faeec1981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51078/0 (socket says 172.21.15.94:51078) 2024-02-17T11:55:08.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faeeb7fe700 1 -- 172.21.15.94:0/548305686 learned_addr learned my addr 172.21.15.94:0/548305686 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:08.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faef0929700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeec103840 0x7faeec19f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:08.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faeebfff700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeec101ca0 0x7faeec197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:08.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faeeb7fe700 1 -- 172.21.15.94:0/548305686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeec103840 msgr2=0x7faeec19f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:08.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faeeb7fe700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeec103840 0x7faeec19f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:08.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faeeb7fe700 1 -- 172.21.15.94:0/548305686 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeec101ca0 msgr2=0x7faeec197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:08.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faeeb7fe700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeec101ca0 0x7faeec197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:08.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faeeb7fe700 1 -- 172.21.15.94:0/548305686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc0095f0 con 0x7faeec102ea0 2024-02-17T11:55:08.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faeeb7fe700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec102ea0 0x7faeec1981f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faee000d950 tx=0x7faee000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:08.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faee97fa700 1 -- 172.21.15.94:0/548305686 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000f890 con 0x7faeec102ea0 2024-02-17T11:55:08.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faee97fa700 1 -- 172.21.15.94:0/548305686 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faee0010460 con 0x7faeec102ea0 2024-02-17T11:55:08.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faef238c700 1 -- 172.21.15.94:0/548305686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeec19fe10 con 0x7faeec102ea0 2024-02-17T11:55:08.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faee97fa700 1 -- 172.21.15.94:0/548305686 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000e610 con 0x7faeec102ea0 2024-02-17T11:55:08.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.594+0000 7faef238c700 1 -- 172.21.15.94:0/548305686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeec1a0280 con 0x7faeec102ea0 2024-02-17T11:55:08.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.595+0000 7faef238c700 1 -- 172.21.15.94:0/548305686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeec109180 con 0x7faeec102ea0 2024-02-17T11:55:08.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.596+0000 7faee97fa700 1 -- 172.21.15.94:0/548305686 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7faee0010960 con 0x7faeec102ea0 2024-02-17T11:55:08.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.597+0000 7faee97fa700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faed40a21d0 0x7faed40a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:08.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.597+0000 7faeebfff700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faed40a21d0 0x7faed40a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:08.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.597+0000 7faee97fa700 1 -- 172.21.15.94:0/548305686 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faee00cd520 con 0x7faeec102ea0 2024-02-17T11:55:08.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.597+0000 7faeebfff700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faed40a21d0 0x7faed40a4690 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faedc006010 tx=0x7faedc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:08.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.600+0000 7faee97fa700 1 -- 172.21.15.94:0/548305686 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faee0096990 con 0x7faeec102ea0 2024-02-17T11:55:08.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.797+0000 7faef238c700 1 -- 172.21.15.94:0/548305686 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7faeec10d5e0 con 0x7faeec102ea0 2024-02-17T11:55:08.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.798+0000 7faee97fa700 1 -- 172.21.15.94:0/548305686 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) v1 ==== 74+0+133 (secure 0 0 0) 0x7faee00b8e60 con 0x7faeec102ea0 2024-02-17T11:55:08.797 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.800+0000 7faef238c700 1 -- 172.21.15.94:0/548305686 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faed40a21d0 msgr2=0x7faed40a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.800+0000 7faef238c700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faed40a21d0 0x7faed40a4690 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faedc006010 tx=0x7faedc000bc0 comp rx=0 tx=0).stop 2024-02-17T11:55:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.800+0000 7faef238c700 1 -- 172.21.15.94:0/548305686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec102ea0 msgr2=0x7faeec1981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.800+0000 7faef238c700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec102ea0 0x7faeec1981f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faee000d950 tx=0x7faee000dc60 comp rx=0 tx=0).stop 2024-02-17T11:55:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.800+0000 7faef238c700 1 -- 172.21.15.94:0/548305686 shutdown_connections 2024-02-17T11:55:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.800+0000 7faef238c700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faed40a21d0 0x7faed40a4690 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.800+0000 7faef238c700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeec103840 0x7faeec19f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:08.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.800+0000 7faef238c700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeec101ca0 0x7faeec197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:08.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.800+0000 7faef238c700 1 --2- 172.21.15.94:0/548305686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeec102ea0 0x7faeec1981f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:08.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.800+0000 7faef238c700 1 -- 172.21.15.94:0/548305686 >> 172.21.15.94:0/548305686 conn(0x7faeec0fd280 msgr2=0x7faeec1062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:08.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.800+0000 7faef238c700 1 -- 172.21.15.94:0/548305686 shutdown_connections 2024-02-17T11:55:08.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:08.800+0000 7faef238c700 1 -- 172.21.15.94:0/548305686 wait complete. 2024-02-17T11:55:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:08 smithi189 ceph-mon[36588]: purged_snaps scrub starts 2024-02-17T11:55:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:08 smithi189 ceph-mon[36588]: purged_snaps scrub ok 2024-02-17T11:55:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:08 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:08 smithi189 ceph-mon[36588]: osd.11 [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186] boot 2024-02-17T11:55:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:08 smithi189 ceph-mon[36588]: osdmap e80: 12 total, 12 up, 12 in 2024-02-17T11:55:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:08 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:08 smithi189 ceph-mon[36588]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:08 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/548305686' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-17T11:55:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:08 smithi125 ceph-mon[36819]: purged_snaps scrub starts 2024-02-17T11:55:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:08 smithi125 ceph-mon[36819]: purged_snaps scrub ok 2024-02-17T11:55:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:08 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:08 smithi125 ceph-mon[36819]: osd.11 [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186] boot 2024-02-17T11:55:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:08 smithi125 ceph-mon[36819]: osdmap e80: 12 total, 12 up, 12 in 2024-02-17T11:55:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:08 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:08 smithi125 ceph-mon[36819]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:08 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/548305686' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-17T11:55:09.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:08 smithi094 ceph-mon[32499]: purged_snaps scrub starts 2024-02-17T11:55:09.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:08 smithi094 ceph-mon[32499]: purged_snaps scrub ok 2024-02-17T11:55:09.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:08 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:09.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:08 smithi094 ceph-mon[32499]: osd.11 [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186] boot 2024-02-17T11:55:09.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:08 smithi094 ceph-mon[32499]: osdmap e80: 12 total, 12 up, 12 in 2024-02-17T11:55:09.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:08 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:55:09.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:08 smithi094 ceph-mon[32499]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-02-17T11:55:09.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:08 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/548305686' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-17T11:55:09.599 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1708170908,"num_in_osds":12,"osd_in_since":1708170888,"num_remapped_pgs":0} 2024-02-17T11:55:09.599 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-17T11:55:09.839 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:55:10.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:10 smithi189 ceph-mon[36588]: osdmap e81: 12 total, 12 up, 12 in 2024-02-17T11:55:10.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:10.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:10.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:55:10.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:55:10.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:55:10.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:55:10.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:10.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:10.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:10.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:10 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:10.663 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:10 smithi094 ceph-mon[32499]: osdmap e81: 12 total, 12 up, 12 in 2024-02-17T11:55:10.663 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:10.663 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:10.663 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:55:10.663 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:55:10.663 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:55:10.663 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:55:10.664 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:10.664 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:10.664 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:10.664 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:10 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:10.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:10 smithi125 ceph-mon[36819]: osdmap e81: 12 total, 12 up, 12 in 2024-02-17T11:55:10.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:10.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:10.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:55:10.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:55:10.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:55:10.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:55:10.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:10.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:10.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:10.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:10 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:11.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:11 smithi125 ceph-mon[36819]: Detected new or changed devices on smithi189 2024-02-17T11:55:11.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:11 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi189 to 4270M 2024-02-17T11:55:11.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:11 smithi125 ceph-mon[36819]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:11.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.527+0000 7fa7482d5700 1 -- 172.21.15.94:0/2876785652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7401017f0 msgr2=0x7fa740105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:11.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.527+0000 7fa7482d5700 1 --2- 172.21.15.94:0/2876785652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7401017f0 0x7fa740105840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa734009990 tx=0x7fa734009ca0 comp rx=0 tx=0).stop 2024-02-17T11:55:11.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.527+0000 7fa7482d5700 1 -- 172.21.15.94:0/2876785652 shutdown_connections 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.527+0000 7fa7482d5700 1 --2- 172.21.15.94:0/2876785652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa740105f20 0x7fa74010daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.527+0000 7fa7482d5700 1 --2- 172.21.15.94:0/2876785652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa740100e40 0x7fa740101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.527+0000 7fa7482d5700 1 --2- 172.21.15.94:0/2876785652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa7401017f0 0x7fa740105840 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.527+0000 7fa7482d5700 1 -- 172.21.15.94:0/2876785652 >> 172.21.15.94:0/2876785652 conn(0x7fa740077ca0 msgr2=0x7fa7400780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.527+0000 7fa7482d5700 1 -- 172.21.15.94:0/2876785652 shutdown_connections 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.527+0000 7fa7482d5700 1 -- 172.21.15.94:0/2876785652 wait complete. 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.528+0000 7fa7482d5700 1 Processor -- start 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.528+0000 7fa7482d5700 1 -- start start 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.528+0000 7fa7482d5700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa740100e40 0x7fa74010e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.528+0000 7fa7482d5700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7401017f0 0x7fa74010e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.528+0000 7fa7482d5700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740105f20 0x7fa74010ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.528+0000 7fa7482d5700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7401a6710 con 0x7fa740105f20 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.528+0000 7fa7482d5700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7401a6880 con 0x7fa7401017f0 2024-02-17T11:55:11.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.528+0000 7fa7482d5700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7401a6a40 con 0x7fa740100e40 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.528+0000 7fa746872700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740105f20 0x7fa74010ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.528+0000 7fa746872700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740105f20 0x7fa74010ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51086/0 (socket says 172.21.15.94:51086) 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.528+0000 7fa746872700 1 -- 172.21.15.94:0/3961472359 learned_addr learned my addr 172.21.15.94:0/3961472359 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.528+0000 7fa746071700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa740100e40 0x7fa74010e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.528+0000 7fa745870700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7401017f0 0x7fa74010e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.529+0000 7fa746872700 1 -- 172.21.15.94:0/3961472359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa740100e40 msgr2=0x7fa74010e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.529+0000 7fa746872700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa740100e40 0x7fa74010e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.529+0000 7fa746872700 1 -- 172.21.15.94:0/3961472359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7401017f0 msgr2=0x7fa74010e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.529+0000 7fa746872700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7401017f0 0x7fa74010e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.529+0000 7fa746872700 1 -- 172.21.15.94:0/3961472359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7340095f0 con 0x7fa740105f20 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.529+0000 7fa746872700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740105f20 0x7fa74010ecc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa73000d980 tx=0x7fa73000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.529+0000 7fa73f7fe700 1 -- 172.21.15.94:0/3961472359 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa730004a30 con 0x7fa740105f20 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.529+0000 7fa7482d5700 1 -- 172.21.15.94:0/3961472359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7401a6d20 con 0x7fa740105f20 2024-02-17T11:55:11.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.529+0000 7fa73f7fe700 1 -- 172.21.15.94:0/3961472359 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa730005500 con 0x7fa740105f20 2024-02-17T11:55:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.529+0000 7fa73f7fe700 1 -- 172.21.15.94:0/3961472359 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73000fc80 con 0x7fa740105f20 2024-02-17T11:55:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.529+0000 7fa7482d5700 1 -- 172.21.15.94:0/3961472359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7401a71f0 con 0x7fa740105f20 2024-02-17T11:55:11.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.530+0000 7fa7482d5700 1 -- 172.21.15.94:0/3961472359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa740109240 con 0x7fa740105f20 2024-02-17T11:55:11.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.532+0000 7fa73f7fe700 1 -- 172.21.15.94:0/3961472359 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa730005000 con 0x7fa740105f20 2024-02-17T11:55:11.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.532+0000 7fa73f7fe700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa7280a2150 0x7fa7280a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:11.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.532+0000 7fa746071700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa7280a2150 0x7fa7280a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:11.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.532+0000 7fa73f7fe700 1 -- 172.21.15.94:0/3961472359 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7300cd9b0 con 0x7fa740105f20 2024-02-17T11:55:11.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.533+0000 7fa746071700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa7280a2150 0x7fa7280a4610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa738009550 tx=0x7fa7380091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:11.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.536+0000 7fa73f7fe700 1 -- 172.21.15.94:0/3961472359 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa730096e20 con 0x7fa740105f20 2024-02-17T11:55:11.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:11 smithi189 ceph-mon[36588]: Detected new or changed devices on smithi189 2024-02-17T11:55:11.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:11 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi189 to 4270M 2024-02-17T11:55:11.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:11 smithi189 ceph-mon[36588]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:11.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:11 smithi094 ceph-mon[32499]: Detected new or changed devices on smithi189 2024-02-17T11:55:11.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:11 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi189 to 4270M 2024-02-17T11:55:11.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:11 smithi094 ceph-mon[32499]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:11.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.729+0000 7fa7482d5700 1 -- 172.21.15.94:0/3961472359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa74010f770 con 0x7fa740105f20 2024-02-17T11:55:11.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.730+0000 7fa73f7fe700 1 -- 172.21.15.94:0/3961472359 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18444 (secure 0 0 0) 0x7fa7300052b0 con 0x7fa740105f20 2024-02-17T11:55:11.729 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:11.729 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:55:09.341614+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:55:11.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.732+0000 7fa7482d5700 1 -- 172.21.15.94:0/3961472359 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa7280a2150 msgr2=0x7fa7280a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:11.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.732+0000 7fa7482d5700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa7280a2150 0x7fa7280a4610 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa738009550 tx=0x7fa7380091b0 comp rx=0 tx=0).stop 2024-02-17T11:55:11.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.732+0000 7fa7482d5700 1 -- 172.21.15.94:0/3961472359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740105f20 msgr2=0x7fa74010ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:11.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.732+0000 7fa7482d5700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740105f20 0x7fa74010ecc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa73000d980 tx=0x7fa73000dc90 comp rx=0 tx=0).stop 2024-02-17T11:55:11.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.732+0000 7fa7482d5700 1 -- 172.21.15.94:0/3961472359 shutdown_connections 2024-02-17T11:55:11.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.732+0000 7fa7482d5700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa7280a2150 0x7fa7280a4610 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:11.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.732+0000 7fa7482d5700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa740105f20 0x7fa74010ecc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:11.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.732+0000 7fa7482d5700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa740100e40 0x7fa74010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:11.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.732+0000 7fa7482d5700 1 --2- 172.21.15.94:0/3961472359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa7401017f0 0x7fa74010e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:11.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.732+0000 7fa7482d5700 1 -- 172.21.15.94:0/3961472359 >> 172.21.15.94:0/3961472359 conn(0x7fa740077ca0 msgr2=0x7fa7400febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:11.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.732+0000 7fa7482d5700 1 -- 172.21.15.94:0/3961472359 shutdown_connections 2024-02-17T11:55:11.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:11.732+0000 7fa7482d5700 1 -- 172.21.15.94:0/3961472359 wait complete. 2024-02-17T11:55:12.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:12 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3961472359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:55:12.542 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T11:51:20.894601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T11:55:12.542 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-17T11:55:12.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:12 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3961472359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:55:12.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:12 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3961472359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:55:12.780 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:55:13.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:13 smithi189 ceph-mon[36588]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:13.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:13 smithi125 ceph-mon[36819]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:13.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:13 smithi094 ceph-mon[32499]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:14.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.490+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1595664637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8edc106d60 msgr2=0x7f8edc107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:14.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.490+0000 7f8ee0f6a700 1 --2- 172.21.15.94:0/1595664637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8edc106d60 0x7f8edc107140 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009990 tx=0x7f8ecc009ca0 comp rx=0 tx=0).stop 2024-02-17T11:55:14.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.490+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1595664637 shutdown_connections 2024-02-17T11:55:14.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.490+0000 7f8ee0f6a700 1 --2- 172.21.15.94:0/1595664637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8edc101810 0x7f8edc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:14.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.490+0000 7f8ee0f6a700 1 --2- 172.21.15.94:0/1595664637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8edc106d60 0x7f8edc107140 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:14.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.490+0000 7f8ee0f6a700 1 --2- 172.21.15.94:0/1595664637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8edc100e50 0x7f8edc1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:14.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.490+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1595664637 >> 172.21.15.94:0/1595664637 conn(0x7f8edc0779d0 msgr2=0x7f8edc077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:14.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.490+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1595664637 shutdown_connections 2024-02-17T11:55:14.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.490+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1595664637 wait complete. 2024-02-17T11:55:14.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.491+0000 7f8ee0f6a700 1 Processor -- start 2024-02-17T11:55:14.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.491+0000 7f8ee0f6a700 1 -- start start 2024-02-17T11:55:14.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.491+0000 7f8ee0f6a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8edc100e50 0x7f8edc198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:14.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.491+0000 7f8ee0f6a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8edc101810 0x7f8edc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:14.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.491+0000 7f8ee0f6a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8edc106d60 0x7f8edc19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:14.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.491+0000 7f8ee0f6a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc19d020 con 0x7f8edc101810 2024-02-17T11:55:14.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.491+0000 7f8ee0f6a700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc19d190 con 0x7f8edc106d60 2024-02-17T11:55:14.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.491+0000 7f8ee0f6a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8edc19d350 con 0x7f8edc100e50 2024-02-17T11:55:14.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.491+0000 7f8ed9d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8edc101810 0x7f8edc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.491+0000 7f8ed9d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8edc101810 0x7f8edc1986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42520/0 (socket says 172.21.15.94:42520) 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.491+0000 7f8ed9d9b700 1 -- 172.21.15.94:0/1335678585 learned_addr learned my addr 172.21.15.94:0/1335678585 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.492+0000 7f8eda59c700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8edc100e50 0x7f8edc198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.492+0000 7f8edad9d700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8edc106d60 0x7f8edc19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.492+0000 7f8ed9d9b700 1 -- 172.21.15.94:0/1335678585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8edc100e50 msgr2=0x7f8edc198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.492+0000 7f8ed9d9b700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8edc100e50 0x7f8edc198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.492+0000 7f8ed9d9b700 1 -- 172.21.15.94:0/1335678585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8edc106d60 msgr2=0x7f8edc19ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.492+0000 7f8ed9d9b700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8edc106d60 0x7f8edc19ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.492+0000 7f8ed9d9b700 1 -- 172.21.15.94:0/1335678585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc0095f0 con 0x7f8edc101810 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.492+0000 7f8ed9d9b700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8edc101810 0x7f8edc1986d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000c9b0 tx=0x7f8ed000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.492+0000 7f8ecb7fe700 1 -- 172.21.15.94:0/1335678585 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0007b00 con 0x7f8edc101810 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.492+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1335678585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8edc19d630 con 0x7f8edc101810 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.492+0000 7f8ecb7fe700 1 -- 172.21.15.94:0/1335678585 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ed0007c60 con 0x7f8edc101810 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.492+0000 7f8ecb7fe700 1 -- 172.21.15.94:0/1335678585 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0018730 con 0x7f8edc101810 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.492+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1335678585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8edc1a4e90 con 0x7f8edc101810 2024-02-17T11:55:14.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.493+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1335678585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8edc1022f0 con 0x7f8edc101810 2024-02-17T11:55:14.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.494+0000 7f8ecb7fe700 1 -- 172.21.15.94:0/1335678585 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8ed0018890 con 0x7f8edc101810 2024-02-17T11:55:14.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.495+0000 7f8ecb7fe700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ec40a2150 0x7f8ec40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:14.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.495+0000 7f8eda59c700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ec40a2150 0x7f8ec40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:14.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.495+0000 7f8ecb7fe700 1 -- 172.21.15.94:0/1335678585 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8ed00cdbb0 con 0x7f8edc101810 2024-02-17T11:55:14.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.495+0000 7f8eda59c700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ec40a2150 0x7f8ec40a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc006010 tx=0x7f8ecc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:14.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.498+0000 7f8ecb7fe700 1 -- 172.21.15.94:0/1335678585 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ed00970b0 con 0x7f8edc101810 2024-02-17T11:55:14.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.691+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1335678585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8edc110c50 con 0x7f8edc101810 2024-02-17T11:55:14.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.692+0000 7f8ecb7fe700 1 -- 172.21.15.94:0/1335678585 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ed009a770 con 0x7f8edc101810 2024-02-17T11:55:14.691 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-02-17T11:55:14.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.694+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1335678585 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ec40a2150 msgr2=0x7f8ec40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:14.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.694+0000 7f8ee0f6a700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ec40a2150 0x7f8ec40a4610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc006010 tx=0x7f8ecc01a040 comp rx=0 tx=0).stop 2024-02-17T11:55:14.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.694+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1335678585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8edc101810 msgr2=0x7f8edc1986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:14.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.694+0000 7f8ee0f6a700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8edc101810 0x7f8edc1986d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000c9b0 tx=0x7f8ed000cd70 comp rx=0 tx=0).stop 2024-02-17T11:55:14.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.694+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1335678585 shutdown_connections 2024-02-17T11:55:14.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.694+0000 7f8ee0f6a700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ec40a2150 0x7f8ec40a4610 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:14.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.694+0000 7f8ee0f6a700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8edc106d60 0x7f8edc19ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:14.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.694+0000 7f8ee0f6a700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8edc100e50 0x7f8edc198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:14.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.694+0000 7f8ee0f6a700 1 --2- 172.21.15.94:0/1335678585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8edc101810 0x7f8edc1986d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:14.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.694+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1335678585 >> 172.21.15.94:0/1335678585 conn(0x7f8edc0779d0 msgr2=0x7f8edc100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:14.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.695+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1335678585 shutdown_connections 2024-02-17T11:55:14.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:14.695+0000 7f8ee0f6a700 1 -- 172.21.15.94:0/1335678585 wait complete. 2024-02-17T11:55:15.345 INFO:tasks.cephadm:Adding mds.a on smithi094 2024-02-17T11:55:15.345 INFO:tasks.cephadm:Adding mds.d on smithi094 2024-02-17T11:55:15.345 INFO:tasks.cephadm:Adding mds.g on smithi094 2024-02-17T11:55:15.345 INFO:tasks.cephadm:Adding mds.j on smithi094 2024-02-17T11:55:15.345 INFO:tasks.cephadm:Adding mds.b on smithi125 2024-02-17T11:55:15.345 INFO:tasks.cephadm:Adding mds.e on smithi125 2024-02-17T11:55:15.345 INFO:tasks.cephadm:Adding mds.h on smithi125 2024-02-17T11:55:15.345 INFO:tasks.cephadm:Adding mds.k on smithi125 2024-02-17T11:55:15.345 INFO:tasks.cephadm:Adding mds.c on smithi189 2024-02-17T11:55:15.346 INFO:tasks.cephadm:Adding mds.f on smithi189 2024-02-17T11:55:15.346 INFO:tasks.cephadm:Adding mds.i on smithi189 2024-02-17T11:55:15.346 INFO:tasks.cephadm:Adding mds.l on smithi189 2024-02-17T11:55:15.346 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph orch apply mds all '12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l' 2024-02-17T11:55:15.598 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T11:55:15.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:15 smithi125 ceph-mon[36819]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:15.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:15 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1335678585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:55:15.937 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:15 smithi189 ceph-mon[36588]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:15.937 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:15 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1335678585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:55:15.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:15 smithi094 ceph-mon[32499]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:15.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:15 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1335678585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:55:17.342 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.341+0000 7f4d1617a700 1 -- 172.21.15.189:0/3899590708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d101050d0 msgr2=0x7f4d101074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:17.343 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.341+0000 7f4d1617a700 1 --2- 172.21.15.189:0/3899590708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d101050d0 0x7f4d101074c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4d04009990 tx=0x7f4d04009ca0 comp rx=0 tx=0).stop 2024-02-17T11:55:17.343 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.341+0000 7f4d1617a700 1 -- 172.21.15.189:0/3899590708 shutdown_connections 2024-02-17T11:55:17.343 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.341+0000 7f4d1617a700 1 --2- 172.21.15.189:0/3899590708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d10107a00 0x7f4d10109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:17.343 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.341+0000 7f4d1617a700 1 --2- 172.21.15.189:0/3899590708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d101027a0 0x7f4d10104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:17.343 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.341+0000 7f4d1617a700 1 --2- 172.21.15.189:0/3899590708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d101050d0 0x7f4d101074c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:17.343 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.341+0000 7f4d1617a700 1 -- 172.21.15.189:0/3899590708 >> 172.21.15.189:0/3899590708 conn(0x7f4d100fc1d0 msgr2=0x7f4d100fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:17.343 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.341+0000 7f4d1617a700 1 -- 172.21.15.189:0/3899590708 shutdown_connections 2024-02-17T11:55:17.343 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.341+0000 7f4d1617a700 1 -- 172.21.15.189:0/3899590708 wait complete. 2024-02-17T11:55:17.343 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.342+0000 7f4d1617a700 1 Processor -- start 2024-02-17T11:55:17.343 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.342+0000 7f4d1617a700 1 -- start start 2024-02-17T11:55:17.343 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.342+0000 7f4d1617a700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d101027a0 0x7f4d10197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:17.343 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.342+0000 7f4d1617a700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d101050d0 0x7f4d10198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:17.343 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.342+0000 7f4d1617a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d10107a00 0x7f4d1019f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:17.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.342+0000 7f4d1617a700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1019f830 con 0x7f4d10107a00 2024-02-17T11:55:17.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.342+0000 7f4d1617a700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1019f970 con 0x7f4d101027a0 2024-02-17T11:55:17.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.342+0000 7f4d1617a700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1019fb30 con 0x7f4d101050d0 2024-02-17T11:55:17.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0effd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d101050d0 0x7f4d10198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:17.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0effd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d101050d0 0x7f4d10198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:41742/0 (socket says 172.21.15.189:41742) 2024-02-17T11:55:17.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0effd700 1 -- 172.21.15.189:0/76105099 learned_addr learned my addr 172.21.15.189:0/76105099 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-02-17T11:55:17.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0ffff700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d10107a00 0x7f4d1019f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:17.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0f7fe700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d101027a0 0x7f4d10197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:17.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0effd700 1 -- 172.21.15.189:0/76105099 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d101027a0 msgr2=0x7f4d10197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:17.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0effd700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d101027a0 0x7f4d10197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:17.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0effd700 1 -- 172.21.15.189:0/76105099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d10107a00 msgr2=0x7f4d1019f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:17.344 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0effd700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d10107a00 0x7f4d1019f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:17.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0effd700 1 -- 172.21.15.189:0/76105099 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d040095f0 con 0x7f4d101050d0 2024-02-17T11:55:17.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0effd700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d101050d0 0x7f4d10198220 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4d04000c00 tx=0x7f4d0400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:17.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0cff9700 1 -- 172.21.15.189:0/76105099 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0401d070 con 0x7f4d101050d0 2024-02-17T11:55:17.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d1617a700 1 -- 172.21.15.189:0/76105099 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1019fdb0 con 0x7f4d101050d0 2024-02-17T11:55:17.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0cff9700 1 -- 172.21.15.189:0/76105099 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d0400fb90 con 0x7f4d101050d0 2024-02-17T11:55:17.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d0cff9700 1 -- 172.21.15.189:0/76105099 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d04017650 con 0x7f4d101050d0 2024-02-17T11:55:17.345 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.343+0000 7f4d1617a700 1 -- 172.21.15.189:0/76105099 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d101a0170 con 0x7f4d101050d0 2024-02-17T11:55:17.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.344+0000 7f4d1617a700 1 -- 172.21.15.189:0/76105099 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d100fdd50 con 0x7f4d101050d0 2024-02-17T11:55:17.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.346+0000 7f4d0cff9700 1 -- 172.21.15.189:0/76105099 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4d0400fd00 con 0x7f4d101050d0 2024-02-17T11:55:17.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.346+0000 7f4d0cff9700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4cf80a2150 0x7f4cf80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:17.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.346+0000 7f4d0cff9700 1 -- 172.21.15.189:0/76105099 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4d040cf460 con 0x7f4d101050d0 2024-02-17T11:55:17.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.346+0000 7f4d0f7fe700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4cf80a2150 0x7f4cf80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:17.346 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.347+0000 7f4d0f7fe700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4cf80a2150 0x7f4cf80a4610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4d101991c0 tx=0x7f4d00009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:17.348 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.350+0000 7f4d0cff9700 1 -- 172.21.15.189:0/76105099 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4d04098960 con 0x7f4d101050d0 2024-02-17T11:55:17.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.551+0000 7f4d1617a700 1 -- 172.21.15.189:0/76105099 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d1002d9d0 con 0x7f4cf80a2150 2024-02-17T11:55:17.558 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.559+0000 7f4d0cff9700 1 -- 172.21.15.189:0/76105099 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f4d1002d9d0 con 0x7f4cf80a2150 2024-02-17T11:55:17.558 INFO:teuthology.orchestra.run.smithi189.stdout:Scheduled mds.all update... 2024-02-17T11:55:17.560 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.561+0000 7f4d1617a700 1 -- 172.21.15.189:0/76105099 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4cf80a2150 msgr2=0x7f4cf80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:17.560 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.561+0000 7f4d1617a700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4cf80a2150 0x7f4cf80a4610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4d101991c0 tx=0x7f4d00009190 comp rx=0 tx=0).stop 2024-02-17T11:55:17.560 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.561+0000 7f4d1617a700 1 -- 172.21.15.189:0/76105099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d101050d0 msgr2=0x7f4d10198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:17.560 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.561+0000 7f4d1617a700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d101050d0 0x7f4d10198220 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4d04000c00 tx=0x7f4d0400f690 comp rx=0 tx=0).stop 2024-02-17T11:55:17.560 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.561+0000 7f4d1617a700 1 -- 172.21.15.189:0/76105099 shutdown_connections 2024-02-17T11:55:17.561 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.561+0000 7f4d1617a700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4cf80a2150 0x7f4cf80a4610 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:17.561 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.561+0000 7f4d1617a700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d10107a00 0x7f4d1019f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:17.561 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.561+0000 7f4d1617a700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d101027a0 0x7f4d10197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:17.561 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.561+0000 7f4d1617a700 1 --2- 172.21.15.189:0/76105099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d101050d0 0x7f4d10198220 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:17.561 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.561+0000 7f4d1617a700 1 -- 172.21.15.189:0/76105099 >> 172.21.15.189:0/76105099 conn(0x7f4d100fc1d0 msgr2=0x7f4d10103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:17.561 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.561+0000 7f4d1617a700 1 -- 172.21.15.189:0/76105099 shutdown_connections 2024-02-17T11:55:17.561 INFO:teuthology.orchestra.run.smithi189.stderr:2024-02-17T11:55:17.561+0000 7f4d1617a700 1 -- 172.21.15.189:0/76105099 wait complete. 2024-02-17T11:55:17.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:17 smithi189 ceph-mon[36588]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:17.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:17 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:17.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:17 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:17.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:17 smithi125 ceph-mon[36819]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:17.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:17 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:17.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:17 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:17.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:17 smithi094 ceph-mon[32499]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:17.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:17 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:17.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:17 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:18.254 DEBUG:teuthology.orchestra.run.smithi094:mds.a> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.a.service 2024-02-17T11:55:18.256 DEBUG:teuthology.orchestra.run.smithi094:mds.d> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.d.service 2024-02-17T11:55:18.300 DEBUG:teuthology.orchestra.run.smithi094:mds.g> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.g.service 2024-02-17T11:55:18.343 DEBUG:teuthology.orchestra.run.smithi094:mds.j> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.j.service 2024-02-17T11:55:18.386 DEBUG:teuthology.orchestra.run.smithi125:mds.b> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.b.service 2024-02-17T11:55:18.389 DEBUG:teuthology.orchestra.run.smithi125:mds.e> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.e.service 2024-02-17T11:55:18.432 DEBUG:teuthology.orchestra.run.smithi125:mds.h> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.h.service 2024-02-17T11:55:18.474 DEBUG:teuthology.orchestra.run.smithi125:mds.k> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.k.service 2024-02-17T11:55:18.518 DEBUG:teuthology.orchestra.run.smithi189:mds.c> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.c.service 2024-02-17T11:55:18.520 DEBUG:teuthology.orchestra.run.smithi189:mds.f> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.f.service 2024-02-17T11:55:18.564 DEBUG:teuthology.orchestra.run.smithi189:mds.i> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.i.service 2024-02-17T11:55:18.607 DEBUG:teuthology.orchestra.run.smithi189:mds.l> sudo journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.l.service 2024-02-17T11:55:18.649 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-02-17T11:55:18.650 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:55:18.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.761+0000 7f52c8b48640 1 -- 172.21.15.94:0/2218639686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52c4101400 msgr2=0x7f52c4107c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:18.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.761+0000 7f52c8b48640 1 --2- 172.21.15.94:0/2218639686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52c4101400 0x7f52c4107c90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009ca0 tx=0x7f52bc02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:55:18.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.761+0000 7f52c8b48640 1 -- 172.21.15.94:0/2218639686 shutdown_connections 2024-02-17T11:55:18.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.761+0000 7f52c8b48640 1 --2- 172.21.15.94:0/2218639686 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52c4101400 0x7f52c4107c90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:18.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.761+0000 7f52c8b48640 1 --2- 172.21.15.94:0/2218639686 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52c4100a40 0x7f52c4100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:18.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.761+0000 7f52c8b48640 1 --2- 172.21.15.94:0/2218639686 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c40ff850 0x7f52c40ffc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:18.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.761+0000 7f52c8b48640 1 -- 172.21.15.94:0/2218639686 >> 172.21.15.94:0/2218639686 conn(0x7f52c40fadc0 msgr2=0x7f52c40fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:18.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.761+0000 7f52c8b48640 1 -- 172.21.15.94:0/2218639686 shutdown_connections 2024-02-17T11:55:18.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.761+0000 7f52c8b48640 1 -- 172.21.15.94:0/2218639686 wait complete. 2024-02-17T11:55:18.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.762+0000 7f52c8b48640 1 Processor -- start 2024-02-17T11:55:18.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.762+0000 7f52c8b48640 1 -- start start 2024-02-17T11:55:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.762+0000 7f52c8b48640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52c40ff850 0x7f52c41971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.762+0000 7f52c8b48640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52c4100a40 0x7f52c4197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.762+0000 7f52c8b48640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c4101400 0x7f52c419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.762+0000 7f52c8b48640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c419ecb0 con 0x7f52c40ff850 2024-02-17T11:55:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.762+0000 7f52c8b48640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c419ee20 con 0x7f52c4100a40 2024-02-17T11:55:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.762+0000 7f52c8b48640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c419efe0 con 0x7f52c4101400 2024-02-17T11:55:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.762+0000 7f52c259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52c40ff850 0x7f52c41971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.762+0000 7f52c259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52c40ff850 0x7f52c41971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42528/0 (socket says 172.21.15.94:42528) 2024-02-17T11:55:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.762+0000 7f52c259c640 1 -- 172.21.15.94:0/3093745511 learned_addr learned my addr 172.21.15.94:0/3093745511 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52bbfff640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52c4100a40 0x7f52c4197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52c2d9d640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c4101400 0x7f52c419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52c259c640 1 -- 172.21.15.94:0/3093745511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c4101400 msgr2=0x7f52c419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:18.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52c259c640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c4101400 0x7f52c419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52c259c640 1 -- 172.21.15.94:0/3093745511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52c4100a40 msgr2=0x7f52c4197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52c259c640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52c4100a40 0x7f52c4197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52c259c640 1 -- 172.21.15.94:0/3093745511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc009900 con 0x7f52c40ff850 2024-02-17T11:55:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52c2d9d640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c4101400 0x7f52c419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:55:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52bbfff640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52c4100a40 0x7f52c4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:55:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52c259c640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52c40ff850 0x7f52c41971c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f52b400d770 tx=0x7f52b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52bb7fe640 1 -- 172.21.15.94:0/3093745511 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4009c30 con 0x7f52c40ff850 2024-02-17T11:55:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52bb7fe640 1 -- 172.21.15.94:0/3093745511 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52b400de80 con 0x7f52c40ff850 2024-02-17T11:55:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52c8b48640 1 -- 172.21.15.94:0/3093745511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c419f2c0 con 0x7f52c40ff850 2024-02-17T11:55:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52bb7fe640 1 -- 172.21.15.94:0/3093745511 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4010460 con 0x7f52c40ff850 2024-02-17T11:55:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.763+0000 7f52c8b48640 1 -- 172.21.15.94:0/3093745511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c419f7d0 con 0x7f52c40ff850 2024-02-17T11:55:18.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.764+0000 7f52c8b48640 1 -- 172.21.15.94:0/3093745511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c4106b10 con 0x7f52c40ff850 2024-02-17T11:55:18.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.765+0000 7f52bb7fe640 1 -- 172.21.15.94:0/3093745511 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f52b4003f60 con 0x7f52c40ff850 2024-02-17T11:55:18.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.765+0000 7f52bb7fe640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f52b00a20c0 0x7f52b00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:18.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.766+0000 7f52bbfff640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f52b00a20c0 0x7f52b00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:18.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.766+0000 7f52bb7fe640 1 -- 172.21.15.94:0/3093745511 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f52b40cd5b0 con 0x7f52c40ff850 2024-02-17T11:55:18.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.766+0000 7f52bbfff640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f52b00a20c0 0x7f52b00a4580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0058c0 tx=0x7f52ac005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:18.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.770+0000 7f52bb7fe640 1 -- 172.21.15.94:0/3093745511 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f52b4096a20 con 0x7f52c40ff850 2024-02-17T11:55:18.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:18 smithi189 ceph-mon[36588]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:55:18.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:18 smithi189 ceph-mon[36588]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-02-17T11:55:18.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:18.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:18.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:18.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:18.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:18.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:18 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:18.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:18 smithi125 ceph-mon[36819]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:55:18.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:18 smithi125 ceph-mon[36819]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-02-17T11:55:18.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:18.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:18.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:18.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:18.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:18.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:18 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:18.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.944+0000 7f52c8b48640 1 -- 172.21.15.94:0/3093745511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52c402ce40 con 0x7f52c40ff850 2024-02-17T11:55:18.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.945+0000 7f52bb7fe640 1 -- 172.21.15.94:0/3093745511 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18444 (secure 0 0 0) 0x7f52b409a0e0 con 0x7f52c40ff850 2024-02-17T11:55:18.944 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:18.945 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:55:09.341614+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:55:18.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.947+0000 7f52c8b48640 1 -- 172.21.15.94:0/3093745511 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f52b00a20c0 msgr2=0x7f52b00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:18.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.947+0000 7f52c8b48640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f52b00a20c0 0x7f52b00a4580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0058c0 tx=0x7f52ac005850 comp rx=0 tx=0).stop 2024-02-17T11:55:18.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.947+0000 7f52c8b48640 1 -- 172.21.15.94:0/3093745511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52c40ff850 msgr2=0x7f52c41971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.947+0000 7f52c8b48640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52c40ff850 0x7f52c41971c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f52b400d770 tx=0x7f52b400dc30 comp rx=0 tx=0).stop 2024-02-17T11:55:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.947+0000 7f52c8b48640 1 -- 172.21.15.94:0/3093745511 shutdown_connections 2024-02-17T11:55:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.947+0000 7f52c8b48640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f52b00a20c0 0x7f52b00a4580 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.947+0000 7f52c8b48640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52c4101400 0x7f52c419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.947+0000 7f52c8b48640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52c4100a40 0x7f52c4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.947+0000 7f52c8b48640 1 --2- 172.21.15.94:0/3093745511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52c40ff850 0x7f52c41971c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.947+0000 7f52c8b48640 1 -- 172.21.15.94:0/3093745511 >> 172.21.15.94:0/3093745511 conn(0x7f52c40fadc0 msgr2=0x7f52c4103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.947+0000 7f52c8b48640 1 -- 172.21.15.94:0/3093745511 shutdown_connections 2024-02-17T11:55:18.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:18.947+0000 7f52c8b48640 1 -- 172.21.15.94:0/3093745511 wait complete. 2024-02-17T11:55:18.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:18 smithi094 ceph-mon[32499]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:55:18.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:18 smithi094 ceph-mon[32499]: Saving service mds.all spec with placement smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-02-17T11:55:18.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:18.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:18.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:18.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:18.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:18.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:18 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:18.957 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T11:51:20.894601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T11:55:18.958 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T11:55:19.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.063+0000 7f1eba702640 1 -- 172.21.15.94:0/1499455341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb4101270 msgr2=0x7f1eb41016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:19.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.063+0000 7f1eba702640 1 --2- 172.21.15.94:0/1499455341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb4101270 0x7f1eb41016f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8009900 tx=0x7f1ea802bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:19.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.063+0000 7f1eba702640 1 -- 172.21.15.94:0/1499455341 shutdown_connections 2024-02-17T11:55:19.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.063+0000 7f1eba702640 1 --2- 172.21.15.94:0/1499455341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1eb4101c30 0x7f1eb41084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.063+0000 7f1eba702640 1 --2- 172.21.15.94:0/1499455341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb4101270 0x7f1eb41016f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.063+0000 7f1eba702640 1 --2- 172.21.15.94:0/1499455341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb4100080 0x7f1eb41004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.063+0000 7f1eba702640 1 -- 172.21.15.94:0/1499455341 >> 172.21.15.94:0/1499455341 conn(0x7f1eb40fb610 msgr2=0x7f1eb40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:19.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.063+0000 7f1eba702640 1 -- 172.21.15.94:0/1499455341 shutdown_connections 2024-02-17T11:55:19.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.063+0000 7f1eba702640 1 -- 172.21.15.94:0/1499455341 wait complete. 2024-02-17T11:55:19.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.064+0000 7f1eba702640 1 Processor -- start 2024-02-17T11:55:19.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.064+0000 7f1eba702640 1 -- start start 2024-02-17T11:55:19.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.064+0000 7f1eba702640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1eb4100080 0x7f1eb4068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:19.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.064+0000 7f1eba702640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb4101270 0x7f1eb419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:19.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.064+0000 7f1eba702640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb4101c30 0x7f1eb419e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:19.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.064+0000 7f1eba702640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb419ed30 con 0x7f1eb4101270 2024-02-17T11:55:19.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.064+0000 7f1eba702640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb419eea0 con 0x7f1eb4100080 2024-02-17T11:55:19.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.064+0000 7f1eba702640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb419f060 con 0x7f1eb4101c30 2024-02-17T11:55:19.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.064+0000 7f1eb37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb4101270 0x7f1eb419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.064+0000 7f1eb37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb4101270 0x7f1eb419bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42544/0 (socket says 172.21.15.94:42544) 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.064+0000 7f1eb37fe640 1 -- 172.21.15.94:0/3978400344 learned_addr learned my addr 172.21.15.94:0/3978400344 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.064+0000 7f1eb3fff640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1eb4100080 0x7f1eb4068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.064+0000 7f1eb8c9f640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb4101c30 0x7f1eb419e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.065+0000 7f1eb37fe640 1 -- 172.21.15.94:0/3978400344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb4101c30 msgr2=0x7f1eb419e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.065+0000 7f1eb37fe640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb4101c30 0x7f1eb419e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.065+0000 7f1eb37fe640 1 -- 172.21.15.94:0/3978400344 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1eb4100080 msgr2=0x7f1eb4068e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.065+0000 7f1eb37fe640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1eb4100080 0x7f1eb4068e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.065+0000 7f1eb37fe640 1 -- 172.21.15.94:0/3978400344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea8009560 con 0x7f1eb4101270 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.065+0000 7f1eb37fe640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb4101270 0x7f1eb419bf60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8006010 tx=0x7f1ea80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.065+0000 7f1eb17fa640 1 -- 172.21.15.94:0/3978400344 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea80044c0 con 0x7f1eb4101270 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.065+0000 7f1eba702640 1 -- 172.21.15.94:0/3978400344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb419f2e0 con 0x7f1eb4101270 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.065+0000 7f1eb17fa640 1 -- 172.21.15.94:0/3978400344 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ea80052d0 con 0x7f1eb4101270 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.065+0000 7f1eb17fa640 1 -- 172.21.15.94:0/3978400344 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8003e10 con 0x7f1eb4101270 2024-02-17T11:55:19.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.065+0000 7f1eba702640 1 -- 172.21.15.94:0/3978400344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb419f840 con 0x7f1eb4101270 2024-02-17T11:55:19.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.066+0000 7f1eba702640 1 -- 172.21.15.94:0/3978400344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb4067f70 con 0x7f1eb4101270 2024-02-17T11:55:19.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.067+0000 7f1eb17fa640 1 -- 172.21.15.94:0/3978400344 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1ea8005440 con 0x7f1eb4101270 2024-02-17T11:55:19.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.067+0000 7f1eb17fa640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1e9c0a20c0 0x7f1e9c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:19.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.068+0000 7f1eb3fff640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1e9c0a20c0 0x7f1e9c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:19.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.068+0000 7f1eb17fa640 1 -- 172.21.15.94:0/3978400344 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1ea80efae0 con 0x7f1eb4101270 2024-02-17T11:55:19.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.068+0000 7f1eb3fff640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1e9c0a20c0 0x7f1e9c0a4580 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1eb41010d0 tx=0x7f1ea4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:19.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.072+0000 7f1eb17fa640 1 -- 172.21.15.94:0/3978400344 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ea80b8f50 con 0x7f1eb4101270 2024-02-17T11:55:19.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.244+0000 7f1eba702640 1 -- 172.21.15.94:0/3978400344 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1eb410ba90 con 0x7f1eb4101270 2024-02-17T11:55:19.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.244+0000 7f1eb17fa640 1 -- 172.21.15.94:0/3978400344 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ea80bc610 con 0x7f1eb4101270 2024-02-17T11:55:19.243 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-02-17T11:55:19.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.246+0000 7f1eba702640 1 -- 172.21.15.94:0/3978400344 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1e9c0a20c0 msgr2=0x7f1e9c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:19.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.246+0000 7f1eba702640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1e9c0a20c0 0x7f1e9c0a4580 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1eb41010d0 tx=0x7f1ea4005f90 comp rx=0 tx=0).stop 2024-02-17T11:55:19.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.246+0000 7f1eba702640 1 -- 172.21.15.94:0/3978400344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb4101270 msgr2=0x7f1eb419bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:19.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.246+0000 7f1eba702640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb4101270 0x7f1eb419bf60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ea8006010 tx=0x7f1ea80040c0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.246+0000 7f1eba702640 1 -- 172.21.15.94:0/3978400344 shutdown_connections 2024-02-17T11:55:19.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.246+0000 7f1eba702640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1e9c0a20c0 0x7f1e9c0a4580 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.246+0000 7f1eba702640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1eb4101c30 0x7f1eb419e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.246+0000 7f1eba702640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1eb4101270 0x7f1eb419bf60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.246+0000 7f1eba702640 1 --2- 172.21.15.94:0/3978400344 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1eb4100080 0x7f1eb4068e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.246+0000 7f1eba702640 1 -- 172.21.15.94:0/3978400344 >> 172.21.15.94:0/3978400344 conn(0x7f1eb40fb610 msgr2=0x7f1eb41045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:19.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.247+0000 7f1eba702640 1 -- 172.21.15.94:0/3978400344 shutdown_connections 2024-02-17T11:55:19.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.247+0000 7f1eba702640 1 -- 172.21.15.94:0/3978400344 wait complete. 2024-02-17T11:55:19.257 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:19.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.362+0000 7fc48c927640 1 -- 172.21.15.94:0/724728222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc488102390 msgr2=0x7fc488102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:19.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.362+0000 7fc48c927640 1 --2- 172.21.15.94:0/724728222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc488102390 0x7fc488102810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc478009900 tx=0x7fc47802bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:19.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.362+0000 7fc48c927640 1 -- 172.21.15.94:0/724728222 shutdown_connections 2024-02-17T11:55:19.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.362+0000 7fc48c927640 1 --2- 172.21.15.94:0/724728222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc488102d50 0x7fc4881095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.362+0000 7fc48c927640 1 --2- 172.21.15.94:0/724728222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc488102390 0x7fc488102810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.362+0000 7fc48c927640 1 --2- 172.21.15.94:0/724728222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4881011a0 0x7fc4881015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.362+0000 7fc48c927640 1 -- 172.21.15.94:0/724728222 >> 172.21.15.94:0/724728222 conn(0x7fc4880fc730 msgr2=0x7fc4880feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:19.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.362+0000 7fc48c927640 1 -- 172.21.15.94:0/724728222 shutdown_connections 2024-02-17T11:55:19.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.362+0000 7fc48c927640 1 -- 172.21.15.94:0/724728222 wait complete. 2024-02-17T11:55:19.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48c927640 1 Processor -- start 2024-02-17T11:55:19.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48c927640 1 -- start start 2024-02-17T11:55:19.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48c927640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4881011a0 0x7fc4881971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:19.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48c927640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc488102390 0x7fc488197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:19.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48c927640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc488102d50 0x7fc48819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:19.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48c927640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48819ece0 con 0x7fc4881011a0 2024-02-17T11:55:19.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48c927640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48819ee50 con 0x7fc488102390 2024-02-17T11:55:19.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48c927640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48819f010 con 0x7fc488102d50 2024-02-17T11:55:19.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4881011a0 0x7fc4881971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:19.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4881011a0 0x7fc4881971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42564/0 (socket says 172.21.15.94:42564) 2024-02-17T11:55:19.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48659c640 1 -- 172.21.15.94:0/379960561 learned_addr learned my addr 172.21.15.94:0/379960561 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:19.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc486d9d640 1 --2- 172.21.15.94:0/379960561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc488102d50 0x7fc48819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:19.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48659c640 1 -- 172.21.15.94:0/379960561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc488102d50 msgr2=0x7fc48819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:19.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48659c640 1 --2- 172.21.15.94:0/379960561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc488102d50 0x7fc48819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48659c640 1 -- 172.21.15.94:0/379960561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc488102390 msgr2=0x7fc488197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:19.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48659c640 1 --2- 172.21.15.94:0/379960561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc488102390 0x7fc488197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48659c640 1 -- 172.21.15.94:0/379960561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc478009560 con 0x7fc4881011a0 2024-02-17T11:55:19.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.363+0000 7fc48659c640 1 --2- 172.21.15.94:0/379960561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4881011a0 0x7fc4881971f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00c970 tx=0x7fc47c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:19.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.364+0000 7fc4777fe640 1 -- 172.21.15.94:0/379960561 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c004220 con 0x7fc4881011a0 2024-02-17T11:55:19.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.364+0000 7fc4777fe640 1 -- 172.21.15.94:0/379960561 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc47c007e90 con 0x7fc4881011a0 2024-02-17T11:55:19.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.364+0000 7fc48c927640 1 -- 172.21.15.94:0/379960561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48819f2f0 con 0x7fc4881011a0 2024-02-17T11:55:19.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.364+0000 7fc4777fe640 1 -- 172.21.15.94:0/379960561 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c00f450 con 0x7fc4881011a0 2024-02-17T11:55:19.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.364+0000 7fc48c927640 1 -- 172.21.15.94:0/379960561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48819f800 con 0x7fc4881011a0 2024-02-17T11:55:19.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.365+0000 7fc48c927640 1 -- 172.21.15.94:0/379960561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4881084c0 con 0x7fc4881011a0 2024-02-17T11:55:19.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.366+0000 7fc4777fe640 1 -- 172.21.15.94:0/379960561 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc47c003610 con 0x7fc4881011a0 2024-02-17T11:55:19.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.366+0000 7fc4777fe640 1 --2- 172.21.15.94:0/379960561 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc4700a20c0 0x7fc4700a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:19.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.366+0000 7fc4777fe640 1 -- 172.21.15.94:0/379960561 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc47c0cd580 con 0x7fc4881011a0 2024-02-17T11:55:19.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.366+0000 7fc485d9b640 1 --2- 172.21.15.94:0/379960561 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc4700a20c0 0x7fc4700a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:19.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.367+0000 7fc485d9b640 1 --2- 172.21.15.94:0/379960561 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc4700a20c0 0x7fc4700a4580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc47802c6a0 tx=0x7fc478000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.371+0000 7fc4777fe640 1 -- 172.21.15.94:0/379960561 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc47c0969f0 con 0x7fc4881011a0 2024-02-17T11:55:19.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.590+0000 7fc48c927640 1 -- 172.21.15.94:0/379960561 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc48810cbb0 con 0x7fc4881011a0 2024-02-17T11:55:19.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.590+0000 7fc4777fe640 1 -- 172.21.15.94:0/379960561 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fc47c09a0b0 con 0x7fc4881011a0 2024-02-17T11:55:19.589 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:19.589 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-02-17T11:55:19.589 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 1 2024-02-17T11:55:19.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.592+0000 7fc48c927640 1 -- 172.21.15.94:0/379960561 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc4700a20c0 msgr2=0x7fc4700a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:19.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.592+0000 7fc48c927640 1 --2- 172.21.15.94:0/379960561 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc4700a20c0 0x7fc4700a4580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc47802c6a0 tx=0x7fc478000bc0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.592+0000 7fc48c927640 1 -- 172.21.15.94:0/379960561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4881011a0 msgr2=0x7fc4881971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:19.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.592+0000 7fc48c927640 1 --2- 172.21.15.94:0/379960561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4881011a0 0x7fc4881971f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00c970 tx=0x7fc47c00ce30 comp rx=0 tx=0).stop 2024-02-17T11:55:19.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.592+0000 7fc48c927640 1 -- 172.21.15.94:0/379960561 shutdown_connections 2024-02-17T11:55:19.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.592+0000 7fc48c927640 1 --2- 172.21.15.94:0/379960561 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc4700a20c0 0x7fc4700a4580 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.592+0000 7fc48c927640 1 --2- 172.21.15.94:0/379960561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc488102d50 0x7fc48819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.592+0000 7fc48c927640 1 --2- 172.21.15.94:0/379960561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc488102390 0x7fc488197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.592+0000 7fc48c927640 1 --2- 172.21.15.94:0/379960561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4881011a0 0x7fc4881971f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:19.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.592+0000 7fc48c927640 1 -- 172.21.15.94:0/379960561 >> 172.21.15.94:0/379960561 conn(0x7fc4880fc730 msgr2=0x7fc488105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:19.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.592+0000 7fc48c927640 1 -- 172.21.15.94:0/379960561 shutdown_connections 2024-02-17T11:55:19.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:19.592+0000 7fc48c927640 1 -- 172.21.15.94:0/379960561 wait complete. 2024-02-17T11:55:19.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:19 smithi125 ceph-mon[36819]: Deploying daemon mds.l on smithi189 2024-02-17T11:55:19.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:19 smithi125 ceph-mon[36819]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:19.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:19 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3093745511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:55:19.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:19 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3978400344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:55:19.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:19 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/379960561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:19 smithi094 ceph-mon[32499]: Deploying daemon mds.l on smithi189 2024-02-17T11:55:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:19 smithi094 ceph-mon[32499]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:19 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3093745511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:55:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:19 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3978400344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:55:19.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:19 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/379960561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:20.008 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:19 smithi189 ceph-mon[36588]: Deploying daemon mds.l on smithi189 2024-02-17T11:55:20.008 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:19 smithi189 ceph-mon[36588]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:20.009 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:19 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3093745511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:55:20.009 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:19 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3978400344' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:55:20.009 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:19 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/379960561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:20.651 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 11:55:20 smithi189 systemd[1]: Starting Ceph mds.l for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:55:20.979 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 11:55:20 smithi189 podman[67057]: 2024-02-17 11:55:20.611297317 +0000 UTC m=+0.018072688 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T11:55:20.980 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 11:55:20 smithi189 podman[67057]: 2024-02-17 11:55:20.75413614 +0000 UTC m=+0.160911497 container create c9250c033e01451b4ec504b338e471cec4cb30d766a0b045ee0213eb2115bd73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, ceph=True, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-17T11:55:20.980 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 11:55:20 smithi189 podman[67057]: 2024-02-17 11:55:20.981721192 +0000 UTC m=+0.388496551 container init c9250c033e01451b4ec504b338e471cec4cb30d766a0b045ee0213eb2115bd73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-17T11:55:21.400 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 11:55:20 smithi189 podman[67057]: 2024-02-17 11:55:20.984419894 +0000 UTC m=+0.391195259 container start c9250c033e01451b4ec504b338e471cec4cb30d766a0b045ee0213eb2115bd73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-17T11:55:21.401 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 11:55:21 smithi189 ceph-mds[67076]: -- 172.21.15.189:0/1533545960 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55728ccf21a0 con 0x55728ccc8800 2024-02-17T11:55:21.401 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 11:55:21 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: starting mds.l at 2024-02-17T11:55:21.401 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 11:55:21 smithi189 bash[67057]: c9250c033e01451b4ec504b338e471cec4cb30d766a0b045ee0213eb2115bd73 2024-02-17T11:55:21.401 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 11:55:21 smithi189 systemd[1]: Started Ceph mds.l for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:55:21.602 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.719+0000 7f08adcc0640 1 -- 172.21.15.94:0/1661215001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a81011a0 msgr2=0x7f08a81015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.719+0000 7f08adcc0640 1 --2- 172.21.15.94:0/1661215001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a81011a0 0x7f08a81015c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0898009900 tx=0x7f089802bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.719+0000 7f08adcc0640 1 -- 172.21.15.94:0/1661215001 shutdown_connections 2024-02-17T11:55:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.719+0000 7f08adcc0640 1 --2- 172.21.15.94:0/1661215001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a8102d50 0x7f08a81095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.719+0000 7f08adcc0640 1 --2- 172.21.15.94:0/1661215001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8102390 0x7f08a8102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.719+0000 7f08adcc0640 1 --2- 172.21.15.94:0/1661215001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a81011a0 0x7f08a81015c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.719+0000 7f08adcc0640 1 -- 172.21.15.94:0/1661215001 >> 172.21.15.94:0/1661215001 conn(0x7f08a80fc730 msgr2=0x7f08a80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.719+0000 7f08adcc0640 1 -- 172.21.15.94:0/1661215001 shutdown_connections 2024-02-17T11:55:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.719+0000 7f08adcc0640 1 -- 172.21.15.94:0/1661215001 wait complete. 2024-02-17T11:55:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.719+0000 7f08adcc0640 1 Processor -- start 2024-02-17T11:55:21.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08adcc0640 1 -- start start 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08adcc0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a81011a0 0x7f08a8074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08adcc0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8102390 0x7f08a8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08adcc0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8102d50 0x7f08a8073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08adcc0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a8074210 con 0x7f08a8102390 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08adcc0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a8074380 con 0x7f08a8102d50 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08adcc0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a81a1250 con 0x7f08a81011a0 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8102390 0x7f08a8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8102390 0x7f08a8073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42580/0 (socket says 172.21.15.94:42580) 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08a6ffd640 1 -- 172.21.15.94:0/1619565391 learned_addr learned my addr 172.21.15.94:0/1619565391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08a7fff640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8102d50 0x7f08a8073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08a77fe640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a81011a0 0x7f08a8074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08a6ffd640 1 -- 172.21.15.94:0/1619565391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a81011a0 msgr2=0x7f08a8074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08a6ffd640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a81011a0 0x7f08a8074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:21.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08a6ffd640 1 -- 172.21.15.94:0/1619565391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8102d50 msgr2=0x7f08a8073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08a6ffd640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8102d50 0x7f08a8073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08a6ffd640 1 -- 172.21.15.94:0/1619565391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0898009560 con 0x7f08a8102390 2024-02-17T11:55:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08a77fe640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a81011a0 0x7f08a8074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:55:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08a6ffd640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8102390 0x7f08a8073570 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f089c00c8c0 tx=0x7f089c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.720+0000 7f08a4ff9640 1 -- 172.21.15.94:0/1619565391 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c004220 con 0x7f08a8102390 2024-02-17T11:55:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.721+0000 7f08a4ff9640 1 -- 172.21.15.94:0/1619565391 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f089c007e90 con 0x7f08a8102390 2024-02-17T11:55:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.721+0000 7f08adcc0640 1 -- 172.21.15.94:0/1619565391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a81a1500 con 0x7f08a8102390 2024-02-17T11:55:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.721+0000 7f08a4ff9640 1 -- 172.21.15.94:0/1619565391 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c004a00 con 0x7f08a8102390 2024-02-17T11:55:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.721+0000 7f08adcc0640 1 -- 172.21.15.94:0/1619565391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a81a1a10 con 0x7f08a8102390 2024-02-17T11:55:21.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.721+0000 7f08adcc0640 1 -- 172.21.15.94:0/1619565391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a81084c0 con 0x7f08a8102390 2024-02-17T11:55:21.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.723+0000 7f08a4ff9640 1 -- 172.21.15.94:0/1619565391 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f089c0079b0 con 0x7f08a8102390 2024-02-17T11:55:21.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.723+0000 7f08a4ff9640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08900a20c0 0x7f08900a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.723+0000 7f08a77fe640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08900a20c0 0x7f08900a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.724+0000 7f08a4ff9640 1 -- 172.21.15.94:0/1619565391 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f089c0cd4f0 con 0x7f08a8102390 2024-02-17T11:55:21.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.724+0000 7f08a77fe640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08900a20c0 0x7f08900a4580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0898006010 tx=0x7f089802c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:21.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.728+0000 7f08a4ff9640 1 -- 172.21.15.94:0/1619565391 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f089c096960 con 0x7f08a8102390 2024-02-17T11:55:21.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.936+0000 7f08adcc0640 1 -- 172.21.15.94:0/1619565391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08a810cb70 con 0x7f08a8102390 2024-02-17T11:55:21.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.937+0000 7f08a4ff9640 1 -- 172.21.15.94:0/1619565391 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f089c09a020 con 0x7f08a8102390 2024-02-17T11:55:21.936 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:21.936 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-17T11:55:21.936 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 2 2024-02-17T11:55:21.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.938+0000 7f08adcc0640 1 -- 172.21.15.94:0/1619565391 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08900a20c0 msgr2=0x7f08900a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:21.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.938+0000 7f08adcc0640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08900a20c0 0x7f08900a4580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0898006010 tx=0x7f089802c800 comp rx=0 tx=0).stop 2024-02-17T11:55:21.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.938+0000 7f08adcc0640 1 -- 172.21.15.94:0/1619565391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8102390 msgr2=0x7f08a8073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:21.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.938+0000 7f08adcc0640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8102390 0x7f08a8073570 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f089c00c8c0 tx=0x7f089c00cd80 comp rx=0 tx=0).stop 2024-02-17T11:55:21.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.939+0000 7f08adcc0640 1 -- 172.21.15.94:0/1619565391 shutdown_connections 2024-02-17T11:55:21.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.939+0000 7f08adcc0640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08900a20c0 0x7f08900a4580 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:21.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.939+0000 7f08adcc0640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08a8102d50 0x7f08a8073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:21.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.939+0000 7f08adcc0640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08a8102390 0x7f08a8073570 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:21.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.939+0000 7f08adcc0640 1 --2- 172.21.15.94:0/1619565391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08a81011a0 0x7f08a8074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:21.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.939+0000 7f08adcc0640 1 -- 172.21.15.94:0/1619565391 >> 172.21.15.94:0/1619565391 conn(0x7f08a80fc730 msgr2=0x7f08a8105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:21.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.939+0000 7f08adcc0640 1 -- 172.21.15.94:0/1619565391 shutdown_connections 2024-02-17T11:55:21.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:21.939+0000 7f08adcc0640 1 -- 172.21.15.94:0/1619565391 wait complete. 2024-02-17T11:55:21.947 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:21 smithi094 ceph-mon[32499]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:21.947 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:21 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.189:6832/1397235152,v1:172.21.15.189:6833/1397235152] up:boot 2024-02-17T11:55:21.947 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:21 smithi094 ceph-mon[32499]: fsmap 1 up:standby 2024-02-17T11:55:21.947 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T11:55:21.947 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:21.948 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:21.948 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:21.948 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:21.948 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:21.948 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:21 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:21.949 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:21 smithi125 ceph-mon[36819]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:21.949 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:21 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.189:6832/1397235152,v1:172.21.15.189:6833/1397235152] up:boot 2024-02-17T11:55:21.949 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:21 smithi125 ceph-mon[36819]: fsmap 1 up:standby 2024-02-17T11:55:21.949 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T11:55:21.949 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:21.949 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:21.949 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:21.949 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:21.949 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:21.949 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:21 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:22.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:21 smithi189 ceph-mon[36588]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:22.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:21 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.189:6832/1397235152,v1:172.21.15.189:6833/1397235152] up:boot 2024-02-17T11:55:22.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:21 smithi189 ceph-mon[36588]: fsmap 1 up:standby 2024-02-17T11:55:22.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T11:55:22.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:22.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:22.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:22.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:22.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:22.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:21 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:22.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:22 smithi094 ceph-mon[32499]: Deploying daemon mds.k on smithi125 2024-02-17T11:55:22.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:22 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1619565391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:22.993 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:22 smithi125 ceph-mon[36819]: Deploying daemon mds.k on smithi125 2024-02-17T11:55:22.993 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:22 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1619565391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:23.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:22 smithi189 ceph-mon[36588]: Deploying daemon mds.k on smithi125 2024-02-17T11:55:23.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:22 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1619565391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:23.372 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 11:55:23 smithi125 systemd[1]: Starting Ceph mds.k for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:55:23.684 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 11:55:23 smithi125 podman[66466]: 2024-02-17 11:55:23.491055789 +0000 UTC m=+0.017767814 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T11:55:23.684 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 11:55:23 smithi125 podman[66466]: 2024-02-17 11:55:23.614626811 +0000 UTC m=+0.141338821 container create db4b881c9c9bebca26bb5e27ab82eeac4e6e8c1c7bedf3098a231527b46adab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-k, io.buildah.version=1.33.5, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-17T11:55:23.951 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:23.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:23 smithi094 ceph-mon[32499]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:23.966 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:23 smithi125 ceph-mon[36819]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:23.966 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 11:55:23 smithi125 podman[66466]: 2024-02-17 11:55:23.850529165 +0000 UTC m=+0.377241186 container init db4b881c9c9bebca26bb5e27ab82eeac4e6e8c1c7bedf3098a231527b46adab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-k, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, io.buildah.version=1.33.5) 2024-02-17T11:55:23.966 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 11:55:23 smithi125 podman[66466]: 2024-02-17 11:55:23.85332918 +0000 UTC m=+0.380041200 container start db4b881c9c9bebca26bb5e27ab82eeac4e6e8c1c7bedf3098a231527b46adab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-02-17T11:55:23.966 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 11:55:23 smithi125 ceph-mds[66485]: -- 172.21.15.125:0/197018058 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b5c99ca1a0 con 0x55b5c99a0c00 2024-02-17T11:55:23.966 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 11:55:23 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-k[66481]: starting mds.k at 2024-02-17T11:55:23.967 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 11:55:23 smithi125 bash[66466]: db4b881c9c9bebca26bb5e27ab82eeac4e6e8c1c7bedf3098a231527b46adab1 2024-02-17T11:55:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.069+0000 7f0eedb7a640 1 -- 172.21.15.94:0/1507721739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee81023b0 msgr2=0x7f0ee8102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.069+0000 7f0eedb7a640 1 --2- 172.21.15.94:0/1507721739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee81023b0 0x7f0ee8102830 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009900 tx=0x7f0edc02bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.069+0000 7f0eedb7a640 1 -- 172.21.15.94:0/1507721739 shutdown_connections 2024-02-17T11:55:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.069+0000 7f0eedb7a640 1 --2- 172.21.15.94:0/1507721739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ee8102d70 0x7f0ee8109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.069+0000 7f0eedb7a640 1 --2- 172.21.15.94:0/1507721739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee81023b0 0x7f0ee8102830 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.069+0000 7f0eedb7a640 1 --2- 172.21.15.94:0/1507721739 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ee81011c0 0x7f0ee81015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.069+0000 7f0eedb7a640 1 -- 172.21.15.94:0/1507721739 >> 172.21.15.94:0/1507721739 conn(0x7f0ee80fc730 msgr2=0x7f0ee80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.069+0000 7f0eedb7a640 1 -- 172.21.15.94:0/1507721739 shutdown_connections 2024-02-17T11:55:24.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.069+0000 7f0eedb7a640 1 -- 172.21.15.94:0/1507721739 wait complete. 2024-02-17T11:55:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.069+0000 7f0eedb7a640 1 Processor -- start 2024-02-17T11:55:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0eedb7a640 1 -- start start 2024-02-17T11:55:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0eedb7a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee81011c0 0x7f0ee8197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0eedb7a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ee81023b0 0x7f0ee8197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0eedb7a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ee8102d70 0x7f0ee819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0eedb7a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee819ed20 con 0x7f0ee81011c0 2024-02-17T11:55:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0eedb7a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee819ee90 con 0x7f0ee8102d70 2024-02-17T11:55:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0eedb7a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee819f050 con 0x7f0ee81023b0 2024-02-17T11:55:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee81011c0 0x7f0ee8197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee81011c0 0x7f0ee8197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42596/0 (socket says 172.21.15.94:42596) 2024-02-17T11:55:24.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0ee77fe640 1 -- 172.21.15.94:0/3911636254 learned_addr learned my addr 172.21.15.94:0/3911636254 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0ee6ffd640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ee81023b0 0x7f0ee8197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0ee7fff640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ee8102d70 0x7f0ee819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0ee77fe640 1 -- 172.21.15.94:0/3911636254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ee81023b0 msgr2=0x7f0ee8197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ee81023b0 0x7f0ee8197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0ee77fe640 1 -- 172.21.15.94:0/3911636254 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ee8102d70 msgr2=0x7f0ee819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ee8102d70 0x7f0ee819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.070+0000 7f0ee77fe640 1 -- 172.21.15.94:0/3911636254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc009560 con 0x7f0ee81011c0 2024-02-17T11:55:24.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.071+0000 7f0ee77fe640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee81011c0 0x7f0ee8197230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800e9c0 tx=0x7f0ed800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.073+0000 7f0ee4ff9640 1 -- 172.21.15.94:0/3911636254 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8009700 con 0x7f0ee81011c0 2024-02-17T11:55:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.073+0000 7f0ee4ff9640 1 -- 172.21.15.94:0/3911636254 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ed800cd80 con 0x7f0ee81011c0 2024-02-17T11:55:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.073+0000 7f0eedb7a640 1 -- 172.21.15.94:0/3911636254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee819f330 con 0x7f0ee81011c0 2024-02-17T11:55:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.073+0000 7f0ee4ff9640 1 -- 172.21.15.94:0/3911636254 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8010430 con 0x7f0ee81011c0 2024-02-17T11:55:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.073+0000 7f0eedb7a640 1 -- 172.21.15.94:0/3911636254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee819f810 con 0x7f0ee81011c0 2024-02-17T11:55:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.074+0000 7f0eedb7a640 1 -- 172.21.15.94:0/3911636254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee8108530 con 0x7f0ee81011c0 2024-02-17T11:55:24.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.075+0000 7f0ee4ff9640 1 -- 172.21.15.94:0/3911636254 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0ed8003680 con 0x7f0ee81011c0 2024-02-17T11:55:24.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.075+0000 7f0ee4ff9640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0ed00a20c0 0x7f0ed00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:24.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.075+0000 7f0ee4ff9640 1 -- 172.21.15.94:0/3911636254 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0ed8014070 con 0x7f0ee81011c0 2024-02-17T11:55:24.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.075+0000 7f0ee6ffd640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0ed00a20c0 0x7f0ed00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:24.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.076+0000 7f0ee6ffd640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0ed00a20c0 0x7f0ed00a4580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009230 tx=0x7f0edc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:24.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.080+0000 7f0ee4ff9640 1 -- 172.21.15.94:0/3911636254 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ed8096fc0 con 0x7f0ee81011c0 2024-02-17T11:55:24.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:23 smithi189 ceph-mon[36588]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:24.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.298+0000 7f0eedb7a640 1 -- 172.21.15.94:0/3911636254 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ee810cbd0 con 0x7f0ee81011c0 2024-02-17T11:55:24.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.298+0000 7f0ee4ff9640 1 -- 172.21.15.94:0/3911636254 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7f0ed809a680 con 0x7f0ee81011c0 2024-02-17T11:55:24.297 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:24.297 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-17T11:55:24.297 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 2 2024-02-17T11:55:24.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.300+0000 7f0eedb7a640 1 -- 172.21.15.94:0/3911636254 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0ed00a20c0 msgr2=0x7f0ed00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:24.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.300+0000 7f0eedb7a640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0ed00a20c0 0x7f0ed00a4580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0edc009230 tx=0x7f0edc005fd0 comp rx=0 tx=0).stop 2024-02-17T11:55:24.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.300+0000 7f0eedb7a640 1 -- 172.21.15.94:0/3911636254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee81011c0 msgr2=0x7f0ee8197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:24.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.300+0000 7f0eedb7a640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee81011c0 0x7f0ee8197230 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800e9c0 tx=0x7f0ed800ee80 comp rx=0 tx=0).stop 2024-02-17T11:55:24.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.300+0000 7f0eedb7a640 1 -- 172.21.15.94:0/3911636254 shutdown_connections 2024-02-17T11:55:24.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.300+0000 7f0eedb7a640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0ed00a20c0 0x7f0ed00a4580 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:24.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.300+0000 7f0eedb7a640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ee8102d70 0x7f0ee819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:24.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.300+0000 7f0eedb7a640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ee81023b0 0x7f0ee8197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:24.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.300+0000 7f0eedb7a640 1 --2- 172.21.15.94:0/3911636254 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ee81011c0 0x7f0ee8197230 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:24.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.300+0000 7f0eedb7a640 1 -- 172.21.15.94:0/3911636254 >> 172.21.15.94:0/3911636254 conn(0x7f0ee80fc730 msgr2=0x7f0ee8105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:24.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.300+0000 7f0eedb7a640 1 -- 172.21.15.94:0/3911636254 shutdown_connections 2024-02-17T11:55:24.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:24.300+0000 7f0eedb7a640 1 -- 172.21.15.94:0/3911636254 wait complete. 2024-02-17T11:55:24.434 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 11:55:23 smithi125 systemd[1]: Started Ceph mds.k for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:55:25.286 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:25.287 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:25.287 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:25.287 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:25.287 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:25.287 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:25.287 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:25 smithi125 ceph-mon[36819]: Deploying daemon mds.j on smithi094 2024-02-17T11:55:25.287 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:25 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3911636254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:25.287 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:25 smithi125 ceph-mon[36819]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:25.287 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:25 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.125:6832/1464839618,v1:172.21.15.125:6833/1464839618] up:boot 2024-02-17T11:55:25.288 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:25 smithi125 ceph-mon[36819]: fsmap 2 up:standby 2024-02-17T11:55:25.288 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:25 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T11:55:25.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:25.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:25 smithi189 ceph-mon[36588]: Deploying daemon mds.j on smithi094 2024-02-17T11:55:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:25 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3911636254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:25 smithi189 ceph-mon[36588]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:25 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.125:6832/1464839618,v1:172.21.15.125:6833/1464839618] up:boot 2024-02-17T11:55:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:25 smithi189 ceph-mon[36588]: fsmap 2 up:standby 2024-02-17T11:55:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:25 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T11:55:25.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:25.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:25.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:25.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:25.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:25.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:25.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:25 smithi094 ceph-mon[32499]: Deploying daemon mds.j on smithi094 2024-02-17T11:55:25.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:25 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3911636254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:25.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:25 smithi094 ceph-mon[32499]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:25.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:25 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.125:6832/1464839618,v1:172.21.15.125:6833/1464839618] up:boot 2024-02-17T11:55:25.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:25 smithi094 ceph-mon[32499]: fsmap 2 up:standby 2024-02-17T11:55:25.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:25 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T11:55:26.311 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:26.339 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 11:55:26 smithi094 systemd[1]: Starting Ceph mds.j for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:55:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.420+0000 7fbe20240640 1 -- 172.21.15.94:0/2465305899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101c20 msgr2=0x7fbe18104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.420+0000 7fbe20240640 1 --2- 172.21.15.94:0/2465305899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101c20 0x7fbe18104010 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009ca0 tx=0x7fbe1402c0f0 comp rx=0 tx=0).stop 2024-02-17T11:55:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 -- 172.21.15.94:0/2465305899 shutdown_connections 2024-02-17T11:55:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 --2- 172.21.15.94:0/2465305899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe18106e80 0x7fbe181092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 --2- 172.21.15.94:0/2465305899 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe18104550 0x7fbe18106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 --2- 172.21.15.94:0/2465305899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101c20 0x7fbe18104010 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 -- 172.21.15.94:0/2465305899 >> 172.21.15.94:0/2465305899 conn(0x7fbe180fb610 msgr2=0x7fbe180fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 -- 172.21.15.94:0/2465305899 shutdown_connections 2024-02-17T11:55:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 -- 172.21.15.94:0/2465305899 wait complete. 2024-02-17T11:55:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 Processor -- start 2024-02-17T11:55:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 -- start start 2024-02-17T11:55:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101c20 0x7fbe181971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe18104550 0x7fbe18197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe18106e80 0x7fbe1819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1819ecc0 con 0x7fbe18101c20 2024-02-17T11:55:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1819ee30 con 0x7fbe18104550 2024-02-17T11:55:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe20240640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1819eff0 con 0x7fbe18106e80 2024-02-17T11:55:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe1dfdc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101c20 0x7fbe181971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe1dfdc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101c20 0x7fbe181971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35550/0 (socket says 172.21.15.94:35550) 2024-02-17T11:55:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe1dfdc640 1 -- 172.21.15.94:0/3938428730 learned_addr learned my addr 172.21.15.94:0/3938428730 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.421+0000 7fbe1d7db640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe18104550 0x7fbe18197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.422+0000 7fbe1e7dd640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe18106e80 0x7fbe1819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.422+0000 7fbe1dfdc640 1 -- 172.21.15.94:0/3938428730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe18106e80 msgr2=0x7fbe1819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.422+0000 7fbe1dfdc640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe18106e80 0x7fbe1819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.422+0000 7fbe1dfdc640 1 -- 172.21.15.94:0/3938428730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe18104550 msgr2=0x7fbe18197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.422+0000 7fbe1dfdc640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe18104550 0x7fbe18197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.422+0000 7fbe1dfdc640 1 -- 172.21.15.94:0/3938428730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe14009900 con 0x7fbe18101c20 2024-02-17T11:55:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.422+0000 7fbe1e7dd640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe18106e80 0x7fbe1819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:55:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.422+0000 7fbe1dfdc640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101c20 0x7fbe181971d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009dc0 tx=0x7fbe14034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.422+0000 7fbe0effd640 1 -- 172.21.15.94:0/3938428730 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe14032eb0 con 0x7fbe18101c20 2024-02-17T11:55:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.422+0000 7fbe20240640 1 -- 172.21.15.94:0/3938428730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1819f270 con 0x7fbe18101c20 2024-02-17T11:55:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.422+0000 7fbe0effd640 1 -- 172.21.15.94:0/3938428730 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbe14003e50 con 0x7fbe18101c20 2024-02-17T11:55:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.422+0000 7fbe0effd640 1 -- 172.21.15.94:0/3938428730 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe140406b0 con 0x7fbe18101c20 2024-02-17T11:55:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.422+0000 7fbe20240640 1 -- 172.21.15.94:0/3938428730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1819f770 con 0x7fbe18101c20 2024-02-17T11:55:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.423+0000 7fbe20240640 1 -- 172.21.15.94:0/3938428730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe180fd100 con 0x7fbe18101c20 2024-02-17T11:55:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.424+0000 7fbe0effd640 1 -- 172.21.15.94:0/3938428730 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fbe1400a040 con 0x7fbe18101c20 2024-02-17T11:55:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.425+0000 7fbe0effd640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbe040a20c0 0x7fbe040a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.425+0000 7fbe1d7db640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbe040a20c0 0x7fbe040a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:26.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.425+0000 7fbe0effd640 1 -- 172.21.15.94:0/3938428730 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbe140f0b60 con 0x7fbe18101c20 2024-02-17T11:55:26.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.425+0000 7fbe1d7db640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbe040a20c0 0x7fbe040a4580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe181986e0 tx=0x7fbe0800a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:26.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.429+0000 7fbe0effd640 1 -- 172.21.15.94:0/3938428730 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe140ba0d0 con 0x7fbe18101c20 2024-02-17T11:55:26.604 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 11:55:26 smithi094 podman[73287]: 2024-02-17 11:55:26.341282101 +0000 UTC m=+0.017870644 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T11:55:26.604 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 11:55:26 smithi094 podman[73287]: 2024-02-17 11:55:26.466307297 +0000 UTC m=+0.142895823 container create 8085e8836ad5a9b7c8dfe8ae8808bba3dd7f074aa9623826d8375d44143683d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-j, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, RELEASE=main-6822c7b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T11:55:26.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.645+0000 7fbe20240640 1 -- 172.21.15.94:0/3938428730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe1810cbb0 con 0x7fbe18101c20 2024-02-17T11:55:26.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.645+0000 7fbe0effd640 1 -- 172.21.15.94:0/3938428730 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7fbe140bd790 con 0x7fbe18101c20 2024-02-17T11:55:26.645 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:26.645 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-17T11:55:26.645 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 3 2024-02-17T11:55:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.647+0000 7fbe20240640 1 -- 172.21.15.94:0/3938428730 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbe040a20c0 msgr2=0x7fbe040a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.647+0000 7fbe20240640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbe040a20c0 0x7fbe040a4580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbe181986e0 tx=0x7fbe0800a2d0 comp rx=0 tx=0).stop 2024-02-17T11:55:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.647+0000 7fbe20240640 1 -- 172.21.15.94:0/3938428730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101c20 msgr2=0x7fbe181971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.647+0000 7fbe20240640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101c20 0x7fbe181971d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009dc0 tx=0x7fbe14034d40 comp rx=0 tx=0).stop 2024-02-17T11:55:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.647+0000 7fbe20240640 1 -- 172.21.15.94:0/3938428730 shutdown_connections 2024-02-17T11:55:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.647+0000 7fbe20240640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbe040a20c0 0x7fbe040a4580 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.647+0000 7fbe20240640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe18106e80 0x7fbe1819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.647+0000 7fbe20240640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbe18104550 0x7fbe18197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.647+0000 7fbe20240640 1 --2- 172.21.15.94:0/3938428730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbe18101c20 0x7fbe181971d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.647+0000 7fbe20240640 1 -- 172.21.15.94:0/3938428730 >> 172.21.15.94:0/3938428730 conn(0x7fbe180fb610 msgr2=0x7fbe181026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.647+0000 7fbe20240640 1 -- 172.21.15.94:0/3938428730 shutdown_connections 2024-02-17T11:55:26.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:26.647+0000 7fbe20240640 1 -- 172.21.15.94:0/3938428730 wait complete. 2024-02-17T11:55:26.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:26 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3938428730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:26.955 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 11:55:26 smithi094 podman[73287]: 2024-02-17 11:55:26.694366052 +0000 UTC m=+0.370954581 container init 8085e8836ad5a9b7c8dfe8ae8808bba3dd7f074aa9623826d8375d44143683d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-j, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , ceph=True) 2024-02-17T11:55:26.955 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 11:55:26 smithi094 podman[73287]: 2024-02-17 11:55:26.697733324 +0000 UTC m=+0.374321849 container start 8085e8836ad5a9b7c8dfe8ae8808bba3dd7f074aa9623826d8375d44143683d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-j, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T11:55:26.955 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 11:55:26 smithi094 ceph-mds[73344]: -- 172.21.15.94:0/3522061776 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5580f42ce1a0 con 0x5580f42a4800 2024-02-17T11:55:26.955 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 11:55:26 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-j[73338]: starting mds.j at 2024-02-17T11:55:26.955 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 11:55:26 smithi094 bash[73287]: 8085e8836ad5a9b7c8dfe8ae8808bba3dd7f074aa9623826d8375d44143683d1 2024-02-17T11:55:26.955 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 11:55:26 smithi094 systemd[1]: Started Ceph mds.j for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:55:27.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:26 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3938428730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:27.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:26 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3938428730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:27.806 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:27 smithi094 ceph-mon[32499]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:27.806 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:27.806 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:27.806 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:27.806 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:27.806 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:27.806 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:27 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:28.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:27 smithi189 ceph-mon[36588]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:28.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:28.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:28.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:28.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:28.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:28.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:27 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:28.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:27 smithi125 ceph-mon[36819]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:28.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:28.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:28.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:28.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:28.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:28.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:27 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:28.658 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:28.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.777+0000 7f440e20e640 1 -- 172.21.15.94:0/1179986473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44081023b0 msgr2=0x7f4408102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:28.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.777+0000 7f440e20e640 1 --2- 172.21.15.94:0/1179986473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44081023b0 0x7f4408102830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009900 tx=0x7f43f002bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:28.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.777+0000 7f440e20e640 1 -- 172.21.15.94:0/1179986473 shutdown_connections 2024-02-17T11:55:28.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.777+0000 7f440e20e640 1 --2- 172.21.15.94:0/1179986473 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4408102d70 0x7f4408109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:28.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.777+0000 7f440e20e640 1 --2- 172.21.15.94:0/1179986473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44081023b0 0x7f4408102830 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:28.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.777+0000 7f440e20e640 1 --2- 172.21.15.94:0/1179986473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44081011c0 0x7f44081015e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:28.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.777+0000 7f440e20e640 1 -- 172.21.15.94:0/1179986473 >> 172.21.15.94:0/1179986473 conn(0x7f44080fc730 msgr2=0x7f44080feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.777+0000 7f440e20e640 1 -- 172.21.15.94:0/1179986473 shutdown_connections 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.777+0000 7f440e20e640 1 -- 172.21.15.94:0/1179986473 wait complete. 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f440e20e640 1 Processor -- start 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f440e20e640 1 -- start start 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f440e20e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44081011c0 0x7f4408197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f440e20e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44081023b0 0x7f44081977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f440e20e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4408102d70 0x7f440819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f440e20e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440819ed80 con 0x7f44081023b0 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f440e20e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440819eef0 con 0x7f4408102d70 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f440e20e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440819f0b0 con 0x7f44081011c0 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f43feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44081023b0 0x7f44081977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f43feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44081023b0 0x7f44081977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35588/0 (socket says 172.21.15.94:35588) 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f43feffd640 1 -- 172.21.15.94:0/1156358461 learned_addr learned my addr 172.21.15.94:0/1156358461 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f44077fe640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44081011c0 0x7f4408197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:28.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f43feffd640 1 -- 172.21.15.94:0/1156358461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44081011c0 msgr2=0x7f4408197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:28.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f4407fff640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4408102d70 0x7f440819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:28.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f43feffd640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44081011c0 0x7f4408197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:28.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f43feffd640 1 -- 172.21.15.94:0/1156358461 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4408102d70 msgr2=0x7f440819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:28.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f43feffd640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4408102d70 0x7f440819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:28.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f43feffd640 1 -- 172.21.15.94:0/1156358461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f0009560 con 0x7f44081023b0 2024-02-17T11:55:28.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.778+0000 7f44077fe640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44081011c0 0x7f4408197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:55:28.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.779+0000 7f43feffd640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44081023b0 0x7f44081977d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f43f0006010 tx=0x7f43f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:28.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.779+0000 7f44057fa640 1 -- 172.21.15.94:0/1156358461 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f00044e0 con 0x7f44081023b0 2024-02-17T11:55:28.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.779+0000 7f44057fa640 1 -- 172.21.15.94:0/1156358461 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43f00054e0 con 0x7f44081023b0 2024-02-17T11:55:28.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.779+0000 7f440e20e640 1 -- 172.21.15.94:0/1156358461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440819f330 con 0x7f44081023b0 2024-02-17T11:55:28.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.779+0000 7f44057fa640 1 -- 172.21.15.94:0/1156358461 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f003e450 con 0x7f44081023b0 2024-02-17T11:55:28.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.779+0000 7f440e20e640 1 -- 172.21.15.94:0/1156358461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440819f840 con 0x7f44081023b0 2024-02-17T11:55:28.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.780+0000 7f440e20e640 1 -- 172.21.15.94:0/1156358461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4408108560 con 0x7f44081023b0 2024-02-17T11:55:28.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.782+0000 7f44057fa640 1 -- 172.21.15.94:0/1156358461 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f43f00035f0 con 0x7f44081023b0 2024-02-17T11:55:28.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.782+0000 7f44057fa640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f43f40a20c0 0x7f43f40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:28.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.782+0000 7f44077fe640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f43f40a20c0 0x7f43f40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:28.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.782+0000 7f44057fa640 1 -- 172.21.15.94:0/1156358461 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f43f00f0260 con 0x7f44081023b0 2024-02-17T11:55:28.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.783+0000 7f44077fe640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f43f40a20c0 0x7f43f40a4580 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4408102210 tx=0x7f43f8005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:28.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:28.786+0000 7f44057fa640 1 -- 172.21.15.94:0/1156358461 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43f00b96d0 con 0x7f44081023b0 2024-02-17T11:55:28.948 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:28 smithi094 ceph-mon[32499]: Deploying daemon mds.d on smithi094 2024-02-17T11:55:28.948 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:28 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.94:6834/2052147813,v1:172.21.15.94:6835/2052147813] up:boot 2024-02-17T11:55:28.948 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:28 smithi094 ceph-mon[32499]: fsmap 3 up:standby 2024-02-17T11:55:28.948 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:28 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-17T11:55:29.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.016+0000 7f440e20e640 1 -- 172.21.15.94:0/1156358461 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f440810cbd0 con 0x7f44081023b0 2024-02-17T11:55:29.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.016+0000 7f44057fa640 1 -- 172.21.15.94:0/1156358461 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2913 (secure 0 0 0) 0x7f43f00bcd90 con 0x7f44081023b0 2024-02-17T11:55:29.015 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:29.015 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-17T11:55:29.016 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 4 2024-02-17T11:55:29.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.018+0000 7f440e20e640 1 -- 172.21.15.94:0/1156358461 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f43f40a20c0 msgr2=0x7f43f40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.018+0000 7f440e20e640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f43f40a20c0 0x7f43f40a4580 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4408102210 tx=0x7f43f8005e70 comp rx=0 tx=0).stop 2024-02-17T11:55:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.018+0000 7f440e20e640 1 -- 172.21.15.94:0/1156358461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44081023b0 msgr2=0x7f44081977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.018+0000 7f440e20e640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44081023b0 0x7f44081977d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f43f0006010 tx=0x7f43f00042f0 comp rx=0 tx=0).stop 2024-02-17T11:55:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.018+0000 7f440e20e640 1 -- 172.21.15.94:0/1156358461 shutdown_connections 2024-02-17T11:55:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.018+0000 7f440e20e640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f43f40a20c0 0x7f43f40a4580 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.018+0000 7f440e20e640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4408102d70 0x7f440819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.018+0000 7f440e20e640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44081023b0 0x7f44081977d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.018+0000 7f440e20e640 1 --2- 172.21.15.94:0/1156358461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44081011c0 0x7f4408197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.018+0000 7f440e20e640 1 -- 172.21.15.94:0/1156358461 >> 172.21.15.94:0/1156358461 conn(0x7f44080fc730 msgr2=0x7f4408105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.019+0000 7f440e20e640 1 -- 172.21.15.94:0/1156358461 shutdown_connections 2024-02-17T11:55:29.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:29.019+0000 7f440e20e640 1 -- 172.21.15.94:0/1156358461 wait complete. 2024-02-17T11:55:29.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:28 smithi189 ceph-mon[36588]: Deploying daemon mds.d on smithi094 2024-02-17T11:55:29.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:28 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.94:6834/2052147813,v1:172.21.15.94:6835/2052147813] up:boot 2024-02-17T11:55:29.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:28 smithi189 ceph-mon[36588]: fsmap 3 up:standby 2024-02-17T11:55:29.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:28 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-17T11:55:29.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:28 smithi125 ceph-mon[36819]: Deploying daemon mds.d on smithi094 2024-02-17T11:55:29.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:28 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.94:6834/2052147813,v1:172.21.15.94:6835/2052147813] up:boot 2024-02-17T11:55:29.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:28 smithi125 ceph-mon[36819]: fsmap 3 up:standby 2024-02-17T11:55:29.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:28 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-17T11:55:29.205 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 11:55:29 smithi094 systemd[1]: Starting Ceph mds.d for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:55:29.658 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 11:55:29 smithi094 podman[73674]: 2024-02-17 11:55:29.291481995 +0000 UTC m=+0.018067855 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T11:55:29.658 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 11:55:29 smithi094 podman[73674]: 2024-02-17 11:55:29.407350986 +0000 UTC m=+0.133936846 container create 005a4583f339358fa83ac416104cbbc5b0368f6845000180e7a572c80186da23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-02-17T11:55:29.658 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 11:55:29 smithi094 podman[73674]: 2024-02-17 11:55:29.660326747 +0000 UTC m=+0.386912608 container init 005a4583f339358fa83ac416104cbbc5b0368f6845000180e7a572c80186da23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-d, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, GIT_CLEAN=True) 2024-02-17T11:55:29.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:29 smithi094 ceph-mon[32499]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:29.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:29 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1156358461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:29.955 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 11:55:29 smithi094 podman[73674]: 2024-02-17 11:55:29.663612949 +0000 UTC m=+0.390198811 container start 005a4583f339358fa83ac416104cbbc5b0368f6845000180e7a572c80186da23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, RELEASE=main-6822c7b) 2024-02-17T11:55:29.955 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 11:55:29 smithi094 ceph-mds[73693]: -- 172.21.15.94:0/1006963165 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555c64ca41a0 con 0x555c64c7a800 2024-02-17T11:55:29.955 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 11:55:29 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-d[73689]: starting mds.d at 2024-02-17T11:55:29.955 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 11:55:29 smithi094 bash[73674]: 005a4583f339358fa83ac416104cbbc5b0368f6845000180e7a572c80186da23 2024-02-17T11:55:29.955 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 11:55:29 smithi094 systemd[1]: Started Ceph mds.d for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:55:30.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:29 smithi189 ceph-mon[36588]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:30.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:29 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1156358461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:30.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:29 smithi125 ceph-mon[36819]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:30.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:29 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1156358461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:30.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:30 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.94:6836/2565649123,v1:172.21.15.94:6837/2565649123] up:boot 2024-02-17T11:55:30.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:30 smithi094 ceph-mon[32499]: fsmap 4 up:standby 2024-02-17T11:55:30.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:30 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-17T11:55:30.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:30 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:30.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:30 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:30.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:30 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:30.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:30 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:30.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:30 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:30.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:30 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:31.034 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:31.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.142+0000 7f4759a6f640 1 -- 172.21.15.94:0/3245520135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4754075140 msgr2=0x7f4754073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:31.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.142+0000 7f4759a6f640 1 --2- 172.21.15.94:0/3245520135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4754075140 0x7f4754073590 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4744009900 tx=0x7f474402bcd0 comp rx=0 tx=0).stop 2024-02-17T11:55:31.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.142+0000 7f4759a6f640 1 -- 172.21.15.94:0/3245520135 shutdown_connections 2024-02-17T11:55:31.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.142+0000 7f4759a6f640 1 --2- 172.21.15.94:0/3245520135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4754109070 0x7f475410b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:31.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.142+0000 7f4759a6f640 1 --2- 172.21.15.94:0/3245520135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4754073b60 0x7f4754073fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:31.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.142+0000 7f4759a6f640 1 --2- 172.21.15.94:0/3245520135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4754075140 0x7f4754073590 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:31.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.142+0000 7f4759a6f640 1 -- 172.21.15.94:0/3245520135 >> 172.21.15.94:0/3245520135 conn(0x7f47540fc730 msgr2=0x7f47540feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:31.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.142+0000 7f4759a6f640 1 -- 172.21.15.94:0/3245520135 shutdown_connections 2024-02-17T11:55:31.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.142+0000 7f4759a6f640 1 -- 172.21.15.94:0/3245520135 wait complete. 2024-02-17T11:55:31.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.142+0000 7f4759a6f640 1 Processor -- start 2024-02-17T11:55:31.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.142+0000 7f4759a6f640 1 -- start start 2024-02-17T11:55:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f4759a6f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4754073b60 0x7f47541993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f4759a6f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4754075140 0x7f4754199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f4759a6f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4754109070 0x7f47541a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f4759a6f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47541a0eb0 con 0x7f4754075140 2024-02-17T11:55:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f4759a6f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47541a1020 con 0x7f4754109070 2024-02-17T11:55:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f4759a6f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47541a11e0 con 0x7f4754073b60 2024-02-17T11:55:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f474affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4754075140 0x7f4754199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f474affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4754075140 0x7f4754199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35632/0 (socket says 172.21.15.94:35632) 2024-02-17T11:55:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f474affd640 1 -- 172.21.15.94:0/2737443470 learned_addr learned my addr 172.21.15.94:0/2737443470 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:31.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f4753fff640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4754109070 0x7f47541a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f47537fe640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4754073b60 0x7f47541993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f474affd640 1 -- 172.21.15.94:0/2737443470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4754073b60 msgr2=0x7f47541993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f474affd640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4754073b60 0x7f47541993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f474affd640 1 -- 172.21.15.94:0/2737443470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4754109070 msgr2=0x7f47541a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f474affd640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4754109070 0x7f47541a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f474affd640 1 -- 172.21.15.94:0/2737443470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4744009560 con 0x7f4754075140 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f4753fff640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4754109070 0x7f47541a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f474affd640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4754075140 0x7f4754199900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c920 tx=0x7f473c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f47517fa640 1 -- 172.21.15.94:0/2737443470 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c007c70 con 0x7f4754075140 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f47517fa640 1 -- 172.21.15.94:0/2737443470 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f473c007dd0 con 0x7f4754075140 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f4759a6f640 1 -- 172.21.15.94:0/2737443470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47541a14c0 con 0x7f4754075140 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f47517fa640 1 -- 172.21.15.94:0/2737443470 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c018860 con 0x7f4754075140 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.143+0000 7f4759a6f640 1 -- 172.21.15.94:0/2737443470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47541a19d0 con 0x7f4754075140 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.144+0000 7f4759a6f640 1 -- 172.21.15.94:0/2737443470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4754107e30 con 0x7f4754075140 2024-02-17T11:55:31.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.145+0000 7f47517fa640 1 -- 172.21.15.94:0/2737443470 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f473c00fac0 con 0x7f4754075140 2024-02-17T11:55:31.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.146+0000 7f47517fa640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f47400a20c0 0x7f47400a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:31.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.146+0000 7f47537fe640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f47400a20c0 0x7f47400a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:31.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.146+0000 7f47517fa640 1 -- 172.21.15.94:0/2737443470 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f473c0cd8a0 con 0x7f4754075140 2024-02-17T11:55:31.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.146+0000 7f47537fe640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f47400a20c0 0x7f47400a4580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4744009230 tx=0x7f4744005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:31.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.150+0000 7f47517fa640 1 -- 172.21.15.94:0/2737443470 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f473c096d10 con 0x7f4754075140 2024-02-17T11:55:31.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:30 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.94:6836/2565649123,v1:172.21.15.94:6837/2565649123] up:boot 2024-02-17T11:55:31.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:30 smithi189 ceph-mon[36588]: fsmap 4 up:standby 2024-02-17T11:55:31.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:30 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-17T11:55:31.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:30 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:31.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:30 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:31.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:30 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:31.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:30 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:31.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:30 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:31.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:30 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:31.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:30 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.94:6836/2565649123,v1:172.21.15.94:6837/2565649123] up:boot 2024-02-17T11:55:31.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:30 smithi125 ceph-mon[36819]: fsmap 4 up:standby 2024-02-17T11:55:31.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:30 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-17T11:55:31.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:30 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:31.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:30 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:31.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:30 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:31.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:30 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:31.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:30 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:31.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:30 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.363+0000 7f4759a6f640 1 -- 172.21.15.94:0/2737443470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f475410eda0 con 0x7f4754075140 2024-02-17T11:55:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.363+0000 7f47517fa640 1 -- 172.21.15.94:0/2737443470 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f473c09a3d0 con 0x7f4754075140 2024-02-17T11:55:31.362 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:31.363 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-02-17T11:55:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 5 2024-02-17T11:55:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.365+0000 7f4759a6f640 1 -- 172.21.15.94:0/2737443470 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f47400a20c0 msgr2=0x7f47400a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.365+0000 7f4759a6f640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f47400a20c0 0x7f47400a4580 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4744009230 tx=0x7f4744005fd0 comp rx=0 tx=0).stop 2024-02-17T11:55:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.365+0000 7f4759a6f640 1 -- 172.21.15.94:0/2737443470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4754075140 msgr2=0x7f4754199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.365+0000 7f4759a6f640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4754075140 0x7f4754199900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f473c00c920 tx=0x7f473c00cde0 comp rx=0 tx=0).stop 2024-02-17T11:55:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.365+0000 7f4759a6f640 1 -- 172.21.15.94:0/2737443470 shutdown_connections 2024-02-17T11:55:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.365+0000 7f4759a6f640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f47400a20c0 0x7f47400a4580 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.365+0000 7f4759a6f640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4754109070 0x7f47541a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.365+0000 7f4759a6f640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4754075140 0x7f4754199900 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.365+0000 7f4759a6f640 1 --2- 172.21.15.94:0/2737443470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4754073b60 0x7f47541993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.365+0000 7f4759a6f640 1 -- 172.21.15.94:0/2737443470 >> 172.21.15.94:0/2737443470 conn(0x7f47540fc730 msgr2=0x7f4754105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.366+0000 7f4759a6f640 1 -- 172.21.15.94:0/2737443470 shutdown_connections 2024-02-17T11:55:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:31.366+0000 7f4759a6f640 1 -- 172.21.15.94:0/2737443470 wait complete. 2024-02-17T11:55:32.140 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:31 smithi125 ceph-mon[36819]: Deploying daemon mds.e on smithi125 2024-02-17T11:55:32.140 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:31 smithi125 ceph-mon[36819]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:32.140 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:31 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2737443470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:32.140 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 11:55:31 smithi125 systemd[1]: Starting Ceph mds.e for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:55:32.141 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 11:55:32 smithi125 podman[66786]: 2024-02-17 11:55:32.044536257 +0000 UTC m=+0.017468672 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T11:55:32.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:31 smithi189 ceph-mon[36588]: Deploying daemon mds.e on smithi125 2024-02-17T11:55:32.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:31 smithi189 ceph-mon[36588]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:32.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:31 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2737443470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:32.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:31 smithi094 ceph-mon[32499]: Deploying daemon mds.e on smithi125 2024-02-17T11:55:32.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:31 smithi094 ceph-mon[32499]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:32.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:31 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2737443470' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:32.411 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 11:55:32 smithi125 podman[66786]: 2024-02-17 11:55:32.143506406 +0000 UTC m=+0.116438806 container create 0eaf5a395ba020c55f4c2b8aaffc650450f2d9804ac3c5efefe3562f09e12847 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-e, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-17T11:55:32.411 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 11:55:32 smithi125 podman[66786]: 2024-02-17 11:55:32.387851358 +0000 UTC m=+0.360783765 container init 0eaf5a395ba020c55f4c2b8aaffc650450f2d9804ac3c5efefe3562f09e12847 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-e, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T11:55:32.411 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 11:55:32 smithi125 podman[66786]: 2024-02-17 11:55:32.390375275 +0000 UTC m=+0.363307685 container start 0eaf5a395ba020c55f4c2b8aaffc650450f2d9804ac3c5efefe3562f09e12847 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-e, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-17T11:55:32.412 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 11:55:32 smithi125 ceph-mds[66805]: -- 172.21.15.125:0/948850253 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55986c4e01a0 con 0x55986c4b7000 2024-02-17T11:55:32.684 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 11:55:32 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-e[66801]: starting mds.e at 2024-02-17T11:55:32.684 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 11:55:32 smithi125 bash[66786]: 0eaf5a395ba020c55f4c2b8aaffc650450f2d9804ac3c5efefe3562f09e12847 2024-02-17T11:55:32.684 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 11:55:32 smithi125 systemd[1]: Started Ceph mds.e for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:55:33.376 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:33.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.491+0000 7f69eb535640 1 -- 172.21.15.94:0/4154114458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69e41000c0 msgr2=0x7f69e41004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:33.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.491+0000 7f69eb535640 1 --2- 172.21.15.94:0/4154114458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69e41000c0 0x7f69e41004e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f69d4009900 tx=0x7f69d402bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:33.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.491+0000 7f69eb535640 1 -- 172.21.15.94:0/4154114458 shutdown_connections 2024-02-17T11:55:33.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.491+0000 7f69eb535640 1 --2- 172.21.15.94:0/4154114458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69e4101c70 0x7f69e4108500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:33.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.491+0000 7f69eb535640 1 --2- 172.21.15.94:0/4154114458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e41012b0 0x7f69e4101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:33.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.491+0000 7f69eb535640 1 --2- 172.21.15.94:0/4154114458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69e41000c0 0x7f69e41004e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:33.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.491+0000 7f69eb535640 1 -- 172.21.15.94:0/4154114458 >> 172.21.15.94:0/4154114458 conn(0x7f69e40fb630 msgr2=0x7f69e40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:33.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.491+0000 7f69eb535640 1 -- 172.21.15.94:0/4154114458 shutdown_connections 2024-02-17T11:55:33.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.491+0000 7f69eb535640 1 -- 172.21.15.94:0/4154114458 wait complete. 2024-02-17T11:55:33.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.491+0000 7f69eb535640 1 Processor -- start 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.491+0000 7f69eb535640 1 -- start start 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69eb535640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e41000c0 0x7f69e41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69eb535640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69e41012b0 0x7f69e4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69eb535640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69e4101c70 0x7f69e419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69eb535640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e419edc0 con 0x7f69e4101c70 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69eb535640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e419ef30 con 0x7f69e41012b0 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69eb535640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e419f0f0 con 0x7f69e41000c0 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69e9ad2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69e4101c70 0x7f69e419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69e9ad2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69e4101c70 0x7f69e419e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35652/0 (socket says 172.21.15.94:35652) 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69e9ad2640 1 -- 172.21.15.94:0/4066606717 learned_addr learned my addr 172.21.15.94:0/4066606717 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69e8ad0640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69e41012b0 0x7f69e4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69e92d1640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e41000c0 0x7f69e41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69e9ad2640 1 -- 172.21.15.94:0/4066606717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e41000c0 msgr2=0x7f69e41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:33.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69e9ad2640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e41000c0 0x7f69e41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:33.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69e9ad2640 1 -- 172.21.15.94:0/4066606717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69e41012b0 msgr2=0x7f69e4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:33.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69e9ad2640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69e41012b0 0x7f69e4197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:33.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69e9ad2640 1 -- 172.21.15.94:0/4066606717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d4009560 con 0x7f69e4101c70 2024-02-17T11:55:33.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69e9ad2640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69e4101c70 0x7f69e419e880 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009fd0 tx=0x7f69dc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:33.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69da7fc640 1 -- 172.21.15.94:0/4066606717 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc00bea0 con 0x7f69e4101c70 2024-02-17T11:55:33.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69da7fc640 1 -- 172.21.15.94:0/4066606717 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69dc009940 con 0x7f69e4101c70 2024-02-17T11:55:33.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69eb535640 1 -- 172.21.15.94:0/4066606717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e419f3d0 con 0x7f69e4101c70 2024-02-17T11:55:33.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69da7fc640 1 -- 172.21.15.94:0/4066606717 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc0106c0 con 0x7f69e4101c70 2024-02-17T11:55:33.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.492+0000 7f69eb535640 1 -- 172.21.15.94:0/4066606717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e419f890 con 0x7f69e4101c70 2024-02-17T11:55:33.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.493+0000 7f69eb535640 1 -- 172.21.15.94:0/4066606717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e410bad0 con 0x7f69e4101c70 2024-02-17T11:55:33.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.494+0000 7f69da7fc640 1 -- 172.21.15.94:0/4066606717 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f69dc004240 con 0x7f69e4101c70 2024-02-17T11:55:33.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.495+0000 7f69da7fc640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f69d00a20c0 0x7f69d00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:33.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.495+0000 7f69e92d1640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f69d00a20c0 0x7f69d00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:33.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.495+0000 7f69da7fc640 1 -- 172.21.15.94:0/4066606717 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f69dc0cd4f0 con 0x7f69e4101c70 2024-02-17T11:55:33.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.495+0000 7f69e92d1640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f69d00a20c0 0x7f69d00a4580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f69d4006010 tx=0x7f69d4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:33.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.498+0000 7f69da7fc640 1 -- 172.21.15.94:0/4066606717 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f69dc096960 con 0x7f69e4101c70 2024-02-17T11:55:33.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:33 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:33.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:33 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:33.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:33 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:33.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:33 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:33.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:33 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:33.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:33 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:33.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:33 smithi094 ceph-mon[32499]: Deploying daemon mds.f on smithi189 2024-02-17T11:55:33.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:33 smithi094 ceph-mon[32499]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:33.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:33 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.125:6834/1238777175,v1:172.21.15.125:6835/1238777175] up:boot 2024-02-17T11:55:33.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:33 smithi094 ceph-mon[32499]: fsmap 5 up:standby 2024-02-17T11:55:33.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:33 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T11:55:33.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.710+0000 7f69eb535640 1 -- 172.21.15.94:0/4066606717 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69e410bc60 con 0x7f69e4101c70 2024-02-17T11:55:33.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.711+0000 7f69da7fc640 1 -- 172.21.15.94:0/4066606717 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4498 (secure 0 0 0) 0x7f69dc09a020 con 0x7f69e4101c70 2024-02-17T11:55:33.710 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:33.710 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-02-17T11:55:33.711 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 6 2024-02-17T11:55:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.713+0000 7f69eb535640 1 -- 172.21.15.94:0/4066606717 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f69d00a20c0 msgr2=0x7f69d00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.713+0000 7f69eb535640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f69d00a20c0 0x7f69d00a4580 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f69d4006010 tx=0x7f69d4000bc0 comp rx=0 tx=0).stop 2024-02-17T11:55:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.713+0000 7f69eb535640 1 -- 172.21.15.94:0/4066606717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69e4101c70 msgr2=0x7f69e419e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.713+0000 7f69eb535640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69e4101c70 0x7f69e419e880 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009fd0 tx=0x7f69dc00b260 comp rx=0 tx=0).stop 2024-02-17T11:55:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.713+0000 7f69eb535640 1 -- 172.21.15.94:0/4066606717 shutdown_connections 2024-02-17T11:55:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.713+0000 7f69eb535640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f69d00a20c0 0x7f69d00a4580 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:33.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.713+0000 7f69eb535640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f69e4101c70 0x7f69e419e880 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.713+0000 7f69eb535640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f69e41012b0 0x7f69e4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.713+0000 7f69eb535640 1 --2- 172.21.15.94:0/4066606717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f69e41000c0 0x7f69e41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.713+0000 7f69eb535640 1 -- 172.21.15.94:0/4066606717 >> 172.21.15.94:0/4066606717 conn(0x7f69e40fb630 msgr2=0x7f69e40697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.713+0000 7f69eb535640 1 -- 172.21.15.94:0/4066606717 shutdown_connections 2024-02-17T11:55:33.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:33.713+0000 7f69eb535640 1 -- 172.21.15.94:0/4066606717 wait complete. 2024-02-17T11:55:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:33 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:33 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:33 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:33 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:33 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:33 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:33 smithi189 ceph-mon[36588]: Deploying daemon mds.f on smithi189 2024-02-17T11:55:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:33 smithi189 ceph-mon[36588]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:33 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.125:6834/1238777175,v1:172.21.15.125:6835/1238777175] up:boot 2024-02-17T11:55:33.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:33 smithi189 ceph-mon[36588]: fsmap 5 up:standby 2024-02-17T11:55:33.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:33 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T11:55:33.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:33 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:33.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:33 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:33.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:33 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:33.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:33 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:33.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:33 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:33.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:33 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:33.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:33 smithi125 ceph-mon[36819]: Deploying daemon mds.f on smithi189 2024-02-17T11:55:33.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:33 smithi125 ceph-mon[36819]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:33.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:33 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.125:6834/1238777175,v1:172.21.15.125:6835/1238777175] up:boot 2024-02-17T11:55:33.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:33 smithi125 ceph-mon[36819]: fsmap 5 up:standby 2024-02-17T11:55:33.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:33 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T11:55:34.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:34 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/4066606717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:34.901 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 11:55:34 smithi189 systemd[1]: Starting Ceph mds.f for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:55:34.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:34 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/4066606717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:34.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:34 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/4066606717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:35.293 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 11:55:35 smithi189 podman[67365]: 2024-02-17 11:55:34.906496617 +0000 UTC m=+0.017829878 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T11:55:35.293 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 11:55:35 smithi189 podman[67365]: 2024-02-17 11:55:35.039824185 +0000 UTC m=+0.151157437 container create 7a8d6cf40c0e5fee39f5632879a333cb731d262723f770445b27adf455d8e095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS) 2024-02-17T11:55:35.293 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 11:55:35 smithi189 podman[67365]: 2024-02-17 11:55:35.294869256 +0000 UTC m=+0.406202536 container init 7a8d6cf40c0e5fee39f5632879a333cb731d262723f770445b27adf455d8e095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-17T11:55:35.621 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 11:55:35 smithi189 podman[67365]: 2024-02-17 11:55:35.297618344 +0000 UTC m=+0.408951606 container start 7a8d6cf40c0e5fee39f5632879a333cb731d262723f770445b27adf455d8e095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True) 2024-02-17T11:55:35.621 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 11:55:35 smithi189 ceph-mds[67384]: -- 172.21.15.189:0/3575934387 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5596fd77c1a0 con 0x5596fd753000 2024-02-17T11:55:35.621 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 11:55:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: starting mds.f at 2024-02-17T11:55:35.621 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 11:55:35 smithi189 bash[67365]: 7a8d6cf40c0e5fee39f5632879a333cb731d262723f770445b27adf455d8e095 2024-02-17T11:55:35.621 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 11:55:35 smithi189 systemd[1]: Started Ceph mds.f for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:55:35.723 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:35.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.838+0000 7fa447040640 1 -- 172.21.15.94:0/684386465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa440101c50 msgr2=0x7fa4401084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.838+0000 7fa447040640 1 --2- 172.21.15.94:0/684386465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa440101c50 0x7fa4401084e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa438009ca0 tx=0x7fa43802c0f0 comp rx=0 tx=0).stop 2024-02-17T11:55:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.838+0000 7fa447040640 1 -- 172.21.15.94:0/684386465 shutdown_connections 2024-02-17T11:55:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.838+0000 7fa447040640 1 --2- 172.21.15.94:0/684386465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa440101c50 0x7fa4401084e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.838+0000 7fa447040640 1 --2- 172.21.15.94:0/684386465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa440101290 0x7fa440101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.838+0000 7fa447040640 1 --2- 172.21.15.94:0/684386465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa4401000a0 0x7fa4401004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.838+0000 7fa447040640 1 -- 172.21.15.94:0/684386465 >> 172.21.15.94:0/684386465 conn(0x7fa4400fb610 msgr2=0x7fa4400fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.838+0000 7fa447040640 1 -- 172.21.15.94:0/684386465 shutdown_connections 2024-02-17T11:55:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.838+0000 7fa447040640 1 -- 172.21.15.94:0/684386465 wait complete. 2024-02-17T11:55:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.838+0000 7fa447040640 1 Processor -- start 2024-02-17T11:55:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.838+0000 7fa447040640 1 -- start start 2024-02-17T11:55:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.838+0000 7fa447040640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4401000a0 0x7fa440196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:35.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa447040640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa440101290 0x7fa4401965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa447040640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa440101c50 0x7fa44019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa447040640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44019db50 con 0x7fa440101290 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa447040640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44019dcc0 con 0x7fa440101c50 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa447040640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44019de80 con 0x7fa4401000a0 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa43ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa440101290 0x7fa4401965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa43ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa440101290 0x7fa4401965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51846/0 (socket says 172.21.15.94:51846) 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa43ffff640 1 -- 172.21.15.94:0/1270874237 learned_addr learned my addr 172.21.15.94:0/1270874237 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa444ddc640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4401000a0 0x7fa440196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa4455dd640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa440101c50 0x7fa44019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa43ffff640 1 -- 172.21.15.94:0/1270874237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4401000a0 msgr2=0x7fa440196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa43ffff640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4401000a0 0x7fa440196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa43ffff640 1 -- 172.21.15.94:0/1270874237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa440101c50 msgr2=0x7fa44019d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa43ffff640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa440101c50 0x7fa44019d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:35.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa43ffff640 1 -- 172.21.15.94:0/1270874237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa438009900 con 0x7fa440101290 2024-02-17T11:55:35.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa43ffff640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa440101290 0x7fa4401965a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00e9f0 tx=0x7fa42c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:35.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa43dffb640 1 -- 172.21.15.94:0/1270874237 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c009720 con 0x7fa440101290 2024-02-17T11:55:35.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa43dffb640 1 -- 172.21.15.94:0/1270874237 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa42c00cd80 con 0x7fa440101290 2024-02-17T11:55:35.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa447040640 1 -- 172.21.15.94:0/1270874237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44019e160 con 0x7fa440101290 2024-02-17T11:55:35.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.839+0000 7fa43dffb640 1 -- 172.21.15.94:0/1270874237 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c010690 con 0x7fa440101290 2024-02-17T11:55:35.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.840+0000 7fa447040640 1 -- 172.21.15.94:0/1270874237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44019e620 con 0x7fa440101290 2024-02-17T11:55:35.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.841+0000 7fa447040640 1 -- 172.21.15.94:0/1270874237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4401072f0 con 0x7fa440101290 2024-02-17T11:55:35.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.841+0000 7fa43dffb640 1 -- 172.21.15.94:0/1270874237 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa42c0046c0 con 0x7fa440101290 2024-02-17T11:55:35.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.842+0000 7fa43dffb640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa4300a20c0 0x7fa4300a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:35.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.842+0000 7fa444ddc640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa4300a20c0 0x7fa4300a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:35.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.842+0000 7fa43dffb640 1 -- 172.21.15.94:0/1270874237 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa42c014070 con 0x7fa440101290 2024-02-17T11:55:35.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.843+0000 7fa444ddc640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa4300a20c0 0x7fa4300a4580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa4401010f0 tx=0x7fa434005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:35.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:35.847+0000 7fa43dffb640 1 -- 172.21.15.94:0/1270874237 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa42c096fd0 con 0x7fa440101290 2024-02-17T11:55:35.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:35 smithi189 ceph-mon[36588]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:35.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:35.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:35.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:35.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:35.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:35.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:35 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:35.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:35 smithi125 ceph-mon[36819]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:35.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:35.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:35.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:35.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:35.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:35.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:35 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:35.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:35 smithi094 ceph-mon[32499]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:35.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:35.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:35.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:35.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:35.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:35.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:35 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:36.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.060+0000 7fa447040640 1 -- 172.21.15.94:0/1270874237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa44010bc60 con 0x7fa440101290 2024-02-17T11:55:36.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.060+0000 7fa43dffb640 1 -- 172.21.15.94:0/1270874237 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5292 (secure 0 0 0) 0x7fa42c09a690 con 0x7fa440101290 2024-02-17T11:55:36.060 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:36.060 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-02-17T11:55:36.060 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 7 2024-02-17T11:55:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.062+0000 7fa447040640 1 -- 172.21.15.94:0/1270874237 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa4300a20c0 msgr2=0x7fa4300a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.062+0000 7fa447040640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa4300a20c0 0x7fa4300a4580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa4401010f0 tx=0x7fa434005f90 comp rx=0 tx=0).stop 2024-02-17T11:55:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.062+0000 7fa447040640 1 -- 172.21.15.94:0/1270874237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa440101290 msgr2=0x7fa4401965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.062+0000 7fa447040640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa440101290 0x7fa4401965a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa42c00e9f0 tx=0x7fa42c00edf0 comp rx=0 tx=0).stop 2024-02-17T11:55:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.062+0000 7fa447040640 1 -- 172.21.15.94:0/1270874237 shutdown_connections 2024-02-17T11:55:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.062+0000 7fa447040640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa4300a20c0 0x7fa4300a4580 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.062+0000 7fa447040640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa440101c50 0x7fa44019d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.062+0000 7fa447040640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa440101290 0x7fa4401965a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.062+0000 7fa447040640 1 --2- 172.21.15.94:0/1270874237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa4401000a0 0x7fa440196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.062+0000 7fa447040640 1 -- 172.21.15.94:0/1270874237 >> 172.21.15.94:0/1270874237 conn(0x7fa4400fb610 msgr2=0x7fa4401044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.062+0000 7fa447040640 1 -- 172.21.15.94:0/1270874237 shutdown_connections 2024-02-17T11:55:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:36.062+0000 7fa447040640 1 -- 172.21.15.94:0/1270874237 wait complete. 2024-02-17T11:55:36.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:36 smithi189 ceph-mon[36588]: Deploying daemon mds.c on smithi189 2024-02-17T11:55:36.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:36 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.189:6834/2720928164,v1:172.21.15.189:6835/2720928164] up:boot 2024-02-17T11:55:36.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:36 smithi189 ceph-mon[36588]: fsmap 6 up:standby 2024-02-17T11:55:36.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:36 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T11:55:36.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:36 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1270874237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:36.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:36 smithi125 ceph-mon[36819]: Deploying daemon mds.c on smithi189 2024-02-17T11:55:36.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:36 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.189:6834/2720928164,v1:172.21.15.189:6835/2720928164] up:boot 2024-02-17T11:55:36.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:36 smithi125 ceph-mon[36819]: fsmap 6 up:standby 2024-02-17T11:55:36.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:36 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T11:55:36.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:36 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1270874237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:36.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:36 smithi094 ceph-mon[32499]: Deploying daemon mds.c on smithi189 2024-02-17T11:55:36.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:36 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.189:6834/2720928164,v1:172.21.15.189:6835/2720928164] up:boot 2024-02-17T11:55:36.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:36 smithi094 ceph-mon[32499]: fsmap 6 up:standby 2024-02-17T11:55:36.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:36 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T11:55:36.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:36 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1270874237' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:37.844 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:37 smithi189 ceph-mon[36588]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:37.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:37 smithi125 ceph-mon[36819]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:37.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:37 smithi094 ceph-mon[32499]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:38.073 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:38.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.185+0000 7fc2480fb640 1 -- 172.21.15.94:0/3385768862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc240101290 msgr2=0x7fc240101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:38.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.185+0000 7fc2480fb640 1 --2- 172.21.15.94:0/3385768862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc240101290 0x7fc240101710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc230009900 tx=0x7fc23002bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.185+0000 7fc2480fb640 1 -- 172.21.15.94:0/3385768862 shutdown_connections 2024-02-17T11:55:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.185+0000 7fc2480fb640 1 --2- 172.21.15.94:0/3385768862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc240101c50 0x7fc2401084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.185+0000 7fc2480fb640 1 --2- 172.21.15.94:0/3385768862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc240101290 0x7fc240101710 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.185+0000 7fc2480fb640 1 --2- 172.21.15.94:0/3385768862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2401000a0 0x7fc2401004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.185+0000 7fc2480fb640 1 -- 172.21.15.94:0/3385768862 >> 172.21.15.94:0/3385768862 conn(0x7fc2400fb610 msgr2=0x7fc2400fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.185+0000 7fc2480fb640 1 -- 172.21.15.94:0/3385768862 shutdown_connections 2024-02-17T11:55:38.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.185+0000 7fc2480fb640 1 -- 172.21.15.94:0/3385768862 wait complete. 2024-02-17T11:55:38.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc2480fb640 1 Processor -- start 2024-02-17T11:55:38.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc2480fb640 1 -- start start 2024-02-17T11:55:38.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc2480fb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2401000a0 0x7fc240068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:38.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc2480fb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc240101290 0x7fc24019e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:38.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc2480fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc240101c50 0x7fc2401a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:38.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc2480fb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2401a0fa0 con 0x7fc240101c50 2024-02-17T11:55:38.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc2480fb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2401a1110 con 0x7fc2401000a0 2024-02-17T11:55:38.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc2480fb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2401a12d0 con 0x7fc240101290 2024-02-17T11:55:38.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc246698640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc240101c50 0x7fc2401a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:38.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc246698640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc240101c50 0x7fc2401a08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51870/0 (socket says 172.21.15.94:51870) 2024-02-17T11:55:38.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc246698640 1 -- 172.21.15.94:0/3254170366 learned_addr learned my addr 172.21.15.94:0/3254170366 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:38.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc245696640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc240101290 0x7fc24019e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:38.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc245e97640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2401000a0 0x7fc240068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:38.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc246698640 1 -- 172.21.15.94:0/3254170366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc240101290 msgr2=0x7fc24019e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:38.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc246698640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc240101290 0x7fc24019e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:38.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc246698640 1 -- 172.21.15.94:0/3254170366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2401000a0 msgr2=0x7fc240068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:38.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc246698640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2401000a0 0x7fc240068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:38.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.186+0000 7fc246698640 1 -- 172.21.15.94:0/3254170366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc230009560 con 0x7fc240101c50 2024-02-17T11:55:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.187+0000 7fc246698640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc240101c50 0x7fc2401a08e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc23800eaa0 tx=0x7fc23800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.187+0000 7fc236ffd640 1 -- 172.21.15.94:0/3254170366 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23800cdb0 con 0x7fc240101c50 2024-02-17T11:55:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.187+0000 7fc236ffd640 1 -- 172.21.15.94:0/3254170366 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc238005020 con 0x7fc240101c50 2024-02-17T11:55:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.187+0000 7fc2480fb640 1 -- 172.21.15.94:0/3254170366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2401a1550 con 0x7fc240101c50 2024-02-17T11:55:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.187+0000 7fc236ffd640 1 -- 172.21.15.94:0/3254170366 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2380116b0 con 0x7fc240101c50 2024-02-17T11:55:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.187+0000 7fc2480fb640 1 -- 172.21.15.94:0/3254170366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2401a1a10 con 0x7fc240101c50 2024-02-17T11:55:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.188+0000 7fc2480fb640 1 -- 172.21.15.94:0/3254170366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2401073c0 con 0x7fc240101c50 2024-02-17T11:55:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.189+0000 7fc236ffd640 1 -- 172.21.15.94:0/3254170366 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc238005520 con 0x7fc240101c50 2024-02-17T11:55:38.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.189+0000 7fc236ffd640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc22c0a20f0 0x7fc22c0a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:38.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.189+0000 7fc245e97640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc22c0a20f0 0x7fc22c0a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:38.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.190+0000 7fc236ffd640 1 -- 172.21.15.94:0/3254170366 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc2380cdac0 con 0x7fc240101c50 2024-02-17T11:55:38.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.190+0000 7fc245e97640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc22c0a20f0 0x7fc22c0a45b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0058c0 tx=0x7fc23c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:38.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.194+0000 7fc236ffd640 1 -- 172.21.15.94:0/3254170366 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc238096f30 con 0x7fc240101c50 2024-02-17T11:55:38.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.413+0000 7fc2480fb640 1 -- 172.21.15.94:0/3254170366 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc24010bc60 con 0x7fc240101c50 2024-02-17T11:55:38.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.414+0000 7fc236ffd640 1 -- 172.21.15.94:0/3254170366 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5292 (secure 0 0 0) 0x7fc23809a5f0 con 0x7fc240101c50 2024-02-17T11:55:38.414 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:38.414 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-02-17T11:55:38.414 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 7 2024-02-17T11:55:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.416+0000 7fc2480fb640 1 -- 172.21.15.94:0/3254170366 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc22c0a20f0 msgr2=0x7fc22c0a45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.416+0000 7fc2480fb640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc22c0a20f0 0x7fc22c0a45b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0058c0 tx=0x7fc23c00a4c0 comp rx=0 tx=0).stop 2024-02-17T11:55:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.416+0000 7fc2480fb640 1 -- 172.21.15.94:0/3254170366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc240101c50 msgr2=0x7fc2401a08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.416+0000 7fc2480fb640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc240101c50 0x7fc2401a08e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc23800eaa0 tx=0x7fc23800ef60 comp rx=0 tx=0).stop 2024-02-17T11:55:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.416+0000 7fc2480fb640 1 -- 172.21.15.94:0/3254170366 shutdown_connections 2024-02-17T11:55:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.416+0000 7fc2480fb640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc22c0a20f0 0x7fc22c0a45b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.416+0000 7fc2480fb640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc240101c50 0x7fc2401a08e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.416+0000 7fc2480fb640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc240101290 0x7fc24019e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.416+0000 7fc2480fb640 1 --2- 172.21.15.94:0/3254170366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2401000a0 0x7fc240068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.416+0000 7fc2480fb640 1 -- 172.21.15.94:0/3254170366 >> 172.21.15.94:0/3254170366 conn(0x7fc2400fb610 msgr2=0x7fc240104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.416+0000 7fc2480fb640 1 -- 172.21.15.94:0/3254170366 shutdown_connections 2024-02-17T11:55:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:38.416+0000 7fc2480fb640 1 -- 172.21.15.94:0/3254170366 wait complete. 2024-02-17T11:55:38.650 INFO:journalctl@ceph.mds.c.smithi189.stdout:Feb 17 11:55:38 smithi189 systemd[1]: Started Ceph mds.c for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:55:39.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:39 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:39.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:39 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:39.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:39 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3254170366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:39.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:39 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:39.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:39 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:39.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:39 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:39.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:39 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:39.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:39 smithi189 ceph-mon[36588]: Deploying daemon mds.h on smithi125 2024-02-17T11:55:39.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:39 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] up:boot 2024-02-17T11:55:39.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:39 smithi189 ceph-mon[36588]: fsmap 7 up:standby 2024-02-17T11:55:39.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:39 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T11:55:39.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:39 smithi189 ceph-mon[36588]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:39.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:39 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:39.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:39 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:39.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:39 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3254170366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:39.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:39 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:39.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:39 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:39.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:39 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:39.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:39 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:39.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:39 smithi125 ceph-mon[36819]: Deploying daemon mds.h on smithi125 2024-02-17T11:55:39.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:39 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] up:boot 2024-02-17T11:55:39.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:39 smithi125 ceph-mon[36819]: fsmap 7 up:standby 2024-02-17T11:55:39.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:39 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T11:55:39.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:39 smithi125 ceph-mon[36819]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:39.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:39.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:39.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:39 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3254170366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:39.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:39.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:39.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:39 smithi094 ceph-mon[32499]: Deploying daemon mds.h on smithi125 2024-02-17T11:55:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:39 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] up:boot 2024-02-17T11:55:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:39 smithi094 ceph-mon[32499]: fsmap 7 up:standby 2024-02-17T11:55:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:39 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T11:55:39.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:39 smithi094 ceph-mon[32499]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:40.429 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:40.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.544+0000 7f6011a02640 1 -- 172.21.15.94:0/3193380315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f600c0ffea0 msgr2=0x7f600c1002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:40.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.544+0000 7f6011a02640 1 --2- 172.21.15.94:0/3193380315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f600c0ffea0 0x7f600c1002c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6000009900 tx=0x7f600002bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:40.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.544+0000 7f6011a02640 1 -- 172.21.15.94:0/3193380315 shutdown_connections 2024-02-17T11:55:40.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.544+0000 7f6011a02640 1 --2- 172.21.15.94:0/3193380315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f600c101a40 0x7f600c108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:40.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.544+0000 7f6011a02640 1 --2- 172.21.15.94:0/3193380315 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f600c101080 0x7f600c101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:40.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.544+0000 7f6011a02640 1 --2- 172.21.15.94:0/3193380315 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f600c0ffea0 0x7f600c1002c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:40.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.544+0000 7f6011a02640 1 -- 172.21.15.94:0/3193380315 >> 172.21.15.94:0/3193380315 conn(0x7f600c0fb460 msgr2=0x7f600c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:40.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.544+0000 7f6011a02640 1 -- 172.21.15.94:0/3193380315 shutdown_connections 2024-02-17T11:55:40.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.544+0000 7f6011a02640 1 -- 172.21.15.94:0/3193380315 wait complete. 2024-02-17T11:55:40.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.544+0000 7f6011a02640 1 Processor -- start 2024-02-17T11:55:40.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f6011a02640 1 -- start start 2024-02-17T11:55:40.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f6011a02640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f600c0ffea0 0x7f600c06a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:40.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f6011a02640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f600c101080 0x7f600c19bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:40.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f6011a02640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f600c101a40 0x7f600c19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:40.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f6011a02640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c19ebb0 con 0x7f600c101a40 2024-02-17T11:55:40.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f6011a02640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c19ed20 con 0x7f600c101080 2024-02-17T11:55:40.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f6011a02640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c19eee0 con 0x7f600c0ffea0 2024-02-17T11:55:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f600b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f600c101a40 0x7f600c19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f600b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f600c101a40 0x7f600c19e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51894/0 (socket says 172.21.15.94:51894) 2024-02-17T11:55:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f600b7fe640 1 -- 172.21.15.94:0/3832035136 learned_addr learned my addr 172.21.15.94:0/3832035136 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f600a7fc640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f600c101080 0x7f600c19bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f600affd640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f600c0ffea0 0x7f600c06a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f600b7fe640 1 -- 172.21.15.94:0/3832035136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f600c0ffea0 msgr2=0x7f600c06a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f600b7fe640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f600c0ffea0 0x7f600c06a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f600b7fe640 1 -- 172.21.15.94:0/3832035136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f600c101080 msgr2=0x7f600c19bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f600b7fe640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f600c101080 0x7f600c19bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.545+0000 7f600b7fe640 1 -- 172.21.15.94:0/3832035136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6000009560 con 0x7f600c101a40 2024-02-17T11:55:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.546+0000 7f600b7fe640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f600c101a40 0x7f600c19e420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f600400cad0 tx=0x7f600400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.546+0000 7f6010a00640 1 -- 172.21.15.94:0/3832035136 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400ae20 con 0x7f600c101a40 2024-02-17T11:55:40.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.546+0000 7f6011a02640 1 -- 172.21.15.94:0/3832035136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c19f1c0 con 0x7f600c101a40 2024-02-17T11:55:40.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.546+0000 7f6010a00640 1 -- 172.21.15.94:0/3832035136 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6004005020 con 0x7f600c101a40 2024-02-17T11:55:40.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.546+0000 7f6010a00640 1 -- 172.21.15.94:0/3832035136 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400f6b0 con 0x7f600c101a40 2024-02-17T11:55:40.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.546+0000 7f6011a02640 1 -- 172.21.15.94:0/3832035136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c19f720 con 0x7f600c101a40 2024-02-17T11:55:40.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.547+0000 7f6011a02640 1 -- 172.21.15.94:0/3832035136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600c1072c0 con 0x7f600c101a40 2024-02-17T11:55:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.548+0000 7f6010a00640 1 -- 172.21.15.94:0/3832035136 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f6004005190 con 0x7f600c101a40 2024-02-17T11:55:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.548+0000 7f6010a00640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5ff40a20c0 0x7f5ff40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.549+0000 7f600affd640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5ff40a20c0 0x7f5ff40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.549+0000 7f6010a00640 1 -- 172.21.15.94:0/3832035136 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f60040cd850 con 0x7f600c101a40 2024-02-17T11:55:40.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.549+0000 7f600affd640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5ff40a20c0 0x7f5ff40a4580 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6000006010 tx=0x7f6000000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:40.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.553+0000 7f6010a00640 1 -- 172.21.15.94:0/3832035136 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f600409a7f0 con 0x7f600c101a40 2024-02-17T11:55:40.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.763+0000 7f6011a02640 1 -- 172.21.15.94:0/3832035136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f600c10bac0 con 0x7f600c101a40 2024-02-17T11:55:40.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.763+0000 7f6010a00640 1 -- 172.21.15.94:0/3832035136 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7f600409a380 con 0x7f600c101a40 2024-02-17T11:55:40.762 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:40.763 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-17T11:55:40.763 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 8 2024-02-17T11:55:40.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.765+0000 7f6011a02640 1 -- 172.21.15.94:0/3832035136 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5ff40a20c0 msgr2=0x7f5ff40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:40.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.765+0000 7f6011a02640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5ff40a20c0 0x7f5ff40a4580 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6000006010 tx=0x7f6000000bc0 comp rx=0 tx=0).stop 2024-02-17T11:55:40.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.765+0000 7f6011a02640 1 -- 172.21.15.94:0/3832035136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f600c101a40 msgr2=0x7f600c19e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:40.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.765+0000 7f6011a02640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f600c101a40 0x7f600c19e420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f600400cad0 tx=0x7f600400cf90 comp rx=0 tx=0).stop 2024-02-17T11:55:40.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.765+0000 7f6011a02640 1 -- 172.21.15.94:0/3832035136 shutdown_connections 2024-02-17T11:55:40.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.765+0000 7f6011a02640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5ff40a20c0 0x7f5ff40a4580 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:40.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.765+0000 7f6011a02640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f600c101a40 0x7f600c19e420 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:40.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.765+0000 7f6011a02640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f600c101080 0x7f600c19bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:40.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.765+0000 7f6011a02640 1 --2- 172.21.15.94:0/3832035136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f600c0ffea0 0x7f600c06a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:40.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.765+0000 7f6011a02640 1 -- 172.21.15.94:0/3832035136 >> 172.21.15.94:0/3832035136 conn(0x7f600c0fb460 msgr2=0x7f600c104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:40.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.765+0000 7f6011a02640 1 -- 172.21.15.94:0/3832035136 shutdown_connections 2024-02-17T11:55:40.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:40.765+0000 7f6011a02640 1 -- 172.21.15.94:0/3832035136 wait complete. 2024-02-17T11:55:40.832 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 11:55:40 smithi125 systemd[1]: Starting Ceph mds.h for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:55:41.110 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 11:55:40 smithi125 podman[67092]: 2024-02-17 11:55:40.736524507 +0000 UTC m=+0.017973303 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T11:55:41.110 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 11:55:40 smithi125 podman[67092]: 2024-02-17 11:55:40.85086484 +0000 UTC m=+0.132313632 container create 0cfe22aa45829914a188a30cadff04a5f0ec492a515ae9ad008d94ade7339afd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-h, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-17T11:55:41.110 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 11:55:41 smithi125 podman[67092]: 2024-02-17 11:55:41.111911846 +0000 UTC m=+0.393360631 container init 0cfe22aa45829914a188a30cadff04a5f0ec492a515ae9ad008d94ade7339afd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-h, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T11:55:41.434 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 11:55:41 smithi125 podman[67092]: 2024-02-17 11:55:41.114640165 +0000 UTC m=+0.396088955 container start 0cfe22aa45829914a188a30cadff04a5f0ec492a515ae9ad008d94ade7339afd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T11:55:41.434 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 11:55:41 smithi125 ceph-mds[67111]: -- 172.21.15.125:0/568668127 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c71d5e81a0 con 0x55c71d5be800 2024-02-17T11:55:41.434 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 11:55:41 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-h[67107]: starting mds.h at 2024-02-17T11:55:41.435 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 11:55:41 smithi125 bash[67092]: 0cfe22aa45829914a188a30cadff04a5f0ec492a515ae9ad008d94ade7339afd 2024-02-17T11:55:41.435 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 11:55:41 smithi125 systemd[1]: Started Ceph mds.h for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:55:41.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:41 smithi125 ceph-mon[36819]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:41.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:41 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3832035136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:41.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:41 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:41.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:41 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:41.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:41 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:41.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:41 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:41.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:41 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:41.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:41 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:41 smithi094 ceph-mon[32499]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:41 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3832035136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:41 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:41 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:41 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:41 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:41.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:41 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:41.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:41 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:42.073 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:41 smithi189 ceph-mon[36588]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:42.073 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:41 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3832035136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:42.073 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:41 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:42.073 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:41 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:42.074 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:41 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:42.074 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:41 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:42.074 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:41 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:42.074 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:41 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:42.776 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:42.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.889+0000 7f0de975e640 1 -- 172.21.15.94:0/4116790857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0de4102440 msgr2=0x7f0de41028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:42.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.889+0000 7f0de975e640 1 --2- 172.21.15.94:0/4116790857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0de4102440 0x7f0de41028c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0dd8009900 tx=0x7f0dd802bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:42.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.889+0000 7f0de975e640 1 -- 172.21.15.94:0/4116790857 shutdown_connections 2024-02-17T11:55:42.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.889+0000 7f0de975e640 1 --2- 172.21.15.94:0/4116790857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0de4102e00 0x7f0de4109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:42.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.889+0000 7f0de975e640 1 --2- 172.21.15.94:0/4116790857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0de4102440 0x7f0de41028c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:42.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.889+0000 7f0de975e640 1 --2- 172.21.15.94:0/4116790857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0de40ff470 0x7f0de40ff890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:42.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.889+0000 7f0de975e640 1 -- 172.21.15.94:0/4116790857 >> 172.21.15.94:0/4116790857 conn(0x7f0de40fada0 msgr2=0x7f0de40fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:42.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.889+0000 7f0de975e640 1 -- 172.21.15.94:0/4116790857 shutdown_connections 2024-02-17T11:55:42.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.889+0000 7f0de975e640 1 -- 172.21.15.94:0/4116790857 wait complete. 2024-02-17T11:55:42.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.890+0000 7f0de975e640 1 Processor -- start 2024-02-17T11:55:42.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.890+0000 7f0de975e640 1 -- start start 2024-02-17T11:55:42.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.890+0000 7f0de975e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0de40ff470 0x7f0de4197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:42.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.890+0000 7f0de975e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0de4102440 0x7f0de4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:42.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.890+0000 7f0de975e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0de4102e00 0x7f0de419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:42.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.890+0000 7f0de975e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de419ed40 con 0x7f0de4102e00 2024-02-17T11:55:42.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.890+0000 7f0de975e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de419eeb0 con 0x7f0de4102440 2024-02-17T11:55:42.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.890+0000 7f0de975e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de419f070 con 0x7f0de40ff470 2024-02-17T11:55:42.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.890+0000 7f0de37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0de4102e00 0x7f0de419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.890+0000 7f0de2ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0de40ff470 0x7f0de4197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.890+0000 7f0de37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0de4102e00 0x7f0de419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51912/0 (socket says 172.21.15.94:51912) 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.890+0000 7f0de37fe640 1 -- 172.21.15.94:0/3487115532 learned_addr learned my addr 172.21.15.94:0/3487115532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.890+0000 7f0de27fc640 1 --2- 172.21.15.94:0/3487115532 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0de4102440 0x7f0de4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.891+0000 7f0de37fe640 1 -- 172.21.15.94:0/3487115532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0de40ff470 msgr2=0x7f0de4197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.891+0000 7f0de37fe640 1 --2- 172.21.15.94:0/3487115532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0de40ff470 0x7f0de4197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.891+0000 7f0de37fe640 1 -- 172.21.15.94:0/3487115532 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0de4102440 msgr2=0x7f0de4197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.891+0000 7f0de37fe640 1 --2- 172.21.15.94:0/3487115532 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0de4102440 0x7f0de4197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.891+0000 7f0de37fe640 1 -- 172.21.15.94:0/3487115532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd8009560 con 0x7f0de4102e00 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.891+0000 7f0de37fe640 1 --2- 172.21.15.94:0/3487115532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0de4102e00 0x7f0de419e800 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00eb10 tx=0x7f0ddc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.891+0000 7f0dcbfff640 1 -- 172.21.15.94:0/3487115532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc004b20 con 0x7f0de4102e00 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.891+0000 7f0dcbfff640 1 -- 172.21.15.94:0/3487115532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ddc004c80 con 0x7f0de4102e00 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.891+0000 7f0de975e640 1 -- 172.21.15.94:0/3487115532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de419f350 con 0x7f0de4102e00 2024-02-17T11:55:42.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.891+0000 7f0dcbfff640 1 -- 172.21.15.94:0/3487115532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc010650 con 0x7f0de4102e00 2024-02-17T11:55:42.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.891+0000 7f0de975e640 1 -- 172.21.15.94:0/3487115532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de419f7e0 con 0x7f0de4102e00 2024-02-17T11:55:42.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.892+0000 7f0de975e640 1 -- 172.21.15.94:0/3487115532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de4108660 con 0x7f0de4102e00 2024-02-17T11:55:42.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.893+0000 7f0dcbfff640 1 -- 172.21.15.94:0/3487115532 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f0ddc005020 con 0x7f0de4102e00 2024-02-17T11:55:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.894+0000 7f0dcbfff640 1 --2- 172.21.15.94:0/3487115532 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0dcc0a20c0 0x7f0dcc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.894+0000 7f0de2ffd640 1 --2- 172.21.15.94:0/3487115532 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0dcc0a20c0 0x7f0dcc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.894+0000 7f0dcbfff640 1 -- 172.21.15.94:0/3487115532 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0ddc014070 con 0x7f0de4102e00 2024-02-17T11:55:42.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.894+0000 7f0de2ffd640 1 --2- 172.21.15.94:0/3487115532 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0dcc0a20c0 0x7f0dcc0a4580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0de40695b0 tx=0x7f0dd4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:42.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:42.898+0000 7f0dcbfff640 1 -- 172.21.15.94:0/3487115532 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ddc097210 con 0x7f0de4102e00 2024-02-17T11:55:42.965 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:42 smithi189 ceph-mon[36588]: Deploying daemon mds.i on smithi189 2024-02-17T11:55:42.965 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:42 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.125:6836/1075914834,v1:172.21.15.125:6837/1075914834] up:boot 2024-02-17T11:55:42.965 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:42 smithi189 ceph-mon[36588]: fsmap 8 up:standby 2024-02-17T11:55:42.965 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:42 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T11:55:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.110+0000 7f0de975e640 1 -- 172.21.15.94:0/3487115532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0de410cbd0 con 0x7f0de4102e00 2024-02-17T11:55:43.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.111+0000 7f0dcbfff640 1 -- 172.21.15.94:0/3487115532 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7f0ddc09a8d0 con 0x7f0de4102e00 2024-02-17T11:55:43.110 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:43.110 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-02-17T11:55:43.110 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 9 2024-02-17T11:55:43.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.113+0000 7f0de975e640 1 -- 172.21.15.94:0/3487115532 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0dcc0a20c0 msgr2=0x7f0dcc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:43.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.113+0000 7f0de975e640 1 --2- 172.21.15.94:0/3487115532 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0dcc0a20c0 0x7f0dcc0a4580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0de40695b0 tx=0x7f0dd4005f90 comp rx=0 tx=0).stop 2024-02-17T11:55:43.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.113+0000 7f0de975e640 1 -- 172.21.15.94:0/3487115532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0de4102e00 msgr2=0x7f0de419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:43.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.113+0000 7f0de975e640 1 --2- 172.21.15.94:0/3487115532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0de4102e00 0x7f0de419e800 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00eb10 tx=0x7f0ddc00c2d0 comp rx=0 tx=0).stop 2024-02-17T11:55:43.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.113+0000 7f0de975e640 1 -- 172.21.15.94:0/3487115532 shutdown_connections 2024-02-17T11:55:43.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.113+0000 7f0de975e640 1 --2- 172.21.15.94:0/3487115532 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0dcc0a20c0 0x7f0dcc0a4580 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:43.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.113+0000 7f0de975e640 1 --2- 172.21.15.94:0/3487115532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0de4102e00 0x7f0de419e800 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:43.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.113+0000 7f0de975e640 1 --2- 172.21.15.94:0/3487115532 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0de4102440 0x7f0de4197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:43.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.113+0000 7f0de975e640 1 --2- 172.21.15.94:0/3487115532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0de40ff470 0x7f0de4197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:43.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.113+0000 7f0de975e640 1 -- 172.21.15.94:0/3487115532 >> 172.21.15.94:0/3487115532 conn(0x7f0de40fada0 msgr2=0x7f0de4105830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:43.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.113+0000 7f0de975e640 1 -- 172.21.15.94:0/3487115532 shutdown_connections 2024-02-17T11:55:43.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:43.113+0000 7f0de975e640 1 -- 172.21.15.94:0/3487115532 wait complete. 2024-02-17T11:55:43.122 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:42 smithi094 ceph-mon[32499]: Deploying daemon mds.i on smithi189 2024-02-17T11:55:43.122 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:42 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.125:6836/1075914834,v1:172.21.15.125:6837/1075914834] up:boot 2024-02-17T11:55:43.122 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:42 smithi094 ceph-mon[32499]: fsmap 8 up:standby 2024-02-17T11:55:43.122 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:42 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T11:55:43.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:42 smithi125 ceph-mon[36819]: Deploying daemon mds.i on smithi189 2024-02-17T11:55:43.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:42 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.125:6836/1075914834,v1:172.21.15.125:6837/1075914834] up:boot 2024-02-17T11:55:43.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:42 smithi125 ceph-mon[36819]: fsmap 8 up:standby 2024-02-17T11:55:43.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:42 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T11:55:43.651 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 11:55:43 smithi189 systemd[1]: Starting Ceph mds.i for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:55:43.928 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:43 smithi189 ceph-mon[36588]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:43.928 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:43 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3487115532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:43.928 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 11:55:43 smithi189 podman[67990]: 2024-02-17 11:55:43.561918711 +0000 UTC m=+0.017761083 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T11:55:43.928 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 11:55:43 smithi189 podman[67990]: 2024-02-17 11:55:43.685777825 +0000 UTC m=+0.141620217 container create d80d879d3207bcdf38e0018843a9278c62ae8705394b0bbd032b78ed53b1f9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-i, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-17T11:55:43.928 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 11:55:43 smithi189 podman[67990]: 2024-02-17 11:55:43.929925367 +0000 UTC m=+0.385767731 container init d80d879d3207bcdf38e0018843a9278c62ae8705394b0bbd032b78ed53b1f9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-i, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T11:55:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:43 smithi125 ceph-mon[36819]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:43 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3487115532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:44.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:43 smithi094 ceph-mon[32499]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:44.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:43 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3487115532' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:44.400 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 11:55:43 smithi189 podman[67990]: 2024-02-17 11:55:43.932614986 +0000 UTC m=+0.388457341 container start d80d879d3207bcdf38e0018843a9278c62ae8705394b0bbd032b78ed53b1f9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-i, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-17T11:55:44.401 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 11:55:43 smithi189 ceph-mds[68009]: -- 172.21.15.189:0/1184133476 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5588ef1cc1a0 con 0x5588ef1a2800 2024-02-17T11:55:44.401 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 11:55:43 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-i[68005]: starting mds.i at 2024-02-17T11:55:44.401 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 11:55:44 smithi189 bash[67990]: d80d879d3207bcdf38e0018843a9278c62ae8705394b0bbd032b78ed53b1f9ee 2024-02-17T11:55:44.401 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 11:55:44 smithi189 systemd[1]: Started Ceph mds.i for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:55:45.124 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:45.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.231+0000 7f4e6478c640 1 -- 172.21.15.94:0/1023734897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e5c102d50 msgr2=0x7f4e5c1095e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:45.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.231+0000 7f4e6478c640 1 --2- 172.21.15.94:0/1023734897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e5c102d50 0x7f4e5c1095e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009ca0 tx=0x7f4e5402c0f0 comp rx=0 tx=0).stop 2024-02-17T11:55:45.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.231+0000 7f4e6478c640 1 -- 172.21.15.94:0/1023734897 shutdown_connections 2024-02-17T11:55:45.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.231+0000 7f4e6478c640 1 --2- 172.21.15.94:0/1023734897 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e5c102d50 0x7f4e5c1095e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:45.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.231+0000 7f4e6478c640 1 --2- 172.21.15.94:0/1023734897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e5c102390 0x7f4e5c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:45.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.231+0000 7f4e6478c640 1 --2- 172.21.15.94:0/1023734897 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e5c1011a0 0x7f4e5c1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:45.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.231+0000 7f4e6478c640 1 -- 172.21.15.94:0/1023734897 >> 172.21.15.94:0/1023734897 conn(0x7f4e5c0fc730 msgr2=0x7f4e5c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:45.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.231+0000 7f4e6478c640 1 -- 172.21.15.94:0/1023734897 shutdown_connections 2024-02-17T11:55:45.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.231+0000 7f4e6478c640 1 -- 172.21.15.94:0/1023734897 wait complete. 2024-02-17T11:55:45.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e6478c640 1 Processor -- start 2024-02-17T11:55:45.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e6478c640 1 -- start start 2024-02-17T11:55:45.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e6478c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e5c1011a0 0x7f4e5c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:45.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e6478c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e5c102390 0x7f4e5c1976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:45.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e6478c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e5c102d50 0x7f4e5c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:45.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e6478c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c19ec50 con 0x7f4e5c102d50 2024-02-17T11:55:45.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e6478c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c19edc0 con 0x7f4e5c102390 2024-02-17T11:55:45.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e6478c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c19ef80 con 0x7f4e5c1011a0 2024-02-17T11:55:45.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e62d29640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e5c102d50 0x7f4e5c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:45.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e62d29640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e5c102d50 0x7f4e5c19e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46994/0 (socket says 172.21.15.94:46994) 2024-02-17T11:55:45.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e62d29640 1 -- 172.21.15.94:0/908746774 learned_addr learned my addr 172.21.15.94:0/908746774 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:45.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e61d27640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e5c102390 0x7f4e5c1976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:45.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e62528640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e5c1011a0 0x7f4e5c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:45.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e62d29640 1 -- 172.21.15.94:0/908746774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e5c1011a0 msgr2=0x7f4e5c197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:45.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e62d29640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e5c1011a0 0x7f4e5c197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:45.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e62d29640 1 -- 172.21.15.94:0/908746774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e5c102390 msgr2=0x7f4e5c1976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:45.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e62d29640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e5c102390 0x7f4e5c1976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:45.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e62d29640 1 -- 172.21.15.94:0/908746774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e54009900 con 0x7f4e5c102d50 2024-02-17T11:55:45.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e61d27640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e5c102390 0x7f4e5c1976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:55:45.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e62d29640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e5c102d50 0x7f4e5c19e710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e54037040 tx=0x7f4e54034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:45.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e537fe640 1 -- 172.21.15.94:0/908746774 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54032eb0 con 0x7f4e5c102d50 2024-02-17T11:55:45.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e537fe640 1 -- 172.21.15.94:0/908746774 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e54003e50 con 0x7f4e5c102d50 2024-02-17T11:55:45.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e6478c640 1 -- 172.21.15.94:0/908746774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c19f200 con 0x7f4e5c102d50 2024-02-17T11:55:45.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.232+0000 7f4e537fe640 1 -- 172.21.15.94:0/908746774 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54040450 con 0x7f4e5c102d50 2024-02-17T11:55:45.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.233+0000 7f4e6478c640 1 -- 172.21.15.94:0/908746774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c19f6b0 con 0x7f4e5c102d50 2024-02-17T11:55:45.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.233+0000 7f4e6478c640 1 -- 172.21.15.94:0/908746774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5c1083f0 con 0x7f4e5c102d50 2024-02-17T11:55:45.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.235+0000 7f4e537fe640 1 -- 172.21.15.94:0/908746774 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4e5400a040 con 0x7f4e5c102d50 2024-02-17T11:55:45.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.235+0000 7f4e537fe640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4e480a20c0 0x7f4e480a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:45.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.235+0000 7f4e62528640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4e480a20c0 0x7f4e480a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:45.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.235+0000 7f4e537fe640 1 -- 172.21.15.94:0/908746774 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4e540f0c90 con 0x7f4e5c102d50 2024-02-17T11:55:45.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.236+0000 7f4e62528640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4e480a20c0 0x7f4e480a4580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4e580058c0 tx=0x7f4e58005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:45.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.239+0000 7f4e537fe640 1 -- 172.21.15.94:0/908746774 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e540ba100 con 0x7f4e5c102d50 2024-02-17T11:55:45.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:45.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:45.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:45.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:45.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:45.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:45.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:45 smithi189 ceph-mon[36588]: Deploying daemon mds.g on smithi094 2024-02-17T11:55:45.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:45 smithi189 ceph-mon[36588]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:45.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:45 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] up:boot 2024-02-17T11:55:45.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:45 smithi189 ceph-mon[36588]: fsmap 9 up:standby 2024-02-17T11:55:45.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:45 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T11:55:45.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:45.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:45.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:45.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:45.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:45.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:45.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:45 smithi125 ceph-mon[36819]: Deploying daemon mds.g on smithi094 2024-02-17T11:55:45.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:45 smithi125 ceph-mon[36819]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:45.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:45 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] up:boot 2024-02-17T11:55:45.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:45 smithi125 ceph-mon[36819]: fsmap 9 up:standby 2024-02-17T11:55:45.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:45 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T11:55:45.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.451+0000 7f4e6478c640 1 -- 172.21.15.94:0/908746774 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e5c10cbb0 con 0x7f4e5c102d50 2024-02-17T11:55:45.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.452+0000 7f4e537fe640 1 -- 172.21.15.94:0/908746774 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7670 (secure 0 0 0) 0x7f4e540bd7c0 con 0x7f4e5c102d50 2024-02-17T11:55:45.451 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:45.451 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-17T11:55:45.451 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 10 2024-02-17T11:55:45.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.454+0000 7f4e6478c640 1 -- 172.21.15.94:0/908746774 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4e480a20c0 msgr2=0x7f4e480a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:45.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.454+0000 7f4e6478c640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4e480a20c0 0x7f4e480a4580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4e580058c0 tx=0x7f4e58005850 comp rx=0 tx=0).stop 2024-02-17T11:55:45.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.454+0000 7f4e6478c640 1 -- 172.21.15.94:0/908746774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e5c102d50 msgr2=0x7f4e5c19e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:45.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.454+0000 7f4e6478c640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e5c102d50 0x7f4e5c19e710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e54037040 tx=0x7f4e54034d40 comp rx=0 tx=0).stop 2024-02-17T11:55:45.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.454+0000 7f4e6478c640 1 -- 172.21.15.94:0/908746774 shutdown_connections 2024-02-17T11:55:45.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.454+0000 7f4e6478c640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4e480a20c0 0x7f4e480a4580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:45.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.454+0000 7f4e6478c640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e5c102d50 0x7f4e5c19e710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:45.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.454+0000 7f4e6478c640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e5c102390 0x7f4e5c1976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:45.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.454+0000 7f4e6478c640 1 --2- 172.21.15.94:0/908746774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e5c1011a0 0x7f4e5c197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:45.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.454+0000 7f4e6478c640 1 -- 172.21.15.94:0/908746774 >> 172.21.15.94:0/908746774 conn(0x7f4e5c0fc730 msgr2=0x7f4e5c1055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:45.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.454+0000 7f4e6478c640 1 -- 172.21.15.94:0/908746774 shutdown_connections 2024-02-17T11:55:45.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:45.454+0000 7f4e6478c640 1 -- 172.21.15.94:0/908746774 wait complete. 2024-02-17T11:55:45.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:45.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:45.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:45.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:45.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:45.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:45.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:45 smithi094 ceph-mon[32499]: Deploying daemon mds.g on smithi094 2024-02-17T11:55:45.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:45 smithi094 ceph-mon[32499]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:45.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:45 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] up:boot 2024-02-17T11:55:45.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:45 smithi094 ceph-mon[32499]: fsmap 9 up:standby 2024-02-17T11:55:45.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:45 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T11:55:46.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:46 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/908746774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:46.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:46 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/908746774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:46.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:46 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/908746774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:46.455 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 11:55:46 smithi094 systemd[1]: Starting Ceph mds.g for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:55:46.774 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 11:55:46 smithi094 podman[74270]: 2024-02-17 11:55:46.403618672 +0000 UTC m=+0.017384123 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T11:55:46.775 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 11:55:46 smithi094 podman[74270]: 2024-02-17 11:55:46.523773163 +0000 UTC m=+0.137538608 container create 2f5bb2bd9bd504f631c3f86312bfb50fab185a93591cb5a19ea3566a4e6662d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T11:55:46.775 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 11:55:46 smithi094 podman[74270]: 2024-02-17 11:55:46.776801836 +0000 UTC m=+0.390567283 container init 2f5bb2bd9bd504f631c3f86312bfb50fab185a93591cb5a19ea3566a4e6662d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.build-date=20240214) 2024-02-17T11:55:47.137 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 11:55:46 smithi094 podman[74270]: 2024-02-17 11:55:46.779703514 +0000 UTC m=+0.393468966 container start 2f5bb2bd9bd504f631c3f86312bfb50fab185a93591cb5a19ea3566a4e6662d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=) 2024-02-17T11:55:47.137 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 11:55:46 smithi094 ceph-mds[74289]: -- 172.21.15.94:0/2623143693 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b4962101a0 con 0x55b4961e7000 2024-02-17T11:55:47.137 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 11:55:46 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-g[74285]: starting mds.g at 2024-02-17T11:55:47.137 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 11:55:46 smithi094 bash[74270]: 2f5bb2bd9bd504f631c3f86312bfb50fab185a93591cb5a19ea3566a4e6662d9 2024-02-17T11:55:47.137 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 11:55:46 smithi094 systemd[1]: Started Ceph mds.g for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:55:47.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:47 smithi189 ceph-mon[36588]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:47.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:47.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:47.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:47.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:47.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:47.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:47 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:47.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:47 smithi125 ceph-mon[36819]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:47.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:47.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:47.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:47.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:47.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:47.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:47 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:47.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:47 smithi094 ceph-mon[32499]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:47.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:47.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:47.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:47.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:47.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:47.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:47 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:47.464 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:47.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.572+0000 7fb621aeb640 1 -- 172.21.15.94:0/4232384524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb61c101c50 msgr2=0x7fb61c1084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.572+0000 7fb621aeb640 1 --2- 172.21.15.94:0/4232384524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb61c101c50 0x7fb61c1084e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb614009ca0 tx=0x7fb61402c0f0 comp rx=0 tx=0).stop 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.572+0000 7fb621aeb640 1 -- 172.21.15.94:0/4232384524 shutdown_connections 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.572+0000 7fb621aeb640 1 --2- 172.21.15.94:0/4232384524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb61c101c50 0x7fb61c1084e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.572+0000 7fb621aeb640 1 --2- 172.21.15.94:0/4232384524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb61c101290 0x7fb61c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.572+0000 7fb621aeb640 1 --2- 172.21.15.94:0/4232384524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb61c1000a0 0x7fb61c1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.572+0000 7fb621aeb640 1 -- 172.21.15.94:0/4232384524 >> 172.21.15.94:0/4232384524 conn(0x7fb61c0fb610 msgr2=0x7fb61c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.572+0000 7fb621aeb640 1 -- 172.21.15.94:0/4232384524 shutdown_connections 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.572+0000 7fb621aeb640 1 -- 172.21.15.94:0/4232384524 wait complete. 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.572+0000 7fb621aeb640 1 Processor -- start 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb621aeb640 1 -- start start 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb621aeb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb61c1000a0 0x7fb61c197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb621aeb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb61c101290 0x7fb61c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb621aeb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb61c101c50 0x7fb61c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:47.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb621aeb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c19ecf0 con 0x7fb61c1000a0 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb621aeb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c19ee60 con 0x7fb61c101c50 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb621aeb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61c19f020 con 0x7fb61c101290 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb61b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb61c1000a0 0x7fb61c197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb61b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb61c1000a0 0x7fb61c197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47042/0 (socket says 172.21.15.94:47042) 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb61b7fe640 1 -- 172.21.15.94:0/4054172222 learned_addr learned my addr 172.21.15.94:0/4054172222 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb61affd640 1 --2- 172.21.15.94:0/4054172222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb61c101290 0x7fb61c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb61b7fe640 1 -- 172.21.15.94:0/4054172222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb61c101290 msgr2=0x7fb61c197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb61b7fe640 1 --2- 172.21.15.94:0/4054172222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb61c101290 0x7fb61c197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb61b7fe640 1 -- 172.21.15.94:0/4054172222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb61c101c50 msgr2=0x7fb61c19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb61b7fe640 1 --2- 172.21.15.94:0/4054172222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb61c101c50 0x7fb61c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb61b7fe640 1 -- 172.21.15.94:0/4054172222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb614009900 con 0x7fb61c1000a0 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb61b7fe640 1 --2- 172.21.15.94:0/4054172222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb61c1000a0 0x7fb61c197200 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00c970 tx=0x7fb60c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb618ff9640 1 -- 172.21.15.94:0/4054172222 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c004220 con 0x7fb61c1000a0 2024-02-17T11:55:47.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb621aeb640 1 -- 172.21.15.94:0/4054172222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c19f300 con 0x7fb61c1000a0 2024-02-17T11:55:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb618ff9640 1 -- 172.21.15.94:0/4054172222 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb60c007e90 con 0x7fb61c1000a0 2024-02-17T11:55:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb618ff9640 1 -- 172.21.15.94:0/4054172222 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60c00f450 con 0x7fb61c1000a0 2024-02-17T11:55:47.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.573+0000 7fb621aeb640 1 -- 172.21.15.94:0/4054172222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61c19f810 con 0x7fb61c1000a0 2024-02-17T11:55:47.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.574+0000 7fb621aeb640 1 -- 172.21.15.94:0/4054172222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61c1073c0 con 0x7fb61c1000a0 2024-02-17T11:55:47.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.575+0000 7fb618ff9640 1 -- 172.21.15.94:0/4054172222 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb60c003610 con 0x7fb61c1000a0 2024-02-17T11:55:47.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.576+0000 7fb618ff9640 1 --2- 172.21.15.94:0/4054172222 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb6040a20c0 0x7fb6040a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:47.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.576+0000 7fb61affd640 1 --2- 172.21.15.94:0/4054172222 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb6040a20c0 0x7fb6040a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:47.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.576+0000 7fb618ff9640 1 -- 172.21.15.94:0/4054172222 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb60c0cd960 con 0x7fb61c1000a0 2024-02-17T11:55:47.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.577+0000 7fb61affd640 1 --2- 172.21.15.94:0/4054172222 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb6040a20c0 0x7fb6040a4580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb610005fd0 tx=0x7fb610005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:47.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.580+0000 7fb618ff9640 1 -- 172.21.15.94:0/4054172222 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb60c096dd0 con 0x7fb61c1000a0 2024-02-17T11:55:47.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.801+0000 7fb621aeb640 1 -- 172.21.15.94:0/4054172222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb61c10cbb0 con 0x7fb61c1000a0 2024-02-17T11:55:47.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.802+0000 7fb618ff9640 1 -- 172.21.15.94:0/4054172222 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8462 (secure 0 0 0) 0x7fb60c09a490 con 0x7fb61c1000a0 2024-02-17T11:55:47.801 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:47.802 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-17T11:55:47.802 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 11 2024-02-17T11:55:47.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.805+0000 7fb621aeb640 1 -- 172.21.15.94:0/4054172222 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb6040a20c0 msgr2=0x7fb6040a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:47.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.805+0000 7fb621aeb640 1 --2- 172.21.15.94:0/4054172222 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb6040a20c0 0x7fb6040a4580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb610005fd0 tx=0x7fb610005850 comp rx=0 tx=0).stop 2024-02-17T11:55:47.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.805+0000 7fb621aeb640 1 -- 172.21.15.94:0/4054172222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb61c1000a0 msgr2=0x7fb61c197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:47.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.805+0000 7fb621aeb640 1 --2- 172.21.15.94:0/4054172222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb61c1000a0 0x7fb61c197200 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00c970 tx=0x7fb60c00ce30 comp rx=0 tx=0).stop 2024-02-17T11:55:47.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.805+0000 7fb621aeb640 1 -- 172.21.15.94:0/4054172222 shutdown_connections 2024-02-17T11:55:47.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.805+0000 7fb621aeb640 1 --2- 172.21.15.94:0/4054172222 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb6040a20c0 0x7fb6040a4580 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:47.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.805+0000 7fb621aeb640 1 --2- 172.21.15.94:0/4054172222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb61c101c50 0x7fb61c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:47.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.805+0000 7fb621aeb640 1 --2- 172.21.15.94:0/4054172222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb61c101290 0x7fb61c197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:47.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.805+0000 7fb621aeb640 1 --2- 172.21.15.94:0/4054172222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb61c1000a0 0x7fb61c197200 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:47.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.805+0000 7fb621aeb640 1 -- 172.21.15.94:0/4054172222 >> 172.21.15.94:0/4054172222 conn(0x7fb61c0fb610 msgr2=0x7fb61c104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:47.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.805+0000 7fb621aeb640 1 -- 172.21.15.94:0/4054172222 shutdown_connections 2024-02-17T11:55:47.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:47.805+0000 7fb621aeb640 1 -- 172.21.15.94:0/4054172222 wait complete. 2024-02-17T11:55:48.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:48 smithi189 ceph-mon[36588]: Deploying daemon mds.a on smithi094 2024-02-17T11:55:48.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:48 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.94:6838/2246951573,v1:172.21.15.94:6839/2246951573] up:boot 2024-02-17T11:55:48.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:48 smithi189 ceph-mon[36588]: fsmap 10 up:standby 2024-02-17T11:55:48.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:48 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T11:55:48.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:48 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/4054172222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:48.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:48 smithi125 ceph-mon[36819]: Deploying daemon mds.a on smithi094 2024-02-17T11:55:48.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:48 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.94:6838/2246951573,v1:172.21.15.94:6839/2246951573] up:boot 2024-02-17T11:55:48.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:48 smithi125 ceph-mon[36819]: fsmap 10 up:standby 2024-02-17T11:55:48.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:48 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T11:55:48.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:48 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/4054172222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:48.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:48 smithi094 ceph-mon[32499]: Deploying daemon mds.a on smithi094 2024-02-17T11:55:48.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:48 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.94:6838/2246951573,v1:172.21.15.94:6839/2246951573] up:boot 2024-02-17T11:55:48.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:48 smithi094 ceph-mon[32499]: fsmap 10 up:standby 2024-02-17T11:55:48.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:48 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T11:55:48.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:48 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/4054172222' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:49.397 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:49 smithi094 ceph-mon[32499]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:49.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:49 smithi189 ceph-mon[36588]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:49.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:49 smithi125 ceph-mon[36819]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:49.705 INFO:journalctl@ceph.mds.a.smithi094.stdout:Feb 17 11:55:49 smithi094 systemd[1]: Started Ceph mds.a for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:55:49.817 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:49.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.924+0000 7ff7bde64640 1 -- 172.21.15.94:0/2679042818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b81000a0 msgr2=0x7ff7b81004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:49.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.924+0000 7ff7bde64640 1 --2- 172.21.15.94:0/2679042818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b81000a0 0x7ff7b81004c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009900 tx=0x7ff7a802bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:49.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 -- 172.21.15.94:0/2679042818 shutdown_connections 2024-02-17T11:55:49.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 --2- 172.21.15.94:0/2679042818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7b8101c50 0x7ff7b81084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:49.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 --2- 172.21.15.94:0/2679042818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7b8101290 0x7ff7b8101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:49.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 --2- 172.21.15.94:0/2679042818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b81000a0 0x7ff7b81004c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:49.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 -- 172.21.15.94:0/2679042818 >> 172.21.15.94:0/2679042818 conn(0x7ff7b80fb610 msgr2=0x7ff7b80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:49.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 -- 172.21.15.94:0/2679042818 shutdown_connections 2024-02-17T11:55:49.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 -- 172.21.15.94:0/2679042818 wait complete. 2024-02-17T11:55:49.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 Processor -- start 2024-02-17T11:55:49.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 -- start start 2024-02-17T11:55:49.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7b81000a0 0x7ff7b8197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:49.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b8101290 0x7ff7b8197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:49.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7b8101c50 0x7ff7b819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:49.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b819ed30 con 0x7ff7b8101290 2024-02-17T11:55:49.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b819eea0 con 0x7ff7b81000a0 2024-02-17T11:55:49.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.925+0000 7ff7bde64640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b819f060 con 0x7ff7b8101c50 2024-02-17T11:55:49.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b8101290 0x7ff7b8197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:49.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b8101290 0x7ff7b8197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47068/0 (socket says 172.21.15.94:47068) 2024-02-17T11:55:49.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b6ffd640 1 -- 172.21.15.94:0/719621632 learned_addr learned my addr 172.21.15.94:0/719621632 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:49.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b77fe640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7b81000a0 0x7ff7b8197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:49.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b7fff640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7b8101c50 0x7ff7b819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:49.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b6ffd640 1 -- 172.21.15.94:0/719621632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7b8101c50 msgr2=0x7ff7b819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:49.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b6ffd640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7b8101c50 0x7ff7b819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:49.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b6ffd640 1 -- 172.21.15.94:0/719621632 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7b81000a0 msgr2=0x7ff7b8197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:49.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b6ffd640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7b81000a0 0x7ff7b8197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:49.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b6ffd640 1 -- 172.21.15.94:0/719621632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a8009560 con 0x7ff7b8101290 2024-02-17T11:55:49.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b7fff640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7b8101c50 0x7ff7b819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:55:49.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b6ffd640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b8101290 0x7ff7b8197780 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000e9f0 tx=0x7ff7a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:49.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b4ff9640 1 -- 172.21.15.94:0/719621632 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0009700 con 0x7ff7b8101290 2024-02-17T11:55:49.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7bde64640 1 -- 172.21.15.94:0/719621632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b819f340 con 0x7ff7b8101290 2024-02-17T11:55:49.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b4ff9640 1 -- 172.21.15.94:0/719621632 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7a000cd80 con 0x7ff7b8101290 2024-02-17T11:55:49.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7b4ff9640 1 -- 172.21.15.94:0/719621632 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a0018920 con 0x7ff7b8101290 2024-02-17T11:55:49.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.926+0000 7ff7bde64640 1 -- 172.21.15.94:0/719621632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b819f8a0 con 0x7ff7b8101290 2024-02-17T11:55:49.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.927+0000 7ff7bde64640 1 -- 172.21.15.94:0/719621632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b81073c0 con 0x7ff7b8101290 2024-02-17T11:55:49.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.928+0000 7ff7b4ff9640 1 -- 172.21.15.94:0/719621632 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff7a0018a80 con 0x7ff7b8101290 2024-02-17T11:55:49.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.929+0000 7ff7b4ff9640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff7a40a20c0 0x7ff7a40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:49.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.929+0000 7ff7b77fe640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff7a40a20c0 0x7ff7a40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:49.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.929+0000 7ff7b4ff9640 1 -- 172.21.15.94:0/719621632 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff7a0014070 con 0x7ff7b8101290 2024-02-17T11:55:49.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.929+0000 7ff7b77fe640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff7a40a20c0 0x7ff7a40a4580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8005ab0 tx=0x7ff7a8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:49.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:49.933+0000 7ff7b4ff9640 1 -- 172.21.15.94:0/719621632 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7a0052d20 con 0x7ff7b8101290 2024-02-17T11:55:50.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.149+0000 7ff7bde64640 1 -- 172.21.15.94:0/719621632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7b810cdb0 con 0x7ff7b8101290 2024-02-17T11:55:50.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.149+0000 7ff7b4ff9640 1 -- 172.21.15.94:0/719621632 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9251 (secure 0 0 0) 0x7ff7a00cfe60 con 0x7ff7b8101290 2024-02-17T11:55:50.148 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:50.149 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-17T11:55:50.149 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-02-17T11:55:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.151+0000 7ff7bde64640 1 -- 172.21.15.94:0/719621632 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff7a40a20c0 msgr2=0x7ff7a40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.151+0000 7ff7bde64640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff7a40a20c0 0x7ff7a40a4580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8005ab0 tx=0x7ff7a8005fd0 comp rx=0 tx=0).stop 2024-02-17T11:55:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.151+0000 7ff7bde64640 1 -- 172.21.15.94:0/719621632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b8101290 msgr2=0x7ff7b8197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.151+0000 7ff7bde64640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b8101290 0x7ff7b8197780 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff7a000e9f0 tx=0x7ff7a000eeb0 comp rx=0 tx=0).stop 2024-02-17T11:55:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.151+0000 7ff7bde64640 1 -- 172.21.15.94:0/719621632 shutdown_connections 2024-02-17T11:55:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.151+0000 7ff7bde64640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff7a40a20c0 0x7ff7a40a4580 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.151+0000 7ff7bde64640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7b8101c50 0x7ff7b819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.151+0000 7ff7bde64640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7b8101290 0x7ff7b8197780 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.151+0000 7ff7bde64640 1 --2- 172.21.15.94:0/719621632 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7b81000a0 0x7ff7b8197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:50.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.151+0000 7ff7bde64640 1 -- 172.21.15.94:0/719621632 >> 172.21.15.94:0/719621632 conn(0x7ff7b80fb610 msgr2=0x7ff7b8104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:50.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.152+0000 7ff7bde64640 1 -- 172.21.15.94:0/719621632 shutdown_connections 2024-02-17T11:55:50.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:50.152+0000 7ff7bde64640 1 -- 172.21.15.94:0/719621632 wait complete. 2024-02-17T11:55:50.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:50 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:50.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:50 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:50.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:50 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:50.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:50 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:50.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:50 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:50.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:50 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:50.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:50 smithi189 ceph-mon[36588]: Deploying daemon mds.b on smithi125 2024-02-17T11:55:50.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:50 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.94:6840/755410555,v1:172.21.15.94:6841/755410555] up:boot 2024-02-17T11:55:50.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:50 smithi189 ceph-mon[36588]: fsmap 11 up:standby 2024-02-17T11:55:50.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:50 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T11:55:50.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:50 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/719621632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:50.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:50 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:50.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:50 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:50.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:50 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:50.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:50 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:50.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:50 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:50.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:50 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:50.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:50 smithi125 ceph-mon[36819]: Deploying daemon mds.b on smithi125 2024-02-17T11:55:50.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:50 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.94:6840/755410555,v1:172.21.15.94:6841/755410555] up:boot 2024-02-17T11:55:50.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:50 smithi125 ceph-mon[36819]: fsmap 11 up:standby 2024-02-17T11:55:50.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:50 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T11:55:50.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:50 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/719621632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:50.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:50 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:50.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:50 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:50.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:50 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:50.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:50 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-17T11:55:50.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:50 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-17T11:55:50.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:50 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:50.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:50 smithi094 ceph-mon[32499]: Deploying daemon mds.b on smithi125 2024-02-17T11:55:50.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:50 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.94:6840/755410555,v1:172.21.15.94:6841/755410555] up:boot 2024-02-17T11:55:50.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:50 smithi094 ceph-mon[32499]: fsmap 11 up:standby 2024-02-17T11:55:50.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:50 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T11:55:50.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:50 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/719621632' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:51.923 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:51 smithi125 ceph-mon[36819]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:51.924 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 11:55:51 smithi125 systemd[1]: Starting Ceph mds.b for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T11:55:51.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:51 smithi094 ceph-mon[32499]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:52.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:51 smithi189 ceph-mon[36588]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:52.163 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:52.184 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 11:55:52 smithi125 podman[67415]: 2024-02-17 11:55:51.957153816 +0000 UTC m=+0.017183554 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T11:55:52.184 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 11:55:52 smithi125 podman[67415]: 2024-02-17 11:55:52.072494351 +0000 UTC m=+0.132524101 container create f403f8ef7044d1cdfe5cbdb3a47628955951dfa9517f2244f736886793f07fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T11:55:52.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.279+0000 7f6718a4e640 1 -- 172.21.15.94:0/805781307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67141045b0 msgr2=0x7f67141069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:52.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.279+0000 7f6718a4e640 1 --2- 172.21.15.94:0/805781307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67141045b0 0x7f67141069a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009900 tx=0x7f66fc02bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:52.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.279+0000 7f6718a4e640 1 -- 172.21.15.94:0/805781307 shutdown_connections 2024-02-17T11:55:52.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.279+0000 7f6718a4e640 1 --2- 172.21.15.94:0/805781307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6714106ee0 0x7f67141092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:52.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.279+0000 7f6718a4e640 1 --2- 172.21.15.94:0/805781307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67141045b0 0x7f67141069a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:52.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.279+0000 7f6718a4e640 1 --2- 172.21.15.94:0/805781307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6714101c80 0x7f6714104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:52.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.279+0000 7f6718a4e640 1 -- 172.21.15.94:0/805781307 >> 172.21.15.94:0/805781307 conn(0x7f67140fb610 msgr2=0x7f67140fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:52.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.279+0000 7f6718a4e640 1 -- 172.21.15.94:0/805781307 shutdown_connections 2024-02-17T11:55:52.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.279+0000 7f6718a4e640 1 -- 172.21.15.94:0/805781307 wait complete. 2024-02-17T11:55:52.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6718a4e640 1 Processor -- start 2024-02-17T11:55:52.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6718a4e640 1 -- start start 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6718a4e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6714101c80 0x7f6714197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6718a4e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67141045b0 0x7f67141977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6718a4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6714106ee0 0x7f671419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6718a4e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671419ed60 con 0x7f6714106ee0 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6718a4e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671419eed0 con 0x7f6714101c80 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6718a4e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671419f090 con 0x7f67141045b0 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6712d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6714106ee0 0x7f671419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6712d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6714106ee0 0x7f671419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47086/0 (socket says 172.21.15.94:47086) 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6712d9d640 1 -- 172.21.15.94:0/3303724844 learned_addr learned my addr 172.21.15.94:0/3303724844 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f670bfff640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67141045b0 0x7f67141977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f671259c640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6714101c80 0x7f6714197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6712d9d640 1 -- 172.21.15.94:0/3303724844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67141045b0 msgr2=0x7f67141977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6712d9d640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67141045b0 0x7f67141977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:52.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6712d9d640 1 -- 172.21.15.94:0/3303724844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6714101c80 msgr2=0x7f6714197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6712d9d640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6714101c80 0x7f6714197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f6712d9d640 1 -- 172.21.15.94:0/3303724844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc009560 con 0x7f6714106ee0 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.280+0000 7f671259c640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6714101c80 0x7f6714197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.281+0000 7f6712d9d640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6714106ee0 0x7f671419e820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f670c00eaa0 tx=0x7f670c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.281+0000 7f670b7fe640 1 -- 172.21.15.94:0/3303724844 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c00cdb0 con 0x7f6714106ee0 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.281+0000 7f670b7fe640 1 -- 172.21.15.94:0/3303724844 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f670c005020 con 0x7f6714106ee0 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.281+0000 7f6718a4e640 1 -- 172.21.15.94:0/3303724844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671419f370 con 0x7f6714106ee0 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.281+0000 7f670b7fe640 1 -- 172.21.15.94:0/3303724844 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670c0116b0 con 0x7f6714106ee0 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.281+0000 7f6718a4e640 1 -- 172.21.15.94:0/3303724844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671419f800 con 0x7f6714106ee0 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.282+0000 7f6718a4e640 1 -- 172.21.15.94:0/3303724844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67140fd100 con 0x7f6714106ee0 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.283+0000 7f670b7fe640 1 -- 172.21.15.94:0/3303724844 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f670c005520 con 0x7f6714106ee0 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.283+0000 7f670b7fe640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f67000a20c0 0x7f67000a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.283+0000 7f671259c640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f67000a20c0 0x7f67000a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:52.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.283+0000 7f670b7fe640 1 -- 172.21.15.94:0/3303724844 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f670c0cdb20 con 0x7f6714106ee0 2024-02-17T11:55:52.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.284+0000 7f671259c640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f67000a20c0 0x7f67000a4580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6704009b40 tx=0x7f6704005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:52.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.288+0000 7f670b7fe640 1 -- 172.21.15.94:0/3303724844 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f670c096f90 con 0x7f6714106ee0 2024-02-17T11:55:52.485 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 11:55:52 smithi125 podman[67415]: 2024-02-17 11:55:52.316728216 +0000 UTC m=+0.376757937 container init f403f8ef7044d1cdfe5cbdb3a47628955951dfa9517f2244f736886793f07fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-17T11:55:52.485 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 11:55:52 smithi125 podman[67415]: 2024-02-17 11:55:52.320027928 +0000 UTC m=+0.380057667 container start f403f8ef7044d1cdfe5cbdb3a47628955951dfa9517f2244f736886793f07fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-17T11:55:52.485 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 11:55:52 smithi125 ceph-mds[67434]: -- 172.21.15.125:0/3328937424 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5619043d81a0 con 0x5619043af000 2024-02-17T11:55:52.485 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 11:55:52 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b[67430]: starting mds.b at 2024-02-17T11:55:52.485 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 11:55:52 smithi125 bash[67415]: f403f8ef7044d1cdfe5cbdb3a47628955951dfa9517f2244f736886793f07fa8 2024-02-17T11:55:52.485 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 11:55:52 smithi125 systemd[1]: Started Ceph mds.b for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T11:55:52.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.504+0000 7f6718a4e640 1 -- 172.21.15.94:0/3303724844 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f671410cbb0 con 0x7f6714106ee0 2024-02-17T11:55:52.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.504+0000 7f670b7fe640 1 -- 172.21.15.94:0/3303724844 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9251 (secure 0 0 0) 0x7f670c09a650 con 0x7f6714106ee0 2024-02-17T11:55:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:52.503 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-02-17T11:55:52.503 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 12 2024-02-17T11:55:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.506+0000 7f6718a4e640 1 -- 172.21.15.94:0/3303724844 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f67000a20c0 msgr2=0x7f67000a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.506+0000 7f6718a4e640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f67000a20c0 0x7f67000a4580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6704009b40 tx=0x7f6704005e50 comp rx=0 tx=0).stop 2024-02-17T11:55:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.506+0000 7f6718a4e640 1 -- 172.21.15.94:0/3303724844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6714106ee0 msgr2=0x7f671419e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:52.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.506+0000 7f6718a4e640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6714106ee0 0x7f671419e820 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f670c00eaa0 tx=0x7f670c00ef60 comp rx=0 tx=0).stop 2024-02-17T11:55:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.506+0000 7f6718a4e640 1 -- 172.21.15.94:0/3303724844 shutdown_connections 2024-02-17T11:55:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.506+0000 7f6718a4e640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f67000a20c0 0x7f67000a4580 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.506+0000 7f6718a4e640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6714106ee0 0x7f671419e820 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.506+0000 7f6718a4e640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67141045b0 0x7f67141977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.506+0000 7f6718a4e640 1 --2- 172.21.15.94:0/3303724844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6714101c80 0x7f6714197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.506+0000 7f6718a4e640 1 -- 172.21.15.94:0/3303724844 >> 172.21.15.94:0/3303724844 conn(0x7f67140fb610 msgr2=0x7f67141026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.506+0000 7f6718a4e640 1 -- 172.21.15.94:0/3303724844 shutdown_connections 2024-02-17T11:55:52.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:52.506+0000 7f6718a4e640 1 -- 172.21.15.94:0/3303724844 wait complete. 2024-02-17T11:55:53.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:53 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3303724844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:53.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:53 smithi189 ceph-mon[36588]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:53.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:53.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:53 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] up:boot 2024-02-17T11:55:53.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:53 smithi189 ceph-mon[36588]: fsmap 12 up:standby 2024-02-17T11:55:53.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:53 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T11:55:53.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:53 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3303724844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:53.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:53 smithi094 ceph-mon[32499]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:53.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:53.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:53 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] up:boot 2024-02-17T11:55:53.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:53 smithi094 ceph-mon[32499]: fsmap 12 up:standby 2024-02-17T11:55:53.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:53 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T11:55:53.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:53 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3303724844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:53.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:53.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:53 smithi125 ceph-mon[36819]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:53.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:53.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:53 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] up:boot 2024-02-17T11:55:53.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:53 smithi125 ceph-mon[36819]: fsmap 12 up:standby 2024-02-17T11:55:53.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:53 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T11:55:54.517 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:55:54.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.625+0000 7f24eba09640 1 -- 172.21.15.94:0/2065193560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e4101c50 msgr2=0x7f24e41084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:54.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.625+0000 7f24eba09640 1 --2- 172.21.15.94:0/2065193560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e4101c50 0x7f24e41084e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009ca0 tx=0x7f24dc02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.625+0000 7f24eba09640 1 -- 172.21.15.94:0/2065193560 shutdown_connections 2024-02-17T11:55:54.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.625+0000 7f24eba09640 1 --2- 172.21.15.94:0/2065193560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e4101c50 0x7f24e41084e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.625+0000 7f24eba09640 1 --2- 172.21.15.94:0/2065193560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e4101290 0x7f24e4101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.625+0000 7f24eba09640 1 --2- 172.21.15.94:0/2065193560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24e41000a0 0x7f24e41004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.625+0000 7f24eba09640 1 -- 172.21.15.94:0/2065193560 >> 172.21.15.94:0/2065193560 conn(0x7f24e40fb610 msgr2=0x7f24e40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:54.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.625+0000 7f24eba09640 1 -- 172.21.15.94:0/2065193560 shutdown_connections 2024-02-17T11:55:54.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.625+0000 7f24eba09640 1 -- 172.21.15.94:0/2065193560 wait complete. 2024-02-17T11:55:54.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24eba09640 1 Processor -- start 2024-02-17T11:55:54.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24eba09640 1 -- start start 2024-02-17T11:55:54.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24eba09640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24e41000a0 0x7f24e41960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:54.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24eba09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e4101290 0x7f24e4196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:54.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24eba09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e4101c50 0x7f24e419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:54.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24eba09640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e419dbd0 con 0x7f24e4101290 2024-02-17T11:55:54.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24eba09640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e419dd40 con 0x7f24e4101c50 2024-02-17T11:55:54.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24eba09640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e419df00 con 0x7f24e41000a0 2024-02-17T11:55:54.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24e8fa4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e4101290 0x7f24e4196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:54.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24e8fa4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e4101290 0x7f24e4196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37448/0 (socket says 172.21.15.94:37448) 2024-02-17T11:55:54.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24e8fa4640 1 -- 172.21.15.94:0/20767090 learned_addr learned my addr 172.21.15.94:0/20767090 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:54.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24e8fa4640 1 -- 172.21.15.94:0/20767090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24e41000a0 msgr2=0x7f24e41960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:54.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24e9fa6640 1 --2- 172.21.15.94:0/20767090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e4101c50 0x7f24e419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:54.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.626+0000 7f24e8fa4640 1 --2- 172.21.15.94:0/20767090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24e41000a0 0x7f24e41960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.627+0000 7f24e8fa4640 1 -- 172.21.15.94:0/20767090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e4101c50 msgr2=0x7f24e419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:54.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.627+0000 7f24e8fa4640 1 --2- 172.21.15.94:0/20767090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e4101c50 0x7f24e419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.627+0000 7f24e8fa4640 1 -- 172.21.15.94:0/20767090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24dc009900 con 0x7f24e4101290 2024-02-17T11:55:54.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.627+0000 7f24e8fa4640 1 --2- 172.21.15.94:0/20767090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e4101290 0x7f24e4196620 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f24d400d950 tx=0x7f24d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:54.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.627+0000 7f24da7fc640 1 -- 172.21.15.94:0/20767090 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4009c30 con 0x7f24e4101290 2024-02-17T11:55:54.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.627+0000 7f24da7fc640 1 -- 172.21.15.94:0/20767090 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24d4009d90 con 0x7f24e4101290 2024-02-17T11:55:54.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.627+0000 7f24eba09640 1 -- 172.21.15.94:0/20767090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e419e1e0 con 0x7f24e4101290 2024-02-17T11:55:54.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.627+0000 7f24da7fc640 1 -- 172.21.15.94:0/20767090 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d400f7e0 con 0x7f24e4101290 2024-02-17T11:55:54.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.627+0000 7f24eba09640 1 -- 172.21.15.94:0/20767090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e419e740 con 0x7f24e4101290 2024-02-17T11:55:54.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.628+0000 7f24eba09640 1 -- 172.21.15.94:0/20767090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e4107360 con 0x7f24e4101290 2024-02-17T11:55:54.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.629+0000 7f24da7fc640 1 -- 172.21.15.94:0/20767090 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f24d4010ad0 con 0x7f24e4101290 2024-02-17T11:55:54.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.630+0000 7f24da7fc640 1 --2- 172.21.15.94:0/20767090 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f24d00a2140 0x7f24d00a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:54.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.630+0000 7f24e97a5640 1 --2- 172.21.15.94:0/20767090 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f24d00a2140 0x7f24d00a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:54.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.630+0000 7f24da7fc640 1 -- 172.21.15.94:0/20767090 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f24d40cd500 con 0x7f24e4101290 2024-02-17T11:55:54.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.630+0000 7f24e97a5640 1 --2- 172.21.15.94:0/20767090 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f24d00a2140 0x7f24d00a4600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005fd0 tx=0x7f24e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:54.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.634+0000 7f24da7fc640 1 -- 172.21.15.94:0/20767090 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f24d40968d0 con 0x7f24e4101290 2024-02-17T11:55:54.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.857+0000 7f24eba09640 1 -- 172.21.15.94:0/20767090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24e4106dd0 con 0x7f24e4101290 2024-02-17T11:55:54.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.858+0000 7f24da7fc640 1 -- 172.21.15.94:0/20767090 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10046 (secure 0 0 0) 0x7f24d4099f90 con 0x7f24e4101290 2024-02-17T11:55:54.857 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:54.857 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24452,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24454,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2024-02-17T11:55:54.857 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 13 2024-02-17T11:55:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.859+0000 7f24eba09640 1 -- 172.21.15.94:0/20767090 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f24d00a2140 msgr2=0x7f24d00a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.859+0000 7f24eba09640 1 --2- 172.21.15.94:0/20767090 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f24d00a2140 0x7f24d00a4600 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005fd0 tx=0x7f24e000b410 comp rx=0 tx=0).stop 2024-02-17T11:55:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.860+0000 7f24eba09640 1 -- 172.21.15.94:0/20767090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e4101290 msgr2=0x7f24e4196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.860+0000 7f24eba09640 1 --2- 172.21.15.94:0/20767090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e4101290 0x7f24e4196620 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f24d400d950 tx=0x7f24d400de10 comp rx=0 tx=0).stop 2024-02-17T11:55:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.860+0000 7f24eba09640 1 -- 172.21.15.94:0/20767090 shutdown_connections 2024-02-17T11:55:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.860+0000 7f24eba09640 1 --2- 172.21.15.94:0/20767090 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f24d00a2140 0x7f24d00a4600 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.860+0000 7f24eba09640 1 --2- 172.21.15.94:0/20767090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f24e4101c50 0x7f24e419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.860+0000 7f24eba09640 1 --2- 172.21.15.94:0/20767090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f24e4101290 0x7f24e4196620 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.860+0000 7f24eba09640 1 --2- 172.21.15.94:0/20767090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24e41000a0 0x7f24e41960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.860+0000 7f24eba09640 1 -- 172.21.15.94:0/20767090 >> 172.21.15.94:0/20767090 conn(0x7f24e40fb610 msgr2=0x7f24e4104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:54.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.860+0000 7f24eba09640 1 -- 172.21.15.94:0/20767090 shutdown_connections 2024-02-17T11:55:54.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.860+0000 7f24eba09640 1 -- 172.21.15.94:0/20767090 wait complete. 2024-02-17T11:55:54.868 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:54 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.868 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:54 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.869 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:54 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.869 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:54 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.869 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:54 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.869 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:54 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.869 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:54 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.870 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:55:54.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:54 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:54 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:54 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:54 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:54 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:54 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:54 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:54 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:54 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:54 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:54 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:54 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:54 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:54 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:54.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.976+0000 7ff6c7193640 1 -- 172.21.15.94:0/978255747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c0101c50 msgr2=0x7ff6c01084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:54.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.976+0000 7ff6c7193640 1 --2- 172.21.15.94:0/978255747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c0101c50 0x7ff6c01084e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8009ca0 tx=0x7ff6b802c0f0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.976+0000 7ff6c7193640 1 -- 172.21.15.94:0/978255747 shutdown_connections 2024-02-17T11:55:54.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.976+0000 7ff6c7193640 1 --2- 172.21.15.94:0/978255747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c0101c50 0x7ff6c01084e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.976+0000 7ff6c7193640 1 --2- 172.21.15.94:0/978255747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6c0101290 0x7ff6c0101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.976+0000 7ff6c7193640 1 --2- 172.21.15.94:0/978255747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6c01000a0 0x7ff6c01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.976+0000 7ff6c7193640 1 -- 172.21.15.94:0/978255747 >> 172.21.15.94:0/978255747 conn(0x7ff6c00fb610 msgr2=0x7ff6c00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.976+0000 7ff6c7193640 1 -- 172.21.15.94:0/978255747 shutdown_connections 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.976+0000 7ff6c7193640 1 -- 172.21.15.94:0/978255747 wait complete. 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.976+0000 7ff6c7193640 1 Processor -- start 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.976+0000 7ff6c7193640 1 -- start start 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c7193640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c01000a0 0x7ff6c01960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c7193640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6c0101290 0x7ff6c0196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c7193640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6c0101c50 0x7ff6c019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c7193640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c019dbd0 con 0x7ff6c01000a0 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c7193640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c019dd40 con 0x7ff6c0101c50 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c7193640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c019df00 con 0x7ff6c0101290 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c4f2f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c01000a0 0x7ff6c01960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c4f2f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c01000a0 0x7ff6c01960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37460/0 (socket says 172.21.15.94:37460) 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c4f2f640 1 -- 172.21.15.94:0/2936567882 learned_addr learned my addr 172.21.15.94:0/2936567882 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:54.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6bffff640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6c0101290 0x7ff6c0196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c5730640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6c0101c50 0x7ff6c019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c4f2f640 1 -- 172.21.15.94:0/2936567882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6c0101290 msgr2=0x7ff6c0196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c4f2f640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6c0101290 0x7ff6c0196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c4f2f640 1 -- 172.21.15.94:0/2936567882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6c0101c50 msgr2=0x7ff6c019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c4f2f640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6c0101c50 0x7ff6c019d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c4f2f640 1 -- 172.21.15.94:0/2936567882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b8009900 con 0x7ff6c01000a0 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6bffff640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6c0101290 0x7ff6c0196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c5730640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6c0101c50 0x7ff6c019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c4f2f640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c01000a0 0x7ff6c01960e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400c970 tx=0x7ff6b400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6bdffb640 1 -- 172.21.15.94:0/2936567882 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4004220 con 0x7ff6c01000a0 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6c7193640 1 -- 172.21.15.94:0/2936567882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c019e1e0 con 0x7ff6c01000a0 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6bdffb640 1 -- 172.21.15.94:0/2936567882 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6b4007e90 con 0x7ff6c01000a0 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.977+0000 7ff6bdffb640 1 -- 172.21.15.94:0/2936567882 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b400f450 con 0x7ff6c01000a0 2024-02-17T11:55:54.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.978+0000 7ff6c7193640 1 -- 172.21.15.94:0/2936567882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c019e6f0 con 0x7ff6c01000a0 2024-02-17T11:55:54.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.979+0000 7ff6c7193640 1 -- 172.21.15.94:0/2936567882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c0107360 con 0x7ff6c01000a0 2024-02-17T11:55:54.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.980+0000 7ff6bdffb640 1 -- 172.21.15.94:0/2936567882 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ff6b4003610 con 0x7ff6c01000a0 2024-02-17T11:55:54.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.980+0000 7ff6bdffb640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff6b00a20c0 0x7ff6b00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:54.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.980+0000 7ff6bffff640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff6b00a20c0 0x7ff6b00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:54.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.980+0000 7ff6bdffb640 1 -- 172.21.15.94:0/2936567882 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6b40cd5a0 con 0x7ff6c01000a0 2024-02-17T11:55:54.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.981+0000 7ff6bffff640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff6b00a20c0 0x7ff6b00a4580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff6c01975f0 tx=0x7ff6ac005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:54.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:54.985+0000 7ff6bdffb640 1 -- 172.21.15.94:0/2936567882 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff6b40969f0 con 0x7ff6c01000a0 2024-02-17T11:55:55.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.168+0000 7ff6c7193640 1 -- 172.21.15.94:0/2936567882 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6c002ce40 con 0x7ff6c01000a0 2024-02-17T11:55:55.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.169+0000 7ff6bdffb640 1 -- 172.21.15.94:0/2936567882 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18444 (secure 0 0 0) 0x7ff6b409a0b0 con 0x7ff6c01000a0 2024-02-17T11:55:55.168 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:55.169 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":81,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:55:09.341614+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:55:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.171+0000 7ff6c7193640 1 -- 172.21.15.94:0/2936567882 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff6b00a20c0 msgr2=0x7ff6b00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.171+0000 7ff6c7193640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff6b00a20c0 0x7ff6b00a4580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff6c01975f0 tx=0x7ff6ac005ec0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.171+0000 7ff6c7193640 1 -- 172.21.15.94:0/2936567882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c01000a0 msgr2=0x7ff6c01960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.171+0000 7ff6c7193640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c01000a0 0x7ff6c01960e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff6b400c970 tx=0x7ff6b400ce30 comp rx=0 tx=0).stop 2024-02-17T11:55:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.171+0000 7ff6c7193640 1 -- 172.21.15.94:0/2936567882 shutdown_connections 2024-02-17T11:55:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.171+0000 7ff6c7193640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff6b00a20c0 0x7ff6b00a4580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.171+0000 7ff6c7193640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff6c0101c50 0x7ff6c019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.171+0000 7ff6c7193640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff6c0101290 0x7ff6c0196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.171+0000 7ff6c7193640 1 --2- 172.21.15.94:0/2936567882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6c01000a0 0x7ff6c01960e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.171+0000 7ff6c7193640 1 -- 172.21.15.94:0/2936567882 >> 172.21.15.94:0/2936567882 conn(0x7ff6c00fb610 msgr2=0x7ff6c0104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.172+0000 7ff6c7193640 1 -- 172.21.15.94:0/2936567882 shutdown_connections 2024-02-17T11:55:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.172+0000 7ff6c7193640 1 -- 172.21.15.94:0/2936567882 wait complete. 2024-02-17T11:55:55.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T11:51:20.894601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T11:55:55.182 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T11:55:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.292+0000 7fd824b3a640 1 -- 172.21.15.94:0/1359183871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd820101c30 msgr2=0x7fd8201084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.292+0000 7fd824b3a640 1 --2- 172.21.15.94:0/1359183871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd820101c30 0x7fd8201084c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd818009ca0 tx=0x7fd81802c0f0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.292+0000 7fd824b3a640 1 -- 172.21.15.94:0/1359183871 shutdown_connections 2024-02-17T11:55:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.292+0000 7fd824b3a640 1 --2- 172.21.15.94:0/1359183871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd820101c30 0x7fd8201084c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.292+0000 7fd824b3a640 1 --2- 172.21.15.94:0/1359183871 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820101270 0x7fd8201016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.292+0000 7fd824b3a640 1 --2- 172.21.15.94:0/1359183871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd820100080 0x7fd8201004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.292+0000 7fd824b3a640 1 -- 172.21.15.94:0/1359183871 >> 172.21.15.94:0/1359183871 conn(0x7fd8200fb610 msgr2=0x7fd8200fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.292+0000 7fd824b3a640 1 -- 172.21.15.94:0/1359183871 shutdown_connections 2024-02-17T11:55:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.292+0000 7fd824b3a640 1 -- 172.21.15.94:0/1359183871 wait complete. 2024-02-17T11:55:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd824b3a640 1 Processor -- start 2024-02-17T11:55:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd824b3a640 1 -- start start 2024-02-17T11:55:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd824b3a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820100080 0x7fd820068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd824b3a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd820101270 0x7fd82019bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd824b3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd820101c30 0x7fd82019e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd824b3a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82019eda0 con 0x7fd820101c30 2024-02-17T11:55:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd824b3a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82019ef10 con 0x7fd820100080 2024-02-17T11:55:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd824b3a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82019f0d0 con 0x7fd820101270 2024-02-17T11:55:55.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd81ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd820101c30 0x7fd82019e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd81ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd820101c30 0x7fd82019e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37488/0 (socket says 172.21.15.94:37488) 2024-02-17T11:55:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd81ed9d640 1 -- 172.21.15.94:0/47918326 learned_addr learned my addr 172.21.15.94:0/47918326 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd81e59c640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820100080 0x7fd820068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd81dd9b640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd820101270 0x7fd82019bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd81ed9d640 1 -- 172.21.15.94:0/47918326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd820101270 msgr2=0x7fd82019bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd81ed9d640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd820101270 0x7fd82019bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd81ed9d640 1 -- 172.21.15.94:0/47918326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820100080 msgr2=0x7fd820068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd81ed9d640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820100080 0x7fd820068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.293+0000 7fd81ed9d640 1 -- 172.21.15.94:0/47918326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd818009900 con 0x7fd820101c30 2024-02-17T11:55:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.294+0000 7fd81ed9d640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd820101c30 0x7fd82019e5c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd818037040 tx=0x7fd818034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.294+0000 7fd80f7fe640 1 -- 172.21.15.94:0/47918326 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818032eb0 con 0x7fd820101c30 2024-02-17T11:55:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.294+0000 7fd80f7fe640 1 -- 172.21.15.94:0/47918326 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd818003e50 con 0x7fd820101c30 2024-02-17T11:55:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.294+0000 7fd824b3a640 1 -- 172.21.15.94:0/47918326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82019f2f0 con 0x7fd820101c30 2024-02-17T11:55:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.294+0000 7fd80f7fe640 1 -- 172.21.15.94:0/47918326 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818040450 con 0x7fd820101c30 2024-02-17T11:55:55.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.294+0000 7fd824b3a640 1 -- 172.21.15.94:0/47918326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82019f7a0 con 0x7fd820101c30 2024-02-17T11:55:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.295+0000 7fd824b3a640 1 -- 172.21.15.94:0/47918326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8201073a0 con 0x7fd820101c30 2024-02-17T11:55:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.296+0000 7fd80f7fe640 1 -- 172.21.15.94:0/47918326 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd81800a040 con 0x7fd820101c30 2024-02-17T11:55:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.297+0000 7fd80f7fe640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd8080a20c0 0x7fd8080a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.297+0000 7fd81e59c640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd8080a20c0 0x7fd8080a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.297+0000 7fd80f7fe640 1 -- 172.21.15.94:0/47918326 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd8180f0bb0 con 0x7fd820101c30 2024-02-17T11:55:55.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.297+0000 7fd81e59c640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd8080a20c0 0x7fd8080a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd8201010d0 tx=0x7fd81000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:55.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.301+0000 7fd80f7fe640 1 -- 172.21.15.94:0/47918326 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd8180b9f80 con 0x7fd820101c30 2024-02-17T11:55:55.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.479+0000 7fd824b3a640 1 -- 172.21.15.94:0/47918326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd82010bcd0 con 0x7fd820101c30 2024-02-17T11:55:55.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.479+0000 7fd80f7fe640 1 -- 172.21.15.94:0/47918326 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8180bd640 con 0x7fd820101c30 2024-02-17T11:55:55.478 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-02-17T11:55:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.482+0000 7fd824b3a640 1 -- 172.21.15.94:0/47918326 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd8080a20c0 msgr2=0x7fd8080a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.482+0000 7fd824b3a640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd8080a20c0 0x7fd8080a4580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd8201010d0 tx=0x7fd81000b390 comp rx=0 tx=0).stop 2024-02-17T11:55:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.482+0000 7fd824b3a640 1 -- 172.21.15.94:0/47918326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd820101c30 msgr2=0x7fd82019e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.482+0000 7fd824b3a640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd820101c30 0x7fd82019e5c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd818037040 tx=0x7fd818034d40 comp rx=0 tx=0).stop 2024-02-17T11:55:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.482+0000 7fd824b3a640 1 -- 172.21.15.94:0/47918326 shutdown_connections 2024-02-17T11:55:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.482+0000 7fd824b3a640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd8080a20c0 0x7fd8080a4580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.482+0000 7fd824b3a640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd820101c30 0x7fd82019e5c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.482+0000 7fd824b3a640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd820101270 0x7fd82019bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.482+0000 7fd824b3a640 1 --2- 172.21.15.94:0/47918326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd820100080 0x7fd820068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.482+0000 7fd824b3a640 1 -- 172.21.15.94:0/47918326 >> 172.21.15.94:0/47918326 conn(0x7fd8200fb610 msgr2=0x7fd820104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:55.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.482+0000 7fd824b3a640 1 -- 172.21.15.94:0/47918326 shutdown_connections 2024-02-17T11:55:55.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.482+0000 7fd824b3a640 1 -- 172.21.15.94:0/47918326 wait complete. 2024-02-17T11:55:55.492 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-17T11:55:55.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.596+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/4161264552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc0106ef0 msgr2=0x7f8bc01092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.596+0000 7f8bc7d3e640 1 --2- 172.21.15.94:0/4161264552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc0106ef0 0x7f8bc01092e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009ca0 tx=0x7f8bbc02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.596+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/4161264552 shutdown_connections 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.596+0000 7f8bc7d3e640 1 --2- 172.21.15.94:0/4161264552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc0106ef0 0x7f8bc01092e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.596+0000 7f8bc7d3e640 1 --2- 172.21.15.94:0/4161264552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc01045c0 0x7f8bc01069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.596+0000 7f8bc7d3e640 1 --2- 172.21.15.94:0/4161264552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bc0101c90 0x7f8bc0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.596+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/4161264552 >> 172.21.15.94:0/4161264552 conn(0x7f8bc00fb640 msgr2=0x7f8bc00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.597+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/4161264552 shutdown_connections 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.597+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/4161264552 wait complete. 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.597+0000 7f8bc7d3e640 1 Processor -- start 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.597+0000 7f8bc7d3e640 1 -- start start 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.597+0000 7f8bc7d3e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc0101c90 0x7f8bc01971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.597+0000 7f8bc7d3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc01045c0 0x7f8bc0197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.597+0000 7f8bc7d3e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bc0106ef0 0x7f8bc019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:55.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.597+0000 7f8bc7d3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc019ecd0 con 0x7f8bc01045c0 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.597+0000 7f8bc7d3e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc019ee40 con 0x7f8bc0106ef0 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.597+0000 7f8bc7d3e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc019f000 con 0x7f8bc0101c90 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.597+0000 7f8bc52d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc01045c0 0x7f8bc0197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.597+0000 7f8bc52d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc01045c0 0x7f8bc0197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37506/0 (socket says 172.21.15.94:37506) 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bc52d9640 1 -- 172.21.15.94:0/613997007 learned_addr learned my addr 172.21.15.94:0/613997007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bc5ada640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc0101c90 0x7f8bc01971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bc62db640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bc0106ef0 0x7f8bc019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bc52d9640 1 -- 172.21.15.94:0/613997007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc0101c90 msgr2=0x7f8bc01971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bc52d9640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc0101c90 0x7f8bc01971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bc52d9640 1 -- 172.21.15.94:0/613997007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bc0106ef0 msgr2=0x7f8bc019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bc52d9640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bc0106ef0 0x7f8bc019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bc52d9640 1 -- 172.21.15.94:0/613997007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc009900 con 0x7f8bc01045c0 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bc52d9640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc01045c0 0x7f8bc0197720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0005fd0 tx=0x7f8bb000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:55.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bb6ffd640 1 -- 172.21.15.94:0/613997007 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0009c30 con 0x7f8bc01045c0 2024-02-17T11:55:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/613997007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc019f2e0 con 0x7f8bc01045c0 2024-02-17T11:55:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bb6ffd640 1 -- 172.21.15.94:0/613997007 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bb0009d90 con 0x7f8bc01045c0 2024-02-17T11:55:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bb6ffd640 1 -- 172.21.15.94:0/613997007 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0010460 con 0x7f8bc01045c0 2024-02-17T11:55:55.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.598+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/613997007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc019f840 con 0x7f8bc01045c0 2024-02-17T11:55:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.599+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/613997007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc00fd130 con 0x7f8bc01045c0 2024-02-17T11:55:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.600+0000 7f8bb6ffd640 1 -- 172.21.15.94:0/613997007 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8bb0003f20 con 0x7f8bc01045c0 2024-02-17T11:55:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.601+0000 7f8bb6ffd640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8bac0a20c0 0x7f8bac0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.601+0000 7f8bc5ada640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8bac0a20c0 0x7f8bac0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.601+0000 7f8bb6ffd640 1 -- 172.21.15.94:0/613997007 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8bb00cd360 con 0x7f8bc01045c0 2024-02-17T11:55:55.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.601+0000 7f8bc5ada640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8bac0a20c0 0x7f8bac0a4580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005d00 tx=0x7f8bb800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:55.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.606+0000 7f8bb6ffd640 1 -- 172.21.15.94:0/613997007 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8bb00967d0 con 0x7f8bc01045c0 2024-02-17T11:55:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.800+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/613997007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8bc010cd70 con 0x7f8bc01045c0 2024-02-17T11:55:55.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.800+0000 7f8bb6ffd640 1 -- 172.21.15.94:0/613997007 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8bb0099e90 con 0x7f8bc01045c0 2024-02-17T11:55:55.799 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:55.799 INFO:teuthology.orchestra.run.smithi094.stdout:[ 2024-02-17T11:55:55.800 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T11:55:55.800 INFO:teuthology.orchestra.run.smithi094.stdout: "poolnum": 1, 2024-02-17T11:55:55.800 INFO:teuthology.orchestra.run.smithi094.stdout: "poolname": ".mgr" 2024-02-17T11:55:55.800 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T11:55:55.800 INFO:teuthology.orchestra.run.smithi094.stdout:] 2024-02-17T11:55:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.802+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/613997007 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8bac0a20c0 msgr2=0x7f8bac0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.802+0000 7f8bc7d3e640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8bac0a20c0 0x7f8bac0a4580 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8005d00 tx=0x7f8bb800b410 comp rx=0 tx=0).stop 2024-02-17T11:55:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.802+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/613997007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc01045c0 msgr2=0x7f8bc0197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.802+0000 7f8bc7d3e640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc01045c0 0x7f8bc0197720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0005fd0 tx=0x7f8bb000dde0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.802+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/613997007 shutdown_connections 2024-02-17T11:55:55.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.802+0000 7f8bc7d3e640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8bac0a20c0 0x7f8bac0a4580 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.802+0000 7f8bc7d3e640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bc0106ef0 0x7f8bc019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.802+0000 7f8bc7d3e640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bc01045c0 0x7f8bc0197720 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.802+0000 7f8bc7d3e640 1 --2- 172.21.15.94:0/613997007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc0101c90 0x7f8bc01971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.802+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/613997007 >> 172.21.15.94:0/613997007 conn(0x7f8bc00fb640 msgr2=0x7f8bc01025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:55.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.802+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/613997007 shutdown_connections 2024-02-17T11:55:55.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.802+0000 7f8bc7d3e640 1 -- 172.21.15.94:0/613997007 wait complete. 2024-02-17T11:55:55.811 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:55 smithi094 ceph-mon[32499]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:55.811 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:55 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/20767090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:55.811 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:55 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:55.811 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:55 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:55.811 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:55 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:55.811 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:55 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:55.811 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:55 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2936567882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:55:55.811 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:55 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/47918326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:55:55.812 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-17T11:55:55.812 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-17T11:55:55.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:55 smithi189 ceph-mon[36588]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:55.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:55 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/20767090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:55.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:55 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:55.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:55 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:55.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:55 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:55.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:55 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:55.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:55 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2936567882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:55:55.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:55 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/47918326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:55:55.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.919+0000 7f788c08d640 1 -- 172.21.15.94:0/3934772670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884101270 msgr2=0x7f78841016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.919+0000 7f788c08d640 1 --2- 172.21.15.94:0/3934772670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884101270 0x7f78841016f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7880009900 tx=0x7f788002bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:55.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.919+0000 7f788c08d640 1 -- 172.21.15.94:0/3934772670 shutdown_connections 2024-02-17T11:55:55.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.919+0000 7f788c08d640 1 --2- 172.21.15.94:0/3934772670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7884101c30 0x7f78841084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.919+0000 7f788c08d640 1 --2- 172.21.15.94:0/3934772670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884101270 0x7f78841016f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.919+0000 7f788c08d640 1 --2- 172.21.15.94:0/3934772670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7884100080 0x7f78841004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.919+0000 7f788c08d640 1 -- 172.21.15.94:0/3934772670 >> 172.21.15.94:0/3934772670 conn(0x7f78840fb610 msgr2=0x7f78840fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:55.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f788c08d640 1 -- 172.21.15.94:0/3934772670 shutdown_connections 2024-02-17T11:55:55.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f788c08d640 1 -- 172.21.15.94:0/3934772670 wait complete. 2024-02-17T11:55:55.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f788c08d640 1 Processor -- start 2024-02-17T11:55:55.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f788c08d640 1 -- start start 2024-02-17T11:55:55.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f788c08d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7884100080 0x7f78841961f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:55.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f788c08d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884101270 0x7f7884196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f788c08d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7884101c30 0x7f788419d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f788c08d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788419dce0 con 0x7f7884101270 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f788c08d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788419de50 con 0x7f7884100080 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f788c08d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788419e010 con 0x7f7884101c30 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f7889628640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884101270 0x7f7884196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f7889628640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884101270 0x7f7884196730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37534/0 (socket says 172.21.15.94:37534) 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f7889628640 1 -- 172.21.15.94:0/74557572 learned_addr learned my addr 172.21.15.94:0/74557572 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.920+0000 7f788a62a640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7884101c30 0x7f788419d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.921+0000 7f7889628640 1 -- 172.21.15.94:0/74557572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7884101c30 msgr2=0x7f788419d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.921+0000 7f7889628640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7884101c30 0x7f788419d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.921+0000 7f7889e29640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7884100080 0x7f78841961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.921+0000 7f7889628640 1 -- 172.21.15.94:0/74557572 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7884100080 msgr2=0x7f78841961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.921+0000 7f7889628640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7884100080 0x7f78841961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:55.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.921+0000 7f7889628640 1 -- 172.21.15.94:0/74557572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7880009560 con 0x7f7884101270 2024-02-17T11:55:55.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.921+0000 7f7889628640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884101270 0x7f7884196730 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7880006010 tx=0x7f78800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:55.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.921+0000 7f787affd640 1 -- 172.21.15.94:0/74557572 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78800044e0 con 0x7f7884101270 2024-02-17T11:55:55.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.921+0000 7f787affd640 1 -- 172.21.15.94:0/74557572 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78800054e0 con 0x7f7884101270 2024-02-17T11:55:55.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.921+0000 7f788c08d640 1 -- 172.21.15.94:0/74557572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788419e290 con 0x7f7884101270 2024-02-17T11:55:55.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.921+0000 7f787affd640 1 -- 172.21.15.94:0/74557572 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788003e450 con 0x7f7884101270 2024-02-17T11:55:55.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.921+0000 7f788c08d640 1 -- 172.21.15.94:0/74557572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788419e870 con 0x7f7884101270 2024-02-17T11:55:55.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.922+0000 7f788c08d640 1 -- 172.21.15.94:0/74557572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78841073c0 con 0x7f7884101270 2024-02-17T11:55:55.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.923+0000 7f787affd640 1 -- 172.21.15.94:0/74557572 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f78800035f0 con 0x7f7884101270 2024-02-17T11:55:55.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.923+0000 7f787affd640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78700a2140 0x7f78700a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:55.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.924+0000 7f7889e29640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78700a2140 0x7f78700a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:55.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.924+0000 7f787affd640 1 -- 172.21.15.94:0/74557572 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f78800f01d0 con 0x7f7884101270 2024-02-17T11:55:55.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.924+0000 7f7889e29640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78700a2140 0x7f78700a4600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f78841010d0 tx=0x7f7874008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:55.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:55.927+0000 7f787affd640 1 -- 172.21.15.94:0/74557572 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f78800b9640 con 0x7f7884101270 2024-02-17T11:55:55.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:55 smithi125 ceph-mon[36819]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:55.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:55 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/20767090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:55:55.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:55 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:55.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:55 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:55.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:55 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:55.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:55 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:55:55.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:55 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2936567882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:55:55.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:55 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/47918326' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:55:56.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.110+0000 7f788c08d640 1 -- 172.21.15.94:0/74557572 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f788410bd60 con 0x7f7884101270 2024-02-17T11:55:56.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.645+0000 7f787affd640 1 -- 172.21.15.94:0/74557572 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7f78800bcd00 con 0x7f7884101270 2024-02-17T11:55:56.644 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' created 2024-02-17T11:55:56.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.648+0000 7f788c08d640 1 -- 172.21.15.94:0/74557572 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78700a2140 msgr2=0x7f78700a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:56.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.648+0000 7f788c08d640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78700a2140 0x7f78700a4600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f78841010d0 tx=0x7f7874008040 comp rx=0 tx=0).stop 2024-02-17T11:55:56.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.648+0000 7f788c08d640 1 -- 172.21.15.94:0/74557572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884101270 msgr2=0x7f7884196730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:56.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.648+0000 7f788c08d640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884101270 0x7f7884196730 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7880006010 tx=0x7f78800042f0 comp rx=0 tx=0).stop 2024-02-17T11:55:56.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.649+0000 7f788c08d640 1 -- 172.21.15.94:0/74557572 shutdown_connections 2024-02-17T11:55:56.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.649+0000 7f788c08d640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78700a2140 0x7f78700a4600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:56.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.649+0000 7f788c08d640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7884101c30 0x7f788419d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:56.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.649+0000 7f788c08d640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884101270 0x7f7884196730 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:56.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.649+0000 7f788c08d640 1 --2- 172.21.15.94:0/74557572 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7884100080 0x7f78841961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:56.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.649+0000 7f788c08d640 1 -- 172.21.15.94:0/74557572 >> 172.21.15.94:0/74557572 conn(0x7f78840fb610 msgr2=0x7f7884104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:56.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.649+0000 7f788c08d640 1 -- 172.21.15.94:0/74557572 shutdown_connections 2024-02-17T11:55:56.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.649+0000 7f788c08d640 1 -- 172.21.15.94:0/74557572 wait complete. 2024-02-17T11:55:56.663 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-02-17T11:55:56.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.785+0000 7fea42440640 1 -- 172.21.15.94:0/788022908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea3c1045d0 msgr2=0x7fea3c1069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:56.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.785+0000 7fea42440640 1 --2- 172.21.15.94:0/788022908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea3c1045d0 0x7fea3c1069c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fea30009900 tx=0x7fea3002bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.785+0000 7fea42440640 1 -- 172.21.15.94:0/788022908 shutdown_connections 2024-02-17T11:55:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.785+0000 7fea42440640 1 --2- 172.21.15.94:0/788022908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea3c106f00 0x7fea3c1092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.785+0000 7fea42440640 1 --2- 172.21.15.94:0/788022908 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea3c1045d0 0x7fea3c1069c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.785+0000 7fea42440640 1 --2- 172.21.15.94:0/788022908 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea3c101ca0 0x7fea3c104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.785+0000 7fea42440640 1 -- 172.21.15.94:0/788022908 >> 172.21.15.94:0/788022908 conn(0x7fea3c0fb5f0 msgr2=0x7fea3c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.785+0000 7fea42440640 1 -- 172.21.15.94:0/788022908 shutdown_connections 2024-02-17T11:55:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.785+0000 7fea42440640 1 -- 172.21.15.94:0/788022908 wait complete. 2024-02-17T11:55:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.786+0000 7fea42440640 1 Processor -- start 2024-02-17T11:55:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.786+0000 7fea42440640 1 -- start start 2024-02-17T11:55:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.786+0000 7fea42440640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea3c101ca0 0x7fea3c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.786+0000 7fea42440640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea3c1045d0 0x7fea3c197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:56.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.786+0000 7fea42440640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea3c106f00 0x7fea3c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:56.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.786+0000 7fea42440640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c19ed40 con 0x7fea3c1045d0 2024-02-17T11:55:56.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.786+0000 7fea42440640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c19eeb0 con 0x7fea3c101ca0 2024-02-17T11:55:56.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.786+0000 7fea42440640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c19f070 con 0x7fea3c106f00 2024-02-17T11:55:56.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.786+0000 7fea3b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea3c1045d0 0x7fea3c197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:56.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.786+0000 7fea3b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea3c1045d0 0x7fea3c197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37546/0 (socket says 172.21.15.94:37546) 2024-02-17T11:55:56.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.786+0000 7fea3b7fe640 1 -- 172.21.15.94:0/1161362824 learned_addr learned my addr 172.21.15.94:0/1161362824 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:56.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.786+0000 7fea3bfff640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea3c101ca0 0x7fea3c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:56.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.786+0000 7fea409dd640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea3c106f00 0x7fea3c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:56.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.787+0000 7fea3b7fe640 1 -- 172.21.15.94:0/1161362824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea3c106f00 msgr2=0x7fea3c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:56.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.787+0000 7fea3b7fe640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea3c106f00 0x7fea3c19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:56.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.787+0000 7fea3b7fe640 1 -- 172.21.15.94:0/1161362824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea3c101ca0 msgr2=0x7fea3c197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:56.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.787+0000 7fea3b7fe640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea3c101ca0 0x7fea3c197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:56.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.787+0000 7fea3b7fe640 1 -- 172.21.15.94:0/1161362824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea30009560 con 0x7fea3c1045d0 2024-02-17T11:55:56.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.787+0000 7fea3b7fe640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea3c1045d0 0x7fea3c197790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea30005ab0 tx=0x7fea30005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:56.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.787+0000 7fea397fa640 1 -- 172.21.15.94:0/1161362824 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300043d0 con 0x7fea3c1045d0 2024-02-17T11:55:56.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.787+0000 7fea42440640 1 -- 172.21.15.94:0/1161362824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3c19f2f0 con 0x7fea3c1045d0 2024-02-17T11:55:56.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.787+0000 7fea42440640 1 -- 172.21.15.94:0/1161362824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3c19f850 con 0x7fea3c1045d0 2024-02-17T11:55:56.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.787+0000 7fea397fa640 1 -- 172.21.15.94:0/1161362824 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea300052d0 con 0x7fea3c1045d0 2024-02-17T11:55:56.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.787+0000 7fea397fa640 1 -- 172.21.15.94:0/1161362824 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30003d40 con 0x7fea3c1045d0 2024-02-17T11:55:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.788+0000 7fea42440640 1 -- 172.21.15.94:0/1161362824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3c0fd0e0 con 0x7fea3c1045d0 2024-02-17T11:55:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.789+0000 7fea397fa640 1 -- 172.21.15.94:0/1161362824 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fea30005440 con 0x7fea3c1045d0 2024-02-17T11:55:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.790+0000 7fea397fa640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fea240a20c0 0x7fea240a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.790+0000 7fea3bfff640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fea240a20c0 0x7fea240a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.790+0000 7fea397fa640 1 -- 172.21.15.94:0/1161362824 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fea300ef7f0 con 0x7fea3c1045d0 2024-02-17T11:55:56.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.791+0000 7fea3bfff640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fea240a20c0 0x7fea240a4580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009bd0 tx=0x7fea2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:56.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.799+0000 7fea397fa640 1 -- 172.21.15.94:0/1161362824 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fea300b8af0 con 0x7fea3c1045d0 2024-02-17T11:55:56.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:56 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/613997007' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-17T11:55:56.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:56 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/74557572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-17T11:55:56.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:56 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/613997007' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-17T11:55:56.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:56 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/74557572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-17T11:55:56.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:56 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/613997007' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-17T11:55:56.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:56 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/74557572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-17T11:55:56.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:56.995+0000 7fea42440640 1 -- 172.21.15.94:0/1161362824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fea3c0fcb50 con 0x7fea3c1045d0 2024-02-17T11:55:57.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.651+0000 7fea397fa640 1 -- 172.21.15.94:0/1161362824 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7fea300bc1b0 con 0x7fea3c1045d0 2024-02-17T11:55:57.650 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' created 2024-02-17T11:55:57.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.654+0000 7fea42440640 1 -- 172.21.15.94:0/1161362824 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fea240a20c0 msgr2=0x7fea240a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:57.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.654+0000 7fea42440640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fea240a20c0 0x7fea240a4580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009bd0 tx=0x7fea2c005f90 comp rx=0 tx=0).stop 2024-02-17T11:55:57.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.654+0000 7fea42440640 1 -- 172.21.15.94:0/1161362824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea3c1045d0 msgr2=0x7fea3c197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:57.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.654+0000 7fea42440640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea3c1045d0 0x7fea3c197790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fea30005ab0 tx=0x7fea30005e50 comp rx=0 tx=0).stop 2024-02-17T11:55:57.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.654+0000 7fea42440640 1 -- 172.21.15.94:0/1161362824 shutdown_connections 2024-02-17T11:55:57.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.654+0000 7fea42440640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fea240a20c0 0x7fea240a4580 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:57.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.654+0000 7fea42440640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea3c106f00 0x7fea3c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:57.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.654+0000 7fea42440640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea3c1045d0 0x7fea3c197790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:57.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.654+0000 7fea42440640 1 --2- 172.21.15.94:0/1161362824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea3c101ca0 0x7fea3c197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:57.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.654+0000 7fea42440640 1 -- 172.21.15.94:0/1161362824 >> 172.21.15.94:0/1161362824 conn(0x7fea3c0fb5f0 msgr2=0x7fea3c102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:57.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.654+0000 7fea42440640 1 -- 172.21.15.94:0/1161362824 shutdown_connections 2024-02-17T11:55:57.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.654+0000 7fea42440640 1 -- 172.21.15.94:0/1161362824 wait complete. 2024-02-17T11:55:57.672 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-17T11:55:57.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.779+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/2966462053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100080 msgr2=0x7fbc541004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:57.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.779+0000 7fbc5bcfb640 1 --2- 172.21.15.94:0/2966462053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100080 0x7fbc541004a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009ca0 tx=0x7fbc5002c0f0 comp rx=0 tx=0).stop 2024-02-17T11:55:57.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.779+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/2966462053 shutdown_connections 2024-02-17T11:55:57.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.779+0000 7fbc5bcfb640 1 --2- 172.21.15.94:0/2966462053 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc54101c30 0x7fbc541084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:57.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.779+0000 7fbc5bcfb640 1 --2- 172.21.15.94:0/2966462053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc54101270 0x7fbc541016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:57.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.779+0000 7fbc5bcfb640 1 --2- 172.21.15.94:0/2966462053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100080 0x7fbc541004a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:57.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.779+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/2966462053 >> 172.21.15.94:0/2966462053 conn(0x7fbc540fb610 msgr2=0x7fbc540fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:57.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.779+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/2966462053 shutdown_connections 2024-02-17T11:55:57.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.779+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/2966462053 wait complete. 2024-02-17T11:55:57.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc5bcfb640 1 Processor -- start 2024-02-17T11:55:57.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc5bcfb640 1 -- start start 2024-02-17T11:55:57.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc5bcfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100080 0x7fbc541972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:57.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc5bcfb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc54101270 0x7fbc54197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:57.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc5bcfb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc54101c30 0x7fbc5419e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:57.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc5bcfb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5419edd0 con 0x7fbc54100080 2024-02-17T11:55:57.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc5bcfb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5419ef40 con 0x7fbc54101270 2024-02-17T11:55:57.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc5bcfb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5419f100 con 0x7fbc54101c30 2024-02-17T11:55:57.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc59a97640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100080 0x7fbc541972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:57.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc59a97640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100080 0x7fbc541972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37570/0 (socket says 172.21.15.94:37570) 2024-02-17T11:55:57.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc59a97640 1 -- 172.21.15.94:0/1723941468 learned_addr learned my addr 172.21.15.94:0/1723941468 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:57.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc59296640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc54101270 0x7fbc54197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:57.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc5a298640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc54101c30 0x7fbc5419e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:57.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc59a97640 1 -- 172.21.15.94:0/1723941468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc54101c30 msgr2=0x7fbc5419e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:57.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc59a97640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc54101c30 0x7fbc5419e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:57.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc59a97640 1 -- 172.21.15.94:0/1723941468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc54101270 msgr2=0x7fbc54197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:57.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.780+0000 7fbc59a97640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc54101270 0x7fbc54197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:57.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.781+0000 7fbc59a97640 1 -- 172.21.15.94:0/1723941468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc50009900 con 0x7fbc54100080 2024-02-17T11:55:57.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.781+0000 7fbc59296640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc54101270 0x7fbc54197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:55:57.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.781+0000 7fbc5a298640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc54101c30 0x7fbc5419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:55:57.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.781+0000 7fbc59a97640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100080 0x7fbc541972e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbc50037040 tx=0x7fbc50034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:57.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.781+0000 7fbc4affd640 1 -- 172.21.15.94:0/1723941468 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50032eb0 con 0x7fbc54100080 2024-02-17T11:55:57.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.781+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/1723941468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5419f380 con 0x7fbc54100080 2024-02-17T11:55:57.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.781+0000 7fbc4affd640 1 -- 172.21.15.94:0/1723941468 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc50003e50 con 0x7fbc54100080 2024-02-17T11:55:57.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.781+0000 7fbc4affd640 1 -- 172.21.15.94:0/1723941468 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc50040470 con 0x7fbc54100080 2024-02-17T11:55:57.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.781+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/1723941468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5419f830 con 0x7fbc54100080 2024-02-17T11:55:57.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.782+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/1723941468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5410ba90 con 0x7fbc54100080 2024-02-17T11:55:57.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.783+0000 7fbc4affd640 1 -- 172.21.15.94:0/1723941468 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fbc5000a040 con 0x7fbc54100080 2024-02-17T11:55:57.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.783+0000 7fbc4affd640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbc400a20c0 0x7fbc400a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:57.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.784+0000 7fbc59296640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbc400a20c0 0x7fbc400a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:57.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.784+0000 7fbc4affd640 1 -- 172.21.15.94:0/1723941468 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fbc500f0e90 con 0x7fbc54100080 2024-02-17T11:55:57.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.784+0000 7fbc59296640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbc400a20c0 0x7fbc400a4580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbc44005d00 tx=0x7fbc44005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:57.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.787+0000 7fbc4affd640 1 -- 172.21.15.94:0/1723941468 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc500ba010 con 0x7fbc54100080 2024-02-17T11:55:57.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:57 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/74557572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-17T11:55:57.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:57 smithi125 ceph-mon[36819]: osdmap e82: 12 total, 12 up, 12 in 2024-02-17T11:55:57.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:57 smithi125 ceph-mon[36819]: pgmap v281: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:57.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:57.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:57.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:57 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1161362824' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-17T11:55:57.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:57 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/74557572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-17T11:55:57.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:57 smithi094 ceph-mon[32499]: osdmap e82: 12 total, 12 up, 12 in 2024-02-17T11:55:57.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:57 smithi094 ceph-mon[32499]: pgmap v281: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:57.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:57.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:57.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:57.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:57 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1161362824' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-17T11:55:57.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:57.998+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/1723941468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbc5419fa50 con 0x7fbc54100080 2024-02-17T11:55:58.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:57 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/74557572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-17T11:55:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:57 smithi189 ceph-mon[36588]: osdmap e82: 12 total, 12 up, 12 in 2024-02-17T11:55:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:57 smithi189 ceph-mon[36588]: pgmap v281: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:55:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:55:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:55:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:57 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1161362824' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-17T11:55:58.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.653+0000 7fbc4affd640 1 -- 172.21.15.94:0/1723941468 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbc500bd6d0 con 0x7fbc54100080 2024-02-17T11:55:58.652 INFO:teuthology.orchestra.run.smithi094.stderr:new fs with metadata pool 2 and data pool 3 2024-02-17T11:55:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.656+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/1723941468 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbc400a20c0 msgr2=0x7fbc400a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.656+0000 7fbc5bcfb640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbc400a20c0 0x7fbc400a4580 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbc44005d00 tx=0x7fbc44005c70 comp rx=0 tx=0).stop 2024-02-17T11:55:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.656+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/1723941468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100080 msgr2=0x7fbc541972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.656+0000 7fbc5bcfb640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100080 0x7fbc541972e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbc50037040 tx=0x7fbc50034d40 comp rx=0 tx=0).stop 2024-02-17T11:55:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.657+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/1723941468 shutdown_connections 2024-02-17T11:55:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.657+0000 7fbc5bcfb640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbc400a20c0 0x7fbc400a4580 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.657+0000 7fbc5bcfb640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc54101c30 0x7fbc5419e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.657+0000 7fbc5bcfb640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc54101270 0x7fbc54197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.657+0000 7fbc5bcfb640 1 --2- 172.21.15.94:0/1723941468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc54100080 0x7fbc541972e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.657+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/1723941468 >> 172.21.15.94:0/1723941468 conn(0x7fbc540fb610 msgr2=0x7fbc54068390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:58.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.657+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/1723941468 shutdown_connections 2024-02-17T11:55:58.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.657+0000 7fbc5bcfb640 1 -- 172.21.15.94:0/1723941468 wait complete. 2024-02-17T11:55:58.679 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:55:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 -- 172.21.15.94:0/107778182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884101310 msgr2=0x7f7884101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 --2- 172.21.15.94:0/107778182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884101310 0x7f7884101790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7870009900 tx=0x7f787002bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 -- 172.21.15.94:0/107778182 shutdown_connections 2024-02-17T11:55:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 --2- 172.21.15.94:0/107778182 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7884101cd0 0x7f7884108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 --2- 172.21.15.94:0/107778182 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884101310 0x7f7884101790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 --2- 172.21.15.94:0/107778182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7884100120 0x7f7884100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 -- 172.21.15.94:0/107778182 >> 172.21.15.94:0/107778182 conn(0x7f78840fb6b0 msgr2=0x7f78840fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 -- 172.21.15.94:0/107778182 shutdown_connections 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 -- 172.21.15.94:0/107778182 wait complete. 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 Processor -- start 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 -- start start 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884100120 0x7f7884197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7884101310 0x7f7884197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7884101cd0 0x7f788419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788419eef0 con 0x7f7884100120 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788419f060 con 0x7f7884101cd0 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f788ad8c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788419f220 con 0x7f7884101310 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f7888b28640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884100120 0x7f7884197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f7888b28640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884100120 0x7f7884197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37588/0 (socket says 172.21.15.94:37588) 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.790+0000 7f7888b28640 1 -- 172.21.15.94:0/455076501 learned_addr learned my addr 172.21.15.94:0/455076501 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.791+0000 7f7889329640 1 --2- 172.21.15.94:0/455076501 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7884101cd0 0x7f788419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:58.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.791+0000 7f7888b28640 1 -- 172.21.15.94:0/455076501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7884101310 msgr2=0x7f7884197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:58.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.791+0000 7f7888b28640 1 --2- 172.21.15.94:0/455076501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7884101310 0x7f7884197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.791+0000 7f7888b28640 1 -- 172.21.15.94:0/455076501 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7884101cd0 msgr2=0x7f788419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:58.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.791+0000 7f7888b28640 1 --2- 172.21.15.94:0/455076501 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7884101cd0 0x7f788419e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.791+0000 7f7888b28640 1 -- 172.21.15.94:0/455076501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7870009560 con 0x7f7884100120 2024-02-17T11:55:58.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.791+0000 7f7888b28640 1 --2- 172.21.15.94:0/455076501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884100120 0x7f7884197450 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f787800b450 tx=0x7f787800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:58.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.791+0000 7f7881ffb640 1 -- 172.21.15.94:0/455076501 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878011890 con 0x7f7884100120 2024-02-17T11:55:58.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.791+0000 7f7881ffb640 1 -- 172.21.15.94:0/455076501 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78780103e0 con 0x7f7884100120 2024-02-17T11:55:58.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.791+0000 7f7881ffb640 1 -- 172.21.15.94:0/455076501 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787800f580 con 0x7f7884100120 2024-02-17T11:55:58.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.791+0000 7f788ad8c640 1 -- 172.21.15.94:0/455076501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788419f500 con 0x7f7884100120 2024-02-17T11:55:58.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.791+0000 7f788ad8c640 1 -- 172.21.15.94:0/455076501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788419fa90 con 0x7f7884100120 2024-02-17T11:55:58.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.792+0000 7f788ad8c640 1 -- 172.21.15.94:0/455076501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788410bba0 con 0x7f7884100120 2024-02-17T11:55:58.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.793+0000 7f7881ffb640 1 -- 172.21.15.94:0/455076501 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f78780108e0 con 0x7f7884100120 2024-02-17T11:55:58.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.793+0000 7f7881ffb640 1 --2- 172.21.15.94:0/455076501 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78740a2140 0x7f78740a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:58.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.794+0000 7f7883fff640 1 --2- 172.21.15.94:0/455076501 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78740a2140 0x7f78740a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:58.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.794+0000 7f7881ffb640 1 -- 172.21.15.94:0/455076501 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f78780cd950 con 0x7f7884100120 2024-02-17T11:55:58.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.794+0000 7f7883fff640 1 --2- 172.21.15.94:0/455076501 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78740a2140 0x7f78740a4600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f787002c6a0 tx=0x7f7870000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:58.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.798+0000 7f7881ffb640 1 -- 172.21.15.94:0/455076501 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7878096a70 con 0x7f7884100120 2024-02-17T11:55:58.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:58 smithi125 ceph-mon[36819]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-17T11:55:58.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:58 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1161362824' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-17T11:55:58.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:58 smithi125 ceph-mon[36819]: osdmap e83: 12 total, 12 up, 12 in 2024-02-17T11:55:58.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:58 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1723941468' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-17T11:55:58.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:58 smithi125 ceph-mon[36819]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-17T11:55:58.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:58 smithi094 ceph-mon[32499]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-17T11:55:58.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:58 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1161362824' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-17T11:55:58.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:58 smithi094 ceph-mon[32499]: osdmap e83: 12 total, 12 up, 12 in 2024-02-17T11:55:58.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:58 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1723941468' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-17T11:55:58.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:58 smithi094 ceph-mon[32499]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-17T11:55:58.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.972+0000 7f788ad8c640 1 -- 172.21.15.94:0/455076501 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f788419fec0 con 0x7f7884100120 2024-02-17T11:55:58.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.974+0000 7f7881ffb640 1 -- 172.21.15.94:0/455076501 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22220 (secure 0 0 0) 0x7f787809a130 con 0x7f7884100120 2024-02-17T11:55:58.973 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:58.973 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":84,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:55:58.000271+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:55:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.976+0000 7f788ad8c640 1 -- 172.21.15.94:0/455076501 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78740a2140 msgr2=0x7f78740a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.976+0000 7f788ad8c640 1 --2- 172.21.15.94:0/455076501 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78740a2140 0x7f78740a4600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f787002c6a0 tx=0x7f7870000bc0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.976+0000 7f788ad8c640 1 -- 172.21.15.94:0/455076501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884100120 msgr2=0x7f7884197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.976+0000 7f788ad8c640 1 --2- 172.21.15.94:0/455076501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884100120 0x7f7884197450 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f787800b450 tx=0x7f787800b910 comp rx=0 tx=0).stop 2024-02-17T11:55:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.976+0000 7f788ad8c640 1 -- 172.21.15.94:0/455076501 shutdown_connections 2024-02-17T11:55:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.976+0000 7f788ad8c640 1 --2- 172.21.15.94:0/455076501 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78740a2140 0x7f78740a4600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.976+0000 7f788ad8c640 1 --2- 172.21.15.94:0/455076501 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7884101cd0 0x7f788419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.976+0000 7f788ad8c640 1 --2- 172.21.15.94:0/455076501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7884101310 0x7f7884197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.976+0000 7f788ad8c640 1 --2- 172.21.15.94:0/455076501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7884100120 0x7f7884197450 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.976+0000 7f788ad8c640 1 -- 172.21.15.94:0/455076501 >> 172.21.15.94:0/455076501 conn(0x7f78840fb6b0 msgr2=0x7f78840697e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:58.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.976+0000 7f788ad8c640 1 -- 172.21.15.94:0/455076501 shutdown_connections 2024-02-17T11:55:58.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:58.976+0000 7f788ad8c640 1 -- 172.21.15.94:0/455076501 wait complete. 2024-02-17T11:55:58.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:55:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.089+0000 7fa54d32c640 1 -- 172.21.15.94:0/3261635213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa548101c70 msgr2=0x7fa548108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:59.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.089+0000 7fa54d32c640 1 --2- 172.21.15.94:0/3261635213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa548101c70 0x7fa548108500 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa540009ca0 tx=0x7fa54002c0f0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.089+0000 7fa54d32c640 1 -- 172.21.15.94:0/3261635213 shutdown_connections 2024-02-17T11:55:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.089+0000 7fa54d32c640 1 --2- 172.21.15.94:0/3261635213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa548101c70 0x7fa548108500 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.089+0000 7fa54d32c640 1 --2- 172.21.15.94:0/3261635213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5481012b0 0x7fa548101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.089+0000 7fa54d32c640 1 --2- 172.21.15.94:0/3261635213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5481000c0 0x7fa5481004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.089+0000 7fa54d32c640 1 -- 172.21.15.94:0/3261635213 >> 172.21.15.94:0/3261635213 conn(0x7fa5480fb630 msgr2=0x7fa5480fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.089+0000 7fa54d32c640 1 -- 172.21.15.94:0/3261635213 shutdown_connections 2024-02-17T11:55:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.089+0000 7fa54d32c640 1 -- 172.21.15.94:0/3261635213 wait complete. 2024-02-17T11:55:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.089+0000 7fa54d32c640 1 Processor -- start 2024-02-17T11:55:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa54d32c640 1 -- start start 2024-02-17T11:55:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa54d32c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5481000c0 0x7fa54806a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa54d32c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5481012b0 0x7fa54819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:59.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa54d32c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa548101c70 0x7fa54819e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa54d32c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54819ed40 con 0x7fa5481000c0 2024-02-17T11:55:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa54d32c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54819eeb0 con 0x7fa5481012b0 2024-02-17T11:55:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa54d32c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54819f070 con 0x7fa548101c70 2024-02-17T11:55:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa546ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5481000c0 0x7fa54806a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa546ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5481000c0 0x7fa54806a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37598/0 (socket says 172.21.15.94:37598) 2024-02-17T11:55:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa546ffd640 1 -- 172.21.15.94:0/755884889 learned_addr learned my addr 172.21.15.94:0/755884889 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa53e7fc640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5481012b0 0x7fa54819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa5477fe640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa548101c70 0x7fa54819e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa546ffd640 1 -- 172.21.15.94:0/755884889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa548101c70 msgr2=0x7fa54819e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:59.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa546ffd640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa548101c70 0x7fa54819e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa546ffd640 1 -- 172.21.15.94:0/755884889 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5481012b0 msgr2=0x7fa54819bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa546ffd640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5481012b0 0x7fa54819bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa546ffd640 1 -- 172.21.15.94:0/755884889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa540009900 con 0x7fa5481000c0 2024-02-17T11:55:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa53e7fc640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5481012b0 0x7fa54819bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:55:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa5477fe640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa548101c70 0x7fa54819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:55:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa546ffd640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5481000c0 0x7fa54806a320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa53800c970 tx=0x7fa53800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa544ff9640 1 -- 172.21.15.94:0/755884889 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538004220 con 0x7fa5481000c0 2024-02-17T11:55:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa544ff9640 1 -- 172.21.15.94:0/755884889 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa538007e90 con 0x7fa5481000c0 2024-02-17T11:55:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa54d32c640 1 -- 172.21.15.94:0/755884889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa54819f2f0 con 0x7fa5481000c0 2024-02-17T11:55:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa544ff9640 1 -- 172.21.15.94:0/755884889 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53800f450 con 0x7fa5481000c0 2024-02-17T11:55:59.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.090+0000 7fa54d32c640 1 -- 172.21.15.94:0/755884889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa54819f800 con 0x7fa5481000c0 2024-02-17T11:55:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.091+0000 7fa54d32c640 1 -- 172.21.15.94:0/755884889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5481073e0 con 0x7fa5481000c0 2024-02-17T11:55:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.092+0000 7fa544ff9640 1 -- 172.21.15.94:0/755884889 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa538003610 con 0x7fa5481000c0 2024-02-17T11:55:59.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.093+0000 7fa544ff9640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa5340a20c0 0x7fa5340a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:59.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.093+0000 7fa53e7fc640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa5340a20c0 0x7fa5340a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:59.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.093+0000 7fa544ff9640 1 -- 172.21.15.94:0/755884889 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa5380cd8d0 con 0x7fa5481000c0 2024-02-17T11:55:59.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.094+0000 7fa53e7fc640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa5340a20c0 0x7fa5340a4580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa54806a090 tx=0x7fa530005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:59.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.098+0000 7fa544ff9640 1 -- 172.21.15.94:0/755884889 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa5380969f0 con 0x7fa5481000c0 2024-02-17T11:55:59.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:58 smithi189 ceph-mon[36588]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-17T11:55:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:58 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1161362824' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-17T11:55:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:58 smithi189 ceph-mon[36588]: osdmap e83: 12 total, 12 up, 12 in 2024-02-17T11:55:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:58 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1723941468' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-17T11:55:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:58 smithi189 ceph-mon[36588]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-17T11:55:59.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.273+0000 7fa54d32c640 1 -- 172.21.15.94:0/755884889 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa54810bd30 con 0x7fa5481000c0 2024-02-17T11:55:59.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.274+0000 7fa544ff9640 1 -- 172.21.15.94:0/755884889 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22220 (secure 0 0 0) 0x7fa53809a0b0 con 0x7fa5481000c0 2024-02-17T11:55:59.273 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:55:59.273 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":84,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:55:58.000271+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:55:59.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.276+0000 7fa54d32c640 1 -- 172.21.15.94:0/755884889 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa5340a20c0 msgr2=0x7fa5340a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:59.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.276+0000 7fa54d32c640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa5340a20c0 0x7fa5340a4580 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa54806a090 tx=0x7fa530005d30 comp rx=0 tx=0).stop 2024-02-17T11:55:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.276+0000 7fa54d32c640 1 -- 172.21.15.94:0/755884889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5481000c0 msgr2=0x7fa54806a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.276+0000 7fa54d32c640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5481000c0 0x7fa54806a320 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa53800c970 tx=0x7fa53800ce30 comp rx=0 tx=0).stop 2024-02-17T11:55:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.276+0000 7fa54d32c640 1 -- 172.21.15.94:0/755884889 shutdown_connections 2024-02-17T11:55:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.276+0000 7fa54d32c640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa5340a20c0 0x7fa5340a4580 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.276+0000 7fa54d32c640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa548101c70 0x7fa54819e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.276+0000 7fa54d32c640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5481012b0 0x7fa54819bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.276+0000 7fa54d32c640 1 --2- 172.21.15.94:0/755884889 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5481000c0 0x7fa54806a320 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.276+0000 7fa54d32c640 1 -- 172.21.15.94:0/755884889 >> 172.21.15.94:0/755884889 conn(0x7fa5480fb630 msgr2=0x7fa5481045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.276+0000 7fa54d32c640 1 -- 172.21.15.94:0/755884889 shutdown_connections 2024-02-17T11:55:59.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.276+0000 7fa54d32c640 1 -- 172.21.15.94:0/755884889 wait complete. 2024-02-17T11:55:59.287 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-17T11:55:59.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.392+0000 7f8ae690f640 1 -- 172.21.15.94:0/2786255334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae0101640 msgr2=0x7f8ae0101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:59.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.392+0000 7f8ae690f640 1 --2- 172.21.15.94:0/2786255334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae0101640 0x7f8ae0101a60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009930 tx=0x7f8ad002bc30 comp rx=0 tx=0).stop 2024-02-17T11:55:59.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.392+0000 7f8ae690f640 1 -- 172.21.15.94:0/2786255334 shutdown_connections 2024-02-17T11:55:59.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.392+0000 7f8ae690f640 1 --2- 172.21.15.94:0/2786255334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ae0100d30 0x7f8ae01085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.392+0000 7f8ae690f640 1 --2- 172.21.15.94:0/2786255334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ae0100370 0x7f8ae01007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.392+0000 7f8ae690f640 1 --2- 172.21.15.94:0/2786255334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae0101640 0x7f8ae0101a60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.392+0000 7f8ae690f640 1 -- 172.21.15.94:0/2786255334 >> 172.21.15.94:0/2786255334 conn(0x7f8ae00fad80 msgr2=0x7f8ae00fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:59.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.392+0000 7f8ae690f640 1 -- 172.21.15.94:0/2786255334 shutdown_connections 2024-02-17T11:55:59.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.392+0000 7f8ae690f640 1 -- 172.21.15.94:0/2786255334 wait complete. 2024-02-17T11:55:59.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae690f640 1 Processor -- start 2024-02-17T11:55:59.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae690f640 1 -- start start 2024-02-17T11:55:59.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae690f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ae0100370 0x7f8ae01961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:59.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae690f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ae0100d30 0x7f8ae01966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:59.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae690f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae0101640 0x7f8ae019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:59.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae690f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae019dc90 con 0x7f8ae0101640 2024-02-17T11:55:59.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae690f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae019de00 con 0x7f8ae0100d30 2024-02-17T11:55:59.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae690f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae019dfc0 con 0x7f8ae0100370 2024-02-17T11:55:59.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae4eac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae0101640 0x7f8ae019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:59.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae4eac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae0101640 0x7f8ae019d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37610/0 (socket says 172.21.15.94:37610) 2024-02-17T11:55:59.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae4eac640 1 -- 172.21.15.94:0/590589715 learned_addr learned my addr 172.21.15.94:0/590589715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:59.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8adffff640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ae0100370 0x7f8ae01961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:59.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8adf7fe640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ae0100d30 0x7f8ae01966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:59.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae4eac640 1 -- 172.21.15.94:0/590589715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ae0100370 msgr2=0x7f8ae01961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:59.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae4eac640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ae0100370 0x7f8ae01961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae4eac640 1 -- 172.21.15.94:0/590589715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ae0100d30 msgr2=0x7f8ae01966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:59.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae4eac640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ae0100d30 0x7f8ae01966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae4eac640 1 -- 172.21.15.94:0/590589715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad0009560 con 0x7f8ae0101640 2024-02-17T11:55:59.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8adffff640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ae0100370 0x7f8ae01961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:55:59.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.393+0000 7f8ae4eac640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae0101640 0x7f8ae019d750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800eaa0 tx=0x7f8ad800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:59.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.394+0000 7f8add7fa640 1 -- 172.21.15.94:0/590589715 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad800cdb0 con 0x7f8ae0101640 2024-02-17T11:55:59.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.394+0000 7f8add7fa640 1 -- 172.21.15.94:0/590589715 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ad8005020 con 0x7f8ae0101640 2024-02-17T11:55:59.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.394+0000 7f8add7fa640 1 -- 172.21.15.94:0/590589715 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80116b0 con 0x7f8ae0101640 2024-02-17T11:55:59.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.394+0000 7f8ae690f640 1 -- 172.21.15.94:0/590589715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae019e2a0 con 0x7f8ae0101640 2024-02-17T11:55:59.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.394+0000 7f8ae690f640 1 -- 172.21.15.94:0/590589715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae019e7b0 con 0x7f8ae0101640 2024-02-17T11:55:59.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.395+0000 7f8ae690f640 1 -- 172.21.15.94:0/590589715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae01075a0 con 0x7f8ae0101640 2024-02-17T11:55:59.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.396+0000 7f8add7fa640 1 -- 172.21.15.94:0/590589715 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8ad8005520 con 0x7f8ae0101640 2024-02-17T11:55:59.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.396+0000 7f8add7fa640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ac80a20c0 0x7f8ac80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:59.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.396+0000 7f8adffff640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ac80a20c0 0x7f8ac80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:59.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.397+0000 7f8add7fa640 1 -- 172.21.15.94:0/590589715 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8ad80cde10 con 0x7f8ae0101640 2024-02-17T11:55:59.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.397+0000 7f8adffff640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ac80a20c0 0x7f8ac80a4580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009900 tx=0x7f8ad0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:59.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.401+0000 7f8add7fa640 1 -- 172.21.15.94:0/590589715 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ad8096f30 con 0x7f8ae0101640 2024-02-17T11:55:59.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.607+0000 7f8ae690f640 1 -- 172.21.15.94:0/590589715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8ae002ce40 con 0x7f8ae0101640 2024-02-17T11:55:59.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.668+0000 7f8add7fa640 1 -- 172.21.15.94:0/590589715 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8ad801f020 con 0x7f8ae0101640 2024-02-17T11:55:59.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.670+0000 7f8ae690f640 1 -- 172.21.15.94:0/590589715 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ac80a20c0 msgr2=0x7f8ac80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:59.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.670+0000 7f8ae690f640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ac80a20c0 0x7f8ac80a4580 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009900 tx=0x7f8ad0005fd0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.670+0000 7f8ae690f640 1 -- 172.21.15.94:0/590589715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae0101640 msgr2=0x7f8ae019d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:59.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.670+0000 7f8ae690f640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae0101640 0x7f8ae019d750 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8ad800eaa0 tx=0x7f8ad800ef60 comp rx=0 tx=0).stop 2024-02-17T11:55:59.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.670+0000 7f8ae690f640 1 -- 172.21.15.94:0/590589715 shutdown_connections 2024-02-17T11:55:59.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.670+0000 7f8ae690f640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ac80a20c0 0x7f8ac80a4580 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.670+0000 7f8ae690f640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ae0101640 0x7f8ae019d750 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.670+0000 7f8ae690f640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ae0100d30 0x7f8ae01966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.670+0000 7f8ae690f640 1 --2- 172.21.15.94:0/590589715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ae0100370 0x7f8ae01961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.670+0000 7f8ae690f640 1 -- 172.21.15.94:0/590589715 >> 172.21.15.94:0/590589715 conn(0x7f8ae00fad80 msgr2=0x7f8ae0104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:59.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.670+0000 7f8ae690f640 1 -- 172.21.15.94:0/590589715 shutdown_connections 2024-02-17T11:55:59.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.670+0000 7f8ae690f640 1 -- 172.21.15.94:0/590589715 wait complete. 2024-02-17T11:55:59.680 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-02-17T11:55:59.680 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-02-17T11:55:59.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.785+0000 7f13563d0640 1 -- 172.21.15.94:0/2219451623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1350102d40 msgr2=0x7f13501095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:59.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.785+0000 7f13563d0640 1 --2- 172.21.15.94:0/2219451623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1350102d40 0x7f13501095d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1348009ca0 tx=0x7f134802c0f0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.785+0000 7f13563d0640 1 -- 172.21.15.94:0/2219451623 shutdown_connections 2024-02-17T11:55:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.785+0000 7f13563d0640 1 --2- 172.21.15.94:0/2219451623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1350102d40 0x7f13501095d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.785+0000 7f13563d0640 1 --2- 172.21.15.94:0/2219451623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1350102380 0x7f1350102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.785+0000 7f13563d0640 1 --2- 172.21.15.94:0/2219451623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1350101190 0x7f13501015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.785+0000 7f13563d0640 1 -- 172.21.15.94:0/2219451623 >> 172.21.15.94:0/2219451623 conn(0x7f13500fc720 msgr2=0x7f13500feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:55:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.785+0000 7f13563d0640 1 -- 172.21.15.94:0/2219451623 shutdown_connections 2024-02-17T11:55:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.785+0000 7f13563d0640 1 -- 172.21.15.94:0/2219451623 wait complete. 2024-02-17T11:55:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f13563d0640 1 Processor -- start 2024-02-17T11:55:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f13563d0640 1 -- start start 2024-02-17T11:55:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f13563d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1350101190 0x7f1350197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f13563d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1350102380 0x7f13501976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f13563d0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1350102d40 0x7f135019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:59.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f13563d0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135019ec60 con 0x7f1350102380 2024-02-17T11:55:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f13563d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135019edd0 con 0x7f1350101190 2024-02-17T11:55:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f13563d0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135019ef90 con 0x7f1350102d40 2024-02-17T11:55:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1350102380 0x7f13501976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1350102380 0x7f13501976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37632/0 (socket says 172.21.15.94:37632) 2024-02-17T11:55:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f134f7fe640 1 -- 172.21.15.94:0/867512879 learned_addr learned my addr 172.21.15.94:0/867512879 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:55:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f134ffff640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1350101190 0x7f1350197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f135496d640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1350102d40 0x7f135019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f134f7fe640 1 -- 172.21.15.94:0/867512879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1350102d40 msgr2=0x7f135019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f134f7fe640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1350102d40 0x7f135019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f134f7fe640 1 -- 172.21.15.94:0/867512879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1350101190 msgr2=0x7f1350197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:55:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f134f7fe640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1350101190 0x7f1350197170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:55:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.786+0000 7f134f7fe640 1 -- 172.21.15.94:0/867512879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1348009900 con 0x7f1350102380 2024-02-17T11:55:59.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.787+0000 7f134f7fe640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1350102380 0x7f13501976b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f134400c920 tx=0x7f134400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.787+0000 7f134d7fa640 1 -- 172.21.15.94:0/867512879 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344007c70 con 0x7f1350102380 2024-02-17T11:55:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.787+0000 7f134d7fa640 1 -- 172.21.15.94:0/867512879 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1344007dd0 con 0x7f1350102380 2024-02-17T11:55:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.787+0000 7f13563d0640 1 -- 172.21.15.94:0/867512879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135019f270 con 0x7f1350102380 2024-02-17T11:55:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.787+0000 7f134d7fa640 1 -- 172.21.15.94:0/867512879 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344018860 con 0x7f1350102380 2024-02-17T11:55:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.787+0000 7f13563d0640 1 -- 172.21.15.94:0/867512879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135019f780 con 0x7f1350102380 2024-02-17T11:55:59.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.788+0000 7f13563d0640 1 -- 172.21.15.94:0/867512879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1350108450 con 0x7f1350102380 2024-02-17T11:55:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.789+0000 7f134d7fa640 1 -- 172.21.15.94:0/867512879 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f134400fac0 con 0x7f1350102380 2024-02-17T11:55:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.789+0000 7f134d7fa640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f13380a20c0 0x7f13380a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:55:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.790+0000 7f134ffff640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f13380a20c0 0x7f13380a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:55:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.790+0000 7f134d7fa640 1 -- 172.21.15.94:0/867512879 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f13440cdca0 con 0x7f1350102380 2024-02-17T11:55:59.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.790+0000 7f134ffff640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f13380a20c0 0x7f13380a4580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1340005fd0 tx=0x7f134000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:55:59.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:55:59.794+0000 7f134d7fa640 1 -- 172.21.15.94:0/867512879 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1344096dc0 con 0x7f1350102380 2024-02-17T11:55:59.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:59 smithi125 ceph-mon[36819]: pgmap v283: 97 pgs: 12 creating+peering, 2 active+clean, 83 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:55:59.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:59 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1723941468' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-17T11:55:59.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:59 smithi125 ceph-mon[36819]: osdmap e84: 12 total, 12 up, 12 in 2024-02-17T11:55:59.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:59 smithi125 ceph-mon[36819]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-17T11:55:59.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:59 smithi125 ceph-mon[36819]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-17T11:55:59.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:59 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/455076501' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:55:59.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:59 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/755884889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:55:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:59 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/590589715' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-17T11:55:59.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:55:59 smithi125 ceph-mon[36819]: osdmap e85: 12 total, 12 up, 12 in 2024-02-17T11:56:00.011 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:59 smithi094 ceph-mon[32499]: pgmap v283: 97 pgs: 12 creating+peering, 2 active+clean, 83 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:56:00.011 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:59 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1723941468' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-17T11:56:00.011 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:59 smithi094 ceph-mon[32499]: osdmap e84: 12 total, 12 up, 12 in 2024-02-17T11:56:00.011 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:59 smithi094 ceph-mon[32499]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-17T11:56:00.011 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:59 smithi094 ceph-mon[32499]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-17T11:56:00.011 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:59 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/455076501' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:56:00.012 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:59 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/755884889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:56:00.012 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:59 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/590589715' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-17T11:56:00.012 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:55:59 smithi094 ceph-mon[32499]: osdmap e85: 12 total, 12 up, 12 in 2024-02-17T11:56:00.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.007+0000 7f13563d0640 1 -- 172.21.15.94:0/867512879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f135002ce40 con 0x7f1350102380 2024-02-17T11:56:00.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.130+0000 7f134d7fa640 1 -- 172.21.15.94:0/867512879 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) v1 ==== 105+0+0 (secure 0 0 0) 0x7f134409a480 con 0x7f1350102380 2024-02-17T11:56:00.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.132+0000 7f13563d0640 1 -- 172.21.15.94:0/867512879 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f13380a20c0 msgr2=0x7f13380a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:00.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.132+0000 7f13563d0640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f13380a20c0 0x7f13380a4580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1340005fd0 tx=0x7f134000b390 comp rx=0 tx=0).stop 2024-02-17T11:56:00.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.132+0000 7f13563d0640 1 -- 172.21.15.94:0/867512879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1350102380 msgr2=0x7f13501976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:00.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.132+0000 7f13563d0640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1350102380 0x7f13501976b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f134400c920 tx=0x7f134400cde0 comp rx=0 tx=0).stop 2024-02-17T11:56:00.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.132+0000 7f13563d0640 1 -- 172.21.15.94:0/867512879 shutdown_connections 2024-02-17T11:56:00.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.132+0000 7f13563d0640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f13380a20c0 0x7f13380a4580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:00.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.132+0000 7f13563d0640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1350102d40 0x7f135019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:00.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.132+0000 7f13563d0640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1350102380 0x7f13501976b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:00.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.132+0000 7f13563d0640 1 --2- 172.21.15.94:0/867512879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1350101190 0x7f1350197170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:00.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.132+0000 7f13563d0640 1 -- 172.21.15.94:0/867512879 >> 172.21.15.94:0/867512879 conn(0x7f13500fc720 msgr2=0x7f1350105620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:00.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.132+0000 7f13563d0640 1 -- 172.21.15.94:0/867512879 shutdown_connections 2024-02-17T11:56:00.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.132+0000 7f13563d0640 1 -- 172.21.15.94:0/867512879 wait complete. 2024-02-17T11:56:00.147 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-02-17T11:56:00.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:59 smithi189 ceph-mon[36588]: pgmap v283: 97 pgs: 12 creating+peering, 2 active+clean, 83 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:56:00.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:59 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1723941468' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-17T11:56:00.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:59 smithi189 ceph-mon[36588]: osdmap e84: 12 total, 12 up, 12 in 2024-02-17T11:56:00.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:59 smithi189 ceph-mon[36588]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-17T11:56:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:59 smithi189 ceph-mon[36588]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-17T11:56:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:59 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/455076501' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:56:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:59 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/755884889' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:56:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:59 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/590589715' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-17T11:56:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:55:59 smithi189 ceph-mon[36588]: osdmap e85: 12 total, 12 up, 12 in 2024-02-17T11:56:00.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.253+0000 7fa2c69a0640 1 -- 172.21.15.94:0/4235300259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c0100250 msgr2=0x7fa2c01006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:00.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.253+0000 7fa2c69a0640 1 --2- 172.21.15.94:0/4235300259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c0100250 0x7fa2c01006d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc009930 tx=0x7fa2bc02bc30 comp rx=0 tx=0).stop 2024-02-17T11:56:00.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.253+0000 7fa2c69a0640 1 -- 172.21.15.94:0/4235300259 shutdown_connections 2024-02-17T11:56:00.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.253+0000 7fa2c69a0640 1 --2- 172.21.15.94:0/4235300259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2c0100c10 0x7fa2c01084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:00.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.253+0000 7fa2c69a0640 1 --2- 172.21.15.94:0/4235300259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c0100250 0x7fa2c01006d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:00.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.253+0000 7fa2c69a0640 1 --2- 172.21.15.94:0/4235300259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c0101520 0x7fa2c0101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:00.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.253+0000 7fa2c69a0640 1 -- 172.21.15.94:0/4235300259 >> 172.21.15.94:0/4235300259 conn(0x7fa2c00fac80 msgr2=0x7fa2c00fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:00.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.253+0000 7fa2c69a0640 1 -- 172.21.15.94:0/4235300259 shutdown_connections 2024-02-17T11:56:00.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.253+0000 7fa2c69a0640 1 -- 172.21.15.94:0/4235300259 wait complete. 2024-02-17T11:56:00.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.253+0000 7fa2c69a0640 1 Processor -- start 2024-02-17T11:56:00.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.253+0000 7fa2c69a0640 1 -- start start 2024-02-17T11:56:00.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.253+0000 7fa2c69a0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2c0100250 0x7fa2c00ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:00.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c69a0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c0100c10 0x7fa2c019c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c69a0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c0101520 0x7fa2c019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c69a0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c019ee30 con 0x7fa2c0100c10 2024-02-17T11:56:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c69a0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c019efa0 con 0x7fa2c0101520 2024-02-17T11:56:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c69a0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c019f160 con 0x7fa2c0100250 2024-02-17T11:56:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c519d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c0100c10 0x7fa2c019c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c519d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c0100c10 0x7fa2c019c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37660/0 (socket says 172.21.15.94:37660) 2024-02-17T11:56:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c519d640 1 -- 172.21.15.94:0/2588293200 learned_addr learned my addr 172.21.15.94:0/2588293200 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c599e640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2c0100250 0x7fa2c00ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c519d640 1 -- 172.21.15.94:0/2588293200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2c0100250 msgr2=0x7fa2c00ffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c619f640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c0101520 0x7fa2c019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:00.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c519d640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2c0100250 0x7fa2c00ffcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c519d640 1 -- 172.21.15.94:0/2588293200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c0101520 msgr2=0x7fa2c019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c519d640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c0101520 0x7fa2c019e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c519d640 1 -- 172.21.15.94:0/2588293200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2bc009560 con 0x7fa2c0100c10 2024-02-17T11:56:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c619f640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c0101520 0x7fa2c019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c519d640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c0100c10 0x7fa2c019c030 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc02c840 tx=0x7fa2bc0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2b6ffd640 1 -- 172.21.15.94:0/2588293200 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc0044d0 con 0x7fa2c0100c10 2024-02-17T11:56:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c69a0640 1 -- 172.21.15.94:0/2588293200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c019f380 con 0x7fa2c0100c10 2024-02-17T11:56:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2b6ffd640 1 -- 172.21.15.94:0/2588293200 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2bc0054e0 con 0x7fa2c0100c10 2024-02-17T11:56:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2b6ffd640 1 -- 172.21.15.94:0/2588293200 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2bc03e6b0 con 0x7fa2c0100c10 2024-02-17T11:56:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.254+0000 7fa2c69a0640 1 -- 172.21.15.94:0/2588293200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c019f930 con 0x7fa2c0100c10 2024-02-17T11:56:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.255+0000 7fa2c69a0640 1 -- 172.21.15.94:0/2588293200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c0107460 con 0x7fa2c0100c10 2024-02-17T11:56:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.257+0000 7fa2b6ffd640 1 -- 172.21.15.94:0/2588293200 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fa2bc0037d0 con 0x7fa2c0100c10 2024-02-17T11:56:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.257+0000 7fa2b6ffd640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa2ac0a20c0 0x7fa2ac0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.258+0000 7fa2c599e640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa2ac0a20c0 0x7fa2ac0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.258+0000 7fa2b6ffd640 1 -- 172.21.15.94:0/2588293200 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa2bc0f0660 con 0x7fa2c0100c10 2024-02-17T11:56:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.258+0000 7fa2c599e640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa2ac0a20c0 0x7fa2ac0a4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0069470 tx=0x7fa2b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.261+0000 7fa2b6ffd640 1 -- 172.21.15.94:0/2588293200 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa2bc0b9780 con 0x7fa2c0100c10 2024-02-17T11:56:00.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:00.473+0000 7fa2c69a0640 1 -- 172.21.15.94:0/2588293200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fa2c010baa0 con 0x7fa2c0100c10 2024-02-17T11:56:00.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:00 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/590589715' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-17T11:56:00.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:00 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] up:active 2024-02-17T11:56:00.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:00 smithi125 ceph-mon[36819]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-17T11:56:00.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:00 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/867512879' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-17T11:56:00.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:00 smithi125 ceph-mon[36819]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-17T11:56:00.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:00 smithi125 ceph-mon[36819]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-17T11:56:00.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:00 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/867512879' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-17T11:56:00.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:00 smithi125 ceph-mon[36819]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-17T11:56:00.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:00 smithi125 ceph-mon[36819]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-17T11:56:00.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:00 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2588293200' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-17T11:56:00.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:00 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/590589715' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-17T11:56:00.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:00 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] up:active 2024-02-17T11:56:00.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:00 smithi094 ceph-mon[32499]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-17T11:56:00.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:00 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/867512879' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-17T11:56:00.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:00 smithi094 ceph-mon[32499]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-17T11:56:00.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:00 smithi094 ceph-mon[32499]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-17T11:56:00.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:00 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/867512879' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-17T11:56:00.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:00 smithi094 ceph-mon[32499]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-17T11:56:00.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:00 smithi094 ceph-mon[32499]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-17T11:56:00.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:00 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2588293200' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-17T11:56:01.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.140+0000 7fa2b6ffd640 1 -- 172.21.15.94:0/2588293200 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v17) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa2bc0bce40 con 0x7fa2c0100c10 2024-02-17T11:56:01.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.142+0000 7fa2c69a0640 1 -- 172.21.15.94:0/2588293200 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa2ac0a20c0 msgr2=0x7fa2ac0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:01.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.142+0000 7fa2c69a0640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa2ac0a20c0 0x7fa2ac0a4580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0069470 tx=0x7fa2b0008040 comp rx=0 tx=0).stop 2024-02-17T11:56:01.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.142+0000 7fa2c69a0640 1 -- 172.21.15.94:0/2588293200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c0100c10 msgr2=0x7fa2c019c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:01.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.142+0000 7fa2c69a0640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c0100c10 0x7fa2c019c030 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc02c840 tx=0x7fa2bc0042e0 comp rx=0 tx=0).stop 2024-02-17T11:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.142+0000 7fa2c69a0640 1 -- 172.21.15.94:0/2588293200 shutdown_connections 2024-02-17T11:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.142+0000 7fa2c69a0640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa2ac0a20c0 0x7fa2ac0a4580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.142+0000 7fa2c69a0640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa2c0101520 0x7fa2c019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.142+0000 7fa2c69a0640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa2c0100c10 0x7fa2c019c030 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.142+0000 7fa2c69a0640 1 --2- 172.21.15.94:0/2588293200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2c0100250 0x7fa2c00ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.142+0000 7fa2c69a0640 1 -- 172.21.15.94:0/2588293200 >> 172.21.15.94:0/2588293200 conn(0x7fa2c00fac80 msgr2=0x7fa2c0104630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.142+0000 7fa2c69a0640 1 -- 172.21.15.94:0/2588293200 shutdown_connections 2024-02-17T11:56:01.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.143+0000 7fa2c69a0640 1 -- 172.21.15.94:0/2588293200 wait complete. 2024-02-17T11:56:01.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:00 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/590589715' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-17T11:56:01.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:00 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] up:active 2024-02-17T11:56:01.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:00 smithi189 ceph-mon[36588]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-17T11:56:01.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:00 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/867512879' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-02-17T11:56:01.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:00 smithi189 ceph-mon[36588]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-17T11:56:01.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:00 smithi189 ceph-mon[36588]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-17T11:56:01.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:00 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/867512879' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-02-17T11:56:01.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:00 smithi189 ceph-mon[36588]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-17T11:56:01.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:00 smithi189 ceph-mon[36588]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-17T11:56:01.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:00 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2588293200' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-17T11:56:01.156 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-02-17T11:56:01.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.269+0000 7f1a31e23640 1 -- 172.21.15.94:0/1742961783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c101c30 msgr2=0x7f1a2c1084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:01.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.269+0000 7f1a31e23640 1 --2- 172.21.15.94:0/1742961783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c101c30 0x7f1a2c1084c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009900 tx=0x7f1a2402bc30 comp rx=0 tx=0).stop 2024-02-17T11:56:01.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.269+0000 7f1a31e23640 1 -- 172.21.15.94:0/1742961783 shutdown_connections 2024-02-17T11:56:01.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.269+0000 7f1a31e23640 1 --2- 172.21.15.94:0/1742961783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c101c30 0x7f1a2c1084c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:01.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.269+0000 7f1a31e23640 1 --2- 172.21.15.94:0/1742961783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a2c101270 0x7f1a2c1016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:01.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.269+0000 7f1a31e23640 1 --2- 172.21.15.94:0/1742961783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a2c100080 0x7f1a2c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:01.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.269+0000 7f1a31e23640 1 -- 172.21.15.94:0/1742961783 >> 172.21.15.94:0/1742961783 conn(0x7f1a2c0fb610 msgr2=0x7f1a2c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:01.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.269+0000 7f1a31e23640 1 -- 172.21.15.94:0/1742961783 shutdown_connections 2024-02-17T11:56:01.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.269+0000 7f1a31e23640 1 -- 172.21.15.94:0/1742961783 wait complete. 2024-02-17T11:56:01.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.270+0000 7f1a31e23640 1 Processor -- start 2024-02-17T11:56:01.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.270+0000 7f1a31e23640 1 -- start start 2024-02-17T11:56:01.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.270+0000 7f1a31e23640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a2c100080 0x7f1a2c197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:01.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.270+0000 7f1a31e23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c101270 0x7f1a2c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:01.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.270+0000 7f1a31e23640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a2c101c30 0x7f1a2c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:01.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.270+0000 7f1a31e23640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c19ecf0 con 0x7f1a2c101270 2024-02-17T11:56:01.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.270+0000 7f1a31e23640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c19ee60 con 0x7f1a2c101c30 2024-02-17T11:56:01.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.270+0000 7f1a31e23640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a2c19f020 con 0x7f1a2c100080 2024-02-17T11:56:01.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.270+0000 7f1a2affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c101270 0x7f1a2c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:01.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.270+0000 7f1a2affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c101270 0x7f1a2c197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37686/0 (socket says 172.21.15.94:37686) 2024-02-17T11:56:01.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.270+0000 7f1a2affd640 1 -- 172.21.15.94:0/227920916 learned_addr learned my addr 172.21.15.94:0/227920916 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:01.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.270+0000 7f1a2b7fe640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a2c100080 0x7f1a2c197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:01.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.270+0000 7f1a2bfff640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a2c101c30 0x7f1a2c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:01.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.271+0000 7f1a2affd640 1 -- 172.21.15.94:0/227920916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a2c100080 msgr2=0x7f1a2c197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:01.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.271+0000 7f1a2affd640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a2c100080 0x7f1a2c197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:01.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.271+0000 7f1a2affd640 1 -- 172.21.15.94:0/227920916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a2c101c30 msgr2=0x7f1a2c19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:01.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.271+0000 7f1a2affd640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a2c101c30 0x7f1a2c19e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:01.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.271+0000 7f1a2affd640 1 -- 172.21.15.94:0/227920916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a24009560 con 0x7f1a2c101270 2024-02-17T11:56:01.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.271+0000 7f1a2affd640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c101270 0x7f1a2c197740 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400c890 tx=0x7f1a1400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:01.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.271+0000 7f1a28ff9640 1 -- 172.21.15.94:0/227920916 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14004220 con 0x7f1a2c101270 2024-02-17T11:56:01.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.271+0000 7f1a28ff9640 1 -- 172.21.15.94:0/227920916 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1a14007e90 con 0x7f1a2c101270 2024-02-17T11:56:01.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.271+0000 7f1a28ff9640 1 -- 172.21.15.94:0/227920916 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a14004a00 con 0x7f1a2c101270 2024-02-17T11:56:01.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.271+0000 7f1a31e23640 1 -- 172.21.15.94:0/227920916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2c19f300 con 0x7f1a2c101270 2024-02-17T11:56:01.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.271+0000 7f1a31e23640 1 -- 172.21.15.94:0/227920916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2c19f860 con 0x7f1a2c101270 2024-02-17T11:56:01.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.272+0000 7f1a31e23640 1 -- 172.21.15.94:0/227920916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a2c1073a0 con 0x7f1a2c101270 2024-02-17T11:56:01.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.273+0000 7f1a28ff9640 1 -- 172.21.15.94:0/227920916 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1a140079b0 con 0x7f1a2c101270 2024-02-17T11:56:01.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.273+0000 7f1a28ff9640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1a180a2140 0x7f1a180a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:01.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.273+0000 7f1a2b7fe640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1a180a2140 0x7f1a180a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:01.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.274+0000 7f1a28ff9640 1 -- 172.21.15.94:0/227920916 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1a140cd7a0 con 0x7f1a2c101270 2024-02-17T11:56:01.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.274+0000 7f1a2b7fe640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1a180a2140 0x7f1a180a4600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c1010d0 tx=0x7f1a1c00a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:01.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.278+0000 7f1a28ff9640 1 -- 172.21.15.94:0/227920916 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a140968c0 con 0x7f1a2c101270 2024-02-17T11:56:01.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:01.491+0000 7f1a31e23640 1 -- 172.21.15.94:0/227920916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f1a2c02ce40 con 0x7f1a2c101270 2024-02-17T11:56:01.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:01 smithi125 ceph-mon[36819]: pgmap v286: 97 pgs: 12 creating+peering, 45 active+clean, 40 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s wr, 8 op/s 2024-02-17T11:56:01.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:01 smithi125 ceph-mon[36819]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-17T11:56:01.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:01 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2588293200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-17T11:56:01.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:01 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] up:active 2024-02-17T11:56:01.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:01 smithi125 ceph-mon[36819]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-17T11:56:01.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:01 smithi125 ceph-mon[36819]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-17T11:56:01.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:01 smithi125 ceph-mon[36819]: Cluster is now healthy 2024-02-17T11:56:01.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:01 smithi125 ceph-mon[36819]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-17T11:56:01.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:01 smithi125 ceph-mon[36819]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-02-17T11:56:01.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:01 smithi125 ceph-mon[36819]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-17T11:56:01.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:01 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/227920916' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-17T11:56:01.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:01 smithi094 ceph-mon[32499]: pgmap v286: 97 pgs: 12 creating+peering, 45 active+clean, 40 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s wr, 8 op/s 2024-02-17T11:56:01.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:01 smithi094 ceph-mon[32499]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-17T11:56:01.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:01 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2588293200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-17T11:56:01.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:01 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] up:active 2024-02-17T11:56:01.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:01 smithi094 ceph-mon[32499]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-17T11:56:01.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:01 smithi094 ceph-mon[32499]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-17T11:56:01.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:01 smithi094 ceph-mon[32499]: Cluster is now healthy 2024-02-17T11:56:01.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:01 smithi094 ceph-mon[32499]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-17T11:56:01.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:01 smithi094 ceph-mon[32499]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-02-17T11:56:01.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:01 smithi094 ceph-mon[32499]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-17T11:56:01.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:01 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/227920916' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-17T11:56:02.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.146+0000 7f1a28ff9640 1 -- 172.21.15.94:0/227920916 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v19) v1 ==== 115+0+0 (secure 0 0 0) 0x7f1a14099f80 con 0x7f1a2c101270 2024-02-17T11:56:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.149+0000 7f1a31e23640 1 -- 172.21.15.94:0/227920916 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1a180a2140 msgr2=0x7f1a180a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.149+0000 7f1a31e23640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1a180a2140 0x7f1a180a4600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c1010d0 tx=0x7f1a1c00a590 comp rx=0 tx=0).stop 2024-02-17T11:56:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.149+0000 7f1a31e23640 1 -- 172.21.15.94:0/227920916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c101270 msgr2=0x7f1a2c197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.149+0000 7f1a31e23640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c101270 0x7f1a2c197740 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400c890 tx=0x7f1a1400cd50 comp rx=0 tx=0).stop 2024-02-17T11:56:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.149+0000 7f1a31e23640 1 -- 172.21.15.94:0/227920916 shutdown_connections 2024-02-17T11:56:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.149+0000 7f1a31e23640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1a180a2140 0x7f1a180a4600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.149+0000 7f1a31e23640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1a2c101c30 0x7f1a2c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.149+0000 7f1a31e23640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1a2c101270 0x7f1a2c197740 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.149+0000 7f1a31e23640 1 --2- 172.21.15.94:0/227920916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1a2c100080 0x7f1a2c197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.149+0000 7f1a31e23640 1 -- 172.21.15.94:0/227920916 >> 172.21.15.94:0/227920916 conn(0x7f1a2c0fb610 msgr2=0x7f1a2c104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:02.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.149+0000 7f1a31e23640 1 -- 172.21.15.94:0/227920916 shutdown_connections 2024-02-17T11:56:02.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.149+0000 7f1a31e23640 1 -- 172.21.15.94:0/227920916 wait complete. 2024-02-17T11:56:02.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:01 smithi189 ceph-mon[36588]: pgmap v286: 97 pgs: 12 creating+peering, 45 active+clean, 40 unknown; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s wr, 8 op/s 2024-02-17T11:56:02.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:01 smithi189 ceph-mon[36588]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-02-17T11:56:02.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:01 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2588293200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-17T11:56:02.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:01 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] up:active 2024-02-17T11:56:02.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:01 smithi189 ceph-mon[36588]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-17T11:56:02.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:01 smithi189 ceph-mon[36588]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-17T11:56:02.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:01 smithi189 ceph-mon[36588]: Cluster is now healthy 2024-02-17T11:56:02.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:01 smithi189 ceph-mon[36588]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-17T11:56:02.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:01 smithi189 ceph-mon[36588]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-02-17T11:56:02.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:01 smithi189 ceph-mon[36588]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-02-17T11:56:02.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:01 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/227920916' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-17T11:56:02.163 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-02-17T11:56:02.163 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-02-17T11:56:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.273+0000 7fb4df839640 1 -- 172.21.15.94:0/3390485424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4d81023a0 msgr2=0x7fb4d8102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.273+0000 7fb4df839640 1 --2- 172.21.15.94:0/3390485424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4d81023a0 0x7fb4d8102820 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb4c8009900 tx=0x7fb4c802bc30 comp rx=0 tx=0).stop 2024-02-17T11:56:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.273+0000 7fb4df839640 1 -- 172.21.15.94:0/3390485424 shutdown_connections 2024-02-17T11:56:02.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.273+0000 7fb4df839640 1 --2- 172.21.15.94:0/3390485424 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb4d8102d60 0x7fb4d81095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.273+0000 7fb4df839640 1 --2- 172.21.15.94:0/3390485424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4d81023a0 0x7fb4d8102820 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.273+0000 7fb4df839640 1 --2- 172.21.15.94:0/3390485424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4d81011b0 0x7fb4d81015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.273+0000 7fb4df839640 1 -- 172.21.15.94:0/3390485424 >> 172.21.15.94:0/3390485424 conn(0x7fb4d80fc720 msgr2=0x7fb4d80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.274+0000 7fb4df839640 1 -- 172.21.15.94:0/3390485424 shutdown_connections 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.274+0000 7fb4df839640 1 -- 172.21.15.94:0/3390485424 wait complete. 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.274+0000 7fb4df839640 1 Processor -- start 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.274+0000 7fb4df839640 1 -- start start 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.274+0000 7fb4df839640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4d81011b0 0x7fb4d8197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.274+0000 7fb4df839640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb4d81023a0 0x7fb4d8197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.274+0000 7fb4df839640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4d8102d60 0x7fb4d819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.274+0000 7fb4df839640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d819ed00 con 0x7fb4d81011b0 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.274+0000 7fb4df839640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d819ee70 con 0x7fb4d81023a0 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.274+0000 7fb4df839640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d819f030 con 0x7fb4d8102d60 2024-02-17T11:56:02.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.274+0000 7fb4dd5d5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4d81011b0 0x7fb4d8197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:02.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.274+0000 7fb4dd5d5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4d81011b0 0x7fb4d8197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37714/0 (socket says 172.21.15.94:37714) 2024-02-17T11:56:02.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.274+0000 7fb4dd5d5640 1 -- 172.21.15.94:0/2002030019 learned_addr learned my addr 172.21.15.94:0/2002030019 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:02.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.275+0000 7fb4dd5d5640 1 -- 172.21.15.94:0/2002030019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4d8102d60 msgr2=0x7fb4d819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.275+0000 7fb4dd5d5640 1 --2- 172.21.15.94:0/2002030019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4d8102d60 0x7fb4d819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.275+0000 7fb4dd5d5640 1 -- 172.21.15.94:0/2002030019 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb4d81023a0 msgr2=0x7fb4d8197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.275+0000 7fb4dd5d5640 1 --2- 172.21.15.94:0/2002030019 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb4d81023a0 0x7fb4d8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.275+0000 7fb4dd5d5640 1 -- 172.21.15.94:0/2002030019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c8009560 con 0x7fb4d81011b0 2024-02-17T11:56:02.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.275+0000 7fb4dd5d5640 1 --2- 172.21.15.94:0/2002030019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4d81011b0 0x7fb4d8197210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c970 tx=0x7fb4d400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:02.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.275+0000 7fb4ce7fc640 1 -- 172.21.15.94:0/2002030019 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4007c10 con 0x7fb4d81011b0 2024-02-17T11:56:02.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.275+0000 7fb4ce7fc640 1 -- 172.21.15.94:0/2002030019 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4d4007d70 con 0x7fb4d81011b0 2024-02-17T11:56:02.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.275+0000 7fb4df839640 1 -- 172.21.15.94:0/2002030019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d819f310 con 0x7fb4d81011b0 2024-02-17T11:56:02.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.275+0000 7fb4ce7fc640 1 -- 172.21.15.94:0/2002030019 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400f8a0 con 0x7fb4d81011b0 2024-02-17T11:56:02.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.275+0000 7fb4df839640 1 -- 172.21.15.94:0/2002030019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d819f8f0 con 0x7fb4d81011b0 2024-02-17T11:56:02.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.276+0000 7fb4df839640 1 -- 172.21.15.94:0/2002030019 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d81084d0 con 0x7fb4d81011b0 2024-02-17T11:56:02.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.277+0000 7fb4ce7fc640 1 -- 172.21.15.94:0/2002030019 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb4d4018af0 con 0x7fb4d81011b0 2024-02-17T11:56:02.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.278+0000 7fb4ce7fc640 1 --2- 172.21.15.94:0/2002030019 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb4c40a2140 0x7fb4c40a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:02.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.278+0000 7fb4dcdd4640 1 --2- 172.21.15.94:0/2002030019 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb4c40a2140 0x7fb4c40a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:02.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.278+0000 7fb4ce7fc640 1 -- 172.21.15.94:0/2002030019 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb4d4013070 con 0x7fb4d81011b0 2024-02-17T11:56:02.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.278+0000 7fb4dcdd4640 1 --2- 172.21.15.94:0/2002030019 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb4c40a2140 0x7fb4c40a4600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4c802c6a0 tx=0x7fb4c8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:02.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.282+0000 7fb4ce7fc640 1 -- 172.21.15.94:0/2002030019 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb4d4097050 con 0x7fb4d81011b0 2024-02-17T11:56:02.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.480+0000 7fb4df839640 1 -- 172.21.15.94:0/2002030019 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4d802cfa0 con 0x7fb4c40a2140 2024-02-17T11:56:02.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.506+0000 7fb4ce7fc640 1 -- 172.21.15.94:0/2002030019 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb4d802cfa0 con 0x7fb4c40a2140 2024-02-17T11:56:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.508+0000 7fb4df839640 1 -- 172.21.15.94:0/2002030019 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb4c40a2140 msgr2=0x7fb4c40a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.508+0000 7fb4df839640 1 --2- 172.21.15.94:0/2002030019 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb4c40a2140 0x7fb4c40a4600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb4c802c6a0 tx=0x7fb4c8000bc0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.508+0000 7fb4df839640 1 -- 172.21.15.94:0/2002030019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4d81011b0 msgr2=0x7fb4d8197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.508+0000 7fb4df839640 1 --2- 172.21.15.94:0/2002030019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4d81011b0 0x7fb4d8197210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c970 tx=0x7fb4d400ce30 comp rx=0 tx=0).stop 2024-02-17T11:56:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.508+0000 7fb4df839640 1 -- 172.21.15.94:0/2002030019 shutdown_connections 2024-02-17T11:56:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.508+0000 7fb4df839640 1 --2- 172.21.15.94:0/2002030019 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb4c40a2140 0x7fb4c40a4600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.508+0000 7fb4df839640 1 --2- 172.21.15.94:0/2002030019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4d8102d60 0x7fb4d819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.508+0000 7fb4df839640 1 --2- 172.21.15.94:0/2002030019 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb4d81023a0 0x7fb4d8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.508+0000 7fb4df839640 1 --2- 172.21.15.94:0/2002030019 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb4d81011b0 0x7fb4d8197210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.508+0000 7fb4df839640 1 -- 172.21.15.94:0/2002030019 >> 172.21.15.94:0/2002030019 conn(0x7fb4d80fc720 msgr2=0x7fb4d81056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.508+0000 7fb4df839640 1 -- 172.21.15.94:0/2002030019 shutdown_connections 2024-02-17T11:56:02.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.508+0000 7fb4df839640 1 -- 172.21.15.94:0/2002030019 wait complete. 2024-02-17T11:56:02.518 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-02-17T11:56:02.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.626+0000 7f4265577640 1 -- 172.21.15.94:0/2617274329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4260101c30 msgr2=0x7f42601084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.626+0000 7f4265577640 1 --2- 172.21.15.94:0/2617274329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4260101c30 0x7f42601084c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4258009ca0 tx=0x7f425802c0f0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.626+0000 7f4265577640 1 -- 172.21.15.94:0/2617274329 shutdown_connections 2024-02-17T11:56:02.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.626+0000 7f4265577640 1 --2- 172.21.15.94:0/2617274329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4260101c30 0x7f42601084c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.626+0000 7f4265577640 1 --2- 172.21.15.94:0/2617274329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4260101270 0x7f42601016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.626+0000 7f4265577640 1 --2- 172.21.15.94:0/2617274329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4260100080 0x7f42601004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.626+0000 7f4265577640 1 -- 172.21.15.94:0/2617274329 >> 172.21.15.94:0/2617274329 conn(0x7f42600fb610 msgr2=0x7f42600fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.626+0000 7f4265577640 1 -- 172.21.15.94:0/2617274329 shutdown_connections 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.626+0000 7f4265577640 1 -- 172.21.15.94:0/2617274329 wait complete. 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.626+0000 7f4265577640 1 Processor -- start 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f4265577640 1 -- start start 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f4265577640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4260100080 0x7f426010a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f4265577640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4260101270 0x7f4260108f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f4265577640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4260101c30 0x7f42601094c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f4265577640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4260109c20 con 0x7f4260101270 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f4265577640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4260109d90 con 0x7f4260101c30 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f4265577640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42601a12c0 con 0x7f4260100080 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f425e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4260101270 0x7f4260108f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f425e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4260101270 0x7f4260108f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37752/0 (socket says 172.21.15.94:37752) 2024-02-17T11:56:02.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f425e7fc640 1 -- 172.21.15.94:0/4189051861 learned_addr learned my addr 172.21.15.94:0/4189051861 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:02.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f425effd640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4260100080 0x7f426010a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:02.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f425f7fe640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4260101c30 0x7f42601094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:02.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f425e7fc640 1 -- 172.21.15.94:0/4189051861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4260100080 msgr2=0x7f426010a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f425e7fc640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4260100080 0x7f426010a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f425e7fc640 1 -- 172.21.15.94:0/4189051861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4260101c30 msgr2=0x7f42601094c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f425e7fc640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4260101c30 0x7f42601094c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f425e7fc640 1 -- 172.21.15.94:0/4189051861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4258009900 con 0x7f4260101270 2024-02-17T11:56:02.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f425effd640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4260100080 0x7f426010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:02.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f425f7fe640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4260101c30 0x7f42601094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:02.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f425e7fc640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4260101270 0x7f4260108f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f425400d980 tx=0x7f425400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:02.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f4247fff640 1 -- 172.21.15.94:0/4189051861 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254009c30 con 0x7f4260101270 2024-02-17T11:56:02.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f4247fff640 1 -- 172.21.15.94:0/4189051861 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f425400b660 con 0x7f4260101270 2024-02-17T11:56:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f4265577640 1 -- 172.21.15.94:0/4189051861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42601a1570 con 0x7f4260101270 2024-02-17T11:56:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f4247fff640 1 -- 172.21.15.94:0/4189051861 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425400f6b0 con 0x7f4260101270 2024-02-17T11:56:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.627+0000 7f4265577640 1 -- 172.21.15.94:0/4189051861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42601a1a80 con 0x7f4260101270 2024-02-17T11:56:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.628+0000 7f4265577640 1 -- 172.21.15.94:0/4189051861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42601073a0 con 0x7f4260101270 2024-02-17T11:56:02.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.630+0000 7f4247fff640 1 -- 172.21.15.94:0/4189051861 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4254010460 con 0x7f4260101270 2024-02-17T11:56:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.630+0000 7f4247fff640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f42480a20c0 0x7f42480a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.630+0000 7f425effd640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f42480a20c0 0x7f42480a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.630+0000 7f4247fff640 1 -- 172.21.15.94:0/4189051861 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f42540cd890 con 0x7f4260101270 2024-02-17T11:56:02.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.631+0000 7f425effd640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f42480a20c0 0x7f42480a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4250006fd0 tx=0x7f4250008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:02.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.634+0000 7f4247fff640 1 -- 172.21.15.94:0/4189051861 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42540969b0 con 0x7f4260101270 2024-02-17T11:56:02.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.835+0000 7f4265577640 1 -- 172.21.15.94:0/4189051861 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f426002ceb0 con 0x7f42480a20c0 2024-02-17T11:56:02.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.859+0000 7f4247fff640 1 -- 172.21.15.94:0/4189051861 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f426002ceb0 con 0x7f42480a20c0 2024-02-17T11:56:02.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.861+0000 7f4265577640 1 -- 172.21.15.94:0/4189051861 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f42480a20c0 msgr2=0x7f42480a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.861+0000 7f4265577640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f42480a20c0 0x7f42480a4580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4250006fd0 tx=0x7f4250008040 comp rx=0 tx=0).stop 2024-02-17T11:56:02.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.861+0000 7f4265577640 1 -- 172.21.15.94:0/4189051861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4260101270 msgr2=0x7f4260108f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.861+0000 7f4265577640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4260101270 0x7f4260108f80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f425400d980 tx=0x7f425400de40 comp rx=0 tx=0).stop 2024-02-17T11:56:02.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.861+0000 7f4265577640 1 -- 172.21.15.94:0/4189051861 shutdown_connections 2024-02-17T11:56:02.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.861+0000 7f4265577640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f42480a20c0 0x7f42480a4580 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.861+0000 7f4265577640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4260101c30 0x7f42601094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.861+0000 7f4265577640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4260101270 0x7f4260108f80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.861+0000 7f4265577640 1 --2- 172.21.15.94:0/4189051861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4260100080 0x7f426010a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.861+0000 7f4265577640 1 -- 172.21.15.94:0/4189051861 >> 172.21.15.94:0/4189051861 conn(0x7f42600fb610 msgr2=0x7f4260104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:02.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.861+0000 7f4265577640 1 -- 172.21.15.94:0/4189051861 shutdown_connections 2024-02-17T11:56:02.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.861+0000 7f4265577640 1 -- 172.21.15.94:0/4189051861 wait complete. 2024-02-17T11:56:02.870 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:56:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:56:02.485+0000 7f1b837e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:56:02.870 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:56:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:56:02.485+0000 7f1b837e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:56:02.870 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:56:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:56:02.485+0000 7f1b837e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:56:02.870 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:56:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:56:02.485+0000 7f1b837e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:56:02.870 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:56:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:56:02.485+0000 7f1b837e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:56:02.871 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-02-17T11:56:02.934 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 11:56:02 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b[67430]: 2024-02-17T11:56:02.660+0000 7f6367f53700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-17T11:56:02.934 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 11:56:02 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b[67430]: 2024-02-17T11:56:02.660+0000 7f6367f53700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-02-17T11:56:02.934 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 11:56:02 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b[67430]: 2024-02-17T11:56:02.660+0000 7f6367f53700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-02-17T11:56:02.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.978+0000 7f3c5304e640 1 -- 172.21.15.94:0/2953845133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c102d40 msgr2=0x7f3c4c1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.978+0000 7f3c5304e640 1 --2- 172.21.15.94:0/2953845133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c102d40 0x7f3c4c1095d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3c44009ca0 tx=0x7f3c4402c0f0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.978+0000 7f3c5304e640 1 -- 172.21.15.94:0/2953845133 shutdown_connections 2024-02-17T11:56:02.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.978+0000 7f3c5304e640 1 --2- 172.21.15.94:0/2953845133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c102d40 0x7f3c4c1095d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.978+0000 7f3c5304e640 1 --2- 172.21.15.94:0/2953845133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c4c102380 0x7f3c4c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.978+0000 7f3c5304e640 1 --2- 172.21.15.94:0/2953845133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4c101190 0x7f3c4c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.978+0000 7f3c5304e640 1 -- 172.21.15.94:0/2953845133 >> 172.21.15.94:0/2953845133 conn(0x7f3c4c0fc720 msgr2=0x7f3c4c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:02.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.978+0000 7f3c5304e640 1 -- 172.21.15.94:0/2953845133 shutdown_connections 2024-02-17T11:56:02.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.978+0000 7f3c5304e640 1 -- 172.21.15.94:0/2953845133 wait complete. 2024-02-17T11:56:02.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c5304e640 1 Processor -- start 2024-02-17T11:56:02.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c5304e640 1 -- start start 2024-02-17T11:56:02.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c5304e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c101190 0x7f3c4c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:02.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c5304e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c4c102380 0x7f3c4c1977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:02.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c5304e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4c102d40 0x7f3c4c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:02.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c5304e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c19ed90 con 0x7f3c4c101190 2024-02-17T11:56:02.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c5304e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c19ef00 con 0x7f3c4c102d40 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c5304e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4c19f0c0 con 0x7f3c4c102380 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c50dea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c101190 0x7f3c4c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c50dea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c101190 0x7f3c4c1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37762/0 (socket says 172.21.15.94:37762) 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c50dea640 1 -- 172.21.15.94:0/2459853454 learned_addr learned my addr 172.21.15.94:0/2459853454 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c50dea640 1 -- 172.21.15.94:0/2459853454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c4c102380 msgr2=0x7f3c4c1977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c50dea640 1 --2- 172.21.15.94:0/2459853454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c4c102380 0x7f3c4c1977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c50dea640 1 -- 172.21.15.94:0/2459853454 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4c102d40 msgr2=0x7f3c4c19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c50dea640 1 --2- 172.21.15.94:0/2459853454 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4c102d40 0x7f3c4c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c50dea640 1 -- 172.21.15.94:0/2459853454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c44009900 con 0x7f3c4c101190 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c50dea640 1 --2- 172.21.15.94:0/2459853454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c101190 0x7f3c4c1972a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000b450 tx=0x7f3c4000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c49ffb640 1 -- 172.21.15.94:0/2459853454 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c40011890 con 0x7f3c4c101190 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.979+0000 7f3c49ffb640 1 -- 172.21.15.94:0/2459853454 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c400103e0 con 0x7f3c4c101190 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.980+0000 7f3c49ffb640 1 -- 172.21.15.94:0/2459853454 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4000f580 con 0x7f3c4c101190 2024-02-17T11:56:02.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.980+0000 7f3c5304e640 1 -- 172.21.15.94:0/2459853454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c4c19f3a0 con 0x7f3c4c101190 2024-02-17T11:56:02.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.980+0000 7f3c5304e640 1 -- 172.21.15.94:0/2459853454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c4c19f930 con 0x7f3c4c101190 2024-02-17T11:56:02.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.981+0000 7f3c5304e640 1 -- 172.21.15.94:0/2459853454 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4c108530 con 0x7f3c4c101190 2024-02-17T11:56:02.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.982+0000 7f3c49ffb640 1 -- 172.21.15.94:0/2459853454 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 16) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f3c400108e0 con 0x7f3c4c101190 2024-02-17T11:56:02.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.982+0000 7f3c49ffb640 1 --2- 172.21.15.94:0/2459853454 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f3c3c0a20c0 0x7f3c3c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:02.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.982+0000 7f3c4bfff640 1 --2- 172.21.15.94:0/2459853454 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f3c3c0a20c0 0x7f3c3c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:02.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.983+0000 7f3c49ffb640 1 -- 172.21.15.94:0/2459853454 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f3c400cd980 con 0x7f3c4c101190 2024-02-17T11:56:02.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.983+0000 7f3c4bfff640 1 --2- 172.21.15.94:0/2459853454 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f3c3c0a20c0 0x7f3c3c0a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c1987b0 tx=0x7f3c38006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:02.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:02.987+0000 7f3c49ffb640 1 -- 172.21.15.94:0/2459853454 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3c40096aa0 con 0x7f3c4c101190 2024-02-17T11:56:03.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.169+0000 7f3c49ffb640 1 -- 172.21.15.94:0/2459853454 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f3c40096e70 con 0x7f3c4c101190 2024-02-17T11:56:03.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.193+0000 7f3c5304e640 1 -- 172.21.15.94:0/2459853454 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c4c02cf10 con 0x7f3c3c0a20c0 2024-02-17T11:56:03.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.197+0000 7f3c49ffb640 1 -- 172.21.15.94:0/2459853454 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7f3c4c02cf10 con 0x7f3c3c0a20c0 2024-02-17T11:56:03.195 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/qa/sv_0/dc6cb3b5-cdfe-43f2-9c70-cc52af1e04e0 2024-02-17T11:56:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.199+0000 7f3c5304e640 1 -- 172.21.15.94:0/2459853454 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f3c3c0a20c0 msgr2=0x7f3c3c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.199+0000 7f3c5304e640 1 --2- 172.21.15.94:0/2459853454 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f3c3c0a20c0 0x7f3c3c0a4580 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c1987b0 tx=0x7f3c38006c90 comp rx=0 tx=0).stop 2024-02-17T11:56:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.199+0000 7f3c5304e640 1 -- 172.21.15.94:0/2459853454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c101190 msgr2=0x7f3c4c1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.199+0000 7f3c5304e640 1 --2- 172.21.15.94:0/2459853454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c101190 0x7f3c4c1972a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3c4000b450 tx=0x7f3c4000b910 comp rx=0 tx=0).stop 2024-02-17T11:56:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.199+0000 7f3c5304e640 1 -- 172.21.15.94:0/2459853454 shutdown_connections 2024-02-17T11:56:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.199+0000 7f3c5304e640 1 --2- 172.21.15.94:0/2459853454 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f3c3c0a20c0 0x7f3c3c0a4580 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.199+0000 7f3c5304e640 1 --2- 172.21.15.94:0/2459853454 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c4c102d40 0x7f3c4c19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.199+0000 7f3c5304e640 1 --2- 172.21.15.94:0/2459853454 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c4c102380 0x7f3c4c1977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.199+0000 7f3c5304e640 1 --2- 172.21.15.94:0/2459853454 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c4c101190 0x7f3c4c1972a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.199+0000 7f3c5304e640 1 -- 172.21.15.94:0/2459853454 >> 172.21.15.94:0/2459853454 conn(0x7f3c4c0fc720 msgr2=0x7f3c4c105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.199+0000 7f3c5304e640 1 -- 172.21.15.94:0/2459853454 shutdown_connections 2024-02-17T11:56:03.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.199+0000 7f3c5304e640 1 -- 172.21.15.94:0/2459853454 wait complete. 2024-02-17T11:56:03.208 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-02-17T11:56:03.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.316+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1754040637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6eb8100080 msgr2=0x7f6eb81004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.316+0000 7f6ebdf01640 1 --2- 172.21.15.94:0/1754040637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6eb8100080 0x7f6eb81004a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8009900 tx=0x7f6ea802bc30 comp rx=0 tx=0).stop 2024-02-17T11:56:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.316+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1754040637 shutdown_connections 2024-02-17T11:56:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.316+0000 7f6ebdf01640 1 --2- 172.21.15.94:0/1754040637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6eb8101c30 0x7f6eb81084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.316+0000 7f6ebdf01640 1 --2- 172.21.15.94:0/1754040637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6eb8101270 0x7f6eb81016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.316+0000 7f6ebdf01640 1 --2- 172.21.15.94:0/1754040637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6eb8100080 0x7f6eb81004a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.316+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1754040637 >> 172.21.15.94:0/1754040637 conn(0x7f6eb80fb610 msgr2=0x7f6eb80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.316+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1754040637 shutdown_connections 2024-02-17T11:56:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.316+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1754040637 wait complete. 2024-02-17T11:56:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.317+0000 7f6ebdf01640 1 Processor -- start 2024-02-17T11:56:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.317+0000 7f6ebdf01640 1 -- start start 2024-02-17T11:56:03.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.317+0000 7f6ebdf01640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6eb8100080 0x7f6eb8197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.317+0000 7f6ebdf01640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6eb8101270 0x7f6eb8197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.317+0000 7f6ebdf01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6eb8101c30 0x7f6eb819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.317+0000 7f6ebdf01640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb819edf0 con 0x7f6eb8101c30 2024-02-17T11:56:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.317+0000 7f6ebdf01640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb819ef60 con 0x7f6eb8101270 2024-02-17T11:56:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.317+0000 7f6ebdf01640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb819f120 con 0x7f6eb8100080 2024-02-17T11:56:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.317+0000 7f6eb7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6eb8101c30 0x7f6eb819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.317+0000 7f6eb7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6eb8101c30 0x7f6eb819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37778/0 (socket says 172.21.15.94:37778) 2024-02-17T11:56:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.317+0000 7f6eb7fff640 1 -- 172.21.15.94:0/1337306629 learned_addr learned my addr 172.21.15.94:0/1337306629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6eb77fe640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6eb8100080 0x7f6eb8197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6eb6ffd640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6eb8101270 0x7f6eb8197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6eb7fff640 1 -- 172.21.15.94:0/1337306629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6eb8100080 msgr2=0x7f6eb8197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6eb7fff640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6eb8100080 0x7f6eb8197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6eb7fff640 1 -- 172.21.15.94:0/1337306629 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6eb8101270 msgr2=0x7f6eb8197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:03.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6eb7fff640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6eb8101270 0x7f6eb8197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6eb7fff640 1 -- 172.21.15.94:0/1337306629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea8009560 con 0x7f6eb8101c30 2024-02-17T11:56:03.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6eb6ffd640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6eb8101270 0x7f6eb8197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:56:03.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6eb7fff640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6eb8101c30 0x7f6eb819e8b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000eaa0 tx=0x7f6eb000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:03.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6eb4ff9640 1 -- 172.21.15.94:0/1337306629 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000cdb0 con 0x7f6eb8101c30 2024-02-17T11:56:03.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6eb4ff9640 1 -- 172.21.15.94:0/1337306629 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6eb0005020 con 0x7f6eb8101c30 2024-02-17T11:56:03.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1337306629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb819f400 con 0x7f6eb8101c30 2024-02-17T11:56:03.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6eb4ff9640 1 -- 172.21.15.94:0/1337306629 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00116b0 con 0x7f6eb8101c30 2024-02-17T11:56:03.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.318+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1337306629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb819f990 con 0x7f6eb8101c30 2024-02-17T11:56:03.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.319+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1337306629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb8107470 con 0x7f6eb8101c30 2024-02-17T11:56:03.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.320+0000 7f6eb4ff9640 1 -- 172.21.15.94:0/1337306629 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f6eb0005520 con 0x7f6eb8101c30 2024-02-17T11:56:03.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.321+0000 7f6eb4ff9640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f6ea00a21d0 0x7f6ea00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:03.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.321+0000 7f6eb77fe640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f6ea00a21d0 0x7f6ea00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:03.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.321+0000 7f6eb4ff9640 1 -- 172.21.15.94:0/1337306629 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6eb00ce250 con 0x7f6eb8101c30 2024-02-17T11:56:03.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.321+0000 7f6eb77fe640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f6ea00a21d0 0x7f6ea00a4690 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8005ab0 tx=0x7f6ea8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:03.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.324+0000 7f6eb4ff9640 1 -- 172.21.15.94:0/1337306629 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6eb00972f0 con 0x7f6eb8101c30 2024-02-17T11:56:03.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:03 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/227920916' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-17T11:56:03.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:03 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] up:active 2024-02-17T11:56:03.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:03 smithi189 ceph-mon[36588]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-02-17T11:56:03.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:03 smithi189 ceph-mon[36588]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T11:56:03.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:03 smithi189 ceph-mon[36588]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:03.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:03 smithi189 ceph-mon[36588]: pgmap v287: 97 pgs: 6 creating+peering, 91 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s wr, 16 op/s 2024-02-17T11:56:03.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:03 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/227920916' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-17T11:56:03.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:03 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] up:active 2024-02-17T11:56:03.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:03 smithi125 ceph-mon[36819]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-02-17T11:56:03.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:03 smithi125 ceph-mon[36819]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T11:56:03.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:03 smithi125 ceph-mon[36819]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:03.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:03 smithi125 ceph-mon[36819]: pgmap v287: 97 pgs: 6 creating+peering, 91 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s wr, 16 op/s 2024-02-17T11:56:03.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:03 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/227920916' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-17T11:56:03.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:03 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] up:active 2024-02-17T11:56:03.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:03 smithi094 ceph-mon[32499]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-02-17T11:56:03.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:03 smithi094 ceph-mon[32499]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T11:56:03.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:03 smithi094 ceph-mon[32499]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:03.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:03 smithi094 ceph-mon[32499]: pgmap v287: 97 pgs: 6 creating+peering, 91 active+clean; 581 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s wr, 16 op/s 2024-02-17T11:56:03.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.526+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1337306629 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f6eb802ceb0 con 0x7f6ea00a21d0 2024-02-17T11:56:03.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.549+0000 7f6eb4ff9640 1 -- 172.21.15.94:0/1337306629 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6eb802ceb0 con 0x7f6ea00a21d0 2024-02-17T11:56:03.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.551+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1337306629 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f6ea00a21d0 msgr2=0x7f6ea00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:03.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.551+0000 7f6ebdf01640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f6ea00a21d0 0x7f6ea00a4690 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8005ab0 tx=0x7f6ea8005fd0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.551+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1337306629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6eb8101c30 msgr2=0x7f6eb819e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:03.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.551+0000 7f6ebdf01640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6eb8101c30 0x7f6eb819e8b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000eaa0 tx=0x7f6eb000ef60 comp rx=0 tx=0).stop 2024-02-17T11:56:03.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.551+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1337306629 shutdown_connections 2024-02-17T11:56:03.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.551+0000 7f6ebdf01640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f6ea00a21d0 0x7f6ea00a4690 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.551+0000 7f6ebdf01640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6eb8101c30 0x7f6eb819e8b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.551+0000 7f6ebdf01640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6eb8101270 0x7f6eb8197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.551+0000 7f6ebdf01640 1 --2- 172.21.15.94:0/1337306629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6eb8100080 0x7f6eb8197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.551+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1337306629 >> 172.21.15.94:0/1337306629 conn(0x7f6eb80fb610 msgr2=0x7f6eb8104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:03.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.551+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1337306629 shutdown_connections 2024-02-17T11:56:03.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.551+0000 7f6ebdf01640 1 -- 172.21.15.94:0/1337306629 wait complete. 2024-02-17T11:56:03.561 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-02-17T11:56:03.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.669+0000 7fb97de09640 1 -- 172.21.15.94:0/3686768652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9780ff470 msgr2=0x7fb9780ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:03.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.669+0000 7fb97de09640 1 --2- 172.21.15.94:0/3686768652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9780ff470 0x7fb9780ff890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb968009900 tx=0x7fb96802bc30 comp rx=0 tx=0).stop 2024-02-17T11:56:03.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.669+0000 7fb97de09640 1 -- 172.21.15.94:0/3686768652 shutdown_connections 2024-02-17T11:56:03.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.669+0000 7fb97de09640 1 --2- 172.21.15.94:0/3686768652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978102e00 0x7fb978109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.669+0000 7fb97de09640 1 --2- 172.21.15.94:0/3686768652 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb978102440 0x7fb9781028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.669+0000 7fb97de09640 1 --2- 172.21.15.94:0/3686768652 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9780ff470 0x7fb9780ff890 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.669+0000 7fb97de09640 1 -- 172.21.15.94:0/3686768652 >> 172.21.15.94:0/3686768652 conn(0x7fb9780fad80 msgr2=0x7fb9780fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:03.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.669+0000 7fb97de09640 1 -- 172.21.15.94:0/3686768652 shutdown_connections 2024-02-17T11:56:03.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.669+0000 7fb97de09640 1 -- 172.21.15.94:0/3686768652 wait complete. 2024-02-17T11:56:03.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.669+0000 7fb97de09640 1 Processor -- start 2024-02-17T11:56:03.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.669+0000 7fb97de09640 1 -- start start 2024-02-17T11:56:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb97de09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9780ff470 0x7fb9781972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb97de09640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb978102440 0x7fb978197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb97de09640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978102e00 0x7fb97819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb97de09640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97819edb0 con 0x7fb9780ff470 2024-02-17T11:56:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb97de09640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97819ef20 con 0x7fb978102440 2024-02-17T11:56:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb97de09640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97819f0e0 con 0x7fb978102e00 2024-02-17T11:56:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb9777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9780ff470 0x7fb9781972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb9777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9780ff470 0x7fb9781972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37794/0 (socket says 172.21.15.94:37794) 2024-02-17T11:56:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb9777fe640 1 -- 172.21.15.94:0/135985493 learned_addr learned my addr 172.21.15.94:0/135985493 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb976ffd640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb978102440 0x7fb978197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb977fff640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978102e00 0x7fb97819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:03.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb9777fe640 1 -- 172.21.15.94:0/135985493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978102e00 msgr2=0x7fb97819e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb9777fe640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978102e00 0x7fb97819e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb9777fe640 1 -- 172.21.15.94:0/135985493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb978102440 msgr2=0x7fb978197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb9777fe640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb978102440 0x7fb978197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb9777fe640 1 -- 172.21.15.94:0/135985493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb968009560 con 0x7fb9780ff470 2024-02-17T11:56:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb977fff640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978102e00 0x7fb97819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb9777fe640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9780ff470 0x7fb9781972c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb968036040 tx=0x7fb9680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb974ff9640 1 -- 172.21.15.94:0/135985493 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb968004510 con 0x7fb9780ff470 2024-02-17T11:56:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb974ff9640 1 -- 172.21.15.94:0/135985493 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9680054e0 con 0x7fb9780ff470 2024-02-17T11:56:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb974ff9640 1 -- 172.21.15.94:0/135985493 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96803e450 con 0x7fb9780ff470 2024-02-17T11:56:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb97de09640 1 -- 172.21.15.94:0/135985493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97819f360 con 0x7fb9780ff470 2024-02-17T11:56:03.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.670+0000 7fb97de09640 1 -- 172.21.15.94:0/135985493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97819f8f0 con 0x7fb9780ff470 2024-02-17T11:56:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.671+0000 7fb97de09640 1 -- 172.21.15.94:0/135985493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb978108570 con 0x7fb9780ff470 2024-02-17T11:56:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.672+0000 7fb974ff9640 1 -- 172.21.15.94:0/135985493 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fb968003710 con 0x7fb9780ff470 2024-02-17T11:56:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.673+0000 7fb974ff9640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb9600a2150 0x7fb9600a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:03.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.673+0000 7fb976ffd640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb9600a2150 0x7fb9600a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:03.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.673+0000 7fb974ff9640 1 -- 172.21.15.94:0/135985493 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb9680f0120 con 0x7fb9780ff470 2024-02-17T11:56:03.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.673+0000 7fb976ffd640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb9600a2150 0x7fb9600a4610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb9781987d0 tx=0x7fb96c00a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:03.675 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.677+0000 7fb974ff9640 1 -- 172.21.15.94:0/135985493 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb9680b9250 con 0x7fb9780ff470 2024-02-17T11:56:03.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.878+0000 7fb97de09640 1 -- 172.21.15.94:0/135985493 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb97802ce10 con 0x7fb9600a2150 2024-02-17T11:56:03.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.882+0000 7fb974ff9640 1 -- 172.21.15.94:0/135985493 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fb97802ce10 con 0x7fb9600a2150 2024-02-17T11:56:03.880 INFO:teuthology.orchestra.run.smithi094.stdout:/volumes/qa/sv_1/38f266b8-4b32-401a-962e-ca40f2c90bf2 2024-02-17T11:56:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.884+0000 7fb97de09640 1 -- 172.21.15.94:0/135985493 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb9600a2150 msgr2=0x7fb9600a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.884+0000 7fb97de09640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb9600a2150 0x7fb9600a4610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb9781987d0 tx=0x7fb96c00a4c0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.884+0000 7fb97de09640 1 -- 172.21.15.94:0/135985493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9780ff470 msgr2=0x7fb9781972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.884+0000 7fb97de09640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9780ff470 0x7fb9781972c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb968036040 tx=0x7fb9680042f0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.884+0000 7fb97de09640 1 -- 172.21.15.94:0/135985493 shutdown_connections 2024-02-17T11:56:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.884+0000 7fb97de09640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb9600a2150 0x7fb9600a4610 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.884+0000 7fb97de09640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb978102e00 0x7fb97819e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.884+0000 7fb97de09640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb978102440 0x7fb978197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.884+0000 7fb97de09640 1 --2- 172.21.15.94:0/135985493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9780ff470 0x7fb9781972c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.884+0000 7fb97de09640 1 -- 172.21.15.94:0/135985493 >> 172.21.15.94:0/135985493 conn(0x7fb9780fad80 msgr2=0x7fb978105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:03.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.884+0000 7fb97de09640 1 -- 172.21.15.94:0/135985493 shutdown_connections 2024-02-17T11:56:03.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.884+0000 7fb97de09640 1 -- 172.21.15.94:0/135985493 wait complete. 2024-02-17T11:56:03.893 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:56:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.996+0000 7ff37d54b640 1 -- 172.21.15.94:0/4060128026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff378104550 msgr2=0x7ff378106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 --2- 172.21.15.94:0/4060128026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff378104550 0x7ff378106940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff360009900 tx=0x7ff36002bc30 comp rx=0 tx=0).stop 2024-02-17T11:56:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 -- 172.21.15.94:0/4060128026 shutdown_connections 2024-02-17T11:56:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 --2- 172.21.15.94:0/4060128026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff378106e80 0x7ff3781092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 --2- 172.21.15.94:0/4060128026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff378104550 0x7ff378106940 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 --2- 172.21.15.94:0/4060128026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff378101c20 0x7ff378104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 -- 172.21.15.94:0/4060128026 >> 172.21.15.94:0/4060128026 conn(0x7ff3780fb610 msgr2=0x7ff3780fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 -- 172.21.15.94:0/4060128026 shutdown_connections 2024-02-17T11:56:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 -- 172.21.15.94:0/4060128026 wait complete. 2024-02-17T11:56:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 Processor -- start 2024-02-17T11:56:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 -- start start 2024-02-17T11:56:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff378101c20 0x7ff378197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff378104550 0x7ff3781977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff378106e80 0x7ff37819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37819ed70 con 0x7ff378106e80 2024-02-17T11:56:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37819eee0 con 0x7ff378104550 2024-02-17T11:56:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff37d54b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37819f0a0 con 0x7ff378101c20 2024-02-17T11:56:03.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff3777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff378106e80 0x7ff37819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff3777fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff378106e80 0x7ff37819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37824/0 (socket says 172.21.15.94:37824) 2024-02-17T11:56:04.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.997+0000 7ff3777fe640 1 -- 172.21.15.94:0/2564034638 learned_addr learned my addr 172.21.15.94:0/2564034638 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:04.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff376ffd640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff378101c20 0x7ff378197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff36ffff640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff378104550 0x7ff3781977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff3777fe640 1 -- 172.21.15.94:0/2564034638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff378101c20 msgr2=0x7ff378197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff3777fe640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff378101c20 0x7ff378197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff3777fe640 1 -- 172.21.15.94:0/2564034638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff378104550 msgr2=0x7ff3781977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff3777fe640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff378104550 0x7ff3781977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff3777fe640 1 -- 172.21.15.94:0/2564034638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff360009560 con 0x7ff378106e80 2024-02-17T11:56:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff376ffd640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff378101c20 0x7ff378197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff3777fe640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff378106e80 0x7ff37819e830 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff370009fd0 tx=0x7ff37000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff374ff9640 1 -- 172.21.15.94:0/2564034638 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37000bea0 con 0x7ff378106e80 2024-02-17T11:56:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff374ff9640 1 -- 172.21.15.94:0/2564034638 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff370009940 con 0x7ff378106e80 2024-02-17T11:56:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff374ff9640 1 -- 172.21.15.94:0/2564034638 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3700106c0 con 0x7ff378106e80 2024-02-17T11:56:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff37d54b640 1 -- 172.21.15.94:0/2564034638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37819f380 con 0x7ff378106e80 2024-02-17T11:56:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.998+0000 7ff37d54b640 1 -- 172.21.15.94:0/2564034638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37819f810 con 0x7ff378106e80 2024-02-17T11:56:04.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:03.999+0000 7ff37d54b640 1 -- 172.21.15.94:0/2564034638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3780fd100 con 0x7ff378106e80 2024-02-17T11:56:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.000+0000 7ff374ff9640 1 -- 172.21.15.94:0/2564034638 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ff370004240 con 0x7ff378106e80 2024-02-17T11:56:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.000+0000 7ff374ff9640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff3640a2150 0x7ff3640a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.001+0000 7ff376ffd640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff3640a2150 0x7ff3640a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.001+0000 7ff374ff9640 1 -- 172.21.15.94:0/2564034638 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff3700cda60 con 0x7ff378106e80 2024-02-17T11:56:04.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.001+0000 7ff376ffd640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff3640a2150 0x7ff3640a4610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff368009b50 tx=0x7ff368005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:04.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.005+0000 7ff374ff9640 1 -- 172.21.15.94:0/2564034638 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff370096a80 con 0x7ff378106e80 2024-02-17T11:56:04.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.219+0000 7ff37d54b640 1 -- 172.21.15.94:0/2564034638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff37810cb70 con 0x7ff378106e80 2024-02-17T11:56:04.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.219+0000 7ff374ff9640 1 -- 172.21.15.94:0/2564034638 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11346 (secure 0 0 0) 0x7ff37009a140 con 0x7ff378106e80 2024-02-17T11:56:04.218 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:56:04.218 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:56:02.151161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T11:56:04.219 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 20 2024-02-17T11:56:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.222+0000 7ff37d54b640 1 -- 172.21.15.94:0/2564034638 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff3640a2150 msgr2=0x7ff3640a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.222+0000 7ff37d54b640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff3640a2150 0x7ff3640a4610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff368009b50 tx=0x7ff368005f90 comp rx=0 tx=0).stop 2024-02-17T11:56:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.222+0000 7ff37d54b640 1 -- 172.21.15.94:0/2564034638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff378106e80 msgr2=0x7ff37819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.222+0000 7ff37d54b640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff378106e80 0x7ff37819e830 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff370009fd0 tx=0x7ff37000b260 comp rx=0 tx=0).stop 2024-02-17T11:56:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.222+0000 7ff37d54b640 1 -- 172.21.15.94:0/2564034638 shutdown_connections 2024-02-17T11:56:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.222+0000 7ff37d54b640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff3640a2150 0x7ff3640a4610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.222+0000 7ff37d54b640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff378106e80 0x7ff37819e830 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.222+0000 7ff37d54b640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff378104550 0x7ff3781977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.222+0000 7ff37d54b640 1 --2- 172.21.15.94:0/2564034638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff378101c20 0x7ff378197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.222+0000 7ff37d54b640 1 -- 172.21.15.94:0/2564034638 >> 172.21.15.94:0/2564034638 conn(0x7ff3780fb610 msgr2=0x7ff3781024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.222+0000 7ff37d54b640 1 -- 172.21.15.94:0/2564034638 shutdown_connections 2024-02-17T11:56:04.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.222+0000 7ff37d54b640 1 -- 172.21.15.94:0/2564034638 wait complete. 2024-02-17T11:56:04.232 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:56:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.336+0000 7fad2a1ac640 1 -- 172.21.15.94:0/2864416739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24101c50 msgr2=0x7fad241084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.336+0000 7fad2a1ac640 1 --2- 172.21.15.94:0/2864416739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24101c50 0x7fad241084e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009ca0 tx=0x7fad1c02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.336+0000 7fad2a1ac640 1 -- 172.21.15.94:0/2864416739 shutdown_connections 2024-02-17T11:56:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.336+0000 7fad2a1ac640 1 --2- 172.21.15.94:0/2864416739 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24101c50 0x7fad241084e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.336+0000 7fad2a1ac640 1 --2- 172.21.15.94:0/2864416739 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad24101290 0x7fad24101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.336+0000 7fad2a1ac640 1 --2- 172.21.15.94:0/2864416739 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad241000a0 0x7fad241004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.336+0000 7fad2a1ac640 1 -- 172.21.15.94:0/2864416739 >> 172.21.15.94:0/2864416739 conn(0x7fad240fb610 msgr2=0x7fad240fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.336+0000 7fad2a1ac640 1 -- 172.21.15.94:0/2864416739 shutdown_connections 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.336+0000 7fad2a1ac640 1 -- 172.21.15.94:0/2864416739 wait complete. 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.336+0000 7fad2a1ac640 1 Processor -- start 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.336+0000 7fad2a1ac640 1 -- start start 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad2a1ac640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad241000a0 0x7fad24197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad2a1ac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad24101290 0x7fad24197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad2a1ac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24101c50 0x7fad2419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad2a1ac640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2419ec10 con 0x7fad24101c50 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad2a1ac640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2419ed80 con 0x7fad24101290 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad2a1ac640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2419ef40 con 0x7fad241000a0 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad23fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24101c50 0x7fad2419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad23fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24101c50 0x7fad2419e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37840/0 (socket says 172.21.15.94:37840) 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad23fff640 1 -- 172.21.15.94:0/3519428427 learned_addr learned my addr 172.21.15.94:0/3519428427 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:04.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad1affd640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad24101290 0x7fad24197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad237fe640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad241000a0 0x7fad24197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad23fff640 1 -- 172.21.15.94:0/3519428427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad241000a0 msgr2=0x7fad24197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad23fff640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad241000a0 0x7fad24197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad23fff640 1 -- 172.21.15.94:0/3519428427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad24101290 msgr2=0x7fad24197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad23fff640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad24101290 0x7fad24197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad23fff640 1 -- 172.21.15.94:0/3519428427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c009900 con 0x7fad24101c50 2024-02-17T11:56:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad237fe640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad241000a0 0x7fad24197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:56:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad23fff640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24101c50 0x7fad2419e6d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fad1c037040 tx=0x7fad1c034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad217fa640 1 -- 172.21.15.94:0/3519428427 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c032eb0 con 0x7fad24101c50 2024-02-17T11:56:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad217fa640 1 -- 172.21.15.94:0/3519428427 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad1c003e50 con 0x7fad24101c50 2024-02-17T11:56:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad2a1ac640 1 -- 172.21.15.94:0/3519428427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2419f1c0 con 0x7fad24101c50 2024-02-17T11:56:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad217fa640 1 -- 172.21.15.94:0/3519428427 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c040450 con 0x7fad24101c50 2024-02-17T11:56:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.337+0000 7fad2a1ac640 1 -- 172.21.15.94:0/3519428427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2419f670 con 0x7fad24101c50 2024-02-17T11:56:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.338+0000 7fad2a1ac640 1 -- 172.21.15.94:0/3519428427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad24107360 con 0x7fad24101c50 2024-02-17T11:56:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.339+0000 7fad217fa640 1 -- 172.21.15.94:0/3519428427 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fad1c00a040 con 0x7fad24101c50 2024-02-17T11:56:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.340+0000 7fad217fa640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fad100a2150 0x7fad100a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.340+0000 7fad237fe640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fad100a2150 0x7fad100a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.340+0000 7fad217fa640 1 -- 172.21.15.94:0/3519428427 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fad1c0f0ae0 con 0x7fad24101c50 2024-02-17T11:56:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.340+0000 7fad237fe640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fad100a2150 0x7fad100a4610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fad140058c0 tx=0x7fad14005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:04.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.344+0000 7fad217fa640 1 -- 172.21.15.94:0/3519428427 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fad1c0b9c10 con 0x7fad24101c50 2024-02-17T11:56:04.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:04 smithi125 ceph-mon[36819]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:04.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:04 smithi125 ceph-mon[36819]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-17T11:56:04.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:04 smithi125 ceph-mon[36819]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:04.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:04 smithi125 ceph-mon[36819]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:04.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:04 smithi094 ceph-mon[32499]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:04.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:04 smithi094 ceph-mon[32499]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-17T11:56:04.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:04 smithi094 ceph-mon[32499]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:04.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:04 smithi094 ceph-mon[32499]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:04.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.518+0000 7fad2a1ac640 1 -- 172.21.15.94:0/3519428427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad2402ce40 con 0x7fad24101c50 2024-02-17T11:56:04.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.520+0000 7fad217fa640 1 -- 172.21.15.94:0/3519428427 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22207 (secure 0 0 0) 0x7fad1c0bd2d0 con 0x7fad24101c50 2024-02-17T11:56:04.519 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:56:04.519 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":85,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:55:59.654265+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:56:04.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.522+0000 7fad2a1ac640 1 -- 172.21.15.94:0/3519428427 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fad100a2150 msgr2=0x7fad100a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.522+0000 7fad2a1ac640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fad100a2150 0x7fad100a4610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fad140058c0 tx=0x7fad14005850 comp rx=0 tx=0).stop 2024-02-17T11:56:04.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.522+0000 7fad2a1ac640 1 -- 172.21.15.94:0/3519428427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24101c50 msgr2=0x7fad2419e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.522+0000 7fad2a1ac640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24101c50 0x7fad2419e6d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fad1c037040 tx=0x7fad1c034d40 comp rx=0 tx=0).stop 2024-02-17T11:56:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.522+0000 7fad2a1ac640 1 -- 172.21.15.94:0/3519428427 shutdown_connections 2024-02-17T11:56:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.522+0000 7fad2a1ac640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fad100a2150 0x7fad100a4610 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.522+0000 7fad2a1ac640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad24101c50 0x7fad2419e6d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.522+0000 7fad2a1ac640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad24101290 0x7fad24197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.522+0000 7fad2a1ac640 1 --2- 172.21.15.94:0/3519428427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad241000a0 0x7fad24197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.522+0000 7fad2a1ac640 1 -- 172.21.15.94:0/3519428427 >> 172.21.15.94:0/3519428427 conn(0x7fad240fb610 msgr2=0x7fad24104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.522+0000 7fad2a1ac640 1 -- 172.21.15.94:0/3519428427 shutdown_connections 2024-02-17T11:56:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.522+0000 7fad2a1ac640 1 -- 172.21.15.94:0/3519428427 wait complete. 2024-02-17T11:56:04.533 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-17T11:56:04.533 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-17T11:56:04.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.636+0000 7ff3b75a7640 1 -- 172.21.15.94:0/293195977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b0101290 msgr2=0x7ff3b0101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.636+0000 7ff3b75a7640 1 --2- 172.21.15.94:0/293195977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b0101290 0x7ff3b0101710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac009900 tx=0x7ff3ac02bc30 comp rx=0 tx=0).stop 2024-02-17T11:56:04.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.636+0000 7ff3b75a7640 1 -- 172.21.15.94:0/293195977 shutdown_connections 2024-02-17T11:56:04.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.636+0000 7ff3b75a7640 1 --2- 172.21.15.94:0/293195977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3b0101c50 0x7ff3b01084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.636+0000 7ff3b75a7640 1 --2- 172.21.15.94:0/293195977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b0101290 0x7ff3b0101710 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.636+0000 7ff3b75a7640 1 --2- 172.21.15.94:0/293195977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3b01000a0 0x7ff3b01004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.636+0000 7ff3b75a7640 1 -- 172.21.15.94:0/293195977 >> 172.21.15.94:0/293195977 conn(0x7ff3b00fb610 msgr2=0x7ff3b00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:04.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.636+0000 7ff3b75a7640 1 -- 172.21.15.94:0/293195977 shutdown_connections 2024-02-17T11:56:04.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.636+0000 7ff3b75a7640 1 -- 172.21.15.94:0/293195977 wait complete. 2024-02-17T11:56:04.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.636+0000 7ff3b75a7640 1 Processor -- start 2024-02-17T11:56:04.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.636+0000 7ff3b75a7640 1 -- start start 2024-02-17T11:56:04.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b75a7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b01000a0 0x7ff3b0196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:04.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b75a7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3b0101290 0x7ff3b0196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:04.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b75a7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3b0101c50 0x7ff3b019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:04.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b75a7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b019db10 con 0x7ff3b01000a0 2024-02-17T11:56:04.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b75a7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b019dc80 con 0x7ff3b0101290 2024-02-17T11:56:04.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b75a7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b019de40 con 0x7ff3b0101c50 2024-02-17T11:56:04.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b5343640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b01000a0 0x7ff3b0196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b5343640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b01000a0 0x7ff3b0196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47812/0 (socket says 172.21.15.94:47812) 2024-02-17T11:56:04.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b5343640 1 -- 172.21.15.94:0/4105488873 learned_addr learned my addr 172.21.15.94:0/4105488873 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:04.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b5343640 1 -- 172.21.15.94:0/4105488873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3b0101c50 msgr2=0x7ff3b019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b5343640 1 --2- 172.21.15.94:0/4105488873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3b0101c50 0x7ff3b019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b5343640 1 -- 172.21.15.94:0/4105488873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3b0101290 msgr2=0x7ff3b0196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b5343640 1 --2- 172.21.15.94:0/4105488873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3b0101290 0x7ff3b0196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b5343640 1 -- 172.21.15.94:0/4105488873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac009560 con 0x7ff3b01000a0 2024-02-17T11:56:04.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b5343640 1 --2- 172.21.15.94:0/4105488873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b01000a0 0x7ff3b0196020 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c970 tx=0x7ff3a000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:04.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3a67fc640 1 -- 172.21.15.94:0/4105488873 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0007c10 con 0x7ff3b01000a0 2024-02-17T11:56:04.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b75a7640 1 -- 172.21.15.94:0/4105488873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b019e120 con 0x7ff3b01000a0 2024-02-17T11:56:04.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3a67fc640 1 -- 172.21.15.94:0/4105488873 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3a0007d70 con 0x7ff3b01000a0 2024-02-17T11:56:04.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3a67fc640 1 -- 172.21.15.94:0/4105488873 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a000f870 con 0x7ff3b01000a0 2024-02-17T11:56:04.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.637+0000 7ff3b75a7640 1 -- 172.21.15.94:0/4105488873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b019e630 con 0x7ff3b01000a0 2024-02-17T11:56:04.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.638+0000 7ff3b75a7640 1 -- 172.21.15.94:0/4105488873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b0107360 con 0x7ff3b01000a0 2024-02-17T11:56:04.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.639+0000 7ff3a67fc640 1 -- 172.21.15.94:0/4105488873 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ff3a0018a60 con 0x7ff3b01000a0 2024-02-17T11:56:04.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.640+0000 7ff3a67fc640 1 --2- 172.21.15.94:0/4105488873 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff39c0a2150 0x7ff39c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:04.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.640+0000 7ff3b4b42640 1 --2- 172.21.15.94:0/4105488873 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff39c0a2150 0x7ff39c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.640+0000 7ff3a67fc640 1 -- 172.21.15.94:0/4105488873 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff3a0013070 con 0x7ff3b01000a0 2024-02-17T11:56:04.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.640+0000 7ff3b4b42640 1 --2- 172.21.15.94:0/4105488873 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff39c0a2150 0x7ff39c0a4610 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac02c6a0 tx=0x7ff3ac000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:04.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.643+0000 7ff3a67fc640 1 -- 172.21.15.94:0/4105488873 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff3a0097460 con 0x7ff3b01000a0 2024-02-17T11:56:04.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:04 smithi189 ceph-mon[36588]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:04.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:04 smithi189 ceph-mon[36588]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-02-17T11:56:04.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:04 smithi189 ceph-mon[36588]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:04.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:04 smithi189 ceph-mon[36588]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:04.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.812+0000 7ff3b75a7640 1 -- 172.21.15.94:0/4105488873 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3b002cfa0 con 0x7ff39c0a2150 2024-02-17T11:56:04.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.823+0000 7ff3a67fc640 1 -- 172.21.15.94:0/4105488873 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336655 (secure 0 0 0) 0x7ff3b002cfa0 con 0x7ff39c0a2150 2024-02-17T11:56:04.826 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:56:04.829 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-02-17T11:56:04.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.826+0000 7ff3b75a7640 1 -- 172.21.15.94:0/4105488873 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff39c0a2150 msgr2=0x7ff39c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.826+0000 7ff3b75a7640 1 --2- 172.21.15.94:0/4105488873 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff39c0a2150 0x7ff39c0a4610 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac02c6a0 tx=0x7ff3ac000bc0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.826+0000 7ff3b75a7640 1 -- 172.21.15.94:0/4105488873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b01000a0 msgr2=0x7ff3b0196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.826+0000 7ff3b75a7640 1 --2- 172.21.15.94:0/4105488873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b01000a0 0x7ff3b0196020 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c970 tx=0x7ff3a000ce30 comp rx=0 tx=0).stop 2024-02-17T11:56:04.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.826+0000 7ff3b75a7640 1 -- 172.21.15.94:0/4105488873 shutdown_connections 2024-02-17T11:56:04.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.826+0000 7ff3b75a7640 1 --2- 172.21.15.94:0/4105488873 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff39c0a2150 0x7ff39c0a4610 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.826+0000 7ff3b75a7640 1 --2- 172.21.15.94:0/4105488873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3b0101c50 0x7ff3b019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.826+0000 7ff3b75a7640 1 --2- 172.21.15.94:0/4105488873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff3b0101290 0x7ff3b0196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.826+0000 7ff3b75a7640 1 --2- 172.21.15.94:0/4105488873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff3b01000a0 0x7ff3b0196020 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.826+0000 7ff3b75a7640 1 -- 172.21.15.94:0/4105488873 >> 172.21.15.94:0/4105488873 conn(0x7ff3b00fb610 msgr2=0x7ff3b0104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:04.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.826+0000 7ff3b75a7640 1 -- 172.21.15.94:0/4105488873 shutdown_connections 2024-02-17T11:56:04.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.826+0000 7ff3b75a7640 1 -- 172.21.15.94:0/4105488873 wait complete. 2024-02-17T11:56:04.836 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-02-17T11:56:04.652981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632176,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":145,"num_read_kb":144,"num_write":306,"num_write_kb":3007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":245,"ondisk_log_size":245,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":247,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360184,"kb_used_data":10960,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581640,"statfs":{"total":1151940427776,"available":1151571599360,"internally_reserved":0,"allocated":11223040,"data_stored":7016889,"data_compressed":108564,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41808,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":9,"num_read_kb":25,"num_write":125,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.000399"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.149529+0000","last_change":"2024-02-17T11:55:58.678553+0000","last_active":"2024-02-17T11:56:00.149529+0000","last_peered":"2024-02-17T11:56:00.149529+0000","last_clean":"2024-02-17T11:56:00.149529+0000","last_became_active":"2024-02-17T11:55:58.678212+0000","last_became_peered":"2024-02-17T11:55:58.678212+0000","last_unstale":"2024-02-17T11:56:00.149529+0000","last_undegraded":"2024-02-17T11:56:00.149529+0000","last_fullsized":"2024-02-17T11:56:00.149529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:09:18.011937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.683245+0000","last_change":"2024-02-17T11:55:58.681666+0000","last_active":"2024-02-17T11:55:58.683245+0000","last_peered":"2024-02-17T11:55:58.683245+0000","last_clean":"2024-02-17T11:55:58.683245+0000","last_became_active":"2024-02-17T11:55:58.681305+0000","last_became_peered":"2024-02-17T11:55:58.681305+0000","last_unstale":"2024-02-17T11:55:58.683245+0000","last_undegraded":"2024-02-17T11:55:58.683245+0000","last_fullsized":"2024-02-17T11:55:58.683245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:44:46.522902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.690117+0000","last_change":"2024-02-17T11:55:58.689465+0000","last_active":"2024-02-17T11:55:58.690117+0000","last_peered":"2024-02-17T11:55:58.690117+0000","last_clean":"2024-02-17T11:55:58.690117+0000","last_became_active":"2024-02-17T11:55:58.689166+0000","last_became_peered":"2024-02-17T11:55:58.689166+0000","last_unstale":"2024-02-17T11:55:58.690117+0000","last_undegraded":"2024-02-17T11:55:58.690117+0000","last_fullsized":"2024-02-17T11:55:58.690117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:31:27.999931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.144551+0000","last_change":"2024-02-17T11:55:58.678701+0000","last_active":"2024-02-17T11:56:00.144551+0000","last_peered":"2024-02-17T11:56:00.144551+0000","last_clean":"2024-02-17T11:56:00.144551+0000","last_became_active":"2024-02-17T11:55:58.678126+0000","last_became_peered":"2024-02-17T11:55:58.678126+0000","last_unstale":"2024-02-17T11:56:00.144551+0000","last_undegraded":"2024-02-17T11:56:00.144551+0000","last_fullsized":"2024-02-17T11:56:00.144551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:57:20.127009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143231+0000","last_change":"2024-02-17T11:55:58.677543+0000","last_active":"2024-02-17T11:56:00.143231+0000","last_peered":"2024-02-17T11:56:00.143231+0000","last_clean":"2024-02-17T11:56:00.143231+0000","last_became_active":"2024-02-17T11:55:58.677223+0000","last_became_peered":"2024-02-17T11:55:58.677223+0000","last_unstale":"2024-02-17T11:56:00.143231+0000","last_undegraded":"2024-02-17T11:56:00.143231+0000","last_fullsized":"2024-02-17T11:56:00.143231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:05:27.194963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140932+0000","last_change":"2024-02-17T11:55:58.673756+0000","last_active":"2024-02-17T11:56:00.140932+0000","last_peered":"2024-02-17T11:56:00.140932+0000","last_clean":"2024-02-17T11:56:00.140932+0000","last_became_active":"2024-02-17T11:55:58.672775+0000","last_became_peered":"2024-02-17T11:55:58.672775+0000","last_unstale":"2024-02-17T11:56:00.140932+0000","last_undegraded":"2024-02-17T11:56:00.140932+0000","last_fullsized":"2024-02-17T11:56:00.140932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:12:18.658414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143129+0000","last_change":"2024-02-17T11:55:58.675720+0000","last_active":"2024-02-17T11:56:00.143129+0000","last_peered":"2024-02-17T11:56:00.143129+0000","last_clean":"2024-02-17T11:56:00.143129+0000","last_became_active":"2024-02-17T11:55:58.675335+0000","last_became_peered":"2024-02-17T11:55:58.675335+0000","last_unstale":"2024-02-17T11:56:00.143129+0000","last_undegraded":"2024-02-17T11:56:00.143129+0000","last_fullsized":"2024-02-17T11:56:00.143129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:40:58.756439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139965+0000","last_change":"2024-02-17T11:55:58.675496+0000","last_active":"2024-02-17T11:56:00.139965+0000","last_peered":"2024-02-17T11:56:00.139965+0000","last_clean":"2024-02-17T11:56:00.139965+0000","last_became_active":"2024-02-17T11:55:58.675193+0000","last_became_peered":"2024-02-17T11:55:58.675193+0000","last_unstale":"2024-02-17T11:56:00.139965+0000","last_undegraded":"2024-02-17T11:56:00.139965+0000","last_fullsized":"2024-02-17T11:56:00.139965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:14:29.648857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.658999+0000","last_change":"2024-02-17T11:55:58.680859+0000","last_active":"2024-02-17T11:55:59.658999+0000","last_peered":"2024-02-17T11:55:59.658999+0000","last_clean":"2024-02-17T11:55:59.658999+0000","last_became_active":"2024-02-17T11:55:58.680489+0000","last_became_peered":"2024-02-17T11:55:58.680489+0000","last_unstale":"2024-02-17T11:55:59.658999+0000","last_undegraded":"2024-02-17T11:55:59.658999+0000","last_fullsized":"2024-02-17T11:55:59.658999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:29.963349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140798+0000","last_change":"2024-02-17T11:55:58.677896+0000","last_active":"2024-02-17T11:56:00.140798+0000","last_peered":"2024-02-17T11:56:00.140798+0000","last_clean":"2024-02-17T11:56:00.140798+0000","last_became_active":"2024-02-17T11:55:58.677540+0000","last_became_peered":"2024-02-17T11:55:58.677540+0000","last_unstale":"2024-02-17T11:56:00.140798+0000","last_undegraded":"2024-02-17T11:56:00.140798+0000","last_fullsized":"2024-02-17T11:56:00.140798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:20:47.921687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143870+0000","last_change":"2024-02-17T11:55:58.688562+0000","last_active":"2024-02-17T11:56:00.143870+0000","last_peered":"2024-02-17T11:56:00.143870+0000","last_clean":"2024-02-17T11:56:00.143870+0000","last_became_active":"2024-02-17T11:55:58.688186+0000","last_became_peered":"2024-02-17T11:55:58.688186+0000","last_unstale":"2024-02-17T11:56:00.143870+0000","last_undegraded":"2024-02-17T11:56:00.143870+0000","last_fullsized":"2024-02-17T11:56:00.143870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:09.668937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.144343+0000","last_change":"2024-02-17T11:55:58.684989+0000","last_active":"2024-02-17T11:56:00.144343+0000","last_peered":"2024-02-17T11:56:00.144343+0000","last_clean":"2024-02-17T11:56:00.144343+0000","last_became_active":"2024-02-17T11:55:58.684656+0000","last_became_peered":"2024-02-17T11:55:58.684656+0000","last_unstale":"2024-02-17T11:56:00.144343+0000","last_undegraded":"2024-02-17T11:56:00.144343+0000","last_fullsized":"2024-02-17T11:56:00.144343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:29:43.480963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657578+0000","last_change":"2024-02-17T11:55:58.662076+0000","last_active":"2024-02-17T11:55:59.657578+0000","last_peered":"2024-02-17T11:55:59.657578+0000","last_clean":"2024-02-17T11:55:59.657578+0000","last_became_active":"2024-02-17T11:55:58.661716+0000","last_became_peered":"2024-02-17T11:55:58.661716+0000","last_unstale":"2024-02-17T11:55:59.657578+0000","last_undegraded":"2024-02-17T11:55:59.657578+0000","last_fullsized":"2024-02-17T11:55:59.657578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:20:39.647774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.691857+0000","last_change":"2024-02-17T11:55:58.690762+0000","last_active":"2024-02-17T11:55:58.691857+0000","last_peered":"2024-02-17T11:55:58.691857+0000","last_clean":"2024-02-17T11:55:58.691857+0000","last_became_active":"2024-02-17T11:55:58.690414+0000","last_became_peered":"2024-02-17T11:55:58.690414+0000","last_unstale":"2024-02-17T11:55:58.691857+0000","last_undegraded":"2024-02-17T11:55:58.691857+0000","last_fullsized":"2024-02-17T11:55:58.691857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:39:55.313442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.683205+0000","last_change":"2024-02-17T11:55:58.682391+0000","last_active":"2024-02-17T11:55:58.683205+0000","last_peered":"2024-02-17T11:55:58.683205+0000","last_clean":"2024-02-17T11:55:58.683205+0000","last_became_active":"2024-02-17T11:55:58.682048+0000","last_became_peered":"2024-02-17T11:55:58.682048+0000","last_unstale":"2024-02-17T11:55:58.683205+0000","last_undegraded":"2024-02-17T11:55:58.683205+0000","last_fullsized":"2024-02-17T11:55:58.683205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:42:43.777522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.658566+0000","last_change":"2024-02-17T11:55:58.685782+0000","last_active":"2024-02-17T11:55:59.658566+0000","last_peered":"2024-02-17T11:55:59.658566+0000","last_clean":"2024-02-17T11:55:59.658566+0000","last_became_active":"2024-02-17T11:55:58.683087+0000","last_became_peered":"2024-02-17T11:55:58.683087+0000","last_unstale":"2024-02-17T11:55:59.658566+0000","last_undegraded":"2024-02-17T11:55:59.658566+0000","last_fullsized":"2024-02-17T11:55:59.658566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:30:39.653717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.149527+0000","last_change":"2024-02-17T11:55:58.688400+0000","last_active":"2024-02-17T11:56:00.149527+0000","last_peered":"2024-02-17T11:56:00.149527+0000","last_clean":"2024-02-17T11:56:00.149527+0000","last_became_active":"2024-02-17T11:55:58.688040+0000","last_became_peered":"2024-02-17T11:55:58.688040+0000","last_unstale":"2024-02-17T11:56:00.149527+0000","last_undegraded":"2024-02-17T11:56:00.149527+0000","last_fullsized":"2024-02-17T11:56:00.149527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:39:02.706547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.896558+0000","last_change":"2024-02-17T11:55:58.687454+0000","last_active":"2024-02-17T11:55:59.896558+0000","last_peered":"2024-02-17T11:55:59.896558+0000","last_clean":"2024-02-17T11:55:59.896558+0000","last_became_active":"2024-02-17T11:55:58.686432+0000","last_became_peered":"2024-02-17T11:55:58.686432+0000","last_unstale":"2024-02-17T11:55:59.896558+0000","last_undegraded":"2024-02-17T11:55:59.896558+0000","last_fullsized":"2024-02-17T11:55:59.896558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:26:31.497782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.662763+0000","last_change":"2024-02-17T11:55:58.681806+0000","last_active":"2024-02-17T11:55:59.662763+0000","last_peered":"2024-02-17T11:55:59.662763+0000","last_clean":"2024-02-17T11:55:59.662763+0000","last_became_active":"2024-02-17T11:55:58.679764+0000","last_became_peered":"2024-02-17T11:55:58.679764+0000","last_unstale":"2024-02-17T11:55:59.662763+0000","last_undegraded":"2024-02-17T11:55:59.662763+0000","last_fullsized":"2024-02-17T11:55:59.662763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:13:14.733417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.156700+0000","last_change":"2024-02-17T11:55:57.672689+0000","last_active":"2024-02-17T11:56:01.156700+0000","last_peered":"2024-02-17T11:56:01.156700+0000","last_clean":"2024-02-17T11:56:01.156700+0000","last_became_active":"2024-02-17T11:55:57.672352+0000","last_became_peered":"2024-02-17T11:55:57.672352+0000","last_unstale":"2024-02-17T11:56:01.156700+0000","last_undegraded":"2024-02-17T11:56:01.156700+0000","last_fullsized":"2024-02-17T11:56:01.156700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:00:34.395594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139808+0000","last_change":"2024-02-17T11:55:58.690679+0000","last_active":"2024-02-17T11:56:00.139808+0000","last_peered":"2024-02-17T11:56:00.139808+0000","last_clean":"2024-02-17T11:56:00.139808+0000","last_became_active":"2024-02-17T11:55:58.690353+0000","last_became_peered":"2024-02-17T11:55:58.690353+0000","last_unstale":"2024-02-17T11:56:00.139808+0000","last_undegraded":"2024-02-17T11:56:00.139808+0000","last_fullsized":"2024-02-17T11:56:00.139808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:45:33.830784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139853+0000","last_change":"2024-02-17T11:55:57.677649+0000","last_active":"2024-02-17T11:56:00.139853+0000","last_peered":"2024-02-17T11:56:00.139853+0000","last_clean":"2024-02-17T11:56:00.139853+0000","last_became_active":"2024-02-17T11:55:57.677331+0000","last_became_peered":"2024-02-17T11:55:57.677331+0000","last_unstale":"2024-02-17T11:56:00.139853+0000","last_undegraded":"2024-02-17T11:56:00.139853+0000","last_fullsized":"2024-02-17T11:56:00.139853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:06.932255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.689917+0000","last_change":"2024-02-17T11:55:58.688575+0000","last_active":"2024-02-17T11:55:58.689917+0000","last_peered":"2024-02-17T11:55:58.689917+0000","last_clean":"2024-02-17T11:55:58.689917+0000","last_became_active":"2024-02-17T11:55:58.688238+0000","last_became_peered":"2024-02-17T11:55:58.688238+0000","last_unstale":"2024-02-17T11:55:58.689917+0000","last_undegraded":"2024-02-17T11:55:58.689917+0000","last_fullsized":"2024-02-17T11:55:58.689917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:57:28.183776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.146001+0000","last_change":"2024-02-17T11:55:57.679676+0000","last_active":"2024-02-17T11:56:00.146001+0000","last_peered":"2024-02-17T11:56:00.146001+0000","last_clean":"2024-02-17T11:56:00.146001+0000","last_became_active":"2024-02-17T11:55:57.679350+0000","last_became_peered":"2024-02-17T11:55:57.679350+0000","last_unstale":"2024-02-17T11:56:00.146001+0000","last_undegraded":"2024-02-17T11:56:00.146001+0000","last_fullsized":"2024-02-17T11:56:00.146001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:52:45.072896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.659027+0000","last_change":"2024-02-17T11:55:58.682212+0000","last_active":"2024-02-17T11:55:59.659027+0000","last_peered":"2024-02-17T11:55:59.659027+0000","last_clean":"2024-02-17T11:55:59.659027+0000","last_became_active":"2024-02-17T11:55:58.681773+0000","last_became_peered":"2024-02-17T11:55:58.681773+0000","last_unstale":"2024-02-17T11:55:59.659027+0000","last_undegraded":"2024-02-17T11:55:59.659027+0000","last_fullsized":"2024-02-17T11:55:59.659027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:28:18.577174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.145333+0000","last_change":"2024-02-17T11:55:57.667320+0000","last_active":"2024-02-17T11:56:00.145333+0000","last_peered":"2024-02-17T11:56:00.145333+0000","last_clean":"2024-02-17T11:56:00.145333+0000","last_became_active":"2024-02-17T11:55:57.666989+0000","last_became_peered":"2024-02-17T11:55:57.666989+0000","last_unstale":"2024-02-17T11:56:00.145333+0000","last_undegraded":"2024-02-17T11:56:00.145333+0000","last_fullsized":"2024-02-17T11:56:00.145333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:00:08.275186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.144329+0000","last_change":"2024-02-17T11:55:58.672771+0000","last_active":"2024-02-17T11:56:00.144329+0000","last_peered":"2024-02-17T11:56:00.144329+0000","last_clean":"2024-02-17T11:56:00.144329+0000","last_became_active":"2024-02-17T11:55:58.671877+0000","last_became_peered":"2024-02-17T11:55:58.671877+0000","last_unstale":"2024-02-17T11:56:00.144329+0000","last_undegraded":"2024-02-17T11:56:00.144329+0000","last_fullsized":"2024-02-17T11:56:00.144329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:38:38.350695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.151798+0000","last_change":"2024-02-17T11:55:57.669396+0000","last_active":"2024-02-17T11:56:00.151798+0000","last_peered":"2024-02-17T11:56:00.151798+0000","last_clean":"2024-02-17T11:56:00.151798+0000","last_became_active":"2024-02-17T11:55:57.669116+0000","last_became_peered":"2024-02-17T11:55:57.669116+0000","last_unstale":"2024-02-17T11:56:00.151798+0000","last_undegraded":"2024-02-17T11:56:00.151798+0000","last_fullsized":"2024-02-17T11:56:00.151798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:05:11.081309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143170+0000","last_change":"2024-02-17T11:55:58.674379+0000","last_active":"2024-02-17T11:56:00.143170+0000","last_peered":"2024-02-17T11:56:00.143170+0000","last_clean":"2024-02-17T11:56:00.143170+0000","last_became_active":"2024-02-17T11:55:58.674015+0000","last_became_peered":"2024-02-17T11:55:58.674015+0000","last_unstale":"2024-02-17T11:56:00.143170+0000","last_undegraded":"2024-02-17T11:56:00.143170+0000","last_fullsized":"2024-02-17T11:56:00.143170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:44:20.410623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.154931+0000","last_change":"2024-02-17T11:55:57.668436+0000","last_active":"2024-02-17T11:56:01.154931+0000","last_peered":"2024-02-17T11:56:01.154931+0000","last_clean":"2024-02-17T11:56:01.154931+0000","last_became_active":"2024-02-17T11:55:57.668004+0000","last_became_peered":"2024-02-17T11:55:57.668004+0000","last_unstale":"2024-02-17T11:56:01.154931+0000","last_undegraded":"2024-02-17T11:56:01.154931+0000","last_fullsized":"2024-02-17T11:56:01.154931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:13:42.566644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.147215+0000","last_change":"2024-02-17T11:55:58.686888+0000","last_active":"2024-02-17T11:56:00.147215+0000","last_peered":"2024-02-17T11:56:00.147215+0000","last_clean":"2024-02-17T11:56:00.147215+0000","last_became_active":"2024-02-17T11:55:58.686376+0000","last_became_peered":"2024-02-17T11:55:58.686376+0000","last_unstale":"2024-02-17T11:56:00.147215+0000","last_undegraded":"2024-02-17T11:56:00.147215+0000","last_fullsized":"2024-02-17T11:56:00.147215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:58:48.292354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.155654+0000","last_change":"2024-02-17T11:55:57.665717+0000","last_active":"2024-02-17T11:56:01.155654+0000","last_peered":"2024-02-17T11:56:01.155654+0000","last_clean":"2024-02-17T11:56:01.155654+0000","last_became_active":"2024-02-17T11:55:57.665381+0000","last_became_peered":"2024-02-17T11:55:57.665381+0000","last_unstale":"2024-02-17T11:56:01.155654+0000","last_undegraded":"2024-02-17T11:56:01.155654+0000","last_fullsized":"2024-02-17T11:56:01.155654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:15:27.424332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139912+0000","last_change":"2024-02-17T11:55:58.673261+0000","last_active":"2024-02-17T11:56:00.139912+0000","last_peered":"2024-02-17T11:56:00.139912+0000","last_clean":"2024-02-17T11:56:00.139912+0000","last_became_active":"2024-02-17T11:55:58.672912+0000","last_became_peered":"2024-02-17T11:55:58.672912+0000","last_unstale":"2024-02-17T11:56:00.139912+0000","last_undegraded":"2024-02-17T11:56:00.139912+0000","last_fullsized":"2024-02-17T11:56:00.139912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:42.605291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148312+0000","last_change":"2024-02-17T11:55:57.667792+0000","last_active":"2024-02-17T11:56:00.148312+0000","last_peered":"2024-02-17T11:56:00.148312+0000","last_clean":"2024-02-17T11:56:00.148312+0000","last_became_active":"2024-02-17T11:55:57.667361+0000","last_became_peered":"2024-02-17T11:55:57.667361+0000","last_unstale":"2024-02-17T11:56:00.148312+0000","last_undegraded":"2024-02-17T11:56:00.148312+0000","last_fullsized":"2024-02-17T11:56:00.148312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:48.073742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.896505+0000","last_change":"2024-02-17T11:55:58.672180+0000","last_active":"2024-02-17T11:55:59.896505+0000","last_peered":"2024-02-17T11:55:59.896505+0000","last_clean":"2024-02-17T11:55:59.896505+0000","last_became_active":"2024-02-17T11:55:58.671790+0000","last_became_peered":"2024-02-17T11:55:58.671790+0000","last_unstale":"2024-02-17T11:55:59.896505+0000","last_undegraded":"2024-02-17T11:55:59.896505+0000","last_fullsized":"2024-02-17T11:55:59.896505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:48:39.982924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:02.159261+0000","last_change":"2024-02-17T11:55:57.670085+0000","last_active":"2024-02-17T11:56:02.159261+0000","last_peered":"2024-02-17T11:56:02.159261+0000","last_clean":"2024-02-17T11:56:02.159261+0000","last_became_active":"2024-02-17T11:55:57.669648+0000","last_became_peered":"2024-02-17T11:55:57.669648+0000","last_unstale":"2024-02-17T11:56:02.159261+0000","last_undegraded":"2024-02-17T11:56:02.159261+0000","last_fullsized":"2024-02-17T11:56:02.159261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:31:34.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.658885+0000","last_change":"2024-02-17T11:55:58.685080+0000","last_active":"2024-02-17T11:55:59.658885+0000","last_peered":"2024-02-17T11:55:59.658885+0000","last_clean":"2024-02-17T11:55:59.658885+0000","last_became_active":"2024-02-17T11:55:58.684763+0000","last_became_peered":"2024-02-17T11:55:58.684763+0000","last_unstale":"2024-02-17T11:55:59.658885+0000","last_undegraded":"2024-02-17T11:55:59.658885+0000","last_fullsized":"2024-02-17T11:55:59.658885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:59:53.763492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148316+0000","last_change":"2024-02-17T11:55:57.676391+0000","last_active":"2024-02-17T11:56:00.148316+0000","last_peered":"2024-02-17T11:56:00.148316+0000","last_clean":"2024-02-17T11:56:00.148316+0000","last_became_active":"2024-02-17T11:55:57.676103+0000","last_became_peered":"2024-02-17T11:55:57.676103+0000","last_unstale":"2024-02-17T11:56:00.148316+0000","last_undegraded":"2024-02-17T11:56:00.148316+0000","last_fullsized":"2024-02-17T11:56:00.148316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:37:52.866379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.682503+0000","last_change":"2024-02-17T11:55:58.679274+0000","last_active":"2024-02-17T11:55:58.682503+0000","last_peered":"2024-02-17T11:55:58.682503+0000","last_clean":"2024-02-17T11:55:58.682503+0000","last_became_active":"2024-02-17T11:55:58.678945+0000","last_became_peered":"2024-02-17T11:55:58.678945+0000","last_unstale":"2024-02-17T11:55:58.682503+0000","last_undegraded":"2024-02-17T11:55:58.682503+0000","last_fullsized":"2024-02-17T11:55:58.682503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:23:25.998999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.147530+0000","last_change":"2024-02-17T11:55:57.672732+0000","last_active":"2024-02-17T11:56:00.147530+0000","last_peered":"2024-02-17T11:56:00.147530+0000","last_clean":"2024-02-17T11:56:00.147530+0000","last_became_active":"2024-02-17T11:55:57.672409+0000","last_became_peered":"2024-02-17T11:55:57.672409+0000","last_unstale":"2024-02-17T11:56:00.147530+0000","last_undegraded":"2024-02-17T11:56:00.147530+0000","last_fullsized":"2024-02-17T11:56:00.147530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:07:31.556508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.658675+0000","last_change":"2024-02-17T11:55:58.689983+0000","last_active":"2024-02-17T11:55:59.658675+0000","last_peered":"2024-02-17T11:55:59.658675+0000","last_clean":"2024-02-17T11:55:59.658675+0000","last_became_active":"2024-02-17T11:55:58.689629+0000","last_became_peered":"2024-02-17T11:55:58.689629+0000","last_unstale":"2024-02-17T11:55:59.658675+0000","last_undegraded":"2024-02-17T11:55:59.658675+0000","last_fullsized":"2024-02-17T11:55:59.658675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:12.321746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'19","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:03.542259+0000","last_change":"2024-02-17T11:55:57.673273+0000","last_active":"2024-02-17T11:56:03.542259+0000","last_peered":"2024-02-17T11:56:03.542259+0000","last_clean":"2024-02-17T11:56:03.542259+0000","last_became_active":"2024-02-17T11:55:57.672865+0000","last_became_peered":"2024-02-17T11:55:57.672865+0000","last_unstale":"2024-02-17T11:56:03.542259+0000","last_undegraded":"2024-02-17T11:56:03.542259+0000","last_fullsized":"2024-02-17T11:56:03.542259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:31:01.593798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38193,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":23,"num_write":20,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.145364+0000","last_change":"2024-02-17T11:55:58.669296+0000","last_active":"2024-02-17T11:56:00.145364+0000","last_peered":"2024-02-17T11:56:00.145364+0000","last_clean":"2024-02-17T11:56:00.145364+0000","last_became_active":"2024-02-17T11:55:58.668927+0000","last_became_peered":"2024-02-17T11:55:58.668927+0000","last_unstale":"2024-02-17T11:56:00.145364+0000","last_undegraded":"2024-02-17T11:56:00.145364+0000","last_fullsized":"2024-02-17T11:56:00.145364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:25.177905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.154885+0000","last_change":"2024-02-17T11:55:57.674523+0000","last_active":"2024-02-17T11:56:01.154885+0000","last_peered":"2024-02-17T11:56:01.154885+0000","last_clean":"2024-02-17T11:56:01.154885+0000","last_became_active":"2024-02-17T11:55:57.672375+0000","last_became_peered":"2024-02-17T11:55:57.672375+0000","last_unstale":"2024-02-17T11:56:01.154885+0000","last_undegraded":"2024-02-17T11:56:01.154885+0000","last_fullsized":"2024-02-17T11:56:01.154885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:15:21.568434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657583+0000","last_change":"2024-02-17T11:55:58.682106+0000","last_active":"2024-02-17T11:55:59.657583+0000","last_peered":"2024-02-17T11:55:59.657583+0000","last_clean":"2024-02-17T11:55:59.657583+0000","last_became_active":"2024-02-17T11:55:58.681420+0000","last_became_peered":"2024-02-17T11:55:58.681420+0000","last_unstale":"2024-02-17T11:55:59.657583+0000","last_undegraded":"2024-02-17T11:55:59.657583+0000","last_fullsized":"2024-02-17T11:55:59.657583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:23:00.521690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.150460+0000","last_change":"2024-02-17T11:55:57.675161+0000","last_active":"2024-02-17T11:56:00.150460+0000","last_peered":"2024-02-17T11:56:00.150460+0000","last_clean":"2024-02-17T11:56:00.150460+0000","last_became_active":"2024-02-17T11:55:57.674776+0000","last_became_peered":"2024-02-17T11:55:57.674776+0000","last_unstale":"2024-02-17T11:56:00.150460+0000","last_undegraded":"2024-02-17T11:56:00.150460+0000","last_fullsized":"2024-02-17T11:56:00.150460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:25:59.695014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143497+0000","last_change":"2024-02-17T11:55:58.679319+0000","last_active":"2024-02-17T11:56:00.143497+0000","last_peered":"2024-02-17T11:56:00.143497+0000","last_clean":"2024-02-17T11:56:00.143497+0000","last_became_active":"2024-02-17T11:55:58.679001+0000","last_became_peered":"2024-02-17T11:55:58.679001+0000","last_unstale":"2024-02-17T11:56:00.143497+0000","last_undegraded":"2024-02-17T11:56:00.143497+0000","last_fullsized":"2024-02-17T11:56:00.143497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:36:48.493338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.154733+0000","last_change":"2024-02-17T11:55:57.670027+0000","last_active":"2024-02-17T11:56:01.154733+0000","last_peered":"2024-02-17T11:56:01.154733+0000","last_clean":"2024-02-17T11:56:01.154733+0000","last_became_active":"2024-02-17T11:55:57.669692+0000","last_became_peered":"2024-02-17T11:55:57.669692+0000","last_unstale":"2024-02-17T11:56:01.154733+0000","last_undegraded":"2024-02-17T11:56:01.154733+0000","last_fullsized":"2024-02-17T11:56:01.154733+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:44:01.001640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143029+0000","last_change":"2024-02-17T11:55:58.672811+0000","last_active":"2024-02-17T11:56:00.143029+0000","last_peered":"2024-02-17T11:56:00.143029+0000","last_clean":"2024-02-17T11:56:00.143029+0000","last_became_active":"2024-02-17T11:55:58.672469+0000","last_became_peered":"2024-02-17T11:55:58.672469+0000","last_unstale":"2024-02-17T11:56:00.143029+0000","last_undegraded":"2024-02-17T11:56:00.143029+0000","last_fullsized":"2024-02-17T11:56:00.143029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:07.451422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'10","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148194+0000","last_change":"2024-02-17T11:55:57.668031+0000","last_active":"2024-02-17T11:56:00.148194+0000","last_peered":"2024-02-17T11:56:00.148194+0000","last_clean":"2024-02-17T11:56:00.148194+0000","last_became_active":"2024-02-17T11:55:57.667344+0000","last_became_peered":"2024-02-17T11:55:57.667344+0000","last_unstale":"2024-02-17T11:56:00.148194+0000","last_undegraded":"2024-02-17T11:56:00.148194+0000","last_fullsized":"2024-02-17T11:56:00.148194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:14:20.773902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143827+0000","last_change":"2024-02-17T11:55:58.682013+0000","last_active":"2024-02-17T11:56:00.143827+0000","last_peered":"2024-02-17T11:56:00.143827+0000","last_clean":"2024-02-17T11:56:00.143827+0000","last_became_active":"2024-02-17T11:55:58.681503+0000","last_became_peered":"2024-02-17T11:55:58.681503+0000","last_unstale":"2024-02-17T11:56:00.143827+0000","last_undegraded":"2024-02-17T11:56:00.143827+0000","last_fullsized":"2024-02-17T11:56:00.143827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:24:50.364090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.146816+0000","last_change":"2024-02-17T11:55:57.666725+0000","last_active":"2024-02-17T11:56:00.146816+0000","last_peered":"2024-02-17T11:56:00.146816+0000","last_clean":"2024-02-17T11:56:00.146816+0000","last_became_active":"2024-02-17T11:55:57.666339+0000","last_became_peered":"2024-02-17T11:55:57.666339+0000","last_unstale":"2024-02-17T11:56:00.146816+0000","last_undegraded":"2024-02-17T11:56:00.146816+0000","last_fullsized":"2024-02-17T11:56:00.146816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:22:47.324016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.147908+0000","last_change":"2024-02-17T11:55:58.675995+0000","last_active":"2024-02-17T11:56:00.147908+0000","last_peered":"2024-02-17T11:56:00.147908+0000","last_clean":"2024-02-17T11:56:00.147908+0000","last_became_active":"2024-02-17T11:55:58.675655+0000","last_became_peered":"2024-02-17T11:55:58.675655+0000","last_unstale":"2024-02-17T11:56:00.147908+0000","last_undegraded":"2024-02-17T11:56:00.147908+0000","last_fullsized":"2024-02-17T11:56:00.147908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:12.465300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.896394+0000","last_change":"2024-02-17T11:55:57.669500+0000","last_active":"2024-02-17T11:55:59.896394+0000","last_peered":"2024-02-17T11:55:59.896394+0000","last_clean":"2024-02-17T11:55:59.896394+0000","last_became_active":"2024-02-17T11:55:57.669148+0000","last_became_peered":"2024-02-17T11:55:57.669148+0000","last_unstale":"2024-02-17T11:55:59.896394+0000","last_undegraded":"2024-02-17T11:55:59.896394+0000","last_fullsized":"2024-02-17T11:55:59.896394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:33.821266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140787+0000","last_change":"2024-02-17T11:55:58.678336+0000","last_active":"2024-02-17T11:56:00.140787+0000","last_peered":"2024-02-17T11:56:00.140787+0000","last_clean":"2024-02-17T11:56:00.140787+0000","last_became_active":"2024-02-17T11:55:58.677967+0000","last_became_peered":"2024-02-17T11:55:58.677967+0000","last_unstale":"2024-02-17T11:56:00.140787+0000","last_undegraded":"2024-02-17T11:56:00.140787+0000","last_fullsized":"2024-02-17T11:56:00.140787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:31:39.406673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143517+0000","last_change":"2024-02-17T11:55:57.674810+0000","last_active":"2024-02-17T11:56:00.143517+0000","last_peered":"2024-02-17T11:56:00.143517+0000","last_clean":"2024-02-17T11:56:00.143517+0000","last_became_active":"2024-02-17T11:55:57.674356+0000","last_became_peered":"2024-02-17T11:55:57.674356+0000","last_unstale":"2024-02-17T11:56:00.143517+0000","last_undegraded":"2024-02-17T11:56:00.143517+0000","last_fullsized":"2024-02-17T11:56:00.143517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:25:57.879239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140758+0000","last_change":"2024-02-17T11:55:58.680928+0000","last_active":"2024-02-17T11:56:00.140758+0000","last_peered":"2024-02-17T11:56:00.140758+0000","last_clean":"2024-02-17T11:56:00.140758+0000","last_became_active":"2024-02-17T11:55:58.680521+0000","last_became_peered":"2024-02-17T11:55:58.680521+0000","last_unstale":"2024-02-17T11:56:00.140758+0000","last_undegraded":"2024-02-17T11:56:00.140758+0000","last_fullsized":"2024-02-17T11:56:00.140758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:15:15.739127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.144262+0000","last_change":"2024-02-17T11:55:58.674771+0000","last_active":"2024-02-17T11:56:00.144262+0000","last_peered":"2024-02-17T11:56:00.144262+0000","last_clean":"2024-02-17T11:56:00.144262+0000","last_became_active":"2024-02-17T11:55:58.674450+0000","last_became_peered":"2024-02-17T11:55:58.674450+0000","last_unstale":"2024-02-17T11:56:00.144262+0000","last_undegraded":"2024-02-17T11:56:00.144262+0000","last_fullsized":"2024-02-17T11:56:00.144262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.031576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.155863+0000","last_change":"2024-02-17T11:55:57.672868+0000","last_active":"2024-02-17T11:56:01.155863+0000","last_peered":"2024-02-17T11:56:01.155863+0000","last_clean":"2024-02-17T11:56:01.155863+0000","last_became_active":"2024-02-17T11:55:57.672584+0000","last_became_peered":"2024-02-17T11:55:57.672584+0000","last_unstale":"2024-02-17T11:56:01.155863+0000","last_undegraded":"2024-02-17T11:56:01.155863+0000","last_fullsized":"2024-02-17T11:56:01.155863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:03:20.307036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140579+0000","last_change":"2024-02-17T11:55:58.692594+0000","last_active":"2024-02-17T11:56:00.140579+0000","last_peered":"2024-02-17T11:56:00.140579+0000","last_clean":"2024-02-17T11:56:00.140579+0000","last_became_active":"2024-02-17T11:55:58.692258+0000","last_became_peered":"2024-02-17T11:55:58.692258+0000","last_unstale":"2024-02-17T11:56:00.140579+0000","last_undegraded":"2024-02-17T11:56:00.140579+0000","last_fullsized":"2024-02-17T11:56:00.140579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:28.497939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.147183+0000","last_change":"2024-02-17T11:55:57.669456+0000","last_active":"2024-02-17T11:56:00.147183+0000","last_peered":"2024-02-17T11:56:00.147183+0000","last_clean":"2024-02-17T11:56:00.147183+0000","last_became_active":"2024-02-17T11:55:57.669067+0000","last_became_peered":"2024-02-17T11:55:57.669067+0000","last_unstale":"2024-02-17T11:56:00.147183+0000","last_undegraded":"2024-02-17T11:56:00.147183+0000","last_fullsized":"2024-02-17T11:56:00.147183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:12.352296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143175+0000","last_change":"2024-02-17T11:54:46.125844+0000","last_active":"2024-02-17T11:56:00.143175+0000","last_peered":"2024-02-17T11:56:00.143175+0000","last_clean":"2024-02-17T11:56:00.143175+0000","last_became_active":"2024-02-17T11:54:46.125387+0000","last_became_peered":"2024-02-17T11:54:46.125387+0000","last_unstale":"2024-02-17T11:56:00.143175+0000","last_undegraded":"2024-02-17T11:56:00.143175+0000","last_fullsized":"2024-02-17T11:56:00.143175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_clean_scrub_stamp":"2024-02-17T11:51:21.581371+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:53:27.219974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657088+0000","last_change":"2024-02-17T11:55:58.681623+0000","last_active":"2024-02-17T11:55:59.657088+0000","last_peered":"2024-02-17T11:55:59.657088+0000","last_clean":"2024-02-17T11:55:59.657088+0000","last_became_active":"2024-02-17T11:55:58.681275+0000","last_became_peered":"2024-02-17T11:55:58.681275+0000","last_unstale":"2024-02-17T11:55:59.657088+0000","last_undegraded":"2024-02-17T11:55:59.657088+0000","last_fullsized":"2024-02-17T11:55:59.657088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:20.685634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.150096+0000","last_change":"2024-02-17T11:55:57.679555+0000","last_active":"2024-02-17T11:56:00.150096+0000","last_peered":"2024-02-17T11:56:00.150096+0000","last_clean":"2024-02-17T11:56:00.150096+0000","last_became_active":"2024-02-17T11:55:57.679241+0000","last_became_peered":"2024-02-17T11:55:57.679241+0000","last_unstale":"2024-02-17T11:56:00.150096+0000","last_undegraded":"2024-02-17T11:56:00.150096+0000","last_fullsized":"2024-02-17T11:56:00.150096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:37.482582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148424+0000","last_change":"2024-02-17T11:55:57.679711+0000","last_active":"2024-02-17T11:56:00.148424+0000","last_peered":"2024-02-17T11:56:00.148424+0000","last_clean":"2024-02-17T11:56:00.148424+0000","last_became_active":"2024-02-17T11:55:57.679415+0000","last_became_peered":"2024-02-17T11:55:57.679415+0000","last_unstale":"2024-02-17T11:56:00.148424+0000","last_undegraded":"2024-02-17T11:56:00.148424+0000","last_fullsized":"2024-02-17T11:56:00.148424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:13.095530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657267+0000","last_change":"2024-02-17T11:55:58.670275+0000","last_active":"2024-02-17T11:55:59.657267+0000","last_peered":"2024-02-17T11:55:59.657267+0000","last_clean":"2024-02-17T11:55:59.657267+0000","last_became_active":"2024-02-17T11:55:58.669926+0000","last_became_peered":"2024-02-17T11:55:58.669926+0000","last_unstale":"2024-02-17T11:55:59.657267+0000","last_undegraded":"2024-02-17T11:55:59.657267+0000","last_fullsized":"2024-02-17T11:55:59.657267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:48:35.078389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.156432+0000","last_change":"2024-02-17T11:55:57.670364+0000","last_active":"2024-02-17T11:56:01.156432+0000","last_peered":"2024-02-17T11:56:01.156432+0000","last_clean":"2024-02-17T11:56:01.156432+0000","last_became_active":"2024-02-17T11:55:57.669977+0000","last_became_peered":"2024-02-17T11:55:57.669977+0000","last_unstale":"2024-02-17T11:56:01.156432+0000","last_undegraded":"2024-02-17T11:56:01.156432+0000","last_fullsized":"2024-02-17T11:56:01.156432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:42:13.820390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139996+0000","last_change":"2024-02-17T11:55:58.681587+0000","last_active":"2024-02-17T11:56:00.139996+0000","last_peered":"2024-02-17T11:56:00.139996+0000","last_clean":"2024-02-17T11:56:00.139996+0000","last_became_active":"2024-02-17T11:55:58.680024+0000","last_became_peered":"2024-02-17T11:55:58.680024+0000","last_unstale":"2024-02-17T11:56:00.139996+0000","last_undegraded":"2024-02-17T11:56:00.139996+0000","last_fullsized":"2024-02-17T11:56:00.139996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:31:40.399439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.149961+0000","last_change":"2024-02-17T11:55:57.674618+0000","last_active":"2024-02-17T11:56:00.149961+0000","last_peered":"2024-02-17T11:56:00.149961+0000","last_clean":"2024-02-17T11:56:00.149961+0000","last_became_active":"2024-02-17T11:55:57.674246+0000","last_became_peered":"2024-02-17T11:55:57.674246+0000","last_unstale":"2024-02-17T11:56:00.149961+0000","last_undegraded":"2024-02-17T11:56:00.149961+0000","last_fullsized":"2024-02-17T11:56:00.149961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:42:26.681061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.681851+0000","last_change":"2024-02-17T11:55:58.680320+0000","last_active":"2024-02-17T11:55:58.681851+0000","last_peered":"2024-02-17T11:55:58.681851+0000","last_clean":"2024-02-17T11:55:58.681851+0000","last_became_active":"2024-02-17T11:55:58.680005+0000","last_became_peered":"2024-02-17T11:55:58.680005+0000","last_unstale":"2024-02-17T11:55:58.681851+0000","last_undegraded":"2024-02-17T11:55:58.681851+0000","last_fullsized":"2024-02-17T11:55:58.681851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:54:49.976680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139356+0000","last_change":"2024-02-17T11:55:57.667998+0000","last_active":"2024-02-17T11:56:00.139356+0000","last_peered":"2024-02-17T11:56:00.139356+0000","last_clean":"2024-02-17T11:56:00.139356+0000","last_became_active":"2024-02-17T11:55:57.667688+0000","last_became_peered":"2024-02-17T11:55:57.667688+0000","last_unstale":"2024-02-17T11:56:00.139356+0000","last_undegraded":"2024-02-17T11:56:00.139356+0000","last_fullsized":"2024-02-17T11:56:00.139356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:07:19.782879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.142836+0000","last_change":"2024-02-17T11:55:58.676644+0000","last_active":"2024-02-17T11:56:00.142836+0000","last_peered":"2024-02-17T11:56:00.142836+0000","last_clean":"2024-02-17T11:56:00.142836+0000","last_became_active":"2024-02-17T11:55:58.674670+0000","last_became_peered":"2024-02-17T11:55:58.674670+0000","last_unstale":"2024-02-17T11:56:00.142836+0000","last_undegraded":"2024-02-17T11:56:00.142836+0000","last_fullsized":"2024-02-17T11:56:00.142836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:43:22.746537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.145300+0000","last_change":"2024-02-17T11:55:57.671330+0000","last_active":"2024-02-17T11:56:00.145300+0000","last_peered":"2024-02-17T11:56:00.145300+0000","last_clean":"2024-02-17T11:56:00.145300+0000","last_became_active":"2024-02-17T11:55:57.670902+0000","last_became_peered":"2024-02-17T11:55:57.670902+0000","last_unstale":"2024-02-17T11:56:00.145300+0000","last_undegraded":"2024-02-17T11:56:00.145300+0000","last_fullsized":"2024-02-17T11:56:00.145300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:22:14.770339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148553+0000","last_change":"2024-02-17T11:55:58.676417+0000","last_active":"2024-02-17T11:56:00.148553+0000","last_peered":"2024-02-17T11:56:00.148553+0000","last_clean":"2024-02-17T11:56:00.148553+0000","last_became_active":"2024-02-17T11:55:58.675960+0000","last_became_peered":"2024-02-17T11:55:58.675960+0000","last_unstale":"2024-02-17T11:56:00.148553+0000","last_undegraded":"2024-02-17T11:56:00.148553+0000","last_fullsized":"2024-02-17T11:56:00.148553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:22:18.566872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.662655+0000","last_change":"2024-02-17T11:55:57.672668+0000","last_active":"2024-02-17T11:55:59.662655+0000","last_peered":"2024-02-17T11:55:59.662655+0000","last_clean":"2024-02-17T11:55:59.662655+0000","last_became_active":"2024-02-17T11:55:57.672363+0000","last_became_peered":"2024-02-17T11:55:57.672363+0000","last_unstale":"2024-02-17T11:55:59.662655+0000","last_undegraded":"2024-02-17T11:55:59.662655+0000","last_fullsized":"2024-02-17T11:55:59.662655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:17:25.878564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140241+0000","last_change":"2024-02-17T11:55:58.690030+0000","last_active":"2024-02-17T11:56:00.140241+0000","last_peered":"2024-02-17T11:56:00.140241+0000","last_clean":"2024-02-17T11:56:00.140241+0000","last_became_active":"2024-02-17T11:55:58.689687+0000","last_became_peered":"2024-02-17T11:55:58.689687+0000","last_unstale":"2024-02-17T11:56:00.140241+0000","last_undegraded":"2024-02-17T11:56:00.140241+0000","last_fullsized":"2024-02-17T11:56:00.140241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:33:43.175234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148085+0000","last_change":"2024-02-17T11:55:57.679734+0000","last_active":"2024-02-17T11:56:00.148085+0000","last_peered":"2024-02-17T11:56:00.148085+0000","last_clean":"2024-02-17T11:56:00.148085+0000","last_became_active":"2024-02-17T11:55:57.679328+0000","last_became_peered":"2024-02-17T11:55:57.679328+0000","last_unstale":"2024-02-17T11:56:00.148085+0000","last_undegraded":"2024-02-17T11:56:00.148085+0000","last_fullsized":"2024-02-17T11:56:00.148085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:50:09.813482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143825+0000","last_change":"2024-02-17T11:55:58.688360+0000","last_active":"2024-02-17T11:56:00.143825+0000","last_peered":"2024-02-17T11:56:00.143825+0000","last_clean":"2024-02-17T11:56:00.143825+0000","last_became_active":"2024-02-17T11:55:58.687994+0000","last_became_peered":"2024-02-17T11:55:58.687994+0000","last_unstale":"2024-02-17T11:56:00.143825+0000","last_undegraded":"2024-02-17T11:56:00.143825+0000","last_fullsized":"2024-02-17T11:56:00.143825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:01:02.163238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.156019+0000","last_change":"2024-02-17T11:55:57.675977+0000","last_active":"2024-02-17T11:56:01.156019+0000","last_peered":"2024-02-17T11:56:01.156019+0000","last_clean":"2024-02-17T11:56:01.156019+0000","last_became_active":"2024-02-17T11:55:57.675250+0000","last_became_peered":"2024-02-17T11:55:57.675250+0000","last_unstale":"2024-02-17T11:56:01.156019+0000","last_undegraded":"2024-02-17T11:56:01.156019+0000","last_fullsized":"2024-02-17T11:56:01.156019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:11:27.611692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143929+0000","last_change":"2024-02-17T11:55:58.688593+0000","last_active":"2024-02-17T11:56:00.143929+0000","last_peered":"2024-02-17T11:56:00.143929+0000","last_clean":"2024-02-17T11:56:00.143929+0000","last_became_active":"2024-02-17T11:55:58.688163+0000","last_became_peered":"2024-02-17T11:55:58.688163+0000","last_unstale":"2024-02-17T11:56:00.143929+0000","last_undegraded":"2024-02-17T11:56:00.143929+0000","last_fullsized":"2024-02-17T11:56:00.143929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:54.597116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.144164+0000","last_change":"2024-02-17T11:55:57.676860+0000","last_active":"2024-02-17T11:56:00.144164+0000","last_peered":"2024-02-17T11:56:00.144164+0000","last_clean":"2024-02-17T11:56:00.144164+0000","last_became_active":"2024-02-17T11:55:57.676534+0000","last_became_peered":"2024-02-17T11:55:57.676534+0000","last_unstale":"2024-02-17T11:56:00.144164+0000","last_undegraded":"2024-02-17T11:56:00.144164+0000","last_fullsized":"2024-02-17T11:56:00.144164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:23:47.726264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143305+0000","last_change":"2024-02-17T11:55:58.679254+0000","last_active":"2024-02-17T11:56:00.143305+0000","last_peered":"2024-02-17T11:56:00.143305+0000","last_clean":"2024-02-17T11:56:00.143305+0000","last_became_active":"2024-02-17T11:55:58.678874+0000","last_became_peered":"2024-02-17T11:55:58.678874+0000","last_unstale":"2024-02-17T11:56:00.143305+0000","last_undegraded":"2024-02-17T11:56:00.143305+0000","last_fullsized":"2024-02-17T11:56:00.143305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:55:44.769818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657137+0000","last_change":"2024-02-17T11:55:57.672274+0000","last_active":"2024-02-17T11:55:59.657137+0000","last_peered":"2024-02-17T11:55:59.657137+0000","last_clean":"2024-02-17T11:55:59.657137+0000","last_became_active":"2024-02-17T11:55:57.671932+0000","last_became_peered":"2024-02-17T11:55:57.671932+0000","last_unstale":"2024-02-17T11:55:59.657137+0000","last_undegraded":"2024-02-17T11:55:59.657137+0000","last_fullsized":"2024-02-17T11:55:59.657137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:20:41.973595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143601+0000","last_change":"2024-02-17T11:55:58.672985+0000","last_active":"2024-02-17T11:56:00.143601+0000","last_peered":"2024-02-17T11:56:00.143601+0000","last_clean":"2024-02-17T11:56:00.143601+0000","last_became_active":"2024-02-17T11:55:58.672622+0000","last_became_peered":"2024-02-17T11:55:58.672622+0000","last_unstale":"2024-02-17T11:56:00.143601+0000","last_undegraded":"2024-02-17T11:56:00.143601+0000","last_fullsized":"2024-02-17T11:56:00.143601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:45.555827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148641+0000","last_change":"2024-02-17T11:55:58.676127+0000","last_active":"2024-02-17T11:56:00.148641+0000","last_peered":"2024-02-17T11:56:00.148641+0000","last_clean":"2024-02-17T11:56:00.148641+0000","last_became_active":"2024-02-17T11:55:58.675784+0000","last_became_peered":"2024-02-17T11:55:58.675784+0000","last_unstale":"2024-02-17T11:56:00.148641+0000","last_undegraded":"2024-02-17T11:56:00.148641+0000","last_fullsized":"2024-02-17T11:56:00.148641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:56:32.648803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.662821+0000","last_change":"2024-02-17T11:55:58.682438+0000","last_active":"2024-02-17T11:55:59.662821+0000","last_peered":"2024-02-17T11:55:59.662821+0000","last_clean":"2024-02-17T11:55:59.662821+0000","last_became_active":"2024-02-17T11:55:58.682067+0000","last_became_peered":"2024-02-17T11:55:58.682067+0000","last_unstale":"2024-02-17T11:55:59.662821+0000","last_undegraded":"2024-02-17T11:55:59.662821+0000","last_fullsized":"2024-02-17T11:55:59.662821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:10:22.906450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.662484+0000","last_change":"2024-02-17T11:55:58.686671+0000","last_active":"2024-02-17T11:55:59.662484+0000","last_peered":"2024-02-17T11:55:59.662484+0000","last_clean":"2024-02-17T11:55:59.662484+0000","last_became_active":"2024-02-17T11:55:58.686315+0000","last_became_peered":"2024-02-17T11:55:58.686315+0000","last_unstale":"2024-02-17T11:55:59.662484+0000","last_undegraded":"2024-02-17T11:55:59.662484+0000","last_fullsized":"2024-02-17T11:55:59.662484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:55:40.864847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.684709+0000","last_change":"2024-02-17T11:55:58.684227+0000","last_active":"2024-02-17T11:55:58.684709+0000","last_peered":"2024-02-17T11:55:58.684709+0000","last_clean":"2024-02-17T11:55:58.684709+0000","last_became_active":"2024-02-17T11:55:58.683875+0000","last_became_peered":"2024-02-17T11:55:58.683875+0000","last_unstale":"2024-02-17T11:55:58.684709+0000","last_undegraded":"2024-02-17T11:55:58.684709+0000","last_fullsized":"2024-02-17T11:55:58.684709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:14:59.935238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657415+0000","last_change":"2024-02-17T11:55:58.685964+0000","last_active":"2024-02-17T11:55:59.657415+0000","last_peered":"2024-02-17T11:55:59.657415+0000","last_clean":"2024-02-17T11:55:59.657415+0000","last_became_active":"2024-02-17T11:55:58.685350+0000","last_became_peered":"2024-02-17T11:55:58.685350+0000","last_unstale":"2024-02-17T11:55:59.657415+0000","last_undegraded":"2024-02-17T11:55:59.657415+0000","last_fullsized":"2024-02-17T11:55:59.657415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:22:34.600604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148280+0000","last_change":"2024-02-17T11:55:58.671291+0000","last_active":"2024-02-17T11:56:00.148280+0000","last_peered":"2024-02-17T11:56:00.148280+0000","last_clean":"2024-02-17T11:56:00.148280+0000","last_became_active":"2024-02-17T11:55:58.670925+0000","last_became_peered":"2024-02-17T11:55:58.670925+0000","last_unstale":"2024-02-17T11:56:00.148280+0000","last_undegraded":"2024-02-17T11:56:00.148280+0000","last_fullsized":"2024-02-17T11:56:00.148280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:26:03.120976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.662972+0000","last_change":"2024-02-17T11:55:58.673122+0000","last_active":"2024-02-17T11:55:59.662972+0000","last_peered":"2024-02-17T11:55:59.662972+0000","last_clean":"2024-02-17T11:55:59.662972+0000","last_became_active":"2024-02-17T11:55:58.672747+0000","last_became_peered":"2024-02-17T11:55:58.672747+0000","last_unstale":"2024-02-17T11:55:59.662972+0000","last_undegraded":"2024-02-17T11:55:59.662972+0000","last_fullsized":"2024-02-17T11:55:59.662972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:55:15.064073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139833+0000","last_change":"2024-02-17T11:55:58.681276+0000","last_active":"2024-02-17T11:56:00.139833+0000","last_peered":"2024-02-17T11:56:00.139833+0000","last_clean":"2024-02-17T11:56:00.139833+0000","last_became_active":"2024-02-17T11:55:58.680889+0000","last_became_peered":"2024-02-17T11:55:58.680889+0000","last_unstale":"2024-02-17T11:56:00.139833+0000","last_undegraded":"2024-02-17T11:56:00.139833+0000","last_fullsized":"2024-02-17T11:56:00.139833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:19:34.330288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143934+0000","last_change":"2024-02-17T11:55:58.667475+0000","last_active":"2024-02-17T11:56:00.143934+0000","last_peered":"2024-02-17T11:56:00.143934+0000","last_clean":"2024-02-17T11:56:00.143934+0000","last_became_active":"2024-02-17T11:55:58.666577+0000","last_became_peered":"2024-02-17T11:55:58.666577+0000","last_unstale":"2024-02-17T11:56:00.143934+0000","last_undegraded":"2024-02-17T11:56:00.143934+0000","last_fullsized":"2024-02-17T11:56:00.143934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:51:35.732035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140151+0000","last_change":"2024-02-17T11:55:58.674557+0000","last_active":"2024-02-17T11:56:00.140151+0000","last_peered":"2024-02-17T11:56:00.140151+0000","last_clean":"2024-02-17T11:56:00.140151+0000","last_became_active":"2024-02-17T11:55:58.673874+0000","last_became_peered":"2024-02-17T11:55:58.673874+0000","last_unstale":"2024-02-17T11:56:00.140151+0000","last_undegraded":"2024-02-17T11:56:00.140151+0000","last_fullsized":"2024-02-17T11:56:00.140151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:18:00.812489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.658591+0000","last_change":"2024-02-17T11:55:58.692725+0000","last_active":"2024-02-17T11:55:59.658591+0000","last_peered":"2024-02-17T11:55:59.658591+0000","last_clean":"2024-02-17T11:55:59.658591+0000","last_became_active":"2024-02-17T11:55:58.690880+0000","last_became_peered":"2024-02-17T11:55:58.690880+0000","last_unstale":"2024-02-17T11:55:59.658591+0000","last_undegraded":"2024-02-17T11:55:59.658591+0000","last_fullsized":"2024-02-17T11:55:59.658591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:03:44.724385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657443+0000","last_change":"2024-02-17T11:55:58.679377+0000","last_active":"2024-02-17T11:55:59.657443+0000","last_peered":"2024-02-17T11:55:59.657443+0000","last_clean":"2024-02-17T11:55:59.657443+0000","last_became_active":"2024-02-17T11:55:58.679041+0000","last_became_peered":"2024-02-17T11:55:58.679041+0000","last_unstale":"2024-02-17T11:55:59.657443+0000","last_undegraded":"2024-02-17T11:55:59.657443+0000","last_fullsized":"2024-02-17T11:55:59.657443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:16:37.396424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.147521+0000","last_change":"2024-02-17T11:55:58.680665+0000","last_active":"2024-02-17T11:56:00.147521+0000","last_peered":"2024-02-17T11:56:00.147521+0000","last_clean":"2024-02-17T11:56:00.147521+0000","last_became_active":"2024-02-17T11:55:58.680311+0000","last_became_peered":"2024-02-17T11:55:58.680311+0000","last_unstale":"2024-02-17T11:56:00.147521+0000","last_undegraded":"2024-02-17T11:56:00.147521+0000","last_fullsized":"2024-02-17T11:56:00.147521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:19:11.455471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284587,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41691,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":9,"num_read_kb":25,"num_write":124,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":69743,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":1039070,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.314},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.40799999999999997}]},{"osd":2,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.60199999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36099999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.374}]},{"osd":5,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65900000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.49399999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.41499999999999998}]},{"osd":8,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61699999999999999}]},{"osd":9,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.46500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808855,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":439305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.78200000000000003}]},{"osd":1,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55900000000000005}]},{"osd":2,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.45800000000000002}]},{"osd":3,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.379},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.35499999999999998}]},{"osd":4,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67500000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.434},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.65900000000000003}]},{"osd":7,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.63400000000000001}]},{"osd":8,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.499},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.752}]},{"osd":10,"last update":"Sat Feb 17 11:55:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.69099999999999995}]}]},{"osd":8,"up_from":59,"seq":253403070492,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":430555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44},{"interface":"front","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.38100000000000001}]},{"osd":1,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76900000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.35899999999999999}]},{"osd":2,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.44500000000000001,"15min":0.435},"min":{"1min":0.311,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.308,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78600000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48199999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.58499999999999996,"15min":0.59499999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69899999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.376,"5min":0.372,"15min":0.372},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.52700000000000002}]},{"osd":4,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73799999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.622}]},{"osd":5,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76200000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56799999999999995}]},{"osd":6,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.67600000000000005,"15min":0.68500000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.5,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.68999999999999995}]},{"osd":9,"last update":"Sat Feb 17 11:55:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.73799999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:55:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332129,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":1020901,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.36699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39600000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41699999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.81899999999999995,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.39000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81499999999999995,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.624}]},{"osd":3,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55600000000000005}]},{"osd":4,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52000000000000002,"15min":0.51500000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.81899999999999995}]},{"osd":5,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94399999999999995,"5min":1.333,"15min":1.333},"last":0.746},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.95399999999999996,"5min":1.486,"15min":1.486},"last":0.69599999999999995}]},{"osd":6,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.85699999999999998}]},{"osd":8,"last update":"Sat Feb 17 11:55:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98599999999999999,"5min":1.446,"15min":1.446},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89600000000000002,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.77900000000000003}]},{"osd":9,"last update":"Sat Feb 17 11:55:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.92000000000000004}]},{"osd":10,"last update":"Sat Feb 17 11:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.94299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.23499999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.51500000000000001,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61699999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.439}]},{"osd":1,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.502}]},{"osd":2,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.42699999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78300000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82699999999999996,"5min":0.871,"15min":0.871},"last":0.624}]},{"osd":3,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.45900000000000002,"15min":0.45100000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.56299999999999994,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.42099999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59799999999999998,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.51900000000000002}]},{"osd":5,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61899999999999999,"15min":0.622},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82199999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66500000000000004,"15min":0.66000000000000003},"min":{"1min":0.57499999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90600000000000003,"5min":0.997,"15min":0.997},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.436,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75700000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.60399999999999998},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55800000000000005}]},{"osd":8,"last update":"Sat Feb 17 11:55:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67100000000000004}]},{"osd":9,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60599999999999998}]},{"osd":10,"last update":"Sat Feb 17 11:55:42 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.65900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.50700000000000001,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Feb 17 11:55:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53200000000000003,"15min":0.51700000000000002},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.71799999999999997}]},{"osd":3,"last update":"Sat Feb 17 11:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.56100000000000005,"15min":0.52300000000000002},"min":{"1min":0.42499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.52000000000000002},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58799999999999997}]},{"osd":4,"last update":"Sat Feb 17 11:55:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.42699999999999999,"15min":0.42099999999999999},"min":{"1min":0.29999999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67200000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.54400000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.59099999999999997,"15min":0.56799999999999995},"min":{"1min":0.44900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.60499999999999998,"15min":0.58499999999999996},"min":{"1min":0.51400000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.746}]},{"osd":6,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66500000000000004}]},{"osd":7,"last update":"Sat Feb 17 11:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54400000000000004,"15min":0.52500000000000002},"min":{"1min":0.40600000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.59799999999999998,"15min":0.58299999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.65000000000000002}]},{"osd":8,"last update":"Sat Feb 17 11:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.72899999999999998}]},{"osd":9,"last update":"Sat Feb 17 11:55:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.86299999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:55:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.84899999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":905216,"data_stored":430041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.47499999999999998,"15min":0.44700000000000001},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.48799999999999999,"15min":0.46700000000000003},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.40100000000000002}]},{"osd":2,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.54300000000000004,"15min":0.503},"min":{"1min":0.35899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.39100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60099999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:54:47 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.39400000000000002,"15min":0.379},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.44800000000000001,"15min":0.435},"min":{"1min":0.36299999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.747,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38}]},{"osd":4,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.52700000000000002,"15min":0.504},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.49399999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.433}]},{"osd":5,"last update":"Sat Feb 17 11:55:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60999999999999999,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60599999999999998,"15min":0.59399999999999997},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.53600000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.622}]},{"osd":7,"last update":"Sat Feb 17 11:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.501,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58399999999999996}]},{"osd":8,"last update":"Sat Feb 17 11:55:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70999999999999996,"15min":0.71099999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.69899999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.63500000000000001}]},{"osd":9,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65200000000000002}]},{"osd":10,"last update":"Sat Feb 17 11:55:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":440789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.49299999999999999},"min":{"1min":0.40400000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.51700000000000002,"15min":0.499},"min":{"1min":0.38500000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46100000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.48099999999999998},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.73599999999999999,"5min":0.751,"15min":0.751},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.51900000000000002,"15min":0.47699999999999998},"min":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85799999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70099999999999996}]},{"osd":3,"last update":"Sat Feb 17 11:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.44700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.755,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.35299999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65000000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Feb 17 11:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54000000000000004,"15min":0.52500000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.747},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56000000000000005,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79400000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.432,"15min":0.41999999999999998},"min":{"1min":0.30299999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.441,"15min":0.436},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.48199999999999998}]},{"osd":6,"last update":"Sat Feb 17 11:55:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79300000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.47099999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.501}]},{"osd":7,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63700000000000001}]},{"osd":8,"last update":"Sat Feb 17 11:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.42099999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73099999999999998}]},{"osd":9,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42899999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:55:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.76400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":438715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.34999999999999998,"15min":0.34100000000000003},"min":{"1min":0.27400000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.35199999999999998,"15min":0.34300000000000003},"min":{"1min":0.254,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.53700000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.30099999999999999}]},{"osd":1,"last update":"Sat Feb 17 11:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52500000000000002,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54100000000000004,"15min":0.53500000000000003},"min":{"1min":0.443,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64700000000000002}]},{"osd":2,"last update":"Sat Feb 17 11:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.57299999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78900000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63700000000000001}]},{"osd":4,"last update":"Sat Feb 17 11:55:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.46600000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.53900000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79200000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56899999999999995}]},{"osd":5,"last update":"Sat Feb 17 11:55:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.46400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52000000000000002}]},{"osd":6,"last update":"Sat Feb 17 11:55:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.45100000000000001,"15min":0.44},"min":{"1min":0.315,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.49199999999999999,"15min":0.47699999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.50800000000000001}]},{"osd":7,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59499999999999997,"15min":0.58399999999999996},"min":{"1min":0.47399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.53700000000000003}]},{"osd":8,"last update":"Sat Feb 17 11:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.49199999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97599999999999998,"5min":1.214,"15min":1.214},"last":0.622}]},{"osd":9,"last update":"Sat Feb 17 11:55:19 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.45000000000000001}]},{"osd":10,"last update":"Sat Feb 17 11:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":439165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.50700000000000001,"15min":0.497},"min":{"1min":0.41099999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41599999999999998}]},{"osd":1,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39600000000000002,"15min":0.38900000000000001},"min":{"1min":0.255,"5min":0.254,"15min":0.254},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.39900000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.61199999999999999,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.0279999999999996,"5min":7.0279999999999996,"15min":7.0279999999999996},"last":0.91600000000000004}]},{"osd":3,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78100000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69799999999999995}]},{"osd":5,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79800000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.57799999999999996,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.96199999999999997}]},{"osd":6,"last update":"Sat Feb 17 11:55:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57399999999999995,"15min":0.56799999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.62},{"interface":"front","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.73899999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.53400000000000003,"15min":0.51100000000000001},"min":{"1min":0.33100000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.373,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66500000000000004}]},{"osd":8,"last update":"Sat Feb 17 11:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.72299999999999998,"15min":0.67300000000000004},"min":{"1min":0.51900000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":6.9660000000000002,"5min":6.9660000000000002,"15min":6.9660000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.54400000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":1.0580000000000001}]},{"osd":9,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":1.028}]},{"osd":10,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":1.087},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":1020501,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.503,"15min":0.48699999999999999},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.50700000000000001,"15min":0.48299999999999998},"min":{"1min":0.41499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.75800000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.66800000000000004}]},{"osd":1,"last update":"Sat Feb 17 11:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.38500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.84799999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.52500000000000002,"15min":0.497},"min":{"1min":0.41499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.71099999999999997}]},{"osd":2,"last update":"Sat Feb 17 11:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.309,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.27100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67300000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.41199999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.40100000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.52600000000000002},"min":{"1min":0.42199999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73599999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat Feb 17 11:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57099999999999995,"15min":0.55400000000000005},"min":{"1min":0.39600000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80900000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.57699999999999996,"15min":0.56000000000000005},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.88100000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72799999999999998}]},{"osd":6,"last update":"Sat Feb 17 11:55:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81000000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.755}]},{"osd":7,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.45500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84399999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.64700000000000002,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77300000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68500000000000005}]},{"osd":8,"last update":"Sat Feb 17 11:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.98799999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65100000000000002,"15min":0.65900000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91800000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.52000000000000002}]},{"osd":9,"last update":"Sat Feb 17 11:55:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.78400000000000003}]},{"osd":10,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.82899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":40162,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T11:56:04.843 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-17T11:56:04.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.979+0000 7feb9b9f4640 1 -- 172.21.15.94:0/1639826922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb941000a0 msgr2=0x7feb941004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.979+0000 7feb9b9f4640 1 --2- 172.21.15.94:0/1639826922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb941000a0 0x7feb941004c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feb8c009ca0 tx=0x7feb8c02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.979+0000 7feb9b9f4640 1 -- 172.21.15.94:0/1639826922 shutdown_connections 2024-02-17T11:56:04.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.979+0000 7feb9b9f4640 1 --2- 172.21.15.94:0/1639826922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb94101c50 0x7feb941084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.979+0000 7feb9b9f4640 1 --2- 172.21.15.94:0/1639826922 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb94101290 0x7feb94101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.979+0000 7feb9b9f4640 1 --2- 172.21.15.94:0/1639826922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb941000a0 0x7feb941004c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.979+0000 7feb9b9f4640 1 -- 172.21.15.94:0/1639826922 >> 172.21.15.94:0/1639826922 conn(0x7feb940fb630 msgr2=0x7feb940fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:04.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.979+0000 7feb9b9f4640 1 -- 172.21.15.94:0/1639826922 shutdown_connections 2024-02-17T11:56:04.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.979+0000 7feb9b9f4640 1 -- 172.21.15.94:0/1639826922 wait complete. 2024-02-17T11:56:04.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.979+0000 7feb9b9f4640 1 Processor -- start 2024-02-17T11:56:04.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.979+0000 7feb9b9f4640 1 -- start start 2024-02-17T11:56:04.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb9b9f4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb941000a0 0x7feb941972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:04.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb9b9f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb94101290 0x7feb941977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:04.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb9b9f4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb94101c50 0x7feb9419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:04.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb9b9f4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9419ed50 con 0x7feb94101290 2024-02-17T11:56:04.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb9b9f4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9419eec0 con 0x7feb941000a0 2024-02-17T11:56:04.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb9b9f4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9419f080 con 0x7feb94101c50 2024-02-17T11:56:04.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb98f8f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb94101290 0x7feb941977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb98f8f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb94101290 0x7feb941977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47824/0 (socket says 172.21.15.94:47824) 2024-02-17T11:56:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb98f8f640 1 -- 172.21.15.94:0/2642884453 learned_addr learned my addr 172.21.15.94:0/2642884453 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb99f91640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb94101c50 0x7feb9419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb98f8f640 1 -- 172.21.15.94:0/2642884453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb94101c50 msgr2=0x7feb9419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb99790640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb941000a0 0x7feb941972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb98f8f640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb94101c50 0x7feb9419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb98f8f640 1 -- 172.21.15.94:0/2642884453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb941000a0 msgr2=0x7feb941972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb98f8f640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb941000a0 0x7feb941972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb98f8f640 1 -- 172.21.15.94:0/2642884453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb8c009900 con 0x7feb94101290 2024-02-17T11:56:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb99f91640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb94101c50 0x7feb9419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb99790640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb941000a0 0x7feb941972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb98f8f640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb94101290 0x7feb941977f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feb8400c920 tx=0x7feb8400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb8a7fc640 1 -- 172.21.15.94:0/2642884453 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84007c70 con 0x7feb94101290 2024-02-17T11:56:04.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb9b9f4640 1 -- 172.21.15.94:0/2642884453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9419f360 con 0x7feb94101290 2024-02-17T11:56:04.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb8a7fc640 1 -- 172.21.15.94:0/2642884453 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb84007dd0 con 0x7feb94101290 2024-02-17T11:56:04.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb8a7fc640 1 -- 172.21.15.94:0/2642884453 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84018860 con 0x7feb94101290 2024-02-17T11:56:04.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.980+0000 7feb9b9f4640 1 -- 172.21.15.94:0/2642884453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb9419f8c0 con 0x7feb94101290 2024-02-17T11:56:04.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.981+0000 7feb9b9f4640 1 -- 172.21.15.94:0/2642884453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb9410baf0 con 0x7feb94101290 2024-02-17T11:56:04.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.983+0000 7feb8a7fc640 1 -- 172.21.15.94:0/2642884453 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7feb8400fac0 con 0x7feb94101290 2024-02-17T11:56:04.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.983+0000 7feb8a7fc640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7feb800a2150 0x7feb800a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:04.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.983+0000 7feb99790640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7feb800a2150 0x7feb800a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:04.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.983+0000 7feb8a7fc640 1 -- 172.21.15.94:0/2642884453 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7feb840ce100 con 0x7feb94101290 2024-02-17T11:56:04.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.983+0000 7feb99790640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7feb800a2150 0x7feb800a4610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb8c032540 tx=0x7feb8c005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:04.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:04.987+0000 7feb8a7fc640 1 -- 172.21.15.94:0/2642884453 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feb840971b0 con 0x7feb94101290 2024-02-17T11:56:05.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.157+0000 7feb9b9f4640 1 -- 172.21.15.94:0/2642884453 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb940fd770 con 0x7feb800a2150 2024-02-17T11:56:05.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.168+0000 7feb8a7fc640 1 -- 172.21.15.94:0/2642884453 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336655 (secure 0 0 0) 0x7feb940fd770 con 0x7feb800a2150 2024-02-17T11:56:05.171 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:56:05.173 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-02-17T11:56:05.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.171+0000 7feb9b9f4640 1 -- 172.21.15.94:0/2642884453 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7feb800a2150 msgr2=0x7feb800a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:05.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.171+0000 7feb9b9f4640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7feb800a2150 0x7feb800a4610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb8c032540 tx=0x7feb8c005c50 comp rx=0 tx=0).stop 2024-02-17T11:56:05.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.171+0000 7feb9b9f4640 1 -- 172.21.15.94:0/2642884453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb94101290 msgr2=0x7feb941977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:05.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.171+0000 7feb9b9f4640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb94101290 0x7feb941977f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7feb8400c920 tx=0x7feb8400cde0 comp rx=0 tx=0).stop 2024-02-17T11:56:05.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.171+0000 7feb9b9f4640 1 -- 172.21.15.94:0/2642884453 shutdown_connections 2024-02-17T11:56:05.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.171+0000 7feb9b9f4640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7feb800a2150 0x7feb800a4610 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:05.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.171+0000 7feb9b9f4640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb94101c50 0x7feb9419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:05.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.171+0000 7feb9b9f4640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb94101290 0x7feb941977f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:05.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.171+0000 7feb9b9f4640 1 --2- 172.21.15.94:0/2642884453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb941000a0 0x7feb941972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:05.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.171+0000 7feb9b9f4640 1 -- 172.21.15.94:0/2642884453 >> 172.21.15.94:0/2642884453 conn(0x7feb940fb630 msgr2=0x7feb940fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:05.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.171+0000 7feb9b9f4640 1 -- 172.21.15.94:0/2642884453 shutdown_connections 2024-02-17T11:56:05.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:05.171+0000 7feb9b9f4640 1 -- 172.21.15.94:0/2642884453 wait complete. 2024-02-17T11:56:05.181 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-02-17T11:56:04.652981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632176,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":145,"num_read_kb":144,"num_write":306,"num_write_kb":3007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":245,"ondisk_log_size":245,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":247,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360184,"kb_used_data":10960,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581640,"statfs":{"total":1151940427776,"available":1151571599360,"internally_reserved":0,"allocated":11223040,"data_stored":7016889,"data_compressed":108564,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41808,"num_objects":54,"num_object_clones":0,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":0,"num_read":9,"num_read_kb":25,"num_write":125,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.000399"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.149529+0000","last_change":"2024-02-17T11:55:58.678553+0000","last_active":"2024-02-17T11:56:00.149529+0000","last_peered":"2024-02-17T11:56:00.149529+0000","last_clean":"2024-02-17T11:56:00.149529+0000","last_became_active":"2024-02-17T11:55:58.678212+0000","last_became_peered":"2024-02-17T11:55:58.678212+0000","last_unstale":"2024-02-17T11:56:00.149529+0000","last_undegraded":"2024-02-17T11:56:00.149529+0000","last_fullsized":"2024-02-17T11:56:00.149529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:09:18.011937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.683245+0000","last_change":"2024-02-17T11:55:58.681666+0000","last_active":"2024-02-17T11:55:58.683245+0000","last_peered":"2024-02-17T11:55:58.683245+0000","last_clean":"2024-02-17T11:55:58.683245+0000","last_became_active":"2024-02-17T11:55:58.681305+0000","last_became_peered":"2024-02-17T11:55:58.681305+0000","last_unstale":"2024-02-17T11:55:58.683245+0000","last_undegraded":"2024-02-17T11:55:58.683245+0000","last_fullsized":"2024-02-17T11:55:58.683245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:44:46.522902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.690117+0000","last_change":"2024-02-17T11:55:58.689465+0000","last_active":"2024-02-17T11:55:58.690117+0000","last_peered":"2024-02-17T11:55:58.690117+0000","last_clean":"2024-02-17T11:55:58.690117+0000","last_became_active":"2024-02-17T11:55:58.689166+0000","last_became_peered":"2024-02-17T11:55:58.689166+0000","last_unstale":"2024-02-17T11:55:58.690117+0000","last_undegraded":"2024-02-17T11:55:58.690117+0000","last_fullsized":"2024-02-17T11:55:58.690117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:31:27.999931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.144551+0000","last_change":"2024-02-17T11:55:58.678701+0000","last_active":"2024-02-17T11:56:00.144551+0000","last_peered":"2024-02-17T11:56:00.144551+0000","last_clean":"2024-02-17T11:56:00.144551+0000","last_became_active":"2024-02-17T11:55:58.678126+0000","last_became_peered":"2024-02-17T11:55:58.678126+0000","last_unstale":"2024-02-17T11:56:00.144551+0000","last_undegraded":"2024-02-17T11:56:00.144551+0000","last_fullsized":"2024-02-17T11:56:00.144551+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:57:20.127009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143231+0000","last_change":"2024-02-17T11:55:58.677543+0000","last_active":"2024-02-17T11:56:00.143231+0000","last_peered":"2024-02-17T11:56:00.143231+0000","last_clean":"2024-02-17T11:56:00.143231+0000","last_became_active":"2024-02-17T11:55:58.677223+0000","last_became_peered":"2024-02-17T11:55:58.677223+0000","last_unstale":"2024-02-17T11:56:00.143231+0000","last_undegraded":"2024-02-17T11:56:00.143231+0000","last_fullsized":"2024-02-17T11:56:00.143231+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:05:27.194963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140932+0000","last_change":"2024-02-17T11:55:58.673756+0000","last_active":"2024-02-17T11:56:00.140932+0000","last_peered":"2024-02-17T11:56:00.140932+0000","last_clean":"2024-02-17T11:56:00.140932+0000","last_became_active":"2024-02-17T11:55:58.672775+0000","last_became_peered":"2024-02-17T11:55:58.672775+0000","last_unstale":"2024-02-17T11:56:00.140932+0000","last_undegraded":"2024-02-17T11:56:00.140932+0000","last_fullsized":"2024-02-17T11:56:00.140932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:12:18.658414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143129+0000","last_change":"2024-02-17T11:55:58.675720+0000","last_active":"2024-02-17T11:56:00.143129+0000","last_peered":"2024-02-17T11:56:00.143129+0000","last_clean":"2024-02-17T11:56:00.143129+0000","last_became_active":"2024-02-17T11:55:58.675335+0000","last_became_peered":"2024-02-17T11:55:58.675335+0000","last_unstale":"2024-02-17T11:56:00.143129+0000","last_undegraded":"2024-02-17T11:56:00.143129+0000","last_fullsized":"2024-02-17T11:56:00.143129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:40:58.756439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139965+0000","last_change":"2024-02-17T11:55:58.675496+0000","last_active":"2024-02-17T11:56:00.139965+0000","last_peered":"2024-02-17T11:56:00.139965+0000","last_clean":"2024-02-17T11:56:00.139965+0000","last_became_active":"2024-02-17T11:55:58.675193+0000","last_became_peered":"2024-02-17T11:55:58.675193+0000","last_unstale":"2024-02-17T11:56:00.139965+0000","last_undegraded":"2024-02-17T11:56:00.139965+0000","last_fullsized":"2024-02-17T11:56:00.139965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:14:29.648857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.658999+0000","last_change":"2024-02-17T11:55:58.680859+0000","last_active":"2024-02-17T11:55:59.658999+0000","last_peered":"2024-02-17T11:55:59.658999+0000","last_clean":"2024-02-17T11:55:59.658999+0000","last_became_active":"2024-02-17T11:55:58.680489+0000","last_became_peered":"2024-02-17T11:55:58.680489+0000","last_unstale":"2024-02-17T11:55:59.658999+0000","last_undegraded":"2024-02-17T11:55:59.658999+0000","last_fullsized":"2024-02-17T11:55:59.658999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:29.963349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140798+0000","last_change":"2024-02-17T11:55:58.677896+0000","last_active":"2024-02-17T11:56:00.140798+0000","last_peered":"2024-02-17T11:56:00.140798+0000","last_clean":"2024-02-17T11:56:00.140798+0000","last_became_active":"2024-02-17T11:55:58.677540+0000","last_became_peered":"2024-02-17T11:55:58.677540+0000","last_unstale":"2024-02-17T11:56:00.140798+0000","last_undegraded":"2024-02-17T11:56:00.140798+0000","last_fullsized":"2024-02-17T11:56:00.140798+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:20:47.921687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143870+0000","last_change":"2024-02-17T11:55:58.688562+0000","last_active":"2024-02-17T11:56:00.143870+0000","last_peered":"2024-02-17T11:56:00.143870+0000","last_clean":"2024-02-17T11:56:00.143870+0000","last_became_active":"2024-02-17T11:55:58.688186+0000","last_became_peered":"2024-02-17T11:55:58.688186+0000","last_unstale":"2024-02-17T11:56:00.143870+0000","last_undegraded":"2024-02-17T11:56:00.143870+0000","last_fullsized":"2024-02-17T11:56:00.143870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:09.668937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.144343+0000","last_change":"2024-02-17T11:55:58.684989+0000","last_active":"2024-02-17T11:56:00.144343+0000","last_peered":"2024-02-17T11:56:00.144343+0000","last_clean":"2024-02-17T11:56:00.144343+0000","last_became_active":"2024-02-17T11:55:58.684656+0000","last_became_peered":"2024-02-17T11:55:58.684656+0000","last_unstale":"2024-02-17T11:56:00.144343+0000","last_undegraded":"2024-02-17T11:56:00.144343+0000","last_fullsized":"2024-02-17T11:56:00.144343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:29:43.480963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657578+0000","last_change":"2024-02-17T11:55:58.662076+0000","last_active":"2024-02-17T11:55:59.657578+0000","last_peered":"2024-02-17T11:55:59.657578+0000","last_clean":"2024-02-17T11:55:59.657578+0000","last_became_active":"2024-02-17T11:55:58.661716+0000","last_became_peered":"2024-02-17T11:55:58.661716+0000","last_unstale":"2024-02-17T11:55:59.657578+0000","last_undegraded":"2024-02-17T11:55:59.657578+0000","last_fullsized":"2024-02-17T11:55:59.657578+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:20:39.647774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.691857+0000","last_change":"2024-02-17T11:55:58.690762+0000","last_active":"2024-02-17T11:55:58.691857+0000","last_peered":"2024-02-17T11:55:58.691857+0000","last_clean":"2024-02-17T11:55:58.691857+0000","last_became_active":"2024-02-17T11:55:58.690414+0000","last_became_peered":"2024-02-17T11:55:58.690414+0000","last_unstale":"2024-02-17T11:55:58.691857+0000","last_undegraded":"2024-02-17T11:55:58.691857+0000","last_fullsized":"2024-02-17T11:55:58.691857+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:39:55.313442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.683205+0000","last_change":"2024-02-17T11:55:58.682391+0000","last_active":"2024-02-17T11:55:58.683205+0000","last_peered":"2024-02-17T11:55:58.683205+0000","last_clean":"2024-02-17T11:55:58.683205+0000","last_became_active":"2024-02-17T11:55:58.682048+0000","last_became_peered":"2024-02-17T11:55:58.682048+0000","last_unstale":"2024-02-17T11:55:58.683205+0000","last_undegraded":"2024-02-17T11:55:58.683205+0000","last_fullsized":"2024-02-17T11:55:58.683205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:42:43.777522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.658566+0000","last_change":"2024-02-17T11:55:58.685782+0000","last_active":"2024-02-17T11:55:59.658566+0000","last_peered":"2024-02-17T11:55:59.658566+0000","last_clean":"2024-02-17T11:55:59.658566+0000","last_became_active":"2024-02-17T11:55:58.683087+0000","last_became_peered":"2024-02-17T11:55:58.683087+0000","last_unstale":"2024-02-17T11:55:59.658566+0000","last_undegraded":"2024-02-17T11:55:59.658566+0000","last_fullsized":"2024-02-17T11:55:59.658566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:30:39.653717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.149527+0000","last_change":"2024-02-17T11:55:58.688400+0000","last_active":"2024-02-17T11:56:00.149527+0000","last_peered":"2024-02-17T11:56:00.149527+0000","last_clean":"2024-02-17T11:56:00.149527+0000","last_became_active":"2024-02-17T11:55:58.688040+0000","last_became_peered":"2024-02-17T11:55:58.688040+0000","last_unstale":"2024-02-17T11:56:00.149527+0000","last_undegraded":"2024-02-17T11:56:00.149527+0000","last_fullsized":"2024-02-17T11:56:00.149527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:39:02.706547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.896558+0000","last_change":"2024-02-17T11:55:58.687454+0000","last_active":"2024-02-17T11:55:59.896558+0000","last_peered":"2024-02-17T11:55:59.896558+0000","last_clean":"2024-02-17T11:55:59.896558+0000","last_became_active":"2024-02-17T11:55:58.686432+0000","last_became_peered":"2024-02-17T11:55:58.686432+0000","last_unstale":"2024-02-17T11:55:59.896558+0000","last_undegraded":"2024-02-17T11:55:59.896558+0000","last_fullsized":"2024-02-17T11:55:59.896558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:26:31.497782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.662763+0000","last_change":"2024-02-17T11:55:58.681806+0000","last_active":"2024-02-17T11:55:59.662763+0000","last_peered":"2024-02-17T11:55:59.662763+0000","last_clean":"2024-02-17T11:55:59.662763+0000","last_became_active":"2024-02-17T11:55:58.679764+0000","last_became_peered":"2024-02-17T11:55:58.679764+0000","last_unstale":"2024-02-17T11:55:59.662763+0000","last_undegraded":"2024-02-17T11:55:59.662763+0000","last_fullsized":"2024-02-17T11:55:59.662763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:13:14.733417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.156700+0000","last_change":"2024-02-17T11:55:57.672689+0000","last_active":"2024-02-17T11:56:01.156700+0000","last_peered":"2024-02-17T11:56:01.156700+0000","last_clean":"2024-02-17T11:56:01.156700+0000","last_became_active":"2024-02-17T11:55:57.672352+0000","last_became_peered":"2024-02-17T11:55:57.672352+0000","last_unstale":"2024-02-17T11:56:01.156700+0000","last_undegraded":"2024-02-17T11:56:01.156700+0000","last_fullsized":"2024-02-17T11:56:01.156700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:00:34.395594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139808+0000","last_change":"2024-02-17T11:55:58.690679+0000","last_active":"2024-02-17T11:56:00.139808+0000","last_peered":"2024-02-17T11:56:00.139808+0000","last_clean":"2024-02-17T11:56:00.139808+0000","last_became_active":"2024-02-17T11:55:58.690353+0000","last_became_peered":"2024-02-17T11:55:58.690353+0000","last_unstale":"2024-02-17T11:56:00.139808+0000","last_undegraded":"2024-02-17T11:56:00.139808+0000","last_fullsized":"2024-02-17T11:56:00.139808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:45:33.830784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139853+0000","last_change":"2024-02-17T11:55:57.677649+0000","last_active":"2024-02-17T11:56:00.139853+0000","last_peered":"2024-02-17T11:56:00.139853+0000","last_clean":"2024-02-17T11:56:00.139853+0000","last_became_active":"2024-02-17T11:55:57.677331+0000","last_became_peered":"2024-02-17T11:55:57.677331+0000","last_unstale":"2024-02-17T11:56:00.139853+0000","last_undegraded":"2024-02-17T11:56:00.139853+0000","last_fullsized":"2024-02-17T11:56:00.139853+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:06.932255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.689917+0000","last_change":"2024-02-17T11:55:58.688575+0000","last_active":"2024-02-17T11:55:58.689917+0000","last_peered":"2024-02-17T11:55:58.689917+0000","last_clean":"2024-02-17T11:55:58.689917+0000","last_became_active":"2024-02-17T11:55:58.688238+0000","last_became_peered":"2024-02-17T11:55:58.688238+0000","last_unstale":"2024-02-17T11:55:58.689917+0000","last_undegraded":"2024-02-17T11:55:58.689917+0000","last_fullsized":"2024-02-17T11:55:58.689917+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:57:28.183776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.146001+0000","last_change":"2024-02-17T11:55:57.679676+0000","last_active":"2024-02-17T11:56:00.146001+0000","last_peered":"2024-02-17T11:56:00.146001+0000","last_clean":"2024-02-17T11:56:00.146001+0000","last_became_active":"2024-02-17T11:55:57.679350+0000","last_became_peered":"2024-02-17T11:55:57.679350+0000","last_unstale":"2024-02-17T11:56:00.146001+0000","last_undegraded":"2024-02-17T11:56:00.146001+0000","last_fullsized":"2024-02-17T11:56:00.146001+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:52:45.072896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.659027+0000","last_change":"2024-02-17T11:55:58.682212+0000","last_active":"2024-02-17T11:55:59.659027+0000","last_peered":"2024-02-17T11:55:59.659027+0000","last_clean":"2024-02-17T11:55:59.659027+0000","last_became_active":"2024-02-17T11:55:58.681773+0000","last_became_peered":"2024-02-17T11:55:58.681773+0000","last_unstale":"2024-02-17T11:55:59.659027+0000","last_undegraded":"2024-02-17T11:55:59.659027+0000","last_fullsized":"2024-02-17T11:55:59.659027+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:28:18.577174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.145333+0000","last_change":"2024-02-17T11:55:57.667320+0000","last_active":"2024-02-17T11:56:00.145333+0000","last_peered":"2024-02-17T11:56:00.145333+0000","last_clean":"2024-02-17T11:56:00.145333+0000","last_became_active":"2024-02-17T11:55:57.666989+0000","last_became_peered":"2024-02-17T11:55:57.666989+0000","last_unstale":"2024-02-17T11:56:00.145333+0000","last_undegraded":"2024-02-17T11:56:00.145333+0000","last_fullsized":"2024-02-17T11:56:00.145333+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:00:08.275186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.144329+0000","last_change":"2024-02-17T11:55:58.672771+0000","last_active":"2024-02-17T11:56:00.144329+0000","last_peered":"2024-02-17T11:56:00.144329+0000","last_clean":"2024-02-17T11:56:00.144329+0000","last_became_active":"2024-02-17T11:55:58.671877+0000","last_became_peered":"2024-02-17T11:55:58.671877+0000","last_unstale":"2024-02-17T11:56:00.144329+0000","last_undegraded":"2024-02-17T11:56:00.144329+0000","last_fullsized":"2024-02-17T11:56:00.144329+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:38:38.350695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'5","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.151798+0000","last_change":"2024-02-17T11:55:57.669396+0000","last_active":"2024-02-17T11:56:00.151798+0000","last_peered":"2024-02-17T11:56:00.151798+0000","last_clean":"2024-02-17T11:56:00.151798+0000","last_became_active":"2024-02-17T11:55:57.669116+0000","last_became_peered":"2024-02-17T11:55:57.669116+0000","last_unstale":"2024-02-17T11:56:00.151798+0000","last_undegraded":"2024-02-17T11:56:00.151798+0000","last_fullsized":"2024-02-17T11:56:00.151798+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:05:11.081309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143170+0000","last_change":"2024-02-17T11:55:58.674379+0000","last_active":"2024-02-17T11:56:00.143170+0000","last_peered":"2024-02-17T11:56:00.143170+0000","last_clean":"2024-02-17T11:56:00.143170+0000","last_became_active":"2024-02-17T11:55:58.674015+0000","last_became_peered":"2024-02-17T11:55:58.674015+0000","last_unstale":"2024-02-17T11:56:00.143170+0000","last_undegraded":"2024-02-17T11:56:00.143170+0000","last_fullsized":"2024-02-17T11:56:00.143170+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:44:20.410623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.154931+0000","last_change":"2024-02-17T11:55:57.668436+0000","last_active":"2024-02-17T11:56:01.154931+0000","last_peered":"2024-02-17T11:56:01.154931+0000","last_clean":"2024-02-17T11:56:01.154931+0000","last_became_active":"2024-02-17T11:55:57.668004+0000","last_became_peered":"2024-02-17T11:55:57.668004+0000","last_unstale":"2024-02-17T11:56:01.154931+0000","last_undegraded":"2024-02-17T11:56:01.154931+0000","last_fullsized":"2024-02-17T11:56:01.154931+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:13:42.566644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.147215+0000","last_change":"2024-02-17T11:55:58.686888+0000","last_active":"2024-02-17T11:56:00.147215+0000","last_peered":"2024-02-17T11:56:00.147215+0000","last_clean":"2024-02-17T11:56:00.147215+0000","last_became_active":"2024-02-17T11:55:58.686376+0000","last_became_peered":"2024-02-17T11:55:58.686376+0000","last_unstale":"2024-02-17T11:56:00.147215+0000","last_undegraded":"2024-02-17T11:56:00.147215+0000","last_fullsized":"2024-02-17T11:56:00.147215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:58:48.292354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.155654+0000","last_change":"2024-02-17T11:55:57.665717+0000","last_active":"2024-02-17T11:56:01.155654+0000","last_peered":"2024-02-17T11:56:01.155654+0000","last_clean":"2024-02-17T11:56:01.155654+0000","last_became_active":"2024-02-17T11:55:57.665381+0000","last_became_peered":"2024-02-17T11:55:57.665381+0000","last_unstale":"2024-02-17T11:56:01.155654+0000","last_undegraded":"2024-02-17T11:56:01.155654+0000","last_fullsized":"2024-02-17T11:56:01.155654+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:15:27.424332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139912+0000","last_change":"2024-02-17T11:55:58.673261+0000","last_active":"2024-02-17T11:56:00.139912+0000","last_peered":"2024-02-17T11:56:00.139912+0000","last_clean":"2024-02-17T11:56:00.139912+0000","last_became_active":"2024-02-17T11:55:58.672912+0000","last_became_peered":"2024-02-17T11:55:58.672912+0000","last_unstale":"2024-02-17T11:56:00.139912+0000","last_undegraded":"2024-02-17T11:56:00.139912+0000","last_fullsized":"2024-02-17T11:56:00.139912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:42.605291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148312+0000","last_change":"2024-02-17T11:55:57.667792+0000","last_active":"2024-02-17T11:56:00.148312+0000","last_peered":"2024-02-17T11:56:00.148312+0000","last_clean":"2024-02-17T11:56:00.148312+0000","last_became_active":"2024-02-17T11:55:57.667361+0000","last_became_peered":"2024-02-17T11:55:57.667361+0000","last_unstale":"2024-02-17T11:56:00.148312+0000","last_undegraded":"2024-02-17T11:56:00.148312+0000","last_fullsized":"2024-02-17T11:56:00.148312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:48.073742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.896505+0000","last_change":"2024-02-17T11:55:58.672180+0000","last_active":"2024-02-17T11:55:59.896505+0000","last_peered":"2024-02-17T11:55:59.896505+0000","last_clean":"2024-02-17T11:55:59.896505+0000","last_became_active":"2024-02-17T11:55:58.671790+0000","last_became_peered":"2024-02-17T11:55:58.671790+0000","last_unstale":"2024-02-17T11:55:59.896505+0000","last_undegraded":"2024-02-17T11:55:59.896505+0000","last_fullsized":"2024-02-17T11:55:59.896505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:48:39.982924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:02.159261+0000","last_change":"2024-02-17T11:55:57.670085+0000","last_active":"2024-02-17T11:56:02.159261+0000","last_peered":"2024-02-17T11:56:02.159261+0000","last_clean":"2024-02-17T11:56:02.159261+0000","last_became_active":"2024-02-17T11:55:57.669648+0000","last_became_peered":"2024-02-17T11:55:57.669648+0000","last_unstale":"2024-02-17T11:56:02.159261+0000","last_undegraded":"2024-02-17T11:56:02.159261+0000","last_fullsized":"2024-02-17T11:56:02.159261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:31:34.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.658885+0000","last_change":"2024-02-17T11:55:58.685080+0000","last_active":"2024-02-17T11:55:59.658885+0000","last_peered":"2024-02-17T11:55:59.658885+0000","last_clean":"2024-02-17T11:55:59.658885+0000","last_became_active":"2024-02-17T11:55:58.684763+0000","last_became_peered":"2024-02-17T11:55:58.684763+0000","last_unstale":"2024-02-17T11:55:59.658885+0000","last_undegraded":"2024-02-17T11:55:59.658885+0000","last_fullsized":"2024-02-17T11:55:59.658885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:59:53.763492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148316+0000","last_change":"2024-02-17T11:55:57.676391+0000","last_active":"2024-02-17T11:56:00.148316+0000","last_peered":"2024-02-17T11:56:00.148316+0000","last_clean":"2024-02-17T11:56:00.148316+0000","last_became_active":"2024-02-17T11:55:57.676103+0000","last_became_peered":"2024-02-17T11:55:57.676103+0000","last_unstale":"2024-02-17T11:56:00.148316+0000","last_undegraded":"2024-02-17T11:56:00.148316+0000","last_fullsized":"2024-02-17T11:56:00.148316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:37:52.866379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.682503+0000","last_change":"2024-02-17T11:55:58.679274+0000","last_active":"2024-02-17T11:55:58.682503+0000","last_peered":"2024-02-17T11:55:58.682503+0000","last_clean":"2024-02-17T11:55:58.682503+0000","last_became_active":"2024-02-17T11:55:58.678945+0000","last_became_peered":"2024-02-17T11:55:58.678945+0000","last_unstale":"2024-02-17T11:55:58.682503+0000","last_undegraded":"2024-02-17T11:55:58.682503+0000","last_fullsized":"2024-02-17T11:55:58.682503+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:23:25.998999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.147530+0000","last_change":"2024-02-17T11:55:57.672732+0000","last_active":"2024-02-17T11:56:00.147530+0000","last_peered":"2024-02-17T11:56:00.147530+0000","last_clean":"2024-02-17T11:56:00.147530+0000","last_became_active":"2024-02-17T11:55:57.672409+0000","last_became_peered":"2024-02-17T11:55:57.672409+0000","last_unstale":"2024-02-17T11:56:00.147530+0000","last_undegraded":"2024-02-17T11:56:00.147530+0000","last_fullsized":"2024-02-17T11:56:00.147530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:07:31.556508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.658675+0000","last_change":"2024-02-17T11:55:58.689983+0000","last_active":"2024-02-17T11:55:59.658675+0000","last_peered":"2024-02-17T11:55:59.658675+0000","last_clean":"2024-02-17T11:55:59.658675+0000","last_became_active":"2024-02-17T11:55:58.689629+0000","last_became_peered":"2024-02-17T11:55:58.689629+0000","last_unstale":"2024-02-17T11:55:59.658675+0000","last_undegraded":"2024-02-17T11:55:59.658675+0000","last_fullsized":"2024-02-17T11:55:59.658675+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:12.321746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'19","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:03.542259+0000","last_change":"2024-02-17T11:55:57.673273+0000","last_active":"2024-02-17T11:56:03.542259+0000","last_peered":"2024-02-17T11:56:03.542259+0000","last_clean":"2024-02-17T11:56:03.542259+0000","last_became_active":"2024-02-17T11:55:57.672865+0000","last_became_peered":"2024-02-17T11:55:57.672865+0000","last_unstale":"2024-02-17T11:56:03.542259+0000","last_undegraded":"2024-02-17T11:56:03.542259+0000","last_fullsized":"2024-02-17T11:56:03.542259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:31:01.593798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38193,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":7,"num_read_kb":23,"num_write":20,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.145364+0000","last_change":"2024-02-17T11:55:58.669296+0000","last_active":"2024-02-17T11:56:00.145364+0000","last_peered":"2024-02-17T11:56:00.145364+0000","last_clean":"2024-02-17T11:56:00.145364+0000","last_became_active":"2024-02-17T11:55:58.668927+0000","last_became_peered":"2024-02-17T11:55:58.668927+0000","last_unstale":"2024-02-17T11:56:00.145364+0000","last_undegraded":"2024-02-17T11:56:00.145364+0000","last_fullsized":"2024-02-17T11:56:00.145364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:25.177905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.154885+0000","last_change":"2024-02-17T11:55:57.674523+0000","last_active":"2024-02-17T11:56:01.154885+0000","last_peered":"2024-02-17T11:56:01.154885+0000","last_clean":"2024-02-17T11:56:01.154885+0000","last_became_active":"2024-02-17T11:55:57.672375+0000","last_became_peered":"2024-02-17T11:55:57.672375+0000","last_unstale":"2024-02-17T11:56:01.154885+0000","last_undegraded":"2024-02-17T11:56:01.154885+0000","last_fullsized":"2024-02-17T11:56:01.154885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:15:21.568434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657583+0000","last_change":"2024-02-17T11:55:58.682106+0000","last_active":"2024-02-17T11:55:59.657583+0000","last_peered":"2024-02-17T11:55:59.657583+0000","last_clean":"2024-02-17T11:55:59.657583+0000","last_became_active":"2024-02-17T11:55:58.681420+0000","last_became_peered":"2024-02-17T11:55:58.681420+0000","last_unstale":"2024-02-17T11:55:59.657583+0000","last_undegraded":"2024-02-17T11:55:59.657583+0000","last_fullsized":"2024-02-17T11:55:59.657583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:23:00.521690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.150460+0000","last_change":"2024-02-17T11:55:57.675161+0000","last_active":"2024-02-17T11:56:00.150460+0000","last_peered":"2024-02-17T11:56:00.150460+0000","last_clean":"2024-02-17T11:56:00.150460+0000","last_became_active":"2024-02-17T11:55:57.674776+0000","last_became_peered":"2024-02-17T11:55:57.674776+0000","last_unstale":"2024-02-17T11:56:00.150460+0000","last_undegraded":"2024-02-17T11:56:00.150460+0000","last_fullsized":"2024-02-17T11:56:00.150460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:25:59.695014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143497+0000","last_change":"2024-02-17T11:55:58.679319+0000","last_active":"2024-02-17T11:56:00.143497+0000","last_peered":"2024-02-17T11:56:00.143497+0000","last_clean":"2024-02-17T11:56:00.143497+0000","last_became_active":"2024-02-17T11:55:58.679001+0000","last_became_peered":"2024-02-17T11:55:58.679001+0000","last_unstale":"2024-02-17T11:56:00.143497+0000","last_undegraded":"2024-02-17T11:56:00.143497+0000","last_fullsized":"2024-02-17T11:56:00.143497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:36:48.493338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.154733+0000","last_change":"2024-02-17T11:55:57.670027+0000","last_active":"2024-02-17T11:56:01.154733+0000","last_peered":"2024-02-17T11:56:01.154733+0000","last_clean":"2024-02-17T11:56:01.154733+0000","last_became_active":"2024-02-17T11:55:57.669692+0000","last_became_peered":"2024-02-17T11:55:57.669692+0000","last_unstale":"2024-02-17T11:56:01.154733+0000","last_undegraded":"2024-02-17T11:56:01.154733+0000","last_fullsized":"2024-02-17T11:56:01.154733+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:44:01.001640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143029+0000","last_change":"2024-02-17T11:55:58.672811+0000","last_active":"2024-02-17T11:56:00.143029+0000","last_peered":"2024-02-17T11:56:00.143029+0000","last_clean":"2024-02-17T11:56:00.143029+0000","last_became_active":"2024-02-17T11:55:58.672469+0000","last_became_peered":"2024-02-17T11:55:58.672469+0000","last_unstale":"2024-02-17T11:56:00.143029+0000","last_undegraded":"2024-02-17T11:56:00.143029+0000","last_fullsized":"2024-02-17T11:56:00.143029+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:07.451422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'10","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148194+0000","last_change":"2024-02-17T11:55:57.668031+0000","last_active":"2024-02-17T11:56:00.148194+0000","last_peered":"2024-02-17T11:56:00.148194+0000","last_clean":"2024-02-17T11:56:00.148194+0000","last_became_active":"2024-02-17T11:55:57.667344+0000","last_became_peered":"2024-02-17T11:55:57.667344+0000","last_unstale":"2024-02-17T11:56:00.148194+0000","last_undegraded":"2024-02-17T11:56:00.148194+0000","last_fullsized":"2024-02-17T11:56:00.148194+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:14:20.773902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143827+0000","last_change":"2024-02-17T11:55:58.682013+0000","last_active":"2024-02-17T11:56:00.143827+0000","last_peered":"2024-02-17T11:56:00.143827+0000","last_clean":"2024-02-17T11:56:00.143827+0000","last_became_active":"2024-02-17T11:55:58.681503+0000","last_became_peered":"2024-02-17T11:55:58.681503+0000","last_unstale":"2024-02-17T11:56:00.143827+0000","last_undegraded":"2024-02-17T11:56:00.143827+0000","last_fullsized":"2024-02-17T11:56:00.143827+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:24:50.364090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.146816+0000","last_change":"2024-02-17T11:55:57.666725+0000","last_active":"2024-02-17T11:56:00.146816+0000","last_peered":"2024-02-17T11:56:00.146816+0000","last_clean":"2024-02-17T11:56:00.146816+0000","last_became_active":"2024-02-17T11:55:57.666339+0000","last_became_peered":"2024-02-17T11:55:57.666339+0000","last_unstale":"2024-02-17T11:56:00.146816+0000","last_undegraded":"2024-02-17T11:56:00.146816+0000","last_fullsized":"2024-02-17T11:56:00.146816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:22:47.324016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.147908+0000","last_change":"2024-02-17T11:55:58.675995+0000","last_active":"2024-02-17T11:56:00.147908+0000","last_peered":"2024-02-17T11:56:00.147908+0000","last_clean":"2024-02-17T11:56:00.147908+0000","last_became_active":"2024-02-17T11:55:58.675655+0000","last_became_peered":"2024-02-17T11:55:58.675655+0000","last_unstale":"2024-02-17T11:56:00.147908+0000","last_undegraded":"2024-02-17T11:56:00.147908+0000","last_fullsized":"2024-02-17T11:56:00.147908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:12.465300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.896394+0000","last_change":"2024-02-17T11:55:57.669500+0000","last_active":"2024-02-17T11:55:59.896394+0000","last_peered":"2024-02-17T11:55:59.896394+0000","last_clean":"2024-02-17T11:55:59.896394+0000","last_became_active":"2024-02-17T11:55:57.669148+0000","last_became_peered":"2024-02-17T11:55:57.669148+0000","last_unstale":"2024-02-17T11:55:59.896394+0000","last_undegraded":"2024-02-17T11:55:59.896394+0000","last_fullsized":"2024-02-17T11:55:59.896394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:33.821266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140787+0000","last_change":"2024-02-17T11:55:58.678336+0000","last_active":"2024-02-17T11:56:00.140787+0000","last_peered":"2024-02-17T11:56:00.140787+0000","last_clean":"2024-02-17T11:56:00.140787+0000","last_became_active":"2024-02-17T11:55:58.677967+0000","last_became_peered":"2024-02-17T11:55:58.677967+0000","last_unstale":"2024-02-17T11:56:00.140787+0000","last_undegraded":"2024-02-17T11:56:00.140787+0000","last_fullsized":"2024-02-17T11:56:00.140787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:31:39.406673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143517+0000","last_change":"2024-02-17T11:55:57.674810+0000","last_active":"2024-02-17T11:56:00.143517+0000","last_peered":"2024-02-17T11:56:00.143517+0000","last_clean":"2024-02-17T11:56:00.143517+0000","last_became_active":"2024-02-17T11:55:57.674356+0000","last_became_peered":"2024-02-17T11:55:57.674356+0000","last_unstale":"2024-02-17T11:56:00.143517+0000","last_undegraded":"2024-02-17T11:56:00.143517+0000","last_fullsized":"2024-02-17T11:56:00.143517+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:25:57.879239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140758+0000","last_change":"2024-02-17T11:55:58.680928+0000","last_active":"2024-02-17T11:56:00.140758+0000","last_peered":"2024-02-17T11:56:00.140758+0000","last_clean":"2024-02-17T11:56:00.140758+0000","last_became_active":"2024-02-17T11:55:58.680521+0000","last_became_peered":"2024-02-17T11:55:58.680521+0000","last_unstale":"2024-02-17T11:56:00.140758+0000","last_undegraded":"2024-02-17T11:56:00.140758+0000","last_fullsized":"2024-02-17T11:56:00.140758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:15:15.739127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.144262+0000","last_change":"2024-02-17T11:55:58.674771+0000","last_active":"2024-02-17T11:56:00.144262+0000","last_peered":"2024-02-17T11:56:00.144262+0000","last_clean":"2024-02-17T11:56:00.144262+0000","last_became_active":"2024-02-17T11:55:58.674450+0000","last_became_peered":"2024-02-17T11:55:58.674450+0000","last_unstale":"2024-02-17T11:56:00.144262+0000","last_undegraded":"2024-02-17T11:56:00.144262+0000","last_fullsized":"2024-02-17T11:56:00.144262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.031576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.155863+0000","last_change":"2024-02-17T11:55:57.672868+0000","last_active":"2024-02-17T11:56:01.155863+0000","last_peered":"2024-02-17T11:56:01.155863+0000","last_clean":"2024-02-17T11:56:01.155863+0000","last_became_active":"2024-02-17T11:55:57.672584+0000","last_became_peered":"2024-02-17T11:55:57.672584+0000","last_unstale":"2024-02-17T11:56:01.155863+0000","last_undegraded":"2024-02-17T11:56:01.155863+0000","last_fullsized":"2024-02-17T11:56:01.155863+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:03:20.307036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140579+0000","last_change":"2024-02-17T11:55:58.692594+0000","last_active":"2024-02-17T11:56:00.140579+0000","last_peered":"2024-02-17T11:56:00.140579+0000","last_clean":"2024-02-17T11:56:00.140579+0000","last_became_active":"2024-02-17T11:55:58.692258+0000","last_became_peered":"2024-02-17T11:55:58.692258+0000","last_unstale":"2024-02-17T11:56:00.140579+0000","last_undegraded":"2024-02-17T11:56:00.140579+0000","last_fullsized":"2024-02-17T11:56:00.140579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:28.497939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.147183+0000","last_change":"2024-02-17T11:55:57.669456+0000","last_active":"2024-02-17T11:56:00.147183+0000","last_peered":"2024-02-17T11:56:00.147183+0000","last_clean":"2024-02-17T11:56:00.147183+0000","last_became_active":"2024-02-17T11:55:57.669067+0000","last_became_peered":"2024-02-17T11:55:57.669067+0000","last_unstale":"2024-02-17T11:56:00.147183+0000","last_undegraded":"2024-02-17T11:56:00.147183+0000","last_fullsized":"2024-02-17T11:56:00.147183+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:12.352296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":349,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143175+0000","last_change":"2024-02-17T11:54:46.125844+0000","last_active":"2024-02-17T11:56:00.143175+0000","last_peered":"2024-02-17T11:56:00.143175+0000","last_clean":"2024-02-17T11:56:00.143175+0000","last_became_active":"2024-02-17T11:54:46.125387+0000","last_became_peered":"2024-02-17T11:54:46.125387+0000","last_unstale":"2024-02-17T11:56:00.143175+0000","last_undegraded":"2024-02-17T11:56:00.143175+0000","last_fullsized":"2024-02-17T11:56:00.143175+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_clean_scrub_stamp":"2024-02-17T11:51:21.581371+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:53:27.219974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657088+0000","last_change":"2024-02-17T11:55:58.681623+0000","last_active":"2024-02-17T11:55:59.657088+0000","last_peered":"2024-02-17T11:55:59.657088+0000","last_clean":"2024-02-17T11:55:59.657088+0000","last_became_active":"2024-02-17T11:55:58.681275+0000","last_became_peered":"2024-02-17T11:55:58.681275+0000","last_unstale":"2024-02-17T11:55:59.657088+0000","last_undegraded":"2024-02-17T11:55:59.657088+0000","last_fullsized":"2024-02-17T11:55:59.657088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:20.685634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.150096+0000","last_change":"2024-02-17T11:55:57.679555+0000","last_active":"2024-02-17T11:56:00.150096+0000","last_peered":"2024-02-17T11:56:00.150096+0000","last_clean":"2024-02-17T11:56:00.150096+0000","last_became_active":"2024-02-17T11:55:57.679241+0000","last_became_peered":"2024-02-17T11:55:57.679241+0000","last_unstale":"2024-02-17T11:56:00.150096+0000","last_undegraded":"2024-02-17T11:56:00.150096+0000","last_fullsized":"2024-02-17T11:56:00.150096+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:37.482582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148424+0000","last_change":"2024-02-17T11:55:57.679711+0000","last_active":"2024-02-17T11:56:00.148424+0000","last_peered":"2024-02-17T11:56:00.148424+0000","last_clean":"2024-02-17T11:56:00.148424+0000","last_became_active":"2024-02-17T11:55:57.679415+0000","last_became_peered":"2024-02-17T11:55:57.679415+0000","last_unstale":"2024-02-17T11:56:00.148424+0000","last_undegraded":"2024-02-17T11:56:00.148424+0000","last_fullsized":"2024-02-17T11:56:00.148424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:13.095530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657267+0000","last_change":"2024-02-17T11:55:58.670275+0000","last_active":"2024-02-17T11:55:59.657267+0000","last_peered":"2024-02-17T11:55:59.657267+0000","last_clean":"2024-02-17T11:55:59.657267+0000","last_became_active":"2024-02-17T11:55:58.669926+0000","last_became_peered":"2024-02-17T11:55:58.669926+0000","last_unstale":"2024-02-17T11:55:59.657267+0000","last_undegraded":"2024-02-17T11:55:59.657267+0000","last_fullsized":"2024-02-17T11:55:59.657267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:48:35.078389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.156432+0000","last_change":"2024-02-17T11:55:57.670364+0000","last_active":"2024-02-17T11:56:01.156432+0000","last_peered":"2024-02-17T11:56:01.156432+0000","last_clean":"2024-02-17T11:56:01.156432+0000","last_became_active":"2024-02-17T11:55:57.669977+0000","last_became_peered":"2024-02-17T11:55:57.669977+0000","last_unstale":"2024-02-17T11:56:01.156432+0000","last_undegraded":"2024-02-17T11:56:01.156432+0000","last_fullsized":"2024-02-17T11:56:01.156432+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:42:13.820390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139996+0000","last_change":"2024-02-17T11:55:58.681587+0000","last_active":"2024-02-17T11:56:00.139996+0000","last_peered":"2024-02-17T11:56:00.139996+0000","last_clean":"2024-02-17T11:56:00.139996+0000","last_became_active":"2024-02-17T11:55:58.680024+0000","last_became_peered":"2024-02-17T11:55:58.680024+0000","last_unstale":"2024-02-17T11:56:00.139996+0000","last_undegraded":"2024-02-17T11:56:00.139996+0000","last_fullsized":"2024-02-17T11:56:00.139996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:31:40.399439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.149961+0000","last_change":"2024-02-17T11:55:57.674618+0000","last_active":"2024-02-17T11:56:00.149961+0000","last_peered":"2024-02-17T11:56:00.149961+0000","last_clean":"2024-02-17T11:56:00.149961+0000","last_became_active":"2024-02-17T11:55:57.674246+0000","last_became_peered":"2024-02-17T11:55:57.674246+0000","last_unstale":"2024-02-17T11:56:00.149961+0000","last_undegraded":"2024-02-17T11:56:00.149961+0000","last_fullsized":"2024-02-17T11:56:00.149961+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:42:26.681061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.681851+0000","last_change":"2024-02-17T11:55:58.680320+0000","last_active":"2024-02-17T11:55:58.681851+0000","last_peered":"2024-02-17T11:55:58.681851+0000","last_clean":"2024-02-17T11:55:58.681851+0000","last_became_active":"2024-02-17T11:55:58.680005+0000","last_became_peered":"2024-02-17T11:55:58.680005+0000","last_unstale":"2024-02-17T11:55:58.681851+0000","last_undegraded":"2024-02-17T11:55:58.681851+0000","last_fullsized":"2024-02-17T11:55:58.681851+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:54:49.976680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139356+0000","last_change":"2024-02-17T11:55:57.667998+0000","last_active":"2024-02-17T11:56:00.139356+0000","last_peered":"2024-02-17T11:56:00.139356+0000","last_clean":"2024-02-17T11:56:00.139356+0000","last_became_active":"2024-02-17T11:55:57.667688+0000","last_became_peered":"2024-02-17T11:55:57.667688+0000","last_unstale":"2024-02-17T11:56:00.139356+0000","last_undegraded":"2024-02-17T11:56:00.139356+0000","last_fullsized":"2024-02-17T11:56:00.139356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:07:19.782879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.142836+0000","last_change":"2024-02-17T11:55:58.676644+0000","last_active":"2024-02-17T11:56:00.142836+0000","last_peered":"2024-02-17T11:56:00.142836+0000","last_clean":"2024-02-17T11:56:00.142836+0000","last_became_active":"2024-02-17T11:55:58.674670+0000","last_became_peered":"2024-02-17T11:55:58.674670+0000","last_unstale":"2024-02-17T11:56:00.142836+0000","last_undegraded":"2024-02-17T11:56:00.142836+0000","last_fullsized":"2024-02-17T11:56:00.142836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:43:22.746537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.145300+0000","last_change":"2024-02-17T11:55:57.671330+0000","last_active":"2024-02-17T11:56:00.145300+0000","last_peered":"2024-02-17T11:56:00.145300+0000","last_clean":"2024-02-17T11:56:00.145300+0000","last_became_active":"2024-02-17T11:55:57.670902+0000","last_became_peered":"2024-02-17T11:55:57.670902+0000","last_unstale":"2024-02-17T11:56:00.145300+0000","last_undegraded":"2024-02-17T11:56:00.145300+0000","last_fullsized":"2024-02-17T11:56:00.145300+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:22:14.770339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148553+0000","last_change":"2024-02-17T11:55:58.676417+0000","last_active":"2024-02-17T11:56:00.148553+0000","last_peered":"2024-02-17T11:56:00.148553+0000","last_clean":"2024-02-17T11:56:00.148553+0000","last_became_active":"2024-02-17T11:55:58.675960+0000","last_became_peered":"2024-02-17T11:55:58.675960+0000","last_unstale":"2024-02-17T11:56:00.148553+0000","last_undegraded":"2024-02-17T11:56:00.148553+0000","last_fullsized":"2024-02-17T11:56:00.148553+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:22:18.566872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.662655+0000","last_change":"2024-02-17T11:55:57.672668+0000","last_active":"2024-02-17T11:55:59.662655+0000","last_peered":"2024-02-17T11:55:59.662655+0000","last_clean":"2024-02-17T11:55:59.662655+0000","last_became_active":"2024-02-17T11:55:57.672363+0000","last_became_peered":"2024-02-17T11:55:57.672363+0000","last_unstale":"2024-02-17T11:55:59.662655+0000","last_undegraded":"2024-02-17T11:55:59.662655+0000","last_fullsized":"2024-02-17T11:55:59.662655+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:17:25.878564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140241+0000","last_change":"2024-02-17T11:55:58.690030+0000","last_active":"2024-02-17T11:56:00.140241+0000","last_peered":"2024-02-17T11:56:00.140241+0000","last_clean":"2024-02-17T11:56:00.140241+0000","last_became_active":"2024-02-17T11:55:58.689687+0000","last_became_peered":"2024-02-17T11:55:58.689687+0000","last_unstale":"2024-02-17T11:56:00.140241+0000","last_undegraded":"2024-02-17T11:56:00.140241+0000","last_fullsized":"2024-02-17T11:56:00.140241+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:33:43.175234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148085+0000","last_change":"2024-02-17T11:55:57.679734+0000","last_active":"2024-02-17T11:56:00.148085+0000","last_peered":"2024-02-17T11:56:00.148085+0000","last_clean":"2024-02-17T11:56:00.148085+0000","last_became_active":"2024-02-17T11:55:57.679328+0000","last_became_peered":"2024-02-17T11:55:57.679328+0000","last_unstale":"2024-02-17T11:56:00.148085+0000","last_undegraded":"2024-02-17T11:56:00.148085+0000","last_fullsized":"2024-02-17T11:56:00.148085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:50:09.813482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143825+0000","last_change":"2024-02-17T11:55:58.688360+0000","last_active":"2024-02-17T11:56:00.143825+0000","last_peered":"2024-02-17T11:56:00.143825+0000","last_clean":"2024-02-17T11:56:00.143825+0000","last_became_active":"2024-02-17T11:55:58.687994+0000","last_became_peered":"2024-02-17T11:55:58.687994+0000","last_unstale":"2024-02-17T11:56:00.143825+0000","last_undegraded":"2024-02-17T11:56:00.143825+0000","last_fullsized":"2024-02-17T11:56:00.143825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:01:02.163238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:01.156019+0000","last_change":"2024-02-17T11:55:57.675977+0000","last_active":"2024-02-17T11:56:01.156019+0000","last_peered":"2024-02-17T11:56:01.156019+0000","last_clean":"2024-02-17T11:56:01.156019+0000","last_became_active":"2024-02-17T11:55:57.675250+0000","last_became_peered":"2024-02-17T11:55:57.675250+0000","last_unstale":"2024-02-17T11:56:01.156019+0000","last_undegraded":"2024-02-17T11:56:01.156019+0000","last_fullsized":"2024-02-17T11:56:01.156019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:11:27.611692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143929+0000","last_change":"2024-02-17T11:55:58.688593+0000","last_active":"2024-02-17T11:56:00.143929+0000","last_peered":"2024-02-17T11:56:00.143929+0000","last_clean":"2024-02-17T11:56:00.143929+0000","last_became_active":"2024-02-17T11:55:58.688163+0000","last_became_peered":"2024-02-17T11:55:58.688163+0000","last_unstale":"2024-02-17T11:56:00.143929+0000","last_undegraded":"2024-02-17T11:56:00.143929+0000","last_fullsized":"2024-02-17T11:56:00.143929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:54.597116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.144164+0000","last_change":"2024-02-17T11:55:57.676860+0000","last_active":"2024-02-17T11:56:00.144164+0000","last_peered":"2024-02-17T11:56:00.144164+0000","last_clean":"2024-02-17T11:56:00.144164+0000","last_became_active":"2024-02-17T11:55:57.676534+0000","last_became_peered":"2024-02-17T11:55:57.676534+0000","last_unstale":"2024-02-17T11:56:00.144164+0000","last_undegraded":"2024-02-17T11:56:00.144164+0000","last_fullsized":"2024-02-17T11:56:00.144164+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:23:47.726264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143305+0000","last_change":"2024-02-17T11:55:58.679254+0000","last_active":"2024-02-17T11:56:00.143305+0000","last_peered":"2024-02-17T11:56:00.143305+0000","last_clean":"2024-02-17T11:56:00.143305+0000","last_became_active":"2024-02-17T11:55:58.678874+0000","last_became_peered":"2024-02-17T11:55:58.678874+0000","last_unstale":"2024-02-17T11:56:00.143305+0000","last_undegraded":"2024-02-17T11:56:00.143305+0000","last_fullsized":"2024-02-17T11:56:00.143305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:55:44.769818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657137+0000","last_change":"2024-02-17T11:55:57.672274+0000","last_active":"2024-02-17T11:55:59.657137+0000","last_peered":"2024-02-17T11:55:59.657137+0000","last_clean":"2024-02-17T11:55:59.657137+0000","last_became_active":"2024-02-17T11:55:57.671932+0000","last_became_peered":"2024-02-17T11:55:57.671932+0000","last_unstale":"2024-02-17T11:55:59.657137+0000","last_undegraded":"2024-02-17T11:55:59.657137+0000","last_fullsized":"2024-02-17T11:55:59.657137+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:20:41.973595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143601+0000","last_change":"2024-02-17T11:55:58.672985+0000","last_active":"2024-02-17T11:56:00.143601+0000","last_peered":"2024-02-17T11:56:00.143601+0000","last_clean":"2024-02-17T11:56:00.143601+0000","last_became_active":"2024-02-17T11:55:58.672622+0000","last_became_peered":"2024-02-17T11:55:58.672622+0000","last_unstale":"2024-02-17T11:56:00.143601+0000","last_undegraded":"2024-02-17T11:56:00.143601+0000","last_fullsized":"2024-02-17T11:56:00.143601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:45.555827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148641+0000","last_change":"2024-02-17T11:55:58.676127+0000","last_active":"2024-02-17T11:56:00.148641+0000","last_peered":"2024-02-17T11:56:00.148641+0000","last_clean":"2024-02-17T11:56:00.148641+0000","last_became_active":"2024-02-17T11:55:58.675784+0000","last_became_peered":"2024-02-17T11:55:58.675784+0000","last_unstale":"2024-02-17T11:56:00.148641+0000","last_undegraded":"2024-02-17T11:56:00.148641+0000","last_fullsized":"2024-02-17T11:56:00.148641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:56:32.648803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.662821+0000","last_change":"2024-02-17T11:55:58.682438+0000","last_active":"2024-02-17T11:55:59.662821+0000","last_peered":"2024-02-17T11:55:59.662821+0000","last_clean":"2024-02-17T11:55:59.662821+0000","last_became_active":"2024-02-17T11:55:58.682067+0000","last_became_peered":"2024-02-17T11:55:58.682067+0000","last_unstale":"2024-02-17T11:55:59.662821+0000","last_undegraded":"2024-02-17T11:55:59.662821+0000","last_fullsized":"2024-02-17T11:55:59.662821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:10:22.906450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.662484+0000","last_change":"2024-02-17T11:55:58.686671+0000","last_active":"2024-02-17T11:55:59.662484+0000","last_peered":"2024-02-17T11:55:59.662484+0000","last_clean":"2024-02-17T11:55:59.662484+0000","last_became_active":"2024-02-17T11:55:58.686315+0000","last_became_peered":"2024-02-17T11:55:58.686315+0000","last_unstale":"2024-02-17T11:55:59.662484+0000","last_undegraded":"2024-02-17T11:55:59.662484+0000","last_fullsized":"2024-02-17T11:55:59.662484+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:55:40.864847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-17T11:55:58.684709+0000","last_change":"2024-02-17T11:55:58.684227+0000","last_active":"2024-02-17T11:55:58.684709+0000","last_peered":"2024-02-17T11:55:58.684709+0000","last_clean":"2024-02-17T11:55:58.684709+0000","last_became_active":"2024-02-17T11:55:58.683875+0000","last_became_peered":"2024-02-17T11:55:58.683875+0000","last_unstale":"2024-02-17T11:55:58.684709+0000","last_undegraded":"2024-02-17T11:55:58.684709+0000","last_fullsized":"2024-02-17T11:55:58.684709+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:14:59.935238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657415+0000","last_change":"2024-02-17T11:55:58.685964+0000","last_active":"2024-02-17T11:55:59.657415+0000","last_peered":"2024-02-17T11:55:59.657415+0000","last_clean":"2024-02-17T11:55:59.657415+0000","last_became_active":"2024-02-17T11:55:58.685350+0000","last_became_peered":"2024-02-17T11:55:58.685350+0000","last_unstale":"2024-02-17T11:55:59.657415+0000","last_undegraded":"2024-02-17T11:55:59.657415+0000","last_fullsized":"2024-02-17T11:55:59.657415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:22:34.600604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.148280+0000","last_change":"2024-02-17T11:55:58.671291+0000","last_active":"2024-02-17T11:56:00.148280+0000","last_peered":"2024-02-17T11:56:00.148280+0000","last_clean":"2024-02-17T11:56:00.148280+0000","last_became_active":"2024-02-17T11:55:58.670925+0000","last_became_peered":"2024-02-17T11:55:58.670925+0000","last_unstale":"2024-02-17T11:56:00.148280+0000","last_undegraded":"2024-02-17T11:56:00.148280+0000","last_fullsized":"2024-02-17T11:56:00.148280+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:26:03.120976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.662972+0000","last_change":"2024-02-17T11:55:58.673122+0000","last_active":"2024-02-17T11:55:59.662972+0000","last_peered":"2024-02-17T11:55:59.662972+0000","last_clean":"2024-02-17T11:55:59.662972+0000","last_became_active":"2024-02-17T11:55:58.672747+0000","last_became_peered":"2024-02-17T11:55:58.672747+0000","last_unstale":"2024-02-17T11:55:59.662972+0000","last_undegraded":"2024-02-17T11:55:59.662972+0000","last_fullsized":"2024-02-17T11:55:59.662972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:55:15.064073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.139833+0000","last_change":"2024-02-17T11:55:58.681276+0000","last_active":"2024-02-17T11:56:00.139833+0000","last_peered":"2024-02-17T11:56:00.139833+0000","last_clean":"2024-02-17T11:56:00.139833+0000","last_became_active":"2024-02-17T11:55:58.680889+0000","last_became_peered":"2024-02-17T11:55:58.680889+0000","last_unstale":"2024-02-17T11:56:00.139833+0000","last_undegraded":"2024-02-17T11:56:00.139833+0000","last_fullsized":"2024-02-17T11:56:00.139833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:19:34.330288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.143934+0000","last_change":"2024-02-17T11:55:58.667475+0000","last_active":"2024-02-17T11:56:00.143934+0000","last_peered":"2024-02-17T11:56:00.143934+0000","last_clean":"2024-02-17T11:56:00.143934+0000","last_became_active":"2024-02-17T11:55:58.666577+0000","last_became_peered":"2024-02-17T11:55:58.666577+0000","last_unstale":"2024-02-17T11:56:00.143934+0000","last_undegraded":"2024-02-17T11:56:00.143934+0000","last_fullsized":"2024-02-17T11:56:00.143934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:51:35.732035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.140151+0000","last_change":"2024-02-17T11:55:58.674557+0000","last_active":"2024-02-17T11:56:00.140151+0000","last_peered":"2024-02-17T11:56:00.140151+0000","last_clean":"2024-02-17T11:56:00.140151+0000","last_became_active":"2024-02-17T11:55:58.673874+0000","last_became_peered":"2024-02-17T11:55:58.673874+0000","last_unstale":"2024-02-17T11:56:00.140151+0000","last_undegraded":"2024-02-17T11:56:00.140151+0000","last_fullsized":"2024-02-17T11:56:00.140151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:18:00.812489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.658591+0000","last_change":"2024-02-17T11:55:58.692725+0000","last_active":"2024-02-17T11:55:59.658591+0000","last_peered":"2024-02-17T11:55:59.658591+0000","last_clean":"2024-02-17T11:55:59.658591+0000","last_became_active":"2024-02-17T11:55:58.690880+0000","last_became_peered":"2024-02-17T11:55:58.690880+0000","last_unstale":"2024-02-17T11:55:59.658591+0000","last_undegraded":"2024-02-17T11:55:59.658591+0000","last_fullsized":"2024-02-17T11:55:59.658591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:03:44.724385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:55:59.657443+0000","last_change":"2024-02-17T11:55:58.679377+0000","last_active":"2024-02-17T11:55:59.657443+0000","last_peered":"2024-02-17T11:55:59.657443+0000","last_clean":"2024-02-17T11:55:59.657443+0000","last_became_active":"2024-02-17T11:55:58.679041+0000","last_became_peered":"2024-02-17T11:55:58.679041+0000","last_unstale":"2024-02-17T11:55:59.657443+0000","last_undegraded":"2024-02-17T11:55:59.657443+0000","last_fullsized":"2024-02-17T11:55:59.657443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:16:37.396424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-17T11:56:00.147521+0000","last_change":"2024-02-17T11:55:58.680665+0000","last_active":"2024-02-17T11:56:00.147521+0000","last_peered":"2024-02-17T11:56:00.147521+0000","last_clean":"2024-02-17T11:56:00.147521+0000","last_became_active":"2024-02-17T11:55:58.680311+0000","last_became_peered":"2024-02-17T11:55:58.680311+0000","last_unstale":"2024-02-17T11:56:00.147521+0000","last_undegraded":"2024-02-17T11:56:00.147521+0000","last_fullsized":"2024-02-17T11:56:00.147521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:19:11.455471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284587,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":41691,"num_objects":53,"num_object_clones":0,"num_object_copies":159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":9,"num_read_kb":25,"num_write":124,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253952,"data_stored":69743,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":917504,"data_stored":439103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645331,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":1039070,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":1,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.314},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.40799999999999997}]},{"osd":2,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.60199999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36099999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.374}]},{"osd":5,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65900000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.49399999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.41499999999999998}]},{"osd":8,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61699999999999999}]},{"osd":9,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.46500000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808855,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":439305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.78200000000000003}]},{"osd":1,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55900000000000005}]},{"osd":2,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.45800000000000002}]},{"osd":3,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.379},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.35499999999999998}]},{"osd":4,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67500000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.48299999999999998}]},{"osd":6,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.434},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.65900000000000003}]},{"osd":7,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.63400000000000001}]},{"osd":8,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.499},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.752}]},{"osd":10,"last update":"Sat Feb 17 11:55:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.069,"5min":1.069,"15min":1.069},"last":0.69099999999999995}]}]},{"osd":8,"up_from":59,"seq":253403070492,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":430555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44},{"interface":"front","average":{"1min":0.497,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.38100000000000001}]},{"osd":1,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.56299999999999994,"15min":0.55900000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57199999999999995,"15min":0.57499999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76900000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.35899999999999999}]},{"osd":2,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.44500000000000001,"15min":0.435},"min":{"1min":0.311,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.48499999999999999,"15min":0.48299999999999998},"min":{"1min":0.308,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78600000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.48199999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.58499999999999996,"15min":0.59499999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69899999999999995,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.376,"5min":0.372,"15min":0.372},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.52700000000000002}]},{"osd":4,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59899999999999998,"15min":0.60199999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73799999999999999,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78700000000000003,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.622}]},{"osd":5,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56399999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38400000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76200000000000001,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.56799999999999995}]},{"osd":6,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.439,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.67600000000000005,"15min":0.68500000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Sat Feb 17 11:55:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.68799999999999994,"15min":0.69199999999999995},"min":{"1min":0.5,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.68999999999999995}]},{"osd":9,"last update":"Sat Feb 17 11:55:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.73799999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:55:44 2024","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.70799999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]},{"osd":7,"up_from":51,"seq":219043332129,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":954368,"data_stored":1020901,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.36699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39600000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41699999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.81899999999999995,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56999999999999995,"15min":0.56799999999999995},"min":{"1min":0.39000000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81499999999999995,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.624}]},{"osd":3,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.59699999999999998,"15min":0.59299999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.55600000000000005}]},{"osd":4,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47099999999999997,"15min":0.46899999999999997},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52000000000000002,"15min":0.51500000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.81899999999999995}]},{"osd":5,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.65900000000000003,"15min":0.65500000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94399999999999995,"5min":1.333,"15min":1.333},"last":0.746},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.95399999999999996,"5min":1.486,"15min":1.486},"last":0.69599999999999995}]},{"osd":6,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59999999999999998,"15min":0.59299999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.68700000000000006,"15min":0.68200000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.85699999999999998}]},{"osd":8,"last update":"Sat Feb 17 11:55:56 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98599999999999999,"5min":1.446,"15min":1.446},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.89600000000000002,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.77900000000000003}]},{"osd":9,"last update":"Sat Feb 17 11:55:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.92000000000000004}]},{"osd":10,"last update":"Sat Feb 17 11:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.94299999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]}]},{"osd":6,"up_from":46,"seq":197568495655,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":917504,"data_stored":438611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.35199999999999998,"15min":0.34599999999999997},"min":{"1min":0.23499999999999999,"5min":0.223,"15min":0.223},"max":{"1min":0.51500000000000001,"5min":0.53900000000000003,"15min":0.53900000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61699999999999999,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.439}]},{"osd":1,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55300000000000005,"15min":0.54500000000000004},"min":{"1min":0.41199999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.502}]},{"osd":2,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.42699999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78300000000000003,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.58699999999999997,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82699999999999996,"5min":0.871,"15min":0.871},"last":0.624}]},{"osd":3,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.45900000000000002,"15min":0.45100000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.441,"15min":0.44500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.56299999999999994,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.42099999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.59799999999999998,"15min":0.59199999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.51900000000000002}]},{"osd":5,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61899999999999999,"15min":0.622},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.82199999999999995,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.66500000000000004,"15min":0.66000000000000003},"min":{"1min":0.57499999999999996,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.90600000000000003,"5min":0.997,"15min":0.997},"last":0.65000000000000002}]},{"osd":7,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.436,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75700000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.61399999999999999,"15min":0.60399999999999998},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55800000000000005}]},{"osd":8,"last update":"Sat Feb 17 11:55:58 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.67100000000000004}]},{"osd":9,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60599999999999998}]},{"osd":10,"last update":"Sat Feb 17 11:55:42 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.65900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":925696,"data_stored":440133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.50700000000000001,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Sat Feb 17 11:55:24 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.53200000000000003,"15min":0.51700000000000002},"min":{"1min":0.38900000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.71799999999999997}]},{"osd":3,"last update":"Sat Feb 17 11:55:49 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.56100000000000005,"15min":0.52300000000000002},"min":{"1min":0.42499999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54300000000000004,"15min":0.52000000000000002},"min":{"1min":0.40300000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58799999999999997}]},{"osd":4,"last update":"Sat Feb 17 11:55:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.42699999999999999,"15min":0.42099999999999999},"min":{"1min":0.29999999999999999,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67200000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.54400000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:55:40 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.59099999999999997,"15min":0.56799999999999995},"min":{"1min":0.44900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.60499999999999998,"15min":0.58499999999999996},"min":{"1min":0.51400000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.746}]},{"osd":6,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.66500000000000004}]},{"osd":7,"last update":"Sat Feb 17 11:55:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.54400000000000004,"15min":0.52500000000000002},"min":{"1min":0.40600000000000003,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.59799999999999998,"15min":0.58299999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.65000000000000002}]},{"osd":8,"last update":"Sat Feb 17 11:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.69199999999999995,"15min":0.67900000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.65300000000000002,"15min":0.65000000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.72899999999999998}]},{"osd":9,"last update":"Sat Feb 17 11:55:19 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.86299999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:55:44 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.84899999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738436,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":905216,"data_stored":430041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.47499999999999998,"15min":0.44700000000000001},"min":{"1min":0.38100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.48799999999999999,"15min":0.46700000000000003},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.40100000000000002}]},{"osd":2,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.54300000000000004,"15min":0.503},"min":{"1min":0.35899999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.54300000000000004,"15min":0.51500000000000001},"min":{"1min":0.39100000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60099999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:54:47 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.39400000000000002,"15min":0.379},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.44800000000000001,"15min":0.435},"min":{"1min":0.36299999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.747,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38}]},{"osd":4,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.52700000000000002,"15min":0.504},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.54200000000000004,"15min":0.51500000000000001},"min":{"1min":0.49399999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.433}]},{"osd":5,"last update":"Sat Feb 17 11:55:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.60999999999999999,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.60599999999999998,"15min":0.59399999999999997},"min":{"1min":0.40100000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.53600000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:55:04 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.622}]},{"osd":7,"last update":"Sat Feb 17 11:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.501,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58399999999999996}]},{"osd":8,"last update":"Sat Feb 17 11:55:56 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70999999999999996,"15min":0.71099999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.69799999999999995,"15min":0.69899999999999995},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.63500000000000001}]},{"osd":9,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.65200000000000002}]},{"osd":10,"last update":"Sat Feb 17 11:55:43 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57199999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":440789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.51400000000000001,"15min":0.49299999999999999},"min":{"1min":0.40400000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.51700000000000002,"15min":0.499},"min":{"1min":0.38500000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.46100000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51000000000000001,"15min":0.48099999999999998},"min":{"1min":0.33600000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.73599999999999999,"5min":0.751,"15min":0.751},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.51900000000000002,"15min":0.47699999999999998},"min":{"1min":0.33100000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85799999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70099999999999996}]},{"osd":3,"last update":"Sat Feb 17 11:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.56399999999999995,"15min":0.53900000000000003},"min":{"1min":0.44700000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.755,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52500000000000002,"15min":0.51300000000000001},"min":{"1min":0.35299999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.65000000000000002,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Feb 17 11:55:13 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54000000000000004,"15min":0.52500000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.747},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.56000000000000005,"15min":0.54200000000000004},"min":{"1min":0.42499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.79400000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.432,"15min":0.41999999999999998},"min":{"1min":0.30299999999999999,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.441,"15min":0.436},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.48199999999999998}]},{"osd":6,"last update":"Sat Feb 17 11:55:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58299999999999996,"15min":0.57499999999999996},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.79300000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.47099999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.501}]},{"osd":7,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63700000000000001}]},{"osd":8,"last update":"Sat Feb 17 11:55:52 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59499999999999997,"15min":0.59999999999999998},"min":{"1min":0.35499999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.60299999999999998,"15min":0.60599999999999998},"min":{"1min":0.42099999999999999,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73099999999999998}]},{"osd":9,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.42899999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:55:44 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.76400000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":3,"up_from":25,"seq":107374182453,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":438715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.34999999999999998,"15min":0.34100000000000003},"min":{"1min":0.27400000000000002,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.35199999999999998,"15min":0.34300000000000003},"min":{"1min":0.254,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.53700000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.30099999999999999}]},{"osd":1,"last update":"Sat Feb 17 11:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52500000000000002,"15min":0.53700000000000003},"min":{"1min":0.40000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.79100000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.54100000000000004,"15min":0.53500000000000003},"min":{"1min":0.443,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.64700000000000002}]},{"osd":2,"last update":"Sat Feb 17 11:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.57299999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78900000000000003,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.40000000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63700000000000001}]},{"osd":4,"last update":"Sat Feb 17 11:55:15 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.57799999999999996,"15min":0.56599999999999995},"min":{"1min":0.46600000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.53900000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.79200000000000004,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56899999999999995}]},{"osd":5,"last update":"Sat Feb 17 11:55:43 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.46400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.41899999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52000000000000002}]},{"osd":6,"last update":"Sat Feb 17 11:55:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.45100000000000001,"15min":0.44},"min":{"1min":0.315,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.49199999999999999,"15min":0.47699999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.50800000000000001}]},{"osd":7,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59499999999999997,"15min":0.58399999999999996},"min":{"1min":0.47399999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91600000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.47599999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.53700000000000003}]},{"osd":8,"last update":"Sat Feb 17 11:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.49199999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97599999999999998,"5min":1.214,"15min":1.214},"last":0.622}]},{"osd":9,"last update":"Sat Feb 17 11:55:19 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.45000000000000001}]},{"osd":10,"last update":"Sat Feb 17 11:55:47 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.58199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920817,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":439165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.50700000000000001,"15min":0.497},"min":{"1min":0.41099999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41599999999999998}]},{"osd":1,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38400000000000001,"15min":0.378},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.39600000000000002,"15min":0.38900000000000001},"min":{"1min":0.255,"5min":0.254,"15min":0.254},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.48099999999999998}]},{"osd":2,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53700000000000003,"15min":0.53900000000000003},"min":{"1min":0.39900000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.61199999999999999,"15min":0.55000000000000004},"min":{"1min":0.42799999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":7.0279999999999996,"5min":7.0279999999999996,"15min":7.0279999999999996},"last":0.91600000000000004}]},{"osd":3,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.75900000000000001,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78100000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.69799999999999995}]},{"osd":5,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79800000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.57799999999999996,"15min":0.55600000000000005},"min":{"1min":0.40600000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.96199999999999997}]},{"osd":6,"last update":"Sat Feb 17 11:55:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57399999999999995,"15min":0.56799999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.62},{"interface":"front","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.73899999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.53400000000000003,"15min":0.51100000000000001},"min":{"1min":0.33100000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.373,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66500000000000004}]},{"osd":8,"last update":"Sat Feb 17 11:55:56 2024","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":0.72299999999999998,"15min":0.67300000000000004},"min":{"1min":0.51900000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":6.9660000000000002,"5min":6.9660000000000002,"15min":6.9660000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.751,"5min":0.71099999999999997,"15min":0.70499999999999996},"min":{"1min":0.54400000000000004,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":1.0580000000000001}]},{"osd":9,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":1.028}]},{"osd":10,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":1.087},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.79700000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757292,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":962560,"data_stored":1020501,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.503,"15min":0.48699999999999999},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.50700000000000001,"15min":0.48299999999999998},"min":{"1min":0.41499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.75800000000000001,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.66800000000000004}]},{"osd":1,"last update":"Sat Feb 17 11:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53100000000000003,"15min":0.505},"min":{"1min":0.38500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.84799999999999998,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.52500000000000002,"15min":0.497},"min":{"1min":0.41499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.71099999999999997}]},{"osd":2,"last update":"Sat Feb 17 11:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.309,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.27100000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67300000000000004,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.41199999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.54400000000000004,"15min":0.52400000000000002},"min":{"1min":0.40100000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.52600000000000002},"min":{"1min":0.42199999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.73599999999999999,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.63500000000000001}]},{"osd":4,"last update":"Sat Feb 17 11:55:41 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.57099999999999995,"15min":0.55400000000000005},"min":{"1min":0.39600000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80900000000000005,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.57699999999999996,"15min":0.56000000000000005},"min":{"1min":0.38300000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.88100000000000001,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72799999999999998}]},{"osd":6,"last update":"Sat Feb 17 11:55:06 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.58999999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.81000000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.755}]},{"osd":7,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61399999999999999,"15min":0.60899999999999999},"min":{"1min":0.45500000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84399999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.64700000000000002,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.77300000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.68500000000000005}]},{"osd":8,"last update":"Sat Feb 17 11:55:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.98799999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65100000000000002,"15min":0.65900000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.91800000000000004,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.52000000000000002}]},{"osd":9,"last update":"Sat Feb 17 11:55:16 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.78400000000000003}]},{"osd":10,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.82899999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":20895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":40162,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T11:56:05.190 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-17T11:56:05.191 INFO:tasks.cephadm:Setting up client nodes... 2024-02-17T11:56:05.191 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-17T11:56:05.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:05 smithi125 ceph-mon[36819]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:05.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:05 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2564034638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:56:05.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:05 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3519428427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:56:05.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:05 smithi125 ceph-mon[36819]: pgmap v288: 97 pgs: 97 active+clean; 617 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-02-17T11:56:05.454 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:05 smithi094 ceph-mon[32499]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:05 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2564034638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:56:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:05 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3519428427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:56:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:05 smithi094 ceph-mon[32499]: pgmap v288: 97 pgs: 97 active+clean; 617 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-02-17T11:56:05.462 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:05.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:05 smithi189 ceph-mon[36588]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:56:05.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:05 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2564034638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:56:05.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:05 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3519428427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:56:05.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:05 smithi189 ceph-mon[36588]: pgmap v288: 97 pgs: 97 active+clean; 617 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9.9 KiB/s wr, 19 op/s 2024-02-17T11:56:06.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:06 smithi125 ceph-mon[36819]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:56:06.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:06 smithi125 ceph-mon[36819]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:56:06.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:06 smithi094 ceph-mon[32499]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:56:06.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:06 smithi094 ceph-mon[32499]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:56:06.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:06 smithi189 ceph-mon[36588]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:56:06.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:06 smithi189 ceph-mon[36588]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:56:07.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.105+0000 7f2120acb700 1 -- 172.21.15.94:0/2282725370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c103860 msgr2=0x7f211c10a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:07.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.105+0000 7f2120acb700 1 --2- 172.21.15.94:0/2282725370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c103860 0x7f211c10a110 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f211400b0d0 tx=0x7f211400b3e0 comp rx=0 tx=0).stop 2024-02-17T11:56:07.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.106+0000 7f2120acb700 1 -- 172.21.15.94:0/2282725370 shutdown_connections 2024-02-17T11:56:07.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.106+0000 7f2120acb700 1 --2- 172.21.15.94:0/2282725370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c103860 0x7f211c10a110 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:07.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.106+0000 7f2120acb700 1 --2- 172.21.15.94:0/2282725370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f211c101cc0 0x7f211c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:07.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.106+0000 7f2120acb700 1 --2- 172.21.15.94:0/2282725370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f211c102ec0 0x7f211c103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:07.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.106+0000 7f2120acb700 1 -- 172.21.15.94:0/2282725370 >> 172.21.15.94:0/2282725370 conn(0x7f211c0fd280 msgr2=0x7f211c0ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:07.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.106+0000 7f2120acb700 1 -- 172.21.15.94:0/2282725370 shutdown_connections 2024-02-17T11:56:07.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.106+0000 7f2120acb700 1 -- 172.21.15.94:0/2282725370 wait complete. 2024-02-17T11:56:07.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.106+0000 7f2120acb700 1 Processor -- start 2024-02-17T11:56:07.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.106+0000 7f2120acb700 1 -- start start 2024-02-17T11:56:07.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f2120acb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c101cc0 0x7f211c075a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:07.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f2120acb700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f211c102ec0 0x7f211c074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:07.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f2120acb700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f211c103860 0x7f211c0745a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:07.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f2120acb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c074d20 con 0x7f211c101cc0 2024-02-17T11:56:07.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f2120acb700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c074e90 con 0x7f211c102ec0 2024-02-17T11:56:07.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f2120acb700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c1a1ca0 con 0x7f211c103860 2024-02-17T11:56:07.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f211a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c101cc0 0x7f211c075a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:07.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f211a59c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c101cc0 0x7f211c075a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47846/0 (socket says 172.21.15.94:47846) 2024-02-17T11:56:07.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f211a59c700 1 -- 172.21.15.94:0/612316770 learned_addr learned my addr 172.21.15.94:0/612316770 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:07.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f211ad9d700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f211c103860 0x7f211c0745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:07.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f2119d9b700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f211c102ec0 0x7f211c074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:07.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f211a59c700 1 -- 172.21.15.94:0/612316770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f211c103860 msgr2=0x7f211c0745a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:07.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f211a59c700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f211c103860 0x7f211c0745a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f211a59c700 1 -- 172.21.15.94:0/612316770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f211c102ec0 msgr2=0x7f211c074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f211a59c700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f211c102ec0 0x7f211c074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f211a59c700 1 -- 172.21.15.94:0/612316770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2114009d00 con 0x7f211c101cc0 2024-02-17T11:56:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f211a59c700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c101cc0 0x7f211c075a10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f211000c970 tx=0x7f211000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f210b7fe700 1 -- 172.21.15.94:0/612316770 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110004220 con 0x7f211c101cc0 2024-02-17T11:56:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f210b7fe700 1 -- 172.21.15.94:0/612316770 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2110004380 con 0x7f211c101cc0 2024-02-17T11:56:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.107+0000 7f210b7fe700 1 -- 172.21.15.94:0/612316770 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110003c00 con 0x7f211c101cc0 2024-02-17T11:56:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.108+0000 7f2120acb700 1 -- 172.21.15.94:0/612316770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c1a1f80 con 0x7f211c101cc0 2024-02-17T11:56:07.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.108+0000 7f2120acb700 1 -- 172.21.15.94:0/612316770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c1a2390 con 0x7f211c101cc0 2024-02-17T11:56:07.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.109+0000 7f2120acb700 1 -- 172.21.15.94:0/612316770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211c108fb0 con 0x7f211c101cc0 2024-02-17T11:56:07.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.110+0000 7f210b7fe700 1 -- 172.21.15.94:0/612316770 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f21100076d0 con 0x7f211c101cc0 2024-02-17T11:56:07.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.110+0000 7f210b7fe700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f21040a2290 0x7f21040a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:07.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.110+0000 7f2119d9b700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f21040a2290 0x7f21040a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:07.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.110+0000 7f210b7fe700 1 -- 172.21.15.94:0/612316770 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f21100cd420 con 0x7f211c101cc0 2024-02-17T11:56:07.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.111+0000 7f2119d9b700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f21040a2290 0x7f21040a4750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f210c005950 tx=0x7f210c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:07.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.115+0000 7f210b7fe700 1 -- 172.21.15.94:0/612316770 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2110096440 con 0x7f211c101cc0 2024-02-17T11:56:07.388 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:07 smithi094 ceph-mon[32499]: pgmap v289: 97 pgs: 97 active+clean; 618 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.9 KiB/s wr, 21 op/s 2024-02-17T11:56:07.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.389+0000 7f2120acb700 1 -- 172.21.15.94:0/612316770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f211c10d6c0 con 0x7f211c101cc0 2024-02-17T11:56:07.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.391+0000 7f210b7fe700 1 -- 172.21.15.94:0/612316770 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f2110099b00 con 0x7f211c101cc0 2024-02-17T11:56:07.390 INFO:teuthology.orchestra.run.smithi094.stdout:[client.0] 2024-02-17T11:56:07.390 INFO:teuthology.orchestra.run.smithi094.stdout: key = AQDXntBlB79OFxAAyN2V2s4dCXO99IxAMkHmyA== 2024-02-17T11:56:07.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.393+0000 7f2120acb700 1 -- 172.21.15.94:0/612316770 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f21040a2290 msgr2=0x7f21040a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:07.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.393+0000 7f2120acb700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f21040a2290 0x7f21040a4750 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f210c005950 tx=0x7f210c00b410 comp rx=0 tx=0).stop 2024-02-17T11:56:07.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.393+0000 7f2120acb700 1 -- 172.21.15.94:0/612316770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c101cc0 msgr2=0x7f211c075a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:07.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.393+0000 7f2120acb700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c101cc0 0x7f211c075a10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f211000c970 tx=0x7f211000cc80 comp rx=0 tx=0).stop 2024-02-17T11:56:07.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.393+0000 7f2120acb700 1 -- 172.21.15.94:0/612316770 shutdown_connections 2024-02-17T11:56:07.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.393+0000 7f2120acb700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f21040a2290 0x7f21040a4750 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:07.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.393+0000 7f2120acb700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f211c103860 0x7f211c0745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:07.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.394+0000 7f2120acb700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f211c101cc0 0x7f211c075a10 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:07.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.394+0000 7f2120acb700 1 --2- 172.21.15.94:0/612316770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f211c102ec0 0x7f211c074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:07.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.394+0000 7f2120acb700 1 -- 172.21.15.94:0/612316770 >> 172.21.15.94:0/612316770 conn(0x7f211c0fd280 msgr2=0x7f211c1060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:07.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.394+0000 7f2120acb700 1 -- 172.21.15.94:0/612316770 shutdown_connections 2024-02-17T11:56:07.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:07.394+0000 7f2120acb700 1 -- 172.21.15.94:0/612316770 wait complete. 2024-02-17T11:56:07.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:07 smithi125 ceph-mon[36819]: pgmap v289: 97 pgs: 97 active+clean; 618 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.9 KiB/s wr, 21 op/s 2024-02-17T11:56:07.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:07 smithi189 ceph-mon[36588]: pgmap v289: 97 pgs: 97 active+clean; 618 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.9 KiB/s wr, 21 op/s 2024-02-17T11:56:08.051 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:56:08.051 DEBUG:teuthology.orchestra.run.smithi094:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-17T11:56:08.051 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-17T11:56:08.095 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-17T11:56:08.095 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-17T11:56:08.095 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph mgr dump --format=json 2024-02-17T11:56:08.373 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:08.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:08 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/612316770' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-17T11:56:08.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:08 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/612316770' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-17T11:56:08.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:08 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/612316770' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-17T11:56:08.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:08 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/612316770' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-17T11:56:08.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:08 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/612316770' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-17T11:56:08.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:08 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/612316770' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-17T11:56:09.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:09 smithi125 ceph-mon[36819]: pgmap v290: 97 pgs: 97 active+clean; 618 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5 KiB/s rd, 8.2 KiB/s wr, 18 op/s 2024-02-17T11:56:09.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:09 smithi094 ceph-mon[32499]: pgmap v290: 97 pgs: 97 active+clean; 618 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5 KiB/s rd, 8.2 KiB/s wr, 18 op/s 2024-02-17T11:56:09.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:09 smithi189 ceph-mon[36588]: pgmap v290: 97 pgs: 97 active+clean; 618 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5 KiB/s rd, 8.2 KiB/s wr, 18 op/s 2024-02-17T11:56:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.043+0000 7f381c2f9700 1 -- 172.21.15.94:0/3281674099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3814107cc0 msgr2=0x7f38141080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.043+0000 7f381c2f9700 1 --2- 172.21.15.94:0/3281674099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3814107cc0 0x7f38141080a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f38100099c0 tx=0x7f3810009cd0 comp rx=0 tx=0).stop 2024-02-17T11:56:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.043+0000 7f381c2f9700 1 -- 172.21.15.94:0/3281674099 shutdown_connections 2024-02-17T11:56:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.043+0000 7f381c2f9700 1 --2- 172.21.15.94:0/3281674099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f381406a2c0 0x7f381410eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.043+0000 7f381c2f9700 1 --2- 172.21.15.94:0/3281674099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3814107cc0 0x7f38141080a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:10.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.043+0000 7f381c2f9700 1 --2- 172.21.15.94:0/3281674099 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814069920 0x7f3814069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:10.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.043+0000 7f381c2f9700 1 -- 172.21.15.94:0/3281674099 >> 172.21.15.94:0/3281674099 conn(0x7f3814077aa0 msgr2=0x7f3814077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:10.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.043+0000 7f381c2f9700 1 -- 172.21.15.94:0/3281674099 shutdown_connections 2024-02-17T11:56:10.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.043+0000 7f381c2f9700 1 -- 172.21.15.94:0/3281674099 wait complete. 2024-02-17T11:56:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381c2f9700 1 Processor -- start 2024-02-17T11:56:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381c2f9700 1 -- start start 2024-02-17T11:56:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381c2f9700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3814069920 0x7f3814100c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381c2f9700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f381406a2c0 0x7f3814101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381c2f9700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814107cc0 0x7f38141016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381c2f9700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141a65b0 con 0x7f3814069920 2024-02-17T11:56:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381c2f9700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141a6720 con 0x7f3814107cc0 2024-02-17T11:56:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381c2f9700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38141a68e0 con 0x7f381406a2c0 2024-02-17T11:56:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381a095700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3814069920 0x7f3814100c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381a095700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3814069920 0x7f3814100c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47864/0 (socket says 172.21.15.94:47864) 2024-02-17T11:56:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381a095700 1 -- 172.21.15.94:0/816813534 learned_addr learned my addr 172.21.15.94:0/816813534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:10.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381a896700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814107cc0 0x7f38141016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:10.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f3819894700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f381406a2c0 0x7f3814101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:10.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381a095700 1 -- 172.21.15.94:0/816813534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f381406a2c0 msgr2=0x7f3814101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:10.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381a095700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f381406a2c0 0x7f3814101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:10.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381a095700 1 -- 172.21.15.94:0/816813534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814107cc0 msgr2=0x7f38141016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:10.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381a095700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814107cc0 0x7f38141016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:10.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381a095700 1 -- 172.21.15.94:0/816813534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38100095f0 con 0x7f3814069920 2024-02-17T11:56:10.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f3819894700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f381406a2c0 0x7f3814101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:10.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381a896700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814107cc0 0x7f38141016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:56:10.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.044+0000 7f381a095700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3814069920 0x7f3814100c40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f381000b580 tx=0x7f3810005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:10.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.045+0000 7f380b7fe700 1 -- 172.21.15.94:0/816813534 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381001d070 con 0x7f3814069920 2024-02-17T11:56:10.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.045+0000 7f381c2f9700 1 -- 172.21.15.94:0/816813534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38141a6b60 con 0x7f3814069920 2024-02-17T11:56:10.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.045+0000 7f380b7fe700 1 -- 172.21.15.94:0/816813534 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3810003fd0 con 0x7f3814069920 2024-02-17T11:56:10.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.045+0000 7f380b7fe700 1 -- 172.21.15.94:0/816813534 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810022e50 con 0x7f3814069920 2024-02-17T11:56:10.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.045+0000 7f381c2f9700 1 -- 172.21.15.94:0/816813534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38141a6f70 con 0x7f3814069920 2024-02-17T11:56:10.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.046+0000 7f381c2f9700 1 -- 172.21.15.94:0/816813534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381410a1e0 con 0x7f3814069920 2024-02-17T11:56:10.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.047+0000 7f380b7fe700 1 -- 172.21.15.94:0/816813534 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f3810022590 con 0x7f3814069920 2024-02-17T11:56:10.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.047+0000 7f380b7fe700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f38000a21e0 0x7f38000a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:10.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.047+0000 7f3819894700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f38000a21e0 0x7f38000a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:10.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.047+0000 7f380b7fe700 1 -- 172.21.15.94:0/816813534 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f38100cef70 con 0x7f3814069920 2024-02-17T11:56:10.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.048+0000 7f3819894700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f38000a21e0 0x7f38000a46a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3804005fd0 tx=0x7f3804005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:10.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.051+0000 7f380b7fe700 1 -- 172.21.15.94:0/816813534 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3810098020 con 0x7f3814069920 2024-02-17T11:56:10.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.304+0000 7f381c2f9700 1 -- 172.21.15.94:0/816813534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3814111da0 con 0x7f3814069920 2024-02-17T11:56:10.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.308+0000 7f380b7fe700 1 -- 172.21.15.94:0/816813534 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+261484 (secure 0 0 0) 0x7f381009b6e0 con 0x7f3814069920 2024-02-17T11:56:10.308 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:56:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.311+0000 7f381c2f9700 1 -- 172.21.15.94:0/816813534 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f38000a21e0 msgr2=0x7f38000a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.311+0000 7f381c2f9700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f38000a21e0 0x7f38000a46a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3804005fd0 tx=0x7f3804005ee0 comp rx=0 tx=0).stop 2024-02-17T11:56:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.311+0000 7f381c2f9700 1 -- 172.21.15.94:0/816813534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3814069920 msgr2=0x7f3814100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.311+0000 7f381c2f9700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3814069920 0x7f3814100c40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f381000b580 tx=0x7f3810005e80 comp rx=0 tx=0).stop 2024-02-17T11:56:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.311+0000 7f381c2f9700 1 -- 172.21.15.94:0/816813534 shutdown_connections 2024-02-17T11:56:10.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.311+0000 7f381c2f9700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f38000a21e0 0x7f38000a46a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.311+0000 7f381c2f9700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3814107cc0 0x7f38141016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.311+0000 7f381c2f9700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3814069920 0x7f3814100c40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.311+0000 7f381c2f9700 1 --2- 172.21.15.94:0/816813534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f381406a2c0 0x7f3814101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.311+0000 7f381c2f9700 1 -- 172.21.15.94:0/816813534 >> 172.21.15.94:0/816813534 conn(0x7f3814077aa0 msgr2=0x7f38140ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.311+0000 7f381c2f9700 1 -- 172.21.15.94:0/816813534 shutdown_connections 2024-02-17T11:56:10.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:10.311+0000 7f381c2f9700 1 -- 172.21.15.94:0/816813534 wait complete. 2024-02-17T11:56:10.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:10 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/816813534' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-17T11:56:10.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:10 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/816813534' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-17T11:56:10.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:10 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/816813534' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-17T11:56:10.952 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6800","nonce":1808883811},{"type":"v1","addr":"172.21.15.94:6801","nonce":1808883811}]},"active_addr":"172.21.15.94:6801/1808883811","active_change":"2024-02-17T11:48:58.555034+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.94:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":1603490109}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":2639307451}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":875984108}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":2865658350}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.94:0","nonce":2019886303}]}]} 2024-02-17T11:56:10.957 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-17T11:56:10.957 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-17T11:56:10.957 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-17T11:56:11.196 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:11.577 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:11 smithi094 ceph-mon[32499]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.8 KiB/s wr, 18 op/s 2024-02-17T11:56:11.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:11 smithi189 ceph-mon[36588]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.8 KiB/s wr, 18 op/s 2024-02-17T11:56:11.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:11 smithi125 ceph-mon[36819]: pgmap v291: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 7.8 KiB/s wr, 18 op/s 2024-02-17T11:56:12.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.906+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/3527740037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3810e670 msgr2=0x7f5e38110a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:12.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.906+0000 7f5e3e8fb700 1 --2- 172.21.15.94:0/3527740037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3810e670 0x7f5e38110a60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b0a0 tx=0x7f5e3000b3b0 comp rx=0 tx=0).stop 2024-02-17T11:56:12.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.906+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/3527740037 shutdown_connections 2024-02-17T11:56:12.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.906+0000 7f5e3e8fb700 1 --2- 172.21.15.94:0/3527740037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e3810e670 0x7f5e38110a60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:12.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.906+0000 7f5e3e8fb700 1 --2- 172.21.15.94:0/3527740037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e38073ea0 0x7f5e38074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:12.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.906+0000 7f5e3e8fb700 1 --2- 172.21.15.94:0/3527740037 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e380747c0 0x7f5e3810e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:12.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.906+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/3527740037 >> 172.21.15.94:0/3527740037 conn(0x7f5e38078da0 msgr2=0x7f5e380791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:12.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.906+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/3527740037 shutdown_connections 2024-02-17T11:56:12.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.906+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/3527740037 wait complete. 2024-02-17T11:56:12.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e3e8fb700 1 Processor -- start 2024-02-17T11:56:12.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e3e8fb700 1 -- start start 2024-02-17T11:56:12.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e3e8fb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e38073ea0 0x7f5e3819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:12.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e3e8fb700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e380747c0 0x7f5e3819a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:12.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e3e8fb700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e3810e670 0x7f5e3819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:12.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e3e8fb700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3819f2c0 con 0x7f5e38073ea0 2024-02-17T11:56:12.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e3e8fb700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3819f430 con 0x7f5e380747c0 2024-02-17T11:56:12.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e3e8fb700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3819f5f0 con 0x7f5e3810e670 2024-02-17T11:56:12.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e37fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e38073ea0 0x7f5e3819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:12.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e37fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e38073ea0 0x7f5e3819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47892/0 (socket says 172.21.15.94:47892) 2024-02-17T11:56:12.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e37fff700 1 -- 172.21.15.94:0/403766197 learned_addr learned my addr 172.21.15.94:0/403766197 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:12.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e3ce98700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e3810e670 0x7f5e3819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:12.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e377fe700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e380747c0 0x7f5e3819a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:12.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e37fff700 1 -- 172.21.15.94:0/403766197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e3810e670 msgr2=0x7f5e3819ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:12.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e37fff700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e3810e670 0x7f5e3819ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:12.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e37fff700 1 -- 172.21.15.94:0/403766197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e380747c0 msgr2=0x7f5e3819a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:12.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e37fff700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e380747c0 0x7f5e3819a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:12.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e37fff700 1 -- 172.21.15.94:0/403766197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e30009d00 con 0x7f5e38073ea0 2024-02-17T11:56:12.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.907+0000 7f5e377fe700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e380747c0 0x7f5e3819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:12.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.908+0000 7f5e37fff700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e38073ea0 0x7f5e3819a3e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800c9b0 tx=0x7f5e2800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:12.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.908+0000 7f5e357fa700 1 -- 172.21.15.94:0/403766197 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28007b00 con 0x7f5e38073ea0 2024-02-17T11:56:12.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.908+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/403766197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3819f8d0 con 0x7f5e38073ea0 2024-02-17T11:56:12.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.908+0000 7f5e357fa700 1 -- 172.21.15.94:0/403766197 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e28007c60 con 0x7f5e38073ea0 2024-02-17T11:56:12.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.908+0000 7f5e357fa700 1 -- 172.21.15.94:0/403766197 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28018730 con 0x7f5e38073ea0 2024-02-17T11:56:12.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.908+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/403766197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e381a7130 con 0x7f5e38073ea0 2024-02-17T11:56:12.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.909+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/403766197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e38068930 con 0x7f5e38073ea0 2024-02-17T11:56:12.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.910+0000 7f5e357fa700 1 -- 172.21.15.94:0/403766197 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f5e28018890 con 0x7f5e38073ea0 2024-02-17T11:56:12.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.910+0000 7f5e357fa700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5e240a21e0 0x7f5e240a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:12.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.911+0000 7f5e377fe700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5e240a21e0 0x7f5e240a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:12.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.911+0000 7f5e357fa700 1 -- 172.21.15.94:0/403766197 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5e280cdd50 con 0x7f5e38073ea0 2024-02-17T11:56:12.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.911+0000 7f5e377fe700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5e240a21e0 0x7f5e240a46a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5e38110170 tx=0x7f5e20009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:12.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:12.915+0000 7f5e357fa700 1 -- 172.21.15.94:0/403766197 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5e28096d70 con 0x7f5e38073ea0 2024-02-17T11:56:13.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.107+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/403766197 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e38113f30 con 0x7f5e38073ea0 2024-02-17T11:56:13.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.109+0000 7f5e357fa700 1 -- 172.21.15.94:0/403766197 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22207 (secure 0 0 0) 0x7f5e2809a430 con 0x7f5e38073ea0 2024-02-17T11:56:13.108 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:56:13.108 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":85,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:55:59.654265+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:56:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.111+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/403766197 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5e240a21e0 msgr2=0x7f5e240a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.111+0000 7f5e3e8fb700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5e240a21e0 0x7f5e240a46a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5e38110170 tx=0x7f5e20009550 comp rx=0 tx=0).stop 2024-02-17T11:56:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.111+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/403766197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e38073ea0 msgr2=0x7f5e3819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.111+0000 7f5e3e8fb700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e38073ea0 0x7f5e3819a3e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800c9b0 tx=0x7f5e2800cd70 comp rx=0 tx=0).stop 2024-02-17T11:56:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.111+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/403766197 shutdown_connections 2024-02-17T11:56:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.111+0000 7f5e3e8fb700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5e240a21e0 0x7f5e240a46a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.111+0000 7f5e3e8fb700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e3810e670 0x7f5e3819ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.111+0000 7f5e3e8fb700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e38073ea0 0x7f5e3819a3e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:13.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.111+0000 7f5e3e8fb700 1 --2- 172.21.15.94:0/403766197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e380747c0 0x7f5e3819a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:13.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.111+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/403766197 >> 172.21.15.94:0/403766197 conn(0x7f5e38078da0 msgr2=0x7f5e380feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:13.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.111+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/403766197 shutdown_connections 2024-02-17T11:56:13.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:13.111+0000 7f5e3e8fb700 1 -- 172.21.15.94:0/403766197 wait complete. 2024-02-17T11:56:13.796 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-17T11:56:13.796 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-17T11:56:14.039 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:14.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:13 smithi189 ceph-mon[36588]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-02-17T11:56:14.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:13 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/403766197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:56:14.161 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:13 smithi094 ceph-mon[32499]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-02-17T11:56:14.162 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:13 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/403766197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:56:14.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:13 smithi125 ceph-mon[36819]: pgmap v292: 97 pgs: 97 active+clean; 623 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-02-17T11:56:14.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:13 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/403766197' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:56:15.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.705+0000 7f0d26a43700 1 -- 172.21.15.94:0/3869542477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20101510 msgr2=0x7f0d2010d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:15.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.705+0000 7f0d26a43700 1 --2- 172.21.15.94:0/3869542477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20101510 0x7f0d2010d9b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800b0a0 tx=0x7f0d1800b3b0 comp rx=0 tx=0).stop 2024-02-17T11:56:15.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.705+0000 7f0d26a43700 1 -- 172.21.15.94:0/3869542477 shutdown_connections 2024-02-17T11:56:15.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.705+0000 7f0d26a43700 1 --2- 172.21.15.94:0/3869542477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20101510 0x7f0d2010d9b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:15.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.705+0000 7f0d26a43700 1 --2- 172.21.15.94:0/3869542477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d20105ab0 0x7f0d20105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:15.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.705+0000 7f0d26a43700 1 --2- 172.21.15.94:0/3869542477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d20100b50 0x7f0d20100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:15.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.705+0000 7f0d26a43700 1 -- 172.21.15.94:0/3869542477 >> 172.21.15.94:0/3869542477 conn(0x7f0d20077b80 msgr2=0x7f0d20077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:15.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.705+0000 7f0d26a43700 1 -- 172.21.15.94:0/3869542477 shutdown_connections 2024-02-17T11:56:15.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.705+0000 7f0d26a43700 1 -- 172.21.15.94:0/3869542477 wait complete. 2024-02-17T11:56:15.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d26a43700 1 Processor -- start 2024-02-17T11:56:15.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d26a43700 1 -- start start 2024-02-17T11:56:15.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d26a43700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20100b50 0x7f0d20198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:15.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d26a43700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d20101510 0x7f0d20198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:15.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d26a43700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d20105ab0 0x7f0d2019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:15.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d26a43700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2019d0a0 con 0x7f0d20100b50 2024-02-17T11:56:15.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d26a43700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2019d210 con 0x7f0d20105ab0 2024-02-17T11:56:15.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d26a43700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2019d3d0 con 0x7f0d20101510 2024-02-17T11:56:15.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d1ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20100b50 0x7f0d20198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:15.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d1ffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20100b50 0x7f0d20198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46962/0 (socket says 172.21.15.94:46962) 2024-02-17T11:56:15.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d1ffff700 1 -- 172.21.15.94:0/3799528665 learned_addr learned my addr 172.21.15.94:0/3799528665 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:15.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d24fe0700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d20105ab0 0x7f0d2019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:15.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d1ffff700 1 -- 172.21.15.94:0/3799528665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d20101510 msgr2=0x7f0d20198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:15.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d1ffff700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d20101510 0x7f0d20198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:15.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d1ffff700 1 -- 172.21.15.94:0/3799528665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d20105ab0 msgr2=0x7f0d2019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:15.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d1ffff700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d20105ab0 0x7f0d2019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:15.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d1ffff700 1 -- 172.21.15.94:0/3799528665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d18009d00 con 0x7f0d20100b50 2024-02-17T11:56:15.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d24fe0700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d20105ab0 0x7f0d2019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:56:15.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.706+0000 7f0d1ffff700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20100b50 0x7f0d20198210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000c9b0 tx=0x7f0d1000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:15.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.707+0000 7f0d1d7fa700 1 -- 172.21.15.94:0/3799528665 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10007b00 con 0x7f0d20100b50 2024-02-17T11:56:15.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.707+0000 7f0d26a43700 1 -- 172.21.15.94:0/3799528665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d2019d6b0 con 0x7f0d20100b50 2024-02-17T11:56:15.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.707+0000 7f0d1d7fa700 1 -- 172.21.15.94:0/3799528665 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0d10007c60 con 0x7f0d20100b50 2024-02-17T11:56:15.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.707+0000 7f0d1d7fa700 1 -- 172.21.15.94:0/3799528665 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10018730 con 0x7f0d20100b50 2024-02-17T11:56:15.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.707+0000 7f0d26a43700 1 -- 172.21.15.94:0/3799528665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d201a4f10 con 0x7f0d20100b50 2024-02-17T11:56:15.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.708+0000 7f0d26a43700 1 -- 172.21.15.94:0/3799528665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d20068930 con 0x7f0d20100b50 2024-02-17T11:56:15.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.709+0000 7f0d1d7fa700 1 -- 172.21.15.94:0/3799528665 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f0d10018890 con 0x7f0d20100b50 2024-02-17T11:56:15.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.709+0000 7f0d1d7fa700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0d080a21e0 0x7f0d080a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:15.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.710+0000 7f0d1f7fe700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0d080a21e0 0x7f0d080a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:15.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.710+0000 7f0d1d7fa700 1 -- 172.21.15.94:0/3799528665 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f0d100cdd40 con 0x7f0d20100b50 2024-02-17T11:56:15.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.710+0000 7f0d1f7fe700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0d080a21e0 0x7f0d080a46a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0d14005950 tx=0x7f0d140058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:15.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.714+0000 7f0d1d7fa700 1 -- 172.21.15.94:0/3799528665 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d10096d60 con 0x7f0d20100b50 2024-02-17T11:56:15.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.908+0000 7f0d26a43700 1 -- 172.21.15.94:0/3799528665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d20110c70 con 0x7f0d20100b50 2024-02-17T11:56:15.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.910+0000 7f0d1d7fa700 1 -- 172.21.15.94:0/3799528665 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22207 (secure 0 0 0) 0x7f0d1009a420 con 0x7f0d20100b50 2024-02-17T11:56:15.909 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:56:15.909 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":85,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:55:59.654265+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:56:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.912+0000 7f0d26a43700 1 -- 172.21.15.94:0/3799528665 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0d080a21e0 msgr2=0x7f0d080a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.912+0000 7f0d26a43700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0d080a21e0 0x7f0d080a46a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0d14005950 tx=0x7f0d140058e0 comp rx=0 tx=0).stop 2024-02-17T11:56:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.912+0000 7f0d26a43700 1 -- 172.21.15.94:0/3799528665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20100b50 msgr2=0x7f0d20198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.912+0000 7f0d26a43700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20100b50 0x7f0d20198210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000c9b0 tx=0x7f0d1000cd70 comp rx=0 tx=0).stop 2024-02-17T11:56:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.912+0000 7f0d26a43700 1 -- 172.21.15.94:0/3799528665 shutdown_connections 2024-02-17T11:56:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.912+0000 7f0d26a43700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0d080a21e0 0x7f0d080a46a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.912+0000 7f0d26a43700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0d20105ab0 0x7f0d2019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.912+0000 7f0d26a43700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0d20100b50 0x7f0d20198210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.912+0000 7f0d26a43700 1 --2- 172.21.15.94:0/3799528665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d20101510 0x7f0d20198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.912+0000 7f0d26a43700 1 -- 172.21.15.94:0/3799528665 >> 172.21.15.94:0/3799528665 conn(0x7f0d20077b80 msgr2=0x7f0d201048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:15.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.912+0000 7f0d26a43700 1 -- 172.21.15.94:0/3799528665 shutdown_connections 2024-02-17T11:56:15.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:15.912+0000 7f0d26a43700 1 -- 172.21.15.94:0/3799528665 wait complete. 2024-02-17T11:56:16.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:15 smithi189 ceph-mon[36588]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 KiB/s wr, 12 op/s 2024-02-17T11:56:16.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:15 smithi125 ceph-mon[36819]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 KiB/s wr, 12 op/s 2024-02-17T11:56:16.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:15 smithi094 ceph-mon[32499]: pgmap v293: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.3 KiB/s wr, 12 op/s 2024-02-17T11:56:16.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-17T11:56:16.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-17T11:56:16.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-17T11:56:16.607 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-17T11:56:16.607 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-17T11:56:16.608 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-17T11:56:16.608 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-17T11:56:16.608 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-17T11:56:16.609 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-02-17T11:56:16.609 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-02-17T11:56:16.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-02-17T11:56:16.610 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-02-17T11:56:16.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:16 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3799528665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:56:17.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:16 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3799528665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:56:17.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:16 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3799528665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:56:17.253 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:17.322 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:17.349 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:17.350 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:18.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:17 smithi189 ceph-mon[36588]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-02-17T11:56:18.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:17 smithi125 ceph-mon[36819]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-02-17T11:56:18.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:17 smithi094 ceph-mon[32499]: pgmap v294: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-02-17T11:56:18.622 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:18.622 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:18.623 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:18.624 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:18.629 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:18.634 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:18.645 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:20.013 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:19 smithi094 ceph-mon[32499]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-02-17T11:56:20.030 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:19 smithi125 ceph-mon[36819]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-02-17T11:56:20.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:19 smithi189 ceph-mon[36588]: pgmap v295: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-02-17T11:56:21.996 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:21 smithi094 ceph-mon[32499]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-02-17T11:56:22.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:21 smithi189 ceph-mon[36588]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-02-17T11:56:22.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:21 smithi125 ceph-mon[36819]: pgmap v296: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-02-17T11:56:23.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.530+0000 7efcbc1ce700 1 -- 172.21.15.94:0/1672627248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcb4100e50 msgr2=0x7efcb41012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:23.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.530+0000 7efcbc1ce700 1 --2- 172.21.15.94:0/1672627248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcb4100e50 0x7efcb41012d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efca40099c0 tx=0x7efca4009cd0 comp rx=0 tx=0).stop 2024-02-17T11:56:23.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.530+0000 7efcbc1ce700 1 -- 172.21.15.94:0/1672627248 shutdown_connections 2024-02-17T11:56:23.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.530+0000 7efcbc1ce700 1 --2- 172.21.15.94:0/1672627248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efcb4101810 0x7efcb410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:23.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.530+0000 7efcbc1ce700 1 --2- 172.21.15.94:0/1672627248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efcb4106ca0 0x7efcb4107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:23.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.530+0000 7efcbc1ce700 1 --2- 172.21.15.94:0/1672627248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcb4100e50 0x7efcb41012d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:23.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.530+0000 7efcbc1ce700 1 -- 172.21.15.94:0/1672627248 >> 172.21.15.94:0/1672627248 conn(0x7efcb4077a30 msgr2=0x7efcb4077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.530+0000 7efcbc1ce700 1 -- 172.21.15.94:0/1672627248 shutdown_connections 2024-02-17T11:56:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.530+0000 7efcbc1ce700 1 -- 172.21.15.94:0/1672627248 wait complete. 2024-02-17T11:56:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.531+0000 7efcbc1ce700 1 Processor -- start 2024-02-17T11:56:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.531+0000 7efcbc1ce700 1 -- start start 2024-02-17T11:56:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.531+0000 7efcbc1ce700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efcb4100e50 0x7efcb4199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.531+0000 7efcbc1ce700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcb4101810 0x7efcb4199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.531+0000 7efcbc1ce700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efcb4106ca0 0x7efcb41a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.531+0000 7efcbc1ce700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb4199de0 con 0x7efcb4101810 2024-02-17T11:56:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.531+0000 7efcbc1ce700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb41a45f0 con 0x7efcb4106ca0 2024-02-17T11:56:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.531+0000 7efcbc1ce700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb41a4730 con 0x7efcb4100e50 2024-02-17T11:56:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.531+0000 7efcb9769700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcb4101810 0x7efcb4199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.531+0000 7efcb9769700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcb4101810 0x7efcb4199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46978/0 (socket says 172.21.15.94:46978) 2024-02-17T11:56:23.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.531+0000 7efcb9769700 1 -- 172.21.15.94:0/795461843 learned_addr learned my addr 172.21.15.94:0/795461843 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:23.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcb9769700 1 -- 172.21.15.94:0/795461843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efcb4100e50 msgr2=0x7efcb4199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:23.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcb9f6a700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efcb4100e50 0x7efcb4199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:23.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcba76b700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efcb4106ca0 0x7efcb41a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:23.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcb9769700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efcb4100e50 0x7efcb4199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:23.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcb9769700 1 -- 172.21.15.94:0/795461843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efcb4106ca0 msgr2=0x7efcb41a40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:23.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcb9769700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efcb4106ca0 0x7efcb41a40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:23.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcb9769700 1 -- 172.21.15.94:0/795461843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca40095f0 con 0x7efcb4101810 2024-02-17T11:56:23.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcb9f6a700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efcb4100e50 0x7efcb4199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:23.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcba76b700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efcb4106ca0 0x7efcb41a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:23.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcb9769700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcb4101810 0x7efcb4199570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efca40098e0 tx=0x7efca40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:23.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcaaffd700 1 -- 172.21.15.94:0/795461843 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca401d070 con 0x7efcb4101810 2024-02-17T11:56:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcaaffd700 1 -- 172.21.15.94:0/795461843 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efca4004050 con 0x7efcb4101810 2024-02-17T11:56:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcbc1ce700 1 -- 172.21.15.94:0/795461843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb41a4960 con 0x7efcb4101810 2024-02-17T11:56:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcaaffd700 1 -- 172.21.15.94:0/795461843 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca400f460 con 0x7efcb4101810 2024-02-17T11:56:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.532+0000 7efcbc1ce700 1 -- 172.21.15.94:0/795461843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb41a4d70 con 0x7efcb4101810 2024-02-17T11:56:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.533+0000 7efcbc1ce700 1 -- 172.21.15.94:0/795461843 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efcb4110f20 con 0x7efcb4101810 2024-02-17T11:56:23.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.534+0000 7efcaaffd700 1 -- 172.21.15.94:0/795461843 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7efca4022690 con 0x7efcb4101810 2024-02-17T11:56:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.535+0000 7efcaaffd700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7efca00a21e0 0x7efca00a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.535+0000 7efcb9f6a700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7efca00a21e0 0x7efca00a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.535+0000 7efcaaffd700 1 -- 172.21.15.94:0/795461843 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7efca40cfa10 con 0x7efcb4101810 2024-02-17T11:56:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.535+0000 7efcaaffd700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244] conn(0x7efca00ab9c0 0x7efca00adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.535+0000 7efcaaffd700 1 -- 172.21.15.94:0/795461843 --> [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efca00ae490 con 0x7efca00ab9c0 2024-02-17T11:56:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.535+0000 7efcaaffd700 1 -- 172.21.15.94:0/795461843 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7efca40cfeb0 con 0x7efcb4101810 2024-02-17T11:56:23.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.535+0000 7efcb9f6a700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7efca00a21e0 0x7efca00a46a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efcb0007900 tx=0x7efcb0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.535+0000 7efcba76b700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244] conn(0x7efca00ab9c0 0x7efca00adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.536+0000 7efcba76b700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244] conn(0x7efca00ab9c0 0x7efca00adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:23.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.536+0000 7efcaaffd700 1 -- 172.21.15.94:0/795461843 <== osd.5 v2:172.21.15.189:6808/1566223244 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efca00ae490 con 0x7efca00ab9c0 2024-02-17T11:56:23.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.552+0000 7efcbc1ce700 1 -- 172.21.15.94:0/795461843 --> [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efcb4062380 con 0x7efca00ab9c0 2024-02-17T11:56:23.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.553+0000 7efcaaffd700 1 -- 172.21.15.94:0/795461843 <== osd.5 v2:172.21.15.189:6808/1566223244 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efcb4062380 con 0x7efca00ab9c0 2024-02-17T11:56:23.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.553+0000 7efcbc1ce700 1 -- 172.21.15.94:0/795461843 >> [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244] conn(0x7efca00ab9c0 msgr2=0x7efca00adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:23.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.553+0000 7efcbc1ce700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244] conn(0x7efca00ab9c0 0x7efca00adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:23.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.553+0000 7efcbc1ce700 1 -- 172.21.15.94:0/795461843 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7efca00a21e0 msgr2=0x7efca00a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:23.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.553+0000 7efcbc1ce700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7efca00a21e0 0x7efca00a46a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efcb0007900 tx=0x7efcb0008040 comp rx=0 tx=0).stop 2024-02-17T11:56:23.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.553+0000 7efcbc1ce700 1 -- 172.21.15.94:0/795461843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcb4101810 msgr2=0x7efcb4199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:23.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.553+0000 7efcbc1ce700 1 --2- 172.21.15.94:0/795461843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efcb4101810 0x7efcb4199570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efca40098e0 tx=0x7efca40057f0 comp rx=0 tx=0).stop 2024-02-17T11:56:23.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.553+0000 7efcba76b700 1 -- 172.21.15.94:0/795461843 reap_dead start 2024-02-17T11:56:23.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.553+0000 7efcbc1ce700 1 -- 172.21.15.94:0/795461843 shutdown_connections 2024-02-17T11:56:23.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.553+0000 7efcbc1ce700 1 -- 172.21.15.94:0/795461843 >> 172.21.15.94:0/795461843 conn(0x7efcb4077a30 msgr2=0x7efcb40ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:23.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.553+0000 7efcbc1ce700 1 -- 172.21.15.94:0/795461843 shutdown_connections 2024-02-17T11:56:23.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:23.553+0000 7efcbc1ce700 1 -- 172.21.15.94:0/795461843 wait complete. 2024-02-17T11:56:24.088 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:23 smithi094 ceph-mon[32499]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-02-17T11:56:24.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:23 smithi189 ceph-mon[36588]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-02-17T11:56:24.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:23 smithi125 ceph-mon[36819]: pgmap v297: 97 pgs: 97 active+clean; 636 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-02-17T11:56:25.999 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:25 smithi094 ceph-mon[32499]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-17T11:56:26.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:25 smithi189 ceph-mon[36588]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-17T11:56:26.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:25 smithi125 ceph-mon[36819]: pgmap v298: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-17T11:56:27.939 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:27 smithi094 ceph-mon[32499]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-17T11:56:28.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:27 smithi189 ceph-mon[36588]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-17T11:56:28.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:27 smithi125 ceph-mon[36819]: pgmap v299: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-17T11:56:30.077 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:29 smithi094 ceph-mon[32499]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-17T11:56:30.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:29 smithi189 ceph-mon[36588]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-17T11:56:30.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:29 smithi125 ceph-mon[36819]: pgmap v300: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-02-17T11:56:30.425 INFO:teuthology.orchestra.run.smithi094.stdout:163208757297 2024-02-17T11:56:30.426 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-17T11:56:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.686+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/4213020800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101530 msgr2=0x7ff9d810d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.686+0000 7ff9ddb6f700 1 --2- 172.21.15.94:0/4213020800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101530 0x7ff9d810d9c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000b0a0 tx=0x7ff9d000b3b0 comp rx=0 tx=0).stop 2024-02-17T11:56:31.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.686+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/4213020800 shutdown_connections 2024-02-17T11:56:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.686+0000 7ff9ddb6f700 1 --2- 172.21.15.94:0/4213020800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101530 0x7ff9d810d9c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.686+0000 7ff9ddb6f700 1 --2- 172.21.15.94:0/4213020800 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d8106b80 0x7ff9d8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.686+0000 7ff9ddb6f700 1 --2- 172.21.15.94:0/4213020800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9d8100b70 0x7ff9d8100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.686+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/4213020800 >> 172.21.15.94:0/4213020800 conn(0x7ff9d8077aa0 msgr2=0x7ff9d8077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:31.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.686+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/4213020800 shutdown_connections 2024-02-17T11:56:31.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.687+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/4213020800 wait complete. 2024-02-17T11:56:31.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.687+0000 7ff9ddb6f700 1 Processor -- start 2024-02-17T11:56:31.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.687+0000 7ff9ddb6f700 1 -- start start 2024-02-17T11:56:31.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ddb6f700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9d8100b70 0x7ff9d81027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:31.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ddb6f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101530 0x7ff9d8102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:31.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ddb6f700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d8106b80 0x7ff9d8103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:31.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ddb6f700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d81a6510 con 0x7ff9d8101530 2024-02-17T11:56:31.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ddb6f700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d81a6680 con 0x7ff9d8106b80 2024-02-17T11:56:31.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ddb6f700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d81a6840 con 0x7ff9d8100b70 2024-02-17T11:56:31.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ceffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101530 0x7ff9d8102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:31.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ceffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101530 0x7ff9d8102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52190/0 (socket says 172.21.15.94:52190) 2024-02-17T11:56:31.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ceffd700 1 -- 172.21.15.94:0/2047698827 learned_addr learned my addr 172.21.15.94:0/2047698827 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:31.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9d77fe700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9d8100b70 0x7ff9d81027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:31.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9d7fff700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d8106b80 0x7ff9d8103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:31.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ceffd700 1 -- 172.21.15.94:0/2047698827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9d8100b70 msgr2=0x7ff9d81027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:31.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ceffd700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9d8100b70 0x7ff9d81027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:31.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ceffd700 1 -- 172.21.15.94:0/2047698827 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d8106b80 msgr2=0x7ff9d8103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:31.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ceffd700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d8106b80 0x7ff9d8103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:31.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ceffd700 1 -- 172.21.15.94:0/2047698827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d0009d00 con 0x7ff9d8101530 2024-02-17T11:56:31.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9d7fff700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9d8106b80 0x7ff9d8103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:56:31.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.688+0000 7ff9ceffd700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101530 0x7ff9d8102cf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000d950 tx=0x7ff9c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:31.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.689+0000 7ff9d57fa700 1 -- 172.21.15.94:0/2047698827 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0009940 con 0x7ff9d8101530 2024-02-17T11:56:31.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.689+0000 7ff9d57fa700 1 -- 172.21.15.94:0/2047698827 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9c0010460 con 0x7ff9d8101530 2024-02-17T11:56:31.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.689+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/2047698827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d81a6b20 con 0x7ff9d8101530 2024-02-17T11:56:31.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.689+0000 7ff9d57fa700 1 -- 172.21.15.94:0/2047698827 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c000f620 con 0x7ff9d8101530 2024-02-17T11:56:31.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.689+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/2047698827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d81a7070 con 0x7ff9d8101530 2024-02-17T11:56:31.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.689+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/2047698827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff9d8110e90 con 0x7ff9d8101530 2024-02-17T11:56:31.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.691+0000 7ff9d57fa700 1 -- 172.21.15.94:0/2047698827 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ff9c00105d0 con 0x7ff9d8101530 2024-02-17T11:56:31.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.691+0000 7ff9d57fa700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff9c40a2260 0x7ff9c40a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:31.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.691+0000 7ff9d77fe700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff9c40a2260 0x7ff9c40a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:31.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.691+0000 7ff9d57fa700 1 -- 172.21.15.94:0/2047698827 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff9c00cdfd0 con 0x7ff9d8101530 2024-02-17T11:56:31.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.691+0000 7ff9d57fa700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236] conn(0x7ff9c40aba40 0x7ff9c40ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:31.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.691+0000 7ff9d57fa700 1 -- 172.21.15.94:0/2047698827 --> [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9c40ae510 con 0x7ff9c40aba40 2024-02-17T11:56:31.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.691+0000 7ff9d57fa700 1 -- 172.21.15.94:0/2047698827 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c00ce470 con 0x7ff9d8101530 2024-02-17T11:56:31.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.692+0000 7ff9d77fe700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff9c40a2260 0x7ff9c40a4720 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800abd0 tx=0x7ff9c800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:31.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.692+0000 7ff9d7fff700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236] conn(0x7ff9c40aba40 0x7ff9c40ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:31.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.692+0000 7ff9d7fff700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236] conn(0x7ff9c40aba40 0x7ff9c40ade60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:31.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.693+0000 7ff9d57fa700 1 -- 172.21.15.94:0/2047698827 <== osd.2 v2:172.21.15.189:6800/1204918236 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff9c40ae510 con 0x7ff9c40aba40 2024-02-17T11:56:31.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.709+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/2047698827 --> [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff9d8104b20 con 0x7ff9c40aba40 2024-02-17T11:56:31.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.710+0000 7ff9d57fa700 1 -- 172.21.15.94:0/2047698827 <== osd.2 v2:172.21.15.189:6800/1204918236 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff9d8104b20 con 0x7ff9c40aba40 2024-02-17T11:56:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.710+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/2047698827 >> [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236] conn(0x7ff9c40aba40 msgr2=0x7ff9c40ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.710+0000 7ff9ddb6f700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236] conn(0x7ff9c40aba40 0x7ff9c40ade60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.710+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/2047698827 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff9c40a2260 msgr2=0x7ff9c40a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.710+0000 7ff9ddb6f700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff9c40a2260 0x7ff9c40a4720 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800abd0 tx=0x7ff9c800a380 comp rx=0 tx=0).stop 2024-02-17T11:56:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.710+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/2047698827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101530 msgr2=0x7ff9d8102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.710+0000 7ff9ddb6f700 1 --2- 172.21.15.94:0/2047698827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9d8101530 0x7ff9d8102cf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000d950 tx=0x7ff9c000dc60 comp rx=0 tx=0).stop 2024-02-17T11:56:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.710+0000 7ff9d7fff700 1 -- 172.21.15.94:0/2047698827 reap_dead start 2024-02-17T11:56:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.710+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/2047698827 shutdown_connections 2024-02-17T11:56:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.710+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/2047698827 >> 172.21.15.94:0/2047698827 conn(0x7ff9d8077aa0 msgr2=0x7ff9d80ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.710+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/2047698827 shutdown_connections 2024-02-17T11:56:31.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:31.710+0000 7ff9ddb6f700 1 -- 172.21.15.94:0/2047698827 wait complete. 2024-02-17T11:56:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:31 smithi094 ceph-mon[32499]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-17T11:56:32.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:31 smithi189 ceph-mon[36588]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-17T11:56:32.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:31 smithi125 ceph-mon[36819]: pgmap v301: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-17T11:56:32.594 INFO:teuthology.orchestra.run.smithi094.stdout:77309411393 2024-02-17T11:56:32.594 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-17T11:56:32.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.750+0000 7f8093530700 1 -- 172.21.15.94:0/3961057091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f808c105ef0 msgr2=0x7f808c10dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:32.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.750+0000 7f8093530700 1 --2- 172.21.15.94:0/3961057091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f808c105ef0 0x7f808c10dac0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f808400b0a0 tx=0x7f808400b3b0 comp rx=0 tx=0).stop 2024-02-17T11:56:32.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.751+0000 7f8093530700 1 -- 172.21.15.94:0/3961057091 shutdown_connections 2024-02-17T11:56:32.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.751+0000 7f8093530700 1 --2- 172.21.15.94:0/3961057091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f808c105ef0 0x7f808c10dac0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:32.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.751+0000 7f8093530700 1 --2- 172.21.15.94:0/3961057091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f808c1015d0 0x7f808c1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:32.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.751+0000 7f8093530700 1 --2- 172.21.15.94:0/3961057091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f808c101f80 0x7f808c105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:32.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.751+0000 7f8093530700 1 -- 172.21.15.94:0/3961057091 >> 172.21.15.94:0/3961057091 conn(0x7f808c077170 msgr2=0x7f808c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:32.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.751+0000 7f8093530700 1 -- 172.21.15.94:0/3961057091 shutdown_connections 2024-02-17T11:56:32.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.751+0000 7f8093530700 1 -- 172.21.15.94:0/3961057091 wait complete. 2024-02-17T11:56:32.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.751+0000 7f8093530700 1 Processor -- start 2024-02-17T11:56:32.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.751+0000 7f8093530700 1 -- start start 2024-02-17T11:56:32.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8093530700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f808c1015d0 0x7f808c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:32.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8093530700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f808c101f80 0x7f808c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:32.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8093530700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f808c105ef0 0x7f808c19cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:32.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8093530700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c19d1e0 con 0x7f808c101f80 2024-02-17T11:56:32.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8093530700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c19d350 con 0x7f808c105ef0 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8093530700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c19d510 con 0x7f808c1015d0 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8090acb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f808c101f80 0x7f808c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8090acb700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f808c101f80 0x7f808c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52204/0 (socket says 172.21.15.94:52204) 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8090acb700 1 -- 172.21.15.94:0/3744167319 learned_addr learned my addr 172.21.15.94:0/3744167319 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f80912cc700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f808c1015d0 0x7f808c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8091acd700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f808c105ef0 0x7f808c19cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8090acb700 1 -- 172.21.15.94:0/3744167319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f808c1015d0 msgr2=0x7f808c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8090acb700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f808c1015d0 0x7f808c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8090acb700 1 -- 172.21.15.94:0/3744167319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f808c105ef0 msgr2=0x7f808c19cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8090acb700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f808c105ef0 0x7f808c19cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8090acb700 1 -- 172.21.15.94:0/3744167319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8084009d00 con 0x7f808c101f80 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f8090acb700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f808c101f80 0x7f808c198890 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f807c00d980 tx=0x7f807c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.752+0000 7f80827fc700 1 -- 172.21.15.94:0/3744167319 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00f890 con 0x7f808c101f80 2024-02-17T11:56:32.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.753+0000 7f80827fc700 1 -- 172.21.15.94:0/3744167319 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f807c010460 con 0x7f808c101f80 2024-02-17T11:56:32.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.753+0000 7f80827fc700 1 -- 172.21.15.94:0/3744167319 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c00e610 con 0x7f808c101f80 2024-02-17T11:56:32.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.753+0000 7f8093530700 1 -- 172.21.15.94:0/3744167319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808c19d7f0 con 0x7f808c101f80 2024-02-17T11:56:32.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.753+0000 7f8093530700 1 -- 172.21.15.94:0/3744167319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c1a5050 con 0x7f808c101f80 2024-02-17T11:56:32.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.753+0000 7f8093530700 1 -- 172.21.15.94:0/3744167319 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f808c110f20 con 0x7f808c101f80 2024-02-17T11:56:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.755+0000 7f80827fc700 1 -- 172.21.15.94:0/3744167319 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f807c010940 con 0x7f808c101f80 2024-02-17T11:56:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.755+0000 7f80827fc700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f80780a21e0 0x7f80780a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.755+0000 7f80912cc700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f80780a21e0 0x7f80780a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.756+0000 7f80827fc700 1 -- 172.21.15.94:0/3744167319 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f807c0cda60 con 0x7f808c101f80 2024-02-17T11:56:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.756+0000 7f80827fc700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927] conn(0x7f80780ab9c0 0x7f80780adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.756+0000 7f80827fc700 1 -- 172.21.15.94:0/3744167319 --> [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80780ae490 con 0x7f80780ab9c0 2024-02-17T11:56:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.756+0000 7f80827fc700 1 -- 172.21.15.94:0/3744167319 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f807c0cdf00 con 0x7f808c101f80 2024-02-17T11:56:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.756+0000 7f80912cc700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f80780a21e0 0x7f80780a46a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8088005950 tx=0x7f808800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.756+0000 7f8091acd700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927] conn(0x7f80780ab9c0 0x7f80780adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:32.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.756+0000 7f8091acd700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927] conn(0x7f80780ab9c0 0x7f80780adde0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:32.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.757+0000 7f80827fc700 1 -- 172.21.15.94:0/3744167319 <== osd.4 v2:172.21.15.125:6808/256065927 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f80780ae490 con 0x7f80780ab9c0 2024-02-17T11:56:32.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.774+0000 7f8093530700 1 -- 172.21.15.94:0/3744167319 --> [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f808c0fea60 con 0x7f80780ab9c0 2024-02-17T11:56:32.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.774+0000 7f80827fc700 1 -- 172.21.15.94:0/3744167319 <== osd.4 v2:172.21.15.125:6808/256065927 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f808c0fea60 con 0x7f80780ab9c0 2024-02-17T11:56:32.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.775+0000 7f8093530700 1 -- 172.21.15.94:0/3744167319 >> [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927] conn(0x7f80780ab9c0 msgr2=0x7f80780adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:32.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.775+0000 7f8093530700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927] conn(0x7f80780ab9c0 0x7f80780adde0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:32.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.775+0000 7f8093530700 1 -- 172.21.15.94:0/3744167319 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f80780a21e0 msgr2=0x7f80780a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:32.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.775+0000 7f8093530700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f80780a21e0 0x7f80780a46a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8088005950 tx=0x7f808800b410 comp rx=0 tx=0).stop 2024-02-17T11:56:32.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.775+0000 7f8093530700 1 -- 172.21.15.94:0/3744167319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f808c101f80 msgr2=0x7f808c198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:32.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.775+0000 7f8093530700 1 --2- 172.21.15.94:0/3744167319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f808c101f80 0x7f808c198890 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f807c00d980 tx=0x7f807c00dc90 comp rx=0 tx=0).stop 2024-02-17T11:56:32.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.775+0000 7f8091acd700 1 -- 172.21.15.94:0/3744167319 reap_dead start 2024-02-17T11:56:32.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.775+0000 7f8093530700 1 -- 172.21.15.94:0/3744167319 shutdown_connections 2024-02-17T11:56:32.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.775+0000 7f8093530700 1 -- 172.21.15.94:0/3744167319 >> 172.21.15.94:0/3744167319 conn(0x7f808c077170 msgr2=0x7f808c104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:32.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.775+0000 7f8093530700 1 -- 172.21.15.94:0/3744167319 shutdown_connections 2024-02-17T11:56:32.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:32.775+0000 7f8093530700 1 -- 172.21.15.94:0/3744167319 wait complete. 2024-02-17T11:56:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.136+0000 7ff251427700 1 -- 172.21.15.94:0/1835902194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff24c101810 msgr2=0x7ff24c10da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.136+0000 7ff251427700 1 --2- 172.21.15.94:0/1835902194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff24c101810 0x7ff24c10da30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff24400b0d0 tx=0x7ff24400b3e0 comp rx=0 tx=0).stop 2024-02-17T11:56:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.136+0000 7ff251427700 1 -- 172.21.15.94:0/1835902194 shutdown_connections 2024-02-17T11:56:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.136+0000 7ff251427700 1 --2- 172.21.15.94:0/1835902194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff24c101810 0x7ff24c10da30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:33.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.136+0000 7ff251427700 1 --2- 172.21.15.94:0/1835902194 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff24c106ca0 0x7ff24c107080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.136+0000 7ff251427700 1 --2- 172.21.15.94:0/1835902194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff24c100e50 0x7ff24c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.136+0000 7ff251427700 1 -- 172.21.15.94:0/1835902194 >> 172.21.15.94:0/1835902194 conn(0x7ff24c077a30 msgr2=0x7ff24c077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.137+0000 7ff251427700 1 -- 172.21.15.94:0/1835902194 shutdown_connections 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.137+0000 7ff251427700 1 -- 172.21.15.94:0/1835902194 wait complete. 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.137+0000 7ff251427700 1 Processor -- start 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.137+0000 7ff251427700 1 -- start start 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.137+0000 7ff251427700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff24c100e50 0x7ff24c199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.137+0000 7ff251427700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff24c101810 0x7ff24c199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.137+0000 7ff251427700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff24c106ca0 0x7ff24c1a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff251427700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c199e30 con 0x7ff24c101810 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff251427700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c1a4610 con 0x7ff24c106ca0 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff251427700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c1a4750 con 0x7ff24c100e50 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff24a7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff24c101810 0x7ff24c199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff24a7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff24c101810 0x7ff24c199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52168/0 (socket says 172.21.15.94:52168) 2024-02-17T11:56:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff24a7fc700 1 -- 172.21.15.94:0/1034783389 learned_addr learned my addr 172.21.15.94:0/1034783389 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff24affd700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff24c100e50 0x7ff24c199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff24b7fe700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff24c106ca0 0x7ff24c1a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff24a7fc700 1 -- 172.21.15.94:0/1034783389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff24c100e50 msgr2=0x7ff24c199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff24a7fc700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff24c100e50 0x7ff24c199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff24a7fc700 1 -- 172.21.15.94:0/1034783389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff24c106ca0 msgr2=0x7ff24c1a40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff24a7fc700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff24c106ca0 0x7ff24c1a40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff24a7fc700 1 -- 172.21.15.94:0/1034783389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff244009d00 con 0x7ff24c101810 2024-02-17T11:56:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff24affd700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff24c100e50 0x7ff24c199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:56:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff24a7fc700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff24c101810 0x7ff24c199590 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff24000d980 tx=0x7ff24000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff233fff700 1 -- 172.21.15.94:0/1034783389 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff240009b40 con 0x7ff24c101810 2024-02-17T11:56:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff233fff700 1 -- 172.21.15.94:0/1034783389 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff24000b6e0 con 0x7ff24c101810 2024-02-17T11:56:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff233fff700 1 -- 172.21.15.94:0/1034783389 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24000f620 con 0x7ff24c101810 2024-02-17T11:56:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff251427700 1 -- 172.21.15.94:0/1034783389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24c1a4a00 con 0x7ff24c101810 2024-02-17T11:56:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.138+0000 7ff251427700 1 -- 172.21.15.94:0/1034783389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24c1a4f50 con 0x7ff24c101810 2024-02-17T11:56:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.139+0000 7ff251427700 1 -- 172.21.15.94:0/1034783389 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff24c110f00 con 0x7ff24c101810 2024-02-17T11:56:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.140+0000 7ff233fff700 1 -- 172.21.15.94:0/1034783389 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ff240010460 con 0x7ff24c101810 2024-02-17T11:56:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.141+0000 7ff233fff700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff2340a2260 0x7ff2340a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.141+0000 7ff24affd700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff2340a2260 0x7ff2340a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.141+0000 7ff233fff700 1 -- 172.21.15.94:0/1034783389 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff2400ce020 con 0x7ff24c101810 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.141+0000 7ff233fff700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115] conn(0x7ff2340aba40 0x7ff2340ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.141+0000 7ff233fff700 1 -- 172.21.15.94:0/1034783389 --> [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2340ae510 con 0x7ff2340aba40 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.141+0000 7ff233fff700 1 -- 172.21.15.94:0/1034783389 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2400ce4c0 con 0x7ff24c101810 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.141+0000 7ff24affd700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff2340a2260 0x7ff2340a4720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff23c005fd0 tx=0x7ff23c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.141+0000 7ff24b7fe700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115] conn(0x7ff2340aba40 0x7ff2340ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.142+0000 7ff24b7fe700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115] conn(0x7ff2340aba40 0x7ff2340ade60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.143+0000 7ff233fff700 1 -- 172.21.15.94:0/1034783389 <== osd.10 v2:172.21.15.125:6824/2398610115 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff2340ae510 con 0x7ff2340aba40 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.159+0000 7ff251427700 1 -- 172.21.15.94:0/1034783389 --> [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff24c100300 con 0x7ff2340aba40 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.160+0000 7ff233fff700 1 -- 172.21.15.94:0/1034783389 <== osd.10 v2:172.21.15.125:6824/2398610115 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff24c100300 con 0x7ff2340aba40 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.160+0000 7ff251427700 1 -- 172.21.15.94:0/1034783389 >> [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115] conn(0x7ff2340aba40 msgr2=0x7ff2340ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.160+0000 7ff251427700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115] conn(0x7ff2340aba40 0x7ff2340ade60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.160+0000 7ff251427700 1 -- 172.21.15.94:0/1034783389 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff2340a2260 msgr2=0x7ff2340a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.160+0000 7ff251427700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ff2340a2260 0x7ff2340a4720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff23c005fd0 tx=0x7ff23c005db0 comp rx=0 tx=0).stop 2024-02-17T11:56:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.160+0000 7ff251427700 1 -- 172.21.15.94:0/1034783389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff24c101810 msgr2=0x7ff24c199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.160+0000 7ff251427700 1 --2- 172.21.15.94:0/1034783389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff24c101810 0x7ff24c199590 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff24000d980 tx=0x7ff24000dd40 comp rx=0 tx=0).stop 2024-02-17T11:56:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.160+0000 7ff24b7fe700 1 -- 172.21.15.94:0/1034783389 reap_dead start 2024-02-17T11:56:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.160+0000 7ff251427700 1 -- 172.21.15.94:0/1034783389 shutdown_connections 2024-02-17T11:56:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.160+0000 7ff251427700 1 -- 172.21.15.94:0/1034783389 >> 172.21.15.94:0/1034783389 conn(0x7ff24c077a30 msgr2=0x7ff24c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.161+0000 7ff251427700 1 -- 172.21.15.94:0/1034783389 shutdown_connections 2024-02-17T11:56:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:29.161+0000 7ff251427700 1 -- 172.21.15.94:0/1034783389 wait complete. 2024-02-17T11:56:34.028 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:33 smithi094 ceph-mon[32499]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-17T11:56:34.058 INFO:teuthology.orchestra.run.smithi094.stdout:309237645337 2024-02-17T11:56:34.059 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-02-17T11:56:34.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:33 smithi189 ceph-mon[36588]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-17T11:56:34.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:33 smithi125 ceph-mon[36819]: pgmap v302: 97 pgs: 97 active+clean; 639 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-02-17T11:56:34.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.554+0000 7f649bc9c700 1 -- 172.21.15.94:0/2211118372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64941017f0 msgr2=0x7f6494105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:34.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.554+0000 7f649bc9c700 1 --2- 172.21.15.94:0/2211118372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64941017f0 0x7f6494105840 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6490009990 tx=0x7f6490009ca0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.554+0000 7f649bc9c700 1 -- 172.21.15.94:0/2211118372 shutdown_connections 2024-02-17T11:56:34.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.554+0000 7f649bc9c700 1 --2- 172.21.15.94:0/2211118372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6494105f20 0x7f649410daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.554+0000 7f649bc9c700 1 --2- 172.21.15.94:0/2211118372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6494100e40 0x7f6494101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.554+0000 7f649bc9c700 1 --2- 172.21.15.94:0/2211118372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64941017f0 0x7f6494105840 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.554+0000 7f649bc9c700 1 -- 172.21.15.94:0/2211118372 >> 172.21.15.94:0/2211118372 conn(0x7f6494077ca0 msgr2=0x7f64940780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:34.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.554+0000 7f649bc9c700 1 -- 172.21.15.94:0/2211118372 shutdown_connections 2024-02-17T11:56:34.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.554+0000 7f649bc9c700 1 -- 172.21.15.94:0/2211118372 wait complete. 2024-02-17T11:56:34.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649bc9c700 1 Processor -- start 2024-02-17T11:56:34.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649bc9c700 1 -- start start 2024-02-17T11:56:34.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649bc9c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6494100e40 0x7f6494198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:34.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649bc9c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64941017f0 0x7f6494198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649bc9c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6494105f20 0x7f649419cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649bc9c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649419d110 con 0x7f6494105f20 2024-02-17T11:56:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649bc9c700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649419d280 con 0x7f6494100e40 2024-02-17T11:56:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649bc9c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649419d440 con 0x7f64941017f0 2024-02-17T11:56:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649a239700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6494105f20 0x7f649419cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649a239700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6494105f20 0x7f649419cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59296/0 (socket says 172.21.15.94:59296) 2024-02-17T11:56:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649a239700 1 -- 172.21.15.94:0/955908701 learned_addr learned my addr 172.21.15.94:0/955908701 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f6499a38700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6494100e40 0x7f6494198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f6499237700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64941017f0 0x7f6494198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649a239700 1 -- 172.21.15.94:0/955908701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64941017f0 msgr2=0x7f6494198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649a239700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64941017f0 0x7f6494198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649a239700 1 -- 172.21.15.94:0/955908701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6494100e40 msgr2=0x7f6494198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:34.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649a239700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6494100e40 0x7f6494198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649a239700 1 -- 172.21.15.94:0/955908701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64900095f0 con 0x7f6494105f20 2024-02-17T11:56:34.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.555+0000 7f649a239700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6494105f20 0x7f649419cb00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f648c00ada0 tx=0x7f648c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:34.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.556+0000 7f648affd700 1 -- 172.21.15.94:0/955908701 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c00f070 con 0x7f6494105f20 2024-02-17T11:56:34.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.556+0000 7f648affd700 1 -- 172.21.15.94:0/955908701 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f648c007b30 con 0x7f6494105f20 2024-02-17T11:56:34.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.556+0000 7f649bc9c700 1 -- 172.21.15.94:0/955908701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f649419d720 con 0x7f6494105f20 2024-02-17T11:56:34.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.556+0000 7f648affd700 1 -- 172.21.15.94:0/955908701 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648c0053c0 con 0x7f6494105f20 2024-02-17T11:56:34.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.556+0000 7f649bc9c700 1 -- 172.21.15.94:0/955908701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64941a4f80 con 0x7f6494105f20 2024-02-17T11:56:34.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.556+0000 7f649bc9c700 1 -- 172.21.15.94:0/955908701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6494110ee0 con 0x7f6494105f20 2024-02-17T11:56:34.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.558+0000 7f648affd700 1 -- 172.21.15.94:0/955908701 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f648c00c980 con 0x7f6494105f20 2024-02-17T11:56:34.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.558+0000 7f648affd700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f64800a21e0 0x7f64800a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:34.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.558+0000 7f6499a38700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f64800a21e0 0x7f64800a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:34.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.558+0000 7f648affd700 1 -- 172.21.15.94:0/955908701 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f648c0cdbf0 con 0x7f6494105f20 2024-02-17T11:56:34.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.558+0000 7f648affd700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621] conn(0x7f64800ab9c0 0x7f64800adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:34.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.558+0000 7f648affd700 1 -- 172.21.15.94:0/955908701 --> [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64800ae490 con 0x7f64800ab9c0 2024-02-17T11:56:34.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.558+0000 7f648affd700 1 -- 172.21.15.94:0/955908701 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f648c0ce090 con 0x7f6494105f20 2024-02-17T11:56:34.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.559+0000 7f6499237700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621] conn(0x7f64800ab9c0 0x7f64800adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:34.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.559+0000 7f6499a38700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f64800a21e0 0x7f64800a46a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6484005950 tx=0x7f648400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:34.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.559+0000 7f6499237700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621] conn(0x7f64800ab9c0 0x7f64800adde0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:34.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.560+0000 7f648affd700 1 -- 172.21.15.94:0/955908701 <== osd.3 v2:172.21.15.94:6810/3136219621 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f64800ae490 con 0x7f64800ab9c0 2024-02-17T11:56:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.576+0000 7f649bc9c700 1 -- 172.21.15.94:0/955908701 --> [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f64940623b0 con 0x7f64800ab9c0 2024-02-17T11:56:34.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.576+0000 7f648affd700 1 -- 172.21.15.94:0/955908701 <== osd.3 v2:172.21.15.94:6810/3136219621 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f64940623b0 con 0x7f64800ab9c0 2024-02-17T11:56:34.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.576+0000 7f649bc9c700 1 -- 172.21.15.94:0/955908701 >> [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621] conn(0x7f64800ab9c0 msgr2=0x7f64800adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:34.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.576+0000 7f649bc9c700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621] conn(0x7f64800ab9c0 0x7f64800adde0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.577+0000 7f649bc9c700 1 -- 172.21.15.94:0/955908701 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f64800a21e0 msgr2=0x7f64800a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:34.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.577+0000 7f649bc9c700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f64800a21e0 0x7f64800a46a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6484005950 tx=0x7f648400a300 comp rx=0 tx=0).stop 2024-02-17T11:56:34.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.577+0000 7f649bc9c700 1 -- 172.21.15.94:0/955908701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6494105f20 msgr2=0x7f649419cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:34.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.577+0000 7f649bc9c700 1 --2- 172.21.15.94:0/955908701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6494105f20 0x7f649419cb00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f648c00ada0 tx=0x7f648c00c330 comp rx=0 tx=0).stop 2024-02-17T11:56:34.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.577+0000 7f649a239700 1 -- 172.21.15.94:0/955908701 reap_dead start 2024-02-17T11:56:34.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.577+0000 7f649bc9c700 1 -- 172.21.15.94:0/955908701 shutdown_connections 2024-02-17T11:56:34.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.577+0000 7f649bc9c700 1 -- 172.21.15.94:0/955908701 >> 172.21.15.94:0/955908701 conn(0x7f6494077ca0 msgr2=0x7f649406a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:34.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.577+0000 7f649bc9c700 1 -- 172.21.15.94:0/955908701 shutdown_connections 2024-02-17T11:56:34.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.577+0000 7f649bc9c700 1 -- 172.21.15.94:0/955908701 wait complete. 2024-02-17T11:56:34.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.745+0000 7fbfa17b8700 1 -- 172.21.15.94:0/2961307904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf9c107cc0 msgr2=0x7fbf9c1080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:34.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.745+0000 7fbfa17b8700 1 --2- 172.21.15.94:0/2961307904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf9c107cc0 0x7fbf9c1080a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0099c0 tx=0x7fbf8c009cd0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.745+0000 7fbfa17b8700 1 -- 172.21.15.94:0/2961307904 shutdown_connections 2024-02-17T11:56:34.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.745+0000 7fbfa17b8700 1 --2- 172.21.15.94:0/2961307904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf9c06a2c0 0x7fbf9c10eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.745+0000 7fbfa17b8700 1 --2- 172.21.15.94:0/2961307904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf9c107cc0 0x7fbf9c1080a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.745+0000 7fbfa17b8700 1 --2- 172.21.15.94:0/2961307904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbf9c069920 0x7fbf9c069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.745+0000 7fbfa17b8700 1 -- 172.21.15.94:0/2961307904 >> 172.21.15.94:0/2961307904 conn(0x7fbf9c077aa0 msgr2=0x7fbf9c077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:34.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.745+0000 7fbfa17b8700 1 -- 172.21.15.94:0/2961307904 shutdown_connections 2024-02-17T11:56:34.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.745+0000 7fbfa17b8700 1 -- 172.21.15.94:0/2961307904 wait complete. 2024-02-17T11:56:34.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbfa17b8700 1 Processor -- start 2024-02-17T11:56:34.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbfa17b8700 1 -- start start 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbfa17b8700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbf9c069920 0x7fbf9c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbfa17b8700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf9c06a2c0 0x7fbf9c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbfa17b8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf9c107cc0 0x7fbf9c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbfa17b8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c19d040 con 0x7fbf9c107cc0 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbfa17b8700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c19d1b0 con 0x7fbf9c069920 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbfa17b8700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9c19d370 con 0x7fbf9c06a2c0 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbf9b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf9c107cc0 0x7fbf9c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbf9b7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf9c107cc0 0x7fbf9c19ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59320/0 (socket says 172.21.15.94:59320) 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbf9b7fe700 1 -- 172.21.15.94:0/831047392 learned_addr learned my addr 172.21.15.94:0/831047392 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbf9affd700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbf9c069920 0x7fbf9c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbf9a7fc700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf9c06a2c0 0x7fbf9c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbf9b7fe700 1 -- 172.21.15.94:0/831047392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf9c06a2c0 msgr2=0x7fbf9c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbf9b7fe700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf9c06a2c0 0x7fbf9c1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbf9b7fe700 1 -- 172.21.15.94:0/831047392 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbf9c069920 msgr2=0x7fbf9c1981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbf9b7fe700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbf9c069920 0x7fbf9c1981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbf9b7fe700 1 -- 172.21.15.94:0/831047392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf8c0095f0 con 0x7fbf9c107cc0 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbf9affd700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbf9c069920 0x7fbf9c1981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.746+0000 7fbf9b7fe700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf9c107cc0 0x7fbf9c19ca30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400ada0 tx=0x7fbf9400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.747+0000 7fbf83fff700 1 -- 172.21.15.94:0/831047392 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9400f070 con 0x7fbf9c107cc0 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.747+0000 7fbfa17b8700 1 -- 172.21.15.94:0/831047392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9c19d650 con 0x7fbf9c107cc0 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.747+0000 7fbf83fff700 1 -- 172.21.15.94:0/831047392 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbf94007b30 con 0x7fbf9c107cc0 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.747+0000 7fbf83fff700 1 -- 172.21.15.94:0/831047392 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94005500 con 0x7fbf9c107cc0 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.747+0000 7fbfa17b8700 1 -- 172.21.15.94:0/831047392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf9c1a4eb0 con 0x7fbf9c107cc0 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.747+0000 7fbfa17b8700 1 -- 172.21.15.94:0/831047392 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbf9c111fb0 con 0x7fbf9c107cc0 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.749+0000 7fbf83fff700 1 -- 172.21.15.94:0/831047392 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fbf94004ad0 con 0x7fbf9c107cc0 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.749+0000 7fbf83fff700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbf840a21e0 0x7fbf840a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.750+0000 7fbf83fff700 1 -- 172.21.15.94:0/831047392 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbf940cdd10 con 0x7fbf9c107cc0 2024-02-17T11:56:34.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.750+0000 7fbf9affd700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbf840a21e0 0x7fbf840a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:34.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.750+0000 7fbf83fff700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455] conn(0x7fbf840ab9c0 0x7fbf840adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:34.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.750+0000 7fbf83fff700 1 -- 172.21.15.94:0/831047392 --> [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbf840ae490 con 0x7fbf840ab9c0 2024-02-17T11:56:34.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.750+0000 7fbf83fff700 1 -- 172.21.15.94:0/831047392 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf940ce1b0 con 0x7fbf9c107cc0 2024-02-17T11:56:34.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.750+0000 7fbf9a7fc700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455] conn(0x7fbf840ab9c0 0x7fbf840adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:34.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.750+0000 7fbf9affd700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbf840a21e0 0x7fbf840a46a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c003800 tx=0x7fbf8c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:34.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.750+0000 7fbf9a7fc700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455] conn(0x7fbf840ab9c0 0x7fbf840adde0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:34.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.751+0000 7fbf83fff700 1 -- 172.21.15.94:0/831047392 <== osd.9 v2:172.21.15.94:6826/3841273455 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbf840ae490 con 0x7fbf840ab9c0 2024-02-17T11:56:34.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.767+0000 7fbfa17b8700 1 -- 172.21.15.94:0/831047392 --> [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbf9c062300 con 0x7fbf840ab9c0 2024-02-17T11:56:34.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.768+0000 7fbf83fff700 1 -- 172.21.15.94:0/831047392 <== osd.9 v2:172.21.15.94:6826/3841273455 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbf9c062300 con 0x7fbf840ab9c0 2024-02-17T11:56:34.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.768+0000 7fbfa17b8700 1 -- 172.21.15.94:0/831047392 >> [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455] conn(0x7fbf840ab9c0 msgr2=0x7fbf840adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:34.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.768+0000 7fbfa17b8700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455] conn(0x7fbf840ab9c0 0x7fbf840adde0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:34.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.768+0000 7fbfa17b8700 1 -- 172.21.15.94:0/831047392 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbf840a21e0 msgr2=0x7fbf840a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:34.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.768+0000 7fbfa17b8700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbf840a21e0 0x7fbf840a46a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c003800 tx=0x7fbf8c009f10 comp rx=0 tx=0).stop 2024-02-17T11:56:34.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.768+0000 7fbfa17b8700 1 -- 172.21.15.94:0/831047392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf9c107cc0 msgr2=0x7fbf9c19ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:34.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.768+0000 7fbfa17b8700 1 --2- 172.21.15.94:0/831047392 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbf9c107cc0 0x7fbf9c19ca30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbf9400ada0 tx=0x7fbf9400c330 comp rx=0 tx=0).stop 2024-02-17T11:56:34.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.768+0000 7fbf9b7fe700 1 -- 172.21.15.94:0/831047392 reap_dead start 2024-02-17T11:56:34.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.768+0000 7fbfa17b8700 1 -- 172.21.15.94:0/831047392 shutdown_connections 2024-02-17T11:56:34.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.768+0000 7fbfa17b8700 1 -- 172.21.15.94:0/831047392 >> 172.21.15.94:0/831047392 conn(0x7fbf9c077aa0 msgr2=0x7fbf9c0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:34.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.768+0000 7fbfa17b8700 1 -- 172.21.15.94:0/831047392 shutdown_connections 2024-02-17T11:56:34.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:34.768+0000 7fbfa17b8700 1 -- 172.21.15.94:0/831047392 wait complete. 2024-02-17T11:56:35.788 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:35 smithi094 ceph-mon[32499]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-17T11:56:35.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.890+0000 7fd922892700 1 -- 172.21.15.94:0/3481472346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd91c06ad80 msgr2=0x7fd91c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:35.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.890+0000 7fd922892700 1 --2- 172.21.15.94:0/3481472346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd91c06ad80 0x7fd91c10eac0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd91400b0d0 tx=0x7fd91400b3e0 comp rx=0 tx=0).stop 2024-02-17T11:56:35.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.890+0000 7fd922892700 1 -- 172.21.15.94:0/3481472346 shutdown_connections 2024-02-17T11:56:35.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.890+0000 7fd922892700 1 --2- 172.21.15.94:0/3481472346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd91c06ad80 0x7fd91c10eac0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:35.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.890+0000 7fd922892700 1 --2- 172.21.15.94:0/3481472346 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd91c107ca0 0x7fd91c108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:35.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.890+0000 7fd922892700 1 --2- 172.21.15.94:0/3481472346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd91c06a3c0 0x7fd91c06a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:35.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.890+0000 7fd922892700 1 -- 172.21.15.94:0/3481472346 >> 172.21.15.94:0/3481472346 conn(0x7fd91c077b80 msgr2=0x7fd91c077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:35.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.890+0000 7fd922892700 1 -- 172.21.15.94:0/3481472346 shutdown_connections 2024-02-17T11:56:35.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.890+0000 7fd922892700 1 -- 172.21.15.94:0/3481472346 wait complete. 2024-02-17T11:56:35.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.891+0000 7fd922892700 1 Processor -- start 2024-02-17T11:56:35.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.891+0000 7fd922892700 1 -- start start 2024-02-17T11:56:35.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.891+0000 7fd922892700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd91c06a3c0 0x7fd91c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:35.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.891+0000 7fd922892700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd91c06ad80 0x7fd91c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:35.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.891+0000 7fd922892700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd91c107ca0 0x7fd91c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:35.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.891+0000 7fd922892700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c19d190 con 0x7fd91c107ca0 2024-02-17T11:56:35.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.891+0000 7fd922892700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c19d300 con 0x7fd91c06ad80 2024-02-17T11:56:35.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.891+0000 7fd922892700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c19d4c0 con 0x7fd91c06a3c0 2024-02-17T11:56:35.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.891+0000 7fd920e2f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd91c107ca0 0x7fd91c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:35.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.891+0000 7fd920e2f700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd91c107ca0 0x7fd91c19cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59340/0 (socket says 172.21.15.94:59340) 2024-02-17T11:56:35.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.891+0000 7fd920e2f700 1 -- 172.21.15.94:0/3104428644 learned_addr learned my addr 172.21.15.94:0/3104428644 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:35.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd91b7fe700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd91c06ad80 0x7fd91c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:35.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd91bfff700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd91c06a3c0 0x7fd91c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:35.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd920e2f700 1 -- 172.21.15.94:0/3104428644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd91c06a3c0 msgr2=0x7fd91c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:35.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd920e2f700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd91c06a3c0 0x7fd91c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:35.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd920e2f700 1 -- 172.21.15.94:0/3104428644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd91c06ad80 msgr2=0x7fd91c1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:35.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd920e2f700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd91c06ad80 0x7fd91c1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:35.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd920e2f700 1 -- 172.21.15.94:0/3104428644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914009d00 con 0x7fd91c107ca0 2024-02-17T11:56:35.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd91b7fe700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd91c06ad80 0x7fd91c1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:56:35.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd91bfff700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd91c06a3c0 0x7fd91c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:35.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd920e2f700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd91c107ca0 0x7fd91c19cb80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd91400d010 tx=0x7fd914007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:35.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd9197fa700 1 -- 172.21.15.94:0/3104428644 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91400e040 con 0x7fd91c107ca0 2024-02-17T11:56:35.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd922892700 1 -- 172.21.15.94:0/3104428644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91c19d740 con 0x7fd91c107ca0 2024-02-17T11:56:35.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd9197fa700 1 -- 172.21.15.94:0/3104428644 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd91401dd70 con 0x7fd91c107ca0 2024-02-17T11:56:35.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd9197fa700 1 -- 172.21.15.94:0/3104428644 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91401b450 con 0x7fd91c107ca0 2024-02-17T11:56:35.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.892+0000 7fd922892700 1 -- 172.21.15.94:0/3104428644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c1a5000 con 0x7fd91c107ca0 2024-02-17T11:56:35.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.893+0000 7fd922892700 1 -- 172.21.15.94:0/3104428644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd91c19da20 con 0x7fd91c107ca0 2024-02-17T11:56:35.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.894+0000 7fd9197fa700 1 -- 172.21.15.94:0/3104428644 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fd91401d890 con 0x7fd91c107ca0 2024-02-17T11:56:35.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.895+0000 7fd9197fa700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd9040a21e0 0x7fd9040a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:35.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.895+0000 7fd91bfff700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd9040a21e0 0x7fd9040a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:35.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.895+0000 7fd9197fa700 1 -- 172.21.15.94:0/3104428644 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd9140cfab0 con 0x7fd91c107ca0 2024-02-17T11:56:35.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.895+0000 7fd9197fa700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800] conn(0x7fd9040ab9c0 0x7fd9040adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:35.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.895+0000 7fd9197fa700 1 -- 172.21.15.94:0/3104428644 --> [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9040ae490 con 0x7fd9040ab9c0 2024-02-17T11:56:35.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.895+0000 7fd9197fa700 1 -- 172.21.15.94:0/3104428644 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9140d42c0 con 0x7fd91c107ca0 2024-02-17T11:56:35.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.895+0000 7fd91b7fe700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800] conn(0x7fd9040ab9c0 0x7fd9040adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:35.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.895+0000 7fd91bfff700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd9040a21e0 0x7fd9040a46a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd90c005fd0 tx=0x7fd90c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:35.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.895+0000 7fd91b7fe700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800] conn(0x7fd9040ab9c0 0x7fd9040adde0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:35.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.896+0000 7fd9197fa700 1 -- 172.21.15.94:0/3104428644 <== osd.0 v2:172.21.15.94:6802/4275068800 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd9040ae490 con 0x7fd9040ab9c0 2024-02-17T11:56:35.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.912+0000 7fd922892700 1 -- 172.21.15.94:0/3104428644 --> [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd91c062380 con 0x7fd9040ab9c0 2024-02-17T11:56:35.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.913+0000 7fd9197fa700 1 -- 172.21.15.94:0/3104428644 <== osd.0 v2:172.21.15.94:6802/4275068800 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd91c062380 con 0x7fd9040ab9c0 2024-02-17T11:56:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.913+0000 7fd922892700 1 -- 172.21.15.94:0/3104428644 >> [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800] conn(0x7fd9040ab9c0 msgr2=0x7fd9040adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.913+0000 7fd922892700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800] conn(0x7fd9040ab9c0 0x7fd9040adde0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.913+0000 7fd922892700 1 -- 172.21.15.94:0/3104428644 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd9040a21e0 msgr2=0x7fd9040a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.913+0000 7fd922892700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd9040a21e0 0x7fd9040a46a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd90c005fd0 tx=0x7fd90c005ee0 comp rx=0 tx=0).stop 2024-02-17T11:56:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.913+0000 7fd922892700 1 -- 172.21.15.94:0/3104428644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd91c107ca0 msgr2=0x7fd91c19cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.913+0000 7fd922892700 1 --2- 172.21.15.94:0/3104428644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd91c107ca0 0x7fd91c19cb80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd91400d010 tx=0x7fd914007920 comp rx=0 tx=0).stop 2024-02-17T11:56:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.913+0000 7fd920e2f700 1 -- 172.21.15.94:0/3104428644 reap_dead start 2024-02-17T11:56:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.913+0000 7fd922892700 1 -- 172.21.15.94:0/3104428644 shutdown_connections 2024-02-17T11:56:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.913+0000 7fd922892700 1 -- 172.21.15.94:0/3104428644 >> 172.21.15.94:0/3104428644 conn(0x7fd91c077b80 msgr2=0x7fd91c0ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:35.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.913+0000 7fd922892700 1 -- 172.21.15.94:0/3104428644 shutdown_connections 2024-02-17T11:56:35.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:35.913+0000 7fd922892700 1 -- 172.21.15.94:0/3104428644 wait complete. 2024-02-17T11:56:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:35 smithi189 ceph-mon[36588]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-17T11:56:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:35 smithi125 ceph-mon[36819]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-17T11:56:37.059 INFO:teuthology.orchestra.run.smithi094.stdout:287762808862 2024-02-17T11:56:37.060 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-02-17T11:56:37.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.781+0000 7fc36301e700 1 -- 172.21.15.94:0/4062406611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc35c101560 msgr2=0x7fc35c10d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:37.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.781+0000 7fc36301e700 1 --2- 172.21.15.94:0/4062406611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc35c101560 0x7fc35c10d9a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc35400b0a0 tx=0x7fc35400b3b0 comp rx=0 tx=0).stop 2024-02-17T11:56:37.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.781+0000 7fc36301e700 1 -- 172.21.15.94:0/4062406611 shutdown_connections 2024-02-17T11:56:37.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.781+0000 7fc36301e700 1 --2- 172.21.15.94:0/4062406611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc35c101560 0x7fc35c10d9a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:37.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.781+0000 7fc36301e700 1 --2- 172.21.15.94:0/4062406611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc35c106bd0 0x7fc35c106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:37.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.781+0000 7fc36301e700 1 --2- 172.21.15.94:0/4062406611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc35c100ba0 0x7fc35c101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:37.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.781+0000 7fc36301e700 1 -- 172.21.15.94:0/4062406611 >> 172.21.15.94:0/4062406611 conn(0x7fc35c076bc0 msgr2=0x7fc35c076fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:37.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.781+0000 7fc36301e700 1 -- 172.21.15.94:0/4062406611 shutdown_connections 2024-02-17T11:56:37.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.781+0000 7fc36301e700 1 -- 172.21.15.94:0/4062406611 wait complete. 2024-02-17T11:56:37.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.781+0000 7fc36301e700 1 Processor -- start 2024-02-17T11:56:37.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.781+0000 7fc36301e700 1 -- start start 2024-02-17T11:56:37.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc36301e700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc35c100ba0 0x7fc35c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:37.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc36301e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc35c101560 0x7fc35c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:37.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc36301e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc35c106bd0 0x7fc35c19cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:37.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc36301e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c19d190 con 0x7fc35c101560 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc36301e700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c19d300 con 0x7fc35c100ba0 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc36301e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35c19d4c0 con 0x7fc35c106bd0 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc35bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc35c101560 0x7fc35c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc35bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc35c101560 0x7fc35c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59360/0 (socket says 172.21.15.94:59360) 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc35bfff700 1 -- 172.21.15.94:0/4037762041 learned_addr learned my addr 172.21.15.94:0/4037762041 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc360dba700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc35c100ba0 0x7fc35c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc3615bb700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc35c106bd0 0x7fc35c19cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc35bfff700 1 -- 172.21.15.94:0/4037762041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc35c106bd0 msgr2=0x7fc35c19cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc35bfff700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc35c106bd0 0x7fc35c19cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc35bfff700 1 -- 172.21.15.94:0/4037762041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc35c100ba0 msgr2=0x7fc35c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc35bfff700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc35c100ba0 0x7fc35c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc35bfff700 1 -- 172.21.15.94:0/4037762041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc354009d00 con 0x7fc35c101560 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc35bfff700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc35c101560 0x7fc35c1987f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc34800d950 tx=0x7fc34800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:37.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc359ffb700 1 -- 172.21.15.94:0/4037762041 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348009b40 con 0x7fc35c101560 2024-02-17T11:56:37.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc36301e700 1 -- 172.21.15.94:0/4037762041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc35c19d7a0 con 0x7fc35c101560 2024-02-17T11:56:37.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc359ffb700 1 -- 172.21.15.94:0/4037762041 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc348009ca0 con 0x7fc35c101560 2024-02-17T11:56:37.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.782+0000 7fc359ffb700 1 -- 172.21.15.94:0/4037762041 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc34800f6b0 con 0x7fc35c101560 2024-02-17T11:56:37.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.783+0000 7fc36301e700 1 -- 172.21.15.94:0/4037762041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc35c1a5000 con 0x7fc35c101560 2024-02-17T11:56:37.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.783+0000 7fc36301e700 1 -- 172.21.15.94:0/4037762041 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc35c1021d0 con 0x7fc35c101560 2024-02-17T11:56:37.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.785+0000 7fc359ffb700 1 -- 172.21.15.94:0/4037762041 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fc348010460 con 0x7fc35c101560 2024-02-17T11:56:37.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.785+0000 7fc359ffb700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc34c0a21e0 0x7fc34c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:37.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.785+0000 7fc360dba700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc34c0a21e0 0x7fc34c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:37.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.785+0000 7fc359ffb700 1 -- 172.21.15.94:0/4037762041 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc3480cde00 con 0x7fc35c101560 2024-02-17T11:56:37.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.785+0000 7fc359ffb700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339] conn(0x7fc34c0ab9c0 0x7fc34c0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:37.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.785+0000 7fc359ffb700 1 -- 172.21.15.94:0/4037762041 --> [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc34c0ae490 con 0x7fc34c0ab9c0 2024-02-17T11:56:37.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.785+0000 7fc359ffb700 1 -- 172.21.15.94:0/4037762041 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3480ce2a0 con 0x7fc35c101560 2024-02-17T11:56:37.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.786+0000 7fc360dba700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc34c0a21e0 0x7fc34c0a46a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc350006fd0 tx=0x7fc350008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:37.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.786+0000 7fc3615bb700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339] conn(0x7fc34c0ab9c0 0x7fc34c0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:37.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.786+0000 7fc3615bb700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339] conn(0x7fc34c0ab9c0 0x7fc34c0adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:37.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.787+0000 7fc359ffb700 1 -- 172.21.15.94:0/4037762041 <== osd.1 v2:172.21.15.125:6800/142575339 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc34c0ae490 con 0x7fc34c0ab9c0 2024-02-17T11:56:37.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.803+0000 7fc36301e700 1 -- 172.21.15.94:0/4037762041 --> [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc35c110f40 con 0x7fc34c0ab9c0 2024-02-17T11:56:37.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.803+0000 7fc359ffb700 1 -- 172.21.15.94:0/4037762041 <== osd.1 v2:172.21.15.125:6800/142575339 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc35c110f40 con 0x7fc34c0ab9c0 2024-02-17T11:56:37.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.804+0000 7fc36301e700 1 -- 172.21.15.94:0/4037762041 >> [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339] conn(0x7fc34c0ab9c0 msgr2=0x7fc34c0adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:37.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.804+0000 7fc36301e700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339] conn(0x7fc34c0ab9c0 0x7fc34c0adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:37.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.804+0000 7fc36301e700 1 -- 172.21.15.94:0/4037762041 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc34c0a21e0 msgr2=0x7fc34c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:37.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.804+0000 7fc36301e700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc34c0a21e0 0x7fc34c0a46a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc350006fd0 tx=0x7fc350008040 comp rx=0 tx=0).stop 2024-02-17T11:56:37.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.804+0000 7fc36301e700 1 -- 172.21.15.94:0/4037762041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc35c101560 msgr2=0x7fc35c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:37.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.804+0000 7fc36301e700 1 --2- 172.21.15.94:0/4037762041 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc35c101560 0x7fc35c1987f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc34800d950 tx=0x7fc34800dd10 comp rx=0 tx=0).stop 2024-02-17T11:56:37.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.804+0000 7fc3615bb700 1 -- 172.21.15.94:0/4037762041 reap_dead start 2024-02-17T11:56:37.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.804+0000 7fc36301e700 1 -- 172.21.15.94:0/4037762041 shutdown_connections 2024-02-17T11:56:37.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.804+0000 7fc36301e700 1 -- 172.21.15.94:0/4037762041 >> 172.21.15.94:0/4037762041 conn(0x7fc35c076bc0 msgr2=0x7fc35c0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:37.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.804+0000 7fc36301e700 1 -- 172.21.15.94:0/4037762041 shutdown_connections 2024-02-17T11:56:37.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.804+0000 7fc36301e700 1 -- 172.21.15.94:0/4037762041 wait complete. 2024-02-17T11:56:37.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:37 smithi094 ceph-mon[32499]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-17T11:56:38.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:37 smithi189 ceph-mon[36588]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-17T11:56:38.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:37 smithi125 ceph-mon[36819]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-17T11:56:38.692 INFO:teuthology.orchestra.run.smithi094.stdout:107374182460 2024-02-17T11:56:38.692 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-17T11:56:38.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.872+0000 7ffb06256700 1 -- 172.21.15.94:0/2397572784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb00102900 msgr2=0x7ffb001068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:38.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.872+0000 7ffb06256700 1 --2- 172.21.15.94:0/2397572784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb00102900 0x7ffb001068a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009990 tx=0x7ffaf4009ca0 comp rx=0 tx=0).stop 2024-02-17T11:56:38.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.872+0000 7ffb06256700 1 -- 172.21.15.94:0/2397572784 shutdown_connections 2024-02-17T11:56:38.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.872+0000 7ffb06256700 1 --2- 172.21.15.94:0/2397572784 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb00106f80 0x7ffb0010eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:38.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.872+0000 7ffb06256700 1 --2- 172.21.15.94:0/2397572784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb00101f50 0x7ffb00102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:38.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.872+0000 7ffb06256700 1 --2- 172.21.15.94:0/2397572784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb00102900 0x7ffb001068a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:38.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.872+0000 7ffb06256700 1 -- 172.21.15.94:0/2397572784 >> 172.21.15.94:0/2397572784 conn(0x7ffb00078d00 msgr2=0x7ffb00079110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:38.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.872+0000 7ffb06256700 1 -- 172.21.15.94:0/2397572784 shutdown_connections 2024-02-17T11:56:38.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.872+0000 7ffb06256700 1 -- 172.21.15.94:0/2397572784 wait complete. 2024-02-17T11:56:38.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.872+0000 7ffb06256700 1 Processor -- start 2024-02-17T11:56:38.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.873+0000 7ffb06256700 1 -- start start 2024-02-17T11:56:38.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.873+0000 7ffb06256700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb00101f50 0x7ffb0019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:38.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.873+0000 7ffb06256700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb00102900 0x7ffb0019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:38.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.873+0000 7ffb06256700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb00106f80 0x7ffb0019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:38.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.873+0000 7ffb06256700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0019f270 con 0x7ffb00106f80 2024-02-17T11:56:38.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.873+0000 7ffb06256700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0019f3e0 con 0x7ffb00101f50 2024-02-17T11:56:38.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.873+0000 7ffb06256700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0019f5a0 con 0x7ffb00102900 2024-02-17T11:56:38.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.873+0000 7ffafffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb00106f80 0x7ffb0019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:38.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.873+0000 7ffafffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb00106f80 0x7ffb0019ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59378/0 (socket says 172.21.15.94:59378) 2024-02-17T11:56:38.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.873+0000 7ffafffff700 1 -- 172.21.15.94:0/1301982775 learned_addr learned my addr 172.21.15.94:0/1301982775 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:38.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.873+0000 7ffafeffd700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb00102900 0x7ffb0019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:38.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.873+0000 7ffaff7fe700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb00101f50 0x7ffb0019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:38.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.874+0000 7ffafffff700 1 -- 172.21.15.94:0/1301982775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb00102900 msgr2=0x7ffb0019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:38.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.874+0000 7ffafffff700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb00102900 0x7ffb0019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:38.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.874+0000 7ffafffff700 1 -- 172.21.15.94:0/1301982775 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb00101f50 msgr2=0x7ffb0019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:38.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.874+0000 7ffafffff700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb00101f50 0x7ffb0019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:38.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.874+0000 7ffafffff700 1 -- 172.21.15.94:0/1301982775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf40095f0 con 0x7ffb00106f80 2024-02-17T11:56:38.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.874+0000 7ffaff7fe700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb00101f50 0x7ffb0019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:56:38.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.874+0000 7ffafffff700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb00106f80 0x7ffb0019ec60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8007ee0 tx=0x7ffaf800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:38.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.874+0000 7ffafcff9700 1 -- 172.21.15.94:0/1301982775 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf800ddd0 con 0x7ffb00106f80 2024-02-17T11:56:38.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.874+0000 7ffb06256700 1 -- 172.21.15.94:0/1301982775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0019f880 con 0x7ffb00106f80 2024-02-17T11:56:38.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.874+0000 7ffafcff9700 1 -- 172.21.15.94:0/1301982775 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffaf800f040 con 0x7ffb00106f80 2024-02-17T11:56:38.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.874+0000 7ffafcff9700 1 -- 172.21.15.94:0/1301982775 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8015400 con 0x7ffb00106f80 2024-02-17T11:56:38.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.874+0000 7ffb06256700 1 -- 172.21.15.94:0/1301982775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb001a70e0 con 0x7ffb00106f80 2024-02-17T11:56:38.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.875+0000 7ffb06256700 1 -- 172.21.15.94:0/1301982775 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffb00112010 con 0x7ffb00106f80 2024-02-17T11:56:38.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.876+0000 7ffafcff9700 1 -- 172.21.15.94:0/1301982775 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7ffaf8003a00 con 0x7ffb00106f80 2024-02-17T11:56:38.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.877+0000 7ffafcff9700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffae80a2260 0x7ffae80a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:38.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.877+0000 7ffafcff9700 1 -- 172.21.15.94:0/1301982775 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ffaf80cdfd0 con 0x7ffb00106f80 2024-02-17T11:56:38.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.877+0000 7ffaff7fe700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffae80a2260 0x7ffae80a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:38.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.877+0000 7ffafcff9700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186] conn(0x7ffae80aba40 0x7ffae80ade60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:38.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.877+0000 7ffafcff9700 1 -- 172.21.15.94:0/1301982775 --> [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffae80ae510 con 0x7ffae80aba40 2024-02-17T11:56:38.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.877+0000 7ffafcff9700 1 -- 172.21.15.94:0/1301982775 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffaf80ce470 con 0x7ffb00106f80 2024-02-17T11:56:38.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.877+0000 7ffafeffd700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186] conn(0x7ffae80aba40 0x7ffae80ade60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:38.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.878+0000 7ffaff7fe700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffae80a2260 0x7ffae80a4720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0005950 tx=0x7ffaf00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:38.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.878+0000 7ffafeffd700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186] conn(0x7ffae80aba40 0x7ffae80ade60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:38.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.879+0000 7ffafcff9700 1 -- 172.21.15.94:0/1301982775 <== osd.11 v2:172.21.15.189:6824/656579186 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffae80ae510 con 0x7ffae80aba40 2024-02-17T11:56:38.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.898+0000 7ffb06256700 1 -- 172.21.15.94:0/1301982775 --> [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffb00075a10 con 0x7ffae80aba40 2024-02-17T11:56:38.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.898+0000 7ffafcff9700 1 -- 172.21.15.94:0/1301982775 <== osd.11 v2:172.21.15.189:6824/656579186 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ffb00075a10 con 0x7ffae80aba40 2024-02-17T11:56:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.899+0000 7ffb06256700 1 -- 172.21.15.94:0/1301982775 >> [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186] conn(0x7ffae80aba40 msgr2=0x7ffae80ade60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.899+0000 7ffb06256700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186] conn(0x7ffae80aba40 0x7ffae80ade60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.899+0000 7ffb06256700 1 -- 172.21.15.94:0/1301982775 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffae80a2260 msgr2=0x7ffae80a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.899+0000 7ffb06256700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffae80a2260 0x7ffae80a4720 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0005950 tx=0x7ffaf00058e0 comp rx=0 tx=0).stop 2024-02-17T11:56:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.899+0000 7ffb06256700 1 -- 172.21.15.94:0/1301982775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb00106f80 msgr2=0x7ffb0019ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:38.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.899+0000 7ffb06256700 1 --2- 172.21.15.94:0/1301982775 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb00106f80 0x7ffb0019ec60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8007ee0 tx=0x7ffaf800d3b0 comp rx=0 tx=0).stop 2024-02-17T11:56:38.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.900+0000 7ffafffff700 1 -- 172.21.15.94:0/1301982775 reap_dead start 2024-02-17T11:56:38.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.900+0000 7ffb06256700 1 -- 172.21.15.94:0/1301982775 shutdown_connections 2024-02-17T11:56:38.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.900+0000 7ffb06256700 1 -- 172.21.15.94:0/1301982775 >> 172.21.15.94:0/1301982775 conn(0x7ffb00078d00 msgr2=0x7ffb0010ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:38.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.900+0000 7ffb06256700 1 -- 172.21.15.94:0/1301982775 shutdown_connections 2024-02-17T11:56:38.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:38.900+0000 7ffb06256700 1 -- 172.21.15.94:0/1301982775 wait complete. 2024-02-17T11:56:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.677+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3821367510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec006a2c0 msgr2=0x7f1ec010eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.677+0000 7f1ec86e0700 1 --2- 172.21.15.94:0/3821367510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec006a2c0 0x7f1ec010eb00 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1eb800b0d0 tx=0x7f1eb800b3e0 comp rx=0 tx=0).stop 2024-02-17T11:56:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.677+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3821367510 shutdown_connections 2024-02-17T11:56:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.677+0000 7f1ec86e0700 1 --2- 172.21.15.94:0/3821367510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec006a2c0 0x7f1ec010eb00 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.677+0000 7f1ec86e0700 1 --2- 172.21.15.94:0/3821367510 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ec0107cc0 0x7f1ec01080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.677+0000 7f1ec86e0700 1 --2- 172.21.15.94:0/3821367510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ec0069920 0x7f1ec0069d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.677+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3821367510 >> 172.21.15.94:0/3821367510 conn(0x7f1ec0077aa0 msgr2=0x7f1ec0077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.677+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3821367510 shutdown_connections 2024-02-17T11:56:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.677+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3821367510 wait complete. 2024-02-17T11:56:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.678+0000 7f1ec86e0700 1 Processor -- start 2024-02-17T11:56:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.678+0000 7f1ec86e0700 1 -- start start 2024-02-17T11:56:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.678+0000 7f1ec86e0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec0069920 0x7f1ec0100c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.678+0000 7f1ec86e0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ec006a2c0 0x7f1ec0101180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.678+0000 7f1ec86e0700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ec0107cc0 0x7f1ec01016c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.678+0000 7f1ec86e0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec01a6680 con 0x7f1ec0069920 2024-02-17T11:56:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.678+0000 7f1ec86e0700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec01a67f0 con 0x7f1ec0107cc0 2024-02-17T11:56:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.678+0000 7f1ec86e0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec01a69b0 con 0x7f1ec006a2c0 2024-02-17T11:56:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.678+0000 7f1ec647c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec0069920 0x7f1ec0100c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.679+0000 7f1ec647c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec0069920 0x7f1ec0100c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59352/0 (socket says 172.21.15.94:59352) 2024-02-17T11:56:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.679+0000 7f1ec647c700 1 -- 172.21.15.94:0/3380867866 learned_addr learned my addr 172.21.15.94:0/3380867866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.679+0000 7f1ec6c7d700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ec0107cc0 0x7f1ec01016c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.679+0000 7f1ec5c7b700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ec006a2c0 0x7f1ec0101180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.679+0000 7f1ec647c700 1 -- 172.21.15.94:0/3380867866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ec006a2c0 msgr2=0x7f1ec0101180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.679+0000 7f1ec647c700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1ec006a2c0 0x7f1ec0101180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.679+0000 7f1ec647c700 1 -- 172.21.15.94:0/3380867866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ec0107cc0 msgr2=0x7f1ec01016c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.679+0000 7f1ec647c700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1ec0107cc0 0x7f1ec01016c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.679+0000 7f1ec647c700 1 -- 172.21.15.94:0/3380867866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb8009d00 con 0x7f1ec0069920 2024-02-17T11:56:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.679+0000 7f1ec647c700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec0069920 0x7f1ec0100c40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00e9a0 tx=0x7f1ebc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:39.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.679+0000 7f1eb77fe700 1 -- 172.21.15.94:0/3380867866 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc009790 con 0x7f1ec0069920 2024-02-17T11:56:39.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.679+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3380867866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec01a6c90 con 0x7f1ec0069920 2024-02-17T11:56:39.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.679+0000 7f1eb77fe700 1 -- 172.21.15.94:0/3380867866 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ebc010430 con 0x7f1ec0069920 2024-02-17T11:56:39.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.680+0000 7f1eb77fe700 1 -- 172.21.15.94:0/3380867866 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc018680 con 0x7f1ec0069920 2024-02-17T11:56:39.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.680+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3380867866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec01a7190 con 0x7f1ec0069920 2024-02-17T11:56:39.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.680+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3380867866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1ec0111f80 con 0x7f1ec0069920 2024-02-17T11:56:39.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.682+0000 7f1eb77fe700 1 -- 172.21.15.94:0/3380867866 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f1ebc010910 con 0x7f1ec0069920 2024-02-17T11:56:39.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.683+0000 7f1eb77fe700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1eac0a21e0 0x7f1eac0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:39.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.683+0000 7f1eb77fe700 1 -- 172.21.15.94:0/3380867866 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1ebc014070 con 0x7f1ec0069920 2024-02-17T11:56:39.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.683+0000 7f1ec5c7b700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1eac0a21e0 0x7f1eac0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:39.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.683+0000 7f1eb77fe700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517] conn(0x7f1eac0ab9c0 0x7f1eac0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:39.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.683+0000 7f1eb77fe700 1 -- 172.21.15.94:0/3380867866 --> [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1eac0ae490 con 0x7f1eac0ab9c0 2024-02-17T11:56:39.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.683+0000 7f1eb77fe700 1 -- 172.21.15.94:0/3380867866 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ebc0cec80 con 0x7f1ec0069920 2024-02-17T11:56:39.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.683+0000 7f1ec5c7b700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1eac0a21e0 0x7f1eac0a46a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0078010 tx=0x7f1eb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:39.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.683+0000 7f1ec6c7d700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517] conn(0x7f1eac0ab9c0 0x7f1eac0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:39.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.684+0000 7f1ec6c7d700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517] conn(0x7f1eac0ab9c0 0x7f1eac0adde0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:39.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.685+0000 7f1eb77fe700 1 -- 172.21.15.94:0/3380867866 <== osd.7 v2:172.21.15.125:6816/2400671517 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1eac0ae490 con 0x7f1eac0ab9c0 2024-02-17T11:56:39.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.708+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3380867866 --> [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1ec0062380 con 0x7f1eac0ab9c0 2024-02-17T11:56:39.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.708+0000 7f1eb77fe700 1 -- 172.21.15.94:0/3380867866 <== osd.7 v2:172.21.15.125:6816/2400671517 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1ec0062380 con 0x7f1eac0ab9c0 2024-02-17T11:56:39.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.709+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3380867866 >> [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517] conn(0x7f1eac0ab9c0 msgr2=0x7f1eac0adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:39.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.709+0000 7f1ec86e0700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517] conn(0x7f1eac0ab9c0 0x7f1eac0adde0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:39.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.709+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3380867866 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1eac0a21e0 msgr2=0x7f1eac0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:39.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.709+0000 7f1ec86e0700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f1eac0a21e0 0x7f1eac0a46a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0078010 tx=0x7f1eb0009260 comp rx=0 tx=0).stop 2024-02-17T11:56:39.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.709+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3380867866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec0069920 msgr2=0x7f1ec0100c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:39.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.709+0000 7f1ec86e0700 1 --2- 172.21.15.94:0/3380867866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1ec0069920 0x7f1ec0100c40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00e9a0 tx=0x7f1ebc00ecb0 comp rx=0 tx=0).stop 2024-02-17T11:56:39.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.709+0000 7f1ec6c7d700 1 -- 172.21.15.94:0/3380867866 reap_dead start 2024-02-17T11:56:39.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.709+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3380867866 shutdown_connections 2024-02-17T11:56:39.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.709+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3380867866 >> 172.21.15.94:0/3380867866 conn(0x7f1ec0077aa0 msgr2=0x7f1ec00ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:39.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.709+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3380867866 shutdown_connections 2024-02-17T11:56:39.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:37.709+0000 7f1ec86e0700 1 -- 172.21.15.94:0/3380867866 wait complete. 2024-02-17T11:56:39.783 INFO:teuthology.orchestra.run.smithi094.stdout:55834574919 2024-02-17T11:56:39.783 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-17T11:56:39.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:39 smithi094 ceph-mon[32499]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-17T11:56:40.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:39 smithi189 ceph-mon[36588]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-17T11:56:40.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:39 smithi125 ceph-mon[36819]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-02-17T11:56:40.203 INFO:teuthology.orchestra.run.smithi094.stdout:219043332137 2024-02-17T11:56:40.203 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-17T11:56:40.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.415+0000 7fce2dea2700 1 -- 172.21.15.94:0/3956932004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce281015d0 msgr2=0x7fce281019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:40.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.415+0000 7fce2dea2700 1 --2- 172.21.15.94:0/3956932004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce281015d0 0x7fce281019b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fce18009990 tx=0x7fce18009ca0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.415+0000 7fce2dea2700 1 -- 172.21.15.94:0/3956932004 shutdown_connections 2024-02-17T11:56:40.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.415+0000 7fce2dea2700 1 --2- 172.21.15.94:0/3956932004 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce28105ef0 0x7fce2810dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.415+0000 7fce2dea2700 1 --2- 172.21.15.94:0/3956932004 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce281015d0 0x7fce281019b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.415+0000 7fce2dea2700 1 --2- 172.21.15.94:0/3956932004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce28101f80 0x7fce28105810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.415+0000 7fce2dea2700 1 -- 172.21.15.94:0/3956932004 >> 172.21.15.94:0/3956932004 conn(0x7fce28077170 msgr2=0x7fce280fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:40.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.415+0000 7fce2dea2700 1 -- 172.21.15.94:0/3956932004 shutdown_connections 2024-02-17T11:56:40.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.415+0000 7fce2dea2700 1 -- 172.21.15.94:0/3956932004 wait complete. 2024-02-17T11:56:40.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.415+0000 7fce2dea2700 1 Processor -- start 2024-02-17T11:56:40.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce2dea2700 1 -- start start 2024-02-17T11:56:40.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce2dea2700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce281015d0 0x7fce28199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:40.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce2dea2700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce28101f80 0x7fce28199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce2dea2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce28105ef0 0x7fce281a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce2dea2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce28199ee0 con 0x7fce28105ef0 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce2dea2700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce281a46c0 con 0x7fce281015d0 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce2dea2700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce281a4800 con 0x7fce28101f80 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce27fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce28105ef0 0x7fce281a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce27fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce28105ef0 0x7fce281a4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59404/0 (socket says 172.21.15.94:59404) 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce27fff700 1 -- 172.21.15.94:0/3118529961 learned_addr learned my addr 172.21.15.94:0/3118529961 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce277fe700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce281015d0 0x7fce28199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce26ffd700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce28101f80 0x7fce28199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce27fff700 1 -- 172.21.15.94:0/3118529961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce28101f80 msgr2=0x7fce28199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce27fff700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce28101f80 0x7fce28199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce27fff700 1 -- 172.21.15.94:0/3118529961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce281015d0 msgr2=0x7fce28199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce27fff700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fce281015d0 0x7fce28199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.416+0000 7fce27fff700 1 -- 172.21.15.94:0/3118529961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce180095f0 con 0x7fce28105ef0 2024-02-17T11:56:40.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.417+0000 7fce27fff700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce28105ef0 0x7fce281a4180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fce2000b9a0 tx=0x7fce2000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:40.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.417+0000 7fce24ff9700 1 -- 172.21.15.94:0/3118529961 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2000d850 con 0x7fce28105ef0 2024-02-17T11:56:40.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.417+0000 7fce2dea2700 1 -- 172.21.15.94:0/3118529961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce281a4ab0 con 0x7fce28105ef0 2024-02-17T11:56:40.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.417+0000 7fce24ff9700 1 -- 172.21.15.94:0/3118529961 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fce2000de90 con 0x7fce28105ef0 2024-02-17T11:56:40.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.417+0000 7fce24ff9700 1 -- 172.21.15.94:0/3118529961 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20017420 con 0x7fce28105ef0 2024-02-17T11:56:40.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.417+0000 7fce2dea2700 1 -- 172.21.15.94:0/3118529961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce28196ef0 con 0x7fce28105ef0 2024-02-17T11:56:40.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.418+0000 7fce2dea2700 1 -- 172.21.15.94:0/3118529961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fce28110f90 con 0x7fce28105ef0 2024-02-17T11:56:40.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.419+0000 7fce24ff9700 1 -- 172.21.15.94:0/3118529961 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fce2000e6b0 con 0x7fce28105ef0 2024-02-17T11:56:40.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.419+0000 7fce24ff9700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fce100a65f0 0x7fce100a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:40.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.420+0000 7fce277fe700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fce100a65f0 0x7fce100a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:40.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.420+0000 7fce24ff9700 1 -- 172.21.15.94:0/3118529961 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fce20006e40 con 0x7fce28105ef0 2024-02-17T11:56:40.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.420+0000 7fce24ff9700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253] conn(0x7fce100afdd0 0x7fce100b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:40.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.420+0000 7fce24ff9700 1 -- 172.21.15.94:0/3118529961 --> [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce100b28a0 con 0x7fce100afdd0 2024-02-17T11:56:40.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.420+0000 7fce24ff9700 1 -- 172.21.15.94:0/3118529961 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce200cdf60 con 0x7fce28105ef0 2024-02-17T11:56:40.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.420+0000 7fce277fe700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fce100a65f0 0x7fce100a8ab0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fce1800b5c0 tx=0x7fce18005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:40.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.420+0000 7fce26ffd700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253] conn(0x7fce100afdd0 0x7fce100b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:40.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.420+0000 7fce26ffd700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253] conn(0x7fce100afdd0 0x7fce100b21f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:40.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.421+0000 7fce24ff9700 1 -- 172.21.15.94:0/3118529961 <== osd.8 v2:172.21.15.189:6816/3010520253 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fce100b28a0 con 0x7fce100afdd0 2024-02-17T11:56:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.438+0000 7fce2dea2700 1 -- 172.21.15.94:0/3118529961 --> [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fce28062300 con 0x7fce100afdd0 2024-02-17T11:56:40.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.439+0000 7fce24ff9700 1 -- 172.21.15.94:0/3118529961 <== osd.8 v2:172.21.15.189:6816/3010520253 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fce28062300 con 0x7fce100afdd0 2024-02-17T11:56:40.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.439+0000 7fce2dea2700 1 -- 172.21.15.94:0/3118529961 >> [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253] conn(0x7fce100afdd0 msgr2=0x7fce100b21f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:40.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.439+0000 7fce2dea2700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253] conn(0x7fce100afdd0 0x7fce100b21f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.439+0000 7fce2dea2700 1 -- 172.21.15.94:0/3118529961 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fce100a65f0 msgr2=0x7fce100a8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.439+0000 7fce2dea2700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fce100a65f0 0x7fce100a8ab0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fce1800b5c0 tx=0x7fce18005960 comp rx=0 tx=0).stop 2024-02-17T11:56:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.439+0000 7fce2dea2700 1 -- 172.21.15.94:0/3118529961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce28105ef0 msgr2=0x7fce281a4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.439+0000 7fce2dea2700 1 --2- 172.21.15.94:0/3118529961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fce28105ef0 0x7fce281a4180 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fce2000b9a0 tx=0x7fce2000bcb0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.439+0000 7fce27fff700 1 -- 172.21.15.94:0/3118529961 reap_dead start 2024-02-17T11:56:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.440+0000 7fce2dea2700 1 -- 172.21.15.94:0/3118529961 shutdown_connections 2024-02-17T11:56:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.440+0000 7fce2dea2700 1 -- 172.21.15.94:0/3118529961 >> 172.21.15.94:0/3118529961 conn(0x7fce28077170 msgr2=0x7fce280fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.440+0000 7fce2dea2700 1 -- 172.21.15.94:0/3118529961 shutdown_connections 2024-02-17T11:56:40.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.440+0000 7fce2dea2700 1 -- 172.21.15.94:0/3118529961 wait complete. 2024-02-17T11:56:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.648+0000 7f11c4fbd700 1 -- 172.21.15.94:0/944421369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c0100e40 msgr2=0x7f11c0101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.648+0000 7f11c4fbd700 1 --2- 172.21.15.94:0/944421369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c0100e40 0x7f11c0101220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f11b0009990 tx=0x7f11b0009ca0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.648+0000 7f11c4fbd700 1 -- 172.21.15.94:0/944421369 shutdown_connections 2024-02-17T11:56:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.648+0000 7f11c4fbd700 1 --2- 172.21.15.94:0/944421369 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11c0105f20 0x7f11c010daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.648+0000 7f11c4fbd700 1 --2- 172.21.15.94:0/944421369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c0100e40 0x7f11c0101220 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.648+0000 7f11c4fbd700 1 --2- 172.21.15.94:0/944421369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11c01017f0 0x7f11c0105840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.648+0000 7f11c4fbd700 1 -- 172.21.15.94:0/944421369 >> 172.21.15.94:0/944421369 conn(0x7f11c0077ca0 msgr2=0x7f11c00780b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.648+0000 7f11c4fbd700 1 -- 172.21.15.94:0/944421369 shutdown_connections 2024-02-17T11:56:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.648+0000 7f11c4fbd700 1 -- 172.21.15.94:0/944421369 wait complete. 2024-02-17T11:56:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.648+0000 7f11c4fbd700 1 Processor -- start 2024-02-17T11:56:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11c4fbd700 1 -- start start 2024-02-17T11:56:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11c4fbd700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11c0100e40 0x7f11c0198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11c4fbd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c01017f0 0x7f11c0198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11c4fbd700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11c0105f20 0x7f11c019c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11c4fbd700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c019cff0 con 0x7f11c01017f0 2024-02-17T11:56:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11c4fbd700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c019d160 con 0x7f11c0105f20 2024-02-17T11:56:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11c4fbd700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c019d320 con 0x7f11c0100e40 2024-02-17T11:56:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11bdd9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c01017f0 0x7f11c0198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11bdd9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c01017f0 0x7f11c0198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59424/0 (socket says 172.21.15.94:59424) 2024-02-17T11:56:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11bdd9b700 1 -- 172.21.15.94:0/2539145233 learned_addr learned my addr 172.21.15.94:0/2539145233 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11be59c700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11c0100e40 0x7f11c0198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11bed9d700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11c0105f20 0x7f11c019c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11bdd9b700 1 -- 172.21.15.94:0/2539145233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11c0100e40 msgr2=0x7f11c0198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:40.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11bdd9b700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f11c0100e40 0x7f11c0198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11bdd9b700 1 -- 172.21.15.94:0/2539145233 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11c0105f20 msgr2=0x7f11c019c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11bdd9b700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11c0105f20 0x7f11c019c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11bdd9b700 1 -- 172.21.15.94:0/2539145233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b00095f0 con 0x7f11c01017f0 2024-02-17T11:56:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11bed9d700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f11c0105f20 0x7f11c019c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11bdd9b700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c01017f0 0x7f11c0198650 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f11b400c9e0 tx=0x7f11b400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11af7fe700 1 -- 172.21.15.94:0/2539145233 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b4007aa0 con 0x7f11c01017f0 2024-02-17T11:56:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11af7fe700 1 -- 172.21.15.94:0/2539145233 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11b4007c00 con 0x7f11c01017f0 2024-02-17T11:56:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11c4fbd700 1 -- 172.21.15.94:0/2539145233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c019d600 con 0x7f11c01017f0 2024-02-17T11:56:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11af7fe700 1 -- 172.21.15.94:0/2539145233 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b40186f0 con 0x7f11c01017f0 2024-02-17T11:56:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.649+0000 7f11c4fbd700 1 -- 172.21.15.94:0/2539145233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c01a4e60 con 0x7f11c01017f0 2024-02-17T11:56:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.650+0000 7f11ad7fa700 1 -- 172.21.15.94:0/2539145233 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f11c0110f00 con 0x7f11c01017f0 2024-02-17T11:56:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.651+0000 7f11af7fe700 1 -- 172.21.15.94:0/2539145233 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f11b400f450 con 0x7f11c01017f0 2024-02-17T11:56:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.652+0000 7f11af7fe700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f11a80a21e0 0x7f11a80a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.652+0000 7f11be59c700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f11a80a21e0 0x7f11a80a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.652+0000 7f11af7fe700 1 -- 172.21.15.94:0/2539145233 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f11b40ce080 con 0x7f11c01017f0 2024-02-17T11:56:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.652+0000 7f11af7fe700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558] conn(0x7f11a80ab9c0 0x7f11a80adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.652+0000 7f11af7fe700 1 -- 172.21.15.94:0/2539145233 --> [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f11a80ae490 con 0x7f11a80ab9c0 2024-02-17T11:56:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.652+0000 7f11af7fe700 1 -- 172.21.15.94:0/2539145233 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11b40ce520 con 0x7f11c01017f0 2024-02-17T11:56:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.652+0000 7f11bed9d700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558] conn(0x7f11a80ab9c0 0x7f11a80adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.653+0000 7f11be59c700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f11a80a21e0 0x7f11a80a46a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f11b0006010 tx=0x7f11b001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:40.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.653+0000 7f11bed9d700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558] conn(0x7f11a80ab9c0 0x7f11a80adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:40.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.654+0000 7f11af7fe700 1 -- 172.21.15.94:0/2539145233 <== osd.6 v2:172.21.15.94:6818/422477558 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f11a80ae490 con 0x7f11a80ab9c0 2024-02-17T11:56:40.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.671+0000 7f11ad7fa700 1 -- 172.21.15.94:0/2539145233 --> [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f11c0062300 con 0x7f11a80ab9c0 2024-02-17T11:56:40.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.671+0000 7f11af7fe700 1 -- 172.21.15.94:0/2539145233 <== osd.6 v2:172.21.15.94:6818/422477558 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f11c0062300 con 0x7f11a80ab9c0 2024-02-17T11:56:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.672+0000 7f11ad7fa700 1 -- 172.21.15.94:0/2539145233 >> [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558] conn(0x7f11a80ab9c0 msgr2=0x7f11a80adde0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.672+0000 7f11ad7fa700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558] conn(0x7f11a80ab9c0 0x7f11a80adde0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.672+0000 7f11ad7fa700 1 -- 172.21.15.94:0/2539145233 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f11a80a21e0 msgr2=0x7f11a80a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.672+0000 7f11ad7fa700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f11a80a21e0 0x7f11a80a46a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f11b0006010 tx=0x7f11b001a040 comp rx=0 tx=0).stop 2024-02-17T11:56:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.672+0000 7f11ad7fa700 1 -- 172.21.15.94:0/2539145233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c01017f0 msgr2=0x7f11c0198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.672+0000 7f11ad7fa700 1 --2- 172.21.15.94:0/2539145233 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f11c01017f0 0x7f11c0198650 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f11b400c9e0 tx=0x7f11b400cda0 comp rx=0 tx=0).stop 2024-02-17T11:56:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.672+0000 7f11bed9d700 1 -- 172.21.15.94:0/2539145233 reap_dead start 2024-02-17T11:56:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.672+0000 7f11ad7fa700 1 -- 172.21.15.94:0/2539145233 shutdown_connections 2024-02-17T11:56:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.672+0000 7f11ad7fa700 1 -- 172.21.15.94:0/2539145233 >> 172.21.15.94:0/2539145233 conn(0x7f11c0077ca0 msgr2=0x7f11c006a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.672+0000 7f11ad7fa700 1 -- 172.21.15.94:0/2539145233 shutdown_connections 2024-02-17T11:56:40.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:40.672+0000 7f11ad7fa700 1 -- 172.21.15.94:0/2539145233 wait complete. 2024-02-17T11:56:41.201 INFO:teuthology.orchestra.run.smithi094.stdout:141733920824 2024-02-17T11:56:41.201 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-17T11:56:41.772 INFO:teuthology.orchestra.run.smithi094.stdout:253403070501 2024-02-17T11:56:41.773 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-02-17T11:56:42.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:41 smithi189 ceph-mon[36588]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-17T11:56:42.153 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:41 smithi094 ceph-mon[32499]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-17T11:56:42.173 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:41 smithi125 ceph-mon[36819]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-17T11:56:42.503 INFO:teuthology.orchestra.run.smithi094.stdout:34359738444 2024-02-17T11:56:42.503 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-17T11:56:42.938 INFO:teuthology.orchestra.run.smithi094.stdout:197568495663 2024-02-17T11:56:42.938 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-17T11:56:43.773 INFO:teuthology.orchestra.run.smithi094.stdout:343597383701 2024-02-17T11:56:43.774 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-02-17T11:56:43.774 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:43.800 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:43.804 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:43.804 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:43.807 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:43.812 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:43.840 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:43.840 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:44.047 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:43 smithi094 ceph-mon[32499]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-17T11:56:44.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:43 smithi189 ceph-mon[36588]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-17T11:56:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:43 smithi125 ceph-mon[36819]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-17T11:56:46.058 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:45 smithi094 ceph-mon[32499]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-17T11:56:46.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:45 smithi189 ceph-mon[36588]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-17T11:56:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:45 smithi125 ceph-mon[36819]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-17T11:56:47.410 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:48.081 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:47 smithi094 ceph-mon[32499]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T11:56:48.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:47 smithi189 ceph-mon[36588]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T11:56:48.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:47 smithi125 ceph-mon[36819]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T11:56:48.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.937+0000 7f281ee15700 1 -- 172.21.15.94:0/2986495040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2818107000 msgr2=0x7f281810eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:48.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.937+0000 7f281ee15700 1 --2- 172.21.15.94:0/2986495040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2818107000 0x7f281810eb80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f281000b0a0 tx=0x7f281000b3b0 comp rx=0 tx=0).stop 2024-02-17T11:56:48.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.937+0000 7f281ee15700 1 -- 172.21.15.94:0/2986495040 shutdown_connections 2024-02-17T11:56:48.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.937+0000 7f281ee15700 1 --2- 172.21.15.94:0/2986495040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2818107000 0x7f281810eb80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:48.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.937+0000 7f281ee15700 1 --2- 172.21.15.94:0/2986495040 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2818101f50 0x7f2818102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:48.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.937+0000 7f281ee15700 1 --2- 172.21.15.94:0/2986495040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2818102900 0x7f28181068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.937+0000 7f281ee15700 1 -- 172.21.15.94:0/2986495040 >> 172.21.15.94:0/2986495040 conn(0x7f2818078d90 msgr2=0x7f28180791a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.937+0000 7f281ee15700 1 -- 172.21.15.94:0/2986495040 shutdown_connections 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.937+0000 7f281ee15700 1 -- 172.21.15.94:0/2986495040 wait complete. 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281ee15700 1 Processor -- start 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281ee15700 1 -- start start 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281ee15700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2818101f50 0x7f2818198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281ee15700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2818102900 0x7f28181987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281ee15700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2818107000 0x7f281819cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281ee15700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281819d150 con 0x7f2818107000 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281ee15700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281819d2c0 con 0x7f2818101f50 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281ee15700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281819d480 con 0x7f2818102900 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281d3b2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2818107000 0x7f281819cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281d3b2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2818107000 0x7f281819cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36850/0 (socket says 172.21.15.94:36850) 2024-02-17T11:56:48.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281d3b2700 1 -- 172.21.15.94:0/1286318694 learned_addr learned my addr 172.21.15.94:0/1286318694 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:48.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281cbb1700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2818101f50 0x7f2818198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:48.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f2817fff700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2818102900 0x7f28181987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:48.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281d3b2700 1 -- 172.21.15.94:0/1286318694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2818102900 msgr2=0x7f28181987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:48.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281d3b2700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2818102900 0x7f28181987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:48.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281d3b2700 1 -- 172.21.15.94:0/1286318694 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2818101f50 msgr2=0x7f2818198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:48.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.938+0000 7f281d3b2700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2818101f50 0x7f2818198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:48.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.939+0000 7f281d3b2700 1 -- 172.21.15.94:0/1286318694 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2810009d00 con 0x7f2818107000 2024-02-17T11:56:48.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.939+0000 7f2817fff700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2818102900 0x7f28181987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:56:48.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.939+0000 7f281d3b2700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2818107000 0x7f281819cb40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f281000b070 tx=0x7f2810004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:48.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.939+0000 7f2815ffb700 1 -- 172.21.15.94:0/1286318694 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281000e040 con 0x7f2818107000 2024-02-17T11:56:48.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.939+0000 7f281ee15700 1 -- 172.21.15.94:0/1286318694 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281819d700 con 0x7f2818107000 2024-02-17T11:56:48.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.939+0000 7f2815ffb700 1 -- 172.21.15.94:0/1286318694 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2810007950 con 0x7f2818107000 2024-02-17T11:56:48.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.939+0000 7f2815ffb700 1 -- 172.21.15.94:0/1286318694 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281001b8f0 con 0x7f2818107000 2024-02-17T11:56:48.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.939+0000 7f281ee15700 1 -- 172.21.15.94:0/1286318694 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28181a4fc0 con 0x7f2818107000 2024-02-17T11:56:48.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.940+0000 7f281ee15700 1 -- 172.21.15.94:0/1286318694 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281810a1f0 con 0x7f2818107000 2024-02-17T11:56:48.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.941+0000 7f2815ffb700 1 -- 172.21.15.94:0/1286318694 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f281001ba50 con 0x7f2818107000 2024-02-17T11:56:48.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.942+0000 7f2815ffb700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f28080a21e0 0x7f28080a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:48.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.942+0000 7f281cbb1700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f28080a21e0 0x7f28080a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:48.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.942+0000 7f2815ffb700 1 -- 172.21.15.94:0/1286318694 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f28100cf6c0 con 0x7f2818107000 2024-02-17T11:56:48.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.942+0000 7f281cbb1700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f28080a21e0 0x7f28080a46a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f280c005950 tx=0x7f280c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:48.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:48.946+0000 7f2815ffb700 1 -- 172.21.15.94:0/1286318694 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2810098670 con 0x7f2818107000 2024-02-17T11:56:49.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.143+0000 7f281ee15700 1 -- 172.21.15.94:0/1286318694 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f28181995d0 con 0x7f2818107000 2024-02-17T11:56:49.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.143+0000 7f2815ffb700 1 -- 172.21.15.94:0/1286318694 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f281009bd30 con 0x7f2818107000 2024-02-17T11:56:49.143 INFO:teuthology.orchestra.run.smithi094.stdout:107374182463 2024-02-17T11:56:49.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.146+0000 7f281ee15700 1 -- 172.21.15.94:0/1286318694 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f28080a21e0 msgr2=0x7f28080a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:49.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.146+0000 7f281ee15700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f28080a21e0 0x7f28080a46a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f280c005950 tx=0x7f280c00b410 comp rx=0 tx=0).stop 2024-02-17T11:56:49.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.146+0000 7f281ee15700 1 -- 172.21.15.94:0/1286318694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2818107000 msgr2=0x7f281819cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:49.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.146+0000 7f281ee15700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2818107000 0x7f281819cb40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f281000b070 tx=0x7f2810004e40 comp rx=0 tx=0).stop 2024-02-17T11:56:49.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.146+0000 7f281ee15700 1 -- 172.21.15.94:0/1286318694 shutdown_connections 2024-02-17T11:56:49.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.146+0000 7f281ee15700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f28080a21e0 0x7f28080a46a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:49.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.146+0000 7f281ee15700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2818107000 0x7f281819cb40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:49.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.146+0000 7f281ee15700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2818101f50 0x7f2818198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:49.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.146+0000 7f281ee15700 1 --2- 172.21.15.94:0/1286318694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2818102900 0x7f28181987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:49.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.146+0000 7f281ee15700 1 -- 172.21.15.94:0/1286318694 >> 172.21.15.94:0/1286318694 conn(0x7f2818078d90 msgr2=0x7f28180ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:49.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.146+0000 7f281ee15700 1 -- 172.21.15.94:0/1286318694 shutdown_connections 2024-02-17T11:56:49.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:49.146+0000 7f281ee15700 1 -- 172.21.15.94:0/1286318694 wait complete. 2024-02-17T11:56:49.236 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:49.450 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:56:49.862 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182460 got 107374182463 for osd.3 2024-02-17T11:56:49.862 DEBUG:teuthology.parallel:result is None 2024-02-17T11:56:49.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:49 smithi094 ceph-mon[32499]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-17T11:56:49.977 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:49 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1286318694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-17T11:56:50.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:49 smithi189 ceph-mon[36588]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-17T11:56:50.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:49 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1286318694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-17T11:56:50.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:49 smithi125 ceph-mon[36819]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-17T11:56:50.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:49 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1286318694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-17T11:56:50.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.259+0000 7faa82fa0700 1 -- 172.21.15.94:0/403291924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c106a60 msgr2=0x7faa7c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:50.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.259+0000 7faa82fa0700 1 --2- 172.21.15.94:0/403291924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c106a60 0x7faa7c106e40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faa78009990 tx=0x7faa78009ca0 comp rx=0 tx=0).stop 2024-02-17T11:56:50.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.259+0000 7faa82fa0700 1 -- 172.21.15.94:0/403291924 shutdown_connections 2024-02-17T11:56:50.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.259+0000 7faa82fa0700 1 --2- 172.21.15.94:0/403291924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa7c101420 0x7faa7c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:50.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.259+0000 7faa82fa0700 1 --2- 172.21.15.94:0/403291924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c106a60 0x7faa7c106e40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:50.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.259+0000 7faa82fa0700 1 --2- 172.21.15.94:0/403291924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa7c100a60 0x7faa7c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:50.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.259+0000 7faa82fa0700 1 -- 172.21.15.94:0/403291924 >> 172.21.15.94:0/403291924 conn(0x7faa7c0fc530 msgr2=0x7faa7c0fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:50.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.259+0000 7faa82fa0700 1 -- 172.21.15.94:0/403291924 shutdown_connections 2024-02-17T11:56:50.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.259+0000 7faa82fa0700 1 -- 172.21.15.94:0/403291924 wait complete. 2024-02-17T11:56:50.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.260+0000 7faa82fa0700 1 Processor -- start 2024-02-17T11:56:50.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.260+0000 7faa82fa0700 1 -- start start 2024-02-17T11:56:50.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.260+0000 7faa82fa0700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa7c100a60 0x7faa7c0fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:50.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.260+0000 7faa82fa0700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c101420 0x7faa7c0fcf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:50.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.260+0000 7faa82fa0700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa7c106a60 0x7faa7c0fd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:50.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.260+0000 7faa82fa0700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c1a6490 con 0x7faa7c101420 2024-02-17T11:56:50.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.260+0000 7faa82fa0700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c1a6600 con 0x7faa7c106a60 2024-02-17T11:56:50.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.260+0000 7faa82fa0700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c1a67c0 con 0x7faa7c100a60 2024-02-17T11:56:50.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.260+0000 7faa8179d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c101420 0x7faa7c0fcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:50.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.260+0000 7faa8179d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c101420 0x7faa7c0fcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36860/0 (socket says 172.21.15.94:36860) 2024-02-17T11:56:50.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa8179d700 1 -- 172.21.15.94:0/913138763 learned_addr learned my addr 172.21.15.94:0/913138763 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:50.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa8279f700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa7c106a60 0x7faa7c0fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:50.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa81f9e700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa7c100a60 0x7faa7c0fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:50.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa8179d700 1 -- 172.21.15.94:0/913138763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa7c100a60 msgr2=0x7faa7c0fc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:50.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa8179d700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa7c100a60 0x7faa7c0fc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:50.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa8179d700 1 -- 172.21.15.94:0/913138763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa7c106a60 msgr2=0x7faa7c0fd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:50.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa8179d700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa7c106a60 0x7faa7c0fd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:50.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa8179d700 1 -- 172.21.15.94:0/913138763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa780095f0 con 0x7faa7c101420 2024-02-17T11:56:50.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa81f9e700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa7c100a60 0x7faa7c0fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:56:50.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa8179d700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c101420 0x7faa7c0fcf20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00c9b0 tx=0x7faa6c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:50.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa72ffd700 1 -- 172.21.15.94:0/913138763 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c007b00 con 0x7faa7c101420 2024-02-17T11:56:50.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa72ffd700 1 -- 172.21.15.94:0/913138763 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faa6c007c60 con 0x7faa7c101420 2024-02-17T11:56:50.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa82fa0700 1 -- 172.21.15.94:0/913138763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7c1a6aa0 con 0x7faa7c101420 2024-02-17T11:56:50.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa72ffd700 1 -- 172.21.15.94:0/913138763 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c018660 con 0x7faa7c101420 2024-02-17T11:56:50.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.261+0000 7faa82fa0700 1 -- 172.21.15.94:0/913138763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c1a6ff0 con 0x7faa7c101420 2024-02-17T11:56:50.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.262+0000 7faa82fa0700 1 -- 172.21.15.94:0/913138763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7c10a070 con 0x7faa7c101420 2024-02-17T11:56:50.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.263+0000 7faa72ffd700 1 -- 172.21.15.94:0/913138763 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7faa6c00f450 con 0x7faa7c101420 2024-02-17T11:56:50.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.264+0000 7faa72ffd700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faa680a2260 0x7faa680a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:50.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.264+0000 7faa81f9e700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faa680a2260 0x7faa680a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:50.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.264+0000 7faa72ffd700 1 -- 172.21.15.94:0/913138763 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7faa6c0cde90 con 0x7faa7c101420 2024-02-17T11:56:50.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.265+0000 7faa81f9e700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faa680a2260 0x7faa680a4720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa78003800 tx=0x7faa78005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:50.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.268+0000 7faa72ffd700 1 -- 172.21.15.94:0/913138763 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa6c096fc0 con 0x7faa7c101420 2024-02-17T11:56:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.468+0000 7faa82fa0700 1 -- 172.21.15.94:0/913138763 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faa7c068930 con 0x7faa7c101420 2024-02-17T11:56:50.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.469+0000 7faa72ffd700 1 -- 172.21.15.94:0/913138763 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faa6c09a680 con 0x7faa7c101420 2024-02-17T11:56:50.468 INFO:teuthology.orchestra.run.smithi094.stdout:34359738446 2024-02-17T11:56:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.471+0000 7faa82fa0700 1 -- 172.21.15.94:0/913138763 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faa680a2260 msgr2=0x7faa680a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.471+0000 7faa82fa0700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faa680a2260 0x7faa680a4720 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7faa78003800 tx=0x7faa78005fd0 comp rx=0 tx=0).stop 2024-02-17T11:56:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.471+0000 7faa82fa0700 1 -- 172.21.15.94:0/913138763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c101420 msgr2=0x7faa7c0fcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.471+0000 7faa82fa0700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c101420 0x7faa7c0fcf20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00c9b0 tx=0x7faa6c00ccc0 comp rx=0 tx=0).stop 2024-02-17T11:56:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.471+0000 7faa82fa0700 1 -- 172.21.15.94:0/913138763 shutdown_connections 2024-02-17T11:56:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.471+0000 7faa82fa0700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faa680a2260 0x7faa680a4720 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.471+0000 7faa82fa0700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faa7c106a60 0x7faa7c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.471+0000 7faa82fa0700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faa7c100a60 0x7faa7c0fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:50.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.471+0000 7faa82fa0700 1 --2- 172.21.15.94:0/913138763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faa7c101420 0x7faa7c0fcf20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.471+0000 7faa82fa0700 1 -- 172.21.15.94:0/913138763 >> 172.21.15.94:0/913138763 conn(0x7faa7c0fc530 msgr2=0x7faa7c06a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.471+0000 7faa82fa0700 1 -- 172.21.15.94:0/913138763 shutdown_connections 2024-02-17T11:56:50.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:50.471+0000 7faa82fa0700 1 -- 172.21.15.94:0/913138763 wait complete. 2024-02-17T11:56:51.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:50 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/913138763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-17T11:56:51.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:50 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/913138763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-17T11:56:51.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:50 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/913138763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-17T11:56:51.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:51 smithi125 ceph-mon[36819]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-17T11:56:52.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:51 smithi189 ceph-mon[36588]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-17T11:56:52.154 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:51 smithi094 ceph-mon[32499]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-17T11:56:53.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.386+0000 7fc90d854700 1 -- 172.21.15.94:0/2827917340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc908107040 msgr2=0x7fc90810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:53.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.386+0000 7fc90d854700 1 --2- 172.21.15.94:0/2827917340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc908107040 0x7fc90810ec50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc90000b0a0 tx=0x7fc90000b3b0 comp rx=0 tx=0).stop 2024-02-17T11:56:53.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.386+0000 7fc90d854700 1 -- 172.21.15.94:0/2827917340 shutdown_connections 2024-02-17T11:56:53.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.386+0000 7fc90d854700 1 --2- 172.21.15.94:0/2827917340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc908107040 0x7fc90810ec50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:53.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.386+0000 7fc90d854700 1 --2- 172.21.15.94:0/2827917340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc908100870 0x7fc908100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:53.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.386+0000 7fc90d854700 1 --2- 172.21.15.94:0/2827917340 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc908101220 0x7fc908106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:53.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.386+0000 7fc90d854700 1 -- 172.21.15.94:0/2827917340 >> 172.21.15.94:0/2827917340 conn(0x7fc908075ef0 msgr2=0x7fc908076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:53.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.386+0000 7fc90d854700 1 -- 172.21.15.94:0/2827917340 shutdown_connections 2024-02-17T11:56:53.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.386+0000 7fc90d854700 1 -- 172.21.15.94:0/2827917340 wait complete. 2024-02-17T11:56:53.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc90d854700 1 Processor -- start 2024-02-17T11:56:53.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc90d854700 1 -- start start 2024-02-17T11:56:53.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc90d854700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc908100870 0x7fc908198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:53.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc90d854700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc908101220 0x7fc9081988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:53.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc90d854700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc908107040 0x7fc90819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:53.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc90d854700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90819d260 con 0x7fc908101220 2024-02-17T11:56:53.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc90d854700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90819d3d0 con 0x7fc908107040 2024-02-17T11:56:53.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc90d854700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90819d590 con 0x7fc908100870 2024-02-17T11:56:53.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc8fe7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc908101220 0x7fc9081988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:53.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc8fe7fc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc908101220 0x7fc9081988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36868/0 (socket says 172.21.15.94:36868) 2024-02-17T11:56:53.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc8fe7fc700 1 -- 172.21.15.94:0/1709902913 learned_addr learned my addr 172.21.15.94:0/1709902913 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:53.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc906ffd700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc908100870 0x7fc908198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc9077fe700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc908107040 0x7fc90819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc8fe7fc700 1 -- 172.21.15.94:0/1709902913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc908100870 msgr2=0x7fc908198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc8fe7fc700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc908100870 0x7fc908198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc8fe7fc700 1 -- 172.21.15.94:0/1709902913 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc908107040 msgr2=0x7fc90819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc8fe7fc700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc908107040 0x7fc90819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc8fe7fc700 1 -- 172.21.15.94:0/1709902913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc900009d00 con 0x7fc908101220 2024-02-17T11:56:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.387+0000 7fc9077fe700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc908107040 0x7fc90819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:56:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.388+0000 7fc8fe7fc700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc908101220 0x7fc9081988c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000d980 tx=0x7fc8f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.388+0000 7fc90c852700 1 -- 172.21.15.94:0/1709902913 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f0009930 con 0x7fc908101220 2024-02-17T11:56:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.388+0000 7fc90c852700 1 -- 172.21.15.94:0/1709902913 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8f0010460 con 0x7fc908101220 2024-02-17T11:56:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.388+0000 7fc90d854700 1 -- 172.21.15.94:0/1709902913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90819d870 con 0x7fc908101220 2024-02-17T11:56:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.388+0000 7fc90c852700 1 -- 172.21.15.94:0/1709902913 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000f620 con 0x7fc908101220 2024-02-17T11:56:53.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.388+0000 7fc90d854700 1 -- 172.21.15.94:0/1709902913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9081a50d0 con 0x7fc908101220 2024-02-17T11:56:53.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.389+0000 7fc90d854700 1 -- 172.21.15.94:0/1709902913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90810a3a0 con 0x7fc908101220 2024-02-17T11:56:53.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.390+0000 7fc90c852700 1 -- 172.21.15.94:0/1709902913 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fc8f00105d0 con 0x7fc908101220 2024-02-17T11:56:53.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.391+0000 7fc90c852700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc8f40a21e0 0x7fc8f40a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:53.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.391+0000 7fc90c852700 1 -- 172.21.15.94:0/1709902913 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc8f00cd750 con 0x7fc908101220 2024-02-17T11:56:53.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.391+0000 7fc906ffd700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc8f40a21e0 0x7fc8f40a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:53.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.391+0000 7fc906ffd700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc8f40a21e0 0x7fc8f40a46a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8005950 tx=0x7fc8f800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:53.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.397+0000 7fc90c852700 1 -- 172.21.15.94:0/1709902913 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc8f009a2a0 con 0x7fc908101220 2024-02-17T11:56:53.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.597+0000 7fc90d854700 1 -- 172.21.15.94:0/1709902913 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc9081996e0 con 0x7fc908101220 2024-02-17T11:56:53.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.598+0000 7fc90c852700 1 -- 172.21.15.94:0/1709902913 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc8f0017020 con 0x7fc908101220 2024-02-17T11:56:53.597 INFO:teuthology.orchestra.run.smithi094.stdout:141733920828 2024-02-17T11:56:53.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.600+0000 7fc90d854700 1 -- 172.21.15.94:0/1709902913 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc8f40a21e0 msgr2=0x7fc8f40a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:53.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.600+0000 7fc90d854700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc8f40a21e0 0x7fc8f40a46a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8005950 tx=0x7fc8f800a300 comp rx=0 tx=0).stop 2024-02-17T11:56:53.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.600+0000 7fc90d854700 1 -- 172.21.15.94:0/1709902913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc908101220 msgr2=0x7fc9081988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:53.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.600+0000 7fc90d854700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc908101220 0x7fc9081988c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc8f000d980 tx=0x7fc8f000dc90 comp rx=0 tx=0).stop 2024-02-17T11:56:53.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.600+0000 7fc90d854700 1 -- 172.21.15.94:0/1709902913 shutdown_connections 2024-02-17T11:56:53.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.600+0000 7fc90d854700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc8f40a21e0 0x7fc8f40a46a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:53.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.600+0000 7fc90d854700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc908107040 0x7fc90819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:53.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.600+0000 7fc90d854700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc908100870 0x7fc908198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:53.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.600+0000 7fc90d854700 1 --2- 172.21.15.94:0/1709902913 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc908101220 0x7fc9081988c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:53.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.600+0000 7fc90d854700 1 -- 172.21.15.94:0/1709902913 >> 172.21.15.94:0/1709902913 conn(0x7fc908075ef0 msgr2=0x7fc9080fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:53.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.600+0000 7fc90d854700 1 -- 172.21.15.94:0/1709902913 shutdown_connections 2024-02-17T11:56:53.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:53.600+0000 7fc90d854700 1 -- 172.21.15.94:0/1709902913 wait complete. 2024-02-17T11:56:53.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:53 smithi094 ceph-mon[32499]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-17T11:56:53.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:53 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1709902913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-17T11:56:54.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:53 smithi189 ceph-mon[36588]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-17T11:56:54.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:53 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1709902913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-17T11:56:54.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:53 smithi125 ceph-mon[36819]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-17T11:56:54.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:53 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1709902913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-17T11:56:54.485 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920824 got 141733920828 for osd.4 2024-02-17T11:56:54.485 DEBUG:teuthology.parallel:result is None 2024-02-17T11:56:54.903 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738444 got 34359738446 for osd.0 2024-02-17T11:56:54.904 DEBUG:teuthology.parallel:result is None 2024-02-17T11:56:55.110 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:54 smithi094 ceph-mon[32499]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-17T11:56:55.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:54 smithi189 ceph-mon[36588]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-17T11:56:55.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:54 smithi125 ceph-mon[36819]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-17T11:56:58.011 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:57 smithi094 ceph-mon[32499]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T11:56:58.011 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:57 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:56:58.019 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:57 smithi125 ceph-mon[36819]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T11:56:58.019 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:57 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:56:58.023 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:57 smithi189 ceph-mon[36588]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T11:56:58.023 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:57 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:56:58.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.230+0000 7fd960b3e700 1 -- 172.21.15.94:0/287782607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd95c100c20 msgr2=0x7fd95c10eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:58.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.230+0000 7fd960b3e700 1 --2- 172.21.15.94:0/287782607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd95c100c20 0x7fd95c10eb80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd95400b0a0 tx=0x7fd95400b3b0 comp rx=0 tx=0).stop 2024-02-17T11:56:58.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.230+0000 7fd960b3e700 1 -- 172.21.15.94:0/287782607 shutdown_connections 2024-02-17T11:56:58.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.230+0000 7fd960b3e700 1 --2- 172.21.15.94:0/287782607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd95c100c20 0x7fd95c10eb80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:58.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.230+0000 7fd960b3e700 1 --2- 172.21.15.94:0/287782607 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd95c106ce0 0x7fd95c1070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:58.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.230+0000 7fd960b3e700 1 --2- 172.21.15.94:0/287782607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd95c100260 0x7fd95c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:58.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.230+0000 7fd960b3e700 1 -- 172.21.15.94:0/287782607 >> 172.21.15.94:0/287782607 conn(0x7fd95c077170 msgr2=0x7fd95c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:58.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.230+0000 7fd960b3e700 1 -- 172.21.15.94:0/287782607 shutdown_connections 2024-02-17T11:56:58.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.230+0000 7fd960b3e700 1 -- 172.21.15.94:0/287782607 wait complete. 2024-02-17T11:56:58.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd960b3e700 1 Processor -- start 2024-02-17T11:56:58.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd960b3e700 1 -- start start 2024-02-17T11:56:58.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd960b3e700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd95c100260 0x7fd95c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:58.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd960b3e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd95c100c20 0x7fd95c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:58.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd960b3e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd95c106ce0 0x7fd95c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:58.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd960b3e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c19d290 con 0x7fd95c106ce0 2024-02-17T11:56:58.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd960b3e700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c19d400 con 0x7fd95c100260 2024-02-17T11:56:58.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd960b3e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c19d5c0 con 0x7fd95c100c20 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd959d9b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd95c100c20 0x7fd95c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd95a59c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd95c100260 0x7fd95c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd95ad9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd95c106ce0 0x7fd95c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd959d9b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd95c100c20 0x7fd95c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:36552/0 (socket says 172.21.15.94:36552) 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd959d9b700 1 -- 172.21.15.94:0/1516842898 learned_addr learned my addr 172.21.15.94:0/1516842898 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd95ad9d700 1 -- 172.21.15.94:0/1516842898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd95c100c20 msgr2=0x7fd95c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd95ad9d700 1 --2- 172.21.15.94:0/1516842898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd95c100c20 0x7fd95c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd95ad9d700 1 -- 172.21.15.94:0/1516842898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd95c100260 msgr2=0x7fd95c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd95ad9d700 1 --2- 172.21.15.94:0/1516842898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd95c100260 0x7fd95c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd95ad9d700 1 -- 172.21.15.94:0/1516842898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd954009d00 con 0x7fd95c106ce0 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.231+0000 7fd95a59c700 1 --2- 172.21.15.94:0/1516842898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd95c100260 0x7fd95c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.232+0000 7fd95ad9d700 1 --2- 172.21.15.94:0/1516842898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd95c106ce0 0x7fd95c19cc80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd95400b0a0 tx=0x7fd954004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.232+0000 7fd94b7fe700 1 -- 172.21.15.94:0/1516842898 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95400e040 con 0x7fd95c106ce0 2024-02-17T11:56:58.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.232+0000 7fd960b3e700 1 -- 172.21.15.94:0/1516842898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd95c19d840 con 0x7fd95c106ce0 2024-02-17T11:56:58.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.232+0000 7fd94b7fe700 1 -- 172.21.15.94:0/1516842898 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd954007a50 con 0x7fd95c106ce0 2024-02-17T11:56:58.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.232+0000 7fd960b3e700 1 -- 172.21.15.94:0/1516842898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95c101540 con 0x7fd95c106ce0 2024-02-17T11:56:58.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.232+0000 7fd94b7fe700 1 -- 172.21.15.94:0/1516842898 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95401b8c0 con 0x7fd95c106ce0 2024-02-17T11:56:58.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.233+0000 7fd960b3e700 1 -- 172.21.15.94:0/1516842898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95c10a2d0 con 0x7fd95c106ce0 2024-02-17T11:56:58.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.234+0000 7fd94b7fe700 1 -- 172.21.15.94:0/1516842898 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fd95401daf0 con 0x7fd95c106ce0 2024-02-17T11:56:58.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.235+0000 7fd94b7fe700 1 --2- 172.21.15.94:0/1516842898 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd9440a21e0 0x7fd9440a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:58.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.235+0000 7fd95a59c700 1 --2- 172.21.15.94:0/1516842898 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd9440a21e0 0x7fd9440a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:58.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.235+0000 7fd94b7fe700 1 -- 172.21.15.94:0/1516842898 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd9540d0000 con 0x7fd95c106ce0 2024-02-17T11:56:58.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.235+0000 7fd95a59c700 1 --2- 172.21.15.94:0/1516842898 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd9440a21e0 0x7fd9440a46a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009de0 tx=0x7fd94c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:58.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.238+0000 7fd94b7fe700 1 -- 172.21.15.94:0/1516842898 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd9540990b0 con 0x7fd95c106ce0 2024-02-17T11:56:58.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.435+0000 7fd960b3e700 1 -- 172.21.15.94:0/1516842898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fd95c068930 con 0x7fd95c106ce0 2024-02-17T11:56:58.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.436+0000 7fd94b7fe700 1 -- 172.21.15.94:0/1516842898 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fd95409c770 con 0x7fd95c106ce0 2024-02-17T11:56:58.435 INFO:teuthology.orchestra.run.smithi094.stdout:309237645342 2024-02-17T11:56:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.438+0000 7fd960b3e700 1 -- 172.21.15.94:0/1516842898 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd9440a21e0 msgr2=0x7fd9440a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.438+0000 7fd960b3e700 1 --2- 172.21.15.94:0/1516842898 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd9440a21e0 0x7fd9440a46a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009de0 tx=0x7fd94c009190 comp rx=0 tx=0).stop 2024-02-17T11:56:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.438+0000 7fd960b3e700 1 -- 172.21.15.94:0/1516842898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd95c106ce0 msgr2=0x7fd95c19cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.438+0000 7fd960b3e700 1 --2- 172.21.15.94:0/1516842898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd95c106ce0 0x7fd95c19cc80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd95400b0a0 tx=0x7fd954004e20 comp rx=0 tx=0).stop 2024-02-17T11:56:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.438+0000 7fd960b3e700 1 -- 172.21.15.94:0/1516842898 shutdown_connections 2024-02-17T11:56:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.438+0000 7fd960b3e700 1 --2- 172.21.15.94:0/1516842898 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd9440a21e0 0x7fd9440a46a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.438+0000 7fd960b3e700 1 --2- 172.21.15.94:0/1516842898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd95c106ce0 0x7fd95c19cc80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.438+0000 7fd960b3e700 1 --2- 172.21.15.94:0/1516842898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd95c100260 0x7fd95c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.438+0000 7fd960b3e700 1 --2- 172.21.15.94:0/1516842898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd95c100c20 0x7fd95c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.438+0000 7fd960b3e700 1 -- 172.21.15.94:0/1516842898 >> 172.21.15.94:0/1516842898 conn(0x7fd95c077170 msgr2=0x7fd95c104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.438+0000 7fd960b3e700 1 -- 172.21.15.94:0/1516842898 shutdown_connections 2024-02-17T11:56:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:58.438+0000 7fd960b3e700 1 -- 172.21.15.94:0/1516842898 wait complete. 2024-02-17T11:56:59.023 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:58 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1516842898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-17T11:56:59.023 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-02-17T11:56:59.023 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-02-17T11:56:59.023 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-02-17T11:56:59.023 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-02-17T11:56:59.023 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-02-17T11:56:59.023 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-02-17T11:56:59.024 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-02-17T11:56:59.024 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-02-17T11:56:59.024 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:58 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-02-17T11:56:59.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:58 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1516842898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-17T11:56:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-02-17T11:56:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-02-17T11:56:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-02-17T11:56:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-02-17T11:56:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-02-17T11:56:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-02-17T11:56:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-02-17T11:56:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-02-17T11:56:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:58 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-02-17T11:56:59.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:58 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1516842898' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-17T11:56:59.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-02-17T11:56:59.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-02-17T11:56:59.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-02-17T11:56:59.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-02-17T11:56:59.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-02-17T11:56:59.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-02-17T11:56:59.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-02-17T11:56:59.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]: dispatch 2024-02-17T11:56:59.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:58 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-02-17T11:56:59.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.372+0000 7f8ec589e700 1 -- 172.21.15.94:0/3983662602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec0106b80 msgr2=0x7f8ec0106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:59.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.372+0000 7f8ec589e700 1 --2- 172.21.15.94:0/3983662602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec0106b80 0x7f8ec0106f60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009990 tx=0x7f8eb0009ca0 comp rx=0 tx=0).stop 2024-02-17T11:56:59.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.372+0000 7f8ec589e700 1 -- 172.21.15.94:0/3983662602 shutdown_connections 2024-02-17T11:56:59.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.372+0000 7f8ec589e700 1 --2- 172.21.15.94:0/3983662602 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ec0101530 0x7f8ec010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:59.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.372+0000 7f8ec589e700 1 --2- 172.21.15.94:0/3983662602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec0106b80 0x7f8ec0106f60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:59.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.372+0000 7f8ec589e700 1 --2- 172.21.15.94:0/3983662602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ec0100b70 0x7f8ec0100ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:59.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.372+0000 7f8ec589e700 1 -- 172.21.15.94:0/3983662602 >> 172.21.15.94:0/3983662602 conn(0x7f8ec0077aa0 msgr2=0x7f8ec0077eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:59.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.372+0000 7f8ec589e700 1 -- 172.21.15.94:0/3983662602 shutdown_connections 2024-02-17T11:56:59.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.372+0000 7f8ec589e700 1 -- 172.21.15.94:0/3983662602 wait complete. 2024-02-17T11:56:59.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.373+0000 7f8ec589e700 1 Processor -- start 2024-02-17T11:56:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.373+0000 7f8ec589e700 1 -- start start 2024-02-17T11:56:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.373+0000 7f8ec589e700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ec0100b70 0x7f8ec0198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.373+0000 7f8ec589e700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ec0101530 0x7f8ec0199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.373+0000 7f8ec589e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec0106b80 0x7f8ec01a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.373+0000 7f8ec589e700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec0199cd0 con 0x7f8ec0106b80 2024-02-17T11:56:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.373+0000 7f8ec589e700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec01a44b0 con 0x7f8ec0101530 2024-02-17T11:56:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.373+0000 7f8ec589e700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec01a45f0 con 0x7f8ec0100b70 2024-02-17T11:56:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.373+0000 7f8ebf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec0106b80 0x7f8ec01a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.373+0000 7f8ebf7fe700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec0106b80 0x7f8ec01a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36426/0 (socket says 172.21.15.94:36426) 2024-02-17T11:56:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.373+0000 7f8ebf7fe700 1 -- 172.21.15.94:0/1312707712 learned_addr learned my addr 172.21.15.94:0/1312707712 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:56:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ebeffd700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ec0100b70 0x7f8ec0198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ebe7fc700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ec0101530 0x7f8ec0199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:59.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ebf7fe700 1 -- 172.21.15.94:0/1312707712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ec0100b70 msgr2=0x7f8ec0198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ebf7fe700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ec0100b70 0x7f8ec0198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ebf7fe700 1 -- 172.21.15.94:0/1312707712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ec0101530 msgr2=0x7f8ec0199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ebf7fe700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ec0101530 0x7f8ec0199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ebf7fe700 1 -- 172.21.15.94:0/1312707712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb00095f0 con 0x7f8ec0106b80 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ebe7fc700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ec0101530 0x7f8ec0199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ebf7fe700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec0106b80 0x7f8ec01a3f70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800ef60 tx=0x7f8eb800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ec489c700 1 -- 172.21.15.94:0/1312707712 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8018070 con 0x7f8ec0106b80 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ec489c700 1 -- 172.21.15.94:0/1312707712 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8eb800f040 con 0x7f8ec0106b80 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ec589e700 1 -- 172.21.15.94:0/1312707712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec01a4840 con 0x7f8ec0106b80 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ec489c700 1 -- 172.21.15.94:0/1312707712 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800ce10 con 0x7f8ec0106b80 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.374+0000 7f8ec589e700 1 -- 172.21.15.94:0/1312707712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec01a4d40 con 0x7f8ec0106b80 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.375+0000 7f8ec589e700 1 -- 172.21.15.94:0/1312707712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec0109020 con 0x7f8ec0106b80 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.377+0000 7f8ec489c700 1 -- 172.21.15.94:0/1312707712 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f8eb8004ad0 con 0x7f8ec0106b80 2024-02-17T11:56:59.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.377+0000 7f8ec489c700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ea80a2260 0x7f8ea80a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:56:59.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.377+0000 7f8ebeffd700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ea80a2260 0x7f8ea80a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:56:59.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.377+0000 7f8ec489c700 1 -- 172.21.15.94:0/1312707712 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8eb80cdec0 con 0x7f8ec0106b80 2024-02-17T11:56:59.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.378+0000 7f8ebeffd700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ea80a2260 0x7f8ea80a4720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00098e0 tx=0x7f8eb0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:56:59.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.381+0000 7f8ec489c700 1 -- 172.21.15.94:0/1312707712 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8eb8096dc0 con 0x7f8ec0106b80 2024-02-17T11:56:59.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.586+0000 7f8ec589e700 1 -- 172.21.15.94:0/1312707712 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8ec0068930 con 0x7f8ec0106b80 2024-02-17T11:56:59.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.587+0000 7f8ec489c700 1 -- 172.21.15.94:0/1312707712 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8eb809a480 con 0x7f8ec0106b80 2024-02-17T11:56:59.586 INFO:teuthology.orchestra.run.smithi094.stdout:55834574924 2024-02-17T11:56:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.589+0000 7f8ec589e700 1 -- 172.21.15.94:0/1312707712 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ea80a2260 msgr2=0x7f8ea80a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.589+0000 7f8ec589e700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ea80a2260 0x7f8ea80a4720 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00098e0 tx=0x7f8eb0005960 comp rx=0 tx=0).stop 2024-02-17T11:56:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.589+0000 7f8ec589e700 1 -- 172.21.15.94:0/1312707712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec0106b80 msgr2=0x7f8ec01a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:56:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.589+0000 7f8ec589e700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec0106b80 0x7f8ec01a3f70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800ef60 tx=0x7f8eb800ef90 comp rx=0 tx=0).stop 2024-02-17T11:56:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.589+0000 7f8ec589e700 1 -- 172.21.15.94:0/1312707712 shutdown_connections 2024-02-17T11:56:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.589+0000 7f8ec589e700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ea80a2260 0x7f8ea80a4720 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.589+0000 7f8ec589e700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ec0106b80 0x7f8ec01a3f70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.589+0000 7f8ec589e700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ec0100b70 0x7f8ec0198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.589+0000 7f8ec589e700 1 --2- 172.21.15.94:0/1312707712 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ec0101530 0x7f8ec0199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:56:59.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.589+0000 7f8ec589e700 1 -- 172.21.15.94:0/1312707712 >> 172.21.15.94:0/1312707712 conn(0x7f8ec0077aa0 msgr2=0x7f8ec00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:56:59.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.589+0000 7f8ec589e700 1 -- 172.21.15.94:0/1312707712 shutdown_connections 2024-02-17T11:56:59.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:56:59.589+0000 7f8ec589e700 1 -- 172.21.15.94:0/1312707712 wait complete. 2024-02-17T11:56:59.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-17T11:56:59.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-02-17T11:56:59.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-02-17T11:56:59.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-02-17T11:56:59.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-02-17T11:56:59.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-02-17T11:56:59.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-02-17T11:56:59.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-02-17T11:56:59.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-02-17T11:56:59.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-02-17T11:56:59.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: osdmap e86: 12 total, 12 up, 12 in 2024-02-17T11:56:59.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:56:59.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:56:59.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:56:59.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:56:59.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:56:59 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1312707712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-17T11:57:00.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-02-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-02-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-02-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-02-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-02-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-02-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-02-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-02-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-02-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: osdmap e86: 12 total, 12 up, 12 in 2024-02-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:00.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:00.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:00.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:56:59 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1312707712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-17T11:57:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-17T11:57:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-02-17T11:57:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-02-17T11:57:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-02-17T11:57:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-02-17T11:57:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-02-17T11:57:00.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-02-17T11:57:00.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-02-17T11:57:00.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-02-17T11:57:00.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-02-17T11:57:00.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: osdmap e86: 12 total, 12 up, 12 in 2024-02-17T11:57:00.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:00.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:00.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:00.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:00.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:56:59 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1312707712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-17T11:57:00.397 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645337 got 309237645342 for osd.10 2024-02-17T11:57:00.397 DEBUG:teuthology.parallel:result is None 2024-02-17T11:57:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.591+0000 7f5778315700 1 -- 172.21.15.94:0/1928151521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5770102860 msgr2=0x7f57701068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.591+0000 7f5778315700 1 --2- 172.21.15.94:0/1928151521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5770102860 0x7f57701068b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f576c009990 tx=0x7f576c009ca0 comp rx=0 tx=0).stop 2024-02-17T11:57:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.591+0000 7f5778315700 1 -- 172.21.15.94:0/1928151521 shutdown_connections 2024-02-17T11:57:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.591+0000 7f5778315700 1 --2- 172.21.15.94:0/1928151521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5770106f90 0x7f577010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.591+0000 7f5778315700 1 --2- 172.21.15.94:0/1928151521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5770101eb0 0x7f5770102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.591+0000 7f5778315700 1 --2- 172.21.15.94:0/1928151521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5770102860 0x7f57701068b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.591+0000 7f5778315700 1 -- 172.21.15.94:0/1928151521 >> 172.21.15.94:0/1928151521 conn(0x7f57700fd720 msgr2=0x7f57700ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.591+0000 7f5778315700 1 -- 172.21.15.94:0/1928151521 shutdown_connections 2024-02-17T11:57:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.591+0000 7f5778315700 1 -- 172.21.15.94:0/1928151521 wait complete. 2024-02-17T11:57:00.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f5778315700 1 Processor -- start 2024-02-17T11:57:00.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f5778315700 1 -- start start 2024-02-17T11:57:00.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f5778315700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5770101eb0 0x7f5770198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:00.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f5778315700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5770102860 0x7f5770198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:00.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f5778315700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5770106f90 0x7f577019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:00.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f5778315700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577019d0f0 con 0x7f5770101eb0 2024-02-17T11:57:00.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f5778315700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577019d260 con 0x7f5770102860 2024-02-17T11:57:00.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f5778315700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577019d420 con 0x7f5770106f90 2024-02-17T11:57:00.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f57760b1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5770101eb0 0x7f5770198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:00.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f57760b1700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5770101eb0 0x7f5770198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36442/0 (socket says 172.21.15.94:36442) 2024-02-17T11:57:00.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f57760b1700 1 -- 172.21.15.94:0/4206144719 learned_addr learned my addr 172.21.15.94:0/4206144719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:00.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f57768b2700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5770106f90 0x7f577019cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:00.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f57758b0700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5770102860 0x7f5770198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:00.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f57760b1700 1 -- 172.21.15.94:0/4206144719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5770106f90 msgr2=0x7f577019cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f57760b1700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5770106f90 0x7f577019cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f57760b1700 1 -- 172.21.15.94:0/4206144719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5770102860 msgr2=0x7f5770198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f57760b1700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5770102860 0x7f5770198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f57760b1700 1 -- 172.21.15.94:0/4206144719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c0095f0 con 0x7f5770101eb0 2024-02-17T11:57:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f57758b0700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5770102860 0x7f5770198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.592+0000 7f57760b1700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5770101eb0 0x7f5770198210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f576000b530 tx=0x7f576000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.593+0000 7f57677fe700 1 -- 172.21.15.94:0/4206144719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760011830 con 0x7f5770101eb0 2024-02-17T11:57:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.593+0000 7f5778315700 1 -- 172.21.15.94:0/4206144719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577019d700 con 0x7f5770101eb0 2024-02-17T11:57:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.593+0000 7f57677fe700 1 -- 172.21.15.94:0/4206144719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5760011e70 con 0x7f5770101eb0 2024-02-17T11:57:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.593+0000 7f5778315700 1 -- 172.21.15.94:0/4206144719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57701a4f60 con 0x7f5770101eb0 2024-02-17T11:57:00.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.593+0000 7f57677fe700 1 -- 172.21.15.94:0/4206144719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576000f2e0 con 0x7f5770101eb0 2024-02-17T11:57:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.593+0000 7f5778315700 1 -- 172.21.15.94:0/4206144719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577010a2b0 con 0x7f5770101eb0 2024-02-17T11:57:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.595+0000 7f57677fe700 1 -- 172.21.15.94:0/4206144719 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f57600105f0 con 0x7f5770101eb0 2024-02-17T11:57:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.596+0000 7f57677fe700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f575c0a21e0 0x7f575c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.596+0000 7f57758b0700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f575c0a21e0 0x7f575c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.596+0000 7f57677fe700 1 -- 172.21.15.94:0/4206144719 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f57600cdcc0 con 0x7f5770101eb0 2024-02-17T11:57:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.596+0000 7f57758b0700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f575c0a21e0 0x7f575c0a46a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f576c00b5c0 tx=0x7f576c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.599+0000 7f57677fe700 1 -- 172.21.15.94:0/4206144719 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5760096c40 con 0x7f5770101eb0 2024-02-17T11:57:00.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.804+0000 7f5778315700 1 -- 172.21.15.94:0/4206144719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f5770199570 con 0x7f5770101eb0 2024-02-17T11:57:00.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.804+0000 7f57677fe700 1 -- 172.21.15.94:0/4206144719 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5760010380 con 0x7f5770101eb0 2024-02-17T11:57:00.803 INFO:teuthology.orchestra.run.smithi094.stdout:253403070505 2024-02-17T11:57:00.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.806+0000 7f5778315700 1 -- 172.21.15.94:0/4206144719 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f575c0a21e0 msgr2=0x7f575c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:00.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.806+0000 7f5778315700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f575c0a21e0 0x7f575c0a46a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f576c00b5c0 tx=0x7f576c005fb0 comp rx=0 tx=0).stop 2024-02-17T11:57:00.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.806+0000 7f5778315700 1 -- 172.21.15.94:0/4206144719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5770101eb0 msgr2=0x7f5770198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:00.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.806+0000 7f5778315700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5770101eb0 0x7f5770198210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f576000b530 tx=0x7f576000b8f0 comp rx=0 tx=0).stop 2024-02-17T11:57:00.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.806+0000 7f5778315700 1 -- 172.21.15.94:0/4206144719 shutdown_connections 2024-02-17T11:57:00.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.806+0000 7f5778315700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f575c0a21e0 0x7f575c0a46a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:00.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.806+0000 7f5778315700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5770106f90 0x7f577019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:00.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.806+0000 7f5778315700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5770101eb0 0x7f5770198210 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:00.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.806+0000 7f5778315700 1 --2- 172.21.15.94:0/4206144719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5770102860 0x7f5770198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:00.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.806+0000 7f5778315700 1 -- 172.21.15.94:0/4206144719 >> 172.21.15.94:0/4206144719 conn(0x7f57700fd720 msgr2=0x7f57700ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:00.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.806+0000 7f5778315700 1 -- 172.21.15.94:0/4206144719 shutdown_connections 2024-02-17T11:57:00.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:00.806+0000 7f5778315700 1 -- 172.21.15.94:0/4206144719 wait complete. 2024-02-17T11:57:01.110 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:00 smithi094 ceph-mon[32499]: osdmap e87: 12 total, 12 up, 12 in 2024-02-17T11:57:01.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:00 smithi189 ceph-mon[36588]: osdmap e87: 12 total, 12 up, 12 in 2024-02-17T11:57:01.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:00 smithi125 ceph-mon[36819]: osdmap e87: 12 total, 12 up, 12 in 2024-02-17T11:57:01.380 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070501 got 253403070505 for osd.8 2024-02-17T11:57:01.380 DEBUG:teuthology.parallel:result is None 2024-02-17T11:57:01.992 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:01 smithi094 ceph-mon[32499]: pgmap v318: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-02-17T11:57:01.992 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:01 smithi094 ceph-mon[32499]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-17T11:57:01.992 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:01 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/4206144719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-17T11:57:01.998 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:57:02.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.004+0000 7fe59ce2b700 1 -- 172.21.15.94:0/2863988757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe59806ad80 msgr2=0x7fe59810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:02.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.004+0000 7fe59ce2b700 1 --2- 172.21.15.94:0/2863988757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe59806ad80 0x7fe59810eac0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe59000b0d0 tx=0x7fe59000b3e0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.004+0000 7fe59ce2b700 1 -- 172.21.15.94:0/2863988757 shutdown_connections 2024-02-17T11:57:02.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.004+0000 7fe59ce2b700 1 --2- 172.21.15.94:0/2863988757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe59806ad80 0x7fe59810eac0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.004+0000 7fe59ce2b700 1 --2- 172.21.15.94:0/2863988757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe598107ca0 0x7fe598108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.004+0000 7fe59ce2b700 1 --2- 172.21.15.94:0/2863988757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe59806a3c0 0x7fe59806a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.004+0000 7fe59ce2b700 1 -- 172.21.15.94:0/2863988757 >> 172.21.15.94:0/2863988757 conn(0x7fe598077b80 msgr2=0x7fe598077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:02.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.004+0000 7fe59ce2b700 1 -- 172.21.15.94:0/2863988757 shutdown_connections 2024-02-17T11:57:02.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.004+0000 7fe59ce2b700 1 -- 172.21.15.94:0/2863988757 wait complete. 2024-02-17T11:57:02.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe59ce2b700 1 Processor -- start 2024-02-17T11:57:02.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe59ce2b700 1 -- start start 2024-02-17T11:57:02.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe59ce2b700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe59806a3c0 0x7fe598198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:02.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe59ce2b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe59806ad80 0x7fe598198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:02.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe59ce2b700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe598107ca0 0x7fe59819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:02.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe59ce2b700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59819d230 con 0x7fe59806ad80 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe59ce2b700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59819d3a0 con 0x7fe598107ca0 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe59ce2b700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59819d560 con 0x7fe59806a3c0 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe595d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe59806ad80 0x7fe598198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe595d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe59806ad80 0x7fe598198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36460/0 (socket says 172.21.15.94:36460) 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe595d9b700 1 -- 172.21.15.94:0/920570102 learned_addr learned my addr 172.21.15.94:0/920570102 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe59659c700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe59806a3c0 0x7fe598198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe595d9b700 1 -- 172.21.15.94:0/920570102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe59806a3c0 msgr2=0x7fe598198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.005+0000 7fe595d9b700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe59806a3c0 0x7fe598198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.006+0000 7fe596d9d700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe598107ca0 0x7fe59819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.006+0000 7fe595d9b700 1 -- 172.21.15.94:0/920570102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe598107ca0 msgr2=0x7fe59819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.006+0000 7fe595d9b700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe598107ca0 0x7fe59819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.006+0000 7fe595d9b700 1 -- 172.21.15.94:0/920570102 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe590009d00 con 0x7fe59806ad80 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.006+0000 7fe59659c700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe59806a3c0 0x7fe598198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:02.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.006+0000 7fe595d9b700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe59806ad80 0x7fe598198890 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00b530 tx=0x7fe58c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:02.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.006+0000 7fe5877fe700 1 -- 172.21.15.94:0/920570102 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c011830 con 0x7fe59806ad80 2024-02-17T11:57:02.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.006+0000 7fe5877fe700 1 -- 172.21.15.94:0/920570102 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe58c011e70 con 0x7fe59806ad80 2024-02-17T11:57:02.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.006+0000 7fe5877fe700 1 -- 172.21.15.94:0/920570102 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c00f2e0 con 0x7fe59806ad80 2024-02-17T11:57:02.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.006+0000 7fe59ce2b700 1 -- 172.21.15.94:0/920570102 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59819d840 con 0x7fe59806ad80 2024-02-17T11:57:02.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.006+0000 7fe59ce2b700 1 -- 172.21.15.94:0/920570102 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5981a50a0 con 0x7fe59806ad80 2024-02-17T11:57:02.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.007+0000 7fe59ce2b700 1 -- 172.21.15.94:0/920570102 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59810a180 con 0x7fe59806ad80 2024-02-17T11:57:02.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.008+0000 7fe5877fe700 1 -- 172.21.15.94:0/920570102 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe58c0105f0 con 0x7fe59806ad80 2024-02-17T11:57:02.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.009+0000 7fe5877fe700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe5800a2260 0x7fe5800a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:02.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.009+0000 7fe59659c700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe5800a2260 0x7fe5800a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:02.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.009+0000 7fe5877fe700 1 -- 172.21.15.94:0/920570102 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe58c0cd830 con 0x7fe59806ad80 2024-02-17T11:57:02.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.010+0000 7fe59659c700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe5800a2260 0x7fe5800a4720 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe5880099d0 tx=0x7fe588008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:02.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.014+0000 7fe5877fe700 1 -- 172.21.15.94:0/920570102 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe58c096770 con 0x7fe59806ad80 2024-02-17T11:57:02.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:01 smithi189 ceph-mon[36588]: pgmap v318: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-02-17T11:57:02.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:01 smithi189 ceph-mon[36588]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-17T11:57:02.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:01 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/4206144719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-17T11:57:02.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:01 smithi125 ceph-mon[36819]: pgmap v318: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s; 14 B/s, 0 objects/s recovering 2024-02-17T11:57:02.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:01 smithi125 ceph-mon[36819]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-17T11:57:02.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:01 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/4206144719' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-17T11:57:02.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.217+0000 7fe59ce2b700 1 -- 172.21.15.94:0/920570102 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe598068930 con 0x7fe59806ad80 2024-02-17T11:57:02.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.218+0000 7fe5877fe700 1 -- 172.21.15.94:0/920570102 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe58c010380 con 0x7fe59806ad80 2024-02-17T11:57:02.217 INFO:teuthology.orchestra.run.smithi094.stdout:197568495667 2024-02-17T11:57:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.220+0000 7fe59ce2b700 1 -- 172.21.15.94:0/920570102 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe5800a2260 msgr2=0x7fe5800a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.220+0000 7fe59ce2b700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe5800a2260 0x7fe5800a4720 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe5880099d0 tx=0x7fe588008040 comp rx=0 tx=0).stop 2024-02-17T11:57:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.220+0000 7fe59ce2b700 1 -- 172.21.15.94:0/920570102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe59806ad80 msgr2=0x7fe598198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.220+0000 7fe59ce2b700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe59806ad80 0x7fe598198890 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe58c00b530 tx=0x7fe58c00b8f0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.220+0000 7fe59ce2b700 1 -- 172.21.15.94:0/920570102 shutdown_connections 2024-02-17T11:57:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.220+0000 7fe59ce2b700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe5800a2260 0x7fe5800a4720 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.220+0000 7fe59ce2b700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe598107ca0 0x7fe59819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.220+0000 7fe59ce2b700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe59806a3c0 0x7fe598198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.220+0000 7fe59ce2b700 1 --2- 172.21.15.94:0/920570102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe59806ad80 0x7fe598198890 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.220+0000 7fe59ce2b700 1 -- 172.21.15.94:0/920570102 >> 172.21.15.94:0/920570102 conn(0x7fe598077b80 msgr2=0x7fe5980ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.220+0000 7fe59ce2b700 1 -- 172.21.15.94:0/920570102 shutdown_connections 2024-02-17T11:57:02.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.220+0000 7fe59ce2b700 1 -- 172.21.15.94:0/920570102 wait complete. 2024-02-17T11:57:02.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.990+0000 7f559d077700 1 -- 172.21.15.94:0/483973432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5598107d40 msgr2=0x7f5598108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:02.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.990+0000 7f559d077700 1 --2- 172.21.15.94:0/483973432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5598107d40 0x7f5598108120 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5588009990 tx=0x7f5588009ca0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.991+0000 7f559d077700 1 -- 172.21.15.94:0/483973432 shutdown_connections 2024-02-17T11:57:02.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.991+0000 7f559d077700 1 --2- 172.21.15.94:0/483973432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f559806ad50 0x7f559810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.991+0000 7f559d077700 1 --2- 172.21.15.94:0/483973432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5598107d40 0x7f5598108120 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.991+0000 7f559d077700 1 --2- 172.21.15.94:0/483973432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f559806a3b0 0x7f559806a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.991+0000 7f559d077700 1 -- 172.21.15.94:0/483973432 >> 172.21.15.94:0/483973432 conn(0x7f55980779d0 msgr2=0x7f5598077de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.991+0000 7f559d077700 1 -- 172.21.15.94:0/483973432 shutdown_connections 2024-02-17T11:57:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.991+0000 7f559d077700 1 -- 172.21.15.94:0/483973432 wait complete. 2024-02-17T11:57:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.991+0000 7f559d077700 1 Processor -- start 2024-02-17T11:57:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f559d077700 1 -- start start 2024-02-17T11:57:02.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f559d077700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f559806a3b0 0x7f55981982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:02.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f559d077700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f559806ad50 0x7f55981987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:02.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f559d077700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5598107d40 0x7f559819cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:02.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f559d077700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559819d190 con 0x7f559806a3b0 2024-02-17T11:57:02.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f559d077700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559819d300 con 0x7f5598107d40 2024-02-17T11:57:02.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f559d077700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559819d4c0 con 0x7f559806ad50 2024-02-17T11:57:02.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f5596d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f559806a3b0 0x7f55981982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:02.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f5596d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f559806a3b0 0x7f55981982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36472/0 (socket says 172.21.15.94:36472) 2024-02-17T11:57:02.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f5596d9d700 1 -- 172.21.15.94:0/1093037752 learned_addr learned my addr 172.21.15.94:0/1093037752 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:02.995 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:02 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/920570102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-17T11:57:02.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f558e59c700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f559806ad50 0x7f55981987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:02.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f559759e700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5598107d40 0x7f559819cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:02.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f5596d9d700 1 -- 172.21.15.94:0/1093037752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f559806ad50 msgr2=0x7f55981987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:02.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f5596d9d700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f559806ad50 0x7f55981987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f5596d9d700 1 -- 172.21.15.94:0/1093037752 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5598107d40 msgr2=0x7f559819cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f5596d9d700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5598107d40 0x7f559819cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f5596d9d700 1 -- 172.21.15.94:0/1093037752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55880095f0 con 0x7f559806a3b0 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f559759e700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5598107d40 0x7f559819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.992+0000 7f5596d9d700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f559806a3b0 0x7f55981982b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f55880098e0 tx=0x7f558800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.993+0000 7f5594d99700 1 -- 172.21.15.94:0/1093037752 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558800fc60 con 0x7f559806a3b0 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.993+0000 7f5594d99700 1 -- 172.21.15.94:0/1093037752 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5588023070 con 0x7f559806a3b0 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.993+0000 7f5594d99700 1 -- 172.21.15.94:0/1093037752 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5588017770 con 0x7f559806a3b0 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.993+0000 7f559d077700 1 -- 172.21.15.94:0/1093037752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559819d740 con 0x7f559806a3b0 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.993+0000 7f559d077700 1 -- 172.21.15.94:0/1093037752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5598100e50 con 0x7f559806a3b0 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.994+0000 7f559d077700 1 -- 172.21.15.94:0/1093037752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559810a170 con 0x7f559806a3b0 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.995+0000 7f5594d99700 1 -- 172.21.15.94:0/1093037752 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f5588025030 con 0x7f559806a3b0 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.995+0000 7f5594d99700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f55840a21e0 0x7f55840a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:02.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.995+0000 7f558e59c700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f55840a21e0 0x7f55840a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:02.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.996+0000 7f5594d99700 1 -- 172.21.15.94:0/1093037752 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f55880d31c0 con 0x7f559806a3b0 2024-02-17T11:57:02.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:02.996+0000 7f558e59c700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f55840a21e0 0x7f55840a46a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5580005fd0 tx=0x7f558000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:02.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.000+0000 7f5594d99700 1 -- 172.21.15.94:0/1093037752 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f558809c1d0 con 0x7f559806a3b0 2024-02-17T11:57:03.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:02 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/920570102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-17T11:57:03.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:02 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/920570102' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-17T11:57:03.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.202+0000 7f559d077700 1 -- 172.21.15.94:0/1093037752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f55981033f0 con 0x7f559806a3b0 2024-02-17T11:57:03.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.203+0000 7f5594d99700 1 -- 172.21.15.94:0/1093037752 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f558809f890 con 0x7f559806a3b0 2024-02-17T11:57:03.202 INFO:teuthology.orchestra.run.smithi094.stdout:163208757305 2024-02-17T11:57:03.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.205+0000 7f559d077700 1 -- 172.21.15.94:0/1093037752 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f55840a21e0 msgr2=0x7f55840a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:03.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.205+0000 7f559d077700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f55840a21e0 0x7f55840a46a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5580005fd0 tx=0x7f558000a560 comp rx=0 tx=0).stop 2024-02-17T11:57:03.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.205+0000 7f559d077700 1 -- 172.21.15.94:0/1093037752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f559806a3b0 msgr2=0x7f55981982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:03.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.205+0000 7f559d077700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f559806a3b0 0x7f55981982b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f55880098e0 tx=0x7f558800f740 comp rx=0 tx=0).stop 2024-02-17T11:57:03.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.205+0000 7f559d077700 1 -- 172.21.15.94:0/1093037752 shutdown_connections 2024-02-17T11:57:03.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.205+0000 7f559d077700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f55840a21e0 0x7f55840a46a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:03.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.205+0000 7f559d077700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5598107d40 0x7f559819cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:03.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.205+0000 7f559d077700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f559806a3b0 0x7f55981982b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:03.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.205+0000 7f559d077700 1 --2- 172.21.15.94:0/1093037752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f559806ad50 0x7f55981987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:03.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.205+0000 7f559d077700 1 -- 172.21.15.94:0/1093037752 >> 172.21.15.94:0/1093037752 conn(0x7f55980779d0 msgr2=0x7f55980ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:03.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.205+0000 7f559d077700 1 -- 172.21.15.94:0/1093037752 shutdown_connections 2024-02-17T11:57:03.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:03.205+0000 7f559d077700 1 -- 172.21.15.94:0/1093037752 wait complete. 2024-02-17T11:57:04.008 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:03 smithi094 ceph-mon[32499]: pgmap v319: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 87 B/s, 0 objects/s recovering 2024-02-17T11:57:04.008 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:03 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1093037752' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-17T11:57:04.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:03 smithi189 ceph-mon[36588]: pgmap v319: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 87 B/s, 0 objects/s recovering 2024-02-17T11:57:04.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:03 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1093037752' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-17T11:57:04.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:03 smithi125 ceph-mon[36819]: pgmap v319: 97 pgs: 3 peering, 94 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 87 B/s, 0 objects/s recovering 2024-02-17T11:57:04.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:03 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1093037752' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-17T11:57:04.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.749+0000 7fdf28bb7700 1 -- 172.21.15.94:0/876948158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf24101ed0 msgr2=0x7fdf241022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:04.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.749+0000 7fdf28bb7700 1 --2- 172.21.15.94:0/876948158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf24101ed0 0x7fdf241022b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009990 tx=0x7fdf14009ca0 comp rx=0 tx=0).stop 2024-02-17T11:57:04.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.749+0000 7fdf28bb7700 1 -- 172.21.15.94:0/876948158 shutdown_connections 2024-02-17T11:57:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.749+0000 7fdf28bb7700 1 --2- 172.21.15.94:0/876948158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf24106fb0 0x7fdf2410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.749+0000 7fdf28bb7700 1 --2- 172.21.15.94:0/876948158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf24101ed0 0x7fdf241022b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.749+0000 7fdf28bb7700 1 --2- 172.21.15.94:0/876948158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf24102880 0x7fdf241068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.749+0000 7fdf28bb7700 1 -- 172.21.15.94:0/876948158 >> 172.21.15.94:0/876948158 conn(0x7fdf240fd740 msgr2=0x7fdf240ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.749+0000 7fdf28bb7700 1 -- 172.21.15.94:0/876948158 shutdown_connections 2024-02-17T11:57:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.749+0000 7fdf28bb7700 1 -- 172.21.15.94:0/876948158 wait complete. 2024-02-17T11:57:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.749+0000 7fdf28bb7700 1 Processor -- start 2024-02-17T11:57:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.749+0000 7fdf28bb7700 1 -- start start 2024-02-17T11:57:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf28bb7700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf24101ed0 0x7fdf241981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf28bb7700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf24102880 0x7fdf241986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf28bb7700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf24106fb0 0x7fdf2419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf28bb7700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2419d080 con 0x7fdf24102880 2024-02-17T11:57:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf28bb7700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2419d1f0 con 0x7fdf24106fb0 2024-02-17T11:57:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf28bb7700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2419d3b0 con 0x7fdf24101ed0 2024-02-17T11:57:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf21d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf24102880 0x7fdf241986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf21d9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf24102880 0x7fdf241986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44772/0 (socket says 172.21.15.94:44772) 2024-02-17T11:57:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf21d9b700 1 -- 172.21.15.94:0/2067426028 learned_addr learned my addr 172.21.15.94:0/2067426028 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf2259c700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf24101ed0 0x7fdf241981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf22d9d700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf24106fb0 0x7fdf2419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf21d9b700 1 -- 172.21.15.94:0/2067426028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf24101ed0 msgr2=0x7fdf241981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf21d9b700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf24101ed0 0x7fdf241981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf21d9b700 1 -- 172.21.15.94:0/2067426028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf24106fb0 msgr2=0x7fdf2419ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf21d9b700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf24106fb0 0x7fdf2419ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf21d9b700 1 -- 172.21.15.94:0/2067426028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf140095f0 con 0x7fdf24102880 2024-02-17T11:57:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf22d9d700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf24106fb0 0x7fdf2419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf21d9b700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf24102880 0x7fdf241986e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800c9e0 tx=0x7fdf1800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:04.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf137fe700 1 -- 172.21.15.94:0/2067426028 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18007aa0 con 0x7fdf24102880 2024-02-17T11:57:04.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf137fe700 1 -- 172.21.15.94:0/2067426028 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf18007c00 con 0x7fdf24102880 2024-02-17T11:57:04.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.750+0000 7fdf28bb7700 1 -- 172.21.15.94:0/2067426028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2419d690 con 0x7fdf24102880 2024-02-17T11:57:04.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.751+0000 7fdf137fe700 1 -- 172.21.15.94:0/2067426028 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18018660 con 0x7fdf24102880 2024-02-17T11:57:04.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.751+0000 7fdf28bb7700 1 -- 172.21.15.94:0/2067426028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf241a4ef0 con 0x7fdf24102880 2024-02-17T11:57:04.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.751+0000 7fdf28bb7700 1 -- 172.21.15.94:0/2067426028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2410a1e0 con 0x7fdf24102880 2024-02-17T11:57:04.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.752+0000 7fdf137fe700 1 -- 172.21.15.94:0/2067426028 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fdf1800f450 con 0x7fdf24102880 2024-02-17T11:57:04.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.753+0000 7fdf137fe700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdf0c0a21e0 0x7fdf0c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:04.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.753+0000 7fdf2259c700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdf0c0a21e0 0x7fdf0c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:04.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.753+0000 7fdf137fe700 1 -- 172.21.15.94:0/2067426028 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdf180cddc0 con 0x7fdf24102880 2024-02-17T11:57:04.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.754+0000 7fdf2259c700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdf0c0a21e0 0x7fdf0c0a46a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf140098e0 tx=0x7fdf14005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:04.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.758+0000 7fdf137fe700 1 -- 172.21.15.94:0/2067426028 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf18096c30 con 0x7fdf24102880 2024-02-17T11:57:04.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.961+0000 7fdf28bb7700 1 -- 172.21.15.94:0/2067426028 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fdf24111d10 con 0x7fdf24102880 2024-02-17T11:57:04.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.961+0000 7fdf137fe700 1 -- 172.21.15.94:0/2067426028 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf1809a2f0 con 0x7fdf24102880 2024-02-17T11:57:04.960 INFO:teuthology.orchestra.run.smithi094.stdout:219043332142 2024-02-17T11:57:04.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.963+0000 7fdf28bb7700 1 -- 172.21.15.94:0/2067426028 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdf0c0a21e0 msgr2=0x7fdf0c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:04.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.963+0000 7fdf28bb7700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdf0c0a21e0 0x7fdf0c0a46a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf140098e0 tx=0x7fdf14005960 comp rx=0 tx=0).stop 2024-02-17T11:57:04.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.963+0000 7fdf28bb7700 1 -- 172.21.15.94:0/2067426028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf24102880 msgr2=0x7fdf241986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.963+0000 7fdf28bb7700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf24102880 0x7fdf241986e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800c9e0 tx=0x7fdf1800ccf0 comp rx=0 tx=0).stop 2024-02-17T11:57:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.963+0000 7fdf28bb7700 1 -- 172.21.15.94:0/2067426028 shutdown_connections 2024-02-17T11:57:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.963+0000 7fdf28bb7700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdf0c0a21e0 0x7fdf0c0a46a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.963+0000 7fdf28bb7700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf24106fb0 0x7fdf2419ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.963+0000 7fdf28bb7700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf24101ed0 0x7fdf241981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.963+0000 7fdf28bb7700 1 --2- 172.21.15.94:0/2067426028 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf24102880 0x7fdf241986e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.963+0000 7fdf28bb7700 1 -- 172.21.15.94:0/2067426028 >> 172.21.15.94:0/2067426028 conn(0x7fdf240fd740 msgr2=0x7fdf240ff540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.964+0000 7fdf28bb7700 1 -- 172.21.15.94:0/2067426028 shutdown_connections 2024-02-17T11:57:04.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.964+0000 7fdf28bb7700 1 -- 172.21.15.94:0/2067426028 wait complete. 2024-02-17T11:57:05.327 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495663 got 197568495667 for osd.6 2024-02-17T11:57:05.327 DEBUG:teuthology.parallel:result is None 2024-02-17T11:57:06.125 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:05 smithi094 ceph-mon[32499]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 87 B/s, 0 objects/s recovering 2024-02-17T11:57:06.125 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:05 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2067426028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-17T11:57:06.125 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:05 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2981321132' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-17T11:57:06.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:05 smithi189 ceph-mon[36588]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 87 B/s, 0 objects/s recovering 2024-02-17T11:57:06.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:05 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2067426028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-17T11:57:06.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:05 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2981321132' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-17T11:57:06.157 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757297 got 163208757305 for osd.5 2024-02-17T11:57:06.157 DEBUG:teuthology.parallel:result is None 2024-02-17T11:57:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:05 smithi125 ceph-mon[36819]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 87 B/s, 0 objects/s recovering 2024-02-17T11:57:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:05 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2067426028' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-17T11:57:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:05 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2981321132' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-17T11:57:06.521 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332137 got 219043332142 for osd.7 2024-02-17T11:57:06.521 DEBUG:teuthology.parallel:result is None 2024-02-17T11:57:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.998+0000 7fef32eae700 1 -- 172.21.15.94:0/242911709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef2c106ce0 msgr2=0x7fef2c1070c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.998+0000 7fef32eae700 1 --2- 172.21.15.94:0/242911709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef2c106ce0 0x7fef2c1070c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fef20009990 tx=0x7fef20009ca0 comp rx=0 tx=0).stop 2024-02-17T11:57:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.998+0000 7fef32eae700 1 -- 172.21.15.94:0/242911709 shutdown_connections 2024-02-17T11:57:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.998+0000 7fef32eae700 1 --2- 172.21.15.94:0/242911709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef2c100c20 0x7fef2c10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.998+0000 7fef32eae700 1 --2- 172.21.15.94:0/242911709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef2c106ce0 0x7fef2c1070c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.998+0000 7fef32eae700 1 --2- 172.21.15.94:0/242911709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef2c100260 0x7fef2c1006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.998+0000 7fef32eae700 1 -- 172.21.15.94:0/242911709 >> 172.21.15.94:0/242911709 conn(0x7fef2c077170 msgr2=0x7fef2c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:06.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.998+0000 7fef32eae700 1 -- 172.21.15.94:0/242911709 shutdown_connections 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.998+0000 7fef32eae700 1 -- 172.21.15.94:0/242911709 wait complete. 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.999+0000 7fef32eae700 1 Processor -- start 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.999+0000 7fef32eae700 1 -- start start 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.999+0000 7fef32eae700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef2c100260 0x7fef2c101440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.999+0000 7fef32eae700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef2c100c20 0x7fef2c101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.999+0000 7fef32eae700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef2c106ce0 0x7fef2c101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.999+0000 7fef32eae700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c1a6780 con 0x7fef2c100c20 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.999+0000 7fef32eae700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c1a68f0 con 0x7fef2c106ce0 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.999+0000 7fef32eae700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c1a6ab0 con 0x7fef2c100260 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.999+0000 7fef2bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef2c100c20 0x7fef2c101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.999+0000 7fef2bfff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef2c100c20 0x7fef2c101980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44790/0 (socket says 172.21.15.94:44790) 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.999+0000 7fef2bfff700 1 -- 172.21.15.94:0/2981321132 learned_addr learned my addr 172.21.15.94:0/2981321132 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.999+0000 7fef30c4a700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef2c100260 0x7fef2c101440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:06.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:04.999+0000 7fef3144b700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef2c106ce0 0x7fef2c101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:06.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.000+0000 7fef2bfff700 1 -- 172.21.15.94:0/2981321132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef2c100260 msgr2=0x7fef2c101440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:06.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.000+0000 7fef2bfff700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef2c100260 0x7fef2c101440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:06.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.000+0000 7fef2bfff700 1 -- 172.21.15.94:0/2981321132 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef2c106ce0 msgr2=0x7fef2c101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:06.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.000+0000 7fef2bfff700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef2c106ce0 0x7fef2c101ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:06.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.000+0000 7fef2bfff700 1 -- 172.21.15.94:0/2981321132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef200095f0 con 0x7fef2c100c20 2024-02-17T11:57:06.623 INFO:teuthology.orchestra.run.smithi094.stdout:77309411399 2024-02-17T11:57:06.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.000+0000 7fef2bfff700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef2c100c20 0x7fef2c101980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fef1800b530 tx=0x7fef1800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.000+0000 7fef29ffb700 1 -- 172.21.15.94:0/2981321132 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef18011830 con 0x7fef2c100c20 2024-02-17T11:57:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.000+0000 7fef29ffb700 1 -- 172.21.15.94:0/2981321132 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef18011e70 con 0x7fef2c100c20 2024-02-17T11:57:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.000+0000 7fef32eae700 1 -- 172.21.15.94:0/2981321132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef2c1a6d90 con 0x7fef2c100c20 2024-02-17T11:57:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.000+0000 7fef29ffb700 1 -- 172.21.15.94:0/2981321132 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1800f2e0 con 0x7fef2c100c20 2024-02-17T11:57:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.000+0000 7fef32eae700 1 -- 172.21.15.94:0/2981321132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef2c1a7230 con 0x7fef2c100c20 2024-02-17T11:57:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.002+0000 7fef32eae700 1 -- 172.21.15.94:0/2981321132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2c10a2b0 con 0x7fef2c100c20 2024-02-17T11:57:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.002+0000 7fef29ffb700 1 -- 172.21.15.94:0/2981321132 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fef18010610 con 0x7fef2c100c20 2024-02-17T11:57:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.003+0000 7fef29ffb700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fef1c0a21e0 0x7fef1c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.003+0000 7fef30c4a700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fef1c0a21e0 0x7fef1c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.003+0000 7fef29ffb700 1 -- 172.21.15.94:0/2981321132 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fef180cd6c0 con 0x7fef2c100c20 2024-02-17T11:57:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.003+0000 7fef30c4a700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fef1c0a21e0 0x7fef1c0a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fef2000b5c0 tx=0x7fef20005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:06.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.007+0000 7fef29ffb700 1 -- 172.21.15.94:0/2981321132 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef18096640 con 0x7fef2c100c20 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.203+0000 7fef32eae700 1 -- 172.21.15.94:0/2981321132 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fef2c103560 con 0x7fef2c100c20 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.204+0000 7fef29ffb700 1 -- 172.21.15.94:0/2981321132 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fef18099d00 con 0x7fef2c100c20 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.206+0000 7fef32eae700 1 -- 172.21.15.94:0/2981321132 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fef1c0a21e0 msgr2=0x7fef1c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.206+0000 7fef32eae700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fef1c0a21e0 0x7fef1c0a46a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fef2000b5c0 tx=0x7fef20005960 comp rx=0 tx=0).stop 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.206+0000 7fef32eae700 1 -- 172.21.15.94:0/2981321132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef2c100c20 msgr2=0x7fef2c101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.206+0000 7fef32eae700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef2c100c20 0x7fef2c101980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fef1800b530 tx=0x7fef1800b840 comp rx=0 tx=0).stop 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.206+0000 7fef32eae700 1 -- 172.21.15.94:0/2981321132 shutdown_connections 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.206+0000 7fef32eae700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fef1c0a21e0 0x7fef1c0a46a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.206+0000 7fef32eae700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef2c106ce0 0x7fef2c101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.206+0000 7fef32eae700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef2c100260 0x7fef2c101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.206+0000 7fef32eae700 1 --2- 172.21.15.94:0/2981321132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef2c100c20 0x7fef2c101980 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.206+0000 7fef32eae700 1 -- 172.21.15.94:0/2981321132 >> 172.21.15.94:0/2981321132 conn(0x7fef2c077170 msgr2=0x7fef2c104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.206+0000 7fef32eae700 1 -- 172.21.15.94:0/2981321132 shutdown_connections 2024-02-17T11:57:06.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:05.206+0000 7fef32eae700 1 -- 172.21.15.94:0/2981321132 wait complete. 2024-02-17T11:57:07.088 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:06 smithi094 ceph-mon[32499]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 87 B/s, 0 objects/s recovering 2024-02-17T11:57:07.088 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:06 smithi094 ceph-mon[32499]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-17T11:57:07.088 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:06 smithi094 ceph-mon[32499]: Cluster is now healthy 2024-02-17T11:57:07.156 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:06 smithi189 ceph-mon[36588]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 87 B/s, 0 objects/s recovering 2024-02-17T11:57:07.156 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:06 smithi189 ceph-mon[36588]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-17T11:57:07.156 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:06 smithi189 ceph-mon[36588]: Cluster is now healthy 2024-02-17T11:57:07.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:06 smithi125 ceph-mon[36819]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 87 B/s, 0 objects/s recovering 2024-02-17T11:57:07.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:06 smithi125 ceph-mon[36819]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-17T11:57:07.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:06 smithi125 ceph-mon[36819]: Cluster is now healthy 2024-02-17T11:57:07.430 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574919 got 55834574924 for osd.1 2024-02-17T11:57:07.430 DEBUG:teuthology.parallel:result is None 2024-02-17T11:57:07.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.631+0000 7fdc771f2700 1 -- 172.21.15.94:0/3715056015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc70106bb0 msgr2=0x7fdc70106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:07.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.631+0000 7fdc771f2700 1 --2- 172.21.15.94:0/3715056015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc70106bb0 0x7fdc70106f90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009990 tx=0x7fdc64009ca0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.631+0000 7fdc771f2700 1 -- 172.21.15.94:0/3715056015 shutdown_connections 2024-02-17T11:57:07.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.631+0000 7fdc771f2700 1 --2- 172.21.15.94:0/3715056015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc70101560 0x7fdc7010d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.631+0000 7fdc771f2700 1 --2- 172.21.15.94:0/3715056015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc70106bb0 0x7fdc70106f90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.631+0000 7fdc771f2700 1 --2- 172.21.15.94:0/3715056015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc70100ba0 0x7fdc70101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.631+0000 7fdc771f2700 1 -- 172.21.15.94:0/3715056015 >> 172.21.15.94:0/3715056015 conn(0x7fdc70076b80 msgr2=0x7fdc70076f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:07.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.631+0000 7fdc771f2700 1 -- 172.21.15.94:0/3715056015 shutdown_connections 2024-02-17T11:57:07.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.631+0000 7fdc771f2700 1 -- 172.21.15.94:0/3715056015 wait complete. 2024-02-17T11:57:07.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc771f2700 1 Processor -- start 2024-02-17T11:57:07.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc771f2700 1 -- start start 2024-02-17T11:57:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc771f2700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc70100ba0 0x7fdc701027c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc771f2700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc70101560 0x7fdc70102d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc771f2700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc70106bb0 0x7fdc70103240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc771f2700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc701a6670 con 0x7fdc70100ba0 2024-02-17T11:57:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc771f2700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc701a67e0 con 0x7fdc70101560 2024-02-17T11:57:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc771f2700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc701a69a0 con 0x7fdc70106bb0 2024-02-17T11:57:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc74f8e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc70100ba0 0x7fdc701027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc74f8e700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc70100ba0 0x7fdc701027c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44820/0 (socket says 172.21.15.94:44820) 2024-02-17T11:57:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc74f8e700 1 -- 172.21.15.94:0/3978409655 learned_addr learned my addr 172.21.15.94:0/3978409655 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc6ffff700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc70101560 0x7fdc70102d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc7578f700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc70106bb0 0x7fdc70103240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc74f8e700 1 -- 172.21.15.94:0/3978409655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc70106bb0 msgr2=0x7fdc70103240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:07.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc74f8e700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc70106bb0 0x7fdc70103240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc74f8e700 1 -- 172.21.15.94:0/3978409655 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc70101560 msgr2=0x7fdc70102d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:07.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc74f8e700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc70101560 0x7fdc70102d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc74f8e700 1 -- 172.21.15.94:0/3978409655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc640095f0 con 0x7fdc70100ba0 2024-02-17T11:57:07.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc7578f700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc70106bb0 0x7fdc70103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:07.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc6ffff700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc70101560 0x7fdc70102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:07.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.632+0000 7fdc74f8e700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc70100ba0 0x7fdc701027c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdc640098e0 tx=0x7fdc6400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:07.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.633+0000 7fdc6dffb700 1 -- 172.21.15.94:0/3978409655 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6401d070 con 0x7fdc70100ba0 2024-02-17T11:57:07.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.633+0000 7fdc6dffb700 1 -- 172.21.15.94:0/3978409655 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc6400fc70 con 0x7fdc70100ba0 2024-02-17T11:57:07.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.633+0000 7fdc771f2700 1 -- 172.21.15.94:0/3978409655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc701a6c20 con 0x7fdc70100ba0 2024-02-17T11:57:07.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.633+0000 7fdc6dffb700 1 -- 172.21.15.94:0/3978409655 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc640176e0 con 0x7fdc70100ba0 2024-02-17T11:57:07.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.633+0000 7fdc771f2700 1 -- 172.21.15.94:0/3978409655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc701a7090 con 0x7fdc70100ba0 2024-02-17T11:57:07.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.634+0000 7fdc771f2700 1 -- 172.21.15.94:0/3978409655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc70068930 con 0x7fdc70100ba0 2024-02-17T11:57:07.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.635+0000 7fdc6dffb700 1 -- 172.21.15.94:0/3978409655 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fdc640229e0 con 0x7fdc70100ba0 2024-02-17T11:57:07.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.635+0000 7fdc6dffb700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdc600a2260 0x7fdc600a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:07.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.635+0000 7fdc6dffb700 1 -- 172.21.15.94:0/3978409655 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdc640cf100 con 0x7fdc70100ba0 2024-02-17T11:57:07.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.635+0000 7fdc6ffff700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdc600a2260 0x7fdc600a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:07.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.636+0000 7fdc6ffff700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdc600a2260 0x7fdc600a4720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc70105970 tx=0x7fdc5c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:07.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.640+0000 7fdc6dffb700 1 -- 172.21.15.94:0/3978409655 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdc6405dd20 con 0x7fdc70100ba0 2024-02-17T11:57:07.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.839+0000 7fdc771f2700 1 -- 172.21.15.94:0/3978409655 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fdc70110c90 con 0x7fdc70100ba0 2024-02-17T11:57:07.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.840+0000 7fdc6dffb700 1 -- 172.21.15.94:0/3978409655 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fdc64098120 con 0x7fdc70100ba0 2024-02-17T11:57:07.838 INFO:teuthology.orchestra.run.smithi094.stdout:343597383707 2024-02-17T11:57:07.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.842+0000 7fdc771f2700 1 -- 172.21.15.94:0/3978409655 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdc600a2260 msgr2=0x7fdc600a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:07.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.842+0000 7fdc771f2700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdc600a2260 0x7fdc600a4720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc70105970 tx=0x7fdc5c008040 comp rx=0 tx=0).stop 2024-02-17T11:57:07.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.842+0000 7fdc771f2700 1 -- 172.21.15.94:0/3978409655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc70100ba0 msgr2=0x7fdc701027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:07.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.842+0000 7fdc771f2700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc70100ba0 0x7fdc701027c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdc640098e0 tx=0x7fdc6400f740 comp rx=0 tx=0).stop 2024-02-17T11:57:07.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.842+0000 7fdc771f2700 1 -- 172.21.15.94:0/3978409655 shutdown_connections 2024-02-17T11:57:07.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.842+0000 7fdc771f2700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdc600a2260 0x7fdc600a4720 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.842+0000 7fdc771f2700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc70106bb0 0x7fdc70103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.842+0000 7fdc771f2700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc70100ba0 0x7fdc701027c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.842+0000 7fdc771f2700 1 --2- 172.21.15.94:0/3978409655 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc70101560 0x7fdc70102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.842+0000 7fdc771f2700 1 -- 172.21.15.94:0/3978409655 >> 172.21.15.94:0/3978409655 conn(0x7fdc70076b80 msgr2=0x7fdc700ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:07.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.842+0000 7fdc771f2700 1 -- 172.21.15.94:0/3978409655 shutdown_connections 2024-02-17T11:57:07.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.842+0000 7fdc771f2700 1 -- 172.21.15.94:0/3978409655 wait complete. 2024-02-17T11:57:07.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.877+0000 7f2a64d61700 1 -- 172.21.15.94:0/3025666055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a601026b0 msgr2=0x7f2a6010eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:07.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.877+0000 7f2a64d61700 1 --2- 172.21.15.94:0/3025666055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a601026b0 0x7f2a6010eb10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800b0a0 tx=0x7f2a5800b3b0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.878+0000 7f2a64d61700 1 -- 172.21.15.94:0/3025666055 shutdown_connections 2024-02-17T11:57:07.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.878+0000 7f2a64d61700 1 --2- 172.21.15.94:0/3025666055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a601026b0 0x7f2a6010eb10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.878+0000 7f2a64d61700 1 --2- 172.21.15.94:0/3025666055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a60107dd0 0x7f2a601081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.878+0000 7f2a64d61700 1 --2- 172.21.15.94:0/3025666055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a60101cf0 0x7f2a60102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.878+0000 7f2a64d61700 1 -- 172.21.15.94:0/3025666055 >> 172.21.15.94:0/3025666055 conn(0x7f2a60078d30 msgr2=0x7f2a60079140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:07.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.878+0000 7f2a64d61700 1 -- 172.21.15.94:0/3025666055 shutdown_connections 2024-02-17T11:57:07.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.878+0000 7f2a64d61700 1 -- 172.21.15.94:0/3025666055 wait complete. 2024-02-17T11:57:07.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.878+0000 7f2a64d61700 1 Processor -- start 2024-02-17T11:57:07.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.878+0000 7f2a64d61700 1 -- start start 2024-02-17T11:57:07.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a64d61700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a60101cf0 0x7f2a601982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:07.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a64d61700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a601026b0 0x7f2a60198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:07.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a64d61700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a60107dd0 0x7f2a6019cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:07.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a64d61700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6019d180 con 0x7f2a60107dd0 2024-02-17T11:57:07.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a64d61700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6019d2f0 con 0x7f2a601026b0 2024-02-17T11:57:07.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a64d61700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6019d4b0 con 0x7f2a60101cf0 2024-02-17T11:57:07.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a5ed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a60107dd0 0x7f2a6019cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:07.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a5ed9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a60107dd0 0x7f2a6019cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44842/0 (socket says 172.21.15.94:44842) 2024-02-17T11:57:07.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a5ed9d700 1 -- 172.21.15.94:0/163282114 learned_addr learned my addr 172.21.15.94:0/163282114 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:07.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a5e59c700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a60101cf0 0x7f2a601982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:07.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a5dd9b700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a601026b0 0x7f2a60198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:07.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a5ed9d700 1 -- 172.21.15.94:0/163282114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a60101cf0 msgr2=0x7f2a601982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:07.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a5ed9d700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a60101cf0 0x7f2a601982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a5ed9d700 1 -- 172.21.15.94:0/163282114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a601026b0 msgr2=0x7f2a60198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:07.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a5ed9d700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a601026b0 0x7f2a60198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:07.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a5ed9d700 1 -- 172.21.15.94:0/163282114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a58009d00 con 0x7f2a60107dd0 2024-02-17T11:57:07.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a5e59c700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a60101cf0 0x7f2a601982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:07.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a5dd9b700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a601026b0 0x7f2a60198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:07.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a5ed9d700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a60107dd0 0x7f2a6019cb70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2a58006130 tx=0x7f2a58004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:07.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a4f7fe700 1 -- 172.21.15.94:0/163282114 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5800e040 con 0x7f2a60107dd0 2024-02-17T11:57:07.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a4f7fe700 1 -- 172.21.15.94:0/163282114 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2a58007a30 con 0x7f2a60107dd0 2024-02-17T11:57:07.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a64d61700 1 -- 172.21.15.94:0/163282114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6019d730 con 0x7f2a60107dd0 2024-02-17T11:57:07.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a4f7fe700 1 -- 172.21.15.94:0/163282114 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a580057c0 con 0x7f2a60107dd0 2024-02-17T11:57:07.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.879+0000 7f2a64d61700 1 -- 172.21.15.94:0/163282114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a601a4ff0 con 0x7f2a60107dd0 2024-02-17T11:57:07.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.881+0000 7f2a64d61700 1 -- 172.21.15.94:0/163282114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6010a260 con 0x7f2a60107dd0 2024-02-17T11:57:07.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.882+0000 7f2a4f7fe700 1 -- 172.21.15.94:0/163282114 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f2a58005920 con 0x7f2a60107dd0 2024-02-17T11:57:07.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.882+0000 7f2a4f7fe700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2a480a21e0 0x7f2a480a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:07.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.882+0000 7f2a4f7fe700 1 -- 172.21.15.94:0/163282114 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2a580cff20 con 0x7f2a60107dd0 2024-02-17T11:57:07.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.882+0000 7f2a5e59c700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2a480a21e0 0x7f2a480a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:07.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.883+0000 7f2a5e59c700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2a480a21e0 0x7f2a480a46a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2a500099d0 tx=0x7f2a50008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:07.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:07.886+0000 7f2a4f7fe700 1 -- 172.21.15.94:0/163282114 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a58098e20 con 0x7f2a60107dd0 2024-02-17T11:57:08.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.082+0000 7f2a64d61700 1 -- 172.21.15.94:0/163282114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f2a60199660 con 0x7f2a60107dd0 2024-02-17T11:57:08.081 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:07 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3978409655' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-17T11:57:08.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.082+0000 7f2a4f7fe700 1 -- 172.21.15.94:0/163282114 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2a5809c4e0 con 0x7f2a60107dd0 2024-02-17T11:57:08.082 INFO:teuthology.orchestra.run.smithi094.stdout:287762808868 2024-02-17T11:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.084+0000 7f2a64d61700 1 -- 172.21.15.94:0/163282114 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2a480a21e0 msgr2=0x7f2a480a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.084+0000 7f2a64d61700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2a480a21e0 0x7f2a480a46a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2a500099d0 tx=0x7f2a50008040 comp rx=0 tx=0).stop 2024-02-17T11:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.084+0000 7f2a64d61700 1 -- 172.21.15.94:0/163282114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a60107dd0 msgr2=0x7f2a6019cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.084+0000 7f2a64d61700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a60107dd0 0x7f2a6019cb70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f2a58006130 tx=0x7f2a58004ef0 comp rx=0 tx=0).stop 2024-02-17T11:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.084+0000 7f2a64d61700 1 -- 172.21.15.94:0/163282114 shutdown_connections 2024-02-17T11:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.084+0000 7f2a64d61700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2a480a21e0 0x7f2a480a46a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.084+0000 7f2a64d61700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2a60107dd0 0x7f2a6019cb70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.084+0000 7f2a64d61700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a60101cf0 0x7f2a601982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.084+0000 7f2a64d61700 1 --2- 172.21.15.94:0/163282114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2a601026b0 0x7f2a60198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:08.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.084+0000 7f2a64d61700 1 -- 172.21.15.94:0/163282114 >> 172.21.15.94:0/163282114 conn(0x7f2a60078d30 msgr2=0x7f2a60100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.085+0000 7f2a64d61700 1 -- 172.21.15.94:0/163282114 shutdown_connections 2024-02-17T11:57:08.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:08.085+0000 7f2a64d61700 1 -- 172.21.15.94:0/163282114 wait complete. 2024-02-17T11:57:08.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:07 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3978409655' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-17T11:57:08.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:07 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3978409655' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-17T11:57:08.696 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808862 got 287762808868 for osd.9 2024-02-17T11:57:08.697 DEBUG:teuthology.parallel:result is None 2024-02-17T11:57:08.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:08 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/163282114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-17T11:57:08.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:08 smithi094 ceph-mon[32499]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s; 70 B/s, 0 objects/s recovering 2024-02-17T11:57:09.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:08 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/163282114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-17T11:57:09.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:08 smithi189 ceph-mon[36588]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s; 70 B/s, 0 objects/s recovering 2024-02-17T11:57:09.158 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383701 got 343597383707 for osd.11 2024-02-17T11:57:09.158 DEBUG:teuthology.parallel:result is None 2024-02-17T11:57:09.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:08 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/163282114' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-17T11:57:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:08 smithi125 ceph-mon[36819]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 4 op/s; 70 B/s, 0 objects/s recovering 2024-02-17T11:57:09.640 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411399 for osd.2 2024-02-17T11:57:09.640 DEBUG:teuthology.parallel:result is None 2024-02-17T11:57:09.641 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-17T11:57:09.641 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-17T11:57:10.323 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:57:11.983 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:11 smithi094 ceph-mon[32499]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 63 B/s, 0 objects/s recovering 2024-02-17T11:57:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.003+0000 7f6629487700 1 -- 172.21.15.94:0/984359284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6624102880 msgr2=0x7f66241068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.003+0000 7f6629487700 1 --2- 172.21.15.94:0/984359284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6624102880 0x7f66241068d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6618009990 tx=0x7f6618009ca0 comp rx=0 tx=0).stop 2024-02-17T11:57:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.003+0000 7f6629487700 1 -- 172.21.15.94:0/984359284 shutdown_connections 2024-02-17T11:57:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.003+0000 7f6629487700 1 --2- 172.21.15.94:0/984359284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6624106fb0 0x7f662410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.003+0000 7f6629487700 1 --2- 172.21.15.94:0/984359284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6624101ed0 0x7f66241022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.003+0000 7f6629487700 1 --2- 172.21.15.94:0/984359284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6624102880 0x7f66241068d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.003+0000 7f6629487700 1 -- 172.21.15.94:0/984359284 >> 172.21.15.94:0/984359284 conn(0x7f66240fd740 msgr2=0x7f66240ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.003+0000 7f6629487700 1 -- 172.21.15.94:0/984359284 shutdown_connections 2024-02-17T11:57:12.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.003+0000 7f6629487700 1 -- 172.21.15.94:0/984359284 wait complete. 2024-02-17T11:57:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.003+0000 7f6629487700 1 Processor -- start 2024-02-17T11:57:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6629487700 1 -- start start 2024-02-17T11:57:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6629487700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6624101ed0 0x7f6624198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6629487700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6624102880 0x7f66241987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6629487700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6624106fb0 0x7f662419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6629487700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662419d150 con 0x7f6624101ed0 2024-02-17T11:57:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6629487700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662419d2c0 con 0x7f6624106fb0 2024-02-17T11:57:12.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6629487700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662419d480 con 0x7f6624102880 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6622ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6624101ed0 0x7f6624198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6622ffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6624101ed0 0x7f6624198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44874/0 (socket says 172.21.15.94:44874) 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6622ffd700 1 -- 172.21.15.94:0/2929205723 learned_addr learned my addr 172.21.15.94:0/2929205723 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f66227fc700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6624102880 0x7f66241987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f66237fe700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6624106fb0 0x7f662419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6622ffd700 1 -- 172.21.15.94:0/2929205723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6624102880 msgr2=0x7f66241987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6622ffd700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6624102880 0x7f66241987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6622ffd700 1 -- 172.21.15.94:0/2929205723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6624106fb0 msgr2=0x7f662419cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6622ffd700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6624106fb0 0x7f662419cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6622ffd700 1 -- 172.21.15.94:0/2929205723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180095f0 con 0x7f6624101ed0 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.004+0000 7f6622ffd700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6624101ed0 0x7f6624198270 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f661400c920 tx=0x7f661400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.005+0000 7f660bfff700 1 -- 172.21.15.94:0/2929205723 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614007b00 con 0x7f6624101ed0 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.005+0000 7f660bfff700 1 -- 172.21.15.94:0/2929205723 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f661400ce70 con 0x7f6624101ed0 2024-02-17T11:57:12.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.005+0000 7f660bfff700 1 -- 172.21.15.94:0/2929205723 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614005610 con 0x7f6624101ed0 2024-02-17T11:57:12.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.005+0000 7f6629487700 1 -- 172.21.15.94:0/2929205723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662419d760 con 0x7f6624101ed0 2024-02-17T11:57:12.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.005+0000 7f6629487700 1 -- 172.21.15.94:0/2929205723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6624074070 con 0x7f6624101ed0 2024-02-17T11:57:12.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.006+0000 7f6629487700 1 -- 172.21.15.94:0/2929205723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662410a2d0 con 0x7f6624101ed0 2024-02-17T11:57:12.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.007+0000 7f660bfff700 1 -- 172.21.15.94:0/2929205723 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f6614004750 con 0x7f6624101ed0 2024-02-17T11:57:12.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.007+0000 7f660bfff700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f660c0a21e0 0x7f660c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:12.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.008+0000 7f66227fc700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f660c0a21e0 0x7f660c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:12.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.008+0000 7f660bfff700 1 -- 172.21.15.94:0/2929205723 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f66140cda20 con 0x7f6624101ed0 2024-02-17T11:57:12.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.008+0000 7f66227fc700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f660c0a21e0 0x7f660c0a46a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6618006010 tx=0x7f6618005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:12.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.012+0000 7f660bfff700 1 -- 172.21.15.94:0/2929205723 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6614096850 con 0x7f6624101ed0 2024-02-17T11:57:12.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:11 smithi189 ceph-mon[36588]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 63 B/s, 0 objects/s recovering 2024-02-17T11:57:12.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:11 smithi125 ceph-mon[36819]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 63 B/s, 0 objects/s recovering 2024-02-17T11:57:12.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.223+0000 7f6629487700 1 -- 172.21.15.94:0/2929205723 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f66241995d0 con 0x7f660c0a21e0 2024-02-17T11:57:12.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.236+0000 7f660bfff700 1 -- 172.21.15.94:0/2929205723 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345566 (secure 0 0 0) 0x7f66241995d0 con 0x7f660c0a21e0 2024-02-17T11:57:12.240 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.239+0000 7f6629487700 1 -- 172.21.15.94:0/2929205723 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f660c0a21e0 msgr2=0x7f660c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.239+0000 7f6629487700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f660c0a21e0 0x7f660c0a46a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6618006010 tx=0x7f6618005960 comp rx=0 tx=0).stop 2024-02-17T11:57:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.239+0000 7f6629487700 1 -- 172.21.15.94:0/2929205723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6624101ed0 msgr2=0x7f6624198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.239+0000 7f6629487700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6624101ed0 0x7f6624198270 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f661400c920 tx=0x7f661400cc30 comp rx=0 tx=0).stop 2024-02-17T11:57:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.239+0000 7f6629487700 1 -- 172.21.15.94:0/2929205723 shutdown_connections 2024-02-17T11:57:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.239+0000 7f6629487700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f660c0a21e0 0x7f660c0a46a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.239+0000 7f6629487700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6624106fb0 0x7f662419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.239+0000 7f6629487700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6624101ed0 0x7f6624198270 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.239+0000 7f6629487700 1 --2- 172.21.15.94:0/2929205723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6624102880 0x7f66241987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.239+0000 7f6629487700 1 -- 172.21.15.94:0/2929205723 >> 172.21.15.94:0/2929205723 conn(0x7f66240fd740 msgr2=0x7f66240ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:12.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.239+0000 7f6629487700 1 -- 172.21.15.94:0/2929205723 shutdown_connections 2024-02-17T11:57:12.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:12.239+0000 7f6629487700 1 -- 172.21.15.94:0/2929205723 wait complete. 2024-02-17T11:57:12.245 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-02-17T11:57:12.923 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-02-17T11:57:10.670470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656040,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":559,"num_read_kb":401,"num_write":340,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360764,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581060,"statfs":{"total":1151940427776,"available":1151571005440,"internally_reserved":0,"allocated":11833344,"data_stored":7436640,"data_compressed":184182,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.942914"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730934+0000","last_change":"2024-02-17T11:55:58.678553+0000","last_active":"2024-02-17T11:56:59.730934+0000","last_peered":"2024-02-17T11:56:59.730934+0000","last_clean":"2024-02-17T11:56:59.730934+0000","last_became_active":"2024-02-17T11:55:58.678212+0000","last_became_peered":"2024-02-17T11:55:58.678212+0000","last_unstale":"2024-02-17T11:56:59.730934+0000","last_undegraded":"2024-02-17T11:56:59.730934+0000","last_fullsized":"2024-02-17T11:56:59.730934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:09:18.011937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728898+0000","last_change":"2024-02-17T11:55:58.681666+0000","last_active":"2024-02-17T11:56:59.728898+0000","last_peered":"2024-02-17T11:56:59.728898+0000","last_clean":"2024-02-17T11:56:59.728898+0000","last_became_active":"2024-02-17T11:55:58.681305+0000","last_became_peered":"2024-02-17T11:55:58.681305+0000","last_unstale":"2024-02-17T11:56:59.728898+0000","last_undegraded":"2024-02-17T11:56:59.728898+0000","last_fullsized":"2024-02-17T11:56:59.728898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:40:03.514130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728930+0000","last_change":"2024-02-17T11:55:58.689465+0000","last_active":"2024-02-17T11:56:59.728930+0000","last_peered":"2024-02-17T11:56:59.728930+0000","last_clean":"2024-02-17T11:56:59.728930+0000","last_became_active":"2024-02-17T11:55:58.689166+0000","last_became_peered":"2024-02-17T11:55:58.689166+0000","last_unstale":"2024-02-17T11:56:59.728930+0000","last_undegraded":"2024-02-17T11:56:59.728930+0000","last_fullsized":"2024-02-17T11:56:59.728930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:43:07.473303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.735853+0000","last_change":"2024-02-17T11:56:59.735287+0000","last_active":"2024-02-17T11:56:59.735853+0000","last_peered":"2024-02-17T11:56:59.735853+0000","last_clean":"2024-02-17T11:56:59.735853+0000","last_became_active":"2024-02-17T11:56:59.734916+0000","last_became_peered":"2024-02-17T11:56:59.734916+0000","last_unstale":"2024-02-17T11:56:59.735853+0000","last_undegraded":"2024-02-17T11:56:59.735853+0000","last_fullsized":"2024-02-17T11:56:59.735853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:25:03.955834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729168+0000","last_change":"2024-02-17T11:55:58.677543+0000","last_active":"2024-02-17T11:56:59.729168+0000","last_peered":"2024-02-17T11:56:59.729168+0000","last_clean":"2024-02-17T11:56:59.729168+0000","last_became_active":"2024-02-17T11:55:58.677223+0000","last_became_peered":"2024-02-17T11:55:58.677223+0000","last_unstale":"2024-02-17T11:56:59.729168+0000","last_undegraded":"2024-02-17T11:56:59.729168+0000","last_fullsized":"2024-02-17T11:56:59.729168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:05:27.194963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.734826+0000","last_change":"2024-02-17T11:55:58.673756+0000","last_active":"2024-02-17T11:56:59.734826+0000","last_peered":"2024-02-17T11:56:59.734826+0000","last_clean":"2024-02-17T11:56:59.734826+0000","last_became_active":"2024-02-17T11:55:58.672775+0000","last_became_peered":"2024-02-17T11:55:58.672775+0000","last_unstale":"2024-02-17T11:56:59.734826+0000","last_undegraded":"2024-02-17T11:56:59.734826+0000","last_fullsized":"2024-02-17T11:56:59.734826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:12:18.658414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.736210+0000","last_change":"2024-02-17T11:56:59.736210+0000","last_active":"2024-02-17T11:56:59.736210+0000","last_peered":"2024-02-17T11:56:59.736210+0000","last_clean":"2024-02-17T11:56:59.736210+0000","last_became_active":"2024-02-17T11:56:59.735834+0000","last_became_peered":"2024-02-17T11:56:59.735834+0000","last_unstale":"2024-02-17T11:56:59.736210+0000","last_undegraded":"2024-02-17T11:56:59.736210+0000","last_fullsized":"2024-02-17T11:56:59.736210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:19:27.677453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732603+0000","last_change":"2024-02-17T11:55:58.675496+0000","last_active":"2024-02-17T11:56:59.732603+0000","last_peered":"2024-02-17T11:56:59.732603+0000","last_clean":"2024-02-17T11:56:59.732603+0000","last_became_active":"2024-02-17T11:55:58.675193+0000","last_became_peered":"2024-02-17T11:55:58.675193+0000","last_unstale":"2024-02-17T11:56:59.732603+0000","last_undegraded":"2024-02-17T11:56:59.732603+0000","last_fullsized":"2024-02-17T11:56:59.732603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:14:29.648857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732537+0000","last_change":"2024-02-17T11:55:58.680859+0000","last_active":"2024-02-17T11:56:59.732537+0000","last_peered":"2024-02-17T11:56:59.732537+0000","last_clean":"2024-02-17T11:56:59.732537+0000","last_became_active":"2024-02-17T11:55:58.680489+0000","last_became_peered":"2024-02-17T11:55:58.680489+0000","last_unstale":"2024-02-17T11:56:59.732537+0000","last_undegraded":"2024-02-17T11:56:59.732537+0000","last_fullsized":"2024-02-17T11:56:59.732537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:29.963349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.737386+0000","last_change":"2024-02-17T11:56:59.737386+0000","last_active":"2024-02-17T11:56:59.737386+0000","last_peered":"2024-02-17T11:56:59.737386+0000","last_clean":"2024-02-17T11:56:59.737386+0000","last_became_active":"2024-02-17T11:56:59.736973+0000","last_became_peered":"2024-02-17T11:56:59.736973+0000","last_unstale":"2024-02-17T11:56:59.737386+0000","last_undegraded":"2024-02-17T11:56:59.737386+0000","last_fullsized":"2024-02-17T11:56:59.737386+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:11:46.180288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732803+0000","last_change":"2024-02-17T11:55:58.688562+0000","last_active":"2024-02-17T11:56:59.732803+0000","last_peered":"2024-02-17T11:56:59.732803+0000","last_clean":"2024-02-17T11:56:59.732803+0000","last_became_active":"2024-02-17T11:55:58.688186+0000","last_became_peered":"2024-02-17T11:55:58.688186+0000","last_unstale":"2024-02-17T11:56:59.732803+0000","last_undegraded":"2024-02-17T11:56:59.732803+0000","last_fullsized":"2024-02-17T11:56:59.732803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:09.668937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.880971+0000","last_change":"2024-02-17T11:55:58.684989+0000","last_active":"2024-02-17T11:56:59.880971+0000","last_peered":"2024-02-17T11:56:59.880971+0000","last_clean":"2024-02-17T11:56:59.880971+0000","last_became_active":"2024-02-17T11:55:58.684656+0000","last_became_peered":"2024-02-17T11:55:58.684656+0000","last_unstale":"2024-02-17T11:56:59.880971+0000","last_undegraded":"2024-02-17T11:56:59.880971+0000","last_fullsized":"2024-02-17T11:56:59.880971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:29:43.480963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731843+0000","last_change":"2024-02-17T11:55:58.662076+0000","last_active":"2024-02-17T11:56:59.731843+0000","last_peered":"2024-02-17T11:56:59.731843+0000","last_clean":"2024-02-17T11:56:59.731843+0000","last_became_active":"2024-02-17T11:55:58.661716+0000","last_became_peered":"2024-02-17T11:55:58.661716+0000","last_unstale":"2024-02-17T11:56:59.731843+0000","last_undegraded":"2024-02-17T11:56:59.731843+0000","last_fullsized":"2024-02-17T11:56:59.731843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:20:39.647774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728662+0000","last_change":"2024-02-17T11:55:58.690762+0000","last_active":"2024-02-17T11:56:59.728662+0000","last_peered":"2024-02-17T11:56:59.728662+0000","last_clean":"2024-02-17T11:56:59.728662+0000","last_became_active":"2024-02-17T11:55:58.690414+0000","last_became_peered":"2024-02-17T11:55:58.690414+0000","last_unstale":"2024-02-17T11:56:59.728662+0000","last_undegraded":"2024-02-17T11:56:59.728662+0000","last_fullsized":"2024-02-17T11:56:59.728662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:37:13.938474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728634+0000","last_change":"2024-02-17T11:55:58.682391+0000","last_active":"2024-02-17T11:56:59.728634+0000","last_peered":"2024-02-17T11:56:59.728634+0000","last_clean":"2024-02-17T11:56:59.728634+0000","last_became_active":"2024-02-17T11:55:58.682048+0000","last_became_peered":"2024-02-17T11:55:58.682048+0000","last_unstale":"2024-02-17T11:56:59.728634+0000","last_undegraded":"2024-02-17T11:56:59.728634+0000","last_fullsized":"2024-02-17T11:56:59.728634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:03:51.667202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732251+0000","last_change":"2024-02-17T11:55:58.685782+0000","last_active":"2024-02-17T11:56:59.732251+0000","last_peered":"2024-02-17T11:56:59.732251+0000","last_clean":"2024-02-17T11:56:59.732251+0000","last_became_active":"2024-02-17T11:55:58.683087+0000","last_became_peered":"2024-02-17T11:55:58.683087+0000","last_unstale":"2024-02-17T11:56:59.732251+0000","last_undegraded":"2024-02-17T11:56:59.732251+0000","last_fullsized":"2024-02-17T11:56:59.732251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:30:39.653717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731016+0000","last_change":"2024-02-17T11:55:58.688400+0000","last_active":"2024-02-17T11:56:59.731016+0000","last_peered":"2024-02-17T11:56:59.731016+0000","last_clean":"2024-02-17T11:56:59.731016+0000","last_became_active":"2024-02-17T11:55:58.688040+0000","last_became_peered":"2024-02-17T11:55:58.688040+0000","last_unstale":"2024-02-17T11:56:59.731016+0000","last_undegraded":"2024-02-17T11:56:59.731016+0000","last_fullsized":"2024-02-17T11:56:59.731016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:39:02.706547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730910+0000","last_change":"2024-02-17T11:55:58.687454+0000","last_active":"2024-02-17T11:56:59.730910+0000","last_peered":"2024-02-17T11:56:59.730910+0000","last_clean":"2024-02-17T11:56:59.730910+0000","last_became_active":"2024-02-17T11:55:58.686432+0000","last_became_peered":"2024-02-17T11:55:58.686432+0000","last_unstale":"2024-02-17T11:56:59.730910+0000","last_undegraded":"2024-02-17T11:56:59.730910+0000","last_fullsized":"2024-02-17T11:56:59.730910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:26:31.497782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733605+0000","last_change":"2024-02-17T11:55:58.681806+0000","last_active":"2024-02-17T11:56:59.733605+0000","last_peered":"2024-02-17T11:56:59.733605+0000","last_clean":"2024-02-17T11:56:59.733605+0000","last_became_active":"2024-02-17T11:55:58.679764+0000","last_became_peered":"2024-02-17T11:55:58.679764+0000","last_unstale":"2024-02-17T11:56:59.733605+0000","last_undegraded":"2024-02-17T11:56:59.733605+0000","last_fullsized":"2024-02-17T11:56:59.733605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:13:14.733417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.887970+0000","last_change":"2024-02-17T11:56:59.887970+0000","last_active":"2024-02-17T11:56:59.887970+0000","last_peered":"2024-02-17T11:56:59.887970+0000","last_clean":"2024-02-17T11:56:59.887970+0000","last_became_active":"2024-02-17T11:56:59.882087+0000","last_became_peered":"2024-02-17T11:56:59.882087+0000","last_unstale":"2024-02-17T11:56:59.887970+0000","last_undegraded":"2024-02-17T11:56:59.887970+0000","last_fullsized":"2024-02-17T11:56:59.887970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:34:08.202837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732852+0000","last_change":"2024-02-17T11:55:58.690679+0000","last_active":"2024-02-17T11:56:59.732852+0000","last_peered":"2024-02-17T11:56:59.732852+0000","last_clean":"2024-02-17T11:56:59.732852+0000","last_became_active":"2024-02-17T11:55:58.690353+0000","last_became_peered":"2024-02-17T11:55:58.690353+0000","last_unstale":"2024-02-17T11:56:59.732852+0000","last_undegraded":"2024-02-17T11:56:59.732852+0000","last_fullsized":"2024-02-17T11:56:59.732852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:45:33.830784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733040+0000","last_change":"2024-02-17T11:55:57.677649+0000","last_active":"2024-02-17T11:56:59.733040+0000","last_peered":"2024-02-17T11:56:59.733040+0000","last_clean":"2024-02-17T11:56:59.733040+0000","last_became_active":"2024-02-17T11:55:57.677331+0000","last_became_peered":"2024-02-17T11:55:57.677331+0000","last_unstale":"2024-02-17T11:56:59.733040+0000","last_undegraded":"2024-02-17T11:56:59.733040+0000","last_fullsized":"2024-02-17T11:56:59.733040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:06.932255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728982+0000","last_change":"2024-02-17T11:55:58.688575+0000","last_active":"2024-02-17T11:56:59.728982+0000","last_peered":"2024-02-17T11:56:59.728982+0000","last_clean":"2024-02-17T11:56:59.728982+0000","last_became_active":"2024-02-17T11:55:58.688238+0000","last_became_peered":"2024-02-17T11:55:58.688238+0000","last_unstale":"2024-02-17T11:56:59.728982+0000","last_undegraded":"2024-02-17T11:56:59.728982+0000","last_fullsized":"2024-02-17T11:56:59.728982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:17:19.774986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729622+0000","last_change":"2024-02-17T11:55:57.679676+0000","last_active":"2024-02-17T11:56:59.729622+0000","last_peered":"2024-02-17T11:56:59.729622+0000","last_clean":"2024-02-17T11:56:59.729622+0000","last_became_active":"2024-02-17T11:55:57.679350+0000","last_became_peered":"2024-02-17T11:55:57.679350+0000","last_unstale":"2024-02-17T11:56:59.729622+0000","last_undegraded":"2024-02-17T11:56:59.729622+0000","last_fullsized":"2024-02-17T11:56:59.729622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:52:45.072896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733182+0000","last_change":"2024-02-17T11:55:58.682212+0000","last_active":"2024-02-17T11:56:59.733182+0000","last_peered":"2024-02-17T11:56:59.733182+0000","last_clean":"2024-02-17T11:56:59.733182+0000","last_became_active":"2024-02-17T11:55:58.681773+0000","last_became_peered":"2024-02-17T11:55:58.681773+0000","last_unstale":"2024-02-17T11:56:59.733182+0000","last_undegraded":"2024-02-17T11:56:59.733182+0000","last_fullsized":"2024-02-17T11:56:59.733182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:28:18.577174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.741288+0000","last_change":"2024-02-17T11:56:59.741288+0000","last_active":"2024-02-17T11:56:59.741288+0000","last_peered":"2024-02-17T11:56:59.741288+0000","last_clean":"2024-02-17T11:56:59.741288+0000","last_became_active":"2024-02-17T11:56:59.734687+0000","last_became_peered":"2024-02-17T11:56:59.734687+0000","last_unstale":"2024-02-17T11:56:59.741288+0000","last_undegraded":"2024-02-17T11:56:59.741288+0000","last_fullsized":"2024-02-17T11:56:59.741288+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:49:15.242774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733252+0000","last_change":"2024-02-17T11:55:58.672771+0000","last_active":"2024-02-17T11:56:59.733252+0000","last_peered":"2024-02-17T11:56:59.733252+0000","last_clean":"2024-02-17T11:56:59.733252+0000","last_became_active":"2024-02-17T11:55:58.671877+0000","last_became_peered":"2024-02-17T11:55:58.671877+0000","last_unstale":"2024-02-17T11:56:59.733252+0000","last_undegraded":"2024-02-17T11:56:59.733252+0000","last_fullsized":"2024-02-17T11:56:59.733252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:38:38.350695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":92,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:04.249705+0000","last_change":"2024-02-17T11:55:57.669396+0000","last_active":"2024-02-17T11:57:04.249705+0000","last_peered":"2024-02-17T11:57:04.249705+0000","last_clean":"2024-02-17T11:57:04.249705+0000","last_became_active":"2024-02-17T11:55:57.669116+0000","last_became_peered":"2024-02-17T11:55:57.669116+0000","last_unstale":"2024-02-17T11:57:04.249705+0000","last_undegraded":"2024-02-17T11:57:04.249705+0000","last_fullsized":"2024-02-17T11:57:04.249705+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:05:11.081309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729167+0000","last_change":"2024-02-17T11:55:58.674379+0000","last_active":"2024-02-17T11:56:59.729167+0000","last_peered":"2024-02-17T11:56:59.729167+0000","last_clean":"2024-02-17T11:56:59.729167+0000","last_became_active":"2024-02-17T11:55:58.674015+0000","last_became_peered":"2024-02-17T11:55:58.674015+0000","last_unstale":"2024-02-17T11:56:59.729167+0000","last_undegraded":"2024-02-17T11:56:59.729167+0000","last_fullsized":"2024-02-17T11:56:59.729167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:44:20.410623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.734826+0000","last_change":"2024-02-17T11:55:57.668436+0000","last_active":"2024-02-17T11:56:59.734826+0000","last_peered":"2024-02-17T11:56:59.734826+0000","last_clean":"2024-02-17T11:56:59.734826+0000","last_became_active":"2024-02-17T11:55:57.668004+0000","last_became_peered":"2024-02-17T11:55:57.668004+0000","last_unstale":"2024-02-17T11:56:59.734826+0000","last_undegraded":"2024-02-17T11:56:59.734826+0000","last_fullsized":"2024-02-17T11:56:59.734826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:13:42.566644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732981+0000","last_change":"2024-02-17T11:55:58.686888+0000","last_active":"2024-02-17T11:56:59.732981+0000","last_peered":"2024-02-17T11:56:59.732981+0000","last_clean":"2024-02-17T11:56:59.732981+0000","last_became_active":"2024-02-17T11:55:58.686376+0000","last_became_peered":"2024-02-17T11:55:58.686376+0000","last_unstale":"2024-02-17T11:56:59.732981+0000","last_undegraded":"2024-02-17T11:56:59.732981+0000","last_fullsized":"2024-02-17T11:56:59.732981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:58:48.292354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731458+0000","last_change":"2024-02-17T11:55:57.665717+0000","last_active":"2024-02-17T11:56:59.731458+0000","last_peered":"2024-02-17T11:56:59.731458+0000","last_clean":"2024-02-17T11:56:59.731458+0000","last_became_active":"2024-02-17T11:55:57.665381+0000","last_became_peered":"2024-02-17T11:55:57.665381+0000","last_unstale":"2024-02-17T11:56:59.731458+0000","last_undegraded":"2024-02-17T11:56:59.731458+0000","last_fullsized":"2024-02-17T11:56:59.731458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:15:27.424332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732499+0000","last_change":"2024-02-17T11:55:58.673261+0000","last_active":"2024-02-17T11:56:59.732499+0000","last_peered":"2024-02-17T11:56:59.732499+0000","last_clean":"2024-02-17T11:56:59.732499+0000","last_became_active":"2024-02-17T11:55:58.672912+0000","last_became_peered":"2024-02-17T11:55:58.672912+0000","last_unstale":"2024-02-17T11:56:59.732499+0000","last_undegraded":"2024-02-17T11:56:59.732499+0000","last_fullsized":"2024-02-17T11:56:59.732499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:42.605291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732598+0000","last_change":"2024-02-17T11:55:57.667792+0000","last_active":"2024-02-17T11:56:59.732598+0000","last_peered":"2024-02-17T11:56:59.732598+0000","last_clean":"2024-02-17T11:56:59.732598+0000","last_became_active":"2024-02-17T11:55:57.667361+0000","last_became_peered":"2024-02-17T11:55:57.667361+0000","last_unstale":"2024-02-17T11:56:59.732598+0000","last_undegraded":"2024-02-17T11:56:59.732598+0000","last_fullsized":"2024-02-17T11:56:59.732598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:48.073742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731060+0000","last_change":"2024-02-17T11:55:58.672180+0000","last_active":"2024-02-17T11:56:59.731060+0000","last_peered":"2024-02-17T11:56:59.731060+0000","last_clean":"2024-02-17T11:56:59.731060+0000","last_became_active":"2024-02-17T11:55:58.671790+0000","last_became_peered":"2024-02-17T11:55:58.671790+0000","last_unstale":"2024-02-17T11:56:59.731060+0000","last_undegraded":"2024-02-17T11:56:59.731060+0000","last_fullsized":"2024-02-17T11:56:59.731060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:48:39.982924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:07.253312+0000","last_change":"2024-02-17T11:55:57.670085+0000","last_active":"2024-02-17T11:57:07.253312+0000","last_peered":"2024-02-17T11:57:07.253312+0000","last_clean":"2024-02-17T11:57:07.253312+0000","last_became_active":"2024-02-17T11:55:57.669648+0000","last_became_peered":"2024-02-17T11:55:57.669648+0000","last_unstale":"2024-02-17T11:57:07.253312+0000","last_undegraded":"2024-02-17T11:57:07.253312+0000","last_fullsized":"2024-02-17T11:57:07.253312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:31:34.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732608+0000","last_change":"2024-02-17T11:55:58.685080+0000","last_active":"2024-02-17T11:56:59.732608+0000","last_peered":"2024-02-17T11:56:59.732608+0000","last_clean":"2024-02-17T11:56:59.732608+0000","last_became_active":"2024-02-17T11:55:58.684763+0000","last_became_peered":"2024-02-17T11:55:58.684763+0000","last_unstale":"2024-02-17T11:56:59.732608+0000","last_undegraded":"2024-02-17T11:56:59.732608+0000","last_fullsized":"2024-02-17T11:56:59.732608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:59:53.763492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732988+0000","last_change":"2024-02-17T11:55:57.676391+0000","last_active":"2024-02-17T11:56:59.732988+0000","last_peered":"2024-02-17T11:56:59.732988+0000","last_clean":"2024-02-17T11:56:59.732988+0000","last_became_active":"2024-02-17T11:55:57.676103+0000","last_became_peered":"2024-02-17T11:55:57.676103+0000","last_unstale":"2024-02-17T11:56:59.732988+0000","last_undegraded":"2024-02-17T11:56:59.732988+0000","last_fullsized":"2024-02-17T11:56:59.732988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:37:52.866379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728974+0000","last_change":"2024-02-17T11:55:58.679274+0000","last_active":"2024-02-17T11:56:59.728974+0000","last_peered":"2024-02-17T11:56:59.728974+0000","last_clean":"2024-02-17T11:56:59.728974+0000","last_became_active":"2024-02-17T11:55:58.678945+0000","last_became_peered":"2024-02-17T11:55:58.678945+0000","last_unstale":"2024-02-17T11:56:59.728974+0000","last_undegraded":"2024-02-17T11:56:59.728974+0000","last_fullsized":"2024-02-17T11:56:59.728974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:12:20.502503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733259+0000","last_change":"2024-02-17T11:55:57.672732+0000","last_active":"2024-02-17T11:56:59.733259+0000","last_peered":"2024-02-17T11:56:59.733259+0000","last_clean":"2024-02-17T11:56:59.733259+0000","last_became_active":"2024-02-17T11:55:57.672409+0000","last_became_peered":"2024-02-17T11:55:57.672409+0000","last_unstale":"2024-02-17T11:56:59.733259+0000","last_undegraded":"2024-02-17T11:56:59.733259+0000","last_fullsized":"2024-02-17T11:56:59.733259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:07:31.556508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733084+0000","last_change":"2024-02-17T11:55:58.689983+0000","last_active":"2024-02-17T11:56:59.733084+0000","last_peered":"2024-02-17T11:56:59.733084+0000","last_clean":"2024-02-17T11:56:59.733084+0000","last_became_active":"2024-02-17T11:55:58.689629+0000","last_became_peered":"2024-02-17T11:55:58.689629+0000","last_unstale":"2024-02-17T11:56:59.733084+0000","last_undegraded":"2024-02-17T11:56:59.733084+0000","last_fullsized":"2024-02-17T11:56:59.733084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:12.321746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'28","reported_seq":189,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:08.272504+0000","last_change":"2024-02-17T11:55:57.673273+0000","last_active":"2024-02-17T11:57:08.272504+0000","last_peered":"2024-02-17T11:57:08.272504+0000","last_clean":"2024-02-17T11:57:08.272504+0000","last_became_active":"2024-02-17T11:55:57.672865+0000","last_became_peered":"2024-02-17T11:55:57.672865+0000","last_unstale":"2024-02-17T11:57:08.272504+0000","last_undegraded":"2024-02-17T11:57:08.272504+0000","last_fullsized":"2024-02-17T11:57:08.272504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:31:01.593798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":142,"num_read_kb":130,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729395+0000","last_change":"2024-02-17T11:55:58.669296+0000","last_active":"2024-02-17T11:56:59.729395+0000","last_peered":"2024-02-17T11:56:59.729395+0000","last_clean":"2024-02-17T11:56:59.729395+0000","last_became_active":"2024-02-17T11:55:58.668927+0000","last_became_peered":"2024-02-17T11:55:58.668927+0000","last_unstale":"2024-02-17T11:56:59.729395+0000","last_undegraded":"2024-02-17T11:56:59.729395+0000","last_fullsized":"2024-02-17T11:56:59.729395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:25.177905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733466+0000","last_change":"2024-02-17T11:55:57.674523+0000","last_active":"2024-02-17T11:56:59.733466+0000","last_peered":"2024-02-17T11:56:59.733466+0000","last_clean":"2024-02-17T11:56:59.733466+0000","last_became_active":"2024-02-17T11:55:57.672375+0000","last_became_peered":"2024-02-17T11:55:57.672375+0000","last_unstale":"2024-02-17T11:56:59.733466+0000","last_undegraded":"2024-02-17T11:56:59.733466+0000","last_fullsized":"2024-02-17T11:56:59.733466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:15:21.568434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731915+0000","last_change":"2024-02-17T11:55:58.682106+0000","last_active":"2024-02-17T11:56:59.731915+0000","last_peered":"2024-02-17T11:56:59.731915+0000","last_clean":"2024-02-17T11:56:59.731915+0000","last_became_active":"2024-02-17T11:55:58.681420+0000","last_became_peered":"2024-02-17T11:55:58.681420+0000","last_unstale":"2024-02-17T11:56:59.731915+0000","last_undegraded":"2024-02-17T11:56:59.731915+0000","last_fullsized":"2024-02-17T11:56:59.731915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:23:00.521690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732537+0000","last_change":"2024-02-17T11:55:57.675161+0000","last_active":"2024-02-17T11:56:59.732537+0000","last_peered":"2024-02-17T11:56:59.732537+0000","last_clean":"2024-02-17T11:56:59.732537+0000","last_became_active":"2024-02-17T11:55:57.674776+0000","last_became_peered":"2024-02-17T11:55:57.674776+0000","last_unstale":"2024-02-17T11:56:59.732537+0000","last_undegraded":"2024-02-17T11:56:59.732537+0000","last_fullsized":"2024-02-17T11:56:59.732537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:25:59.695014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732385+0000","last_change":"2024-02-17T11:55:58.679319+0000","last_active":"2024-02-17T11:56:59.732385+0000","last_peered":"2024-02-17T11:56:59.732385+0000","last_clean":"2024-02-17T11:56:59.732385+0000","last_became_active":"2024-02-17T11:55:58.679001+0000","last_became_peered":"2024-02-17T11:55:58.679001+0000","last_unstale":"2024-02-17T11:56:59.732385+0000","last_undegraded":"2024-02-17T11:56:59.732385+0000","last_fullsized":"2024-02-17T11:56:59.732385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:36:48.493338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":91,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:05.250378+0000","last_change":"2024-02-17T11:55:57.670027+0000","last_active":"2024-02-17T11:57:05.250378+0000","last_peered":"2024-02-17T11:57:05.250378+0000","last_clean":"2024-02-17T11:57:05.250378+0000","last_became_active":"2024-02-17T11:55:57.669692+0000","last_became_peered":"2024-02-17T11:55:57.669692+0000","last_unstale":"2024-02-17T11:57:05.250378+0000","last_undegraded":"2024-02-17T11:57:05.250378+0000","last_fullsized":"2024-02-17T11:57:05.250378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:44:01.001640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731924+0000","last_change":"2024-02-17T11:55:58.672811+0000","last_active":"2024-02-17T11:56:59.731924+0000","last_peered":"2024-02-17T11:56:59.731924+0000","last_clean":"2024-02-17T11:56:59.731924+0000","last_became_active":"2024-02-17T11:55:58.672469+0000","last_became_peered":"2024-02-17T11:55:58.672469+0000","last_unstale":"2024-02-17T11:56:59.731924+0000","last_undegraded":"2024-02-17T11:56:59.731924+0000","last_fullsized":"2024-02-17T11:56:59.731924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:07.451422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729069+0000","last_change":"2024-02-17T11:55:57.668031+0000","last_active":"2024-02-17T11:56:59.729069+0000","last_peered":"2024-02-17T11:56:59.729069+0000","last_clean":"2024-02-17T11:56:59.729069+0000","last_became_active":"2024-02-17T11:55:57.667344+0000","last_became_peered":"2024-02-17T11:55:57.667344+0000","last_unstale":"2024-02-17T11:56:59.729069+0000","last_undegraded":"2024-02-17T11:56:59.729069+0000","last_fullsized":"2024-02-17T11:56:59.729069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:14:20.773902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732837+0000","last_change":"2024-02-17T11:55:58.682013+0000","last_active":"2024-02-17T11:56:59.732837+0000","last_peered":"2024-02-17T11:56:59.732837+0000","last_clean":"2024-02-17T11:56:59.732837+0000","last_became_active":"2024-02-17T11:55:58.681503+0000","last_became_peered":"2024-02-17T11:55:58.681503+0000","last_unstale":"2024-02-17T11:56:59.732837+0000","last_undegraded":"2024-02-17T11:56:59.732837+0000","last_fullsized":"2024-02-17T11:56:59.732837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:24:50.364090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729587+0000","last_change":"2024-02-17T11:55:57.666725+0000","last_active":"2024-02-17T11:56:59.729587+0000","last_peered":"2024-02-17T11:56:59.729587+0000","last_clean":"2024-02-17T11:56:59.729587+0000","last_became_active":"2024-02-17T11:55:57.666339+0000","last_became_peered":"2024-02-17T11:55:57.666339+0000","last_unstale":"2024-02-17T11:56:59.729587+0000","last_undegraded":"2024-02-17T11:56:59.729587+0000","last_fullsized":"2024-02-17T11:56:59.729587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:22:47.324016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730770+0000","last_change":"2024-02-17T11:55:58.675995+0000","last_active":"2024-02-17T11:56:59.730770+0000","last_peered":"2024-02-17T11:56:59.730770+0000","last_clean":"2024-02-17T11:56:59.730770+0000","last_became_active":"2024-02-17T11:55:58.675655+0000","last_became_peered":"2024-02-17T11:55:58.675655+0000","last_unstale":"2024-02-17T11:56:59.730770+0000","last_undegraded":"2024-02-17T11:56:59.730770+0000","last_fullsized":"2024-02-17T11:56:59.730770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:12.465300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730891+0000","last_change":"2024-02-17T11:55:57.669500+0000","last_active":"2024-02-17T11:56:59.730891+0000","last_peered":"2024-02-17T11:56:59.730891+0000","last_clean":"2024-02-17T11:56:59.730891+0000","last_became_active":"2024-02-17T11:55:57.669148+0000","last_became_peered":"2024-02-17T11:55:57.669148+0000","last_unstale":"2024-02-17T11:56:59.730891+0000","last_undegraded":"2024-02-17T11:56:59.730891+0000","last_fullsized":"2024-02-17T11:56:59.730891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:33.821266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.735074+0000","last_change":"2024-02-17T11:55:58.678336+0000","last_active":"2024-02-17T11:56:59.735074+0000","last_peered":"2024-02-17T11:56:59.735074+0000","last_clean":"2024-02-17T11:56:59.735074+0000","last_became_active":"2024-02-17T11:55:58.677967+0000","last_became_peered":"2024-02-17T11:55:58.677967+0000","last_unstale":"2024-02-17T11:56:59.735074+0000","last_undegraded":"2024-02-17T11:56:59.735074+0000","last_fullsized":"2024-02-17T11:56:59.735074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:31:39.406673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732692+0000","last_change":"2024-02-17T11:55:57.674810+0000","last_active":"2024-02-17T11:56:59.732692+0000","last_peered":"2024-02-17T11:56:59.732692+0000","last_clean":"2024-02-17T11:56:59.732692+0000","last_became_active":"2024-02-17T11:55:57.674356+0000","last_became_peered":"2024-02-17T11:55:57.674356+0000","last_unstale":"2024-02-17T11:56:59.732692+0000","last_undegraded":"2024-02-17T11:56:59.732692+0000","last_fullsized":"2024-02-17T11:56:59.732692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:25:57.879239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.735206+0000","last_change":"2024-02-17T11:55:58.680928+0000","last_active":"2024-02-17T11:56:59.735206+0000","last_peered":"2024-02-17T11:56:59.735206+0000","last_clean":"2024-02-17T11:56:59.735206+0000","last_became_active":"2024-02-17T11:55:58.680521+0000","last_became_peered":"2024-02-17T11:55:58.680521+0000","last_unstale":"2024-02-17T11:56:59.735206+0000","last_undegraded":"2024-02-17T11:56:59.735206+0000","last_fullsized":"2024-02-17T11:56:59.735206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:15:15.739127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732913+0000","last_change":"2024-02-17T11:55:58.674771+0000","last_active":"2024-02-17T11:56:59.732913+0000","last_peered":"2024-02-17T11:56:59.732913+0000","last_clean":"2024-02-17T11:56:59.732913+0000","last_became_active":"2024-02-17T11:55:58.674450+0000","last_became_peered":"2024-02-17T11:55:58.674450+0000","last_unstale":"2024-02-17T11:56:59.732913+0000","last_undegraded":"2024-02-17T11:56:59.732913+0000","last_fullsized":"2024-02-17T11:56:59.732913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.031576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":88,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:05.249806+0000","last_change":"2024-02-17T11:55:57.672868+0000","last_active":"2024-02-17T11:57:05.249806+0000","last_peered":"2024-02-17T11:57:05.249806+0000","last_clean":"2024-02-17T11:57:05.249806+0000","last_became_active":"2024-02-17T11:55:57.672584+0000","last_became_peered":"2024-02-17T11:55:57.672584+0000","last_unstale":"2024-02-17T11:57:05.249806+0000","last_undegraded":"2024-02-17T11:57:05.249806+0000","last_fullsized":"2024-02-17T11:57:05.249806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:03:20.307036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732918+0000","last_change":"2024-02-17T11:55:58.692594+0000","last_active":"2024-02-17T11:56:59.732918+0000","last_peered":"2024-02-17T11:56:59.732918+0000","last_clean":"2024-02-17T11:56:59.732918+0000","last_became_active":"2024-02-17T11:55:58.692258+0000","last_became_peered":"2024-02-17T11:55:58.692258+0000","last_unstale":"2024-02-17T11:56:59.732918+0000","last_undegraded":"2024-02-17T11:56:59.732918+0000","last_fullsized":"2024-02-17T11:56:59.732918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:28.497939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.880997+0000","last_change":"2024-02-17T11:55:57.669456+0000","last_active":"2024-02-17T11:56:59.880997+0000","last_peered":"2024-02-17T11:56:59.880997+0000","last_clean":"2024-02-17T11:56:59.880997+0000","last_became_active":"2024-02-17T11:55:57.669067+0000","last_became_peered":"2024-02-17T11:55:57.669067+0000","last_unstale":"2024-02-17T11:56:59.880997+0000","last_undegraded":"2024-02-17T11:56:59.880997+0000","last_fullsized":"2024-02-17T11:56:59.880997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:12.352296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732072+0000","last_change":"2024-02-17T11:54:46.125844+0000","last_active":"2024-02-17T11:56:59.732072+0000","last_peered":"2024-02-17T11:56:59.732072+0000","last_clean":"2024-02-17T11:56:59.732072+0000","last_became_active":"2024-02-17T11:54:46.125387+0000","last_became_peered":"2024-02-17T11:54:46.125387+0000","last_unstale":"2024-02-17T11:56:59.732072+0000","last_undegraded":"2024-02-17T11:56:59.732072+0000","last_fullsized":"2024-02-17T11:56:59.732072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_clean_scrub_stamp":"2024-02-17T11:51:21.581371+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:53:27.219974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731217+0000","last_change":"2024-02-17T11:55:58.681623+0000","last_active":"2024-02-17T11:56:59.731217+0000","last_peered":"2024-02-17T11:56:59.731217+0000","last_clean":"2024-02-17T11:56:59.731217+0000","last_became_active":"2024-02-17T11:55:58.681275+0000","last_became_peered":"2024-02-17T11:55:58.681275+0000","last_unstale":"2024-02-17T11:56:59.731217+0000","last_undegraded":"2024-02-17T11:56:59.731217+0000","last_fullsized":"2024-02-17T11:56:59.731217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:20.685634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732237+0000","last_change":"2024-02-17T11:55:57.679555+0000","last_active":"2024-02-17T11:56:59.732237+0000","last_peered":"2024-02-17T11:56:59.732237+0000","last_clean":"2024-02-17T11:56:59.732237+0000","last_became_active":"2024-02-17T11:55:57.679241+0000","last_became_peered":"2024-02-17T11:55:57.679241+0000","last_unstale":"2024-02-17T11:56:59.732237+0000","last_undegraded":"2024-02-17T11:56:59.732237+0000","last_fullsized":"2024-02-17T11:56:59.732237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:37.482582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729501+0000","last_change":"2024-02-17T11:55:57.679711+0000","last_active":"2024-02-17T11:56:59.729501+0000","last_peered":"2024-02-17T11:56:59.729501+0000","last_clean":"2024-02-17T11:56:59.729501+0000","last_became_active":"2024-02-17T11:55:57.679415+0000","last_became_peered":"2024-02-17T11:55:57.679415+0000","last_unstale":"2024-02-17T11:56:59.729501+0000","last_undegraded":"2024-02-17T11:56:59.729501+0000","last_fullsized":"2024-02-17T11:56:59.729501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:13.095530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731646+0000","last_change":"2024-02-17T11:55:58.670275+0000","last_active":"2024-02-17T11:56:59.731646+0000","last_peered":"2024-02-17T11:56:59.731646+0000","last_clean":"2024-02-17T11:56:59.731646+0000","last_became_active":"2024-02-17T11:55:58.669926+0000","last_became_peered":"2024-02-17T11:55:58.669926+0000","last_unstale":"2024-02-17T11:56:59.731646+0000","last_undegraded":"2024-02-17T11:56:59.731646+0000","last_fullsized":"2024-02-17T11:56:59.731646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:48:35.078389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730945+0000","last_change":"2024-02-17T11:55:57.670364+0000","last_active":"2024-02-17T11:56:59.730945+0000","last_peered":"2024-02-17T11:56:59.730945+0000","last_clean":"2024-02-17T11:56:59.730945+0000","last_became_active":"2024-02-17T11:55:57.669977+0000","last_became_peered":"2024-02-17T11:55:57.669977+0000","last_unstale":"2024-02-17T11:56:59.730945+0000","last_undegraded":"2024-02-17T11:56:59.730945+0000","last_fullsized":"2024-02-17T11:56:59.730945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:42:13.820390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732778+0000","last_change":"2024-02-17T11:55:58.681587+0000","last_active":"2024-02-17T11:56:59.732778+0000","last_peered":"2024-02-17T11:56:59.732778+0000","last_clean":"2024-02-17T11:56:59.732778+0000","last_became_active":"2024-02-17T11:55:58.680024+0000","last_became_peered":"2024-02-17T11:55:58.680024+0000","last_unstale":"2024-02-17T11:56:59.732778+0000","last_undegraded":"2024-02-17T11:56:59.732778+0000","last_fullsized":"2024-02-17T11:56:59.732778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:31:40.399439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.881143+0000","last_change":"2024-02-17T11:55:57.674618+0000","last_active":"2024-02-17T11:56:59.881143+0000","last_peered":"2024-02-17T11:56:59.881143+0000","last_clean":"2024-02-17T11:56:59.881143+0000","last_became_active":"2024-02-17T11:55:57.674246+0000","last_became_peered":"2024-02-17T11:55:57.674246+0000","last_unstale":"2024-02-17T11:56:59.881143+0000","last_undegraded":"2024-02-17T11:56:59.881143+0000","last_fullsized":"2024-02-17T11:56:59.881143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:42:26.681061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.734355+0000","last_change":"2024-02-17T11:56:59.734355+0000","last_active":"2024-02-17T11:56:59.734355+0000","last_peered":"2024-02-17T11:56:59.734355+0000","last_clean":"2024-02-17T11:56:59.734355+0000","last_became_active":"2024-02-17T11:56:59.733975+0000","last_became_peered":"2024-02-17T11:56:59.733975+0000","last_unstale":"2024-02-17T11:56:59.734355+0000","last_undegraded":"2024-02-17T11:56:59.734355+0000","last_fullsized":"2024-02-17T11:56:59.734355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:37:51.426702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732149+0000","last_change":"2024-02-17T11:55:57.667998+0000","last_active":"2024-02-17T11:56:59.732149+0000","last_peered":"2024-02-17T11:56:59.732149+0000","last_clean":"2024-02-17T11:56:59.732149+0000","last_became_active":"2024-02-17T11:55:57.667688+0000","last_became_peered":"2024-02-17T11:55:57.667688+0000","last_unstale":"2024-02-17T11:56:59.732149+0000","last_undegraded":"2024-02-17T11:56:59.732149+0000","last_fullsized":"2024-02-17T11:56:59.732149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:07:19.782879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729069+0000","last_change":"2024-02-17T11:55:58.676644+0000","last_active":"2024-02-17T11:56:59.729069+0000","last_peered":"2024-02-17T11:56:59.729069+0000","last_clean":"2024-02-17T11:56:59.729069+0000","last_became_active":"2024-02-17T11:55:58.674670+0000","last_became_peered":"2024-02-17T11:55:58.674670+0000","last_unstale":"2024-02-17T11:56:59.729069+0000","last_undegraded":"2024-02-17T11:56:59.729069+0000","last_fullsized":"2024-02-17T11:56:59.729069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:43:22.746537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732754+0000","last_change":"2024-02-17T11:55:57.671330+0000","last_active":"2024-02-17T11:56:59.732754+0000","last_peered":"2024-02-17T11:56:59.732754+0000","last_clean":"2024-02-17T11:56:59.732754+0000","last_became_active":"2024-02-17T11:55:57.670902+0000","last_became_peered":"2024-02-17T11:55:57.670902+0000","last_unstale":"2024-02-17T11:56:59.732754+0000","last_undegraded":"2024-02-17T11:56:59.732754+0000","last_fullsized":"2024-02-17T11:56:59.732754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:22:14.770339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730447+0000","last_change":"2024-02-17T11:55:58.676417+0000","last_active":"2024-02-17T11:56:59.730447+0000","last_peered":"2024-02-17T11:56:59.730447+0000","last_clean":"2024-02-17T11:56:59.730447+0000","last_became_active":"2024-02-17T11:55:58.675960+0000","last_became_peered":"2024-02-17T11:55:58.675960+0000","last_unstale":"2024-02-17T11:56:59.730447+0000","last_undegraded":"2024-02-17T11:56:59.730447+0000","last_fullsized":"2024-02-17T11:56:59.730447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:22:18.566872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.737823+0000","last_change":"2024-02-17T11:56:59.737823+0000","last_active":"2024-02-17T11:56:59.737823+0000","last_peered":"2024-02-17T11:56:59.737823+0000","last_clean":"2024-02-17T11:56:59.737823+0000","last_became_active":"2024-02-17T11:56:59.734043+0000","last_became_peered":"2024-02-17T11:56:59.734043+0000","last_unstale":"2024-02-17T11:56:59.737823+0000","last_undegraded":"2024-02-17T11:56:59.737823+0000","last_fullsized":"2024-02-17T11:56:59.737823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:11:09.171845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732972+0000","last_change":"2024-02-17T11:55:58.690030+0000","last_active":"2024-02-17T11:56:59.732972+0000","last_peered":"2024-02-17T11:56:59.732972+0000","last_clean":"2024-02-17T11:56:59.732972+0000","last_became_active":"2024-02-17T11:55:58.689687+0000","last_became_peered":"2024-02-17T11:55:58.689687+0000","last_unstale":"2024-02-17T11:56:59.732972+0000","last_undegraded":"2024-02-17T11:56:59.732972+0000","last_fullsized":"2024-02-17T11:56:59.732972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:33:43.175234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729187+0000","last_change":"2024-02-17T11:55:57.679734+0000","last_active":"2024-02-17T11:56:59.729187+0000","last_peered":"2024-02-17T11:56:59.729187+0000","last_clean":"2024-02-17T11:56:59.729187+0000","last_became_active":"2024-02-17T11:55:57.679328+0000","last_became_peered":"2024-02-17T11:55:57.679328+0000","last_unstale":"2024-02-17T11:56:59.729187+0000","last_undegraded":"2024-02-17T11:56:59.729187+0000","last_fullsized":"2024-02-17T11:56:59.729187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:50:09.813482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732863+0000","last_change":"2024-02-17T11:55:58.688360+0000","last_active":"2024-02-17T11:56:59.732863+0000","last_peered":"2024-02-17T11:56:59.732863+0000","last_clean":"2024-02-17T11:56:59.732863+0000","last_became_active":"2024-02-17T11:55:58.687994+0000","last_became_peered":"2024-02-17T11:55:58.687994+0000","last_unstale":"2024-02-17T11:56:59.732863+0000","last_undegraded":"2024-02-17T11:56:59.732863+0000","last_fullsized":"2024-02-17T11:56:59.732863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:01:02.163238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733202+0000","last_change":"2024-02-17T11:55:57.675977+0000","last_active":"2024-02-17T11:56:59.733202+0000","last_peered":"2024-02-17T11:56:59.733202+0000","last_clean":"2024-02-17T11:56:59.733202+0000","last_became_active":"2024-02-17T11:55:57.675250+0000","last_became_peered":"2024-02-17T11:55:57.675250+0000","last_unstale":"2024-02-17T11:56:59.733202+0000","last_undegraded":"2024-02-17T11:56:59.733202+0000","last_fullsized":"2024-02-17T11:56:59.733202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:11:27.611692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732873+0000","last_change":"2024-02-17T11:55:58.688593+0000","last_active":"2024-02-17T11:56:59.732873+0000","last_peered":"2024-02-17T11:56:59.732873+0000","last_clean":"2024-02-17T11:56:59.732873+0000","last_became_active":"2024-02-17T11:55:58.688163+0000","last_became_peered":"2024-02-17T11:55:58.688163+0000","last_unstale":"2024-02-17T11:56:59.732873+0000","last_undegraded":"2024-02-17T11:56:59.732873+0000","last_fullsized":"2024-02-17T11:56:59.732873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:54.597116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733115+0000","last_change":"2024-02-17T11:55:57.676860+0000","last_active":"2024-02-17T11:56:59.733115+0000","last_peered":"2024-02-17T11:56:59.733115+0000","last_clean":"2024-02-17T11:56:59.733115+0000","last_became_active":"2024-02-17T11:55:57.676534+0000","last_became_peered":"2024-02-17T11:55:57.676534+0000","last_unstale":"2024-02-17T11:56:59.733115+0000","last_undegraded":"2024-02-17T11:56:59.733115+0000","last_fullsized":"2024-02-17T11:56:59.733115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:23:47.726264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.739615+0000","last_change":"2024-02-17T11:56:59.739615+0000","last_active":"2024-02-17T11:56:59.739615+0000","last_peered":"2024-02-17T11:56:59.739615+0000","last_clean":"2024-02-17T11:56:59.739615+0000","last_became_active":"2024-02-17T11:56:59.736220+0000","last_became_peered":"2024-02-17T11:56:59.736220+0000","last_unstale":"2024-02-17T11:56:59.739615+0000","last_undegraded":"2024-02-17T11:56:59.739615+0000","last_fullsized":"2024-02-17T11:56:59.739615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:22:07.463713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.739390+0000","last_change":"2024-02-17T11:56:59.739390+0000","last_active":"2024-02-17T11:56:59.739390+0000","last_peered":"2024-02-17T11:56:59.739390+0000","last_clean":"2024-02-17T11:56:59.739390+0000","last_became_active":"2024-02-17T11:56:59.734852+0000","last_became_peered":"2024-02-17T11:56:59.734852+0000","last_unstale":"2024-02-17T11:56:59.739390+0000","last_undegraded":"2024-02-17T11:56:59.739390+0000","last_fullsized":"2024-02-17T11:56:59.739390+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:40:09.174557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732359+0000","last_change":"2024-02-17T11:55:58.672985+0000","last_active":"2024-02-17T11:56:59.732359+0000","last_peered":"2024-02-17T11:56:59.732359+0000","last_clean":"2024-02-17T11:56:59.732359+0000","last_became_active":"2024-02-17T11:55:58.672622+0000","last_became_peered":"2024-02-17T11:55:58.672622+0000","last_unstale":"2024-02-17T11:56:59.732359+0000","last_undegraded":"2024-02-17T11:56:59.732359+0000","last_fullsized":"2024-02-17T11:56:59.732359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:45.555827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730520+0000","last_change":"2024-02-17T11:55:58.676127+0000","last_active":"2024-02-17T11:56:59.730520+0000","last_peered":"2024-02-17T11:56:59.730520+0000","last_clean":"2024-02-17T11:56:59.730520+0000","last_became_active":"2024-02-17T11:55:58.675784+0000","last_became_peered":"2024-02-17T11:55:58.675784+0000","last_unstale":"2024-02-17T11:56:59.730520+0000","last_undegraded":"2024-02-17T11:56:59.730520+0000","last_fullsized":"2024-02-17T11:56:59.730520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:56:32.648803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733209+0000","last_change":"2024-02-17T11:55:58.682438+0000","last_active":"2024-02-17T11:56:59.733209+0000","last_peered":"2024-02-17T11:56:59.733209+0000","last_clean":"2024-02-17T11:56:59.733209+0000","last_became_active":"2024-02-17T11:55:58.682067+0000","last_became_peered":"2024-02-17T11:55:58.682067+0000","last_unstale":"2024-02-17T11:56:59.733209+0000","last_undegraded":"2024-02-17T11:56:59.733209+0000","last_fullsized":"2024-02-17T11:56:59.733209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:10:22.906450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733833+0000","last_change":"2024-02-17T11:55:58.686671+0000","last_active":"2024-02-17T11:56:59.733833+0000","last_peered":"2024-02-17T11:56:59.733833+0000","last_clean":"2024-02-17T11:56:59.733833+0000","last_became_active":"2024-02-17T11:55:58.686315+0000","last_became_peered":"2024-02-17T11:55:58.686315+0000","last_unstale":"2024-02-17T11:56:59.733833+0000","last_undegraded":"2024-02-17T11:56:59.733833+0000","last_fullsized":"2024-02-17T11:56:59.733833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:55:40.864847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728763+0000","last_change":"2024-02-17T11:55:58.684227+0000","last_active":"2024-02-17T11:56:59.728763+0000","last_peered":"2024-02-17T11:56:59.728763+0000","last_clean":"2024-02-17T11:56:59.728763+0000","last_became_active":"2024-02-17T11:55:58.683875+0000","last_became_peered":"2024-02-17T11:55:58.683875+0000","last_unstale":"2024-02-17T11:56:59.728763+0000","last_undegraded":"2024-02-17T11:56:59.728763+0000","last_fullsized":"2024-02-17T11:56:59.728763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:11:41.283161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731741+0000","last_change":"2024-02-17T11:55:58.685964+0000","last_active":"2024-02-17T11:56:59.731741+0000","last_peered":"2024-02-17T11:56:59.731741+0000","last_clean":"2024-02-17T11:56:59.731741+0000","last_became_active":"2024-02-17T11:55:58.685350+0000","last_became_peered":"2024-02-17T11:55:58.685350+0000","last_unstale":"2024-02-17T11:56:59.731741+0000","last_undegraded":"2024-02-17T11:56:59.731741+0000","last_fullsized":"2024-02-17T11:56:59.731741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:22:34.600604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730677+0000","last_change":"2024-02-17T11:55:58.671291+0000","last_active":"2024-02-17T11:56:59.730677+0000","last_peered":"2024-02-17T11:56:59.730677+0000","last_clean":"2024-02-17T11:56:59.730677+0000","last_became_active":"2024-02-17T11:55:58.670925+0000","last_became_peered":"2024-02-17T11:55:58.670925+0000","last_unstale":"2024-02-17T11:56:59.730677+0000","last_undegraded":"2024-02-17T11:56:59.730677+0000","last_fullsized":"2024-02-17T11:56:59.730677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:26:03.120976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.734015+0000","last_change":"2024-02-17T11:55:58.673122+0000","last_active":"2024-02-17T11:56:59.734015+0000","last_peered":"2024-02-17T11:56:59.734015+0000","last_clean":"2024-02-17T11:56:59.734015+0000","last_became_active":"2024-02-17T11:55:58.672747+0000","last_became_peered":"2024-02-17T11:55:58.672747+0000","last_unstale":"2024-02-17T11:56:59.734015+0000","last_undegraded":"2024-02-17T11:56:59.734015+0000","last_fullsized":"2024-02-17T11:56:59.734015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:55:15.064073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732668+0000","last_change":"2024-02-17T11:55:58.681276+0000","last_active":"2024-02-17T11:56:59.732668+0000","last_peered":"2024-02-17T11:56:59.732668+0000","last_clean":"2024-02-17T11:56:59.732668+0000","last_became_active":"2024-02-17T11:55:58.680889+0000","last_became_peered":"2024-02-17T11:55:58.680889+0000","last_unstale":"2024-02-17T11:56:59.732668+0000","last_undegraded":"2024-02-17T11:56:59.732668+0000","last_fullsized":"2024-02-17T11:56:59.732668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:19:34.330288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732681+0000","last_change":"2024-02-17T11:55:58.667475+0000","last_active":"2024-02-17T11:56:59.732681+0000","last_peered":"2024-02-17T11:56:59.732681+0000","last_clean":"2024-02-17T11:56:59.732681+0000","last_became_active":"2024-02-17T11:55:58.666577+0000","last_became_peered":"2024-02-17T11:55:58.666577+0000","last_unstale":"2024-02-17T11:56:59.732681+0000","last_undegraded":"2024-02-17T11:56:59.732681+0000","last_fullsized":"2024-02-17T11:56:59.732681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:51:35.732035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732861+0000","last_change":"2024-02-17T11:55:58.674557+0000","last_active":"2024-02-17T11:56:59.732861+0000","last_peered":"2024-02-17T11:56:59.732861+0000","last_clean":"2024-02-17T11:56:59.732861+0000","last_became_active":"2024-02-17T11:55:58.673874+0000","last_became_peered":"2024-02-17T11:55:58.673874+0000","last_unstale":"2024-02-17T11:56:59.732861+0000","last_undegraded":"2024-02-17T11:56:59.732861+0000","last_fullsized":"2024-02-17T11:56:59.732861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:18:00.812489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732369+0000","last_change":"2024-02-17T11:55:58.692725+0000","last_active":"2024-02-17T11:56:59.732369+0000","last_peered":"2024-02-17T11:56:59.732369+0000","last_clean":"2024-02-17T11:56:59.732369+0000","last_became_active":"2024-02-17T11:55:58.690880+0000","last_became_peered":"2024-02-17T11:55:58.690880+0000","last_unstale":"2024-02-17T11:56:59.732369+0000","last_undegraded":"2024-02-17T11:56:59.732369+0000","last_fullsized":"2024-02-17T11:56:59.732369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:03:44.724385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731750+0000","last_change":"2024-02-17T11:55:58.679377+0000","last_active":"2024-02-17T11:56:59.731750+0000","last_peered":"2024-02-17T11:56:59.731750+0000","last_clean":"2024-02-17T11:56:59.731750+0000","last_became_active":"2024-02-17T11:55:58.679041+0000","last_became_peered":"2024-02-17T11:55:58.679041+0000","last_unstale":"2024-02-17T11:56:59.731750+0000","last_undegraded":"2024-02-17T11:56:59.731750+0000","last_fullsized":"2024-02-17T11:56:59.731750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:16:37.396424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731242+0000","last_change":"2024-02-17T11:55:58.680665+0000","last_active":"2024-02-17T11:56:59.731242+0000","last_peered":"2024-02-17T11:56:59.731242+0000","last_clean":"2024-02-17T11:56:59.731242+0000","last_became_active":"2024-02-17T11:55:58.680311+0000","last_became_peered":"2024-02-17T11:55:58.680311+0000","last_unstale":"2024-02-17T11:56:59.731242+0000","last_undegraded":"2024-02-17T11:56:59.731242+0000","last_fullsized":"2024-02-17T11:56:59.731242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:19:11.455471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284587,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65438,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":423,"num_read_kb":282,"num_write":157,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":196314,"data_compressed":5598,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383708,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":456292,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.437}]},{"osd":1,"last update":"Sat Feb 17 11:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.83499999999999996}]},{"osd":2,"last update":"Sat Feb 17 11:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Sat Feb 17 11:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.85999999999999999}]},{"osd":5,"last update":"Sat Feb 17 11:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.53100000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Feb 17 11:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.75900000000000001}]},{"osd":8,"last update":"Sat Feb 17 11:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.745}]},{"osd":9,"last update":"Sat Feb 17 11:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.67900000000000005}]},{"osd":10,"last update":"Sat Feb 17 11:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.81899999999999995}]}]},{"osd":10,"up_from":72,"seq":309237645345,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":1109210,"data_compressed":43889,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.47699999999999998}]},{"osd":1,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.46400000000000002,"15min":0.47499999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63400000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.46999999999999997,"15min":0.47699999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60199999999999998,"5min":0.746,"15min":0.746},"last":0.44900000000000001}]},{"osd":2,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81799999999999995,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.442,"5min":0.44,"15min":0.44},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.67300000000000004}]},{"osd":3,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81799999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.54800000000000004}]},{"osd":4,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68500000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.499},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.52300000000000002,"15min":0.53600000000000003},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.622,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.58899999999999997}]},{"osd":5,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.50700000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79600000000000004,"5min":0.871,"15min":0.871},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67800000000000005,"15min":0.68000000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.84499999999999997,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65200000000000002}]},{"osd":6,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.66700000000000004,"15min":0.68100000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73399999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.622},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82199999999999995,"5min":1.101,"15min":1.101},"last":0.60999999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.41499999999999998}]},{"osd":8,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72199999999999998,"15min":0.72899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92600000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.74299999999999999,"15min":0.75900000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82799999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.80300000000000005}]},{"osd":9,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.73599999999999999,"15min":0.754},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88300000000000001,"5min":1.014,"15min":1.014},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.74399999999999999,"15min":0.76500000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.86099999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.69299999999999995}]},{"osd":11,"last update":"Sat Feb 17 11:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.86399999999999999}]}]},{"osd":9,"up_from":67,"seq":287762808869,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":456645,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.41399999999999998,"15min":0.42299999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.504,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.40999999999999998}]},{"osd":1,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72699999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78300000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.78600000000000003}]},{"osd":2,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.73999999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.629},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55000000000000004,"15min":0.55600000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.47999999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.50600000000000001,"15min":0.52300000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.57199999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.48999999999999999,"15min":0.50600000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.54800000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.441}]},{"osd":4,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.66300000000000003,"15min":0.68100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71499999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.69699999999999995}]},{"osd":5,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.626,"15min":0.64000000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.67600000000000005,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69999999999999996,"15min":0.71999999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81899999999999995,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.59199999999999997,"15min":0.61199999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.64600000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.61799999999999999,"15min":0.63100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78300000000000003,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.46200000000000002}]},{"osd":7,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.73699999999999999,"15min":0.75600000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.78600000000000003,"5min":1.05,"15min":1.05},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.70199999999999996,"15min":0.71699999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.74399999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.71499999999999997}]},{"osd":8,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.72399999999999998,"5min":1.016,"15min":1.016},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.86299999999999999,"5min":1.034,"15min":1.034},"last":0.67500000000000004}]},{"osd":10,"last update":"Sat Feb 17 11:56:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56399999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0129999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.57699999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.036,"5min":1.069,"15min":1.069},"last":0.751}]}]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":455866,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.44,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70399999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.38100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.69599999999999995}]},{"osd":1,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88900000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.72999999999999998}]},{"osd":2,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46000000000000002,"15min":0.44},"min":{"1min":0.35999999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46899999999999997}]},{"osd":3,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.59199999999999997},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.76200000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.61699999999999999,"15min":0.60799999999999998},"min":{"1min":0.503,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85299999999999998,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.44600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.77100000000000002}]},{"osd":5,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56999999999999995,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73399999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.625}]},{"osd":6,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85099999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.52500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.52300000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.53400000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.84799999999999998}]},{"osd":9,"last update":"Sat Feb 17 11:56:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67400000000000004,"15min":0.67100000000000004},"min":{"1min":0.55800000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.82999999999999996,"5min":0.871,"15min":0.871},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66600000000000004,"15min":0.65900000000000003},"min":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67400000000000004}]},{"osd":10,"last update":"Sat Feb 17 11:56:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.749,"15min":0.73799999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.749}]},{"osd":11,"last update":"Sat Feb 17 11:57:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.79400000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332144,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":1048769,"data_compressed":42023,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.36699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39600000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29699999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.58599999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.311},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.30099999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64000000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51800000000000002}]},{"osd":2,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.35599999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.82799999999999996,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.751},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80400000000000005,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.503}]},{"osd":3,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81999999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.42499999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.34300000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81899999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.40300000000000002}]},{"osd":5,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84299999999999997,"5min":1.333,"15min":1.333},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":1.486,"15min":1.486},"last":0.60799999999999998}]},{"osd":6,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.44800000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89100000000000001,"5min":1.087,"15min":1.087},"last":0.47699999999999998}]},{"osd":8,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.55900000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96999999999999997,"5min":1.446,"15min":1.446},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92400000000000004,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.64700000000000002}]},{"osd":9,"last update":"Sat Feb 17 11:56:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68200000000000005,"15min":0.68700000000000006},"min":{"1min":0.51100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91900000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.746,"15min":0.755},"min":{"1min":0.52600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.93600000000000005,"5min":1.052,"15min":1.052},"last":0.53800000000000003}]},{"osd":10,"last update":"Sat Feb 17 11:56:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.498,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95599999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75700000000000001,"15min":0.76100000000000001},"min":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0009999999999999,"5min":1.02,"15min":1.02},"last":0.68000000000000005}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73299999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495669,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":455917,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35499999999999998,"15min":0.34699999999999998},"min":{"1min":0.26600000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.38500000000000001,"15min":0.38300000000000001},"min":{"1min":0.28599999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.67100000000000004,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.29999999999999999}]},{"osd":1,"last update":"Sat Feb 17 11:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56100000000000005,"15min":0.54800000000000004},"min":{"1min":0.436,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76400000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.502},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71199999999999997,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.378}]},{"osd":2,"last update":"Sat Feb 17 11:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56299999999999994,"15min":0.55600000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58299999999999996,"15min":0.57999999999999996},"min":{"1min":0.40200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.79400000000000004,"5min":0.871,"15min":0.871},"last":0.57799999999999996}]},{"osd":3,"last update":"Sat Feb 17 11:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.45300000000000001,"15min":0.44900000000000001},"min":{"1min":0.316,"5min":0.313,"15min":0.313},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.42599999999999999,"15min":0.44},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.57899999999999996,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.27600000000000002}]},{"osd":4,"last update":"Sat Feb 17 11:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40300000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.77600000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.78900000000000003,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.47099999999999997}]},{"osd":5,"last update":"Sat Feb 17 11:56:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.39700000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72999999999999998,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.746,"5min":0.997,"15min":0.997},"last":0.69199999999999995}]},{"osd":7,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81200000000000006,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51700000000000002}]},{"osd":8,"last update":"Sat Feb 17 11:57:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.71799999999999997,"15min":0.70099999999999996},"min":{"1min":0.46400000000000002,"5min":0.434,"15min":0.434},"max":{"1min":5.4269999999999996,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.55700000000000005}]},{"osd":9,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.68999999999999995,"5min":0.879,"15min":0.879},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.67100000000000004,"15min":0.68799999999999994},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78800000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.53300000000000003}]},{"osd":10,"last update":"Sat Feb 17 11:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.71999999999999997,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.86799999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":516756,"data_compressed":7701,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54500000000000004,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77800000000000002,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.498},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71199999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.71199999999999997}]},{"osd":2,"last update":"Sat Feb 17 11:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55100000000000005,"15min":0.52300000000000002},"min":{"1min":0.33000000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.54600000000000004,"15min":0.52200000000000002},"min":{"1min":0.371,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.76200000000000001}]},{"osd":3,"last update":"Sat Feb 17 11:56:51 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58399999999999996,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.626},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.59299999999999997}]},{"osd":4,"last update":"Sat Feb 17 11:56:21 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.443,"15min":0.42599999999999999},"min":{"1min":0.32200000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.437,"15min":0.41099999999999998},"min":{"1min":0.29599999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.55200000000000005}]},{"osd":5,"last update":"Sat Feb 17 11:56:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.621,"15min":0.57799999999999996},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89400000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.64200000000000002,"15min":0.59699999999999998},"min":{"1min":0.58199999999999996,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.78800000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63300000000000001,"15min":0.60099999999999998},"min":{"1min":0.52000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.88300000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.62,"15min":0.59699999999999998},"min":{"1min":0.51000000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78600000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.747}]},{"osd":7,"last update":"Sat Feb 17 11:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56499999999999995,"15min":0.53200000000000003},"min":{"1min":0.40300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91200000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61699999999999999,"15min":0.58899999999999997},"min":{"1min":0.46200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.018,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.89000000000000001}]},{"osd":8,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.71799999999999997,"15min":0.68799999999999994},"min":{"1min":0.622,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98599999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66500000000000004,"15min":0.65400000000000003},"min":{"1min":0.56899999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.83799999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.80100000000000005}]},{"osd":9,"last update":"Sat Feb 17 11:56:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93899999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77900000000000003,"15min":0.77400000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.82899999999999996}]},{"osd":10,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.878,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78300000000000003,"15min":0.77400000000000002},"min":{"1min":0.623,"5min":0.501,"15min":0.501},"max":{"1min":0.96999999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.81499999999999995}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.874}]}]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":457188,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.53700000000000003,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52600000000000002,"15min":0.47999999999999998},"min":{"1min":0.40400000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73199999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48699999999999999}]},{"osd":2,"last update":"Sat Feb 17 11:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56699999999999995,"15min":0.51100000000000001},"min":{"1min":0.45400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78400000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.51900000000000002},"min":{"1min":0.40500000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.54700000000000004}]},{"osd":3,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.32600000000000001}]},{"osd":4,"last update":"Sat Feb 17 11:56:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.50900000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.434},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55500000000000005,"15min":0.51900000000000002},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46899999999999997}]},{"osd":5,"last update":"Sat Feb 17 11:56:46 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.59399999999999997},"min":{"1min":0.48999999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.45400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.498,"15min":0.502},"min":{"1min":0.307,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64200000000000002,"5min":0.877,"15min":0.877},"last":0.376},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68799999999999994,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.39000000000000001}]},{"osd":7,"last update":"Sat Feb 17 11:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.501,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58399999999999996}]},{"osd":8,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.69499999999999995,"15min":0.70599999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.46600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.497}]},{"osd":9,"last update":"Sat Feb 17 11:56:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.36299999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:56:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.73899999999999999,"15min":0.753},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.44800000000000001}]},{"osd":11,"last update":"Sat Feb 17 11:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.64600000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":518497,"data_compressed":7701,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51400000000000001,"15min":0.49299999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71499999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52300000000000002,"15min":0.501},"min":{"1min":0.33700000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.76300000000000001}]},{"osd":1,"last update":"Sat Feb 17 11:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52500000000000002,"15min":0.48599999999999999},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.73399999999999999,"5min":0.751,"15min":0.751},"last":0.438},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.54900000000000004,"15min":0.48699999999999999},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85099999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.54200000000000004}]},{"osd":3,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.54500000000000004},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.753,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.4750000000000001,"5min":0.71899999999999997,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":16.718,"5min":16.718,"15min":16.718},"last":0.502}]},{"osd":4,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.58999999999999997}]},{"osd":5,"last update":"Sat Feb 17 11:56:43 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.44500000000000001,"15min":0.42499999999999999},"min":{"1min":0.35199999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.621,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.45200000000000001,"15min":0.44},"min":{"1min":0.34100000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71899999999999997,"5min":0.878,"15min":0.878},"last":0.40600000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60799999999999998,"15min":0.58399999999999996},"min":{"1min":0.40799999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.72799999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.52900000000000003}]},{"osd":7,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63700000000000001}]},{"osd":8,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.41199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.628}]},{"osd":9,"last update":"Sat Feb 17 11:56:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.52900000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77300000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66600000000000004}]},{"osd":10,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.46500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78300000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.59099999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.84699999999999998,"5min":1.008,"15min":1.008},"last":0.81399999999999995}]},{"osd":11,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.77800000000000002}]}]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456111,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.35799999999999998,"15min":0.34399999999999997},"min":{"1min":0.26100000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.745,"5min":0.746,"15min":0.746},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.34499999999999997},"min":{"1min":0.26400000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.53000000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.39600000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.41699999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.53600000000000003},"min":{"1min":0.36799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78300000000000003,"5min":0.87,"15min":0.87},"last":0.59899999999999998}]},{"osd":2,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77100000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78700000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.46400000000000002}]},{"osd":4,"last update":"Sat Feb 17 11:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75700000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.57899999999999996},"min":{"1min":0.42099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.68899999999999995}]},{"osd":5,"last update":"Sat Feb 17 11:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.41499999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79500000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.63300000000000001}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.44400000000000001},"min":{"1min":0.307,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82999999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95799999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.49099999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":12.301,"5min":12.301,"15min":12.301},"last":0.71399999999999997}]},{"osd":8,"last update":"Sat Feb 17 11:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.49199999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97599999999999998,"5min":1.214,"15min":1.214},"last":0.622}]},{"osd":9,"last update":"Sat Feb 17 11:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.60199999999999998,"15min":0.61599999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76600000000000001,"5min":0.875,"15min":0.875},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.69499999999999995,"15min":0.71499999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82699999999999996,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.53200000000000003}]},{"osd":10,"last update":"Sat Feb 17 11:56:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.86499999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.82399999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.72299999999999998}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.64800000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920831,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":457327,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.50700000000000001,"15min":0.497},"min":{"1min":0.41099999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41599999999999998}]},{"osd":1,"last update":"Sat Feb 17 11:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.26800000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.57299999999999995,"5min":0.872,"15min":0.872},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.40500000000000003,"15min":0.39200000000000002},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.63,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.46000000000000002}]},{"osd":2,"last update":"Sat Feb 17 11:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77600000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63500000000000001,"15min":0.55700000000000005},"min":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91600000000000004,"5min":7.0279999999999996,"15min":7.0279999999999996},"last":0.61699999999999999}]},{"osd":3,"last update":"Sat Feb 17 11:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.49399999999999999}]},{"osd":5,"last update":"Sat Feb 17 11:56:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.40799999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59699999999999998,"15min":0.56200000000000006},"min":{"1min":0.39300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.66800000000000004}]},{"osd":6,"last update":"Sat Feb 17 11:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58299999999999996,"15min":0.57099999999999995},"min":{"1min":0.438,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.60099999999999998,"15min":0.59099999999999997},"min":{"1min":0.42399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84099999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57599999999999996}]},{"osd":7,"last update":"Sat Feb 17 11:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55400000000000005,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56100000000000005,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83999999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.63700000000000001}]},{"osd":8,"last update":"Sat Feb 17 11:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":0.67700000000000005},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90400000000000003,"5min":6.9660000000000002,"15min":6.9660000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72299999999999998,"15min":0.70899999999999996},"min":{"1min":0.52800000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79200000000000004}]},{"osd":9,"last update":"Sat Feb 17 11:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.48299999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.85899999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.751,"15min":0.753},"min":{"1min":0.53800000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.028,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.72599999999999998}]},{"osd":10,"last update":"Sat Feb 17 11:56:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.77500000000000002,"15min":0.78400000000000003},"min":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73199999999999998,"15min":0.73999999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92100000000000004,"5min":1.004,"15min":1.004},"last":0.74299999999999999}]},{"osd":11,"last update":"Sat Feb 17 11:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.70999999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757306,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":1048062,"data_compressed":42023,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51600000000000001,"15min":0.49199999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81899999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52300000000000002,"15min":0.48799999999999999},"min":{"1min":0.39000000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.75,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51200000000000001}]},{"osd":1,"last update":"Sat Feb 17 11:56:42 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55800000000000005,"15min":0.51400000000000001},"min":{"1min":0.503,"5min":0.313,"15min":0.313},"max":{"1min":0.88300000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.55700000000000005,"15min":0.50800000000000001},"min":{"1min":0.46400000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80700000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Sat Feb 17 11:56:42 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.42499999999999999,"15min":0.41499999999999998},"min":{"1min":0.34200000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65700000000000003,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.44,"15min":0.41999999999999998},"min":{"1min":0.32500000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82399999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.48299999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:56:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.629},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.53100000000000003},"min":{"1min":0.44500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.58899999999999997}]},{"osd":4,"last update":"Sat Feb 17 11:56:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.60199999999999998,"15min":0.56399999999999995},"min":{"1min":0.51700000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91300000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59699999999999998,"15min":0.56699999999999995},"min":{"1min":0.53100000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.71099999999999997}]},{"osd":6,"last update":"Sat Feb 17 11:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64900000000000002,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.90000000000000002,"5min":1.087,"15min":1.087},"last":0.74099999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.63700000000000001,"15min":0.61699999999999999},"min":{"1min":0.47699999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.77600000000000002}]},{"osd":8,"last update":"Sat Feb 17 11:56:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.626,"15min":0.61599999999999999},"min":{"1min":0.46700000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.90900000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92800000000000005,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80700000000000005}]},{"osd":9,"last update":"Sat Feb 17 11:56:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.51500000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.99399999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.53600000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93899999999999995,"5min":1.056,"15min":1.056},"last":0.72799999999999998}]},{"osd":10,"last update":"Sat Feb 17 11:56:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.755,"15min":0.745},"min":{"1min":0.66000000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76500000000000001,"15min":0.75900000000000001},"min":{"1min":0.67000000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.755}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61046,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62787,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63015,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T11:57:12.931 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-17T11:57:13.214 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:57:13.460 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:13.460 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:13.460 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:13 smithi094 ceph-mon[32499]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:13.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:13 smithi094 ceph-mon[32499]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-02-17T11:57:13.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:57:13.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:57:13.461 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:13 smithi094 ceph-mon[32499]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:13.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:13.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:13.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:13 smithi189 ceph-mon[36588]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:13.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:13 smithi189 ceph-mon[36588]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-02-17T11:57:13.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:57:13.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:57:13.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:13 smithi189 ceph-mon[36588]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:13.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:13.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:13.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:13 smithi125 ceph-mon[36819]: from='client.14943 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:13.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:13 smithi125 ceph-mon[36819]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-02-17T11:57:13.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:57:13.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:57:13.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:13 smithi125 ceph-mon[36819]: from='mgr.14152 172.21.15.94:0/872745640' entity='mgr.x' 2024-02-17T11:57:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.776+0000 7f9e59c22700 1 -- 172.21.15.94:0/1852856732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5406a3c0 msgr2=0x7f9e5406a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.776+0000 7f9e59c22700 1 --2- 172.21.15.94:0/1852856732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5406a3c0 0x7f9e5406a840 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0099c0 tx=0x7f9e3c009cd0 comp rx=0 tx=0).stop 2024-02-17T11:57:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.776+0000 7f9e59c22700 1 -- 172.21.15.94:0/1852856732 shutdown_connections 2024-02-17T11:57:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.776+0000 7f9e59c22700 1 --2- 172.21.15.94:0/1852856732 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e5406ad80 0x7f9e5410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.776+0000 7f9e59c22700 1 --2- 172.21.15.94:0/1852856732 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e54107ca0 0x7f9e54108080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.776+0000 7f9e59c22700 1 --2- 172.21.15.94:0/1852856732 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e5406a3c0 0x7f9e5406a840 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:14.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.776+0000 7f9e59c22700 1 -- 172.21.15.94:0/1852856732 >> 172.21.15.94:0/1852856732 conn(0x7f9e54077b80 msgr2=0x7f9e54077f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.776+0000 7f9e59c22700 1 -- 172.21.15.94:0/1852856732 shutdown_connections 2024-02-17T11:57:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.776+0000 7f9e59c22700 1 -- 172.21.15.94:0/1852856732 wait complete. 2024-02-17T11:57:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.777+0000 7f9e59c22700 1 Processor -- start 2024-02-17T11:57:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.777+0000 7f9e59c22700 1 -- start start 2024-02-17T11:57:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.777+0000 7f9e59c22700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e5406a3c0 0x7f9e54100c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.777+0000 7f9e59c22700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e5406ad80 0x7f9e54101160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.777+0000 7f9e59c22700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e54107ca0 0x7f9e541016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.777+0000 7f9e59c22700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e541a6570 con 0x7f9e54107ca0 2024-02-17T11:57:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.777+0000 7f9e59c22700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e541a66e0 con 0x7f9e5406a3c0 2024-02-17T11:57:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.777+0000 7f9e59c22700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e541a68a0 con 0x7f9e5406ad80 2024-02-17T11:57:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e53fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e54107ca0 0x7f9e541016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e53fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e54107ca0 0x7f9e541016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42714/0 (socket says 172.21.15.94:42714) 2024-02-17T11:57:14.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e53fff700 1 -- 172.21.15.94:0/497306920 learned_addr learned my addr 172.21.15.94:0/497306920 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e52ffd700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e5406ad80 0x7f9e54101160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e537fe700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e5406a3c0 0x7f9e54100c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e53fff700 1 -- 172.21.15.94:0/497306920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e5406ad80 msgr2=0x7f9e54101160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e53fff700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e5406ad80 0x7f9e54101160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e53fff700 1 -- 172.21.15.94:0/497306920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e5406a3c0 msgr2=0x7f9e54100c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e53fff700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e5406a3c0 0x7f9e54100c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e53fff700 1 -- 172.21.15.94:0/497306920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0095f0 con 0x7f9e54107ca0 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e52ffd700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e5406ad80 0x7f9e54101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e53fff700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e54107ca0 0x7f9e541016a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400b550 tx=0x7f9e4400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e50ff9700 1 -- 172.21.15.94:0/497306920 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44011830 con 0x7f9e54107ca0 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e59c22700 1 -- 172.21.15.94:0/497306920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e541a6b80 con 0x7f9e54107ca0 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e50ff9700 1 -- 172.21.15.94:0/497306920 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e44011e70 con 0x7f9e54107ca0 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e50ff9700 1 -- 172.21.15.94:0/497306920 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4400f2e0 con 0x7f9e54107ca0 2024-02-17T11:57:14.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.778+0000 7f9e59c22700 1 -- 172.21.15.94:0/497306920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e541a7050 con 0x7f9e54107ca0 2024-02-17T11:57:14.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.779+0000 7f9e59c22700 1 -- 172.21.15.94:0/497306920 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e54068930 con 0x7f9e54107ca0 2024-02-17T11:57:14.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.781+0000 7f9e50ff9700 1 -- 172.21.15.94:0/497306920 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f9e44010610 con 0x7f9e54107ca0 2024-02-17T11:57:14.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.781+0000 7f9e50ff9700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f9e400a21e0 0x7f9e400a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:14.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.782+0000 7f9e537fe700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f9e400a21e0 0x7f9e400a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:14.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.782+0000 7f9e50ff9700 1 -- 172.21.15.94:0/497306920 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9e440cdb50 con 0x7f9e54107ca0 2024-02-17T11:57:14.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.782+0000 7f9e537fe700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f9e400a21e0 0x7f9e400a46a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00a400 tx=0x7f9e4c00d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:14.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.785+0000 7f9e50ff9700 1 -- 172.21.15.94:0/497306920 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9e44096ad0 con 0x7f9e54107ca0 2024-02-17T11:57:14.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.975+0000 7f9e59c22700 1 -- 172.21.15.94:0/497306920 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e54102150 con 0x7f9e400a21e0 2024-02-17T11:57:14.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.985+0000 7f9e50ff9700 1 -- 172.21.15.94:0/497306920 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345477 (secure 0 0 0) 0x7f9e54102150 con 0x7f9e400a21e0 2024-02-17T11:57:14.988 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:14.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.990+0000 7f9e59c22700 1 -- 172.21.15.94:0/497306920 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f9e400a21e0 msgr2=0x7f9e400a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:14.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.990+0000 7f9e59c22700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f9e400a21e0 0x7f9e400a46a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00a400 tx=0x7f9e4c00d040 comp rx=0 tx=0).stop 2024-02-17T11:57:14.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.990+0000 7f9e59c22700 1 -- 172.21.15.94:0/497306920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e54107ca0 msgr2=0x7f9e541016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:14.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.990+0000 7f9e59c22700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e54107ca0 0x7f9e541016a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400b550 tx=0x7f9e4400b860 comp rx=0 tx=0).stop 2024-02-17T11:57:14.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.990+0000 7f9e59c22700 1 -- 172.21.15.94:0/497306920 shutdown_connections 2024-02-17T11:57:14.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.990+0000 7f9e59c22700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f9e400a21e0 0x7f9e400a46a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:14.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.990+0000 7f9e59c22700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e54107ca0 0x7f9e541016a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:14.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.990+0000 7f9e59c22700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e5406a3c0 0x7f9e54100c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:14.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.990+0000 7f9e59c22700 1 --2- 172.21.15.94:0/497306920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e5406ad80 0x7f9e54101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:14.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.990+0000 7f9e59c22700 1 -- 172.21.15.94:0/497306920 >> 172.21.15.94:0/497306920 conn(0x7f9e54077b80 msgr2=0x7f9e540ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:14.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.990+0000 7f9e59c22700 1 -- 172.21.15.94:0/497306920 shutdown_connections 2024-02-17T11:57:14.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:14.990+0000 7f9e59c22700 1 -- 172.21.15.94:0/497306920 wait complete. 2024-02-17T11:57:14.992 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-02-17T11:57:15.654 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-02-17T11:57:14.671557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656040,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":589,"num_read_kb":416,"num_write":340,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360764,"kb_used_data":11556,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581060,"statfs":{"total":1151940427776,"available":1151571005440,"internally_reserved":0,"allocated":11833344,"data_stored":7436640,"data_compressed":184182,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003341"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730934+0000","last_change":"2024-02-17T11:55:58.678553+0000","last_active":"2024-02-17T11:56:59.730934+0000","last_peered":"2024-02-17T11:56:59.730934+0000","last_clean":"2024-02-17T11:56:59.730934+0000","last_became_active":"2024-02-17T11:55:58.678212+0000","last_became_peered":"2024-02-17T11:55:58.678212+0000","last_unstale":"2024-02-17T11:56:59.730934+0000","last_undegraded":"2024-02-17T11:56:59.730934+0000","last_fullsized":"2024-02-17T11:56:59.730934+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:09:18.011937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728898+0000","last_change":"2024-02-17T11:55:58.681666+0000","last_active":"2024-02-17T11:56:59.728898+0000","last_peered":"2024-02-17T11:56:59.728898+0000","last_clean":"2024-02-17T11:56:59.728898+0000","last_became_active":"2024-02-17T11:55:58.681305+0000","last_became_peered":"2024-02-17T11:55:58.681305+0000","last_unstale":"2024-02-17T11:56:59.728898+0000","last_undegraded":"2024-02-17T11:56:59.728898+0000","last_fullsized":"2024-02-17T11:56:59.728898+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:40:03.514130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728930+0000","last_change":"2024-02-17T11:55:58.689465+0000","last_active":"2024-02-17T11:56:59.728930+0000","last_peered":"2024-02-17T11:56:59.728930+0000","last_clean":"2024-02-17T11:56:59.728930+0000","last_became_active":"2024-02-17T11:55:58.689166+0000","last_became_peered":"2024-02-17T11:55:58.689166+0000","last_unstale":"2024-02-17T11:56:59.728930+0000","last_undegraded":"2024-02-17T11:56:59.728930+0000","last_fullsized":"2024-02-17T11:56:59.728930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:43:07.473303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.735853+0000","last_change":"2024-02-17T11:56:59.735287+0000","last_active":"2024-02-17T11:56:59.735853+0000","last_peered":"2024-02-17T11:56:59.735853+0000","last_clean":"2024-02-17T11:56:59.735853+0000","last_became_active":"2024-02-17T11:56:59.734916+0000","last_became_peered":"2024-02-17T11:56:59.734916+0000","last_unstale":"2024-02-17T11:56:59.735853+0000","last_undegraded":"2024-02-17T11:56:59.735853+0000","last_fullsized":"2024-02-17T11:56:59.735853+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:25:03.955834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729168+0000","last_change":"2024-02-17T11:55:58.677543+0000","last_active":"2024-02-17T11:56:59.729168+0000","last_peered":"2024-02-17T11:56:59.729168+0000","last_clean":"2024-02-17T11:56:59.729168+0000","last_became_active":"2024-02-17T11:55:58.677223+0000","last_became_peered":"2024-02-17T11:55:58.677223+0000","last_unstale":"2024-02-17T11:56:59.729168+0000","last_undegraded":"2024-02-17T11:56:59.729168+0000","last_fullsized":"2024-02-17T11:56:59.729168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:05:27.194963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.734826+0000","last_change":"2024-02-17T11:55:58.673756+0000","last_active":"2024-02-17T11:56:59.734826+0000","last_peered":"2024-02-17T11:56:59.734826+0000","last_clean":"2024-02-17T11:56:59.734826+0000","last_became_active":"2024-02-17T11:55:58.672775+0000","last_became_peered":"2024-02-17T11:55:58.672775+0000","last_unstale":"2024-02-17T11:56:59.734826+0000","last_undegraded":"2024-02-17T11:56:59.734826+0000","last_fullsized":"2024-02-17T11:56:59.734826+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:12:18.658414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.736210+0000","last_change":"2024-02-17T11:56:59.736210+0000","last_active":"2024-02-17T11:56:59.736210+0000","last_peered":"2024-02-17T11:56:59.736210+0000","last_clean":"2024-02-17T11:56:59.736210+0000","last_became_active":"2024-02-17T11:56:59.735834+0000","last_became_peered":"2024-02-17T11:56:59.735834+0000","last_unstale":"2024-02-17T11:56:59.736210+0000","last_undegraded":"2024-02-17T11:56:59.736210+0000","last_fullsized":"2024-02-17T11:56:59.736210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:19:27.677453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732603+0000","last_change":"2024-02-17T11:55:58.675496+0000","last_active":"2024-02-17T11:56:59.732603+0000","last_peered":"2024-02-17T11:56:59.732603+0000","last_clean":"2024-02-17T11:56:59.732603+0000","last_became_active":"2024-02-17T11:55:58.675193+0000","last_became_peered":"2024-02-17T11:55:58.675193+0000","last_unstale":"2024-02-17T11:56:59.732603+0000","last_undegraded":"2024-02-17T11:56:59.732603+0000","last_fullsized":"2024-02-17T11:56:59.732603+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:14:29.648857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732537+0000","last_change":"2024-02-17T11:55:58.680859+0000","last_active":"2024-02-17T11:56:59.732537+0000","last_peered":"2024-02-17T11:56:59.732537+0000","last_clean":"2024-02-17T11:56:59.732537+0000","last_became_active":"2024-02-17T11:55:58.680489+0000","last_became_peered":"2024-02-17T11:55:58.680489+0000","last_unstale":"2024-02-17T11:56:59.732537+0000","last_undegraded":"2024-02-17T11:56:59.732537+0000","last_fullsized":"2024-02-17T11:56:59.732537+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:29.963349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.737386+0000","last_change":"2024-02-17T11:56:59.737386+0000","last_active":"2024-02-17T11:56:59.737386+0000","last_peered":"2024-02-17T11:56:59.737386+0000","last_clean":"2024-02-17T11:56:59.737386+0000","last_became_active":"2024-02-17T11:56:59.736973+0000","last_became_peered":"2024-02-17T11:56:59.736973+0000","last_unstale":"2024-02-17T11:56:59.737386+0000","last_undegraded":"2024-02-17T11:56:59.737386+0000","last_fullsized":"2024-02-17T11:56:59.737386+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:11:46.180288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732803+0000","last_change":"2024-02-17T11:55:58.688562+0000","last_active":"2024-02-17T11:56:59.732803+0000","last_peered":"2024-02-17T11:56:59.732803+0000","last_clean":"2024-02-17T11:56:59.732803+0000","last_became_active":"2024-02-17T11:55:58.688186+0000","last_became_peered":"2024-02-17T11:55:58.688186+0000","last_unstale":"2024-02-17T11:56:59.732803+0000","last_undegraded":"2024-02-17T11:56:59.732803+0000","last_fullsized":"2024-02-17T11:56:59.732803+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:09.668937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.880971+0000","last_change":"2024-02-17T11:55:58.684989+0000","last_active":"2024-02-17T11:56:59.880971+0000","last_peered":"2024-02-17T11:56:59.880971+0000","last_clean":"2024-02-17T11:56:59.880971+0000","last_became_active":"2024-02-17T11:55:58.684656+0000","last_became_peered":"2024-02-17T11:55:58.684656+0000","last_unstale":"2024-02-17T11:56:59.880971+0000","last_undegraded":"2024-02-17T11:56:59.880971+0000","last_fullsized":"2024-02-17T11:56:59.880971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:29:43.480963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731843+0000","last_change":"2024-02-17T11:55:58.662076+0000","last_active":"2024-02-17T11:56:59.731843+0000","last_peered":"2024-02-17T11:56:59.731843+0000","last_clean":"2024-02-17T11:56:59.731843+0000","last_became_active":"2024-02-17T11:55:58.661716+0000","last_became_peered":"2024-02-17T11:55:58.661716+0000","last_unstale":"2024-02-17T11:56:59.731843+0000","last_undegraded":"2024-02-17T11:56:59.731843+0000","last_fullsized":"2024-02-17T11:56:59.731843+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:20:39.647774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728662+0000","last_change":"2024-02-17T11:55:58.690762+0000","last_active":"2024-02-17T11:56:59.728662+0000","last_peered":"2024-02-17T11:56:59.728662+0000","last_clean":"2024-02-17T11:56:59.728662+0000","last_became_active":"2024-02-17T11:55:58.690414+0000","last_became_peered":"2024-02-17T11:55:58.690414+0000","last_unstale":"2024-02-17T11:56:59.728662+0000","last_undegraded":"2024-02-17T11:56:59.728662+0000","last_fullsized":"2024-02-17T11:56:59.728662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:37:13.938474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728634+0000","last_change":"2024-02-17T11:55:58.682391+0000","last_active":"2024-02-17T11:56:59.728634+0000","last_peered":"2024-02-17T11:56:59.728634+0000","last_clean":"2024-02-17T11:56:59.728634+0000","last_became_active":"2024-02-17T11:55:58.682048+0000","last_became_peered":"2024-02-17T11:55:58.682048+0000","last_unstale":"2024-02-17T11:56:59.728634+0000","last_undegraded":"2024-02-17T11:56:59.728634+0000","last_fullsized":"2024-02-17T11:56:59.728634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:03:51.667202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732251+0000","last_change":"2024-02-17T11:55:58.685782+0000","last_active":"2024-02-17T11:56:59.732251+0000","last_peered":"2024-02-17T11:56:59.732251+0000","last_clean":"2024-02-17T11:56:59.732251+0000","last_became_active":"2024-02-17T11:55:58.683087+0000","last_became_peered":"2024-02-17T11:55:58.683087+0000","last_unstale":"2024-02-17T11:56:59.732251+0000","last_undegraded":"2024-02-17T11:56:59.732251+0000","last_fullsized":"2024-02-17T11:56:59.732251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:30:39.653717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731016+0000","last_change":"2024-02-17T11:55:58.688400+0000","last_active":"2024-02-17T11:56:59.731016+0000","last_peered":"2024-02-17T11:56:59.731016+0000","last_clean":"2024-02-17T11:56:59.731016+0000","last_became_active":"2024-02-17T11:55:58.688040+0000","last_became_peered":"2024-02-17T11:55:58.688040+0000","last_unstale":"2024-02-17T11:56:59.731016+0000","last_undegraded":"2024-02-17T11:56:59.731016+0000","last_fullsized":"2024-02-17T11:56:59.731016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:39:02.706547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730910+0000","last_change":"2024-02-17T11:55:58.687454+0000","last_active":"2024-02-17T11:56:59.730910+0000","last_peered":"2024-02-17T11:56:59.730910+0000","last_clean":"2024-02-17T11:56:59.730910+0000","last_became_active":"2024-02-17T11:55:58.686432+0000","last_became_peered":"2024-02-17T11:55:58.686432+0000","last_unstale":"2024-02-17T11:56:59.730910+0000","last_undegraded":"2024-02-17T11:56:59.730910+0000","last_fullsized":"2024-02-17T11:56:59.730910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:26:31.497782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733605+0000","last_change":"2024-02-17T11:55:58.681806+0000","last_active":"2024-02-17T11:56:59.733605+0000","last_peered":"2024-02-17T11:56:59.733605+0000","last_clean":"2024-02-17T11:56:59.733605+0000","last_became_active":"2024-02-17T11:55:58.679764+0000","last_became_peered":"2024-02-17T11:55:58.679764+0000","last_unstale":"2024-02-17T11:56:59.733605+0000","last_undegraded":"2024-02-17T11:56:59.733605+0000","last_fullsized":"2024-02-17T11:56:59.733605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:13:14.733417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.887970+0000","last_change":"2024-02-17T11:56:59.887970+0000","last_active":"2024-02-17T11:56:59.887970+0000","last_peered":"2024-02-17T11:56:59.887970+0000","last_clean":"2024-02-17T11:56:59.887970+0000","last_became_active":"2024-02-17T11:56:59.882087+0000","last_became_peered":"2024-02-17T11:56:59.882087+0000","last_unstale":"2024-02-17T11:56:59.887970+0000","last_undegraded":"2024-02-17T11:56:59.887970+0000","last_fullsized":"2024-02-17T11:56:59.887970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:34:08.202837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732852+0000","last_change":"2024-02-17T11:55:58.690679+0000","last_active":"2024-02-17T11:56:59.732852+0000","last_peered":"2024-02-17T11:56:59.732852+0000","last_clean":"2024-02-17T11:56:59.732852+0000","last_became_active":"2024-02-17T11:55:58.690353+0000","last_became_peered":"2024-02-17T11:55:58.690353+0000","last_unstale":"2024-02-17T11:56:59.732852+0000","last_undegraded":"2024-02-17T11:56:59.732852+0000","last_fullsized":"2024-02-17T11:56:59.732852+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:45:33.830784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733040+0000","last_change":"2024-02-17T11:55:57.677649+0000","last_active":"2024-02-17T11:56:59.733040+0000","last_peered":"2024-02-17T11:56:59.733040+0000","last_clean":"2024-02-17T11:56:59.733040+0000","last_became_active":"2024-02-17T11:55:57.677331+0000","last_became_peered":"2024-02-17T11:55:57.677331+0000","last_unstale":"2024-02-17T11:56:59.733040+0000","last_undegraded":"2024-02-17T11:56:59.733040+0000","last_fullsized":"2024-02-17T11:56:59.733040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:06.932255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728982+0000","last_change":"2024-02-17T11:55:58.688575+0000","last_active":"2024-02-17T11:56:59.728982+0000","last_peered":"2024-02-17T11:56:59.728982+0000","last_clean":"2024-02-17T11:56:59.728982+0000","last_became_active":"2024-02-17T11:55:58.688238+0000","last_became_peered":"2024-02-17T11:55:58.688238+0000","last_unstale":"2024-02-17T11:56:59.728982+0000","last_undegraded":"2024-02-17T11:56:59.728982+0000","last_fullsized":"2024-02-17T11:56:59.728982+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:17:19.774986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729622+0000","last_change":"2024-02-17T11:55:57.679676+0000","last_active":"2024-02-17T11:56:59.729622+0000","last_peered":"2024-02-17T11:56:59.729622+0000","last_clean":"2024-02-17T11:56:59.729622+0000","last_became_active":"2024-02-17T11:55:57.679350+0000","last_became_peered":"2024-02-17T11:55:57.679350+0000","last_unstale":"2024-02-17T11:56:59.729622+0000","last_undegraded":"2024-02-17T11:56:59.729622+0000","last_fullsized":"2024-02-17T11:56:59.729622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:52:45.072896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733182+0000","last_change":"2024-02-17T11:55:58.682212+0000","last_active":"2024-02-17T11:56:59.733182+0000","last_peered":"2024-02-17T11:56:59.733182+0000","last_clean":"2024-02-17T11:56:59.733182+0000","last_became_active":"2024-02-17T11:55:58.681773+0000","last_became_peered":"2024-02-17T11:55:58.681773+0000","last_unstale":"2024-02-17T11:56:59.733182+0000","last_undegraded":"2024-02-17T11:56:59.733182+0000","last_fullsized":"2024-02-17T11:56:59.733182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:28:18.577174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.741288+0000","last_change":"2024-02-17T11:56:59.741288+0000","last_active":"2024-02-17T11:56:59.741288+0000","last_peered":"2024-02-17T11:56:59.741288+0000","last_clean":"2024-02-17T11:56:59.741288+0000","last_became_active":"2024-02-17T11:56:59.734687+0000","last_became_peered":"2024-02-17T11:56:59.734687+0000","last_unstale":"2024-02-17T11:56:59.741288+0000","last_undegraded":"2024-02-17T11:56:59.741288+0000","last_fullsized":"2024-02-17T11:56:59.741288+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:49:15.242774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733252+0000","last_change":"2024-02-17T11:55:58.672771+0000","last_active":"2024-02-17T11:56:59.733252+0000","last_peered":"2024-02-17T11:56:59.733252+0000","last_clean":"2024-02-17T11:56:59.733252+0000","last_became_active":"2024-02-17T11:55:58.671877+0000","last_became_peered":"2024-02-17T11:55:58.671877+0000","last_unstale":"2024-02-17T11:56:59.733252+0000","last_undegraded":"2024-02-17T11:56:59.733252+0000","last_fullsized":"2024-02-17T11:56:59.733252+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:38:38.350695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":97,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:09.256539+0000","last_change":"2024-02-17T11:55:57.669396+0000","last_active":"2024-02-17T11:57:09.256539+0000","last_peered":"2024-02-17T11:57:09.256539+0000","last_clean":"2024-02-17T11:57:09.256539+0000","last_became_active":"2024-02-17T11:55:57.669116+0000","last_became_peered":"2024-02-17T11:55:57.669116+0000","last_unstale":"2024-02-17T11:57:09.256539+0000","last_undegraded":"2024-02-17T11:57:09.256539+0000","last_fullsized":"2024-02-17T11:57:09.256539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:05:11.081309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729167+0000","last_change":"2024-02-17T11:55:58.674379+0000","last_active":"2024-02-17T11:56:59.729167+0000","last_peered":"2024-02-17T11:56:59.729167+0000","last_clean":"2024-02-17T11:56:59.729167+0000","last_became_active":"2024-02-17T11:55:58.674015+0000","last_became_peered":"2024-02-17T11:55:58.674015+0000","last_unstale":"2024-02-17T11:56:59.729167+0000","last_undegraded":"2024-02-17T11:56:59.729167+0000","last_fullsized":"2024-02-17T11:56:59.729167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:44:20.410623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.734826+0000","last_change":"2024-02-17T11:55:57.668436+0000","last_active":"2024-02-17T11:56:59.734826+0000","last_peered":"2024-02-17T11:56:59.734826+0000","last_clean":"2024-02-17T11:56:59.734826+0000","last_became_active":"2024-02-17T11:55:57.668004+0000","last_became_peered":"2024-02-17T11:55:57.668004+0000","last_unstale":"2024-02-17T11:56:59.734826+0000","last_undegraded":"2024-02-17T11:56:59.734826+0000","last_fullsized":"2024-02-17T11:56:59.734826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:13:42.566644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732981+0000","last_change":"2024-02-17T11:55:58.686888+0000","last_active":"2024-02-17T11:56:59.732981+0000","last_peered":"2024-02-17T11:56:59.732981+0000","last_clean":"2024-02-17T11:56:59.732981+0000","last_became_active":"2024-02-17T11:55:58.686376+0000","last_became_peered":"2024-02-17T11:55:58.686376+0000","last_unstale":"2024-02-17T11:56:59.732981+0000","last_undegraded":"2024-02-17T11:56:59.732981+0000","last_fullsized":"2024-02-17T11:56:59.732981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:58:48.292354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731458+0000","last_change":"2024-02-17T11:55:57.665717+0000","last_active":"2024-02-17T11:56:59.731458+0000","last_peered":"2024-02-17T11:56:59.731458+0000","last_clean":"2024-02-17T11:56:59.731458+0000","last_became_active":"2024-02-17T11:55:57.665381+0000","last_became_peered":"2024-02-17T11:55:57.665381+0000","last_unstale":"2024-02-17T11:56:59.731458+0000","last_undegraded":"2024-02-17T11:56:59.731458+0000","last_fullsized":"2024-02-17T11:56:59.731458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:15:27.424332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732499+0000","last_change":"2024-02-17T11:55:58.673261+0000","last_active":"2024-02-17T11:56:59.732499+0000","last_peered":"2024-02-17T11:56:59.732499+0000","last_clean":"2024-02-17T11:56:59.732499+0000","last_became_active":"2024-02-17T11:55:58.672912+0000","last_became_peered":"2024-02-17T11:55:58.672912+0000","last_unstale":"2024-02-17T11:56:59.732499+0000","last_undegraded":"2024-02-17T11:56:59.732499+0000","last_fullsized":"2024-02-17T11:56:59.732499+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:42.605291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732598+0000","last_change":"2024-02-17T11:55:57.667792+0000","last_active":"2024-02-17T11:56:59.732598+0000","last_peered":"2024-02-17T11:56:59.732598+0000","last_clean":"2024-02-17T11:56:59.732598+0000","last_became_active":"2024-02-17T11:55:57.667361+0000","last_became_peered":"2024-02-17T11:55:57.667361+0000","last_unstale":"2024-02-17T11:56:59.732598+0000","last_undegraded":"2024-02-17T11:56:59.732598+0000","last_fullsized":"2024-02-17T11:56:59.732598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:48.073742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731060+0000","last_change":"2024-02-17T11:55:58.672180+0000","last_active":"2024-02-17T11:56:59.731060+0000","last_peered":"2024-02-17T11:56:59.731060+0000","last_clean":"2024-02-17T11:56:59.731060+0000","last_became_active":"2024-02-17T11:55:58.671790+0000","last_became_peered":"2024-02-17T11:55:58.671790+0000","last_unstale":"2024-02-17T11:56:59.731060+0000","last_undegraded":"2024-02-17T11:56:59.731060+0000","last_fullsized":"2024-02-17T11:56:59.731060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:48:39.982924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":94,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:12.260433+0000","last_change":"2024-02-17T11:55:57.670085+0000","last_active":"2024-02-17T11:57:12.260433+0000","last_peered":"2024-02-17T11:57:12.260433+0000","last_clean":"2024-02-17T11:57:12.260433+0000","last_became_active":"2024-02-17T11:55:57.669648+0000","last_became_peered":"2024-02-17T11:55:57.669648+0000","last_unstale":"2024-02-17T11:57:12.260433+0000","last_undegraded":"2024-02-17T11:57:12.260433+0000","last_fullsized":"2024-02-17T11:57:12.260433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:31:34.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732608+0000","last_change":"2024-02-17T11:55:58.685080+0000","last_active":"2024-02-17T11:56:59.732608+0000","last_peered":"2024-02-17T11:56:59.732608+0000","last_clean":"2024-02-17T11:56:59.732608+0000","last_became_active":"2024-02-17T11:55:58.684763+0000","last_became_peered":"2024-02-17T11:55:58.684763+0000","last_unstale":"2024-02-17T11:56:59.732608+0000","last_undegraded":"2024-02-17T11:56:59.732608+0000","last_fullsized":"2024-02-17T11:56:59.732608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:59:53.763492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732988+0000","last_change":"2024-02-17T11:55:57.676391+0000","last_active":"2024-02-17T11:56:59.732988+0000","last_peered":"2024-02-17T11:56:59.732988+0000","last_clean":"2024-02-17T11:56:59.732988+0000","last_became_active":"2024-02-17T11:55:57.676103+0000","last_became_peered":"2024-02-17T11:55:57.676103+0000","last_unstale":"2024-02-17T11:56:59.732988+0000","last_undegraded":"2024-02-17T11:56:59.732988+0000","last_fullsized":"2024-02-17T11:56:59.732988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:37:52.866379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728974+0000","last_change":"2024-02-17T11:55:58.679274+0000","last_active":"2024-02-17T11:56:59.728974+0000","last_peered":"2024-02-17T11:56:59.728974+0000","last_clean":"2024-02-17T11:56:59.728974+0000","last_became_active":"2024-02-17T11:55:58.678945+0000","last_became_peered":"2024-02-17T11:55:58.678945+0000","last_unstale":"2024-02-17T11:56:59.728974+0000","last_undegraded":"2024-02-17T11:56:59.728974+0000","last_fullsized":"2024-02-17T11:56:59.728974+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:12:20.502503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733259+0000","last_change":"2024-02-17T11:55:57.672732+0000","last_active":"2024-02-17T11:56:59.733259+0000","last_peered":"2024-02-17T11:56:59.733259+0000","last_clean":"2024-02-17T11:56:59.733259+0000","last_became_active":"2024-02-17T11:55:57.672409+0000","last_became_peered":"2024-02-17T11:55:57.672409+0000","last_unstale":"2024-02-17T11:56:59.733259+0000","last_undegraded":"2024-02-17T11:56:59.733259+0000","last_fullsized":"2024-02-17T11:56:59.733259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:07:31.556508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733084+0000","last_change":"2024-02-17T11:55:58.689983+0000","last_active":"2024-02-17T11:56:59.733084+0000","last_peered":"2024-02-17T11:56:59.733084+0000","last_clean":"2024-02-17T11:56:59.733084+0000","last_became_active":"2024-02-17T11:55:58.689629+0000","last_became_peered":"2024-02-17T11:55:58.689629+0000","last_unstale":"2024-02-17T11:56:59.733084+0000","last_undegraded":"2024-02-17T11:56:59.733084+0000","last_fullsized":"2024-02-17T11:56:59.733084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:12.321746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'28","reported_seq":199,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:13.280852+0000","last_change":"2024-02-17T11:55:57.673273+0000","last_active":"2024-02-17T11:57:13.280852+0000","last_peered":"2024-02-17T11:57:13.280852+0000","last_clean":"2024-02-17T11:57:13.280852+0000","last_became_active":"2024-02-17T11:55:57.672865+0000","last_became_peered":"2024-02-17T11:55:57.672865+0000","last_unstale":"2024-02-17T11:57:13.280852+0000","last_undegraded":"2024-02-17T11:57:13.280852+0000","last_fullsized":"2024-02-17T11:57:13.280852+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:31:01.593798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":152,"num_read_kb":135,"num_write":29,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729395+0000","last_change":"2024-02-17T11:55:58.669296+0000","last_active":"2024-02-17T11:56:59.729395+0000","last_peered":"2024-02-17T11:56:59.729395+0000","last_clean":"2024-02-17T11:56:59.729395+0000","last_became_active":"2024-02-17T11:55:58.668927+0000","last_became_peered":"2024-02-17T11:55:58.668927+0000","last_unstale":"2024-02-17T11:56:59.729395+0000","last_undegraded":"2024-02-17T11:56:59.729395+0000","last_fullsized":"2024-02-17T11:56:59.729395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:25.177905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733466+0000","last_change":"2024-02-17T11:55:57.674523+0000","last_active":"2024-02-17T11:56:59.733466+0000","last_peered":"2024-02-17T11:56:59.733466+0000","last_clean":"2024-02-17T11:56:59.733466+0000","last_became_active":"2024-02-17T11:55:57.672375+0000","last_became_peered":"2024-02-17T11:55:57.672375+0000","last_unstale":"2024-02-17T11:56:59.733466+0000","last_undegraded":"2024-02-17T11:56:59.733466+0000","last_fullsized":"2024-02-17T11:56:59.733466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:15:21.568434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731915+0000","last_change":"2024-02-17T11:55:58.682106+0000","last_active":"2024-02-17T11:56:59.731915+0000","last_peered":"2024-02-17T11:56:59.731915+0000","last_clean":"2024-02-17T11:56:59.731915+0000","last_became_active":"2024-02-17T11:55:58.681420+0000","last_became_peered":"2024-02-17T11:55:58.681420+0000","last_unstale":"2024-02-17T11:56:59.731915+0000","last_undegraded":"2024-02-17T11:56:59.731915+0000","last_fullsized":"2024-02-17T11:56:59.731915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:23:00.521690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732537+0000","last_change":"2024-02-17T11:55:57.675161+0000","last_active":"2024-02-17T11:56:59.732537+0000","last_peered":"2024-02-17T11:56:59.732537+0000","last_clean":"2024-02-17T11:56:59.732537+0000","last_became_active":"2024-02-17T11:55:57.674776+0000","last_became_peered":"2024-02-17T11:55:57.674776+0000","last_unstale":"2024-02-17T11:56:59.732537+0000","last_undegraded":"2024-02-17T11:56:59.732537+0000","last_fullsized":"2024-02-17T11:56:59.732537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:25:59.695014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732385+0000","last_change":"2024-02-17T11:55:58.679319+0000","last_active":"2024-02-17T11:56:59.732385+0000","last_peered":"2024-02-17T11:56:59.732385+0000","last_clean":"2024-02-17T11:56:59.732385+0000","last_became_active":"2024-02-17T11:55:58.679001+0000","last_became_peered":"2024-02-17T11:55:58.679001+0000","last_unstale":"2024-02-17T11:56:59.732385+0000","last_undegraded":"2024-02-17T11:56:59.732385+0000","last_fullsized":"2024-02-17T11:56:59.732385+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:36:48.493338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'4","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:10.257408+0000","last_change":"2024-02-17T11:55:57.670027+0000","last_active":"2024-02-17T11:57:10.257408+0000","last_peered":"2024-02-17T11:57:10.257408+0000","last_clean":"2024-02-17T11:57:10.257408+0000","last_became_active":"2024-02-17T11:55:57.669692+0000","last_became_peered":"2024-02-17T11:55:57.669692+0000","last_unstale":"2024-02-17T11:57:10.257408+0000","last_undegraded":"2024-02-17T11:57:10.257408+0000","last_fullsized":"2024-02-17T11:57:10.257408+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:44:01.001640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731924+0000","last_change":"2024-02-17T11:55:58.672811+0000","last_active":"2024-02-17T11:56:59.731924+0000","last_peered":"2024-02-17T11:56:59.731924+0000","last_clean":"2024-02-17T11:56:59.731924+0000","last_became_active":"2024-02-17T11:55:58.672469+0000","last_became_peered":"2024-02-17T11:55:58.672469+0000","last_unstale":"2024-02-17T11:56:59.731924+0000","last_undegraded":"2024-02-17T11:56:59.731924+0000","last_fullsized":"2024-02-17T11:56:59.731924+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:07.451422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729069+0000","last_change":"2024-02-17T11:55:57.668031+0000","last_active":"2024-02-17T11:56:59.729069+0000","last_peered":"2024-02-17T11:56:59.729069+0000","last_clean":"2024-02-17T11:56:59.729069+0000","last_became_active":"2024-02-17T11:55:57.667344+0000","last_became_peered":"2024-02-17T11:55:57.667344+0000","last_unstale":"2024-02-17T11:56:59.729069+0000","last_undegraded":"2024-02-17T11:56:59.729069+0000","last_fullsized":"2024-02-17T11:56:59.729069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:14:20.773902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732837+0000","last_change":"2024-02-17T11:55:58.682013+0000","last_active":"2024-02-17T11:56:59.732837+0000","last_peered":"2024-02-17T11:56:59.732837+0000","last_clean":"2024-02-17T11:56:59.732837+0000","last_became_active":"2024-02-17T11:55:58.681503+0000","last_became_peered":"2024-02-17T11:55:58.681503+0000","last_unstale":"2024-02-17T11:56:59.732837+0000","last_undegraded":"2024-02-17T11:56:59.732837+0000","last_fullsized":"2024-02-17T11:56:59.732837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:24:50.364090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729587+0000","last_change":"2024-02-17T11:55:57.666725+0000","last_active":"2024-02-17T11:56:59.729587+0000","last_peered":"2024-02-17T11:56:59.729587+0000","last_clean":"2024-02-17T11:56:59.729587+0000","last_became_active":"2024-02-17T11:55:57.666339+0000","last_became_peered":"2024-02-17T11:55:57.666339+0000","last_unstale":"2024-02-17T11:56:59.729587+0000","last_undegraded":"2024-02-17T11:56:59.729587+0000","last_fullsized":"2024-02-17T11:56:59.729587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:22:47.324016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730770+0000","last_change":"2024-02-17T11:55:58.675995+0000","last_active":"2024-02-17T11:56:59.730770+0000","last_peered":"2024-02-17T11:56:59.730770+0000","last_clean":"2024-02-17T11:56:59.730770+0000","last_became_active":"2024-02-17T11:55:58.675655+0000","last_became_peered":"2024-02-17T11:55:58.675655+0000","last_unstale":"2024-02-17T11:56:59.730770+0000","last_undegraded":"2024-02-17T11:56:59.730770+0000","last_fullsized":"2024-02-17T11:56:59.730770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:12.465300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730891+0000","last_change":"2024-02-17T11:55:57.669500+0000","last_active":"2024-02-17T11:56:59.730891+0000","last_peered":"2024-02-17T11:56:59.730891+0000","last_clean":"2024-02-17T11:56:59.730891+0000","last_became_active":"2024-02-17T11:55:57.669148+0000","last_became_peered":"2024-02-17T11:55:57.669148+0000","last_unstale":"2024-02-17T11:56:59.730891+0000","last_undegraded":"2024-02-17T11:56:59.730891+0000","last_fullsized":"2024-02-17T11:56:59.730891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:33.821266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.735074+0000","last_change":"2024-02-17T11:55:58.678336+0000","last_active":"2024-02-17T11:56:59.735074+0000","last_peered":"2024-02-17T11:56:59.735074+0000","last_clean":"2024-02-17T11:56:59.735074+0000","last_became_active":"2024-02-17T11:55:58.677967+0000","last_became_peered":"2024-02-17T11:55:58.677967+0000","last_unstale":"2024-02-17T11:56:59.735074+0000","last_undegraded":"2024-02-17T11:56:59.735074+0000","last_fullsized":"2024-02-17T11:56:59.735074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:31:39.406673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732692+0000","last_change":"2024-02-17T11:55:57.674810+0000","last_active":"2024-02-17T11:56:59.732692+0000","last_peered":"2024-02-17T11:56:59.732692+0000","last_clean":"2024-02-17T11:56:59.732692+0000","last_became_active":"2024-02-17T11:55:57.674356+0000","last_became_peered":"2024-02-17T11:55:57.674356+0000","last_unstale":"2024-02-17T11:56:59.732692+0000","last_undegraded":"2024-02-17T11:56:59.732692+0000","last_fullsized":"2024-02-17T11:56:59.732692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:25:57.879239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.735206+0000","last_change":"2024-02-17T11:55:58.680928+0000","last_active":"2024-02-17T11:56:59.735206+0000","last_peered":"2024-02-17T11:56:59.735206+0000","last_clean":"2024-02-17T11:56:59.735206+0000","last_became_active":"2024-02-17T11:55:58.680521+0000","last_became_peered":"2024-02-17T11:55:58.680521+0000","last_unstale":"2024-02-17T11:56:59.735206+0000","last_undegraded":"2024-02-17T11:56:59.735206+0000","last_fullsized":"2024-02-17T11:56:59.735206+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:15:15.739127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732913+0000","last_change":"2024-02-17T11:55:58.674771+0000","last_active":"2024-02-17T11:56:59.732913+0000","last_peered":"2024-02-17T11:56:59.732913+0000","last_clean":"2024-02-17T11:56:59.732913+0000","last_became_active":"2024-02-17T11:55:58.674450+0000","last_became_peered":"2024-02-17T11:55:58.674450+0000","last_unstale":"2024-02-17T11:56:59.732913+0000","last_undegraded":"2024-02-17T11:56:59.732913+0000","last_fullsized":"2024-02-17T11:56:59.732913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.031576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":93,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:10.256629+0000","last_change":"2024-02-17T11:55:57.672868+0000","last_active":"2024-02-17T11:57:10.256629+0000","last_peered":"2024-02-17T11:57:10.256629+0000","last_clean":"2024-02-17T11:57:10.256629+0000","last_became_active":"2024-02-17T11:55:57.672584+0000","last_became_peered":"2024-02-17T11:55:57.672584+0000","last_unstale":"2024-02-17T11:57:10.256629+0000","last_undegraded":"2024-02-17T11:57:10.256629+0000","last_fullsized":"2024-02-17T11:57:10.256629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:03:20.307036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":70,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732918+0000","last_change":"2024-02-17T11:55:58.692594+0000","last_active":"2024-02-17T11:56:59.732918+0000","last_peered":"2024-02-17T11:56:59.732918+0000","last_clean":"2024-02-17T11:56:59.732918+0000","last_became_active":"2024-02-17T11:55:58.692258+0000","last_became_peered":"2024-02-17T11:55:58.692258+0000","last_unstale":"2024-02-17T11:56:59.732918+0000","last_undegraded":"2024-02-17T11:56:59.732918+0000","last_fullsized":"2024-02-17T11:56:59.732918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:28.497939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.880997+0000","last_change":"2024-02-17T11:55:57.669456+0000","last_active":"2024-02-17T11:56:59.880997+0000","last_peered":"2024-02-17T11:56:59.880997+0000","last_clean":"2024-02-17T11:56:59.880997+0000","last_became_active":"2024-02-17T11:55:57.669067+0000","last_became_peered":"2024-02-17T11:55:57.669067+0000","last_unstale":"2024-02-17T11:56:59.880997+0000","last_undegraded":"2024-02-17T11:56:59.880997+0000","last_fullsized":"2024-02-17T11:56:59.880997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:12.352296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":353,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732072+0000","last_change":"2024-02-17T11:54:46.125844+0000","last_active":"2024-02-17T11:56:59.732072+0000","last_peered":"2024-02-17T11:56:59.732072+0000","last_clean":"2024-02-17T11:56:59.732072+0000","last_became_active":"2024-02-17T11:54:46.125387+0000","last_became_peered":"2024-02-17T11:54:46.125387+0000","last_unstale":"2024-02-17T11:56:59.732072+0000","last_undegraded":"2024-02-17T11:56:59.732072+0000","last_fullsized":"2024-02-17T11:56:59.732072+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_clean_scrub_stamp":"2024-02-17T11:51:21.581371+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:53:27.219974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731217+0000","last_change":"2024-02-17T11:55:58.681623+0000","last_active":"2024-02-17T11:56:59.731217+0000","last_peered":"2024-02-17T11:56:59.731217+0000","last_clean":"2024-02-17T11:56:59.731217+0000","last_became_active":"2024-02-17T11:55:58.681275+0000","last_became_peered":"2024-02-17T11:55:58.681275+0000","last_unstale":"2024-02-17T11:56:59.731217+0000","last_undegraded":"2024-02-17T11:56:59.731217+0000","last_fullsized":"2024-02-17T11:56:59.731217+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:20.685634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732237+0000","last_change":"2024-02-17T11:55:57.679555+0000","last_active":"2024-02-17T11:56:59.732237+0000","last_peered":"2024-02-17T11:56:59.732237+0000","last_clean":"2024-02-17T11:56:59.732237+0000","last_became_active":"2024-02-17T11:55:57.679241+0000","last_became_peered":"2024-02-17T11:55:57.679241+0000","last_unstale":"2024-02-17T11:56:59.732237+0000","last_undegraded":"2024-02-17T11:56:59.732237+0000","last_fullsized":"2024-02-17T11:56:59.732237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:37.482582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729501+0000","last_change":"2024-02-17T11:55:57.679711+0000","last_active":"2024-02-17T11:56:59.729501+0000","last_peered":"2024-02-17T11:56:59.729501+0000","last_clean":"2024-02-17T11:56:59.729501+0000","last_became_active":"2024-02-17T11:55:57.679415+0000","last_became_peered":"2024-02-17T11:55:57.679415+0000","last_unstale":"2024-02-17T11:56:59.729501+0000","last_undegraded":"2024-02-17T11:56:59.729501+0000","last_fullsized":"2024-02-17T11:56:59.729501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:13.095530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731646+0000","last_change":"2024-02-17T11:55:58.670275+0000","last_active":"2024-02-17T11:56:59.731646+0000","last_peered":"2024-02-17T11:56:59.731646+0000","last_clean":"2024-02-17T11:56:59.731646+0000","last_became_active":"2024-02-17T11:55:58.669926+0000","last_became_peered":"2024-02-17T11:55:58.669926+0000","last_unstale":"2024-02-17T11:56:59.731646+0000","last_undegraded":"2024-02-17T11:56:59.731646+0000","last_fullsized":"2024-02-17T11:56:59.731646+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:48:35.078389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730945+0000","last_change":"2024-02-17T11:55:57.670364+0000","last_active":"2024-02-17T11:56:59.730945+0000","last_peered":"2024-02-17T11:56:59.730945+0000","last_clean":"2024-02-17T11:56:59.730945+0000","last_became_active":"2024-02-17T11:55:57.669977+0000","last_became_peered":"2024-02-17T11:55:57.669977+0000","last_unstale":"2024-02-17T11:56:59.730945+0000","last_undegraded":"2024-02-17T11:56:59.730945+0000","last_fullsized":"2024-02-17T11:56:59.730945+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:42:13.820390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732778+0000","last_change":"2024-02-17T11:55:58.681587+0000","last_active":"2024-02-17T11:56:59.732778+0000","last_peered":"2024-02-17T11:56:59.732778+0000","last_clean":"2024-02-17T11:56:59.732778+0000","last_became_active":"2024-02-17T11:55:58.680024+0000","last_became_peered":"2024-02-17T11:55:58.680024+0000","last_unstale":"2024-02-17T11:56:59.732778+0000","last_undegraded":"2024-02-17T11:56:59.732778+0000","last_fullsized":"2024-02-17T11:56:59.732778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:31:40.399439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.881143+0000","last_change":"2024-02-17T11:55:57.674618+0000","last_active":"2024-02-17T11:56:59.881143+0000","last_peered":"2024-02-17T11:56:59.881143+0000","last_clean":"2024-02-17T11:56:59.881143+0000","last_became_active":"2024-02-17T11:55:57.674246+0000","last_became_peered":"2024-02-17T11:55:57.674246+0000","last_unstale":"2024-02-17T11:56:59.881143+0000","last_undegraded":"2024-02-17T11:56:59.881143+0000","last_fullsized":"2024-02-17T11:56:59.881143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:42:26.681061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.734355+0000","last_change":"2024-02-17T11:56:59.734355+0000","last_active":"2024-02-17T11:56:59.734355+0000","last_peered":"2024-02-17T11:56:59.734355+0000","last_clean":"2024-02-17T11:56:59.734355+0000","last_became_active":"2024-02-17T11:56:59.733975+0000","last_became_peered":"2024-02-17T11:56:59.733975+0000","last_unstale":"2024-02-17T11:56:59.734355+0000","last_undegraded":"2024-02-17T11:56:59.734355+0000","last_fullsized":"2024-02-17T11:56:59.734355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:37:51.426702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732149+0000","last_change":"2024-02-17T11:55:57.667998+0000","last_active":"2024-02-17T11:56:59.732149+0000","last_peered":"2024-02-17T11:56:59.732149+0000","last_clean":"2024-02-17T11:56:59.732149+0000","last_became_active":"2024-02-17T11:55:57.667688+0000","last_became_peered":"2024-02-17T11:55:57.667688+0000","last_unstale":"2024-02-17T11:56:59.732149+0000","last_undegraded":"2024-02-17T11:56:59.732149+0000","last_fullsized":"2024-02-17T11:56:59.732149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:07:19.782879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729069+0000","last_change":"2024-02-17T11:55:58.676644+0000","last_active":"2024-02-17T11:56:59.729069+0000","last_peered":"2024-02-17T11:56:59.729069+0000","last_clean":"2024-02-17T11:56:59.729069+0000","last_became_active":"2024-02-17T11:55:58.674670+0000","last_became_peered":"2024-02-17T11:55:58.674670+0000","last_unstale":"2024-02-17T11:56:59.729069+0000","last_undegraded":"2024-02-17T11:56:59.729069+0000","last_fullsized":"2024-02-17T11:56:59.729069+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:43:22.746537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732754+0000","last_change":"2024-02-17T11:55:57.671330+0000","last_active":"2024-02-17T11:56:59.732754+0000","last_peered":"2024-02-17T11:56:59.732754+0000","last_clean":"2024-02-17T11:56:59.732754+0000","last_became_active":"2024-02-17T11:55:57.670902+0000","last_became_peered":"2024-02-17T11:55:57.670902+0000","last_unstale":"2024-02-17T11:56:59.732754+0000","last_undegraded":"2024-02-17T11:56:59.732754+0000","last_fullsized":"2024-02-17T11:56:59.732754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:22:14.770339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730447+0000","last_change":"2024-02-17T11:55:58.676417+0000","last_active":"2024-02-17T11:56:59.730447+0000","last_peered":"2024-02-17T11:56:59.730447+0000","last_clean":"2024-02-17T11:56:59.730447+0000","last_became_active":"2024-02-17T11:55:58.675960+0000","last_became_peered":"2024-02-17T11:55:58.675960+0000","last_unstale":"2024-02-17T11:56:59.730447+0000","last_undegraded":"2024-02-17T11:56:59.730447+0000","last_fullsized":"2024-02-17T11:56:59.730447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:22:18.566872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.737823+0000","last_change":"2024-02-17T11:56:59.737823+0000","last_active":"2024-02-17T11:56:59.737823+0000","last_peered":"2024-02-17T11:56:59.737823+0000","last_clean":"2024-02-17T11:56:59.737823+0000","last_became_active":"2024-02-17T11:56:59.734043+0000","last_became_peered":"2024-02-17T11:56:59.734043+0000","last_unstale":"2024-02-17T11:56:59.737823+0000","last_undegraded":"2024-02-17T11:56:59.737823+0000","last_fullsized":"2024-02-17T11:56:59.737823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:11:09.171845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732972+0000","last_change":"2024-02-17T11:55:58.690030+0000","last_active":"2024-02-17T11:56:59.732972+0000","last_peered":"2024-02-17T11:56:59.732972+0000","last_clean":"2024-02-17T11:56:59.732972+0000","last_became_active":"2024-02-17T11:55:58.689687+0000","last_became_peered":"2024-02-17T11:55:58.689687+0000","last_unstale":"2024-02-17T11:56:59.732972+0000","last_undegraded":"2024-02-17T11:56:59.732972+0000","last_fullsized":"2024-02-17T11:56:59.732972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:33:43.175234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729187+0000","last_change":"2024-02-17T11:55:57.679734+0000","last_active":"2024-02-17T11:56:59.729187+0000","last_peered":"2024-02-17T11:56:59.729187+0000","last_clean":"2024-02-17T11:56:59.729187+0000","last_became_active":"2024-02-17T11:55:57.679328+0000","last_became_peered":"2024-02-17T11:55:57.679328+0000","last_unstale":"2024-02-17T11:56:59.729187+0000","last_undegraded":"2024-02-17T11:56:59.729187+0000","last_fullsized":"2024-02-17T11:56:59.729187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:50:09.813482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732863+0000","last_change":"2024-02-17T11:55:58.688360+0000","last_active":"2024-02-17T11:56:59.732863+0000","last_peered":"2024-02-17T11:56:59.732863+0000","last_clean":"2024-02-17T11:56:59.732863+0000","last_became_active":"2024-02-17T11:55:58.687994+0000","last_became_peered":"2024-02-17T11:55:58.687994+0000","last_unstale":"2024-02-17T11:56:59.732863+0000","last_undegraded":"2024-02-17T11:56:59.732863+0000","last_fullsized":"2024-02-17T11:56:59.732863+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:01:02.163238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733202+0000","last_change":"2024-02-17T11:55:57.675977+0000","last_active":"2024-02-17T11:56:59.733202+0000","last_peered":"2024-02-17T11:56:59.733202+0000","last_clean":"2024-02-17T11:56:59.733202+0000","last_became_active":"2024-02-17T11:55:57.675250+0000","last_became_peered":"2024-02-17T11:55:57.675250+0000","last_unstale":"2024-02-17T11:56:59.733202+0000","last_undegraded":"2024-02-17T11:56:59.733202+0000","last_fullsized":"2024-02-17T11:56:59.733202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:11:27.611692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732873+0000","last_change":"2024-02-17T11:55:58.688593+0000","last_active":"2024-02-17T11:56:59.732873+0000","last_peered":"2024-02-17T11:56:59.732873+0000","last_clean":"2024-02-17T11:56:59.732873+0000","last_became_active":"2024-02-17T11:55:58.688163+0000","last_became_peered":"2024-02-17T11:55:58.688163+0000","last_unstale":"2024-02-17T11:56:59.732873+0000","last_undegraded":"2024-02-17T11:56:59.732873+0000","last_fullsized":"2024-02-17T11:56:59.732873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:54.597116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733115+0000","last_change":"2024-02-17T11:55:57.676860+0000","last_active":"2024-02-17T11:56:59.733115+0000","last_peered":"2024-02-17T11:56:59.733115+0000","last_clean":"2024-02-17T11:56:59.733115+0000","last_became_active":"2024-02-17T11:55:57.676534+0000","last_became_peered":"2024-02-17T11:55:57.676534+0000","last_unstale":"2024-02-17T11:56:59.733115+0000","last_undegraded":"2024-02-17T11:56:59.733115+0000","last_fullsized":"2024-02-17T11:56:59.733115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:23:47.726264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.739615+0000","last_change":"2024-02-17T11:56:59.739615+0000","last_active":"2024-02-17T11:56:59.739615+0000","last_peered":"2024-02-17T11:56:59.739615+0000","last_clean":"2024-02-17T11:56:59.739615+0000","last_became_active":"2024-02-17T11:56:59.736220+0000","last_became_peered":"2024-02-17T11:56:59.736220+0000","last_unstale":"2024-02-17T11:56:59.739615+0000","last_undegraded":"2024-02-17T11:56:59.739615+0000","last_fullsized":"2024-02-17T11:56:59.739615+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:22:07.463713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.739390+0000","last_change":"2024-02-17T11:56:59.739390+0000","last_active":"2024-02-17T11:56:59.739390+0000","last_peered":"2024-02-17T11:56:59.739390+0000","last_clean":"2024-02-17T11:56:59.739390+0000","last_became_active":"2024-02-17T11:56:59.734852+0000","last_became_peered":"2024-02-17T11:56:59.734852+0000","last_unstale":"2024-02-17T11:56:59.739390+0000","last_undegraded":"2024-02-17T11:56:59.739390+0000","last_fullsized":"2024-02-17T11:56:59.739390+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:40:09.174557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732359+0000","last_change":"2024-02-17T11:55:58.672985+0000","last_active":"2024-02-17T11:56:59.732359+0000","last_peered":"2024-02-17T11:56:59.732359+0000","last_clean":"2024-02-17T11:56:59.732359+0000","last_became_active":"2024-02-17T11:55:58.672622+0000","last_became_peered":"2024-02-17T11:55:58.672622+0000","last_unstale":"2024-02-17T11:56:59.732359+0000","last_undegraded":"2024-02-17T11:56:59.732359+0000","last_fullsized":"2024-02-17T11:56:59.732359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:45.555827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730520+0000","last_change":"2024-02-17T11:55:58.676127+0000","last_active":"2024-02-17T11:56:59.730520+0000","last_peered":"2024-02-17T11:56:59.730520+0000","last_clean":"2024-02-17T11:56:59.730520+0000","last_became_active":"2024-02-17T11:55:58.675784+0000","last_became_peered":"2024-02-17T11:55:58.675784+0000","last_unstale":"2024-02-17T11:56:59.730520+0000","last_undegraded":"2024-02-17T11:56:59.730520+0000","last_fullsized":"2024-02-17T11:56:59.730520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:56:32.648803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733209+0000","last_change":"2024-02-17T11:55:58.682438+0000","last_active":"2024-02-17T11:56:59.733209+0000","last_peered":"2024-02-17T11:56:59.733209+0000","last_clean":"2024-02-17T11:56:59.733209+0000","last_became_active":"2024-02-17T11:55:58.682067+0000","last_became_peered":"2024-02-17T11:55:58.682067+0000","last_unstale":"2024-02-17T11:56:59.733209+0000","last_undegraded":"2024-02-17T11:56:59.733209+0000","last_fullsized":"2024-02-17T11:56:59.733209+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:10:22.906450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733833+0000","last_change":"2024-02-17T11:55:58.686671+0000","last_active":"2024-02-17T11:56:59.733833+0000","last_peered":"2024-02-17T11:56:59.733833+0000","last_clean":"2024-02-17T11:56:59.733833+0000","last_became_active":"2024-02-17T11:55:58.686315+0000","last_became_peered":"2024-02-17T11:55:58.686315+0000","last_unstale":"2024-02-17T11:56:59.733833+0000","last_undegraded":"2024-02-17T11:56:59.733833+0000","last_fullsized":"2024-02-17T11:56:59.733833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:55:40.864847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.728763+0000","last_change":"2024-02-17T11:55:58.684227+0000","last_active":"2024-02-17T11:56:59.728763+0000","last_peered":"2024-02-17T11:56:59.728763+0000","last_clean":"2024-02-17T11:56:59.728763+0000","last_became_active":"2024-02-17T11:55:58.683875+0000","last_became_peered":"2024-02-17T11:55:58.683875+0000","last_unstale":"2024-02-17T11:56:59.728763+0000","last_undegraded":"2024-02-17T11:56:59.728763+0000","last_fullsized":"2024-02-17T11:56:59.728763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:11:41.283161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731741+0000","last_change":"2024-02-17T11:55:58.685964+0000","last_active":"2024-02-17T11:56:59.731741+0000","last_peered":"2024-02-17T11:56:59.731741+0000","last_clean":"2024-02-17T11:56:59.731741+0000","last_became_active":"2024-02-17T11:55:58.685350+0000","last_became_peered":"2024-02-17T11:55:58.685350+0000","last_unstale":"2024-02-17T11:56:59.731741+0000","last_undegraded":"2024-02-17T11:56:59.731741+0000","last_fullsized":"2024-02-17T11:56:59.731741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:22:34.600604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730677+0000","last_change":"2024-02-17T11:55:58.671291+0000","last_active":"2024-02-17T11:56:59.730677+0000","last_peered":"2024-02-17T11:56:59.730677+0000","last_clean":"2024-02-17T11:56:59.730677+0000","last_became_active":"2024-02-17T11:55:58.670925+0000","last_became_peered":"2024-02-17T11:55:58.670925+0000","last_unstale":"2024-02-17T11:56:59.730677+0000","last_undegraded":"2024-02-17T11:56:59.730677+0000","last_fullsized":"2024-02-17T11:56:59.730677+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:26:03.120976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.734015+0000","last_change":"2024-02-17T11:55:58.673122+0000","last_active":"2024-02-17T11:56:59.734015+0000","last_peered":"2024-02-17T11:56:59.734015+0000","last_clean":"2024-02-17T11:56:59.734015+0000","last_became_active":"2024-02-17T11:55:58.672747+0000","last_became_peered":"2024-02-17T11:55:58.672747+0000","last_unstale":"2024-02-17T11:56:59.734015+0000","last_undegraded":"2024-02-17T11:56:59.734015+0000","last_fullsized":"2024-02-17T11:56:59.734015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:55:15.064073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732668+0000","last_change":"2024-02-17T11:55:58.681276+0000","last_active":"2024-02-17T11:56:59.732668+0000","last_peered":"2024-02-17T11:56:59.732668+0000","last_clean":"2024-02-17T11:56:59.732668+0000","last_became_active":"2024-02-17T11:55:58.680889+0000","last_became_peered":"2024-02-17T11:55:58.680889+0000","last_unstale":"2024-02-17T11:56:59.732668+0000","last_undegraded":"2024-02-17T11:56:59.732668+0000","last_fullsized":"2024-02-17T11:56:59.732668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:19:34.330288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732681+0000","last_change":"2024-02-17T11:55:58.667475+0000","last_active":"2024-02-17T11:56:59.732681+0000","last_peered":"2024-02-17T11:56:59.732681+0000","last_clean":"2024-02-17T11:56:59.732681+0000","last_became_active":"2024-02-17T11:55:58.666577+0000","last_became_peered":"2024-02-17T11:55:58.666577+0000","last_unstale":"2024-02-17T11:56:59.732681+0000","last_undegraded":"2024-02-17T11:56:59.732681+0000","last_fullsized":"2024-02-17T11:56:59.732681+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:51:35.732035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732861+0000","last_change":"2024-02-17T11:55:58.674557+0000","last_active":"2024-02-17T11:56:59.732861+0000","last_peered":"2024-02-17T11:56:59.732861+0000","last_clean":"2024-02-17T11:56:59.732861+0000","last_became_active":"2024-02-17T11:55:58.673874+0000","last_became_peered":"2024-02-17T11:55:58.673874+0000","last_unstale":"2024-02-17T11:56:59.732861+0000","last_undegraded":"2024-02-17T11:56:59.732861+0000","last_fullsized":"2024-02-17T11:56:59.732861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:18:00.812489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732369+0000","last_change":"2024-02-17T11:55:58.692725+0000","last_active":"2024-02-17T11:56:59.732369+0000","last_peered":"2024-02-17T11:56:59.732369+0000","last_clean":"2024-02-17T11:56:59.732369+0000","last_became_active":"2024-02-17T11:55:58.690880+0000","last_became_peered":"2024-02-17T11:55:58.690880+0000","last_unstale":"2024-02-17T11:56:59.732369+0000","last_undegraded":"2024-02-17T11:56:59.732369+0000","last_fullsized":"2024-02-17T11:56:59.732369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:03:44.724385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731750+0000","last_change":"2024-02-17T11:55:58.679377+0000","last_active":"2024-02-17T11:56:59.731750+0000","last_peered":"2024-02-17T11:56:59.731750+0000","last_clean":"2024-02-17T11:56:59.731750+0000","last_became_active":"2024-02-17T11:55:58.679041+0000","last_became_peered":"2024-02-17T11:55:58.679041+0000","last_unstale":"2024-02-17T11:56:59.731750+0000","last_undegraded":"2024-02-17T11:56:59.731750+0000","last_fullsized":"2024-02-17T11:56:59.731750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:16:37.396424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731242+0000","last_change":"2024-02-17T11:55:58.680665+0000","last_active":"2024-02-17T11:56:59.731242+0000","last_peered":"2024-02-17T11:56:59.731242+0000","last_clean":"2024-02-17T11:56:59.731242+0000","last_became_active":"2024-02-17T11:55:58.680311+0000","last_became_peered":"2024-02-17T11:55:58.680311+0000","last_unstale":"2024-02-17T11:56:59.731242+0000","last_undegraded":"2024-02-17T11:56:59.731242+0000","last_fullsized":"2024-02-17T11:56:59.731242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:19:11.455471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284587,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65438,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":453,"num_read_kb":297,"num_write":157,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":196314,"data_compressed":5598,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383709,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":942080,"data_stored":456292,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.39300000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.70699999999999996}]},{"osd":2,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.49199999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.52000000000000002,"15min":0.52800000000000002},"min":{"1min":0.317,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68899999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.35899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":4,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.504,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.67500000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.372,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78000000000000003,"5min":1.484,"15min":1.484},"last":0.621},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84299999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.55600000000000005}]},{"osd":6,"last update":"Sat Feb 17 11:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82199999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.92300000000000004,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.755}]},{"osd":8,"last update":"Sat Feb 17 11:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.82499999999999996}]},{"osd":9,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.78500000000000003,"15min":0.79400000000000004},"min":{"1min":0.54000000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.89500000000000002,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.73499999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90700000000000003,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.69299999999999995}]},{"osd":10,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.61199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95099999999999996,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0529999999999999,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.77400000000000002}]}]},{"osd":10,"up_from":72,"seq":309237645346,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":1109210,"data_compressed":43889,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.66000000000000003}]},{"osd":1,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.46400000000000002,"15min":0.47499999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.63400000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.46999999999999997,"15min":0.47699999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.60199999999999998,"5min":0.746,"15min":0.746},"last":0.53100000000000003}]},{"osd":2,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.61599999999999999,"15min":0.61799999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81799999999999995,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.59799999999999998},"min":{"1min":0.442,"5min":0.44,"15min":0.44},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.71999999999999997}]},{"osd":3,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81799999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.59999999999999998,"15min":0.61199999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.67600000000000005}]},{"osd":4,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.56299999999999994,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.68500000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.52300000000000002,"15min":0.53600000000000003},"min":{"1min":0.309,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.622,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.50700000000000001}]},{"osd":5,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.50700000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.79600000000000004,"5min":0.871,"15min":0.871},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.67800000000000005,"15min":0.68000000000000005},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.84499999999999997,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.69499999999999995}]},{"osd":6,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.66700000000000004,"15min":0.68100000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73399999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.748},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.82199999999999995,"5min":1.101,"15min":1.101},"last":0.73499999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.41499999999999998}]},{"osd":8,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72199999999999998,"15min":0.72899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.92600000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.74299999999999999,"15min":0.75900000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82799999999999996,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.70799999999999996}]},{"osd":9,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.73599999999999999,"15min":0.754},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.88300000000000001,"5min":1.014,"15min":1.014},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.74399999999999999,"15min":0.76500000000000001},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.86099999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.83699999999999997}]},{"osd":11,"last update":"Sat Feb 17 11:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.56299999999999994,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.84699999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.60999999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.81299999999999994}]}]},{"osd":9,"up_from":67,"seq":287762808870,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":456645,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.41399999999999998,"15min":0.42299999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.504,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.41299999999999998,"15min":0.41899999999999998},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.42199999999999999}]},{"osd":1,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.60599999999999998,"15min":0.621},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.72699999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57499999999999996,"15min":0.58299999999999996},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78300000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.71199999999999997}]},{"osd":2,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.60299999999999998,"15min":0.61499999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.73999999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.55000000000000004,"15min":0.55600000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60399999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.50600000000000001,"15min":0.52300000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.57199999999999995,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.443},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.48999999999999999,"15min":0.50600000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.54800000000000004,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.46200000000000002}]},{"osd":4,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71399999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.66300000000000003,"15min":0.68100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.71499999999999997,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.66700000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.626,"15min":0.64000000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.67600000000000005,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69999999999999996,"15min":0.71999999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81899999999999995,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.69399999999999995}]},{"osd":6,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.59199999999999997,"15min":0.61199999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.64600000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.61799999999999999,"15min":0.63100000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78300000000000003,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.61499999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.73699999999999999,"15min":0.75600000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.78600000000000003,"5min":1.05,"15min":1.05},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.70199999999999996,"15min":0.71699999999999997},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.74399999999999999,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.73899999999999999}]},{"osd":8,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.70899999999999996,"15min":0.72999999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.72399999999999998,"5min":1.016,"15min":1.016},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.71499999999999997,"15min":0.72699999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.86299999999999999,"5min":1.034,"15min":1.034},"last":0.83999999999999997}]},{"osd":10,"last update":"Sat Feb 17 11:56:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.56399999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.0129999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.57699999999999996,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.036,"5min":1.069,"15min":1.069},"last":0.80200000000000005}]}]},{"osd":8,"up_from":59,"seq":253403070507,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":455866,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.44,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70399999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.38100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.69599999999999995}]},{"osd":1,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88900000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.72999999999999998}]},{"osd":2,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46000000000000002,"15min":0.44},"min":{"1min":0.35999999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46899999999999997}]},{"osd":3,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.59199999999999997},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.76200000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.61699999999999999,"15min":0.60799999999999998},"min":{"1min":0.503,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85299999999999998,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.44600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.77100000000000002}]},{"osd":5,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56999999999999995,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73399999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.625}]},{"osd":6,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85099999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.52500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.52300000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.53400000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.84799999999999998}]},{"osd":9,"last update":"Sat Feb 17 11:56:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.67400000000000004,"15min":0.67100000000000004},"min":{"1min":0.55800000000000005,"5min":0.505,"15min":0.505},"max":{"1min":0.82999999999999996,"5min":0.871,"15min":0.871},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66600000000000004,"15min":0.65900000000000003},"min":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.67400000000000004}]},{"osd":10,"last update":"Sat Feb 17 11:56:49 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.749,"15min":0.73799999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.75,"15min":0.73999999999999999},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.749}]},{"osd":11,"last update":"Sat Feb 17 11:57:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.79400000000000004}]}]},{"osd":7,"up_from":51,"seq":219043332144,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":1048769,"data_compressed":42023,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.36699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39600000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.29699999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.58599999999999997,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.311},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.42599999999999999},"min":{"1min":0.30099999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64000000000000001,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51800000000000002}]},{"osd":2,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.35599999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.82799999999999996,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.751},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80400000000000005,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.503}]},{"osd":3,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.55500000000000005},"min":{"1min":0.38400000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.81999999999999995,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58599999999999997,"15min":0.58899999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.745,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.42499999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47099999999999997},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.51600000000000001,"15min":0.51400000000000001},"min":{"1min":0.34300000000000003,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.81899999999999995,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.40300000000000002}]},{"osd":5,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.433,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84299999999999997,"5min":1.333,"15min":1.333},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.42899999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83299999999999996,"5min":1.486,"15min":1.486},"last":0.60799999999999998}]},{"osd":6,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60199999999999998,"15min":0.59399999999999997},"min":{"1min":0.44800000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.89100000000000001,"5min":1.087,"15min":1.087},"last":0.47699999999999998}]},{"osd":8,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.55900000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96999999999999997,"5min":1.446,"15min":1.446},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92400000000000004,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.64700000000000002}]},{"osd":9,"last update":"Sat Feb 17 11:56:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.68200000000000005,"15min":0.68700000000000006},"min":{"1min":0.51100000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91900000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.746,"15min":0.755},"min":{"1min":0.52600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.93600000000000005,"5min":1.052,"15min":1.052},"last":0.53800000000000003}]},{"osd":10,"last update":"Sat Feb 17 11:56:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.498,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95599999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75700000000000001,"15min":0.76100000000000001},"min":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0009999999999999,"5min":1.02,"15min":1.02},"last":0.68000000000000005}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73299999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495670,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":455917,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.35099999999999998},"min":{"1min":0.309,"5min":0.223,"15min":0.223},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38500000000000001,"15min":0.38300000000000001},"min":{"1min":0.28799999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53300000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.29499999999999998}]},{"osd":1,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70099999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.32500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.53800000000000003}]},{"osd":2,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71199999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":0.871,"15min":0.871},"last":0.41699999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64400000000000002,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.317},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.439},"min":{"1min":0.27600000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.628,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.34100000000000003}]},{"osd":4,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.39300000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.39500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.79300000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.504}]},{"osd":5,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59299999999999997,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.503,"5min":0.432,"15min":0.432},"max":{"1min":0.86899999999999999,"5min":0.997,"15min":0.997},"last":0.55400000000000005}]},{"osd":7,"last update":"Sat Feb 17 11:56:27 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57999999999999996,"15min":0.57799999999999996},"min":{"1min":0.38400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.61599999999999999,"15min":0.60499999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.81200000000000006,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51300000000000001}]},{"osd":8,"last update":"Sat Feb 17 11:57:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.71799999999999997,"15min":0.70099999999999996},"min":{"1min":0.46400000000000002,"5min":0.434,"15min":0.434},"max":{"1min":5.4269999999999996,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.61799999999999999}]},{"osd":9,"last update":"Sat Feb 17 11:56:17 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.64300000000000002,"15min":0.66300000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.68999999999999995,"5min":0.879,"15min":0.879},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.67100000000000004,"15min":0.68799999999999994},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.78800000000000003,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54900000000000004}]},{"osd":10,"last update":"Sat Feb 17 11:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.71999999999999997,"15min":0.72699999999999998},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.86799999999999999,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.57799999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":516756,"data_compressed":7701,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54500000000000004,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77800000000000002,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.434},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71199999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.54700000000000004}]},{"osd":2,"last update":"Sat Feb 17 11:56:24 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55100000000000005,"15min":0.52300000000000002},"min":{"1min":0.33000000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.63},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.54600000000000004,"15min":0.52200000000000002},"min":{"1min":0.371,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.68300000000000005}]},{"osd":3,"last update":"Sat Feb 17 11:56:51 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58399999999999996,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.57299999999999995}]},{"osd":4,"last update":"Sat Feb 17 11:56:21 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.443,"15min":0.42599999999999999},"min":{"1min":0.32200000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.437,"15min":0.41099999999999998},"min":{"1min":0.29599999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60299999999999998}]},{"osd":5,"last update":"Sat Feb 17 11:56:43 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.621,"15min":0.57799999999999996},"min":{"1min":0.45700000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89400000000000002,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.64200000000000002,"15min":0.59699999999999998},"min":{"1min":0.58199999999999996,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.76300000000000001}]},{"osd":6,"last update":"Sat Feb 17 11:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63300000000000001,"15min":0.60099999999999998},"min":{"1min":0.52000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.88300000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.497},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.62,"15min":0.59699999999999998},"min":{"1min":0.51000000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78600000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.65200000000000002}]},{"osd":7,"last update":"Sat Feb 17 11:56:29 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.56499999999999995,"15min":0.53200000000000003},"min":{"1min":0.40300000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91200000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.61699999999999999,"15min":0.58899999999999997},"min":{"1min":0.46200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.018,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.77900000000000003}]},{"osd":8,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.71799999999999997,"15min":0.68799999999999994},"min":{"1min":0.622,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98599999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66500000000000004,"15min":0.65400000000000003},"min":{"1min":0.56899999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.83799999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.71499999999999997}]},{"osd":9,"last update":"Sat Feb 17 11:56:21 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.45100000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.93899999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.77900000000000003,"15min":0.77400000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.83399999999999996}]},{"osd":10,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.49099999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.878,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.751},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78300000000000003,"15min":0.77400000000000002},"min":{"1min":0.623,"5min":0.501,"15min":0.501},"max":{"1min":0.96999999999999997,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.79600000000000004}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.87},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.85599999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":966656,"data_stored":457188,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.53700000000000003,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52600000000000002,"15min":0.47999999999999998},"min":{"1min":0.40400000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73199999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.48699999999999999}]},{"osd":2,"last update":"Sat Feb 17 11:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.56699999999999995,"15min":0.51100000000000001},"min":{"1min":0.45400000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78400000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55700000000000005,"15min":0.51900000000000002},"min":{"1min":0.40500000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.54700000000000004}]},{"osd":3,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.32600000000000001}]},{"osd":4,"last update":"Sat Feb 17 11:56:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.54200000000000004,"15min":0.50900000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.434},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55500000000000005,"15min":0.51900000000000002},"min":{"1min":0.36599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46899999999999997}]},{"osd":5,"last update":"Sat Feb 17 11:56:46 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.61699999999999999,"15min":0.59399999999999997},"min":{"1min":0.48999999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.45400000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.66300000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.498,"15min":0.502},"min":{"1min":0.307,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64200000000000002,"5min":0.877,"15min":0.877},"last":0.376},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68799999999999994,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.39000000000000001}]},{"osd":7,"last update":"Sat Feb 17 11:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.501,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58399999999999996}]},{"osd":8,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.69499999999999995,"15min":0.70599999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.46600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.497}]},{"osd":9,"last update":"Sat Feb 17 11:56:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.64100000000000001,"15min":0.65400000000000003},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.68000000000000005,"15min":0.68899999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.36299999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:56:46 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70199999999999996,"15min":0.70999999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.73899999999999999,"15min":0.753},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.44800000000000001}]},{"osd":11,"last update":"Sat Feb 17 11:56:09 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.64600000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":518497,"data_compressed":7701,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51400000000000001,"15min":0.49299999999999999},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71499999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52300000000000002,"15min":0.501},"min":{"1min":0.33700000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67800000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.66400000000000003}]},{"osd":1,"last update":"Sat Feb 17 11:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.52500000000000002,"15min":0.48599999999999999},"min":{"1min":0.35499999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.73399999999999999,"5min":0.751,"15min":0.751},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.54900000000000004,"15min":0.48699999999999999},"min":{"1min":0.38900000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85099999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.85499999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.54500000000000004},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.753,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.72999999999999998},{"interface":"front","average":{"1min":1.4750000000000001,"5min":0.71899999999999997,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":16.718,"5min":16.718,"15min":16.718},"last":0.61699999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.95599999999999996}]},{"osd":5,"last update":"Sat Feb 17 11:56:43 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.44500000000000001,"15min":0.42499999999999999},"min":{"1min":0.35199999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.621,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.45200000000000001,"15min":0.44},"min":{"1min":0.34100000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71899999999999997,"5min":0.878,"15min":0.878},"last":0.64100000000000001}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60799999999999998,"15min":0.58399999999999996},"min":{"1min":0.40799999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.751},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.72799999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.76700000000000002}]},{"osd":7,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63700000000000001}]},{"osd":8,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.41199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.78800000000000003}]},{"osd":9,"last update":"Sat Feb 17 11:56:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.52900000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65700000000000003,"15min":0.66500000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77300000000000002,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.80600000000000005}]},{"osd":10,"last update":"Sat Feb 17 11:56:45 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.61399999999999999,"15min":0.60599999999999998},"min":{"1min":0.46500000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78300000000000003,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.876},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69799999999999995,"15min":0.69099999999999995},"min":{"1min":0.59099999999999997,"5min":0.502,"15min":0.502},"max":{"1min":0.84699999999999998,"5min":1.008,"15min":1.008},"last":0.93600000000000005}]},{"osd":11,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.97599999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":456111,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.35799999999999998,"15min":0.34399999999999997},"min":{"1min":0.26100000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.745,"5min":0.746,"15min":0.746},"last":0.38},{"interface":"front","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.34499999999999997},"min":{"1min":0.26400000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.53000000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.40500000000000003}]},{"osd":1,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.41699999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.53600000000000003},"min":{"1min":0.36799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78300000000000003,"5min":0.87,"15min":0.87},"last":0.58199999999999996}]},{"osd":2,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77100000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78700000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53000000000000003}]},{"osd":4,"last update":"Sat Feb 17 11:56:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.75700000000000001,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58799999999999997,"15min":0.57899999999999996},"min":{"1min":0.42099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.748}]},{"osd":5,"last update":"Sat Feb 17 11:56:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.41499999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56399999999999995,"15min":0.56000000000000005},"min":{"1min":0.40400000000000003,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.79500000000000004,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60299999999999998}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.44400000000000001},"min":{"1min":0.307,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82999999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95799999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.51000000000000001}]},{"osd":7,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":12.301,"5min":12.301,"15min":12.301},"last":0.80100000000000005}]},{"osd":8,"last update":"Sat Feb 17 11:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.49199999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97599999999999998,"5min":1.214,"15min":1.214},"last":0.622}]},{"osd":9,"last update":"Sat Feb 17 11:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.60199999999999998,"15min":0.61599999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76600000000000001,"5min":0.875,"15min":0.875},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.69499999999999995,"15min":0.71499999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.82699999999999996,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.55700000000000005}]},{"osd":10,"last update":"Sat Feb 17 11:56:48 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.74099999999999999,"15min":0.753},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.86499999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.68300000000000005,"15min":0.68899999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.82399999999999995,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.76100000000000001}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.70799999999999996}]}]},{"osd":4,"up_from":33,"seq":141733920832,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":457327,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.50700000000000001,"15min":0.497},"min":{"1min":0.41099999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41599999999999998}]},{"osd":1,"last update":"Sat Feb 17 11:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.26800000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.57299999999999995,"5min":0.872,"15min":0.872},"last":0.433},{"interface":"front","average":{"1min":0.432,"5min":0.40500000000000003,"15min":0.39200000000000002},"min":{"1min":0.29199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.63,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.40899999999999997}]},{"osd":2,"last update":"Sat Feb 17 11:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77600000000000002,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.746},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63500000000000001,"15min":0.55700000000000005},"min":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91600000000000004,"5min":7.0279999999999996,"15min":7.0279999999999996},"last":0.66300000000000003}]},{"osd":3,"last update":"Sat Feb 17 11:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.74299999999999999,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.56299999999999994,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Sat Feb 17 11:56:46 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.40799999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79500000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59699999999999998,"15min":0.56200000000000006},"min":{"1min":0.39300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.61799999999999999}]},{"osd":6,"last update":"Sat Feb 17 11:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59099999999999997,"15min":0.57399999999999995},"min":{"1min":0.41999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77800000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83999999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.503}]},{"osd":7,"last update":"Sat Feb 17 11:56:30 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55400000000000005,"15min":0.51800000000000002},"min":{"1min":0.34999999999999998,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56100000000000005,"15min":0.54600000000000004},"min":{"1min":0.36799999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83999999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64900000000000002}]},{"osd":8,"last update":"Sat Feb 17 11:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":0.67700000000000005},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90400000000000003,"5min":6.9660000000000002,"15min":6.9660000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72299999999999998,"15min":0.70899999999999996},"min":{"1min":0.52800000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.76100000000000001}]},{"osd":9,"last update":"Sat Feb 17 11:56:19 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.48299999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.85899999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.751,"15min":0.753},"min":{"1min":0.53800000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.028,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.73399999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:56:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.77500000000000002,"15min":0.78400000000000003},"min":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73199999999999998,"15min":0.73999999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.92100000000000004,"5min":1.004,"15min":1.004},"last":0.84799999999999998}]},{"osd":11,"last update":"Sat Feb 17 11:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.72099999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":1048062,"data_compressed":42023,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:42 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51600000000000001,"15min":0.49199999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.81899999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52300000000000002,"15min":0.48799999999999999},"min":{"1min":0.39000000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.75,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.64300000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:56:42 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55800000000000005,"15min":0.51400000000000001},"min":{"1min":0.503,"5min":0.313,"15min":0.313},"max":{"1min":0.88300000000000001,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.55700000000000005,"15min":0.50800000000000001},"min":{"1min":0.46400000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.80700000000000005,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.54900000000000004}]},{"osd":2,"last update":"Sat Feb 17 11:56:42 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.42499999999999999,"15min":0.41499999999999998},"min":{"1min":0.34200000000000003,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65700000000000003,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.44,"15min":0.41999999999999998},"min":{"1min":0.32500000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82399999999999995,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.42999999999999999}]},{"osd":3,"last update":"Sat Feb 17 11:56:42 2024","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.52800000000000002},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.81399999999999995,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55700000000000005,"15min":0.53100000000000003},"min":{"1min":0.44500000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.753}]},{"osd":4,"last update":"Sat Feb 17 11:56:42 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.60199999999999998,"15min":0.56399999999999995},"min":{"1min":0.51700000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91300000000000003,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.59699999999999998,"15min":0.56699999999999995},"min":{"1min":0.53100000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.83499999999999996,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.47199999999999998}]},{"osd":6,"last update":"Sat Feb 17 11:56:11 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.45100000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.627},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.64900000000000002,"15min":0.63900000000000001},"min":{"1min":0.47299999999999998,"5min":0.439,"15min":0.439},"max":{"1min":0.90000000000000002,"5min":1.087,"15min":1.087},"last":0.73599999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:56:32 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.63700000000000001,"15min":0.61699999999999999},"min":{"1min":0.47699999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.67000000000000004,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.53300000000000003}]},{"osd":8,"last update":"Sat Feb 17 11:56:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.626,"15min":0.61599999999999999},"min":{"1min":0.46700000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.90900000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92800000000000005,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.70499999999999996}]},{"osd":9,"last update":"Sat Feb 17 11:56:18 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.51500000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.99399999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.70599999999999996,"15min":0.69599999999999995},"min":{"1min":0.53600000000000003,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93899999999999995,"5min":1.056,"15min":1.056},"last":0.67600000000000005}]},{"osd":10,"last update":"Sat Feb 17 11:56:48 2024","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.755,"15min":0.745},"min":{"1min":0.66000000000000003,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.76500000000000001,"15min":0.75900000000000001},"min":{"1min":0.67000000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":61046,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":62787,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63015,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T11:57:15.661 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-17T11:57:15.661 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-17T11:57:15.661 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-17T11:57:15.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph health --format=json 2024-02-17T11:57:15.903 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:57:15.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:15 smithi094 ceph-mon[32499]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-02-17T11:57:16.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:15 smithi189 ceph-mon[36588]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-02-17T11:57:16.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:15 smithi125 ceph-mon[36819]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-02-17T11:57:17.037 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:16 smithi094 ceph-mon[32499]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:17.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:16 smithi189 ceph-mon[36588]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:17.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:16 smithi125 ceph-mon[36819]: from='client.14949 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:17.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.524+0000 7fadc58dc700 1 -- 172.21.15.94:0/3808312133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadc006af20 msgr2=0x7fadc01068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:17.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.524+0000 7fadc58dc700 1 --2- 172.21.15.94:0/3808312133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadc006af20 0x7fadc01068b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fada8009990 tx=0x7fada8009ca0 comp rx=0 tx=0).stop 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.524+0000 7fadc58dc700 1 -- 172.21.15.94:0/3808312133 shutdown_connections 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.524+0000 7fadc58dc700 1 --2- 172.21.15.94:0/3808312133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadc0106fe0 0x7fadc010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.524+0000 7fadc58dc700 1 --2- 172.21.15.94:0/3808312133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadc006a570 0x7fadc006a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.524+0000 7fadc58dc700 1 --2- 172.21.15.94:0/3808312133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadc006af20 0x7fadc01068b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.524+0000 7fadc58dc700 1 -- 172.21.15.94:0/3808312133 >> 172.21.15.94:0/3808312133 conn(0x7fadc0077ae0 msgr2=0x7fadc0077ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.524+0000 7fadc58dc700 1 -- 172.21.15.94:0/3808312133 shutdown_connections 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.524+0000 7fadc58dc700 1 -- 172.21.15.94:0/3808312133 wait complete. 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.524+0000 7fadc58dc700 1 Processor -- start 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.524+0000 7fadc58dc700 1 -- start start 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadc58dc700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadc006a570 0x7fadc01981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadc58dc700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadc006af20 0x7fadc0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadc58dc700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadc0106fe0 0x7fadc019cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadc58dc700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc019d0c0 con 0x7fadc006a570 2024-02-17T11:57:17.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadc58dc700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc019d230 con 0x7fadc0106fe0 2024-02-17T11:57:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadc58dc700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc019d3f0 con 0x7fadc006af20 2024-02-17T11:57:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadbeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadc006a570 0x7fadc01981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadbeffd700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadc006a570 0x7fadc01981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42724/0 (socket says 172.21.15.94:42724) 2024-02-17T11:57:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadbeffd700 1 -- 172.21.15.94:0/16972141 learned_addr learned my addr 172.21.15.94:0/16972141 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadb67fc700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadc006af20 0x7fadc0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadbf7fe700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadc0106fe0 0x7fadc019cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadbeffd700 1 -- 172.21.15.94:0/16972141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadc006af20 msgr2=0x7fadc0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadbeffd700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadc006af20 0x7fadc0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadbeffd700 1 -- 172.21.15.94:0/16972141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadc0106fe0 msgr2=0x7fadc019cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadbeffd700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadc0106fe0 0x7fadc019cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadbeffd700 1 -- 172.21.15.94:0/16972141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada80095f0 con 0x7fadc006a570 2024-02-17T11:57:17.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.525+0000 7fadbeffd700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadc006a570 0x7fadc01981e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fadb000d950 tx=0x7fadb000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.526+0000 7fadc48da700 1 -- 172.21.15.94:0/16972141 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb000f890 con 0x7fadc006a570 2024-02-17T11:57:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.526+0000 7fadc48da700 1 -- 172.21.15.94:0/16972141 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadb0010460 con 0x7fadc006a570 2024-02-17T11:57:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.526+0000 7fadc58dc700 1 -- 172.21.15.94:0/16972141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc019d6d0 con 0x7fadc006a570 2024-02-17T11:57:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.526+0000 7fadc48da700 1 -- 172.21.15.94:0/16972141 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb000e610 con 0x7fadc006a570 2024-02-17T11:57:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.526+0000 7fadc58dc700 1 -- 172.21.15.94:0/16972141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc01a4f30 con 0x7fadc006a570 2024-02-17T11:57:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.527+0000 7fadc58dc700 1 -- 172.21.15.94:0/16972141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc010a1d0 con 0x7fadc006a570 2024-02-17T11:57:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.528+0000 7fadc48da700 1 -- 172.21.15.94:0/16972141 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fadb0010960 con 0x7fadc006a570 2024-02-17T11:57:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.528+0000 7fadc48da700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadac0a21e0 0x7fadac0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.528+0000 7fadb67fc700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadac0a21e0 0x7fadac0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.528+0000 7fadc48da700 1 -- 172.21.15.94:0/16972141 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fadb00cdb90 con 0x7fadc006a570 2024-02-17T11:57:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.529+0000 7fadb67fc700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadac0a21e0 0x7fadac0a46a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fada80059d0 tx=0x7fada8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:17.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.532+0000 7fadc48da700 1 -- 172.21.15.94:0/16972141 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadb0096a00 con 0x7fadc006a570 2024-02-17T11:57:17.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.777+0000 7fadc58dc700 1 -- 172.21.15.94:0/16972141 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fadc0103390 con 0x7fadc006a570 2024-02-17T11:57:17.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.778+0000 7fadc48da700 1 -- 172.21.15.94:0/16972141 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fadb009a0c0 con 0x7fadc006a570 2024-02-17T11:57:17.777 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:17.777 INFO:teuthology.orchestra.run.smithi094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-17T11:57:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.780+0000 7fadc58dc700 1 -- 172.21.15.94:0/16972141 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadac0a21e0 msgr2=0x7fadac0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.780+0000 7fadc58dc700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadac0a21e0 0x7fadac0a46a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fada80059d0 tx=0x7fada8005960 comp rx=0 tx=0).stop 2024-02-17T11:57:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.780+0000 7fadc58dc700 1 -- 172.21.15.94:0/16972141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadc006a570 msgr2=0x7fadc01981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.780+0000 7fadc58dc700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadc006a570 0x7fadc01981e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fadb000d950 tx=0x7fadb000dc60 comp rx=0 tx=0).stop 2024-02-17T11:57:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.780+0000 7fadc58dc700 1 -- 172.21.15.94:0/16972141 shutdown_connections 2024-02-17T11:57:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.780+0000 7fadc58dc700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadac0a21e0 0x7fadac0a46a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.780+0000 7fadc58dc700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadc0106fe0 0x7fadc019cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.780+0000 7fadc58dc700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadc006a570 0x7fadc01981e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.780+0000 7fadc58dc700 1 --2- 172.21.15.94:0/16972141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadc006af20 0x7fadc0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.780+0000 7fadc58dc700 1 -- 172.21.15.94:0/16972141 >> 172.21.15.94:0/16972141 conn(0x7fadc0077ae0 msgr2=0x7fadc00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.780+0000 7fadc58dc700 1 -- 172.21.15.94:0/16972141 shutdown_connections 2024-02-17T11:57:17.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:17.780+0000 7fadc58dc700 1 -- 172.21.15.94:0/16972141 wait complete. 2024-02-17T11:57:18.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:17 smithi189 ceph-mon[36588]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T11:57:18.152 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:17 smithi094 ceph-mon[32499]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T11:57:18.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:17 smithi125 ceph-mon[36819]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T11:57:18.480 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-17T11:57:18.481 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:57:18.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.586+0000 7fe235fbc640 1 -- 172.21.15.94:0/460824827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe230101290 msgr2=0x7fe230101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:18.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.586+0000 7fe235fbc640 1 --2- 172.21.15.94:0/460824827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe230101290 0x7fe230101710 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe218009900 tx=0x7fe21802bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:18.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.586+0000 7fe235fbc640 1 -- 172.21.15.94:0/460824827 shutdown_connections 2024-02-17T11:57:18.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.586+0000 7fe235fbc640 1 --2- 172.21.15.94:0/460824827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe230101c50 0x7fe2301084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.586+0000 7fe235fbc640 1 --2- 172.21.15.94:0/460824827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe230101290 0x7fe230101710 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.586+0000 7fe235fbc640 1 --2- 172.21.15.94:0/460824827 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2301000a0 0x7fe2301004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.586+0000 7fe235fbc640 1 -- 172.21.15.94:0/460824827 >> 172.21.15.94:0/460824827 conn(0x7fe2300fb610 msgr2=0x7fe2300fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:18.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.586+0000 7fe235fbc640 1 -- 172.21.15.94:0/460824827 shutdown_connections 2024-02-17T11:57:18.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.586+0000 7fe235fbc640 1 -- 172.21.15.94:0/460824827 wait complete. 2024-02-17T11:57:18.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.586+0000 7fe235fbc640 1 Processor -- start 2024-02-17T11:57:18.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe235fbc640 1 -- start start 2024-02-17T11:57:18.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe235fbc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2301000a0 0x7fe2301972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:18.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe235fbc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe230101290 0x7fe230197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:18.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe235fbc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe230101c50 0x7fe23019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:18.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe235fbc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23019edd0 con 0x7fe230101c50 2024-02-17T11:57:18.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe235fbc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23019ef40 con 0x7fe2301000a0 2024-02-17T11:57:18.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe235fbc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23019f100 con 0x7fe230101290 2024-02-17T11:57:18.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe230101c50 0x7fe23019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:18.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe230101c50 0x7fe23019e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42746/0 (socket says 172.21.15.94:42746) 2024-02-17T11:57:18.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22ffff640 1 -- 172.21.15.94:0/2416348879 learned_addr learned my addr 172.21.15.94:0/2416348879 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:18.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22effd640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe230101290 0x7fe230197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:18.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22f7fe640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2301000a0 0x7fe2301972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:18.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22ffff640 1 -- 172.21.15.94:0/2416348879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe230101290 msgr2=0x7fe230197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:18.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22ffff640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe230101290 0x7fe230197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22ffff640 1 -- 172.21.15.94:0/2416348879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2301000a0 msgr2=0x7fe2301972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:18.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22ffff640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2301000a0 0x7fe2301972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22ffff640 1 -- 172.21.15.94:0/2416348879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe218009560 con 0x7fe230101c50 2024-02-17T11:57:18.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22f7fe640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2301000a0 0x7fe2301972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:18.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22ffff640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe230101c50 0x7fe23019e890 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe22800d750 tx=0x7fe22800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:18.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22cff9640 1 -- 172.21.15.94:0/2416348879 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228009bd0 con 0x7fe230101c50 2024-02-17T11:57:18.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe235fbc640 1 -- 172.21.15.94:0/2416348879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe23019f3e0 con 0x7fe230101c50 2024-02-17T11:57:18.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22cff9640 1 -- 172.21.15.94:0/2416348879 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe22800de60 con 0x7fe230101c50 2024-02-17T11:57:18.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe22cff9640 1 -- 172.21.15.94:0/2416348879 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228004e70 con 0x7fe230101c50 2024-02-17T11:57:18.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.587+0000 7fe235fbc640 1 -- 172.21.15.94:0/2416348879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe23019f910 con 0x7fe230101c50 2024-02-17T11:57:18.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.589+0000 7fe235fbc640 1 -- 172.21.15.94:0/2416348879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23010bab0 con 0x7fe230101c50 2024-02-17T11:57:18.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.589+0000 7fe22cff9640 1 -- 172.21.15.94:0/2416348879 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe228005020 con 0x7fe230101c50 2024-02-17T11:57:18.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.590+0000 7fe22cff9640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe21c0a2150 0x7fe21c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:18.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.590+0000 7fe22f7fe640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe21c0a2150 0x7fe21c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:18.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.590+0000 7fe22cff9640 1 -- 172.21.15.94:0/2416348879 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe2280cd7c0 con 0x7fe230101c50 2024-02-17T11:57:18.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.591+0000 7fe22f7fe640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe21c0a2150 0x7fe21c0a4610 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe2301010f0 tx=0x7fe220009540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:18.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.595+0000 7fe22cff9640 1 -- 172.21.15.94:0/2416348879 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe22809a160 con 0x7fe230101c50 2024-02-17T11:57:18.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.779+0000 7fe235fbc640 1 -- 172.21.15.94:0/2416348879 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe23019fcf0 con 0x7fe230101c50 2024-02-17T11:57:18.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.781+0000 7fe22cff9640 1 -- 172.21.15.94:0/2416348879 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22686 (secure 0 0 0) 0x7fe228099cf0 con 0x7fe230101c50 2024-02-17T11:57:18.780 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:18.780 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":87,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:56:59.725646+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:57:18.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.783+0000 7fe235fbc640 1 -- 172.21.15.94:0/2416348879 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe21c0a2150 msgr2=0x7fe21c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:18.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.783+0000 7fe235fbc640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe21c0a2150 0x7fe21c0a4610 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe2301010f0 tx=0x7fe220009540 comp rx=0 tx=0).stop 2024-02-17T11:57:18.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.783+0000 7fe235fbc640 1 -- 172.21.15.94:0/2416348879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe230101c50 msgr2=0x7fe23019e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:18.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.783+0000 7fe235fbc640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe230101c50 0x7fe23019e890 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe22800d750 tx=0x7fe22800dc10 comp rx=0 tx=0).stop 2024-02-17T11:57:18.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.783+0000 7fe235fbc640 1 -- 172.21.15.94:0/2416348879 shutdown_connections 2024-02-17T11:57:18.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.783+0000 7fe235fbc640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe21c0a2150 0x7fe21c0a4610 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.783+0000 7fe235fbc640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe230101c50 0x7fe23019e890 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.783+0000 7fe235fbc640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe230101290 0x7fe230197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.783+0000 7fe235fbc640 1 --2- 172.21.15.94:0/2416348879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe2301000a0 0x7fe2301972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.783+0000 7fe235fbc640 1 -- 172.21.15.94:0/2416348879 >> 172.21.15.94:0/2416348879 conn(0x7fe2300fb610 msgr2=0x7fe2300fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.783+0000 7fe235fbc640 1 -- 172.21.15.94:0/2416348879 shutdown_connections 2024-02-17T11:57:18.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.783+0000 7fe235fbc640 1 -- 172.21.15.94:0/2416348879 wait complete. 2024-02-17T11:57:18.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T11:51:20.894601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T11:55:56.349040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T11:55:57.234916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T11:57:18.794 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T11:57:18.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.900+0000 7f7341dd7640 1 -- 172.21.15.94:0/2881830208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c101290 msgr2=0x7f733c101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:18.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.900+0000 7f7341dd7640 1 --2- 172.21.15.94:0/2881830208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c101290 0x7f733c101710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7330009900 tx=0x7f733002bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:18.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.900+0000 7f7341dd7640 1 -- 172.21.15.94:0/2881830208 shutdown_connections 2024-02-17T11:57:18.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.900+0000 7f7341dd7640 1 --2- 172.21.15.94:0/2881830208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f733c101c50 0x7f733c1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.900+0000 7f7341dd7640 1 --2- 172.21.15.94:0/2881830208 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c101290 0x7f733c101710 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.900+0000 7f7341dd7640 1 --2- 172.21.15.94:0/2881830208 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f733c1000a0 0x7f733c1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.900+0000 7f7341dd7640 1 -- 172.21.15.94:0/2881830208 >> 172.21.15.94:0/2881830208 conn(0x7f733c0fb630 msgr2=0x7f733c0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:18.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.900+0000 7f7341dd7640 1 -- 172.21.15.94:0/2881830208 shutdown_connections 2024-02-17T11:57:18.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.900+0000 7f7341dd7640 1 -- 172.21.15.94:0/2881830208 wait complete. 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.900+0000 7f7341dd7640 1 Processor -- start 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f7341dd7640 1 -- start start 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f7341dd7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c1000a0 0x7f733c10a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f7341dd7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f733c101290 0x7f733c108fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f7341dd7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f733c101c50 0x7f733c1094e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f7341dd7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c109c40 con 0x7f733c1000a0 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f7341dd7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c109db0 con 0x7f733c101290 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f7341dd7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c1a1330 con 0x7f733c101c50 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f733b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c1000a0 0x7f733c10a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f733b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c1000a0 0x7f733c10a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42764/0 (socket says 172.21.15.94:42764) 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f733b7fe640 1 -- 172.21.15.94:0/3310480336 learned_addr learned my addr 172.21.15.94:0/3310480336 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f733affd640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f733c101290 0x7f733c108fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f733bfff640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f733c101c50 0x7f733c1094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:18.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f733b7fe640 1 -- 172.21.15.94:0/3310480336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f733c101c50 msgr2=0x7f733c1094e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:18.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f733b7fe640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f733c101c50 0x7f733c1094e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f733b7fe640 1 -- 172.21.15.94:0/3310480336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f733c101290 msgr2=0x7f733c108fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:18.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f733b7fe640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f733c101290 0x7f733c108fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:18.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f733b7fe640 1 -- 172.21.15.94:0/3310480336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7330009560 con 0x7f733c1000a0 2024-02-17T11:57:18.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f733b7fe640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c1000a0 0x7f733c10a960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f732c00b480 tx=0x7f732c00b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:18.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f7338ff9640 1 -- 172.21.15.94:0/3310480336 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c010870 con 0x7f733c1000a0 2024-02-17T11:57:18.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f7338ff9640 1 -- 172.21.15.94:0/3310480336 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f732c00f3c0 con 0x7f733c1000a0 2024-02-17T11:57:18.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f7341dd7640 1 -- 172.21.15.94:0/3310480336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733c1a15e0 con 0x7f733c1000a0 2024-02-17T11:57:18.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f7338ff9640 1 -- 172.21.15.94:0/3310480336 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c0175c0 con 0x7f733c1000a0 2024-02-17T11:57:18.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.901+0000 7f7341dd7640 1 -- 172.21.15.94:0/3310480336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733c1a1b40 con 0x7f733c1000a0 2024-02-17T11:57:18.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.902+0000 7f7341dd7640 1 -- 172.21.15.94:0/3310480336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733c0683c0 con 0x7f733c1000a0 2024-02-17T11:57:18.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.903+0000 7f7338ff9640 1 -- 172.21.15.94:0/3310480336 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f732c00f8c0 con 0x7f733c1000a0 2024-02-17T11:57:18.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.904+0000 7f7338ff9640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f73240a2150 0x7f73240a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:18.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.904+0000 7f733affd640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f73240a2150 0x7f73240a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:18.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.904+0000 7f7338ff9640 1 -- 172.21.15.94:0/3310480336 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f732c0cda80 con 0x7f733c1000a0 2024-02-17T11:57:18.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.904+0000 7f733affd640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f73240a2150 0x7f73240a4610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7330000c00 tx=0x7f733002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:18.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:18.908+0000 7f7338ff9640 1 -- 172.21.15.94:0/3310480336 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f732c096b40 con 0x7f733c1000a0 2024-02-17T11:57:19.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.081+0000 7f7341dd7640 1 -- 172.21.15.94:0/3310480336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f733c10cc60 con 0x7f733c1000a0 2024-02-17T11:57:19.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.082+0000 7f7338ff9640 1 -- 172.21.15.94:0/3310480336 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f733c10cc60 con 0x7f733c1000a0 2024-02-17T11:57:19.081 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-02-17T11:57:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.084+0000 7f7341dd7640 1 -- 172.21.15.94:0/3310480336 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f73240a2150 msgr2=0x7f73240a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.084+0000 7f7341dd7640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f73240a2150 0x7f73240a4610 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7330000c00 tx=0x7f733002c7e0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.084+0000 7f7341dd7640 1 -- 172.21.15.94:0/3310480336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c1000a0 msgr2=0x7f733c10a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.084+0000 7f7341dd7640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c1000a0 0x7f733c10a960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f732c00b480 tx=0x7f732c00b940 comp rx=0 tx=0).stop 2024-02-17T11:57:19.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.084+0000 7f7341dd7640 1 -- 172.21.15.94:0/3310480336 shutdown_connections 2024-02-17T11:57:19.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.084+0000 7f7341dd7640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f73240a2150 0x7f73240a4610 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.084+0000 7f7341dd7640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f733c101c50 0x7f733c1094e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.084+0000 7f7341dd7640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f733c101290 0x7f733c108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.084+0000 7f7341dd7640 1 --2- 172.21.15.94:0/3310480336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f733c1000a0 0x7f733c10a960 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.084+0000 7f7341dd7640 1 -- 172.21.15.94:0/3310480336 >> 172.21.15.94:0/3310480336 conn(0x7f733c0fb630 msgr2=0x7f733c104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:19.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.084+0000 7f7341dd7640 1 -- 172.21.15.94:0/3310480336 shutdown_connections 2024-02-17T11:57:19.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.084+0000 7f7341dd7640 1 -- 172.21.15.94:0/3310480336 wait complete. 2024-02-17T11:57:19.093 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:18 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/16972141' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-17T11:57:19.094 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T11:57:19.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:18 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/16972141' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-17T11:57:19.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:18 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/16972141' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-17T11:57:19.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.201+0000 7fde0569d640 1 -- 172.21.15.94:0/1600355537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde00100080 msgr2=0x7fde001004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.201+0000 7fde0569d640 1 --2- 172.21.15.94:0/1600355537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde00100080 0x7fde001004a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fddf0009900 tx=0x7fddf002bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.201+0000 7fde0569d640 1 -- 172.21.15.94:0/1600355537 shutdown_connections 2024-02-17T11:57:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.201+0000 7fde0569d640 1 --2- 172.21.15.94:0/1600355537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde00101c30 0x7fde001084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.201+0000 7fde0569d640 1 --2- 172.21.15.94:0/1600355537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde00101270 0x7fde001016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.201+0000 7fde0569d640 1 --2- 172.21.15.94:0/1600355537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde00100080 0x7fde001004a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.201+0000 7fde0569d640 1 -- 172.21.15.94:0/1600355537 >> 172.21.15.94:0/1600355537 conn(0x7fde000fb610 msgr2=0x7fde000fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.201+0000 7fde0569d640 1 -- 172.21.15.94:0/1600355537 shutdown_connections 2024-02-17T11:57:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.201+0000 7fde0569d640 1 -- 172.21.15.94:0/1600355537 wait complete. 2024-02-17T11:57:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.202+0000 7fde0569d640 1 Processor -- start 2024-02-17T11:57:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.202+0000 7fde0569d640 1 -- start start 2024-02-17T11:57:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.202+0000 7fde0569d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde00100080 0x7fde00068e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.202+0000 7fde0569d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde00101270 0x7fde0019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:19.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.202+0000 7fde0569d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde00101c30 0x7fde0019e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.202+0000 7fde0569d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0019ed00 con 0x7fde00100080 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.202+0000 7fde0569d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0019ee70 con 0x7fde00101c30 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.202+0000 7fde0569d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0019f030 con 0x7fde00101270 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.202+0000 7fddfeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde00100080 0x7fde00068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.202+0000 7fddfeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde00100080 0x7fde00068e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42786/0 (socket says 172.21.15.94:42786) 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.202+0000 7fddfeffd640 1 -- 172.21.15.94:0/3707537859 learned_addr learned my addr 172.21.15.94:0/3707537859 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fddf67fc640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde00101270 0x7fde0019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fddff7fe640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde00101c30 0x7fde0019e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fddfeffd640 1 -- 172.21.15.94:0/3707537859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde00101270 msgr2=0x7fde0019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fddfeffd640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde00101270 0x7fde0019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fddfeffd640 1 -- 172.21.15.94:0/3707537859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde00101c30 msgr2=0x7fde0019e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fddfeffd640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde00101c30 0x7fde0019e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fddfeffd640 1 -- 172.21.15.94:0/3707537859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf0009560 con 0x7fde00100080 2024-02-17T11:57:19.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fddf67fc640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde00101270 0x7fde0019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:19.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fddff7fe640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde00101c30 0x7fde0019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:19.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fddfeffd640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde00100080 0x7fde00068e70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fddf0000c00 tx=0x7fddf0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:19.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fddfcff9640 1 -- 172.21.15.94:0/3707537859 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf0004df0 con 0x7fde00100080 2024-02-17T11:57:19.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fddfcff9640 1 -- 172.21.15.94:0/3707537859 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fddf0005430 con 0x7fde00100080 2024-02-17T11:57:19.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fde0569d640 1 -- 172.21.15.94:0/3707537859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0019f2b0 con 0x7fde00100080 2024-02-17T11:57:19.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fddfcff9640 1 -- 172.21.15.94:0/3707537859 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddf003e450 con 0x7fde00100080 2024-02-17T11:57:19.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.203+0000 7fde0569d640 1 -- 172.21.15.94:0/3707537859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0019f7c0 con 0x7fde00100080 2024-02-17T11:57:19.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.204+0000 7fde0569d640 1 -- 172.21.15.94:0/3707537859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde00107420 con 0x7fde00100080 2024-02-17T11:57:19.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.205+0000 7fddfcff9640 1 -- 172.21.15.94:0/3707537859 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fddf0003680 con 0x7fde00100080 2024-02-17T11:57:19.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.206+0000 7fddfcff9640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fddec0a2150 0x7fddec0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:19.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.206+0000 7fddf67fc640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fddec0a2150 0x7fddec0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:19.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.206+0000 7fddfcff9640 1 -- 172.21.15.94:0/3707537859 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fddf00f0310 con 0x7fde00100080 2024-02-17T11:57:19.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.206+0000 7fddf67fc640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fddec0a2150 0x7fddec0a4610 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde00068be0 tx=0x7fdde8005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:19.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.210+0000 7fddfcff9640 1 -- 172.21.15.94:0/3707537859 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fddf00b9290 con 0x7fde00100080 2024-02-17T11:57:19.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.390+0000 7fde0569d640 1 -- 172.21.15.94:0/3707537859 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde0002cda0 con 0x7fde00100080 2024-02-17T11:57:19.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.391+0000 7fddfcff9640 1 -- 172.21.15.94:0/3707537859 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fddf00bc950 con 0x7fde00100080 2024-02-17T11:57:19.389 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-02-17T11:57:19.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.393+0000 7fde0569d640 1 -- 172.21.15.94:0/3707537859 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fddec0a2150 msgr2=0x7fddec0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.393+0000 7fde0569d640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fddec0a2150 0x7fddec0a4610 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fde00068be0 tx=0x7fdde8005d90 comp rx=0 tx=0).stop 2024-02-17T11:57:19.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.393+0000 7fde0569d640 1 -- 172.21.15.94:0/3707537859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde00100080 msgr2=0x7fde00068e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.393+0000 7fde0569d640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde00100080 0x7fde00068e70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fddf0000c00 tx=0x7fddf0004270 comp rx=0 tx=0).stop 2024-02-17T11:57:19.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.393+0000 7fde0569d640 1 -- 172.21.15.94:0/3707537859 shutdown_connections 2024-02-17T11:57:19.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.393+0000 7fde0569d640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fddec0a2150 0x7fddec0a4610 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.393+0000 7fde0569d640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde00101c30 0x7fde0019e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.393+0000 7fde0569d640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde00101270 0x7fde0019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.393+0000 7fde0569d640 1 --2- 172.21.15.94:0/3707537859 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde00100080 0x7fde00068e70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.393+0000 7fde0569d640 1 -- 172.21.15.94:0/3707537859 >> 172.21.15.94:0/3707537859 conn(0x7fde000fb610 msgr2=0x7fde001045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:19.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.393+0000 7fde0569d640 1 -- 172.21.15.94:0/3707537859 shutdown_connections 2024-02-17T11:57:19.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.393+0000 7fde0569d640 1 -- 172.21.15.94:0/3707537859 wait complete. 2024-02-17T11:57:19.403 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T11:57:19.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.512+0000 7fb0d74dd640 1 -- 172.21.15.94:0/2440918751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d0101c00 msgr2=0x7fb0d0103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.512+0000 7fb0d74dd640 1 --2- 172.21.15.94:0/2440918751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d0101c00 0x7fb0d0103ff0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009900 tx=0x7fb0cc02bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:19.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.512+0000 7fb0d74dd640 1 -- 172.21.15.94:0/2440918751 shutdown_connections 2024-02-17T11:57:19.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.512+0000 7fb0d74dd640 1 --2- 172.21.15.94:0/2440918751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0d0106e60 0x7fb0d01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.512+0000 7fb0d74dd640 1 --2- 172.21.15.94:0/2440918751 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0d0104530 0x7fb0d0106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.512+0000 7fb0d74dd640 1 --2- 172.21.15.94:0/2440918751 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d0101c00 0x7fb0d0103ff0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.512+0000 7fb0d74dd640 1 -- 172.21.15.94:0/2440918751 >> 172.21.15.94:0/2440918751 conn(0x7fb0d00fb610 msgr2=0x7fb0d00fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:19.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.512+0000 7fb0d74dd640 1 -- 172.21.15.94:0/2440918751 shutdown_connections 2024-02-17T11:57:19.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.512+0000 7fb0d74dd640 1 -- 172.21.15.94:0/2440918751 wait complete. 2024-02-17T11:57:19.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d74dd640 1 Processor -- start 2024-02-17T11:57:19.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d74dd640 1 -- start start 2024-02-17T11:57:19.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d74dd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0d0101c00 0x7fb0d01972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:19.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d74dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d0104530 0x7fb0d0197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:19.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d74dd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0d0106e60 0x7fb0d019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:19.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d74dd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d019edc0 con 0x7fb0d0104530 2024-02-17T11:57:19.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d74dd640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d019ef30 con 0x7fb0d0106e60 2024-02-17T11:57:19.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d74dd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d019f0f0 con 0x7fb0d0101c00 2024-02-17T11:57:19.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d4a78640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d0104530 0x7fb0d0197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:19.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d4a78640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d0104530 0x7fb0d0197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42810/0 (socket says 172.21.15.94:42810) 2024-02-17T11:57:19.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d4a78640 1 -- 172.21.15.94:0/3844086880 learned_addr learned my addr 172.21.15.94:0/3844086880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:19.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d5279640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0d0101c00 0x7fb0d01972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:19.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d5a7a640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0d0106e60 0x7fb0d019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:19.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d4a78640 1 -- 172.21.15.94:0/3844086880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0d0101c00 msgr2=0x7fb0d01972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d4a78640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0d0101c00 0x7fb0d01972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d4a78640 1 -- 172.21.15.94:0/3844086880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0d0106e60 msgr2=0x7fb0d019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d4a78640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0d0106e60 0x7fb0d019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d4a78640 1 -- 172.21.15.94:0/3844086880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc009560 con 0x7fb0d0104530 2024-02-17T11:57:19.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0d4a78640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d0104530 0x7fb0d0197810 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000d770 tx=0x7fb0c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:19.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.513+0000 7fb0c67fc640 1 -- 172.21.15.94:0/3844086880 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0009c30 con 0x7fb0d0104530 2024-02-17T11:57:19.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.514+0000 7fb0c67fc640 1 -- 172.21.15.94:0/3844086880 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb0c000de80 con 0x7fb0d0104530 2024-02-17T11:57:19.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.514+0000 7fb0d74dd640 1 -- 172.21.15.94:0/3844086880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d019f3d0 con 0x7fb0d0104530 2024-02-17T11:57:19.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.514+0000 7fb0c67fc640 1 -- 172.21.15.94:0/3844086880 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000f6b0 con 0x7fb0d0104530 2024-02-17T11:57:19.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.514+0000 7fb0d74dd640 1 -- 172.21.15.94:0/3844086880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d019f9b0 con 0x7fb0d0104530 2024-02-17T11:57:19.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.515+0000 7fb0d74dd640 1 -- 172.21.15.94:0/3844086880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d00fd100 con 0x7fb0d0104530 2024-02-17T11:57:19.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.516+0000 7fb0c67fc640 1 -- 172.21.15.94:0/3844086880 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fb0c0010460 con 0x7fb0d0104530 2024-02-17T11:57:19.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.516+0000 7fb0c67fc640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb0bc0a21d0 0x7fb0bc0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:19.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.517+0000 7fb0c67fc640 1 -- 172.21.15.94:0/3844086880 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb0c00cdbf0 con 0x7fb0d0104530 2024-02-17T11:57:19.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.517+0000 7fb0d5279640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb0bc0a21d0 0x7fb0bc0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:19.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.517+0000 7fb0d5279640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb0bc0a21d0 0x7fb0bc0a4690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc02c840 tx=0x7fb0cc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:19.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.521+0000 7fb0c67fc640 1 -- 172.21.15.94:0/3844086880 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb0c0096a60 con 0x7fb0d0104530 2024-02-17T11:57:19.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.705+0000 7fb0d74dd640 1 -- 172.21.15.94:0/3844086880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0d010ce40 con 0x7fb0d0104530 2024-02-17T11:57:19.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.706+0000 7fb0c67fc640 1 -- 172.21.15.94:0/3844086880 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0c009a120 con 0x7fb0d0104530 2024-02-17T11:57:19.705 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-02-17T11:57:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.708+0000 7fb0d74dd640 1 -- 172.21.15.94:0/3844086880 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb0bc0a21d0 msgr2=0x7fb0bc0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.708+0000 7fb0d74dd640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb0bc0a21d0 0x7fb0bc0a4690 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc02c840 tx=0x7fb0cc000bc0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.708+0000 7fb0d74dd640 1 -- 172.21.15.94:0/3844086880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d0104530 msgr2=0x7fb0d0197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.708+0000 7fb0d74dd640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d0104530 0x7fb0d0197810 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000d770 tx=0x7fb0c000dc30 comp rx=0 tx=0).stop 2024-02-17T11:57:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.708+0000 7fb0d74dd640 1 -- 172.21.15.94:0/3844086880 shutdown_connections 2024-02-17T11:57:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.708+0000 7fb0d74dd640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb0bc0a21d0 0x7fb0bc0a4690 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.708+0000 7fb0d74dd640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb0d0106e60 0x7fb0d019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.708+0000 7fb0d74dd640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb0d0104530 0x7fb0d0197810 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.708+0000 7fb0d74dd640 1 --2- 172.21.15.94:0/3844086880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0d0101c00 0x7fb0d01972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.708+0000 7fb0d74dd640 1 -- 172.21.15.94:0/3844086880 >> 172.21.15.94:0/3844086880 conn(0x7fb0d00fb610 msgr2=0x7fb0d0102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:19.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.708+0000 7fb0d74dd640 1 -- 172.21.15.94:0/3844086880 shutdown_connections 2024-02-17T11:57:19.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.708+0000 7fb0d74dd640 1 -- 172.21.15.94:0/3844086880 wait complete. 2024-02-17T11:57:19.718 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:57:19.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.823+0000 7f312a89d640 1 -- 172.21.15.94:0/605262178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31241003f0 msgr2=0x7f3124100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.823+0000 7f312a89d640 1 --2- 172.21.15.94:0/605262178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31241003f0 0x7f3124100850 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3118009900 tx=0x7f311802bcd0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.823+0000 7f312a89d640 1 -- 172.21.15.94:0/605262178 shutdown_connections 2024-02-17T11:57:19.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.823+0000 7f312a89d640 1 --2- 172.21.15.94:0/605262178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3124100d90 0x7f3124101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.823+0000 7f312a89d640 1 --2- 172.21.15.94:0/605262178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31241003f0 0x7f3124100850 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.823+0000 7f312a89d640 1 --2- 172.21.15.94:0/605262178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31240ffa90 0x7f31240ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.823+0000 7f312a89d640 1 -- 172.21.15.94:0/605262178 >> 172.21.15.94:0/605262178 conn(0x7f31240fb610 msgr2=0x7f31240fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.823+0000 7f312a89d640 1 -- 172.21.15.94:0/605262178 shutdown_connections 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.823+0000 7f312a89d640 1 -- 172.21.15.94:0/605262178 wait complete. 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f312a89d640 1 Processor -- start 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f312a89d640 1 -- start start 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f312a89d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31240ffa90 0x7f31241993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f312a89d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31241003f0 0x7f3124199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f312a89d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3124100d90 0x7f31241a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f312a89d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31241a0eb0 con 0x7f31241003f0 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f312a89d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31241a1020 con 0x7f31240ffa90 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f312a89d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31241a11e0 con 0x7f3124100d90 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f31237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31241003f0 0x7f3124199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f31237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31241003f0 0x7f3124199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42822/0 (socket says 172.21.15.94:42822) 2024-02-17T11:57:19.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f31237fe640 1 -- 172.21.15.94:0/896785215 learned_addr learned my addr 172.21.15.94:0/896785215 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f31237fe640 1 -- 172.21.15.94:0/896785215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3124100d90 msgr2=0x7f31241a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f3128e3a640 1 --2- 172.21.15.94:0/896785215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3124100d90 0x7f31241a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f31237fe640 1 --2- 172.21.15.94:0/896785215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3124100d90 0x7f31241a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f31237fe640 1 -- 172.21.15.94:0/896785215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31240ffa90 msgr2=0x7f31241993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f31237fe640 1 --2- 172.21.15.94:0/896785215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31240ffa90 0x7f31241993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.824+0000 7f31237fe640 1 -- 172.21.15.94:0/896785215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3118009560 con 0x7f31241003f0 2024-02-17T11:57:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.825+0000 7f31237fe640 1 --2- 172.21.15.94:0/896785215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31241003f0 0x7f3124199900 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3118006010 tx=0x7f31180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.825+0000 7f31217fa640 1 -- 172.21.15.94:0/896785215 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31180044e0 con 0x7f31241003f0 2024-02-17T11:57:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.825+0000 7f31217fa640 1 -- 172.21.15.94:0/896785215 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f31180054e0 con 0x7f31241003f0 2024-02-17T11:57:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.825+0000 7f312a89d640 1 -- 172.21.15.94:0/896785215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31241a1460 con 0x7f31241003f0 2024-02-17T11:57:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.825+0000 7f31217fa640 1 -- 172.21.15.94:0/896785215 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311803e450 con 0x7f31241003f0 2024-02-17T11:57:19.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.825+0000 7f312a89d640 1 -- 172.21.15.94:0/896785215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31241a1970 con 0x7f31241003f0 2024-02-17T11:57:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.826+0000 7f312a89d640 1 -- 172.21.15.94:0/896785215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312410a970 con 0x7f31241003f0 2024-02-17T11:57:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.827+0000 7f31217fa640 1 -- 172.21.15.94:0/896785215 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f31180035f0 con 0x7f31241003f0 2024-02-17T11:57:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.827+0000 7f31217fa640 1 --2- 172.21.15.94:0/896785215 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f310c0a2150 0x7f310c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.827+0000 7f3123fff640 1 --2- 172.21.15.94:0/896785215 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f310c0a2150 0x7f310c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:19.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.827+0000 7f31217fa640 1 -- 172.21.15.94:0/896785215 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f31180f0e70 con 0x7f31241003f0 2024-02-17T11:57:19.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.828+0000 7f3123fff640 1 --2- 172.21.15.94:0/896785215 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f310c0a2150 0x7f310c0a4610 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3114005fd0 tx=0x7f3114005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:19.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:19.831+0000 7f31217fa640 1 -- 172.21.15.94:0/896785215 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31180b9df0 con 0x7f31241003f0 2024-02-17T11:57:20.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.046+0000 7f312a89d640 1 -- 172.21.15.94:0/896785215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f312410eed0 con 0x7f31241003f0 2024-02-17T11:57:20.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.047+0000 7f31217fa640 1 -- 172.21.15.94:0/896785215 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f31180bd4b0 con 0x7f31241003f0 2024-02-17T11:57:20.046 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:20.046 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:56:02.151161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T11:57:20.046 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 20 2024-02-17T11:57:20.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.049+0000 7f312a89d640 1 -- 172.21.15.94:0/896785215 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f310c0a2150 msgr2=0x7f310c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.049+0000 7f312a89d640 1 --2- 172.21.15.94:0/896785215 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f310c0a2150 0x7f310c0a4610 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3114005fd0 tx=0x7f3114005850 comp rx=0 tx=0).stop 2024-02-17T11:57:20.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.049+0000 7f312a89d640 1 -- 172.21.15.94:0/896785215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31241003f0 msgr2=0x7f3124199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.049+0000 7f312a89d640 1 --2- 172.21.15.94:0/896785215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31241003f0 0x7f3124199900 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3118006010 tx=0x7f31180042f0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.049+0000 7f312a89d640 1 -- 172.21.15.94:0/896785215 shutdown_connections 2024-02-17T11:57:20.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.049+0000 7f312a89d640 1 --2- 172.21.15.94:0/896785215 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f310c0a2150 0x7f310c0a4610 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.049+0000 7f312a89d640 1 --2- 172.21.15.94:0/896785215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3124100d90 0x7f31241a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.049+0000 7f312a89d640 1 --2- 172.21.15.94:0/896785215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f31241003f0 0x7f3124199900 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.049+0000 7f312a89d640 1 --2- 172.21.15.94:0/896785215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f31240ffa90 0x7f31241993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.049+0000 7f312a89d640 1 -- 172.21.15.94:0/896785215 >> 172.21.15.94:0/896785215 conn(0x7f31240fb610 msgr2=0x7f31240fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:20.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.049+0000 7f312a89d640 1 -- 172.21.15.94:0/896785215 shutdown_connections 2024-02-17T11:57:20.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.049+0000 7f312a89d640 1 -- 172.21.15.94:0/896785215 wait complete. 2024-02-17T11:57:20.058 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:19 smithi094 ceph-mon[32499]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-17T11:57:20.058 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:19 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2416348879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:20.058 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:19 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3310480336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:20.058 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:19 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3707537859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:20.058 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:19 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3844086880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:20.060 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:57:20.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:19 smithi189 ceph-mon[36588]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-17T11:57:20.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:19 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2416348879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:20.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:19 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3310480336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:20.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:19 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3707537859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:20.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:19 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3844086880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.164+0000 7fadf9e42640 1 -- 172.21.15.94:0/723490375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf41000a0 msgr2=0x7fadf41004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.164+0000 7fadf9e42640 1 --2- 172.21.15.94:0/723490375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf41000a0 0x7fadf41004c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fade4009900 tx=0x7fade402bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 -- 172.21.15.94:0/723490375 shutdown_connections 2024-02-17T11:57:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 --2- 172.21.15.94:0/723490375 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadf4101c50 0x7fadf41084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 --2- 172.21.15.94:0/723490375 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadf4101290 0x7fadf4101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 --2- 172.21.15.94:0/723490375 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf41000a0 0x7fadf41004c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 -- 172.21.15.94:0/723490375 >> 172.21.15.94:0/723490375 conn(0x7fadf40fb610 msgr2=0x7fadf40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 -- 172.21.15.94:0/723490375 shutdown_connections 2024-02-17T11:57:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 -- 172.21.15.94:0/723490375 wait complete. 2024-02-17T11:57:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 Processor -- start 2024-02-17T11:57:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 -- start start 2024-02-17T11:57:20.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadf41000a0 0x7fadf4068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadf4101290 0x7fadf419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf4101c50 0x7fadf419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf419ee00 con 0x7fadf4101c50 2024-02-17T11:57:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf419ef70 con 0x7fadf41000a0 2024-02-17T11:57:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.165+0000 7fadf9e42640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf419f130 con 0x7fadf4101290 2024-02-17T11:57:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf4101c50 0x7fadf419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf4101c50 0x7fadf419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42846/0 (socket says 172.21.15.94:42846) 2024-02-17T11:57:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf3fff640 1 -- 172.21.15.94:0/3746246468 learned_addr learned my addr 172.21.15.94:0/3746246468 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf2ffd640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadf4101290 0x7fadf419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf37fe640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadf41000a0 0x7fadf4068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf3fff640 1 -- 172.21.15.94:0/3746246468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadf4101290 msgr2=0x7fadf419c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf3fff640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadf4101290 0x7fadf419c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf3fff640 1 -- 172.21.15.94:0/3746246468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadf41000a0 msgr2=0x7fadf4068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf3fff640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadf41000a0 0x7fadf4068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf3fff640 1 -- 172.21.15.94:0/3746246468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4009560 con 0x7fadf4101c50 2024-02-17T11:57:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf3fff640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf4101c50 0x7fadf419e620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fadec00eaa0 tx=0x7fadec00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf0ff9640 1 -- 172.21.15.94:0/3746246468 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec004d20 con 0x7fadf4101c50 2024-02-17T11:57:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf9e42640 1 -- 172.21.15.94:0/3746246468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf419f3b0 con 0x7fadf4101c50 2024-02-17T11:57:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf0ff9640 1 -- 172.21.15.94:0/3746246468 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadec004e80 con 0x7fadf4101c50 2024-02-17T11:57:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf0ff9640 1 -- 172.21.15.94:0/3746246468 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec005a10 con 0x7fadf4101c50 2024-02-17T11:57:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.166+0000 7fadf9e42640 1 -- 172.21.15.94:0/3746246468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf419f8f0 con 0x7fadf4101c50 2024-02-17T11:57:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.167+0000 7fadf9e42640 1 -- 172.21.15.94:0/3746246468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf41073c0 con 0x7fadf4101c50 2024-02-17T11:57:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.169+0000 7fadf0ff9640 1 -- 172.21.15.94:0/3746246468 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fadec005000 con 0x7fadf4101c50 2024-02-17T11:57:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.169+0000 7fadf0ff9640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faddc0a2150 0x7faddc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.169+0000 7fadf37fe640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faddc0a2150 0x7faddc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.169+0000 7fadf0ff9640 1 -- 172.21.15.94:0/3746246468 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fadec0ce4b0 con 0x7fadf4101c50 2024-02-17T11:57:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.170+0000 7fadf37fe640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faddc0a2150 0x7faddc0a4610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fade402c240 tx=0x7fade4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:20.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.173+0000 7fadf0ff9640 1 -- 172.21.15.94:0/3746246468 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadec097320 con 0x7fadf4101c50 2024-02-17T11:57:20.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:19 smithi125 ceph-mon[36819]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-17T11:57:20.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:19 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2416348879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:20.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:19 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3310480336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:20.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:19 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3707537859' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:20.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:19 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3844086880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:20.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.357+0000 7fadf9e42640 1 -- 172.21.15.94:0/3746246468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadf410bd40 con 0x7fadf4101c50 2024-02-17T11:57:20.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.359+0000 7fadf0ff9640 1 -- 172.21.15.94:0/3746246468 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22686 (secure 0 0 0) 0x7fadec01f070 con 0x7fadf4101c50 2024-02-17T11:57:20.358 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:20.358 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":87,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:56:59.725646+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:57:20.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.361+0000 7fadf9e42640 1 -- 172.21.15.94:0/3746246468 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faddc0a2150 msgr2=0x7faddc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.361+0000 7fadf9e42640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faddc0a2150 0x7faddc0a4610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fade402c240 tx=0x7fade4005fd0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.361+0000 7fadf9e42640 1 -- 172.21.15.94:0/3746246468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf4101c50 msgr2=0x7fadf419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.361+0000 7fadf9e42640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf4101c50 0x7fadf419e620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fadec00eaa0 tx=0x7fadec00eea0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.361+0000 7fadf9e42640 1 -- 172.21.15.94:0/3746246468 shutdown_connections 2024-02-17T11:57:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.361+0000 7fadf9e42640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faddc0a2150 0x7faddc0a4610 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.361+0000 7fadf9e42640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadf4101c50 0x7fadf419e620 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.361+0000 7fadf9e42640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadf4101290 0x7fadf419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.361+0000 7fadf9e42640 1 --2- 172.21.15.94:0/3746246468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadf41000a0 0x7fadf4068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.361+0000 7fadf9e42640 1 -- 172.21.15.94:0/3746246468 >> 172.21.15.94:0/3746246468 conn(0x7fadf40fb610 msgr2=0x7fadf4104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.361+0000 7fadf9e42640 1 -- 172.21.15.94:0/3746246468 shutdown_connections 2024-02-17T11:57:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.361+0000 7fadf9e42640 1 -- 172.21.15.94:0/3746246468 wait complete. 2024-02-17T11:57:20.371 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T11:51:20.894601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T11:55:56.349040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T11:55:57.234916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T11:57:20.371 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T11:57:20.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.478+0000 7f987aa88640 1 -- 172.21.15.94:0/808245790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9874100080 msgr2=0x7f98741004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.478+0000 7f987aa88640 1 --2- 172.21.15.94:0/808245790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9874100080 0x7f98741004a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9864009900 tx=0x7f986402bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:20.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.478+0000 7f987aa88640 1 -- 172.21.15.94:0/808245790 shutdown_connections 2024-02-17T11:57:20.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.478+0000 7f987aa88640 1 --2- 172.21.15.94:0/808245790 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874101c30 0x7f98741084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.478+0000 7f987aa88640 1 --2- 172.21.15.94:0/808245790 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874101270 0x7f98741016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.478+0000 7f987aa88640 1 --2- 172.21.15.94:0/808245790 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9874100080 0x7f98741004a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.478+0000 7f987aa88640 1 -- 172.21.15.94:0/808245790 >> 172.21.15.94:0/808245790 conn(0x7f98740fb610 msgr2=0x7f98740fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:20.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.478+0000 7f987aa88640 1 -- 172.21.15.94:0/808245790 shutdown_connections 2024-02-17T11:57:20.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.478+0000 7f987aa88640 1 -- 172.21.15.94:0/808245790 wait complete. 2024-02-17T11:57:20.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f987aa88640 1 Processor -- start 2024-02-17T11:57:20.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f987aa88640 1 -- start start 2024-02-17T11:57:20.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f987aa88640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874100080 0x7f9874068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:20.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f987aa88640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874101270 0x7f987419bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:20.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f987aa88640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9874101c30 0x7f987419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:20.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f987aa88640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987419edd0 con 0x7f9874101c30 2024-02-17T11:57:20.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f987aa88640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987419ef40 con 0x7f9874100080 2024-02-17T11:57:20.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f987aa88640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987419f100 con 0x7f9874101270 2024-02-17T11:57:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f9879025640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9874101c30 0x7f987419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f9879025640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9874101c30 0x7f987419e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42862/0 (socket says 172.21.15.94:42862) 2024-02-17T11:57:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f9879025640 1 -- 172.21.15.94:0/20778938 learned_addr learned my addr 172.21.15.94:0/20778938 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f9873fff640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874101270 0x7f987419bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f9878824640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874100080 0x7f9874068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f9879025640 1 -- 172.21.15.94:0/20778938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874101270 msgr2=0x7f987419bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f9879025640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874101270 0x7f987419bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f9879025640 1 -- 172.21.15.94:0/20778938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874100080 msgr2=0x7f9874068e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f9879025640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874100080 0x7f9874068e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f9879025640 1 -- 172.21.15.94:0/20778938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864009560 con 0x7f9874101c30 2024-02-17T11:57:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.479+0000 7f9873fff640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874101270 0x7f987419bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.480+0000 7f9879025640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9874101c30 0x7f987419e5f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f986c00eaa0 tx=0x7f986c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:20.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.480+0000 7f9871ffb640 1 -- 172.21.15.94:0/20778938 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c00cdb0 con 0x7f9874101c30 2024-02-17T11:57:20.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.480+0000 7f9871ffb640 1 -- 172.21.15.94:0/20778938 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f986c005020 con 0x7f9874101c30 2024-02-17T11:57:20.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.480+0000 7f987aa88640 1 -- 172.21.15.94:0/20778938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987419f380 con 0x7f9874101c30 2024-02-17T11:57:20.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.480+0000 7f9871ffb640 1 -- 172.21.15.94:0/20778938 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c0116b0 con 0x7f9874101c30 2024-02-17T11:57:20.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.480+0000 7f987aa88640 1 -- 172.21.15.94:0/20778938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987419f890 con 0x7f9874101c30 2024-02-17T11:57:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.481+0000 7f987aa88640 1 -- 172.21.15.94:0/20778938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9874107470 con 0x7f9874101c30 2024-02-17T11:57:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.482+0000 7f9871ffb640 1 -- 172.21.15.94:0/20778938 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f986c005520 con 0x7f9874101c30 2024-02-17T11:57:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.483+0000 7f9871ffb640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f985c0a21d0 0x7f985c0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.483+0000 7f9878824640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f985c0a21d0 0x7f985c0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.483+0000 7f9871ffb640 1 -- 172.21.15.94:0/20778938 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f986c0ce0a0 con 0x7f9874101c30 2024-02-17T11:57:20.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.483+0000 7f9878824640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f985c0a21d0 0x7f985c0a4690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9864006010 tx=0x7f9864000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:20.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.487+0000 7f9871ffb640 1 -- 172.21.15.94:0/20778938 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f986c096f10 con 0x7f9874101c30 2024-02-17T11:57:20.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.661+0000 7f987aa88640 1 -- 172.21.15.94:0/20778938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f987410bc50 con 0x7f9874101c30 2024-02-17T11:57:20.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.661+0000 7f9871ffb640 1 -- 172.21.15.94:0/20778938 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f986c01f0a0 con 0x7f9874101c30 2024-02-17T11:57:20.660 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-02-17T11:57:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.663+0000 7f987aa88640 1 -- 172.21.15.94:0/20778938 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f985c0a21d0 msgr2=0x7f985c0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.663+0000 7f987aa88640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f985c0a21d0 0x7f985c0a4690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9864006010 tx=0x7f9864000bc0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.663+0000 7f987aa88640 1 -- 172.21.15.94:0/20778938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9874101c30 msgr2=0x7f987419e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.663+0000 7f987aa88640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9874101c30 0x7f987419e5f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f986c00eaa0 tx=0x7f986c00ef60 comp rx=0 tx=0).stop 2024-02-17T11:57:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.663+0000 7f987aa88640 1 -- 172.21.15.94:0/20778938 shutdown_connections 2024-02-17T11:57:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.663+0000 7f987aa88640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f985c0a21d0 0x7f985c0a4690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.663+0000 7f987aa88640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9874101c30 0x7f987419e5f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.663+0000 7f987aa88640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9874101270 0x7f987419bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.663+0000 7f987aa88640 1 --2- 172.21.15.94:0/20778938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9874100080 0x7f9874068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.663+0000 7f987aa88640 1 -- 172.21.15.94:0/20778938 >> 172.21.15.94:0/20778938 conn(0x7f98740fb610 msgr2=0x7f9874104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:20.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.663+0000 7f987aa88640 1 -- 172.21.15.94:0/20778938 shutdown_connections 2024-02-17T11:57:20.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.663+0000 7f987aa88640 1 -- 172.21.15.94:0/20778938 wait complete. 2024-02-17T11:57:20.673 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T11:57:20.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.780+0000 7f4c972d4640 1 -- 172.21.15.94:0/126488036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c90102790 msgr2=0x7f4c90102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.780+0000 7f4c972d4640 1 --2- 172.21.15.94:0/126488036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c90102790 0x7f4c90102bb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4c80009900 tx=0x7f4c8002bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:20.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.780+0000 7f4c972d4640 1 -- 172.21.15.94:0/126488036 shutdown_connections 2024-02-17T11:57:20.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.780+0000 7f4c972d4640 1 --2- 172.21.15.94:0/126488036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c900ffbc0 0x7f4c90109770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.780+0000 7f4c972d4640 1 --2- 172.21.15.94:0/126488036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c900ff200 0x7f4c900ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.780+0000 7f4c972d4640 1 --2- 172.21.15.94:0/126488036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c90102790 0x7f4c90102bb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.780+0000 7f4c972d4640 1 -- 172.21.15.94:0/126488036 >> 172.21.15.94:0/126488036 conn(0x7f4c900fad80 msgr2=0x7f4c900fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.780+0000 7f4c972d4640 1 -- 172.21.15.94:0/126488036 shutdown_connections 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.780+0000 7f4c972d4640 1 -- 172.21.15.94:0/126488036 wait complete. 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.780+0000 7f4c972d4640 1 Processor -- start 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.780+0000 7f4c972d4640 1 -- start start 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c972d4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c900ff200 0x7f4c90197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c972d4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c900ffbc0 0x7f4c901977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c972d4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c90102790 0x7f4c9019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c972d4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9019ed70 con 0x7f4c90102790 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c972d4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9019eee0 con 0x7f4c900ffbc0 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c972d4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9019f0a0 con 0x7f4c900ff200 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c95871640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c90102790 0x7f4c9019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c95871640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c90102790 0x7f4c9019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42888/0 (socket says 172.21.15.94:42888) 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c95871640 1 -- 172.21.15.94:0/2947910460 learned_addr learned my addr 172.21.15.94:0/2947910460 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:20.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c95070640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c900ff200 0x7f4c90197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:20.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c9486f640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c900ffbc0 0x7f4c901977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:20.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c95871640 1 -- 172.21.15.94:0/2947910460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c900ff200 msgr2=0x7f4c90197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c95871640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c900ff200 0x7f4c90197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c95871640 1 -- 172.21.15.94:0/2947910460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c900ffbc0 msgr2=0x7f4c901977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c95871640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c900ffbc0 0x7f4c901977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c95871640 1 -- 172.21.15.94:0/2947910460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c80009560 con 0x7f4c90102790 2024-02-17T11:57:20.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c95871640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c90102790 0x7f4c9019e830 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800cad0 tx=0x7f4c8800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:20.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c867fc640 1 -- 172.21.15.94:0/2947910460 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800ae20 con 0x7f4c90102790 2024-02-17T11:57:20.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c867fc640 1 -- 172.21.15.94:0/2947910460 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c88005020 con 0x7f4c90102790 2024-02-17T11:57:20.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c867fc640 1 -- 172.21.15.94:0/2947910460 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8800f6b0 con 0x7f4c90102790 2024-02-17T11:57:20.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c972d4640 1 -- 172.21.15.94:0/2947910460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9019f380 con 0x7f4c90102790 2024-02-17T11:57:20.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.781+0000 7f4c972d4640 1 -- 172.21.15.94:0/2947910460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9019f840 con 0x7f4c90102790 2024-02-17T11:57:20.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.782+0000 7f4c972d4640 1 -- 172.21.15.94:0/2947910460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c90108720 con 0x7f4c90102790 2024-02-17T11:57:20.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.783+0000 7f4c867fc640 1 -- 172.21.15.94:0/2947910460 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f4c88005520 con 0x7f4c90102790 2024-02-17T11:57:20.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.784+0000 7f4c867fc640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4c7c0a2150 0x7f4c7c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:20.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.784+0000 7f4c95070640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4c7c0a2150 0x7f4c7c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:20.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.784+0000 7f4c867fc640 1 -- 172.21.15.94:0/2947910460 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4c880ce2c0 con 0x7f4c90102790 2024-02-17T11:57:20.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.785+0000 7f4c95070640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4c7c0a2150 0x7f4c7c0a4610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4c80005ab0 tx=0x7f4c80005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:20.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.788+0000 7f4c867fc640 1 -- 172.21.15.94:0/2947910460 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c880970f0 con 0x7f4c90102790 2024-02-17T11:57:20.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:20 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/896785215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:20.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:20 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3746246468' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:20.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:20 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/20778938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:20.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.961+0000 7f4c972d4640 1 -- 172.21.15.94:0/2947910460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4c9010cbf0 con 0x7f4c90102790 2024-02-17T11:57:20.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.961+0000 7f4c867fc640 1 -- 172.21.15.94:0/2947910460 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4c8801d0a0 con 0x7f4c90102790 2024-02-17T11:57:20.960 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-02-17T11:57:20.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.963+0000 7f4c972d4640 1 -- 172.21.15.94:0/2947910460 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4c7c0a2150 msgr2=0x7f4c7c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.963+0000 7f4c972d4640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4c7c0a2150 0x7f4c7c0a4610 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4c80005ab0 tx=0x7f4c80005fd0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.963+0000 7f4c972d4640 1 -- 172.21.15.94:0/2947910460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c90102790 msgr2=0x7f4c9019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:20.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.963+0000 7f4c972d4640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c90102790 0x7f4c9019e830 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800cad0 tx=0x7f4c8800cf90 comp rx=0 tx=0).stop 2024-02-17T11:57:20.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.963+0000 7f4c972d4640 1 -- 172.21.15.94:0/2947910460 shutdown_connections 2024-02-17T11:57:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.963+0000 7f4c972d4640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4c7c0a2150 0x7f4c7c0a4610 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.963+0000 7f4c972d4640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c90102790 0x7f4c9019e830 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.963+0000 7f4c972d4640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c900ffbc0 0x7f4c901977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.963+0000 7f4c972d4640 1 --2- 172.21.15.94:0/2947910460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c900ff200 0x7f4c90197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.963+0000 7f4c972d4640 1 -- 172.21.15.94:0/2947910460 >> 172.21.15.94:0/2947910460 conn(0x7f4c900fad80 msgr2=0x7f4c901058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.963+0000 7f4c972d4640 1 -- 172.21.15.94:0/2947910460 shutdown_connections 2024-02-17T11:57:20.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:20.963+0000 7f4c972d4640 1 -- 172.21.15.94:0/2947910460 wait complete. 2024-02-17T11:57:20.973 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T11:57:21.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.078+0000 7f128717a640 1 -- 172.21.15.94:0/2680832811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12801033c0 msgr2=0x7f12801037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.078+0000 7f128717a640 1 --2- 172.21.15.94:0/2680832811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12801033c0 0x7f12801037e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1270009900 tx=0x7f127002bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:21.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.078+0000 7f128717a640 1 -- 172.21.15.94:0/2680832811 shutdown_connections 2024-02-17T11:57:21.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.078+0000 7f128717a640 1 --2- 172.21.15.94:0/2680832811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1280104f70 0x7f128010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.078+0000 7f128717a640 1 --2- 172.21.15.94:0/2680832811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12801045b0 0x7f1280104a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.078+0000 7f128717a640 1 --2- 172.21.15.94:0/2680832811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12801033c0 0x7f12801037e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.078+0000 7f128717a640 1 -- 172.21.15.94:0/2680832811 >> 172.21.15.94:0/2680832811 conn(0x7f12800fe930 msgr2=0x7f1280100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:21.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.078+0000 7f128717a640 1 -- 172.21.15.94:0/2680832811 shutdown_connections 2024-02-17T11:57:21.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.078+0000 7f128717a640 1 -- 172.21.15.94:0/2680832811 wait complete. 2024-02-17T11:57:21.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f128717a640 1 Processor -- start 2024-02-17T11:57:21.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f128717a640 1 -- start start 2024-02-17T11:57:21.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f128717a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12801033c0 0x7f12801994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:21.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f128717a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12801045b0 0x7f12801999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:21.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f128717a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1280104f70 0x7f12801a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:21.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f128717a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801a0fa0 con 0x7f12801045b0 2024-02-17T11:57:21.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f128717a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801a1110 con 0x7f12801033c0 2024-02-17T11:57:21.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f128717a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12801a12d0 con 0x7f1280104f70 2024-02-17T11:57:21.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f127ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12801045b0 0x7f12801999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:21.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f127ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12801045b0 0x7f12801999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42904/0 (socket says 172.21.15.94:42904) 2024-02-17T11:57:21.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f127ffff640 1 -- 172.21.15.94:0/3425264493 learned_addr learned my addr 172.21.15.94:0/3425264493 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:21.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f1284f16640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12801033c0 0x7f12801994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:21.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f127ffff640 1 -- 172.21.15.94:0/3425264493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1280104f70 msgr2=0x7f12801a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f127ffff640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1280104f70 0x7f12801a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f127ffff640 1 -- 172.21.15.94:0/3425264493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12801033c0 msgr2=0x7f12801994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f127ffff640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12801033c0 0x7f12801994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f127ffff640 1 -- 172.21.15.94:0/3425264493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1270009560 con 0x7f12801045b0 2024-02-17T11:57:21.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f1284f16640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12801033c0 0x7f12801994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:21.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.079+0000 7f127ffff640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12801045b0 0x7f12801999f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f127400b4a0 tx=0x7f127400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:21.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.080+0000 7f127dffb640 1 -- 172.21.15.94:0/3425264493 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274010610 con 0x7f12801045b0 2024-02-17T11:57:21.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.080+0000 7f127dffb640 1 -- 172.21.15.94:0/3425264493 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1274010c50 con 0x7f12801045b0 2024-02-17T11:57:21.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.080+0000 7f128717a640 1 -- 172.21.15.94:0/3425264493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12801a15b0 con 0x7f12801045b0 2024-02-17T11:57:21.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.080+0000 7f127dffb640 1 -- 172.21.15.94:0/3425264493 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274017e20 con 0x7f12801045b0 2024-02-17T11:57:21.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.080+0000 7f128717a640 1 -- 172.21.15.94:0/3425264493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12801a1b90 con 0x7f12801045b0 2024-02-17T11:57:21.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.081+0000 7f128717a640 1 -- 172.21.15.94:0/3425264493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128010a6e0 con 0x7f12801045b0 2024-02-17T11:57:21.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.082+0000 7f127dffb640 1 -- 172.21.15.94:0/3425264493 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f1274017480 con 0x7f12801045b0 2024-02-17T11:57:21.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.082+0000 7f127dffb640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f12680a21d0 0x7f12680a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:21.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.082+0000 7f1284f16640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f12680a21d0 0x7f12680a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:21.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.083+0000 7f127dffb640 1 -- 172.21.15.94:0/3425264493 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f12740cd3c0 con 0x7f12801045b0 2024-02-17T11:57:21.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.083+0000 7f1284f16640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f12680a21d0 0x7f12680a4690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1270036040 tx=0x7f1270000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:21.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.087+0000 7f127dffb640 1 -- 172.21.15.94:0/3425264493 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1274096340 con 0x7f12801045b0 2024-02-17T11:57:21.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:20 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/896785215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:21.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:20 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3746246468' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:21.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:20 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/20778938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:21.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:20 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/896785215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:21.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:20 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3746246468' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:21.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:20 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/20778938' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.259+0000 7f128717a640 1 -- 172.21.15.94:0/3425264493 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f128002cea0 con 0x7f12801045b0 2024-02-17T11:57:21.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.259+0000 7f127dffb640 1 -- 172.21.15.94:0/3425264493 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1274099a00 con 0x7f12801045b0 2024-02-17T11:57:21.258 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-02-17T11:57:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.261+0000 7f128717a640 1 -- 172.21.15.94:0/3425264493 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f12680a21d0 msgr2=0x7f12680a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.261+0000 7f128717a640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f12680a21d0 0x7f12680a4690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1270036040 tx=0x7f1270000bc0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.261+0000 7f128717a640 1 -- 172.21.15.94:0/3425264493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12801045b0 msgr2=0x7f12801999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.261+0000 7f128717a640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12801045b0 0x7f12801999f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f127400b4a0 tx=0x7f127400b960 comp rx=0 tx=0).stop 2024-02-17T11:57:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.261+0000 7f128717a640 1 -- 172.21.15.94:0/3425264493 shutdown_connections 2024-02-17T11:57:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.261+0000 7f128717a640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f12680a21d0 0x7f12680a4690 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.261+0000 7f128717a640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1280104f70 0x7f12801a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.261+0000 7f128717a640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12801045b0 0x7f12801999f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.261+0000 7f128717a640 1 --2- 172.21.15.94:0/3425264493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12801033c0 0x7f12801994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.261+0000 7f128717a640 1 -- 172.21.15.94:0/3425264493 >> 172.21.15.94:0/3425264493 conn(0x7f12800fe930 msgr2=0x7f12801078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:21.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.261+0000 7f128717a640 1 -- 172.21.15.94:0/3425264493 shutdown_connections 2024-02-17T11:57:21.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.261+0000 7f128717a640 1 -- 172.21.15.94:0/3425264493 wait complete. 2024-02-17T11:57:21.271 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:57:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.374+0000 7f80e388d640 1 -- 172.21.15.94:0/3904646693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc0ffa10 msgr2=0x7f80dc101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.374+0000 7f80e388d640 1 --2- 172.21.15.94:0/3904646693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc0ffa10 0x7f80dc101e40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f80d4009ca0 tx=0x7f80d402c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.374+0000 7f80e388d640 1 -- 172.21.15.94:0/3904646693 shutdown_connections 2024-02-17T11:57:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.374+0000 7f80e388d640 1 --2- 172.21.15.94:0/3904646693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80dc104d30 0x7f80dc1051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.374+0000 7f80e388d640 1 --2- 172.21.15.94:0/3904646693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80dc102380 0x7f80dc1047f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.374+0000 7f80e388d640 1 --2- 172.21.15.94:0/3904646693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc0ffa10 0x7f80dc101e40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.374+0000 7f80e388d640 1 -- 172.21.15.94:0/3904646693 >> 172.21.15.94:0/3904646693 conn(0x7f80dc0fb610 msgr2=0x7f80dc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.374+0000 7f80e388d640 1 -- 172.21.15.94:0/3904646693 shutdown_connections 2024-02-17T11:57:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.374+0000 7f80e388d640 1 -- 172.21.15.94:0/3904646693 wait complete. 2024-02-17T11:57:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.374+0000 7f80e388d640 1 Processor -- start 2024-02-17T11:57:21.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.374+0000 7f80e388d640 1 -- start start 2024-02-17T11:57:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e388d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80dc0ffa10 0x7f80dc19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e388d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc102380 0x7f80dc19dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e388d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80dc104d30 0x7f80dc1a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e388d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc1a52c0 con 0x7f80dc102380 2024-02-17T11:57:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e388d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc1a5430 con 0x7f80dc104d30 2024-02-17T11:57:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e388d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc1a55f0 con 0x7f80dc0ffa10 2024-02-17T11:57:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e0e28640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc102380 0x7f80dc19dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e0e28640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc102380 0x7f80dc19dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42914/0 (socket says 172.21.15.94:42914) 2024-02-17T11:57:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e0e28640 1 -- 172.21.15.94:0/1072914410 learned_addr learned my addr 172.21.15.94:0/1072914410 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e1629640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80dc0ffa10 0x7f80dc19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e1e2a640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80dc104d30 0x7f80dc1a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e0e28640 1 -- 172.21.15.94:0/1072914410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80dc0ffa10 msgr2=0x7f80dc19d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e0e28640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80dc0ffa10 0x7f80dc19d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e0e28640 1 -- 172.21.15.94:0/1072914410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80dc104d30 msgr2=0x7f80dc1a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e0e28640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80dc104d30 0x7f80dc1a4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e0e28640 1 -- 172.21.15.94:0/1072914410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d4009900 con 0x7f80dc102380 2024-02-17T11:57:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e1e2a640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80dc104d30 0x7f80dc1a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e0e28640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc102380 0x7f80dc19dd10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c920 tx=0x7f80cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80d27fc640 1 -- 172.21.15.94:0/1072914410 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc007c70 con 0x7f80dc102380 2024-02-17T11:57:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e388d640 1 -- 172.21.15.94:0/1072914410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80dc1a58d0 con 0x7f80dc102380 2024-02-17T11:57:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80d27fc640 1 -- 172.21.15.94:0/1072914410 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80cc007dd0 con 0x7f80dc102380 2024-02-17T11:57:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80d27fc640 1 -- 172.21.15.94:0/1072914410 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc018860 con 0x7f80dc102380 2024-02-17T11:57:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.375+0000 7f80e388d640 1 -- 172.21.15.94:0/1072914410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80dc1a5de0 con 0x7f80dc102380 2024-02-17T11:57:21.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.376+0000 7f80e388d640 1 -- 172.21.15.94:0/1072914410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80dc0fd100 con 0x7f80dc102380 2024-02-17T11:57:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.377+0000 7f80d27fc640 1 -- 172.21.15.94:0/1072914410 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f80cc00fac0 con 0x7f80dc102380 2024-02-17T11:57:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.378+0000 7f80d27fc640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f80c80a2150 0x7f80c80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.378+0000 7f80e1629640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f80c80a2150 0x7f80c80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.378+0000 7f80d27fc640 1 -- 172.21.15.94:0/1072914410 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f80cc0cdfa0 con 0x7f80dc102380 2024-02-17T11:57:21.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.378+0000 7f80e1629640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f80c80a2150 0x7f80c80a4610 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f80d402c700 tx=0x7f80d4032e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:21.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.382+0000 7f80d27fc640 1 -- 172.21.15.94:0/1072914410 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f80cc096e10 con 0x7f80dc102380 2024-02-17T11:57:21.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.594+0000 7f80e388d640 1 -- 172.21.15.94:0/1072914410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80dc113190 con 0x7f80dc102380 2024-02-17T11:57:21.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.594+0000 7f80d27fc640 1 -- 172.21.15.94:0/1072914410 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f80cc09a4d0 con 0x7f80dc102380 2024-02-17T11:57:21.593 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:21.594 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:56:02.151161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T11:57:21.594 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 20 2024-02-17T11:57:21.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.596+0000 7f80e388d640 1 -- 172.21.15.94:0/1072914410 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f80c80a2150 msgr2=0x7f80c80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.596+0000 7f80e388d640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f80c80a2150 0x7f80c80a4610 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f80d402c700 tx=0x7f80d4032e50 comp rx=0 tx=0).stop 2024-02-17T11:57:21.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.596+0000 7f80e388d640 1 -- 172.21.15.94:0/1072914410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc102380 msgr2=0x7f80dc19dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.596+0000 7f80e388d640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc102380 0x7f80dc19dd10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f80cc00c920 tx=0x7f80cc00cde0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.596+0000 7f80e388d640 1 -- 172.21.15.94:0/1072914410 shutdown_connections 2024-02-17T11:57:21.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.596+0000 7f80e388d640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f80c80a2150 0x7f80c80a4610 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.596+0000 7f80e388d640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f80dc104d30 0x7f80dc1a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.596+0000 7f80e388d640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f80dc102380 0x7f80dc19dd10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.596+0000 7f80e388d640 1 --2- 172.21.15.94:0/1072914410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80dc0ffa10 0x7f80dc19d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.596+0000 7f80e388d640 1 -- 172.21.15.94:0/1072914410 >> 172.21.15.94:0/1072914410 conn(0x7f80dc0fb610 msgr2=0x7f80dc102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:21.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.596+0000 7f80e388d640 1 -- 172.21.15.94:0/1072914410 shutdown_connections 2024-02-17T11:57:21.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.596+0000 7f80e388d640 1 -- 172.21.15.94:0/1072914410 wait complete. 2024-02-17T11:57:21.606 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:57:21.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.709+0000 7fb416f20640 1 -- 172.21.15.94:0/3035635750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb410101c50 msgr2=0x7fb4101084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.709+0000 7fb416f20640 1 --2- 172.21.15.94:0/3035635750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb410101c50 0x7fb4101084e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb408009ca0 tx=0x7fb40802c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.709+0000 7fb416f20640 1 -- 172.21.15.94:0/3035635750 shutdown_connections 2024-02-17T11:57:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.709+0000 7fb416f20640 1 --2- 172.21.15.94:0/3035635750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb410101c50 0x7fb4101084e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.709+0000 7fb416f20640 1 --2- 172.21.15.94:0/3035635750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb410101290 0x7fb410101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.709+0000 7fb416f20640 1 --2- 172.21.15.94:0/3035635750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb4101000a0 0x7fb4101004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.709+0000 7fb416f20640 1 -- 172.21.15.94:0/3035635750 >> 172.21.15.94:0/3035635750 conn(0x7fb4100fb610 msgr2=0x7fb4100fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.709+0000 7fb416f20640 1 -- 172.21.15.94:0/3035635750 shutdown_connections 2024-02-17T11:57:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.709+0000 7fb416f20640 1 -- 172.21.15.94:0/3035635750 wait complete. 2024-02-17T11:57:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb416f20640 1 Processor -- start 2024-02-17T11:57:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb416f20640 1 -- start start 2024-02-17T11:57:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb416f20640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb4101000a0 0x7fb410197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:21.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb416f20640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb410101290 0x7fb410197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb416f20640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb410101c50 0x7fb41019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb416f20640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41019ecf0 con 0x7fb410101c50 2024-02-17T11:57:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb416f20640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41019ee60 con 0x7fb4101000a0 2024-02-17T11:57:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb416f20640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41019f020 con 0x7fb410101290 2024-02-17T11:57:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb4154bd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb410101c50 0x7fb41019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb4154bd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb410101c50 0x7fb41019e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42926/0 (socket says 172.21.15.94:42926) 2024-02-17T11:57:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb4154bd640 1 -- 172.21.15.94:0/3604970860 learned_addr learned my addr 172.21.15.94:0/3604970860 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb414cbc640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb4101000a0 0x7fb410197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb40ffff640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb410101290 0x7fb410197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb4154bd640 1 -- 172.21.15.94:0/3604970860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb410101290 msgr2=0x7fb410197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.710+0000 7fb4154bd640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb410101290 0x7fb410197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.711+0000 7fb4154bd640 1 -- 172.21.15.94:0/3604970860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb4101000a0 msgr2=0x7fb410197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.711+0000 7fb4154bd640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb4101000a0 0x7fb410197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.711+0000 7fb4154bd640 1 -- 172.21.15.94:0/3604970860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb408009900 con 0x7fb410101c50 2024-02-17T11:57:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.711+0000 7fb414cbc640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb4101000a0 0x7fb410197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.711+0000 7fb4154bd640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb410101c50 0x7fb41019e7b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb408037040 tx=0x7fb408034d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.711+0000 7fb40dffb640 1 -- 172.21.15.94:0/3604970860 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408032eb0 con 0x7fb410101c50 2024-02-17T11:57:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.711+0000 7fb40dffb640 1 -- 172.21.15.94:0/3604970860 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb408003e50 con 0x7fb410101c50 2024-02-17T11:57:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.711+0000 7fb416f20640 1 -- 172.21.15.94:0/3604970860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41019f2a0 con 0x7fb410101c50 2024-02-17T11:57:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.711+0000 7fb40dffb640 1 -- 172.21.15.94:0/3604970860 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408040450 con 0x7fb410101c50 2024-02-17T11:57:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.711+0000 7fb416f20640 1 -- 172.21.15.94:0/3604970860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41019f750 con 0x7fb410101c50 2024-02-17T11:57:21.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.712+0000 7fb416f20640 1 -- 172.21.15.94:0/3604970860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4101073c0 con 0x7fb410101c50 2024-02-17T11:57:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.713+0000 7fb40dffb640 1 -- 172.21.15.94:0/3604970860 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fb40800a040 con 0x7fb410101c50 2024-02-17T11:57:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.714+0000 7fb40dffb640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb4000a2150 0x7fb4000a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.714+0000 7fb414cbc640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb4000a2150 0x7fb4000a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:21.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.714+0000 7fb40dffb640 1 -- 172.21.15.94:0/3604970860 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb4080f0d60 con 0x7fb410101c50 2024-02-17T11:57:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.714+0000 7fb414cbc640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb4000a2150 0x7fb4000a4610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb4040058c0 tx=0x7fb404005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:21.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.718+0000 7fb40dffb640 1 -- 172.21.15.94:0/3604970860 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb4080b9ce0 con 0x7fb410101c50 2024-02-17T11:57:21.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.893+0000 7fb416f20640 1 -- 172.21.15.94:0/3604970860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb41002ce40 con 0x7fb410101c50 2024-02-17T11:57:21.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.894+0000 7fb40dffb640 1 -- 172.21.15.94:0/3604970860 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22686 (secure 0 0 0) 0x7fb4080bd3a0 con 0x7fb410101c50 2024-02-17T11:57:21.893 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:21.893 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":87,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:56:59.725646+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:57:21.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.896+0000 7fb416f20640 1 -- 172.21.15.94:0/3604970860 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb4000a2150 msgr2=0x7fb4000a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.896+0000 7fb416f20640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb4000a2150 0x7fb4000a4610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb4040058c0 tx=0x7fb404005850 comp rx=0 tx=0).stop 2024-02-17T11:57:21.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.896+0000 7fb416f20640 1 -- 172.21.15.94:0/3604970860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb410101c50 msgr2=0x7fb41019e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:21.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.896+0000 7fb416f20640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb410101c50 0x7fb41019e7b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb408037040 tx=0x7fb408034d40 comp rx=0 tx=0).stop 2024-02-17T11:57:21.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.896+0000 7fb416f20640 1 -- 172.21.15.94:0/3604970860 shutdown_connections 2024-02-17T11:57:21.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.896+0000 7fb416f20640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb4000a2150 0x7fb4000a4610 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.896+0000 7fb416f20640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb410101c50 0x7fb41019e7b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.896+0000 7fb416f20640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb410101290 0x7fb410197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.896+0000 7fb416f20640 1 --2- 172.21.15.94:0/3604970860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb4101000a0 0x7fb410197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:21.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.896+0000 7fb416f20640 1 -- 172.21.15.94:0/3604970860 >> 172.21.15.94:0/3604970860 conn(0x7fb4100fb610 msgr2=0x7fb410104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:21.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.896+0000 7fb416f20640 1 -- 172.21.15.94:0/3604970860 shutdown_connections 2024-02-17T11:57:21.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:21.896+0000 7fb416f20640 1 -- 172.21.15.94:0/3604970860 wait complete. 2024-02-17T11:57:21.905 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:21 smithi094 ceph-mon[32499]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-17T11:57:21.905 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:21 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2947910460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:21.905 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:21 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3425264493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:21.905 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:21 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1072914410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:21.906 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:57:22.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.008+0000 7f4c38d8a640 1 -- 172.21.15.94:0/2578768960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c341011a0 msgr2=0x7f4c341015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.008+0000 7f4c38d8a640 1 --2- 172.21.15.94:0/2578768960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c341011a0 0x7f4c341015c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4c28009900 tx=0x7f4c2802bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:22.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 -- 172.21.15.94:0/2578768960 shutdown_connections 2024-02-17T11:57:22.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 --2- 172.21.15.94:0/2578768960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c34102d50 0x7f4c341095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 --2- 172.21.15.94:0/2578768960 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c34102390 0x7f4c34102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 --2- 172.21.15.94:0/2578768960 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c341011a0 0x7f4c341015c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 -- 172.21.15.94:0/2578768960 >> 172.21.15.94:0/2578768960 conn(0x7f4c340fc730 msgr2=0x7f4c340feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 -- 172.21.15.94:0/2578768960 shutdown_connections 2024-02-17T11:57:22.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 -- 172.21.15.94:0/2578768960 wait complete. 2024-02-17T11:57:22.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 Processor -- start 2024-02-17T11:57:22.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 -- start start 2024-02-17T11:57:22.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c341011a0 0x7f4c34074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c34102390 0x7f4c34073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c34102d50 0x7f4c34073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34074210 con 0x7f4c34102d50 2024-02-17T11:57:22.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34074380 con 0x7f4c341011a0 2024-02-17T11:57:22.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.009+0000 7f4c38d8a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c341a1300 con 0x7f4c34102390 2024-02-17T11:57:22.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c32d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c34102d50 0x7f4c34073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c32d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c34102d50 0x7f4c34073ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42936/0 (socket says 172.21.15.94:42936) 2024-02-17T11:57:22.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c32d9d640 1 -- 172.21.15.94:0/3383773257 learned_addr learned my addr 172.21.15.94:0/3383773257 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c3259c640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c341011a0 0x7f4c34074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c31d9b640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c34102390 0x7f4c34073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c32d9d640 1 -- 172.21.15.94:0/3383773257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c34102390 msgr2=0x7f4c34073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c32d9d640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c34102390 0x7f4c34073570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c32d9d640 1 -- 172.21.15.94:0/3383773257 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c341011a0 msgr2=0x7f4c34074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c32d9d640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c341011a0 0x7f4c34074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c32d9d640 1 -- 172.21.15.94:0/3383773257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c28009560 con 0x7f4c34102d50 2024-02-17T11:57:22.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c3259c640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c341011a0 0x7f4c34074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:22.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c31d9b640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c34102390 0x7f4c34073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:22.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c32d9d640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c34102d50 0x7f4c34073ab0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00b9a0 tx=0x7f4c2c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c237fe640 1 -- 172.21.15.94:0/3383773257 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00cba0 con 0x7f4c34102d50 2024-02-17T11:57:22.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c38d8a640 1 -- 172.21.15.94:0/3383773257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c341a15b0 con 0x7f4c34102d50 2024-02-17T11:57:22.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c237fe640 1 -- 172.21.15.94:0/3383773257 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4c2c00cd00 con 0x7f4c34102d50 2024-02-17T11:57:22.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c237fe640 1 -- 172.21.15.94:0/3383773257 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0076f0 con 0x7f4c34102d50 2024-02-17T11:57:22.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.010+0000 7f4c38d8a640 1 -- 172.21.15.94:0/3383773257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c341a1b40 con 0x7f4c34102d50 2024-02-17T11:57:22.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.011+0000 7f4c38d8a640 1 -- 172.21.15.94:0/3383773257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c341084c0 con 0x7f4c34102d50 2024-02-17T11:57:22.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.012+0000 7f4c237fe640 1 -- 172.21.15.94:0/3383773257 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f4c2c014a20 con 0x7f4c34102d50 2024-02-17T11:57:22.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.013+0000 7f4c237fe640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4c1c0a2150 0x7f4c1c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.013+0000 7f4c3259c640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4c1c0a2150 0x7f4c1c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.013+0000 7f4c237fe640 1 -- 172.21.15.94:0/3383773257 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4c2c0cd6a0 con 0x7f4c34102d50 2024-02-17T11:57:22.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.013+0000 7f4c3259c640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4c1c0a2150 0x7f4c1c0a4610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c28006010 tx=0x7f4c2802c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.017+0000 7f4c237fe640 1 -- 172.21.15.94:0/3383773257 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c2c04fd20 con 0x7f4c34102d50 2024-02-17T11:57:22.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:21 smithi189 ceph-mon[36588]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-17T11:57:22.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:21 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2947910460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:22.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:21 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3425264493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:22.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:21 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1072914410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:22.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:21 smithi125 ceph-mon[36819]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-17T11:57:22.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:21 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2947910460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:22.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:21 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3425264493' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:22.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:21 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1072914410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:22.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.226+0000 7f4c38d8a640 1 -- 172.21.15.94:0/3383773257 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c3402cdd0 con 0x7f4c34102d50 2024-02-17T11:57:22.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.227+0000 7f4c237fe640 1 -- 172.21.15.94:0/3383773257 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f4c2c096620 con 0x7f4c34102d50 2024-02-17T11:57:22.226 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:22.226 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:56:02.151161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T11:57:22.226 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 20 2024-02-17T11:57:22.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.229+0000 7f4c38d8a640 1 -- 172.21.15.94:0/3383773257 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4c1c0a2150 msgr2=0x7f4c1c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.229+0000 7f4c38d8a640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4c1c0a2150 0x7f4c1c0a4610 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c28006010 tx=0x7f4c2802c800 comp rx=0 tx=0).stop 2024-02-17T11:57:22.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.229+0000 7f4c38d8a640 1 -- 172.21.15.94:0/3383773257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c34102d50 msgr2=0x7f4c34073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.229+0000 7f4c38d8a640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c34102d50 0x7f4c34073ab0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c00b9a0 tx=0x7f4c2c00be60 comp rx=0 tx=0).stop 2024-02-17T11:57:22.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.229+0000 7f4c38d8a640 1 -- 172.21.15.94:0/3383773257 shutdown_connections 2024-02-17T11:57:22.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.229+0000 7f4c38d8a640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4c1c0a2150 0x7f4c1c0a4610 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.229+0000 7f4c38d8a640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4c34102d50 0x7f4c34073ab0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.229+0000 7f4c38d8a640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4c34102390 0x7f4c34073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.229+0000 7f4c38d8a640 1 --2- 172.21.15.94:0/3383773257 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4c341011a0 0x7f4c34074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.229+0000 7f4c38d8a640 1 -- 172.21.15.94:0/3383773257 >> 172.21.15.94:0/3383773257 conn(0x7f4c340fc730 msgr2=0x7f4c34105690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.229+0000 7f4c38d8a640 1 -- 172.21.15.94:0/3383773257 shutdown_connections 2024-02-17T11:57:22.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.229+0000 7f4c38d8a640 1 -- 172.21.15.94:0/3383773257 wait complete. 2024-02-17T11:57:22.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-17T11:55:58.000681+0000', 'modified': '2024-02-17T11:56:02.151161+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24488, 'mds_1': 24487, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.125:6835/1238777175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6834', 'nonce': 1238777175}, {'type': 'v1', 'addr': '172.21.15.125:6835', 'nonce': 1238777175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.125:6837/1075914834', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6836', 'nonce': 1075914834}, {'type': 'v1', 'addr': '172.21.15.125:6837', 'nonce': 1075914834}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6835/2720928164', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 2720928164}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 2720928164}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.189:6837/860722582', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 860722582}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 860722582}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 16, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.189:6839/308580523', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 308580523}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 308580523}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6839/1158324088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6838', 'nonce': 1158324088}, {'type': 'v1', 'addr': '172.21.15.125:6839', 'nonce': 1158324088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-17T11:57:22.239 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-17T11:57:22.240 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-17T11:57:22.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.342+0000 7f859d042640 1 -- 172.21.15.94:0/1341316140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8598101c30 msgr2=0x7f85981084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.342+0000 7f859d042640 1 --2- 172.21.15.94:0/1341316140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8598101c30 0x7f85981084c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8590009ca0 tx=0x7f859002c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.342+0000 7f859d042640 1 -- 172.21.15.94:0/1341316140 shutdown_connections 2024-02-17T11:57:22.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.342+0000 7f859d042640 1 --2- 172.21.15.94:0/1341316140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8598101c30 0x7f85981084c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.342+0000 7f859d042640 1 --2- 172.21.15.94:0/1341316140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8598101270 0x7f85981016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.342+0000 7f859d042640 1 --2- 172.21.15.94:0/1341316140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8598100080 0x7f85981004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.342+0000 7f859d042640 1 -- 172.21.15.94:0/1341316140 >> 172.21.15.94:0/1341316140 conn(0x7f85980fb610 msgr2=0x7f85980fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.342+0000 7f859d042640 1 -- 172.21.15.94:0/1341316140 shutdown_connections 2024-02-17T11:57:22.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.342+0000 7f859d042640 1 -- 172.21.15.94:0/1341316140 wait complete. 2024-02-17T11:57:22.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f859d042640 1 Processor -- start 2024-02-17T11:57:22.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f859d042640 1 -- start start 2024-02-17T11:57:22.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f859d042640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8598100080 0x7f85981971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f859d042640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8598101270 0x7f8598197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f859d042640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8598101c30 0x7f859819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f859d042640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859819ecd0 con 0x7f8598100080 2024-02-17T11:57:22.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f859d042640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859819ee40 con 0x7f8598101c30 2024-02-17T11:57:22.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f859d042640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f859819f000 con 0x7f8598101270 2024-02-17T11:57:22.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f8596d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8598100080 0x7f85981971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f8596d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8598100080 0x7f85981971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42968/0 (socket says 172.21.15.94:42968) 2024-02-17T11:57:22.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f8596d9d640 1 -- 172.21.15.94:0/1853884695 learned_addr learned my addr 172.21.15.94:0/1853884695 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f859759e640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8598101c30 0x7f859819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f858ffff640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8598101270 0x7f8598197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f8596d9d640 1 -- 172.21.15.94:0/1853884695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8598101270 msgr2=0x7f8598197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f8596d9d640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8598101270 0x7f8598197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f8596d9d640 1 -- 172.21.15.94:0/1853884695 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8598101c30 msgr2=0x7f859819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f8596d9d640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8598101c30 0x7f859819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f8596d9d640 1 -- 172.21.15.94:0/1853884695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8590009900 con 0x7f8598100080 2024-02-17T11:57:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f858ffff640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8598101270 0x7f8598197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.343+0000 7f8596d9d640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8598100080 0x7f85981971e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f858800d950 tx=0x7f858800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.344+0000 7f8594d99640 1 -- 172.21.15.94:0/1853884695 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8588009c30 con 0x7f8598100080 2024-02-17T11:57:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.344+0000 7f859d042640 1 -- 172.21.15.94:0/1853884695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859819f2e0 con 0x7f8598100080 2024-02-17T11:57:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.344+0000 7f8594d99640 1 -- 172.21.15.94:0/1853884695 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8588009d90 con 0x7f8598100080 2024-02-17T11:57:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.344+0000 7f8594d99640 1 -- 172.21.15.94:0/1853884695 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858800f720 con 0x7f8598100080 2024-02-17T11:57:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.344+0000 7f859d042640 1 -- 172.21.15.94:0/1853884695 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859819f7f0 con 0x7f8598100080 2024-02-17T11:57:22.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.346+0000 7f8594d99640 1 -- 172.21.15.94:0/1853884695 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f8588010a60 con 0x7f8598100080 2024-02-17T11:57:22.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.346+0000 7f859fc0b740 1 Processor -- start 2024-02-17T11:57:22.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.346+0000 7f859fc0b740 1 -- start start 2024-02-17T11:57:22.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859fc0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 0x5565f6e573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859fc0b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e57970 0x5565f6e5c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859fc0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e5cae0 0x5565f6e5ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859fc0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f6e12cd0 con 0x5565f6e56f80 2024-02-17T11:57:22.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859fc0b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f6dc5f20 con 0x5565f6e57970 2024-02-17T11:57:22.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859fc0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f6dc3450 con 0x5565f6e5cae0 2024-02-17T11:57:22.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f8594d99640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f85840a2150 0x7f85840a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 0x5565f6e573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 0x5565f6e573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42970/0 (socket says 172.21.15.94:42970) 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859759e640 1 -- 172.21.15.94:0/1982236728 learned_addr learned my addr 172.21.15.94:0/1982236728 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859759e640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f85840a2150 0x7f85840a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f8594d99640 1 -- 172.21.15.94:0/1853884695 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f85880cdc30 con 0x7f8598100080 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f858ffff640 1 --2- 172.21.15.94:0/1982236728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e5cae0 0x5565f6e5ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f8596d9d640 1 --2- 172.21.15.94:0/1982236728 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e57970 0x5565f6e5c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859759e640 1 -- 172.21.15.94:0/1982236728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e5cae0 msgr2=0x5565f6e5ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859759e640 1 --2- 172.21.15.94:0/1982236728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e5cae0 0x5565f6e5ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859759e640 1 -- 172.21.15.94:0/1982236728 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e57970 msgr2=0x5565f6e5c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859759e640 1 --2- 172.21.15.94:0/1982236728 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e57970 0x5565f6e5c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859759e640 1 -- 172.21.15.94:0/1982236728 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565f6d73f20 con 0x5565f6e56f80 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.347+0000 7f859759e640 1 --2- 172.21.15.94:0/1982236728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 0x5565f6e573a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f85900325e0 tx=0x7f85900347f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc812df7b7e0524d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.348+0000 7f859759e640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f85840a2150 0x7f85840a4610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8590009ca0 tx=0x7f859003c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.348+0000 7f858d7fa640 1 -- 172.21.15.94:0/1982236728 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590048070 con 0x5565f6e56f80 2024-02-17T11:57:22.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.348+0000 7f858d7fa640 1 -- 172.21.15.94:0/1982236728 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8590034560 con 0x5565f6e56f80 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.348+0000 7f858d7fa640 1 -- 172.21.15.94:0/1982236728 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590004ea0 con 0x5565f6e56f80 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.348+0000 7f859fc0b740 1 -- 172.21.15.94:0/1982236728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 msgr2=0x5565f6e573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.348+0000 7f859fc0b740 1 --2- 172.21.15.94:0/1982236728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 0x5565f6e573a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f85900325e0 tx=0x7f85900347f0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.348+0000 7f859fc0b740 1 -- 172.21.15.94:0/1982236728 shutdown_connections 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.348+0000 7f859fc0b740 1 --2- 172.21.15.94:0/1982236728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e5cae0 0x5565f6e5ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.348+0000 7f859fc0b740 1 --2- 172.21.15.94:0/1982236728 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e57970 0x5565f6e5c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.348+0000 7f859fc0b740 1 --2- 172.21.15.94:0/1982236728 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 0x5565f6e573a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.348+0000 7f859fc0b740 1 -- 172.21.15.94:0/1982236728 >> 172.21.15.94:0/1982236728 conn(0x5565f6e524f0 msgr2=0x5565f6e528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f859fc0b740 1 -- 172.21.15.94:0/1982236728 shutdown_connections 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f859fc0b740 1 -- 172.21.15.94:0/1982236728 wait complete. 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f859fc0b740 1 -- start start 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f859fc0b740 1 Processor -- start 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f859fc0b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e56f80 0x5565f6e61190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f859fc0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e57970 0x5565f6e616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f859fc0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e5cae0 0x5565f6e68710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f859fc0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f6dbec30 con 0x5565f6e57970 2024-02-17T11:57:22.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f859fc0b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f6d63d30 con 0x5565f6e56f80 2024-02-17T11:57:22.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f859fc0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f6a92310 con 0x5565f6e5cae0 2024-02-17T11:57:22.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f859fc0b740 10 client.0 _ll_register_callbacks cb 0x5565f6d6bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:22.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f858ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e57970 0x5565f6e616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f858ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e57970 0x5565f6e616d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42982/0 (socket says 172.21.15.94:42982) 2024-02-17T11:57:22.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f858ffff640 1 -- 172.21.15.94:0/2133818160 learned_addr learned my addr 172.21.15.94:0/2133818160 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f8596d9d640 1 --2- 172.21.15.94:0/2133818160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e56f80 0x5565f6e61190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f859759e640 1 --2- 172.21.15.94:0/2133818160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e5cae0 0x5565f6e68710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f858ffff640 1 -- 172.21.15.94:0/2133818160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e5cae0 msgr2=0x5565f6e68710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f858ffff640 1 --2- 172.21.15.94:0/2133818160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e5cae0 0x5565f6e68710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f858ffff640 1 -- 172.21.15.94:0/2133818160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e56f80 msgr2=0x5565f6e61190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f858ffff640 1 --2- 172.21.15.94:0/2133818160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e56f80 0x5565f6e61190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.349+0000 7f858ffff640 1 -- 172.21.15.94:0/2133818160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8598108dc0 con 0x5565f6e57970 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f858ffff640 1 --2- 172.21.15.94:0/2133818160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e57970 0x5565f6e616d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8580009d40 tx=0x7f85800039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48d0b0d3d709554 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f857e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f857e7fc640 1 -- 172.21.15.94:0/2133818160 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580012860 con 0x5565f6e57970 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f857e7fc640 1 -- 172.21.15.94:0/2133818160 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f85800129c0 con 0x5565f6e57970 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f859fc0b740 1 -- 172.21.15.94:0/2133818160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565f6b0f270 con 0x5565f6e57970 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f857e7fc640 1 -- 172.21.15.94:0/2133818160 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8580012b80 con 0x5565f6e57970 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f859fc0b740 10 client.15039 fetch_fsmap learned FSMap version 20 2024-02-17T11:57:22.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f859fc0b740 1 -- 172.21.15.94:0/2133818160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5565f6dbd2b0 con 0x5565f6e57970 2024-02-17T11:57:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f857e7fc640 1 -- 172.21.15.94:0/2133818160 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5565f6b0f270 con 0x5565f6e57970 2024-02-17T11:57:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f857e7fc640 1 -- 172.21.15.94:0/2133818160 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8580011450 con 0x5565f6e57970 2024-02-17T11:57:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f859fc0b740 10 client.15039 fetch_fsmap finished waiting for FSMap version 20 2024-02-17T11:57:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f859fc0b740 10 client.15039 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-17T11:57:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f859fc0b740 20 client.15039 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f859fc0b740 1 --2- 172.21.15.94:0/2133818160 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5565f6e6ce90 0x5565f6e71f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f859fc0b740 4 client.15039 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:57:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.350+0000 7f859fc0b740 1 -- 172.21.15.94:0/2133818160 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565f6afb610 con 0x5565f6e6ce90 2024-02-17T11:57:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.351+0000 7f8596d9d640 1 --2- 172.21.15.94:0/2133818160 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5565f6e6ce90 0x5565f6e71f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.351+0000 7f857e7fc640 10 client.15039 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T11:57:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.351+0000 7f8596d9d640 1 --2- 172.21.15.94:0/2133818160 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5565f6e6ce90 0x5565f6e71f10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb070510a7a1f6f3 server_cookie=e455ad315867df83 in_seq=0 out_seq=0 2024-02-17T11:57:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f857e7fc640 1 -- 172.21.15.94:0/2133818160 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565f6afb610 con 0x5565f6e6ce90 2024-02-17T11:57:22.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f857e7fc640 10 client.15039 handle_command_reply: tid=0 2024-02-17T11:57:22.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f859fc0b740 1 client.15039 shutdown 2024-02-17T11:57:22.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f859fc0b740 1 -- 172.21.15.94:0/2133818160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e57970 msgr2=0x5565f6e616d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f859fc0b740 1 --2- 172.21.15.94:0/2133818160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e57970 0x5565f6e616d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8580009d40 tx=0x7f85800039c0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f859fc0b740 1 -- 172.21.15.94:0/2133818160 shutdown_connections 2024-02-17T11:57:22.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f859fc0b740 1 --2- 172.21.15.94:0/2133818160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e5cae0 0x5565f6e68710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f859fc0b740 1 --2- 172.21.15.94:0/2133818160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e57970 0x5565f6e616d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f859fc0b740 1 --2- 172.21.15.94:0/2133818160 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5565f6e6ce90 0x5565f6e71f10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f859fc0b740 1 --2- 172.21.15.94:0/2133818160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e56f80 0x5565f6e61190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f857e7fc640 0 client.15039 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T11:57:22.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f859fc0b740 1 -- 172.21.15.94:0/2133818160 >> 172.21.15.94:0/2133818160 conn(0x5565f6e524f0 msgr2=0x5565f6e517d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f859fc0b740 1 -- 172.21.15.94:0/2133818160 shutdown_connections 2024-02-17T11:57:22.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f859fc0b740 1 -- 172.21.15.94:0/2133818160 wait complete. 2024-02-17T11:57:22.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.352+0000 7f859fc0b740 20 client.15039 trim_cache size 0 max 16384 2024-02-17T11:57:22.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.367+0000 7f859fc0b740 1 Processor -- start 2024-02-17T11:57:22.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.367+0000 7f859fc0b740 1 -- start start 2024-02-17T11:57:22.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f859fc0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 0x5565f6e528d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f859fc0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e57970 0x5565f6e686d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f859fc0b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e5cae0 0x5565f6e60da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f859fc0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f6d44490 con 0x5565f6e56f80 2024-02-17T11:57:22.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f859fc0b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f6e69390 con 0x5565f6e5cae0 2024-02-17T11:57:22.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f859fc0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f6afb610 con 0x5565f6e57970 2024-02-17T11:57:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f8596d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 0x5565f6e528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f8596d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 0x5565f6e528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42988/0 (socket says 172.21.15.94:42988) 2024-02-17T11:57:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f8596d9d640 1 -- 172.21.15.94:0/3776953513 learned_addr learned my addr 172.21.15.94:0/3776953513 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f8596d9d640 1 -- 172.21.15.94:0/3776953513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e57970 msgr2=0x5565f6e686d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f8596d9d640 1 --2- 172.21.15.94:0/3776953513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e57970 0x5565f6e686d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f8596d9d640 1 -- 172.21.15.94:0/3776953513 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e5cae0 msgr2=0x5565f6e60da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f8596d9d640 1 --2- 172.21.15.94:0/3776953513 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e5cae0 0x5565f6e60da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f8596d9d640 1 -- 172.21.15.94:0/3776953513 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565f6d59a20 con 0x5565f6e56f80 2024-02-17T11:57:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f8596d9d640 1 --2- 172.21.15.94:0/3776953513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 0x5565f6e528d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f858809a4c0 tx=0x7f85880cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb41677df8e127df server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f858cff9640 1 -- 172.21.15.94:0/3776953513 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858809a7c0 con 0x5565f6e56f80 2024-02-17T11:57:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f858cff9640 1 -- 172.21.15.94:0/3776953513 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f858809e910 con 0x5565f6e56f80 2024-02-17T11:57:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.368+0000 7f858cff9640 1 -- 172.21.15.94:0/3776953513 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85880cb7f0 con 0x5565f6e56f80 2024-02-17T11:57:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.369+0000 7f859fc0b740 1 -- 172.21.15.94:0/3776953513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 msgr2=0x5565f6e528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.369+0000 7f859fc0b740 1 --2- 172.21.15.94:0/3776953513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 0x5565f6e528d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f858809a4c0 tx=0x7f85880cd040 comp rx=0 tx=0).stop 2024-02-17T11:57:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.369+0000 7f859fc0b740 1 -- 172.21.15.94:0/3776953513 shutdown_connections 2024-02-17T11:57:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.369+0000 7f859fc0b740 1 --2- 172.21.15.94:0/3776953513 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e5cae0 0x5565f6e60da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.369+0000 7f859fc0b740 1 --2- 172.21.15.94:0/3776953513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e57970 0x5565f6e686d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.369+0000 7f859fc0b740 1 --2- 172.21.15.94:0/3776953513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e56f80 0x5565f6e528d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.369+0000 7f859fc0b740 1 -- 172.21.15.94:0/3776953513 >> 172.21.15.94:0/3776953513 conn(0x5565f6e524f0 msgr2=0x5565f6e583c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.369+0000 7f859fc0b740 1 -- 172.21.15.94:0/3776953513 shutdown_connections 2024-02-17T11:57:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.369+0000 7f859fc0b740 1 -- 172.21.15.94:0/3776953513 wait complete. 2024-02-17T11:57:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.369+0000 7f859fc0b740 1 -- start start 2024-02-17T11:57:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.369+0000 7f859fc0b740 1 Processor -- start 2024-02-17T11:57:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859fc0b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e56f80 0x5565f6e68530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859fc0b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e57970 0x5565f6e64fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859fc0b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e5cae0 0x5565f6e65510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859fc0b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f6e65b70 con 0x5565f6e5cae0 2024-02-17T11:57:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859fc0b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f6ec3bd0 con 0x5565f6e57970 2024-02-17T11:57:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859fc0b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f6ec3d10 con 0x5565f6e56f80 2024-02-17T11:57:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859fc0b740 10 client.0 _ll_register_callbacks cb 0x7f8580005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e5cae0 0x5565f6e65510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859759e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e5cae0 0x5565f6e65510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42992/0 (socket says 172.21.15.94:42992) 2024-02-17T11:57:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859759e640 1 -- 172.21.15.94:0/1657745256 learned_addr learned my addr 172.21.15.94:0/1657745256 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f858ffff640 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e57970 0x5565f6e64fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f8596d9d640 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e56f80 0x5565f6e68530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859759e640 1 -- 172.21.15.94:0/1657745256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e56f80 msgr2=0x5565f6e68530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859759e640 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e56f80 0x5565f6e68530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859759e640 1 -- 172.21.15.94:0/1657745256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e57970 msgr2=0x5565f6e64fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859759e640 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e57970 0x5565f6e64fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859759e640 1 -- 172.21.15.94:0/1657745256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859819f7f0 con 0x5565f6e5cae0 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f858ffff640 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e57970 0x5565f6e64fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859759e640 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e5cae0 0x5565f6e65510 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f85900095c0 tx=0x7f8590004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737cbafd35bf847c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f857e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f857e7fc640 1 -- 172.21.15.94:0/1657745256 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859005d070 con 0x5565f6e5cae0 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f857e7fc640 1 -- 172.21.15.94:0/1657745256 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8590004040 con 0x5565f6e5cae0 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859fc0b740 1 -- 172.21.15.94:0/1657745256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565f6e6d7a0 con 0x5565f6e5cae0 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f857e7fc640 1 -- 172.21.15.94:0/1657745256 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8590048040 con 0x5565f6e5cae0 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859fc0b740 10 client.15045 fetch_fsmap learned FSMap version 20 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f859fc0b740 1 -- 172.21.15.94:0/1657745256 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5565f6d7f780 con 0x5565f6e5cae0 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.370+0000 7f857e7fc640 1 -- 172.21.15.94:0/1657745256 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8590068020 con 0x5565f6e5cae0 2024-02-17T11:57:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.371+0000 7f857e7fc640 1 -- 172.21.15.94:0/1657745256 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8590060020 con 0x5565f6e5cae0 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.371+0000 7f859fc0b740 10 client.15045 fetch_fsmap finished waiting for FSMap version 20 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.371+0000 7f859fc0b740 10 client.15045 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.371+0000 7f859fc0b740 20 client.15045 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.371+0000 7f859fc0b740 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5565f6ec8e00 0x5565f6ec5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.371+0000 7f859fc0b740 4 client.15045 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.371+0000 7f859fc0b740 1 -- 172.21.15.94:0/1657745256 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5565f6e6e460 con 0x5565f6ec8e00 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.371+0000 7f8596d9d640 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5565f6ec8e00 0x5565f6ec5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.371+0000 7f8596d9d640 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5565f6ec8e00 0x5565f6ec5300 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dcc03b452de92584 server_cookie=cb7dd9ea38e71cdc in_seq=0 out_seq=0 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.371+0000 7f857e7fc640 10 client.15045 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.371+0000 7f857e7fc640 1 -- 172.21.15.94:0/1657745256 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5565f6e6e460 con 0x5565f6ec8e00 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.371+0000 7f857e7fc640 10 client.15045 handle_command_reply: tid=0 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.371+0000 7f859fc0b740 1 client.15045 shutdown 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859fc0b740 1 -- 172.21.15.94:0/1657745256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e5cae0 msgr2=0x5565f6e65510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859fc0b740 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e5cae0 0x5565f6e65510 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f85900095c0 tx=0x7f8590004f70 comp rx=0 tx=0).stop 2024-02-17T11:57:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859fc0b740 1 -- 172.21.15.94:0/1657745256 shutdown_connections 2024-02-17T11:57:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859fc0b740 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565f6e5cae0 0x5565f6e65510 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859fc0b740 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5565f6ec8e00 0x5565f6ec5300 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859fc0b740 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565f6e57970 0x5565f6e64fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859fc0b740 1 --2- 172.21.15.94:0/1657745256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565f6e56f80 0x5565f6e68530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f857e7fc640 0 client.15045 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T11:57:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859fc0b740 1 -- 172.21.15.94:0/1657745256 >> 172.21.15.94:0/1657745256 conn(0x5565f6e524f0 msgr2=0x5565f6e58200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859fc0b740 1 -- 172.21.15.94:0/1657745256 shutdown_connections 2024-02-17T11:57:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859fc0b740 1 -- 172.21.15.94:0/1657745256 wait complete. 2024-02-17T11:57:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859fc0b740 20 client.15045 trim_cache size 0 max 16384 2024-02-17T11:57:22.381 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:57:22.382 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "3bc5cfc8-cd8a-11ee-95ba-87774f69a715", 2024-02-17T11:57:22.382 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 2, 2024-02-17T11:57:22.382 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24469, 2024-02-17T11:57:22.382 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-02-17T11:57:22.382 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-02-17T11:57:22.382 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-02-17T11:57:22.382 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 81.226683312999995, 2024-02-17T11:57:22.382 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 20, 2024-02-17T11:57:22.382 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 87, 2024-02-17T11:57:22.382 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 85, 2024-02-17T11:57:22.382 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 104.094142244 2024-02-17T11:57:22.382 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:57:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859d042640 1 -- 172.21.15.94:0/1853884695 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f85840a2150 msgr2=0x7f85840a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859d042640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f85840a2150 0x7f85840a4610 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8590009ca0 tx=0x7f859003c000 comp rx=0 tx=0).stop 2024-02-17T11:57:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859d042640 1 -- 172.21.15.94:0/1853884695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8598100080 msgr2=0x7f85981971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859d042640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8598100080 0x7f85981971e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f858800d950 tx=0x7f858800de10 comp rx=0 tx=0).stop 2024-02-17T11:57:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859d042640 1 -- 172.21.15.94:0/1853884695 shutdown_connections 2024-02-17T11:57:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859d042640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f85840a2150 0x7f85840a4610 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859d042640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8598101c30 0x7f859819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859d042640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8598101270 0x7f8598197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859d042640 1 --2- 172.21.15.94:0/1853884695 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8598100080 0x7f85981971e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859d042640 1 -- 172.21.15.94:0/1853884695 >> 172.21.15.94:0/1853884695 conn(0x7f85980fb610 msgr2=0x7f85980fd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859d042640 1 -- 172.21.15.94:0/1853884695 shutdown_connections 2024-02-17T11:57:22.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.372+0000 7f859d042640 1 -- 172.21.15.94:0/1853884695 wait complete. 2024-02-17T11:57:22.385 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-17T11:57:22.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.489+0000 7f32317d0640 1 -- 172.21.15.94:0/2603939973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c100360 msgr2=0x7f322c1007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.489+0000 7f32317d0640 1 --2- 172.21.15.94:0/2603939973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c100360 0x7f322c1007e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3214009930 tx=0x7f321402bc20 comp rx=0 tx=0).stop 2024-02-17T11:57:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.489+0000 7f32317d0640 1 -- 172.21.15.94:0/2603939973 shutdown_connections 2024-02-17T11:57:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.489+0000 7f32317d0640 1 --2- 172.21.15.94:0/2603939973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f322c100d20 0x7f322c1085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.489+0000 7f32317d0640 1 --2- 172.21.15.94:0/2603939973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c100360 0x7f322c1007e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.489+0000 7f32317d0640 1 --2- 172.21.15.94:0/2603939973 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f322c101630 0x7f322c101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.489+0000 7f32317d0640 1 -- 172.21.15.94:0/2603939973 >> 172.21.15.94:0/2603939973 conn(0x7f322c0fadb0 msgr2=0x7f322c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.489+0000 7f32317d0640 1 -- 172.21.15.94:0/2603939973 shutdown_connections 2024-02-17T11:57:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.489+0000 7f32317d0640 1 -- 172.21.15.94:0/2603939973 wait complete. 2024-02-17T11:57:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.489+0000 7f32317d0640 1 Processor -- start 2024-02-17T11:57:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.489+0000 7f32317d0640 1 -- start start 2024-02-17T11:57:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f32317d0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f322c100360 0x7f322c0ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f32317d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f322c100d20 0x7f322c19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f32317d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c101630 0x7f322c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f32317d0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c19ed60 con 0x7f322c101630 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f32317d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c19eed0 con 0x7f322c100d20 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f32317d0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c19f090 con 0x7f322c100360 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f322b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c101630 0x7f322c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f322b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c101630 0x7f322c19e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43012/0 (socket says 172.21.15.94:43012) 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f322b7fe640 1 -- 172.21.15.94:0/1496187533 learned_addr learned my addr 172.21.15.94:0/1496187533 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f322affd640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f322c100360 0x7f322c0ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f322a7fc640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f322c100d20 0x7f322c19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f322b7fe640 1 -- 172.21.15.94:0/1496187533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f322c100360 msgr2=0x7f322c0ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f322b7fe640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f322c100360 0x7f322c0ffdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f322b7fe640 1 -- 172.21.15.94:0/1496187533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f322c100d20 msgr2=0x7f322c19bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f322b7fe640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f322c100d20 0x7f322c19bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f322b7fe640 1 -- 172.21.15.94:0/1496187533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3214009560 con 0x7f322c101630 2024-02-17T11:57:22.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f322a7fc640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f322c100d20 0x7f322c19bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:22.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f322b7fe640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c101630 0x7f322c19e580 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f321c00cce0 tx=0x7f321c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f3223fff640 1 -- 172.21.15.94:0/1496187533 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c00f450 con 0x7f322c101630 2024-02-17T11:57:22.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f32317d0640 1 -- 172.21.15.94:0/1496187533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c19f310 con 0x7f322c101630 2024-02-17T11:57:22.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f3223fff640 1 -- 172.21.15.94:0/1496187533 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f321c00fa90 con 0x7f322c101630 2024-02-17T11:57:22.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f3223fff640 1 -- 172.21.15.94:0/1496187533 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c018b90 con 0x7f322c101630 2024-02-17T11:57:22.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.490+0000 7f32317d0640 1 -- 172.21.15.94:0/1496187533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c19f820 con 0x7f322c101630 2024-02-17T11:57:22.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.492+0000 7f3223fff640 1 -- 172.21.15.94:0/1496187533 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f321c00fc00 con 0x7f322c101630 2024-02-17T11:57:22.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f32343a3740 1 Processor -- start 2024-02-17T11:57:22.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f32343a3740 1 -- start start 2024-02-17T11:57:22.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f32343a3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 0x559a48bf33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f32343a3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48bf8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f32343a3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 0x559a48bfaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f32343a3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a48baecd0 con 0x559a48bf3970 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f32343a3740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a48b61f20 con 0x559a48bf2f80 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f32343a3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a48b5f450 con 0x559a48bf8ae0 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f322affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48bf8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f322affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48bf8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43028/0 (socket says 172.21.15.94:43028) 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f322affd640 1 -- 172.21.15.94:0/1795527962 learned_addr learned my addr 172.21.15.94:0/1795527962 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f3223fff640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f32180a2150 0x7f32180a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f322a7fc640 1 --2- 172.21.15.94:0/1795527962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 0x559a48bf33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f322a7fc640 1 --2- 172.21.15.94:0/1795527962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 0x559a48bfaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f322affd640 1 -- 172.21.15.94:0/1795527962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 msgr2=0x559a48bfaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f322b7fe640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f32180a2150 0x7f32180a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f322affd640 1 --2- 172.21.15.94:0/1795527962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 0x559a48bfaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f322affd640 1 -- 172.21.15.94:0/1795527962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 msgr2=0x559a48bf33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f322affd640 1 --2- 172.21.15.94:0/1795527962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 0x559a48bf33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f322affd640 1 -- 172.21.15.94:0/1795527962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a48b0ff20 con 0x559a48bf3970 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f322a7fc640 1 --2- 172.21.15.94:0/1795527962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 0x559a48bfaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.493+0000 7f3223fff640 1 -- 172.21.15.94:0/1496187533 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f321c0ce6c0 con 0x7f322c101630 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.494+0000 7f322affd640 1 --2- 172.21.15.94:0/1795527962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48bf8470 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f322400cbc0 tx=0x7f322400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89a491b700592b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.494+0000 7f32217fa640 1 -- 172.21.15.94:0/1795527962 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224004220 con 0x559a48bf3970 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.494+0000 7f322b7fe640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f32180a2150 0x7f32180a4610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f321c01b000 tx=0x7f321c0667d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.494+0000 7f32217fa640 1 -- 172.21.15.94:0/1795527962 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3224004380 con 0x559a48bf3970 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.494+0000 7f32217fa640 1 -- 172.21.15.94:0/1795527962 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224004d70 con 0x559a48bf3970 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.494+0000 7f32343a3740 1 -- 172.21.15.94:0/1795527962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 msgr2=0x559a48bf8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.494+0000 7f32343a3740 1 --2- 172.21.15.94:0/1795527962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48bf8470 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f322400cbc0 tx=0x7f322400e720 comp rx=0 tx=0).stop 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 -- 172.21.15.94:0/1795527962 shutdown_connections 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 --2- 172.21.15.94:0/1795527962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 0x559a48bfaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 --2- 172.21.15.94:0/1795527962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48bf8470 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 --2- 172.21.15.94:0/1795527962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 0x559a48bf33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 -- 172.21.15.94:0/1795527962 >> 172.21.15.94:0/1795527962 conn(0x559a48bee4f0 msgr2=0x559a48bee8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 -- 172.21.15.94:0/1795527962 shutdown_connections 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 -- 172.21.15.94:0/1795527962 wait complete. 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 -- start start 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 Processor -- start 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 0x559a48bfd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48bfd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 0x559a48c045f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a48b5ac30 con 0x559a48bf3970 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a48affd30 con 0x559a48bf2f80 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a4882e310 con 0x559a48bf8ae0 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f32343a3740 10 client.0 _ll_register_callbacks cb 0x559a48b07d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f322affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48bfd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f322affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48bfd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43038/0 (socket says 172.21.15.94:43038) 2024-02-17T11:57:22.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f322affd640 1 -- 172.21.15.94:0/4177946372 learned_addr learned my addr 172.21.15.94:0/4177946372 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f322a7fc640 1 --2- 172.21.15.94:0/4177946372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 0x559a48c045f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f322affd640 1 -- 172.21.15.94:0/4177946372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 msgr2=0x559a48c045f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f322affd640 1 --2- 172.21.15.94:0/4177946372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 0x559a48c045f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f322affd640 1 -- 172.21.15.94:0/4177946372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 msgr2=0x559a48bfd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.495+0000 7f322affd640 1 --2- 172.21.15.94:0/4177946372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 0x559a48bfd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f322affd640 1 -- 172.21.15.94:0/4177946372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322400c7f0 con 0x559a48bf3970 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f322affd640 1 --2- 172.21.15.94:0/4177946372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48bfd5b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f322400ec20 tx=0x7f3224005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e7cf8f163987f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32127fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32127fc640 1 -- 172.21.15.94:0/4177946372 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224021070 con 0x559a48bf3970 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32127fc640 1 -- 172.21.15.94:0/4177946372 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f32240046c0 con 0x559a48bf3970 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32343a3740 1 -- 172.21.15.94:0/4177946372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559a4885a6c0 con 0x559a48bf3970 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32127fc640 1 -- 172.21.15.94:0/4177946372 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f322400cec0 con 0x559a48bf3970 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32343a3740 10 client.15057 fetch_fsmap learned FSMap version 20 2024-02-17T11:57:22.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32343a3740 1 -- 172.21.15.94:0/4177946372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559a48b592b0 con 0x559a48bf3970 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32127fc640 1 -- 172.21.15.94:0/4177946372 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559a4885a6c0 con 0x559a48bf3970 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32127fc640 1 -- 172.21.15.94:0/4177946372 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3224010430 con 0x559a48bf3970 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32343a3740 10 client.15057 fetch_fsmap finished waiting for FSMap version 20 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32343a3740 10 client.15057 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32343a3740 20 client.15057 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32343a3740 1 --2- 172.21.15.94:0/4177946372 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x559a48c09920 0x559a48c08bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32343a3740 4 client.15057 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.496+0000 7f32343a3740 1 -- 172.21.15.94:0/4177946372 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559a488ab270 con 0x559a48c09920 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.497+0000 7f322a7fc640 1 --2- 172.21.15.94:0/4177946372 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x559a48c09920 0x559a48c08bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.497+0000 7f32127fc640 10 client.15057 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.497+0000 7f322a7fc640 1 --2- 172.21.15.94:0/4177946372 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x559a48c09920 0x559a48c08bf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa5cd2aabea4e0b9 server_cookie=939147bd7fb71830 in_seq=0 out_seq=0 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32127fc640 1 -- 172.21.15.94:0/4177946372 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559a488ab270 con 0x559a48c09920 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32127fc640 10 client.15057 handle_command_reply: tid=0 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32343a3740 1 client.15057 shutdown 2024-02-17T11:57:22.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32343a3740 1 -- 172.21.15.94:0/4177946372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 msgr2=0x559a48bfd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32343a3740 1 --2- 172.21.15.94:0/4177946372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48bfd5b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f322400ec20 tx=0x7f3224005270 comp rx=0 tx=0).stop 2024-02-17T11:57:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32343a3740 1 -- 172.21.15.94:0/4177946372 shutdown_connections 2024-02-17T11:57:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32343a3740 1 --2- 172.21.15.94:0/4177946372 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x559a48c09920 0x559a48c08bf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32343a3740 1 --2- 172.21.15.94:0/4177946372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 0x559a48c045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32127fc640 0 client.15057 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T11:57:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32343a3740 1 --2- 172.21.15.94:0/4177946372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48bfd5b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32343a3740 1 --2- 172.21.15.94:0/4177946372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 0x559a48bfd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32343a3740 1 -- 172.21.15.94:0/4177946372 >> 172.21.15.94:0/4177946372 conn(0x559a48bee4f0 msgr2=0x559a48bed7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32343a3740 1 -- 172.21.15.94:0/4177946372 shutdown_connections 2024-02-17T11:57:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32343a3740 1 -- 172.21.15.94:0/4177946372 wait complete. 2024-02-17T11:57:22.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.498+0000 7f32343a3740 20 client.15057 trim_cache size 0 max 16384 2024-02-17T11:57:22.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.513+0000 7f32343a3740 1 Processor -- start 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f32343a3740 1 -- start start 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f32343a3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 0x559a48bfbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f32343a3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48c045b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f32343a3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 0x559a48bf4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f32343a3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a48ae0490 con 0x559a48bf3970 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f32343a3740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a48beefe0 con 0x559a48bf2f80 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f32343a3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a48bfb630 con 0x559a48bf8ae0 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f322affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48c045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f322affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48c045b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43052/0 (socket says 172.21.15.94:43052) 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f322affd640 1 -- 172.21.15.94:0/3597306786 learned_addr learned my addr 172.21.15.94:0/3597306786 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f322a7fc640 1 --2- 172.21.15.94:0/3597306786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 0x559a48bf4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f322a7fc640 1 --2- 172.21.15.94:0/3597306786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 0x559a48bfbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f322affd640 1 -- 172.21.15.94:0/3597306786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 msgr2=0x559a48bf4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f322affd640 1 --2- 172.21.15.94:0/3597306786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 0x559a48bf4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f322affd640 1 -- 172.21.15.94:0/3597306786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 msgr2=0x559a48bfbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f322affd640 1 --2- 172.21.15.94:0/3597306786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 0x559a48bfbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f322affd640 1 -- 172.21.15.94:0/3597306786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559a48af5a20 con 0x559a48bf3970 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f322affd640 1 --2- 172.21.15.94:0/3597306786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48c045b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f322400ec50 tx=0x7f322400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3492385e051da1c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.514+0000 7f3220ff9640 1 -- 172.21.15.94:0/3597306786 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224021040 con 0x559a48bf3970 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.515+0000 7f3220ff9640 1 -- 172.21.15.94:0/3597306786 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f32240057f0 con 0x559a48bf3970 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.515+0000 7f3220ff9640 1 -- 172.21.15.94:0/3597306786 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322401a690 con 0x559a48bf3970 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.515+0000 7f32343a3740 1 -- 172.21.15.94:0/3597306786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 msgr2=0x559a48c045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.515+0000 7f32343a3740 1 --2- 172.21.15.94:0/3597306786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48c045b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f322400ec50 tx=0x7f322400ec80 comp rx=0 tx=0).stop 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.515+0000 7f32343a3740 1 -- 172.21.15.94:0/3597306786 shutdown_connections 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.515+0000 7f32343a3740 1 --2- 172.21.15.94:0/3597306786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf8ae0 0x559a48bf4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.515+0000 7f32343a3740 1 --2- 172.21.15.94:0/3597306786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf3970 0x559a48c045b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.515+0000 7f32343a3740 1 --2- 172.21.15.94:0/3597306786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf2f80 0x559a48bfbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.515+0000 7f32343a3740 1 -- 172.21.15.94:0/3597306786 >> 172.21.15.94:0/3597306786 conn(0x559a48bee4f0 msgr2=0x559a48bee8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.515+0000 7f32343a3740 1 -- 172.21.15.94:0/3597306786 shutdown_connections 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.515+0000 7f32343a3740 1 -- 172.21.15.94:0/3597306786 wait complete. 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.515+0000 7f32343a3740 1 -- start start 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f32343a3740 1 Processor -- start 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f32343a3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf2f80 0x559a48c04520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f32343a3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf3970 0x559a48c01010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f32343a3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf8ae0 0x559a48c01550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f32343a3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a48c01c50 con 0x559a48bf2f80 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f32343a3740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a48c5f520 con 0x559a48bf8ae0 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f32343a3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559a48c5f6e0 con 0x559a48bf3970 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f32343a3740 10 client.0 _ll_register_callbacks cb 0x7f322400c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f322a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf2f80 0x559a48c04520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f322a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf2f80 0x559a48c04520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43054/0 (socket says 172.21.15.94:43054) 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f322a7fc640 1 -- 172.21.15.94:0/3727665707 learned_addr learned my addr 172.21.15.94:0/3727665707 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f322a7fc640 1 -- 172.21.15.94:0/3727665707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf3970 msgr2=0x559a48c01010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f322a7fc640 1 --2- 172.21.15.94:0/3727665707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf3970 0x559a48c01010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f322a7fc640 1 -- 172.21.15.94:0/3727665707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf8ae0 msgr2=0x559a48c01550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f322a7fc640 1 --2- 172.21.15.94:0/3727665707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf8ae0 0x559a48c01550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f322a7fc640 1 -- 172.21.15.94:0/3727665707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559a48b592b0 con 0x559a48bf2f80 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f322a7fc640 1 --2- 172.21.15.94:0/3727665707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf2f80 0x559a48c04520 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3214009900 tx=0x7f321402c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f519ef031ef1fed server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f32127fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f32127fc640 1 -- 172.21.15.94:0/3727665707 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214004740 con 0x559a48bf2f80 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f32127fc640 1 -- 172.21.15.94:0/3727665707 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f32140048a0 con 0x559a48bf2f80 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.516+0000 7f32343a3740 1 -- 172.21.15.94:0/3727665707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3224004380 con 0x559a48bf2f80 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.517+0000 7f32127fc640 1 -- 172.21.15.94:0/3727665707 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3214004a60 con 0x559a48bf2f80 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.517+0000 7f32343a3740 10 client.15063 fetch_fsmap learned FSMap version 20 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.517+0000 7f32343a3740 1 -- 172.21.15.94:0/3727665707 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x559a48b1b780 con 0x559a48bf2f80 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.517+0000 7f32127fc640 1 -- 172.21.15.94:0/3727665707 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3224004380 con 0x559a48bf2f80 2024-02-17T11:57:22.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.517+0000 7f32127fc640 1 -- 172.21.15.94:0/3727665707 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3214005460 con 0x559a48bf2f80 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.517+0000 7f32343a3740 10 client.15063 fetch_fsmap finished waiting for FSMap version 20 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.517+0000 7f32343a3740 10 client.15063 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.517+0000 7f32343a3740 20 client.15063 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.517+0000 7f32343a3740 1 --2- 172.21.15.94:0/3727665707 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x559a48c09920 0x559a48c60ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.517+0000 7f32343a3740 4 client.15063 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.517+0000 7f32343a3740 1 -- 172.21.15.94:0/3727665707 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559a48897610 con 0x559a48c09920 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.517+0000 7f322affd640 1 --2- 172.21.15.94:0/3727665707 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x559a48c09920 0x559a48c60ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32127fc640 10 client.15063 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f322affd640 1 --2- 172.21.15.94:0/3727665707 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x559a48c09920 0x559a48c60ef0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9eef599c6c43c8d server_cookie=cc17d089ae9c0615 in_seq=0 out_seq=0 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32127fc640 1 -- 172.21.15.94:0/3727665707 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x559a48897610 con 0x559a48c09920 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32127fc640 10 client.15063 handle_command_reply: tid=0 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32343a3740 1 client.15063 shutdown 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32343a3740 1 -- 172.21.15.94:0/3727665707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf2f80 msgr2=0x559a48c04520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32343a3740 1 --2- 172.21.15.94:0/3727665707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf2f80 0x559a48c04520 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3214009900 tx=0x7f321402c320 comp rx=0 tx=0).stop 2024-02-17T11:57:22.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32343a3740 1 -- 172.21.15.94:0/3727665707 shutdown_connections 2024-02-17T11:57:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32343a3740 1 --2- 172.21.15.94:0/3727665707 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x559a48c09920 0x559a48c60ef0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32343a3740 1 --2- 172.21.15.94:0/3727665707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559a48bf8ae0 0x559a48c01550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32127fc640 0 client.15063 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T11:57:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32343a3740 1 --2- 172.21.15.94:0/3727665707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559a48bf3970 0x559a48c01010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32343a3740 1 --2- 172.21.15.94:0/3727665707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559a48bf2f80 0x559a48c04520 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32343a3740 1 -- 172.21.15.94:0/3727665707 >> 172.21.15.94:0/3727665707 conn(0x559a48bee4f0 msgr2=0x559a48bf4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32343a3740 1 -- 172.21.15.94:0/3727665707 shutdown_connections 2024-02-17T11:57:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32343a3740 1 -- 172.21.15.94:0/3727665707 wait complete. 2024-02-17T11:57:22.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.518+0000 7f32343a3740 20 client.15063 trim_cache size 0 max 16384 2024-02-17T11:57:22.523 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:57:22.523 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "3bc5cfc8-cd8a-11ee-95ba-87774f69a715", 2024-02-17T11:57:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 1, 2024-02-17T11:57:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24487, 2024-02-17T11:57:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-02-17T11:57:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-02-17T11:57:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-02-17T11:57:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 82.385978571999999, 2024-02-17T11:57:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 20, 2024-02-17T11:57:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 87, 2024-02-17T11:57:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 85, 2024-02-17T11:57:22.524 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 98.561358428000005 2024-02-17T11:57:22.524 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:57:22.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.519+0000 7f32317d0640 1 -- 172.21.15.94:0/1496187533 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f32180a2150 msgr2=0x7f32180a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.519+0000 7f32317d0640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f32180a2150 0x7f32180a4610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f321c01b000 tx=0x7f321c0667d0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.519+0000 7f32317d0640 1 -- 172.21.15.94:0/1496187533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c101630 msgr2=0x7f322c19e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.519+0000 7f32317d0640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c101630 0x7f322c19e580 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f321c00cce0 tx=0x7f321c00a620 comp rx=0 tx=0).stop 2024-02-17T11:57:22.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.519+0000 7f32317d0640 1 -- 172.21.15.94:0/1496187533 shutdown_connections 2024-02-17T11:57:22.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.519+0000 7f32317d0640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f32180a2150 0x7f32180a4610 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.519+0000 7f32317d0640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f322c101630 0x7f322c19e580 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.519+0000 7f32317d0640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f322c100d20 0x7f322c19bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.519+0000 7f32317d0640 1 --2- 172.21.15.94:0/1496187533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f322c100360 0x7f322c0ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.519+0000 7f32317d0640 1 -- 172.21.15.94:0/1496187533 >> 172.21.15.94:0/1496187533 conn(0x7f322c0fadb0 msgr2=0x7f322c1045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.519+0000 7f32317d0640 1 -- 172.21.15.94:0/1496187533 shutdown_connections 2024-02-17T11:57:22.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.519+0000 7f32317d0640 1 -- 172.21.15.94:0/1496187533 wait complete. 2024-02-17T11:57:22.528 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-17T11:57:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.631+0000 7f1163335640 1 -- 172.21.15.94:0/202190523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f115c101c00 msgr2=0x7f115c103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.631+0000 7f1163335640 1 --2- 172.21.15.94:0/202190523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f115c101c00 0x7f115c103ff0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1158009900 tx=0x7f115802bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.631+0000 7f1163335640 1 -- 172.21.15.94:0/202190523 shutdown_connections 2024-02-17T11:57:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.631+0000 7f1163335640 1 --2- 172.21.15.94:0/202190523 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c106e60 0x7f115c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.631+0000 7f1163335640 1 --2- 172.21.15.94:0/202190523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f115c104530 0x7f115c106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.631+0000 7f1163335640 1 --2- 172.21.15.94:0/202190523 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f115c101c00 0x7f115c103ff0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.631+0000 7f1163335640 1 -- 172.21.15.94:0/202190523 >> 172.21.15.94:0/202190523 conn(0x7f115c0fb610 msgr2=0x7f115c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.631+0000 7f1163335640 1 -- 172.21.15.94:0/202190523 shutdown_connections 2024-02-17T11:57:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.631+0000 7f1163335640 1 -- 172.21.15.94:0/202190523 wait complete. 2024-02-17T11:57:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.631+0000 7f1163335640 1 Processor -- start 2024-02-17T11:57:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f1163335640 1 -- start start 2024-02-17T11:57:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f1163335640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c101c00 0x7f115c197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f1163335640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f115c104530 0x7f115c1978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f1163335640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f115c106e60 0x7f115c19e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f1163335640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c19ee80 con 0x7f115c106e60 2024-02-17T11:57:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f1163335640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c19eff0 con 0x7f115c101c00 2024-02-17T11:57:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f1163335640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115c19f1b0 con 0x7f115c104530 2024-02-17T11:57:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11618d2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f115c106e60 0x7f115c19e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11618d2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f115c106e60 0x7f115c19e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43060/0 (socket says 172.21.15.94:43060) 2024-02-17T11:57:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11618d2640 1 -- 172.21.15.94:0/1546497586 learned_addr learned my addr 172.21.15.94:0/1546497586 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11608d0640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f115c104530 0x7f115c1978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11610d1640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c101c00 0x7f115c197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11618d2640 1 -- 172.21.15.94:0/1546497586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f115c104530 msgr2=0x7f115c1978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11618d2640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f115c104530 0x7f115c1978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11618d2640 1 -- 172.21.15.94:0/1546497586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c101c00 msgr2=0x7f115c197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11618d2640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c101c00 0x7f115c197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11618d2640 1 -- 172.21.15.94:0/1546497586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1158009560 con 0x7f115c106e60 2024-02-17T11:57:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11608d0640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f115c104530 0x7f115c1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11610d1640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c101c00 0x7f115c197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11618d2640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f115c106e60 0x7f115c19e940 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f115400b970 tx=0x7f115400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11527fc640 1 -- 172.21.15.94:0/1546497586 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115400cb40 con 0x7f115c106e60 2024-02-17T11:57:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11527fc640 1 -- 172.21.15.94:0/1546497586 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f115400cca0 con 0x7f115c106e60 2024-02-17T11:57:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f1163335640 1 -- 172.21.15.94:0/1546497586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115c19f490 con 0x7f115c106e60 2024-02-17T11:57:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f11527fc640 1 -- 172.21.15.94:0/1546497586 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11540077d0 con 0x7f115c106e60 2024-02-17T11:57:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.632+0000 7f1163335640 1 -- 172.21.15.94:0/1546497586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115c19f9d0 con 0x7f115c106e60 2024-02-17T11:57:22.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.634+0000 7f11527fc640 1 -- 172.21.15.94:0/1546497586 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f1154014440 con 0x7f115c106e60 2024-02-17T11:57:22.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f1165fe0740 1 Processor -- start 2024-02-17T11:57:22.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f1165fe0740 1 -- start start 2024-02-17T11:57:22.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f1165fe0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x55871253f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f1165fe0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x558712544470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f1165fe0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558712544ae0 0x558712546f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f1165fe0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587124facd0 con 0x55871253ef80 2024-02-17T11:57:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f1165fe0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587124adf20 con 0x558712544ae0 2024-02-17T11:57:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f1165fe0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587124ab450 con 0x55871253f970 2024-02-17T11:57:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f11527fc640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f11480a2150 0x7f11480a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f11608d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x55871253f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f11608d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x55871253f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43066/0 (socket says 172.21.15.94:43066) 2024-02-17T11:57:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f11608d0640 1 -- 172.21.15.94:0/2102800783 learned_addr learned my addr 172.21.15.94:0/2102800783 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f11618d2640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f11480a2150 0x7f11480a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f11608d0640 1 --2- 172.21.15.94:0/2102800783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558712544ae0 0x558712546f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.635+0000 7f11527fc640 1 -- 172.21.15.94:0/1546497586 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f11540cd6a0 con 0x7f115c106e60 2024-02-17T11:57:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.636+0000 7f11610d1640 1 --2- 172.21.15.94:0/2102800783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x558712544470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.636+0000 7f11608d0640 1 -- 172.21.15.94:0/2102800783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 msgr2=0x558712544470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.636+0000 7f11608d0640 1 --2- 172.21.15.94:0/2102800783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x558712544470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.636+0000 7f11608d0640 1 -- 172.21.15.94:0/2102800783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558712544ae0 msgr2=0x558712546f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.636+0000 7f11608d0640 1 --2- 172.21.15.94:0/2102800783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558712544ae0 0x558712546f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.636+0000 7f11608d0640 1 -- 172.21.15.94:0/2102800783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55871245bf20 con 0x55871253ef80 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.636+0000 7f11610d1640 1 --2- 172.21.15.94:0/2102800783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x558712544470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.636+0000 7f11608d0640 1 --2- 172.21.15.94:0/2102800783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x55871253f3a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f114c007f00 tx=0x7f114c0073e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c96242af801e59 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.636+0000 7f11618d2640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f11480a2150 0x7f11480a4610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f11540967e0 tx=0x7f115406d440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.636+0000 7f11477fe640 1 -- 172.21.15.94:0/2102800783 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c017400 con 0x55871253ef80 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.636+0000 7f11477fe640 1 -- 172.21.15.94:0/2102800783 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f114c017a40 con 0x55871253ef80 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.636+0000 7f11477fe640 1 -- 172.21.15.94:0/2102800783 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c014980 con 0x55871253ef80 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 -- 172.21.15.94:0/2102800783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 msgr2=0x55871253f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 --2- 172.21.15.94:0/2102800783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x55871253f3a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f114c007f00 tx=0x7f114c0073e0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 -- 172.21.15.94:0/2102800783 shutdown_connections 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 --2- 172.21.15.94:0/2102800783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558712544ae0 0x558712546f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 --2- 172.21.15.94:0/2102800783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x558712544470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 --2- 172.21.15.94:0/2102800783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x55871253f3a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 -- 172.21.15.94:0/2102800783 >> 172.21.15.94:0/2102800783 conn(0x55871253a4f0 msgr2=0x55871253a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 -- 172.21.15.94:0/2102800783 shutdown_connections 2024-02-17T11:57:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 -- 172.21.15.94:0/2102800783 wait complete. 2024-02-17T11:57:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 -- start start 2024-02-17T11:57:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 Processor -- start 2024-02-17T11:57:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x5587125491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871253f970 0x558712549700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558712544ae0 0x558712550740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587124a6c30 con 0x55871253ef80 2024-02-17T11:57:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871244bd30 con 0x55871253f970 2024-02-17T11:57:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.637+0000 7f1165fe0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871217a310 con 0x558712544ae0 2024-02-17T11:57:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f11608d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x5587125491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f1165fe0740 10 client.0 _ll_register_callbacks cb 0x558712453d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f11608d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x5587125491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43072/0 (socket says 172.21.15.94:43072) 2024-02-17T11:57:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f11608d0640 1 -- 172.21.15.94:0/1753642273 learned_addr learned my addr 172.21.15.94:0/1753642273 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f11610d1640 1 --2- 172.21.15.94:0/1753642273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871253f970 0x558712549700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f11608d0640 1 -- 172.21.15.94:0/1753642273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558712544ae0 msgr2=0x558712550740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f11608d0640 1 --2- 172.21.15.94:0/1753642273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558712544ae0 0x558712550740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f11608d0640 1 -- 172.21.15.94:0/1753642273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871253f970 msgr2=0x558712549700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f11608d0640 1 --2- 172.21.15.94:0/1753642273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871253f970 0x558712549700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f11608d0640 1 -- 172.21.15.94:0/1753642273 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c007a80 con 0x55871253ef80 2024-02-17T11:57:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f11610d1640 1 --2- 172.21.15.94:0/1753642273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871253f970 0x558712549700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f11608d0640 1 --2- 172.21.15.94:0/1753642273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x5587125491c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f114c01a040 tx=0x7f114c017630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6893a36dc80aaa server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f1144ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f1144ff9640 1 -- 172.21.15.94:0/1753642273 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c0059c0 con 0x55871253ef80 2024-02-17T11:57:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f1144ff9640 1 -- 172.21.15.94:0/1753642273 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f114c005b20 con 0x55871253ef80 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f1165fe0740 1 -- 172.21.15.94:0/1753642273 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587121a66c0 con 0x55871253ef80 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f1144ff9640 1 -- 172.21.15.94:0/1753642273 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f114c005ce0 con 0x55871253ef80 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f1165fe0740 10 client.15075 fetch_fsmap learned FSMap version 20 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f1165fe0740 1 -- 172.21.15.94:0/1753642273 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x5587124a52b0 con 0x55871253ef80 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.638+0000 7f1144ff9640 1 -- 172.21.15.94:0/1753642273 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5587121a66c0 con 0x55871253ef80 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.639+0000 7f1144ff9640 1 -- 172.21.15.94:0/1753642273 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f114c006250 con 0x55871253ef80 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.639+0000 7f1165fe0740 10 client.15075 fetch_fsmap finished waiting for FSMap version 20 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.639+0000 7f1165fe0740 10 client.15075 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.639+0000 7f1165fe0740 20 client.15075 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.639+0000 7f1165fe0740 1 --2- 172.21.15.94:0/1753642273 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558712555330 0x558712559df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.639+0000 7f1165fe0740 4 client.15075 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.639+0000 7f1165fe0740 1 -- 172.21.15.94:0/1753642273 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587121f7270 con 0x558712555330 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.639+0000 7f11610d1640 1 --2- 172.21.15.94:0/1753642273 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558712555330 0x558712559df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.639+0000 7f1144ff9640 10 client.15075 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.639+0000 7f11610d1640 1 --2- 172.21.15.94:0/1753642273 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558712555330 0x558712559df0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d74324ad95440ba server_cookie=9dadf9139ac1a911 in_seq=0 out_seq=0 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1144ff9640 1 -- 172.21.15.94:0/1753642273 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587121f7270 con 0x558712555330 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1144ff9640 10 client.15075 handle_command_reply: tid=0 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1165fe0740 1 client.15075 shutdown 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1165fe0740 1 -- 172.21.15.94:0/1753642273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 msgr2=0x5587125491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1165fe0740 1 --2- 172.21.15.94:0/1753642273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x5587125491c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f114c01a040 tx=0x7f114c017630 comp rx=0 tx=0).stop 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1165fe0740 1 -- 172.21.15.94:0/1753642273 shutdown_connections 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1165fe0740 1 --2- 172.21.15.94:0/1753642273 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558712555330 0x558712559df0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1165fe0740 1 --2- 172.21.15.94:0/1753642273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558712544ae0 0x558712550740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1144ff9640 0 client.15075 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1165fe0740 1 --2- 172.21.15.94:0/1753642273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871253f970 0x558712549700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1165fe0740 1 --2- 172.21.15.94:0/1753642273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x5587125491c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1165fe0740 1 -- 172.21.15.94:0/1753642273 >> 172.21.15.94:0/1753642273 conn(0x55871253a4f0 msgr2=0x5587125397d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1165fe0740 1 -- 172.21.15.94:0/1753642273 shutdown_connections 2024-02-17T11:57:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1165fe0740 1 -- 172.21.15.94:0/1753642273 wait complete. 2024-02-17T11:57:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.640+0000 7f1165fe0740 20 client.15075 trim_cache size 0 max 16384 2024-02-17T11:57:22.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f1165fe0740 1 Processor -- start 2024-02-17T11:57:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f1165fe0740 1 -- start start 2024-02-17T11:57:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f1165fe0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871253ef80 0x558712548dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f1165fe0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x5587125506d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f1165fe0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558712544ae0 0x558712549310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f1165fe0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871242c490 con 0x558712544ae0 2024-02-17T11:57:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f1165fe0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558712547630 con 0x55871253ef80 2024-02-17T11:57:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f1165fe0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587121f7270 con 0x55871253f970 2024-02-17T11:57:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f11608d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558712544ae0 0x558712549310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f11608d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558712544ae0 0x558712549310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43088/0 (socket says 172.21.15.94:43088) 2024-02-17T11:57:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f11608d0640 1 -- 172.21.15.94:0/368000290 learned_addr learned my addr 172.21.15.94:0/368000290 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f11610d1640 1 --2- 172.21.15.94:0/368000290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x5587125506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f11608d0640 1 -- 172.21.15.94:0/368000290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 msgr2=0x5587125506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f11608d0640 1 --2- 172.21.15.94:0/368000290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x5587125506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f11608d0640 1 -- 172.21.15.94:0/368000290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871253ef80 msgr2=0x558712548dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f11608d0640 1 --2- 172.21.15.94:0/368000290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871253ef80 0x558712548dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f11608d0640 1 -- 172.21.15.94:0/368000290 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558712555020 con 0x558712544ae0 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f11608d0640 1 --2- 172.21.15.94:0/368000290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558712544ae0 0x558712549310 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f114c006b50 tx=0x7f114c00ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d71ca051d475113 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f1146ffd640 1 -- 172.21.15.94:0/368000290 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c007ca0 con 0x558712544ae0 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f1146ffd640 1 -- 172.21.15.94:0/368000290 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f114c02a980 con 0x558712544ae0 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.656+0000 7f1146ffd640 1 -- 172.21.15.94:0/368000290 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c021670 con 0x558712544ae0 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.657+0000 7f1165fe0740 1 -- 172.21.15.94:0/368000290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558712544ae0 msgr2=0x558712549310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.657+0000 7f1165fe0740 1 --2- 172.21.15.94:0/368000290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558712544ae0 0x558712549310 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f114c006b50 tx=0x7f114c00ffd0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.657+0000 7f1165fe0740 1 -- 172.21.15.94:0/368000290 shutdown_connections 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.657+0000 7f1165fe0740 1 --2- 172.21.15.94:0/368000290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558712544ae0 0x558712549310 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.657+0000 7f1165fe0740 1 --2- 172.21.15.94:0/368000290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x5587125506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.657+0000 7f1165fe0740 1 --2- 172.21.15.94:0/368000290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871253ef80 0x558712548dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.657+0000 7f1165fe0740 1 -- 172.21.15.94:0/368000290 >> 172.21.15.94:0/368000290 conn(0x55871253a4f0 msgr2=0x55871253a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.657+0000 7f1165fe0740 1 -- 172.21.15.94:0/368000290 shutdown_connections 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.657+0000 7f1165fe0740 1 -- 172.21.15.94:0/368000290 wait complete. 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.657+0000 7f1165fe0740 1 -- start start 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.657+0000 7f1165fe0740 1 Processor -- start 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1165fe0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x5587125501d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1165fe0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x558712550710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1165fe0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558712544ae0 0x55871254cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1165fe0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871254d620 con 0x55871253ef80 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1165fe0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871254d790 con 0x558712544ae0 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1165fe0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871254d950 con 0x55871253f970 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1165fe0740 10 client.0 _ll_register_callbacks cb 0x7f114c007240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f11608d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x5587125501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f11608d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x5587125501d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43098/0 (socket says 172.21.15.94:43098) 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f11608d0640 1 -- 172.21.15.94:0/1917418929 learned_addr learned my addr 172.21.15.94:0/1917418929 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f11610d1640 1 --2- 172.21.15.94:0/1917418929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x558712550710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f11608d0640 1 -- 172.21.15.94:0/1917418929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 msgr2=0x558712550710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f11608d0640 1 --2- 172.21.15.94:0/1917418929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x558712550710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f11608d0640 1 -- 172.21.15.94:0/1917418929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558712544ae0 msgr2=0x55871254cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f11608d0640 1 --2- 172.21.15.94:0/1917418929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558712544ae0 0x55871254cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f11608d0640 1 -- 172.21.15.94:0/1917418929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587124a52b0 con 0x55871253ef80 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f11610d1640 1 --2- 172.21.15.94:0/1917418929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x558712550710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f11608d0640 1 --2- 172.21.15.94:0/1917418929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x5587125501d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f114c017670 tx=0x7f114c007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01c8ae903582430 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1144ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1144ff9640 1 -- 172.21.15.94:0/1917418929 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c02aaf0 con 0x55871253ef80 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1144ff9640 1 -- 172.21.15.94:0/1917418929 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f114c000e40 con 0x55871253ef80 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1165fe0740 1 -- 172.21.15.94:0/1917418929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f114c017a40 con 0x55871253ef80 2024-02-17T11:57:22.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1144ff9640 1 -- 172.21.15.94:0/1917418929 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f114c02ac50 con 0x55871253ef80 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1165fe0740 10 client.15081 fetch_fsmap learned FSMap version 20 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1165fe0740 1 -- 172.21.15.94:0/1917418929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x558712441a20 con 0x55871253ef80 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.658+0000 7f1144ff9640 1 -- 172.21.15.94:0/1917418929 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f114c017a40 con 0x55871253ef80 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.659+0000 7f1144ff9640 1 -- 172.21.15.94:0/1917418929 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f114c021410 con 0x55871253ef80 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.659+0000 7f1165fe0740 10 client.15081 fetch_fsmap finished waiting for FSMap version 20 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.659+0000 7f1165fe0740 10 client.15081 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.659+0000 7f1165fe0740 20 client.15081 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.659+0000 7f1165fe0740 1 --2- 172.21.15.94:0/1917418929 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5587125b0690 0x5587125acb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.659+0000 7f1165fe0740 4 client.15081 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.659+0000 7f1165fe0740 1 -- 172.21.15.94:0/1917418929 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5587121e3610 con 0x5587125b0690 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.659+0000 7f11610d1640 1 --2- 172.21.15.94:0/1917418929 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5587125b0690 0x5587125acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.659+0000 7f1144ff9640 10 client.15081 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.659+0000 7f11610d1640 1 --2- 172.21.15.94:0/1917418929 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5587125b0690 0x5587125acb30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88dba9b1f7577b61 server_cookie=a2c52d607339f1e6 in_seq=0 out_seq=0 2024-02-17T11:57:22.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1144ff9640 1 -- 172.21.15.94:0/1917418929 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5587121e3610 con 0x5587125b0690 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1144ff9640 10 client.15081 handle_command_reply: tid=0 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1165fe0740 1 client.15081 shutdown 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1165fe0740 1 -- 172.21.15.94:0/1917418929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 msgr2=0x5587125501d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1165fe0740 1 --2- 172.21.15.94:0/1917418929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x5587125501d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f114c017670 tx=0x7f114c007810 comp rx=0 tx=0).stop 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1165fe0740 1 -- 172.21.15.94:0/1917418929 shutdown_connections 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1165fe0740 1 --2- 172.21.15.94:0/1917418929 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5587125b0690 0x5587125acb30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1165fe0740 1 --2- 172.21.15.94:0/1917418929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558712544ae0 0x55871254cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1165fe0740 1 --2- 172.21.15.94:0/1917418929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871253f970 0x558712550710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1165fe0740 1 --2- 172.21.15.94:0/1917418929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871253ef80 0x5587125501d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1165fe0740 1 -- 172.21.15.94:0/1917418929 >> 172.21.15.94:0/1917418929 conn(0x55871253a4f0 msgr2=0x558712548dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1144ff9640 0 client.15081 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1165fe0740 1 -- 172.21.15.94:0/1917418929 shutdown_connections 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1165fe0740 1 -- 172.21.15.94:0/1917418929 wait complete. 2024-02-17T11:57:22.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1165fe0740 20 client.15081 trim_cache size 0 max 16384 2024-02-17T11:57:22.666 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:57:22.666 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "3bc5cfc8-cd8a-11ee-95ba-87774f69a715", 2024-02-17T11:57:22.666 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-02-17T11:57:22.666 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24488, 2024-02-17T11:57:22.666 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-02-17T11:57:22.666 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-02-17T11:57:22.666 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-02-17T11:57:22.666 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 84.004488652000006, 2024-02-17T11:57:22.666 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 20, 2024-02-17T11:57:22.666 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 87, 2024-02-17T11:57:22.666 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 85, 2024-02-17T11:57:22.667 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 90.315336372999994 2024-02-17T11:57:22.667 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:57:22.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1163335640 1 -- 172.21.15.94:0/1546497586 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f11480a2150 msgr2=0x7f11480a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1163335640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f11480a2150 0x7f11480a4610 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f11540967e0 tx=0x7f115406d440 comp rx=0 tx=0).stop 2024-02-17T11:57:22.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1163335640 1 -- 172.21.15.94:0/1546497586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f115c106e60 msgr2=0x7f115c19e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:22.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1163335640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f115c106e60 0x7f115c19e940 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f115400b970 tx=0x7f115400be30 comp rx=0 tx=0).stop 2024-02-17T11:57:22.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1163335640 1 -- 172.21.15.94:0/1546497586 shutdown_connections 2024-02-17T11:57:22.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1163335640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f11480a2150 0x7f11480a4610 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1163335640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f115c106e60 0x7f115c19e940 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1163335640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f115c104530 0x7f115c1978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1163335640 1 --2- 172.21.15.94:0/1546497586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f115c101c00 0x7f115c197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:22.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1163335640 1 -- 172.21.15.94:0/1546497586 >> 172.21.15.94:0/1546497586 conn(0x7f115c0fb610 msgr2=0x7f115c1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:22.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.660+0000 7f1163335640 1 -- 172.21.15.94:0/1546497586 shutdown_connections 2024-02-17T11:57:22.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:22.661+0000 7f1163335640 1 -- 172.21.15.94:0/1546497586 wait complete. 2024-02-17T11:57:22.670 INFO:tasks.cephadm:Setup complete, yielding 2024-02-17T11:57:22.670 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-17T11:57:22.681 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-02-17T11:57:22.682 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- bash -c 'ceph orch status' 2024-02-17T11:57:22.925 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:57:22.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:22 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3604970860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:22.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:22 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3383773257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:23.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:22 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3604970860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:23.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:22 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3383773257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:23.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:22 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3604970860' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:23.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:22 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3383773257' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:24.012 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:23 smithi094 ceph-mon[32499]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-17T11:57:24.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:23 smithi189 ceph-mon[36588]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-17T11:57:24.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:23 smithi125 ceph-mon[36819]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-02-17T11:57:24.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.577+0000 7fbeb5b65700 1 -- 172.21.15.94:0/3156975078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb0100ba0 msgr2=0x7fbeb0100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:24.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.577+0000 7fbeb5b65700 1 --2- 172.21.15.94:0/3156975078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb0100ba0 0x7fbeb0100fc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbea00099c0 tx=0x7fbea0009cd0 comp rx=0 tx=0).stop 2024-02-17T11:57:24.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.577+0000 7fbeb5b65700 1 -- 172.21.15.94:0/3156975078 shutdown_connections 2024-02-17T11:57:24.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.577+0000 7fbeb5b65700 1 --2- 172.21.15.94:0/3156975078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb0102740 0x7fbeb0108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:24.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.577+0000 7fbeb5b65700 1 --2- 172.21.15.94:0/3156975078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb0100ba0 0x7fbeb0100fc0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:24.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.577+0000 7fbeb5b65700 1 --2- 172.21.15.94:0/3156975078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbeb0101da0 0x7fbeb0102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:24.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.577+0000 7fbeb5b65700 1 -- 172.21.15.94:0/3156975078 >> 172.21.15.94:0/3156975078 conn(0x7fbeb00fc140 msgr2=0x7fbeb00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:24.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.577+0000 7fbeb5b65700 1 -- 172.21.15.94:0/3156975078 shutdown_connections 2024-02-17T11:57:24.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.577+0000 7fbeb5b65700 1 -- 172.21.15.94:0/3156975078 wait complete. 2024-02-17T11:57:24.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.577+0000 7fbeb5b65700 1 Processor -- start 2024-02-17T11:57:24.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.577+0000 7fbeb5b65700 1 -- start start 2024-02-17T11:57:24.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeb5b65700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbeb0100ba0 0x7fbeb006a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:24.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeb5b65700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb0101da0 0x7fbeb019ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:24.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeb5b65700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb0102740 0x7fbeb019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:24.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeb5b65700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb019f830 con 0x7fbeb0102740 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeb5b65700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb019f9a0 con 0x7fbeb0101da0 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeb5b65700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb019fb60 con 0x7fbeb0100ba0 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeaffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb0102740 0x7fbeb019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeaffff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb0102740 0x7fbeb019f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45452/0 (socket says 172.21.15.94:45452) 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeaffff700 1 -- 172.21.15.94:0/849127166 learned_addr learned my addr 172.21.15.94:0/849127166 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeaf7fe700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbeb0100ba0 0x7fbeb006a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbea6ffd700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb0101da0 0x7fbeb019ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeaffff700 1 -- 172.21.15.94:0/849127166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbeb0100ba0 msgr2=0x7fbeb006a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeaffff700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbeb0100ba0 0x7fbeb006a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeaffff700 1 -- 172.21.15.94:0/849127166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb0101da0 msgr2=0x7fbeb019ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeaffff700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb0101da0 0x7fbeb019ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeaffff700 1 -- 172.21.15.94:0/849127166 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea00095f0 con 0x7fbeb0102740 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeaf7fe700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbeb0100ba0 0x7fbeb006a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:24.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbeaffff700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb0102740 0x7fbeb019f090 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbea8007ee0 tx=0x7fbea800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:24.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.578+0000 7fbead7fa700 1 -- 172.21.15.94:0/849127166 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8017070 con 0x7fbeb0102740 2024-02-17T11:57:24.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.579+0000 7fbead7fa700 1 -- 172.21.15.94:0/849127166 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbea800f040 con 0x7fbeb0102740 2024-02-17T11:57:24.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.579+0000 7fbeb5b65700 1 -- 172.21.15.94:0/849127166 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb019fe40 con 0x7fbeb0102740 2024-02-17T11:57:24.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.579+0000 7fbead7fa700 1 -- 172.21.15.94:0/849127166 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8005760 con 0x7fbeb0102740 2024-02-17T11:57:24.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.579+0000 7fbeb5b65700 1 -- 172.21.15.94:0/849127166 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb01a0260 con 0x7fbeb0102740 2024-02-17T11:57:24.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.580+0000 7fbeb5b65700 1 -- 172.21.15.94:0/849127166 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb0108030 con 0x7fbeb0102740 2024-02-17T11:57:24.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.581+0000 7fbead7fa700 1 -- 172.21.15.94:0/849127166 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fbea80058c0 con 0x7fbeb0102740 2024-02-17T11:57:24.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.581+0000 7fbead7fa700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbe9c0a21e0 0x7fbe9c0a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:24.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.581+0000 7fbeaf7fe700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbe9c0a21e0 0x7fbe9c0a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:24.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.581+0000 7fbead7fa700 1 -- 172.21.15.94:0/849127166 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fbea80cd660 con 0x7fbeb0102740 2024-02-17T11:57:24.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.582+0000 7fbeaf7fe700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbe9c0a21e0 0x7fbe9c0a46a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbea0000c00 tx=0x7fbea0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:24.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.585+0000 7fbead7fa700 1 -- 172.21.15.94:0/849127166 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbea80964d0 con 0x7fbeb0102740 2024-02-17T11:57:24.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.800+0000 7fbeb5b65700 1 -- 172.21.15.94:0/849127166 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbeb002d8d0 con 0x7fbe9c0a21e0 2024-02-17T11:57:24.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.801+0000 7fbead7fa700 1 -- 172.21.15.94:0/849127166 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fbeb002d8d0 con 0x7fbe9c0a21e0 2024-02-17T11:57:24.799 INFO:teuthology.orchestra.run.smithi094.stdout:Backend: cephadm 2024-02-17T11:57:24.800 INFO:teuthology.orchestra.run.smithi094.stdout:Available: Yes 2024-02-17T11:57:24.800 INFO:teuthology.orchestra.run.smithi094.stdout:Paused: No 2024-02-17T11:57:24.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.803+0000 7fbeb5b65700 1 -- 172.21.15.94:0/849127166 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbe9c0a21e0 msgr2=0x7fbe9c0a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.803+0000 7fbeb5b65700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbe9c0a21e0 0x7fbe9c0a46a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbea0000c00 tx=0x7fbea0005fb0 comp rx=0 tx=0).stop 2024-02-17T11:57:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.803+0000 7fbeb5b65700 1 -- 172.21.15.94:0/849127166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb0102740 msgr2=0x7fbeb019f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.803+0000 7fbeb5b65700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb0102740 0x7fbeb019f090 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbea8007ee0 tx=0x7fbea800d3b0 comp rx=0 tx=0).stop 2024-02-17T11:57:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.803+0000 7fbeb5b65700 1 -- 172.21.15.94:0/849127166 shutdown_connections 2024-02-17T11:57:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.803+0000 7fbeb5b65700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fbe9c0a21e0 0x7fbe9c0a46a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.803+0000 7fbeb5b65700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb0102740 0x7fbeb019f090 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.803+0000 7fbeb5b65700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbeb0100ba0 0x7fbeb006a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.803+0000 7fbeb5b65700 1 --2- 172.21.15.94:0/849127166 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb0101da0 0x7fbeb019ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.803+0000 7fbeb5b65700 1 -- 172.21.15.94:0/849127166 >> 172.21.15.94:0/849127166 conn(0x7fbeb00fc140 msgr2=0x7fbeb0105170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.803+0000 7fbeb5b65700 1 -- 172.21.15.94:0/849127166 shutdown_connections 2024-02-17T11:57:24.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:24.803+0000 7fbeb5b65700 1 -- 172.21.15.94:0/849127166 wait complete. 2024-02-17T11:57:25.387 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-17T11:57:25.627 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:57:26.060 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:25 smithi094 ceph-mon[32499]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-17T11:57:26.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:25 smithi189 ceph-mon[36588]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-17T11:57:26.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:25 smithi125 ceph-mon[36819]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-02-17T11:57:27.063 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:26 smithi094 ceph-mon[32499]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:27.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:26 smithi189 ceph-mon[36588]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:27.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:26 smithi125 ceph-mon[36819]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:27.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.289+0000 7fe33d1c4700 1 -- 172.21.15.94:0/3517378262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3381050b0 msgr2=0x7fe3381074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:27.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.289+0000 7fe33d1c4700 1 --2- 172.21.15.94:0/3517378262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3381050b0 0x7fe3381074a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009990 tx=0x7fe32c009ca0 comp rx=0 tx=0).stop 2024-02-17T11:57:27.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.289+0000 7fe33d1c4700 1 -- 172.21.15.94:0/3517378262 shutdown_connections 2024-02-17T11:57:27.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.289+0000 7fe33d1c4700 1 --2- 172.21.15.94:0/3517378262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3381079e0 0x7fe338109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:27.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.289+0000 7fe33d1c4700 1 --2- 172.21.15.94:0/3517378262 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe338102780 0x7fe338104b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:27.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.289+0000 7fe33d1c4700 1 --2- 172.21.15.94:0/3517378262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe3381050b0 0x7fe3381074a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:27.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.289+0000 7fe33d1c4700 1 -- 172.21.15.94:0/3517378262 >> 172.21.15.94:0/3517378262 conn(0x7fe3380fc130 msgr2=0x7fe3380fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:27.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.290+0000 7fe33d1c4700 1 -- 172.21.15.94:0/3517378262 shutdown_connections 2024-02-17T11:57:27.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.290+0000 7fe33d1c4700 1 -- 172.21.15.94:0/3517378262 wait complete. 2024-02-17T11:57:27.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.290+0000 7fe33d1c4700 1 Processor -- start 2024-02-17T11:57:27.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.290+0000 7fe33d1c4700 1 -- start start 2024-02-17T11:57:27.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe33d1c4700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe338102780 0x7fe338197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:27.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe33d1c4700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3381050b0 0x7fe338198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:27.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe33d1c4700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3381079e0 0x7fe33819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe33d1c4700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33819f8c0 con 0x7fe338102780 2024-02-17T11:57:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe33d1c4700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33819fa00 con 0x7fe3381050b0 2024-02-17T11:57:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe33d1c4700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33819fb90 con 0x7fe3381079e0 2024-02-17T11:57:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe336d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe338102780 0x7fe338197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe336d9d700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe338102780 0x7fe338197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45458/0 (socket says 172.21.15.94:45458) 2024-02-17T11:57:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe336d9d700 1 -- 172.21.15.94:0/1923014532 learned_addr learned my addr 172.21.15.94:0/1923014532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe33659c700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3381050b0 0x7fe338198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe33759e700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3381079e0 0x7fe33819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe336d9d700 1 -- 172.21.15.94:0/1923014532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3381079e0 msgr2=0x7fe33819f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe336d9d700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3381079e0 0x7fe33819f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe336d9d700 1 -- 172.21.15.94:0/1923014532 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3381050b0 msgr2=0x7fe338198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe336d9d700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3381050b0 0x7fe338198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:27.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe336d9d700 1 -- 172.21.15.94:0/1923014532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c0095f0 con 0x7fe338102780 2024-02-17T11:57:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe336d9d700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe338102780 0x7fe338197d30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe33000ba80 tx=0x7fe33000bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe31ffff700 1 -- 172.21.15.94:0/1923014532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33000c930 con 0x7fe338102780 2024-02-17T11:57:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe33d1c4700 1 -- 172.21.15.94:0/1923014532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33819fe70 con 0x7fe338102780 2024-02-17T11:57:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe31ffff700 1 -- 172.21.15.94:0/1923014532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe330014440 con 0x7fe338102780 2024-02-17T11:57:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.291+0000 7fe31ffff700 1 -- 172.21.15.94:0/1923014532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3300075d0 con 0x7fe338102780 2024-02-17T11:57:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.292+0000 7fe33d1c4700 1 -- 172.21.15.94:0/1923014532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3381a0300 con 0x7fe338102780 2024-02-17T11:57:27.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.293+0000 7fe33d1c4700 1 -- 172.21.15.94:0/1923014532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3380fdcb0 con 0x7fe338102780 2024-02-17T11:57:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.293+0000 7fe31ffff700 1 -- 172.21.15.94:0/1923014532 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 17) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe3300145b0 con 0x7fe338102780 2024-02-17T11:57:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.294+0000 7fe31ffff700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe3200a21e0 0x7fe3200a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.294+0000 7fe33659c700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe3200a21e0 0x7fe3200a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.294+0000 7fe31ffff700 1 -- 172.21.15.94:0/1923014532 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe330006e40 con 0x7fe338102780 2024-02-17T11:57:27.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.294+0000 7fe33659c700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe3200a21e0 0x7fe3200a46a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe32c003790 tx=0x7fe32c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:27.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.298+0000 7fe31ffff700 1 -- 172.21.15.94:0/1923014532 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe330096440 con 0x7fe338102780 2024-02-17T11:57:27.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.501+0000 7fe33d1c4700 1 -- 172.21.15.94:0/1923014532 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe33802d870 con 0x7fe3200a21e0 2024-02-17T11:57:27.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.515+0000 7fe31ffff700 1 -- 172.21.15.94:0/1923014532 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fe33802d870 con 0x7fe3200a21e0 2024-02-17T11:57:27.514 INFO:teuthology.orchestra.run.smithi094.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-17T11:57:27.514 INFO:teuthology.orchestra.run.smithi094.stdout:mds.a smithi094 running (98s) 15s ago 97s 15.1M - 19.0.0-1515-g6822c7b5 32fa8af9a902 8d200e9967fa 2024-02-17T11:57:27.514 INFO:teuthology.orchestra.run.smithi094.stdout:mds.b smithi125 running (95s) 28s ago 95s 28.0M - 19.0.0-1515-g6822c7b5 32fa8af9a902 f403f8ef7044 2024-02-17T11:57:27.514 INFO:teuthology.orchestra.run.smithi094.stdout:mds.c smithi189 running (109s) 28s ago 109s 17.0M - 19.0.0-1515-g6822c7b5 32fa8af9a902 1241d9f934b6 2024-02-17T11:57:27.514 INFO:teuthology.orchestra.run.smithi094.stdout:mds.d smithi094 running (118s) 15s ago 117s 15.3M - 19.0.0-1515-g6822c7b5 32fa8af9a902 005a4583f339 2024-02-17T11:57:27.515 INFO:teuthology.orchestra.run.smithi094.stdout:mds.e smithi125 running (115s) 28s ago 114s 18.1M - 19.0.0-1515-g6822c7b5 32fa8af9a902 0eaf5a395ba0 2024-02-17T11:57:27.515 INFO:teuthology.orchestra.run.smithi094.stdout:mds.f smithi189 running (112s) 28s ago 112s 17.9M - 19.0.0-1515-g6822c7b5 32fa8af9a902 7a8d6cf40c0e 2024-02-17T11:57:27.515 INFO:teuthology.orchestra.run.smithi094.stdout:mds.g smithi094 running (101s) 15s ago 100s 16.2M - 19.0.0-1515-g6822c7b5 32fa8af9a902 2f5bb2bd9bd5 2024-02-17T11:57:27.515 INFO:teuthology.orchestra.run.smithi094.stdout:mds.h smithi125 running (106s) 28s ago 106s 19.1M - 19.0.0-1515-g6822c7b5 32fa8af9a902 0cfe22aa4582 2024-02-17T11:57:27.515 INFO:teuthology.orchestra.run.smithi094.stdout:mds.i smithi189 running (103s) 28s ago 103s 19.8M - 19.0.0-1515-g6822c7b5 32fa8af9a902 d80d879d3207 2024-02-17T11:57:27.515 INFO:teuthology.orchestra.run.smithi094.stdout:mds.j smithi094 running (2m) 15s ago 2m 15.2M - 19.0.0-1515-g6822c7b5 32fa8af9a902 8085e8836ad5 2024-02-17T11:57:27.515 INFO:teuthology.orchestra.run.smithi094.stdout:mds.k smithi125 running (2m) 28s ago 2m 16.1M - 19.0.0-1515-g6822c7b5 32fa8af9a902 db4b881c9c9b 2024-02-17T11:57:27.515 INFO:teuthology.orchestra.run.smithi094.stdout:mds.l smithi189 running (2m) 28s ago 2m 13.0M - 19.0.0-1515-g6822c7b5 32fa8af9a902 c9250c033e01 2024-02-17T11:57:27.515 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.x smithi094 *:9283,8765 running (9m) 15s ago 9m 512M - 19.0.0-1515-g6822c7b5 32fa8af9a902 452c06f58c50 2024-02-17T11:57:27.515 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.y smithi125 *:8443,8765 running (7m) 28s ago 7m 453M - 19.0.0-1515-g6822c7b5 32fa8af9a902 bb76ce397900 2024-02-17T11:57:27.515 INFO:teuthology.orchestra.run.smithi094.stdout:mgr.z smithi189 *:8443,8765 running (7m) 28s ago 7m 450M - 19.0.0-1515-g6822c7b5 32fa8af9a902 907011b69dcd 2024-02-17T11:57:27.515 INFO:teuthology.orchestra.run.smithi094.stdout:mon.a smithi094 running (9m) 15s ago 9m 52.7M 2048M 19.0.0-1515-g6822c7b5 32fa8af9a902 fb2eb803b897 2024-02-17T11:57:27.516 INFO:teuthology.orchestra.run.smithi094.stdout:mon.b smithi125 running (7m) 28s ago 7m 38.0M 2048M 19.0.0-1515-g6822c7b5 32fa8af9a902 232c18f9347d 2024-02-17T11:57:27.516 INFO:teuthology.orchestra.run.smithi094.stdout:mon.c smithi189 running (7m) 28s ago 7m 37.4M 2048M 19.0.0-1515-g6822c7b5 32fa8af9a902 a90006e43b15 2024-02-17T11:57:27.516 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 smithi094 running (7m) 15s ago 7m 55.7M 4272M 19.0.0-1515-g6822c7b5 32fa8af9a902 4e27d1ad4c35 2024-02-17T11:57:27.516 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 smithi125 running (6m) 28s ago 6m 44.8M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 ae4c64f2631d 2024-02-17T11:57:27.516 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 smithi189 running (6m) 28s ago 6m 46.5M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 3483965cd0cd 2024-02-17T11:57:27.516 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 smithi094 running (5m) 15s ago 5m 43.9M 4272M 19.0.0-1515-g6822c7b5 32fa8af9a902 0058133c7446 2024-02-17T11:57:27.516 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 smithi125 running (5m) 28s ago 5m 43.0M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 3818f7c8493b 2024-02-17T11:57:27.516 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 smithi189 running (4m) 28s ago 4m 55.7M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 085ff22ec9d9 2024-02-17T11:57:27.516 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 smithi094 running (4m) 15s ago 4m 42.4M 4272M 19.0.0-1515-g6822c7b5 32fa8af9a902 3fd1c36b839e 2024-02-17T11:57:27.516 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 smithi125 running (4m) 28s ago 4m 42.9M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 54c335f255a2 2024-02-17T11:57:27.516 INFO:teuthology.orchestra.run.smithi094.stdout:osd.8 smithi189 running (3m) 28s ago 3m 56.7M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 44dce335494f 2024-02-17T11:57:27.517 INFO:teuthology.orchestra.run.smithi094.stdout:osd.9 smithi094 running (3m) 15s ago 3m 40.7M 4272M 19.0.0-1515-g6822c7b5 32fa8af9a902 e205b66edec5 2024-02-17T11:57:27.517 INFO:teuthology.orchestra.run.smithi094.stdout:osd.10 smithi125 running (2m) 28s ago 2m 53.4M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 bf1642c5b51c 2024-02-17T11:57:27.517 INFO:teuthology.orchestra.run.smithi094.stdout:osd.11 smithi189 running (2m) 28s ago 2m 53.7M 4270M 19.0.0-1515-g6822c7b5 32fa8af9a902 14c9be00b6c3 2024-02-17T11:57:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.517+0000 7fe33d1c4700 1 -- 172.21.15.94:0/1923014532 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe3200a21e0 msgr2=0x7fe3200a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.517+0000 7fe33d1c4700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe3200a21e0 0x7fe3200a46a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe32c003790 tx=0x7fe32c005fd0 comp rx=0 tx=0).stop 2024-02-17T11:57:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.517+0000 7fe33d1c4700 1 -- 172.21.15.94:0/1923014532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe338102780 msgr2=0x7fe338197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.517+0000 7fe33d1c4700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe338102780 0x7fe338197d30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe33000ba80 tx=0x7fe33000bd90 comp rx=0 tx=0).stop 2024-02-17T11:57:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.517+0000 7fe33d1c4700 1 -- 172.21.15.94:0/1923014532 shutdown_connections 2024-02-17T11:57:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.517+0000 7fe33d1c4700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe3200a21e0 0x7fe3200a46a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.517+0000 7fe33d1c4700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3381079e0 0x7fe33819f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.517+0000 7fe33d1c4700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe338102780 0x7fe338197d30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:27.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.517+0000 7fe33d1c4700 1 --2- 172.21.15.94:0/1923014532 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe3381050b0 0x7fe338198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.517+0000 7fe33d1c4700 1 -- 172.21.15.94:0/1923014532 >> 172.21.15.94:0/1923014532 conn(0x7fe3380fc130 msgr2=0x7fe3381030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.517+0000 7fe33d1c4700 1 -- 172.21.15.94:0/1923014532 shutdown_connections 2024-02-17T11:57:27.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:27.517+0000 7fe33d1c4700 1 -- 172.21.15.94:0/1923014532 wait complete. 2024-02-17T11:57:28.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:27 smithi189 ceph-mon[36588]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T11:57:28.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:27 smithi125 ceph-mon[36819]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T11:57:28.184 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:27 smithi094 ceph-mon[32499]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T11:57:28.213 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-17T11:57:28.453 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:57:28.831 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:28 smithi094 ceph-mon[32499]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:29.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:28 smithi189 ceph-mon[36588]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:29.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:28 smithi125 ceph-mon[36819]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:29.903 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:29 smithi094 ceph-mon[32499]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-17T11:57:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.143+0000 7f466fcd8700 1 -- 172.21.15.94:0/2279563104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4668100ba0 msgr2=0x7f4668100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.143+0000 7f466fcd8700 1 --2- 172.21.15.94:0/2279563104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4668100ba0 0x7f4668100fc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f46580099c0 tx=0x7f4658009cd0 comp rx=0 tx=0).stop 2024-02-17T11:57:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.143+0000 7f466fcd8700 1 -- 172.21.15.94:0/2279563104 shutdown_connections 2024-02-17T11:57:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.143+0000 7f466fcd8700 1 --2- 172.21.15.94:0/2279563104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4668102740 0x7f4668108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.143+0000 7f466fcd8700 1 --2- 172.21.15.94:0/2279563104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4668100ba0 0x7f4668100fc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.143+0000 7f466fcd8700 1 --2- 172.21.15.94:0/2279563104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4668101da0 0x7f4668102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.143+0000 7f466fcd8700 1 -- 172.21.15.94:0/2279563104 >> 172.21.15.94:0/2279563104 conn(0x7f46680fc140 msgr2=0x7f46680fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.143+0000 7f466fcd8700 1 -- 172.21.15.94:0/2279563104 shutdown_connections 2024-02-17T11:57:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.143+0000 7f466fcd8700 1 -- 172.21.15.94:0/2279563104 wait complete. 2024-02-17T11:57:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466fcd8700 1 Processor -- start 2024-02-17T11:57:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466fcd8700 1 -- start start 2024-02-17T11:57:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466fcd8700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4668100ba0 0x7f4668104b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466fcd8700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4668101da0 0x7f4668103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466fcd8700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4668102740 0x7f46681036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466fcd8700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4668103e40 con 0x7f4668100ba0 2024-02-17T11:57:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466fcd8700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4668103fb0 con 0x7f4668101da0 2024-02-17T11:57:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466fcd8700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46681a1d90 con 0x7f4668102740 2024-02-17T11:57:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466da74700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4668100ba0 0x7f4668104b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466da74700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4668100ba0 0x7f4668104b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45474/0 (socket says 172.21.15.94:45474) 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466da74700 1 -- 172.21.15.94:0/2851321866 learned_addr learned my addr 172.21.15.94:0/2851321866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466e275700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4668102740 0x7f46681036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466d273700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4668101da0 0x7f4668103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466da74700 1 -- 172.21.15.94:0/2851321866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4668102740 msgr2=0x7f46681036c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466da74700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4668102740 0x7f46681036c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466da74700 1 -- 172.21.15.94:0/2851321866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4668101da0 msgr2=0x7f4668103180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466da74700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4668101da0 0x7f4668103180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466da74700 1 -- 172.21.15.94:0/2851321866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46580095f0 con 0x7f4668100ba0 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466e275700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4668102740 0x7f46681036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f466da74700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4668100ba0 0x7f4668104b30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4658000c00 tx=0x7f4658009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.144+0000 7f465effd700 1 -- 172.21.15.94:0/2851321866 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465801d070 con 0x7f4668100ba0 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.145+0000 7f466fcd8700 1 -- 172.21.15.94:0/2851321866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46681a2010 con 0x7f4668100ba0 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.145+0000 7f465effd700 1 -- 172.21.15.94:0/2851321866 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46580040d0 con 0x7f4668100ba0 2024-02-17T11:57:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.145+0000 7f465effd700 1 -- 172.21.15.94:0/2851321866 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4658022e00 con 0x7f4668100ba0 2024-02-17T11:57:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.145+0000 7f466fcd8700 1 -- 172.21.15.94:0/2851321866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46681a2500 con 0x7f4668100ba0 2024-02-17T11:57:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.145+0000 7f466fcd8700 1 -- 172.21.15.94:0/2851321866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4668068930 con 0x7f4668100ba0 2024-02-17T11:57:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.147+0000 7f465effd700 1 -- 172.21.15.94:0/2851321866 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4658022590 con 0x7f4668100ba0 2024-02-17T11:57:30.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.147+0000 7f465effd700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f46540a2150 0x7f46540a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:30.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.147+0000 7f466d273700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f46540a2150 0x7f46540a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:30.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.147+0000 7f465effd700 1 -- 172.21.15.94:0/2851321866 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f46580cf810 con 0x7f4668100ba0 2024-02-17T11:57:30.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.148+0000 7f466d273700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f46540a2150 0x7f46540a4610 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4664005950 tx=0x7f46640058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:30.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.151+0000 7f465effd700 1 -- 172.21.15.94:0/2851321866 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4658098870 con 0x7f4668100ba0 2024-02-17T11:57:30.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:29 smithi189 ceph-mon[36588]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-17T11:57:30.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:29 smithi125 ceph-mon[36819]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-02-17T11:57:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.352+0000 7f466fcd8700 1 -- 172.21.15.94:0/2851321866 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f46680fe370 con 0x7f46540a2150 2024-02-17T11:57:30.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.358+0000 7f465effd700 1 -- 172.21.15.94:0/2851321866 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f46680fe370 con 0x7f46540a2150 2024-02-17T11:57:30.357 INFO:teuthology.orchestra.run.smithi094.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-17T11:57:30.357 INFO:teuthology.orchestra.run.smithi094.stdout:mds.all 12/12 31s ago 2m smithi094=a;smithi094=d;smithi094=g;smithi094=j;smithi125=b;smithi125=e;smithi125=h;smithi125=k;smithi189=c;smithi189=f;smithi189=i;smithi189=l;count:12 2024-02-17T11:57:30.357 INFO:teuthology.orchestra.run.smithi094.stdout:mgr 3/3 31s ago 7m smithi094=x;smithi125=y;smithi189=z;count:3 2024-02-17T11:57:30.357 INFO:teuthology.orchestra.run.smithi094.stdout:mon 3/3 31s ago 7m smithi094:172.21.15.94=a;smithi125:172.21.15.125=b;smithi189:172.21.15.189=c;count:3 2024-02-17T11:57:30.357 INFO:teuthology.orchestra.run.smithi094.stdout:osd 12 31s ago - 2024-02-17T11:57:30.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.360+0000 7f466fcd8700 1 -- 172.21.15.94:0/2851321866 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f46540a2150 msgr2=0x7f46540a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:30.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.360+0000 7f466fcd8700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f46540a2150 0x7f46540a4610 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4664005950 tx=0x7f46640058e0 comp rx=0 tx=0).stop 2024-02-17T11:57:30.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.360+0000 7f466fcd8700 1 -- 172.21.15.94:0/2851321866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4668100ba0 msgr2=0x7f4668104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:30.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.360+0000 7f466fcd8700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4668100ba0 0x7f4668104b30 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4658000c00 tx=0x7f4658009580 comp rx=0 tx=0).stop 2024-02-17T11:57:30.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.360+0000 7f466fcd8700 1 -- 172.21.15.94:0/2851321866 shutdown_connections 2024-02-17T11:57:30.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.360+0000 7f466fcd8700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f46540a2150 0x7f46540a4610 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:30.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.360+0000 7f466fcd8700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4668102740 0x7f46681036c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:30.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.360+0000 7f466fcd8700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4668100ba0 0x7f4668104b30 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:30.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.360+0000 7f466fcd8700 1 --2- 172.21.15.94:0/2851321866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4668101da0 0x7f4668103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:30.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.360+0000 7f466fcd8700 1 -- 172.21.15.94:0/2851321866 >> 172.21.15.94:0/2851321866 conn(0x7f46680fc140 msgr2=0x7f46680fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:30.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.361+0000 7f466fcd8700 1 -- 172.21.15.94:0/2851321866 shutdown_connections 2024-02-17T11:57:30.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:30.361+0000 7f466fcd8700 1 -- 172.21.15.94:0/2851321866 wait complete. 2024-02-17T11:57:30.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:30 smithi094 ceph-mon[32499]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-02-17T11:57:31.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-17T11:57:31.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:30 smithi189 ceph-mon[36588]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-02-17T11:57:31.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:30 smithi125 ceph-mon[36819]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-02-17T11:57:31.296 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:57:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:31 smithi094 ceph-mon[32499]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:31.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:31 smithi094 ceph-mon[32499]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-17T11:57:32.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:31 smithi189 ceph-mon[36588]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:32.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:31 smithi189 ceph-mon[36588]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-17T11:57:32.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:31 smithi125 ceph-mon[36819]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:32.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:31 smithi125 ceph-mon[36819]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 341 B/s wr, 6 op/s 2024-02-17T11:57:33.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.025+0000 7f8ef62ad700 1 -- 172.21.15.94:0/192016822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0100ba0 msgr2=0x7f8ef0100fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:33.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.025+0000 7f8ef62ad700 1 --2- 172.21.15.94:0/192016822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0100ba0 0x7f8ef0100fc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8ee00099c0 tx=0x7f8ee0009cd0 comp rx=0 tx=0).stop 2024-02-17T11:57:33.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.025+0000 7f8ef62ad700 1 -- 172.21.15.94:0/192016822 shutdown_connections 2024-02-17T11:57:33.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.025+0000 7f8ef62ad700 1 --2- 172.21.15.94:0/192016822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ef0102740 0x7f8ef0108ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:33.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.025+0000 7f8ef62ad700 1 --2- 172.21.15.94:0/192016822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0100ba0 0x7f8ef0100fc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:33.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.025+0000 7f8ef62ad700 1 --2- 172.21.15.94:0/192016822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ef0101da0 0x7f8ef0102200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:33.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.025+0000 7f8ef62ad700 1 -- 172.21.15.94:0/192016822 >> 172.21.15.94:0/192016822 conn(0x7f8ef00fc160 msgr2=0x7f8ef00fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:33.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.025+0000 7f8ef62ad700 1 -- 172.21.15.94:0/192016822 shutdown_connections 2024-02-17T11:57:33.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.025+0000 7f8ef62ad700 1 -- 172.21.15.94:0/192016822 wait complete. 2024-02-17T11:57:33.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.026+0000 7f8ef62ad700 1 Processor -- start 2024-02-17T11:57:33.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.026+0000 7f8ef62ad700 1 -- start start 2024-02-17T11:57:33.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.026+0000 7f8ef62ad700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ef0100ba0 0x7f8ef0104b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:33.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.026+0000 7f8ef62ad700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ef0101da0 0x7f8ef0103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:33.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.026+0000 7f8ef62ad700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0102740 0x7f8ef01036c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:33.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.026+0000 7f8ef62ad700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef0103e40 con 0x7f8ef0102740 2024-02-17T11:57:33.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.026+0000 7f8ef62ad700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef0103fb0 con 0x7f8ef0100ba0 2024-02-17T11:57:33.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.026+0000 7f8ef62ad700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef01a1d40 con 0x7f8ef0101da0 2024-02-17T11:57:33.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.026+0000 7f8ef484a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0102740 0x7f8ef01036c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.026+0000 7f8ef484a700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0102740 0x7f8ef01036c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45480/0 (socket says 172.21.15.94:45480) 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8ef484a700 1 -- 172.21.15.94:0/1833367877 learned_addr learned my addr 172.21.15.94:0/1833367877 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8eef7fe700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ef0101da0 0x7f8ef0103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8eeffff700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ef0100ba0 0x7f8ef0104b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8ef484a700 1 -- 172.21.15.94:0/1833367877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ef0101da0 msgr2=0x7f8ef0103180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8ef484a700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ef0101da0 0x7f8ef0103180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8ef484a700 1 -- 172.21.15.94:0/1833367877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ef0100ba0 msgr2=0x7f8ef0104b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8ef484a700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ef0100ba0 0x7f8ef0104b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8ef484a700 1 -- 172.21.15.94:0/1833367877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee00095f0 con 0x7f8ef0102740 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8eeffff700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ef0100ba0 0x7f8ef0104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8ef484a700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0102740 0x7f8ef01036c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800b970 tx=0x7f8ee800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8eed7fa700 1 -- 172.21.15.94:0/1833367877 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800c930 con 0x7f8ef0102740 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8eed7fa700 1 -- 172.21.15.94:0/1833367877 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8ee8014440 con 0x7f8ef0102740 2024-02-17T11:57:33.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8ef62ad700 1 -- 172.21.15.94:0/1833367877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef01a1fc0 con 0x7f8ef0102740 2024-02-17T11:57:33.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8eed7fa700 1 -- 172.21.15.94:0/1833367877 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80075d0 con 0x7f8ef0102740 2024-02-17T11:57:33.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.027+0000 7f8ef62ad700 1 -- 172.21.15.94:0/1833367877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef01a2440 con 0x7f8ef0102740 2024-02-17T11:57:33.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.028+0000 7f8ef62ad700 1 -- 172.21.15.94:0/1833367877 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef0068930 con 0x7f8ef0102740 2024-02-17T11:57:33.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.029+0000 7f8eed7fa700 1 -- 172.21.15.94:0/1833367877 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f8ee80145b0 con 0x7f8ef0102740 2024-02-17T11:57:33.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.030+0000 7f8eed7fa700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ed80a2150 0x7f8ed80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:33.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.030+0000 7f8eeffff700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ed80a2150 0x7f8ed80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:33.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.030+0000 7f8eed7fa700 1 -- 172.21.15.94:0/1833367877 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8ee8006e40 con 0x7f8ef0102740 2024-02-17T11:57:33.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.030+0000 7f8eeffff700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ed80a2150 0x7f8ed80a4610 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000b580 tx=0x7f8ee0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:33.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.034+0000 7f8eed7fa700 1 -- 172.21.15.94:0/1833367877 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8ee8096320 con 0x7f8ef0102740 2024-02-17T11:57:33.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.237+0000 7f8ef62ad700 1 -- 172.21.15.94:0/1833367877 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ef00fe340 con 0x7f8ed80a2150 2024-02-17T11:57:33.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.238+0000 7f8eed7fa700 1 -- 172.21.15.94:0/1833367877 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f8ef00fe340 con 0x7f8ed80a2150 2024-02-17T11:57:33.237 INFO:teuthology.orchestra.run.smithi094.stdout:HOST ADDR LABELS STATUS 2024-02-17T11:57:33.237 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 172.21.15.94 2024-02-17T11:57:33.237 INFO:teuthology.orchestra.run.smithi094.stdout:smithi125 172.21.15.125 2024-02-17T11:57:33.237 INFO:teuthology.orchestra.run.smithi094.stdout:smithi189 172.21.15.189 2024-02-17T11:57:33.237 INFO:teuthology.orchestra.run.smithi094.stdout:3 hosts in cluster 2024-02-17T11:57:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.240+0000 7f8ef62ad700 1 -- 172.21.15.94:0/1833367877 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ed80a2150 msgr2=0x7f8ed80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.240+0000 7f8ef62ad700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ed80a2150 0x7f8ed80a4610 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000b580 tx=0x7f8ee0005fb0 comp rx=0 tx=0).stop 2024-02-17T11:57:33.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.240+0000 7f8ef62ad700 1 -- 172.21.15.94:0/1833367877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0102740 msgr2=0x7f8ef01036c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:33.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.240+0000 7f8ef62ad700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0102740 0x7f8ef01036c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800b970 tx=0x7f8ee800bc80 comp rx=0 tx=0).stop 2024-02-17T11:57:33.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.240+0000 7f8ef62ad700 1 -- 172.21.15.94:0/1833367877 shutdown_connections 2024-02-17T11:57:33.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.240+0000 7f8ef62ad700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8ed80a2150 0x7f8ed80a4610 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:33.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.240+0000 7f8ef62ad700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8ef0102740 0x7f8ef01036c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:33.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.240+0000 7f8ef62ad700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8ef0100ba0 0x7f8ef0104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:33.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.240+0000 7f8ef62ad700 1 --2- 172.21.15.94:0/1833367877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ef0101da0 0x7f8ef0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:33.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.240+0000 7f8ef62ad700 1 -- 172.21.15.94:0/1833367877 >> 172.21.15.94:0/1833367877 conn(0x7f8ef00fc160 msgr2=0x7f8ef00fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:33.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.241+0000 7f8ef62ad700 1 -- 172.21.15.94:0/1833367877 shutdown_connections 2024-02-17T11:57:33.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:33.241+0000 7f8ef62ad700 1 -- 172.21.15.94:0/1833367877 wait complete. 2024-02-17T11:57:33.965 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-17T11:57:34.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:33 smithi189 ceph-mon[36588]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-17T11:57:34.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:33 smithi125 ceph-mon[36819]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-17T11:57:34.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:33 smithi094 ceph-mon[32499]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-17T11:57:34.207 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:57:34.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:34 smithi094 ceph-mon[32499]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:35.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:34 smithi189 ceph-mon[36588]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:35.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:34 smithi125 ceph-mon[36819]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:35.897 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:35 smithi094 ceph-mon[32499]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-17T11:57:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 -- 172.21.15.94:0/1456819450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0101ca0 msgr2=0x7f58b01020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 --2- 172.21.15.94:0/1456819450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0101ca0 0x7f58b01020c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f58a80099c0 tx=0x7f58a8009cd0 comp rx=0 tx=0).stop 2024-02-17T11:57:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 -- 172.21.15.94:0/1456819450 shutdown_connections 2024-02-17T11:57:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 --2- 172.21.15.94:0/1456819450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b0103840 0x7f58b010a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 --2- 172.21.15.94:0/1456819450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0101ca0 0x7f58b01020c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 --2- 172.21.15.94:0/1456819450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58b0102ea0 0x7f58b0103300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 -- 172.21.15.94:0/1456819450 >> 172.21.15.94:0/1456819450 conn(0x7f58b00fd280 msgr2=0x7f58b00ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 -- 172.21.15.94:0/1456819450 shutdown_connections 2024-02-17T11:57:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 -- 172.21.15.94:0/1456819450 wait complete. 2024-02-17T11:57:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 Processor -- start 2024-02-17T11:57:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 -- start start 2024-02-17T11:57:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b0101ca0 0x7f58b0197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58b0102ea0 0x7f58b0198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0103840 0x7f58b019f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b019f850 con 0x7f58b0103840 2024-02-17T11:57:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b019f990 con 0x7f58b0101ca0 2024-02-17T11:57:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.997+0000 7f58b6aaf700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b019fb50 con 0x7f58b0102ea0 2024-02-17T11:57:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58b504c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0103840 0x7f58b019f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58b504c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0103840 0x7f58b019f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48248/0 (socket says 172.21.15.94:48248) 2024-02-17T11:57:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58b504c700 1 -- 172.21.15.94:0/3356863259 learned_addr learned my addr 172.21.15.94:0/3356863259 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58b484b700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b0101ca0 0x7f58b0197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58affff700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58b0102ea0 0x7f58b0198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58b504c700 1 -- 172.21.15.94:0/3356863259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58b0102ea0 msgr2=0x7f58b0198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58b504c700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58b0102ea0 0x7f58b0198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58b504c700 1 -- 172.21.15.94:0/3356863259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b0101ca0 msgr2=0x7f58b0197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58b504c700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b0101ca0 0x7f58b0197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58b504c700 1 -- 172.21.15.94:0/3356863259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a80095f0 con 0x7f58b0103840 2024-02-17T11:57:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58affff700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58b0102ea0 0x7f58b0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58b484b700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b0101ca0 0x7f58b0197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58b504c700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0103840 0x7f58b019f280 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f589800d980 tx=0x7f589800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58adffb700 1 -- 172.21.15.94:0/3356863259 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5898009940 con 0x7f58b0103840 2024-02-17T11:57:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58b6aaf700 1 -- 172.21.15.94:0/3356863259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b019fe30 con 0x7f58b0103840 2024-02-17T11:57:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58adffb700 1 -- 172.21.15.94:0/3356863259 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5898004830 con 0x7f58b0103840 2024-02-17T11:57:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58adffb700 1 -- 172.21.15.94:0/3356863259 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58980054b0 con 0x7f58b0103840 2024-02-17T11:57:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.998+0000 7f58b6aaf700 1 -- 172.21.15.94:0/3356863259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b01a0240 con 0x7f58b0103840 2024-02-17T11:57:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:35.999+0000 7f58b6aaf700 1 -- 172.21.15.94:0/3356863259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b01090e0 con 0x7f58b0103840 2024-02-17T11:57:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.000+0000 7f58adffb700 1 -- 172.21.15.94:0/3356863259 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5898004d30 con 0x7f58b0103840 2024-02-17T11:57:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.001+0000 7f58adffb700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f589c0a2150 0x7f589c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.001+0000 7f58adffb700 1 -- 172.21.15.94:0/3356863259 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f58980cdc20 con 0x7f58b0103840 2024-02-17T11:57:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.001+0000 7f58b484b700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f589c0a2150 0x7f589c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.001+0000 7f58b484b700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f589c0a2150 0x7f589c0a4610 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009990 tx=0x7f58a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.005+0000 7f58adffb700 1 -- 172.21.15.94:0/3356863259 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5898096bf0 con 0x7f58b0103840 2024-02-17T11:57:36.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:35 smithi189 ceph-mon[36588]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-17T11:57:36.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:35 smithi125 ceph-mon[36819]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-17T11:57:36.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.207+0000 7f58b6aaf700 1 -- 172.21.15.94:0/3356863259 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f58b002da20 con 0x7f589c0a2150 2024-02-17T11:57:36.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.209+0000 7f58adffb700 1 -- 172.21.15.94:0/3356863259 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f58b002da20 con 0x7f589c0a2150 2024-02-17T11:57:36.208 INFO:teuthology.orchestra.run.smithi094.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-17T11:57:36.208 INFO:teuthology.orchestra.run.smithi094.stdout:smithi094 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001L480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-02-17T11:57:36.208 INFO:teuthology.orchestra.run.smithi094.stdout:smithi125 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040123400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-17T11:57:36.208 INFO:teuthology.orchestra.run.smithi094.stdout:smithi189 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232004L400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-17T11:57:36.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.211+0000 7f58b6aaf700 1 -- 172.21.15.94:0/3356863259 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f589c0a2150 msgr2=0x7f589c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:36.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.211+0000 7f58b6aaf700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f589c0a2150 0x7f589c0a4610 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f58a8009990 tx=0x7f58a8005fb0 comp rx=0 tx=0).stop 2024-02-17T11:57:36.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.211+0000 7f58b6aaf700 1 -- 172.21.15.94:0/3356863259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0103840 msgr2=0x7f58b019f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:36.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.211+0000 7f58b6aaf700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0103840 0x7f58b019f280 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f589800d980 tx=0x7f589800dd40 comp rx=0 tx=0).stop 2024-02-17T11:57:36.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.211+0000 7f58b6aaf700 1 -- 172.21.15.94:0/3356863259 shutdown_connections 2024-02-17T11:57:36.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.211+0000 7f58b6aaf700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f589c0a2150 0x7f589c0a4610 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:36.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.211+0000 7f58b6aaf700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f58b0103840 0x7f58b019f280 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:36.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.211+0000 7f58b6aaf700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f58b0101ca0 0x7f58b0197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:36.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.211+0000 7f58b6aaf700 1 --2- 172.21.15.94:0/3356863259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58b0102ea0 0x7f58b0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:36.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.211+0000 7f58b6aaf700 1 -- 172.21.15.94:0/3356863259 >> 172.21.15.94:0/3356863259 conn(0x7f58b00fd280 msgr2=0x7f58b0106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:36.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.211+0000 7f58b6aaf700 1 -- 172.21.15.94:0/3356863259 shutdown_connections 2024-02-17T11:57:36.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:36.211+0000 7f58b6aaf700 1 -- 172.21.15.94:0/3356863259 wait complete. 2024-02-17T11:57:36.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:36 smithi094 ceph-mon[32499]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:36.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:36 smithi094 ceph-mon[32499]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-17T11:57:37.023 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-17T11:57:37.032 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-02-17T11:57:37.032 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-17T11:57:37.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:36 smithi189 ceph-mon[36588]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:37.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:36 smithi189 ceph-mon[36588]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-17T11:57:37.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:36 smithi125 ceph-mon[36819]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:37.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:36 smithi125 ceph-mon[36819]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-17T11:57:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:57:39.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.005+0000 7fc3e7789700 1 -- 172.21.15.94:0/4007657817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3e0102f90 msgr2=0x7fc3e01033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:39.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.005+0000 7fc3e7789700 1 --2- 172.21.15.94:0/4007657817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3e0102f90 0x7fc3e01033f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009990 tx=0x7fc3dc009ca0 comp rx=0 tx=0).stop 2024-02-17T11:57:39.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.005+0000 7fc3e7789700 1 -- 172.21.15.94:0/4007657817 shutdown_connections 2024-02-17T11:57:39.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.005+0000 7fc3e7789700 1 --2- 172.21.15.94:0/4007657817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3e0103930 0x7fc3e010a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:39.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.005+0000 7fc3e7789700 1 --2- 172.21.15.94:0/4007657817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3e00fffb0 0x7fc3e01003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:39.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.005+0000 7fc3e7789700 1 --2- 172.21.15.94:0/4007657817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3e0102f90 0x7fc3e01033f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:39.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.005+0000 7fc3e7789700 1 -- 172.21.15.94:0/4007657817 >> 172.21.15.94:0/4007657817 conn(0x7fc3e00fb910 msgr2=0x7fc3e00fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:39.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.005+0000 7fc3e7789700 1 -- 172.21.15.94:0/4007657817 shutdown_connections 2024-02-17T11:57:39.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.005+0000 7fc3e7789700 1 -- 172.21.15.94:0/4007657817 wait complete. 2024-02-17T11:57:39.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.006+0000 7fc3e7789700 1 Processor -- start 2024-02-17T11:57:39.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.006+0000 7fc3e7789700 1 -- start start 2024-02-17T11:57:39.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.006+0000 7fc3e7789700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3e00fffb0 0x7fc3e01028f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:39.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.006+0000 7fc3e7789700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3e0102f90 0x7fc3e0100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:39.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.006+0000 7fc3e7789700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3e0103930 0x7fc3e0101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:39.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.006+0000 7fc3e7789700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e0101c00 con 0x7fc3e0103930 2024-02-17T11:57:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.006+0000 7fc3e7789700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e0101d70 con 0x7fc3e00fffb0 2024-02-17T11:57:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.006+0000 7fc3e7789700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e01a1d90 con 0x7fc3e0102f90 2024-02-17T11:57:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.006+0000 7fc3e5d26700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3e0103930 0x7fc3e0101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.006+0000 7fc3e5d26700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3e0103930 0x7fc3e0101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48266/0 (socket says 172.21.15.94:48266) 2024-02-17T11:57:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.006+0000 7fc3e5d26700 1 -- 172.21.15.94:0/84411632 learned_addr learned my addr 172.21.15.94:0/84411632 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.006+0000 7fc3e5525700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3e00fffb0 0x7fc3e01028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.007+0000 7fc3e4d24700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3e0102f90 0x7fc3e0100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.007+0000 7fc3e5d26700 1 -- 172.21.15.94:0/84411632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3e0102f90 msgr2=0x7fc3e0100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.007+0000 7fc3e5d26700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3e0102f90 0x7fc3e0100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.007+0000 7fc3e5d26700 1 -- 172.21.15.94:0/84411632 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3e00fffb0 msgr2=0x7fc3e01028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.007+0000 7fc3e5d26700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3e00fffb0 0x7fc3e01028f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.007+0000 7fc3e5d26700 1 -- 172.21.15.94:0/84411632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc0095f0 con 0x7fc3e0103930 2024-02-17T11:57:39.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.007+0000 7fc3e5d26700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3e0103930 0x7fc3e0101480 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800b970 tx=0x7fc3d800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.007+0000 7fc3d67fc700 1 -- 172.21.15.94:0/84411632 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d800c9b0 con 0x7fc3e0103930 2024-02-17T11:57:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.007+0000 7fc3d67fc700 1 -- 172.21.15.94:0/84411632 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3d8014440 con 0x7fc3e0103930 2024-02-17T11:57:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.007+0000 7fc3e7789700 1 -- 172.21.15.94:0/84411632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e01a2070 con 0x7fc3e0103930 2024-02-17T11:57:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.007+0000 7fc3d67fc700 1 -- 172.21.15.94:0/84411632 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d80075d0 con 0x7fc3e0103930 2024-02-17T11:57:39.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.007+0000 7fc3e7789700 1 -- 172.21.15.94:0/84411632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e01a25f0 con 0x7fc3e0103930 2024-02-17T11:57:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.008+0000 7fc3e7789700 1 -- 172.21.15.94:0/84411632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e01090f0 con 0x7fc3e0103930 2024-02-17T11:57:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.009+0000 7fc3d67fc700 1 -- 172.21.15.94:0/84411632 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fc3d8014920 con 0x7fc3e0103930 2024-02-17T11:57:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.010+0000 7fc3d67fc700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc3cc0a21d0 0x7fc3cc0a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.010+0000 7fc3e5525700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc3cc0a21d0 0x7fc3cc0a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.010+0000 7fc3d67fc700 1 -- 172.21.15.94:0/84411632 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc3d8006e40 con 0x7fc3e0103930 2024-02-17T11:57:39.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.010+0000 7fc3e5525700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc3cc0a21d0 0x7fc3cc0a4690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0006fd0 tx=0x7fc3d0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:39.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.014+0000 7fc3d67fc700 1 -- 172.21.15.94:0/84411632 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc3d8096350 con 0x7fc3e0103930 2024-02-17T11:57:39.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.250+0000 7fc3e7789700 1 -- 172.21.15.94:0/84411632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc3e0068930 con 0x7fc3e0103930 2024-02-17T11:57:39.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.251+0000 7fc3d67fc700 1 -- 172.21.15.94:0/84411632 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+2943 (secure 0 0 0) 0x7fc3d8019b90 con 0x7fc3e0103930 2024-02-17T11:57:39.250 INFO:teuthology.orchestra.run.smithi094.stdout:e20 2024-02-17T11:57:39.250 INFO:teuthology.orchestra.run.smithi094.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-17T11:57:39.250 INFO:teuthology.orchestra.run.smithi094.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-17T11:57:39.250 INFO:teuthology.orchestra.run.smithi094.stdout:legacy client fscid: 1 2024-02-17T11:57:39.250 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:39.250 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem 'cephfs' (1) 2024-02-17T11:57:39.251 INFO:teuthology.orchestra.run.smithi094.stdout:fs_name cephfs 2024-02-17T11:57:39.251 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 20 2024-02-17T11:57:39.251 INFO:teuthology.orchestra.run.smithi094.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-17T11:57:39.251 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-02-17T11:55:58.000681+0000 2024-02-17T11:57:39.251 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-02-17T11:56:02.151161+0000 2024-02-17T11:57:39.251 INFO:teuthology.orchestra.run.smithi094.stdout:tableserver 0 2024-02-17T11:57:39.251 INFO:teuthology.orchestra.run.smithi094.stdout:root 0 2024-02-17T11:57:39.251 INFO:teuthology.orchestra.run.smithi094.stdout:session_timeout 300 2024-02-17T11:57:39.251 INFO:teuthology.orchestra.run.smithi094.stdout:session_autoclose 300 2024-02-17T11:57:39.251 INFO:teuthology.orchestra.run.smithi094.stdout:max_file_size 1099511627776 2024-02-17T11:57:39.252 INFO:teuthology.orchestra.run.smithi094.stdout:max_xattr_size 65536 2024-02-17T11:57:39.252 INFO:teuthology.orchestra.run.smithi094.stdout:required_client_features {} 2024-02-17T11:57:39.252 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure 0 2024-02-17T11:57:39.252 INFO:teuthology.orchestra.run.smithi094.stdout:last_failure_osd_epoch 0 2024-02-17T11:57:39.252 INFO:teuthology.orchestra.run.smithi094.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-17T11:57:39.252 INFO:teuthology.orchestra.run.smithi094.stdout:max_mds 3 2024-02-17T11:57:39.252 INFO:teuthology.orchestra.run.smithi094.stdout:in 0,1,2 2024-02-17T11:57:39.252 INFO:teuthology.orchestra.run.smithi094.stdout:up {0=24488,1=24487,2=24469} 2024-02-17T11:57:39.252 INFO:teuthology.orchestra.run.smithi094.stdout:failed 2024-02-17T11:57:39.253 INFO:teuthology.orchestra.run.smithi094.stdout:damaged 2024-02-17T11:57:39.253 INFO:teuthology.orchestra.run.smithi094.stdout:stopped 2024-02-17T11:57:39.253 INFO:teuthology.orchestra.run.smithi094.stdout:data_pools [3] 2024-02-17T11:57:39.253 INFO:teuthology.orchestra.run.smithi094.stdout:metadata_pool 2 2024-02-17T11:57:39.253 INFO:teuthology.orchestra.run.smithi094.stdout:inline_data disabled 2024-02-17T11:57:39.253 INFO:teuthology.orchestra.run.smithi094.stdout:balancer 2024-02-17T11:57:39.253 INFO:teuthology.orchestra.run.smithi094.stdout:bal_rank_mask -1 2024-02-17T11:57:39.253 INFO:teuthology.orchestra.run.smithi094.stdout:standby_count_wanted 0 2024-02-17T11:57:39.253 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.b{0:24488} state up:active seq 3 addr [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T11:57:39.254 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.f{0:24454} state up:standby-replay seq 1 addr [v2:172.21.15.189:6834/2720928164,v1:172.21.15.189:6835/2720928164] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T11:57:39.254 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.i{1:24487} state up:active seq 6 addr [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T11:57:39.254 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.h{1:24452} state up:standby-replay seq 1 addr [v2:172.21.15.125:6836/1075914834,v1:172.21.15.125:6837/1075914834] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T11:57:39.254 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.c{2:24469} state up:active seq 7 addr [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T11:57:39.254 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.e{2:24422} state up:standby-replay seq 1 addr [v2:172.21.15.125:6834/1238777175,v1:172.21.15.125:6835/1238777175] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T11:57:39.254 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:39.254 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:39.254 INFO:teuthology.orchestra.run.smithi094.stdout:Standby daemons: 2024-02-17T11:57:39.254 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:39.254 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.94:6834/2052147813,v1:172.21.15.94:6835/2052147813] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T11:57:39.255 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.94:6836/2565649123,v1:172.21.15.94:6837/2565649123] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T11:57:39.255 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.g{-1:14601} state up:standby seq 1 addr [v2:172.21.15.94:6838/2246951573,v1:172.21.15.94:6839/2246951573] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T11:57:39.255 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.a{-1:14613} state up:standby seq 1 addr [v2:172.21.15.94:6840/755410555,v1:172.21.15.94:6841/755410555] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T11:57:39.255 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.k{-1:24398} state up:standby seq 1 addr [v2:172.21.15.125:6832/1464839618,v1:172.21.15.125:6833/1464839618] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T11:57:39.255 INFO:teuthology.orchestra.run.smithi094.stdout:[mds.l{-1:24418} state up:standby seq 1 addr [v2:172.21.15.189:6832/1397235152,v1:172.21.15.189:6833/1397235152] compat {c=[1],r=[1],i=[fff]}] 2024-02-17T11:57:39.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.253+0000 7fc3e7789700 1 -- 172.21.15.94:0/84411632 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc3cc0a21d0 msgr2=0x7fc3cc0a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:39.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.253+0000 7fc3e7789700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc3cc0a21d0 0x7fc3cc0a4690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0006fd0 tx=0x7fc3d0008040 comp rx=0 tx=0).stop 2024-02-17T11:57:39.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.253+0000 7fc3e7789700 1 -- 172.21.15.94:0/84411632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3e0103930 msgr2=0x7fc3e0101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:39.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.253+0000 7fc3e7789700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3e0103930 0x7fc3e0101480 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc3d800b970 tx=0x7fc3d800bd30 comp rx=0 tx=0).stop 2024-02-17T11:57:39.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.253+0000 7fc3e7789700 1 -- 172.21.15.94:0/84411632 shutdown_connections 2024-02-17T11:57:39.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.253+0000 7fc3e7789700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc3cc0a21d0 0x7fc3cc0a4690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:39.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.253+0000 7fc3e7789700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc3e0103930 0x7fc3e0101480 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:39.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.253+0000 7fc3e7789700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc3e00fffb0 0x7fc3e01028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:39.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.253+0000 7fc3e7789700 1 --2- 172.21.15.94:0/84411632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3e0102f90 0x7fc3e0100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:39.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.253+0000 7fc3e7789700 1 -- 172.21.15.94:0/84411632 >> 172.21.15.94:0/84411632 conn(0x7fc3e00fb910 msgr2=0x7fc3e0106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:39.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.253+0000 7fc3e7789700 1 -- 172.21.15.94:0/84411632 shutdown_connections 2024-02-17T11:57:39.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:39.253+0000 7fc3e7789700 1 -- 172.21.15.94:0/84411632 wait complete. 2024-02-17T11:57:39.258 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 20 2024-02-17T11:57:39.930 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:39 smithi094 ceph-mon[32499]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-17T11:57:39.930 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:39 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/84411632' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-17T11:57:39.960 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- bash -c 'ceph osd dump' 2024-02-17T11:57:40.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:39 smithi189 ceph-mon[36588]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-17T11:57:40.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:39 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/84411632' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-17T11:57:40.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:39 smithi125 ceph-mon[36819]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-02-17T11:57:40.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:39 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/84411632' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-17T11:57:40.201 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:57:41.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.920+0000 7f0894a68700 1 -- 172.21.15.94:0/2276205654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08901005a0 msgr2=0x7f08901009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:41.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.920+0000 7f0894a68700 1 --2- 172.21.15.94:0/2276205654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08901005a0 0x7f08901009c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0880009990 tx=0x7f0880009ca0 comp rx=0 tx=0).stop 2024-02-17T11:57:41.921 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:41 smithi094 ceph-mon[32499]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-17T11:57:41.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.920+0000 7f0894a68700 1 -- 172.21.15.94:0/2276205654 shutdown_connections 2024-02-17T11:57:41.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.920+0000 7f0894a68700 1 --2- 172.21.15.94:0/2276205654 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08901018c0 0x7f0890101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:41.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.920+0000 7f0894a68700 1 --2- 172.21.15.94:0/2276205654 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08901005a0 0x7f08901009c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:41.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.920+0000 7f0894a68700 1 --2- 172.21.15.94:0/2276205654 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0890100f00 0x7f0890101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:41.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.920+0000 7f0894a68700 1 -- 172.21.15.94:0/2276205654 >> 172.21.15.94:0/2276205654 conn(0x7f08900fc160 msgr2=0x7f08900fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:41.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.920+0000 7f0894a68700 1 -- 172.21.15.94:0/2276205654 shutdown_connections 2024-02-17T11:57:41.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.921+0000 7f0894a68700 1 -- 172.21.15.94:0/2276205654 wait complete. 2024-02-17T11:57:41.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.921+0000 7f0894a68700 1 Processor -- start 2024-02-17T11:57:41.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.921+0000 7f0894a68700 1 -- start start 2024-02-17T11:57:41.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f0894a68700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08901005a0 0x7f0890199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:41.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f0894a68700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0890100f00 0x7f089019a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:41.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f0894a68700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08901018c0 0x7f08901a13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:41.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f0894a68700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08901a1970 con 0x7f0890100f00 2024-02-17T11:57:41.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f0894a68700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08901a1ab0 con 0x7f08901018c0 2024-02-17T11:57:41.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f0894a68700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08901a1c40 con 0x7f08901005a0 2024-02-17T11:57:41.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f088dd9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0890100f00 0x7f089019a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:41.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f088dd9b700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0890100f00 0x7f089019a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48284/0 (socket says 172.21.15.94:48284) 2024-02-17T11:57:41.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f088dd9b700 1 -- 172.21.15.94:0/84169316 learned_addr learned my addr 172.21.15.94:0/84169316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:41.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f088ed9d700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08901018c0 0x7f08901a13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:41.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f088e59c700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08901005a0 0x7f0890199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:41.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f088dd9b700 1 -- 172.21.15.94:0/84169316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08901005a0 msgr2=0x7f0890199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:41.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f088dd9b700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08901005a0 0x7f0890199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:41.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f088dd9b700 1 -- 172.21.15.94:0/84169316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08901018c0 msgr2=0x7f08901a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:41.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f088dd9b700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08901018c0 0x7f08901a13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:41.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f088dd9b700 1 -- 172.21.15.94:0/84169316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800095f0 con 0x7f0890100f00 2024-02-17T11:57:41.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.922+0000 7f088dd9b700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0890100f00 0x7f089019a320 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f088400c9e0 tx=0x7f088400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:41.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.923+0000 7f087f7fe700 1 -- 172.21.15.94:0/84169316 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884007ac0 con 0x7f0890100f00 2024-02-17T11:57:41.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.923+0000 7f0894a68700 1 -- 172.21.15.94:0/84169316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08901a1f20 con 0x7f0890100f00 2024-02-17T11:57:41.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.923+0000 7f087f7fe700 1 -- 172.21.15.94:0/84169316 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0884007c20 con 0x7f0890100f00 2024-02-17T11:57:41.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.923+0000 7f087f7fe700 1 -- 172.21.15.94:0/84169316 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884018660 con 0x7f0890100f00 2024-02-17T11:57:41.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.923+0000 7f0894a68700 1 -- 172.21.15.94:0/84169316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08901a23e0 con 0x7f0890100f00 2024-02-17T11:57:41.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.924+0000 7f0894a68700 1 -- 172.21.15.94:0/84169316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089010b380 con 0x7f0890100f00 2024-02-17T11:57:41.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.925+0000 7f087f7fe700 1 -- 172.21.15.94:0/84169316 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f088400f450 con 0x7f0890100f00 2024-02-17T11:57:41.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.925+0000 7f087f7fe700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08780a2150 0x7f08780a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:41.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.925+0000 7f087f7fe700 1 -- 172.21.15.94:0/84169316 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f08840cdea0 con 0x7f0890100f00 2024-02-17T11:57:41.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.925+0000 7f088e59c700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08780a2150 0x7f08780a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:41.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.926+0000 7f088e59c700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08780a2150 0x7f08780a4610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f088000b5c0 tx=0x7f0880005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:41.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:41.929+0000 7f087f7fe700 1 -- 172.21.15.94:0/84169316 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0884096f00 con 0x7f0890100f00 2024-02-17T11:57:42.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.122+0000 7f0894a68700 1 -- 172.21.15.94:0/84169316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f0890068930 con 0x7f0890100f00 2024-02-17T11:57:42.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.123+0000 7f087f7fe700 1 -- 172.21.15.94:0/84169316 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5304 (secure 0 0 0) 0x7f088409a5c0 con 0x7f0890100f00 2024-02-17T11:57:42.123 INFO:teuthology.orchestra.run.smithi094.stdout:epoch 87 2024-02-17T11:57:42.123 INFO:teuthology.orchestra.run.smithi094.stdout:fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T11:57:42.123 INFO:teuthology.orchestra.run.smithi094.stdout:created 2024-02-17T11:48:04.980164+0000 2024-02-17T11:57:42.123 INFO:teuthology.orchestra.run.smithi094.stdout:modified 2024-02-17T11:56:59.725646+0000 2024-02-17T11:57:42.123 INFO:teuthology.orchestra.run.smithi094.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-02-17T11:57:42.123 INFO:teuthology.orchestra.run.smithi094.stdout:crush_version 26 2024-02-17T11:57:42.123 INFO:teuthology.orchestra.run.smithi094.stdout:full_ratio 0.95 2024-02-17T11:57:42.124 INFO:teuthology.orchestra.run.smithi094.stdout:backfillfull_ratio 0.9 2024-02-17T11:57:42.124 INFO:teuthology.orchestra.run.smithi094.stdout:nearfull_ratio 0.85 2024-02-17T11:57:42.124 INFO:teuthology.orchestra.run.smithi094.stdout:require_min_compat_client luminous 2024-02-17T11:57:42.124 INFO:teuthology.orchestra.run.smithi094.stdout:min_compat_client luminous 2024-02-17T11:57:42.124 INFO:teuthology.orchestra.run.smithi094.stdout:require_osd_release squid 2024-02-17T11:57:42.124 INFO:teuthology.orchestra.run.smithi094.stdout:stretch_mode_enabled false 2024-02-17T11:57:42.124 INFO:teuthology.orchestra.run.smithi094.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-02-17T11:57:42.124 INFO:teuthology.orchestra.run.smithi094.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-02-17T11:57:42.124 INFO:teuthology.orchestra.run.smithi094.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-02-17T11:57:42.124 INFO:teuthology.orchestra.run.smithi094.stdout:max_osd 12 2024-02-17T11:57:42.124 INFO:teuthology.orchestra.run.smithi094.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6802/4275068800,v1:172.21.15.94:6803/4275068800] [v2:172.21.15.94:6804/4275068800,v1:172.21.15.94:6805/4275068800] exists,up e7ed7cb6-3fc6-4264-8da2-fb1c340fe308 2024-02-17T11:57:42.124 INFO:teuthology.orchestra.run.smithi094.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.125:6800/142575339,v1:172.21.15.125:6801/142575339] [v2:172.21.15.125:6802/142575339,v1:172.21.15.125:6803/142575339] exists,up bf2a8735-cd15-4041-9013-178ba85852c2 2024-02-17T11:57:42.124 INFO:teuthology.orchestra.run.smithi094.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6800/1204918236,v1:172.21.15.189:6801/1204918236] [v2:172.21.15.189:6802/1204918236,v1:172.21.15.189:6803/1204918236] exists,up 65b51d55-af5c-411c-988a-05331aad4a52 2024-02-17T11:57:42.125 INFO:teuthology.orchestra.run.smithi094.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6810/3136219621,v1:172.21.15.94:6811/3136219621] [v2:172.21.15.94:6812/3136219621,v1:172.21.15.94:6813/3136219621] exists,up 36e54a46-a0de-4a53-8bb4-6a49a457c91e 2024-02-17T11:57:42.125 INFO:teuthology.orchestra.run.smithi094.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.125:6808/256065927,v1:172.21.15.125:6809/256065927] [v2:172.21.15.125:6810/256065927,v1:172.21.15.125:6811/256065927] exists,up dbe91382-c694-4eb0-8e3a-1f9cdbd45356 2024-02-17T11:57:42.125 INFO:teuthology.orchestra.run.smithi094.stdout:osd.5 up in weight 1 up_from 38 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6808/1566223244,v1:172.21.15.189:6809/1566223244] [v2:172.21.15.189:6810/1566223244,v1:172.21.15.189:6811/1566223244] exists,up aa318c31-a0a3-4375-9980-5786a855c103 2024-02-17T11:57:42.125 INFO:teuthology.orchestra.run.smithi094.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6818/422477558,v1:172.21.15.94:6819/422477558] [v2:172.21.15.94:6820/422477558,v1:172.21.15.94:6821/422477558] exists,up 3fbef085-0c9b-4eec-9516-74e139980355 2024-02-17T11:57:42.125 INFO:teuthology.orchestra.run.smithi094.stdout:osd.7 up in weight 1 up_from 51 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.125:6816/2400671517,v1:172.21.15.125:6817/2400671517] [v2:172.21.15.125:6818/2400671517,v1:172.21.15.125:6819/2400671517] exists,up 3c76060b-420b-47f4-b746-75064957a855 2024-02-17T11:57:42.125 INFO:teuthology.orchestra.run.smithi094.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6816/3010520253,v1:172.21.15.189:6817/3010520253] [v2:172.21.15.189:6818/3010520253,v1:172.21.15.189:6819/3010520253] exists,up 9f6c6b96-ee0a-48c7-abfd-e73733791afc 2024-02-17T11:57:42.125 INFO:teuthology.orchestra.run.smithi094.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.94:6826/3841273455,v1:172.21.15.94:6827/3841273455] [v2:172.21.15.94:6828/3841273455,v1:172.21.15.94:6829/3841273455] exists,up bad201d9-439f-4236-bbf7-e736d0b12a87 2024-02-17T11:57:42.125 INFO:teuthology.orchestra.run.smithi094.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.125:6824/2398610115,v1:172.21.15.125:6825/2398610115] [v2:172.21.15.125:6826/2398610115,v1:172.21.15.125:6827/2398610115] exists,up 61683c46-c206-4556-b68d-d69457f6fd61 2024-02-17T11:57:42.125 INFO:teuthology.orchestra.run.smithi094.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.189:6824/656579186,v1:172.21.15.189:6825/656579186] [v2:172.21.15.189:6826/656579186,v1:172.21.15.189:6827/656579186] exists,up 52468b82-565a-4bc9-a784-43acd747d719 2024-02-17T11:57:42.125 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.a [2,11] 2024-02-17T11:57:42.125 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.e [2,1] 2024-02-17T11:57:42.125 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.1c [8,11] 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 2.1f [8,4] 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.6 [4,6] 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.c [3,9] 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.36 [4,9] 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.39 [5,8] 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:pg_upmap_items 3.3c [4,9] 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/1237669817 expires 2024-02-18T11:48:58.554916+0000 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6801/2384585889 expires 2024-02-18T11:48:58.554916+0000 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6801/281060900 expires 2024-02-18T11:48:31.902937+0000 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6800/2384585889 expires 2024-02-18T11:48:58.554916+0000 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/3511647110 expires 2024-02-18T11:48:58.554916+0000 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/1681584721 expires 2024-02-18T11:48:31.902937+0000 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/2416598774 expires 2024-02-18T11:48:58.554916+0000 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/2293278094 expires 2024-02-18T11:48:31.902937+0000 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:6800/281060900 expires 2024-02-18T11:48:31.902937+0000 2024-02-17T11:57:42.126 INFO:teuthology.orchestra.run.smithi094.stdout:blocklist 172.21.15.94:0/3895514831 expires 2024-02-18T11:48:31.902937+0000 2024-02-17T11:57:42.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.125+0000 7f0894a68700 1 -- 172.21.15.94:0/84169316 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08780a2150 msgr2=0x7f08780a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:42.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.125+0000 7f0894a68700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08780a2150 0x7f08780a4610 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f088000b5c0 tx=0x7f0880005fb0 comp rx=0 tx=0).stop 2024-02-17T11:57:42.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.125+0000 7f0894a68700 1 -- 172.21.15.94:0/84169316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0890100f00 msgr2=0x7f089019a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:42.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.125+0000 7f0894a68700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0890100f00 0x7f089019a320 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f088400c9e0 tx=0x7f088400ccf0 comp rx=0 tx=0).stop 2024-02-17T11:57:42.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.126+0000 7f0894a68700 1 -- 172.21.15.94:0/84169316 shutdown_connections 2024-02-17T11:57:42.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.126+0000 7f0894a68700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08780a2150 0x7f08780a4610 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:42.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.126+0000 7f0894a68700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08901018c0 0x7f08901a13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:42.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.126+0000 7f0894a68700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08901005a0 0x7f0890199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:42.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.126+0000 7f0894a68700 1 --2- 172.21.15.94:0/84169316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0890100f00 0x7f089019a320 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:42.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.126+0000 7f0894a68700 1 -- 172.21.15.94:0/84169316 >> 172.21.15.94:0/84169316 conn(0x7f08900fc160 msgr2=0x7f08900fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:42.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.126+0000 7f0894a68700 1 -- 172.21.15.94:0/84169316 shutdown_connections 2024-02-17T11:57:42.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.126+0000 7f0894a68700 1 -- 172.21.15.94:0/84169316 wait complete. 2024-02-17T11:57:42.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:41 smithi189 ceph-mon[36588]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-17T11:57:42.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:41 smithi125 ceph-mon[36819]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-02-17T11:57:42.836 INFO:teuthology.run_tasks:Running task fs.ready... 2024-02-17T11:57:42.848 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:57:42.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.952+0000 7f2abc102640 1 -- 172.21.15.94:0/817413808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ab4104610 msgr2=0x7f2ab4106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:42.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.952+0000 7f2abc102640 1 --2- 172.21.15.94:0/817413808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ab4104610 0x7f2ab4106a00 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009900 tx=0x7f2aa402bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:42.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.952+0000 7f2abc102640 1 -- 172.21.15.94:0/817413808 shutdown_connections 2024-02-17T11:57:42.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.952+0000 7f2abc102640 1 --2- 172.21.15.94:0/817413808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ab4106f40 0x7f2ab4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:42.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.952+0000 7f2abc102640 1 --2- 172.21.15.94:0/817413808 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ab4104610 0x7f2ab4106a00 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:42.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.952+0000 7f2abc102640 1 --2- 172.21.15.94:0/817413808 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4101ce0 0x7f2ab41040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:42.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.952+0000 7f2abc102640 1 -- 172.21.15.94:0/817413808 >> 172.21.15.94:0/817413808 conn(0x7f2ab40fb630 msgr2=0x7f2ab40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:42.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.952+0000 7f2abc102640 1 -- 172.21.15.94:0/817413808 shutdown_connections 2024-02-17T11:57:42.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.952+0000 7f2abc102640 1 -- 172.21.15.94:0/817413808 wait complete. 2024-02-17T11:57:42.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.953+0000 7f2abc102640 1 Processor -- start 2024-02-17T11:57:42.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.953+0000 7f2abc102640 1 -- start start 2024-02-17T11:57:42.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.953+0000 7f2abc102640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ab4101ce0 0x7f2ab4101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.953+0000 7f2abc102640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ab4104610 0x7f2ab40ffc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.953+0000 7f2abc102640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4106f40 0x7f2ab41001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.953+0000 7f2abc102640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab4100930 con 0x7f2ab4101ce0 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.953+0000 7f2abc102640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab4100aa0 con 0x7f2ab4106f40 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.953+0000 7f2abc102640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab41a12f0 con 0x7f2ab4104610 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.953+0000 7f2ab9e9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ab4101ce0 0x7f2ab4101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.953+0000 7f2ab9e9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ab4101ce0 0x7f2ab4101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48306/0 (socket says 172.21.15.94:48306) 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2ab9e9e640 1 -- 172.21.15.94:0/390916160 learned_addr learned my addr 172.21.15.94:0/390916160 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2ab969d640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ab4104610 0x7f2ab40ffc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2ab9e9e640 1 -- 172.21.15.94:0/390916160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ab4104610 msgr2=0x7f2ab40ffc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2aba69f640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4106f40 0x7f2ab41001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2ab9e9e640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ab4104610 0x7f2ab40ffc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2ab9e9e640 1 -- 172.21.15.94:0/390916160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4106f40 msgr2=0x7f2ab41001d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:42.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2ab9e9e640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4106f40 0x7f2ab41001d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:42.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2ab9e9e640 1 -- 172.21.15.94:0/390916160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa4009560 con 0x7f2ab4101ce0 2024-02-17T11:57:42.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2aba69f640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4106f40 0x7f2ab41001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:42.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2ab9e9e640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ab4101ce0 0x7f2ab4101650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000ea20 tx=0x7f2ab000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:42.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2aaaffd640 1 -- 172.21.15.94:0/390916160 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0009700 con 0x7f2ab4101ce0 2024-02-17T11:57:42.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2abc102640 1 -- 172.21.15.94:0/390916160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab41a15a0 con 0x7f2ab4101ce0 2024-02-17T11:57:42.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2aaaffd640 1 -- 172.21.15.94:0/390916160 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ab000cd80 con 0x7f2ab4101ce0 2024-02-17T11:57:42.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2aaaffd640 1 -- 172.21.15.94:0/390916160 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0018840 con 0x7f2ab4101ce0 2024-02-17T11:57:42.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.954+0000 7f2abc102640 1 -- 172.21.15.94:0/390916160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab41a1ad0 con 0x7f2ab4101ce0 2024-02-17T11:57:42.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.955+0000 7f2abc102640 1 -- 172.21.15.94:0/390916160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab40fd120 con 0x7f2ab4101ce0 2024-02-17T11:57:42.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.957+0000 7f2aaaffd640 1 -- 172.21.15.94:0/390916160 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2ab0010a80 con 0x7f2ab4101ce0 2024-02-17T11:57:42.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.957+0000 7f2aaaffd640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2aa00a20c0 0x7f2aa00a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:42.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.957+0000 7f2ab969d640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2aa00a20c0 0x7f2aa00a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:42.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.958+0000 7f2aaaffd640 1 -- 172.21.15.94:0/390916160 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2ab0014070 con 0x7f2ab4101ce0 2024-02-17T11:57:42.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.958+0000 7f2ab969d640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2aa00a20c0 0x7f2aa00a4580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4006010 tx=0x7f2aa4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:42.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:42.961+0000 7f2aaaffd640 1 -- 172.21.15.94:0/390916160 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2ab0097520 con 0x7f2ab4101ce0 2024-02-17T11:57:43.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.134+0000 7f2abc102640 1 -- 172.21.15.94:0/390916160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ab40fcb90 con 0x7f2ab4101ce0 2024-02-17T11:57:43.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.136+0000 7f2aaaffd640 1 -- 172.21.15.94:0/390916160 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22686 (secure 0 0 0) 0x7f2ab0026090 con 0x7f2ab4101ce0 2024-02-17T11:57:43.135 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:43.135 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":87,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:56:59.725646+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:57:43.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.138+0000 7f2abc102640 1 -- 172.21.15.94:0/390916160 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2aa00a20c0 msgr2=0x7f2aa00a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.138+0000 7f2abc102640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2aa00a20c0 0x7f2aa00a4580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4006010 tx=0x7f2aa4000bc0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.138+0000 7f2abc102640 1 -- 172.21.15.94:0/390916160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ab4101ce0 msgr2=0x7f2ab4101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.138+0000 7f2abc102640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ab4101ce0 0x7f2ab4101650 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000ea20 tx=0x7f2ab000eee0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.138+0000 7f2abc102640 1 -- 172.21.15.94:0/390916160 shutdown_connections 2024-02-17T11:57:43.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.138+0000 7f2abc102640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2aa00a20c0 0x7f2aa00a4580 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.138+0000 7f2abc102640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2ab4106f40 0x7f2ab41001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.138+0000 7f2abc102640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ab4104610 0x7f2ab40ffc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.138+0000 7f2abc102640 1 --2- 172.21.15.94:0/390916160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2ab4101ce0 0x7f2ab4101650 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.138+0000 7f2abc102640 1 -- 172.21.15.94:0/390916160 >> 172.21.15.94:0/390916160 conn(0x7f2ab40fb630 msgr2=0x7f2ab4102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:43.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.138+0000 7f2abc102640 1 -- 172.21.15.94:0/390916160 shutdown_connections 2024-02-17T11:57:43.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.138+0000 7f2abc102640 1 -- 172.21.15.94:0/390916160 wait complete. 2024-02-17T11:57:43.147 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:42 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/84169316' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-17T11:57:43.149 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T11:51:20.894601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T11:55:56.349040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T11:55:57.234916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T11:57:43.149 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T11:57:43.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:42 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/84169316' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-17T11:57:43.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:42 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/84169316' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-17T11:57:43.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.254+0000 7f267409a640 1 -- 172.21.15.94:0/4151659141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f266c1027b0 msgr2=0x7f266c102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.254+0000 7f267409a640 1 --2- 172.21.15.94:0/4151659141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f266c1027b0 0x7f266c102bd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2668009900 tx=0x7f266802bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:43.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.254+0000 7f267409a640 1 -- 172.21.15.94:0/4151659141 shutdown_connections 2024-02-17T11:57:43.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.254+0000 7f267409a640 1 --2- 172.21.15.94:0/4151659141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f266c0ffbe0 0x7f266c109790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.254+0000 7f267409a640 1 --2- 172.21.15.94:0/4151659141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f266c0ff220 0x7f266c0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.254+0000 7f267409a640 1 --2- 172.21.15.94:0/4151659141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f266c1027b0 0x7f266c102bd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.254+0000 7f267409a640 1 -- 172.21.15.94:0/4151659141 >> 172.21.15.94:0/4151659141 conn(0x7f266c0fad80 msgr2=0x7f266c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.254+0000 7f267409a640 1 -- 172.21.15.94:0/4151659141 shutdown_connections 2024-02-17T11:57:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.254+0000 7f267409a640 1 -- 172.21.15.94:0/4151659141 wait complete. 2024-02-17T11:57:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f267409a640 1 Processor -- start 2024-02-17T11:57:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f267409a640 1 -- start start 2024-02-17T11:57:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f267409a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f266c0ff220 0x7f266c101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f267409a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f266c0ffbe0 0x7f266c1004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f267409a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f266c1027b0 0x7f266c100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f267409a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c101190 con 0x7f266c1027b0 2024-02-17T11:57:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f267409a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c101300 con 0x7f266c0ff220 2024-02-17T11:57:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f267409a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c1a1320 con 0x7f266c0ffbe0 2024-02-17T11:57:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f2672637640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f266c1027b0 0x7f266c100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:43.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f2672637640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f266c1027b0 0x7f266c100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48316/0 (socket says 172.21.15.94:48316) 2024-02-17T11:57:43.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f2672637640 1 -- 172.21.15.94:0/3939381168 learned_addr learned my addr 172.21.15.94:0/3939381168 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:43.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f2671e36640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f266c0ff220 0x7f266c101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:43.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f2672637640 1 -- 172.21.15.94:0/3939381168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f266c0ffbe0 msgr2=0x7f266c1004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f2672637640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f266c0ffbe0 0x7f266c1004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f2672637640 1 -- 172.21.15.94:0/3939381168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f266c0ff220 msgr2=0x7f266c101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f2672637640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f266c0ff220 0x7f266c101eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f2672637640 1 -- 172.21.15.94:0/3939381168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2668009560 con 0x7f266c1027b0 2024-02-17T11:57:43.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.255+0000 7f2671e36640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f266c0ff220 0x7f266c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:43.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.256+0000 7f2672637640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f266c1027b0 0x7f266c100a30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f266400db20 tx=0x7f266400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:43.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.256+0000 7f2662ffd640 1 -- 172.21.15.94:0/3939381168 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266400bea0 con 0x7f266c1027b0 2024-02-17T11:57:43.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.256+0000 7f2662ffd640 1 -- 172.21.15.94:0/3939381168 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2664009940 con 0x7f266c1027b0 2024-02-17T11:57:43.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.256+0000 7f267409a640 1 -- 172.21.15.94:0/3939381168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266c1a15d0 con 0x7f266c1027b0 2024-02-17T11:57:43.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.256+0000 7f2662ffd640 1 -- 172.21.15.94:0/3939381168 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26640106c0 con 0x7f266c1027b0 2024-02-17T11:57:43.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.256+0000 7f267409a640 1 -- 172.21.15.94:0/3939381168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266c1a1ae0 con 0x7f266c1027b0 2024-02-17T11:57:43.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.257+0000 7f267409a640 1 -- 172.21.15.94:0/3939381168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266c108740 con 0x7f266c1027b0 2024-02-17T11:57:43.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.258+0000 7f2662ffd640 1 -- 172.21.15.94:0/3939381168 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2664004240 con 0x7f266c1027b0 2024-02-17T11:57:43.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.258+0000 7f2662ffd640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f26580a2140 0x7f26580a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:43.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.258+0000 7f2671e36640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f26580a2140 0x7f26580a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:43.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.258+0000 7f2662ffd640 1 -- 172.21.15.94:0/3939381168 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f26640cd9a0 con 0x7f266c1027b0 2024-02-17T11:57:43.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.259+0000 7f2671e36640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f26580a2140 0x7f26580a4600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2668036040 tx=0x7f2668000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:43.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.262+0000 7f2662ffd640 1 -- 172.21.15.94:0/3939381168 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f26640969f0 con 0x7f266c1027b0 2024-02-17T11:57:43.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.435+0000 7f267409a640 1 -- 172.21.15.94:0/3939381168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f266c10ce60 con 0x7f266c1027b0 2024-02-17T11:57:43.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.435+0000 7f2662ffd640 1 -- 172.21.15.94:0/3939381168 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2664017020 con 0x7f266c1027b0 2024-02-17T11:57:43.434 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-02-17T11:57:43.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.437+0000 7f267409a640 1 -- 172.21.15.94:0/3939381168 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f26580a2140 msgr2=0x7f26580a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.437+0000 7f267409a640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f26580a2140 0x7f26580a4600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2668036040 tx=0x7f2668000bc0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.437+0000 7f267409a640 1 -- 172.21.15.94:0/3939381168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f266c1027b0 msgr2=0x7f266c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.437+0000 7f267409a640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f266c1027b0 0x7f266c100a30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f266400db20 tx=0x7f266400b260 comp rx=0 tx=0).stop 2024-02-17T11:57:43.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.437+0000 7f267409a640 1 -- 172.21.15.94:0/3939381168 shutdown_connections 2024-02-17T11:57:43.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.437+0000 7f267409a640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f26580a2140 0x7f26580a4600 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.437+0000 7f267409a640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f266c1027b0 0x7f266c100a30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.437+0000 7f267409a640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f266c0ffbe0 0x7f266c1004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.437+0000 7f267409a640 1 --2- 172.21.15.94:0/3939381168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f266c0ff220 0x7f266c101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.437+0000 7f267409a640 1 -- 172.21.15.94:0/3939381168 >> 172.21.15.94:0/3939381168 conn(0x7f266c0fad80 msgr2=0x7f266c105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:43.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.437+0000 7f267409a640 1 -- 172.21.15.94:0/3939381168 shutdown_connections 2024-02-17T11:57:43.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.437+0000 7f267409a640 1 -- 172.21.15.94:0/3939381168 wait complete. 2024-02-17T11:57:43.447 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T11:57:43.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.554+0000 7f2bcebca640 1 -- 172.21.15.94:0/2770583154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc8101190 msgr2=0x7f2bc81015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.554+0000 7f2bcebca640 1 --2- 172.21.15.94:0/2770583154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc8101190 0x7f2bc81015b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009900 tx=0x7f2bbc02bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:43.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.554+0000 7f2bcebca640 1 -- 172.21.15.94:0/2770583154 shutdown_connections 2024-02-17T11:57:43.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.554+0000 7f2bcebca640 1 --2- 172.21.15.94:0/2770583154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bc8102d40 0x7f2bc81095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.554+0000 7f2bcebca640 1 --2- 172.21.15.94:0/2770583154 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bc8102380 0x7f2bc8102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.554+0000 7f2bcebca640 1 --2- 172.21.15.94:0/2770583154 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc8101190 0x7f2bc81015b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.554+0000 7f2bcebca640 1 -- 172.21.15.94:0/2770583154 >> 172.21.15.94:0/2770583154 conn(0x7f2bc80fc720 msgr2=0x7f2bc80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:43.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.554+0000 7f2bcebca640 1 -- 172.21.15.94:0/2770583154 shutdown_connections 2024-02-17T11:57:43.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.554+0000 7f2bcebca640 1 -- 172.21.15.94:0/2770583154 wait complete. 2024-02-17T11:57:43.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.554+0000 7f2bcebca640 1 Processor -- start 2024-02-17T11:57:43.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.554+0000 7f2bcebca640 1 -- start start 2024-02-17T11:57:43.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcebca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc8101190 0x7f2bc8197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:43.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcebca640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bc8102380 0x7f2bc81977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:43.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcebca640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bc8102d40 0x7f2bc819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:43.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcebca640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc819ed80 con 0x7f2bc8101190 2024-02-17T11:57:43.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcebca640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc819eef0 con 0x7f2bc8102380 2024-02-17T11:57:43.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcebca640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc819f0b0 con 0x7f2bc8102d40 2024-02-17T11:57:43.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcc966640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc8101190 0x7f2bc8197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:43.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcc966640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc8101190 0x7f2bc8197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48338/0 (socket says 172.21.15.94:48338) 2024-02-17T11:57:43.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcc966640 1 -- 172.21.15.94:0/3702004282 learned_addr learned my addr 172.21.15.94:0/3702004282 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:43.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bc7fff640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bc8102380 0x7f2bc81977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:43.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcd167640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bc8102d40 0x7f2bc819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:43.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcc966640 1 -- 172.21.15.94:0/3702004282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bc8102d40 msgr2=0x7f2bc819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcc966640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bc8102d40 0x7f2bc819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcc966640 1 -- 172.21.15.94:0/3702004282 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bc8102380 msgr2=0x7f2bc81977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcc966640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bc8102380 0x7f2bc81977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcc966640 1 -- 172.21.15.94:0/3702004282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc009560 con 0x7f2bc8101190 2024-02-17T11:57:43.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcc966640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc8101190 0x7f2bc8197290 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc005ab0 tx=0x7f2bbc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:43.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bc5ffb640 1 -- 172.21.15.94:0/3702004282 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc004aa0 con 0x7f2bc8101190 2024-02-17T11:57:43.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bc5ffb640 1 -- 172.21.15.94:0/3702004282 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bbc0052d0 con 0x7f2bc8101190 2024-02-17T11:57:43.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bcebca640 1 -- 172.21.15.94:0/3702004282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc819f330 con 0x7f2bc8101190 2024-02-17T11:57:43.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.555+0000 7f2bc5ffb640 1 -- 172.21.15.94:0/3702004282 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc003e80 con 0x7f2bc8101190 2024-02-17T11:57:43.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.556+0000 7f2bcebca640 1 -- 172.21.15.94:0/3702004282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc819f840 con 0x7f2bc8101190 2024-02-17T11:57:43.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.556+0000 7f2bcebca640 1 -- 172.21.15.94:0/3702004282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc8108500 con 0x7f2bc8101190 2024-02-17T11:57:43.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.557+0000 7f2bc5ffb640 1 -- 172.21.15.94:0/3702004282 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f2bbc005440 con 0x7f2bc8101190 2024-02-17T11:57:43.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.558+0000 7f2bc5ffb640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2bb80a20c0 0x7f2bb80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:43.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.558+0000 7f2bc7fff640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2bb80a20c0 0x7f2bb80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:43.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.558+0000 7f2bc5ffb640 1 -- 172.21.15.94:0/3702004282 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f2bbc0f0150 con 0x7f2bc8101190 2024-02-17T11:57:43.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.558+0000 7f2bc7fff640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2bb80a20c0 0x7f2bb80a4580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2bc81987a0 tx=0x7f2bb4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:43.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.562+0000 7f2bc5ffb640 1 -- 172.21.15.94:0/3702004282 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2bbc0b9120 con 0x7f2bc8101190 2024-02-17T11:57:43.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.736+0000 7f2bcebca640 1 -- 172.21.15.94:0/3702004282 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2bc802ce40 con 0x7f2bc8101190 2024-02-17T11:57:43.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.737+0000 7f2bc5ffb640 1 -- 172.21.15.94:0/3702004282 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2bbc0bc7e0 con 0x7f2bc8101190 2024-02-17T11:57:43.737 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-02-17T11:57:43.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.739+0000 7f2bcebca640 1 -- 172.21.15.94:0/3702004282 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2bb80a20c0 msgr2=0x7f2bb80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.739+0000 7f2bcebca640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2bb80a20c0 0x7f2bb80a4580 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2bc81987a0 tx=0x7f2bb4005f90 comp rx=0 tx=0).stop 2024-02-17T11:57:43.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.739+0000 7f2bcebca640 1 -- 172.21.15.94:0/3702004282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc8101190 msgr2=0x7f2bc8197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.739+0000 7f2bcebca640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc8101190 0x7f2bc8197290 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc005ab0 tx=0x7f2bbc004270 comp rx=0 tx=0).stop 2024-02-17T11:57:43.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.739+0000 7f2bcebca640 1 -- 172.21.15.94:0/3702004282 shutdown_connections 2024-02-17T11:57:43.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.739+0000 7f2bcebca640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f2bb80a20c0 0x7f2bb80a4580 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.739+0000 7f2bcebca640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bc8102d40 0x7f2bc819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.739+0000 7f2bcebca640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bc8102380 0x7f2bc81977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.739+0000 7f2bcebca640 1 --2- 172.21.15.94:0/3702004282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bc8101190 0x7f2bc8197290 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.739+0000 7f2bcebca640 1 -- 172.21.15.94:0/3702004282 >> 172.21.15.94:0/3702004282 conn(0x7f2bc80fc720 msgr2=0x7f2bc81056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:43.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.739+0000 7f2bcebca640 1 -- 172.21.15.94:0/3702004282 shutdown_connections 2024-02-17T11:57:43.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.739+0000 7f2bcebca640 1 -- 172.21.15.94:0/3702004282 wait complete. 2024-02-17T11:57:43.752 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T11:57:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.862+0000 7f4ad080f640 1 -- 172.21.15.94:0/131191631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc106e60 msgr2=0x7f4acc1092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.862+0000 7f4ad080f640 1 --2- 172.21.15.94:0/131191631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc106e60 0x7f4acc1092b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009900 tx=0x7f4ac402bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 -- 172.21.15.94:0/131191631 shutdown_connections 2024-02-17T11:57:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 --2- 172.21.15.94:0/131191631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc106e60 0x7f4acc1092b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 --2- 172.21.15.94:0/131191631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4acc104530 0x7f4acc106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 --2- 172.21.15.94:0/131191631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4acc101c00 0x7f4acc103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 -- 172.21.15.94:0/131191631 >> 172.21.15.94:0/131191631 conn(0x7f4acc0fb610 msgr2=0x7f4acc0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 -- 172.21.15.94:0/131191631 shutdown_connections 2024-02-17T11:57:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 -- 172.21.15.94:0/131191631 wait complete. 2024-02-17T11:57:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 Processor -- start 2024-02-17T11:57:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 -- start start 2024-02-17T11:57:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc101c00 0x7f4acc197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4acc104530 0x7f4acc197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4acc106e60 0x7f4acc19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc19ed00 con 0x7f4acc101c00 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc19ee70 con 0x7f4acc106e60 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.863+0000 7f4ad080f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4acc19f030 con 0x7f4acc104530 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4aca59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc101c00 0x7f4acc197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4aca59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc101c00 0x7f4acc197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48350/0 (socket says 172.21.15.94:48350) 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4aca59c640 1 -- 172.21.15.94:0/214396749 learned_addr learned my addr 172.21.15.94:0/214396749 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4ac9d9b640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4acc104530 0x7f4acc197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4aca59c640 1 -- 172.21.15.94:0/214396749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4acc104530 msgr2=0x7f4acc197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4aca59c640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4acc104530 0x7f4acc197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4acad9d640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4acc106e60 0x7f4acc19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4aca59c640 1 -- 172.21.15.94:0/214396749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4acc106e60 msgr2=0x7f4acc19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4aca59c640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4acc106e60 0x7f4acc19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4aca59c640 1 -- 172.21.15.94:0/214396749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac4009560 con 0x7f4acc101c00 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4acad9d640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4acc106e60 0x7f4acc19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4aca59c640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc101c00 0x7f4acc197210 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00ea80 tx=0x7f4abc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4abb7fe640 1 -- 172.21.15.94:0/214396749 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc00ce40 con 0x7f4acc101c00 2024-02-17T11:57:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4abb7fe640 1 -- 172.21.15.94:0/214396749 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4abc005020 con 0x7f4acc101c00 2024-02-17T11:57:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4ad080f640 1 -- 172.21.15.94:0/214396749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4acc19f310 con 0x7f4acc101c00 2024-02-17T11:57:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4abb7fe640 1 -- 172.21.15.94:0/214396749 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0116d0 con 0x7f4acc101c00 2024-02-17T11:57:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.864+0000 7f4ad080f640 1 -- 172.21.15.94:0/214396749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4acc19f7d0 con 0x7f4acc101c00 2024-02-17T11:57:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.865+0000 7f4ad080f640 1 -- 172.21.15.94:0/214396749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4acc0fd100 con 0x7f4acc101c00 2024-02-17T11:57:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.866+0000 7f4abb7fe640 1 -- 172.21.15.94:0/214396749 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f4abc005190 con 0x7f4acc101c00 2024-02-17T11:57:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.867+0000 7f4abb7fe640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4ab40a20c0 0x7f4ab40a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.867+0000 7f4ac9d9b640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4ab40a20c0 0x7f4ab40a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.867+0000 7f4abb7fe640 1 -- 172.21.15.94:0/214396749 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4abc0cdff0 con 0x7f4acc101c00 2024-02-17T11:57:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.867+0000 7f4ac9d9b640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4ab40a20c0 0x7f4ab40a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0005fd0 tx=0x7f4ac0009470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:43.871+0000 7f4abb7fe640 1 -- 172.21.15.94:0/214396749 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4abc101d20 con 0x7f4acc101c00 2024-02-17T11:57:44.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.044+0000 7f4ad080f640 1 -- 172.21.15.94:0/214396749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4acc10cb90 con 0x7f4acc101c00 2024-02-17T11:57:44.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.045+0000 7f4abb7fe640 1 -- 172.21.15.94:0/214396749 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4abc01f020 con 0x7f4acc101c00 2024-02-17T11:57:44.043 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-02-17T11:57:44.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.046+0000 7f4ad080f640 1 -- 172.21.15.94:0/214396749 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4ab40a20c0 msgr2=0x7f4ab40a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.046+0000 7f4ad080f640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4ab40a20c0 0x7f4ab40a4580 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0005fd0 tx=0x7f4ac0009470 comp rx=0 tx=0).stop 2024-02-17T11:57:44.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.046+0000 7f4ad080f640 1 -- 172.21.15.94:0/214396749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc101c00 msgr2=0x7f4acc197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.046+0000 7f4ad080f640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc101c00 0x7f4acc197210 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00ea80 tx=0x7f4abc00ef40 comp rx=0 tx=0).stop 2024-02-17T11:57:44.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.046+0000 7f4ad080f640 1 -- 172.21.15.94:0/214396749 shutdown_connections 2024-02-17T11:57:44.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.046+0000 7f4ad080f640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4ab40a20c0 0x7f4ab40a4580 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.046+0000 7f4ad080f640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4acc106e60 0x7f4acc19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.046+0000 7f4ad080f640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4acc104530 0x7f4acc197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.046+0000 7f4ad080f640 1 --2- 172.21.15.94:0/214396749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4acc101c00 0x7f4acc197210 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.047+0000 7f4ad080f640 1 -- 172.21.15.94:0/214396749 >> 172.21.15.94:0/214396749 conn(0x7f4acc0fb610 msgr2=0x7f4acc102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:44.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.047+0000 7f4ad080f640 1 -- 172.21.15.94:0/214396749 shutdown_connections 2024-02-17T11:57:44.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.047+0000 7f4ad080f640 1 -- 172.21.15.94:0/214396749 wait complete. 2024-02-17T11:57:44.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:43 smithi094 ceph-mon[32499]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-17T11:57:44.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:43 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/390916160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:44.055 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:43 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3939381168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:44.056 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:43 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3702004282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:44.057 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:57:44.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:43 smithi189 ceph-mon[36588]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-17T11:57:44.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:43 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/390916160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:44.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:43 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3939381168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:44.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:43 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3702004282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 -- 172.21.15.94:0/840317423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc1000a0 msgr2=0x7f94fc1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 --2- 172.21.15.94:0/840317423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc1000a0 0x7f94fc1004c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009900 tx=0x7f94ec02bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 -- 172.21.15.94:0/840317423 shutdown_connections 2024-02-17T11:57:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 --2- 172.21.15.94:0/840317423 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94fc101c50 0x7f94fc1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 --2- 172.21.15.94:0/840317423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94fc101290 0x7f94fc101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 --2- 172.21.15.94:0/840317423 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc1000a0 0x7f94fc1004c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 -- 172.21.15.94:0/840317423 >> 172.21.15.94:0/840317423 conn(0x7f94fc0fb610 msgr2=0x7f94fc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 -- 172.21.15.94:0/840317423 shutdown_connections 2024-02-17T11:57:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 -- 172.21.15.94:0/840317423 wait complete. 2024-02-17T11:57:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 Processor -- start 2024-02-17T11:57:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 -- start start 2024-02-17T11:57:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc1000a0 0x7f94fc068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94fc101290 0x7f94fc19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94fc101c50 0x7f94fc19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.160+0000 7f9503579640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc19ee10 con 0x7f94fc1000a0 2024-02-17T11:57:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9503579640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc19ef80 con 0x7f94fc101c50 2024-02-17T11:57:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9503579640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc19f140 con 0x7f94fc101290 2024-02-17T11:57:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9501315640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc1000a0 0x7f94fc068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9501315640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc1000a0 0x7f94fc068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48354/0 (socket says 172.21.15.94:48354) 2024-02-17T11:57:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9501315640 1 -- 172.21.15.94:0/803134018 learned_addr learned my addr 172.21.15.94:0/803134018 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9500b14640 1 --2- 172.21.15.94:0/803134018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94fc101290 0x7f94fc19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9501315640 1 -- 172.21.15.94:0/803134018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94fc101290 msgr2=0x7f94fc19c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9501315640 1 --2- 172.21.15.94:0/803134018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94fc101290 0x7f94fc19c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9501315640 1 -- 172.21.15.94:0/803134018 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94fc101c50 msgr2=0x7f94fc19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9501315640 1 --2- 172.21.15.94:0/803134018 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94fc101c50 0x7f94fc19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9501315640 1 -- 172.21.15.94:0/803134018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec009560 con 0x7f94fc1000a0 2024-02-17T11:57:44.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9501315640 1 --2- 172.21.15.94:0/803134018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc1000a0 0x7f94fc068e50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f94ec000c00 tx=0x7f94ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:44.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f94f27fc640 1 -- 172.21.15.94:0/803134018 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec004df0 con 0x7f94fc1000a0 2024-02-17T11:57:44.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f94f27fc640 1 -- 172.21.15.94:0/803134018 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f94ec005430 con 0x7f94fc1000a0 2024-02-17T11:57:44.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9503579640 1 -- 172.21.15.94:0/803134018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94fc19f360 con 0x7f94fc1000a0 2024-02-17T11:57:44.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f94f27fc640 1 -- 172.21.15.94:0/803134018 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec03e450 con 0x7f94fc1000a0 2024-02-17T11:57:44.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.161+0000 7f9503579640 1 -- 172.21.15.94:0/803134018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc19f8f0 con 0x7f94fc1000a0 2024-02-17T11:57:44.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.162+0000 7f9503579640 1 -- 172.21.15.94:0/803134018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94fc107440 con 0x7f94fc1000a0 2024-02-17T11:57:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.163+0000 7f94f27fc640 1 -- 172.21.15.94:0/803134018 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f94ec003680 con 0x7f94fc1000a0 2024-02-17T11:57:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.164+0000 7f94f27fc640 1 --2- 172.21.15.94:0/803134018 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f94e80a20c0 0x7f94e80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.164+0000 7f9500b14640 1 --2- 172.21.15.94:0/803134018 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f94e80a20c0 0x7f94e80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.164+0000 7f94f27fc640 1 -- 172.21.15.94:0/803134018 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f94ec0f0840 con 0x7f94fc1000a0 2024-02-17T11:57:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.164+0000 7f9500b14640 1 --2- 172.21.15.94:0/803134018 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f94e80a20c0 0x7f94e80a4580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005fd0 tx=0x7f94f8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:44.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.167+0000 7f94f27fc640 1 -- 172.21.15.94:0/803134018 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f94ec0b9810 con 0x7f94fc1000a0 2024-02-17T11:57:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:43 smithi125 ceph-mon[36819]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 6 op/s 2024-02-17T11:57:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:43 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/390916160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:43 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3939381168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:43 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3702004282' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:44.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.378+0000 7f9503579640 1 -- 172.21.15.94:0/803134018 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94fc10bb10 con 0x7f94fc1000a0 2024-02-17T11:57:44.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.379+0000 7f94f27fc640 1 -- 172.21.15.94:0/803134018 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f94ec0bced0 con 0x7f94fc1000a0 2024-02-17T11:57:44.378 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:44.378 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:56:02.151161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T11:57:44.378 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 20 2024-02-17T11:57:44.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.381+0000 7f9503579640 1 -- 172.21.15.94:0/803134018 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f94e80a20c0 msgr2=0x7f94e80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.381+0000 7f9503579640 1 --2- 172.21.15.94:0/803134018 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f94e80a20c0 0x7f94e80a4580 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94f8005fd0 tx=0x7f94f8005850 comp rx=0 tx=0).stop 2024-02-17T11:57:44.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.381+0000 7f9503579640 1 -- 172.21.15.94:0/803134018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc1000a0 msgr2=0x7f94fc068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.381+0000 7f9503579640 1 --2- 172.21.15.94:0/803134018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc1000a0 0x7f94fc068e50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f94ec000c00 tx=0x7f94ec004270 comp rx=0 tx=0).stop 2024-02-17T11:57:44.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.381+0000 7f9503579640 1 -- 172.21.15.94:0/803134018 shutdown_connections 2024-02-17T11:57:44.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.381+0000 7f9503579640 1 --2- 172.21.15.94:0/803134018 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f94e80a20c0 0x7f94e80a4580 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.381+0000 7f9503579640 1 --2- 172.21.15.94:0/803134018 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f94fc101c50 0x7f94fc19e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.381+0000 7f9503579640 1 --2- 172.21.15.94:0/803134018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94fc101290 0x7f94fc19c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.381+0000 7f9503579640 1 --2- 172.21.15.94:0/803134018 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94fc1000a0 0x7f94fc068e50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.381+0000 7f9503579640 1 -- 172.21.15.94:0/803134018 >> 172.21.15.94:0/803134018 conn(0x7f94fc0fb610 msgr2=0x7f94fc104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:44.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.381+0000 7f9503579640 1 -- 172.21.15.94:0/803134018 shutdown_connections 2024-02-17T11:57:44.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.381+0000 7f9503579640 1 -- 172.21.15.94:0/803134018 wait complete. 2024-02-17T11:57:44.391 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:57:44.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.494+0000 7fb954c9d640 1 -- 172.21.15.94:0/1941185912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb950101c70 msgr2=0x7fb950108500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.494+0000 7fb954c9d640 1 --2- 172.21.15.94:0/1941185912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb950101c70 0x7fb950108500 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb948009ca0 tx=0x7fb94802c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.494+0000 7fb954c9d640 1 -- 172.21.15.94:0/1941185912 shutdown_connections 2024-02-17T11:57:44.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.494+0000 7fb954c9d640 1 --2- 172.21.15.94:0/1941185912 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb950101c70 0x7fb950108500 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.494+0000 7fb954c9d640 1 --2- 172.21.15.94:0/1941185912 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9501012b0 0x7fb950101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.494+0000 7fb954c9d640 1 --2- 172.21.15.94:0/1941185912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9501000c0 0x7fb9501004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.494+0000 7fb954c9d640 1 -- 172.21.15.94:0/1941185912 >> 172.21.15.94:0/1941185912 conn(0x7fb9500fb630 msgr2=0x7fb9500fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:44.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.494+0000 7fb954c9d640 1 -- 172.21.15.94:0/1941185912 shutdown_connections 2024-02-17T11:57:44.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.494+0000 7fb954c9d640 1 -- 172.21.15.94:0/1941185912 wait complete. 2024-02-17T11:57:44.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb954c9d640 1 Processor -- start 2024-02-17T11:57:44.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb954c9d640 1 -- start start 2024-02-17T11:57:44.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb954c9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9501000c0 0x7fb95006a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:44.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb954c9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9501012b0 0x7fb95019beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:44.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb954c9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950101c70 0x7fb95019e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:44.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb954c9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95019ec80 con 0x7fb9501000c0 2024-02-17T11:57:44.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb954c9d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95019edf0 con 0x7fb9501012b0 2024-02-17T11:57:44.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb954c9d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95019efb0 con 0x7fb950101c70 2024-02-17T11:57:44.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb94e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9501000c0 0x7fb95006a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:44.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb94e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9501000c0 0x7fb95006a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57622/0 (socket says 172.21.15.94:57622) 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb94e59c640 1 -- 172.21.15.94:0/2074488049 learned_addr learned my addr 172.21.15.94:0/2074488049 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb94ed9d640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950101c70 0x7fb95019e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb94e59c640 1 -- 172.21.15.94:0/2074488049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950101c70 msgr2=0x7fb95019e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb94e59c640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950101c70 0x7fb95019e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb94e59c640 1 -- 172.21.15.94:0/2074488049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9501012b0 msgr2=0x7fb95019beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb945d9b640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9501012b0 0x7fb95019beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb94e59c640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9501012b0 0x7fb95019beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb94e59c640 1 -- 172.21.15.94:0/2074488049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb948009900 con 0x7fb9501000c0 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb94ed9d640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950101c70 0x7fb95019e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.495+0000 7fb94e59c640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9501000c0 0x7fb95006a340 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb94000d950 tx=0x7fb94000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.496+0000 7fb947fff640 1 -- 172.21.15.94:0/2074488049 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940009c30 con 0x7fb9501000c0 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.496+0000 7fb947fff640 1 -- 172.21.15.94:0/2074488049 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb940009d90 con 0x7fb9501000c0 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.496+0000 7fb954c9d640 1 -- 172.21.15.94:0/2074488049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95019f290 con 0x7fb9501000c0 2024-02-17T11:57:44.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.496+0000 7fb947fff640 1 -- 172.21.15.94:0/2074488049 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94000f720 con 0x7fb9501000c0 2024-02-17T11:57:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.496+0000 7fb954c9d640 1 -- 172.21.15.94:0/2074488049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95019f750 con 0x7fb9501000c0 2024-02-17T11:57:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.497+0000 7fb954c9d640 1 -- 172.21.15.94:0/2074488049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb950107310 con 0x7fb9501000c0 2024-02-17T11:57:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.498+0000 7fb947fff640 1 -- 172.21.15.94:0/2074488049 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb940010a60 con 0x7fb9501000c0 2024-02-17T11:57:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.498+0000 7fb947fff640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb93c0a20c0 0x7fb93c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:44.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.498+0000 7fb945d9b640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb93c0a20c0 0x7fb93c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.498+0000 7fb947fff640 1 -- 172.21.15.94:0/2074488049 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb9400cdaa0 con 0x7fb9501000c0 2024-02-17T11:57:44.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.499+0000 7fb945d9b640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb93c0a20c0 0x7fb93c0a4580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb95006a0b0 tx=0x7fb938005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:44.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.503+0000 7fb947fff640 1 -- 172.21.15.94:0/2074488049 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb940096a70 con 0x7fb9501000c0 2024-02-17T11:57:44.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.678+0000 7fb954c9d640 1 -- 172.21.15.94:0/2074488049 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb95002ce40 con 0x7fb9501000c0 2024-02-17T11:57:44.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.679+0000 7fb947fff640 1 -- 172.21.15.94:0/2074488049 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22686 (secure 0 0 0) 0x7fb94009a130 con 0x7fb9501000c0 2024-02-17T11:57:44.678 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:44.679 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":87,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:56:59.725646+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:57:44.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.681+0000 7fb954c9d640 1 -- 172.21.15.94:0/2074488049 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb93c0a20c0 msgr2=0x7fb93c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.681+0000 7fb954c9d640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb93c0a20c0 0x7fb93c0a4580 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb95006a0b0 tx=0x7fb938005cc0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.681+0000 7fb954c9d640 1 -- 172.21.15.94:0/2074488049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9501000c0 msgr2=0x7fb95006a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.681+0000 7fb954c9d640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9501000c0 0x7fb95006a340 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb94000d950 tx=0x7fb94000de10 comp rx=0 tx=0).stop 2024-02-17T11:57:44.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.681+0000 7fb954c9d640 1 -- 172.21.15.94:0/2074488049 shutdown_connections 2024-02-17T11:57:44.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.681+0000 7fb954c9d640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb93c0a20c0 0x7fb93c0a4580 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.681+0000 7fb954c9d640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb950101c70 0x7fb95019e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.681+0000 7fb954c9d640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9501012b0 0x7fb95019beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.681+0000 7fb954c9d640 1 --2- 172.21.15.94:0/2074488049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9501000c0 0x7fb95006a340 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.681+0000 7fb954c9d640 1 -- 172.21.15.94:0/2074488049 >> 172.21.15.94:0/2074488049 conn(0x7fb9500fb630 msgr2=0x7fb9501044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:44.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.682+0000 7fb954c9d640 1 -- 172.21.15.94:0/2074488049 shutdown_connections 2024-02-17T11:57:44.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.682+0000 7fb954c9d640 1 -- 172.21.15.94:0/2074488049 wait complete. 2024-02-17T11:57:44.692 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T11:51:20.894601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T11:55:56.349040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T11:55:57.234916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T11:57:44.692 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T11:57:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.799+0000 7fb9f764e640 1 -- 172.21.15.94:0/3594560107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f0100a10 msgr2=0x7fb9f0100e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.799+0000 7fb9f764e640 1 --2- 172.21.15.94:0/3594560107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f0100a10 0x7fb9f0100e90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0009900 tx=0x7fb9e002bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:44.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.799+0000 7fb9f764e640 1 -- 172.21.15.94:0/3594560107 shutdown_connections 2024-02-17T11:57:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.799+0000 7fb9f764e640 1 --2- 172.21.15.94:0/3594560107 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9f01013d0 0x7fb9f0107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.799+0000 7fb9f764e640 1 --2- 172.21.15.94:0/3594560107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f0100a10 0x7fb9f0100e90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.799+0000 7fb9f764e640 1 --2- 172.21.15.94:0/3594560107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9f00ff820 0x7fb9f00ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.799+0000 7fb9f764e640 1 -- 172.21.15.94:0/3594560107 >> 172.21.15.94:0/3594560107 conn(0x7fb9f00fad70 msgr2=0x7fb9f00fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f764e640 1 -- 172.21.15.94:0/3594560107 shutdown_connections 2024-02-17T11:57:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f764e640 1 -- 172.21.15.94:0/3594560107 wait complete. 2024-02-17T11:57:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f764e640 1 Processor -- start 2024-02-17T11:57:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f764e640 1 -- start start 2024-02-17T11:57:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f764e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9f00ff820 0x7fb9f01090d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f764e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9f0100a10 0x7fb9f019bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f764e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f01013d0 0x7fb9f019e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f764e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f019ed10 con 0x7fb9f01013d0 2024-02-17T11:57:44.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f764e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f019ee80 con 0x7fb9f0100a10 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f764e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f019f040 con 0x7fb9f00ff820 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f5beb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f01013d0 0x7fb9f019e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f5beb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f01013d0 0x7fb9f019e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57648/0 (socket says 172.21.15.94:57648) 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f5beb640 1 -- 172.21.15.94:0/3242508880 learned_addr learned my addr 172.21.15.94:0/3242508880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.800+0000 7fb9f4be9640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9f0100a10 0x7fb9f019bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.801+0000 7fb9f5beb640 1 -- 172.21.15.94:0/3242508880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9f00ff820 msgr2=0x7fb9f01090d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.801+0000 7fb9f53ea640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9f00ff820 0x7fb9f01090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.801+0000 7fb9f5beb640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9f00ff820 0x7fb9f01090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.801+0000 7fb9f5beb640 1 -- 172.21.15.94:0/3242508880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9f0100a10 msgr2=0x7fb9f019bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.801+0000 7fb9f5beb640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9f0100a10 0x7fb9f019bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.801+0000 7fb9f5beb640 1 -- 172.21.15.94:0/3242508880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e0009560 con 0x7fb9f01013d0 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.801+0000 7fb9f5beb640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f01013d0 0x7fb9f019e530 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800eb10 tx=0x7fb9e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.801+0000 7fb9e67fc640 1 -- 172.21.15.94:0/3242508880 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8004b20 con 0x7fb9f01013d0 2024-02-17T11:57:44.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.801+0000 7fb9e67fc640 1 -- 172.21.15.94:0/3242508880 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9e8004c80 con 0x7fb9f01013d0 2024-02-17T11:57:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.801+0000 7fb9f764e640 1 -- 172.21.15.94:0/3242508880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f019f2c0 con 0x7fb9f01013d0 2024-02-17T11:57:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.801+0000 7fb9e67fc640 1 -- 172.21.15.94:0/3242508880 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8010650 con 0x7fb9f01013d0 2024-02-17T11:57:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.801+0000 7fb9f764e640 1 -- 172.21.15.94:0/3242508880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f019f750 con 0x7fb9f01013d0 2024-02-17T11:57:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.802+0000 7fb9f764e640 1 -- 172.21.15.94:0/3242508880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f0106ae0 con 0x7fb9f01013d0 2024-02-17T11:57:44.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.803+0000 7fb9e67fc640 1 -- 172.21.15.94:0/3242508880 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fb9e8005020 con 0x7fb9f01013d0 2024-02-17T11:57:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.803+0000 7fb9e67fc640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb9dc0a20c0 0x7fb9dc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.804+0000 7fb9f53ea640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb9dc0a20c0 0x7fb9dc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.804+0000 7fb9e67fc640 1 -- 172.21.15.94:0/3242508880 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb9e8014070 con 0x7fb9f01013d0 2024-02-17T11:57:44.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.804+0000 7fb9f53ea640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb9dc0a20c0 0x7fb9dc0a4580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0100870 tx=0x7fb9ec005cc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:44.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.808+0000 7fb9e67fc640 1 -- 172.21.15.94:0/3242508880 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb9e80972c0 con 0x7fb9f01013d0 2024-02-17T11:57:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:44 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/214396749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:44 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/803134018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:44.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:44 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2074488049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:44.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.981+0000 7fb9f764e640 1 -- 172.21.15.94:0/3242508880 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9f002cea0 con 0x7fb9f01013d0 2024-02-17T11:57:44.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.981+0000 7fb9e67fc640 1 -- 172.21.15.94:0/3242508880 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb9e809a980 con 0x7fb9f01013d0 2024-02-17T11:57:44.980 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-02-17T11:57:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.983+0000 7fb9f764e640 1 -- 172.21.15.94:0/3242508880 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb9dc0a20c0 msgr2=0x7fb9dc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.983+0000 7fb9f764e640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb9dc0a20c0 0x7fb9dc0a4580 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0100870 tx=0x7fb9ec005cc0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.983+0000 7fb9f764e640 1 -- 172.21.15.94:0/3242508880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f01013d0 msgr2=0x7fb9f019e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.983+0000 7fb9f764e640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f01013d0 0x7fb9f019e530 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb9e800eb10 tx=0x7fb9e800c2d0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.983+0000 7fb9f764e640 1 -- 172.21.15.94:0/3242508880 shutdown_connections 2024-02-17T11:57:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.983+0000 7fb9f764e640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb9dc0a20c0 0x7fb9dc0a4580 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.983+0000 7fb9f764e640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9f01013d0 0x7fb9f019e530 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.983+0000 7fb9f764e640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9f0100a10 0x7fb9f019bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.983+0000 7fb9f764e640 1 --2- 172.21.15.94:0/3242508880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9f00ff820 0x7fb9f01090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:44.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.983+0000 7fb9f764e640 1 -- 172.21.15.94:0/3242508880 >> 172.21.15.94:0/3242508880 conn(0x7fb9f00fad70 msgr2=0x7fb9f0103cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:44.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.983+0000 7fb9f764e640 1 -- 172.21.15.94:0/3242508880 shutdown_connections 2024-02-17T11:57:44.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:44.983+0000 7fb9f764e640 1 -- 172.21.15.94:0/3242508880 wait complete. 2024-02-17T11:57:44.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T11:57:45.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 -- 172.21.15.94:0/3263236770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f029c1002d0 msgr2=0x7f029c100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 --2- 172.21.15.94:0/3263236770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f029c1002d0 0x7f029c100750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0284009900 tx=0x7f028402bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 -- 172.21.15.94:0/3263236770 shutdown_connections 2024-02-17T11:57:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 --2- 172.21.15.94:0/3263236770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f029c100c90 0x7f029c108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 --2- 172.21.15.94:0/3263236770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f029c1002d0 0x7f029c100750 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 --2- 172.21.15.94:0/3263236770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f029c101680 0x7f029c101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 -- 172.21.15.94:0/3263236770 >> 172.21.15.94:0/3263236770 conn(0x7f029c0fad80 msgr2=0x7f029c0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 -- 172.21.15.94:0/3263236770 shutdown_connections 2024-02-17T11:57:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 -- 172.21.15.94:0/3263236770 wait complete. 2024-02-17T11:57:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 Processor -- start 2024-02-17T11:57:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 -- start start 2024-02-17T11:57:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f029c1002d0 0x7f029c197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:45.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f029c100c90 0x7f029c197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f029c101680 0x7f029c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.099+0000 7f02a0902640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c19ed20 con 0x7f029c100c90 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f02a0902640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c19ee90 con 0x7f029c1002d0 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f02a0902640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c19f050 con 0x7f029c101680 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f0293fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f029c100c90 0x7f029c197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f0293fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f029c100c90 0x7f029c197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57670/0 (socket says 172.21.15.94:57670) 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f0293fff640 1 -- 172.21.15.94:0/4138771530 learned_addr learned my addr 172.21.15.94:0/4138771530 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f029a59c640 1 --2- 172.21.15.94:0/4138771530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f029c1002d0 0x7f029c197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f0293fff640 1 -- 172.21.15.94:0/4138771530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f029c101680 msgr2=0x7f029c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f0293fff640 1 --2- 172.21.15.94:0/4138771530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f029c101680 0x7f029c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f0293fff640 1 -- 172.21.15.94:0/4138771530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f029c1002d0 msgr2=0x7f029c197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f0293fff640 1 --2- 172.21.15.94:0/4138771530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f029c1002d0 0x7f029c197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f0293fff640 1 -- 172.21.15.94:0/4138771530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0284009560 con 0x7f029c100c90 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f0293fff640 1 --2- 172.21.15.94:0/4138771530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f029c100c90 0x7f029c197770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f028402c840 tx=0x7f0284004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f02937fe640 1 -- 172.21.15.94:0/4138771530 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284004df0 con 0x7f029c100c90 2024-02-17T11:57:45.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f02937fe640 1 -- 172.21.15.94:0/4138771530 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0284005430 con 0x7f029c100c90 2024-02-17T11:57:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f02a0902640 1 -- 172.21.15.94:0/4138771530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029c19f2d0 con 0x7f029c100c90 2024-02-17T11:57:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f02937fe640 1 -- 172.21.15.94:0/4138771530 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284003e60 con 0x7f029c100c90 2024-02-17T11:57:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.100+0000 7f02a0902640 1 -- 172.21.15.94:0/4138771530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029c19f7e0 con 0x7f029c100c90 2024-02-17T11:57:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.101+0000 7f02a0902640 1 -- 172.21.15.94:0/4138771530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029c1073b0 con 0x7f029c100c90 2024-02-17T11:57:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.102+0000 7f02937fe640 1 -- 172.21.15.94:0/4138771530 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f02840035f0 con 0x7f029c100c90 2024-02-17T11:57:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.103+0000 7f02937fe640 1 --2- 172.21.15.94:0/4138771530 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f02880a20c0 0x7f02880a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:45.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.103+0000 7f02937fe640 1 -- 172.21.15.94:0/4138771530 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f02840f07d0 con 0x7f029c100c90 2024-02-17T11:57:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.103+0000 7f029a59c640 1 --2- 172.21.15.94:0/4138771530 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f02880a20c0 0x7f02880a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:45.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.103+0000 7f029a59c640 1 --2- 172.21.15.94:0/4138771530 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f02880a20c0 0x7f02880a4580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f028c0058c0 tx=0x7f028c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:45.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.106+0000 7f02937fe640 1 -- 172.21.15.94:0/4138771530 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f02840b97a0 con 0x7f029c100c90 2024-02-17T11:57:45.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:44 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/214396749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:45.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:44 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/803134018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:45.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:44 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2074488049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:45.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:44 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/214396749' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:45.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:44 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/803134018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:45.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:44 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2074488049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:45.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.278+0000 7f02a0902640 1 -- 172.21.15.94:0/4138771530 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f029c10cc10 con 0x7f029c100c90 2024-02-17T11:57:45.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.279+0000 7f02937fe640 1 -- 172.21.15.94:0/4138771530 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f02840bce60 con 0x7f029c100c90 2024-02-17T11:57:45.278 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-02-17T11:57:45.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.281+0000 7f02a0902640 1 -- 172.21.15.94:0/4138771530 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f02880a20c0 msgr2=0x7f02880a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.281+0000 7f02a0902640 1 --2- 172.21.15.94:0/4138771530 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f02880a20c0 0x7f02880a4580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f028c0058c0 tx=0x7f028c005850 comp rx=0 tx=0).stop 2024-02-17T11:57:45.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.281+0000 7f02a0902640 1 -- 172.21.15.94:0/4138771530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f029c100c90 msgr2=0x7f029c197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.281+0000 7f02a0902640 1 --2- 172.21.15.94:0/4138771530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f029c100c90 0x7f029c197770 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f028402c840 tx=0x7f0284004270 comp rx=0 tx=0).stop 2024-02-17T11:57:45.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.281+0000 7f02a0902640 1 -- 172.21.15.94:0/4138771530 shutdown_connections 2024-02-17T11:57:45.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.281+0000 7f02a0902640 1 --2- 172.21.15.94:0/4138771530 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f02880a20c0 0x7f02880a4580 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.281+0000 7f02a0902640 1 --2- 172.21.15.94:0/4138771530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f029c101680 0x7f029c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.281+0000 7f02a0902640 1 --2- 172.21.15.94:0/4138771530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f029c100c90 0x7f029c197770 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.281+0000 7f02a0902640 1 --2- 172.21.15.94:0/4138771530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f029c1002d0 0x7f029c197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.281+0000 7f02a0902640 1 -- 172.21.15.94:0/4138771530 >> 172.21.15.94:0/4138771530 conn(0x7f029c0fad80 msgr2=0x7f029c104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:45.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.281+0000 7f02a0902640 1 -- 172.21.15.94:0/4138771530 shutdown_connections 2024-02-17T11:57:45.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.281+0000 7f02a0902640 1 -- 172.21.15.94:0/4138771530 wait complete. 2024-02-17T11:57:45.291 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T11:57:45.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.396+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/3331079323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa4101c50 msgr2=0x7f5fa41084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.397+0000 7f5fa8b3a640 1 --2- 172.21.15.94:0/3331079323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa4101c50 0x7f5fa41084e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009ca0 tx=0x7f5f9c02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.397+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/3331079323 shutdown_connections 2024-02-17T11:57:45.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.397+0000 7f5fa8b3a640 1 --2- 172.21.15.94:0/3331079323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa4101c50 0x7f5fa41084e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.397+0000 7f5fa8b3a640 1 --2- 172.21.15.94:0/3331079323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5fa4101290 0x7f5fa4101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.397+0000 7f5fa8b3a640 1 --2- 172.21.15.94:0/3331079323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fa41000a0 0x7f5fa41004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.397+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/3331079323 >> 172.21.15.94:0/3331079323 conn(0x7f5fa40fb630 msgr2=0x7f5fa40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:45.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.397+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/3331079323 shutdown_connections 2024-02-17T11:57:45.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.397+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/3331079323 wait complete. 2024-02-17T11:57:45.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.397+0000 7f5fa8b3a640 1 Processor -- start 2024-02-17T11:57:45.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.397+0000 7f5fa8b3a640 1 -- start start 2024-02-17T11:57:45.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa8b3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa41000a0 0x7f5fa41971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:45.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa8b3a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5fa4101290 0x7f5fa4197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:45.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa8b3a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fa4101c50 0x7f5fa419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:45.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa8b3a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa419ecb0 con 0x7f5fa41000a0 2024-02-17T11:57:45.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa8b3a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa419ee20 con 0x7f5fa4101c50 2024-02-17T11:57:45.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa8b3a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa419efe0 con 0x7f5fa4101290 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa41000a0 0x7f5fa41971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa41000a0 0x7f5fa41971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57690/0 (socket says 172.21.15.94:57690) 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa259c640 1 -- 172.21.15.94:0/1768218498 learned_addr learned my addr 172.21.15.94:0/1768218498 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa2d9d640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fa4101c50 0x7f5fa419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa1d9b640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5fa4101290 0x7f5fa4197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa259c640 1 -- 172.21.15.94:0/1768218498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5fa4101290 msgr2=0x7f5fa4197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa259c640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5fa4101290 0x7f5fa4197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa259c640 1 -- 172.21.15.94:0/1768218498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fa4101c50 msgr2=0x7f5fa419e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa259c640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fa4101c50 0x7f5fa419e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa259c640 1 -- 172.21.15.94:0/1768218498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c009900 con 0x7f5fa41000a0 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa1d9b640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5fa4101290 0x7f5fa4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa259c640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa41000a0 0x7f5fa41971c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800d770 tx=0x7f5f9800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5f937fe640 1 -- 172.21.15.94:0/1768218498 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98009c30 con 0x7f5fa41000a0 2024-02-17T11:57:45.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5f937fe640 1 -- 172.21.15.94:0/1768218498 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f9800de80 con 0x7f5fa41000a0 2024-02-17T11:57:45.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/1768218498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa419f2c0 con 0x7f5fa41000a0 2024-02-17T11:57:45.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5f937fe640 1 -- 172.21.15.94:0/1768218498 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98010460 con 0x7f5fa41000a0 2024-02-17T11:57:45.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.398+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/1768218498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa419f7d0 con 0x7f5fa41000a0 2024-02-17T11:57:45.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.399+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/1768218498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa410bab0 con 0x7f5fa41000a0 2024-02-17T11:57:45.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.400+0000 7f5f937fe640 1 -- 172.21.15.94:0/1768218498 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f5f98003f60 con 0x7f5fa41000a0 2024-02-17T11:57:45.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.401+0000 7f5f937fe640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5f8c0a20c0 0x7f5f8c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:45.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.401+0000 7f5fa1d9b640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5f8c0a20c0 0x7f5f8c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:45.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.401+0000 7f5f937fe640 1 -- 172.21.15.94:0/1768218498 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f5f980cd9a0 con 0x7f5fa41000a0 2024-02-17T11:57:45.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.401+0000 7f5fa1d9b640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5f8c0a20c0 0x7f5f8c0a4580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f940058c0 tx=0x7f5f94005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:45.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.405+0000 7f5f937fe640 1 -- 172.21.15.94:0/1768218498 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f98096970 con 0x7f5fa41000a0 2024-02-17T11:57:45.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.579+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/1768218498 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5fa419fc90 con 0x7f5fa41000a0 2024-02-17T11:57:45.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.579+0000 7f5f937fe640 1 -- 172.21.15.94:0/1768218498 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5f9809a030 con 0x7f5fa41000a0 2024-02-17T11:57:45.578 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-02-17T11:57:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.581+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/1768218498 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5f8c0a20c0 msgr2=0x7f5f8c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.581+0000 7f5fa8b3a640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5f8c0a20c0 0x7f5f8c0a4580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f940058c0 tx=0x7f5f94005850 comp rx=0 tx=0).stop 2024-02-17T11:57:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.581+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/1768218498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa41000a0 msgr2=0x7f5fa41971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.581+0000 7f5fa8b3a640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa41000a0 0x7f5fa41971c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800d770 tx=0x7f5f9800dc30 comp rx=0 tx=0).stop 2024-02-17T11:57:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.581+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/1768218498 shutdown_connections 2024-02-17T11:57:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.581+0000 7f5fa8b3a640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f5f8c0a20c0 0x7f5f8c0a4580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.581+0000 7f5fa8b3a640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5fa4101c50 0x7f5fa419e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.581+0000 7f5fa8b3a640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5fa4101290 0x7f5fa4197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.581+0000 7f5fa8b3a640 1 --2- 172.21.15.94:0/1768218498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5fa41000a0 0x7f5fa41971c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.581+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/1768218498 >> 172.21.15.94:0/1768218498 conn(0x7f5fa40fb630 msgr2=0x7f5fa4069750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.581+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/1768218498 shutdown_connections 2024-02-17T11:57:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.581+0000 7f5fa8b3a640 1 -- 172.21.15.94:0/1768218498 wait complete. 2024-02-17T11:57:45.591 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:57:45.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.694+0000 7f06b9c81640 1 -- 172.21.15.94:0/3869386160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b41000a0 msgr2=0x7f06b41004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.694+0000 7f06b9c81640 1 --2- 172.21.15.94:0/3869386160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b41000a0 0x7f06b41004c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009900 tx=0x7f06a402bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:45.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.694+0000 7f06b9c81640 1 -- 172.21.15.94:0/3869386160 shutdown_connections 2024-02-17T11:57:45.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.694+0000 7f06b9c81640 1 --2- 172.21.15.94:0/3869386160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06b4101c50 0x7f06b41084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.694+0000 7f06b9c81640 1 --2- 172.21.15.94:0/3869386160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06b4101290 0x7f06b4101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.694+0000 7f06b9c81640 1 --2- 172.21.15.94:0/3869386160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b41000a0 0x7f06b41004c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.694+0000 7f06b9c81640 1 -- 172.21.15.94:0/3869386160 >> 172.21.15.94:0/3869386160 conn(0x7f06b40fb610 msgr2=0x7f06b40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.694+0000 7f06b9c81640 1 -- 172.21.15.94:0/3869386160 shutdown_connections 2024-02-17T11:57:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.694+0000 7f06b9c81640 1 -- 172.21.15.94:0/3869386160 wait complete. 2024-02-17T11:57:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.695+0000 7f06b9c81640 1 Processor -- start 2024-02-17T11:57:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.695+0000 7f06b9c81640 1 -- start start 2024-02-17T11:57:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.695+0000 7f06b9c81640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06b41000a0 0x7f06b41972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.695+0000 7f06b9c81640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b4101290 0x7f06b4197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.695+0000 7f06b9c81640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06b4101c50 0x7f06b419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.695+0000 7f06b9c81640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b419ed60 con 0x7f06b4101290 2024-02-17T11:57:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.695+0000 7f06b9c81640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b419eed0 con 0x7f06b4101c50 2024-02-17T11:57:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.695+0000 7f06b9c81640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b419f090 con 0x7f06b41000a0 2024-02-17T11:57:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.695+0000 7f06b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b4101290 0x7f06b4197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:45.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.695+0000 7f06b2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b4101290 0x7f06b4197800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57716/0 (socket says 172.21.15.94:57716) 2024-02-17T11:57:45.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.695+0000 7f06b2ffd640 1 -- 172.21.15.94:0/1840364333 learned_addr learned my addr 172.21.15.94:0/1840364333 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:45.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.696+0000 7f06b37fe640 1 --2- 172.21.15.94:0/1840364333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06b41000a0 0x7f06b41972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:45.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.696+0000 7f06b2ffd640 1 -- 172.21.15.94:0/1840364333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06b41000a0 msgr2=0x7f06b41972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.696+0000 7f06b2ffd640 1 --2- 172.21.15.94:0/1840364333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06b41000a0 0x7f06b41972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.696+0000 7f06b2ffd640 1 -- 172.21.15.94:0/1840364333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06b4101c50 msgr2=0x7f06b419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.696+0000 7f06b2ffd640 1 --2- 172.21.15.94:0/1840364333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06b4101c50 0x7f06b419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.696+0000 7f06b2ffd640 1 -- 172.21.15.94:0/1840364333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a4009560 con 0x7f06b4101290 2024-02-17T11:57:45.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.696+0000 7f06b2ffd640 1 --2- 172.21.15.94:0/1840364333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b4101290 0x7f06b4197800 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f06a800c970 tx=0x7f06a800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:45.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.696+0000 7f06b0ff9640 1 -- 172.21.15.94:0/1840364333 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a8007bb0 con 0x7f06b4101290 2024-02-17T11:57:45.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.696+0000 7f06b0ff9640 1 -- 172.21.15.94:0/1840364333 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06a8007d10 con 0x7f06b4101290 2024-02-17T11:57:45.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.696+0000 7f06b9c81640 1 -- 172.21.15.94:0/1840364333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b419f370 con 0x7f06b4101290 2024-02-17T11:57:45.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.696+0000 7f06b0ff9640 1 -- 172.21.15.94:0/1840364333 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a800f450 con 0x7f06b4101290 2024-02-17T11:57:45.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.696+0000 7f06b9c81640 1 -- 172.21.15.94:0/1840364333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b419f830 con 0x7f06b4101290 2024-02-17T11:57:45.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.697+0000 7f06b9c81640 1 -- 172.21.15.94:0/1840364333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b41073c0 con 0x7f06b4101290 2024-02-17T11:57:45.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.698+0000 7f06b0ff9640 1 -- 172.21.15.94:0/1840364333 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f06a8004f70 con 0x7f06b4101290 2024-02-17T11:57:45.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.699+0000 7f06b0ff9640 1 --2- 172.21.15.94:0/1840364333 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f069c0a20c0 0x7f069c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:45.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.699+0000 7f06b37fe640 1 --2- 172.21.15.94:0/1840364333 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f069c0a20c0 0x7f069c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:45.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.699+0000 7f06b0ff9640 1 -- 172.21.15.94:0/1840364333 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f06a80cda90 con 0x7f06b4101290 2024-02-17T11:57:45.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.699+0000 7f06b37fe640 1 --2- 172.21.15.94:0/1840364333 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f069c0a20c0 0x7f069c0a4580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f06a402c6a0 tx=0x7f06a4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:45.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.703+0000 7f06b0ff9640 1 -- 172.21.15.94:0/1840364333 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f06a8096a60 con 0x7f06b4101290 2024-02-17T11:57:45.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.914+0000 7f06b9c81640 1 -- 172.21.15.94:0/1840364333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f06b402cea0 con 0x7f06b4101290 2024-02-17T11:57:45.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.914+0000 7f06b0ff9640 1 -- 172.21.15.94:0/1840364333 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f06a809a120 con 0x7f06b4101290 2024-02-17T11:57:45.913 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:45.913 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:56:02.151161+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T11:57:45.914 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 20 2024-02-17T11:57:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.916+0000 7f06b9c81640 1 -- 172.21.15.94:0/1840364333 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f069c0a20c0 msgr2=0x7f069c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.916+0000 7f06b9c81640 1 --2- 172.21.15.94:0/1840364333 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f069c0a20c0 0x7f069c0a4580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f06a402c6a0 tx=0x7f06a4005fd0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.916+0000 7f06b9c81640 1 -- 172.21.15.94:0/1840364333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b4101290 msgr2=0x7f06b4197800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:45.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.916+0000 7f06b9c81640 1 --2- 172.21.15.94:0/1840364333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b4101290 0x7f06b4197800 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f06a800c970 tx=0x7f06a800ce30 comp rx=0 tx=0).stop 2024-02-17T11:57:45.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.917+0000 7f06b9c81640 1 -- 172.21.15.94:0/1840364333 shutdown_connections 2024-02-17T11:57:45.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.917+0000 7f06b9c81640 1 --2- 172.21.15.94:0/1840364333 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f069c0a20c0 0x7f069c0a4580 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.917+0000 7f06b9c81640 1 --2- 172.21.15.94:0/1840364333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06b4101c50 0x7f06b419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.917+0000 7f06b9c81640 1 --2- 172.21.15.94:0/1840364333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06b4101290 0x7f06b4197800 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.917+0000 7f06b9c81640 1 --2- 172.21.15.94:0/1840364333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06b41000a0 0x7f06b41972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:45.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.917+0000 7f06b9c81640 1 -- 172.21.15.94:0/1840364333 >> 172.21.15.94:0/1840364333 conn(0x7f06b40fb610 msgr2=0x7f06b4104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:45.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.917+0000 7f06b9c81640 1 -- 172.21.15.94:0/1840364333 shutdown_connections 2024-02-17T11:57:45.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:45.917+0000 7f06b9c81640 1 -- 172.21.15.94:0/1840364333 wait complete. 2024-02-17T11:57:45.925 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:45 smithi094 ceph-mon[32499]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-02-17T11:57:45.925 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:45 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3242508880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:45.926 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:45 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/4138771530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:45.926 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:45 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1768218498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:45.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:57:46.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.029+0000 7ffbba33a640 1 -- 172.21.15.94:0/1721344717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb4100120 msgr2=0x7ffbb4100540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.029+0000 7ffbba33a640 1 --2- 172.21.15.94:0/1721344717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb4100120 0x7ffbb4100540 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ffba8009900 tx=0x7ffba802bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:46.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.029+0000 7ffbba33a640 1 -- 172.21.15.94:0/1721344717 shutdown_connections 2024-02-17T11:57:46.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.029+0000 7ffbba33a640 1 --2- 172.21.15.94:0/1721344717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbb4101cd0 0x7ffbb4108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.029+0000 7ffbba33a640 1 --2- 172.21.15.94:0/1721344717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb4101310 0x7ffbb4101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.029+0000 7ffbba33a640 1 --2- 172.21.15.94:0/1721344717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb4100120 0x7ffbb4100540 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.029+0000 7ffbba33a640 1 -- 172.21.15.94:0/1721344717 >> 172.21.15.94:0/1721344717 conn(0x7ffbb40fb6b0 msgr2=0x7ffbb40fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.029+0000 7ffbba33a640 1 -- 172.21.15.94:0/1721344717 shutdown_connections 2024-02-17T11:57:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.029+0000 7ffbba33a640 1 -- 172.21.15.94:0/1721344717 wait complete. 2024-02-17T11:57:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbba33a640 1 Processor -- start 2024-02-17T11:57:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbba33a640 1 -- start start 2024-02-17T11:57:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbba33a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb4100120 0x7ffbb41962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbba33a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbb4101310 0x7ffbb41967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbba33a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb4101cd0 0x7ffbb419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbba33a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb419dda0 con 0x7ffbb4100120 2024-02-17T11:57:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbba33a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb419df10 con 0x7ffbb4101cd0 2024-02-17T11:57:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbba33a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb419e0d0 con 0x7ffbb4101310 2024-02-17T11:57:46.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbb3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb4100120 0x7ffbb41962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbb3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb4100120 0x7ffbb41962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57734/0 (socket says 172.21.15.94:57734) 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbb3fff640 1 -- 172.21.15.94:0/1676651250 learned_addr learned my addr 172.21.15.94:0/1676651250 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbb3fff640 1 -- 172.21.15.94:0/1676651250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbb4101310 msgr2=0x7ffbb41967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbb37fe640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbb4101310 0x7ffbb41967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbb88d7640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb4101cd0 0x7ffbb419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbb3fff640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbb4101310 0x7ffbb41967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbb3fff640 1 -- 172.21.15.94:0/1676651250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb4101cd0 msgr2=0x7ffbb419d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbb3fff640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb4101cd0 0x7ffbb419d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbb3fff640 1 -- 172.21.15.94:0/1676651250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba8009560 con 0x7ffbb4100120 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.030+0000 7ffbb37fe640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbb4101310 0x7ffbb41967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.031+0000 7ffbb3fff640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb4100120 0x7ffbb41962b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ffba8036040 tx=0x7ffba80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.031+0000 7ffbb17fa640 1 -- 172.21.15.94:0/1676651250 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba80044e0 con 0x7ffbb4100120 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.031+0000 7ffbb17fa640 1 -- 172.21.15.94:0/1676651250 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffba80054e0 con 0x7ffbb4100120 2024-02-17T11:57:46.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.031+0000 7ffbba33a640 1 -- 172.21.15.94:0/1676651250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb419e350 con 0x7ffbb4100120 2024-02-17T11:57:46.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.031+0000 7ffbb17fa640 1 -- 172.21.15.94:0/1676651250 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba803e450 con 0x7ffbb4100120 2024-02-17T11:57:46.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.031+0000 7ffbba33a640 1 -- 172.21.15.94:0/1676651250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb419e930 con 0x7ffbb4100120 2024-02-17T11:57:46.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.032+0000 7ffbba33a640 1 -- 172.21.15.94:0/1676651250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb4107510 con 0x7ffbb4100120 2024-02-17T11:57:46.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.033+0000 7ffbb17fa640 1 -- 172.21.15.94:0/1676651250 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ffba8003710 con 0x7ffbb4100120 2024-02-17T11:57:46.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.033+0000 7ffbb17fa640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffb9c0a2140 0x7ffb9c0a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.034+0000 7ffbb37fe640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffb9c0a2140 0x7ffb9c0a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.034+0000 7ffbb17fa640 1 -- 172.21.15.94:0/1676651250 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffba80f06d0 con 0x7ffbb4100120 2024-02-17T11:57:46.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.034+0000 7ffbb37fe640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffb9c0a2140 0x7ffb9c0a4600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffba4006fd0 tx=0x7ffba4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.037+0000 7ffbb17fa640 1 -- 172.21.15.94:0/1676651250 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffba80b96a0 con 0x7ffbb4100120 2024-02-17T11:57:46.051 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:45 smithi189 ceph-mon[36588]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-02-17T11:57:46.052 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:45 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3242508880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:46.052 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:45 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/4138771530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:46.052 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:45 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1768218498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:45 smithi125 ceph-mon[36819]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-02-17T11:57:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:45 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3242508880' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:45 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/4138771530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:45 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1768218498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:46.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.212+0000 7ffbba33a640 1 -- 172.21.15.94:0/1676651250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffbb402ce40 con 0x7ffbb4100120 2024-02-17T11:57:46.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.214+0000 7ffbb17fa640 1 -- 172.21.15.94:0/1676651250 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22686 (secure 0 0 0) 0x7ffba80bcd60 con 0x7ffbb4100120 2024-02-17T11:57:46.213 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:46.213 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":87,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:56:59.725646+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:57:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.216+0000 7ffbba33a640 1 -- 172.21.15.94:0/1676651250 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffb9c0a2140 msgr2=0x7ffb9c0a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.216+0000 7ffbba33a640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffb9c0a2140 0x7ffb9c0a4600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffba4006fd0 tx=0x7ffba4008040 comp rx=0 tx=0).stop 2024-02-17T11:57:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.216+0000 7ffbba33a640 1 -- 172.21.15.94:0/1676651250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb4100120 msgr2=0x7ffbb41962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.216+0000 7ffbba33a640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb4100120 0x7ffbb41962b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ffba8036040 tx=0x7ffba80042f0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.216+0000 7ffbba33a640 1 -- 172.21.15.94:0/1676651250 shutdown_connections 2024-02-17T11:57:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.216+0000 7ffbba33a640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffb9c0a2140 0x7ffb9c0a4600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.216+0000 7ffbba33a640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffbb4101cd0 0x7ffbb419d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.216+0000 7ffbba33a640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffbb4101310 0x7ffbb41967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.216+0000 7ffbba33a640 1 --2- 172.21.15.94:0/1676651250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffbb4100120 0x7ffbb41962b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.216+0000 7ffbba33a640 1 -- 172.21.15.94:0/1676651250 >> 172.21.15.94:0/1676651250 conn(0x7ffbb40fb6b0 msgr2=0x7ffbb41046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.216+0000 7ffbba33a640 1 -- 172.21.15.94:0/1676651250 shutdown_connections 2024-02-17T11:57:46.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.216+0000 7ffbba33a640 1 -- 172.21.15.94:0/1676651250 wait complete. 2024-02-17T11:57:46.226 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-17T11:55:58.000681+0000', 'modified': '2024-02-17T11:56:02.151161+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24488, 'mds_1': 24487, 'mds_2': 24469}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.125:6835/1238777175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6834', 'nonce': 1238777175}, {'type': 'v1', 'addr': '172.21.15.125:6835', 'nonce': 1238777175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24452': {'gid': 24452, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.125:6837/1075914834', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6836', 'nonce': 1075914834}, {'type': 'v1', 'addr': '172.21.15.125:6837', 'nonce': 1075914834}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24454': {'gid': 24454, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.189:6835/2720928164', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 2720928164}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 2720928164}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 2, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.189:6837/860722582', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 860722582}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 860722582}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 1, 'incarnation': 16, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.189:6839/308580523', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6838', 'nonce': 308580523}, {'type': 'v1', 'addr': '172.21.15.189:6839', 'nonce': 308580523}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24488': {'gid': 24488, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.125:6839/1158324088', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.125:6838', 'nonce': 1158324088}, {'type': 'v1', 'addr': '172.21.15.125:6839', 'nonce': 1158324088}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-17T11:57:46.226 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-02-17T11:57:46.227 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-17T11:57:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.331+0000 7f91163e2640 1 -- 172.21.15.94:0/2951035384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91101016b0 msgr2=0x7f9110101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.331+0000 7f91163e2640 1 --2- 172.21.15.94:0/2951035384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91101016b0 0x7f9110101ad0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9100009900 tx=0x7f910002bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.331+0000 7f91163e2640 1 -- 172.21.15.94:0/2951035384 shutdown_connections 2024-02-17T11:57:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.331+0000 7f91163e2640 1 --2- 172.21.15.94:0/2951035384 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9110100d10 0x7f91101085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.331+0000 7f91163e2640 1 --2- 172.21.15.94:0/2951035384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9110100350 0x7f91101007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.331+0000 7f91163e2640 1 --2- 172.21.15.94:0/2951035384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91101016b0 0x7f9110101ad0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.331+0000 7f91163e2640 1 -- 172.21.15.94:0/2951035384 >> 172.21.15.94:0/2951035384 conn(0x7f91100fad80 msgr2=0x7f91100fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.331+0000 7f91163e2640 1 -- 172.21.15.94:0/2951035384 shutdown_connections 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.331+0000 7f91163e2640 1 -- 172.21.15.94:0/2951035384 wait complete. 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.331+0000 7f91163e2640 1 Processor -- start 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.331+0000 7f91163e2640 1 -- start start 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f91163e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9110100350 0x7f91100ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f91163e2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9110100d10 0x7f911019c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f91163e2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91101016b0 0x7f911019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f91163e2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911019ee10 con 0x7f9110100350 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f91163e2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911019ef80 con 0x7f9110100d10 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f91163e2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911019f140 con 0x7f91101016b0 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9110100350 0x7f91100ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9110100350 0x7f91100ffdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57748/0 (socket says 172.21.15.94:57748) 2024-02-17T11:57:46.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910ffff640 1 -- 172.21.15.94:0/1294141670 learned_addr learned my addr 172.21.15.94:0/1294141670 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f911497f640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91101016b0 0x7f911019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910f7fe640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9110100d10 0x7f911019c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910ffff640 1 -- 172.21.15.94:0/1294141670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91101016b0 msgr2=0x7f911019e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910ffff640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91101016b0 0x7f911019e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910ffff640 1 -- 172.21.15.94:0/1294141670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9110100d10 msgr2=0x7f911019c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910ffff640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9110100d10 0x7f911019c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910ffff640 1 -- 172.21.15.94:0/1294141670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9100009560 con 0x7f9110100350 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910f7fe640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9110100d10 0x7f911019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910ffff640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9110100350 0x7f91100ffdb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9100000c00 tx=0x7f9100004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910d7fa640 1 -- 172.21.15.94:0/1294141670 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100004df0 con 0x7f9110100350 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910d7fa640 1 -- 172.21.15.94:0/1294141670 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9100005430 con 0x7f9110100350 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f910d7fa640 1 -- 172.21.15.94:0/1294141670 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910003e450 con 0x7f9110100350 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.332+0000 7f91163e2640 1 -- 172.21.15.94:0/1294141670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911019f360 con 0x7f9110100350 2024-02-17T11:57:46.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.333+0000 7f91163e2640 1 -- 172.21.15.94:0/1294141670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911019f8f0 con 0x7f9110100350 2024-02-17T11:57:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.334+0000 7f910d7fa640 1 -- 172.21.15.94:0/1294141670 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f91000055a0 con 0x7f9110100350 2024-02-17T11:57:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f9118fb8740 1 Processor -- start 2024-02-17T11:57:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f9118fb8740 1 -- start start 2024-02-17T11:57:46.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f9118fb8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaee390 0x562f8aaee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f9118fb8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaeed80 0x562f8aaf3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f9118fb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaf3ef0 0x562f8aaf6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f9118fb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8aaa5cd0 con 0x562f8aaf3ef0 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f9118fb8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8aa58f20 con 0x562f8aaeed80 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f9118fb8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8aa56450 con 0x562f8aaee390 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f910d7fa640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f90f80a20c0 0x7f90f80a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f910f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaf3ef0 0x562f8aaf6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f911497f640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f90f80a20c0 0x7f90f80a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f910f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaf3ef0 0x562f8aaf6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57750/0 (socket says 172.21.15.94:57750) 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f910f7fe640 1 -- 172.21.15.94:0/3782068586 learned_addr learned my addr 172.21.15.94:0/3782068586 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f910d7fa640 1 -- 172.21.15.94:0/1294141670 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f91000f0280 con 0x7f9110100350 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f910ffff640 1 --2- 172.21.15.94:0/3782068586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaeed80 0x562f8aaf3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f911497f640 1 --2- 172.21.15.94:0/3782068586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaee390 0x562f8aaee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f910f7fe640 1 -- 172.21.15.94:0/3782068586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaee390 msgr2=0x562f8aaee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f910f7fe640 1 --2- 172.21.15.94:0/3782068586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaee390 0x562f8aaee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f910f7fe640 1 -- 172.21.15.94:0/3782068586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaeed80 msgr2=0x562f8aaf3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f910f7fe640 1 --2- 172.21.15.94:0/3782068586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaeed80 0x562f8aaf3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.335+0000 7f910f7fe640 1 -- 172.21.15.94:0/3782068586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f8aa06f20 con 0x562f8aaf3ef0 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f910f7fe640 1 --2- 172.21.15.94:0/3782068586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaf3ef0 0x562f8aaf6380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9104007800 tx=0x7f9104007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17347ce99117f173 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f911497f640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f90f80a20c0 0x7f90f80a4580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9108005fd0 tx=0x7f9108012a30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f90f67fc640 1 -- 172.21.15.94:0/3782068586 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104007e50 con 0x562f8aaf3ef0 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f90f67fc640 1 -- 172.21.15.94:0/3782068586 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9104007440 con 0x562f8aaf3ef0 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f90f67fc640 1 -- 172.21.15.94:0/3782068586 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91040123e0 con 0x562f8aaf3ef0 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f9118fb8740 1 -- 172.21.15.94:0/3782068586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaf3ef0 msgr2=0x562f8aaf6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f9118fb8740 1 --2- 172.21.15.94:0/3782068586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaf3ef0 0x562f8aaf6380 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9104007800 tx=0x7f9104007c00 comp rx=0 tx=0).stop 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f9118fb8740 1 -- 172.21.15.94:0/3782068586 shutdown_connections 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f9118fb8740 1 --2- 172.21.15.94:0/3782068586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaf3ef0 0x562f8aaf6380 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f9118fb8740 1 --2- 172.21.15.94:0/3782068586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaeed80 0x562f8aaf3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f9118fb8740 1 --2- 172.21.15.94:0/3782068586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaee390 0x562f8aaee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f9118fb8740 1 -- 172.21.15.94:0/3782068586 >> 172.21.15.94:0/3782068586 conn(0x562f8aae54f0 msgr2=0x562f8aae58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f9118fb8740 1 -- 172.21.15.94:0/3782068586 shutdown_connections 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.336+0000 7f9118fb8740 1 -- 172.21.15.94:0/3782068586 wait complete. 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f9118fb8740 1 -- start start 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f9118fb8740 1 Processor -- start 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f9118fb8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaee390 0x562f8aaf8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f9118fb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 0x562f8aaf8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f9118fb8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaf3ef0 0x562f8aaffa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f9118fb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8aa51c30 con 0x562f8aaeed80 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f9118fb8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8a9f6d30 con 0x562f8aaee390 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f9118fb8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8a725310 con 0x562f8aaf3ef0 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f9118fb8740 10 client.0 _ll_register_callbacks cb 0x562f8a9fed40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f910f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 0x562f8aaf8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f910f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 0x562f8aaf8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57756/0 (socket says 172.21.15.94:57756) 2024-02-17T11:57:46.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f910f7fe640 1 -- 172.21.15.94:0/3003381688 learned_addr learned my addr 172.21.15.94:0/3003381688 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f910ffff640 1 --2- 172.21.15.94:0/3003381688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaee390 0x562f8aaf8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f910f7fe640 1 -- 172.21.15.94:0/3003381688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaf3ef0 msgr2=0x562f8aaffa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f910f7fe640 1 --2- 172.21.15.94:0/3003381688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaf3ef0 0x562f8aaffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f910f7fe640 1 -- 172.21.15.94:0/3003381688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaee390 msgr2=0x562f8aaf8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f910f7fe640 1 --2- 172.21.15.94:0/3003381688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaee390 0x562f8aaf8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.337+0000 7f910f7fe640 1 -- 172.21.15.94:0/3003381688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910400e040 con 0x562f8aaeed80 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f910f7fe640 1 --2- 172.21.15.94:0/3003381688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 0x562f8aaf8a80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9104000c00 tx=0x7f91040136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf1943bc6b3017e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f90ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f90ef7fe640 1 -- 172.21.15.94:0/3003381688 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910400dca0 con 0x562f8aaeed80 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f90ef7fe640 1 -- 172.21.15.94:0/3003381688 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9104027410 con 0x562f8aaeed80 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f9118fb8740 1 -- 172.21.15.94:0/3003381688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f8a7a2270 con 0x562f8aaeed80 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f90ef7fe640 1 -- 172.21.15.94:0/3003381688 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f910400de00 con 0x562f8aaeed80 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f9118fb8740 10 client.15201 fetch_fsmap learned FSMap version 20 2024-02-17T11:57:46.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f9118fb8740 1 -- 172.21.15.94:0/3003381688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x562f8aa502b0 con 0x562f8aaeed80 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f90ef7fe640 1 -- 172.21.15.94:0/3003381688 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562f8a7a2270 con 0x562f8aaeed80 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f90ef7fe640 1 -- 172.21.15.94:0/3003381688 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9104032770 con 0x562f8aaeed80 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f9118fb8740 10 client.15201 fetch_fsmap finished waiting for FSMap version 20 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f9118fb8740 10 client.15201 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f9118fb8740 20 client.15201 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f9118fb8740 1 --2- 172.21.15.94:0/3003381688 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x562f8ab040a0 0x562f8ab09110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f9118fb8740 4 client.15201 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.338+0000 7f9118fb8740 1 -- 172.21.15.94:0/3003381688 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f8a78e610 con 0x562f8ab040a0 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.339+0000 7f910ffff640 1 --2- 172.21.15.94:0/3003381688 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x562f8ab040a0 0x562f8ab09110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.339+0000 7f90ef7fe640 10 client.15201 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.339+0000 7f910ffff640 1 --2- 172.21.15.94:0/3003381688 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x562f8ab040a0 0x562f8ab09110 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b51dceb09b7e3ab server_cookie=d86c56357f71ba73 in_seq=0 out_seq=0 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f90ef7fe640 1 -- 172.21.15.94:0/3003381688 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562f8a78e610 con 0x562f8ab040a0 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f90ef7fe640 10 client.15201 handle_command_reply: tid=0 2024-02-17T11:57:46.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f9118fb8740 1 client.15201 shutdown 2024-02-17T11:57:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f9118fb8740 1 -- 172.21.15.94:0/3003381688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 msgr2=0x562f8aaf8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f9118fb8740 1 --2- 172.21.15.94:0/3003381688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 0x562f8aaf8a80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9104000c00 tx=0x7f91040136b0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f9118fb8740 1 -- 172.21.15.94:0/3003381688 shutdown_connections 2024-02-17T11:57:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f9118fb8740 1 --2- 172.21.15.94:0/3003381688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaf3ef0 0x562f8aaffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f9118fb8740 1 --2- 172.21.15.94:0/3003381688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 0x562f8aaf8a80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f9118fb8740 1 --2- 172.21.15.94:0/3003381688 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x562f8ab040a0 0x562f8ab09110 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f9118fb8740 1 --2- 172.21.15.94:0/3003381688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaee390 0x562f8aaf8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f90ef7fe640 0 client.15201 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T11:57:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f9118fb8740 1 -- 172.21.15.94:0/3003381688 >> 172.21.15.94:0/3003381688 conn(0x562f8aae54f0 msgr2=0x562f8aae47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f9118fb8740 1 -- 172.21.15.94:0/3003381688 shutdown_connections 2024-02-17T11:57:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f9118fb8740 1 -- 172.21.15.94:0/3003381688 wait complete. 2024-02-17T11:57:46.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.340+0000 7f9118fb8740 20 client.15201 trim_cache size 0 max 16384 2024-02-17T11:57:46.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.355+0000 7f9118fb8740 1 Processor -- start 2024-02-17T11:57:46.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.355+0000 7f9118fb8740 1 -- start start 2024-02-17T11:57:46.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f9118fb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaee390 0x562f8aaf7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f9118fb8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaeed80 0x562f8aaffa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f9118fb8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaf3ef0 0x562f8aaf8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f9118fb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8a9d7490 con 0x562f8aaee390 2024-02-17T11:57:46.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f9118fb8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8a78e640 con 0x562f8aaf3ef0 2024-02-17T11:57:46.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f9118fb8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8aaef470 con 0x562f8aaeed80 2024-02-17T11:57:46.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f910ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaee390 0x562f8aaf7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f910ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaee390 0x562f8aaf7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57764/0 (socket says 172.21.15.94:57764) 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f910ffff640 1 -- 172.21.15.94:0/1555302614 learned_addr learned my addr 172.21.15.94:0/1555302614 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f911497f640 1 --2- 172.21.15.94:0/1555302614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaf3ef0 0x562f8aaf8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f910ffff640 1 -- 172.21.15.94:0/1555302614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaeed80 msgr2=0x562f8aaffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f910ffff640 1 --2- 172.21.15.94:0/1555302614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaeed80 0x562f8aaffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f910ffff640 1 -- 172.21.15.94:0/1555302614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaf3ef0 msgr2=0x562f8aaf8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f910ffff640 1 --2- 172.21.15.94:0/1555302614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaf3ef0 0x562f8aaf8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f910ffff640 1 -- 172.21.15.94:0/1555302614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f8a9eca20 con 0x562f8aaee390 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f911497f640 1 --2- 172.21.15.94:0/1555302614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaf3ef0 0x562f8aaf8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f910ffff640 1 --2- 172.21.15.94:0/1555302614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaee390 0x562f8aaf7b60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f91000bccb0 tx=0x7f91000edc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b928b7a55bc19175 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f90f5ffb640 1 -- 172.21.15.94:0/1555302614 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91000c5750 con 0x562f8aaee390 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f90f5ffb640 1 -- 172.21.15.94:0/1555302614 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91000bb2b0 con 0x562f8aaee390 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.356+0000 7f90f5ffb640 1 -- 172.21.15.94:0/1555302614 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100003680 con 0x562f8aaee390 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 -- 172.21.15.94:0/1555302614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaee390 msgr2=0x562f8aaf7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 --2- 172.21.15.94:0/1555302614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaee390 0x562f8aaf7b60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f91000bccb0 tx=0x7f91000edc50 comp rx=0 tx=0).stop 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 -- 172.21.15.94:0/1555302614 shutdown_connections 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 --2- 172.21.15.94:0/1555302614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaf3ef0 0x562f8aaf8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 --2- 172.21.15.94:0/1555302614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaeed80 0x562f8aaffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 --2- 172.21.15.94:0/1555302614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaee390 0x562f8aaf7b60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 -- 172.21.15.94:0/1555302614 >> 172.21.15.94:0/1555302614 conn(0x562f8aae54f0 msgr2=0x562f8aae58d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 -- 172.21.15.94:0/1555302614 shutdown_connections 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 -- 172.21.15.94:0/1555302614 wait complete. 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 -- start start 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 Processor -- start 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaee390 0x562f8aaff810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 0x562f8aafc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaf3ef0 0x562f8aafc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8aafcde0 con 0x562f8aaeed80 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8aafcf50 con 0x562f8aaf3ef0 2024-02-17T11:57:46.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f8ab5ae50 con 0x562f8aaee390 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f9118fb8740 10 client.0 _ll_register_callbacks cb 0x7f9104007220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.357+0000 7f910f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 0x562f8aafc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f910f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 0x562f8aafc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57770/0 (socket says 172.21.15.94:57770) 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f910f7fe640 1 -- 172.21.15.94:0/490947969 learned_addr learned my addr 172.21.15.94:0/490947969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f910ffff640 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaee390 0x562f8aaff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f911497f640 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaf3ef0 0x562f8aafc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f910f7fe640 1 -- 172.21.15.94:0/490947969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaee390 msgr2=0x562f8aaff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f910f7fe640 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaee390 0x562f8aaff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f910f7fe640 1 -- 172.21.15.94:0/490947969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaf3ef0 msgr2=0x562f8aafc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f910f7fe640 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaf3ef0 0x562f8aafc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f910f7fe640 1 -- 172.21.15.94:0/490947969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911019f8f0 con 0x562f8aaeed80 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f911497f640 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaf3ef0 0x562f8aafc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f910f7fe640 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 0x562f8aafc2e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x562f8aaf99b0 tx=0x7f9104006f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2319e7e2e7269646 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f90ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:46.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f90ef7fe640 1 -- 172.21.15.94:0/490947969 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91040137e0 con 0x562f8aaeed80 2024-02-17T11:57:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f90ef7fe640 1 -- 172.21.15.94:0/490947969 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f9104000e40 con 0x562f8aaeed80 2024-02-17T11:57:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f9118fb8740 1 -- 172.21.15.94:0/490947969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f8ab05000 con 0x562f8aaeed80 2024-02-17T11:57:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f90ef7fe640 1 -- 172.21.15.94:0/490947969 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9104013940 con 0x562f8aaeed80 2024-02-17T11:57:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f9118fb8740 10 client.15207 fetch_fsmap learned FSMap version 20 2024-02-17T11:57:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f9118fb8740 1 -- 172.21.15.94:0/490947969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x562f8aa12780 con 0x562f8aaeed80 2024-02-17T11:57:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f90ef7fe640 1 -- 172.21.15.94:0/490947969 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562f8ab05000 con 0x562f8aaeed80 2024-02-17T11:57:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f90ef7fe640 1 -- 172.21.15.94:0/490947969 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9104038030 con 0x562f8aaeed80 2024-02-17T11:57:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.358+0000 7f9118fb8740 10 client.15207 fetch_fsmap finished waiting for FSMap version 20 2024-02-17T11:57:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.359+0000 7f9118fb8740 10 client.15207 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-17T11:57:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.359+0000 7f9118fb8740 20 client.15207 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.359+0000 7f9118fb8740 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x562f8ab5ff20 0x562f8ab5c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.359+0000 7f9118fb8740 4 client.15207 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T11:57:46.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.359+0000 7f9118fb8740 1 -- 172.21.15.94:0/490947969 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562f8a7516c0 con 0x562f8ab5ff20 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.359+0000 7f910ffff640 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x562f8ab5ff20 0x562f8ab5c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.359+0000 7f90ef7fe640 10 client.15207 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.359+0000 7f910ffff640 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x562f8ab5ff20 0x562f8ab5c3c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d4b9a5f5fd9a45cd server_cookie=b51db5620e6a5b4c in_seq=0 out_seq=0 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.359+0000 7f90ef7fe640 1 -- 172.21.15.94:0/490947969 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x562f8a7516c0 con 0x562f8ab5ff20 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.359+0000 7f90ef7fe640 10 client.15207 handle_command_reply: tid=0 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.359+0000 7f9118fb8740 1 client.15207 shutdown 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f9118fb8740 1 -- 172.21.15.94:0/490947969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 msgr2=0x562f8aafc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f9118fb8740 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 0x562f8aafc2e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x562f8aaf99b0 tx=0x7f9104006f40 comp rx=0 tx=0).stop 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f9118fb8740 1 -- 172.21.15.94:0/490947969 shutdown_connections 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f9118fb8740 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x562f8ab5ff20 0x562f8ab5c3c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f9118fb8740 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562f8aaf3ef0 0x562f8aafc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f9118fb8740 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562f8aaeed80 0x562f8aafc2e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f90ef7fe640 0 client.15207 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T11:57:46.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f9118fb8740 1 --2- 172.21.15.94:0/490947969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562f8aaee390 0x562f8aaff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f9118fb8740 1 -- 172.21.15.94:0/490947969 >> 172.21.15.94:0/490947969 conn(0x562f8aae54f0 msgr2=0x562f8aaf8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f9118fb8740 1 -- 172.21.15.94:0/490947969 shutdown_connections 2024-02-17T11:57:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f9118fb8740 1 -- 172.21.15.94:0/490947969 wait complete. 2024-02-17T11:57:46.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f9118fb8740 20 client.15207 trim_cache size 0 max 16384 2024-02-17T11:57:46.366 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:57:46.366 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "3bc5cfc8-cd8a-11ee-95ba-87774f69a715", 2024-02-17T11:57:46.366 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 2, 2024-02-17T11:57:46.366 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24469, 2024-02-17T11:57:46.366 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-02-17T11:57:46.366 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-02-17T11:57:46.366 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-02-17T11:57:46.366 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 105.214641212, 2024-02-17T11:57:46.367 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 20, 2024-02-17T11:57:46.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 87, 2024-02-17T11:57:46.367 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 85, 2024-02-17T11:57:46.367 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 128.082099035 2024-02-17T11:57:46.367 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:57:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f91163e2640 1 -- 172.21.15.94:0/1294141670 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f90f80a20c0 msgr2=0x7f90f80a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f91163e2640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f90f80a20c0 0x7f90f80a4580 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f9108005fd0 tx=0x7f9108012a30 comp rx=0 tx=0).stop 2024-02-17T11:57:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f91163e2640 1 -- 172.21.15.94:0/1294141670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9110100350 msgr2=0x7f91100ffdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f91163e2640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9110100350 0x7f91100ffdb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9100000c00 tx=0x7f9100004270 comp rx=0 tx=0).stop 2024-02-17T11:57:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f91163e2640 1 -- 172.21.15.94:0/1294141670 shutdown_connections 2024-02-17T11:57:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f91163e2640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f90f80a20c0 0x7f90f80a4580 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f91163e2640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91101016b0 0x7f911019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f91163e2640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9110100d10 0x7f911019c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f91163e2640 1 --2- 172.21.15.94:0/1294141670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9110100350 0x7f91100ffdb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f91163e2640 1 -- 172.21.15.94:0/1294141670 >> 172.21.15.94:0/1294141670 conn(0x7f91100fad80 msgr2=0x7f9110104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f91163e2640 1 -- 172.21.15.94:0/1294141670 shutdown_connections 2024-02-17T11:57:46.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.360+0000 7f91163e2640 1 -- 172.21.15.94:0/1294141670 wait complete. 2024-02-17T11:57:46.370 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-17T11:57:46.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.473+0000 7f08db055640 1 -- 172.21.15.94:0/1553613851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d4101c00 msgr2=0x7f08d4103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.473+0000 7f08db055640 1 --2- 172.21.15.94:0/1553613851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d4101c00 0x7f08d4103ff0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f08c4009900 tx=0x7f08c402bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.473+0000 7f08db055640 1 -- 172.21.15.94:0/1553613851 shutdown_connections 2024-02-17T11:57:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.473+0000 7f08db055640 1 --2- 172.21.15.94:0/1553613851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d4106e60 0x7f08d41092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.473+0000 7f08db055640 1 --2- 172.21.15.94:0/1553613851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08d4104530 0x7f08d4106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.473+0000 7f08db055640 1 --2- 172.21.15.94:0/1553613851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d4101c00 0x7f08d4103ff0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08db055640 1 -- 172.21.15.94:0/1553613851 >> 172.21.15.94:0/1553613851 conn(0x7f08d40fb610 msgr2=0x7f08d40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08db055640 1 -- 172.21.15.94:0/1553613851 shutdown_connections 2024-02-17T11:57:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08db055640 1 -- 172.21.15.94:0/1553613851 wait complete. 2024-02-17T11:57:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08db055640 1 Processor -- start 2024-02-17T11:57:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08db055640 1 -- start start 2024-02-17T11:57:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08db055640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d4101c00 0x7f08d41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08db055640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d4104530 0x7f08d4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08db055640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08d4106e60 0x7f08d419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08db055640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d419edc0 con 0x7f08d4104530 2024-02-17T11:57:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08db055640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d419ef30 con 0x7f08d4106e60 2024-02-17T11:57:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08db055640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d419f0f0 con 0x7f08d4101c00 2024-02-17T11:57:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d4104530 0x7f08d4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d4104530 0x7f08d4197810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57798/0 (socket says 172.21.15.94:57798) 2024-02-17T11:57:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.474+0000 7f08d3fff640 1 -- 172.21.15.94:0/4022337451 learned_addr learned my addr 172.21.15.94:0/4022337451 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.475+0000 7f08d8df1640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d4101c00 0x7f08d41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.475+0000 7f08d95f2640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08d4106e60 0x7f08d419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.475+0000 7f08d3fff640 1 -- 172.21.15.94:0/4022337451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d4101c00 msgr2=0x7f08d41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.475+0000 7f08d3fff640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d4101c00 0x7f08d41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.475+0000 7f08d3fff640 1 -- 172.21.15.94:0/4022337451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08d4106e60 msgr2=0x7f08d419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.475+0000 7f08d3fff640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08d4106e60 0x7f08d419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.475+0000 7f08d3fff640 1 -- 172.21.15.94:0/4022337451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c4009560 con 0x7f08d4104530 2024-02-17T11:57:46.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.475+0000 7f08d3fff640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d4104530 0x7f08d4197810 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f08c800d770 tx=0x7f08c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.475+0000 7f08d1ffb640 1 -- 172.21.15.94:0/4022337451 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8009c30 con 0x7f08d4104530 2024-02-17T11:57:46.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.475+0000 7f08d1ffb640 1 -- 172.21.15.94:0/4022337451 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08c800de80 con 0x7f08d4104530 2024-02-17T11:57:46.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.475+0000 7f08db055640 1 -- 172.21.15.94:0/4022337451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d419f3d0 con 0x7f08d4104530 2024-02-17T11:57:46.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.475+0000 7f08d1ffb640 1 -- 172.21.15.94:0/4022337451 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c800f6b0 con 0x7f08d4104530 2024-02-17T11:57:46.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.475+0000 7f08db055640 1 -- 172.21.15.94:0/4022337451 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d419f960 con 0x7f08d4104530 2024-02-17T11:57:46.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.477+0000 7f08d1ffb640 1 -- 172.21.15.94:0/4022337451 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f08c8010460 con 0x7f08d4104530 2024-02-17T11:57:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.477+0000 7f08ddc2f740 1 Processor -- start 2024-02-17T11:57:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.477+0000 7f08ddc2f740 1 -- start start 2024-02-17T11:57:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08ddc2f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1df80 0x563b02d1e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08ddc2f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1e970 0x563b02d23470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08ddc2f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d23ae0 0x563b02d25f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08ddc2f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02cd9cd0 con 0x563b02d1df80 2024-02-17T11:57:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08ddc2f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02c8cf20 con 0x563b02d23ae0 2024-02-17T11:57:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08ddc2f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02c8a450 con 0x563b02d1e970 2024-02-17T11:57:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d95f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1df80 0x563b02d1e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d95f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1df80 0x563b02d1e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57808/0 (socket says 172.21.15.94:57808) 2024-02-17T11:57:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d95f2640 1 -- 172.21.15.94:0/3924857831 learned_addr learned my addr 172.21.15.94:0/3924857831 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d1ffb640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08bc0a20c0 0x7f08bc0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d8df1640 1 --2- 172.21.15.94:0/3924857831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1e970 0x563b02d23470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d95f2640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08bc0a20c0 0x7f08bc0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d1ffb640 1 -- 172.21.15.94:0/4022337451 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f08c80cdaf0 con 0x7f08d4104530 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d95f2640 1 -- 172.21.15.94:0/3924857831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1e970 msgr2=0x563b02d23470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d3fff640 1 --2- 172.21.15.94:0/3924857831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d23ae0 0x563b02d25f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d95f2640 1 --2- 172.21.15.94:0/3924857831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1e970 0x563b02d23470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d95f2640 1 -- 172.21.15.94:0/3924857831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d23ae0 msgr2=0x563b02d25f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d95f2640 1 --2- 172.21.15.94:0/3924857831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d23ae0 0x563b02d25f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d95f2640 1 -- 172.21.15.94:0/3924857831 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b02c3af20 con 0x563b02d1df80 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d8df1640 1 --2- 172.21.15.94:0/3924857831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1e970 0x563b02d23470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d3fff640 1 --2- 172.21.15.94:0/3924857831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d23ae0 0x563b02d25f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08d95f2640 1 --2- 172.21.15.94:0/3924857831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1df80 0x563b02d1e3a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00cb20 tx=0x7f08cc0072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9964fec307144b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08baffd640 1 -- 172.21.15.94:0/3924857831 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc01a640 con 0x563b02d1df80 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08baffd640 1 -- 172.21.15.94:0/3924857831 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08cc00c560 con 0x563b02d1df80 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.478+0000 7f08baffd640 1 -- 172.21.15.94:0/3924857831 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc00a8e0 con 0x563b02d1df80 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.479+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3924857831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1df80 msgr2=0x563b02d1e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.479+0000 7f08d95f2640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08bc0a20c0 0x7f08bc0a4580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f08cc01f000 tx=0x7f08cc01b000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.479+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3924857831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1df80 0x563b02d1e3a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00cb20 tx=0x7f08cc0072b0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.479+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3924857831 shutdown_connections 2024-02-17T11:57:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.479+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3924857831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d23ae0 0x563b02d25f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.479+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3924857831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1e970 0x563b02d23470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.479+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3924857831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1df80 0x563b02d1e3a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.479+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3924857831 >> 172.21.15.94:0/3924857831 conn(0x563b02d194f0 msgr2=0x563b02d198f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.479+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3924857831 shutdown_connections 2024-02-17T11:57:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.479+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3924857831 wait complete. 2024-02-17T11:57:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.479+0000 7f08ddc2f740 1 -- start start 2024-02-17T11:57:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.479+0000 7f08ddc2f740 1 Processor -- start 2024-02-17T11:57:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08ddc2f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d1df80 0x563b02d28170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08ddc2f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1e970 0x563b02d286b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08ddc2f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d23ae0 0x563b02d2f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08ddc2f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02c85c30 con 0x563b02d23ae0 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08ddc2f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02c2ad30 con 0x563b02d1df80 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08ddc2f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02959310 con 0x563b02d1e970 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08ddc2f740 10 client.0 _ll_register_callbacks cb 0x563b02c32d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08d95f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d23ae0 0x563b02d2f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08d95f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d23ae0 0x563b02d2f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57812/0 (socket says 172.21.15.94:57812) 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08d95f2640 1 -- 172.21.15.94:0/4288415171 learned_addr learned my addr 172.21.15.94:0/4288415171 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08d8df1640 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d1df80 0x563b02d28170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08d3fff640 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1e970 0x563b02d286b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08d95f2640 1 -- 172.21.15.94:0/4288415171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1e970 msgr2=0x563b02d286b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08d95f2640 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1e970 0x563b02d286b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08d95f2640 1 -- 172.21.15.94:0/4288415171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d1df80 msgr2=0x563b02d28170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08d95f2640 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d1df80 0x563b02d28170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08d95f2640 1 -- 172.21.15.94:0/4288415171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08cc00c700 con 0x563b02d23ae0 2024-02-17T11:57:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08d3fff640 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1e970 0x563b02d286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08d8df1640 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d1df80 0x563b02d28170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08d95f2640 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d23ae0 0x563b02d2f6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f08cc007930 tx=0x7f08cc00a9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e8a8d61ba8a0a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.480+0000 7f08b3fff640 1 -- 172.21.15.94:0/4288415171 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc01a640 con 0x563b02d23ae0 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08b3fff640 1 -- 172.21.15.94:0/4288415171 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f08cc01a7a0 con 0x563b02d23ae0 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08ddc2f740 1 -- 172.21.15.94:0/4288415171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b029856c0 con 0x563b02d23ae0 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08b3fff640 1 -- 172.21.15.94:0/4288415171 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08cc01a960 con 0x563b02d23ae0 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08ddc2f740 10 client.15219 fetch_fsmap learned FSMap version 20 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08ddc2f740 1 -- 172.21.15.94:0/4288415171 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563b02c842b0 con 0x563b02d23ae0 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08b3fff640 1 -- 172.21.15.94:0/4288415171 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563b029856c0 con 0x563b02d23ae0 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08b3fff640 1 -- 172.21.15.94:0/4288415171 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f08cc008040 con 0x563b02d23ae0 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08ddc2f740 10 client.15219 fetch_fsmap finished waiting for FSMap version 20 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08ddc2f740 10 client.15219 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08ddc2f740 20 client.15219 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x563b02d34a40 0x563b02d33d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08ddc2f740 4 client.15219 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08ddc2f740 1 -- 172.21.15.94:0/4288415171 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b029d6270 con 0x563b02d34a40 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.481+0000 7f08d8df1640 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x563b02d34a40 0x563b02d33d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.482+0000 7f08b3fff640 10 client.15219 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.482+0000 7f08d8df1640 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x563b02d34a40 0x563b02d33d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d8a9e5ed91dba99 server_cookie=58929f2ca1d76b76 in_seq=0 out_seq=0 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.482+0000 7f08b3fff640 1 -- 172.21.15.94:0/4288415171 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b029d6270 con 0x563b02d34a40 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.482+0000 7f08b3fff640 10 client.15219 handle_command_reply: tid=0 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.483+0000 7f08ddc2f740 1 client.15219 shutdown 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.483+0000 7f08ddc2f740 1 -- 172.21.15.94:0/4288415171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d23ae0 msgr2=0x563b02d2f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.483+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d23ae0 0x563b02d2f6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f08cc007930 tx=0x7f08cc00a9d0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.483+0000 7f08ddc2f740 1 -- 172.21.15.94:0/4288415171 shutdown_connections 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.483+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x563b02d34a40 0x563b02d33d20 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.483+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d23ae0 0x563b02d2f6f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.483+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1e970 0x563b02d286b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.483+0000 7f08b3fff640 0 client.15219 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.483+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/4288415171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d1df80 0x563b02d28170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.483+0000 7f08ddc2f740 1 -- 172.21.15.94:0/4288415171 >> 172.21.15.94:0/4288415171 conn(0x563b02d194f0 msgr2=0x563b02d187d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.483+0000 7f08ddc2f740 1 -- 172.21.15.94:0/4288415171 shutdown_connections 2024-02-17T11:57:46.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.483+0000 7f08ddc2f740 1 -- 172.21.15.94:0/4288415171 wait complete. 2024-02-17T11:57:46.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.483+0000 7f08ddc2f740 20 client.15219 trim_cache size 0 max 16384 2024-02-17T11:57:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.498+0000 7f08ddc2f740 1 Processor -- start 2024-02-17T11:57:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.498+0000 7f08ddc2f740 1 -- start start 2024-02-17T11:57:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08ddc2f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d1df80 0x563b02d1fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08ddc2f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 0x563b02d2f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08ddc2f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d23ae0 0x563b02d203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08ddc2f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02c0b490 con 0x563b02d1e970 2024-02-17T11:57:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08ddc2f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02d19f40 con 0x563b02d1df80 2024-02-17T11:57:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08ddc2f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02d26630 con 0x563b02d23ae0 2024-02-17T11:57:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 0x563b02d2f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 0x563b02d2f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57814/0 (socket says 172.21.15.94:57814) 2024-02-17T11:57:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08d3fff640 1 -- 172.21.15.94:0/3699486295 learned_addr learned my addr 172.21.15.94:0/3699486295 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08d8df1640 1 --2- 172.21.15.94:0/3699486295 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d1df80 0x563b02d1fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08d95f2640 1 --2- 172.21.15.94:0/3699486295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d23ae0 0x563b02d203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08d3fff640 1 -- 172.21.15.94:0/3699486295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d23ae0 msgr2=0x563b02d203b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08d3fff640 1 --2- 172.21.15.94:0/3699486295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d23ae0 0x563b02d203b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08d3fff640 1 -- 172.21.15.94:0/3699486295 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d1df80 msgr2=0x563b02d1fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08d3fff640 1 --2- 172.21.15.94:0/3699486295 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d1df80 0x563b02d1fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08d3fff640 1 -- 172.21.15.94:0/3699486295 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b02c20a20 con 0x563b02d1e970 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08d95f2640 1 --2- 172.21.15.94:0/3699486295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d23ae0 0x563b02d203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08d3fff640 1 --2- 172.21.15.94:0/3699486295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 0x563b02d2f6b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f08c809a520 tx=0x7f08c80ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5464a01863042dbf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08ba7fc640 1 -- 172.21.15.94:0/3699486295 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c809e970 con 0x563b02d1e970 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08ba7fc640 1 -- 172.21.15.94:0/3699486295 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f08c80a2fc0 con 0x563b02d1e970 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.499+0000 7f08ba7fc640 1 -- 172.21.15.94:0/3699486295 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c809bb50 con 0x563b02d1e970 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3699486295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 msgr2=0x563b02d2f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3699486295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 0x563b02d2f6b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f08c809a520 tx=0x7f08c80ba040 comp rx=0 tx=0).stop 2024-02-17T11:57:46.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3699486295 shutdown_connections 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3699486295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d23ae0 0x563b02d203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3699486295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 0x563b02d2f6b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3699486295 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d1df80 0x563b02d1fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3699486295 >> 172.21.15.94:0/3699486295 conn(0x563b02d194f0 msgr2=0x563b02d3cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3699486295 shutdown_connections 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3699486295 wait complete. 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 -- start start 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 Processor -- start 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1df80 0x563b02d2c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 0x563b02d2caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d23ae0 0x563b02d8a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02d2d1a0 con 0x563b02d1e970 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02d8ab80 con 0x563b02d23ae0 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b02d8acc0 con 0x563b02d1df80 2024-02-17T11:57:46.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.500+0000 7f08ddc2f740 10 client.0 _ll_register_callbacks cb 0x7f08cc0097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 0x563b02d2caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 0x563b02d2caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57820/0 (socket says 172.21.15.94:57820) 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08d3fff640 1 -- 172.21.15.94:0/3913404290 learned_addr learned my addr 172.21.15.94:0/3913404290 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08d8df1640 1 --2- 172.21.15.94:0/3913404290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1df80 0x563b02d2c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08d3fff640 1 -- 172.21.15.94:0/3913404290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1df80 msgr2=0x563b02d2c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08d3fff640 1 --2- 172.21.15.94:0/3913404290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1df80 0x563b02d2c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08d3fff640 1 -- 172.21.15.94:0/3913404290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d23ae0 msgr2=0x563b02d8a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08d3fff640 1 --2- 172.21.15.94:0/3913404290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d23ae0 0x563b02d8a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08d3fff640 1 -- 172.21.15.94:0/3913404290 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d419f960 con 0x563b02d1e970 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08d8df1640 1 --2- 172.21.15.94:0/3913404290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1df80 0x563b02d2c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08d3fff640 1 --2- 172.21.15.94:0/3913404290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 0x563b02d2caa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f08c8096c80 tx=0x7f08c80cb590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980d2dabb2309f3b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08b3fff640 1 -- 172.21.15.94:0/3913404290 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8097070 con 0x563b02d1e970 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08b3fff640 1 -- 172.21.15.94:0/3913404290 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f08c809e970 con 0x563b02d1e970 2024-02-17T11:57:46.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3913404290 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b02d34750 con 0x563b02d1e970 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08b3fff640 1 -- 172.21.15.94:0/3913404290 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08c80850b0 con 0x563b02d1e970 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08ddc2f740 10 client.15225 fetch_fsmap learned FSMap version 20 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3913404290 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x563b02c46780 con 0x563b02d1e970 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08b3fff640 1 -- 172.21.15.94:0/3913404290 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563b02d34750 con 0x563b02d1e970 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08b3fff640 1 -- 172.21.15.94:0/3913404290 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f08c8099c70 con 0x563b02d1e970 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.501+0000 7f08ddc2f740 10 client.15225 fetch_fsmap finished waiting for FSMap version 20 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.502+0000 7f08ddc2f740 10 client.15225 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.502+0000 7f08ddc2f740 20 client.15225 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.502+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3913404290 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x563b02d8fbd0 0x563b02d8c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.502+0000 7f08ddc2f740 4 client.15225 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.502+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3913404290 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f08cc00c560 con 0x563b02d8fbd0 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.502+0000 7f08d8df1640 1 --2- 172.21.15.94:0/3913404290 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x563b02d8fbd0 0x563b02d8c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.502+0000 7f08b3fff640 10 client.15225 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T11:57:46.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.502+0000 7f08d8df1640 1 --2- 172.21.15.94:0/3913404290 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x563b02d8fbd0 0x563b02d8c010 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eed7a7dee5416a97 server_cookie=c4c2936ef1b3c567 in_seq=0 out_seq=0 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08b3fff640 1 -- 172.21.15.94:0/3913404290 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x7f08cc00c560 con 0x563b02d8fbd0 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08b3fff640 10 client.15225 handle_command_reply: tid=0 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08ddc2f740 1 client.15225 shutdown 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3913404290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 msgr2=0x563b02d2caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3913404290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 0x563b02d2caa0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f08c8096c80 tx=0x7f08c80cb590 comp rx=0 tx=0).stop 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3913404290 shutdown_connections 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3913404290 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x563b02d8fbd0 0x563b02d8c010 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3913404290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b02d23ae0 0x563b02d8a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08b3fff640 0 client.15225 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3913404290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b02d1e970 0x563b02d2caa0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08ddc2f740 1 --2- 172.21.15.94:0/3913404290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b02d1df80 0x563b02d2c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3913404290 >> 172.21.15.94:0/3913404290 conn(0x563b02d194f0 msgr2=0x563b02d1fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3913404290 shutdown_connections 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08ddc2f740 1 -- 172.21.15.94:0/3913404290 wait complete. 2024-02-17T11:57:46.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08ddc2f740 20 client.15225 trim_cache size 0 max 16384 2024-02-17T11:57:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:57:46.509 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "3bc5cfc8-cd8a-11ee-95ba-87774f69a715", 2024-02-17T11:57:46.509 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 1, 2024-02-17T11:57:46.509 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24487, 2024-02-17T11:57:46.509 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-02-17T11:57:46.509 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-02-17T11:57:46.509 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-02-17T11:57:46.509 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 106.370763041, 2024-02-17T11:57:46.509 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 20, 2024-02-17T11:57:46.509 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 87, 2024-02-17T11:57:46.509 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 85, 2024-02-17T11:57:46.509 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 122.546142104 2024-02-17T11:57:46.509 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:57:46.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08db055640 1 -- 172.21.15.94:0/4022337451 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08bc0a20c0 msgr2=0x7f08bc0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08db055640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08bc0a20c0 0x7f08bc0a4580 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f08cc01f000 tx=0x7f08cc01b000 comp rx=0 tx=0).stop 2024-02-17T11:57:46.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08db055640 1 -- 172.21.15.94:0/4022337451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d4104530 msgr2=0x7f08d4197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08db055640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d4104530 0x7f08d4197810 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f08c800d770 tx=0x7f08c800dc30 comp rx=0 tx=0).stop 2024-02-17T11:57:46.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08db055640 1 -- 172.21.15.94:0/4022337451 shutdown_connections 2024-02-17T11:57:46.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08db055640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f08bc0a20c0 0x7f08bc0a4580 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08db055640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f08d4106e60 0x7f08d419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08db055640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f08d4104530 0x7f08d4197810 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.503+0000 7f08db055640 1 --2- 172.21.15.94:0/4022337451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d4101c00 0x7f08d41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.504+0000 7f08db055640 1 -- 172.21.15.94:0/4022337451 >> 172.21.15.94:0/4022337451 conn(0x7f08d40fb610 msgr2=0x7f08d4102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.504+0000 7f08db055640 1 -- 172.21.15.94:0/4022337451 shutdown_connections 2024-02-17T11:57:46.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.504+0000 7f08db055640 1 -- 172.21.15.94:0/4022337451 wait complete. 2024-02-17T11:57:46.513 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-17T11:57:46.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 -- 172.21.15.94:0/3115465227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7850102d60 msgr2=0x7f78501095f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 --2- 172.21.15.94:0/3115465227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7850102d60 0x7f78501095f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7840009900 tx=0x7f784002bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 -- 172.21.15.94:0/3115465227 shutdown_connections 2024-02-17T11:57:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 --2- 172.21.15.94:0/3115465227 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7850102d60 0x7f78501095f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 --2- 172.21.15.94:0/3115465227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78501023a0 0x7f7850102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 --2- 172.21.15.94:0/3115465227 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f78501011b0 0x7f78501015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 -- 172.21.15.94:0/3115465227 >> 172.21.15.94:0/3115465227 conn(0x7f78500fc720 msgr2=0x7f78500feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 -- 172.21.15.94:0/3115465227 shutdown_connections 2024-02-17T11:57:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 -- 172.21.15.94:0/3115465227 wait complete. 2024-02-17T11:57:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 Processor -- start 2024-02-17T11:57:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 -- start start 2024-02-17T11:57:46.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501011b0 0x7f7850197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78501023a0 0x7f78501977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7850102d60 0x7f785019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785019ed60 con 0x7f78501011b0 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785019eed0 con 0x7f7850102d60 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.617+0000 7f7856c64640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785019f090 con 0x7f78501023a0 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f7854a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501011b0 0x7f7850197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f7854a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501011b0 0x7f7850197270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57834/0 (socket says 172.21.15.94:57834) 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f7854a00640 1 -- 172.21.15.94:0/693849372 learned_addr learned my addr 172.21.15.94:0/693849372 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f7855201640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7850102d60 0x7f785019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f7854a00640 1 -- 172.21.15.94:0/693849372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78501023a0 msgr2=0x7f78501977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f7854a00640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78501023a0 0x7f78501977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f7854a00640 1 -- 172.21.15.94:0/693849372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7850102d60 msgr2=0x7f785019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f7854a00640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7850102d60 0x7f785019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f7854a00640 1 -- 172.21.15.94:0/693849372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7840009560 con 0x7f78501011b0 2024-02-17T11:57:46.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f7855201640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7850102d60 0x7f785019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:46.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f7854a00640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501011b0 0x7f7850197270 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f784800ea80 tx=0x7f784800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f784dffb640 1 -- 172.21.15.94:0/693849372 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784800ce40 con 0x7f78501011b0 2024-02-17T11:57:46.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f7856c64640 1 -- 172.21.15.94:0/693849372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785019f370 con 0x7f78501011b0 2024-02-17T11:57:46.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f784dffb640 1 -- 172.21.15.94:0/693849372 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7848011950 con 0x7f78501011b0 2024-02-17T11:57:46.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f784dffb640 1 -- 172.21.15.94:0/693849372 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784801ab40 con 0x7f78501011b0 2024-02-17T11:57:46.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.618+0000 7f7856c64640 1 -- 172.21.15.94:0/693849372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785019f880 con 0x7f78501011b0 2024-02-17T11:57:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.620+0000 7f784dffb640 1 -- 172.21.15.94:0/693849372 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f7848011ac0 con 0x7f78501011b0 2024-02-17T11:57:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f7859838740 1 Processor -- start 2024-02-17T11:57:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f7859838740 1 -- start start 2024-02-17T11:57:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f7859838740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4aff80 0x555a0d4b03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f7859838740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b0970 0x555a0d4b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f7859838740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4b5ae0 0x555a0d4b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f7859838740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0d46bcd0 con 0x555a0d4b5ae0 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f7859838740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0d41ef20 con 0x555a0d4b0970 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f784dffb640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78380a20c0 0x7f78380a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f7859838740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0d41c450 con 0x555a0d4aff80 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f784ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4b5ae0 0x555a0d4b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f7855201640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78380a20c0 0x7f78380a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f784ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4b5ae0 0x555a0d4b7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57850/0 (socket says 172.21.15.94:57850) 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f784ffff640 1 -- 172.21.15.94:0/3887933604 learned_addr learned my addr 172.21.15.94:0/3887933604 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f784dffb640 1 -- 172.21.15.94:0/693849372 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f78480ce2e0 con 0x7f78501011b0 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f7855201640 1 --2- 172.21.15.94:0/3887933604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4aff80 0x555a0d4b03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f784ffff640 1 -- 172.21.15.94:0/3887933604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4aff80 msgr2=0x555a0d4b03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f784ffff640 1 --2- 172.21.15.94:0/3887933604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4aff80 0x555a0d4b03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f784ffff640 1 -- 172.21.15.94:0/3887933604 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b0970 msgr2=0x555a0d4b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f784ffff640 1 --2- 172.21.15.94:0/3887933604 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b0970 0x555a0d4b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.621+0000 7f784ffff640 1 -- 172.21.15.94:0/3887933604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a0d3ccf20 con 0x555a0d4b5ae0 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.622+0000 7f7855201640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78380a20c0 0x7f78380a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f784002c840 tx=0x7f7840005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.622+0000 7f784ffff640 1 --2- 172.21.15.94:0/3887933604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4b5ae0 0x555a0d4b7f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7844009be0 tx=0x7f784400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b076122f6e564c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.622+0000 7f7836ffd640 1 -- 172.21.15.94:0/3887933604 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844009730 con 0x555a0d4b5ae0 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.622+0000 7f7836ffd640 1 -- 172.21.15.94:0/3887933604 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7844005500 con 0x555a0d4b5ae0 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.622+0000 7f7836ffd640 1 -- 172.21.15.94:0/3887933604 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844010690 con 0x555a0d4b5ae0 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.622+0000 7f7859838740 1 -- 172.21.15.94:0/3887933604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4b5ae0 msgr2=0x555a0d4b7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.622+0000 7f7859838740 1 --2- 172.21.15.94:0/3887933604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4b5ae0 0x555a0d4b7f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7844009be0 tx=0x7f784400e720 comp rx=0 tx=0).stop 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.622+0000 7f7859838740 1 -- 172.21.15.94:0/3887933604 shutdown_connections 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.622+0000 7f7859838740 1 --2- 172.21.15.94:0/3887933604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4b5ae0 0x555a0d4b7f70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.622+0000 7f7859838740 1 --2- 172.21.15.94:0/3887933604 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b0970 0x555a0d4b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.622+0000 7f7859838740 1 --2- 172.21.15.94:0/3887933604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4aff80 0x555a0d4b03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.622+0000 7f7859838740 1 -- 172.21.15.94:0/3887933604 >> 172.21.15.94:0/3887933604 conn(0x555a0d4ab4f0 msgr2=0x555a0d4ab8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.622+0000 7f7859838740 1 -- 172.21.15.94:0/3887933604 shutdown_connections 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.623+0000 7f7859838740 1 -- 172.21.15.94:0/3887933604 wait complete. 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.623+0000 7f7859838740 1 -- start start 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.623+0000 7f7859838740 1 Processor -- start 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.623+0000 7f7859838740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4ba190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.623+0000 7f7859838740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b0970 0x555a0d4ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.623+0000 7f7859838740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b5ae0 0x555a0d4c1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.623+0000 7f7859838740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0d417c30 con 0x555a0d4aff80 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.623+0000 7f7859838740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0d3bcd30 con 0x555a0d4b0970 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.623+0000 7f7859838740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0d0eb310 con 0x555a0d4b5ae0 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.623+0000 7f7854a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4ba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.623+0000 7f7859838740 10 client.0 _ll_register_callbacks cb 0x555a0d3c4d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.623+0000 7f7854a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4ba190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57864/0 (socket says 172.21.15.94:57864) 2024-02-17T11:57:46.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.623+0000 7f7854a00640 1 -- 172.21.15.94:0/406383146 learned_addr learned my addr 172.21.15.94:0/406383146 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7855201640 1 --2- 172.21.15.94:0/406383146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b5ae0 0x555a0d4c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7854a00640 1 -- 172.21.15.94:0/406383146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b5ae0 msgr2=0x555a0d4c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7854a00640 1 --2- 172.21.15.94:0/406383146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b5ae0 0x555a0d4c1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7854a00640 1 -- 172.21.15.94:0/406383146 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b0970 msgr2=0x555a0d4ba6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7854a00640 1 --2- 172.21.15.94:0/406383146 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b0970 0x555a0d4ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7854a00640 1 -- 172.21.15.94:0/406383146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7844009890 con 0x555a0d4aff80 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7855201640 1 --2- 172.21.15.94:0/406383146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b5ae0 0x555a0d4c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7854a00640 1 --2- 172.21.15.94:0/406383146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4ba190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f78480979e0 tx=0x7f784809ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc289e594cc8b2a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f782ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f782ffff640 1 -- 172.21.15.94:0/406383146 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784801aea0 con 0x555a0d4aff80 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f782ffff640 1 -- 172.21.15.94:0/406383146 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f784809f160 con 0x555a0d4aff80 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7859838740 1 -- 172.21.15.94:0/406383146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a0d1176c0 con 0x555a0d4aff80 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f782ffff640 1 -- 172.21.15.94:0/406383146 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78480a37b0 con 0x555a0d4aff80 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7859838740 10 client.15237 fetch_fsmap learned FSMap version 20 2024-02-17T11:57:46.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7859838740 1 -- 172.21.15.94:0/406383146 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555a0d4162b0 con 0x555a0d4aff80 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f782ffff640 1 -- 172.21.15.94:0/406383146 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555a0d1176c0 con 0x555a0d4aff80 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f782ffff640 1 -- 172.21.15.94:0/406383146 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7848018020 con 0x555a0d4aff80 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7859838740 10 client.15237 fetch_fsmap finished waiting for FSMap version 20 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7859838740 10 client.15237 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7859838740 20 client.15237 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7859838740 1 --2- 172.21.15.94:0/406383146 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a0d4c6aa0 0x555a0d4caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7859838740 4 client.15237 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.624+0000 7f7859838740 1 -- 172.21.15.94:0/406383146 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a0d168270 con 0x555a0d4c6aa0 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.625+0000 7f784ffff640 1 --2- 172.21.15.94:0/406383146 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a0d4c6aa0 0x555a0d4caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.625+0000 7f782ffff640 10 client.15237 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.625+0000 7f784ffff640 1 --2- 172.21.15.94:0/406383146 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a0d4c6aa0 0x555a0d4caf20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3faf61eaa7ce538e server_cookie=d1d6f125a58b4d12 in_seq=0 out_seq=0 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f782ffff640 1 -- 172.21.15.94:0/406383146 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a0d168270 con 0x555a0d4c6aa0 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f782ffff640 10 client.15237 handle_command_reply: tid=0 2024-02-17T11:57:46.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f7859838740 1 client.15237 shutdown 2024-02-17T11:57:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f7859838740 1 -- 172.21.15.94:0/406383146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 msgr2=0x555a0d4ba190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f7859838740 1 --2- 172.21.15.94:0/406383146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4ba190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f78480979e0 tx=0x7f784809ac50 comp rx=0 tx=0).stop 2024-02-17T11:57:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f7859838740 1 -- 172.21.15.94:0/406383146 shutdown_connections 2024-02-17T11:57:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f7859838740 1 --2- 172.21.15.94:0/406383146 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a0d4c6aa0 0x555a0d4caf20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f7859838740 1 --2- 172.21.15.94:0/406383146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b5ae0 0x555a0d4c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f7859838740 1 --2- 172.21.15.94:0/406383146 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b0970 0x555a0d4ba6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f782ffff640 0 client.15237 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f7859838740 1 --2- 172.21.15.94:0/406383146 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4ba190 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f7859838740 1 -- 172.21.15.94:0/406383146 >> 172.21.15.94:0/406383146 conn(0x555a0d4ab4f0 msgr2=0x555a0d4aa7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f7859838740 1 -- 172.21.15.94:0/406383146 shutdown_connections 2024-02-17T11:57:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f7859838740 1 -- 172.21.15.94:0/406383146 wait complete. 2024-02-17T11:57:46.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.626+0000 7f7859838740 20 client.15237 trim_cache size 0 max 16384 2024-02-17T11:57:46.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.641+0000 7f7859838740 1 Processor -- start 2024-02-17T11:57:46.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.641+0000 7f7859838740 1 -- start start 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7859838740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7859838740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b0970 0x555a0d4c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7859838740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b5ae0 0x555a0d4b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7859838740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0d39d490 con 0x555a0d4aff80 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7859838740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0d4b8630 con 0x555a0d4b0970 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7859838740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0d168270 con 0x555a0d4b5ae0 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7854a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7854a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4b7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57878/0 (socket says 172.21.15.94:57878) 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7854a00640 1 -- 172.21.15.94:0/2459066437 learned_addr learned my addr 172.21.15.94:0/2459066437 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7854a00640 1 -- 172.21.15.94:0/2459066437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b5ae0 msgr2=0x555a0d4b9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7855201640 1 --2- 172.21.15.94:0/2459066437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b5ae0 0x555a0d4b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7854a00640 1 --2- 172.21.15.94:0/2459066437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b5ae0 0x555a0d4b9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f784ffff640 1 --2- 172.21.15.94:0/2459066437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b0970 0x555a0d4c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7854a00640 1 -- 172.21.15.94:0/2459066437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b0970 msgr2=0x555a0d4c16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7854a00640 1 --2- 172.21.15.94:0/2459066437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b0970 0x555a0d4c16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7854a00640 1 -- 172.21.15.94:0/2459066437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a0d3b2a20 con 0x555a0d4aff80 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7855201640 1 --2- 172.21.15.94:0/2459066437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b5ae0 0x555a0d4b9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f7854a00640 1 --2- 172.21.15.94:0/2459066437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4b7ed0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f784809ad10 tx=0x7f784801a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17f96ea391eb49c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f78367fc640 1 -- 172.21.15.94:0/2459066437 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784809f160 con 0x555a0d4aff80 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f78367fc640 1 -- 172.21.15.94:0/2459066437 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78480a37b0 con 0x555a0d4aff80 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.642+0000 7f78367fc640 1 -- 172.21.15.94:0/2459066437 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784809a6c0 con 0x555a0d4aff80 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.643+0000 7f7859838740 1 -- 172.21.15.94:0/2459066437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 msgr2=0x555a0d4b7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.643+0000 7f7859838740 1 --2- 172.21.15.94:0/2459066437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4b7ed0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f784809ad10 tx=0x7f784801a5a0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.643+0000 7f7859838740 1 -- 172.21.15.94:0/2459066437 shutdown_connections 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.643+0000 7f7859838740 1 --2- 172.21.15.94:0/2459066437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b5ae0 0x555a0d4b9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.643+0000 7f7859838740 1 --2- 172.21.15.94:0/2459066437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b0970 0x555a0d4c16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.643+0000 7f7859838740 1 --2- 172.21.15.94:0/2459066437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4b7ed0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.643+0000 7f7859838740 1 -- 172.21.15.94:0/2459066437 >> 172.21.15.94:0/2459066437 conn(0x555a0d4ab4f0 msgr2=0x555a0d4b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.643+0000 7f7859838740 1 -- 172.21.15.94:0/2459066437 shutdown_connections 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.643+0000 7f7859838740 1 -- 172.21.15.94:0/2459066437 wait complete. 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.643+0000 7f7859838740 1 -- start start 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.643+0000 7f7859838740 1 Processor -- start 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7859838740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4c1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7859838740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b0970 0x555a0d4c1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7859838740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b5ae0 0x555a0d4bdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7859838740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0d4be2d0 con 0x555a0d4aff80 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7859838740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0d4be440 con 0x555a0d4b5ae0 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7859838740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a0d4be600 con 0x555a0d4b0970 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7859838740 10 client.0 _ll_register_callbacks cb 0x7f7844009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7854a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4c1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7854a00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4c1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57882/0 (socket says 172.21.15.94:57882) 2024-02-17T11:57:46.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7854a00640 1 -- 172.21.15.94:0/2446284793 learned_addr learned my addr 172.21.15.94:0/2446284793 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:46.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7854a00640 1 -- 172.21.15.94:0/2446284793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b0970 msgr2=0x555a0d4c1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7854a00640 1 --2- 172.21.15.94:0/2446284793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b0970 0x555a0d4c1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7854a00640 1 -- 172.21.15.94:0/2446284793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b5ae0 msgr2=0x555a0d4bdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:46.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7855201640 1 --2- 172.21.15.94:0/2446284793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b5ae0 0x555a0d4bdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7854a00640 1 --2- 172.21.15.94:0/2446284793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b5ae0 0x555a0d4bdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7854a00640 1 -- 172.21.15.94:0/2446284793 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a0d4162b0 con 0x555a0d4aff80 2024-02-17T11:57:46.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7855201640 1 --2- 172.21.15.94:0/2446284793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b5ae0 0x555a0d4bdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:46.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7854a00640 1 --2- 172.21.15.94:0/2446284793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4c1110 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f78480ce5d0 tx=0x7f7848095db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0d1ee61e9cf9a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:46.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f782ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:46.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f782ffff640 1 -- 172.21.15.94:0/2446284793 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f784809ab50 con 0x555a0d4aff80 2024-02-17T11:57:46.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f782ffff640 1 -- 172.21.15.94:0/2446284793 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f784809f160 con 0x555a0d4aff80 2024-02-17T11:57:46.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7859838740 1 -- 172.21.15.94:0/2446284793 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a0d4a6b10 con 0x555a0d4aff80 2024-02-17T11:57:46.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f782ffff640 1 -- 172.21.15.94:0/2446284793 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7848011470 con 0x555a0d4aff80 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7859838740 10 client.15243 fetch_fsmap learned FSMap version 20 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f7859838740 1 -- 172.21.15.94:0/2446284793 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) v3 -- 0x555a0d3d8780 con 0x555a0d4aff80 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.644+0000 7f782ffff640 1 -- 172.21.15.94:0/2446284793 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555a0d4a6b10 con 0x555a0d4aff80 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.645+0000 7f782ffff640 1 -- 172.21.15.94:0/2446284793 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 20) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7848018050 con 0x555a0d4aff80 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.645+0000 7f7859838740 10 client.15243 fetch_fsmap finished waiting for FSMap version 20 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.645+0000 7f7859838740 10 client.15243 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.645+0000 7f7859838740 20 client.15243 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.645+0000 7f7859838740 1 --2- 172.21.15.94:0/2446284793 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a0d521520 0x555a0d51da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.645+0000 7f7859838740 4 client.15243 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.645+0000 7f7859838740 1 -- 172.21.15.94:0/2446284793 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f7844005500 con 0x555a0d521520 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.645+0000 7f784ffff640 1 --2- 172.21.15.94:0/2446284793 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a0d521520 0x555a0d51da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.645+0000 7f784ffff640 1 --2- 172.21.15.94:0/2446284793 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a0d521520 0x555a0d51da80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41477d38f3bf6563 server_cookie=bd2acc8b150d520e in_seq=0 out_seq=0 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.645+0000 7f782ffff640 10 client.15243 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.645+0000 7f782ffff640 1 -- 172.21.15.94:0/2446284793 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x7f7844005500 con 0x555a0d521520 2024-02-17T11:57:46.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.645+0000 7f782ffff640 10 client.15243 handle_command_reply: tid=0 2024-02-17T11:57:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.645+0000 7f7859838740 1 client.15243 shutdown 2024-02-17T11:57:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7859838740 1 -- 172.21.15.94:0/2446284793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 msgr2=0x555a0d4c1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7859838740 1 --2- 172.21.15.94:0/2446284793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4c1110 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f78480ce5d0 tx=0x7f7848095db0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7859838740 1 -- 172.21.15.94:0/2446284793 shutdown_connections 2024-02-17T11:57:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7859838740 1 --2- 172.21.15.94:0/2446284793 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a0d521520 0x555a0d51da80 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7859838740 1 --2- 172.21.15.94:0/2446284793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a0d4b5ae0 0x555a0d4bdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7859838740 1 --2- 172.21.15.94:0/2446284793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a0d4b0970 0x555a0d4c1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7859838740 1 --2- 172.21.15.94:0/2446284793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a0d4aff80 0x555a0d4c1110 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f782ffff640 0 client.15243 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7859838740 1 -- 172.21.15.94:0/2446284793 >> 172.21.15.94:0/2446284793 conn(0x555a0d4ab4f0 msgr2=0x555a0d4b9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7859838740 1 -- 172.21.15.94:0/2446284793 shutdown_connections 2024-02-17T11:57:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7859838740 1 -- 172.21.15.94:0/2446284793 wait complete. 2024-02-17T11:57:46.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7859838740 20 client.15243 trim_cache size 0 max 16384 2024-02-17T11:57:46.652 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:57:46.652 INFO:teuthology.orchestra.run.smithi094.stdout: "cluster_fsid": "3bc5cfc8-cd8a-11ee-95ba-87774f69a715", 2024-02-17T11:57:46.652 INFO:teuthology.orchestra.run.smithi094.stdout: "whoami": 0, 2024-02-17T11:57:46.652 INFO:teuthology.orchestra.run.smithi094.stdout: "id": 24488, 2024-02-17T11:57:46.652 INFO:teuthology.orchestra.run.smithi094.stdout: "want_state": "up:active", 2024-02-17T11:57:46.652 INFO:teuthology.orchestra.run.smithi094.stdout: "state": "up:active", 2024-02-17T11:57:46.652 INFO:teuthology.orchestra.run.smithi094.stdout: "fs_name": "cephfs", 2024-02-17T11:57:46.652 INFO:teuthology.orchestra.run.smithi094.stdout: "rank_uptime": 107.990413022, 2024-02-17T11:57:46.652 INFO:teuthology.orchestra.run.smithi094.stdout: "mdsmap_epoch": 20, 2024-02-17T11:57:46.652 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch": 87, 2024-02-17T11:57:46.652 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_epoch_barrier": 85, 2024-02-17T11:57:46.652 INFO:teuthology.orchestra.run.smithi094.stdout: "uptime": 114.301258515 2024-02-17T11:57:46.652 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:57:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7856c64640 1 -- 172.21.15.94:0/693849372 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78380a20c0 msgr2=0x7f78380a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7856c64640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78380a20c0 0x7f78380a4580 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f784002c840 tx=0x7f7840005a20 comp rx=0 tx=0).stop 2024-02-17T11:57:46.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7856c64640 1 -- 172.21.15.94:0/693849372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501011b0 msgr2=0x7f7850197270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7856c64640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501011b0 0x7f7850197270 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f784800ea80 tx=0x7f784800ef40 comp rx=0 tx=0).stop 2024-02-17T11:57:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7856c64640 1 -- 172.21.15.94:0/693849372 shutdown_connections 2024-02-17T11:57:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7856c64640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f78380a20c0 0x7f78380a4580 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7856c64640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7850102d60 0x7f785019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7856c64640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f78501023a0 0x7f78501977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7856c64640 1 --2- 172.21.15.94:0/693849372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f78501011b0 0x7f7850197270 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7856c64640 1 -- 172.21.15.94:0/693849372 >> 172.21.15.94:0/693849372 conn(0x7f78500fc720 msgr2=0x7f78501056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7856c64640 1 -- 172.21.15.94:0/693849372 shutdown_connections 2024-02-17T11:57:46.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:46.646+0000 7f7856c64640 1 -- 172.21.15.94:0/693849372 wait complete. 2024-02-17T11:57:46.656 INFO:teuthology.run_tasks:Running task kclient... 2024-02-17T11:57:46.677 INFO:tasks.kclient:Mounting kernel clients... 2024-02-17T11:57:46.677 INFO:tasks.kclient:config is {'client.0': {}} 2024-02-17T11:57:46.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:46.678 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-02-17T11:57:46.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:46.693 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-02-17T11:57:46.762 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-02-17T11:57:46.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:46 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1840364333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:46.762 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:46 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1676651250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:46.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T11:57:46.763 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v2', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-02-17T11:57:46.763 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-17T11:57:46.763 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-02-17T11:57:46.764 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-02-17T11:57:46.764 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph fs ls 2024-02-17T11:57:47.044 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T11:57:47.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:46 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1840364333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:47.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:46 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1676651250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:47.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:46 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1840364333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:47.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:46 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1676651250' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:48.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:47 smithi189 ceph-mon[36588]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-02-17T11:57:48.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:47 smithi125 ceph-mon[36819]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-02-17T11:57:48.190 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:47 smithi094 ceph-mon[32499]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-02-17T11:57:48.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.885+0000 7fae8ed8c700 1 -- 172.21.15.94:0/3682912963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae88100810 msgr2=0x7fae88100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:48.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.885+0000 7fae8ed8c700 1 --2- 172.21.15.94:0/3682912963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae88100810 0x7fae88100bf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009990 tx=0x7fae7c009ca0 comp rx=0 tx=0).stop 2024-02-17T11:57:48.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.885+0000 7fae8ed8c700 1 -- 172.21.15.94:0/3682912963 shutdown_connections 2024-02-17T11:57:48.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.885+0000 7fae8ed8c700 1 --2- 172.21.15.94:0/3682912963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae88107030 0x7fae8810ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:48.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.885+0000 7fae8ed8c700 1 --2- 172.21.15.94:0/3682912963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae88100810 0x7fae88100bf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:48.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.885+0000 7fae8ed8c700 1 --2- 172.21.15.94:0/3682912963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae881011c0 0x7fae88106900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:48.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.885+0000 7fae8ed8c700 1 -- 172.21.15.94:0/3682912963 >> 172.21.15.94:0/3682912963 conn(0x7fae88075ed0 msgr2=0x7fae880762e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:48.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.886+0000 7fae8ed8c700 1 -- 172.21.15.94:0/3682912963 shutdown_connections 2024-02-17T11:57:48.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.886+0000 7fae8ed8c700 1 -- 172.21.15.94:0/3682912963 wait complete. 2024-02-17T11:57:48.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.886+0000 7fae8ed8c700 1 Processor -- start 2024-02-17T11:57:48.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.886+0000 7fae8ed8c700 1 -- start start 2024-02-17T11:57:48.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae8ed8c700 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae88100810 0x7fae88101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:48.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae8ed8c700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae881011c0 0x7fae88101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae8ed8c700 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae88107030 0x7fae88102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae8ed8c700 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae881a67a0 con 0x7fae881011c0 2024-02-17T11:57:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae8ed8c700 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae881a6910 con 0x7fae88107030 2024-02-17T11:57:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae8ed8c700 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae881a6ad0 con 0x7fae88100810 2024-02-17T11:57:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae87fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae881011c0 0x7fae88101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae87fff700 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae881011c0 0x7fae88101cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57906/0 (socket says 172.21.15.94:57906) 2024-02-17T11:57:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae87fff700 1 -- 172.21.15.94:0/1625781540 learned_addr learned my addr 172.21.15.94:0/1625781540 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae8cb28700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae88100810 0x7fae88101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae8d329700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae88107030 0x7fae88102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae87fff700 1 -- 172.21.15.94:0/1625781540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae88100810 msgr2=0x7fae88101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae87fff700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae88100810 0x7fae88101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae87fff700 1 -- 172.21.15.94:0/1625781540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae88107030 msgr2=0x7fae88102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae87fff700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae88107030 0x7fae88102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae87fff700 1 -- 172.21.15.94:0/1625781540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c0095f0 con 0x7fae881011c0 2024-02-17T11:57:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae8d329700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae88107030 0x7fae88102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae87fff700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae881011c0 0x7fae88101cd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fae7400b4d0 tx=0x7fae7400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae85ffb700 1 -- 172.21.15.94:0/1625781540 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae74011830 con 0x7fae881011c0 2024-02-17T11:57:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae8ed8c700 1 -- 172.21.15.94:0/1625781540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae881a6db0 con 0x7fae881011c0 2024-02-17T11:57:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.887+0000 7fae85ffb700 1 -- 172.21.15.94:0/1625781540 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae74011e70 con 0x7fae881011c0 2024-02-17T11:57:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.888+0000 7fae85ffb700 1 -- 172.21.15.94:0/1625781540 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7400f2e0 con 0x7fae881011c0 2024-02-17T11:57:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.888+0000 7fae8ed8c700 1 -- 172.21.15.94:0/1625781540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae881a7220 con 0x7fae881011c0 2024-02-17T11:57:48.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.889+0000 7fae8ed8c700 1 -- 172.21.15.94:0/1625781540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8810a290 con 0x7fae881011c0 2024-02-17T11:57:48.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.890+0000 7fae85ffb700 1 -- 172.21.15.94:0/1625781540 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fae740105a0 con 0x7fae881011c0 2024-02-17T11:57:48.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.890+0000 7fae85ffb700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fae780a2150 0x7fae780a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:48.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.891+0000 7fae8cb28700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fae780a2150 0x7fae780a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:48.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.891+0000 7fae85ffb700 1 -- 172.21.15.94:0/1625781540 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fae740cdad0 con 0x7fae881011c0 2024-02-17T11:57:48.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.891+0000 7fae8cb28700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fae780a2150 0x7fae780a4610 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fae7c003800 tx=0x7fae7c00b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:48.894+0000 7fae85ffb700 1 -- 172.21.15.94:0/1625781540 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae74096aa0 con 0x7fae881011c0 2024-02-17T11:57:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.132+0000 7fae8ed8c700 1 -- 172.21.15.94:0/1625781540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fae88068930 con 0x7fae881011c0 2024-02-17T11:57:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.133+0000 7fae85ffb700 1 -- 172.21.15.94:0/1625781540 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+73 (secure 0 0 0) 0x7fae7409a160 con 0x7fae881011c0 2024-02-17T11:57:49.132 INFO:teuthology.orchestra.run.smithi094.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-02-17T11:57:49.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.135+0000 7fae8ed8c700 1 -- 172.21.15.94:0/1625781540 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fae780a2150 msgr2=0x7fae780a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.135+0000 7fae8ed8c700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fae780a2150 0x7fae780a4610 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fae7c003800 tx=0x7fae7c00b560 comp rx=0 tx=0).stop 2024-02-17T11:57:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.135+0000 7fae8ed8c700 1 -- 172.21.15.94:0/1625781540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae881011c0 msgr2=0x7fae88101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.135+0000 7fae8ed8c700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae881011c0 0x7fae88101cd0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fae7400b4d0 tx=0x7fae7400b7e0 comp rx=0 tx=0).stop 2024-02-17T11:57:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.135+0000 7fae8ed8c700 1 -- 172.21.15.94:0/1625781540 shutdown_connections 2024-02-17T11:57:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.135+0000 7fae8ed8c700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fae780a2150 0x7fae780a4610 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.135+0000 7fae8ed8c700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fae88107030 0x7fae88102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.135+0000 7fae8ed8c700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae88100810 0x7fae88101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.135+0000 7fae8ed8c700 1 --2- 172.21.15.94:0/1625781540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fae881011c0 0x7fae88101cd0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.135+0000 7fae8ed8c700 1 -- 172.21.15.94:0/1625781540 >> 172.21.15.94:0/1625781540 conn(0x7fae88075ed0 msgr2=0x7fae880fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.135+0000 7fae8ed8c700 1 -- 172.21.15.94:0/1625781540 shutdown_connections 2024-02-17T11:57:49.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:49.135+0000 7fae8ed8c700 1 -- 172.21.15.94:0/1625781540 wait complete. 2024-02-17T11:57:49.937 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-17T11:57:49.937 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-17T11:57:49.937 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi094.front.sepia.ceph.com 2024-02-17T11:57:49.937 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-17T11:57:49.937 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-02-17T11:57:49.937 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-17T11:57:49.937 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-17T11:57:49.937 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-17T11:57:49.937 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-02-17T11:57:49.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:49.937 DEBUG:teuthology.orchestra.run.smithi094:> ip addr 2024-02-17T11:57:49.957 INFO:teuthology.orchestra.run.smithi094.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-17T11:57:49.957 INFO:teuthology.orchestra.run.smithi094.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-17T11:57:49.957 INFO:teuthology.orchestra.run.smithi094.stdout: inet 127.0.0.1/8 scope host lo 2024-02-17T11:57:49.957 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-02-17T11:57:49.957 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 ::1/128 scope host 2024-02-17T11:57:49.957 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-02-17T11:57:49.957 INFO:teuthology.orchestra.run.smithi094.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-17T11:57:49.957 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4c brd ff:ff:ff:ff:ff:ff 2024-02-17T11:57:49.957 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f0 2024-02-17T11:57:49.957 INFO:teuthology.orchestra.run.smithi094.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-17T11:57:49.958 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:d8:db:4d brd ff:ff:ff:ff:ff:ff 2024-02-17T11:57:49.958 INFO:teuthology.orchestra.run.smithi094.stdout: altname enp7s0f1 2024-02-17T11:57:49.958 INFO:teuthology.orchestra.run.smithi094.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-17T11:57:49.958 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:62 brd ff:ff:ff:ff:ff:ff 2024-02-17T11:57:49.958 INFO:teuthology.orchestra.run.smithi094.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-17T11:57:49.958 INFO:teuthology.orchestra.run.smithi094.stdout: link/ether 0c:c4:7a:8f:d7:63 brd ff:ff:ff:ff:ff:ff 2024-02-17T11:57:49.958 INFO:teuthology.orchestra.run.smithi094.stdout: inet 172.21.15.94/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-17T11:57:49.958 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft 42135sec preferred_lft 42135sec 2024-02-17T11:57:49.958 INFO:teuthology.orchestra.run.smithi094.stdout: inet6 fe80::ec4:7aff:fe8f:d763/64 scope link 2024-02-17T11:57:49.958 INFO:teuthology.orchestra.run.smithi094.stdout: valid_lft forever preferred_lft forever 2024-02-17T11:57:49.958 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-17T11:57:49.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:49.959 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2024-02-17T11:57:49.959 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-02-17T11:57:49.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add name ceph-brx type bridge 2024-02-17T11:57:49.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr flush dev ceph-brx 2024-02-17T11:57:49.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set ceph-brx up 2024-02-17T11:57:49.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-17T11:57:49.959 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-02-17T11:57:50.023 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:49 smithi094 ceph-mon[32499]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-02-17T11:57:50.023 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:49 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1625781540' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-17T11:57:50.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:50.133 DEBUG:teuthology.orchestra.run.smithi094:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-17T11:57:50.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:49 smithi189 ceph-mon[36588]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-02-17T11:57:50.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:49 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1625781540' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-17T11:57:50.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:49 smithi125 ceph-mon[36819]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-02-17T11:57:50.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:49 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1625781540' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-17T11:57:50.200 INFO:teuthology.orchestra.run.smithi094.stdout:1 2024-02-17T11:57:50.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:50.202 DEBUG:teuthology.orchestra.run.smithi094:> route 2024-02-17T11:57:50.267 INFO:teuthology.orchestra.run.smithi094.stdout:Kernel IP routing table 2024-02-17T11:57:50.267 INFO:teuthology.orchestra.run.smithi094.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-17T11:57:50.268 INFO:teuthology.orchestra.run.smithi094.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-17T11:57:50.268 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-17T11:57:50.268 INFO:teuthology.orchestra.run.smithi094.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-17T11:57:50.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:50.268 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2024-02-17T11:57:50.268 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-02-17T11:57:50.269 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-17T11:57:50.269 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-17T11:57:50.269 DEBUG:teuthology.orchestra.run.smithi094:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-17T11:57:50.269 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-02-17T11:57:50.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:50.508 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-02-17T11:57:50.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:50.524 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list-id 2024-02-17T11:57:50.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:50.578 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2024-02-17T11:57:50.578 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-02-17T11:57:50.578 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns add ceph-ns--tmp-cephfs 2024-02-17T11:57:50.578 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-02-17T11:57:50.578 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-02-17T11:57:50.660 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-02-17T11:57:50.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:50.661 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2024-02-17T11:57:50.661 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-02-17T11:57:50.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-02-17T11:57:50.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-17T11:57:50.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-02-17T11:57:50.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-02-17T11:57:50.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-02-17T11:57:50.661 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-02-17T11:57:50.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:50.797 DEBUG:teuthology.orchestra.run.smithi094:> (cd / && exec bash -c ' 2024-02-17T11:57:50.797 DEBUG:teuthology.orchestra.run.smithi094:> set -e 2024-02-17T11:57:50.797 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set brx.0 up 2024-02-17T11:57:50.798 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link set dev brx.0 master ceph-brx 2024-02-17T11:57:50.798 DEBUG:teuthology.orchestra.run.smithi094:> ') 2024-02-17T11:57:50.839 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-17T11:57:50.839 DEBUG:teuthology.orchestra.run.smithi094:> mkdir -p -v /tmp/cephfs 2024-02-17T11:57:50.895 INFO:teuthology.orchestra.run.smithi094.stdout:mkdir: created directory '/tmp/cephfs' 2024-02-17T11:57:50.895 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-17T11:57:50.896 DEBUG:teuthology.orchestra.run.smithi094:> chmod 0000 /tmp/cephfs 2024-02-17T11:57:50.950 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-02-17T11:57:50.950 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-02-17T11:57:50.950 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/38f266b8-4b32-401a-962e-ca40f2c90bf2 2024-02-17T11:57:50.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:50.951 DEBUG:teuthology.orchestra.run.smithi094:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/38f266b8-4b32-401a-962e-ca40f2c90bf2 /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,context=system_u:object_r:postgresql_db_t:s0,nofallback 2024-02-17T11:57:51.036 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-02-17T11:57:51.036 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-02-17T11:57:51.156 INFO:teuthology.orchestra.run.smithi094.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0",nofallback 2024-02-17T11:57:51.157 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-02-17T11:57:51.157 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: resolved to: "172.21.15.94:3300,172.21.15.125:3300,172.21.15.189:3300" 2024-02-17T11:57:51.157 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: trying mount with new device syntax: 0@3bc5cfc8-cd8a-11ee-95ba-87774f69a715.cephfs=/volumes/qa/sv_1/38f266b8-4b32-401a-962e-ca40f2c90bf2 2024-02-17T11:57:51.157 INFO:teuthology.orchestra.run.smithi094.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,nowsync,context="system_u:object_r:postgresql_db_t:s0",name=0,key=0,mon_addr=172.21.15.94:3300/172.21.15.125:3300/172.21.15.189:3300" will pass to kernel 2024-02-17T11:57:51.159 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-17T11:57:51.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:51.159 DEBUG:teuthology.orchestra.run.smithi094:> sudo chmod 1777 /tmp/cephfs 2024-02-17T11:57:51.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:51.189 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-17T11:57:51.246 INFO:teuthology.orchestra.run.smithi094.stdout:client24829 2024-02-17T11:57:51.246 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T11:57:51.247 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-17T11:57:51.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:51.303 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-17T11:57:51.361 INFO:teuthology.orchestra.run.smithi094.stdout:client24829 2024-02-17T11:57:51.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:51.361 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/cephfs && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-17T11:57:51.418 INFO:teuthology.orchestra.run.smithi094.stdout:client24829 2024-02-17T11:57:51.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:51.419 DEBUG:teuthology.orchestra.run.smithi094:> (cd /tmp/cephfs && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715.client24829/status') 2024-02-17T11:57:51.485 INFO:teuthology.orchestra.run.smithi094.stdout:instance: client.24829 (3)172.21.15.94:0/2128676296 2024-02-17T11:57:51.485 INFO:teuthology.orchestra.run.smithi094.stdout:blocklisted: false 2024-02-17T11:57:51.485 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records in 2024-02-17T11:57:51.486 INFO:teuthology.orchestra.run.smithi094.stderr:0+1 records out 2024-02-17T11:57:51.486 INFO:teuthology.orchestra.run.smithi094.stderr:71 bytes copied, 2.2047e-05 s, 3.2 MB/s 2024-02-17T11:57:51.487 INFO:teuthology.run_tasks:Running task exec... 2024-02-17T11:57:51.497 INFO:teuthology.task.exec:Executing custom commands... 2024-02-17T11:57:51.497 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-02-17T11:57:51.497 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-02-17T11:57:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.630+0000 7fdb405f4640 1 -- 172.21.15.94:0/995716971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb381000a0 msgr2=0x7fdb381004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.630+0000 7fdb405f4640 1 --2- 172.21.15.94:0/995716971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb381000a0 0x7fdb381004c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009900 tx=0x7fdb3002bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.630+0000 7fdb405f4640 1 -- 172.21.15.94:0/995716971 shutdown_connections 2024-02-17T11:57:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.630+0000 7fdb405f4640 1 --2- 172.21.15.94:0/995716971 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb38101c50 0x7fdb381084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.630+0000 7fdb405f4640 1 --2- 172.21.15.94:0/995716971 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb38101290 0x7fdb38101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.630+0000 7fdb405f4640 1 --2- 172.21.15.94:0/995716971 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb381000a0 0x7fdb381004c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.630+0000 7fdb405f4640 1 -- 172.21.15.94:0/995716971 >> 172.21.15.94:0/995716971 conn(0x7fdb380fb610 msgr2=0x7fdb380fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.630+0000 7fdb405f4640 1 -- 172.21.15.94:0/995716971 shutdown_connections 2024-02-17T11:57:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.630+0000 7fdb405f4640 1 -- 172.21.15.94:0/995716971 wait complete. 2024-02-17T11:57:51.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.630+0000 7fdb405f4640 1 Processor -- start 2024-02-17T11:57:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.630+0000 7fdb405f4640 1 -- start start 2024-02-17T11:57:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb405f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb381000a0 0x7fdb38197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb405f4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb38101290 0x7fdb38197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb405f4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb38101c50 0x7fdb3819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb405f4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3819ed20 con 0x7fdb381000a0 2024-02-17T11:57:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb405f4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3819ee90 con 0x7fdb38101c50 2024-02-17T11:57:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb405f4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3819f050 con 0x7fdb38101290 2024-02-17T11:57:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb3e390640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb381000a0 0x7fdb38197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb3e390640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb381000a0 0x7fdb38197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57926/0 (socket says 172.21.15.94:57926) 2024-02-17T11:57:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb3e390640 1 -- 172.21.15.94:0/378616215 learned_addr learned my addr 172.21.15.94:0/378616215 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb3eb91640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb38101c50 0x7fdb3819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:51.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb3db8f640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb38101290 0x7fdb38197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb3e390640 1 -- 172.21.15.94:0/378616215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb38101290 msgr2=0x7fdb38197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb3e390640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb38101290 0x7fdb38197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb3e390640 1 -- 172.21.15.94:0/378616215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb38101c50 msgr2=0x7fdb3819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb3e390640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb38101c50 0x7fdb3819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb3e390640 1 -- 172.21.15.94:0/378616215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb30009560 con 0x7fdb381000a0 2024-02-17T11:57:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb3db8f640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb38101290 0x7fdb38197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb3e390640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb381000a0 0x7fdb38197230 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdb30000c00 tx=0x7fdb30004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb377fe640 1 -- 172.21.15.94:0/378616215 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30004df0 con 0x7fdb381000a0 2024-02-17T11:57:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb377fe640 1 -- 172.21.15.94:0/378616215 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb30005430 con 0x7fdb381000a0 2024-02-17T11:57:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb405f4640 1 -- 172.21.15.94:0/378616215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3819f2d0 con 0x7fdb381000a0 2024-02-17T11:57:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb377fe640 1 -- 172.21.15.94:0/378616215 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3003e450 con 0x7fdb381000a0 2024-02-17T11:57:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.631+0000 7fdb405f4640 1 -- 172.21.15.94:0/378616215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3819f860 con 0x7fdb381000a0 2024-02-17T11:57:51.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.632+0000 7fdb405f4640 1 -- 172.21.15.94:0/378616215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb38107440 con 0x7fdb381000a0 2024-02-17T11:57:51.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.633+0000 7fdb377fe640 1 -- 172.21.15.94:0/378616215 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fdb30003680 con 0x7fdb381000a0 2024-02-17T11:57:51.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.634+0000 7fdb377fe640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdb200a2160 0x7fdb200a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:51.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.634+0000 7fdb3db8f640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdb200a2160 0x7fdb200a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:51.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.634+0000 7fdb377fe640 1 -- 172.21.15.94:0/378616215 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fdb300f0840 con 0x7fdb381000a0 2024-02-17T11:57:51.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.634+0000 7fdb3db8f640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdb200a2160 0x7fdb200a4620 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdb38198740 tx=0x7fdb2c0093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:51.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.637+0000 7fdb377fe640 1 -- 172.21.15.94:0/378616215 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb300b9810 con 0x7fdb381000a0 2024-02-17T11:57:51.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:51.853+0000 7fdb405f4640 1 -- 172.21.15.94:0/378616215 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7fdb3810cbf0 con 0x7fdb381000a0 2024-02-17T11:57:52.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:51 smithi189 ceph-mon[36588]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-02-17T11:57:52.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:51 smithi125 ceph-mon[36819]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-02-17T11:57:52.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:51 smithi094 ceph-mon[32499]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-02-17T11:57:52.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.766+0000 7fdb377fe640 1 -- 172.21.15.94:0/378616215 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v21) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdb300bced0 con 0x7fdb381000a0 2024-02-17T11:57:52.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.769+0000 7fdb405f4640 1 -- 172.21.15.94:0/378616215 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdb200a2160 msgr2=0x7fdb200a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:52.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.769+0000 7fdb405f4640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdb200a2160 0x7fdb200a4620 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdb38198740 tx=0x7fdb2c0093e0 comp rx=0 tx=0).stop 2024-02-17T11:57:52.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.769+0000 7fdb405f4640 1 -- 172.21.15.94:0/378616215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb381000a0 msgr2=0x7fdb38197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:52.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.769+0000 7fdb405f4640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb381000a0 0x7fdb38197230 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdb30000c00 tx=0x7fdb30004270 comp rx=0 tx=0).stop 2024-02-17T11:57:52.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.769+0000 7fdb405f4640 1 -- 172.21.15.94:0/378616215 shutdown_connections 2024-02-17T11:57:52.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.769+0000 7fdb405f4640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fdb200a2160 0x7fdb200a4620 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:52.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.769+0000 7fdb405f4640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb38101c50 0x7fdb3819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:52.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.769+0000 7fdb405f4640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb38101290 0x7fdb38197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:52.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.769+0000 7fdb405f4640 1 --2- 172.21.15.94:0/378616215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb381000a0 0x7fdb38197230 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:52.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.769+0000 7fdb405f4640 1 -- 172.21.15.94:0/378616215 >> 172.21.15.94:0/378616215 conn(0x7fdb380fb610 msgr2=0x7fdb38104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.769+0000 7fdb405f4640 1 -- 172.21.15.94:0/378616215 shutdown_connections 2024-02-17T11:57:52.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.769+0000 7fdb405f4640 1 -- 172.21.15.94:0/378616215 wait complete. 2024-02-17T11:57:52.781 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-02-17T11:57:52.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.893+0000 7f173c965640 1 -- 172.21.15.94:0/2147361831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1738101670 msgr2=0x7f1738101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.893+0000 7f173c965640 1 --2- 172.21.15.94:0/2147361831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1738101670 0x7f1738101a90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1728009900 tx=0x7f172802bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.893+0000 7f173c965640 1 -- 172.21.15.94:0/2147361831 shutdown_connections 2024-02-17T11:57:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.893+0000 7f173c965640 1 --2- 172.21.15.94:0/2147361831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1738100d60 0x7f1738108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.893+0000 7f173c965640 1 --2- 172.21.15.94:0/2147361831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17381003a0 0x7f1738100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.893+0000 7f173c965640 1 --2- 172.21.15.94:0/2147361831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1738101670 0x7f1738101a90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.893+0000 7f173c965640 1 -- 172.21.15.94:0/2147361831 >> 172.21.15.94:0/2147361831 conn(0x7f17380fadd0 msgr2=0x7f17380fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.893+0000 7f173c965640 1 -- 172.21.15.94:0/2147361831 shutdown_connections 2024-02-17T11:57:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.893+0000 7f173c965640 1 -- 172.21.15.94:0/2147361831 wait complete. 2024-02-17T11:57:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.894+0000 7f173c965640 1 Processor -- start 2024-02-17T11:57:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.894+0000 7f173c965640 1 -- start start 2024-02-17T11:57:52.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.894+0000 7f173c965640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17381003a0 0x7f173810a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.894+0000 7f173c965640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1738100d60 0x7f1738108d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.894+0000 7f173c965640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1738101670 0x7f1738109240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.894+0000 7f173c965640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17381099a0 con 0x7f1738101670 2024-02-17T11:57:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.894+0000 7f173c965640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1738109b10 con 0x7f17381003a0 2024-02-17T11:57:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.894+0000 7f173c965640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17381a1280 con 0x7f1738100d60 2024-02-17T11:57:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.894+0000 7f1736d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1738101670 0x7f1738109240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.894+0000 7f1736d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1738101670 0x7f1738109240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57944/0 (socket says 172.21.15.94:57944) 2024-02-17T11:57:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.894+0000 7f1736d9d640 1 -- 172.21.15.94:0/774539213 learned_addr learned my addr 172.21.15.94:0/774539213 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.894+0000 7f173659c640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17381003a0 0x7f173810a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:52.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.894+0000 7f1735d9b640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1738100d60 0x7f1738108d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.895+0000 7f1736d9d640 1 -- 172.21.15.94:0/774539213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1738100d60 msgr2=0x7f1738108d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.895+0000 7f1736d9d640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1738100d60 0x7f1738108d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.895+0000 7f1736d9d640 1 -- 172.21.15.94:0/774539213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17381003a0 msgr2=0x7f173810a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.895+0000 7f1736d9d640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17381003a0 0x7f173810a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.895+0000 7f1736d9d640 1 -- 172.21.15.94:0/774539213 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728009560 con 0x7f1738101670 2024-02-17T11:57:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.895+0000 7f1736d9d640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1738101670 0x7f1738109240 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f173000db20 tx=0x7f173000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.895+0000 7f17277fe640 1 -- 172.21.15.94:0/774539213 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f173000bea0 con 0x7f1738101670 2024-02-17T11:57:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.895+0000 7f173c965640 1 -- 172.21.15.94:0/774539213 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17381a1530 con 0x7f1738101670 2024-02-17T11:57:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.895+0000 7f17277fe640 1 -- 172.21.15.94:0/774539213 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1730009940 con 0x7f1738101670 2024-02-17T11:57:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.895+0000 7f17277fe640 1 -- 172.21.15.94:0/774539213 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17300106c0 con 0x7f1738101670 2024-02-17T11:57:52.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.895+0000 7f173c965640 1 -- 172.21.15.94:0/774539213 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17381a1a40 con 0x7f1738101670 2024-02-17T11:57:52.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.896+0000 7f173c965640 1 -- 172.21.15.94:0/774539213 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17381074e0 con 0x7f1738101670 2024-02-17T11:57:52.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.897+0000 7f17277fe640 1 -- 172.21.15.94:0/774539213 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7f1730004240 con 0x7f1738101670 2024-02-17T11:57:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.898+0000 7f17277fe640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f17200a2160 0x7f17200a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.898+0000 7f173659c640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f17200a2160 0x7f17200a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.898+0000 7f17277fe640 1 -- 172.21.15.94:0/774539213 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f17300cd920 con 0x7f1738101670 2024-02-17T11:57:52.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.898+0000 7f173659c640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f17200a2160 0x7f17200a4620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1728005ab0 tx=0x7f1728005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:52.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:52.902+0000 7f17277fe640 1 -- 172.21.15.94:0/774539213 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17300969a0 con 0x7f1738101670 2024-02-17T11:57:53.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.100+0000 7f173c965640 1 -- 172.21.15.94:0/774539213 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7f173802ce10 con 0x7f17200a2160 2024-02-17T11:57:53.104 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:52 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/378616215' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-17T11:57:53.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.120+0000 7f17277fe640 1 -- 172.21.15.94:0/774539213 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7f173802ce10 con 0x7f17200a2160 2024-02-17T11:57:53.118 INFO:teuthology.orchestra.run.smithi094.stdout:{} 2024-02-17T11:57:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.122+0000 7f173c965640 1 -- 172.21.15.94:0/774539213 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f17200a2160 msgr2=0x7f17200a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.122+0000 7f173c965640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f17200a2160 0x7f17200a4620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1728005ab0 tx=0x7f1728005fd0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.122+0000 7f173c965640 1 -- 172.21.15.94:0/774539213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1738101670 msgr2=0x7f1738109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.122+0000 7f173c965640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1738101670 0x7f1738109240 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f173000db20 tx=0x7f173000b260 comp rx=0 tx=0).stop 2024-02-17T11:57:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.122+0000 7f173c965640 1 -- 172.21.15.94:0/774539213 shutdown_connections 2024-02-17T11:57:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.122+0000 7f173c965640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f17200a2160 0x7f17200a4620 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.122+0000 7f173c965640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1738101670 0x7f1738109240 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.122+0000 7f173c965640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1738100d60 0x7f1738108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.122+0000 7f173c965640 1 --2- 172.21.15.94:0/774539213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f17381003a0 0x7f173810a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.122+0000 7f173c965640 1 -- 172.21.15.94:0/774539213 >> 172.21.15.94:0/774539213 conn(0x7f17380fadd0 msgr2=0x7f17381046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.122+0000 7f173c965640 1 -- 172.21.15.94:0/774539213 shutdown_connections 2024-02-17T11:57:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.122+0000 7f173c965640 1 -- 172.21.15.94:0/774539213 wait complete. 2024-02-17T11:57:53.131 INFO:teuthology.run_tasks:Running task check-counter... 2024-02-17T11:57:53.144 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-02-17T11:57:53.159 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:57:53.160 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:52 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/378616215' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-17T11:57:53.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:52 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/378616215' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-02-17T11:57:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.272+0000 7fd787756640 1 -- 172.21.15.94:0/1163494633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd780101c40 msgr2=0x7fd7801084d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.272+0000 7fd787756640 1 --2- 172.21.15.94:0/1163494633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd780101c40 0x7fd7801084d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd778009ca0 tx=0x7fd77802c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.272+0000 7fd787756640 1 -- 172.21.15.94:0/1163494633 shutdown_connections 2024-02-17T11:57:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.272+0000 7fd787756640 1 --2- 172.21.15.94:0/1163494633 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd780101c40 0x7fd7801084d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.272+0000 7fd787756640 1 --2- 172.21.15.94:0/1163494633 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd780101280 0x7fd780101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.272+0000 7fd787756640 1 --2- 172.21.15.94:0/1163494633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd780100090 0x7fd7801004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.272+0000 7fd787756640 1 -- 172.21.15.94:0/1163494633 >> 172.21.15.94:0/1163494633 conn(0x7fd7800fb620 msgr2=0x7fd7800fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.272+0000 7fd787756640 1 -- 172.21.15.94:0/1163494633 shutdown_connections 2024-02-17T11:57:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.272+0000 7fd787756640 1 -- 172.21.15.94:0/1163494633 wait complete. 2024-02-17T11:57:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd787756640 1 Processor -- start 2024-02-17T11:57:53.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd787756640 1 -- start start 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd787756640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd780100090 0x7fd780199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd787756640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd780101280 0x7fd780199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd787756640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd780101c40 0x7fd7801a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd787756640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7801a0f00 con 0x7fd780100090 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd787756640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7801a1070 con 0x7fd780101280 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd787756640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7801a1230 con 0x7fd780101c40 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd7854f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd780100090 0x7fd780199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd7854f2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd780100090 0x7fd780199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57968/0 (socket says 172.21.15.94:57968) 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd7854f2640 1 -- 172.21.15.94:0/3760071391 learned_addr learned my addr 172.21.15.94:0/3760071391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd785cf3640 1 --2- 172.21.15.94:0/3760071391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd780101c40 0x7fd7801a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd7854f2640 1 -- 172.21.15.94:0/3760071391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd780101c40 msgr2=0x7fd7801a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd7854f2640 1 --2- 172.21.15.94:0/3760071391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd780101c40 0x7fd7801a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd7854f2640 1 -- 172.21.15.94:0/3760071391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd780101280 msgr2=0x7fd780199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd7854f2640 1 --2- 172.21.15.94:0/3760071391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd780101280 0x7fd780199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd7854f2640 1 -- 172.21.15.94:0/3760071391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd778009900 con 0x7fd780100090 2024-02-17T11:57:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd7854f2640 1 --2- 172.21.15.94:0/3760071391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd780100090 0x7fd780199410 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00d950 tx=0x7fd77c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd7767fc640 1 -- 172.21.15.94:0/3760071391 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c009c30 con 0x7fd780100090 2024-02-17T11:57:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd7767fc640 1 -- 172.21.15.94:0/3760071391 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd77c009d90 con 0x7fd780100090 2024-02-17T11:57:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.273+0000 7fd787756640 1 -- 172.21.15.94:0/3760071391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7801a1510 con 0x7fd780100090 2024-02-17T11:57:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.274+0000 7fd7767fc640 1 -- 172.21.15.94:0/3760071391 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c00f720 con 0x7fd780100090 2024-02-17T11:57:53.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.274+0000 7fd787756640 1 -- 172.21.15.94:0/3760071391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7801a19d0 con 0x7fd780100090 2024-02-17T11:57:53.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.275+0000 7fd787756640 1 -- 172.21.15.94:0/3760071391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78010baa0 con 0x7fd780100090 2024-02-17T11:57:53.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.275+0000 7fd7767fc640 1 -- 172.21.15.94:0/3760071391 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7fd77c010a60 con 0x7fd780100090 2024-02-17T11:57:53.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.276+0000 7fd7767fc640 1 --2- 172.21.15.94:0/3760071391 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd76c0a20c0 0x7fd76c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:53.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.276+0000 7fd7767fc640 1 -- 172.21.15.94:0/3760071391 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fd77c0cdaa0 con 0x7fd780100090 2024-02-17T11:57:53.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.276+0000 7fd784cf1640 1 --2- 172.21.15.94:0/3760071391 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd76c0a20c0 0x7fd76c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:53.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.276+0000 7fd784cf1640 1 --2- 172.21.15.94:0/3760071391 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd76c0a20c0 0x7fd76c0a4580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd770005fd0 tx=0x7fd770005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:53.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.280+0000 7fd7767fc640 1 -- 172.21.15.94:0/3760071391 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd77c096a70 con 0x7fd780100090 2024-02-17T11:57:53.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.454+0000 7fd787756640 1 -- 172.21.15.94:0/3760071391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd78010bc50 con 0x7fd780100090 2024-02-17T11:57:53.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.455+0000 7fd7767fc640 1 -- 172.21.15.94:0/3760071391 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22686 (secure 0 0 0) 0x7fd77c09a130 con 0x7fd780100090 2024-02-17T11:57:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:53.455 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":87,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:56:59.725646+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:57:53.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:57:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:57:53.105+0000 7f1b837e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:57:53.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:57:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:57:53.105+0000 7f1b837e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:57:53.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:57:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:57:53.105+0000 7f1b837e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:57:53.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:57:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:57:53.105+0000 7f1b837e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:57:53.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:57:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:57:53.105+0000 7f1b837e5700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:57:53.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.458+0000 7fd787756640 1 -- 172.21.15.94:0/3760071391 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd76c0a20c0 msgr2=0x7fd76c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.458+0000 7fd787756640 1 --2- 172.21.15.94:0/3760071391 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd76c0a20c0 0x7fd76c0a4580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd770005fd0 tx=0x7fd770005850 comp rx=0 tx=0).stop 2024-02-17T11:57:53.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.458+0000 7fd787756640 1 -- 172.21.15.94:0/3760071391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd780100090 msgr2=0x7fd780199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.458+0000 7fd787756640 1 --2- 172.21.15.94:0/3760071391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd780100090 0x7fd780199410 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00d950 tx=0x7fd77c00de10 comp rx=0 tx=0).stop 2024-02-17T11:57:53.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.458+0000 7fd787756640 1 -- 172.21.15.94:0/3760071391 shutdown_connections 2024-02-17T11:57:53.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.458+0000 7fd787756640 1 --2- 172.21.15.94:0/3760071391 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fd76c0a20c0 0x7fd76c0a4580 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.458+0000 7fd787756640 1 --2- 172.21.15.94:0/3760071391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd780101c40 0x7fd7801a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.458+0000 7fd787756640 1 --2- 172.21.15.94:0/3760071391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd780101280 0x7fd780199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.458+0000 7fd787756640 1 --2- 172.21.15.94:0/3760071391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd780100090 0x7fd780199410 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.458+0000 7fd787756640 1 -- 172.21.15.94:0/3760071391 >> 172.21.15.94:0/3760071391 conn(0x7fd7800fb620 msgr2=0x7fd7800fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:53.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.458+0000 7fd787756640 1 -- 172.21.15.94:0/3760071391 shutdown_connections 2024-02-17T11:57:53.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.458+0000 7fd787756640 1 -- 172.21.15.94:0/3760071391 wait complete. 2024-02-17T11:57:53.468 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T11:51:20.894601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T11:55:56.349040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T11:55:57.234916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T11:57:53.468 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T11:57:53.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2738695129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa20100080 msgr2=0x7ffa201004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 --2- 172.21.15.94:0/2738695129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa20100080 0x7ffa201004a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009ca0 tx=0x7ffa1c02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2738695129 shutdown_connections 2024-02-17T11:57:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 --2- 172.21.15.94:0/2738695129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa20101c30 0x7ffa201084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 --2- 172.21.15.94:0/2738695129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa20101270 0x7ffa201016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 --2- 172.21.15.94:0/2738695129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa20100080 0x7ffa201004a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2738695129 >> 172.21.15.94:0/2738695129 conn(0x7ffa200fb610 msgr2=0x7ffa200fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2738695129 shutdown_connections 2024-02-17T11:57:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2738695129 wait complete. 2024-02-17T11:57:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 Processor -- start 2024-02-17T11:57:53.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 -- start start 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa20100080 0x7ffa201961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa20101270 0x7ffa201966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa20101c30 0x7ffa2019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2019dca0 con 0x7ffa20101c30 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2019de10 con 0x7ffa20100080 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa27ba7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2019dfd0 con 0x7ffa20101270 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.575+0000 7ffa26144640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa20101c30 0x7ffa2019d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa26144640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa20101c30 0x7ffa2019d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57978/0 (socket says 172.21.15.94:57978) 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa26144640 1 -- 172.21.15.94:0/2308968237 learned_addr learned my addr 172.21.15.94:0/2308968237 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa25142640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa20101270 0x7ffa201966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa25943640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa20100080 0x7ffa201961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa26144640 1 -- 172.21.15.94:0/2308968237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa20101270 msgr2=0x7ffa201966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa26144640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa20101270 0x7ffa201966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa26144640 1 -- 172.21.15.94:0/2308968237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa20100080 msgr2=0x7ffa201961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa26144640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa20100080 0x7ffa201961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa26144640 1 -- 172.21.15.94:0/2308968237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c009900 con 0x7ffa20101c30 2024-02-17T11:57:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa26144640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa20101c30 0x7ffa2019d760 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800c970 tx=0x7ffa1800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa16ffd640 1 -- 172.21.15.94:0/2308968237 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18007bb0 con 0x7ffa20101c30 2024-02-17T11:57:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa16ffd640 1 -- 172.21.15.94:0/2308968237 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffa18007d10 con 0x7ffa20101c30 2024-02-17T11:57:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2308968237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2019e2b0 con 0x7ffa20101c30 2024-02-17T11:57:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa16ffd640 1 -- 172.21.15.94:0/2308968237 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1800f450 con 0x7ffa20101c30 2024-02-17T11:57:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.576+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2308968237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2019e770 con 0x7ffa20101c30 2024-02-17T11:57:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.577+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2308968237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa20107440 con 0x7ffa20101c30 2024-02-17T11:57:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.578+0000 7ffa16ffd640 1 -- 172.21.15.94:0/2308968237 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 18) v1 ==== 136768+0+0 (secure 0 0 0) 0x7ffa18004f00 con 0x7ffa20101c30 2024-02-17T11:57:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.579+0000 7ffa16ffd640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffa0c0a20c0 0x7ffa0c0a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.579+0000 7ffa25943640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffa0c0a20c0 0x7ffa0c0a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.579+0000 7ffa16ffd640 1 -- 172.21.15.94:0/2308968237 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7ffa180cd9f0 con 0x7ffa20101c30 2024-02-17T11:57:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.580+0000 7ffa25943640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffa0c0a20c0 0x7ffa0c0a4580 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c037040 tx=0x7ffa1c009890 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.584+0000 7ffa16ffd640 1 -- 172.21.15.94:0/2308968237 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa180969c0 con 0x7ffa20101c30 2024-02-17T11:57:53.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.760+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2308968237 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa2010ba90 con 0x7ffa20101c30 2024-02-17T11:57:53.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.760+0000 7ffa16ffd640 1 -- 172.21.15.94:0/2308968237 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa2010ba90 con 0x7ffa20101c30 2024-02-17T11:57:53.759 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-02-17T11:57:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.762+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2308968237 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffa0c0a20c0 msgr2=0x7ffa0c0a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.762+0000 7ffa27ba7640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffa0c0a20c0 0x7ffa0c0a4580 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c037040 tx=0x7ffa1c009890 comp rx=0 tx=0).stop 2024-02-17T11:57:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.762+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2308968237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa20101c30 msgr2=0x7ffa2019d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.762+0000 7ffa27ba7640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa20101c30 0x7ffa2019d760 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800c970 tx=0x7ffa1800ce30 comp rx=0 tx=0).stop 2024-02-17T11:57:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.762+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2308968237 shutdown_connections 2024-02-17T11:57:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.762+0000 7ffa27ba7640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7ffa0c0a20c0 0x7ffa0c0a4580 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.762+0000 7ffa27ba7640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffa20101c30 0x7ffa2019d760 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.762+0000 7ffa27ba7640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa20101270 0x7ffa201966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.762+0000 7ffa27ba7640 1 --2- 172.21.15.94:0/2308968237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffa20100080 0x7ffa201961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.762+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2308968237 >> 172.21.15.94:0/2308968237 conn(0x7ffa200fb610 msgr2=0x7ffa20104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.762+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2308968237 shutdown_connections 2024-02-17T11:57:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.762+0000 7ffa27ba7640 1 -- 172.21.15.94:0/2308968237 wait complete. 2024-02-17T11:57:53.773 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T11:57:53.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.881+0000 7fb6fc676640 1 -- 172.21.15.94:0/2107556735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6f41000a0 msgr2=0x7fb6f41004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.881+0000 7fb6fc676640 1 --2- 172.21.15.94:0/2107556735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6f41000a0 0x7fb6f41004c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009ca0 tx=0x7fb6f002c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.881+0000 7fb6fc676640 1 -- 172.21.15.94:0/2107556735 shutdown_connections 2024-02-17T11:57:53.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.881+0000 7fb6fc676640 1 --2- 172.21.15.94:0/2107556735 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6f4101c50 0x7fb6f41084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.881+0000 7fb6fc676640 1 --2- 172.21.15.94:0/2107556735 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6f4101290 0x7fb6f4101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.881+0000 7fb6fc676640 1 --2- 172.21.15.94:0/2107556735 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6f41000a0 0x7fb6f41004c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.881+0000 7fb6fc676640 1 -- 172.21.15.94:0/2107556735 >> 172.21.15.94:0/2107556735 conn(0x7fb6f40fb630 msgr2=0x7fb6f40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:53.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.881+0000 7fb6fc676640 1 -- 172.21.15.94:0/2107556735 shutdown_connections 2024-02-17T11:57:53.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.881+0000 7fb6fc676640 1 -- 172.21.15.94:0/2107556735 wait complete. 2024-02-17T11:57:53.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6fc676640 1 Processor -- start 2024-02-17T11:57:53.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6fc676640 1 -- start start 2024-02-17T11:57:53.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6fc676640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6f41000a0 0x7fb6f4197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:53.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6fc676640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6f4101290 0x7fb6f4197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:53.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6fc676640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6f4101c50 0x7fb6f419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:53.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6fc676640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f419ed40 con 0x7fb6f4101290 2024-02-17T11:57:53.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6fc676640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f419eeb0 con 0x7fb6f4101c50 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6fc676640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f419f070 con 0x7fb6f41000a0 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6f9c11640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6f4101290 0x7fb6f4197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6f9c11640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6f4101290 0x7fb6f4197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57990/0 (socket says 172.21.15.94:57990) 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6f9c11640 1 -- 172.21.15.94:0/2706592460 learned_addr learned my addr 172.21.15.94:0/2706592460 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6fac13640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6f4101c50 0x7fb6f419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6fa412640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6f41000a0 0x7fb6f4197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6f9c11640 1 -- 172.21.15.94:0/2706592460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6f41000a0 msgr2=0x7fb6f4197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6f9c11640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6f41000a0 0x7fb6f4197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6f9c11640 1 -- 172.21.15.94:0/2706592460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6f4101c50 msgr2=0x7fb6f419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6f9c11640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6f4101c50 0x7fb6f419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6f9c11640 1 -- 172.21.15.94:0/2706592460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f0009900 con 0x7fb6f4101290 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6fac13640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6f4101c50 0x7fb6f419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.882+0000 7fb6f9c11640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6f4101290 0x7fb6f4197790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400d770 tx=0x7fb6e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:53.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.883+0000 7fb6eb7fe640 1 -- 172.21.15.94:0/2706592460 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4009c30 con 0x7fb6f4101290 2024-02-17T11:57:53.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.883+0000 7fb6eb7fe640 1 -- 172.21.15.94:0/2706592460 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb6e400de80 con 0x7fb6f4101290 2024-02-17T11:57:53.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.883+0000 7fb6fc676640 1 -- 172.21.15.94:0/2706592460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f419f350 con 0x7fb6f4101290 2024-02-17T11:57:53.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.883+0000 7fb6eb7fe640 1 -- 172.21.15.94:0/2706592460 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4004e80 con 0x7fb6f4101290 2024-02-17T11:57:53.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.883+0000 7fb6fc676640 1 -- 172.21.15.94:0/2706592460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f419f860 con 0x7fb6f4101290 2024-02-17T11:57:53.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.884+0000 7fb6fc676640 1 -- 172.21.15.94:0/2706592460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f410bab0 con 0x7fb6f4101290 2024-02-17T11:57:53.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.885+0000 7fb6eb7fe640 1 -- 172.21.15.94:0/2706592460 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fb6e4004fe0 con 0x7fb6f4101290 2024-02-17T11:57:53.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.885+0000 7fb6eb7fe640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb6e00a21d0 0x7fb6e00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:53.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.885+0000 7fb6fa412640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb6e00a21d0 0x7fb6e00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:53.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.885+0000 7fb6eb7fe640 1 -- 172.21.15.94:0/2706592460 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fb6e40cdd00 con 0x7fb6f4101290 2024-02-17T11:57:53.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.886+0000 7fb6fa412640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb6e00a21d0 0x7fb6e00a4690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb6f002c700 tx=0x7fb6f0009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:53.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:53.889+0000 7fb6eb7fe640 1 -- 172.21.15.94:0/2706592460 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb6e4096bf0 con 0x7fb6f4101290 2024-02-17T11:57:54.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.062+0000 7fb6fc676640 1 -- 172.21.15.94:0/2706592460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb6f419fd00 con 0x7fb6f4101290 2024-02-17T11:57:54.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.063+0000 7fb6eb7fe640 1 -- 172.21.15.94:0/2706592460 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb6e4017020 con 0x7fb6f4101290 2024-02-17T11:57:54.062 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-02-17T11:57:54.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.065+0000 7fb6fc676640 1 -- 172.21.15.94:0/2706592460 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb6e00a21d0 msgr2=0x7fb6e00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.065+0000 7fb6fc676640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb6e00a21d0 0x7fb6e00a4690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb6f002c700 tx=0x7fb6f0009800 comp rx=0 tx=0).stop 2024-02-17T11:57:54.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.065+0000 7fb6fc676640 1 -- 172.21.15.94:0/2706592460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6f4101290 msgr2=0x7fb6f4197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.065+0000 7fb6fc676640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6f4101290 0x7fb6f4197790 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400d770 tx=0x7fb6e400dc30 comp rx=0 tx=0).stop 2024-02-17T11:57:54.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.065+0000 7fb6fc676640 1 -- 172.21.15.94:0/2706592460 shutdown_connections 2024-02-17T11:57:54.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.065+0000 7fb6fc676640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fb6e00a21d0 0x7fb6e00a4690 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.065+0000 7fb6fc676640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb6f4101c50 0x7fb6f419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.065+0000 7fb6fc676640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6f4101290 0x7fb6f4197790 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.065+0000 7fb6fc676640 1 --2- 172.21.15.94:0/2706592460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6f41000a0 0x7fb6f4197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.065+0000 7fb6fc676640 1 -- 172.21.15.94:0/2706592460 >> 172.21.15.94:0/2706592460 conn(0x7fb6f40fb630 msgr2=0x7fb6f40fd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:54.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.065+0000 7fb6fc676640 1 -- 172.21.15.94:0/2706592460 shutdown_connections 2024-02-17T11:57:54.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.065+0000 7fb6fc676640 1 -- 172.21.15.94:0/2706592460 wait complete. 2024-02-17T11:57:54.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:53 smithi094 ceph-mon[32499]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-02-17T11:57:54.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:53 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/378616215' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-17T11:57:54.073 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:53 smithi094 ceph-mon[32499]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T11:57:54.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:53 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3760071391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:54.074 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:53 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2308968237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:54.075 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T11:57:54.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:53 smithi189 ceph-mon[36588]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-02-17T11:57:54.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:53 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/378616215' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-17T11:57:54.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:53 smithi189 ceph-mon[36588]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T11:57:54.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:53 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3760071391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:54.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:53 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2308968237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 -- 172.21.15.94:0/2080241666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102d40 msgr2=0x7fa76c1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 --2- 172.21.15.94:0/2080241666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102d40 0x7fa76c1095d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa768009ca0 tx=0x7fa76802c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 -- 172.21.15.94:0/2080241666 shutdown_connections 2024-02-17T11:57:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 --2- 172.21.15.94:0/2080241666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102d40 0x7fa76c1095d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 --2- 172.21.15.94:0/2080241666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c102380 0x7fa76c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 --2- 172.21.15.94:0/2080241666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76c101190 0x7fa76c1015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 -- 172.21.15.94:0/2080241666 >> 172.21.15.94:0/2080241666 conn(0x7fa76c0fc720 msgr2=0x7fa76c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 -- 172.21.15.94:0/2080241666 shutdown_connections 2024-02-17T11:57:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 -- 172.21.15.94:0/2080241666 wait complete. 2024-02-17T11:57:54.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 Processor -- start 2024-02-17T11:57:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 -- start start 2024-02-17T11:57:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76c101190 0x7fa76c074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c102380 0x7fa76c073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102d40 0x7fa76c073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c074210 con 0x7fa76c102d40 2024-02-17T11:57:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c074380 con 0x7fa76c102380 2024-02-17T11:57:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.180+0000 7fa77338a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c1a1260 con 0x7fa76c101190 2024-02-17T11:57:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa771927640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102d40 0x7fa76c073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa771927640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102d40 0x7fa76c073ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58004/0 (socket says 172.21.15.94:58004) 2024-02-17T11:57:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa771927640 1 -- 172.21.15.94:0/3753437988 learned_addr learned my addr 172.21.15.94:0/3753437988 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa771126640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76c101190 0x7fa76c074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:54.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa770925640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c102380 0x7fa76c073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa771927640 1 -- 172.21.15.94:0/3753437988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76c101190 msgr2=0x7fa76c074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa771927640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76c101190 0x7fa76c074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa771927640 1 -- 172.21.15.94:0/3753437988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c102380 msgr2=0x7fa76c073570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa771927640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c102380 0x7fa76c073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa771927640 1 -- 172.21.15.94:0/3753437988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768009900 con 0x7fa76c102d40 2024-02-17T11:57:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa771126640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76c101190 0x7fa76c074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:54.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa771927640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102d40 0x7fa76c073ab0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa768033000 tx=0x7fa768034a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa7627fc640 1 -- 172.21.15.94:0/3753437988 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680329b0 con 0x7fa76c102d40 2024-02-17T11:57:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa7627fc640 1 -- 172.21.15.94:0/3753437988 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa768003e50 con 0x7fa76c102d40 2024-02-17T11:57:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa7627fc640 1 -- 172.21.15.94:0/3753437988 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768040450 con 0x7fa76c102d40 2024-02-17T11:57:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa77338a640 1 -- 172.21.15.94:0/3753437988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76c1a14b0 con 0x7fa76c102d40 2024-02-17T11:57:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.181+0000 7fa77338a640 1 -- 172.21.15.94:0/3753437988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76c1a1960 con 0x7fa76c102d40 2024-02-17T11:57:54.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.182+0000 7fa77338a640 1 -- 172.21.15.94:0/3753437988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76c1084b0 con 0x7fa76c102d40 2024-02-17T11:57:54.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:53 smithi125 ceph-mon[36819]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-02-17T11:57:54.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:53 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/378616215' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-02-17T11:57:54.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:53 smithi125 ceph-mon[36819]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T11:57:54.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:53 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3760071391' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:54.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:53 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2308968237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:54.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.183+0000 7fa7627fc640 1 -- 172.21.15.94:0/3753437988 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa76800a040 con 0x7fa76c102d40 2024-02-17T11:57:54.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.184+0000 7fa7627fc640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa7580a2150 0x7fa7580a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:54.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.184+0000 7fa7627fc640 1 -- 172.21.15.94:0/3753437988 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa7680f10e0 con 0x7fa76c102d40 2024-02-17T11:57:54.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.184+0000 7fa771126640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa7580a2150 0x7fa7580a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:54.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.184+0000 7fa771126640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa7580a2150 0x7fa7580a4610 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa76c1021e0 tx=0x7fa764005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:54.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.187+0000 7fa7627fc640 1 -- 172.21.15.94:0/3753437988 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa7680ba060 con 0x7fa76c102d40 2024-02-17T11:57:54.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.360+0000 7fa77338a640 1 -- 172.21.15.94:0/3753437988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa76c10cba0 con 0x7fa76c102d40 2024-02-17T11:57:54.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.360+0000 7fa7627fc640 1 -- 172.21.15.94:0/3753437988 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa7680bd720 con 0x7fa76c102d40 2024-02-17T11:57:54.359 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-02-17T11:57:54.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.362+0000 7fa77338a640 1 -- 172.21.15.94:0/3753437988 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa7580a2150 msgr2=0x7fa7580a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.362+0000 7fa77338a640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa7580a2150 0x7fa7580a4610 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa76c1021e0 tx=0x7fa764005f90 comp rx=0 tx=0).stop 2024-02-17T11:57:54.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.362+0000 7fa77338a640 1 -- 172.21.15.94:0/3753437988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102d40 msgr2=0x7fa76c073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.362+0000 7fa77338a640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102d40 0x7fa76c073ab0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa768033000 tx=0x7fa768034a90 comp rx=0 tx=0).stop 2024-02-17T11:57:54.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.362+0000 7fa77338a640 1 -- 172.21.15.94:0/3753437988 shutdown_connections 2024-02-17T11:57:54.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.362+0000 7fa77338a640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa7580a2150 0x7fa7580a4610 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.362+0000 7fa77338a640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102d40 0x7fa76c073ab0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.362+0000 7fa77338a640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c102380 0x7fa76c073570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.362+0000 7fa77338a640 1 --2- 172.21.15.94:0/3753437988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76c101190 0x7fa76c074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.362+0000 7fa77338a640 1 -- 172.21.15.94:0/3753437988 >> 172.21.15.94:0/3753437988 conn(0x7fa76c0fc720 msgr2=0x7fa76c105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:54.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.362+0000 7fa77338a640 1 -- 172.21.15.94:0/3753437988 shutdown_connections 2024-02-17T11:57:54.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.362+0000 7fa77338a640 1 -- 172.21.15.94:0/3753437988 wait complete. 2024-02-17T11:57:54.372 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:57:54.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.476+0000 7f96e3c17640 1 -- 172.21.15.94:0/355173420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc1000a0 msgr2=0x7f96dc1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.476+0000 7f96e3c17640 1 --2- 172.21.15.94:0/355173420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc1000a0 0x7f96dc1004c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009ca0 tx=0x7f96d402c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.476+0000 7f96e3c17640 1 -- 172.21.15.94:0/355173420 shutdown_connections 2024-02-17T11:57:54.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.476+0000 7f96e3c17640 1 --2- 172.21.15.94:0/355173420 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96dc101c50 0x7f96dc1084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.476+0000 7f96e3c17640 1 --2- 172.21.15.94:0/355173420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96dc101290 0x7f96dc101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.476+0000 7f96e3c17640 1 --2- 172.21.15.94:0/355173420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc1000a0 0x7f96dc1004c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.476+0000 7f96e3c17640 1 -- 172.21.15.94:0/355173420 >> 172.21.15.94:0/355173420 conn(0x7f96dc0fb610 msgr2=0x7f96dc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:54.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.476+0000 7f96e3c17640 1 -- 172.21.15.94:0/355173420 shutdown_connections 2024-02-17T11:57:54.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.476+0000 7f96e3c17640 1 -- 172.21.15.94:0/355173420 wait complete. 2024-02-17T11:57:54.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.477+0000 7f96e3c17640 1 Processor -- start 2024-02-17T11:57:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.477+0000 7f96e3c17640 1 -- start start 2024-02-17T11:57:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.477+0000 7f96e3c17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96dc1000a0 0x7f96dc196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.477+0000 7f96e3c17640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96dc101290 0x7f96dc196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.477+0000 7f96e3c17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc101c50 0x7f96dc19d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.477+0000 7f96e3c17640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc19dc40 con 0x7f96dc101c50 2024-02-17T11:57:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.477+0000 7f96e3c17640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc19ddb0 con 0x7f96dc1000a0 2024-02-17T11:57:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.477+0000 7f96e3c17640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc19df70 con 0x7f96dc101290 2024-02-17T11:57:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.477+0000 7f96e21b4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc101c50 0x7f96dc19d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.477+0000 7f96e21b4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc101c50 0x7f96dc19d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53190/0 (socket says 172.21.15.94:53190) 2024-02-17T11:57:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.477+0000 7f96e21b4640 1 -- 172.21.15.94:0/1452096776 learned_addr learned my addr 172.21.15.94:0/1452096776 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.477+0000 7f96e19b3640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96dc1000a0 0x7f96dc196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:54.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.477+0000 7f96e11b2640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96dc101290 0x7f96dc196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.478+0000 7f96e21b4640 1 -- 172.21.15.94:0/1452096776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96dc101290 msgr2=0x7f96dc196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.478+0000 7f96e21b4640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96dc101290 0x7f96dc196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.478+0000 7f96e21b4640 1 -- 172.21.15.94:0/1452096776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96dc1000a0 msgr2=0x7f96dc196150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.478+0000 7f96e21b4640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96dc1000a0 0x7f96dc196150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.478+0000 7f96e21b4640 1 -- 172.21.15.94:0/1452096776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d4009900 con 0x7f96dc101c50 2024-02-17T11:57:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.478+0000 7f96e21b4640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc101c50 0x7f96dc19d700 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f96d800d750 tx=0x7f96d800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.478+0000 7f96d2ffd640 1 -- 172.21.15.94:0/1452096776 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8009bd0 con 0x7f96dc101c50 2024-02-17T11:57:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.478+0000 7f96d2ffd640 1 -- 172.21.15.94:0/1452096776 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96d800de60 con 0x7f96dc101c50 2024-02-17T11:57:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.478+0000 7f96d2ffd640 1 -- 172.21.15.94:0/1452096776 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8004e70 con 0x7f96dc101c50 2024-02-17T11:57:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.478+0000 7f96e3c17640 1 -- 172.21.15.94:0/1452096776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96dc19e250 con 0x7f96dc101c50 2024-02-17T11:57:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.478+0000 7f96e3c17640 1 -- 172.21.15.94:0/1452096776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc19e710 con 0x7f96dc101c50 2024-02-17T11:57:54.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.479+0000 7f96e3c17640 1 -- 172.21.15.94:0/1452096776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc107440 con 0x7f96dc101c50 2024-02-17T11:57:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.481+0000 7f96d2ffd640 1 -- 172.21.15.94:0/1452096776 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f96d8005000 con 0x7f96dc101c50 2024-02-17T11:57:54.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.481+0000 7f96d2ffd640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f96c80a2150 0x7f96c80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.481+0000 7f96e19b3640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f96c80a2150 0x7f96c80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.481+0000 7f96d2ffd640 1 -- 172.21.15.94:0/1452096776 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f96d80cdb10 con 0x7f96dc101c50 2024-02-17T11:57:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.482+0000 7f96e19b3640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f96c80a2150 0x7f96c80a4610 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f96d402c630 tx=0x7f96d4009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:54.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.485+0000 7f96d2ffd640 1 -- 172.21.15.94:0/1452096776 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96d809a560 con 0x7f96dc101c50 2024-02-17T11:57:54.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.661+0000 7f96e3c17640 1 -- 172.21.15.94:0/1452096776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96dc02ce40 con 0x7f96dc101c50 2024-02-17T11:57:54.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.662+0000 7f96d2ffd640 1 -- 172.21.15.94:0/1452096776 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22686 (secure 0 0 0) 0x7f96d8017090 con 0x7f96dc101c50 2024-02-17T11:57:54.661 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:54.662 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":87,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:56:59.725646+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:57:54.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.664+0000 7f96e3c17640 1 -- 172.21.15.94:0/1452096776 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f96c80a2150 msgr2=0x7f96c80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.664+0000 7f96e3c17640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f96c80a2150 0x7f96c80a4610 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f96d402c630 tx=0x7f96d4009800 comp rx=0 tx=0).stop 2024-02-17T11:57:54.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.664+0000 7f96e3c17640 1 -- 172.21.15.94:0/1452096776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc101c50 msgr2=0x7f96dc19d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.664+0000 7f96e3c17640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc101c50 0x7f96dc19d700 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f96d800d750 tx=0x7f96d800dc10 comp rx=0 tx=0).stop 2024-02-17T11:57:54.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.664+0000 7f96e3c17640 1 -- 172.21.15.94:0/1452096776 shutdown_connections 2024-02-17T11:57:54.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.664+0000 7f96e3c17640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f96c80a2150 0x7f96c80a4610 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.664+0000 7f96e3c17640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96dc101c50 0x7f96dc19d700 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.664+0000 7f96e3c17640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96dc101290 0x7f96dc196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.664+0000 7f96e3c17640 1 --2- 172.21.15.94:0/1452096776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96dc1000a0 0x7f96dc196150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.664+0000 7f96e3c17640 1 -- 172.21.15.94:0/1452096776 >> 172.21.15.94:0/1452096776 conn(0x7f96dc0fb610 msgr2=0x7f96dc104610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:54.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.664+0000 7f96e3c17640 1 -- 172.21.15.94:0/1452096776 shutdown_connections 2024-02-17T11:57:54.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.664+0000 7f96e3c17640 1 -- 172.21.15.94:0/1452096776 wait complete. 2024-02-17T11:57:54.675 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T11:51:20.894601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T11:55:56.349040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T11:55:57.234916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T11:57:54.675 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T11:57:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.783+0000 7f4ccac40640 1 -- 172.21.15.94:0/3335354935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc4101c30 msgr2=0x7f4cc41084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.783+0000 7f4ccac40640 1 --2- 172.21.15.94:0/3335354935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc4101c30 0x7f4cc41084c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009ca0 tx=0x7f4cbc02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.783+0000 7f4ccac40640 1 -- 172.21.15.94:0/3335354935 shutdown_connections 2024-02-17T11:57:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.783+0000 7f4ccac40640 1 --2- 172.21.15.94:0/3335354935 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc4101c30 0x7f4cc41084c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.783+0000 7f4ccac40640 1 --2- 172.21.15.94:0/3335354935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc4101270 0x7f4cc41016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.783+0000 7f4ccac40640 1 --2- 172.21.15.94:0/3335354935 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc4100080 0x7f4cc41004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.783+0000 7f4ccac40640 1 -- 172.21.15.94:0/3335354935 >> 172.21.15.94:0/3335354935 conn(0x7f4cc40fb610 msgr2=0x7f4cc40fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.783+0000 7f4ccac40640 1 -- 172.21.15.94:0/3335354935 shutdown_connections 2024-02-17T11:57:54.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.783+0000 7f4ccac40640 1 -- 172.21.15.94:0/3335354935 wait complete. 2024-02-17T11:57:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4ccac40640 1 Processor -- start 2024-02-17T11:57:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4ccac40640 1 -- start start 2024-02-17T11:57:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4ccac40640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc4100080 0x7f4cc4196110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4ccac40640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc4101270 0x7f4cc4196650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4ccac40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc4101c30 0x7f4cc419d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4ccac40640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc419dc00 con 0x7f4cc4101c30 2024-02-17T11:57:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4ccac40640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc419dd70 con 0x7f4cc4100080 2024-02-17T11:57:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4ccac40640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc4068360 con 0x7f4cc4101270 2024-02-17T11:57:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc91dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc4101c30 0x7f4cc419d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc91dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc4101c30 0x7f4cc419d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53216/0 (socket says 172.21.15.94:53216) 2024-02-17T11:57:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc91dd640 1 -- 172.21.15.94:0/2589401321 learned_addr learned my addr 172.21.15.94:0/2589401321 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc89dc640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc4100080 0x7f4cc4196110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:54.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc3fff640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc4101270 0x7f4cc4196650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc91dd640 1 -- 172.21.15.94:0/2589401321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc4101270 msgr2=0x7f4cc4196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc91dd640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc4101270 0x7f4cc4196650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc91dd640 1 -- 172.21.15.94:0/2589401321 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc4100080 msgr2=0x7f4cc4196110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc91dd640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc4100080 0x7f4cc4196110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc91dd640 1 -- 172.21.15.94:0/2589401321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cbc009900 con 0x7f4cc4101c30 2024-02-17T11:57:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc3fff640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc4101270 0x7f4cc4196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc91dd640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc4101c30 0x7f4cc419d6c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc02c630 tx=0x7f4cbc032f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc1ffb640 1 -- 172.21.15.94:0/2589401321 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc034e20 con 0x7f4cc4101c30 2024-02-17T11:57:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc1ffb640 1 -- 172.21.15.94:0/2589401321 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4cbc003e50 con 0x7f4cc4101c30 2024-02-17T11:57:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4ccac40640 1 -- 172.21.15.94:0/2589401321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc40685e0 con 0x7f4cc4101c30 2024-02-17T11:57:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.784+0000 7f4cc1ffb640 1 -- 172.21.15.94:0/2589401321 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc004880 con 0x7f4cc4101c30 2024-02-17T11:57:54.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.785+0000 7f4ccac40640 1 -- 172.21.15.94:0/2589401321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc4068a90 con 0x7f4cc4101c30 2024-02-17T11:57:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.786+0000 7f4ccac40640 1 -- 172.21.15.94:0/2589401321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc41073a0 con 0x7f4cc4101c30 2024-02-17T11:57:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.787+0000 7f4cc1ffb640 1 -- 172.21.15.94:0/2589401321 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f4cbc00a040 con 0x7f4cc4101c30 2024-02-17T11:57:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.787+0000 7f4cc1ffb640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4cb40a2150 0x7f4cb40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.787+0000 7f4cc89dc640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4cb40a2150 0x7f4cb40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.787+0000 7f4cc1ffb640 1 -- 172.21.15.94:0/2589401321 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f4cbc0f0b20 con 0x7f4cc4101c30 2024-02-17T11:57:54.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.788+0000 7f4cc89dc640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4cb40a2150 0x7f4cb40a4610 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4cc41010d0 tx=0x7f4cb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:54.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.791+0000 7f4cc1ffb640 1 -- 172.21.15.94:0/2589401321 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4cbc0b9aa0 con 0x7f4cc4101c30 2024-02-17T11:57:54.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:54 smithi094 ceph-mon[32499]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:54.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:54 smithi094 ceph-mon[32499]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-02-17T11:57:54.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:54 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2706592460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:54.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:54 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3753437988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:54.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:54 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1452096776' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:54.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.966+0000 7f4ccac40640 1 -- 172.21.15.94:0/2589401321 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4cc402ce40 con 0x7f4cc4101c30 2024-02-17T11:57:54.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.967+0000 7f4cc1ffb640 1 -- 172.21.15.94:0/2589401321 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4cbc0bd160 con 0x7f4cc4101c30 2024-02-17T11:57:54.965 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-02-17T11:57:54.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.969+0000 7f4ccac40640 1 -- 172.21.15.94:0/2589401321 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4cb40a2150 msgr2=0x7f4cb40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.969+0000 7f4ccac40640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4cb40a2150 0x7f4cb40a4610 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4cc41010d0 tx=0x7f4cb8005f90 comp rx=0 tx=0).stop 2024-02-17T11:57:54.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.969+0000 7f4ccac40640 1 -- 172.21.15.94:0/2589401321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc4101c30 msgr2=0x7f4cc419d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:54.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.969+0000 7f4ccac40640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc4101c30 0x7f4cc419d6c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc02c630 tx=0x7f4cbc032f80 comp rx=0 tx=0).stop 2024-02-17T11:57:54.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.969+0000 7f4ccac40640 1 -- 172.21.15.94:0/2589401321 shutdown_connections 2024-02-17T11:57:54.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.969+0000 7f4ccac40640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f4cb40a2150 0x7f4cb40a4610 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.969+0000 7f4ccac40640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4cc4101c30 0x7f4cc419d6c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.969+0000 7f4ccac40640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4cc4101270 0x7f4cc4196650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.969+0000 7f4ccac40640 1 --2- 172.21.15.94:0/2589401321 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4cc4100080 0x7f4cc4196110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:54.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.969+0000 7f4ccac40640 1 -- 172.21.15.94:0/2589401321 >> 172.21.15.94:0/2589401321 conn(0x7f4cc40fb610 msgr2=0x7f4cc4104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:54.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.969+0000 7f4ccac40640 1 -- 172.21.15.94:0/2589401321 shutdown_connections 2024-02-17T11:57:54.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:54.969+0000 7f4ccac40640 1 -- 172.21.15.94:0/2589401321 wait complete. 2024-02-17T11:57:54.979 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T11:57:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.086+0000 7f8be16b5640 1 -- 172.21.15.94:0/1168385703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bdc101190 msgr2=0x7f8bdc1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.086+0000 7f8be16b5640 1 --2- 172.21.15.94:0/1168385703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bdc101190 0x7f8bdc1015b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009ca0 tx=0x7f8bcc02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.086+0000 7f8be16b5640 1 -- 172.21.15.94:0/1168385703 shutdown_connections 2024-02-17T11:57:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.086+0000 7f8be16b5640 1 --2- 172.21.15.94:0/1168385703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bdc102d40 0x7f8bdc1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.086+0000 7f8be16b5640 1 --2- 172.21.15.94:0/1168385703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bdc102380 0x7f8bdc102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.086+0000 7f8be16b5640 1 --2- 172.21.15.94:0/1168385703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bdc101190 0x7f8bdc1015b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.086+0000 7f8be16b5640 1 -- 172.21.15.94:0/1168385703 >> 172.21.15.94:0/1168385703 conn(0x7f8bdc0fc720 msgr2=0x7f8bdc0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:55.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.086+0000 7f8be16b5640 1 -- 172.21.15.94:0/1168385703 shutdown_connections 2024-02-17T11:57:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.086+0000 7f8be16b5640 1 -- 172.21.15.94:0/1168385703 wait complete. 2024-02-17T11:57:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.087+0000 7f8be16b5640 1 Processor -- start 2024-02-17T11:57:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.087+0000 7f8be16b5640 1 -- start start 2024-02-17T11:57:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.087+0000 7f8be16b5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bdc101190 0x7f8bdc197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.087+0000 7f8be16b5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bdc102380 0x7f8bdc1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.087+0000 7f8be16b5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bdc102d40 0x7f8bdc19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.087+0000 7f8be16b5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc19ed60 con 0x7f8bdc102d40 2024-02-17T11:57:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.087+0000 7f8be16b5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc19eed0 con 0x7f8bdc102380 2024-02-17T11:57:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.087+0000 7f8be16b5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc19f090 con 0x7f8bdc101190 2024-02-17T11:57:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.087+0000 7f8bdb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bdc102d40 0x7f8bdc19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.087+0000 7f8bdb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bdc102d40 0x7f8bdc19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53238/0 (socket says 172.21.15.94:53238) 2024-02-17T11:57:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.087+0000 7f8bdb7fe640 1 -- 172.21.15.94:0/3785879465 learned_addr learned my addr 172.21.15.94:0/3785879465 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:55.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.087+0000 7f8bda7fc640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bdc102380 0x7f8bdc1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.087+0000 7f8bdaffd640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bdc101190 0x7f8bdc197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.088+0000 7f8bdb7fe640 1 -- 172.21.15.94:0/3785879465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bdc101190 msgr2=0x7f8bdc197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.088+0000 7f8bdb7fe640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bdc101190 0x7f8bdc197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.088+0000 7f8bdb7fe640 1 -- 172.21.15.94:0/3785879465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bdc102380 msgr2=0x7f8bdc1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.088+0000 7f8bdb7fe640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bdc102380 0x7f8bdc1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.088+0000 7f8bdb7fe640 1 -- 172.21.15.94:0/3785879465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc009900 con 0x7f8bdc102d40 2024-02-17T11:57:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.088+0000 7f8bdaffd640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bdc101190 0x7f8bdc197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.088+0000 7f8bdb7fe640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bdc102d40 0x7f8bdc19e820 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400d950 tx=0x7f8bd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.088+0000 7f8bc3fff640 1 -- 172.21.15.94:0/3785879465 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4009a30 con 0x7f8bdc102d40 2024-02-17T11:57:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.088+0000 7f8be16b5640 1 -- 172.21.15.94:0/3785879465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bdc19f370 con 0x7f8bdc102d40 2024-02-17T11:57:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.088+0000 7f8bc3fff640 1 -- 172.21.15.94:0/3785879465 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bd4009b90 con 0x7f8bdc102d40 2024-02-17T11:57:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.088+0000 7f8bc3fff640 1 -- 172.21.15.94:0/3785879465 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4003d60 con 0x7f8bdc102d40 2024-02-17T11:57:55.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.088+0000 7f8be16b5640 1 -- 172.21.15.94:0/3785879465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bdc19f880 con 0x7f8bdc102d40 2024-02-17T11:57:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.089+0000 7f8be16b5640 1 -- 172.21.15.94:0/3785879465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bdc1085d0 con 0x7f8bdc102d40 2024-02-17T11:57:55.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.090+0000 7f8bc3fff640 1 -- 172.21.15.94:0/3785879465 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f8bd4009d00 con 0x7f8bdc102d40 2024-02-17T11:57:55.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.090+0000 7f8bc3fff640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8bc40a2150 0x7f8bc40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:55.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.091+0000 7f8bdaffd640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8bc40a2150 0x7f8bc40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:55.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.091+0000 7f8bc3fff640 1 -- 172.21.15.94:0/3785879465 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f8bd40cd8e0 con 0x7f8bdc102d40 2024-02-17T11:57:55.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.091+0000 7f8bdaffd640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8bc40a2150 0x7f8bc40a4610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc02c700 tx=0x7f8bcc0326d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:55.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.095+0000 7f8bc3fff640 1 -- 172.21.15.94:0/3785879465 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8bd4096750 con 0x7f8bdc102d40 2024-02-17T11:57:55.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:54 smithi189 ceph-mon[36588]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:55.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:54 smithi189 ceph-mon[36588]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-02-17T11:57:55.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:54 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2706592460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:55.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:54 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3753437988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:55.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:54 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1452096776' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:55.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:54 smithi125 ceph-mon[36819]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:57:55.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:54 smithi125 ceph-mon[36819]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-02-17T11:57:55.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:54 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2706592460' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:55.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:54 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3753437988' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:55.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:54 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1452096776' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:55.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.279+0000 7f8be16b5640 1 -- 172.21.15.94:0/3785879465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8bdc02cdd0 con 0x7f8bdc102d40 2024-02-17T11:57:55.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.279+0000 7f8bc3fff640 1 -- 172.21.15.94:0/3785879465 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8bd4099e10 con 0x7f8bdc102d40 2024-02-17T11:57:55.278 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-02-17T11:57:55.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.281+0000 7f8be16b5640 1 -- 172.21.15.94:0/3785879465 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8bc40a2150 msgr2=0x7f8bc40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.281+0000 7f8be16b5640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8bc40a2150 0x7f8bc40a4610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc02c700 tx=0x7f8bcc0326d0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.281+0000 7f8be16b5640 1 -- 172.21.15.94:0/3785879465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bdc102d40 msgr2=0x7f8bdc19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.281+0000 7f8be16b5640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bdc102d40 0x7f8bdc19e820 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400d950 tx=0x7f8bd400de10 comp rx=0 tx=0).stop 2024-02-17T11:57:55.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.281+0000 7f8be16b5640 1 -- 172.21.15.94:0/3785879465 shutdown_connections 2024-02-17T11:57:55.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.281+0000 7f8be16b5640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f8bc40a2150 0x7f8bc40a4610 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.281+0000 7f8be16b5640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8bdc102d40 0x7f8bdc19e820 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.281+0000 7f8be16b5640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8bdc102380 0x7f8bdc1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.281+0000 7f8be16b5640 1 --2- 172.21.15.94:0/3785879465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bdc101190 0x7f8bdc197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.281+0000 7f8be16b5640 1 -- 172.21.15.94:0/3785879465 >> 172.21.15.94:0/3785879465 conn(0x7f8bdc0fc720 msgr2=0x7f8bdc1057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:55.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.281+0000 7f8be16b5640 1 -- 172.21.15.94:0/3785879465 shutdown_connections 2024-02-17T11:57:55.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.281+0000 7f8be16b5640 1 -- 172.21.15.94:0/3785879465 wait complete. 2024-02-17T11:57:55.291 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T11:57:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.398+0000 7f5173fff640 1 -- 172.21.15.94:0/3907022872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5174101aa0 msgr2=0x7f51741083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.398+0000 7f5173fff640 1 --2- 172.21.15.94:0/3907022872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5174101aa0 0x7f51741083c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f516c009ca0 tx=0x7f516c02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.398+0000 7f5173fff640 1 -- 172.21.15.94:0/3907022872 shutdown_connections 2024-02-17T11:57:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.398+0000 7f5173fff640 1 --2- 172.21.15.94:0/3907022872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5174101aa0 0x7f51741083c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.398+0000 7f5173fff640 1 --2- 172.21.15.94:0/3907022872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f51741010e0 0x7f5174101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.398+0000 7f5173fff640 1 --2- 172.21.15.94:0/3907022872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51740fff10 0x7f5174100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.398+0000 7f5173fff640 1 -- 172.21.15.94:0/3907022872 >> 172.21.15.94:0/3907022872 conn(0x7f51740fb4f0 msgr2=0x7f51740fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:55.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.398+0000 7f5173fff640 1 -- 172.21.15.94:0/3907022872 shutdown_connections 2024-02-17T11:57:55.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.398+0000 7f5173fff640 1 -- 172.21.15.94:0/3907022872 wait complete. 2024-02-17T11:57:55.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.398+0000 7f5173fff640 1 Processor -- start 2024-02-17T11:57:55.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f5173fff640 1 -- start start 2024-02-17T11:57:55.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f5173fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51740fff10 0x7f5174195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:55.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f5173fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51741010e0 0x7f51741964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:55.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f5173fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5174101aa0 0x7f517419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:55.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f5173fff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517419daa0 con 0x7f51741010e0 2024-02-17T11:57:55.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f5173fff640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517419dc10 con 0x7f5174101aa0 2024-02-17T11:57:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f5173fff640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517419ddd0 con 0x7f51740fff10 2024-02-17T11:57:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f516a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51741010e0 0x7f51741964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f516a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51741010e0 0x7f51741964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53268/0 (socket says 172.21.15.94:53268) 2024-02-17T11:57:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f516a7fc640 1 -- 172.21.15.94:0/1999820338 learned_addr learned my addr 172.21.15.94:0/1999820338 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f5172ffd640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51740fff10 0x7f5174195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f516a7fc640 1 -- 172.21.15.94:0/1999820338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51740fff10 msgr2=0x7f5174195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f516a7fc640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51740fff10 0x7f5174195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f516a7fc640 1 -- 172.21.15.94:0/1999820338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5174101aa0 msgr2=0x7f517419d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f51737fe640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5174101aa0 0x7f517419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f516a7fc640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5174101aa0 0x7f517419d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f516a7fc640 1 -- 172.21.15.94:0/1999820338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c009900 con 0x7f51741010e0 2024-02-17T11:57:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.399+0000 7f5172ffd640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51740fff10 0x7f5174195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:55.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.400+0000 7f516a7fc640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51741010e0 0x7f51741964f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f515c00d950 tx=0x7f515c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:55.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.400+0000 7f5178a00640 1 -- 172.21.15.94:0/1999820338 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c009c30 con 0x7f51741010e0 2024-02-17T11:57:55.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.400+0000 7f5173fff640 1 -- 172.21.15.94:0/1999820338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517419e0b0 con 0x7f51741010e0 2024-02-17T11:57:55.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.400+0000 7f5178a00640 1 -- 172.21.15.94:0/1999820338 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f515c009d90 con 0x7f51741010e0 2024-02-17T11:57:55.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.400+0000 7f5178a00640 1 -- 172.21.15.94:0/1999820338 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c00f7e0 con 0x7f51741010e0 2024-02-17T11:57:55.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.400+0000 7f5173fff640 1 -- 172.21.15.94:0/1999820338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517419e610 con 0x7f51741010e0 2024-02-17T11:57:55.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.401+0000 7f5173fff640 1 -- 172.21.15.94:0/1999820338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5174107320 con 0x7f51741010e0 2024-02-17T11:57:55.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.402+0000 7f5178a00640 1 -- 172.21.15.94:0/1999820338 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f515c010ad0 con 0x7f51741010e0 2024-02-17T11:57:55.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.402+0000 7f5178a00640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f51600a2150 0x7f51600a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:55.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.402+0000 7f5178a00640 1 -- 172.21.15.94:0/1999820338 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f515c0cdea0 con 0x7f51741010e0 2024-02-17T11:57:55.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.402+0000 7f5172ffd640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f51600a2150 0x7f51600a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:55.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.403+0000 7f5172ffd640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f51600a2150 0x7f51600a4610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f51640058c0 tx=0x7f5164005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:55.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.407+0000 7f5178a00640 1 -- 172.21.15.94:0/1999820338 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f515c096d10 con 0x7f51741010e0 2024-02-17T11:57:55.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.580+0000 7f5173fff640 1 -- 172.21.15.94:0/1999820338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f517410bb40 con 0x7f51741010e0 2024-02-17T11:57:55.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.581+0000 7f5178a00640 1 -- 172.21.15.94:0/1999820338 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f515c09a3d0 con 0x7f51741010e0 2024-02-17T11:57:55.580 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-02-17T11:57:55.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.583+0000 7f5173fff640 1 -- 172.21.15.94:0/1999820338 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f51600a2150 msgr2=0x7f51600a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.583+0000 7f5173fff640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f51600a2150 0x7f51600a4610 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f51640058c0 tx=0x7f5164005850 comp rx=0 tx=0).stop 2024-02-17T11:57:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.583+0000 7f5173fff640 1 -- 172.21.15.94:0/1999820338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51741010e0 msgr2=0x7f51741964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.583+0000 7f5173fff640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51741010e0 0x7f51741964f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f515c00d950 tx=0x7f515c00de10 comp rx=0 tx=0).stop 2024-02-17T11:57:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.583+0000 7f5173fff640 1 -- 172.21.15.94:0/1999820338 shutdown_connections 2024-02-17T11:57:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.583+0000 7f5173fff640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f51600a2150 0x7f51600a4610 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.583+0000 7f5173fff640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5174101aa0 0x7f517419d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.583+0000 7f5173fff640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f51741010e0 0x7f51741964f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.583+0000 7f5173fff640 1 --2- 172.21.15.94:0/1999820338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51740fff10 0x7f5174195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.583+0000 7f5173fff640 1 -- 172.21.15.94:0/1999820338 >> 172.21.15.94:0/1999820338 conn(0x7f51740fb4f0 msgr2=0x7f51741044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.583+0000 7f5173fff640 1 -- 172.21.15.94:0/1999820338 shutdown_connections 2024-02-17T11:57:55.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.583+0000 7f5173fff640 1 -- 172.21.15.94:0/1999820338 wait complete. 2024-02-17T11:57:55.593 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-02-17T11:57:55.593 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:57:55.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.694+0000 7fadd7c1c640 1 -- 172.21.15.94:0/1646210370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd01011b0 msgr2=0x7fadd01015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.694+0000 7fadd7c1c640 1 --2- 172.21.15.94:0/1646210370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd01011b0 0x7fadd01015d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009ca0 tx=0x7fadcc02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.694+0000 7fadd7c1c640 1 -- 172.21.15.94:0/1646210370 shutdown_connections 2024-02-17T11:57:55.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.694+0000 7fadd7c1c640 1 --2- 172.21.15.94:0/1646210370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadd0102d60 0x7fadd01095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.694+0000 7fadd7c1c640 1 --2- 172.21.15.94:0/1646210370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadd01023a0 0x7fadd0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.694+0000 7fadd7c1c640 1 --2- 172.21.15.94:0/1646210370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd01011b0 0x7fadd01015d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.694+0000 7fadd7c1c640 1 -- 172.21.15.94:0/1646210370 >> 172.21.15.94:0/1646210370 conn(0x7fadd00fc760 msgr2=0x7fadd00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:55.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd7c1c640 1 -- 172.21.15.94:0/1646210370 shutdown_connections 2024-02-17T11:57:55.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd7c1c640 1 -- 172.21.15.94:0/1646210370 wait complete. 2024-02-17T11:57:55.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd7c1c640 1 Processor -- start 2024-02-17T11:57:55.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd7c1c640 1 -- start start 2024-02-17T11:57:55.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd7c1c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadd01011b0 0x7fadd01972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:55.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd7c1c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd01023a0 0x7fadd01977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd7c1c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadd0102d60 0x7fadd019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd7c1c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd019ed90 con 0x7fadd01023a0 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd7c1c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd019ef00 con 0x7fadd01011b0 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd7c1c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd019f0c0 con 0x7fadd0102d60 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd51b7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd01023a0 0x7fadd01977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd51b7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd01023a0 0x7fadd01977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53284/0 (socket says 172.21.15.94:53284) 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd51b7640 1 -- 172.21.15.94:0/3153798007 learned_addr learned my addr 172.21.15.94:0/3153798007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd59b8640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadd01011b0 0x7fadd01972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.695+0000 7fadd61b9640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadd0102d60 0x7fadd019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.696+0000 7fadd51b7640 1 -- 172.21.15.94:0/3153798007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadd0102d60 msgr2=0x7fadd019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.696+0000 7fadd51b7640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadd0102d60 0x7fadd019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.696+0000 7fadd51b7640 1 -- 172.21.15.94:0/3153798007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadd01011b0 msgr2=0x7fadd01972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.696+0000 7fadd51b7640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadd01011b0 0x7fadd01972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.696+0000 7fadd51b7640 1 -- 172.21.15.94:0/3153798007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc009900 con 0x7fadd01023a0 2024-02-17T11:57:55.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.696+0000 7fadd61b9640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadd0102d60 0x7fadd019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.696+0000 7fadd51b7640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd01023a0 0x7fadd01977e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fadc000e9f0 tx=0x7fadc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.696+0000 7fadc6ffd640 1 -- 172.21.15.94:0/3153798007 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0009700 con 0x7fadd01023a0 2024-02-17T11:57:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.696+0000 7fadc6ffd640 1 -- 172.21.15.94:0/3153798007 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadc000cd80 con 0x7fadd01023a0 2024-02-17T11:57:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.696+0000 7fadc6ffd640 1 -- 172.21.15.94:0/3153798007 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0018920 con 0x7fadd01023a0 2024-02-17T11:57:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.696+0000 7fadd7c1c640 1 -- 172.21.15.94:0/3153798007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd019f3a0 con 0x7fadd01023a0 2024-02-17T11:57:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.696+0000 7fadd7c1c640 1 -- 172.21.15.94:0/3153798007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd019f8b0 con 0x7fadd01023a0 2024-02-17T11:57:55.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.697+0000 7fadd7c1c640 1 -- 172.21.15.94:0/3153798007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd0108590 con 0x7fadd01023a0 2024-02-17T11:57:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.698+0000 7fadc6ffd640 1 -- 172.21.15.94:0/3153798007 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fadc0018a80 con 0x7fadd01023a0 2024-02-17T11:57:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.699+0000 7fadc6ffd640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadbc0a2150 0x7fadbc0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.699+0000 7fadd59b8640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadbc0a2150 0x7fadbc0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.699+0000 7fadc6ffd640 1 -- 172.21.15.94:0/3153798007 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fadc0014070 con 0x7fadd01023a0 2024-02-17T11:57:55.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.699+0000 7fadd59b8640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadbc0a2150 0x7fadbc0a4610 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fadcc02c700 tx=0x7fadcc0326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:55.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.703+0000 7fadc6ffd640 1 -- 172.21.15.94:0/3153798007 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadc0052d20 con 0x7fadd01023a0 2024-02-17T11:57:55.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.918+0000 7fadd7c1c640 1 -- 172.21.15.94:0/3153798007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadd010cd30 con 0x7fadd01023a0 2024-02-17T11:57:55.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.918+0000 7fadc6ffd640 1 -- 172.21.15.94:0/3153798007 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11346 (secure 0 0 0) 0x7fadc00970d0 con 0x7fadd01023a0 2024-02-17T11:57:55.917 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:55.917 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:57:51.854453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T11:57:55.918 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 21 2024-02-17T11:57:55.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.920+0000 7fadd7c1c640 1 -- 172.21.15.94:0/3153798007 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadbc0a2150 msgr2=0x7fadbc0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.920+0000 7fadd7c1c640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadbc0a2150 0x7fadbc0a4610 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fadcc02c700 tx=0x7fadcc0326b0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.920+0000 7fadd7c1c640 1 -- 172.21.15.94:0/3153798007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd01023a0 msgr2=0x7fadd01977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:55.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.920+0000 7fadd7c1c640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd01023a0 0x7fadd01977e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fadc000e9f0 tx=0x7fadc000eeb0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.920+0000 7fadd7c1c640 1 -- 172.21.15.94:0/3153798007 shutdown_connections 2024-02-17T11:57:55.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.920+0000 7fadd7c1c640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadbc0a2150 0x7fadbc0a4610 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.920+0000 7fadd7c1c640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadd0102d60 0x7fadd019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.920+0000 7fadd7c1c640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadd01023a0 0x7fadd01977e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.920+0000 7fadd7c1c640 1 --2- 172.21.15.94:0/3153798007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadd01011b0 0x7fadd01972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:55.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.920+0000 7fadd7c1c640 1 -- 172.21.15.94:0/3153798007 >> 172.21.15.94:0/3153798007 conn(0x7fadd00fc760 msgr2=0x7fadd0105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:55.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.921+0000 7fadd7c1c640 1 -- 172.21.15.94:0/3153798007 shutdown_connections 2024-02-17T11:57:55.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:55.921+0000 7fadd7c1c640 1 -- 172.21.15.94:0/3153798007 wait complete. 2024-02-17T11:57:55.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:55 smithi094 ceph-mon[32499]: pgmap v345: 97 pgs: 97 active+clean; 647 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-02-17T11:57:55.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:55 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2589401321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:55.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:55 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3785879465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:55.929 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:55 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1999820338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:55.930 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-02-17T11:57:55.931 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-02-17T11:57:55.931 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-17T11:57:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.033+0000 7fe4c2793640 1 -- 172.21.15.94:0/2740615317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4bc1045d0 msgr2=0x7fe4bc1069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.033+0000 7fe4c2793640 1 --2- 172.21.15.94:0/2740615317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4bc1045d0 0x7fe4bc1069c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009900 tx=0x7fe4ac02bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.034+0000 7fe4c2793640 1 -- 172.21.15.94:0/2740615317 shutdown_connections 2024-02-17T11:57:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.034+0000 7fe4c2793640 1 --2- 172.21.15.94:0/2740615317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4bc106f00 0x7fe4bc1092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.034+0000 7fe4c2793640 1 --2- 172.21.15.94:0/2740615317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4bc1045d0 0x7fe4bc1069c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.034+0000 7fe4c2793640 1 --2- 172.21.15.94:0/2740615317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4bc101ca0 0x7fe4bc104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.034+0000 7fe4c2793640 1 -- 172.21.15.94:0/2740615317 >> 172.21.15.94:0/2740615317 conn(0x7fe4bc0fb630 msgr2=0x7fe4bc0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.034+0000 7fe4c2793640 1 -- 172.21.15.94:0/2740615317 shutdown_connections 2024-02-17T11:57:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.034+0000 7fe4c2793640 1 -- 172.21.15.94:0/2740615317 wait complete. 2024-02-17T11:57:56.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.034+0000 7fe4c2793640 1 Processor -- start 2024-02-17T11:57:56.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.034+0000 7fe4c2793640 1 -- start start 2024-02-17T11:57:56.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.034+0000 7fe4c2793640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4bc101ca0 0x7fe4bc197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:56.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c2793640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4bc1045d0 0x7fe4bc197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:56.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c2793640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4bc106f00 0x7fe4bc19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:56.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c2793640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc19ee10 con 0x7fe4bc106f00 2024-02-17T11:57:56.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c2793640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc19ef80 con 0x7fe4bc101ca0 2024-02-17T11:57:56.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c2793640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc19f140 con 0x7fe4bc1045d0 2024-02-17T11:57:56.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c0d30640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4bc106f00 0x7fe4bc19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:56.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c0d30640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4bc106f00 0x7fe4bc19e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53310/0 (socket says 172.21.15.94:53310) 2024-02-17T11:57:56.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c0d30640 1 -- 172.21.15.94:0/2893975081 learned_addr learned my addr 172.21.15.94:0/2893975081 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:56.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4bbfff640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4bc101ca0 0x7fe4bc197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:56.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4bb7fe640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4bc1045d0 0x7fe4bc197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:56.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c0d30640 1 -- 172.21.15.94:0/2893975081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4bc1045d0 msgr2=0x7fe4bc197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c0d30640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4bc1045d0 0x7fe4bc197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c0d30640 1 -- 172.21.15.94:0/2893975081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4bc101ca0 msgr2=0x7fe4bc197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c0d30640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4bc101ca0 0x7fe4bc197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c0d30640 1 -- 172.21.15.94:0/2893975081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac009560 con 0x7fe4bc106f00 2024-02-17T11:57:56.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4bb7fe640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4bc1045d0 0x7fe4bc197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:56.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c0d30640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4bc106f00 0x7fe4bc19e8d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400eaa0 tx=0x7fe4b400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:56.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4b97fa640 1 -- 172.21.15.94:0/2893975081 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400cdb0 con 0x7fe4bc106f00 2024-02-17T11:57:56.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c2793640 1 -- 172.21.15.94:0/2893975081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4bc19f420 con 0x7fe4bc106f00 2024-02-17T11:57:56.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4b97fa640 1 -- 172.21.15.94:0/2893975081 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4b4005020 con 0x7fe4bc106f00 2024-02-17T11:57:56.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4b97fa640 1 -- 172.21.15.94:0/2893975081 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b40116b0 con 0x7fe4bc106f00 2024-02-17T11:57:56.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.035+0000 7fe4c2793640 1 -- 172.21.15.94:0/2893975081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4bc19f930 con 0x7fe4bc106f00 2024-02-17T11:57:56.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.036+0000 7fe4c2793640 1 -- 172.21.15.94:0/2893975081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4bc0fd120 con 0x7fe4bc106f00 2024-02-17T11:57:56.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.037+0000 7fe4b97fa640 1 -- 172.21.15.94:0/2893975081 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe4b4005520 con 0x7fe4bc106f00 2024-02-17T11:57:56.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.038+0000 7fe4b97fa640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe4a40a2150 0x7fe4a40a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:56.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.038+0000 7fe4bbfff640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe4a40a2150 0x7fe4a40a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:56.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.038+0000 7fe4b97fa640 1 -- 172.21.15.94:0/2893975081 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe4b40cdfb0 con 0x7fe4bc106f00 2024-02-17T11:57:56.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.038+0000 7fe4bbfff640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe4a40a2150 0x7fe4a40a4610 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0005fd0 tx=0x7fe4b000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:56.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.042+0000 7fe4b97fa640 1 -- 172.21.15.94:0/2893975081 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe4b4096ee0 con 0x7fe4bc106f00 2024-02-17T11:57:56.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:55 smithi189 ceph-mon[36588]: pgmap v345: 97 pgs: 97 active+clean; 647 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-02-17T11:57:56.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:55 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2589401321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:56.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:55 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3785879465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:56.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:55 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1999820338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:56.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:55 smithi125 ceph-mon[36819]: pgmap v345: 97 pgs: 97 active+clean; 647 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1023 B/s wr, 7 op/s 2024-02-17T11:57:56.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:55 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2589401321' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:56.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:55 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3785879465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:56.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:55 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1999820338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:56.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.212+0000 7fe4c2793640 1 -- 172.21.15.94:0/2893975081 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4bc02ce40 con 0x7fe4a40a2150 2024-02-17T11:57:56.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.223+0000 7fe4b97fa640 1 -- 172.21.15.94:0/2893975081 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345529 (secure 0 0 0) 0x7fe4bc02ce40 con 0x7fe4a40a2150 2024-02-17T11:57:56.226 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:56.228 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-02-17T11:57:56.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.227+0000 7fe4c2793640 1 -- 172.21.15.94:0/2893975081 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe4a40a2150 msgr2=0x7fe4a40a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.227+0000 7fe4c2793640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe4a40a2150 0x7fe4a40a4610 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0005fd0 tx=0x7fe4b000b390 comp rx=0 tx=0).stop 2024-02-17T11:57:56.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.227+0000 7fe4c2793640 1 -- 172.21.15.94:0/2893975081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4bc106f00 msgr2=0x7fe4bc19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.227+0000 7fe4c2793640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4bc106f00 0x7fe4bc19e8d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400eaa0 tx=0x7fe4b400ef60 comp rx=0 tx=0).stop 2024-02-17T11:57:56.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.227+0000 7fe4c2793640 1 -- 172.21.15.94:0/2893975081 shutdown_connections 2024-02-17T11:57:56.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.227+0000 7fe4c2793640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe4a40a2150 0x7fe4a40a4610 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.227+0000 7fe4c2793640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4bc106f00 0x7fe4bc19e8d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.227+0000 7fe4c2793640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4bc1045d0 0x7fe4bc197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.227+0000 7fe4c2793640 1 --2- 172.21.15.94:0/2893975081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4bc101ca0 0x7fe4bc197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.227+0000 7fe4c2793640 1 -- 172.21.15.94:0/2893975081 >> 172.21.15.94:0/2893975081 conn(0x7fe4bc0fb630 msgr2=0x7fe4bc103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:56.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.227+0000 7fe4c2793640 1 -- 172.21.15.94:0/2893975081 shutdown_connections 2024-02-17T11:57:56.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.227+0000 7fe4c2793640 1 -- 172.21.15.94:0/2893975081 wait complete. 2024-02-17T11:57:56.237 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-02-17T11:57:54.680570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662643,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":833,"num_read_kb":545,"num_write":358,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364884,"kb_used_data":11572,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576940,"statfs":{"total":1151940427776,"available":1151566786560,"internally_reserved":0,"allocated":11849728,"data_stored":7449595,"data_compressed":184182,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6072,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":9,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002682"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.673752+0000","last_change":"2024-02-17T11:55:58.678553+0000","last_active":"2024-02-17T11:57:28.673752+0000","last_peered":"2024-02-17T11:57:28.673752+0000","last_clean":"2024-02-17T11:57:28.673752+0000","last_became_active":"2024-02-17T11:55:58.678212+0000","last_became_peered":"2024-02-17T11:55:58.678212+0000","last_unstale":"2024-02-17T11:57:28.673752+0000","last_undegraded":"2024-02-17T11:57:28.673752+0000","last_fullsized":"2024-02-17T11:57:28.673752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:09:18.011937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672220+0000","last_change":"2024-02-17T11:55:58.681666+0000","last_active":"2024-02-17T11:57:28.672220+0000","last_peered":"2024-02-17T11:57:28.672220+0000","last_clean":"2024-02-17T11:57:28.672220+0000","last_became_active":"2024-02-17T11:55:58.681305+0000","last_became_peered":"2024-02-17T11:55:58.681305+0000","last_unstale":"2024-02-17T11:57:28.672220+0000","last_undegraded":"2024-02-17T11:57:28.672220+0000","last_fullsized":"2024-02-17T11:57:28.672220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:40:03.514130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.682419+0000","last_change":"2024-02-17T11:55:58.689465+0000","last_active":"2024-02-17T11:57:28.682419+0000","last_peered":"2024-02-17T11:57:28.682419+0000","last_clean":"2024-02-17T11:57:28.682419+0000","last_became_active":"2024-02-17T11:55:58.689166+0000","last_became_peered":"2024-02-17T11:55:58.689166+0000","last_unstale":"2024-02-17T11:57:28.682419+0000","last_undegraded":"2024-02-17T11:57:28.682419+0000","last_fullsized":"2024-02-17T11:57:28.682419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:43:07.473303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.652932+0000","last_change":"2024-02-17T11:56:59.735287+0000","last_active":"2024-02-17T11:57:28.652932+0000","last_peered":"2024-02-17T11:57:28.652932+0000","last_clean":"2024-02-17T11:57:28.652932+0000","last_became_active":"2024-02-17T11:56:59.734916+0000","last_became_peered":"2024-02-17T11:56:59.734916+0000","last_unstale":"2024-02-17T11:57:28.652932+0000","last_undegraded":"2024-02-17T11:57:28.652932+0000","last_fullsized":"2024-02-17T11:57:28.652932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:25:03.955834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.669070+0000","last_change":"2024-02-17T11:55:58.677543+0000","last_active":"2024-02-17T11:57:28.669070+0000","last_peered":"2024-02-17T11:57:28.669070+0000","last_clean":"2024-02-17T11:57:28.669070+0000","last_became_active":"2024-02-17T11:55:58.677223+0000","last_became_peered":"2024-02-17T11:55:58.677223+0000","last_unstale":"2024-02-17T11:57:28.669070+0000","last_undegraded":"2024-02-17T11:57:28.669070+0000","last_fullsized":"2024-02-17T11:57:28.669070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:05:27.194963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.677305+0000","last_change":"2024-02-17T11:55:58.673756+0000","last_active":"2024-02-17T11:57:28.677305+0000","last_peered":"2024-02-17T11:57:28.677305+0000","last_clean":"2024-02-17T11:57:28.677305+0000","last_became_active":"2024-02-17T11:55:58.672775+0000","last_became_peered":"2024-02-17T11:55:58.672775+0000","last_unstale":"2024-02-17T11:57:28.677305+0000","last_undegraded":"2024-02-17T11:57:28.677305+0000","last_fullsized":"2024-02-17T11:57:28.677305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:12:18.658414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671796+0000","last_change":"2024-02-17T11:56:59.736210+0000","last_active":"2024-02-17T11:57:28.671796+0000","last_peered":"2024-02-17T11:57:28.671796+0000","last_clean":"2024-02-17T11:57:28.671796+0000","last_became_active":"2024-02-17T11:56:59.735834+0000","last_became_peered":"2024-02-17T11:56:59.735834+0000","last_unstale":"2024-02-17T11:57:28.671796+0000","last_undegraded":"2024-02-17T11:57:28.671796+0000","last_fullsized":"2024-02-17T11:57:28.671796+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:19:27.677453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671910+0000","last_change":"2024-02-17T11:55:58.675496+0000","last_active":"2024-02-17T11:57:28.671910+0000","last_peered":"2024-02-17T11:57:28.671910+0000","last_clean":"2024-02-17T11:57:28.671910+0000","last_became_active":"2024-02-17T11:55:58.675193+0000","last_became_peered":"2024-02-17T11:55:58.675193+0000","last_unstale":"2024-02-17T11:57:28.671910+0000","last_undegraded":"2024-02-17T11:57:28.671910+0000","last_fullsized":"2024-02-17T11:57:28.671910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:14:29.648857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685041+0000","last_change":"2024-02-17T11:55:58.680859+0000","last_active":"2024-02-17T11:57:28.685041+0000","last_peered":"2024-02-17T11:57:28.685041+0000","last_clean":"2024-02-17T11:57:28.685041+0000","last_became_active":"2024-02-17T11:55:58.680489+0000","last_became_peered":"2024-02-17T11:55:58.680489+0000","last_unstale":"2024-02-17T11:57:28.685041+0000","last_undegraded":"2024-02-17T11:57:28.685041+0000","last_fullsized":"2024-02-17T11:57:28.685041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:29.963349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.674284+0000","last_change":"2024-02-17T11:56:59.737386+0000","last_active":"2024-02-17T11:57:28.674284+0000","last_peered":"2024-02-17T11:57:28.674284+0000","last_clean":"2024-02-17T11:57:28.674284+0000","last_became_active":"2024-02-17T11:56:59.736973+0000","last_became_peered":"2024-02-17T11:56:59.736973+0000","last_unstale":"2024-02-17T11:57:28.674284+0000","last_undegraded":"2024-02-17T11:57:28.674284+0000","last_fullsized":"2024-02-17T11:57:28.674284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:11:46.180288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.683763+0000","last_change":"2024-02-17T11:55:58.688562+0000","last_active":"2024-02-17T11:57:28.683763+0000","last_peered":"2024-02-17T11:57:28.683763+0000","last_clean":"2024-02-17T11:57:28.683763+0000","last_became_active":"2024-02-17T11:55:58.688186+0000","last_became_peered":"2024-02-17T11:55:58.688186+0000","last_unstale":"2024-02-17T11:57:28.683763+0000","last_undegraded":"2024-02-17T11:57:28.683763+0000","last_fullsized":"2024-02-17T11:57:28.683763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:09.668937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.676809+0000","last_change":"2024-02-17T11:55:58.684989+0000","last_active":"2024-02-17T11:57:28.676809+0000","last_peered":"2024-02-17T11:57:28.676809+0000","last_clean":"2024-02-17T11:57:28.676809+0000","last_became_active":"2024-02-17T11:55:58.684656+0000","last_became_peered":"2024-02-17T11:55:58.684656+0000","last_unstale":"2024-02-17T11:57:28.676809+0000","last_undegraded":"2024-02-17T11:57:28.676809+0000","last_fullsized":"2024-02-17T11:57:28.676809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:29:43.480963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.667676+0000","last_change":"2024-02-17T11:55:58.662076+0000","last_active":"2024-02-17T11:57:28.667676+0000","last_peered":"2024-02-17T11:57:28.667676+0000","last_clean":"2024-02-17T11:57:28.667676+0000","last_became_active":"2024-02-17T11:55:58.661716+0000","last_became_peered":"2024-02-17T11:55:58.661716+0000","last_unstale":"2024-02-17T11:57:28.667676+0000","last_undegraded":"2024-02-17T11:57:28.667676+0000","last_fullsized":"2024-02-17T11:57:28.667676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:20:39.647774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671962+0000","last_change":"2024-02-17T11:55:58.690762+0000","last_active":"2024-02-17T11:57:28.671962+0000","last_peered":"2024-02-17T11:57:28.671962+0000","last_clean":"2024-02-17T11:57:28.671962+0000","last_became_active":"2024-02-17T11:55:58.690414+0000","last_became_peered":"2024-02-17T11:55:58.690414+0000","last_unstale":"2024-02-17T11:57:28.671962+0000","last_undegraded":"2024-02-17T11:57:28.671962+0000","last_fullsized":"2024-02-17T11:57:28.671962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:37:13.938474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.674249+0000","last_change":"2024-02-17T11:55:58.682391+0000","last_active":"2024-02-17T11:57:28.674249+0000","last_peered":"2024-02-17T11:57:28.674249+0000","last_clean":"2024-02-17T11:57:28.674249+0000","last_became_active":"2024-02-17T11:55:58.682048+0000","last_became_peered":"2024-02-17T11:55:58.682048+0000","last_unstale":"2024-02-17T11:57:28.674249+0000","last_undegraded":"2024-02-17T11:57:28.674249+0000","last_fullsized":"2024-02-17T11:57:28.674249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:03:51.667202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685713+0000","last_change":"2024-02-17T11:55:58.685782+0000","last_active":"2024-02-17T11:57:28.685713+0000","last_peered":"2024-02-17T11:57:28.685713+0000","last_clean":"2024-02-17T11:57:28.685713+0000","last_became_active":"2024-02-17T11:55:58.683087+0000","last_became_peered":"2024-02-17T11:55:58.683087+0000","last_unstale":"2024-02-17T11:57:28.685713+0000","last_undegraded":"2024-02-17T11:57:28.685713+0000","last_fullsized":"2024-02-17T11:57:28.685713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:30:39.653717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.684630+0000","last_change":"2024-02-17T11:55:58.688400+0000","last_active":"2024-02-17T11:57:28.684630+0000","last_peered":"2024-02-17T11:57:28.684630+0000","last_clean":"2024-02-17T11:57:28.684630+0000","last_became_active":"2024-02-17T11:55:58.688040+0000","last_became_peered":"2024-02-17T11:55:58.688040+0000","last_unstale":"2024-02-17T11:57:28.684630+0000","last_undegraded":"2024-02-17T11:57:28.684630+0000","last_fullsized":"2024-02-17T11:57:28.684630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:39:02.706547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.684948+0000","last_change":"2024-02-17T11:55:58.687454+0000","last_active":"2024-02-17T11:57:28.684948+0000","last_peered":"2024-02-17T11:57:28.684948+0000","last_clean":"2024-02-17T11:57:28.684948+0000","last_became_active":"2024-02-17T11:55:58.686432+0000","last_became_peered":"2024-02-17T11:55:58.686432+0000","last_unstale":"2024-02-17T11:57:28.684948+0000","last_undegraded":"2024-02-17T11:57:28.684948+0000","last_fullsized":"2024-02-17T11:57:28.684948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:26:31.497782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670573+0000","last_change":"2024-02-17T11:55:58.681806+0000","last_active":"2024-02-17T11:57:28.670573+0000","last_peered":"2024-02-17T11:57:28.670573+0000","last_clean":"2024-02-17T11:57:28.670573+0000","last_became_active":"2024-02-17T11:55:58.679764+0000","last_became_peered":"2024-02-17T11:55:58.679764+0000","last_unstale":"2024-02-17T11:57:28.670573+0000","last_undegraded":"2024-02-17T11:57:28.670573+0000","last_fullsized":"2024-02-17T11:57:28.670573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:13:14.733417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.887970+0000","last_change":"2024-02-17T11:56:59.887970+0000","last_active":"2024-02-17T11:56:59.887970+0000","last_peered":"2024-02-17T11:56:59.887970+0000","last_clean":"2024-02-17T11:56:59.887970+0000","last_became_active":"2024-02-17T11:56:59.882087+0000","last_became_peered":"2024-02-17T11:56:59.882087+0000","last_unstale":"2024-02-17T11:56:59.887970+0000","last_undegraded":"2024-02-17T11:56:59.887970+0000","last_fullsized":"2024-02-17T11:56:59.887970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:34:08.202837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672008+0000","last_change":"2024-02-17T11:55:58.690679+0000","last_active":"2024-02-17T11:57:28.672008+0000","last_peered":"2024-02-17T11:57:28.672008+0000","last_clean":"2024-02-17T11:57:28.672008+0000","last_became_active":"2024-02-17T11:55:58.690353+0000","last_became_peered":"2024-02-17T11:55:58.690353+0000","last_unstale":"2024-02-17T11:57:28.672008+0000","last_undegraded":"2024-02-17T11:57:28.672008+0000","last_fullsized":"2024-02-17T11:57:28.672008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:45:33.830784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733040+0000","last_change":"2024-02-17T11:55:57.677649+0000","last_active":"2024-02-17T11:56:59.733040+0000","last_peered":"2024-02-17T11:56:59.733040+0000","last_clean":"2024-02-17T11:56:59.733040+0000","last_became_active":"2024-02-17T11:55:57.677331+0000","last_became_peered":"2024-02-17T11:55:57.677331+0000","last_unstale":"2024-02-17T11:56:59.733040+0000","last_undegraded":"2024-02-17T11:56:59.733040+0000","last_fullsized":"2024-02-17T11:56:59.733040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:06.932255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.684630+0000","last_change":"2024-02-17T11:55:58.688575+0000","last_active":"2024-02-17T11:57:28.684630+0000","last_peered":"2024-02-17T11:57:28.684630+0000","last_clean":"2024-02-17T11:57:28.684630+0000","last_became_active":"2024-02-17T11:55:58.688238+0000","last_became_peered":"2024-02-17T11:55:58.688238+0000","last_unstale":"2024-02-17T11:57:28.684630+0000","last_undegraded":"2024-02-17T11:57:28.684630+0000","last_fullsized":"2024-02-17T11:57:28.684630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:17:19.774986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729622+0000","last_change":"2024-02-17T11:55:57.679676+0000","last_active":"2024-02-17T11:56:59.729622+0000","last_peered":"2024-02-17T11:56:59.729622+0000","last_clean":"2024-02-17T11:56:59.729622+0000","last_became_active":"2024-02-17T11:55:57.679350+0000","last_became_peered":"2024-02-17T11:55:57.679350+0000","last_unstale":"2024-02-17T11:56:59.729622+0000","last_undegraded":"2024-02-17T11:56:59.729622+0000","last_fullsized":"2024-02-17T11:56:59.729622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:52:45.072896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685733+0000","last_change":"2024-02-17T11:55:58.682212+0000","last_active":"2024-02-17T11:57:28.685733+0000","last_peered":"2024-02-17T11:57:28.685733+0000","last_clean":"2024-02-17T11:57:28.685733+0000","last_became_active":"2024-02-17T11:55:58.681773+0000","last_became_peered":"2024-02-17T11:55:58.681773+0000","last_unstale":"2024-02-17T11:57:28.685733+0000","last_undegraded":"2024-02-17T11:57:28.685733+0000","last_fullsized":"2024-02-17T11:57:28.685733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:28:18.577174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.741288+0000","last_change":"2024-02-17T11:56:59.741288+0000","last_active":"2024-02-17T11:56:59.741288+0000","last_peered":"2024-02-17T11:56:59.741288+0000","last_clean":"2024-02-17T11:56:59.741288+0000","last_became_active":"2024-02-17T11:56:59.734687+0000","last_became_peered":"2024-02-17T11:56:59.734687+0000","last_unstale":"2024-02-17T11:56:59.741288+0000","last_undegraded":"2024-02-17T11:56:59.741288+0000","last_fullsized":"2024-02-17T11:56:59.741288+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:49:15.242774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.678159+0000","last_change":"2024-02-17T11:55:58.672771+0000","last_active":"2024-02-17T11:57:28.678159+0000","last_peered":"2024-02-17T11:57:28.678159+0000","last_clean":"2024-02-17T11:57:28.678159+0000","last_became_active":"2024-02-17T11:55:58.671877+0000","last_became_peered":"2024-02-17T11:55:58.671877+0000","last_unstale":"2024-02-17T11:57:28.678159+0000","last_undegraded":"2024-02-17T11:57:28.678159+0000","last_fullsized":"2024-02-17T11:57:28.678159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:38:38.350695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":139,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:49.315935+0000","last_change":"2024-02-17T11:55:57.669396+0000","last_active":"2024-02-17T11:57:49.315935+0000","last_peered":"2024-02-17T11:57:49.315935+0000","last_clean":"2024-02-17T11:57:49.315935+0000","last_became_active":"2024-02-17T11:55:57.669116+0000","last_became_peered":"2024-02-17T11:55:57.669116+0000","last_unstale":"2024-02-17T11:57:49.315935+0000","last_undegraded":"2024-02-17T11:57:49.315935+0000","last_fullsized":"2024-02-17T11:57:49.315935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:05:11.081309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.659888+0000","last_change":"2024-02-17T11:55:58.674379+0000","last_active":"2024-02-17T11:57:28.659888+0000","last_peered":"2024-02-17T11:57:28.659888+0000","last_clean":"2024-02-17T11:57:28.659888+0000","last_became_active":"2024-02-17T11:55:58.674015+0000","last_became_peered":"2024-02-17T11:55:58.674015+0000","last_unstale":"2024-02-17T11:57:28.659888+0000","last_undegraded":"2024-02-17T11:57:28.659888+0000","last_fullsized":"2024-02-17T11:57:28.659888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:44:20.410623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.734826+0000","last_change":"2024-02-17T11:55:57.668436+0000","last_active":"2024-02-17T11:56:59.734826+0000","last_peered":"2024-02-17T11:56:59.734826+0000","last_clean":"2024-02-17T11:56:59.734826+0000","last_became_active":"2024-02-17T11:55:57.668004+0000","last_became_peered":"2024-02-17T11:55:57.668004+0000","last_unstale":"2024-02-17T11:56:59.734826+0000","last_undegraded":"2024-02-17T11:56:59.734826+0000","last_fullsized":"2024-02-17T11:56:59.734826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:13:42.566644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.678282+0000","last_change":"2024-02-17T11:55:58.686888+0000","last_active":"2024-02-17T11:57:28.678282+0000","last_peered":"2024-02-17T11:57:28.678282+0000","last_clean":"2024-02-17T11:57:28.678282+0000","last_became_active":"2024-02-17T11:55:58.686376+0000","last_became_peered":"2024-02-17T11:55:58.686376+0000","last_unstale":"2024-02-17T11:57:28.678282+0000","last_undegraded":"2024-02-17T11:57:28.678282+0000","last_fullsized":"2024-02-17T11:57:28.678282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:58:48.292354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731458+0000","last_change":"2024-02-17T11:55:57.665717+0000","last_active":"2024-02-17T11:56:59.731458+0000","last_peered":"2024-02-17T11:56:59.731458+0000","last_clean":"2024-02-17T11:56:59.731458+0000","last_became_active":"2024-02-17T11:55:57.665381+0000","last_became_peered":"2024-02-17T11:55:57.665381+0000","last_unstale":"2024-02-17T11:56:59.731458+0000","last_undegraded":"2024-02-17T11:56:59.731458+0000","last_fullsized":"2024-02-17T11:56:59.731458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:15:27.424332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670008+0000","last_change":"2024-02-17T11:55:58.673261+0000","last_active":"2024-02-17T11:57:28.670008+0000","last_peered":"2024-02-17T11:57:28.670008+0000","last_clean":"2024-02-17T11:57:28.670008+0000","last_became_active":"2024-02-17T11:55:58.672912+0000","last_became_peered":"2024-02-17T11:55:58.672912+0000","last_unstale":"2024-02-17T11:57:28.670008+0000","last_undegraded":"2024-02-17T11:57:28.670008+0000","last_fullsized":"2024-02-17T11:57:28.670008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:42.605291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732598+0000","last_change":"2024-02-17T11:55:57.667792+0000","last_active":"2024-02-17T11:56:59.732598+0000","last_peered":"2024-02-17T11:56:59.732598+0000","last_clean":"2024-02-17T11:56:59.732598+0000","last_became_active":"2024-02-17T11:55:57.667361+0000","last_became_peered":"2024-02-17T11:55:57.667361+0000","last_unstale":"2024-02-17T11:56:59.732598+0000","last_undegraded":"2024-02-17T11:56:59.732598+0000","last_fullsized":"2024-02-17T11:56:59.732598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:48.073742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.668179+0000","last_change":"2024-02-17T11:55:58.672180+0000","last_active":"2024-02-17T11:57:28.668179+0000","last_peered":"2024-02-17T11:57:28.668179+0000","last_clean":"2024-02-17T11:57:28.668179+0000","last_became_active":"2024-02-17T11:55:58.671790+0000","last_became_peered":"2024-02-17T11:55:58.671790+0000","last_unstale":"2024-02-17T11:57:28.668179+0000","last_undegraded":"2024-02-17T11:57:28.668179+0000","last_fullsized":"2024-02-17T11:57:28.668179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:48:39.982924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":136,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:53.110454+0000","last_change":"2024-02-17T11:55:57.670085+0000","last_active":"2024-02-17T11:57:53.110454+0000","last_peered":"2024-02-17T11:57:53.110454+0000","last_clean":"2024-02-17T11:57:53.110454+0000","last_became_active":"2024-02-17T11:55:57.669648+0000","last_became_peered":"2024-02-17T11:55:57.669648+0000","last_unstale":"2024-02-17T11:57:53.110454+0000","last_undegraded":"2024-02-17T11:57:53.110454+0000","last_fullsized":"2024-02-17T11:57:53.110454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:31:34.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685714+0000","last_change":"2024-02-17T11:55:58.685080+0000","last_active":"2024-02-17T11:57:28.685714+0000","last_peered":"2024-02-17T11:57:28.685714+0000","last_clean":"2024-02-17T11:57:28.685714+0000","last_became_active":"2024-02-17T11:55:58.684763+0000","last_became_peered":"2024-02-17T11:55:58.684763+0000","last_unstale":"2024-02-17T11:57:28.685714+0000","last_undegraded":"2024-02-17T11:57:28.685714+0000","last_fullsized":"2024-02-17T11:57:28.685714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:59:53.763492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732988+0000","last_change":"2024-02-17T11:55:57.676391+0000","last_active":"2024-02-17T11:56:59.732988+0000","last_peered":"2024-02-17T11:56:59.732988+0000","last_clean":"2024-02-17T11:56:59.732988+0000","last_became_active":"2024-02-17T11:55:57.676103+0000","last_became_peered":"2024-02-17T11:55:57.676103+0000","last_unstale":"2024-02-17T11:56:59.732988+0000","last_undegraded":"2024-02-17T11:56:59.732988+0000","last_fullsized":"2024-02-17T11:56:59.732988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:37:52.866379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670655+0000","last_change":"2024-02-17T11:55:58.679274+0000","last_active":"2024-02-17T11:57:28.670655+0000","last_peered":"2024-02-17T11:57:28.670655+0000","last_clean":"2024-02-17T11:57:28.670655+0000","last_became_active":"2024-02-17T11:55:58.678945+0000","last_became_peered":"2024-02-17T11:55:58.678945+0000","last_unstale":"2024-02-17T11:57:28.670655+0000","last_undegraded":"2024-02-17T11:57:28.670655+0000","last_fullsized":"2024-02-17T11:57:28.670655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:12:20.502503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733259+0000","last_change":"2024-02-17T11:55:57.672732+0000","last_active":"2024-02-17T11:56:59.733259+0000","last_peered":"2024-02-17T11:56:59.733259+0000","last_clean":"2024-02-17T11:56:59.733259+0000","last_became_active":"2024-02-17T11:55:57.672409+0000","last_became_peered":"2024-02-17T11:55:57.672409+0000","last_unstale":"2024-02-17T11:56:59.733259+0000","last_undegraded":"2024-02-17T11:56:59.733259+0000","last_fullsized":"2024-02-17T11:56:59.733259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:07:31.556508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685399+0000","last_change":"2024-02-17T11:55:58.689983+0000","last_active":"2024-02-17T11:57:28.685399+0000","last_peered":"2024-02-17T11:57:28.685399+0000","last_clean":"2024-02-17T11:57:28.685399+0000","last_became_active":"2024-02-17T11:55:58.689629+0000","last_became_peered":"2024-02-17T11:55:58.689629+0000","last_unstale":"2024-02-17T11:57:28.685399+0000","last_undegraded":"2024-02-17T11:57:28.685399+0000","last_fullsized":"2024-02-17T11:57:28.685399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:12.321746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":289,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:53.344302+0000","last_change":"2024-02-17T11:55:57.673273+0000","last_active":"2024-02-17T11:57:53.344302+0000","last_peered":"2024-02-17T11:57:53.344302+0000","last_clean":"2024-02-17T11:57:53.344302+0000","last_became_active":"2024-02-17T11:55:57.672865+0000","last_became_peered":"2024-02-17T11:55:57.672865+0000","last_unstale":"2024-02-17T11:57:53.344302+0000","last_undegraded":"2024-02-17T11:57:53.344302+0000","last_fullsized":"2024-02-17T11:57:53.344302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:31:01.593798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67367,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":235,"num_read_kb":183,"num_write":36,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670386+0000","last_change":"2024-02-17T11:55:58.669296+0000","last_active":"2024-02-17T11:57:28.670386+0000","last_peered":"2024-02-17T11:57:28.670386+0000","last_clean":"2024-02-17T11:57:28.670386+0000","last_became_active":"2024-02-17T11:55:58.668927+0000","last_became_peered":"2024-02-17T11:55:58.668927+0000","last_unstale":"2024-02-17T11:57:28.670386+0000","last_undegraded":"2024-02-17T11:57:28.670386+0000","last_fullsized":"2024-02-17T11:57:28.670386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:25.177905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733466+0000","last_change":"2024-02-17T11:55:57.674523+0000","last_active":"2024-02-17T11:56:59.733466+0000","last_peered":"2024-02-17T11:56:59.733466+0000","last_clean":"2024-02-17T11:56:59.733466+0000","last_became_active":"2024-02-17T11:55:57.672375+0000","last_became_peered":"2024-02-17T11:55:57.672375+0000","last_unstale":"2024-02-17T11:56:59.733466+0000","last_undegraded":"2024-02-17T11:56:59.733466+0000","last_fullsized":"2024-02-17T11:56:59.733466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:15:21.568434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.666966+0000","last_change":"2024-02-17T11:55:58.682106+0000","last_active":"2024-02-17T11:57:28.666966+0000","last_peered":"2024-02-17T11:57:28.666966+0000","last_clean":"2024-02-17T11:57:28.666966+0000","last_became_active":"2024-02-17T11:55:58.681420+0000","last_became_peered":"2024-02-17T11:55:58.681420+0000","last_unstale":"2024-02-17T11:57:28.666966+0000","last_undegraded":"2024-02-17T11:57:28.666966+0000","last_fullsized":"2024-02-17T11:57:28.666966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:23:00.521690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732537+0000","last_change":"2024-02-17T11:55:57.675161+0000","last_active":"2024-02-17T11:56:59.732537+0000","last_peered":"2024-02-17T11:56:59.732537+0000","last_clean":"2024-02-17T11:56:59.732537+0000","last_became_active":"2024-02-17T11:55:57.674776+0000","last_became_peered":"2024-02-17T11:55:57.674776+0000","last_unstale":"2024-02-17T11:56:59.732537+0000","last_undegraded":"2024-02-17T11:56:59.732537+0000","last_fullsized":"2024-02-17T11:56:59.732537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:25:59.695014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.680450+0000","last_change":"2024-02-17T11:55:58.679319+0000","last_active":"2024-02-17T11:57:28.680450+0000","last_peered":"2024-02-17T11:57:28.680450+0000","last_clean":"2024-02-17T11:57:28.680450+0000","last_became_active":"2024-02-17T11:55:58.679001+0000","last_became_peered":"2024-02-17T11:55:58.679001+0000","last_unstale":"2024-02-17T11:57:28.680450+0000","last_undegraded":"2024-02-17T11:57:28.680450+0000","last_fullsized":"2024-02-17T11:57:28.680450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:36:48.493338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":138,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:50.316679+0000","last_change":"2024-02-17T11:55:57.670027+0000","last_active":"2024-02-17T11:57:50.316679+0000","last_peered":"2024-02-17T11:57:50.316679+0000","last_clean":"2024-02-17T11:57:50.316679+0000","last_became_active":"2024-02-17T11:55:57.669692+0000","last_became_peered":"2024-02-17T11:55:57.669692+0000","last_unstale":"2024-02-17T11:57:50.316679+0000","last_undegraded":"2024-02-17T11:57:50.316679+0000","last_fullsized":"2024-02-17T11:57:50.316679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:44:01.001640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.674132+0000","last_change":"2024-02-17T11:55:58.672811+0000","last_active":"2024-02-17T11:57:28.674132+0000","last_peered":"2024-02-17T11:57:28.674132+0000","last_clean":"2024-02-17T11:57:28.674132+0000","last_became_active":"2024-02-17T11:55:58.672469+0000","last_became_peered":"2024-02-17T11:55:58.672469+0000","last_unstale":"2024-02-17T11:57:28.674132+0000","last_undegraded":"2024-02-17T11:57:28.674132+0000","last_fullsized":"2024-02-17T11:57:28.674132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:07.451422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729069+0000","last_change":"2024-02-17T11:55:57.668031+0000","last_active":"2024-02-17T11:56:59.729069+0000","last_peered":"2024-02-17T11:56:59.729069+0000","last_clean":"2024-02-17T11:56:59.729069+0000","last_became_active":"2024-02-17T11:55:57.667344+0000","last_became_peered":"2024-02-17T11:55:57.667344+0000","last_unstale":"2024-02-17T11:56:59.729069+0000","last_undegraded":"2024-02-17T11:56:59.729069+0000","last_fullsized":"2024-02-17T11:56:59.729069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:14:20.773902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.676780+0000","last_change":"2024-02-17T11:55:58.682013+0000","last_active":"2024-02-17T11:57:28.676780+0000","last_peered":"2024-02-17T11:57:28.676780+0000","last_clean":"2024-02-17T11:57:28.676780+0000","last_became_active":"2024-02-17T11:55:58.681503+0000","last_became_peered":"2024-02-17T11:55:58.681503+0000","last_unstale":"2024-02-17T11:57:28.676780+0000","last_undegraded":"2024-02-17T11:57:28.676780+0000","last_fullsized":"2024-02-17T11:57:28.676780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:24:50.364090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729587+0000","last_change":"2024-02-17T11:55:57.666725+0000","last_active":"2024-02-17T11:56:59.729587+0000","last_peered":"2024-02-17T11:56:59.729587+0000","last_clean":"2024-02-17T11:56:59.729587+0000","last_became_active":"2024-02-17T11:55:57.666339+0000","last_became_peered":"2024-02-17T11:55:57.666339+0000","last_unstale":"2024-02-17T11:56:59.729587+0000","last_undegraded":"2024-02-17T11:56:59.729587+0000","last_fullsized":"2024-02-17T11:56:59.729587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:22:47.324016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.678483+0000","last_change":"2024-02-17T11:55:58.675995+0000","last_active":"2024-02-17T11:57:28.678483+0000","last_peered":"2024-02-17T11:57:28.678483+0000","last_clean":"2024-02-17T11:57:28.678483+0000","last_became_active":"2024-02-17T11:55:58.675655+0000","last_became_peered":"2024-02-17T11:55:58.675655+0000","last_unstale":"2024-02-17T11:57:28.678483+0000","last_undegraded":"2024-02-17T11:57:28.678483+0000","last_fullsized":"2024-02-17T11:57:28.678483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:12.465300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730891+0000","last_change":"2024-02-17T11:55:57.669500+0000","last_active":"2024-02-17T11:56:59.730891+0000","last_peered":"2024-02-17T11:56:59.730891+0000","last_clean":"2024-02-17T11:56:59.730891+0000","last_became_active":"2024-02-17T11:55:57.669148+0000","last_became_peered":"2024-02-17T11:55:57.669148+0000","last_unstale":"2024-02-17T11:56:59.730891+0000","last_undegraded":"2024-02-17T11:56:59.730891+0000","last_fullsized":"2024-02-17T11:56:59.730891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:33.821266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.668037+0000","last_change":"2024-02-17T11:55:58.678336+0000","last_active":"2024-02-17T11:57:28.668037+0000","last_peered":"2024-02-17T11:57:28.668037+0000","last_clean":"2024-02-17T11:57:28.668037+0000","last_became_active":"2024-02-17T11:55:58.677967+0000","last_became_peered":"2024-02-17T11:55:58.677967+0000","last_unstale":"2024-02-17T11:57:28.668037+0000","last_undegraded":"2024-02-17T11:57:28.668037+0000","last_fullsized":"2024-02-17T11:57:28.668037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:31:39.406673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732692+0000","last_change":"2024-02-17T11:55:57.674810+0000","last_active":"2024-02-17T11:56:59.732692+0000","last_peered":"2024-02-17T11:56:59.732692+0000","last_clean":"2024-02-17T11:56:59.732692+0000","last_became_active":"2024-02-17T11:55:57.674356+0000","last_became_peered":"2024-02-17T11:55:57.674356+0000","last_unstale":"2024-02-17T11:56:59.732692+0000","last_undegraded":"2024-02-17T11:56:59.732692+0000","last_fullsized":"2024-02-17T11:56:59.732692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:25:57.879239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.677955+0000","last_change":"2024-02-17T11:55:58.680928+0000","last_active":"2024-02-17T11:57:28.677955+0000","last_peered":"2024-02-17T11:57:28.677955+0000","last_clean":"2024-02-17T11:57:28.677955+0000","last_became_active":"2024-02-17T11:55:58.680521+0000","last_became_peered":"2024-02-17T11:55:58.680521+0000","last_unstale":"2024-02-17T11:57:28.677955+0000","last_undegraded":"2024-02-17T11:57:28.677955+0000","last_fullsized":"2024-02-17T11:57:28.677955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:15:15.739127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.675398+0000","last_change":"2024-02-17T11:55:58.674771+0000","last_active":"2024-02-17T11:57:28.675398+0000","last_peered":"2024-02-17T11:57:28.675398+0000","last_clean":"2024-02-17T11:57:28.675398+0000","last_became_active":"2024-02-17T11:55:58.674450+0000","last_became_peered":"2024-02-17T11:55:58.674450+0000","last_unstale":"2024-02-17T11:57:28.675398+0000","last_undegraded":"2024-02-17T11:57:28.675398+0000","last_fullsized":"2024-02-17T11:57:28.675398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.031576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":133,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:49.314502+0000","last_change":"2024-02-17T11:55:57.672868+0000","last_active":"2024-02-17T11:57:49.314502+0000","last_peered":"2024-02-17T11:57:49.314502+0000","last_clean":"2024-02-17T11:57:49.314502+0000","last_became_active":"2024-02-17T11:55:57.672584+0000","last_became_peered":"2024-02-17T11:55:57.672584+0000","last_unstale":"2024-02-17T11:57:49.314502+0000","last_undegraded":"2024-02-17T11:57:49.314502+0000","last_fullsized":"2024-02-17T11:57:49.314502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:03:20.307036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.684813+0000","last_change":"2024-02-17T11:55:58.692594+0000","last_active":"2024-02-17T11:57:28.684813+0000","last_peered":"2024-02-17T11:57:28.684813+0000","last_clean":"2024-02-17T11:57:28.684813+0000","last_became_active":"2024-02-17T11:55:58.692258+0000","last_became_peered":"2024-02-17T11:55:58.692258+0000","last_unstale":"2024-02-17T11:57:28.684813+0000","last_undegraded":"2024-02-17T11:57:28.684813+0000","last_fullsized":"2024-02-17T11:57:28.684813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:28.497939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.880997+0000","last_change":"2024-02-17T11:55:57.669456+0000","last_active":"2024-02-17T11:56:59.880997+0000","last_peered":"2024-02-17T11:56:59.880997+0000","last_clean":"2024-02-17T11:56:59.880997+0000","last_became_active":"2024-02-17T11:55:57.669067+0000","last_became_peered":"2024-02-17T11:55:57.669067+0000","last_unstale":"2024-02-17T11:56:59.880997+0000","last_undegraded":"2024-02-17T11:56:59.880997+0000","last_fullsized":"2024-02-17T11:56:59.880997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:12.352296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732120+0000","last_change":"2024-02-17T11:54:46.125844+0000","last_active":"2024-02-17T11:56:59.732120+0000","last_peered":"2024-02-17T11:56:59.732120+0000","last_clean":"2024-02-17T11:56:59.732120+0000","last_became_active":"2024-02-17T11:54:46.125387+0000","last_became_peered":"2024-02-17T11:54:46.125387+0000","last_unstale":"2024-02-17T11:56:59.732120+0000","last_undegraded":"2024-02-17T11:56:59.732120+0000","last_fullsized":"2024-02-17T11:56:59.732120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_clean_scrub_stamp":"2024-02-17T11:51:21.581371+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:53:27.219974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671351+0000","last_change":"2024-02-17T11:55:58.681623+0000","last_active":"2024-02-17T11:57:28.671351+0000","last_peered":"2024-02-17T11:57:28.671351+0000","last_clean":"2024-02-17T11:57:28.671351+0000","last_became_active":"2024-02-17T11:55:58.681275+0000","last_became_peered":"2024-02-17T11:55:58.681275+0000","last_unstale":"2024-02-17T11:57:28.671351+0000","last_undegraded":"2024-02-17T11:57:28.671351+0000","last_fullsized":"2024-02-17T11:57:28.671351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:20.685634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732237+0000","last_change":"2024-02-17T11:55:57.679555+0000","last_active":"2024-02-17T11:56:59.732237+0000","last_peered":"2024-02-17T11:56:59.732237+0000","last_clean":"2024-02-17T11:56:59.732237+0000","last_became_active":"2024-02-17T11:55:57.679241+0000","last_became_peered":"2024-02-17T11:55:57.679241+0000","last_unstale":"2024-02-17T11:56:59.732237+0000","last_undegraded":"2024-02-17T11:56:59.732237+0000","last_fullsized":"2024-02-17T11:56:59.732237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:37.482582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729501+0000","last_change":"2024-02-17T11:55:57.679711+0000","last_active":"2024-02-17T11:56:59.729501+0000","last_peered":"2024-02-17T11:56:59.729501+0000","last_clean":"2024-02-17T11:56:59.729501+0000","last_became_active":"2024-02-17T11:55:57.679415+0000","last_became_peered":"2024-02-17T11:55:57.679415+0000","last_unstale":"2024-02-17T11:56:59.729501+0000","last_undegraded":"2024-02-17T11:56:59.729501+0000","last_fullsized":"2024-02-17T11:56:59.729501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:13.095530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.660308+0000","last_change":"2024-02-17T11:55:58.670275+0000","last_active":"2024-02-17T11:57:28.660308+0000","last_peered":"2024-02-17T11:57:28.660308+0000","last_clean":"2024-02-17T11:57:28.660308+0000","last_became_active":"2024-02-17T11:55:58.669926+0000","last_became_peered":"2024-02-17T11:55:58.669926+0000","last_unstale":"2024-02-17T11:57:28.660308+0000","last_undegraded":"2024-02-17T11:57:28.660308+0000","last_fullsized":"2024-02-17T11:57:28.660308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:48:35.078389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.675754+0000","last_change":"2024-02-17T11:55:57.670364+0000","last_active":"2024-02-17T11:57:28.675754+0000","last_peered":"2024-02-17T11:57:28.675754+0000","last_clean":"2024-02-17T11:57:28.675754+0000","last_became_active":"2024-02-17T11:55:57.669977+0000","last_became_peered":"2024-02-17T11:55:57.669977+0000","last_unstale":"2024-02-17T11:57:28.675754+0000","last_undegraded":"2024-02-17T11:57:28.675754+0000","last_fullsized":"2024-02-17T11:57:28.675754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:42:13.820390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.673177+0000","last_change":"2024-02-17T11:55:58.681587+0000","last_active":"2024-02-17T11:57:28.673177+0000","last_peered":"2024-02-17T11:57:28.673177+0000","last_clean":"2024-02-17T11:57:28.673177+0000","last_became_active":"2024-02-17T11:55:58.680024+0000","last_became_peered":"2024-02-17T11:55:58.680024+0000","last_unstale":"2024-02-17T11:57:28.673177+0000","last_undegraded":"2024-02-17T11:57:28.673177+0000","last_fullsized":"2024-02-17T11:57:28.673177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:31:40.399439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.881143+0000","last_change":"2024-02-17T11:55:57.674618+0000","last_active":"2024-02-17T11:56:59.881143+0000","last_peered":"2024-02-17T11:56:59.881143+0000","last_clean":"2024-02-17T11:56:59.881143+0000","last_became_active":"2024-02-17T11:55:57.674246+0000","last_became_peered":"2024-02-17T11:55:57.674246+0000","last_unstale":"2024-02-17T11:56:59.881143+0000","last_undegraded":"2024-02-17T11:56:59.881143+0000","last_fullsized":"2024-02-17T11:56:59.881143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:42:26.681061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671477+0000","last_change":"2024-02-17T11:56:59.734355+0000","last_active":"2024-02-17T11:57:28.671477+0000","last_peered":"2024-02-17T11:57:28.671477+0000","last_clean":"2024-02-17T11:57:28.671477+0000","last_became_active":"2024-02-17T11:56:59.733975+0000","last_became_peered":"2024-02-17T11:56:59.733975+0000","last_unstale":"2024-02-17T11:57:28.671477+0000","last_undegraded":"2024-02-17T11:57:28.671477+0000","last_fullsized":"2024-02-17T11:57:28.671477+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:37:51.426702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732149+0000","last_change":"2024-02-17T11:55:57.667998+0000","last_active":"2024-02-17T11:56:59.732149+0000","last_peered":"2024-02-17T11:56:59.732149+0000","last_clean":"2024-02-17T11:56:59.732149+0000","last_became_active":"2024-02-17T11:55:57.667688+0000","last_became_peered":"2024-02-17T11:55:57.667688+0000","last_unstale":"2024-02-17T11:56:59.732149+0000","last_undegraded":"2024-02-17T11:56:59.732149+0000","last_fullsized":"2024-02-17T11:56:59.732149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:07:19.782879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.659157+0000","last_change":"2024-02-17T11:55:58.676644+0000","last_active":"2024-02-17T11:57:28.659157+0000","last_peered":"2024-02-17T11:57:28.659157+0000","last_clean":"2024-02-17T11:57:28.659157+0000","last_became_active":"2024-02-17T11:55:58.674670+0000","last_became_peered":"2024-02-17T11:55:58.674670+0000","last_unstale":"2024-02-17T11:57:28.659157+0000","last_undegraded":"2024-02-17T11:57:28.659157+0000","last_fullsized":"2024-02-17T11:57:28.659157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:43:22.746537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732754+0000","last_change":"2024-02-17T11:55:57.671330+0000","last_active":"2024-02-17T11:56:59.732754+0000","last_peered":"2024-02-17T11:56:59.732754+0000","last_clean":"2024-02-17T11:56:59.732754+0000","last_became_active":"2024-02-17T11:55:57.670902+0000","last_became_peered":"2024-02-17T11:55:57.670902+0000","last_unstale":"2024-02-17T11:56:59.732754+0000","last_undegraded":"2024-02-17T11:56:59.732754+0000","last_fullsized":"2024-02-17T11:56:59.732754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:22:14.770339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672152+0000","last_change":"2024-02-17T11:55:58.676417+0000","last_active":"2024-02-17T11:57:28.672152+0000","last_peered":"2024-02-17T11:57:28.672152+0000","last_clean":"2024-02-17T11:57:28.672152+0000","last_became_active":"2024-02-17T11:55:58.675960+0000","last_became_peered":"2024-02-17T11:55:58.675960+0000","last_unstale":"2024-02-17T11:57:28.672152+0000","last_undegraded":"2024-02-17T11:57:28.672152+0000","last_fullsized":"2024-02-17T11:57:28.672152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:22:18.566872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.738907+0000","last_change":"2024-02-17T11:56:59.737823+0000","last_active":"2024-02-17T11:56:59.738907+0000","last_peered":"2024-02-17T11:56:59.738907+0000","last_clean":"2024-02-17T11:56:59.738907+0000","last_became_active":"2024-02-17T11:56:59.734043+0000","last_became_peered":"2024-02-17T11:56:59.734043+0000","last_unstale":"2024-02-17T11:56:59.738907+0000","last_undegraded":"2024-02-17T11:56:59.738907+0000","last_fullsized":"2024-02-17T11:56:59.738907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:11:09.171845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670581+0000","last_change":"2024-02-17T11:55:58.690030+0000","last_active":"2024-02-17T11:57:28.670581+0000","last_peered":"2024-02-17T11:57:28.670581+0000","last_clean":"2024-02-17T11:57:28.670581+0000","last_became_active":"2024-02-17T11:55:58.689687+0000","last_became_peered":"2024-02-17T11:55:58.689687+0000","last_unstale":"2024-02-17T11:57:28.670581+0000","last_undegraded":"2024-02-17T11:57:28.670581+0000","last_fullsized":"2024-02-17T11:57:28.670581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:33:43.175234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729187+0000","last_change":"2024-02-17T11:55:57.679734+0000","last_active":"2024-02-17T11:56:59.729187+0000","last_peered":"2024-02-17T11:56:59.729187+0000","last_clean":"2024-02-17T11:56:59.729187+0000","last_became_active":"2024-02-17T11:55:57.679328+0000","last_became_peered":"2024-02-17T11:55:57.679328+0000","last_unstale":"2024-02-17T11:56:59.729187+0000","last_undegraded":"2024-02-17T11:56:59.729187+0000","last_fullsized":"2024-02-17T11:56:59.729187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:50:09.813482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.683947+0000","last_change":"2024-02-17T11:55:58.688360+0000","last_active":"2024-02-17T11:57:28.683947+0000","last_peered":"2024-02-17T11:57:28.683947+0000","last_clean":"2024-02-17T11:57:28.683947+0000","last_became_active":"2024-02-17T11:55:58.687994+0000","last_became_peered":"2024-02-17T11:55:58.687994+0000","last_unstale":"2024-02-17T11:57:28.683947+0000","last_undegraded":"2024-02-17T11:57:28.683947+0000","last_fullsized":"2024-02-17T11:57:28.683947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:01:02.163238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733202+0000","last_change":"2024-02-17T11:55:57.675977+0000","last_active":"2024-02-17T11:56:59.733202+0000","last_peered":"2024-02-17T11:56:59.733202+0000","last_clean":"2024-02-17T11:56:59.733202+0000","last_became_active":"2024-02-17T11:55:57.675250+0000","last_became_peered":"2024-02-17T11:55:57.675250+0000","last_unstale":"2024-02-17T11:56:59.733202+0000","last_undegraded":"2024-02-17T11:56:59.733202+0000","last_fullsized":"2024-02-17T11:56:59.733202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:11:27.611692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685664+0000","last_change":"2024-02-17T11:55:58.688593+0000","last_active":"2024-02-17T11:57:28.685664+0000","last_peered":"2024-02-17T11:57:28.685664+0000","last_clean":"2024-02-17T11:57:28.685664+0000","last_became_active":"2024-02-17T11:55:58.688163+0000","last_became_peered":"2024-02-17T11:55:58.688163+0000","last_unstale":"2024-02-17T11:57:28.685664+0000","last_undegraded":"2024-02-17T11:57:28.685664+0000","last_fullsized":"2024-02-17T11:57:28.685664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:54.597116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733115+0000","last_change":"2024-02-17T11:55:57.676860+0000","last_active":"2024-02-17T11:56:59.733115+0000","last_peered":"2024-02-17T11:56:59.733115+0000","last_clean":"2024-02-17T11:56:59.733115+0000","last_became_active":"2024-02-17T11:55:57.676534+0000","last_became_peered":"2024-02-17T11:55:57.676534+0000","last_unstale":"2024-02-17T11:56:59.733115+0000","last_undegraded":"2024-02-17T11:56:59.733115+0000","last_fullsized":"2024-02-17T11:56:59.733115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:23:47.726264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.675151+0000","last_change":"2024-02-17T11:56:59.739615+0000","last_active":"2024-02-17T11:57:28.675151+0000","last_peered":"2024-02-17T11:57:28.675151+0000","last_clean":"2024-02-17T11:57:28.675151+0000","last_became_active":"2024-02-17T11:56:59.736220+0000","last_became_peered":"2024-02-17T11:56:59.736220+0000","last_unstale":"2024-02-17T11:57:28.675151+0000","last_undegraded":"2024-02-17T11:57:28.675151+0000","last_fullsized":"2024-02-17T11:57:28.675151+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:22:07.463713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.739390+0000","last_change":"2024-02-17T11:56:59.739390+0000","last_active":"2024-02-17T11:56:59.739390+0000","last_peered":"2024-02-17T11:56:59.739390+0000","last_clean":"2024-02-17T11:56:59.739390+0000","last_became_active":"2024-02-17T11:56:59.734852+0000","last_became_peered":"2024-02-17T11:56:59.734852+0000","last_unstale":"2024-02-17T11:56:59.739390+0000","last_undegraded":"2024-02-17T11:56:59.739390+0000","last_fullsized":"2024-02-17T11:56:59.739390+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:40:09.174557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.675878+0000","last_change":"2024-02-17T11:55:58.672985+0000","last_active":"2024-02-17T11:57:28.675878+0000","last_peered":"2024-02-17T11:57:28.675878+0000","last_clean":"2024-02-17T11:57:28.675878+0000","last_became_active":"2024-02-17T11:55:58.672622+0000","last_became_peered":"2024-02-17T11:55:58.672622+0000","last_unstale":"2024-02-17T11:57:28.675878+0000","last_undegraded":"2024-02-17T11:57:28.675878+0000","last_fullsized":"2024-02-17T11:57:28.675878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:45.555827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672536+0000","last_change":"2024-02-17T11:55:58.676127+0000","last_active":"2024-02-17T11:57:28.672536+0000","last_peered":"2024-02-17T11:57:28.672536+0000","last_clean":"2024-02-17T11:57:28.672536+0000","last_became_active":"2024-02-17T11:55:58.675784+0000","last_became_peered":"2024-02-17T11:55:58.675784+0000","last_unstale":"2024-02-17T11:57:28.672536+0000","last_undegraded":"2024-02-17T11:57:28.672536+0000","last_fullsized":"2024-02-17T11:57:28.672536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:56:32.648803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.663712+0000","last_change":"2024-02-17T11:55:58.682438+0000","last_active":"2024-02-17T11:57:28.663712+0000","last_peered":"2024-02-17T11:57:28.663712+0000","last_clean":"2024-02-17T11:57:28.663712+0000","last_became_active":"2024-02-17T11:55:58.682067+0000","last_became_peered":"2024-02-17T11:55:58.682067+0000","last_unstale":"2024-02-17T11:57:28.663712+0000","last_undegraded":"2024-02-17T11:57:28.663712+0000","last_fullsized":"2024-02-17T11:57:28.663712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:10:22.906450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671367+0000","last_change":"2024-02-17T11:55:58.686671+0000","last_active":"2024-02-17T11:57:28.671367+0000","last_peered":"2024-02-17T11:57:28.671367+0000","last_clean":"2024-02-17T11:57:28.671367+0000","last_became_active":"2024-02-17T11:55:58.686315+0000","last_became_peered":"2024-02-17T11:55:58.686315+0000","last_unstale":"2024-02-17T11:57:28.671367+0000","last_undegraded":"2024-02-17T11:57:28.671367+0000","last_fullsized":"2024-02-17T11:57:28.671367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:55:40.864847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671037+0000","last_change":"2024-02-17T11:55:58.684227+0000","last_active":"2024-02-17T11:57:28.671037+0000","last_peered":"2024-02-17T11:57:28.671037+0000","last_clean":"2024-02-17T11:57:28.671037+0000","last_became_active":"2024-02-17T11:55:58.683875+0000","last_became_peered":"2024-02-17T11:55:58.683875+0000","last_unstale":"2024-02-17T11:57:28.671037+0000","last_undegraded":"2024-02-17T11:57:28.671037+0000","last_fullsized":"2024-02-17T11:57:28.671037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:11:41.283161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.677463+0000","last_change":"2024-02-17T11:55:58.685964+0000","last_active":"2024-02-17T11:57:28.677463+0000","last_peered":"2024-02-17T11:57:28.677463+0000","last_clean":"2024-02-17T11:57:28.677463+0000","last_became_active":"2024-02-17T11:55:58.685350+0000","last_became_peered":"2024-02-17T11:55:58.685350+0000","last_unstale":"2024-02-17T11:57:28.677463+0000","last_undegraded":"2024-02-17T11:57:28.677463+0000","last_fullsized":"2024-02-17T11:57:28.677463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:22:34.600604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671590+0000","last_change":"2024-02-17T11:55:58.671291+0000","last_active":"2024-02-17T11:57:28.671590+0000","last_peered":"2024-02-17T11:57:28.671590+0000","last_clean":"2024-02-17T11:57:28.671590+0000","last_became_active":"2024-02-17T11:55:58.670925+0000","last_became_peered":"2024-02-17T11:55:58.670925+0000","last_unstale":"2024-02-17T11:57:28.671590+0000","last_undegraded":"2024-02-17T11:57:28.671590+0000","last_fullsized":"2024-02-17T11:57:28.671590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:26:03.120976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.667565+0000","last_change":"2024-02-17T11:55:58.673122+0000","last_active":"2024-02-17T11:57:28.667565+0000","last_peered":"2024-02-17T11:57:28.667565+0000","last_clean":"2024-02-17T11:57:28.667565+0000","last_became_active":"2024-02-17T11:55:58.672747+0000","last_became_peered":"2024-02-17T11:55:58.672747+0000","last_unstale":"2024-02-17T11:57:28.667565+0000","last_undegraded":"2024-02-17T11:57:28.667565+0000","last_fullsized":"2024-02-17T11:57:28.667565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:55:15.064073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.677328+0000","last_change":"2024-02-17T11:55:58.681276+0000","last_active":"2024-02-17T11:57:28.677328+0000","last_peered":"2024-02-17T11:57:28.677328+0000","last_clean":"2024-02-17T11:57:28.677328+0000","last_became_active":"2024-02-17T11:55:58.680889+0000","last_became_peered":"2024-02-17T11:55:58.680889+0000","last_unstale":"2024-02-17T11:57:28.677328+0000","last_undegraded":"2024-02-17T11:57:28.677328+0000","last_fullsized":"2024-02-17T11:57:28.677328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:19:34.330288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670939+0000","last_change":"2024-02-17T11:55:58.667475+0000","last_active":"2024-02-17T11:57:28.670939+0000","last_peered":"2024-02-17T11:57:28.670939+0000","last_clean":"2024-02-17T11:57:28.670939+0000","last_became_active":"2024-02-17T11:55:58.666577+0000","last_became_peered":"2024-02-17T11:55:58.666577+0000","last_unstale":"2024-02-17T11:57:28.670939+0000","last_undegraded":"2024-02-17T11:57:28.670939+0000","last_fullsized":"2024-02-17T11:57:28.670939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:51:35.732035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672928+0000","last_change":"2024-02-17T11:55:58.674557+0000","last_active":"2024-02-17T11:57:28.672928+0000","last_peered":"2024-02-17T11:57:28.672928+0000","last_clean":"2024-02-17T11:57:28.672928+0000","last_became_active":"2024-02-17T11:55:58.673874+0000","last_became_peered":"2024-02-17T11:55:58.673874+0000","last_unstale":"2024-02-17T11:57:28.672928+0000","last_undegraded":"2024-02-17T11:57:28.672928+0000","last_fullsized":"2024-02-17T11:57:28.672928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:18:00.812489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685895+0000","last_change":"2024-02-17T11:55:58.692725+0000","last_active":"2024-02-17T11:57:28.685895+0000","last_peered":"2024-02-17T11:57:28.685895+0000","last_clean":"2024-02-17T11:57:28.685895+0000","last_became_active":"2024-02-17T11:55:58.690880+0000","last_became_peered":"2024-02-17T11:55:58.690880+0000","last_unstale":"2024-02-17T11:57:28.685895+0000","last_undegraded":"2024-02-17T11:57:28.685895+0000","last_fullsized":"2024-02-17T11:57:28.685895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:03:44.724385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.666977+0000","last_change":"2024-02-17T11:55:58.679377+0000","last_active":"2024-02-17T11:57:28.666977+0000","last_peered":"2024-02-17T11:57:28.666977+0000","last_clean":"2024-02-17T11:57:28.666977+0000","last_became_active":"2024-02-17T11:55:58.679041+0000","last_became_peered":"2024-02-17T11:55:58.679041+0000","last_unstale":"2024-02-17T11:57:28.666977+0000","last_undegraded":"2024-02-17T11:57:28.666977+0000","last_fullsized":"2024-02-17T11:57:28.666977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:16:37.396424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.678575+0000","last_change":"2024-02-17T11:55:58.680665+0000","last_active":"2024-02-17T11:57:28.678575+0000","last_peered":"2024-02-17T11:57:28.678575+0000","last_clean":"2024-02-17T11:57:28.678575+0000","last_became_active":"2024-02-17T11:55:58.680311+0000","last_became_peered":"2024-02-17T11:55:58.680311+0000","last_unstale":"2024-02-17T11:57:28.678575+0000","last_undegraded":"2024-02-17T11:57:28.678575+0000","last_fullsized":"2024-02-17T11:57:28.678575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:19:11.455471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284587,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72041,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":697,"num_read_kb":426,"num_write":175,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":210750,"data_compressed":5598,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":456292,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.372},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.55100000000000005}]},{"osd":1,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63800000000000001}]},{"osd":2,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.49199999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.52000000000000002,"15min":0.52800000000000002},"min":{"1min":0.317,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68899999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.504,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52800000000000002}]},{"osd":5,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.372,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78000000000000003,"5min":1.484,"15min":1.484},"last":0.755},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84299999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.40100000000000002}]},{"osd":6,"last update":"Sat Feb 17 11:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82199999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.62},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.92300000000000004,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.58799999999999997}]},{"osd":8,"last update":"Sat Feb 17 11:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.70399999999999996}]},{"osd":9,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.78500000000000003,"15min":0.79400000000000004},"min":{"1min":0.54000000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.89500000000000002,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.73499999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90700000000000003,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.66100000000000003}]},{"osd":10,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.61199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95099999999999996,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0529999999999999,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.83899999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1115281,"data_compressed":43889,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.89300000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46300000000000002,"15min":0.47499999999999998},"min":{"1min":0.32400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.36399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73899999999999999,"5min":0.746,"15min":0.746},"last":0.435}]},{"osd":2,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81899999999999995,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.53500000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.79500000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.41699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71899999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.60999999999999999},"min":{"1min":0.48299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77500000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69299999999999995}]},{"osd":4,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.57499999999999996},"min":{"1min":0.373,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82899999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.31,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.47399999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.47799999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.68000000000000005}]},{"osd":6,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66500000000000004,"15min":0.68100000000000005},"min":{"1min":0.51800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93400000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.748},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80200000000000005,"5min":1.101,"15min":1.101},"last":0.86199999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.41499999999999998}]},{"osd":8,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71999999999999997,"15min":0.72799999999999998},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88700000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73999999999999999,"15min":0.75800000000000001},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94699999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.71199999999999997}]},{"osd":9,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73099999999999998,"15min":0.752},"min":{"1min":0.59799999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91800000000000004,"5min":1.014,"15min":1.014},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.73899999999999999,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.434,"15min":0.434},"max":{"1min":0.997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.92100000000000004}]},{"osd":11,"last update":"Sat Feb 17 11:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.56299999999999994,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.84699999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.877},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.60999999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77900000000000003}]}]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456645,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.42299999999999999},"min":{"1min":0.29999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.40500000000000003,"15min":0.41699999999999998},"min":{"1min":0.27300000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44500000000000001}]},{"osd":1,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60099999999999998,"15min":0.61899999999999999},"min":{"1min":0.45600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.499},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78600000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.43099999999999999}]},{"osd":2,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90700000000000003,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.40300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54000000000000004}]},{"osd":3,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.496,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47399999999999998,"15min":0.5},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.71899999999999997,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.317}]},{"osd":4,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.41199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85299999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.65000000000000002,"15min":0.67700000000000005},"min":{"1min":0.42499999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.87,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47499999999999998}]},{"osd":5,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60799999999999998,"15min":0.63400000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84799999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68500000000000005,"15min":0.71499999999999997},"min":{"1min":0.503,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98099999999999998,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.52900000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57099999999999995,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81299999999999994,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60199999999999998,"15min":0.626},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91800000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55300000000000005}]},{"osd":7,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72599999999999998,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.95299999999999996,"5min":1.05,"15min":1.05},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69199999999999995,"15min":0.71399999999999997},"min":{"1min":0.47699999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93700000000000006,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.59099999999999997}]},{"osd":8,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69799999999999995,"15min":0.72599999999999998},"min":{"1min":0.55200000000000005,"5min":0.442,"15min":0.442},"max":{"1min":0.99299999999999999,"5min":1.016,"15min":1.016},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70399999999999996,"15min":0.72299999999999998},"min":{"1min":0.51200000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.96599999999999997,"5min":1.034,"15min":1.034},"last":0.61799999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.70899999999999996,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.78800000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.57399999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.80500000000000005,"5min":1.069,"15min":1.069},"last":0.66800000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070515,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":455866,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.44,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70399999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.38100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.442}]},{"osd":1,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88900000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66400000000000003}]},{"osd":2,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46000000000000002,"15min":0.44},"min":{"1min":0.35999999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.40799999999999997}]},{"osd":3,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.59199999999999997},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.76200000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.61699999999999999,"15min":0.60799999999999998},"min":{"1min":0.503,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85299999999999998,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.44600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56999999999999995,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73399999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.82199999999999995}]},{"osd":6,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85099999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.753},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.52500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.628}]},{"osd":7,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.52300000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.53400000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70599999999999996}]},{"osd":9,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.51100000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.68700000000000006,"15min":0.66600000000000004},"min":{"1min":0.54300000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.84099999999999997}]},{"osd":10,"last update":"Sat Feb 17 11:57:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76400000000000001,"15min":0.74299999999999999},"min":{"1min":0.54900000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.75900000000000001,"15min":0.74299999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.90100000000000002}]},{"osd":11,"last update":"Sat Feb 17 11:57:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.88200000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":1049051,"data_compressed":42023,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.36699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39600000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.41899999999999998},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65300000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.45100000000000001,"15min":0.433},"min":{"1min":0.33700000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68000000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.52900000000000003}]},{"osd":2,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.38600000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.89100000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.65500000000000003}]},{"osd":3,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.45600000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71899999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.41999999999999998}]},{"osd":4,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48599999999999999,"15min":0.47399999999999998},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.373},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85499999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.39200000000000002}]},{"osd":5,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67500000000000004,"15min":0.66100000000000003},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93200000000000005,"5min":1.333,"15min":1.333},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.47599999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83199999999999996,"5min":1.486,"15min":1.486},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.47699999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91000000000000003,"5min":1.087,"15min":1.087},"last":0.57199999999999995}]},{"osd":8,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.55900000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96999999999999997,"5min":1.446,"15min":1.446},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92400000000000004,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.70099999999999996}]},{"osd":9,"last update":"Sat Feb 17 11:57:26 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47299999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.746,"15min":0.755},"min":{"1min":0.49299999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.034,"5min":1.052,"15min":1.052},"last":0.61499999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:56:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.498,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95599999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75700000000000001,"15min":0.76100000000000001},"min":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0009999999999999,"5min":1.02,"15min":1.02},"last":0.71499999999999997}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.754}]}]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":455917,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.35099999999999998},"min":{"1min":0.309,"5min":0.223,"15min":0.223},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.436},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38500000000000001,"15min":0.38300000000000001},"min":{"1min":0.28799999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53300000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.42599999999999999}]},{"osd":1,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70099999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.32500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71199999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":0.871,"15min":0.871},"last":0.496}]},{"osd":3,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64400000000000002,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.439},"min":{"1min":0.27600000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.628,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.30499999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.39300000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.39500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.79300000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.65100000000000002}]},{"osd":5,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59299999999999997,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.503,"5min":0.432,"15min":0.432},"max":{"1min":0.86899999999999999,"5min":0.997,"15min":0.997},"last":0.47799999999999998}]},{"osd":7,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.622},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.45400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.71099999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59599999999999997}]},{"osd":8,"last update":"Sat Feb 17 11:57:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.71799999999999997,"15min":0.70099999999999996},"min":{"1min":0.46400000000000002,"5min":0.434,"15min":0.434},"max":{"1min":5.4269999999999996,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.50600000000000001}]},{"osd":9,"last update":"Sat Feb 17 11:57:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.625,"15min":0.65700000000000003},"min":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79300000000000004,"5min":0.879,"15min":0.879},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.65600000000000003,"15min":0.68300000000000005},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82699999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56899999999999995}]},{"osd":10,"last update":"Sat Feb 17 11:57:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.73599999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.70199999999999996,"15min":0.72099999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.753,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":522686,"data_compressed":7701,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54500000000000004,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77800000000000002,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.496},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71199999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57899999999999996}]},{"osd":2,"last update":"Sat Feb 17 11:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57799999999999996,"15min":0.53200000000000003},"min":{"1min":0.41099999999999998,"5min":0.313,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57899999999999996,"15min":0.53300000000000003},"min":{"1min":0.52800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81899999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.74399999999999999}]},{"osd":3,"last update":"Sat Feb 17 11:56:51 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58399999999999996,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.46500000000000002,"15min":0.433},"min":{"1min":0.376,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.46899999999999997,"15min":0.42099999999999999},"min":{"1min":0.35699999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.56100000000000005}]},{"osd":5,"last update":"Sat Feb 17 11:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.66100000000000003,"15min":0.59099999999999997},"min":{"1min":0.58099999999999996,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67900000000000005,"15min":0.60899999999999999},"min":{"1min":0.64300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.871,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79200000000000004}]},{"osd":6,"last update":"Sat Feb 17 11:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63300000000000001,"15min":0.60099999999999998},"min":{"1min":0.52000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.88300000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.62,"15min":0.59699999999999998},"min":{"1min":0.51000000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78600000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45100000000000001}]},{"osd":7,"last update":"Sat Feb 17 11:57:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91100000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65100000000000002,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93799999999999994,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.64400000000000002}]},{"osd":8,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.71799999999999997,"15min":0.68799999999999994},"min":{"1min":0.622,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98599999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66500000000000004,"15min":0.65400000000000003},"min":{"1min":0.56899999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.83799999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63200000000000001}]},{"osd":9,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.59699999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89900000000000002,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78900000000000003,"15min":0.77700000000000002},"min":{"1min":0.66300000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92200000000000004,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72899999999999998}]},{"osd":10,"last update":"Sat Feb 17 11:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.48899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92500000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79000000000000004,"15min":0.77600000000000002},"min":{"1min":0.66800000000000004,"5min":0.501,"15min":0.501},"max":{"1min":1.002,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.82899999999999996}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.86899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":457188,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.53700000000000003,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52600000000000002,"15min":0.47999999999999998},"min":{"1min":0.40400000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73199999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57899999999999996}]},{"osd":2,"last update":"Sat Feb 17 11:57:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.52300000000000002},"min":{"1min":0.442,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57999999999999996,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.61499999999999999}]},{"osd":3,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat Feb 17 11:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54900000000000004,"15min":0.51100000000000001},"min":{"1min":0.39200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67700000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.52300000000000002},"min":{"1min":0.35099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75800000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63}]},{"osd":5,"last update":"Sat Feb 17 11:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.62,"15min":0.59499999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.46899999999999997}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.498,"15min":0.502},"min":{"1min":0.307,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64200000000000002,"5min":0.877,"15min":0.877},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68799999999999994,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.44500000000000001}]},{"osd":7,"last update":"Sat Feb 17 11:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.501,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58399999999999996}]},{"osd":8,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.69499999999999995,"15min":0.70599999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.46600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.53500000000000003}]},{"osd":9,"last update":"Sat Feb 17 11:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.61499999999999999,"15min":0.64500000000000002},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86099999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.498},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.68200000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76700000000000002,"5min":1.335,"15min":1.335},"last":0.48499999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68500000000000005,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.79400000000000004,"5min":1.107,"15min":1.107},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.745},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89900000000000002,"5min":1.236,"15min":1.236},"last":0.70399999999999996}]},{"osd":11,"last update":"Sat Feb 17 11:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70199999999999996,"15min":0.70499999999999996},"min":{"1min":0.55600000000000005,"5min":0.495,"15min":0.495},"max":{"1min":0.84399999999999997,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.67000000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":518887,"data_compressed":7701,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.498},"min":{"1min":0.32200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76700000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.54600000000000004,"15min":0.50800000000000001},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.29199999999999998},"max":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60899999999999999}]},{"osd":1,"last update":"Sat Feb 17 11:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58999999999999997,"15min":0.501},"min":{"1min":0.45100000000000001,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.89400000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70799999999999996}]},{"osd":3,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.54500000000000004},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.753,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.4750000000000001,"5min":0.71899999999999997,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":16.718,"5min":16.718,"15min":16.718},"last":0.48099999999999998}]},{"osd":4,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82599999999999996}]},{"osd":5,"last update":"Sat Feb 17 11:57:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.48599999999999999,"15min":0.438},"min":{"1min":0.378,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.47299999999999998,"15min":0.44700000000000001},"min":{"1min":0.35399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66100000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60799999999999998,"15min":0.58399999999999996},"min":{"1min":0.40799999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.72799999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57799999999999996}]},{"osd":7,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63700000000000001}]},{"osd":8,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.41199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79000000000000004}]},{"osd":9,"last update":"Sat Feb 17 11:57:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90500000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.66400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80600000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.64300000000000002}]},{"osd":10,"last update":"Sat Feb 17 11:57:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.63900000000000001,"15min":0.61399999999999999},"min":{"1min":0.438,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.749},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.85999999999999999}]},{"osd":11,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.76100000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456111,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.35799999999999998,"15min":0.34399999999999997},"min":{"1min":0.26100000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.745,"5min":0.746,"15min":0.746},"last":0.309},{"interface":"front","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.34499999999999997},"min":{"1min":0.26400000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.53000000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.35699999999999998}]},{"osd":1,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.41699999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.53600000000000003},"min":{"1min":0.36799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78300000000000003,"5min":0.87,"15min":0.87},"last":0.504}]},{"osd":2,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77100000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78700000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51300000000000001}]},{"osd":4,"last update":"Sat Feb 17 11:57:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":0.71399999999999997,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":12.353,"5min":12.353,"15min":12.353},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60499999999999998,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58199999999999996}]},{"osd":5,"last update":"Sat Feb 17 11:57:48 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":0.68999999999999995,"15min":0.60999999999999999},"min":{"1min":0.46700000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":12.337999999999999,"5min":12.337999999999999,"15min":12.337999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.749,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.496}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.44400000000000001},"min":{"1min":0.307,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82999999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.373},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95799999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.39100000000000001}]},{"osd":7,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":12.301,"5min":12.301,"15min":12.301},"last":0.69799999999999995}]},{"osd":8,"last update":"Sat Feb 17 11:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.49199999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97599999999999998,"5min":1.214,"15min":1.214},"last":0.622}]},{"osd":9,"last update":"Sat Feb 17 11:57:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.57399999999999995,"15min":0.60699999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64100000000000001,"5min":0.875,"15min":0.875},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.66700000000000004,"15min":0.70599999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82299999999999995,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.41799999999999998}]},{"osd":10,"last update":"Sat Feb 17 11:57:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":0.83799999999999997,"15min":0.78600000000000003},"min":{"1min":0.51400000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":12.319000000000001,"5min":12.319000000000001,"15min":12.319000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.71999999999999997}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66600000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":457468,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.50700000000000001,"15min":0.497},"min":{"1min":0.41099999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41599999999999998}]},{"osd":1,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.39900000000000002,"15min":0.38300000000000001},"min":{"1min":0.308,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.60799999999999998,"5min":0.872,"15min":0.872},"last":0.497},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.41799999999999998,"15min":0.39600000000000002},"min":{"1min":0.30199999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.69099999999999995,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.47399999999999998}]},{"osd":2,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.36599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.65300000000000002,"15min":0.56299999999999994},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":7.0279999999999996,"15min":7.0279999999999996},"last":0.69799999999999995}]},{"osd":3,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80500000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54800000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.60199999999999998,"15min":0.56200000000000006},"min":{"1min":0.45800000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.622,"15min":0.56999999999999995},"min":{"1min":0.499,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82799999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.747}]},{"osd":6,"last update":"Sat Feb 17 11:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59099999999999997,"15min":0.57399999999999995},"min":{"1min":0.41999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77800000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83999999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.66000000000000003}]},{"osd":7,"last update":"Sat Feb 17 11:57:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56999999999999995,"15min":0.52300000000000002},"min":{"1min":0.42899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.872,"5min":0.997,"15min":0.997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.374,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67900000000000005}]},{"osd":8,"last update":"Sat Feb 17 11:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":0.67700000000000005},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90400000000000003,"5min":6.9660000000000002,"15min":6.9660000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72299999999999998,"15min":0.70899999999999996},"min":{"1min":0.52800000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.85199999999999998}]},{"osd":9,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.50800000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.94699999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.53900000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90500000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78300000000000003}]},{"osd":10,"last update":"Sat Feb 17 11:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76800000000000002,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0369999999999999,"5min":1.087,"15min":1.087},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.61299999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.87}]},{"osd":11,"last update":"Sat Feb 17 11:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.83499999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":1048203,"data_compressed":42023,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.495},"min":{"1min":0.42599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53300000000000003,"15min":0.49099999999999999},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.69399999999999995,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.621}]},{"osd":1,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57499999999999996,"15min":0.51900000000000002},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.89700000000000002,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.51600000000000001},"min":{"1min":0.443,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73799999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42899999999999999,"15min":0.41699999999999998},"min":{"1min":0.317,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44500000000000001,"15min":0.42099999999999999},"min":{"1min":0.311,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68100000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.48999999999999999}]},{"osd":3,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.53000000000000003},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.53300000000000003},"min":{"1min":0.40999999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.51200000000000001}]},{"osd":4,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61699999999999999,"15min":0.56899999999999995},"min":{"1min":0.39900000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79900000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.56999999999999995},"min":{"1min":0.40400000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.86499999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.75}]},{"osd":6,"last update":"Sat Feb 17 11:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.45200000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66100000000000003,"15min":0.64300000000000002},"min":{"1min":0.54700000000000004,"5min":0.439,"15min":0.439},"max":{"1min":0.94899999999999995,"5min":1.087,"15min":1.087},"last":0.66500000000000004}]},{"osd":7,"last update":"Sat Feb 17 11:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84399999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.52400000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.879,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.874}]},{"osd":8,"last update":"Sat Feb 17 11:56:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.626,"15min":0.61599999999999999},"min":{"1min":0.46700000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.90900000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92800000000000005,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84599999999999997}]},{"osd":9,"last update":"Sat Feb 17 11:57:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.52100000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.91400000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.56100000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93500000000000005,"5min":1.056,"15min":1.056},"last":0.72299999999999998}]},{"osd":10,"last update":"Sat Feb 17 11:57:49 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.745},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96399999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.82799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":67367,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63485,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":69868,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T11:57:56.244 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-17T11:57:56.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.380+0000 7f408db25640 1 -- 172.21.15.94:0/983180566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4088102380 msgr2=0x7f4088102800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.380+0000 7f408db25640 1 --2- 172.21.15.94:0/983180566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4088102380 0x7f4088102800 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f407c009900 tx=0x7f407c02bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:56.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.380+0000 7f408db25640 1 -- 172.21.15.94:0/983180566 shutdown_connections 2024-02-17T11:57:56.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.380+0000 7f408db25640 1 --2- 172.21.15.94:0/983180566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4088102d40 0x7f40881095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.380+0000 7f408db25640 1 --2- 172.21.15.94:0/983180566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4088102380 0x7f4088102800 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.380+0000 7f408db25640 1 --2- 172.21.15.94:0/983180566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4088101190 0x7f40881015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.380+0000 7f408db25640 1 -- 172.21.15.94:0/983180566 >> 172.21.15.94:0/983180566 conn(0x7f40880fc760 msgr2=0x7f40880feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:56.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.380+0000 7f408db25640 1 -- 172.21.15.94:0/983180566 shutdown_connections 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.380+0000 7f408db25640 1 -- 172.21.15.94:0/983180566 wait complete. 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.380+0000 7f408db25640 1 Processor -- start 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f408db25640 1 -- start start 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f408db25640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4088101190 0x7f4088197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f408db25640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4088102380 0x7f40881977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f408db25640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4088102d40 0x7f408819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f408db25640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408819ed50 con 0x7f4088101190 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f408db25640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408819eec0 con 0x7f4088102d40 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f408db25640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408819f080 con 0x7f4088102380 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f40877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4088101190 0x7f4088197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f40877fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4088101190 0x7f4088197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53320/0 (socket says 172.21.15.94:53320) 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f40877fe640 1 -- 172.21.15.94:0/2274373900 learned_addr learned my addr 172.21.15.94:0/2274373900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f4086ffd640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4088102380 0x7f40881977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:56.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f40877fe640 1 -- 172.21.15.94:0/2274373900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4088102380 msgr2=0x7f40881977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f40877fe640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4088102380 0x7f40881977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f40877fe640 1 -- 172.21.15.94:0/2274373900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4088102d40 msgr2=0x7f408819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f40877fe640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4088102d40 0x7f408819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f40877fe640 1 -- 172.21.15.94:0/2274373900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c009560 con 0x7f4088101190 2024-02-17T11:57:56.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f4086ffd640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4088102380 0x7f40881977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:56.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f40877fe640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4088101190 0x7f4088197260 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f407800d950 tx=0x7f407800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:56.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.381+0000 7f4084ff9640 1 -- 172.21.15.94:0/2274373900 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4078009c30 con 0x7f4088101190 2024-02-17T11:57:56.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.382+0000 7f408db25640 1 -- 172.21.15.94:0/2274373900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408819f360 con 0x7f4088101190 2024-02-17T11:57:56.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.382+0000 7f4084ff9640 1 -- 172.21.15.94:0/2274373900 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4078009d90 con 0x7f4088101190 2024-02-17T11:57:56.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.382+0000 7f4084ff9640 1 -- 172.21.15.94:0/2274373900 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407800f720 con 0x7f4088101190 2024-02-17T11:57:56.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.382+0000 7f408db25640 1 -- 172.21.15.94:0/2274373900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408819f820 con 0x7f4088101190 2024-02-17T11:57:56.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.382+0000 7f408db25640 1 -- 172.21.15.94:0/2274373900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40881084d0 con 0x7f4088101190 2024-02-17T11:57:56.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.383+0000 7f4084ff9640 1 -- 172.21.15.94:0/2274373900 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f4078010a60 con 0x7f4088101190 2024-02-17T11:57:56.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.384+0000 7f4084ff9640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f40700a2150 0x7f40700a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:56.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.384+0000 7f4086ffd640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f40700a2150 0x7f40700a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:56.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.384+0000 7f4084ff9640 1 -- 172.21.15.94:0/2274373900 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f40780cdca0 con 0x7f4088101190 2024-02-17T11:57:56.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.385+0000 7f4086ffd640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f40700a2150 0x7f40700a4610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4088198770 tx=0x7f407c005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:56.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.388+0000 7f4084ff9640 1 -- 172.21.15.94:0/2274373900 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4078096b10 con 0x7f4088101190 2024-02-17T11:57:56.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.559+0000 7f408db25640 1 -- 172.21.15.94:0/2274373900 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f408802cf50 con 0x7f40700a2150 2024-02-17T11:57:56.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.569+0000 7f4084ff9640 1 -- 172.21.15.94:0/2274373900 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345529 (secure 0 0 0) 0x7f408802cf50 con 0x7f40700a2150 2024-02-17T11:57:56.572 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:56.574 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-02-17T11:57:56.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.573+0000 7f408db25640 1 -- 172.21.15.94:0/2274373900 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f40700a2150 msgr2=0x7f40700a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.573+0000 7f408db25640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f40700a2150 0x7f40700a4610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4088198770 tx=0x7f407c005be0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.573+0000 7f408db25640 1 -- 172.21.15.94:0/2274373900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4088101190 msgr2=0x7f4088197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.573+0000 7f408db25640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4088101190 0x7f4088197260 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f407800d950 tx=0x7f407800de10 comp rx=0 tx=0).stop 2024-02-17T11:57:56.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.573+0000 7f408db25640 1 -- 172.21.15.94:0/2274373900 shutdown_connections 2024-02-17T11:57:56.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.573+0000 7f408db25640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f40700a2150 0x7f40700a4610 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.573+0000 7f408db25640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4088102d40 0x7f408819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.573+0000 7f408db25640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4088102380 0x7f40881977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.573+0000 7f408db25640 1 --2- 172.21.15.94:0/2274373900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4088101190 0x7f4088197260 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.573+0000 7f408db25640 1 -- 172.21.15.94:0/2274373900 >> 172.21.15.94:0/2274373900 conn(0x7f40880fc760 msgr2=0x7f40881056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:56.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.573+0000 7f408db25640 1 -- 172.21.15.94:0/2274373900 shutdown_connections 2024-02-17T11:57:56.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.573+0000 7f408db25640 1 -- 172.21.15.94:0/2274373900 wait complete. 2024-02-17T11:57:56.583 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-02-17T11:57:54.680570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662643,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":833,"num_read_kb":545,"num_write":358,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":790,"ondisk_log_size":790,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364884,"kb_used_data":11572,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576940,"statfs":{"total":1151940427776,"available":1151566786560,"internally_reserved":0,"allocated":11849728,"data_stored":7449595,"data_compressed":184182,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6072,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":76,"num_read_kb":46,"num_write":9,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002682"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.673752+0000","last_change":"2024-02-17T11:55:58.678553+0000","last_active":"2024-02-17T11:57:28.673752+0000","last_peered":"2024-02-17T11:57:28.673752+0000","last_clean":"2024-02-17T11:57:28.673752+0000","last_became_active":"2024-02-17T11:55:58.678212+0000","last_became_peered":"2024-02-17T11:55:58.678212+0000","last_unstale":"2024-02-17T11:57:28.673752+0000","last_undegraded":"2024-02-17T11:57:28.673752+0000","last_fullsized":"2024-02-17T11:57:28.673752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:09:18.011937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672220+0000","last_change":"2024-02-17T11:55:58.681666+0000","last_active":"2024-02-17T11:57:28.672220+0000","last_peered":"2024-02-17T11:57:28.672220+0000","last_clean":"2024-02-17T11:57:28.672220+0000","last_became_active":"2024-02-17T11:55:58.681305+0000","last_became_peered":"2024-02-17T11:55:58.681305+0000","last_unstale":"2024-02-17T11:57:28.672220+0000","last_undegraded":"2024-02-17T11:57:28.672220+0000","last_fullsized":"2024-02-17T11:57:28.672220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:40:03.514130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.682419+0000","last_change":"2024-02-17T11:55:58.689465+0000","last_active":"2024-02-17T11:57:28.682419+0000","last_peered":"2024-02-17T11:57:28.682419+0000","last_clean":"2024-02-17T11:57:28.682419+0000","last_became_active":"2024-02-17T11:55:58.689166+0000","last_became_peered":"2024-02-17T11:55:58.689166+0000","last_unstale":"2024-02-17T11:57:28.682419+0000","last_undegraded":"2024-02-17T11:57:28.682419+0000","last_fullsized":"2024-02-17T11:57:28.682419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:43:07.473303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.652932+0000","last_change":"2024-02-17T11:56:59.735287+0000","last_active":"2024-02-17T11:57:28.652932+0000","last_peered":"2024-02-17T11:57:28.652932+0000","last_clean":"2024-02-17T11:57:28.652932+0000","last_became_active":"2024-02-17T11:56:59.734916+0000","last_became_peered":"2024-02-17T11:56:59.734916+0000","last_unstale":"2024-02-17T11:57:28.652932+0000","last_undegraded":"2024-02-17T11:57:28.652932+0000","last_fullsized":"2024-02-17T11:57:28.652932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:25:03.955834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.669070+0000","last_change":"2024-02-17T11:55:58.677543+0000","last_active":"2024-02-17T11:57:28.669070+0000","last_peered":"2024-02-17T11:57:28.669070+0000","last_clean":"2024-02-17T11:57:28.669070+0000","last_became_active":"2024-02-17T11:55:58.677223+0000","last_became_peered":"2024-02-17T11:55:58.677223+0000","last_unstale":"2024-02-17T11:57:28.669070+0000","last_undegraded":"2024-02-17T11:57:28.669070+0000","last_fullsized":"2024-02-17T11:57:28.669070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:05:27.194963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.677305+0000","last_change":"2024-02-17T11:55:58.673756+0000","last_active":"2024-02-17T11:57:28.677305+0000","last_peered":"2024-02-17T11:57:28.677305+0000","last_clean":"2024-02-17T11:57:28.677305+0000","last_became_active":"2024-02-17T11:55:58.672775+0000","last_became_peered":"2024-02-17T11:55:58.672775+0000","last_unstale":"2024-02-17T11:57:28.677305+0000","last_undegraded":"2024-02-17T11:57:28.677305+0000","last_fullsized":"2024-02-17T11:57:28.677305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:12:18.658414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671796+0000","last_change":"2024-02-17T11:56:59.736210+0000","last_active":"2024-02-17T11:57:28.671796+0000","last_peered":"2024-02-17T11:57:28.671796+0000","last_clean":"2024-02-17T11:57:28.671796+0000","last_became_active":"2024-02-17T11:56:59.735834+0000","last_became_peered":"2024-02-17T11:56:59.735834+0000","last_unstale":"2024-02-17T11:57:28.671796+0000","last_undegraded":"2024-02-17T11:57:28.671796+0000","last_fullsized":"2024-02-17T11:57:28.671796+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:19:27.677453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671910+0000","last_change":"2024-02-17T11:55:58.675496+0000","last_active":"2024-02-17T11:57:28.671910+0000","last_peered":"2024-02-17T11:57:28.671910+0000","last_clean":"2024-02-17T11:57:28.671910+0000","last_became_active":"2024-02-17T11:55:58.675193+0000","last_became_peered":"2024-02-17T11:55:58.675193+0000","last_unstale":"2024-02-17T11:57:28.671910+0000","last_undegraded":"2024-02-17T11:57:28.671910+0000","last_fullsized":"2024-02-17T11:57:28.671910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:14:29.648857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685041+0000","last_change":"2024-02-17T11:55:58.680859+0000","last_active":"2024-02-17T11:57:28.685041+0000","last_peered":"2024-02-17T11:57:28.685041+0000","last_clean":"2024-02-17T11:57:28.685041+0000","last_became_active":"2024-02-17T11:55:58.680489+0000","last_became_peered":"2024-02-17T11:55:58.680489+0000","last_unstale":"2024-02-17T11:57:28.685041+0000","last_undegraded":"2024-02-17T11:57:28.685041+0000","last_fullsized":"2024-02-17T11:57:28.685041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:29.963349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.674284+0000","last_change":"2024-02-17T11:56:59.737386+0000","last_active":"2024-02-17T11:57:28.674284+0000","last_peered":"2024-02-17T11:57:28.674284+0000","last_clean":"2024-02-17T11:57:28.674284+0000","last_became_active":"2024-02-17T11:56:59.736973+0000","last_became_peered":"2024-02-17T11:56:59.736973+0000","last_unstale":"2024-02-17T11:57:28.674284+0000","last_undegraded":"2024-02-17T11:57:28.674284+0000","last_fullsized":"2024-02-17T11:57:28.674284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:11:46.180288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.683763+0000","last_change":"2024-02-17T11:55:58.688562+0000","last_active":"2024-02-17T11:57:28.683763+0000","last_peered":"2024-02-17T11:57:28.683763+0000","last_clean":"2024-02-17T11:57:28.683763+0000","last_became_active":"2024-02-17T11:55:58.688186+0000","last_became_peered":"2024-02-17T11:55:58.688186+0000","last_unstale":"2024-02-17T11:57:28.683763+0000","last_undegraded":"2024-02-17T11:57:28.683763+0000","last_fullsized":"2024-02-17T11:57:28.683763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:09.668937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.676809+0000","last_change":"2024-02-17T11:55:58.684989+0000","last_active":"2024-02-17T11:57:28.676809+0000","last_peered":"2024-02-17T11:57:28.676809+0000","last_clean":"2024-02-17T11:57:28.676809+0000","last_became_active":"2024-02-17T11:55:58.684656+0000","last_became_peered":"2024-02-17T11:55:58.684656+0000","last_unstale":"2024-02-17T11:57:28.676809+0000","last_undegraded":"2024-02-17T11:57:28.676809+0000","last_fullsized":"2024-02-17T11:57:28.676809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:29:43.480963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.667676+0000","last_change":"2024-02-17T11:55:58.662076+0000","last_active":"2024-02-17T11:57:28.667676+0000","last_peered":"2024-02-17T11:57:28.667676+0000","last_clean":"2024-02-17T11:57:28.667676+0000","last_became_active":"2024-02-17T11:55:58.661716+0000","last_became_peered":"2024-02-17T11:55:58.661716+0000","last_unstale":"2024-02-17T11:57:28.667676+0000","last_undegraded":"2024-02-17T11:57:28.667676+0000","last_fullsized":"2024-02-17T11:57:28.667676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:20:39.647774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671962+0000","last_change":"2024-02-17T11:55:58.690762+0000","last_active":"2024-02-17T11:57:28.671962+0000","last_peered":"2024-02-17T11:57:28.671962+0000","last_clean":"2024-02-17T11:57:28.671962+0000","last_became_active":"2024-02-17T11:55:58.690414+0000","last_became_peered":"2024-02-17T11:55:58.690414+0000","last_unstale":"2024-02-17T11:57:28.671962+0000","last_undegraded":"2024-02-17T11:57:28.671962+0000","last_fullsized":"2024-02-17T11:57:28.671962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:37:13.938474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.674249+0000","last_change":"2024-02-17T11:55:58.682391+0000","last_active":"2024-02-17T11:57:28.674249+0000","last_peered":"2024-02-17T11:57:28.674249+0000","last_clean":"2024-02-17T11:57:28.674249+0000","last_became_active":"2024-02-17T11:55:58.682048+0000","last_became_peered":"2024-02-17T11:55:58.682048+0000","last_unstale":"2024-02-17T11:57:28.674249+0000","last_undegraded":"2024-02-17T11:57:28.674249+0000","last_fullsized":"2024-02-17T11:57:28.674249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:03:51.667202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685713+0000","last_change":"2024-02-17T11:55:58.685782+0000","last_active":"2024-02-17T11:57:28.685713+0000","last_peered":"2024-02-17T11:57:28.685713+0000","last_clean":"2024-02-17T11:57:28.685713+0000","last_became_active":"2024-02-17T11:55:58.683087+0000","last_became_peered":"2024-02-17T11:55:58.683087+0000","last_unstale":"2024-02-17T11:57:28.685713+0000","last_undegraded":"2024-02-17T11:57:28.685713+0000","last_fullsized":"2024-02-17T11:57:28.685713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:30:39.653717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.684630+0000","last_change":"2024-02-17T11:55:58.688400+0000","last_active":"2024-02-17T11:57:28.684630+0000","last_peered":"2024-02-17T11:57:28.684630+0000","last_clean":"2024-02-17T11:57:28.684630+0000","last_became_active":"2024-02-17T11:55:58.688040+0000","last_became_peered":"2024-02-17T11:55:58.688040+0000","last_unstale":"2024-02-17T11:57:28.684630+0000","last_undegraded":"2024-02-17T11:57:28.684630+0000","last_fullsized":"2024-02-17T11:57:28.684630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:39:02.706547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.684948+0000","last_change":"2024-02-17T11:55:58.687454+0000","last_active":"2024-02-17T11:57:28.684948+0000","last_peered":"2024-02-17T11:57:28.684948+0000","last_clean":"2024-02-17T11:57:28.684948+0000","last_became_active":"2024-02-17T11:55:58.686432+0000","last_became_peered":"2024-02-17T11:55:58.686432+0000","last_unstale":"2024-02-17T11:57:28.684948+0000","last_undegraded":"2024-02-17T11:57:28.684948+0000","last_fullsized":"2024-02-17T11:57:28.684948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:26:31.497782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670573+0000","last_change":"2024-02-17T11:55:58.681806+0000","last_active":"2024-02-17T11:57:28.670573+0000","last_peered":"2024-02-17T11:57:28.670573+0000","last_clean":"2024-02-17T11:57:28.670573+0000","last_became_active":"2024-02-17T11:55:58.679764+0000","last_became_peered":"2024-02-17T11:55:58.679764+0000","last_unstale":"2024-02-17T11:57:28.670573+0000","last_undegraded":"2024-02-17T11:57:28.670573+0000","last_fullsized":"2024-02-17T11:57:28.670573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:13:14.733417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.887970+0000","last_change":"2024-02-17T11:56:59.887970+0000","last_active":"2024-02-17T11:56:59.887970+0000","last_peered":"2024-02-17T11:56:59.887970+0000","last_clean":"2024-02-17T11:56:59.887970+0000","last_became_active":"2024-02-17T11:56:59.882087+0000","last_became_peered":"2024-02-17T11:56:59.882087+0000","last_unstale":"2024-02-17T11:56:59.887970+0000","last_undegraded":"2024-02-17T11:56:59.887970+0000","last_fullsized":"2024-02-17T11:56:59.887970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:34:08.202837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672008+0000","last_change":"2024-02-17T11:55:58.690679+0000","last_active":"2024-02-17T11:57:28.672008+0000","last_peered":"2024-02-17T11:57:28.672008+0000","last_clean":"2024-02-17T11:57:28.672008+0000","last_became_active":"2024-02-17T11:55:58.690353+0000","last_became_peered":"2024-02-17T11:55:58.690353+0000","last_unstale":"2024-02-17T11:57:28.672008+0000","last_undegraded":"2024-02-17T11:57:28.672008+0000","last_fullsized":"2024-02-17T11:57:28.672008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:45:33.830784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733040+0000","last_change":"2024-02-17T11:55:57.677649+0000","last_active":"2024-02-17T11:56:59.733040+0000","last_peered":"2024-02-17T11:56:59.733040+0000","last_clean":"2024-02-17T11:56:59.733040+0000","last_became_active":"2024-02-17T11:55:57.677331+0000","last_became_peered":"2024-02-17T11:55:57.677331+0000","last_unstale":"2024-02-17T11:56:59.733040+0000","last_undegraded":"2024-02-17T11:56:59.733040+0000","last_fullsized":"2024-02-17T11:56:59.733040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:06.932255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.684630+0000","last_change":"2024-02-17T11:55:58.688575+0000","last_active":"2024-02-17T11:57:28.684630+0000","last_peered":"2024-02-17T11:57:28.684630+0000","last_clean":"2024-02-17T11:57:28.684630+0000","last_became_active":"2024-02-17T11:55:58.688238+0000","last_became_peered":"2024-02-17T11:55:58.688238+0000","last_unstale":"2024-02-17T11:57:28.684630+0000","last_undegraded":"2024-02-17T11:57:28.684630+0000","last_fullsized":"2024-02-17T11:57:28.684630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:17:19.774986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729622+0000","last_change":"2024-02-17T11:55:57.679676+0000","last_active":"2024-02-17T11:56:59.729622+0000","last_peered":"2024-02-17T11:56:59.729622+0000","last_clean":"2024-02-17T11:56:59.729622+0000","last_became_active":"2024-02-17T11:55:57.679350+0000","last_became_peered":"2024-02-17T11:55:57.679350+0000","last_unstale":"2024-02-17T11:56:59.729622+0000","last_undegraded":"2024-02-17T11:56:59.729622+0000","last_fullsized":"2024-02-17T11:56:59.729622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:52:45.072896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685733+0000","last_change":"2024-02-17T11:55:58.682212+0000","last_active":"2024-02-17T11:57:28.685733+0000","last_peered":"2024-02-17T11:57:28.685733+0000","last_clean":"2024-02-17T11:57:28.685733+0000","last_became_active":"2024-02-17T11:55:58.681773+0000","last_became_peered":"2024-02-17T11:55:58.681773+0000","last_unstale":"2024-02-17T11:57:28.685733+0000","last_undegraded":"2024-02-17T11:57:28.685733+0000","last_fullsized":"2024-02-17T11:57:28.685733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:28:18.577174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.741288+0000","last_change":"2024-02-17T11:56:59.741288+0000","last_active":"2024-02-17T11:56:59.741288+0000","last_peered":"2024-02-17T11:56:59.741288+0000","last_clean":"2024-02-17T11:56:59.741288+0000","last_became_active":"2024-02-17T11:56:59.734687+0000","last_became_peered":"2024-02-17T11:56:59.734687+0000","last_unstale":"2024-02-17T11:56:59.741288+0000","last_undegraded":"2024-02-17T11:56:59.741288+0000","last_fullsized":"2024-02-17T11:56:59.741288+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:49:15.242774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.678159+0000","last_change":"2024-02-17T11:55:58.672771+0000","last_active":"2024-02-17T11:57:28.678159+0000","last_peered":"2024-02-17T11:57:28.678159+0000","last_clean":"2024-02-17T11:57:28.678159+0000","last_became_active":"2024-02-17T11:55:58.671877+0000","last_became_peered":"2024-02-17T11:55:58.671877+0000","last_unstale":"2024-02-17T11:57:28.678159+0000","last_undegraded":"2024-02-17T11:57:28.678159+0000","last_fullsized":"2024-02-17T11:57:28.678159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:38:38.350695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'7","reported_seq":139,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:49.315935+0000","last_change":"2024-02-17T11:55:57.669396+0000","last_active":"2024-02-17T11:57:49.315935+0000","last_peered":"2024-02-17T11:57:49.315935+0000","last_clean":"2024-02-17T11:57:49.315935+0000","last_became_active":"2024-02-17T11:55:57.669116+0000","last_became_peered":"2024-02-17T11:55:57.669116+0000","last_unstale":"2024-02-17T11:57:49.315935+0000","last_undegraded":"2024-02-17T11:57:49.315935+0000","last_fullsized":"2024-02-17T11:57:49.315935+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:05:11.081309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.659888+0000","last_change":"2024-02-17T11:55:58.674379+0000","last_active":"2024-02-17T11:57:28.659888+0000","last_peered":"2024-02-17T11:57:28.659888+0000","last_clean":"2024-02-17T11:57:28.659888+0000","last_became_active":"2024-02-17T11:55:58.674015+0000","last_became_peered":"2024-02-17T11:55:58.674015+0000","last_unstale":"2024-02-17T11:57:28.659888+0000","last_undegraded":"2024-02-17T11:57:28.659888+0000","last_fullsized":"2024-02-17T11:57:28.659888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:44:20.410623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.734826+0000","last_change":"2024-02-17T11:55:57.668436+0000","last_active":"2024-02-17T11:56:59.734826+0000","last_peered":"2024-02-17T11:56:59.734826+0000","last_clean":"2024-02-17T11:56:59.734826+0000","last_became_active":"2024-02-17T11:55:57.668004+0000","last_became_peered":"2024-02-17T11:55:57.668004+0000","last_unstale":"2024-02-17T11:56:59.734826+0000","last_undegraded":"2024-02-17T11:56:59.734826+0000","last_fullsized":"2024-02-17T11:56:59.734826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:13:42.566644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.678282+0000","last_change":"2024-02-17T11:55:58.686888+0000","last_active":"2024-02-17T11:57:28.678282+0000","last_peered":"2024-02-17T11:57:28.678282+0000","last_clean":"2024-02-17T11:57:28.678282+0000","last_became_active":"2024-02-17T11:55:58.686376+0000","last_became_peered":"2024-02-17T11:55:58.686376+0000","last_unstale":"2024-02-17T11:57:28.678282+0000","last_undegraded":"2024-02-17T11:57:28.678282+0000","last_fullsized":"2024-02-17T11:57:28.678282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:58:48.292354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731458+0000","last_change":"2024-02-17T11:55:57.665717+0000","last_active":"2024-02-17T11:56:59.731458+0000","last_peered":"2024-02-17T11:56:59.731458+0000","last_clean":"2024-02-17T11:56:59.731458+0000","last_became_active":"2024-02-17T11:55:57.665381+0000","last_became_peered":"2024-02-17T11:55:57.665381+0000","last_unstale":"2024-02-17T11:56:59.731458+0000","last_undegraded":"2024-02-17T11:56:59.731458+0000","last_fullsized":"2024-02-17T11:56:59.731458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:15:27.424332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670008+0000","last_change":"2024-02-17T11:55:58.673261+0000","last_active":"2024-02-17T11:57:28.670008+0000","last_peered":"2024-02-17T11:57:28.670008+0000","last_clean":"2024-02-17T11:57:28.670008+0000","last_became_active":"2024-02-17T11:55:58.672912+0000","last_became_peered":"2024-02-17T11:55:58.672912+0000","last_unstale":"2024-02-17T11:57:28.670008+0000","last_undegraded":"2024-02-17T11:57:28.670008+0000","last_fullsized":"2024-02-17T11:57:28.670008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:42.605291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732598+0000","last_change":"2024-02-17T11:55:57.667792+0000","last_active":"2024-02-17T11:56:59.732598+0000","last_peered":"2024-02-17T11:56:59.732598+0000","last_clean":"2024-02-17T11:56:59.732598+0000","last_became_active":"2024-02-17T11:55:57.667361+0000","last_became_peered":"2024-02-17T11:55:57.667361+0000","last_unstale":"2024-02-17T11:56:59.732598+0000","last_undegraded":"2024-02-17T11:56:59.732598+0000","last_fullsized":"2024-02-17T11:56:59.732598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:48.073742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.668179+0000","last_change":"2024-02-17T11:55:58.672180+0000","last_active":"2024-02-17T11:57:28.668179+0000","last_peered":"2024-02-17T11:57:28.668179+0000","last_clean":"2024-02-17T11:57:28.668179+0000","last_became_active":"2024-02-17T11:55:58.671790+0000","last_became_peered":"2024-02-17T11:55:58.671790+0000","last_unstale":"2024-02-17T11:57:28.668179+0000","last_undegraded":"2024-02-17T11:57:28.668179+0000","last_fullsized":"2024-02-17T11:57:28.668179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:48:39.982924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":136,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:53.110454+0000","last_change":"2024-02-17T11:55:57.670085+0000","last_active":"2024-02-17T11:57:53.110454+0000","last_peered":"2024-02-17T11:57:53.110454+0000","last_clean":"2024-02-17T11:57:53.110454+0000","last_became_active":"2024-02-17T11:55:57.669648+0000","last_became_peered":"2024-02-17T11:55:57.669648+0000","last_unstale":"2024-02-17T11:57:53.110454+0000","last_undegraded":"2024-02-17T11:57:53.110454+0000","last_fullsized":"2024-02-17T11:57:53.110454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:31:34.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685714+0000","last_change":"2024-02-17T11:55:58.685080+0000","last_active":"2024-02-17T11:57:28.685714+0000","last_peered":"2024-02-17T11:57:28.685714+0000","last_clean":"2024-02-17T11:57:28.685714+0000","last_became_active":"2024-02-17T11:55:58.684763+0000","last_became_peered":"2024-02-17T11:55:58.684763+0000","last_unstale":"2024-02-17T11:57:28.685714+0000","last_undegraded":"2024-02-17T11:57:28.685714+0000","last_fullsized":"2024-02-17T11:57:28.685714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:59:53.763492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732988+0000","last_change":"2024-02-17T11:55:57.676391+0000","last_active":"2024-02-17T11:56:59.732988+0000","last_peered":"2024-02-17T11:56:59.732988+0000","last_clean":"2024-02-17T11:56:59.732988+0000","last_became_active":"2024-02-17T11:55:57.676103+0000","last_became_peered":"2024-02-17T11:55:57.676103+0000","last_unstale":"2024-02-17T11:56:59.732988+0000","last_undegraded":"2024-02-17T11:56:59.732988+0000","last_fullsized":"2024-02-17T11:56:59.732988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:37:52.866379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670655+0000","last_change":"2024-02-17T11:55:58.679274+0000","last_active":"2024-02-17T11:57:28.670655+0000","last_peered":"2024-02-17T11:57:28.670655+0000","last_clean":"2024-02-17T11:57:28.670655+0000","last_became_active":"2024-02-17T11:55:58.678945+0000","last_became_peered":"2024-02-17T11:55:58.678945+0000","last_unstale":"2024-02-17T11:57:28.670655+0000","last_undegraded":"2024-02-17T11:57:28.670655+0000","last_fullsized":"2024-02-17T11:57:28.670655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:12:20.502503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733259+0000","last_change":"2024-02-17T11:55:57.672732+0000","last_active":"2024-02-17T11:56:59.733259+0000","last_peered":"2024-02-17T11:56:59.733259+0000","last_clean":"2024-02-17T11:56:59.733259+0000","last_became_active":"2024-02-17T11:55:57.672409+0000","last_became_peered":"2024-02-17T11:55:57.672409+0000","last_unstale":"2024-02-17T11:56:59.733259+0000","last_undegraded":"2024-02-17T11:56:59.733259+0000","last_fullsized":"2024-02-17T11:56:59.733259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:07:31.556508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685399+0000","last_change":"2024-02-17T11:55:58.689983+0000","last_active":"2024-02-17T11:57:28.685399+0000","last_peered":"2024-02-17T11:57:28.685399+0000","last_clean":"2024-02-17T11:57:28.685399+0000","last_became_active":"2024-02-17T11:55:58.689629+0000","last_became_peered":"2024-02-17T11:55:58.689629+0000","last_unstale":"2024-02-17T11:57:28.685399+0000","last_undegraded":"2024-02-17T11:57:28.685399+0000","last_fullsized":"2024-02-17T11:57:28.685399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:12.321746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":289,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:53.344302+0000","last_change":"2024-02-17T11:55:57.673273+0000","last_active":"2024-02-17T11:57:53.344302+0000","last_peered":"2024-02-17T11:57:53.344302+0000","last_clean":"2024-02-17T11:57:53.344302+0000","last_became_active":"2024-02-17T11:55:57.672865+0000","last_became_peered":"2024-02-17T11:55:57.672865+0000","last_unstale":"2024-02-17T11:57:53.344302+0000","last_undegraded":"2024-02-17T11:57:53.344302+0000","last_fullsized":"2024-02-17T11:57:53.344302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:31:01.593798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67367,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":235,"num_read_kb":183,"num_write":36,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670386+0000","last_change":"2024-02-17T11:55:58.669296+0000","last_active":"2024-02-17T11:57:28.670386+0000","last_peered":"2024-02-17T11:57:28.670386+0000","last_clean":"2024-02-17T11:57:28.670386+0000","last_became_active":"2024-02-17T11:55:58.668927+0000","last_became_peered":"2024-02-17T11:55:58.668927+0000","last_unstale":"2024-02-17T11:57:28.670386+0000","last_undegraded":"2024-02-17T11:57:28.670386+0000","last_fullsized":"2024-02-17T11:57:28.670386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:25.177905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733466+0000","last_change":"2024-02-17T11:55:57.674523+0000","last_active":"2024-02-17T11:56:59.733466+0000","last_peered":"2024-02-17T11:56:59.733466+0000","last_clean":"2024-02-17T11:56:59.733466+0000","last_became_active":"2024-02-17T11:55:57.672375+0000","last_became_peered":"2024-02-17T11:55:57.672375+0000","last_unstale":"2024-02-17T11:56:59.733466+0000","last_undegraded":"2024-02-17T11:56:59.733466+0000","last_fullsized":"2024-02-17T11:56:59.733466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:15:21.568434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.666966+0000","last_change":"2024-02-17T11:55:58.682106+0000","last_active":"2024-02-17T11:57:28.666966+0000","last_peered":"2024-02-17T11:57:28.666966+0000","last_clean":"2024-02-17T11:57:28.666966+0000","last_became_active":"2024-02-17T11:55:58.681420+0000","last_became_peered":"2024-02-17T11:55:58.681420+0000","last_unstale":"2024-02-17T11:57:28.666966+0000","last_undegraded":"2024-02-17T11:57:28.666966+0000","last_fullsized":"2024-02-17T11:57:28.666966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:23:00.521690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732537+0000","last_change":"2024-02-17T11:55:57.675161+0000","last_active":"2024-02-17T11:56:59.732537+0000","last_peered":"2024-02-17T11:56:59.732537+0000","last_clean":"2024-02-17T11:56:59.732537+0000","last_became_active":"2024-02-17T11:55:57.674776+0000","last_became_peered":"2024-02-17T11:55:57.674776+0000","last_unstale":"2024-02-17T11:56:59.732537+0000","last_undegraded":"2024-02-17T11:56:59.732537+0000","last_fullsized":"2024-02-17T11:56:59.732537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:25:59.695014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.680450+0000","last_change":"2024-02-17T11:55:58.679319+0000","last_active":"2024-02-17T11:57:28.680450+0000","last_peered":"2024-02-17T11:57:28.680450+0000","last_clean":"2024-02-17T11:57:28.680450+0000","last_became_active":"2024-02-17T11:55:58.679001+0000","last_became_peered":"2024-02-17T11:55:58.679001+0000","last_unstale":"2024-02-17T11:57:28.680450+0000","last_undegraded":"2024-02-17T11:57:28.680450+0000","last_fullsized":"2024-02-17T11:57:28.680450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:36:48.493338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":138,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:50.316679+0000","last_change":"2024-02-17T11:55:57.670027+0000","last_active":"2024-02-17T11:57:50.316679+0000","last_peered":"2024-02-17T11:57:50.316679+0000","last_clean":"2024-02-17T11:57:50.316679+0000","last_became_active":"2024-02-17T11:55:57.669692+0000","last_became_peered":"2024-02-17T11:55:57.669692+0000","last_unstale":"2024-02-17T11:57:50.316679+0000","last_undegraded":"2024-02-17T11:57:50.316679+0000","last_fullsized":"2024-02-17T11:57:50.316679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:44:01.001640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.674132+0000","last_change":"2024-02-17T11:55:58.672811+0000","last_active":"2024-02-17T11:57:28.674132+0000","last_peered":"2024-02-17T11:57:28.674132+0000","last_clean":"2024-02-17T11:57:28.674132+0000","last_became_active":"2024-02-17T11:55:58.672469+0000","last_became_peered":"2024-02-17T11:55:58.672469+0000","last_unstale":"2024-02-17T11:57:28.674132+0000","last_undegraded":"2024-02-17T11:57:28.674132+0000","last_fullsized":"2024-02-17T11:57:28.674132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:07.451422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729069+0000","last_change":"2024-02-17T11:55:57.668031+0000","last_active":"2024-02-17T11:56:59.729069+0000","last_peered":"2024-02-17T11:56:59.729069+0000","last_clean":"2024-02-17T11:56:59.729069+0000","last_became_active":"2024-02-17T11:55:57.667344+0000","last_became_peered":"2024-02-17T11:55:57.667344+0000","last_unstale":"2024-02-17T11:56:59.729069+0000","last_undegraded":"2024-02-17T11:56:59.729069+0000","last_fullsized":"2024-02-17T11:56:59.729069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:14:20.773902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.676780+0000","last_change":"2024-02-17T11:55:58.682013+0000","last_active":"2024-02-17T11:57:28.676780+0000","last_peered":"2024-02-17T11:57:28.676780+0000","last_clean":"2024-02-17T11:57:28.676780+0000","last_became_active":"2024-02-17T11:55:58.681503+0000","last_became_peered":"2024-02-17T11:55:58.681503+0000","last_unstale":"2024-02-17T11:57:28.676780+0000","last_undegraded":"2024-02-17T11:57:28.676780+0000","last_fullsized":"2024-02-17T11:57:28.676780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:24:50.364090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729587+0000","last_change":"2024-02-17T11:55:57.666725+0000","last_active":"2024-02-17T11:56:59.729587+0000","last_peered":"2024-02-17T11:56:59.729587+0000","last_clean":"2024-02-17T11:56:59.729587+0000","last_became_active":"2024-02-17T11:55:57.666339+0000","last_became_peered":"2024-02-17T11:55:57.666339+0000","last_unstale":"2024-02-17T11:56:59.729587+0000","last_undegraded":"2024-02-17T11:56:59.729587+0000","last_fullsized":"2024-02-17T11:56:59.729587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:22:47.324016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.678483+0000","last_change":"2024-02-17T11:55:58.675995+0000","last_active":"2024-02-17T11:57:28.678483+0000","last_peered":"2024-02-17T11:57:28.678483+0000","last_clean":"2024-02-17T11:57:28.678483+0000","last_became_active":"2024-02-17T11:55:58.675655+0000","last_became_peered":"2024-02-17T11:55:58.675655+0000","last_unstale":"2024-02-17T11:57:28.678483+0000","last_undegraded":"2024-02-17T11:57:28.678483+0000","last_fullsized":"2024-02-17T11:57:28.678483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:12.465300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730891+0000","last_change":"2024-02-17T11:55:57.669500+0000","last_active":"2024-02-17T11:56:59.730891+0000","last_peered":"2024-02-17T11:56:59.730891+0000","last_clean":"2024-02-17T11:56:59.730891+0000","last_became_active":"2024-02-17T11:55:57.669148+0000","last_became_peered":"2024-02-17T11:55:57.669148+0000","last_unstale":"2024-02-17T11:56:59.730891+0000","last_undegraded":"2024-02-17T11:56:59.730891+0000","last_fullsized":"2024-02-17T11:56:59.730891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:33.821266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.668037+0000","last_change":"2024-02-17T11:55:58.678336+0000","last_active":"2024-02-17T11:57:28.668037+0000","last_peered":"2024-02-17T11:57:28.668037+0000","last_clean":"2024-02-17T11:57:28.668037+0000","last_became_active":"2024-02-17T11:55:58.677967+0000","last_became_peered":"2024-02-17T11:55:58.677967+0000","last_unstale":"2024-02-17T11:57:28.668037+0000","last_undegraded":"2024-02-17T11:57:28.668037+0000","last_fullsized":"2024-02-17T11:57:28.668037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:31:39.406673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732692+0000","last_change":"2024-02-17T11:55:57.674810+0000","last_active":"2024-02-17T11:56:59.732692+0000","last_peered":"2024-02-17T11:56:59.732692+0000","last_clean":"2024-02-17T11:56:59.732692+0000","last_became_active":"2024-02-17T11:55:57.674356+0000","last_became_peered":"2024-02-17T11:55:57.674356+0000","last_unstale":"2024-02-17T11:56:59.732692+0000","last_undegraded":"2024-02-17T11:56:59.732692+0000","last_fullsized":"2024-02-17T11:56:59.732692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:25:57.879239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.677955+0000","last_change":"2024-02-17T11:55:58.680928+0000","last_active":"2024-02-17T11:57:28.677955+0000","last_peered":"2024-02-17T11:57:28.677955+0000","last_clean":"2024-02-17T11:57:28.677955+0000","last_became_active":"2024-02-17T11:55:58.680521+0000","last_became_peered":"2024-02-17T11:55:58.680521+0000","last_unstale":"2024-02-17T11:57:28.677955+0000","last_undegraded":"2024-02-17T11:57:28.677955+0000","last_fullsized":"2024-02-17T11:57:28.677955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:15:15.739127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.675398+0000","last_change":"2024-02-17T11:55:58.674771+0000","last_active":"2024-02-17T11:57:28.675398+0000","last_peered":"2024-02-17T11:57:28.675398+0000","last_clean":"2024-02-17T11:57:28.675398+0000","last_became_active":"2024-02-17T11:55:58.674450+0000","last_became_peered":"2024-02-17T11:55:58.674450+0000","last_unstale":"2024-02-17T11:57:28.675398+0000","last_undegraded":"2024-02-17T11:57:28.675398+0000","last_fullsized":"2024-02-17T11:57:28.675398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.031576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":133,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:49.314502+0000","last_change":"2024-02-17T11:55:57.672868+0000","last_active":"2024-02-17T11:57:49.314502+0000","last_peered":"2024-02-17T11:57:49.314502+0000","last_clean":"2024-02-17T11:57:49.314502+0000","last_became_active":"2024-02-17T11:55:57.672584+0000","last_became_peered":"2024-02-17T11:55:57.672584+0000","last_unstale":"2024-02-17T11:57:49.314502+0000","last_undegraded":"2024-02-17T11:57:49.314502+0000","last_fullsized":"2024-02-17T11:57:49.314502+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:03:20.307036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.684813+0000","last_change":"2024-02-17T11:55:58.692594+0000","last_active":"2024-02-17T11:57:28.684813+0000","last_peered":"2024-02-17T11:57:28.684813+0000","last_clean":"2024-02-17T11:57:28.684813+0000","last_became_active":"2024-02-17T11:55:58.692258+0000","last_became_peered":"2024-02-17T11:55:58.692258+0000","last_unstale":"2024-02-17T11:57:28.684813+0000","last_undegraded":"2024-02-17T11:57:28.684813+0000","last_fullsized":"2024-02-17T11:57:28.684813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:28.497939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.880997+0000","last_change":"2024-02-17T11:55:57.669456+0000","last_active":"2024-02-17T11:56:59.880997+0000","last_peered":"2024-02-17T11:56:59.880997+0000","last_clean":"2024-02-17T11:56:59.880997+0000","last_became_active":"2024-02-17T11:55:57.669067+0000","last_became_peered":"2024-02-17T11:55:57.669067+0000","last_unstale":"2024-02-17T11:56:59.880997+0000","last_undegraded":"2024-02-17T11:56:59.880997+0000","last_fullsized":"2024-02-17T11:56:59.880997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:12.352296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732120+0000","last_change":"2024-02-17T11:54:46.125844+0000","last_active":"2024-02-17T11:56:59.732120+0000","last_peered":"2024-02-17T11:56:59.732120+0000","last_clean":"2024-02-17T11:56:59.732120+0000","last_became_active":"2024-02-17T11:54:46.125387+0000","last_became_peered":"2024-02-17T11:54:46.125387+0000","last_unstale":"2024-02-17T11:56:59.732120+0000","last_undegraded":"2024-02-17T11:56:59.732120+0000","last_fullsized":"2024-02-17T11:56:59.732120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_clean_scrub_stamp":"2024-02-17T11:51:21.581371+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:53:27.219974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671351+0000","last_change":"2024-02-17T11:55:58.681623+0000","last_active":"2024-02-17T11:57:28.671351+0000","last_peered":"2024-02-17T11:57:28.671351+0000","last_clean":"2024-02-17T11:57:28.671351+0000","last_became_active":"2024-02-17T11:55:58.681275+0000","last_became_peered":"2024-02-17T11:55:58.681275+0000","last_unstale":"2024-02-17T11:57:28.671351+0000","last_undegraded":"2024-02-17T11:57:28.671351+0000","last_fullsized":"2024-02-17T11:57:28.671351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:20.685634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732237+0000","last_change":"2024-02-17T11:55:57.679555+0000","last_active":"2024-02-17T11:56:59.732237+0000","last_peered":"2024-02-17T11:56:59.732237+0000","last_clean":"2024-02-17T11:56:59.732237+0000","last_became_active":"2024-02-17T11:55:57.679241+0000","last_became_peered":"2024-02-17T11:55:57.679241+0000","last_unstale":"2024-02-17T11:56:59.732237+0000","last_undegraded":"2024-02-17T11:56:59.732237+0000","last_fullsized":"2024-02-17T11:56:59.732237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:37.482582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729501+0000","last_change":"2024-02-17T11:55:57.679711+0000","last_active":"2024-02-17T11:56:59.729501+0000","last_peered":"2024-02-17T11:56:59.729501+0000","last_clean":"2024-02-17T11:56:59.729501+0000","last_became_active":"2024-02-17T11:55:57.679415+0000","last_became_peered":"2024-02-17T11:55:57.679415+0000","last_unstale":"2024-02-17T11:56:59.729501+0000","last_undegraded":"2024-02-17T11:56:59.729501+0000","last_fullsized":"2024-02-17T11:56:59.729501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:13.095530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.660308+0000","last_change":"2024-02-17T11:55:58.670275+0000","last_active":"2024-02-17T11:57:28.660308+0000","last_peered":"2024-02-17T11:57:28.660308+0000","last_clean":"2024-02-17T11:57:28.660308+0000","last_became_active":"2024-02-17T11:55:58.669926+0000","last_became_peered":"2024-02-17T11:55:58.669926+0000","last_unstale":"2024-02-17T11:57:28.660308+0000","last_undegraded":"2024-02-17T11:57:28.660308+0000","last_fullsized":"2024-02-17T11:57:28.660308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:48:35.078389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.675754+0000","last_change":"2024-02-17T11:55:57.670364+0000","last_active":"2024-02-17T11:57:28.675754+0000","last_peered":"2024-02-17T11:57:28.675754+0000","last_clean":"2024-02-17T11:57:28.675754+0000","last_became_active":"2024-02-17T11:55:57.669977+0000","last_became_peered":"2024-02-17T11:55:57.669977+0000","last_unstale":"2024-02-17T11:57:28.675754+0000","last_undegraded":"2024-02-17T11:57:28.675754+0000","last_fullsized":"2024-02-17T11:57:28.675754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:42:13.820390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.673177+0000","last_change":"2024-02-17T11:55:58.681587+0000","last_active":"2024-02-17T11:57:28.673177+0000","last_peered":"2024-02-17T11:57:28.673177+0000","last_clean":"2024-02-17T11:57:28.673177+0000","last_became_active":"2024-02-17T11:55:58.680024+0000","last_became_peered":"2024-02-17T11:55:58.680024+0000","last_unstale":"2024-02-17T11:57:28.673177+0000","last_undegraded":"2024-02-17T11:57:28.673177+0000","last_fullsized":"2024-02-17T11:57:28.673177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:31:40.399439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.881143+0000","last_change":"2024-02-17T11:55:57.674618+0000","last_active":"2024-02-17T11:56:59.881143+0000","last_peered":"2024-02-17T11:56:59.881143+0000","last_clean":"2024-02-17T11:56:59.881143+0000","last_became_active":"2024-02-17T11:55:57.674246+0000","last_became_peered":"2024-02-17T11:55:57.674246+0000","last_unstale":"2024-02-17T11:56:59.881143+0000","last_undegraded":"2024-02-17T11:56:59.881143+0000","last_fullsized":"2024-02-17T11:56:59.881143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:42:26.681061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671477+0000","last_change":"2024-02-17T11:56:59.734355+0000","last_active":"2024-02-17T11:57:28.671477+0000","last_peered":"2024-02-17T11:57:28.671477+0000","last_clean":"2024-02-17T11:57:28.671477+0000","last_became_active":"2024-02-17T11:56:59.733975+0000","last_became_peered":"2024-02-17T11:56:59.733975+0000","last_unstale":"2024-02-17T11:57:28.671477+0000","last_undegraded":"2024-02-17T11:57:28.671477+0000","last_fullsized":"2024-02-17T11:57:28.671477+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:37:51.426702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732149+0000","last_change":"2024-02-17T11:55:57.667998+0000","last_active":"2024-02-17T11:56:59.732149+0000","last_peered":"2024-02-17T11:56:59.732149+0000","last_clean":"2024-02-17T11:56:59.732149+0000","last_became_active":"2024-02-17T11:55:57.667688+0000","last_became_peered":"2024-02-17T11:55:57.667688+0000","last_unstale":"2024-02-17T11:56:59.732149+0000","last_undegraded":"2024-02-17T11:56:59.732149+0000","last_fullsized":"2024-02-17T11:56:59.732149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:07:19.782879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.659157+0000","last_change":"2024-02-17T11:55:58.676644+0000","last_active":"2024-02-17T11:57:28.659157+0000","last_peered":"2024-02-17T11:57:28.659157+0000","last_clean":"2024-02-17T11:57:28.659157+0000","last_became_active":"2024-02-17T11:55:58.674670+0000","last_became_peered":"2024-02-17T11:55:58.674670+0000","last_unstale":"2024-02-17T11:57:28.659157+0000","last_undegraded":"2024-02-17T11:57:28.659157+0000","last_fullsized":"2024-02-17T11:57:28.659157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:43:22.746537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732754+0000","last_change":"2024-02-17T11:55:57.671330+0000","last_active":"2024-02-17T11:56:59.732754+0000","last_peered":"2024-02-17T11:56:59.732754+0000","last_clean":"2024-02-17T11:56:59.732754+0000","last_became_active":"2024-02-17T11:55:57.670902+0000","last_became_peered":"2024-02-17T11:55:57.670902+0000","last_unstale":"2024-02-17T11:56:59.732754+0000","last_undegraded":"2024-02-17T11:56:59.732754+0000","last_fullsized":"2024-02-17T11:56:59.732754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:22:14.770339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672152+0000","last_change":"2024-02-17T11:55:58.676417+0000","last_active":"2024-02-17T11:57:28.672152+0000","last_peered":"2024-02-17T11:57:28.672152+0000","last_clean":"2024-02-17T11:57:28.672152+0000","last_became_active":"2024-02-17T11:55:58.675960+0000","last_became_peered":"2024-02-17T11:55:58.675960+0000","last_unstale":"2024-02-17T11:57:28.672152+0000","last_undegraded":"2024-02-17T11:57:28.672152+0000","last_fullsized":"2024-02-17T11:57:28.672152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:22:18.566872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.738907+0000","last_change":"2024-02-17T11:56:59.737823+0000","last_active":"2024-02-17T11:56:59.738907+0000","last_peered":"2024-02-17T11:56:59.738907+0000","last_clean":"2024-02-17T11:56:59.738907+0000","last_became_active":"2024-02-17T11:56:59.734043+0000","last_became_peered":"2024-02-17T11:56:59.734043+0000","last_unstale":"2024-02-17T11:56:59.738907+0000","last_undegraded":"2024-02-17T11:56:59.738907+0000","last_fullsized":"2024-02-17T11:56:59.738907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:11:09.171845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670581+0000","last_change":"2024-02-17T11:55:58.690030+0000","last_active":"2024-02-17T11:57:28.670581+0000","last_peered":"2024-02-17T11:57:28.670581+0000","last_clean":"2024-02-17T11:57:28.670581+0000","last_became_active":"2024-02-17T11:55:58.689687+0000","last_became_peered":"2024-02-17T11:55:58.689687+0000","last_unstale":"2024-02-17T11:57:28.670581+0000","last_undegraded":"2024-02-17T11:57:28.670581+0000","last_fullsized":"2024-02-17T11:57:28.670581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:33:43.175234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729187+0000","last_change":"2024-02-17T11:55:57.679734+0000","last_active":"2024-02-17T11:56:59.729187+0000","last_peered":"2024-02-17T11:56:59.729187+0000","last_clean":"2024-02-17T11:56:59.729187+0000","last_became_active":"2024-02-17T11:55:57.679328+0000","last_became_peered":"2024-02-17T11:55:57.679328+0000","last_unstale":"2024-02-17T11:56:59.729187+0000","last_undegraded":"2024-02-17T11:56:59.729187+0000","last_fullsized":"2024-02-17T11:56:59.729187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:50:09.813482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.683947+0000","last_change":"2024-02-17T11:55:58.688360+0000","last_active":"2024-02-17T11:57:28.683947+0000","last_peered":"2024-02-17T11:57:28.683947+0000","last_clean":"2024-02-17T11:57:28.683947+0000","last_became_active":"2024-02-17T11:55:58.687994+0000","last_became_peered":"2024-02-17T11:55:58.687994+0000","last_unstale":"2024-02-17T11:57:28.683947+0000","last_undegraded":"2024-02-17T11:57:28.683947+0000","last_fullsized":"2024-02-17T11:57:28.683947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:01:02.163238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733202+0000","last_change":"2024-02-17T11:55:57.675977+0000","last_active":"2024-02-17T11:56:59.733202+0000","last_peered":"2024-02-17T11:56:59.733202+0000","last_clean":"2024-02-17T11:56:59.733202+0000","last_became_active":"2024-02-17T11:55:57.675250+0000","last_became_peered":"2024-02-17T11:55:57.675250+0000","last_unstale":"2024-02-17T11:56:59.733202+0000","last_undegraded":"2024-02-17T11:56:59.733202+0000","last_fullsized":"2024-02-17T11:56:59.733202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:11:27.611692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685664+0000","last_change":"2024-02-17T11:55:58.688593+0000","last_active":"2024-02-17T11:57:28.685664+0000","last_peered":"2024-02-17T11:57:28.685664+0000","last_clean":"2024-02-17T11:57:28.685664+0000","last_became_active":"2024-02-17T11:55:58.688163+0000","last_became_peered":"2024-02-17T11:55:58.688163+0000","last_unstale":"2024-02-17T11:57:28.685664+0000","last_undegraded":"2024-02-17T11:57:28.685664+0000","last_fullsized":"2024-02-17T11:57:28.685664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:54.597116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733115+0000","last_change":"2024-02-17T11:55:57.676860+0000","last_active":"2024-02-17T11:56:59.733115+0000","last_peered":"2024-02-17T11:56:59.733115+0000","last_clean":"2024-02-17T11:56:59.733115+0000","last_became_active":"2024-02-17T11:55:57.676534+0000","last_became_peered":"2024-02-17T11:55:57.676534+0000","last_unstale":"2024-02-17T11:56:59.733115+0000","last_undegraded":"2024-02-17T11:56:59.733115+0000","last_fullsized":"2024-02-17T11:56:59.733115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:23:47.726264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.675151+0000","last_change":"2024-02-17T11:56:59.739615+0000","last_active":"2024-02-17T11:57:28.675151+0000","last_peered":"2024-02-17T11:57:28.675151+0000","last_clean":"2024-02-17T11:57:28.675151+0000","last_became_active":"2024-02-17T11:56:59.736220+0000","last_became_peered":"2024-02-17T11:56:59.736220+0000","last_unstale":"2024-02-17T11:57:28.675151+0000","last_undegraded":"2024-02-17T11:57:28.675151+0000","last_fullsized":"2024-02-17T11:57:28.675151+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:22:07.463713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.739390+0000","last_change":"2024-02-17T11:56:59.739390+0000","last_active":"2024-02-17T11:56:59.739390+0000","last_peered":"2024-02-17T11:56:59.739390+0000","last_clean":"2024-02-17T11:56:59.739390+0000","last_became_active":"2024-02-17T11:56:59.734852+0000","last_became_peered":"2024-02-17T11:56:59.734852+0000","last_unstale":"2024-02-17T11:56:59.739390+0000","last_undegraded":"2024-02-17T11:56:59.739390+0000","last_fullsized":"2024-02-17T11:56:59.739390+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:40:09.174557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.675878+0000","last_change":"2024-02-17T11:55:58.672985+0000","last_active":"2024-02-17T11:57:28.675878+0000","last_peered":"2024-02-17T11:57:28.675878+0000","last_clean":"2024-02-17T11:57:28.675878+0000","last_became_active":"2024-02-17T11:55:58.672622+0000","last_became_peered":"2024-02-17T11:55:58.672622+0000","last_unstale":"2024-02-17T11:57:28.675878+0000","last_undegraded":"2024-02-17T11:57:28.675878+0000","last_fullsized":"2024-02-17T11:57:28.675878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:45.555827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672536+0000","last_change":"2024-02-17T11:55:58.676127+0000","last_active":"2024-02-17T11:57:28.672536+0000","last_peered":"2024-02-17T11:57:28.672536+0000","last_clean":"2024-02-17T11:57:28.672536+0000","last_became_active":"2024-02-17T11:55:58.675784+0000","last_became_peered":"2024-02-17T11:55:58.675784+0000","last_unstale":"2024-02-17T11:57:28.672536+0000","last_undegraded":"2024-02-17T11:57:28.672536+0000","last_fullsized":"2024-02-17T11:57:28.672536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:56:32.648803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.663712+0000","last_change":"2024-02-17T11:55:58.682438+0000","last_active":"2024-02-17T11:57:28.663712+0000","last_peered":"2024-02-17T11:57:28.663712+0000","last_clean":"2024-02-17T11:57:28.663712+0000","last_became_active":"2024-02-17T11:55:58.682067+0000","last_became_peered":"2024-02-17T11:55:58.682067+0000","last_unstale":"2024-02-17T11:57:28.663712+0000","last_undegraded":"2024-02-17T11:57:28.663712+0000","last_fullsized":"2024-02-17T11:57:28.663712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:10:22.906450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671367+0000","last_change":"2024-02-17T11:55:58.686671+0000","last_active":"2024-02-17T11:57:28.671367+0000","last_peered":"2024-02-17T11:57:28.671367+0000","last_clean":"2024-02-17T11:57:28.671367+0000","last_became_active":"2024-02-17T11:55:58.686315+0000","last_became_peered":"2024-02-17T11:55:58.686315+0000","last_unstale":"2024-02-17T11:57:28.671367+0000","last_undegraded":"2024-02-17T11:57:28.671367+0000","last_fullsized":"2024-02-17T11:57:28.671367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:55:40.864847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671037+0000","last_change":"2024-02-17T11:55:58.684227+0000","last_active":"2024-02-17T11:57:28.671037+0000","last_peered":"2024-02-17T11:57:28.671037+0000","last_clean":"2024-02-17T11:57:28.671037+0000","last_became_active":"2024-02-17T11:55:58.683875+0000","last_became_peered":"2024-02-17T11:55:58.683875+0000","last_unstale":"2024-02-17T11:57:28.671037+0000","last_undegraded":"2024-02-17T11:57:28.671037+0000","last_fullsized":"2024-02-17T11:57:28.671037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:11:41.283161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.677463+0000","last_change":"2024-02-17T11:55:58.685964+0000","last_active":"2024-02-17T11:57:28.677463+0000","last_peered":"2024-02-17T11:57:28.677463+0000","last_clean":"2024-02-17T11:57:28.677463+0000","last_became_active":"2024-02-17T11:55:58.685350+0000","last_became_peered":"2024-02-17T11:55:58.685350+0000","last_unstale":"2024-02-17T11:57:28.677463+0000","last_undegraded":"2024-02-17T11:57:28.677463+0000","last_fullsized":"2024-02-17T11:57:28.677463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:22:34.600604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671590+0000","last_change":"2024-02-17T11:55:58.671291+0000","last_active":"2024-02-17T11:57:28.671590+0000","last_peered":"2024-02-17T11:57:28.671590+0000","last_clean":"2024-02-17T11:57:28.671590+0000","last_became_active":"2024-02-17T11:55:58.670925+0000","last_became_peered":"2024-02-17T11:55:58.670925+0000","last_unstale":"2024-02-17T11:57:28.671590+0000","last_undegraded":"2024-02-17T11:57:28.671590+0000","last_fullsized":"2024-02-17T11:57:28.671590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:26:03.120976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.667565+0000","last_change":"2024-02-17T11:55:58.673122+0000","last_active":"2024-02-17T11:57:28.667565+0000","last_peered":"2024-02-17T11:57:28.667565+0000","last_clean":"2024-02-17T11:57:28.667565+0000","last_became_active":"2024-02-17T11:55:58.672747+0000","last_became_peered":"2024-02-17T11:55:58.672747+0000","last_unstale":"2024-02-17T11:57:28.667565+0000","last_undegraded":"2024-02-17T11:57:28.667565+0000","last_fullsized":"2024-02-17T11:57:28.667565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:55:15.064073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.677328+0000","last_change":"2024-02-17T11:55:58.681276+0000","last_active":"2024-02-17T11:57:28.677328+0000","last_peered":"2024-02-17T11:57:28.677328+0000","last_clean":"2024-02-17T11:57:28.677328+0000","last_became_active":"2024-02-17T11:55:58.680889+0000","last_became_peered":"2024-02-17T11:55:58.680889+0000","last_unstale":"2024-02-17T11:57:28.677328+0000","last_undegraded":"2024-02-17T11:57:28.677328+0000","last_fullsized":"2024-02-17T11:57:28.677328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:19:34.330288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670939+0000","last_change":"2024-02-17T11:55:58.667475+0000","last_active":"2024-02-17T11:57:28.670939+0000","last_peered":"2024-02-17T11:57:28.670939+0000","last_clean":"2024-02-17T11:57:28.670939+0000","last_became_active":"2024-02-17T11:55:58.666577+0000","last_became_peered":"2024-02-17T11:55:58.666577+0000","last_unstale":"2024-02-17T11:57:28.670939+0000","last_undegraded":"2024-02-17T11:57:28.670939+0000","last_fullsized":"2024-02-17T11:57:28.670939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:51:35.732035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672928+0000","last_change":"2024-02-17T11:55:58.674557+0000","last_active":"2024-02-17T11:57:28.672928+0000","last_peered":"2024-02-17T11:57:28.672928+0000","last_clean":"2024-02-17T11:57:28.672928+0000","last_became_active":"2024-02-17T11:55:58.673874+0000","last_became_peered":"2024-02-17T11:55:58.673874+0000","last_unstale":"2024-02-17T11:57:28.672928+0000","last_undegraded":"2024-02-17T11:57:28.672928+0000","last_fullsized":"2024-02-17T11:57:28.672928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:18:00.812489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685895+0000","last_change":"2024-02-17T11:55:58.692725+0000","last_active":"2024-02-17T11:57:28.685895+0000","last_peered":"2024-02-17T11:57:28.685895+0000","last_clean":"2024-02-17T11:57:28.685895+0000","last_became_active":"2024-02-17T11:55:58.690880+0000","last_became_peered":"2024-02-17T11:55:58.690880+0000","last_unstale":"2024-02-17T11:57:28.685895+0000","last_undegraded":"2024-02-17T11:57:28.685895+0000","last_fullsized":"2024-02-17T11:57:28.685895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:03:44.724385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.666977+0000","last_change":"2024-02-17T11:55:58.679377+0000","last_active":"2024-02-17T11:57:28.666977+0000","last_peered":"2024-02-17T11:57:28.666977+0000","last_clean":"2024-02-17T11:57:28.666977+0000","last_became_active":"2024-02-17T11:55:58.679041+0000","last_became_peered":"2024-02-17T11:55:58.679041+0000","last_unstale":"2024-02-17T11:57:28.666977+0000","last_undegraded":"2024-02-17T11:57:28.666977+0000","last_fullsized":"2024-02-17T11:57:28.666977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:16:37.396424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.678575+0000","last_change":"2024-02-17T11:55:58.680665+0000","last_active":"2024-02-17T11:57:28.678575+0000","last_peered":"2024-02-17T11:57:28.678575+0000","last_clean":"2024-02-17T11:57:28.678575+0000","last_became_active":"2024-02-17T11:55:58.680311+0000","last_became_peered":"2024-02-17T11:55:58.680311+0000","last_unstale":"2024-02-17T11:57:28.678575+0000","last_undegraded":"2024-02-17T11:57:28.678575+0000","last_fullsized":"2024-02-17T11:57:28.678575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:19:11.455471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284587,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72041,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":697,"num_read_kb":426,"num_write":175,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":210750,"data_compressed":5598,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":456292,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.372},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.55100000000000005}]},{"osd":1,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63800000000000001}]},{"osd":2,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.49199999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.52000000000000002,"15min":0.52800000000000002},"min":{"1min":0.317,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68899999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.504,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52800000000000002}]},{"osd":5,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.372,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78000000000000003,"5min":1.484,"15min":1.484},"last":0.755},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84299999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.40100000000000002}]},{"osd":6,"last update":"Sat Feb 17 11:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82199999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.62},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.92300000000000004,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.58799999999999997}]},{"osd":8,"last update":"Sat Feb 17 11:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.70399999999999996}]},{"osd":9,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.78500000000000003,"15min":0.79400000000000004},"min":{"1min":0.54000000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.89500000000000002,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.73499999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90700000000000003,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.66100000000000003}]},{"osd":10,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.61199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95099999999999996,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0529999999999999,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.83899999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1115281,"data_compressed":43889,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.89300000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46300000000000002,"15min":0.47499999999999998},"min":{"1min":0.32400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.36399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73899999999999999,"5min":0.746,"15min":0.746},"last":0.435}]},{"osd":2,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81899999999999995,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.53500000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.79500000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.41699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71899999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.60999999999999999},"min":{"1min":0.48299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77500000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69299999999999995}]},{"osd":4,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.57499999999999996},"min":{"1min":0.373,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82899999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.31,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.47399999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.47799999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.68000000000000005}]},{"osd":6,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66500000000000004,"15min":0.68100000000000005},"min":{"1min":0.51800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93400000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.748},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80200000000000005,"5min":1.101,"15min":1.101},"last":0.86199999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.41499999999999998}]},{"osd":8,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71999999999999997,"15min":0.72799999999999998},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88700000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73999999999999999,"15min":0.75800000000000001},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94699999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.71199999999999997}]},{"osd":9,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73099999999999998,"15min":0.752},"min":{"1min":0.59799999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91800000000000004,"5min":1.014,"15min":1.014},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.73899999999999999,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.434,"15min":0.434},"max":{"1min":0.997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.92100000000000004}]},{"osd":11,"last update":"Sat Feb 17 11:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.56299999999999994,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.84699999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.877},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.60999999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77900000000000003}]}]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456645,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.42299999999999999},"min":{"1min":0.29999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.40500000000000003,"15min":0.41699999999999998},"min":{"1min":0.27300000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44500000000000001}]},{"osd":1,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60099999999999998,"15min":0.61899999999999999},"min":{"1min":0.45600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.499},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78600000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.43099999999999999}]},{"osd":2,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90700000000000003,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.40300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54000000000000004}]},{"osd":3,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.496,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47399999999999998,"15min":0.5},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.71899999999999997,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.317}]},{"osd":4,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.41199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85299999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.65000000000000002,"15min":0.67700000000000005},"min":{"1min":0.42499999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.87,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47499999999999998}]},{"osd":5,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60799999999999998,"15min":0.63400000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84799999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68500000000000005,"15min":0.71499999999999997},"min":{"1min":0.503,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98099999999999998,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.52900000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57099999999999995,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81299999999999994,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60199999999999998,"15min":0.626},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91800000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55300000000000005}]},{"osd":7,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72599999999999998,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.95299999999999996,"5min":1.05,"15min":1.05},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69199999999999995,"15min":0.71399999999999997},"min":{"1min":0.47699999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93700000000000006,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.59099999999999997}]},{"osd":8,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69799999999999995,"15min":0.72599999999999998},"min":{"1min":0.55200000000000005,"5min":0.442,"15min":0.442},"max":{"1min":0.99299999999999999,"5min":1.016,"15min":1.016},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70399999999999996,"15min":0.72299999999999998},"min":{"1min":0.51200000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.96599999999999997,"5min":1.034,"15min":1.034},"last":0.61799999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.70899999999999996,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.78800000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.57399999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.80500000000000005,"5min":1.069,"15min":1.069},"last":0.66800000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070515,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":455866,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.44,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70399999999999996,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.38100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.442}]},{"osd":1,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88900000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66400000000000003}]},{"osd":2,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46000000000000002,"15min":0.44},"min":{"1min":0.35999999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71499999999999997,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.40799999999999997}]},{"osd":3,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57399999999999995,"15min":0.59199999999999997},"min":{"1min":0.39000000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.45400000000000001,"5min":0.372,"15min":0.372},"max":{"1min":0.76200000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.48999999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.61699999999999999,"15min":0.60799999999999998},"min":{"1min":0.503,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85299999999999998,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.629,"15min":0.61699999999999999},"min":{"1min":0.44600000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81100000000000005,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56999999999999995,"15min":0.56699999999999995},"min":{"1min":0.379,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.78800000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.55200000000000005,"15min":0.54400000000000004},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73399999999999999,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.82199999999999995}]},{"osd":6,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85099999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.753},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.52500000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.628}]},{"osd":7,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.70199999999999996,"15min":0.69599999999999995},"min":{"1min":0.52300000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.69199999999999995,"15min":0.67800000000000005},"min":{"1min":0.53400000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70599999999999996}]},{"osd":9,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.51100000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.68700000000000006,"15min":0.66600000000000004},"min":{"1min":0.54300000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.84099999999999997}]},{"osd":10,"last update":"Sat Feb 17 11:57:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76400000000000001,"15min":0.74299999999999999},"min":{"1min":0.54900000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.75900000000000001,"15min":0.74299999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.90100000000000002}]},{"osd":11,"last update":"Sat Feb 17 11:57:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.88200000000000001}]}]},{"osd":7,"up_from":51,"seq":219043332152,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":1049051,"data_compressed":42023,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.36699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39600000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.41899999999999998},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65300000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.45100000000000001,"15min":0.433},"min":{"1min":0.33700000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68000000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.52900000000000003}]},{"osd":2,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.38600000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.89100000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.65500000000000003}]},{"osd":3,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.45600000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71899999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.41999999999999998}]},{"osd":4,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48599999999999999,"15min":0.47399999999999998},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.373},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85499999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.39200000000000002}]},{"osd":5,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67500000000000004,"15min":0.66100000000000003},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93200000000000005,"5min":1.333,"15min":1.333},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.47599999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83199999999999996,"5min":1.486,"15min":1.486},"last":0.55400000000000005}]},{"osd":6,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.47699999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91000000000000003,"5min":1.087,"15min":1.087},"last":0.57199999999999995}]},{"osd":8,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.55900000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96999999999999997,"5min":1.446,"15min":1.446},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92400000000000004,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.70099999999999996}]},{"osd":9,"last update":"Sat Feb 17 11:57:26 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47299999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.746,"15min":0.755},"min":{"1min":0.49299999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.034,"5min":1.052,"15min":1.052},"last":0.61499999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:56:50 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.498,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95599999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75700000000000001,"15min":0.76100000000000001},"min":{"1min":0.48599999999999999,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0009999999999999,"5min":1.02,"15min":1.02},"last":0.71499999999999997}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.754}]}]},{"osd":6,"up_from":46,"seq":197568495678,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":455917,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.35099999999999998},"min":{"1min":0.309,"5min":0.223,"15min":0.223},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.436},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38500000000000001,"15min":0.38300000000000001},"min":{"1min":0.28799999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53300000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.42599999999999999}]},{"osd":1,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70099999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.32500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.57899999999999996}]},{"osd":2,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71199999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":0.871,"15min":0.871},"last":0.496}]},{"osd":3,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64400000000000002,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.439},"min":{"1min":0.27600000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.628,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.30499999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.39300000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.39500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.79300000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.65100000000000002}]},{"osd":5,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59299999999999997,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.503,"5min":0.432,"15min":0.432},"max":{"1min":0.86899999999999999,"5min":0.997,"15min":0.997},"last":0.47799999999999998}]},{"osd":7,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.622},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.45400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.71099999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59599999999999997}]},{"osd":8,"last update":"Sat Feb 17 11:57:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.71799999999999997,"15min":0.70099999999999996},"min":{"1min":0.46400000000000002,"5min":0.434,"15min":0.434},"max":{"1min":5.4269999999999996,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.50600000000000001}]},{"osd":9,"last update":"Sat Feb 17 11:57:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.625,"15min":0.65700000000000003},"min":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79300000000000004,"5min":0.879,"15min":0.879},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.62,"5min":0.65600000000000003,"15min":0.68300000000000005},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82699999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.56899999999999995}]},{"osd":10,"last update":"Sat Feb 17 11:57:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.73599999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.70199999999999996,"15min":0.72099999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.753,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":522686,"data_compressed":7701,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54500000000000004,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77800000000000002,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.496},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71199999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57899999999999996}]},{"osd":2,"last update":"Sat Feb 17 11:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57799999999999996,"15min":0.53200000000000003},"min":{"1min":0.41099999999999998,"5min":0.313,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57899999999999996,"15min":0.53300000000000003},"min":{"1min":0.52800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81899999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.74399999999999999}]},{"osd":3,"last update":"Sat Feb 17 11:56:51 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58399999999999996,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.46500000000000002,"15min":0.433},"min":{"1min":0.376,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.46899999999999997,"15min":0.42099999999999999},"min":{"1min":0.35699999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.56100000000000005}]},{"osd":5,"last update":"Sat Feb 17 11:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.66100000000000003,"15min":0.59099999999999997},"min":{"1min":0.58099999999999996,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67900000000000005,"15min":0.60899999999999999},"min":{"1min":0.64300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.871,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79200000000000004}]},{"osd":6,"last update":"Sat Feb 17 11:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63300000000000001,"15min":0.60099999999999998},"min":{"1min":0.52000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.88300000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.62,"15min":0.59699999999999998},"min":{"1min":0.51000000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78600000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45100000000000001}]},{"osd":7,"last update":"Sat Feb 17 11:57:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91100000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65100000000000002,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93799999999999994,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.64400000000000002}]},{"osd":8,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.71799999999999997,"15min":0.68799999999999994},"min":{"1min":0.622,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98599999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66500000000000004,"15min":0.65400000000000003},"min":{"1min":0.56899999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.83799999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63200000000000001}]},{"osd":9,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.59699999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89900000000000002,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78900000000000003,"15min":0.77700000000000002},"min":{"1min":0.66300000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92200000000000004,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72899999999999998}]},{"osd":10,"last update":"Sat Feb 17 11:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.48899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92500000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79000000000000004,"15min":0.77600000000000002},"min":{"1min":0.66800000000000004,"5min":0.501,"15min":0.501},"max":{"1min":1.002,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.82899999999999996}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.86899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":457188,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.53700000000000003,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52600000000000002,"15min":0.47999999999999998},"min":{"1min":0.40400000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73199999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57899999999999996}]},{"osd":2,"last update":"Sat Feb 17 11:57:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.52300000000000002},"min":{"1min":0.442,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57999999999999996,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.61499999999999999}]},{"osd":3,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat Feb 17 11:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54900000000000004,"15min":0.51100000000000001},"min":{"1min":0.39200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67700000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.52300000000000002},"min":{"1min":0.35099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75800000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63}]},{"osd":5,"last update":"Sat Feb 17 11:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.62,"15min":0.59499999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.46899999999999997}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.498,"15min":0.502},"min":{"1min":0.307,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64200000000000002,"5min":0.877,"15min":0.877},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68799999999999994,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.44500000000000001}]},{"osd":7,"last update":"Sat Feb 17 11:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.501,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58399999999999996}]},{"osd":8,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.69499999999999995,"15min":0.70599999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.46600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.53500000000000003}]},{"osd":9,"last update":"Sat Feb 17 11:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.61499999999999999,"15min":0.64500000000000002},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86099999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.498},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.68200000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76700000000000002,"5min":1.335,"15min":1.335},"last":0.48499999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68500000000000005,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.79400000000000004,"5min":1.107,"15min":1.107},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.745},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89900000000000002,"5min":1.236,"15min":1.236},"last":0.70399999999999996}]},{"osd":11,"last update":"Sat Feb 17 11:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70199999999999996,"15min":0.70499999999999996},"min":{"1min":0.55600000000000005,"5min":0.495,"15min":0.495},"max":{"1min":0.84399999999999997,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.67000000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":518887,"data_compressed":7701,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.498},"min":{"1min":0.32200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76700000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.54600000000000004,"15min":0.50800000000000001},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.29199999999999998},"max":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60899999999999999}]},{"osd":1,"last update":"Sat Feb 17 11:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58999999999999997,"15min":0.501},"min":{"1min":0.45100000000000001,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.89400000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.70799999999999996}]},{"osd":3,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.54500000000000004},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.753,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.4750000000000001,"5min":0.71899999999999997,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":16.718,"5min":16.718,"15min":16.718},"last":0.48099999999999998}]},{"osd":4,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.82599999999999996}]},{"osd":5,"last update":"Sat Feb 17 11:57:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.48599999999999999,"15min":0.438},"min":{"1min":0.378,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.47299999999999998,"15min":0.44700000000000001},"min":{"1min":0.35399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66100000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60799999999999998,"15min":0.58399999999999996},"min":{"1min":0.40799999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.72799999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57799999999999996}]},{"osd":7,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63700000000000001}]},{"osd":8,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.41199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.79000000000000004}]},{"osd":9,"last update":"Sat Feb 17 11:57:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90500000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.66400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80600000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.64300000000000002}]},{"osd":10,"last update":"Sat Feb 17 11:57:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.63900000000000001,"15min":0.61399999999999999},"min":{"1min":0.438,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.749},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.85999999999999999}]},{"osd":11,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.76100000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456111,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.35799999999999998,"15min":0.34399999999999997},"min":{"1min":0.26100000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.745,"5min":0.746,"15min":0.746},"last":0.309},{"interface":"front","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.34499999999999997},"min":{"1min":0.26400000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.53000000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.35699999999999998}]},{"osd":1,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.41699999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.53600000000000003},"min":{"1min":0.36799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78300000000000003,"5min":0.87,"15min":0.87},"last":0.504}]},{"osd":2,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77100000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78700000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51300000000000001}]},{"osd":4,"last update":"Sat Feb 17 11:57:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":0.71399999999999997,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":12.353,"5min":12.353,"15min":12.353},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60499999999999998,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.58199999999999996}]},{"osd":5,"last update":"Sat Feb 17 11:57:48 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":0.68999999999999995,"15min":0.60999999999999999},"min":{"1min":0.46700000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":12.337999999999999,"5min":12.337999999999999,"15min":12.337999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.749,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.496}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.44400000000000001},"min":{"1min":0.307,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82999999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.373},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95799999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.39100000000000001}]},{"osd":7,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":12.301,"5min":12.301,"15min":12.301},"last":0.69799999999999995}]},{"osd":8,"last update":"Sat Feb 17 11:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.49199999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97599999999999998,"5min":1.214,"15min":1.214},"last":0.622}]},{"osd":9,"last update":"Sat Feb 17 11:57:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.57399999999999995,"15min":0.60699999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64100000000000001,"5min":0.875,"15min":0.875},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.66700000000000004,"15min":0.70599999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82299999999999995,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.41799999999999998}]},{"osd":10,"last update":"Sat Feb 17 11:57:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":0.83799999999999997,"15min":0.78600000000000003},"min":{"1min":0.51400000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":12.319000000000001,"5min":12.319000000000001,"15min":12.319000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.71999999999999997}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66600000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":457468,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.50700000000000001,"15min":0.497},"min":{"1min":0.41099999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41599999999999998}]},{"osd":1,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.39900000000000002,"15min":0.38300000000000001},"min":{"1min":0.308,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.60799999999999998,"5min":0.872,"15min":0.872},"last":0.497},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.41799999999999998,"15min":0.39600000000000002},"min":{"1min":0.30199999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.69099999999999995,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.47399999999999998}]},{"osd":2,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.36599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.65300000000000002,"15min":0.56299999999999994},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":7.0279999999999996,"15min":7.0279999999999996},"last":0.69799999999999995}]},{"osd":3,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80500000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54800000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.60199999999999998,"15min":0.56200000000000006},"min":{"1min":0.45800000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.622,"15min":0.56999999999999995},"min":{"1min":0.499,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82799999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.747}]},{"osd":6,"last update":"Sat Feb 17 11:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59099999999999997,"15min":0.57399999999999995},"min":{"1min":0.41999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77800000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83999999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.66000000000000003}]},{"osd":7,"last update":"Sat Feb 17 11:57:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56999999999999995,"15min":0.52300000000000002},"min":{"1min":0.42899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.872,"5min":0.997,"15min":0.997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.374,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67900000000000005}]},{"osd":8,"last update":"Sat Feb 17 11:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":0.67700000000000005},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90400000000000003,"5min":6.9660000000000002,"15min":6.9660000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72299999999999998,"15min":0.70899999999999996},"min":{"1min":0.52800000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.85199999999999998}]},{"osd":9,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.50800000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.94699999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.53900000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90500000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78300000000000003}]},{"osd":10,"last update":"Sat Feb 17 11:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76800000000000002,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0369999999999999,"5min":1.087,"15min":1.087},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.61299999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.87}]},{"osd":11,"last update":"Sat Feb 17 11:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.83499999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757315,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":1048203,"data_compressed":42023,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.495},"min":{"1min":0.42599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53300000000000003,"15min":0.49099999999999999},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.69399999999999995,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.621}]},{"osd":1,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57499999999999996,"15min":0.51900000000000002},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.89700000000000002,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.51600000000000001},"min":{"1min":0.443,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73799999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42899999999999999,"15min":0.41699999999999998},"min":{"1min":0.317,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44500000000000001,"15min":0.42099999999999999},"min":{"1min":0.311,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68100000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.48999999999999999}]},{"osd":3,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.53000000000000003},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.53300000000000003},"min":{"1min":0.40999999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.51200000000000001}]},{"osd":4,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61699999999999999,"15min":0.56899999999999995},"min":{"1min":0.39900000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79900000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.56999999999999995},"min":{"1min":0.40400000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.86499999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.75}]},{"osd":6,"last update":"Sat Feb 17 11:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.45200000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66100000000000003,"15min":0.64300000000000002},"min":{"1min":0.54700000000000004,"5min":0.439,"15min":0.439},"max":{"1min":0.94899999999999995,"5min":1.087,"15min":1.087},"last":0.66500000000000004}]},{"osd":7,"last update":"Sat Feb 17 11:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84399999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.52400000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.879,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.874}]},{"osd":8,"last update":"Sat Feb 17 11:56:54 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.626,"15min":0.61599999999999999},"min":{"1min":0.46700000000000003,"5min":0.313,"15min":0.313},"max":{"1min":0.90900000000000003,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65900000000000003,"15min":0.66100000000000003},"min":{"1min":0.52000000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.92800000000000005,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.84599999999999997}]},{"osd":9,"last update":"Sat Feb 17 11:57:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.52100000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.91400000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.56100000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93500000000000005,"5min":1.056,"15min":1.056},"last":0.72299999999999998}]},{"osd":10,"last update":"Sat Feb 17 11:57:49 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.745},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96399999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.82799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":67367,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63485,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":69868,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T11:57:56.588 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-02-17T11:57:56.589 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-17T11:57:56.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.728+0000 7fc87f088640 1 -- 172.21.15.94:0/3529944027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8781000b0 msgr2=0x7fc8781004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.728+0000 7fc87f088640 1 --2- 172.21.15.94:0/3529944027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8781000b0 0x7fc8781004d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009900 tx=0x7fc86c02bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:56.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.728+0000 7fc87f088640 1 -- 172.21.15.94:0/3529944027 shutdown_connections 2024-02-17T11:57:56.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.728+0000 7fc87f088640 1 --2- 172.21.15.94:0/3529944027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc878101c60 0x7fc8781084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.728+0000 7fc87f088640 1 --2- 172.21.15.94:0/3529944027 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8781012a0 0x7fc878101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.728+0000 7fc87f088640 1 --2- 172.21.15.94:0/3529944027 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc8781000b0 0x7fc8781004d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.728+0000 7fc87f088640 1 -- 172.21.15.94:0/3529944027 >> 172.21.15.94:0/3529944027 conn(0x7fc8780fb660 msgr2=0x7fc8780fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:56.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87f088640 1 -- 172.21.15.94:0/3529944027 shutdown_connections 2024-02-17T11:57:56.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87f088640 1 -- 172.21.15.94:0/3529944027 wait complete. 2024-02-17T11:57:56.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87f088640 1 Processor -- start 2024-02-17T11:57:56.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87f088640 1 -- start start 2024-02-17T11:57:56.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87f088640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8781000b0 0x7fc87806a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:56.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87f088640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8781012a0 0x7fc87819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:56.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87f088640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc878101c60 0x7fc87819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:56.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87f088640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87819ed70 con 0x7fc878101c60 2024-02-17T11:57:56.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87f088640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87819eee0 con 0x7fc8781012a0 2024-02-17T11:57:56.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87f088640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87819f0a0 con 0x7fc8781000b0 2024-02-17T11:57:56.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87d625640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc878101c60 0x7fc87819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:56.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87d625640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc878101c60 0x7fc87819e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53338/0 (socket says 172.21.15.94:53338) 2024-02-17T11:57:56.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87d625640 1 -- 172.21.15.94:0/3403658725 learned_addr learned my addr 172.21.15.94:0/3403658725 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:56.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.729+0000 7fc87ce24640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8781000b0 0x7fc87806a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:56.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.730+0000 7fc877fff640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8781012a0 0x7fc87819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:56.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.730+0000 7fc87d625640 1 -- 172.21.15.94:0/3403658725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8781000b0 msgr2=0x7fc87806a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.730+0000 7fc87d625640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8781000b0 0x7fc87806a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.730+0000 7fc87d625640 1 -- 172.21.15.94:0/3403658725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8781012a0 msgr2=0x7fc87819bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.730+0000 7fc87d625640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8781012a0 0x7fc87819bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.730+0000 7fc87d625640 1 -- 172.21.15.94:0/3403658725 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c009560 con 0x7fc878101c60 2024-02-17T11:57:56.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.730+0000 7fc87d625640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc878101c60 0x7fc87819e590 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc87000eb10 tx=0x7fc87000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:56.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.730+0000 7fc875ffb640 1 -- 172.21.15.94:0/3403658725 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870004b20 con 0x7fc878101c60 2024-02-17T11:57:56.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.730+0000 7fc875ffb640 1 -- 172.21.15.94:0/3403658725 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc870004c80 con 0x7fc878101c60 2024-02-17T11:57:56.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.730+0000 7fc87f088640 1 -- 172.21.15.94:0/3403658725 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87819f320 con 0x7fc878101c60 2024-02-17T11:57:56.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.730+0000 7fc875ffb640 1 -- 172.21.15.94:0/3403658725 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870010650 con 0x7fc878101c60 2024-02-17T11:57:56.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.730+0000 7fc87f088640 1 -- 172.21.15.94:0/3403658725 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87819f880 con 0x7fc878101c60 2024-02-17T11:57:56.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.731+0000 7fc87f088640 1 -- 172.21.15.94:0/3403658725 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8781073d0 con 0x7fc878101c60 2024-02-17T11:57:56.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.732+0000 7fc875ffb640 1 -- 172.21.15.94:0/3403658725 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fc870005020 con 0x7fc878101c60 2024-02-17T11:57:56.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.733+0000 7fc875ffb640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc8680a21d0 0x7fc8680a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:56.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.733+0000 7fc87ce24640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc8680a21d0 0x7fc8680a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:56.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.733+0000 7fc875ffb640 1 -- 172.21.15.94:0/3403658725 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fc870014070 con 0x7fc878101c60 2024-02-17T11:57:56.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.733+0000 7fc87ce24640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc8680a21d0 0x7fc8680a4690 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc86c005ab0 tx=0x7fc86c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:56.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.737+0000 7fc875ffb640 1 -- 172.21.15.94:0/3403658725 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc870097240 con 0x7fc878101c60 2024-02-17T11:57:56.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.910+0000 7fc87f088640 1 -- 172.21.15.94:0/3403658725 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc87802cee0 con 0x7fc8680a21d0 2024-02-17T11:57:56.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.920+0000 7fc875ffb640 1 -- 172.21.15.94:0/3403658725 <== mgr.14152 v2:172.21.15.94:6800/1808883811 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345555 (secure 0 0 0) 0x7fc87802cee0 con 0x7fc8680a21d0 2024-02-17T11:57:56.923 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:56.925 INFO:teuthology.orchestra.run.smithi094.stderr:dumped all 2024-02-17T11:57:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.923+0000 7fc87f088640 1 -- 172.21.15.94:0/3403658725 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc8680a21d0 msgr2=0x7fc8680a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.923+0000 7fc87f088640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc8680a21d0 0x7fc8680a4690 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fc86c005ab0 tx=0x7fc86c005fd0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.924+0000 7fc87f088640 1 -- 172.21.15.94:0/3403658725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc878101c60 msgr2=0x7fc87819e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.924+0000 7fc87f088640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc878101c60 0x7fc87819e590 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc87000eb10 tx=0x7fc87000c2d0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.924+0000 7fc87f088640 1 -- 172.21.15.94:0/3403658725 shutdown_connections 2024-02-17T11:57:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.924+0000 7fc87f088640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fc8680a21d0 0x7fc8680a4690 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.924+0000 7fc87f088640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc878101c60 0x7fc87819e590 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.924+0000 7fc87f088640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc8781012a0 0x7fc87819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.924+0000 7fc87f088640 1 --2- 172.21.15.94:0/3403658725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc8781000b0 0x7fc87806a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.924+0000 7fc87f088640 1 -- 172.21.15.94:0/3403658725 >> 172.21.15.94:0/3403658725 conn(0x7fc8780fb660 msgr2=0x7fc8781045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:56.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.924+0000 7fc87f088640 1 -- 172.21.15.94:0/3403658725 shutdown_connections 2024-02-17T11:57:56.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:56.924+0000 7fc87f088640 1 -- 172.21.15.94:0/3403658725 wait complete. 2024-02-17T11:57:56.932 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:56 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3153798007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:56.933 INFO:teuthology.orchestra.run.smithi094.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-02-17T11:57:56.680978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663034,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":844,"num_read_kb":551,"num_write":360,"num_write_kb":3070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1181435,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":792,"ondisk_log_size":792,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364892,"kb_used_data":11580,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576932,"statfs":{"total":1151940427776,"available":1151566778368,"internally_reserved":0,"allocated":11857920,"data_stored":7457231,"data_compressed":184182,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19141,"internal_metadata":361674043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6463,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":72,"num_read_kb":42,"num_write":11,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002626"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.673752+0000","last_change":"2024-02-17T11:55:58.678553+0000","last_active":"2024-02-17T11:57:28.673752+0000","last_peered":"2024-02-17T11:57:28.673752+0000","last_clean":"2024-02-17T11:57:28.673752+0000","last_became_active":"2024-02-17T11:55:58.678212+0000","last_became_peered":"2024-02-17T11:55:58.678212+0000","last_unstale":"2024-02-17T11:57:28.673752+0000","last_undegraded":"2024-02-17T11:57:28.673752+0000","last_fullsized":"2024-02-17T11:57:28.673752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:09:18.011937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672220+0000","last_change":"2024-02-17T11:55:58.681666+0000","last_active":"2024-02-17T11:57:28.672220+0000","last_peered":"2024-02-17T11:57:28.672220+0000","last_clean":"2024-02-17T11:57:28.672220+0000","last_became_active":"2024-02-17T11:55:58.681305+0000","last_became_peered":"2024-02-17T11:55:58.681305+0000","last_unstale":"2024-02-17T11:57:28.672220+0000","last_undegraded":"2024-02-17T11:57:28.672220+0000","last_fullsized":"2024-02-17T11:57:28.672220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:40:03.514130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.682419+0000","last_change":"2024-02-17T11:55:58.689465+0000","last_active":"2024-02-17T11:57:28.682419+0000","last_peered":"2024-02-17T11:57:28.682419+0000","last_clean":"2024-02-17T11:57:28.682419+0000","last_became_active":"2024-02-17T11:55:58.689166+0000","last_became_peered":"2024-02-17T11:55:58.689166+0000","last_unstale":"2024-02-17T11:57:28.682419+0000","last_undegraded":"2024-02-17T11:57:28.682419+0000","last_fullsized":"2024-02-17T11:57:28.682419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:43:07.473303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.652932+0000","last_change":"2024-02-17T11:56:59.735287+0000","last_active":"2024-02-17T11:57:28.652932+0000","last_peered":"2024-02-17T11:57:28.652932+0000","last_clean":"2024-02-17T11:57:28.652932+0000","last_became_active":"2024-02-17T11:56:59.734916+0000","last_became_peered":"2024-02-17T11:56:59.734916+0000","last_unstale":"2024-02-17T11:57:28.652932+0000","last_undegraded":"2024-02-17T11:57:28.652932+0000","last_fullsized":"2024-02-17T11:57:28.652932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:25:03.955834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.669070+0000","last_change":"2024-02-17T11:55:58.677543+0000","last_active":"2024-02-17T11:57:28.669070+0000","last_peered":"2024-02-17T11:57:28.669070+0000","last_clean":"2024-02-17T11:57:28.669070+0000","last_became_active":"2024-02-17T11:55:58.677223+0000","last_became_peered":"2024-02-17T11:55:58.677223+0000","last_unstale":"2024-02-17T11:57:28.669070+0000","last_undegraded":"2024-02-17T11:57:28.669070+0000","last_fullsized":"2024-02-17T11:57:28.669070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:05:27.194963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.677305+0000","last_change":"2024-02-17T11:55:58.673756+0000","last_active":"2024-02-17T11:57:28.677305+0000","last_peered":"2024-02-17T11:57:28.677305+0000","last_clean":"2024-02-17T11:57:28.677305+0000","last_became_active":"2024-02-17T11:55:58.672775+0000","last_became_peered":"2024-02-17T11:55:58.672775+0000","last_unstale":"2024-02-17T11:57:28.677305+0000","last_undegraded":"2024-02-17T11:57:28.677305+0000","last_fullsized":"2024-02-17T11:57:28.677305+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:12:18.658414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671796+0000","last_change":"2024-02-17T11:56:59.736210+0000","last_active":"2024-02-17T11:57:28.671796+0000","last_peered":"2024-02-17T11:57:28.671796+0000","last_clean":"2024-02-17T11:57:28.671796+0000","last_became_active":"2024-02-17T11:56:59.735834+0000","last_became_peered":"2024-02-17T11:56:59.735834+0000","last_unstale":"2024-02-17T11:57:28.671796+0000","last_undegraded":"2024-02-17T11:57:28.671796+0000","last_fullsized":"2024-02-17T11:57:28.671796+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:19:27.677453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671910+0000","last_change":"2024-02-17T11:55:58.675496+0000","last_active":"2024-02-17T11:57:28.671910+0000","last_peered":"2024-02-17T11:57:28.671910+0000","last_clean":"2024-02-17T11:57:28.671910+0000","last_became_active":"2024-02-17T11:55:58.675193+0000","last_became_peered":"2024-02-17T11:55:58.675193+0000","last_unstale":"2024-02-17T11:57:28.671910+0000","last_undegraded":"2024-02-17T11:57:28.671910+0000","last_fullsized":"2024-02-17T11:57:28.671910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:14:29.648857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685041+0000","last_change":"2024-02-17T11:55:58.680859+0000","last_active":"2024-02-17T11:57:28.685041+0000","last_peered":"2024-02-17T11:57:28.685041+0000","last_clean":"2024-02-17T11:57:28.685041+0000","last_became_active":"2024-02-17T11:55:58.680489+0000","last_became_peered":"2024-02-17T11:55:58.680489+0000","last_unstale":"2024-02-17T11:57:28.685041+0000","last_undegraded":"2024-02-17T11:57:28.685041+0000","last_fullsized":"2024-02-17T11:57:28.685041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:29.963349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":39,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.674284+0000","last_change":"2024-02-17T11:56:59.737386+0000","last_active":"2024-02-17T11:57:28.674284+0000","last_peered":"2024-02-17T11:57:28.674284+0000","last_clean":"2024-02-17T11:57:28.674284+0000","last_became_active":"2024-02-17T11:56:59.736973+0000","last_became_peered":"2024-02-17T11:56:59.736973+0000","last_unstale":"2024-02-17T11:57:28.674284+0000","last_undegraded":"2024-02-17T11:57:28.674284+0000","last_fullsized":"2024-02-17T11:57:28.674284+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:11:46.180288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.683763+0000","last_change":"2024-02-17T11:55:58.688562+0000","last_active":"2024-02-17T11:57:28.683763+0000","last_peered":"2024-02-17T11:57:28.683763+0000","last_clean":"2024-02-17T11:57:28.683763+0000","last_became_active":"2024-02-17T11:55:58.688186+0000","last_became_peered":"2024-02-17T11:55:58.688186+0000","last_unstale":"2024-02-17T11:57:28.683763+0000","last_undegraded":"2024-02-17T11:57:28.683763+0000","last_fullsized":"2024-02-17T11:57:28.683763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:29:09.668937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.676809+0000","last_change":"2024-02-17T11:55:58.684989+0000","last_active":"2024-02-17T11:57:28.676809+0000","last_peered":"2024-02-17T11:57:28.676809+0000","last_clean":"2024-02-17T11:57:28.676809+0000","last_became_active":"2024-02-17T11:55:58.684656+0000","last_became_peered":"2024-02-17T11:55:58.684656+0000","last_unstale":"2024-02-17T11:57:28.676809+0000","last_undegraded":"2024-02-17T11:57:28.676809+0000","last_fullsized":"2024-02-17T11:57:28.676809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:29:43.480963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.667676+0000","last_change":"2024-02-17T11:55:58.662076+0000","last_active":"2024-02-17T11:57:28.667676+0000","last_peered":"2024-02-17T11:57:28.667676+0000","last_clean":"2024-02-17T11:57:28.667676+0000","last_became_active":"2024-02-17T11:55:58.661716+0000","last_became_peered":"2024-02-17T11:55:58.661716+0000","last_unstale":"2024-02-17T11:57:28.667676+0000","last_undegraded":"2024-02-17T11:57:28.667676+0000","last_fullsized":"2024-02-17T11:57:28.667676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:20:39.647774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671962+0000","last_change":"2024-02-17T11:55:58.690762+0000","last_active":"2024-02-17T11:57:28.671962+0000","last_peered":"2024-02-17T11:57:28.671962+0000","last_clean":"2024-02-17T11:57:28.671962+0000","last_became_active":"2024-02-17T11:55:58.690414+0000","last_became_peered":"2024-02-17T11:55:58.690414+0000","last_unstale":"2024-02-17T11:57:28.671962+0000","last_undegraded":"2024-02-17T11:57:28.671962+0000","last_fullsized":"2024-02-17T11:57:28.671962+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:37:13.938474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.674249+0000","last_change":"2024-02-17T11:55:58.682391+0000","last_active":"2024-02-17T11:57:28.674249+0000","last_peered":"2024-02-17T11:57:28.674249+0000","last_clean":"2024-02-17T11:57:28.674249+0000","last_became_active":"2024-02-17T11:55:58.682048+0000","last_became_peered":"2024-02-17T11:55:58.682048+0000","last_unstale":"2024-02-17T11:57:28.674249+0000","last_undegraded":"2024-02-17T11:57:28.674249+0000","last_fullsized":"2024-02-17T11:57:28.674249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:03:51.667202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685713+0000","last_change":"2024-02-17T11:55:58.685782+0000","last_active":"2024-02-17T11:57:28.685713+0000","last_peered":"2024-02-17T11:57:28.685713+0000","last_clean":"2024-02-17T11:57:28.685713+0000","last_became_active":"2024-02-17T11:55:58.683087+0000","last_became_peered":"2024-02-17T11:55:58.683087+0000","last_unstale":"2024-02-17T11:57:28.685713+0000","last_undegraded":"2024-02-17T11:57:28.685713+0000","last_fullsized":"2024-02-17T11:57:28.685713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:30:39.653717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.684630+0000","last_change":"2024-02-17T11:55:58.688400+0000","last_active":"2024-02-17T11:57:28.684630+0000","last_peered":"2024-02-17T11:57:28.684630+0000","last_clean":"2024-02-17T11:57:28.684630+0000","last_became_active":"2024-02-17T11:55:58.688040+0000","last_became_peered":"2024-02-17T11:55:58.688040+0000","last_unstale":"2024-02-17T11:57:28.684630+0000","last_undegraded":"2024-02-17T11:57:28.684630+0000","last_fullsized":"2024-02-17T11:57:28.684630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:39:02.706547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.684948+0000","last_change":"2024-02-17T11:55:58.687454+0000","last_active":"2024-02-17T11:57:28.684948+0000","last_peered":"2024-02-17T11:57:28.684948+0000","last_clean":"2024-02-17T11:57:28.684948+0000","last_became_active":"2024-02-17T11:55:58.686432+0000","last_became_peered":"2024-02-17T11:55:58.686432+0000","last_unstale":"2024-02-17T11:57:28.684948+0000","last_undegraded":"2024-02-17T11:57:28.684948+0000","last_fullsized":"2024-02-17T11:57:28.684948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:26:31.497782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670573+0000","last_change":"2024-02-17T11:55:58.681806+0000","last_active":"2024-02-17T11:57:28.670573+0000","last_peered":"2024-02-17T11:57:28.670573+0000","last_clean":"2024-02-17T11:57:28.670573+0000","last_became_active":"2024-02-17T11:55:58.679764+0000","last_became_peered":"2024-02-17T11:55:58.679764+0000","last_unstale":"2024-02-17T11:57:28.670573+0000","last_undegraded":"2024-02-17T11:57:28.670573+0000","last_fullsized":"2024-02-17T11:57:28.670573+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:13:14.733417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":45,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.887970+0000","last_change":"2024-02-17T11:56:59.887970+0000","last_active":"2024-02-17T11:56:59.887970+0000","last_peered":"2024-02-17T11:56:59.887970+0000","last_clean":"2024-02-17T11:56:59.887970+0000","last_became_active":"2024-02-17T11:56:59.882087+0000","last_became_peered":"2024-02-17T11:56:59.882087+0000","last_unstale":"2024-02-17T11:56:59.887970+0000","last_undegraded":"2024-02-17T11:56:59.887970+0000","last_fullsized":"2024-02-17T11:56:59.887970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:34:08.202837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672008+0000","last_change":"2024-02-17T11:55:58.690679+0000","last_active":"2024-02-17T11:57:28.672008+0000","last_peered":"2024-02-17T11:57:28.672008+0000","last_clean":"2024-02-17T11:57:28.672008+0000","last_became_active":"2024-02-17T11:55:58.690353+0000","last_became_peered":"2024-02-17T11:55:58.690353+0000","last_unstale":"2024-02-17T11:57:28.672008+0000","last_undegraded":"2024-02-17T11:57:28.672008+0000","last_fullsized":"2024-02-17T11:57:28.672008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:45:33.830784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733040+0000","last_change":"2024-02-17T11:55:57.677649+0000","last_active":"2024-02-17T11:56:59.733040+0000","last_peered":"2024-02-17T11:56:59.733040+0000","last_clean":"2024-02-17T11:56:59.733040+0000","last_became_active":"2024-02-17T11:55:57.677331+0000","last_became_peered":"2024-02-17T11:55:57.677331+0000","last_unstale":"2024-02-17T11:56:59.733040+0000","last_undegraded":"2024-02-17T11:56:59.733040+0000","last_fullsized":"2024-02-17T11:56:59.733040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:06.932255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.684630+0000","last_change":"2024-02-17T11:55:58.688575+0000","last_active":"2024-02-17T11:57:28.684630+0000","last_peered":"2024-02-17T11:57:28.684630+0000","last_clean":"2024-02-17T11:57:28.684630+0000","last_became_active":"2024-02-17T11:55:58.688238+0000","last_became_peered":"2024-02-17T11:55:58.688238+0000","last_unstale":"2024-02-17T11:57:28.684630+0000","last_undegraded":"2024-02-17T11:57:28.684630+0000","last_fullsized":"2024-02-17T11:57:28.684630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:17:19.774986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729622+0000","last_change":"2024-02-17T11:55:57.679676+0000","last_active":"2024-02-17T11:56:59.729622+0000","last_peered":"2024-02-17T11:56:59.729622+0000","last_clean":"2024-02-17T11:56:59.729622+0000","last_became_active":"2024-02-17T11:55:57.679350+0000","last_became_peered":"2024-02-17T11:55:57.679350+0000","last_unstale":"2024-02-17T11:56:59.729622+0000","last_undegraded":"2024-02-17T11:56:59.729622+0000","last_fullsized":"2024-02-17T11:56:59.729622+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:52:45.072896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685733+0000","last_change":"2024-02-17T11:55:58.682212+0000","last_active":"2024-02-17T11:57:28.685733+0000","last_peered":"2024-02-17T11:57:28.685733+0000","last_clean":"2024-02-17T11:57:28.685733+0000","last_became_active":"2024-02-17T11:55:58.681773+0000","last_became_peered":"2024-02-17T11:55:58.681773+0000","last_unstale":"2024-02-17T11:57:28.685733+0000","last_undegraded":"2024-02-17T11:57:28.685733+0000","last_fullsized":"2024-02-17T11:57:28.685733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:28:18.577174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.741288+0000","last_change":"2024-02-17T11:56:59.741288+0000","last_active":"2024-02-17T11:56:59.741288+0000","last_peered":"2024-02-17T11:56:59.741288+0000","last_clean":"2024-02-17T11:56:59.741288+0000","last_became_active":"2024-02-17T11:56:59.734687+0000","last_became_peered":"2024-02-17T11:56:59.734687+0000","last_unstale":"2024-02-17T11:56:59.741288+0000","last_undegraded":"2024-02-17T11:56:59.741288+0000","last_fullsized":"2024-02-17T11:56:59.741288+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:49:15.242774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.678159+0000","last_change":"2024-02-17T11:55:58.672771+0000","last_active":"2024-02-17T11:57:28.678159+0000","last_peered":"2024-02-17T11:57:28.678159+0000","last_clean":"2024-02-17T11:57:28.678159+0000","last_became_active":"2024-02-17T11:55:58.671877+0000","last_became_peered":"2024-02-17T11:55:58.671877+0000","last_unstale":"2024-02-17T11:57:28.678159+0000","last_undegraded":"2024-02-17T11:57:28.678159+0000","last_fullsized":"2024-02-17T11:57:28.678159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:38:38.350695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":146,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:54.323871+0000","last_change":"2024-02-17T11:55:57.669396+0000","last_active":"2024-02-17T11:57:54.323871+0000","last_peered":"2024-02-17T11:57:54.323871+0000","last_clean":"2024-02-17T11:57:54.323871+0000","last_became_active":"2024-02-17T11:55:57.669116+0000","last_became_peered":"2024-02-17T11:55:57.669116+0000","last_unstale":"2024-02-17T11:57:54.323871+0000","last_undegraded":"2024-02-17T11:57:54.323871+0000","last_fullsized":"2024-02-17T11:57:54.323871+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:05:11.081309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1545,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.659888+0000","last_change":"2024-02-17T11:55:58.674379+0000","last_active":"2024-02-17T11:57:28.659888+0000","last_peered":"2024-02-17T11:57:28.659888+0000","last_clean":"2024-02-17T11:57:28.659888+0000","last_became_active":"2024-02-17T11:55:58.674015+0000","last_became_peered":"2024-02-17T11:55:58.674015+0000","last_unstale":"2024-02-17T11:57:28.659888+0000","last_undegraded":"2024-02-17T11:57:28.659888+0000","last_fullsized":"2024-02-17T11:57:28.659888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:44:20.410623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.734826+0000","last_change":"2024-02-17T11:55:57.668436+0000","last_active":"2024-02-17T11:56:59.734826+0000","last_peered":"2024-02-17T11:56:59.734826+0000","last_clean":"2024-02-17T11:56:59.734826+0000","last_became_active":"2024-02-17T11:55:57.668004+0000","last_became_peered":"2024-02-17T11:55:57.668004+0000","last_unstale":"2024-02-17T11:56:59.734826+0000","last_undegraded":"2024-02-17T11:56:59.734826+0000","last_fullsized":"2024-02-17T11:56:59.734826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:13:42.566644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.678282+0000","last_change":"2024-02-17T11:55:58.686888+0000","last_active":"2024-02-17T11:57:28.678282+0000","last_peered":"2024-02-17T11:57:28.678282+0000","last_clean":"2024-02-17T11:57:28.678282+0000","last_became_active":"2024-02-17T11:55:58.686376+0000","last_became_peered":"2024-02-17T11:55:58.686376+0000","last_unstale":"2024-02-17T11:57:28.678282+0000","last_undegraded":"2024-02-17T11:57:28.678282+0000","last_fullsized":"2024-02-17T11:57:28.678282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:58:48.292354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.731458+0000","last_change":"2024-02-17T11:55:57.665717+0000","last_active":"2024-02-17T11:56:59.731458+0000","last_peered":"2024-02-17T11:56:59.731458+0000","last_clean":"2024-02-17T11:56:59.731458+0000","last_became_active":"2024-02-17T11:55:57.665381+0000","last_became_peered":"2024-02-17T11:55:57.665381+0000","last_unstale":"2024-02-17T11:56:59.731458+0000","last_undegraded":"2024-02-17T11:56:59.731458+0000","last_fullsized":"2024-02-17T11:56:59.731458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:15:27.424332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670008+0000","last_change":"2024-02-17T11:55:58.673261+0000","last_active":"2024-02-17T11:57:28.670008+0000","last_peered":"2024-02-17T11:57:28.670008+0000","last_clean":"2024-02-17T11:57:28.670008+0000","last_became_active":"2024-02-17T11:55:58.672912+0000","last_became_peered":"2024-02-17T11:55:58.672912+0000","last_unstale":"2024-02-17T11:57:28.670008+0000","last_undegraded":"2024-02-17T11:57:28.670008+0000","last_fullsized":"2024-02-17T11:57:28.670008+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:42.605291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732598+0000","last_change":"2024-02-17T11:55:57.667792+0000","last_active":"2024-02-17T11:56:59.732598+0000","last_peered":"2024-02-17T11:56:59.732598+0000","last_clean":"2024-02-17T11:56:59.732598+0000","last_became_active":"2024-02-17T11:55:57.667361+0000","last_became_peered":"2024-02-17T11:55:57.667361+0000","last_unstale":"2024-02-17T11:56:59.732598+0000","last_undegraded":"2024-02-17T11:56:59.732598+0000","last_fullsized":"2024-02-17T11:56:59.732598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:22:48.073742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.668179+0000","last_change":"2024-02-17T11:55:58.672180+0000","last_active":"2024-02-17T11:57:28.668179+0000","last_peered":"2024-02-17T11:57:28.668179+0000","last_clean":"2024-02-17T11:57:28.668179+0000","last_became_active":"2024-02-17T11:55:58.671790+0000","last_became_peered":"2024-02-17T11:55:58.671790+0000","last_unstale":"2024-02-17T11:57:28.668179+0000","last_undegraded":"2024-02-17T11:57:28.668179+0000","last_fullsized":"2024-02-17T11:57:28.668179+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:48:39.982924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'5","reported_seq":136,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:53.110454+0000","last_change":"2024-02-17T11:55:57.670085+0000","last_active":"2024-02-17T11:57:53.110454+0000","last_peered":"2024-02-17T11:57:53.110454+0000","last_clean":"2024-02-17T11:57:53.110454+0000","last_became_active":"2024-02-17T11:55:57.669648+0000","last_became_peered":"2024-02-17T11:55:57.669648+0000","last_unstale":"2024-02-17T11:57:53.110454+0000","last_undegraded":"2024-02-17T11:57:53.110454+0000","last_fullsized":"2024-02-17T11:57:53.110454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:31:34.052355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685714+0000","last_change":"2024-02-17T11:55:58.685080+0000","last_active":"2024-02-17T11:57:28.685714+0000","last_peered":"2024-02-17T11:57:28.685714+0000","last_clean":"2024-02-17T11:57:28.685714+0000","last_became_active":"2024-02-17T11:55:58.684763+0000","last_became_peered":"2024-02-17T11:55:58.684763+0000","last_unstale":"2024-02-17T11:57:28.685714+0000","last_undegraded":"2024-02-17T11:57:28.685714+0000","last_fullsized":"2024-02-17T11:57:28.685714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:59:53.763492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732988+0000","last_change":"2024-02-17T11:55:57.676391+0000","last_active":"2024-02-17T11:56:59.732988+0000","last_peered":"2024-02-17T11:56:59.732988+0000","last_clean":"2024-02-17T11:56:59.732988+0000","last_became_active":"2024-02-17T11:55:57.676103+0000","last_became_peered":"2024-02-17T11:55:57.676103+0000","last_unstale":"2024-02-17T11:56:59.732988+0000","last_undegraded":"2024-02-17T11:56:59.732988+0000","last_fullsized":"2024-02-17T11:56:59.732988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:37:52.866379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670655+0000","last_change":"2024-02-17T11:55:58.679274+0000","last_active":"2024-02-17T11:57:28.670655+0000","last_peered":"2024-02-17T11:57:28.670655+0000","last_clean":"2024-02-17T11:57:28.670655+0000","last_became_active":"2024-02-17T11:55:58.678945+0000","last_became_peered":"2024-02-17T11:55:58.678945+0000","last_unstale":"2024-02-17T11:57:28.670655+0000","last_undegraded":"2024-02-17T11:57:28.670655+0000","last_fullsized":"2024-02-17T11:57:28.670655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:12:20.502503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733259+0000","last_change":"2024-02-17T11:55:57.672732+0000","last_active":"2024-02-17T11:56:59.733259+0000","last_peered":"2024-02-17T11:56:59.733259+0000","last_clean":"2024-02-17T11:56:59.733259+0000","last_became_active":"2024-02-17T11:55:57.672409+0000","last_became_peered":"2024-02-17T11:55:57.672409+0000","last_unstale":"2024-02-17T11:56:59.733259+0000","last_undegraded":"2024-02-17T11:56:59.733259+0000","last_fullsized":"2024-02-17T11:56:59.733259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:07:31.556508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685399+0000","last_change":"2024-02-17T11:55:58.689983+0000","last_active":"2024-02-17T11:57:28.685399+0000","last_peered":"2024-02-17T11:57:28.685399+0000","last_clean":"2024-02-17T11:57:28.685399+0000","last_became_active":"2024-02-17T11:55:58.689629+0000","last_became_peered":"2024-02-17T11:55:58.689629+0000","last_unstale":"2024-02-17T11:57:28.685399+0000","last_undegraded":"2024-02-17T11:57:28.685399+0000","last_fullsized":"2024-02-17T11:57:28.685399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:34:12.321746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'35","reported_seq":289,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:53.344302+0000","last_change":"2024-02-17T11:55:57.673273+0000","last_active":"2024-02-17T11:57:53.344302+0000","last_peered":"2024-02-17T11:57:53.344302+0000","last_clean":"2024-02-17T11:57:53.344302+0000","last_became_active":"2024-02-17T11:55:57.672865+0000","last_became_peered":"2024-02-17T11:55:57.672865+0000","last_unstale":"2024-02-17T11:57:53.344302+0000","last_undegraded":"2024-02-17T11:57:53.344302+0000","last_fullsized":"2024-02-17T11:57:53.344302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:31:01.593798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67367,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":235,"num_read_kb":183,"num_write":36,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670386+0000","last_change":"2024-02-17T11:55:58.669296+0000","last_active":"2024-02-17T11:57:28.670386+0000","last_peered":"2024-02-17T11:57:28.670386+0000","last_clean":"2024-02-17T11:57:28.670386+0000","last_became_active":"2024-02-17T11:55:58.668927+0000","last_became_peered":"2024-02-17T11:55:58.668927+0000","last_unstale":"2024-02-17T11:57:28.670386+0000","last_undegraded":"2024-02-17T11:57:28.670386+0000","last_fullsized":"2024-02-17T11:57:28.670386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:25.177905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733466+0000","last_change":"2024-02-17T11:55:57.674523+0000","last_active":"2024-02-17T11:56:59.733466+0000","last_peered":"2024-02-17T11:56:59.733466+0000","last_clean":"2024-02-17T11:56:59.733466+0000","last_became_active":"2024-02-17T11:55:57.672375+0000","last_became_peered":"2024-02-17T11:55:57.672375+0000","last_unstale":"2024-02-17T11:56:59.733466+0000","last_undegraded":"2024-02-17T11:56:59.733466+0000","last_fullsized":"2024-02-17T11:56:59.733466+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:15:21.568434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.666966+0000","last_change":"2024-02-17T11:55:58.682106+0000","last_active":"2024-02-17T11:57:28.666966+0000","last_peered":"2024-02-17T11:57:28.666966+0000","last_clean":"2024-02-17T11:57:28.666966+0000","last_became_active":"2024-02-17T11:55:58.681420+0000","last_became_peered":"2024-02-17T11:55:58.681420+0000","last_unstale":"2024-02-17T11:57:28.666966+0000","last_undegraded":"2024-02-17T11:57:28.666966+0000","last_fullsized":"2024-02-17T11:57:28.666966+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:23:00.521690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732537+0000","last_change":"2024-02-17T11:55:57.675161+0000","last_active":"2024-02-17T11:56:59.732537+0000","last_peered":"2024-02-17T11:56:59.732537+0000","last_clean":"2024-02-17T11:56:59.732537+0000","last_became_active":"2024-02-17T11:55:57.674776+0000","last_became_peered":"2024-02-17T11:55:57.674776+0000","last_unstale":"2024-02-17T11:56:59.732537+0000","last_undegraded":"2024-02-17T11:56:59.732537+0000","last_fullsized":"2024-02-17T11:56:59.732537+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:25:59.695014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.680450+0000","last_change":"2024-02-17T11:55:58.679319+0000","last_active":"2024-02-17T11:57:28.680450+0000","last_peered":"2024-02-17T11:57:28.680450+0000","last_clean":"2024-02-17T11:57:28.680450+0000","last_became_active":"2024-02-17T11:55:58.679001+0000","last_became_peered":"2024-02-17T11:55:58.679001+0000","last_unstale":"2024-02-17T11:57:28.680450+0000","last_undegraded":"2024-02-17T11:57:28.680450+0000","last_fullsized":"2024-02-17T11:57:28.680450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:36:48.493338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'5","reported_seq":138,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:50.316679+0000","last_change":"2024-02-17T11:55:57.670027+0000","last_active":"2024-02-17T11:57:50.316679+0000","last_peered":"2024-02-17T11:57:50.316679+0000","last_clean":"2024-02-17T11:57:50.316679+0000","last_became_active":"2024-02-17T11:55:57.669692+0000","last_became_peered":"2024-02-17T11:55:57.669692+0000","last_unstale":"2024-02-17T11:57:50.316679+0000","last_undegraded":"2024-02-17T11:57:50.316679+0000","last_fullsized":"2024-02-17T11:57:50.316679+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:44:01.001640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.674132+0000","last_change":"2024-02-17T11:55:58.672811+0000","last_active":"2024-02-17T11:57:28.674132+0000","last_peered":"2024-02-17T11:57:28.674132+0000","last_clean":"2024-02-17T11:57:28.674132+0000","last_became_active":"2024-02-17T11:55:58.672469+0000","last_became_peered":"2024-02-17T11:55:58.672469+0000","last_unstale":"2024-02-17T11:57:28.674132+0000","last_undegraded":"2024-02-17T11:57:28.674132+0000","last_fullsized":"2024-02-17T11:57:28.674132+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:07.451422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729069+0000","last_change":"2024-02-17T11:55:57.668031+0000","last_active":"2024-02-17T11:56:59.729069+0000","last_peered":"2024-02-17T11:56:59.729069+0000","last_clean":"2024-02-17T11:56:59.729069+0000","last_became_active":"2024-02-17T11:55:57.667344+0000","last_became_peered":"2024-02-17T11:55:57.667344+0000","last_unstale":"2024-02-17T11:56:59.729069+0000","last_undegraded":"2024-02-17T11:56:59.729069+0000","last_fullsized":"2024-02-17T11:56:59.729069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:14:20.773902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.676780+0000","last_change":"2024-02-17T11:55:58.682013+0000","last_active":"2024-02-17T11:57:28.676780+0000","last_peered":"2024-02-17T11:57:28.676780+0000","last_clean":"2024-02-17T11:57:28.676780+0000","last_became_active":"2024-02-17T11:55:58.681503+0000","last_became_peered":"2024-02-17T11:55:58.681503+0000","last_unstale":"2024-02-17T11:57:28.676780+0000","last_undegraded":"2024-02-17T11:57:28.676780+0000","last_fullsized":"2024-02-17T11:57:28.676780+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:24:50.364090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729587+0000","last_change":"2024-02-17T11:55:57.666725+0000","last_active":"2024-02-17T11:56:59.729587+0000","last_peered":"2024-02-17T11:56:59.729587+0000","last_clean":"2024-02-17T11:56:59.729587+0000","last_became_active":"2024-02-17T11:55:57.666339+0000","last_became_peered":"2024-02-17T11:55:57.666339+0000","last_unstale":"2024-02-17T11:56:59.729587+0000","last_undegraded":"2024-02-17T11:56:59.729587+0000","last_fullsized":"2024-02-17T11:56:59.729587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:22:47.324016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.678483+0000","last_change":"2024-02-17T11:55:58.675995+0000","last_active":"2024-02-17T11:57:28.678483+0000","last_peered":"2024-02-17T11:57:28.678483+0000","last_clean":"2024-02-17T11:57:28.678483+0000","last_became_active":"2024-02-17T11:55:58.675655+0000","last_became_peered":"2024-02-17T11:55:58.675655+0000","last_unstale":"2024-02-17T11:57:28.678483+0000","last_undegraded":"2024-02-17T11:57:28.678483+0000","last_fullsized":"2024-02-17T11:57:28.678483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:12.465300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.730891+0000","last_change":"2024-02-17T11:55:57.669500+0000","last_active":"2024-02-17T11:56:59.730891+0000","last_peered":"2024-02-17T11:56:59.730891+0000","last_clean":"2024-02-17T11:56:59.730891+0000","last_became_active":"2024-02-17T11:55:57.669148+0000","last_became_peered":"2024-02-17T11:55:57.669148+0000","last_unstale":"2024-02-17T11:56:59.730891+0000","last_undegraded":"2024-02-17T11:56:59.730891+0000","last_fullsized":"2024-02-17T11:56:59.730891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:49:33.821266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.668037+0000","last_change":"2024-02-17T11:55:58.678336+0000","last_active":"2024-02-17T11:57:28.668037+0000","last_peered":"2024-02-17T11:57:28.668037+0000","last_clean":"2024-02-17T11:57:28.668037+0000","last_became_active":"2024-02-17T11:55:58.677967+0000","last_became_peered":"2024-02-17T11:55:58.677967+0000","last_unstale":"2024-02-17T11:57:28.668037+0000","last_undegraded":"2024-02-17T11:57:28.668037+0000","last_fullsized":"2024-02-17T11:57:28.668037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:31:39.406673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732692+0000","last_change":"2024-02-17T11:55:57.674810+0000","last_active":"2024-02-17T11:56:59.732692+0000","last_peered":"2024-02-17T11:56:59.732692+0000","last_clean":"2024-02-17T11:56:59.732692+0000","last_became_active":"2024-02-17T11:55:57.674356+0000","last_became_peered":"2024-02-17T11:55:57.674356+0000","last_unstale":"2024-02-17T11:56:59.732692+0000","last_undegraded":"2024-02-17T11:56:59.732692+0000","last_fullsized":"2024-02-17T11:56:59.732692+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:25:57.879239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.677955+0000","last_change":"2024-02-17T11:55:58.680928+0000","last_active":"2024-02-17T11:57:28.677955+0000","last_peered":"2024-02-17T11:57:28.677955+0000","last_clean":"2024-02-17T11:57:28.677955+0000","last_became_active":"2024-02-17T11:55:58.680521+0000","last_became_peered":"2024-02-17T11:55:58.680521+0000","last_unstale":"2024-02-17T11:57:28.677955+0000","last_undegraded":"2024-02-17T11:57:28.677955+0000","last_fullsized":"2024-02-17T11:57:28.677955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:15:15.739127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.675398+0000","last_change":"2024-02-17T11:55:58.674771+0000","last_active":"2024-02-17T11:57:28.675398+0000","last_peered":"2024-02-17T11:57:28.675398+0000","last_clean":"2024-02-17T11:57:28.675398+0000","last_became_active":"2024-02-17T11:55:58.674450+0000","last_became_peered":"2024-02-17T11:55:58.674450+0000","last_unstale":"2024-02-17T11:57:28.675398+0000","last_undegraded":"2024-02-17T11:57:28.675398+0000","last_fullsized":"2024-02-17T11:57:28.675398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:00:59.031576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'6","reported_seq":139,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:54.322499+0000","last_change":"2024-02-17T11:55:57.672868+0000","last_active":"2024-02-17T11:57:54.322499+0000","last_peered":"2024-02-17T11:57:54.322499+0000","last_clean":"2024-02-17T11:57:54.322499+0000","last_became_active":"2024-02-17T11:55:57.672584+0000","last_became_peered":"2024-02-17T11:55:57.672584+0000","last_unstale":"2024-02-17T11:57:54.322499+0000","last_undegraded":"2024-02-17T11:57:54.322499+0000","last_fullsized":"2024-02-17T11:57:54.322499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:03:20.307036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.684813+0000","last_change":"2024-02-17T11:55:58.692594+0000","last_active":"2024-02-17T11:57:28.684813+0000","last_peered":"2024-02-17T11:57:28.684813+0000","last_clean":"2024-02-17T11:57:28.684813+0000","last_became_active":"2024-02-17T11:55:58.692258+0000","last_became_peered":"2024-02-17T11:55:58.692258+0000","last_unstale":"2024-02-17T11:57:28.684813+0000","last_undegraded":"2024-02-17T11:57:28.684813+0000","last_fullsized":"2024-02-17T11:57:28.684813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:55:28.497939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.880997+0000","last_change":"2024-02-17T11:55:57.669456+0000","last_active":"2024-02-17T11:56:59.880997+0000","last_peered":"2024-02-17T11:56:59.880997+0000","last_clean":"2024-02-17T11:56:59.880997+0000","last_became_active":"2024-02-17T11:55:57.669067+0000","last_became_peered":"2024-02-17T11:55:57.669067+0000","last_unstale":"2024-02-17T11:56:59.880997+0000","last_undegraded":"2024-02-17T11:56:59.880997+0000","last_fullsized":"2024-02-17T11:56:59.880997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:10:12.352296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":354,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732120+0000","last_change":"2024-02-17T11:54:46.125844+0000","last_active":"2024-02-17T11:56:59.732120+0000","last_peered":"2024-02-17T11:56:59.732120+0000","last_clean":"2024-02-17T11:56:59.732120+0000","last_became_active":"2024-02-17T11:54:46.125387+0000","last_became_peered":"2024-02-17T11:54:46.125387+0000","last_unstale":"2024-02-17T11:56:59.732120+0000","last_undegraded":"2024-02-17T11:56:59.732120+0000","last_fullsized":"2024-02-17T11:56:59.732120+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:51:21.581371+0000","last_clean_scrub_stamp":"2024-02-17T11:51:21.581371+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T13:53:27.219974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671351+0000","last_change":"2024-02-17T11:55:58.681623+0000","last_active":"2024-02-17T11:57:28.671351+0000","last_peered":"2024-02-17T11:57:28.671351+0000","last_clean":"2024-02-17T11:57:28.671351+0000","last_became_active":"2024-02-17T11:55:58.681275+0000","last_became_peered":"2024-02-17T11:55:58.681275+0000","last_unstale":"2024-02-17T11:57:28.671351+0000","last_undegraded":"2024-02-17T11:57:28.671351+0000","last_fullsized":"2024-02-17T11:57:28.671351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:05:20.685634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732237+0000","last_change":"2024-02-17T11:55:57.679555+0000","last_active":"2024-02-17T11:56:59.732237+0000","last_peered":"2024-02-17T11:56:59.732237+0000","last_clean":"2024-02-17T11:56:59.732237+0000","last_became_active":"2024-02-17T11:55:57.679241+0000","last_became_peered":"2024-02-17T11:55:57.679241+0000","last_unstale":"2024-02-17T11:56:59.732237+0000","last_undegraded":"2024-02-17T11:56:59.732237+0000","last_fullsized":"2024-02-17T11:56:59.732237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:41:37.482582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729501+0000","last_change":"2024-02-17T11:55:57.679711+0000","last_active":"2024-02-17T11:56:59.729501+0000","last_peered":"2024-02-17T11:56:59.729501+0000","last_clean":"2024-02-17T11:56:59.729501+0000","last_became_active":"2024-02-17T11:55:57.679415+0000","last_became_peered":"2024-02-17T11:55:57.679415+0000","last_unstale":"2024-02-17T11:56:59.729501+0000","last_undegraded":"2024-02-17T11:56:59.729501+0000","last_fullsized":"2024-02-17T11:56:59.729501+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:32:13.095530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.660308+0000","last_change":"2024-02-17T11:55:58.670275+0000","last_active":"2024-02-17T11:57:28.660308+0000","last_peered":"2024-02-17T11:57:28.660308+0000","last_clean":"2024-02-17T11:57:28.660308+0000","last_became_active":"2024-02-17T11:55:58.669926+0000","last_became_peered":"2024-02-17T11:55:58.669926+0000","last_unstale":"2024-02-17T11:57:28.660308+0000","last_undegraded":"2024-02-17T11:57:28.660308+0000","last_fullsized":"2024-02-17T11:57:28.660308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:48:35.078389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.675754+0000","last_change":"2024-02-17T11:55:57.670364+0000","last_active":"2024-02-17T11:57:28.675754+0000","last_peered":"2024-02-17T11:57:28.675754+0000","last_clean":"2024-02-17T11:57:28.675754+0000","last_became_active":"2024-02-17T11:55:57.669977+0000","last_became_peered":"2024-02-17T11:55:57.669977+0000","last_unstale":"2024-02-17T11:57:28.675754+0000","last_undegraded":"2024-02-17T11:57:28.675754+0000","last_fullsized":"2024-02-17T11:57:28.675754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:42:13.820390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.673177+0000","last_change":"2024-02-17T11:55:58.681587+0000","last_active":"2024-02-17T11:57:28.673177+0000","last_peered":"2024-02-17T11:57:28.673177+0000","last_clean":"2024-02-17T11:57:28.673177+0000","last_became_active":"2024-02-17T11:55:58.680024+0000","last_became_peered":"2024-02-17T11:55:58.680024+0000","last_unstale":"2024-02-17T11:57:28.673177+0000","last_undegraded":"2024-02-17T11:57:28.673177+0000","last_fullsized":"2024-02-17T11:57:28.673177+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T18:31:40.399439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.881143+0000","last_change":"2024-02-17T11:55:57.674618+0000","last_active":"2024-02-17T11:56:59.881143+0000","last_peered":"2024-02-17T11:56:59.881143+0000","last_clean":"2024-02-17T11:56:59.881143+0000","last_became_active":"2024-02-17T11:55:57.674246+0000","last_became_peered":"2024-02-17T11:55:57.674246+0000","last_unstale":"2024-02-17T11:56:59.881143+0000","last_undegraded":"2024-02-17T11:56:59.881143+0000","last_fullsized":"2024-02-17T11:56:59.881143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:42:26.681061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671477+0000","last_change":"2024-02-17T11:56:59.734355+0000","last_active":"2024-02-17T11:57:28.671477+0000","last_peered":"2024-02-17T11:57:28.671477+0000","last_clean":"2024-02-17T11:57:28.671477+0000","last_became_active":"2024-02-17T11:56:59.733975+0000","last_became_peered":"2024-02-17T11:56:59.733975+0000","last_unstale":"2024-02-17T11:57:28.671477+0000","last_undegraded":"2024-02-17T11:57:28.671477+0000","last_fullsized":"2024-02-17T11:57:28.671477+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:37:51.426702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732149+0000","last_change":"2024-02-17T11:55:57.667998+0000","last_active":"2024-02-17T11:56:59.732149+0000","last_peered":"2024-02-17T11:56:59.732149+0000","last_clean":"2024-02-17T11:56:59.732149+0000","last_became_active":"2024-02-17T11:55:57.667688+0000","last_became_peered":"2024-02-17T11:55:57.667688+0000","last_unstale":"2024-02-17T11:56:59.732149+0000","last_undegraded":"2024-02-17T11:56:59.732149+0000","last_fullsized":"2024-02-17T11:56:59.732149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:07:19.782879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.659157+0000","last_change":"2024-02-17T11:55:58.676644+0000","last_active":"2024-02-17T11:57:28.659157+0000","last_peered":"2024-02-17T11:57:28.659157+0000","last_clean":"2024-02-17T11:57:28.659157+0000","last_became_active":"2024-02-17T11:55:58.674670+0000","last_became_peered":"2024-02-17T11:55:58.674670+0000","last_unstale":"2024-02-17T11:57:28.659157+0000","last_undegraded":"2024-02-17T11:57:28.659157+0000","last_fullsized":"2024-02-17T11:57:28.659157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:43:22.746537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.732754+0000","last_change":"2024-02-17T11:55:57.671330+0000","last_active":"2024-02-17T11:56:59.732754+0000","last_peered":"2024-02-17T11:56:59.732754+0000","last_clean":"2024-02-17T11:56:59.732754+0000","last_became_active":"2024-02-17T11:55:57.670902+0000","last_became_peered":"2024-02-17T11:55:57.670902+0000","last_unstale":"2024-02-17T11:56:59.732754+0000","last_undegraded":"2024-02-17T11:56:59.732754+0000","last_fullsized":"2024-02-17T11:56:59.732754+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:22:14.770339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672152+0000","last_change":"2024-02-17T11:55:58.676417+0000","last_active":"2024-02-17T11:57:28.672152+0000","last_peered":"2024-02-17T11:57:28.672152+0000","last_clean":"2024-02-17T11:57:28.672152+0000","last_became_active":"2024-02-17T11:55:58.675960+0000","last_became_peered":"2024-02-17T11:55:58.675960+0000","last_unstale":"2024-02-17T11:57:28.672152+0000","last_undegraded":"2024-02-17T11:57:28.672152+0000","last_fullsized":"2024-02-17T11:57:28.672152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:22:18.566872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.738907+0000","last_change":"2024-02-17T11:56:59.737823+0000","last_active":"2024-02-17T11:56:59.738907+0000","last_peered":"2024-02-17T11:56:59.738907+0000","last_clean":"2024-02-17T11:56:59.738907+0000","last_became_active":"2024-02-17T11:56:59.734043+0000","last_became_peered":"2024-02-17T11:56:59.734043+0000","last_unstale":"2024-02-17T11:56:59.738907+0000","last_undegraded":"2024-02-17T11:56:59.738907+0000","last_fullsized":"2024-02-17T11:56:59.738907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:11:09.171845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670581+0000","last_change":"2024-02-17T11:55:58.690030+0000","last_active":"2024-02-17T11:57:28.670581+0000","last_peered":"2024-02-17T11:57:28.670581+0000","last_clean":"2024-02-17T11:57:28.670581+0000","last_became_active":"2024-02-17T11:55:58.689687+0000","last_became_peered":"2024-02-17T11:55:58.689687+0000","last_unstale":"2024-02-17T11:57:28.670581+0000","last_undegraded":"2024-02-17T11:57:28.670581+0000","last_fullsized":"2024-02-17T11:57:28.670581+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:33:43.175234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.729187+0000","last_change":"2024-02-17T11:55:57.679734+0000","last_active":"2024-02-17T11:56:59.729187+0000","last_peered":"2024-02-17T11:56:59.729187+0000","last_clean":"2024-02-17T11:56:59.729187+0000","last_became_active":"2024-02-17T11:55:57.679328+0000","last_became_peered":"2024-02-17T11:55:57.679328+0000","last_unstale":"2024-02-17T11:56:59.729187+0000","last_undegraded":"2024-02-17T11:56:59.729187+0000","last_fullsized":"2024-02-17T11:56:59.729187+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:50:09.813482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.683947+0000","last_change":"2024-02-17T11:55:58.688360+0000","last_active":"2024-02-17T11:57:28.683947+0000","last_peered":"2024-02-17T11:57:28.683947+0000","last_clean":"2024-02-17T11:57:28.683947+0000","last_became_active":"2024-02-17T11:55:58.687994+0000","last_became_peered":"2024-02-17T11:55:58.687994+0000","last_unstale":"2024-02-17T11:57:28.683947+0000","last_undegraded":"2024-02-17T11:57:28.683947+0000","last_fullsized":"2024-02-17T11:57:28.683947+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T20:01:02.163238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733202+0000","last_change":"2024-02-17T11:55:57.675977+0000","last_active":"2024-02-17T11:56:59.733202+0000","last_peered":"2024-02-17T11:56:59.733202+0000","last_clean":"2024-02-17T11:56:59.733202+0000","last_became_active":"2024-02-17T11:55:57.675250+0000","last_became_peered":"2024-02-17T11:55:57.675250+0000","last_unstale":"2024-02-17T11:56:59.733202+0000","last_undegraded":"2024-02-17T11:56:59.733202+0000","last_fullsized":"2024-02-17T11:56:59.733202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:11:27.611692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685664+0000","last_change":"2024-02-17T11:55:58.688593+0000","last_active":"2024-02-17T11:57:28.685664+0000","last_peered":"2024-02-17T11:57:28.685664+0000","last_clean":"2024-02-17T11:57:28.685664+0000","last_became_active":"2024-02-17T11:55:58.688163+0000","last_became_peered":"2024-02-17T11:55:58.688163+0000","last_unstale":"2024-02-17T11:57:28.685664+0000","last_undegraded":"2024-02-17T11:57:28.685664+0000","last_fullsized":"2024-02-17T11:57:28.685664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:41:54.597116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.733115+0000","last_change":"2024-02-17T11:55:57.676860+0000","last_active":"2024-02-17T11:56:59.733115+0000","last_peered":"2024-02-17T11:56:59.733115+0000","last_clean":"2024-02-17T11:56:59.733115+0000","last_became_active":"2024-02-17T11:55:57.676534+0000","last_became_peered":"2024-02-17T11:55:57.676534+0000","last_unstale":"2024-02-17T11:56:59.733115+0000","last_undegraded":"2024-02-17T11:56:59.733115+0000","last_fullsized":"2024-02-17T11:56:59.733115+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:23:47.726264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.675151+0000","last_change":"2024-02-17T11:56:59.739615+0000","last_active":"2024-02-17T11:57:28.675151+0000","last_peered":"2024-02-17T11:57:28.675151+0000","last_clean":"2024-02-17T11:57:28.675151+0000","last_became_active":"2024-02-17T11:56:59.736220+0000","last_became_peered":"2024-02-17T11:56:59.736220+0000","last_unstale":"2024-02-17T11:57:28.675151+0000","last_undegraded":"2024-02-17T11:57:28.675151+0000","last_fullsized":"2024-02-17T11:57:28.675151+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:22:07.463713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:56:59.739390+0000","last_change":"2024-02-17T11:56:59.739390+0000","last_active":"2024-02-17T11:56:59.739390+0000","last_peered":"2024-02-17T11:56:59.739390+0000","last_clean":"2024-02-17T11:56:59.739390+0000","last_became_active":"2024-02-17T11:56:59.734852+0000","last_became_peered":"2024-02-17T11:56:59.734852+0000","last_unstale":"2024-02-17T11:56:59.739390+0000","last_undegraded":"2024-02-17T11:56:59.739390+0000","last_fullsized":"2024-02-17T11:56:59.739390+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:56.641409+0000","last_clean_scrub_stamp":"2024-02-17T11:55:56.641409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:40:09.174557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.675878+0000","last_change":"2024-02-17T11:55:58.672985+0000","last_active":"2024-02-17T11:57:28.675878+0000","last_peered":"2024-02-17T11:57:28.675878+0000","last_clean":"2024-02-17T11:57:28.675878+0000","last_became_active":"2024-02-17T11:55:58.672622+0000","last_became_peered":"2024-02-17T11:55:58.672622+0000","last_unstale":"2024-02-17T11:57:28.675878+0000","last_undegraded":"2024-02-17T11:57:28.675878+0000","last_fullsized":"2024-02-17T11:57:28.675878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:13:45.555827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672536+0000","last_change":"2024-02-17T11:55:58.676127+0000","last_active":"2024-02-17T11:57:28.672536+0000","last_peered":"2024-02-17T11:57:28.672536+0000","last_clean":"2024-02-17T11:57:28.672536+0000","last_became_active":"2024-02-17T11:55:58.675784+0000","last_became_peered":"2024-02-17T11:55:58.675784+0000","last_unstale":"2024-02-17T11:57:28.672536+0000","last_undegraded":"2024-02-17T11:57:28.672536+0000","last_fullsized":"2024-02-17T11:57:28.672536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T16:56:32.648803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.663712+0000","last_change":"2024-02-17T11:55:58.682438+0000","last_active":"2024-02-17T11:57:28.663712+0000","last_peered":"2024-02-17T11:57:28.663712+0000","last_clean":"2024-02-17T11:57:28.663712+0000","last_became_active":"2024-02-17T11:55:58.682067+0000","last_became_peered":"2024-02-17T11:55:58.682067+0000","last_unstale":"2024-02-17T11:57:28.663712+0000","last_undegraded":"2024-02-17T11:57:28.663712+0000","last_fullsized":"2024-02-17T11:57:28.663712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T23:10:22.906450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671367+0000","last_change":"2024-02-17T11:55:58.686671+0000","last_active":"2024-02-17T11:57:28.671367+0000","last_peered":"2024-02-17T11:57:28.671367+0000","last_clean":"2024-02-17T11:57:28.671367+0000","last_became_active":"2024-02-17T11:55:58.686315+0000","last_became_peered":"2024-02-17T11:55:58.686315+0000","last_unstale":"2024-02-17T11:57:28.671367+0000","last_undegraded":"2024-02-17T11:57:28.671367+0000","last_fullsized":"2024-02-17T11:57:28.671367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:55:40.864847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671037+0000","last_change":"2024-02-17T11:55:58.684227+0000","last_active":"2024-02-17T11:57:28.671037+0000","last_peered":"2024-02-17T11:57:28.671037+0000","last_clean":"2024-02-17T11:57:28.671037+0000","last_became_active":"2024-02-17T11:55:58.683875+0000","last_became_peered":"2024-02-17T11:55:58.683875+0000","last_unstale":"2024-02-17T11:57:28.671037+0000","last_undegraded":"2024-02-17T11:57:28.671037+0000","last_fullsized":"2024-02-17T11:57:28.671037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T12:11:41.283161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.677463+0000","last_change":"2024-02-17T11:55:58.685964+0000","last_active":"2024-02-17T11:57:28.677463+0000","last_peered":"2024-02-17T11:57:28.677463+0000","last_clean":"2024-02-17T11:57:28.677463+0000","last_became_active":"2024-02-17T11:55:58.685350+0000","last_became_peered":"2024-02-17T11:55:58.685350+0000","last_unstale":"2024-02-17T11:57:28.677463+0000","last_undegraded":"2024-02-17T11:57:28.677463+0000","last_fullsized":"2024-02-17T11:57:28.677463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:22:34.600604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.671590+0000","last_change":"2024-02-17T11:55:58.671291+0000","last_active":"2024-02-17T11:57:28.671590+0000","last_peered":"2024-02-17T11:57:28.671590+0000","last_clean":"2024-02-17T11:57:28.671590+0000","last_became_active":"2024-02-17T11:55:58.670925+0000","last_became_peered":"2024-02-17T11:55:58.670925+0000","last_unstale":"2024-02-17T11:57:28.671590+0000","last_undegraded":"2024-02-17T11:57:28.671590+0000","last_fullsized":"2024-02-17T11:57:28.671590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:26:03.120976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.667565+0000","last_change":"2024-02-17T11:55:58.673122+0000","last_active":"2024-02-17T11:57:28.667565+0000","last_peered":"2024-02-17T11:57:28.667565+0000","last_clean":"2024-02-17T11:57:28.667565+0000","last_became_active":"2024-02-17T11:55:58.672747+0000","last_became_peered":"2024-02-17T11:55:58.672747+0000","last_unstale":"2024-02-17T11:57:28.667565+0000","last_undegraded":"2024-02-17T11:57:28.667565+0000","last_fullsized":"2024-02-17T11:57:28.667565+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T14:55:15.064073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.677328+0000","last_change":"2024-02-17T11:55:58.681276+0000","last_active":"2024-02-17T11:57:28.677328+0000","last_peered":"2024-02-17T11:57:28.677328+0000","last_clean":"2024-02-17T11:57:28.677328+0000","last_became_active":"2024-02-17T11:55:58.680889+0000","last_became_peered":"2024-02-17T11:55:58.680889+0000","last_unstale":"2024-02-17T11:57:28.677328+0000","last_undegraded":"2024-02-17T11:57:28.677328+0000","last_fullsized":"2024-02-17T11:57:28.677328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:19:34.330288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.670939+0000","last_change":"2024-02-17T11:55:58.667475+0000","last_active":"2024-02-17T11:57:28.670939+0000","last_peered":"2024-02-17T11:57:28.670939+0000","last_clean":"2024-02-17T11:57:28.670939+0000","last_became_active":"2024-02-17T11:55:58.666577+0000","last_became_peered":"2024-02-17T11:55:58.666577+0000","last_unstale":"2024-02-17T11:57:28.670939+0000","last_undegraded":"2024-02-17T11:57:28.670939+0000","last_fullsized":"2024-02-17T11:57:28.670939+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T17:51:35.732035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.672928+0000","last_change":"2024-02-17T11:55:58.674557+0000","last_active":"2024-02-17T11:57:28.672928+0000","last_peered":"2024-02-17T11:57:28.672928+0000","last_clean":"2024-02-17T11:57:28.672928+0000","last_became_active":"2024-02-17T11:55:58.673874+0000","last_became_peered":"2024-02-17T11:55:58.673874+0000","last_unstale":"2024-02-17T11:57:28.672928+0000","last_undegraded":"2024-02-17T11:57:28.672928+0000","last_fullsized":"2024-02-17T11:57:28.672928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T15:18:00.812489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.685895+0000","last_change":"2024-02-17T11:55:58.692725+0000","last_active":"2024-02-17T11:57:28.685895+0000","last_peered":"2024-02-17T11:57:28.685895+0000","last_clean":"2024-02-17T11:57:28.685895+0000","last_became_active":"2024-02-17T11:55:58.690880+0000","last_became_peered":"2024-02-17T11:55:58.690880+0000","last_unstale":"2024-02-17T11:57:28.685895+0000","last_undegraded":"2024-02-17T11:57:28.685895+0000","last_fullsized":"2024-02-17T11:57:28.685895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T21:03:44.724385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.666977+0000","last_change":"2024-02-17T11:55:58.679377+0000","last_active":"2024-02-17T11:57:28.666977+0000","last_peered":"2024-02-17T11:57:28.666977+0000","last_clean":"2024-02-17T11:57:28.666977+0000","last_became_active":"2024-02-17T11:55:58.679041+0000","last_became_peered":"2024-02-17T11:55:58.679041+0000","last_unstale":"2024-02-17T11:57:28.666977+0000","last_undegraded":"2024-02-17T11:57:28.666977+0000","last_fullsized":"2024-02-17T11:57:28.666977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T22:16:37.396424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-17T11:57:28.678575+0000","last_change":"2024-02-17T11:55:58.680665+0000","last_active":"2024-02-17T11:57:28.678575+0000","last_peered":"2024-02-17T11:57:28.678575+0000","last_clean":"2024-02-17T11:57:28.678575+0000","last_became_active":"2024-02-17T11:55:58.680311+0000","last_became_peered":"2024-02-17T11:55:58.680311+0000","last_unstale":"2024-02-17T11:57:28.678575+0000","last_undegraded":"2024-02-17T11:57:28.678575+0000","last_fullsized":"2024-02-17T11:57:28.678575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-17T11:55:57.646879+0000","last_clean_scrub_stamp":"2024-02-17T11:55:57.646879+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-18T19:19:11.455471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284587,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72432,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":708,"num_read_kb":432,"num_write":177,"num_write_kb":130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":218078,"data_compressed":5598,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383717,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":456292,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.372},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.55100000000000005}]},{"osd":1,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58099999999999996,"15min":0.57799999999999996},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.59799999999999998,"15min":0.58899999999999997},"min":{"1min":0.443,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63800000000000001}]},{"osd":2,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.49199999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.52000000000000002,"15min":0.52800000000000002},"min":{"1min":0.317,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.68899999999999995,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.73699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61299999999999999,"15min":0.60799999999999998},"min":{"1min":0.45100000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.66800000000000004,"15min":0.66400000000000003},"min":{"1min":0.504,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52800000000000002}]},{"osd":5,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.60399999999999998,"15min":0.61299999999999999},"min":{"1min":0.372,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78000000000000003,"5min":1.484,"15min":1.484},"last":0.755},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59399999999999997,"15min":0.60199999999999998},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84299999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.40100000000000002}]},{"osd":6,"last update":"Sat Feb 17 11:56:08 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.55,"5min":1.55,"15min":1.55},"last":0.46100000000000002}]},{"osd":7,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82199999999999995,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.62},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75900000000000001,"15min":0.76500000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.92300000000000004,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.58799999999999997}]},{"osd":8,"last update":"Sat Feb 17 11:56:59 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.70399999999999996}]},{"osd":9,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.78500000000000003,"15min":0.79400000000000004},"min":{"1min":0.54000000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.89500000000000002,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.72899999999999998,"15min":0.73499999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.90700000000000003,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.66100000000000003}]},{"osd":10,"last update":"Sat Feb 17 11:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.61199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.95099999999999996,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.82099999999999995,"15min":0.82499999999999996},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0529999999999999,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.83899999999999997}]}]},{"osd":10,"up_from":72,"seq":309237645354,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":1115281,"data_compressed":43889,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.89300000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46300000000000002,"15min":0.47499999999999998},"min":{"1min":0.32400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.36399999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73899999999999999,"5min":0.746,"15min":0.746},"last":0.435}]},{"osd":2,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.61699999999999999,"15min":0.61899999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81899999999999995,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.61899999999999999,"15min":0.60299999999999998},"min":{"1min":0.53500000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.79500000000000004,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.59999999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58199999999999996,"15min":0.59199999999999997},"min":{"1min":0.41699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71899999999999997,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.59199999999999997,"15min":0.60999999999999999},"min":{"1min":0.48299999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77500000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69299999999999995}]},{"osd":4,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55700000000000005,"15min":0.57499999999999996},"min":{"1min":0.373,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.82899999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.52400000000000002,"15min":0.53600000000000003},"min":{"1min":0.31,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.84299999999999997,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54000000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67000000000000004,"15min":0.67200000000000004},"min":{"1min":0.47399999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.47799999999999998,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.68000000000000005}]},{"osd":6,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.66500000000000004,"15min":0.68100000000000005},"min":{"1min":0.51800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.93400000000000005,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.748},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.80200000000000005,"5min":1.101,"15min":1.101},"last":0.86199999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:55:45 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.41499999999999998}]},{"osd":8,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71999999999999997,"15min":0.72799999999999998},"min":{"1min":0.502,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88700000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73999999999999999,"15min":0.75800000000000001},"min":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94699999999999995,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.71199999999999997}]},{"osd":9,"last update":"Sat Feb 17 11:57:46 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73099999999999998,"15min":0.752},"min":{"1min":0.59799999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.91800000000000004,"5min":1.014,"15min":1.014},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.73899999999999999,"15min":0.76300000000000001},"min":{"1min":0.55600000000000005,"5min":0.434,"15min":0.434},"max":{"1min":0.997,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.92100000000000004}]},{"osd":11,"last update":"Sat Feb 17 11:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.74399999999999999,"15min":0.74199999999999999},"min":{"1min":0.56299999999999994,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.84699999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.877},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.60999999999999999,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.77900000000000003}]}]},{"osd":9,"up_from":67,"seq":287762808878,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":456645,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.42299999999999999},"min":{"1min":0.29999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.40500000000000003,"15min":0.41699999999999998},"min":{"1min":0.27300000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44500000000000001}]},{"osd":1,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60099999999999998,"15min":0.61899999999999999},"min":{"1min":0.45600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.499},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57399999999999995,"15min":0.58299999999999996},"min":{"1min":0.443,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.78600000000000003,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.43099999999999999}]},{"osd":2,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.443,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90700000000000003,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.40300000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.54000000000000004}]},{"osd":3,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.496,"15min":0.52000000000000002},"min":{"1min":0.32100000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.47399999999999998,"15min":0.5},"min":{"1min":0.32600000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.71899999999999997,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.317}]},{"osd":4,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.41199999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.85299999999999998,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.65000000000000002,"15min":0.67700000000000005},"min":{"1min":0.42499999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.87,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.47499999999999998}]},{"osd":5,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60799999999999998,"15min":0.63400000000000001},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.84799999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68500000000000005,"15min":0.71499999999999997},"min":{"1min":0.503,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.98099999999999998,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.52900000000000003}]},{"osd":6,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57099999999999995,"15min":0.60499999999999998},"min":{"1min":0.39100000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.81299999999999994,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60199999999999998,"15min":0.626},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.91800000000000004,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.55300000000000005}]},{"osd":7,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72599999999999998,"15min":0.752},"min":{"1min":0.54000000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.95299999999999996,"5min":1.05,"15min":1.05},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69199999999999995,"15min":0.71399999999999997},"min":{"1min":0.47699999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93700000000000006,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.59099999999999997}]},{"osd":8,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69799999999999995,"15min":0.72599999999999998},"min":{"1min":0.55200000000000005,"5min":0.442,"15min":0.442},"max":{"1min":0.99299999999999999,"5min":1.016,"15min":1.016},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.70399999999999996,"15min":0.72299999999999998},"min":{"1min":0.51200000000000001,"5min":0.504,"15min":0.504},"max":{"1min":0.96599999999999997,"5min":1.034,"15min":1.034},"last":0.61799999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.70899999999999996,"15min":0.71799999999999997},"min":{"1min":0.54600000000000004,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.78800000000000003,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.57399999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.80500000000000005,"5min":1.069,"15min":1.069},"last":0.66800000000000004}]}]},{"osd":8,"up_from":59,"seq":253403070516,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":954368,"data_stored":455866,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.317,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.72699999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52600000000000002,"15min":0.52500000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71199999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.53300000000000003}]},{"osd":1,"last update":"Sat Feb 17 11:56:53 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.58399999999999996,"15min":0.56599999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.59499999999999997,"15min":0.58299999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88900000000000001,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.66400000000000003}]},{"osd":2,"last update":"Sat Feb 17 11:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.47399999999999998,"15min":0.44400000000000001},"min":{"1min":0.29799999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68700000000000006,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.49099999999999999,"15min":0.48499999999999999},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.80600000000000005}]},{"osd":3,"last update":"Sat Feb 17 11:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.58799999999999997},"min":{"1min":0.38400000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81999999999999995,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58999999999999997}]},{"osd":4,"last update":"Sat Feb 17 11:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.628,"15min":0.61199999999999999},"min":{"1min":0.52600000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.85799999999999998,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.64200000000000002,"15min":0.622},"min":{"1min":0.441,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.872}]},{"osd":5,"last update":"Sat Feb 17 11:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58199999999999996,"15min":0.57099999999999995},"min":{"1min":0.41399999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.56899999999999995,"15min":0.55000000000000004},"min":{"1min":0.42099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.88900000000000001}]},{"osd":6,"last update":"Sat Feb 17 11:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.63800000000000001,"15min":0.63600000000000001},"min":{"1min":0.49399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86799999999999999,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68200000000000005,"15min":0.68700000000000006},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.85299999999999998,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.86199999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.72099999999999997,"15min":0.70299999999999996},"min":{"1min":0.52400000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71199999999999997,"15min":0.68400000000000005},"min":{"1min":0.56599999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.98399999999999999}]},{"osd":9,"last update":"Sat Feb 17 11:57:18 2024","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.69099999999999995,"15min":0.67700000000000005},"min":{"1min":0.51100000000000001,"5min":0.505,"15min":0.505},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.68700000000000006,"15min":0.66600000000000004},"min":{"1min":0.54300000000000004,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.999}]},{"osd":10,"last update":"Sat Feb 17 11:57:49 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.76400000000000001,"15min":0.74299999999999999},"min":{"1min":0.54900000000000004,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.75900000000000001,"15min":0.74299999999999999},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.96699999999999997}]},{"osd":11,"last update":"Sat Feb 17 11:57:00 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":1.01},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":1.0249999999999999}]}]},{"osd":7,"up_from":51,"seq":219043332153,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":1049833,"data_compressed":42023,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.36699999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.81399999999999995,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.53100000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.39600000000000002}]},{"osd":1,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.41899999999999998},"min":{"1min":0.30599999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.65300000000000002,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.45100000000000001,"15min":0.433},"min":{"1min":0.33700000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.68000000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.52700000000000002}]},{"osd":2,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.61099999999999999,"15min":0.60999999999999999},"min":{"1min":0.38600000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.89100000000000001,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77400000000000002,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.504}]},{"osd":3,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57199999999999995,"15min":0.55800000000000005},"min":{"1min":0.45600000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.80800000000000005,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.38900000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.71899999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.59299999999999997}]},{"osd":4,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.48599999999999999,"15min":0.47399999999999998},"min":{"1min":0.31,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.52600000000000002,"15min":0.51700000000000002},"min":{"1min":0.34899999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85499999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67500000000000004,"15min":0.66100000000000003},"min":{"1min":0.44400000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.93200000000000005,"5min":1.333,"15min":1.333},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63800000000000001,"15min":0.64600000000000002},"min":{"1min":0.47599999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83199999999999996,"5min":1.486,"15min":1.486},"last":0.54000000000000004}]},{"osd":6,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.61599999999999999,"15min":0.59799999999999998},"min":{"1min":0.44,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.68200000000000005,"15min":0.68000000000000005},"min":{"1min":0.47699999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.91000000000000003,"5min":1.087,"15min":1.087},"last":0.55900000000000005}]},{"osd":8,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.55900000000000005,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96999999999999997,"5min":1.446,"15min":1.446},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.92400000000000004,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.628}]},{"osd":9,"last update":"Sat Feb 17 11:57:26 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47299999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.746,"15min":0.755},"min":{"1min":0.49299999999999999,"5min":0.437,"15min":0.437},"max":{"1min":1.034,"5min":1.052,"15min":1.052},"last":0.65200000000000002}]},{"osd":10,"last update":"Sat Feb 17 11:57:55 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.70399999999999996,"15min":0.69999999999999996},"min":{"1min":0.52500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.76200000000000001,"15min":0.76300000000000001},"min":{"1min":0.58399999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.002,"5min":1.02,"15min":1.02},"last":0.69599999999999995}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.64100000000000001}]}]},{"osd":6,"up_from":46,"seq":197568495679,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":455917,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.35099999999999998},"min":{"1min":0.309,"5min":0.223,"15min":0.223},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38500000000000001,"15min":0.38300000000000001},"min":{"1min":0.28799999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.53300000000000003,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.34599999999999997}]},{"osd":1,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70099999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.62},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.52900000000000003},"min":{"1min":0.32500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.45500000000000002}]},{"osd":2,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56100000000000005,"15min":0.55500000000000005},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.71199999999999997,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.437},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.41699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.745,"5min":0.871,"15min":0.871},"last":0.48199999999999998}]},{"osd":3,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.45600000000000002,"15min":0.45000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.64400000000000002,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42099999999999999,"15min":0.439},"min":{"1min":0.27600000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.628,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.36899999999999999}]},{"osd":4,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.39300000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.39500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.79300000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.54200000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.59299999999999997,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.83399999999999996,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.64700000000000002,"15min":0.65400000000000003},"min":{"1min":0.503,"5min":0.432,"15min":0.432},"max":{"1min":0.86899999999999999,"5min":0.997,"15min":0.997},"last":0.496}]},{"osd":7,"last update":"Sat Feb 17 11:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57899999999999996,"15min":0.57799999999999996},"min":{"1min":0.42199999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78000000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60799999999999998,"15min":0.60199999999999998},"min":{"1min":0.45400000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.71099999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.60899999999999999}]},{"osd":8,"last update":"Sat Feb 17 11:57:00 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.67200000000000004,"15min":0.68100000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.71799999999999997,"15min":0.70099999999999996},"min":{"1min":0.46400000000000002,"5min":0.434,"15min":0.434},"max":{"1min":5.4269999999999996,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.629}]},{"osd":9,"last update":"Sat Feb 17 11:57:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.625,"15min":0.65700000000000003},"min":{"1min":0.377,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79300000000000004,"5min":0.879,"15min":0.879},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.65600000000000003,"15min":0.68300000000000005},"min":{"1min":0.44700000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82699999999999996,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.65300000000000002}]},{"osd":10,"last update":"Sat Feb 17 11:57:49 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.628,"15min":0.63200000000000001},"min":{"1min":0.47199999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.73599999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.70199999999999996,"15min":0.72099999999999997},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.753,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":522686,"data_compressed":7701,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.54500000000000004,"15min":0.50900000000000001},"min":{"1min":0.371,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77800000000000002,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.496},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.52800000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.71199999999999997,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.57899999999999996}]},{"osd":2,"last update":"Sat Feb 17 11:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57799999999999996,"15min":0.53200000000000003},"min":{"1min":0.41099999999999998,"5min":0.313,"15min":0.309},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.57899999999999996,"15min":0.53300000000000003},"min":{"1min":0.52800000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81899999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.74399999999999999}]},{"osd":3,"last update":"Sat Feb 17 11:56:51 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.58399999999999996,"15min":0.53100000000000003},"min":{"1min":0.37,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55400000000000005,"15min":0.52400000000000002},"min":{"1min":0.40600000000000003,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54400000000000004}]},{"osd":4,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.46500000000000002,"15min":0.433},"min":{"1min":0.376,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69199999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.46899999999999997,"15min":0.42099999999999999},"min":{"1min":0.35699999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.56100000000000005}]},{"osd":5,"last update":"Sat Feb 17 11:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.66100000000000003,"15min":0.59099999999999997},"min":{"1min":0.58099999999999996,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.67900000000000005,"15min":0.60899999999999999},"min":{"1min":0.64300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.871,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.79200000000000004}]},{"osd":6,"last update":"Sat Feb 17 11:57:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.63300000000000001,"15min":0.60099999999999998},"min":{"1min":0.52000000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.88300000000000001,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.62,"15min":0.59699999999999998},"min":{"1min":0.51000000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.78600000000000003,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.45100000000000001}]},{"osd":7,"last update":"Sat Feb 17 11:57:30 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.59799999999999998,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91100000000000003,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.65100000000000002,"15min":0.60099999999999998},"min":{"1min":0.42999999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93799999999999994,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.64400000000000002}]},{"osd":8,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.71799999999999997,"15min":0.68799999999999994},"min":{"1min":0.622,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98599999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66500000000000004,"15min":0.65400000000000003},"min":{"1min":0.56899999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.83799999999999997,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63200000000000001}]},{"osd":9,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.59699999999999998,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.89900000000000002,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78900000000000003,"15min":0.77700000000000002},"min":{"1min":0.66300000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.92200000000000004,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72899999999999998}]},{"osd":10,"last update":"Sat Feb 17 11:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72099999999999997,"15min":0.71699999999999997},"min":{"1min":0.48899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92500000000000004,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.79000000000000004,"15min":0.77600000000000002},"min":{"1min":0.66800000000000004,"5min":0.501,"15min":0.501},"max":{"1min":1.002,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.82899999999999996}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.86899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":966656,"data_stored":457188,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.53700000000000003,"15min":0.46800000000000003},"min":{"1min":0.36299999999999999,"5min":0.31,"15min":0.31},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52600000000000002,"15min":0.47999999999999998},"min":{"1min":0.40400000000000003,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.73199999999999998,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57899999999999996}]},{"osd":2,"last update":"Sat Feb 17 11:57:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.60299999999999998,"15min":0.52300000000000002},"min":{"1min":0.442,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57999999999999996,"15min":0.52700000000000002},"min":{"1min":0.372,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.61499999999999999}]},{"osd":3,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.55300000000000005}]},{"osd":4,"last update":"Sat Feb 17 11:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.54900000000000004,"15min":0.51100000000000001},"min":{"1min":0.39200000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67700000000000005,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56699999999999995,"15min":0.52300000000000002},"min":{"1min":0.35099999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.75800000000000001,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63}]},{"osd":5,"last update":"Sat Feb 17 11:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.62,"15min":0.59499999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.59799999999999998},"min":{"1min":0.40899999999999997,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.518,"5min":1.518,"15min":1.518},"last":0.46899999999999997}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.498,"15min":0.502},"min":{"1min":0.307,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64200000000000002,"5min":0.877,"15min":0.877},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.318,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68799999999999994,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.44500000000000001}]},{"osd":7,"last update":"Sat Feb 17 11:55:31 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.501,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.66200000000000003,"15min":0.66400000000000003},"min":{"1min":0.436,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.58399999999999996}]},{"osd":8,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.69499999999999995,"15min":0.70599999999999996},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.79100000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.46600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.53500000000000003}]},{"osd":9,"last update":"Sat Feb 17 11:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.61499999999999999,"15min":0.64500000000000002},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86099999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.498},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.66000000000000003,"15min":0.68200000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76700000000000002,"5min":1.335,"15min":1.335},"last":0.48499999999999999}]},{"osd":10,"last update":"Sat Feb 17 11:57:52 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.68500000000000005,"15min":0.70499999999999996},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.79400000000000004,"5min":1.107,"15min":1.107},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.745},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.89900000000000002,"5min":1.236,"15min":1.236},"last":0.70399999999999996}]},{"osd":11,"last update":"Sat Feb 17 11:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70199999999999996,"15min":0.70499999999999996},"min":{"1min":0.55600000000000005,"5min":0.495,"15min":0.495},"max":{"1min":0.84399999999999997,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.67000000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":525350,"data_compressed":7701,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.52900000000000003,"15min":0.498},"min":{"1min":0.32200000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.76700000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.54600000000000004,"15min":0.50800000000000001},"min":{"1min":0.377,"5min":0.33700000000000002,"15min":0.29199999999999998},"max":{"1min":0.77300000000000002,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.498}]},{"osd":1,"last update":"Sat Feb 17 11:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.55300000000000005,"15min":0.495},"min":{"1min":0.34100000000000003,"5min":0.318,"15min":0.318},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.746},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.58999999999999997,"15min":0.501},"min":{"1min":0.45100000000000001,"5min":0.33100000000000002,"15min":0.32500000000000001},"max":{"1min":0.89400000000000002,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.65300000000000002}]},{"osd":3,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.54500000000000004},"min":{"1min":0.439,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.753,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.4750000000000001,"5min":0.71899999999999997,"15min":0.57699999999999996},"min":{"1min":0.378,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":16.718,"5min":16.718,"15min":16.718},"last":0.53600000000000003}]},{"osd":4,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.76000000000000001}]},{"osd":5,"last update":"Sat Feb 17 11:57:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.48599999999999999,"15min":0.438},"min":{"1min":0.378,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.432},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.47299999999999998,"15min":0.44700000000000001},"min":{"1min":0.35399999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.372}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.60799999999999998,"15min":0.58399999999999996},"min":{"1min":0.40799999999999997,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.72799999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.46300000000000002}]},{"osd":7,"last update":"Sat Feb 17 11:55:29 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64000000000000001,"15min":0.64100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67500000000000004,"15min":0.67700000000000005},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.93999999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.63700000000000001}]},{"osd":8,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.60499999999999998,"15min":0.60299999999999998},"min":{"1min":0.41199999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.60999999999999999,"15min":0.60799999999999998},"min":{"1min":0.45700000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.51600000000000001}]},{"osd":9,"last update":"Sat Feb 17 11:57:21 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.90500000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.66400000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.80600000000000005,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57599999999999996}]},{"osd":10,"last update":"Sat Feb 17 11:57:47 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.63900000000000001,"15min":0.61399999999999999},"min":{"1min":0.438,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.73099999999999998}]},{"osd":11,"last update":"Sat Feb 17 11:56:58 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.69599999999999995}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":958464,"data_stored":456111,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.35799999999999998,"15min":0.34399999999999997},"min":{"1min":0.26100000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.745,"5min":0.746,"15min":0.746},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.35899999999999999,"15min":0.34499999999999997},"min":{"1min":0.26400000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.53000000000000003,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.36899999999999999}]},{"osd":1,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.41699999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.54600000000000004,"15min":0.53600000000000003},"min":{"1min":0.36799999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.78300000000000003,"5min":0.87,"15min":0.87},"last":0.53000000000000003}]},{"osd":2,"last update":"Sat Feb 17 11:56:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55800000000000005,"15min":0.57099999999999995},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77100000000000002,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56399999999999995},"min":{"1min":0.39400000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78700000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51400000000000001}]},{"osd":4,"last update":"Sat Feb 17 11:57:19 2024","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":0.71399999999999997,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":12.353,"5min":12.353,"15min":12.353},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.60499999999999998,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60199999999999998}]},{"osd":5,"last update":"Sat Feb 17 11:57:48 2024","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":0.68999999999999995,"15min":0.60999999999999999},"min":{"1min":0.46700000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":12.337999999999999,"5min":12.337999999999999,"15min":12.337999999999999},"last":0.443},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.749,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59099999999999997}]},{"osd":6,"last update":"Sat Feb 17 11:57:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.44400000000000001},"min":{"1min":0.307,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.82999999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.52000000000000002,"15min":0.48599999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.95799999999999996,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.47599999999999998}]},{"osd":7,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.61799999999999999},{"interface":"front","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":12.301,"5min":12.301,"15min":12.301},"last":0.56699999999999995}]},{"osd":8,"last update":"Sat Feb 17 11:55:54 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.49199999999999999,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.68100000000000005,"15min":0.67800000000000005},"min":{"1min":0.46400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.97599999999999998,"5min":1.214,"15min":1.214},"last":0.622}]},{"osd":9,"last update":"Sat Feb 17 11:57:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.57399999999999995,"15min":0.60699999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.64100000000000001,"5min":0.875,"15min":0.875},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.66700000000000004,"15min":0.70599999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.82299999999999995,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.627}]},{"osd":10,"last update":"Sat Feb 17 11:57:48 2024","interfaces":[{"interface":"back","average":{"1min":1.2450000000000001,"5min":0.83799999999999997,"15min":0.78600000000000003},"min":{"1min":0.51400000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":12.319000000000001,"5min":12.319000000000001,"15min":12.319000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.64700000000000002}]},{"osd":11,"last update":"Sat Feb 17 11:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66600000000000004}]}]},{"osd":4,"up_from":33,"seq":141733920840,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":954368,"data_stored":457468,"data_compressed":5835,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:55:17 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.32800000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.50700000000000001,"15min":0.497},"min":{"1min":0.41099999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.41599999999999998}]},{"osd":1,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.39900000000000002,"15min":0.38300000000000001},"min":{"1min":0.308,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.60799999999999998,"5min":0.872,"15min":0.872},"last":0.497},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.41799999999999998,"15min":0.39600000000000002},"min":{"1min":0.30199999999999999,"5min":0.254,"15min":0.254},"max":{"1min":0.69099999999999995,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.47399999999999998}]},{"osd":2,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.56799999999999995,"15min":0.55000000000000004},"min":{"1min":0.36599999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.65300000000000002,"15min":0.56299999999999994},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.755,"5min":7.0279999999999996,"15min":7.0279999999999996},"last":0.69799999999999995}]},{"osd":3,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.56299999999999994,"15min":0.55300000000000005},"min":{"1min":0.39800000000000002,"5min":0.371,"15min":0.371},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80500000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54800000000000004}]},{"osd":5,"last update":"Sat Feb 17 11:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.60199999999999998,"15min":0.56200000000000006},"min":{"1min":0.45800000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.622,"15min":0.56999999999999995},"min":{"1min":0.499,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82799999999999996,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.747}]},{"osd":6,"last update":"Sat Feb 17 11:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59099999999999997,"15min":0.57399999999999995},"min":{"1min":0.41999999999999998,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77800000000000002,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61199999999999999,"15min":0.59399999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83999999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.66000000000000003}]},{"osd":7,"last update":"Sat Feb 17 11:57:32 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.56999999999999995,"15min":0.52300000000000002},"min":{"1min":0.42899999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.872,"5min":0.997,"15min":0.997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.54900000000000004},"min":{"1min":0.374,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.67900000000000005}]},{"osd":8,"last update":"Sat Feb 17 11:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":0.67700000000000005},"min":{"1min":0.51000000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90400000000000003,"5min":6.9660000000000002,"15min":6.9660000000000002},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72299999999999998,"15min":0.70899999999999996},"min":{"1min":0.52800000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.85199999999999998}]},{"osd":9,"last update":"Sat Feb 17 11:57:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.50800000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.94699999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.74299999999999999,"15min":0.751},"min":{"1min":0.53900000000000003,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.90500000000000003,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.78300000000000003}]},{"osd":10,"last update":"Sat Feb 17 11:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.76800000000000002,"15min":0.78200000000000003},"min":{"1min":0.58999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0369999999999999,"5min":1.087,"15min":1.087},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73799999999999999,"15min":0.74199999999999999},"min":{"1min":0.61299999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.87}]},{"osd":11,"last update":"Sat Feb 17 11:56:57 2024","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.83499999999999996}]}]},{"osd":5,"up_from":38,"seq":163208757316,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1019904,"data_stored":1048594,"data_compressed":42023,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.495},"min":{"1min":0.42599999999999999,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.53300000000000003,"15min":0.49099999999999999},"min":{"1min":0.38500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.69399999999999995,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.36899999999999999}]},{"osd":1,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57499999999999996,"15min":0.51900000000000002},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.89700000000000002,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57999999999999996,"15min":0.51600000000000001},"min":{"1min":0.443,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.73799999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66300000000000003}]},{"osd":2,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.42899999999999999,"15min":0.41699999999999998},"min":{"1min":0.317,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.44500000000000001,"15min":0.42099999999999999},"min":{"1min":0.311,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68100000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.39100000000000001}]},{"osd":3,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.53000000000000003},"min":{"1min":0.38100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56299999999999994,"15min":0.53300000000000003},"min":{"1min":0.40999999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.753,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.51400000000000001}]},{"osd":4,"last update":"Sat Feb 17 11:57:44 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61699999999999999,"15min":0.56899999999999995},"min":{"1min":0.39900000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.79900000000000004,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.56999999999999995},"min":{"1min":0.40400000000000003,"5min":0.378,"15min":0.378},"max":{"1min":0.86499999999999999,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59099999999999997}]},{"osd":6,"last update":"Sat Feb 17 11:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60499999999999998,"15min":0.59099999999999997},"min":{"1min":0.45200000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.82199999999999995,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66100000000000003,"15min":0.64300000000000002},"min":{"1min":0.54700000000000004,"5min":0.439,"15min":0.439},"max":{"1min":0.94899999999999995,"5min":1.087,"15min":1.087},"last":0.61799999999999999}]},{"osd":7,"last update":"Sat Feb 17 11:57:33 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.61899999999999999},"min":{"1min":0.42799999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84399999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67300000000000004,"15min":0.65900000000000003},"min":{"1min":0.52400000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.879,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.68000000000000005}]},{"osd":8,"last update":"Sat Feb 17 11:57:54 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63700000000000001,"15min":0.62},"min":{"1min":0.42799999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.85999999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.46500000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95199999999999996,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.64200000000000002}]},{"osd":9,"last update":"Sat Feb 17 11:57:21 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71899999999999997,"15min":0.70899999999999996},"min":{"1min":0.52100000000000002,"5min":0.499,"15min":0.499},"max":{"1min":0.91400000000000003,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71099999999999997,"15min":0.69699999999999995},"min":{"1min":0.56100000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93500000000000005,"5min":1.056,"15min":1.056},"last":0.60399999999999998}]},{"osd":10,"last update":"Sat Feb 17 11:57:49 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.75600000000000001,"15min":0.745},"min":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.96399999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76700000000000002,"15min":0.75900000000000001},"min":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.72799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34549,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36188,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":67367,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69640,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":69868,"data_compressed":1866,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-17T11:57:56.940 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:57:57.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.076+0000 7fa359a15640 1 -- 172.21.15.94:0/2783872239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa354101290 msgr2=0x7fa354101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.076+0000 7fa359a15640 1 --2- 172.21.15.94:0/2783872239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa354101290 0x7fa354101710 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa348009900 tx=0x7fa34802bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.076+0000 7fa359a15640 1 -- 172.21.15.94:0/2783872239 shutdown_connections 2024-02-17T11:57:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.076+0000 7fa359a15640 1 --2- 172.21.15.94:0/2783872239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa354101c50 0x7fa3541084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.076+0000 7fa359a15640 1 --2- 172.21.15.94:0/2783872239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa354101290 0x7fa354101710 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.076+0000 7fa359a15640 1 --2- 172.21.15.94:0/2783872239 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa3541000a0 0x7fa3541004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.076+0000 7fa359a15640 1 -- 172.21.15.94:0/2783872239 >> 172.21.15.94:0/2783872239 conn(0x7fa3540fb610 msgr2=0x7fa3540fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.076+0000 7fa359a15640 1 -- 172.21.15.94:0/2783872239 shutdown_connections 2024-02-17T11:57:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.077+0000 7fa359a15640 1 -- 172.21.15.94:0/2783872239 wait complete. 2024-02-17T11:57:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.077+0000 7fa359a15640 1 Processor -- start 2024-02-17T11:57:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.077+0000 7fa359a15640 1 -- start start 2024-02-17T11:57:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.077+0000 7fa359a15640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3541000a0 0x7fa354197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.077+0000 7fa359a15640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa354101290 0x7fa354197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.077+0000 7fa359a15640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa354101c50 0x7fa35419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.077+0000 7fa359a15640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35419edf0 con 0x7fa3541000a0 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.077+0000 7fa359a15640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35419ef60 con 0x7fa354101c50 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.077+0000 7fa359a15640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35419f120 con 0x7fa354101290 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.077+0000 7fa352ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3541000a0 0x7fa354197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.077+0000 7fa352ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3541000a0 0x7fa354197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53348/0 (socket says 172.21.15.94:53348) 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.077+0000 7fa352ffd640 1 -- 172.21.15.94:0/753958819 learned_addr learned my addr 172.21.15.94:0/753958819 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa3537fe640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa354101c50 0x7fa35419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa3527fc640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa354101290 0x7fa354197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa352ffd640 1 -- 172.21.15.94:0/753958819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa354101290 msgr2=0x7fa354197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa352ffd640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa354101290 0x7fa354197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa352ffd640 1 -- 172.21.15.94:0/753958819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa354101c50 msgr2=0x7fa35419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa352ffd640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa354101c50 0x7fa35419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa352ffd640 1 -- 172.21.15.94:0/753958819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa348009560 con 0x7fa3541000a0 2024-02-17T11:57:57.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa3527fc640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa354101290 0x7fa354197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:57.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa352ffd640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3541000a0 0x7fa354197300 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa34400b9c0 tx=0x7fa34400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:57.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa358a13640 1 -- 172.21.15.94:0/753958819 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34400cba0 con 0x7fa3541000a0 2024-02-17T11:57:57.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa359a15640 1 -- 172.21.15.94:0/753958819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35419f400 con 0x7fa3541000a0 2024-02-17T11:57:57.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa358a13640 1 -- 172.21.15.94:0/753958819 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa34400cd00 con 0x7fa3541000a0 2024-02-17T11:57:57.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa358a13640 1 -- 172.21.15.94:0/753958819 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344012720 con 0x7fa3541000a0 2024-02-17T11:57:57.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.078+0000 7fa359a15640 1 -- 172.21.15.94:0/753958819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35419f9e0 con 0x7fa3541000a0 2024-02-17T11:57:57.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.079+0000 7fa359a15640 1 -- 172.21.15.94:0/753958819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35410bb10 con 0x7fa3541000a0 2024-02-17T11:57:57.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.080+0000 7fa358a13640 1 -- 172.21.15.94:0/753958819 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fa344014a40 con 0x7fa3541000a0 2024-02-17T11:57:57.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.080+0000 7fa358a13640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa33c0a2150 0x7fa33c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.081+0000 7fa3527fc640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa33c0a2150 0x7fa33c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.081+0000 7fa358a13640 1 -- 172.21.15.94:0/753958819 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fa3440cd700 con 0x7fa3541000a0 2024-02-17T11:57:57.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.081+0000 7fa3527fc640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa33c0a2150 0x7fa33c0a4610 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa348036040 tx=0x7fa348000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:57.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.085+0000 7fa358a13640 1 -- 172.21.15.94:0/753958819 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa34404fd20 con 0x7fa3541000a0 2024-02-17T11:57:57.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:56 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3153798007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:57.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:56 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3153798007' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:57.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.261+0000 7fa359a15640 1 -- 172.21.15.94:0/753958819 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa35410bca0 con 0x7fa3541000a0 2024-02-17T11:57:57.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.262+0000 7fa358a13640 1 -- 172.21.15.94:0/753958819 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22686 (secure 0 0 0) 0x7fa344096680 con 0x7fa3541000a0 2024-02-17T11:57:57.261 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:57.262 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":87,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:56:59.725646+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:57:57.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.264+0000 7fa359a15640 1 -- 172.21.15.94:0/753958819 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa33c0a2150 msgr2=0x7fa33c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.264+0000 7fa359a15640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa33c0a2150 0x7fa33c0a4610 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa348036040 tx=0x7fa348000bc0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.264+0000 7fa359a15640 1 -- 172.21.15.94:0/753958819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3541000a0 msgr2=0x7fa354197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.264+0000 7fa359a15640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3541000a0 0x7fa354197300 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa34400b9c0 tx=0x7fa34400be80 comp rx=0 tx=0).stop 2024-02-17T11:57:57.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.265+0000 7fa359a15640 1 -- 172.21.15.94:0/753958819 shutdown_connections 2024-02-17T11:57:57.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.265+0000 7fa359a15640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fa33c0a2150 0x7fa33c0a4610 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.265+0000 7fa359a15640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa354101c50 0x7fa35419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.265+0000 7fa359a15640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa354101290 0x7fa354197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.265+0000 7fa359a15640 1 --2- 172.21.15.94:0/753958819 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa3541000a0 0x7fa354197300 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.265+0000 7fa359a15640 1 -- 172.21.15.94:0/753958819 >> 172.21.15.94:0/753958819 conn(0x7fa3540fb610 msgr2=0x7fa3540fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:57.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.265+0000 7fa359a15640 1 -- 172.21.15.94:0/753958819 shutdown_connections 2024-02-17T11:57:57.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.265+0000 7fa359a15640 1 -- 172.21.15.94:0/753958819 wait complete. 2024-02-17T11:57:57.274 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T11:51:20.894601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T11:55:56.349040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T11:55:57.234916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T11:57:57.275 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T11:57:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.382+0000 7f68725a0640 1 -- 172.21.15.94:0/3242420978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f686c101170 msgr2=0x7f686c1015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.382+0000 7f68725a0640 1 --2- 172.21.15.94:0/3242420978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f686c101170 0x7f686c1015f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f685c009900 tx=0x7f685c02bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.382+0000 7f68725a0640 1 -- 172.21.15.94:0/3242420978 shutdown_connections 2024-02-17T11:57:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.382+0000 7f68725a0640 1 --2- 172.21.15.94:0/3242420978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f686c101b30 0x7f686c1083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.382+0000 7f68725a0640 1 --2- 172.21.15.94:0/3242420978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f686c101170 0x7f686c1015f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.382+0000 7f68725a0640 1 --2- 172.21.15.94:0/3242420978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f686c0fff80 0x7f686c1003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.382+0000 7f68725a0640 1 -- 172.21.15.94:0/3242420978 >> 172.21.15.94:0/3242420978 conn(0x7f686c0fb4f0 msgr2=0x7f686c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.382+0000 7f68725a0640 1 -- 172.21.15.94:0/3242420978 shutdown_connections 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.382+0000 7f68725a0640 1 -- 172.21.15.94:0/3242420978 wait complete. 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f68725a0640 1 Processor -- start 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f68725a0640 1 -- start start 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f68725a0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f686c0fff80 0x7f686c1970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f68725a0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f686c101170 0x7f686c197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f68725a0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f686c101b30 0x7f686c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f68725a0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c19ebd0 con 0x7f686c101170 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f68725a0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c19ed40 con 0x7f686c0fff80 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f68725a0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f686c19ef00 con 0x7f686c101b30 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f6870d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f686c101170 0x7f686c197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f6870d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f686c101170 0x7f686c197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53368/0 (socket says 172.21.15.94:53368) 2024-02-17T11:57:57.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f6870d9d640 1 -- 172.21.15.94:0/2614714942 learned_addr learned my addr 172.21.15.94:0/2614714942 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f6871d9f640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f686c101b30 0x7f686c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f687159e640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f686c0fff80 0x7f686c1970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f6870d9d640 1 -- 172.21.15.94:0/2614714942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f686c101b30 msgr2=0x7f686c19e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f6870d9d640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f686c101b30 0x7f686c19e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f6870d9d640 1 -- 172.21.15.94:0/2614714942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f686c0fff80 msgr2=0x7f686c1970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f6870d9d640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f686c0fff80 0x7f686c1970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f6870d9d640 1 -- 172.21.15.94:0/2614714942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c009560 con 0x7f686c101170 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.383+0000 7f6870d9d640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f686c101170 0x7f686c197620 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f685c006010 tx=0x7f685c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.384+0000 7f68627fc640 1 -- 172.21.15.94:0/2614714942 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c0044e0 con 0x7f686c101170 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.384+0000 7f68627fc640 1 -- 172.21.15.94:0/2614714942 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f685c0054e0 con 0x7f686c101170 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.384+0000 7f68725a0640 1 -- 172.21.15.94:0/2614714942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f686c19f180 con 0x7f686c101170 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.384+0000 7f68627fc640 1 -- 172.21.15.94:0/2614714942 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685c03e450 con 0x7f686c101170 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.384+0000 7f68725a0640 1 -- 172.21.15.94:0/2614714942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f686c19f690 con 0x7f686c101170 2024-02-17T11:57:57.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.385+0000 7f68725a0640 1 -- 172.21.15.94:0/2614714942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f686c10b980 con 0x7f686c101170 2024-02-17T11:57:57.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.386+0000 7f68627fc640 1 -- 172.21.15.94:0/2614714942 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f685c0035f0 con 0x7f686c101170 2024-02-17T11:57:57.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.386+0000 7f68627fc640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f68580a2150 0x7f68580a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.386+0000 7f687159e640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f68580a2150 0x7f68580a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.386+0000 7f68627fc640 1 -- 172.21.15.94:0/2614714942 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f685c0f0190 con 0x7f686c101170 2024-02-17T11:57:57.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.387+0000 7f687159e640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f68580a2150 0x7f68580a4610 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6868005fd0 tx=0x7f6868005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:57.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.390+0000 7f68627fc640 1 -- 172.21.15.94:0/2614714942 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f685c0b9110 con 0x7f686c101170 2024-02-17T11:57:57.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.564+0000 7f68725a0640 1 -- 172.21.15.94:0/2614714942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f686c19fa40 con 0x7f686c101170 2024-02-17T11:57:57.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.564+0000 7f68627fc640 1 -- 172.21.15.94:0/2614714942 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f685c0bc7d0 con 0x7f686c101170 2024-02-17T11:57:57.563 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-02-17T11:57:57.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.567+0000 7f68725a0640 1 -- 172.21.15.94:0/2614714942 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f68580a2150 msgr2=0x7f68580a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.567+0000 7f68725a0640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f68580a2150 0x7f68580a4610 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6868005fd0 tx=0x7f6868005850 comp rx=0 tx=0).stop 2024-02-17T11:57:57.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.567+0000 7f68725a0640 1 -- 172.21.15.94:0/2614714942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f686c101170 msgr2=0x7f686c197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.567+0000 7f68725a0640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f686c101170 0x7f686c197620 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f685c006010 tx=0x7f685c0042f0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.567+0000 7f68725a0640 1 -- 172.21.15.94:0/2614714942 shutdown_connections 2024-02-17T11:57:57.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.567+0000 7f68725a0640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f68580a2150 0x7f68580a4610 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.567+0000 7f68725a0640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f686c101b30 0x7f686c19e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.567+0000 7f68725a0640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f686c101170 0x7f686c197620 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.567+0000 7f68725a0640 1 --2- 172.21.15.94:0/2614714942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f686c0fff80 0x7f686c1970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.567+0000 7f68725a0640 1 -- 172.21.15.94:0/2614714942 >> 172.21.15.94:0/2614714942 conn(0x7f686c0fb4f0 msgr2=0x7f686c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:57.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.567+0000 7f68725a0640 1 -- 172.21.15.94:0/2614714942 shutdown_connections 2024-02-17T11:57:57.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.567+0000 7f68725a0640 1 -- 172.21.15.94:0/2614714942 wait complete. 2024-02-17T11:57:57.576 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T11:57:57.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.683+0000 7f0c15fa1640 1 -- 172.21.15.94:0/2050700096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c10102df0 msgr2=0x7f0c10109680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.683+0000 7f0c15fa1640 1 --2- 172.21.15.94:0/2050700096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c10102df0 0x7f0c10109680 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009ca0 tx=0x7f0c0802c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.683+0000 7f0c15fa1640 1 -- 172.21.15.94:0/2050700096 shutdown_connections 2024-02-17T11:57:57.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.683+0000 7f0c15fa1640 1 --2- 172.21.15.94:0/2050700096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c10102df0 0x7f0c10109680 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.683+0000 7f0c15fa1640 1 --2- 172.21.15.94:0/2050700096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c10102430 0x7f0c101028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.683+0000 7f0c15fa1640 1 --2- 172.21.15.94:0/2050700096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c100ff460 0x7f0c100ff880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.683+0000 7f0c15fa1640 1 -- 172.21.15.94:0/2050700096 >> 172.21.15.94:0/2050700096 conn(0x7f0c100fadb0 msgr2=0x7f0c100fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:57.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.683+0000 7f0c15fa1640 1 -- 172.21.15.94:0/2050700096 shutdown_connections 2024-02-17T11:57:57.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.683+0000 7f0c15fa1640 1 -- 172.21.15.94:0/2050700096 wait complete. 2024-02-17T11:57:57.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c15fa1640 1 Processor -- start 2024-02-17T11:57:57.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c15fa1640 1 -- start start 2024-02-17T11:57:57.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c15fa1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c100ff460 0x7f0c101971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c15fa1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c10102430 0x7f0c10197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c15fa1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c10102df0 0x7f0c1019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c15fa1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1019ecb0 con 0x7f0c10102df0 2024-02-17T11:57:57.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c15fa1640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1019ee20 con 0x7f0c10102430 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c15fa1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1019efe0 con 0x7f0c100ff460 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c0ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c10102df0 0x7f0c1019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c0ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c10102df0 0x7f0c1019e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53398/0 (socket says 172.21.15.94:53398) 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c0ffff640 1 -- 172.21.15.94:0/962177671 learned_addr learned my addr 172.21.15.94:0/962177671 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c0f7fe640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c100ff460 0x7f0c101971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c0effd640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c10102430 0x7f0c10197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c0ffff640 1 -- 172.21.15.94:0/962177671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c100ff460 msgr2=0x7f0c101971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c0ffff640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c100ff460 0x7f0c101971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c0ffff640 1 -- 172.21.15.94:0/962177671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c10102430 msgr2=0x7f0c10197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c0ffff640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c10102430 0x7f0c10197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c0ffff640 1 -- 172.21.15.94:0/962177671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c08009900 con 0x7f0c10102df0 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.684+0000 7f0c0effd640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c10102430 0x7f0c10197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.685+0000 7f0c0ffff640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c10102df0 0x7f0c1019e770 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0c08037040 tx=0x7f0c080346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:57.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.685+0000 7f0c0cff9640 1 -- 172.21.15.94:0/962177671 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08032e70 con 0x7f0c10102df0 2024-02-17T11:57:57.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.685+0000 7f0c0cff9640 1 -- 172.21.15.94:0/962177671 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c08003e50 con 0x7f0c10102df0 2024-02-17T11:57:57.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.685+0000 7f0c15fa1640 1 -- 172.21.15.94:0/962177671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1019f260 con 0x7f0c10102df0 2024-02-17T11:57:57.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.685+0000 7f0c0cff9640 1 -- 172.21.15.94:0/962177671 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08040450 con 0x7f0c10102df0 2024-02-17T11:57:57.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.685+0000 7f0c15fa1640 1 -- 172.21.15.94:0/962177671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1019f710 con 0x7f0c10102df0 2024-02-17T11:57:57.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.686+0000 7f0c15fa1640 1 -- 172.21.15.94:0/962177671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c10108500 con 0x7f0c10102df0 2024-02-17T11:57:57.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.687+0000 7f0c0cff9640 1 -- 172.21.15.94:0/962177671 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f0c0800a040 con 0x7f0c10102df0 2024-02-17T11:57:57.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.688+0000 7f0c0cff9640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0bf80a2150 0x7f0bf80a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.688+0000 7f0c0f7fe640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0bf80a2150 0x7f0bf80a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.688+0000 7f0c0cff9640 1 -- 172.21.15.94:0/962177671 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f0c080f0cc0 con 0x7f0c10102df0 2024-02-17T11:57:57.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.688+0000 7f0c0f7fe640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0bf80a2150 0x7f0bf80a4610 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0c100695d0 tx=0x7f0c0000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:57.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.692+0000 7f0c0cff9640 1 -- 172.21.15.94:0/962177671 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0c080b9c40 con 0x7f0c10102df0 2024-02-17T11:57:57.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.866+0000 7f0c15fa1640 1 -- 172.21.15.94:0/962177671 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0c1002ce40 con 0x7f0c10102df0 2024-02-17T11:57:57.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.866+0000 7f0c0cff9640 1 -- 172.21.15.94:0/962177671 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0c080bd300 con 0x7f0c10102df0 2024-02-17T11:57:57.865 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-02-17T11:57:57.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.868+0000 7f0c15fa1640 1 -- 172.21.15.94:0/962177671 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0bf80a2150 msgr2=0x7f0bf80a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.868+0000 7f0c15fa1640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0bf80a2150 0x7f0bf80a4610 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0c100695d0 tx=0x7f0c0000a2d0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.868+0000 7f0c15fa1640 1 -- 172.21.15.94:0/962177671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c10102df0 msgr2=0x7f0c1019e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.868+0000 7f0c15fa1640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c10102df0 0x7f0c1019e770 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0c08037040 tx=0x7f0c080346f0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.869+0000 7f0c15fa1640 1 -- 172.21.15.94:0/962177671 shutdown_connections 2024-02-17T11:57:57.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.869+0000 7f0c15fa1640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f0bf80a2150 0x7f0bf80a4610 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.869+0000 7f0c15fa1640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0c10102df0 0x7f0c1019e770 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.869+0000 7f0c15fa1640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0c10102430 0x7f0c10197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.869+0000 7f0c15fa1640 1 --2- 172.21.15.94:0/962177671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c100ff460 0x7f0c101971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.869+0000 7f0c15fa1640 1 -- 172.21.15.94:0/962177671 >> 172.21.15.94:0/962177671 conn(0x7f0c100fadb0 msgr2=0x7f0c101056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:57.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.869+0000 7f0c15fa1640 1 -- 172.21.15.94:0/962177671 shutdown_connections 2024-02-17T11:57:57.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.869+0000 7f0c15fa1640 1 -- 172.21.15.94:0/962177671 wait complete. 2024-02-17T11:57:57.877 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:57 smithi094 ceph-mon[32499]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:57.877 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:57 smithi094 ceph-mon[32499]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:57.877 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:57 smithi094 ceph-mon[32499]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-02-17T11:57:57.877 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:57 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/753958819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:57.878 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:57 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2614714942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:57.878 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T11:57:57.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.984+0000 7feb3186f640 1 -- 172.21.15.94:0/901565329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb2c101c00 msgr2=0x7feb2c103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.984+0000 7feb3186f640 1 --2- 172.21.15.94:0/901565329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb2c101c00 0x7feb2c103ff0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009ca0 tx=0x7feb1c02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.984+0000 7feb3186f640 1 -- 172.21.15.94:0/901565329 shutdown_connections 2024-02-17T11:57:57.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.984+0000 7feb3186f640 1 --2- 172.21.15.94:0/901565329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb2c106e60 0x7feb2c1092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.984+0000 7feb3186f640 1 --2- 172.21.15.94:0/901565329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb2c104530 0x7feb2c106920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.984+0000 7feb3186f640 1 --2- 172.21.15.94:0/901565329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb2c101c00 0x7feb2c103ff0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.984+0000 7feb3186f640 1 -- 172.21.15.94:0/901565329 >> 172.21.15.94:0/901565329 conn(0x7feb2c0fb610 msgr2=0x7feb2c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.984+0000 7feb3186f640 1 -- 172.21.15.94:0/901565329 shutdown_connections 2024-02-17T11:57:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.984+0000 7feb3186f640 1 -- 172.21.15.94:0/901565329 wait complete. 2024-02-17T11:57:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.985+0000 7feb3186f640 1 Processor -- start 2024-02-17T11:57:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.985+0000 7feb3186f640 1 -- start start 2024-02-17T11:57:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.985+0000 7feb3186f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb2c101c00 0x7feb2c197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.985+0000 7feb3186f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb2c104530 0x7feb2c1977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.985+0000 7feb3186f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb2c106e60 0x7feb2c19e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.985+0000 7feb3186f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c19ed60 con 0x7feb2c106e60 2024-02-17T11:57:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.985+0000 7feb3186f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c19eed0 con 0x7feb2c104530 2024-02-17T11:57:57.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.985+0000 7feb3186f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2c19f090 con 0x7feb2c101c00 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.985+0000 7feb2b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb2c106e60 0x7feb2c19e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.985+0000 7feb2b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb2c106e60 0x7feb2c19e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53418/0 (socket says 172.21.15.94:53418) 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.985+0000 7feb2b7fe640 1 -- 172.21.15.94:0/211458928 learned_addr learned my addr 172.21.15.94:0/211458928 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.985+0000 7feb2a7fc640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb2c104530 0x7feb2c1977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.985+0000 7feb2affd640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb2c101c00 0x7feb2c197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.986+0000 7feb2b7fe640 1 -- 172.21.15.94:0/211458928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb2c101c00 msgr2=0x7feb2c197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.986+0000 7feb2b7fe640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb2c101c00 0x7feb2c197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.986+0000 7feb2b7fe640 1 -- 172.21.15.94:0/211458928 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb2c104530 msgr2=0x7feb2c1977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.986+0000 7feb2b7fe640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb2c104530 0x7feb2c1977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.986+0000 7feb2b7fe640 1 -- 172.21.15.94:0/211458928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1c009900 con 0x7feb2c106e60 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.986+0000 7feb2a7fc640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb2c104530 0x7feb2c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.986+0000 7feb2b7fe640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb2c106e60 0x7feb2c19e820 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feb2400d750 tx=0x7feb2400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.986+0000 7feb3086d640 1 -- 172.21.15.94:0/211458928 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24009bd0 con 0x7feb2c106e60 2024-02-17T11:57:57.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.986+0000 7feb3186f640 1 -- 172.21.15.94:0/211458928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2c19f370 con 0x7feb2c106e60 2024-02-17T11:57:57.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.986+0000 7feb3086d640 1 -- 172.21.15.94:0/211458928 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb2400de60 con 0x7feb2c106e60 2024-02-17T11:57:57.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.986+0000 7feb3086d640 1 -- 172.21.15.94:0/211458928 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24004e70 con 0x7feb2c106e60 2024-02-17T11:57:57.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.986+0000 7feb3186f640 1 -- 172.21.15.94:0/211458928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2c19f830 con 0x7feb2c106e60 2024-02-17T11:57:57.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.987+0000 7feb3186f640 1 -- 172.21.15.94:0/211458928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c0fd100 con 0x7feb2c106e60 2024-02-17T11:57:57.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.988+0000 7feb3086d640 1 -- 172.21.15.94:0/211458928 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7feb24005020 con 0x7feb2c106e60 2024-02-17T11:57:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.988+0000 7feb3086d640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7feb140a2150 0x7feb140a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.988+0000 7feb2affd640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7feb140a2150 0x7feb140a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.988+0000 7feb3086d640 1 -- 172.21.15.94:0/211458928 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7feb240cd8e0 con 0x7feb2c106e60 2024-02-17T11:57:57.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.989+0000 7feb2affd640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7feb140a2150 0x7feb140a4610 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7feb1c006010 tx=0x7feb1c0326b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:57.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:57.993+0000 7feb3086d640 1 -- 172.21.15.94:0/211458928 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feb2409a280 con 0x7feb2c106e60 2024-02-17T11:57:58.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:57 smithi189 ceph-mon[36588]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:58.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:57 smithi189 ceph-mon[36588]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:58.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:57 smithi189 ceph-mon[36588]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-02-17T11:57:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:57 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/753958819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:57 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2614714942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:58.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.165+0000 7feb3186f640 1 -- 172.21.15.94:0/211458928 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feb2c02ce40 con 0x7feb2c106e60 2024-02-17T11:57:58.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.165+0000 7feb3086d640 1 -- 172.21.15.94:0/211458928 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7feb240200a0 con 0x7feb2c106e60 2024-02-17T11:57:58.164 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-02-17T11:57:58.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.167+0000 7feb3186f640 1 -- 172.21.15.94:0/211458928 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7feb140a2150 msgr2=0x7feb140a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.167+0000 7feb3186f640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7feb140a2150 0x7feb140a4610 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7feb1c006010 tx=0x7feb1c0326b0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.167+0000 7feb3186f640 1 -- 172.21.15.94:0/211458928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb2c106e60 msgr2=0x7feb2c19e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.167+0000 7feb3186f640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb2c106e60 0x7feb2c19e820 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7feb2400d750 tx=0x7feb2400dc10 comp rx=0 tx=0).stop 2024-02-17T11:57:58.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.167+0000 7feb3186f640 1 -- 172.21.15.94:0/211458928 shutdown_connections 2024-02-17T11:57:58.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.167+0000 7feb3186f640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7feb140a2150 0x7feb140a4610 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.167+0000 7feb3186f640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb2c106e60 0x7feb2c19e820 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.167+0000 7feb3186f640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb2c104530 0x7feb2c1977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.167+0000 7feb3186f640 1 --2- 172.21.15.94:0/211458928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb2c101c00 0x7feb2c197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.167+0000 7feb3186f640 1 -- 172.21.15.94:0/211458928 >> 172.21.15.94:0/211458928 conn(0x7feb2c0fb610 msgr2=0x7feb2c102700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:58.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.167+0000 7feb3186f640 1 -- 172.21.15.94:0/211458928 shutdown_connections 2024-02-17T11:57:58.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.167+0000 7feb3186f640 1 -- 172.21.15.94:0/211458928 wait complete. 2024-02-17T11:57:58.177 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:57:58.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:57 smithi125 ceph-mon[36819]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:58.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:57 smithi125 ceph-mon[36819]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:58.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:57 smithi125 ceph-mon[36819]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-02-17T11:57:58.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:57 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/753958819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:57:58.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:57 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2614714942' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T11:57:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.279+0000 7febcb3f4640 1 -- 172.21.15.94:0/3028603730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc4102460 msgr2=0x7febc41028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.279+0000 7febcb3f4640 1 --2- 172.21.15.94:0/3028603730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc4102460 0x7febc41028e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7febb4009900 tx=0x7febb402bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 -- 172.21.15.94:0/3028603730 shutdown_connections 2024-02-17T11:57:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 --2- 172.21.15.94:0/3028603730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc4102e20 0x7febc41096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 --2- 172.21.15.94:0/3028603730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc4102460 0x7febc41028e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 --2- 172.21.15.94:0/3028603730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc40ff490 0x7febc40ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 -- 172.21.15.94:0/3028603730 >> 172.21.15.94:0/3028603730 conn(0x7febc40fadc0 msgr2=0x7febc40fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 -- 172.21.15.94:0/3028603730 shutdown_connections 2024-02-17T11:57:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 -- 172.21.15.94:0/3028603730 wait complete. 2024-02-17T11:57:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 Processor -- start 2024-02-17T11:57:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 -- start start 2024-02-17T11:57:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc40ff490 0x7febc4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc4102460 0x7febc4197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc4102e20 0x7febc419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc419ecf0 con 0x7febc40ff490 2024-02-17T11:57:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc419ee60 con 0x7febc4102460 2024-02-17T11:57:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febcb3f4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc419f020 con 0x7febc4102e20 2024-02-17T11:57:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febc9190640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc40ff490 0x7febc4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febc9190640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc40ff490 0x7febc4197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53436/0 (socket says 172.21.15.94:53436) 2024-02-17T11:57:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.280+0000 7febc9190640 1 -- 172.21.15.94:0/1495762682 learned_addr learned my addr 172.21.15.94:0/1495762682 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febc898f640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc4102460 0x7febc4197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febc9991640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc4102e20 0x7febc419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febc9190640 1 -- 172.21.15.94:0/1495762682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc4102e20 msgr2=0x7febc419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febc9190640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc4102e20 0x7febc419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febc9190640 1 -- 172.21.15.94:0/1495762682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc4102460 msgr2=0x7febc4197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febc9190640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc4102460 0x7febc4197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febc9190640 1 -- 172.21.15.94:0/1495762682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb4009560 con 0x7febc40ff490 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febc898f640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc4102460 0x7febc4197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febc9991640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc4102e20 0x7febc419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febc9190640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc40ff490 0x7febc4197200 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c970 tx=0x7febbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febba7fc640 1 -- 172.21.15.94:0/1495762682 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc007c10 con 0x7febc40ff490 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febba7fc640 1 -- 172.21.15.94:0/1495762682 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febbc007d70 con 0x7febc40ff490 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febcb3f4640 1 -- 172.21.15.94:0/1495762682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc419f300 con 0x7febc40ff490 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febba7fc640 1 -- 172.21.15.94:0/1495762682 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc00f8a0 con 0x7febc40ff490 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.281+0000 7febcb3f4640 1 -- 172.21.15.94:0/1495762682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc419f860 con 0x7febc40ff490 2024-02-17T11:57:58.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.282+0000 7febcb3f4640 1 -- 172.21.15.94:0/1495762682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc4108590 con 0x7febc40ff490 2024-02-17T11:57:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.283+0000 7febba7fc640 1 -- 172.21.15.94:0/1495762682 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7febbc018af0 con 0x7febc40ff490 2024-02-17T11:57:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.284+0000 7febba7fc640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7febb00a21d0 0x7febb00a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.284+0000 7febc898f640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7febb00a21d0 0x7febb00a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.284+0000 7febba7fc640 1 -- 172.21.15.94:0/1495762682 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7febbc013070 con 0x7febc40ff490 2024-02-17T11:57:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.284+0000 7febc898f640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7febb00a21d0 0x7febb00a4690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7febc4198710 tx=0x7febb4005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.288+0000 7febba7fc640 1 -- 172.21.15.94:0/1495762682 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7febbc0970a0 con 0x7febc40ff490 2024-02-17T11:57:58.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.499+0000 7febcb3f4640 1 -- 172.21.15.94:0/1495762682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febc410cbf0 con 0x7febc40ff490 2024-02-17T11:57:58.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.500+0000 7febba7fc640 1 -- 172.21.15.94:0/1495762682 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11346 (secure 0 0 0) 0x7febbc09a760 con 0x7febc40ff490 2024-02-17T11:57:58.500 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:58.500 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:57:51.854453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T11:57:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 21 2024-02-17T11:57:58.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.502+0000 7febcb3f4640 1 -- 172.21.15.94:0/1495762682 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7febb00a21d0 msgr2=0x7febb00a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.502+0000 7febcb3f4640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7febb00a21d0 0x7febb00a4690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7febc4198710 tx=0x7febb4005be0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.502+0000 7febcb3f4640 1 -- 172.21.15.94:0/1495762682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc40ff490 msgr2=0x7febc4197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.502+0000 7febcb3f4640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc40ff490 0x7febc4197200 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7febbc00c970 tx=0x7febbc00ce30 comp rx=0 tx=0).stop 2024-02-17T11:57:58.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.502+0000 7febcb3f4640 1 -- 172.21.15.94:0/1495762682 shutdown_connections 2024-02-17T11:57:58.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.502+0000 7febcb3f4640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7febb00a21d0 0x7febb00a4690 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.502+0000 7febcb3f4640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc4102e20 0x7febc419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.502+0000 7febcb3f4640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc4102460 0x7febc4197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.502+0000 7febcb3f4640 1 --2- 172.21.15.94:0/1495762682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc40ff490 0x7febc4197200 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.502+0000 7febcb3f4640 1 -- 172.21.15.94:0/1495762682 >> 172.21.15.94:0/1495762682 conn(0x7febc40fadc0 msgr2=0x7febc4105760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:58.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.502+0000 7febcb3f4640 1 -- 172.21.15.94:0/1495762682 shutdown_connections 2024-02-17T11:57:58.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.502+0000 7febcb3f4640 1 -- 172.21.15.94:0/1495762682 wait complete. 2024-02-17T11:57:58.512 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T11:57:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.614+0000 7f28368d4640 1 -- 172.21.15.94:0/4082122515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2830101190 msgr2=0x7f28301015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.614+0000 7f28368d4640 1 --2- 172.21.15.94:0/4082122515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2830101190 0x7f28301015b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2820009900 tx=0x7f282002bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.614+0000 7f28368d4640 1 -- 172.21.15.94:0/4082122515 shutdown_connections 2024-02-17T11:57:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.614+0000 7f28368d4640 1 --2- 172.21.15.94:0/4082122515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2830102d40 0x7f28301095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.614+0000 7f28368d4640 1 --2- 172.21.15.94:0/4082122515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2830102380 0x7f2830102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.614+0000 7f28368d4640 1 --2- 172.21.15.94:0/4082122515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2830101190 0x7f28301015b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.614+0000 7f28368d4640 1 -- 172.21.15.94:0/4082122515 >> 172.21.15.94:0/4082122515 conn(0x7f28300fc760 msgr2=0x7f28300feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.614+0000 7f28368d4640 1 -- 172.21.15.94:0/4082122515 shutdown_connections 2024-02-17T11:57:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.614+0000 7f28368d4640 1 -- 172.21.15.94:0/4082122515 wait complete. 2024-02-17T11:57:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.614+0000 7f28368d4640 1 Processor -- start 2024-02-17T11:57:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.614+0000 7f28368d4640 1 -- start start 2024-02-17T11:57:58.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f28368d4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2830101190 0x7f2830197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f28368d4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2830102380 0x7f28301977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f28368d4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2830102d40 0x7f283019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f28368d4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283019ed80 con 0x7f2830102380 2024-02-17T11:57:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f28368d4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283019eef0 con 0x7f2830101190 2024-02-17T11:57:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f28368d4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283019f0b0 con 0x7f2830102d40 2024-02-17T11:57:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f282f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2830102380 0x7f28301977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f282f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2830102380 0x7f28301977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53458/0 (socket says 172.21.15.94:53458) 2024-02-17T11:57:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f282f7fe640 1 -- 172.21.15.94:0/1353174837 learned_addr learned my addr 172.21.15.94:0/1353174837 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:58.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f282ffff640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2830101190 0x7f2830197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f2834e71640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2830102d40 0x7f283019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f282f7fe640 1 -- 172.21.15.94:0/1353174837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2830102d40 msgr2=0x7f283019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f282f7fe640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2830102d40 0x7f283019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f282f7fe640 1 -- 172.21.15.94:0/1353174837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2830101190 msgr2=0x7f2830197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f282f7fe640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2830101190 0x7f2830197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f282f7fe640 1 -- 172.21.15.94:0/1353174837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2820009560 con 0x7f2830102380 2024-02-17T11:57:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f282f7fe640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2830102380 0x7f28301977d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f282400d770 tx=0x7f282400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f282d7fa640 1 -- 172.21.15.94:0/1353174837 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824009c30 con 0x7f2830102380 2024-02-17T11:57:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f282d7fa640 1 -- 172.21.15.94:0/1353174837 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f282400de80 con 0x7f2830102380 2024-02-17T11:57:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f28368d4640 1 -- 172.21.15.94:0/1353174837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283019f390 con 0x7f2830102380 2024-02-17T11:57:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f282d7fa640 1 -- 172.21.15.94:0/1353174837 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2824004e80 con 0x7f2830102380 2024-02-17T11:57:58.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.615+0000 7f28368d4640 1 -- 172.21.15.94:0/1353174837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283019f850 con 0x7f2830102380 2024-02-17T11:57:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.617+0000 7f28368d4640 1 -- 172.21.15.94:0/1353174837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2830108580 con 0x7f2830102380 2024-02-17T11:57:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.618+0000 7f282d7fa640 1 -- 172.21.15.94:0/1353174837 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7f2824005000 con 0x7f2830102380 2024-02-17T11:57:58.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.618+0000 7f282d7fa640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f28180a2150 0x7f28180a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.618+0000 7f282ffff640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f28180a2150 0x7f28180a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.618+0000 7f282d7fa640 1 -- 172.21.15.94:0/1353174837 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f28240cdbf0 con 0x7f2830102380 2024-02-17T11:57:58.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.619+0000 7f282ffff640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f28180a2150 0x7f28180a4610 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2820005ab0 tx=0x7f2820005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:58.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.622+0000 7f282d7fa640 1 -- 172.21.15.94:0/1353174837 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2824096b10 con 0x7f2830102380 2024-02-17T11:57:58.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.814+0000 7f28368d4640 1 -- 172.21.15.94:0/1353174837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f283010cdf0 con 0x7f2830102380 2024-02-17T11:57:58.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.815+0000 7f282d7fa640 1 -- 172.21.15.94:0/1353174837 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22686 (secure 0 0 0) 0x7f2824017090 con 0x7f2830102380 2024-02-17T11:57:58.814 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:57:58.814 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":87,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T11:56:59.725646+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T11:57:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.817+0000 7f28368d4640 1 -- 172.21.15.94:0/1353174837 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f28180a2150 msgr2=0x7f28180a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.817+0000 7f28368d4640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f28180a2150 0x7f28180a4610 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2820005ab0 tx=0x7f2820005fd0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.817+0000 7f28368d4640 1 -- 172.21.15.94:0/1353174837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2830102380 msgr2=0x7f28301977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.817+0000 7f28368d4640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2830102380 0x7f28301977d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f282400d770 tx=0x7f282400dc30 comp rx=0 tx=0).stop 2024-02-17T11:57:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.817+0000 7f28368d4640 1 -- 172.21.15.94:0/1353174837 shutdown_connections 2024-02-17T11:57:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.817+0000 7f28368d4640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f28180a2150 0x7f28180a4610 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.817+0000 7f28368d4640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2830102d40 0x7f283019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.817+0000 7f28368d4640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2830102380 0x7f28301977d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.817+0000 7f28368d4640 1 --2- 172.21.15.94:0/1353174837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2830101190 0x7f2830197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.817+0000 7f28368d4640 1 -- 172.21.15.94:0/1353174837 >> 172.21.15.94:0/1353174837 conn(0x7f28300fc760 msgr2=0x7f2830105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:58.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.817+0000 7f28368d4640 1 -- 172.21.15.94:0/1353174837 shutdown_connections 2024-02-17T11:57:58.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.817+0000 7f28368d4640 1 -- 172.21.15.94:0/1353174837 wait complete. 2024-02-17T11:57:58.828 DEBUG:tasks.fwd_scrub:Yielding 2024-02-17T11:57:58.828 INFO:teuthology.run_tasks:Running task exec... 2024-02-17T11:57:58.837 INFO:teuthology.task.exec:Executing custom commands... 2024-02-17T11:57:58.837 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-02-17T11:57:58.837 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-02-17T11:57:58.837 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-02-17T11:57:58.837 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T11:57:58.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:57:58.838 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T11:57:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.942+0000 7fadd2bb5640 1 -- 172.21.15.94:0/1669908026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadcc1045b0 msgr2=0x7fadcc1069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.942+0000 7fadd2bb5640 1 --2- 172.21.15.94:0/1669908026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadcc1045b0 0x7fadcc1069a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009900 tx=0x7fadb802bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:58.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.942+0000 7fadd2bb5640 1 -- 172.21.15.94:0/1669908026 shutdown_connections 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.942+0000 7fadd2bb5640 1 --2- 172.21.15.94:0/1669908026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadcc106ee0 0x7fadcc1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.942+0000 7fadd2bb5640 1 --2- 172.21.15.94:0/1669908026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadcc1045b0 0x7fadcc1069a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.942+0000 7fadd2bb5640 1 --2- 172.21.15.94:0/1669908026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadcc101c80 0x7fadcc104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.942+0000 7fadd2bb5640 1 -- 172.21.15.94:0/1669908026 >> 172.21.15.94:0/1669908026 conn(0x7fadcc0fb610 msgr2=0x7fadcc0fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.942+0000 7fadd2bb5640 1 -- 172.21.15.94:0/1669908026 shutdown_connections 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.942+0000 7fadd2bb5640 1 -- 172.21.15.94:0/1669908026 wait complete. 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.942+0000 7fadd2bb5640 1 Processor -- start 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.942+0000 7fadd2bb5640 1 -- start start 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.942+0000 7fadd2bb5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadcc101c80 0x7fadcc197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.942+0000 7fadd2bb5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadcc1045b0 0x7fadcc197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd2bb5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadcc106ee0 0x7fadcc19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd2bb5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc19ec30 con 0x7fadcc101c80 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd2bb5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc19eda0 con 0x7fadcc1045b0 2024-02-17T11:57:58.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd2bb5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc19ef60 con 0x7fadcc106ee0 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd0951640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadcc101c80 0x7fadcc197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd0951640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadcc101c80 0x7fadcc197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53468/0 (socket says 172.21.15.94:53468) 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd0951640 1 -- 172.21.15.94:0/523503502 learned_addr learned my addr 172.21.15.94:0/523503502 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadcbfff640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadcc1045b0 0x7fadcc197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd1152640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadcc106ee0 0x7fadcc19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd0951640 1 -- 172.21.15.94:0/523503502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadcc106ee0 msgr2=0x7fadcc19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd0951640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadcc106ee0 0x7fadcc19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd0951640 1 -- 172.21.15.94:0/523503502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadcc1045b0 msgr2=0x7fadcc197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd0951640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadcc1045b0 0x7fadcc197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd0951640 1 -- 172.21.15.94:0/523503502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb8009560 con 0x7fadcc101c80 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadcbfff640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadcc1045b0 0x7fadcc197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd0951640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadcc101c80 0x7fadcc197140 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c920 tx=0x7fadc000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadc9ffb640 1 -- 172.21.15.94:0/523503502 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0007c70 con 0x7fadcc101c80 2024-02-17T11:57:58.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadd2bb5640 1 -- 172.21.15.94:0/523503502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc19f240 con 0x7fadcc101c80 2024-02-17T11:57:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.943+0000 7fadc9ffb640 1 -- 172.21.15.94:0/523503502 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadc0007dd0 con 0x7fadcc101c80 2024-02-17T11:57:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.944+0000 7fadc9ffb640 1 -- 172.21.15.94:0/523503502 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0018860 con 0x7fadcc101c80 2024-02-17T11:57:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.944+0000 7fadd2bb5640 1 -- 172.21.15.94:0/523503502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc19f770 con 0x7fadcc101c80 2024-02-17T11:57:58.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.945+0000 7fadd2bb5640 1 -- 172.21.15.94:0/523503502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc0fd100 con 0x7fadcc101c80 2024-02-17T11:57:58.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.945+0000 7fe923add640 1 -- 172.21.15.94:0/945185476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe91c101270 msgr2=0x7fe91c1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.945+0000 7fe923add640 1 --2- 172.21.15.94:0/945185476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe91c101270 0x7fe91c1016f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009900 tx=0x7fe90c02bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:58.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 -- 172.21.15.94:0/945185476 shutdown_connections 2024-02-17T11:57:58.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 --2- 172.21.15.94:0/945185476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe91c101c30 0x7fe91c1084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 --2- 172.21.15.94:0/945185476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe91c101270 0x7fe91c1016f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 --2- 172.21.15.94:0/945185476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe91c100080 0x7fe91c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 -- 172.21.15.94:0/945185476 >> 172.21.15.94:0/945185476 conn(0x7fe91c0fb610 msgr2=0x7fe91c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:58.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 -- 172.21.15.94:0/945185476 shutdown_connections 2024-02-17T11:57:58.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 -- 172.21.15.94:0/945185476 wait complete. 2024-02-17T11:57:58.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fadc9ffb640 1 -- 172.21.15.94:0/523503502 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fadc000fac0 con 0x7fadcc101c80 2024-02-17T11:57:58.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 Processor -- start 2024-02-17T11:57:58.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fadc9ffb640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadbc0a2280 0x7fadbc0a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 -- start start 2024-02-17T11:57:58.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe91c100080 0x7fe91c197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe91c101270 0x7fe91c1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe91c101c30 0x7fe91c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c19ed80 con 0x7fe91c100080 2024-02-17T11:57:58.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c19eef0 con 0x7fe91c101270 2024-02-17T11:57:58.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fe923add640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91c19f0b0 con 0x7fe91c101c30 2024-02-17T11:57:58.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.946+0000 7fadc9ffb640 1 -- 172.21.15.94:0/523503502 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fadc00cdfc0 con 0x7fadcc101c80 2024-02-17T11:57:58.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.947+0000 7fadcbfff640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadbc0a2280 0x7fadbc0a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.947+0000 7fe921879640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe91c100080 0x7fe91c197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.947+0000 7fe921879640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe91c100080 0x7fe91c197290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53490/0 (socket says 172.21.15.94:53490) 2024-02-17T11:57:58.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.947+0000 7fe921879640 1 -- 172.21.15.94:0/3850365204 learned_addr learned my addr 172.21.15.94:0/3850365204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:58.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.947+0000 7fe921078640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe91c101270 0x7fe91c1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.947+0000 7fe92207a640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe91c101c30 0x7fe91c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.947+0000 7fe921879640 1 -- 172.21.15.94:0/3850365204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe91c101c30 msgr2=0x7fe91c19e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.947+0000 7fe921879640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe91c101c30 0x7fe91c19e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.947+0000 7fe921879640 1 -- 172.21.15.94:0/3850365204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe91c101270 msgr2=0x7fe91c1977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.947+0000 7fe921879640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe91c101270 0x7fe91c1977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.947+0000 7fe921879640 1 -- 172.21.15.94:0/3850365204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c009560 con 0x7fe91c100080 2024-02-17T11:57:58.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.947+0000 7fadcbfff640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadbc0a2280 0x7fadbc0a4740 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fadb8006010 tx=0x7fadb8005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:58.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.947+0000 7fe921879640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe91c100080 0x7fe91c197290 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe91800e9c0 tx=0x7fe91800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:58.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.951+0000 7fe912ffd640 1 -- 172.21.15.94:0/3850365204 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918009700 con 0x7fe91c100080 2024-02-17T11:57:58.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.951+0000 7fe912ffd640 1 -- 172.21.15.94:0/3850365204 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe91800cd80 con 0x7fe91c100080 2024-02-17T11:57:58.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.951+0000 7fe923add640 1 -- 172.21.15.94:0/3850365204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91c19f390 con 0x7fe91c100080 2024-02-17T11:57:58.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.951+0000 7fe912ffd640 1 -- 172.21.15.94:0/3850365204 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe918010430 con 0x7fe91c100080 2024-02-17T11:57:58.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.951+0000 7fadc9ffb640 1 -- 172.21.15.94:0/523503502 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fadc0096e30 con 0x7fadcc101c80 2024-02-17T11:57:58.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.951+0000 7fe923add640 1 -- 172.21.15.94:0/3850365204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91c19f870 con 0x7fe91c100080 2024-02-17T11:57:58.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.953+0000 7fe912ffd640 1 -- 172.21.15.94:0/3850365204 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fe918010590 con 0x7fe91c100080 2024-02-17T11:57:58.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.953+0000 7fe926782740 1 Processor -- start 2024-02-17T11:57:58.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe926782740 1 -- start start 2024-02-17T11:57:58.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe926782740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c74c0 0x55cbd05c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe926782740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 0x55cbd05c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe926782740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c9050 0x55cbd05cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe926782740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd0536f90 con 0x55cbd05c8690 2024-02-17T11:57:58.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe926782740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd05344c0 con 0x55cbd05c9050 2024-02-17T11:57:58.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe926782740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd052fca0 con 0x55cbd05c74c0 2024-02-17T11:57:58.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe921879640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 0x55cbd05c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe921879640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 0x55cbd05c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53492/0 (socket says 172.21.15.94:53492) 2024-02-17T11:57:58.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe921879640 1 -- 172.21.15.94:0/1510518157 learned_addr learned my addr 172.21.15.94:0/1510518157 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:58.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe92207a640 1 --2- 172.21.15.94:0/1510518157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c74c0 0x55cbd05c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe921078640 1 --2- 172.21.15.94:0/1510518157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c9050 0x55cbd05cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe921879640 1 -- 172.21.15.94:0/1510518157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c74c0 msgr2=0x55cbd05c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe921879640 1 --2- 172.21.15.94:0/1510518157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c74c0 0x55cbd05c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe921879640 1 -- 172.21.15.94:0/1510518157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c9050 msgr2=0x55cbd05cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe921879640 1 --2- 172.21.15.94:0/1510518157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c9050 0x55cbd05cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe921879640 1 -- 172.21.15.94:0/1510518157 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbd04f20b0 con 0x55cbd05c8690 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe921078640 1 --2- 172.21.15.94:0/1510518157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c9050 0x55cbd05cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe912ffd640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe9080a2150 0x7fe9080a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.954+0000 7fe92207a640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe9080a2150 0x7fe9080a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.955+0000 7fe921879640 1 --2- 172.21.15.94:0/1510518157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 0x55cbd05c8b10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe9180cbbe0 tx=0x7fe9180d6580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890d3f91e55e6e54 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.955+0000 7fe912ffd640 1 -- 172.21.15.94:0/3850365204 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe918014070 con 0x7fe91c100080 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.955+0000 7fe907fff640 1 -- 172.21.15.94:0/1510518157 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91809ac90 con 0x55cbd05c8690 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.955+0000 7fe907fff640 1 -- 172.21.15.94:0/1510518157 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe91809aa00 con 0x55cbd05c8690 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.955+0000 7fe907fff640 1 -- 172.21.15.94:0/1510518157 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91809aec0 con 0x55cbd05c8690 2024-02-17T11:57:58.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.956+0000 7fe926782740 1 -- 172.21.15.94:0/1510518157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 msgr2=0x55cbd05c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.956+0000 7fe92207a640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe9080a2150 0x7fe9080a4610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55cbd05c8590 tx=0x7fe914009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:58.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.956+0000 7fe926782740 1 --2- 172.21.15.94:0/1510518157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 0x55cbd05c8b10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe9180cbbe0 tx=0x7fe9180d6580 comp rx=0 tx=0).stop 2024-02-17T11:57:58.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.956+0000 7fe926782740 1 -- 172.21.15.94:0/1510518157 shutdown_connections 2024-02-17T11:57:58.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.956+0000 7fe926782740 1 --2- 172.21.15.94:0/1510518157 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c9050 0x55cbd05cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.956+0000 7fe926782740 1 --2- 172.21.15.94:0/1510518157 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 0x55cbd05c8b10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.956+0000 7fe926782740 1 --2- 172.21.15.94:0/1510518157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c74c0 0x55cbd05c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.956+0000 7fe926782740 1 -- 172.21.15.94:0/1510518157 >> 172.21.15.94:0/1510518157 conn(0x55cbd05c26a0 msgr2=0x55cbd05c2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:58.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.956+0000 7fe926782740 1 -- 172.21.15.94:0/1510518157 shutdown_connections 2024-02-17T11:57:58.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.956+0000 7fe926782740 1 -- 172.21.15.94:0/1510518157 wait complete. 2024-02-17T11:57:58.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.956+0000 7fe926782740 1 -- start start 2024-02-17T11:57:58.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.956+0000 7fe926782740 1 Processor -- start 2024-02-17T11:57:58.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe926782740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c74c0 0x55cbd05d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe926782740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 0x55cbd05d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe926782740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c9050 0x55cbd05d8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe926782740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd04d4f40 con 0x55cbd05c8690 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe926782740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd02034b0 con 0x55cbd05c9050 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe926782740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd04b5670 con 0x55cbd05c74c0 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe926782740 10 client.0 _ll_register_callbacks cb 0x55cbd0522f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe921078640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 0x55cbd05d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe921078640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 0x55cbd05d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53504/0 (socket says 172.21.15.94:53504) 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe921078640 1 -- 172.21.15.94:0/2507833238 learned_addr learned my addr 172.21.15.94:0/2507833238 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe921879640 1 --2- 172.21.15.94:0/2507833238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c74c0 0x55cbd05d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe92207a640 1 --2- 172.21.15.94:0/2507833238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c9050 0x55cbd05d8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe921078640 1 -- 172.21.15.94:0/2507833238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c74c0 msgr2=0x55cbd05d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe921078640 1 --2- 172.21.15.94:0/2507833238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c74c0 0x55cbd05d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe921078640 1 -- 172.21.15.94:0/2507833238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c9050 msgr2=0x55cbd05d8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:58.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe921078640 1 --2- 172.21.15.94:0/2507833238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c9050 0x55cbd05d8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe921078640 1 -- 172.21.15.94:0/2507833238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe91c19f870 con 0x55cbd05c8690 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe921078640 1 --2- 172.21.15.94:0/2507833238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 0x55cbd05d1bf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe90c005ab0 tx=0x7fe90c02cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a7a6e064df4af server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.957+0000 7fe9057fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.958+0000 7fe9057fa640 1 -- 172.21.15.94:0/2507833238 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe90c004040 con 0x55cbd05c8690 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.958+0000 7fe9057fa640 1 -- 172.21.15.94:0/2507833238 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe90c045070 con 0x55cbd05c8690 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.958+0000 7fe926782740 1 -- 172.21.15.94:0/2507833238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbd0509f40 con 0x55cbd05c8690 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.958+0000 7fe9057fa640 1 -- 172.21.15.94:0/2507833238 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe90c0041a0 con 0x55cbd05c8690 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.958+0000 7fe926782740 10 client.15393 fetch_fsmap learned FSMap version 21 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.958+0000 7fe926782740 1 -- 172.21.15.94:0/2507833238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55cbd052e320 con 0x55cbd05c8690 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.958+0000 7fe9057fa640 1 -- 172.21.15.94:0/2507833238 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cbd0509f40 con 0x55cbd05c8690 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.959+0000 7fe9057fa640 1 -- 172.21.15.94:0/2507833238 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe90c03c040 con 0x55cbd05c8690 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.959+0000 7fe926782740 10 client.15393 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.959+0000 7fe926782740 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.959+0000 7fe926782740 20 client.15393 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:58.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.959+0000 7fe926782740 1 --2- 172.21.15.94:0/2507833238 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cbd05dd900 0x55cbd05dfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.959+0000 7fe926782740 4 client.15393 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:57:58.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.959+0000 7fe926782740 1 -- 172.21.15.94:0/2507833238 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbd0526310 con 0x55cbd05dd900 2024-02-17T11:57:58.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.959+0000 7fe921879640 1 --2- 172.21.15.94:0/2507833238 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cbd05dd900 0x55cbd05dfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.959+0000 7fe9057fa640 10 client.15393 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:58.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.959+0000 7fe921879640 1 --2- 172.21.15.94:0/2507833238 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cbd05dd900 0x55cbd05dfd20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2926faf3d7afd265 server_cookie=626a812ef2e5d022 in_seq=0 out_seq=0 2024-02-17T11:57:58.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe9057fa640 1 -- 172.21.15.94:0/2507833238 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cbd0526310 con 0x55cbd05dd900 2024-02-17T11:57:58.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe9057fa640 10 client.15393 handle_command_reply: tid=0 2024-02-17T11:57:58.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe926782740 1 client.15393 shutdown 2024-02-17T11:57:58.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe926782740 1 -- 172.21.15.94:0/2507833238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 msgr2=0x55cbd05d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe926782740 1 --2- 172.21.15.94:0/2507833238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 0x55cbd05d1bf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe90c005ab0 tx=0x7fe90c02cd90 comp rx=0 tx=0).stop 2024-02-17T11:57:58.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe926782740 1 -- 172.21.15.94:0/2507833238 shutdown_connections 2024-02-17T11:57:58.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe926782740 1 --2- 172.21.15.94:0/2507833238 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cbd05dd900 0x55cbd05dfd20 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe926782740 1 --2- 172.21.15.94:0/2507833238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c9050 0x55cbd05d8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe9057fa640 0 client.15393 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:58.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe926782740 1 --2- 172.21.15.94:0/2507833238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c8690 0x55cbd05d1bf0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe926782740 1 --2- 172.21.15.94:0/2507833238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c74c0 0x55cbd05d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe926782740 1 -- 172.21.15.94:0/2507833238 >> 172.21.15.94:0/2507833238 conn(0x55cbd05c26a0 msgr2=0x55cbd05c18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:58.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe926782740 1 -- 172.21.15.94:0/2507833238 shutdown_connections 2024-02-17T11:57:58.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.960+0000 7fe926782740 1 -- 172.21.15.94:0/2507833238 wait complete. 2024-02-17T11:57:58.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.961+0000 7fe926782740 20 client.15393 trim_cache size 0 max 16384 2024-02-17T11:57:58.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe926782740 1 Processor -- start 2024-02-17T11:57:58.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe926782740 1 -- start start 2024-02-17T11:57:58.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe926782740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c74c0 0x55cbd05e4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe926782740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c8690 0x55cbd05d8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe926782740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 0x55cbd05d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe926782740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd05d0820 con 0x55cbd05c9050 2024-02-17T11:57:58.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe926782740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd0526310 con 0x55cbd05c74c0 2024-02-17T11:57:58.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe926782740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd0509f40 con 0x55cbd05c8690 2024-02-17T11:57:58.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe92207a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 0x55cbd05d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe92207a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 0x55cbd05d16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53516/0 (socket says 172.21.15.94:53516) 2024-02-17T11:57:58.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe92207a640 1 -- 172.21.15.94:0/2741845952 learned_addr learned my addr 172.21.15.94:0/2741845952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:58.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe921879640 1 --2- 172.21.15.94:0/2741845952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c74c0 0x55cbd05e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe921078640 1 --2- 172.21.15.94:0/2741845952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c8690 0x55cbd05d8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe92207a640 1 -- 172.21.15.94:0/2741845952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c8690 msgr2=0x55cbd05d8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:58.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe92207a640 1 --2- 172.21.15.94:0/2741845952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c8690 0x55cbd05d8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe92207a640 1 -- 172.21.15.94:0/2741845952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c74c0 msgr2=0x55cbd05e4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:58.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.991+0000 7fe92207a640 1 --2- 172.21.15.94:0/2741845952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c74c0 0x55cbd05e4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.992+0000 7fe92207a640 1 -- 172.21.15.94:0/2741845952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbd04cac00 con 0x55cbd05c9050 2024-02-17T11:57:58.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.992+0000 7fe921078640 1 --2- 172.21.15.94:0/2741845952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c8690 0x55cbd05d8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:58.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.992+0000 7fe92207a640 1 --2- 172.21.15.94:0/2741845952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 0x55cbd05d16b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe91401e550 tx=0x7fe914019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e770359ad1e14db1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:58.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.992+0000 7fe9077fe640 1 -- 172.21.15.94:0/2741845952 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe914024040 con 0x55cbd05c9050 2024-02-17T11:57:58.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.992+0000 7fe9077fe640 1 -- 172.21.15.94:0/2741845952 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe914011aa0 con 0x55cbd05c9050 2024-02-17T11:57:58.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.992+0000 7fe9077fe640 1 -- 172.21.15.94:0/2741845952 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe914016070 con 0x55cbd05c9050 2024-02-17T11:57:58.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.992+0000 7fe926782740 1 -- 172.21.15.94:0/2741845952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 msgr2=0x55cbd05d16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:58.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.992+0000 7fe926782740 1 --2- 172.21.15.94:0/2741845952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 0x55cbd05d16b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe91401e550 tx=0x7fe914019000 comp rx=0 tx=0).stop 2024-02-17T11:57:58.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 -- 172.21.15.94:0/2741845952 shutdown_connections 2024-02-17T11:57:58.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 --2- 172.21.15.94:0/2741845952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 0x55cbd05d16b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 --2- 172.21.15.94:0/2741845952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c8690 0x55cbd05d8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 --2- 172.21.15.94:0/2741845952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c74c0 0x55cbd05e4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 -- 172.21.15.94:0/2741845952 >> 172.21.15.94:0/2741845952 conn(0x55cbd05c26a0 msgr2=0x55cbd05cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:58.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 -- 172.21.15.94:0/2741845952 shutdown_connections 2024-02-17T11:57:58.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 -- 172.21.15.94:0/2741845952 wait complete. 2024-02-17T11:57:58.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 -- start start 2024-02-17T11:57:58.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 Processor -- start 2024-02-17T11:57:58.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c74c0 0x55cbd05d8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c8690 0x55cbd05d54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 0x55cbd05d59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:58.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd05d5ff0 con 0x55cbd05c9050 2024-02-17T11:57:58.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd0633df0 con 0x55cbd05c74c0 2024-02-17T11:57:58.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbd0633f30 con 0x55cbd05c8690 2024-02-17T11:57:58.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe926782740 10 client.0 _ll_register_callbacks cb 0x7fe9180d1af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:58.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe92207a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 0x55cbd05d59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe92207a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 0x55cbd05d59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53522/0 (socket says 172.21.15.94:53522) 2024-02-17T11:57:58.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe92207a640 1 -- 172.21.15.94:0/1986991537 learned_addr learned my addr 172.21.15.94:0/1986991537 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:58.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.993+0000 7fe921879640 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c74c0 0x55cbd05d8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe921078640 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c8690 0x55cbd05d54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:58.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe92207a640 1 -- 172.21.15.94:0/1986991537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c8690 msgr2=0x55cbd05d54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:58.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe92207a640 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c8690 0x55cbd05d54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe92207a640 1 -- 172.21.15.94:0/1986991537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c74c0 msgr2=0x55cbd05d8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:58.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe92207a640 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c74c0 0x55cbd05d8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:58.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe92207a640 1 -- 172.21.15.94:0/1986991537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe914004770 con 0x55cbd05c9050 2024-02-17T11:57:58.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe921879640 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c74c0 0x55cbd05d8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:57:58.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe92207a640 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 0x55cbd05d59e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe91400a010 tx=0x7fe914011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8755994bec59c69 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe9057fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe9057fa640 1 -- 172.21.15.94:0/1986991537 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe914016070 con 0x55cbd05c9050 2024-02-17T11:57:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe926782740 1 -- 172.21.15.94:0/1986991537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbd0555900 con 0x55cbd05c9050 2024-02-17T11:57:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe9057fa640 1 -- 172.21.15.94:0/1986991537 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fe914011880 con 0x55cbd05c9050 2024-02-17T11:57:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe9057fa640 1 -- 172.21.15.94:0/1986991537 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe914004990 con 0x55cbd05c9050 2024-02-17T11:57:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe926782740 10 client.15399 fetch_fsmap learned FSMap version 21 2024-02-17T11:57:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe926782740 1 -- 172.21.15.94:0/1986991537 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55cbd04f0910 con 0x55cbd05c9050 2024-02-17T11:57:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.994+0000 7fe9057fa640 1 -- 172.21.15.94:0/1986991537 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cbd0555900 con 0x55cbd05c9050 2024-02-17T11:57:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.995+0000 7fe9057fa640 1 -- 172.21.15.94:0/1986991537 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe914005200 con 0x55cbd05c9050 2024-02-17T11:57:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.995+0000 7fe926782740 10 client.15399 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:57:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.995+0000 7fe926782740 10 client.15399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:57:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.995+0000 7fe926782740 20 client.15399 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:58.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.995+0000 7fe926782740 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cbd06357c0 0x55cbd0637be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.995+0000 7fe926782740 4 client.15399 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T11:57:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.995+0000 7fe926782740 1 -- 172.21.15.94:0/1986991537 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cbd022f860 con 0x55cbd06357c0 2024-02-17T11:57:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.995+0000 7fe921879640 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cbd06357c0 0x55cbd0637be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.995+0000 7fe9057fa640 10 client.15399 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.995+0000 7fe921879640 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cbd06357c0 0x55cbd0637be0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40e5ff11f402f8b9 server_cookie=bf78889306bdc6bb in_seq=0 out_seq=0 2024-02-17T11:57:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.996+0000 7fe9057fa640 1 -- 172.21.15.94:0/1986991537 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cbd022f860 con 0x55cbd06357c0 2024-02-17T11:57:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.996+0000 7fe9057fa640 10 client.15399 handle_command_reply: tid=0 2024-02-17T11:57:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.996+0000 7fe926782740 1 client.15399 shutdown 2024-02-17T11:57:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe926782740 1 -- 172.21.15.94:0/1986991537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 msgr2=0x55cbd05d59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe926782740 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 0x55cbd05d59e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe91400a010 tx=0x7fe914011d70 comp rx=0 tx=0).stop 2024-02-17T11:57:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe926782740 1 -- 172.21.15.94:0/1986991537 shutdown_connections 2024-02-17T11:57:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe926782740 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cbd06357c0 0x55cbd0637be0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe926782740 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cbd05c9050 0x55cbd05d59e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe9057fa640 0 client.15399 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe926782740 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cbd05c8690 0x55cbd05d54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe926782740 1 --2- 172.21.15.94:0/1986991537 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cbd05c74c0 0x55cbd05d8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe926782740 1 -- 172.21.15.94:0/1986991537 >> 172.21.15.94:0/1986991537 conn(0x55cbd05c26a0 msgr2=0x55cbd05c5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe926782740 1 -- 172.21.15.94:0/1986991537 shutdown_connections 2024-02-17T11:57:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe926782740 1 -- 172.21.15.94:0/1986991537 wait complete. 2024-02-17T11:57:59.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe926782740 20 client.15399 trim_cache size 0 max 16384 2024-02-17T11:57:59.002 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:57:59.003 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T11:57:59.003 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "575b04f9-6d61-4fff-889c-47af4e5245ab", 2024-02-17T11:57:59.003 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T11:57:59.003 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:57:59.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe923add640 1 -- 172.21.15.94:0/3850365204 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe9080a2150 msgr2=0x7fe9080a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe923add640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe9080a2150 0x7fe9080a4610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55cbd05c8590 tx=0x7fe914009500 comp rx=0 tx=0).stop 2024-02-17T11:57:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe923add640 1 -- 172.21.15.94:0/3850365204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe91c100080 msgr2=0x7fe91c197290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe923add640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe91c100080 0x7fe91c197290 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe91800e9c0 tx=0x7fe91800ee80 comp rx=0 tx=0).stop 2024-02-17T11:57:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe923add640 1 -- 172.21.15.94:0/3850365204 shutdown_connections 2024-02-17T11:57:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe923add640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fe9080a2150 0x7fe9080a4610 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe923add640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe91c101c30 0x7fe91c19e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe923add640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe91c101270 0x7fe91c1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe923add640 1 --2- 172.21.15.94:0/3850365204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe91c100080 0x7fe91c197290 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.997+0000 7fe923add640 1 -- 172.21.15.94:0/3850365204 >> 172.21.15.94:0/3850365204 conn(0x7fe91c0fb610 msgr2=0x7fe91c0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.998+0000 7fe923add640 1 -- 172.21.15.94:0/3850365204 shutdown_connections 2024-02-17T11:57:59.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:58.998+0000 7fe923add640 1 -- 172.21.15.94:0/3850365204 wait complete. 2024-02-17T11:57:59.007 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T11:57:59.007 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T11:57:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 -- 172.21.15.94:0/2564805643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab58101c30 msgr2=0x7fab581084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 --2- 172.21.15.94:0/2564805643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab58101c30 0x7fab581084c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fab50009ca0 tx=0x7fab5002c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 -- 172.21.15.94:0/2564805643 shutdown_connections 2024-02-17T11:57:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 --2- 172.21.15.94:0/2564805643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab58101c30 0x7fab581084c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 --2- 172.21.15.94:0/2564805643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab58101270 0x7fab581016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 --2- 172.21.15.94:0/2564805643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab58100080 0x7fab581004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 -- 172.21.15.94:0/2564805643 >> 172.21.15.94:0/2564805643 conn(0x7fab580fb5f0 msgr2=0x7fab580fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 -- 172.21.15.94:0/2564805643 shutdown_connections 2024-02-17T11:57:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 -- 172.21.15.94:0/2564805643 wait complete. 2024-02-17T11:57:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 Processor -- start 2024-02-17T11:57:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 -- start start 2024-02-17T11:57:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab58100080 0x7fab58068e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab58101270 0x7fab5819bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab58101c30 0x7fab5819e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5819ece0 con 0x7fab58100080 2024-02-17T11:57:59.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5819ee50 con 0x7fab58101c30 2024-02-17T11:57:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.114+0000 7fab5da4e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5819f010 con 0x7fab58101270 2024-02-17T11:57:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab56ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab58100080 0x7fab58068e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab56ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab58100080 0x7fab58068e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53540/0 (socket says 172.21.15.94:53540) 2024-02-17T11:57:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab56ffd640 1 -- 172.21.15.94:0/4203521250 learned_addr learned my addr 172.21.15.94:0/4203521250 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab567fc640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab58101270 0x7fab5819bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab577fe640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab58101c30 0x7fab5819e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab56ffd640 1 -- 172.21.15.94:0/4203521250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab58101270 msgr2=0x7fab5819bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab56ffd640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab58101270 0x7fab5819bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab56ffd640 1 -- 172.21.15.94:0/4203521250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab58101c30 msgr2=0x7fab5819e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab56ffd640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab58101c30 0x7fab5819e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab56ffd640 1 -- 172.21.15.94:0/4203521250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab48009490 con 0x7fab58100080 2024-02-17T11:57:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab577fe640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab58101c30 0x7fab5819e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:59.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab56ffd640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab58100080 0x7fab58068e50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00e9c0 tx=0x7fab4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:59.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab5ca4c640 1 -- 172.21.15.94:0/4203521250 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c009700 con 0x7fab58100080 2024-02-17T11:57:59.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab5da4e640 1 -- 172.21.15.94:0/4203521250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab50009900 con 0x7fab58100080 2024-02-17T11:57:59.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab5ca4c640 1 -- 172.21.15.94:0/4203521250 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab4c00cd80 con 0x7fab58100080 2024-02-17T11:57:59.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab5ca4c640 1 -- 172.21.15.94:0/4203521250 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c010430 con 0x7fab58100080 2024-02-17T11:57:59.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.115+0000 7fab5da4e640 1 -- 172.21.15.94:0/4203521250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5819f610 con 0x7fab58100080 2024-02-17T11:57:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab5ca4c640 1 -- 172.21.15.94:0/4203521250 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 19) v1 ==== 136819+0+0 (secure 0 0 0) 0x7fab4c003680 con 0x7fab58100080 2024-02-17T11:57:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab60628740 1 Processor -- start 2024-02-17T11:57:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab60628740 1 -- start start 2024-02-17T11:57:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab60628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5e96e0 0x55734e5e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab60628740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ea0d0 0x55734e5eebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab60628740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ef240 0x55734e5f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab60628740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734e555280 con 0x55734e5ea0d0 2024-02-17T11:57:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab60628740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734e5527b0 con 0x55734e5ef240 2024-02-17T11:57:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab60628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734e54df90 con 0x55734e5e96e0 2024-02-17T11:57:59.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab56ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ea0d0 0x55734e5eebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab56ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ea0d0 0x55734e5eebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53544/0 (socket says 172.21.15.94:53544) 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab56ffd640 1 -- 172.21.15.94:0/1572979096 learned_addr learned my addr 172.21.15.94:0/1572979096 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab5ca4c640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fab400a2150 0x7fab400a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab577fe640 1 --2- 172.21.15.94:0/1572979096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5e96e0 0x55734e5e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab567fc640 1 --2- 172.21.15.94:0/1572979096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ef240 0x55734e5f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab577fe640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fab400a2150 0x7fab400a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.118+0000 7fab5ca4c640 1 -- 172.21.15.94:0/4203521250 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fab4c014070 con 0x7fab58100080 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.119+0000 7fab56ffd640 1 -- 172.21.15.94:0/1572979096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5e96e0 msgr2=0x55734e5e9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.119+0000 7fab56ffd640 1 --2- 172.21.15.94:0/1572979096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5e96e0 0x55734e5e9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.119+0000 7fab56ffd640 1 -- 172.21.15.94:0/1572979096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ef240 msgr2=0x55734e5f16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.119+0000 7fab56ffd640 1 --2- 172.21.15.94:0/1572979096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ef240 0x55734e5f16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.119+0000 7fab56ffd640 1 -- 172.21.15.94:0/1572979096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55734e50c570 con 0x55734e5ea0d0 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.119+0000 7fab567fc640 1 --2- 172.21.15.94:0/1572979096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ef240 0x55734e5f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:59.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.119+0000 7fab56ffd640 1 --2- 172.21.15.94:0/1572979096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ea0d0 0x55734e5eebd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0cbc50 tx=0x7fab4c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba54b76c365c572f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.119+0000 7fab3dffb640 1 -- 172.21.15.94:0/1572979096 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0d1870 con 0x55734e5ea0d0 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.119+0000 7fab3dffb640 1 -- 172.21.15.94:0/1572979096 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab4c09b600 con 0x55734e5ea0d0 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.119+0000 7fab577fe640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fab400a2150 0x7fab400a4610 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fab5002c630 tx=0x7fab5002c0f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.119+0000 7fab3dffb640 1 -- 172.21.15.94:0/1572979096 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0cc040 con 0x55734e5ea0d0 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 -- 172.21.15.94:0/1572979096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ea0d0 msgr2=0x55734e5eebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 --2- 172.21.15.94:0/1572979096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ea0d0 0x55734e5eebd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0cbc50 tx=0x7fab4c0035f0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 -- 172.21.15.94:0/1572979096 shutdown_connections 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 --2- 172.21.15.94:0/1572979096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ef240 0x55734e5f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 --2- 172.21.15.94:0/1572979096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ea0d0 0x55734e5eebd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 --2- 172.21.15.94:0/1572979096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5e96e0 0x55734e5e9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 -- 172.21.15.94:0/1572979096 >> 172.21.15.94:0/1572979096 conn(0x55734e5e07f0 msgr2=0x55734e5e0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 -- 172.21.15.94:0/1572979096 shutdown_connections 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 -- 172.21.15.94:0/1572979096 wait complete. 2024-02-17T11:57:59.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 -- start start 2024-02-17T11:57:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 Processor -- start 2024-02-17T11:57:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 0x55734e5f3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ea0d0 0x55734e5f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ef240 0x55734e5faba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734e221430 con 0x55734e5e96e0 2024-02-17T11:57:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734e1dde00 con 0x55734e5ef240 2024-02-17T11:57:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734e4d2fd0 con 0x55734e5ea0d0 2024-02-17T11:57:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.120+0000 7fab60628740 10 client.0 _ll_register_callbacks cb 0x55734e528fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab56ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 0x55734e5f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab56ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 0x55734e5f3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53560/0 (socket says 172.21.15.94:53560) 2024-02-17T11:57:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab56ffd640 1 -- 172.21.15.94:0/2422192898 learned_addr learned my addr 172.21.15.94:0/2422192898 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab577fe640 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ef240 0x55734e5faba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab567fc640 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ea0d0 0x55734e5f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab56ffd640 1 -- 172.21.15.94:0/2422192898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ea0d0 msgr2=0x55734e5f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab56ffd640 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ea0d0 0x55734e5f3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab56ffd640 1 -- 172.21.15.94:0/2422192898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ef240 msgr2=0x55734e5faba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab56ffd640 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ef240 0x55734e5faba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab56ffd640 1 -- 172.21.15.94:0/2422192898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5819f610 con 0x55734e5e96e0 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab567fc640 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ea0d0 0x55734e5f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab577fe640 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ef240 0x55734e5faba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab56ffd640 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 0x55734e5f3690 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fab4c097360 tx=0x7fab4c096b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d49f758510fdad server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab36ffd640 1 -- 172.21.15.94:0/2422192898 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c09a550 con 0x55734e5e96e0 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab36ffd640 1 -- 172.21.15.94:0/2422192898 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fab4c09af00 con 0x55734e5e96e0 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab60628740 1 -- 172.21.15.94:0/2422192898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55734e24d7e0 con 0x55734e5e96e0 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab36ffd640 1 -- 172.21.15.94:0/2422192898 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab4c09a6b0 con 0x55734e5e96e0 2024-02-17T11:57:59.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab60628740 10 client.15411 fetch_fsmap learned FSMap version 21 2024-02-17T11:57:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.121+0000 7fab60628740 1 -- 172.21.15.94:0/2422192898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55734e54c610 con 0x55734e5e96e0 2024-02-17T11:57:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.122+0000 7fab36ffd640 1 -- 172.21.15.94:0/2422192898 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55734e24d7e0 con 0x55734e5e96e0 2024-02-17T11:57:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.122+0000 7fab36ffd640 1 -- 172.21.15.94:0/2422192898 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fab4c01f060 con 0x55734e5e96e0 2024-02-17T11:57:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.122+0000 7fab60628740 10 client.15411 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:57:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.122+0000 7fab60628740 10 client.15411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:57:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.122+0000 7fab60628740 20 client.15411 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.122+0000 7fab60628740 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55734e5ff430 0x55734e601840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.122+0000 7fab60628740 4 client.15411 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:57:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.122+0000 7fab60628740 1 -- 172.21.15.94:0/2422192898 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55734e28a720 con 0x55734e5ff430 2024-02-17T11:57:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.122+0000 7fab567fc640 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55734e5ff430 0x55734e601840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.122+0000 7fab36ffd640 10 client.15411 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.122+0000 7fab567fc640 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55734e5ff430 0x55734e601840 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3cb1ea7a969fca7 server_cookie=4deb39bd648bffcd in_seq=0 out_seq=0 2024-02-17T11:57:59.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab36ffd640 1 -- 172.21.15.94:0/2422192898 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55734e28a720 con 0x55734e5ff430 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab36ffd640 10 client.15411 handle_command_reply: tid=0 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab60628740 1 client.15411 shutdown 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab60628740 1 -- 172.21.15.94:0/2422192898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 msgr2=0x55734e5f3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab60628740 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 0x55734e5f3690 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fab4c097360 tx=0x7fab4c096b70 comp rx=0 tx=0).stop 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab60628740 1 -- 172.21.15.94:0/2422192898 shutdown_connections 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab60628740 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55734e5ff430 0x55734e601840 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab60628740 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ef240 0x55734e5faba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab36ffd640 0 client.15411 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab60628740 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ea0d0 0x55734e5f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab60628740 1 --2- 172.21.15.94:0/2422192898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 0x55734e5f3690 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab60628740 1 -- 172.21.15.94:0/2422192898 >> 172.21.15.94:0/2422192898 conn(0x55734e5e07f0 msgr2=0x55734e5dfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab60628740 1 -- 172.21.15.94:0/2422192898 shutdown_connections 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab60628740 1 -- 172.21.15.94:0/2422192898 wait complete. 2024-02-17T11:57:59.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.123+0000 7fab60628740 20 client.15411 trim_cache size 0 max 16384 2024-02-17T11:57:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab60628740 1 Processor -- start 2024-02-17T11:57:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab60628740 1 -- start start 2024-02-17T11:57:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab60628740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 0x55734e5f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab60628740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ea0d0 0x55734e5fab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab60628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ef240 0x55734e5f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab60628740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734e5dbeb0 con 0x55734e5e96e0 2024-02-17T11:57:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab60628740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734e28a750 con 0x55734e5ea0d0 2024-02-17T11:57:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab60628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734e5ea590 con 0x55734e5ef240 2024-02-17T11:57:59.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab56ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 0x55734e5f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab56ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 0x55734e5f32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53564/0 (socket says 172.21.15.94:53564) 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab56ffd640 1 -- 172.21.15.94:0/3292815596 learned_addr learned my addr 172.21.15.94:0/3292815596 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab577fe640 1 --2- 172.21.15.94:0/3292815596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ef240 0x55734e5f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab56ffd640 1 -- 172.21.15.94:0/3292815596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ef240 msgr2=0x55734e5f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab56ffd640 1 --2- 172.21.15.94:0/3292815596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ef240 0x55734e5f37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab56ffd640 1 -- 172.21.15.94:0/3292815596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ea0d0 msgr2=0x55734e5fab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab567fc640 1 --2- 172.21.15.94:0/3292815596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ea0d0 0x55734e5fab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab56ffd640 1 --2- 172.21.15.94:0/3292815596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ea0d0 0x55734e5fab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab56ffd640 1 -- 172.21.15.94:0/3292815596 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55734e4e8ac0 con 0x55734e5e96e0 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.138+0000 7fab577fe640 1 --2- 172.21.15.94:0/3292815596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ef240 0x55734e5f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.139+0000 7fab56ffd640 1 --2- 172.21.15.94:0/3292815596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 0x55734e5f32a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0991b0 tx=0x7fab4c096a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39e4c164cb4fbf9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.139+0000 7fab3d7fa640 1 -- 172.21.15.94:0/3292815596 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0a0370 con 0x55734e5e96e0 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.139+0000 7fab3d7fa640 1 -- 172.21.15.94:0/3292815596 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fab4c0a3710 con 0x55734e5e96e0 2024-02-17T11:57:59.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.139+0000 7fab3d7fa640 1 -- 172.21.15.94:0/3292815596 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4c0b9470 con 0x55734e5e96e0 2024-02-17T11:57:59.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.139+0000 7fab60628740 1 -- 172.21.15.94:0/3292815596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 msgr2=0x55734e5f32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.139+0000 7fab60628740 1 --2- 172.21.15.94:0/3292815596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 0x55734e5f32a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0991b0 tx=0x7fab4c096a80 comp rx=0 tx=0).stop 2024-02-17T11:57:59.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.139+0000 7fab60628740 1 -- 172.21.15.94:0/3292815596 shutdown_connections 2024-02-17T11:57:59.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.139+0000 7fab60628740 1 --2- 172.21.15.94:0/3292815596 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ef240 0x55734e5f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.139+0000 7fab60628740 1 --2- 172.21.15.94:0/3292815596 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5ea0d0 0x55734e5fab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.139+0000 7fab60628740 1 --2- 172.21.15.94:0/3292815596 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5e96e0 0x55734e5f32a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.139+0000 7fab60628740 1 -- 172.21.15.94:0/3292815596 >> 172.21.15.94:0/3292815596 conn(0x55734e5e07f0 msgr2=0x55734e5eb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:59.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.139+0000 7fab60628740 1 -- 172.21.15.94:0/3292815596 shutdown_connections 2024-02-17T11:57:59.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.139+0000 7fab60628740 1 -- 172.21.15.94:0/3292815596 wait complete. 2024-02-17T11:57:59.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab60628740 1 -- start start 2024-02-17T11:57:59.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab60628740 1 Processor -- start 2024-02-17T11:57:59.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab60628740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5e96e0 0x55734e5fa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab60628740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ea0d0 0x55734e5f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab60628740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ef240 0x55734e5f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab60628740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734e5f9b10 con 0x55734e5ef240 2024-02-17T11:57:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab60628740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734e5f9c80 con 0x55734e5e96e0 2024-02-17T11:57:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab60628740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55734e5f9e40 con 0x55734e5ea0d0 2024-02-17T11:57:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab60628740 10 client.0 _ll_register_callbacks cb 0x7fab4c097220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:57:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ef240 0x55734e5f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ef240 0x55734e5f94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53568/0 (socket says 172.21.15.94:53568) 2024-02-17T11:57:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab577fe640 1 -- 172.21.15.94:0/2169668304 learned_addr learned my addr 172.21.15.94:0/2169668304 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab567fc640 1 --2- 172.21.15.94:0/2169668304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ea0d0 0x55734e5f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.140+0000 7fab56ffd640 1 --2- 172.21.15.94:0/2169668304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5e96e0 0x55734e5fa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab577fe640 1 -- 172.21.15.94:0/2169668304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ea0d0 msgr2=0x55734e5f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab577fe640 1 --2- 172.21.15.94:0/2169668304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ea0d0 0x55734e5f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab577fe640 1 -- 172.21.15.94:0/2169668304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5e96e0 msgr2=0x55734e5fa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab577fe640 1 --2- 172.21.15.94:0/2169668304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5e96e0 0x55734e5fa980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab577fe640 1 -- 172.21.15.94:0/2169668304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55734e54c610 con 0x55734e5ef240 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab577fe640 1 --2- 172.21.15.94:0/2169668304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ef240 0x55734e5f94d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fab50036000 tx=0x7fab500095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21600495c84c7ac server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab36ffd640 1 -- 172.21.15.94:0/2169668304 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab50003b80 con 0x55734e5ef240 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab36ffd640 1 -- 172.21.15.94:0/2169668304 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7fab50048040 con 0x55734e5ef240 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab60628740 1 -- 172.21.15.94:0/2169668304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55734e2d9890 con 0x55734e5ef240 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab36ffd640 1 -- 172.21.15.94:0/2169668304 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab50003ce0 con 0x55734e5ef240 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab60628740 10 client.15417 fetch_fsmap learned FSMap version 21 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab60628740 1 -- 172.21.15.94:0/2169668304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55734e50e6a0 con 0x55734e5ef240 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab36ffd640 1 -- 172.21.15.94:0/2169668304 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55734e2d9890 con 0x55734e5ef240 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab36ffd640 1 -- 172.21.15.94:0/2169668304 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fab5003e070 con 0x55734e5ef240 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab60628740 10 client.15417 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:57:59.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab60628740 10 client.15417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab60628740 20 client.15417 populate_metadata read hostname 'smithi094' 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.141+0000 7fab60628740 1 --2- 172.21.15.94:0/2169668304 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55734e6572b0 0x55734e6596d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab60628740 4 client.15417 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab60628740 1 -- 172.21.15.94:0/2169668304 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55734e23c4e0 con 0x55734e6572b0 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab56ffd640 1 --2- 172.21.15.94:0/2169668304 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55734e6572b0 0x55734e6596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab36ffd640 10 client.15417 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab56ffd640 1 --2- 172.21.15.94:0/2169668304 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55734e6572b0 0x55734e6596d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf6b1981c31c58a server_cookie=e9f558212d081ea8 in_seq=0 out_seq=0 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab36ffd640 1 -- 172.21.15.94:0/2169668304 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55734e23c4e0 con 0x55734e6572b0 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab36ffd640 10 client.15417 handle_command_reply: tid=0 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab60628740 1 client.15417 shutdown 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab60628740 1 -- 172.21.15.94:0/2169668304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ef240 msgr2=0x55734e5f94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab60628740 1 --2- 172.21.15.94:0/2169668304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ef240 0x55734e5f94d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fab50036000 tx=0x7fab500095c0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab60628740 1 -- 172.21.15.94:0/2169668304 shutdown_connections 2024-02-17T11:57:59.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab60628740 1 --2- 172.21.15.94:0/2169668304 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55734e6572b0 0x55734e6596d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab60628740 1 --2- 172.21.15.94:0/2169668304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55734e5ef240 0x55734e5f94d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab60628740 1 --2- 172.21.15.94:0/2169668304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55734e5ea0d0 0x55734e5f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab36ffd640 0 client.15417 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:57:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab60628740 1 --2- 172.21.15.94:0/2169668304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55734e5e96e0 0x55734e5fa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.142+0000 7fab60628740 1 -- 172.21.15.94:0/2169668304 >> 172.21.15.94:0/2169668304 conn(0x55734e5e07f0 msgr2=0x55734e5f3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab60628740 1 -- 172.21.15.94:0/2169668304 shutdown_connections 2024-02-17T11:57:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab60628740 1 -- 172.21.15.94:0/2169668304 wait complete. 2024-02-17T11:57:59.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab60628740 20 client.15417 trim_cache size 0 max 16384 2024-02-17T11:57:59.148 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:57:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T11:57:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T11:57:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "575b04f9-6d61-4fff-889c-47af4e5245ab": { 2024-02-17T11:57:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T11:57:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "575b04f9-6d61-4fff-889c-47af4e5245ab", 2024-02-17T11:57:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T11:57:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T11:57:59.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T11:57:59.149 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:57:59.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab5da4e640 1 -- 172.21.15.94:0/4203521250 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fab400a2150 msgr2=0x7fab400a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab5da4e640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fab400a2150 0x7fab400a4610 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fab5002c630 tx=0x7fab5002c0f0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab5da4e640 1 -- 172.21.15.94:0/4203521250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab58100080 msgr2=0x7fab58068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab5da4e640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab58100080 0x7fab58068e50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00e9c0 tx=0x7fab4c00ee80 comp rx=0 tx=0).stop 2024-02-17T11:57:59.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab5da4e640 1 -- 172.21.15.94:0/4203521250 shutdown_connections 2024-02-17T11:57:59.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab5da4e640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fab400a2150 0x7fab400a4610 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab5da4e640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fab58101c30 0x7fab5819e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab5da4e640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fab58101270 0x7fab5819bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab5da4e640 1 --2- 172.21.15.94:0/4203521250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fab58100080 0x7fab58068e50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab5da4e640 1 -- 172.21.15.94:0/4203521250 >> 172.21.15.94:0/4203521250 conn(0x7fab580fb5f0 msgr2=0x7fab58104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:59.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab5da4e640 1 -- 172.21.15.94:0/4203521250 shutdown_connections 2024-02-17T11:57:59.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.143+0000 7fab5da4e640 1 -- 172.21.15.94:0/4203521250 wait complete. 2024-02-17T11:57:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:58 smithi189 ceph-mon[36588]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:58 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/962177671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:58 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/211458928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:59.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:58 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1495762682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:59.152 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:58 smithi094 ceph-mon[32499]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:59.152 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:58 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/962177671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:59.152 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:58 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/211458928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:59.152 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:58 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1495762682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:59.153 INFO:tasks.cephfs.filesystem:scrub status for tag:575b04f9-6d61-4fff-889c-47af4e5245ab - {'path': '/', 'tag': '575b04f9-6d61-4fff-889c-47af4e5245ab', 'options': 'recursive,force'} 2024-02-17T11:57:59.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:58 smithi125 ceph-mon[36819]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-17T11:57:59.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:58 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/962177671' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T11:57:59.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:58 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/211458928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T11:57:59.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:58 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1495762682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:57:59.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.202+0000 7fadd2bb5640 1 -- 172.21.15.94:0/523503502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fadcc10cb70 con 0x7fadcc101c80 2024-02-17T11:57:59.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.800+0000 7fadc9ffb640 1 -- 172.21.15.94:0/523503502 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v20) v1 ==== 92+0+0 (secure 0 0 0) 0x7fadc009a4f0 con 0x7fadcc101c80 2024-02-17T11:57:59.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.803+0000 7fadd2bb5640 1 -- 172.21.15.94:0/523503502 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadbc0a2280 msgr2=0x7fadbc0a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.803+0000 7fadd2bb5640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadbc0a2280 0x7fadbc0a4740 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fadb8006010 tx=0x7fadb8005be0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.803+0000 7fadd2bb5640 1 -- 172.21.15.94:0/523503502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadcc101c80 msgr2=0x7fadcc197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.803+0000 7fadd2bb5640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadcc101c80 0x7fadcc197140 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c920 tx=0x7fadc000cde0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.803+0000 7fadd2bb5640 1 -- 172.21.15.94:0/523503502 shutdown_connections 2024-02-17T11:57:59.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.803+0000 7fadd2bb5640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7fadbc0a2280 0x7fadbc0a4740 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.803+0000 7fadd2bb5640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadcc106ee0 0x7fadcc19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.803+0000 7fadd2bb5640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadcc1045b0 0x7fadcc197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.803+0000 7fadd2bb5640 1 --2- 172.21.15.94:0/523503502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadcc101c80 0x7fadcc197140 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.803+0000 7fadd2bb5640 1 -- 172.21.15.94:0/523503502 >> 172.21.15.94:0/523503502 conn(0x7fadcc0fb610 msgr2=0x7fadcc1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:59.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.805+0000 7fadd2bb5640 1 -- 172.21.15.94:0/523503502 shutdown_connections 2024-02-17T11:57:59.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.805+0000 7fadd2bb5640 1 -- 172.21.15.94:0/523503502 wait complete. 2024-02-17T11:57:59.820 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-02-17T11:57:59.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.944+0000 7faaeafd7640 1 -- 172.21.15.94:0/2611084160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faae41000a0 msgr2=0x7faae41004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.944+0000 7faaeafd7640 1 --2- 172.21.15.94:0/2611084160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faae41000a0 0x7faae41004c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faae0009900 tx=0x7faae002bc30 comp rx=0 tx=0).stop 2024-02-17T11:57:59.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.944+0000 7faaeafd7640 1 -- 172.21.15.94:0/2611084160 shutdown_connections 2024-02-17T11:57:59.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.944+0000 7faaeafd7640 1 --2- 172.21.15.94:0/2611084160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faae4101c50 0x7faae41084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.944+0000 7faaeafd7640 1 --2- 172.21.15.94:0/2611084160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faae4101290 0x7faae4101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.944+0000 7faaeafd7640 1 --2- 172.21.15.94:0/2611084160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faae41000a0 0x7faae41004c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.944+0000 7faaeafd7640 1 -- 172.21.15.94:0/2611084160 >> 172.21.15.94:0/2611084160 conn(0x7faae40fb610 msgr2=0x7faae40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.944+0000 7faaeafd7640 1 -- 172.21.15.94:0/2611084160 shutdown_connections 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.944+0000 7faaeafd7640 1 -- 172.21.15.94:0/2611084160 wait complete. 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faaeafd7640 1 Processor -- start 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faaeafd7640 1 -- start start 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faaeafd7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faae41000a0 0x7faae4069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faaeafd7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faae4101290 0x7faae419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faaeafd7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faae4101c50 0x7faae419e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faaeafd7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae419ed00 con 0x7faae4101290 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faaeafd7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae419ee70 con 0x7faae41000a0 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faaeafd7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae419f030 con 0x7faae4101c50 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faadbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faae4101290 0x7faae419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faadbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faae4101290 0x7faae419bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53612/0 (socket says 172.21.15.94:53612) 2024-02-17T11:57:59.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faadbfff640 1 -- 172.21.15.94:0/3090431852 learned_addr learned my addr 172.21.15.94:0/3090431852 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:57:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faae8d73640 1 --2- 172.21.15.94:0/3090431852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faae41000a0 0x7faae4069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faae9574640 1 --2- 172.21.15.94:0/3090431852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faae4101c50 0x7faae419e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:57:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faadbfff640 1 -- 172.21.15.94:0/3090431852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faae4101c50 msgr2=0x7faae419e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faadbfff640 1 --2- 172.21.15.94:0/3090431852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faae4101c50 0x7faae419e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faadbfff640 1 -- 172.21.15.94:0/3090431852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faae41000a0 msgr2=0x7faae4069370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:57:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faadbfff640 1 --2- 172.21.15.94:0/3090431852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faae41000a0 0x7faae4069370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:57:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faadbfff640 1 -- 172.21.15.94:0/3090431852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae0009560 con 0x7faae4101290 2024-02-17T11:57:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.945+0000 7faadbfff640 1 --2- 172.21.15.94:0/3090431852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faae4101290 0x7faae419bf00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faad400d950 tx=0x7faad400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:57:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.946+0000 7faad9ffb640 1 -- 172.21.15.94:0/3090431852 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4009a30 con 0x7faae4101290 2024-02-17T11:57:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.946+0000 7faad9ffb640 1 -- 172.21.15.94:0/3090431852 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faad4009b90 con 0x7faae4101290 2024-02-17T11:57:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.946+0000 7faaeafd7640 1 -- 172.21.15.94:0/3090431852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae419f2b0 con 0x7faae4101290 2024-02-17T11:57:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.946+0000 7faad9ffb640 1 -- 172.21.15.94:0/3090431852 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad4004a00 con 0x7faae4101290 2024-02-17T11:57:59.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.946+0000 7faaeafd7640 1 -- 172.21.15.94:0/3090431852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae419f7c0 con 0x7faae4101290 2024-02-17T11:57:59.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.947+0000 7faaeafd7640 1 -- 172.21.15.94:0/3090431852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae4107360 con 0x7faae4101290 2024-02-17T11:57:59.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.949+0000 7faad9ffb640 1 -- 172.21.15.94:0/3090431852 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) v1 ==== 136836+0+0 (secure 0 0 0) 0x7faad4009d00 con 0x7faae4101290 2024-02-17T11:57:59.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.949+0000 7faad9ffb640 1 --2- 172.21.15.94:0/3090431852 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faacc0a2250 0x7faacc0a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:57:59.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.949+0000 7faae8d73640 1 -- 172.21.15.94:0/3090431852 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faacc0a2250 msgr2=0x7faacc0a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1808883811 2024-02-17T11:57:59.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.949+0000 7faad9ffb640 1 -- 172.21.15.94:0/3090431852 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7faad40cd950 con 0x7faae4101290 2024-02-17T11:57:59.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.949+0000 7faae8d73640 1 --2- 172.21.15.94:0/3090431852 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faacc0a2250 0x7faacc0a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T11:57:59.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:57:59.953+0000 7faad9ffb640 1 -- 172.21.15.94:0/3090431852 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faad40967c0 con 0x7faae4101290 2024-02-17T11:58:00.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.132+0000 7faaeafd7640 1 -- 172.21.15.94:0/3090431852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7faae410bab0 con 0x7faae4101290 2024-02-17T11:58:00.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.136+0000 7faad9ffb640 1 -- 172.21.15.94:0/3090431852 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) v1 ==== 138+0+0 (secure 0 0 0) 0x7faad4099e80 con 0x7faae4101290 2024-02-17T11:58:00.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.139+0000 7faaeafd7640 1 -- 172.21.15.94:0/3090431852 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faacc0a2250 msgr2=0x7faacc0a4710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T11:58:00.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.139+0000 7faaeafd7640 1 --2- 172.21.15.94:0/3090431852 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faacc0a2250 0x7faacc0a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.139+0000 7faaeafd7640 1 -- 172.21.15.94:0/3090431852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faae4101290 msgr2=0x7faae419bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:00.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.139+0000 7faaeafd7640 1 --2- 172.21.15.94:0/3090431852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faae4101290 0x7faae419bf00 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faad400d950 tx=0x7faad400de10 comp rx=0 tx=0).stop 2024-02-17T11:58:00.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.139+0000 7faaeafd7640 1 -- 172.21.15.94:0/3090431852 shutdown_connections 2024-02-17T11:58:00.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.139+0000 7faaeafd7640 1 --2- 172.21.15.94:0/3090431852 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7faacc0a2250 0x7faacc0a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.139+0000 7faaeafd7640 1 --2- 172.21.15.94:0/3090431852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faae4101c50 0x7faae419e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.139+0000 7faaeafd7640 1 --2- 172.21.15.94:0/3090431852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faae4101290 0x7faae419bf00 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.139+0000 7faaeafd7640 1 --2- 172.21.15.94:0/3090431852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faae41000a0 0x7faae4069370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.139+0000 7faaeafd7640 1 -- 172.21.15.94:0/3090431852 >> 172.21.15.94:0/3090431852 conn(0x7faae40fb610 msgr2=0x7faae4104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:00.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.139+0000 7faaeafd7640 1 -- 172.21.15.94:0/3090431852 shutdown_connections 2024-02-17T11:58:00.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.139+0000 7faaeafd7640 1 -- 172.21.15.94:0/3090431852 wait complete. 2024-02-17T11:58:00.148 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:59 smithi094 ceph-mon[32499]: pgmap v347: 97 pgs: 97 active+clean; 648 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-17T11:58:00.148 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:59 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1353174837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:58:00.148 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:59 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T11:58:00.148 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:59 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T11:58:00.148 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:59 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T11:58:00.148 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:59 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T11:58:00.149 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:57:59 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/523503502' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-17T11:58:00.149 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:57:59 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: ignoring --setuser ceph since I am not root 2024-02-17T11:58:00.149 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:57:59 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: ignoring --setgroup ceph since I am not root 2024-02-17T11:58:00.150 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-02-17T11:58:00.150 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:57:59 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: ignoring --setuser ceph since I am not root 2024-02-17T11:58:00.150 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:57:59 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: ignoring --setgroup ceph since I am not root 2024-02-17T11:58:00.150 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:57:59 smithi189 ceph-mgr[38455]: -- 172.21.15.189:0/4101287013 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55860ca781a0 con 0x55860ca4e800 2024-02-17T11:58:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:59 smithi189 ceph-mon[36588]: pgmap v347: 97 pgs: 97 active+clean; 648 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-17T11:58:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:59 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1353174837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:58:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:59 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T11:58:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:59 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T11:58:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:59 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T11:58:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:59 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T11:58:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:57:59 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/523503502' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-17T11:58:00.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:59 smithi125 ceph-mon[36819]: pgmap v347: 97 pgs: 97 active+clean; 648 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-02-17T11:58:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:59 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1353174837' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T11:58:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:59 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T11:58:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:59 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T11:58:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:59 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T11:58:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:59 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T11:58:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:57:59 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/523503502' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-17T11:58:00.184 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:57:59 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: ignoring --setuser ceph since I am not root 2024-02-17T11:58:00.184 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:57:59 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: ignoring --setgroup ceph since I am not root 2024-02-17T11:58:00.184 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:57:59 smithi125 ceph-mgr[38210]: -- 172.21.15.125:0/3982625417 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55bfa019e1a0 con 0x55bfa0175000 2024-02-17T11:58:00.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.257+0000 7f93ade40640 1 -- 172.21.15.94:0/753306295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93a8102d90 msgr2=0x7f93a8109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.257+0000 7f93ade40640 1 --2- 172.21.15.94:0/753306295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93a8102d90 0x7f93a8109620 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f93a0009ca0 tx=0x7f93a002c0f0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.257+0000 7f93ade40640 1 -- 172.21.15.94:0/753306295 shutdown_connections 2024-02-17T11:58:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.257+0000 7f93ade40640 1 --2- 172.21.15.94:0/753306295 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93a8102d90 0x7f93a8109620 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.257+0000 7f93ade40640 1 --2- 172.21.15.94:0/753306295 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93a81023d0 0x7f93a8102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.257+0000 7f93ade40640 1 --2- 172.21.15.94:0/753306295 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93a81011e0 0x7f93a8101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.257+0000 7f93ade40640 1 -- 172.21.15.94:0/753306295 >> 172.21.15.94:0/753306295 conn(0x7f93a80fc750 msgr2=0x7f93a80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.257+0000 7f93ade40640 1 -- 172.21.15.94:0/753306295 shutdown_connections 2024-02-17T11:58:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.257+0000 7f93ade40640 1 -- 172.21.15.94:0/753306295 wait complete. 2024-02-17T11:58:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.257+0000 7f93ade40640 1 Processor -- start 2024-02-17T11:58:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.257+0000 7f93ade40640 1 -- start start 2024-02-17T11:58:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93ade40640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93a81011e0 0x7f93a8074f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:00.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93ade40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93a81023d0 0x7f93a8073570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93ade40640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93a8102d90 0x7f93a8073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93ade40640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8074210 con 0x7f93a81023d0 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93ade40640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8074380 con 0x7f93a81011e0 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93ade40640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81a11c0 con 0x7f93a8102d90 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93a81023d0 0x7f93a8073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93a8102d90 0x7f93a8073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93a81023d0 0x7f93a8073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53632/0 (socket says 172.21.15.94:53632) 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93a8102d90 0x7f93a8073ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:53196/0 (socket says 172.21.15.94:53196) 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a6ffd640 1 -- 172.21.15.94:0/2392393145 learned_addr learned my addr 172.21.15.94:0/2392393145 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a77fe640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93a81011e0 0x7f93a8074f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a6ffd640 1 -- 172.21.15.94:0/2392393145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93a8102d90 msgr2=0x7f93a8073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a6ffd640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93a8102d90 0x7f93a8073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a6ffd640 1 -- 172.21.15.94:0/2392393145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93a81011e0 msgr2=0x7f93a8074f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:00.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a6ffd640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93a81011e0 0x7f93a8074f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a6ffd640 1 -- 172.21.15.94:0/2392393145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a0009900 con 0x7f93a81023d0 2024-02-17T11:58:00.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a7fff640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93a8102d90 0x7f93a8073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:00.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a77fe640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93a81011e0 0x7f93a8074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:00.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a6ffd640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93a81023d0 0x7f93a8073570 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f939c00d7a0 tx=0x7f939c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:00.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.258+0000 7f93a4ff9640 1 -- 172.21.15.94:0/2392393145 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c009c30 con 0x7f93a81023d0 2024-02-17T11:58:00.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.259+0000 7f93ade40640 1 -- 172.21.15.94:0/2392393145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a81a1470 con 0x7f93a81023d0 2024-02-17T11:58:00.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.259+0000 7f93ade40640 1 -- 172.21.15.94:0/2392393145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a81a1980 con 0x7f93a81023d0 2024-02-17T11:58:00.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.259+0000 7f93a4ff9640 1 -- 172.21.15.94:0/2392393145 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f939c00deb0 con 0x7f93a81023d0 2024-02-17T11:58:00.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.259+0000 7f93a4ff9640 1 -- 172.21.15.94:0/2392393145 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939c00f690 con 0x7f93a81023d0 2024-02-17T11:58:00.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.260+0000 7f93ade40640 1 -- 172.21.15.94:0/2392393145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93940052c0 con 0x7f93a81023d0 2024-02-17T11:58:00.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.261+0000 7f93a4ff9640 1 -- 172.21.15.94:0/2392393145 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) v1 ==== 136836+0+0 (secure 0 0 0) 0x7f939c00f7f0 con 0x7f93a81023d0 2024-02-17T11:58:00.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.262+0000 7f93a4ff9640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f93900a2250 0x7f93900a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:00.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.262+0000 7f93a77fe640 1 -- 172.21.15.94:0/2392393145 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f93900a2250 msgr2=0x7f93900a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1808883811 2024-02-17T11:58:00.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.262+0000 7f93a4ff9640 1 -- 172.21.15.94:0/2392393145 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f939c0cd790 con 0x7f93a81023d0 2024-02-17T11:58:00.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.262+0000 7f93a77fe640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f93900a2250 0x7f93900a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T11:58:00.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.266+0000 7f93a4ff9640 1 -- 172.21.15.94:0/2392393145 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f939c0966b0 con 0x7f93a81023d0 2024-02-17T11:58:00.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.448+0000 7f93ade40640 1 -- 172.21.15.94:0/2392393145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f9394005570 con 0x7f93a81023d0 2024-02-17T11:58:00.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.452+0000 7f93a4ff9640 1 -- 172.21.15.94:0/2392393145 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) v1 ==== 133+0+0 (secure 0 0 0) 0x7f939c099d70 con 0x7f93a81023d0 2024-02-17T11:58:00.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.454+0000 7f93ade40640 1 -- 172.21.15.94:0/2392393145 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f93900a2250 msgr2=0x7f93900a4710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T11:58:00.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.454+0000 7f93ade40640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f93900a2250 0x7f93900a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.454+0000 7f93ade40640 1 -- 172.21.15.94:0/2392393145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93a81023d0 msgr2=0x7f93a8073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:00.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.454+0000 7f93ade40640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93a81023d0 0x7f93a8073570 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f939c00d7a0 tx=0x7f939c00dc60 comp rx=0 tx=0).stop 2024-02-17T11:58:00.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.454+0000 7f93ade40640 1 -- 172.21.15.94:0/2392393145 shutdown_connections 2024-02-17T11:58:00.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.454+0000 7f93ade40640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f93900a2250 0x7f93900a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.454+0000 7f93ade40640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93a8102d90 0x7f93a8073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.454+0000 7f93ade40640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f93a81023d0 0x7f93a8073570 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.454+0000 7f93ade40640 1 --2- 172.21.15.94:0/2392393145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f93a81011e0 0x7f93a8074f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.454+0000 7f93ade40640 1 -- 172.21.15.94:0/2392393145 >> 172.21.15.94:0/2392393145 conn(0x7f93a80fc750 msgr2=0x7f93a8105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:00.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.454+0000 7f93ade40640 1 -- 172.21.15.94:0/2392393145 shutdown_connections 2024-02-17T11:58:00.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.455+0000 7f93ade40640 1 -- 172.21.15.94:0/2392393145 wait complete. 2024-02-17T11:58:00.465 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-02-17T11:58:00.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.572+0000 7f303af81640 1 -- 172.21.15.94:0/980246742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034104550 msgr2=0x7f3034106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:00.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.572+0000 7f303af81640 1 --2- 172.21.15.94:0/980246742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034104550 0x7f3034106940 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3020009900 tx=0x7f302002bc30 comp rx=0 tx=0).stop 2024-02-17T11:58:00.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.572+0000 7f303af81640 1 -- 172.21.15.94:0/980246742 shutdown_connections 2024-02-17T11:58:00.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.572+0000 7f303af81640 1 --2- 172.21.15.94:0/980246742 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3034106e80 0x7f30341092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.572+0000 7f303af81640 1 --2- 172.21.15.94:0/980246742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034104550 0x7f3034106940 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.572+0000 7f303af81640 1 --2- 172.21.15.94:0/980246742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3034101c20 0x7f3034104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.572+0000 7f303af81640 1 -- 172.21.15.94:0/980246742 >> 172.21.15.94:0/980246742 conn(0x7f30340fb610 msgr2=0x7f30340fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:00.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.572+0000 7f303af81640 1 -- 172.21.15.94:0/980246742 shutdown_connections 2024-02-17T11:58:00.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.572+0000 7f303af81640 1 -- 172.21.15.94:0/980246742 wait complete. 2024-02-17T11:58:00.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.573+0000 7f303af81640 1 Processor -- start 2024-02-17T11:58:00.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.573+0000 7f303af81640 1 -- start start 2024-02-17T11:58:00.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.573+0000 7f303af81640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3034101c20 0x7f3034197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:00.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.573+0000 7f303af81640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3034104550 0x7f30341976d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:00.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.573+0000 7f303af81640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034106e80 0x7f303419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:00.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.573+0000 7f303af81640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303419ec80 con 0x7f3034106e80 2024-02-17T11:58:00.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.573+0000 7f303af81640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303419edf0 con 0x7f3034101c20 2024-02-17T11:58:00.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.573+0000 7f303af81640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303419efb0 con 0x7f3034104550 2024-02-17T11:58:00.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.573+0000 7f303951e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034106e80 0x7f303419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:00.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.573+0000 7f303951e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034106e80 0x7f303419e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53644/0 (socket says 172.21.15.94:53644) 2024-02-17T11:58:00.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.573+0000 7f303951e640 1 -- 172.21.15.94:0/3706794781 learned_addr learned my addr 172.21.15.94:0/3706794781 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:00.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.573+0000 7f3038d1d640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3034101c20 0x7f3034197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:00.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.573+0000 7f3033fff640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3034104550 0x7f30341976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:00.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.574+0000 7f303951e640 1 -- 172.21.15.94:0/3706794781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3034104550 msgr2=0x7f30341976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:00.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.574+0000 7f303951e640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3034104550 0x7f30341976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.574+0000 7f303951e640 1 -- 172.21.15.94:0/3706794781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3034101c20 msgr2=0x7f3034197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:00.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.574+0000 7f303951e640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3034101c20 0x7f3034197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:00.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.574+0000 7f303951e640 1 -- 172.21.15.94:0/3706794781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3020009560 con 0x7f3034106e80 2024-02-17T11:58:00.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.574+0000 7f303951e640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034106e80 0x7f303419e740 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f302c00eaa0 tx=0x7f302c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:00.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.574+0000 7f3031ffb640 1 -- 172.21.15.94:0/3706794781 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c00cdb0 con 0x7f3034106e80 2024-02-17T11:58:00.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.574+0000 7f3031ffb640 1 -- 172.21.15.94:0/3706794781 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f302c005020 con 0x7f3034106e80 2024-02-17T11:58:00.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.574+0000 7f303af81640 1 -- 172.21.15.94:0/3706794781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f303419f290 con 0x7f3034106e80 2024-02-17T11:58:00.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.574+0000 7f3031ffb640 1 -- 172.21.15.94:0/3706794781 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c0116b0 con 0x7f3034106e80 2024-02-17T11:58:00.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.574+0000 7f303af81640 1 -- 172.21.15.94:0/3706794781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303419f720 con 0x7f3034106e80 2024-02-17T11:58:00.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.575+0000 7f303af81640 1 -- 172.21.15.94:0/3706794781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30340fd100 con 0x7f3034106e80 2024-02-17T11:58:00.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.576+0000 7f3031ffb640 1 -- 172.21.15.94:0/3706794781 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 20) v1 ==== 136836+0+0 (secure 0 0 0) 0x7f302c005520 con 0x7f3034106e80 2024-02-17T11:58:00.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.576+0000 7f3031ffb640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f30240a2250 0x7f30240a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:00.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.576+0000 7f3038d1d640 1 -- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f30240a2250 msgr2=0x7f30240a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1808883811 2024-02-17T11:58:00.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.577+0000 7f3031ffb640 1 -- 172.21.15.94:0/3706794781 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8240+0+0 (secure 0 0 0) 0x7f302c0cdf80 con 0x7f3034106e80 2024-02-17T11:58:00.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.577+0000 7f3038d1d640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f30240a2250 0x7f30240a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-17T11:58:00.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.581+0000 7f3031ffb640 1 -- 172.21.15.94:0/3706794781 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f302c096f60 con 0x7f3034106e80 2024-02-17T11:58:00.650 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:00 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:00.365+0000 7f738cc96200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T11:58:00.650 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:00 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:00.460+0000 7f738cc96200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T11:58:00.684 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:00 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:00.365+0000 7fdb8f7f0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T11:58:00.684 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:00 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:00.459+0000 7fdb8f7f0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T11:58:00.704 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:00 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:00.427+0000 7f2cdad32200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-17T11:58:00.705 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:00 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:00.531+0000 7f2cdad32200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-17T11:58:00.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.778+0000 7f3038d1d640 1 -- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f30240a2250 msgr2=0x7f30240a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1808883811 2024-02-17T11:58:00.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.778+0000 7f3038d1d640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f30240a2250 0x7f30240a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-17T11:58:00.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:00.782+0000 7f303af81640 1 -- 172.21.15.94:0/3706794781 --> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f303402ce10 con 0x7f30240a2250 2024-02-17T11:58:01.089 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:00 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/523503502' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-17T11:58:01.089 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:00 smithi125 ceph-mon[36819]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-02-17T11:58:01.090 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:00 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3090431852' entity='client.admin' 2024-02-17T11:58:01.090 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:00 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2392393145' entity='client.admin' 2024-02-17T11:58:01.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:00 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/523503502' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-17T11:58:01.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:00 smithi189 ceph-mon[36588]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-02-17T11:58:01.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:00 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3090431852' entity='client.admin' 2024-02-17T11:58:01.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:00 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2392393145' entity='client.admin' 2024-02-17T11:58:01.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:00 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/523503502' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-17T11:58:01.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:00 smithi094 ceph-mon[32499]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-02-17T11:58:01.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:00 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3090431852' entity='client.admin' 2024-02-17T11:58:01.174 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:00 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2392393145' entity='client.admin' 2024-02-17T11:58:01.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:01.179+0000 7f3038d1d640 1 -- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f30240a2250 msgr2=0x7f30240a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1808883811 2024-02-17T11:58:01.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:01.179+0000 7f3038d1d640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f30240a2250 0x7f30240a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-17T11:58:01.400 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:01.090+0000 7f738cc96200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T11:58:01.400 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:01.183+0000 7f738cc96200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T11:58:01.400 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:01.324+0000 7f738cc96200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T11:58:01.418 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:01.090+0000 7fdb8f7f0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T11:58:01.418 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:01.183+0000 7fdb8f7f0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T11:58:01.418 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:01.324+0000 7fdb8f7f0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T11:58:01.418 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:01.419+0000 7fdb8f7f0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T11:58:01.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:01 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:01.175+0000 7f2cdad32200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-17T11:58:01.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:01 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:01.269+0000 7f2cdad32200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-17T11:58:01.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:01 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:01.411+0000 7f2cdad32200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-17T11:58:01.684 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:01.501+0000 7fdb8f7f0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T11:58:01.761 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:01.420+0000 7f738cc96200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T11:58:01.761 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:01.501+0000 7f738cc96200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T11:58:01.853 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:01 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:01.507+0000 7f2cdad32200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-17T11:58:01.853 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:01 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:01.591+0000 7f2cdad32200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-17T11:58:01.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:01.980+0000 7f3038d1d640 1 -- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f30240a2250 msgr2=0x7f30240a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1808883811 2024-02-17T11:58:01.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:01.980+0000 7f3038d1d640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f30240a2250 0x7f30240a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-17T11:58:02.048 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:01.762+0000 7f738cc96200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T11:58:02.048 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:01.871+0000 7f738cc96200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T11:58:02.048 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:01.956+0000 7f738cc96200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T11:58:02.049 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:01.763+0000 7fdb8f7f0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T11:58:02.049 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:01.871+0000 7fdb8f7f0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T11:58:02.049 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:01.957+0000 7fdb8f7f0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T11:58:02.049 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:02 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:02.050+0000 7fdb8f7f0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T11:58:02.145 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:01 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:01.854+0000 7f2cdad32200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-17T11:58:02.146 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:01 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:01.964+0000 7f2cdad32200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-17T11:58:02.146 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:02.051+0000 7f2cdad32200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-17T11:58:02.146 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:02.146+0000 7f2cdad32200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T11:58:02.400 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:02.049+0000 7f738cc96200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-17T11:58:02.400 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:02.225+0000 7f738cc96200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T11:58:02.413 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:02.323+0000 7f2cdad32200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T11:58:02.413 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:02 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:02.224+0000 7fdb8f7f0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-17T11:58:02.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:02 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T11:58:02.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:02 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T11:58:02.900 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:02 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T11:58:03.434 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:03 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:03.177+0000 7fdb8f7f0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T11:58:03.434 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:03 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:03.306+0000 7fdb8f7f0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T11:58:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:03.581+0000 7f3038d1d640 1 -- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f30240a2250 msgr2=0x7f30240a4710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.94:6800/1808883811 2024-02-17T11:58:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:03.581+0000 7f3038d1d640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f30240a2250 0x7f30240a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-17T11:58:03.650 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:03.185+0000 7f738cc96200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T11:58:03.650 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:03.315+0000 7f738cc96200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T11:58:03.704 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:03 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:03.290+0000 7f2cdad32200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-17T11:58:03.705 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:03 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:03.421+0000 7f2cdad32200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-17T11:58:03.916 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:03 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:03.635+0000 7fdb8f7f0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T11:58:03.916 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:03 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:03.724+0000 7fdb8f7f0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T11:58:03.917 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:03 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:03.917+0000 7fdb8f7f0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:58:03.935 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:03.651+0000 7f738cc96200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T11:58:03.935 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:03.742+0000 7f738cc96200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T11:58:03.935 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:03.936+0000 7f738cc96200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:58:04.044 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:03 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:03.755+0000 7f2cdad32200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-17T11:58:04.044 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:03 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:03.847+0000 7f2cdad32200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-17T11:58:04.184 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:04 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:04.100+0000 7fdb8f7f0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T11:58:04.400 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:04 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:04.122+0000 7f738cc96200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T11:58:04.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:04 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:04.045+0000 7f2cdad32200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:58:04.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:04 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:04.232+0000 7f2cdad32200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-17T11:58:04.637 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:04 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:04.381+0000 7fdb8f7f0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:58:04.670 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:04 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:04.408+0000 7f738cc96200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:58:04.777 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:04 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:04.518+0000 7f2cdad32200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-17T11:58:04.934 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:04 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:04.637+0000 7fdb8f7f0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T11:58:04.934 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:04 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:04.872+0000 7fdb8f7f0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T11:58:05.150 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:04 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:04.671+0000 7f738cc96200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T11:58:05.150 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:04 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:04.906+0000 7f738cc96200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T11:58:05.204 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:04 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:04.778+0000 7f2cdad32200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-17T11:58:05.205 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:05 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:05.018+0000 7f2cdad32200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-17T11:58:05.934 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:05 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:05.666+0000 7fdb8f7f0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T11:58:06.150 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:05 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:05.715+0000 7f738cc96200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T11:58:06.204 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:05 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:05.829+0000 7f2cdad32200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-17T11:58:06.538 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:06 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:06.229+0000 7fdb8f7f0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T11:58:06.538 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:06 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:06.325+0000 7fdb8f7f0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T11:58:06.603 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:06 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:06.288+0000 7f738cc96200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T11:58:06.603 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:06 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:06.387+0000 7f738cc96200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T11:58:06.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:06.613+0000 7f3031ffb640 1 -- 172.21.15.94:0/3706794781 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mgrmap(e 21) v1 ==== 136836+0+0 (secure 0 0 0) 0x7f302c097330 con 0x7f3034106e80 2024-02-17T11:58:06.704 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-mon[32499]: Standby manager daemon y restarted 2024-02-17T11:58:06.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-mon[32499]: Standby manager daemon y started 2024-02-17T11:58:06.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.125:0/177715052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-17T11:58:06.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.125:0/177715052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T11:58:06.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.125:0/177715052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-17T11:58:06.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.125:0/177715052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T11:58:06.705 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:06.399+0000 7f2cdad32200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-17T11:58:06.705 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:06.499+0000 7f2cdad32200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-17T11:58:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:06.754+0000 7f3031ffb640 1 -- 172.21.15.94:0/3706794781 <== mon.0 v2:172.21.15.94:3300/0 8 ==== mgrmap(e 22) v1 ==== 136453+0+0 (secure 0 0 0) 0x7f302c001250 con 0x7f3034106e80 2024-02-17T11:58:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:06.754+0000 7f3031ffb640 1 -- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f30240a2250 msgr2=0x7f30240a4710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T11:58:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:06.754+0000 7f3031ffb640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/1808883811,v1:172.21.15.94:6801/1808883811] conn(0x7f30240a2250 0x7f30240a4710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:06.881 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 11:58:06 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y[38206]: 2024-02-17T11:58:06.539+0000 7fdb8f7f0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T11:58:06.881 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:06 smithi125 ceph-mon[36819]: Standby manager daemon y restarted 2024-02-17T11:58:06.882 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:06 smithi125 ceph-mon[36819]: Standby manager daemon y started 2024-02-17T11:58:06.882 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:06 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.125:0/177715052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-17T11:58:06.882 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:06 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.125:0/177715052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T11:58:06.882 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:06 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.125:0/177715052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-17T11:58:06.882 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:06 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.125:0/177715052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T11:58:06.883 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 11:58:06 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z[38451]: 2024-02-17T11:58:06.604+0000 7f738cc96200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T11:58:06.883 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:06 smithi189 ceph-mon[36588]: Standby manager daemon y restarted 2024-02-17T11:58:06.883 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:06 smithi189 ceph-mon[36588]: Standby manager daemon y started 2024-02-17T11:58:06.883 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:06 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.125:0/177715052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-17T11:58:06.883 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:06 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.125:0/177715052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T11:58:06.883 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:06 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.125:0/177715052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-17T11:58:06.884 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:06 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.125:0/177715052' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T11:58:06.983 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:06.730+0000 7f2cdad32200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-17T11:58:07.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:06.988+0000 7f2c3a7db700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:58:07.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:06.988+0000 7f2c3a7db700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:58:07.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:06.988+0000 7f2c3a7db700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:58:07.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:06.988+0000 7f2c3a7db700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:58:07.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:06.988+0000 7f2c3a7db700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:58:07.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:06.989+0000 7f2c3cfe0700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:58:07.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:06.989+0000 7f2c3cfe0700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:58:07.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:06.989+0000 7f2c3cfe0700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:58:07.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:06.989+0000 7f2c3cfe0700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:58:07.455 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:58:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:58:06.989+0000 7f2c3cfe0700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:58:07.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.763+0000 7f3031ffb640 1 -- 172.21.15.94:0/3706794781 <== mon.0 v2:172.21.15.94:3300/0 9 ==== mgrmap(e 23) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f302c005190 con 0x7f3034106e80 2024-02-17T11:58:07.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.764+0000 7f3031ffb640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3024035090 0x7f3024035470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:07.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.764+0000 7f3031ffb640 1 -- 172.21.15.94:0/3706794781 --> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f303402ce10 con 0x7f3024035090 2024-02-17T11:58:07.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.764+0000 7f3038d1d640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3024035090 0x7f3024035470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:07.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.765+0000 7f3038d1d640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3024035090 0x7f3024035470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3028005d00 tx=0x7f302800b410 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:07.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.826+0000 7f3031ffb640 1 -- 172.21.15.94:0/3706794781 <== mgr.15435 v2:172.21.15.94:6800/2106228828 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f303402ce10 con 0x7f3024035090 2024-02-17T11:58:07.825 INFO:teuthology.orchestra.run.smithi094.stdout:Schedule set for path / 2024-02-17T11:58:07.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.828+0000 7f303af81640 1 -- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3024035090 msgr2=0x7f3024035470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:07.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.828+0000 7f303af81640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3024035090 0x7f3024035470 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3028005d00 tx=0x7f302800b410 comp rx=0 tx=0).stop 2024-02-17T11:58:07.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.828+0000 7f303af81640 1 -- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034106e80 msgr2=0x7f303419e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:07.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.828+0000 7f303af81640 1 --2- 172.21.15.94:0/3706794781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3034106e80 0x7f303419e740 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f302c00eaa0 tx=0x7f302c00ef60 comp rx=0 tx=0).stop 2024-02-17T11:58:07.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.828+0000 7f303951e640 1 -- 172.21.15.94:0/3706794781 reap_dead start 2024-02-17T11:58:07.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.828+0000 7f303af81640 1 -- 172.21.15.94:0/3706794781 shutdown_connections 2024-02-17T11:58:07.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.828+0000 7f303af81640 1 -- 172.21.15.94:0/3706794781 >> 172.21.15.94:0/3706794781 conn(0x7f30340fb610 msgr2=0x7f3034102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:07.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.828+0000 7f303af81640 1 -- 172.21.15.94:0/3706794781 shutdown_connections 2024-02-17T11:58:07.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.828+0000 7f303af81640 1 -- 172.21.15.94:0/3706794781 wait complete. 2024-02-17T11:58:07.838 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-02-17T11:58:07.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: mgrmap e21: x(active, since 9m), standbys: z, y 2024-02-17T11:58:07.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.189:0/3924156684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-17T11:58:07.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.189:0/3924156684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T11:58:07.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: Standby manager daemon z restarted 2024-02-17T11:58:07.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: Standby manager daemon z started 2024-02-17T11:58:07.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.189:0/3924156684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-17T11:58:07.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.? 172.21.15.189:0/3924156684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T11:58:07.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: Active manager daemon x restarted 2024-02-17T11:58:07.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: Activating manager daemon x 2024-02-17T11:58:07.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: osdmap e88: 12 total, 12 up, 12 in 2024-02-17T11:58:07.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: mgrmap e22: x(active, starting, since 0.0115349s), standbys: y, z 2024-02-17T11:58:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:58:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:58:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:58:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-17T11:58:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-17T11:58:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T11:58:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T11:58:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T11:58:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T11:58:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T11:58:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T11:58:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T11:58:07.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:58:07.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:58:07.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:58:07.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:58:07.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:58:07.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-17T11:58:07.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-17T11:58:07.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-17T11:58:07.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: Manager daemon x is now available 2024-02-17T11:58:07.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:58:07.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:58:07.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-17T11:58:07.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:07 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-17T11:58:07.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: mgrmap e21: x(active, since 9m), standbys: z, y 2024-02-17T11:58:07.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.189:0/3924156684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-17T11:58:07.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.189:0/3924156684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T11:58:07.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: Standby manager daemon z restarted 2024-02-17T11:58:07.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: Standby manager daemon z started 2024-02-17T11:58:07.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.189:0/3924156684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-17T11:58:07.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.? 172.21.15.189:0/3924156684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T11:58:07.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: Active manager daemon x restarted 2024-02-17T11:58:07.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: Activating manager daemon x 2024-02-17T11:58:07.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: osdmap e88: 12 total, 12 up, 12 in 2024-02-17T11:58:07.935 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: mgrmap e22: x(active, starting, since 0.0115349s), standbys: y, z 2024-02-17T11:58:07.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:58:07.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:58:07.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:58:07.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-17T11:58:07.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-17T11:58:07.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T11:58:07.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T11:58:07.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T11:58:07.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T11:58:07.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T11:58:07.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T11:58:07.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T11:58:07.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T11:58:07.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T11:58:07.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T11:58:07.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-17T11:58:07.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-17T11:58:07.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-17T11:58:07.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:58:07.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:58:07.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:58:07.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:58:07.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:58:07.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:58:07.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:58:07.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:58:07.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:58:07.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:58:07.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:58:07.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:58:07.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-17T11:58:07.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-17T11:58:07.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-17T11:58:07.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: Manager daemon x is now available 2024-02-17T11:58:07.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:58:07.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:58:07.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-17T11:58:07.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:07 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-17T11:58:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.946+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3955554784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec1000b0 msgr2=0x7f5bec1004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.946+0000 7f5bf19b6640 1 --2- 172.21.15.94:0/3955554784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec1000b0 0x7f5bec1004d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009900 tx=0x7f5bdc02bc30 comp rx=0 tx=0).stop 2024-02-17T11:58:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.946+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3955554784 shutdown_connections 2024-02-17T11:58:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.946+0000 7f5bf19b6640 1 --2- 172.21.15.94:0/3955554784 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5bec101c60 0x7f5bec1084f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.946+0000 7f5bf19b6640 1 --2- 172.21.15.94:0/3955554784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bec1012a0 0x7f5bec101720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.946+0000 7f5bf19b6640 1 --2- 172.21.15.94:0/3955554784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec1000b0 0x7f5bec1004d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.946+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3955554784 >> 172.21.15.94:0/3955554784 conn(0x7f5bec0fb620 msgr2=0x7f5bec0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.946+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3955554784 shutdown_connections 2024-02-17T11:58:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.946+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3955554784 wait complete. 2024-02-17T11:58:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.946+0000 7f5bf19b6640 1 Processor -- start 2024-02-17T11:58:07.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bf19b6640 1 -- start start 2024-02-17T11:58:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bf19b6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bec1000b0 0x7f5bec06a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bf19b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec1012a0 0x7f5bec19bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bf19b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5bec101c60 0x7f5bec19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bf19b6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec19ed10 con 0x7f5bec1012a0 2024-02-17T11:58:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bf19b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec19ee80 con 0x7f5bec101c60 2024-02-17T11:58:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bf19b6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec19f040 con 0x7f5bec1000b0 2024-02-17T11:58:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bea7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec1012a0 0x7f5bec19bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bea7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec1012a0 0x7f5bec19bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56042/0 (socket says 172.21.15.94:56042) 2024-02-17T11:58:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5beaffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bec1000b0 0x7f5bec06a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bea7fc640 1 -- 172.21.15.94:0/3308147250 learned_addr learned my addr 172.21.15.94:0/3308147250 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5beb7fe640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5bec101c60 0x7f5bec19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bea7fc640 1 -- 172.21.15.94:0/3308147250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bec1000b0 msgr2=0x7f5bec06a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:07.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bea7fc640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bec1000b0 0x7f5bec06a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:07.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bea7fc640 1 -- 172.21.15.94:0/3308147250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5bec101c60 msgr2=0x7f5bec19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:07.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bea7fc640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5bec101c60 0x7f5bec19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:07.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bea7fc640 1 -- 172.21.15.94:0/3308147250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc009560 con 0x7f5bec1012a0 2024-02-17T11:58:07.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5beb7fe640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5bec101c60 0x7f5bec19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:58:07.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bea7fc640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec1012a0 0x7f5bec19bf10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5be000e9f0 tx=0x7f5be000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:07.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.947+0000 7f5bf09b4640 1 -- 172.21.15.94:0/3308147250 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0009720 con 0x7f5bec1012a0 2024-02-17T11:58:07.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.948+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3308147250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bec19f320 con 0x7f5bec1012a0 2024-02-17T11:58:07.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.948+0000 7f5bf09b4640 1 -- 172.21.15.94:0/3308147250 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5be000cd80 con 0x7f5bec1012a0 2024-02-17T11:58:07.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.948+0000 7f5bf09b4640 1 -- 172.21.15.94:0/3308147250 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be0010690 con 0x7f5bec1012a0 2024-02-17T11:58:07.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.948+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3308147250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec19f830 con 0x7f5bec1012a0 2024-02-17T11:58:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.949+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3308147250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bec107300 con 0x7f5bec1012a0 2024-02-17T11:58:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.950+0000 7f5bf09b4640 1 -- 172.21.15.94:0/3308147250 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f5be00046c0 con 0x7f5bec1012a0 2024-02-17T11:58:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.950+0000 7f5bf09b4640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5bd40a1f70 0x7f5bd40a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.950+0000 7f5beaffd640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5bd40a1f70 0x7f5bd40a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.951+0000 7f5bf09b4640 1 -- 172.21.15.94:0/3308147250 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5be0014070 con 0x7f5bec1012a0 2024-02-17T11:58:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.951+0000 7f5beaffd640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5bd40a1f70 0x7f5bd40a4430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005ab0 tx=0x7f5bdc005fd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:07.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:07.954+0000 7f5bf09b4640 1 -- 172.21.15.94:0/3308147250 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5be00971b0 con 0x7f5bec1012a0 2024-02-17T11:58:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: mgrmap e21: x(active, since 9m), standbys: z, y 2024-02-17T11:58:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.189:0/3924156684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-17T11:58:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.189:0/3924156684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-17T11:58:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: Standby manager daemon z restarted 2024-02-17T11:58:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: Standby manager daemon z started 2024-02-17T11:58:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.189:0/3924156684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-17T11:58:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.? 172.21.15.189:0/3924156684' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-17T11:58:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: Active manager daemon x restarted 2024-02-17T11:58:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: Activating manager daemon x 2024-02-17T11:58:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: osdmap e88: 12 total, 12 up, 12 in 2024-02-17T11:58:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: mgrmap e22: x(active, starting, since 0.0115349s), standbys: y, z 2024-02-17T11:58:07.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T11:58:07.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T11:58:07.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-17T11:58:07.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-17T11:58:07.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-17T11:58:07.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-17T11:58:07.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-17T11:58:07.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-17T11:58:07.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-17T11:58:07.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-17T11:58:07.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-17T11:58:07.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-17T11:58:07.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-17T11:58:07.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-17T11:58:07.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-17T11:58:07.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-17T11:58:07.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-17T11:58:07.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-17T11:58:07.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-17T11:58:07.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-17T11:58:07.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: Manager daemon x is now available 2024-02-17T11:58:07.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:58:07.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:58:07.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-17T11:58:07.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:07 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-17T11:58:08.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.159+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3308147250 --> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bec02ce40 con 0x7f5bd40a1f70 2024-02-17T11:58:08.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.217+0000 7f5bf09b4640 1 -- 172.21.15.94:0/3308147250 <== mgr.15435 v2:172.21.15.94:6800/2106228828 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f5bec02ce40 con 0x7f5bd40a1f70 2024-02-17T11:58:08.216 INFO:teuthology.orchestra.run.smithi094.stdout:Retention added to path / 2024-02-17T11:58:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.219+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3308147250 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5bd40a1f70 msgr2=0x7f5bd40a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.219+0000 7f5bf19b6640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5bd40a1f70 0x7f5bd40a4430 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005ab0 tx=0x7f5bdc005fd0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.219+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3308147250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec1012a0 msgr2=0x7f5bec19bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.219+0000 7f5bf19b6640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec1012a0 0x7f5bec19bf10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5be000e9f0 tx=0x7f5be000edf0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.219+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3308147250 shutdown_connections 2024-02-17T11:58:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.219+0000 7f5bf19b6640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5bd40a1f70 0x7f5bd40a4430 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.219+0000 7f5bf19b6640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5bec101c60 0x7f5bec19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.219+0000 7f5bf19b6640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5bec1012a0 0x7f5bec19bf10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.219+0000 7f5bf19b6640 1 --2- 172.21.15.94:0/3308147250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5bec1000b0 0x7f5bec06a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.219+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3308147250 >> 172.21.15.94:0/3308147250 conn(0x7f5bec0fb620 msgr2=0x7f5bec1044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:08.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.219+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3308147250 shutdown_connections 2024-02-17T11:58:08.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.219+0000 7f5bf19b6640 1 -- 172.21.15.94:0/3308147250 wait complete. 2024-02-17T11:58:08.229 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-17T11:58:08.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.334+0000 7fe94bb13640 1 -- 172.21.15.94:0/2497800202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe944101290 msgr2=0x7fe944101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.334+0000 7fe94bb13640 1 --2- 172.21.15.94:0/2497800202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe944101290 0x7fe944101710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe934009900 tx=0x7fe93402bc30 comp rx=0 tx=0).stop 2024-02-17T11:58:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.334+0000 7fe94bb13640 1 -- 172.21.15.94:0/2497800202 shutdown_connections 2024-02-17T11:58:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.334+0000 7fe94bb13640 1 --2- 172.21.15.94:0/2497800202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe944101c50 0x7fe9441084e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.334+0000 7fe94bb13640 1 --2- 172.21.15.94:0/2497800202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe944101290 0x7fe944101710 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.334+0000 7fe94bb13640 1 --2- 172.21.15.94:0/2497800202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe9441000a0 0x7fe9441004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.334+0000 7fe94bb13640 1 -- 172.21.15.94:0/2497800202 >> 172.21.15.94:0/2497800202 conn(0x7fe9440fb610 msgr2=0x7fe9440fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe94bb13640 1 -- 172.21.15.94:0/2497800202 shutdown_connections 2024-02-17T11:58:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe94bb13640 1 -- 172.21.15.94:0/2497800202 wait complete. 2024-02-17T11:58:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe94bb13640 1 Processor -- start 2024-02-17T11:58:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe94bb13640 1 -- start start 2024-02-17T11:58:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe94bb13640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9441000a0 0x7fe944069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe94bb13640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe944101290 0x7fe94419bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:08.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe94bb13640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe944101c50 0x7fe94419e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:08.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe94bb13640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94419ecd0 con 0x7fe9441000a0 2024-02-17T11:58:08.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe94bb13640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94419ee40 con 0x7fe944101290 2024-02-17T11:58:08.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe94bb13640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94419f000 con 0x7fe944101c50 2024-02-17T11:58:08.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe9498af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9441000a0 0x7fe944069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:08.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe9498af640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9441000a0 0x7fe944069370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56064/0 (socket says 172.21.15.94:56064) 2024-02-17T11:58:08.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe9498af640 1 -- 172.21.15.94:0/309695308 learned_addr learned my addr 172.21.15.94:0/309695308 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:08.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe94a0b0640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe944101c50 0x7fe94419e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:08.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.335+0000 7fe9490ae640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe944101290 0x7fe94419bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:08.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.336+0000 7fe9498af640 1 -- 172.21.15.94:0/309695308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe944101c50 msgr2=0x7fe94419e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:08.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.336+0000 7fe9498af640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe944101c50 0x7fe94419e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.336+0000 7fe9498af640 1 -- 172.21.15.94:0/309695308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe944101290 msgr2=0x7fe94419bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.336+0000 7fe9498af640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe944101290 0x7fe94419bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.336+0000 7fe9498af640 1 -- 172.21.15.94:0/309695308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe934009560 con 0x7fe9441000a0 2024-02-17T11:58:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.336+0000 7fe9498af640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9441000a0 0x7fe944069370 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe94000ea80 tx=0x7fe94000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.336+0000 7fe93affd640 1 -- 172.21.15.94:0/309695308 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94000ce40 con 0x7fe9441000a0 2024-02-17T11:58:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.336+0000 7fe93affd640 1 -- 172.21.15.94:0/309695308 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe940011950 con 0x7fe9441000a0 2024-02-17T11:58:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.336+0000 7fe94bb13640 1 -- 172.21.15.94:0/309695308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94419f280 con 0x7fe9441000a0 2024-02-17T11:58:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.336+0000 7fe93affd640 1 -- 172.21.15.94:0/309695308 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94001ab40 con 0x7fe9441000a0 2024-02-17T11:58:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.336+0000 7fe94bb13640 1 -- 172.21.15.94:0/309695308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94419f760 con 0x7fe9441000a0 2024-02-17T11:58:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.337+0000 7fe94bb13640 1 -- 172.21.15.94:0/309695308 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe944107360 con 0x7fe9441000a0 2024-02-17T11:58:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.338+0000 7fe93affd640 1 -- 172.21.15.94:0/309695308 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) v1 ==== 136580+0+0 (secure 0 0 0) 0x7fe940011ac0 con 0x7fe9441000a0 2024-02-17T11:58:08.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.338+0000 7fe93affd640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe9300a1f70 0x7fe9300a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:08.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.339+0000 7fe9490ae640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe9300a1f70 0x7fe9300a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:08.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.339+0000 7fe93affd640 1 -- 172.21.15.94:0/309695308 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe9400cdf40 con 0x7fe9441000a0 2024-02-17T11:58:08.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.339+0000 7fe9490ae640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe9300a1f70 0x7fe9300a4430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe934009230 tx=0x7fe934005fd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:08.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.343+0000 7fe93affd640 1 -- 172.21.15.94:0/309695308 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe9400970d0 con 0x7fe9441000a0 2024-02-17T11:58:08.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.552+0000 7fe94bb13640 1 -- 172.21.15.94:0/309695308 --> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fe94402cee0 con 0x7fe9300a1f70 2024-02-17T11:58:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.561+0000 7fe93affd640 1 -- 172.21.15.94:0/309695308 <== mgr.15435 v2:172.21.15.94:6800/2106228828 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+298 (secure 0 0 0) 0x7fe94402cee0 con 0x7fe9300a1f70 2024-02-17T11:58:08.559 INFO:teuthology.orchestra.run.smithi094.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-17T00:00:00", "created": "2024-02-17T11:58:07", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-02-17T11:58:08.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.563+0000 7fe94bb13640 1 -- 172.21.15.94:0/309695308 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe9300a1f70 msgr2=0x7fe9300a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:08.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.563+0000 7fe94bb13640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe9300a1f70 0x7fe9300a4430 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe934009230 tx=0x7fe934005fd0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.563+0000 7fe94bb13640 1 -- 172.21.15.94:0/309695308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9441000a0 msgr2=0x7fe944069370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:08.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.563+0000 7fe94bb13640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9441000a0 0x7fe944069370 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe94000ea80 tx=0x7fe94000ef40 comp rx=0 tx=0).stop 2024-02-17T11:58:08.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.563+0000 7fe94bb13640 1 -- 172.21.15.94:0/309695308 shutdown_connections 2024-02-17T11:58:08.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.563+0000 7fe94bb13640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe9300a1f70 0x7fe9300a4430 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.563+0000 7fe94bb13640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe944101c50 0x7fe94419e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.563+0000 7fe94bb13640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe944101290 0x7fe94419bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.563+0000 7fe94bb13640 1 --2- 172.21.15.94:0/309695308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe9441000a0 0x7fe944069370 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.563+0000 7fe94bb13640 1 -- 172.21.15.94:0/309695308 >> 172.21.15.94:0/309695308 conn(0x7fe9440fb610 msgr2=0x7fe944104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:08.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.563+0000 7fe94bb13640 1 -- 172.21.15.94:0/309695308 shutdown_connections 2024-02-17T11:58:08.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.563+0000 7fe94bb13640 1 -- 172.21.15.94:0/309695308 wait complete. 2024-02-17T11:58:08.574 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-17T11:58:08.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.689+0000 7f7ecd240640 1 -- 172.21.15.94:0/2743317358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ec8106e80 msgr2=0x7f7ec81092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.689+0000 7f7ecd240640 1 --2- 172.21.15.94:0/2743317358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ec8106e80 0x7f7ec81092d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0009900 tx=0x7f7eb002bc30 comp rx=0 tx=0).stop 2024-02-17T11:58:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.689+0000 7f7ecd240640 1 -- 172.21.15.94:0/2743317358 shutdown_connections 2024-02-17T11:58:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.689+0000 7f7ecd240640 1 --2- 172.21.15.94:0/2743317358 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ec8106e80 0x7f7ec81092d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.689+0000 7f7ecd240640 1 --2- 172.21.15.94:0/2743317358 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec8104550 0x7f7ec8106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.689+0000 7f7ecd240640 1 --2- 172.21.15.94:0/2743317358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec8101c20 0x7f7ec8104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.689+0000 7f7ecd240640 1 -- 172.21.15.94:0/2743317358 >> 172.21.15.94:0/2743317358 conn(0x7f7ec80fb610 msgr2=0x7f7ec80fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.689+0000 7f7ecd240640 1 -- 172.21.15.94:0/2743317358 shutdown_connections 2024-02-17T11:58:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.690+0000 7f7ecd240640 1 -- 172.21.15.94:0/2743317358 wait complete. 2024-02-17T11:58:08.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.690+0000 7f7ecd240640 1 Processor -- start 2024-02-17T11:58:08.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.690+0000 7f7ecd240640 1 -- start start 2024-02-17T11:58:08.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ecd240640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec8101c20 0x7f7ec8197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:08.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ecd240640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ec8104550 0x7f7ec8197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:08.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ecd240640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec8106e80 0x7f7ec819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:08.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ecd240640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec819ec30 con 0x7f7ec8104550 2024-02-17T11:58:08.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ecd240640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec819eda0 con 0x7f7ec8101c20 2024-02-17T11:58:08.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ecd240640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec819ef60 con 0x7f7ec8106e80 2024-02-17T11:58:08.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ec659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ec8104550 0x7f7ec8197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:08.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ec659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ec8104550 0x7f7ec8197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56080/0 (socket says 172.21.15.94:56080) 2024-02-17T11:58:08.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ec659c640 1 -- 172.21.15.94:0/2474751625 learned_addr learned my addr 172.21.15.94:0/2474751625 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:08.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ec759e640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec8106e80 0x7f7ec819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:08.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ec6d9d640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec8101c20 0x7f7ec8197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:08.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ec659c640 1 -- 172.21.15.94:0/2474751625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec8106e80 msgr2=0x7f7ec819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:08.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ec659c640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec8106e80 0x7f7ec819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ec659c640 1 -- 172.21.15.94:0/2474751625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec8101c20 msgr2=0x7f7ec8197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:08.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ec659c640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec8101c20 0x7f7ec8197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ec659c640 1 -- 172.21.15.94:0/2474751625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb0009560 con 0x7f7ec8104550 2024-02-17T11:58:08.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ec659c640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ec8104550 0x7f7ec8197680 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800e9c0 tx=0x7f7eb800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:08.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ebffff640 1 -- 172.21.15.94:0/2474751625 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8009700 con 0x7f7ec8104550 2024-02-17T11:58:08.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ebffff640 1 -- 172.21.15.94:0/2474751625 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7eb800cd80 con 0x7f7ec8104550 2024-02-17T11:58:08.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ecd240640 1 -- 172.21.15.94:0/2474751625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec819f240 con 0x7f7ec8104550 2024-02-17T11:58:08.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ebffff640 1 -- 172.21.15.94:0/2474751625 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8010430 con 0x7f7ec8104550 2024-02-17T11:58:08.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.691+0000 7f7ecd240640 1 -- 172.21.15.94:0/2474751625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec819f750 con 0x7f7ec8104550 2024-02-17T11:58:08.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.693+0000 7f7ecd240640 1 -- 172.21.15.94:0/2474751625 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec80fd100 con 0x7f7ec8104550 2024-02-17T11:58:08.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.694+0000 7f7ebffff640 1 -- 172.21.15.94:0/2474751625 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 23) v1 ==== 136580+0+0 (secure 0 0 0) 0x7f7eb8005000 con 0x7f7ec8104550 2024-02-17T11:58:08.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.694+0000 7f7ebffff640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7eb40a1f70 0x7f7eb40a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:08.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.694+0000 7f7ec6d9d640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7eb40a1f70 0x7f7eb40a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:08.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.694+0000 7f7ebffff640 1 -- 172.21.15.94:0/2474751625 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7eb8014070 con 0x7f7ec8104550 2024-02-17T11:58:08.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.695+0000 7f7ec6d9d640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7eb40a1f70 0x7f7eb40a4430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0005fd0 tx=0x7f7ec0009500 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:08.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.699+0000 7f7ebffff640 1 -- 172.21.15.94:0/2474751625 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7eb8096f40 con 0x7f7ec8104550 2024-02-17T11:58:08.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.937+0000 7f7ecd240640 1 -- 172.21.15.94:0/2474751625 --> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7ec802ceb0 con 0x7f7eb40a1f70 2024-02-17T11:58:08.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.945+0000 7f7ebffff640 1 -- 172.21.15.94:0/2474751625 <== mgr.15435 v2:172.21.15.94:6800/2106228828 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f7ec802ceb0 con 0x7f7eb40a1f70 2024-02-17T11:58:08.944 INFO:teuthology.orchestra.run.smithi094.stdout:/ 1m 6m3h 2024-02-17T11:58:08.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.947+0000 7f7ecd240640 1 -- 172.21.15.94:0/2474751625 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7eb40a1f70 msgr2=0x7f7eb40a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:08.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.947+0000 7f7ecd240640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7eb40a1f70 0x7f7eb40a4430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0005fd0 tx=0x7f7ec0009500 comp rx=0 tx=0).stop 2024-02-17T11:58:08.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.947+0000 7f7ecd240640 1 -- 172.21.15.94:0/2474751625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ec8104550 msgr2=0x7f7ec8197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:08.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.947+0000 7f7ecd240640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ec8104550 0x7f7ec8197680 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7eb800e9c0 tx=0x7f7eb800ee80 comp rx=0 tx=0).stop 2024-02-17T11:58:08.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.947+0000 7f7ecd240640 1 -- 172.21.15.94:0/2474751625 shutdown_connections 2024-02-17T11:58:08.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.947+0000 7f7ecd240640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7eb40a1f70 0x7f7eb40a4430 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.947+0000 7f7ecd240640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec8106e80 0x7f7ec819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.947+0000 7f7ecd240640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ec8104550 0x7f7ec8197680 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.947+0000 7f7ecd240640 1 --2- 172.21.15.94:0/2474751625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ec8101c20 0x7f7ec8197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:08.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.947+0000 7f7ecd240640 1 -- 172.21.15.94:0/2474751625 >> 172.21.15.94:0/2474751625 conn(0x7f7ec80fb610 msgr2=0x7f7ec81024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:08.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.947+0000 7f7ecd240640 1 -- 172.21.15.94:0/2474751625 shutdown_connections 2024-02-17T11:58:08.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:08.947+0000 7f7ecd240640 1 -- 172.21.15.94:0/2474751625 wait complete. 2024-02-17T11:58:08.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:08 smithi094 ceph-mon[32499]: [17/Feb/2024:11:58:07] ENGINE Bus STARTING 2024-02-17T11:58:08.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:08 smithi094 ceph-mon[32499]: [17/Feb/2024:11:58:07] ENGINE Serving on https://172.21.15.94:7150 2024-02-17T11:58:08.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:08 smithi094 ceph-mon[32499]: [17/Feb/2024:11:58:07] ENGINE Serving on http://172.21.15.94:8765 2024-02-17T11:58:08.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:08 smithi094 ceph-mon[32499]: [17/Feb/2024:11:58:07] ENGINE Bus STARTED 2024-02-17T11:58:08.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:08 smithi094 ceph-mon[32499]: mgrmap e23: x(active, since 1.01999s), standbys: y, z 2024-02-17T11:58:08.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:08 smithi094 ceph-mon[32499]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:58:08.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:08 smithi094 ceph-mon[32499]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:58:08.958 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-02-17T11:58:08.988 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-02-17T11:58:09.033 INFO:teuthology.run_tasks:Running task background_exec... 2024-02-17T11:58:09.043 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-02-17T11:58:09.044 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-02-17T11:58:09.046 INFO:teuthology.run_tasks:Running task exec... 2024-02-17T11:58:09.059 INFO:teuthology.task.exec:Executing custom commands... 2024-02-17T11:58:09.059 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi094.front.sepia.ceph.com 2024-02-17T11:58:09.059 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-02-17T11:58:09.098 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:09.115 INFO:teuthology.orchestra.run.smithi094.stdout:Filesystem Size Used Avail Use% Mounted on 2024-02-17T11:58:09.115 INFO:teuthology.orchestra.run.smithi094.stdout:0@3bc5cfc8-cd8a-11ee-95ba-87774f69a715.cephfs=/volumes/qa/sv_1/38f266b8-4b32-401a-962e-ca40f2c90bf2 340G 0 340G 0% /tmp/cephfs 2024-02-17T11:58:09.117 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-02-17T11:58:09.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:08 smithi189 ceph-mon[36588]: [17/Feb/2024:11:58:07] ENGINE Bus STARTING 2024-02-17T11:58:09.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:08 smithi189 ceph-mon[36588]: [17/Feb/2024:11:58:07] ENGINE Serving on https://172.21.15.94:7150 2024-02-17T11:58:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:08 smithi189 ceph-mon[36588]: [17/Feb/2024:11:58:07] ENGINE Serving on http://172.21.15.94:8765 2024-02-17T11:58:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:08 smithi189 ceph-mon[36588]: [17/Feb/2024:11:58:07] ENGINE Bus STARTED 2024-02-17T11:58:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:08 smithi189 ceph-mon[36588]: mgrmap e23: x(active, since 1.01999s), standbys: y, z 2024-02-17T11:58:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:08 smithi189 ceph-mon[36588]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:58:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:08 smithi189 ceph-mon[36588]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:58:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:08 smithi125 ceph-mon[36819]: [17/Feb/2024:11:58:07] ENGINE Bus STARTING 2024-02-17T11:58:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:08 smithi125 ceph-mon[36819]: [17/Feb/2024:11:58:07] ENGINE Serving on https://172.21.15.94:7150 2024-02-17T11:58:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:08 smithi125 ceph-mon[36819]: [17/Feb/2024:11:58:07] ENGINE Serving on http://172.21.15.94:8765 2024-02-17T11:58:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:08 smithi125 ceph-mon[36819]: [17/Feb/2024:11:58:07] ENGINE Bus STARTED 2024-02-17T11:58:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:08 smithi125 ceph-mon[36819]: mgrmap e23: x(active, since 1.01999s), standbys: y, z 2024-02-17T11:58:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:08 smithi125 ceph-mon[36819]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:58:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:08 smithi125 ceph-mon[36819]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:58:09.215 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-02-17T11:58:09.259 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-02-17T11:58:10.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:09 smithi189 ceph-mon[36588]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:58:10.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:09 smithi189 ceph-mon[36588]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:58:10.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:09 smithi189 ceph-mon[36588]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:58:10.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:09 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:09 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:09 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:09 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:09 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:09 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:09 smithi125 ceph-mon[36819]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:58:10.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:09 smithi125 ceph-mon[36819]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:58:10.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:09 smithi125 ceph-mon[36819]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:58:10.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:09 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:09 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:09 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:09 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:09 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:09 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:09 smithi094 ceph-mon[32499]: from='client.15489 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:58:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:09 smithi094 ceph-mon[32499]: from='client.15495 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:58:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:09 smithi094 ceph-mon[32499]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:58:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:09 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:09 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:09 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:09 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:09 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:09 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.774 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='client.15501 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:58:10.774 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: mgrmap e24: x(active, since 3s), standbys: y, z 2024-02-17T11:58:10.775 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.775 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:10.775 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:10.775 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='client.15501 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:58:11.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: mgrmap e24: x(active, since 3s), standbys: y, z 2024-02-17T11:58:11.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi189 to 174.1M 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: Unable to set osd_memory_target on smithi189 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:58:11.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:10 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:58:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi189 to 174.1M 2024-02-17T11:58:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: Unable to set osd_memory_target on smithi189 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-02-17T11:58:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:58:11.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:10 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:58:11.190 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='client.15501 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T11:58:11.190 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: mgrmap e24: x(active, since 3s), standbys: y, z 2024-02-17T11:58:11.190 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.190 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.190 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.190 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.190 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.190 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.190 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi189 to 174.1M 2024-02-17T11:58:11.190 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: Unable to set osd_memory_target on smithi189 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-02-17T11:58:11.190 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.190 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:11.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T11:58:11.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:58:11.191 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:10 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:58:11.303 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-02-17T11:58:11.344 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-02-17T11:58:11.347 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-02-17T11:58:11.430 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-02-17T11:58:11.515 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/: 2024-02-17T11:58:11.515 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:11.516 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 . 2024-02-17T11:58:11.516 INFO:teuthology.orchestra.run.smithi094.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 .. 2024-02-17T11:58:11.516 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 data 2024-02-17T11:58:11.516 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:11.516 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data: 2024-02-17T11:58:11.516 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:11.516 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:11.516 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 .. 2024-02-17T11:58:11.518 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-02-17T11:58:11.570 INFO:teuthology.orchestra.run.smithi094.stdout:[Service] 2024-02-17T11:58:11.570 INFO:teuthology.orchestra.run.smithi094.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-02-17T11:58:11.570 INFO:teuthology.orchestra.run.smithi094.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-02-17T11:58:11.573 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-02-17T11:58:11.659 INFO:teuthology.orchestra.run.smithi094.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-02-17T11:58:12.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi125 to 174.1M 2024-02-17T11:58:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: Unable to set osd_memory_target on smithi125 to 182611968: error parsing value: Value '182611968' is below minimum 939524096 2024-02-17T11:58:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-02-17T11:58:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: Unable to set osd_memory_target on smithi094 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-02-17T11:58:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-17T11:58:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: Updating smithi125:/etc/ceph/ceph.conf 2024-02-17T11:58:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-17T11:58:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: Updating smithi094:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:58:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:58:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:58:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:58:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:58:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: Updating smithi125:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:58:12.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:58:12.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-02-17T11:58:12.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:58:12.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:58:12.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:58:12.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:11 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi125 to 174.1M 2024-02-17T11:58:12.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: Unable to set osd_memory_target on smithi125 to 182611968: error parsing value: Value '182611968' is below minimum 939524096 2024-02-17T11:58:12.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-02-17T11:58:12.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: Unable to set osd_memory_target on smithi094 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-02-17T11:58:12.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-17T11:58:12.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: Updating smithi125:/etc/ceph/ceph.conf 2024-02-17T11:58:12.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-17T11:58:12.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: Updating smithi094:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:58:12.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:58:12.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:58:12.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:58:12.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:58:12.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: Updating smithi125:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:58:12.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:58:12.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-02-17T11:58:12.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:58:12.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:58:12.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:58:12.188 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:11 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi125 to 174.1M 2024-02-17T11:58:12.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: Unable to set osd_memory_target on smithi125 to 182611968: error parsing value: Value '182611968' is below minimum 939524096 2024-02-17T11:58:12.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-02-17T11:58:12.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: Unable to set osd_memory_target on smithi094 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-02-17T11:58:12.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: Updating smithi094:/etc/ceph/ceph.conf 2024-02-17T11:58:12.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: Updating smithi125:/etc/ceph/ceph.conf 2024-02-17T11:58:12.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: Updating smithi189:/etc/ceph/ceph.conf 2024-02-17T11:58:12.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: Updating smithi094:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:58:12.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:58:12.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.conf 2024-02-17T11:58:12.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: Updating smithi094:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: Updating smithi125:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: Updating smithi189:/etc/ceph/ceph.client.admin.keyring 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:58:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:11 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:58:13.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:12 smithi189 ceph-mon[36588]: Updating smithi094:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:58:13.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:12 smithi189 ceph-mon[36588]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:58:13.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:12 smithi189 ceph-mon[36588]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:58:13.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:12 smithi125 ceph-mon[36819]: Updating smithi094:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:58:13.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:12 smithi125 ceph-mon[36819]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:58:13.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:12 smithi125 ceph-mon[36819]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:58:13.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:12 smithi094 ceph-mon[32499]: Updating smithi094:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:58:13.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:12 smithi094 ceph-mon[32499]: Updating smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:58:13.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:12 smithi094 ceph-mon[32499]: Updating smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/config/ceph.client.admin.keyring 2024-02-17T11:58:13.996 INFO:teuthology.orchestra.run.smithi094.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-02-17T11:58:13.998 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-02-17T11:58:14.035 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:13 smithi094 ceph-mon[32499]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 105 KiB/s wr, 69 op/s 2024-02-17T11:58:14.044 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/: 2024-02-17T11:58:14.044 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.045 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 . 2024-02-17T11:58:14.045 INFO:teuthology.orchestra.run.smithi094.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 .. 2024-02-17T11:58:14.045 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 data 2024-02-17T11:58:14.045 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.045 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data: 2024-02-17T11:58:14.045 INFO:teuthology.orchestra.run.smithi094.stdout:total 35 2024-02-17T11:58:14.045 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 . 2024-02-17T11:58:14.045 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 .. 2024-02-17T11:58:14.045 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 base 2024-02-17T11:58:14.045 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Feb 17 11:58 global 2024-02-17T11:58:14.045 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 log 2024-02-17T11:58:14.045 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_commit_ts 2024-02-17T11:58:14.045 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_dynshmem 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Feb 17 11:58 pg_hba.conf 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Feb 17 11:58 pg_ident.conf 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 pg_logical 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 17 11:58 pg_multixact 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_notify 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_replslot 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_serial 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_snapshots 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_stat 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_stat_tmp 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 pg_subtrans 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_tblspc 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_twophase 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 PG_VERSION 2024-02-17T11:58:14.046 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 17 11:58 pg_wal 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 pg_xact 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Feb 17 11:58 postgresql.auto.conf 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Feb 17 11:58 postgresql.conf 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base: 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 . 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 17 11:58 1 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 17 11:58 13493 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 17 11:58 13494 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/1: 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:total 7777 2024-02-17T11:58:14.047 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 17 11:58 . 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 .. 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 112 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 113 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 17 11:58 1247 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1247_fsm 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1247_vm 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Feb 17 11:58 1249 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1249_fsm 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1249_vm 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Feb 17 11:58 1255 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1255_fsm 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1255_vm 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Feb 17 11:58 1259 2024-02-17T11:58:14.048 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1259_fsm 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1259_vm 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Feb 17 11:58 13344 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13344_fsm 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13344_vm 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13346 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13348 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13349 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13349_fsm 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13349_vm 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13351 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13353 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13354 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13354_fsm 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13354_vm 2024-02-17T11:58:14.049 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13356 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13358 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13359 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13359_fsm 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13359_vm 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13361 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13363 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 1417 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 1418 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 174 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 175 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2187 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2224 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2328 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2336 2024-02-17T11:58:14.050 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2337 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2579 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2600 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2600_fsm 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2600_vm 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2601 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2601_fsm 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2601_vm 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2602 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2602_fsm 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2602_vm 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2603 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2603_fsm 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2603_vm 2024-02-17T11:58:14.051 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2604 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2605 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2605_fsm 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2605_vm 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2606 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2606_fsm 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2606_vm 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2607 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2607_fsm 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2607_vm 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Feb 17 11:58 2608 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2608_fsm 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2608_vm 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Feb 17 11:58 2609 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2609_fsm 2024-02-17T11:58:14.052 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2609_vm 2024-02-17T11:58:14.053 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2610 2024-02-17T11:58:14.053 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2610_fsm 2024-02-17T11:58:14.053 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2610_vm 2024-02-17T11:58:14.053 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2611 2024-02-17T11:58:14.053 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2612 2024-02-17T11:58:14.053 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2612_fsm 2024-02-17T11:58:14.053 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2612_vm 2024-02-17T11:58:14.053 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2613 2024-02-17T11:58:14.053 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2615 2024-02-17T11:58:14.053 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2615_fsm 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2615_vm 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2616 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2616_fsm 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2616_vm 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Feb 17 11:58 2617 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2617_fsm 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2617_vm 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Feb 17 11:58 2618 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2618_fsm 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2618_vm 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Feb 17 11:58 2619 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2619_fsm 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2619_vm 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2620 2024-02-17T11:58:14.054 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2650 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2651 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2652 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2653 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2654 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2655 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2656 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2657 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Feb 17 11:58 2658 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 17 11:58 2659 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2660 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2661 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2662 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2663 2024-02-17T11:58:14.055 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2664 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2665 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2666 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2667 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2668 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2669 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2670 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Feb 17 11:58 2673 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Feb 17 11:58 2674 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Feb 17 11:58 2675 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2678 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2679 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2680 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2681 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2682 2024-02-17T11:58:14.056 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2683 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2684 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2685 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2686 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2687 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2688 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2689 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Feb 17 11:58 2690 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Feb 17 11:58 2691 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2692 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2693 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2696 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2699 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2701 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2702 2024-02-17T11:58:14.057 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2703 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2704 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2753 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2753_fsm 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2753_vm 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2754 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2755 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2756 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2757 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2830 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2831 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2832 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2833 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2834 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2835 2024-02-17T11:58:14.058 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2836 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2837 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Feb 17 11:58 2838 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2838_fsm 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2838_vm 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2839 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2840 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2840_fsm 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2840_vm 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2841 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2995 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2996 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3079 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3079_fsm 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3079_vm 2024-02-17T11:58:14.059 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3080 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3081 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 3085 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3118 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3119 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 17 11:58 3164 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3256 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3257 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3258 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3350 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3351 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3379 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3380 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3381 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3394 2024-02-17T11:58:14.060 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3394_fsm 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3394_vm 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3395 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3429 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3430 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3431 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3433 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3439 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3440 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 3455 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Feb 17 11:58 3456 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3456_fsm 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3456_vm 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3466 2024-02-17T11:58:14.061 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3467 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3468 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3501 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3502 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3503 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3534 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3541 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3541_fsm 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3541_vm 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3542 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3574 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3575 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3576 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3596 2024-02-17T11:58:14.062 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3597 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3598 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3599 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3600 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3600_fsm 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3600_vm 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3601 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3601_fsm 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3601_vm 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3602 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3602_fsm 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3602_vm 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3603 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3603_fsm 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3603_vm 2024-02-17T11:58:14.063 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3604 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3605 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3606 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3607 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3608 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 3609 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3712 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3764 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3764_fsm 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3764_vm 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3766 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3767 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3997 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4143 2024-02-17T11:58:14.064 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4144 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4145 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4146 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4147 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4148 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4149 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4150 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4151 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4152 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4153 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4154 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4155 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4156 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4157 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4158 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4159 2024-02-17T11:58:14.065 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4160 2024-02-17T11:58:14.066 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4161 2024-02-17T11:58:14.066 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4162 2024-02-17T11:58:14.066 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4163 2024-02-17T11:58:14.066 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4164 2024-02-17T11:58:14.066 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4165 2024-02-17T11:58:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4166 2024-02-17T11:58:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4167 2024-02-17T11:58:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4168 2024-02-17T11:58:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4169 2024-02-17T11:58:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4170 2024-02-17T11:58:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4171 2024-02-17T11:58:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4172 2024-02-17T11:58:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4173 2024-02-17T11:58:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4174 2024-02-17T11:58:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 5002 2024-02-17T11:58:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 548 2024-02-17T11:58:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 549 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6102 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6104 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6106 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6110 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6111 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6112 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6113 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6117 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 826 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 827 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 828 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 17 11:58 pg_filenode.map 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 PG_VERSION 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.070 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:total 7777 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 17 11:58 . 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 .. 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 112 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 113 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 17 11:58 1247 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1247_fsm 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1247_vm 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Feb 17 11:58 1249 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1249_fsm 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1249_vm 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Feb 17 11:58 1255 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1255_fsm 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1255_vm 2024-02-17T11:58:14.071 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Feb 17 11:58 1259 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1259_fsm 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1259_vm 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Feb 17 11:58 13344 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13344_fsm 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13344_vm 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13346 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13348 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13349 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13349_fsm 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13349_vm 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13351 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13353 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13354 2024-02-17T11:58:14.072 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13354_fsm 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13354_vm 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13356 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13358 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13359 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13359_fsm 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13359_vm 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13361 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13363 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 1417 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 1418 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 174 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 175 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2187 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2224 2024-02-17T11:58:14.073 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2328 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2336 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2337 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2579 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2600 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2600_fsm 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2600_vm 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2601 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2601_fsm 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2601_vm 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2602 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2602_fsm 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2602_vm 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2603 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2603_fsm 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2603_vm 2024-02-17T11:58:14.074 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2604 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2605 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2605_fsm 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2605_vm 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2606 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2606_fsm 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2606_vm 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2607 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2607_fsm 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2607_vm 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Feb 17 11:58 2608 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2608_fsm 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2608_vm 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Feb 17 11:58 2609 2024-02-17T11:58:14.075 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2609_fsm 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2609_vm 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2610 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2610_fsm 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2610_vm 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2611 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2612 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2612_fsm 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2612_vm 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2613 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2615 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2615_fsm 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2615_vm 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2616 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2616_fsm 2024-02-17T11:58:14.076 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2616_vm 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Feb 17 11:58 2617 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2617_fsm 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2617_vm 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Feb 17 11:58 2618 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2618_fsm 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2618_vm 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Feb 17 11:58 2619 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2619_fsm 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2619_vm 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2620 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2650 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2651 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2652 2024-02-17T11:58:14.077 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2653 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2654 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2655 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2656 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2657 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Feb 17 11:58 2658 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 17 11:58 2659 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2660 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2661 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2662 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2663 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2664 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2665 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2666 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2667 2024-02-17T11:58:14.078 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2668 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2669 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2670 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Feb 17 11:58 2673 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Feb 17 11:58 2674 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Feb 17 11:58 2675 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2678 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2679 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2680 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2681 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2682 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2683 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2684 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2685 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2686 2024-02-17T11:58:14.079 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2687 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2688 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2689 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Feb 17 11:58 2690 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Feb 17 11:58 2691 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2692 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2693 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2696 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2699 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2701 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2702 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2703 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2704 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2753 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2753_fsm 2024-02-17T11:58:14.080 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2753_vm 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2754 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2755 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2756 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2757 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2830 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2831 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2832 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2833 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2834 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2835 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2836 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2837 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Feb 17 11:58 2838 2024-02-17T11:58:14.081 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2838_fsm 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2838_vm 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2839 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2840 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2840_fsm 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2840_vm 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2841 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2995 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2996 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3079 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3079_fsm 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3079_vm 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3080 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3081 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 3085 2024-02-17T11:58:14.082 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3118 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3119 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 17 11:58 3164 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3256 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3257 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3258 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3350 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3351 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3379 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3380 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3381 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3394 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3394_fsm 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3394_vm 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3395 2024-02-17T11:58:14.083 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3429 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3430 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3431 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3433 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3439 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3440 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 3455 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Feb 17 11:58 3456 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3456_fsm 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3456_vm 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3466 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3467 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3468 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3501 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3502 2024-02-17T11:58:14.084 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3503 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3534 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3541 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3541_fsm 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3541_vm 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3542 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3574 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3575 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3576 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3596 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3597 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3598 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3599 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3600 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3600_fsm 2024-02-17T11:58:14.085 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3600_vm 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3601 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3601_fsm 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3601_vm 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3602 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3602_fsm 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3602_vm 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3603 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3603_fsm 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3603_vm 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3604 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3605 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3606 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3607 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3608 2024-02-17T11:58:14.086 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 3609 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3712 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3764 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3764_fsm 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3764_vm 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3766 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3767 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3997 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4143 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4144 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4145 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4146 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4147 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4148 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4149 2024-02-17T11:58:14.087 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4150 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4151 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4152 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4153 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4154 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4155 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4156 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4157 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4158 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4159 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4160 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4161 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4162 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4163 2024-02-17T11:58:14.088 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4164 2024-02-17T11:58:14.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4165 2024-02-17T11:58:14.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4166 2024-02-17T11:58:14.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4167 2024-02-17T11:58:14.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4168 2024-02-17T11:58:14.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4169 2024-02-17T11:58:14.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4170 2024-02-17T11:58:14.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4171 2024-02-17T11:58:14.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4172 2024-02-17T11:58:14.089 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4173 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4174 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 5002 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 548 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 549 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6102 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6104 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6106 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6110 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6111 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6112 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6113 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6117 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 826 2024-02-17T11:58:14.091 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 827 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 828 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 17 11:58 pg_filenode.map 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 PG_VERSION 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:total 7777 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 17 11:58 . 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 .. 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 112 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 113 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 17 11:58 1247 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1247_fsm 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1247_vm 2024-02-17T11:58:14.092 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Feb 17 11:58 1249 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1249_fsm 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1249_vm 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Feb 17 11:58 1255 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1255_fsm 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1255_vm 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Feb 17 11:58 1259 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1259_fsm 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1259_vm 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Feb 17 11:58 13344 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13344_fsm 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13344_vm 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13346 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13348 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13349 2024-02-17T11:58:14.093 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13349_fsm 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13349_vm 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13351 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13353 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13354 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13354_fsm 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13354_vm 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13356 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13358 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13359 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13359_fsm 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13359_vm 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13361 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13363 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 1417 2024-02-17T11:58:14.094 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 1418 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 174 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 175 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2187 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2224 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2328 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2336 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2337 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2579 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2600 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2600_fsm 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2600_vm 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2601 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2601_fsm 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2601_vm 2024-02-17T11:58:14.095 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2602 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2602_fsm 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2602_vm 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2603 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2603_fsm 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2603_vm 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2604 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2605 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2605_fsm 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2605_vm 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2606 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2606_fsm 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2606_vm 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2607 2024-02-17T11:58:14.096 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2607_fsm 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2607_vm 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Feb 17 11:58 2608 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2608_fsm 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2608_vm 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Feb 17 11:58 2609 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2609_fsm 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2609_vm 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2610 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2610_fsm 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2610_vm 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2611 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2612 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2612_fsm 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2612_vm 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2613 2024-02-17T11:58:14.097 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2615 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2615_fsm 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2615_vm 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2616 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2616_fsm 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2616_vm 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Feb 17 11:58 2617 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2617_fsm 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2617_vm 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Feb 17 11:58 2618 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2618_fsm 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2618_vm 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Feb 17 11:58 2619 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2619_fsm 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2619_vm 2024-02-17T11:58:14.098 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2620 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2650 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2651 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2652 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2653 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2654 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2655 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2656 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2657 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Feb 17 11:58 2658 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 17 11:58 2659 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2660 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2661 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2662 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2663 2024-02-17T11:58:14.099 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2664 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2665 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2666 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2667 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2668 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2669 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2670 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Feb 17 11:58 2673 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Feb 17 11:58 2674 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Feb 17 11:58 2675 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2678 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2679 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2680 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2681 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2682 2024-02-17T11:58:14.100 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2683 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2684 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2685 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2686 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2687 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2688 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2689 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Feb 17 11:58 2690 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Feb 17 11:58 2691 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2692 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2693 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2696 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2699 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2701 2024-02-17T11:58:14.101 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2702 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2703 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2704 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2753 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2753_fsm 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2753_vm 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2754 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2755 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2756 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2757 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2830 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2831 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2832 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2833 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2834 2024-02-17T11:58:14.102 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2835 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2836 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2837 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Feb 17 11:58 2838 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2838_fsm 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2838_vm 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2839 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2840 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2840_fsm 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2840_vm 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2841 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2995 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2996 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3079 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3079_fsm 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3079_vm 2024-02-17T11:58:14.103 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3080 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3081 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 3085 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3118 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3119 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 17 11:58 3164 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3256 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3257 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3258 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3350 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3351 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3379 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3380 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3381 2024-02-17T11:58:14.104 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3394 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3394_fsm 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3394_vm 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3395 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3429 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3430 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3431 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3433 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3439 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3440 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 3455 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Feb 17 11:58 3456 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3456_fsm 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3456_vm 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3466 2024-02-17T11:58:14.105 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3467 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3468 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3501 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3502 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3503 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3534 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3541 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3541_fsm 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3541_vm 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3542 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3574 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3575 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3576 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3596 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3597 2024-02-17T11:58:14.106 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3598 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3599 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3600 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3600_fsm 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3600_vm 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3601 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3601_fsm 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3601_vm 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3602 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3602_fsm 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3602_vm 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3603 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3603_fsm 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3603_vm 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3604 2024-02-17T11:58:14.107 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3605 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3606 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3607 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3608 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 3609 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3712 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3764 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3764_fsm 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3764_vm 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3766 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3767 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3997 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4143 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4144 2024-02-17T11:58:14.108 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4145 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4146 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4147 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4148 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4149 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4150 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4151 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4152 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4153 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4154 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4155 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4156 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4157 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4158 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4159 2024-02-17T11:58:14.109 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4160 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4161 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4162 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4163 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4164 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4165 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4166 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4167 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4168 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4169 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4170 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4171 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4172 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4173 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4174 2024-02-17T11:58:14.110 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 5002 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 548 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 549 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6102 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6104 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6106 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6110 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6111 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6112 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6113 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6117 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 826 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 827 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 828 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 17 11:58 pg_filenode.map 2024-02-17T11:58:14.111 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 PG_VERSION 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/global: 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:total 537 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Feb 17 11:58 . 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1213 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1213_fsm 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1213_vm 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1214 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1214_fsm 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1214_vm 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 1232 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 1233 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1260 2024-02-17T11:58:14.112 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1260_fsm 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1260_vm 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1261 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1261_fsm 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1261_vm 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1262 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1262_fsm 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1262_vm 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2396 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2396_fsm 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2396_vm 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2397 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2671 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2672 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2676 2024-02-17T11:58:14.113 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2677 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2694 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2695 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2697 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2698 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2846 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2847 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2964 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2965 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2966 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2967 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3592 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3593 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4060 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4061 2024-02-17T11:58:14.114 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4175 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4176 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4177 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4178 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4181 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4182 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4183 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4184 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4185 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4186 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6000 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6001 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6002 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6100 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6114 2024-02-17T11:58:14.115 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6115 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 pg_control 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 17 11:58 pg_filenode.map 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/log: 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.116 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 . 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 mappings 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Feb 17 11:58 replorigin_checkpoint 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 snapshots 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.117 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 .. 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 .. 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 17 11:58 . 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 members 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 offsets 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-02-17T11:58:14.118 INFO:teuthology.orchestra.run.smithi094.stdout:total 8 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 . 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 17 11:58 .. 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 0000 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout:total 8 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 . 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 17 11:58 .. 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 0000 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.119 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.120 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout:total 8 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 . 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.121 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 0000 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout:total 16384 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 17 11:58 . 2024-02-17T11:58:14.122 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 11:58 000000010000000000000001 2024-02-17T11:58:14.123 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 archive_status 2024-02-17T11:58:14.123 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.123 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-02-17T11:58:14.123 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T11:58:14.123 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T11:58:14.123 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 17 11:58 .. 2024-02-17T11:58:14.123 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:14.123 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-02-17T11:58:14.123 INFO:teuthology.orchestra.run.smithi094.stdout:total 8 2024-02-17T11:58:14.123 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 . 2024-02-17T11:58:14.123 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Feb 17 11:58 .. 2024-02-17T11:58:14.123 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 0000 2024-02-17T11:58:14.124 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-02-17T11:58:14.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:13 smithi189 ceph-mon[36588]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 105 KiB/s wr, 69 op/s 2024-02-17T11:58:14.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:13 smithi125 ceph-mon[36819]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 105 KiB/s wr, 69 op/s 2024-02-17T11:58:14.263 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-02-17T11:58:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:dropping old tables... 2024-02-17T11:58:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-02-17T11:58:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-02-17T11:58:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-02-17T11:58:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-02-17T11:58:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:creating tables... 2024-02-17T11:58:14.318 INFO:teuthology.orchestra.run.smithi094.stderr:generating data (client-side)... 2024-02-17T11:58:14.387 INFO:teuthology.orchestra.run.smithi094.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.74 s) 2024-02-17T11:58:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.21 s, remaining 3.18 s) 2024-02-17T11:58:14.621 INFO:teuthology.orchestra.run.smithi094.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.29 s, remaining 2.80 s) 2024-02-17T11:58:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.43 s, remaining 3.04 s) 2024-02-17T11:58:14.911 INFO:teuthology.orchestra.run.smithi094.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.58 s, remaining 3.13 s) 2024-02-17T11:58:14.992 INFO:teuthology.orchestra.run.smithi094.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.66 s, remaining 2.87 s) 2024-02-17T11:58:15.131 INFO:teuthology.orchestra.run.smithi094.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.80 s, remaining 2.86 s) 2024-02-17T11:58:15.290 INFO:teuthology.orchestra.run.smithi094.stderr:800000 of 3200000 tuples (25%) done (elapsed 0.96 s, remaining 2.88 s) 2024-02-17T11:58:15.375 INFO:teuthology.orchestra.run.smithi094.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.04 s, remaining 2.67 s) 2024-02-17T11:58:15.499 INFO:teuthology.orchestra.run.smithi094.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.17 s, remaining 2.57 s) 2024-02-17T11:58:15.683 INFO:teuthology.orchestra.run.smithi094.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.35 s, remaining 2.58 s) 2024-02-17T11:58:15.755 INFO:teuthology.orchestra.run.smithi094.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.42 s, remaining 2.37 s) 2024-02-17T11:58:15.919 INFO:teuthology.orchestra.run.smithi094.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.59 s, remaining 2.32 s) 2024-02-17T11:58:16.114 INFO:teuthology.orchestra.run.smithi094.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.78 s, remaining 2.29 s) 2024-02-17T11:58:16.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:15 smithi189 ceph-mon[36588]: pgmap v7: 97 pgs: 97 active+clean; 2.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 279 KiB/s wr, 82 op/s 2024-02-17T11:58:16.171 INFO:teuthology.orchestra.run.smithi094.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.84 s, remaining 2.09 s) 2024-02-17T11:58:16.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:15 smithi125 ceph-mon[36819]: pgmap v7: 97 pgs: 97 active+clean; 2.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 279 KiB/s wr, 82 op/s 2024-02-17T11:58:16.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:15 smithi094 ceph-mon[32499]: pgmap v7: 97 pgs: 97 active+clean; 2.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 279 KiB/s wr, 82 op/s 2024-02-17T11:58:16.337 INFO:teuthology.orchestra.run.smithi094.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.01 s, remaining 2.01 s) 2024-02-17T11:58:16.487 INFO:teuthology.orchestra.run.smithi094.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.16 s, remaining 1.90 s) 2024-02-17T11:58:16.572 INFO:teuthology.orchestra.run.smithi094.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.24 s, remaining 1.74 s) 2024-02-17T11:58:16.717 INFO:teuthology.orchestra.run.smithi094.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.39 s, remaining 1.63 s) 2024-02-17T11:58:16.824 INFO:teuthology.orchestra.run.smithi094.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.49 s, remaining 1.50 s) 2024-02-17T11:58:16.962 INFO:teuthology.orchestra.run.smithi094.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.63 s, remaining 1.38 s) 2024-02-17T11:58:17.124 INFO:teuthology.orchestra.run.smithi094.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.79 s, remaining 1.27 s) 2024-02-17T11:58:17.219 INFO:teuthology.orchestra.run.smithi094.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.89 s, remaining 1.13 s) 2024-02-17T11:58:17.346 INFO:teuthology.orchestra.run.smithi094.stderr:2400000 of 3200000 tuples (75%) done (elapsed 3.02 s, remaining 1.01 s) 2024-02-17T11:58:17.507 INFO:teuthology.orchestra.run.smithi094.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.18 s, remaining 0.89 s) 2024-02-17T11:58:17.601 INFO:teuthology.orchestra.run.smithi094.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.27 s, remaining 0.75 s) 2024-02-17T11:58:17.728 INFO:teuthology.orchestra.run.smithi094.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.40 s, remaining 0.63 s) 2024-02-17T11:58:17.840 INFO:teuthology.orchestra.run.smithi094.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.51 s, remaining 0.50 s) 2024-02-17T11:58:17.977 INFO:teuthology.orchestra.run.smithi094.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.65 s, remaining 0.38 s) 2024-02-17T11:58:18.131 INFO:teuthology.orchestra.run.smithi094.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.80 s, remaining 0.25 s) 2024-02-17T11:58:18.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:17 smithi189 ceph-mon[36588]: pgmap v8: 97 pgs: 97 active+clean; 2.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 217 KiB/s wr, 64 op/s 2024-02-17T11:58:18.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:17 smithi125 ceph-mon[36819]: pgmap v8: 97 pgs: 97 active+clean; 2.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 217 KiB/s wr, 64 op/s 2024-02-17T11:58:18.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:17 smithi094 ceph-mon[32499]: pgmap v8: 97 pgs: 97 active+clean; 2.2 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 217 KiB/s wr, 64 op/s 2024-02-17T11:58:18.271 INFO:teuthology.orchestra.run.smithi094.stderr:3100000 of 3200000 tuples (96%) done (elapsed 3.94 s, remaining 0.13 s) 2024-02-17T11:58:18.351 INFO:teuthology.orchestra.run.smithi094.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.02 s, remaining 0.00 s) 2024-02-17T11:58:18.387 INFO:teuthology.orchestra.run.smithi094.stderr:vacuuming... 2024-02-17T11:58:18.992 INFO:teuthology.orchestra.run.smithi094.stderr:creating primary keys... 2024-02-17T11:58:20.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:19 smithi189 ceph-mon[36588]: pgmap v9: 97 pgs: 97 active+clean; 317 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 54 MiB/s wr, 231 op/s 2024-02-17T11:58:20.181 INFO:teuthology.orchestra.run.smithi094.stderr:done in 5.87 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 4.07 s, vacuum 0.60 s, primary keys 1.19 s). 2024-02-17T11:58:20.181 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:19 smithi094 ceph-mon[32499]: pgmap v9: 97 pgs: 97 active+clean; 317 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 54 MiB/s wr, 231 op/s 2024-02-17T11:58:20.184 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-02-17T11:58:20.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:19 smithi125 ceph-mon[36819]: pgmap v9: 97 pgs: 97 active+clean; 317 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 54 MiB/s wr, 231 op/s 2024-02-17T11:58:20.241 INFO:teuthology.orchestra.run.smithi094.stderr:starting vacuum...end. 2024-02-17T11:58:22.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:21 smithi189 ceph-mon[36588]: pgmap v10: 97 pgs: 97 active+clean; 317 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 49 MiB/s wr, 211 op/s 2024-02-17T11:58:22.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:21 smithi125 ceph-mon[36819]: pgmap v10: 97 pgs: 97 active+clean; 317 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 49 MiB/s wr, 211 op/s 2024-02-17T11:58:22.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:21 smithi094 ceph-mon[32499]: pgmap v10: 97 pgs: 97 active+clean; 317 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 49 MiB/s wr, 211 op/s 2024-02-17T11:58:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.212+0000 7f967b96c640 1 -- 172.21.15.94:0/1549611002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96741002d0 msgr2=0x7f9674100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.212+0000 7f967b96c640 1 --2- 172.21.15.94:0/1549611002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96741002d0 0x7f9674100730 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f9664009900 tx=0x7f966402bc80 comp rx=0 tx=0).stop 2024-02-17T11:58:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.212+0000 7f967b96c640 1 -- 172.21.15.94:0/1549611002 shutdown_connections 2024-02-17T11:58:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.212+0000 7f967b96c640 1 --2- 172.21.15.94:0/1549611002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9674100c70 0x7f967410ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.212+0000 7f967b96c640 1 --2- 172.21.15.94:0/1549611002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96741002d0 0x7f9674100730 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.212+0000 7f967b96c640 1 --2- 172.21.15.94:0/1549611002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9674105040 0x7f9674105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.212+0000 7f967b96c640 1 -- 172.21.15.94:0/1549611002 >> 172.21.15.94:0/1549611002 conn(0x7f9674076ee0 msgr2=0x7f96740772f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.212+0000 7f967b96c640 1 -- 172.21.15.94:0/1549611002 shutdown_connections 2024-02-17T11:58:22.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.212+0000 7f967b96c640 1 -- 172.21.15.94:0/1549611002 wait complete. 2024-02-17T11:58:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f967b96c640 1 Processor -- start 2024-02-17T11:58:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f967b96c640 1 -- start start 2024-02-17T11:58:22.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f967b96c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96741002d0 0x7f96741966c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f967b96c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9674100c70 0x7f9674196c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f967b96c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9674105040 0x7f967419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f967b96c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967419b530 con 0x7f9674105040 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f967b96c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967419b6a0 con 0x7f9674100c70 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f967b96c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967419b860 con 0x7f96741002d0 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f9679f09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9674105040 0x7f967419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f9679f09640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9674105040 0x7f967419af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35900/0 (socket says 172.21.15.94:35900) 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f9679f09640 1 -- 172.21.15.94:0/2381770789 learned_addr learned my addr 172.21.15.94:0/2381770789 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f9678f07640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9674100c70 0x7f9674196c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f9679708640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96741002d0 0x7f96741966c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f9679f09640 1 -- 172.21.15.94:0/2381770789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96741002d0 msgr2=0x7f96741966c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f9679f09640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96741002d0 0x7f96741966c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f9679f09640 1 -- 172.21.15.94:0/2381770789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9674100c70 msgr2=0x7f9674196c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f9679f09640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9674100c70 0x7f9674196c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f9679f09640 1 -- 172.21.15.94:0/2381770789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9664009560 con 0x7f9674105040 2024-02-17T11:58:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.213+0000 7f9678f07640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9674100c70 0x7f9674196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.214+0000 7f9679f09640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9674105040 0x7f967419af30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f966c00ea60 tx=0x7f966c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.214+0000 7f966a7fc640 1 -- 172.21.15.94:0/2381770789 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c00ce40 con 0x7f9674105040 2024-02-17T11:58:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.214+0000 7f966a7fc640 1 -- 172.21.15.94:0/2381770789 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f966c005020 con 0x7f9674105040 2024-02-17T11:58:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.214+0000 7f967b96c640 1 -- 172.21.15.94:0/2381770789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967419bb40 con 0x7f9674105040 2024-02-17T11:58:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.214+0000 7f966a7fc640 1 -- 172.21.15.94:0/2381770789 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966c010690 con 0x7f9674105040 2024-02-17T11:58:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.214+0000 7f967b96c640 1 -- 172.21.15.94:0/2381770789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96741a33c0 con 0x7f9674105040 2024-02-17T11:58:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.217+0000 7f967e61e740 1 Processor -- start 2024-02-17T11:58:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.217+0000 7f966a7fc640 1 -- 172.21.15.94:0/2381770789 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f966c005500 con 0x7f9674105040 2024-02-17T11:58:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.217+0000 7f967e61e740 1 -- start start 2024-02-17T11:58:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f967e61e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea280ad0 0x55b2ea27f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f967e61e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea27fd60 0x55b2ea2801e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f967e61e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea2892e0 0x55b2ea28b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f967e61e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ea1ee9f0 con 0x55b2ea280ad0 2024-02-17T11:58:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f967e61e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ea1ebf20 con 0x55b2ea27fd60 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f967e61e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ea1e7700 con 0x55b2ea2892e0 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f9678f07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea280ad0 0x55b2ea27f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f9678f07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea280ad0 0x55b2ea27f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35904/0 (socket says 172.21.15.94:35904) 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f9678f07640 1 -- 172.21.15.94:0/2015081988 learned_addr learned my addr 172.21.15.94:0/2015081988 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f9679708640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea27fd60 0x55b2ea2801e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f9678f07640 1 --2- 172.21.15.94:0/2015081988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea2892e0 0x55b2ea28b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f966a7fc640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96600a2370 0x7f96600a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f9679708640 1 -- 172.21.15.94:0/2015081988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea2892e0 msgr2=0x55b2ea28b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f9679708640 1 --2- 172.21.15.94:0/2015081988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea2892e0 0x55b2ea28b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f9679708640 1 -- 172.21.15.94:0/2015081988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea280ad0 msgr2=0x55b2ea27f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f9679708640 1 --2- 172.21.15.94:0/2015081988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea280ad0 0x55b2ea27f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f9679708640 1 -- 172.21.15.94:0/2015081988 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2ea17d7b0 con 0x55b2ea27fd60 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f9678f07640 1 --2- 172.21.15.94:0/2015081988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea2892e0 0x55b2ea28b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:58:22.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f966a7fc640 1 -- 172.21.15.94:0/2381770789 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f966c014070 con 0x7f9674105040 2024-02-17T11:58:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f9679f09640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96600a2370 0x7f96600a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.218+0000 7f9679708640 1 --2- 172.21.15.94:0/2015081988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea27fd60 0x55b2ea2801e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9670009c20 tx=0x7f9670011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ec49f858278d4e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.219+0000 7f965f7fe640 1 -- 172.21.15.94:0/2015081988 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670004050 con 0x55b2ea27fd60 2024-02-17T11:58:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.219+0000 7f965f7fe640 1 -- 172.21.15.94:0/2015081988 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9670005500 con 0x55b2ea27fd60 2024-02-17T11:58:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.219+0000 7f965f7fe640 1 -- 172.21.15.94:0/2015081988 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670004050 con 0x55b2ea27fd60 2024-02-17T11:58:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.220+0000 7f967e61e740 1 -- 172.21.15.94:0/2015081988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea27fd60 msgr2=0x55b2ea2801e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.220+0000 7f9679f09640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96600a2370 0x7f96600a4830 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f966c0cbf00 tx=0x7f966c01df90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.220+0000 7f967e61e740 1 --2- 172.21.15.94:0/2015081988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea27fd60 0x55b2ea2801e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9670009c20 tx=0x7f9670011810 comp rx=0 tx=0).stop 2024-02-17T11:58:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.220+0000 7f967e61e740 1 -- 172.21.15.94:0/2015081988 shutdown_connections 2024-02-17T11:58:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.220+0000 7f967e61e740 1 --2- 172.21.15.94:0/2015081988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea2892e0 0x55b2ea28b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.220+0000 7f967e61e740 1 --2- 172.21.15.94:0/2015081988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea27fd60 0x55b2ea2801e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.220+0000 7f967e61e740 1 --2- 172.21.15.94:0/2015081988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea280ad0 0x55b2ea27f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.220+0000 7f967e61e740 1 -- 172.21.15.94:0/2015081988 >> 172.21.15.94:0/2015081988 conn(0x55b2ea247550 msgr2=0x55b2ea27aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.220+0000 7f967e61e740 1 -- 172.21.15.94:0/2015081988 shutdown_connections 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.221+0000 7f967e61e740 1 -- 172.21.15.94:0/2015081988 wait complete. 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.221+0000 7f967e61e740 1 -- start start 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.221+0000 7f967e61e740 1 Processor -- start 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.221+0000 7f967e61e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea27fd60 0x55b2ea2862e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.221+0000 7f967e61e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea280ad0 0x55b2ea286820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.221+0000 7f967e61e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea2892e0 0x55b2ea286d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.221+0000 7f967e61e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ea18cbf0 con 0x55b2ea2892e0 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.221+0000 7f967e61e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2e9ebb250 con 0x55b2ea280ad0 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.221+0000 7f967e61e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ea16d200 con 0x55b2ea27fd60 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.221+0000 7f967e61e740 10 client.0 _ll_register_callbacks cb 0x55b2ea1dbd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:22.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.221+0000 7f9678f07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea2892e0 0x55b2ea286d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.221+0000 7f9679708640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea280ad0 0x55b2ea286820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.221+0000 7f9678f07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea2892e0 0x55b2ea286d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35916/0 (socket says 172.21.15.94:35916) 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f9678f07640 1 -- 172.21.15.94:0/1241480825 learned_addr learned my addr 172.21.15.94:0/1241480825 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f9678f07640 1 --2- 172.21.15.94:0/1241480825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea27fd60 0x55b2ea2862e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f9679708640 1 -- 172.21.15.94:0/1241480825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea27fd60 msgr2=0x55b2ea2862e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f9679708640 1 --2- 172.21.15.94:0/1241480825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea27fd60 0x55b2ea2862e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f9679708640 1 -- 172.21.15.94:0/1241480825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea2892e0 msgr2=0x55b2ea286d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f9679708640 1 --2- 172.21.15.94:0/1241480825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea2892e0 0x55b2ea286d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f9679708640 1 -- 172.21.15.94:0/1241480825 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9670009850 con 0x55b2ea280ad0 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f9678f07640 1 --2- 172.21.15.94:0/1241480825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea2892e0 0x55b2ea286d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f9679708640 1 --2- 172.21.15.94:0/1241480825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea280ad0 0x55b2ea286820 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9670011d10 tx=0x7f96700038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd631aed3570c4bb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f965cff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f965cff9640 1 -- 172.21.15.94:0/1241480825 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670004050 con 0x55b2ea280ad0 2024-02-17T11:58:22.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f967e61e740 1 -- 172.21.15.94:0/1241480825 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b2e9ee7600 con 0x55b2ea280ad0 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f965cff9640 1 -- 172.21.15.94:0/1241480825 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f96700046c0 con 0x55b2ea280ad0 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f965cff9640 1 -- 172.21.15.94:0/1241480825 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9670003ca0 con 0x55b2ea280ad0 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f967e61e740 10 client.25016 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.222+0000 7f967e61e740 1 -- 172.21.15.94:0/1241480825 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55b2ea1e5d80 con 0x55b2ea280ad0 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.223+0000 7f965cff9640 1 -- 172.21.15.94:0/1241480825 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b2e9ee7600 con 0x55b2ea280ad0 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.223+0000 7f965cff9640 1 -- 172.21.15.94:0/1241480825 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9670021040 con 0x55b2ea280ad0 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.223+0000 7f967e61e740 10 client.25016 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.223+0000 7f967e61e740 10 client.25016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.223+0000 7f967e61e740 20 client.25016 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.223+0000 7f967e61e740 1 --2- 172.21.15.94:0/1241480825 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b2ea299660 0x55b2ea29ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.223+0000 7f967e61e740 4 client.25016 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.223+0000 7f967e61e740 1 -- 172.21.15.94:0/1241480825 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b2e9f381b0 con 0x55b2ea299660 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.224+0000 7f9678f07640 1 --2- 172.21.15.94:0/1241480825 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b2ea299660 0x55b2ea29ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.224+0000 7f9678f07640 1 --2- 172.21.15.94:0/1241480825 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b2ea299660 0x55b2ea29ba80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=441b78b6beb794b6 server_cookie=e11c1d7cf53af487 in_seq=0 out_seq=0 2024-02-17T11:58:22.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.224+0000 7f965cff9640 10 client.25016 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f965cff9640 1 -- 172.21.15.94:0/1241480825 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f966404ea90 con 0x55b2ea299660 2024-02-17T11:58:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f965cff9640 10 client.25016 handle_command_reply: tid=0 2024-02-17T11:58:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f967e61e740 1 client.25016 shutdown 2024-02-17T11:58:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f967e61e740 1 -- 172.21.15.94:0/1241480825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea280ad0 msgr2=0x55b2ea286820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f967e61e740 1 --2- 172.21.15.94:0/1241480825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea280ad0 0x55b2ea286820 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9670011d10 tx=0x7f96700038a0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f967e61e740 1 -- 172.21.15.94:0/1241480825 shutdown_connections 2024-02-17T11:58:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f967e61e740 1 --2- 172.21.15.94:0/1241480825 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b2ea299660 0x55b2ea29ba80 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f965cff9640 0 client.25016 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f967e61e740 1 --2- 172.21.15.94:0/1241480825 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea2892e0 0x55b2ea286d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f967e61e740 1 --2- 172.21.15.94:0/1241480825 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea280ad0 0x55b2ea286820 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f967e61e740 1 --2- 172.21.15.94:0/1241480825 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea27fd60 0x55b2ea2862e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f967e61e740 1 -- 172.21.15.94:0/1241480825 >> 172.21.15.94:0/1241480825 conn(0x55b2ea247550 msgr2=0x55b2ea27c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f967e61e740 1 -- 172.21.15.94:0/1241480825 shutdown_connections 2024-02-17T11:58:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.225+0000 7f967e61e740 1 -- 172.21.15.94:0/1241480825 wait complete. 2024-02-17T11:58:22.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.226+0000 7f967e61e740 20 client.25016 trim_cache size 0 max 16384 2024-02-17T11:58:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.242+0000 7f967e61e740 1 Processor -- start 2024-02-17T11:58:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.242+0000 7f967e61e740 1 -- start start 2024-02-17T11:58:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.242+0000 7f967e61e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea27fd60 0x55b2ea288a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.242+0000 7f967e61e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea280ad0 0x55b2ea294830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.242+0000 7f967e61e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea2892e0 0x55b2ea286830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.242+0000 7f967e61e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ea281f30 con 0x55b2ea27fd60 2024-02-17T11:58:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.242+0000 7f967e61e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ea284860 con 0x55b2ea280ad0 2024-02-17T11:58:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.242+0000 7f967e61e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ea28b6d0 con 0x55b2ea2892e0 2024-02-17T11:58:22.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.242+0000 7f9678f07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea27fd60 0x55b2ea288a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.242+0000 7f9678f07640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea27fd60 0x55b2ea288a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35922/0 (socket says 172.21.15.94:35922) 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.242+0000 7f9678f07640 1 -- 172.21.15.94:0/1234783993 learned_addr learned my addr 172.21.15.94:0/1234783993 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.243+0000 7f9679708640 1 --2- 172.21.15.94:0/1234783993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea280ad0 0x55b2ea294830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.243+0000 7f9678f07640 1 --2- 172.21.15.94:0/1234783993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea2892e0 0x55b2ea286830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.243+0000 7f9679708640 1 -- 172.21.15.94:0/1234783993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea2892e0 msgr2=0x55b2ea286830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.243+0000 7f9679708640 1 --2- 172.21.15.94:0/1234783993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea2892e0 0x55b2ea286830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.243+0000 7f9679708640 1 -- 172.21.15.94:0/1234783993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea27fd60 msgr2=0x55b2ea288a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.243+0000 7f9679708640 1 --2- 172.21.15.94:0/1234783993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea27fd60 0x55b2ea288a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.243+0000 7f9679708640 1 -- 172.21.15.94:0/1234783993 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b2ea1827b0 con 0x55b2ea280ad0 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.243+0000 7f9678f07640 1 --2- 172.21.15.94:0/1234783993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea27fd60 0x55b2ea288a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.243+0000 7f9679708640 1 --2- 172.21.15.94:0/1234783993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea280ad0 0x55b2ea294830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9670018460 tx=0x7f9670011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72881761bf4fa60e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.243+0000 7f965effd640 1 -- 172.21.15.94:0/1234783993 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670003910 con 0x55b2ea280ad0 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.243+0000 7f965effd640 1 -- 172.21.15.94:0/1234783993 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f967000fe70 con 0x55b2ea280ad0 2024-02-17T11:58:22.245 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.243+0000 7f965effd640 1 -- 172.21.15.94:0/1234783993 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9670021320 con 0x55b2ea280ad0 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.244+0000 7f967e61e740 1 -- 172.21.15.94:0/1234783993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea280ad0 msgr2=0x55b2ea294830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.244+0000 7f967e61e740 1 --2- 172.21.15.94:0/1234783993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea280ad0 0x55b2ea294830 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9670018460 tx=0x7f9670011810 comp rx=0 tx=0).stop 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.244+0000 7f967e61e740 1 -- 172.21.15.94:0/1234783993 shutdown_connections 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.244+0000 7f967e61e740 1 --2- 172.21.15.94:0/1234783993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea2892e0 0x55b2ea286830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.244+0000 7f967e61e740 1 --2- 172.21.15.94:0/1234783993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea280ad0 0x55b2ea294830 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.244+0000 7f967e61e740 1 --2- 172.21.15.94:0/1234783993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea27fd60 0x55b2ea288a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.244+0000 7f967e61e740 1 -- 172.21.15.94:0/1234783993 >> 172.21.15.94:0/1234783993 conn(0x55b2ea247550 msgr2=0x55b2ea294120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.244+0000 7f967e61e740 1 -- 172.21.15.94:0/1234783993 shutdown_connections 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.244+0000 7f967e61e740 1 -- 172.21.15.94:0/1234783993 wait complete. 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.245+0000 7f967e61e740 1 -- start start 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.245+0000 7f967e61e740 1 Processor -- start 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.245+0000 7f967e61e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea27fd60 0x55b2ea2a0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.245+0000 7f967e61e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea280ad0 0x55b2ea2a0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.245+0000 7f967e61e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea2892e0 0x55b2ea29c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.245+0000 7f967e61e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ea29cf70 con 0x55b2ea280ad0 2024-02-17T11:58:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.245+0000 7f967e61e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ea29d0e0 con 0x55b2ea2892e0 2024-02-17T11:58:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.245+0000 7f967e61e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b2ea29d2a0 con 0x55b2ea27fd60 2024-02-17T11:58:22.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.245+0000 7f967e61e740 10 client.0 _ll_register_callbacks cb 0x7f96700097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.246+0000 7f9678f07640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea27fd60 0x55b2ea2a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.246+0000 7f9678f07640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea27fd60 0x55b2ea2a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:53870/0 (socket says 172.21.15.94:53870) 2024-02-17T11:58:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.246+0000 7f9678f07640 1 -- 172.21.15.94:0/630415607 learned_addr learned my addr 172.21.15.94:0/630415607 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:22.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.246+0000 7f9678f07640 1 --2- 172.21.15.94:0/630415607 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea2892e0 0x55b2ea29c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.246+0000 7f9678f07640 1 -- 172.21.15.94:0/630415607 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea2892e0 msgr2=0x55b2ea29c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.246+0000 7f9678f07640 1 --2- 172.21.15.94:0/630415607 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea2892e0 0x55b2ea29c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.246+0000 7f9678f07640 1 -- 172.21.15.94:0/630415607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea280ad0 msgr2=0x55b2ea2a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.246+0000 7f9678f07640 1 --2- 172.21.15.94:0/630415607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea280ad0 0x55b2ea2a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.246+0000 7f9678f07640 1 -- 172.21.15.94:0/630415607 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966404c040 con 0x55b2ea27fd60 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.247+0000 7f9678f07640 1 --2- 172.21.15.94:0/630415607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea27fd60 0x55b2ea2a0320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9664000c00 tx=0x7f9664046040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2b7304e66aee6b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.247+0000 7f965cff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.247+0000 7f965cff9640 1 -- 172.21.15.94:0/630415607 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f966405f850 con 0x55b2ea27fd60 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.247+0000 7f965cff9640 1 -- 172.21.15.94:0/630415607 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9664000e40 con 0x55b2ea27fd60 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.247+0000 7f967e61e740 1 -- 172.21.15.94:0/630415607 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9670005500 con 0x55b2ea27fd60 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.247+0000 7f965cff9640 1 -- 172.21.15.94:0/630415607 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f966405f9b0 con 0x55b2ea27fd60 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.247+0000 7f967e61e740 10 client.25066 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.247+0000 7f967e61e740 1 -- 172.21.15.94:0/630415607 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55b2ea1e5d80 con 0x55b2ea27fd60 2024-02-17T11:58:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.248+0000 7f965cff9640 1 -- 172.21.15.94:0/630415607 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9670005500 con 0x55b2ea27fd60 2024-02-17T11:58:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.248+0000 7f965cff9640 1 -- 172.21.15.94:0/630415607 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f966403c040 con 0x55b2ea27fd60 2024-02-17T11:58:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.248+0000 7f967e61e740 10 client.25066 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.248+0000 7f967e61e740 10 client.25066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.248+0000 7f967e61e740 20 client.25066 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.248+0000 7f967e61e740 1 --2- 172.21.15.94:0/630415607 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b2ea299660 0x55b2ea2f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.248+0000 7f967e61e740 4 client.25066 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T11:58:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.248+0000 7f967e61e740 1 -- 172.21.15.94:0/630415607 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b2e9f24550 con 0x55b2ea299660 2024-02-17T11:58:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.248+0000 7f9679708640 1 --2- 172.21.15.94:0/630415607 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b2ea299660 0x55b2ea2f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.249+0000 7f965cff9640 10 client.25066 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.249+0000 7f9679708640 1 --2- 172.21.15.94:0/630415607 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b2ea299660 0x55b2ea2f2f40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47544f6cc3058fa server_cookie=f6ae777b159649b9 in_seq=0 out_seq=0 2024-02-17T11:58:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.447+0000 7f965cff9640 1 -- 172.21.15.94:0/630415607 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b2e9f24550 con 0x55b2ea299660 2024-02-17T11:58:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.447+0000 7f965cff9640 10 client.25066 handle_command_reply: tid=0 2024-02-17T11:58:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.447+0000 7f967e61e740 1 client.25066 shutdown 2024-02-17T11:58:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.448+0000 7f967e61e740 1 -- 172.21.15.94:0/630415607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea27fd60 msgr2=0x55b2ea2a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.448+0000 7f967e61e740 1 --2- 172.21.15.94:0/630415607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea27fd60 0x55b2ea2a0320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9664000c00 tx=0x7f9664046040 comp rx=0 tx=0).stop 2024-02-17T11:58:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.448+0000 7f967e61e740 1 -- 172.21.15.94:0/630415607 shutdown_connections 2024-02-17T11:58:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.448+0000 7f967e61e740 1 --2- 172.21.15.94:0/630415607 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b2ea299660 0x55b2ea2f2f40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.448+0000 7f967e61e740 1 --2- 172.21.15.94:0/630415607 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b2ea2892e0 0x55b2ea29c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.448+0000 7f967e61e740 1 --2- 172.21.15.94:0/630415607 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b2ea280ad0 0x55b2ea2a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.448+0000 7f967e61e740 1 --2- 172.21.15.94:0/630415607 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b2ea27fd60 0x55b2ea2a0320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.448+0000 7f965cff9640 0 client.25066 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.448+0000 7f967e61e740 1 -- 172.21.15.94:0/630415607 >> 172.21.15.94:0/630415607 conn(0x55b2ea247550 msgr2=0x55b2ea279b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.448+0000 7f967e61e740 1 -- 172.21.15.94:0/630415607 shutdown_connections 2024-02-17T11:58:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.448+0000 7f967e61e740 1 -- 172.21.15.94:0/630415607 wait complete. 2024-02-17T11:58:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.448+0000 7f967e61e740 20 client.25066 trim_cache size 0 max 16384 2024-02-17T11:58:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T11:58:22.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T11:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T11:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T11:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.449+0000 7f967b96c640 1 -- 172.21.15.94:0/2381770789 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96600a2370 msgr2=0x7f96600a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.449+0000 7f967b96c640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96600a2370 0x7f96600a4830 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f966c0cbf00 tx=0x7f966c01df90 comp rx=0 tx=0).stop 2024-02-17T11:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.449+0000 7f967b96c640 1 -- 172.21.15.94:0/2381770789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9674105040 msgr2=0x7f967419af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.449+0000 7f967b96c640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9674105040 0x7f967419af30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f966c00ea60 tx=0x7f966c00ef20 comp rx=0 tx=0).stop 2024-02-17T11:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.449+0000 7f967b96c640 1 -- 172.21.15.94:0/2381770789 shutdown_connections 2024-02-17T11:58:22.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.449+0000 7f967b96c640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96600a2370 0x7f96600a4830 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.449+0000 7f967b96c640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9674105040 0x7f967419af30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.449+0000 7f967b96c640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9674100c70 0x7f9674196c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.449+0000 7f967b96c640 1 --2- 172.21.15.94:0/2381770789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96741002d0 0x7f96741966c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.449+0000 7f967b96c640 1 -- 172.21.15.94:0/2381770789 >> 172.21.15.94:0/2381770789 conn(0x7f9674076ee0 msgr2=0x7f96740fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.449+0000 7f967b96c640 1 -- 172.21.15.94:0/2381770789 shutdown_connections 2024-02-17T11:58:22.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:22.449+0000 7f967b96c640 1 -- 172.21.15.94:0/2381770789 wait complete. 2024-02-17T11:58:24.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:23 smithi189 ceph-mon[36588]: pgmap v11: 97 pgs: 97 active+clean; 371 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 55 MiB/s wr, 274 op/s 2024-02-17T11:58:24.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:23 smithi125 ceph-mon[36819]: pgmap v11: 97 pgs: 97 active+clean; 371 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 55 MiB/s wr, 274 op/s 2024-02-17T11:58:24.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:23 smithi094 ceph-mon[32499]: pgmap v11: 97 pgs: 97 active+clean; 371 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 55 MiB/s wr, 274 op/s 2024-02-17T11:58:25.241 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 5.0 s, 4007.2 tps, lat 24.571 ms stddev 26.346 2024-02-17T11:58:26.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:25 smithi189 ceph-mon[36588]: pgmap v12: 97 pgs: 97 active+clean; 512 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 77 MiB/s wr, 458 op/s 2024-02-17T11:58:26.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:25 smithi125 ceph-mon[36819]: pgmap v12: 97 pgs: 97 active+clean; 512 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 77 MiB/s wr, 458 op/s 2024-02-17T11:58:26.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:25 smithi094 ceph-mon[32499]: pgmap v12: 97 pgs: 97 active+clean; 512 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 77 MiB/s wr, 458 op/s 2024-02-17T11:58:28.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:27 smithi189 ceph-mon[36588]: pgmap v13: 97 pgs: 97 active+clean; 512 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 77 MiB/s wr, 439 op/s 2024-02-17T11:58:28.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:27 smithi125 ceph-mon[36819]: pgmap v13: 97 pgs: 97 active+clean; 512 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 77 MiB/s wr, 439 op/s 2024-02-17T11:58:28.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:27 smithi094 ceph-mon[32499]: pgmap v13: 97 pgs: 97 active+clean; 512 MiB data, 714 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 77 MiB/s wr, 439 op/s 2024-02-17T11:58:29.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T11:58:29.154 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T11:58:29.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.316+0000 7fef846ae640 1 -- 172.21.15.94:0/25091910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef7c1002a0 msgr2=0x7fef7c100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.316+0000 7fef846ae640 1 --2- 172.21.15.94:0/25091910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef7c1002a0 0x7fef7c100680 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fef78009cd0 tx=0x7fef7802bdc0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.316+0000 7fef846ae640 1 -- 172.21.15.94:0/25091910 shutdown_connections 2024-02-17T11:58:29.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.316+0000 7fef846ae640 1 --2- 172.21.15.94:0/25091910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef7c105370 0x7fef7c10cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.316+0000 7fef846ae640 1 --2- 172.21.15.94:0/25091910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef7c100c50 0x7fef7c104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.316+0000 7fef846ae640 1 --2- 172.21.15.94:0/25091910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef7c1002a0 0x7fef7c100680 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.316+0000 7fef846ae640 1 -- 172.21.15.94:0/25091910 >> 172.21.15.94:0/25091910 conn(0x7fef7c077210 msgr2=0x7fef7c077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:29.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.316+0000 7fef846ae640 1 -- 172.21.15.94:0/25091910 shutdown_connections 2024-02-17T11:58:29.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.316+0000 7fef846ae640 1 -- 172.21.15.94:0/25091910 wait complete. 2024-02-17T11:58:29.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.318+0000 7fef846ae640 1 Processor -- start 2024-02-17T11:58:29.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.319+0000 7fef846ae640 1 -- start start 2024-02-17T11:58:29.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.319+0000 7fef846ae640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef7c1002a0 0x7fef7c19ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.319+0000 7fef846ae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef7c105370 0x7fef7c19e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.319+0000 7fef846ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef7c1a26b0 0x7fef7c1a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.319+0000 7fef846ae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1a30a0 con 0x7fef7c1a26b0 2024-02-17T11:58:29.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.319+0000 7fef846ae640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1a3210 con 0x7fef7c1002a0 2024-02-17T11:58:29.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.319+0000 7fef846ae640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c1a3380 con 0x7fef7c105370 2024-02-17T11:58:29.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.320+0000 7fef8244a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef7c1002a0 0x7fef7c19ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.320+0000 7fef8244a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef7c1002a0 0x7fef7c19ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45388/0 (socket says 172.21.15.94:45388) 2024-02-17T11:58:29.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.320+0000 7fef8244a640 1 -- 172.21.15.94:0/443255334 learned_addr learned my addr 172.21.15.94:0/443255334 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:29.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.320+0000 7fef82c4b640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef7c1a26b0 0x7fef7c1a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.320+0000 7fef81c49640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef7c105370 0x7fef7c19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.320+0000 7fef8244a640 1 -- 172.21.15.94:0/443255334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef7c105370 msgr2=0x7fef7c19e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.320+0000 7fef8244a640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef7c105370 0x7fef7c19e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.320+0000 7fef8244a640 1 -- 172.21.15.94:0/443255334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef7c1a26b0 msgr2=0x7fef7c1a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.320+0000 7fef8244a640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef7c1a26b0 0x7fef7c1a2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.320+0000 7fef8244a640 1 -- 172.21.15.94:0/443255334 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c00f790 con 0x7fef7c1002a0 2024-02-17T11:58:29.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.321+0000 7fef8244a640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef7c1002a0 0x7fef7c19ddf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fef78000f80 tx=0x7fef78032690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:29.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.321+0000 7fef737fe640 1 -- 172.21.15.94:0/443255334 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78032880 con 0x7fef7c1002a0 2024-02-17T11:58:29.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.321+0000 7fef737fe640 1 -- 172.21.15.94:0/443255334 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef78003d80 con 0x7fef7c1002a0 2024-02-17T11:58:29.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.321+0000 7fef737fe640 1 -- 172.21.15.94:0/443255334 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78034900 con 0x7fef7c1002a0 2024-02-17T11:58:29.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.321+0000 7fef846ae640 1 -- 172.21.15.94:0/443255334 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef78009900 con 0x7fef7c1002a0 2024-02-17T11:58:29.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.321+0000 7fef846ae640 1 -- 172.21.15.94:0/443255334 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c1aaba0 con 0x7fef7c1002a0 2024-02-17T11:58:29.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.324+0000 7fef737fe640 1 -- 172.21.15.94:0/443255334 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7fef7803b040 con 0x7fef7c1002a0 2024-02-17T11:58:29.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.324+0000 7fef87280740 1 Processor -- start 2024-02-17T11:58:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.325+0000 7fef87280740 1 -- start start 2024-02-17T11:58:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.325+0000 7fef87280740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b7e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.325+0000 7fef87280740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b87270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.325+0000 7fef87280740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b877b0 0x55f5a5b89ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.325+0000 7fef87280740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a5aed280 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.325+0000 7fef87280740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a5aea7b0 con 0x55f5a5b877b0 2024-02-17T11:58:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.325+0000 7fef87280740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a5ae5f90 con 0x55f5a5b7e2b0 2024-02-17T11:58:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.325+0000 7fef8244a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b87270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.325+0000 7fef81c49640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b877b0 0x55f5a5b89ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.325+0000 7fef82c4b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b7e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.325+0000 7fef81c49640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b877b0 0x55f5a5b89ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45396/0 (socket says 172.21.15.94:45396) 2024-02-17T11:58:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.325+0000 7fef82c4b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b7e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57896/0 (socket says 172.21.15.94:57896) 2024-02-17T11:58:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.325+0000 7fef81c49640 1 -- 172.21.15.94:0/15850543 learned_addr learned my addr 172.21.15.94:0/15850543 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.326+0000 7fef8244a640 1 -- 172.21.15.94:0/15850543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7e2b0 msgr2=0x55f5a5b7e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.326+0000 7fef8244a640 1 --2- 172.21.15.94:0/15850543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b7e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.326+0000 7fef8244a640 1 -- 172.21.15.94:0/15850543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b877b0 msgr2=0x55f5a5b89ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:29.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.326+0000 7fef8244a640 1 --2- 172.21.15.94:0/15850543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b877b0 0x55f5a5b89ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.326+0000 7fef8244a640 1 -- 172.21.15.94:0/15850543 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5a5aa4570 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.326+0000 7fef81c49640 1 --2- 172.21.15.94:0/15850543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b877b0 0x55f5a5b89ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:58:29.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.326+0000 7fef737fe640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fef680a2340 0x7fef680a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.326+0000 7fef8244a640 1 --2- 172.21.15.94:0/15850543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b87270 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fef780ed7d0 tx=0x7fef780f3e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2358c83ca649c16 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:29.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.326+0000 7fef82c4b640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fef680a2340 0x7fef680a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.327+0000 7fef70ff9640 1 -- 172.21.15.94:0/15850543 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef780b8d10 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.327+0000 7fef70ff9640 1 -- 172.21.15.94:0/15850543 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef780bc840 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.327+0000 7fef70ff9640 1 -- 172.21.15.94:0/15850543 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef780a0630 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.327+0000 7fef737fe640 1 -- 172.21.15.94:0/443255334 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fef7802ce40 con 0x7fef7c1002a0 2024-02-17T11:58:29.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.327+0000 7fef82c4b640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fef680a2340 0x7fef680a4800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fef74009a70 tx=0x7fef740090e0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:29.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.328+0000 7fef87280740 1 -- 172.21.15.94:0/15850543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b7ebd0 msgr2=0x55f5a5b87270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.328+0000 7fef87280740 1 --2- 172.21.15.94:0/15850543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b87270 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fef780ed7d0 tx=0x7fef780f3e00 comp rx=0 tx=0).stop 2024-02-17T11:58:29.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.328+0000 7fef87280740 1 -- 172.21.15.94:0/15850543 shutdown_connections 2024-02-17T11:58:29.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.328+0000 7fef87280740 1 --2- 172.21.15.94:0/15850543 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b877b0 0x55f5a5b89ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.328+0000 7fef87280740 1 --2- 172.21.15.94:0/15850543 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b87270 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.329+0000 7fef87280740 1 --2- 172.21.15.94:0/15850543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b7e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.329+0000 7fef87280740 1 -- 172.21.15.94:0/15850543 >> 172.21.15.94:0/15850543 conn(0x55f5a5b78990 msgr2=0x55f5a5b78d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:29.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.330+0000 7fef87280740 1 -- 172.21.15.94:0/15850543 shutdown_connections 2024-02-17T11:58:29.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.330+0000 7fef87280740 1 -- 172.21.15.94:0/15850543 wait complete. 2024-02-17T11:58:29.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.330+0000 7fef87280740 1 -- start start 2024-02-17T11:58:29.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.330+0000 7fef87280740 1 Processor -- start 2024-02-17T11:58:29.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.331+0000 7fef87280740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b87260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.331+0000 7fef87280740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.331+0000 7fef87280740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b877b0 0x55f5a5b84d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.331+0000 7fef8244a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b87260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.331+0000 7fef8244a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b87260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45412/0 (socket says 172.21.15.94:45412) 2024-02-17T11:58:29.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.331+0000 7fef8244a640 1 -- 172.21.15.94:0/250931961 learned_addr learned my addr 172.21.15.94:0/250931961 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:29.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.331+0000 7fef87280740 1 -- 172.21.15.94:0/250931961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a57b9430 con 0x55f5a5b877b0 2024-02-17T11:58:29.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.331+0000 7fef87280740 1 -- 172.21.15.94:0/250931961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a5775e00 con 0x55f5a5b7e2b0 2024-02-17T11:58:29.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.331+0000 7fef87280740 1 -- 172.21.15.94:0/250931961 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a5a6afd0 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.331+0000 7fef81c49640 1 --2- 172.21.15.94:0/250931961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.331+0000 7fef82c4b640 1 --2- 172.21.15.94:0/250931961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b877b0 0x55f5a5b84d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.331+0000 7fef87280740 10 client.0 _ll_register_callbacks cb 0x55f5a5ac0fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:29.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.332+0000 7fef8244a640 1 -- 172.21.15.94:0/250931961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7ebd0 msgr2=0x55f5a5b845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:29.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.332+0000 7fef8244a640 1 --2- 172.21.15.94:0/250931961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b845f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.332+0000 7fef8244a640 1 -- 172.21.15.94:0/250931961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b877b0 msgr2=0x55f5a5b84d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:29.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.332+0000 7fef8244a640 1 --2- 172.21.15.94:0/250931961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b877b0 0x55f5a5b84d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.332+0000 7fef8244a640 1 -- 172.21.15.94:0/250931961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c1aaba0 con 0x55f5a5b7e2b0 2024-02-17T11:58:29.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.332+0000 7fef8244a640 1 --2- 172.21.15.94:0/250931961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b87260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef780bca00 tx=0x7fef7802c5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94329358380744a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:29.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.332+0000 7fef65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:58:29.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.332+0000 7fef65ffb640 1 -- 172.21.15.94:0/250931961 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef780dc070 con 0x55f5a5b7e2b0 2024-02-17T11:58:29.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.332+0000 7fef87280740 1 -- 172.21.15.94:0/250931961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5a57e57e0 con 0x55f5a5b7e2b0 2024-02-17T11:58:29.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.333+0000 7fef65ffb640 1 -- 172.21.15.94:0/250931961 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fef780bca70 con 0x55f5a5b7e2b0 2024-02-17T11:58:29.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.333+0000 7fef65ffb640 1 -- 172.21.15.94:0/250931961 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef780f3480 con 0x55f5a5b7e2b0 2024-02-17T11:58:29.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.333+0000 7fef87280740 10 client.25034 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:29.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.333+0000 7fef87280740 1 -- 172.21.15.94:0/250931961 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55f5a5ae4610 con 0x55f5a5b7e2b0 2024-02-17T11:58:29.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.334+0000 7fef65ffb640 1 -- 172.21.15.94:0/250931961 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef780b8d10 con 0x55f5a5b7e2b0 2024-02-17T11:58:29.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.334+0000 7fef65ffb640 1 -- 172.21.15.94:0/250931961 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fef780c4cd0 con 0x55f5a5b7e2b0 2024-02-17T11:58:29.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.335+0000 7fef87280740 10 client.25034 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:29.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.335+0000 7fef87280740 10 client.25034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:29.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.335+0000 7fef87280740 20 client.25034 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:29.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.335+0000 7fef87280740 1 --2- 172.21.15.94:0/250931961 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5a5b97140 0x55f5a5b99550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.335+0000 7fef87280740 4 client.25034 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:58:29.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.335+0000 7fef87280740 1 -- 172.21.15.94:0/250931961 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5a57e57e0 con 0x55f5a5b97140 2024-02-17T11:58:29.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.335+0000 7fef81c49640 1 --2- 172.21.15.94:0/250931961 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5a5b97140 0x55f5a5b99550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.335+0000 7fef81c49640 1 --2- 172.21.15.94:0/250931961 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5a5b97140 0x55f5a5b99550 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a81bc6cf4ec61e76 server_cookie=696df616beb85a64 in_seq=0 out_seq=0 2024-02-17T11:58:29.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.335+0000 7fef65ffb640 10 client.25034 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef65ffb640 1 -- 172.21.15.94:0/250931961 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f5a57e57e0 con 0x55f5a5b97140 2024-02-17T11:58:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef65ffb640 10 client.25034 handle_command_reply: tid=0 2024-02-17T11:58:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef87280740 1 client.25034 shutdown 2024-02-17T11:58:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef87280740 1 -- 172.21.15.94:0/250931961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7e2b0 msgr2=0x55f5a5b87260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef87280740 1 --2- 172.21.15.94:0/250931961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b87260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fef780bca00 tx=0x7fef7802c5e0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef87280740 1 -- 172.21.15.94:0/250931961 shutdown_connections 2024-02-17T11:58:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef87280740 1 --2- 172.21.15.94:0/250931961 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5a5b97140 0x55f5a5b99550 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef87280740 1 --2- 172.21.15.94:0/250931961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b877b0 0x55f5a5b84d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef65ffb640 0 client.25034 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:29.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef87280740 1 --2- 172.21.15.94:0/250931961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef87280740 1 --2- 172.21.15.94:0/250931961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b87260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef87280740 1 -- 172.21.15.94:0/250931961 >> 172.21.15.94:0/250931961 conn(0x55f5a5b78990 msgr2=0x55f5a5b7a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:29.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef87280740 1 -- 172.21.15.94:0/250931961 shutdown_connections 2024-02-17T11:58:29.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef87280740 1 -- 172.21.15.94:0/250931961 wait complete. 2024-02-17T11:58:29.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.336+0000 7fef87280740 20 client.25034 trim_cache size 0 max 16384 2024-02-17T11:58:29.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.358+0000 7fef87280740 1 Processor -- start 2024-02-17T11:58:29.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.358+0000 7fef87280740 1 -- start start 2024-02-17T11:58:29.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.358+0000 7fef87280740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b845b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef8244a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b845b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef87280740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b8e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef87280740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b877b0 0x55f5a5b8e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef87280740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a5b740f0 con 0x55f5a5b7e2b0 2024-02-17T11:58:29.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef87280740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a5b7d3a0 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef87280740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a5b74320 con 0x55f5a5b877b0 2024-02-17T11:58:29.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef8244a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b845b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37274/0 (socket says 172.21.15.94:37274) 2024-02-17T11:58:29.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef8244a640 1 -- 172.21.15.94:0/1469701938 learned_addr learned my addr 172.21.15.94:0/1469701938 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:29.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef81c49640 1 --2- 172.21.15.94:0/1469701938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b8e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef82c4b640 1 --2- 172.21.15.94:0/1469701938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b877b0 0x55f5a5b8e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef81c49640 1 -- 172.21.15.94:0/1469701938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b877b0 msgr2=0x55f5a5b8e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:29.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef81c49640 1 --2- 172.21.15.94:0/1469701938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b877b0 0x55f5a5b8e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef81c49640 1 -- 172.21.15.94:0/1469701938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b7e2b0 msgr2=0x55f5a5b845b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef81c49640 1 --2- 172.21.15.94:0/1469701938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b845b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef81c49640 1 -- 172.21.15.94:0/1469701938 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5a5a80ac0 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.359+0000 7fef8244a640 1 --2- 172.21.15.94:0/1469701938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b845b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.360+0000 7fef81c49640 1 --2- 172.21.15.94:0/1469701938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b8e390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55f5a5b99e00 tx=0x7fef6c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dec416f2bc70703c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.360+0000 7fef67fff640 1 -- 172.21.15.94:0/1469701938 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c014070 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.360+0000 7fef67fff640 1 -- 172.21.15.94:0/1469701938 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef6c015040 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.360+0000 7fef67fff640 1 -- 172.21.15.94:0/1469701938 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c021070 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 -- 172.21.15.94:0/1469701938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7ebd0 msgr2=0x55f5a5b8e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 --2- 172.21.15.94:0/1469701938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b8e390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55f5a5b99e00 tx=0x7fef6c005f90 comp rx=0 tx=0).stop 2024-02-17T11:58:29.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 -- 172.21.15.94:0/1469701938 shutdown_connections 2024-02-17T11:58:29.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 --2- 172.21.15.94:0/1469701938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b877b0 0x55f5a5b8e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 --2- 172.21.15.94:0/1469701938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5b8e390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 --2- 172.21.15.94:0/1469701938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5b845b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 -- 172.21.15.94:0/1469701938 >> 172.21.15.94:0/1469701938 conn(0x55f5a5b78990 msgr2=0x55f5a5b7f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:29.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 -- 172.21.15.94:0/1469701938 shutdown_connections 2024-02-17T11:58:29.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 -- 172.21.15.94:0/1469701938 wait complete. 2024-02-17T11:58:29.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 -- start start 2024-02-17T11:58:29.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 Processor -- start 2024-02-17T11:58:29.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5be7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5be80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b877b0 0x55f5a5b8d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a5b8dde0 con 0x55f5a5b877b0 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a5b8df50 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5a5b8e110 con 0x55f5a5b7e2b0 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.361+0000 7fef87280740 10 client.0 _ll_register_callbacks cb 0x7fef780ed660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef81c49640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5be80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef81c49640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5be80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45420/0 (socket says 172.21.15.94:45420) 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef81c49640 1 -- 172.21.15.94:0/3896996138 learned_addr learned my addr 172.21.15.94:0/3896996138 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef82c4b640 1 --2- 172.21.15.94:0/3896996138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b877b0 0x55f5a5b8d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef8244a640 1 --2- 172.21.15.94:0/3896996138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5be7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef81c49640 1 -- 172.21.15.94:0/3896996138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7e2b0 msgr2=0x55f5a5be7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef81c49640 1 --2- 172.21.15.94:0/3896996138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5be7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef81c49640 1 -- 172.21.15.94:0/3896996138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b877b0 msgr2=0x55f5a5b8d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef81c49640 1 --2- 172.21.15.94:0/3896996138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b877b0 0x55f5a5b8d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef81c49640 1 -- 172.21.15.94:0/3896996138 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5a5ae4610 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef81c49640 1 --2- 172.21.15.94:0/3896996138 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5be80b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fef6c028040 tx=0x7fef6c004200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e6fddb38ac992e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:29.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:58:29.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef65ffb640 1 -- 172.21.15.94:0/3896996138 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c014070 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.362+0000 7fef65ffb640 1 -- 172.21.15.94:0/3896996138 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fef6c01f030 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.363+0000 7fef87280740 1 -- 172.21.15.94:0/3896996138 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef780bc840 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.364+0000 7fef65ffb640 1 -- 172.21.15.94:0/3896996138 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef6c021040 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.364+0000 7fef87280740 10 client.25040 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:29.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.364+0000 7fef87280740 1 -- 172.21.15.94:0/3896996138 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x7fef6c00fbd0 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.364+0000 7fef65ffb640 1 -- 172.21.15.94:0/3896996138 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef6c0046c0 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.364+0000 7fef65ffb640 1 -- 172.21.15.94:0/3896996138 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fef6c02c020 con 0x55f5a5b7ebd0 2024-02-17T11:58:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.364+0000 7fef87280740 10 client.25040 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.364+0000 7fef87280740 10 client.25040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.364+0000 7fef87280740 20 client.25040 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.364+0000 7fef87280740 1 --2- 172.21.15.94:0/3896996138 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5a5bef9f0 0x55f5a5bf1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.364+0000 7fef87280740 4 client.25040 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T11:58:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.364+0000 7fef87280740 1 -- 172.21.15.94:0/3896996138 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fef780bc840 con 0x55f5a5bef9f0 2024-02-17T11:58:29.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.365+0000 7fef8244a640 1 --2- 172.21.15.94:0/3896996138 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5a5bef9f0 0x55f5a5bf1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.365+0000 7fef65ffb640 10 client.25040 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.366+0000 7fef8244a640 1 --2- 172.21.15.94:0/3896996138 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5a5bef9f0 0x55f5a5bf1e10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e312cdfc42fb480 server_cookie=1b172957aeda58e1 in_seq=0 out_seq=0 2024-02-17T11:58:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.366+0000 7fef65ffb640 1 -- 172.21.15.94:0/3896996138 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7fef780bc840 con 0x55f5a5bef9f0 2024-02-17T11:58:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.366+0000 7fef65ffb640 10 client.25040 handle_command_reply: tid=0 2024-02-17T11:58:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.366+0000 7fef87280740 1 client.25040 shutdown 2024-02-17T11:58:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.366+0000 7fef87280740 1 -- 172.21.15.94:0/3896996138 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7ebd0 msgr2=0x55f5a5be80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.366+0000 7fef87280740 1 --2- 172.21.15.94:0/3896996138 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5be80b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fef6c028040 tx=0x7fef6c004200 comp rx=0 tx=0).stop 2024-02-17T11:58:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.366+0000 7fef87280740 1 -- 172.21.15.94:0/3896996138 shutdown_connections 2024-02-17T11:58:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.366+0000 7fef87280740 1 --2- 172.21.15.94:0/3896996138 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5a5bef9f0 0x55f5a5bf1e10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.366+0000 7fef65ffb640 0 client.25040 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.366+0000 7fef87280740 1 --2- 172.21.15.94:0/3896996138 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5a5b877b0 0x55f5a5b8d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.367+0000 7fef87280740 1 --2- 172.21.15.94:0/3896996138 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5a5b7ebd0 0x55f5a5be80b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.367+0000 7fef87280740 1 --2- 172.21.15.94:0/3896996138 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5a5b7e2b0 0x55f5a5be7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.367+0000 7fef87280740 1 -- 172.21.15.94:0/3896996138 >> 172.21.15.94:0/3896996138 conn(0x55f5a5b78990 msgr2=0x55f5a5b82ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:29.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.367+0000 7fef87280740 1 -- 172.21.15.94:0/3896996138 shutdown_connections 2024-02-17T11:58:29.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.367+0000 7fef87280740 1 -- 172.21.15.94:0/3896996138 wait complete. 2024-02-17T11:58:29.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.367+0000 7fef87280740 20 client.25040 trim_cache size 0 max 16384 2024-02-17T11:58:29.371 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:58:29.371 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T11:58:29.371 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T11:58:29.371 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:58:29.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.368+0000 7fef846ae640 1 -- 172.21.15.94:0/443255334 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fef680a2340 msgr2=0x7fef680a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.368+0000 7fef846ae640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fef680a2340 0x7fef680a4800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fef74009a70 tx=0x7fef740090e0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.368+0000 7fef846ae640 1 -- 172.21.15.94:0/443255334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef7c1002a0 msgr2=0x7fef7c19ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.368+0000 7fef846ae640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef7c1002a0 0x7fef7c19ddf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fef78000f80 tx=0x7fef78032690 comp rx=0 tx=0).stop 2024-02-17T11:58:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.368+0000 7fef846ae640 1 -- 172.21.15.94:0/443255334 shutdown_connections 2024-02-17T11:58:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.368+0000 7fef846ae640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fef680a2340 0x7fef680a4800 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.368+0000 7fef846ae640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef7c1a26b0 0x7fef7c1a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.368+0000 7fef846ae640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef7c105370 0x7fef7c19e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.368+0000 7fef846ae640 1 --2- 172.21.15.94:0/443255334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef7c1002a0 0x7fef7c19ddf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.368+0000 7fef846ae640 1 -- 172.21.15.94:0/443255334 >> 172.21.15.94:0/443255334 conn(0x7fef7c077210 msgr2=0x7fef7c0fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.368+0000 7fef846ae640 1 -- 172.21.15.94:0/443255334 shutdown_connections 2024-02-17T11:58:29.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.368+0000 7fef846ae640 1 -- 172.21.15.94:0/443255334 wait complete. 2024-02-17T11:58:29.385 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T11:58:29.385 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:58:29.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.553+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/17579207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b1410a440 msgr2=0x7f9b14112b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.553+0000 7f9b1ba9a640 1 --2- 172.21.15.94:0/17579207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b1410a440 0x7f9b14112b90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009930 tx=0x7f9b0402bc30 comp rx=0 tx=0).stop 2024-02-17T11:58:29.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.554+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/17579207 shutdown_connections 2024-02-17T11:58:29.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.554+0000 7f9b1ba9a640 1 --2- 172.21.15.94:0/17579207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b141130d0 0x7f9b141154c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.554+0000 7f9b1ba9a640 1 --2- 172.21.15.94:0/17579207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b1410a440 0x7f9b14112b90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.554+0000 7f9b1ba9a640 1 --2- 172.21.15.94:0/17579207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9b14109b20 0x7f9b14109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.554+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/17579207 >> 172.21.15.94:0/17579207 conn(0x7f9b1406c6d0 msgr2=0x7f9b1406cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:29.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.554+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/17579207 shutdown_connections 2024-02-17T11:58:29.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.554+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/17579207 wait complete. 2024-02-17T11:58:29.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.554+0000 7f9b1ba9a640 1 Processor -- start 2024-02-17T11:58:29.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.554+0000 7f9b1ba9a640 1 -- start start 2024-02-17T11:58:29.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.555+0000 7f9b1ba9a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b14109b20 0x7f9b1419ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.555+0000 7f9b1ba9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b1410a440 0x7f9b141a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.555+0000 7f9b1ba9a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9b141130d0 0x7f9b141a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.555+0000 7f9b1ba9a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b141a4e50 con 0x7f9b1410a440 2024-02-17T11:58:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.555+0000 7f9b1ba9a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b141a4fc0 con 0x7f9b141130d0 2024-02-17T11:58:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.555+0000 7f9b1ba9a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b141a5180 con 0x7f9b14109b20 2024-02-17T11:58:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.555+0000 7f9b19836640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b14109b20 0x7f9b1419ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.555+0000 7f9b1a037640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9b141130d0 0x7f9b141a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.555+0000 7f9b19836640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b14109b20 0x7f9b1419ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57946/0 (socket says 172.21.15.94:57946) 2024-02-17T11:58:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.555+0000 7f9b19836640 1 -- 172.21.15.94:0/3368801167 learned_addr learned my addr 172.21.15.94:0/3368801167 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:29.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.555+0000 7f9b19035640 1 --2- 172.21.15.94:0/3368801167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b1410a440 0x7f9b141a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.556+0000 7f9b19035640 1 -- 172.21.15.94:0/3368801167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b14109b20 msgr2=0x7f9b1419ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.556+0000 7f9b19035640 1 --2- 172.21.15.94:0/3368801167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b14109b20 0x7f9b1419ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.556+0000 7f9b19035640 1 -- 172.21.15.94:0/3368801167 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9b141130d0 msgr2=0x7f9b141a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.556+0000 7f9b19035640 1 --2- 172.21.15.94:0/3368801167 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9b141130d0 0x7f9b141a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.556+0000 7f9b19035640 1 -- 172.21.15.94:0/3368801167 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b04009560 con 0x7f9b1410a440 2024-02-17T11:58:29.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.557+0000 7f9b19035640 1 --2- 172.21.15.94:0/3368801167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b1410a440 0x7f9b141a04d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9b0402c170 tx=0x7f9b04004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:29.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.557+0000 7f9b0affd640 1 -- 172.21.15.94:0/3368801167 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04005220 con 0x7f9b1410a440 2024-02-17T11:58:29.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.557+0000 7f9b0affd640 1 -- 172.21.15.94:0/3368801167 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b04004df0 con 0x7f9b1410a440 2024-02-17T11:58:29.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.557+0000 7f9b0affd640 1 -- 172.21.15.94:0/3368801167 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04003c00 con 0x7f9b1410a440 2024-02-17T11:58:29.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.557+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/3368801167 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b141a5400 con 0x7f9b1410a440 2024-02-17T11:58:29.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.557+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/3368801167 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1406fc50 con 0x7f9b1410a440 2024-02-17T11:58:29.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.558+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/3368801167 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b14067f70 con 0x7f9b1410a440 2024-02-17T11:58:29.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.559+0000 7f9b0affd640 1 -- 172.21.15.94:0/3368801167 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f9b040035f0 con 0x7f9b1410a440 2024-02-17T11:58:29.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.560+0000 7f9b0affd640 1 --2- 172.21.15.94:0/3368801167 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9b000a2350 0x7f9b000a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:29.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.560+0000 7f9b0affd640 1 -- 172.21.15.94:0/3368801167 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9b040f0da0 con 0x7f9b1410a440 2024-02-17T11:58:29.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.562+0000 7f9b19836640 1 --2- 172.21.15.94:0/3368801167 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9b000a2350 0x7f9b000a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:29.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.562+0000 7f9b19836640 1 --2- 172.21.15.94:0/3368801167 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9b000a2350 0x7f9b000a4810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9b10005fd0 tx=0x7f9b1000b4a0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:29.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.567+0000 7f9b0affd640 1 -- 172.21.15.94:0/3368801167 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b040b9960 con 0x7f9b1410a440 2024-02-17T11:58:29.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.918+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/3368801167 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b1410f200 con 0x7f9b1410a440 2024-02-17T11:58:29.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.919+0000 7f9b0affd640 1 -- 172.21.15.94:0/3368801167 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f9b1410f200 con 0x7f9b1410a440 2024-02-17T11:58:29.918 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:58:29.918 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:57:51.854453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T11:58:29.918 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 21 2024-02-17T11:58:29.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.921+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/3368801167 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9b000a2350 msgr2=0x7f9b000a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.921+0000 7f9b1ba9a640 1 --2- 172.21.15.94:0/3368801167 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9b000a2350 0x7f9b000a4810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9b10005fd0 tx=0x7f9b1000b4a0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.921+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/3368801167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b1410a440 msgr2=0x7f9b141a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:29.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.921+0000 7f9b1ba9a640 1 --2- 172.21.15.94:0/3368801167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b1410a440 0x7f9b141a04d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9b0402c170 tx=0x7f9b04004970 comp rx=0 tx=0).stop 2024-02-17T11:58:29.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.921+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/3368801167 shutdown_connections 2024-02-17T11:58:29.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.921+0000 7f9b1ba9a640 1 --2- 172.21.15.94:0/3368801167 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9b000a2350 0x7f9b000a4810 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.921+0000 7f9b1ba9a640 1 --2- 172.21.15.94:0/3368801167 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9b141130d0 0x7f9b141a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.921+0000 7f9b1ba9a640 1 --2- 172.21.15.94:0/3368801167 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b1410a440 0x7f9b141a04d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.921+0000 7f9b1ba9a640 1 --2- 172.21.15.94:0/3368801167 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b14109b20 0x7f9b1419ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:29.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.921+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/3368801167 >> 172.21.15.94:0/3368801167 conn(0x7f9b1406c6d0 msgr2=0x7f9b14108db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:29.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.921+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/3368801167 shutdown_connections 2024-02-17T11:58:29.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:29.921+0000 7f9b1ba9a640 1 -- 172.21.15.94:0/3368801167 wait complete. 2024-02-17T11:58:29.936 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T11:58:29.936 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T11:58:30.066 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:29 smithi094 ceph-mon[32499]: pgmap v14: 97 pgs: 97 active+clean; 915 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 81 MiB/s wr, 766 op/s 2024-02-17T11:58:30.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.103+0000 7fc6b1cc3640 1 -- 172.21.15.94:0/98928748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6ac103c40 msgr2=0x7fc6ac1040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.103+0000 7fc6b1cc3640 1 --2- 172.21.15.94:0/98928748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6ac103c40 0x7fc6ac1040c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009930 tx=0x7fc69c02bc40 comp rx=0 tx=0).stop 2024-02-17T11:58:30.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.105+0000 7fc6b1cc3640 1 -- 172.21.15.94:0/98928748 shutdown_connections 2024-02-17T11:58:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.105+0000 7fc6b1cc3640 1 --2- 172.21.15.94:0/98928748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6ac103c40 0x7fc6ac1040c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.105+0000 7fc6b1cc3640 1 --2- 172.21.15.94:0/98928748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6ac0688c0 0x7fc6ac068d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.105+0000 7fc6b1cc3640 1 --2- 172.21.15.94:0/98928748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6ac1060d0 0x7fc6ac068380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.105+0000 7fc6b1cc3640 1 -- 172.21.15.94:0/98928748 >> 172.21.15.94:0/98928748 conn(0x7fc6ac077180 msgr2=0x7fc6ac077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.105+0000 7fc6b1cc3640 1 -- 172.21.15.94:0/98928748 shutdown_connections 2024-02-17T11:58:30.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.105+0000 7fc6b1cc3640 1 -- 172.21.15.94:0/98928748 wait complete. 2024-02-17T11:58:30.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.106+0000 7fc6b1cc3640 1 Processor -- start 2024-02-17T11:58:30.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.106+0000 7fc6b1cc3640 1 -- start start 2024-02-17T11:58:30.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.106+0000 7fc6b1cc3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6ac0688c0 0x7fc6ac1a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.106+0000 7fc6b1cc3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6ac103c40 0x7fc6ac1a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.106+0000 7fc6b1cc3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6ac1060d0 0x7fc6ac19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.106+0000 7fc6b1cc3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac19e2c0 con 0x7fc6ac0688c0 2024-02-17T11:58:30.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.106+0000 7fc6b1cc3640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac19e430 con 0x7fc6ac1060d0 2024-02-17T11:58:30.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.106+0000 7fc6b1cc3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ac19e5f0 con 0x7fc6ac103c40 2024-02-17T11:58:30.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.107+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6ac0688c0 0x7fc6ac1a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.107+0000 7fc6aaffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6ac103c40 0x7fc6ac1a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.107+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6ac0688c0 0x7fc6ac1a3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37330/0 (socket says 172.21.15.94:37330) 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.107+0000 7fc6ab7fe640 1 -- 172.21.15.94:0/1565224902 learned_addr learned my addr 172.21.15.94:0/1565224902 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.107+0000 7fc6abfff640 1 --2- 172.21.15.94:0/1565224902 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6ac1060d0 0x7fc6ac19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.107+0000 7fc6aaffd640 1 -- 172.21.15.94:0/1565224902 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6ac1060d0 msgr2=0x7fc6ac19db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.107+0000 7fc6aaffd640 1 --2- 172.21.15.94:0/1565224902 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6ac1060d0 0x7fc6ac19db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.107+0000 7fc6aaffd640 1 -- 172.21.15.94:0/1565224902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6ac0688c0 msgr2=0x7fc6ac1a3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.107+0000 7fc6aaffd640 1 --2- 172.21.15.94:0/1565224902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6ac0688c0 0x7fc6ac1a3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.107+0000 7fc6aaffd640 1 -- 172.21.15.94:0/1565224902 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c009560 con 0x7fc6ac103c40 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.108+0000 7fc6aaffd640 1 --2- 172.21.15.94:0/1565224902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6ac103c40 0x7fc6ac1a3fb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009fd0 tx=0x7fc6a000b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.108+0000 7fc6a8ff9640 1 -- 172.21.15.94:0/1565224902 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0011830 con 0x7fc6ac103c40 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.108+0000 7fc6b1cc3640 1 -- 172.21.15.94:0/1565224902 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ac19e8d0 con 0x7fc6ac103c40 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.108+0000 7fc6a8ff9640 1 -- 172.21.15.94:0/1565224902 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6a0011e70 con 0x7fc6ac103c40 2024-02-17T11:58:30.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.108+0000 7fc6b1cc3640 1 -- 172.21.15.94:0/1565224902 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac1aaa70 con 0x7fc6ac103c40 2024-02-17T11:58:30.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.109+0000 7fc6a8ff9640 1 -- 172.21.15.94:0/1565224902 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a000f2e0 con 0x7fc6ac103c40 2024-02-17T11:58:30.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.112+0000 7fc6b4974740 1 Processor -- start 2024-02-17T11:58:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.112+0000 7fc6b4974740 1 -- start start 2024-02-17T11:58:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.112+0000 7fc6b4974740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9a250 0x55c0a9d9a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.112+0000 7fc6b4974740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.112+0000 7fc6b4974740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9da3750 0x55c0a9da5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.112+0000 7fc6b4974740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a9d55960 con 0x55c0a9d9a250 2024-02-17T11:58:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.112+0000 7fc6b4974740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a9d09310 con 0x55c0a9da3750 2024-02-17T11:58:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.112+0000 7fc6b4974740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a9d06840 con 0x55c0a9d9ab70 2024-02-17T11:58:30.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.113+0000 7fc6abfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9a250 0x55c0a9d9a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.113+0000 7fc6abfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9a250 0x55c0a9d9a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37336/0 (socket says 172.21.15.94:37336) 2024-02-17T11:58:30.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.113+0000 7fc6abfff640 1 -- 172.21.15.94:0/2767510947 learned_addr learned my addr 172.21.15.94:0/2767510947 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.113+0000 7fc6a8ff9640 1 -- 172.21.15.94:0/1565224902 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7fc6a000f540 con 0x7fc6ac103c40 2024-02-17T11:58:30.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.113+0000 7fc6ab7fe640 1 --2- 172.21.15.94:0/2767510947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.113+0000 7fc6abfff640 1 -- 172.21.15.94:0/2767510947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9ab70 msgr2=0x55c0a9da3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.113+0000 7fc6abfff640 1 --2- 172.21.15.94:0/2767510947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.113+0000 7fc6abfff640 1 -- 172.21.15.94:0/2767510947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9da3750 msgr2=0x55c0a9da5b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T11:58:30.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.113+0000 7fc6abfff640 1 --2- 172.21.15.94:0/2767510947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9da3750 0x55c0a9da5b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.113+0000 7fc6abfff640 1 -- 172.21.15.94:0/2767510947 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0a9cc4060 con 0x55c0a9d9a250 2024-02-17T11:58:30.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.114+0000 7fc6abfff640 1 --2- 172.21.15.94:0/2767510947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9a250 0x55c0a9d9a630 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009930 tx=0x7fc69c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f534005c7d44794 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.114+0000 7fc6a8ff9640 1 --2- 172.21.15.94:0/1565224902 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6940a22c0 0x7fc6940a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.114+0000 7fc691ffb640 1 -- 172.21.15.94:0/2767510947 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c004970 con 0x55c0a9d9a250 2024-02-17T11:58:30.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.114+0000 7fc691ffb640 1 -- 172.21.15.94:0/2767510947 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc69c004ad0 con 0x55c0a9d9a250 2024-02-17T11:58:30.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.114+0000 7fc691ffb640 1 -- 172.21.15.94:0/2767510947 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c005210 con 0x55c0a9d9a250 2024-02-17T11:58:30.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.115+0000 7fc6a8ff9640 1 -- 172.21.15.94:0/1565224902 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc6a009acb0 con 0x7fc6ac103c40 2024-02-17T11:58:30.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.115+0000 7fc6b4974740 1 -- 172.21.15.94:0/2767510947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9a250 msgr2=0x55c0a9d9a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.115+0000 7fc6b4974740 1 --2- 172.21.15.94:0/2767510947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9a250 0x55c0a9d9a630 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009930 tx=0x7fc69c0046c0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.115+0000 7fc6b4974740 1 -- 172.21.15.94:0/2767510947 shutdown_connections 2024-02-17T11:58:30.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.115+0000 7fc6b4974740 1 --2- 172.21.15.94:0/2767510947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9da3750 0x55c0a9da5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.115+0000 7fc6b4974740 1 --2- 172.21.15.94:0/2767510947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.115+0000 7fc6b4974740 1 --2- 172.21.15.94:0/2767510947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9a250 0x55c0a9d9a630 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.115+0000 7fc6b4974740 1 -- 172.21.15.94:0/2767510947 >> 172.21.15.94:0/2767510947 conn(0x55c0a9d94980 msgr2=0x55c0a9d94d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.116+0000 7fc6ab7fe640 1 --2- 172.21.15.94:0/1565224902 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6940a22c0 0x7fc6940a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.117+0000 7fc6ab7fe640 1 --2- 172.21.15.94:0/1565224902 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6940a22c0 0x7fc6940a4780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009fd0 tx=0x7fc6a4009500 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.117+0000 7fc6b4974740 1 -- 172.21.15.94:0/2767510947 shutdown_connections 2024-02-17T11:58:30.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.117+0000 7fc6b4974740 1 -- 172.21.15.94:0/2767510947 wait complete. 2024-02-17T11:58:30.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.118+0000 7fc6b4974740 1 -- start start 2024-02-17T11:58:30.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.118+0000 7fc6b4974740 1 Processor -- start 2024-02-17T11:58:30.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.119+0000 7fc6b4974740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9d9a250 0x55c0a9da0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.119+0000 7fc6b4974740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.119+0000 7fc6b4974740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9da3750 0x55c0a9da12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.119+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9d9a250 0x55c0a9da0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.119+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9d9a250 0x55c0a9da0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45474/0 (socket says 172.21.15.94:45474) 2024-02-17T11:58:30.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.119+0000 7fc6aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.119+0000 7fc6aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37346/0 (socket says 172.21.15.94:37346) 2024-02-17T11:58:30.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.119+0000 7fc6abfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9da3750 0x55c0a9da12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.119+0000 7fc6ab7fe640 1 -- 172.21.15.94:0/190559884 learned_addr learned my addr 172.21.15.94:0/190559884 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.119+0000 7fc6b4974740 1 -- 172.21.15.94:0/190559884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a9d02020 con 0x55c0a9d9ab70 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.119+0000 7fc6b4974740 1 -- 172.21.15.94:0/190559884 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a99d5430 con 0x55c0a9d9a250 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.119+0000 7fc6b4974740 1 -- 172.21.15.94:0/190559884 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a9991e00 con 0x55c0a9da3750 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.119+0000 7fc6b4974740 10 client.0 _ll_register_callbacks cb 0x55c0a9ca9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.120+0000 7fc6abfff640 1 -- 172.21.15.94:0/190559884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9d9a250 msgr2=0x55c0a9da0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.120+0000 7fc6abfff640 1 --2- 172.21.15.94:0/190559884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9d9a250 0x55c0a9da0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.120+0000 7fc6abfff640 1 -- 172.21.15.94:0/190559884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9ab70 msgr2=0x55c0a9da0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.120+0000 7fc6abfff640 1 --2- 172.21.15.94:0/190559884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.120+0000 7fc6abfff640 1 -- 172.21.15.94:0/190559884 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ac114f10 con 0x55c0a9da3750 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.120+0000 7fc6abfff640 1 --2- 172.21.15.94:0/190559884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9da3750 0x55c0a9da12a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009900 tx=0x7fc69c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c16ba42dfe2d0e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.120+0000 7fc68affd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.120+0000 7fc68affd640 1 -- 172.21.15.94:0/190559884 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c004970 con 0x55c0a9da3750 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.120+0000 7fc68affd640 1 -- 172.21.15.94:0/190559884 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc69c04a450 con 0x55c0a9da3750 2024-02-17T11:58:30.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.120+0000 7fc6b4974740 1 -- 172.21.15.94:0/190559884 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c0a9a017e0 con 0x55c0a9da3750 2024-02-17T11:58:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.121+0000 7fc68affd640 1 -- 172.21.15.94:0/190559884 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc69c004440 con 0x55c0a9da3750 2024-02-17T11:58:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.121+0000 7fc6b4974740 10 client.25096 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.121+0000 7fc6b4974740 1 -- 172.21.15.94:0/190559884 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55c0a9d006a0 con 0x55c0a9da3750 2024-02-17T11:58:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.121+0000 7fc68affd640 1 -- 172.21.15.94:0/190559884 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c0a9a017e0 con 0x55c0a9da3750 2024-02-17T11:58:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.121+0000 7fc68affd640 1 -- 172.21.15.94:0/190559884 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc69c03c050 con 0x55c0a9da3750 2024-02-17T11:58:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.121+0000 7fc6b4974740 10 client.25096 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.121+0000 7fc6b4974740 10 client.25096 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T11:58:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.121+0000 7fc6b4974740 20 client.25096 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.121+0000 7fc6b4974740 1 --2- 172.21.15.94:0/190559884 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c0a9db3640 0x55c0a9db5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.121+0000 7fc6b4974740 4 client.25096 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:58:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.121+0000 7fc6b4974740 1 -- 172.21.15.94:0/190559884 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c0a9a3e720 con 0x55c0a9db3640 2024-02-17T11:58:30.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.121+0000 7fc6ab7fe640 1 --2- 172.21.15.94:0/190559884 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c0a9db3640 0x55c0a9db5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.122+0000 7fc6ab7fe640 1 --2- 172.21.15.94:0/190559884 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c0a9db3640 0x55c0a9db5a30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=395b79b263ba922 server_cookie=4fb5b3c08582cf81 in_seq=0 out_seq=0 2024-02-17T11:58:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.122+0000 7fc68affd640 10 client.25096 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T11:58:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc68affd640 1 -- 172.21.15.94:0/190559884 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c0a9a3e720 con 0x55c0a9db3640 2024-02-17T11:58:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc68affd640 10 client.25096 handle_command_reply: tid=0 2024-02-17T11:58:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc6b4974740 1 client.25096 shutdown 2024-02-17T11:58:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc6b4974740 1 -- 172.21.15.94:0/190559884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9da3750 msgr2=0x55c0a9da12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc6b4974740 1 --2- 172.21.15.94:0/190559884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9da3750 0x55c0a9da12a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc69c009900 tx=0x7fc69c005fd0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc6b4974740 1 -- 172.21.15.94:0/190559884 shutdown_connections 2024-02-17T11:58:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc6b4974740 1 --2- 172.21.15.94:0/190559884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9da3750 0x55c0a9da12a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc6b4974740 1 --2- 172.21.15.94:0/190559884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc6b4974740 1 --2- 172.21.15.94:0/190559884 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c0a9db3640 0x55c0a9db5a30 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc68affd640 0 client.25096 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T11:58:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc6b4974740 1 --2- 172.21.15.94:0/190559884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9d9a250 0x55c0a9da0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc6b4974740 1 -- 172.21.15.94:0/190559884 >> 172.21.15.94:0/190559884 conn(0x55c0a9d94980 msgr2=0x55c0a9d966d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc6b4974740 1 -- 172.21.15.94:0/190559884 shutdown_connections 2024-02-17T11:58:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc6b4974740 1 -- 172.21.15.94:0/190559884 wait complete. 2024-02-17T11:58:30.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.123+0000 7fc6b4974740 20 client.25096 trim_cache size 0 max 16384 2024-02-17T11:58:30.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6b4974740 1 Processor -- start 2024-02-17T11:58:30.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6b4974740 1 -- start start 2024-02-17T11:58:30.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6b4974740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9a250 0x55c0a9da2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6b4974740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9d9ab70 0x55c0a9d9bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6b4974740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9da3750 0x55c0a9d9c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6b4974740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a9c87130 con 0x55c0a9da3750 2024-02-17T11:58:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6b4974740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a9da17f0 con 0x55c0a9d9ab70 2024-02-17T11:58:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6b4974740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a9dbac50 con 0x55c0a9d9a250 2024-02-17T11:58:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6abfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9da3750 0x55c0a9d9c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6abfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9da3750 0x55c0a9d9c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37354/0 (socket says 172.21.15.94:37354) 2024-02-17T11:58:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6abfff640 1 -- 172.21.15.94:0/4276101519 learned_addr learned my addr 172.21.15.94:0/4276101519 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6ab7fe640 1 --2- 172.21.15.94:0/4276101519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9a250 0x55c0a9da2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6aaffd640 1 --2- 172.21.15.94:0/4276101519 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9d9ab70 0x55c0a9d9bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6abfff640 1 -- 172.21.15.94:0/4276101519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9a250 msgr2=0x55c0a9da2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.139+0000 7fc6abfff640 1 --2- 172.21.15.94:0/4276101519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9a250 0x55c0a9da2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.140+0000 7fc6abfff640 1 -- 172.21.15.94:0/4276101519 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9d9ab70 msgr2=0x55c0a9d9bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.140+0000 7fc6abfff640 1 --2- 172.21.15.94:0/4276101519 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9d9ab70 0x55c0a9d9bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.140+0000 7fc6abfff640 1 -- 172.21.15.94:0/4276101519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c0a9c9cc20 con 0x55c0a9da3750 2024-02-17T11:58:30.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.140+0000 7fc6abfff640 1 --2- 172.21.15.94:0/4276101519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9da3750 0x55c0a9d9c050 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc69c02c180 tx=0x7fc69c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39ec2440950af46 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.140+0000 7fc6917fa640 1 -- 172.21.15.94:0/4276101519 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c03f850 con 0x55c0a9da3750 2024-02-17T11:58:30.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.140+0000 7fc6917fa640 1 -- 172.21.15.94:0/4276101519 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc69c02cb00 con 0x55c0a9da3750 2024-02-17T11:58:30.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.140+0000 7fc6917fa640 1 -- 172.21.15.94:0/4276101519 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c005790 con 0x55c0a9da3750 2024-02-17T11:58:30.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.141+0000 7fc6b4974740 1 -- 172.21.15.94:0/4276101519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9da3750 msgr2=0x55c0a9d9c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.141+0000 7fc6b4974740 1 --2- 172.21.15.94:0/4276101519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9da3750 0x55c0a9d9c050 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc69c02c180 tx=0x7fc69c005a00 comp rx=0 tx=0).stop 2024-02-17T11:58:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.141+0000 7fc6b4974740 1 -- 172.21.15.94:0/4276101519 shutdown_connections 2024-02-17T11:58:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.141+0000 7fc6b4974740 1 --2- 172.21.15.94:0/4276101519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9da3750 0x55c0a9d9c050 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.141+0000 7fc6b4974740 1 --2- 172.21.15.94:0/4276101519 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9d9ab70 0x55c0a9d9bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.141+0000 7fc6b4974740 1 --2- 172.21.15.94:0/4276101519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9a250 0x55c0a9da2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.141+0000 7fc6b4974740 1 -- 172.21.15.94:0/4276101519 >> 172.21.15.94:0/4276101519 conn(0x55c0a9d94980 msgr2=0x55c0a9d8fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.141+0000 7fc6b4974740 1 -- 172.21.15.94:0/4276101519 shutdown_connections 2024-02-17T11:58:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.142+0000 7fc6b4974740 1 -- 172.21.15.94:0/4276101519 wait complete. 2024-02-17T11:58:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.142+0000 7fc6b4974740 1 -- start start 2024-02-17T11:58:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.143+0000 7fc6b4974740 1 Processor -- start 2024-02-17T11:58:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.143+0000 7fc6b4974740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9a250 0x55c0a9da8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.143+0000 7fc6b4974740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.143+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9a250 0x55c0a9da8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6ab7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9a250 0x55c0a9da8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:58002/0 (socket says 172.21.15.94:58002) 2024-02-17T11:58:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6ab7fe640 1 -- 172.21.15.94:0/2056910133 learned_addr learned my addr 172.21.15.94:0/2056910133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6b4974740 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9da3750 0x55c0a9da9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6b4974740 1 -- 172.21.15.94:0/2056910133 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a9e09b80 con 0x55c0a9d9ab70 2024-02-17T11:58:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6b4974740 1 -- 172.21.15.94:0/2056910133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a9e09cf0 con 0x55c0a9da3750 2024-02-17T11:58:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6b4974740 1 -- 172.21.15.94:0/2056910133 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c0a9e09eb0 con 0x55c0a9d9a250 2024-02-17T11:58:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6abfff640 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9da3750 0x55c0a9da9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6aaffd640 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6b4974740 10 client.0 _ll_register_callbacks cb 0x7fc69c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6abfff640 1 -- 172.21.15.94:0/2056910133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9a250 msgr2=0x55c0a9da8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6abfff640 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9a250 0x55c0a9da8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6abfff640 1 -- 172.21.15.94:0/2056910133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9ab70 msgr2=0x55c0a9da9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6abfff640 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6abfff640 1 -- 172.21.15.94:0/2056910133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c0a9d006a0 con 0x55c0a9da3750 2024-02-17T11:58:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6b4974740 1 -- 172.21.15.94:0/2056910133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc69c004ad0 con 0x55c0a9da3750 2024-02-17T11:58:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.144+0000 7fc6aaffd640 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:58:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.145+0000 7fc6abfff640 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9da3750 0x55c0a9da9a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc69c055040 tx=0x7fc69c005a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a4b6b1d91df02 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.145+0000 7fc68affd640 10 client.25061 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:58:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.145+0000 7fc68affd640 1 -- 172.21.15.94:0/2056910133 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c03fd00 con 0x55c0a9da3750 2024-02-17T11:58:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.145+0000 7fc68affd640 1 -- 172.21.15.94:0/2056910133 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc69c004030 con 0x55c0a9da3750 2024-02-17T11:58:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.145+0000 7fc68affd640 1 -- 172.21.15.94:0/2056910133 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc69c004250 con 0x55c0a9da3750 2024-02-17T11:58:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.145+0000 7fc6b4974740 10 client.25061 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:30.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.145+0000 7fc6b4974740 1 -- 172.21.15.94:0/2056910133 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({fsmap=21}) v3 -- 0x55c0a9cc28c0 con 0x55c0a9da3750 2024-02-17T11:58:30.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.146+0000 7fc68affd640 1 -- 172.21.15.94:0/2056910133 <== mon.2 v2:172.21.15.125:3300/0 4 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc69c03c040 con 0x55c0a9da3750 2024-02-17T11:58:30.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.147+0000 7fc6b4974740 10 client.25061 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:30.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.147+0000 7fc6b4974740 10 client.25061 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T11:58:30.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.147+0000 7fc6b4974740 20 client.25061 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:30.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.147+0000 7fc6b4974740 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c0a9e0b0c0 0x55c0a9e0d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.147+0000 7fc6b4974740 4 client.25061 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T11:58:30.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.147+0000 7fc6b4974740 1 -- 172.21.15.94:0/2056910133 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c0a9a8d890 con 0x55c0a9e0b0c0 2024-02-17T11:58:30.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.148+0000 7fc6ab7fe640 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c0a9e0b0c0 0x55c0a9e0d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.149+0000 7fc6ab7fe640 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c0a9e0b0c0 0x55c0a9e0d4b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74ee8118d902dc6a server_cookie=d2103614845f8763 in_seq=0 out_seq=0 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.149+0000 7fc68affd640 10 client.25061 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc68affd640 1 -- 172.21.15.94:0/2056910133 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c0a9a8d890 con 0x55c0a9e0b0c0 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc68affd640 10 client.25061 handle_command_reply: tid=0 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc6b4974740 1 client.25061 shutdown 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc6b4974740 1 -- 172.21.15.94:0/2056910133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9da3750 msgr2=0x55c0a9da9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc6b4974740 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9da3750 0x55c0a9da9a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc69c055040 tx=0x7fc69c005a70 comp rx=0 tx=0).stop 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc6b4974740 1 -- 172.21.15.94:0/2056910133 shutdown_connections 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc6b4974740 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c0a9e0b0c0 0x55c0a9e0d4b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc68affd640 0 client.25061 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc6b4974740 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c0a9da3750 0x55c0a9da9a40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc6b4974740 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c0a9d9ab70 0x55c0a9da9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc6b4974740 1 --2- 172.21.15.94:0/2056910133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c0a9d9a250 0x55c0a9da8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc6b4974740 1 -- 172.21.15.94:0/2056910133 >> 172.21.15.94:0/2056910133 conn(0x55c0a9d94980 msgr2=0x55c0a9d9bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc6b4974740 1 -- 172.21.15.94:0/2056910133 shutdown_connections 2024-02-17T11:58:30.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc6b4974740 1 -- 172.21.15.94:0/2056910133 wait complete. 2024-02-17T11:58:30.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.150+0000 7fc6b4974740 20 client.25061 trim_cache size 0 max 16384 2024-02-17T11:58:30.152 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T11:58:30.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.151+0000 7fc6b1cc3640 1 -- 172.21.15.94:0/1565224902 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6940a22c0 msgr2=0x7fc6940a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.151+0000 7fc6b1cc3640 1 --2- 172.21.15.94:0/1565224902 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6940a22c0 0x7fc6940a4780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009fd0 tx=0x7fc6a4009500 comp rx=0 tx=0).stop 2024-02-17T11:58:30.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.151+0000 7fc6b1cc3640 1 -- 172.21.15.94:0/1565224902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6ac103c40 msgr2=0x7fc6ac1a3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.151+0000 7fc6b1cc3640 1 --2- 172.21.15.94:0/1565224902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6ac103c40 0x7fc6ac1a3fb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009fd0 tx=0x7fc6a000b870 comp rx=0 tx=0).stop 2024-02-17T11:58:30.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.151+0000 7fc6b1cc3640 1 -- 172.21.15.94:0/1565224902 shutdown_connections 2024-02-17T11:58:30.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.151+0000 7fc6b1cc3640 1 --2- 172.21.15.94:0/1565224902 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6940a22c0 0x7fc6940a4780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.151+0000 7fc6b1cc3640 1 --2- 172.21.15.94:0/1565224902 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6ac1060d0 0x7fc6ac19db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.151+0000 7fc6b1cc3640 1 --2- 172.21.15.94:0/1565224902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6ac103c40 0x7fc6ac1a3fb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.151+0000 7fc6b1cc3640 1 --2- 172.21.15.94:0/1565224902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6ac0688c0 0x7fc6ac1a3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.151+0000 7fc6b1cc3640 1 -- 172.21.15.94:0/1565224902 >> 172.21.15.94:0/1565224902 conn(0x7fc6ac077180 msgr2=0x7fc6ac100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.152+0000 7fc6b1cc3640 1 -- 172.21.15.94:0/1565224902 shutdown_connections 2024-02-17T11:58:30.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.152+0000 7fc6b1cc3640 1 -- 172.21.15.94:0/1565224902 wait complete. 2024-02-17T11:58:30.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:29 smithi189 ceph-mon[36588]: pgmap v14: 97 pgs: 97 active+clean; 915 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 81 MiB/s wr, 766 op/s 2024-02-17T11:58:30.167 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T11:58:30.167 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T11:58:30.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:29 smithi125 ceph-mon[36819]: pgmap v14: 97 pgs: 97 active+clean; 915 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 81 MiB/s wr, 766 op/s 2024-02-17T11:58:30.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 10.0 s, 3954.3 tps, lat 25.306 ms stddev 26.087 2024-02-17T11:58:30.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.327+0000 7f4e582c3640 1 -- 172.21.15.94:0/2419694014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e50108360 msgr2=0x7f4e50108740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.327+0000 7f4e582c3640 1 --2- 172.21.15.94:0/2419694014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e50108360 0x7f4e50108740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009900 tx=0x7f4e4402bc40 comp rx=0 tx=0).stop 2024-02-17T11:58:30.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.327+0000 7f4e582c3640 1 -- 172.21.15.94:0/2419694014 shutdown_connections 2024-02-17T11:58:30.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.327+0000 7f4e582c3640 1 --2- 172.21.15.94:0/2419694014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e501142d0 0x7f4e501166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.327+0000 7f4e582c3640 1 --2- 172.21.15.94:0/2419694014 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e50108c80 0x7f4e50113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.327+0000 7f4e582c3640 1 --2- 172.21.15.94:0/2419694014 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e50108360 0x7f4e50108740 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.327+0000 7f4e582c3640 1 -- 172.21.15.94:0/2419694014 >> 172.21.15.94:0/2419694014 conn(0x7f4e5006c260 msgr2=0x7f4e5006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.327+0000 7f4e582c3640 1 -- 172.21.15.94:0/2419694014 shutdown_connections 2024-02-17T11:58:30.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.327+0000 7f4e582c3640 1 -- 172.21.15.94:0/2419694014 wait complete. 2024-02-17T11:58:30.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.328+0000 7f4e582c3640 1 Processor -- start 2024-02-17T11:58:30.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.328+0000 7f4e582c3640 1 -- start start 2024-02-17T11:58:30.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.328+0000 7f4e582c3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e50108360 0x7f4e50113690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.328+0000 7f4e582c3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e50108c80 0x7f4e50113bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.328+0000 7f4e582c3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e501142d0 0x7f4e5010d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.328+0000 7f4e582c3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5010df50 con 0x7f4e50108360 2024-02-17T11:58:30.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.328+0000 7f4e582c3640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5010e0c0 con 0x7f4e50108c80 2024-02-17T11:58:30.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.328+0000 7f4e582c3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5010e280 con 0x7f4e501142d0 2024-02-17T11:58:30.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.328+0000 7f4e56860640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e501142d0 0x7f4e5010d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.328+0000 7f4e56860640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e501142d0 0x7f4e5010d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:58018/0 (socket says 172.21.15.94:58018) 2024-02-17T11:58:30.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.328+0000 7f4e56860640 1 -- 172.21.15.94:0/3233702393 learned_addr learned my addr 172.21.15.94:0/3233702393 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.328+0000 7f4e5605f640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e50108360 0x7f4e50113690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.329+0000 7f4e5585e640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e50108c80 0x7f4e50113bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.329+0000 7f4e56860640 1 -- 172.21.15.94:0/3233702393 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e50108c80 msgr2=0x7f4e50113bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.329+0000 7f4e56860640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e50108c80 0x7f4e50113bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.329+0000 7f4e56860640 1 -- 172.21.15.94:0/3233702393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e50108360 msgr2=0x7f4e50113690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.329+0000 7f4e56860640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e50108360 0x7f4e50113690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.329+0000 7f4e56860640 1 -- 172.21.15.94:0/3233702393 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e44009560 con 0x7f4e501142d0 2024-02-17T11:58:30.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.329+0000 7f4e56860640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e501142d0 0x7f4e5010d7c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800e9d0 tx=0x7f4e4800ee90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.329+0000 7f4e437fe640 1 -- 172.21.15.94:0/3233702393 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4800cd80 con 0x7f4e501142d0 2024-02-17T11:58:30.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.330+0000 7f4e437fe640 1 -- 172.21.15.94:0/3233702393 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e48005020 con 0x7f4e501142d0 2024-02-17T11:58:30.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.330+0000 7f4e582c3640 1 -- 172.21.15.94:0/3233702393 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5010e560 con 0x7f4e501142d0 2024-02-17T11:58:30.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.330+0000 7f4e437fe640 1 -- 172.21.15.94:0/3233702393 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480116d0 con 0x7f4e501142d0 2024-02-17T11:58:30.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.330+0000 7f4e582c3640 1 -- 172.21.15.94:0/3233702393 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e501bba90 con 0x7f4e501142d0 2024-02-17T11:58:30.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.333+0000 7f4e437fe640 1 -- 172.21.15.94:0/3233702393 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f4e48005500 con 0x7f4e501142d0 2024-02-17T11:58:30.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.336+0000 7f4e437fe640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4e3c0a2330 0x7f4e3c0a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.336+0000 7f4e5af72740 1 Processor -- start 2024-02-17T11:58:30.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.336+0000 7f4e437fe640 1 -- 172.21.15.94:0/3233702393 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4e480ce3b0 con 0x7f4e501142d0 2024-02-17T11:58:30.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.336+0000 7f4e5605f640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4e3c0a2330 0x7f4e3c0a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.336+0000 7f4e5af72740 1 -- start start 2024-02-17T11:58:30.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.336+0000 7f4e5af72740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0db250 0x55cf8b0db630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.336+0000 7f4e5af72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0e4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.337+0000 7f4e5585e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.337+0000 7f4e5585e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0e4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37390/0 (socket says 172.21.15.94:37390) 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.338+0000 7f4e5af72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0e4750 0x55cf8b0e6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.338+0000 7f4e5af72740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8b096960 con 0x55cf8b0dbb70 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.338+0000 7f4e5af72740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8b04a310 con 0x55cf8b0db250 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.338+0000 7f4e5af72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8b047840 con 0x55cf8b0e4750 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.338+0000 7f4e5585e640 1 -- 172.21.15.94:0/105700530 learned_addr learned my addr 172.21.15.94:0/105700530 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.338+0000 7f4e56860640 1 --2- 172.21.15.94:0/105700530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0e4750 0x55cf8b0e6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.338+0000 7f4e5605f640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4e3c0a2330 0x7f4e3c0a47f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e44005ab0 tx=0x7f4e44005a20 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.338+0000 7f4e5605f640 1 --2- 172.21.15.94:0/105700530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0db250 0x55cf8b0db630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.339+0000 7f4e56860640 1 -- 172.21.15.94:0/105700530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0db250 msgr2=0x55cf8b0db630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.339+0000 7f4e56860640 1 --2- 172.21.15.94:0/105700530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0db250 0x55cf8b0db630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.339+0000 7f4e56860640 1 -- 172.21.15.94:0/105700530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0dbb70 msgr2=0x55cf8b0e4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.339+0000 7f4e56860640 1 --2- 172.21.15.94:0/105700530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0e4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.339+0000 7f4e56860640 1 -- 172.21.15.94:0/105700530 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf8b005060 con 0x55cf8b0e4750 2024-02-17T11:58:30.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.339+0000 7f4e5605f640 1 --2- 172.21.15.94:0/105700530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0db250 0x55cf8b0db630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:58:30.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.339+0000 7f4e5585e640 1 --2- 172.21.15.94:0/105700530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:30.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.339+0000 7f4e56860640 1 --2- 172.21.15.94:0/105700530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0e4750 0x55cf8b0e6b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e480cba30 tx=0x7f4e48096ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb46738d4f81eec1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.339+0000 7f4e40bfe640 1 -- 172.21.15.94:0/105700530 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4809ed70 con 0x55cf8b0e4750 2024-02-17T11:58:30.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.339+0000 7f4e40bfe640 1 -- 172.21.15.94:0/105700530 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e480a33c0 con 0x55cf8b0e4750 2024-02-17T11:58:30.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.339+0000 7f4e40bfe640 1 -- 172.21.15.94:0/105700530 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4809a2d0 con 0x55cf8b0e4750 2024-02-17T11:58:30.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.340+0000 7f4e5af72740 1 -- 172.21.15.94:0/105700530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0e4750 msgr2=0x55cf8b0e6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.340+0000 7f4e5af72740 1 --2- 172.21.15.94:0/105700530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0e4750 0x55cf8b0e6b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e480cba30 tx=0x7f4e48096ff0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.341+0000 7f4e5af72740 1 -- 172.21.15.94:0/105700530 shutdown_connections 2024-02-17T11:58:30.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.341+0000 7f4e5af72740 1 --2- 172.21.15.94:0/105700530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0e4750 0x55cf8b0e6b40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.341+0000 7f4e5af72740 1 --2- 172.21.15.94:0/105700530 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.341+0000 7f4e5af72740 1 --2- 172.21.15.94:0/105700530 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0db250 0x55cf8b0db630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.341+0000 7f4e5af72740 1 -- 172.21.15.94:0/105700530 >> 172.21.15.94:0/105700530 conn(0x55cf8b0d5980 msgr2=0x55cf8b0d5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.341+0000 7f4e5af72740 1 -- 172.21.15.94:0/105700530 shutdown_connections 2024-02-17T11:58:30.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.341+0000 7f4e5af72740 1 -- 172.21.15.94:0/105700530 wait complete. 2024-02-17T11:58:30.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.341+0000 7f4e5af72740 1 -- start start 2024-02-17T11:58:30.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.341+0000 7f4e5af72740 1 Processor -- start 2024-02-17T11:58:30.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.342+0000 7f4e5af72740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0db250 0x55cf8b0e17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.342+0000 7f4e5af72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0e1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.342+0000 7f4e5af72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0e4750 0x55cf8b0e2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.342+0000 7f4e5605f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0db250 0x55cf8b0e17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.342+0000 7f4e5605f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0db250 0x55cf8b0e17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45540/0 (socket says 172.21.15.94:45540) 2024-02-17T11:58:30.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.342+0000 7f4e5605f640 1 -- 172.21.15.94:0/534313614 learned_addr learned my addr 172.21.15.94:0/534313614 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.342+0000 7f4e5af72740 1 -- 172.21.15.94:0/534313614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8b043020 con 0x55cf8b0e4750 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.342+0000 7f4e5af72740 1 -- 172.21.15.94:0/534313614 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8ad16430 con 0x55cf8b0db250 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.342+0000 7f4e5af72740 1 -- 172.21.15.94:0/534313614 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8acd2e00 con 0x55cf8b0dbb70 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.342+0000 7f4e5af72740 10 client.0 _ll_register_callbacks cb 0x55cf8afeacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.342+0000 7f4e56860640 1 --2- 172.21.15.94:0/534313614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0e4750 0x55cf8b0e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.343+0000 7f4e56860640 1 -- 172.21.15.94:0/534313614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0dbb70 msgr2=0x55cf8b0e1d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.343+0000 7f4e56860640 1 --2- 172.21.15.94:0/534313614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0e1d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.343+0000 7f4e56860640 1 -- 172.21.15.94:0/534313614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0db250 msgr2=0x55cf8b0e17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.343+0000 7f4e56860640 1 --2- 172.21.15.94:0/534313614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0db250 0x55cf8b0e17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.343+0000 7f4e56860640 1 -- 172.21.15.94:0/534313614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e501bba90 con 0x55cf8b0e4750 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.343+0000 7f4e5605f640 1 --2- 172.21.15.94:0/534313614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0db250 0x55cf8b0e17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.343+0000 7f4e56860640 1 --2- 172.21.15.94:0/534313614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0e4750 0x55cf8b0e2260 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4e4809ade0 tx=0x7f4e4809a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e762ea100061981b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.344+0000 7f4e39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.344+0000 7f4e39ffb640 1 -- 172.21.15.94:0/534313614 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4809ed70 con 0x55cf8b0e4750 2024-02-17T11:58:30.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.344+0000 7f4e39ffb640 1 -- 172.21.15.94:0/534313614 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4e48096b80 con 0x55cf8b0e4750 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.344+0000 7f4e5af72740 1 -- 172.21.15.94:0/534313614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf8ad427e0 con 0x55cf8b0e4750 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.344+0000 7f4e39ffb640 1 -- 172.21.15.94:0/534313614 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e48099ae0 con 0x55cf8b0e4750 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.344+0000 7f4e5af72740 10 client.15576 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.344+0000 7f4e5af72740 1 -- 172.21.15.94:0/534313614 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55cf8b0416a0 con 0x55cf8b0e4750 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.344+0000 7f4e39ffb640 1 -- 172.21.15.94:0/534313614 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e4807d080 con 0x55cf8b0e4750 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.344+0000 7f4e39ffb640 1 -- 172.21.15.94:0/534313614 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4e480ce070 con 0x55cf8b0e4750 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.344+0000 7f4e5af72740 10 client.15576 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.345+0000 7f4e5af72740 10 client.15576 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.345+0000 7f4e5af72740 20 client.15576 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.345+0000 7f4e5af72740 1 --2- 172.21.15.94:0/534313614 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cf8b0f4560 0x55cf8b0f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.345+0000 7f4e5af72740 4 client.15576 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.345+0000 7f4e5af72740 1 -- 172.21.15.94:0/534313614 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf8ad427e0 con 0x55cf8b0f4560 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.345+0000 7f4e5605f640 1 --2- 172.21.15.94:0/534313614 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cf8b0f4560 0x55cf8b0f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.346+0000 7f4e5605f640 1 --2- 172.21.15.94:0/534313614 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cf8b0f4560 0x55cf8b0f69b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=634e00ec1f06b653 server_cookie=494c124d1d5c030a in_seq=0 out_seq=0 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.346+0000 7f4e39ffb640 10 client.15576 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e39ffb640 1 -- 172.21.15.94:0/534313614 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cf8ad427e0 con 0x55cf8b0f4560 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e39ffb640 10 client.15576 handle_command_reply: tid=0 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e5af72740 1 client.15576 shutdown 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e5af72740 1 -- 172.21.15.94:0/534313614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0e4750 msgr2=0x55cf8b0e2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e5af72740 1 --2- 172.21.15.94:0/534313614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0e4750 0x55cf8b0e2260 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4e4809ade0 tx=0x7f4e4809a2a0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e5af72740 1 -- 172.21.15.94:0/534313614 shutdown_connections 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e5af72740 1 --2- 172.21.15.94:0/534313614 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cf8b0f4560 0x55cf8b0f69b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e5af72740 1 --2- 172.21.15.94:0/534313614 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0e4750 0x55cf8b0e2260 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e39ffb640 0 client.15576 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e5af72740 1 --2- 172.21.15.94:0/534313614 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0e1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e5af72740 1 --2- 172.21.15.94:0/534313614 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0db250 0x55cf8b0e17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e5af72740 1 -- 172.21.15.94:0/534313614 >> 172.21.15.94:0/534313614 conn(0x55cf8b0d5980 msgr2=0x55cf8b0d76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e5af72740 1 -- 172.21.15.94:0/534313614 shutdown_connections 2024-02-17T11:58:30.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.347+0000 7f4e5af72740 1 -- 172.21.15.94:0/534313614 wait complete. 2024-02-17T11:58:30.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.348+0000 7f4e5af72740 20 client.15576 trim_cache size 0 max 16384 2024-02-17T11:58:30.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.369+0000 7f4e5af72740 1 Processor -- start 2024-02-17T11:58:30.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e5af72740 1 -- start start 2024-02-17T11:58:30.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e5af72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0db250 0x55cf8b0e3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e5af72740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0fb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e5af72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0e4750 0x55cf8b0dea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e5af72740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8afc8130 con 0x55cf8b0e4750 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e5af72740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8b0d6980 con 0x55cf8b0dbb70 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e5af72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8b0da620 con 0x55cf8b0db250 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e56860640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0e4750 0x55cf8b0dea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e5605f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0db250 0x55cf8b0e3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e56860640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0e4750 0x55cf8b0dea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37402/0 (socket says 172.21.15.94:37402) 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e56860640 1 -- 172.21.15.94:0/2098000632 learned_addr learned my addr 172.21.15.94:0/2098000632 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e56860640 1 -- 172.21.15.94:0/2098000632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0db250 msgr2=0x55cf8b0e3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e56860640 1 --2- 172.21.15.94:0/2098000632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0db250 0x55cf8b0e3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e56860640 1 -- 172.21.15.94:0/2098000632 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0dbb70 msgr2=0x55cf8b0fb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e5585e640 1 --2- 172.21.15.94:0/2098000632 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0fb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e56860640 1 --2- 172.21.15.94:0/2098000632 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0fb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e56860640 1 -- 172.21.15.94:0/2098000632 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf8afddc20 con 0x55cf8b0e4750 2024-02-17T11:58:30.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.370+0000 7f4e56860640 1 --2- 172.21.15.94:0/2098000632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0e4750 0x55cf8b0dea40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4e480cbb10 tx=0x7f4e4809b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f5329a1fc62408 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.371+0000 7f4e3bfff640 1 -- 172.21.15.94:0/2098000632 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480cbef0 con 0x55cf8b0e4750 2024-02-17T11:58:30.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.371+0000 7f4e3bfff640 1 -- 172.21.15.94:0/2098000632 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e4809a390 con 0x55cf8b0e4750 2024-02-17T11:58:30.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.371+0000 7f4e3bfff640 1 -- 172.21.15.94:0/2098000632 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e48098fb0 con 0x55cf8b0e4750 2024-02-17T11:58:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.371+0000 7f4e5af72740 1 -- 172.21.15.94:0/2098000632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0e4750 msgr2=0x55cf8b0dea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.371+0000 7f4e5af72740 1 --2- 172.21.15.94:0/2098000632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0e4750 0x55cf8b0dea40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4e480cbb10 tx=0x7f4e4809b3a0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.372+0000 7f4e5af72740 1 -- 172.21.15.94:0/2098000632 shutdown_connections 2024-02-17T11:58:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.372+0000 7f4e5af72740 1 --2- 172.21.15.94:0/2098000632 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0e4750 0x55cf8b0dea40 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.372+0000 7f4e5af72740 1 --2- 172.21.15.94:0/2098000632 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.372+0000 7f4e5af72740 1 --2- 172.21.15.94:0/2098000632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0db250 0x55cf8b0e3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.373+0000 7f4e5af72740 1 -- 172.21.15.94:0/2098000632 >> 172.21.15.94:0/2098000632 conn(0x55cf8b0d5980 msgr2=0x55cf8b0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.373+0000 7f4e5af72740 1 -- 172.21.15.94:0/2098000632 shutdown_connections 2024-02-17T11:58:30.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.373+0000 7f4e5af72740 1 -- 172.21.15.94:0/2098000632 wait complete. 2024-02-17T11:58:30.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.374+0000 7f4e5af72740 1 -- start start 2024-02-17T11:58:30.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.374+0000 7f4e5af72740 1 Processor -- start 2024-02-17T11:58:30.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.374+0000 7f4e5af72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0db250 0x55cf8b0ea190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.374+0000 7f4e5af72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0ea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.374+0000 7f4e5af72740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0e4750 0x55cf8b0eac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.374+0000 7f4e5af72740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8b14abc0 con 0x55cf8b0dbb70 2024-02-17T11:58:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.374+0000 7f4e5af72740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8b14ad30 con 0x55cf8b0e4750 2024-02-17T11:58:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.374+0000 7f4e5af72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf8b14aef0 con 0x55cf8b0db250 2024-02-17T11:58:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.374+0000 7f4e5af72740 10 client.0 _ll_register_callbacks cb 0x7f4e480cb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.375+0000 7f4e5605f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0db250 0x55cf8b0ea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.375+0000 7f4e5605f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0db250 0x55cf8b0ea190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:58060/0 (socket says 172.21.15.94:58060) 2024-02-17T11:58:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.375+0000 7f4e56860640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0e4750 0x55cf8b0eac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.375+0000 7f4e5605f640 1 -- 172.21.15.94:0/1995589617 learned_addr learned my addr 172.21.15.94:0/1995589617 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.375+0000 7f4e5585e640 1 --2- 172.21.15.94:0/1995589617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0ea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.375+0000 7f4e56860640 1 -- 172.21.15.94:0/1995589617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0db250 msgr2=0x55cf8b0ea190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.375+0000 7f4e56860640 1 --2- 172.21.15.94:0/1995589617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0db250 0x55cf8b0ea190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.375+0000 7f4e56860640 1 -- 172.21.15.94:0/1995589617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0dbb70 msgr2=0x55cf8b0ea6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.375+0000 7f4e56860640 1 --2- 172.21.15.94:0/1995589617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0ea6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.375+0000 7f4e56860640 1 -- 172.21.15.94:0/1995589617 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cf8b0416a0 con 0x55cf8b0e4750 2024-02-17T11:58:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.376+0000 7f4e56860640 1 --2- 172.21.15.94:0/1995589617 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0e4750 0x55cf8b0eac10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e480975a0 tx=0x7f4e48099d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b01c952f88b7169 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.376+0000 7f4e39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:58:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.376+0000 7f4e39ffb640 1 -- 172.21.15.94:0/1995589617 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e480a00a0 con 0x55cf8b0e4750 2024-02-17T11:58:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.376+0000 7f4e5af72740 1 -- 172.21.15.94:0/1995589617 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e480a33c0 con 0x55cf8b0e4750 2024-02-17T11:58:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.376+0000 7f4e39ffb640 1 -- 172.21.15.94:0/1995589617 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4e4809ed70 con 0x55cf8b0e4750 2024-02-17T11:58:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.376+0000 7f4e39ffb640 1 -- 172.21.15.94:0/1995589617 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e48098f90 con 0x55cf8b0e4750 2024-02-17T11:58:30.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.376+0000 7f4e5af72740 10 client.25073 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:30.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.376+0000 7f4e5af72740 1 -- 172.21.15.94:0/1995589617 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55cf8b0038c0 con 0x55cf8b0e4750 2024-02-17T11:58:30.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.377+0000 7f4e39ffb640 1 -- 172.21.15.94:0/1995589617 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4e480a33c0 con 0x55cf8b0e4750 2024-02-17T11:58:30.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.377+0000 7f4e39ffb640 1 -- 172.21.15.94:0/1995589617 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4e480ce040 con 0x55cf8b0e4750 2024-02-17T11:58:30.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.377+0000 7f4e5af72740 10 client.25073 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:30.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.377+0000 7f4e5af72740 10 client.25073 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T11:58:30.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.377+0000 7f4e5af72740 20 client.25073 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:30.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.377+0000 7f4e5af72740 1 --2- 172.21.15.94:0/1995589617 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cf8b0f4560 0x55cf8b14e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.377+0000 7f4e5af72740 4 client.25073 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T11:58:30.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.377+0000 7f4e5af72740 1 -- 172.21.15.94:0/1995589617 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cf8ad7f720 con 0x55cf8b0f4560 2024-02-17T11:58:30.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.378+0000 7f4e5605f640 1 --2- 172.21.15.94:0/1995589617 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cf8b0f4560 0x55cf8b14e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.379+0000 7f4e39ffb640 10 client.25073 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T11:58:30.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.379+0000 7f4e5605f640 1 --2- 172.21.15.94:0/1995589617 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cf8b0f4560 0x55cf8b14e1f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f31134e5110e44a8 server_cookie=2ba3d46df31061dc in_seq=0 out_seq=0 2024-02-17T11:58:30.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.379+0000 7f4e39ffb640 1 -- 172.21.15.94:0/1995589617 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cf8ad7f720 con 0x55cf8b0f4560 2024-02-17T11:58:30.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.379+0000 7f4e39ffb640 10 client.25073 handle_command_reply: tid=0 2024-02-17T11:58:30.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.379+0000 7f4e5af72740 1 client.25073 shutdown 2024-02-17T11:58:30.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.381+0000 7f4e5af72740 1 -- 172.21.15.94:0/1995589617 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0e4750 msgr2=0x55cf8b0eac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.381+0000 7f4e5af72740 1 --2- 172.21.15.94:0/1995589617 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0e4750 0x55cf8b0eac10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e480975a0 tx=0x7f4e48099d10 comp rx=0 tx=0).stop 2024-02-17T11:58:30.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.381+0000 7f4e5af72740 1 -- 172.21.15.94:0/1995589617 shutdown_connections 2024-02-17T11:58:30.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.381+0000 7f4e5af72740 1 --2- 172.21.15.94:0/1995589617 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cf8b0f4560 0x55cf8b14e1f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.381+0000 7f4e5af72740 1 --2- 172.21.15.94:0/1995589617 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cf8b0e4750 0x55cf8b0eac10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.381+0000 7f4e5af72740 1 --2- 172.21.15.94:0/1995589617 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cf8b0dbb70 0x55cf8b0ea6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.381+0000 7f4e5af72740 1 --2- 172.21.15.94:0/1995589617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cf8b0db250 0x55cf8b0ea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.381+0000 7f4e39ffb640 0 client.25073 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T11:58:30.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.381+0000 7f4e5af72740 1 -- 172.21.15.94:0/1995589617 >> 172.21.15.94:0/1995589617 conn(0x55cf8b0d5980 msgr2=0x55cf8b0fb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.381+0000 7f4e5af72740 1 -- 172.21.15.94:0/1995589617 shutdown_connections 2024-02-17T11:58:30.381 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T11:58:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.381+0000 7f4e5af72740 1 -- 172.21.15.94:0/1995589617 wait complete. 2024-02-17T11:58:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.382+0000 7f4e5af72740 20 client.25073 trim_cache size 0 max 16384 2024-02-17T11:58:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.383+0000 7f4e582c3640 1 -- 172.21.15.94:0/3233702393 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4e3c0a2330 msgr2=0x7f4e3c0a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.383+0000 7f4e582c3640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4e3c0a2330 0x7f4e3c0a47f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4e44005ab0 tx=0x7f4e44005a20 comp rx=0 tx=0).stop 2024-02-17T11:58:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.384+0000 7f4e582c3640 1 -- 172.21.15.94:0/3233702393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e501142d0 msgr2=0x7f4e5010d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.384+0000 7f4e582c3640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e501142d0 0x7f4e5010d7c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800e9d0 tx=0x7f4e4800ee90 comp rx=0 tx=0).stop 2024-02-17T11:58:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.384+0000 7f4e582c3640 1 -- 172.21.15.94:0/3233702393 shutdown_connections 2024-02-17T11:58:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.384+0000 7f4e582c3640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4e3c0a2330 0x7f4e3c0a47f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.384+0000 7f4e582c3640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e501142d0 0x7f4e5010d7c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.384+0000 7f4e582c3640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e50108c80 0x7f4e50113bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.384+0000 7f4e582c3640 1 --2- 172.21.15.94:0/3233702393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e50108360 0x7f4e50113690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.384+0000 7f4e582c3640 1 -- 172.21.15.94:0/3233702393 >> 172.21.15.94:0/3233702393 conn(0x7f4e5006c260 msgr2=0x7f4e5006fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.384+0000 7f4e582c3640 1 -- 172.21.15.94:0/3233702393 shutdown_connections 2024-02-17T11:58:30.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.384+0000 7f4e582c3640 1 -- 172.21.15.94:0/3233702393 wait complete. 2024-02-17T11:58:30.397 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T11:58:30.397 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T11:58:30.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.555+0000 7f12618c9640 1 -- 172.21.15.94:0/2986464891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f125c100260 msgr2=0x7f125c100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.555+0000 7f12618c9640 1 --2- 172.21.15.94:0/2986464891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f125c100260 0x7f125c100640 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f124c009930 tx=0x7f124c02bc30 comp rx=0 tx=0).stop 2024-02-17T11:58:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.555+0000 7f12618c9640 1 -- 172.21.15.94:0/2986464891 shutdown_connections 2024-02-17T11:58:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.555+0000 7f12618c9640 1 --2- 172.21.15.94:0/2986464891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125c105330 0x7f125c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.555+0000 7f12618c9640 1 --2- 172.21.15.94:0/2986464891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f125c100c10 0x7f125c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.555+0000 7f12618c9640 1 --2- 172.21.15.94:0/2986464891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f125c100260 0x7f125c100640 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.555+0000 7f12618c9640 1 -- 172.21.15.94:0/2986464891 >> 172.21.15.94:0/2986464891 conn(0x7f125c077180 msgr2=0x7f125c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.556+0000 7f12618c9640 1 -- 172.21.15.94:0/2986464891 shutdown_connections 2024-02-17T11:58:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.556+0000 7f12618c9640 1 -- 172.21.15.94:0/2986464891 wait complete. 2024-02-17T11:58:30.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.557+0000 7f12618c9640 1 Processor -- start 2024-02-17T11:58:30.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.557+0000 7f12618c9640 1 -- start start 2024-02-17T11:58:30.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.557+0000 7f12618c9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125c100260 0x7f125c108700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.557+0000 7f12618c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f125c105330 0x7f125c108c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.557+0000 7f12618c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f125c1a7b60 0x7f125c1a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.557+0000 7f12618c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c1093a0 con 0x7f125c105330 2024-02-17T11:58:30.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.557+0000 7f12618c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c1aa490 con 0x7f125c1a7b60 2024-02-17T11:58:30.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.557+0000 7f12618c9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f125c1aa650 con 0x7f125c100260 2024-02-17T11:58:30.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.558+0000 7f125affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125c100260 0x7f125c108700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.558+0000 7f125affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125c100260 0x7f125c108700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:58066/0 (socket says 172.21.15.94:58066) 2024-02-17T11:58:30.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.558+0000 7f125affd640 1 -- 172.21.15.94:0/1930528822 learned_addr learned my addr 172.21.15.94:0/1930528822 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.558+0000 7f125a7fc640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f125c105330 0x7f125c108c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.558+0000 7f125b7fe640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f125c1a7b60 0x7f125c1a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.558+0000 7f125affd640 1 -- 172.21.15.94:0/1930528822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f125c1a7b60 msgr2=0x7f125c1a9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.558+0000 7f125affd640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f125c1a7b60 0x7f125c1a9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.558+0000 7f125affd640 1 -- 172.21.15.94:0/1930528822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f125c105330 msgr2=0x7f125c108c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.558+0000 7f125affd640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f125c105330 0x7f125c108c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.558+0000 7f125affd640 1 -- 172.21.15.94:0/1930528822 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f124c009560 con 0x7f125c100260 2024-02-17T11:58:30.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.559+0000 7f125affd640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125c100260 0x7f125c108700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f124c000c00 tx=0x7f124c02cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.559+0000 7f12608c7640 1 -- 172.21.15.94:0/1930528822 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c004410 con 0x7f125c100260 2024-02-17T11:58:30.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.559+0000 7f12608c7640 1 -- 172.21.15.94:0/1930528822 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f124c0052d0 con 0x7f125c100260 2024-02-17T11:58:30.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.559+0000 7f12618c9640 1 -- 172.21.15.94:0/1930528822 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f125c1aa8d0 con 0x7f125c100260 2024-02-17T11:58:30.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.559+0000 7f12618c9640 1 -- 172.21.15.94:0/1930528822 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f125c1aae30 con 0x7f125c100260 2024-02-17T11:58:30.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.559+0000 7f12608c7640 1 -- 172.21.15.94:0/1930528822 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c003ca0 con 0x7f125c100260 2024-02-17T11:58:30.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.562+0000 7f12608c7640 1 -- 172.21.15.94:0/1930528822 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f124c03e450 con 0x7f125c100260 2024-02-17T11:58:30.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.564+0000 7f12608c7640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12440a2380 0x7f12440a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.564+0000 7f12608c7640 1 -- 172.21.15.94:0/1930528822 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f124c0f18d0 con 0x7f125c100260 2024-02-17T11:58:30.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f125a7fc640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12440a2380 0x7f12440a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f1264570740 1 Processor -- start 2024-02-17T11:58:30.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f1264570740 1 -- start start 2024-02-17T11:58:30.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f1264570740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e55250 0x561187e55630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f1264570740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55b70 0x561187e5e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f1264570740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e5e750 0x561187e60b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f1264570740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561187e10960 con 0x561187e55250 2024-02-17T11:58:30.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f1264570740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561187dc4310 con 0x561187e55b70 2024-02-17T11:58:30.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f1264570740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561187dc1840 con 0x561187e5e750 2024-02-17T11:58:30.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f125b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e5e750 0x561187e60b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f125b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e5e750 0x561187e60b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:58080/0 (socket says 172.21.15.94:58080) 2024-02-17T11:58:30.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f125b7fe640 1 -- 172.21.15.94:0/493163598 learned_addr learned my addr 172.21.15.94:0/493163598 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f125affd640 1 --2- 172.21.15.94:0/493163598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e55250 0x561187e55630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.565+0000 7f125a7fc640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12440a2380 0x7f12440a4840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f125c109ce0 tx=0x7f125000b410 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.566+0000 7f125a7fc640 1 --2- 172.21.15.94:0/493163598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55b70 0x561187e5e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.566+0000 7f125b7fe640 1 -- 172.21.15.94:0/493163598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55b70 msgr2=0x561187e5e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.566+0000 7f125b7fe640 1 --2- 172.21.15.94:0/493163598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55b70 0x561187e5e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.566+0000 7f125b7fe640 1 -- 172.21.15.94:0/493163598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e55250 msgr2=0x561187e55630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.566+0000 7f125b7fe640 1 --2- 172.21.15.94:0/493163598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e55250 0x561187e55630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.566+0000 7f125b7fe640 1 -- 172.21.15.94:0/493163598 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561187d7f060 con 0x561187e5e750 2024-02-17T11:58:30.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.566+0000 7f125a7fc640 1 --2- 172.21.15.94:0/493163598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55b70 0x561187e5e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:58:30.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.566+0000 7f125b7fe640 1 --2- 172.21.15.94:0/493163598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e5e750 0x561187e60b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1254009f40 tx=0x7f1254010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b41d9f0cb8f666b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.567+0000 7f1241ffb640 1 -- 172.21.15.94:0/493163598 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254009a90 con 0x561187e5e750 2024-02-17T11:58:30.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.567+0000 7f1241ffb640 1 -- 172.21.15.94:0/493163598 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1254015070 con 0x561187e5e750 2024-02-17T11:58:30.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.567+0000 7f1241ffb640 1 -- 172.21.15.94:0/493163598 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254011830 con 0x561187e5e750 2024-02-17T11:58:30.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.568+0000 7f1264570740 1 -- 172.21.15.94:0/493163598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e5e750 msgr2=0x561187e60b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.568+0000 7f1264570740 1 --2- 172.21.15.94:0/493163598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e5e750 0x561187e60b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1254009f40 tx=0x7f1254010860 comp rx=0 tx=0).stop 2024-02-17T11:58:30.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.568+0000 7f1264570740 1 -- 172.21.15.94:0/493163598 shutdown_connections 2024-02-17T11:58:30.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.568+0000 7f1264570740 1 --2- 172.21.15.94:0/493163598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e5e750 0x561187e60b40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.568+0000 7f1264570740 1 --2- 172.21.15.94:0/493163598 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55b70 0x561187e5e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.568+0000 7f1264570740 1 --2- 172.21.15.94:0/493163598 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e55250 0x561187e55630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.568+0000 7f1264570740 1 -- 172.21.15.94:0/493163598 >> 172.21.15.94:0/493163598 conn(0x561187e4f980 msgr2=0x561187e4fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.568+0000 7f1264570740 1 -- 172.21.15.94:0/493163598 shutdown_connections 2024-02-17T11:58:30.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.568+0000 7f1264570740 1 -- 172.21.15.94:0/493163598 wait complete. 2024-02-17T11:58:30.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.568+0000 7f1264570740 1 -- start start 2024-02-17T11:58:30.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.569+0000 7f1264570740 1 Processor -- start 2024-02-17T11:58:30.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.569+0000 7f1264570740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55250 0x561187e5b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.569+0000 7f1264570740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55b70 0x561187e5bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.569+0000 7f1264570740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e5e750 0x561187e5c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.569+0000 7f1264570740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561187dbd020 con 0x561187e5e750 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.569+0000 7f1264570740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561187a90430 con 0x561187e55250 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.569+0000 7f1264570740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561187a4ce00 con 0x561187e55b70 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.569+0000 7f1264570740 10 client.0 _ll_register_callbacks cb 0x561187d64cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.570+0000 7f125affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55250 0x561187e5b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.570+0000 7f125affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55250 0x561187e5b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45614/0 (socket says 172.21.15.94:45614) 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.570+0000 7f125affd640 1 -- 172.21.15.94:0/2222365116 learned_addr learned my addr 172.21.15.94:0/2222365116 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.570+0000 7f125a7fc640 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55b70 0x561187e5bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.570+0000 7f125b7fe640 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e5e750 0x561187e5c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.570+0000 7f125affd640 1 -- 172.21.15.94:0/2222365116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55b70 msgr2=0x561187e5bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.570+0000 7f125affd640 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55b70 0x561187e5bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.570+0000 7f125affd640 1 -- 172.21.15.94:0/2222365116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e5e750 msgr2=0x561187e5c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.570+0000 7f125affd640 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e5e750 0x561187e5c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.570+0000 7f125affd640 1 -- 172.21.15.94:0/2222365116 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1254009bf0 con 0x561187e55250 2024-02-17T11:58:30.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.570+0000 7f125a7fc640 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55b70 0x561187e5bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:58:30.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.571+0000 7f125affd640 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55250 0x561187e5b780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f124c0bac20 tx=0x7f124c0be950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3aaeb6645dff28e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.571+0000 7f123affd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:58:30.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.571+0000 7f123affd640 1 -- 172.21.15.94:0/2222365116 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f124c0c69f0 con 0x561187e55250 2024-02-17T11:58:30.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.571+0000 7f123affd640 1 -- 172.21.15.94:0/2222365116 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f124c005440 con 0x561187e55250 2024-02-17T11:58:30.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.571+0000 7f1264570740 1 -- 172.21.15.94:0/2222365116 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561187abc7e0 con 0x561187e55250 2024-02-17T11:58:30.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.571+0000 7f123affd640 1 -- 172.21.15.94:0/2222365116 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f124c005600 con 0x561187e55250 2024-02-17T11:58:30.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.571+0000 7f1264570740 10 client.25085 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:30.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.571+0000 7f1264570740 1 -- 172.21.15.94:0/2222365116 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x561187dbb6a0 con 0x561187e55250 2024-02-17T11:58:30.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.572+0000 7f123affd640 1 -- 172.21.15.94:0/2222365116 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561187abc7e0 con 0x561187e55250 2024-02-17T11:58:30.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.572+0000 7f123affd640 1 -- 172.21.15.94:0/2222365116 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f124c03c070 con 0x561187e55250 2024-02-17T11:58:30.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.572+0000 7f1264570740 10 client.25085 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:30.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.572+0000 7f1264570740 10 client.25085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:30.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.572+0000 7f1264570740 20 client.25085 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:30.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.572+0000 7f1264570740 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561187e7fb20 0x561187e81f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.572+0000 7f1264570740 4 client.25085 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:58:30.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.572+0000 7f1264570740 1 -- 172.21.15.94:0/2222365116 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561187af9720 con 0x561187e7fb20 2024-02-17T11:58:30.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.572+0000 7f125a7fc640 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561187e7fb20 0x561187e81f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.572+0000 7f123affd640 10 client.25085 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:30.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.572+0000 7f125a7fc640 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561187e7fb20 0x561187e81f40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a277d07e465eada server_cookie=992540b454e37a4f in_seq=0 out_seq=0 2024-02-17T11:58:30.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f123affd640 1 -- 172.21.15.94:0/2222365116 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561187af9720 con 0x561187e7fb20 2024-02-17T11:58:30.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f123affd640 10 client.25085 handle_command_reply: tid=0 2024-02-17T11:58:30.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f1264570740 1 client.25085 shutdown 2024-02-17T11:58:30.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f1264570740 1 -- 172.21.15.94:0/2222365116 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55250 msgr2=0x561187e5b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f1264570740 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55250 0x561187e5b780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f124c0bac20 tx=0x7f124c0be950 comp rx=0 tx=0).stop 2024-02-17T11:58:30.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f1264570740 1 -- 172.21.15.94:0/2222365116 shutdown_connections 2024-02-17T11:58:30.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f1264570740 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561187e7fb20 0x561187e81f40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f1264570740 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e5e750 0x561187e5c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f1264570740 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55b70 0x561187e5bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f1264570740 1 --2- 172.21.15.94:0/2222365116 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55250 0x561187e5b780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f1264570740 1 -- 172.21.15.94:0/2222365116 >> 172.21.15.94:0/2222365116 conn(0x561187e4f980 msgr2=0x561187e516d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f123affd640 0 client.25085 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:30.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f1264570740 1 -- 172.21.15.94:0/2222365116 shutdown_connections 2024-02-17T11:58:30.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f1264570740 1 -- 172.21.15.94:0/2222365116 wait complete. 2024-02-17T11:58:30.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.574+0000 7f1264570740 20 client.25085 trim_cache size 0 max 16384 2024-02-17T11:58:30.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.596+0000 7f1264570740 1 Processor -- start 2024-02-17T11:58:30.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.596+0000 7f1264570740 1 -- start start 2024-02-17T11:58:30.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f1264570740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e55250 0x561187e5deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f1264570740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55b70 0x561187e5bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f1264570740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e5e750 0x561187e5c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f1264570740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561187d42130 con 0x561187e55250 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f1264570740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561187e509b0 con 0x561187e5e750 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f1264570740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561187e54eb0 con 0x561187e55b70 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f125a7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55b70 0x561187e5bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f125a7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55b70 0x561187e5bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:58094/0 (socket says 172.21.15.94:58094) 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f125a7fc640 1 -- 172.21.15.94:0/1962271062 learned_addr learned my addr 172.21.15.94:0/1962271062 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f125affd640 1 --2- 172.21.15.94:0/1962271062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e55250 0x561187e5deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f125a7fc640 1 -- 172.21.15.94:0/1962271062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e5e750 msgr2=0x561187e5c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f125a7fc640 1 --2- 172.21.15.94:0/1962271062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e5e750 0x561187e5c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f125a7fc640 1 -- 172.21.15.94:0/1962271062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e55250 msgr2=0x561187e5deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f125a7fc640 1 --2- 172.21.15.94:0/1962271062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e55250 0x561187e5deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.597+0000 7f125a7fc640 1 -- 172.21.15.94:0/1962271062 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561187d57c20 con 0x561187e55b70 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.598+0000 7f125a7fc640 1 --2- 172.21.15.94:0/1962271062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55b70 0x561187e5bcd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1250005fd0 tx=0x7f1250016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b23e55daebf6a654 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.598+0000 7f12417fa640 1 -- 172.21.15.94:0/1962271062 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1250024040 con 0x561187e55b70 2024-02-17T11:58:30.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.598+0000 7f12417fa640 1 -- 172.21.15.94:0/1962271062 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1250015070 con 0x561187e55b70 2024-02-17T11:58:30.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.598+0000 7f12417fa640 1 -- 172.21.15.94:0/1962271062 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125001e7c0 con 0x561187e55b70 2024-02-17T11:58:30.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.599+0000 7f1264570740 1 -- 172.21.15.94:0/1962271062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55b70 msgr2=0x561187e5bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.599+0000 7f1264570740 1 --2- 172.21.15.94:0/1962271062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55b70 0x561187e5bcd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1250005fd0 tx=0x7f1250016000 comp rx=0 tx=0).stop 2024-02-17T11:58:30.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.599+0000 7f1264570740 1 -- 172.21.15.94:0/1962271062 shutdown_connections 2024-02-17T11:58:30.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.599+0000 7f1264570740 1 --2- 172.21.15.94:0/1962271062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e5e750 0x561187e5c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.599+0000 7f1264570740 1 --2- 172.21.15.94:0/1962271062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55b70 0x561187e5bcd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.599+0000 7f1264570740 1 --2- 172.21.15.94:0/1962271062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e55250 0x561187e5deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.599+0000 7f1264570740 1 -- 172.21.15.94:0/1962271062 >> 172.21.15.94:0/1962271062 conn(0x561187e4f980 msgr2=0x561187e86860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.599+0000 7f1264570740 1 -- 172.21.15.94:0/1962271062 shutdown_connections 2024-02-17T11:58:30.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.599+0000 7f1264570740 1 -- 172.21.15.94:0/1962271062 wait complete. 2024-02-17T11:58:30.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.600+0000 7f1264570740 1 -- start start 2024-02-17T11:58:30.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.600+0000 7f1264570740 1 Processor -- start 2024-02-17T11:58:30.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.601+0000 7f1264570740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55250 0x561187e75850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.601+0000 7f1264570740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55b70 0x561187e75d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.601+0000 7f1264570740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e5e750 0x561187ed5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.601+0000 7f1264570740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561187ed6530 con 0x561187e5e750 2024-02-17T11:58:30.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.601+0000 7f1264570740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561187ed6670 con 0x561187e55b70 2024-02-17T11:58:30.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.601+0000 7f1264570740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561187ed67b0 con 0x561187e55250 2024-02-17T11:58:30.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.601+0000 7f1264570740 10 client.0 _ll_register_callbacks cb 0x7f12540099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:30.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.601+0000 7f125b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e5e750 0x561187ed5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.601+0000 7f125b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e5e750 0x561187ed5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37470/0 (socket says 172.21.15.94:37470) 2024-02-17T11:58:30.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.601+0000 7f125b7fe640 1 -- 172.21.15.94:0/1604734578 learned_addr learned my addr 172.21.15.94:0/1604734578 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:30.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.601+0000 7f125affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55250 0x561187e75850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.601+0000 7f125a7fc640 1 --2- 172.21.15.94:0/1604734578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55b70 0x561187e75d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.602+0000 7f125b7fe640 1 -- 172.21.15.94:0/1604734578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55250 msgr2=0x561187e75850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.602+0000 7f125b7fe640 1 --2- 172.21.15.94:0/1604734578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55250 0x561187e75850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.602+0000 7f125b7fe640 1 -- 172.21.15.94:0/1604734578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55b70 msgr2=0x561187e75d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:30.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.602+0000 7f125b7fe640 1 --2- 172.21.15.94:0/1604734578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55b70 0x561187e75d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.602+0000 7f125b7fe640 1 -- 172.21.15.94:0/1604734578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1250011660 con 0x561187e5e750 2024-02-17T11:58:30.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.602+0000 7f125a7fc640 1 --2- 172.21.15.94:0/1604734578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55b70 0x561187e75d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:30.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.602+0000 7f125b7fe640 1 --2- 172.21.15.94:0/1604734578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e5e750 0x561187ed5ff0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1254009ee0 tx=0x7f1254004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa769dc499093cff server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:30.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.602+0000 7f123affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:58:30.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.603+0000 7f123affd640 1 -- 172.21.15.94:0/1604734578 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1254005450 con 0x561187e5e750 2024-02-17T11:58:30.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.603+0000 7f123affd640 1 -- 172.21.15.94:0/1604734578 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1254017070 con 0x561187e5e750 2024-02-17T11:58:30.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.603+0000 7f1264570740 1 -- 172.21.15.94:0/1604734578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1254015070 con 0x561187e5e750 2024-02-17T11:58:30.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.603+0000 7f123affd640 1 -- 172.21.15.94:0/1604734578 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12540055b0 con 0x561187e5e750 2024-02-17T11:58:30.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.603+0000 7f1264570740 10 client.15600 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:30.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.603+0000 7f1264570740 1 -- 172.21.15.94:0/1604734578 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x561187d7d8c0 con 0x561187e5e750 2024-02-17T11:58:30.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.603+0000 7f123affd640 1 -- 172.21.15.94:0/1604734578 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1254033080 con 0x561187e5e750 2024-02-17T11:58:30.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.603+0000 7f123affd640 1 -- 172.21.15.94:0/1604734578 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1254011440 con 0x561187e5e750 2024-02-17T11:58:30.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.603+0000 7f1264570740 10 client.15600 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:30.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.603+0000 7f1264570740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:30.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.603+0000 7f1264570740 20 client.15600 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:30.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.604+0000 7f1264570740 1 --2- 172.21.15.94:0/1604734578 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561187ed7e60 0x561187eda280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:30.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.604+0000 7f1264570740 4 client.15600 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T11:58:30.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.604+0000 7f1264570740 1 -- 172.21.15.94:0/1604734578 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561187b48890 con 0x561187ed7e60 2024-02-17T11:58:30.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.604+0000 7f125affd640 1 --2- 172.21.15.94:0/1604734578 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561187ed7e60 0x561187eda280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:30.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.604+0000 7f125affd640 1 --2- 172.21.15.94:0/1604734578 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561187ed7e60 0x561187eda280 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37bfdbd1caf2bd30 server_cookie=f4f2e9271b8bf34 in_seq=0 out_seq=0 2024-02-17T11:58:30.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.604+0000 7f123affd640 10 client.15600 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:30.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.605+0000 7f123affd640 1 -- 172.21.15.94:0/1604734578 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561187b48890 con 0x561187ed7e60 2024-02-17T11:58:30.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.605+0000 7f123affd640 10 client.15600 handle_command_reply: tid=0 2024-02-17T11:58:30.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.605+0000 7f1264570740 1 client.15600 shutdown 2024-02-17T11:58:30.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.606+0000 7f1264570740 1 -- 172.21.15.94:0/1604734578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e5e750 msgr2=0x561187ed5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.606+0000 7f1264570740 1 --2- 172.21.15.94:0/1604734578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e5e750 0x561187ed5ff0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1254009ee0 tx=0x7f1254004f90 comp rx=0 tx=0).stop 2024-02-17T11:58:30.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.606+0000 7f1264570740 1 -- 172.21.15.94:0/1604734578 shutdown_connections 2024-02-17T11:58:30.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.606+0000 7f1264570740 1 --2- 172.21.15.94:0/1604734578 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561187ed7e60 0x561187eda280 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.606+0000 7f1264570740 1 --2- 172.21.15.94:0/1604734578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561187e5e750 0x561187ed5ff0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.606+0000 7f1264570740 1 --2- 172.21.15.94:0/1604734578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561187e55b70 0x561187e75d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.606+0000 7f1264570740 1 --2- 172.21.15.94:0/1604734578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561187e55250 0x561187e75850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.606+0000 7f1264570740 1 -- 172.21.15.94:0/1604734578 >> 172.21.15.94:0/1604734578 conn(0x561187e4f980 msgr2=0x561187e4e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.606+0000 7f123affd640 0 client.15600 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:30.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.606+0000 7f1264570740 1 -- 172.21.15.94:0/1604734578 shutdown_connections 2024-02-17T11:58:30.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.606+0000 7f1264570740 1 -- 172.21.15.94:0/1604734578 wait complete. 2024-02-17T11:58:30.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.606+0000 7f1264570740 20 client.15600 trim_cache size 0 max 16384 2024-02-17T11:58:30.614 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T11:58:30.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.607+0000 7f12618c9640 1 -- 172.21.15.94:0/1930528822 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12440a2380 msgr2=0x7f12440a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.607+0000 7f12618c9640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12440a2380 0x7f12440a4840 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f125c109ce0 tx=0x7f125000b410 comp rx=0 tx=0).stop 2024-02-17T11:58:30.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.607+0000 7f12618c9640 1 -- 172.21.15.94:0/1930528822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125c100260 msgr2=0x7f125c108700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:30.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.607+0000 7f12618c9640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125c100260 0x7f125c108700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f124c000c00 tx=0x7f124c02cd80 comp rx=0 tx=0).stop 2024-02-17T11:58:30.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.607+0000 7f12618c9640 1 -- 172.21.15.94:0/1930528822 shutdown_connections 2024-02-17T11:58:30.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.607+0000 7f12618c9640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12440a2380 0x7f12440a4840 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.607+0000 7f12618c9640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f125c1a7b60 0x7f125c1a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.607+0000 7f12618c9640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f125c105330 0x7f125c108c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.607+0000 7f12618c9640 1 --2- 172.21.15.94:0/1930528822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125c100260 0x7f125c108700 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:30.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.607+0000 7f12618c9640 1 -- 172.21.15.94:0/1930528822 >> 172.21.15.94:0/1930528822 conn(0x7f125c077180 msgr2=0x7f125c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:30.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.607+0000 7f12618c9640 1 -- 172.21.15.94:0/1930528822 shutdown_connections 2024-02-17T11:58:30.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:30.607+0000 7f12618c9640 1 -- 172.21.15.94:0/1930528822 wait complete. 2024-02-17T11:58:31.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:30 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3368801167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:58:31.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:30 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3368801167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:58:31.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:30 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3368801167' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:58:31.625 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T11:58:31.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:58:31.625 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T11:58:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.781+0000 7f3bc7bb6640 1 -- 172.21.15.94:0/1584279215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bc0101380 msgr2=0x7f3bc0101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.781+0000 7f3bc7bb6640 1 --2- 172.21.15.94:0/1584279215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bc0101380 0x7f3bc0101760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009930 tx=0x7f3bbc02bc50 comp rx=0 tx=0).stop 2024-02-17T11:58:31.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.781+0000 7f3bc7bb6640 1 -- 172.21.15.94:0/1584279215 shutdown_connections 2024-02-17T11:58:31.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.781+0000 7f3bc7bb6640 1 --2- 172.21.15.94:0/1584279215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bc0106450 0x7f3bc010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.781+0000 7f3bc7bb6640 1 --2- 172.21.15.94:0/1584279215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc0101d30 0x7f3bc0105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.782+0000 7f3bc7bb6640 1 --2- 172.21.15.94:0/1584279215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bc0101380 0x7f3bc0101760 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.782+0000 7f3bc7bb6640 1 -- 172.21.15.94:0/1584279215 >> 172.21.15.94:0/1584279215 conn(0x7f3bc0078580 msgr2=0x7f3bc0078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:31.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.782+0000 7f3bc7bb6640 1 -- 172.21.15.94:0/1584279215 shutdown_connections 2024-02-17T11:58:31.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.783+0000 7f3bc7bb6640 1 -- 172.21.15.94:0/1584279215 wait complete. 2024-02-17T11:58:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.784+0000 7f3bc7bb6640 1 Processor -- start 2024-02-17T11:58:31.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.784+0000 7f3bc7bb6640 1 -- start start 2024-02-17T11:58:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.784+0000 7f3bc7bb6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc0101380 0x7f3bc019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.784+0000 7f3bc7bb6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bc0106450 0x7f3bc019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.784+0000 7f3bc7bb6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bc0197620 0x7f3bc0197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.784+0000 7f3bc7bb6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc0198010 con 0x7f3bc0101380 2024-02-17T11:58:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.784+0000 7f3bc7bb6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc0198180 con 0x7f3bc0197620 2024-02-17T11:58:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.784+0000 7f3bc7bb6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc0198340 con 0x7f3bc0106450 2024-02-17T11:58:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.784+0000 7f3bc6153640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bc0197620 0x7f3bc0197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.784+0000 7f3bc6153640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bc0197620 0x7f3bc0197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45644/0 (socket says 172.21.15.94:45644) 2024-02-17T11:58:31.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.784+0000 7f3bc5952640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc0101380 0x7f3bc019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.784+0000 7f3bc6153640 1 -- 172.21.15.94:0/3153508149 learned_addr learned my addr 172.21.15.94:0/3153508149 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:31.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.784+0000 7f3bc5151640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bc0106450 0x7f3bc019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.785+0000 7f3bc6153640 1 -- 172.21.15.94:0/3153508149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bc0106450 msgr2=0x7f3bc019d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.785+0000 7f3bc6153640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bc0106450 0x7f3bc019d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.785+0000 7f3bc6153640 1 -- 172.21.15.94:0/3153508149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc0101380 msgr2=0x7f3bc019d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.785+0000 7f3bc6153640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc0101380 0x7f3bc019d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.785+0000 7f3bc6153640 1 -- 172.21.15.94:0/3153508149 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc009560 con 0x7f3bc0197620 2024-02-17T11:58:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.785+0000 7f3bc5952640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc0101380 0x7f3bc019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:58:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.785+0000 7f3bc6153640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bc0197620 0x7f3bc0197ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009c40 tx=0x7f3bb000b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:31.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.785+0000 7f3bb6ffd640 1 -- 172.21.15.94:0/3153508149 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0015070 con 0x7f3bc0197620 2024-02-17T11:58:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.785+0000 7f3bb6ffd640 1 -- 172.21.15.94:0/3153508149 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bb0009e50 con 0x7f3bc0197620 2024-02-17T11:58:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.785+0000 7f3bc7bb6640 1 -- 172.21.15.94:0/3153508149 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc00733b0 con 0x7f3bc0197620 2024-02-17T11:58:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.785+0000 7f3bc7bb6640 1 -- 172.21.15.94:0/3153508149 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc00738c0 con 0x7f3bc0197620 2024-02-17T11:58:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.785+0000 7f3bb6ffd640 1 -- 172.21.15.94:0/3153508149 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0019410 con 0x7f3bc0197620 2024-02-17T11:58:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.789+0000 7f3bb6ffd640 1 -- 172.21.15.94:0/3153508149 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f3bb0019570 con 0x7f3bc0197620 2024-02-17T11:58:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.789+0000 7f3bca787740 1 Processor -- start 2024-02-17T11:58:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.789+0000 7f3bca787740 1 -- start start 2024-02-17T11:58:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.789+0000 7f3bca787740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254380 0x56234b254760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.789+0000 7f3bca787740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b254d30 0x56234b25d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.789+0000 7f3bca787740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b25d8b0 0x56234b25fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.789+0000 7f3bca787740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56234b1c2f90 con 0x56234b254380 2024-02-17T11:58:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.789+0000 7f3bca787740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56234b1c04c0 con 0x56234b25d8b0 2024-02-17T11:58:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.789+0000 7f3bca787740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56234b1bbca0 con 0x56234b254d30 2024-02-17T11:58:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.789+0000 7f3bb6ffd640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3bac0a2380 0x7f3bac0a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.789+0000 7f3bc5151640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254380 0x56234b254760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.789+0000 7f3bc5151640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254380 0x56234b254760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37502/0 (socket says 172.21.15.94:37502) 2024-02-17T11:58:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.789+0000 7f3bc5151640 1 -- 172.21.15.94:0/2300225706 learned_addr learned my addr 172.21.15.94:0/2300225706 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3bc5952640 1 --2- 172.21.15.94:0/2300225706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b254d30 0x56234b25d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3bc6153640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3bac0a2380 0x7f3bac0a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3bc5151640 1 --2- 172.21.15.94:0/2300225706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b25d8b0 0x56234b25fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3bb6ffd640 1 -- 172.21.15.94:0/3153508149 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3bb00d19f0 con 0x7f3bc0197620 2024-02-17T11:58:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3bc5952640 1 -- 172.21.15.94:0/2300225706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b25d8b0 msgr2=0x56234b25fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3bc5952640 1 --2- 172.21.15.94:0/2300225706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b25d8b0 0x56234b25fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3bc5952640 1 -- 172.21.15.94:0/2300225706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254380 msgr2=0x56234b254760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3bc5952640 1 --2- 172.21.15.94:0/2300225706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254380 0x56234b254760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3bc5952640 1 -- 172.21.15.94:0/2300225706 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56234b17e0b0 con 0x56234b254d30 2024-02-17T11:58:31.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3bc6153640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3bac0a2380 0x7f3bac0a4840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00cf0d0 tx=0x7f3bb0070220 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3bc5952640 1 --2- 172.21.15.94:0/2300225706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b254d30 0x56234b25d2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009850 tx=0x7f3bbc02c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9b14f9e09023d0c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3babfff640 1 -- 172.21.15.94:0/2300225706 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc004940 con 0x56234b254d30 2024-02-17T11:58:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3babfff640 1 -- 172.21.15.94:0/2300225706 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bbc004aa0 con 0x56234b254d30 2024-02-17T11:58:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.790+0000 7f3babfff640 1 -- 172.21.15.94:0/2300225706 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc0051f0 con 0x56234b254d30 2024-02-17T11:58:31.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.791+0000 7f3bca787740 1 -- 172.21.15.94:0/2300225706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b254d30 msgr2=0x56234b25d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:31.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.791+0000 7f3bca787740 1 --2- 172.21.15.94:0/2300225706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b254d30 0x56234b25d2e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009850 tx=0x7f3bbc02c680 comp rx=0 tx=0).stop 2024-02-17T11:58:31.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.791+0000 7f3bca787740 1 -- 172.21.15.94:0/2300225706 shutdown_connections 2024-02-17T11:58:31.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.791+0000 7f3bca787740 1 --2- 172.21.15.94:0/2300225706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b25d8b0 0x56234b25fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.791+0000 7f3bca787740 1 --2- 172.21.15.94:0/2300225706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b254d30 0x56234b25d2e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.791+0000 7f3bca787740 1 --2- 172.21.15.94:0/2300225706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254380 0x56234b254760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.792+0000 7f3bca787740 1 -- 172.21.15.94:0/2300225706 >> 172.21.15.94:0/2300225706 conn(0x56234b24e840 msgr2=0x56234b24ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:31.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.792+0000 7f3bca787740 1 -- 172.21.15.94:0/2300225706 shutdown_connections 2024-02-17T11:58:31.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.792+0000 7f3bca787740 1 -- 172.21.15.94:0/2300225706 wait complete. 2024-02-17T11:58:31.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.792+0000 7f3bca787740 1 -- start start 2024-02-17T11:58:31.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.792+0000 7f3bca787740 1 Processor -- start 2024-02-17T11:58:31.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bca787740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254380 0x56234b25a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bca787740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254d30 0x56234b25ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bca787740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 0x56234b25b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bca787740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56234b160f40 con 0x56234b254380 2024-02-17T11:58:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bca787740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56234ae8f4b0 con 0x56234b254d30 2024-02-17T11:58:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bca787740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56234b141670 con 0x56234b25d8b0 2024-02-17T11:58:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bc5151640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254380 0x56234b25a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bc5151640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254380 0x56234b25a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37504/0 (socket says 172.21.15.94:37504) 2024-02-17T11:58:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bc5151640 1 -- 172.21.15.94:0/2940652326 learned_addr learned my addr 172.21.15.94:0/2940652326 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bc5952640 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254d30 0x56234b25ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bc5151640 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 0x56234b25b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bca787740 10 client.0 _ll_register_callbacks cb 0x56234b1aef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bc5151640 1 -- 172.21.15.94:0/2940652326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 msgr2=0x56234b25b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bc5151640 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 0x56234b25b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bc5151640 1 -- 172.21.15.94:0/2940652326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254d30 msgr2=0x56234b25ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bc5151640 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254d30 0x56234b25ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bc5151640 1 -- 172.21.15.94:0/2940652326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc010e830 con 0x56234b254380 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bc5952640 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254d30 0x56234b25ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.793+0000 7f3bc5151640 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254380 0x56234b25a740 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8015e90 tx=0x7f3bb8009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca182456c5ba4cc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3ba97fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3ba97fa640 1 -- 172.21.15.94:0/2940652326 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8015080 con 0x56234b254380 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3bca787740 1 -- 172.21.15.94:0/2940652326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56234b195f40 con 0x56234b254380 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3ba97fa640 1 -- 172.21.15.94:0/2940652326 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3bb80151e0 con 0x56234b254380 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3ba97fa640 1 -- 172.21.15.94:0/2940652326 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bb80255b0 con 0x56234b254380 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3bca787740 10 client.15612 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3bca787740 1 -- 172.21.15.94:0/2940652326 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x56234b1ba320 con 0x56234b254380 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3ba97fa640 1 -- 172.21.15.94:0/2940652326 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56234b195f40 con 0x56234b254380 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3ba97fa640 1 -- 172.21.15.94:0/2940652326 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3bb8012450 con 0x56234b254380 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3bca787740 10 client.15612 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3bca787740 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3bca787740 20 client.15612 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:31.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3bca787740 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56234b26dd50 0x56234b270190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3bca787740 4 client.15612 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:58:31.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3bca787740 1 -- 172.21.15.94:0/2940652326 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56234b1b2310 con 0x56234b26dd50 2024-02-17T11:58:31.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.794+0000 7f3bc5952640 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56234b26dd50 0x56234b270190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.795+0000 7f3bc5952640 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56234b26dd50 0x56234b270190 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae529bcba99ceb53 server_cookie=723c3660d53d1876 in_seq=0 out_seq=0 2024-02-17T11:58:31.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.795+0000 7f3ba97fa640 10 client.15612 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:31.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.796+0000 7f3ba97fa640 1 -- 172.21.15.94:0/2940652326 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56234b1b2310 con 0x56234b26dd50 2024-02-17T11:58:31.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.796+0000 7f3ba97fa640 10 client.15612 handle_command_reply: tid=0 2024-02-17T11:58:31.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.796+0000 7f3bca787740 1 client.15612 shutdown 2024-02-17T11:58:31.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.796+0000 7f3bca787740 1 -- 172.21.15.94:0/2940652326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254380 msgr2=0x56234b25a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:31.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.796+0000 7f3bca787740 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254380 0x56234b25a740 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8015e90 tx=0x7f3bb8009aa0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.797+0000 7f3bca787740 1 -- 172.21.15.94:0/2940652326 shutdown_connections 2024-02-17T11:58:31.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.797+0000 7f3bca787740 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56234b26dd50 0x56234b270190 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.797+0000 7f3bca787740 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 0x56234b25b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.797+0000 7f3bca787740 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254d30 0x56234b25ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.797+0000 7f3bca787740 1 --2- 172.21.15.94:0/2940652326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254380 0x56234b25a740 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.797+0000 7f3ba97fa640 0 client.15612 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:31.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.797+0000 7f3bca787740 1 -- 172.21.15.94:0/2940652326 >> 172.21.15.94:0/2940652326 conn(0x56234b24e840 msgr2=0x56234b2507a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:31.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.797+0000 7f3bca787740 1 -- 172.21.15.94:0/2940652326 shutdown_connections 2024-02-17T11:58:31.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.797+0000 7f3bca787740 1 -- 172.21.15.94:0/2940652326 wait complete. 2024-02-17T11:58:31.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.797+0000 7f3bca787740 20 client.15612 trim_cache size 0 max 16384 2024-02-17T11:58:31.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.816+0000 7f3bca787740 1 Processor -- start 2024-02-17T11:58:31.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bca787740 1 -- start start 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bca787740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254380 0x56234b25ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bca787740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254d30 0x56234b259720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bca787740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 0x56234b2579a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bca787740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56234b195f70 con 0x56234b254d30 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bca787740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56234b253d00 con 0x56234b254380 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bca787740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56234b24f410 con 0x56234b25d8b0 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bc5952640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254d30 0x56234b259720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bc5151640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 0x56234b2579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bc5952640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254d30 0x56234b259720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37514/0 (socket says 172.21.15.94:37514) 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bc5952640 1 -- 172.21.15.94:0/1443816348 learned_addr learned my addr 172.21.15.94:0/1443816348 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bc5151640 1 --2- 172.21.15.94:0/1443816348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254380 0x56234b25ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bc5952640 1 -- 172.21.15.94:0/1443816348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 msgr2=0x56234b2579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bc5952640 1 --2- 172.21.15.94:0/1443816348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 0x56234b2579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bc5952640 1 -- 172.21.15.94:0/1443816348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254380 msgr2=0x56234b25ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:31.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bc5952640 1 --2- 172.21.15.94:0/1443816348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254380 0x56234b25ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bc5952640 1 -- 172.21.15.94:0/1443816348 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56234b156c00 con 0x56234b254d30 2024-02-17T11:58:31.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.817+0000 7f3bc5151640 1 --2- 172.21.15.94:0/1443816348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 0x56234b2579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:31.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.818+0000 7f3bc5952640 1 --2- 172.21.15.94:0/1443816348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254d30 0x56234b259720 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009930 tx=0x7f3bbc02bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8840d5018fbfae3d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:31.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.818+0000 7f3bab7fe640 1 -- 172.21.15.94:0/1443816348 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc04e040 con 0x56234b254d30 2024-02-17T11:58:31.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.818+0000 7f3bab7fe640 1 -- 172.21.15.94:0/1443816348 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3bbc050040 con 0x56234b254d30 2024-02-17T11:58:31.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.818+0000 7f3bab7fe640 1 -- 172.21.15.94:0/1443816348 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc005210 con 0x56234b254d30 2024-02-17T11:58:31.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.819+0000 7f3bca787740 1 -- 172.21.15.94:0/1443816348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254d30 msgr2=0x56234b259720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.819+0000 7f3bca787740 1 --2- 172.21.15.94:0/1443816348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254d30 0x56234b259720 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009930 tx=0x7f3bbc02bc50 comp rx=0 tx=0).stop 2024-02-17T11:58:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.819+0000 7f3bca787740 1 -- 172.21.15.94:0/1443816348 shutdown_connections 2024-02-17T11:58:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.819+0000 7f3bca787740 1 --2- 172.21.15.94:0/1443816348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 0x56234b2579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.819+0000 7f3bca787740 1 --2- 172.21.15.94:0/1443816348 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254d30 0x56234b259720 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.819+0000 7f3bca787740 1 --2- 172.21.15.94:0/1443816348 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254380 0x56234b25ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.819+0000 7f3bca787740 1 -- 172.21.15.94:0/1443816348 >> 172.21.15.94:0/1443816348 conn(0x56234b24e840 msgr2=0x56234b259010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.820+0000 7f3bca787740 1 -- 172.21.15.94:0/1443816348 shutdown_connections 2024-02-17T11:58:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.820+0000 7f3bca787740 1 -- 172.21.15.94:0/1443816348 wait complete. 2024-02-17T11:58:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.820+0000 7f3bca787740 1 -- start start 2024-02-17T11:58:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.820+0000 7f3bca787740 1 Processor -- start 2024-02-17T11:58:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bca787740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254380 0x56234b264cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bca787740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254d30 0x56234b2630b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bca787740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 0x56234b263670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bca787740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56234b263dc0 con 0x56234b254d30 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bca787740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56234b263f30 con 0x56234b254380 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bc5151640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254380 0x56234b264cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bc5952640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254d30 0x56234b2630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bc5151640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254380 0x56234b264cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45690/0 (socket says 172.21.15.94:45690) 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bc5151640 1 -- 172.21.15.94:0/2564197536 learned_addr learned my addr 172.21.15.94:0/2564197536 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bca787740 1 -- 172.21.15.94:0/2564197536 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56234b2ccac0 con 0x56234b25d8b0 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bca787740 10 client.0 _ll_register_callbacks cb 0x7f3bb0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bc5151640 1 -- 172.21.15.94:0/2564197536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 msgr2=0x56234b263670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bc5151640 1 --2- 172.21.15.94:0/2564197536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 0x56234b263670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bc5151640 1 -- 172.21.15.94:0/2564197536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254d30 msgr2=0x56234b2630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bc5151640 1 --2- 172.21.15.94:0/2564197536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254d30 0x56234b2630b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bc5151640 1 -- 172.21.15.94:0/2564197536 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56234b1ba320 con 0x56234b254380 2024-02-17T11:58:31.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3bc5151640 1 --2- 172.21.15.94:0/2564197536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254380 0x56234b264cc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8004680 tx=0x7f3bb8005ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e198153209fe38 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3ba97fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.821+0000 7f3ba97fa640 1 -- 172.21.15.94:0/2564197536 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb80157e0 con 0x56234b254380 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.822+0000 7f3ba97fa640 1 -- 172.21.15.94:0/2564197536 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3bb8012d30 con 0x56234b254380 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.822+0000 7f3bca787740 1 -- 172.21.15.94:0/2564197536 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56234b1e1900 con 0x56234b254380 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.822+0000 7f3ba97fa640 1 -- 172.21.15.94:0/2564197536 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3bb8003cb0 con 0x56234b254380 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.822+0000 7f3bca787740 10 client.25106 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.822+0000 7f3bca787740 1 -- 172.21.15.94:0/2564197536 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x56234b17e0b0 con 0x56234b254380 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.822+0000 7f3ba97fa640 1 -- 172.21.15.94:0/2564197536 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56234b1e1900 con 0x56234b254380 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.823+0000 7f3ba97fa640 1 -- 172.21.15.94:0/2564197536 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3bb8023bf0 con 0x56234b254380 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.823+0000 7f3bca787740 10 client.25106 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.823+0000 7f3bca787740 10 client.25106 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.823+0000 7f3bca787740 20 client.25106 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.823+0000 7f3bca787740 1 --2- 172.21.15.94:0/2564197536 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56234b2ce1b0 0x56234b2d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:31.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.823+0000 7f3bca787740 4 client.25106 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T11:58:31.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.823+0000 7f3bca787740 1 -- 172.21.15.94:0/2564197536 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56234aebb860 con 0x56234b2ce1b0 2024-02-17T11:58:31.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.824+0000 7f3bc5952640 1 --2- 172.21.15.94:0/2564197536 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56234b2ce1b0 0x56234b2d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:31.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.824+0000 7f3bc5952640 1 --2- 172.21.15.94:0/2564197536 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56234b2ce1b0 0x56234b2d05d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d000286834bc4be8 server_cookie=d1068974443dc013 in_seq=0 out_seq=0 2024-02-17T11:58:31.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.824+0000 7f3ba97fa640 10 client.25106 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:31.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.825+0000 7f3ba97fa640 1 -- 172.21.15.94:0/2564197536 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56234aebb860 con 0x56234b2ce1b0 2024-02-17T11:58:31.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.825+0000 7f3ba97fa640 10 client.25106 handle_command_reply: tid=0 2024-02-17T11:58:31.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.825+0000 7f3bca787740 1 client.25106 shutdown 2024-02-17T11:58:31.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.826+0000 7f3bca787740 1 -- 172.21.15.94:0/2564197536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254380 msgr2=0x56234b264cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:31.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.826+0000 7f3bca787740 1 --2- 172.21.15.94:0/2564197536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254380 0x56234b264cc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8004680 tx=0x7f3bb8005ae0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.826+0000 7f3bca787740 1 -- 172.21.15.94:0/2564197536 shutdown_connections 2024-02-17T11:58:31.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.826+0000 7f3bca787740 1 --2- 172.21.15.94:0/2564197536 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56234b2ce1b0 0x56234b2d05d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.826+0000 7f3bca787740 1 --2- 172.21.15.94:0/2564197536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56234b25d8b0 0x56234b263670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.826+0000 7f3ba97fa640 0 client.25106 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:31.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.826+0000 7f3bca787740 1 --2- 172.21.15.94:0/2564197536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56234b254d30 0x56234b2630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.826+0000 7f3bca787740 1 --2- 172.21.15.94:0/2564197536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56234b254380 0x56234b264cc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.826+0000 7f3bca787740 1 -- 172.21.15.94:0/2564197536 >> 172.21.15.94:0/2564197536 conn(0x56234b24e840 msgr2=0x56234b2598a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:31.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.826+0000 7f3bca787740 1 -- 172.21.15.94:0/2564197536 shutdown_connections 2024-02-17T11:58:31.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.826+0000 7f3bca787740 1 -- 172.21.15.94:0/2564197536 wait complete. 2024-02-17T11:58:31.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.826+0000 7f3bca787740 20 client.25106 trim_cache size 0 max 16384 2024-02-17T11:58:31.827 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:58:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T11:58:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "9d05eff0-32d1-4099-9585-e1f8541f92ec", 2024-02-17T11:58:31.828 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T11:58:31.828 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:58:31.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.827+0000 7f3bc7bb6640 1 -- 172.21.15.94:0/3153508149 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3bac0a2380 msgr2=0x7f3bac0a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:31.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.827+0000 7f3bc7bb6640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3bac0a2380 0x7f3bac0a4840 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00cf0d0 tx=0x7f3bb0070220 comp rx=0 tx=0).stop 2024-02-17T11:58:31.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.827+0000 7f3bc7bb6640 1 -- 172.21.15.94:0/3153508149 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bc0197620 msgr2=0x7f3bc0197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:31.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.827+0000 7f3bc7bb6640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bc0197620 0x7f3bc0197ad0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009c40 tx=0x7f3bb000b690 comp rx=0 tx=0).stop 2024-02-17T11:58:31.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.827+0000 7f3bc7bb6640 1 -- 172.21.15.94:0/3153508149 shutdown_connections 2024-02-17T11:58:31.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.827+0000 7f3bc7bb6640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3bac0a2380 0x7f3bac0a4840 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.827+0000 7f3bc7bb6640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3bc0197620 0x7f3bc0197ad0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.827+0000 7f3bc7bb6640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bc0106450 0x7f3bc019d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.827+0000 7f3bc7bb6640 1 --2- 172.21.15.94:0/3153508149 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3bc0101380 0x7f3bc019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:31.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.827+0000 7f3bc7bb6640 1 -- 172.21.15.94:0/3153508149 >> 172.21.15.94:0/3153508149 conn(0x7f3bc0078580 msgr2=0x7f3bc00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:31.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.827+0000 7f3bc7bb6640 1 -- 172.21.15.94:0/3153508149 shutdown_connections 2024-02-17T11:58:31.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:31.827+0000 7f3bc7bb6640 1 -- 172.21.15.94:0/3153508149 wait complete. 2024-02-17T11:58:31.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T11:58:31.841 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T11:58:31.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:31 smithi094 ceph-mon[32499]: pgmap v15: 97 pgs: 97 active+clean; 915 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 MiB/s wr, 602 op/s 2024-02-17T11:58:32.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.007+0000 7fc31559a640 1 -- 172.21.15.94:0/3995252873 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310032aa0 con 0x7fc318111fd0 2024-02-17T11:58:32.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.008+0000 7fc31cd93640 1 -- 172.21.15.94:0/3995252873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc318111fd0 msgr2=0x7fc3181143c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:32.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.008+0000 7fc31cd93640 1 --2- 172.21.15.94:0/3995252873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc318111fd0 0x7fc3181143c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc310009ca0 tx=0x7fc31002c0f0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.008+0000 7fc31cd93640 1 -- 172.21.15.94:0/3995252873 shutdown_connections 2024-02-17T11:58:32.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.008+0000 7fc31cd93640 1 --2- 172.21.15.94:0/3995252873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc318111fd0 0x7fc3181143c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.008+0000 7fc31cd93640 1 --2- 172.21.15.94:0/3995252873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc318075fd0 0x7fc318111a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.008+0000 7fc31cd93640 1 --2- 172.21.15.94:0/3995252873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3180756b0 0x7fc318075a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.008+0000 7fc31cd93640 1 -- 172.21.15.94:0/3995252873 >> 172.21.15.94:0/3995252873 conn(0x7fc3180fee00 msgr2=0x7fc318101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:32.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.008+0000 7fc31cd93640 1 -- 172.21.15.94:0/3995252873 shutdown_connections 2024-02-17T11:58:32.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.008+0000 7fc31cd93640 1 -- 172.21.15.94:0/3995252873 wait complete. 2024-02-17T11:58:32.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc31cd93640 1 Processor -- start 2024-02-17T11:58:32.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc31cd93640 1 -- start start 2024-02-17T11:58:32.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc31cd93640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3180756b0 0x7fc31819dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc31cd93640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc318075fd0 0x7fc31819e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc31cd93640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc318111fd0 0x7fc3181a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc31cd93640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3181a2c20 con 0x7fc318111fd0 2024-02-17T11:58:32.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc31cd93640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3181a2d90 con 0x7fc318075fd0 2024-02-17T11:58:32.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc31cd93640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3181a2f50 con 0x7fc3180756b0 2024-02-17T11:58:32.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc316d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc318111fd0 0x7fc3181a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc316d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc318111fd0 0x7fc3181a2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37542/0 (socket says 172.21.15.94:37542) 2024-02-17T11:58:32.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc316d9d640 1 -- 172.21.15.94:0/2973059767 learned_addr learned my addr 172.21.15.94:0/2973059767 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:32.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc315d9b640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc318075fd0 0x7fc31819e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc31659c640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3180756b0 0x7fc31819dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc316d9d640 1 -- 172.21.15.94:0/2973059767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3180756b0 msgr2=0x7fc31819dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:32.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc316d9d640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3180756b0 0x7fc31819dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc316d9d640 1 -- 172.21.15.94:0/2973059767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc318075fd0 msgr2=0x7fc31819e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:32.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc316d9d640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc318075fd0 0x7fc31819e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc316d9d640 1 -- 172.21.15.94:0/2973059767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc310009900 con 0x7fc318111fd0 2024-02-17T11:58:32.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.009+0000 7fc315d9b640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc318075fd0 0x7fc31819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:32.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.010+0000 7fc316d9d640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc318111fd0 0x7fc3181a2620 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc310009bf0 tx=0x7fc3100325e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:32.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.010+0000 7fc3077fe640 1 -- 172.21.15.94:0/2973059767 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3100326a0 con 0x7fc318111fd0 2024-02-17T11:58:32.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.010+0000 7fc31cd93640 1 -- 172.21.15.94:0/2973059767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3181a31d0 con 0x7fc318111fd0 2024-02-17T11:58:32.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.010+0000 7fc31cd93640 1 -- 172.21.15.94:0/2973059767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3181aaab0 con 0x7fc318111fd0 2024-02-17T11:58:32.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.010+0000 7fc3077fe640 1 -- 172.21.15.94:0/2973059767 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc310003e50 con 0x7fc318111fd0 2024-02-17T11:58:32.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.010+0000 7fc3077fe640 1 -- 172.21.15.94:0/2973059767 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310004670 con 0x7fc318111fd0 2024-02-17T11:58:32.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.013+0000 7fc3077fe640 1 -- 172.21.15.94:0/2973059767 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7fc31003b040 con 0x7fc318111fd0 2024-02-17T11:58:32.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.013+0000 7fc31f96d740 1 Processor -- start 2024-02-17T11:58:32.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.013+0000 7fc31f96d740 1 -- start start 2024-02-17T11:58:32.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc31f96d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197794b2b0 0x56197794b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc31f96d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x561977954270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc31f96d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619779547b0 0x561977956ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc31f96d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619778ba280 con 0x56197794b2b0 2024-02-17T11:58:32.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc31f96d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619778b77b0 con 0x56197794bbd0 2024-02-17T11:58:32.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc31f96d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619778b2f90 con 0x5619779547b0 2024-02-17T11:58:32.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc3077fe640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc3000a2380 0x7fc3000a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc315d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197794b2b0 0x56197794b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc31659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x561977954270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc315d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197794b2b0 0x56197794b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37548/0 (socket says 172.21.15.94:37548) 2024-02-17T11:58:32.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc315d9b640 1 -- 172.21.15.94:0/1595421199 learned_addr learned my addr 172.21.15.94:0/1595421199 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:32.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc316d9d640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc3000a2380 0x7fc3000a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc315d9b640 1 --2- 172.21.15.94:0/1595421199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619779547b0 0x561977956ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc3077fe640 1 -- 172.21.15.94:0/2973059767 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc3100f02b0 con 0x7fc318111fd0 2024-02-17T11:58:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc31659c640 1 -- 172.21.15.94:0/1595421199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619779547b0 msgr2=0x561977956ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc31659c640 1 --2- 172.21.15.94:0/1595421199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619779547b0 0x561977956ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc31659c640 1 -- 172.21.15.94:0/1595421199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197794b2b0 msgr2=0x56197794b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc31659c640 1 --2- 172.21.15.94:0/1595421199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197794b2b0 0x56197794b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc31659c640 1 -- 172.21.15.94:0/1595421199 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561977871570 con 0x56197794bbd0 2024-02-17T11:58:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.014+0000 7fc315d9b640 1 --2- 172.21.15.94:0/1595421199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197794b2b0 0x56197794b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.015+0000 7fc316d9d640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc3000a2380 0x7fc3000a4840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc3100edaa0 tx=0x7fc3100b2300 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.016+0000 7fc31659c640 1 --2- 172.21.15.94:0/1595421199 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x561977954270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc308009c20 tx=0x7fc308011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a566d196ecfb9180 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.016+0000 7fc304ff9640 1 -- 172.21.15.94:0/1595421199 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308004050 con 0x56197794bbd0 2024-02-17T11:58:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.016+0000 7fc304ff9640 1 -- 172.21.15.94:0/1595421199 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc308005500 con 0x56197794bbd0 2024-02-17T11:58:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.016+0000 7fc304ff9640 1 -- 172.21.15.94:0/1595421199 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308004c30 con 0x56197794bbd0 2024-02-17T11:58:32.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.017+0000 7fc31f96d740 1 -- 172.21.15.94:0/1595421199 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 msgr2=0x561977954270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:32.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.017+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1595421199 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x561977954270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc308009c20 tx=0x7fc308011810 comp rx=0 tx=0).stop 2024-02-17T11:58:32.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.017+0000 7fc31f96d740 1 -- 172.21.15.94:0/1595421199 shutdown_connections 2024-02-17T11:58:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.017+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1595421199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619779547b0 0x561977956ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.017+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1595421199 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x561977954270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.017+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1595421199 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197794b2b0 0x56197794b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.017+0000 7fc31f96d740 1 -- 172.21.15.94:0/1595421199 >> 172.21.15.94:0/1595421199 conn(0x561977945990 msgr2=0x561977945d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.017+0000 7fc31f96d740 1 -- 172.21.15.94:0/1595421199 shutdown_connections 2024-02-17T11:58:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.017+0000 7fc31f96d740 1 -- 172.21.15.94:0/1595421199 wait complete. 2024-02-17T11:58:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.017+0000 7fc31f96d740 1 -- start start 2024-02-17T11:58:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31f96d740 1 Processor -- start 2024-02-17T11:58:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31f96d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794b2b0 0x561977954120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31f96d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x5619779514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31f96d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619779547b0 0x561977951a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31f96d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561977586430 con 0x5619779547b0 2024-02-17T11:58:32.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31f96d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561977542e00 con 0x56197794bbd0 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31f96d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561977837fd0 con 0x56197794b2b0 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31f96d740 10 client.0 _ll_register_callbacks cb 0x56197788dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc315d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619779547b0 0x561977951a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x5619779514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x5619779514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45716/0 (socket says 172.21.15.94:45716) 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc315d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619779547b0 0x561977951a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:37556/0 (socket says 172.21.15.94:37556) 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31659c640 1 -- 172.21.15.94:0/1279837087 learned_addr learned my addr 172.21.15.94:0/1279837087 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc315d9b640 1 --2- 172.21.15.94:0/1279837087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794b2b0 0x561977954120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31659c640 1 -- 172.21.15.94:0/1279837087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794b2b0 msgr2=0x561977954120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31659c640 1 --2- 172.21.15.94:0/1279837087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794b2b0 0x561977954120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31659c640 1 -- 172.21.15.94:0/1279837087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619779547b0 msgr2=0x561977951a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31659c640 1 --2- 172.21.15.94:0/1279837087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619779547b0 0x561977951a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc31659c640 1 -- 172.21.15.94:0/1279837087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc308009850 con 0x56197794bbd0 2024-02-17T11:58:32.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.018+0000 7fc315d9b640 1 --2- 172.21.15.94:0/1279837087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619779547b0 0x561977951a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.019+0000 7fc31659c640 1 --2- 172.21.15.94:0/1279837087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x5619779514b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc308005e00 tx=0x7fc308005670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b527677d8776d6e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:32.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.019+0000 7fc2fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:58:32.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.019+0000 7fc2fdffb640 1 -- 172.21.15.94:0/1279837087 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308003a00 con 0x56197794bbd0 2024-02-17T11:58:32.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.019+0000 7fc31f96d740 1 -- 172.21.15.94:0/1279837087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619775b27e0 con 0x56197794bbd0 2024-02-17T11:58:32.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.019+0000 7fc2fdffb640 1 -- 172.21.15.94:0/1279837087 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc308003b60 con 0x56197794bbd0 2024-02-17T11:58:32.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.019+0000 7fc2fdffb640 1 -- 172.21.15.94:0/1279837087 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc308004510 con 0x56197794bbd0 2024-02-17T11:58:32.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.020+0000 7fc31f96d740 10 client.25118 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:32.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.020+0000 7fc31f96d740 1 -- 172.21.15.94:0/1279837087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5619778b1610 con 0x56197794bbd0 2024-02-17T11:58:32.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.020+0000 7fc2fdffb640 1 -- 172.21.15.94:0/1279837087 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5619775b27e0 con 0x56197794bbd0 2024-02-17T11:58:32.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.020+0000 7fc2fdffb640 1 -- 172.21.15.94:0/1279837087 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc308021040 con 0x56197794bbd0 2024-02-17T11:58:32.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.020+0000 7fc31f96d740 10 client.25118 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.020+0000 7fc31f96d740 10 client.25118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.021+0000 7fc31f96d740 20 client.25118 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.021+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1279837087 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561977964850 0x561977966c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.021+0000 7fc31f96d740 4 client.25118 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:58:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.021+0000 7fc31f96d740 1 -- 172.21.15.94:0/1279837087 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619775ef720 con 0x561977964850 2024-02-17T11:58:32.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.022+0000 7fc315d9b640 1 --2- 172.21.15.94:0/1279837087 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561977964850 0x561977966c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.022+0000 7fc2fdffb640 10 client.25118 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:32.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.022+0000 7fc315d9b640 1 --2- 172.21.15.94:0/1279837087 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561977964850 0x561977966c70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=761646d788b5960 server_cookie=86b382b14bc6afe0 in_seq=0 out_seq=0 2024-02-17T11:58:32.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc2fdffb640 1 -- 172.21.15.94:0/1279837087 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5619775ef720 con 0x561977964850 2024-02-17T11:58:32.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc2fdffb640 10 client.25118 handle_command_reply: tid=0 2024-02-17T11:58:32.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc31f96d740 1 client.25118 shutdown 2024-02-17T11:58:32.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc31f96d740 1 -- 172.21.15.94:0/1279837087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 msgr2=0x5619779514b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:32.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1279837087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x5619779514b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc308005e00 tx=0x7fc308005670 comp rx=0 tx=0).stop 2024-02-17T11:58:32.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc31f96d740 1 -- 172.21.15.94:0/1279837087 shutdown_connections 2024-02-17T11:58:32.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1279837087 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561977964850 0x561977966c70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1279837087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619779547b0 0x561977951a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1279837087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x5619779514b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1279837087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794b2b0 0x561977954120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc2fdffb640 0 client.25118 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:32.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc31f96d740 1 -- 172.21.15.94:0/1279837087 >> 172.21.15.94:0/1279837087 conn(0x561977945990 msgr2=0x561977947730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:32.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc31f96d740 1 -- 172.21.15.94:0/1279837087 shutdown_connections 2024-02-17T11:58:32.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc31f96d740 1 -- 172.21.15.94:0/1279837087 wait complete. 2024-02-17T11:58:32.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.023+0000 7fc31f96d740 20 client.25118 trim_cache size 0 max 16384 2024-02-17T11:58:32.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.043+0000 7fc31f96d740 1 Processor -- start 2024-02-17T11:58:32.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.043+0000 7fc31f96d740 1 -- start start 2024-02-17T11:58:32.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.043+0000 7fc31f96d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197794b2b0 0x561977953be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.043+0000 7fc31f96d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794bbd0 0x561977954120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.043+0000 7fc31f96d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619779547b0 0x561977950d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.043+0000 7fc31f96d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561977957480 con 0x56197794b2b0 2024-02-17T11:58:32.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.043+0000 7fc31f96d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561977960d60 con 0x5619779547b0 2024-02-17T11:58:32.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.043+0000 7fc31f96d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561977946790 con 0x56197794bbd0 2024-02-17T11:58:32.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.043+0000 7fc315d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197794b2b0 0x561977953be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.043+0000 7fc31659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794bbd0 0x561977954120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.043+0000 7fc31659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794bbd0 0x561977954120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:58188/0 (socket says 172.21.15.94:58188) 2024-02-17T11:58:32.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.044+0000 7fc31659c640 1 -- 172.21.15.94:0/312905429 learned_addr learned my addr 172.21.15.94:0/312905429 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:32.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.044+0000 7fc315d9b640 1 --2- 172.21.15.94:0/312905429 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619779547b0 0x561977950d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.044+0000 7fc315d9b640 1 -- 172.21.15.94:0/312905429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794bbd0 msgr2=0x561977954120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:32.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.045+0000 7fc315d9b640 1 --2- 172.21.15.94:0/312905429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794bbd0 0x561977954120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.045+0000 7fc315d9b640 1 -- 172.21.15.94:0/312905429 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619779547b0 msgr2=0x561977950d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:32.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.045+0000 7fc315d9b640 1 --2- 172.21.15.94:0/312905429 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619779547b0 0x561977950d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.045+0000 7fc315d9b640 1 -- 172.21.15.94:0/312905429 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56197784dac0 con 0x56197794b2b0 2024-02-17T11:58:32.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.045+0000 7fc315d9b640 1 --2- 172.21.15.94:0/312905429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197794b2b0 0x561977953be0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x561977964c30 tx=0x7fc30c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5aefdf850a1b76 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:32.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.045+0000 7fc2fffff640 1 -- 172.21.15.94:0/312905429 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c003c80 con 0x56197794b2b0 2024-02-17T11:58:32.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.045+0000 7fc2fffff640 1 -- 172.21.15.94:0/312905429 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc30c02a040 con 0x56197794b2b0 2024-02-17T11:58:32.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.045+0000 7fc2fffff640 1 -- 172.21.15.94:0/312905429 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c004b90 con 0x56197794b2b0 2024-02-17T11:58:32.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.046+0000 7fc31f96d740 1 -- 172.21.15.94:0/312905429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197794b2b0 msgr2=0x561977953be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:32.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.046+0000 7fc31f96d740 1 --2- 172.21.15.94:0/312905429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197794b2b0 0x561977953be0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x561977964c30 tx=0x7fc30c004730 comp rx=0 tx=0).stop 2024-02-17T11:58:32.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.046+0000 7fc31f96d740 1 -- 172.21.15.94:0/312905429 shutdown_connections 2024-02-17T11:58:32.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.046+0000 7fc31f96d740 1 --2- 172.21.15.94:0/312905429 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619779547b0 0x561977950d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.046+0000 7fc31f96d740 1 --2- 172.21.15.94:0/312905429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794bbd0 0x561977954120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.046+0000 7fc31f96d740 1 --2- 172.21.15.94:0/312905429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56197794b2b0 0x561977953be0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.046+0000 7fc31f96d740 1 -- 172.21.15.94:0/312905429 >> 172.21.15.94:0/312905429 conn(0x561977945990 msgr2=0x56197794fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:32.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.046+0000 7fc31f96d740 1 -- 172.21.15.94:0/312905429 shutdown_connections 2024-02-17T11:58:32.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.046+0000 7fc31f96d740 1 -- 172.21.15.94:0/312905429 wait complete. 2024-02-17T11:58:32.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31f96d740 1 -- start start 2024-02-17T11:58:32.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31f96d740 1 Processor -- start 2024-02-17T11:58:32.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31f96d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794b2b0 0x561977958380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31f96d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x5619779588c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31f96d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619779547b0 0x561977958e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31f96d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619779bae80 con 0x5619779547b0 2024-02-17T11:58:32.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31f96d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619779baff0 con 0x56197794bbd0 2024-02-17T11:58:32.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31f96d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619779bb1b0 con 0x56197794b2b0 2024-02-17T11:58:32.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31f96d740 10 client.0 _ll_register_callbacks cb 0x7fc3080097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:32.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc315d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794b2b0 0x561977958380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc315d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794b2b0 0x561977958380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:58202/0 (socket says 172.21.15.94:58202) 2024-02-17T11:58:32.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x5619779588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc315d9b640 1 -- 172.21.15.94:0/1970628893 learned_addr learned my addr 172.21.15.94:0/1970628893 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc315d9b640 1 --2- 172.21.15.94:0/1970628893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619779547b0 0x561977958e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31659c640 1 -- 172.21.15.94:0/1970628893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794b2b0 msgr2=0x561977958380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31659c640 1 --2- 172.21.15.94:0/1970628893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794b2b0 0x561977958380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31659c640 1 -- 172.21.15.94:0/1970628893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619779547b0 msgr2=0x561977958e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31659c640 1 --2- 172.21.15.94:0/1970628893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619779547b0 0x561977958e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.047+0000 7fc31659c640 1 -- 172.21.15.94:0/1970628893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30c0044f0 con 0x56197794bbd0 2024-02-17T11:58:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc31659c640 1 --2- 172.21.15.94:0/1970628893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x5619779588c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc308000c00 tx=0x7fc308004590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffa16b407e8eda7e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc2fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:58:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc2fdffb640 1 -- 172.21.15.94:0/1970628893 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308023ea0 con 0x56197794bbd0 2024-02-17T11:58:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc2fdffb640 1 -- 172.21.15.94:0/1970628893 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc308000e40 con 0x56197794bbd0 2024-02-17T11:58:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc31f96d740 1 -- 172.21.15.94:0/1970628893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc308005500 con 0x56197794bbd0 2024-02-17T11:58:32.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc2fdffb640 1 -- 172.21.15.94:0/1970628893 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc308009a70 con 0x56197794bbd0 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc31f96d740 10 client.25124 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc31f96d740 1 -- 172.21.15.94:0/1970628893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5619778736a0 con 0x56197794bbd0 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc2fdffb640 1 -- 172.21.15.94:0/1970628893 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc308005500 con 0x56197794bbd0 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc2fdffb640 1 -- 172.21.15.94:0/1970628893 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc30802e6a0 con 0x56197794bbd0 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc31f96d740 10 client.25124 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc31f96d740 10 client.25124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc31f96d740 20 client.25124 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1970628893 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5619779bc5c0 0x5619779be9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc31f96d740 4 client.25124 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc31f96d740 1 -- 172.21.15.94:0/1970628893 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56197763e890 con 0x5619779bc5c0 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.048+0000 7fc315d9b640 1 --2- 172.21.15.94:0/1970628893 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5619779bc5c0 0x5619779be9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.049+0000 7fc2fdffb640 10 client.25124 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.049+0000 7fc315d9b640 1 --2- 172.21.15.94:0/1970628893 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5619779bc5c0 0x5619779be9b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44e24c55b6b1bda server_cookie=4eca7d9064ca94b0 in_seq=0 out_seq=0 2024-02-17T11:58:32.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.049+0000 7fc2fdffb640 1 -- 172.21.15.94:0/1970628893 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56197763e890 con 0x5619779bc5c0 2024-02-17T11:58:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.049+0000 7fc2fdffb640 10 client.25124 handle_command_reply: tid=0 2024-02-17T11:58:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.049+0000 7fc31f96d740 1 client.25124 shutdown 2024-02-17T11:58:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.050+0000 7fc31f96d740 1 -- 172.21.15.94:0/1970628893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 msgr2=0x5619779588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.050+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1970628893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x5619779588c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc308000c00 tx=0x7fc308004590 comp rx=0 tx=0).stop 2024-02-17T11:58:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.050+0000 7fc31f96d740 1 -- 172.21.15.94:0/1970628893 shutdown_connections 2024-02-17T11:58:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.050+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1970628893 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5619779bc5c0 0x5619779be9b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.050+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1970628893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619779547b0 0x561977958e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.050+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1970628893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56197794bbd0 0x5619779588c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.050+0000 7fc31f96d740 1 --2- 172.21.15.94:0/1970628893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56197794b2b0 0x561977958380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.050+0000 7fc31f96d740 1 -- 172.21.15.94:0/1970628893 >> 172.21.15.94:0/1970628893 conn(0x561977945990 msgr2=0x561977950d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.050+0000 7fc2fdffb640 0 client.25124 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.050+0000 7fc31f96d740 1 -- 172.21.15.94:0/1970628893 shutdown_connections 2024-02-17T11:58:32.055 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:58:32.055 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T11:58:32.055 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T11:58:32.055 INFO:teuthology.orchestra.run.smithi094.stdout: "9d05eff0-32d1-4099-9585-e1f8541f92ec": { 2024-02-17T11:58:32.055 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T11:58:32.055 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "9d05eff0-32d1-4099-9585-e1f8541f92ec", 2024-02-17T11:58:32.055 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T11:58:32.055 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T11:58:32.055 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T11:58:32.055 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:58:32.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.051+0000 7fc31f96d740 1 -- 172.21.15.94:0/1970628893 wait complete. 2024-02-17T11:58:32.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.051+0000 7fc31f96d740 20 client.25124 trim_cache size 0 max 16384 2024-02-17T11:58:32.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.053+0000 7fc31cd93640 1 -- 172.21.15.94:0/2973059767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc3000a2380 msgr2=0x7fc3000a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:32.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.053+0000 7fc31cd93640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc3000a2380 0x7fc3000a4840 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc3100edaa0 tx=0x7fc3100b2300 comp rx=0 tx=0).stop 2024-02-17T11:58:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.053+0000 7fc31cd93640 1 -- 172.21.15.94:0/2973059767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc318111fd0 msgr2=0x7fc3181a2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.053+0000 7fc31cd93640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc318111fd0 0x7fc3181a2620 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc310009bf0 tx=0x7fc3100325e0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.053+0000 7fc31cd93640 1 -- 172.21.15.94:0/2973059767 shutdown_connections 2024-02-17T11:58:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.053+0000 7fc31cd93640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc3000a2380 0x7fc3000a4840 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.053+0000 7fc31cd93640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc318111fd0 0x7fc3181a2620 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.053+0000 7fc31cd93640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc318075fd0 0x7fc31819e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.053+0000 7fc31cd93640 1 --2- 172.21.15.94:0/2973059767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3180756b0 0x7fc31819dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.053+0000 7fc31cd93640 1 -- 172.21.15.94:0/2973059767 >> 172.21.15.94:0/2973059767 conn(0x7fc3180fee00 msgr2=0x7fc318113a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.053+0000 7fc31cd93640 1 -- 172.21.15.94:0/2973059767 shutdown_connections 2024-02-17T11:58:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:58:32.053+0000 7fc31cd93640 1 -- 172.21.15.94:0/2973059767 wait complete. 2024-02-17T11:58:32.064 INFO:tasks.cephfs.filesystem:scrub status for tag:9d05eff0-32d1-4099-9585-e1f8541f92ec - {'path': '/', 'tag': '9d05eff0-32d1-4099-9585-e1f8541f92ec', 'options': 'recursive,force'} 2024-02-17T11:58:32.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:31 smithi189 ceph-mon[36588]: pgmap v15: 97 pgs: 97 active+clean; 915 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 MiB/s wr, 602 op/s 2024-02-17T11:58:32.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:31 smithi125 ceph-mon[36819]: pgmap v15: 97 pgs: 97 active+clean; 915 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 33 MiB/s wr, 602 op/s 2024-02-17T11:58:33.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:32 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T11:58:33.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:32 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T11:58:33.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:32 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T11:58:33.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:32 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T11:58:33.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:32 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T11:58:33.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:32 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T11:58:33.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:32 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T11:58:33.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:32 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T11:58:33.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:32 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T11:58:33.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:32 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T11:58:33.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:32 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T11:58:33.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:32 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T11:58:33.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:32 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T11:58:33.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:32 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T11:58:33.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:32 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T11:58:34.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:33 smithi189 ceph-mon[36588]: pgmap v16: 97 pgs: 97 active+clean; 918 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 33 MiB/s wr, 690 op/s 2024-02-17T11:58:34.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:33 smithi125 ceph-mon[36819]: pgmap v16: 97 pgs: 97 active+clean; 918 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 33 MiB/s wr, 690 op/s 2024-02-17T11:58:34.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:33 smithi094 ceph-mon[32499]: pgmap v16: 97 pgs: 97 active+clean; 918 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 33 MiB/s wr, 690 op/s 2024-02-17T11:58:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 15.0 s, 3975.1 tps, lat 25.114 ms stddev 25.838 2024-02-17T11:58:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.564+0000 7f73602ae640 1 -- 172.21.15.94:0/754609934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73581015b0 msgr2=0x7f735810cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.564+0000 7f73602ae640 1 --2- 172.21.15.94:0/754609934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73581015b0 0x7f735810cf10 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7350009cd0 tx=0x7f735002c0f0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.564+0000 7f73602ae640 1 -- 172.21.15.94:0/754609934 shutdown_connections 2024-02-17T11:58:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.564+0000 7f73602ae640 1 --2- 172.21.15.94:0/754609934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73581015b0 0x7f735810cf10 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.564+0000 7f73602ae640 1 --2- 172.21.15.94:0/754609934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7358100bf0 0x7f7358101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.564+0000 7f73602ae640 1 --2- 172.21.15.94:0/754609934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7358105080 0x7f7358105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.564+0000 7f73602ae640 1 -- 172.21.15.94:0/754609934 >> 172.21.15.94:0/754609934 conn(0x7f7358076210 msgr2=0x7f73580fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.564+0000 7f73602ae640 1 -- 172.21.15.94:0/754609934 shutdown_connections 2024-02-17T11:58:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.564+0000 7f73602ae640 1 -- 172.21.15.94:0/754609934 wait complete. 2024-02-17T11:58:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f73602ae640 1 Processor -- start 2024-02-17T11:58:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f73602ae640 1 -- start start 2024-02-17T11:58:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f73602ae640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7358100bf0 0x7f7358195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f73602ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73581015b0 0x7f7358195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f73602ae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7358105080 0x7f7358199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f73602ae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73581a2090 con 0x7f73581015b0 2024-02-17T11:58:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f73602ae640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73581a2200 con 0x7f7358100bf0 2024-02-17T11:58:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f73602ae640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735819a2f0 con 0x7f7358105080 2024-02-17T11:58:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f735d849640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73581015b0 0x7f7358195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f735d849640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73581015b0 0x7f7358195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59714/0 (socket says 172.21.15.94:59714) 2024-02-17T11:58:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f735d849640 1 -- 172.21.15.94:0/2147449755 learned_addr learned my addr 172.21.15.94:0/2147449755 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f735e84b640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7358105080 0x7f7358199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f735e04a640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7358100bf0 0x7f7358195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f735d849640 1 -- 172.21.15.94:0/2147449755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7358105080 msgr2=0x7f7358199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f735d849640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7358105080 0x7f7358199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f735d849640 1 -- 172.21.15.94:0/2147449755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7358100bf0 msgr2=0x7f7358195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.565+0000 7f735d849640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7358100bf0 0x7f7358195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.566+0000 7f735d849640 1 -- 172.21.15.94:0/2147449755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7350009900 con 0x7f73581015b0 2024-02-17T11:58:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.566+0000 7f735e04a640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7358100bf0 0x7f7358195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:58:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.566+0000 7f735d849640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73581015b0 0x7f7358195f50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f735400e9f0 tx=0x7f735400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.566+0000 7f734f7fe640 1 -- 172.21.15.94:0/2147449755 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354009720 con 0x7f73581015b0 2024-02-17T11:58:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.566+0000 7f734f7fe640 1 -- 172.21.15.94:0/2147449755 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f735400cd80 con 0x7f73581015b0 2024-02-17T11:58:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.566+0000 7f73602ae640 1 -- 172.21.15.94:0/2147449755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735819a5d0 con 0x7f73581015b0 2024-02-17T11:58:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.566+0000 7f734f7fe640 1 -- 172.21.15.94:0/2147449755 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354010690 con 0x7f73581015b0 2024-02-17T11:58:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.566+0000 7f73602ae640 1 -- 172.21.15.94:0/2147449755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735819aa90 con 0x7f73581015b0 2024-02-17T11:58:35.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.569+0000 7f734f7fe640 1 -- 172.21.15.94:0/2147449755 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f73540046c0 con 0x7f73581015b0 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.569+0000 7f7362b22740 1 Processor -- start 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.570+0000 7f7362b22740 1 -- start start 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.570+0000 7f7362b22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9108ad0 0x55a2e9107820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.570+0000 7f7362b22740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e9107d60 0x55a2e91081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.570+0000 7f7362b22740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e91112e0 0x55a2e91136d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.570+0000 7f7362b22740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e90769f0 con 0x55a2e9107d60 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.570+0000 7f7362b22740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e9073f20 con 0x55a2e91112e0 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.570+0000 7f7362b22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e906f700 con 0x55a2e9108ad0 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.570+0000 7f734f7fe640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f73440a23f0 0x7f73440a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.570+0000 7f735e84b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9108ad0 0x55a2e9107820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.570+0000 7f735e04a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e9107d60 0x55a2e91081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.570+0000 7f735e84b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9108ad0 0x55a2e9107820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39346/0 (socket says 172.21.15.94:39346) 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.570+0000 7f735e84b640 1 -- 172.21.15.94:0/3557650964 learned_addr learned my addr 172.21.15.94:0/3557650964 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:35.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.570+0000 7f735e84b640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f73440a23f0 0x7f73440a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.571+0000 7f734f7fe640 1 -- 172.21.15.94:0/2147449755 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7354014070 con 0x7f73581015b0 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.571+0000 7f735d849640 1 --2- 172.21.15.94:0/3557650964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e91112e0 0x55a2e91136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.571+0000 7f735e84b640 1 -- 172.21.15.94:0/3557650964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e91112e0 msgr2=0x55a2e91136d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.571+0000 7f735e84b640 1 --2- 172.21.15.94:0/3557650964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e91112e0 0x55a2e91136d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.571+0000 7f735e84b640 1 -- 172.21.15.94:0/3557650964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e9107d60 msgr2=0x55a2e91081e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.571+0000 7f735e84b640 1 --2- 172.21.15.94:0/3557650964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e9107d60 0x55a2e91081e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.571+0000 7f735e84b640 1 -- 172.21.15.94:0/3557650964 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2e90057b0 con 0x55a2e9108ad0 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.571+0000 7f735e04a640 1 --2- 172.21.15.94:0/3557650964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e9107d60 0x55a2e91081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.571+0000 7f735e84b640 1 --2- 172.21.15.94:0/3557650964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9108ad0 0x55a2e9107820 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7350005e30 tx=0x7f7350034860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bb479a85c6ecf77 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.572+0000 7f734cbfe640 1 -- 172.21.15.94:0/3557650964 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350048070 con 0x55a2e9108ad0 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.572+0000 7f734cbfe640 1 -- 172.21.15.94:0/3557650964 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7350032e70 con 0x55a2e9108ad0 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.572+0000 7f734cbfe640 1 -- 172.21.15.94:0/3557650964 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350004ea0 con 0x55a2e9108ad0 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.573+0000 7f7362b22740 1 -- 172.21.15.94:0/3557650964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9108ad0 msgr2=0x55a2e9107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.573+0000 7f7362b22740 1 --2- 172.21.15.94:0/3557650964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9108ad0 0x55a2e9107820 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7350005e30 tx=0x7f7350034860 comp rx=0 tx=0).stop 2024-02-17T11:58:35.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.573+0000 7f735e84b640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f73440a23f0 0x7f73440a48b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7350009ca0 tx=0x7f735003c000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.573+0000 7f7362b22740 1 -- 172.21.15.94:0/3557650964 shutdown_connections 2024-02-17T11:58:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.573+0000 7f7362b22740 1 --2- 172.21.15.94:0/3557650964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e91112e0 0x55a2e91136d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.573+0000 7f7362b22740 1 --2- 172.21.15.94:0/3557650964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e9107d60 0x55a2e91081e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.573+0000 7f7362b22740 1 --2- 172.21.15.94:0/3557650964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9108ad0 0x55a2e9107820 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.573+0000 7f7362b22740 1 -- 172.21.15.94:0/3557650964 >> 172.21.15.94:0/3557650964 conn(0x55a2e90cf550 msgr2=0x55a2e9102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.573+0000 7f7362b22740 1 -- 172.21.15.94:0/3557650964 shutdown_connections 2024-02-17T11:58:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.573+0000 7f7362b22740 1 -- 172.21.15.94:0/3557650964 wait complete. 2024-02-17T11:58:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.573+0000 7f7362b22740 1 -- start start 2024-02-17T11:58:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.573+0000 7f7362b22740 1 Processor -- start 2024-02-17T11:58:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f7362b22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9107d60 0x55a2e910e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f7362b22740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e9108ad0 0x55a2e910e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f7362b22740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e91112e0 0x55a2e910ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f7362b22740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e9014bf0 con 0x55a2e9108ad0 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f7362b22740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e8d43250 con 0x55a2e91112e0 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f7362b22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e8ff5200 con 0x55a2e9107d60 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f7362b22740 10 client.0 _ll_register_callbacks cb 0x55a2e9063d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f735d849640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e9108ad0 0x55a2e910e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f735d849640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e9108ad0 0x55a2e910e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59718/0 (socket says 172.21.15.94:59718) 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f735d849640 1 -- 172.21.15.94:0/3482044463 learned_addr learned my addr 172.21.15.94:0/3482044463 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f735e04a640 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9107d60 0x55a2e910e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f735e84b640 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e91112e0 0x55a2e910ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f735d849640 1 -- 172.21.15.94:0/3482044463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9107d60 msgr2=0x55a2e910e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f735d849640 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9107d60 0x55a2e910e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f735d849640 1 -- 172.21.15.94:0/3482044463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e91112e0 msgr2=0x55a2e910ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f735d849640 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e91112e0 0x55a2e910ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f735d849640 1 -- 172.21.15.94:0/3482044463 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735810d810 con 0x55a2e9108ad0 2024-02-17T11:58:35.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f735e04a640 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9107d60 0x55a2e910e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f735d849640 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e9108ad0 0x55a2e910e830 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f735400e9c0 tx=0x7f73540645b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8676f7b9c21835 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.574+0000 7f7341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7341ffb640 1 -- 172.21.15.94:0/3482044463 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735409aee0 con 0x55a2e9108ad0 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7341ffb640 1 -- 172.21.15.94:0/3482044463 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f735409f030 con 0x55a2e9108ad0 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7362b22740 1 -- 172.21.15.94:0/3482044463 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2e8d6f600 con 0x55a2e9108ad0 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7341ffb640 1 -- 172.21.15.94:0/3482044463 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73540a3680 con 0x55a2e9108ad0 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7362b22740 10 client.15648 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7362b22740 1 -- 172.21.15.94:0/3482044463 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55a2e906dd80 con 0x55a2e9108ad0 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7341ffb640 1 -- 172.21.15.94:0/3482044463 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f73540b92a0 con 0x55a2e9108ad0 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7341ffb640 1 -- 172.21.15.94:0/3482044463 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f735401f060 con 0x55a2e9108ad0 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7362b22740 10 client.15648 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7362b22740 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7362b22740 20 client.15648 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7362b22740 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a2e9121520 0x55a2e9123980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7362b22740 4 client.15648 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.575+0000 7f7362b22740 1 -- 172.21.15.94:0/3482044463 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2e8d6f600 con 0x55a2e9121520 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.576+0000 7f735e04a640 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a2e9121520 0x55a2e9123980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.576+0000 7f735e04a640 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a2e9121520 0x55a2e9123980 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a6a6889b64249a5 server_cookie=1f9c2b138d3011dc in_seq=0 out_seq=0 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.576+0000 7f7341ffb640 10 client.15648 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7341ffb640 1 -- 172.21.15.94:0/3482044463 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a2e8d6f600 con 0x55a2e9121520 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7341ffb640 10 client.15648 handle_command_reply: tid=0 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7362b22740 1 client.15648 shutdown 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7362b22740 1 -- 172.21.15.94:0/3482044463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e9108ad0 msgr2=0x55a2e910e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7362b22740 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e9108ad0 0x55a2e910e830 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f735400e9c0 tx=0x7f73540645b0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7362b22740 1 -- 172.21.15.94:0/3482044463 shutdown_connections 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7362b22740 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a2e9121520 0x55a2e9123980 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7341ffb640 0 client.15648 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7362b22740 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e91112e0 0x55a2e910ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7362b22740 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e9108ad0 0x55a2e910e830 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7362b22740 1 --2- 172.21.15.94:0/3482044463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9107d60 0x55a2e910e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7362b22740 1 -- 172.21.15.94:0/3482044463 >> 172.21.15.94:0/3482044463 conn(0x55a2e90cf550 msgr2=0x55a2e91047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7362b22740 1 -- 172.21.15.94:0/3482044463 shutdown_connections 2024-02-17T11:58:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.577+0000 7f7362b22740 1 -- 172.21.15.94:0/3482044463 wait complete. 2024-02-17T11:58:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.578+0000 7f7362b22740 20 client.15648 trim_cache size 0 max 16384 2024-02-17T11:58:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.600+0000 7f7362b22740 1 Processor -- start 2024-02-17T11:58:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.601+0000 7f7362b22740 1 -- start start 2024-02-17T11:58:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.601+0000 7f7362b22740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e9107d60 0x55a2e9109600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.601+0000 7f7362b22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9108ad0 0x55a2e9109b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.601+0000 7f7362b22740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e91112e0 0x55a2e910cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.602+0000 7f735e04a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e9107d60 0x55a2e9109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.602+0000 7f735e04a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e9107d60 0x55a2e9109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45626/0 (socket says 172.21.15.94:45626) 2024-02-17T11:58:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.602+0000 7f735d849640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9108ad0 0x55a2e9109b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.602+0000 7f735d849640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9108ad0 0x55a2e9109b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39368/0 (socket says 172.21.15.94:39368) 2024-02-17T11:58:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.602+0000 7f735e04a640 1 -- 172.21.15.94:0/3831265336 learned_addr learned my addr 172.21.15.94:0/3831265336 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.602+0000 7f7362b22740 1 -- 172.21.15.94:0/3831265336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e90f74a0 con 0x55a2e91112e0 2024-02-17T11:58:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.602+0000 7f7362b22740 1 -- 172.21.15.94:0/3831265336 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e9107af0 con 0x55a2e9107d60 2024-02-17T11:58:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.602+0000 7f7362b22740 1 -- 172.21.15.94:0/3831265336 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e8d6f600 con 0x55a2e9108ad0 2024-02-17T11:58:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.602+0000 7f735e84b640 1 --2- 172.21.15.94:0/3831265336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e91112e0 0x55a2e910cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.603+0000 7f735e04a640 1 -- 172.21.15.94:0/3831265336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9108ad0 msgr2=0x55a2e9109b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.603+0000 7f735e04a640 1 --2- 172.21.15.94:0/3831265336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9108ad0 0x55a2e9109b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.603+0000 7f735e04a640 1 -- 172.21.15.94:0/3831265336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e91112e0 msgr2=0x55a2e910cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:35.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.603+0000 7f735e04a640 1 --2- 172.21.15.94:0/3831265336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e91112e0 0x55a2e910cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.603+0000 7f735e04a640 1 -- 172.21.15.94:0/3831265336 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2e900a7b0 con 0x55a2e9107d60 2024-02-17T11:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.603+0000 7f735e84b640 1 --2- 172.21.15.94:0/3831265336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e91112e0 0x55a2e910cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.604+0000 7f735e04a640 1 --2- 172.21.15.94:0/3831265336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e9107d60 0x55a2e9109600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f73480112e0 tx=0x7f7348012950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ba8fcd3d87391d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.604+0000 7f7343fff640 1 -- 172.21.15.94:0/3831265336 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348016040 con 0x55a2e9107d60 2024-02-17T11:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.604+0000 7f7343fff640 1 -- 172.21.15.94:0/3831265336 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7348018040 con 0x55a2e9107d60 2024-02-17T11:58:35.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.604+0000 7f7343fff640 1 -- 172.21.15.94:0/3831265336 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348025630 con 0x55a2e9107d60 2024-02-17T11:58:35.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.605+0000 7f7362b22740 1 -- 172.21.15.94:0/3831265336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e9107d60 msgr2=0x55a2e9109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.605+0000 7f7362b22740 1 --2- 172.21.15.94:0/3831265336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e9107d60 0x55a2e9109600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f73480112e0 tx=0x7f7348012950 comp rx=0 tx=0).stop 2024-02-17T11:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.606+0000 7f7362b22740 1 -- 172.21.15.94:0/3831265336 shutdown_connections 2024-02-17T11:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.606+0000 7f7362b22740 1 --2- 172.21.15.94:0/3831265336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e91112e0 0x55a2e910cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.606+0000 7f7362b22740 1 --2- 172.21.15.94:0/3831265336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9108ad0 0x55a2e9109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.606+0000 7f7362b22740 1 --2- 172.21.15.94:0/3831265336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e9107d60 0x55a2e9109600 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.606+0000 7f7362b22740 1 -- 172.21.15.94:0/3831265336 >> 172.21.15.94:0/3831265336 conn(0x55a2e90cf550 msgr2=0x55a2e91282e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.606+0000 7f7362b22740 1 -- 172.21.15.94:0/3831265336 shutdown_connections 2024-02-17T11:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.606+0000 7f7362b22740 1 -- 172.21.15.94:0/3831265336 wait complete. 2024-02-17T11:58:35.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.607+0000 7f7362b22740 1 -- start start 2024-02-17T11:58:35.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.607+0000 7f7362b22740 1 Processor -- start 2024-02-17T11:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.608+0000 7f7362b22740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9107d60 0x55a2e9118410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.608+0000 7f7362b22740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e9108ad0 0x55a2e91167b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.608+0000 7f7362b22740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e91112e0 0x55a2e9116cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.608+0000 7f7362b22740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e91173c0 con 0x55a2e91112e0 2024-02-17T11:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.608+0000 7f7362b22740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e9117530 con 0x55a2e9108ad0 2024-02-17T11:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.608+0000 7f7362b22740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e9177a70 con 0x55a2e9107d60 2024-02-17T11:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.608+0000 7f7362b22740 10 client.0 _ll_register_callbacks cb 0x7f7354005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.608+0000 7f735e04a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9107d60 0x55a2e9118410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.608+0000 7f735e04a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9107d60 0x55a2e9118410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39372/0 (socket says 172.21.15.94:39372) 2024-02-17T11:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.608+0000 7f735e04a640 1 -- 172.21.15.94:0/2815960288 learned_addr learned my addr 172.21.15.94:0/2815960288 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.608+0000 7f735e84b640 1 --2- 172.21.15.94:0/2815960288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e91112e0 0x55a2e9116cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.608+0000 7f735d849640 1 --2- 172.21.15.94:0/2815960288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e9108ad0 0x55a2e91167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.609+0000 7f735e84b640 1 -- 172.21.15.94:0/2815960288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9107d60 msgr2=0x55a2e9118410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:35.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.609+0000 7f735e84b640 1 --2- 172.21.15.94:0/2815960288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9107d60 0x55a2e9118410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.609+0000 7f735e84b640 1 -- 172.21.15.94:0/2815960288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e9108ad0 msgr2=0x55a2e91167b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.609+0000 7f735e84b640 1 --2- 172.21.15.94:0/2815960288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e9108ad0 0x55a2e91167b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.609+0000 7f735e84b640 1 -- 172.21.15.94:0/2815960288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7348011050 con 0x55a2e91112e0 2024-02-17T11:58:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.609+0000 7f735e84b640 1 --2- 172.21.15.94:0/2815960288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e91112e0 0x55a2e9116cf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7350033000 tx=0x7f7350034790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929b7fdc73632d2a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.609+0000 7f7341ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:58:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.609+0000 7f7341ffb640 1 -- 172.21.15.94:0/2815960288 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350048040 con 0x55a2e91112e0 2024-02-17T11:58:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.609+0000 7f7341ffb640 1 -- 172.21.15.94:0/2815960288 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f735005c070 con 0x55a2e91112e0 2024-02-17T11:58:35.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.609+0000 7f7362b22740 1 -- 172.21.15.94:0/2815960288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7350032e70 con 0x55a2e91112e0 2024-02-17T11:58:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.610+0000 7f7341ffb640 1 -- 172.21.15.94:0/2815960288 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7350034560 con 0x55a2e91112e0 2024-02-17T11:58:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.610+0000 7f7362b22740 10 client.15654 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.610+0000 7f7362b22740 1 -- 172.21.15.94:0/2815960288 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55a2e9030540 con 0x55a2e91112e0 2024-02-17T11:58:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.611+0000 7f7341ffb640 1 -- 172.21.15.94:0/2815960288 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7350068030 con 0x55a2e91112e0 2024-02-17T11:58:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.611+0000 7f7341ffb640 1 -- 172.21.15.94:0/2815960288 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7350053070 con 0x55a2e91112e0 2024-02-17T11:58:35.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.611+0000 7f7362b22740 10 client.15654 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.611+0000 7f7362b22740 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.611+0000 7f7362b22740 20 client.15654 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.611+0000 7f7362b22740 1 --2- 172.21.15.94:0/2815960288 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a2e9121520 0x55a2e917ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.611+0000 7f7362b22740 4 client.15654 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T11:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.611+0000 7f7362b22740 1 -- 172.21.15.94:0/2815960288 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a2e8dc01b0 con 0x55a2e9121520 2024-02-17T11:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.611+0000 7f735e04a640 1 --2- 172.21.15.94:0/2815960288 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a2e9121520 0x55a2e917ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.613+0000 7f735e04a640 1 --2- 172.21.15.94:0/2815960288 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a2e9121520 0x55a2e917ae90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b47361e834770cff server_cookie=cbdc469ea5d2e6ea in_seq=0 out_seq=0 2024-02-17T11:58:35.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.613+0000 7f7341ffb640 10 client.15654 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7341ffb640 1 -- 172.21.15.94:0/2815960288 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a2e8dc01b0 con 0x55a2e9121520 2024-02-17T11:58:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7341ffb640 10 client.15654 handle_command_reply: tid=0 2024-02-17T11:58:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7362b22740 1 client.15654 shutdown 2024-02-17T11:58:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7362b22740 1 -- 172.21.15.94:0/2815960288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e91112e0 msgr2=0x55a2e9116cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7362b22740 1 --2- 172.21.15.94:0/2815960288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e91112e0 0x55a2e9116cf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7350033000 tx=0x7f7350034790 comp rx=0 tx=0).stop 2024-02-17T11:58:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7362b22740 1 -- 172.21.15.94:0/2815960288 shutdown_connections 2024-02-17T11:58:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7362b22740 1 --2- 172.21.15.94:0/2815960288 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a2e9121520 0x55a2e917ae90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7362b22740 1 --2- 172.21.15.94:0/2815960288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a2e91112e0 0x55a2e9116cf0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7362b22740 1 --2- 172.21.15.94:0/2815960288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a2e9108ad0 0x55a2e91167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7362b22740 1 --2- 172.21.15.94:0/2815960288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a2e9107d60 0x55a2e9118410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7362b22740 1 -- 172.21.15.94:0/2815960288 >> 172.21.15.94:0/2815960288 conn(0x55a2e90cf550 msgr2=0x55a2e9109600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7341ffb640 0 client.15654 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7362b22740 1 -- 172.21.15.94:0/2815960288 shutdown_connections 2024-02-17T11:58:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7362b22740 1 -- 172.21.15.94:0/2815960288 wait complete. 2024-02-17T11:58:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.631+0000 7f7362b22740 20 client.15654 trim_cache size 0 max 16384 2024-02-17T11:58:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T11:58:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T11:58:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T11:58:35.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T11:58:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.632+0000 7f73602ae640 1 -- 172.21.15.94:0/2147449755 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f73440a23f0 msgr2=0x7f73440a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:35.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.632+0000 7f73602ae640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f73440a23f0 0x7f73440a48b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7350009ca0 tx=0x7f735003c000 comp rx=0 tx=0).stop 2024-02-17T11:58:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.632+0000 7f73602ae640 1 -- 172.21.15.94:0/2147449755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73581015b0 msgr2=0x7f7358195f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.632+0000 7f73602ae640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73581015b0 0x7f7358195f50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f735400e9f0 tx=0x7f735400edf0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.632+0000 7f73602ae640 1 -- 172.21.15.94:0/2147449755 shutdown_connections 2024-02-17T11:58:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.632+0000 7f73602ae640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f73440a23f0 0x7f73440a48b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.632+0000 7f73602ae640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7358105080 0x7f7358199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.632+0000 7f73602ae640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73581015b0 0x7f7358195f50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.632+0000 7f73602ae640 1 --2- 172.21.15.94:0/2147449755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7358100bf0 0x7f7358195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.632+0000 7f73602ae640 1 -- 172.21.15.94:0/2147449755 >> 172.21.15.94:0/2147449755 conn(0x7f7358076210 msgr2=0x7f7358101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.632+0000 7f73602ae640 1 -- 172.21.15.94:0/2147449755 shutdown_connections 2024-02-17T11:58:35.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:35.632+0000 7f73602ae640 1 -- 172.21.15.94:0/2147449755 wait complete. 2024-02-17T11:58:36.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:35 smithi189 ceph-mon[36588]: pgmap v17: 97 pgs: 97 active+clean; 928 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 30 MiB/s wr, 945 op/s 2024-02-17T11:58:36.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:35 smithi125 ceph-mon[36819]: pgmap v17: 97 pgs: 97 active+clean; 928 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 30 MiB/s wr, 945 op/s 2024-02-17T11:58:36.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:35 smithi094 ceph-mon[32499]: pgmap v17: 97 pgs: 97 active+clean; 928 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 30 MiB/s wr, 945 op/s 2024-02-17T11:58:38.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:37 smithi189 ceph-mon[36588]: pgmap v18: 97 pgs: 97 active+clean; 928 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.2 MiB/s wr, 732 op/s 2024-02-17T11:58:38.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:37 smithi125 ceph-mon[36819]: pgmap v18: 97 pgs: 97 active+clean; 928 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.2 MiB/s wr, 732 op/s 2024-02-17T11:58:38.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:37 smithi094 ceph-mon[32499]: pgmap v18: 97 pgs: 97 active+clean; 928 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 8.2 MiB/s wr, 732 op/s 2024-02-17T11:58:40.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:39 smithi189 ceph-mon[36588]: pgmap v19: 97 pgs: 97 active+clean; 951 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-02-17T11:58:40.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:39 smithi125 ceph-mon[36819]: pgmap v19: 97 pgs: 97 active+clean; 951 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-02-17T11:58:40.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:39 smithi094 ceph-mon[32499]: pgmap v19: 97 pgs: 97 active+clean; 951 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 13 MiB/s wr, 1.06k op/s 2024-02-17T11:58:40.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 20.0 s, 3780.8 tps, lat 26.476 ms stddev 30.220 2024-02-17T11:58:42.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:41 smithi189 ceph-mon[36588]: pgmap v20: 97 pgs: 97 active+clean; 951 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.0 MiB/s wr, 733 op/s 2024-02-17T11:58:42.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:41 smithi125 ceph-mon[36819]: pgmap v20: 97 pgs: 97 active+clean; 951 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.0 MiB/s wr, 733 op/s 2024-02-17T11:58:42.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:41 smithi094 ceph-mon[32499]: pgmap v20: 97 pgs: 97 active+clean; 951 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.0 MiB/s wr, 733 op/s 2024-02-17T11:58:44.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:43 smithi189 ceph-mon[36588]: pgmap v21: 97 pgs: 97 active+clean; 953 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 11 MiB/s wr, 978 op/s 2024-02-17T11:58:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:43 smithi125 ceph-mon[36819]: pgmap v21: 97 pgs: 97 active+clean; 953 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 11 MiB/s wr, 978 op/s 2024-02-17T11:58:44.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:43 smithi094 ceph-mon[32499]: pgmap v21: 97 pgs: 97 active+clean; 953 MiB data, 1011 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 11 MiB/s wr, 978 op/s 2024-02-17T11:58:45.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 25.0 s, 4077.4 tps, lat 24.549 ms stddev 25.761 2024-02-17T11:58:46.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:45 smithi189 ceph-mon[36588]: pgmap v22: 97 pgs: 97 active+clean; 954 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 969 op/s 2024-02-17T11:58:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:45 smithi125 ceph-mon[36819]: pgmap v22: 97 pgs: 97 active+clean; 954 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 969 op/s 2024-02-17T11:58:46.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:45 smithi094 ceph-mon[32499]: pgmap v22: 97 pgs: 97 active+clean; 954 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 969 op/s 2024-02-17T11:58:48.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:47 smithi189 ceph-mon[36588]: pgmap v23: 97 pgs: 97 active+clean; 954 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.0 MiB/s wr, 651 op/s 2024-02-17T11:58:48.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:47 smithi125 ceph-mon[36819]: pgmap v23: 97 pgs: 97 active+clean; 954 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.0 MiB/s wr, 651 op/s 2024-02-17T11:58:48.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:47 smithi094 ceph-mon[32499]: pgmap v23: 97 pgs: 97 active+clean; 954 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.0 MiB/s wr, 651 op/s 2024-02-17T11:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.746+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/1253044672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cec1002a0 msgr2=0x7f6cec105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.746+0000 7f6cf0bd0640 1 --2- 172.21.15.94:0/1253044672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cec1002a0 0x7f6cec105dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009900 tx=0x7f6cd402bc40 comp rx=0 tx=0).stop 2024-02-17T11:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.746+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/1253044672 shutdown_connections 2024-02-17T11:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.746+0000 7f6cf0bd0640 1 --2- 172.21.15.94:0/1253044672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cec106500 0x7f6cec10e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.746+0000 7f6cf0bd0640 1 --2- 172.21.15.94:0/1253044672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cec1002a0 0x7f6cec105dd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.746+0000 7f6cf0bd0640 1 --2- 172.21.15.94:0/1253044672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cec0ff8f0 0x7f6cec0ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.746+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/1253044672 >> 172.21.15.94:0/1253044672 conn(0x7f6cec076610 msgr2=0x7f6cec0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.746+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/1253044672 shutdown_connections 2024-02-17T11:58:48.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.746+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/1253044672 wait complete. 2024-02-17T11:58:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.747+0000 7f6cf0bd0640 1 Processor -- start 2024-02-17T11:58:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.747+0000 7f6cf0bd0640 1 -- start start 2024-02-17T11:58:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.748+0000 7f6cf0bd0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cec0ff8f0 0x7f6cec1954d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.748+0000 7f6cf0bd0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cec106500 0x7f6cec195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.748+0000 7f6cea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cec0ff8f0 0x7f6cec1954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.748+0000 7f6cea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cec0ff8f0 0x7f6cec1954d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46022/0 (socket says 172.21.15.94:46022) 2024-02-17T11:58:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.748+0000 7f6cea59c640 1 -- 172.21.15.94:0/2995821513 learned_addr learned my addr 172.21.15.94:0/2995821513 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.748+0000 7f6cf0bd0640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cec199d40 0x7f6cec19a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.748+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/2995821513 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec19a730 con 0x7f6cec0ff8f0 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.748+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/2995821513 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec19a8a0 con 0x7f6cec199d40 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.748+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/2995821513 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec19aa10 con 0x7f6cec106500 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.748+0000 7f6ce1d9b640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cec106500 0x7f6cec195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.749+0000 7f6cea59c640 1 -- 172.21.15.94:0/2995821513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cec106500 msgr2=0x7f6cec195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.749+0000 7f6cea59c640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cec106500 0x7f6cec195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.749+0000 7f6cea59c640 1 -- 172.21.15.94:0/2995821513 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cec199d40 msgr2=0x7f6cec19a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.749+0000 7f6cead9d640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cec199d40 0x7f6cec19a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.749+0000 7f6cea59c640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cec199d40 0x7f6cec19a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.749+0000 7f6cea59c640 1 -- 172.21.15.94:0/2995821513 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd4009560 con 0x7f6cec0ff8f0 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.749+0000 7f6cea59c640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cec0ff8f0 0x7f6cec1954d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c920 tx=0x7f6cdc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.749+0000 7f6ce3fff640 1 -- 172.21.15.94:0/2995821513 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc007c70 con 0x7f6cec0ff8f0 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.749+0000 7f6ce3fff640 1 -- 172.21.15.94:0/2995821513 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cdc007dd0 con 0x7f6cec0ff8f0 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.750+0000 7f6ce3fff640 1 -- 172.21.15.94:0/2995821513 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc018860 con 0x7f6cec0ff8f0 2024-02-17T11:58:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.750+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/2995821513 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec1a1ff0 con 0x7f6cec0ff8f0 2024-02-17T11:58:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.750+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/2995821513 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec1a2360 con 0x7f6cec0ff8f0 2024-02-17T11:58:48.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.754+0000 7f6ce3fff640 1 -- 172.21.15.94:0/2995821513 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 25) v1 ==== 136944+0+0 (secure 0 0 0) 0x7f6cdc0189c0 con 0x7f6cec0ff8f0 2024-02-17T11:58:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6cf37a2740 1 Processor -- start 2024-02-17T11:58:48.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6ce3fff640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6cd80a22c0 0x7f6cd80a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6ce3fff640 1 -- 172.21.15.94:0/2995821513 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6cdc0cdc20 con 0x7f6cec0ff8f0 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6cead9d640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6cd80a22c0 0x7f6cd80a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6cf37a2740 1 -- start start 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6cf37a2740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d228cad0 0x5575d228b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6cf37a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228bd60 0x5575d228c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6cf37a2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d22952e0 0x5575d22976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6cf37a2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d21fa9f0 con 0x5575d22952e0 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6cf37a2740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d21f7f20 con 0x5575d228cad0 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6cf37a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d21f3700 con 0x5575d228bd60 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6ce1d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228bd60 0x5575d228c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6ce1d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228bd60 0x5575d228c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:54952/0 (socket says 172.21.15.94:54952) 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6ce1d9b640 1 -- 172.21.15.94:0/1708295815 learned_addr learned my addr 172.21.15.94:0/1708295815 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6cead9d640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6cd80a22c0 0x7f6cd80a4780 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4011fd0 tx=0x7f6ce4011040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6cead9d640 1 --2- 172.21.15.94:0/1708295815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d22952e0 0x5575d22976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.755+0000 7f6cea59c640 1 --2- 172.21.15.94:0/1708295815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d228cad0 0x5575d228b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.756+0000 7f6cead9d640 1 -- 172.21.15.94:0/1708295815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228bd60 msgr2=0x5575d228c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.756+0000 7f6cead9d640 1 --2- 172.21.15.94:0/1708295815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228bd60 0x5575d228c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.756+0000 7f6cead9d640 1 -- 172.21.15.94:0/1708295815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d228cad0 msgr2=0x5575d228b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.756+0000 7f6cead9d640 1 --2- 172.21.15.94:0/1708295815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d228cad0 0x5575d228b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.756+0000 7f6cead9d640 1 -- 172.21.15.94:0/1708295815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575d21897b0 con 0x5575d22952e0 2024-02-17T11:58:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.756+0000 7f6ce1d9b640 1 --2- 172.21.15.94:0/1708295815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228bd60 0x5575d228c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.756+0000 7f6cead9d640 1 --2- 172.21.15.94:0/1708295815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d22952e0 0x5575d22976d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4005fd0 tx=0x7f6ce4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa47cb27f5b8ed6b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.757+0000 7f6ce0d99640 1 -- 172.21.15.94:0/1708295815 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4025070 con 0x5575d22952e0 2024-02-17T11:58:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.757+0000 7f6ce0d99640 1 -- 172.21.15.94:0/1708295815 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ce401d5a0 con 0x5575d22952e0 2024-02-17T11:58:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.757+0000 7f6ce0d99640 1 -- 172.21.15.94:0/1708295815 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4005280 con 0x5575d22952e0 2024-02-17T11:58:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.758+0000 7f6cf37a2740 1 -- 172.21.15.94:0/1708295815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d22952e0 msgr2=0x5575d22976d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.758+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/1708295815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d22952e0 0x5575d22976d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4005fd0 tx=0x7f6ce4019000 comp rx=0 tx=0).stop 2024-02-17T11:58:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.758+0000 7f6cf37a2740 1 -- 172.21.15.94:0/1708295815 shutdown_connections 2024-02-17T11:58:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.758+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/1708295815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d22952e0 0x5575d22976d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.758+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/1708295815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228bd60 0x5575d228c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.758+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/1708295815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d228cad0 0x5575d228b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.758+0000 7f6cf37a2740 1 -- 172.21.15.94:0/1708295815 >> 172.21.15.94:0/1708295815 conn(0x5575d2253550 msgr2=0x5575d2286a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.761+0000 7f6cf37a2740 1 -- 172.21.15.94:0/1708295815 shutdown_connections 2024-02-17T11:58:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.764+0000 7f6cf37a2740 1 -- 172.21.15.94:0/1708295815 wait complete. 2024-02-17T11:58:48.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.765+0000 7f6cf37a2740 1 -- start start 2024-02-17T11:58:48.764 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.765+0000 7f6cf37a2740 1 Processor -- start 2024-02-17T11:58:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.767+0000 7f6cf37a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228bd60 0x5575d22922a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.767+0000 7f6cf37a2740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d228cad0 0x5575d22927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.767+0000 7f6cf37a2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d22952e0 0x5575d2292d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.767+0000 7f6cf37a2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d2198bf0 con 0x5575d22952e0 2024-02-17T11:58:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.767+0000 7f6cf37a2740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d1ec7250 con 0x5575d228cad0 2024-02-17T11:58:48.766 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.767+0000 7f6cf37a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d2179200 con 0x5575d228bd60 2024-02-17T11:58:48.767 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.767+0000 7f6cf37a2740 10 client.0 _ll_register_callbacks cb 0x5575d21e7d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.768+0000 7f6cead9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d22952e0 0x5575d2292d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.768 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.768+0000 7f6cead9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d22952e0 0x5575d2292d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46048/0 (socket says 172.21.15.94:46048) 2024-02-17T11:58:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.768+0000 7f6cead9d640 1 -- 172.21.15.94:0/4240819872 learned_addr learned my addr 172.21.15.94:0/4240819872 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.768+0000 7f6cea59c640 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228bd60 0x5575d22922a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.769+0000 7f6ce1d9b640 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d228cad0 0x5575d22927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.769+0000 7f6cead9d640 1 -- 172.21.15.94:0/4240819872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228bd60 msgr2=0x5575d22922a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.769+0000 7f6cead9d640 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228bd60 0x5575d22922a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.769+0000 7f6cead9d640 1 -- 172.21.15.94:0/4240819872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d228cad0 msgr2=0x5575d22927e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.769+0000 7f6cead9d640 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d228cad0 0x5575d22927e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.769+0000 7f6cead9d640 1 -- 172.21.15.94:0/4240819872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce4020440 con 0x5575d22952e0 2024-02-17T11:58:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.769+0000 7f6ce1d9b640 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d228cad0 0x5575d22927e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:58:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.769+0000 7f6cea59c640 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228bd60 0x5575d22922a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.769+0000 7f6cead9d640 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d22952e0 0x5575d2292d20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6ce401d940 tx=0x7f6ce401d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521dbcd27439a72d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:48.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.769+0000 7f6cd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:58:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.769+0000 7f6cd1ffb640 1 -- 172.21.15.94:0/4240819872 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4025030 con 0x5575d22952e0 2024-02-17T11:58:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.770+0000 7f6cd1ffb640 1 -- 172.21.15.94:0/4240819872 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6ce4003970 con 0x5575d22952e0 2024-02-17T11:58:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.770+0000 7f6cf37a2740 1 -- 172.21.15.94:0/4240819872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575d1ef3600 con 0x5575d22952e0 2024-02-17T11:58:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.770+0000 7f6cd1ffb640 1 -- 172.21.15.94:0/4240819872 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ce4003b30 con 0x5575d22952e0 2024-02-17T11:58:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.770+0000 7f6cf37a2740 10 client.15663 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:48.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.770+0000 7f6cf37a2740 1 -- 172.21.15.94:0/4240819872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5575d21f1d80 con 0x5575d22952e0 2024-02-17T11:58:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.771+0000 7f6cd1ffb640 1 -- 172.21.15.94:0/4240819872 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5575d1ef3600 con 0x5575d22952e0 2024-02-17T11:58:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.771+0000 7f6cd1ffb640 1 -- 172.21.15.94:0/4240819872 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6ce4004f10 con 0x5575d22952e0 2024-02-17T11:58:48.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.771+0000 7f6cf37a2740 10 client.15663 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:48.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.771+0000 7f6cf37a2740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:48.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.772+0000 7f6cf37a2740 20 client.15663 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:48.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.772+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5575d22b65a0 0x5575d22b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.772+0000 7f6cf37a2740 4 client.15663 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:58:48.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.772+0000 7f6cf37a2740 1 -- 172.21.15.94:0/4240819872 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575d1f441b0 con 0x5575d22b65a0 2024-02-17T11:58:48.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.772+0000 7f6cea59c640 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5575d22b65a0 0x5575d22b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.773+0000 7f6cea59c640 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5575d22b65a0 0x5575d22b8990 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23490bf5943cbcf5 server_cookie=f617f7c266b9f44a in_seq=0 out_seq=0 2024-02-17T11:58:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.773+0000 7f6cd1ffb640 10 client.15663 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.774+0000 7f6cd1ffb640 1 -- 172.21.15.94:0/4240819872 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5575d1f441b0 con 0x5575d22b65a0 2024-02-17T11:58:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.774+0000 7f6cd1ffb640 10 client.15663 handle_command_reply: tid=0 2024-02-17T11:58:48.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.774+0000 7f6cf37a2740 1 client.15663 shutdown 2024-02-17T11:58:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.775+0000 7f6cf37a2740 1 -- 172.21.15.94:0/4240819872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d22952e0 msgr2=0x5575d2292d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.775+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d22952e0 0x5575d2292d20 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6ce401d940 tx=0x7f6ce401d970 comp rx=0 tx=0).stop 2024-02-17T11:58:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.775+0000 7f6cf37a2740 1 -- 172.21.15.94:0/4240819872 shutdown_connections 2024-02-17T11:58:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.775+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5575d22b65a0 0x5575d22b8990 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.775+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d22952e0 0x5575d2292d20 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.775+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d228cad0 0x5575d22927e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.775+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/4240819872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228bd60 0x5575d22922a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.775+0000 7f6cf37a2740 1 -- 172.21.15.94:0/4240819872 >> 172.21.15.94:0/4240819872 conn(0x5575d2253550 msgr2=0x5575d22887a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:48.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.775+0000 7f6cd1ffb640 0 client.15663 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.776+0000 7f6cf37a2740 1 -- 172.21.15.94:0/4240819872 shutdown_connections 2024-02-17T11:58:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.776+0000 7f6cf37a2740 1 -- 172.21.15.94:0/4240819872 wait complete. 2024-02-17T11:58:48.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.776+0000 7f6cf37a2740 20 client.15663 trim_cache size 0 max 16384 2024-02-17T11:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cf37a2740 1 Processor -- start 2024-02-17T11:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cf37a2740 1 -- start start 2024-02-17T11:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cf37a2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228bd60 0x5575d22949d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cf37a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228cad0 0x5575d2292810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cf37a2740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d22952e0 0x5575d2292d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cf37a2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d228d130 con 0x5575d228bd60 2024-02-17T11:58:48.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cf37a2740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d22b63a0 con 0x5575d22952e0 2024-02-17T11:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cf37a2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d2282310 con 0x5575d228cad0 2024-02-17T11:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228bd60 0x5575d22949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228bd60 0x5575d22949d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46050/0 (socket says 172.21.15.94:46050) 2024-02-17T11:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cea59c640 1 -- 172.21.15.94:0/3048524274 learned_addr learned my addr 172.21.15.94:0/3048524274 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6ce1d9b640 1 --2- 172.21.15.94:0/3048524274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228cad0 0x5575d2292810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cead9d640 1 --2- 172.21.15.94:0/3048524274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d22952e0 0x5575d2292d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cea59c640 1 -- 172.21.15.94:0/3048524274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228cad0 msgr2=0x5575d2292810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cea59c640 1 --2- 172.21.15.94:0/3048524274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228cad0 0x5575d2292810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cea59c640 1 -- 172.21.15.94:0/3048524274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d22952e0 msgr2=0x5575d2292d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cea59c640 1 --2- 172.21.15.94:0/3048524274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d22952e0 0x5575d2292d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.793+0000 7f6cea59c640 1 -- 172.21.15.94:0/3048524274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575d218e7b0 con 0x5575d228bd60 2024-02-17T11:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.794+0000 7f6cea59c640 1 --2- 172.21.15.94:0/3048524274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228bd60 0x5575d22949d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc09a3d0 tx=0x7f6cdc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5a408a151eff54 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.794+0000 7f6cd3fff640 1 -- 172.21.15.94:0/3048524274 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc098790 con 0x5575d228bd60 2024-02-17T11:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.794+0000 7f6cd3fff640 1 -- 172.21.15.94:0/3048524274 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cdc0b2070 con 0x5575d228bd60 2024-02-17T11:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.794+0000 7f6cd3fff640 1 -- 172.21.15.94:0/3048524274 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0998e0 con 0x5575d228bd60 2024-02-17T11:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.795+0000 7f6cf37a2740 1 -- 172.21.15.94:0/3048524274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228bd60 msgr2=0x5575d22949d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.795+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/3048524274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228bd60 0x5575d22949d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc09a3d0 tx=0x7f6cdc000bc0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.795+0000 7f6cf37a2740 1 -- 172.21.15.94:0/3048524274 shutdown_connections 2024-02-17T11:58:48.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.795+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/3048524274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d22952e0 0x5575d2292d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.795+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/3048524274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d228cad0 0x5575d2292810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.795+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/3048524274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228bd60 0x5575d22949d0 secure :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc09a3d0 tx=0x7f6cdc000bc0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.795+0000 7f6cf37a2740 1 -- 172.21.15.94:0/3048524274 >> 172.21.15.94:0/3048524274 conn(0x5575d2253550 msgr2=0x5575d228d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.796+0000 7f6cf37a2740 1 -- 172.21.15.94:0/3048524274 shutdown_connections 2024-02-17T11:58:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.796+0000 7f6cf37a2740 1 -- 172.21.15.94:0/3048524274 wait complete. 2024-02-17T11:58:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6cf37a2740 1 -- start start 2024-02-17T11:58:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6cf37a2740 1 Processor -- start 2024-02-17T11:58:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6cf37a2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228cad0 0x5575d229c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6cf37a2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d22952e0 0x5575d229a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6cea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228cad0 0x5575d229c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6cea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228cad0 0x5575d229c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46060/0 (socket says 172.21.15.94:46060) 2024-02-17T11:58:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6cea59c640 1 -- 172.21.15.94:0/542730259 learned_addr learned my addr 172.21.15.94:0/542730259 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:58:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d229acb0 0x5575d229b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6cf37a2740 1 -- 172.21.15.94:0/542730259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d230ca20 con 0x5575d228cad0 2024-02-17T11:58:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6cf37a2740 1 -- 172.21.15.94:0/542730259 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d230cb60 con 0x5575d229acb0 2024-02-17T11:58:48.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6cf37a2740 1 -- 172.21.15.94:0/542730259 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d230cca0 con 0x5575d22952e0 2024-02-17T11:58:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6cf37a2740 10 client.0 _ll_register_callbacks cb 0x7f6ce400fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:58:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6ce1d9b640 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d22952e0 0x5575d229a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.797+0000 7f6cead9d640 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d229acb0 0x5575d229b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cea59c640 1 -- 172.21.15.94:0/542730259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d22952e0 msgr2=0x5575d229a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cea59c640 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d22952e0 0x5575d229a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cea59c640 1 -- 172.21.15.94:0/542730259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d229acb0 msgr2=0x5575d229b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:58:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cea59c640 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d229acb0 0x5575d229b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cea59c640 1 -- 172.21.15.94:0/542730259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec1a2360 con 0x5575d228cad0 2024-02-17T11:58:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cead9d640 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d229acb0 0x5575d229b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:58:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6ce1d9b640 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d22952e0 0x5575d229a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:58:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cea59c640 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228cad0 0x5575d229c470 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0968f0 tx=0x7f6cdc0cb210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fbd93e92fb2f72 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:58:48.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cd1ffb640 1 -- 172.21.15.94:0/542730259 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0cb750 con 0x5575d228cad0 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cd1ffb640 1 -- 172.21.15.94:0/542730259 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6cdc09e170 con 0x5575d228cad0 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cf37a2740 1 -- 172.21.15.94:0/542730259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ce401d5a0 con 0x5575d228cad0 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cd1ffb640 1 -- 172.21.15.94:0/542730259 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cdc09e330 con 0x5575d228cad0 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cf37a2740 10 client.15669 fetch_fsmap learned FSMap version 21 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.798+0000 7f6cf37a2740 1 -- 172.21.15.94:0/542730259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5575d21b4540 con 0x5575d228cad0 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.799+0000 7f6cd1ffb640 1 -- 172.21.15.94:0/542730259 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cdc07f080 con 0x5575d228cad0 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.799+0000 7f6cd1ffb640 1 -- 172.21.15.94:0/542730259 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6cdc016020 con 0x5575d228cad0 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.799+0000 7f6cf37a2740 10 client.15669 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.799+0000 7f6cf37a2740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.799+0000 7f6cf37a2740 20 client.15669 populate_metadata read hostname 'smithi094' 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.799+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5575d230df20 0x5575d2310310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.799+0000 7f6cf37a2740 4 client.15669 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.799+0000 7f6cf37a2740 1 -- 172.21.15.94:0/542730259 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5575d1f30550 con 0x5575d230df20 2024-02-17T11:58:48.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.799+0000 7f6ce1d9b640 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5575d230df20 0x5575d2310310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:58:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.800+0000 7f6ce1d9b640 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5575d230df20 0x5575d2310310 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5aebe91902c7dd server_cookie=8eb684f0a6cbdcf3 in_seq=0 out_seq=0 2024-02-17T11:58:48.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.800+0000 7f6cd1ffb640 10 client.15669 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cd1ffb640 1 -- 172.21.15.94:0/542730259 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5575d1f30550 con 0x5575d230df20 2024-02-17T11:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cd1ffb640 10 client.15669 handle_command_reply: tid=0 2024-02-17T11:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cf37a2740 1 client.15669 shutdown 2024-02-17T11:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cf37a2740 1 -- 172.21.15.94:0/542730259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228cad0 msgr2=0x5575d229c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228cad0 0x5575d229c470 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0968f0 tx=0x7f6cdc0cb210 comp rx=0 tx=0).stop 2024-02-17T11:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cf37a2740 1 -- 172.21.15.94:0/542730259 shutdown_connections 2024-02-17T11:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5575d230df20 0x5575d2310310 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575d229acb0 0x5575d229b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cd1ffb640 0 client.15669 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:58:48.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575d22952e0 0x5575d229a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cf37a2740 1 --2- 172.21.15.94:0/542730259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575d228cad0 0x5575d229c470 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cf37a2740 1 -- 172.21.15.94:0/542730259 >> 172.21.15.94:0/542730259 conn(0x5575d2253550 msgr2=0x5575d2290af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cf37a2740 1 -- 172.21.15.94:0/542730259 shutdown_connections 2024-02-17T11:58:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cf37a2740 1 -- 172.21.15.94:0/542730259 wait complete. 2024-02-17T11:58:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.820+0000 7f6cf37a2740 20 client.15669 trim_cache size 0 max 16384 2024-02-17T11:58:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T11:58:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T11:58:48.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T11:58:48.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T11:58:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.822+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/2995821513 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6cd80a22c0 msgr2=0x7f6cd80a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.822+0000 7f6cf0bd0640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6cd80a22c0 0x7f6cd80a4780 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4011fd0 tx=0x7f6ce4011040 comp rx=0 tx=0).stop 2024-02-17T11:58:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.822+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/2995821513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cec0ff8f0 msgr2=0x7f6cec1954d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:58:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.822+0000 7f6cf0bd0640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cec0ff8f0 0x7f6cec1954d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c920 tx=0x7f6cdc00cde0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.822+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/2995821513 shutdown_connections 2024-02-17T11:58:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.822+0000 7f6cf0bd0640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6cd80a22c0 0x7f6cd80a4780 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.822+0000 7f6cf0bd0640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cec199d40 0x7f6cec19a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.822+0000 7f6cf0bd0640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cec106500 0x7f6cec195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.822+0000 7f6cf0bd0640 1 --2- 172.21.15.94:0/2995821513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cec0ff8f0 0x7f6cec1954d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:58:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.822+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/2995821513 >> 172.21.15.94:0/2995821513 conn(0x7f6cec076610 msgr2=0x7f6cec104b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:58:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.822+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/2995821513 shutdown_connections 2024-02-17T11:58:48.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:58:48.822+0000 7f6cf0bd0640 1 -- 172.21.15.94:0/2995821513 wait complete. 2024-02-17T11:58:50.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:49 smithi189 ceph-mon[36588]: pgmap v24: 97 pgs: 97 active+clean; 974 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 1.00k op/s 2024-02-17T11:58:50.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:49 smithi125 ceph-mon[36819]: pgmap v24: 97 pgs: 97 active+clean; 974 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 1.00k op/s 2024-02-17T11:58:50.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:49 smithi094 ceph-mon[32499]: pgmap v24: 97 pgs: 97 active+clean; 974 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 1.00k op/s 2024-02-17T11:58:50.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 30.0 s, 4075.6 tps, lat 24.174 ms stddev 25.295 2024-02-17T11:58:52.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:51 smithi189 ceph-mon[36588]: pgmap v25: 97 pgs: 97 active+clean; 974 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.9 MiB/s wr, 677 op/s 2024-02-17T11:58:52.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:51 smithi125 ceph-mon[36819]: pgmap v25: 97 pgs: 97 active+clean; 974 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.9 MiB/s wr, 677 op/s 2024-02-17T11:58:52.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:51 smithi094 ceph-mon[32499]: pgmap v25: 97 pgs: 97 active+clean; 974 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 8.9 MiB/s wr, 677 op/s 2024-02-17T11:58:54.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:53 smithi189 ceph-mon[36588]: pgmap v26: 97 pgs: 97 active+clean; 982 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 882 op/s 2024-02-17T11:58:54.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:53 smithi125 ceph-mon[36819]: pgmap v26: 97 pgs: 97 active+clean; 982 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 882 op/s 2024-02-17T11:58:54.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:53 smithi094 ceph-mon[32499]: pgmap v26: 97 pgs: 97 active+clean; 982 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 MiB/s wr, 882 op/s 2024-02-17T11:58:55.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 35.0 s, 4221.2 tps, lat 23.995 ms stddev 27.649 2024-02-17T11:58:56.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:55 smithi189 ceph-mon[36588]: pgmap v27: 97 pgs: 97 active+clean; 992 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 783 op/s 2024-02-17T11:58:56.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:55 smithi125 ceph-mon[36819]: pgmap v27: 97 pgs: 97 active+clean; 992 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 783 op/s 2024-02-17T11:58:56.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:55 smithi094 ceph-mon[32499]: pgmap v27: 97 pgs: 97 active+clean; 992 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 17 MiB/s wr, 783 op/s 2024-02-17T11:58:58.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:57 smithi189 ceph-mon[36588]: pgmap v28: 97 pgs: 97 active+clean; 992 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 705 op/s 2024-02-17T11:58:58.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:57 smithi125 ceph-mon[36819]: pgmap v28: 97 pgs: 97 active+clean; 992 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 705 op/s 2024-02-17T11:58:58.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:57 smithi094 ceph-mon[32499]: pgmap v28: 97 pgs: 97 active+clean; 992 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 705 op/s 2024-02-17T11:59:00.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:58:59 smithi189 ceph-mon[36588]: pgmap v29: 97 pgs: 97 active+clean; 996 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.06k op/s 2024-02-17T11:59:00.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:58:59 smithi125 ceph-mon[36819]: pgmap v29: 97 pgs: 97 active+clean; 996 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.06k op/s 2024-02-17T11:59:00.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:58:59 smithi094 ceph-mon[32499]: pgmap v29: 97 pgs: 97 active+clean; 996 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 20 MiB/s wr, 1.06k op/s 2024-02-17T11:59:00.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 40.0 s, 4330.4 tps, lat 23.114 ms stddev 23.309 2024-02-17T11:59:00.705 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:59:00 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:59:00.229+0000 7f2c1b35d700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:59:00.705 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:59:00 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:59:00.230+0000 7f2c1b35d700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:59:00.705 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:59:00 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:59:00.230+0000 7f2c1b35d700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:59:00.705 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:59:00 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:59:00.230+0000 7f2c1b35d700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:59:00.705 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 11:59:00 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T11:59:00.230+0000 7f2c1b35d700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-17T11:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.934+0000 7fec4a439640 1 -- 172.21.15.94:0/2881332729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec441086a0 msgr2=0x7fec44110270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.934+0000 7fec4a439640 1 --2- 172.21.15.94:0/2881332729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec441086a0 0x7fec44110270 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009cd0 tx=0x7fec3c02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.934+0000 7fec4a439640 1 -- 172.21.15.94:0/2881332729 shutdown_connections 2024-02-17T11:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.934+0000 7fec4a439640 1 --2- 172.21.15.94:0/2881332729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec441086a0 0x7fec44110270 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.934+0000 7fec4a439640 1 --2- 172.21.15.94:0/2881332729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec44103f80 0x7fec44107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.934+0000 7fec4a439640 1 --2- 172.21.15.94:0/2881332729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec441035d0 0x7fec441039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.934+0000 7fec4a439640 1 -- 172.21.15.94:0/2881332729 >> 172.21.15.94:0/2881332729 conn(0x7fec440fee30 msgr2=0x7fec44101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.934+0000 7fec4a439640 1 -- 172.21.15.94:0/2881332729 shutdown_connections 2024-02-17T11:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.934+0000 7fec4a439640 1 -- 172.21.15.94:0/2881332729 wait complete. 2024-02-17T11:59:01.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.934+0000 7fec4a439640 1 Processor -- start 2024-02-17T11:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.934+0000 7fec4a439640 1 -- start start 2024-02-17T11:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec4a439640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec441035d0 0x7fec4419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec4a439640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec441086a0 0x7fec4419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec4a439640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec441997f0 0x7fec44199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec4a439640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4419a2a0 con 0x7fec441035d0 2024-02-17T11:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec4a439640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4419a410 con 0x7fec441997f0 2024-02-17T11:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec4a439640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4419a580 con 0x7fec441086a0 2024-02-17T11:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec43fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec441035d0 0x7fec4419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec43fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec441035d0 0x7fec4419f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35226/0 (socket says 172.21.15.94:35226) 2024-02-17T11:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec43fff640 1 -- 172.21.15.94:0/1928425322 learned_addr learned my addr 172.21.15.94:0/1928425322 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec437fe640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec441086a0 0x7fec4419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec489d6640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec441997f0 0x7fec44199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec43fff640 1 -- 172.21.15.94:0/1928425322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec441086a0 msgr2=0x7fec4419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec43fff640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec441086a0 0x7fec4419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec43fff640 1 -- 172.21.15.94:0/1928425322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec441997f0 msgr2=0x7fec44199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec43fff640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec441997f0 0x7fec44199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec43fff640 1 -- 172.21.15.94:0/1928425322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c009900 con 0x7fec441035d0 2024-02-17T11:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec43fff640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec441035d0 0x7fec4419f630 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fec3400c970 tx=0x7fec3400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.935+0000 7fec417fa640 1 -- 172.21.15.94:0/1928425322 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34007bb0 con 0x7fec441035d0 2024-02-17T11:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.936+0000 7fec417fa640 1 -- 172.21.15.94:0/1928425322 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec34007d10 con 0x7fec441035d0 2024-02-17T11:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.936+0000 7fec4a439640 1 -- 172.21.15.94:0/1928425322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec441a64e0 con 0x7fec441035d0 2024-02-17T11:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.936+0000 7fec417fa640 1 -- 172.21.15.94:0/1928425322 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3400f450 con 0x7fec441035d0 2024-02-17T11:59:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.936+0000 7fec4a439640 1 -- 172.21.15.94:0/1928425322 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec441a6970 con 0x7fec441035d0 2024-02-17T11:59:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec417fa640 1 -- 172.21.15.94:0/1928425322 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fec34004f70 con 0x7fec441035d0 2024-02-17T11:59:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec4d007740 1 Processor -- start 2024-02-17T11:59:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec4d007740 1 -- start start 2024-02-17T11:59:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec4d007740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f467539ad0 0x55f467538820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec417fa640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec2c0a23f0 0x7fec2c0a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec4d007740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f4675391e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec4d007740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4675422e0 0x55f4675446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec4d007740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4674a79f0 con 0x55f467538d60 2024-02-17T11:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec4d007740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4674a4f20 con 0x55f4675422e0 2024-02-17T11:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec4d007740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4674a0700 con 0x55f467539ad0 2024-02-17T11:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec489d6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f467539ad0 0x55f467538820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f4675391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec43fff640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec2c0a23f0 0x7fec2c0a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec489d6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f467539ad0 0x55f467538820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:43756/0 (socket says 172.21.15.94:43756) 2024-02-17T11:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec489d6640 1 -- 172.21.15.94:0/3814929183 learned_addr learned my addr 172.21.15.94:0/3814929183 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec489d6640 1 --2- 172.21.15.94:0/3814929183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4675422e0 0x55f4675446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.938+0000 7fec417fa640 1 -- 172.21.15.94:0/1928425322 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fec340ce010 con 0x7fec441035d0 2024-02-17T11:59:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.939+0000 7fec437fe640 1 -- 172.21.15.94:0/3814929183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f467539ad0 msgr2=0x55f467538820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.939+0000 7fec437fe640 1 --2- 172.21.15.94:0/3814929183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f467539ad0 0x55f467538820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.939+0000 7fec437fe640 1 -- 172.21.15.94:0/3814929183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4675422e0 msgr2=0x55f4675446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.939+0000 7fec437fe640 1 --2- 172.21.15.94:0/3814929183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4675422e0 0x55f4675446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.939+0000 7fec437fe640 1 -- 172.21.15.94:0/3814929183 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4674367b0 con 0x55f467538d60 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.939+0000 7fec43fff640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec2c0a23f0 0x7fec2c0a48b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fec340cb630 tx=0x7fec34085040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.939+0000 7fec437fe640 1 --2- 172.21.15.94:0/3814929183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f4675391e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fec4419aaf0 tx=0x7fec38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99db8f79d41cf66 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.939+0000 7fec2a7fc640 1 -- 172.21.15.94:0/3814929183 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38004050 con 0x55f467538d60 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.939+0000 7fec2a7fc640 1 -- 172.21.15.94:0/3814929183 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec38005500 con 0x55f467538d60 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.939+0000 7fec2a7fc640 1 -- 172.21.15.94:0/3814929183 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38004bc0 con 0x55f467538d60 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.940+0000 7fec4d007740 1 -- 172.21.15.94:0/3814929183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 msgr2=0x55f4675391e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.940+0000 7fec4d007740 1 --2- 172.21.15.94:0/3814929183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f4675391e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fec4419aaf0 tx=0x7fec38011810 comp rx=0 tx=0).stop 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.940+0000 7fec4d007740 1 -- 172.21.15.94:0/3814929183 shutdown_connections 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.940+0000 7fec4d007740 1 --2- 172.21.15.94:0/3814929183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f4675422e0 0x55f4675446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.940+0000 7fec4d007740 1 --2- 172.21.15.94:0/3814929183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f4675391e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.940+0000 7fec4d007740 1 --2- 172.21.15.94:0/3814929183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f467539ad0 0x55f467538820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.940+0000 7fec4d007740 1 -- 172.21.15.94:0/3814929183 >> 172.21.15.94:0/3814929183 conn(0x55f467500550 msgr2=0x55f467533a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.940+0000 7fec4d007740 1 -- 172.21.15.94:0/3814929183 shutdown_connections 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.940+0000 7fec4d007740 1 -- 172.21.15.94:0/3814929183 wait complete. 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.940+0000 7fec4d007740 1 -- start start 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.940+0000 7fec4d007740 1 Processor -- start 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec4d007740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f46753f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec4d007740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467539ad0 0x55f46753f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec4d007740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 0x55f46753fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec4d007740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f467445bf0 con 0x55f467538d60 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec489d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f46753f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec489d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f46753f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35248/0 (socket says 172.21.15.94:35248) 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec489d6640 1 -- 172.21.15.94:0/296073678 learned_addr learned my addr 172.21.15.94:0/296073678 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec4d007740 1 -- 172.21.15.94:0/296073678 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f467174250 con 0x55f467539ad0 2024-02-17T11:59:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec4d007740 1 -- 172.21.15.94:0/296073678 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f467426200 con 0x55f4675422e0 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec437fe640 1 --2- 172.21.15.94:0/296073678 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467539ad0 0x55f46753f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec4d007740 10 client.0 _ll_register_callbacks cb 0x55f467494d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec489d6640 1 -- 172.21.15.94:0/296073678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 msgr2=0x55f46753fe00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec489d6640 1 --2- 172.21.15.94:0/296073678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 0x55f46753fe00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec489d6640 1 -- 172.21.15.94:0/296073678 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467539ad0 msgr2=0x55f46753f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec489d6640 1 --2- 172.21.15.94:0/296073678 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467539ad0 0x55f46753f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec489d6640 1 -- 172.21.15.94:0/296073678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec38009850 con 0x55f467538d60 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.941+0000 7fec437fe640 1 --2- 172.21.15.94:0/296073678 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467539ad0 0x55f46753f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.942+0000 7fec489d6640 1 --2- 172.21.15.94:0/296073678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f46753f380 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fec3c037040 tx=0x7fec3c034d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2592cbf0facbf081 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.942+0000 7fec237fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.942+0000 7fec237fe640 1 -- 172.21.15.94:0/296073678 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0328c0 con 0x55f467538d60 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.942+0000 7fec4d007740 1 -- 172.21.15.94:0/296073678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4671a0600 con 0x55f467538d60 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.942+0000 7fec237fe640 1 -- 172.21.15.94:0/296073678 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fec3c032a20 con 0x55f467538d60 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.942+0000 7fec237fe640 1 -- 172.21.15.94:0/296073678 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec3c003970 con 0x55f467538d60 2024-02-17T11:59:01.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.942+0000 7fec4d007740 10 client.15687 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.942+0000 7fec4d007740 1 -- 172.21.15.94:0/296073678 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55f46749ed80 con 0x55f467538d60 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.942+0000 7fec237fe640 1 -- 172.21.15.94:0/296073678 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f4671a0600 con 0x55f467538d60 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.943+0000 7fec237fe640 1 -- 172.21.15.94:0/296073678 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fec3c040430 con 0x55f467538d60 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.943+0000 7fec4d007740 10 client.15687 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.943+0000 7fec4d007740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.943+0000 7fec4d007740 20 client.15687 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.943+0000 7fec4d007740 1 --2- 172.21.15.94:0/296073678 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f467552640 0x55f467554a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.943+0000 7fec4d007740 4 client.15687 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.943+0000 7fec4d007740 1 -- 172.21.15.94:0/296073678 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4671f11b0 con 0x55f467552640 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.943+0000 7fec437fe640 1 --2- 172.21.15.94:0/296073678 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f467552640 0x55f467554a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.943+0000 7fec437fe640 1 --2- 172.21.15.94:0/296073678 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f467552640 0x55f467554a30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44541f76cc6c0870 server_cookie=250925ce62fe843 in_seq=0 out_seq=0 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.943+0000 7fec237fe640 10 client.15687 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec237fe640 1 -- 172.21.15.94:0/296073678 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4671f11b0 con 0x55f467552640 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec237fe640 10 client.15687 handle_command_reply: tid=0 2024-02-17T11:59:01.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec4d007740 1 client.15687 shutdown 2024-02-17T11:59:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec4d007740 1 -- 172.21.15.94:0/296073678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 msgr2=0x55f46753f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec4d007740 1 --2- 172.21.15.94:0/296073678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f46753f380 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fec3c037040 tx=0x7fec3c034d10 comp rx=0 tx=0).stop 2024-02-17T11:59:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec4d007740 1 -- 172.21.15.94:0/296073678 shutdown_connections 2024-02-17T11:59:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec4d007740 1 --2- 172.21.15.94:0/296073678 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f467552640 0x55f467554a30 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec4d007740 1 --2- 172.21.15.94:0/296073678 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 0x55f46753fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec4d007740 1 --2- 172.21.15.94:0/296073678 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467539ad0 0x55f46753f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec4d007740 1 --2- 172.21.15.94:0/296073678 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f46753f380 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec237fe640 0 client.15687 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec4d007740 1 -- 172.21.15.94:0/296073678 >> 172.21.15.94:0/296073678 conn(0x55f467500550 msgr2=0x55f4675357a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec4d007740 1 -- 172.21.15.94:0/296073678 shutdown_connections 2024-02-17T11:59:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec4d007740 1 -- 172.21.15.94:0/296073678 wait complete. 2024-02-17T11:59:01.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.944+0000 7fec4d007740 20 client.15687 trim_cache size 0 max 16384 2024-02-17T11:59:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.958+0000 7fec4d007740 1 Processor -- start 2024-02-17T11:59:01.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.958+0000 7fec4d007740 1 -- start start 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec4d007740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467538d60 0x55f467541ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec4d007740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467539ad0 0x55f467549100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec4d007740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 0x55f467549640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec4d007740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f467534840 con 0x55f467539ad0 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec4d007740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46753aff0 con 0x55f467538d60 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec4d007740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f467552440 con 0x55f4675422e0 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467539ad0 0x55f467549100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec489d6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467538d60 0x55f467541ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec489d6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467538d60 0x55f467541ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:51714/0 (socket says 172.21.15.94:51714) 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467539ad0 0x55f467549100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35252/0 (socket says 172.21.15.94:35252) 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec489d6640 1 -- 172.21.15.94:0/3274007807 learned_addr learned my addr 172.21.15.94:0/3274007807 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec489d6640 1 --2- 172.21.15.94:0/3274007807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 0x55f467549640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec489d6640 1 -- 172.21.15.94:0/3274007807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 msgr2=0x55f467549640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:01.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec489d6640 1 --2- 172.21.15.94:0/3274007807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 0x55f467549640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec489d6640 1 -- 172.21.15.94:0/3274007807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467539ad0 msgr2=0x55f467549100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec489d6640 1 --2- 172.21.15.94:0/3274007807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467539ad0 0x55f467549100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec489d6640 1 -- 172.21.15.94:0/3274007807 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f46743b7b0 con 0x55f467538d60 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec437fe640 1 --2- 172.21.15.94:0/3274007807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467539ad0 0x55f467549100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec489d6640 1 --2- 172.21.15.94:0/3274007807 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467538d60 0x55f467541ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec3c02c630 tx=0x7fec3c02c0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=518d1eb2c9835cf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec29ffb640 1 -- 172.21.15.94:0/3274007807 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c040430 con 0x55f467538d60 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec29ffb640 1 -- 172.21.15.94:0/3274007807 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec3c040590 con 0x55f467538d60 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.959+0000 7fec29ffb640 1 -- 172.21.15.94:0/3274007807 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c004260 con 0x55f467538d60 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.960+0000 7fec4d007740 1 -- 172.21.15.94:0/3274007807 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467538d60 msgr2=0x55f467541ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.960+0000 7fec4d007740 1 --2- 172.21.15.94:0/3274007807 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467538d60 0x55f467541ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec3c02c630 tx=0x7fec3c02c0f0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.960+0000 7fec4d007740 1 -- 172.21.15.94:0/3274007807 shutdown_connections 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.960+0000 7fec4d007740 1 --2- 172.21.15.94:0/3274007807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 0x55f467549640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.960+0000 7fec4d007740 1 --2- 172.21.15.94:0/3274007807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467539ad0 0x55f467549100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.960+0000 7fec4d007740 1 --2- 172.21.15.94:0/3274007807 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467538d60 0x55f467541ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.960+0000 7fec4d007740 1 -- 172.21.15.94:0/3274007807 >> 172.21.15.94:0/3274007807 conn(0x55f467500550 msgr2=0x55f467533580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.960+0000 7fec4d007740 1 -- 172.21.15.94:0/3274007807 shutdown_connections 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.960+0000 7fec4d007740 1 -- 172.21.15.94:0/3274007807 wait complete. 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.960+0000 7fec4d007740 1 -- start start 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec4d007740 1 Processor -- start 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec4d007740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f467549110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec4d007740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467539ad0 0x55f467549650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec4d007740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 0x55f46753b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec4d007740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46753c080 con 0x55f467538d60 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec4d007740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46753c1f0 con 0x55f467539ad0 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec4d007740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f46753c3b0 con 0x55f4675422e0 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec4d007740 10 client.0 _ll_register_callbacks cb 0x7fec380097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec489d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f467549110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec489d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f467549110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35258/0 (socket says 172.21.15.94:35258) 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec489d6640 1 -- 172.21.15.94:0/2885481933 learned_addr learned my addr 172.21.15.94:0/2885481933 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec437fe640 1 --2- 172.21.15.94:0/2885481933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467539ad0 0x55f467549650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec489d6640 1 --2- 172.21.15.94:0/2885481933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 0x55f46753b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec489d6640 1 -- 172.21.15.94:0/2885481933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 msgr2=0x55f46753b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec489d6640 1 --2- 172.21.15.94:0/2885481933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 0x55f46753b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec489d6640 1 -- 172.21.15.94:0/2885481933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467539ad0 msgr2=0x55f467549650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec489d6640 1 --2- 172.21.15.94:0/2885481933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467539ad0 0x55f467549650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec489d6640 1 -- 172.21.15.94:0/2885481933 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4674367b0 con 0x55f467538d60 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec489d6640 1 --2- 172.21.15.94:0/2885481933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f467549110 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fec3c033000 tx=0x7fec3c032960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc93ec52d4c494e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.961+0000 7fec237fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec237fe640 1 -- 172.21.15.94:0/2885481933 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c040430 con 0x55f467538d60 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec237fe640 1 -- 172.21.15.94:0/2885481933 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fec3c003d80 con 0x55f467538d60 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec4d007740 1 -- 172.21.15.94:0/2885481933 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec38005500 con 0x55f467538d60 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec237fe640 1 -- 172.21.15.94:0/2885481933 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec3c004000 con 0x55f467538d60 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec4d007740 10 client.15693 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec4d007740 1 -- 172.21.15.94:0/2885481933 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55f46749ed80 con 0x55f467538d60 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec237fe640 1 -- 172.21.15.94:0/2885481933 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec3c071080 con 0x55f467538d60 2024-02-17T11:59:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec237fe640 1 -- 172.21.15.94:0/2885481933 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fec3c061070 con 0x55f467538d60 2024-02-17T11:59:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec4d007740 10 client.15693 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec4d007740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec4d007740 20 client.15693 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec4d007740 1 --2- 172.21.15.94:0/2885481933 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f4675a98e0 0x55f4675abd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec4d007740 4 client.15693 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T11:59:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.962+0000 7fec4d007740 1 -- 172.21.15.94:0/2885481933 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f4671dd550 con 0x55f4675a98e0 2024-02-17T11:59:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.963+0000 7fec437fe640 1 --2- 172.21.15.94:0/2885481933 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f4675a98e0 0x55f4675abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.963+0000 7fec237fe640 10 client.15693 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:01.963+0000 7fec437fe640 1 --2- 172.21.15.94:0/2885481933 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f4675a98e0 0x55f4675abd00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c97321a225970bc server_cookie=5add4ce982551bcc in_seq=0 out_seq=0 2024-02-17T11:59:02.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T11:59:02.065 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T11:59:02.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:01 smithi189 ceph-mon[36588]: pgmap v30: 97 pgs: 97 active+clean; 996 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 702 op/s 2024-02-17T11:59:02.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:01 smithi125 ceph-mon[36819]: pgmap v30: 97 pgs: 97 active+clean; 996 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 702 op/s 2024-02-17T11:59:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.190+0000 7f76358dd640 1 -- 172.21.15.94:0/3205708796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7630101380 msgr2=0x7f7630101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.190+0000 7f76358dd640 1 --2- 172.21.15.94:0/3205708796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7630101380 0x7f7630101760 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7620009930 tx=0x7f762002bc30 comp rx=0 tx=0).stop 2024-02-17T11:59:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.190+0000 7f76358dd640 1 -- 172.21.15.94:0/3205708796 shutdown_connections 2024-02-17T11:59:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.190+0000 7f76358dd640 1 --2- 172.21.15.94:0/3205708796 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7630106450 0x7f763010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.190+0000 7f76358dd640 1 --2- 172.21.15.94:0/3205708796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7630101d30 0x7f7630105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.190+0000 7f76358dd640 1 --2- 172.21.15.94:0/3205708796 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7630101380 0x7f7630101760 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.190+0000 7f76358dd640 1 -- 172.21.15.94:0/3205708796 >> 172.21.15.94:0/3205708796 conn(0x7f76300fcc20 msgr2=0x7f76300ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.190+0000 7f76358dd640 1 -- 172.21.15.94:0/3205708796 shutdown_connections 2024-02-17T11:59:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.190+0000 7f76358dd640 1 -- 172.21.15.94:0/3205708796 wait complete. 2024-02-17T11:59:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.191+0000 7f76358dd640 1 Processor -- start 2024-02-17T11:59:02.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.191+0000 7f76358dd640 1 -- start start 2024-02-17T11:59:02.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.191+0000 7f76358dd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7630101380 0x7f763019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.191+0000 7f76358dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7630106450 0x7f763019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.191+0000 7f76358dd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7630197620 0x7f7630197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.191+0000 7f76358dd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7630198010 con 0x7f7630106450 2024-02-17T11:59:02.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.191+0000 7f76358dd640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7630198180 con 0x7f7630197620 2024-02-17T11:59:02.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.191+0000 7f76358dd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7630198340 con 0x7f7630101380 2024-02-17T11:59:02.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76267fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7630106450 0x7f763019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76267fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7630106450 0x7f763019d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35268/0 (socket says 172.21.15.94:35268) 2024-02-17T11:59:02.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f762f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7630197620 0x7f7630197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.192 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76267fc640 1 -- 172.21.15.94:0/3002646168 learned_addr learned my addr 172.21.15.94:0/3002646168 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f762effd640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7630101380 0x7f763019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76267fc640 1 -- 172.21.15.94:0/3002646168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7630101380 msgr2=0x7f763019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76267fc640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7630101380 0x7f763019d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76267fc640 1 -- 172.21.15.94:0/3002646168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7630197620 msgr2=0x7f7630197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76267fc640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7630197620 0x7f7630197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76267fc640 1 -- 172.21.15.94:0/3002646168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7620009560 con 0x7f7630106450 2024-02-17T11:59:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f762f7fe640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7630197620 0x7f7630197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76267fc640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7630106450 0x7f763019d9b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f761800e9f0 tx=0x7f761800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76348db640 1 -- 172.21.15.94:0/3002646168 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180096c0 con 0x7f7630106450 2024-02-17T11:59:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76348db640 1 -- 172.21.15.94:0/3002646168 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7618005500 con 0x7f7630106450 2024-02-17T11:59:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76358dd640 1 -- 172.21.15.94:0/3002646168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76301a4330 con 0x7f7630106450 2024-02-17T11:59:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76348db640 1 -- 172.21.15.94:0/3002646168 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618010430 con 0x7f7630106450 2024-02-17T11:59:02.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.192+0000 7f76358dd640 1 -- 172.21.15.94:0/3002646168 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76301a47f0 con 0x7f7630106450 2024-02-17T11:59:02.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.195+0000 7f76348db640 1 -- 172.21.15.94:0/3002646168 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f7618005020 con 0x7f7630106450 2024-02-17T11:59:02.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.195+0000 7f7638582740 1 Processor -- start 2024-02-17T11:59:02.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.195+0000 7f7638582740 1 -- start start 2024-02-17T11:59:02.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.195+0000 7f7638582740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f81690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.195+0000 7f7638582740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.195+0000 7f7638582740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f8cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.195+0000 7f76348db640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f761c0a2350 0x7f761c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.195+0000 7f7638582740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074ef0280 con 0x55f074f812b0 2024-02-17T11:59:02.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.195+0000 7f7638582740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074eed7b0 con 0x55f074f81bd0 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.195+0000 7f7638582740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074ee8f90 con 0x55f074f8a7b0 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f762effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f762f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f81690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f762effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:51754/0 (socket says 172.21.15.94:51754) 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f762effd640 1 -- 172.21.15.94:0/50718086 learned_addr learned my addr 172.21.15.94:0/50718086 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f76267fc640 1 --2- 172.21.15.94:0/50718086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f8cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f762f7fe640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f761c0a2350 0x7f761c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f76348db640 1 -- 172.21.15.94:0/3002646168 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7618014070 con 0x7f7630106450 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f762f7fe640 1 -- 172.21.15.94:0/50718086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 msgr2=0x55f074f8cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f762f7fe640 1 --2- 172.21.15.94:0/50718086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f8cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f762f7fe640 1 -- 172.21.15.94:0/50718086 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 msgr2=0x55f074f8a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f762f7fe640 1 --2- 172.21.15.94:0/50718086 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f762f7fe640 1 -- 172.21.15.94:0/50718086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f074ea7570 con 0x55f074f812b0 2024-02-17T11:59:02.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f76267fc640 1 --2- 172.21.15.94:0/50718086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f8cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f762effd640 1 --2- 172.21.15.94:0/50718086 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f762f7fe640 1 --2- 172.21.15.94:0/50718086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f81690 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7628009fd0 tx=0x7f762800f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde99199ab0958a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f762f7fe640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f761c0a2350 0x7f761c0a4810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76280099e0 tx=0x7f762801e000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f7625ffb640 1 -- 172.21.15.94:0/50718086 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628011bd0 con 0x55f074f812b0 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f7625ffb640 1 -- 172.21.15.94:0/50718086 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7628011540 con 0x55f074f812b0 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.196+0000 7f7625ffb640 1 -- 172.21.15.94:0/50718086 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762801e660 con 0x55f074f812b0 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.197+0000 7f7638582740 1 -- 172.21.15.94:0/50718086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 msgr2=0x55f074f81690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.197+0000 7f7638582740 1 --2- 172.21.15.94:0/50718086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f81690 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7628009fd0 tx=0x7f762800f950 comp rx=0 tx=0).stop 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.197+0000 7f7638582740 1 -- 172.21.15.94:0/50718086 shutdown_connections 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.197+0000 7f7638582740 1 --2- 172.21.15.94:0/50718086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f8cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.197+0000 7f7638582740 1 --2- 172.21.15.94:0/50718086 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.197+0000 7f7638582740 1 --2- 172.21.15.94:0/50718086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f81690 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.197+0000 7f7638582740 1 -- 172.21.15.94:0/50718086 >> 172.21.15.94:0/50718086 conn(0x55f074f7b990 msgr2=0x55f074f7bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.198 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.197+0000 7f7638582740 1 -- 172.21.15.94:0/50718086 shutdown_connections 2024-02-17T11:59:02.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.197+0000 7f7638582740 1 -- 172.21.15.94:0/50718086 wait complete. 2024-02-17T11:59:02.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.198+0000 7f7638582740 1 -- start start 2024-02-17T11:59:02.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.198+0000 7f7638582740 1 Processor -- start 2024-02-17T11:59:02.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.198+0000 7f7638582740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f8a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.198+0000 7f7638582740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f87600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.198+0000 7f7638582740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f87d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.198+0000 7f7638582740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074bbc430 con 0x55f074f812b0 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.198+0000 7f7638582740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074b78e00 con 0x55f074f81bd0 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.198+0000 7f7638582740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074e6dfd0 con 0x55f074f8a7b0 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.198+0000 7f7638582740 10 client.0 _ll_register_callbacks cb 0x55f074ec3fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.198+0000 7f762effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f8a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.198+0000 7f762effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f8a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35288/0 (socket says 172.21.15.94:35288) 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.198+0000 7f762effd640 1 -- 172.21.15.94:0/1911853616 learned_addr learned my addr 172.21.15.94:0/1911853616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.198+0000 7f762f7fe640 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f87d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f76267fc640 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f87600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f762f7fe640 1 -- 172.21.15.94:0/1911853616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 msgr2=0x55f074f87600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f762f7fe640 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f87600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f762f7fe640 1 -- 172.21.15.94:0/1911853616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 msgr2=0x55f074f8a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f762f7fe640 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f8a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f762f7fe640 1 -- 172.21.15.94:0/1911853616 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7628019040 con 0x55f074f8a7b0 2024-02-17T11:59:02.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f762effd640 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f8a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f762f7fe640 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f87d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7628027fd0 tx=0x7f7628026d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7be15c442195680 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f760effd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f760effd640 1 -- 172.21.15.94:0/1911853616 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628038ea0 con 0x55f074f8a7b0 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f760effd640 1 -- 172.21.15.94:0/1911853616 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7628004c00 con 0x55f074f8a7b0 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f7638582740 1 -- 172.21.15.94:0/1911853616 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f074be87e0 con 0x55f074f8a7b0 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f760effd640 1 -- 172.21.15.94:0/1911853616 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7628004dc0 con 0x55f074f8a7b0 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f7638582740 10 client.25234 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.199+0000 7f7638582740 1 -- 172.21.15.94:0/1911853616 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55f074ee7610 con 0x55f074f8a7b0 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.200+0000 7f760effd640 1 -- 172.21.15.94:0/1911853616 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f074be87e0 con 0x55f074f8a7b0 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.200+0000 7f760effd640 1 -- 172.21.15.94:0/1911853616 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f762801e890 con 0x55f074f8a7b0 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.200+0000 7f7638582740 10 client.25234 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.200+0000 7f7638582740 10 client.25234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.200+0000 7f7638582740 20 client.25234 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:02.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.200+0000 7f7638582740 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f074f9a910 0x55f074f9cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.200+0000 7f7638582740 4 client.25234 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:02.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.200+0000 7f7638582740 1 -- 172.21.15.94:0/1911853616 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f074c25720 con 0x55f074f9a910 2024-02-17T11:59:02.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.200+0000 7f762effd640 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f074f9a910 0x55f074f9cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.201+0000 7f762effd640 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f074f9a910 0x55f074f9cd00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c667646ec24aef71 server_cookie=2eded9bf15d86196 in_seq=0 out_seq=0 2024-02-17T11:59:02.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.201+0000 7f760effd640 10 client.25234 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:02.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:01 smithi094 ceph-mon[32499]: pgmap v30: 97 pgs: 97 active+clean; 996 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 14 MiB/s wr, 702 op/s 2024-02-17T11:59:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.371+0000 7fec237fe640 1 -- 172.21.15.94:0/2885481933 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f4671dd550 con 0x55f4675a98e0 2024-02-17T11:59:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.371+0000 7fec237fe640 10 client.15693 handle_command_reply: tid=0 2024-02-17T11:59:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.371+0000 7fec4d007740 1 client.15693 shutdown 2024-02-17T11:59:02.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.371+0000 7fec4d007740 1 -- 172.21.15.94:0/2885481933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 msgr2=0x55f467549110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.371+0000 7fec4d007740 1 --2- 172.21.15.94:0/2885481933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f467549110 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fec3c033000 tx=0x7fec3c032960 comp rx=0 tx=0).stop 2024-02-17T11:59:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.371+0000 7fec4d007740 1 -- 172.21.15.94:0/2885481933 shutdown_connections 2024-02-17T11:59:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.371+0000 7fec4d007740 1 --2- 172.21.15.94:0/2885481933 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f4675a98e0 0x55f4675abd00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.371+0000 7fec4d007740 1 --2- 172.21.15.94:0/2885481933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4675422e0 0x55f46753b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.371+0000 7fec4d007740 1 --2- 172.21.15.94:0/2885481933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f467539ad0 0x55f467549650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.371+0000 7fec237fe640 0 client.15693 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.371+0000 7fec4d007740 1 --2- 172.21.15.94:0/2885481933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f467538d60 0x55f467549110 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.371+0000 7fec4d007740 1 -- 172.21.15.94:0/2885481933 >> 172.21.15.94:0/2885481933 conn(0x55f467500550 msgr2=0x55f46753dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.371+0000 7f760effd640 1 -- 172.21.15.94:0/1911853616 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f074c25720 con 0x55f074f9a910 2024-02-17T11:59:02.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.371+0000 7f760effd640 10 client.25234 handle_command_reply: tid=0 2024-02-17T11:59:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.371+0000 7fec4d007740 1 -- 172.21.15.94:0/2885481933 shutdown_connections 2024-02-17T11:59:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4d007740 1 -- 172.21.15.94:0/2885481933 wait complete. 2024-02-17T11:59:02.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.371+0000 7f7638582740 1 client.25234 shutdown 2024-02-17T11:59:02.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4d007740 20 client.15693 trim_cache size 0 max 16384 2024-02-17T11:59:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.372+0000 7f7638582740 1 -- 172.21.15.94:0/1911853616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 msgr2=0x55f074f87d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.372+0000 7f7638582740 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f87d40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7628027fd0 tx=0x7f7628026d80 comp rx=0 tx=0).stop 2024-02-17T11:59:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T11:59:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T11:59:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T11:59:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T11:59:02.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.372+0000 7f7638582740 1 -- 172.21.15.94:0/1911853616 shutdown_connections 2024-02-17T11:59:02.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.372+0000 7f7638582740 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f074f9a910 0x55f074f9cd00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.372+0000 7f7638582740 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f87d40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.372+0000 7f7638582740 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f87600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.372+0000 7f7638582740 1 --2- 172.21.15.94:0/1911853616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f8a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.372+0000 7f760effd640 0 client.25234 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:02.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.372+0000 7f7638582740 1 -- 172.21.15.94:0/1911853616 >> 172.21.15.94:0/1911853616 conn(0x55f074f7b990 msgr2=0x55f074f7d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.372+0000 7f7638582740 1 -- 172.21.15.94:0/1911853616 shutdown_connections 2024-02-17T11:59:02.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.372+0000 7f7638582740 1 -- 172.21.15.94:0/1911853616 wait complete. 2024-02-17T11:59:02.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.372+0000 7f7638582740 20 client.25234 trim_cache size 0 max 16384 2024-02-17T11:59:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4a439640 1 -- 172.21.15.94:0/1928425322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec2c0a23f0 msgr2=0x7fec2c0a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4a439640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec2c0a23f0 0x7fec2c0a48b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fec340cb630 tx=0x7fec34085040 comp rx=0 tx=0).stop 2024-02-17T11:59:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4a439640 1 -- 172.21.15.94:0/1928425322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec441035d0 msgr2=0x7fec4419f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4a439640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec441035d0 0x7fec4419f630 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fec3400c970 tx=0x7fec3400ce30 comp rx=0 tx=0).stop 2024-02-17T11:59:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4a439640 1 -- 172.21.15.94:0/1928425322 shutdown_connections 2024-02-17T11:59:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4a439640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec2c0a23f0 0x7fec2c0a48b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4a439640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec441997f0 0x7fec44199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4a439640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec441086a0 0x7fec4419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4a439640 1 --2- 172.21.15.94:0/1928425322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec441035d0 0x7fec4419f630 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4a439640 1 -- 172.21.15.94:0/1928425322 >> 172.21.15.94:0/1928425322 conn(0x7fec440fee30 msgr2=0x7fec440ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4a439640 1 -- 172.21.15.94:0/1928425322 shutdown_connections 2024-02-17T11:59:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:02.372+0000 7fec4a439640 1 -- 172.21.15.94:0/1928425322 wait complete. 2024-02-17T11:59:02.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f7638582740 1 Processor -- start 2024-02-17T11:59:02.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f7638582740 1 -- start start 2024-02-17T11:59:02.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f7638582740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f812b0 0x55f074f89d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f7638582740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f7638582740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f8a7b0 0x55f074f91600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f7638582740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074f80f50 con 0x55f074f8a7b0 2024-02-17T11:59:02.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f7638582740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074f770f0 con 0x55f074f81bd0 2024-02-17T11:59:02.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f7638582740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074f9a760 con 0x55f074f812b0 2024-02-17T11:59:02.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f762f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f8a7b0 0x55f074f91600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f762f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f8a7b0 0x55f074f91600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35290/0 (socket says 172.21.15.94:35290) 2024-02-17T11:59:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f762f7fe640 1 -- 172.21.15.94:0/4286825526 learned_addr learned my addr 172.21.15.94:0/4286825526 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f762effd640 1 --2- 172.21.15.94:0/4286825526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f812b0 0x55f074f89d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f76267fc640 1 --2- 172.21.15.94:0/4286825526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f762f7fe640 1 -- 172.21.15.94:0/4286825526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f812b0 msgr2=0x55f074f89d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f762f7fe640 1 --2- 172.21.15.94:0/4286825526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f812b0 0x55f074f89d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f762f7fe640 1 -- 172.21.15.94:0/4286825526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 msgr2=0x55f074f8a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f762f7fe640 1 --2- 172.21.15.94:0/4286825526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.390+0000 7f762f7fe640 1 -- 172.21.15.94:0/4286825526 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f074e83ac0 con 0x55f074f8a7b0 2024-02-17T11:59:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.391+0000 7f762f7fe640 1 --2- 172.21.15.94:0/4286825526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f8a7b0 0x55f074f91600 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f762800f990 tx=0x7f7628005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ef96346f2209bf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.391+0000 7f76257fa640 1 -- 172.21.15.94:0/4286825526 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628009570 con 0x55f074f8a7b0 2024-02-17T11:59:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.391+0000 7f76257fa640 1 -- 172.21.15.94:0/4286825526 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f762801edc0 con 0x55f074f8a7b0 2024-02-17T11:59:02.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.391+0000 7f76257fa640 1 -- 172.21.15.94:0/4286825526 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628030330 con 0x55f074f8a7b0 2024-02-17T11:59:02.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.391+0000 7f7638582740 1 -- 172.21.15.94:0/4286825526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f8a7b0 msgr2=0x55f074f91600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.391+0000 7f7638582740 1 --2- 172.21.15.94:0/4286825526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f8a7b0 0x55f074f91600 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f762800f990 tx=0x7f7628005c50 comp rx=0 tx=0).stop 2024-02-17T11:59:02.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.391+0000 7f7638582740 1 -- 172.21.15.94:0/4286825526 shutdown_connections 2024-02-17T11:59:02.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.391+0000 7f7638582740 1 --2- 172.21.15.94:0/4286825526 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f8a7b0 0x55f074f91600 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.391+0000 7f7638582740 1 --2- 172.21.15.94:0/4286825526 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.391+0000 7f7638582740 1 --2- 172.21.15.94:0/4286825526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f812b0 0x55f074f89d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.392+0000 7f7638582740 1 -- 172.21.15.94:0/4286825526 >> 172.21.15.94:0/4286825526 conn(0x55f074f7b990 msgr2=0x55f074f82910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.392+0000 7f7638582740 1 -- 172.21.15.94:0/4286825526 shutdown_connections 2024-02-17T11:59:02.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.392+0000 7f7638582740 1 -- 172.21.15.94:0/4286825526 wait complete. 2024-02-17T11:59:02.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.392+0000 7f7638582740 1 -- start start 2024-02-17T11:59:02.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.392+0000 7f7638582740 1 Processor -- start 2024-02-17T11:59:02.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.392+0000 7f7638582740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f91a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.392+0000 7f7638582740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.392+0000 7f7638582740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f90490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.392+0000 7f7638582740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074f90b90 con 0x55f074f812b0 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.392+0000 7f7638582740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074ff0df0 con 0x55f074f81bd0 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.392+0000 7f7638582740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f074ff0f30 con 0x55f074f8a7b0 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.392+0000 7f7638582740 10 client.0 _ll_register_callbacks cb 0x7f76280116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f76267fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f762effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f91a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f76267fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:51776/0 (socket says 172.21.15.94:51776) 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f76267fc640 1 -- 172.21.15.94:0/1502935600 learned_addr learned my addr 172.21.15.94:0/1502935600 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f762f7fe640 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f90490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f762f7fe640 1 -- 172.21.15.94:0/1502935600 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 msgr2=0x55f074f8fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f762f7fe640 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f762f7fe640 1 -- 172.21.15.94:0/1502935600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 msgr2=0x55f074f91a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f762f7fe640 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f91a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f762f7fe640 1 -- 172.21.15.94:0/1502935600 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f074ee7610 con 0x55f074f8a7b0 2024-02-17T11:59:02.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f762effd640 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f91a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:02.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f76267fc640 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:02.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f762f7fe640 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f90490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f762800a010 tx=0x7f762801e710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=602df4db5197cb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f760effd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T11:59:02.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f760effd640 1 -- 172.21.15.94:0/1502935600 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628004c00 con 0x55f074f8a7b0 2024-02-17T11:59:02.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f760effd640 1 -- 172.21.15.94:0/1502935600 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7628004d60 con 0x55f074f8a7b0 2024-02-17T11:59:02.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f7638582740 1 -- 172.21.15.94:0/1502935600 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7628011540 con 0x55f074f8a7b0 2024-02-17T11:59:02.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f760effd640 1 -- 172.21.15.94:0/1502935600 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f762801ea50 con 0x55f074f8a7b0 2024-02-17T11:59:02.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f7638582740 10 client.25240 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:02.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.393+0000 7f7638582740 1 -- 172.21.15.94:0/1502935600 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55f074ea96a0 con 0x55f074f8a7b0 2024-02-17T11:59:02.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.394+0000 7f760effd640 1 -- 172.21.15.94:0/1502935600 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7628061020 con 0x55f074f8a7b0 2024-02-17T11:59:02.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.394+0000 7f760effd640 1 -- 172.21.15.94:0/1502935600 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7628005040 con 0x55f074f8a7b0 2024-02-17T11:59:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.394+0000 7f7638582740 10 client.25240 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.394+0000 7f7638582740 10 client.25240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.394+0000 7f7638582740 20 client.25240 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.394+0000 7f7638582740 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f074ff2160 0x55f074ff4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.394+0000 7f7638582740 4 client.25240 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T11:59:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.394+0000 7f7638582740 1 -- 172.21.15.94:0/1502935600 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f074c74890 con 0x55f074ff2160 2024-02-17T11:59:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.394+0000 7f762effd640 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f074ff2160 0x55f074ff4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.394+0000 7f762effd640 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f074ff2160 0x55f074ff4550 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6e7131b1dd227f server_cookie=cdd9c3e9f036c348 in_seq=0 out_seq=0 2024-02-17T11:59:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.394+0000 7f760effd640 10 client.25240 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f760effd640 1 -- 172.21.15.94:0/1502935600 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f074c74890 con 0x55f074ff2160 2024-02-17T11:59:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f760effd640 10 client.25240 handle_command_reply: tid=0 2024-02-17T11:59:02.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f7638582740 1 client.25240 shutdown 2024-02-17T11:59:02.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f7638582740 1 -- 172.21.15.94:0/1502935600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 msgr2=0x55f074f90490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f7638582740 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f90490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f762800a010 tx=0x7f762801e710 comp rx=0 tx=0).stop 2024-02-17T11:59:02.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f7638582740 1 -- 172.21.15.94:0/1502935600 shutdown_connections 2024-02-17T11:59:02.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f7638582740 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f074ff2160 0x55f074ff4550 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f7638582740 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f074f8a7b0 0x55f074f90490 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f760effd640 0 client.25240 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:02.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f7638582740 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f074f81bd0 0x55f074f8fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f7638582740 1 --2- 172.21.15.94:0/1502935600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f074f812b0 0x55f074f91a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f7638582740 1 -- 172.21.15.94:0/1502935600 >> 172.21.15.94:0/1502935600 conn(0x55f074f7b990 msgr2=0x55f074f85e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f7638582740 1 -- 172.21.15.94:0/1502935600 shutdown_connections 2024-02-17T11:59:02.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f7638582740 1 -- 172.21.15.94:0/1502935600 wait complete. 2024-02-17T11:59:02.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f7638582740 20 client.25240 trim_cache size 0 max 16384 2024-02-17T11:59:02.402 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:59:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T11:59:02.402 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T11:59:02.402 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:59:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f76358dd640 1 -- 172.21.15.94:0/3002646168 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f761c0a2350 msgr2=0x7f761c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f76358dd640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f761c0a2350 0x7f761c0a4810 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f76280099e0 tx=0x7f762801e000 comp rx=0 tx=0).stop 2024-02-17T11:59:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f76358dd640 1 -- 172.21.15.94:0/3002646168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7630106450 msgr2=0x7f763019d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.395+0000 7f76358dd640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7630106450 0x7f763019d9b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f761800e9f0 tx=0x7f761800eeb0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.396+0000 7f76358dd640 1 -- 172.21.15.94:0/3002646168 shutdown_connections 2024-02-17T11:59:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.396+0000 7f76358dd640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f761c0a2350 0x7f761c0a4810 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.396+0000 7f76358dd640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7630197620 0x7f7630197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.396+0000 7f76358dd640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7630106450 0x7f763019d9b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.396+0000 7f76358dd640 1 --2- 172.21.15.94:0/3002646168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7630101380 0x7f763019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.396+0000 7f76358dd640 1 -- 172.21.15.94:0/3002646168 >> 172.21.15.94:0/3002646168 conn(0x7f76300fcc20 msgr2=0x7f76300fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.396+0000 7f76358dd640 1 -- 172.21.15.94:0/3002646168 shutdown_connections 2024-02-17T11:59:02.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.396+0000 7f76358dd640 1 -- 172.21.15.94:0/3002646168 wait complete. 2024-02-17T11:59:02.405 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T11:59:02.406 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:59:02.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.524+0000 7f65835a7640 1 -- 172.21.15.94:0/2742517115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c1023f0 msgr2=0x7f657c1027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.524+0000 7f65835a7640 1 --2- 172.21.15.94:0/2742517115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c1023f0 0x7f657c1027d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6578009900 tx=0x7f657802bc30 comp rx=0 tx=0).stop 2024-02-17T11:59:02.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.524+0000 7f65835a7640 1 -- 172.21.15.94:0/2742517115 shutdown_connections 2024-02-17T11:59:02.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.524+0000 7f65835a7640 1 --2- 172.21.15.94:0/2742517115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f657c10b7c0 0x7f657c10dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.524+0000 7f65835a7640 1 --2- 172.21.15.94:0/2742517115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f657c102d10 0x7f657c10b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.524+0000 7f65835a7640 1 --2- 172.21.15.94:0/2742517115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c1023f0 0x7f657c1027d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.524+0000 7f65835a7640 1 -- 172.21.15.94:0/2742517115 >> 172.21.15.94:0/2742517115 conn(0x7f657c077180 msgr2=0x7f657c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.525+0000 7f65835a7640 1 -- 172.21.15.94:0/2742517115 shutdown_connections 2024-02-17T11:59:02.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.525+0000 7f65835a7640 1 -- 172.21.15.94:0/2742517115 wait complete. 2024-02-17T11:59:02.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f65835a7640 1 Processor -- start 2024-02-17T11:59:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f65835a7640 1 -- start start 2024-02-17T11:59:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f65835a7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f657c1023f0 0x7f657c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f65835a7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c102d10 0x7f657c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f65835a7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f657c10b7c0 0x7f657c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f65835a7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c197c30 con 0x7f657c1023f0 2024-02-17T11:59:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f65835a7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c197da0 con 0x7f657c102d10 2024-02-17T11:59:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f65835a7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c197f60 con 0x7f657c10b7c0 2024-02-17T11:59:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f6581343640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f657c1023f0 0x7f657c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f6581343640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f657c1023f0 0x7f657c19d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35318/0 (socket says 172.21.15.94:35318) 2024-02-17T11:59:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f6581343640 1 -- 172.21.15.94:0/3803915235 learned_addr learned my addr 172.21.15.94:0/3803915235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f6580b42640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c102d10 0x7f657c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f6581b44640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f657c10b7c0 0x7f657c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f6580b42640 1 -- 172.21.15.94:0/3803915235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f657c10b7c0 msgr2=0x7f657c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f6580b42640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f657c10b7c0 0x7f657c1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f6580b42640 1 -- 172.21.15.94:0/3803915235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f657c1023f0 msgr2=0x7f657c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f6580b42640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f657c1023f0 0x7f657c19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f6580b42640 1 -- 172.21.15.94:0/3803915235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6578009560 con 0x7f657c102d10 2024-02-17T11:59:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f6581343640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f657c1023f0 0x7f657c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.526+0000 7f6581b44640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f657c10b7c0 0x7f657c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.527+0000 7f6580b42640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c102d10 0x7f657c19d970 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f656c009bc0 tx=0x7f656c00b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.527+0000 7f65727fc640 1 -- 172.21.15.94:0/3803915235 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c015070 con 0x7f657c102d10 2024-02-17T11:59:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.527+0000 7f65727fc640 1 -- 172.21.15.94:0/3803915235 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f656c00be60 con 0x7f657c102d10 2024-02-17T11:59:02.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.527+0000 7f65727fc640 1 -- 172.21.15.94:0/3803915235 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c0187d0 con 0x7f657c102d10 2024-02-17T11:59:02.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.527+0000 7f65835a7640 1 -- 172.21.15.94:0/3803915235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f657c198240 con 0x7f657c102d10 2024-02-17T11:59:02.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.527+0000 7f65835a7640 1 -- 172.21.15.94:0/3803915235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657c100150 con 0x7f657c102d10 2024-02-17T11:59:02.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.528+0000 7f65835a7640 1 -- 172.21.15.94:0/3803915235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f657c067f70 con 0x7f657c102d10 2024-02-17T11:59:02.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.530+0000 7f65727fc640 1 -- 172.21.15.94:0/3803915235 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f656c019b10 con 0x7f657c102d10 2024-02-17T11:59:02.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.530+0000 7f65727fc640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f65680a23d0 0x7f65680a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.530+0000 7f6581343640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f65680a23d0 0x7f65680a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.530+0000 7f65727fc640 1 -- 172.21.15.94:0/3803915235 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f656c0d20e0 con 0x7f657c102d10 2024-02-17T11:59:02.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.531+0000 7f6581343640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f65680a23d0 0x7f65680a4890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6578000c00 tx=0x7f65780058d0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.534+0000 7f65727fc640 1 -- 172.21.15.94:0/3803915235 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f656c09ab60 con 0x7f657c102d10 2024-02-17T11:59:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.799+0000 7f65835a7640 1 -- 172.21.15.94:0/3803915235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f657c1983d0 con 0x7f657c102d10 2024-02-17T11:59:02.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.800+0000 7f65727fc640 1 -- 172.21.15.94:0/3803915235 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f657c1983d0 con 0x7f657c102d10 2024-02-17T11:59:02.799 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:59:02.799 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:57:51.854453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T11:59:02.799 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 21 2024-02-17T11:59:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.802+0000 7f65835a7640 1 -- 172.21.15.94:0/3803915235 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f65680a23d0 msgr2=0x7f65680a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.802+0000 7f65835a7640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f65680a23d0 0x7f65680a4890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6578000c00 tx=0x7f65780058d0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.802+0000 7f65835a7640 1 -- 172.21.15.94:0/3803915235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c102d10 msgr2=0x7f657c19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.802+0000 7f65835a7640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c102d10 0x7f657c19d970 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f656c009bc0 tx=0x7f656c00b690 comp rx=0 tx=0).stop 2024-02-17T11:59:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.803+0000 7f65835a7640 1 -- 172.21.15.94:0/3803915235 shutdown_connections 2024-02-17T11:59:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.803+0000 7f65835a7640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f65680a23d0 0x7f65680a4890 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.803+0000 7f65835a7640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f657c10b7c0 0x7f657c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.803+0000 7f65835a7640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f657c102d10 0x7f657c19d970 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.803+0000 7f65835a7640 1 --2- 172.21.15.94:0/3803915235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f657c1023f0 0x7f657c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.803+0000 7f65835a7640 1 -- 172.21.15.94:0/3803915235 >> 172.21.15.94:0/3803915235 conn(0x7f657c077180 msgr2=0x7f657c0fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.803+0000 7f65835a7640 1 -- 172.21.15.94:0/3803915235 shutdown_connections 2024-02-17T11:59:02.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.803+0000 7f65835a7640 1 -- 172.21.15.94:0/3803915235 wait complete. 2024-02-17T11:59:02.813 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T11:59:02.813 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T11:59:02.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.936+0000 7f7b3e2eb640 1 -- 172.21.15.94:0/1746558863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b38100260 msgr2=0x7f7b38100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.936+0000 7f7b3e2eb640 1 --2- 172.21.15.94:0/1746558863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b38100260 0x7f7b38100640 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009930 tx=0x7f7b2802bc30 comp rx=0 tx=0).stop 2024-02-17T11:59:02.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.936+0000 7f7b3e2eb640 1 -- 172.21.15.94:0/1746558863 shutdown_connections 2024-02-17T11:59:02.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.936+0000 7f7b3e2eb640 1 --2- 172.21.15.94:0/1746558863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b38105330 0x7f7b3810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.936+0000 7f7b3e2eb640 1 --2- 172.21.15.94:0/1746558863 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b38100c10 0x7f7b38104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.936+0000 7f7b3e2eb640 1 --2- 172.21.15.94:0/1746558863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b38100260 0x7f7b38100640 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.936+0000 7f7b3e2eb640 1 -- 172.21.15.94:0/1746558863 >> 172.21.15.94:0/1746558863 conn(0x7f7b38077180 msgr2=0x7f7b38077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.936+0000 7f7b3e2eb640 1 -- 172.21.15.94:0/1746558863 shutdown_connections 2024-02-17T11:59:02.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.936+0000 7f7b3e2eb640 1 -- 172.21.15.94:0/1746558863 wait complete. 2024-02-17T11:59:02.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.937+0000 7f7b3e2eb640 1 Processor -- start 2024-02-17T11:59:02.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.937+0000 7f7b3e2eb640 1 -- start start 2024-02-17T11:59:02.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.937+0000 7f7b3e2eb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b38100260 0x7f7b38198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.937+0000 7f7b3e2eb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b38105330 0x7f7b38198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.937+0000 7f7b3e2eb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b381990f0 0x7f7b381a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.937+0000 7f7b3e2eb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381a3d70 con 0x7f7b38105330 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.937+0000 7f7b3e2eb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381a3ee0 con 0x7f7b38100260 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.937+0000 7f7b3e2eb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b381a40a0 con 0x7f7b381990f0 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.937+0000 7f7b377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b38105330 0x7f7b38198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b38105330 0x7f7b38198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35340/0 (socket says 172.21.15.94:35340) 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b377fe640 1 -- 172.21.15.94:0/1833205164 learned_addr learned my addr 172.21.15.94:0/1833205164 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b3c888640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b381990f0 0x7f7b381a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b37fff640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b38100260 0x7f7b38198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b377fe640 1 -- 172.21.15.94:0/1833205164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b381990f0 msgr2=0x7f7b381a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b377fe640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b381990f0 0x7f7b381a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b377fe640 1 -- 172.21.15.94:0/1833205164 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b38100260 msgr2=0x7f7b38198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b377fe640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b38100260 0x7f7b38198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b377fe640 1 -- 172.21.15.94:0/1833205164 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b28009560 con 0x7f7b38105330 2024-02-17T11:59:02.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b37fff640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b38100260 0x7f7b38198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:02.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b377fe640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b38105330 0x7f7b38198b60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00e9c0 tx=0x7f7b2c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b357fa640 1 -- 172.21.15.94:0/1833205164 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c009700 con 0x7f7b38105330 2024-02-17T11:59:02.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b3e2eb640 1 -- 172.21.15.94:0/1833205164 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b381a4380 con 0x7f7b38105330 2024-02-17T11:59:02.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b357fa640 1 -- 172.21.15.94:0/1833205164 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b2c00cd80 con 0x7f7b38105330 2024-02-17T11:59:02.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b357fa640 1 -- 172.21.15.94:0/1833205164 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c010430 con 0x7f7b38105330 2024-02-17T11:59:02.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.938+0000 7f7b3e2eb640 1 -- 172.21.15.94:0/1833205164 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b381a48e0 con 0x7f7b38105330 2024-02-17T11:59:02.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.940+0000 7f7b357fa640 1 -- 172.21.15.94:0/1833205164 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f7b2c0105f0 con 0x7f7b38105330 2024-02-17T11:59:02.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b40f99740 1 Processor -- start 2024-02-17T11:59:02.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b40f99740 1 -- start start 2024-02-17T11:59:02.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b357fa640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7b200a2350 0x7f7b200a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b40f99740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 0x556e9f49a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b40f99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f49ab70 0x556e9f4a3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b40f99740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f4a3750 0x556e9f4a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b3c888640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7b200a2350 0x7f7b200a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b40f99740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9f455960 con 0x556e9f4a3750 2024-02-17T11:59:02.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b40f99740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9f409310 con 0x556e9f49a250 2024-02-17T11:59:02.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b40f99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9f406840 con 0x556e9f49ab70 2024-02-17T11:59:02.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b357fa640 1 -- 172.21.15.94:0/1833205164 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7b2c014070 con 0x7f7b38105330 2024-02-17T11:59:02.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 0x556e9f49a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b3c888640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f4a3750 0x556e9f4a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 0x556e9f49a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:51830/0 (socket says 172.21.15.94:51830) 2024-02-17T11:59:02.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b37fff640 1 -- 172.21.15.94:0/2874507467 learned_addr learned my addr 172.21.15.94:0/2874507467 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:02.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.941+0000 7f7b377fe640 1 --2- 172.21.15.94:0/2874507467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f49ab70 0x556e9f4a3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.942+0000 7f7b37fff640 1 -- 172.21.15.94:0/2874507467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f49ab70 msgr2=0x556e9f4a3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.942+0000 7f7b37fff640 1 --2- 172.21.15.94:0/2874507467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f49ab70 0x556e9f4a3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.942+0000 7f7b37fff640 1 -- 172.21.15.94:0/2874507467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f4a3750 msgr2=0x556e9f4a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.942+0000 7f7b37fff640 1 --2- 172.21.15.94:0/2874507467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f4a3750 0x556e9f4a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.942+0000 7f7b37fff640 1 -- 172.21.15.94:0/2874507467 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e9f3c4060 con 0x556e9f49a250 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.942+0000 7f7b3c888640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7b200a2350 0x7f7b200a4810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009fd0 tx=0x7f7b30009be0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.942+0000 7f7b37fff640 1 --2- 172.21.15.94:0/2874507467 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 0x556e9f49a630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b28036040 tx=0x7f7b2802c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5bffe78a8baf622 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.942+0000 7f7b1e7fc640 1 -- 172.21.15.94:0/2874507467 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28004af0 con 0x556e9f49a250 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.942+0000 7f7b1e7fc640 1 -- 172.21.15.94:0/2874507467 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b28004030 con 0x556e9f49a250 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.942+0000 7f7b1e7fc640 1 -- 172.21.15.94:0/2874507467 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b280053e0 con 0x556e9f49a250 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.943+0000 7f7b40f99740 1 -- 172.21.15.94:0/2874507467 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 msgr2=0x556e9f49a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.943+0000 7f7b40f99740 1 --2- 172.21.15.94:0/2874507467 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 0x556e9f49a630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7b28036040 tx=0x7f7b2802c660 comp rx=0 tx=0).stop 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.943+0000 7f7b40f99740 1 -- 172.21.15.94:0/2874507467 shutdown_connections 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.943+0000 7f7b40f99740 1 --2- 172.21.15.94:0/2874507467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f4a3750 0x556e9f4a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.943+0000 7f7b40f99740 1 --2- 172.21.15.94:0/2874507467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f49ab70 0x556e9f4a3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.943+0000 7f7b40f99740 1 --2- 172.21.15.94:0/2874507467 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 0x556e9f49a630 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.943+0000 7f7b40f99740 1 -- 172.21.15.94:0/2874507467 >> 172.21.15.94:0/2874507467 conn(0x556e9f494980 msgr2=0x556e9f494d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.943+0000 7f7b40f99740 1 -- 172.21.15.94:0/2874507467 shutdown_connections 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.943+0000 7f7b40f99740 1 -- 172.21.15.94:0/2874507467 wait complete. 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.943+0000 7f7b40f99740 1 -- start start 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.943+0000 7f7b40f99740 1 Processor -- start 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.943+0000 7f7b40f99740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 0x556e9f4a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.943+0000 7f7b40f99740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49ab70 0x556e9f4a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b40f99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f4a3750 0x556e9f4a1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b40f99740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9f402020 con 0x556e9f49ab70 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b40f99740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9f0d5430 con 0x556e9f49a250 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b40f99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9f091e00 con 0x556e9f4a3750 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b40f99740 10 client.0 _ll_register_callbacks cb 0x556e9f3a9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 0x556e9f4a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49ab70 0x556e9f4a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 0x556e9f4a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:51840/0 (socket says 172.21.15.94:51840) 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b377fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49ab70 0x556e9f4a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35352/0 (socket says 172.21.15.94:35352) 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b37fff640 1 -- 172.21.15.94:0/3146764885 learned_addr learned my addr 172.21.15.94:0/3146764885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b3c888640 1 --2- 172.21.15.94:0/3146764885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f4a3750 0x556e9f4a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b37fff640 1 -- 172.21.15.94:0/3146764885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f4a3750 msgr2=0x556e9f4a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b37fff640 1 --2- 172.21.15.94:0/3146764885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f4a3750 0x556e9f4a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b37fff640 1 -- 172.21.15.94:0/3146764885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49ab70 msgr2=0x556e9f4a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b37fff640 1 --2- 172.21.15.94:0/3146764885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49ab70 0x556e9f4a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b37fff640 1 -- 172.21.15.94:0/3146764885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3810d710 con 0x556e9f49a250 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b377fe640 1 --2- 172.21.15.94:0/3146764885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49ab70 0x556e9f4a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b37fff640 1 --2- 172.21.15.94:0/3146764885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 0x556e9f4a06b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b2802ca90 tx=0x7f7b2802cac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6df86bca68f98115 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b177fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b177fe640 1 -- 172.21.15.94:0/3146764885 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28004af0 con 0x556e9f49a250 2024-02-17T11:59:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b177fe640 1 -- 172.21.15.94:0/3146764885 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7b2804d070 con 0x556e9f49a250 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.944+0000 7f7b40f99740 1 -- 172.21.15.94:0/3146764885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e9f1017e0 con 0x556e9f49a250 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.945+0000 7f7b177fe640 1 -- 172.21.15.94:0/3146764885 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b28004df0 con 0x556e9f49a250 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.945+0000 7f7b40f99740 10 client.25208 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.945+0000 7f7b40f99740 1 -- 172.21.15.94:0/3146764885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x556e9f4006a0 con 0x556e9f49a250 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.945+0000 7f7b177fe640 1 -- 172.21.15.94:0/3146764885 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556e9f1017e0 con 0x556e9f49a250 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.945+0000 7f7b177fe640 1 -- 172.21.15.94:0/3146764885 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7b2803c050 con 0x556e9f49a250 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.945+0000 7f7b40f99740 10 client.25208 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.945+0000 7f7b40f99740 10 client.25208 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.945+0000 7f7b40f99740 20 client.25208 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.945+0000 7f7b40f99740 1 --2- 172.21.15.94:0/3146764885 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x556e9f4b3730 0x556e9f4b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.945+0000 7f7b40f99740 4 client.25208 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.945+0000 7f7b40f99740 1 -- 172.21.15.94:0/3146764885 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e9f13e720 con 0x556e9f4b3730 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.945+0000 7f7b377fe640 1 --2- 172.21.15.94:0/3146764885 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x556e9f4b3730 0x556e9f4b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b177fe640 10 client.25208 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T11:59:02.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b377fe640 1 --2- 172.21.15.94:0/3146764885 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x556e9f4b3730 0x556e9f4b5b40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a2be330c5f1c9997 server_cookie=faf9350b1231c9c0 in_seq=0 out_seq=0 2024-02-17T11:59:02.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b177fe640 1 -- 172.21.15.94:0/3146764885 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556e9f13e720 con 0x556e9f4b3730 2024-02-17T11:59:02.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b177fe640 10 client.25208 handle_command_reply: tid=0 2024-02-17T11:59:02.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b40f99740 1 client.25208 shutdown 2024-02-17T11:59:02.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b40f99740 1 -- 172.21.15.94:0/3146764885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 msgr2=0x556e9f4a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b40f99740 1 --2- 172.21.15.94:0/3146764885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 0x556e9f4a06b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7b2802ca90 tx=0x7f7b2802cac0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b40f99740 1 -- 172.21.15.94:0/3146764885 shutdown_connections 2024-02-17T11:59:02.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b40f99740 1 --2- 172.21.15.94:0/3146764885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f4a3750 0x556e9f4a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b40f99740 1 --2- 172.21.15.94:0/3146764885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49ab70 0x556e9f4a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b40f99740 1 --2- 172.21.15.94:0/3146764885 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x556e9f4b3730 0x556e9f4b5b40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b40f99740 1 --2- 172.21.15.94:0/3146764885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49a250 0x556e9f4a06b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b177fe640 0 client.25208 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T11:59:02.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.946+0000 7f7b40f99740 1 -- 172.21.15.94:0/3146764885 >> 172.21.15.94:0/3146764885 conn(0x556e9f494980 msgr2=0x556e9f4966d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.947+0000 7f7b40f99740 1 -- 172.21.15.94:0/3146764885 shutdown_connections 2024-02-17T11:59:02.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.947+0000 7f7b40f99740 1 -- 172.21.15.94:0/3146764885 wait complete. 2024-02-17T11:59:02.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.947+0000 7f7b40f99740 20 client.25208 trim_cache size 0 max 16384 2024-02-17T11:59:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b40f99740 1 Processor -- start 2024-02-17T11:59:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b40f99740 1 -- start start 2024-02-17T11:59:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b40f99740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49a250 0x556e9f493a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b40f99740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49ab70 0x556e9f4aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b40f99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f4a3750 0x556e9f4aaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b40f99740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9f387130 con 0x556e9f49a250 2024-02-17T11:59:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b40f99740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9f4b3530 con 0x556e9f49ab70 2024-02-17T11:59:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b40f99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9f499eb0 con 0x556e9f4a3750 2024-02-17T11:59:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49a250 0x556e9f493a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b37fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49a250 0x556e9f493a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35364/0 (socket says 172.21.15.94:35364) 2024-02-17T11:59:02.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b37fff640 1 -- 172.21.15.94:0/1150247477 learned_addr learned my addr 172.21.15.94:0/1150247477 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b3c888640 1 --2- 172.21.15.94:0/1150247477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f4a3750 0x556e9f4aaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b377fe640 1 --2- 172.21.15.94:0/1150247477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49ab70 0x556e9f4aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b37fff640 1 -- 172.21.15.94:0/1150247477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f4a3750 msgr2=0x556e9f4aaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b37fff640 1 --2- 172.21.15.94:0/1150247477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f4a3750 0x556e9f4aaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b37fff640 1 -- 172.21.15.94:0/1150247477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49ab70 msgr2=0x556e9f4aa5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b37fff640 1 --2- 172.21.15.94:0/1150247477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49ab70 0x556e9f4aa5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.959+0000 7f7b37fff640 1 -- 172.21.15.94:0/1150247477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e9f39cc20 con 0x556e9f49a250 2024-02-17T11:59:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.960+0000 7f7b37fff640 1 --2- 172.21.15.94:0/1150247477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49a250 0x556e9f493a00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009800 tx=0x7f7b2803f440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4dd9d982fbf931 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.960+0000 7f7b1dffb640 1 -- 172.21.15.94:0/1150247477 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b28004af0 con 0x556e9f49a250 2024-02-17T11:59:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.960+0000 7f7b1dffb640 1 -- 172.21.15.94:0/1150247477 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b2802cb30 con 0x556e9f49a250 2024-02-17T11:59:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.960+0000 7f7b1dffb640 1 -- 172.21.15.94:0/1150247477 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b280035f0 con 0x556e9f49a250 2024-02-17T11:59:02.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.960+0000 7f7b40f99740 1 -- 172.21.15.94:0/1150247477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49a250 msgr2=0x556e9f493a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.960+0000 7f7b40f99740 1 --2- 172.21.15.94:0/1150247477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49a250 0x556e9f493a00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009800 tx=0x7f7b2803f440 comp rx=0 tx=0).stop 2024-02-17T11:59:02.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 -- 172.21.15.94:0/1150247477 shutdown_connections 2024-02-17T11:59:02.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 --2- 172.21.15.94:0/1150247477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f4a3750 0x556e9f4aaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 --2- 172.21.15.94:0/1150247477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49ab70 0x556e9f4aa5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 --2- 172.21.15.94:0/1150247477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f49a250 0x556e9f493a00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 -- 172.21.15.94:0/1150247477 >> 172.21.15.94:0/1150247477 conn(0x556e9f494980 msgr2=0x556e9f4932f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 -- 172.21.15.94:0/1150247477 shutdown_connections 2024-02-17T11:59:02.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 -- 172.21.15.94:0/1150247477 wait complete. 2024-02-17T11:59:02.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 -- start start 2024-02-17T11:59:02.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 Processor -- start 2024-02-17T11:59:02.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f49a250 0x556e9f4a9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49ab70 0x556e9f4a9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f4a3750 0x556e9f4a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9f509d40 con 0x556e9f4a3750 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9f509eb0 con 0x556e9f49ab70 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e9f50a070 con 0x556e9f49a250 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b40f99740 10 client.0 _ll_register_callbacks cb 0x7f7b300093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b3c888640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f4a3750 0x556e9f4a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b3c888640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f4a3750 0x556e9f4a9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35366/0 (socket says 172.21.15.94:35366) 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b3c888640 1 -- 172.21.15.94:0/3009648336 learned_addr learned my addr 172.21.15.94:0/3009648336 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b377fe640 1 --2- 172.21.15.94:0/3009648336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49ab70 0x556e9f4a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.961+0000 7f7b37fff640 1 --2- 172.21.15.94:0/3009648336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f49a250 0x556e9f4a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b3c888640 1 -- 172.21.15.94:0/3009648336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f49a250 msgr2=0x556e9f4a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b3c888640 1 --2- 172.21.15.94:0/3009648336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f49a250 0x556e9f4a9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b3c888640 1 -- 172.21.15.94:0/3009648336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49ab70 msgr2=0x556e9f4a9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b3c888640 1 --2- 172.21.15.94:0/3009648336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49ab70 0x556e9f4a9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b3c888640 1 -- 172.21.15.94:0/3009648336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e9f4006a0 con 0x556e9f4a3750 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b3c888640 1 --2- 172.21.15.94:0/3009648336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f4a3750 0x556e9f4a9ba0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7b30020680 tx=0x7f7b30004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f896149127ec37af server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b177fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b177fe640 1 -- 172.21.15.94:0/3009648336 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30018040 con 0x556e9f4a3750 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b177fe640 1 -- 172.21.15.94:0/3009648336 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7b30025050 con 0x556e9f4a3750 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b40f99740 1 -- 172.21.15.94:0/3009648336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b28004030 con 0x556e9f4a3750 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b177fe640 1 -- 172.21.15.94:0/3009648336 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b3000e880 con 0x556e9f4a3750 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b40f99740 10 client.15735 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b40f99740 1 -- 172.21.15.94:0/3009648336 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x556e9f3c28c0 con 0x556e9f4a3750 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.962+0000 7f7b177fe640 1 -- 172.21.15.94:0/3009648336 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b28004030 con 0x556e9f4a3750 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.963+0000 7f7b177fe640 1 -- 172.21.15.94:0/3009648336 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7b3002ccf0 con 0x556e9f4a3750 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.963+0000 7f7b40f99740 10 client.15735 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.963+0000 7f7b40f99740 10 client.15735 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.963+0000 7f7b40f99740 20 client.15735 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.963+0000 7f7b40f99740 1 --2- 172.21.15.94:0/3009648336 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x556e9f50b920 0x556e9f50dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:02.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.963+0000 7f7b40f99740 4 client.15735 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T11:59:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.963+0000 7f7b40f99740 1 -- 172.21.15.94:0/3009648336 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556e9f18d890 con 0x556e9f50b920 2024-02-17T11:59:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.963+0000 7f7b37fff640 1 --2- 172.21.15.94:0/3009648336 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x556e9f50b920 0x556e9f50dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.963+0000 7f7b177fe640 10 client.15735 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T11:59:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.963+0000 7f7b37fff640 1 --2- 172.21.15.94:0/3009648336 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x556e9f50b920 0x556e9f50dd40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e75099911be55652 server_cookie=675a1d44fd087734 in_seq=0 out_seq=0 2024-02-17T11:59:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.963+0000 7f7b177fe640 1 -- 172.21.15.94:0/3009648336 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556e9f18d890 con 0x556e9f50b920 2024-02-17T11:59:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.963+0000 7f7b177fe640 10 client.15735 handle_command_reply: tid=0 2024-02-17T11:59:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.963+0000 7f7b40f99740 1 client.15735 shutdown 2024-02-17T11:59:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.964+0000 7f7b40f99740 1 -- 172.21.15.94:0/3009648336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f4a3750 msgr2=0x556e9f4a9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.964+0000 7f7b40f99740 1 --2- 172.21.15.94:0/3009648336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f4a3750 0x556e9f4a9ba0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7b30020680 tx=0x7f7b30004ab0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.964+0000 7f7b40f99740 1 -- 172.21.15.94:0/3009648336 shutdown_connections 2024-02-17T11:59:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.964+0000 7f7b40f99740 1 --2- 172.21.15.94:0/3009648336 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e9f4a3750 0x556e9f4a9ba0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.964+0000 7f7b40f99740 1 --2- 172.21.15.94:0/3009648336 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x556e9f50b920 0x556e9f50dd40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.964+0000 7f7b40f99740 1 --2- 172.21.15.94:0/3009648336 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e9f49ab70 0x556e9f4a9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.964+0000 7f7b40f99740 1 --2- 172.21.15.94:0/3009648336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e9f49a250 0x556e9f4a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.964+0000 7f7b177fe640 0 client.15735 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T11:59:02.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.964+0000 7f7b40f99740 1 -- 172.21.15.94:0/3009648336 >> 172.21.15.94:0/3009648336 conn(0x556e9f494980 msgr2=0x556e9f493b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.965+0000 7f7b40f99740 1 -- 172.21.15.94:0/3009648336 shutdown_connections 2024-02-17T11:59:02.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.966+0000 7f7b40f99740 1 -- 172.21.15.94:0/3009648336 wait complete. 2024-02-17T11:59:02.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.966+0000 7f7b40f99740 20 client.15735 trim_cache size 0 max 16384 2024-02-17T11:59:02.970 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T11:59:02.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.967+0000 7f7b3e2eb640 1 -- 172.21.15.94:0/1833205164 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7b200a2350 msgr2=0x7f7b200a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.967+0000 7f7b3e2eb640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7b200a2350 0x7f7b200a4810 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009fd0 tx=0x7f7b30009be0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.967+0000 7f7b3e2eb640 1 -- 172.21.15.94:0/1833205164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b38105330 msgr2=0x7f7b38198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:02.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.967+0000 7f7b3e2eb640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b38105330 0x7f7b38198b60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00e9c0 tx=0x7f7b2c00ee80 comp rx=0 tx=0).stop 2024-02-17T11:59:02.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.967+0000 7f7b3e2eb640 1 -- 172.21.15.94:0/1833205164 shutdown_connections 2024-02-17T11:59:02.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.967+0000 7f7b3e2eb640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7b200a2350 0x7f7b200a4810 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.967+0000 7f7b3e2eb640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b381990f0 0x7f7b381a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.967+0000 7f7b3e2eb640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b38105330 0x7f7b38198b60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.967+0000 7f7b3e2eb640 1 --2- 172.21.15.94:0/1833205164 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b38100260 0x7f7b38198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:02.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.967+0000 7f7b3e2eb640 1 -- 172.21.15.94:0/1833205164 >> 172.21.15.94:0/1833205164 conn(0x7f7b38077180 msgr2=0x7f7b380fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:02.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.967+0000 7f7b3e2eb640 1 -- 172.21.15.94:0/1833205164 shutdown_connections 2024-02-17T11:59:02.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:02.967+0000 7f7b3e2eb640 1 -- 172.21.15.94:0/1833205164 wait complete. 2024-02-17T11:59:02.979 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T11:59:02.979 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T11:59:03.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.098+0000 7ff7401e2640 1 -- 172.21.15.94:0/1675668499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff738106100 msgr2=0x7ff7381064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.098+0000 7ff7401e2640 1 --2- 172.21.15.94:0/1675668499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff738106100 0x7ff7381064e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff734009900 tx=0x7ff73402bcd0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.098+0000 7ff7401e2640 1 -- 172.21.15.94:0/1675668499 shutdown_connections 2024-02-17T11:59:03.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.098+0000 7ff7401e2640 1 --2- 172.21.15.94:0/1675668499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7381009e0 0x7ff73810cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.098+0000 7ff7401e2640 1 --2- 172.21.15.94:0/1675668499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff738100040 0x7ff7381004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.098+0000 7ff7401e2640 1 --2- 172.21.15.94:0/1675668499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff738106100 0x7ff7381064e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.098+0000 7ff7401e2640 1 -- 172.21.15.94:0/1675668499 >> 172.21.15.94:0/1675668499 conn(0x7ff738077110 msgr2=0x7ff738077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:03.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.098+0000 7ff7401e2640 1 -- 172.21.15.94:0/1675668499 shutdown_connections 2024-02-17T11:59:03.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.098+0000 7ff7401e2640 1 -- 172.21.15.94:0/1675668499 wait complete. 2024-02-17T11:59:03.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff7401e2640 1 Processor -- start 2024-02-17T11:59:03.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff7401e2640 1 -- start start 2024-02-17T11:59:03.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff7401e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff738100040 0x7ff738197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff7401e2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7381009e0 0x7ff738197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff7401e2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff738106100 0x7ff73819bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff7401e2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73819c5d0 con 0x7ff738100040 2024-02-17T11:59:03.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff7401e2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73819c740 con 0x7ff7381009e0 2024-02-17T11:59:03.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff7401e2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73819c900 con 0x7ff738106100 2024-02-17T11:59:03.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff73df7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff738100040 0x7ff738197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff73df7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff738100040 0x7ff738197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35382/0 (socket says 172.21.15.94:35382) 2024-02-17T11:59:03.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff73df7e640 1 -- 172.21.15.94:0/1122939584 learned_addr learned my addr 172.21.15.94:0/1122939584 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:03.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff73d77d640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7381009e0 0x7ff738197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff73e77f640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff738106100 0x7ff73819bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff73df7e640 1 -- 172.21.15.94:0/1122939584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff738106100 msgr2=0x7ff73819bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff73df7e640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff738106100 0x7ff73819bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff73df7e640 1 -- 172.21.15.94:0/1122939584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7381009e0 msgr2=0x7ff738197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff73df7e640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7381009e0 0x7ff738197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.099+0000 7ff73df7e640 1 -- 172.21.15.94:0/1122939584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff734009560 con 0x7ff738100040 2024-02-17T11:59:03.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.100+0000 7ff73df7e640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff738100040 0x7ff738197710 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff734005940 tx=0x7ff73402c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:03.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.100+0000 7ff72effd640 1 -- 172.21.15.94:0/1122939584 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734004350 con 0x7ff738100040 2024-02-17T11:59:03.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.100+0000 7ff72effd640 1 -- 172.21.15.94:0/1122939584 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7340044b0 con 0x7ff738100040 2024-02-17T11:59:03.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.100+0000 7ff7401e2640 1 -- 172.21.15.94:0/1122939584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73819cb80 con 0x7ff738100040 2024-02-17T11:59:03.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.100+0000 7ff72effd640 1 -- 172.21.15.94:0/1122939584 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734003d30 con 0x7ff738100040 2024-02-17T11:59:03.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.100+0000 7ff7401e2640 1 -- 172.21.15.94:0/1122939584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7381a4460 con 0x7ff738100040 2024-02-17T11:59:03.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.102+0000 7ff72effd640 1 -- 172.21.15.94:0/1122939584 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7ff734004df0 con 0x7ff738100040 2024-02-17T11:59:03.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.102+0000 7ff742db8740 1 Processor -- start 2024-02-17T11:59:03.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff742db8740 1 -- start start 2024-02-17T11:59:03.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff742db8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1250 0x55b11bff1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff742db8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1b70 0x55b11bffa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff742db8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11bffcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff742db8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bfac960 con 0x55b11bff1250 2024-02-17T11:59:03.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff742db8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bf60310 con 0x55b11bff1b70 2024-02-17T11:59:03.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff742db8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bf5d840 con 0x55b11bffa750 2024-02-17T11:59:03.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff72effd640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7240a2350 0x7ff7240a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff73e77f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1250 0x55b11bff1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff73d77d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11bffcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff73d77d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11bffcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:43958/0 (socket says 172.21.15.94:43958) 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff73e77f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1250 0x55b11bff1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35384/0 (socket says 172.21.15.94:35384) 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff73d77d640 1 -- 172.21.15.94:0/528284333 learned_addr learned my addr 172.21.15.94:0/528284333 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff73df7e640 1 --2- 172.21.15.94:0/528284333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1b70 0x55b11bffa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff73e77f640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7240a2350 0x7ff7240a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff72effd640 1 -- 172.21.15.94:0/1122939584 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff7340f08f0 con 0x7ff738100040 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff73d77d640 1 -- 172.21.15.94:0/528284333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1b70 msgr2=0x55b11bffa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff73d77d640 1 --2- 172.21.15.94:0/528284333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1b70 0x55b11bffa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff73d77d640 1 -- 172.21.15.94:0/528284333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1250 msgr2=0x55b11bff1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff73d77d640 1 --2- 172.21.15.94:0/528284333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1250 0x55b11bff1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.103+0000 7ff73d77d640 1 -- 172.21.15.94:0/528284333 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b11bf1b060 con 0x55b11bffa750 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.104+0000 7ff73e77f640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7240a2350 0x7ff7240a4810 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff730009fd0 tx=0x7ff730009500 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:03.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.104+0000 7ff73d77d640 1 --2- 172.21.15.94:0/528284333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11bffcb40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff728009ca0 tx=0x7ff728011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a091bd327df439fe server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.104+0000 7ff723fff640 1 -- 172.21.15.94:0/528284333 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728019630 con 0x55b11bffa750 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.104+0000 7ff723fff640 1 -- 172.21.15.94:0/528284333 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff728019c70 con 0x55b11bffa750 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.104+0000 7ff723fff640 1 -- 172.21.15.94:0/528284333 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72800ebc0 con 0x55b11bffa750 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.105+0000 7ff742db8740 1 -- 172.21.15.94:0/528284333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 msgr2=0x55b11bffcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.105+0000 7ff742db8740 1 --2- 172.21.15.94:0/528284333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11bffcb40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff728009ca0 tx=0x7ff728011810 comp rx=0 tx=0).stop 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.105+0000 7ff742db8740 1 -- 172.21.15.94:0/528284333 shutdown_connections 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.105+0000 7ff742db8740 1 --2- 172.21.15.94:0/528284333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11bffcb40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.105+0000 7ff742db8740 1 --2- 172.21.15.94:0/528284333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1b70 0x55b11bffa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.105+0000 7ff742db8740 1 --2- 172.21.15.94:0/528284333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1250 0x55b11bff1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.105+0000 7ff742db8740 1 -- 172.21.15.94:0/528284333 >> 172.21.15.94:0/528284333 conn(0x55b11bfeb980 msgr2=0x55b11bfebd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.105+0000 7ff742db8740 1 -- 172.21.15.94:0/528284333 shutdown_connections 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.105+0000 7ff742db8740 1 -- 172.21.15.94:0/528284333 wait complete. 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.105+0000 7ff742db8740 1 -- start start 2024-02-17T11:59:03.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.105+0000 7ff742db8740 1 Processor -- start 2024-02-17T11:59:03.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff742db8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11bff76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff742db8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bff1b70 0x55b11bff7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff742db8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bffa750 0x55b11bff8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff742db8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bf59020 con 0x55b11bffa750 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff742db8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bc2c430 con 0x55b11bff1250 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff742db8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bbe8e00 con 0x55b11bff1b70 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff742db8740 10 client.0 _ll_register_callbacks cb 0x55b11bf00cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff73e77f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bffa750 0x55b11bff8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff73d77d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bff1b70 0x55b11bff7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff73e77f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bffa750 0x55b11bff8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35386/0 (socket says 172.21.15.94:35386) 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff73d77d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bff1b70 0x55b11bff7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:43960/0 (socket says 172.21.15.94:43960) 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff73e77f640 1 -- 172.21.15.94:0/2439203372 learned_addr learned my addr 172.21.15.94:0/2439203372 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff73df7e640 1 --2- 172.21.15.94:0/2439203372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11bff76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff73e77f640 1 -- 172.21.15.94:0/2439203372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bff1b70 msgr2=0x55b11bff7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff73e77f640 1 --2- 172.21.15.94:0/2439203372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bff1b70 0x55b11bff7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff73e77f640 1 -- 172.21.15.94:0/2439203372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 msgr2=0x55b11bff76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff73e77f640 1 --2- 172.21.15.94:0/2439203372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11bff76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff73e77f640 1 -- 172.21.15.94:0/2439203372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7280098d0 con 0x55b11bffa750 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff73d77d640 1 --2- 172.21.15.94:0/2439203372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bff1b70 0x55b11bff7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff73e77f640 1 --2- 172.21.15.94:0/2439203372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bffa750 0x55b11bff8160 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff73000ffd0 tx=0x7ff730004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a40369852aaf11 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.106+0000 7ff7217fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff7217fa640 1 -- 172.21.15.94:0/2439203372 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730003e00 con 0x55b11bffa750 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff742db8740 1 -- 172.21.15.94:0/2439203372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b11bc587e0 con 0x55b11bffa750 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff7217fa640 1 -- 172.21.15.94:0/2439203372 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff730026070 con 0x55b11bffa750 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff7217fa640 1 -- 172.21.15.94:0/2439203372 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7300047c0 con 0x55b11bffa750 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff742db8740 10 client.15747 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff742db8740 1 -- 172.21.15.94:0/2439203372 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55b11bf576a0 con 0x55b11bffa750 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff7217fa640 1 -- 172.21.15.94:0/2439203372 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b11bc587e0 con 0x55b11bffa750 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff7217fa640 1 -- 172.21.15.94:0/2439203372 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff73001f4e0 con 0x55b11bffa750 2024-02-17T11:59:03.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff742db8740 10 client.15747 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff742db8740 10 client.15747 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff742db8740 20 client.15747 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff742db8740 1 --2- 172.21.15.94:0/2439203372 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55b11c00ab00 0x55b11c00cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff742db8740 4 client.15747 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff742db8740 1 -- 172.21.15.94:0/2439203372 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b11bc95720 con 0x55b11c00ab00 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.107+0000 7ff73df7e640 1 --2- 172.21.15.94:0/2439203372 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55b11c00ab00 0x55b11c00cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.108+0000 7ff7217fa640 10 client.15747 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.108+0000 7ff73df7e640 1 --2- 172.21.15.94:0/2439203372 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55b11c00ab00 0x55b11c00cf20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d3accc6e9ca5c57 server_cookie=28c35807be87ce9 in_seq=0 out_seq=0 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff7217fa640 1 -- 172.21.15.94:0/2439203372 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b11bc95720 con 0x55b11c00ab00 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff7217fa640 10 client.15747 handle_command_reply: tid=0 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff742db8740 1 client.15747 shutdown 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff742db8740 1 -- 172.21.15.94:0/2439203372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bffa750 msgr2=0x55b11bff8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff742db8740 1 --2- 172.21.15.94:0/2439203372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bffa750 0x55b11bff8160 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ff73000ffd0 tx=0x7ff730004a40 comp rx=0 tx=0).stop 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff742db8740 1 -- 172.21.15.94:0/2439203372 shutdown_connections 2024-02-17T11:59:03.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff742db8740 1 --2- 172.21.15.94:0/2439203372 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55b11c00ab00 0x55b11c00cf20 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff742db8740 1 --2- 172.21.15.94:0/2439203372 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bffa750 0x55b11bff8160 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff742db8740 1 --2- 172.21.15.94:0/2439203372 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bff1b70 0x55b11bff7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff7217fa640 0 client.15747 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T11:59:03.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff742db8740 1 --2- 172.21.15.94:0/2439203372 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11bff76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff742db8740 1 -- 172.21.15.94:0/2439203372 >> 172.21.15.94:0/2439203372 conn(0x55b11bfeb980 msgr2=0x55b11bfed6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:03.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff742db8740 1 -- 172.21.15.94:0/2439203372 shutdown_connections 2024-02-17T11:59:03.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff742db8740 1 -- 172.21.15.94:0/2439203372 wait complete. 2024-02-17T11:59:03.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.109+0000 7ff742db8740 20 client.15747 trim_cache size 0 max 16384 2024-02-17T11:59:03.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.123+0000 7ff742db8740 1 Processor -- start 2024-02-17T11:59:03.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.123+0000 7ff742db8740 1 -- start start 2024-02-17T11:59:03.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff742db8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11bff9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff742db8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1b70 0x55b11bff7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff742db8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11bff4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff742db8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bede130 con 0x55b11bff1b70 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff742db8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bfec980 con 0x55b11bff1250 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff742db8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11bff32e0 con 0x55b11bffa750 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff73d77d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1b70 0x55b11bff7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff73df7e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11bff9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff73d77d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1b70 0x55b11bff7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35394/0 (socket says 172.21.15.94:35394) 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff73d77d640 1 -- 172.21.15.94:0/570456616 learned_addr learned my addr 172.21.15.94:0/570456616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff73e77f640 1 --2- 172.21.15.94:0/570456616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11bff4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff73d77d640 1 -- 172.21.15.94:0/570456616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 msgr2=0x55b11bff4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff73d77d640 1 --2- 172.21.15.94:0/570456616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11bff4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff73d77d640 1 -- 172.21.15.94:0/570456616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 msgr2=0x55b11bff9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff73d77d640 1 --2- 172.21.15.94:0/570456616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11bff9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff73d77d640 1 -- 172.21.15.94:0/570456616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b11bef3c20 con 0x55b11bff1b70 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff73e77f640 1 --2- 172.21.15.94:0/570456616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11bff4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:03.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.124+0000 7ff73df7e640 1 --2- 172.21.15.94:0/570456616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11bff9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:59:03.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.125+0000 7ff73d77d640 1 --2- 172.21.15.94:0/570456616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1b70 0x55b11bff7f80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55b11bfecb10 tx=0x7ff72800e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef38392aebac07e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:03.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.125+0000 7ff7237fe640 1 -- 172.21.15.94:0/570456616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72800e5b0 con 0x55b11bff1b70 2024-02-17T11:59:03.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.125+0000 7ff7237fe640 1 -- 172.21.15.94:0/570456616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff728005520 con 0x55b11bff1b70 2024-02-17T11:59:03.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.125+0000 7ff7237fe640 1 -- 172.21.15.94:0/570456616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728019940 con 0x55b11bff1b70 2024-02-17T11:59:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.126+0000 7ff742db8740 1 -- 172.21.15.94:0/570456616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1b70 msgr2=0x55b11bff7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.126+0000 7ff742db8740 1 --2- 172.21.15.94:0/570456616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1b70 0x55b11bff7f80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55b11bfecb10 tx=0x7ff72800e880 comp rx=0 tx=0).stop 2024-02-17T11:59:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.126+0000 7ff742db8740 1 -- 172.21.15.94:0/570456616 shutdown_connections 2024-02-17T11:59:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.126+0000 7ff742db8740 1 --2- 172.21.15.94:0/570456616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11bff4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.126+0000 7ff742db8740 1 --2- 172.21.15.94:0/570456616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1b70 0x55b11bff7f80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.126+0000 7ff742db8740 1 --2- 172.21.15.94:0/570456616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11bff9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.126+0000 7ff742db8740 1 -- 172.21.15.94:0/570456616 >> 172.21.15.94:0/570456616 conn(0x55b11bfeb980 msgr2=0x55b11c0118a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.126+0000 7ff742db8740 1 -- 172.21.15.94:0/570456616 shutdown_connections 2024-02-17T11:59:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.126+0000 7ff742db8740 1 -- 172.21.15.94:0/570456616 wait complete. 2024-02-17T11:59:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.127+0000 7ff742db8740 1 -- start start 2024-02-17T11:59:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.127+0000 7ff742db8740 1 Processor -- start 2024-02-17T11:59:03.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.127+0000 7ff742db8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11c0008f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.127+0000 7ff742db8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1b70 0x55b11c000e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.127+0000 7ff742db8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11c05f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.127+0000 7ff742db8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11c05f9c0 con 0x55b11bff1b70 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.127+0000 7ff742db8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11c05fb30 con 0x55b11bff1250 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.127+0000 7ff742db8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b11c05fcf0 con 0x55b11bffa750 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.128+0000 7ff73d77d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1b70 0x55b11c000e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.128+0000 7ff73df7e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11c0008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.128+0000 7ff73d77d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1b70 0x55b11c000e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35410/0 (socket says 172.21.15.94:35410) 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.128+0000 7ff742db8740 10 client.0 _ll_register_callbacks cb 0x7ff728009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.128+0000 7ff73d77d640 1 -- 172.21.15.94:0/953140356 learned_addr learned my addr 172.21.15.94:0/953140356 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.128+0000 7ff73e77f640 1 --2- 172.21.15.94:0/953140356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11c05f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.128+0000 7ff73df7e640 1 -- 172.21.15.94:0/953140356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 msgr2=0x55b11c05f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.128+0000 7ff73df7e640 1 --2- 172.21.15.94:0/953140356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11c05f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.128+0000 7ff73df7e640 1 -- 172.21.15.94:0/953140356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1b70 msgr2=0x55b11c000e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.128+0000 7ff73df7e640 1 --2- 172.21.15.94:0/953140356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1b70 0x55b11c000e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.128+0000 7ff73df7e640 1 -- 172.21.15.94:0/953140356 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b11bf576a0 con 0x55b11bff1250 2024-02-17T11:59:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.128+0000 7ff73df7e640 1 --2- 172.21.15.94:0/953140356 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11c0008f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff7340edfc0 tx=0x7ff7340b2860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc4f33e4f67a9b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.128+0000 7ff7217fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:59:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.129+0000 7ff7217fa640 1 -- 172.21.15.94:0/953140356 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7340bd190 con 0x55b11bff1250 2024-02-17T11:59:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.129+0000 7ff7217fa640 1 -- 172.21.15.94:0/953140356 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff7340c12e0 con 0x55b11bff1250 2024-02-17T11:59:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.129+0000 7ff742db8740 1 -- 172.21.15.94:0/953140356 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff728019c70 con 0x55b11bff1250 2024-02-17T11:59:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.129+0000 7ff7217fa640 1 -- 172.21.15.94:0/953140356 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7340c5930 con 0x55b11bff1250 2024-02-17T11:59:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.129+0000 7ff742db8740 10 client.25229 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.129+0000 7ff742db8740 1 -- 172.21.15.94:0/953140356 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55b11bf1b060 con 0x55b11bff1250 2024-02-17T11:59:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.129+0000 7ff7217fa640 1 -- 172.21.15.94:0/953140356 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff728019c70 con 0x55b11bff1250 2024-02-17T11:59:03.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.129+0000 7ff7217fa640 1 -- 172.21.15.94:0/953140356 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff73403c070 con 0x55b11bff1250 2024-02-17T11:59:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.130+0000 7ff742db8740 10 client.25229 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.130+0000 7ff742db8740 10 client.25229 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T11:59:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.130+0000 7ff742db8740 20 client.25229 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.130+0000 7ff742db8740 1 --2- 172.21.15.94:0/953140356 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55b11c067040 0x55b11c069460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.130+0000 7ff742db8740 4 client.25229 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T11:59:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.130+0000 7ff742db8740 1 -- 172.21.15.94:0/953140356 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b11bce4890 con 0x55b11c067040 2024-02-17T11:59:03.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.131+0000 7ff73d77d640 1 --2- 172.21.15.94:0/953140356 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55b11c067040 0x55b11c069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.133+0000 7ff7217fa640 10 client.25229 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T11:59:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.133+0000 7ff73d77d640 1 --2- 172.21.15.94:0/953140356 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55b11c067040 0x55b11c069460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe2299c7aab5a2a1 server_cookie=487edc3f5f6179d1 in_seq=0 out_seq=0 2024-02-17T11:59:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.133+0000 7ff7217fa640 1 -- 172.21.15.94:0/953140356 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b11bce4890 con 0x55b11c067040 2024-02-17T11:59:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.133+0000 7ff7217fa640 10 client.25229 handle_command_reply: tid=0 2024-02-17T11:59:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.133+0000 7ff742db8740 1 client.25229 shutdown 2024-02-17T11:59:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.134+0000 7ff742db8740 1 -- 172.21.15.94:0/953140356 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 msgr2=0x55b11c0008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.134+0000 7ff742db8740 1 --2- 172.21.15.94:0/953140356 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11c0008f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff7340edfc0 tx=0x7ff7340b2860 comp rx=0 tx=0).stop 2024-02-17T11:59:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.134+0000 7ff742db8740 1 -- 172.21.15.94:0/953140356 shutdown_connections 2024-02-17T11:59:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.134+0000 7ff742db8740 1 --2- 172.21.15.94:0/953140356 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55b11c067040 0x55b11c069460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.134+0000 7ff742db8740 1 --2- 172.21.15.94:0/953140356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b11bffa750 0x55b11c05f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.134+0000 7ff742db8740 1 --2- 172.21.15.94:0/953140356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b11bff1b70 0x55b11c000e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.134+0000 7ff742db8740 1 --2- 172.21.15.94:0/953140356 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b11bff1250 0x55b11c0008f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.134+0000 7ff7217fa640 0 client.25229 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T11:59:03.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.134+0000 7ff742db8740 1 -- 172.21.15.94:0/953140356 >> 172.21.15.94:0/953140356 conn(0x55b11bfeb980 msgr2=0x55b11bfecdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:03.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.134+0000 7ff742db8740 1 -- 172.21.15.94:0/953140356 shutdown_connections 2024-02-17T11:59:03.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.134+0000 7ff742db8740 1 -- 172.21.15.94:0/953140356 wait complete. 2024-02-17T11:59:03.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.134+0000 7ff742db8740 20 client.25229 trim_cache size 0 max 16384 2024-02-17T11:59:03.134 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T11:59:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.135+0000 7ff7401e2640 1 -- 172.21.15.94:0/1122939584 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7240a2350 msgr2=0x7ff7240a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.135+0000 7ff7401e2640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7240a2350 0x7ff7240a4810 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff730009fd0 tx=0x7ff730009500 comp rx=0 tx=0).stop 2024-02-17T11:59:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.135+0000 7ff7401e2640 1 -- 172.21.15.94:0/1122939584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff738100040 msgr2=0x7ff738197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.135+0000 7ff7401e2640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff738100040 0x7ff738197710 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff734005940 tx=0x7ff73402c710 comp rx=0 tx=0).stop 2024-02-17T11:59:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.135+0000 7ff7401e2640 1 -- 172.21.15.94:0/1122939584 shutdown_connections 2024-02-17T11:59:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.135+0000 7ff7401e2640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7240a2350 0x7ff7240a4810 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.135+0000 7ff7401e2640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff738106100 0x7ff73819bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.135+0000 7ff7401e2640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7381009e0 0x7ff738197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.135+0000 7ff7401e2640 1 --2- 172.21.15.94:0/1122939584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff738100040 0x7ff738197710 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.135+0000 7ff7401e2640 1 -- 172.21.15.94:0/1122939584 >> 172.21.15.94:0/1122939584 conn(0x7ff738077110 msgr2=0x7ff7380fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.135+0000 7ff7401e2640 1 -- 172.21.15.94:0/1122939584 shutdown_connections 2024-02-17T11:59:03.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.135+0000 7ff7401e2640 1 -- 172.21.15.94:0/1122939584 wait complete. 2024-02-17T11:59:03.146 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:02 smithi094 ceph-mon[32499]: mgrmap e26: x(active, since 55s), standbys: y, z 2024-02-17T11:59:03.146 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:02 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3803915235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:59:03.148 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T11:59:03.148 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T11:59:03.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:02 smithi189 ceph-mon[36588]: mgrmap e26: x(active, since 55s), standbys: y, z 2024-02-17T11:59:03.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:02 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3803915235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:59:03.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:02 smithi125 ceph-mon[36819]: mgrmap e26: x(active, since 55s), standbys: y, z 2024-02-17T11:59:03.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:02 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3803915235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.268+0000 7fb547da1640 1 -- 172.21.15.94:0/4247980901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb540100260 msgr2=0x7fb540100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.268+0000 7fb547da1640 1 --2- 172.21.15.94:0/4247980901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb540100260 0x7fb540100640 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009930 tx=0x7fb53c02bc30 comp rx=0 tx=0).stop 2024-02-17T11:59:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.270+0000 7fb547da1640 1 -- 172.21.15.94:0/4247980901 shutdown_connections 2024-02-17T11:59:03.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.270+0000 7fb547da1640 1 --2- 172.21.15.94:0/4247980901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb540105330 0x7fb54010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.270+0000 7fb547da1640 1 --2- 172.21.15.94:0/4247980901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb540100c10 0x7fb540104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.270+0000 7fb547da1640 1 --2- 172.21.15.94:0/4247980901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb540100260 0x7fb540100640 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.270+0000 7fb547da1640 1 -- 172.21.15.94:0/4247980901 >> 172.21.15.94:0/4247980901 conn(0x7fb540077180 msgr2=0x7fb540077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:03.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.270+0000 7fb547da1640 1 -- 172.21.15.94:0/4247980901 shutdown_connections 2024-02-17T11:59:03.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.271+0000 7fb547da1640 1 -- 172.21.15.94:0/4247980901 wait complete. 2024-02-17T11:59:03.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.272+0000 7fb547da1640 1 Processor -- start 2024-02-17T11:59:03.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.272+0000 7fb547da1640 1 -- start start 2024-02-17T11:59:03.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.272+0000 7fb547da1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb540100260 0x7fb5401966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.272+0000 7fb547da1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb540105330 0x7fb540196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.272+0000 7fb547da1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb54019af90 0x7fb54019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.272+0000 7fb54533c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb540105330 0x7fb540196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.272+0000 7fb547da1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54019b980 con 0x7fb540105330 2024-02-17T11:59:03.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.272+0000 7fb547da1640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54019baf0 con 0x7fb54019af90 2024-02-17T11:59:03.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.272+0000 7fb547da1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54019bc60 con 0x7fb540100260 2024-02-17T11:59:03.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.272+0000 7fb54533c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb540105330 0x7fb540196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35428/0 (socket says 172.21.15.94:35428) 2024-02-17T11:59:03.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.272+0000 7fb54533c640 1 -- 172.21.15.94:0/3169807958 learned_addr learned my addr 172.21.15.94:0/3169807958 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:03.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.272+0000 7fb545b3d640 1 --2- 172.21.15.94:0/3169807958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb540100260 0x7fb5401966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.273+0000 7fb545b3d640 1 -- 172.21.15.94:0/3169807958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb54019af90 msgr2=0x7fb54019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.273+0000 7fb545b3d640 1 --2- 172.21.15.94:0/3169807958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb54019af90 0x7fb54019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.273+0000 7fb545b3d640 1 -- 172.21.15.94:0/3169807958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb540105330 msgr2=0x7fb540196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.273+0000 7fb545b3d640 1 --2- 172.21.15.94:0/3169807958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb540105330 0x7fb540196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.273+0000 7fb545b3d640 1 -- 172.21.15.94:0/3169807958 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53c009560 con 0x7fb540100260 2024-02-17T11:59:03.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.274+0000 7fb545b3d640 1 --2- 172.21.15.94:0/3169807958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb540100260 0x7fb5401966d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009230 tx=0x7fb53c02c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:03.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.274+0000 7fb536ffd640 1 -- 172.21.15.94:0/3169807958 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0042d0 con 0x7fb540100260 2024-02-17T11:59:03.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.274+0000 7fb536ffd640 1 -- 172.21.15.94:0/3169807958 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb53c004430 con 0x7fb540100260 2024-02-17T11:59:03.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.274+0000 7fb536ffd640 1 -- 172.21.15.94:0/3169807958 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c003bd0 con 0x7fb540100260 2024-02-17T11:59:03.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.274+0000 7fb547da1640 1 -- 172.21.15.94:0/3169807958 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5401a3240 con 0x7fb540100260 2024-02-17T11:59:03.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.274+0000 7fb547da1640 1 -- 172.21.15.94:0/3169807958 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5400684c0 con 0x7fb540100260 2024-02-17T11:59:03.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.277+0000 7fb536ffd640 1 -- 172.21.15.94:0/3169807958 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fb53c003d30 con 0x7fb540100260 2024-02-17T11:59:03.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.278+0000 7fb54a97b740 1 Processor -- start 2024-02-17T11:59:03.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb54a97b740 1 -- start start 2024-02-17T11:59:03.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb536ffd640 1 --2- 172.21.15.94:0/3169807958 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb52c0a2520 0x7fb52c0a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb54a97b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebcbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb54a97b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cebd4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb54a97b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebd4f60 0x55c9cebd7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb54a97b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9ceb7e960 con 0x55c9cebcc380 2024-02-17T11:59:03.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb54a97b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9ceb32310 con 0x55c9cebcba60 2024-02-17T11:59:03.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb54a97b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9ceb2f840 con 0x55c9cebd4f60 2024-02-17T11:59:03.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb536ffd640 1 -- 172.21.15.94:0/3169807958 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb53c0f0240 con 0x7fb540100260 2024-02-17T11:59:03.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb54633e640 1 --2- 172.21.15.94:0/3169807958 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb52c0a2520 0x7fb52c0a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb545b3d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebcbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb545b3d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebcbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:51948/0 (socket says 172.21.15.94:51948) 2024-02-17T11:59:03.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb545b3d640 1 -- 172.21.15.94:0/387949371 learned_addr learned my addr 172.21.15.94:0/387949371 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:03.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb54533c640 1 --2- 172.21.15.94:0/387949371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cebd4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.279+0000 7fb54633e640 1 --2- 172.21.15.94:0/387949371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebd4f60 0x55c9cebd7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.280+0000 7fb545b3d640 1 -- 172.21.15.94:0/387949371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebd4f60 msgr2=0x55c9cebd7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.280+0000 7fb545b3d640 1 --2- 172.21.15.94:0/387949371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebd4f60 0x55c9cebd7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.280+0000 7fb545b3d640 1 -- 172.21.15.94:0/387949371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 msgr2=0x55c9cebd4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.280+0000 7fb545b3d640 1 --2- 172.21.15.94:0/387949371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cebd4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.280+0000 7fb545b3d640 1 -- 172.21.15.94:0/387949371 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9ceaed060 con 0x55c9cebcba60 2024-02-17T11:59:03.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.280+0000 7fb54533c640 1 --2- 172.21.15.94:0/387949371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cebd4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:03.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.280+0000 7fb545b3d640 1 --2- 172.21.15.94:0/387949371 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebcbe40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0b8ee0 tx=0x7fb53c02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeabb9c25db600ad server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:03.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.280+0000 7fb52bfff640 1 -- 172.21.15.94:0/387949371 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0c0bd0 con 0x55c9cebcba60 2024-02-17T11:59:03.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.280+0000 7fb52bfff640 1 -- 172.21.15.94:0/387949371 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb53c0c5220 con 0x55c9cebcba60 2024-02-17T11:59:03.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.280+0000 7fb52bfff640 1 -- 172.21.15.94:0/387949371 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c0c4ff0 con 0x55c9cebcba60 2024-02-17T11:59:03.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.281+0000 7fb54633e640 1 --2- 172.21.15.94:0/3169807958 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb52c0a2520 0x7fb52c0a49e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb538009270 tx=0x7fb538009d30 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:03.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.281+0000 7fb54a97b740 1 -- 172.21.15.94:0/387949371 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 msgr2=0x55c9cebcbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.281+0000 7fb54a97b740 1 --2- 172.21.15.94:0/387949371 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebcbe40 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0b8ee0 tx=0x7fb53c02bc30 comp rx=0 tx=0).stop 2024-02-17T11:59:03.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.282+0000 7fb54a97b740 1 -- 172.21.15.94:0/387949371 shutdown_connections 2024-02-17T11:59:03.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.282+0000 7fb54a97b740 1 --2- 172.21.15.94:0/387949371 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebd4f60 0x55c9cebd7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.282+0000 7fb54a97b740 1 --2- 172.21.15.94:0/387949371 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cebd4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.282+0000 7fb54a97b740 1 --2- 172.21.15.94:0/387949371 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebcbe40 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.282+0000 7fb54a97b740 1 -- 172.21.15.94:0/387949371 >> 172.21.15.94:0/387949371 conn(0x55c9cebc6190 msgr2=0x55c9cebc6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.282+0000 7fb54a97b740 1 -- 172.21.15.94:0/387949371 shutdown_connections 2024-02-17T11:59:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.282+0000 7fb54a97b740 1 -- 172.21.15.94:0/387949371 wait complete. 2024-02-17T11:59:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.282+0000 7fb54a97b740 1 -- start start 2024-02-17T11:59:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.283+0000 7fb54a97b740 1 Processor -- start 2024-02-17T11:59:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.283+0000 7fb54a97b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebd1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.283+0000 7fb54a97b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebcc380 0x55c9cebd2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.283+0000 7fb54a97b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebd4f60 0x55c9cebd2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.283+0000 7fb54533c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebcc380 0x55c9cebd2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.283+0000 7fb54633e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebd4f60 0x55c9cebd2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.283+0000 7fb54533c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebcc380 0x55c9cebd2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44022/0 (socket says 172.21.15.94:44022) 2024-02-17T11:59:03.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.283+0000 7fb54633e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebd4f60 0x55c9cebd2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35450/0 (socket says 172.21.15.94:35450) 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.283+0000 7fb54633e640 1 -- 172.21.15.94:0/3536790235 learned_addr learned my addr 172.21.15.94:0/3536790235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.283+0000 7fb54a97b740 1 -- 172.21.15.94:0/3536790235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9ceb2b020 con 0x55c9cebd4f60 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.283+0000 7fb545b3d640 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebd1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.283+0000 7fb54a97b740 1 -- 172.21.15.94:0/3536790235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9ce7fe430 con 0x55c9cebcba60 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.284+0000 7fb54a97b740 1 -- 172.21.15.94:0/3536790235 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9ce7bae00 con 0x55c9cebcc380 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.284+0000 7fb54a97b740 10 client.0 _ll_register_callbacks cb 0x55c9cead2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.284+0000 7fb54633e640 1 -- 172.21.15.94:0/3536790235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebcc380 msgr2=0x55c9cebd2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.284+0000 7fb54633e640 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebcc380 0x55c9cebd2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.284+0000 7fb54633e640 1 -- 172.21.15.94:0/3536790235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 msgr2=0x55c9cebd1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.284+0000 7fb54633e640 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebd1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.284+0000 7fb54633e640 1 -- 172.21.15.94:0/3536790235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5400684c0 con 0x55c9cebd4f60 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.284+0000 7fb54533c640 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebcc380 0x55c9cebd2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.284+0000 7fb545b3d640 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebd1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.284+0000 7fb54633e640 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebd4f60 0x55c9cebd2a60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb538003bf0 tx=0x7fb538009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262a7198c0668a07 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.284+0000 7fb5297fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:59:03.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.284+0000 7fb5297fa640 1 -- 172.21.15.94:0/3536790235 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538004440 con 0x55c9cebd4f60 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.284+0000 7fb5297fa640 1 -- 172.21.15.94:0/3536790235 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb538024070 con 0x55c9cebd4f60 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.285+0000 7fb54a97b740 1 -- 172.21.15.94:0/3536790235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9ce82a7e0 con 0x55c9cebd4f60 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.285+0000 7fb5297fa640 1 -- 172.21.15.94:0/3536790235 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5380045a0 con 0x55c9cebd4f60 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.285+0000 7fb54a97b740 10 client.15765 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.285+0000 7fb54a97b740 1 -- 172.21.15.94:0/3536790235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55c9ceb296a0 con 0x55c9cebd4f60 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.286+0000 7fb5297fa640 1 -- 172.21.15.94:0/3536790235 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c9ce82a7e0 con 0x55c9cebd4f60 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.286+0000 7fb5297fa640 1 -- 172.21.15.94:0/3536790235 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb53801c070 con 0x55c9cebd4f60 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.286+0000 7fb54a97b740 10 client.15765 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.286+0000 7fb54a97b740 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.286+0000 7fb54a97b740 20 client.15765 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.286+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c9cebe5210 0x55c9cebe7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.286+0000 7fb54a97b740 4 client.15765 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.286+0000 7fb54a97b740 1 -- 172.21.15.94:0/3536790235 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9ce867720 con 0x55c9cebe5210 2024-02-17T11:59:03.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.286+0000 7fb545b3d640 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c9cebe5210 0x55c9cebe7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.286+0000 7fb5297fa640 10 client.15765 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:03.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.287+0000 7fb545b3d640 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c9cebe5210 0x55c9cebe7630 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32b221acceb9fa4f server_cookie=1e0ffcf05f0e8f82 in_seq=0 out_seq=0 2024-02-17T11:59:03.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb5297fa640 1 -- 172.21.15.94:0/3536790235 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c9ce867720 con 0x55c9cebe5210 2024-02-17T11:59:03.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb5297fa640 10 client.15765 handle_command_reply: tid=0 2024-02-17T11:59:03.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb54a97b740 1 client.15765 shutdown 2024-02-17T11:59:03.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb54a97b740 1 -- 172.21.15.94:0/3536790235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebd4f60 msgr2=0x55c9cebd2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebd4f60 0x55c9cebd2a60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb538003bf0 tx=0x7fb538009500 comp rx=0 tx=0).stop 2024-02-17T11:59:03.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb54a97b740 1 -- 172.21.15.94:0/3536790235 shutdown_connections 2024-02-17T11:59:03.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c9cebe5210 0x55c9cebe7630 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebd4f60 0x55c9cebd2a60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb5297fa640 0 client.15765 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:03.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebcc380 0x55c9cebd2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3536790235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebd1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb54a97b740 1 -- 172.21.15.94:0/3536790235 >> 172.21.15.94:0/3536790235 conn(0x55c9cebc6190 msgr2=0x55c9cebc7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:03.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb54a97b740 1 -- 172.21.15.94:0/3536790235 shutdown_connections 2024-02-17T11:59:03.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb54a97b740 1 -- 172.21.15.94:0/3536790235 wait complete. 2024-02-17T11:59:03.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.288+0000 7fb54a97b740 20 client.15765 trim_cache size 0 max 16384 2024-02-17T11:59:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.303+0000 7fb54a97b740 1 Processor -- start 2024-02-17T11:59:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.303+0000 7fb54a97b740 1 -- start start 2024-02-17T11:59:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.303+0000 7fb54a97b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebd4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.303+0000 7fb54a97b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cebdbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.303+0000 7fb54a97b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebd4f60 0x55c9cebdc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.303+0000 7fb54a97b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9ceab0130 con 0x55c9cebcc380 2024-02-17T11:59:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.303+0000 7fb54a97b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cebe1c80 con 0x55c9cebcba60 2024-02-17T11:59:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.303+0000 7fb54a97b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cebcade0 con 0x55c9cebd4f60 2024-02-17T11:59:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.303+0000 7fb54533c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cebdbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.303+0000 7fb545b3d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebd4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.303+0000 7fb54533c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cebdbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35456/0 (socket says 172.21.15.94:35456) 2024-02-17T11:59:03.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.303+0000 7fb54533c640 1 -- 172.21.15.94:0/3788558510 learned_addr learned my addr 172.21.15.94:0/3788558510 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:03.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.304+0000 7fb54633e640 1 --2- 172.21.15.94:0/3788558510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebd4f60 0x55c9cebdc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.304+0000 7fb54533c640 1 -- 172.21.15.94:0/3788558510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebd4f60 msgr2=0x55c9cebdc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.304+0000 7fb54533c640 1 --2- 172.21.15.94:0/3788558510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebd4f60 0x55c9cebdc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.304+0000 7fb54533c640 1 -- 172.21.15.94:0/3788558510 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 msgr2=0x55c9cebd4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.304+0000 7fb54533c640 1 --2- 172.21.15.94:0/3788558510 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebd4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.304+0000 7fb54533c640 1 -- 172.21.15.94:0/3788558510 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9ceac5c20 con 0x55c9cebcc380 2024-02-17T11:59:03.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.304+0000 7fb545b3d640 1 --2- 172.21.15.94:0/3788558510 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebd4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:03.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.304+0000 7fb54533c640 1 --2- 172.21.15.94:0/3788558510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cebdbbd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb530012530 tx=0x7fb530005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c10e24e60b046b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:03.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.304+0000 7fb52b7fe640 1 -- 172.21.15.94:0/3788558510 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530020070 con 0x55c9cebcc380 2024-02-17T11:59:03.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.304+0000 7fb52b7fe640 1 -- 172.21.15.94:0/3788558510 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb530018040 con 0x55c9cebcc380 2024-02-17T11:59:03.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.304+0000 7fb52b7fe640 1 -- 172.21.15.94:0/3788558510 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb530017050 con 0x55c9cebcc380 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.305+0000 7fb54a97b740 1 -- 172.21.15.94:0/3788558510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 msgr2=0x55c9cebdbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.305+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3788558510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cebdbbd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb530012530 tx=0x7fb530005f90 comp rx=0 tx=0).stop 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.305+0000 7fb54a97b740 1 -- 172.21.15.94:0/3788558510 shutdown_connections 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.305+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3788558510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebd4f60 0x55c9cebdc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.305+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3788558510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cebdbbd0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.305+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3788558510 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebcba60 0x55c9cebd4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.305+0000 7fb54a97b740 1 -- 172.21.15.94:0/3788558510 >> 172.21.15.94:0/3788558510 conn(0x55c9cebc6190 msgr2=0x55c9cebcd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.305+0000 7fb54a97b740 1 -- 172.21.15.94:0/3788558510 shutdown_connections 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.305+0000 7fb54a97b740 1 -- 172.21.15.94:0/3788558510 wait complete. 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.305+0000 7fb54a97b740 1 -- start start 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.305+0000 7fb54a97b740 1 Processor -- start 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54a97b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebcba60 0x55c9cec35aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54a97b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cec35fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54a97b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebd4f60 0x55c9cebdadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54a97b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cebdb3d0 con 0x55c9cebcc380 2024-02-17T11:59:03.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54a97b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cebdb540 con 0x55c9cebd4f60 2024-02-17T11:59:03.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54a97b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9cebdb700 con 0x55c9cebcba60 2024-02-17T11:59:03.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54a97b740 10 client.0 _ll_register_callbacks cb 0x7fb53c0045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:03.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54533c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cec35fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54633e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebd4f60 0x55c9cebdadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54533c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cec35fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35458/0 (socket says 172.21.15.94:35458) 2024-02-17T11:59:03.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54533c640 1 -- 172.21.15.94:0/3298580300 learned_addr learned my addr 172.21.15.94:0/3298580300 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:03.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb545b3d640 1 --2- 172.21.15.94:0/3298580300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebcba60 0x55c9cec35aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54633e640 1 -- 172.21.15.94:0/3298580300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebcba60 msgr2=0x55c9cec35aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54633e640 1 --2- 172.21.15.94:0/3298580300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebcba60 0x55c9cec35aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54633e640 1 -- 172.21.15.94:0/3298580300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 msgr2=0x55c9cec35fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:03.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54633e640 1 --2- 172.21.15.94:0/3298580300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cec35fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.306+0000 7fb54633e640 1 -- 172.21.15.94:0/3298580300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9ceaed060 con 0x55c9cebd4f60 2024-02-17T11:59:03.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.307+0000 7fb54633e640 1 --2- 172.21.15.94:0/3298580300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebd4f60 0x55c9cebdadc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb538020f80 tx=0x7fb538010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71d9cafb5bd51c3a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:03.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.307+0000 7fb5297fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:59:03.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.307+0000 7fb5297fa640 1 -- 172.21.15.94:0/3298580300 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538033b40 con 0x55c9cebd4f60 2024-02-17T11:59:03.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.307+0000 7fb54a97b740 1 -- 172.21.15.94:0/3298580300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb53c0c5220 con 0x55c9cebd4f60 2024-02-17T11:59:03.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.307+0000 7fb5297fa640 1 -- 172.21.15.94:0/3298580300 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb538039030 con 0x55c9cebd4f60 2024-02-17T11:59:03.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.307+0000 7fb5297fa640 1 -- 172.21.15.94:0/3298580300 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb538009ec0 con 0x55c9cebd4f60 2024-02-17T11:59:03.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.307+0000 7fb54a97b740 10 client.25244 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:03.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.307+0000 7fb54a97b740 1 -- 172.21.15.94:0/3298580300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x7fb530009d90 con 0x55c9cebd4f60 2024-02-17T11:59:03.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.308+0000 7fb5297fa640 1 -- 172.21.15.94:0/3298580300 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb53c0c5220 con 0x55c9cebd4f60 2024-02-17T11:59:03.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.308+0000 7fb5297fa640 1 -- 172.21.15.94:0/3298580300 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb53801c040 con 0x55c9cebd4f60 2024-02-17T11:59:03.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.308+0000 7fb54a97b740 10 client.25244 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:03.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.308+0000 7fb54a97b740 10 client.25244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:03.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.308+0000 7fb54a97b740 20 client.25244 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:03.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.308+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3298580300 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c9cec3d8c0 0x55c9cec3fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:03.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.308+0000 7fb54a97b740 4 client.25244 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T11:59:03.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.308+0000 7fb54a97b740 1 -- 172.21.15.94:0/3298580300 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c9ce8b6890 con 0x55c9cec3d8c0 2024-02-17T11:59:03.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.308+0000 7fb545b3d640 1 --2- 172.21.15.94:0/3298580300 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c9cec3d8c0 0x55c9cec3fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:03.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb545b3d640 1 --2- 172.21.15.94:0/3298580300 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c9cec3d8c0 0x55c9cec3fce0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=767c20ebc4bccf0 server_cookie=c07d756bd75d4982 in_seq=0 out_seq=0 2024-02-17T11:59:03.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb5297fa640 10 client.25244 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:03.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb5297fa640 1 -- 172.21.15.94:0/3298580300 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c9ce8b6890 con 0x55c9cec3d8c0 2024-02-17T11:59:03.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb5297fa640 10 client.25244 handle_command_reply: tid=0 2024-02-17T11:59:03.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb54a97b740 1 client.25244 shutdown 2024-02-17T11:59:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb54a97b740 1 -- 172.21.15.94:0/3298580300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebd4f60 msgr2=0x55c9cebdadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3298580300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebd4f60 0x55c9cebdadc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb538020f80 tx=0x7fb538010860 comp rx=0 tx=0).stop 2024-02-17T11:59:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb54a97b740 1 -- 172.21.15.94:0/3298580300 shutdown_connections 2024-02-17T11:59:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3298580300 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c9cec3d8c0 0x55c9cec3fce0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3298580300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9cebd4f60 0x55c9cebdadc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3298580300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9cebcc380 0x55c9cec35fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb5297fa640 0 client.25244 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb54a97b740 1 --2- 172.21.15.94:0/3298580300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9cebcba60 0x55c9cec35aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb54a97b740 1 -- 172.21.15.94:0/3298580300 >> 172.21.15.94:0/3298580300 conn(0x55c9cebc6190 msgr2=0x55c9cebc75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb54a97b740 1 -- 172.21.15.94:0/3298580300 shutdown_connections 2024-02-17T11:59:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb54a97b740 1 -- 172.21.15.94:0/3298580300 wait complete. 2024-02-17T11:59:03.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.309+0000 7fb54a97b740 20 client.25244 trim_cache size 0 max 16384 2024-02-17T11:59:03.313 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T11:59:03.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.310+0000 7fb547da1640 1 -- 172.21.15.94:0/3169807958 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb52c0a2520 msgr2=0x7fb52c0a49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.310+0000 7fb547da1640 1 --2- 172.21.15.94:0/3169807958 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb52c0a2520 0x7fb52c0a49e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb538009270 tx=0x7fb538009d30 comp rx=0 tx=0).stop 2024-02-17T11:59:03.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.310+0000 7fb547da1640 1 -- 172.21.15.94:0/3169807958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb540100260 msgr2=0x7fb5401966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:03.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.310+0000 7fb547da1640 1 --2- 172.21.15.94:0/3169807958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb540100260 0x7fb5401966d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009230 tx=0x7fb53c02c5d0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.310+0000 7fb547da1640 1 -- 172.21.15.94:0/3169807958 shutdown_connections 2024-02-17T11:59:03.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.310+0000 7fb547da1640 1 --2- 172.21.15.94:0/3169807958 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb52c0a2520 0x7fb52c0a49e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.310+0000 7fb547da1640 1 --2- 172.21.15.94:0/3169807958 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb54019af90 0x7fb54019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.310+0000 7fb547da1640 1 --2- 172.21.15.94:0/3169807958 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb540105330 0x7fb540196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.310+0000 7fb547da1640 1 --2- 172.21.15.94:0/3169807958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb540100260 0x7fb5401966d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:03.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.310+0000 7fb547da1640 1 -- 172.21.15.94:0/3169807958 >> 172.21.15.94:0/3169807958 conn(0x7fb540077180 msgr2=0x7fb5400fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:03.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.310+0000 7fb547da1640 1 -- 172.21.15.94:0/3169807958 shutdown_connections 2024-02-17T11:59:03.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:03.310+0000 7fb547da1640 1 -- 172.21.15.94:0/3169807958 wait complete. 2024-02-17T11:59:04.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:03 smithi189 ceph-mon[36588]: pgmap v31: 97 pgs: 97 active+clean; 996 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 914 op/s 2024-02-17T11:59:04.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:03 smithi125 ceph-mon[36819]: pgmap v31: 97 pgs: 97 active+clean; 996 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 914 op/s 2024-02-17T11:59:04.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:03 smithi094 ceph-mon[32499]: pgmap v31: 97 pgs: 97 active+clean; 996 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 914 op/s 2024-02-17T11:59:04.322 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T11:59:04.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:59:04.322 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T11:59:04.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.466+0000 7fa289db6640 1 -- 172.21.15.94:0/1888283740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa284101360 msgr2=0x7fa284101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.466+0000 7fa289db6640 1 --2- 172.21.15.94:0/1888283740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa284101360 0x7fa284101740 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa27400fb40 tx=0x7fa27402fda0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.466+0000 7fa289db6640 1 -- 172.21.15.94:0/1888283740 shutdown_connections 2024-02-17T11:59:04.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.466+0000 7fa289db6640 1 --2- 172.21.15.94:0/1888283740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa284106430 0x7fa28410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.466+0000 7fa289db6640 1 --2- 172.21.15.94:0/1888283740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa284101d10 0x7fa284105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.466+0000 7fa289db6640 1 --2- 172.21.15.94:0/1888283740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa284101360 0x7fa284101740 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.466+0000 7fa289db6640 1 -- 172.21.15.94:0/1888283740 >> 172.21.15.94:0/1888283740 conn(0x7fa2840fcbe0 msgr2=0x7fa2840ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:04.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.466+0000 7fa289db6640 1 -- 172.21.15.94:0/1888283740 shutdown_connections 2024-02-17T11:59:04.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.466+0000 7fa289db6640 1 -- 172.21.15.94:0/1888283740 wait complete. 2024-02-17T11:59:04.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.467+0000 7fa289db6640 1 Processor -- start 2024-02-17T11:59:04.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.467+0000 7fa289db6640 1 -- start start 2024-02-17T11:59:04.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.467+0000 7fa289db6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa284101360 0x7fa2841956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.467+0000 7fa289db6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa284106430 0x7fa284195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.467+0000 7fa289db6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa284199f90 0x7fa28419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.467+0000 7fa289db6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28419a980 con 0x7fa284199f90 2024-02-17T11:59:04.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.467+0000 7fa289db6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28419aaf0 con 0x7fa284101360 2024-02-17T11:59:04.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.467+0000 7fa289db6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28419ac60 con 0x7fa284106430 2024-02-17T11:59:04.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.467+0000 7fa283fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa284199f90 0x7fa28419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.467+0000 7fa283fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa284199f90 0x7fa28419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55206/0 (socket says 172.21.15.94:55206) 2024-02-17T11:59:04.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.467+0000 7fa283fff640 1 -- 172.21.15.94:0/2581896255 learned_addr learned my addr 172.21.15.94:0/2581896255 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:04.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.467+0000 7fa282ffd640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa284106430 0x7fa284195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.467+0000 7fa2837fe640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa284101360 0x7fa2841956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.468+0000 7fa283fff640 1 -- 172.21.15.94:0/2581896255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa284106430 msgr2=0x7fa284195c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.468+0000 7fa283fff640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa284106430 0x7fa284195c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.468+0000 7fa283fff640 1 -- 172.21.15.94:0/2581896255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa284101360 msgr2=0x7fa2841956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.468+0000 7fa283fff640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa284101360 0x7fa2841956d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.468+0000 7fa283fff640 1 -- 172.21.15.94:0/2581896255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c009840 con 0x7fa284199f90 2024-02-17T11:59:04.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.468+0000 7fa2837fe640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa284101360 0x7fa2841956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:59:04.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.468+0000 7fa283fff640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa284199f90 0x7fa28419a440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00efd0 tx=0x7fa27c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:04.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.468+0000 7fa280ff9640 1 -- 172.21.15.94:0/2581896255 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c009b40 con 0x7fa284199f90 2024-02-17T11:59:04.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.468+0000 7fa289db6640 1 -- 172.21.15.94:0/2581896255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27400f770 con 0x7fa284199f90 2024-02-17T11:59:04.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.468+0000 7fa280ff9640 1 -- 172.21.15.94:0/2581896255 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa27c003b70 con 0x7fa284199f90 2024-02-17T11:59:04.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.468+0000 7fa289db6640 1 -- 172.21.15.94:0/2581896255 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2841a2560 con 0x7fa284199f90 2024-02-17T11:59:04.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.468+0000 7fa280ff9640 1 -- 172.21.15.94:0/2581896255 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0108d0 con 0x7fa284199f90 2024-02-17T11:59:04.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.473+0000 7fa280ff9640 1 -- 172.21.15.94:0/2581896255 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fa27c003ce0 con 0x7fa284199f90 2024-02-17T11:59:04.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.473+0000 7fa28c987740 1 Processor -- start 2024-02-17T11:59:04.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.474+0000 7fa28c987740 1 -- start start 2024-02-17T11:59:04.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.474+0000 7fa28c987740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e12320 0x55f971e12700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.474+0000 7fa28c987740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e23af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.474+0000 7fa28c987740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e240c0 0x55f971e264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.474+0000 7fa28c987740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f971d80f90 con 0x55f971e12320 2024-02-17T11:59:04.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.474+0000 7fa282ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e12320 0x55f971e12700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.474+0000 7fa282ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e12320 0x55f971e12700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55208/0 (socket says 172.21.15.94:55208) 2024-02-17T11:59:04.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.474+0000 7fa282ffd640 1 -- 172.21.15.94:0/1469928436 learned_addr learned my addr 172.21.15.94:0/1469928436 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:04.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.474+0000 7fa28c987740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f971d7e4c0 con 0x55f971e12cd0 2024-02-17T11:59:04.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.474+0000 7fa28c987740 1 -- 172.21.15.94:0/1469928436 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f971d79ca0 con 0x55f971e240c0 2024-02-17T11:59:04.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.474+0000 7fa2837fe640 1 --2- 172.21.15.94:0/1469928436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e23af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.475+0000 7fa280ff9640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa26c0a2420 0x7fa26c0a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.475+0000 7fa282ffd640 1 --2- 172.21.15.94:0/1469928436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e240c0 0x55f971e264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.475+0000 7fa280ff9640 1 -- 172.21.15.94:0/2581896255 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa27c014070 con 0x7fa284199f90 2024-02-17T11:59:04.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.475+0000 7fa283fff640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa26c0a2420 0x7fa26c0a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.475+0000 7fa2837fe640 1 -- 172.21.15.94:0/1469928436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e240c0 msgr2=0x55f971e264d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.475+0000 7fa2837fe640 1 --2- 172.21.15.94:0/1469928436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e240c0 0x55f971e264d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.475+0000 7fa2837fe640 1 -- 172.21.15.94:0/1469928436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e12320 msgr2=0x55f971e12700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.475+0000 7fa2837fe640 1 --2- 172.21.15.94:0/1469928436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e12320 0x55f971e12700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.475+0000 7fa2837fe640 1 -- 172.21.15.94:0/1469928436 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f971d3c0b0 con 0x55f971e12cd0 2024-02-17T11:59:04.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.475+0000 7fa2837fe640 1 --2- 172.21.15.94:0/1469928436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e23af0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa274006010 tx=0x7fa274004bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce495bd8ab37ba2b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:04.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.476+0000 7fa283fff640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa26c0a2420 0x7fa26c0a48e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0cc300 tx=0x7fa27c009f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:04.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.476+0000 7fa269ffb640 1 -- 172.21.15.94:0/1469928436 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2740048c0 con 0x55f971e12cd0 2024-02-17T11:59:04.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.476+0000 7fa269ffb640 1 -- 172.21.15.94:0/1469928436 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa274004a20 con 0x55f971e12cd0 2024-02-17T11:59:04.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.476+0000 7fa269ffb640 1 -- 172.21.15.94:0/1469928436 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274030ac0 con 0x55f971e12cd0 2024-02-17T11:59:04.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.477+0000 7fa28c987740 1 -- 172.21.15.94:0/1469928436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 msgr2=0x55f971e23af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.477+0000 7fa28c987740 1 --2- 172.21.15.94:0/1469928436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e23af0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa274006010 tx=0x7fa274004bf0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.477+0000 7fa28c987740 1 -- 172.21.15.94:0/1469928436 shutdown_connections 2024-02-17T11:59:04.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.477+0000 7fa28c987740 1 --2- 172.21.15.94:0/1469928436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e240c0 0x55f971e264d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.477+0000 7fa28c987740 1 --2- 172.21.15.94:0/1469928436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e23af0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.477+0000 7fa28c987740 1 --2- 172.21.15.94:0/1469928436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e12320 0x55f971e12700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.477+0000 7fa28c987740 1 -- 172.21.15.94:0/1469928436 >> 172.21.15.94:0/1469928436 conn(0x55f971e0c840 msgr2=0x55f971e0cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:04.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.477+0000 7fa28c987740 1 -- 172.21.15.94:0/1469928436 shutdown_connections 2024-02-17T11:59:04.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.478+0000 7fa28c987740 1 -- 172.21.15.94:0/1469928436 wait complete. 2024-02-17T11:59:04.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.478+0000 7fa28c987740 1 -- start start 2024-02-17T11:59:04.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.478+0000 7fa28c987740 1 Processor -- start 2024-02-17T11:59:04.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa28c987740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12320 0x55f971e21090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa28c987740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e12cd0 0x55f971e215d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa28c987740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e240c0 0x55f971e21b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa28c987740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f971d1ef40 con 0x55f971e12cd0 2024-02-17T11:59:04.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa2837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e12cd0 0x55f971e215d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa2837fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e12cd0 0x55f971e215d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55216/0 (socket says 172.21.15.94:55216) 2024-02-17T11:59:04.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa2837fe640 1 -- 172.21.15.94:0/473746310 learned_addr learned my addr 172.21.15.94:0/473746310 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:04.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa28c987740 1 -- 172.21.15.94:0/473746310 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f971a4d4b0 con 0x55f971e12320 2024-02-17T11:59:04.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa28c987740 1 -- 172.21.15.94:0/473746310 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f971cff670 con 0x55f971e240c0 2024-02-17T11:59:04.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa282ffd640 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12320 0x55f971e21090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa28c987740 10 client.0 _ll_register_callbacks cb 0x55f971d6cf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:04.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa282ffd640 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e240c0 0x55f971e21b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa2837fe640 1 -- 172.21.15.94:0/473746310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e240c0 msgr2=0x55f971e21b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa2837fe640 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e240c0 0x55f971e21b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa2837fe640 1 -- 172.21.15.94:0/473746310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12320 msgr2=0x55f971e21090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa2837fe640 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12320 0x55f971e21090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa2837fe640 1 -- 172.21.15.94:0/473746310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28410e810 con 0x55f971e12cd0 2024-02-17T11:59:04.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa282ffd640 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12320 0x55f971e21090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:04.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa2837fe640 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e12cd0 0x55f971e215d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa27400fa60 tx=0x7fa274004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe67c2aca2a99dd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:04.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.479+0000 7fa262ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:59:04.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.480+0000 7fa262ffd640 1 -- 172.21.15.94:0/473746310 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27400f520 con 0x55f971e12cd0 2024-02-17T11:59:04.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.480+0000 7fa262ffd640 1 -- 172.21.15.94:0/473746310 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa274030740 con 0x55f971e12cd0 2024-02-17T11:59:04.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.480+0000 7fa28c987740 1 -- 172.21.15.94:0/473746310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f971d53f40 con 0x55f971e12cd0 2024-02-17T11:59:04.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.480+0000 7fa262ffd640 1 -- 172.21.15.94:0/473746310 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa274030900 con 0x55f971e12cd0 2024-02-17T11:59:04.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.480+0000 7fa28c987740 10 client.15783 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:04.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.480+0000 7fa28c987740 1 -- 172.21.15.94:0/473746310 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55f971d78320 con 0x55f971e12cd0 2024-02-17T11:59:04.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.481+0000 7fa262ffd640 1 -- 172.21.15.94:0/473746310 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa274040440 con 0x55f971e12cd0 2024-02-17T11:59:04.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.481+0000 7fa262ffd640 1 -- 172.21.15.94:0/473746310 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa274049430 con 0x55f971e12cd0 2024-02-17T11:59:04.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.481+0000 7fa28c987740 10 client.15783 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:04.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.481+0000 7fa28c987740 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:04.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.481+0000 7fa28c987740 20 client.15783 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:04.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.481+0000 7fa28c987740 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f971e34150 0x55f971e365f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.481+0000 7fa28c987740 4 client.15783 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:04.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.481+0000 7fa28c987740 1 -- 172.21.15.94:0/473746310 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f971d53f40 con 0x55f971e34150 2024-02-17T11:59:04.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.481+0000 7fa282ffd640 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f971e34150 0x55f971e365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.482+0000 7fa262ffd640 10 client.15783 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:04.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.482+0000 7fa282ffd640 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f971e34150 0x55f971e365f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=347a9141e3c37c35 server_cookie=ea1df1b77f5c87d3 in_seq=0 out_seq=0 2024-02-17T11:59:04.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.482+0000 7fa262ffd640 1 -- 172.21.15.94:0/473746310 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f971d53f40 con 0x55f971e34150 2024-02-17T11:59:04.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.482+0000 7fa262ffd640 10 client.15783 handle_command_reply: tid=0 2024-02-17T11:59:04.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.482+0000 7fa28c987740 1 client.15783 shutdown 2024-02-17T11:59:04.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.483+0000 7fa28c987740 1 -- 172.21.15.94:0/473746310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e12cd0 msgr2=0x55f971e215d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.483+0000 7fa28c987740 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e12cd0 0x55f971e215d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa27400fa60 tx=0x7fa274004a40 comp rx=0 tx=0).stop 2024-02-17T11:59:04.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.483+0000 7fa28c987740 1 -- 172.21.15.94:0/473746310 shutdown_connections 2024-02-17T11:59:04.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.483+0000 7fa28c987740 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f971e34150 0x55f971e365f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.483+0000 7fa28c987740 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e240c0 0x55f971e21b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.483+0000 7fa28c987740 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e12cd0 0x55f971e215d0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.483+0000 7fa28c987740 1 --2- 172.21.15.94:0/473746310 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12320 0x55f971e21090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.483+0000 7fa262ffd640 0 client.15783 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:04.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.483+0000 7fa28c987740 1 -- 172.21.15.94:0/473746310 >> 172.21.15.94:0/473746310 conn(0x55f971e0c840 msgr2=0x55f971e0e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:04.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.483+0000 7fa28c987740 1 -- 172.21.15.94:0/473746310 shutdown_connections 2024-02-17T11:59:04.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.483+0000 7fa28c987740 1 -- 172.21.15.94:0/473746310 wait complete. 2024-02-17T11:59:04.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.483+0000 7fa28c987740 20 client.15783 trim_cache size 0 max 16384 2024-02-17T11:59:04.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.508+0000 7fa28c987740 1 Processor -- start 2024-02-17T11:59:04.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.508+0000 7fa28c987740 1 -- start start 2024-02-17T11:59:04.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.508+0000 7fa28c987740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 0x55f971e21930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.508+0000 7fa28c987740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e1f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa282ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 0x55f971e21930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa282ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 0x55f971e21930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:54296/0 (socket says 172.21.15.94:54296) 2024-02-17T11:59:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa282ffd640 1 -- 172.21.15.94:0/1396304340 learned_addr learned my addr 172.21.15.94:0/1396304340 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa28c987740 1 --2- 172.21.15.94:0/1396304340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e240c0 0x55f971e1fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa28c987740 1 -- 172.21.15.94:0/1396304340 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f971e26ed0 con 0x55f971e240c0 2024-02-17T11:59:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa28c987740 1 -- 172.21.15.94:0/1396304340 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f971e0c0d0 con 0x55f971e12cd0 2024-02-17T11:59:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa28c987740 1 -- 172.21.15.94:0/1396304340 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f971e23b60 con 0x55f971e12320 2024-02-17T11:59:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa2837fe640 1 --2- 172.21.15.94:0/1396304340 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e1f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa282ffd640 1 --2- 172.21.15.94:0/1396304340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e240c0 0x55f971e1fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa2837fe640 1 -- 172.21.15.94:0/1396304340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 msgr2=0x55f971e21930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa2837fe640 1 --2- 172.21.15.94:0/1396304340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 0x55f971e21930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa2837fe640 1 -- 172.21.15.94:0/1396304340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e240c0 msgr2=0x55f971e1fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa2837fe640 1 --2- 172.21.15.94:0/1396304340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e240c0 0x55f971e1fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa2837fe640 1 -- 172.21.15.94:0/1396304340 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f971d14c00 con 0x55f971e12cd0 2024-02-17T11:59:04.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.509+0000 7fa282ffd640 1 --2- 172.21.15.94:0/1396304340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 0x55f971e21930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:04.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.510+0000 7fa2837fe640 1 --2- 172.21.15.94:0/1396304340 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e1f960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa274049840 tx=0x7fa27402fda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b55376319f9e45fe server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:04.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.510+0000 7fa2697fa640 1 -- 172.21.15.94:0/1396304340 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27405f070 con 0x55f971e12cd0 2024-02-17T11:59:04.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.510+0000 7fa2697fa640 1 -- 172.21.15.94:0/1396304340 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa274004bf0 con 0x55f971e12cd0 2024-02-17T11:59:04.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.510+0000 7fa2697fa640 1 -- 172.21.15.94:0/1396304340 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274049c20 con 0x55f971e12cd0 2024-02-17T11:59:04.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.511+0000 7fa28c987740 1 -- 172.21.15.94:0/1396304340 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 msgr2=0x55f971e1f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.511+0000 7fa28c987740 1 --2- 172.21.15.94:0/1396304340 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e1f960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa274049840 tx=0x7fa27402fda0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.512+0000 7fa28c987740 1 -- 172.21.15.94:0/1396304340 shutdown_connections 2024-02-17T11:59:04.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.512+0000 7fa28c987740 1 --2- 172.21.15.94:0/1396304340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e240c0 0x55f971e1fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.512+0000 7fa28c987740 1 --2- 172.21.15.94:0/1396304340 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e1f960 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.512+0000 7fa28c987740 1 --2- 172.21.15.94:0/1396304340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 0x55f971e21930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.512+0000 7fa28c987740 1 -- 172.21.15.94:0/1396304340 >> 172.21.15.94:0/1396304340 conn(0x55f971e0c840 msgr2=0x55f971e1c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:04.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.513+0000 7fa28c987740 1 -- 172.21.15.94:0/1396304340 shutdown_connections 2024-02-17T11:59:04.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.513+0000 7fa28c987740 1 -- 172.21.15.94:0/1396304340 wait complete. 2024-02-17T11:59:04.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.513+0000 7fa28c987740 1 -- start start 2024-02-17T11:59:04.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.514+0000 7fa28c987740 1 Processor -- start 2024-02-17T11:59:04.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.514+0000 7fa28c987740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 0x55f971e29410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.514+0000 7fa28c987740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e29950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.514+0000 7fa28c987740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e240c0 0x55f971e83750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.514+0000 7fa28c987740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f971e83c90 con 0x55f971e240c0 2024-02-17T11:59:04.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.514+0000 7fa28c987740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f971e83e00 con 0x55f971e12cd0 2024-02-17T11:59:04.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.514+0000 7fa28c987740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f971e83fc0 con 0x55f971e12320 2024-02-17T11:59:04.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.514+0000 7fa28c987740 10 client.0 _ll_register_callbacks cb 0x7fa278005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:04.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.514+0000 7fa282ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 0x55f971e29410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.514+0000 7fa282ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 0x55f971e29410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:54298/0 (socket says 172.21.15.94:54298) 2024-02-17T11:59:04.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.514+0000 7fa282ffd640 1 -- 172.21.15.94:0/2950520962 learned_addr learned my addr 172.21.15.94:0/2950520962 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:04.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.514+0000 7fa282ffd640 1 --2- 172.21.15.94:0/2950520962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e240c0 0x55f971e83750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.514+0000 7fa2837fe640 1 --2- 172.21.15.94:0/2950520962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e29950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.516+0000 7fa282ffd640 1 -- 172.21.15.94:0/2950520962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 msgr2=0x55f971e29950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.516+0000 7fa282ffd640 1 --2- 172.21.15.94:0/2950520962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e29950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.516+0000 7fa282ffd640 1 -- 172.21.15.94:0/2950520962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e240c0 msgr2=0x55f971e83750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.516+0000 7fa282ffd640 1 --2- 172.21.15.94:0/2950520962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e240c0 0x55f971e83750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.516+0000 7fa282ffd640 1 -- 172.21.15.94:0/2950520962 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27801b040 con 0x55f971e12320 2024-02-17T11:59:04.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.516+0000 7fa282ffd640 1 --2- 172.21.15.94:0/2950520962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 0x55f971e29410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa278000c00 tx=0x7fa2740049d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48d044079f08277 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:04.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.516+0000 7fa262ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T11:59:04.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.516+0000 7fa262ffd640 1 -- 172.21.15.94:0/2950520962 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa278018040 con 0x55f971e12320 2024-02-17T11:59:04.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.516+0000 7fa262ffd640 1 -- 172.21.15.94:0/2950520962 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa278023860 con 0x55f971e12320 2024-02-17T11:59:04.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.516+0000 7fa28c987740 1 -- 172.21.15.94:0/2950520962 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f971d9f900 con 0x55f971e12320 2024-02-17T11:59:04.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.516+0000 7fa262ffd640 1 -- 172.21.15.94:0/2950520962 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa278012070 con 0x55f971e12320 2024-02-17T11:59:04.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.516+0000 7fa28c987740 10 client.25318 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.516+0000 7fa28c987740 1 -- 172.21.15.94:0/2950520962 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55f971d78320 con 0x55f971e12320 2024-02-17T11:59:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.517+0000 7fa262ffd640 1 -- 172.21.15.94:0/2950520962 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa278005500 con 0x55f971e12320 2024-02-17T11:59:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.517+0000 7fa262ffd640 1 -- 172.21.15.94:0/2950520962 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa27802f070 con 0x55f971e12320 2024-02-17T11:59:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.517+0000 7fa28c987740 10 client.25318 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.517+0000 7fa28c987740 10 client.25318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.517+0000 7fa28c987740 20 client.25318 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.517+0000 7fa28c987740 1 --2- 172.21.15.94:0/2950520962 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f971e8b910 0x55f971e8dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.517+0000 7fa28c987740 4 client.25318 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T11:59:04.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.517+0000 7fa28c987740 1 -- 172.21.15.94:0/2950520962 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f971d9f900 con 0x55f971e8b910 2024-02-17T11:59:04.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.517+0000 7fa2837fe640 1 --2- 172.21.15.94:0/2950520962 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f971e8b910 0x55f971e8dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.518+0000 7fa2837fe640 1 --2- 172.21.15.94:0/2950520962 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f971e8b910 0x55f971e8dd30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89eb3376b954dcc4 server_cookie=f0ce4d71ad474073 in_seq=0 out_seq=0 2024-02-17T11:59:04.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.518+0000 7fa262ffd640 10 client.25318 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:04.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.519+0000 7fa262ffd640 1 -- 172.21.15.94:0/2950520962 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f971d9f900 con 0x55f971e8b910 2024-02-17T11:59:04.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.519+0000 7fa262ffd640 10 client.25318 handle_command_reply: tid=0 2024-02-17T11:59:04.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.519+0000 7fa28c987740 1 client.25318 shutdown 2024-02-17T11:59:04.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.520+0000 7fa28c987740 1 -- 172.21.15.94:0/2950520962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 msgr2=0x55f971e29410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.520+0000 7fa28c987740 1 --2- 172.21.15.94:0/2950520962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 0x55f971e29410 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa278000c00 tx=0x7fa2740049d0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.520+0000 7fa28c987740 1 -- 172.21.15.94:0/2950520962 shutdown_connections 2024-02-17T11:59:04.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.520+0000 7fa28c987740 1 --2- 172.21.15.94:0/2950520962 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f971e8b910 0x55f971e8dd30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.520+0000 7fa28c987740 1 --2- 172.21.15.94:0/2950520962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f971e240c0 0x55f971e83750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.520+0000 7fa28c987740 1 --2- 172.21.15.94:0/2950520962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f971e12cd0 0x55f971e29950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.520+0000 7fa28c987740 1 --2- 172.21.15.94:0/2950520962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f971e12320 0x55f971e29410 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa278000c00 tx=0x7fa2740049d0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.520+0000 7fa28c987740 1 -- 172.21.15.94:0/2950520962 >> 172.21.15.94:0/2950520962 conn(0x55f971e0c840 msgr2=0x55f971e1f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:04.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.520+0000 7fa262ffd640 0 client.25318 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:04.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.521+0000 7fa28c987740 1 -- 172.21.15.94:0/2950520962 shutdown_connections 2024-02-17T11:59:04.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.521+0000 7fa28c987740 1 -- 172.21.15.94:0/2950520962 wait complete. 2024-02-17T11:59:04.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.521+0000 7fa28c987740 20 client.25318 trim_cache size 0 max 16384 2024-02-17T11:59:04.527 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:59:04.527 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T11:59:04.527 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "24bf619e-7d2a-4b97-b6f5-b1b0ff420381", 2024-02-17T11:59:04.528 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T11:59:04.528 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:59:04.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.523+0000 7fa289db6640 1 -- 172.21.15.94:0/2581896255 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa26c0a2420 msgr2=0x7fa26c0a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.523+0000 7fa289db6640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa26c0a2420 0x7fa26c0a48e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa27c0cc300 tx=0x7fa27c009f90 comp rx=0 tx=0).stop 2024-02-17T11:59:04.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.523+0000 7fa289db6640 1 -- 172.21.15.94:0/2581896255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa284199f90 msgr2=0x7fa28419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.523+0000 7fa289db6640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa284199f90 0x7fa28419a440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00efd0 tx=0x7fa27c00c6a0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.523+0000 7fa289db6640 1 -- 172.21.15.94:0/2581896255 shutdown_connections 2024-02-17T11:59:04.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.523+0000 7fa289db6640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa26c0a2420 0x7fa26c0a48e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.523+0000 7fa289db6640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa284199f90 0x7fa28419a440 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.523+0000 7fa289db6640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa284106430 0x7fa284195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.523+0000 7fa289db6640 1 --2- 172.21.15.94:0/2581896255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa284101360 0x7fa2841956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.524+0000 7fa289db6640 1 -- 172.21.15.94:0/2581896255 >> 172.21.15.94:0/2581896255 conn(0x7fa2840fcbe0 msgr2=0x7fa2840fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:04.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.524+0000 7fa289db6640 1 -- 172.21.15.94:0/2581896255 shutdown_connections 2024-02-17T11:59:04.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.524+0000 7fa289db6640 1 -- 172.21.15.94:0/2581896255 wait complete. 2024-02-17T11:59:04.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T11:59:04.540 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T11:59:04.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.705+0000 7fadbfceb640 1 -- 172.21.15.94:0/2576683910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb8108670 msgr2=0x7fadb8110240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.705+0000 7fadbfceb640 1 --2- 172.21.15.94:0/2576683910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb8108670 0x7fadb8110240 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009cd0 tx=0x7fadb002c0f0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.705+0000 7fadbfceb640 1 -- 172.21.15.94:0/2576683910 shutdown_connections 2024-02-17T11:59:04.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.705+0000 7fadbfceb640 1 --2- 172.21.15.94:0/2576683910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb8108670 0x7fadb8110240 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.705+0000 7fadbfceb640 1 --2- 172.21.15.94:0/2576683910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb8103f50 0x7fadb8107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.705+0000 7fadbfceb640 1 --2- 172.21.15.94:0/2576683910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadb81035a0 0x7fadb8103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.705+0000 7fadbfceb640 1 -- 172.21.15.94:0/2576683910 >> 172.21.15.94:0/2576683910 conn(0x7fadb80fee00 msgr2=0x7fadb8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:04.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.705+0000 7fadbfceb640 1 -- 172.21.15.94:0/2576683910 shutdown_connections 2024-02-17T11:59:04.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.705+0000 7fadbfceb640 1 -- 172.21.15.94:0/2576683910 wait complete. 2024-02-17T11:59:04.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.706+0000 7fadbfceb640 1 Processor -- start 2024-02-17T11:59:04.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.706+0000 7fadbfceb640 1 -- start start 2024-02-17T11:59:04.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.706+0000 7fadbfceb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadb81035a0 0x7fadb819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.706+0000 7fadbfceb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb8108670 0x7fadb819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.706+0000 7fadbfceb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb8199880 0x7fadb8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.706+0000 7fadbfceb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb819a330 con 0x7fadb8199880 2024-02-17T11:59:04.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.706+0000 7fadbfceb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb819a4a0 con 0x7fadb81035a0 2024-02-17T11:59:04.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.706+0000 7fadbfceb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb819a610 con 0x7fadb8108670 2024-02-17T11:59:04.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.706+0000 7fadbd286640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb8108670 0x7fadb819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.707+0000 7fadbd286640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb8108670 0x7fadb819fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:54316/0 (socket says 172.21.15.94:54316) 2024-02-17T11:59:04.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.707+0000 7fadbd286640 1 -- 172.21.15.94:0/1548487631 learned_addr learned my addr 172.21.15.94:0/1548487631 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:04.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.707+0000 7fadbe288640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb8199880 0x7fadb8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.707+0000 7fadbda87640 1 --2- 172.21.15.94:0/1548487631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadb81035a0 0x7fadb819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.707+0000 7fadbd286640 1 -- 172.21.15.94:0/1548487631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadb81035a0 msgr2=0x7fadb819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.707+0000 7fadbd286640 1 --2- 172.21.15.94:0/1548487631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadb81035a0 0x7fadb819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.707+0000 7fadbd286640 1 -- 172.21.15.94:0/1548487631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb8199880 msgr2=0x7fadb8199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.707+0000 7fadbd286640 1 --2- 172.21.15.94:0/1548487631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb8199880 0x7fadb8199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.707+0000 7fadbd286640 1 -- 172.21.15.94:0/1548487631 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb0009900 con 0x7fadb8108670 2024-02-17T11:59:04.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.707+0000 7fadbd286640 1 --2- 172.21.15.94:0/1548487631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb8108670 0x7fadb819fc00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fada800e9f0 tx=0x7fada800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:04.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.707+0000 7fadaeffd640 1 -- 172.21.15.94:0/1548487631 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada80096c0 con 0x7fadb8108670 2024-02-17T11:59:04.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.707+0000 7fadaeffd640 1 -- 172.21.15.94:0/1548487631 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fada8005500 con 0x7fadb8108670 2024-02-17T11:59:04.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.707+0000 7fadbfceb640 1 -- 172.21.15.94:0/1548487631 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb81a6570 con 0x7fadb8108670 2024-02-17T11:59:04.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.708+0000 7fadaeffd640 1 -- 172.21.15.94:0/1548487631 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada8010430 con 0x7fadb8108670 2024-02-17T11:59:04.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.708+0000 7fadbfceb640 1 -- 172.21.15.94:0/1548487631 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb81a69f0 con 0x7fadb8108670 2024-02-17T11:59:04.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.711+0000 7fadaeffd640 1 -- 172.21.15.94:0/1548487631 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fada80046c0 con 0x7fadb8108670 2024-02-17T11:59:04.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.711+0000 7fadaeffd640 1 --2- 172.21.15.94:0/1548487631 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fada40a24a0 0x7fada40a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.712+0000 7fadaeffd640 1 -- 172.21.15.94:0/1548487631 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fada8014070 con 0x7fadb8108670 2024-02-17T11:59:04.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.712+0000 7fadbda87640 1 --2- 172.21.15.94:0/1548487631 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fada40a24a0 0x7fada40a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.712+0000 7fadbda87640 1 --2- 172.21.15.94:0/1548487631 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fada40a24a0 0x7fada40a4960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009b60 tx=0x7fadb4005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:04.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.713+0000 7fadc299a740 1 Processor -- start 2024-02-17T11:59:04.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.714+0000 7fadc299a740 1 -- start start 2024-02-17T11:59:04.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.714+0000 7fadc299a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1c0c0 0x562a0bf1c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.714+0000 7fadc299a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 0x562a0bf2da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.714+0000 7fadc299a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf2dfc0 0x562a0bf303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.714+0000 7fadc299a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0be8b280 con 0x562a0bf1ca70 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.714+0000 7fadc299a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0be887b0 con 0x562a0bf2dfc0 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.714+0000 7fadc299a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0be83f90 con 0x562a0bf1c0c0 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.714+0000 7fadbd286640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 0x562a0bf2da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.714+0000 7fadbd286640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 0x562a0bf2da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55270/0 (socket says 172.21.15.94:55270) 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.714+0000 7fadbd286640 1 -- 172.21.15.94:0/3928487682 learned_addr learned my addr 172.21.15.94:0/3928487682 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.714+0000 7fadbda87640 1 --2- 172.21.15.94:0/3928487682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1c0c0 0x562a0bf1c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.714+0000 7fadbe288640 1 --2- 172.21.15.94:0/3928487682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf2dfc0 0x562a0bf303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.715+0000 7fadbda87640 1 -- 172.21.15.94:0/3928487682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf2dfc0 msgr2=0x562a0bf303b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.715+0000 7fadbda87640 1 --2- 172.21.15.94:0/3928487682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf2dfc0 0x562a0bf303b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.715+0000 7fadbda87640 1 -- 172.21.15.94:0/3928487682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 msgr2=0x562a0bf2da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.715+0000 7fadbda87640 1 --2- 172.21.15.94:0/3928487682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 0x562a0bf2da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.715+0000 7fadbda87640 1 -- 172.21.15.94:0/3928487682 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a0be42570 con 0x562a0bf1c0c0 2024-02-17T11:59:04.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.715+0000 7fadbd286640 1 --2- 172.21.15.94:0/3928487682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 0x562a0bf2da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:04.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.715+0000 7fadbda87640 1 --2- 172.21.15.94:0/3928487682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1c0c0 0x562a0bf1c4a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fadb4011f00 tx=0x7fadb4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbb324500d60d08f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:04.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.715+0000 7fada3fff640 1 -- 172.21.15.94:0/3928487682 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4004220 con 0x562a0bf1c0c0 2024-02-17T11:59:04.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.715+0000 7fada3fff640 1 -- 172.21.15.94:0/3928487682 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadb4016070 con 0x562a0bf1c0c0 2024-02-17T11:59:04.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.715+0000 7fada3fff640 1 -- 172.21.15.94:0/3928487682 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb401f660 con 0x562a0bf1c0c0 2024-02-17T11:59:04.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.716+0000 7fadc299a740 1 -- 172.21.15.94:0/3928487682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1c0c0 msgr2=0x562a0bf1c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.716+0000 7fadc299a740 1 --2- 172.21.15.94:0/3928487682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1c0c0 0x562a0bf1c4a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fadb4011f00 tx=0x7fadb4014000 comp rx=0 tx=0).stop 2024-02-17T11:59:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.716+0000 7fadc299a740 1 -- 172.21.15.94:0/3928487682 shutdown_connections 2024-02-17T11:59:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.716+0000 7fadc299a740 1 --2- 172.21.15.94:0/3928487682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf2dfc0 0x562a0bf303b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.716+0000 7fadc299a740 1 --2- 172.21.15.94:0/3928487682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 0x562a0bf2da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.716+0000 7fadc299a740 1 --2- 172.21.15.94:0/3928487682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1c0c0 0x562a0bf1c4a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.716+0000 7fadc299a740 1 -- 172.21.15.94:0/3928487682 >> 172.21.15.94:0/3928487682 conn(0x562a0bf16990 msgr2=0x562a0bf16d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.716+0000 7fadc299a740 1 -- 172.21.15.94:0/3928487682 shutdown_connections 2024-02-17T11:59:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.716+0000 7fadc299a740 1 -- 172.21.15.94:0/3928487682 wait complete. 2024-02-17T11:59:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.716+0000 7fadc299a740 1 -- start start 2024-02-17T11:59:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.716+0000 7fadc299a740 1 Processor -- start 2024-02-17T11:59:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadc299a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 0x562a0bf2da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadc299a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1ca70 0x562a0bf2adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadc299a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf2dfc0 0x562a0bf2b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadc299a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0bb57430 con 0x562a0bf2dfc0 2024-02-17T11:59:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadc299a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0bb13e00 con 0x562a0bf1c0c0 2024-02-17T11:59:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadc299a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0be08fd0 con 0x562a0bf1ca70 2024-02-17T11:59:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadbda87640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 0x562a0bf2da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadc299a740 10 client.0 _ll_register_callbacks cb 0x562a0be5efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadbda87640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 0x562a0bf2da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45066/0 (socket says 172.21.15.94:45066) 2024-02-17T11:59:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadbda87640 1 -- 172.21.15.94:0/611471151 learned_addr learned my addr 172.21.15.94:0/611471151 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadbd286640 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1ca70 0x562a0bf2adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadbe288640 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf2dfc0 0x562a0bf2b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadbda87640 1 -- 172.21.15.94:0/611471151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1ca70 msgr2=0x562a0bf2adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadbda87640 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1ca70 0x562a0bf2adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadbda87640 1 -- 172.21.15.94:0/611471151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf2dfc0 msgr2=0x562a0bf2b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadbda87640 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf2dfc0 0x562a0bf2b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadbda87640 1 -- 172.21.15.94:0/611471151 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb4011be0 con 0x562a0bf1c0c0 2024-02-17T11:59:04.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadbe288640 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf2dfc0 0x562a0bf2b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:04.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fadbda87640 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 0x562a0bf2da20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009a20 tx=0x7fadb4005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=194cd1b71f85e126 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:04.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.717+0000 7fada17fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:59:04.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.718+0000 7fada17fa640 1 -- 172.21.15.94:0/611471151 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb402d070 con 0x562a0bf1c0c0 2024-02-17T11:59:04.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.718+0000 7fada17fa640 1 -- 172.21.15.94:0/611471151 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fadb4018070 con 0x562a0bf1c0c0 2024-02-17T11:59:04.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.718+0000 7fadc299a740 1 -- 172.21.15.94:0/611471151 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a0bb837e0 con 0x562a0bf1c0c0 2024-02-17T11:59:04.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.718+0000 7fada17fa640 1 -- 172.21.15.94:0/611471151 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadb4005320 con 0x562a0bf1c0c0 2024-02-17T11:59:04.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.719+0000 7fadc299a740 10 client.25271 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:04.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.720+0000 7fadc299a740 1 -- 172.21.15.94:0/611471151 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x562a0be82610 con 0x562a0bf1c0c0 2024-02-17T11:59:04.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.720+0000 7fada17fa640 1 -- 172.21.15.94:0/611471151 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadb4003b50 con 0x562a0bf1c0c0 2024-02-17T11:59:04.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.720+0000 7fada17fa640 1 -- 172.21.15.94:0/611471151 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fadb4025030 con 0x562a0bf1c0c0 2024-02-17T11:59:04.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.720+0000 7fadc299a740 10 client.25271 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:04.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.720+0000 7fadc299a740 10 client.25271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:04.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.720+0000 7fadc299a740 20 client.25271 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:04.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.721+0000 7fadc299a740 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562a0bf3e050 0x562a0bf40480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.721+0000 7fadc299a740 4 client.25271 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.721+0000 7fadc299a740 1 -- 172.21.15.94:0/611471151 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a0bb837e0 con 0x562a0bf3e050 2024-02-17T11:59:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.721+0000 7fadbd286640 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562a0bf3e050 0x562a0bf40480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.721+0000 7fada17fa640 10 client.25271 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.721+0000 7fadbd286640 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562a0bf3e050 0x562a0bf40480 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56a12ef805a1db25 server_cookie=30f0e4186a5b2e15 in_seq=0 out_seq=0 2024-02-17T11:59:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fada17fa640 1 -- 172.21.15.94:0/611471151 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a0bb837e0 con 0x562a0bf3e050 2024-02-17T11:59:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fada17fa640 10 client.25271 handle_command_reply: tid=0 2024-02-17T11:59:04.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fadc299a740 1 client.25271 shutdown 2024-02-17T11:59:04.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fadc299a740 1 -- 172.21.15.94:0/611471151 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 msgr2=0x562a0bf2da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fadc299a740 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 0x562a0bf2da20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009a20 tx=0x7fadb4005750 comp rx=0 tx=0).stop 2024-02-17T11:59:04.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fadc299a740 1 -- 172.21.15.94:0/611471151 shutdown_connections 2024-02-17T11:59:04.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fadc299a740 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562a0bf3e050 0x562a0bf40480 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fadc299a740 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf2dfc0 0x562a0bf2b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fada17fa640 0 client.25271 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:04.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fadc299a740 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1ca70 0x562a0bf2adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fadc299a740 1 --2- 172.21.15.94:0/611471151 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 0x562a0bf2da20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fadc299a740 1 -- 172.21.15.94:0/611471151 >> 172.21.15.94:0/611471151 conn(0x562a0bf16990 msgr2=0x562a0bf18730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:04.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fadc299a740 1 -- 172.21.15.94:0/611471151 shutdown_connections 2024-02-17T11:59:04.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fadc299a740 1 -- 172.21.15.94:0/611471151 wait complete. 2024-02-17T11:59:04.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.722+0000 7fadc299a740 20 client.25271 trim_cache size 0 max 16384 2024-02-17T11:59:04.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.748+0000 7fadc299a740 1 Processor -- start 2024-02-17T11:59:04.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.748+0000 7fadc299a740 1 -- start start 2024-02-17T11:59:04.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.748+0000 7fadc299a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1c0c0 0x562a0bf2ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.748+0000 7fadc299a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 0x562a0bf2d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.748+0000 7fadc299a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf2dfc0 0x562a0bf2da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.748+0000 7fadc299a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0bf1b610 con 0x562a0bf1ca70 2024-02-17T11:59:04.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.748+0000 7fadc299a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0bf1bbd0 con 0x562a0bf2dfc0 2024-02-17T11:59:04.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.748+0000 7fadc299a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0bf2bad0 con 0x562a0bf1c0c0 2024-02-17T11:59:04.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.748+0000 7fadbd286640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 0x562a0bf2d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.748+0000 7fadbd286640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 0x562a0bf2d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55292/0 (socket says 172.21.15.94:55292) 2024-02-17T11:59:04.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.748+0000 7fadbd286640 1 -- 172.21.15.94:0/1139650002 learned_addr learned my addr 172.21.15.94:0/1139650002 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:04.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.748+0000 7fadbe288640 1 --2- 172.21.15.94:0/1139650002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf2dfc0 0x562a0bf2da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.749+0000 7fadbda87640 1 --2- 172.21.15.94:0/1139650002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1c0c0 0x562a0bf2ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.749+0000 7fadbd286640 1 -- 172.21.15.94:0/1139650002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1c0c0 msgr2=0x562a0bf2ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.749+0000 7fadbd286640 1 --2- 172.21.15.94:0/1139650002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1c0c0 0x562a0bf2ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.749+0000 7fadbd286640 1 -- 172.21.15.94:0/1139650002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf2dfc0 msgr2=0x562a0bf2da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.749+0000 7fadbd286640 1 --2- 172.21.15.94:0/1139650002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf2dfc0 0x562a0bf2da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.749+0000 7fadbd286640 1 -- 172.21.15.94:0/1139650002 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a0be1eac0 con 0x562a0bf1ca70 2024-02-17T11:59:04.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.749+0000 7fadbd286640 1 --2- 172.21.15.94:0/1139650002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 0x562a0bf2d4e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fada809aa80 tx=0x7fada809ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c17d3543014b61 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:04.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.749+0000 7fada37fe640 1 -- 172.21.15.94:0/1139650002 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada809eed0 con 0x562a0bf1ca70 2024-02-17T11:59:04.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.750+0000 7fada37fe640 1 -- 172.21.15.94:0/1139650002 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fada80a3520 con 0x562a0bf1ca70 2024-02-17T11:59:04.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.750+0000 7fada37fe640 1 -- 172.21.15.94:0/1139650002 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada80a3090 con 0x562a0bf1ca70 2024-02-17T11:59:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.750+0000 7fadc299a740 1 -- 172.21.15.94:0/1139650002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 msgr2=0x562a0bf2d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.750+0000 7fadc299a740 1 --2- 172.21.15.94:0/1139650002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 0x562a0bf2d4e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fada809aa80 tx=0x7fada809ac80 comp rx=0 tx=0).stop 2024-02-17T11:59:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.750+0000 7fadc299a740 1 -- 172.21.15.94:0/1139650002 shutdown_connections 2024-02-17T11:59:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.750+0000 7fadc299a740 1 --2- 172.21.15.94:0/1139650002 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf2dfc0 0x562a0bf2da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.751+0000 7fadc299a740 1 --2- 172.21.15.94:0/1139650002 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf1ca70 0x562a0bf2d4e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.751+0000 7fadc299a740 1 --2- 172.21.15.94:0/1139650002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1c0c0 0x562a0bf2ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.751+0000 7fadc299a740 1 -- 172.21.15.94:0/1139650002 >> 172.21.15.94:0/1139650002 conn(0x562a0bf16990 msgr2=0x562a0bf2a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.752+0000 7fadc299a740 1 -- 172.21.15.94:0/1139650002 shutdown_connections 2024-02-17T11:59:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.752+0000 7fadc299a740 1 -- 172.21.15.94:0/1139650002 wait complete. 2024-02-17T11:59:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.752+0000 7fadc299a740 1 -- start start 2024-02-17T11:59:04.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.752+0000 7fadc299a740 1 Processor -- start 2024-02-17T11:59:04.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.753+0000 7fadc299a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 0x562a0bf33fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.753+0000 7fadc299a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1ca70 0x562a0bf344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.753+0000 7fadc299a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf2dfc0 0x562a0bf945b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.753+0000 7fadc299a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0bf34c70 con 0x562a0bf2dfc0 2024-02-17T11:59:04.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.753+0000 7fadc299a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0bf94af0 con 0x562a0bf1c0c0 2024-02-17T11:59:04.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.753+0000 7fadc299a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a0bf94cb0 con 0x562a0bf1ca70 2024-02-17T11:59:04.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.753+0000 7fadc299a740 10 client.0 _ll_register_callbacks cb 0x7fadb4009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:04.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.754+0000 7fadbda87640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 0x562a0bf33fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.754+0000 7fadbda87640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 0x562a0bf33fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45090/0 (socket says 172.21.15.94:45090) 2024-02-17T11:59:04.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.754+0000 7fadbda87640 1 -- 172.21.15.94:0/2947004170 learned_addr learned my addr 172.21.15.94:0/2947004170 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:04.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.754+0000 7fadbe288640 1 --2- 172.21.15.94:0/2947004170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf2dfc0 0x562a0bf945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.754+0000 7fadbda87640 1 -- 172.21.15.94:0/2947004170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1ca70 msgr2=0x562a0bf344e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T11:59:04.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.754+0000 7fadbda87640 1 --2- 172.21.15.94:0/2947004170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1ca70 0x562a0bf344e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.754+0000 7fadbda87640 1 -- 172.21.15.94:0/2947004170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf2dfc0 msgr2=0x562a0bf945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:04.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.754+0000 7fadbda87640 1 --2- 172.21.15.94:0/2947004170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf2dfc0 0x562a0bf945b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.754+0000 7fadbda87640 1 -- 172.21.15.94:0/2947004170 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb81a6570 con 0x562a0bf1c0c0 2024-02-17T11:59:04.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.755+0000 7fadbda87640 1 --2- 172.21.15.94:0/2947004170 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 0x562a0bf33fa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fadb4012fd0 tx=0x7fadb4004910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e74b074c408aac7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:04.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.755+0000 7fada17fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:59:04.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.755+0000 7fada17fa640 1 -- 172.21.15.94:0/2947004170 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb402d040 con 0x562a0bf1c0c0 2024-02-17T11:59:04.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.755+0000 7fada17fa640 1 -- 172.21.15.94:0/2947004170 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fadb4018070 con 0x562a0bf1c0c0 2024-02-17T11:59:04.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.755+0000 7fadc299a740 1 -- 172.21.15.94:0/2947004170 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadb4016070 con 0x562a0bf1c0c0 2024-02-17T11:59:04.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.757+0000 7fada17fa640 1 -- 172.21.15.94:0/2947004170 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadb4011940 con 0x562a0bf1c0c0 2024-02-17T11:59:04.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.757+0000 7fadc299a740 10 client.25277 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:04.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.757+0000 7fadc299a740 1 -- 172.21.15.94:0/2947004170 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x562a0be446a0 con 0x562a0bf1c0c0 2024-02-17T11:59:04.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.757+0000 7fada17fa640 1 -- 172.21.15.94:0/2947004170 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadb4016070 con 0x562a0bf1c0c0 2024-02-17T11:59:04.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.758+0000 7fada17fa640 1 -- 172.21.15.94:0/2947004170 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fadb4042020 con 0x562a0bf1c0c0 2024-02-17T11:59:04.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.758+0000 7fadc299a740 10 client.25277 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:04.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.758+0000 7fadc299a740 10 client.25277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:04.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.758+0000 7fadc299a740 20 client.25277 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:04.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.758+0000 7fadc299a740 1 --2- 172.21.15.94:0/2947004170 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562a0bf96290 0x562a0bf986b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:04.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.758+0000 7fadc299a740 4 client.25277 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T11:59:04.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.758+0000 7fadc299a740 1 -- 172.21.15.94:0/2947004170 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562a0bbc0720 con 0x562a0bf96290 2024-02-17T11:59:04.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.758+0000 7fadbd286640 1 --2- 172.21.15.94:0/2947004170 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562a0bf96290 0x562a0bf986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:04.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.760+0000 7fadbd286640 1 --2- 172.21.15.94:0/2947004170 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562a0bf96290 0x562a0bf986b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f612f59cd77cba server_cookie=d8a7e12708c178ff in_seq=0 out_seq=0 2024-02-17T11:59:04.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.760+0000 7fada17fa640 10 client.25277 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:04.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.760+0000 7fada17fa640 1 -- 172.21.15.94:0/2947004170 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562a0bbc0720 con 0x562a0bf96290 2024-02-17T11:59:04.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.760+0000 7fada17fa640 10 client.25277 handle_command_reply: tid=0 2024-02-17T11:59:04.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.760+0000 7fadc299a740 1 client.25277 shutdown 2024-02-17T11:59:04.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.760+0000 7fadc299a740 1 -- 172.21.15.94:0/2947004170 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 msgr2=0x562a0bf33fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.760+0000 7fadc299a740 1 --2- 172.21.15.94:0/2947004170 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 0x562a0bf33fa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fadb4012fd0 tx=0x7fadb4004910 comp rx=0 tx=0).stop 2024-02-17T11:59:04.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.760+0000 7fadc299a740 1 -- 172.21.15.94:0/2947004170 shutdown_connections 2024-02-17T11:59:04.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.760+0000 7fadc299a740 1 --2- 172.21.15.94:0/2947004170 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562a0bf96290 0x562a0bf986b0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.761+0000 7fadc299a740 1 --2- 172.21.15.94:0/2947004170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a0bf2dfc0 0x562a0bf945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.761+0000 7fadc299a740 1 --2- 172.21.15.94:0/2947004170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a0bf1ca70 0x562a0bf344e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.761+0000 7fadc299a740 1 --2- 172.21.15.94:0/2947004170 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a0bf1c0c0 0x562a0bf33fa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.761+0000 7fadc299a740 1 -- 172.21.15.94:0/2947004170 >> 172.21.15.94:0/2947004170 conn(0x562a0bf16990 msgr2=0x562a0bf15a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:04.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.761+0000 7fada17fa640 0 client.25277 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:04.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.761+0000 7fadc299a740 1 -- 172.21.15.94:0/2947004170 shutdown_connections 2024-02-17T11:59:04.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.761+0000 7fadc299a740 1 -- 172.21.15.94:0/2947004170 wait complete. 2024-02-17T11:59:04.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.761+0000 7fadc299a740 20 client.25277 trim_cache size 0 max 16384 2024-02-17T11:59:04.762 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:59:04.762 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T11:59:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T11:59:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "24bf619e-7d2a-4b97-b6f5-b1b0ff420381": { 2024-02-17T11:59:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T11:59:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "24bf619e-7d2a-4b97-b6f5-b1b0ff420381", 2024-02-17T11:59:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T11:59:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T11:59:04.763 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T11:59:04.763 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:59:04.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.762+0000 7fadbfceb640 1 -- 172.21.15.94:0/1548487631 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fada40a24a0 msgr2=0x7fada40a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.762+0000 7fadbfceb640 1 --2- 172.21.15.94:0/1548487631 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fada40a24a0 0x7fada40a4960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fadb4009b60 tx=0x7fadb4005f90 comp rx=0 tx=0).stop 2024-02-17T11:59:04.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.762+0000 7fadbfceb640 1 -- 172.21.15.94:0/1548487631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb8108670 msgr2=0x7fadb819fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:04.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.762+0000 7fadbfceb640 1 --2- 172.21.15.94:0/1548487631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb8108670 0x7fadb819fc00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fada800e9f0 tx=0x7fada800eeb0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.762+0000 7fadbfceb640 1 -- 172.21.15.94:0/1548487631 shutdown_connections 2024-02-17T11:59:04.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.762+0000 7fadbfceb640 1 --2- 172.21.15.94:0/1548487631 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fada40a24a0 0x7fada40a4960 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.762+0000 7fadbfceb640 1 --2- 172.21.15.94:0/1548487631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fadb8199880 0x7fadb8199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.762+0000 7fadbfceb640 1 --2- 172.21.15.94:0/1548487631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb8108670 0x7fadb819fc00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.762+0000 7fadbfceb640 1 --2- 172.21.15.94:0/1548487631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fadb81035a0 0x7fadb819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:04.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.762+0000 7fadbfceb640 1 -- 172.21.15.94:0/1548487631 >> 172.21.15.94:0/1548487631 conn(0x7fadb80fee00 msgr2=0x7fadb8101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:04.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.762+0000 7fadbfceb640 1 -- 172.21.15.94:0/1548487631 shutdown_connections 2024-02-17T11:59:04.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:04.763+0000 7fadbfceb640 1 -- 172.21.15.94:0/1548487631 wait complete. 2024-02-17T11:59:04.781 INFO:tasks.cephfs.filesystem:scrub status for tag:24bf619e-7d2a-4b97-b6f5-b1b0ff420381 - {'path': '/', 'tag': '24bf619e-7d2a-4b97-b6f5-b1b0ff420381', 'options': 'recursive,force'} 2024-02-17T11:59:05.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:04 smithi189 ceph-mon[36588]: pgmap v32: 97 pgs: 97 active+clean; 997 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 867 op/s 2024-02-17T11:59:05.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:04 smithi125 ceph-mon[36819]: pgmap v32: 97 pgs: 97 active+clean; 997 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 867 op/s 2024-02-17T11:59:05.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:04 smithi094 ceph-mon[32499]: pgmap v32: 97 pgs: 97 active+clean; 997 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 867 op/s 2024-02-17T11:59:05.241 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 45.0 s, 1144.4 tps, lat 87.260 ms stddev 123.426 2024-02-17T11:59:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:05 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T11:59:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:05 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T11:59:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:05 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T11:59:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:05 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T11:59:06.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:05 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T11:59:06.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:05 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T11:59:06.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:05 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T11:59:06.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:05 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T11:59:06.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:05 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T11:59:06.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:05 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T11:59:06.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:05 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T11:59:06.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:05 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T11:59:07.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:06 smithi125 ceph-mon[36819]: pgmap v33: 97 pgs: 97 active+clean; 997 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.6 MiB/s wr, 722 op/s 2024-02-17T11:59:07.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:06 smithi094 ceph-mon[32499]: pgmap v33: 97 pgs: 97 active+clean; 997 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.6 MiB/s wr, 722 op/s 2024-02-17T11:59:07.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:06 smithi189 ceph-mon[36588]: pgmap v33: 97 pgs: 97 active+clean; 997 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.6 MiB/s wr, 722 op/s 2024-02-17T11:59:08.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:07 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T11:59:08.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:07 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T11:59:08.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:07 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T11:59:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:08 smithi125 ceph-mon[36819]: pgmap v34: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 45 MiB/s wr, 1.73k op/s 2024-02-17T11:59:09.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:08 smithi094 ceph-mon[32499]: pgmap v34: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 45 MiB/s wr, 1.73k op/s 2024-02-17T11:59:09.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:08 smithi189 ceph-mon[36588]: pgmap v34: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 45 MiB/s wr, 1.73k op/s 2024-02-17T11:59:10.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 50.0 s, 4179.8 tps, lat 23.963 ms stddev 24.337 2024-02-17T11:59:12.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:11 smithi189 ceph-mon[36588]: pgmap v35: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 41 MiB/s wr, 1.38k op/s 2024-02-17T11:59:12.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:11 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:59:12.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:11 smithi125 ceph-mon[36819]: pgmap v35: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 41 MiB/s wr, 1.38k op/s 2024-02-17T11:59:12.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:11 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:59:12.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:11 smithi094 ceph-mon[32499]: pgmap v35: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 41 MiB/s wr, 1.38k op/s 2024-02-17T11:59:12.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:11 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T11:59:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:13 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:59:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:13 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:59:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:13 smithi125 ceph-mon[36819]: pgmap v36: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 44 MiB/s wr, 1.67k op/s 2024-02-17T11:59:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:13 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:59:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:13 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:59:13.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:13 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:59:13.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:13 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:59:13.954 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:13 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:59:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:13 smithi094 ceph-mon[32499]: pgmap v36: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 44 MiB/s wr, 1.67k op/s 2024-02-17T11:59:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:13 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:59:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:13 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:59:13.955 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:13 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:59:14.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:13 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:59:14.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:13 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:59:14.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:13 smithi189 ceph-mon[36588]: pgmap v36: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 44 MiB/s wr, 1.67k op/s 2024-02-17T11:59:14.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:13 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T11:59:14.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:13 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T11:59:14.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:13 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T11:59:15.241 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 55.0 s, 3744.6 tps, lat 26.593 ms stddev 27.280 2024-02-17T11:59:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.508+0000 7fbb147d0640 1 -- 172.21.15.94:0/4014553911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0c1060b0 msgr2=0x7fbb0c106490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.508+0000 7fbb147d0640 1 --2- 172.21.15.94:0/4014553911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0c1060b0 0x7fbb0c106490 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009ca0 tx=0x7fbb0402c0f0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.508+0000 7fbb147d0640 1 -- 172.21.15.94:0/4014553911 shutdown_connections 2024-02-17T11:59:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.508+0000 7fbb147d0640 1 --2- 172.21.15.94:0/4014553911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb0c100c70 0x7fbb0c069960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.508+0000 7fbb147d0640 1 --2- 172.21.15.94:0/4014553911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb0c1002d0 0x7fbb0c100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.508+0000 7fbb147d0640 1 --2- 172.21.15.94:0/4014553911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0c1060b0 0x7fbb0c106490 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.508+0000 7fbb147d0640 1 -- 172.21.15.94:0/4014553911 >> 172.21.15.94:0/4014553911 conn(0x7fbb0c076ea0 msgr2=0x7fbb0c0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.508+0000 7fbb147d0640 1 -- 172.21.15.94:0/4014553911 shutdown_connections 2024-02-17T11:59:15.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.508+0000 7fbb147d0640 1 -- 172.21.15.94:0/4014553911 wait complete. 2024-02-17T11:59:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.508+0000 7fbb147d0640 1 Processor -- start 2024-02-17T11:59:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.509+0000 7fbb147d0640 1 -- start start 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.509+0000 7fbb147d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb0c1002d0 0x7fbb0c103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.509+0000 7fbb147d0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb0c100c70 0x7fbb0c103590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.509+0000 7fbb147d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0c1060b0 0x7fbb0c1a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.509+0000 7fbb147d0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c06aba0 con 0x7fbb0c1060b0 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.509+0000 7fbb147d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c06ad10 con 0x7fbb0c1002d0 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.509+0000 7fbb147d0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c06aed0 con 0x7fbb0c100c70 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.509+0000 7fbb12d6d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0c1060b0 0x7fbb0c1a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.509+0000 7fbb11d6b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb0c100c70 0x7fbb0c103590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.509+0000 7fbb1256c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb0c1002d0 0x7fbb0c103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.509+0000 7fbb11d6b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb0c100c70 0x7fbb0c103590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39494/0 (socket says 172.21.15.94:39494) 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.509+0000 7fbb11d6b640 1 -- 172.21.15.94:0/2889746713 learned_addr learned my addr 172.21.15.94:0/2889746713 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.509+0000 7fbb1256c640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb0c1002d0 0x7fbb0c103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53966/0 (socket says 172.21.15.94:53966) 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.510+0000 7fbb1256c640 1 -- 172.21.15.94:0/2889746713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb0c100c70 msgr2=0x7fbb0c103590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.510+0000 7fbb1256c640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb0c100c70 0x7fbb0c103590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.510+0000 7fbb1256c640 1 -- 172.21.15.94:0/2889746713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0c1060b0 msgr2=0x7fbb0c1a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.510+0000 7fbb1256c640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0c1060b0 0x7fbb0c1a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.510+0000 7fbb1256c640 1 -- 172.21.15.94:0/2889746713 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb04009900 con 0x7fbb0c1002d0 2024-02-17T11:59:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.510+0000 7fbb11d6b640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb0c100c70 0x7fbb0c103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:59:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.510+0000 7fbb12d6d640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0c1060b0 0x7fbb0c1a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:59:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.510+0000 7fbb1256c640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb0c1002d0 0x7fbb0c103050 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402c700 tx=0x7fbb04034cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.511+0000 7fbb037fe640 1 -- 172.21.15.94:0/2889746713 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040328c0 con 0x7fbb0c1002d0 2024-02-17T11:59:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.511+0000 7fbb037fe640 1 -- 172.21.15.94:0/2889746713 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb04003e50 con 0x7fbb0c1002d0 2024-02-17T11:59:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.511+0000 7fbb147d0640 1 -- 172.21.15.94:0/2889746713 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0c06b150 con 0x7fbb0c1002d0 2024-02-17T11:59:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.511+0000 7fbb147d0640 1 -- 172.21.15.94:0/2889746713 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0c06b5d0 con 0x7fbb0c1002d0 2024-02-17T11:59:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.511+0000 7fbb037fe640 1 -- 172.21.15.94:0/2889746713 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb04004800 con 0x7fbb0c1002d0 2024-02-17T11:59:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.514+0000 7fbb037fe640 1 -- 172.21.15.94:0/2889746713 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fbb0400a040 con 0x7fbb0c1002d0 2024-02-17T11:59:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.514+0000 7fbb173a5740 1 Processor -- start 2024-02-17T11:59:15.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.515+0000 7fbb173a5740 1 -- start start 2024-02-17T11:59:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.515+0000 7fbb173a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2384ad0 0x55c3c2383820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.515+0000 7fbb173a5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c2383d60 0x55c3c23841e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.515+0000 7fbb173a5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c238d2e0 0x55c3c238f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.515+0000 7fbb173a5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c22f29f0 con 0x55c3c238d2e0 2024-02-17T11:59:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.515+0000 7fbb173a5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c22eff20 con 0x55c3c2383d60 2024-02-17T11:59:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.515+0000 7fbb173a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c22eb700 con 0x55c3c2384ad0 2024-02-17T11:59:15.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.515+0000 7fbb037fe640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbaf80a2620 0x7fbaf80a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.515+0000 7fbb1256c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c2383d60 0x55c3c23841e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.515+0000 7fbb1256c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c2383d60 0x55c3c23841e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53970/0 (socket says 172.21.15.94:53970) 2024-02-17T11:59:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.515+0000 7fbb1256c640 1 -- 172.21.15.94:0/1373119241 learned_addr learned my addr 172.21.15.94:0/1373119241 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.516+0000 7fbb1256c640 1 -- 172.21.15.94:0/1373119241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2384ad0 msgr2=0x55c3c2383820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.516+0000 7fbb12d6d640 1 --2- 172.21.15.94:0/1373119241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2384ad0 0x55c3c2383820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.516+0000 7fbb1256c640 1 --2- 172.21.15.94:0/1373119241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2384ad0 0x55c3c2383820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.516+0000 7fbb1256c640 1 -- 172.21.15.94:0/1373119241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c238d2e0 msgr2=0x55c3c238f6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T11:59:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.516+0000 7fbb1256c640 1 --2- 172.21.15.94:0/1373119241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c238d2e0 0x55c3c238f6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.516+0000 7fbb1256c640 1 -- 172.21.15.94:0/1373119241 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3c22817b0 con 0x55c3c2383d60 2024-02-17T11:59:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.516+0000 7fbb12d6d640 1 --2- 172.21.15.94:0/1373119241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2384ad0 0x55c3c2383820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:59:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.516+0000 7fbb12d6d640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbaf80a2620 0x7fbaf80a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.516+0000 7fbb037fe640 1 -- 172.21.15.94:0/2889746713 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fbb040f0ed0 con 0x7fbb0c1002d0 2024-02-17T11:59:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.517+0000 7fbb1256c640 1 --2- 172.21.15.94:0/1373119241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c2383d60 0x55c3c23841e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb040ee5d0 tx=0x7fbb0407df40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3cd2c615708b63 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.517+0000 7fbb12d6d640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbaf80a2620 0x7fbaf80a4ae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb080058c0 tx=0x7fbb08005850 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.517+0000 7fbb00ff9640 1 -- 172.21.15.94:0/1373119241 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb04087a40 con 0x55c3c2383d60 2024-02-17T11:59:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.517+0000 7fbb00ff9640 1 -- 172.21.15.94:0/1373119241 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb040bde10 con 0x55c3c2383d60 2024-02-17T11:59:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.517+0000 7fbb00ff9640 1 -- 172.21.15.94:0/1373119241 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040c5f40 con 0x55c3c2383d60 2024-02-17T11:59:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.518+0000 7fbb173a5740 1 -- 172.21.15.94:0/1373119241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c2383d60 msgr2=0x55c3c23841e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.518+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1373119241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c2383d60 0x55c3c23841e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb040ee5d0 tx=0x7fbb0407df40 comp rx=0 tx=0).stop 2024-02-17T11:59:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.519+0000 7fbb173a5740 1 -- 172.21.15.94:0/1373119241 shutdown_connections 2024-02-17T11:59:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.519+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1373119241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c238d2e0 0x55c3c238f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.519+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1373119241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c2383d60 0x55c3c23841e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.519+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1373119241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2384ad0 0x55c3c2383820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.519+0000 7fbb173a5740 1 -- 172.21.15.94:0/1373119241 >> 172.21.15.94:0/1373119241 conn(0x55c3c234b550 msgr2=0x55c3c237ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.519+0000 7fbb173a5740 1 -- 172.21.15.94:0/1373119241 shutdown_connections 2024-02-17T11:59:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.519+0000 7fbb173a5740 1 -- 172.21.15.94:0/1373119241 wait complete. 2024-02-17T11:59:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.520+0000 7fbb173a5740 1 -- start start 2024-02-17T11:59:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.520+0000 7fbb173a5740 1 Processor -- start 2024-02-17T11:59:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.521+0000 7fbb173a5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2383d60 0x55c3c238a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.521+0000 7fbb173a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2384ad0 0x55c3c238a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.521+0000 7fbb1256c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2383d60 0x55c3c238a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.521+0000 7fbb1256c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2383d60 0x55c3c238a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41852/0 (socket says 172.21.15.94:41852) 2024-02-17T11:59:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.521+0000 7fbb1256c640 1 -- 172.21.15.94:0/1996150417 learned_addr learned my addr 172.21.15.94:0/1996150417 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.521+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1996150417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 0x55c3c238ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.521+0000 7fbb173a5740 1 -- 172.21.15.94:0/1996150417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c2290bf0 con 0x55c3c2383d60 2024-02-17T11:59:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.521+0000 7fbb173a5740 1 -- 172.21.15.94:0/1996150417 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c1fbf250 con 0x55c3c238d2e0 2024-02-17T11:59:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.521+0000 7fbb173a5740 1 -- 172.21.15.94:0/1996150417 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c2271200 con 0x55c3c2384ad0 2024-02-17T11:59:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.521+0000 7fbb173a5740 10 client.0 _ll_register_callbacks cb 0x55c3c22dfd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.521+0000 7fbb12d6d640 1 --2- 172.21.15.94:0/1996150417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 0x55c3c238ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.522+0000 7fbb1256c640 1 -- 172.21.15.94:0/1996150417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2384ad0 msgr2=0x55c3c238a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.522+0000 7fbb1256c640 1 --2- 172.21.15.94:0/1996150417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2384ad0 0x55c3c238a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.522+0000 7fbb1256c640 1 -- 172.21.15.94:0/1996150417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 msgr2=0x55c3c238ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.522+0000 7fbb1256c640 1 --2- 172.21.15.94:0/1996150417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 0x55c3c238ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.522+0000 7fbb1256c640 1 -- 172.21.15.94:0/1996150417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0c06b5d0 con 0x55c3c2383d60 2024-02-17T11:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.522+0000 7fbb1256c640 1 --2- 172.21.15.94:0/1996150417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2383d60 0x55c3c238a390 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402c6d0 tx=0x7fbb04087a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e68e877b954234 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.522+0000 7fbaf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.523+0000 7fbaf5ffb640 1 -- 172.21.15.94:0/1996150417 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040c1860 con 0x55c3c2383d60 2024-02-17T11:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.523+0000 7fbaf5ffb640 1 -- 172.21.15.94:0/1996150417 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fbb040c5eb0 con 0x55c3c2383d60 2024-02-17T11:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.523+0000 7fbb173a5740 1 -- 172.21.15.94:0/1996150417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3c1feb600 con 0x55c3c2383d60 2024-02-17T11:59:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.523+0000 7fbaf5ffb640 1 -- 172.21.15.94:0/1996150417 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb04050300 con 0x55c3c2383d60 2024-02-17T11:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.523+0000 7fbb173a5740 10 client.15816 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.523+0000 7fbb173a5740 1 -- 172.21.15.94:0/1996150417 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55c3c22e9d80 con 0x55c3c2383d60 2024-02-17T11:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.523+0000 7fbaf5ffb640 1 -- 172.21.15.94:0/1996150417 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c3c1feb600 con 0x55c3c2383d60 2024-02-17T11:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.524+0000 7fbaf5ffb640 1 -- 172.21.15.94:0/1996150417 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbb040dd0d0 con 0x55c3c2383d60 2024-02-17T11:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.524+0000 7fbb173a5740 10 client.15816 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.524+0000 7fbb173a5740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.524+0000 7fbb173a5740 20 client.15816 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.524+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1996150417 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c3c239cdb0 0x55c3c239f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.524+0000 7fbb173a5740 4 client.15816 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.524+0000 7fbb173a5740 1 -- 172.21.15.94:0/1996150417 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3c203c1b0 con 0x55c3c239cdb0 2024-02-17T11:59:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.525+0000 7fbb11d6b640 1 --2- 172.21.15.94:0/1996150417 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c3c239cdb0 0x55c3c239f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.526+0000 7fbb11d6b640 1 --2- 172.21.15.94:0/1996150417 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c3c239cdb0 0x55c3c239f1a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a91ddeff2bdacef server_cookie=4bc03bc0c4261b71 in_seq=0 out_seq=0 2024-02-17T11:59:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.526+0000 7fbaf5ffb640 10 client.15816 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.526+0000 7fbaf5ffb640 1 -- 172.21.15.94:0/1996150417 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3c203c1b0 con 0x55c3c239cdb0 2024-02-17T11:59:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.526+0000 7fbaf5ffb640 10 client.15816 handle_command_reply: tid=0 2024-02-17T11:59:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.526+0000 7fbb173a5740 1 client.15816 shutdown 2024-02-17T11:59:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.527+0000 7fbb173a5740 1 -- 172.21.15.94:0/1996150417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2383d60 msgr2=0x55c3c238a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.527+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1996150417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2383d60 0x55c3c238a390 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402c6d0 tx=0x7fbb04087a40 comp rx=0 tx=0).stop 2024-02-17T11:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.527+0000 7fbb173a5740 1 -- 172.21.15.94:0/1996150417 shutdown_connections 2024-02-17T11:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.527+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1996150417 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c3c239cdb0 0x55c3c239f1a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.527+0000 7fbaf5ffb640 0 client.15816 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.527+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1996150417 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 0x55c3c238ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.527+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1996150417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2384ad0 0x55c3c238a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.527+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1996150417 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2383d60 0x55c3c238a390 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.527+0000 7fbb173a5740 1 -- 172.21.15.94:0/1996150417 >> 172.21.15.94:0/1996150417 conn(0x55c3c234b550 msgr2=0x55c3c23807a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.528+0000 7fbb173a5740 1 -- 172.21.15.94:0/1996150417 shutdown_connections 2024-02-17T11:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.528+0000 7fbb173a5740 1 -- 172.21.15.94:0/1996150417 wait complete. 2024-02-17T11:59:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.528+0000 7fbb173a5740 20 client.15816 trim_cache size 0 max 16384 2024-02-17T11:59:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.548+0000 7fbb173a5740 1 Processor -- start 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.548+0000 7fbb173a5740 1 -- start start 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb173a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2383d60 0x55c3c238cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb173a5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2384ad0 0x55c3c238ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb173a5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 0x55c3c2393e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb173a5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c2385ff0 con 0x55c3c2384ad0 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb173a5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c2383030 con 0x55c3c238d2e0 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb173a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c2383b60 con 0x55c3c2383d60 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb11d6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2384ad0 0x55c3c238ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb11d6b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2384ad0 0x55c3c238ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41866/0 (socket says 172.21.15.94:41866) 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb11d6b640 1 -- 172.21.15.94:0/1450359755 learned_addr learned my addr 172.21.15.94:0/1450359755 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb1256c640 1 --2- 172.21.15.94:0/1450359755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2383d60 0x55c3c238cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb12d6d640 1 --2- 172.21.15.94:0/1450359755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 0x55c3c2393e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb11d6b640 1 -- 172.21.15.94:0/1450359755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2383d60 msgr2=0x55c3c238cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb11d6b640 1 --2- 172.21.15.94:0/1450359755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2383d60 0x55c3c238cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb11d6b640 1 -- 172.21.15.94:0/1450359755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 msgr2=0x55c3c2393e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb11d6b640 1 --2- 172.21.15.94:0/1450359755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 0x55c3c2393e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb11d6b640 1 -- 172.21.15.94:0/1450359755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3c22867b0 con 0x55c3c2384ad0 2024-02-17T11:59:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbb11d6b640 1 --2- 172.21.15.94:0/1450359755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2384ad0 0x55c3c238ac30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c1046c0 tx=0x7fbafc0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa83fe13f56d4f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.549+0000 7fbaf7fff640 1 -- 172.21.15.94:0/1450359755 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc017040 con 0x55c3c2384ad0 2024-02-17T11:59:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.550+0000 7fbaf7fff640 1 -- 172.21.15.94:0/1450359755 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbafc01e070 con 0x55c3c2384ad0 2024-02-17T11:59:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.550+0000 7fbaf7fff640 1 -- 172.21.15.94:0/1450359755 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc005690 con 0x55c3c2384ad0 2024-02-17T11:59:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.550+0000 7fbb173a5740 1 -- 172.21.15.94:0/1450359755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2384ad0 msgr2=0x55c3c238ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.550+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1450359755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2384ad0 0x55c3c238ac30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c1046c0 tx=0x7fbafc0095d0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.551+0000 7fbb173a5740 1 -- 172.21.15.94:0/1450359755 shutdown_connections 2024-02-17T11:59:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.551+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1450359755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 0x55c3c2393e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.551+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1450359755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2384ad0 0x55c3c238ac30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.551+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1450359755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2383d60 0x55c3c238cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.551+0000 7fbb173a5740 1 -- 172.21.15.94:0/1450359755 >> 172.21.15.94:0/1450359755 conn(0x55c3c234b550 msgr2=0x55c3c237e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.552+0000 7fbb173a5740 1 -- 172.21.15.94:0/1450359755 shutdown_connections 2024-02-17T11:59:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.552+0000 7fbb173a5740 1 -- 172.21.15.94:0/1450359755 wait complete. 2024-02-17T11:59:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.552+0000 7fbb173a5740 1 -- start start 2024-02-17T11:59:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.552+0000 7fbb173a5740 1 Processor -- start 2024-02-17T11:59:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.553+0000 7fbb173a5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2383d60 0x55c3c2394360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.553+0000 7fbb173a5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2384ad0 0x55c3c2386eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.553+0000 7fbb173a5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 0x55c3c23875f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.553+0000 7fbb173a5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c2387cc0 con 0x55c3c2384ad0 2024-02-17T11:59:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.553+0000 7fbb173a5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c23f32a0 con 0x55c3c238d2e0 2024-02-17T11:59:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.553+0000 7fbb173a5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c23f33e0 con 0x55c3c2383d60 2024-02-17T11:59:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.553+0000 7fbb173a5740 10 client.0 _ll_register_callbacks cb 0x7fbb040bd560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.553+0000 7fbb1256c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2383d60 0x55c3c2394360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.553+0000 7fbb1256c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2383d60 0x55c3c2394360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39536/0 (socket says 172.21.15.94:39536) 2024-02-17T11:59:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.553+0000 7fbb1256c640 1 -- 172.21.15.94:0/1473704643 learned_addr learned my addr 172.21.15.94:0/1473704643 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.553+0000 7fbb12d6d640 1 --2- 172.21.15.94:0/1473704643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 0x55c3c23875f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.553+0000 7fbb11d6b640 1 --2- 172.21.15.94:0/1473704643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2384ad0 0x55c3c2386eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.554+0000 7fbb1256c640 1 -- 172.21.15.94:0/1473704643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 msgr2=0x55c3c23875f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.554+0000 7fbb1256c640 1 --2- 172.21.15.94:0/1473704643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 0x55c3c23875f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.554+0000 7fbb1256c640 1 -- 172.21.15.94:0/1473704643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2384ad0 msgr2=0x55c3c2386eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.554+0000 7fbb1256c640 1 --2- 172.21.15.94:0/1473704643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2384ad0 0x55c3c2386eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.554+0000 7fbb1256c640 1 -- 172.21.15.94:0/1473704643 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbafc0099c0 con 0x55c3c2383d60 2024-02-17T11:59:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.555+0000 7fbb1256c640 1 --2- 172.21.15.94:0/1473704643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2383d60 0x55c3c2394360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbb040ba0e0 tx=0x7fbb040bdd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=720fce530f27da3c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.555+0000 7fbaf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T11:59:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.555+0000 7fbaf5ffb640 1 -- 172.21.15.94:0/1473704643 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb04050500 con 0x55c3c2383d60 2024-02-17T11:59:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.555+0000 7fbaf5ffb640 1 -- 172.21.15.94:0/1473704643 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fbb040c5eb0 con 0x55c3c2383d60 2024-02-17T11:59:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.555+0000 7fbb173a5740 1 -- 172.21.15.94:0/1473704643 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb040bde10 con 0x55c3c2383d60 2024-02-17T11:59:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.555+0000 7fbaf5ffb640 1 -- 172.21.15.94:0/1473704643 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb040b99b0 con 0x55c3c2383d60 2024-02-17T11:59:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.555+0000 7fbb173a5740 10 client.25345 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.555+0000 7fbb173a5740 1 -- 172.21.15.94:0/1473704643 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55c3c22ac540 con 0x55c3c2383d60 2024-02-17T11:59:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.556+0000 7fbaf5ffb640 1 -- 172.21.15.94:0/1473704643 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbb04009540 con 0x55c3c2383d60 2024-02-17T11:59:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.556+0000 7fbaf5ffb640 1 -- 172.21.15.94:0/1473704643 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbb040c5a20 con 0x55c3c2383d60 2024-02-17T11:59:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.556+0000 7fbb173a5740 10 client.25345 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.556+0000 7fbb173a5740 10 client.25345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.556+0000 7fbb173a5740 20 client.25345 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.556+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1473704643 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c3c23f45e0 0x55c3c23f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.556+0000 7fbb173a5740 4 client.25345 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T11:59:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.556+0000 7fbb173a5740 1 -- 172.21.15.94:0/1473704643 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fbb040bde10 con 0x55c3c23f45e0 2024-02-17T11:59:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.556+0000 7fbb11d6b640 1 --2- 172.21.15.94:0/1473704643 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c3c23f45e0 0x55c3c23f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.556+0000 7fbaf5ffb640 10 client.25345 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.556+0000 7fbb11d6b640 1 --2- 172.21.15.94:0/1473704643 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c3c23f45e0 0x55c3c23f69d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8809554e1991823 server_cookie=3f5fb5cf4bb480c9 in_seq=0 out_seq=0 2024-02-17T11:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.576+0000 7fbaf5ffb640 1 -- 172.21.15.94:0/1473704643 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fbb040bde10 con 0x55c3c23f45e0 2024-02-17T11:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.576+0000 7fbaf5ffb640 10 client.25345 handle_command_reply: tid=0 2024-02-17T11:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.576+0000 7fbb173a5740 1 client.25345 shutdown 2024-02-17T11:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.576+0000 7fbb173a5740 1 -- 172.21.15.94:0/1473704643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2383d60 msgr2=0x55c3c2394360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.576+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1473704643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2383d60 0x55c3c2394360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbb040ba0e0 tx=0x7fbb040bdd10 comp rx=0 tx=0).stop 2024-02-17T11:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.576+0000 7fbb173a5740 1 -- 172.21.15.94:0/1473704643 shutdown_connections 2024-02-17T11:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.576+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1473704643 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c3c23f45e0 0x55c3c23f69d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.576+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1473704643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c3c238d2e0 0x55c3c23875f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.576+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1473704643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c3c2384ad0 0x55c3c2386eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.576+0000 7fbb173a5740 1 --2- 172.21.15.94:0/1473704643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c2383d60 0x55c3c2394360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.576+0000 7fbb173a5740 1 -- 172.21.15.94:0/1473704643 >> 172.21.15.94:0/1473704643 conn(0x55c3c234b550 msgr2=0x55c3c2388c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.576+0000 7fbaf5ffb640 0 client.25345 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.578+0000 7fbb173a5740 1 -- 172.21.15.94:0/1473704643 shutdown_connections 2024-02-17T11:59:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.578+0000 7fbb173a5740 1 -- 172.21.15.94:0/1473704643 wait complete. 2024-02-17T11:59:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.578+0000 7fbb173a5740 20 client.25345 trim_cache size 0 max 16384 2024-02-17T11:59:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T11:59:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T11:59:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T11:59:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T11:59:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.579+0000 7fbb147d0640 1 -- 172.21.15.94:0/2889746713 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbaf80a2620 msgr2=0x7fbaf80a4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.579+0000 7fbb147d0640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbaf80a2620 0x7fbaf80a4ae0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbb080058c0 tx=0x7fbb08005850 comp rx=0 tx=0).stop 2024-02-17T11:59:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.579+0000 7fbb147d0640 1 -- 172.21.15.94:0/2889746713 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb0c1002d0 msgr2=0x7fbb0c103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.579+0000 7fbb147d0640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb0c1002d0 0x7fbb0c103050 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbb0402c700 tx=0x7fbb04034cc0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.579+0000 7fbb147d0640 1 -- 172.21.15.94:0/2889746713 shutdown_connections 2024-02-17T11:59:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.579+0000 7fbb147d0640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbaf80a2620 0x7fbaf80a4ae0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.579+0000 7fbb147d0640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbb0c1060b0 0x7fbb0c1a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.579+0000 7fbb147d0640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb0c100c70 0x7fbb0c103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.579+0000 7fbb147d0640 1 --2- 172.21.15.94:0/2889746713 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbb0c1002d0 0x7fbb0c103050 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.579+0000 7fbb147d0640 1 -- 172.21.15.94:0/2889746713 >> 172.21.15.94:0/2889746713 conn(0x7fbb0c076ea0 msgr2=0x7fbb0c10f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.579+0000 7fbb147d0640 1 -- 172.21.15.94:0/2889746713 shutdown_connections 2024-02-17T11:59:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:15.579+0000 7fbb147d0640 1 -- 172.21.15.94:0/2889746713 wait complete. 2024-02-17T11:59:16.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:15 smithi189 ceph-mon[36588]: pgmap v37: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 43 MiB/s wr, 1.50k op/s 2024-02-17T11:59:16.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:15 smithi125 ceph-mon[36819]: pgmap v37: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 43 MiB/s wr, 1.50k op/s 2024-02-17T11:59:16.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:15 smithi094 ceph-mon[32499]: pgmap v37: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 43 MiB/s wr, 1.50k op/s 2024-02-17T11:59:18.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:17 smithi189 ceph-mon[36588]: pgmap v38: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 42 MiB/s wr, 1.35k op/s 2024-02-17T11:59:18.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:17 smithi125 ceph-mon[36819]: pgmap v38: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 42 MiB/s wr, 1.35k op/s 2024-02-17T11:59:18.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:17 smithi094 ceph-mon[32499]: pgmap v38: 97 pgs: 97 active+clean; 1.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 42 MiB/s wr, 1.35k op/s 2024-02-17T11:59:20.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:19 smithi189 ceph-mon[36588]: pgmap v39: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 48 MiB/s wr, 1.74k op/s 2024-02-17T11:59:20.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:19 smithi125 ceph-mon[36819]: pgmap v39: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 48 MiB/s wr, 1.74k op/s 2024-02-17T11:59:20.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:19 smithi094 ceph-mon[32499]: pgmap v39: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 48 MiB/s wr, 1.74k op/s 2024-02-17T11:59:20.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 60.0 s, 4067.4 tps, lat 24.646 ms stddev 25.778 2024-02-17T11:59:22.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:21 smithi189 ceph-mon[36588]: pgmap v40: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.9 MiB/s wr, 732 op/s 2024-02-17T11:59:22.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:21 smithi125 ceph-mon[36819]: pgmap v40: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.9 MiB/s wr, 732 op/s 2024-02-17T11:59:22.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:21 smithi094 ceph-mon[32499]: pgmap v40: 97 pgs: 97 active+clean; 1.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.9 MiB/s wr, 732 op/s 2024-02-17T11:59:24.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:23 smithi189 ceph-mon[36588]: pgmap v41: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-02-17T11:59:24.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:23 smithi125 ceph-mon[36819]: pgmap v41: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-02-17T11:59:24.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:23 smithi094 ceph-mon[32499]: pgmap v41: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-02-17T11:59:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 65.0 s, 3697.0 tps, lat 26.925 ms stddev 27.198 2024-02-17T11:59:26.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:25 smithi189 ceph-mon[36588]: pgmap v42: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 814 op/s 2024-02-17T11:59:26.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:25 smithi125 ceph-mon[36819]: pgmap v42: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 814 op/s 2024-02-17T11:59:26.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:25 smithi094 ceph-mon[32499]: pgmap v42: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 11 MiB/s wr, 814 op/s 2024-02-17T11:59:28.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:27 smithi189 ceph-mon[36588]: pgmap v43: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 771 op/s 2024-02-17T11:59:28.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:27 smithi125 ceph-mon[36819]: pgmap v43: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 771 op/s 2024-02-17T11:59:28.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:27 smithi094 ceph-mon[32499]: pgmap v43: 97 pgs: 97 active+clean; 1.1 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 11 MiB/s wr, 771 op/s 2024-02-17T11:59:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.708+0000 7ff42a348640 1 -- 172.21.15.94:0/175850779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff424069420 msgr2=0x7ff424105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.708+0000 7ff42a348640 1 --2- 172.21.15.94:0/175850779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff424069420 0x7ff424105dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff418009930 tx=0x7ff41802bc20 comp rx=0 tx=0).stop 2024-02-17T11:59:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.708+0000 7ff42a348640 1 -- 172.21.15.94:0/175850779 shutdown_connections 2024-02-17T11:59:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.708+0000 7ff42a348640 1 --2- 172.21.15.94:0/175850779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4241064a0 0x7ff42410e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.708+0000 7ff42a348640 1 --2- 172.21.15.94:0/175850779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff424069420 0x7ff424105dc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.708+0000 7ff42a348640 1 --2- 172.21.15.94:0/175850779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff424068a70 0x7ff424068e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.708+0000 7ff42a348640 1 -- 172.21.15.94:0/175850779 >> 172.21.15.94:0/175850779 conn(0x7ff424076ed0 msgr2=0x7ff4240772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.708+0000 7ff42a348640 1 -- 172.21.15.94:0/175850779 shutdown_connections 2024-02-17T11:59:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.708+0000 7ff42a348640 1 -- 172.21.15.94:0/175850779 wait complete. 2024-02-17T11:59:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.709+0000 7ff42a348640 1 Processor -- start 2024-02-17T11:59:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.709+0000 7ff42a348640 1 -- start start 2024-02-17T11:59:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.709+0000 7ff42a348640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff424068a70 0x7ff424197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.709+0000 7ff42a348640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4241064a0 0x7ff424197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.709+0000 7ff42a348640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff42419c000 0x7ff42419c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.709+0000 7ff42a348640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42419c9f0 con 0x7ff42419c000 2024-02-17T11:59:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.709+0000 7ff42a348640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42419cb60 con 0x7ff424068a70 2024-02-17T11:59:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.709+0000 7ff42a348640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42419ccd0 con 0x7ff4241064a0 2024-02-17T11:59:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.709+0000 7ff4288e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff42419c000 0x7ff42419c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.710+0000 7ff4288e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff42419c000 0x7ff42419c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53622/0 (socket says 172.21.15.94:53622) 2024-02-17T11:59:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.710+0000 7ff4288e5640 1 -- 172.21.15.94:0/130506268 learned_addr learned my addr 172.21.15.94:0/130506268 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.710+0000 7ff423fff640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff424068a70 0x7ff424197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.710+0000 7ff4237fe640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4241064a0 0x7ff424197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.710+0000 7ff423fff640 1 -- 172.21.15.94:0/130506268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4241064a0 msgr2=0x7ff424197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.710+0000 7ff423fff640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4241064a0 0x7ff424197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.710+0000 7ff423fff640 1 -- 172.21.15.94:0/130506268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff42419c000 msgr2=0x7ff42419c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.710+0000 7ff423fff640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff42419c000 0x7ff42419c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.710+0000 7ff423fff640 1 -- 172.21.15.94:0/130506268 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff418009560 con 0x7ff424068a70 2024-02-17T11:59:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.711+0000 7ff423fff640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff424068a70 0x7ff424197740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff41400d6d0 tx=0x7ff41400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.711+0000 7ff4217fa640 1 -- 172.21.15.94:0/130506268 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414005220 con 0x7ff424068a70 2024-02-17T11:59:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.711+0000 7ff4217fa640 1 -- 172.21.15.94:0/130506268 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff414009d00 con 0x7ff424068a70 2024-02-17T11:59:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.711+0000 7ff4217fa640 1 -- 172.21.15.94:0/130506268 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414004dd0 con 0x7ff424068a70 2024-02-17T11:59:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.711+0000 7ff42a348640 1 -- 172.21.15.94:0/130506268 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4241a42b0 con 0x7ff424068a70 2024-02-17T11:59:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.711+0000 7ff42a348640 1 -- 172.21.15.94:0/130506268 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4241a46e0 con 0x7ff424068a70 2024-02-17T11:59:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.715+0000 7ff4217fa640 1 -- 172.21.15.94:0/130506268 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7ff414009820 con 0x7ff424068a70 2024-02-17T11:59:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.715+0000 7ff42cffa740 1 Processor -- start 2024-02-17T11:59:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.715+0000 7ff4217fa640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff40c0a25d0 0x7ff40c0a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.715+0000 7ff42cffa740 1 -- start start 2024-02-17T11:59:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff42cffa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a870ad0 0x56312a86f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff4288e5640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff40c0a25d0 0x7ff40c0a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff42cffa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a86fd60 0x56312a8701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff4217fa640 1 -- 172.21.15.94:0/130506268 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff4140cddb0 con 0x7ff424068a70 2024-02-17T11:59:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff42cffa740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a8792e0 0x56312a87b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff42cffa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312a7de9f0 con 0x56312a86fd60 2024-02-17T11:59:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff42cffa740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312a7dbf20 con 0x56312a8792e0 2024-02-17T11:59:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff42cffa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312a7d7700 con 0x56312a870ad0 2024-02-17T11:59:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a86fd60 0x56312a8701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a86fd60 0x56312a8701e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53628/0 (socket says 172.21.15.94:53628) 2024-02-17T11:59:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff423fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a870ad0 0x56312a86f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff4237fe640 1 -- 172.21.15.94:0/2047780490 learned_addr learned my addr 172.21.15.94:0/2047780490 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff4288e5640 1 --2- 172.21.15.94:0/2047780490 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a8792e0 0x56312a87b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff423fff640 1 -- 172.21.15.94:0/2047780490 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a8792e0 msgr2=0x56312a87b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff423fff640 1 --2- 172.21.15.94:0/2047780490 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a8792e0 0x56312a87b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff423fff640 1 -- 172.21.15.94:0/2047780490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a86fd60 msgr2=0x56312a8701e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff423fff640 1 --2- 172.21.15.94:0/2047780490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a86fd60 0x56312a8701e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.716+0000 7ff423fff640 1 -- 172.21.15.94:0/2047780490 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56312a76d7b0 con 0x56312a870ad0 2024-02-17T11:59:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.717+0000 7ff4288e5640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff40c0a25d0 0x7ff40c0a4a90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009fd0 tx=0x7ff41c013c20 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.717+0000 7ff423fff640 1 --2- 172.21.15.94:0/2047780490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a870ad0 0x56312a86f820 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff4140cb310 tx=0x7ff414004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ce13a6159fc8d06 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.717+0000 7ff40a7fc640 1 -- 172.21.15.94:0/2047780490 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41409a590 con 0x56312a870ad0 2024-02-17T11:59:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.717+0000 7ff40a7fc640 1 -- 172.21.15.94:0/2047780490 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff41409e630 con 0x56312a870ad0 2024-02-17T11:59:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.717+0000 7ff40a7fc640 1 -- 172.21.15.94:0/2047780490 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414096c00 con 0x56312a870ad0 2024-02-17T11:59:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.718+0000 7ff42cffa740 1 -- 172.21.15.94:0/2047780490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a870ad0 msgr2=0x56312a86f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.718+0000 7ff42cffa740 1 --2- 172.21.15.94:0/2047780490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a870ad0 0x56312a86f820 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff4140cb310 tx=0x7ff414004fb0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.718+0000 7ff42cffa740 1 -- 172.21.15.94:0/2047780490 shutdown_connections 2024-02-17T11:59:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.718+0000 7ff42cffa740 1 --2- 172.21.15.94:0/2047780490 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a8792e0 0x56312a87b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.718+0000 7ff42cffa740 1 --2- 172.21.15.94:0/2047780490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a86fd60 0x56312a8701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.718+0000 7ff42cffa740 1 --2- 172.21.15.94:0/2047780490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a870ad0 0x56312a86f820 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.718+0000 7ff42cffa740 1 -- 172.21.15.94:0/2047780490 >> 172.21.15.94:0/2047780490 conn(0x56312a837550 msgr2=0x56312a86aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.719+0000 7ff42cffa740 1 -- 172.21.15.94:0/2047780490 shutdown_connections 2024-02-17T11:59:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.719+0000 7ff42cffa740 1 -- 172.21.15.94:0/2047780490 wait complete. 2024-02-17T11:59:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.720+0000 7ff42cffa740 1 -- start start 2024-02-17T11:59:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.720+0000 7ff42cffa740 1 Processor -- start 2024-02-17T11:59:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.720+0000 7ff42cffa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a86fd60 0x56312a8763d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.720+0000 7ff42cffa740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a870ad0 0x56312a876910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.720+0000 7ff42cffa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a8792e0 0x56312a876e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.720+0000 7ff42cffa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312a77cbf0 con 0x56312a86fd60 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.720+0000 7ff42cffa740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312a4ab250 con 0x56312a870ad0 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.720+0000 7ff42cffa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312a75d200 con 0x56312a8792e0 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.721+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a870ad0 0x56312a876910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.721+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a870ad0 0x56312a876910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36662/0 (socket says 172.21.15.94:36662) 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.721+0000 7ff4237fe640 1 -- 172.21.15.94:0/2904854988 learned_addr learned my addr 172.21.15.94:0/2904854988 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.721+0000 7ff423fff640 1 --2- 172.21.15.94:0/2904854988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a86fd60 0x56312a8763d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.721+0000 7ff4288e5640 1 --2- 172.21.15.94:0/2904854988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a8792e0 0x56312a876e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.721+0000 7ff42cffa740 10 client.0 _ll_register_callbacks cb 0x56312a7cbd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.722+0000 7ff423fff640 1 -- 172.21.15.94:0/2904854988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a8792e0 msgr2=0x56312a876e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.722+0000 7ff423fff640 1 --2- 172.21.15.94:0/2904854988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a8792e0 0x56312a876e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.722+0000 7ff423fff640 1 -- 172.21.15.94:0/2904854988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a870ad0 msgr2=0x56312a876910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.722+0000 7ff423fff640 1 --2- 172.21.15.94:0/2904854988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a870ad0 0x56312a876910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.722+0000 7ff423fff640 1 -- 172.21.15.94:0/2904854988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4241a46e0 con 0x56312a86fd60 2024-02-17T11:59:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.722+0000 7ff4037fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:59:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.722+0000 7ff423fff640 1 --2- 172.21.15.94:0/2904854988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a86fd60 0x56312a8763d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff4140cb310 tx=0x7ff4140047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad71891dbbfe7a20 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.723+0000 7ff4037fe640 1 -- 172.21.15.94:0/2904854988 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4140a2c80 con 0x56312a86fd60 2024-02-17T11:59:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.723+0000 7ff4037fe640 1 -- 172.21.15.94:0/2904854988 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff41409a590 con 0x56312a86fd60 2024-02-17T11:59:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.723+0000 7ff42cffa740 1 -- 172.21.15.94:0/2904854988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56312a4d7600 con 0x56312a86fd60 2024-02-17T11:59:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.723+0000 7ff4037fe640 1 -- 172.21.15.94:0/2904854988 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff414096c00 con 0x56312a86fd60 2024-02-17T11:59:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.723+0000 7ff42cffa740 10 client.15834 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.723+0000 7ff42cffa740 1 -- 172.21.15.94:0/2904854988 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x56312a7d5d80 con 0x56312a86fd60 2024-02-17T11:59:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.724+0000 7ff4037fe640 1 -- 172.21.15.94:0/2904854988 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56312a4d7600 con 0x56312a86fd60 2024-02-17T11:59:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.724+0000 7ff4037fe640 1 -- 172.21.15.94:0/2904854988 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff4140b5070 con 0x56312a86fd60 2024-02-17T11:59:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.724+0000 7ff42cffa740 10 client.15834 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.724+0000 7ff42cffa740 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.724+0000 7ff42cffa740 20 client.15834 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.724+0000 7ff42cffa740 1 --2- 172.21.15.94:0/2904854988 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56312a891a60 0x56312a893e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.724+0000 7ff42cffa740 4 client.15834 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.724+0000 7ff42cffa740 1 -- 172.21.15.94:0/2904854988 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56312a5281b0 con 0x56312a891a60 2024-02-17T11:59:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.724+0000 7ff4237fe640 1 --2- 172.21.15.94:0/2904854988 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56312a891a60 0x56312a893e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.724+0000 7ff4037fe640 10 client.15834 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.724+0000 7ff4237fe640 1 --2- 172.21.15.94:0/2904854988 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56312a891a60 0x56312a893e50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967eadd0d3929754 server_cookie=9e376c7e188c85e7 in_seq=0 out_seq=0 2024-02-17T11:59:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.725+0000 7ff4037fe640 1 -- 172.21.15.94:0/2904854988 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56312a5281b0 con 0x56312a891a60 2024-02-17T11:59:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.725+0000 7ff4037fe640 10 client.15834 handle_command_reply: tid=0 2024-02-17T11:59:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.725+0000 7ff42cffa740 1 client.15834 shutdown 2024-02-17T11:59:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.726+0000 7ff42cffa740 1 -- 172.21.15.94:0/2904854988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a86fd60 msgr2=0x56312a8763d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.726+0000 7ff42cffa740 1 --2- 172.21.15.94:0/2904854988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a86fd60 0x56312a8763d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff4140cb310 tx=0x7ff4140047f0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.726+0000 7ff42cffa740 1 -- 172.21.15.94:0/2904854988 shutdown_connections 2024-02-17T11:59:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.726+0000 7ff42cffa740 1 --2- 172.21.15.94:0/2904854988 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56312a891a60 0x56312a893e50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.726+0000 7ff42cffa740 1 --2- 172.21.15.94:0/2904854988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a8792e0 0x56312a876e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.726+0000 7ff4037fe640 0 client.15834 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.726+0000 7ff42cffa740 1 --2- 172.21.15.94:0/2904854988 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a870ad0 0x56312a876910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.726+0000 7ff42cffa740 1 --2- 172.21.15.94:0/2904854988 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a86fd60 0x56312a8763d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.726+0000 7ff42cffa740 1 -- 172.21.15.94:0/2904854988 >> 172.21.15.94:0/2904854988 conn(0x56312a837550 msgr2=0x56312a86c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.726+0000 7ff42cffa740 1 -- 172.21.15.94:0/2904854988 shutdown_connections 2024-02-17T11:59:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.726+0000 7ff42cffa740 1 -- 172.21.15.94:0/2904854988 wait complete. 2024-02-17T11:59:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.726+0000 7ff42cffa740 20 client.15834 trim_cache size 0 max 16384 2024-02-17T11:59:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.746+0000 7ff42cffa740 1 Processor -- start 2024-02-17T11:59:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.746+0000 7ff42cffa740 1 -- start start 2024-02-17T11:59:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.746+0000 7ff42cffa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a86fd60 0x56312a878b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.746+0000 7ff42cffa740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a870ad0 0x56312a876c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.746+0000 7ff42cffa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a8792e0 0x56312a8888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.746+0000 7ff42cffa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312a866310 con 0x56312a8792e0 2024-02-17T11:59:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.746+0000 7ff42cffa740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312a874920 con 0x56312a870ad0 2024-02-17T11:59:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.746+0000 7ff42cffa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312a86fb10 con 0x56312a86fd60 2024-02-17T11:59:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.747+0000 7ff423fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a86fd60 0x56312a878b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.747+0000 7ff423fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a86fd60 0x56312a878b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:53216/0 (socket says 172.21.15.94:53216) 2024-02-17T11:59:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.747+0000 7ff423fff640 1 -- 172.21.15.94:0/1725523624 learned_addr learned my addr 172.21.15.94:0/1725523624 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.747+0000 7ff4237fe640 1 --2- 172.21.15.94:0/1725523624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a870ad0 0x56312a876c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.747+0000 7ff4288e5640 1 --2- 172.21.15.94:0/1725523624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a8792e0 0x56312a8888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.747+0000 7ff4237fe640 1 -- 172.21.15.94:0/1725523624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a86fd60 msgr2=0x56312a878b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.747+0000 7ff4237fe640 1 --2- 172.21.15.94:0/1725523624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a86fd60 0x56312a878b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.747+0000 7ff4237fe640 1 -- 172.21.15.94:0/1725523624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a8792e0 msgr2=0x56312a8888b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.747+0000 7ff4237fe640 1 --2- 172.21.15.94:0/1725523624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a8792e0 0x56312a8888b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.747+0000 7ff4237fe640 1 -- 172.21.15.94:0/1725523624 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56312a7727b0 con 0x56312a870ad0 2024-02-17T11:59:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.747+0000 7ff4237fe640 1 --2- 172.21.15.94:0/1725523624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a870ad0 0x56312a876c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff418009230 tx=0x7ff418009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24ed8834b95ce5da server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.747+0000 7ff409ffb640 1 -- 172.21.15.94:0/1725523624 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418004820 con 0x56312a870ad0 2024-02-17T11:59:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.748+0000 7ff409ffb640 1 -- 172.21.15.94:0/1725523624 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff418049be0 con 0x56312a870ad0 2024-02-17T11:59:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.748+0000 7ff409ffb640 1 -- 172.21.15.94:0/1725523624 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418049600 con 0x56312a870ad0 2024-02-17T11:59:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.749+0000 7ff42cffa740 1 -- 172.21.15.94:0/1725523624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a870ad0 msgr2=0x56312a876c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.749+0000 7ff42cffa740 1 --2- 172.21.15.94:0/1725523624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a870ad0 0x56312a876c70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff418009230 tx=0x7ff418009260 comp rx=0 tx=0).stop 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.749+0000 7ff42cffa740 1 -- 172.21.15.94:0/1725523624 shutdown_connections 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.749+0000 7ff42cffa740 1 --2- 172.21.15.94:0/1725523624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a8792e0 0x56312a8888b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.749+0000 7ff42cffa740 1 --2- 172.21.15.94:0/1725523624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a870ad0 0x56312a876c70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.749+0000 7ff42cffa740 1 --2- 172.21.15.94:0/1725523624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a86fd60 0x56312a878b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.749+0000 7ff42cffa740 1 -- 172.21.15.94:0/1725523624 >> 172.21.15.94:0/1725523624 conn(0x56312a837550 msgr2=0x56312a86a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.749+0000 7ff42cffa740 1 -- 172.21.15.94:0/1725523624 shutdown_connections 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.750+0000 7ff42cffa740 1 -- 172.21.15.94:0/1725523624 wait complete. 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.750+0000 7ff42cffa740 1 -- start start 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.750+0000 7ff42cffa740 1 Processor -- start 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.750+0000 7ff42cffa740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a86fd60 0x56312a873240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff42cffa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a870ad0 0x56312a873780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff42cffa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a8792e0 0x56312a873cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff42cffa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312a8e7fa0 con 0x56312a870ad0 2024-02-17T11:59:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff42cffa740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312a8e8110 con 0x56312a86fd60 2024-02-17T11:59:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff42cffa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56312a8e82d0 con 0x56312a8792e0 2024-02-17T11:59:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff423fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a86fd60 0x56312a873240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a870ad0 0x56312a873780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff423fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a86fd60 0x56312a873240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36678/0 (socket says 172.21.15.94:36678) 2024-02-17T11:59:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff4237fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a870ad0 0x56312a873780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53652/0 (socket says 172.21.15.94:53652) 2024-02-17T11:59:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff423fff640 1 -- 172.21.15.94:0/621024060 learned_addr learned my addr 172.21.15.94:0/621024060 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff42cffa740 10 client.0 _ll_register_callbacks cb 0x7ff414096170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff4288e5640 1 --2- 172.21.15.94:0/621024060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a8792e0 0x56312a873cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff423fff640 1 -- 172.21.15.94:0/621024060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a8792e0 msgr2=0x56312a873cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff423fff640 1 --2- 172.21.15.94:0/621024060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a8792e0 0x56312a873cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff423fff640 1 -- 172.21.15.94:0/621024060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a870ad0 msgr2=0x56312a873780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff423fff640 1 --2- 172.21.15.94:0/621024060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a870ad0 0x56312a873780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff423fff640 1 -- 172.21.15.94:0/621024060 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42410e860 con 0x56312a86fd60 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff423fff640 1 --2- 172.21.15.94:0/621024060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a86fd60 0x56312a873240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff41400d6a0 tx=0x7ff414009740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9606fc7d713b9a36 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff4037fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff4037fe640 1 -- 172.21.15.94:0/621024060 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414010670 con 0x56312a86fd60 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff4037fe640 1 -- 172.21.15.94:0/621024060 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff41409e120 con 0x56312a86fd60 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.751+0000 7ff42cffa740 1 -- 172.21.15.94:0/621024060 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff41409e630 con 0x56312a86fd60 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.752+0000 7ff4037fe640 1 -- 172.21.15.94:0/621024060 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff414096830 con 0x56312a86fd60 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.752+0000 7ff42cffa740 10 client.25313 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.752+0000 7ff42cffa740 1 -- 172.21.15.94:0/621024060 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x56312a798540 con 0x56312a86fd60 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.752+0000 7ff4037fe640 1 -- 172.21.15.94:0/621024060 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff41409e630 con 0x56312a86fd60 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.752+0000 7ff4037fe640 1 -- 172.21.15.94:0/621024060 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff4140b5070 con 0x56312a86fd60 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.752+0000 7ff42cffa740 10 client.25313 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.752+0000 7ff42cffa740 10 client.25313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.752+0000 7ff42cffa740 20 client.25313 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.752+0000 7ff42cffa740 1 --2- 172.21.15.94:0/621024060 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56312a891a60 0x56312a8eb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.752+0000 7ff42cffa740 4 client.25313 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T11:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.752+0000 7ff42cffa740 1 -- 172.21.15.94:0/621024060 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56312a514550 con 0x56312a891a60 2024-02-17T11:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.752+0000 7ff4237fe640 1 --2- 172.21.15.94:0/621024060 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56312a891a60 0x56312a8eb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.753+0000 7ff4037fe640 10 client.25313 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.753+0000 7ff4237fe640 1 --2- 172.21.15.94:0/621024060 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56312a891a60 0x56312a8eb5a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2e6e8d0dfe60ab5 server_cookie=e25cec70fbdde5fe in_seq=0 out_seq=0 2024-02-17T11:59:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.771+0000 7ff4037fe640 1 -- 172.21.15.94:0/621024060 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56312a514550 con 0x56312a891a60 2024-02-17T11:59:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.771+0000 7ff4037fe640 10 client.25313 handle_command_reply: tid=0 2024-02-17T11:59:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.771+0000 7ff42cffa740 1 client.25313 shutdown 2024-02-17T11:59:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.771+0000 7ff42cffa740 1 -- 172.21.15.94:0/621024060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a86fd60 msgr2=0x56312a873240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.771+0000 7ff42cffa740 1 --2- 172.21.15.94:0/621024060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a86fd60 0x56312a873240 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff41400d6a0 tx=0x7ff414009740 comp rx=0 tx=0).stop 2024-02-17T11:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.771+0000 7ff42cffa740 1 -- 172.21.15.94:0/621024060 shutdown_connections 2024-02-17T11:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.771+0000 7ff42cffa740 1 --2- 172.21.15.94:0/621024060 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56312a891a60 0x56312a8eb5a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.771+0000 7ff42cffa740 1 --2- 172.21.15.94:0/621024060 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56312a8792e0 0x56312a873cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.771+0000 7ff42cffa740 1 --2- 172.21.15.94:0/621024060 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56312a870ad0 0x56312a873780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.771+0000 7ff42cffa740 1 --2- 172.21.15.94:0/621024060 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56312a86fd60 0x56312a873240 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.771+0000 7ff42cffa740 1 -- 172.21.15.94:0/621024060 >> 172.21.15.94:0/621024060 conn(0x56312a837550 msgr2=0x56312a874c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.771+0000 7ff4037fe640 0 client.25313 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42cffa740 1 -- 172.21.15.94:0/621024060 shutdown_connections 2024-02-17T11:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42cffa740 1 -- 172.21.15.94:0/621024060 wait complete. 2024-02-17T11:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42cffa740 20 client.25313 trim_cache size 0 max 16384 2024-02-17T11:59:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T11:59:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T11:59:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T11:59:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T11:59:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42a348640 1 -- 172.21.15.94:0/130506268 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff40c0a25d0 msgr2=0x7ff40c0a4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42a348640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff40c0a25d0 0x7ff40c0a4a90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff41c009fd0 tx=0x7ff41c013c20 comp rx=0 tx=0).stop 2024-02-17T11:59:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42a348640 1 -- 172.21.15.94:0/130506268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff424068a70 msgr2=0x7ff424197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42a348640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff424068a70 0x7ff424197740 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff41400d6d0 tx=0x7ff41400db90 comp rx=0 tx=0).stop 2024-02-17T11:59:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42a348640 1 -- 172.21.15.94:0/130506268 shutdown_connections 2024-02-17T11:59:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42a348640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff40c0a25d0 0x7ff40c0a4a90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42a348640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff42419c000 0x7ff42419c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42a348640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4241064a0 0x7ff424197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42a348640 1 --2- 172.21.15.94:0/130506268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff424068a70 0x7ff424197740 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42a348640 1 -- 172.21.15.94:0/130506268 >> 172.21.15.94:0/130506268 conn(0x7ff424076ed0 msgr2=0x7ff4240fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42a348640 1 -- 172.21.15.94:0/130506268 shutdown_connections 2024-02-17T11:59:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:28.772+0000 7ff42a348640 1 -- 172.21.15.94:0/130506268 wait complete. 2024-02-17T11:59:30.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:29 smithi189 ceph-mon[36588]: pgmap v44: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-02-17T11:59:30.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:29 smithi125 ceph-mon[36819]: pgmap v44: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-02-17T11:59:30.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:29 smithi094 ceph-mon[32499]: pgmap v44: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 MiB/s wr, 1.09k op/s 2024-02-17T11:59:30.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 70.0 s, 3832.0 tps, lat 26.238 ms stddev 29.122 2024-02-17T11:59:32.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:31 smithi189 ceph-mon[36588]: pgmap v45: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.4 MiB/s wr, 693 op/s 2024-02-17T11:59:32.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:31 smithi125 ceph-mon[36819]: pgmap v45: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.4 MiB/s wr, 693 op/s 2024-02-17T11:59:32.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:31 smithi094 ceph-mon[32499]: pgmap v45: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.4 MiB/s wr, 693 op/s 2024-02-17T11:59:34.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:33 smithi189 ceph-mon[36588]: pgmap v46: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 1.02k op/s 2024-02-17T11:59:34.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:33 smithi125 ceph-mon[36819]: pgmap v46: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 1.02k op/s 2024-02-17T11:59:34.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:33 smithi094 ceph-mon[32499]: pgmap v46: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 14 MiB/s wr, 1.02k op/s 2024-02-17T11:59:34.782 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T11:59:34.782 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T11:59:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.919+0000 7f59b486f640 1 -- 172.21.15.94:0/3345175461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b0105780 msgr2=0x7f59b0105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.919+0000 7f59b486f640 1 --2- 172.21.15.94:0/3345175461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b0105780 0x7f59b0105b60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f59a0020b50 tx=0x7f59a0040dc0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.919+0000 7f59b486f640 1 -- 172.21.15.94:0/3345175461 shutdown_connections 2024-02-17T11:59:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.919+0000 7f59b486f640 1 --2- 172.21.15.94:0/3345175461 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b0100130 0x7f59b010c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.919+0000 7f59b486f640 1 --2- 172.21.15.94:0/3345175461 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59b00ff770 0x7f59b00ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.919+0000 7f59b486f640 1 --2- 172.21.15.94:0/3345175461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b0105780 0x7f59b0105b60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.919+0000 7f59b486f640 1 -- 172.21.15.94:0/3345175461 >> 172.21.15.94:0/3345175461 conn(0x7f59b00765b0 msgr2=0x7f59b00fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:34.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.920+0000 7f59b486f640 1 -- 172.21.15.94:0/3345175461 shutdown_connections 2024-02-17T11:59:34.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.920+0000 7f59b486f640 1 -- 172.21.15.94:0/3345175461 wait complete. 2024-02-17T11:59:34.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.921+0000 7f59b486f640 1 Processor -- start 2024-02-17T11:59:34.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59b486f640 1 -- start start 2024-02-17T11:59:34.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59b486f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b00ff770 0x7f59b01078d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59b486f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59b0100130 0x7f59b0107e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59b486f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b0105780 0x7f59b0108350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59b486f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0198f10 con 0x7f59b0105780 2024-02-17T11:59:34.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59b486f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0199050 con 0x7f59b0100130 2024-02-17T11:59:34.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59b486f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b0199210 con 0x7f59b00ff770 2024-02-17T11:59:34.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b0105780 0x7f59b0108350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59a5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59b0100130 0x7f59b0107e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59ae59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b00ff770 0x7f59b01078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59a5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59b0100130 0x7f59b0107e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44396/0 (socket says 172.21.15.94:44396) 2024-02-17T11:59:34.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b0105780 0x7f59b0108350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39598/0 (socket says 172.21.15.94:39598) 2024-02-17T11:59:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59a5d9b640 1 -- 172.21.15.94:0/3690636525 learned_addr learned my addr 172.21.15.94:0/3690636525 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59a5d9b640 1 -- 172.21.15.94:0/3690636525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b00ff770 msgr2=0x7f59b01078d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59a5d9b640 1 --2- 172.21.15.94:0/3690636525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b00ff770 0x7f59b01078d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59a5d9b640 1 -- 172.21.15.94:0/3690636525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b0105780 msgr2=0x7f59b0108350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59a5d9b640 1 --2- 172.21.15.94:0/3690636525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b0105780 0x7f59b0108350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.922+0000 7f59a5d9b640 1 -- 172.21.15.94:0/3690636525 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a0020780 con 0x7f59b0100130 2024-02-17T11:59:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.923+0000 7f59a5d9b640 1 --2- 172.21.15.94:0/3690636525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59b0100130 0x7f59b0107e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f599800a8b0 tx=0x7f599800ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.923+0000 7f59a7fff640 1 -- 172.21.15.94:0/3690636525 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998004170 con 0x7f59b0100130 2024-02-17T11:59:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.923+0000 7f59a7fff640 1 -- 172.21.15.94:0/3690636525 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f599800ee40 con 0x7f59b0100130 2024-02-17T11:59:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.923+0000 7f59b486f640 1 -- 172.21.15.94:0/3690636525 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b01994f0 con 0x7f59b0100130 2024-02-17T11:59:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.923+0000 7f59a7fff640 1 -- 172.21.15.94:0/3690636525 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599800c870 con 0x7f59b0100130 2024-02-17T11:59:34.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.923+0000 7f59b486f640 1 -- 172.21.15.94:0/3690636525 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b01999b0 con 0x7f59b0100130 2024-02-17T11:59:34.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.926+0000 7f59b7440740 1 Processor -- start 2024-02-17T11:59:34.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.926+0000 7f59b7440740 1 -- start start 2024-02-17T11:59:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.926+0000 7f59b7440740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb28fd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.926+0000 7f59b7440740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2906270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.926+0000 7f59b7440740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2908ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.926+0000 7f59b7440740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb286c280 con 0x558eb28fdbd0 2024-02-17T11:59:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.926+0000 7f59b7440740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb28697b0 con 0x558eb28fd2b0 2024-02-17T11:59:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.926+0000 7f59b7440740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb2864f90 con 0x558eb29067b0 2024-02-17T11:59:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59a7fff640 1 -- 172.21.15.94:0/3690636525 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f599800ca30 con 0x7f59b0100130 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59aed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb28fd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59aed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb28fd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44410/0 (socket says 172.21.15.94:44410) 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59aed9d640 1 -- 172.21.15.94:0/1502139570 learned_addr learned my addr 172.21.15.94:0/1502139570 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59ae59c640 1 --2- 172.21.15.94:0/1502139570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2906270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59a5d9b640 1 --2- 172.21.15.94:0/1502139570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2908ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59aed9d640 1 -- 172.21.15.94:0/1502139570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 msgr2=0x558eb2908ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59aed9d640 1 --2- 172.21.15.94:0/1502139570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2908ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59aed9d640 1 -- 172.21.15.94:0/1502139570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 msgr2=0x558eb2906270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59aed9d640 1 --2- 172.21.15.94:0/1502139570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2906270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59aed9d640 1 -- 172.21.15.94:0/1502139570 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558eb2823570 con 0x558eb28fd2b0 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59a5d9b640 1 --2- 172.21.15.94:0/1502139570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2908ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59aed9d640 1 --2- 172.21.15.94:0/1502139570 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb28fd690 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f59a8003fa0 tx=0x7f59a8008380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55254a91b1f1a224 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59a4d99640 1 -- 172.21.15.94:0/1502139570 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a801a070 con 0x558eb28fd2b0 2024-02-17T11:59:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59a7fff640 1 --2- 172.21.15.94:0/3690636525 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f599c0a2350 0x7f599c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59a4d99640 1 -- 172.21.15.94:0/1502139570 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59a8008e70 con 0x558eb28fd2b0 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59a4d99640 1 -- 172.21.15.94:0/1502139570 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8010040 con 0x558eb28fd2b0 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.927+0000 7f59ae59c640 1 --2- 172.21.15.94:0/3690636525 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f599c0a2350 0x7f599c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.928+0000 7f59a7fff640 1 -- 172.21.15.94:0/3690636525 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f59980ce3f0 con 0x7f59b0100130 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.928+0000 7f59b7440740 1 -- 172.21.15.94:0/1502139570 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 msgr2=0x558eb28fd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.928+0000 7f59ae59c640 1 --2- 172.21.15.94:0/3690636525 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f599c0a2350 0x7f599c0a4810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f59a0020b20 tx=0x7f59a0005670 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.928+0000 7f59b7440740 1 --2- 172.21.15.94:0/1502139570 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb28fd690 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f59a8003fa0 tx=0x7f59a8008380 comp rx=0 tx=0).stop 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.928+0000 7f59b7440740 1 -- 172.21.15.94:0/1502139570 shutdown_connections 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.928+0000 7f59b7440740 1 --2- 172.21.15.94:0/1502139570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2908ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.928+0000 7f59b7440740 1 --2- 172.21.15.94:0/1502139570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2906270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.928+0000 7f59b7440740 1 --2- 172.21.15.94:0/1502139570 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb28fd690 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.928+0000 7f59b7440740 1 -- 172.21.15.94:0/1502139570 >> 172.21.15.94:0/1502139570 conn(0x558eb28f7990 msgr2=0x558eb28f7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.928+0000 7f59b7440740 1 -- 172.21.15.94:0/1502139570 shutdown_connections 2024-02-17T11:59:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.928+0000 7f59b7440740 1 -- 172.21.15.94:0/1502139570 wait complete. 2024-02-17T11:59:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.928+0000 7f59b7440740 1 -- start start 2024-02-17T11:59:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.929+0000 7f59b7440740 1 Processor -- start 2024-02-17T11:59:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.929+0000 7f59b7440740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb29061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.929+0000 7f59b7440740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2903550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.929+0000 7f59b7440740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2903b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.929+0000 7f59ae59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb29061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.929+0000 7f59ae59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb29061c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44424/0 (socket says 172.21.15.94:44424) 2024-02-17T11:59:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.929+0000 7f59ae59c640 1 -- 172.21.15.94:0/2658856919 learned_addr learned my addr 172.21.15.94:0/2658856919 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.929+0000 7f59b7440740 1 -- 172.21.15.94:0/2658856919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb2538430 con 0x558eb28fdbd0 2024-02-17T11:59:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.929+0000 7f59b7440740 1 -- 172.21.15.94:0/2658856919 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb24f4e00 con 0x558eb28fd2b0 2024-02-17T11:59:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.929+0000 7f59b7440740 1 -- 172.21.15.94:0/2658856919 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb27e9fd0 con 0x558eb29067b0 2024-02-17T11:59:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.929+0000 7f59a5d9b640 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2903550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.929+0000 7f59aed9d640 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2903b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.930+0000 7f59ae59c640 1 -- 172.21.15.94:0/2658856919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 msgr2=0x558eb2903b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.930+0000 7f59ae59c640 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2903b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.930+0000 7f59ae59c640 1 -- 172.21.15.94:0/2658856919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 msgr2=0x558eb2903550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.930+0000 7f59ae59c640 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2903550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.930+0000 7f59ae59c640 1 -- 172.21.15.94:0/2658856919 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a8003c50 con 0x558eb28fd2b0 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.930+0000 7f59aed9d640 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2903b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.930+0000 7f59b7440740 10 client.0 _ll_register_callbacks cb 0x558eb283ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.930+0000 7f59b7440740 1 -- 172.21.15.94:0/2658856919 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558eb25647e0 con 0x558eb28fd2b0 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.930+0000 7f59ae59c640 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb29061c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f59a0004fd0 tx=0x7f59a0040dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aff9480f6e043d1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.930+0000 7f5995ffb640 10 client.25325 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.930+0000 7f5995ffb640 1 -- 172.21.15.94:0/2658856919 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a0004e20 con 0x558eb28fd2b0 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.931+0000 7f5995ffb640 1 -- 172.21.15.94:0/2658856919 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f59a0004040 con 0x558eb28fd2b0 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.931+0000 7f5995ffb640 1 -- 172.21.15.94:0/2658856919 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59a0004240 con 0x558eb28fd2b0 2024-02-17T11:59:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.931+0000 7f59b7440740 10 client.25325 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:34.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.931+0000 7f59b7440740 1 -- 172.21.15.94:0/2658856919 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({fsmap=21}) v3 -- 0x558eb2863610 con 0x558eb28fd2b0 2024-02-17T11:59:34.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.931+0000 7f5995ffb640 1 -- 172.21.15.94:0/2658856919 <== mon.2 v2:172.21.15.125:3300/0 4 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59a00043a0 con 0x558eb28fd2b0 2024-02-17T11:59:34.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.931+0000 7f59b7440740 10 client.25325 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:34.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.931+0000 7f59b7440740 10 client.25325 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:34.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.931+0000 7f59b7440740 20 client.25325 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:34.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.931+0000 7f59b7440740 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558eb291ed40 0x558eb2921200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.931+0000 7f59b7440740 4 client.25325 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:34.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.931+0000 7f59b7440740 1 -- 172.21.15.94:0/2658856919 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558eb25a1720 con 0x558eb291ed40 2024-02-17T11:59:34.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.931+0000 7f59a5d9b640 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558eb291ed40 0x558eb2921200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.932+0000 7f5995ffb640 10 client.25325 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:34.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.932+0000 7f59a5d9b640 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558eb291ed40 0x558eb2921200 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16b7939f819ba38f server_cookie=73641250104df9bb in_seq=0 out_seq=0 2024-02-17T11:59:34.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.932+0000 7f5995ffb640 1 -- 172.21.15.94:0/2658856919 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558eb25a1720 con 0x558eb291ed40 2024-02-17T11:59:34.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.932+0000 7f5995ffb640 10 client.25325 handle_command_reply: tid=0 2024-02-17T11:59:34.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.932+0000 7f59b7440740 1 client.25325 shutdown 2024-02-17T11:59:34.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.932+0000 7f59b7440740 1 -- 172.21.15.94:0/2658856919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 msgr2=0x558eb29061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:34.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.932+0000 7f59b7440740 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb29061c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f59a0004fd0 tx=0x7f59a0040dc0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.932+0000 7f59b7440740 1 -- 172.21.15.94:0/2658856919 shutdown_connections 2024-02-17T11:59:34.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.932+0000 7f59b7440740 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558eb291ed40 0x558eb2921200 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.932+0000 7f59b7440740 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2903b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.932+0000 7f5995ffb640 0 client.25325 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:34.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.932+0000 7f59b7440740 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2903550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.932+0000 7f59b7440740 1 --2- 172.21.15.94:0/2658856919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb29061c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.933+0000 7f59b7440740 1 -- 172.21.15.94:0/2658856919 >> 172.21.15.94:0/2658856919 conn(0x558eb28f7990 msgr2=0x558eb28f9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:34.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.933+0000 7f59b7440740 1 -- 172.21.15.94:0/2658856919 shutdown_connections 2024-02-17T11:59:34.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.933+0000 7f59b7440740 1 -- 172.21.15.94:0/2658856919 wait complete. 2024-02-17T11:59:34.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.933+0000 7f59b7440740 20 client.25325 trim_cache size 0 max 16384 2024-02-17T11:59:34.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.955+0000 7f59b7440740 1 Processor -- start 2024-02-17T11:59:34.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.955+0000 7f59b7440740 1 -- start start 2024-02-17T11:59:34.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.956+0000 7f59b7440740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fd2b0 0x558eb2905c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.956+0000 7f59b7440740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb28fdbd0 0x558eb29061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.956+0000 7f59ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fd2b0 0x558eb2905c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.956+0000 7f59b7440740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb29067b0 0x558eb2915ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.956+0000 7f59b7440740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb28f8a10 con 0x558eb28fd2b0 2024-02-17T11:59:34.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.956+0000 7f59ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fd2b0 0x558eb2905c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39616/0 (socket says 172.21.15.94:39616) 2024-02-17T11:59:34.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.956+0000 7f59ae59c640 1 -- 172.21.15.94:0/1470322779 learned_addr learned my addr 172.21.15.94:0/1470322779 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:34.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.956+0000 7f59b7440740 1 -- 172.21.15.94:0/1470322779 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb28fe2e0 con 0x558eb29067b0 2024-02-17T11:59:34.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.956+0000 7f59b7440740 1 -- 172.21.15.94:0/1470322779 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb2908f10 con 0x558eb28fdbd0 2024-02-17T11:59:34.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.956+0000 7f59aed9d640 1 --2- 172.21.15.94:0/1470322779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb29067b0 0x558eb2915ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.956+0000 7f59a5d9b640 1 --2- 172.21.15.94:0/1470322779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb28fdbd0 0x558eb29061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.957+0000 7f59ae59c640 1 -- 172.21.15.94:0/1470322779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb28fdbd0 msgr2=0x558eb29061c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:34.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.957+0000 7f59ae59c640 1 --2- 172.21.15.94:0/1470322779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb28fdbd0 0x558eb29061c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.957+0000 7f59ae59c640 1 -- 172.21.15.94:0/1470322779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb29067b0 msgr2=0x558eb2915ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:34.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.957+0000 7f59ae59c640 1 --2- 172.21.15.94:0/1470322779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb29067b0 0x558eb2915ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.957+0000 7f59ae59c640 1 -- 172.21.15.94:0/1470322779 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558eb27ffac0 con 0x558eb28fd2b0 2024-02-17T11:59:34.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.957+0000 7f59aed9d640 1 --2- 172.21.15.94:0/1470322779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb29067b0 0x558eb2915ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:34.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.958+0000 7f59ae59c640 1 --2- 172.21.15.94:0/1470322779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fd2b0 0x558eb2905c80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f59a0020b50 tx=0x7f59a001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba180f921b26666 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:34.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.958+0000 7f5997fff640 1 -- 172.21.15.94:0/1470322779 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a00544e0 con 0x558eb28fd2b0 2024-02-17T11:59:34.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.958+0000 7f5997fff640 1 -- 172.21.15.94:0/1470322779 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558eb25647e0 con 0x558eb28fd2b0 2024-02-17T11:59:34.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.958+0000 7f5997fff640 1 -- 172.21.15.94:0/1470322779 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a005e7f0 con 0x558eb28fd2b0 2024-02-17T11:59:34.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.959+0000 7f59b7440740 1 -- 172.21.15.94:0/1470322779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fd2b0 msgr2=0x558eb2905c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:34.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.959+0000 7f59b7440740 1 --2- 172.21.15.94:0/1470322779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fd2b0 0x558eb2905c80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f59a0020b50 tx=0x7f59a001c000 comp rx=0 tx=0).stop 2024-02-17T11:59:34.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.959+0000 7f59b7440740 1 -- 172.21.15.94:0/1470322779 shutdown_connections 2024-02-17T11:59:34.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.959+0000 7f59b7440740 1 --2- 172.21.15.94:0/1470322779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb29067b0 0x558eb2915ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.959+0000 7f59b7440740 1 --2- 172.21.15.94:0/1470322779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb28fdbd0 0x558eb29061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.959+0000 7f59b7440740 1 --2- 172.21.15.94:0/1470322779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fd2b0 0x558eb2905c80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.959+0000 7f59b7440740 1 -- 172.21.15.94:0/1470322779 >> 172.21.15.94:0/1470322779 conn(0x558eb28f7990 msgr2=0x558eb28fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:34.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.959+0000 7f59b7440740 1 -- 172.21.15.94:0/1470322779 shutdown_connections 2024-02-17T11:59:34.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.959+0000 7f59b7440740 1 -- 172.21.15.94:0/1470322779 wait complete. 2024-02-17T11:59:34.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.959+0000 7f59b7440740 1 -- start start 2024-02-17T11:59:34.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.959+0000 7f59b7440740 1 Processor -- start 2024-02-17T11:59:34.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59b7440740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb2912880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59b7440740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2912dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59b7440740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2913300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59b7440740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb2916840 con 0x558eb28fdbd0 2024-02-17T11:59:34.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59b7440740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb2975320 con 0x558eb28fd2b0 2024-02-17T11:59:34.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59b7440740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558eb29754b0 con 0x558eb29067b0 2024-02-17T11:59:34.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59a5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2912dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59ae59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb2912880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59a5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2912dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39628/0 (socket says 172.21.15.94:39628) 2024-02-17T11:59:34.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59a5d9b640 1 -- 172.21.15.94:0/4059121349 learned_addr learned my addr 172.21.15.94:0/4059121349 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:34.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59b7440740 10 client.0 _ll_register_callbacks cb 0x7f59a8003b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:34.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59aed9d640 1 --2- 172.21.15.94:0/4059121349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2913300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59a5d9b640 1 -- 172.21.15.94:0/4059121349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 msgr2=0x558eb2913300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:34.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59a5d9b640 1 --2- 172.21.15.94:0/4059121349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2913300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59a5d9b640 1 -- 172.21.15.94:0/4059121349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 msgr2=0x558eb2912880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:34.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59a5d9b640 1 --2- 172.21.15.94:0/4059121349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb2912880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.960+0000 7f59a5d9b640 1 -- 172.21.15.94:0/4059121349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558eb2823570 con 0x558eb28fdbd0 2024-02-17T11:59:34.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.961+0000 7f59a5d9b640 1 --2- 172.21.15.94:0/4059121349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2912dc0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f59980cba10 tx=0x7f5998016f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967482f09cb150e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:34.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.961+0000 7f5995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:59:34.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.961+0000 7f5995ffb640 1 -- 172.21.15.94:0/4059121349 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59980972a0 con 0x558eb28fdbd0 2024-02-17T11:59:34.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.961+0000 7f5995ffb640 1 -- 172.21.15.94:0/4059121349 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5998097400 con 0x558eb28fdbd0 2024-02-17T11:59:34.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.962+0000 7f59b7440740 1 -- 172.21.15.94:0/4059121349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59a8008e70 con 0x558eb28fdbd0 2024-02-17T11:59:34.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.962+0000 7f5995ffb640 1 -- 172.21.15.94:0/4059121349 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f599800cce0 con 0x558eb28fdbd0 2024-02-17T11:59:34.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.962+0000 7f59b7440740 10 client.15858 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:34.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.962+0000 7f59b7440740 1 -- 172.21.15.94:0/4059121349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x558eb2863610 con 0x558eb28fdbd0 2024-02-17T11:59:34.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.962+0000 7f5995ffb640 1 -- 172.21.15.94:0/4059121349 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59a8008e70 con 0x558eb28fdbd0 2024-02-17T11:59:34.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.962+0000 7f5995ffb640 1 -- 172.21.15.94:0/4059121349 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59980cbc90 con 0x558eb28fdbd0 2024-02-17T11:59:34.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.963+0000 7f59b7440740 10 client.15858 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:34.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.963+0000 7f59b7440740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:34.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.963+0000 7f59b7440740 20 client.15858 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:34.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.963+0000 7f59b7440740 1 --2- 172.21.15.94:0/4059121349 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558eb291ed40 0x558eb2978d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:34.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.963+0000 7f59b7440740 4 client.15858 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T11:59:34.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.963+0000 7f59b7440740 1 -- 172.21.15.94:0/4059121349 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558eb25f0890 con 0x558eb291ed40 2024-02-17T11:59:34.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.963+0000 7f59ae59c640 1 --2- 172.21.15.94:0/4059121349 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558eb291ed40 0x558eb2978d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:34.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.963+0000 7f5995ffb640 10 client.15858 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:34.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.963+0000 7f59ae59c640 1 --2- 172.21.15.94:0/4059121349 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558eb291ed40 0x558eb2978d00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf309f2e85b0632 server_cookie=6c4d188e41db7a61 in_seq=0 out_seq=0 2024-02-17T11:59:34.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.963+0000 7f5995ffb640 1 -- 172.21.15.94:0/4059121349 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558eb25f0890 con 0x558eb291ed40 2024-02-17T11:59:34.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.963+0000 7f5995ffb640 10 client.15858 handle_command_reply: tid=0 2024-02-17T11:59:34.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.964+0000 7f59b7440740 1 client.15858 shutdown 2024-02-17T11:59:34.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.964+0000 7f59b7440740 1 -- 172.21.15.94:0/4059121349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 msgr2=0x558eb2912dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:34.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.964+0000 7f59b7440740 1 --2- 172.21.15.94:0/4059121349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2912dc0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f59980cba10 tx=0x7f5998016f90 comp rx=0 tx=0).stop 2024-02-17T11:59:34.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.964+0000 7f59b7440740 1 -- 172.21.15.94:0/4059121349 shutdown_connections 2024-02-17T11:59:34.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.964+0000 7f59b7440740 1 --2- 172.21.15.94:0/4059121349 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558eb291ed40 0x558eb2978d00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.964+0000 7f59b7440740 1 --2- 172.21.15.94:0/4059121349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558eb29067b0 0x558eb2913300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.964+0000 7f59b7440740 1 --2- 172.21.15.94:0/4059121349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558eb28fdbd0 0x558eb2912dc0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.964+0000 7f59b7440740 1 --2- 172.21.15.94:0/4059121349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558eb28fd2b0 0x558eb2912880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.964+0000 7f59b7440740 1 -- 172.21.15.94:0/4059121349 >> 172.21.15.94:0/4059121349 conn(0x558eb28f7990 msgr2=0x558eb2901d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:34.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.964+0000 7f5995ffb640 0 client.15858 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:34.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.964+0000 7f59b7440740 1 -- 172.21.15.94:0/4059121349 shutdown_connections 2024-02-17T11:59:34.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.964+0000 7f59b7440740 1 -- 172.21.15.94:0/4059121349 wait complete. 2024-02-17T11:59:34.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.964+0000 7f59b7440740 20 client.15858 trim_cache size 0 max 16384 2024-02-17T11:59:34.971 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:59:34.972 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T11:59:34.972 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T11:59:34.972 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:59:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.966+0000 7f59b486f640 1 -- 172.21.15.94:0/3690636525 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f599c0a2350 msgr2=0x7f599c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.966+0000 7f59b486f640 1 --2- 172.21.15.94:0/3690636525 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f599c0a2350 0x7f599c0a4810 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f59a0020b20 tx=0x7f59a0005670 comp rx=0 tx=0).stop 2024-02-17T11:59:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.966+0000 7f59b486f640 1 -- 172.21.15.94:0/3690636525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59b0100130 msgr2=0x7f59b0107e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.966+0000 7f59b486f640 1 --2- 172.21.15.94:0/3690636525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59b0100130 0x7f59b0107e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f599800a8b0 tx=0x7f599800ad70 comp rx=0 tx=0).stop 2024-02-17T11:59:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.966+0000 7f59b486f640 1 -- 172.21.15.94:0/3690636525 shutdown_connections 2024-02-17T11:59:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.966+0000 7f59b486f640 1 --2- 172.21.15.94:0/3690636525 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f599c0a2350 0x7f599c0a4810 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.966+0000 7f59b486f640 1 --2- 172.21.15.94:0/3690636525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b0105780 0x7f59b0108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.966+0000 7f59b486f640 1 --2- 172.21.15.94:0/3690636525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59b0100130 0x7f59b0107e10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.966+0000 7f59b486f640 1 --2- 172.21.15.94:0/3690636525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b00ff770 0x7f59b01078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.966+0000 7f59b486f640 1 -- 172.21.15.94:0/3690636525 >> 172.21.15.94:0/3690636525 conn(0x7f59b00765b0 msgr2=0x7f59b01027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.966+0000 7f59b486f640 1 -- 172.21.15.94:0/3690636525 shutdown_connections 2024-02-17T11:59:34.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:34.967+0000 7f59b486f640 1 -- 172.21.15.94:0/3690636525 wait complete. 2024-02-17T11:59:34.978 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T11:59:34.978 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T11:59:35.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.128+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/1890220702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bec1080f0 msgr2=0x7f4bec114600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.128+0000 7f4bf0ad4640 1 --2- 172.21.15.94:0/1890220702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bec1080f0 0x7f4bec114600 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009ca0 tx=0x7f4be402c100 comp rx=0 tx=0).stop 2024-02-17T11:59:35.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.128+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/1890220702 shutdown_connections 2024-02-17T11:59:35.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.128+0000 7f4bf0ad4640 1 --2- 172.21.15.94:0/1890220702 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bec1080f0 0x7f4bec114600 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.128+0000 7f4bf0ad4640 1 --2- 172.21.15.94:0/1890220702 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bec107730 0x7f4bec107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.128+0000 7f4bf0ad4640 1 --2- 172.21.15.94:0/1890220702 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bec10d740 0x7f4bec10db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.128+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/1890220702 >> 172.21.15.94:0/1890220702 conn(0x7f4bec0756d0 msgr2=0x7f4bec077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.129+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/1890220702 shutdown_connections 2024-02-17T11:59:35.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.129+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/1890220702 wait complete. 2024-02-17T11:59:35.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.130+0000 7f4bf0ad4640 1 Processor -- start 2024-02-17T11:59:35.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.130+0000 7f4bf0ad4640 1 -- start start 2024-02-17T11:59:35.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bf0ad4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bec107730 0x7f4bec1a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bf0ad4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bec1080f0 0x7f4bec1a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bf0ad4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bec10d740 0x7f4bec19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bf0ad4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec19e080 con 0x7f4bec1080f0 2024-02-17T11:59:35.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bf0ad4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec19e1f0 con 0x7f4bec10d740 2024-02-17T11:59:35.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bf0ad4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec19e3b0 con 0x7f4bec107730 2024-02-17T11:59:35.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bea59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bec107730 0x7f4bec1a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4be9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bec1080f0 0x7f4bec1a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bea59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bec107730 0x7f4bec1a3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33128/0 (socket says 172.21.15.94:33128) 2024-02-17T11:59:35.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bea59c640 1 -- 172.21.15.94:0/92816919 learned_addr learned my addr 172.21.15.94:0/92816919 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:35.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bead9d640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bec10d740 0x7f4bec19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4be9d9b640 1 -- 172.21.15.94:0/92816919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bec107730 msgr2=0x7f4bec1a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4be9d9b640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bec107730 0x7f4bec1a3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4be9d9b640 1 -- 172.21.15.94:0/92816919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bec10d740 msgr2=0x7f4bec19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4be9d9b640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bec10d740 0x7f4bec19da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4be9d9b640 1 -- 172.21.15.94:0/92816919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be4009900 con 0x7f4bec1080f0 2024-02-17T11:59:35.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bea59c640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bec107730 0x7f4bec1a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:35.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bead9d640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bec10d740 0x7f4bec19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:35.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4be9d9b640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bec1080f0 0x7f4bec1a3d70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4be000e9f0 tx=0x7f4be000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bdb7fe640 1 -- 172.21.15.94:0/92816919 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be000ccc0 con 0x7f4bec1080f0 2024-02-17T11:59:35.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.131+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/92816919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec19e690 con 0x7f4bec1080f0 2024-02-17T11:59:35.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.132+0000 7f4bdb7fe640 1 -- 172.21.15.94:0/92816919 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4be000ce20 con 0x7f4bec1080f0 2024-02-17T11:59:35.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.132+0000 7f4bdb7fe640 1 -- 172.21.15.94:0/92816919 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0018920 con 0x7f4bec1080f0 2024-02-17T11:59:35.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.132+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/92816919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec1aa8d0 con 0x7f4bec1080f0 2024-02-17T11:59:35.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.133+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/92816919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bec0683c0 con 0x7f4bec1080f0 2024-02-17T11:59:35.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.134+0000 7f4bdb7fe640 1 -- 172.21.15.94:0/92816919 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f4be0018a80 con 0x7f4bec1080f0 2024-02-17T11:59:35.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.135+0000 7f4bdb7fe640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4bd40a2350 0x7f4bd40a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.135+0000 7f4bea59c640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4bd40a2350 0x7f4bd40a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.135+0000 7f4bdb7fe640 1 -- 172.21.15.94:0/92816919 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4be0014070 con 0x7f4bec1080f0 2024-02-17T11:59:35.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.136+0000 7f4bea59c640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4bd40a2350 0x7f4bd40a4810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc005fd0 tx=0x7f4bdc00b480 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.141+0000 7f4bdb7fe640 1 -- 172.21.15.94:0/92816919 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4be0052d20 con 0x7f4bec1080f0 2024-02-17T11:59:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 75.0 s, 3830.8 tps, lat 26.008 ms stddev 27.171 2024-02-17T11:59:35.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.462+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/92816919 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bec1178d0 con 0x7f4bec1080f0 2024-02-17T11:59:35.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.463+0000 7f4bdb7fe640 1 -- 172.21.15.94:0/92816919 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f4be00970e0 con 0x7f4bec1080f0 2024-02-17T11:59:35.462 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T11:59:35.463 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:57:51.854453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T11:59:35.463 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 21 2024-02-17T11:59:35.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.466+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/92816919 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4bd40a2350 msgr2=0x7f4bd40a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.466+0000 7f4bf0ad4640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4bd40a2350 0x7f4bd40a4810 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc005fd0 tx=0x7f4bdc00b480 comp rx=0 tx=0).stop 2024-02-17T11:59:35.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.466+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/92816919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bec1080f0 msgr2=0x7f4bec1a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.466+0000 7f4bf0ad4640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bec1080f0 0x7f4bec1a3d70 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4be000e9f0 tx=0x7f4be000eeb0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.466+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/92816919 shutdown_connections 2024-02-17T11:59:35.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.466+0000 7f4bf0ad4640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4bd40a2350 0x7f4bd40a4810 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.466+0000 7f4bf0ad4640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bec10d740 0x7f4bec19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.466+0000 7f4bf0ad4640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bec1080f0 0x7f4bec1a3d70 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.466+0000 7f4bf0ad4640 1 --2- 172.21.15.94:0/92816919 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bec107730 0x7f4bec1a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.466+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/92816919 >> 172.21.15.94:0/92816919 conn(0x7f4bec0756d0 msgr2=0x7f4bec075b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.466+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/92816919 shutdown_connections 2024-02-17T11:59:35.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.467+0000 7f4bf0ad4640 1 -- 172.21.15.94:0/92816919 wait complete. 2024-02-17T11:59:35.479 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T11:59:35.479 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T11:59:35.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.615+0000 7fd85df93640 1 -- 172.21.15.94:0/377058111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd85810b750 msgr2=0x7fd85810dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.615+0000 7fd85df93640 1 --2- 172.21.15.94:0/377058111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd85810b750 0x7fd85810dba0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd850009be0 tx=0x7fd85002bf80 comp rx=0 tx=0).stop 2024-02-17T11:59:35.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.615+0000 7fd85df93640 1 -- 172.21.15.94:0/377058111 shutdown_connections 2024-02-17T11:59:35.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.615+0000 7fd85df93640 1 --2- 172.21.15.94:0/377058111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd85810b750 0x7fd85810dba0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.615+0000 7fd85df93640 1 --2- 172.21.15.94:0/377058111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858102ca0 0x7fd85810b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.615+0000 7fd85df93640 1 --2- 172.21.15.94:0/377058111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd858102380 0x7fd858102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.615+0000 7fd85df93640 1 -- 172.21.15.94:0/377058111 >> 172.21.15.94:0/377058111 conn(0x7fd858076e30 msgr2=0x7fd8580fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.615+0000 7fd85df93640 1 -- 172.21.15.94:0/377058111 shutdown_connections 2024-02-17T11:59:35.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.615+0000 7fd85df93640 1 -- 172.21.15.94:0/377058111 wait complete. 2024-02-17T11:59:35.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.616+0000 7fd85df93640 1 Processor -- start 2024-02-17T11:59:35.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.616+0000 7fd85df93640 1 -- start start 2024-02-17T11:59:35.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.616+0000 7fd85df93640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd858102380 0x7fd8580fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.616+0000 7fd85df93640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858102ca0 0x7fd858100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd85df93640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd85810b750 0x7fd858100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd85df93640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8581a5ae0 con 0x7fd858102380 2024-02-17T11:59:35.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd85df93640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8581a5c50 con 0x7fd858102ca0 2024-02-17T11:59:35.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd85df93640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8581a5e10 con 0x7fd85810b750 2024-02-17T11:59:35.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd856ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858102ca0 0x7fd858100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd856ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858102ca0 0x7fd858100510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44460/0 (socket says 172.21.15.94:44460) 2024-02-17T11:59:35.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd856ffd640 1 -- 172.21.15.94:0/1487777275 learned_addr learned my addr 172.21.15.94:0/1487777275 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:35.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd857fff640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd85810b750 0x7fd858100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd8577fe640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd858102380 0x7fd8580fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd857fff640 1 -- 172.21.15.94:0/1487777275 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858102ca0 msgr2=0x7fd858100510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd857fff640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858102ca0 0x7fd858100510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd857fff640 1 -- 172.21.15.94:0/1487777275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd858102380 msgr2=0x7fd8580fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd857fff640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd858102380 0x7fd8580fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd857fff640 1 -- 172.21.15.94:0/1487777275 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd848009560 con 0x7fd85810b750 2024-02-17T11:59:35.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.617+0000 7fd856ffd640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858102ca0 0x7fd858100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:35.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.618+0000 7fd857fff640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd85810b750 0x7fd858100a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd850009b30 tx=0x7fd8500043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.618+0000 7fd854ff9640 1 -- 172.21.15.94:0/1487777275 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850004770 con 0x7fd85810b750 2024-02-17T11:59:35.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.618+0000 7fd854ff9640 1 -- 172.21.15.94:0/1487777275 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd850034910 con 0x7fd85810b750 2024-02-17T11:59:35.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.618+0000 7fd85df93640 1 -- 172.21.15.94:0/1487777275 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd850009840 con 0x7fd85810b750 2024-02-17T11:59:35.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.618+0000 7fd854ff9640 1 -- 172.21.15.94:0/1487777275 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85003e790 con 0x7fd85810b750 2024-02-17T11:59:35.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.618+0000 7fd85df93640 1 -- 172.21.15.94:0/1487777275 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8581a6460 con 0x7fd85810b750 2024-02-17T11:59:35.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.620+0000 7fd854ff9640 1 -- 172.21.15.94:0/1487777275 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fd850034a80 con 0x7fd85810b750 2024-02-17T11:59:35.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.621+0000 7fd860b6e740 1 Processor -- start 2024-02-17T11:59:35.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.621+0000 7fd860b6e740 1 -- start start 2024-02-17T11:59:35.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.621+0000 7fd860b6e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02aba60 0x564cf02abe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.621+0000 7fd860b6e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02ac380 0x564cf02b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.621+0000 7fd860b6e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02b4f60 0x564cf02b7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.621+0000 7fd860b6e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf025e960 con 0x564cf02ac380 2024-02-17T11:59:35.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.621+0000 7fd860b6e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf0212310 con 0x564cf02aba60 2024-02-17T11:59:35.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd860b6e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf020f840 con 0x564cf02b4f60 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd8577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02ac380 0x564cf02b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd856ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02aba60 0x564cf02abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd856ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02aba60 0x564cf02abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44466/0 (socket says 172.21.15.94:44466) 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd8577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02ac380 0x564cf02b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39680/0 (socket says 172.21.15.94:39680) 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd856ffd640 1 -- 172.21.15.94:0/2705576517 learned_addr learned my addr 172.21.15.94:0/2705576517 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd854ff9640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd8400a24a0 0x7fd8400a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd856ffd640 1 --2- 172.21.15.94:0/2705576517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02b4f60 0x564cf02b7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd857fff640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd8400a24a0 0x7fd8400a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd856ffd640 1 -- 172.21.15.94:0/2705576517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02b4f60 msgr2=0x564cf02b7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd856ffd640 1 --2- 172.21.15.94:0/2705576517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02b4f60 0x564cf02b7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd856ffd640 1 -- 172.21.15.94:0/2705576517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02ac380 msgr2=0x564cf02b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd856ffd640 1 --2- 172.21.15.94:0/2705576517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02ac380 0x564cf02b4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd856ffd640 1 -- 172.21.15.94:0/2705576517 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cf01cd060 con 0x564cf02aba60 2024-02-17T11:59:35.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd854ff9640 1 -- 172.21.15.94:0/1487777275 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd8500f0db0 con 0x7fd85810b750 2024-02-17T11:59:35.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.622+0000 7fd856ffd640 1 --2- 172.21.15.94:0/2705576517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02aba60 0x564cf02abe40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd84c012bd0 tx=0x7fd84c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b593efe0b086417c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.623+0000 7fd857fff640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd8400a24a0 0x7fd8400a4960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd8500ee480 tx=0x7fd85009e040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.625+0000 7fd83dffb640 1 -- 172.21.15.94:0/2705576517 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c012050 con 0x564cf02aba60 2024-02-17T11:59:35.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.625+0000 7fd83dffb640 1 -- 172.21.15.94:0/2705576517 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd84c004510 con 0x564cf02aba60 2024-02-17T11:59:35.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.625+0000 7fd83dffb640 1 -- 172.21.15.94:0/2705576517 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c003850 con 0x564cf02aba60 2024-02-17T11:59:35.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.627+0000 7fd860b6e740 1 -- 172.21.15.94:0/2705576517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02aba60 msgr2=0x564cf02abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.627+0000 7fd860b6e740 1 --2- 172.21.15.94:0/2705576517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02aba60 0x564cf02abe40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd84c012bd0 tx=0x7fd84c004730 comp rx=0 tx=0).stop 2024-02-17T11:59:35.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.627+0000 7fd860b6e740 1 -- 172.21.15.94:0/2705576517 shutdown_connections 2024-02-17T11:59:35.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.627+0000 7fd860b6e740 1 --2- 172.21.15.94:0/2705576517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02b4f60 0x564cf02b7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.627+0000 7fd860b6e740 1 --2- 172.21.15.94:0/2705576517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02ac380 0x564cf02b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.627+0000 7fd860b6e740 1 --2- 172.21.15.94:0/2705576517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02aba60 0x564cf02abe40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.627+0000 7fd860b6e740 1 -- 172.21.15.94:0/2705576517 >> 172.21.15.94:0/2705576517 conn(0x564cf02a6190 msgr2=0x564cf02a6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.627+0000 7fd860b6e740 1 -- 172.21.15.94:0/2705576517 shutdown_connections 2024-02-17T11:59:35.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.627+0000 7fd860b6e740 1 -- 172.21.15.94:0/2705576517 wait complete. 2024-02-17T11:59:35.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.628+0000 7fd860b6e740 1 -- start start 2024-02-17T11:59:35.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.628+0000 7fd860b6e740 1 Processor -- start 2024-02-17T11:59:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.628+0000 7fd860b6e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02aba60 0x564cf02b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.629+0000 7fd860b6e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 0x564cf02b2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.629+0000 7fd860b6e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02b4f60 0x564cf02b2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.629+0000 7fd860b6e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf020b020 con 0x564cf02b4f60 2024-02-17T11:59:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.629+0000 7fd860b6e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cefede430 con 0x564cf02aba60 2024-02-17T11:59:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.629+0000 7fd860b6e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cefe9ae00 con 0x564cf02ac380 2024-02-17T11:59:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.629+0000 7fd8577fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 0x564cf02b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.629+0000 7fd8577fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 0x564cf02b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33166/0 (socket says 172.21.15.94:33166) 2024-02-17T11:59:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.629+0000 7fd8577fe640 1 -- 172.21.15.94:0/1240859727 learned_addr learned my addr 172.21.15.94:0/1240859727 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:35.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.629+0000 7fd860b6e740 10 client.0 _ll_register_callbacks cb 0x564cf01b2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:35.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.631+0000 7fd8577fe640 1 -- 172.21.15.94:0/1240859727 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02aba60 msgr2=0x564cf02b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.631+0000 7fd8577fe640 1 --2- 172.21.15.94:0/1240859727 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02aba60 0x564cf02b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.631+0000 7fd8577fe640 1 -- 172.21.15.94:0/1240859727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02b4f60 msgr2=0x564cf02b2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.631+0000 7fd8577fe640 1 --2- 172.21.15.94:0/1240859727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02b4f60 0x564cf02b2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.631+0000 7fd8577fe640 1 -- 172.21.15.94:0/1240859727 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84c01a040 con 0x564cf02ac380 2024-02-17T11:59:35.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.631+0000 7fd8577fe640 1 --2- 172.21.15.94:0/1240859727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 0x564cf02b2600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd848012980 tx=0x7fd848012e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2209f2ecee312c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.632+0000 7fd836ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T11:59:35.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.633+0000 7fd836ffd640 1 -- 172.21.15.94:0/1240859727 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848017070 con 0x564cf02ac380 2024-02-17T11:59:35.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.634+0000 7fd860b6e740 1 -- 172.21.15.94:0/1240859727 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ceff0a7e0 con 0x564cf02ac380 2024-02-17T11:59:35.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.634+0000 7fd836ffd640 1 -- 172.21.15.94:0/1240859727 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd848004020 con 0x564cf02ac380 2024-02-17T11:59:35.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.634+0000 7fd836ffd640 1 -- 172.21.15.94:0/1240859727 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8480122e0 con 0x564cf02ac380 2024-02-17T11:59:35.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.634+0000 7fd860b6e740 10 client.25396 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:35.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.634+0000 7fd860b6e740 1 -- 172.21.15.94:0/1240859727 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x564cf02096a0 con 0x564cf02ac380 2024-02-17T11:59:35.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.635+0000 7fd836ffd640 1 -- 172.21.15.94:0/1240859727 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564ceff0a7e0 con 0x564cf02ac380 2024-02-17T11:59:35.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.635+0000 7fd836ffd640 1 -- 172.21.15.94:0/1240859727 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd848003a90 con 0x564cf02ac380 2024-02-17T11:59:35.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.635+0000 7fd860b6e740 10 client.25396 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:35.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.635+0000 7fd860b6e740 10 client.25396 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T11:59:35.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.635+0000 7fd860b6e740 20 client.25396 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:35.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.635+0000 7fd860b6e740 1 --2- 172.21.15.94:0/1240859727 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564cf02c53b0 0x564cf02c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.635+0000 7fd860b6e740 4 client.25396 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:35.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.635+0000 7fd860b6e740 1 -- 172.21.15.94:0/1240859727 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ceff47720 con 0x564cf02c53b0 2024-02-17T11:59:35.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.636+0000 7fd856ffd640 1 --2- 172.21.15.94:0/1240859727 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564cf02c53b0 0x564cf02c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.638+0000 7fd856ffd640 1 --2- 172.21.15.94:0/1240859727 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564cf02c53b0 0x564cf02c77d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b57ab7b7c17f0a88 server_cookie=36707f06cd86082a in_seq=0 out_seq=0 2024-02-17T11:59:35.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.638+0000 7fd836ffd640 10 client.25396 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T11:59:35.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.639+0000 7fd836ffd640 1 -- 172.21.15.94:0/1240859727 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564ceff47720 con 0x564cf02c53b0 2024-02-17T11:59:35.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.639+0000 7fd836ffd640 10 client.25396 handle_command_reply: tid=0 2024-02-17T11:59:35.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.639+0000 7fd860b6e740 1 client.25396 shutdown 2024-02-17T11:59:35.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.639+0000 7fd860b6e740 1 -- 172.21.15.94:0/1240859727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 msgr2=0x564cf02b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.639+0000 7fd860b6e740 1 --2- 172.21.15.94:0/1240859727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 0x564cf02b2600 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd848012980 tx=0x7fd848012e40 comp rx=0 tx=0).stop 2024-02-17T11:59:35.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.639+0000 7fd860b6e740 1 -- 172.21.15.94:0/1240859727 shutdown_connections 2024-02-17T11:59:35.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.639+0000 7fd860b6e740 1 --2- 172.21.15.94:0/1240859727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02b4f60 0x564cf02b2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.639+0000 7fd860b6e740 1 --2- 172.21.15.94:0/1240859727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 0x564cf02b2600 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.639+0000 7fd860b6e740 1 --2- 172.21.15.94:0/1240859727 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564cf02c53b0 0x564cf02c77d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.639+0000 7fd860b6e740 1 --2- 172.21.15.94:0/1240859727 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02aba60 0x564cf02b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.639+0000 7fd836ffd640 0 client.25396 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T11:59:35.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.639+0000 7fd860b6e740 1 -- 172.21.15.94:0/1240859727 >> 172.21.15.94:0/1240859727 conn(0x564cf02a6190 msgr2=0x564cf02a7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.640+0000 7fd860b6e740 1 -- 172.21.15.94:0/1240859727 shutdown_connections 2024-02-17T11:59:35.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.640+0000 7fd860b6e740 1 -- 172.21.15.94:0/1240859727 wait complete. 2024-02-17T11:59:35.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.640+0000 7fd860b6e740 20 client.25396 trim_cache size 0 max 16384 2024-02-17T11:59:35.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.657+0000 7fd860b6e740 1 Processor -- start 2024-02-17T11:59:35.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.657+0000 7fd860b6e740 1 -- start start 2024-02-17T11:59:35.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.657+0000 7fd860b6e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02aba60 0x564cf02b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.657+0000 7fd860b6e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02ac380 0x564cf02bbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd860b6e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02b4f60 0x564cf02bc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd860b6e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf0190130 con 0x564cf02aba60 2024-02-17T11:59:35.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd860b6e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ceff0a7e0 con 0x564cf02ac380 2024-02-17T11:59:35.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd860b6e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf028fa40 con 0x564cf02b4f60 2024-02-17T11:59:35.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd856ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02aba60 0x564cf02b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd856ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02aba60 0x564cf02b47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39688/0 (socket says 172.21.15.94:39688) 2024-02-17T11:59:35.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd856ffd640 1 -- 172.21.15.94:0/2138478783 learned_addr learned my addr 172.21.15.94:0/2138478783 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:35.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd8577fe640 1 --2- 172.21.15.94:0/2138478783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02ac380 0x564cf02bbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd856ffd640 1 --2- 172.21.15.94:0/2138478783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02b4f60 0x564cf02bc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd856ffd640 1 -- 172.21.15.94:0/2138478783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02b4f60 msgr2=0x564cf02bc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd856ffd640 1 --2- 172.21.15.94:0/2138478783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02b4f60 0x564cf02bc350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd856ffd640 1 -- 172.21.15.94:0/2138478783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02ac380 msgr2=0x564cf02bbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd856ffd640 1 --2- 172.21.15.94:0/2138478783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02ac380 0x564cf02bbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd856ffd640 1 -- 172.21.15.94:0/2138478783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564cf01a5c20 con 0x564cf02aba60 2024-02-17T11:59:35.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd8577fe640 1 --2- 172.21.15.94:0/2138478783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02ac380 0x564cf02bbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:35.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.658+0000 7fd856ffd640 1 --2- 172.21.15.94:0/2138478783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02aba60 0x564cf02b47f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd84c012b70 tx=0x7fd84c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9194d79d3f5e4ce7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.659+0000 7fd83d7fa640 1 -- 172.21.15.94:0/2138478783 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c024040 con 0x564cf02aba60 2024-02-17T11:59:35.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.659+0000 7fd83d7fa640 1 -- 172.21.15.94:0/2138478783 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd84c032020 con 0x564cf02aba60 2024-02-17T11:59:35.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.659+0000 7fd83d7fa640 1 -- 172.21.15.94:0/2138478783 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c01f6c0 con 0x564cf02aba60 2024-02-17T11:59:35.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.660+0000 7fd860b6e740 1 -- 172.21.15.94:0/2138478783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02aba60 msgr2=0x564cf02b47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.660+0000 7fd860b6e740 1 --2- 172.21.15.94:0/2138478783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02aba60 0x564cf02b47f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd84c012b70 tx=0x7fd84c00bf90 comp rx=0 tx=0).stop 2024-02-17T11:59:35.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.661+0000 7fd860b6e740 1 -- 172.21.15.94:0/2138478783 shutdown_connections 2024-02-17T11:59:35.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.661+0000 7fd860b6e740 1 --2- 172.21.15.94:0/2138478783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02b4f60 0x564cf02bc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.661+0000 7fd860b6e740 1 --2- 172.21.15.94:0/2138478783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02ac380 0x564cf02bbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.661+0000 7fd860b6e740 1 --2- 172.21.15.94:0/2138478783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02aba60 0x564cf02b47f0 secure :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd84c012b70 tx=0x7fd84c00bf90 comp rx=0 tx=0).stop 2024-02-17T11:59:35.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.661+0000 7fd860b6e740 1 -- 172.21.15.94:0/2138478783 >> 172.21.15.94:0/2138478783 conn(0x564cf02a6190 msgr2=0x564cf02ad380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.661+0000 7fd860b6e740 1 -- 172.21.15.94:0/2138478783 shutdown_connections 2024-02-17T11:59:35.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.661+0000 7fd860b6e740 1 -- 172.21.15.94:0/2138478783 wait complete. 2024-02-17T11:59:35.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.661+0000 7fd860b6e740 1 -- start start 2024-02-17T11:59:35.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.662+0000 7fd860b6e740 1 Processor -- start 2024-02-17T11:59:35.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.662+0000 7fd860b6e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 0x564cf02ba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.662+0000 7fd860b6e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02b4f60 0x564cf02bae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.662+0000 7fd860b6e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02bb370 0x564cf031b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.662+0000 7fd860b6e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf02bc750 con 0x564cf02b4f60 2024-02-17T11:59:35.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.662+0000 7fd860b6e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf031be30 con 0x564cf02bb370 2024-02-17T11:59:35.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.662+0000 7fd860b6e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564cf031bff0 con 0x564cf02ac380 2024-02-17T11:59:35.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.662+0000 7fd860b6e740 10 client.0 _ll_register_callbacks cb 0x7fd84c00bee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:35.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.662+0000 7fd856ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 0x564cf02ba8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.662+0000 7fd856ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 0x564cf02ba8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33190/0 (socket says 172.21.15.94:33190) 2024-02-17T11:59:35.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.663+0000 7fd856ffd640 1 -- 172.21.15.94:0/4061234518 learned_addr learned my addr 172.21.15.94:0/4061234518 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:35.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.663+0000 7fd8577fe640 1 --2- 172.21.15.94:0/4061234518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02b4f60 0x564cf02bae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.663+0000 7fd856ffd640 1 --2- 172.21.15.94:0/4061234518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02bb370 0x564cf031b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.663+0000 7fd856ffd640 1 -- 172.21.15.94:0/4061234518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02bb370 msgr2=0x564cf031b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.663+0000 7fd856ffd640 1 --2- 172.21.15.94:0/4061234518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02bb370 0x564cf031b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.663+0000 7fd856ffd640 1 -- 172.21.15.94:0/4061234518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02b4f60 msgr2=0x564cf02bae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.663+0000 7fd856ffd640 1 --2- 172.21.15.94:0/4061234518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02b4f60 0x564cf02bae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.663+0000 7fd856ffd640 1 -- 172.21.15.94:0/4061234518 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564cf02096a0 con 0x564cf02ac380 2024-02-17T11:59:35.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd856ffd640 1 --2- 172.21.15.94:0/4061234518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 0x564cf02ba8f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd84c021040 tx=0x7fd84c012570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=144295556daa41a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd836ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd836ffd640 1 -- 172.21.15.94:0/4061234518 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c024050 con 0x564cf02ac380 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd836ffd640 1 -- 172.21.15.94:0/4061234518 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd84c03f060 con 0x564cf02ac380 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd860b6e740 1 -- 172.21.15.94:0/4061234518 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd84c004510 con 0x564cf02ac380 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd836ffd640 1 -- 172.21.15.94:0/4061234518 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd84c012800 con 0x564cf02ac380 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd860b6e740 10 client.25402 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd860b6e740 1 -- 172.21.15.94:0/4061234518 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x564cf01cd060 con 0x564cf02ac380 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd836ffd640 1 -- 172.21.15.94:0/4061234518 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd84c047020 con 0x564cf02ac380 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd836ffd640 1 -- 172.21.15.94:0/4061234518 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd84c029070 con 0x564cf02ac380 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd860b6e740 10 client.25402 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd860b6e740 10 client.25402 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd860b6e740 20 client.25402 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd860b6e740 1 --2- 172.21.15.94:0/4061234518 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564cf031d490 0x564cf031f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.664+0000 7fd860b6e740 4 client.25402 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T11:59:35.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd860b6e740 1 -- 172.21.15.94:0/4061234518 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fd84c004510 con 0x564cf031d490 2024-02-17T11:59:35.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd8577fe640 1 --2- 172.21.15.94:0/4061234518 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564cf031d490 0x564cf031f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd836ffd640 10 client.25402 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T11:59:35.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd8577fe640 1 --2- 172.21.15.94:0/4061234518 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564cf031d490 0x564cf031f880 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0d23ff0b382562d server_cookie=dd7822a27e769e22 in_seq=0 out_seq=0 2024-02-17T11:59:35.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd836ffd640 1 -- 172.21.15.94:0/4061234518 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fd84c004510 con 0x564cf031d490 2024-02-17T11:59:35.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd836ffd640 10 client.25402 handle_command_reply: tid=0 2024-02-17T11:59:35.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd860b6e740 1 client.25402 shutdown 2024-02-17T11:59:35.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd860b6e740 1 -- 172.21.15.94:0/4061234518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 msgr2=0x564cf02ba8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd860b6e740 1 --2- 172.21.15.94:0/4061234518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 0x564cf02ba8f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd84c021040 tx=0x7fd84c012570 comp rx=0 tx=0).stop 2024-02-17T11:59:35.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd860b6e740 1 -- 172.21.15.94:0/4061234518 shutdown_connections 2024-02-17T11:59:35.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd860b6e740 1 --2- 172.21.15.94:0/4061234518 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564cf031d490 0x564cf031f880 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd860b6e740 1 --2- 172.21.15.94:0/4061234518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564cf02bb370 0x564cf031b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd860b6e740 1 --2- 172.21.15.94:0/4061234518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564cf02b4f60 0x564cf02bae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.665+0000 7fd860b6e740 1 --2- 172.21.15.94:0/4061234518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564cf02ac380 0x564cf02ba8f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.666+0000 7fd860b6e740 1 -- 172.21.15.94:0/4061234518 >> 172.21.15.94:0/4061234518 conn(0x564cf02a6190 msgr2=0x564cf02a75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.666+0000 7fd836ffd640 0 client.25402 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T11:59:35.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.666+0000 7fd860b6e740 1 -- 172.21.15.94:0/4061234518 shutdown_connections 2024-02-17T11:59:35.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.666+0000 7fd860b6e740 1 -- 172.21.15.94:0/4061234518 wait complete. 2024-02-17T11:59:35.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.666+0000 7fd860b6e740 20 client.25402 trim_cache size 0 max 16384 2024-02-17T11:59:35.668 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T11:59:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.666+0000 7fd85df93640 1 -- 172.21.15.94:0/1487777275 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd8400a24a0 msgr2=0x7fd8400a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.666+0000 7fd85df93640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd8400a24a0 0x7fd8400a4960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd8500ee480 tx=0x7fd85009e040 comp rx=0 tx=0).stop 2024-02-17T11:59:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.666+0000 7fd85df93640 1 -- 172.21.15.94:0/1487777275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd85810b750 msgr2=0x7fd858100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.666+0000 7fd85df93640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd85810b750 0x7fd858100a50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd850009b30 tx=0x7fd8500043b0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.667+0000 7fd85df93640 1 -- 172.21.15.94:0/1487777275 shutdown_connections 2024-02-17T11:59:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.667+0000 7fd85df93640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd8400a24a0 0x7fd8400a4960 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.667+0000 7fd85df93640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd85810b750 0x7fd858100a50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.667+0000 7fd85df93640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd858102ca0 0x7fd858100510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.667+0000 7fd85df93640 1 --2- 172.21.15.94:0/1487777275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd858102380 0x7fd8580fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.667+0000 7fd85df93640 1 -- 172.21.15.94:0/1487777275 >> 172.21.15.94:0/1487777275 conn(0x7fd858076e30 msgr2=0x7fd8580fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.667+0000 7fd85df93640 1 -- 172.21.15.94:0/1487777275 shutdown_connections 2024-02-17T11:59:35.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.667+0000 7fd85df93640 1 -- 172.21.15.94:0/1487777275 wait complete. 2024-02-17T11:59:35.679 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T11:59:35.679 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T11:59:35.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.817+0000 7fcaf50ae640 1 -- 172.21.15.94:0/606101109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf0101d30 msgr2=0x7fcaf0105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.817+0000 7fcaf50ae640 1 --2- 172.21.15.94:0/606101109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf0101d30 0x7fcaf0105d70 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009930 tx=0x7fcad802bc30 comp rx=0 tx=0).stop 2024-02-17T11:59:35.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.817+0000 7fcaf50ae640 1 -- 172.21.15.94:0/606101109 shutdown_connections 2024-02-17T11:59:35.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.817+0000 7fcaf50ae640 1 --2- 172.21.15.94:0/606101109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaf0106450 0x7fcaf010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.817+0000 7fcaf50ae640 1 --2- 172.21.15.94:0/606101109 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf0101d30 0x7fcaf0105d70 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.817+0000 7fcaf50ae640 1 --2- 172.21.15.94:0/606101109 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaf0101380 0x7fcaf0101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.817+0000 7fcaf50ae640 1 -- 172.21.15.94:0/606101109 >> 172.21.15.94:0/606101109 conn(0x7fcaf00fcc20 msgr2=0x7fcaf00ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.817+0000 7fcaf50ae640 1 -- 172.21.15.94:0/606101109 shutdown_connections 2024-02-17T11:59:35.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.817+0000 7fcaf50ae640 1 -- 172.21.15.94:0/606101109 wait complete. 2024-02-17T11:59:35.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.818+0000 7fcaf50ae640 1 Processor -- start 2024-02-17T11:59:35.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.818+0000 7fcaf50ae640 1 -- start start 2024-02-17T11:59:35.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.818+0000 7fcaf50ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf0101380 0x7fcaf019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.818+0000 7fcaf50ae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaf0106450 0x7fcaf019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.818+0000 7fcaf50ae640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaf0197690 0x7fcaf0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.818+0000 7fcaf50ae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0198160 con 0x7fcaf0101380 2024-02-17T11:59:35.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.818+0000 7fcaeed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf0101380 0x7fcaf019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.818+0000 7fcaeed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf0101380 0x7fcaf019d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39722/0 (socket says 172.21.15.94:39722) 2024-02-17T11:59:35.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.818+0000 7fcaeed9d640 1 -- 172.21.15.94:0/3134198058 learned_addr learned my addr 172.21.15.94:0/3134198058 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:35.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.818+0000 7fcae7fff640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaf0106450 0x7fcaf019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.818+0000 7fcaf50ae640 1 -- 172.21.15.94:0/3134198058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf01982d0 con 0x7fcaf0197690 2024-02-17T11:59:35.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.818+0000 7fcaf50ae640 1 -- 172.21.15.94:0/3134198058 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf0198490 con 0x7fcaf0106450 2024-02-17T11:59:35.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.818+0000 7fcaef59e640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaf0197690 0x7fcaf0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.819+0000 7fcaef59e640 1 -- 172.21.15.94:0/3134198058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaf0106450 msgr2=0x7fcaf019dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.819+0000 7fcaef59e640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaf0106450 0x7fcaf019dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.819+0000 7fcaef59e640 1 -- 172.21.15.94:0/3134198058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf0101380 msgr2=0x7fcaf019d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.819+0000 7fcaef59e640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf0101380 0x7fcaf019d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.819+0000 7fcaef59e640 1 -- 172.21.15.94:0/3134198058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad8009560 con 0x7fcaf0197690 2024-02-17T11:59:35.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.819+0000 7fcae7fff640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaf0106450 0x7fcaf019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:35.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.819+0000 7fcaef59e640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaf0197690 0x7fcaf0197b10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009fd0 tx=0x7fcae800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.819+0000 7fcaecd99640 1 -- 172.21.15.94:0/3134198058 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae800ce20 con 0x7fcaf0197690 2024-02-17T11:59:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.819+0000 7fcaf50ae640 1 -- 172.21.15.94:0/3134198058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf0073470 con 0x7fcaf0197690 2024-02-17T11:59:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.820+0000 7fcaf50ae640 1 -- 172.21.15.94:0/3134198058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf00739a0 con 0x7fcaf0197690 2024-02-17T11:59:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.820+0000 7fcaecd99640 1 -- 172.21.15.94:0/3134198058 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcae8005020 con 0x7fcaf0197690 2024-02-17T11:59:35.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.820+0000 7fcaecd99640 1 -- 172.21.15.94:0/3134198058 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8010690 con 0x7fcaf0197690 2024-02-17T11:59:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.822+0000 7fcaecd99640 1 -- 172.21.15.94:0/3134198058 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7fcae8004a40 con 0x7fcaf0197690 2024-02-17T11:59:35.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.822+0000 7fcaf7c80740 1 Processor -- start 2024-02-17T11:59:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.822+0000 7fcaf7c80740 1 -- start start 2024-02-17T11:59:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.822+0000 7fcaecd99640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcadc0aac60 0x7fcadc0ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaecd99640 1 -- 172.21.15.94:0/3134198058 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcae8014070 con 0x7fcaf0197690 2024-02-17T11:59:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaf7c80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5250 0x5645406d5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaf7c80740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5b70 0x5645406de210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaf7c80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406de750 0x5645406e0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaf7c80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564540690960 con 0x5645406de750 2024-02-17T11:59:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaf7c80740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564540644310 con 0x5645406d5b70 2024-02-17T11:59:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaf7c80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564540641840 con 0x5645406d5250 2024-02-17T11:59:35.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcae7fff640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcadc0aac60 0x7fcadc0ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaeed9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5250 0x5645406d5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaef59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406de750 0x5645406e0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaeed9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5250 0x5645406d5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33212/0 (socket says 172.21.15.94:33212) 2024-02-17T11:59:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaeed9d640 1 -- 172.21.15.94:0/3876967174 learned_addr learned my addr 172.21.15.94:0/3876967174 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcae7fff640 1 --2- 172.21.15.94:0/3876967174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5b70 0x5645406de210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaeed9d640 1 -- 172.21.15.94:0/3876967174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5b70 msgr2=0x5645406de210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaeed9d640 1 --2- 172.21.15.94:0/3876967174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5b70 0x5645406de210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaeed9d640 1 -- 172.21.15.94:0/3876967174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406de750 msgr2=0x5645406e0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaeed9d640 1 --2- 172.21.15.94:0/3876967174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406de750 0x5645406e0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaeed9d640 1 -- 172.21.15.94:0/3876967174 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645405ff060 con 0x5645406d5250 2024-02-17T11:59:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaef59e640 1 --2- 172.21.15.94:0/3876967174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406de750 0x5645406e0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcae7fff640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcadc0aac60 0x7fcadc0ad120 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcad8000c00 tx=0x7fcad802c9c0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.823+0000 7fcaeed9d640 1 --2- 172.21.15.94:0/3876967174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5250 0x5645406d5630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009dc0 tx=0x7fcae0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d4aa9924c5cb433 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.824+0000 7fcae57fa640 1 -- 172.21.15.94:0/3876967174 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0005220 con 0x5645406d5250 2024-02-17T11:59:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.824+0000 7fcae57fa640 1 -- 172.21.15.94:0/3876967174 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcae0004c80 con 0x5645406d5250 2024-02-17T11:59:35.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.824+0000 7fcae57fa640 1 -- 172.21.15.94:0/3876967174 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0018870 con 0x5645406d5250 2024-02-17T11:59:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.825+0000 7fcaf7c80740 1 -- 172.21.15.94:0/3876967174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5250 msgr2=0x5645406d5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.825+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/3876967174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5250 0x5645406d5630 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009dc0 tx=0x7fcae0011810 comp rx=0 tx=0).stop 2024-02-17T11:59:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.825+0000 7fcaf7c80740 1 -- 172.21.15.94:0/3876967174 shutdown_connections 2024-02-17T11:59:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.825+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/3876967174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406de750 0x5645406e0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.825+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/3876967174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5b70 0x5645406de210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.825+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/3876967174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5250 0x5645406d5630 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.825+0000 7fcaf7c80740 1 -- 172.21.15.94:0/3876967174 >> 172.21.15.94:0/3876967174 conn(0x5645406cf980 msgr2=0x5645406cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.825+0000 7fcaf7c80740 1 -- 172.21.15.94:0/3876967174 shutdown_connections 2024-02-17T11:59:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.825+0000 7fcaf7c80740 1 -- 172.21.15.94:0/3876967174 wait complete. 2024-02-17T11:59:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcaf7c80740 1 -- start start 2024-02-17T11:59:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcaf7c80740 1 Processor -- start 2024-02-17T11:59:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcaf7c80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5250 0x5645406db610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcaf7c80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5b70 0x5645406dbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcaf7c80740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406de750 0x5645406dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcaf7c80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56454063d020 con 0x5645406d5250 2024-02-17T11:59:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcaf7c80740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564540310430 con 0x5645406de750 2024-02-17T11:59:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcaf7c80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645402cce00 con 0x5645406d5b70 2024-02-17T11:59:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcaf7c80740 10 client.0 _ll_register_callbacks cb 0x5645405e4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:35.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcaeed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5250 0x5645406db610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcae7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5b70 0x5645406dbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcae7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5b70 0x5645406dbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33216/0 (socket says 172.21.15.94:33216) 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcae7fff640 1 -- 172.21.15.94:0/860354809 learned_addr learned my addr 172.21.15.94:0/860354809 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.826+0000 7fcaef59e640 1 --2- 172.21.15.94:0/860354809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406de750 0x5645406dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.827+0000 7fcae7fff640 1 -- 172.21.15.94:0/860354809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406de750 msgr2=0x5645406dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.827+0000 7fcae7fff640 1 --2- 172.21.15.94:0/860354809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406de750 0x5645406dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.827+0000 7fcae7fff640 1 -- 172.21.15.94:0/860354809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5250 msgr2=0x5645406db610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.827+0000 7fcae7fff640 1 --2- 172.21.15.94:0/860354809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5250 0x5645406db610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.827+0000 7fcae7fff640 1 -- 172.21.15.94:0/860354809 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae00099f0 con 0x5645406d5b70 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.827+0000 7fcaef59e640 1 --2- 172.21.15.94:0/860354809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406de750 0x5645406dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.827+0000 7fcae7fff640 1 --2- 172.21.15.94:0/860354809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5b70 0x5645406dbb50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcad8035000 tx=0x7fcad802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92512b0b547fb574 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.827+0000 7fcad67fc640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.827+0000 7fcad67fc640 1 -- 172.21.15.94:0/860354809 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8005370 con 0x5645406d5b70 2024-02-17T11:59:35.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.827+0000 7fcad67fc640 1 -- 172.21.15.94:0/860354809 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcad80054d0 con 0x5645406d5b70 2024-02-17T11:59:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.827+0000 7fcaf7c80740 1 -- 172.21.15.94:0/860354809 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56454033c7e0 con 0x5645406d5b70 2024-02-17T11:59:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.828+0000 7fcad67fc640 1 -- 172.21.15.94:0/860354809 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcad8005690 con 0x5645406d5b70 2024-02-17T11:59:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.828+0000 7fcaf7c80740 10 client.25414 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.828+0000 7fcaf7c80740 1 -- 172.21.15.94:0/860354809 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x56454063b6a0 con 0x5645406d5b70 2024-02-17T11:59:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.828+0000 7fcad67fc640 1 -- 172.21.15.94:0/860354809 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56454033c7e0 con 0x5645406d5b70 2024-02-17T11:59:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.828+0000 7fcad67fc640 1 -- 172.21.15.94:0/860354809 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcad8042670 con 0x5645406d5b70 2024-02-17T11:59:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.828+0000 7fcaf7c80740 10 client.25414 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.828+0000 7fcaf7c80740 10 client.25414 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T11:59:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.828+0000 7fcaf7c80740 20 client.25414 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.828+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/860354809 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5645406eea90 0x5645406f0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.828+0000 7fcaf7c80740 4 client.25414 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.828+0000 7fcaf7c80740 1 -- 172.21.15.94:0/860354809 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564540379720 con 0x5645406eea90 2024-02-17T11:59:35.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.829+0000 7fcaeed9d640 1 --2- 172.21.15.94:0/860354809 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5645406eea90 0x5645406f0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.829+0000 7fcaeed9d640 1 --2- 172.21.15.94:0/860354809 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5645406eea90 0x5645406f0eb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94298dca1793bde9 server_cookie=3507b943c80a135a in_seq=0 out_seq=0 2024-02-17T11:59:35.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.829+0000 7fcad67fc640 10 client.25414 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T11:59:35.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.830+0000 7fcad67fc640 1 -- 172.21.15.94:0/860354809 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564540379720 con 0x5645406eea90 2024-02-17T11:59:35.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.831+0000 7fcad67fc640 10 client.25414 handle_command_reply: tid=0 2024-02-17T11:59:35.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.831+0000 7fcaf7c80740 1 client.25414 shutdown 2024-02-17T11:59:35.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.832+0000 7fcaf7c80740 1 -- 172.21.15.94:0/860354809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5b70 msgr2=0x5645406dbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.832+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/860354809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5b70 0x5645406dbb50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcad8035000 tx=0x7fcad802bc30 comp rx=0 tx=0).stop 2024-02-17T11:59:35.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.832+0000 7fcaf7c80740 1 -- 172.21.15.94:0/860354809 shutdown_connections 2024-02-17T11:59:35.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.832+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/860354809 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5645406eea90 0x5645406f0eb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.832+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/860354809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406de750 0x5645406dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.832+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/860354809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406d5b70 0x5645406dbb50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.832+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/860354809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5250 0x5645406db610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.832+0000 7fcaf7c80740 1 -- 172.21.15.94:0/860354809 >> 172.21.15.94:0/860354809 conn(0x5645406cf980 msgr2=0x5645406d16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.832+0000 7fcad67fc640 0 client.25414 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T11:59:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.833+0000 7fcaf7c80740 1 -- 172.21.15.94:0/860354809 shutdown_connections 2024-02-17T11:59:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.833+0000 7fcaf7c80740 1 -- 172.21.15.94:0/860354809 wait complete. 2024-02-17T11:59:35.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.833+0000 7fcaf7c80740 20 client.25414 trim_cache size 0 max 16384 2024-02-17T11:59:35.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.859+0000 7fcaf7c80740 1 Processor -- start 2024-02-17T11:59:35.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.864+0000 7fcaf7c80740 1 -- start start 2024-02-17T11:59:35.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.865+0000 7fcaf7c80740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5250 0x5645406ddd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.865+0000 7fcaf7c80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5b70 0x5645406dbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.865+0000 7fcaf7c80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406de750 0x5645406d8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.865+0000 7fcaf7c80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645405c2130 con 0x5645406d5b70 2024-02-17T11:59:35.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.865+0000 7fcaf7c80740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645406d4ef0 con 0x5645406d5250 2024-02-17T11:59:35.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.865+0000 7fcaf7c80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645406cb610 con 0x5645406de750 2024-02-17T11:59:35.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.865+0000 7fcaef59e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406de750 0x5645406d8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.865+0000 7fcaef59e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406de750 0x5645406d8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33230/0 (socket says 172.21.15.94:33230) 2024-02-17T11:59:35.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.865+0000 7fcaef59e640 1 -- 172.21.15.94:0/1838339152 learned_addr learned my addr 172.21.15.94:0/1838339152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:35.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.866+0000 7fcaeed9d640 1 --2- 172.21.15.94:0/1838339152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5250 0x5645406ddd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.866+0000 7fcaef59e640 1 -- 172.21.15.94:0/1838339152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5250 msgr2=0x5645406ddd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.866+0000 7fcaef59e640 1 --2- 172.21.15.94:0/1838339152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5250 0x5645406ddd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.866+0000 7fcaef59e640 1 -- 172.21.15.94:0/1838339152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5b70 msgr2=0x5645406dbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.866+0000 7fcaef59e640 1 --2- 172.21.15.94:0/1838339152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5b70 0x5645406dbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.866+0000 7fcaef59e640 1 -- 172.21.15.94:0/1838339152 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645405d7c20 con 0x5645406de750 2024-02-17T11:59:35.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.866+0000 7fcaeed9d640 1 --2- 172.21.15.94:0/1838339152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5250 0x5645406ddd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:59:35.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.866+0000 7fcaef59e640 1 --2- 172.21.15.94:0/1838339152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406de750 0x5645406d8870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcae8017000 tx=0x7fcae809b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a260322c29996a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.868+0000 7fcae4ff9640 1 -- 172.21.15.94:0/1838339152 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae80a3790 con 0x5645406de750 2024-02-17T11:59:35.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.868+0000 7fcae4ff9640 1 -- 172.21.15.94:0/1838339152 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcae8004cf0 con 0x5645406de750 2024-02-17T11:59:35.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.868+0000 7fcae4ff9640 1 -- 172.21.15.94:0/1838339152 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8095dc0 con 0x5645406de750 2024-02-17T11:59:35.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 -- 172.21.15.94:0/1838339152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406de750 msgr2=0x5645406d8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/1838339152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406de750 0x5645406d8870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcae8017000 tx=0x7fcae809b7b0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 -- 172.21.15.94:0/1838339152 shutdown_connections 2024-02-17T11:59:35.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/1838339152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406de750 0x5645406d8870 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/1838339152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5b70 0x5645406dbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/1838339152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5250 0x5645406ddd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 -- 172.21.15.94:0/1838339152 >> 172.21.15.94:0/1838339152 conn(0x5645406cf980 msgr2=0x5645406d6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 -- 172.21.15.94:0/1838339152 shutdown_connections 2024-02-17T11:59:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 -- 172.21.15.94:0/1838339152 wait complete. 2024-02-17T11:59:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 -- start start 2024-02-17T11:59:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 Processor -- start 2024-02-17T11:59:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5250 0x5645406e4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5b70 0x5645406e4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406de750 0x56454073ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645406e5010 con 0x5645406d5b70 2024-02-17T11:59:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645406e5180 con 0x5645406d5250 2024-02-17T11:59:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.870+0000 7fcaf7c80740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56454073f470 con 0x5645406de750 2024-02-17T11:59:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.871+0000 7fcaf7c80740 10 client.0 _ll_register_callbacks cb 0x7fcae0009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:35.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.871+0000 7fcae7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5b70 0x5645406e4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.871+0000 7fcae7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5b70 0x5645406e4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39764/0 (socket says 172.21.15.94:39764) 2024-02-17T11:59:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.871+0000 7fcae7fff640 1 -- 172.21.15.94:0/2064734460 learned_addr learned my addr 172.21.15.94:0/2064734460 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.871+0000 7fcaeed9d640 1 --2- 172.21.15.94:0/2064734460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5250 0x5645406e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.871+0000 7fcaef59e640 1 --2- 172.21.15.94:0/2064734460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406de750 0x56454073ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.871+0000 7fcaeed9d640 1 -- 172.21.15.94:0/2064734460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406de750 msgr2=0x56454073ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.871+0000 7fcaeed9d640 1 --2- 172.21.15.94:0/2064734460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406de750 0x56454073ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.871+0000 7fcaeed9d640 1 -- 172.21.15.94:0/2064734460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5b70 msgr2=0x5645406e4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.871+0000 7fcaeed9d640 1 --2- 172.21.15.94:0/2064734460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5b70 0x5645406e4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.871+0000 7fcaeed9d640 1 -- 172.21.15.94:0/2064734460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf0073470 con 0x5645406d5250 2024-02-17T11:59:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.871+0000 7fcaeed9d640 1 --2- 172.21.15.94:0/2064734460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5250 0x5645406e4310 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcae0005780 tx=0x7fcae0005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd9f748ce8bfae0c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.871+0000 7fcad67fc640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:59:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcad67fc640 1 -- 172.21.15.94:0/2064734460 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0029070 con 0x5645406d5250 2024-02-17T11:59:35.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcad67fc640 1 -- 172.21.15.94:0/2064734460 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fcae00050e0 con 0x5645406d5250 2024-02-17T11:59:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcaf7c80740 1 -- 172.21.15.94:0/2064734460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcae0004c80 con 0x5645406d5250 2024-02-17T11:59:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcad67fc640 1 -- 172.21.15.94:0/2064734460 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcae0011d10 con 0x5645406d5250 2024-02-17T11:59:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcaf7c80740 10 client.25367 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcaf7c80740 1 -- 172.21.15.94:0/2064734460 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5645405fd8c0 con 0x5645406d5250 2024-02-17T11:59:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcad67fc640 1 -- 172.21.15.94:0/2064734460 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcae0004c80 con 0x5645406d5250 2024-02-17T11:59:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcad67fc640 1 -- 172.21.15.94:0/2064734460 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcae0018440 con 0x5645406d5250 2024-02-17T11:59:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcaf7c80740 10 client.25367 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcaf7c80740 10 client.25367 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T11:59:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcaf7c80740 20 client.25367 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/2064734460 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564540757f60 0x56454075a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcaf7c80740 4 client.25367 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T11:59:35.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.872+0000 7fcaf7c80740 1 -- 172.21.15.94:0/2064734460 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5645403c8890 con 0x564540757f60 2024-02-17T11:59:35.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.873+0000 7fcae7fff640 1 --2- 172.21.15.94:0/2064734460 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564540757f60 0x56454075a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:35.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.873+0000 7fcad67fc640 10 client.25367 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T11:59:35.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.873+0000 7fcae7fff640 1 --2- 172.21.15.94:0/2064734460 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564540757f60 0x56454075a350 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b478010313379a2c server_cookie=9c6b4b131e711a21 in_seq=0 out_seq=0 2024-02-17T11:59:35.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.873+0000 7fcad67fc640 1 -- 172.21.15.94:0/2064734460 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5645403c8890 con 0x564540757f60 2024-02-17T11:59:35.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.873+0000 7fcad67fc640 10 client.25367 handle_command_reply: tid=0 2024-02-17T11:59:35.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.874+0000 7fcaf7c80740 1 client.25367 shutdown 2024-02-17T11:59:35.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.874+0000 7fcaf7c80740 1 -- 172.21.15.94:0/2064734460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5250 msgr2=0x5645406e4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.874+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/2064734460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5250 0x5645406e4310 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcae0005780 tx=0x7fcae0005da0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.874+0000 7fcaf7c80740 1 -- 172.21.15.94:0/2064734460 shutdown_connections 2024-02-17T11:59:35.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.874+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/2064734460 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564540757f60 0x56454075a350 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.874+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/2064734460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645406de750 0x56454073ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.874+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/2064734460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645406d5b70 0x5645406e4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.874+0000 7fcaf7c80740 1 --2- 172.21.15.94:0/2064734460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645406d5250 0x5645406e4310 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.874+0000 7fcaf7c80740 1 -- 172.21.15.94:0/2064734460 >> 172.21.15.94:0/2064734460 conn(0x5645406cf980 msgr2=0x5645406ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.874+0000 7fcad67fc640 0 client.25367 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T11:59:35.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.874+0000 7fcaf7c80740 1 -- 172.21.15.94:0/2064734460 shutdown_connections 2024-02-17T11:59:35.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.874+0000 7fcaf7c80740 1 -- 172.21.15.94:0/2064734460 wait complete. 2024-02-17T11:59:35.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.874+0000 7fcaf7c80740 20 client.25367 trim_cache size 0 max 16384 2024-02-17T11:59:35.877 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T11:59:35.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.875+0000 7fcaf50ae640 1 -- 172.21.15.94:0/3134198058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcadc0aac60 msgr2=0x7fcadc0ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.875+0000 7fcaf50ae640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcadc0aac60 0x7fcadc0ad120 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcad8000c00 tx=0x7fcad802c9c0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.875+0000 7fcaf50ae640 1 -- 172.21.15.94:0/3134198058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaf0197690 msgr2=0x7fcaf0197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:35.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.875+0000 7fcaf50ae640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaf0197690 0x7fcaf0197b10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcae8009fd0 tx=0x7fcae800ef10 comp rx=0 tx=0).stop 2024-02-17T11:59:35.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.875+0000 7fcaf50ae640 1 -- 172.21.15.94:0/3134198058 shutdown_connections 2024-02-17T11:59:35.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.875+0000 7fcaf50ae640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcadc0aac60 0x7fcadc0ad120 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.875+0000 7fcaf50ae640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcaf0197690 0x7fcaf0197b10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.875+0000 7fcaf50ae640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcaf0106450 0x7fcaf019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.875+0000 7fcaf50ae640 1 --2- 172.21.15.94:0/3134198058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcaf0101380 0x7fcaf019d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:35.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.875+0000 7fcaf50ae640 1 -- 172.21.15.94:0/3134198058 >> 172.21.15.94:0/3134198058 conn(0x7fcaf00fcc20 msgr2=0x7fcaf00fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:35.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.875+0000 7fcaf50ae640 1 -- 172.21.15.94:0/3134198058 shutdown_connections 2024-02-17T11:59:35.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:35.875+0000 7fcaf50ae640 1 -- 172.21.15.94:0/3134198058 wait complete. 2024-02-17T11:59:35.886 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T11:59:35.886 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T11:59:36.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.024+0000 7f49204b5640 1 -- 172.21.15.94:0/1201962999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4918105350 msgr2=0x7f491810cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:36.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.024+0000 7f49204b5640 1 --2- 172.21.15.94:0/1201962999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4918105350 0x7f491810cf20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4910009ca0 tx=0x7f491002c100 comp rx=0 tx=0).stop 2024-02-17T11:59:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.024+0000 7f49204b5640 1 -- 172.21.15.94:0/1201962999 shutdown_connections 2024-02-17T11:59:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.024+0000 7f49204b5640 1 --2- 172.21.15.94:0/1201962999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4918105350 0x7f491810cf20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.024+0000 7f49204b5640 1 --2- 172.21.15.94:0/1201962999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4918100c30 0x7f4918104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.024+0000 7f49204b5640 1 --2- 172.21.15.94:0/1201962999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4918100280 0x7f4918100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.024+0000 7f49204b5640 1 -- 172.21.15.94:0/1201962999 >> 172.21.15.94:0/1201962999 conn(0x7f4918076e30 msgr2=0x7f49180fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.025+0000 7f49204b5640 1 -- 172.21.15.94:0/1201962999 shutdown_connections 2024-02-17T11:59:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.025+0000 7f49204b5640 1 -- 172.21.15.94:0/1201962999 wait complete. 2024-02-17T11:59:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.025+0000 7f49204b5640 1 Processor -- start 2024-02-17T11:59:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.026+0000 7f49204b5640 1 -- start start 2024-02-17T11:59:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.026+0000 7f49204b5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4918100280 0x7f491819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.026+0000 7f49204b5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4918105350 0x7f491819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.026+0000 7f49204b5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4918197610 0x7f4918197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.026+0000 7f49204b5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4918198000 con 0x7f4918197610 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.026+0000 7f491e251640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4918100280 0x7f491819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.026+0000 7f49204b5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4918198170 con 0x7f4918100280 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.026+0000 7f49204b5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4918198330 con 0x7f4918105350 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.027+0000 7f491e251640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4918100280 0x7f491819d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44566/0 (socket says 172.21.15.94:44566) 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.027+0000 7f491ea52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4918197610 0x7f4918197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.027+0000 7f491e251640 1 -- 172.21.15.94:0/650761020 learned_addr learned my addr 172.21.15.94:0/650761020 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.027+0000 7f491da50640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4918105350 0x7f491819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.027+0000 7f491e251640 1 -- 172.21.15.94:0/650761020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4918105350 msgr2=0x7f491819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.027+0000 7f491e251640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4918105350 0x7f491819d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.027+0000 7f491e251640 1 -- 172.21.15.94:0/650761020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4918197610 msgr2=0x7f4918197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.027+0000 7f491e251640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4918197610 0x7f4918197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.027+0000 7f491e251640 1 -- 172.21.15.94:0/650761020 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4910009900 con 0x7f4918100280 2024-02-17T11:59:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.027+0000 7f491ea52640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4918197610 0x7f4918197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.027+0000 7f491e251640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4918100280 0x7f491819d460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f491400d950 tx=0x7f491400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.027+0000 7f490f7fe640 1 -- 172.21.15.94:0/650761020 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4914009c30 con 0x7f4918100280 2024-02-17T11:59:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.027+0000 7f49204b5640 1 -- 172.21.15.94:0/650761020 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49181a4320 con 0x7f4918100280 2024-02-17T11:59:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.028+0000 7f490f7fe640 1 -- 172.21.15.94:0/650761020 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4914009d90 con 0x7f4918100280 2024-02-17T11:59:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.028+0000 7f490f7fe640 1 -- 172.21.15.94:0/650761020 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491400f7e0 con 0x7f4918100280 2024-02-17T11:59:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.028+0000 7f49204b5640 1 -- 172.21.15.94:0/650761020 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49181a4880 con 0x7f4918100280 2024-02-17T11:59:36.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.030+0000 7f490f7fe640 1 -- 172.21.15.94:0/650761020 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f4914010ad0 con 0x7f4918100280 2024-02-17T11:59:36.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f4923163740 1 Processor -- start 2024-02-17T11:59:36.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f4923163740 1 -- start start 2024-02-17T11:59:36.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f4923163740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56146099b250 0x56146099b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f4923163740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 0x5614609a4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f4923163740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614609a4750 0x5614609a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f4923163740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561460956960 con 0x56146099b250 2024-02-17T11:59:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f4923163740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56146090a310 con 0x56146099bb70 2024-02-17T11:59:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f4923163740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561460907840 con 0x5614609a4750 2024-02-17T11:59:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f491e251640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 0x5614609a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f491ea52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56146099b250 0x56146099b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f491da50640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614609a4750 0x5614609a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f491da50640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614609a4750 0x5614609a6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33278/0 (socket says 172.21.15.94:33278) 2024-02-17T11:59:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f491ea52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56146099b250 0x56146099b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39808/0 (socket says 172.21.15.94:39808) 2024-02-17T11:59:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f491da50640 1 -- 172.21.15.94:0/2782035299 learned_addr learned my addr 172.21.15.94:0/2782035299 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f491e251640 1 --2- 172.21.15.94:0/2782035299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 0x5614609a4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44574/0 (socket says 172.21.15.94:44574) 2024-02-17T11:59:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f490f7fe640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f49040a2350 0x7f49040a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.031+0000 7f491ea52640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f49040a2350 0x7f49040a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.032+0000 7f491da50640 1 -- 172.21.15.94:0/2782035299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 msgr2=0x5614609a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:36.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.032+0000 7f491da50640 1 --2- 172.21.15.94:0/2782035299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 0x5614609a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.032+0000 7f491da50640 1 -- 172.21.15.94:0/2782035299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56146099b250 msgr2=0x56146099b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:36.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.032+0000 7f491da50640 1 --2- 172.21.15.94:0/2782035299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56146099b250 0x56146099b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.032+0000 7f491da50640 1 -- 172.21.15.94:0/2782035299 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614608c5060 con 0x5614609a4750 2024-02-17T11:59:36.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.032+0000 7f491e251640 1 --2- 172.21.15.94:0/2782035299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 0x5614609a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:59:36.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.032+0000 7f491ea52640 1 --2- 172.21.15.94:0/2782035299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56146099b250 0x56146099b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:36.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.032+0000 7f491da50640 1 --2- 172.21.15.94:0/2782035299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614609a4750 0x5614609a6b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f490800bbb0 tx=0x7f49080107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b1fefaf2efe26c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:36.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.032+0000 7f490cbfe640 1 -- 172.21.15.94:0/2782035299 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908004030 con 0x5614609a4750 2024-02-17T11:59:36.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.033+0000 7f490cbfe640 1 -- 172.21.15.94:0/2782035299 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4908015070 con 0x5614609a4750 2024-02-17T11:59:36.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.033+0000 7f490cbfe640 1 -- 172.21.15.94:0/2782035299 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908004a40 con 0x5614609a4750 2024-02-17T11:59:36.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.033+0000 7f491ea52640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f49040a2350 0x7f49040a4810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4910004af0 tx=0x7f491002c100 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:36.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.033+0000 7f490f7fe640 1 -- 172.21.15.94:0/650761020 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f49140cdf70 con 0x7f4918100280 2024-02-17T11:59:36.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.034+0000 7f4923163740 1 -- 172.21.15.94:0/2782035299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614609a4750 msgr2=0x5614609a6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:36.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.034+0000 7f4923163740 1 --2- 172.21.15.94:0/2782035299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614609a4750 0x5614609a6b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f490800bbb0 tx=0x7f49080107a0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.034+0000 7f4923163740 1 -- 172.21.15.94:0/2782035299 shutdown_connections 2024-02-17T11:59:36.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.034+0000 7f4923163740 1 --2- 172.21.15.94:0/2782035299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614609a4750 0x5614609a6b40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.034+0000 7f4923163740 1 --2- 172.21.15.94:0/2782035299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 0x5614609a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.034+0000 7f4923163740 1 --2- 172.21.15.94:0/2782035299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56146099b250 0x56146099b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.034+0000 7f4923163740 1 -- 172.21.15.94:0/2782035299 >> 172.21.15.94:0/2782035299 conn(0x561460995980 msgr2=0x561460995d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:36.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.035+0000 7f4923163740 1 -- 172.21.15.94:0/2782035299 shutdown_connections 2024-02-17T11:59:36.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.035+0000 7f4923163740 1 -- 172.21.15.94:0/2782035299 wait complete. 2024-02-17T11:59:36.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.035+0000 7f4923163740 1 -- start start 2024-02-17T11:59:36.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.035+0000 7f4923163740 1 Processor -- start 2024-02-17T11:59:36.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.035+0000 7f4923163740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 0x5614609ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.035+0000 7f4923163740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 0x5614609aebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.035+0000 7f4923163740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609a4750 0x5614609a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.035+0000 7f4923163740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561460903020 con 0x5614609a4750 2024-02-17T11:59:36.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.035+0000 7f4923163740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614605d6430 con 0x56146099bb70 2024-02-17T11:59:36.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.035+0000 7f4923163740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561460592e00 con 0x56146099b250 2024-02-17T11:59:36.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.036+0000 7f491da50640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 0x5614609aebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.036+0000 7f4923163740 10 client.0 _ll_register_callbacks cb 0x5614608aacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:36.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.036+0000 7f491da50640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 0x5614609aebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44576/0 (socket says 172.21.15.94:44576) 2024-02-17T11:59:36.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.036+0000 7f491da50640 1 -- 172.21.15.94:0/267943196 learned_addr learned my addr 172.21.15.94:0/267943196 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:36.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.036+0000 7f491ea52640 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609a4750 0x5614609a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.036+0000 7f491e251640 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 0x5614609ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.036+0000 7f491da50640 1 -- 172.21.15.94:0/267943196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 msgr2=0x5614609ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.036+0000 7f491da50640 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 0x5614609ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.036+0000 7f491da50640 1 -- 172.21.15.94:0/267943196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609a4750 msgr2=0x5614609a8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.036+0000 7f491da50640 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609a4750 0x5614609a8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.036+0000 7f491da50640 1 -- 172.21.15.94:0/267943196 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490800b7e0 con 0x56146099bb70 2024-02-17T11:59:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.036+0000 7f491ea52640 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609a4750 0x5614609a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:59:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.036+0000 7f491e251640 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 0x5614609ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:59:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.037+0000 7f491da50640 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 0x5614609aebe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4908011010 tx=0x7f490800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3091b7ae9b4a764 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.037+0000 7f4901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:59:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.037+0000 7f4901ffb640 1 -- 172.21.15.94:0/267943196 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908021070 con 0x56146099bb70 2024-02-17T11:59:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.037+0000 7f4901ffb640 1 -- 172.21.15.94:0/267943196 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4908017070 con 0x56146099bb70 2024-02-17T11:59:36.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.037+0000 7f4923163740 1 -- 172.21.15.94:0/267943196 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614606027e0 con 0x56146099bb70 2024-02-17T11:59:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.038+0000 7f4901ffb640 1 -- 172.21.15.94:0/267943196 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4908004310 con 0x56146099bb70 2024-02-17T11:59:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.038+0000 7f4923163740 10 client.25379 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.038+0000 7f4923163740 1 -- 172.21.15.94:0/267943196 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5614609016a0 con 0x56146099bb70 2024-02-17T11:59:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.038+0000 7f4901ffb640 1 -- 172.21.15.94:0/267943196 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5614606027e0 con 0x56146099bb70 2024-02-17T11:59:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.039+0000 7f4901ffb640 1 -- 172.21.15.94:0/267943196 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f490802d070 con 0x56146099bb70 2024-02-17T11:59:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.039+0000 7f4923163740 10 client.25379 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.039+0000 7f4923163740 10 client.25379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.039+0000 7f4923163740 20 client.25379 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.039+0000 7f4923163740 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614609b8e30 0x5614609bb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.039+0000 7f4923163740 4 client.25379 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.039+0000 7f4923163740 1 -- 172.21.15.94:0/267943196 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56146063f720 con 0x5614609b8e30 2024-02-17T11:59:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.039+0000 7f491e251640 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614609b8e30 0x5614609bb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.039+0000 7f4901ffb640 10 client.25379 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.039+0000 7f491e251640 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614609b8e30 0x5614609bb220 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c774680238f3608 server_cookie=2afba360b1584ac8 in_seq=0 out_seq=0 2024-02-17T11:59:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.040+0000 7f4901ffb640 1 -- 172.21.15.94:0/267943196 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56146063f720 con 0x5614609b8e30 2024-02-17T11:59:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.040+0000 7f4901ffb640 10 client.25379 handle_command_reply: tid=0 2024-02-17T11:59:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.040+0000 7f4923163740 1 client.25379 shutdown 2024-02-17T11:59:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.040+0000 7f4923163740 1 -- 172.21.15.94:0/267943196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 msgr2=0x5614609aebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.040+0000 7f4923163740 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 0x5614609aebe0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4908011010 tx=0x7f490800bf90 comp rx=0 tx=0).stop 2024-02-17T11:59:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.040+0000 7f4923163740 1 -- 172.21.15.94:0/267943196 shutdown_connections 2024-02-17T11:59:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.040+0000 7f4923163740 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614609b8e30 0x5614609bb220 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.040+0000 7f4923163740 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609a4750 0x5614609a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.040+0000 7f4923163740 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56146099bb70 0x5614609aebe0 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4908011010 tx=0x7f490800bf90 comp rx=0 tx=0).stop 2024-02-17T11:59:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.040+0000 7f4923163740 1 --2- 172.21.15.94:0/267943196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 0x5614609ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.040+0000 7f4923163740 1 -- 172.21.15.94:0/267943196 >> 172.21.15.94:0/267943196 conn(0x561460995980 msgr2=0x5614609976d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.041+0000 7f4901ffb640 0 client.25379 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.041+0000 7f4923163740 1 -- 172.21.15.94:0/267943196 shutdown_connections 2024-02-17T11:59:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.041+0000 7f4923163740 1 -- 172.21.15.94:0/267943196 wait complete. 2024-02-17T11:59:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.041+0000 7f4923163740 20 client.25379 trim_cache size 0 max 16384 2024-02-17T11:59:36.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.060+0000 7f4923163740 1 Processor -- start 2024-02-17T11:59:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.060+0000 7f4923163740 1 -- start start 2024-02-17T11:59:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.061+0000 7f4923163740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 0x5614609bfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.061+0000 7f4923163740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614609a4750 0x5614609a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.061+0000 7f4923163740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609c0100 0x5614609a1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.061+0000 7f4923163740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561460888130 con 0x5614609c0100 2024-02-17T11:59:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.061+0000 7f4923163740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561460987a40 con 0x5614609a4750 2024-02-17T11:59:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.061+0000 7f4923163740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614609a7110 con 0x56146099b250 2024-02-17T11:59:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.061+0000 7f491ea52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609c0100 0x5614609a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.061+0000 7f491ea52640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609c0100 0x5614609a1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39834/0 (socket says 172.21.15.94:39834) 2024-02-17T11:59:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.061+0000 7f491ea52640 1 -- 172.21.15.94:0/2985718049 learned_addr learned my addr 172.21.15.94:0/2985718049 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:36.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.061+0000 7f491da50640 1 --2- 172.21.15.94:0/2985718049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614609a4750 0x5614609a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.062+0000 7f491da50640 1 -- 172.21.15.94:0/2985718049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 msgr2=0x5614609bfbc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T11:59:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.062+0000 7f491da50640 1 --2- 172.21.15.94:0/2985718049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 0x5614609bfbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.062+0000 7f491da50640 1 -- 172.21.15.94:0/2985718049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609c0100 msgr2=0x5614609a1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.062+0000 7f491da50640 1 --2- 172.21.15.94:0/2985718049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609c0100 0x5614609a1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.062+0000 7f491da50640 1 -- 172.21.15.94:0/2985718049 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56146089dc20 con 0x5614609a4750 2024-02-17T11:59:36.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.063+0000 7f491da50640 1 --2- 172.21.15.94:0/2985718049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614609a4750 0x5614609a0ee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5614609a9b70 tx=0x7f490800b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b535ac0ed8da82 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.063+0000 7f4903fff640 1 -- 172.21.15.94:0/2985718049 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4908021030 con 0x5614609a4750 2024-02-17T11:59:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.063+0000 7f4903fff640 1 -- 172.21.15.94:0/2985718049 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4908005380 con 0x5614609a4750 2024-02-17T11:59:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.063+0000 7f4903fff640 1 -- 172.21.15.94:0/2985718049 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490802f420 con 0x5614609a4750 2024-02-17T11:59:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.064+0000 7f4923163740 1 -- 172.21.15.94:0/2985718049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614609a4750 msgr2=0x5614609a0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:36.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.064+0000 7f4923163740 1 --2- 172.21.15.94:0/2985718049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614609a4750 0x5614609a0ee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5614609a9b70 tx=0x7f490800b390 comp rx=0 tx=0).stop 2024-02-17T11:59:36.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.069+0000 7f4923163740 1 -- 172.21.15.94:0/2985718049 shutdown_connections 2024-02-17T11:59:36.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.069+0000 7f4923163740 1 --2- 172.21.15.94:0/2985718049 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609c0100 0x5614609a1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.069+0000 7f4923163740 1 --2- 172.21.15.94:0/2985718049 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614609a4750 0x5614609a0ee0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.069+0000 7f4923163740 1 --2- 172.21.15.94:0/2985718049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 0x5614609bfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.069+0000 7f4923163740 1 -- 172.21.15.94:0/2985718049 >> 172.21.15.94:0/2985718049 conn(0x561460995980 msgr2=0x561460990a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:36.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.070+0000 7f4923163740 1 -- 172.21.15.94:0/2985718049 shutdown_connections 2024-02-17T11:59:36.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.070+0000 7f4923163740 1 -- 172.21.15.94:0/2985718049 wait complete. 2024-02-17T11:59:36.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.070+0000 7f4923163740 1 -- start start 2024-02-17T11:59:36.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.070+0000 7f4923163740 1 Processor -- start 2024-02-17T11:59:36.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.071+0000 7f4923163740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 0x561460a144d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.071+0000 7f4923163740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609a4750 0x561460a14a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.071+0000 7f4923163740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614609c0100 0x561460a193a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.071+0000 7f4923163740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561460a198e0 con 0x5614609a4750 2024-02-17T11:59:36.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.071+0000 7f4923163740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561460a19a50 con 0x5614609c0100 2024-02-17T11:59:36.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.071+0000 7f4923163740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561460a19c10 con 0x56146099b250 2024-02-17T11:59:36.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.071+0000 7f4923163740 10 client.0 _ll_register_callbacks cb 0x7f490800b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:36.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.071+0000 7f491ea52640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614609c0100 0x561460a193a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.072+0000 7f491ea52640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614609c0100 0x561460a193a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44600/0 (socket says 172.21.15.94:44600) 2024-02-17T11:59:36.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.072+0000 7f491ea52640 1 -- 172.21.15.94:0/2285118405 learned_addr learned my addr 172.21.15.94:0/2285118405 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:36.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.072+0000 7f491e251640 1 --2- 172.21.15.94:0/2285118405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 0x561460a144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.072+0000 7f491da50640 1 --2- 172.21.15.94:0/2285118405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609a4750 0x561460a14a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.072+0000 7f491ea52640 1 -- 172.21.15.94:0/2285118405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 msgr2=0x561460a144d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:36.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.072+0000 7f491ea52640 1 --2- 172.21.15.94:0/2285118405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 0x561460a144d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.072+0000 7f491ea52640 1 -- 172.21.15.94:0/2285118405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609a4750 msgr2=0x561460a14a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:36.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.072+0000 7f491ea52640 1 --2- 172.21.15.94:0/2285118405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609a4750 0x561460a14a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.072+0000 7f491ea52640 1 -- 172.21.15.94:0/2285118405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614609016a0 con 0x5614609c0100 2024-02-17T11:59:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.073+0000 7f491ea52640 1 --2- 172.21.15.94:0/2285118405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614609c0100 0x561460a193a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4910005cc0 tx=0x7f49100095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da207ecb2e4a398f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.073+0000 7f4901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:59:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.073+0000 7f4901ffb640 1 -- 172.21.15.94:0/2285118405 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910004250 con 0x5614609c0100 2024-02-17T11:59:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.074+0000 7f4923163740 1 -- 172.21.15.94:0/2285118405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4908015070 con 0x5614609c0100 2024-02-17T11:59:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.074+0000 7f4901ffb640 1 -- 172.21.15.94:0/2285118405 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f49100043b0 con 0x5614609c0100 2024-02-17T11:59:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.074+0000 7f4901ffb640 1 -- 172.21.15.94:0/2285118405 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4910004570 con 0x5614609c0100 2024-02-17T11:59:36.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.074+0000 7f4923163740 10 client.25385 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.074+0000 7f4923163740 1 -- 172.21.15.94:0/2285118405 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5614608c38c0 con 0x5614609c0100 2024-02-17T11:59:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.074+0000 7f4901ffb640 1 -- 172.21.15.94:0/2285118405 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4908015070 con 0x5614609c0100 2024-02-17T11:59:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.075+0000 7f4901ffb640 1 -- 172.21.15.94:0/2285118405 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4910044440 con 0x5614609c0100 2024-02-17T11:59:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.075+0000 7f4923163740 10 client.25385 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.075+0000 7f4923163740 10 client.25385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.075+0000 7f4923163740 20 client.25385 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.075+0000 7f4923163740 1 --2- 172.21.15.94:0/2285118405 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561460a03a80 0x561460a05ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.075+0000 7f4923163740 4 client.25385 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T11:59:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.075+0000 7f4923163740 1 -- 172.21.15.94:0/2285118405 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56146068e890 con 0x561460a03a80 2024-02-17T11:59:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.075+0000 7f491e251640 1 --2- 172.21.15.94:0/2285118405 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561460a03a80 0x561460a05ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.075+0000 7f4901ffb640 10 client.25385 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.075+0000 7f491e251640 1 --2- 172.21.15.94:0/2285118405 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561460a03a80 0x561460a05ea0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=952f4ad7d2fa8f02 server_cookie=fc3fe7f59e2fc3fe in_seq=0 out_seq=0 2024-02-17T11:59:36.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.075+0000 7f4901ffb640 1 -- 172.21.15.94:0/2285118405 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56146068e890 con 0x561460a03a80 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.075+0000 7f4901ffb640 10 client.25385 handle_command_reply: tid=0 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.075+0000 7f4923163740 1 client.25385 shutdown 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f4923163740 1 -- 172.21.15.94:0/2285118405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614609c0100 msgr2=0x561460a193a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f4923163740 1 --2- 172.21.15.94:0/2285118405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614609c0100 0x561460a193a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4910005cc0 tx=0x7f49100095c0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f4923163740 1 -- 172.21.15.94:0/2285118405 shutdown_connections 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f4923163740 1 --2- 172.21.15.94:0/2285118405 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561460a03a80 0x561460a05ea0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f4923163740 1 --2- 172.21.15.94:0/2285118405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614609c0100 0x561460a193a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f4923163740 1 --2- 172.21.15.94:0/2285118405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614609a4750 0x561460a14a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f4923163740 1 --2- 172.21.15.94:0/2285118405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56146099b250 0x561460a144d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f4901ffb640 0 client.25385 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f4923163740 1 -- 172.21.15.94:0/2285118405 >> 172.21.15.94:0/2285118405 conn(0x561460995980 msgr2=0x5614609a0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f4923163740 1 -- 172.21.15.94:0/2285118405 shutdown_connections 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f4923163740 1 -- 172.21.15.94:0/2285118405 wait complete. 2024-02-17T11:59:36.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f4923163740 20 client.25385 trim_cache size 0 max 16384 2024-02-17T11:59:36.078 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T11:59:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f49204b5640 1 -- 172.21.15.94:0/650761020 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f49040a2350 msgr2=0x7f49040a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f49204b5640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f49040a2350 0x7f49040a4810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4910004af0 tx=0x7f491002c100 comp rx=0 tx=0).stop 2024-02-17T11:59:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f49204b5640 1 -- 172.21.15.94:0/650761020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4918100280 msgr2=0x7f491819d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f49204b5640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4918100280 0x7f491819d460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f491400d950 tx=0x7f491400de10 comp rx=0 tx=0).stop 2024-02-17T11:59:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f49204b5640 1 -- 172.21.15.94:0/650761020 shutdown_connections 2024-02-17T11:59:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f49204b5640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f49040a2350 0x7f49040a4810 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f49204b5640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4918197610 0x7f4918197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f49204b5640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4918105350 0x7f491819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f49204b5640 1 --2- 172.21.15.94:0/650761020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4918100280 0x7f491819d460 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f49204b5640 1 -- 172.21.15.94:0/650761020 >> 172.21.15.94:0/650761020 conn(0x7f4918076e30 msgr2=0x7f49181065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f49204b5640 1 -- 172.21.15.94:0/650761020 shutdown_connections 2024-02-17T11:59:36.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:36.076+0000 7f49204b5640 1 -- 172.21.15.94:0/650761020 wait complete. 2024-02-17T11:59:36.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:35 smithi189 ceph-mon[36588]: pgmap v47: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.0 MiB/s wr, 676 op/s 2024-02-17T11:59:36.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:35 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/92816919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:59:36.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:35 smithi125 ceph-mon[36819]: pgmap v47: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.0 MiB/s wr, 676 op/s 2024-02-17T11:59:36.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:35 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/92816919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:59:36.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:35 smithi094 ceph-mon[32499]: pgmap v47: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.0 MiB/s wr, 676 op/s 2024-02-17T11:59:36.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:35 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/92816919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T11:59:37.089 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T11:59:37.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T11:59:37.090 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T11:59:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.238+0000 7f1d82c88640 1 -- 172.21.15.94:0/497321434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d7c108180 msgr2=0x7f1d7c108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.238+0000 7f1d82c88640 1 --2- 172.21.15.94:0/497321434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d7c108180 0x7f1d7c108560 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1d70009900 tx=0x7f1d7002bce0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.238+0000 7f1d82c88640 1 -- 172.21.15.94:0/497321434 shutdown_connections 2024-02-17T11:59:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.238+0000 7f1d82c88640 1 --2- 172.21.15.94:0/497321434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c1142d0 0x7f1d7c1166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.238+0000 7f1d82c88640 1 --2- 172.21.15.94:0/497321434 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d7c108aa0 0x7f1d7c113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.238+0000 7f1d82c88640 1 --2- 172.21.15.94:0/497321434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d7c108180 0x7f1d7c108560 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.238+0000 7f1d82c88640 1 -- 172.21.15.94:0/497321434 >> 172.21.15.94:0/497321434 conn(0x7f1d7c06c6c0 msgr2=0x7f1d7c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.238+0000 7f1d82c88640 1 -- 172.21.15.94:0/497321434 shutdown_connections 2024-02-17T11:59:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.238+0000 7f1d82c88640 1 -- 172.21.15.94:0/497321434 wait complete. 2024-02-17T11:59:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.238+0000 7f1d82c88640 1 Processor -- start 2024-02-17T11:59:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.238+0000 7f1d82c88640 1 -- start start 2024-02-17T11:59:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d82c88640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d7c108180 0x7f1d7c1a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d82c88640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c108aa0 0x7f1d7c1a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d82c88640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d7c1142d0 0x7f1d7c1a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d82c88640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c1a4f10 con 0x7f1d7c108aa0 2024-02-17T11:59:37.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d82c88640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c1a5080 con 0x7f1d7c108180 2024-02-17T11:59:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d82c88640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c1a5240 con 0x7f1d7c1142d0 2024-02-17T11:59:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c108aa0 0x7f1d7c1a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d81225640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d7c1142d0 0x7f1d7c1a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d81225640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d7c1142d0 0x7f1d7c1a4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33318/0 (socket says 172.21.15.94:33318) 2024-02-17T11:59:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d81225640 1 -- 172.21.15.94:0/2229058225 learned_addr learned my addr 172.21.15.94:0/2229058225 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d80a24640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d7c108180 0x7f1d7c1a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d81225640 1 -- 172.21.15.94:0/2229058225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d7c108180 msgr2=0x7f1d7c1a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d81225640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d7c108180 0x7f1d7c1a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d81225640 1 -- 172.21.15.94:0/2229058225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c108aa0 msgr2=0x7f1d7c1a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d81225640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c108aa0 0x7f1d7c1a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d81225640 1 -- 172.21.15.94:0/2229058225 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d74009840 con 0x7f1d7c1142d0 2024-02-17T11:59:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d7bfff640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c108aa0 0x7f1d7c1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:37.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d81225640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d7c1142d0 0x7f1d7c1a4910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400ec80 tx=0x7f1d7400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:37.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d79ffb640 1 -- 172.21.15.94:0/2229058225 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74003b70 con 0x7f1d7c1142d0 2024-02-17T11:59:37.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.239+0000 7f1d79ffb640 1 -- 172.21.15.94:0/2229058225 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d74003cd0 con 0x7f1d7c1142d0 2024-02-17T11:59:37.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.240+0000 7f1d79ffb640 1 -- 172.21.15.94:0/2229058225 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d740108f0 con 0x7f1d7c1142d0 2024-02-17T11:59:37.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.240+0000 7f1d82c88640 1 -- 172.21.15.94:0/2229058225 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d70009560 con 0x7f1d7c1142d0 2024-02-17T11:59:37.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.241+0000 7f1d82c88640 1 -- 172.21.15.94:0/2229058225 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c1acbc0 con 0x7f1d7c1142d0 2024-02-17T11:59:37.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.243+0000 7f1d79ffb640 1 -- 172.21.15.94:0/2229058225 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f1d74010a50 con 0x7f1d7c1142d0 2024-02-17T11:59:37.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.243+0000 7f1d85866740 1 Processor -- start 2024-02-17T11:59:37.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.243+0000 7f1d85866740 1 -- start start 2024-02-17T11:59:37.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d79ffb640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1d6c0a23d0 0x7f1d6c0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d85866740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b145887120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d85866740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458876f0 0x55b14588fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d85866740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458902f0 0x55b1458926e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d85866740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1457ecf90 con 0x55b1458876f0 2024-02-17T11:59:37.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d85866740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1457ea4c0 con 0x55b1458902f0 2024-02-17T11:59:37.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d85866740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1457e5ca0 con 0x55b145886d40 2024-02-17T11:59:37.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d7bfff640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1d6c0a23d0 0x7f1d6c0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d79ffb640 1 -- 172.21.15.94:0/2229058225 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1d74014070 con 0x7f1d7c1142d0 2024-02-17T11:59:37.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458876f0 0x55b14588fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458876f0 0x55b14588fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39864/0 (socket says 172.21.15.94:39864) 2024-02-17T11:59:37.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d7bfff640 1 -- 172.21.15.94:0/1202691754 learned_addr learned my addr 172.21.15.94:0/1202691754 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:37.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d80a24640 1 --2- 172.21.15.94:0/1202691754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b145887120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.244+0000 7f1d81225640 1 --2- 172.21.15.94:0/1202691754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458902f0 0x55b1458926e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.245+0000 7f1d80a24640 1 -- 172.21.15.94:0/1202691754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458902f0 msgr2=0x55b1458926e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.245+0000 7f1d80a24640 1 --2- 172.21.15.94:0/1202691754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458902f0 0x55b1458926e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.245+0000 7f1d80a24640 1 -- 172.21.15.94:0/1202691754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458876f0 msgr2=0x55b14588fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.245+0000 7f1d80a24640 1 --2- 172.21.15.94:0/1202691754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458876f0 0x55b14588fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.245+0000 7f1d80a24640 1 -- 172.21.15.94:0/1202691754 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1457a80b0 con 0x55b145886d40 2024-02-17T11:59:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.245+0000 7f1d80a24640 1 --2- 172.21.15.94:0/1202691754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b145887120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d7002c220 tx=0x7f1d700046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb9c5b941e12f898 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.245+0000 7f1d7bfff640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1d6c0a23d0 0x7f1d6c0a4890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009fd0 tx=0x7f1d68009d60 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.245+0000 7f1d66ffd640 1 -- 172.21.15.94:0/1202691754 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d700048b0 con 0x55b145886d40 2024-02-17T11:59:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.245+0000 7f1d66ffd640 1 -- 172.21.15.94:0/1202691754 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d70004a10 con 0x55b145886d40 2024-02-17T11:59:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.245+0000 7f1d66ffd640 1 -- 172.21.15.94:0/1202691754 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70005050 con 0x55b145886d40 2024-02-17T11:59:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.247+0000 7f1d85866740 1 -- 172.21.15.94:0/1202691754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 msgr2=0x55b145887120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.247+0000 7f1d85866740 1 --2- 172.21.15.94:0/1202691754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b145887120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d7002c220 tx=0x7f1d700046c0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.247+0000 7f1d85866740 1 -- 172.21.15.94:0/1202691754 shutdown_connections 2024-02-17T11:59:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.247+0000 7f1d85866740 1 --2- 172.21.15.94:0/1202691754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458902f0 0x55b1458926e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.247+0000 7f1d85866740 1 --2- 172.21.15.94:0/1202691754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458876f0 0x55b14588fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.247+0000 7f1d85866740 1 --2- 172.21.15.94:0/1202691754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b145887120 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.247+0000 7f1d85866740 1 -- 172.21.15.94:0/1202691754 >> 172.21.15.94:0/1202691754 conn(0x55b14587b1d0 msgr2=0x55b14587b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.247+0000 7f1d85866740 1 -- 172.21.15.94:0/1202691754 shutdown_connections 2024-02-17T11:59:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.247+0000 7f1d85866740 1 -- 172.21.15.94:0/1202691754 wait complete. 2024-02-17T11:59:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.247+0000 7f1d85866740 1 -- start start 2024-02-17T11:59:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.247+0000 7f1d85866740 1 Processor -- start 2024-02-17T11:59:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d85866740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b14588d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d85866740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458876f0 0x55b14588d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d85866740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458902f0 0x55b14588ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d85866740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14578af40 con 0x55b1458902f0 2024-02-17T11:59:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d85866740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1454b94b0 con 0x55b1458876f0 2024-02-17T11:59:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d85866740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b14576b670 con 0x55b145886d40 2024-02-17T11:59:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d85866740 10 client.0 _ll_register_callbacks cb 0x55b1457d8f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d81225640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458902f0 0x55b14588ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d81225640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458902f0 0x55b14588ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39874/0 (socket says 172.21.15.94:39874) 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d81225640 1 -- 172.21.15.94:0/1608138211 learned_addr learned my addr 172.21.15.94:0/1608138211 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d7bfff640 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458876f0 0x55b14588d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d80a24640 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b14588d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d80a24640 1 -- 172.21.15.94:0/1608138211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458876f0 msgr2=0x55b14588d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d80a24640 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458876f0 0x55b14588d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d80a24640 1 -- 172.21.15.94:0/1608138211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458902f0 msgr2=0x55b14588ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d80a24640 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458902f0 0x55b14588ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d80a24640 1 -- 172.21.15.94:0/1608138211 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c117010 con 0x55b145886d40 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d7bfff640 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458876f0 0x55b14588d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.248+0000 7f1d81225640 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458902f0 0x55b14588ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d80a24640 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b14588d330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d70005ab0 tx=0x7f1d7002cb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=348538945a771199 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d5bfff640 1 -- 172.21.15.94:0/1608138211 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70004440 con 0x55b145886d40 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d85866740 1 -- 172.21.15.94:0/1608138211 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1457bff40 con 0x55b145886d40 2024-02-17T11:59:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d5bfff640 1 -- 172.21.15.94:0/1608138211 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1d7002cd10 con 0x55b145886d40 2024-02-17T11:59:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d5bfff640 1 -- 172.21.15.94:0/1608138211 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d700035f0 con 0x55b145886d40 2024-02-17T11:59:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d85866740 10 client.25444 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d85866740 1 -- 172.21.15.94:0/1608138211 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55b1457e4320 con 0x55b145886d40 2024-02-17T11:59:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d5bfff640 1 -- 172.21.15.94:0/1608138211 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d70056080 con 0x55b145886d40 2024-02-17T11:59:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d5bfff640 1 -- 172.21.15.94:0/1608138211 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1d7003c050 con 0x55b145886d40 2024-02-17T11:59:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d85866740 10 client.25444 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d85866740 10 client.25444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d85866740 20 client.25444 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d85866740 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b1458a0360 0x55b1458a2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d85866740 4 client.25444 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.249+0000 7f1d85866740 1 -- 172.21.15.94:0/1608138211 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1457dc310 con 0x55b1458a0360 2024-02-17T11:59:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.250+0000 7f1d7bfff640 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b1458a0360 0x55b1458a2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.251+0000 7f1d5bfff640 10 client.25444 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.251+0000 7f1d7bfff640 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b1458a0360 0x55b1458a2780 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c8b004b21568052 server_cookie=42d0038dc1f3ba88 in_seq=0 out_seq=0 2024-02-17T11:59:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.251+0000 7f1d5bfff640 1 -- 172.21.15.94:0/1608138211 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b1457dc310 con 0x55b1458a0360 2024-02-17T11:59:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.251+0000 7f1d5bfff640 10 client.25444 handle_command_reply: tid=0 2024-02-17T11:59:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.252+0000 7f1d85866740 1 client.25444 shutdown 2024-02-17T11:59:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.252+0000 7f1d85866740 1 -- 172.21.15.94:0/1608138211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 msgr2=0x55b14588d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.252+0000 7f1d85866740 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b14588d330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1d70005ab0 tx=0x7f1d7002cb60 comp rx=0 tx=0).stop 2024-02-17T11:59:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.252+0000 7f1d85866740 1 -- 172.21.15.94:0/1608138211 shutdown_connections 2024-02-17T11:59:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.252+0000 7f1d85866740 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b1458a0360 0x55b1458a2780 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.252+0000 7f1d85866740 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458902f0 0x55b14588ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.252+0000 7f1d5bfff640 0 client.25444 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.252+0000 7f1d85866740 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458876f0 0x55b14588d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.252+0000 7f1d85866740 1 --2- 172.21.15.94:0/1608138211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b14588d330 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.252+0000 7f1d85866740 1 -- 172.21.15.94:0/1608138211 >> 172.21.15.94:0/1608138211 conn(0x55b14587b1d0 msgr2=0x55b14587a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.252+0000 7f1d85866740 1 -- 172.21.15.94:0/1608138211 shutdown_connections 2024-02-17T11:59:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.252+0000 7f1d85866740 1 -- 172.21.15.94:0/1608138211 wait complete. 2024-02-17T11:59:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.252+0000 7f1d85866740 20 client.25444 trim_cache size 0 max 16384 2024-02-17T11:59:37.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.273+0000 7f1d85866740 1 Processor -- start 2024-02-17T11:59:37.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.273+0000 7f1d85866740 1 -- start start 2024-02-17T11:59:37.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.273+0000 7f1d85866740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b14588fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.273+0000 7f1d85866740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458876f0 0x55b14588c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.273+0000 7f1d85866740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458902f0 0x55b14588dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.273+0000 7f1d85866740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145879c00 con 0x55b1458902f0 2024-02-17T11:59:37.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.273+0000 7f1d85866740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145878ea0 con 0x55b1458876f0 2024-02-17T11:59:37.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.273+0000 7f1d85866740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145888ee0 con 0x55b145886d40 2024-02-17T11:59:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.273+0000 7f1d7bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458876f0 0x55b14588c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.273+0000 7f1d7bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458876f0 0x55b14588c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44622/0 (socket says 172.21.15.94:44622) 2024-02-17T11:59:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.273+0000 7f1d81225640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458902f0 0x55b14588dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.273+0000 7f1d7bfff640 1 -- 172.21.15.94:0/1851680831 learned_addr learned my addr 172.21.15.94:0/1851680831 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.274+0000 7f1d80a24640 1 --2- 172.21.15.94:0/1851680831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b14588fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.274+0000 7f1d7bfff640 1 -- 172.21.15.94:0/1851680831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 msgr2=0x55b14588fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.274+0000 7f1d7bfff640 1 --2- 172.21.15.94:0/1851680831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b14588fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.274+0000 7f1d7bfff640 1 -- 172.21.15.94:0/1851680831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458902f0 msgr2=0x55b14588dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.274+0000 7f1d7bfff640 1 --2- 172.21.15.94:0/1851680831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458902f0 0x55b14588dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.274+0000 7f1d7bfff640 1 -- 172.21.15.94:0/1851680831 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b145780c00 con 0x55b1458876f0 2024-02-17T11:59:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.274+0000 7f1d7bfff640 1 --2- 172.21.15.94:0/1851680831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458876f0 0x55b14588c330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009780 tx=0x7f1d6801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff3bc5fd5dbf2f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.274+0000 7f1d667fc640 1 -- 172.21.15.94:0/1851680831 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6802a070 con 0x55b1458876f0 2024-02-17T11:59:37.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.274+0000 7f1d667fc640 1 -- 172.21.15.94:0/1851680831 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d6801c070 con 0x55b1458876f0 2024-02-17T11:59:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.274+0000 7f1d667fc640 1 -- 172.21.15.94:0/1851680831 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d68030510 con 0x55b1458876f0 2024-02-17T11:59:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.275+0000 7f1d85866740 1 -- 172.21.15.94:0/1851680831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458876f0 msgr2=0x55b14588c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.275+0000 7f1d85866740 1 --2- 172.21.15.94:0/1851680831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458876f0 0x55b14588c330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009780 tx=0x7f1d6801a000 comp rx=0 tx=0).stop 2024-02-17T11:59:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.275+0000 7f1d85866740 1 -- 172.21.15.94:0/1851680831 shutdown_connections 2024-02-17T11:59:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.275+0000 7f1d85866740 1 --2- 172.21.15.94:0/1851680831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458902f0 0x55b14588dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.275+0000 7f1d85866740 1 --2- 172.21.15.94:0/1851680831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458876f0 0x55b14588c330 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.275+0000 7f1d85866740 1 --2- 172.21.15.94:0/1851680831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b14588fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.275+0000 7f1d85866740 1 -- 172.21.15.94:0/1851680831 >> 172.21.15.94:0/1851680831 conn(0x55b14587b1d0 msgr2=0x55b14588bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.275+0000 7f1d85866740 1 -- 172.21.15.94:0/1851680831 shutdown_connections 2024-02-17T11:59:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.275+0000 7f1d85866740 1 -- 172.21.15.94:0/1851680831 wait complete. 2024-02-17T11:59:37.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.275+0000 7f1d85866740 1 -- start start 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.275+0000 7f1d85866740 1 Processor -- start 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d85866740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b1458963d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d85866740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458876f0 0x55b145896910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d85866740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458902f0 0x55b1458805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d85866740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145880b10 con 0x55b1458876f0 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d85866740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145880c80 con 0x55b1458902f0 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d85866740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b145880e40 con 0x55b145886d40 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d85866740 10 client.0 _ll_register_callbacks cb 0x7f1d70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d80a24640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b1458963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d81225640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458902f0 0x55b1458805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d80a24640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b1458963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33360/0 (socket says 172.21.15.94:33360) 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d80a24640 1 -- 172.21.15.94:0/1943433824 learned_addr learned my addr 172.21.15.94:0/1943433824 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d81225640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458902f0 0x55b1458805d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44636/0 (socket says 172.21.15.94:44636) 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d7bfff640 1 --2- 172.21.15.94:0/1943433824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458876f0 0x55b145896910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d81225640 1 -- 172.21.15.94:0/1943433824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 msgr2=0x55b1458963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d81225640 1 --2- 172.21.15.94:0/1943433824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b1458963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d81225640 1 -- 172.21.15.94:0/1943433824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458876f0 msgr2=0x55b145896910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d81225640 1 --2- 172.21.15.94:0/1943433824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458876f0 0x55b145896910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.276+0000 7f1d81225640 1 -- 172.21.15.94:0/1943433824 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d680097e0 con 0x55b1458902f0 2024-02-17T11:59:37.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d81225640 1 --2- 172.21.15.94:0/1943433824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458902f0 0x55b1458805d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400cba0 tx=0x7f1d7409bad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffd8899b43063205 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:37.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:59:37.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d5bfff640 1 -- 172.21.15.94:0/1943433824 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74052150 con 0x55b1458902f0 2024-02-17T11:59:37.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d5bfff640 1 -- 172.21.15.94:0/1943433824 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1d740cc3a0 con 0x55b1458902f0 2024-02-17T11:59:37.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d85866740 1 -- 172.21.15.94:0/1943433824 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b14580b900 con 0x55b1458902f0 2024-02-17T11:59:37.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d5bfff640 1 -- 172.21.15.94:0/1943433824 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d740cc560 con 0x55b1458902f0 2024-02-17T11:59:37.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d85866740 10 client.25400 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:37.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d85866740 1 -- 172.21.15.94:0/1943433824 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55b1457a6910 con 0x55b1458902f0 2024-02-17T11:59:37.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d5bfff640 1 -- 172.21.15.94:0/1943433824 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b14580b900 con 0x55b1458902f0 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d5bfff640 1 -- 172.21.15.94:0/1943433824 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1d74050b40 con 0x55b1458902f0 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d85866740 10 client.25400 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d85866740 10 client.25400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d85866740 20 client.25400 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d85866740 1 --2- 172.21.15.94:0/1943433824 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b145884790 0x55b145903880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d85866740 4 client.25400 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.277+0000 7f1d85866740 1 -- 172.21.15.94:0/1943433824 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b1454e5860 con 0x55b145884790 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.278+0000 7f1d80a24640 1 --2- 172.21.15.94:0/1943433824 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b145884790 0x55b145903880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.278+0000 7f1d5bfff640 10 client.25400 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.279+0000 7f1d80a24640 1 --2- 172.21.15.94:0/1943433824 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b145884790 0x55b145903880 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7344782cdb4b827d server_cookie=32c7d0ebedabdf9b in_seq=0 out_seq=0 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d5bfff640 1 -- 172.21.15.94:0/1943433824 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b1454e5860 con 0x55b145884790 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d5bfff640 10 client.25400 handle_command_reply: tid=0 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d85866740 1 client.25400 shutdown 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d85866740 1 -- 172.21.15.94:0/1943433824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458902f0 msgr2=0x55b1458805d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d85866740 1 --2- 172.21.15.94:0/1943433824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458902f0 0x55b1458805d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400cba0 tx=0x7f1d7409bad0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d85866740 1 -- 172.21.15.94:0/1943433824 shutdown_connections 2024-02-17T11:59:37.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d85866740 1 --2- 172.21.15.94:0/1943433824 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b145884790 0x55b145903880 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d85866740 1 --2- 172.21.15.94:0/1943433824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b1458902f0 0x55b1458805d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d85866740 1 --2- 172.21.15.94:0/1943433824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b1458876f0 0x55b145896910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d85866740 1 --2- 172.21.15.94:0/1943433824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b145886d40 0x55b1458963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d85866740 1 -- 172.21.15.94:0/1943433824 >> 172.21.15.94:0/1943433824 conn(0x55b14587b1d0 msgr2=0x55b145877050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:37.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d5bfff640 0 client.25400 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:37.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d85866740 1 -- 172.21.15.94:0/1943433824 shutdown_connections 2024-02-17T11:59:37.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.280+0000 7f1d85866740 1 -- 172.21.15.94:0/1943433824 wait complete. 2024-02-17T11:59:37.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.281+0000 7f1d85866740 20 client.25400 trim_cache size 0 max 16384 2024-02-17T11:59:37.282 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:59:37.282 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T11:59:37.283 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "2fea7fc2-729f-466f-a345-de27adf0a057", 2024-02-17T11:59:37.283 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T11:59:37.283 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:59:37.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.282+0000 7f1d82c88640 1 -- 172.21.15.94:0/2229058225 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1d6c0a23d0 msgr2=0x7f1d6c0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.282+0000 7f1d82c88640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1d6c0a23d0 0x7f1d6c0a4890 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009fd0 tx=0x7f1d68009d60 comp rx=0 tx=0).stop 2024-02-17T11:59:37.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.282+0000 7f1d82c88640 1 -- 172.21.15.94:0/2229058225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d7c1142d0 msgr2=0x7f1d7c1a4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.282+0000 7f1d82c88640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d7c1142d0 0x7f1d7c1a4910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400ec80 tx=0x7f1d7400c6a0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.282+0000 7f1d82c88640 1 -- 172.21.15.94:0/2229058225 shutdown_connections 2024-02-17T11:59:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.282+0000 7f1d82c88640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1d6c0a23d0 0x7f1d6c0a4890 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.282+0000 7f1d82c88640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d7c1142d0 0x7f1d7c1a4910 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.282+0000 7f1d82c88640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d7c108aa0 0x7f1d7c1a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.282+0000 7f1d82c88640 1 --2- 172.21.15.94:0/2229058225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d7c108180 0x7f1d7c1a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.282+0000 7f1d82c88640 1 -- 172.21.15.94:0/2229058225 >> 172.21.15.94:0/2229058225 conn(0x7f1d7c06c6c0 msgr2=0x7f1d7c06fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.282+0000 7f1d82c88640 1 -- 172.21.15.94:0/2229058225 shutdown_connections 2024-02-17T11:59:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.282+0000 7f1d82c88640 1 -- 172.21.15.94:0/2229058225 wait complete. 2024-02-17T11:59:37.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T11:59:37.297 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T11:59:37.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.461+0000 7f4a05288640 1 -- 172.21.15.94:0/2408286319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a001060b0 msgr2=0x7f4a000ffcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.461+0000 7f4a05288640 1 --2- 172.21.15.94:0/2408286319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a001060b0 0x7f4a000ffcb0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f49f8009ca0 tx=0x7f49f802c140 comp rx=0 tx=0).stop 2024-02-17T11:59:37.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.461+0000 7f4a05288640 1 -- 172.21.15.94:0/2408286319 shutdown_connections 2024-02-17T11:59:37.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.461+0000 7f4a05288640 1 --2- 172.21.15.94:0/2408286319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a001060b0 0x7f4a000ffcb0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.462+0000 7f4a05288640 1 --2- 172.21.15.94:0/2408286319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a00105c30 0x7f4a000ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.462+0000 7f4a05288640 1 --2- 172.21.15.94:0/2408286319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a00067ae0 0x7f4a001056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.462+0000 7f4a05288640 1 -- 172.21.15.94:0/2408286319 >> 172.21.15.94:0/2408286319 conn(0x7f4a000765b0 msgr2=0x7f4a000fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:37.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.462+0000 7f4a05288640 1 -- 172.21.15.94:0/2408286319 shutdown_connections 2024-02-17T11:59:37.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.462+0000 7f4a05288640 1 -- 172.21.15.94:0/2408286319 wait complete. 2024-02-17T11:59:37.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.462+0000 7f4a05288640 1 Processor -- start 2024-02-17T11:59:37.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.463+0000 7f4a05288640 1 -- start start 2024-02-17T11:59:37.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.463+0000 7f4a05288640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a00067ae0 0x7f4a0019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.463+0000 7f4a05288640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a00105c30 0x7f4a0019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.463+0000 7f4a05288640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a001060b0 0x7f4a00199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.463+0000 7f4a05288640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00199f00 con 0x7f4a00067ae0 2024-02-17T11:59:37.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.463+0000 7f4a05288640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0019a070 con 0x7f4a00105c30 2024-02-17T11:59:37.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.463+0000 7f4a05288640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0019a230 con 0x7f4a001060b0 2024-02-17T11:59:37.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.464+0000 7f49ff7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a001060b0 0x7f4a00199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.464+0000 7f49ff7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a001060b0 0x7f4a00199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33366/0 (socket says 172.21.15.94:33366) 2024-02-17T11:59:37.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.464+0000 7f49ff7fe640 1 -- 172.21.15.94:0/1014156967 learned_addr learned my addr 172.21.15.94:0/1014156967 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:37.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.464+0000 7f49feffd640 1 --2- 172.21.15.94:0/1014156967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a00067ae0 0x7f4a0019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.464+0000 7f49ff7fe640 1 -- 172.21.15.94:0/1014156967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a00105c30 msgr2=0x7f4a0019fbf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-17T11:59:37.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.464+0000 7f49ff7fe640 1 --2- 172.21.15.94:0/1014156967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a00105c30 0x7f4a0019fbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.464+0000 7f49ff7fe640 1 -- 172.21.15.94:0/1014156967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a00067ae0 msgr2=0x7f4a0019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.464+0000 7f49ff7fe640 1 --2- 172.21.15.94:0/1014156967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a00067ae0 0x7f4a0019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.464+0000 7f49ff7fe640 1 -- 172.21.15.94:0/1014156967 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f0009490 con 0x7f4a001060b0 2024-02-17T11:59:37.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.464+0000 7f49ff7fe640 1 --2- 172.21.15.94:0/1014156967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a001060b0 0x7f4a00199770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f49f8005e30 tx=0x7f49f80325e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:37.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.465+0000 7f49fcff9640 1 -- 172.21.15.94:0/1014156967 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8034e20 con 0x7f4a001060b0 2024-02-17T11:59:37.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.465+0000 7f4a05288640 1 -- 172.21.15.94:0/1014156967 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e8009490 con 0x7f4a001060b0 2024-02-17T11:59:37.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.465+0000 7f49fcff9640 1 -- 172.21.15.94:0/1014156967 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49f8003e50 con 0x7f4a001060b0 2024-02-17T11:59:37.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.465+0000 7f49fcff9640 1 -- 172.21.15.94:0/1014156967 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f80047c0 con 0x7f4a001060b0 2024-02-17T11:59:37.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.465+0000 7f4a05288640 1 -- 172.21.15.94:0/1014156967 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f8009900 con 0x7f4a001060b0 2024-02-17T11:59:37.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.468+0000 7f49fcff9640 1 -- 172.21.15.94:0/1014156967 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 26) v1 ==== 137001+0+0 (secure 0 0 0) 0x7f49f800a040 con 0x7f4a001060b0 2024-02-17T11:59:37.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.468+0000 7f4a07e68740 1 Processor -- start 2024-02-17T11:59:37.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.468+0000 7f4a07e68740 1 -- start start 2024-02-17T11:59:37.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.469+0000 7f4a07e68740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cd0c0 0x55ad216cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.469+0000 7f4a07e68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 0x55ad216dea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.469+0000 7f4a07e68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216defc0 0x55ad216e13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.469+0000 7f4a07e68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad2163c280 con 0x55ad216cda70 2024-02-17T11:59:37.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.469+0000 7f4a07e68740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad216397b0 con 0x55ad216cd0c0 2024-02-17T11:59:37.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.469+0000 7f4a07e68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21634f90 con 0x55ad216defc0 2024-02-17T11:59:37.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.469+0000 7f49feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 0x55ad216dea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.469+0000 7f49feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 0x55ad216dea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39926/0 (socket says 172.21.15.94:39926) 2024-02-17T11:59:37.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.469+0000 7f49feffd640 1 -- 172.21.15.94:0/2707308649 learned_addr learned my addr 172.21.15.94:0/2707308649 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:37.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.470+0000 7f49f7fff640 1 --2- 172.21.15.94:0/2707308649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cd0c0 0x55ad216cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.470+0000 7f49feffd640 1 -- 172.21.15.94:0/2707308649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216defc0 msgr2=0x55ad216e13b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T11:59:37.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.470+0000 7f49feffd640 1 --2- 172.21.15.94:0/2707308649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216defc0 0x55ad216e13b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.470+0000 7f49feffd640 1 -- 172.21.15.94:0/2707308649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cd0c0 msgr2=0x55ad216cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.470+0000 7f49feffd640 1 --2- 172.21.15.94:0/2707308649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cd0c0 0x55ad216cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.470+0000 7f49feffd640 1 -- 172.21.15.94:0/2707308649 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad215f3570 con 0x55ad216cda70 2024-02-17T11:59:37.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.470+0000 7f49f7fff640 1 --2- 172.21.15.94:0/2707308649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cd0c0 0x55ad216cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:59:37.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.470+0000 7f49feffd640 1 --2- 172.21.15.94:0/2707308649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 0x55ad216dea80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009e80 tx=0x7f49f0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c684cc5d56fa0600 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:37.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.470+0000 7f49fcff9640 1 --2- 172.21.15.94:0/1014156967 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f49ec0a2350 0x7f49ec0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.470+0000 7f49f57fa640 1 -- 172.21.15.94:0/2707308649 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0019410 con 0x55ad216cda70 2024-02-17T11:59:37.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.470+0000 7f49f57fa640 1 -- 172.21.15.94:0/2707308649 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49f0019a50 con 0x55ad216cda70 2024-02-17T11:59:37.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.470+0000 7f49f57fa640 1 -- 172.21.15.94:0/2707308649 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0018c00 con 0x55ad216cda70 2024-02-17T11:59:37.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.471+0000 7f49f7fff640 1 --2- 172.21.15.94:0/1014156967 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f49ec0a2350 0x7f49ec0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.471+0000 7f49fcff9640 1 -- 172.21.15.94:0/1014156967 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f49f80f0ea0 con 0x7f4a001060b0 2024-02-17T11:59:37.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.471+0000 7f4a07e68740 1 -- 172.21.15.94:0/2707308649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 msgr2=0x55ad216dea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.471+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2707308649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 0x55ad216dea80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009e80 tx=0x7f49f0011810 comp rx=0 tx=0).stop 2024-02-17T11:59:37.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.471+0000 7f49f7fff640 1 --2- 172.21.15.94:0/1014156967 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f49ec0a2350 0x7f49ec0a4810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55ad216c8e00 tx=0x7f49e8005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:37.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.472+0000 7f4a07e68740 1 -- 172.21.15.94:0/2707308649 shutdown_connections 2024-02-17T11:59:37.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.472+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2707308649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216defc0 0x55ad216e13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.472+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2707308649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 0x55ad216dea80 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.472+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2707308649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cd0c0 0x55ad216cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.472+0000 7f4a07e68740 1 -- 172.21.15.94:0/2707308649 >> 172.21.15.94:0/2707308649 conn(0x55ad216c7990 msgr2=0x55ad216c7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:37.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.472+0000 7f4a07e68740 1 -- 172.21.15.94:0/2707308649 shutdown_connections 2024-02-17T11:59:37.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.472+0000 7f4a07e68740 1 -- 172.21.15.94:0/2707308649 wait complete. 2024-02-17T11:59:37.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.472+0000 7f4a07e68740 1 -- start start 2024-02-17T11:59:37.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.472+0000 7f4a07e68740 1 Processor -- start 2024-02-17T11:59:37.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.473+0000 7f4a07e68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216cd0c0 0x55ad216dea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.473+0000 7f4a07e68740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cda70 0x55ad216dbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.473+0000 7f4a07e68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216defc0 0x55ad216dc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.473+0000 7f4a07e68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21308430 con 0x55ad216defc0 2024-02-17T11:59:37.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.473+0000 7f4a07e68740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad212c4e00 con 0x55ad216cda70 2024-02-17T11:59:37.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.473+0000 7f4a07e68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad215b9fd0 con 0x55ad216cd0c0 2024-02-17T11:59:37.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.473+0000 7f4a07e68740 10 client.0 _ll_register_callbacks cb 0x55ad2160ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:37.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.473+0000 7f49feffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216cd0c0 0x55ad216dea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.474+0000 7f49feffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216cd0c0 0x55ad216dea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33374/0 (socket says 172.21.15.94:33374) 2024-02-17T11:59:37.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.474+0000 7f49feffd640 1 -- 172.21.15.94:0/1545775625 learned_addr learned my addr 172.21.15.94:0/1545775625 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:37.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.474+0000 7f49ff7fe640 1 --2- 172.21.15.94:0/1545775625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216defc0 0x55ad216dc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.474+0000 7f49feffd640 1 -- 172.21.15.94:0/1545775625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cda70 msgr2=0x55ad216dbde0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T11:59:37.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.474+0000 7f49feffd640 1 --2- 172.21.15.94:0/1545775625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cda70 0x55ad216dbde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.474+0000 7f49feffd640 1 -- 172.21.15.94:0/1545775625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216defc0 msgr2=0x55ad216dc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.474+0000 7f49feffd640 1 --2- 172.21.15.94:0/1545775625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216defc0 0x55ad216dc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.474+0000 7f49feffd640 1 -- 172.21.15.94:0/1545775625 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f0009ae0 con 0x55ad216cd0c0 2024-02-17T11:59:37.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.474+0000 7f49feffd640 1 --2- 172.21.15.94:0/1545775625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216cd0c0 0x55ad216dea50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f49f000a010 tx=0x7f49f0019640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b8ececd0b1c12c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:37.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.474+0000 7f49e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T11:59:37.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.475+0000 7f49e67fc640 1 -- 172.21.15.94:0/1545775625 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f00183b0 con 0x55ad216cd0c0 2024-02-17T11:59:37.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.475+0000 7f49e67fc640 1 -- 172.21.15.94:0/1545775625 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f49f0004020 con 0x55ad216cd0c0 2024-02-17T11:59:37.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.475+0000 7f4a07e68740 1 -- 172.21.15.94:0/1545775625 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad213347e0 con 0x55ad216cd0c0 2024-02-17T11:59:37.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.475+0000 7f49e67fc640 1 -- 172.21.15.94:0/1545775625 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49f00041e0 con 0x55ad216cd0c0 2024-02-17T11:59:37.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.475+0000 7f4a07e68740 10 client.25456 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:37.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.475+0000 7f4a07e68740 1 -- 172.21.15.94:0/1545775625 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55ad21633610 con 0x55ad216cd0c0 2024-02-17T11:59:37.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.476+0000 7f49e67fc640 1 -- 172.21.15.94:0/1545775625 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ad213347e0 con 0x55ad216cd0c0 2024-02-17T11:59:37.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.476+0000 7f49e67fc640 1 -- 172.21.15.94:0/1545775625 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f49f00054b0 con 0x55ad216cd0c0 2024-02-17T11:59:37.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.476+0000 7f4a07e68740 10 client.25456 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:37.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.476+0000 7f4a07e68740 10 client.25456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:37.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.476+0000 7f4a07e68740 20 client.25456 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:37.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.476+0000 7f4a07e68740 1 --2- 172.21.15.94:0/1545775625 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ad21700020 0x55ad21702440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.476+0000 7f4a07e68740 4 client.25456 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:37.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.476+0000 7f4a07e68740 1 -- 172.21.15.94:0/1545775625 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad21371720 con 0x55ad21700020 2024-02-17T11:59:37.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.476+0000 7f49f7fff640 1 --2- 172.21.15.94:0/1545775625 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ad21700020 0x55ad21702440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.477+0000 7f49e67fc640 10 client.25456 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:37.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.477+0000 7f49f7fff640 1 --2- 172.21.15.94:0/1545775625 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ad21700020 0x55ad21702440 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2220e56032e3140e server_cookie=6b890fd338b9428a in_seq=0 out_seq=0 2024-02-17T11:59:37.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.478+0000 7f49e67fc640 1 -- 172.21.15.94:0/1545775625 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad21371720 con 0x55ad21700020 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.478+0000 7f49e67fc640 10 client.25456 handle_command_reply: tid=0 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.478+0000 7f4a07e68740 1 client.25456 shutdown 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.479+0000 7f4a07e68740 1 -- 172.21.15.94:0/1545775625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216cd0c0 msgr2=0x55ad216dea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.479+0000 7f4a07e68740 1 --2- 172.21.15.94:0/1545775625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216cd0c0 0x55ad216dea50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f49f000a010 tx=0x7f49f0019640 comp rx=0 tx=0).stop 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.479+0000 7f4a07e68740 1 -- 172.21.15.94:0/1545775625 shutdown_connections 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.479+0000 7f4a07e68740 1 --2- 172.21.15.94:0/1545775625 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ad21700020 0x55ad21702440 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.479+0000 7f4a07e68740 1 --2- 172.21.15.94:0/1545775625 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216defc0 0x55ad216dc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.479+0000 7f4a07e68740 1 --2- 172.21.15.94:0/1545775625 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cda70 0x55ad216dbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.479+0000 7f4a07e68740 1 --2- 172.21.15.94:0/1545775625 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216cd0c0 0x55ad216dea50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.479+0000 7f4a07e68740 1 -- 172.21.15.94:0/1545775625 >> 172.21.15.94:0/1545775625 conn(0x55ad216c7990 msgr2=0x55ad216c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.479+0000 7f49e67fc640 0 client.25456 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.479+0000 7f4a07e68740 1 -- 172.21.15.94:0/1545775625 shutdown_connections 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.479+0000 7f4a07e68740 1 -- 172.21.15.94:0/1545775625 wait complete. 2024-02-17T11:59:37.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.479+0000 7f4a07e68740 20 client.25456 trim_cache size 0 max 16384 2024-02-17T11:59:37.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.504+0000 7f4a07e68740 1 Processor -- start 2024-02-17T11:59:37.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.504+0000 7f4a07e68740 1 -- start start 2024-02-17T11:59:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.504+0000 7f4a07e68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cd0c0 0x55ad216de510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.504+0000 7f4a07e68740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cda70 0x55ad216dea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.504+0000 7f4a07e68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216defc0 0x55ad216d70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.504+0000 7f49f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cda70 0x55ad216dea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.504+0000 7f49feffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cd0c0 0x55ad216de510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.504+0000 7f49f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cda70 0x55ad216dea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44684/0 (socket says 172.21.15.94:44684) 2024-02-17T11:59:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.504+0000 7f49f7fff640 1 -- 172.21.15.94:0/2987852524 learned_addr learned my addr 172.21.15.94:0/2987852524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.504+0000 7f4a07e68740 1 -- 172.21.15.94:0/2987852524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad216c30d0 con 0x55ad216cd0c0 2024-02-17T11:59:37.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.504+0000 7f4a07e68740 1 -- 172.21.15.94:0/2987852524 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad216eadc0 con 0x55ad216cda70 2024-02-17T11:59:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.504+0000 7f4a07e68740 1 -- 172.21.15.94:0/2987852524 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad216c8790 con 0x55ad216defc0 2024-02-17T11:59:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.505+0000 7f49ff7fe640 1 --2- 172.21.15.94:0/2987852524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216defc0 0x55ad216d70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.505+0000 7f49ff7fe640 1 -- 172.21.15.94:0/2987852524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cda70 msgr2=0x55ad216dea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.505+0000 7f49ff7fe640 1 --2- 172.21.15.94:0/2987852524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cda70 0x55ad216dea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.505+0000 7f49ff7fe640 1 -- 172.21.15.94:0/2987852524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cd0c0 msgr2=0x55ad216de510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.505+0000 7f49ff7fe640 1 --2- 172.21.15.94:0/2987852524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cd0c0 0x55ad216de510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.505+0000 7f49ff7fe640 1 -- 172.21.15.94:0/2987852524 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad215cfac0 con 0x55ad216defc0 2024-02-17T11:59:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.505+0000 7f49ff7fe640 1 --2- 172.21.15.94:0/2987852524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216defc0 0x55ad216d70c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f49f80ee570 tx=0x7f49f80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b2c3ba1bd4ee96b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.505+0000 7f49f4ff9640 1 -- 172.21.15.94:0/2987852524 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f80bde40 con 0x55ad216defc0 2024-02-17T11:59:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.505+0000 7f49f4ff9640 1 -- 172.21.15.94:0/2987852524 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49f80bd740 con 0x55ad216defc0 2024-02-17T11:59:37.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.505+0000 7f49f4ff9640 1 -- 172.21.15.94:0/2987852524 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f80a8170 con 0x55ad216defc0 2024-02-17T11:59:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.506+0000 7f4a07e68740 1 -- 172.21.15.94:0/2987852524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216defc0 msgr2=0x55ad216d70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.506+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2987852524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216defc0 0x55ad216d70c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f49f80ee570 tx=0x7f49f80095c0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.506+0000 7f4a07e68740 1 -- 172.21.15.94:0/2987852524 shutdown_connections 2024-02-17T11:59:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.506+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2987852524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216defc0 0x55ad216d70c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.506+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2987852524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216cda70 0x55ad216dea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.506+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2987852524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cd0c0 0x55ad216de510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.506+0000 7f4a07e68740 1 -- 172.21.15.94:0/2987852524 >> 172.21.15.94:0/2987852524 conn(0x55ad216c7990 msgr2=0x55ad21706de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.506+0000 7f4a07e68740 1 -- 172.21.15.94:0/2987852524 shutdown_connections 2024-02-17T11:59:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.506+0000 7f4a07e68740 1 -- 172.21.15.94:0/2987852524 wait complete. 2024-02-17T11:59:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f4a07e68740 1 -- start start 2024-02-17T11:59:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f4a07e68740 1 Processor -- start 2024-02-17T11:59:37.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f4a07e68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216cd0c0 0x55ad216e61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f4a07e68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 0x55ad216e2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f4a07e68740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216defc0 0x55ad216e2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f4a07e68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad216e31a0 con 0x55ad216cda70 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f4a07e68740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad216e3310 con 0x55ad216defc0 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f4a07e68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad21756560 con 0x55ad216cd0c0 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f49f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 0x55ad216e2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f49f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 0x55ad216e2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39968/0 (socket says 172.21.15.94:39968) 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f49f7fff640 1 -- 172.21.15.94:0/2149629817 learned_addr learned my addr 172.21.15.94:0/2149629817 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f4a07e68740 10 client.0 _ll_register_callbacks cb 0x7f49f0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f49feffd640 1 --2- 172.21.15.94:0/2149629817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216cd0c0 0x55ad216e61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.507+0000 7f49ff7fe640 1 --2- 172.21.15.94:0/2149629817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216defc0 0x55ad216e2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f49f7fff640 1 -- 172.21.15.94:0/2149629817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216cd0c0 msgr2=0x55ad216e61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f49f7fff640 1 --2- 172.21.15.94:0/2149629817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216cd0c0 0x55ad216e61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f49f7fff640 1 -- 172.21.15.94:0/2149629817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216defc0 msgr2=0x55ad216e2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:37.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f49f7fff640 1 --2- 172.21.15.94:0/2149629817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216defc0 0x55ad216e2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f49f7fff640 1 -- 172.21.15.94:0/2149629817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f8009900 con 0x55ad216cda70 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f49f7fff640 1 --2- 172.21.15.94:0/2149629817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 0x55ad216e2560 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4a0019afc0 tx=0x7f49e8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fac888dd1e8345 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f49e67fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f49e67fc640 1 -- 172.21.15.94:0/2149629817 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e80257c0 con 0x55ad216cda70 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f49e67fc640 1 -- 172.21.15.94:0/2149629817 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f49e8004030 con 0x55ad216cda70 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f4a07e68740 1 -- 172.21.15.94:0/2149629817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49f0019a50 con 0x55ad216cda70 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f49e67fc640 1 -- 172.21.15.94:0/2149629817 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49e8025920 con 0x55ad216cda70 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f4a07e68740 10 client.15948 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f4a07e68740 1 -- 172.21.15.94:0/2149629817 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55ad215f56a0 con 0x55ad216cda70 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f49e67fc640 1 -- 172.21.15.94:0/2149629817 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f49e80041f0 con 0x55ad216cda70 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f49e67fc640 1 -- 172.21.15.94:0/2149629817 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f49e802e600 con 0x55ad216cda70 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f4a07e68740 10 client.15948 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f4a07e68740 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:37.512 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f4a07e68740 20 client.15948 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2149629817 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ad21757cf0 0x55ad2175a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.508+0000 7f4a07e68740 4 client.15948 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f4a07e68740 1 -- 172.21.15.94:0/2149629817 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f49f0019a50 con 0x55ad21757cf0 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f49feffd640 1 --2- 172.21.15.94:0/2149629817 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ad21757cf0 0x55ad2175a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f49feffd640 1 --2- 172.21.15.94:0/2149629817 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ad21757cf0 0x55ad2175a110 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=850707070ca2f105 server_cookie=f0b178fde43abc73 in_seq=0 out_seq=0 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f49e67fc640 10 client.15948 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f49e67fc640 1 -- 172.21.15.94:0/2149629817 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f49f0019a50 con 0x55ad21757cf0 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f49e67fc640 10 client.15948 handle_command_reply: tid=0 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f4a07e68740 1 client.15948 shutdown 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f4a07e68740 1 -- 172.21.15.94:0/2149629817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 msgr2=0x55ad216e2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2149629817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 0x55ad216e2560 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f4a0019afc0 tx=0x7f49e8016000 comp rx=0 tx=0).stop 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f4a07e68740 1 -- 172.21.15.94:0/2149629817 shutdown_connections 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2149629817 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ad21757cf0 0x55ad2175a110 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f49e67fc640 0 client.15948 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:37.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2149629817 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ad216defc0 0x55ad216e2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2149629817 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ad216cda70 0x55ad216e2560 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f4a07e68740 1 --2- 172.21.15.94:0/2149629817 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad216cd0c0 0x55ad216e61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f4a07e68740 1 -- 172.21.15.94:0/2149629817 >> 172.21.15.94:0/2149629817 conn(0x55ad216c7990 msgr2=0x55ad216d70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:37.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f4a07e68740 1 -- 172.21.15.94:0/2149629817 shutdown_connections 2024-02-17T11:59:37.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f4a07e68740 1 -- 172.21.15.94:0/2149629817 wait complete. 2024-02-17T11:59:37.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.509+0000 7f4a07e68740 20 client.15948 trim_cache size 0 max 16384 2024-02-17T11:59:37.514 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T11:59:37.514 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T11:59:37.514 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T11:59:37.514 INFO:teuthology.orchestra.run.smithi094.stdout: "2fea7fc2-729f-466f-a345-de27adf0a057": { 2024-02-17T11:59:37.514 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T11:59:37.514 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "2fea7fc2-729f-466f-a345-de27adf0a057", 2024-02-17T11:59:37.515 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T11:59:37.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T11:59:37.515 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T11:59:37.515 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T11:59:37.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.510+0000 7f4a05288640 1 -- 172.21.15.94:0/1014156967 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f49ec0a2350 msgr2=0x7f49ec0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.511+0000 7f4a05288640 1 --2- 172.21.15.94:0/1014156967 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f49ec0a2350 0x7f49ec0a4810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55ad216c8e00 tx=0x7f49e8005f90 comp rx=0 tx=0).stop 2024-02-17T11:59:37.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.511+0000 7f4a05288640 1 -- 172.21.15.94:0/1014156967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a001060b0 msgr2=0x7f4a00199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:37.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.511+0000 7f4a05288640 1 --2- 172.21.15.94:0/1014156967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a001060b0 0x7f4a00199770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f49f8005e30 tx=0x7f49f80325e0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.511+0000 7f4a05288640 1 -- 172.21.15.94:0/1014156967 shutdown_connections 2024-02-17T11:59:37.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.511+0000 7f4a05288640 1 --2- 172.21.15.94:0/1014156967 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f49ec0a2350 0x7f49ec0a4810 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.511+0000 7f4a05288640 1 --2- 172.21.15.94:0/1014156967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a001060b0 0x7f4a00199770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.511+0000 7f4a05288640 1 --2- 172.21.15.94:0/1014156967 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a00105c30 0x7f4a0019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.511+0000 7f4a05288640 1 --2- 172.21.15.94:0/1014156967 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a00067ae0 0x7f4a0019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:37.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.511+0000 7f4a05288640 1 -- 172.21.15.94:0/1014156967 >> 172.21.15.94:0/1014156967 conn(0x7f4a000765b0 msgr2=0x7f4a00101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:37.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.511+0000 7f4a05288640 1 -- 172.21.15.94:0/1014156967 shutdown_connections 2024-02-17T11:59:37.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T11:59:37.511+0000 7f4a05288640 1 -- 172.21.15.94:0/1014156967 wait complete. 2024-02-17T11:59:37.525 INFO:tasks.cephfs.filesystem:scrub status for tag:2fea7fc2-729f-466f-a345-de27adf0a057 - {'path': '/', 'tag': '2fea7fc2-729f-466f-a345-de27adf0a057', 'options': 'recursive,force'} 2024-02-17T11:59:38.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:37 smithi189 ceph-mon[36588]: pgmap v48: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 659 op/s 2024-02-17T11:59:38.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:37 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T11:59:38.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:37 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T11:59:38.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:37 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T11:59:38.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:37 smithi125 ceph-mon[36819]: pgmap v48: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 659 op/s 2024-02-17T11:59:38.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:37 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T11:59:38.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:37 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T11:59:38.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:37 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T11:59:38.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:37 smithi094 ceph-mon[32499]: pgmap v48: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 8.9 MiB/s wr, 659 op/s 2024-02-17T11:59:38.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:37 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T11:59:38.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:37 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T11:59:38.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:37 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T11:59:39.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:38 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T11:59:39.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:38 smithi189 ceph-mon[36588]: mgrmap e27: x(active, since 91s), standbys: y, z 2024-02-17T11:59:39.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:38 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T11:59:39.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:38 smithi125 ceph-mon[36819]: mgrmap e27: x(active, since 91s), standbys: y, z 2024-02-17T11:59:39.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:38 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T11:59:39.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:38 smithi094 ceph-mon[32499]: mgrmap e27: x(active, since 91s), standbys: y, z 2024-02-17T11:59:40.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:39 smithi189 ceph-mon[36588]: pgmap v49: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-02-17T11:59:40.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:39 smithi125 ceph-mon[36819]: pgmap v49: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-02-17T11:59:40.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:39 smithi094 ceph-mon[32499]: pgmap v49: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 12 MiB/s wr, 1.07k op/s 2024-02-17T11:59:40.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 80.0 s, 3580.0 tps, lat 27.959 ms stddev 28.819 2024-02-17T11:59:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.892+0000 7fc702249640 1 -- 172.21.15.94:0/1189385511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6fc108cf0 msgr2=0x7fc6fc113d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:41.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.892+0000 7fc702249640 1 --2- 172.21.15.94:0/1189385511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6fc108cf0 0x7fc6fc113d90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009930 tx=0x7fc6e4030040 comp rx=0 tx=0).stop 2024-02-17T11:59:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.893+0000 7fc702249640 1 -- 172.21.15.94:0/1189385511 shutdown_connections 2024-02-17T11:59:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.893+0000 7fc702249640 1 --2- 172.21.15.94:0/1189385511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6fc1142d0 0x7fc6fc1166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.893+0000 7fc702249640 1 --2- 172.21.15.94:0/1189385511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6fc108cf0 0x7fc6fc113d90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.893+0000 7fc702249640 1 --2- 172.21.15.94:0/1189385511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6fc1083d0 0x7fc6fc1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.893+0000 7fc702249640 1 -- 172.21.15.94:0/1189385511 >> 172.21.15.94:0/1189385511 conn(0x7fc6fc06c220 msgr2=0x7fc6fc06c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.893+0000 7fc702249640 1 -- 172.21.15.94:0/1189385511 shutdown_connections 2024-02-17T11:59:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.893+0000 7fc702249640 1 -- 172.21.15.94:0/1189385511 wait complete. 2024-02-17T11:59:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.894+0000 7fc702249640 1 Processor -- start 2024-02-17T11:59:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.894+0000 7fc702249640 1 -- start start 2024-02-17T11:59:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.895+0000 7fc702249640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6fc1083d0 0x7fc6fc1a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.895+0000 7fc702249640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6fc108cf0 0x7fc6fc1a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.895+0000 7fc702249640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6fc1142d0 0x7fc6fc19db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.895+0000 7fc6fb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6fc1083d0 0x7fc6fc1a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.895+0000 7fc6fb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6fc1083d0 0x7fc6fc1a3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44714/0 (socket says 172.21.15.94:44714) 2024-02-17T11:59:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.895+0000 7fc702249640 1 -- 172.21.15.94:0/1973922748 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc19e170 con 0x7fc6fc108cf0 2024-02-17T11:59:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.895+0000 7fc702249640 1 -- 172.21.15.94:0/1973922748 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc19e2e0 con 0x7fc6fc1083d0 2024-02-17T11:59:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.895+0000 7fc702249640 1 -- 172.21.15.94:0/1973922748 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc19e4a0 con 0x7fc6fc1142d0 2024-02-17T11:59:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.895+0000 7fc6fb7fe640 1 -- 172.21.15.94:0/1973922748 learned_addr learned my addr 172.21.15.94:0/1973922748 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.895+0000 7fc6f3fff640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6fc108cf0 0x7fc6fc1a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.895+0000 7fc6fbfff640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6fc1142d0 0x7fc6fc19db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.895+0000 7fc6fbfff640 1 -- 172.21.15.94:0/1973922748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6fc1083d0 msgr2=0x7fc6fc1a3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.896+0000 7fc6fbfff640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6fc1083d0 0x7fc6fc1a3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.896+0000 7fc6fbfff640 1 -- 172.21.15.94:0/1973922748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6fc108cf0 msgr2=0x7fc6fc1a3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.896+0000 7fc6fbfff640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6fc108cf0 0x7fc6fc1a3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.896+0000 7fc6fbfff640 1 -- 172.21.15.94:0/1973922748 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec009490 con 0x7fc6fc1142d0 2024-02-17T11:59:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.896+0000 7fc6f3fff640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6fc108cf0 0x7fc6fc1a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.896+0000 7fc6fbfff640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6fc1142d0 0x7fc6fc19db20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009fd0 tx=0x7fc6f400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.896+0000 7fc6f97fa640 1 -- 172.21.15.94:0/1973922748 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f400ce20 con 0x7fc6fc1142d0 2024-02-17T11:59:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.896+0000 7fc6f97fa640 1 -- 172.21.15.94:0/1973922748 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6f4005020 con 0x7fc6fc1142d0 2024-02-17T11:59:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.896+0000 7fc6f97fa640 1 -- 172.21.15.94:0/1973922748 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4010690 con 0x7fc6fc1142d0 2024-02-17T11:59:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.896+0000 7fc702249640 1 -- 172.21.15.94:0/1973922748 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e4009560 con 0x7fc6fc1142d0 2024-02-17T11:59:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.896+0000 7fc702249640 1 -- 172.21.15.94:0/1973922748 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc1aa7e0 con 0x7fc6fc1142d0 2024-02-17T11:59:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.899+0000 7fc6f97fa640 1 -- 172.21.15.94:0/1973922748 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc6f4004b20 con 0x7fc6fc1142d0 2024-02-17T11:59:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc6f97fa640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6e80a2350 0x7fc6e80a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc704e17740 1 Processor -- start 2024-02-17T11:59:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc704e17740 1 -- start start 2024-02-17T11:59:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc704e17740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb1729ad0 0x55ddb1728820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc704e17740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb17291e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc704e17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb173aaf0 0x55ddb173cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc704e17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddb16979f0 con 0x55ddb173aaf0 2024-02-17T11:59:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc704e17740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddb1694f20 con 0x55ddb1729ad0 2024-02-17T11:59:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc704e17740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddb1690700 con 0x55ddb1728d60 2024-02-17T11:59:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc6f97fa640 1 -- 172.21.15.94:0/1973922748 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc6f4014070 con 0x7fc6fc1142d0 2024-02-17T11:59:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc6f3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb17291e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc6fb7fe640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6e80a2350 0x7fc6e80a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc6f3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb17291e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33406/0 (socket says 172.21.15.94:33406) 2024-02-17T11:59:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc6f3fff640 1 -- 172.21.15.94:0/2765959804 learned_addr learned my addr 172.21.15.94:0/2765959804 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc6fbfff640 1 --2- 172.21.15.94:0/2765959804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb173aaf0 0x55ddb173cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.900+0000 7fc6fb7fe640 1 --2- 172.21.15.94:0/2765959804 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb1729ad0 0x55ddb1728820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.901+0000 7fc6f3fff640 1 -- 172.21.15.94:0/2765959804 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb1729ad0 msgr2=0x55ddb1728820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.901+0000 7fc6f3fff640 1 --2- 172.21.15.94:0/2765959804 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb1729ad0 0x55ddb1728820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.901+0000 7fc6f3fff640 1 -- 172.21.15.94:0/2765959804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb173aaf0 msgr2=0x55ddb173cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.901+0000 7fc6f3fff640 1 --2- 172.21.15.94:0/2765959804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb173aaf0 0x55ddb173cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.901+0000 7fc6f3fff640 1 -- 172.21.15.94:0/2765959804 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddb16267b0 con 0x55ddb1728d60 2024-02-17T11:59:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.901+0000 7fc6f3fff640 1 --2- 172.21.15.94:0/2765959804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb17291e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4006010 tx=0x7fc6e4005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d1f4565ab8e037b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.901+0000 7fc6fb7fe640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6e80a2350 0x7fc6e80a4810 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0098e0 tx=0x7fc6ec009870 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.901+0000 7fc6f1ffb640 1 -- 172.21.15.94:0/2765959804 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4010430 con 0x55ddb1728d60 2024-02-17T11:59:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.901+0000 7fc6f1ffb640 1 -- 172.21.15.94:0/2765959804 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6e40309e0 con 0x55ddb1728d60 2024-02-17T11:59:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.901+0000 7fc6f1ffb640 1 -- 172.21.15.94:0/2765959804 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4005590 con 0x55ddb1728d60 2024-02-17T11:59:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.902+0000 7fc704e17740 1 -- 172.21.15.94:0/2765959804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 msgr2=0x55ddb17291e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.902+0000 7fc704e17740 1 --2- 172.21.15.94:0/2765959804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb17291e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4006010 tx=0x7fc6e4005bd0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.902+0000 7fc704e17740 1 -- 172.21.15.94:0/2765959804 shutdown_connections 2024-02-17T11:59:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.902+0000 7fc704e17740 1 --2- 172.21.15.94:0/2765959804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb173aaf0 0x55ddb173cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.902+0000 7fc704e17740 1 --2- 172.21.15.94:0/2765959804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb17291e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.902+0000 7fc704e17740 1 --2- 172.21.15.94:0/2765959804 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb1729ad0 0x55ddb1728820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.902+0000 7fc704e17740 1 -- 172.21.15.94:0/2765959804 >> 172.21.15.94:0/2765959804 conn(0x55ddb16f0550 msgr2=0x55ddb1723a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.902+0000 7fc704e17740 1 -- 172.21.15.94:0/2765959804 shutdown_connections 2024-02-17T11:59:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.902+0000 7fc704e17740 1 -- 172.21.15.94:0/2765959804 wait complete. 2024-02-17T11:59:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.903+0000 7fc704e17740 1 -- start start 2024-02-17T11:59:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.903+0000 7fc704e17740 1 Processor -- start 2024-02-17T11:59:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.904+0000 7fc704e17740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb1737b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.904+0000 7fc704e17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb1729ad0 0x55ddb17380b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.904+0000 7fc704e17740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb173aaf0 0x55ddb17385f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.904+0000 7fc704e17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddb1635bf0 con 0x55ddb1729ad0 2024-02-17T11:59:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.904+0000 7fc704e17740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddb1364250 con 0x55ddb173aaf0 2024-02-17T11:59:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.904+0000 7fc704e17740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddb1616200 con 0x55ddb1728d60 2024-02-17T11:59:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.904+0000 7fc704e17740 10 client.0 _ll_register_callbacks cb 0x55ddb1684d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.904+0000 7fc6fb7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb1737b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.904+0000 7fc6fb7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb1737b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33416/0 (socket says 172.21.15.94:33416) 2024-02-17T11:59:41.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.904+0000 7fc6fb7fe640 1 -- 172.21.15.94:0/2794856438 learned_addr learned my addr 172.21.15.94:0/2794856438 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.904+0000 7fc6f3fff640 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb1729ad0 0x55ddb17380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc6fbfff640 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb173aaf0 0x55ddb17385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc6fb7fe640 1 -- 172.21.15.94:0/2794856438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb173aaf0 msgr2=0x55ddb17385f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc6fb7fe640 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb173aaf0 0x55ddb17385f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc6fb7fe640 1 -- 172.21.15.94:0/2794856438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb1729ad0 msgr2=0x55ddb17380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc6fb7fe640 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb1729ad0 0x55ddb17380b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc6fb7fe640 1 -- 172.21.15.94:0/2794856438 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc117050 con 0x55ddb1728d60 2024-02-17T11:59:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc6fbfff640 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb173aaf0 0x55ddb17385f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T11:59:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc6fb7fe640 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb1737b70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec010000 tx=0x7fc6ec004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d982b9fb44f8e4cd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc6daffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T11:59:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc6daffd640 1 -- 172.21.15.94:0/2794856438 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec020860 con 0x55ddb1728d60 2024-02-17T11:59:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc704e17740 1 -- 172.21.15.94:0/2794856438 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ddb1390600 con 0x55ddb1728d60 2024-02-17T11:59:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc6daffd640 1 -- 172.21.15.94:0/2794856438 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc6ec0209c0 con 0x55ddb1728d60 2024-02-17T11:59:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc6daffd640 1 -- 172.21.15.94:0/2794856438 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6ec01feb0 con 0x55ddb1728d60 2024-02-17T11:59:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.905+0000 7fc704e17740 10 client.25474 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.906+0000 7fc704e17740 1 -- 172.21.15.94:0/2794856438 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55ddb168ed80 con 0x55ddb1728d60 2024-02-17T11:59:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.907+0000 7fc6daffd640 1 -- 172.21.15.94:0/2794856438 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ddb1390600 con 0x55ddb1728d60 2024-02-17T11:59:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.907+0000 7fc6daffd640 1 -- 172.21.15.94:0/2794856438 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc6ec028030 con 0x55ddb1728d60 2024-02-17T11:59:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.907+0000 7fc704e17740 10 client.25474 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.907+0000 7fc704e17740 10 client.25474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.907+0000 7fc704e17740 20 client.25474 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.907+0000 7fc704e17740 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ddb174af90 0x55ddb174d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.907+0000 7fc704e17740 4 client.25474 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.907+0000 7fc704e17740 1 -- 172.21.15.94:0/2794856438 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ddb13e11b0 con 0x55ddb174af90 2024-02-17T11:59:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.907+0000 7fc6f3fff640 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ddb174af90 0x55ddb174d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.908+0000 7fc6daffd640 10 client.25474 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.908+0000 7fc6f3fff640 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ddb174af90 0x55ddb174d3b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3540b90183970e2 server_cookie=4b1ceb7a82dd148e in_seq=0 out_seq=0 2024-02-17T11:59:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.909+0000 7fc6daffd640 1 -- 172.21.15.94:0/2794856438 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ddb13e11b0 con 0x55ddb174af90 2024-02-17T11:59:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.909+0000 7fc6daffd640 10 client.25474 handle_command_reply: tid=0 2024-02-17T11:59:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.909+0000 7fc704e17740 1 client.25474 shutdown 2024-02-17T11:59:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.910+0000 7fc704e17740 1 -- 172.21.15.94:0/2794856438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 msgr2=0x55ddb1737b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.910+0000 7fc704e17740 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb1737b70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec010000 tx=0x7fc6ec004730 comp rx=0 tx=0).stop 2024-02-17T11:59:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.910+0000 7fc704e17740 1 -- 172.21.15.94:0/2794856438 shutdown_connections 2024-02-17T11:59:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.910+0000 7fc704e17740 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ddb174af90 0x55ddb174d3b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.910+0000 7fc704e17740 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb173aaf0 0x55ddb17385f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.910+0000 7fc704e17740 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb1729ad0 0x55ddb17380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.910+0000 7fc704e17740 1 --2- 172.21.15.94:0/2794856438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb1737b70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.910+0000 7fc6daffd640 0 client.25474 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.910+0000 7fc704e17740 1 -- 172.21.15.94:0/2794856438 >> 172.21.15.94:0/2794856438 conn(0x55ddb16f0550 msgr2=0x55ddb17257a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.910+0000 7fc704e17740 1 -- 172.21.15.94:0/2794856438 shutdown_connections 2024-02-17T11:59:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.910+0000 7fc704e17740 1 -- 172.21.15.94:0/2794856438 wait complete. 2024-02-17T11:59:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.910+0000 7fc704e17740 20 client.25474 trim_cache size 0 max 16384 2024-02-17T11:59:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.929+0000 7fc704e17740 1 Processor -- start 2024-02-17T11:59:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.929+0000 7fc704e17740 1 -- start start 2024-02-17T11:59:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.930+0000 7fc704e17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb1728d60 0x55ddb173a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.930+0000 7fc704e17740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1729ad0 0x55ddb1738410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.930+0000 7fc6fb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb1728d60 0x55ddb173a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.930+0000 7fc6fb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb1728d60 0x55ddb173a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40002/0 (socket says 172.21.15.94:40002) 2024-02-17T11:59:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.930+0000 7fc6fb7fe640 1 -- 172.21.15.94:0/2221820207 learned_addr learned my addr 172.21.15.94:0/2221820207 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.930+0000 7fc704e17740 1 --2- 172.21.15.94:0/2221820207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb173aaf0 0x55ddb17418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.930+0000 7fc704e17740 1 -- 172.21.15.94:0/2221820207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddb1390630 con 0x55ddb1728d60 2024-02-17T11:59:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.930+0000 7fc704e17740 1 -- 172.21.15.94:0/2221820207 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddb17479a0 con 0x55ddb173aaf0 2024-02-17T11:59:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.930+0000 7fc704e17740 1 -- 172.21.15.94:0/2221820207 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddb1733760 con 0x55ddb1729ad0 2024-02-17T11:59:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.930+0000 7fc6f3fff640 1 --2- 172.21.15.94:0/2221820207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1729ad0 0x55ddb1738410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.930+0000 7fc6fbfff640 1 --2- 172.21.15.94:0/2221820207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb173aaf0 0x55ddb17418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.931+0000 7fc6fb7fe640 1 -- 172.21.15.94:0/2221820207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1729ad0 msgr2=0x55ddb1738410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.931+0000 7fc6fb7fe640 1 --2- 172.21.15.94:0/2221820207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1729ad0 0x55ddb1738410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.931+0000 7fc6fb7fe640 1 -- 172.21.15.94:0/2221820207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb173aaf0 msgr2=0x55ddb17418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.931+0000 7fc6fb7fe640 1 --2- 172.21.15.94:0/2221820207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb173aaf0 0x55ddb17418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.931+0000 7fc6fb7fe640 1 -- 172.21.15.94:0/2221820207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddb162b7b0 con 0x55ddb1728d60 2024-02-17T11:59:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.931+0000 7fc6fb7fe640 1 --2- 172.21.15.94:0/2221820207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb1728d60 0x55ddb173a2a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009e40 tx=0x7fc6ec029450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a28fbf908caa40e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.931+0000 7fc6f17fa640 1 -- 172.21.15.94:0/2221820207 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec029a80 con 0x55ddb1728d60 2024-02-17T11:59:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.932+0000 7fc6f17fa640 1 -- 172.21.15.94:0/2221820207 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6ec024070 con 0x55ddb1728d60 2024-02-17T11:59:41.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.932+0000 7fc6f17fa640 1 -- 172.21.15.94:0/2221820207 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec020da0 con 0x55ddb1728d60 2024-02-17T11:59:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.933+0000 7fc704e17740 1 -- 172.21.15.94:0/2221820207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb1728d60 msgr2=0x55ddb173a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.933+0000 7fc704e17740 1 --2- 172.21.15.94:0/2221820207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb1728d60 0x55ddb173a2a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec009e40 tx=0x7fc6ec029450 comp rx=0 tx=0).stop 2024-02-17T11:59:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.933+0000 7fc704e17740 1 -- 172.21.15.94:0/2221820207 shutdown_connections 2024-02-17T11:59:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.933+0000 7fc704e17740 1 --2- 172.21.15.94:0/2221820207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb173aaf0 0x55ddb17418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.933+0000 7fc704e17740 1 --2- 172.21.15.94:0/2221820207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1729ad0 0x55ddb1738410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.933+0000 7fc704e17740 1 --2- 172.21.15.94:0/2221820207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb1728d60 0x55ddb173a2a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.933+0000 7fc704e17740 1 -- 172.21.15.94:0/2221820207 >> 172.21.15.94:0/2221820207 conn(0x55ddb16f0550 msgr2=0x55ddb1723580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.933+0000 7fc704e17740 1 -- 172.21.15.94:0/2221820207 shutdown_connections 2024-02-17T11:59:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.933+0000 7fc704e17740 1 -- 172.21.15.94:0/2221820207 wait complete. 2024-02-17T11:59:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.933+0000 7fc704e17740 1 -- start start 2024-02-17T11:59:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.933+0000 7fc704e17740 1 Processor -- start 2024-02-17T11:59:41.936 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.934+0000 7fc704e17740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb173e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.934+0000 7fc704e17740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb1729ad0 0x55ddb173e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.934+0000 7fc704e17740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb173aaf0 0x55ddb173eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.934+0000 7fc704e17740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddb17a14b0 con 0x55ddb173aaf0 2024-02-17T11:59:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.934+0000 7fc704e17740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddb17a1620 con 0x55ddb1729ad0 2024-02-17T11:59:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.934+0000 7fc704e17740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddb17a17e0 con 0x55ddb1728d60 2024-02-17T11:59:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.934+0000 7fc704e17740 10 client.0 _ll_register_callbacks cb 0x7fc6f40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.934+0000 7fc6f3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb1729ad0 0x55ddb173e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.934+0000 7fc6f3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb1729ad0 0x55ddb173e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44758/0 (socket says 172.21.15.94:44758) 2024-02-17T11:59:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.934+0000 7fc6f3fff640 1 -- 172.21.15.94:0/2223913852 learned_addr learned my addr 172.21.15.94:0/2223913852 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.934+0000 7fc6fbfff640 1 --2- 172.21.15.94:0/2223913852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb173aaf0 0x55ddb173eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.935+0000 7fc6fb7fe640 1 --2- 172.21.15.94:0/2223913852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb173e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.935+0000 7fc6f3fff640 1 -- 172.21.15.94:0/2223913852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 msgr2=0x55ddb173e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.935+0000 7fc6f3fff640 1 --2- 172.21.15.94:0/2223913852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb173e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.935+0000 7fc6f3fff640 1 -- 172.21.15.94:0/2223913852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb173aaf0 msgr2=0x55ddb173eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.935+0000 7fc6f3fff640 1 --2- 172.21.15.94:0/2223913852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb173aaf0 0x55ddb173eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.935+0000 7fc6f3fff640 1 -- 172.21.15.94:0/2223913852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddb168ed80 con 0x55ddb1729ad0 2024-02-17T11:59:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.935+0000 7fc6f3fff640 1 --2- 172.21.15.94:0/2223913852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb1729ad0 0x55ddb173e960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4005e90 tx=0x7fc6e4005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=991c5a58c4701153 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.935+0000 7fc6daffd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T11:59:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc6daffd640 1 -- 172.21.15.94:0/2223913852 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4052040 con 0x55ddb1729ad0 2024-02-17T11:59:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc6daffd640 1 -- 172.21.15.94:0/2223913852 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc6e40104f0 con 0x55ddb1729ad0 2024-02-17T11:59:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc704e17740 1 -- 172.21.15.94:0/2223913852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6e40309e0 con 0x55ddb1729ad0 2024-02-17T11:59:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc6daffd640 1 -- 172.21.15.94:0/2223913852 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6e4005100 con 0x55ddb1729ad0 2024-02-17T11:59:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc704e17740 10 client.25424 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc704e17740 1 -- 172.21.15.94:0/2223913852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55ddb1651540 con 0x55ddb1729ad0 2024-02-17T11:59:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc6daffd640 1 -- 172.21.15.94:0/2223913852 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6e4066080 con 0x55ddb1729ad0 2024-02-17T11:59:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc6daffd640 1 -- 172.21.15.94:0/2223913852 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc6e4055050 con 0x55ddb1729ad0 2024-02-17T11:59:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc704e17740 10 client.25424 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc704e17740 10 client.25424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc704e17740 20 client.25424 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc704e17740 1 --2- 172.21.15.94:0/2223913852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ddb17a2ca0 0x55ddb17a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc704e17740 4 client.25424 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T11:59:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.936+0000 7fc704e17740 1 -- 172.21.15.94:0/2223913852 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fc6e40309e0 con 0x55ddb17a2ca0 2024-02-17T11:59:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.937+0000 7fc6fb7fe640 1 --2- 172.21.15.94:0/2223913852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ddb17a2ca0 0x55ddb17a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.938+0000 7fc6fb7fe640 1 --2- 172.21.15.94:0/2223913852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ddb17a2ca0 0x55ddb17a50c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a97591662570367 server_cookie=d87c3be62693b2ae in_seq=0 out_seq=0 2024-02-17T11:59:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.938+0000 7fc6daffd640 10 client.25424 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.960+0000 7fc6daffd640 1 -- 172.21.15.94:0/2223913852 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fc6e40309e0 con 0x55ddb17a2ca0 2024-02-17T11:59:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.960+0000 7fc6daffd640 10 client.25424 handle_command_reply: tid=0 2024-02-17T11:59:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.960+0000 7fc704e17740 1 client.25424 shutdown 2024-02-17T11:59:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc704e17740 1 -- 172.21.15.94:0/2223913852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb1729ad0 msgr2=0x55ddb173e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc704e17740 1 --2- 172.21.15.94:0/2223913852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb1729ad0 0x55ddb173e960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4005e90 tx=0x7fc6e4005e20 comp rx=0 tx=0).stop 2024-02-17T11:59:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc704e17740 1 -- 172.21.15.94:0/2223913852 shutdown_connections 2024-02-17T11:59:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc704e17740 1 --2- 172.21.15.94:0/2223913852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ddb17a2ca0 0x55ddb17a50c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc704e17740 1 --2- 172.21.15.94:0/2223913852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ddb173aaf0 0x55ddb173eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc6daffd640 0 client.25424 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc704e17740 1 --2- 172.21.15.94:0/2223913852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ddb1729ad0 0x55ddb173e960 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc704e17740 1 --2- 172.21.15.94:0/2223913852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ddb1728d60 0x55ddb173e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc704e17740 1 -- 172.21.15.94:0/2223913852 >> 172.21.15.94:0/2223913852 conn(0x55ddb16f0550 msgr2=0x55ddb17362d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc704e17740 1 -- 172.21.15.94:0/2223913852 shutdown_connections 2024-02-17T11:59:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc704e17740 1 -- 172.21.15.94:0/2223913852 wait complete. 2024-02-17T11:59:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc704e17740 20 client.25424 trim_cache size 0 max 16384 2024-02-17T11:59:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T11:59:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T11:59:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T11:59:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T11:59:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc702249640 1 -- 172.21.15.94:0/1973922748 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6e80a2350 msgr2=0x7fc6e80a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc702249640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6e80a2350 0x7fc6e80a4810 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec0098e0 tx=0x7fc6ec009870 comp rx=0 tx=0).stop 2024-02-17T11:59:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc702249640 1 -- 172.21.15.94:0/1973922748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6fc1142d0 msgr2=0x7fc6fc19db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc702249640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6fc1142d0 0x7fc6fc19db20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009fd0 tx=0x7fc6f400ef10 comp rx=0 tx=0).stop 2024-02-17T11:59:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc702249640 1 -- 172.21.15.94:0/1973922748 shutdown_connections 2024-02-17T11:59:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc702249640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6e80a2350 0x7fc6e80a4810 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc702249640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6fc1142d0 0x7fc6fc19db20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc702249640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6fc108cf0 0x7fc6fc1a3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc702249640 1 --2- 172.21.15.94:0/1973922748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6fc1083d0 0x7fc6fc1a3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.961+0000 7fc702249640 1 -- 172.21.15.94:0/1973922748 >> 172.21.15.94:0/1973922748 conn(0x7fc6fc06c220 msgr2=0x7fc6fc06fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.962+0000 7fc702249640 1 -- 172.21.15.94:0/1973922748 shutdown_connections 2024-02-17T11:59:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:41.962+0000 7fc702249640 1 -- 172.21.15.94:0/1973922748 wait complete. 2024-02-17T11:59:42.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:41 smithi189 ceph-mon[36588]: pgmap v50: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.9 MiB/s wr, 749 op/s 2024-02-17T11:59:42.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:41 smithi125 ceph-mon[36819]: pgmap v50: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.9 MiB/s wr, 749 op/s 2024-02-17T11:59:42.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:41 smithi094 ceph-mon[32499]: pgmap v50: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 7.9 MiB/s wr, 749 op/s 2024-02-17T11:59:43.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:42 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T11:59:43.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:42 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T11:59:43.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:42 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T11:59:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:43 smithi125 ceph-mon[36819]: pgmap v51: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-02-17T11:59:44.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:43 smithi094 ceph-mon[32499]: pgmap v51: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-02-17T11:59:44.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:43 smithi189 ceph-mon[36588]: pgmap v51: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 15 MiB/s wr, 1.16k op/s 2024-02-17T11:59:45.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:44 smithi125 ceph-mon[36819]: pgmap v52: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 11 MiB/s wr, 860 op/s 2024-02-17T11:59:45.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:44 smithi094 ceph-mon[32499]: pgmap v52: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 11 MiB/s wr, 860 op/s 2024-02-17T11:59:45.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 85.0 s, 3425.4 tps, lat 29.181 ms stddev 32.533 2024-02-17T11:59:45.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:44 smithi189 ceph-mon[36588]: pgmap v52: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 11 MiB/s wr, 860 op/s 2024-02-17T11:59:48.102 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:47 smithi189 ceph-mon[36588]: pgmap v53: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-02-17T11:59:48.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:47 smithi125 ceph-mon[36819]: pgmap v53: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-02-17T11:59:48.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:47 smithi094 ceph-mon[32499]: pgmap v53: 97 pgs: 97 active+clean; 1.2 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 11 MiB/s wr, 843 op/s 2024-02-17T11:59:50.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:49 smithi189 ceph-mon[36588]: pgmap v54: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2024-02-17T11:59:50.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:49 smithi125 ceph-mon[36819]: pgmap v54: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2024-02-17T11:59:50.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:49 smithi094 ceph-mon[32499]: pgmap v54: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 15 MiB/s wr, 1.18k op/s 2024-02-17T11:59:50.241 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 90.0 s, 3647.4 tps, lat 27.420 ms stddev 27.607 2024-02-17T11:59:52.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:51 smithi189 ceph-mon[36588]: pgmap v55: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 765 op/s 2024-02-17T11:59:52.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:51 smithi125 ceph-mon[36819]: pgmap v55: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 765 op/s 2024-02-17T11:59:52.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:51 smithi094 ceph-mon[32499]: pgmap v55: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 11 MiB/s wr, 765 op/s 2024-02-17T11:59:54.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:53 smithi189 ceph-mon[36588]: pgmap v56: 97 pgs: 97 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-02-17T11:59:54.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:53 smithi125 ceph-mon[36819]: pgmap v56: 97 pgs: 97 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-02-17T11:59:54.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:53 smithi094 ceph-mon[32499]: pgmap v56: 97 pgs: 97 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 1.03k op/s 2024-02-17T11:59:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.083+0000 7f07df7dd640 1 -- 172.21.15.94:0/2561963888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d81013a0 msgr2=0x7f07d8101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.083+0000 7f07df7dd640 1 --2- 172.21.15.94:0/2561963888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d81013a0 0x7f07d8101780 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009930 tx=0x7f07d402bc30 comp rx=0 tx=0).stop 2024-02-17T11:59:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.083+0000 7f07df7dd640 1 -- 172.21.15.94:0/2561963888 shutdown_connections 2024-02-17T11:59:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.083+0000 7f07df7dd640 1 --2- 172.21.15.94:0/2561963888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d8106470 0x7f07d810e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.083+0000 7f07df7dd640 1 --2- 172.21.15.94:0/2561963888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07d8101d50 0x7f07d8105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.083+0000 7f07df7dd640 1 --2- 172.21.15.94:0/2561963888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d81013a0 0x7f07d8101780 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.083+0000 7f07df7dd640 1 -- 172.21.15.94:0/2561963888 >> 172.21.15.94:0/2561963888 conn(0x7f07d80fcc40 msgr2=0x7f07d80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.083+0000 7f07df7dd640 1 -- 172.21.15.94:0/2561963888 shutdown_connections 2024-02-17T11:59:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.083+0000 7f07df7dd640 1 -- 172.21.15.94:0/2561963888 wait complete. 2024-02-17T11:59:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.084+0000 7f07df7dd640 1 Processor -- start 2024-02-17T11:59:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.084+0000 7f07df7dd640 1 -- start start 2024-02-17T11:59:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.084+0000 7f07df7dd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d81013a0 0x7f07d819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07df7dd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07d8106470 0x7f07d819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07df7dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d810cff0 0x7f07d810d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07dd579640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d81013a0 0x7f07d819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07dd579640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d81013a0 0x7f07d819a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:50800/0 (socket says 172.21.15.94:50800) 2024-02-17T11:59:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07dd579640 1 -- 172.21.15.94:0/1621441999 learned_addr learned my addr 172.21.15.94:0/1621441999 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07df7dd640 1 -- 172.21.15.94:0/1621441999 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d810dc30 con 0x7f07d810cff0 2024-02-17T11:59:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07df7dd640 1 -- 172.21.15.94:0/1621441999 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d810dda0 con 0x7f07d8106470 2024-02-17T11:59:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07dcd78640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07d8106470 0x7f07d819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07df7dd640 1 -- 172.21.15.94:0/1621441999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d819df70 con 0x7f07d81013a0 2024-02-17T11:59:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07ddd7a640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d810cff0 0x7f07d810d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07dd579640 1 -- 172.21.15.94:0/1621441999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07d8106470 msgr2=0x7f07d819a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07dd579640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07d8106470 0x7f07d819a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07dd579640 1 -- 172.21.15.94:0/1621441999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d810cff0 msgr2=0x7f07d810d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07dd579640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d810cff0 0x7f07d810d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07dd579640 1 -- 172.21.15.94:0/1621441999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c8009490 con 0x7f07d81013a0 2024-02-17T11:59:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.085+0000 7f07dcd78640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07d8106470 0x7f07d819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.086+0000 7f07dd579640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d81013a0 0x7f07d819a270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009230 tx=0x7f07d402c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.086+0000 7f07ce7fc640 1 -- 172.21.15.94:0/1621441999 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4004350 con 0x7f07d81013a0 2024-02-17T11:59:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.086+0000 7f07df7dd640 1 -- 172.21.15.94:0/1621441999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07d4009560 con 0x7f07d81013a0 2024-02-17T11:59:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.086+0000 7f07ce7fc640 1 -- 172.21.15.94:0/1621441999 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07d40044b0 con 0x7f07d81013a0 2024-02-17T11:59:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.086+0000 7f07df7dd640 1 -- 172.21.15.94:0/1621441999 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d819e570 con 0x7f07d81013a0 2024-02-17T11:59:55.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.086+0000 7f07ce7fc640 1 -- 172.21.15.94:0/1621441999 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4003d00 con 0x7f07d81013a0 2024-02-17T11:59:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.089+0000 7f07e23a8740 1 Processor -- start 2024-02-17T11:59:55.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.089+0000 7f07e23a8740 1 -- start start 2024-02-17T11:59:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.090+0000 7f07ce7fc640 1 -- 172.21.15.94:0/1621441999 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f07d4004df0 con 0x7f07d81013a0 2024-02-17T11:59:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.090+0000 7f07e23a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088fad0 0x560d0088e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.090+0000 7f07e23a8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088ed60 0x560d0088f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.090+0000 7f07e23a8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.090+0000 7f07e23a8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d007fd9f0 con 0x560d008982e0 2024-02-17T11:59:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.090+0000 7f07e23a8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d007faf20 con 0x560d0088ed60 2024-02-17T11:59:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.090+0000 7f07e23a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d007f6700 con 0x560d0088fad0 2024-02-17T11:59:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.090+0000 7f07dcd78640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.090+0000 7f07dcd78640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54706/0 (socket says 172.21.15.94:54706) 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.090+0000 7f07dcd78640 1 -- 172.21.15.94:0/670329107 learned_addr learned my addr 172.21.15.94:0/670329107 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.090+0000 7f07ddd7a640 1 --2- 172.21.15.94:0/670329107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088fad0 0x560d0088e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.090+0000 7f07dd579640 1 --2- 172.21.15.94:0/670329107 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088ed60 0x560d0088f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.091+0000 7f07dcd78640 1 -- 172.21.15.94:0/670329107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088fad0 msgr2=0x560d0088e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.091+0000 7f07dcd78640 1 --2- 172.21.15.94:0/670329107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088fad0 0x560d0088e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.091+0000 7f07dcd78640 1 -- 172.21.15.94:0/670329107 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088ed60 msgr2=0x560d0088f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.091+0000 7f07dcd78640 1 --2- 172.21.15.94:0/670329107 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088ed60 0x560d0088f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.091+0000 7f07dcd78640 1 -- 172.21.15.94:0/670329107 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d0078c7b0 con 0x560d008982e0 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.091+0000 7f07ddd7a640 1 --2- 172.21.15.94:0/670329107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088fad0 0x560d0088e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.091+0000 7f07dcd78640 1 --2- 172.21.15.94:0/670329107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089a6d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f07c80115f0 tx=0x7f07c8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d56002efcf580ec server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.091+0000 7f07ce7fc640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f07c40a22d0 0x7f07c40a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.091+0000 7f07c37fe640 1 -- 172.21.15.94:0/670329107 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c800f780 con 0x560d008982e0 2024-02-17T11:59:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.091+0000 7f07c37fe640 1 -- 172.21.15.94:0/670329107 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07c800fdc0 con 0x560d008982e0 2024-02-17T11:59:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.091+0000 7f07c37fe640 1 -- 172.21.15.94:0/670329107 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c8019de0 con 0x560d008982e0 2024-02-17T11:59:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.091+0000 7f07ddd7a640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f07c40a22d0 0x7f07c40a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.091+0000 7f07ce7fc640 1 -- 172.21.15.94:0/1621441999 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f07d40efff0 con 0x7f07d81013a0 2024-02-17T11:59:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.092+0000 7f07e23a8740 1 -- 172.21.15.94:0/670329107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 msgr2=0x560d0089a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.092+0000 7f07ddd7a640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f07c40a22d0 0x7f07c40a4790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009fd0 tx=0x7f07d0009500 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.092+0000 7f07e23a8740 1 --2- 172.21.15.94:0/670329107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089a6d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f07c80115f0 tx=0x7f07c8011ab0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.092+0000 7f07e23a8740 1 -- 172.21.15.94:0/670329107 shutdown_connections 2024-02-17T11:59:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.092+0000 7f07e23a8740 1 --2- 172.21.15.94:0/670329107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089a6d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.092+0000 7f07e23a8740 1 --2- 172.21.15.94:0/670329107 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088ed60 0x560d0088f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.092+0000 7f07e23a8740 1 --2- 172.21.15.94:0/670329107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088fad0 0x560d0088e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.092+0000 7f07e23a8740 1 -- 172.21.15.94:0/670329107 >> 172.21.15.94:0/670329107 conn(0x560d00856550 msgr2=0x560d00889a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.092+0000 7f07e23a8740 1 -- 172.21.15.94:0/670329107 shutdown_connections 2024-02-17T11:59:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.092+0000 7f07e23a8740 1 -- 172.21.15.94:0/670329107 wait complete. 2024-02-17T11:59:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.093+0000 7f07e23a8740 1 -- start start 2024-02-17T11:59:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.093+0000 7f07e23a8740 1 Processor -- start 2024-02-17T11:59:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07e23a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d008952b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07e23a8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d0088fad0 0x560d008957f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07e23a8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d008982e0 0x560d00895d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07e23a8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0079bbf0 con 0x560d0088fad0 2024-02-17T11:59:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07e23a8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d004ca250 con 0x560d008982e0 2024-02-17T11:59:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07e23a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0077c200 con 0x560d0088ed60 2024-02-17T11:59:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07e23a8740 10 client.0 _ll_register_callbacks cb 0x560d007ead70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07dd579640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d008952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07dd579640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d008952b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:50828/0 (socket says 172.21.15.94:50828) 2024-02-17T11:59:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07dd579640 1 -- 172.21.15.94:0/2503287364 learned_addr learned my addr 172.21.15.94:0/2503287364 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07dcd78640 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d0088fad0 0x560d008957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07ddd7a640 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d008982e0 0x560d00895d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07dd579640 1 -- 172.21.15.94:0/2503287364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d008982e0 msgr2=0x560d00895d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07dd579640 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d008982e0 0x560d00895d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07dd579640 1 -- 172.21.15.94:0/2503287364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d0088fad0 msgr2=0x560d008957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07dd579640 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d0088fad0 0x560d008957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07dd579640 1 -- 172.21.15.94:0/2503287364 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c8009dd0 con 0x560d0088ed60 2024-02-17T11:59:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.094+0000 7f07ddd7a640 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d008982e0 0x560d00895d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T11:59:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.095+0000 7f07dd579640 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d008952b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009430 tx=0x7f07d4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f982bbee05fb3dc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.095+0000 7f07c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.095+0000 7f07c0ff9640 1 -- 172.21.15.94:0/2503287364 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d40edb50 con 0x560d0088ed60 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.095+0000 7f07e23a8740 1 -- 172.21.15.94:0/2503287364 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d004f6600 con 0x560d0088ed60 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.095+0000 7f07c0ff9640 1 -- 172.21.15.94:0/2503287364 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f07d40035f0 con 0x560d0088ed60 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.095+0000 7f07c0ff9640 1 -- 172.21.15.94:0/2503287364 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07d4003e60 con 0x560d0088ed60 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.095+0000 7f07e23a8740 10 client.25489 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.095+0000 7f07e23a8740 1 -- 172.21.15.94:0/2503287364 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x560d007f4d80 con 0x560d0088ed60 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.096+0000 7f07c0ff9640 1 -- 172.21.15.94:0/2503287364 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560d004f6600 con 0x560d0088ed60 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.096+0000 7f07c0ff9640 1 -- 172.21.15.94:0/2503287364 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f07d403c070 con 0x560d0088ed60 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.096+0000 7f07e23a8740 10 client.25489 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.096+0000 7f07e23a8740 10 client.25489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.096+0000 7f07e23a8740 20 client.25489 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.096+0000 7f07e23a8740 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560d008a8500 0x560d008aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.096+0000 7f07e23a8740 4 client.25489 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T11:59:55.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.096+0000 7f07e23a8740 1 -- 172.21.15.94:0/2503287364 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d005471b0 con 0x560d008a8500 2024-02-17T11:59:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.096+0000 7f07dcd78640 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560d008a8500 0x560d008aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.097+0000 7f07dcd78640 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560d008a8500 0x560d008aa8f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98efafde5c4a1a3c server_cookie=827d5780dc045e09 in_seq=0 out_seq=0 2024-02-17T11:59:55.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.097+0000 7f07c0ff9640 10 client.25489 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.097+0000 7f07c0ff9640 1 -- 172.21.15.94:0/2503287364 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d005471b0 con 0x560d008a8500 2024-02-17T11:59:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.097+0000 7f07c0ff9640 10 client.25489 handle_command_reply: tid=0 2024-02-17T11:59:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.097+0000 7f07e23a8740 1 client.25489 shutdown 2024-02-17T11:59:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.098+0000 7f07e23a8740 1 -- 172.21.15.94:0/2503287364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 msgr2=0x560d008952b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.098+0000 7f07e23a8740 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d008952b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009430 tx=0x7f07d4009260 comp rx=0 tx=0).stop 2024-02-17T11:59:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.098+0000 7f07e23a8740 1 -- 172.21.15.94:0/2503287364 shutdown_connections 2024-02-17T11:59:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.098+0000 7f07e23a8740 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560d008a8500 0x560d008aa8f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.098+0000 7f07e23a8740 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d008982e0 0x560d00895d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.098+0000 7f07e23a8740 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d0088fad0 0x560d008957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.098+0000 7f07e23a8740 1 --2- 172.21.15.94:0/2503287364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d008952b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.098+0000 7f07c0ff9640 0 client.25489 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.098+0000 7f07e23a8740 1 -- 172.21.15.94:0/2503287364 >> 172.21.15.94:0/2503287364 conn(0x560d00856550 msgr2=0x560d0088b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.099+0000 7f07e23a8740 1 -- 172.21.15.94:0/2503287364 shutdown_connections 2024-02-17T11:59:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.099+0000 7f07e23a8740 1 -- 172.21.15.94:0/2503287364 wait complete. 2024-02-17T11:59:55.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.099+0000 7f07e23a8740 20 client.25489 trim_cache size 0 max 16384 2024-02-17T11:59:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.116+0000 7f07e23a8740 1 Processor -- start 2024-02-17T11:59:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.116+0000 7f07e23a8740 1 -- start start 2024-02-17T11:59:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.116+0000 7f07e23a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d008979e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.116+0000 7f07e23a8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088fad0 0x560d00895b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.116+0000 7f07e23a8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.116+0000 7f07e23a8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d008a5170 con 0x560d008982e0 2024-02-17T11:59:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.116+0000 7f07e23a8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d008a8300 con 0x560d0088fad0 2024-02-17T11:59:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.116+0000 7f07e23a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0089aef0 con 0x560d0088ed60 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07ddd7a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07dd579640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d008979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07ddd7a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54724/0 (socket says 172.21.15.94:54724) 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07dd579640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d008979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:50834/0 (socket says 172.21.15.94:50834) 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07ddd7a640 1 -- 172.21.15.94:0/3529391332 learned_addr learned my addr 172.21.15.94:0/3529391332 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07dcd78640 1 --2- 172.21.15.94:0/3529391332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088fad0 0x560d00895b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07ddd7a640 1 -- 172.21.15.94:0/3529391332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 msgr2=0x560d008979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07ddd7a640 1 --2- 172.21.15.94:0/3529391332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d008979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07ddd7a640 1 -- 172.21.15.94:0/3529391332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088fad0 msgr2=0x560d00895b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07ddd7a640 1 --2- 172.21.15.94:0/3529391332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088fad0 0x560d00895b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07ddd7a640 1 -- 172.21.15.94:0/3529391332 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d007917b0 con 0x560d008982e0 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07ddd7a640 1 --2- 172.21.15.94:0/3529391332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089f160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f07d00114e0 tx=0x7f07d0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63852e25eb45304 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07c2ffd640 1 -- 172.21.15.94:0/3529391332 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0024070 con 0x560d008982e0 2024-02-17T11:59:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07c2ffd640 1 -- 172.21.15.94:0/3529391332 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07d0003970 con 0x560d008982e0 2024-02-17T11:59:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.117+0000 7f07c2ffd640 1 -- 172.21.15.94:0/3529391332 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0016070 con 0x560d008982e0 2024-02-17T11:59:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.118+0000 7f07e23a8740 1 -- 172.21.15.94:0/3529391332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 msgr2=0x560d0089f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.118+0000 7f07e23a8740 1 --2- 172.21.15.94:0/3529391332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089f160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f07d00114e0 tx=0x7f07d0016000 comp rx=0 tx=0).stop 2024-02-17T11:59:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.118+0000 7f07e23a8740 1 -- 172.21.15.94:0/3529391332 shutdown_connections 2024-02-17T11:59:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.118+0000 7f07e23a8740 1 --2- 172.21.15.94:0/3529391332 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089f160 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.118+0000 7f07e23a8740 1 --2- 172.21.15.94:0/3529391332 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088fad0 0x560d00895b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.118+0000 7f07e23a8740 1 --2- 172.21.15.94:0/3529391332 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d008979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.118+0000 7f07e23a8740 1 -- 172.21.15.94:0/3529391332 >> 172.21.15.94:0/3529391332 conn(0x560d00856550 msgr2=0x560d00889580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.118+0000 7f07e23a8740 1 -- 172.21.15.94:0/3529391332 shutdown_connections 2024-02-17T11:59:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.118+0000 7f07e23a8740 1 -- 172.21.15.94:0/3529391332 wait complete. 2024-02-17T11:59:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.118+0000 7f07e23a8740 1 -- start start 2024-02-17T11:59:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07e23a8740 1 Processor -- start 2024-02-17T11:59:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07e23a8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d0089f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07e23a8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088fad0 0x560d0089b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07e23a8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07e23a8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0089c500 con 0x560d008982e0 2024-02-17T11:59:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07e23a8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d0089c670 con 0x560d0088fad0 2024-02-17T11:59:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07e23a8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d008fe9a0 con 0x560d0088ed60 2024-02-17T11:59:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07e23a8740 10 client.0 _ll_register_callbacks cb 0x7f07c8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T11:59:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07dd579640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d0089f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07ddd7a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07dd579640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d0089f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:50842/0 (socket says 172.21.15.94:50842) 2024-02-17T11:59:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07dd579640 1 -- 172.21.15.94:0/2794738495 learned_addr learned my addr 172.21.15.94:0/2794738495 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T11:59:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07dcd78640 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088fad0 0x560d0089b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07ddd7a640 1 -- 172.21.15.94:0/2794738495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 msgr2=0x560d0089f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07ddd7a640 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d0089f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07ddd7a640 1 -- 172.21.15.94:0/2794738495 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088fad0 msgr2=0x560d0089b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T11:59:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.119+0000 7f07ddd7a640 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088fad0 0x560d0089b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.120+0000 7f07ddd7a640 1 -- 172.21.15.94:0/2794738495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d0011190 con 0x560d008982e0 2024-02-17T11:59:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.120+0000 7f07dd579640 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d0089f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:55.123 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.120+0000 7f07dcd78640 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088fad0 0x560d0089b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T11:59:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.120+0000 7f07ddd7a640 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089be00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f07d0018040 tx=0x7f07d0003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3619ba56cdcb1d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T11:59:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.120+0000 7f07c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T11:59:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.120+0000 7f07c0ff9640 1 -- 172.21.15.94:0/2794738495 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0024040 con 0x560d008982e0 2024-02-17T11:59:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.120+0000 7f07c0ff9640 1 -- 172.21.15.94:0/2794738495 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f07d0004e10 con 0x560d008982e0 2024-02-17T11:59:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.120+0000 7f07e23a8740 1 -- 172.21.15.94:0/2794738495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f07c800fdc0 con 0x560d008982e0 2024-02-17T11:59:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.121+0000 7f07c0ff9640 1 -- 172.21.15.94:0/2794738495 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07d00043b0 con 0x560d008982e0 2024-02-17T11:59:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.121+0000 7f07e23a8740 10 client.15984 fetch_fsmap learned FSMap version 21 2024-02-17T11:59:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.121+0000 7f07e23a8740 1 -- 172.21.15.94:0/2794738495 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x560d007b7540 con 0x560d008982e0 2024-02-17T11:59:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.121+0000 7f07c0ff9640 1 -- 172.21.15.94:0/2794738495 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f07d0039030 con 0x560d008982e0 2024-02-17T11:59:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.121+0000 7f07c0ff9640 1 -- 172.21.15.94:0/2794738495 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f07d0005040 con 0x560d008982e0 2024-02-17T11:59:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.121+0000 7f07e23a8740 10 client.15984 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T11:59:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.121+0000 7f07e23a8740 10 client.15984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T11:59:55.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.121+0000 7f07e23a8740 20 client.15984 populate_metadata read hostname 'smithi094' 2024-02-17T11:59:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.121+0000 7f07e23a8740 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560d008ffff0 0x560d00902410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T11:59:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.121+0000 7f07e23a8740 4 client.15984 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T11:59:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.121+0000 7f07e23a8740 1 -- 172.21.15.94:0/2794738495 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560d00533550 con 0x560d008ffff0 2024-02-17T11:59:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.121+0000 7f07dd579640 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560d008ffff0 0x560d00902410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T11:59:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.122+0000 7f07c0ff9640 10 client.15984 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:55.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.122+0000 7f07dd579640 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560d008ffff0 0x560d00902410 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc43af2bae374716 server_cookie=72ff1292443e140d in_seq=0 out_seq=0 2024-02-17T11:59:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07c0ff9640 1 -- 172.21.15.94:0/2794738495 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560d00533550 con 0x560d008ffff0 2024-02-17T11:59:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07c0ff9640 10 client.15984 handle_command_reply: tid=0 2024-02-17T11:59:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07e23a8740 1 client.15984 shutdown 2024-02-17T11:59:55.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07e23a8740 1 -- 172.21.15.94:0/2794738495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 msgr2=0x560d0089be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07e23a8740 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089be00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f07d0018040 tx=0x7f07d0003f50 comp rx=0 tx=0).stop 2024-02-17T11:59:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07e23a8740 1 -- 172.21.15.94:0/2794738495 shutdown_connections 2024-02-17T11:59:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07e23a8740 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560d008ffff0 0x560d00902410 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07e23a8740 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560d008982e0 0x560d0089be00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07c0ff9640 0 client.15984 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T11:59:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07e23a8740 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560d0088fad0 0x560d0089b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07e23a8740 1 --2- 172.21.15.94:0/2794738495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560d0088ed60 0x560d0089f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07e23a8740 1 -- 172.21.15.94:0/2794738495 >> 172.21.15.94:0/2794738495 conn(0x560d00856550 msgr2=0x560d00893780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07e23a8740 1 -- 172.21.15.94:0/2794738495 shutdown_connections 2024-02-17T11:59:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.140+0000 7f07e23a8740 1 -- 172.21.15.94:0/2794738495 wait complete. 2024-02-17T11:59:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.141+0000 7f07e23a8740 20 client.15984 trim_cache size 0 max 16384 2024-02-17T11:59:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T11:59:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T11:59:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T11:59:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T11:59:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.141+0000 7f07df7dd640 1 -- 172.21.15.94:0/1621441999 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f07c40a22d0 msgr2=0x7f07c40a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.141+0000 7f07df7dd640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f07c40a22d0 0x7f07c40a4790 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009fd0 tx=0x7f07d0009500 comp rx=0 tx=0).stop 2024-02-17T11:59:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.141+0000 7f07df7dd640 1 -- 172.21.15.94:0/1621441999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d81013a0 msgr2=0x7f07d819a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T11:59:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.141+0000 7f07df7dd640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d81013a0 0x7f07d819a270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f07d4009230 tx=0x7f07d402c650 comp rx=0 tx=0).stop 2024-02-17T11:59:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.141+0000 7f07df7dd640 1 -- 172.21.15.94:0/1621441999 shutdown_connections 2024-02-17T11:59:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.141+0000 7f07df7dd640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f07c40a22d0 0x7f07c40a4790 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.141+0000 7f07df7dd640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f07d810cff0 0x7f07d810d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.141+0000 7f07df7dd640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f07d8106470 0x7f07d819a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.141+0000 7f07df7dd640 1 --2- 172.21.15.94:0/1621441999 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f07d81013a0 0x7f07d819a270 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T11:59:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.141+0000 7f07df7dd640 1 -- 172.21.15.94:0/1621441999 >> 172.21.15.94:0/1621441999 conn(0x7f07d80fcc40 msgr2=0x7f07d80fdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T11:59:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.141+0000 7f07df7dd640 1 -- 172.21.15.94:0/1621441999 shutdown_connections 2024-02-17T11:59:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T11:59:55.141+0000 7f07df7dd640 1 -- 172.21.15.94:0/1621441999 wait complete. 2024-02-17T11:59:55.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 95.0 s, 3517.4 tps, lat 28.389 ms stddev 29.134 2024-02-17T11:59:56.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:55 smithi189 ceph-mon[36588]: pgmap v57: 97 pgs: 97 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 699 op/s 2024-02-17T11:59:56.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:55 smithi125 ceph-mon[36819]: pgmap v57: 97 pgs: 97 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 699 op/s 2024-02-17T11:59:56.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:55 smithi094 ceph-mon[32499]: pgmap v57: 97 pgs: 97 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 9.1 MiB/s wr, 699 op/s 2024-02-17T11:59:58.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:57 smithi189 ceph-mon[36588]: pgmap v58: 97 pgs: 97 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.0 MiB/s wr, 680 op/s 2024-02-17T11:59:58.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:57 smithi125 ceph-mon[36819]: pgmap v58: 97 pgs: 97 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.0 MiB/s wr, 680 op/s 2024-02-17T11:59:58.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:57 smithi094 ceph-mon[32499]: pgmap v58: 97 pgs: 97 active+clean; 1.3 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.0 MiB/s wr, 680 op/s 2024-02-17T12:00:00.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 11:59:59 smithi189 ceph-mon[36588]: pgmap v59: 97 pgs: 97 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 999 op/s 2024-02-17T12:00:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 11:59:59 smithi125 ceph-mon[36819]: pgmap v59: 97 pgs: 97 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 999 op/s 2024-02-17T12:00:00.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 11:59:59 smithi094 ceph-mon[32499]: pgmap v59: 97 pgs: 97 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 12 MiB/s wr, 999 op/s 2024-02-17T12:00:00.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 100.0 s, 3577.2 tps, lat 28.021 ms stddev 30.698 2024-02-17T12:00:01.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:00 smithi189 ceph-mon[36588]: overall HEALTH_OK 2024-02-17T12:00:01.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:00 smithi125 ceph-mon[36819]: overall HEALTH_OK 2024-02-17T12:00:01.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:00 smithi094 ceph-mon[32499]: overall HEALTH_OK 2024-02-17T12:00:02.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:01 smithi189 ceph-mon[36588]: pgmap v60: 97 pgs: 97 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 667 op/s 2024-02-17T12:00:02.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:01 smithi125 ceph-mon[36819]: pgmap v60: 97 pgs: 97 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 667 op/s 2024-02-17T12:00:02.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:01 smithi094 ceph-mon[32499]: pgmap v60: 97 pgs: 97 active+clean; 1.3 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.2 MiB/s wr, 667 op/s 2024-02-17T12:00:04.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:03 smithi189 ceph-mon[36588]: pgmap v61: 97 pgs: 97 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.6 MiB/s wr, 872 op/s 2024-02-17T12:00:04.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:03 smithi125 ceph-mon[36819]: pgmap v61: 97 pgs: 97 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.6 MiB/s wr, 872 op/s 2024-02-17T12:00:04.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:03 smithi094 ceph-mon[32499]: pgmap v61: 97 pgs: 97 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.6 MiB/s wr, 872 op/s 2024-02-17T12:00:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 105.0 s, 473.6 tps, lat 209.825 ms stddev 239.118 2024-02-17T12:00:06.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:05 smithi189 ceph-mon[36588]: pgmap v62: 97 pgs: 97 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 716 op/s 2024-02-17T12:00:06.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:05 smithi125 ceph-mon[36819]: pgmap v62: 97 pgs: 97 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 716 op/s 2024-02-17T12:00:06.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:05 smithi094 ceph-mon[32499]: pgmap v62: 97 pgs: 97 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 716 op/s 2024-02-17T12:00:07.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:00:07.527 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:00:07.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.694+0000 7f52f0464640 1 -- 172.21.15.94:0/1725245995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52e8103590 msgr2=0x7f52e8103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.694+0000 7f52f0464640 1 --2- 172.21.15.94:0/1725245995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52e8103590 0x7f52e8103970 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009930 tx=0x7f52e402bc20 comp rx=0 tx=0).stop 2024-02-17T12:00:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.695+0000 7f52f0464640 1 -- 172.21.15.94:0/1725245995 shutdown_connections 2024-02-17T12:00:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.695+0000 7f52f0464640 1 --2- 172.21.15.94:0/1725245995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52e8108660 0x7f52e8110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.695+0000 7f52f0464640 1 --2- 172.21.15.94:0/1725245995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52e8103f40 0x7f52e8107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.695+0000 7f52f0464640 1 --2- 172.21.15.94:0/1725245995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52e8103590 0x7f52e8103970 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.695+0000 7f52f0464640 1 -- 172.21.15.94:0/1725245995 >> 172.21.15.94:0/1725245995 conn(0x7f52e80fee70 msgr2=0x7f52e8101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.695+0000 7f52f0464640 1 -- 172.21.15.94:0/1725245995 shutdown_connections 2024-02-17T12:00:07.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.695+0000 7f52f0464640 1 -- 172.21.15.94:0/1725245995 wait complete. 2024-02-17T12:00:07.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.696+0000 7f52f0464640 1 Processor -- start 2024-02-17T12:00:07.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.696+0000 7f52f0464640 1 -- start start 2024-02-17T12:00:07.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.696+0000 7f52f0464640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52e8103590 0x7f52e819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52f0464640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52e8108660 0x7f52e819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52f0464640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52e8199910 0x7f52e8199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52f0464640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e819a300 con 0x7f52e8199910 2024-02-17T12:00:07.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52f0464640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e819a470 con 0x7f52e8108660 2024-02-17T12:00:07.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52f0464640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e819a630 con 0x7f52e8103590 2024-02-17T12:00:07.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52ee200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52e8103590 0x7f52e819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52ee200640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52e8103590 0x7f52e819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60838/0 (socket says 172.21.15.94:60838) 2024-02-17T12:00:07.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52eea01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52e8199910 0x7f52e8199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52ee200640 1 -- 172.21.15.94:0/2875891161 learned_addr learned my addr 172.21.15.94:0/2875891161 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:07.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52ed9ff640 1 --2- 172.21.15.94:0/2875891161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52e8108660 0x7f52e819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52ee200640 1 -- 172.21.15.94:0/2875891161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52e8108660 msgr2=0x7f52e819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:07.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52ee200640 1 --2- 172.21.15.94:0/2875891161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52e8108660 0x7f52e819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52ee200640 1 -- 172.21.15.94:0/2875891161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52e8199910 msgr2=0x7f52e8199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:07.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52ee200640 1 --2- 172.21.15.94:0/2875891161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52e8199910 0x7f52e8199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52ee200640 1 -- 172.21.15.94:0/2875891161 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4009560 con 0x7f52e8103590 2024-02-17T12:00:07.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.697+0000 7f52ee200640 1 --2- 172.21.15.94:0/2875891161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52e8103590 0x7f52e819f710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f52e4000c00 tx=0x7f52e4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:07.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.698+0000 7f52db7fe640 1 -- 172.21.15.94:0/2875891161 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e402dea0 con 0x7f52e8103590 2024-02-17T12:00:07.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.698+0000 7f52f0464640 1 -- 172.21.15.94:0/2875891161 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e81a65d0 con 0x7f52e8103590 2024-02-17T12:00:07.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.698+0000 7f52db7fe640 1 -- 172.21.15.94:0/2875891161 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52e4004440 con 0x7f52e8103590 2024-02-17T12:00:07.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.698+0000 7f52f0464640 1 -- 172.21.15.94:0/2875891161 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e81a6ad0 con 0x7f52e8103590 2024-02-17T12:00:07.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.698+0000 7f52db7fe640 1 -- 172.21.15.94:0/2875891161 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e4003b80 con 0x7f52e8103590 2024-02-17T12:00:07.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.701+0000 7f52db7fe640 1 -- 172.21.15.94:0/2875891161 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f52e4003ce0 con 0x7f52e8103590 2024-02-17T12:00:07.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.702+0000 7f52f3039740 1 Processor -- start 2024-02-17T12:00:07.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.702+0000 7f52f3039740 1 -- start start 2024-02-17T12:00:07.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52f3039740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e334e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52db7fe640 1 --2- 172.21.15.94:0/2875891161 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f52d40a2330 0x7f52d40a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52f3039740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e334ea70 0x5601e335fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52f3039740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e335ffc0 0x5601e33623b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52f3039740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e32bd280 con 0x5601e334e0c0 2024-02-17T12:00:07.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52f3039740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e32ba7b0 con 0x5601e335ffc0 2024-02-17T12:00:07.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52f3039740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e32b5f90 con 0x5601e334ea70 2024-02-17T12:00:07.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52ee200640 1 --2- 172.21.15.94:0/2875891161 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f52d40a2330 0x7f52d40a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52eea01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e334e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52db7fe640 1 -- 172.21.15.94:0/2875891161 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f52e40f0430 con 0x7f52e8103590 2024-02-17T12:00:07.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52eea01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e334e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33054/0 (socket says 172.21.15.94:33054) 2024-02-17T12:00:07.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52eea01640 1 -- 172.21.15.94:0/1049667582 learned_addr learned my addr 172.21.15.94:0/1049667582 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:07.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52eea01640 1 --2- 172.21.15.94:0/1049667582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e335ffc0 0x5601e33623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.703+0000 7f52ed9ff640 1 --2- 172.21.15.94:0/1049667582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e334ea70 0x5601e335fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.704+0000 7f52eea01640 1 -- 172.21.15.94:0/1049667582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e334ea70 msgr2=0x5601e335fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:07.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.704+0000 7f52eea01640 1 --2- 172.21.15.94:0/1049667582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e334ea70 0x5601e335fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.704+0000 7f52eea01640 1 -- 172.21.15.94:0/1049667582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 msgr2=0x5601e334e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:07.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.704+0000 7f52eea01640 1 --2- 172.21.15.94:0/1049667582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e334e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.704+0000 7f52eea01640 1 -- 172.21.15.94:0/1049667582 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601e3274570 con 0x5601e335ffc0 2024-02-17T12:00:07.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.704+0000 7f52ee200640 1 --2- 172.21.15.94:0/2875891161 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f52d40a2330 0x7f52d40a47f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009900 tx=0x7f52e40894d0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:07.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.704+0000 7f52eea01640 1 --2- 172.21.15.94:0/1049667582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e335ffc0 0x5601e33623b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f52e0012af0 tx=0x7f52e0009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f491c1774849439 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:07.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.704+0000 7f52d8bfe640 1 -- 172.21.15.94:0/1049667582 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0004b20 con 0x5601e335ffc0 2024-02-17T12:00:07.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.704+0000 7f52d8bfe640 1 -- 172.21.15.94:0/1049667582 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52e000acb0 con 0x5601e335ffc0 2024-02-17T12:00:07.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.704+0000 7f52d8bfe640 1 -- 172.21.15.94:0/1049667582 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e00052a0 con 0x5601e335ffc0 2024-02-17T12:00:07.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.705+0000 7f52f3039740 1 -- 172.21.15.94:0/1049667582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e335ffc0 msgr2=0x5601e33623b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:07.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.705+0000 7f52f3039740 1 --2- 172.21.15.94:0/1049667582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e335ffc0 0x5601e33623b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f52e0012af0 tx=0x7f52e0009600 comp rx=0 tx=0).stop 2024-02-17T12:00:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.705+0000 7f52f3039740 1 -- 172.21.15.94:0/1049667582 shutdown_connections 2024-02-17T12:00:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.705+0000 7f52f3039740 1 --2- 172.21.15.94:0/1049667582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e335ffc0 0x5601e33623b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.705+0000 7f52f3039740 1 --2- 172.21.15.94:0/1049667582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e334ea70 0x5601e335fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.705+0000 7f52f3039740 1 --2- 172.21.15.94:0/1049667582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e334e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.705+0000 7f52f3039740 1 -- 172.21.15.94:0/1049667582 >> 172.21.15.94:0/1049667582 conn(0x5601e3348990 msgr2=0x5601e3348d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.707+0000 7f52f3039740 1 -- 172.21.15.94:0/1049667582 shutdown_connections 2024-02-17T12:00:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.707+0000 7f52f3039740 1 -- 172.21.15.94:0/1049667582 wait complete. 2024-02-17T12:00:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.707+0000 7f52f3039740 1 -- start start 2024-02-17T12:00:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.707+0000 7f52f3039740 1 Processor -- start 2024-02-17T12:00:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52f3039740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e3359780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52f3039740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e334ea70 0x5601e3359cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52eea01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e3359780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.715 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52f3039740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e335ffc0 0x5601e335f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52eea01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e3359780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33060/0 (socket says 172.21.15.94:33060) 2024-02-17T12:00:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52f3039740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e2f89430 con 0x5601e334e0c0 2024-02-17T12:00:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52eea01640 1 -- 172.21.15.94:0/2492609245 learned_addr learned my addr 172.21.15.94:0/2492609245 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52f3039740 1 -- 172.21.15.94:0/2492609245 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e2f45e00 con 0x5601e334ea70 2024-02-17T12:00:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52f3039740 1 -- 172.21.15.94:0/2492609245 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e323afd0 con 0x5601e335ffc0 2024-02-17T12:00:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52f3039740 10 client.0 _ll_register_callbacks cb 0x5601e3290fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52ed9ff640 1 --2- 172.21.15.94:0/2492609245 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e334ea70 0x5601e3359cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52eea01640 1 -- 172.21.15.94:0/2492609245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e335ffc0 msgr2=0x5601e335f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52eea01640 1 --2- 172.21.15.94:0/2492609245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e335ffc0 0x5601e335f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52eea01640 1 -- 172.21.15.94:0/2492609245 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e334ea70 msgr2=0x5601e3359cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52eea01640 1 --2- 172.21.15.94:0/2492609245 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e334ea70 0x5601e3359cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52eea01640 1 -- 172.21.15.94:0/2492609245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e001a040 con 0x5601e334e0c0 2024-02-17T12:00:07.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52ed9ff640 1 --2- 172.21.15.94:0/2492609245 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e334ea70 0x5601e3359cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52eea01640 1 --2- 172.21.15.94:0/2492609245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e3359780 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005100 tx=0x7f52e0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328f78eb885a2dbc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.708+0000 7f52d1ffb640 1 -- 172.21.15.94:0/2492609245 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0004e70 con 0x5601e334e0c0 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.709+0000 7f52d1ffb640 1 -- 172.21.15.94:0/2492609245 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f52e000a7d0 con 0x5601e334e0c0 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.709+0000 7f52f3039740 1 -- 172.21.15.94:0/2492609245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601e2fb57e0 con 0x5601e334e0c0 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.709+0000 7f52d1ffb640 1 -- 172.21.15.94:0/2492609245 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52e000a9b0 con 0x5601e334e0c0 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.709+0000 7f52f3039740 10 client.15996 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.709+0000 7f52f3039740 1 -- 172.21.15.94:0/2492609245 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5601e32b4610 con 0x5601e334e0c0 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.709+0000 7f52d1ffb640 1 -- 172.21.15.94:0/2492609245 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5601e2fb57e0 con 0x5601e334e0c0 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.709+0000 7f52d1ffb640 1 -- 172.21.15.94:0/2492609245 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f52e0005550 con 0x5601e334e0c0 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.709+0000 7f52f3039740 10 client.15996 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.709+0000 7f52f3039740 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.709+0000 7f52f3039740 20 client.15996 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:07.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.709+0000 7f52f3039740 1 --2- 172.21.15.94:0/2492609245 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601e3378a30 0x5601e337ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.709+0000 7f52f3039740 4 client.15996 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:07.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.709+0000 7f52f3039740 1 -- 172.21.15.94:0/2492609245 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601e2ff2720 con 0x5601e3378a30 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.710+0000 7f52ed9ff640 1 --2- 172.21.15.94:0/2492609245 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601e3378a30 0x5601e337ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.711+0000 7f52d1ffb640 10 client.15996 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.711+0000 7f52ed9ff640 1 --2- 172.21.15.94:0/2492609245 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601e3378a30 0x5601e337ae50 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=747bf7381b58ddea server_cookie=db991f05552639d5 in_seq=0 out_seq=0 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.711+0000 7f52d1ffb640 1 -- 172.21.15.94:0/2492609245 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5601e2ff2720 con 0x5601e3378a30 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.711+0000 7f52d1ffb640 10 client.15996 handle_command_reply: tid=0 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.711+0000 7f52f3039740 1 client.15996 shutdown 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.711+0000 7f52f3039740 1 -- 172.21.15.94:0/2492609245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 msgr2=0x5601e3359780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.711+0000 7f52f3039740 1 --2- 172.21.15.94:0/2492609245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e3359780 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005100 tx=0x7f52e0009f90 comp rx=0 tx=0).stop 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.712+0000 7f52f3039740 1 -- 172.21.15.94:0/2492609245 shutdown_connections 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.712+0000 7f52f3039740 1 --2- 172.21.15.94:0/2492609245 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601e3378a30 0x5601e337ae50 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.712+0000 7f52f3039740 1 --2- 172.21.15.94:0/2492609245 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e335ffc0 0x5601e335f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.712+0000 7f52f3039740 1 --2- 172.21.15.94:0/2492609245 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e334ea70 0x5601e3359cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.712+0000 7f52f3039740 1 --2- 172.21.15.94:0/2492609245 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e3359780 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.712+0000 7f52f3039740 1 -- 172.21.15.94:0/2492609245 >> 172.21.15.94:0/2492609245 conn(0x5601e3348990 msgr2=0x5601e334a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:07.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.712+0000 7f52d1ffb640 0 client.15996 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:07.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.714+0000 7f52f3039740 1 -- 172.21.15.94:0/2492609245 shutdown_connections 2024-02-17T12:00:07.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.714+0000 7f52f3039740 1 -- 172.21.15.94:0/2492609245 wait complete. 2024-02-17T12:00:07.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.714+0000 7f52f3039740 20 client.15996 trim_cache size 0 max 16384 2024-02-17T12:00:07.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.741+0000 7f52f3039740 1 Processor -- start 2024-02-17T12:00:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.741+0000 7f52f3039740 1 -- start start 2024-02-17T12:00:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.741+0000 7f52f3039740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e334e0c0 0x5601e3353450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52f3039740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334ea70 0x5601e3353990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52f3039740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e335ffc0 0x5601e3351de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52f3039740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e33440f0 con 0x5601e334ea70 2024-02-17T12:00:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52f3039740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e334d3a0 con 0x5601e335ffc0 2024-02-17T12:00:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52f3039740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e334f190 con 0x5601e334e0c0 2024-02-17T12:00:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52eea01640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e334e0c0 0x5601e3353450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52ed9ff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334ea70 0x5601e3353990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52ed9ff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334ea70 0x5601e3353990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33062/0 (socket says 172.21.15.94:33062) 2024-02-17T12:00:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52eea01640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e334e0c0 0x5601e3353450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60870/0 (socket says 172.21.15.94:60870) 2024-02-17T12:00:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52ed9ff640 1 -- 172.21.15.94:0/1454516914 learned_addr learned my addr 172.21.15.94:0/1454516914 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:07.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52eea01640 1 -- 172.21.15.94:0/1454516914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e335ffc0 msgr2=0x5601e3351de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:07.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52eea01640 1 --2- 172.21.15.94:0/1454516914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e335ffc0 0x5601e3351de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52eea01640 1 -- 172.21.15.94:0/1454516914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334ea70 msgr2=0x5601e3353990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:07.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52eea01640 1 --2- 172.21.15.94:0/1454516914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334ea70 0x5601e3353990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.742+0000 7f52eea01640 1 -- 172.21.15.94:0/1454516914 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601e3250ac0 con 0x5601e334e0c0 2024-02-17T12:00:07.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.743+0000 7f52eea01640 1 --2- 172.21.15.94:0/1454516914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e334e0c0 0x5601e3353450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005130 tx=0x7f52e0012a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f67ba8ff873d6752 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.744+0000 7f52d3fff640 1 -- 172.21.15.94:0/1454516914 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0012450 con 0x5601e334e0c0 2024-02-17T12:00:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.744+0000 7f52d3fff640 1 -- 172.21.15.94:0/1454516914 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f52e0024070 con 0x5601e334e0c0 2024-02-17T12:00:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.744+0000 7f52d3fff640 1 -- 172.21.15.94:0/1454516914 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0004470 con 0x5601e334e0c0 2024-02-17T12:00:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.745+0000 7f52f3039740 1 -- 172.21.15.94:0/1454516914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e334e0c0 msgr2=0x5601e3353450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.745+0000 7f52f3039740 1 --2- 172.21.15.94:0/1454516914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e334e0c0 0x5601e3353450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005130 tx=0x7f52e0012a70 comp rx=0 tx=0).stop 2024-02-17T12:00:07.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.745+0000 7f52f3039740 1 -- 172.21.15.94:0/1454516914 shutdown_connections 2024-02-17T12:00:07.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.745+0000 7f52f3039740 1 --2- 172.21.15.94:0/1454516914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e335ffc0 0x5601e3351de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.745+0000 7f52f3039740 1 --2- 172.21.15.94:0/1454516914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334ea70 0x5601e3353990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.745+0000 7f52f3039740 1 --2- 172.21.15.94:0/1454516914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e334e0c0 0x5601e3353450 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.745+0000 7f52f3039740 1 -- 172.21.15.94:0/1454516914 >> 172.21.15.94:0/1454516914 conn(0x5601e3348990 msgr2=0x5601e334fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:07.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.745+0000 7f52f3039740 1 -- 172.21.15.94:0/1454516914 shutdown_connections 2024-02-17T12:00:07.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.745+0000 7f52f3039740 1 -- 172.21.15.94:0/1454516914 wait complete. 2024-02-17T12:00:07.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.745+0000 7f52f3039740 1 -- start start 2024-02-17T12:00:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.746+0000 7f52f3039740 1 Processor -- start 2024-02-17T12:00:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.746+0000 7f52f3039740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e335f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.747+0000 7f52f3039740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e334ea70 0x5601e335d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.747+0000 7f52eea01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e335f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.747+0000 7f52eea01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e335f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33064/0 (socket says 172.21.15.94:33064) 2024-02-17T12:00:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.747+0000 7f52eea01640 1 -- 172.21.15.94:0/1327630767 learned_addr learned my addr 172.21.15.94:0/1327630767 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.747+0000 7f52f3039740 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e335ffc0 0x5601e335d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.747+0000 7f52f3039740 1 -- 172.21.15.94:0/1327630767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e335df10 con 0x5601e334e0c0 2024-02-17T12:00:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.747+0000 7f52f3039740 1 -- 172.21.15.94:0/1327630767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e335e080 con 0x5601e334ea70 2024-02-17T12:00:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.747+0000 7f52f3039740 1 -- 172.21.15.94:0/1327630767 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e335e240 con 0x5601e335ffc0 2024-02-17T12:00:07.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.747+0000 7f52ed9ff640 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e334ea70 0x5601e335d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.748+0000 7f52eea01640 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e335ffc0 0x5601e335d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.748+0000 7f52eea01640 1 -- 172.21.15.94:0/1327630767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e335ffc0 msgr2=0x5601e335d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.748+0000 7f52eea01640 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e335ffc0 0x5601e335d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.748+0000 7f52eea01640 1 -- 172.21.15.94:0/1327630767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e334ea70 msgr2=0x5601e335d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.748+0000 7f52eea01640 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e334ea70 0x5601e335d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.748+0000 7f52eea01640 1 -- 172.21.15.94:0/1327630767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601e32b4610 con 0x5601e334e0c0 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.748+0000 7f52f3039740 10 client.0 _ll_register_callbacks cb 0x7f52e0009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.748+0000 7f52eea01640 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e335f0f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005d70 tx=0x7f52e0004fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb93869d368b5638 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.748+0000 7f52f3039740 1 -- 172.21.15.94:0/1327630767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52e000acb0 con 0x5601e334e0c0 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.748+0000 7f52d1ffb640 10 client.16002 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.749+0000 7f52d1ffb640 1 -- 172.21.15.94:0/1327630767 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e003d070 con 0x5601e334e0c0 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.749+0000 7f52d1ffb640 1 -- 172.21.15.94:0/1327630767 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f52e0009920 con 0x5601e334e0c0 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.749+0000 7f52d1ffb640 1 -- 172.21.15.94:0/1327630767 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52e000a7d0 con 0x5601e334e0c0 2024-02-17T12:00:07.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.749+0000 7f52f3039740 10 client.16002 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:07.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.749+0000 7f52f3039740 1 -- 172.21.15.94:0/1327630767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({fsmap=21}) v3 -- 0x5601e32766a0 con 0x5601e334e0c0 2024-02-17T12:00:07.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.749+0000 7f52d1ffb640 1 -- 172.21.15.94:0/1327630767 <== mon.0 v2:172.21.15.94:3300/0 4 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f52e0047020 con 0x5601e334e0c0 2024-02-17T12:00:07.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.749+0000 7f52f3039740 10 client.16002 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:07.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.749+0000 7f52f3039740 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:07.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.749+0000 7f52f3039740 20 client.16002 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:07.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.749+0000 7f52f3039740 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601e33cfcd0 0x5601e33d20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.749+0000 7f52f3039740 4 client.16002 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:00:07.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.749+0000 7f52f3039740 1 -- 172.21.15.94:0/1327630767 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f52e000acb0 con 0x5601e33cfcd0 2024-02-17T12:00:07.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.750+0000 7f52ed9ff640 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601e33cfcd0 0x5601e33d20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.750+0000 7f52d1ffb640 10 client.16002 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:07.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.750+0000 7f52ed9ff640 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601e33cfcd0 0x5601e33d20f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=217d8a446c0b694e server_cookie=8c15d29fa918dbd7 in_seq=0 out_seq=0 2024-02-17T12:00:07.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.750+0000 7f52d1ffb640 1 -- 172.21.15.94:0/1327630767 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f52e000acb0 con 0x5601e33cfcd0 2024-02-17T12:00:07.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.750+0000 7f52d1ffb640 10 client.16002 handle_command_reply: tid=0 2024-02-17T12:00:07.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.750+0000 7f52f3039740 1 client.16002 shutdown 2024-02-17T12:00:07.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.752+0000 7f52f3039740 1 -- 172.21.15.94:0/1327630767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 msgr2=0x5601e335f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:07.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.752+0000 7f52f3039740 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e335f0f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005d70 tx=0x7f52e0004fe0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.752+0000 7f52f3039740 1 -- 172.21.15.94:0/1327630767 shutdown_connections 2024-02-17T12:00:07.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.752+0000 7f52f3039740 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601e33cfcd0 0x5601e33d20f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.752+0000 7f52f3039740 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601e335ffc0 0x5601e335d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.752+0000 7f52d1ffb640 0 client.16002 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:07.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.753+0000 7f52f3039740 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601e334ea70 0x5601e335d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.753+0000 7f52f3039740 1 --2- 172.21.15.94:0/1327630767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601e334e0c0 0x5601e335f0f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.753+0000 7f52f3039740 1 -- 172.21.15.94:0/1327630767 >> 172.21.15.94:0/1327630767 conn(0x5601e3348990 msgr2=0x5601e33533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:07.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.758+0000 7f52f3039740 1 -- 172.21.15.94:0/1327630767 shutdown_connections 2024-02-17T12:00:07.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.759+0000 7f52f3039740 1 -- 172.21.15.94:0/1327630767 wait complete. 2024-02-17T12:00:07.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.762+0000 7f52f3039740 20 client.16002 trim_cache size 0 max 16384 2024-02-17T12:00:07.762 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:00:07.762 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:00:07.762 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:00:07.762 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:00:07.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.768+0000 7f52f0464640 1 -- 172.21.15.94:0/2875891161 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f52d40a2330 msgr2=0x7f52d40a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:07.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.768+0000 7f52f0464640 1 --2- 172.21.15.94:0/2875891161 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f52d40a2330 0x7f52d40a47f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009900 tx=0x7f52e40894d0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.769+0000 7f52f0464640 1 -- 172.21.15.94:0/2875891161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52e8103590 msgr2=0x7f52e819f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:07.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.769+0000 7f52f0464640 1 --2- 172.21.15.94:0/2875891161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52e8103590 0x7f52e819f710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f52e4000c00 tx=0x7f52e4004040 comp rx=0 tx=0).stop 2024-02-17T12:00:07.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.773+0000 7f52f0464640 1 -- 172.21.15.94:0/2875891161 shutdown_connections 2024-02-17T12:00:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.773+0000 7f52f0464640 1 --2- 172.21.15.94:0/2875891161 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f52d40a2330 0x7f52d40a47f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.773+0000 7f52f0464640 1 --2- 172.21.15.94:0/2875891161 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f52e8199910 0x7f52e8199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.773+0000 7f52f0464640 1 --2- 172.21.15.94:0/2875891161 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f52e8108660 0x7f52e819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.773+0000 7f52f0464640 1 --2- 172.21.15.94:0/2875891161 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f52e8103590 0x7f52e819f710 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.773+0000 7f52f0464640 1 -- 172.21.15.94:0/2875891161 >> 172.21.15.94:0/2875891161 conn(0x7f52e80fee70 msgr2=0x7f52e8100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:07.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.776+0000 7f52f0464640 1 -- 172.21.15.94:0/2875891161 shutdown_connections 2024-02-17T12:00:07.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.779+0000 7f52f0464640 1 -- 172.21.15.94:0/2875891161 wait complete. 2024-02-17T12:00:07.798 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:00:07.799 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:00:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.951+0000 7ff9f89a7640 1 -- 172.21.15.94:0/3069625985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9f4106510 msgr2=0x7ff9f410cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.951+0000 7ff9f89a7640 1 --2- 172.21.15.94:0/3069625985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9f4106510 0x7ff9f410cf20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff9ec009cd0 tx=0x7ff9ec02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.951+0000 7ff9f89a7640 1 -- 172.21.15.94:0/3069625985 shutdown_connections 2024-02-17T12:00:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.951+0000 7ff9f89a7640 1 --2- 172.21.15.94:0/3069625985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9f4106510 0x7ff9f410cf20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.951+0000 7ff9f89a7640 1 --2- 172.21.15.94:0/3069625985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9f4101990 0x7ff9f4105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.951+0000 7ff9f89a7640 1 --2- 172.21.15.94:0/3069625985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4100fe0 0x7ff9f41013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.951+0000 7ff9f89a7640 1 -- 172.21.15.94:0/3069625985 >> 172.21.15.94:0/3069625985 conn(0x7ff9f40753a0 msgr2=0x7ff9f40757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:07.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.951+0000 7ff9f89a7640 1 -- 172.21.15.94:0/3069625985 shutdown_connections 2024-02-17T12:00:07.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.951+0000 7ff9f89a7640 1 -- 172.21.15.94:0/3069625985 wait complete. 2024-02-17T12:00:07.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.953+0000 7ff9f89a7640 1 Processor -- start 2024-02-17T12:00:07.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.953+0000 7ff9f89a7640 1 -- start start 2024-02-17T12:00:07.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.953+0000 7ff9f89a7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9f4100fe0 0x7ff9f4072cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.953+0000 7ff9f89a7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9f4106510 0x7ff9f4101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.953+0000 7ff9f89a7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4102400 0x7ff9f41028b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.953+0000 7ff9f89a7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f4105ef0 con 0x7ff9f4100fe0 2024-02-17T12:00:07.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.953+0000 7ff9f89a7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f4106060 con 0x7ff9f4102400 2024-02-17T12:00:07.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.953+0000 7ff9f89a7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f41b2f80 con 0x7ff9f4106510 2024-02-17T12:00:07.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.953+0000 7ff9f259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9f4100fe0 0x7ff9f4072cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.953+0000 7ff9f259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9f4100fe0 0x7ff9f4072cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33082/0 (socket says 172.21.15.94:33082) 2024-02-17T12:00:07.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.953+0000 7ff9f259c640 1 -- 172.21.15.94:0/2463926685 learned_addr learned my addr 172.21.15.94:0/2463926685 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:07.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.953+0000 7ff9ebfff640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9f4106510 0x7ff9f4101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.953+0000 7ff9f2d9d640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4102400 0x7ff9f41028b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.954+0000 7ff9f259c640 1 -- 172.21.15.94:0/2463926685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9f4106510 msgr2=0x7ff9f4101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:07.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.954+0000 7ff9f259c640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9f4106510 0x7ff9f4101ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.954+0000 7ff9f259c640 1 -- 172.21.15.94:0/2463926685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4102400 msgr2=0x7ff9f41028b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:07.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.954+0000 7ff9f259c640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4102400 0x7ff9f41028b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:07.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.954+0000 7ff9f259c640 1 -- 172.21.15.94:0/2463926685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec009900 con 0x7ff9f4100fe0 2024-02-17T12:00:07.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.954+0000 7ff9ebfff640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9f4106510 0x7ff9f4101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:00:07.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.954+0000 7ff9f259c640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9f4100fe0 0x7ff9f4072cc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400e9f0 tx=0x7ff9e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:07.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.954+0000 7ff9eb7fe640 1 -- 172.21.15.94:0/2463926685 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4009700 con 0x7ff9f4100fe0 2024-02-17T12:00:07.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.954+0000 7ff9f89a7640 1 -- 172.21.15.94:0/2463926685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f41b3260 con 0x7ff9f4100fe0 2024-02-17T12:00:07.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.954+0000 7ff9eb7fe640 1 -- 172.21.15.94:0/2463926685 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff9e400cd20 con 0x7ff9f4100fe0 2024-02-17T12:00:07.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.954+0000 7ff9eb7fe640 1 -- 172.21.15.94:0/2463926685 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4018860 con 0x7ff9f4100fe0 2024-02-17T12:00:07.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.955+0000 7ff9f89a7640 1 -- 172.21.15.94:0/2463926685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f41b37c0 con 0x7ff9f4100fe0 2024-02-17T12:00:07.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.957+0000 7ff9f89a7640 1 -- 172.21.15.94:0/2463926685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f4067cd0 con 0x7ff9f4100fe0 2024-02-17T12:00:07.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.958+0000 7ff9eb7fe640 1 -- 172.21.15.94:0/2463926685 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff9e4010af0 con 0x7ff9f4100fe0 2024-02-17T12:00:07.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.958+0000 7ff9eb7fe640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff9e00a22f0 0x7ff9e00a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:07.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.959+0000 7ff9eb7fe640 1 -- 172.21.15.94:0/2463926685 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff9e4014070 con 0x7ff9f4100fe0 2024-02-17T12:00:07.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.959+0000 7ff9ebfff640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff9e00a22f0 0x7ff9e00a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:07.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.960+0000 7ff9ebfff640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff9e00a22f0 0x7ff9e00a47b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc005fd0 tx=0x7ff9dc00b390 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:07.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:07.965+0000 7ff9eb7fe640 1 -- 172.21.15.94:0/2463926685 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9e4096f40 con 0x7ff9f4100fe0 2024-02-17T12:00:08.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:07 smithi189 ceph-mon[36588]: pgmap v63: 97 pgs: 97 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.4 MiB/s wr, 636 op/s 2024-02-17T12:00:08.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:07 smithi125 ceph-mon[36819]: pgmap v63: 97 pgs: 97 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.4 MiB/s wr, 636 op/s 2024-02-17T12:00:08.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:07 smithi094 ceph-mon[32499]: pgmap v63: 97 pgs: 97 active+clean; 1.3 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.4 MiB/s wr, 636 op/s 2024-02-17T12:00:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.281+0000 7fb61f59e640 1 -- 172.21.15.94:0/3242485439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6200fff20 msgr2=0x7fb620100300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.281+0000 7fb61f59e640 1 --2- 172.21.15.94:0/3242485439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6200fff20 0x7fb620100300 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb610009900 tx=0x7fb61002bd10 comp rx=0 tx=0).stop 2024-02-17T12:00:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.281+0000 7fb61f59e640 1 -- 172.21.15.94:0/3242485439 shutdown_connections 2024-02-17T12:00:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.281+0000 7fb61f59e640 1 --2- 172.21.15.94:0/3242485439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb620105320 0x7fb6201057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.281+0000 7fb61f59e640 1 --2- 172.21.15.94:0/3242485439 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb620100840 0x7fb620104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.281+0000 7fb61f59e640 1 --2- 172.21.15.94:0/3242485439 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb6200fff20 0x7fb620100300 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.281+0000 7fb61f59e640 1 -- 172.21.15.94:0/3242485439 >> 172.21.15.94:0/3242485439 conn(0x7fb6200fb990 msgr2=0x7fb6200fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.281+0000 7fb61f59e640 1 -- 172.21.15.94:0/3242485439 shutdown_connections 2024-02-17T12:00:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.281+0000 7fb61f59e640 1 -- 172.21.15.94:0/3242485439 wait complete. 2024-02-17T12:00:08.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.282+0000 7fb61f59e640 1 Processor -- start 2024-02-17T12:00:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.282+0000 7fb61f59e640 1 -- start start 2024-02-17T12:00:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.282+0000 7fb61f59e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6200fff20 0x7fb6201a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.282+0000 7fb61f59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb620100840 0x7fb6201a3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.282+0000 7fb61f59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb620105320 0x7fb62019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.282+0000 7fb61f59e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62019dff0 con 0x7fb620105320 2024-02-17T12:00:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.282+0000 7fb61f59e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62019e160 con 0x7fb620100840 2024-02-17T12:00:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.282+0000 7fb61f59e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62019e320 con 0x7fb6200fff20 2024-02-17T12:00:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.283+0000 7fb61dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb620100840 0x7fb6201a3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.283+0000 7fb61dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb620100840 0x7fb6201a3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59116/0 (socket says 172.21.15.94:59116) 2024-02-17T12:00:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.283+0000 7fb61dd9b640 1 -- 172.21.15.94:0/3251760382 learned_addr learned my addr 172.21.15.94:0/3251760382 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.283+0000 7fb61ed9d640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb620105320 0x7fb62019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.283+0000 7fb61e59c640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6200fff20 0x7fb6201a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.283+0000 7fb61ed9d640 1 -- 172.21.15.94:0/3251760382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6200fff20 msgr2=0x7fb6201a37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.284+0000 7fb61ed9d640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6200fff20 0x7fb6201a37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.284+0000 7fb61ed9d640 1 -- 172.21.15.94:0/3251760382 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb620100840 msgr2=0x7fb6201a3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.284+0000 7fb61ed9d640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb620100840 0x7fb6201a3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.284+0000 7fb61ed9d640 1 -- 172.21.15.94:0/3251760382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb610009560 con 0x7fb620105320 2024-02-17T12:00:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.284+0000 7fb61e59c640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6200fff20 0x7fb6201a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:00:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.284+0000 7fb61ed9d640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb620105320 0x7fb62019d9a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb61400e970 tx=0x7fb61400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.284+0000 7fb60f7fe640 1 -- 172.21.15.94:0/3251760382 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614009620 con 0x7fb620105320 2024-02-17T12:00:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.284+0000 7fb61f59e640 1 -- 172.21.15.94:0/3251760382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb62019e600 con 0x7fb620105320 2024-02-17T12:00:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.284+0000 7fb61f59e640 1 -- 172.21.15.94:0/3251760382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6201aa890 con 0x7fb620105320 2024-02-17T12:00:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.285+0000 7fb60f7fe640 1 -- 172.21.15.94:0/3251760382 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb61400ccc0 con 0x7fb620105320 2024-02-17T12:00:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.285+0000 7fb60f7fe640 1 -- 172.21.15.94:0/3251760382 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6140049a0 con 0x7fb620105320 2024-02-17T12:00:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.287+0000 7fb60f7fe640 1 -- 172.21.15.94:0/3251760382 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb614004b00 con 0x7fb620105320 2024-02-17T12:00:08.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.287+0000 7fb62741e740 1 Processor -- start 2024-02-17T12:00:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.288+0000 7fb62741e740 1 -- start start 2024-02-17T12:00:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.288+0000 7fb60f7fe640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb6080a2320 0x7fb6080a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.288+0000 7fb62741e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 0x560e6e3689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.288+0000 7fb62741e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e368ef0 0x560e6e369370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.288+0000 7fb62741e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e372470 0x560e6e374860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.288+0000 7fb62741e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6e2d79f0 con 0x560e6e372470 2024-02-17T12:00:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.288+0000 7fb62741e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6e2d4f20 con 0x560e6e368ef0 2024-02-17T12:00:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.288+0000 7fb62741e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6e2d0700 con 0x560e6e369c60 2024-02-17T12:00:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.289+0000 7fb60f7fe640 1 -- 172.21.15.94:0/3251760382 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb614014070 con 0x7fb620105320 2024-02-17T12:00:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.289+0000 7fb61dd9b640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb6080a2320 0x7fb6080a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.289+0000 7fb61ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e372470 0x560e6e374860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.289+0000 7fb61ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e372470 0x560e6e374860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33114/0 (socket says 172.21.15.94:33114) 2024-02-17T12:00:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.289+0000 7fb61ed9d640 1 -- 172.21.15.94:0/3617881700 learned_addr learned my addr 172.21.15.94:0/3617881700 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.289+0000 7fb61dd9b640 1 --2- 172.21.15.94:0/3617881700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e368ef0 0x560e6e369370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.289+0000 7fb61ed9d640 1 -- 172.21.15.94:0/3617881700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 msgr2=0x560e6e3689b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:00:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.289+0000 7fb61ed9d640 1 --2- 172.21.15.94:0/3617881700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 0x560e6e3689b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.289+0000 7fb61ed9d640 1 -- 172.21.15.94:0/3617881700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e368ef0 msgr2=0x560e6e369370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.289+0000 7fb61ed9d640 1 --2- 172.21.15.94:0/3617881700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e368ef0 0x560e6e369370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.289+0000 7fb61ed9d640 1 -- 172.21.15.94:0/3617881700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e6e2667b0 con 0x560e6e372470 2024-02-17T12:00:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.289+0000 7fb61ed9d640 1 --2- 172.21.15.94:0/3617881700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e372470 0x560e6e374860 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb614097270 tx=0x7fb61409a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b313618eae9c1f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.289+0000 7fb61dd9b640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb6080a2320 0x7fb6080a47e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb618009fd0 tx=0x7fb618009f60 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.290+0000 7fb60cff9640 1 -- 172.21.15.94:0/3617881700 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61409afa0 con 0x560e6e372470 2024-02-17T12:00:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.290+0000 7fb60cff9640 1 -- 172.21.15.94:0/3617881700 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb61409a8a0 con 0x560e6e372470 2024-02-17T12:00:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.290+0000 7fb60cff9640 1 -- 172.21.15.94:0/3617881700 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6140cb880 con 0x560e6e372470 2024-02-17T12:00:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.291+0000 7fb62741e740 1 -- 172.21.15.94:0/3617881700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e372470 msgr2=0x560e6e374860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.291+0000 7fb62741e740 1 --2- 172.21.15.94:0/3617881700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e372470 0x560e6e374860 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb614097270 tx=0x7fb61409a200 comp rx=0 tx=0).stop 2024-02-17T12:00:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.291+0000 7fb62741e740 1 -- 172.21.15.94:0/3617881700 shutdown_connections 2024-02-17T12:00:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.291+0000 7fb62741e740 1 --2- 172.21.15.94:0/3617881700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e372470 0x560e6e374860 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.291+0000 7fb62741e740 1 --2- 172.21.15.94:0/3617881700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e368ef0 0x560e6e369370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.291+0000 7fb62741e740 1 --2- 172.21.15.94:0/3617881700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 0x560e6e3689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.291+0000 7fb62741e740 1 -- 172.21.15.94:0/3617881700 >> 172.21.15.94:0/3617881700 conn(0x560e6e330550 msgr2=0x560e6e363be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.291+0000 7fb62741e740 1 -- 172.21.15.94:0/3617881700 shutdown_connections 2024-02-17T12:00:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.291+0000 7fb62741e740 1 -- 172.21.15.94:0/3617881700 wait complete. 2024-02-17T12:00:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.291+0000 7fb62741e740 1 -- start start 2024-02-17T12:00:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.292+0000 7fb62741e740 1 Processor -- start 2024-02-17T12:00:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.292+0000 7fb62741e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e368ef0 0x560e6e36f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.292+0000 7fb62741e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 0x560e6e36fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.292+0000 7fb62741e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e372470 0x560e6e36ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.292+0000 7fb62741e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6e275bf0 con 0x560e6e372470 2024-02-17T12:00:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.292+0000 7fb62741e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6dfa4250 con 0x560e6e368ef0 2024-02-17T12:00:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.292+0000 7fb62741e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6e256200 con 0x560e6e369c60 2024-02-17T12:00:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.292+0000 7fb62741e740 10 client.0 _ll_register_callbacks cb 0x560e6e2c4d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.292+0000 7fb61dd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 0x560e6e36fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.292+0000 7fb61e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e368ef0 0x560e6e36f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.292+0000 7fb61e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e368ef0 0x560e6e36f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59128/0 (socket says 172.21.15.94:59128) 2024-02-17T12:00:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.292+0000 7fb61e59c640 1 -- 172.21.15.94:0/2701394226 learned_addr learned my addr 172.21.15.94:0/2701394226 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.293+0000 7fb61ed9d640 1 --2- 172.21.15.94:0/2701394226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e372470 0x560e6e36ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.293+0000 7fb61e59c640 1 -- 172.21.15.94:0/2701394226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 msgr2=0x560e6e36fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.293+0000 7fb61e59c640 1 --2- 172.21.15.94:0/2701394226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 0x560e6e36fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.293+0000 7fb61e59c640 1 -- 172.21.15.94:0/2701394226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e372470 msgr2=0x560e6e36ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.293+0000 7fb61e59c640 1 --2- 172.21.15.94:0/2701394226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e372470 0x560e6e36ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.293+0000 7fb61e59c640 1 -- 172.21.15.94:0/2701394226 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6201aa890 con 0x560e6e368ef0 2024-02-17T12:00:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.293+0000 7fb61ed9d640 1 --2- 172.21.15.94:0/2701394226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e372470 0x560e6e36ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:00:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.293+0000 7fb605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:00:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.293+0000 7fb61e59c640 1 --2- 172.21.15.94:0/2701394226 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e368ef0 0x560e6e36f4e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb610005ab0 tx=0x7fb61002cd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c60520d35d4ad199 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.293+0000 7fb605ffb640 1 -- 172.21.15.94:0/2701394226 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610005220 con 0x560e6e368ef0 2024-02-17T12:00:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.294+0000 7fb605ffb640 1 -- 172.21.15.94:0/2701394226 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb610004df0 con 0x560e6e368ef0 2024-02-17T12:00:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.294+0000 7fb62741e740 1 -- 172.21.15.94:0/2701394226 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e6dfd0600 con 0x560e6e368ef0 2024-02-17T12:00:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.294+0000 7fb605ffb640 1 -- 172.21.15.94:0/2701394226 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb61002c650 con 0x560e6e368ef0 2024-02-17T12:00:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.295+0000 7fb62741e740 10 client.25469 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.295+0000 7fb62741e740 1 -- 172.21.15.94:0/2701394226 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x560e6e2ced80 con 0x560e6e368ef0 2024-02-17T12:00:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.295+0000 7fb605ffb640 1 -- 172.21.15.94:0/2701394226 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560e6dfd0600 con 0x560e6e368ef0 2024-02-17T12:00:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.295+0000 7fb605ffb640 1 -- 172.21.15.94:0/2701394226 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb6100037b0 con 0x560e6e368ef0 2024-02-17T12:00:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.295+0000 7fb62741e740 10 client.25469 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.295+0000 7fb62741e740 10 client.25469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.295+0000 7fb62741e740 20 client.25469 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.295+0000 7fb62741e740 1 --2- 172.21.15.94:0/2701394226 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e6e3829c0 0x560e6e384e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.295+0000 7fb62741e740 4 client.25469 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.295+0000 7fb62741e740 1 -- 172.21.15.94:0/2701394226 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e6e0211b0 con 0x560e6e3829c0 2024-02-17T12:00:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.296+0000 7fb61dd9b640 1 --2- 172.21.15.94:0/2701394226 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e6e3829c0 0x560e6e384e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.296+0000 7fb605ffb640 10 client.25469 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.296+0000 7fb61dd9b640 1 --2- 172.21.15.94:0/2701394226 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e6e3829c0 0x560e6e384e00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0400b74f866a8c server_cookie=c463727da858aab6 in_seq=0 out_seq=0 2024-02-17T12:00:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb605ffb640 1 -- 172.21.15.94:0/2701394226 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e6e0211b0 con 0x560e6e3829c0 2024-02-17T12:00:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb605ffb640 10 client.25469 handle_command_reply: tid=0 2024-02-17T12:00:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb62741e740 1 client.25469 shutdown 2024-02-17T12:00:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb62741e740 1 -- 172.21.15.94:0/2701394226 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e368ef0 msgr2=0x560e6e36f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb62741e740 1 --2- 172.21.15.94:0/2701394226 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e368ef0 0x560e6e36f4e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb610005ab0 tx=0x7fb61002cd20 comp rx=0 tx=0).stop 2024-02-17T12:00:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb62741e740 1 -- 172.21.15.94:0/2701394226 shutdown_connections 2024-02-17T12:00:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb62741e740 1 --2- 172.21.15.94:0/2701394226 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e6e3829c0 0x560e6e384e00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb62741e740 1 --2- 172.21.15.94:0/2701394226 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e372470 0x560e6e36ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb62741e740 1 --2- 172.21.15.94:0/2701394226 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 0x560e6e36fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb62741e740 1 --2- 172.21.15.94:0/2701394226 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e368ef0 0x560e6e36f4e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb62741e740 1 -- 172.21.15.94:0/2701394226 >> 172.21.15.94:0/2701394226 conn(0x560e6e330550 msgr2=0x560e6e365930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb605ffb640 0 client.25469 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb62741e740 1 -- 172.21.15.94:0/2701394226 shutdown_connections 2024-02-17T12:00:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb62741e740 1 -- 172.21.15.94:0/2701394226 wait complete. 2024-02-17T12:00:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.297+0000 7fb62741e740 20 client.25469 trim_cache size 0 max 16384 2024-02-17T12:00:08.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.316+0000 7fb62741e740 1 Processor -- start 2024-02-17T12:00:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.316+0000 7fb62741e740 1 -- start start 2024-02-17T12:00:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.316+0000 7fb62741e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e368ef0 0x560e6e36ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.316+0000 7fb62741e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 0x560e6e362c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.316+0000 7fb62741e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e372470 0x560e6e36c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.316+0000 7fb62741e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6e37e300 con 0x560e6e368ef0 2024-02-17T12:00:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.316+0000 7fb62741e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6e375080 con 0x560e6e372470 2024-02-17T12:00:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.316+0000 7fb62741e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6e3681c0 con 0x560e6e369c60 2024-02-17T12:00:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.316+0000 7fb61e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e368ef0 0x560e6e36ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.316+0000 7fb61e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e368ef0 0x560e6e36ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33136/0 (socket says 172.21.15.94:33136) 2024-02-17T12:00:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.316+0000 7fb61e59c640 1 -- 172.21.15.94:0/2102172772 learned_addr learned my addr 172.21.15.94:0/2102172772 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.316+0000 7fb61dd9b640 1 --2- 172.21.15.94:0/2102172772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 0x560e6e362c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.316+0000 7fb61ed9d640 1 --2- 172.21.15.94:0/2102172772 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e372470 0x560e6e36c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.317+0000 7fb61e59c640 1 -- 172.21.15.94:0/2102172772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 msgr2=0x560e6e362c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.317+0000 7fb61e59c640 1 --2- 172.21.15.94:0/2102172772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 0x560e6e362c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.317+0000 7fb61e59c640 1 -- 172.21.15.94:0/2102172772 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e372470 msgr2=0x560e6e36c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.317+0000 7fb61e59c640 1 --2- 172.21.15.94:0/2102172772 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e372470 0x560e6e36c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.317+0000 7fb61e59c640 1 -- 172.21.15.94:0/2102172772 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e6e26b7b0 con 0x560e6e368ef0 2024-02-17T12:00:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.317+0000 7fb61ed9d640 1 --2- 172.21.15.94:0/2102172772 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e372470 0x560e6e36c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:00:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.317+0000 7fb61e59c640 1 --2- 172.21.15.94:0/2102172772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e368ef0 0x560e6e36ede0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb61003f000 tx=0x7fb61002bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226552a24a936f68 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.317+0000 7fb607fff640 1 -- 172.21.15.94:0/2102172772 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610005220 con 0x560e6e368ef0 2024-02-17T12:00:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.317+0000 7fb607fff640 1 -- 172.21.15.94:0/2102172772 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb610009780 con 0x560e6e368ef0 2024-02-17T12:00:08.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.317+0000 7fb607fff640 1 -- 172.21.15.94:0/2102172772 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610051510 con 0x560e6e368ef0 2024-02-17T12:00:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.318+0000 7fb62741e740 1 -- 172.21.15.94:0/2102172772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e368ef0 msgr2=0x560e6e36ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.318+0000 7fb62741e740 1 --2- 172.21.15.94:0/2102172772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e368ef0 0x560e6e36ede0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb61003f000 tx=0x7fb61002bd10 comp rx=0 tx=0).stop 2024-02-17T12:00:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.318+0000 7fb62741e740 1 -- 172.21.15.94:0/2102172772 shutdown_connections 2024-02-17T12:00:08.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.318+0000 7fb62741e740 1 --2- 172.21.15.94:0/2102172772 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e372470 0x560e6e36c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.318+0000 7fb62741e740 1 --2- 172.21.15.94:0/2102172772 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e369c60 0x560e6e362c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.318+0000 7fb62741e740 1 --2- 172.21.15.94:0/2102172772 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e368ef0 0x560e6e36ede0 secure :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb61003f000 tx=0x7fb61002bd10 comp rx=0 tx=0).stop 2024-02-17T12:00:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.318+0000 7fb62741e740 1 -- 172.21.15.94:0/2102172772 >> 172.21.15.94:0/2102172772 conn(0x560e6e330550 msgr2=0x560e6e362550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.319+0000 7fb62741e740 1 -- 172.21.15.94:0/2102172772 shutdown_connections 2024-02-17T12:00:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.319+0000 7fb62741e740 1 -- 172.21.15.94:0/2102172772 wait complete. 2024-02-17T12:00:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.320+0000 7fb62741e740 1 -- start start 2024-02-17T12:00:08.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.320+0000 7fb62741e740 1 Processor -- start 2024-02-17T12:00:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.320+0000 7fb62741e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e369c60 0x560e6e377c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.320+0000 7fb62741e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e372470 0x560e6e378180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.320+0000 7fb62741e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e3786c0 0x560e6e3d8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.320+0000 7fb62741e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6e379a80 con 0x560e6e3786c0 2024-02-17T12:00:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.320+0000 7fb62741e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6e379bf0 con 0x560e6e369c60 2024-02-17T12:00:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.320+0000 7fb62741e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e6e3d9460 con 0x560e6e372470 2024-02-17T12:00:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.320+0000 7fb62741e740 10 client.0 _ll_register_callbacks cb 0x7fb614005740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb61ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e3786c0 0x560e6e3d8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb61ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e3786c0 0x560e6e3d8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33138/0 (socket says 172.21.15.94:33138) 2024-02-17T12:00:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb61ed9d640 1 -- 172.21.15.94:0/2926174729 learned_addr learned my addr 172.21.15.94:0/2926174729 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb61e59c640 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e369c60 0x560e6e377c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb61dd9b640 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e372470 0x560e6e378180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb61e59c640 1 -- 172.21.15.94:0/2926174729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e372470 msgr2=0x560e6e378180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb61e59c640 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e372470 0x560e6e378180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb61e59c640 1 -- 172.21.15.94:0/2926174729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e3786c0 msgr2=0x560e6e3d8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb61e59c640 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e3786c0 0x560e6e3d8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb61e59c640 1 -- 172.21.15.94:0/2926174729 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e6e2ced80 con 0x560e6e369c60 2024-02-17T12:00:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb61ed9d640 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e3786c0 0x560e6e3d8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb61e59c640 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e369c60 0x560e6e377c40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb61002cfd0 tx=0x7fb610005180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e15ef288bd47ae server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb605ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:00:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.321+0000 7fb605ffb640 1 -- 172.21.15.94:0/2926174729 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61005a070 con 0x560e6e369c60 2024-02-17T12:00:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.322+0000 7fb605ffb640 1 -- 172.21.15.94:0/2926174729 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb6100035f0 con 0x560e6e369c60 2024-02-17T12:00:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.322+0000 7fb62741e740 1 -- 172.21.15.94:0/2926174729 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb61409a8a0 con 0x560e6e369c60 2024-02-17T12:00:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.322+0000 7fb605ffb640 1 -- 172.21.15.94:0/2926174729 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb610005790 con 0x560e6e369c60 2024-02-17T12:00:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.322+0000 7fb62741e740 10 client.25472 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:08.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.322+0000 7fb62741e740 1 -- 172.21.15.94:0/2926174729 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x560e6e291540 con 0x560e6e369c60 2024-02-17T12:00:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.322+0000 7fb605ffb640 1 -- 172.21.15.94:0/2926174729 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb61409a8a0 con 0x560e6e369c60 2024-02-17T12:00:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.322+0000 7fb605ffb640 1 -- 172.21.15.94:0/2926174729 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb61005d020 con 0x560e6e369c60 2024-02-17T12:00:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.322+0000 7fb62741e740 10 client.25472 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.322+0000 7fb62741e740 10 client.25472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.322+0000 7fb62741e740 20 client.25472 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.322+0000 7fb62741e740 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e6e3da6d0 0x560e6e3dcac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.322+0000 7fb62741e740 4 client.25472 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:00:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.322+0000 7fb62741e740 1 -- 172.21.15.94:0/2926174729 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560e6e00d550 con 0x560e6e3da6d0 2024-02-17T12:00:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.323+0000 7fb61dd9b640 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e6e3da6d0 0x560e6e3dcac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.323+0000 7fb605ffb640 10 client.25472 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.323+0000 7fb61dd9b640 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e6e3da6d0 0x560e6e3dcac0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bad6309d9cd6efe server_cookie=a8e9a3dc6265edb1 in_seq=0 out_seq=0 2024-02-17T12:00:08.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.334+0000 7ff9f89a7640 1 -- 172.21.15.94:0/2463926685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9f40676e0 con 0x7ff9f4100fe0 2024-02-17T12:00:08.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.335+0000 7ff9eb7fe640 1 -- 172.21.15.94:0/2463926685 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11346 (secure 0 0 0) 0x7ff9e409a600 con 0x7ff9f4100fe0 2024-02-17T12:00:08.334 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:00:08.334 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:57:51.854453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:00:08.334 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 21 2024-02-17T12:00:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.338+0000 7ff9f89a7640 1 -- 172.21.15.94:0/2463926685 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff9e00a22f0 msgr2=0x7ff9e00a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.338+0000 7ff9f89a7640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff9e00a22f0 0x7ff9e00a47b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc005fd0 tx=0x7ff9dc00b390 comp rx=0 tx=0).stop 2024-02-17T12:00:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.338+0000 7ff9f89a7640 1 -- 172.21.15.94:0/2463926685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9f4100fe0 msgr2=0x7ff9f4072cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.339+0000 7ff9f89a7640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9f4100fe0 0x7ff9f4072cc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff9e400e9f0 tx=0x7ff9e400eeb0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.339+0000 7ff9f89a7640 1 -- 172.21.15.94:0/2463926685 shutdown_connections 2024-02-17T12:00:08.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.339+0000 7ff9f89a7640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff9e00a22f0 0x7ff9e00a47b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.339+0000 7ff9f89a7640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9f4102400 0x7ff9f41028b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.339+0000 7ff9f89a7640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff9f4106510 0x7ff9f4101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.339+0000 7ff9f89a7640 1 --2- 172.21.15.94:0/2463926685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9f4100fe0 0x7ff9f4072cc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.340+0000 7ff9f89a7640 1 -- 172.21.15.94:0/2463926685 >> 172.21.15.94:0/2463926685 conn(0x7ff9f40753a0 msgr2=0x7ff9f410e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.340+0000 7ff9f89a7640 1 -- 172.21.15.94:0/2463926685 shutdown_connections 2024-02-17T12:00:08.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.340+0000 7ff9f89a7640 1 -- 172.21.15.94:0/2463926685 wait complete. 2024-02-17T12:00:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.341+0000 7fb605ffb640 1 -- 172.21.15.94:0/2926174729 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560e6e00d550 con 0x560e6e3da6d0 2024-02-17T12:00:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.341+0000 7fb605ffb640 10 client.25472 handle_command_reply: tid=0 2024-02-17T12:00:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.341+0000 7fb62741e740 1 client.25472 shutdown 2024-02-17T12:00:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.341+0000 7fb62741e740 1 -- 172.21.15.94:0/2926174729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e369c60 msgr2=0x560e6e377c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.341+0000 7fb62741e740 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e369c60 0x560e6e377c40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb61002cfd0 tx=0x7fb610005180 comp rx=0 tx=0).stop 2024-02-17T12:00:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.341+0000 7fb62741e740 1 -- 172.21.15.94:0/2926174729 shutdown_connections 2024-02-17T12:00:08.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.341+0000 7fb62741e740 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e6e3da6d0 0x560e6e3dcac0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.341+0000 7fb62741e740 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e6e3786c0 0x560e6e3d8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.341+0000 7fb62741e740 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e6e372470 0x560e6e378180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.341+0000 7fb62741e740 1 --2- 172.21.15.94:0/2926174729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e6e369c60 0x560e6e377c40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.341+0000 7fb62741e740 1 -- 172.21.15.94:0/2926174729 >> 172.21.15.94:0/2926174729 conn(0x560e6e330550 msgr2=0x560e6e362de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.341+0000 7fb605ffb640 0 client.25472 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.342+0000 7fb62741e740 1 -- 172.21.15.94:0/2926174729 shutdown_connections 2024-02-17T12:00:08.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.342+0000 7fb62741e740 1 -- 172.21.15.94:0/2926174729 wait complete. 2024-02-17T12:00:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.342+0000 7fb62741e740 20 client.25472 trim_cache size 0 max 16384 2024-02-17T12:00:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:00:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:00:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:00:08.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:00:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.343+0000 7fb61f59e640 1 -- 172.21.15.94:0/3251760382 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb6080a2320 msgr2=0x7fb6080a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.343+0000 7fb61f59e640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb6080a2320 0x7fb6080a47e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb618009fd0 tx=0x7fb618009f60 comp rx=0 tx=0).stop 2024-02-17T12:00:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.343+0000 7fb61f59e640 1 -- 172.21.15.94:0/3251760382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb620105320 msgr2=0x7fb62019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.344+0000 7fb61f59e640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb620105320 0x7fb62019d9a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb61400e970 tx=0x7fb61400ee30 comp rx=0 tx=0).stop 2024-02-17T12:00:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.344+0000 7fb61f59e640 1 -- 172.21.15.94:0/3251760382 shutdown_connections 2024-02-17T12:00:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.344+0000 7fb61f59e640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb6080a2320 0x7fb6080a47e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.344+0000 7fb61f59e640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb620105320 0x7fb62019d9a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.344+0000 7fb61f59e640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb620100840 0x7fb6201a3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.344+0000 7fb61f59e640 1 --2- 172.21.15.94:0/3251760382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6200fff20 0x7fb6201a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.344+0000 7fb61f59e640 1 -- 172.21.15.94:0/3251760382 >> 172.21.15.94:0/3251760382 conn(0x7fb6200fb990 msgr2=0x7fb6200fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.344+0000 7fb61f59e640 1 -- 172.21.15.94:0/3251760382 shutdown_connections 2024-02-17T12:00:08.344 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:08.344+0000 7fb61f59e640 1 -- 172.21.15.94:0/3251760382 wait complete. 2024-02-17T12:00:08.357 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:00:08.357 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:00:08.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.515+0000 7f02d728f640 1 -- 172.21.15.94:0/2767213581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d0105370 msgr2=0x7f02d010cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.515+0000 7f02d728f640 1 --2- 172.21.15.94:0/2767213581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d0105370 0x7f02d010cf40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f02c8009cd0 tx=0x7f02c802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.515+0000 7f02d728f640 1 -- 172.21.15.94:0/2767213581 shutdown_connections 2024-02-17T12:00:08.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.515+0000 7f02d728f640 1 --2- 172.21.15.94:0/2767213581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d0105370 0x7f02d010cf40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.515+0000 7f02d728f640 1 --2- 172.21.15.94:0/2767213581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02d0100ca0 0x7f02d0104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.515+0000 7f02d728f640 1 --2- 172.21.15.94:0/2767213581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02d01002f0 0x7f02d01006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.515+0000 7f02d728f640 1 -- 172.21.15.94:0/2767213581 >> 172.21.15.94:0/2767213581 conn(0x7f02d00771e0 msgr2=0x7f02d00775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.517+0000 7f02d728f640 1 -- 172.21.15.94:0/2767213581 shutdown_connections 2024-02-17T12:00:08.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.517+0000 7f02d728f640 1 -- 172.21.15.94:0/2767213581 wait complete. 2024-02-17T12:00:08.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.518+0000 7f02d728f640 1 Processor -- start 2024-02-17T12:00:08.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.518+0000 7f02d728f640 1 -- start start 2024-02-17T12:00:08.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.518+0000 7f02d728f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02d01002f0 0x7f02d019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.518+0000 7f02d728f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d0105370 0x7f02d019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.518+0000 7f02d728f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02d0199810 0x7f02d0199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.518+0000 7f02d728f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d019a2c0 con 0x7f02d0105370 2024-02-17T12:00:08.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.518+0000 7f02d728f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d019a430 con 0x7f02d0199810 2024-02-17T12:00:08.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.518+0000 7f02d728f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d019a5a0 con 0x7f02d01002f0 2024-02-17T12:00:08.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.518+0000 7f02d502b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02d01002f0 0x7f02d019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.518+0000 7f02d502b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02d01002f0 0x7f02d019f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60974/0 (socket says 172.21.15.94:60974) 2024-02-17T12:00:08.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.518+0000 7f02d502b640 1 -- 172.21.15.94:0/1905456672 learned_addr learned my addr 172.21.15.94:0/1905456672 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.519+0000 7f02d582c640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02d0199810 0x7f02d0199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.519+0000 7f02d482a640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d0105370 0x7f02d019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.519+0000 7f02d502b640 1 -- 172.21.15.94:0/1905456672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02d0199810 msgr2=0x7f02d0199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.519+0000 7f02d502b640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02d0199810 0x7f02d0199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.519+0000 7f02d502b640 1 -- 172.21.15.94:0/1905456672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d0105370 msgr2=0x7f02d019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.519+0000 7f02d502b640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d0105370 0x7f02d019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.519+0000 7f02d502b640 1 -- 172.21.15.94:0/1905456672 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c8009900 con 0x7f02d01002f0 2024-02-17T12:00:08.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.519+0000 7f02d582c640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02d0199810 0x7f02d0199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:08.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.519+0000 7f02d482a640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d0105370 0x7f02d019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:00:08.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.520+0000 7f02d502b640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02d01002f0 0x7f02d019f650 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00d950 tx=0x7f02cc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.520+0000 7f02c67fc640 1 -- 172.21.15.94:0/1905456672 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc009a30 con 0x7f02d01002f0 2024-02-17T12:00:08.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.520+0000 7f02c67fc640 1 -- 172.21.15.94:0/1905456672 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02cc009b90 con 0x7f02d01002f0 2024-02-17T12:00:08.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.520+0000 7f02c67fc640 1 -- 172.21.15.94:0/1905456672 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc004a00 con 0x7f02d01002f0 2024-02-17T12:00:08.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.520+0000 7f02d728f640 1 -- 172.21.15.94:0/1905456672 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d01a6500 con 0x7f02d01002f0 2024-02-17T12:00:08.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.520+0000 7f02d728f640 1 -- 172.21.15.94:0/1905456672 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d01a69e0 con 0x7f02d01002f0 2024-02-17T12:00:08.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.523+0000 7f02c67fc640 1 -- 172.21.15.94:0/1905456672 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f02cc004b60 con 0x7f02d01002f0 2024-02-17T12:00:08.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.523+0000 7f02d9e5d740 1 Processor -- start 2024-02-17T12:00:08.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.523+0000 7f02d9e5d740 1 -- start start 2024-02-17T12:00:08.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.523+0000 7f02d9e5d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 0x55978b17c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.523+0000 7f02d9e5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17cb70 0x55978b185210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.523+0000 7f02d9e5d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b185750 0x55978b187b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.523+0000 7f02d9e5d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978b137960 con 0x55978b17c250 2024-02-17T12:00:08.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.523+0000 7f02d9e5d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978b0eb310 con 0x55978b185750 2024-02-17T12:00:08.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.523+0000 7f02d9e5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978b0e8840 con 0x55978b17cb70 2024-02-17T12:00:08.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.524+0000 7f02d482a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b185750 0x55978b187b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.524+0000 7f02c67fc640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02bc0a2380 0x7f02bc0a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.524+0000 7f02d482a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b185750 0x55978b187b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59188/0 (socket says 172.21.15.94:59188) 2024-02-17T12:00:08.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.524+0000 7f02d482a640 1 -- 172.21.15.94:0/2496897655 learned_addr learned my addr 172.21.15.94:0/2496897655 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.524+0000 7f02d582c640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02bc0a2380 0x7f02bc0a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.524+0000 7f02d502b640 1 --2- 172.21.15.94:0/2496897655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17cb70 0x55978b185210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.524+0000 7f02c67fc640 1 -- 172.21.15.94:0/1905456672 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f02cc0cda30 con 0x7f02d01002f0 2024-02-17T12:00:08.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.524+0000 7f02d502b640 1 -- 172.21.15.94:0/2496897655 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b185750 msgr2=0x55978b187b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.524+0000 7f02d502b640 1 --2- 172.21.15.94:0/2496897655 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b185750 0x55978b187b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.524+0000 7f02d502b640 1 -- 172.21.15.94:0/2496897655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 msgr2=0x55978b17c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.524+0000 7f02d502b640 1 --2- 172.21.15.94:0/2496897655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 0x55978b17c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.524+0000 7f02d502b640 1 -- 172.21.15.94:0/2496897655 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55978b0a6060 con 0x55978b17cb70 2024-02-17T12:00:08.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.525+0000 7f02d502b640 1 --2- 172.21.15.94:0/2496897655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17cb70 0x55978b185210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0cb170 tx=0x7f02cc08fae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d2ba6c444f72f10 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.525+0000 7f02bb7fe640 1 -- 172.21.15.94:0/2496897655 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc099d70 con 0x55978b17cb70 2024-02-17T12:00:08.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.525+0000 7f02bb7fe640 1 -- 172.21.15.94:0/2496897655 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02cc09ab10 con 0x55978b17cb70 2024-02-17T12:00:08.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.526+0000 7f02d9e5d740 1 -- 172.21.15.94:0/2496897655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17cb70 msgr2=0x55978b185210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.526+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/2496897655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17cb70 0x55978b185210 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0cb170 tx=0x7f02cc08fae0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.526+0000 7f02d582c640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02bc0a2380 0x7f02bc0a4840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f02c802c630 tx=0x7f02c8005f50 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.526+0000 7f02d9e5d740 1 -- 172.21.15.94:0/2496897655 shutdown_connections 2024-02-17T12:00:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.526+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/2496897655 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b185750 0x55978b187b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.526+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/2496897655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17cb70 0x55978b185210 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.526+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/2496897655 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 0x55978b17c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.526+0000 7f02d9e5d740 1 -- 172.21.15.94:0/2496897655 >> 172.21.15.94:0/2496897655 conn(0x55978b176980 msgr2=0x55978b176d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.526+0000 7f02d9e5d740 1 -- 172.21.15.94:0/2496897655 shutdown_connections 2024-02-17T12:00:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.526+0000 7f02d9e5d740 1 -- 172.21.15.94:0/2496897655 wait complete. 2024-02-17T12:00:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.527+0000 7f02d9e5d740 1 -- start start 2024-02-17T12:00:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.527+0000 7f02d9e5d740 1 Processor -- start 2024-02-17T12:00:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.527+0000 7f02d9e5d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 0x55978b182760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.527+0000 7f02d9e5d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b17cb70 0x55978b182ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d9e5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b185750 0x55978b1831e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d9e5d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978b0e4020 con 0x55978b17c250 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d9e5d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978adb7430 con 0x55978b17cb70 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d9e5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978ad73e00 con 0x55978b185750 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d9e5d740 10 client.0 _ll_register_callbacks cb 0x55978b08bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d482a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b17cb70 0x55978b182ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d482a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b17cb70 0x55978b182ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59204/0 (socket says 172.21.15.94:59204) 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d482a640 1 -- 172.21.15.94:0/2484578202 learned_addr learned my addr 172.21.15.94:0/2484578202 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d582c640 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b185750 0x55978b1831e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d502b640 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 0x55978b182760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d482a640 1 -- 172.21.15.94:0/2484578202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b185750 msgr2=0x55978b1831e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d482a640 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b185750 0x55978b1831e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d482a640 1 -- 172.21.15.94:0/2484578202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 msgr2=0x55978b182760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d482a640 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 0x55978b182760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d482a640 1 -- 172.21.15.94:0/2484578202 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d01a69e0 con 0x55978b17cb70 2024-02-17T12:00:08.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d502b640 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 0x55978b182760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.528+0000 7f02d482a640 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b17cb70 0x55978b182ca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02c000d9d0 tx=0x7f02c000de90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d66de76c3934d74e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02b8ff9640 1 -- 172.21.15.94:0/2484578202 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c001b440 con 0x55978b17cb70 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02b8ff9640 1 -- 172.21.15.94:0/2484578202 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f02c0011440 con 0x55978b17cb70 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02d9e5d740 1 -- 172.21.15.94:0/2484578202 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55978ade37e0 con 0x55978b17cb70 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02b8ff9640 1 -- 172.21.15.94:0/2484578202 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02c0011600 con 0x55978b17cb70 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02d9e5d740 10 client.25484 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02d9e5d740 1 -- 172.21.15.94:0/2484578202 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55978b0e26a0 con 0x55978b17cb70 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02b8ff9640 1 -- 172.21.15.94:0/2484578202 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02c0027080 con 0x55978b17cb70 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02b8ff9640 1 -- 172.21.15.94:0/2484578202 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02c0023a50 con 0x55978b17cb70 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02d9e5d740 10 client.25484 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02d9e5d740 10 client.25484 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02d9e5d740 20 client.25484 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:08.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55978b19e4e0 0x55978b1a0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02d9e5d740 4 client.25484 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:08.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.529+0000 7f02d9e5d740 1 -- 172.21.15.94:0/2484578202 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55978ade37e0 con 0x55978b19e4e0 2024-02-17T12:00:08.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.530+0000 7f02d502b640 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55978b19e4e0 0x55978b1a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.532+0000 7f02d502b640 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55978b19e4e0 0x55978b1a0960 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bdc9053eb9cd9113 server_cookie=2f3250cb63e43109 in_seq=0 out_seq=0 2024-02-17T12:00:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.532+0000 7f02b8ff9640 10 client.25484 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:00:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.532+0000 7f02b8ff9640 1 -- 172.21.15.94:0/2484578202 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55978ade37e0 con 0x55978b19e4e0 2024-02-17T12:00:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.532+0000 7f02b8ff9640 10 client.25484 handle_command_reply: tid=0 2024-02-17T12:00:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.533+0000 7f02d9e5d740 1 client.25484 shutdown 2024-02-17T12:00:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.533+0000 7f02d9e5d740 1 -- 172.21.15.94:0/2484578202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b17cb70 msgr2=0x55978b182ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.533+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b17cb70 0x55978b182ca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f02c000d9d0 tx=0x7f02c000de90 comp rx=0 tx=0).stop 2024-02-17T12:00:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.533+0000 7f02d9e5d740 1 -- 172.21.15.94:0/2484578202 shutdown_connections 2024-02-17T12:00:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.533+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b185750 0x55978b1831e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.533+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55978b19e4e0 0x55978b1a0960 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.533+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b17cb70 0x55978b182ca0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.533+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/2484578202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 0x55978b182760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.533+0000 7f02d9e5d740 1 -- 172.21.15.94:0/2484578202 >> 172.21.15.94:0/2484578202 conn(0x55978b176980 msgr2=0x55978b1786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.533+0000 7f02b8ff9640 0 client.25484 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:00:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.533+0000 7f02d9e5d740 1 -- 172.21.15.94:0/2484578202 shutdown_connections 2024-02-17T12:00:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.533+0000 7f02d9e5d740 1 -- 172.21.15.94:0/2484578202 wait complete. 2024-02-17T12:00:08.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.533+0000 7f02d9e5d740 20 client.25484 trim_cache size 0 max 16384 2024-02-17T12:00:08.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.555+0000 7f02d9e5d740 1 Processor -- start 2024-02-17T12:00:08.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.555+0000 7f02d9e5d740 1 -- start start 2024-02-17T12:00:08.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.556+0000 7f02d9e5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17c250 0x55978b184e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.556+0000 7f02d9e5d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b17cb70 0x55978b183000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.556+0000 7f02d502b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17c250 0x55978b184e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.556+0000 7f02d502b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17c250 0x55978b184e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:32770/0 (socket says 172.21.15.94:32770) 2024-02-17T12:00:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.556+0000 7f02d502b640 1 -- 172.21.15.94:0/1232978560 learned_addr learned my addr 172.21.15.94:0/1232978560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.556+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/1232978560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b185750 0x55978b17f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.556+0000 7f02d9e5d740 1 -- 172.21.15.94:0/1232978560 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978b069130 con 0x55978b185750 2024-02-17T12:00:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.556+0000 7f02d9e5d740 1 -- 172.21.15.94:0/1232978560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978b190c30 con 0x55978b17cb70 2024-02-17T12:00:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.556+0000 7f02d9e5d740 1 -- 172.21.15.94:0/1232978560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978b177980 con 0x55978b17c250 2024-02-17T12:00:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.556+0000 7f02d482a640 1 --2- 172.21.15.94:0/1232978560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b17cb70 0x55978b183000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.557+0000 7f02d582c640 1 --2- 172.21.15.94:0/1232978560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b185750 0x55978b17f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.557+0000 7f02d502b640 1 -- 172.21.15.94:0/1232978560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b17cb70 msgr2=0x55978b183000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.557+0000 7f02d502b640 1 --2- 172.21.15.94:0/1232978560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b17cb70 0x55978b183000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.557+0000 7f02d502b640 1 -- 172.21.15.94:0/1232978560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b185750 msgr2=0x55978b17f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.557+0000 7f02d502b640 1 --2- 172.21.15.94:0/1232978560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b185750 0x55978b17f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.557+0000 7f02d502b640 1 -- 172.21.15.94:0/1232978560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55978b07ec20 con 0x55978b17c250 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.557+0000 7f02d482a640 1 --2- 172.21.15.94:0/1232978560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b17cb70 0x55978b183000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.557+0000 7f02d502b640 1 --2- 172.21.15.94:0/1232978560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17c250 0x55978b184e90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0cb220 tx=0x7f02cc0ba040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10a74d4d6ceb53a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.557+0000 7f02baffd640 1 -- 172.21.15.94:0/1232978560 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc0a2bb0 con 0x55978b17c250 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.557+0000 7f02baffd640 1 -- 172.21.15.94:0/1232978560 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02cc005680 con 0x55978b17c250 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.557+0000 7f02baffd640 1 -- 172.21.15.94:0/1232978560 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc09e170 con 0x55978b17c250 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.558+0000 7f02d9e5d740 1 -- 172.21.15.94:0/1232978560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17c250 msgr2=0x55978b184e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.558+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/1232978560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17c250 0x55978b184e90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f02cc0cb220 tx=0x7f02cc0ba040 comp rx=0 tx=0).stop 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.558+0000 7f02d9e5d740 1 -- 172.21.15.94:0/1232978560 shutdown_connections 2024-02-17T12:00:08.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.558+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/1232978560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b185750 0x55978b17f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.558+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/1232978560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b17cb70 0x55978b183000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.558+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/1232978560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17c250 0x55978b184e90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.558+0000 7f02d9e5d740 1 -- 172.21.15.94:0/1232978560 >> 172.21.15.94:0/1232978560 conn(0x55978b176980 msgr2=0x55978b17da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.559+0000 7f02d9e5d740 1 -- 172.21.15.94:0/1232978560 shutdown_connections 2024-02-17T12:00:08.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.559+0000 7f02d9e5d740 1 -- 172.21.15.94:0/1232978560 wait complete. 2024-02-17T12:00:08.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.560+0000 7f02d9e5d740 1 -- start start 2024-02-17T12:00:08.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.560+0000 7f02d9e5d740 1 Processor -- start 2024-02-17T12:00:08.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.560+0000 7f02d9e5d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 0x55978b1953e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d9e5d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17cb70 0x55978b193780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d9e5d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b185750 0x55978b193e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d9e5d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978b194570 con 0x55978b17c250 2024-02-17T12:00:08.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d9e5d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978b1f4a90 con 0x55978b185750 2024-02-17T12:00:08.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d9e5d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55978b1f4bd0 con 0x55978b17cb70 2024-02-17T12:00:08.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d482a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17cb70 0x55978b193780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d582c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b185750 0x55978b193e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d482a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17cb70 0x55978b193780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:32780/0 (socket says 172.21.15.94:32780) 2024-02-17T12:00:08.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d482a640 1 -- 172.21.15.94:0/3911753438 learned_addr learned my addr 172.21.15.94:0/3911753438 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d502b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 0x55978b1953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d9e5d740 10 client.0 _ll_register_callbacks cb 0x7f02cc096040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:08.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d482a640 1 -- 172.21.15.94:0/3911753438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b185750 msgr2=0x55978b193e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d482a640 1 --2- 172.21.15.94:0/3911753438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b185750 0x55978b193e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d482a640 1 -- 172.21.15.94:0/3911753438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 msgr2=0x55978b1953e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d482a640 1 --2- 172.21.15.94:0/3911753438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 0x55978b1953e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.561+0000 7f02d482a640 1 -- 172.21.15.94:0/3911753438 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55978b0a6060 con 0x55978b17cb70 2024-02-17T12:00:08.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.562+0000 7f02d482a640 1 --2- 172.21.15.94:0/3911753438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17cb70 0x55978b193780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55978b181fc0 tx=0x7f02c0003af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=628ffdf32c946e96 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.562+0000 7f02b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:00:08.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.562+0000 7f02b8ff9640 1 -- 172.21.15.94:0/3911753438 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c001a5e0 con 0x55978b17cb70 2024-02-17T12:00:08.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.562+0000 7f02b8ff9640 1 -- 172.21.15.94:0/3911753438 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f02c0000e40 con 0x55978b17cb70 2024-02-17T12:00:08.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.562+0000 7f02d9e5d740 1 -- 172.21.15.94:0/3911753438 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02cc09ab10 con 0x55978b17cb70 2024-02-17T12:00:08.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.562+0000 7f02b8ff9640 1 -- 172.21.15.94:0/3911753438 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02c001a760 con 0x55978b17cb70 2024-02-17T12:00:08.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.562+0000 7f02d9e5d740 10 client.25546 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:08.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.562+0000 7f02d9e5d740 1 -- 172.21.15.94:0/3911753438 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55978b0a48c0 con 0x55978b17cb70 2024-02-17T12:00:08.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.563+0000 7f02b8ff9640 1 -- 172.21.15.94:0/3911753438 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02c0038080 con 0x55978b17cb70 2024-02-17T12:00:08.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.563+0000 7f02b8ff9640 1 -- 172.21.15.94:0/3911753438 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02c00239b0 con 0x55978b17cb70 2024-02-17T12:00:08.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.563+0000 7f02d9e5d740 10 client.25546 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:08.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.563+0000 7f02d9e5d740 10 client.25546 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:00:08.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.563+0000 7f02d9e5d740 20 client.25546 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:08.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.563+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/3911753438 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55978b1f6330 0x55978b1f8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.563+0000 7f02d9e5d740 4 client.25546 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:00:08.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.563+0000 7f02d9e5d740 1 -- 172.21.15.94:0/3911753438 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f02cc09ab10 con 0x55978b1f6330 2024-02-17T12:00:08.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.563+0000 7f02d502b640 1 --2- 172.21.15.94:0/3911753438 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55978b1f6330 0x55978b1f8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.565+0000 7f02b8ff9640 10 client.25546 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:00:08.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.565+0000 7f02d502b640 1 --2- 172.21.15.94:0/3911753438 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55978b1f6330 0x55978b1f8750 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=55184ec389562ce5 server_cookie=e5c8d996336a6532 in_seq=0 out_seq=0 2024-02-17T12:00:08.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.565+0000 7f02b8ff9640 1 -- 172.21.15.94:0/3911753438 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f02cc09ab10 con 0x55978b1f6330 2024-02-17T12:00:08.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.565+0000 7f02b8ff9640 10 client.25546 handle_command_reply: tid=0 2024-02-17T12:00:08.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.565+0000 7f02d9e5d740 1 client.25546 shutdown 2024-02-17T12:00:08.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.565+0000 7f02d9e5d740 1 -- 172.21.15.94:0/3911753438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17cb70 msgr2=0x55978b193780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.565+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/3911753438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17cb70 0x55978b193780 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55978b181fc0 tx=0x7f02c0003af0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.566+0000 7f02d9e5d740 1 -- 172.21.15.94:0/3911753438 shutdown_connections 2024-02-17T12:00:08.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.566+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/3911753438 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55978b1f6330 0x55978b1f8750 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.566+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/3911753438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55978b185750 0x55978b193e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.566+0000 7f02b8ff9640 0 client.25546 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:00:08.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.566+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/3911753438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55978b17cb70 0x55978b193780 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.566+0000 7f02d9e5d740 1 --2- 172.21.15.94:0/3911753438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55978b17c250 0x55978b1953e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.566+0000 7f02d9e5d740 1 -- 172.21.15.94:0/3911753438 >> 172.21.15.94:0/3911753438 conn(0x55978b176980 msgr2=0x55978b1752f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.566+0000 7f02d9e5d740 1 -- 172.21.15.94:0/3911753438 shutdown_connections 2024-02-17T12:00:08.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.566+0000 7f02d9e5d740 1 -- 172.21.15.94:0/3911753438 wait complete. 2024-02-17T12:00:08.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.566+0000 7f02d9e5d740 20 client.25546 trim_cache size 0 max 16384 2024-02-17T12:00:08.569 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:00:08.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.567+0000 7f02d728f640 1 -- 172.21.15.94:0/1905456672 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02bc0a2380 msgr2=0x7f02bc0a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.567+0000 7f02d728f640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02bc0a2380 0x7f02bc0a4840 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f02c802c630 tx=0x7f02c8005f50 comp rx=0 tx=0).stop 2024-02-17T12:00:08.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.567+0000 7f02d728f640 1 -- 172.21.15.94:0/1905456672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02d01002f0 msgr2=0x7f02d019f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.567+0000 7f02d728f640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02d01002f0 0x7f02d019f650 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00d950 tx=0x7f02cc00de10 comp rx=0 tx=0).stop 2024-02-17T12:00:08.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.567+0000 7f02d728f640 1 -- 172.21.15.94:0/1905456672 shutdown_connections 2024-02-17T12:00:08.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.567+0000 7f02d728f640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02bc0a2380 0x7f02bc0a4840 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.567+0000 7f02d728f640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02d0199810 0x7f02d0199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.567+0000 7f02d728f640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02d0105370 0x7f02d019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.567+0000 7f02d728f640 1 --2- 172.21.15.94:0/1905456672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02d01002f0 0x7f02d019f650 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.567+0000 7f02d728f640 1 -- 172.21.15.94:0/1905456672 >> 172.21.15.94:0/1905456672 conn(0x7f02d00771e0 msgr2=0x7f02d00fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.567+0000 7f02d728f640 1 -- 172.21.15.94:0/1905456672 shutdown_connections 2024-02-17T12:00:08.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.567+0000 7f02d728f640 1 -- 172.21.15.94:0/1905456672 wait complete. 2024-02-17T12:00:08.581 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:00:08.581 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:00:08.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.743+0000 7f4e08639640 1 -- 172.21.15.94:0/2984782879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e00103f40 msgr2=0x7f4e00107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.743+0000 7f4e08639640 1 --2- 172.21.15.94:0/2984782879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e00103f40 0x7f4e00107f80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009930 tx=0x7f4dfc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:00:08.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.743+0000 7f4e08639640 1 -- 172.21.15.94:0/2984782879 shutdown_connections 2024-02-17T12:00:08.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.743+0000 7f4e08639640 1 --2- 172.21.15.94:0/2984782879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00108660 0x7f4e00110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.743+0000 7f4e08639640 1 --2- 172.21.15.94:0/2984782879 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e00103f40 0x7f4e00107f80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.743+0000 7f4e08639640 1 --2- 172.21.15.94:0/2984782879 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e00103590 0x7f4e00103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.743+0000 7f4e08639640 1 -- 172.21.15.94:0/2984782879 >> 172.21.15.94:0/2984782879 conn(0x7f4e000fedf0 msgr2=0x7f4e00101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.743+0000 7f4e08639640 1 -- 172.21.15.94:0/2984782879 shutdown_connections 2024-02-17T12:00:08.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.743+0000 7f4e08639640 1 -- 172.21.15.94:0/2984782879 wait complete. 2024-02-17T12:00:08.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.744+0000 7f4e08639640 1 Processor -- start 2024-02-17T12:00:08.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.744+0000 7f4e08639640 1 -- start start 2024-02-17T12:00:08.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.744+0000 7f4e08639640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e00103590 0x7f4e00110020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.744+0000 7f4e08639640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00108660 0x7f4e0010d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.744+0000 7f4e08639640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e0010d650 0x7f4e0010db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.744+0000 7f4e08639640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e001aa3a0 con 0x7f4e00103590 2024-02-17T12:00:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.744+0000 7f4e08639640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e001aa510 con 0x7f4e0010d650 2024-02-17T12:00:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.744+0000 7f4e08639640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e001aa6d0 con 0x7f4e00108660 2024-02-17T12:00:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e063d5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e00103590 0x7f4e00110020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e05bd4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00108660 0x7f4e0010d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e063d5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e00103590 0x7f4e00110020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33212/0 (socket says 172.21.15.94:33212) 2024-02-17T12:00:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e063d5640 1 -- 172.21.15.94:0/3655442185 learned_addr learned my addr 172.21.15.94:0/3655442185 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e05bd4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00108660 0x7f4e0010d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:32802/0 (socket says 172.21.15.94:32802) 2024-02-17T12:00:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e06bd6640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e0010d650 0x7f4e0010db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e06bd6640 1 -- 172.21.15.94:0/3655442185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00108660 msgr2=0x7f4e0010d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e06bd6640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00108660 0x7f4e0010d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e06bd6640 1 -- 172.21.15.94:0/3655442185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e00103590 msgr2=0x7f4e00110020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e06bd6640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e00103590 0x7f4e00110020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e06bd6640 1 -- 172.21.15.94:0/3655442185 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc009560 con 0x7f4e0010d650 2024-02-17T12:00:08.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e05bd4640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00108660 0x7f4e0010d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:08.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e063d5640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e00103590 0x7f4e00110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:08.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.745+0000 7f4e06bd6640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e0010d650 0x7f4e0010db00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009fd0 tx=0x7f4df800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.746+0000 7f4df77fe640 1 -- 172.21.15.94:0/3655442185 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df800ce20 con 0x7f4e0010d650 2024-02-17T12:00:08.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.746+0000 7f4e08639640 1 -- 172.21.15.94:0/3655442185 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e00198b60 con 0x7f4e0010d650 2024-02-17T12:00:08.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.746+0000 7f4df77fe640 1 -- 172.21.15.94:0/3655442185 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4df8005020 con 0x7f4e0010d650 2024-02-17T12:00:08.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.746+0000 7f4e08639640 1 -- 172.21.15.94:0/3655442185 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e00199040 con 0x7f4e0010d650 2024-02-17T12:00:08.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.746+0000 7f4df77fe640 1 -- 172.21.15.94:0/3655442185 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df8010690 con 0x7f4e0010d650 2024-02-17T12:00:08.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.749+0000 7f4e0b209740 1 Processor -- start 2024-02-17T12:00:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.750+0000 7f4df77fe640 1 -- 172.21.15.94:0/3655442185 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4df8005500 con 0x7f4e0010d650 2024-02-17T12:00:08.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e0b209740 1 -- start start 2024-02-17T12:00:08.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e0b209740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c0250 0x55fd077c0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e0b209740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0b70 0x55fd077c9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e0b209740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c9750 0x55fd077cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e0b209740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0777b960 con 0x55fd077c9750 2024-02-17T12:00:08.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e0b209740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0772f310 con 0x55fd077c0b70 2024-02-17T12:00:08.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e0b209740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0772c840 con 0x55fd077c0250 2024-02-17T12:00:08.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e05bd4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c0250 0x55fd077c0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e05bd4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c0250 0x55fd077c0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:32808/0 (socket says 172.21.15.94:32808) 2024-02-17T12:00:08.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e05bd4640 1 -- 172.21.15.94:0/151941947 learned_addr learned my addr 172.21.15.94:0/151941947 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e063d5640 1 --2- 172.21.15.94:0/151941947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0b70 0x55fd077c9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e05bd4640 1 --2- 172.21.15.94:0/151941947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c9750 0x55fd077cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4df77fe640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4dec0a2330 0x7f4dec0a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e063d5640 1 -- 172.21.15.94:0/151941947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c0250 msgr2=0x55fd077c0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e063d5640 1 --2- 172.21.15.94:0/151941947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c0250 0x55fd077c0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e063d5640 1 -- 172.21.15.94:0/151941947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c9750 msgr2=0x55fd077cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e063d5640 1 --2- 172.21.15.94:0/151941947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c9750 0x55fd077cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.751+0000 7f4e063d5640 1 -- 172.21.15.94:0/151941947 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd076ea060 con 0x55fd077c0b70 2024-02-17T12:00:08.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.752+0000 7f4e06bd6640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4dec0a2330 0x7f4dec0a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.752+0000 7f4e063d5640 1 --2- 172.21.15.94:0/151941947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0b70 0x55fd077c9210 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4df00114f0 tx=0x7f4df00119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7108b7de22c9564 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.752+0000 7f4df77fe640 1 -- 172.21.15.94:0/3655442185 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4df8014070 con 0x7f4e0010d650 2024-02-17T12:00:08.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.752+0000 7f4df4bfe640 1 -- 172.21.15.94:0/151941947 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0004030 con 0x55fd077c0b70 2024-02-17T12:00:08.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.752+0000 7f4df4bfe640 1 -- 172.21.15.94:0/151941947 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4df00035f0 con 0x55fd077c0b70 2024-02-17T12:00:08.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.752+0000 7f4df4bfe640 1 -- 172.21.15.94:0/151941947 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df000f8c0 con 0x55fd077c0b70 2024-02-17T12:00:08.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.753+0000 7f4e0b209740 1 -- 172.21.15.94:0/151941947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0b70 msgr2=0x55fd077c9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.753+0000 7f4e0b209740 1 --2- 172.21.15.94:0/151941947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0b70 0x55fd077c9210 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4df00114f0 tx=0x7f4df00119b0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.753+0000 7f4e06bd6640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4dec0a2330 0x7f4dec0a47f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4df809ae20 tx=0x7f4df8097460 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.753+0000 7f4e0b209740 1 -- 172.21.15.94:0/151941947 shutdown_connections 2024-02-17T12:00:08.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.753+0000 7f4e0b209740 1 --2- 172.21.15.94:0/151941947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c9750 0x55fd077cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.753+0000 7f4e0b209740 1 --2- 172.21.15.94:0/151941947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0b70 0x55fd077c9210 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.753+0000 7f4e0b209740 1 --2- 172.21.15.94:0/151941947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c0250 0x55fd077c0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.753+0000 7f4e0b209740 1 -- 172.21.15.94:0/151941947 >> 172.21.15.94:0/151941947 conn(0x55fd077ba980 msgr2=0x55fd077bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.754+0000 7f4e0b209740 1 -- 172.21.15.94:0/151941947 shutdown_connections 2024-02-17T12:00:08.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.755+0000 7f4e0b209740 1 -- 172.21.15.94:0/151941947 wait complete. 2024-02-17T12:00:08.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.755+0000 7f4e0b209740 1 -- start start 2024-02-17T12:00:08.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.755+0000 7f4e0b209740 1 Processor -- start 2024-02-17T12:00:08.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e0b209740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c0250 0x55fd077c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e0b209740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0b70 0x55fd077c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e0b209740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c9750 0x55fd077c7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e0b209740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd07728020 con 0x55fd077c0b70 2024-02-17T12:00:08.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e0b209740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd073fb430 con 0x55fd077c9750 2024-02-17T12:00:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e0b209740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd073b7e00 con 0x55fd077c0250 2024-02-17T12:00:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e0b209740 10 client.0 _ll_register_callbacks cb 0x55fd076cfcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e063d5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0b70 0x55fd077c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e063d5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0b70 0x55fd077c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33224/0 (socket says 172.21.15.94:33224) 2024-02-17T12:00:08.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e063d5640 1 -- 172.21.15.94:0/3340005147 learned_addr learned my addr 172.21.15.94:0/3340005147 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e05bd4640 1 --2- 172.21.15.94:0/3340005147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c9750 0x55fd077c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e05bd4640 1 --2- 172.21.15.94:0/3340005147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c0250 0x55fd077c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e063d5640 1 -- 172.21.15.94:0/3340005147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c0250 msgr2=0x55fd077c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e063d5640 1 --2- 172.21.15.94:0/3340005147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c0250 0x55fd077c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e063d5640 1 -- 172.21.15.94:0/3340005147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c9750 msgr2=0x55fd077c7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e063d5640 1 --2- 172.21.15.94:0/3340005147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c9750 0x55fd077c7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e063d5640 1 -- 172.21.15.94:0/3340005147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df0009cd0 con 0x55fd077c0b70 2024-02-17T12:00:08.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4e063d5640 1 --2- 172.21.15.94:0/3340005147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0b70 0x55fd077c6c00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4df000f5a0 tx=0x7f4df00041f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0db1dbb5f909f5d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.756+0000 7f4de9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:00:08.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4de9ffb640 1 -- 172.21.15.94:0/3340005147 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0003d80 con 0x55fd077c0b70 2024-02-17T12:00:08.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4de9ffb640 1 -- 172.21.15.94:0/3340005147 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4df0005030 con 0x55fd077c0b70 2024-02-17T12:00:08.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4e0b209740 1 -- 172.21.15.94:0/3340005147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd074277e0 con 0x55fd077c0b70 2024-02-17T12:00:08.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4de9ffb640 1 -- 172.21.15.94:0/3340005147 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4df00051f0 con 0x55fd077c0b70 2024-02-17T12:00:08.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4e0b209740 10 client.16047 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:08.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4e0b209740 1 -- 172.21.15.94:0/3340005147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55fd077266a0 con 0x55fd077c0b70 2024-02-17T12:00:08.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4de9ffb640 1 -- 172.21.15.94:0/3340005147 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fd074277e0 con 0x55fd077c0b70 2024-02-17T12:00:08.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4de9ffb640 1 -- 172.21.15.94:0/3340005147 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4df000f640 con 0x55fd077c0b70 2024-02-17T12:00:08.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4e0b209740 10 client.16047 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:08.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4e0b209740 10 client.16047 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:00:08.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4e0b209740 20 client.16047 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:08.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4e0b209740 1 --2- 172.21.15.94:0/3340005147 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55fd077d9b30 0x55fd077dbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4e0b209740 4 client.16047 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:08.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.757+0000 7f4e0b209740 1 -- 172.21.15.94:0/3340005147 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd07464720 con 0x55fd077d9b30 2024-02-17T12:00:08.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.758+0000 7f4e05bd4640 1 --2- 172.21.15.94:0/3340005147 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55fd077d9b30 0x55fd077dbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.758+0000 7f4de9ffb640 10 client.16047 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:00:08.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.758+0000 7f4e05bd4640 1 --2- 172.21.15.94:0/3340005147 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55fd077d9b30 0x55fd077dbf50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1ee51cbc2c1bc5 server_cookie=a03b6be3b9654bcf in_seq=0 out_seq=0 2024-02-17T12:00:08.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.759+0000 7f4de9ffb640 1 -- 172.21.15.94:0/3340005147 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f4dfc05ea90 con 0x55fd077d9b30 2024-02-17T12:00:08.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.759+0000 7f4de9ffb640 10 client.16047 handle_command_reply: tid=0 2024-02-17T12:00:08.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.759+0000 7f4e0b209740 1 client.16047 shutdown 2024-02-17T12:00:08.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.760+0000 7f4e0b209740 1 -- 172.21.15.94:0/3340005147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0b70 msgr2=0x55fd077c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.760+0000 7f4e0b209740 1 --2- 172.21.15.94:0/3340005147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0b70 0x55fd077c6c00 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4df000f5a0 tx=0x7f4df00041f0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.760+0000 7f4e0b209740 1 -- 172.21.15.94:0/3340005147 shutdown_connections 2024-02-17T12:00:08.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.760+0000 7f4e0b209740 1 --2- 172.21.15.94:0/3340005147 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55fd077d9b30 0x55fd077dbf50 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.760+0000 7f4e0b209740 1 --2- 172.21.15.94:0/3340005147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c9750 0x55fd077c7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.760+0000 7f4e0b209740 1 --2- 172.21.15.94:0/3340005147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0b70 0x55fd077c6c00 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.760+0000 7f4e0b209740 1 --2- 172.21.15.94:0/3340005147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c0250 0x55fd077c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.760+0000 7f4e0b209740 1 -- 172.21.15.94:0/3340005147 >> 172.21.15.94:0/3340005147 conn(0x55fd077ba980 msgr2=0x55fd077bc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.761+0000 7f4de9ffb640 0 client.16047 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:00:08.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.761+0000 7f4e0b209740 1 -- 172.21.15.94:0/3340005147 shutdown_connections 2024-02-17T12:00:08.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.761+0000 7f4e0b209740 1 -- 172.21.15.94:0/3340005147 wait complete. 2024-02-17T12:00:08.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.761+0000 7f4e0b209740 20 client.16047 trim_cache size 0 max 16384 2024-02-17T12:00:08.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.778+0000 7f4e0b209740 1 Processor -- start 2024-02-17T12:00:08.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.778+0000 7f4e0b209740 1 -- start start 2024-02-17T12:00:08.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.778+0000 7f4e0b209740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0250 0x55fd077c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e0b209740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0b70 0x55fd077c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e05bd4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0250 0x55fd077c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e0b209740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c9750 0x55fd077c3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e0b209740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd076ad130 con 0x55fd077c0250 2024-02-17T12:00:08.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e05bd4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0250 0x55fd077c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33228/0 (socket says 172.21.15.94:33228) 2024-02-17T12:00:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e0b209740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd077b4f90 con 0x55fd077c0b70 2024-02-17T12:00:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e0b209740 1 -- 172.21.15.94:0/2578986314 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd077b62b0 con 0x55fd077c9750 2024-02-17T12:00:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e05bd4640 1 -- 172.21.15.94:0/2578986314 learned_addr learned my addr 172.21.15.94:0/2578986314 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e063d5640 1 --2- 172.21.15.94:0/2578986314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0b70 0x55fd077c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e05bd4640 1 -- 172.21.15.94:0/2578986314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c9750 msgr2=0x55fd077c3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e05bd4640 1 --2- 172.21.15.94:0/2578986314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c9750 0x55fd077c3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e05bd4640 1 -- 172.21.15.94:0/2578986314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0b70 msgr2=0x55fd077c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e05bd4640 1 --2- 172.21.15.94:0/2578986314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0b70 0x55fd077c6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e05bd4640 1 -- 172.21.15.94:0/2578986314 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd076c2c20 con 0x55fd077c0250 2024-02-17T12:00:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4e05bd4640 1 --2- 172.21.15.94:0/2578986314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0250 0x55fd077c8df0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009230 tx=0x7f4dfc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4422fd8319d89d5b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4debfff640 1 -- 172.21.15.94:0/2578986314 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc004100 con 0x55fd077c0250 2024-02-17T12:00:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4debfff640 1 -- 172.21.15.94:0/2578986314 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dfc05a040 con 0x55fd077c0250 2024-02-17T12:00:08.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.779+0000 7f4debfff640 1 -- 172.21.15.94:0/2578986314 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc003850 con 0x55fd077c0250 2024-02-17T12:00:08.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.780+0000 7f4e0b209740 1 -- 172.21.15.94:0/2578986314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0250 msgr2=0x55fd077c8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.780+0000 7f4e0b209740 1 --2- 172.21.15.94:0/2578986314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0250 0x55fd077c8df0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009230 tx=0x7f4dfc009260 comp rx=0 tx=0).stop 2024-02-17T12:00:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.781+0000 7f4e0b209740 1 -- 172.21.15.94:0/2578986314 shutdown_connections 2024-02-17T12:00:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.781+0000 7f4e0b209740 1 --2- 172.21.15.94:0/2578986314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c9750 0x55fd077c3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.781+0000 7f4e0b209740 1 --2- 172.21.15.94:0/2578986314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0b70 0x55fd077c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.781+0000 7f4e0b209740 1 --2- 172.21.15.94:0/2578986314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0250 0x55fd077c8df0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.781+0000 7f4e0b209740 1 -- 172.21.15.94:0/2578986314 >> 172.21.15.94:0/2578986314 conn(0x55fd077ba980 msgr2=0x55fd077e08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.781+0000 7f4e0b209740 1 -- 172.21.15.94:0/2578986314 shutdown_connections 2024-02-17T12:00:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.781+0000 7f4e0b209740 1 -- 172.21.15.94:0/2578986314 wait complete. 2024-02-17T12:00:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.781+0000 7f4e0b209740 1 -- start start 2024-02-17T12:00:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.781+0000 7f4e0b209740 1 Processor -- start 2024-02-17T12:00:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.782+0000 7f4e0b209740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0250 0x55fd077cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.782+0000 7f4e0b209740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0b70 0x55fd077cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.782+0000 7f4e05bd4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0250 0x55fd077cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.782+0000 7f4e05bd4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0250 0x55fd077cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59280/0 (socket says 172.21.15.94:59280) 2024-02-17T12:00:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.782+0000 7f4e0b209740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c9750 0x55fd077cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.782+0000 7f4e063d5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0b70 0x55fd077cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.782+0000 7f4e05bd4640 1 -- 172.21.15.94:0/3086608048 learned_addr learned my addr 172.21.15.94:0/3086608048 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.782+0000 7f4e0b209740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd078301b0 con 0x55fd077c0b70 2024-02-17T12:00:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.782+0000 7f4e0b209740 1 -- 172.21.15.94:0/3086608048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd07830320 con 0x55fd077c0250 2024-02-17T12:00:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.782+0000 7f4e0b209740 1 -- 172.21.15.94:0/3086608048 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd078304e0 con 0x55fd077c9750 2024-02-17T12:00:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.782+0000 7f4e0b209740 10 client.0 _ll_register_callbacks cb 0x7f4df0009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.782+0000 7f4e05bd4640 1 --2- 172.21.15.94:0/3086608048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c9750 0x55fd077cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4e05bd4640 1 -- 172.21.15.94:0/3086608048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c9750 msgr2=0x55fd077cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4e05bd4640 1 --2- 172.21.15.94:0/3086608048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c9750 0x55fd077cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4e05bd4640 1 -- 172.21.15.94:0/3086608048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0b70 msgr2=0x55fd077cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4e05bd4640 1 --2- 172.21.15.94:0/3086608048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0b70 0x55fd077cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4e05bd4640 1 -- 172.21.15.94:0/3086608048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e00110a40 con 0x55fd077c0250 2024-02-17T12:00:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4e063d5640 1 --2- 172.21.15.94:0/3086608048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0b70 0x55fd077cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4e05bd4640 1 --2- 172.21.15.94:0/3086608048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0250 0x55fd077cf2c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc036040 tx=0x7f4dfc005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93429f72a8dc1131 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4de9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:00:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4de9ffb640 1 -- 172.21.15.94:0/3086608048 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc004eb0 con 0x55fd077c0250 2024-02-17T12:00:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4de9ffb640 1 -- 172.21.15.94:0/3086608048 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4dfc005010 con 0x55fd077c0250 2024-02-17T12:00:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4e0b209740 1 -- 172.21.15.94:0/3086608048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4df00035f0 con 0x55fd077c0250 2024-02-17T12:00:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4de9ffb640 1 -- 172.21.15.94:0/3086608048 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dfc0053e0 con 0x55fd077c0250 2024-02-17T12:00:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4e0b209740 10 client.25508 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4e0b209740 1 -- 172.21.15.94:0/3086608048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55fd076e88c0 con 0x55fd077c0250 2024-02-17T12:00:08.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.783+0000 7f4de9ffb640 1 -- 172.21.15.94:0/3086608048 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4dfc07a080 con 0x55fd077c0250 2024-02-17T12:00:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.784+0000 7f4de9ffb640 1 -- 172.21.15.94:0/3086608048 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4dfc06a040 con 0x55fd077c0250 2024-02-17T12:00:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.784+0000 7f4e0b209740 10 client.25508 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.784+0000 7f4e0b209740 10 client.25508 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:00:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.784+0000 7f4e0b209740 20 client.25508 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.784+0000 7f4e0b209740 1 --2- 172.21.15.94:0/3086608048 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55fd078319c0 0x55fd07833db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.784+0000 7f4e0b209740 4 client.25508 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:00:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.784+0000 7f4e0b209740 1 -- 172.21.15.94:0/3086608048 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fd074b3890 con 0x55fd078319c0 2024-02-17T12:00:08.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.784+0000 7f4e063d5640 1 --2- 172.21.15.94:0/3086608048 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55fd078319c0 0x55fd07833db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4e063d5640 1 --2- 172.21.15.94:0/3086608048 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55fd078319c0 0x55fd07833db0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f410a8669300ed9d server_cookie=6a6cff7a8a3f3e52 in_seq=0 out_seq=0 2024-02-17T12:00:08.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4de9ffb640 10 client.25508 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:00:08.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4de9ffb640 1 -- 172.21.15.94:0/3086608048 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fd074b3890 con 0x55fd078319c0 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4de9ffb640 10 client.25508 handle_command_reply: tid=0 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4e0b209740 1 client.25508 shutdown 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4e0b209740 1 -- 172.21.15.94:0/3086608048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0250 msgr2=0x55fd077cf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4e0b209740 1 --2- 172.21.15.94:0/3086608048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0250 0x55fd077cf2c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc036040 tx=0x7f4dfc005850 comp rx=0 tx=0).stop 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4e0b209740 1 -- 172.21.15.94:0/3086608048 shutdown_connections 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4e0b209740 1 --2- 172.21.15.94:0/3086608048 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55fd078319c0 0x55fd07833db0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4e0b209740 1 --2- 172.21.15.94:0/3086608048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd077c9750 0x55fd077cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4e0b209740 1 --2- 172.21.15.94:0/3086608048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd077c0b70 0x55fd077cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4e0b209740 1 --2- 172.21.15.94:0/3086608048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd077c0250 0x55fd077cf2c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4e0b209740 1 -- 172.21.15.94:0/3086608048 >> 172.21.15.94:0/3086608048 conn(0x55fd077ba980 msgr2=0x55fd077b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4de9ffb640 0 client.25508 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4e0b209740 1 -- 172.21.15.94:0/3086608048 shutdown_connections 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.785+0000 7f4e0b209740 1 -- 172.21.15.94:0/3086608048 wait complete. 2024-02-17T12:00:08.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.786+0000 7f4e0b209740 20 client.25508 trim_cache size 0 max 16384 2024-02-17T12:00:08.789 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:00:08.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.786+0000 7f4e08639640 1 -- 172.21.15.94:0/3655442185 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4dec0a2330 msgr2=0x7f4dec0a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.786+0000 7f4e08639640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4dec0a2330 0x7f4dec0a47f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4df809ae20 tx=0x7f4df8097460 comp rx=0 tx=0).stop 2024-02-17T12:00:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.786+0000 7f4e08639640 1 -- 172.21.15.94:0/3655442185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e0010d650 msgr2=0x7f4e0010db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.786+0000 7f4e08639640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e0010d650 0x7f4e0010db00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4df8009fd0 tx=0x7f4df800ef10 comp rx=0 tx=0).stop 2024-02-17T12:00:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.786+0000 7f4e08639640 1 -- 172.21.15.94:0/3655442185 shutdown_connections 2024-02-17T12:00:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.786+0000 7f4e08639640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4dec0a2330 0x7f4dec0a47f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.786+0000 7f4e08639640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4e0010d650 0x7f4e0010db00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.786+0000 7f4e08639640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e00108660 0x7f4e0010d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.786+0000 7f4e08639640 1 --2- 172.21.15.94:0/3655442185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4e00103590 0x7f4e00110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.786+0000 7f4e08639640 1 -- 172.21.15.94:0/3655442185 >> 172.21.15.94:0/3655442185 conn(0x7f4e000fedf0 msgr2=0x7f4e001001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.788+0000 7f4e08639640 1 -- 172.21.15.94:0/3655442185 shutdown_connections 2024-02-17T12:00:08.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.788+0000 7f4e08639640 1 -- 172.21.15.94:0/3655442185 wait complete. 2024-02-17T12:00:08.804 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:00:08.804 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:00:08.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.962+0000 7f33dc28d640 1 -- 172.21.15.94:0/4246532467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d4101d40 msgr2=0x7f33d4105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.962+0000 7f33dc28d640 1 --2- 172.21.15.94:0/4246532467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d4101d40 0x7f33d4105d80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009930 tx=0x7f33c402bc20 comp rx=0 tx=0).stop 2024-02-17T12:00:08.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.962+0000 7f33dc28d640 1 -- 172.21.15.94:0/4246532467 shutdown_connections 2024-02-17T12:00:08.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.962+0000 7f33dc28d640 1 --2- 172.21.15.94:0/4246532467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33d4106460 0x7f33d410e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.962+0000 7f33dc28d640 1 --2- 172.21.15.94:0/4246532467 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d4101d40 0x7f33d4105d80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.962+0000 7f33dc28d640 1 --2- 172.21.15.94:0/4246532467 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d4101390 0x7f33d4101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.962+0000 7f33dc28d640 1 -- 172.21.15.94:0/4246532467 >> 172.21.15.94:0/4246532467 conn(0x7f33d40fcbb0 msgr2=0x7f33d40fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.962+0000 7f33dc28d640 1 -- 172.21.15.94:0/4246532467 shutdown_connections 2024-02-17T12:00:08.962 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.962+0000 7f33dc28d640 1 -- 172.21.15.94:0/4246532467 wait complete. 2024-02-17T12:00:08.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.963+0000 7f33dc28d640 1 Processor -- start 2024-02-17T12:00:08.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.963+0000 7f33dc28d640 1 -- start start 2024-02-17T12:00:08.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33dc28d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33d4101390 0x7f33d4075080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33dc28d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d4106460 0x7f33d4073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33dc28d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d4073a10 0x7f33d4073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33dc28d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d41aa410 con 0x7f33d4073a10 2024-02-17T12:00:08.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33dc28d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d41aa580 con 0x7f33d4106460 2024-02-17T12:00:08.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33dc28d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d41aa740 con 0x7f33d4101390 2024-02-17T12:00:08.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33da82a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d4073a10 0x7f33d4073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33d9828640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d4106460 0x7f33d4073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33da82a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d4073a10 0x7f33d4073ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33258/0 (socket says 172.21.15.94:33258) 2024-02-17T12:00:08.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33da82a640 1 -- 172.21.15.94:0/1299494915 learned_addr learned my addr 172.21.15.94:0/1299494915 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33da029640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33d4101390 0x7f33d4075080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33da82a640 1 -- 172.21.15.94:0/1299494915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33d4101390 msgr2=0x7f33d4075080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33da82a640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33d4101390 0x7f33d4075080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33da82a640 1 -- 172.21.15.94:0/1299494915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d4106460 msgr2=0x7f33d4073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33da82a640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d4106460 0x7f33d4073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33da82a640 1 -- 172.21.15.94:0/1299494915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc009840 con 0x7f33d4073a10 2024-02-17T12:00:08.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.964+0000 7f33d9828640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d4106460 0x7f33d4073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:08.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.965+0000 7f33da82a640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d4073a10 0x7f33d4073ea0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f33cc00efd0 tx=0x7f33cc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.965+0000 7f33cb7fe640 1 -- 172.21.15.94:0/1299494915 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc009b40 con 0x7f33d4073a10 2024-02-17T12:00:08.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.965+0000 7f33cb7fe640 1 -- 172.21.15.94:0/1299494915 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33cc003b70 con 0x7f33d4073a10 2024-02-17T12:00:08.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.965+0000 7f33cb7fe640 1 -- 172.21.15.94:0/1299494915 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0108f0 con 0x7f33d4073a10 2024-02-17T12:00:08.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.965+0000 7f33dc28d640 1 -- 172.21.15.94:0/1299494915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c4009560 con 0x7f33d4073a10 2024-02-17T12:00:08.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.965+0000 7f33dc28d640 1 -- 172.21.15.94:0/1299494915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d41aad70 con 0x7f33d4073a10 2024-02-17T12:00:08.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.970+0000 7f33cb7fe640 1 -- 172.21.15.94:0/1299494915 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f33cc003ce0 con 0x7f33d4073a10 2024-02-17T12:00:08.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.971+0000 7f33dee5c740 1 Processor -- start 2024-02-17T12:00:08.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.971+0000 7f33cb7fe640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f33c00a22b0 0x7f33c00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.971+0000 7f33cb7fe640 1 -- 172.21.15.94:0/1299494915 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f33cc014070 con 0x7f33d4073a10 2024-02-17T12:00:08.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.971+0000 7f33d9828640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f33c00a22b0 0x7f33c00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.971+0000 7f33dee5c740 1 -- start start 2024-02-17T12:00:08.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33dee5c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2250 0x560042ad2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33dee5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2b70 0x560042adb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33dee5c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 0x560042addb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33dee5c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560042a8d960 con 0x560042ad2250 2024-02-17T12:00:08.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33dee5c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560042a41310 con 0x560042adb750 2024-02-17T12:00:08.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33dee5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560042a3e840 con 0x560042ad2b70 2024-02-17T12:00:08.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33da029640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2250 0x560042ad2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33da029640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2250 0x560042ad2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33260/0 (socket says 172.21.15.94:33260) 2024-02-17T12:00:08.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33da029640 1 -- 172.21.15.94:0/4146753952 learned_addr learned my addr 172.21.15.94:0/4146753952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33da82a640 1 --2- 172.21.15.94:0/4146753952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 0x560042addb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33d9828640 1 --2- 172.21.15.94:0/4146753952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2b70 0x560042adb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33d9828640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f33c00a22b0 0x7f33c00a4770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f33c4005ab0 tx=0x7f33c4005a20 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33da029640 1 -- 172.21.15.94:0/4146753952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2b70 msgr2=0x560042adb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33da029640 1 --2- 172.21.15.94:0/4146753952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2b70 0x560042adb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33da029640 1 -- 172.21.15.94:0/4146753952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 msgr2=0x560042addb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33da029640 1 --2- 172.21.15.94:0/4146753952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 0x560042addb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.972+0000 7f33da029640 1 -- 172.21.15.94:0/4146753952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600429fc060 con 0x560042ad2250 2024-02-17T12:00:08.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.973+0000 7f33da029640 1 --2- 172.21.15.94:0/4146753952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2250 0x560042ad2630 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009d20 tx=0x7f33d0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9997dcbd1d1068 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.973+0000 7f33c8bfe640 1 -- 172.21.15.94:0/4146753952 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0019630 con 0x560042ad2250 2024-02-17T12:00:08.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.974+0000 7f33c8bfe640 1 -- 172.21.15.94:0/4146753952 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33d0019c70 con 0x560042ad2250 2024-02-17T12:00:08.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.974+0000 7f33c8bfe640 1 -- 172.21.15.94:0/4146753952 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d000ec70 con 0x560042ad2250 2024-02-17T12:00:08.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.975+0000 7f33dee5c740 1 -- 172.21.15.94:0/4146753952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2250 msgr2=0x560042ad2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.975+0000 7f33dee5c740 1 --2- 172.21.15.94:0/4146753952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2250 0x560042ad2630 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009d20 tx=0x7f33d0011810 comp rx=0 tx=0).stop 2024-02-17T12:00:08.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.975+0000 7f33dee5c740 1 -- 172.21.15.94:0/4146753952 shutdown_connections 2024-02-17T12:00:08.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.975+0000 7f33dee5c740 1 --2- 172.21.15.94:0/4146753952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 0x560042addb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.975+0000 7f33dee5c740 1 --2- 172.21.15.94:0/4146753952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2b70 0x560042adb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.975+0000 7f33dee5c740 1 --2- 172.21.15.94:0/4146753952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2250 0x560042ad2630 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.975+0000 7f33dee5c740 1 -- 172.21.15.94:0/4146753952 >> 172.21.15.94:0/4146753952 conn(0x560042acc980 msgr2=0x560042accd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.975+0000 7f33dee5c740 1 -- 172.21.15.94:0/4146753952 shutdown_connections 2024-02-17T12:00:08.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.975+0000 7f33dee5c740 1 -- 172.21.15.94:0/4146753952 wait complete. 2024-02-17T12:00:08.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.976+0000 7f33dee5c740 1 -- start start 2024-02-17T12:00:08.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.976+0000 7f33dee5c740 1 Processor -- start 2024-02-17T12:00:08.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.977+0000 7f33dee5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2250 0x560042ad87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.977+0000 7f33dee5c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2b70 0x560042ad8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.977+0000 7f33dee5c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 0x560042ad9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.977+0000 7f33dee5c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560042a3a020 con 0x560042ad2b70 2024-02-17T12:00:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.977+0000 7f33dee5c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56004270d430 con 0x560042adb750 2024-02-17T12:00:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.977+0000 7f33dee5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600426c9e00 con 0x560042ad2250 2024-02-17T12:00:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.977+0000 7f33dee5c740 10 client.0 _ll_register_callbacks cb 0x5600429e1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.978+0000 7f33d9828640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2b70 0x560042ad8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.978+0000 7f33d9828640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2b70 0x560042ad8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33268/0 (socket says 172.21.15.94:33268) 2024-02-17T12:00:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.978+0000 7f33d9828640 1 -- 172.21.15.94:0/3597572185 learned_addr learned my addr 172.21.15.94:0/3597572185 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.978+0000 7f33da029640 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2250 0x560042ad87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.978+0000 7f33da82a640 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 0x560042ad9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.978+0000 7f33d9828640 1 -- 172.21.15.94:0/3597572185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2250 msgr2=0x560042ad87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.978+0000 7f33d9828640 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2250 0x560042ad87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.978+0000 7f33d9828640 1 -- 172.21.15.94:0/3597572185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 msgr2=0x560042ad9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:08.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.978+0000 7f33d9828640 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 0x560042ad9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.978+0000 7f33d9828640 1 -- 172.21.15.94:0/3597572185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d0009950 con 0x560042ad2b70 2024-02-17T12:00:08.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.978+0000 7f33da029640 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2250 0x560042ad87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:00:08.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.978+0000 7f33da82a640 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 0x560042ad9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:00:08.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.979+0000 7f33d9828640 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2b70 0x560042ad8ce0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f33c402c780 tx=0x7f33c4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b9968437acbaa4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:08.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.979+0000 7f33bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:00:08.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.980+0000 7f33bdffb640 1 -- 172.21.15.94:0/3597572185 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c40044d0 con 0x560042ad2b70 2024-02-17T12:00:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.980+0000 7f33bdffb640 1 -- 172.21.15.94:0/3597572185 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f33c4047040 con 0x560042ad2b70 2024-02-17T12:00:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.980+0000 7f33dee5c740 1 -- 172.21.15.94:0/3597572185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600427397e0 con 0x560042ad2b70 2024-02-17T12:00:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.980+0000 7f33bdffb640 1 -- 172.21.15.94:0/3597572185 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33c4005000 con 0x560042ad2b70 2024-02-17T12:00:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.981+0000 7f33dee5c740 10 client.16065 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.981+0000 7f33dee5c740 1 -- 172.21.15.94:0/3597572185 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x560042a386a0 con 0x560042ad2b70 2024-02-17T12:00:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.981+0000 7f33bdffb640 1 -- 172.21.15.94:0/3597572185 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33c405b030 con 0x560042ad2b70 2024-02-17T12:00:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.981+0000 7f33bdffb640 1 -- 172.21.15.94:0/3597572185 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f33c4003c50 con 0x560042ad2b70 2024-02-17T12:00:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.981+0000 7f33dee5c740 10 client.16065 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.981+0000 7f33dee5c740 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.981+0000 7f33dee5c740 20 client.16065 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.981+0000 7f33dee5c740 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560042aeba20 0x560042aede40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:08.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.981+0000 7f33dee5c740 4 client.16065 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:08.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.981+0000 7f33dee5c740 1 -- 172.21.15.94:0/3597572185 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560042776720 con 0x560042aeba20 2024-02-17T12:00:08.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.982+0000 7f33da029640 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560042aeba20 0x560042aede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:08.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.982+0000 7f33da029640 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560042aeba20 0x560042aede40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a29d68678e03976 server_cookie=7d43dc52ef5df1ba in_seq=0 out_seq=0 2024-02-17T12:00:08.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.982+0000 7f33bdffb640 10 client.16065 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:08.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.983+0000 7f33bdffb640 1 -- 172.21.15.94:0/3597572185 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560042776720 con 0x560042aeba20 2024-02-17T12:00:08.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.983+0000 7f33bdffb640 10 client.16065 handle_command_reply: tid=0 2024-02-17T12:00:08.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.983+0000 7f33dee5c740 1 client.16065 shutdown 2024-02-17T12:00:08.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.984+0000 7f33dee5c740 1 -- 172.21.15.94:0/3597572185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2b70 msgr2=0x560042ad8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:08.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.984+0000 7f33dee5c740 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2b70 0x560042ad8ce0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f33c402c780 tx=0x7f33c4009260 comp rx=0 tx=0).stop 2024-02-17T12:00:08.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.984+0000 7f33dee5c740 1 -- 172.21.15.94:0/3597572185 shutdown_connections 2024-02-17T12:00:08.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.984+0000 7f33dee5c740 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560042aeba20 0x560042aede40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.984+0000 7f33dee5c740 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 0x560042ad9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.984+0000 7f33bdffb640 0 client.16065 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:08.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.984+0000 7f33dee5c740 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2b70 0x560042ad8ce0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.984+0000 7f33dee5c740 1 --2- 172.21.15.94:0/3597572185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2250 0x560042ad87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:08.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.984+0000 7f33dee5c740 1 -- 172.21.15.94:0/3597572185 >> 172.21.15.94:0/3597572185 conn(0x560042acc980 msgr2=0x560042ace6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:08.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.984+0000 7f33dee5c740 1 -- 172.21.15.94:0/3597572185 shutdown_connections 2024-02-17T12:00:08.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.985+0000 7f33dee5c740 1 -- 172.21.15.94:0/3597572185 wait complete. 2024-02-17T12:00:08.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:08.985+0000 7f33dee5c740 20 client.16065 trim_cache size 0 max 16384 2024-02-17T12:00:09.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.003+0000 7f33dee5c740 1 Processor -- start 2024-02-17T12:00:09.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.003+0000 7f33dee5c740 1 -- start start 2024-02-17T12:00:09.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.003+0000 7f33dee5c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2250 0x560042adaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:09.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.003+0000 7f33dee5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2b70 0x560042ad9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:09.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.003+0000 7f33dee5c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 0x560042ad5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:09.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.003+0000 7f33dee5c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600429bf130 con 0x560042ad2250 2024-02-17T12:00:09.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.003+0000 7f33dee5c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560042ac82b0 con 0x560042adb750 2024-02-17T12:00:09.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.003+0000 7f33dee5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560042af3050 con 0x560042ad2b70 2024-02-17T12:00:09.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.003+0000 7f33da029640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2250 0x560042adaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:09.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.003+0000 7f33d9828640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2b70 0x560042ad9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:09.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.003+0000 7f33da029640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2250 0x560042adaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33284/0 (socket says 172.21.15.94:33284) 2024-02-17T12:00:09.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.003+0000 7f33da029640 1 -- 172.21.15.94:0/1503235800 learned_addr learned my addr 172.21.15.94:0/1503235800 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:09.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.003+0000 7f33da82a640 1 --2- 172.21.15.94:0/1503235800 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 0x560042ad5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:09.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.004+0000 7f33d9828640 1 -- 172.21.15.94:0/1503235800 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 msgr2=0x560042ad5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:09.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.004+0000 7f33d9828640 1 --2- 172.21.15.94:0/1503235800 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 0x560042ad5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.004+0000 7f33d9828640 1 -- 172.21.15.94:0/1503235800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2250 msgr2=0x560042adaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:09.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.004+0000 7f33d9828640 1 --2- 172.21.15.94:0/1503235800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2250 0x560042adaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.004+0000 7f33d9828640 1 -- 172.21.15.94:0/1503235800 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600429d4c20 con 0x560042ad2b70 2024-02-17T12:00:09.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.005+0000 7f33d9828640 1 --2- 172.21.15.94:0/1503235800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2b70 0x560042ad9040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33c403f440 tx=0x7f33c40044d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35b8f6bbadba40c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:09.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.005+0000 7f33bffff640 1 -- 172.21.15.94:0/1503235800 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4004df0 con 0x560042ad2b70 2024-02-17T12:00:09.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.005+0000 7f33bffff640 1 -- 172.21.15.94:0/1503235800 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33c405b030 con 0x560042ad2b70 2024-02-17T12:00:09.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.005+0000 7f33bffff640 1 -- 172.21.15.94:0/1503235800 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4004df0 con 0x560042ad2b70 2024-02-17T12:00:09.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.006+0000 7f33dee5c740 1 -- 172.21.15.94:0/1503235800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2b70 msgr2=0x560042ad9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:09.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.006+0000 7f33dee5c740 1 --2- 172.21.15.94:0/1503235800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2b70 0x560042ad9040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33c403f440 tx=0x7f33c40044d0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.006+0000 7f33dee5c740 1 -- 172.21.15.94:0/1503235800 shutdown_connections 2024-02-17T12:00:09.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.006+0000 7f33dee5c740 1 --2- 172.21.15.94:0/1503235800 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042adb750 0x560042ad5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.006+0000 7f33dee5c740 1 --2- 172.21.15.94:0/1503235800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042ad2b70 0x560042ad9040 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.006+0000 7f33dee5c740 1 --2- 172.21.15.94:0/1503235800 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042ad2250 0x560042adaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.006+0000 7f33dee5c740 1 -- 172.21.15.94:0/1503235800 >> 172.21.15.94:0/1503235800 conn(0x560042acc980 msgr2=0x560042ad3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:09.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.006+0000 7f33dee5c740 1 -- 172.21.15.94:0/1503235800 shutdown_connections 2024-02-17T12:00:09.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.006+0000 7f33dee5c740 1 -- 172.21.15.94:0/1503235800 wait complete. 2024-02-17T12:00:09.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.007+0000 7f33dee5c740 1 -- start start 2024-02-17T12:00:09.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.007+0000 7f33dee5c740 1 Processor -- start 2024-02-17T12:00:09.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.007+0000 7f33dee5c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042ad2250 0x560042ae0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:09.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.007+0000 7f33dee5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042adb750 0x560042ae1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:09.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.007+0000 7f33dee5c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042aeba20 0x560042ae1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:09.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.007+0000 7f33dee5c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560042b41f30 con 0x560042aeba20 2024-02-17T12:00:09.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.007+0000 7f33dee5c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560042b42070 con 0x560042ad2250 2024-02-17T12:00:09.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.007+0000 7f33dee5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560042b421b0 con 0x560042adb750 2024-02-17T12:00:09.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.007+0000 7f33dee5c740 10 client.0 _ll_register_callbacks cb 0x7f33d0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:09.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.007+0000 7f33da029640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042ad2250 0x560042ae0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:09.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.007+0000 7f33d9828640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042adb750 0x560042ae1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:09.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.007+0000 7f33d9828640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042adb750 0x560042ae1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:32870/0 (socket says 172.21.15.94:32870) 2024-02-17T12:00:09.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.007+0000 7f33d9828640 1 -- 172.21.15.94:0/3709332546 learned_addr learned my addr 172.21.15.94:0/3709332546 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:09.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.008+0000 7f33da82a640 1 --2- 172.21.15.94:0/3709332546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042aeba20 0x560042ae1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:09.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.008+0000 7f33da82a640 1 -- 172.21.15.94:0/3709332546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042adb750 msgr2=0x560042ae1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:09.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.008+0000 7f33da82a640 1 --2- 172.21.15.94:0/3709332546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042adb750 0x560042ae1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.008+0000 7f33da82a640 1 -- 172.21.15.94:0/3709332546 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042ad2250 msgr2=0x560042ae0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:09.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.008+0000 7f33da82a640 1 --2- 172.21.15.94:0/3709332546 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042ad2250 0x560042ae0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.008+0000 7f33da82a640 1 -- 172.21.15.94:0/3709332546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560042a386a0 con 0x560042aeba20 2024-02-17T12:00:09.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.008+0000 7f33da82a640 1 --2- 172.21.15.94:0/3709332546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042aeba20 0x560042ae1a40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0cbcd0 tx=0x7f33cc09b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb07bbab05794d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:09.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.009+0000 7f33bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:00:09.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.009+0000 7f33bdffb640 1 -- 172.21.15.94:0/3709332546 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0ce2d0 con 0x560042aeba20 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.009+0000 7f33bdffb640 1 -- 172.21.15.94:0/3709332546 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f33cc004b40 con 0x560042aeba20 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.009+0000 7f33dee5c740 1 -- 172.21.15.94:0/3709332546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33d0019c70 con 0x560042aeba20 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.009+0000 7f33bdffb640 1 -- 172.21.15.94:0/3709332546 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33cc0ce430 con 0x560042aeba20 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.009+0000 7f33dee5c740 10 client.16071 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.009+0000 7f33dee5c740 1 -- 172.21.15.94:0/3709332546 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x5600429fa8c0 con 0x560042aeba20 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.009+0000 7f33bdffb640 1 -- 172.21.15.94:0/3709332546 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33cc07d080 con 0x560042aeba20 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.010+0000 7f33bdffb640 1 -- 172.21.15.94:0/3709332546 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f33cc09a450 con 0x560042aeba20 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.010+0000 7f33dee5c740 10 client.16071 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.010+0000 7f33dee5c740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.010+0000 7f33dee5c740 20 client.16071 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.010+0000 7f33dee5c740 1 --2- 172.21.15.94:0/3709332546 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560042b43a80 0x560042b45ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.010+0000 7f33dee5c740 4 client.16071 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.010+0000 7f33dee5c740 1 -- 172.21.15.94:0/3709332546 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5600427c5890 con 0x560042b43a80 2024-02-17T12:00:09.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.010+0000 7f33da029640 1 --2- 172.21.15.94:0/3709332546 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560042b43a80 0x560042b45ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.011+0000 7f33bdffb640 10 client.16071 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.011+0000 7f33da029640 1 --2- 172.21.15.94:0/3709332546 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560042b43a80 0x560042b45ea0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5d99ef4cd7d45f server_cookie=b6545be15f69bbe6 in_seq=0 out_seq=0 2024-02-17T12:00:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.011+0000 7f33bdffb640 1 -- 172.21.15.94:0/3709332546 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5600427c5890 con 0x560042b43a80 2024-02-17T12:00:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.011+0000 7f33bdffb640 10 client.16071 handle_command_reply: tid=0 2024-02-17T12:00:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.011+0000 7f33dee5c740 1 client.16071 shutdown 2024-02-17T12:00:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.011+0000 7f33dee5c740 1 -- 172.21.15.94:0/3709332546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042aeba20 msgr2=0x560042ae1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.011+0000 7f33dee5c740 1 --2- 172.21.15.94:0/3709332546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042aeba20 0x560042ae1a40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0cbcd0 tx=0x7f33cc09b7b0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.011+0000 7f33dee5c740 1 -- 172.21.15.94:0/3709332546 shutdown_connections 2024-02-17T12:00:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.011+0000 7f33dee5c740 1 --2- 172.21.15.94:0/3709332546 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560042b43a80 0x560042b45ea0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.011+0000 7f33dee5c740 1 --2- 172.21.15.94:0/3709332546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560042aeba20 0x560042ae1a40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.011+0000 7f33dee5c740 1 --2- 172.21.15.94:0/3709332546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560042adb750 0x560042ae1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.011+0000 7f33dee5c740 1 --2- 172.21.15.94:0/3709332546 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560042ad2250 0x560042ae0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.011+0000 7f33dee5c740 1 -- 172.21.15.94:0/3709332546 >> 172.21.15.94:0/3709332546 conn(0x560042acc980 msgr2=0x560042acb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.012+0000 7f33bdffb640 0 client.16071 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.012+0000 7f33dee5c740 1 -- 172.21.15.94:0/3709332546 shutdown_connections 2024-02-17T12:00:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.012+0000 7f33dee5c740 1 -- 172.21.15.94:0/3709332546 wait complete. 2024-02-17T12:00:09.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.012+0000 7f33dee5c740 20 client.16071 trim_cache size 0 max 16384 2024-02-17T12:00:09.013 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:00:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.013+0000 7f33dc28d640 1 -- 172.21.15.94:0/1299494915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f33c00a22b0 msgr2=0x7f33c00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.013+0000 7f33dc28d640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f33c00a22b0 0x7f33c00a4770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f33c4005ab0 tx=0x7f33c4005a20 comp rx=0 tx=0).stop 2024-02-17T12:00:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.013+0000 7f33dc28d640 1 -- 172.21.15.94:0/1299494915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d4073a10 msgr2=0x7f33d4073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.013+0000 7f33dc28d640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d4073a10 0x7f33d4073ea0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f33cc00efd0 tx=0x7f33cc00c6a0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.013+0000 7f33dc28d640 1 -- 172.21.15.94:0/1299494915 shutdown_connections 2024-02-17T12:00:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.013+0000 7f33dc28d640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f33c00a22b0 0x7f33c00a4770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.013+0000 7f33dc28d640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33d4073a10 0x7f33d4073ea0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.013+0000 7f33dc28d640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33d4106460 0x7f33d4073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.013+0000 7f33dc28d640 1 --2- 172.21.15.94:0/1299494915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33d4101390 0x7f33d4075080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:09.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.013+0000 7f33dc28d640 1 -- 172.21.15.94:0/1299494915 >> 172.21.15.94:0/1299494915 conn(0x7f33d40fcbb0 msgr2=0x7f33d40fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:09.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.013+0000 7f33dc28d640 1 -- 172.21.15.94:0/1299494915 shutdown_connections 2024-02-17T12:00:09.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:09.013+0000 7f33dc28d640 1 -- 172.21.15.94:0/1299494915 wait complete. 2024-02-17T12:00:09.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:08 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2463926685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:00:09.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:08 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2463926685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:00:09.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:08 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2463926685' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:00:10.024 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:00:10.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:00:10.025 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:00:10.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:09 smithi189 ceph-mon[36588]: pgmap v64: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 1.74k op/s 2024-02-17T12:00:10.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.168+0000 7f9e4a826640 1 -- 172.21.15.94:0/3217592925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e3c096b50 msgr2=0x7f9e3c09a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.168+0000 7f9e4a826640 1 --2- 172.21.15.94:0/3217592925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e3c096b50 0x7f9e3c09a960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009930 tx=0x7f9e3402bc30 comp rx=0 tx=0).stop 2024-02-17T12:00:10.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.168+0000 7f9e4a826640 1 -- 172.21.15.94:0/3217592925 shutdown_connections 2024-02-17T12:00:10.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.168+0000 7f9e4a826640 1 --2- 172.21.15.94:0/3217592925 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e3c09b040 0x7f9e3c0a2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.168+0000 7f9e4a826640 1 --2- 172.21.15.94:0/3217592925 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e3c096b50 0x7f9e3c09a960 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.168+0000 7f9e4a826640 1 --2- 172.21.15.94:0/3217592925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e3c0961a0 0x7f9e3c096580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.168+0000 7f9e4a826640 1 -- 172.21.15.94:0/3217592925 >> 172.21.15.94:0/3217592925 conn(0x7f9e3c00b840 msgr2=0x7f9e3c00bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:10.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.168+0000 7f9e4a826640 1 -- 172.21.15.94:0/3217592925 shutdown_connections 2024-02-17T12:00:10.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.168+0000 7f9e4a826640 1 -- 172.21.15.94:0/3217592925 wait complete. 2024-02-17T12:00:10.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e4a826640 1 Processor -- start 2024-02-17T12:00:10.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e4a826640 1 -- start start 2024-02-17T12:00:10.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e4a826640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e3c0961a0 0x7f9e3c0076f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e4a826640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e3c09b040 0x7f9e3c007c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e4a826640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e3c008170 0x7f9e3c13a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e4a826640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c13ab80 con 0x7f9e3c0961a0 2024-02-17T12:00:10.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e4a826640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c13acf0 con 0x7f9e3c008170 2024-02-17T12:00:10.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e4a826640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c13aeb0 con 0x7f9e3c09b040 2024-02-17T12:00:10.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e49824640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e3c0961a0 0x7f9e3c0076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e49824640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e3c0961a0 0x7f9e3c0076f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33312/0 (socket says 172.21.15.94:33312) 2024-02-17T12:00:10.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e49824640 1 -- 172.21.15.94:0/1497653438 learned_addr learned my addr 172.21.15.94:0/1497653438 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:10.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e4a025640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e3c008170 0x7f9e3c13a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e49023640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e3c09b040 0x7f9e3c007c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e49824640 1 -- 172.21.15.94:0/1497653438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e3c09b040 msgr2=0x7f9e3c007c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e49824640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e3c09b040 0x7f9e3c007c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e49824640 1 -- 172.21.15.94:0/1497653438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e3c008170 msgr2=0x7f9e3c13a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e49824640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e3c008170 0x7f9e3c13a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e49824640 1 -- 172.21.15.94:0/1497653438 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e34009560 con 0x7f9e3c0961a0 2024-02-17T12:00:10.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.169+0000 7f9e4a025640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e3c008170 0x7f9e3c13a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:10.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.170+0000 7f9e49824640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e3c0961a0 0x7f9e3c0076f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000e9f0 tx=0x7f9e4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:10.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.170+0000 7f9e3affd640 1 -- 172.21.15.94:0/1497653438 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40009700 con 0x7f9e3c0961a0 2024-02-17T12:00:10.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.170+0000 7f9e3affd640 1 -- 172.21.15.94:0/1497653438 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e4000cd80 con 0x7f9e3c0961a0 2024-02-17T12:00:10.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.170+0000 7f9e4a826640 1 -- 172.21.15.94:0/1497653438 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c13b190 con 0x7f9e3c0961a0 2024-02-17T12:00:10.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.170+0000 7f9e3affd640 1 -- 172.21.15.94:0/1497653438 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40018920 con 0x7f9e3c0961a0 2024-02-17T12:00:10.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.170+0000 7f9e4a826640 1 -- 172.21.15.94:0/1497653438 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c13b670 con 0x7f9e3c0961a0 2024-02-17T12:00:10.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.173+0000 7f9e3affd640 1 -- 172.21.15.94:0/1497653438 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9e40018a80 con 0x7f9e3c0961a0 2024-02-17T12:00:10.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.173+0000 7f9e4e666740 1 Processor -- start 2024-02-17T12:00:10.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.173+0000 7f9e4e666740 1 -- start start 2024-02-17T12:00:10.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.173+0000 7f9e4e666740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22550 0x56491fe22930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.173+0000 7f9e4e666740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe22f00 0x56491fe2b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e4e666740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe2ba80 0x56491fe2ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e4e666740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491fd90f90 con 0x56491fe2ba80 2024-02-17T12:00:10.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e4e666740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491fd8e4c0 con 0x56491fe22f00 2024-02-17T12:00:10.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e4e666740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491fd89ca0 con 0x56491fe22550 2024-02-17T12:00:10.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e49824640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe22f00 0x56491fe2b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e49824640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe22f00 0x56491fe2b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59368/0 (socket says 172.21.15.94:59368) 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e49824640 1 -- 172.21.15.94:0/748571578 learned_addr learned my addr 172.21.15.94:0/748571578 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e4a025640 1 --2- 172.21.15.94:0/748571578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22550 0x56491fe22930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e49023640 1 --2- 172.21.15.94:0/748571578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe2ba80 0x56491fe2ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e49824640 1 -- 172.21.15.94:0/748571578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22550 msgr2=0x56491fe22930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e49824640 1 --2- 172.21.15.94:0/748571578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22550 0x56491fe22930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e49824640 1 -- 172.21.15.94:0/748571578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe2ba80 msgr2=0x56491fe2ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e3affd640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9e300a2260 0x7f9e300a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e49824640 1 --2- 172.21.15.94:0/748571578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe2ba80 0x56491fe2ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.174+0000 7f9e49824640 1 -- 172.21.15.94:0/748571578 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56491fd4c0b0 con 0x56491fe22f00 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.175+0000 7f9e4a025640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9e300a2260 0x7f9e300a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.175+0000 7f9e49824640 1 --2- 172.21.15.94:0/748571578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe22f00 0x56491fe2b4b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9e400cbad0 tx=0x7f9e400aea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=973eb1fccd3650d1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.175+0000 7f9e3affd640 1 -- 172.21.15.94:0/1497653438 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9e40014070 con 0x7f9e3c0961a0 2024-02-17T12:00:10.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.175+0000 7f9e2ffff640 1 -- 172.21.15.94:0/748571578 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4009d050 con 0x56491fe22f00 2024-02-17T12:00:10.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.175+0000 7f9e2ffff640 1 -- 172.21.15.94:0/748571578 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e4009ac80 con 0x56491fe22f00 2024-02-17T12:00:10.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.175+0000 7f9e2ffff640 1 -- 172.21.15.94:0/748571578 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4009aeb0 con 0x56491fe22f00 2024-02-17T12:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.176+0000 7f9e4e666740 1 -- 172.21.15.94:0/748571578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe22f00 msgr2=0x56491fe2b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.176+0000 7f9e4e666740 1 --2- 172.21.15.94:0/748571578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe22f00 0x56491fe2b4b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9e400cbad0 tx=0x7f9e400aea90 comp rx=0 tx=0).stop 2024-02-17T12:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.176+0000 7f9e4a025640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9e300a2260 0x7f9e300a4720 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9e440668b0 tx=0x7f9e44078040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.176+0000 7f9e4e666740 1 -- 172.21.15.94:0/748571578 shutdown_connections 2024-02-17T12:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.176+0000 7f9e4e666740 1 --2- 172.21.15.94:0/748571578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe2ba80 0x56491fe2ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.176+0000 7f9e4e666740 1 --2- 172.21.15.94:0/748571578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe22f00 0x56491fe2b4b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.176+0000 7f9e4e666740 1 --2- 172.21.15.94:0/748571578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22550 0x56491fe22930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.176+0000 7f9e4e666740 1 -- 172.21.15.94:0/748571578 >> 172.21.15.94:0/748571578 conn(0x56491fe1ca10 msgr2=0x56491fe1cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.177+0000 7f9e4e666740 1 -- 172.21.15.94:0/748571578 shutdown_connections 2024-02-17T12:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.177+0000 7f9e4e666740 1 -- 172.21.15.94:0/748571578 wait complete. 2024-02-17T12:00:10.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.177+0000 7f9e4e666740 1 -- start start 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.177+0000 7f9e4e666740 1 Processor -- start 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e4e666740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 0x56491fe289c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e4e666740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22f00 0x56491fe28f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e4e666740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe2ba80 0x56491fe29440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e4e666740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491fd2ef40 con 0x56491fe22550 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e4e666740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491fa5d4b0 con 0x56491fe2ba80 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e4e666740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491fd0f670 con 0x56491fe22f00 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e4e666740 10 client.0 _ll_register_callbacks cb 0x56491fd7cf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e4a025640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe2ba80 0x56491fe29440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e49023640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22f00 0x56491fe28f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e49824640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 0x56491fe289c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e49824640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 0x56491fe289c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33334/0 (socket says 172.21.15.94:33334) 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e4a025640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe2ba80 0x56491fe29440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59370/0 (socket says 172.21.15.94:59370) 2024-02-17T12:00:10.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e49023640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22f00 0x56491fe28f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:32886/0 (socket says 172.21.15.94:32886) 2024-02-17T12:00:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e49824640 1 -- 172.21.15.94:0/901351993 learned_addr learned my addr 172.21.15.94:0/901351993 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e49824640 1 -- 172.21.15.94:0/901351993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22f00 msgr2=0x56491fe28f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e49824640 1 --2- 172.21.15.94:0/901351993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22f00 0x56491fe28f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e49824640 1 -- 172.21.15.94:0/901351993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe2ba80 msgr2=0x56491fe29440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e49824640 1 --2- 172.21.15.94:0/901351993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe2ba80 0x56491fe29440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.178+0000 7f9e49824640 1 -- 172.21.15.94:0/901351993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c13b670 con 0x56491fe22550 2024-02-17T12:00:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.179+0000 7f9e49824640 1 --2- 172.21.15.94:0/901351993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 0x56491fe289c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9e400cc160 tx=0x7f9e40096b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d3a308a39cd2e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.179+0000 7f9e2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:00:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.179+0000 7f9e4a025640 1 --2- 172.21.15.94:0/901351993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe2ba80 0x56491fe29440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-02-17T12:00:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.179+0000 7f9e49023640 1 --2- 172.21.15.94:0/901351993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22f00 0x56491fe28f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-02-17T12:00:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.179+0000 7f9e2d7fa640 1 -- 172.21.15.94:0/901351993 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4009ac80 con 0x56491fe22550 2024-02-17T12:00:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.179+0000 7f9e2d7fa640 1 -- 172.21.15.94:0/901351993 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9e4009ef50 con 0x56491fe22550 2024-02-17T12:00:10.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.179+0000 7f9e4e666740 1 -- 172.21.15.94:0/901351993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56491fd63f40 con 0x56491fe22550 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.179+0000 7f9e2d7fa640 1 -- 172.21.15.94:0/901351993 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e4009a9f0 con 0x56491fe22550 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.179+0000 7f9e4e666740 10 client.16083 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.179+0000 7f9e4e666740 1 -- 172.21.15.94:0/901351993 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x56491fd88320 con 0x56491fe22550 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.179+0000 7f9e2d7fa640 1 -- 172.21.15.94:0/901351993 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56491fd63f40 con 0x56491fe22550 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.180+0000 7f9e2d7fa640 1 -- 172.21.15.94:0/901351993 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9e400cd070 con 0x56491fe22550 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.180+0000 7f9e4e666740 10 client.16083 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.180+0000 7f9e4e666740 10 client.16083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.180+0000 7f9e4e666740 20 client.16083 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.180+0000 7f9e4e666740 1 --2- 172.21.15.94:0/901351993 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56491fe3bbf0 0x56491fe3dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.180+0000 7f9e4e666740 4 client.16083 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.180+0000 7f9e4e666740 1 -- 172.21.15.94:0/901351993 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56491fd80310 con 0x56491fe3bbf0 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.180+0000 7f9e49023640 1 --2- 172.21.15.94:0/901351993 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56491fe3bbf0 0x56491fe3dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.180+0000 7f9e2d7fa640 10 client.16083 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:10.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.180+0000 7f9e49023640 1 --2- 172.21.15.94:0/901351993 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56491fe3bbf0 0x56491fe3dfe0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c090451d7e6287 server_cookie=6137f483bc4eab2 in_seq=0 out_seq=0 2024-02-17T12:00:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.181+0000 7f9e2d7fa640 1 -- 172.21.15.94:0/901351993 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56491fd80310 con 0x56491fe3bbf0 2024-02-17T12:00:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.181+0000 7f9e2d7fa640 10 client.16083 handle_command_reply: tid=0 2024-02-17T12:00:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.181+0000 7f9e4e666740 1 client.16083 shutdown 2024-02-17T12:00:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.181+0000 7f9e4e666740 1 -- 172.21.15.94:0/901351993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 msgr2=0x56491fe289c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.181+0000 7f9e4e666740 1 --2- 172.21.15.94:0/901351993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 0x56491fe289c0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9e400cc160 tx=0x7f9e40096b50 comp rx=0 tx=0).stop 2024-02-17T12:00:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.181+0000 7f9e4e666740 1 -- 172.21.15.94:0/901351993 shutdown_connections 2024-02-17T12:00:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.181+0000 7f9e4e666740 1 --2- 172.21.15.94:0/901351993 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56491fe3bbf0 0x56491fe3dfe0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.181+0000 7f9e4e666740 1 --2- 172.21.15.94:0/901351993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe2ba80 0x56491fe29440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.181+0000 7f9e4e666740 1 --2- 172.21.15.94:0/901351993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22f00 0x56491fe28f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.181+0000 7f9e4e666740 1 --2- 172.21.15.94:0/901351993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 0x56491fe289c0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.182+0000 7f9e4e666740 1 -- 172.21.15.94:0/901351993 >> 172.21.15.94:0/901351993 conn(0x56491fe1ca10 msgr2=0x56491fe1bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:10.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.182+0000 7f9e2d7fa640 0 client.16083 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:10.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.182+0000 7f9e4e666740 1 -- 172.21.15.94:0/901351993 shutdown_connections 2024-02-17T12:00:10.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.182+0000 7f9e4e666740 1 -- 172.21.15.94:0/901351993 wait complete. 2024-02-17T12:00:10.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.182+0000 7f9e4e666740 20 client.16083 trim_cache size 0 max 16384 2024-02-17T12:00:10.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:09 smithi125 ceph-mon[36819]: pgmap v64: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 1.74k op/s 2024-02-17T12:00:10.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.201+0000 7f9e4e666740 1 Processor -- start 2024-02-17T12:00:10.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.201+0000 7f9e4e666740 1 -- start start 2024-02-17T12:00:10.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e4e666740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 0x56491fe2b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e4e666740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe22f00 0x56491fe29260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e4e666740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe2ba80 0x56491fe1bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e4e666740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491fe43270 con 0x56491fe22550 2024-02-17T12:00:10.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e4e666740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491fe0e610 con 0x56491fe22f00 2024-02-17T12:00:10.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e4e666740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491fe245f0 con 0x56491fe2ba80 2024-02-17T12:00:10.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e49824640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 0x56491fe2b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e49824640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 0x56491fe2b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33340/0 (socket says 172.21.15.94:33340) 2024-02-17T12:00:10.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e49824640 1 -- 172.21.15.94:0/604414506 learned_addr learned my addr 172.21.15.94:0/604414506 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:10.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e4a025640 1 --2- 172.21.15.94:0/604414506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe2ba80 0x56491fe1bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e49023640 1 --2- 172.21.15.94:0/604414506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe22f00 0x56491fe29260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e49824640 1 -- 172.21.15.94:0/604414506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe2ba80 msgr2=0x56491fe1bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e49824640 1 --2- 172.21.15.94:0/604414506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe2ba80 0x56491fe1bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e49824640 1 -- 172.21.15.94:0/604414506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe22f00 msgr2=0x56491fe29260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e49824640 1 --2- 172.21.15.94:0/604414506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe22f00 0x56491fe29260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e49824640 1 -- 172.21.15.94:0/604414506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56491fd24c00 con 0x56491fe22550 2024-02-17T12:00:10.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.202+0000 7f9e49824640 1 --2- 172.21.15.94:0/604414506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 0x56491fe2b0f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000c6a0 tx=0x7f9e40010f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00402d16e9dcd64 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:10.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.203+0000 7f9e2f7fe640 1 -- 172.21.15.94:0/604414506 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4009d020 con 0x56491fe22550 2024-02-17T12:00:10.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.203+0000 7f9e2f7fe640 1 -- 172.21.15.94:0/604414506 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e400d5030 con 0x56491fe22550 2024-02-17T12:00:10.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.203+0000 7f9e2f7fe640 1 -- 172.21.15.94:0/604414506 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4009a9f0 con 0x56491fe22550 2024-02-17T12:00:10.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.204+0000 7f9e4e666740 1 -- 172.21.15.94:0/604414506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 msgr2=0x56491fe2b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.204+0000 7f9e4e666740 1 --2- 172.21.15.94:0/604414506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 0x56491fe2b0f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000c6a0 tx=0x7f9e40010f50 comp rx=0 tx=0).stop 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.204+0000 7f9e4e666740 1 -- 172.21.15.94:0/604414506 shutdown_connections 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.204+0000 7f9e4e666740 1 --2- 172.21.15.94:0/604414506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe2ba80 0x56491fe1bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.204+0000 7f9e4e666740 1 --2- 172.21.15.94:0/604414506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe22f00 0x56491fe29260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.204+0000 7f9e4e666740 1 --2- 172.21.15.94:0/604414506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22550 0x56491fe2b0f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.204+0000 7f9e4e666740 1 -- 172.21.15.94:0/604414506 >> 172.21.15.94:0/604414506 conn(0x56491fe1ca10 msgr2=0x56491fe1b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.204+0000 7f9e4e666740 1 -- 172.21.15.94:0/604414506 shutdown_connections 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.204+0000 7f9e4e666740 1 -- 172.21.15.94:0/604414506 wait complete. 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.204+0000 7f9e4e666740 1 -- start start 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.204+0000 7f9e4e666740 1 Processor -- start 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4e666740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22550 0x56491fe8c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4e666740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22f00 0x56491fe8cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4e666740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe2ba80 0x56491fe32000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4e666740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491fe326d0 con 0x56491fe22f00 2024-02-17T12:00:10.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4e666740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491fe32840 con 0x56491fe2ba80 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4e666740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56491fe32a00 con 0x56491fe22550 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4e666740 10 client.0 _ll_register_callbacks cb 0x7f9e400d1c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e49023640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22f00 0x56491fe8cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e49824640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22550 0x56491fe8c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4a025640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe2ba80 0x56491fe32000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e49023640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22f00 0x56491fe8cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33356/0 (socket says 172.21.15.94:33356) 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e49023640 1 -- 172.21.15.94:0/25447494 learned_addr learned my addr 172.21.15.94:0/25447494 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e49824640 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22550 0x56491fe8c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:32904/0 (socket says 172.21.15.94:32904) 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4a025640 1 -- 172.21.15.94:0/25447494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22550 msgr2=0x56491fe8c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4a025640 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22550 0x56491fe8c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4a025640 1 -- 172.21.15.94:0/25447494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22f00 msgr2=0x56491fe8cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4a025640 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22f00 0x56491fe8cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4a025640 1 -- 172.21.15.94:0/25447494 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56491fd88320 con 0x56491fe2ba80 2024-02-17T12:00:10.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e49824640 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22550 0x56491fe8c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e49023640 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22f00 0x56491fe8cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e4a025640 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe2ba80 0x56491fe32000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9e44066e50 tx=0x7f9e44076040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b09d952130dfc0e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.205+0000 7f9e2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:00:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.206+0000 7f9e2d7fa640 1 -- 172.21.15.94:0/25447494 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e44062220 con 0x56491fe2ba80 2024-02-17T12:00:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.206+0000 7f9e4e666740 1 -- 172.21.15.94:0/25447494 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56491fdaf900 con 0x56491fe2ba80 2024-02-17T12:00:10.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.206+0000 7f9e2d7fa640 1 -- 172.21.15.94:0/25447494 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9e44062380 con 0x56491fe2ba80 2024-02-17T12:00:10.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:09 smithi094 ceph-mon[32499]: pgmap v64: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 38 MiB/s wr, 1.74k op/s 2024-02-17T12:00:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.206+0000 7f9e2d7fa640 1 -- 172.21.15.94:0/25447494 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e440615c0 con 0x56491fe2ba80 2024-02-17T12:00:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.206+0000 7f9e4e666740 10 client.25535 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.206+0000 7f9e4e666740 1 -- 172.21.15.94:0/25447494 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x56491fd4a910 con 0x56491fe2ba80 2024-02-17T12:00:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.206+0000 7f9e2d7fa640 1 -- 172.21.15.94:0/25447494 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56491fdaf900 con 0x56491fe2ba80 2024-02-17T12:00:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.207+0000 7f9e2d7fa640 1 -- 172.21.15.94:0/25447494 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9e44081040 con 0x56491fe2ba80 2024-02-17T12:00:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.207+0000 7f9e4e666740 10 client.25535 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.207+0000 7f9e4e666740 10 client.25535 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.207+0000 7f9e4e666740 20 client.25535 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.207+0000 7f9e4e666740 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56491fe946e0 0x56491fe96b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.207+0000 7f9e4e666740 4 client.25535 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:00:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.207+0000 7f9e4e666740 1 -- 172.21.15.94:0/25447494 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56491fa89860 con 0x56491fe946e0 2024-02-17T12:00:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.207+0000 7f9e49824640 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56491fe946e0 0x56491fe96b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.208+0000 7f9e49824640 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56491fe946e0 0x56491fe96b00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f72508fde4526311 server_cookie=717671fbe892a379 in_seq=0 out_seq=0 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.208+0000 7f9e2d7fa640 10 client.25535 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.210+0000 7f9e2d7fa640 1 -- 172.21.15.94:0/25447494 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56491fa89860 con 0x56491fe946e0 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.210+0000 7f9e2d7fa640 10 client.25535 handle_command_reply: tid=0 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.210+0000 7f9e4e666740 1 client.25535 shutdown 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.210+0000 7f9e4e666740 1 -- 172.21.15.94:0/25447494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe2ba80 msgr2=0x56491fe32000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.210+0000 7f9e4e666740 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe2ba80 0x56491fe32000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9e44066e50 tx=0x7f9e44076040 comp rx=0 tx=0).stop 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.210+0000 7f9e4e666740 1 -- 172.21.15.94:0/25447494 shutdown_connections 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.210+0000 7f9e4e666740 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56491fe946e0 0x56491fe96b00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.210+0000 7f9e4e666740 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56491fe2ba80 0x56491fe32000 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.210+0000 7f9e2d7fa640 0 client.25535 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.210+0000 7f9e4e666740 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56491fe22f00 0x56491fe8cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.210+0000 7f9e4e666740 1 --2- 172.21.15.94:0/25447494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56491fe22550 0x56491fe8c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.210+0000 7f9e4e666740 1 -- 172.21.15.94:0/25447494 >> 172.21.15.94:0/25447494 conn(0x56491fe1ca10 msgr2=0x56491fe1bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:10.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.211+0000 7f9e4e666740 1 -- 172.21.15.94:0/25447494 shutdown_connections 2024-02-17T12:00:10.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.211+0000 7f9e4e666740 1 -- 172.21.15.94:0/25447494 wait complete. 2024-02-17T12:00:10.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.211+0000 7f9e4e666740 20 client.25535 trim_cache size 0 max 16384 2024-02-17T12:00:10.213 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:00:10.213 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:00:10.213 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "ad1e184e-3a86-4702-a773-f0ff3113b420", 2024-02-17T12:00:10.213 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:00:10.213 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:00:10.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.211+0000 7f9e4a826640 1 -- 172.21.15.94:0/1497653438 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9e300a2260 msgr2=0x7f9e300a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.211+0000 7f9e4a826640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9e300a2260 0x7f9e300a4720 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9e440668b0 tx=0x7f9e44078040 comp rx=0 tx=0).stop 2024-02-17T12:00:10.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.211+0000 7f9e4a826640 1 -- 172.21.15.94:0/1497653438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e3c0961a0 msgr2=0x7f9e3c0076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.211+0000 7f9e4a826640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e3c0961a0 0x7f9e3c0076f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9e4000e9f0 tx=0x7f9e4000eeb0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.211+0000 7f9e4a826640 1 -- 172.21.15.94:0/1497653438 shutdown_connections 2024-02-17T12:00:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.211+0000 7f9e4a826640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9e300a2260 0x7f9e300a4720 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.211+0000 7f9e4a826640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e3c008170 0x7f9e3c13a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.211+0000 7f9e4a826640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e3c09b040 0x7f9e3c007c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.211+0000 7f9e4a826640 1 --2- 172.21.15.94:0/1497653438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e3c0961a0 0x7f9e3c0076f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.211+0000 7f9e4a826640 1 -- 172.21.15.94:0/1497653438 >> 172.21.15.94:0/1497653438 conn(0x7f9e3c00b840 msgr2=0x7f9e3c094180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.212+0000 7f9e4a826640 1 -- 172.21.15.94:0/1497653438 shutdown_connections 2024-02-17T12:00:10.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.212+0000 7f9e4a826640 1 -- 172.21.15.94:0/1497653438 wait complete. 2024-02-17T12:00:10.224 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:00:10.224 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:00:10.241 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 110.0 s, 3437.0 tps, lat 29.291 ms stddev 32.624 2024-02-17T12:00:10.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.381+0000 7fedb478c640 1 -- 172.21.15.94:0/4100919197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedac072ea0 msgr2=0x7fedac06ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.381+0000 7fedb478c640 1 --2- 172.21.15.94:0/4100919197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedac072ea0 0x7fedac06ac80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feda8009900 tx=0x7feda802bce0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.381+0000 7fedb478c640 1 -- 172.21.15.94:0/4100919197 shutdown_connections 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.381+0000 7fedb478c640 1 --2- 172.21.15.94:0/4100919197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedac112f40 0x7fedac115330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.382+0000 7fedb478c640 1 --2- 172.21.15.94:0/4100919197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedac06b1c0 0x7fedac06b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.382+0000 7fedb478c640 1 --2- 172.21.15.94:0/4100919197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedac072ea0 0x7fedac06ac80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.382+0000 7fedb478c640 1 -- 172.21.15.94:0/4100919197 >> 172.21.15.94:0/4100919197 conn(0x7fedac077230 msgr2=0x7fedac077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.382+0000 7fedb478c640 1 -- 172.21.15.94:0/4100919197 shutdown_connections 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.382+0000 7fedb478c640 1 -- 172.21.15.94:0/4100919197 wait complete. 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.383+0000 7fedb478c640 1 Processor -- start 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.383+0000 7fedb478c640 1 -- start start 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.383+0000 7fedb478c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedac06b1c0 0x7fedac11beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.383+0000 7fedb478c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedac072ea0 0x7fedac118f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.383+0000 7fedb478c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedac112f40 0x7fedac1194b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.383+0000 7fedb478c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac119c40 con 0x7fedac112f40 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.383+0000 7fedb478c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac119db0 con 0x7fedac06b1c0 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.383+0000 7fedb478c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac1b2ae0 con 0x7fedac072ea0 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.383+0000 7fedb1d27640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedac072ea0 0x7fedac118f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.383+0000 7fedb1d27640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedac072ea0 0x7fedac118f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:32912/0 (socket says 172.21.15.94:32912) 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.383+0000 7fedb1d27640 1 -- 172.21.15.94:0/1789548378 learned_addr learned my addr 172.21.15.94:0/1789548378 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.383+0000 7fedb2528640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedac06b1c0 0x7fedac11beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.384+0000 7fedb2d29640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedac112f40 0x7fedac1194b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.384+0000 7fedb2d29640 1 -- 172.21.15.94:0/1789548378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedac072ea0 msgr2=0x7fedac118f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.384+0000 7fedb2d29640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedac072ea0 0x7fedac118f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.384+0000 7fedb2d29640 1 -- 172.21.15.94:0/1789548378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedac06b1c0 msgr2=0x7fedac11beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.384+0000 7fedb2d29640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedac06b1c0 0x7fedac11beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.384+0000 7fedb2d29640 1 -- 172.21.15.94:0/1789548378 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda8009560 con 0x7fedac112f40 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.385+0000 7fedb2d29640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedac112f40 0x7fedac1194b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feda4009fd0 tx=0x7feda400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:10.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.385+0000 7feda37fe640 1 -- 172.21.15.94:0/1789548378 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4004d20 con 0x7fedac112f40 2024-02-17T12:00:10.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.385+0000 7feda37fe640 1 -- 172.21.15.94:0/1789548378 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feda4004e80 con 0x7fedac112f40 2024-02-17T12:00:10.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.385+0000 7fedb478c640 1 -- 172.21.15.94:0/1789548378 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedac1b2dc0 con 0x7fedac112f40 2024-02-17T12:00:10.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.385+0000 7feda37fe640 1 -- 172.21.15.94:0/1789548378 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda40116d0 con 0x7fedac112f40 2024-02-17T12:00:10.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.385+0000 7fedb478c640 1 -- 172.21.15.94:0/1789548378 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac1b32d0 con 0x7fedac112f40 2024-02-17T12:00:10.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.388+0000 7fedb7367740 1 Processor -- start 2024-02-17T12:00:10.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.388+0000 7feda37fe640 1 -- 172.21.15.94:0/1789548378 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7feda400cb20 con 0x7fedac112f40 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb7367740 1 -- start start 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb7367740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fa2b0 0x55af4f0fa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb7367740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fabd0 0x55af4f103270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb7367740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f105ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb7367740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f069280 con 0x55af4f0fabd0 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb7367740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f0667b0 con 0x55af4f1037b0 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb7367740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f061f90 con 0x55af4f0fa2b0 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb1d27640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f105ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb2528640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fabd0 0x55af4f103270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb1d27640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f105ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59400/0 (socket says 172.21.15.94:59400) 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb1d27640 1 -- 172.21.15.94:0/1256715924 learned_addr learned my addr 172.21.15.94:0/1256715924 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7feda37fe640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fed980a23c0 0x7fed980a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb1d27640 1 --2- 172.21.15.94:0/1256715924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fa2b0 0x55af4f0fa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb1d27640 1 -- 172.21.15.94:0/1256715924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fa2b0 msgr2=0x55af4f0fa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7feda37fe640 1 -- 172.21.15.94:0/1789548378 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7feda40ce080 con 0x7fedac112f40 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb1d27640 1 --2- 172.21.15.94:0/1256715924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fa2b0 0x55af4f0fa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb1d27640 1 -- 172.21.15.94:0/1256715924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fabd0 msgr2=0x55af4f103270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.389+0000 7fedb1d27640 1 --2- 172.21.15.94:0/1256715924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fabd0 0x55af4f103270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.390+0000 7fedb1d27640 1 -- 172.21.15.94:0/1256715924 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af4f020570 con 0x55af4f1037b0 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.390+0000 7fedb2d29640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fed980a23c0 0x7fed980a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.390+0000 7fedb1d27640 1 --2- 172.21.15.94:0/1256715924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f105ba0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fed9c006a50 tx=0x7fed9c009590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f765962e7f44403 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.390+0000 7feda0ff9640 1 -- 172.21.15.94:0/1256715924 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0157c0 con 0x55af4f1037b0 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.390+0000 7feda0ff9640 1 -- 172.21.15.94:0/1256715924 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed9c015e00 con 0x55af4f1037b0 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.390+0000 7feda0ff9640 1 -- 172.21.15.94:0/1256715924 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c018df0 con 0x55af4f1037b0 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.391+0000 7fedb7367740 1 -- 172.21.15.94:0/1256715924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 msgr2=0x55af4f105ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.391+0000 7fedb7367740 1 --2- 172.21.15.94:0/1256715924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f105ba0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fed9c006a50 tx=0x7fed9c009590 comp rx=0 tx=0).stop 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.391+0000 7fedb2d29640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fed980a23c0 0x7fed980a4880 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feda4096f70 tx=0x7feda4005f50 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:10.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.391+0000 7fedb7367740 1 -- 172.21.15.94:0/1256715924 shutdown_connections 2024-02-17T12:00:10.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.391+0000 7fedb7367740 1 --2- 172.21.15.94:0/1256715924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f105ba0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.391+0000 7fedb7367740 1 --2- 172.21.15.94:0/1256715924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fabd0 0x55af4f103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.391+0000 7fedb7367740 1 --2- 172.21.15.94:0/1256715924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fa2b0 0x55af4f0fa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.391+0000 7fedb7367740 1 -- 172.21.15.94:0/1256715924 >> 172.21.15.94:0/1256715924 conn(0x55af4f0f4990 msgr2=0x55af4f0f4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:10.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.392+0000 7fedb7367740 1 -- 172.21.15.94:0/1256715924 shutdown_connections 2024-02-17T12:00:10.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.392+0000 7fedb7367740 1 -- 172.21.15.94:0/1256715924 wait complete. 2024-02-17T12:00:10.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.392+0000 7fedb7367740 1 -- start start 2024-02-17T12:00:10.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.392+0000 7fedb7367740 1 Processor -- start 2024-02-17T12:00:10.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb7367740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 0x55af4f1031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb7367740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f100570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb7367740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f100cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb1d27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 0x55af4f1031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb1d27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 0x55af4f1031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33378/0 (socket says 172.21.15.94:33378) 2024-02-17T12:00:10.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb1d27640 1 -- 172.21.15.94:0/226764985 learned_addr learned my addr 172.21.15.94:0/226764985 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:10.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb7367740 1 -- 172.21.15.94:0/226764985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4ed35430 con 0x55af4f0fa2b0 2024-02-17T12:00:10.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb7367740 1 -- 172.21.15.94:0/226764985 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4ecf1e00 con 0x55af4f1037b0 2024-02-17T12:00:10.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb7367740 1 -- 172.21.15.94:0/226764985 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4efe6fd0 con 0x55af4f0fabd0 2024-02-17T12:00:10.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb1d27640 1 --2- 172.21.15.94:0/226764985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f100cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb2528640 1 --2- 172.21.15.94:0/226764985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb7367740 10 client.0 _ll_register_callbacks cb 0x55af4f03cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb1d27640 1 -- 172.21.15.94:0/226764985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 msgr2=0x55af4f100570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb1d27640 1 --2- 172.21.15.94:0/226764985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f100570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb1d27640 1 -- 172.21.15.94:0/226764985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 msgr2=0x55af4f100cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb1d27640 1 --2- 172.21.15.94:0/226764985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f100cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb1d27640 1 -- 172.21.15.94:0/226764985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9c017460 con 0x55af4f0fa2b0 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb7367740 1 -- 172.21.15.94:0/226764985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af4ed617e0 con 0x55af4f0fa2b0 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fedb1d27640 1 --2- 172.21.15.94:0/226764985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 0x55af4f1031e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00a010 tx=0x7fed9c0189d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36ad75289143a93 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fed95ffb640 10 client.16101 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fed95ffb640 1 -- 172.21.15.94:0/226764985 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c015790 con 0x55af4f0fa2b0 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.393+0000 7fed95ffb640 1 -- 172.21.15.94:0/226764985 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fed9c0158f0 con 0x55af4f0fa2b0 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.394+0000 7fed95ffb640 1 -- 172.21.15.94:0/226764985 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed9c015ab0 con 0x55af4f0fa2b0 2024-02-17T12:00:10.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.394+0000 7fedb7367740 10 client.16101 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.394+0000 7fedb7367740 1 -- 172.21.15.94:0/226764985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({fsmap=21}) v3 -- 0x55af4f060610 con 0x55af4f0fa2b0 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.394+0000 7fed95ffb640 1 -- 172.21.15.94:0/226764985 <== mon.0 v2:172.21.15.94:3300/0 4 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fed9c023410 con 0x55af4f0fa2b0 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.394+0000 7fedb7367740 10 client.16101 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.394+0000 7fedb7367740 10 client.16101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.394+0000 7fedb7367740 20 client.16101 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.394+0000 7fedb7367740 1 --2- 172.21.15.94:0/226764985 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55af4f113550 0x55af4f115970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.394+0000 7fedb7367740 4 client.16101 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.394+0000 7fedb7367740 1 -- 172.21.15.94:0/226764985 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af4ed9e720 con 0x55af4f113550 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.394+0000 7fedb2528640 1 --2- 172.21.15.94:0/226764985 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55af4f113550 0x55af4f115970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.394+0000 7fed95ffb640 10 client.16101 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.394+0000 7fedb2528640 1 --2- 172.21.15.94:0/226764985 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55af4f113550 0x55af4f115970 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f65eb6d4aef3a76c server_cookie=c4a6c4c22ee78048 in_seq=0 out_seq=0 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.395+0000 7fed95ffb640 1 -- 172.21.15.94:0/226764985 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55af4ed9e720 con 0x55af4f113550 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.395+0000 7fed95ffb640 10 client.16101 handle_command_reply: tid=0 2024-02-17T12:00:10.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.396+0000 7fedb7367740 1 client.16101 shutdown 2024-02-17T12:00:10.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.396+0000 7fedb7367740 1 -- 172.21.15.94:0/226764985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 msgr2=0x55af4f1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.396+0000 7fedb7367740 1 --2- 172.21.15.94:0/226764985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 0x55af4f1031e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00a010 tx=0x7fed9c0189d0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.396+0000 7fedb7367740 1 -- 172.21.15.94:0/226764985 shutdown_connections 2024-02-17T12:00:10.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.396+0000 7fedb7367740 1 --2- 172.21.15.94:0/226764985 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55af4f113550 0x55af4f115970 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.396+0000 7fedb7367740 1 --2- 172.21.15.94:0/226764985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.396+0000 7fed95ffb640 0 client.16101 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:10.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.396+0000 7fedb7367740 1 --2- 172.21.15.94:0/226764985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.396+0000 7fedb7367740 1 --2- 172.21.15.94:0/226764985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 0x55af4f1031e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.396+0000 7fedb7367740 1 -- 172.21.15.94:0/226764985 >> 172.21.15.94:0/226764985 conn(0x55af4f0f4990 msgr2=0x55af4f0f6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:10.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.396+0000 7fedb7367740 1 -- 172.21.15.94:0/226764985 shutdown_connections 2024-02-17T12:00:10.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.396+0000 7fedb7367740 1 -- 172.21.15.94:0/226764985 wait complete. 2024-02-17T12:00:10.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.396+0000 7fedb7367740 20 client.16101 trim_cache size 0 max 16384 2024-02-17T12:00:10.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb7367740 1 Processor -- start 2024-02-17T12:00:10.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb7367740 1 -- start start 2024-02-17T12:00:10.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb7367740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f0fa2b0 0x55af4f0ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb7367740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb7367740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f1037b0 0x55af4f1031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb7367740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f0fc1a0 con 0x55af4f1037b0 2024-02-17T12:00:10.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb7367740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f113350 con 0x55af4f0fa2b0 2024-02-17T12:00:10.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb7367740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f0f59e0 con 0x55af4f0fabd0 2024-02-17T12:00:10.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb2528640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb1d27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f1037b0 0x55af4f1031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb2528640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:32934/0 (socket says 172.21.15.94:32934) 2024-02-17T12:00:10.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb2528640 1 -- 172.21.15.94:0/2093801650 learned_addr learned my addr 172.21.15.94:0/2093801650 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:10.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb1d27640 1 --2- 172.21.15.94:0/2093801650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f0fa2b0 0x55af4f0ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb1d27640 1 -- 172.21.15.94:0/2093801650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 msgr2=0x55af4f102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb1d27640 1 --2- 172.21.15.94:0/2093801650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f102ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb1d27640 1 -- 172.21.15.94:0/2093801650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f0fa2b0 msgr2=0x55af4f0ffe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb1d27640 1 --2- 172.21.15.94:0/2093801650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f0fa2b0 0x55af4f0ffe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb1d27640 1 -- 172.21.15.94:0/2093801650 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af4effcac0 con 0x55af4f1037b0 2024-02-17T12:00:10.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.416+0000 7fedb2528640 1 --2- 172.21.15.94:0/2093801650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:10.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.417+0000 7fedb1d27640 1 --2- 172.21.15.94:0/2093801650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f1037b0 0x55af4f1031e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fed9c016010 tx=0x7fed9c027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27b64e05c4d20ba server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:10.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.417+0000 7fed97fff640 1 -- 172.21.15.94:0/2093801650 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c009420 con 0x55af4f1037b0 2024-02-17T12:00:10.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.417+0000 7fed97fff640 1 -- 172.21.15.94:0/2093801650 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed9c03f070 con 0x55af4f1037b0 2024-02-17T12:00:10.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.417+0000 7fed97fff640 1 -- 172.21.15.94:0/2093801650 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c023a50 con 0x55af4f1037b0 2024-02-17T12:00:10.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.418+0000 7fedb7367740 1 -- 172.21.15.94:0/2093801650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f1037b0 msgr2=0x55af4f1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.418+0000 7fedb7367740 1 --2- 172.21.15.94:0/2093801650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f1037b0 0x55af4f1031e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fed9c016010 tx=0x7fed9c027000 comp rx=0 tx=0).stop 2024-02-17T12:00:10.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.418+0000 7fedb7367740 1 -- 172.21.15.94:0/2093801650 shutdown_connections 2024-02-17T12:00:10.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.418+0000 7fedb7367740 1 --2- 172.21.15.94:0/2093801650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f1037b0 0x55af4f1031e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.418+0000 7fedb7367740 1 --2- 172.21.15.94:0/2093801650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.418+0000 7fedb7367740 1 --2- 172.21.15.94:0/2093801650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f0fa2b0 0x55af4f0ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.418+0000 7fedb7367740 1 -- 172.21.15.94:0/2093801650 >> 172.21.15.94:0/2093801650 conn(0x55af4f0f4990 msgr2=0x55af4f11a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:10.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.418+0000 7fedb7367740 1 -- 172.21.15.94:0/2093801650 shutdown_connections 2024-02-17T12:00:10.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.418+0000 7fedb7367740 1 -- 172.21.15.94:0/2093801650 wait complete. 2024-02-17T12:00:10.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.418+0000 7fedb7367740 1 -- start start 2024-02-17T12:00:10.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.418+0000 7fedb7367740 1 Processor -- start 2024-02-17T12:00:10.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb7367740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 0x55af4f108c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb7367740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f109190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb1d27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 0x55af4f108c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb1d27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 0x55af4f108c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33402/0 (socket says 172.21.15.94:33402) 2024-02-17T12:00:10.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb7367740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f10a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb1d27640 1 -- 172.21.15.94:0/3937556349 learned_addr learned my addr 172.21.15.94:0/3937556349 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:10.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb7367740 1 -- 172.21.15.94:0/3937556349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f10a690 con 0x55af4f0fa2b0 2024-02-17T12:00:10.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb7367740 1 -- 172.21.15.94:0/3937556349 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f10a800 con 0x55af4f1037b0 2024-02-17T12:00:10.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb7367740 1 -- 172.21.15.94:0/3937556349 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f10a9c0 con 0x55af4f0fabd0 2024-02-17T12:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb7367740 10 client.0 _ll_register_callbacks cb 0x7fed9c0098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb2528640 1 --2- 172.21.15.94:0/3937556349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb1d27640 1 --2- 172.21.15.94:0/3937556349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f10a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb1d27640 1 -- 172.21.15.94:0/3937556349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 msgr2=0x55af4f109190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb1d27640 1 --2- 172.21.15.94:0/3937556349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f109190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb1d27640 1 -- 172.21.15.94:0/3937556349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 msgr2=0x55af4f10a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb1d27640 1 --2- 172.21.15.94:0/3937556349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f10a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb1d27640 1 -- 172.21.15.94:0/3937556349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac0682e0 con 0x55af4f0fa2b0 2024-02-17T12:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb1d27640 1 --2- 172.21.15.94:0/3937556349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 0x55af4f108c50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fed9c032040 tx=0x7fed9c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18453d2c7251cee9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fed95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fed95ffb640 1 -- 172.21.15.94:0/3937556349 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c009210 con 0x55af4f0fa2b0 2024-02-17T12:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fed95ffb640 1 -- 172.21.15.94:0/3937556349 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fed9c009630 con 0x55af4f0fa2b0 2024-02-17T12:00:10.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.419+0000 7fedb7367740 1 -- 172.21.15.94:0/3937556349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed9c015e00 con 0x55af4f0fa2b0 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.420+0000 7fed95ffb640 1 -- 172.21.15.94:0/3937556349 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed9c015790 con 0x55af4f0fa2b0 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.420+0000 7fedb7367740 10 client.16107 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.420+0000 7fedb7367740 1 -- 172.21.15.94:0/3937556349 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55af4f060610 con 0x55af4f0fa2b0 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.420+0000 7fed95ffb640 1 -- 172.21.15.94:0/3937556349 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fed9c015e00 con 0x55af4f0fa2b0 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.420+0000 7fed95ffb640 1 -- 172.21.15.94:0/3937556349 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fed9c003690 con 0x55af4f0fa2b0 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.420+0000 7fedb7367740 10 client.16107 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.420+0000 7fedb7367740 10 client.16107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.420+0000 7fedb7367740 20 client.16107 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.420+0000 7fedb7367740 1 --2- 172.21.15.94:0/3937556349 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55af4f113550 0x55af4f16c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.420+0000 7fedb7367740 4 client.16107 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.420+0000 7fedb7367740 1 -- 172.21.15.94:0/3937556349 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55af4eded890 con 0x55af4f113550 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.421+0000 7fedb2528640 1 --2- 172.21.15.94:0/3937556349 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55af4f113550 0x55af4f16c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.421+0000 7fed95ffb640 10 client.16107 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:10.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fedb2528640 1 --2- 172.21.15.94:0/3937556349 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55af4f113550 0x55af4f16c960 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86c5d53ab14ee832 server_cookie=dc6ac439594997b4 in_seq=0 out_seq=0 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fed95ffb640 1 -- 172.21.15.94:0/3937556349 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55af4eded890 con 0x55af4f113550 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fed95ffb640 10 client.16107 handle_command_reply: tid=0 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fedb7367740 1 client.16107 shutdown 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fedb7367740 1 -- 172.21.15.94:0/3937556349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 msgr2=0x55af4f108c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fedb7367740 1 --2- 172.21.15.94:0/3937556349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 0x55af4f108c50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fed9c032040 tx=0x7fed9c009b20 comp rx=0 tx=0).stop 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fedb7367740 1 -- 172.21.15.94:0/3937556349 shutdown_connections 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fedb7367740 1 --2- 172.21.15.94:0/3937556349 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55af4f113550 0x55af4f16c960 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fedb7367740 1 --2- 172.21.15.94:0/3937556349 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55af4f1037b0 0x55af4f10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fed95ffb640 0 client.16107 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fedb7367740 1 --2- 172.21.15.94:0/3937556349 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55af4f0fabd0 0x55af4f109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fedb7367740 1 --2- 172.21.15.94:0/3937556349 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55af4f0fa2b0 0x55af4f108c50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fedb7367740 1 -- 172.21.15.94:0/3937556349 >> 172.21.15.94:0/3937556349 conn(0x55af4f0f4990 msgr2=0x55af4f0f31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fedb7367740 1 -- 172.21.15.94:0/3937556349 shutdown_connections 2024-02-17T12:00:10.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fedb7367740 1 -- 172.21.15.94:0/3937556349 wait complete. 2024-02-17T12:00:10.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.422+0000 7fedb7367740 20 client.16107 trim_cache size 0 max 16384 2024-02-17T12:00:10.427 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:00:10.427 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:00:10.427 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:00:10.427 INFO:teuthology.orchestra.run.smithi094.stdout: "ad1e184e-3a86-4702-a773-f0ff3113b420": { 2024-02-17T12:00:10.427 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:00:10.427 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "ad1e184e-3a86-4702-a773-f0ff3113b420", 2024-02-17T12:00:10.427 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:00:10.428 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:00:10.428 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:00:10.428 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:00:10.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.424+0000 7fedb478c640 1 -- 172.21.15.94:0/1789548378 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fed980a23c0 msgr2=0x7fed980a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.424+0000 7fedb478c640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fed980a23c0 0x7fed980a4880 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7feda4096f70 tx=0x7feda4005f50 comp rx=0 tx=0).stop 2024-02-17T12:00:10.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.424+0000 7fedb478c640 1 -- 172.21.15.94:0/1789548378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedac112f40 msgr2=0x7fedac1194b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:10.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.424+0000 7fedb478c640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedac112f40 0x7fedac1194b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feda4009fd0 tx=0x7feda400edf0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.424+0000 7fedb478c640 1 -- 172.21.15.94:0/1789548378 shutdown_connections 2024-02-17T12:00:10.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.424+0000 7fedb478c640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fed980a23c0 0x7fed980a4880 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.424+0000 7fedb478c640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fedac112f40 0x7fedac1194b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.424+0000 7fedb478c640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedac072ea0 0x7fedac118f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.424+0000 7fedb478c640 1 --2- 172.21.15.94:0/1789548378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fedac06b1c0 0x7fedac11beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:10.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.424+0000 7fedb478c640 1 -- 172.21.15.94:0/1789548378 >> 172.21.15.94:0/1789548378 conn(0x7fedac077230 msgr2=0x7fedac070b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:10.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.424+0000 7fedb478c640 1 -- 172.21.15.94:0/1789548378 shutdown_connections 2024-02-17T12:00:10.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:10.424+0000 7fedb478c640 1 -- 172.21.15.94:0/1789548378 wait complete. 2024-02-17T12:00:10.437 INFO:tasks.cephfs.filesystem:scrub status for tag:ad1e184e-3a86-4702-a773-f0ff3113b420 - {'path': '/', 'tag': 'ad1e184e-3a86-4702-a773-f0ff3113b420', 'options': 'recursive,force'} 2024-02-17T12:00:11.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:10 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:00:11.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:10 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:00:11.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:10 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:00:11.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:10 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:00:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:10 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:00:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:10 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:00:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:10 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:00:11.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:10 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:00:11.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:10 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:00:11.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:10 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:00:11.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:10 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:00:11.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:10 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:00:12.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:11 smithi189 ceph-mon[36588]: pgmap v65: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.42k op/s 2024-02-17T12:00:12.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:11 smithi125 ceph-mon[36819]: pgmap v65: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.42k op/s 2024-02-17T12:00:12.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:11 smithi094 ceph-mon[32499]: pgmap v65: 97 pgs: 97 active+clean; 1.6 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 1.42k op/s 2024-02-17T12:00:13.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:12 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:00:13.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:12 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:00:13.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:12 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:00:14.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:13 smithi189 ceph-mon[36588]: pgmap v66: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 38 MiB/s wr, 1.74k op/s 2024-02-17T12:00:14.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:13 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:00:14.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:13 smithi125 ceph-mon[36819]: pgmap v66: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 38 MiB/s wr, 1.74k op/s 2024-02-17T12:00:14.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:13 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:00:14.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:13 smithi094 ceph-mon[32499]: pgmap v66: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 38 MiB/s wr, 1.74k op/s 2024-02-17T12:00:14.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:13 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:00:15.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 115.0 s, 3653.0 tps, lat 27.327 ms stddev 28.726 2024-02-17T12:00:15.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:15 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:15 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:15 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.650 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:15 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:15 smithi189 ceph-mon[36588]: pgmap v67: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 37 MiB/s wr, 1.63k op/s 2024-02-17T12:00:15.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:15 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:00:15.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:15 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:00:15.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:15 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:15 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:15 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:15 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:15 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:15 smithi125 ceph-mon[36819]: pgmap v67: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 37 MiB/s wr, 1.63k op/s 2024-02-17T12:00:15.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:15 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:00:15.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:15 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:00:15.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:15 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:15 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:15 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:15 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:15 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:15.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:15 smithi094 ceph-mon[32499]: pgmap v67: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 37 MiB/s wr, 1.63k op/s 2024-02-17T12:00:15.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:15 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:00:15.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:15 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:00:15.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:15 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:00:18.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:17 smithi189 ceph-mon[36588]: pgmap v68: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 36 MiB/s wr, 1.52k op/s 2024-02-17T12:00:18.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:17 smithi125 ceph-mon[36819]: pgmap v68: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 36 MiB/s wr, 1.52k op/s 2024-02-17T12:00:18.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:17 smithi094 ceph-mon[32499]: pgmap v68: 97 pgs: 97 active+clean; 1.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 36 MiB/s wr, 1.52k op/s 2024-02-17T12:00:20.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:19 smithi189 ceph-mon[36588]: pgmap v69: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 41 MiB/s wr, 1.88k op/s 2024-02-17T12:00:20.183 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:19 smithi125 ceph-mon[36819]: pgmap v69: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 41 MiB/s wr, 1.88k op/s 2024-02-17T12:00:20.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:19 smithi094 ceph-mon[32499]: pgmap v69: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 41 MiB/s wr, 1.88k op/s 2024-02-17T12:00:20.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 120.0 s, 3529.0 tps, lat 28.417 ms stddev 33.072 2024-02-17T12:00:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.480+0000 7fa5f5e57640 1 -- 172.21.15.94:0/3339751681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5f0100c60 msgr2=0x7fa5f0104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.480+0000 7fa5f5e57640 1 --2- 172.21.15.94:0/3339751681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5f0100c60 0x7fa5f0104ca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009860 tx=0x7fa5e402bb90 comp rx=0 tx=0).stop 2024-02-17T12:00:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.480+0000 7fa5f5e57640 1 -- 172.21.15.94:0/3339751681 shutdown_connections 2024-02-17T12:00:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.480+0000 7fa5f5e57640 1 --2- 172.21.15.94:0/3339751681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5f0105380 0x7fa5f010cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.480+0000 7fa5f5e57640 1 --2- 172.21.15.94:0/3339751681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5f0100c60 0x7fa5f0104ca0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.480+0000 7fa5f5e57640 1 --2- 172.21.15.94:0/3339751681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f01002b0 0x7fa5f0100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.480+0000 7fa5f5e57640 1 -- 172.21.15.94:0/3339751681 >> 172.21.15.94:0/3339751681 conn(0x7fa5f00771b0 msgr2=0x7fa5f00775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:21.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.480+0000 7fa5f5e57640 1 -- 172.21.15.94:0/3339751681 shutdown_connections 2024-02-17T12:00:21.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.480+0000 7fa5f5e57640 1 -- 172.21.15.94:0/3339751681 wait complete. 2024-02-17T12:00:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.481+0000 7fa5f5e57640 1 Processor -- start 2024-02-17T12:00:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.481+0000 7fa5f5e57640 1 -- start start 2024-02-17T12:00:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.481+0000 7fa5f5e57640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5f01002b0 0x7fa5f01974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.481+0000 7fa5f5e57640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f0105380 0x7fa5f0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.482+0000 7fa5f5e57640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5f019bda0 0x7fa5f019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.482+0000 7fa5f5e57640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f019c790 con 0x7fa5f01002b0 2024-02-17T12:00:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.482+0000 7fa5f5e57640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f019c900 con 0x7fa5f0105380 2024-02-17T12:00:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.482+0000 7fa5f5e57640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f019ca70 con 0x7fa5f019bda0 2024-02-17T12:00:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.482+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5f01002b0 0x7fa5f01974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.482+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5f01002b0 0x7fa5f01974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58628/0 (socket says 172.21.15.94:58628) 2024-02-17T12:00:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.482+0000 7fa5ef7fe640 1 -- 172.21.15.94:0/580415272 learned_addr learned my addr 172.21.15.94:0/580415272 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.482+0000 7fa5eeffd640 1 --2- 172.21.15.94:0/580415272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f0105380 0x7fa5f0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.483+0000 7fa5eeffd640 1 -- 172.21.15.94:0/580415272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5f019bda0 msgr2=0x7fa5f019c250 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-17T12:00:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.483+0000 7fa5eeffd640 1 --2- 172.21.15.94:0/580415272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5f019bda0 0x7fa5f019c250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.483+0000 7fa5eeffd640 1 -- 172.21.15.94:0/580415272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5f01002b0 msgr2=0x7fa5f01974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.483+0000 7fa5eeffd640 1 --2- 172.21.15.94:0/580415272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5f01002b0 0x7fa5f01974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.483+0000 7fa5eeffd640 1 -- 172.21.15.94:0/580415272 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e4009490 con 0x7fa5f0105380 2024-02-17T12:00:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.483+0000 7fa5eeffd640 1 --2- 172.21.15.94:0/580415272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f0105380 0x7fa5f0197a20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009780 tx=0x7fa5e40346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.484+0000 7fa5ecff9640 1 -- 172.21.15.94:0/580415272 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4005250 con 0x7fa5f0105380 2024-02-17T12:00:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.484+0000 7fa5ecff9640 1 -- 172.21.15.94:0/580415272 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5e40323b0 con 0x7fa5f0105380 2024-02-17T12:00:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.484+0000 7fa5ecff9640 1 -- 172.21.15.94:0/580415272 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4004af0 con 0x7fa5f0105380 2024-02-17T12:00:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.484+0000 7fa5f5e57640 1 -- 172.21.15.94:0/580415272 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f01a4050 con 0x7fa5f0105380 2024-02-17T12:00:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.484+0000 7fa5f5e57640 1 -- 172.21.15.94:0/580415272 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f010d7b0 con 0x7fa5f0105380 2024-02-17T12:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.488+0000 7fa5f8a20740 1 Processor -- start 2024-02-17T12:00:21.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.488+0000 7fa5ecff9640 1 -- 172.21.15.94:0/580415272 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa5e403b040 con 0x7fa5f0105380 2024-02-17T12:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.489+0000 7fa5ecff9640 1 --2- 172.21.15.94:0/580415272 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5d80a24b0 0x7fa5d80a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.489+0000 7fa5ecff9640 1 -- 172.21.15.94:0/580415272 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa5e40efcd0 con 0x7fa5f0105380 2024-02-17T12:00:21.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.490+0000 7fa5f8a20740 1 -- start start 2024-02-17T12:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.490+0000 7fa5effff640 1 --2- 172.21.15.94:0/580415272 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5d80a24b0 0x7fa5d80a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.490+0000 7fa5f8a20740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 0x5619a60ba820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.490+0000 7fa5f8a20740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bad60 0x5619a60bb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.490+0000 7fa5f8a20740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.490+0000 7fa5f8a20740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a60299f0 con 0x5619a60c42e0 2024-02-17T12:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.490+0000 7fa5f8a20740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a6026f20 con 0x5619a60bad60 2024-02-17T12:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.490+0000 7fa5f8a20740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a6022700 con 0x5619a60bbad0 2024-02-17T12:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.490+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 0x5619a60ba820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.490+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 0x5619a60ba820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:59042/0 (socket says 172.21.15.94:59042) 2024-02-17T12:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.490+0000 7fa5ef7fe640 1 -- 172.21.15.94:0/3838620493 learned_addr learned my addr 172.21.15.94:0/3838620493 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.491+0000 7fa5eeffd640 1 --2- 172.21.15.94:0/3838620493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bad60 0x5619a60bb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.491+0000 7fa5effff640 1 --2- 172.21.15.94:0/3838620493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.491+0000 7fa5ef7fe640 1 -- 172.21.15.94:0/3838620493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bad60 msgr2=0x5619a60bb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.491+0000 7fa5ef7fe640 1 --2- 172.21.15.94:0/3838620493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bad60 0x5619a60bb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.491+0000 7fa5effff640 1 --2- 172.21.15.94:0/580415272 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5d80a24b0 0x7fa5d80a4970 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8005fd0 tx=0x7fa5e800e7c0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.491+0000 7fa5ef7fe640 1 -- 172.21.15.94:0/3838620493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 msgr2=0x5619a60c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.491+0000 7fa5ef7fe640 1 --2- 172.21.15.94:0/3838620493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.491+0000 7fa5ef7fe640 1 -- 172.21.15.94:0/3838620493 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619a5fb87b0 con 0x5619a60bbad0 2024-02-17T12:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.491+0000 7fa5eeffd640 1 --2- 172.21.15.94:0/3838620493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bad60 0x5619a60bb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.491+0000 7fa5ef7fe640 1 --2- 172.21.15.94:0/3838620493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 0x5619a60ba820 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009dc0 tx=0x7fa5e0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=160cf98c03731ebd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.491+0000 7fa5d5ffb640 1 -- 172.21.15.94:0/3838620493 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e0005220 con 0x5619a60bbad0 2024-02-17T12:00:21.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.491+0000 7fa5d5ffb640 1 -- 172.21.15.94:0/3838620493 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5e0004c80 con 0x5619a60bbad0 2024-02-17T12:00:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.491+0000 7fa5d5ffb640 1 -- 172.21.15.94:0/3838620493 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e00198a0 con 0x5619a60bbad0 2024-02-17T12:00:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.492+0000 7fa5f8a20740 1 -- 172.21.15.94:0/3838620493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 msgr2=0x5619a60ba820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.492+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/3838620493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 0x5619a60ba820 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009dc0 tx=0x7fa5e0011810 comp rx=0 tx=0).stop 2024-02-17T12:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.492+0000 7fa5f8a20740 1 -- 172.21.15.94:0/3838620493 shutdown_connections 2024-02-17T12:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.493+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/3838620493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.493+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/3838620493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bad60 0x5619a60bb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.493+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/3838620493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 0x5619a60ba820 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.493+0000 7fa5f8a20740 1 -- 172.21.15.94:0/3838620493 >> 172.21.15.94:0/3838620493 conn(0x5619a6082550 msgr2=0x5619a60b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.493+0000 7fa5f8a20740 1 -- 172.21.15.94:0/3838620493 shutdown_connections 2024-02-17T12:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.493+0000 7fa5f8a20740 1 -- 172.21.15.94:0/3838620493 wait complete. 2024-02-17T12:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.493+0000 7fa5f8a20740 1 -- start start 2024-02-17T12:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.494+0000 7fa5f8a20740 1 Processor -- start 2024-02-17T12:00:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.494+0000 7fa5f8a20740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bad60 0x5619a60c1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.494+0000 7fa5f8a20740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 0x5619a60c1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.494+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bad60 0x5619a60c1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.494+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bad60 0x5619a60c1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:48146/0 (socket says 172.21.15.94:48146) 2024-02-17T12:00:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.494+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 0x5619a60c1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.494+0000 7fa5eeffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 0x5619a60c1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:59048/0 (socket says 172.21.15.94:59048) 2024-02-17T12:00:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5ef7fe640 1 -- 172.21.15.94:0/136671134 learned_addr learned my addr 172.21.15.94:0/136671134 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/136671134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5f8a20740 1 -- 172.21.15.94:0/136671134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a5fc7bf0 con 0x5619a60c42e0 2024-02-17T12:00:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5f8a20740 1 -- 172.21.15.94:0/136671134 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a5cf6250 con 0x5619a60bad60 2024-02-17T12:00:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5f8a20740 1 -- 172.21.15.94:0/136671134 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a5fa8200 con 0x5619a60bbad0 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5effff640 1 --2- 172.21.15.94:0/136671134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5f8a20740 10 client.0 _ll_register_callbacks cb 0x5619a6016d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5eeffd640 1 -- 172.21.15.94:0/136671134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bad60 msgr2=0x5619a60c1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5eeffd640 1 --2- 172.21.15.94:0/136671134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bad60 0x5619a60c1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5eeffd640 1 -- 172.21.15.94:0/136671134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 msgr2=0x5619a60c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5eeffd640 1 --2- 172.21.15.94:0/136671134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5eeffd640 1 -- 172.21.15.94:0/136671134 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e00099f0 con 0x5619a60bbad0 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5eeffd640 1 --2- 172.21.15.94:0/136671134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 0x5619a60c1840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40ed4c0 tx=0x7fa5e40bc180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d095fdb58c329d69 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.495+0000 7fa5f8a20740 1 -- 172.21.15.94:0/136671134 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5619a5d22600 con 0x5619a60bbad0 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.496+0000 7fa5ceffd640 1 -- 172.21.15.94:0/136671134 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40d4070 con 0x5619a60bbad0 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.496+0000 7fa5ceffd640 1 -- 172.21.15.94:0/136671134 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa5e40d5260 con 0x5619a60bbad0 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.496+0000 7fa5ceffd640 1 -- 172.21.15.94:0/136671134 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5e40c4f00 con 0x5619a60bbad0 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.496+0000 7fa5f8a20740 10 client.25618 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.496+0000 7fa5f8a20740 1 -- 172.21.15.94:0/136671134 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=21}) v3 -- 0x5619a6020d80 con 0x5619a60bbad0 2024-02-17T12:00:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.497+0000 7fa5ceffd640 1 -- 172.21.15.94:0/136671134 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa5e4004c50 con 0x5619a60bbad0 2024-02-17T12:00:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.497+0000 7fa5f8a20740 10 client.25618 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.497+0000 7fa5f8a20740 10 client.25618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.497+0000 7fa5f8a20740 20 client.25618 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.497+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/136671134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5619a60dc9c0 0x5619a60dee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.497+0000 7fa5f8a20740 4 client.25618 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.497+0000 7fa5f8a20740 1 -- 172.21.15.94:0/136671134 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619a5d22600 con 0x5619a60dc9c0 2024-02-17T12:00:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.497+0000 7fa5ef7fe640 1 --2- 172.21.15.94:0/136671134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5619a60dc9c0 0x5619a60dee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.498+0000 7fa5ef7fe640 1 --2- 172.21.15.94:0/136671134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5619a60dc9c0 0x5619a60dee10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82c39fa2e73e9e25 server_cookie=319434732baeb865 in_seq=0 out_seq=0 2024-02-17T12:00:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.498+0000 7fa5ceffd640 10 client.25618 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.498+0000 7fa5ceffd640 1 -- 172.21.15.94:0/136671134 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5619a5d22600 con 0x5619a60dc9c0 2024-02-17T12:00:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.498+0000 7fa5ceffd640 10 client.25618 handle_command_reply: tid=0 2024-02-17T12:00:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.498+0000 7fa5f8a20740 1 client.25618 shutdown 2024-02-17T12:00:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.498+0000 7fa5f8a20740 1 -- 172.21.15.94:0/136671134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 msgr2=0x5619a60c1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.498+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/136671134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 0x5619a60c1840 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa5e40ed4c0 tx=0x7fa5e40bc180 comp rx=0 tx=0).stop 2024-02-17T12:00:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.498+0000 7fa5f8a20740 1 -- 172.21.15.94:0/136671134 shutdown_connections 2024-02-17T12:00:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.499+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/136671134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5619a60dc9c0 0x5619a60dee10 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.499+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/136671134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.499+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/136671134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bbad0 0x5619a60c1840 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.499+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/136671134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bad60 0x5619a60c1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.499+0000 7fa5f8a20740 1 -- 172.21.15.94:0/136671134 >> 172.21.15.94:0/136671134 conn(0x5619a6082550 msgr2=0x5619a60b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.499+0000 7fa5ceffd640 0 client.25618 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.499+0000 7fa5f8a20740 1 -- 172.21.15.94:0/136671134 shutdown_connections 2024-02-17T12:00:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.499+0000 7fa5f8a20740 1 -- 172.21.15.94:0/136671134 wait complete. 2024-02-17T12:00:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.499+0000 7fa5f8a20740 20 client.25618 trim_cache size 0 max 16384 2024-02-17T12:00:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.523+0000 7fa5f8a20740 1 Processor -- start 2024-02-17T12:00:21.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.523+0000 7fa5f8a20740 1 -- start start 2024-02-17T12:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.523+0000 7fa5f8a20740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bad60 0x5619a60c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.523+0000 7fa5f8a20740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bbad0 0x5619a60bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.523+0000 7fa5f8a20740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.523+0000 7fa5f8a20740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a60dc7c0 con 0x5619a60c42e0 2024-02-17T12:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.523+0000 7fa5f8a20740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a60c10f0 con 0x5619a60bbad0 2024-02-17T12:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.523+0000 7fa5f8a20740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a60c0650 con 0x5619a60bad60 2024-02-17T12:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.524+0000 7fa5effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.524+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bad60 0x5619a60c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.524+0000 7fa5effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58648/0 (socket says 172.21.15.94:58648) 2024-02-17T12:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.524+0000 7fa5effff640 1 -- 172.21.15.94:0/2066740750 learned_addr learned my addr 172.21.15.94:0/2066740750 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:21.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.524+0000 7fa5eeffd640 1 --2- 172.21.15.94:0/2066740750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bbad0 0x5619a60bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.524+0000 7fa5effff640 1 -- 172.21.15.94:0/2066740750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bad60 msgr2=0x5619a60c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.524+0000 7fa5effff640 1 --2- 172.21.15.94:0/2066740750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bad60 0x5619a60c3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.524+0000 7fa5effff640 1 -- 172.21.15.94:0/2066740750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bbad0 msgr2=0x5619a60bfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.524+0000 7fa5effff640 1 --2- 172.21.15.94:0/2066740750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bbad0 0x5619a60bfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.524+0000 7fa5effff640 1 -- 172.21.15.94:0/2066740750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619a5fbd7b0 con 0x5619a60c42e0 2024-02-17T12:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.524+0000 7fa5ef7fe640 1 --2- 172.21.15.94:0/2066740750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bad60 0x5619a60c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:21.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.524+0000 7fa5effff640 1 --2- 172.21.15.94:0/2066740750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c0110 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa5e801b7a0 tx=0x7fa5e801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45abf03a1fa26cd4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:21.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.525+0000 7fa5d57fa640 1 -- 172.21.15.94:0/2066740750 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8020070 con 0x5619a60c42e0 2024-02-17T12:00:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.525+0000 7fa5d57fa640 1 -- 172.21.15.94:0/2066740750 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5e8005020 con 0x5619a60c42e0 2024-02-17T12:00:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.525+0000 7fa5d57fa640 1 -- 172.21.15.94:0/2066740750 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80196a0 con 0x5619a60c42e0 2024-02-17T12:00:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.526+0000 7fa5f8a20740 1 -- 172.21.15.94:0/2066740750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 msgr2=0x5619a60c0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.526+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/2066740750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c0110 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa5e801b7a0 tx=0x7fa5e801a000 comp rx=0 tx=0).stop 2024-02-17T12:00:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.526+0000 7fa5f8a20740 1 -- 172.21.15.94:0/2066740750 shutdown_connections 2024-02-17T12:00:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.526+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/2066740750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60c42e0 0x5619a60c0110 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.526+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/2066740750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bbad0 0x5619a60bfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.526+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/2066740750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60bad60 0x5619a60c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.526+0000 7fa5f8a20740 1 -- 172.21.15.94:0/2066740750 >> 172.21.15.94:0/2066740750 conn(0x5619a6082550 msgr2=0x5619a60b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.526+0000 7fa5f8a20740 1 -- 172.21.15.94:0/2066740750 shutdown_connections 2024-02-17T12:00:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.527+0000 7fa5f8a20740 1 -- 172.21.15.94:0/2066740750 wait complete. 2024-02-17T12:00:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.527+0000 7fa5f8a20740 1 -- start start 2024-02-17T12:00:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.527+0000 7fa5f8a20740 1 Processor -- start 2024-02-17T12:00:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.527+0000 7fa5f8a20740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60bad60 0x5619a60d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.527+0000 7fa5f8a20740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bbad0 0x5619a60d2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.527+0000 7fa5f8a20740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60c42e0 0x5619a60d3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.528+0000 7fa5f8a20740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a61330a0 con 0x5619a60bad60 2024-02-17T12:00:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.528+0000 7fa5f8a20740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a6133210 con 0x5619a60bbad0 2024-02-17T12:00:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.528+0000 7fa5f8a20740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619a61333d0 con 0x5619a60c42e0 2024-02-17T12:00:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.528+0000 7fa5f8a20740 10 client.0 _ll_register_callbacks cb 0x7fa5e0009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.528+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60bad60 0x5619a60d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.528+0000 7fa5effff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60c42e0 0x5619a60d3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.528+0000 7fa5ef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60bad60 0x5619a60d26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58658/0 (socket says 172.21.15.94:58658) 2024-02-17T12:00:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.528+0000 7fa5effff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60c42e0 0x5619a60d3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:59058/0 (socket says 172.21.15.94:59058) 2024-02-17T12:00:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.528+0000 7fa5ef7fe640 1 -- 172.21.15.94:0/17431918 learned_addr learned my addr 172.21.15.94:0/17431918 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.528+0000 7fa5eeffd640 1 --2- 172.21.15.94:0/17431918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bbad0 0x5619a60d2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.529+0000 7fa5effff640 1 -- 172.21.15.94:0/17431918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bbad0 msgr2=0x5619a60d2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.529+0000 7fa5effff640 1 --2- 172.21.15.94:0/17431918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bbad0 0x5619a60d2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.529+0000 7fa5effff640 1 -- 172.21.15.94:0/17431918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60bad60 msgr2=0x5619a60d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.529+0000 7fa5effff640 1 --2- 172.21.15.94:0/17431918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60bad60 0x5619a60d26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.529+0000 7fa5effff640 1 -- 172.21.15.94:0/17431918 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619a5fb87b0 con 0x5619a60c42e0 2024-02-17T12:00:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.530+0000 7fa5effff640 1 --2- 172.21.15.94:0/17431918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60c42e0 0x5619a60d3160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8026040 tx=0x7fa5e8005ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99901e8d234da677 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.530+0000 7fa5ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:00:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.530+0000 7fa5ceffd640 1 -- 172.21.15.94:0/17431918 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8020040 con 0x5619a60c42e0 2024-02-17T12:00:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.530+0000 7fa5ceffd640 1 -- 172.21.15.94:0/17431918 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa5e8004dd0 con 0x5619a60c42e0 2024-02-17T12:00:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.531+0000 7fa5f8a20740 1 -- 172.21.15.94:0/17431918 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5e0004c80 con 0x5619a60c42e0 2024-02-17T12:00:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.531+0000 7fa5ceffd640 1 -- 172.21.15.94:0/17431918 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5e8019460 con 0x5619a60c42e0 2024-02-17T12:00:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.531+0000 7fa5f8a20740 10 client.25624 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.531+0000 7fa5f8a20740 1 -- 172.21.15.94:0/17431918 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x7fa5e801b500 con 0x5619a60c42e0 2024-02-17T12:00:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.531+0000 7fa5ceffd640 1 -- 172.21.15.94:0/17431918 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5e803b020 con 0x5619a60c42e0 2024-02-17T12:00:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.532+0000 7fa5ceffd640 1 -- 172.21.15.94:0/17431918 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa5e802e6e0 con 0x5619a60c42e0 2024-02-17T12:00:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.532+0000 7fa5f8a20740 10 client.25624 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.532+0000 7fa5f8a20740 10 client.25624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.532+0000 7fa5f8a20740 20 client.25624 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.532+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/17431918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5619a6134ad0 0x5619a6136ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.532+0000 7fa5f8a20740 4 client.25624 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:00:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.532+0000 7fa5f8a20740 1 -- 172.21.15.94:0/17431918 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fa5e0004c80 con 0x5619a6134ad0 2024-02-17T12:00:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.532+0000 7fa5ef7fe640 1 --2- 172.21.15.94:0/17431918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5619a6134ad0 0x5619a6136ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.533+0000 7fa5ef7fe640 1 --2- 172.21.15.94:0/17431918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5619a6134ad0 0x5619a6136ec0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8215ef02d8f4d server_cookie=be40b27b9a0bdc30 in_seq=0 out_seq=0 2024-02-17T12:00:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.533+0000 7fa5ceffd640 10 client.25624 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.552+0000 7fa5ceffd640 1 -- 172.21.15.94:0/17431918 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fa5e0004c80 con 0x5619a6134ad0 2024-02-17T12:00:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.552+0000 7fa5ceffd640 10 client.25624 handle_command_reply: tid=0 2024-02-17T12:00:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.552+0000 7fa5f8a20740 1 client.25624 shutdown 2024-02-17T12:00:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.552+0000 7fa5f8a20740 1 -- 172.21.15.94:0/17431918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60c42e0 msgr2=0x5619a60d3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.552+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/17431918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60c42e0 0x5619a60d3160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8026040 tx=0x7fa5e8005ab0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.552+0000 7fa5f8a20740 1 -- 172.21.15.94:0/17431918 shutdown_connections 2024-02-17T12:00:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.552+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/17431918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5619a6134ad0 0x5619a6136ec0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.552+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/17431918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619a60c42e0 0x5619a60d3160 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.552+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/17431918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5619a60bbad0 0x5619a60d2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.552+0000 7fa5f8a20740 1 --2- 172.21.15.94:0/17431918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5619a60bad60 0x5619a60d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.552+0000 7fa5f8a20740 1 -- 172.21.15.94:0/17431918 >> 172.21.15.94:0/17431918 conn(0x5619a6082550 msgr2=0x5619a60b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.552+0000 7fa5ceffd640 0 client.25624 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.552+0000 7fa5f8a20740 1 -- 172.21.15.94:0/17431918 shutdown_connections 2024-02-17T12:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.553+0000 7fa5f8a20740 1 -- 172.21.15.94:0/17431918 wait complete. 2024-02-17T12:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.553+0000 7fa5f8a20740 20 client.25624 trim_cache size 0 max 16384 2024-02-17T12:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:00:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:00:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.553+0000 7fa5f5e57640 1 -- 172.21.15.94:0/580415272 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5d80a24b0 msgr2=0x7fa5d80a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:21.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.553+0000 7fa5f5e57640 1 --2- 172.21.15.94:0/580415272 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5d80a24b0 0x7fa5d80a4970 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8005fd0 tx=0x7fa5e800e7c0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.553+0000 7fa5f5e57640 1 -- 172.21.15.94:0/580415272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f0105380 msgr2=0x7fa5f0197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.553+0000 7fa5f5e57640 1 --2- 172.21.15.94:0/580415272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f0105380 0x7fa5f0197a20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4009780 tx=0x7fa5e40346f0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.554+0000 7fa5f5e57640 1 -- 172.21.15.94:0/580415272 shutdown_connections 2024-02-17T12:00:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.554+0000 7fa5f5e57640 1 --2- 172.21.15.94:0/580415272 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5d80a24b0 0x7fa5d80a4970 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.554+0000 7fa5f5e57640 1 --2- 172.21.15.94:0/580415272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5f019bda0 0x7fa5f019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.554+0000 7fa5f5e57640 1 --2- 172.21.15.94:0/580415272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5f0105380 0x7fa5f0197a20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.554+0000 7fa5f5e57640 1 --2- 172.21.15.94:0/580415272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5f01002b0 0x7fa5f01974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.554+0000 7fa5f5e57640 1 -- 172.21.15.94:0/580415272 >> 172.21.15.94:0/580415272 conn(0x7fa5f00771b0 msgr2=0x7fa5f00fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.554+0000 7fa5f5e57640 1 -- 172.21.15.94:0/580415272 shutdown_connections 2024-02-17T12:00:21.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:21.554+0000 7fa5f5e57640 1 -- 172.21.15.94:0/580415272 wait complete. 2024-02-17T12:00:22.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:21 smithi189 ceph-mon[36588]: pgmap v70: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 8.7 MiB/s wr, 778 op/s 2024-02-17T12:00:22.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:21 smithi125 ceph-mon[36819]: pgmap v70: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 8.7 MiB/s wr, 778 op/s 2024-02-17T12:00:22.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:21 smithi094 ceph-mon[32499]: pgmap v70: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 8.7 MiB/s wr, 778 op/s 2024-02-17T12:00:24.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:23 smithi189 ceph-mon[36588]: pgmap v71: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 12 MiB/s wr, 1.08k op/s 2024-02-17T12:00:24.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:23 smithi125 ceph-mon[36819]: pgmap v71: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 12 MiB/s wr, 1.08k op/s 2024-02-17T12:00:24.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:23 smithi094 ceph-mon[32499]: pgmap v71: 97 pgs: 97 active+clean; 1.6 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 12 MiB/s wr, 1.08k op/s 2024-02-17T12:00:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 125.0 s, 3516.2 tps, lat 28.410 ms stddev 31.554 2024-02-17T12:00:26.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:25 smithi189 ceph-mon[36588]: pgmap v72: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.8 MiB/s wr, 852 op/s 2024-02-17T12:00:26.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:25 smithi125 ceph-mon[36819]: pgmap v72: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.8 MiB/s wr, 852 op/s 2024-02-17T12:00:26.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:25 smithi094 ceph-mon[32499]: pgmap v72: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 9.8 MiB/s wr, 852 op/s 2024-02-17T12:00:28.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:27 smithi189 ceph-mon[36588]: pgmap v73: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.0 MiB/s wr, 751 op/s 2024-02-17T12:00:28.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:27 smithi125 ceph-mon[36819]: pgmap v73: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.0 MiB/s wr, 751 op/s 2024-02-17T12:00:28.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:27 smithi094 ceph-mon[32499]: pgmap v73: 97 pgs: 97 active+clean; 1.7 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.0 MiB/s wr, 751 op/s 2024-02-17T12:00:30.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:29 smithi189 ceph-mon[36588]: pgmap v74: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2024-02-17T12:00:30.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:29 smithi125 ceph-mon[36819]: pgmap v74: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2024-02-17T12:00:30.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:29 smithi094 ceph-mon[32499]: pgmap v74: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 13 MiB/s wr, 1.07k op/s 2024-02-17T12:00:30.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 130.0 s, 3678.6 tps, lat 27.053 ms stddev 28.930 2024-02-17T12:00:32.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:31 smithi189 ceph-mon[36588]: pgmap v75: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.4 MiB/s wr, 707 op/s 2024-02-17T12:00:32.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:31 smithi125 ceph-mon[36819]: pgmap v75: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.4 MiB/s wr, 707 op/s 2024-02-17T12:00:32.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:31 smithi094 ceph-mon[32499]: pgmap v75: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.4 MiB/s wr, 707 op/s 2024-02-17T12:00:34.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:33 smithi189 ceph-mon[36588]: pgmap v76: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 915 op/s 2024-02-17T12:00:34.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:33 smithi125 ceph-mon[36819]: pgmap v76: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 915 op/s 2024-02-17T12:00:34.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:33 smithi094 ceph-mon[32499]: pgmap v76: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 10 MiB/s wr, 915 op/s 2024-02-17T12:00:34.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.682+0000 7f2be169d640 1 -- 172.21.15.94:0/3650844435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bdc10fd30 msgr2=0x7f2bdc10da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.682+0000 7f2be169d640 1 --2- 172.21.15.94:0/3650844435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bdc10fd30 0x7f2bdc10da20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009900 tx=0x7f2bcc02bc40 comp rx=0 tx=0).stop 2024-02-17T12:00:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.682+0000 7f2be169d640 1 -- 172.21.15.94:0/3650844435 shutdown_connections 2024-02-17T12:00:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.682+0000 7f2be169d640 1 --2- 172.21.15.94:0/3650844435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bdc11a960 0x7f2bdc11cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.682+0000 7f2be169d640 1 --2- 172.21.15.94:0/3650844435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bdc10fd30 0x7f2bdc10da20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.682+0000 7f2be169d640 1 --2- 172.21.15.94:0/3650844435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bdc10f8c0 0x7f2bdc10d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.682+0000 7f2be169d640 1 -- 172.21.15.94:0/3650844435 >> 172.21.15.94:0/3650844435 conn(0x7f2bdc06c220 msgr2=0x7f2bdc06c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.682+0000 7f2be169d640 1 -- 172.21.15.94:0/3650844435 shutdown_connections 2024-02-17T12:00:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.682+0000 7f2be169d640 1 -- 172.21.15.94:0/3650844435 wait complete. 2024-02-17T12:00:34.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.683+0000 7f2be169d640 1 Processor -- start 2024-02-17T12:00:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.683+0000 7f2be169d640 1 -- start start 2024-02-17T12:00:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.683+0000 7f2be169d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bdc10f8c0 0x7f2bdc1a6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.683+0000 7f2be169d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bdc10fd30 0x7f2bdc1a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.683+0000 7f2be169d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bdc11a960 0x7f2bdc1aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.683+0000 7f2be169d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc1ab420 con 0x7f2bdc10fd30 2024-02-17T12:00:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.683+0000 7f2be169d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc1ab590 con 0x7f2bdc11a960 2024-02-17T12:00:34.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.683+0000 7f2be169d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bdc1ab750 con 0x7f2bdc10f8c0 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.684+0000 7f2bdb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bdc11a960 0x7f2bdc1aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.684+0000 7f2bda7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bdc10fd30 0x7f2bdc1a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.684+0000 7f2bdb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bdc11a960 0x7f2bdc1aae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36608/0 (socket says 172.21.15.94:36608) 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.684+0000 7f2bdb7fe640 1 -- 172.21.15.94:0/2797917345 learned_addr learned my addr 172.21.15.94:0/2797917345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.684+0000 7f2bdaffd640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bdc10f8c0 0x7f2bdc1a6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.684+0000 7f2bdb7fe640 1 -- 172.21.15.94:0/2797917345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bdc10f8c0 msgr2=0x7f2bdc1a6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.684+0000 7f2bdb7fe640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bdc10f8c0 0x7f2bdc1a6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.684+0000 7f2bdb7fe640 1 -- 172.21.15.94:0/2797917345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bdc10fd30 msgr2=0x7f2bdc1a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.684+0000 7f2bdb7fe640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bdc10fd30 0x7f2bdc1a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.684+0000 7f2bdb7fe640 1 -- 172.21.15.94:0/2797917345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc009560 con 0x7f2bdc11a960 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.684+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bdc10fd30 0x7f2bdc1a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.684+0000 7f2bdb7fe640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bdc11a960 0x7f2bdc1aae20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400e9d0 tx=0x7f2bd400ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.685+0000 7f2bc3fff640 1 -- 172.21.15.94:0/2797917345 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd400cd80 con 0x7f2bdc11a960 2024-02-17T12:00:34.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.685+0000 7f2bc3fff640 1 -- 172.21.15.94:0/2797917345 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bd4005020 con 0x7f2bdc11a960 2024-02-17T12:00:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.685+0000 7f2bc3fff640 1 -- 172.21.15.94:0/2797917345 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40116d0 con 0x7f2bdc11a960 2024-02-17T12:00:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.685+0000 7f2be169d640 1 -- 172.21.15.94:0/2797917345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bdc1aba30 con 0x7f2bdc11a960 2024-02-17T12:00:34.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.685+0000 7f2be169d640 1 -- 172.21.15.94:0/2797917345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bdc1b32b0 con 0x7f2bdc11a960 2024-02-17T12:00:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.687+0000 7f2bc3fff640 1 -- 172.21.15.94:0/2797917345 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2bd4005500 con 0x7f2bdc11a960 2024-02-17T12:00:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2be4275740 1 Processor -- start 2024-02-17T12:00:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2be4275740 1 -- start start 2024-02-17T12:00:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2be4275740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e7ad0 0x5588250e6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2be4275740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e6d60 0x5588250e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2be4275740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2be4275740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588250559f0 con 0x5588250e6d60 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2be4275740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558825052f20 con 0x5588250f02e0 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2be4275740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882504e700 con 0x5588250e7ad0 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2bdaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e6d60 0x5588250e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2bdaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e6d60 0x5588250e71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43362/0 (socket says 172.21.15.94:43362) 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2bdaffd640 1 -- 172.21.15.94:0/3960223106 learned_addr learned my addr 172.21.15.94:0/3960223106 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2bc3fff640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2bc40a2430 0x7f2bc40a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/3960223106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2bdb7fe640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2bc40a2430 0x7f2bc40a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.688+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/3960223106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e7ad0 0x5588250e6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.689+0000 7f2bdaffd640 1 -- 172.21.15.94:0/3960223106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e7ad0 msgr2=0x5588250e6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.689+0000 7f2bdaffd640 1 --2- 172.21.15.94:0/3960223106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e7ad0 0x5588250e6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.689+0000 7f2bdaffd640 1 -- 172.21.15.94:0/3960223106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 msgr2=0x5588250f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.689+0000 7f2bc3fff640 1 -- 172.21.15.94:0/2797917345 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2bd40ce1d0 con 0x7f2bdc11a960 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.689+0000 7f2bdaffd640 1 --2- 172.21.15.94:0/3960223106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.689+0000 7f2bdaffd640 1 -- 172.21.15.94:0/3960223106 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558824fe47b0 con 0x5588250e6d60 2024-02-17T12:00:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.689+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/3960223106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.689+0000 7f2bdaffd640 1 --2- 172.21.15.94:0/3960223106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e6d60 0x5588250e71e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000ac20 tx=0x7f2bd000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be657340f3687f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.689+0000 7f2bdb7fe640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2bc40a2430 0x7f2bc40a48f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40cbf10 tx=0x7f2bd406dca0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.689+0000 7f2bc17fa640 1 -- 172.21.15.94:0/3960223106 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd000a720 con 0x5588250e6d60 2024-02-17T12:00:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.689+0000 7f2bc17fa640 1 -- 172.21.15.94:0/3960223106 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0018b20 con 0x5588250e6d60 2024-02-17T12:00:34.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.689+0000 7f2bc17fa640 1 -- 172.21.15.94:0/3960223106 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0010c80 con 0x5588250e6d60 2024-02-17T12:00:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.690+0000 7f2be4275740 1 -- 172.21.15.94:0/3960223106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e6d60 msgr2=0x5588250e71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:34.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.690+0000 7f2be4275740 1 --2- 172.21.15.94:0/3960223106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e6d60 0x5588250e71e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000ac20 tx=0x7f2bd000f790 comp rx=0 tx=0).stop 2024-02-17T12:00:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.690+0000 7f2be4275740 1 -- 172.21.15.94:0/3960223106 shutdown_connections 2024-02-17T12:00:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.690+0000 7f2be4275740 1 --2- 172.21.15.94:0/3960223106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.690+0000 7f2be4275740 1 --2- 172.21.15.94:0/3960223106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e6d60 0x5588250e71e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.690+0000 7f2be4275740 1 --2- 172.21.15.94:0/3960223106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e7ad0 0x5588250e6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.690+0000 7f2be4275740 1 -- 172.21.15.94:0/3960223106 >> 172.21.15.94:0/3960223106 conn(0x5588250ae550 msgr2=0x5588250e1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.690+0000 7f2be4275740 1 -- 172.21.15.94:0/3960223106 shutdown_connections 2024-02-17T12:00:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.690+0000 7f2be4275740 1 -- 172.21.15.94:0/3960223106 wait complete. 2024-02-17T12:00:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.690+0000 7f2be4275740 1 -- start start 2024-02-17T12:00:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.690+0000 7f2be4275740 1 Processor -- start 2024-02-17T12:00:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2be4275740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250e6d60 0x5588250ed360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2be4275740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e7ad0 0x5588250ed8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2be4275740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250f02e0 0x5588250edde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2be4275740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558824ff3bf0 con 0x5588250e7ad0 2024-02-17T12:00:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2be4275740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558824d22250 con 0x5588250e6d60 2024-02-17T12:00:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2be4275740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558824fd4200 con 0x5588250f02e0 2024-02-17T12:00:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2be4275740 10 client.0 _ll_register_callbacks cb 0x558825042d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2bda7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250e6d60 0x5588250ed360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2bdaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e7ad0 0x5588250ed8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2bda7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250e6d60 0x5588250ed360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36638/0 (socket says 172.21.15.94:36638) 2024-02-17T12:00:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2bda7fc640 1 -- 172.21.15.94:0/2991007385 learned_addr learned my addr 172.21.15.94:0/2991007385 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/2991007385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250f02e0 0x5588250edde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2bda7fc640 1 -- 172.21.15.94:0/2991007385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250f02e0 msgr2=0x5588250edde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/2991007385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250f02e0 0x5588250edde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2bda7fc640 1 -- 172.21.15.94:0/2991007385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e7ad0 msgr2=0x5588250ed8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:34.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/2991007385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e7ad0 0x5588250ed8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2bda7fc640 1 -- 172.21.15.94:0/2991007385 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd000a880 con 0x5588250e6d60 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2bdaffd640 1 --2- 172.21.15.94:0/2991007385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e7ad0 0x5588250ed8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/2991007385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250e6d60 0x5588250ed360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009900 tx=0x7f2bcc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9d98f6c8d4bcd48 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.691+0000 7f2bbeff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2bbeff5640 1 -- 172.21.15.94:0/2991007385 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc004df0 con 0x5588250e6d60 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2bbeff5640 1 -- 172.21.15.94:0/2991007385 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2bcc004f50 con 0x5588250e6d60 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2be4275740 1 -- 172.21.15.94:0/2991007385 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558824d4e600 con 0x5588250e6d60 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2bbeff5640 1 -- 172.21.15.94:0/2991007385 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bcc005110 con 0x5588250e6d60 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2be4275740 10 client.25577 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2be4275740 1 -- 172.21.15.94:0/2991007385 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55882504cd80 con 0x5588250e6d60 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2bbeff5640 1 -- 172.21.15.94:0/2991007385 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558824d4e600 con 0x5588250e6d60 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2bbeff5640 1 -- 172.21.15.94:0/2991007385 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2bcc003c40 con 0x5588250e6d60 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2be4275740 10 client.25577 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2be4275740 10 client.25577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:34.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2be4275740 20 client.25577 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2be4275740 1 --2- 172.21.15.94:0/2991007385 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558825111680 0x558825113a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2be4275740 4 client.25577 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.692+0000 7f2be4275740 1 -- 172.21.15.94:0/2991007385 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558824d9f1b0 con 0x558825111680 2024-02-17T12:00:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.693+0000 7f2bdaffd640 1 --2- 172.21.15.94:0/2991007385 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558825111680 0x558825113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.693+0000 7f2bbeff5640 10 client.25577 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.693+0000 7f2bdaffd640 1 --2- 172.21.15.94:0/2991007385 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558825111680 0x558825113a70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b1fb5de713cc179 server_cookie=6606d5c67d600fc7 in_seq=0 out_seq=0 2024-02-17T12:00:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.693+0000 7f2bbeff5640 1 -- 172.21.15.94:0/2991007385 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558824d9f1b0 con 0x558825111680 2024-02-17T12:00:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.693+0000 7f2bbeff5640 10 client.25577 handle_command_reply: tid=0 2024-02-17T12:00:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.693+0000 7f2be4275740 1 client.25577 shutdown 2024-02-17T12:00:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.694+0000 7f2be4275740 1 -- 172.21.15.94:0/2991007385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250e6d60 msgr2=0x5588250ed360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.694+0000 7f2be4275740 1 --2- 172.21.15.94:0/2991007385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250e6d60 0x5588250ed360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009900 tx=0x7f2bcc009260 comp rx=0 tx=0).stop 2024-02-17T12:00:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.694+0000 7f2be4275740 1 -- 172.21.15.94:0/2991007385 shutdown_connections 2024-02-17T12:00:34.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.694+0000 7f2be4275740 1 --2- 172.21.15.94:0/2991007385 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558825111680 0x558825113a70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.694+0000 7f2be4275740 1 --2- 172.21.15.94:0/2991007385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250f02e0 0x5588250edde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.694+0000 7f2be4275740 1 --2- 172.21.15.94:0/2991007385 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e7ad0 0x5588250ed8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.694+0000 7f2bbeff5640 0 client.25577 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.694+0000 7f2be4275740 1 --2- 172.21.15.94:0/2991007385 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250e6d60 0x5588250ed360 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.694+0000 7f2be4275740 1 -- 172.21.15.94:0/2991007385 >> 172.21.15.94:0/2991007385 conn(0x5588250ae550 msgr2=0x5588250e37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.694+0000 7f2be4275740 1 -- 172.21.15.94:0/2991007385 shutdown_connections 2024-02-17T12:00:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.694+0000 7f2be4275740 1 -- 172.21.15.94:0/2991007385 wait complete. 2024-02-17T12:00:34.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.695+0000 7f2be4275740 20 client.25577 trim_cache size 0 max 16384 2024-02-17T12:00:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.711+0000 7f2be4275740 1 Processor -- start 2024-02-17T12:00:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.711+0000 7f2be4275740 1 -- start start 2024-02-17T12:00:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.712+0000 7f2be4275740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e6d60 0x5588250efa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.712+0000 7f2be4275740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e7ad0 0x5588250f7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.712+0000 7f2be4275740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.712+0000 7f2be4275740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558824d4e600 con 0x5588250e6d60 2024-02-17T12:00:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.712+0000 7f2be4275740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588250e80c0 con 0x5588250f02e0 2024-02-17T12:00:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.712+0000 7f2be4275740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55882510e1d0 con 0x5588250e7ad0 2024-02-17T12:00:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.712+0000 7f2bdaffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e7ad0 0x5588250f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.713+0000 7f2bdaffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e7ad0 0x5588250f7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39590/0 (socket says 172.21.15.94:39590) 2024-02-17T12:00:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.713+0000 7f2bdaffd640 1 -- 172.21.15.94:0/3358305431 learned_addr learned my addr 172.21.15.94:0/3358305431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.713+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/3358305431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.713+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/3358305431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e6d60 0x5588250efa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.713+0000 7f2bdaffd640 1 -- 172.21.15.94:0/3358305431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 msgr2=0x5588250f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.713+0000 7f2bdaffd640 1 --2- 172.21.15.94:0/3358305431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.713+0000 7f2bdaffd640 1 -- 172.21.15.94:0/3358305431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e6d60 msgr2=0x5588250efa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.713+0000 7f2bdaffd640 1 --2- 172.21.15.94:0/3358305431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e6d60 0x5588250efa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.713+0000 7f2bdaffd640 1 -- 172.21.15.94:0/3358305431 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc04ef60 con 0x5588250e7ad0 2024-02-17T12:00:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.713+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/3358305431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:00:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.714+0000 7f2bdaffd640 1 --2- 172.21.15.94:0/3358305431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e7ad0 0x5588250f7100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2bd001b040 tx=0x7f2bd000ab50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d35d3a0a4da98fd5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.714+0000 7f2bc0ff9640 1 -- 172.21.15.94:0/3358305431 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0018640 con 0x5588250e7ad0 2024-02-17T12:00:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.714+0000 7f2bc0ff9640 1 -- 172.21.15.94:0/3358305431 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0010d50 con 0x5588250e7ad0 2024-02-17T12:00:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.714+0000 7f2bc0ff9640 1 -- 172.21.15.94:0/3358305431 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00054e0 con 0x5588250e7ad0 2024-02-17T12:00:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.715+0000 7f2be4275740 1 -- 172.21.15.94:0/3358305431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e7ad0 msgr2=0x5588250f7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.715+0000 7f2be4275740 1 --2- 172.21.15.94:0/3358305431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e7ad0 0x5588250f7100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2bd001b040 tx=0x7f2bd000ab50 comp rx=0 tx=0).stop 2024-02-17T12:00:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.715+0000 7f2be4275740 1 -- 172.21.15.94:0/3358305431 shutdown_connections 2024-02-17T12:00:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.715+0000 7f2be4275740 1 --2- 172.21.15.94:0/3358305431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.715+0000 7f2be4275740 1 --2- 172.21.15.94:0/3358305431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e7ad0 0x5588250f7100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.715+0000 7f2be4275740 1 --2- 172.21.15.94:0/3358305431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e6d60 0x5588250efa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.715+0000 7f2be4275740 1 -- 172.21.15.94:0/3358305431 >> 172.21.15.94:0/3358305431 conn(0x5588250ae550 msgr2=0x5588250e1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.715+0000 7f2be4275740 1 -- 172.21.15.94:0/3358305431 shutdown_connections 2024-02-17T12:00:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.715+0000 7f2be4275740 1 -- 172.21.15.94:0/3358305431 wait complete. 2024-02-17T12:00:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.715+0000 7f2be4275740 1 -- start start 2024-02-17T12:00:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.715+0000 7f2be4275740 1 Processor -- start 2024-02-17T12:00:34.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2be4275740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e6d60 0x5588250f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2be4275740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e7ad0 0x5588250f3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2be4275740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2be4275740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588250f4460 con 0x5588250e7ad0 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2be4275740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588250f45d0 con 0x5588250f02e0 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2be4275740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558825167b50 con 0x5588250e6d60 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2be4275740 10 client.0 _ll_register_callbacks cb 0x7f2bd000a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2bdaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e7ad0 0x5588250f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2bda7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e6d60 0x5588250f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2bdaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e7ad0 0x5588250f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43392/0 (socket says 172.21.15.94:43392) 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2bdaffd640 1 -- 172.21.15.94:0/1425997236 learned_addr learned my addr 172.21.15.94:0/1425997236 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/1425997236 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2bda7fc640 1 -- 172.21.15.94:0/1425997236 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 msgr2=0x5588250f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/1425997236 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.718 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2bda7fc640 1 -- 172.21.15.94:0/1425997236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e7ad0 msgr2=0x5588250f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/1425997236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e7ad0 0x5588250f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2bda7fc640 1 -- 172.21.15.94:0/1425997236 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558824fe47b0 con 0x5588250e6d60 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2bdaffd640 1 --2- 172.21.15.94:0/1425997236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e7ad0 0x5588250f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2bda7fc640 1 --2- 172.21.15.94:0/1425997236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e6d60 0x5588250f7450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x5588250f2ce0 tx=0x7f2bcc004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13da0757cec87d74 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.716+0000 7f2bbeff5640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2bbeff5640 1 -- 172.21.15.94:0/1425997236 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc02c2c0 con 0x5588250e6d60 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2be4275740 1 -- 172.21.15.94:0/1425997236 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bd0018b20 con 0x5588250e6d60 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2bbeff5640 1 -- 172.21.15.94:0/1425997236 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2bcc000e40 con 0x5588250e6d60 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2bbeff5640 1 -- 172.21.15.94:0/1425997236 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bcc043970 con 0x5588250e6d60 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2be4275740 10 client.25642 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2be4275740 1 -- 172.21.15.94:0/1425997236 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x558824fe97b0 con 0x5588250e6d60 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2bbeff5640 1 -- 172.21.15.94:0/1425997236 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0018b20 con 0x5588250e6d60 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2bbeff5640 1 -- 172.21.15.94:0/1425997236 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2bcc005050 con 0x5588250e6d60 2024-02-17T12:00:34.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2be4275740 10 client.25642 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2be4275740 10 client.25642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2be4275740 20 client.25642 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2be4275740 1 --2- 172.21.15.94:0/1425997236 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558825111680 0x55882516af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2be4275740 4 client.25642 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:00:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2be4275740 1 -- 172.21.15.94:0/1425997236 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558824d8b550 con 0x558825111680 2024-02-17T12:00:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.717+0000 7f2bdaffd640 1 --2- 172.21.15.94:0/1425997236 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558825111680 0x55882516af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.718+0000 7f2bdaffd640 1 --2- 172.21.15.94:0/1425997236 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558825111680 0x55882516af90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73f0372e6fae7e1c server_cookie=d334cb3d9146a169 in_seq=0 out_seq=0 2024-02-17T12:00:34.720 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.718+0000 7f2bbeff5640 10 client.25642 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.739+0000 7f2bbeff5640 1 -- 172.21.15.94:0/1425997236 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558824d8b550 con 0x558825111680 2024-02-17T12:00:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.739+0000 7f2bbeff5640 10 client.25642 handle_command_reply: tid=0 2024-02-17T12:00:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.739+0000 7f2be4275740 1 client.25642 shutdown 2024-02-17T12:00:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.740+0000 7f2be4275740 1 -- 172.21.15.94:0/1425997236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e6d60 msgr2=0x5588250f7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:34.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.740+0000 7f2be4275740 1 --2- 172.21.15.94:0/1425997236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e6d60 0x5588250f7450 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x5588250f2ce0 tx=0x7f2bcc004740 comp rx=0 tx=0).stop 2024-02-17T12:00:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.740+0000 7f2be4275740 1 -- 172.21.15.94:0/1425997236 shutdown_connections 2024-02-17T12:00:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.740+0000 7f2be4275740 1 --2- 172.21.15.94:0/1425997236 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558825111680 0x55882516af90 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.740+0000 7f2be4275740 1 --2- 172.21.15.94:0/1425997236 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588250f02e0 0x5588250f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.740+0000 7f2be4275740 1 --2- 172.21.15.94:0/1425997236 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588250e7ad0 0x5588250f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.740+0000 7f2be4275740 1 --2- 172.21.15.94:0/1425997236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588250e6d60 0x5588250f7450 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.740+0000 7f2bbeff5640 0 client.25642 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.740+0000 7f2be4275740 1 -- 172.21.15.94:0/1425997236 >> 172.21.15.94:0/1425997236 conn(0x5588250ae550 msgr2=0x5588250ebac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.740+0000 7f2be4275740 1 -- 172.21.15.94:0/1425997236 shutdown_connections 2024-02-17T12:00:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.740+0000 7f2be4275740 1 -- 172.21.15.94:0/1425997236 wait complete. 2024-02-17T12:00:34.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.740+0000 7f2be4275740 20 client.25642 trim_cache size 0 max 16384 2024-02-17T12:00:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:00:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:00:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:00:34.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:00:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.741+0000 7f2be169d640 1 -- 172.21.15.94:0/2797917345 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2bc40a2430 msgr2=0x7f2bc40a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.741+0000 7f2be169d640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2bc40a2430 0x7f2bc40a48f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2bd40cbf10 tx=0x7f2bd406dca0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.741+0000 7f2be169d640 1 -- 172.21.15.94:0/2797917345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bdc11a960 msgr2=0x7f2bdc1aae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.741+0000 7f2be169d640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bdc11a960 0x7f2bdc1aae20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400e9d0 tx=0x7f2bd400ee90 comp rx=0 tx=0).stop 2024-02-17T12:00:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.741+0000 7f2be169d640 1 -- 172.21.15.94:0/2797917345 shutdown_connections 2024-02-17T12:00:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.741+0000 7f2be169d640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2bc40a2430 0x7f2bc40a48f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.741+0000 7f2be169d640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2bdc11a960 0x7f2bdc1aae20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.741+0000 7f2be169d640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2bdc10fd30 0x7f2bdc1a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.741+0000 7f2be169d640 1 --2- 172.21.15.94:0/2797917345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2bdc10f8c0 0x7f2bdc1a6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.741+0000 7f2be169d640 1 -- 172.21.15.94:0/2797917345 >> 172.21.15.94:0/2797917345 conn(0x7f2bdc06c220 msgr2=0x7f2bdc1193e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.741+0000 7f2be169d640 1 -- 172.21.15.94:0/2797917345 shutdown_connections 2024-02-17T12:00:34.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:34.741+0000 7f2be169d640 1 -- 172.21.15.94:0/2797917345 wait complete. 2024-02-17T12:00:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 135.0 s, 3524.2 tps, lat 28.506 ms stddev 29.509 2024-02-17T12:00:36.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:35 smithi189 ceph-mon[36588]: pgmap v77: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 726 op/s 2024-02-17T12:00:36.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:35 smithi125 ceph-mon[36819]: pgmap v77: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 726 op/s 2024-02-17T12:00:36.204 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:35 smithi094 ceph-mon[32499]: pgmap v77: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 8.4 MiB/s wr, 726 op/s 2024-02-17T12:00:38.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:37 smithi189 ceph-mon[36588]: pgmap v78: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 634 op/s 2024-02-17T12:00:38.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:37 smithi125 ceph-mon[36819]: pgmap v78: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 634 op/s 2024-02-17T12:00:38.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:37 smithi094 ceph-mon[32499]: pgmap v78: 97 pgs: 97 active+clean; 1.7 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.7 MiB/s wr, 634 op/s 2024-02-17T12:00:40.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:39 smithi189 ceph-mon[36588]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 949 op/s 2024-02-17T12:00:40.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:39 smithi125 ceph-mon[36819]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 949 op/s 2024-02-17T12:00:40.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:39 smithi094 ceph-mon[32499]: pgmap v79: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 12 MiB/s wr, 949 op/s 2024-02-17T12:00:40.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 140.0 s, 3257.4 tps, lat 30.647 ms stddev 32.132 2024-02-17T12:00:40.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:00:40.438 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:00:40.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.595+0000 7f9cbca9d640 1 -- 172.21.15.94:0/3765773512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cb8100190 msgr2=0x7f9cb8100570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:40.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.595+0000 7f9cbca9d640 1 --2- 172.21.15.94:0/3765773512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cb8100190 0x7f9cb8100570 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009860 tx=0x7f9ca802bb70 comp rx=0 tx=0).stop 2024-02-17T12:00:40.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.599+0000 7f9cbca9d640 1 -- 172.21.15.94:0/3765773512 shutdown_connections 2024-02-17T12:00:40.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.599+0000 7f9cbca9d640 1 --2- 172.21.15.94:0/3765773512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb8105590 0x7f9cb8105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.599+0000 7f9cbca9d640 1 --2- 172.21.15.94:0/3765773512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb8100ab0 0x7f9cb8104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.599+0000 7f9cbca9d640 1 --2- 172.21.15.94:0/3765773512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cb8100190 0x7f9cb8100570 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.599+0000 7f9cbca9d640 1 -- 172.21.15.94:0/3765773512 >> 172.21.15.94:0/3765773512 conn(0x7f9cb8077210 msgr2=0x7f9cb8077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:40.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.600+0000 7f9cbca9d640 1 -- 172.21.15.94:0/3765773512 shutdown_connections 2024-02-17T12:00:40.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.600+0000 7f9cbca9d640 1 -- 172.21.15.94:0/3765773512 wait complete. 2024-02-17T12:00:40.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.601+0000 7f9cbca9d640 1 Processor -- start 2024-02-17T12:00:40.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.602+0000 7f9cbca9d640 1 -- start start 2024-02-17T12:00:40.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.602+0000 7f9cbca9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb8100190 0x7f9cb819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.602+0000 7f9cbca9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb8100ab0 0x7f9cb819e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.602+0000 7f9cbca9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cb8105590 0x7f9cb81a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.602+0000 7f9cbca9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb81a2cf0 con 0x7f9cb8100190 2024-02-17T12:00:40.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.602+0000 7f9cbca9d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb81a2e60 con 0x7f9cb8100ab0 2024-02-17T12:00:40.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.602+0000 7f9cbca9d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb81a3020 con 0x7f9cb8105590 2024-02-17T12:00:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.602+0000 7f9cb5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb8100ab0 0x7f9cb819e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.602+0000 7f9cb5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb8100ab0 0x7f9cb819e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36672/0 (socket says 172.21.15.94:36672) 2024-02-17T12:00:40.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.602+0000 7f9cb5d9b640 1 -- 172.21.15.94:0/579406797 learned_addr learned my addr 172.21.15.94:0/579406797 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:40.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.604+0000 7f9cb5d9b640 1 -- 172.21.15.94:0/579406797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cb8105590 msgr2=0x7f9cb81a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:40.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.604+0000 7f9cb5d9b640 1 --2- 172.21.15.94:0/579406797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cb8105590 0x7f9cb81a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.604+0000 7f9cb5d9b640 1 -- 172.21.15.94:0/579406797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb8100190 msgr2=0x7f9cb819de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:40.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.604+0000 7f9cb5d9b640 1 --2- 172.21.15.94:0/579406797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb8100190 0x7f9cb819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.604+0000 7f9cb5d9b640 1 -- 172.21.15.94:0/579406797 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca8009490 con 0x7f9cb8100ab0 2024-02-17T12:00:40.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.604+0000 7f9cb5d9b640 1 --2- 172.21.15.94:0/579406797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb8100ab0 0x7f9cb819e370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00e9c0 tx=0x7f9cac00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:40.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.605+0000 7f9ca77fe640 1 -- 172.21.15.94:0/579406797 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac00ccc0 con 0x7f9cb8100ab0 2024-02-17T12:00:40.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.605+0000 7f9cbca9d640 1 -- 172.21.15.94:0/579406797 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb81a3300 con 0x7f9cb8100ab0 2024-02-17T12:00:40.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.605+0000 7f9cbca9d640 1 -- 172.21.15.94:0/579406797 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb81aab80 con 0x7f9cb8100ab0 2024-02-17T12:00:40.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.605+0000 7f9ca77fe640 1 -- 172.21.15.94:0/579406797 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cac00ce20 con 0x7f9cb8100ab0 2024-02-17T12:00:40.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.605+0000 7f9ca77fe640 1 -- 172.21.15.94:0/579406797 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac010430 con 0x7f9cb8100ab0 2024-02-17T12:00:40.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.610+0000 7f9ca77fe640 1 -- 172.21.15.94:0/579406797 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9cac0046c0 con 0x7f9cb8100ab0 2024-02-17T12:00:40.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.612+0000 7f9ca77fe640 1 --2- 172.21.15.94:0/579406797 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9ca00a2440 0x7f9ca00a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.613+0000 7f9ca77fe640 1 -- 172.21.15.94:0/579406797 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9cac014070 con 0x7f9cb8100ab0 2024-02-17T12:00:40.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.617+0000 7f9cbf678740 1 Processor -- start 2024-02-17T12:00:40.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.617+0000 7f9cbf678740 1 -- start start 2024-02-17T12:00:40.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.618+0000 7f9cbf678740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d62b0 0x561fd41d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.618+0000 7f9cbf678740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d6bd0 0x561fd41df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.618+0000 7f9cbf678740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41df7b0 0x561fd41e1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.618+0000 7f9cbf678740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd4145280 con 0x561fd41d6bd0 2024-02-17T12:00:40.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.618+0000 7f9cbf678740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd41427b0 con 0x561fd41d62b0 2024-02-17T12:00:40.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.618+0000 7f9cbf678740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd413df90 con 0x561fd41df7b0 2024-02-17T12:00:40.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.618+0000 7f9cb6d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41df7b0 0x561fd41e1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.619+0000 7f9cb6d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41df7b0 0x561fd41e1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39628/0 (socket says 172.21.15.94:39628) 2024-02-17T12:00:40.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.619+0000 7f9cb6d9d640 1 -- 172.21.15.94:0/1291481855 learned_addr learned my addr 172.21.15.94:0/1291481855 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:40.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.619+0000 7f9cb5d9b640 1 --2- 172.21.15.94:0/1291481855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d6bd0 0x561fd41df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.619+0000 7f9cb659c640 1 --2- 172.21.15.94:0/579406797 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9ca00a2440 0x7f9ca00a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.620+0000 7f9cb659c640 1 --2- 172.21.15.94:0/1291481855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d62b0 0x561fd41d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.620+0000 7f9cb6d9d640 1 -- 172.21.15.94:0/1291481855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d62b0 msgr2=0x561fd41d6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.620+0000 7f9cb6d9d640 1 --2- 172.21.15.94:0/1291481855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d62b0 0x561fd41d6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.620+0000 7f9cb6d9d640 1 -- 172.21.15.94:0/1291481855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d6bd0 msgr2=0x561fd41df270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.620+0000 7f9cb6d9d640 1 --2- 172.21.15.94:0/1291481855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d6bd0 0x561fd41df270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.620+0000 7f9cb6d9d640 1 -- 172.21.15.94:0/1291481855 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fd40fc570 con 0x561fd41df7b0 2024-02-17T12:00:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.620+0000 7f9cb5d9b640 1 --2- 172.21.15.94:0/1291481855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d6bd0 0x561fd41df270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.620+0000 7f9cb659c640 1 --2- 172.21.15.94:0/579406797 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9ca00a2440 0x7f9ca00a4900 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009860 tx=0x7f9ca8035e60 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.621+0000 7f9cb6d9d640 1 --2- 172.21.15.94:0/1291481855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41df7b0 0x561fd41e1ba0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009d10 tx=0x7f9cb000e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f196f396944c36f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.621+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/1291481855 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0004b20 con 0x561fd41df7b0 2024-02-17T12:00:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.621+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/1291481855 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cb0004c80 con 0x561fd41df7b0 2024-02-17T12:00:40.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.621+0000 7f9ca4ff9640 1 -- 172.21.15.94:0/1291481855 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb00116d0 con 0x561fd41df7b0 2024-02-17T12:00:40.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.622+0000 7f9cbf678740 1 -- 172.21.15.94:0/1291481855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41df7b0 msgr2=0x561fd41e1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:40.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.622+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1291481855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41df7b0 0x561fd41e1ba0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009d10 tx=0x7f9cb000e6c0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.622+0000 7f9cbf678740 1 -- 172.21.15.94:0/1291481855 shutdown_connections 2024-02-17T12:00:40.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.622+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1291481855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41df7b0 0x561fd41e1ba0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.622+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1291481855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d6bd0 0x561fd41df270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.622+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1291481855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d62b0 0x561fd41d6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.622+0000 7f9cbf678740 1 -- 172.21.15.94:0/1291481855 >> 172.21.15.94:0/1291481855 conn(0x561fd41d0990 msgr2=0x561fd41d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:40.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.623+0000 7f9cbf678740 1 -- 172.21.15.94:0/1291481855 shutdown_connections 2024-02-17T12:00:40.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.623+0000 7f9cbf678740 1 -- 172.21.15.94:0/1291481855 wait complete. 2024-02-17T12:00:40.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.624+0000 7f9cbf678740 1 -- start start 2024-02-17T12:00:40.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.625+0000 7f9cbf678740 1 Processor -- start 2024-02-17T12:00:40.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.625+0000 7f9cbf678740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d62b0 0x561fd41df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.625+0000 7f9cbf678740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d6bd0 0x561fd41dc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.625+0000 7f9cbf678740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41df7b0 0x561fd41dc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cb5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d6bd0 0x561fd41dc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cb5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d6bd0 0x561fd41dc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43434/0 (socket says 172.21.15.94:43434) 2024-02-17T12:00:40.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cb5d9b640 1 -- 172.21.15.94:0/1862033527 learned_addr learned my addr 172.21.15.94:0/1862033527 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:40.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cb6d9d640 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41df7b0 0x561fd41dc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cbf678740 1 -- 172.21.15.94:0/1862033527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd3e11430 con 0x561fd41d6bd0 2024-02-17T12:00:40.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cbf678740 1 -- 172.21.15.94:0/1862033527 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd3dcde00 con 0x561fd41d62b0 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cbf678740 1 -- 172.21.15.94:0/1862033527 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd40c2fd0 con 0x561fd41df7b0 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cbf678740 10 client.0 _ll_register_callbacks cb 0x561fd4118fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cb659c640 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d62b0 0x561fd41df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cb5d9b640 1 -- 172.21.15.94:0/1862033527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41df7b0 msgr2=0x561fd41dc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cb5d9b640 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41df7b0 0x561fd41dc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cb5d9b640 1 -- 172.21.15.94:0/1862033527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d62b0 msgr2=0x561fd41df0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cb5d9b640 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d62b0 0x561fd41df0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cb5d9b640 1 -- 172.21.15.94:0/1862033527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb0009940 con 0x561fd41d6bd0 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.626+0000 7f9cb659c640 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d62b0 0x561fd41df0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.627+0000 7f9cb5d9b640 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d6bd0 0x561fd41dc460 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00c6a0 tx=0x7f9cac09ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781a3c7924896b10 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.627+0000 7f9c9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.627+0000 7f9c9dffb640 1 -- 172.21.15.94:0/1862033527 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac0cd040 con 0x561fd41d6bd0 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.627+0000 7f9c9dffb640 1 -- 172.21.15.94:0/1862033527 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9cac0a2f90 con 0x561fd41d6bd0 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.627+0000 7f9cbf678740 1 -- 172.21.15.94:0/1862033527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fd3e3d7e0 con 0x561fd41d6bd0 2024-02-17T12:00:40.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.627+0000 7f9c9dffb640 1 -- 172.21.15.94:0/1862033527 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cac0cbe60 con 0x561fd41d6bd0 2024-02-17T12:00:40.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.627+0000 7f9cbf678740 10 client.16149 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:40.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.627+0000 7f9cbf678740 1 -- 172.21.15.94:0/1862033527 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x561fd413c610 con 0x561fd41d6bd0 2024-02-17T12:00:40.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.627+0000 7f9c9dffb640 1 -- 172.21.15.94:0/1862033527 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561fd3e3d7e0 con 0x561fd41d6bd0 2024-02-17T12:00:40.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.629+0000 7f9c9dffb640 1 -- 172.21.15.94:0/1862033527 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9cac01f060 con 0x561fd41d6bd0 2024-02-17T12:00:40.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.629+0000 7f9cbf678740 10 client.16149 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:40.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.629+0000 7f9cbf678740 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:40.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.629+0000 7f9cbf678740 20 client.16149 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:40.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.629+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561fd41ef820 0x561fd41f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.629+0000 7f9cbf678740 4 client.16149 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:40.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.629+0000 7f9cbf678740 1 -- 172.21.15.94:0/1862033527 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fd3e7a720 con 0x561fd41ef820 2024-02-17T12:00:40.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.629+0000 7f9cb659c640 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561fd41ef820 0x561fd41f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.630+0000 7f9cb659c640 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561fd41ef820 0x561fd41f1c40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dce8a1eae63e551 server_cookie=5c5642dad538be in_seq=0 out_seq=0 2024-02-17T12:00:40.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.630+0000 7f9c9dffb640 10 client.16149 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:40.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9c9dffb640 1 -- 172.21.15.94:0/1862033527 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561fd3e7a720 con 0x561fd41ef820 2024-02-17T12:00:40.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9c9dffb640 10 client.16149 handle_command_reply: tid=0 2024-02-17T12:00:40.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9cbf678740 1 client.16149 shutdown 2024-02-17T12:00:40.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9cbf678740 1 -- 172.21.15.94:0/1862033527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d6bd0 msgr2=0x561fd41dc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:40.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d6bd0 0x561fd41dc460 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00c6a0 tx=0x7f9cac09ad40 comp rx=0 tx=0).stop 2024-02-17T12:00:40.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9cbf678740 1 -- 172.21.15.94:0/1862033527 shutdown_connections 2024-02-17T12:00:40.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561fd41ef820 0x561fd41f1c40 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41df7b0 0x561fd41dc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d6bd0 0x561fd41dc460 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1862033527 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d62b0 0x561fd41df0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9c9dffb640 0 client.16149 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:40.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9cbf678740 1 -- 172.21.15.94:0/1862033527 >> 172.21.15.94:0/1862033527 conn(0x561fd41d0990 msgr2=0x561fd41d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:40.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9cbf678740 1 -- 172.21.15.94:0/1862033527 shutdown_connections 2024-02-17T12:00:40.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9cbf678740 1 -- 172.21.15.94:0/1862033527 wait complete. 2024-02-17T12:00:40.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.631+0000 7f9cbf678740 20 client.16149 trim_cache size 0 max 16384 2024-02-17T12:00:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.652+0000 7f9cbf678740 1 Processor -- start 2024-02-17T12:00:40.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.652+0000 7f9cbf678740 1 -- start start 2024-02-17T12:00:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cbf678740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41d62b0 0x561fd41deb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cbf678740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d6bd0 0x561fd41df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cbf678740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41df7b0 0x561fd41e66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cbf678740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd41ec3a0 con 0x561fd41df7b0 2024-02-17T12:00:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cbf678740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd41cc0d0 con 0x561fd41d6bd0 2024-02-17T12:00:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cbf678740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd41d5630 con 0x561fd41d62b0 2024-02-17T12:00:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cb659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41d62b0 0x561fd41deb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cb659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41d62b0 0x561fd41deb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39648/0 (socket says 172.21.15.94:39648) 2024-02-17T12:00:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cb6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41df7b0 0x561fd41e66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cb659c640 1 -- 172.21.15.94:0/1278995000 learned_addr learned my addr 172.21.15.94:0/1278995000 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cb5d9b640 1 --2- 172.21.15.94:0/1278995000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d6bd0 0x561fd41df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cb659c640 1 -- 172.21.15.94:0/1278995000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d6bd0 msgr2=0x561fd41df0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cb659c640 1 --2- 172.21.15.94:0/1278995000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d6bd0 0x561fd41df0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cb659c640 1 -- 172.21.15.94:0/1278995000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41df7b0 msgr2=0x561fd41e66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cb659c640 1 --2- 172.21.15.94:0/1278995000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41df7b0 0x561fd41e66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cb659c640 1 -- 172.21.15.94:0/1278995000 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fd40d8ac0 con 0x561fd41d62b0 2024-02-17T12:00:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.653+0000 7f9cb6d9d640 1 --2- 172.21.15.94:0/1278995000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41df7b0 0x561fd41e66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.654+0000 7f9cb659c640 1 --2- 172.21.15.94:0/1278995000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41d62b0 0x561fd41deb90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9ca802c180 tx=0x7f9ca8047000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f19bce041179631f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.654+0000 7f9c9ffff640 1 -- 172.21.15.94:0/1278995000 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca802c510 con 0x561fd41d62b0 2024-02-17T12:00:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.654+0000 7f9c9ffff640 1 -- 172.21.15.94:0/1278995000 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ca802cb50 con 0x561fd41d62b0 2024-02-17T12:00:40.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.654+0000 7f9c9ffff640 1 -- 172.21.15.94:0/1278995000 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8032600 con 0x561fd41d62b0 2024-02-17T12:00:40.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 -- 172.21.15.94:0/1278995000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41d62b0 msgr2=0x561fd41deb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:40.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1278995000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41d62b0 0x561fd41deb90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9ca802c180 tx=0x7f9ca8047000 comp rx=0 tx=0).stop 2024-02-17T12:00:40.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 -- 172.21.15.94:0/1278995000 shutdown_connections 2024-02-17T12:00:40.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1278995000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41df7b0 0x561fd41e66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1278995000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41d6bd0 0x561fd41df0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 --2- 172.21.15.94:0/1278995000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41d62b0 0x561fd41deb90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 -- 172.21.15.94:0/1278995000 >> 172.21.15.94:0/1278995000 conn(0x561fd41d0990 msgr2=0x561fd41f65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 -- 172.21.15.94:0/1278995000 shutdown_connections 2024-02-17T12:00:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 -- 172.21.15.94:0/1278995000 wait complete. 2024-02-17T12:00:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 -- start start 2024-02-17T12:00:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 Processor -- start 2024-02-17T12:00:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d62b0 0x561fd41e6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41d6bd0 0x561fd41e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41df7b0 0x561fd41e2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd41e3330 con 0x561fd41d62b0 2024-02-17T12:00:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd41e34a0 con 0x561fd41df7b0 2024-02-17T12:00:40.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.655+0000 7f9cbf678740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd41e3660 con 0x561fd41d6bd0 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.656+0000 7f9cbf678740 10 client.0 _ll_register_callbacks cb 0x7f9cb0009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.656+0000 7f9cb6d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41df7b0 0x561fd41e2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.656+0000 7f9cb6d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41df7b0 0x561fd41e2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36696/0 (socket says 172.21.15.94:36696) 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.656+0000 7f9cb6d9d640 1 -- 172.21.15.94:0/3327796239 learned_addr learned my addr 172.21.15.94:0/3327796239 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.656+0000 7f9cb659c640 1 --2- 172.21.15.94:0/3327796239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d62b0 0x561fd41e6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.656+0000 7f9cb5d9b640 1 --2- 172.21.15.94:0/3327796239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41d6bd0 0x561fd41e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.656+0000 7f9cb659c640 1 -- 172.21.15.94:0/3327796239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41d6bd0 msgr2=0x561fd41e6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.656+0000 7f9cb659c640 1 --2- 172.21.15.94:0/3327796239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41d6bd0 0x561fd41e6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.656+0000 7f9cb659c640 1 -- 172.21.15.94:0/3327796239 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41df7b0 msgr2=0x561fd41e2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.656+0000 7f9cb659c640 1 --2- 172.21.15.94:0/3327796239 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41df7b0 0x561fd41e2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.656+0000 7f9cb659c640 1 -- 172.21.15.94:0/3327796239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fd40fc570 con 0x561fd41d62b0 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.657+0000 7f9cb659c640 1 --2- 172.21.15.94:0/3327796239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d62b0 0x561fd41e6660 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8037040 tx=0x7f9ca80538e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1e7a30e5593144 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.657+0000 7f9c9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:00:40.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.657+0000 7f9c9dffb640 1 -- 172.21.15.94:0/3327796239 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8065070 con 0x561fd41d62b0 2024-02-17T12:00:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.657+0000 7f9c9dffb640 1 -- 172.21.15.94:0/3327796239 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f9ca8053e40 con 0x561fd41d62b0 2024-02-17T12:00:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.657+0000 7f9cbf678740 1 -- 172.21.15.94:0/3327796239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cb0004c80 con 0x561fd41d62b0 2024-02-17T12:00:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.657+0000 7f9c9dffb640 1 -- 172.21.15.94:0/3327796239 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ca8034990 con 0x561fd41d62b0 2024-02-17T12:00:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.657+0000 7f9cbf678740 10 client.16155 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.657+0000 7f9cbf678740 1 -- 172.21.15.94:0/3327796239 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x7f9cb8114ec0 con 0x561fd41d62b0 2024-02-17T12:00:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.658+0000 7f9c9dffb640 1 -- 172.21.15.94:0/3327796239 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9cb0004c80 con 0x561fd41d62b0 2024-02-17T12:00:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.658+0000 7f9c9dffb640 1 -- 172.21.15.94:0/3327796239 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9ca8068030 con 0x561fd41d62b0 2024-02-17T12:00:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.658+0000 7f9cbf678740 10 client.16155 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.658+0000 7f9cbf678740 10 client.16155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.658+0000 7f9cbf678740 20 client.16155 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.658+0000 7f9cbf678740 1 --2- 172.21.15.94:0/3327796239 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561fd4246cf0 0x561fd42490e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.658+0000 7f9cbf678740 4 client.16155 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:00:40.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.658+0000 7f9cbf678740 1 -- 172.21.15.94:0/3327796239 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561fd3ec9890 con 0x561fd4246cf0 2024-02-17T12:00:40.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.658+0000 7f9cb5d9b640 1 --2- 172.21.15.94:0/3327796239 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561fd4246cf0 0x561fd42490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.659+0000 7f9c9dffb640 10 client.16155 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:40.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.659+0000 7f9cb5d9b640 1 --2- 172.21.15.94:0/3327796239 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561fd4246cf0 0x561fd42490e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=135049e7ab0728f server_cookie=6264e7fa13e1a970 in_seq=0 out_seq=0 2024-02-17T12:00:40.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.659+0000 7f9c9dffb640 1 -- 172.21.15.94:0/3327796239 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561fd3ec9890 con 0x561fd4246cf0 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.659+0000 7f9c9dffb640 10 client.16155 handle_command_reply: tid=0 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.659+0000 7f9cbf678740 1 client.16155 shutdown 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.660+0000 7f9cbf678740 1 -- 172.21.15.94:0/3327796239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d62b0 msgr2=0x561fd41e6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.660+0000 7f9cbf678740 1 --2- 172.21.15.94:0/3327796239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d62b0 0x561fd41e6660 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8037040 tx=0x7f9ca80538e0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.660+0000 7f9cbf678740 1 -- 172.21.15.94:0/3327796239 shutdown_connections 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.660+0000 7f9cbf678740 1 --2- 172.21.15.94:0/3327796239 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561fd4246cf0 0x561fd42490e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.660+0000 7f9cbf678740 1 --2- 172.21.15.94:0/3327796239 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd41df7b0 0x561fd41e2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.660+0000 7f9c9dffb640 0 client.16155 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.660+0000 7f9cbf678740 1 --2- 172.21.15.94:0/3327796239 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd41d6bd0 0x561fd41e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.660+0000 7f9cbf678740 1 --2- 172.21.15.94:0/3327796239 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd41d62b0 0x561fd41e6660 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.660+0000 7f9cbf678740 1 -- 172.21.15.94:0/3327796239 >> 172.21.15.94:0/3327796239 conn(0x561fd41d0990 msgr2=0x561fd41dac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.660+0000 7f9cbf678740 1 -- 172.21.15.94:0/3327796239 shutdown_connections 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.660+0000 7f9cbf678740 1 -- 172.21.15.94:0/3327796239 wait complete. 2024-02-17T12:00:40.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.661+0000 7f9cbf678740 20 client.16155 trim_cache size 0 max 16384 2024-02-17T12:00:40.664 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:00:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:00:40.664 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:00:40.664 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:00:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.661+0000 7f9cbca9d640 1 -- 172.21.15.94:0/579406797 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9ca00a2440 msgr2=0x7f9ca00a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.661+0000 7f9cbca9d640 1 --2- 172.21.15.94:0/579406797 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9ca00a2440 0x7f9ca00a4900 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009860 tx=0x7f9ca8035e60 comp rx=0 tx=0).stop 2024-02-17T12:00:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.661+0000 7f9cbca9d640 1 -- 172.21.15.94:0/579406797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb8100ab0 msgr2=0x7f9cb819e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.661+0000 7f9cbca9d640 1 --2- 172.21.15.94:0/579406797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb8100ab0 0x7f9cb819e370 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00e9c0 tx=0x7f9cac00ee80 comp rx=0 tx=0).stop 2024-02-17T12:00:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.661+0000 7f9cbca9d640 1 -- 172.21.15.94:0/579406797 shutdown_connections 2024-02-17T12:00:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.661+0000 7f9cbca9d640 1 --2- 172.21.15.94:0/579406797 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9ca00a2440 0x7f9ca00a4900 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.661+0000 7f9cbca9d640 1 --2- 172.21.15.94:0/579406797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cb8105590 0x7f9cb81a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.662+0000 7f9cbca9d640 1 --2- 172.21.15.94:0/579406797 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9cb8100ab0 0x7f9cb819e370 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.662+0000 7f9cbca9d640 1 --2- 172.21.15.94:0/579406797 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9cb8100190 0x7f9cb819de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.662+0000 7f9cbca9d640 1 -- 172.21.15.94:0/579406797 >> 172.21.15.94:0/579406797 conn(0x7f9cb8077210 msgr2=0x7f9cb8102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:40.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.662+0000 7f9cbca9d640 1 -- 172.21.15.94:0/579406797 shutdown_connections 2024-02-17T12:00:40.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.662+0000 7f9cbca9d640 1 -- 172.21.15.94:0/579406797 wait complete. 2024-02-17T12:00:40.676 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:00:40.677 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:00:40.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.829+0000 7fba1d240640 1 -- 172.21.15.94:0/3709094395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba18100c70 msgr2=0x7fba18104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:40.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.829+0000 7fba1d240640 1 --2- 172.21.15.94:0/3709094395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba18100c70 0x7fba18104cb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fba00009930 tx=0x7fba0002bc30 comp rx=0 tx=0).stop 2024-02-17T12:00:40.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.829+0000 7fba1d240640 1 -- 172.21.15.94:0/3709094395 shutdown_connections 2024-02-17T12:00:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.829+0000 7fba1d240640 1 --2- 172.21.15.94:0/3709094395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba18105390 0x7fba1810cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.829+0000 7fba1d240640 1 --2- 172.21.15.94:0/3709094395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba18100c70 0x7fba18104cb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.829+0000 7fba1d240640 1 --2- 172.21.15.94:0/3709094395 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba181002c0 0x7fba181006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.829+0000 7fba1d240640 1 -- 172.21.15.94:0/3709094395 >> 172.21.15.94:0/3709094395 conn(0x7fba18076400 msgr2=0x7fba18076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.829+0000 7fba1d240640 1 -- 172.21.15.94:0/3709094395 shutdown_connections 2024-02-17T12:00:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.829+0000 7fba1d240640 1 -- 172.21.15.94:0/3709094395 wait complete. 2024-02-17T12:00:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.830+0000 7fba1d240640 1 Processor -- start 2024-02-17T12:00:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.830+0000 7fba1d240640 1 -- start start 2024-02-17T12:00:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.830+0000 7fba1d240640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba181002c0 0x7fba18196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.830+0000 7fba1d240640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba18105390 0x7fba18196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.830+0000 7fba16d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba181002c0 0x7fba18196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.830+0000 7fba16d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba181002c0 0x7fba18196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39690/0 (socket says 172.21.15.94:39690) 2024-02-17T12:00:40.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.830+0000 7fba1d240640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba18196ee0 0x7fba1819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.830+0000 7fba1d240640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1819bb30 con 0x7fba18105390 2024-02-17T12:00:40.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.830+0000 7fba1d240640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1819bca0 con 0x7fba18196ee0 2024-02-17T12:00:40.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.830+0000 7fba1d240640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1819be60 con 0x7fba181002c0 2024-02-17T12:00:40.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.831+0000 7fba16d9d640 1 -- 172.21.15.94:0/159146367 learned_addr learned my addr 172.21.15.94:0/159146367 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:40.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.831+0000 7fba1759e640 1 --2- 172.21.15.94:0/159146367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba18196ee0 0x7fba1819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.831+0000 7fba16d9d640 1 -- 172.21.15.94:0/159146367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba18196ee0 msgr2=0x7fba1819b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:40.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.831+0000 7fba16d9d640 1 --2- 172.21.15.94:0/159146367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba18196ee0 0x7fba1819b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.831+0000 7fba16d9d640 1 -- 172.21.15.94:0/159146367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba18105390 msgr2=0x7fba18196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:40.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.831+0000 7fba16d9d640 1 --2- 172.21.15.94:0/159146367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba18105390 0x7fba18196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:40.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.831+0000 7fba16d9d640 1 -- 172.21.15.94:0/159146367 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba00009560 con 0x7fba181002c0 2024-02-17T12:00:40.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.831+0000 7fba16d9d640 1 --2- 172.21.15.94:0/159146367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba181002c0 0x7fba18196410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba0800d950 tx=0x7fba0800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:40.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.832+0000 7fba14d99640 1 -- 172.21.15.94:0/159146367 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba08009a30 con 0x7fba181002c0 2024-02-17T12:00:40.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.832+0000 7fba14d99640 1 -- 172.21.15.94:0/159146367 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fba08009b90 con 0x7fba181002c0 2024-02-17T12:00:40.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.832+0000 7fba14d99640 1 -- 172.21.15.94:0/159146367 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba08004a00 con 0x7fba181002c0 2024-02-17T12:00:40.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.832+0000 7fba1d240640 1 -- 172.21.15.94:0/159146367 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1819c140 con 0x7fba181002c0 2024-02-17T12:00:40.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.832+0000 7fba1d240640 1 -- 172.21.15.94:0/159146367 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1819c5a0 con 0x7fba181002c0 2024-02-17T12:00:40.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.833+0000 7fba1d240640 1 -- 172.21.15.94:0/159146367 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba18101790 con 0x7fba181002c0 2024-02-17T12:00:40.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.834+0000 7fba14d99640 1 -- 172.21.15.94:0/159146367 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fba08009d00 con 0x7fba181002c0 2024-02-17T12:00:40.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.835+0000 7fba14d99640 1 --2- 172.21.15.94:0/159146367 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fba040a2330 0x7fba040a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:40.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.835+0000 7fba14d99640 1 -- 172.21.15.94:0/159146367 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fba080cdb70 con 0x7fba181002c0 2024-02-17T12:00:40.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.835+0000 7fba0ffff640 1 --2- 172.21.15.94:0/159146367 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fba040a2330 0x7fba040a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:40.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.836+0000 7fba0ffff640 1 --2- 172.21.15.94:0/159146367 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fba040a2330 0x7fba040a47f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fba00036040 tx=0x7fba00000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:40.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:40.840+0000 7fba14d99640 1 -- 172.21.15.94:0/159146367 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fba08096790 con 0x7fba181002c0 2024-02-17T12:00:41.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.163+0000 7fba1d240640 1 -- 172.21.15.94:0/159146367 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba180683c0 con 0x7fba181002c0 2024-02-17T12:00:41.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.164+0000 7fba14d99640 1 -- 172.21.15.94:0/159146367 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11346 (secure 0 0 0) 0x7fba08099e50 con 0x7fba181002c0 2024-02-17T12:00:41.163 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:00:41.163 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:57:51.854453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:00:41.163 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 21 2024-02-17T12:00:41.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.166+0000 7fba1d240640 1 -- 172.21.15.94:0/159146367 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fba040a2330 msgr2=0x7fba040a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.166+0000 7fba1d240640 1 --2- 172.21.15.94:0/159146367 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fba040a2330 0x7fba040a47f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fba00036040 tx=0x7fba00000bc0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.166+0000 7fba1d240640 1 -- 172.21.15.94:0/159146367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba181002c0 msgr2=0x7fba18196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.166+0000 7fba1d240640 1 --2- 172.21.15.94:0/159146367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba181002c0 0x7fba18196410 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fba0800d950 tx=0x7fba0800de10 comp rx=0 tx=0).stop 2024-02-17T12:00:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.166+0000 7fba1d240640 1 -- 172.21.15.94:0/159146367 shutdown_connections 2024-02-17T12:00:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.166+0000 7fba1d240640 1 --2- 172.21.15.94:0/159146367 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fba040a2330 0x7fba040a47f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.166+0000 7fba1d240640 1 --2- 172.21.15.94:0/159146367 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fba18196ee0 0x7fba1819b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.166+0000 7fba1d240640 1 --2- 172.21.15.94:0/159146367 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fba18105390 0x7fba18196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.166+0000 7fba1d240640 1 --2- 172.21.15.94:0/159146367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba181002c0 0x7fba18196410 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.166+0000 7fba1d240640 1 -- 172.21.15.94:0/159146367 >> 172.21.15.94:0/159146367 conn(0x7fba18076400 msgr2=0x7fba180fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.166+0000 7fba1d240640 1 -- 172.21.15.94:0/159146367 shutdown_connections 2024-02-17T12:00:41.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.166+0000 7fba1d240640 1 -- 172.21.15.94:0/159146367 wait complete. 2024-02-17T12:00:41.180 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:00:41.180 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:00:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.334+0000 7f61404ff640 1 -- 172.21.15.94:0/608206831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61381130d0 msgr2=0x7f61381154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.334+0000 7f61404ff640 1 --2- 172.21.15.94:0/608206831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61381130d0 0x7f61381154c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f613000bd70 tx=0x7f613002dd80 comp rx=0 tx=0).stop 2024-02-17T12:00:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.335+0000 7f61404ff640 1 -- 172.21.15.94:0/608206831 shutdown_connections 2024-02-17T12:00:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.335+0000 7f61404ff640 1 --2- 172.21.15.94:0/608206831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61381130d0 0x7f61381154c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.335+0000 7f61404ff640 1 --2- 172.21.15.94:0/608206831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f613810a440 0x7f6138112b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.335+0000 7f61404ff640 1 --2- 172.21.15.94:0/608206831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6138109b20 0x7f6138109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.335+0000 7f61404ff640 1 -- 172.21.15.94:0/608206831 >> 172.21.15.94:0/608206831 conn(0x7f613806c260 msgr2=0x7f613806c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.335+0000 7f61404ff640 1 -- 172.21.15.94:0/608206831 shutdown_connections 2024-02-17T12:00:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.335+0000 7f61404ff640 1 -- 172.21.15.94:0/608206831 wait complete. 2024-02-17T12:00:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f61404ff640 1 Processor -- start 2024-02-17T12:00:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f61404ff640 1 -- start start 2024-02-17T12:00:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f61404ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6138109b20 0x7f61381ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f61404ff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f613810a440 0x7f61381ac920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f61404ff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61381130d0 0x7f6138111550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f61404ff640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6138111bf0 con 0x7f61381130d0 2024-02-17T12:00:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f61404ff640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6138111d60 con 0x7f6138109b20 2024-02-17T12:00:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f61404ff640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6138111f20 con 0x7f613810a440 2024-02-17T12:00:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f613e29b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6138109b20 0x7f61381ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f613ea9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61381130d0 0x7f6138111550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f613e29b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6138109b20 0x7f61381ac3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36730/0 (socket says 172.21.15.94:36730) 2024-02-17T12:00:41.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f613e29b640 1 -- 172.21.15.94:0/3695708798 learned_addr learned my addr 172.21.15.94:0/3695708798 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f613da9a640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f613810a440 0x7f61381ac920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f613e29b640 1 -- 172.21.15.94:0/3695708798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f613810a440 msgr2=0x7f61381ac920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f613e29b640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f613810a440 0x7f61381ac920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f613e29b640 1 -- 172.21.15.94:0/3695708798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61381130d0 msgr2=0x7f6138111550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f613e29b640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61381130d0 0x7f6138111550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f613e29b640 1 -- 172.21.15.94:0/3695708798 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6128009560 con 0x7f6138109b20 2024-02-17T12:00:41.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.336+0000 7f613ea9c640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61381130d0 0x7f6138111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:41.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.337+0000 7f613e29b640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6138109b20 0x7f61381ac3e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f613400d770 tx=0x7f613400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.337+0000 7f612f7fe640 1 -- 172.21.15.94:0/3695708798 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134009c30 con 0x7f6138109b20 2024-02-17T12:00:41.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.337+0000 7f61404ff640 1 -- 172.21.15.94:0/3695708798 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613000ba50 con 0x7f6138109b20 2024-02-17T12:00:41.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.337+0000 7f612f7fe640 1 -- 172.21.15.94:0/3695708798 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f613400de80 con 0x7f6138109b20 2024-02-17T12:00:41.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.337+0000 7f612f7fe640 1 -- 172.21.15.94:0/3695708798 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134004e80 con 0x7f6138109b20 2024-02-17T12:00:41.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.337+0000 7f61404ff640 1 -- 172.21.15.94:0/3695708798 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61381b31d0 con 0x7f6138109b20 2024-02-17T12:00:41.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.340+0000 7f61431aa740 1 Processor -- start 2024-02-17T12:00:41.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.340+0000 7f612f7fe640 1 -- 172.21.15.94:0/3695708798 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6134005020 con 0x7f6138109b20 2024-02-17T12:00:41.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f61431aa740 1 -- start start 2024-02-17T12:00:41.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f61431aa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649efc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f61431aa740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f05210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f61431aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f07b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f61431aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555649eb7960 con 0x555649f05750 2024-02-17T12:00:41.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f61431aa740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555649e6b310 con 0x555649efcb70 2024-02-17T12:00:41.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f61431aa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555649e68840 con 0x555649efc250 2024-02-17T12:00:41.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f613e29b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f05210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f613da9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f07b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f613ea9c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649efc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f613e29b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f05210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36740/0 (socket says 172.21.15.94:36740) 2024-02-17T12:00:41.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f613e29b640 1 -- 172.21.15.94:0/3122894491 learned_addr learned my addr 172.21.15.94:0/3122894491 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f613da9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f07b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43480/0 (socket says 172.21.15.94:43480) 2024-02-17T12:00:41.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.341+0000 7f612f7fe640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f61240a2270 0x7f61240a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.342+0000 7f613ea9c640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f61240a2270 0x7f61240a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.342+0000 7f613ea9c640 1 -- 172.21.15.94:0/3122894491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 msgr2=0x555649f05210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.342+0000 7f612f7fe640 1 -- 172.21.15.94:0/3695708798 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f61340cda60 con 0x7f6138109b20 2024-02-17T12:00:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.342+0000 7f613ea9c640 1 --2- 172.21.15.94:0/3122894491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f05210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.342+0000 7f613ea9c640 1 -- 172.21.15.94:0/3122894491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 msgr2=0x555649f07b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.342+0000 7f613ea9c640 1 --2- 172.21.15.94:0/3122894491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f07b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.342+0000 7f613ea9c640 1 -- 172.21.15.94:0/3122894491 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555649e26060 con 0x555649efc250 2024-02-17T12:00:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.342+0000 7f613da9a640 1 --2- 172.21.15.94:0/3122894491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f07b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:00:41.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.342+0000 7f613ea9c640 1 --2- 172.21.15.94:0/3122894491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649efc630 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f613000bd40 tx=0x7f6130036f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ee652f9cb0b5235 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.343+0000 7f613ea9c640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f61240a2270 0x7f61240a4730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f613004a000 tx=0x7f6130049000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.343+0000 7f612cbfe640 1 -- 172.21.15.94:0/3122894491 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130036a00 con 0x555649efc250 2024-02-17T12:00:41.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.343+0000 7f612cbfe640 1 -- 172.21.15.94:0/3122894491 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6130049070 con 0x555649efc250 2024-02-17T12:00:41.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.343+0000 7f612cbfe640 1 -- 172.21.15.94:0/3122894491 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6130045480 con 0x555649efc250 2024-02-17T12:00:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.344+0000 7f61431aa740 1 -- 172.21.15.94:0/3122894491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 msgr2=0x555649efc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.344+0000 7f61431aa740 1 --2- 172.21.15.94:0/3122894491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649efc630 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f613000bd40 tx=0x7f6130036f60 comp rx=0 tx=0).stop 2024-02-17T12:00:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.344+0000 7f61431aa740 1 -- 172.21.15.94:0/3122894491 shutdown_connections 2024-02-17T12:00:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.344+0000 7f61431aa740 1 --2- 172.21.15.94:0/3122894491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f07b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.344+0000 7f61431aa740 1 --2- 172.21.15.94:0/3122894491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f05210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.344+0000 7f61431aa740 1 --2- 172.21.15.94:0/3122894491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649efc630 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.344+0000 7f61431aa740 1 -- 172.21.15.94:0/3122894491 >> 172.21.15.94:0/3122894491 conn(0x555649ef6980 msgr2=0x555649ef6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.344+0000 7f61431aa740 1 -- 172.21.15.94:0/3122894491 shutdown_connections 2024-02-17T12:00:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.344+0000 7f61431aa740 1 -- 172.21.15.94:0/3122894491 wait complete. 2024-02-17T12:00:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.344+0000 7f61431aa740 1 -- start start 2024-02-17T12:00:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.344+0000 7f61431aa740 1 Processor -- start 2024-02-17T12:00:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f61431aa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649f02810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f61431aa740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f02d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f61431aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f03290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f61431aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555649e64020 con 0x555649f05750 2024-02-17T12:00:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f61431aa740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555649b37430 con 0x555649efcb70 2024-02-17T12:00:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f61431aa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555649af3e00 con 0x555649efc250 2024-02-17T12:00:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f61431aa740 10 client.0 _ll_register_callbacks cb 0x555649e0bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f613da9a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f02d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f613da9a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f02d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36756/0 (socket says 172.21.15.94:36756) 2024-02-17T12:00:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f613e29b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649f02810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f613da9a640 1 -- 172.21.15.94:0/1592339262 learned_addr learned my addr 172.21.15.94:0/1592339262 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f613ea9c640 1 --2- 172.21.15.94:0/1592339262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f03290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f613da9a640 1 -- 172.21.15.94:0/1592339262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 msgr2=0x555649f02810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f613da9a640 1 --2- 172.21.15.94:0/1592339262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649f02810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f613da9a640 1 -- 172.21.15.94:0/1592339262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 msgr2=0x555649f03290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f613da9a640 1 --2- 172.21.15.94:0/1592339262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f03290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f613da9a640 1 -- 172.21.15.94:0/1592339262 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6138115f20 con 0x555649efcb70 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f613e29b640 1 --2- 172.21.15.94:0/1592339262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649f02810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f613da9a640 1 --2- 172.21.15.94:0/1592339262 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f02d50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6128011790 tx=0x7f6128011b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e623d0c8da5bac4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f6121ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f6121ffb640 1 -- 172.21.15.94:0/1592339262 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128016070 con 0x555649efcb70 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.345+0000 7f6121ffb640 1 -- 172.21.15.94:0/1592339262 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f612800fbb0 con 0x555649efcb70 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.346+0000 7f61431aa740 1 -- 172.21.15.94:0/1592339262 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555649b637e0 con 0x555649efcb70 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.347+0000 7f6121ffb640 1 -- 172.21.15.94:0/1592339262 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f612800fd70 con 0x555649efcb70 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.347+0000 7f61431aa740 10 client.25619 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.347+0000 7f61431aa740 1 -- 172.21.15.94:0/1592339262 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x555649e626a0 con 0x555649efcb70 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.347+0000 7f6121ffb640 1 -- 172.21.15.94:0/1592339262 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6128005500 con 0x555649efcb70 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.347+0000 7f6121ffb640 1 -- 172.21.15.94:0/1592339262 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f61280233f0 con 0x555649efcb70 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.347+0000 7f61431aa740 10 client.25619 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.347+0000 7f61431aa740 10 client.25619 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:00:41.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.347+0000 7f61431aa740 20 client.25619 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:41.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.347+0000 7f61431aa740 1 --2- 172.21.15.94:0/1592339262 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555649f15c60 0x555649f180a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.347+0000 7f61431aa740 4 client.25619 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:41.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.347+0000 7f61431aa740 1 -- 172.21.15.94:0/1592339262 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555649b637e0 con 0x555649f15c60 2024-02-17T12:00:41.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.348+0000 7f613e29b640 1 --2- 172.21.15.94:0/1592339262 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555649f15c60 0x555649f180a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.348+0000 7f6121ffb640 10 client.25619 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:00:41.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.348+0000 7f613e29b640 1 --2- 172.21.15.94:0/1592339262 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555649f15c60 0x555649f180a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd9540722d220841 server_cookie=453dcd75a401d216 in_seq=0 out_seq=0 2024-02-17T12:00:41.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.349+0000 7f6121ffb640 1 -- 172.21.15.94:0/1592339262 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555649b637e0 con 0x555649f15c60 2024-02-17T12:00:41.352 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.349+0000 7f6121ffb640 10 client.25619 handle_command_reply: tid=0 2024-02-17T12:00:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.349+0000 7f61431aa740 1 client.25619 shutdown 2024-02-17T12:00:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.349+0000 7f61431aa740 1 -- 172.21.15.94:0/1592339262 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 msgr2=0x555649f02d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.349+0000 7f61431aa740 1 --2- 172.21.15.94:0/1592339262 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f02d50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6128011790 tx=0x7f6128011b90 comp rx=0 tx=0).stop 2024-02-17T12:00:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.349+0000 7f61431aa740 1 -- 172.21.15.94:0/1592339262 shutdown_connections 2024-02-17T12:00:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.349+0000 7f61431aa740 1 --2- 172.21.15.94:0/1592339262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f03290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.349+0000 7f61431aa740 1 --2- 172.21.15.94:0/1592339262 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555649f15c60 0x555649f180a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.349+0000 7f61431aa740 1 --2- 172.21.15.94:0/1592339262 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f02d50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.349+0000 7f61431aa740 1 --2- 172.21.15.94:0/1592339262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649f02810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.349+0000 7f61431aa740 1 -- 172.21.15.94:0/1592339262 >> 172.21.15.94:0/1592339262 conn(0x555649ef6980 msgr2=0x555649ef86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.349+0000 7f6121ffb640 0 client.25619 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:00:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.349+0000 7f61431aa740 1 -- 172.21.15.94:0/1592339262 shutdown_connections 2024-02-17T12:00:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.350+0000 7f61431aa740 1 -- 172.21.15.94:0/1592339262 wait complete. 2024-02-17T12:00:41.353 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.350+0000 7f61431aa740 20 client.25619 trim_cache size 0 max 16384 2024-02-17T12:00:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.367+0000 7f61431aa740 1 Processor -- start 2024-02-17T12:00:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.367+0000 7f61431aa740 1 -- start start 2024-02-17T12:00:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.367+0000 7f61431aa740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efc250 0x555649f04f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.367+0000 7f61431aa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efcb70 0x555649f030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.367+0000 7f61431aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649effa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.367+0000 7f61431aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555649de9130 con 0x555649f05750 2024-02-17T12:00:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.367+0000 7f61431aa740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555649b637e0 con 0x555649efc250 2024-02-17T12:00:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.367+0000 7f61431aa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555649efbef0 con 0x555649efcb70 2024-02-17T12:00:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.367+0000 7f613e29b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efc250 0x555649f04f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.367+0000 7f613ea9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649effa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.367+0000 7f613e29b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efc250 0x555649f04f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36762/0 (socket says 172.21.15.94:36762) 2024-02-17T12:00:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.367+0000 7f613e29b640 1 -- 172.21.15.94:0/491397934 learned_addr learned my addr 172.21.15.94:0/491397934 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.367+0000 7f613da9a640 1 --2- 172.21.15.94:0/491397934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efcb70 0x555649f030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.368+0000 7f613e29b640 1 -- 172.21.15.94:0/491397934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efcb70 msgr2=0x555649f030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.368+0000 7f613e29b640 1 --2- 172.21.15.94:0/491397934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efcb70 0x555649f030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.368+0000 7f613e29b640 1 -- 172.21.15.94:0/491397934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 msgr2=0x555649effa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.368+0000 7f613e29b640 1 --2- 172.21.15.94:0/491397934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649effa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.368+0000 7f613e29b640 1 -- 172.21.15.94:0/491397934 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555649dfec20 con 0x555649efc250 2024-02-17T12:00:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.368+0000 7f613e29b640 1 --2- 172.21.15.94:0/491397934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efc250 0x555649f04f40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6134009610 tx=0x7f61340cff30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7c912b0c4b367ff server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.368+0000 7f6123fff640 1 -- 172.21.15.94:0/491397934 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134099e50 con 0x555649efc250 2024-02-17T12:00:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.368+0000 7f6123fff640 1 -- 172.21.15.94:0/491397934 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f613409a4f0 con 0x555649efc250 2024-02-17T12:00:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.368+0000 7f6123fff640 1 -- 172.21.15.94:0/491397934 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134098880 con 0x555649efc250 2024-02-17T12:00:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.369+0000 7f61431aa740 1 -- 172.21.15.94:0/491397934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efc250 msgr2=0x555649f04f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.369+0000 7f61431aa740 1 --2- 172.21.15.94:0/491397934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efc250 0x555649f04f40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6134009610 tx=0x7f61340cff30 comp rx=0 tx=0).stop 2024-02-17T12:00:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.369+0000 7f61431aa740 1 -- 172.21.15.94:0/491397934 shutdown_connections 2024-02-17T12:00:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.369+0000 7f61431aa740 1 --2- 172.21.15.94:0/491397934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649effa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.369+0000 7f61431aa740 1 --2- 172.21.15.94:0/491397934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efcb70 0x555649f030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.369+0000 7f61431aa740 1 --2- 172.21.15.94:0/491397934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efc250 0x555649f04f40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.369+0000 7f61431aa740 1 -- 172.21.15.94:0/491397934 >> 172.21.15.94:0/491397934 conn(0x555649ef6980 msgr2=0x555649f010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.369+0000 7f61431aa740 1 -- 172.21.15.94:0/491397934 shutdown_connections 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.369+0000 7f61431aa740 1 -- 172.21.15.94:0/491397934 wait complete. 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.369+0000 7f61431aa740 1 -- start start 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.369+0000 7f61431aa740 1 Processor -- start 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f61431aa740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649f0ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f61431aa740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f0ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f61431aa740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f0b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f61431aa740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555649f0ba90 con 0x555649f05750 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f61431aa740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555649f0bc00 con 0x555649efcb70 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f61431aa740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555649f6c1d0 con 0x555649efc250 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f61431aa740 10 client.0 _ll_register_callbacks cb 0x7f61381ad040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f613ea9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f0b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f613ea9c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f0b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43510/0 (socket says 172.21.15.94:43510) 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f613da9a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f0ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f613ea9c640 1 -- 172.21.15.94:0/956909643 learned_addr learned my addr 172.21.15.94:0/956909643 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f613e29b640 1 --2- 172.21.15.94:0/956909643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649f0ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f613da9a640 1 -- 172.21.15.94:0/956909643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 msgr2=0x555649f0ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f613da9a640 1 --2- 172.21.15.94:0/956909643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649f0ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f613da9a640 1 -- 172.21.15.94:0/956909643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 msgr2=0x555649f0b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f613da9a640 1 --2- 172.21.15.94:0/956909643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f0b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f613da9a640 1 -- 172.21.15.94:0/956909643 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61381b31d0 con 0x555649efcb70 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f613da9a640 1 --2- 172.21.15.94:0/956909643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f0ae10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6128023820 tx=0x7f6128023620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea2c6cccec881c99 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.370+0000 7f6121ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f6121ffb640 1 -- 172.21.15.94:0/956909643 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128016040 con 0x555649efcb70 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f61431aa740 1 -- 172.21.15.94:0/956909643 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6130049070 con 0x555649efcb70 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f6121ffb640 1 -- 172.21.15.94:0/956909643 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6128023e40 con 0x555649efcb70 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f6121ffb640 1 -- 172.21.15.94:0/956909643 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6128011030 con 0x555649efcb70 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f61431aa740 10 client.25625 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:41.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f61431aa740 1 -- 172.21.15.94:0/956909643 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x555649e248c0 con 0x555649efcb70 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f6121ffb640 1 -- 172.21.15.94:0/956909643 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6128038020 con 0x555649efcb70 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f6121ffb640 1 -- 172.21.15.94:0/956909643 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6128033070 con 0x555649efcb70 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f61431aa740 10 client.25625 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f61431aa740 10 client.25625 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f61431aa740 20 client.25625 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f61431aa740 1 --2- 172.21.15.94:0/956909643 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555649f6da40 0x555649f6fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f61431aa740 4 client.25625 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f61431aa740 1 -- 172.21.15.94:0/956909643 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555649ba0720 con 0x555649f6da40 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.371+0000 7f613e29b640 1 --2- 172.21.15.94:0/956909643 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555649f6da40 0x555649f6fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f6121ffb640 10 client.25625 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f613e29b640 1 --2- 172.21.15.94:0/956909643 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555649f6da40 0x555649f6fe60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6892481b1568021 server_cookie=dc2daa3b0352d864 in_seq=0 out_seq=0 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f6121ffb640 1 -- 172.21.15.94:0/956909643 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555649ba0720 con 0x555649f6da40 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f6121ffb640 10 client.25625 handle_command_reply: tid=0 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f61431aa740 1 client.25625 shutdown 2024-02-17T12:00:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f61431aa740 1 -- 172.21.15.94:0/956909643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 msgr2=0x555649f0ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f61431aa740 1 --2- 172.21.15.94:0/956909643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f0ae10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6128023820 tx=0x7f6128023620 comp rx=0 tx=0).stop 2024-02-17T12:00:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f61431aa740 1 -- 172.21.15.94:0/956909643 shutdown_connections 2024-02-17T12:00:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f61431aa740 1 --2- 172.21.15.94:0/956909643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555649f05750 0x555649f0b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f61431aa740 1 --2- 172.21.15.94:0/956909643 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555649f6da40 0x555649f6fe60 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f61431aa740 1 --2- 172.21.15.94:0/956909643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555649efcb70 0x555649f0ae10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f6121ffb640 0 client.25625 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:00:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f61431aa740 1 --2- 172.21.15.94:0/956909643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555649efc250 0x555649f0ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f61431aa740 1 -- 172.21.15.94:0/956909643 >> 172.21.15.94:0/956909643 conn(0x555649ef6980 msgr2=0x555649ef7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f61431aa740 1 -- 172.21.15.94:0/956909643 shutdown_connections 2024-02-17T12:00:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f61431aa740 1 -- 172.21.15.94:0/956909643 wait complete. 2024-02-17T12:00:41.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.372+0000 7f61431aa740 20 client.25625 trim_cache size 0 max 16384 2024-02-17T12:00:41.379 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:00:41.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.373+0000 7f61404ff640 1 -- 172.21.15.94:0/3695708798 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f61240a2270 msgr2=0x7f61240a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.373+0000 7f61404ff640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f61240a2270 0x7f61240a4730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f613004a000 tx=0x7f6130049000 comp rx=0 tx=0).stop 2024-02-17T12:00:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.373+0000 7f61404ff640 1 -- 172.21.15.94:0/3695708798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6138109b20 msgr2=0x7f61381ac3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.373+0000 7f61404ff640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6138109b20 0x7f61381ac3e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f613400d770 tx=0x7f613400dc30 comp rx=0 tx=0).stop 2024-02-17T12:00:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.373+0000 7f61404ff640 1 -- 172.21.15.94:0/3695708798 shutdown_connections 2024-02-17T12:00:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.373+0000 7f61404ff640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f61240a2270 0x7f61240a4730 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.373+0000 7f61404ff640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f61381130d0 0x7f6138111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.373+0000 7f61404ff640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f613810a440 0x7f61381ac920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.373+0000 7f61404ff640 1 --2- 172.21.15.94:0/3695708798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6138109b20 0x7f61381ac3e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.373+0000 7f61404ff640 1 -- 172.21.15.94:0/3695708798 >> 172.21.15.94:0/3695708798 conn(0x7f613806c260 msgr2=0x7f6138117c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.374+0000 7f61404ff640 1 -- 172.21.15.94:0/3695708798 shutdown_connections 2024-02-17T12:00:41.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.374+0000 7f61404ff640 1 -- 172.21.15.94:0/3695708798 wait complete. 2024-02-17T12:00:41.389 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:00:41.390 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:00:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.545+0000 7f00b805b640 1 -- 172.21.15.94:0/1156322473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b0103590 msgr2=0x7f00b0103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.545+0000 7f00b805b640 1 --2- 172.21.15.94:0/1156322473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b0103590 0x7f00b0103970 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009930 tx=0x7f00a002bc30 comp rx=0 tx=0).stop 2024-02-17T12:00:41.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.545+0000 7f00b805b640 1 -- 172.21.15.94:0/1156322473 shutdown_connections 2024-02-17T12:00:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.545+0000 7f00b805b640 1 --2- 172.21.15.94:0/1156322473 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00b0108660 0x7f00b0110230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.545+0000 7f00b805b640 1 --2- 172.21.15.94:0/1156322473 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00b0103f40 0x7f00b0107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.545+0000 7f00b805b640 1 --2- 172.21.15.94:0/1156322473 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b0103590 0x7f00b0103970 secure :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009930 tx=0x7f00a002bc30 comp rx=0 tx=0).stop 2024-02-17T12:00:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.545+0000 7f00b805b640 1 -- 172.21.15.94:0/1156322473 >> 172.21.15.94:0/1156322473 conn(0x7f00b00fedd0 msgr2=0x7f00b01011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.545+0000 7f00b805b640 1 -- 172.21.15.94:0/1156322473 shutdown_connections 2024-02-17T12:00:41.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.546+0000 7f00b805b640 1 -- 172.21.15.94:0/1156322473 wait complete. 2024-02-17T12:00:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.546+0000 7f00b805b640 1 Processor -- start 2024-02-17T12:00:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.546+0000 7f00b805b640 1 -- start start 2024-02-17T12:00:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b805b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00b0108660 0x7f00b010edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b805b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b010f2f0 0x7f00b010f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b805b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00b01b0860 0x7f00b01b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b805b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b01b3190 con 0x7f00b010f2f0 2024-02-17T12:00:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b805b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b01b32d0 con 0x7f00b0108660 2024-02-17T12:00:41.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b805b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b01b3410 con 0x7f00b01b0860 2024-02-17T12:00:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b5df7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00b0108660 0x7f00b010edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b5df7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00b0108660 0x7f00b010edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36790/0 (socket says 172.21.15.94:36790) 2024-02-17T12:00:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b55f6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b010f2f0 0x7f00b010f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b5df7640 1 -- 172.21.15.94:0/941548708 learned_addr learned my addr 172.21.15.94:0/941548708 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b65f8640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00b01b0860 0x7f00b01b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b5df7640 1 -- 172.21.15.94:0/941548708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00b01b0860 msgr2=0x7f00b01b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b5df7640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00b01b0860 0x7f00b01b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b5df7640 1 -- 172.21.15.94:0/941548708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b010f2f0 msgr2=0x7f00b010f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b5df7640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b010f2f0 0x7f00b010f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b5df7640 1 -- 172.21.15.94:0/941548708 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a0009560 con 0x7f00b0108660 2024-02-17T12:00:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b65f8640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00b01b0860 0x7f00b01b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b5df7640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00b0108660 0x7f00b010edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009900 tx=0x7f00a0004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00a6ffd640 1 -- 172.21.15.94:0/941548708 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0004440 con 0x7f00b0108660 2024-02-17T12:00:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00b805b640 1 -- 172.21.15.94:0/941548708 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b01b3690 con 0x7f00b0108660 2024-02-17T12:00:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00a6ffd640 1 -- 172.21.15.94:0/941548708 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00a00052d0 con 0x7f00b0108660 2024-02-17T12:00:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.548+0000 7f00b805b640 1 -- 172.21.15.94:0/941548708 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b01b3b70 con 0x7f00b0108660 2024-02-17T12:00:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.547+0000 7f00a6ffd640 1 -- 172.21.15.94:0/941548708 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0003d10 con 0x7f00b0108660 2024-02-17T12:00:41.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.551+0000 7f00a6ffd640 1 -- 172.21.15.94:0/941548708 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f00a0005440 con 0x7f00b0108660 2024-02-17T12:00:41.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.551+0000 7f00bac2a740 1 Processor -- start 2024-02-17T12:00:41.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.551+0000 7f00bac2a740 1 -- start start 2024-02-17T12:00:41.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.551+0000 7f00a6ffd640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f009c0a2230 0x7f009c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.551+0000 7f00bac2a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6250 0x55ebbd7e6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.551+0000 7f00bac2a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ef210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.551+0000 7f00bac2a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7ef750 0x55ebbd7f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.551+0000 7f00bac2a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbd7a1960 con 0x55ebbd7e6250 2024-02-17T12:00:41.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.551+0000 7f00bac2a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbd755310 con 0x55ebbd7ef750 2024-02-17T12:00:41.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.551+0000 7f00bac2a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbd752840 con 0x55ebbd7e6b70 2024-02-17T12:00:41.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b5df7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6250 0x55ebbd7e6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b65f8640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f009c0a2230 0x7f009c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b5df7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6250 0x55ebbd7e6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43532/0 (socket says 172.21.15.94:43532) 2024-02-17T12:00:41.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b5df7640 1 -- 172.21.15.94:0/792522477 learned_addr learned my addr 172.21.15.94:0/792522477 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00a6ffd640 1 -- 172.21.15.94:0/941548708 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f00a00f0670 con 0x7f00b0108660 2024-02-17T12:00:41.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b65f8640 1 --2- 172.21.15.94:0/792522477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7ef750 0x55ebbd7f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b55f6640 1 --2- 172.21.15.94:0/792522477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b5df7640 1 -- 172.21.15.94:0/792522477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7e6b70 msgr2=0x55ebbd7ef210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b5df7640 1 --2- 172.21.15.94:0/792522477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ef210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b5df7640 1 -- 172.21.15.94:0/792522477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7ef750 msgr2=0x55ebbd7f1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b5df7640 1 --2- 172.21.15.94:0/792522477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7ef750 0x55ebbd7f1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b5df7640 1 -- 172.21.15.94:0/792522477 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebbd710060 con 0x55ebbd7e6250 2024-02-17T12:00:41.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b55f6640 1 --2- 172.21.15.94:0/792522477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ef210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:00:41.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b65f8640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f009c0a2230 0x7f009c0a46f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009fd0 tx=0x7f00a8009ec0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f00b5df7640 1 --2- 172.21.15.94:0/792522477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6250 0x55ebbd7e6630 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009850 tx=0x7f00a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7fdcc9def2e9fb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f009bfff640 1 -- 172.21.15.94:0/792522477 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a00bcbe0 con 0x55ebbd7e6250 2024-02-17T12:00:41.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f009bfff640 1 -- 172.21.15.94:0/792522477 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00a00bd980 con 0x55ebbd7e6250 2024-02-17T12:00:41.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.552+0000 7f009bfff640 1 -- 172.21.15.94:0/792522477 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a00bb610 con 0x55ebbd7e6250 2024-02-17T12:00:41.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.553+0000 7f00bac2a740 1 -- 172.21.15.94:0/792522477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6250 msgr2=0x55ebbd7e6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.553+0000 7f00bac2a740 1 --2- 172.21.15.94:0/792522477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6250 0x55ebbd7e6630 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009850 tx=0x7f00a0005fd0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.554+0000 7f00bac2a740 1 -- 172.21.15.94:0/792522477 shutdown_connections 2024-02-17T12:00:41.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.554+0000 7f00bac2a740 1 --2- 172.21.15.94:0/792522477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7ef750 0x55ebbd7f1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.554+0000 7f00bac2a740 1 --2- 172.21.15.94:0/792522477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ef210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.554+0000 7f00bac2a740 1 --2- 172.21.15.94:0/792522477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6250 0x55ebbd7e6630 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.554+0000 7f00bac2a740 1 -- 172.21.15.94:0/792522477 >> 172.21.15.94:0/792522477 conn(0x55ebbd7e0980 msgr2=0x55ebbd7e0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.554+0000 7f00bac2a740 1 -- 172.21.15.94:0/792522477 shutdown_connections 2024-02-17T12:00:41.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.554+0000 7f00bac2a740 1 -- 172.21.15.94:0/792522477 wait complete. 2024-02-17T12:00:41.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.555+0000 7f00bac2a740 1 -- start start 2024-02-17T12:00:41.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.556+0000 7f00bac2a740 1 Processor -- start 2024-02-17T12:00:41.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00bac2a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00bac2a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ecd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00bac2a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7ef750 0x55ebbd7ed250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00bac2a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbd74e020 con 0x55ebbd7ef750 2024-02-17T12:00:41.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00bac2a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbd421430 con 0x55ebbd7e6250 2024-02-17T12:00:41.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00bac2a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbd3dde00 con 0x55ebbd7e6b70 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00b5df7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00b5df7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7ec7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36802/0 (socket says 172.21.15.94:36802) 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00b65f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7ef750 0x55ebbd7ed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00b5df7640 1 -- 172.21.15.94:0/4142428881 learned_addr learned my addr 172.21.15.94:0/4142428881 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00bac2a740 10 client.0 _ll_register_callbacks cb 0x55ebbd6f5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00b55f6640 1 --2- 172.21.15.94:0/4142428881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ecd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00b5df7640 1 -- 172.21.15.94:0/4142428881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7e6b70 msgr2=0x55ebbd7ecd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00b5df7640 1 --2- 172.21.15.94:0/4142428881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ecd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00b5df7640 1 -- 172.21.15.94:0/4142428881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7ef750 msgr2=0x55ebbd7ed250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00b5df7640 1 --2- 172.21.15.94:0/4142428881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7ef750 0x55ebbd7ed250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00b5df7640 1 -- 172.21.15.94:0/4142428881 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b01b3b70 con 0x55ebbd7e6250 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00b5df7640 1 --2- 172.21.15.94:0/4142428881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7ec7d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f00a00bd210 tx=0x7f00a00c13d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58e787eaad7becb9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00997fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:00:41.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.557+0000 7f00997fa640 1 -- 172.21.15.94:0/4142428881 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a00e7ea0 con 0x55ebbd7e6250 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00997fa640 1 -- 172.21.15.94:0/4142428881 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f00a0000e40 con 0x55ebbd7e6250 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00bac2a740 1 -- 172.21.15.94:0/4142428881 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebbd44d7e0 con 0x55ebbd7e6250 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00997fa640 1 -- 172.21.15.94:0/4142428881 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00a00bd280 con 0x55ebbd7e6250 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00bac2a740 10 client.25637 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00bac2a740 1 -- 172.21.15.94:0/4142428881 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55ebbd74c6a0 con 0x55ebbd7e6250 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00997fa640 1 -- 172.21.15.94:0/4142428881 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ebbd44d7e0 con 0x55ebbd7e6250 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00997fa640 1 -- 172.21.15.94:0/4142428881 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f00a003c070 con 0x55ebbd7e6250 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00bac2a740 10 client.25637 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00bac2a740 10 client.25637 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00bac2a740 20 client.25637 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00bac2a740 1 --2- 172.21.15.94:0/4142428881 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ebbd7ff500 0x55ebbd8018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00bac2a740 4 client.25637 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00bac2a740 1 -- 172.21.15.94:0/4142428881 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebbd48a720 con 0x55ebbd7ff500 2024-02-17T12:00:41.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.558+0000 7f00b55f6640 1 --2- 172.21.15.94:0/4142428881 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ebbd7ff500 0x55ebbd8018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.559+0000 7f00b55f6640 1 --2- 172.21.15.94:0/4142428881 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ebbd7ff500 0x55ebbd8018f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c97dd6c7234b49b1 server_cookie=44d2041d5cd15492 in_seq=0 out_seq=0 2024-02-17T12:00:41.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.559+0000 7f00997fa640 10 client.25637 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:00:41.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.560+0000 7f00997fa640 1 -- 172.21.15.94:0/4142428881 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ebbd48a720 con 0x55ebbd7ff500 2024-02-17T12:00:41.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.560+0000 7f00997fa640 10 client.25637 handle_command_reply: tid=0 2024-02-17T12:00:41.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.560+0000 7f00bac2a740 1 client.25637 shutdown 2024-02-17T12:00:41.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.560+0000 7f00bac2a740 1 -- 172.21.15.94:0/4142428881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 msgr2=0x55ebbd7ec7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.560+0000 7f00bac2a740 1 --2- 172.21.15.94:0/4142428881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7ec7d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f00a00bd210 tx=0x7f00a00c13d0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.561+0000 7f00bac2a740 1 -- 172.21.15.94:0/4142428881 shutdown_connections 2024-02-17T12:00:41.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.561+0000 7f00bac2a740 1 --2- 172.21.15.94:0/4142428881 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ebbd7ff500 0x55ebbd8018f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.561+0000 7f00bac2a740 1 --2- 172.21.15.94:0/4142428881 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7ef750 0x55ebbd7ed250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.561+0000 7f00bac2a740 1 --2- 172.21.15.94:0/4142428881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ecd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.561+0000 7f00997fa640 0 client.25637 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:00:41.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.561+0000 7f00bac2a740 1 --2- 172.21.15.94:0/4142428881 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7ec7d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.561+0000 7f00bac2a740 1 -- 172.21.15.94:0/4142428881 >> 172.21.15.94:0/4142428881 conn(0x55ebbd7e0980 msgr2=0x55ebbd7e26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.561+0000 7f00bac2a740 1 -- 172.21.15.94:0/4142428881 shutdown_connections 2024-02-17T12:00:41.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.561+0000 7f00bac2a740 1 -- 172.21.15.94:0/4142428881 wait complete. 2024-02-17T12:00:41.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.561+0000 7f00bac2a740 20 client.25637 trim_cache size 0 max 16384 2024-02-17T12:00:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00bac2a740 1 Processor -- start 2024-02-17T12:00:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00bac2a740 1 -- start start 2024-02-17T12:00:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00bac2a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7eef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00bac2a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ed070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00bac2a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7ef750 0x55ebbd7e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00bac2a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbd6d3130 con 0x55ebbd7e6b70 2024-02-17T12:00:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00bac2a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbd44d7e0 con 0x55ebbd7e6250 2024-02-17T12:00:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00bac2a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbd7e1770 con 0x55ebbd7ef750 2024-02-17T12:00:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00b55f6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00b5df7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7eef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00b55f6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ed070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43554/0 (socket says 172.21.15.94:43554) 2024-02-17T12:00:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00b55f6640 1 -- 172.21.15.94:0/447387000 learned_addr learned my addr 172.21.15.94:0/447387000 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00b65f8640 1 --2- 172.21.15.94:0/447387000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7ef750 0x55ebbd7e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00b55f6640 1 -- 172.21.15.94:0/447387000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7ef750 msgr2=0x55ebbd7e7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00b55f6640 1 --2- 172.21.15.94:0/447387000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7ef750 0x55ebbd7e7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00b55f6640 1 -- 172.21.15.94:0/447387000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 msgr2=0x55ebbd7eef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00b55f6640 1 --2- 172.21.15.94:0/447387000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7eef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00b55f6640 1 -- 172.21.15.94:0/447387000 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebbd6e8c20 con 0x55ebbd7e6b70 2024-02-17T12:00:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.577+0000 7f00b5df7640 1 --2- 172.21.15.94:0/447387000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7eef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.578+0000 7f00b55f6640 1 --2- 172.21.15.94:0/447387000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ed070 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x55ebbd7e5d20 tx=0x7f00ac0138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cad6194630e87ad server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.578+0000 7f009b7fe640 1 -- 172.21.15.94:0/447387000 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac017040 con 0x55ebbd7e6b70 2024-02-17T12:00:41.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.578+0000 7f009b7fe640 1 -- 172.21.15.94:0/447387000 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00ac020070 con 0x55ebbd7e6b70 2024-02-17T12:00:41.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.578+0000 7f009b7fe640 1 -- 172.21.15.94:0/447387000 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac011d00 con 0x55ebbd7e6b70 2024-02-17T12:00:41.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.579+0000 7f00bac2a740 1 -- 172.21.15.94:0/447387000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6b70 msgr2=0x55ebbd7ed070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.579+0000 7f00bac2a740 1 --2- 172.21.15.94:0/447387000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ed070 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x55ebbd7e5d20 tx=0x7f00ac0138f0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.579+0000 7f00bac2a740 1 -- 172.21.15.94:0/447387000 shutdown_connections 2024-02-17T12:00:41.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.579+0000 7f00bac2a740 1 --2- 172.21.15.94:0/447387000 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7ef750 0x55ebbd7e7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.579+0000 7f00bac2a740 1 --2- 172.21.15.94:0/447387000 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7ed070 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.579+0000 7f00bac2a740 1 --2- 172.21.15.94:0/447387000 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7eef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.579+0000 7f00bac2a740 1 -- 172.21.15.94:0/447387000 >> 172.21.15.94:0/447387000 conn(0x55ebbd7e0980 msgr2=0x55ebbd7dba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.579+0000 7f00bac2a740 1 -- 172.21.15.94:0/447387000 shutdown_connections 2024-02-17T12:00:41.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.579+0000 7f00bac2a740 1 -- 172.21.15.94:0/447387000 wait complete. 2024-02-17T12:00:41.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.580+0000 7f00bac2a740 1 -- start start 2024-02-17T12:00:41.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.580+0000 7f00bac2a740 1 Processor -- start 2024-02-17T12:00:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.580+0000 7f00bac2a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7f6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00bac2a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7f4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00bac2a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7ef750 0x55ebbd7f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00b5df7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00b55f6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7f4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00b5df7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7f6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36816/0 (socket says 172.21.15.94:36816) 2024-02-17T12:00:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00b5df7640 1 -- 172.21.15.94:0/1508381400 learned_addr learned my addr 172.21.15.94:0/1508381400 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00b65f8640 1 --2- 172.21.15.94:0/1508381400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7ef750 0x55ebbd7f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00bac2a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbd7f5400 con 0x55ebbd7e6b70 2024-02-17T12:00:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00bac2a740 1 -- 172.21.15.94:0/1508381400 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbd7f5570 con 0x55ebbd7e6250 2024-02-17T12:00:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00bac2a740 1 -- 172.21.15.94:0/1508381400 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebbd7f5730 con 0x55ebbd7ef750 2024-02-17T12:00:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00bac2a740 10 client.0 _ll_register_callbacks cb 0x7f00a00b8ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:41.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00b5df7640 1 -- 172.21.15.94:0/1508381400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7ef750 msgr2=0x55ebbd7f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00b5df7640 1 --2- 172.21.15.94:0/1508381400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7ef750 0x55ebbd7f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00b5df7640 1 -- 172.21.15.94:0/1508381400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6b70 msgr2=0x55ebbd7f4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.581+0000 7f00b5df7640 1 --2- 172.21.15.94:0/1508381400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7f4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00b5df7640 1 -- 172.21.15.94:0/1508381400 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebbd74c6a0 con 0x55ebbd7e6250 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00b55f6640 1 --2- 172.21.15.94:0/1508381400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7f4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00b5df7640 1 --2- 172.21.15.94:0/1508381400 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7f6460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f00a00c54f0 tx=0x7f00a00bd120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26e88db93fbf4daf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00997fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00997fa640 1 -- 172.21.15.94:0/1508381400 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a00bcbe0 con 0x55ebbd7e6250 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00997fa640 1 -- 172.21.15.94:0/1508381400 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f00a0000e40 con 0x55ebbd7e6250 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00bac2a740 1 -- 172.21.15.94:0/1508381400 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00a00bd980 con 0x55ebbd7e6250 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00997fa640 1 -- 172.21.15.94:0/1508381400 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00a00bd280 con 0x55ebbd7e6250 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00bac2a740 10 client.25643 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00bac2a740 1 -- 172.21.15.94:0/1508381400 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x7f00ac00add0 con 0x55ebbd7e6250 2024-02-17T12:00:41.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00997fa640 1 -- 172.21.15.94:0/1508381400 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00a00ca080 con 0x55ebbd7e6250 2024-02-17T12:00:41.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00997fa640 1 -- 172.21.15.94:0/1508381400 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f00a00b8fd0 con 0x55ebbd7e6250 2024-02-17T12:00:41.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00bac2a740 10 client.25643 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:41.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00bac2a740 10 client.25643 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:00:41.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00bac2a740 20 client.25643 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:41.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00bac2a740 1 --2- 172.21.15.94:0/1508381400 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ebbd7ff500 0x55ebbd8589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00bac2a740 4 client.25643 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:00:41.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.582+0000 7f00bac2a740 1 -- 172.21.15.94:0/1508381400 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ebbd4d9890 con 0x55ebbd7ff500 2024-02-17T12:00:41.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00b55f6640 1 --2- 172.21.15.94:0/1508381400 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ebbd7ff500 0x55ebbd8589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00997fa640 10 client.25643 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:00:41.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00b55f6640 1 --2- 172.21.15.94:0/1508381400 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ebbd7ff500 0x55ebbd8589c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b234f8da9ec6f9a server_cookie=36f3968e0874eab6 in_seq=0 out_seq=0 2024-02-17T12:00:41.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00997fa640 1 -- 172.21.15.94:0/1508381400 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ebbd4d9890 con 0x55ebbd7ff500 2024-02-17T12:00:41.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00997fa640 10 client.25643 handle_command_reply: tid=0 2024-02-17T12:00:41.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00bac2a740 1 client.25643 shutdown 2024-02-17T12:00:41.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00bac2a740 1 -- 172.21.15.94:0/1508381400 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 msgr2=0x55ebbd7f6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00bac2a740 1 --2- 172.21.15.94:0/1508381400 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7f6460 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f00a00c54f0 tx=0x7f00a00bd120 comp rx=0 tx=0).stop 2024-02-17T12:00:41.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00bac2a740 1 -- 172.21.15.94:0/1508381400 shutdown_connections 2024-02-17T12:00:41.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00bac2a740 1 --2- 172.21.15.94:0/1508381400 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ebbd7ff500 0x55ebbd8589c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00bac2a740 1 --2- 172.21.15.94:0/1508381400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ebbd7ef750 0x55ebbd7f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00bac2a740 1 --2- 172.21.15.94:0/1508381400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ebbd7e6b70 0x55ebbd7f4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00bac2a740 1 --2- 172.21.15.94:0/1508381400 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ebbd7e6250 0x55ebbd7f6460 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00997fa640 0 client.25643 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:00:41.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00bac2a740 1 -- 172.21.15.94:0/1508381400 >> 172.21.15.94:0/1508381400 conn(0x55ebbd7e0980 msgr2=0x55ebbd7e7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00bac2a740 1 -- 172.21.15.94:0/1508381400 shutdown_connections 2024-02-17T12:00:41.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.583+0000 7f00bac2a740 1 -- 172.21.15.94:0/1508381400 wait complete. 2024-02-17T12:00:41.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.584+0000 7f00bac2a740 20 client.25643 trim_cache size 0 max 16384 2024-02-17T12:00:41.590 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:00:41.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.585+0000 7f00b805b640 1 -- 172.21.15.94:0/941548708 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f009c0a2230 msgr2=0x7f009c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.585+0000 7f00b805b640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f009c0a2230 0x7f009c0a46f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009fd0 tx=0x7f00a8009ec0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.585+0000 7f00b805b640 1 -- 172.21.15.94:0/941548708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00b0108660 msgr2=0x7f00b010edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.585+0000 7f00b805b640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00b0108660 0x7f00b010edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f00a0009900 tx=0x7f00a0004040 comp rx=0 tx=0).stop 2024-02-17T12:00:41.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.585+0000 7f00b805b640 1 -- 172.21.15.94:0/941548708 shutdown_connections 2024-02-17T12:00:41.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.585+0000 7f00b805b640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f009c0a2230 0x7f009c0a46f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.585+0000 7f00b805b640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00b01b0860 0x7f00b01b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.585+0000 7f00b805b640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00b010f2f0 0x7f00b010f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.585+0000 7f00b805b640 1 --2- 172.21.15.94:0/941548708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00b0108660 0x7f00b010edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.585+0000 7f00b805b640 1 -- 172.21.15.94:0/941548708 >> 172.21.15.94:0/941548708 conn(0x7f00b00fedd0 msgr2=0x7f00b0100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.585+0000 7f00b805b640 1 -- 172.21.15.94:0/941548708 shutdown_connections 2024-02-17T12:00:41.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.585+0000 7f00b805b640 1 -- 172.21.15.94:0/941548708 wait complete. 2024-02-17T12:00:41.599 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:00:41.599 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:00:41.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.750+0000 7f3016c81640 1 -- 172.21.15.94:0/1433346274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3010109630 msgr2=0x7f301010cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.750+0000 7f3016c81640 1 --2- 172.21.15.94:0/1433346274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3010109630 0x7f301010cc40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3000009930 tx=0x7f300002bc30 comp rx=0 tx=0).stop 2024-02-17T12:00:41.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.750+0000 7f3016c81640 1 -- 172.21.15.94:0/1433346274 shutdown_connections 2024-02-17T12:00:41.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.750+0000 7f3016c81640 1 --2- 172.21.15.94:0/1433346274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f301010d370 0x7f3010114eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.750+0000 7f3016c81640 1 --2- 172.21.15.94:0/1433346274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3010109630 0x7f301010cc40 secure :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3000009930 tx=0x7f300002bc30 comp rx=0 tx=0).stop 2024-02-17T12:00:41.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.750+0000 7f3016c81640 1 --2- 172.21.15.94:0/1433346274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010108c80 0x7f3010109060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.750+0000 7f3016c81640 1 -- 172.21.15.94:0/1433346274 >> 172.21.15.94:0/1433346274 conn(0x7f301006c6c0 msgr2=0x7f301006cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.751+0000 7f3016c81640 1 -- 172.21.15.94:0/1433346274 shutdown_connections 2024-02-17T12:00:41.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.751+0000 7f3016c81640 1 -- 172.21.15.94:0/1433346274 wait complete. 2024-02-17T12:00:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f3016c81640 1 Processor -- start 2024-02-17T12:00:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f3016c81640 1 -- start start 2024-02-17T12:00:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f3016c81640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010108c80 0x7f30101a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f3016c81640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f301010d370 0x7f30101a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f3016c81640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30101a4af0 0x7f30101a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f3016c81640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30101a54e0 con 0x7f301010d370 2024-02-17T12:00:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f3016c81640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30101a5650 con 0x7f30101a4af0 2024-02-17T12:00:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f3016c81640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30101a57c0 con 0x7f3010108c80 2024-02-17T12:00:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f300ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f301010d370 0x7f30101a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f300ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f301010d370 0x7f30101a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43582/0 (socket says 172.21.15.94:43582) 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f300ffff640 1 -- 172.21.15.94:0/577038403 learned_addr learned my addr 172.21.15.94:0/577038403 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f301521e640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30101a4af0 0x7f30101a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f3014a1d640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010108c80 0x7f30101a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f300ffff640 1 -- 172.21.15.94:0/577038403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010108c80 msgr2=0x7f30101a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f300ffff640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010108c80 0x7f30101a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f300ffff640 1 -- 172.21.15.94:0/577038403 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30101a4af0 msgr2=0x7f30101a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f300ffff640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30101a4af0 0x7f30101a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f300ffff640 1 -- 172.21.15.94:0/577038403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3000009560 con 0x7f301010d370 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.752+0000 7f301521e640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30101a4af0 0x7f30101a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.753+0000 7f300ffff640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f301010d370 0x7f30101a07c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3000009230 tx=0x7f3000004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.753+0000 7f300dffb640 1 -- 172.21.15.94:0/577038403 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000004350 con 0x7f301010d370 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.753+0000 7f3016c81640 1 -- 172.21.15.94:0/577038403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30101acda0 con 0x7f301010d370 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.753+0000 7f300dffb640 1 -- 172.21.15.94:0/577038403 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30000044b0 con 0x7f301010d370 2024-02-17T12:00:41.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.753+0000 7f300dffb640 1 -- 172.21.15.94:0/577038403 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000003b90 con 0x7f301010d370 2024-02-17T12:00:41.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.753+0000 7f3016c81640 1 -- 172.21.15.94:0/577038403 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30101ad1d0 con 0x7f301010d370 2024-02-17T12:00:41.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.756+0000 7f301984f740 1 Processor -- start 2024-02-17T12:00:41.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.756+0000 7f301984f740 1 -- start start 2024-02-17T12:00:41.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f300dffb640 1 -- 172.21.15.94:0/577038403 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3000003cf0 con 0x7f301010d370 2024-02-17T12:00:41.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f301984f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8d250 0x564a94f8d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f301984f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f8db70 0x564a94f96210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f301984f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f96750 0x564a94f98b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f301984f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94f48960 con 0x564a94f96750 2024-02-17T12:00:41.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f301984f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94efc310 con 0x564a94f8db70 2024-02-17T12:00:41.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f301984f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94ef9840 con 0x564a94f8d250 2024-02-17T12:00:41.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f300ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f96750 0x564a94f98b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f300ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f96750 0x564a94f98b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43592/0 (socket says 172.21.15.94:43592) 2024-02-17T12:00:41.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f300ffff640 1 -- 172.21.15.94:0/2656345703 learned_addr learned my addr 172.21.15.94:0/2656345703 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f3014a1d640 1 --2- 172.21.15.94:0/2656345703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f8db70 0x564a94f96210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f301521e640 1 --2- 172.21.15.94:0/2656345703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8d250 0x564a94f8d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f300ffff640 1 -- 172.21.15.94:0/2656345703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8d250 msgr2=0x564a94f8d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f300ffff640 1 --2- 172.21.15.94:0/2656345703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8d250 0x564a94f8d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f300ffff640 1 -- 172.21.15.94:0/2656345703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f8db70 msgr2=0x564a94f96210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f300ffff640 1 --2- 172.21.15.94:0/2656345703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f8db70 0x564a94f96210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f300ffff640 1 -- 172.21.15.94:0/2656345703 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a94eb7060 con 0x564a94f96750 2024-02-17T12:00:41.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f300dffb640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2ff80a2230 0x7f2ff80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.757+0000 7f300ffff640 1 --2- 172.21.15.94:0/2656345703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f96750 0x564a94f98b40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f30000edae0 tx=0x7f30000f8470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ff2ed512baddb5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.758+0000 7f301521e640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2ff80a2230 0x7f2ff80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.758+0000 7f300dffb640 1 -- 172.21.15.94:0/577038403 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f30000f0320 con 0x7f301010d370 2024-02-17T12:00:41.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.758+0000 7f2ff6ffd640 1 -- 172.21.15.94:0/2656345703 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30000b90b0 con 0x564a94f96750 2024-02-17T12:00:41.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.758+0000 7f2ff6ffd640 1 -- 172.21.15.94:0/2656345703 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30000bf020 con 0x564a94f96750 2024-02-17T12:00:41.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.758+0000 7f2ff6ffd640 1 -- 172.21.15.94:0/2656345703 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30000bc770 con 0x564a94f96750 2024-02-17T12:00:41.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.759+0000 7f301521e640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2ff80a2230 0x7f2ff80a46f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f300400e560 tx=0x7f3004010040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.759+0000 7f301984f740 1 -- 172.21.15.94:0/2656345703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f96750 msgr2=0x564a94f98b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.759+0000 7f301984f740 1 --2- 172.21.15.94:0/2656345703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f96750 0x564a94f98b40 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f30000edae0 tx=0x7f30000f8470 comp rx=0 tx=0).stop 2024-02-17T12:00:41.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.759+0000 7f301984f740 1 -- 172.21.15.94:0/2656345703 shutdown_connections 2024-02-17T12:00:41.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.759+0000 7f301984f740 1 --2- 172.21.15.94:0/2656345703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f96750 0x564a94f98b40 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.759+0000 7f301984f740 1 --2- 172.21.15.94:0/2656345703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f8db70 0x564a94f96210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.759+0000 7f301984f740 1 --2- 172.21.15.94:0/2656345703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8d250 0x564a94f8d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.759+0000 7f301984f740 1 -- 172.21.15.94:0/2656345703 >> 172.21.15.94:0/2656345703 conn(0x564a94f87980 msgr2=0x564a94f87d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.759+0000 7f301984f740 1 -- 172.21.15.94:0/2656345703 shutdown_connections 2024-02-17T12:00:41.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.760+0000 7f301984f740 1 -- 172.21.15.94:0/2656345703 wait complete. 2024-02-17T12:00:41.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.760+0000 7f301984f740 1 -- start start 2024-02-17T12:00:41.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.760+0000 7f301984f740 1 Processor -- start 2024-02-17T12:00:41.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.760+0000 7f301984f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8d250 0x564a94f93760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f301984f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8db70 0x564a94f93ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f301984f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94f941e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f301984f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94ef5020 con 0x564a94f8db70 2024-02-17T12:00:41.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f301984f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94bc8430 con 0x564a94f96750 2024-02-17T12:00:41.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f301984f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94b84e00 con 0x564a94f8d250 2024-02-17T12:00:41.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f300ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8db70 0x564a94f93ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f300ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8db70 0x564a94f93ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43602/0 (socket says 172.21.15.94:43602) 2024-02-17T12:00:41.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f300ffff640 1 -- 172.21.15.94:0/3194317050 learned_addr learned my addr 172.21.15.94:0/3194317050 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f3014a1d640 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8d250 0x564a94f93760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f301521e640 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94f941e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f301984f740 10 client.0 _ll_register_callbacks cb 0x564a94e9ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:41.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f300ffff640 1 -- 172.21.15.94:0/3194317050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8d250 msgr2=0x564a94f93760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f300ffff640 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8d250 0x564a94f93760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f300ffff640 1 -- 172.21.15.94:0/3194317050 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 msgr2=0x564a94f941e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f300ffff640 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94f941e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f300ffff640 1 -- 172.21.15.94:0/3194317050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30101ad1d0 con 0x564a94f8db70 2024-02-17T12:00:41.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f3014a1d640 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8d250 0x564a94f93760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:41.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f300ffff640 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8db70 0x564a94f93ca0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f30000c5080 tx=0x7f3000005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448401575ad93506 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.761+0000 7f2feffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:00:41.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f2feffff640 1 -- 172.21.15.94:0/3194317050 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30000f8e70 con 0x564a94f8db70 2024-02-17T12:00:41.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f2feffff640 1 -- 172.21.15.94:0/3194317050 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f30000c0070 con 0x564a94f8db70 2024-02-17T12:00:41.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f301984f740 1 -- 172.21.15.94:0/3194317050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a94bf47e0 con 0x564a94f8db70 2024-02-17T12:00:41.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f2feffff640 1 -- 172.21.15.94:0/3194317050 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30000b90b0 con 0x564a94f8db70 2024-02-17T12:00:41.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f301984f740 10 client.16206 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:41.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f301984f740 1 -- 172.21.15.94:0/3194317050 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x564a94ef36a0 con 0x564a94f8db70 2024-02-17T12:00:41.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f2feffff640 1 -- 172.21.15.94:0/3194317050 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3000086070 con 0x564a94f8db70 2024-02-17T12:00:41.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f2feffff640 1 -- 172.21.15.94:0/3194317050 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f300004f020 con 0x564a94f8db70 2024-02-17T12:00:41.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f301984f740 10 client.16206 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:41.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f301984f740 10 client.16206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:41.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f301984f740 20 client.16206 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:41.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f301984f740 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a94fa61d0 0x564a94fa85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f301984f740 4 client.16206 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:41.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.762+0000 7f301984f740 1 -- 172.21.15.94:0/3194317050 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a94c31720 con 0x564a94fa61d0 2024-02-17T12:00:41.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.763+0000 7f3014a1d640 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a94fa61d0 0x564a94fa85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.763+0000 7f3014a1d640 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a94fa61d0 0x564a94fa85c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4be9616a5e50fcc0 server_cookie=6d3a0a84462b137 in_seq=0 out_seq=0 2024-02-17T12:00:41.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.763+0000 7f2feffff640 10 client.16206 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:41.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.764+0000 7f2feffff640 1 -- 172.21.15.94:0/3194317050 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a94c31720 con 0x564a94fa61d0 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.764+0000 7f2feffff640 10 client.16206 handle_command_reply: tid=0 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.764+0000 7f301984f740 1 client.16206 shutdown 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.764+0000 7f301984f740 1 -- 172.21.15.94:0/3194317050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8db70 msgr2=0x564a94f93ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.764+0000 7f301984f740 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8db70 0x564a94f93ca0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f30000c5080 tx=0x7f3000005670 comp rx=0 tx=0).stop 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.765+0000 7f301984f740 1 -- 172.21.15.94:0/3194317050 shutdown_connections 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.765+0000 7f301984f740 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a94fa61d0 0x564a94fa85c0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.765+0000 7f2feffff640 0 client.16206 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.765+0000 7f301984f740 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94f941e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.765+0000 7f301984f740 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8db70 0x564a94f93ca0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.765+0000 7f301984f740 1 --2- 172.21.15.94:0/3194317050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8d250 0x564a94f93760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.765+0000 7f301984f740 1 -- 172.21.15.94:0/3194317050 >> 172.21.15.94:0/3194317050 conn(0x564a94f87980 msgr2=0x564a94f896d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.765+0000 7f301984f740 1 -- 172.21.15.94:0/3194317050 shutdown_connections 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.765+0000 7f301984f740 1 -- 172.21.15.94:0/3194317050 wait complete. 2024-02-17T12:00:41.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.765+0000 7f301984f740 20 client.16206 trim_cache size 0 max 16384 2024-02-17T12:00:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.781+0000 7f301984f740 1 Processor -- start 2024-02-17T12:00:41.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.782+0000 7f301984f740 1 -- start start 2024-02-17T12:00:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.782+0000 7f301984f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8d250 0x564a94f95e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.782+0000 7f301984f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8db70 0x564a94f909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.782+0000 7f301984f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94f90f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.782+0000 7f301984f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94e7a130 con 0x564a94f8d250 2024-02-17T12:00:41.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.782+0000 7f301984f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94c31720 con 0x564a94f96750 2024-02-17T12:00:41.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.782+0000 7f301984f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94f99110 con 0x564a94f8db70 2024-02-17T12:00:41.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.782+0000 7f301521e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94f90f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f301521e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94f90f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36848/0 (socket says 172.21.15.94:36848) 2024-02-17T12:00:41.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f301521e640 1 -- 172.21.15.94:0/2581887050 learned_addr learned my addr 172.21.15.94:0/2581887050 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f3014a1d640 1 --2- 172.21.15.94:0/2581887050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8d250 0x564a94f95e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f300ffff640 1 --2- 172.21.15.94:0/2581887050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8db70 0x564a94f909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f301521e640 1 -- 172.21.15.94:0/2581887050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8db70 msgr2=0x564a94f909e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f301521e640 1 --2- 172.21.15.94:0/2581887050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8db70 0x564a94f909e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f301521e640 1 -- 172.21.15.94:0/2581887050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8d250 msgr2=0x564a94f95e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f301521e640 1 --2- 172.21.15.94:0/2581887050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8d250 0x564a94f95e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f301521e640 1 -- 172.21.15.94:0/2581887050 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a94e8fc20 con 0x564a94f96750 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f300ffff640 1 --2- 172.21.15.94:0/2581887050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8db70 0x564a94f909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f301521e640 1 --2- 172.21.15.94:0/2581887050 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94f90f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f300400da50 tx=0x7f3004015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e5e8fdd0af60a76 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f2ff67fc640 1 -- 172.21.15.94:0/2581887050 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004013040 con 0x564a94f96750 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f2ff67fc640 1 -- 172.21.15.94:0/2581887050 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f300401f040 con 0x564a94f96750 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.783+0000 7f2ff67fc640 1 -- 172.21.15.94:0/2581887050 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004025dd0 con 0x564a94f96750 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.784+0000 7f301984f740 1 -- 172.21.15.94:0/2581887050 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 msgr2=0x564a94f90f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.784+0000 7f301984f740 1 --2- 172.21.15.94:0/2581887050 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94f90f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f300400da50 tx=0x7f3004015000 comp rx=0 tx=0).stop 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.784+0000 7f301984f740 1 -- 172.21.15.94:0/2581887050 shutdown_connections 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.784+0000 7f301984f740 1 --2- 172.21.15.94:0/2581887050 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94f90f20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.784+0000 7f301984f740 1 --2- 172.21.15.94:0/2581887050 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8db70 0x564a94f909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.784+0000 7f301984f740 1 --2- 172.21.15.94:0/2581887050 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8d250 0x564a94f95e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.784+0000 7f301984f740 1 -- 172.21.15.94:0/2581887050 >> 172.21.15.94:0/2581887050 conn(0x564a94f87980 msgr2=0x564a94f8ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.784+0000 7f301984f740 1 -- 172.21.15.94:0/2581887050 shutdown_connections 2024-02-17T12:00:41.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.784+0000 7f301984f740 1 -- 172.21.15.94:0/2581887050 wait complete. 2024-02-17T12:00:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.784+0000 7f301984f740 1 -- start start 2024-02-17T12:00:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.785+0000 7f301984f740 1 Processor -- start 2024-02-17T12:00:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.785+0000 7f301984f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8d250 0x564a94f9c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.785+0000 7f301984f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8db70 0x564a94f9cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.785+0000 7f301984f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94ff6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.785+0000 7f301984f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94ff6aa0 con 0x564a94f8d250 2024-02-17T12:00:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.785+0000 7f301984f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94ff6be0 con 0x564a94f96750 2024-02-17T12:00:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.785+0000 7f301984f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a94ff6da0 con 0x564a94f8db70 2024-02-17T12:00:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.785+0000 7f301984f740 10 client.0 _ll_register_callbacks cb 0x7f30000f3d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.785+0000 7f3014a1d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8d250 0x564a94f9c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.785+0000 7f3014a1d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8d250 0x564a94f9c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43628/0 (socket says 172.21.15.94:43628) 2024-02-17T12:00:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.785+0000 7f3014a1d640 1 -- 172.21.15.94:0/2045164721 learned_addr learned my addr 172.21.15.94:0/2045164721 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:41.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.785+0000 7f301521e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94ff6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.785+0000 7f300ffff640 1 --2- 172.21.15.94:0/2045164721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8db70 0x564a94f9cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.786+0000 7f301521e640 1 -- 172.21.15.94:0/2045164721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8db70 msgr2=0x564a94f9cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.786+0000 7f301521e640 1 --2- 172.21.15.94:0/2045164721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8db70 0x564a94f9cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.786+0000 7f301521e640 1 -- 172.21.15.94:0/2045164721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8d250 msgr2=0x564a94f9c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.786+0000 7f301521e640 1 --2- 172.21.15.94:0/2045164721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8d250 0x564a94f9c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.786+0000 7f301521e640 1 -- 172.21.15.94:0/2045164721 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300400d6d0 con 0x564a94f96750 2024-02-17T12:00:41.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.787+0000 7f301521e640 1 --2- 172.21.15.94:0/2045164721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94ff6560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f300400edf0 tx=0x7f3004026e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cd4e39f260684f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:41.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.787+0000 7f2feffff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.787+0000 7f2feffff640 1 -- 172.21.15.94:0/2045164721 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3004013040 con 0x564a94f96750 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.787+0000 7f2feffff640 1 -- 172.21.15.94:0/2045164721 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3004019bd0 con 0x564a94f96750 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.787+0000 7f301984f740 1 -- 172.21.15.94:0/2045164721 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a94f88300 con 0x564a94f96750 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.787+0000 7f2feffff640 1 -- 172.21.15.94:0/2045164721 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3004019d90 con 0x564a94f96750 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.787+0000 7f301984f740 10 client.25661 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.787+0000 7f301984f740 1 -- 172.21.15.94:0/2045164721 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x564a94eb58c0 con 0x564a94f96750 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.788+0000 7f2feffff640 1 -- 172.21.15.94:0/2045164721 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564a94f88300 con 0x564a94f96750 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.788+0000 7f2feffff640 1 -- 172.21.15.94:0/2045164721 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3004016030 con 0x564a94f96750 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.788+0000 7f301984f740 10 client.25661 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.788+0000 7f301984f740 10 client.25661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.788+0000 7f301984f740 20 client.25661 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.789+0000 7f301984f740 1 --2- 172.21.15.94:0/2045164721 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a94fa61d0 0x564a95000560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.789+0000 7f301984f740 4 client.25661 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:00:41.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.789+0000 7f301984f740 1 -- 172.21.15.94:0/2045164721 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f30000bf020 con 0x564a94fa61d0 2024-02-17T12:00:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.789+0000 7f3014a1d640 1 --2- 172.21.15.94:0/2045164721 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a94fa61d0 0x564a95000560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.789+0000 7f2feffff640 10 client.25661 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.789+0000 7f3014a1d640 1 --2- 172.21.15.94:0/2045164721 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a94fa61d0 0x564a95000560 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2749fc46377bed3 server_cookie=45339d356a5edde6 in_seq=0 out_seq=0 2024-02-17T12:00:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.790+0000 7f2feffff640 1 -- 172.21.15.94:0/2045164721 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f30000bf020 con 0x564a94fa61d0 2024-02-17T12:00:41.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.790+0000 7f2feffff640 10 client.25661 handle_command_reply: tid=0 2024-02-17T12:00:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.791+0000 7f301984f740 1 client.25661 shutdown 2024-02-17T12:00:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.791+0000 7f301984f740 1 -- 172.21.15.94:0/2045164721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 msgr2=0x564a94ff6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.791+0000 7f301984f740 1 --2- 172.21.15.94:0/2045164721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94ff6560 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f300400edf0 tx=0x7f3004026e90 comp rx=0 tx=0).stop 2024-02-17T12:00:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.791+0000 7f301984f740 1 -- 172.21.15.94:0/2045164721 shutdown_connections 2024-02-17T12:00:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.791+0000 7f301984f740 1 --2- 172.21.15.94:0/2045164721 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a94fa61d0 0x564a95000560 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.791+0000 7f301984f740 1 --2- 172.21.15.94:0/2045164721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a94f96750 0x564a94ff6560 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.791+0000 7f301984f740 1 --2- 172.21.15.94:0/2045164721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a94f8db70 0x564a94f9cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.791+0000 7f2feffff640 0 client.25661 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:41.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.791+0000 7f301984f740 1 --2- 172.21.15.94:0/2045164721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a94f8d250 0x564a94f9c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.791+0000 7f301984f740 1 -- 172.21.15.94:0/2045164721 >> 172.21.15.94:0/2045164721 conn(0x564a94f87980 msgr2=0x564a94f862f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f301984f740 1 -- 172.21.15.94:0/2045164721 shutdown_connections 2024-02-17T12:00:41.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f301984f740 1 -- 172.21.15.94:0/2045164721 wait complete. 2024-02-17T12:00:41.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f301984f740 20 client.25661 trim_cache size 0 max 16384 2024-02-17T12:00:41.794 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:00:41.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f3016c81640 1 -- 172.21.15.94:0/577038403 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2ff80a2230 msgr2=0x7f2ff80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f3016c81640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2ff80a2230 0x7f2ff80a46f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f300400e560 tx=0x7f3004010040 comp rx=0 tx=0).stop 2024-02-17T12:00:41.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f3016c81640 1 -- 172.21.15.94:0/577038403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f301010d370 msgr2=0x7f30101a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:41.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f3016c81640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f301010d370 0x7f30101a07c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3000009230 tx=0x7f3000004c00 comp rx=0 tx=0).stop 2024-02-17T12:00:41.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f3016c81640 1 -- 172.21.15.94:0/577038403 shutdown_connections 2024-02-17T12:00:41.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f3016c81640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2ff80a2230 0x7f2ff80a46f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f3016c81640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30101a4af0 0x7f30101a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f3016c81640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f301010d370 0x7f30101a07c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f3016c81640 1 --2- 172.21.15.94:0/577038403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3010108c80 0x7f30101a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:41.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f3016c81640 1 -- 172.21.15.94:0/577038403 >> 172.21.15.94:0/577038403 conn(0x7f301006c6c0 msgr2=0x7f3010070c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:41.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f3016c81640 1 -- 172.21.15.94:0/577038403 shutdown_connections 2024-02-17T12:00:41.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:41.792+0000 7f3016c81640 1 -- 172.21.15.94:0/577038403 wait complete. 2024-02-17T12:00:42.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:41 smithi189 ceph-mon[36588]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.5 MiB/s wr, 634 op/s 2024-02-17T12:00:42.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:41 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/159146367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:00:42.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:41 smithi125 ceph-mon[36819]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.5 MiB/s wr, 634 op/s 2024-02-17T12:00:42.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:41 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/159146367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:00:42.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:41 smithi094 ceph-mon[32499]: pgmap v80: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.5 MiB/s wr, 634 op/s 2024-02-17T12:00:42.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:41 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/159146367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:00:42.807 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:00:42.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:00:42.807 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:00:42.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.974+0000 7f64a4c88640 1 -- 172.21.15.94:0/3874257475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a010d9d0 msgr2=0x7f64a010de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:42.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.974+0000 7f64a4c88640 1 --2- 172.21.15.94:0/3874257475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a010d9d0 0x7f64a010de50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f6494009900 tx=0x7f649402bc60 comp rx=0 tx=0).stop 2024-02-17T12:00:42.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.974+0000 7f64a4c88640 1 -- 172.21.15.94:0/3874257475 shutdown_connections 2024-02-17T12:00:42.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.974+0000 7f64a4c88640 1 --2- 172.21.15.94:0/3874257475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a011a960 0x7f64a011cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.974+0000 7f64a4c88640 1 --2- 172.21.15.94:0/3874257475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a010d9d0 0x7f64a010de50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.974+0000 7f64a4c88640 1 --2- 172.21.15.94:0/3874257475 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64a010f9a0 0x7f64a010d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.974+0000 7f64a4c88640 1 -- 172.21.15.94:0/3874257475 >> 172.21.15.94:0/3874257475 conn(0x7f64a006c260 msgr2=0x7f64a006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:42.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.975+0000 7f64a4c88640 1 -- 172.21.15.94:0/3874257475 shutdown_connections 2024-02-17T12:00:42.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.975+0000 7f64a4c88640 1 -- 172.21.15.94:0/3874257475 wait complete. 2024-02-17T12:00:42.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.976+0000 7f64a4c88640 1 Processor -- start 2024-02-17T12:00:42.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.977+0000 7f64a4c88640 1 -- start start 2024-02-17T12:00:42.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.977+0000 7f64a4c88640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a010d9d0 0x7f64a01a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:42.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.977+0000 7f64a4c88640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64a010f9a0 0x7f64a01a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:42.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.977+0000 7f64a4c88640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a011a960 0x7f64a01aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:42.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.977+0000 7f64a4c88640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a01ab510 con 0x7f64a010d9d0 2024-02-17T12:00:42.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.977+0000 7f64a4c88640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a01ab680 con 0x7f64a010f9a0 2024-02-17T12:00:42.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.977+0000 7f64a4c88640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a01ab840 con 0x7f64a011a960 2024-02-17T12:00:42.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.977+0000 7f649e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a010d9d0 0x7f64a01a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:42.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.977+0000 7f649dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64a010f9a0 0x7f64a01a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:42.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.977+0000 7f649dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64a010f9a0 0x7f64a01a6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36878/0 (socket says 172.21.15.94:36878) 2024-02-17T12:00:42.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.977+0000 7f649dd9b640 1 -- 172.21.15.94:0/3428730431 learned_addr learned my addr 172.21.15.94:0/3428730431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:42.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.977+0000 7f649ed9d640 1 --2- 172.21.15.94:0/3428730431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a011a960 0x7f64a01aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:42.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.978+0000 7f649dd9b640 1 -- 172.21.15.94:0/3428730431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a011a960 msgr2=0x7f64a01aaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:42.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.978+0000 7f649dd9b640 1 --2- 172.21.15.94:0/3428730431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a011a960 0x7f64a01aaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.978+0000 7f649dd9b640 1 -- 172.21.15.94:0/3428730431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a010d9d0 msgr2=0x7f64a01a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:42.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.978+0000 7f649dd9b640 1 --2- 172.21.15.94:0/3428730431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a010d9d0 0x7f64a01a6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.978+0000 7f649dd9b640 1 -- 172.21.15.94:0/3428730431 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6490009490 con 0x7f64a010f9a0 2024-02-17T12:00:42.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.978+0000 7f649dd9b640 1 --2- 172.21.15.94:0/3428730431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64a010f9a0 0x7f64a01a6b90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6494005ab0 tx=0x7f649402c6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:42.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.978+0000 7f648f7fe640 1 -- 172.21.15.94:0/3428730431 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494004350 con 0x7f64a010f9a0 2024-02-17T12:00:42.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.978+0000 7f64a4c88640 1 -- 172.21.15.94:0/3428730431 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6494009560 con 0x7f64a010f9a0 2024-02-17T12:00:42.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.978+0000 7f648f7fe640 1 -- 172.21.15.94:0/3428730431 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64940044b0 con 0x7f64a010f9a0 2024-02-17T12:00:42.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.978+0000 7f648f7fe640 1 -- 172.21.15.94:0/3428730431 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494003d30 con 0x7f64a010f9a0 2024-02-17T12:00:42.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.978+0000 7f64a4c88640 1 -- 172.21.15.94:0/3428730431 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a01b31c0 con 0x7f64a010f9a0 2024-02-17T12:00:42.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.982+0000 7f64a7865740 1 Processor -- start 2024-02-17T12:00:42.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.982+0000 7f64a7865740 1 -- start start 2024-02-17T12:00:42.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.983+0000 7f64a7865740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x563561fc4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:42.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.983+0000 7f64a7865740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fc4d30 0x563561fcd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.983+0000 7f64a7865740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fcd8b0 0x563561fcfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.983+0000 7f64a7865740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563561f32f90 con 0x563561fc4d30 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.983+0000 7f64a7865740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563561f304c0 con 0x563561fcd8b0 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.983+0000 7f64a7865740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563561f2bca0 con 0x563561fc4380 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.983+0000 7f649ed9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x563561fc4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.983+0000 7f649ed9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x563561fc4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39888/0 (socket says 172.21.15.94:39888) 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.983+0000 7f649ed9d640 1 -- 172.21.15.94:0/413264560 learned_addr learned my addr 172.21.15.94:0/413264560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.983+0000 7f648f7fe640 1 -- 172.21.15.94:0/3428730431 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6494004df0 con 0x7f64a010f9a0 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.983+0000 7f649e59c640 1 --2- 172.21.15.94:0/413264560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fc4d30 0x563561fcd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.983+0000 7f649dd9b640 1 --2- 172.21.15.94:0/413264560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fcd8b0 0x563561fcfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.984+0000 7f649ed9d640 1 -- 172.21.15.94:0/413264560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fcd8b0 msgr2=0x563561fcfd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.984+0000 7f649ed9d640 1 --2- 172.21.15.94:0/413264560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fcd8b0 0x563561fcfd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.984+0000 7f649ed9d640 1 -- 172.21.15.94:0/413264560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fc4d30 msgr2=0x563561fcd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:42.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.984+0000 7f649ed9d640 1 --2- 172.21.15.94:0/413264560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fc4d30 0x563561fcd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.984+0000 7f649ed9d640 1 -- 172.21.15.94:0/413264560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563561eee0b0 con 0x563561fc4380 2024-02-17T12:00:42.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.985+0000 7f649ed9d640 1 --2- 172.21.15.94:0/413264560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x563561fc4760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f649800b500 tx=0x7f6498009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1d6666cde66ca3d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:42.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.985+0000 7f648cff9640 1 -- 172.21.15.94:0/413264560 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498010040 con 0x563561fc4380 2024-02-17T12:00:42.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.985+0000 7f648cff9640 1 -- 172.21.15.94:0/413264560 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f649800ae50 con 0x563561fc4380 2024-02-17T12:00:42.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.985+0000 7f648cff9640 1 -- 172.21.15.94:0/413264560 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6498013380 con 0x563561fc4380 2024-02-17T12:00:42.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.985+0000 7f648f7fe640 1 --2- 172.21.15.94:0/3428730431 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f64880a22b0 0x7f64880a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:42.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.985+0000 7f648f7fe640 1 -- 172.21.15.94:0/3428730431 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f64940f0520 con 0x7f64a010f9a0 2024-02-17T12:00:42.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.986+0000 7f649e59c640 1 --2- 172.21.15.94:0/3428730431 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f64880a22b0 0x7f64880a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:42.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.986+0000 7f64a7865740 1 -- 172.21.15.94:0/413264560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 msgr2=0x563561fc4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:42.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.986+0000 7f64a7865740 1 --2- 172.21.15.94:0/413264560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x563561fc4760 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f649800b500 tx=0x7f6498009630 comp rx=0 tx=0).stop 2024-02-17T12:00:42.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.986+0000 7f649e59c640 1 --2- 172.21.15.94:0/3428730431 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f64880a22b0 0x7f64880a4770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x563561fbfac0 tx=0x7f6490005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:42.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.987+0000 7f64a7865740 1 -- 172.21.15.94:0/413264560 shutdown_connections 2024-02-17T12:00:42.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.987+0000 7f64a7865740 1 --2- 172.21.15.94:0/413264560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fcd8b0 0x563561fcfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.987+0000 7f64a7865740 1 --2- 172.21.15.94:0/413264560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fc4d30 0x563561fcd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.987+0000 7f64a7865740 1 --2- 172.21.15.94:0/413264560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x563561fc4760 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.987+0000 7f64a7865740 1 -- 172.21.15.94:0/413264560 >> 172.21.15.94:0/413264560 conn(0x563561fbe840 msgr2=0x563561fbec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:42.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.987+0000 7f64a7865740 1 -- 172.21.15.94:0/413264560 shutdown_connections 2024-02-17T12:00:42.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.987+0000 7f64a7865740 1 -- 172.21.15.94:0/413264560 wait complete. 2024-02-17T12:00:42.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.987+0000 7f64a7865740 1 -- start start 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f64a7865740 1 Processor -- start 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f64a7865740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4380 0x563561fca940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f64a7865740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fc4d30 0x563561fcae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f64a7865740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fcd8b0 0x563561fcb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f64a7865740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563561ed0f40 con 0x563561fc4d30 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f64a7865740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563561bff4b0 con 0x563561fc4380 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f64a7865740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563561eb1670 con 0x563561fcd8b0 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f64a7865740 10 client.0 _ll_register_callbacks cb 0x563561f1ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f649dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fc4d30 0x563561fcae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f649dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fc4d30 0x563561fcae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43668/0 (socket says 172.21.15.94:43668) 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f649dd9b640 1 -- 172.21.15.94:0/499423158 learned_addr learned my addr 172.21.15.94:0/499423158 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f649ed9d640 1 --2- 172.21.15.94:0/499423158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fcd8b0 0x563561fcb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f649e59c640 1 --2- 172.21.15.94:0/499423158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4380 0x563561fca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:42.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f649dd9b640 1 -- 172.21.15.94:0/499423158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fcd8b0 msgr2=0x563561fcb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:42.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f649dd9b640 1 --2- 172.21.15.94:0/499423158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fcd8b0 0x563561fcb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f649dd9b640 1 -- 172.21.15.94:0/499423158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4380 msgr2=0x563561fca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:42.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f649dd9b640 1 --2- 172.21.15.94:0/499423158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4380 0x563561fca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.988+0000 7f649dd9b640 1 -- 172.21.15.94:0/499423158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6498009ce0 con 0x563561fc4d30 2024-02-17T12:00:42.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.989+0000 7f649dd9b640 1 --2- 172.21.15.94:0/499423158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fc4d30 0x563561fcae80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6494006010 tx=0x7f6494003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4638e0af676c0fdc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:42.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.989+0000 7f6485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:00:42.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.989+0000 7f6485ffb640 1 -- 172.21.15.94:0/499423158 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64940c5520 con 0x563561fc4d30 2024-02-17T12:00:42.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.989+0000 7f6485ffb640 1 -- 172.21.15.94:0/499423158 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f64940035f0 con 0x563561fc4d30 2024-02-17T12:00:42.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.989+0000 7f64a7865740 1 -- 172.21.15.94:0/499423158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563561f05f40 con 0x563561fc4d30 2024-02-17T12:00:42.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.990+0000 7f6485ffb640 1 -- 172.21.15.94:0/499423158 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64940bcde0 con 0x563561fc4d30 2024-02-17T12:00:42.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.990+0000 7f64a7865740 10 client.16224 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:42.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.990+0000 7f64a7865740 1 -- 172.21.15.94:0/499423158 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x563561f2a320 con 0x563561fc4d30 2024-02-17T12:00:42.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.990+0000 7f6485ffb640 1 -- 172.21.15.94:0/499423158 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563561f05f40 con 0x563561fc4d30 2024-02-17T12:00:42.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.991+0000 7f6485ffb640 1 -- 172.21.15.94:0/499423158 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f649403c070 con 0x563561fc4d30 2024-02-17T12:00:42.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.991+0000 7f64a7865740 10 client.16224 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:42.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.991+0000 7f64a7865740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:42.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.991+0000 7f64a7865740 20 client.16224 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:42.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.991+0000 7f64a7865740 1 --2- 172.21.15.94:0/499423158 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563561fddd10 0x563561fe0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.991+0000 7f64a7865740 4 client.16224 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.991+0000 7f64a7865740 1 -- 172.21.15.94:0/499423158 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563561f22310 con 0x563561fddd10 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.991+0000 7f649e59c640 1 --2- 172.21.15.94:0/499423158 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563561fddd10 0x563561fe0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.991+0000 7f6485ffb640 10 client.16224 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.991+0000 7f649e59c640 1 --2- 172.21.15.94:0/499423158 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563561fddd10 0x563561fe0100 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69fbb80271ee2d2e server_cookie=21fd736a85538273 in_seq=0 out_seq=0 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.992+0000 7f6485ffb640 1 -- 172.21.15.94:0/499423158 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563561f22310 con 0x563561fddd10 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.992+0000 7f6485ffb640 10 client.16224 handle_command_reply: tid=0 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.992+0000 7f64a7865740 1 client.16224 shutdown 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.993+0000 7f64a7865740 1 -- 172.21.15.94:0/499423158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fc4d30 msgr2=0x563561fcae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.993+0000 7f64a7865740 1 --2- 172.21.15.94:0/499423158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fc4d30 0x563561fcae80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6494006010 tx=0x7f6494003f90 comp rx=0 tx=0).stop 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.993+0000 7f64a7865740 1 -- 172.21.15.94:0/499423158 shutdown_connections 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.993+0000 7f64a7865740 1 --2- 172.21.15.94:0/499423158 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563561fddd10 0x563561fe0100 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.993+0000 7f64a7865740 1 --2- 172.21.15.94:0/499423158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fcd8b0 0x563561fcb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.993+0000 7f64a7865740 1 --2- 172.21.15.94:0/499423158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fc4d30 0x563561fcae80 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.993+0000 7f64a7865740 1 --2- 172.21.15.94:0/499423158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4380 0x563561fca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:42.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.993+0000 7f64a7865740 1 -- 172.21.15.94:0/499423158 >> 172.21.15.94:0/499423158 conn(0x563561fbe840 msgr2=0x563561fc07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:42.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.993+0000 7f6485ffb640 0 client.16224 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:42.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.993+0000 7f64a7865740 1 -- 172.21.15.94:0/499423158 shutdown_connections 2024-02-17T12:00:42.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.993+0000 7f64a7865740 1 -- 172.21.15.94:0/499423158 wait complete. 2024-02-17T12:00:42.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:42.993+0000 7f64a7865740 20 client.16224 trim_cache size 0 max 16384 2024-02-17T12:00:43.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.018+0000 7f64a7865740 1 Processor -- start 2024-02-17T12:00:43.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.018+0000 7f64a7865740 1 -- start start 2024-02-17T12:00:43.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.019+0000 7f64a7865740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x563561fcb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.019+0000 7f64a7865740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4d30 0x563561fc0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.019+0000 7f64a7865740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fcd8b0 0x563561fd48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.019+0000 7f64a7865740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563561f22310 con 0x563561fcd8b0 2024-02-17T12:00:43.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.019+0000 7f64a7865740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563561f05f40 con 0x563561fc4d30 2024-02-17T12:00:43.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.019+0000 7f64a7865740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563561fbe060 con 0x563561fc4380 2024-02-17T12:00:43.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.019+0000 7f649ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fcd8b0 0x563561fd48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.019+0000 7f649ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fcd8b0 0x563561fd48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43682/0 (socket says 172.21.15.94:43682) 2024-02-17T12:00:43.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.019+0000 7f649dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4d30 0x563561fc0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.019+0000 7f649ed9d640 1 -- 172.21.15.94:0/1508923876 learned_addr learned my addr 172.21.15.94:0/1508923876 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:43.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.019+0000 7f649e59c640 1 --2- 172.21.15.94:0/1508923876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x563561fcb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.020+0000 7f649e59c640 1 -- 172.21.15.94:0/1508923876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4d30 msgr2=0x563561fc0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:43.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.020+0000 7f649e59c640 1 --2- 172.21.15.94:0/1508923876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4d30 0x563561fc0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.020+0000 7f649e59c640 1 -- 172.21.15.94:0/1508923876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fcd8b0 msgr2=0x563561fd48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:43.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.020+0000 7f649e59c640 1 --2- 172.21.15.94:0/1508923876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fcd8b0 0x563561fd48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.020+0000 7f649e59c640 1 -- 172.21.15.94:0/1508923876 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563561ec6c00 con 0x563561fc4380 2024-02-17T12:00:43.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.020+0000 7f649ed9d640 1 --2- 172.21.15.94:0/1508923876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fcd8b0 0x563561fd48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:00:43.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.020+0000 7f649e59c640 1 --2- 172.21.15.94:0/1508923876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x563561fcb1e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x563561fe0950 tx=0x7f6490019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13a486365e3d050e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:43.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.020+0000 7f6487fff640 1 -- 172.21.15.94:0/1508923876 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490023030 con 0x563561fc4380 2024-02-17T12:00:43.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.020+0000 7f6487fff640 1 -- 172.21.15.94:0/1508923876 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6490024040 con 0x563561fc4380 2024-02-17T12:00:43.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.020+0000 7f6487fff640 1 -- 172.21.15.94:0/1508923876 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490017070 con 0x563561fc4380 2024-02-17T12:00:43.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.022+0000 7f64a7865740 1 -- 172.21.15.94:0/1508923876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 msgr2=0x563561fcb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:43.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.022+0000 7f64a7865740 1 --2- 172.21.15.94:0/1508923876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x563561fcb1e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x563561fe0950 tx=0x7f6490019000 comp rx=0 tx=0).stop 2024-02-17T12:00:43.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.022+0000 7f64a7865740 1 -- 172.21.15.94:0/1508923876 shutdown_connections 2024-02-17T12:00:43.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.022+0000 7f64a7865740 1 --2- 172.21.15.94:0/1508923876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fcd8b0 0x563561fd48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.022+0000 7f64a7865740 1 --2- 172.21.15.94:0/1508923876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4d30 0x563561fc0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.022+0000 7f64a7865740 1 --2- 172.21.15.94:0/1508923876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x563561fcb1e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.022+0000 7f64a7865740 1 -- 172.21.15.94:0/1508923876 >> 172.21.15.94:0/1508923876 conn(0x563561fbe840 msgr2=0x563561fbfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:43.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.022+0000 7f64a7865740 1 -- 172.21.15.94:0/1508923876 shutdown_connections 2024-02-17T12:00:43.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.022+0000 7f64a7865740 1 -- 172.21.15.94:0/1508923876 wait complete. 2024-02-17T12:00:43.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.022+0000 7f64a7865740 1 -- start start 2024-02-17T12:00:43.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.023+0000 7f64a7865740 1 Processor -- start 2024-02-17T12:00:43.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.023+0000 7f64a7865740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x56356202d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.023+0000 7f64a7865740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4d30 0x56356202dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.023+0000 7f64a7865740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fcd8b0 0x563561fd3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.023+0000 7f64a7865740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563561fd4560 con 0x563561fcd8b0 2024-02-17T12:00:43.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.023+0000 7f649dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4d30 0x56356202dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f64a7865740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563561fd46d0 con 0x563561fc4d30 2024-02-17T12:00:43.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f64a7865740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563561fd4890 con 0x563561fc4380 2024-02-17T12:00:43.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f649dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4d30 0x56356202dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36910/0 (socket says 172.21.15.94:36910) 2024-02-17T12:00:43.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f649dd9b640 1 -- 172.21.15.94:0/1752879482 learned_addr learned my addr 172.21.15.94:0/1752879482 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:43.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f649e59c640 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x56356202d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f649ed9d640 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fcd8b0 0x563561fd3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f64a7865740 10 client.0 _ll_register_callbacks cb 0x7f6498009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:43.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f649e59c640 1 -- 172.21.15.94:0/1752879482 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4d30 msgr2=0x56356202dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:43.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f649e59c640 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4d30 0x56356202dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f649e59c640 1 -- 172.21.15.94:0/1752879482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fcd8b0 msgr2=0x563561fd3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:43.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f649e59c640 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fcd8b0 0x563561fd3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f649e59c640 1 -- 172.21.15.94:0/1752879482 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563561eee0b0 con 0x563561fc4380 2024-02-17T12:00:43.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f649ed9d640 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fcd8b0 0x563561fd3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:00:43.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f649e59c640 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x56356202d710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f649001d5e0 tx=0x7f649001df60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e06abb967cdbb3b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:43.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.024+0000 7f6485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:00:43.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.025+0000 7f6485ffb640 1 -- 172.21.15.94:0/1752879482 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490023070 con 0x563561fc4380 2024-02-17T12:00:43.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.025+0000 7f6485ffb640 1 -- 172.21.15.94:0/1752879482 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f649002a630 con 0x563561fc4380 2024-02-17T12:00:43.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.025+0000 7f64a7865740 1 -- 172.21.15.94:0/1752879482 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563561f51900 con 0x563561fc4380 2024-02-17T12:00:43.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.025+0000 7f6485ffb640 1 -- 172.21.15.94:0/1752879482 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6490017040 con 0x563561fc4380 2024-02-17T12:00:43.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.026+0000 7f64a7865740 10 client.25723 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:43.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.026+0000 7f64a7865740 1 -- 172.21.15.94:0/1752879482 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x7f649001d690 con 0x563561fc4380 2024-02-17T12:00:43.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.027+0000 7f6485ffb640 1 -- 172.21.15.94:0/1752879482 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6490038070 con 0x563561fc4380 2024-02-17T12:00:43.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.027+0000 7f6485ffb640 1 -- 172.21.15.94:0/1752879482 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6490031020 con 0x563561fc4380 2024-02-17T12:00:43.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.027+0000 7f64a7865740 10 client.25723 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:43.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.027+0000 7f64a7865740 10 client.25723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:43.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.027+0000 7f64a7865740 20 client.25723 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:43.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.027+0000 7f64a7865740 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563562036650 0x563562038a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.027+0000 7f64a7865740 4 client.25723 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:00:43.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.027+0000 7f64a7865740 1 -- 172.21.15.94:0/1752879482 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563561f51900 con 0x563562036650 2024-02-17T12:00:43.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.027+0000 7f649dd9b640 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563562036650 0x563562038a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.028+0000 7f649dd9b640 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563562036650 0x563562038a40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80bd05a170875322 server_cookie=84ea6ced61831763 in_seq=0 out_seq=0 2024-02-17T12:00:43.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.028+0000 7f6485ffb640 10 client.25723 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:43.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.030+0000 7f6485ffb640 1 -- 172.21.15.94:0/1752879482 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563561f51900 con 0x563562036650 2024-02-17T12:00:43.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.030+0000 7f6485ffb640 10 client.25723 handle_command_reply: tid=0 2024-02-17T12:00:43.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.030+0000 7f64a7865740 1 client.25723 shutdown 2024-02-17T12:00:43.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.031+0000 7f64a7865740 1 -- 172.21.15.94:0/1752879482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 msgr2=0x56356202d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:43.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.031+0000 7f64a7865740 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x56356202d710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f649001d5e0 tx=0x7f649001df60 comp rx=0 tx=0).stop 2024-02-17T12:00:43.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.031+0000 7f64a7865740 1 -- 172.21.15.94:0/1752879482 shutdown_connections 2024-02-17T12:00:43.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.031+0000 7f64a7865740 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563562036650 0x563562038a40 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.031+0000 7f64a7865740 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563561fcd8b0 0x563561fd3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.031+0000 7f64a7865740 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563561fc4d30 0x56356202dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.031+0000 7f64a7865740 1 --2- 172.21.15.94:0/1752879482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563561fc4380 0x56356202d710 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.031+0000 7f64a7865740 1 -- 172.21.15.94:0/1752879482 >> 172.21.15.94:0/1752879482 conn(0x563561fbe840 msgr2=0x563561fc0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:43.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.031+0000 7f6485ffb640 0 client.25723 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:43.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.032+0000 7f64a7865740 1 -- 172.21.15.94:0/1752879482 shutdown_connections 2024-02-17T12:00:43.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.032+0000 7f64a7865740 1 -- 172.21.15.94:0/1752879482 wait complete. 2024-02-17T12:00:43.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.032+0000 7f64a7865740 20 client.25723 trim_cache size 0 max 16384 2024-02-17T12:00:43.034 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:00:43.034 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:00:43.034 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "db02b663-394c-413c-a1cf-b816f9704140", 2024-02-17T12:00:43.034 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:00:43.034 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:00:43.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.033+0000 7f64a4c88640 1 -- 172.21.15.94:0/3428730431 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f64880a22b0 msgr2=0x7f64880a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:43.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.033+0000 7f64a4c88640 1 --2- 172.21.15.94:0/3428730431 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f64880a22b0 0x7f64880a4770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x563561fbfac0 tx=0x7f6490005f90 comp rx=0 tx=0).stop 2024-02-17T12:00:43.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.033+0000 7f64a4c88640 1 -- 172.21.15.94:0/3428730431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64a010f9a0 msgr2=0x7f64a01a6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:43.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.033+0000 7f64a4c88640 1 --2- 172.21.15.94:0/3428730431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64a010f9a0 0x7f64a01a6b90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6494005ab0 tx=0x7f649402c6b0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.033+0000 7f64a4c88640 1 -- 172.21.15.94:0/3428730431 shutdown_connections 2024-02-17T12:00:43.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.033+0000 7f64a4c88640 1 --2- 172.21.15.94:0/3428730431 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f64880a22b0 0x7f64880a4770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.033+0000 7f64a4c88640 1 --2- 172.21.15.94:0/3428730431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64a011a960 0x7f64a01aaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.033+0000 7f64a4c88640 1 --2- 172.21.15.94:0/3428730431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f64a010f9a0 0x7f64a01a6b90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.033+0000 7f64a4c88640 1 --2- 172.21.15.94:0/3428730431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f64a010d9d0 0x7f64a01a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.033+0000 7f64a4c88640 1 -- 172.21.15.94:0/3428730431 >> 172.21.15.94:0/3428730431 conn(0x7f64a006c260 msgr2=0x7f64a0071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:43.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.034+0000 7f64a4c88640 1 -- 172.21.15.94:0/3428730431 shutdown_connections 2024-02-17T12:00:43.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.034+0000 7f64a4c88640 1 -- 172.21.15.94:0/3428730431 wait complete. 2024-02-17T12:00:43.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:00:43.050 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:00:43.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.215+0000 7fef12847640 1 -- 172.21.15.94:0/2227496289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef0c101d30 msgr2=0x7fef0c105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:43.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.215+0000 7fef12847640 1 --2- 172.21.15.94:0/2227496289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef0c101d30 0x7fef0c105d70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fef00009930 tx=0x7fef0002bc30 comp rx=0 tx=0).stop 2024-02-17T12:00:43.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.215+0000 7fef12847640 1 -- 172.21.15.94:0/2227496289 shutdown_connections 2024-02-17T12:00:43.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.215+0000 7fef12847640 1 --2- 172.21.15.94:0/2227496289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef0c106450 0x7fef0c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.215+0000 7fef12847640 1 --2- 172.21.15.94:0/2227496289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef0c101d30 0x7fef0c105d70 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.215+0000 7fef12847640 1 --2- 172.21.15.94:0/2227496289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef0c101380 0x7fef0c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.215+0000 7fef12847640 1 -- 172.21.15.94:0/2227496289 >> 172.21.15.94:0/2227496289 conn(0x7fef0c0fcc40 msgr2=0x7fef0c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:43.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.215+0000 7fef12847640 1 -- 172.21.15.94:0/2227496289 shutdown_connections 2024-02-17T12:00:43.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.215+0000 7fef12847640 1 -- 172.21.15.94:0/2227496289 wait complete. 2024-02-17T12:00:43.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.216+0000 7fef12847640 1 Processor -- start 2024-02-17T12:00:43.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.216+0000 7fef12847640 1 -- start start 2024-02-17T12:00:43.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.216+0000 7fef12847640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef0c101380 0x7fef0c195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.216+0000 7fef12847640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef0c106450 0x7fef0c195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.216+0000 7fef12847640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef0c199e80 0x7fef0c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.216+0000 7fef12847640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c19a870 con 0x7fef0c106450 2024-02-17T12:00:43.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.216+0000 7fef12847640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c19a9e0 con 0x7fef0c101380 2024-02-17T12:00:43.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.216+0000 7fef12847640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c19ab50 con 0x7fef0c199e80 2024-02-17T12:00:43.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.216+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef0c101380 0x7fef0c195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.216+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef0c101380 0x7fef0c195610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36926/0 (socket says 172.21.15.94:36926) 2024-02-17T12:00:43.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.216+0000 7fef0bfff640 1 -- 172.21.15.94:0/4067469813 learned_addr learned my addr 172.21.15.94:0/4067469813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:43.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.216+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef0c106450 0x7fef0c195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.216+0000 7fef10de4640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef0c199e80 0x7fef0c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.217+0000 7fef0bfff640 1 -- 172.21.15.94:0/4067469813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef0c199e80 msgr2=0x7fef0c19a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:43.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.217+0000 7fef0bfff640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef0c199e80 0x7fef0c19a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.217+0000 7fef0bfff640 1 -- 172.21.15.94:0/4067469813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef0c106450 msgr2=0x7fef0c195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:43.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.217+0000 7fef0bfff640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef0c106450 0x7fef0c195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.217+0000 7fef0bfff640 1 -- 172.21.15.94:0/4067469813 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef00009560 con 0x7fef0c101380 2024-02-17T12:00:43.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.217+0000 7fef0bfff640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef0c101380 0x7fef0c195610 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feefc00e9f0 tx=0x7feefc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:43.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.218+0000 7fef097fa640 1 -- 172.21.15.94:0/4067469813 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc009700 con 0x7fef0c101380 2024-02-17T12:00:43.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.218+0000 7fef097fa640 1 -- 172.21.15.94:0/4067469813 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feefc00cd80 con 0x7fef0c101380 2024-02-17T12:00:43.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.219+0000 7fef097fa640 1 -- 172.21.15.94:0/4067469813 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc018920 con 0x7fef0c101380 2024-02-17T12:00:43.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.219+0000 7fef12847640 1 -- 172.21.15.94:0/4067469813 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0c06aba0 con 0x7fef0c101380 2024-02-17T12:00:43.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.219+0000 7fef12847640 1 -- 172.21.15.94:0/4067469813 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0c06afd0 con 0x7fef0c101380 2024-02-17T12:00:43.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.222+0000 7fef097fa640 1 -- 172.21.15.94:0/4067469813 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7feefc018a80 con 0x7fef0c101380 2024-02-17T12:00:43.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.223+0000 7fef097fa640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feef40a6850 0x7feef40a8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.224+0000 7fef15418740 1 Processor -- start 2024-02-17T12:00:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.224+0000 7fef15418740 1 -- start start 2024-02-17T12:00:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.224+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feef40a6850 0x7feef40a8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.223+0000 7fef097fa640 1 -- 172.21.15.94:0/4067469813 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7feefc014070 con 0x7fef0c101380 2024-02-17T12:00:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.224+0000 7fef15418740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e89f7f0 0x55ea7e89fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.224+0000 7fef15418740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8a0110 0x55ea7e8992e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.224+0000 7fef15418740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e8b1a30 0x55ea7e8b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feef40a6850 0x7feef40a8d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fef00009850 tx=0x7fef00000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:43.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef15418740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e806280 con 0x55ea7e8a0110 2024-02-17T12:00:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e89f7f0 0x55ea7e89fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e89f7f0 0x55ea7e89fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36934/0 (socket says 172.21.15.94:36934) 2024-02-17T12:00:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef0bfff640 1 -- 172.21.15.94:0/1591680664 learned_addr learned my addr 172.21.15.94:0/1591680664 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/1591680664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8a0110 0x55ea7e8992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef10de4640 1 --2- 172.21.15.94:0/1591680664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e8b1a30 0x55ea7e8b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef15418740 1 -- 172.21.15.94:0/1591680664 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e8037b0 con 0x55ea7e89f7f0 2024-02-17T12:00:43.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef15418740 1 -- 172.21.15.94:0/1591680664 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e7fef90 con 0x55ea7e8b1a30 2024-02-17T12:00:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef10de4640 1 -- 172.21.15.94:0/1591680664 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e89f7f0 msgr2=0x55ea7e89fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef10de4640 1 --2- 172.21.15.94:0/1591680664 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e89f7f0 0x55ea7e89fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef10de4640 1 -- 172.21.15.94:0/1591680664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8a0110 msgr2=0x55ea7e8992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef10de4640 1 --2- 172.21.15.94:0/1591680664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8a0110 0x55ea7e8992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.225+0000 7fef10de4640 1 -- 172.21.15.94:0/1591680664 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea7e7bd570 con 0x55ea7e8b1a30 2024-02-17T12:00:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.226+0000 7fef10de4640 1 --2- 172.21.15.94:0/1591680664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e8b1a30 0x55ea7e8b3e20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef04009da0 tx=0x7fef04021750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2118d2674514b11 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.226+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/1591680664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8a0110 0x55ea7e8992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:00:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.226+0000 7feef27fc640 1 -- 172.21.15.94:0/1591680664 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04004d20 con 0x55ea7e8b1a30 2024-02-17T12:00:43.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.227+0000 7feef27fc640 1 -- 172.21.15.94:0/1591680664 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef04026070 con 0x55ea7e8b1a30 2024-02-17T12:00:43.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.227+0000 7feef27fc640 1 -- 172.21.15.94:0/1591680664 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0402b800 con 0x55ea7e8b1a30 2024-02-17T12:00:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.228+0000 7fef15418740 1 -- 172.21.15.94:0/1591680664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e8b1a30 msgr2=0x55ea7e8b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.228+0000 7fef15418740 1 --2- 172.21.15.94:0/1591680664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e8b1a30 0x55ea7e8b3e20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef04009da0 tx=0x7fef04021750 comp rx=0 tx=0).stop 2024-02-17T12:00:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.228+0000 7fef15418740 1 -- 172.21.15.94:0/1591680664 shutdown_connections 2024-02-17T12:00:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.228+0000 7fef15418740 1 --2- 172.21.15.94:0/1591680664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e8b1a30 0x55ea7e8b3e20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.228+0000 7fef15418740 1 --2- 172.21.15.94:0/1591680664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8a0110 0x55ea7e8992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.228+0000 7fef15418740 1 --2- 172.21.15.94:0/1591680664 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e89f7f0 0x55ea7e89fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.228+0000 7fef15418740 1 -- 172.21.15.94:0/1591680664 >> 172.21.15.94:0/1591680664 conn(0x55ea7e89a1a0 msgr2=0x55ea7e89a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.228+0000 7fef15418740 1 -- 172.21.15.94:0/1591680664 shutdown_connections 2024-02-17T12:00:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.228+0000 7fef15418740 1 -- 172.21.15.94:0/1591680664 wait complete. 2024-02-17T12:00:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.229+0000 7fef15418740 1 -- start start 2024-02-17T12:00:43.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.229+0000 7fef15418740 1 Processor -- start 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.229+0000 7fef15418740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e89f7f0 0x55ea7e899200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.229+0000 7fef15418740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8a0110 0x55ea7e896590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.229+0000 7fef15418740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8b1a30 0x55ea7e896ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.229+0000 7fef15418740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e4d2430 con 0x55ea7e8b1a30 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.229+0000 7fef15418740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e48ee00 con 0x55ea7e8a0110 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.229+0000 7fef15418740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e783fd0 con 0x55ea7e89f7f0 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.229+0000 7fef15418740 10 client.0 _ll_register_callbacks cb 0x55ea7e7d9fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e89f7f0 0x55ea7e899200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e89f7f0 0x55ea7e899200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39948/0 (socket says 172.21.15.94:39948) 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7fef0bfff640 1 -- 172.21.15.94:0/3525450638 learned_addr learned my addr 172.21.15.94:0/3525450638 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7fef10de4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8b1a30 0x55ea7e896ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/3525450638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8a0110 0x55ea7e896590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7fef0bfff640 1 -- 172.21.15.94:0/3525450638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8a0110 msgr2=0x55ea7e896590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7fef0bfff640 1 --2- 172.21.15.94:0/3525450638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8a0110 0x55ea7e896590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7fef0bfff640 1 -- 172.21.15.94:0/3525450638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8b1a30 msgr2=0x55ea7e896ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:43.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7fef0bfff640 1 --2- 172.21.15.94:0/3525450638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8b1a30 0x55ea7e896ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7fef0bfff640 1 -- 172.21.15.94:0/3525450638 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef04009a00 con 0x55ea7e89f7f0 2024-02-17T12:00:43.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7fef0bfff640 1 --2- 172.21.15.94:0/3525450638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e89f7f0 0x55ea7e899200 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feefc00c6a0 tx=0x7feefc09ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f15718f556bc87 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:43.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7feeeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:00:43.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7feeeb7fe640 1 -- 172.21.15.94:0/3525450638 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc0cc430 con 0x55ea7e89f7f0 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.230+0000 7feeeb7fe640 1 -- 172.21.15.94:0/3525450638 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7feefc09f160 con 0x55ea7e89f7f0 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.231+0000 7fef15418740 1 -- 172.21.15.94:0/3525450638 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea7e4fe7e0 con 0x55ea7e89f7f0 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.231+0000 7feeeb7fe640 1 -- 172.21.15.94:0/3525450638 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7feefc0a39f0 con 0x55ea7e89f7f0 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.231+0000 7fef15418740 10 client.25735 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.231+0000 7fef15418740 1 -- 172.21.15.94:0/3525450638 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55ea7e7fd610 con 0x55ea7e89f7f0 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.231+0000 7feeeb7fe640 1 -- 172.21.15.94:0/3525450638 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feefc09a6a0 con 0x55ea7e89f7f0 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.231+0000 7feeeb7fe640 1 -- 172.21.15.94:0/3525450638 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feefc0b2070 con 0x55ea7e89f7f0 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.231+0000 7fef15418740 10 client.25735 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.232+0000 7fef15418740 10 client.25735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.232+0000 7fef15418740 20 client.25735 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.232+0000 7fef15418740 1 --2- 172.21.15.94:0/3525450638 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ea7e8c19a0 0x55ea7e8c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.232+0000 7fef15418740 4 client.25735 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.232+0000 7fef15418740 1 -- 172.21.15.94:0/3525450638 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea7e4fe7e0 con 0x55ea7e8c19a0 2024-02-17T12:00:43.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.232+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/3525450638 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ea7e8c19a0 0x55ea7e8c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.233+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/3525450638 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ea7e8c19a0 0x55ea7e8c3df0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853c420ed93ad7e6 server_cookie=e041fa2e961bd51c in_seq=0 out_seq=0 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.233+0000 7feeeb7fe640 10 client.25735 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7feeeb7fe640 1 -- 172.21.15.94:0/3525450638 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ea7e4fe7e0 con 0x55ea7e8c19a0 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7feeeb7fe640 10 client.25735 handle_command_reply: tid=0 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7fef15418740 1 client.25735 shutdown 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7fef15418740 1 -- 172.21.15.94:0/3525450638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e89f7f0 msgr2=0x55ea7e899200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7fef15418740 1 --2- 172.21.15.94:0/3525450638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e89f7f0 0x55ea7e899200 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feefc00c6a0 tx=0x7feefc09ba90 comp rx=0 tx=0).stop 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7fef15418740 1 -- 172.21.15.94:0/3525450638 shutdown_connections 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7fef15418740 1 --2- 172.21.15.94:0/3525450638 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ea7e8c19a0 0x55ea7e8c3df0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7fef15418740 1 --2- 172.21.15.94:0/3525450638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8b1a30 0x55ea7e896ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7fef15418740 1 --2- 172.21.15.94:0/3525450638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8a0110 0x55ea7e896590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7fef15418740 1 --2- 172.21.15.94:0/3525450638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e89f7f0 0x55ea7e899200 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7fef15418740 1 -- 172.21.15.94:0/3525450638 >> 172.21.15.94:0/3525450638 conn(0x55ea7e89a1a0 msgr2=0x55ea7e89bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:43.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7feeeb7fe640 0 client.25735 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7fef15418740 1 -- 172.21.15.94:0/3525450638 shutdown_connections 2024-02-17T12:00:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7fef15418740 1 -- 172.21.15.94:0/3525450638 wait complete. 2024-02-17T12:00:43.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.234+0000 7fef15418740 20 client.25735 trim_cache size 0 max 16384 2024-02-17T12:00:43.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.260+0000 7fef15418740 1 Processor -- start 2024-02-17T12:00:43.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.260+0000 7fef15418740 1 -- start start 2024-02-17T12:00:43.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.260+0000 7fef15418740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e89f7f0 0x55ea7e8b8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.260+0000 7fef15418740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8a0110 0x55ea7e8b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.260+0000 7fef15418740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8b1a30 0x55ea7e895e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.260+0000 7fef15418740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e4fe7e0 con 0x55ea7e8a0110 2024-02-17T12:00:43.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.260+0000 7fef15418740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e88d0f0 con 0x55ea7e8b1a30 2024-02-17T12:00:43.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.260+0000 7fef15418740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e89afa0 con 0x55ea7e89f7f0 2024-02-17T12:00:43.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.260+0000 7fef0b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8a0110 0x55ea7e8b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.261+0000 7fef0b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8a0110 0x55ea7e8b8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43734/0 (socket says 172.21.15.94:43734) 2024-02-17T12:00:43.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.261+0000 7fef0b7fe640 1 -- 172.21.15.94:0/3597128571 learned_addr learned my addr 172.21.15.94:0/3597128571 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:43.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.261+0000 7fef0bfff640 1 --2- 172.21.15.94:0/3597128571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e89f7f0 0x55ea7e8b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.261+0000 7fef10de4640 1 --2- 172.21.15.94:0/3597128571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8b1a30 0x55ea7e895e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.261+0000 7fef0b7fe640 1 -- 172.21.15.94:0/3597128571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e89f7f0 msgr2=0x55ea7e8b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:43.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.261+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/3597128571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e89f7f0 0x55ea7e8b8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.261+0000 7fef0b7fe640 1 -- 172.21.15.94:0/3597128571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8b1a30 msgr2=0x55ea7e895e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:43.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.261+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/3597128571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8b1a30 0x55ea7e895e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.261+0000 7fef0b7fe640 1 -- 172.21.15.94:0/3597128571 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea7e799ac0 con 0x55ea7e8a0110 2024-02-17T12:00:43.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.261+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/3597128571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8a0110 0x55ea7e8b8c80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fef00009900 tx=0x7fef00043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ef430b847ba26c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:43.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.261+0000 7feef1ffb640 1 -- 172.21.15.94:0/3597128571 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00054b00 con 0x55ea7e8a0110 2024-02-17T12:00:43.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.261+0000 7feef1ffb640 1 -- 172.21.15.94:0/3597128571 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef00054c60 con 0x55ea7e8a0110 2024-02-17T12:00:43.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.261+0000 7feef1ffb640 1 -- 172.21.15.94:0/3597128571 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef000035f0 con 0x55ea7e8a0110 2024-02-17T12:00:43.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.262+0000 7fef15418740 1 -- 172.21.15.94:0/3597128571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8a0110 msgr2=0x55ea7e8b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:43.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.262+0000 7fef15418740 1 --2- 172.21.15.94:0/3597128571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8a0110 0x55ea7e8b8c80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fef00009900 tx=0x7fef00043000 comp rx=0 tx=0).stop 2024-02-17T12:00:43.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.262+0000 7fef15418740 1 -- 172.21.15.94:0/3597128571 shutdown_connections 2024-02-17T12:00:43.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.262+0000 7fef15418740 1 --2- 172.21.15.94:0/3597128571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8b1a30 0x55ea7e895e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.262+0000 7fef15418740 1 --2- 172.21.15.94:0/3597128571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e8a0110 0x55ea7e8b8c80 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.262+0000 7fef15418740 1 --2- 172.21.15.94:0/3597128571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e89f7f0 0x55ea7e8b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.262+0000 7fef15418740 1 -- 172.21.15.94:0/3597128571 >> 172.21.15.94:0/3597128571 conn(0x55ea7e89a1a0 msgr2=0x55ea7e8901c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:43.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.262+0000 7fef15418740 1 -- 172.21.15.94:0/3597128571 shutdown_connections 2024-02-17T12:00:43.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.262+0000 7fef15418740 1 -- 172.21.15.94:0/3597128571 wait complete. 2024-02-17T12:00:43.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.262+0000 7fef15418740 1 -- start start 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.262+0000 7fef15418740 1 Processor -- start 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef15418740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e89f7f0 0x55ea7e8b7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef15418740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e8a0110 0x55ea7e8b7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef15418740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8b1a30 0x55ea7e917e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef15418740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e8b8290 con 0x55ea7e89f7f0 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef15418740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e9183c0 con 0x55ea7e8b1a30 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef15418740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea7e918500 con 0x55ea7e8a0110 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef15418740 10 client.0 _ll_register_callbacks cb 0x7fef04009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e89f7f0 0x55ea7e8b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e89f7f0 0x55ea7e8b7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43738/0 (socket says 172.21.15.94:43738) 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef0bfff640 1 -- 172.21.15.94:0/3913943316 learned_addr learned my addr 172.21.15.94:0/3913943316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e8a0110 0x55ea7e8b7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef10de4640 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8b1a30 0x55ea7e917e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef0bfff640 1 -- 172.21.15.94:0/3913943316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e8a0110 msgr2=0x55ea7e8b7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef0bfff640 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e8a0110 0x55ea7e8b7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef0bfff640 1 -- 172.21.15.94:0/3913943316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8b1a30 msgr2=0x55ea7e917e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef0bfff640 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8b1a30 0x55ea7e917e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef0bfff640 1 -- 172.21.15.94:0/3913943316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0c10e830 con 0x55ea7e89f7f0 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e8a0110 0x55ea7e8b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef0bfff640 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e89f7f0 0x55ea7e8b7510 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7feefc097c20 tx=0x7feefc09a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c942c94fda553e69 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7feeeb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7feeeb7fe640 1 -- 172.21.15.94:0/3913943316 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc09f160 con 0x55ea7e89f7f0 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7feeeb7fe640 1 -- 172.21.15.94:0/3913943316 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7feefc0a39f0 con 0x55ea7e89f7f0 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef15418740 1 -- 172.21.15.94:0/3913943316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef04026070 con 0x55ea7e89f7f0 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7feeeb7fe640 1 -- 172.21.15.94:0/3913943316 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7feefc0a0430 con 0x55ea7e89f7f0 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef15418740 10 client.16248 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.263+0000 7fef15418740 1 -- 172.21.15.94:0/3913943316 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55ea7e7bf6a0 con 0x55ea7e89f7f0 2024-02-17T12:00:43.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.264+0000 7feeeb7fe640 1 -- 172.21.15.94:0/3913943316 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fef04026070 con 0x55ea7e89f7f0 2024-02-17T12:00:43.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.264+0000 7feeeb7fe640 1 -- 172.21.15.94:0/3913943316 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feefc01f020 con 0x55ea7e89f7f0 2024-02-17T12:00:43.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.264+0000 7fef15418740 10 client.16248 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:43.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.264+0000 7fef15418740 10 client.16248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:43.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.264+0000 7fef15418740 20 client.16248 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:43.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.264+0000 7fef15418740 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ea7e919a00 0x55ea7e91bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:43.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.264+0000 7fef15418740 4 client.16248 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:00:43.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.264+0000 7fef15418740 1 -- 172.21.15.94:0/3913943316 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ea7e53b720 con 0x55ea7e919a00 2024-02-17T12:00:43.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.264+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ea7e919a00 0x55ea7e91bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:43.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.264+0000 7fef0b7fe640 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ea7e919a00 0x55ea7e91bdf0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7e36afb278f24a server_cookie=70a74ee1adbbed7b in_seq=0 out_seq=0 2024-02-17T12:00:43.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.264+0000 7feeeb7fe640 10 client.16248 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:43.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7feeeb7fe640 1 -- 172.21.15.94:0/3913943316 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ea7e53b720 con 0x55ea7e919a00 2024-02-17T12:00:43.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7feeeb7fe640 10 client.16248 handle_command_reply: tid=0 2024-02-17T12:00:43.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7fef15418740 1 client.16248 shutdown 2024-02-17T12:00:43.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7fef15418740 1 -- 172.21.15.94:0/3913943316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e89f7f0 msgr2=0x55ea7e8b7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:43.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7fef15418740 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e89f7f0 0x55ea7e8b7510 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7feefc097c20 tx=0x7feefc09a8d0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7fef15418740 1 -- 172.21.15.94:0/3913943316 shutdown_connections 2024-02-17T12:00:43.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7fef15418740 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ea7e919a00 0x55ea7e91bdf0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7fef15418740 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ea7e8b1a30 0x55ea7e917e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7fef15418740 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ea7e8a0110 0x55ea7e8b7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7feeeb7fe640 0 client.16248 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:43.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7fef15418740 1 --2- 172.21.15.94:0/3913943316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ea7e89f7f0 0x55ea7e8b7510 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7fef15418740 1 -- 172.21.15.94:0/3913943316 >> 172.21.15.94:0/3913943316 conn(0x55ea7e89a1a0 msgr2=0x55ea7e89b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:43.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7fef15418740 1 -- 172.21.15.94:0/3913943316 shutdown_connections 2024-02-17T12:00:43.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7fef15418740 1 -- 172.21.15.94:0/3913943316 wait complete. 2024-02-17T12:00:43.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.265+0000 7fef15418740 20 client.16248 trim_cache size 0 max 16384 2024-02-17T12:00:43.273 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:00:43.274 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:00:43.274 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:00:43.274 INFO:teuthology.orchestra.run.smithi094.stdout: "db02b663-394c-413c-a1cf-b816f9704140": { 2024-02-17T12:00:43.274 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:00:43.274 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "db02b663-394c-413c-a1cf-b816f9704140", 2024-02-17T12:00:43.274 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:00:43.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:00:43.274 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:00:43.274 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:00:43.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.266+0000 7fef12847640 1 -- 172.21.15.94:0/4067469813 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feef40a6850 msgr2=0x7feef40a8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:43.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.266+0000 7fef12847640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feef40a6850 0x7feef40a8d10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fef00009850 tx=0x7fef00000bc0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.266+0000 7fef12847640 1 -- 172.21.15.94:0/4067469813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef0c101380 msgr2=0x7fef0c195610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:43.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.266+0000 7fef12847640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef0c101380 0x7fef0c195610 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7feefc00e9f0 tx=0x7feefc00eeb0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.266+0000 7fef12847640 1 -- 172.21.15.94:0/4067469813 shutdown_connections 2024-02-17T12:00:43.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.266+0000 7fef12847640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feef40a6850 0x7feef40a8d10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.266+0000 7fef12847640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef0c199e80 0x7fef0c19a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.266+0000 7fef12847640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fef0c106450 0x7fef0c195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.266+0000 7fef12847640 1 --2- 172.21.15.94:0/4067469813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fef0c101380 0x7fef0c195610 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:43.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.266+0000 7fef12847640 1 -- 172.21.15.94:0/4067469813 >> 172.21.15.94:0/4067469813 conn(0x7fef0c0fcc40 msgr2=0x7fef0c0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:43.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.266+0000 7fef12847640 1 -- 172.21.15.94:0/4067469813 shutdown_connections 2024-02-17T12:00:43.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:00:43.266+0000 7fef12847640 1 -- 172.21.15.94:0/4067469813 wait complete. 2024-02-17T12:00:43.277 INFO:tasks.cephfs.filesystem:scrub status for tag:db02b663-394c-413c-a1cf-b816f9704140 - {'path': '/', 'tag': 'db02b663-394c-413c-a1cf-b816f9704140', 'options': 'recursive,force'} 2024-02-17T12:00:44.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:43 smithi189 ceph-mon[36588]: pgmap v81: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.4 MiB/s wr, 827 op/s 2024-02-17T12:00:44.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:43 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:00:44.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:43 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:00:44.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:43 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:00:44.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:43 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:00:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:43 smithi125 ceph-mon[36819]: pgmap v81: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.4 MiB/s wr, 827 op/s 2024-02-17T12:00:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:43 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:00:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:43 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:00:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:43 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:00:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:43 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:00:44.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:43 smithi094 ceph-mon[32499]: pgmap v81: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.4 MiB/s wr, 827 op/s 2024-02-17T12:00:44.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:43 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:00:44.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:43 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:00:44.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:43 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:00:44.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:43 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:00:45.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 145.0 s, 3876.0 tps, lat 25.820 ms stddev 27.442 2024-02-17T12:00:46.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:45 smithi189 ceph-mon[36588]: pgmap v82: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.0 MiB/s wr, 737 op/s 2024-02-17T12:00:46.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:45 smithi125 ceph-mon[36819]: pgmap v82: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.0 MiB/s wr, 737 op/s 2024-02-17T12:00:46.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:45 smithi094 ceph-mon[32499]: pgmap v82: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 9.0 MiB/s wr, 737 op/s 2024-02-17T12:00:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.871+0000 7f2feaaa3640 1 -- 172.21.15.94:0/2409542168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fe4106500 msgr2=0x7f2fe410e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.871+0000 7f2feaaa3640 1 --2- 172.21.15.94:0/2409542168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fe4106500 0x7f2fe410e060 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009ca0 tx=0x7f2fdc02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.874+0000 7f2feaaa3640 1 -- 172.21.15.94:0/2409542168 shutdown_connections 2024-02-17T12:00:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.875+0000 7f2feaaa3640 1 --2- 172.21.15.94:0/2409542168 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fe4106500 0x7f2fe410e060 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.875+0000 7f2feaaa3640 1 --2- 172.21.15.94:0/2409542168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fe41002a0 0x7f2fe4105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.875+0000 7f2feaaa3640 1 --2- 172.21.15.94:0/2409542168 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe40ff8f0 0x7f2fe40ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.875+0000 7f2feaaa3640 1 -- 172.21.15.94:0/2409542168 >> 172.21.15.94:0/2409542168 conn(0x7f2fe4076610 msgr2=0x7f2fe40fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.875+0000 7f2feaaa3640 1 -- 172.21.15.94:0/2409542168 shutdown_connections 2024-02-17T12:00:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.875+0000 7f2feaaa3640 1 -- 172.21.15.94:0/2409542168 wait complete. 2024-02-17T12:00:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.876+0000 7f2feaaa3640 1 Processor -- start 2024-02-17T12:00:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.876+0000 7f2feaaa3640 1 -- start start 2024-02-17T12:00:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.877+0000 7f2feaaa3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fe40ff8f0 0x7f2fe41976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.877+0000 7f2feaaa3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe4106500 0x7f2fe4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.877+0000 7f2feaaa3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fe419bf50 0x7f2fe419c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.877+0000 7f2fe3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe4106500 0x7f2fe4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.877+0000 7f2feaaa3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe419c940 con 0x7f2fe419bf50 2024-02-17T12:00:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.877+0000 7f2fe883f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fe40ff8f0 0x7f2fe41976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.878+0000 7f2fe3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe4106500 0x7f2fe4197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:60652/0 (socket says 172.21.15.94:60652) 2024-02-17T12:00:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.878+0000 7f2fe9040640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fe419bf50 0x7f2fe419c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.878+0000 7f2fe883f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fe40ff8f0 0x7f2fe41976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56288/0 (socket says 172.21.15.94:56288) 2024-02-17T12:00:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.878+0000 7f2fe883f640 1 -- 172.21.15.94:0/1168485096 learned_addr learned my addr 172.21.15.94:0/1168485096 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.878+0000 7f2feaaa3640 1 -- 172.21.15.94:0/1168485096 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe419cab0 con 0x7f2fe4106500 2024-02-17T12:00:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.878+0000 7f2feaaa3640 1 -- 172.21.15.94:0/1168485096 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe419cc20 con 0x7f2fe40ff8f0 2024-02-17T12:00:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.878+0000 7f2fe9040640 1 -- 172.21.15.94:0/1168485096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fe40ff8f0 msgr2=0x7f2fe41976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.878+0000 7f2fe9040640 1 --2- 172.21.15.94:0/1168485096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fe40ff8f0 0x7f2fe41976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.878+0000 7f2fe9040640 1 -- 172.21.15.94:0/1168485096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe4106500 msgr2=0x7f2fe4197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.878+0000 7f2fe9040640 1 --2- 172.21.15.94:0/1168485096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe4106500 0x7f2fe4197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.878+0000 7f2fe9040640 1 -- 172.21.15.94:0/1168485096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc009900 con 0x7f2fe419bf50 2024-02-17T12:00:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.880+0000 7f2fe9040640 1 --2- 172.21.15.94:0/1168485096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fe419bf50 0x7f2fe419c400 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc02c700 tx=0x7f2fdc034b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.882+0000 7f2fe1ffb640 1 -- 172.21.15.94:0/1168485096 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc0328b0 con 0x7f2fe419bf50 2024-02-17T12:00:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.882+0000 7f2fe1ffb640 1 -- 172.21.15.94:0/1168485096 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fdc043070 con 0x7f2fe419bf50 2024-02-17T12:00:47.881 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.882+0000 7f2fe1ffb640 1 -- 172.21.15.94:0/1168485096 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc03fd40 con 0x7f2fe419bf50 2024-02-17T12:00:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.882+0000 7f2feaaa3640 1 -- 172.21.15.94:0/1168485096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe4100790 con 0x7f2fe419bf50 2024-02-17T12:00:47.882 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.883+0000 7f2feaaa3640 1 -- 172.21.15.94:0/1168485096 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe4100b00 con 0x7f2fe419bf50 2024-02-17T12:00:47.885 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.887+0000 7f2fed670740 1 Processor -- start 2024-02-17T12:00:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.887+0000 7f2fe1ffb640 1 -- 172.21.15.94:0/1168485096 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2fdc00a040 con 0x7f2fe419bf50 2024-02-17T12:00:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.888+0000 7f2fe1ffb640 1 --2- 172.21.15.94:0/1168485096 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2fd40a24b0 0x7f2fd40a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.889+0000 7f2fe1ffb640 1 -- 172.21.15.94:0/1168485096 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2fdc045070 con 0x7f2fe419bf50 2024-02-17T12:00:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.889+0000 7f2fe3fff640 1 --2- 172.21.15.94:0/1168485096 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2fd40a24b0 0x7f2fd40a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.889+0000 7f2fed670740 1 -- start start 2024-02-17T12:00:47.888 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.889+0000 7f2fed670740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b863cdad0 0x562b863cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.890+0000 7f2fed670740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863ccd60 0x562b863cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.890+0000 7f2fed670740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863d62e0 0x562b863d86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.890+0000 7f2fed670740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8633b9f0 con 0x562b863ccd60 2024-02-17T12:00:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.890+0000 7f2fed670740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b86338f20 con 0x562b863cdad0 2024-02-17T12:00:47.889 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.890+0000 7f2fed670740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b86334700 con 0x562b863d62e0 2024-02-17T12:00:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.890+0000 7f2fe3fff640 1 --2- 172.21.15.94:0/1168485096 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2fd40a24b0 0x7f2fd40a4970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40fefc0 tx=0x7f2fd000b590 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.891+0000 7f2fe3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863ccd60 0x562b863cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.891+0000 7f2fe3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863ccd60 0x562b863cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53366/0 (socket says 172.21.15.94:53366) 2024-02-17T12:00:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.891+0000 7f2fe3fff640 1 -- 172.21.15.94:0/2402889916 learned_addr learned my addr 172.21.15.94:0/2402889916 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.892+0000 7f2fe883f640 1 --2- 172.21.15.94:0/2402889916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b863cdad0 0x562b863cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.892 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.892+0000 7f2fe9040640 1 --2- 172.21.15.94:0/2402889916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863d62e0 0x562b863d86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.894+0000 7f2fe883f640 1 -- 172.21.15.94:0/2402889916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863d62e0 msgr2=0x562b863d86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:47.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.894+0000 7f2fe883f640 1 --2- 172.21.15.94:0/2402889916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863d62e0 0x562b863d86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.894+0000 7f2fe883f640 1 -- 172.21.15.94:0/2402889916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863ccd60 msgr2=0x562b863cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.894+0000 7f2fe883f640 1 --2- 172.21.15.94:0/2402889916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863ccd60 0x562b863cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.894+0000 7f2fe883f640 1 -- 172.21.15.94:0/2402889916 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b862ca7b0 con 0x562b863cdad0 2024-02-17T12:00:47.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.895+0000 7f2fe883f640 1 --2- 172.21.15.94:0/2402889916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b863cdad0 0x562b863cc820 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009c80 tx=0x7f2fd8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86b23863a2618cf8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.897+0000 7f2fceffd640 1 -- 172.21.15.94:0/2402889916 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8004230 con 0x562b863cdad0 2024-02-17T12:00:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.897+0000 7f2fceffd640 1 -- 172.21.15.94:0/2402889916 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fd8004390 con 0x562b863cdad0 2024-02-17T12:00:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.897+0000 7f2fceffd640 1 -- 172.21.15.94:0/2402889916 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8004df0 con 0x562b863cdad0 2024-02-17T12:00:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.898+0000 7f2fed670740 1 -- 172.21.15.94:0/2402889916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b863cdad0 msgr2=0x562b863cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.898+0000 7f2fed670740 1 --2- 172.21.15.94:0/2402889916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b863cdad0 0x562b863cc820 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8009c80 tx=0x7f2fd8011810 comp rx=0 tx=0).stop 2024-02-17T12:00:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.898+0000 7f2fed670740 1 -- 172.21.15.94:0/2402889916 shutdown_connections 2024-02-17T12:00:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.898+0000 7f2fed670740 1 --2- 172.21.15.94:0/2402889916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863d62e0 0x562b863d86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.898+0000 7f2fed670740 1 --2- 172.21.15.94:0/2402889916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863ccd60 0x562b863cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.898+0000 7f2fed670740 1 --2- 172.21.15.94:0/2402889916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b863cdad0 0x562b863cc820 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.898+0000 7f2fed670740 1 -- 172.21.15.94:0/2402889916 >> 172.21.15.94:0/2402889916 conn(0x562b86394550 msgr2=0x562b863c7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.899+0000 7f2fed670740 1 -- 172.21.15.94:0/2402889916 shutdown_connections 2024-02-17T12:00:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.899+0000 7f2fed670740 1 -- 172.21.15.94:0/2402889916 wait complete. 2024-02-17T12:00:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.899+0000 7f2fed670740 1 -- start start 2024-02-17T12:00:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.899+0000 7f2fed670740 1 Processor -- start 2024-02-17T12:00:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.900+0000 7f2fed670740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863ccd60 0x562b863d3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.900+0000 7f2fed670740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b863cdad0 0x562b863d38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.901+0000 7f2fed670740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863d62e0 0x562b863d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.901+0000 7f2fe883f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863ccd60 0x562b863d3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.901+0000 7f2fe883f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863ccd60 0x562b863d3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56320/0 (socket says 172.21.15.94:56320) 2024-02-17T12:00:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.901+0000 7f2fe9040640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863d62e0 0x562b863d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.901+0000 7f2fe9040640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863d62e0 0x562b863d3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53378/0 (socket says 172.21.15.94:53378) 2024-02-17T12:00:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.901+0000 7f2fe883f640 1 -- 172.21.15.94:0/1336428705 learned_addr learned my addr 172.21.15.94:0/1336428705 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.901+0000 7f2fed670740 1 -- 172.21.15.94:0/1336428705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b862d9bf0 con 0x562b863d62e0 2024-02-17T12:00:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.901+0000 7f2fed670740 1 -- 172.21.15.94:0/1336428705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b86008250 con 0x562b863cdad0 2024-02-17T12:00:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.901+0000 7f2fed670740 1 -- 172.21.15.94:0/1336428705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b862ba200 con 0x562b863ccd60 2024-02-17T12:00:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.902+0000 7f2fe883f640 1 -- 172.21.15.94:0/1336428705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b863cdad0 msgr2=0x562b863d38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.902+0000 7f2fe883f640 1 --2- 172.21.15.94:0/1336428705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b863cdad0 0x562b863d38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.902+0000 7f2fe883f640 1 -- 172.21.15.94:0/1336428705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863d62e0 msgr2=0x562b863d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.902+0000 7f2fe883f640 1 --2- 172.21.15.94:0/1336428705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863d62e0 0x562b863d3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.902+0000 7f2fe883f640 1 -- 172.21.15.94:0/1336428705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd80098e0 con 0x562b863ccd60 2024-02-17T12:00:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.902+0000 7f2fed670740 10 client.0 _ll_register_callbacks cb 0x562b86328d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.902+0000 7f2fe883f640 1 --2- 172.21.15.94:0/1336428705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863ccd60 0x562b863d3380 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8019bf0 tx=0x7f2fd8019c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1263587426fb420b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.902+0000 7f2fc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:00:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.902+0000 7f2fc3fff640 1 -- 172.21.15.94:0/1336428705 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd800e7c0 con 0x562b863ccd60 2024-02-17T12:00:47.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.902+0000 7f2fc3fff640 1 -- 172.21.15.94:0/1336428705 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2fd80046c0 con 0x562b863ccd60 2024-02-17T12:00:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.903+0000 7f2fed670740 1 -- 172.21.15.94:0/1336428705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b86034600 con 0x562b863ccd60 2024-02-17T12:00:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.903+0000 7f2fc3fff640 1 -- 172.21.15.94:0/1336428705 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fd800e920 con 0x562b863ccd60 2024-02-17T12:00:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.903+0000 7f2fed670740 10 client.25753 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:47.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.903+0000 7f2fed670740 1 -- 172.21.15.94:0/1336428705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=21}) v3 -- 0x562b86332d80 con 0x562b863ccd60 2024-02-17T12:00:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.905+0000 7f2fc3fff640 1 -- 172.21.15.94:0/1336428705 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2fd800ea80 con 0x562b863ccd60 2024-02-17T12:00:47.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.905+0000 7f2fed670740 10 client.25753 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.905+0000 7f2fed670740 10 client.25753 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.906+0000 7f2fed670740 20 client.25753 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.906+0000 7f2fed670740 1 --2- 172.21.15.94:0/1336428705 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b863e6190 0x562b863e85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.906+0000 7f2fed670740 4 client.25753 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:00:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.906+0000 7f2fed670740 1 -- 172.21.15.94:0/1336428705 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b86034600 con 0x562b863e6190 2024-02-17T12:00:47.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.906+0000 7f2fe3fff640 1 --2- 172.21.15.94:0/1336428705 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b863e6190 0x562b863e85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.907+0000 7f2fc3fff640 10 client.25753 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:47.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.907+0000 7f2fe3fff640 1 --2- 172.21.15.94:0/1336428705 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b863e6190 0x562b863e85d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4b973cd83e15ea7 server_cookie=82adc44f31b17d6c in_seq=0 out_seq=0 2024-02-17T12:00:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.907+0000 7f2fc3fff640 1 -- 172.21.15.94:0/1336428705 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b86034600 con 0x562b863e6190 2024-02-17T12:00:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.907+0000 7f2fc3fff640 10 client.25753 handle_command_reply: tid=0 2024-02-17T12:00:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.907+0000 7f2fed670740 1 client.25753 shutdown 2024-02-17T12:00:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.907+0000 7f2fed670740 1 -- 172.21.15.94:0/1336428705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863ccd60 msgr2=0x562b863d3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.907+0000 7f2fed670740 1 --2- 172.21.15.94:0/1336428705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863ccd60 0x562b863d3380 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8019bf0 tx=0x7f2fd8019c20 comp rx=0 tx=0).stop 2024-02-17T12:00:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.908+0000 7f2fed670740 1 -- 172.21.15.94:0/1336428705 shutdown_connections 2024-02-17T12:00:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.908+0000 7f2fed670740 1 --2- 172.21.15.94:0/1336428705 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b863e6190 0x562b863e85d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.908+0000 7f2fc3fff640 0 client.25753 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.908+0000 7f2fed670740 1 --2- 172.21.15.94:0/1336428705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863d62e0 0x562b863d3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.908+0000 7f2fed670740 1 --2- 172.21.15.94:0/1336428705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b863cdad0 0x562b863d38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.908+0000 7f2fed670740 1 --2- 172.21.15.94:0/1336428705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863ccd60 0x562b863d3380 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.908+0000 7f2fed670740 1 -- 172.21.15.94:0/1336428705 >> 172.21.15.94:0/1336428705 conn(0x562b86394550 msgr2=0x562b863c97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.908+0000 7f2fed670740 1 -- 172.21.15.94:0/1336428705 shutdown_connections 2024-02-17T12:00:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.908+0000 7f2fed670740 1 -- 172.21.15.94:0/1336428705 wait complete. 2024-02-17T12:00:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.908+0000 7f2fed670740 20 client.25753 trim_cache size 0 max 16384 2024-02-17T12:00:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.926+0000 7f2fed670740 1 Processor -- start 2024-02-17T12:00:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.926+0000 7f2fed670740 1 -- start start 2024-02-17T12:00:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.926+0000 7f2fed670740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 0x562b863d5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.926+0000 7f2fed670740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863ccd60 0x562b863d3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.926+0000 7f2fed670740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863cdad0 0x562b863dd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.926+0000 7f2fed670740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b863e31b0 con 0x562b863ccd60 2024-02-17T12:00:47.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.926+0000 7f2fed670740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b863c8a50 con 0x562b86394550 2024-02-17T12:00:47.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.926+0000 7f2fed670740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b863ed820 con 0x562b863cdad0 2024-02-17T12:00:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.927+0000 7f2fe883f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 0x562b863d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.927+0000 7f2fe883f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 0x562b863d5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:60670/0 (socket says 172.21.15.94:60670) 2024-02-17T12:00:47.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.927+0000 7f2fe883f640 1 -- 172.21.15.94:0/3551382836 learned_addr learned my addr 172.21.15.94:0/3551382836 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.927+0000 7f2fe3fff640 1 --2- 172.21.15.94:0/3551382836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863ccd60 0x562b863d3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.927+0000 7f2fe9040640 1 --2- 172.21.15.94:0/3551382836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863cdad0 0x562b863dd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.927+0000 7f2fe883f640 1 -- 172.21.15.94:0/3551382836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863cdad0 msgr2=0x562b863dd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.927+0000 7f2fe883f640 1 --2- 172.21.15.94:0/3551382836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863cdad0 0x562b863dd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.927+0000 7f2fe883f640 1 -- 172.21.15.94:0/3551382836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863ccd60 msgr2=0x562b863d3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.927+0000 7f2fe883f640 1 --2- 172.21.15.94:0/3551382836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863ccd60 0x562b863d3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.927+0000 7f2fe883f640 1 -- 172.21.15.94:0/3551382836 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b862cf7b0 con 0x562b86394550 2024-02-17T12:00:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.927+0000 7f2fe9040640 1 --2- 172.21.15.94:0/3551382836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863cdad0 0x562b863dd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:00:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.927+0000 7f2fe883f640 1 --2- 172.21.15.94:0/3551382836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 0x562b863d5ab0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800b010 tx=0x7f2fd801e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb20772fb464de4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.927+0000 7f2fce7fc640 1 -- 172.21.15.94:0/3551382836 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd800e7c0 con 0x562b86394550 2024-02-17T12:00:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.928+0000 7f2fce7fc640 1 -- 172.21.15.94:0/3551382836 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fd800e920 con 0x562b86394550 2024-02-17T12:00:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.928+0000 7f2fce7fc640 1 -- 172.21.15.94:0/3551382836 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8018390 con 0x562b86394550 2024-02-17T12:00:47.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.928+0000 7f2fed670740 1 -- 172.21.15.94:0/3551382836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 msgr2=0x562b863d5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.928+0000 7f2fed670740 1 --2- 172.21.15.94:0/3551382836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 0x562b863d5ab0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800b010 tx=0x7f2fd801e000 comp rx=0 tx=0).stop 2024-02-17T12:00:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.928+0000 7f2fed670740 1 -- 172.21.15.94:0/3551382836 shutdown_connections 2024-02-17T12:00:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.928+0000 7f2fed670740 1 --2- 172.21.15.94:0/3551382836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863cdad0 0x562b863dd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.928+0000 7f2fed670740 1 --2- 172.21.15.94:0/3551382836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863ccd60 0x562b863d3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.928+0000 7f2fed670740 1 --2- 172.21.15.94:0/3551382836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 0x562b863d5ab0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.928+0000 7f2fed670740 1 -- 172.21.15.94:0/3551382836 >> 172.21.15.94:0/3551382836 conn(0x562b863d88b0 msgr2=0x562b863d1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.928+0000 7f2fed670740 1 -- 172.21.15.94:0/3551382836 shutdown_connections 2024-02-17T12:00:47.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.928+0000 7f2fed670740 1 -- 172.21.15.94:0/3551382836 wait complete. 2024-02-17T12:00:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.929+0000 7f2fed670740 1 -- start start 2024-02-17T12:00:47.930 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.929+0000 7f2fed670740 1 Processor -- start 2024-02-17T12:00:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.929+0000 7f2fed670740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 0x562b863d0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fed670740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863ccd60 0x562b863d06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fed670740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863cdad0 0x562b863d0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fed670740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8643c720 con 0x562b863cdad0 2024-02-17T12:00:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fed670740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8643c890 con 0x562b86394550 2024-02-17T12:00:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fed670740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8643ca50 con 0x562b863ccd60 2024-02-17T12:00:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fed670740 10 client.0 _ll_register_callbacks cb 0x7f2fd8009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:00:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fe883f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 0x562b863d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fe883f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 0x562b863d0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:60682/0 (socket says 172.21.15.94:60682) 2024-02-17T12:00:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fe883f640 1 -- 172.21.15.94:0/4145530601 learned_addr learned my addr 172.21.15.94:0/4145530601 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:00:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fe9040640 1 --2- 172.21.15.94:0/4145530601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863cdad0 0x562b863d0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fe3fff640 1 --2- 172.21.15.94:0/4145530601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863ccd60 0x562b863d06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fe883f640 1 -- 172.21.15.94:0/4145530601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863ccd60 msgr2=0x562b863d06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fe883f640 1 --2- 172.21.15.94:0/4145530601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863ccd60 0x562b863d06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fe883f640 1 -- 172.21.15.94:0/4145530601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863cdad0 msgr2=0x562b863d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:00:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fe883f640 1 --2- 172.21.15.94:0/4145530601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863cdad0 0x562b863d0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.930+0000 7f2fe883f640 1 -- 172.21.15.94:0/4145530601 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b86332d80 con 0x562b86394550 2024-02-17T12:00:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.931+0000 7f2fe883f640 1 --2- 172.21.15.94:0/4145530601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 0x562b863d0180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800ef90 tx=0x7f2fd8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4eb2bf30b7826cd7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:00:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.931+0000 7f2fc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:00:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.931+0000 7f2fc3fff640 1 -- 172.21.15.94:0/4145530601 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd800e7c0 con 0x562b86394550 2024-02-17T12:00:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.931+0000 7f2fc3fff640 1 -- 172.21.15.94:0/4145530601 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2fd80245b0 con 0x562b86394550 2024-02-17T12:00:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.931+0000 7f2fed670740 1 -- 172.21.15.94:0/4145530601 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fd8004390 con 0x562b86394550 2024-02-17T12:00:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.932+0000 7f2fc3fff640 1 -- 172.21.15.94:0/4145530601 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fd8009310 con 0x562b86394550 2024-02-17T12:00:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.933+0000 7f2fed670740 10 client.25712 fetch_fsmap learned FSMap version 21 2024-02-17T12:00:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.933+0000 7f2fed670740 1 -- 172.21.15.94:0/4145530601 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x562b862f5540 con 0x562b86394550 2024-02-17T12:00:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.933+0000 7f2fc3fff640 1 -- 172.21.15.94:0/4145530601 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2fd804b470 con 0x562b86394550 2024-02-17T12:00:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.933+0000 7f2fc3fff640 1 -- 172.21.15.94:0/4145530601 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2fd802e450 con 0x562b86394550 2024-02-17T12:00:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.933+0000 7f2fed670740 10 client.25712 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:00:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.933+0000 7f2fed670740 10 client.25712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:00:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.933+0000 7f2fed670740 20 client.25712 populate_metadata read hostname 'smithi094' 2024-02-17T12:00:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.933+0000 7f2fed670740 1 --2- 172.21.15.94:0/4145530601 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b863d62e0 0x562b8643fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:00:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.933+0000 7f2fed670740 4 client.25712 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:00:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.933+0000 7f2fed670740 1 -- 172.21.15.94:0/4145530601 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f2fd8004390 con 0x562b863d62e0 2024-02-17T12:00:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.933+0000 7f2fe3fff640 1 --2- 172.21.15.94:0/4145530601 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b863d62e0 0x562b8643fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:00:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.934+0000 7f2fc3fff640 10 client.25712 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.934+0000 7f2fe3fff640 1 --2- 172.21.15.94:0/4145530601 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b863d62e0 0x562b8643fd50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59bb6c2858875fd9 server_cookie=56fe3d8cbefe5197 in_seq=0 out_seq=0 2024-02-17T12:00:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.953+0000 7f2fc3fff640 1 -- 172.21.15.94:0/4145530601 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f2fd8004390 con 0x562b863d62e0 2024-02-17T12:00:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.953+0000 7f2fc3fff640 10 client.25712 handle_command_reply: tid=0 2024-02-17T12:00:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.953+0000 7f2fed670740 1 client.25712 shutdown 2024-02-17T12:00:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.954+0000 7f2fed670740 1 -- 172.21.15.94:0/4145530601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 msgr2=0x562b863d0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.954+0000 7f2fed670740 1 --2- 172.21.15.94:0/4145530601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 0x562b863d0180 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800ef90 tx=0x7f2fd8011810 comp rx=0 tx=0).stop 2024-02-17T12:00:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.955+0000 7f2fed670740 1 -- 172.21.15.94:0/4145530601 shutdown_connections 2024-02-17T12:00:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.955+0000 7f2fed670740 1 --2- 172.21.15.94:0/4145530601 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b863d62e0 0x562b8643fd50 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.955+0000 7f2fed670740 1 --2- 172.21.15.94:0/4145530601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b863cdad0 0x562b863d0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.955+0000 7f2fc3fff640 0 client.25712 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.955+0000 7f2fed670740 1 --2- 172.21.15.94:0/4145530601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b863ccd60 0x562b863d06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.955+0000 7f2fed670740 1 --2- 172.21.15.94:0/4145530601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b86394550 0x562b863d0180 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.955+0000 7f2fed670740 1 -- 172.21.15.94:0/4145530601 >> 172.21.15.94:0/4145530601 conn(0x562b863d88b0 msgr2=0x562b863c6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.955+0000 7f2fed670740 1 -- 172.21.15.94:0/4145530601 shutdown_connections 2024-02-17T12:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.955+0000 7f2fed670740 1 -- 172.21.15.94:0/4145530601 wait complete. 2024-02-17T12:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.955+0000 7f2fed670740 20 client.25712 trim_cache size 0 max 16384 2024-02-17T12:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:00:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.957+0000 7f2feaaa3640 1 -- 172.21.15.94:0/1168485096 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2fd40a24b0 msgr2=0x7f2fd40a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.957+0000 7f2feaaa3640 1 --2- 172.21.15.94:0/1168485096 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2fd40a24b0 0x7f2fd40a4970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40fefc0 tx=0x7f2fd000b590 comp rx=0 tx=0).stop 2024-02-17T12:00:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.957+0000 7f2feaaa3640 1 -- 172.21.15.94:0/1168485096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fe419bf50 msgr2=0x7f2fe419c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:00:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.957+0000 7f2feaaa3640 1 --2- 172.21.15.94:0/1168485096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fe419bf50 0x7f2fe419c400 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc02c700 tx=0x7f2fdc034b10 comp rx=0 tx=0).stop 2024-02-17T12:00:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.957+0000 7f2feaaa3640 1 -- 172.21.15.94:0/1168485096 shutdown_connections 2024-02-17T12:00:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.957+0000 7f2feaaa3640 1 --2- 172.21.15.94:0/1168485096 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2fd40a24b0 0x7f2fd40a4970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.957+0000 7f2feaaa3640 1 --2- 172.21.15.94:0/1168485096 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2fe419bf50 0x7f2fe419c400 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.957+0000 7f2feaaa3640 1 --2- 172.21.15.94:0/1168485096 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2fe4106500 0x7f2fe4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.957+0000 7f2feaaa3640 1 --2- 172.21.15.94:0/1168485096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fe40ff8f0 0x7f2fe41976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:00:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.957+0000 7f2feaaa3640 1 -- 172.21.15.94:0/1168485096 >> 172.21.15.94:0/1168485096 conn(0x7f2fe4076610 msgr2=0x7f2fe4104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:00:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.957+0000 7f2feaaa3640 1 -- 172.21.15.94:0/1168485096 shutdown_connections 2024-02-17T12:00:47.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:00:47.957+0000 7f2feaaa3640 1 -- 172.21.15.94:0/1168485096 wait complete. 2024-02-17T12:00:48.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:47 smithi189 ceph-mon[36588]: pgmap v83: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 625 op/s 2024-02-17T12:00:48.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:47 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:00:48.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:47 smithi125 ceph-mon[36819]: pgmap v83: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 625 op/s 2024-02-17T12:00:48.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:47 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:00:48.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:47 smithi094 ceph-mon[32499]: pgmap v83: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 7.8 MiB/s wr, 625 op/s 2024-02-17T12:00:48.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:47 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:00:50.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:49 smithi189 ceph-mon[36588]: pgmap v84: 97 pgs: 97 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-02-17T12:00:50.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:49 smithi125 ceph-mon[36819]: pgmap v84: 97 pgs: 97 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-02-17T12:00:50.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:49 smithi094 ceph-mon[32499]: pgmap v84: 97 pgs: 97 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 13 MiB/s wr, 1.03k op/s 2024-02-17T12:00:50.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 150.0 s, 3398.4 tps, lat 29.318 ms stddev 32.295 2024-02-17T12:00:52.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:51 smithi189 ceph-mon[36588]: pgmap v85: 97 pgs: 97 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.8 MiB/s wr, 718 op/s 2024-02-17T12:00:52.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:51 smithi125 ceph-mon[36819]: pgmap v85: 97 pgs: 97 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.8 MiB/s wr, 718 op/s 2024-02-17T12:00:52.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:51 smithi094 ceph-mon[32499]: pgmap v85: 97 pgs: 97 active+clean; 1.8 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 8.8 MiB/s wr, 718 op/s 2024-02-17T12:00:54.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:53 smithi189 ceph-mon[36588]: pgmap v86: 97 pgs: 97 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 909 op/s 2024-02-17T12:00:54.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:53 smithi125 ceph-mon[36819]: pgmap v86: 97 pgs: 97 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 909 op/s 2024-02-17T12:00:54.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:53 smithi094 ceph-mon[32499]: pgmap v86: 97 pgs: 97 active+clean; 1.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 11 MiB/s wr, 909 op/s 2024-02-17T12:00:55.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 155.0 s, 3357.4 tps, lat 29.903 ms stddev 32.663 2024-02-17T12:00:56.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:55 smithi189 ceph-mon[36588]: pgmap v87: 97 pgs: 97 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 11 MiB/s wr, 860 op/s 2024-02-17T12:00:56.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:55 smithi125 ceph-mon[36819]: pgmap v87: 97 pgs: 97 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 11 MiB/s wr, 860 op/s 2024-02-17T12:00:56.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:55 smithi094 ceph-mon[32499]: pgmap v87: 97 pgs: 97 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 11 MiB/s wr, 860 op/s 2024-02-17T12:00:58.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:57 smithi125 ceph-mon[36819]: pgmap v88: 97 pgs: 97 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 9.1 MiB/s wr, 743 op/s 2024-02-17T12:00:58.215 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:57 smithi094 ceph-mon[32499]: pgmap v88: 97 pgs: 97 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 9.1 MiB/s wr, 743 op/s 2024-02-17T12:00:58.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:57 smithi189 ceph-mon[36588]: pgmap v88: 97 pgs: 97 active+clean; 1.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 9.1 MiB/s wr, 743 op/s 2024-02-17T12:01:00.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:00:59 smithi125 ceph-mon[36819]: pgmap v89: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-02-17T12:01:00.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:00:59 smithi094 ceph-mon[32499]: pgmap v89: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-02-17T12:01:00.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 160.0 s, 3571.6 tps, lat 27.985 ms stddev 30.810 2024-02-17T12:01:00.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:00:59 smithi189 ceph-mon[36588]: pgmap v89: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 14 MiB/s wr, 1.09k op/s 2024-02-17T12:01:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.057+0000 7fa877c78640 1 -- 172.21.15.94:0/842656621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8700691a0 msgr2=0x7fa87010e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.057+0000 7fa877c78640 1 --2- 172.21.15.94:0/842656621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8700691a0 0x7fa87010e010 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa868009ca0 tx=0x7fa86802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.057+0000 7fa877c78640 1 -- 172.21.15.94:0/842656621 shutdown_connections 2024-02-17T12:01:01.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.057+0000 7fa877c78640 1 --2- 172.21.15.94:0/842656621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8700691a0 0x7fa87010e010 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.057+0000 7fa877c78640 1 --2- 172.21.15.94:0/842656621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa870068800 0x7fa870068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.057+0000 7fa877c78640 1 --2- 172.21.15.94:0/842656621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa870107290 0x7fa870107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.057+0000 7fa877c78640 1 -- 172.21.15.94:0/842656621 >> 172.21.15.94:0/842656621 conn(0x7fa870076ed0 msgr2=0x7fa8700772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.057+0000 7fa877c78640 1 -- 172.21.15.94:0/842656621 shutdown_connections 2024-02-17T12:01:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.057+0000 7fa877c78640 1 -- 172.21.15.94:0/842656621 wait complete. 2024-02-17T12:01:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa877c78640 1 Processor -- start 2024-02-17T12:01:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa877c78640 1 -- start start 2024-02-17T12:01:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa877c78640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa870068800 0x7fa870197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa877c78640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8700691a0 0x7fa870197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa877c78640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa870107290 0x7fa87019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa877c78640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87019c4e0 con 0x7fa8700691a0 2024-02-17T12:01:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa877c78640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87019c650 con 0x7fa870107290 2024-02-17T12:01:01.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa877c78640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87019c810 con 0x7fa870068800 2024-02-17T12:01:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa875213640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8700691a0 0x7fa870197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa875213640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8700691a0 0x7fa870197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44386/0 (socket says 172.21.15.94:44386) 2024-02-17T12:01:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa875a14640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa870068800 0x7fa870197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa875213640 1 -- 172.21.15.94:0/3983319497 learned_addr learned my addr 172.21.15.94:0/3983319497 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa876215640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa870107290 0x7fa87019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.058+0000 7fa875213640 1 -- 172.21.15.94:0/3983319497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa870068800 msgr2=0x7fa870197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.059+0000 7fa875213640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa870068800 0x7fa870197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.059+0000 7fa875213640 1 -- 172.21.15.94:0/3983319497 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa870107290 msgr2=0x7fa87019bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.059+0000 7fa875213640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa870107290 0x7fa87019bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.059+0000 7fa875213640 1 -- 172.21.15.94:0/3983319497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa868009900 con 0x7fa8700691a0 2024-02-17T12:01:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.059+0000 7fa875a14640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa870068800 0x7fa870197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.059+0000 7fa876215640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa870107290 0x7fa87019bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:01.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.059+0000 7fa875213640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8700691a0 0x7fa870197bb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa86000e9f0 tx=0x7fa86000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.059+0000 7fa866ffd640 1 -- 172.21.15.94:0/3983319497 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860009720 con 0x7fa8700691a0 2024-02-17T12:01:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.059+0000 7fa866ffd640 1 -- 172.21.15.94:0/3983319497 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa86000cd80 con 0x7fa8700691a0 2024-02-17T12:01:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.059+0000 7fa866ffd640 1 -- 172.21.15.94:0/3983319497 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860010690 con 0x7fa8700691a0 2024-02-17T12:01:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.059+0000 7fa877c78640 1 -- 172.21.15.94:0/3983319497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87019caf0 con 0x7fa8700691a0 2024-02-17T12:01:01.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.059+0000 7fa877c78640 1 -- 172.21.15.94:0/3983319497 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8701a4370 con 0x7fa8700691a0 2024-02-17T12:01:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.062+0000 7fa866ffd640 1 -- 172.21.15.94:0/3983319497 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa8600046c0 con 0x7fa8700691a0 2024-02-17T12:01:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.062+0000 7fa87a845740 1 Processor -- start 2024-02-17T12:01:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.062+0000 7fa87a845740 1 -- start start 2024-02-17T12:01:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa87a845740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe679ad0 0x55aafe678820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa87a845740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe678d60 0x55aafe6791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa87a845740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe6822e0 0x55aafe6846d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa87a845740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafe5e79f0 con 0x55aafe678d60 2024-02-17T12:01:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa87a845740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafe5e4f20 con 0x55aafe679ad0 2024-02-17T12:01:01.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa87a845740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafe5e0700 con 0x55aafe6822e0 2024-02-17T12:01:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa876215640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe679ad0 0x55aafe678820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa875a14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe678d60 0x55aafe6791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa876215640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe679ad0 0x55aafe678820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:37300/0 (socket says 172.21.15.94:37300) 2024-02-17T12:01:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa876215640 1 -- 172.21.15.94:0/2675379669 learned_addr learned my addr 172.21.15.94:0/2675379669 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa875213640 1 --2- 172.21.15.94:0/2675379669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe6822e0 0x55aafe6846d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa866ffd640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa85c0a2350 0x7fa85c0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa876215640 1 -- 172.21.15.94:0/2675379669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe6822e0 msgr2=0x55aafe6846d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa876215640 1 --2- 172.21.15.94:0/2675379669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe6822e0 0x55aafe6846d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa876215640 1 -- 172.21.15.94:0/2675379669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe678d60 msgr2=0x55aafe6791e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa876215640 1 --2- 172.21.15.94:0/2675379669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe678d60 0x55aafe6791e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa876215640 1 -- 172.21.15.94:0/2675379669 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aafe5767b0 con 0x55aafe679ad0 2024-02-17T12:01:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa875a14640 1 --2- 172.21.15.94:0/2675379669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe678d60 0x55aafe6791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.063+0000 7fa866ffd640 1 -- 172.21.15.94:0/3983319497 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa860014070 con 0x7fa8700691a0 2024-02-17T12:01:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.064+0000 7fa876215640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa85c0a2350 0x7fa85c0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.064+0000 7fa876215640 1 --2- 172.21.15.94:0/2675379669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe679ad0 0x55aafe678820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa868009ca0 tx=0x7fa868005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2092778c5d85612 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.064+0000 7fa85bfff640 1 -- 172.21.15.94:0/2675379669 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868034c90 con 0x55aafe679ad0 2024-02-17T12:01:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.064+0000 7fa85bfff640 1 -- 172.21.15.94:0/2675379669 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa868003e50 con 0x55aafe679ad0 2024-02-17T12:01:01.065 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.064+0000 7fa85bfff640 1 -- 172.21.15.94:0/2675379669 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86804c460 con 0x55aafe679ad0 2024-02-17T12:01:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.065+0000 7fa87a845740 1 -- 172.21.15.94:0/2675379669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe679ad0 msgr2=0x55aafe678820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.065+0000 7fa876215640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa85c0a2350 0x7fa85c0a4810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa86804b000 tx=0x7fa86804e000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:01.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.065+0000 7fa87a845740 1 --2- 172.21.15.94:0/2675379669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe679ad0 0x55aafe678820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa868009ca0 tx=0x7fa868005ef0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.065+0000 7fa87a845740 1 -- 172.21.15.94:0/2675379669 shutdown_connections 2024-02-17T12:01:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.065+0000 7fa87a845740 1 --2- 172.21.15.94:0/2675379669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe6822e0 0x55aafe6846d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.065+0000 7fa87a845740 1 --2- 172.21.15.94:0/2675379669 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe678d60 0x55aafe6791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.065+0000 7fa87a845740 1 --2- 172.21.15.94:0/2675379669 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe679ad0 0x55aafe678820 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.065+0000 7fa87a845740 1 -- 172.21.15.94:0/2675379669 >> 172.21.15.94:0/2675379669 conn(0x55aafe640550 msgr2=0x55aafe673a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.065+0000 7fa87a845740 1 -- 172.21.15.94:0/2675379669 shutdown_connections 2024-02-17T12:01:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.065+0000 7fa87a845740 1 -- 172.21.15.94:0/2675379669 wait complete. 2024-02-17T12:01:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.065+0000 7fa87a845740 1 -- start start 2024-02-17T12:01:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.065+0000 7fa87a845740 1 Processor -- start 2024-02-17T12:01:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa87a845740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe678d60 0x55aafe67f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa87a845740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 0x55aafe67f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa87a845740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe6822e0 0x55aafe67fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.071 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa87a845740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafe585bf0 con 0x55aafe679ad0 2024-02-17T12:01:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa87a845740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafe2b4250 con 0x55aafe6822e0 2024-02-17T12:01:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa87a845740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafe566200 con 0x55aafe678d60 2024-02-17T12:01:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa87a845740 10 client.0 _ll_register_callbacks cb 0x55aafe5d4d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa875213640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 0x55aafe67f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa876215640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe6822e0 0x55aafe67fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa875213640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 0x55aafe67f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44416/0 (socket says 172.21.15.94:44416) 2024-02-17T12:01:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa875213640 1 -- 172.21.15.94:0/1120922863 learned_addr learned my addr 172.21.15.94:0/1120922863 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa875a14640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe678d60 0x55aafe67f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.072 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa876215640 1 -- 172.21.15.94:0/1120922863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe678d60 msgr2=0x55aafe67f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa876215640 1 --2- 172.21.15.94:0/1120922863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe678d60 0x55aafe67f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa876215640 1 -- 172.21.15.94:0/1120922863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 msgr2=0x55aafe67f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa876215640 1 --2- 172.21.15.94:0/1120922863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 0x55aafe67f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa876215640 1 -- 172.21.15.94:0/1120922863 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87010e910 con 0x55aafe6822e0 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa875a14640 1 --2- 172.21.15.94:0/1120922863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe678d60 0x55aafe67f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.066+0000 7fa875213640 1 --2- 172.21.15.94:0/1120922863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 0x55aafe67f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa876215640 1 --2- 172.21.15.94:0/1120922863 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe6822e0 0x55aafe67fd80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa868004af0 tx=0x7fa868004950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0bfd129df7ecdb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa8597fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa8597fa640 1 -- 172.21.15.94:0/1120922863 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86804c460 con 0x55aafe6822e0 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa8597fa640 1 -- 172.21.15.94:0/1120922863 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa868032b00 con 0x55aafe6822e0 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa87a845740 1 -- 172.21.15.94:0/1120922863 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aafe2e0600 con 0x55aafe6822e0 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa8597fa640 1 -- 172.21.15.94:0/1120922863 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa868034ca0 con 0x55aafe6822e0 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa87a845740 10 client.25724 fetch_fsmap learned FSMap version 21 2024-02-17T12:01:01.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa87a845740 1 -- 172.21.15.94:0/1120922863 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55aafe5ded80 con 0x55aafe6822e0 2024-02-17T12:01:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa8597fa640 1 -- 172.21.15.94:0/1120922863 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aafe2e0600 con 0x55aafe6822e0 2024-02-17T12:01:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa8597fa640 1 -- 172.21.15.94:0/1120922863 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa86803f420 con 0x55aafe6822e0 2024-02-17T12:01:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa87a845740 10 client.25724 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:01:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa87a845740 10 client.25724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa87a845740 20 client.25724 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa87a845740 1 --2- 172.21.15.94:0/1120922863 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aafe6921c0 0x55aafe6945b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa87a845740 4 client.25724 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.067+0000 7fa87a845740 1 -- 172.21.15.94:0/1120922863 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aafe3311b0 con 0x55aafe6921c0 2024-02-17T12:01:01.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.068+0000 7fa875a14640 1 --2- 172.21.15.94:0/1120922863 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aafe6921c0 0x55aafe6945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.068+0000 7fa8597fa640 10 client.25724 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.068+0000 7fa875a14640 1 --2- 172.21.15.94:0/1120922863 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aafe6921c0 0x55aafe6945b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b025e53dcf45bfcc server_cookie=f1d2c55041e904f5 in_seq=0 out_seq=0 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa8597fa640 1 -- 172.21.15.94:0/1120922863 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aafe3311b0 con 0x55aafe6921c0 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa8597fa640 10 client.25724 handle_command_reply: tid=0 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa87a845740 1 client.25724 shutdown 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa87a845740 1 -- 172.21.15.94:0/1120922863 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe6822e0 msgr2=0x55aafe67fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa87a845740 1 --2- 172.21.15.94:0/1120922863 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe6822e0 0x55aafe67fd80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa868004af0 tx=0x7fa868004950 comp rx=0 tx=0).stop 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa87a845740 1 -- 172.21.15.94:0/1120922863 shutdown_connections 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa87a845740 1 --2- 172.21.15.94:0/1120922863 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aafe6921c0 0x55aafe6945b0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa87a845740 1 --2- 172.21.15.94:0/1120922863 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe6822e0 0x55aafe67fd80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa87a845740 1 --2- 172.21.15.94:0/1120922863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 0x55aafe67f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa8597fa640 0 client.25724 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa87a845740 1 --2- 172.21.15.94:0/1120922863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe678d60 0x55aafe67f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa87a845740 1 -- 172.21.15.94:0/1120922863 >> 172.21.15.94:0/1120922863 conn(0x55aafe640550 msgr2=0x55aafe6757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa87a845740 1 -- 172.21.15.94:0/1120922863 shutdown_connections 2024-02-17T12:01:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa87a845740 1 -- 172.21.15.94:0/1120922863 wait complete. 2024-02-17T12:01:01.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.069+0000 7fa87a845740 20 client.25724 trim_cache size 0 max 16384 2024-02-17T12:01:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.083+0000 7fa87a845740 1 Processor -- start 2024-02-17T12:01:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa87a845740 1 -- start start 2024-02-17T12:01:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa87a845740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe678d60 0x55aafe681a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa87a845740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 0x55aafe67c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa87a845740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe6822e0 0x55aafe67cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa87a845740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafe678b60 con 0x55aafe679ad0 2024-02-17T12:01:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa87a845740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafe699550 con 0x55aafe678d60 2024-02-17T12:01:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa87a845740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafe3311b0 con 0x55aafe6822e0 2024-02-17T12:01:01.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa875213640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 0x55aafe67c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa875213640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 0x55aafe67c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44418/0 (socket says 172.21.15.94:44418) 2024-02-17T12:01:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa875213640 1 -- 172.21.15.94:0/1688089924 learned_addr learned my addr 172.21.15.94:0/1688089924 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa875a14640 1 --2- 172.21.15.94:0/1688089924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe678d60 0x55aafe681a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa876215640 1 --2- 172.21.15.94:0/1688089924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe6822e0 0x55aafe67cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa875213640 1 -- 172.21.15.94:0/1688089924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe6822e0 msgr2=0x55aafe67cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa875213640 1 --2- 172.21.15.94:0/1688089924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe6822e0 0x55aafe67cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa875213640 1 -- 172.21.15.94:0/1688089924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe678d60 msgr2=0x55aafe681a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa875213640 1 --2- 172.21.15.94:0/1688089924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe678d60 0x55aafe681a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa875213640 1 -- 172.21.15.94:0/1688089924 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aafe57b7b0 con 0x55aafe679ad0 2024-02-17T12:01:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa875213640 1 --2- 172.21.15.94:0/1688089924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 0x55aafe67c5c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa8600a1f50 tx=0x7fa860004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4316ab93fe5ba032 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.084+0000 7fa85b7fe640 1 -- 172.21.15.94:0/1688089924 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86009a8d0 con 0x55aafe679ad0 2024-02-17T12:01:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.085+0000 7fa85b7fe640 1 -- 172.21.15.94:0/1688089924 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa86009a6f0 con 0x55aafe679ad0 2024-02-17T12:01:01.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.085+0000 7fa85b7fe640 1 -- 172.21.15.94:0/1688089924 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8600a30a0 con 0x55aafe679ad0 2024-02-17T12:01:01.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.085+0000 7fa87a845740 1 -- 172.21.15.94:0/1688089924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 msgr2=0x55aafe67c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:01.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.085+0000 7fa87a845740 1 --2- 172.21.15.94:0/1688089924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 0x55aafe67c5c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa8600a1f50 tx=0x7fa860004a30 comp rx=0 tx=0).stop 2024-02-17T12:01:01.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.085+0000 7fa87a845740 1 -- 172.21.15.94:0/1688089924 shutdown_connections 2024-02-17T12:01:01.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.085+0000 7fa87a845740 1 --2- 172.21.15.94:0/1688089924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe6822e0 0x55aafe67cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.085+0000 7fa87a845740 1 --2- 172.21.15.94:0/1688089924 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe679ad0 0x55aafe67c5c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.085+0000 7fa87a845740 1 --2- 172.21.15.94:0/1688089924 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe678d60 0x55aafe681a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.086+0000 7fa87a845740 1 -- 172.21.15.94:0/1688089924 >> 172.21.15.94:0/1688089924 conn(0x55aafe640550 msgr2=0x55aafe67dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.086+0000 7fa87a845740 1 -- 172.21.15.94:0/1688089924 shutdown_connections 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.086+0000 7fa87a845740 1 -- 172.21.15.94:0/1688089924 wait complete. 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.086+0000 7fa87a845740 1 -- start start 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.086+0000 7fa87a845740 1 Processor -- start 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.086+0000 7fa87a845740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe678d60 0x55aafe687cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.086+0000 7fa87a845740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe679ad0 0x55aafe688200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.086+0000 7fa87a845740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe6822e0 0x55aafe688740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.086+0000 7fa87a845740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafe6e86f0 con 0x55aafe678d60 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.086+0000 7fa87a845740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafe6e8860 con 0x55aafe6822e0 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.086+0000 7fa87a845740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aafe6e8a20 con 0x55aafe679ad0 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa87a845740 10 client.0 _ll_register_callbacks cb 0x7fa868037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa875a14640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe678d60 0x55aafe687cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa875213640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe679ad0 0x55aafe688200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa875213640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe679ad0 0x55aafe688200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:34308/0 (socket says 172.21.15.94:34308) 2024-02-17T12:01:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa875213640 1 -- 172.21.15.94:0/777723948 learned_addr learned my addr 172.21.15.94:0/777723948 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa876215640 1 --2- 172.21.15.94:0/777723948 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe6822e0 0x55aafe688740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa875213640 1 -- 172.21.15.94:0/777723948 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe6822e0 msgr2=0x55aafe688740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa875213640 1 --2- 172.21.15.94:0/777723948 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe6822e0 0x55aafe688740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa875213640 1 -- 172.21.15.94:0/777723948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe678d60 msgr2=0x55aafe687cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa875213640 1 --2- 172.21.15.94:0/777723948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe678d60 0x55aafe687cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa875213640 1 -- 172.21.15.94:0/777723948 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8701a4370 con 0x55aafe679ad0 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa876215640 1 --2- 172.21.15.94:0/777723948 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe6822e0 0x55aafe688740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa875213640 1 --2- 172.21.15.94:0/777723948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe679ad0 0x55aafe688200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa86000e9c0 tx=0x7fa8600992a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b408566ecea68ef2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa8597fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa8597fa640 1 -- 172.21.15.94:0/777723948 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86009a8d0 con 0x55aafe679ad0 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa8597fa640 1 -- 172.21.15.94:0/777723948 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fa8600d3e40 con 0x55aafe679ad0 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.087+0000 7fa87a845740 1 -- 172.21.15.94:0/777723948 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa868003e50 con 0x55aafe679ad0 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.088+0000 7fa8597fa640 1 -- 172.21.15.94:0/777723948 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8600a3530 con 0x55aafe679ad0 2024-02-17T12:01:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.088+0000 7fa87a845740 10 client.25774 fetch_fsmap learned FSMap version 21 2024-02-17T12:01:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.088+0000 7fa87a845740 1 -- 172.21.15.94:0/777723948 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55aafe5a1540 con 0x55aafe679ad0 2024-02-17T12:01:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.088+0000 7fa8597fa640 1 -- 172.21.15.94:0/777723948 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa868003e50 con 0x55aafe679ad0 2024-02-17T12:01:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.088+0000 7fa8597fa640 1 -- 172.21.15.94:0/777723948 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa8600ce040 con 0x55aafe679ad0 2024-02-17T12:01:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.088+0000 7fa87a845740 10 client.25774 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:01:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.088+0000 7fa87a845740 10 client.25774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.088+0000 7fa87a845740 20 client.25774 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.088+0000 7fa87a845740 1 --2- 172.21.15.94:0/777723948 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aafe6e9d90 0x55aafe6ec180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.088+0000 7fa87a845740 4 client.25774 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:01:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.088+0000 7fa87a845740 1 -- 172.21.15.94:0/777723948 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aafe31d550 con 0x55aafe6e9d90 2024-02-17T12:01:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.088+0000 7fa875a14640 1 --2- 172.21.15.94:0/777723948 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aafe6e9d90 0x55aafe6ec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.089+0000 7fa8597fa640 10 client.25774 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:01.089+0000 7fa875a14640 1 --2- 172.21.15.94:0/777723948 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aafe6e9d90 0x55aafe6ec180 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50503d47c68ee7cb server_cookie=9f7223c67f4f59f9 in_seq=0 out_seq=0 2024-02-17T12:01:02.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:01 smithi125 ceph-mon[36819]: pgmap v90: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 680 op/s 2024-02-17T12:01:02.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:01 smithi094 ceph-mon[32499]: pgmap v90: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 680 op/s 2024-02-17T12:01:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.375+0000 7fa8597fa640 1 -- 172.21.15.94:0/777723948 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aafe31d550 con 0x55aafe6e9d90 2024-02-17T12:01:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.375+0000 7fa8597fa640 10 client.25774 handle_command_reply: tid=0 2024-02-17T12:01:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.375+0000 7fa87a845740 1 client.25774 shutdown 2024-02-17T12:01:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa87a845740 1 -- 172.21.15.94:0/777723948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe679ad0 msgr2=0x55aafe688200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa87a845740 1 --2- 172.21.15.94:0/777723948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe679ad0 0x55aafe688200 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa86000e9c0 tx=0x7fa8600992a0 comp rx=0 tx=0).stop 2024-02-17T12:01:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa87a845740 1 -- 172.21.15.94:0/777723948 shutdown_connections 2024-02-17T12:01:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa87a845740 1 --2- 172.21.15.94:0/777723948 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aafe6e9d90 0x55aafe6ec180 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa87a845740 1 --2- 172.21.15.94:0/777723948 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aafe6822e0 0x55aafe688740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa87a845740 1 --2- 172.21.15.94:0/777723948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aafe679ad0 0x55aafe688200 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa8597fa640 0 client.25774 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa87a845740 1 --2- 172.21.15.94:0/777723948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aafe678d60 0x55aafe687cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa87a845740 1 -- 172.21.15.94:0/777723948 >> 172.21.15.94:0/777723948 conn(0x55aafe640550 msgr2=0x55aafe672b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa87a845740 1 -- 172.21.15.94:0/777723948 shutdown_connections 2024-02-17T12:01:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa87a845740 1 -- 172.21.15.94:0/777723948 wait complete. 2024-02-17T12:01:02.376 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa87a845740 20 client.25774 trim_cache size 0 max 16384 2024-02-17T12:01:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:01:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:01:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:01:02.377 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:01:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa877c78640 1 -- 172.21.15.94:0/3983319497 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa85c0a2350 msgr2=0x7fa85c0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa877c78640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa85c0a2350 0x7fa85c0a4810 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa86804b000 tx=0x7fa86804e000 comp rx=0 tx=0).stop 2024-02-17T12:01:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa877c78640 1 -- 172.21.15.94:0/3983319497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8700691a0 msgr2=0x7fa870197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.376+0000 7fa877c78640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8700691a0 0x7fa870197bb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa86000e9f0 tx=0x7fa86000edf0 comp rx=0 tx=0).stop 2024-02-17T12:01:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.377+0000 7fa877c78640 1 -- 172.21.15.94:0/3983319497 shutdown_connections 2024-02-17T12:01:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.377+0000 7fa877c78640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa85c0a2350 0x7fa85c0a4810 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.377+0000 7fa877c78640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa870107290 0x7fa87019bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.377+0000 7fa877c78640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8700691a0 0x7fa870197bb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.377+0000 7fa877c78640 1 --2- 172.21.15.94:0/3983319497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa870068800 0x7fa870197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.377+0000 7fa877c78640 1 -- 172.21.15.94:0/3983319497 >> 172.21.15.94:0/3983319497 conn(0x7fa870076ed0 msgr2=0x7fa8700fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.377+0000 7fa877c78640 1 -- 172.21.15.94:0/3983319497 shutdown_connections 2024-02-17T12:01:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:02.377+0000 7fa877c78640 1 -- 172.21.15.94:0/3983319497 wait complete. 2024-02-17T12:01:02.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:01 smithi189 ceph-mon[36588]: pgmap v90: 97 pgs: 97 active+clean; 1.8 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 9.1 MiB/s wr, 680 op/s 2024-02-17T12:01:03.401 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: *** Caught signal (Segmentation fault) ** 2024-02-17T12:01:03.401 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: in thread 7f0b2aef5700 thread_name:md_log_replay 2024-02-17T12:01:03.401 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:01:03.401 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0b3c2b6d20] 2024-02-17T12:01:03.401 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 2: /usr/bin/ceph-mds(+0x50481f) [0x5596fc51f81f] 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5596fc521a53] 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 4: (MDLog::_replay_thread()+0x647) [0x5596fc57e277] 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5596fc588111] 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0b3c2ac1ca] 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 7: clone() 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 2024-02-17T12:01:03.116+0000 7f0b2aef5700 -1 *** Caught signal (Segmentation fault) ** 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: in thread 7f0b2aef5700 thread_name:md_log_replay 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0b3c2b6d20] 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 2: /usr/bin/ceph-mds(+0x50481f) [0x5596fc51f81f] 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5596fc521a53] 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 4: (MDLog::_replay_thread()+0x647) [0x5596fc57e277] 2024-02-17T12:01:03.402 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5596fc588111] 2024-02-17T12:01:03.403 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0b3c2ac1ca] 2024-02-17T12:01:03.403 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 7: clone() 2024-02-17T12:01:03.403 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-17T12:01:03.403 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 2024-02-17T12:01:03.403 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 0> 2024-02-17T12:01:03.116+0000 7f0b2aef5700 -1 *** Caught signal (Segmentation fault) ** 2024-02-17T12:01:03.403 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: in thread 7f0b2aef5700 thread_name:md_log_replay 2024-02-17T12:01:03.403 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 2024-02-17T12:01:03.403 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:01:03.403 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0b3c2b6d20] 2024-02-17T12:01:03.403 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 2: /usr/bin/ceph-mds(+0x50481f) [0x5596fc51f81f] 2024-02-17T12:01:03.403 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5596fc521a53] 2024-02-17T12:01:03.404 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 4: (MDLog::_replay_thread()+0x647) [0x5596fc57e277] 2024-02-17T12:01:03.404 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5596fc588111] 2024-02-17T12:01:03.404 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0b3c2ac1ca] 2024-02-17T12:01:03.404 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 7: clone() 2024-02-17T12:01:03.404 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-17T12:01:03.404 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 2024-02-17T12:01:03.404 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 0> 2024-02-17T12:01:03.116+0000 7f0b2aef5700 -1 *** Caught signal (Segmentation fault) ** 2024-02-17T12:01:03.404 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: in thread 7f0b2aef5700 thread_name:md_log_replay 2024-02-17T12:01:03.404 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 2024-02-17T12:01:03.404 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:01:03.404 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0b3c2b6d20] 2024-02-17T12:01:03.404 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 2: /usr/bin/ceph-mds(+0x50481f) [0x5596fc51f81f] 2024-02-17T12:01:03.404 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x5596fc521a53] 2024-02-17T12:01:03.405 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 4: (MDLog::_replay_thread()+0x647) [0x5596fc57e277] 2024-02-17T12:01:03.405 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 5: (MDLog::ReplayThread::entry()+0x11) [0x5596fc588111] 2024-02-17T12:01:03.405 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f0b3c2ac1ca] 2024-02-17T12:01:03.405 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 7: clone() 2024-02-17T12:01:03.405 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-17T12:01:03.405 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[67380]: 2024-02-17T12:01:03.405 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 podman[71317]: 2024-02-17 12:01:03.180804887 +0000 UTC m=+0.020549565 container died 7a8d6cf40c0e5fee39f5632879a333cb731d262723f770445b27adf455d8e095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-17T12:01:04.136 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:03 smithi189 ceph-mon[36588]: pgmap v91: 97 pgs: 97 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 905 op/s 2024-02-17T12:01:04.136 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 podman[71317]: 2024-02-17 12:01:03.876267078 +0000 UTC m=+0.716011757 container remove 7a8d6cf40c0e5fee39f5632879a333cb731d262723f770445b27adf455d8e095 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:01:04.136 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:03 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.f.service: Main process exited, code=exited, status=139/n/a 2024-02-17T12:01:04.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:03 smithi125 ceph-mon[36819]: pgmap v91: 97 pgs: 97 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 905 op/s 2024-02-17T12:01:04.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:03 smithi094 ceph-mon[32499]: pgmap v91: 97 pgs: 97 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 11 MiB/s wr, 905 op/s 2024-02-17T12:01:04.400 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:04 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.f.service: Failed with result 'exit-code'. 2024-02-17T12:01:04.401 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:04 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.f.service: Consumed 1.685s CPU time. 2024-02-17T12:01:05.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 165.0 s, 461.2 tps, lat 206.005 ms stddev 190.419 2024-02-17T12:01:05.400 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:04 smithi189 ceph-mon[36588]: pgmap v92: 97 pgs: 97 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.4 MiB/s wr, 800 op/s 2024-02-17T12:01:05.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:04 smithi125 ceph-mon[36819]: pgmap v92: 97 pgs: 97 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.4 MiB/s wr, 800 op/s 2024-02-17T12:01:05.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:04 smithi094 ceph-mon[32499]: pgmap v92: 97 pgs: 97 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 9.4 MiB/s wr, 800 op/s 2024-02-17T12:01:08.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:07 smithi189 ceph-mon[36588]: pgmap v93: 97 pgs: 97 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.5 MiB/s wr, 656 op/s 2024-02-17T12:01:08.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:07 smithi125 ceph-mon[36819]: pgmap v93: 97 pgs: 97 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.5 MiB/s wr, 656 op/s 2024-02-17T12:01:08.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:07 smithi094 ceph-mon[32499]: pgmap v93: 97 pgs: 97 active+clean; 1.9 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 7.5 MiB/s wr, 656 op/s 2024-02-17T12:01:10.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:09 smithi189 ceph-mon[36588]: pgmap v94: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 MiB/s wr, 1.70k op/s 2024-02-17T12:01:10.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:09 smithi125 ceph-mon[36819]: pgmap v94: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 MiB/s wr, 1.70k op/s 2024-02-17T12:01:10.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:09 smithi094 ceph-mon[32499]: pgmap v94: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 MiB/s wr, 1.70k op/s 2024-02-17T12:01:10.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 170.0 s, 2707.2 tps, lat 38.800 ms stddev 116.158 2024-02-17T12:01:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:11 smithi189 ceph-mon[36588]: pgmap v95: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 MiB/s wr, 1.35k op/s 2024-02-17T12:01:12.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:11 smithi125 ceph-mon[36819]: pgmap v95: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 MiB/s wr, 1.35k op/s 2024-02-17T12:01:12.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:11 smithi094 ceph-mon[32499]: pgmap v95: 97 pgs: 97 active+clean; 2.1 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 MiB/s wr, 1.35k op/s 2024-02-17T12:01:13.277 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:01:13.278 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:01:13.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.462+0000 7ff40770d640 1 -- 172.21.15.94:0/2546989720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40010cc40 msgr2=0x7ff40010d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:13.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.462+0000 7ff40770d640 1 --2- 172.21.15.94:0/2546989720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40010cc40 0x7ff40010d0d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0009ca0 tx=0x7ff3f002c120 comp rx=0 tx=0).stop 2024-02-17T12:01:13.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.463+0000 7ff40770d640 1 -- 172.21.15.94:0/2546989720 shutdown_connections 2024-02-17T12:01:13.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.463+0000 7ff40770d640 1 --2- 172.21.15.94:0/2546989720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40010cc40 0x7ff40010d0d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.463+0000 7ff40770d640 1 --2- 172.21.15.94:0/2546989720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4001085f0 0x7ff4001091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.463+0000 7ff40770d640 1 --2- 172.21.15.94:0/2546989720 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff400108180 0x7ff400108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.463+0000 7ff40770d640 1 -- 172.21.15.94:0/2546989720 >> 172.21.15.94:0/2546989720 conn(0x7ff40006c270 msgr2=0x7ff40006c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.463+0000 7ff40770d640 1 -- 172.21.15.94:0/2546989720 shutdown_connections 2024-02-17T12:01:13.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.463+0000 7ff40770d640 1 -- 172.21.15.94:0/2546989720 wait complete. 2024-02-17T12:01:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.465+0000 7ff40770d640 1 Processor -- start 2024-02-17T12:01:13.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.465+0000 7ff40770d640 1 -- start start 2024-02-17T12:01:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.465+0000 7ff40770d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff400108180 0x7ff4001a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.465+0000 7ff40770d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4001085f0 0x7ff4001a2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.465+0000 7ff40770d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40010cc40 0x7ff4001a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.465+0000 7ff40770d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4001a7090 con 0x7ff40010cc40 2024-02-17T12:01:13.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.465+0000 7ff40770d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4001a7200 con 0x7ff4001085f0 2024-02-17T12:01:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.465+0000 7ff40770d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4001a73c0 con 0x7ff400108180 2024-02-17T12:01:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.466+0000 7ff405caa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40010cc40 0x7ff4001a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.466+0000 7ff405caa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40010cc40 0x7ff4001a6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40856/0 (socket says 172.21.15.94:40856) 2024-02-17T12:01:13.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.466+0000 7ff405caa640 1 -- 172.21.15.94:0/3594796650 learned_addr learned my addr 172.21.15.94:0/3594796650 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.467+0000 7ff404ca8640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4001085f0 0x7ff4001a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.468+0000 7ff4054a9640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff400108180 0x7ff4001a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.468+0000 7ff404ca8640 1 -- 172.21.15.94:0/3594796650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff400108180 msgr2=0x7ff4001a2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.468+0000 7ff404ca8640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff400108180 0x7ff4001a2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.468+0000 7ff404ca8640 1 -- 172.21.15.94:0/3594796650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40010cc40 msgr2=0x7ff4001a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.468+0000 7ff404ca8640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40010cc40 0x7ff4001a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.468+0000 7ff404ca8640 1 -- 172.21.15.94:0/3594796650 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f0009900 con 0x7ff4001085f0 2024-02-17T12:01:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.468+0000 7ff4054a9640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff400108180 0x7ff4001a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:01:13.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.468+0000 7ff404ca8640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4001085f0 0x7ff4001a2760 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400d6a0 tx=0x7ff3f400db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.469+0000 7ff3fe7fc640 1 -- 172.21.15.94:0/3594796650 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4005220 con 0x7ff4001085f0 2024-02-17T12:01:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.469+0000 7ff3fe7fc640 1 -- 172.21.15.94:0/3594796650 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3f4009d00 con 0x7ff4001085f0 2024-02-17T12:01:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.469+0000 7ff40770d640 1 -- 172.21.15.94:0/3594796650 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4001a76a0 con 0x7ff4001085f0 2024-02-17T12:01:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.470+0000 7ff40770d640 1 -- 172.21.15.94:0/3594796650 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4001aef20 con 0x7ff4001085f0 2024-02-17T12:01:13.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.470+0000 7ff3fe7fc640 1 -- 172.21.15.94:0/3594796650 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4004d90 con 0x7ff4001085f0 2024-02-17T12:01:13.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.473+0000 7ff3fe7fc640 1 -- 172.21.15.94:0/3594796650 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff3f4010460 con 0x7ff4001085f0 2024-02-17T12:01:13.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.474+0000 7ff40a3bb740 1 Processor -- start 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.474+0000 7ff40a3bb740 1 -- start start 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.474+0000 7ff40a3bb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2e7b0 0x55b08ae2eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.475+0000 7ff40a3bb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae2f0d0 0x55b08ae282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.475+0000 7ff40a3bb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae40a30 0x55b08ae42e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.475+0000 7ff40a3bb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08ad95280 con 0x55b08ae2e7b0 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.475+0000 7ff40a3bb740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08ad927b0 con 0x55b08ae2f0d0 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.475+0000 7ff40a3bb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08ad8df90 con 0x55b08ae40a30 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.475+0000 7ff404ca8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae40a30 0x55b08ae42e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.475+0000 7ff404ca8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae40a30 0x55b08ae42e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39384/0 (socket says 172.21.15.94:39384) 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.475+0000 7ff404ca8640 1 -- 172.21.15.94:0/4118502275 learned_addr learned my addr 172.21.15.94:0/4118502275 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.475+0000 7ff405caa640 1 --2- 172.21.15.94:0/4118502275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2e7b0 0x55b08ae2eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.475+0000 7ff3fe7fc640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff3e80a2440 0x7ff3e80a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.476+0000 7ff404ca8640 1 -- 172.21.15.94:0/4118502275 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae2f0d0 msgr2=0x55b08ae282a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:01:13.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.476+0000 7ff404ca8640 1 --2- 172.21.15.94:0/4118502275 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae2f0d0 0x55b08ae282a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.476+0000 7ff404ca8640 1 -- 172.21.15.94:0/4118502275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2e7b0 msgr2=0x55b08ae2eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:13.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.476+0000 7ff404ca8640 1 --2- 172.21.15.94:0/4118502275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2e7b0 0x55b08ae2eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.476+0000 7ff404ca8640 1 -- 172.21.15.94:0/4118502275 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b08ad4c570 con 0x55b08ae40a30 2024-02-17T12:01:13.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.476+0000 7ff404ca8640 1 --2- 172.21.15.94:0/4118502275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae40a30 0x55b08ae42e20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40cbba0 tx=0x7ff3f4012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b71cda8ab234ba8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:13.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.476+0000 7ff3fe7fc640 1 -- 172.21.15.94:0/3594796650 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff3f40ce580 con 0x7ff4001085f0 2024-02-17T12:01:13.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.477+0000 7ff3e77fe640 1 -- 172.21.15.94:0/4118502275 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4009b70 con 0x55b08ae40a30 2024-02-17T12:01:13.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.477+0000 7ff3e77fe640 1 -- 172.21.15.94:0/4118502275 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3f40971a0 con 0x55b08ae40a30 2024-02-17T12:01:13.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.477+0000 7ff3e77fe640 1 -- 172.21.15.94:0/4118502275 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f409f0c0 con 0x55b08ae40a30 2024-02-17T12:01:13.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.477+0000 7ff405caa640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff3e80a2440 0x7ff3e80a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.478+0000 7ff40a3bb740 1 -- 172.21.15.94:0/4118502275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae40a30 msgr2=0x55b08ae42e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:13.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.478+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/4118502275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae40a30 0x55b08ae42e20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff3f40cbba0 tx=0x7ff3f4012000 comp rx=0 tx=0).stop 2024-02-17T12:01:13.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.478+0000 7ff40a3bb740 1 -- 172.21.15.94:0/4118502275 shutdown_connections 2024-02-17T12:01:13.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.478+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/4118502275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae40a30 0x55b08ae42e20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.478+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/4118502275 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae2f0d0 0x55b08ae282a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.478+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/4118502275 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2e7b0 0x55b08ae2eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.478+0000 7ff40a3bb740 1 -- 172.21.15.94:0/4118502275 >> 172.21.15.94:0/4118502275 conn(0x55b08ae291a0 msgr2=0x55b08ae29580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:13.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.479+0000 7ff405caa640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff3e80a2440 0x7ff3e80a4900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0033000 tx=0x7ff3f00095c0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:13.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.481+0000 7ff40a3bb740 1 -- 172.21.15.94:0/4118502275 shutdown_connections 2024-02-17T12:01:13.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.481+0000 7ff40a3bb740 1 -- 172.21.15.94:0/4118502275 wait complete. 2024-02-17T12:01:13.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.482+0000 7ff40a3bb740 1 -- start start 2024-02-17T12:01:13.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.482+0000 7ff40a3bb740 1 Processor -- start 2024-02-17T12:01:13.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.482+0000 7ff40a3bb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae2e7b0 0x55b08ae28180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.482+0000 7ff40a3bb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae25510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.482+0000 7ff40a3bb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae40a30 0x55b08ae25ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.483+0000 7ff40a3bb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08aa61430 con 0x55b08ae2f0d0 2024-02-17T12:01:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.483+0000 7ff40a3bb740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08aa1de00 con 0x55b08ae40a30 2024-02-17T12:01:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.483+0000 7ff40a3bb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08ad12fd0 con 0x55b08ae2e7b0 2024-02-17T12:01:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.483+0000 7ff405caa640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae40a30 0x55b08ae25ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.483+0000 7ff405caa640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae40a30 0x55b08ae25ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:48260/0 (socket says 172.21.15.94:48260) 2024-02-17T12:01:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.483+0000 7ff405caa640 1 -- 172.21.15.94:0/2006345021 learned_addr learned my addr 172.21.15.94:0/2006345021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.483+0000 7ff4054a9640 1 --2- 172.21.15.94:0/2006345021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae2e7b0 0x55b08ae28180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.483+0000 7ff405caa640 1 -- 172.21.15.94:0/2006345021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae2e7b0 msgr2=0x55b08ae28180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.483+0000 7ff405caa640 1 --2- 172.21.15.94:0/2006345021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae2e7b0 0x55b08ae28180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.483+0000 7ff405caa640 1 -- 172.21.15.94:0/2006345021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 msgr2=0x55b08ae25510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.483+0000 7ff405caa640 1 --2- 172.21.15.94:0/2006345021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae25510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.483+0000 7ff405caa640 1 -- 172.21.15.94:0/2006345021 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4001aef20 con 0x55b08ae40a30 2024-02-17T12:01:13.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.484+0000 7ff40a3bb740 10 client.0 _ll_register_callbacks cb 0x55b08ad68fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:13.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.484+0000 7ff40a3bb740 1 -- 172.21.15.94:0/2006345021 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b08aa8d7e0 con 0x55b08ae40a30 2024-02-17T12:01:13.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.484+0000 7ff405caa640 1 --2- 172.21.15.94:0/2006345021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae40a30 0x55b08ae25ab0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0032b60 tx=0x7ff3f002c120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddfaf07c939cef52 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:13.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.484+0000 7ff3e4ff9640 10 client.25736 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:13.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.484+0000 7ff3e4ff9640 1 -- 172.21.15.94:0/2006345021 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0003970 con 0x55b08ae40a30 2024-02-17T12:01:13.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.485+0000 7ff3e4ff9640 1 -- 172.21.15.94:0/2006345021 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff3f0003ad0 con 0x55b08ae40a30 2024-02-17T12:01:13.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.485+0000 7ff3e4ff9640 1 -- 172.21.15.94:0/2006345021 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3f0003cd0 con 0x55b08ae40a30 2024-02-17T12:01:13.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.485+0000 7ff40a3bb740 10 client.25736 fetch_fsmap learned FSMap version 21 2024-02-17T12:01:13.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.485+0000 7ff40a3bb740 1 -- 172.21.15.94:0/2006345021 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({fsmap=21}) v3 -- 0x55b08ad8c610 con 0x55b08ae40a30 2024-02-17T12:01:13.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.486+0000 7ff3e4ff9640 1 -- 172.21.15.94:0/2006345021 <== mon.2 v2:172.21.15.125:3300/0 4 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff3f0052020 con 0x55b08ae40a30 2024-02-17T12:01:13.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.486+0000 7ff40a3bb740 10 client.25736 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:01:13.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.486+0000 7ff40a3bb740 10 client.25736 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:13.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.486+0000 7ff40a3bb740 20 client.25736 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:13.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.486+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/2006345021 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b08ae3d030 0x55b08ae3f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.486+0000 7ff40a3bb740 4 client.25736 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:13.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.486+0000 7ff40a3bb740 1 -- 172.21.15.94:0/2006345021 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b08aaca720 con 0x55b08ae3d030 2024-02-17T12:01:13.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.486+0000 7ff4054a9640 1 --2- 172.21.15.94:0/2006345021 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b08ae3d030 0x55b08ae3f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.487+0000 7ff3e4ff9640 10 client.25736 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:13.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.487+0000 7ff4054a9640 1 --2- 172.21.15.94:0/2006345021 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b08ae3d030 0x55b08ae3f4d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73f7e283de92c55b server_cookie=65ac758e24035595 in_seq=0 out_seq=0 2024-02-17T12:01:13.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.487+0000 7ff3e4ff9640 1 -- 172.21.15.94:0/2006345021 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b08aaca720 con 0x55b08ae3d030 2024-02-17T12:01:13.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.487+0000 7ff3e4ff9640 10 client.25736 handle_command_reply: tid=0 2024-02-17T12:01:13.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.487+0000 7ff40a3bb740 1 client.25736 shutdown 2024-02-17T12:01:13.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.488+0000 7ff40a3bb740 1 -- 172.21.15.94:0/2006345021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae40a30 msgr2=0x55b08ae25ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:13.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.488+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/2006345021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae40a30 0x55b08ae25ab0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0032b60 tx=0x7ff3f002c120 comp rx=0 tx=0).stop 2024-02-17T12:01:13.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.488+0000 7ff40a3bb740 1 -- 172.21.15.94:0/2006345021 shutdown_connections 2024-02-17T12:01:13.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.488+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/2006345021 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b08ae3d030 0x55b08ae3f4d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.488+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/2006345021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae40a30 0x55b08ae25ab0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.488+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/2006345021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae25510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.488+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/2006345021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae2e7b0 0x55b08ae28180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.488+0000 7ff40a3bb740 1 -- 172.21.15.94:0/2006345021 >> 172.21.15.94:0/2006345021 conn(0x55b08ae291a0 msgr2=0x55b08ae2af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:13.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.488+0000 7ff3e4ff9640 0 client.25736 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:13.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.488+0000 7ff40a3bb740 1 -- 172.21.15.94:0/2006345021 shutdown_connections 2024-02-17T12:01:13.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.488+0000 7ff40a3bb740 1 -- 172.21.15.94:0/2006345021 wait complete. 2024-02-17T12:01:13.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.488+0000 7ff40a3bb740 20 client.25736 trim_cache size 0 max 16384 2024-02-17T12:01:13.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.505+0000 7ff40a3bb740 1 Processor -- start 2024-02-17T12:01:13.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.505+0000 7ff40a3bb740 1 -- start start 2024-02-17T12:01:13.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.505+0000 7ff40a3bb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae2e7b0 0x55b08ae27c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.505+0000 7ff40a3bb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae28180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.505+0000 7ff40a3bb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae40a30 0x55b08ae33df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.505+0000 7ff404ca8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae28180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.505+0000 7ff404ca8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae28180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40874/0 (socket says 172.21.15.94:40874) 2024-02-17T12:01:13.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.506+0000 7ff404ca8640 1 -- 172.21.15.94:0/3635745388 learned_addr learned my addr 172.21.15.94:0/3635745388 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:13.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.506+0000 7ff40a3bb740 1 -- 172.21.15.94:0/3635745388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08ae25300 con 0x55b08ae2f0d0 2024-02-17T12:01:13.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.506+0000 7ff40a3bb740 1 -- 172.21.15.94:0/3635745388 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08ae29d70 con 0x55b08ae2e7b0 2024-02-17T12:01:13.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.506+0000 7ff40a3bb740 1 -- 172.21.15.94:0/3635745388 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08ae2f4b0 con 0x55b08ae40a30 2024-02-17T12:01:13.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.506+0000 7ff404ca8640 1 -- 172.21.15.94:0/3635745388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae40a30 msgr2=0x55b08ae33df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:13.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.506+0000 7ff404ca8640 1 --2- 172.21.15.94:0/3635745388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae40a30 0x55b08ae33df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.506+0000 7ff404ca8640 1 -- 172.21.15.94:0/3635745388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae2e7b0 msgr2=0x55b08ae27c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:13.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.506+0000 7ff404ca8640 1 --2- 172.21.15.94:0/3635745388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae2e7b0 0x55b08ae27c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.506+0000 7ff404ca8640 1 -- 172.21.15.94:0/3635745388 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b08ad28ac0 con 0x55b08ae2f0d0 2024-02-17T12:01:13.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.506+0000 7ff404ca8640 1 --2- 172.21.15.94:0/3635745388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae28180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4097310 tx=0x7ff3f40048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37661597c08fc210 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:13.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.507+0000 7ff3e6ffd640 1 -- 172.21.15.94:0/3635745388 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f40cbbd0 con 0x55b08ae2f0d0 2024-02-17T12:01:13.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.507+0000 7ff3e6ffd640 1 -- 172.21.15.94:0/3635745388 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3f4009b70 con 0x55b08ae2f0d0 2024-02-17T12:01:13.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.507+0000 7ff3e6ffd640 1 -- 172.21.15.94:0/3635745388 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f40a36a0 con 0x55b08ae2f0d0 2024-02-17T12:01:13.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.508+0000 7ff40a3bb740 1 -- 172.21.15.94:0/3635745388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 msgr2=0x55b08ae28180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:13.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.508+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/3635745388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae28180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4097310 tx=0x7ff3f40048b0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.508+0000 7ff40a3bb740 1 -- 172.21.15.94:0/3635745388 shutdown_connections 2024-02-17T12:01:13.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.508+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/3635745388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae40a30 0x55b08ae33df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.508+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/3635745388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae28180 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.508+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/3635745388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae2e7b0 0x55b08ae27c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.508+0000 7ff40a3bb740 1 -- 172.21.15.94:0/3635745388 >> 172.21.15.94:0/3635745388 conn(0x55b08ae291a0 msgr2=0x55b08ae20770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:13.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.508+0000 7ff40a3bb740 1 -- 172.21.15.94:0/3635745388 shutdown_connections 2024-02-17T12:01:13.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.509+0000 7ff40a3bb740 1 -- 172.21.15.94:0/3635745388 wait complete. 2024-02-17T12:01:13.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.509+0000 7ff40a3bb740 1 -- start start 2024-02-17T12:01:13.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.510+0000 7ff40a3bb740 1 Processor -- start 2024-02-17T12:01:13.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.510+0000 7ff40a3bb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae2e7b0 0x55b08ae320b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.510+0000 7ff40a3bb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae325f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.510+0000 7ff40a3bb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae40a30 0x55b08ae8fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.510+0000 7ff40a3bb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08ae90070 con 0x55b08ae2f0d0 2024-02-17T12:01:13.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.511+0000 7ff40a3bb740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08ae901e0 con 0x55b08ae40a30 2024-02-17T12:01:13.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.511+0000 7ff40a3bb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b08ae903a0 con 0x55b08ae2e7b0 2024-02-17T12:01:13.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.511+0000 7ff40a3bb740 10 client.0 _ll_register_callbacks cb 0x7ff3f4004800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:13.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.511+0000 7ff404ca8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae325f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.511+0000 7ff404ca8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae325f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40888/0 (socket says 172.21.15.94:40888) 2024-02-17T12:01:13.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.511+0000 7ff404ca8640 1 -- 172.21.15.94:0/15435152 learned_addr learned my addr 172.21.15.94:0/15435152 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:13.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.511+0000 7ff405caa640 1 --2- 172.21.15.94:0/15435152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae40a30 0x55b08ae8fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.511+0000 7ff404ca8640 1 -- 172.21.15.94:0/15435152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae2e7b0 msgr2=0x55b08ae320b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:13.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.511+0000 7ff404ca8640 1 --2- 172.21.15.94:0/15435152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae2e7b0 0x55b08ae320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.511+0000 7ff404ca8640 1 -- 172.21.15.94:0/15435152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae40a30 msgr2=0x55b08ae8fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:13.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.511+0000 7ff404ca8640 1 --2- 172.21.15.94:0/15435152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae40a30 0x55b08ae8fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.511+0000 7ff404ca8640 1 -- 172.21.15.94:0/15435152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b08ad4c570 con 0x55b08ae2f0d0 2024-02-17T12:01:13.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.512+0000 7ff404ca8640 1 --2- 172.21.15.94:0/15435152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae325f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4004940 tx=0x7ff3f409b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8486416505108f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:13.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.512+0000 7ff3e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:01:13.513 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.512+0000 7ff3e4ff9640 1 -- 172.21.15.94:0/15435152 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f409a860 con 0x55b08ae2f0d0 2024-02-17T12:01:13.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.512+0000 7ff3e4ff9640 1 -- 172.21.15.94:0/15435152 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff3f409af00 con 0x55b08ae2f0d0 2024-02-17T12:01:13.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.512+0000 7ff40a3bb740 1 -- 172.21.15.94:0/15435152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3f40971a0 con 0x55b08ae2f0d0 2024-02-17T12:01:13.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.512+0000 7ff3e4ff9640 1 -- 172.21.15.94:0/15435152 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3f409acd0 con 0x55b08ae2f0d0 2024-02-17T12:01:13.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.513+0000 7ff40a3bb740 10 client.16299 fetch_fsmap learned FSMap version 21 2024-02-17T12:01:13.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.513+0000 7ff40a3bb740 1 -- 172.21.15.94:0/15435152 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55b08ad4e6a0 con 0x55b08ae2f0d0 2024-02-17T12:01:13.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.513+0000 7ff3e4ff9640 1 -- 172.21.15.94:0/15435152 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff3f40971a0 con 0x55b08ae2f0d0 2024-02-17T12:01:13.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.513+0000 7ff3e4ff9640 1 -- 172.21.15.94:0/15435152 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff3f40cbee0 con 0x55b08ae2f0d0 2024-02-17T12:01:13.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.513+0000 7ff40a3bb740 10 client.16299 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:01:13.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.513+0000 7ff40a3bb740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:13.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.513+0000 7ff40a3bb740 20 client.16299 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:13.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.513+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/15435152 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b08ae97740 0x55b08ae99b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.513+0000 7ff40a3bb740 4 client.16299 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:01:13.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.513+0000 7ff40a3bb740 1 -- 172.21.15.94:0/15435152 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b08ab19890 con 0x55b08ae97740 2024-02-17T12:01:13.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.514+0000 7ff4054a9640 1 --2- 172.21.15.94:0/15435152 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b08ae97740 0x55b08ae99b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.515+0000 7ff4054a9640 1 --2- 172.21.15.94:0/15435152 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b08ae97740 0x55b08ae99b60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf1734e13cc7d0f3 server_cookie=954b7b80c8071af9 in_seq=0 out_seq=0 2024-02-17T12:01:13.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.515+0000 7ff3e4ff9640 10 client.16299 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:13.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.515+0000 7ff3e4ff9640 1 -- 172.21.15.94:0/15435152 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b08ab19890 con 0x55b08ae97740 2024-02-17T12:01:13.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.516+0000 7ff3e4ff9640 10 client.16299 handle_command_reply: tid=0 2024-02-17T12:01:13.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.516+0000 7ff40a3bb740 1 client.16299 shutdown 2024-02-17T12:01:13.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.516+0000 7ff40a3bb740 1 -- 172.21.15.94:0/15435152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 msgr2=0x55b08ae325f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:13.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.516+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/15435152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae325f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4004940 tx=0x7ff3f409b620 comp rx=0 tx=0).stop 2024-02-17T12:01:13.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.516+0000 7ff40a3bb740 1 -- 172.21.15.94:0/15435152 shutdown_connections 2024-02-17T12:01:13.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.516+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/15435152 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b08ae97740 0x55b08ae99b60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.516+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/15435152 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b08ae40a30 0x55b08ae8fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.516+0000 7ff3e4ff9640 0 client.16299 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:13.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.516+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/15435152 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b08ae2f0d0 0x55b08ae325f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.516+0000 7ff40a3bb740 1 --2- 172.21.15.94:0/15435152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b08ae2e7b0 0x55b08ae320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.516+0000 7ff40a3bb740 1 -- 172.21.15.94:0/15435152 >> 172.21.15.94:0/15435152 conn(0x55b08ae291a0 msgr2=0x55b08ae3fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:13.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.517+0000 7ff40a3bb740 1 -- 172.21.15.94:0/15435152 shutdown_connections 2024-02-17T12:01:13.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.517+0000 7ff40a3bb740 1 -- 172.21.15.94:0/15435152 wait complete. 2024-02-17T12:01:13.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.517+0000 7ff40a3bb740 20 client.16299 trim_cache size 0 max 16384 2024-02-17T12:01:13.519 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:01:13.519 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:01:13.520 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:01:13.520 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:01:13.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.518+0000 7ff40770d640 1 -- 172.21.15.94:0/3594796650 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff3e80a2440 msgr2=0x7ff3e80a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:13.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.518+0000 7ff40770d640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff3e80a2440 0x7ff3e80a4900 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0033000 tx=0x7ff3f00095c0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.518+0000 7ff40770d640 1 -- 172.21.15.94:0/3594796650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4001085f0 msgr2=0x7ff4001a2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:13.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.518+0000 7ff40770d640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4001085f0 0x7ff4001a2760 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400d6a0 tx=0x7ff3f400db60 comp rx=0 tx=0).stop 2024-02-17T12:01:13.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.518+0000 7ff40770d640 1 -- 172.21.15.94:0/3594796650 shutdown_connections 2024-02-17T12:01:13.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.518+0000 7ff40770d640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff3e80a2440 0x7ff3e80a4900 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.518+0000 7ff40770d640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff40010cc40 0x7ff4001a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.518+0000 7ff40770d640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff4001085f0 0x7ff4001a2760 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.518+0000 7ff40770d640 1 --2- 172.21.15.94:0/3594796650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff400108180 0x7ff4001a2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.518+0000 7ff40770d640 1 -- 172.21.15.94:0/3594796650 >> 172.21.15.94:0/3594796650 conn(0x7ff40006c270 msgr2=0x7ff40010bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:13.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.518+0000 7ff40770d640 1 -- 172.21.15.94:0/3594796650 shutdown_connections 2024-02-17T12:01:13.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.518+0000 7ff40770d640 1 -- 172.21.15.94:0/3594796650 wait complete. 2024-02-17T12:01:13.530 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:01:13.530 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:01:13.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.689+0000 7f33a5b7a640 1 -- 172.21.15.94:0/3989872591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33a01035b0 msgr2=0x7f33a0103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:13.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.689+0000 7f33a5b7a640 1 --2- 172.21.15.94:0/3989872591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33a01035b0 0x7f33a0103990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3390009930 tx=0x7f339002bc50 comp rx=0 tx=0).stop 2024-02-17T12:01:13.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.690+0000 7f33a5b7a640 1 -- 172.21.15.94:0/3989872591 shutdown_connections 2024-02-17T12:01:13.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.690+0000 7f33a5b7a640 1 --2- 172.21.15.94:0/3989872591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33a0108680 0x7f33a0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.690+0000 7f33a5b7a640 1 --2- 172.21.15.94:0/3989872591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33a0103f60 0x7f33a0107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.690+0000 7f33a5b7a640 1 --2- 172.21.15.94:0/3989872591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33a01035b0 0x7f33a0103990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.690+0000 7f33a5b7a640 1 -- 172.21.15.94:0/3989872591 >> 172.21.15.94:0/3989872591 conn(0x7f33a00fee30 msgr2=0x7f33a0101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:13.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.690+0000 7f33a5b7a640 1 -- 172.21.15.94:0/3989872591 shutdown_connections 2024-02-17T12:01:13.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.691+0000 7f33a5b7a640 1 -- 172.21.15.94:0/3989872591 wait complete. 2024-02-17T12:01:13.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.691+0000 7f33a5b7a640 1 Processor -- start 2024-02-17T12:01:13.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f33a5b7a640 1 -- start start 2024-02-17T12:01:13.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f33a5b7a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33a01035b0 0x7f33a019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f33a5b7a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33a0108680 0x7f33a019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f33a5b7a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33a0199810 0x7f33a0199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f33a5b7a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a019a2c0 con 0x7f33a0199810 2024-02-17T12:01:13.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f33a5b7a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a019a430 con 0x7f33a01035b0 2024-02-17T12:01:13.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f33a5b7a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a019a5a0 con 0x7f33a0108680 2024-02-17T12:01:13.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f339f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33a01035b0 0x7f33a019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f339ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33a0199810 0x7f33a0199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f339f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33a01035b0 0x7f33a019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:48306/0 (socket says 172.21.15.94:48306) 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f339ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33a0199810 0x7f33a0199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40898/0 (socket says 172.21.15.94:40898) 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f339f7fe640 1 -- 172.21.15.94:0/2375235235 learned_addr learned my addr 172.21.15.94:0/2375235235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f339effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33a0108680 0x7f33a019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f339f7fe640 1 -- 172.21.15.94:0/2375235235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33a0108680 msgr2=0x7f33a019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f339f7fe640 1 --2- 172.21.15.94:0/2375235235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33a0108680 0x7f33a019fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f339f7fe640 1 -- 172.21.15.94:0/2375235235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33a0199810 msgr2=0x7f33a0199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f339f7fe640 1 --2- 172.21.15.94:0/2375235235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33a0199810 0x7f33a0199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.692+0000 7f339f7fe640 1 -- 172.21.15.94:0/2375235235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3390009560 con 0x7f33a01035b0 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.693+0000 7f339effd640 1 --2- 172.21.15.94:0/2375235235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33a0108680 0x7f33a019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.693+0000 7f339f7fe640 1 --2- 172.21.15.94:0/2375235235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33a01035b0 0x7f33a019f6a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3390000c00 tx=0x7f339002cda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.693+0000 7f339cff9640 1 -- 172.21.15.94:0/2375235235 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33900043d0 con 0x7f33a01035b0 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.693+0000 7f339cff9640 1 -- 172.21.15.94:0/2375235235 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33900052d0 con 0x7f33a01035b0 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.693+0000 7f33a5b7a640 1 -- 172.21.15.94:0/2375235235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a01a64f0 con 0x7f33a01035b0 2024-02-17T12:01:13.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.693+0000 7f33a5b7a640 1 -- 172.21.15.94:0/2375235235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a01a69d0 con 0x7f33a01035b0 2024-02-17T12:01:13.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.693+0000 7f339cff9640 1 -- 172.21.15.94:0/2375235235 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390003d30 con 0x7f33a01035b0 2024-02-17T12:01:13.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.695+0000 7f33a5b7a640 1 -- 172.21.15.94:0/2375235235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a0104a80 con 0x7f33a01035b0 2024-02-17T12:01:13.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.697+0000 7f339cff9640 1 -- 172.21.15.94:0/2375235235 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3390004df0 con 0x7f33a01035b0 2024-02-17T12:01:13.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.698+0000 7f339cff9640 1 --2- 172.21.15.94:0/2375235235 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f33880a2230 0x7f33880a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:13.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.698+0000 7f339effd640 1 --2- 172.21.15.94:0/2375235235 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f33880a2230 0x7f33880a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:13.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.698+0000 7f339effd640 1 --2- 172.21.15.94:0/2375235235 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f33880a2230 0x7f33880a46f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f33a00ff2e0 tx=0x7f339400b390 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:13.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.698+0000 7f339cff9640 1 -- 172.21.15.94:0/2375235235 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f33900f07a0 con 0x7f33a01035b0 2024-02-17T12:01:13.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:13.705+0000 7f339cff9640 1 -- 172.21.15.94:0/2375235235 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f33900b9650 con 0x7f33a01035b0 2024-02-17T12:01:14.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.067+0000 7f33a5b7a640 1 -- 172.21.15.94:0/2375235235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33a0067f70 con 0x7f33a01035b0 2024-02-17T12:01:14.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.069+0000 7f339cff9640 1 -- 172.21.15.94:0/2375235235 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+11346 (secure 0 0 0) 0x7f3390043090 con 0x7f33a01035b0 2024-02-17T12:01:14.068 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:01:14.069 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T11:57:51.854453+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24454":{"gid":24454,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6835/2720928164","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2720928164},{"type":"v1","addr":"172.21.15.189:6835","nonce":2720928164}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:01:14.069 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 21 2024-02-17T12:01:14.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.074+0000 7f33a5b7a640 1 -- 172.21.15.94:0/2375235235 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f33880a2230 msgr2=0x7f33880a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.074+0000 7f33a5b7a640 1 --2- 172.21.15.94:0/2375235235 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f33880a2230 0x7f33880a46f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f33a00ff2e0 tx=0x7f339400b390 comp rx=0 tx=0).stop 2024-02-17T12:01:14.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.074+0000 7f33a5b7a640 1 -- 172.21.15.94:0/2375235235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33a01035b0 msgr2=0x7f33a019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.074+0000 7f33a5b7a640 1 --2- 172.21.15.94:0/2375235235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33a01035b0 0x7f33a019f6a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3390000c00 tx=0x7f339002cda0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.077+0000 7f33a5b7a640 1 -- 172.21.15.94:0/2375235235 shutdown_connections 2024-02-17T12:01:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.077+0000 7f33a5b7a640 1 --2- 172.21.15.94:0/2375235235 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f33880a2230 0x7f33880a46f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.077+0000 7f33a5b7a640 1 --2- 172.21.15.94:0/2375235235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f33a0199810 0x7f33a0199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.077+0000 7f33a5b7a640 1 --2- 172.21.15.94:0/2375235235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33a0108680 0x7f33a019fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.077+0000 7f33a5b7a640 1 --2- 172.21.15.94:0/2375235235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f33a01035b0 0x7f33a019f6a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.077+0000 7f33a5b7a640 1 -- 172.21.15.94:0/2375235235 >> 172.21.15.94:0/2375235235 conn(0x7f33a00fee30 msgr2=0x7f33a0100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.079+0000 7f33a5b7a640 1 -- 172.21.15.94:0/2375235235 shutdown_connections 2024-02-17T12:01:14.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.079+0000 7f33a5b7a640 1 -- 172.21.15.94:0/2375235235 wait complete. 2024-02-17T12:01:14.098 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:01:14.099 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:01:14.133 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:13 smithi094 ceph-mon[32499]: pgmap v96: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 MiB/s wr, 1.62k op/s 2024-02-17T12:01:14.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:13 smithi189 ceph-mon[36588]: pgmap v96: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 MiB/s wr, 1.62k op/s 2024-02-17T12:01:14.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:13 smithi125 ceph-mon[36819]: pgmap v96: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 40 MiB/s wr, 1.62k op/s 2024-02-17T12:01:14.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.250+0000 7f135560f640 1 -- 172.21.15.94:0/1529159474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1350109e50 msgr2=0x7f13500ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.251+0000 7f135560f640 1 --2- 172.21.15.94:0/1529159474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1350109e50 0x7f13500ff250 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1344009900 tx=0x7f134402bc30 comp rx=0 tx=0).stop 2024-02-17T12:01:14.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.251+0000 7f135560f640 1 -- 172.21.15.94:0/1529159474 shutdown_connections 2024-02-17T12:01:14.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.251+0000 7f135560f640 1 --2- 172.21.15.94:0/1529159474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f13500ff980 0x7f13501102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.251+0000 7f135560f640 1 --2- 172.21.15.94:0/1529159474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1350109e50 0x7f13500ff250 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.251+0000 7f135560f640 1 --2- 172.21.15.94:0/1529159474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13501094a0 0x7f1350109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.251+0000 7f135560f640 1 -- 172.21.15.94:0/1529159474 >> 172.21.15.94:0/1529159474 conn(0x7f13500761d0 msgr2=0x7f13500fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.251+0000 7f135560f640 1 -- 172.21.15.94:0/1529159474 shutdown_connections 2024-02-17T12:01:14.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.251+0000 7f135560f640 1 -- 172.21.15.94:0/1529159474 wait complete. 2024-02-17T12:01:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.252+0000 7f135560f640 1 Processor -- start 2024-02-17T12:01:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.252+0000 7f135560f640 1 -- start start 2024-02-17T12:01:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f135560f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13500ff980 0x7f1350110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f135560f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13501094a0 0x7f1350072b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f135560f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f135010d140 0x7f135010d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f135560f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1350073110 con 0x7f13500ff980 2024-02-17T12:01:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f135560f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135010db30 con 0x7f135010d140 2024-02-17T12:01:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f135560f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135010dcf0 con 0x7f13501094a0 2024-02-17T12:01:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f134effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13500ff980 0x7f1350110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f134effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13500ff980 0x7f1350110050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40920/0 (socket says 172.21.15.94:40920) 2024-02-17T12:01:14.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f134effd640 1 -- 172.21.15.94:0/1991413335 learned_addr learned my addr 172.21.15.94:0/1991413335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f134f7fe640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f135010d140 0x7f135010d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f134e7fc640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13501094a0 0x7f1350072b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f134effd640 1 -- 172.21.15.94:0/1991413335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13501094a0 msgr2=0x7f1350072b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f134effd640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13501094a0 0x7f1350072b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f134effd640 1 -- 172.21.15.94:0/1991413335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f135010d140 msgr2=0x7f135010d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f134effd640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f135010d140 0x7f135010d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f134effd640 1 -- 172.21.15.94:0/1991413335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1344009560 con 0x7f13500ff980 2024-02-17T12:01:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f134effd640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13500ff980 0x7f1350110050 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f134000e9c0 tx=0x7f134000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.253+0000 7f1337fff640 1 -- 172.21.15.94:0/1991413335 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400096e0 con 0x7f13500ff980 2024-02-17T12:01:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.254+0000 7f135560f640 1 -- 172.21.15.94:0/1991413335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13501b3430 con 0x7f13500ff980 2024-02-17T12:01:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.254+0000 7f135560f640 1 -- 172.21.15.94:0/1991413335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13501b37a0 con 0x7f13500ff980 2024-02-17T12:01:14.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.254+0000 7f1337fff640 1 -- 172.21.15.94:0/1991413335 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f134000cd80 con 0x7f13500ff980 2024-02-17T12:01:14.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.254+0000 7f1337fff640 1 -- 172.21.15.94:0/1991413335 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1340010430 con 0x7f13500ff980 2024-02-17T12:01:14.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.257+0000 7f1337fff640 1 -- 172.21.15.94:0/1991413335 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1340003680 con 0x7f13500ff980 2024-02-17T12:01:14.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.257+0000 7f13581dc740 1 Processor -- start 2024-02-17T12:01:14.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.257+0000 7f13581dc740 1 -- start start 2024-02-17T12:01:14.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.257+0000 7f13581dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40250 0x55a961d40630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.257+0000 7f13581dc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d40b70 0x55a961d49210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.257+0000 7f13581dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d49750 0x55a961d4bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.257+0000 7f13581dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961cfb960 con 0x55a961d49750 2024-02-17T12:01:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.257+0000 7f13581dc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961caf310 con 0x55a961d40b70 2024-02-17T12:01:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.257+0000 7f13581dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961cac840 con 0x55a961d40250 2024-02-17T12:01:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40250 0x55a961d40630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f134effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d40b70 0x55a961d49210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40250 0x55a961d40630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39444/0 (socket says 172.21.15.94:39444) 2024-02-17T12:01:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f134f7fe640 1 -- 172.21.15.94:0/688742891 learned_addr learned my addr 172.21.15.94:0/688742891 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f134e7fc640 1 --2- 172.21.15.94:0/688742891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d49750 0x55a961d4bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f1337fff640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f13380a2230 0x7f13380a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f134f7fe640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f13380a2230 0x7f13380a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f1337fff640 1 -- 172.21.15.94:0/1991413335 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1340014070 con 0x7f13500ff980 2024-02-17T12:01:14.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f134e7fc640 1 -- 172.21.15.94:0/688742891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40250 msgr2=0x55a961d40630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f134e7fc640 1 --2- 172.21.15.94:0/688742891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40250 0x55a961d40630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f134e7fc640 1 -- 172.21.15.94:0/688742891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d40b70 msgr2=0x55a961d49210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f134e7fc640 1 --2- 172.21.15.94:0/688742891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d40b70 0x55a961d49210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f134e7fc640 1 -- 172.21.15.94:0/688742891 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a961c6a060 con 0x55a961d49750 2024-02-17T12:01:14.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f134e7fc640 1 --2- 172.21.15.94:0/688742891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d49750 0x55a961d4bb40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1344036040 tx=0x7f134402c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76dfbfaedbc90d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f13357fa640 1 -- 172.21.15.94:0/688742891 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344004900 con 0x55a961d49750 2024-02-17T12:01:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.258+0000 7f134f7fe640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f13380a2230 0x7f13380a46f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1348009fd0 tx=0x7f1348004ab0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.259+0000 7f13357fa640 1 -- 172.21.15.94:0/688742891 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1344004a60 con 0x55a961d49750 2024-02-17T12:01:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.259+0000 7f13357fa640 1 -- 172.21.15.94:0/688742891 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1344005170 con 0x55a961d49750 2024-02-17T12:01:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.259+0000 7f13581dc740 1 -- 172.21.15.94:0/688742891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d49750 msgr2=0x55a961d4bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.259+0000 7f13581dc740 1 --2- 172.21.15.94:0/688742891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d49750 0x55a961d4bb40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1344036040 tx=0x7f134402c720 comp rx=0 tx=0).stop 2024-02-17T12:01:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.261+0000 7f13581dc740 1 -- 172.21.15.94:0/688742891 shutdown_connections 2024-02-17T12:01:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.261+0000 7f13581dc740 1 --2- 172.21.15.94:0/688742891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d49750 0x55a961d4bb40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.261+0000 7f13581dc740 1 --2- 172.21.15.94:0/688742891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d40b70 0x55a961d49210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.261+0000 7f13581dc740 1 --2- 172.21.15.94:0/688742891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40250 0x55a961d40630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.261+0000 7f13581dc740 1 -- 172.21.15.94:0/688742891 >> 172.21.15.94:0/688742891 conn(0x55a961d3a980 msgr2=0x55a961d3ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.261+0000 7f13581dc740 1 -- 172.21.15.94:0/688742891 shutdown_connections 2024-02-17T12:01:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.261+0000 7f13581dc740 1 -- 172.21.15.94:0/688742891 wait complete. 2024-02-17T12:01:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.261+0000 7f13581dc740 1 -- start start 2024-02-17T12:01:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f13581dc740 1 Processor -- start 2024-02-17T12:01:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f13581dc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d40250 0x55a961d46740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f13581dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40b70 0x55a961d46c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f13581dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d49750 0x55a961d471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f13581dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961ca8020 con 0x55a961d49750 2024-02-17T12:01:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f13581dc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a96197b430 con 0x55a961d40250 2024-02-17T12:01:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f13581dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961937e00 con 0x55a961d40b70 2024-02-17T12:01:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f13581dc740 10 client.0 _ll_register_callbacks cb 0x55a961c4fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d49750 0x55a961d471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f134effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d40250 0x55a961d46740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f134effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d40250 0x55a961d46740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:48360/0 (socket says 172.21.15.94:48360) 2024-02-17T12:01:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d49750 0x55a961d471c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40926/0 (socket says 172.21.15.94:40926) 2024-02-17T12:01:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f134effd640 1 -- 172.21.15.94:0/3237418433 learned_addr learned my addr 172.21.15.94:0/3237418433 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f134e7fc640 1 --2- 172.21.15.94:0/3237418433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40b70 0x55a961d46c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f134effd640 1 -- 172.21.15.94:0/3237418433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40b70 msgr2=0x55a961d46c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f134effd640 1 --2- 172.21.15.94:0/3237418433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40b70 0x55a961d46c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f134effd640 1 -- 172.21.15.94:0/3237418433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d49750 msgr2=0x55a961d471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f134effd640 1 --2- 172.21.15.94:0/3237418433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d49750 0x55a961d471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.262+0000 7f134effd640 1 -- 172.21.15.94:0/3237418433 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1350110aa0 con 0x55a961d40250 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f134effd640 1 --2- 172.21.15.94:0/3237418433 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d40250 0x55a961d46740 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f13400cbc10 tx=0x7f134009a990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1052c8d494b31d52 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f1332ff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f1332ff5640 1 -- 172.21.15.94:0/3237418433 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400aeea0 con 0x55a961d40250 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f1332ff5640 1 -- 172.21.15.94:0/3237418433 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f134009ae50 con 0x55a961d40250 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f13581dc740 1 -- 172.21.15.94:0/3237418433 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9619a77e0 con 0x55a961d40250 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f1332ff5640 1 -- 172.21.15.94:0/3237418433 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f134009efa0 con 0x55a961d40250 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f13581dc740 10 client.25757 fetch_fsmap learned FSMap version 21 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f13581dc740 1 -- 172.21.15.94:0/3237418433 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55a961ca66a0 con 0x55a961d40250 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f1332ff5640 1 -- 172.21.15.94:0/3237418433 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13400b7020 con 0x55a961d40250 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f1332ff5640 1 -- 172.21.15.94:0/3237418433 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f134001f060 con 0x55a961d40250 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f13581dc740 10 client.25757 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f13581dc740 10 client.25757 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:01:14.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f13581dc740 20 client.25757 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:14.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.263+0000 7f13581dc740 1 --2- 172.21.15.94:0/3237418433 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55a961d59bb0 0x55a961d5bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.264+0000 7f13581dc740 4 client.25757 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:14.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.264+0000 7f13581dc740 1 -- 172.21.15.94:0/3237418433 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9619e4720 con 0x55a961d59bb0 2024-02-17T12:01:14.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.264+0000 7f134e7fc640 1 --2- 172.21.15.94:0/3237418433 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55a961d59bb0 0x55a961d5bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.282+0000 7f134e7fc640 1 --2- 172.21.15.94:0/3237418433 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55a961d59bb0 0x55a961d5bfa0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27ed38716e751190 server_cookie=6b368289e290101c in_seq=0 out_seq=0 2024-02-17T12:01:14.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.282+0000 7f1332ff5640 10 client.25757 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:01:14.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.284+0000 7f1332ff5640 1 -- 172.21.15.94:0/3237418433 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9619e4720 con 0x55a961d59bb0 2024-02-17T12:01:14.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.284+0000 7f1332ff5640 10 client.25757 handle_command_reply: tid=0 2024-02-17T12:01:14.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.284+0000 7f13581dc740 1 client.25757 shutdown 2024-02-17T12:01:14.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.284+0000 7f13581dc740 1 -- 172.21.15.94:0/3237418433 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d40250 msgr2=0x55a961d46740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.284+0000 7f13581dc740 1 --2- 172.21.15.94:0/3237418433 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d40250 0x55a961d46740 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f13400cbc10 tx=0x7f134009a990 comp rx=0 tx=0).stop 2024-02-17T12:01:14.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.284+0000 7f13581dc740 1 -- 172.21.15.94:0/3237418433 shutdown_connections 2024-02-17T12:01:14.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.284+0000 7f13581dc740 1 --2- 172.21.15.94:0/3237418433 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d49750 0x55a961d471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.284+0000 7f13581dc740 1 --2- 172.21.15.94:0/3237418433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40b70 0x55a961d46c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.284+0000 7f13581dc740 1 --2- 172.21.15.94:0/3237418433 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55a961d59bb0 0x55a961d5bfa0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.284+0000 7f13581dc740 1 --2- 172.21.15.94:0/3237418433 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d40250 0x55a961d46740 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.284+0000 7f13581dc740 1 -- 172.21.15.94:0/3237418433 >> 172.21.15.94:0/3237418433 conn(0x55a961d3a980 msgr2=0x55a961d3c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.284+0000 7f1332ff5640 0 client.25757 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:01:14.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.285+0000 7f13581dc740 1 -- 172.21.15.94:0/3237418433 shutdown_connections 2024-02-17T12:01:14.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.285+0000 7f13581dc740 1 -- 172.21.15.94:0/3237418433 wait complete. 2024-02-17T12:01:14.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.286+0000 7f13581dc740 20 client.25757 trim_cache size 0 max 16384 2024-02-17T12:01:14.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.308+0000 7f13581dc740 1 Processor -- start 2024-02-17T12:01:14.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.309+0000 7f13581dc740 1 -- start start 2024-02-17T12:01:14.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.309+0000 7f13581dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40250 0x55a961d39a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.309+0000 7f13581dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d40b70 0x55a961d46fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.309+0000 7f13581dc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d49750 0x55a961d505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.309+0000 7f13581dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961c2d130 con 0x55a961d40b70 2024-02-17T12:01:14.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.309+0000 7f13581dc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961d362b0 con 0x55a961d49750 2024-02-17T12:01:14.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.309+0000 7f13581dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961d3f9f0 con 0x55a961d40250 2024-02-17T12:01:14.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.309+0000 7f134effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40250 0x55a961d39a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.309+0000 7f134effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40250 0x55a961d39a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39448/0 (socket says 172.21.15.94:39448) 2024-02-17T12:01:14.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.309+0000 7f134effd640 1 -- 172.21.15.94:0/3893441030 learned_addr learned my addr 172.21.15.94:0/3893441030 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.309+0000 7f134e7fc640 1 --2- 172.21.15.94:0/3893441030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d40b70 0x55a961d46fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.310+0000 7f134f7fe640 1 --2- 172.21.15.94:0/3893441030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d49750 0x55a961d505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.310+0000 7f134effd640 1 -- 172.21.15.94:0/3893441030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d49750 msgr2=0x55a961d505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.310+0000 7f134effd640 1 --2- 172.21.15.94:0/3893441030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d49750 0x55a961d505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.310+0000 7f134effd640 1 -- 172.21.15.94:0/3893441030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d40b70 msgr2=0x55a961d46fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.310+0000 7f134effd640 1 --2- 172.21.15.94:0/3893441030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d40b70 0x55a961d46fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.310+0000 7f134effd640 1 -- 172.21.15.94:0/3893441030 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a961c42c20 con 0x55a961d40250 2024-02-17T12:01:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.310+0000 7f134effd640 1 --2- 172.21.15.94:0/3893441030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40250 0x55a961d39a00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f13400058c0 tx=0x7f13400976d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=443441e274d50b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.311+0000 7f1334ff9640 1 -- 172.21.15.94:0/3893441030 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400aeea0 con 0x55a961d40250 2024-02-17T12:01:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.311+0000 7f1334ff9640 1 -- 172.21.15.94:0/3893441030 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13400b7030 con 0x55a961d40250 2024-02-17T12:01:14.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.311+0000 7f1334ff9640 1 -- 172.21.15.94:0/3893441030 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134009efa0 con 0x55a961d40250 2024-02-17T12:01:14.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.311+0000 7f13581dc740 1 -- 172.21.15.94:0/3893441030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40250 msgr2=0x55a961d39a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.311+0000 7f13581dc740 1 --2- 172.21.15.94:0/3893441030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40250 0x55a961d39a00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f13400058c0 tx=0x7f13400976d0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.311+0000 7f13581dc740 1 -- 172.21.15.94:0/3893441030 shutdown_connections 2024-02-17T12:01:14.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.311+0000 7f13581dc740 1 --2- 172.21.15.94:0/3893441030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d49750 0x55a961d505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.311+0000 7f13581dc740 1 --2- 172.21.15.94:0/3893441030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d40b70 0x55a961d46fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.311+0000 7f13581dc740 1 --2- 172.21.15.94:0/3893441030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40250 0x55a961d39a00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.311+0000 7f13581dc740 1 -- 172.21.15.94:0/3893441030 >> 172.21.15.94:0/3893441030 conn(0x55a961d3a980 msgr2=0x55a961d392f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f13581dc740 1 -- 172.21.15.94:0/3893441030 shutdown_connections 2024-02-17T12:01:14.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f13581dc740 1 -- 172.21.15.94:0/3893441030 wait complete. 2024-02-17T12:01:14.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f13581dc740 1 -- start start 2024-02-17T12:01:14.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f13581dc740 1 Processor -- start 2024-02-17T12:01:14.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f13581dc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d40250 0x55a961daa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f13581dc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40b70 0x55a961daa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f13581dc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d49750 0x55a961d4f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f13581dc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961d4fe60 con 0x55a961d40250 2024-02-17T12:01:14.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f13581dc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961d4ffd0 con 0x55a961d49750 2024-02-17T12:01:14.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f13581dc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a961d50190 con 0x55a961d40b70 2024-02-17T12:01:14.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f13581dc740 10 client.0 _ll_register_callbacks cb 0x7f1344009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:14.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d49750 0x55a961d4f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f134effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d40250 0x55a961daa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f134effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d40250 0x55a961daa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40958/0 (socket says 172.21.15.94:40958) 2024-02-17T12:01:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f134f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d49750 0x55a961d4f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:48378/0 (socket says 172.21.15.94:48378) 2024-02-17T12:01:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f134effd640 1 -- 172.21.15.94:0/1307502916 learned_addr learned my addr 172.21.15.94:0/1307502916 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.312+0000 7f134e7fc640 1 --2- 172.21.15.94:0/1307502916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40b70 0x55a961daa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f134effd640 1 -- 172.21.15.94:0/1307502916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40b70 msgr2=0x55a961daa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f134effd640 1 --2- 172.21.15.94:0/1307502916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40b70 0x55a961daa9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f134effd640 1 -- 172.21.15.94:0/1307502916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d49750 msgr2=0x55a961d4f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f134effd640 1 --2- 172.21.15.94:0/1307502916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d49750 0x55a961d4f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f134effd640 1 -- 172.21.15.94:0/1307502916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a961ca66a0 con 0x55a961d40250 2024-02-17T12:01:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f134e7fc640 1 --2- 172.21.15.94:0/1307502916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40b70 0x55a961daa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f134effd640 1 --2- 172.21.15.94:0/1307502916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d40250 0x55a961daa480 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1340099100 tx=0x7f13400a34b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c352226b7d7edfd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f1332ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:01:14.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f1332ff5640 1 -- 172.21.15.94:0/1307502916 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13400aeea0 con 0x55a961d40250 2024-02-17T12:01:14.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f1332ff5640 1 -- 172.21.15.94:0/1307502916 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f13400b8080 con 0x55a961d40250 2024-02-17T12:01:14.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f13581dc740 1 -- 172.21.15.94:0/1307502916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1344004a60 con 0x55a961d40250 2024-02-17T12:01:14.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f1332ff5640 1 -- 172.21.15.94:0/1307502916 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13400a35f0 con 0x55a961d40250 2024-02-17T12:01:14.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f13581dc740 10 client.16323 fetch_fsmap learned FSMap version 21 2024-02-17T12:01:14.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.313+0000 7f13581dc740 1 -- 172.21.15.94:0/1307502916 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x55a961c688c0 con 0x55a961d40250 2024-02-17T12:01:14.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.314+0000 7f1332ff5640 1 -- 172.21.15.94:0/1307502916 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1344004a60 con 0x55a961d40250 2024-02-17T12:01:14.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.314+0000 7f1332ff5640 1 -- 172.21.15.94:0/1307502916 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f134001f070 con 0x55a961d40250 2024-02-17T12:01:14.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.314+0000 7f13581dc740 10 client.16323 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:01:14.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.314+0000 7f13581dc740 10 client.16323 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:01:14.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.314+0000 7f13581dc740 20 client.16323 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:14.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.314+0000 7f13581dc740 1 --2- 172.21.15.94:0/1307502916 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55a961db1e80 0x55a961db4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.314+0000 7f13581dc740 4 client.16323 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:01:14.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.314+0000 7f13581dc740 1 -- 172.21.15.94:0/1307502916 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a961a33890 con 0x55a961db1e80 2024-02-17T12:01:14.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.315+0000 7f134e7fc640 1 --2- 172.21.15.94:0/1307502916 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55a961db1e80 0x55a961db4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.315+0000 7f1332ff5640 10 client.16323 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:01:14.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.315+0000 7f134e7fc640 1 --2- 172.21.15.94:0/1307502916 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55a961db1e80 0x55a961db4270 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2741887ae1c8ce79 server_cookie=cf31cb6a5f7fb14e in_seq=0 out_seq=0 2024-02-17T12:01:14.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.315+0000 7f1332ff5640 1 -- 172.21.15.94:0/1307502916 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a961a33890 con 0x55a961db1e80 2024-02-17T12:01:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.315+0000 7f1332ff5640 10 client.16323 handle_command_reply: tid=0 2024-02-17T12:01:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.316+0000 7f13581dc740 1 client.16323 shutdown 2024-02-17T12:01:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.316+0000 7f13581dc740 1 -- 172.21.15.94:0/1307502916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d40250 msgr2=0x55a961daa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.316+0000 7f13581dc740 1 --2- 172.21.15.94:0/1307502916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d40250 0x55a961daa480 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1340099100 tx=0x7f13400a34b0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.316+0000 7f13581dc740 1 -- 172.21.15.94:0/1307502916 shutdown_connections 2024-02-17T12:01:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.316+0000 7f13581dc740 1 --2- 172.21.15.94:0/1307502916 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55a961db1e80 0x55a961db4270 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.316+0000 7f13581dc740 1 --2- 172.21.15.94:0/1307502916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a961d49750 0x55a961d4f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.316+0000 7f13581dc740 1 --2- 172.21.15.94:0/1307502916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a961d40b70 0x55a961daa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.316+0000 7f1332ff5640 0 client.16323 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:01:14.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.316+0000 7f13581dc740 1 --2- 172.21.15.94:0/1307502916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a961d40250 0x55a961daa480 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.316+0000 7f13581dc740 1 -- 172.21.15.94:0/1307502916 >> 172.21.15.94:0/1307502916 conn(0x55a961d3a980 msgr2=0x55a961d39b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.316+0000 7f13581dc740 1 -- 172.21.15.94:0/1307502916 shutdown_connections 2024-02-17T12:01:14.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.316+0000 7f13581dc740 1 -- 172.21.15.94:0/1307502916 wait complete. 2024-02-17T12:01:14.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.316+0000 7f13581dc740 20 client.16323 trim_cache size 0 max 16384 2024-02-17T12:01:14.321 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:01:14.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.317+0000 7f135560f640 1 -- 172.21.15.94:0/1991413335 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f13380a2230 msgr2=0x7f13380a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.317+0000 7f135560f640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f13380a2230 0x7f13380a46f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1348009fd0 tx=0x7f1348004ab0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.317+0000 7f135560f640 1 -- 172.21.15.94:0/1991413335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13500ff980 msgr2=0x7f1350110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.317+0000 7f135560f640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13500ff980 0x7f1350110050 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f134000e9c0 tx=0x7f134000ee80 comp rx=0 tx=0).stop 2024-02-17T12:01:14.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.317+0000 7f135560f640 1 -- 172.21.15.94:0/1991413335 shutdown_connections 2024-02-17T12:01:14.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.317+0000 7f135560f640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f13380a2230 0x7f13380a46f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.317+0000 7f135560f640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f135010d140 0x7f135010d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.317+0000 7f135560f640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13501094a0 0x7f1350072b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.317+0000 7f135560f640 1 --2- 172.21.15.94:0/1991413335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f13500ff980 0x7f1350110050 secure :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f134000e9c0 tx=0x7f134000ee80 comp rx=0 tx=0).stop 2024-02-17T12:01:14.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.317+0000 7f135560f640 1 -- 172.21.15.94:0/1991413335 >> 172.21.15.94:0/1991413335 conn(0x7f13500761d0 msgr2=0x7f1350100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.318+0000 7f135560f640 1 -- 172.21.15.94:0/1991413335 shutdown_connections 2024-02-17T12:01:14.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.319+0000 7f135560f640 1 -- 172.21.15.94:0/1991413335 wait complete. 2024-02-17T12:01:14.333 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:01:14.334 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:01:14.408 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:14 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.f.service: Scheduled restart job, restart counter is at 1. 2024-02-17T12:01:14.408 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:14 smithi189 systemd[1]: Stopped Ceph mds.f for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T12:01:14.408 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:14 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.f.service: Consumed 1.685s CPU time. 2024-02-17T12:01:14.408 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:14 smithi189 systemd[1]: Starting Ceph mds.f for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:01:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.539+0000 7ff747a0e640 1 -- 172.21.15.94:0/1432327221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7401002a0 msgr2=0x7ff740105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.539+0000 7ff747a0e640 1 --2- 172.21.15.94:0/1432327221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7401002a0 0x7ff740105dd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff730009900 tx=0x7ff73002bc30 comp rx=0 tx=0).stop 2024-02-17T12:01:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.539+0000 7ff747a0e640 1 -- 172.21.15.94:0/1432327221 shutdown_connections 2024-02-17T12:01:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.539+0000 7ff747a0e640 1 --2- 172.21.15.94:0/1432327221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff740106500 0x7ff74010e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.539+0000 7ff747a0e640 1 --2- 172.21.15.94:0/1432327221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff7401002a0 0x7ff740105dd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.539+0000 7ff747a0e640 1 --2- 172.21.15.94:0/1432327221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7400ff8f0 0x7ff7400ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.539+0000 7ff747a0e640 1 -- 172.21.15.94:0/1432327221 >> 172.21.15.94:0/1432327221 conn(0x7ff7400765a0 msgr2=0x7ff7400fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.540+0000 7ff747a0e640 1 -- 172.21.15.94:0/1432327221 shutdown_connections 2024-02-17T12:01:14.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.540+0000 7ff747a0e640 1 -- 172.21.15.94:0/1432327221 wait complete. 2024-02-17T12:01:14.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.541+0000 7ff747a0e640 1 Processor -- start 2024-02-17T12:01:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.541+0000 7ff747a0e640 1 -- start start 2024-02-17T12:01:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.541+0000 7ff747a0e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7400ff8f0 0x7ff740197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.541+0000 7ff747a0e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff740106500 0x7ff740197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.541+0000 7ff747a0e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74019c010 0x7ff74019c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.541+0000 7ff747a0e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74019ca00 con 0x7ff7400ff8f0 2024-02-17T12:01:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.541+0000 7ff747a0e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74019cb70 con 0x7ff740106500 2024-02-17T12:01:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.541+0000 7ff747a0e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74019cce0 con 0x7ff74019c010 2024-02-17T12:01:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff7457aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7400ff8f0 0x7ff740197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff745fab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74019c010 0x7ff74019c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff744fa9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff740106500 0x7ff740197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff744fa9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff740106500 0x7ff740197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41686/0 (socket says 172.21.15.94:41686) 2024-02-17T12:01:14.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff745fab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74019c010 0x7ff74019c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60386/0 (socket says 172.21.15.94:60386) 2024-02-17T12:01:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff744fa9640 1 -- 172.21.15.94:0/97937280 learned_addr learned my addr 172.21.15.94:0/97937280 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff744fa9640 1 -- 172.21.15.94:0/97937280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74019c010 msgr2=0x7ff74019c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff744fa9640 1 --2- 172.21.15.94:0/97937280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74019c010 0x7ff74019c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff744fa9640 1 -- 172.21.15.94:0/97937280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7400ff8f0 msgr2=0x7ff740197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff744fa9640 1 --2- 172.21.15.94:0/97937280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7400ff8f0 0x7ff740197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff744fa9640 1 -- 172.21.15.94:0/97937280 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c009490 con 0x7ff740106500 2024-02-17T12:01:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff745fab640 1 --2- 172.21.15.94:0/97937280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74019c010 0x7ff74019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff744fa9640 1 --2- 172.21.15.94:0/97937280 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff740106500 0x7ff740197d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff730000c00 tx=0x7ff73002cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff7367fc640 1 -- 172.21.15.94:0/97937280 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730004350 con 0x7ff740106500 2024-02-17T12:01:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff7367fc640 1 -- 172.21.15.94:0/97937280 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7300044b0 con 0x7ff740106500 2024-02-17T12:01:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.542+0000 7ff7367fc640 1 -- 172.21.15.94:0/97937280 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730003d00 con 0x7ff740106500 2024-02-17T12:01:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.543+0000 7ff747a0e640 1 -- 172.21.15.94:0/97937280 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff730009560 con 0x7ff740106500 2024-02-17T12:01:14.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.543+0000 7ff747a0e640 1 -- 172.21.15.94:0/97937280 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff740100a30 con 0x7ff740106500 2024-02-17T12:01:14.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.545+0000 7ff7367fc640 1 -- 172.21.15.94:0/97937280 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff73003e450 con 0x7ff740106500 2024-02-17T12:01:14.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.546+0000 7ff74a5e7740 1 Processor -- start 2024-02-17T12:01:14.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.547+0000 7ff7367fc640 1 --2- 172.21.15.94:0/97937280 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff72c0a6640 0x7ff72c0a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.547+0000 7ff7367fc640 1 -- 172.21.15.94:0/97937280 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff7300f0640 con 0x7ff740106500 2024-02-17T12:01:14.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.547+0000 7ff74a5e7740 1 -- start start 2024-02-17T12:01:14.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.547+0000 7ff745fab640 1 --2- 172.21.15.94:0/97937280 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff72c0a6640 0x7ff72c0a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.547+0000 7ff74a5e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f441250 0x564a7f441630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.547+0000 7ff74a5e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f44a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.547+0000 7ff74a5e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f44a750 0x564a7f44cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.547+0000 7ff74a5e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f3fc960 con 0x564a7f441250 2024-02-17T12:01:14.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.547+0000 7ff74a5e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f3b0310 con 0x564a7f44a750 2024-02-17T12:01:14.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.547+0000 7ff74a5e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f3ad840 con 0x564a7f441b70 2024-02-17T12:01:14.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff7457aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f441250 0x564a7f441630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff745fab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f44a750 0x564a7f44cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff7457aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f441250 0x564a7f441630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55022/0 (socket says 172.21.15.94:55022) 2024-02-17T12:01:14.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff745fab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f44a750 0x564a7f44cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41702/0 (socket says 172.21.15.94:41702) 2024-02-17T12:01:14.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff7457aa640 1 -- 172.21.15.94:0/1853270285 learned_addr learned my addr 172.21.15.94:0/1853270285 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff744fa9640 1 --2- 172.21.15.94:0/1853270285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f44a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff744fa9640 1 -- 172.21.15.94:0/1853270285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f44a750 msgr2=0x564a7f44cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff744fa9640 1 --2- 172.21.15.94:0/1853270285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f44a750 0x564a7f44cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff744fa9640 1 -- 172.21.15.94:0/1853270285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f441250 msgr2=0x564a7f441630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff744fa9640 1 --2- 172.21.15.94:0/1853270285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f441250 0x564a7f441630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff744fa9640 1 -- 172.21.15.94:0/1853270285 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a7f36b060 con 0x564a7f441b70 2024-02-17T12:01:14.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff745fab640 1 --2- 172.21.15.94:0/1853270285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f44a750 0x564a7f44cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:14.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff745fab640 1 --2- 172.21.15.94:0/97937280 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff72c0a6640 0x7ff72c0a8b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff738005fd0 tx=0x7ff738012630 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff744fa9640 1 --2- 172.21.15.94:0/1853270285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f44a210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff7300ede90 tx=0x7ff7300bcbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e08d96dfdd25b393 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.548+0000 7ff72b7fe640 1 -- 172.21.15.94:0/1853270285 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7300c1120 con 0x564a7f441b70 2024-02-17T12:01:14.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.549+0000 7ff72b7fe640 1 -- 172.21.15.94:0/1853270285 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7300035f0 con 0x564a7f441b70 2024-02-17T12:01:14.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.549+0000 7ff72b7fe640 1 -- 172.21.15.94:0/1853270285 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7300ee180 con 0x564a7f441b70 2024-02-17T12:01:14.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.549+0000 7ff74a5e7740 1 -- 172.21.15.94:0/1853270285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 msgr2=0x564a7f44a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.549+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/1853270285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f44a210 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff7300ede90 tx=0x7ff7300bcbe0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.549+0000 7ff74a5e7740 1 -- 172.21.15.94:0/1853270285 shutdown_connections 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.549+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/1853270285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f44a750 0x564a7f44cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.549+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/1853270285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f44a210 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.549+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/1853270285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f441250 0x564a7f441630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.549+0000 7ff74a5e7740 1 -- 172.21.15.94:0/1853270285 >> 172.21.15.94:0/1853270285 conn(0x564a7f43b980 msgr2=0x564a7f43bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.549+0000 7ff74a5e7740 1 -- 172.21.15.94:0/1853270285 shutdown_connections 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.549+0000 7ff74a5e7740 1 -- 172.21.15.94:0/1853270285 wait complete. 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.550+0000 7ff74a5e7740 1 -- start start 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.550+0000 7ff74a5e7740 1 Processor -- start 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.550+0000 7ff74a5e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441250 0x564a7f447780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.550+0000 7ff74a5e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f441b70 0x564a7f447cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.550+0000 7ff74a5e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f44a750 0x564a7f448200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.550+0000 7ff74a5e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f3a9020 con 0x564a7f441b70 2024-02-17T12:01:14.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.550+0000 7ff74a5e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f07c430 con 0x564a7f44a750 2024-02-17T12:01:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.550+0000 7ff74a5e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f038e00 con 0x564a7f441250 2024-02-17T12:01:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.551+0000 7ff74a5e7740 10 client.0 _ll_register_callbacks cb 0x564a7f350cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.551+0000 7ff744fa9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f441b70 0x564a7f447cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.551+0000 7ff744fa9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f441b70 0x564a7f447cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55026/0 (socket says 172.21.15.94:55026) 2024-02-17T12:01:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.551+0000 7ff744fa9640 1 -- 172.21.15.94:0/1074273353 learned_addr learned my addr 172.21.15.94:0/1074273353 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.551+0000 7ff7457aa640 1 --2- 172.21.15.94:0/1074273353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441250 0x564a7f447780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.551+0000 7ff744fa9640 1 -- 172.21.15.94:0/1074273353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441250 msgr2=0x564a7f447780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.551+0000 7ff744fa9640 1 --2- 172.21.15.94:0/1074273353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441250 0x564a7f447780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.551+0000 7ff744fa9640 1 -- 172.21.15.94:0/1074273353 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f44a750 msgr2=0x564a7f448200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:01:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.551+0000 7ff744fa9640 1 --2- 172.21.15.94:0/1074273353 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f44a750 0x564a7f448200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.551+0000 7ff744fa9640 1 -- 172.21.15.94:0/1074273353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff740100a30 con 0x564a7f441b70 2024-02-17T12:01:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.552+0000 7ff744fa9640 1 --2- 172.21.15.94:0/1074273353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f441b70 0x564a7f447cc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff7300bd150 tx=0x7ff7300bd1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a5e45719cfe36d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.552+0000 7ff728ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.552+0000 7ff728ff9640 1 -- 172.21.15.94:0/1074273353 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7300c1120 con 0x564a7f441b70 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.552+0000 7ff74a5e7740 1 -- 172.21.15.94:0/1074273353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a7f0a87e0 con 0x564a7f441b70 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.552+0000 7ff728ff9640 1 -- 172.21.15.94:0/1074273353 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff7300bd8c0 con 0x564a7f441b70 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.553+0000 7ff728ff9640 1 -- 172.21.15.94:0/1074273353 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7300c58d0 con 0x564a7f441b70 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.553+0000 7ff74a5e7740 10 client.16335 fetch_fsmap learned FSMap version 21 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.553+0000 7ff74a5e7740 1 -- 172.21.15.94:0/1074273353 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x564a7f3a76a0 con 0x564a7f441b70 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.553+0000 7ff728ff9640 1 -- 172.21.15.94:0/1074273353 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564a7f0a87e0 con 0x564a7f441b70 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.553+0000 7ff728ff9640 1 -- 172.21.15.94:0/1074273353 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff73003c070 con 0x564a7f441b70 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.553+0000 7ff74a5e7740 10 client.16335 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.553+0000 7ff74a5e7740 10 client.16335 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.553+0000 7ff74a5e7740 20 client.16335 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.553+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/1074273353 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a7f451580 0x564a7f4539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.553+0000 7ff74a5e7740 4 client.16335 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:14.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.553+0000 7ff74a5e7740 1 -- 172.21.15.94:0/1074273353 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a7f0e5720 con 0x564a7f451580 2024-02-17T12:01:14.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.554+0000 7ff7457aa640 1 --2- 172.21.15.94:0/1074273353 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a7f451580 0x564a7f4539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.554+0000 7ff728ff9640 10 client.16335 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:01:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.555+0000 7ff7457aa640 1 --2- 172.21.15.94:0/1074273353 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a7f451580 0x564a7f4539a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=63086214ea9e8540 server_cookie=b2f86d9c5be98982 in_seq=0 out_seq=0 2024-02-17T12:01:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.555+0000 7ff728ff9640 1 -- 172.21.15.94:0/1074273353 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a7f0e5720 con 0x564a7f451580 2024-02-17T12:01:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.555+0000 7ff728ff9640 10 client.16335 handle_command_reply: tid=0 2024-02-17T12:01:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.555+0000 7ff74a5e7740 1 client.16335 shutdown 2024-02-17T12:01:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.555+0000 7ff74a5e7740 1 -- 172.21.15.94:0/1074273353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f441b70 msgr2=0x564a7f447cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.555+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/1074273353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f441b70 0x564a7f447cc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff7300bd150 tx=0x7ff7300bd1c0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.555+0000 7ff74a5e7740 1 -- 172.21.15.94:0/1074273353 shutdown_connections 2024-02-17T12:01:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.555+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/1074273353 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a7f451580 0x564a7f4539a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.556+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/1074273353 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f44a750 0x564a7f448200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.556+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/1074273353 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f441b70 0x564a7f447cc0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.556+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/1074273353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441250 0x564a7f447780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.556+0000 7ff74a5e7740 1 -- 172.21.15.94:0/1074273353 >> 172.21.15.94:0/1074273353 conn(0x564a7f43b980 msgr2=0x564a7f43d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.556+0000 7ff728ff9640 0 client.16335 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:01:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.556+0000 7ff74a5e7740 1 -- 172.21.15.94:0/1074273353 shutdown_connections 2024-02-17T12:01:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.556+0000 7ff74a5e7740 1 -- 172.21.15.94:0/1074273353 wait complete. 2024-02-17T12:01:14.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.556+0000 7ff74a5e7740 20 client.16335 trim_cache size 0 max 16384 2024-02-17T12:01:14.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.573+0000 7ff74a5e7740 1 Processor -- start 2024-02-17T12:01:14.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.573+0000 7ff74a5e7740 1 -- start start 2024-02-17T12:01:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.573+0000 7ff74a5e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f441250 0x564a7f473ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.573+0000 7ff74a5e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f458300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.573+0000 7ff74a5e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 0x564a7f458840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.573+0000 7ff74a5e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f32e130 con 0x564a7f44a750 2024-02-17T12:01:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.573+0000 7ff74a5e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f44d110 con 0x564a7f441250 2024-02-17T12:01:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.573+0000 7ff74a5e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f43c9b0 con 0x564a7f441b70 2024-02-17T12:01:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff745fab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 0x564a7f458840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff745fab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 0x564a7f458840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55036/0 (socket says 172.21.15.94:55036) 2024-02-17T12:01:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff745fab640 1 -- 172.21.15.94:0/2665192680 learned_addr learned my addr 172.21.15.94:0/2665192680 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff7457aa640 1 --2- 172.21.15.94:0/2665192680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f441250 0x564a7f473ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff744fa9640 1 --2- 172.21.15.94:0/2665192680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f458300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff745fab640 1 -- 172.21.15.94:0/2665192680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 msgr2=0x564a7f458300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff745fab640 1 --2- 172.21.15.94:0/2665192680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f458300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff745fab640 1 -- 172.21.15.94:0/2665192680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f441250 msgr2=0x564a7f473ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff745fab640 1 --2- 172.21.15.94:0/2665192680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f441250 0x564a7f473ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff745fab640 1 -- 172.21.15.94:0/2665192680 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a7f343c20 con 0x564a7f44a750 2024-02-17T12:01:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff745fab640 1 --2- 172.21.15.94:0/2665192680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 0x564a7f458840 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff738012bf0 tx=0x7ff7380096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a57f62400a02f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff72affd640 1 -- 172.21.15.94:0/2665192680 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73801edb0 con 0x564a7f44a750 2024-02-17T12:01:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff72affd640 1 -- 172.21.15.94:0/2665192680 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff738003e50 con 0x564a7f44a750 2024-02-17T12:01:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.574+0000 7ff72affd640 1 -- 172.21.15.94:0/2665192680 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff738004810 con 0x564a7f44a750 2024-02-17T12:01:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.575+0000 7ff74a5e7740 1 -- 172.21.15.94:0/2665192680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 msgr2=0x564a7f458840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.575+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/2665192680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 0x564a7f458840 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7ff738012bf0 tx=0x7ff7380096a0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.575+0000 7ff74a5e7740 1 -- 172.21.15.94:0/2665192680 shutdown_connections 2024-02-17T12:01:14.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.575+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/2665192680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 0x564a7f458840 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.575+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/2665192680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f458300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.575+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/2665192680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f441250 0x564a7f473ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.575+0000 7ff74a5e7740 1 -- 172.21.15.94:0/2665192680 >> 172.21.15.94:0/2665192680 conn(0x564a7f43b980 msgr2=0x564a7f436a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.575+0000 7ff74a5e7740 1 -- 172.21.15.94:0/2665192680 shutdown_connections 2024-02-17T12:01:14.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.575+0000 7ff74a5e7740 1 -- 172.21.15.94:0/2665192680 wait complete. 2024-02-17T12:01:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.575+0000 7ff74a5e7740 1 -- start start 2024-02-17T12:01:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.576+0000 7ff74a5e7740 1 Processor -- start 2024-02-17T12:01:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.576+0000 7ff74a5e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f441250 0x564a7f457f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.576+0000 7ff74a5e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f458450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.576+0000 7ff74a5e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 0x564a7f4553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.576+0000 7ff74a5e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f455a00 con 0x564a7f44a750 2024-02-17T12:01:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.576+0000 7ff74a5e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f455b70 con 0x564a7f441250 2024-02-17T12:01:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.576+0000 7ff74a5e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7f455d30 con 0x564a7f441b70 2024-02-17T12:01:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.576+0000 7ff74a5e7740 10 client.0 _ll_register_callbacks cb 0x7ff730005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:14.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.576+0000 7ff745fab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 0x564a7f4553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.576+0000 7ff745fab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 0x564a7f4553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55048/0 (socket says 172.21.15.94:55048) 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.576+0000 7ff745fab640 1 -- 172.21.15.94:0/2516922663 learned_addr learned my addr 172.21.15.94:0/2516922663 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.576+0000 7ff744fa9640 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f458450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.576+0000 7ff7457aa640 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f441250 0x564a7f457f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff744fa9640 1 -- 172.21.15.94:0/2516922663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f441250 msgr2=0x564a7f457f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff744fa9640 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f441250 0x564a7f457f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff744fa9640 1 -- 172.21.15.94:0/2516922663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 msgr2=0x564a7f4553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff744fa9640 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 0x564a7f4553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff744fa9640 1 -- 172.21.15.94:0/2516922663 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff738012980 con 0x564a7f441b70 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff745fab640 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 0x564a7f4553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff7457aa640 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f441250 0x564a7f457f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff744fa9640 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f458450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x564a7f448d50 tx=0x7ff7300092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a84a6c24badca9dd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff728ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:01:14.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff728ff9640 1 -- 172.21.15.94:0/2516922663 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7300bcd00 con 0x564a7f441b70 2024-02-17T12:01:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff74a5e7740 1 -- 172.21.15.94:0/2516922663 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7300035f0 con 0x564a7f441b70 2024-02-17T12:01:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff728ff9640 1 -- 172.21.15.94:0/2516922663 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff7300bd8c0 con 0x564a7f441b70 2024-02-17T12:01:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff728ff9640 1 -- 172.21.15.94:0/2516922663 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7300b9460 con 0x564a7f441b70 2024-02-17T12:01:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff74a5e7740 10 client.25825 fetch_fsmap learned FSMap version 21 2024-02-17T12:01:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff74a5e7740 1 -- 172.21.15.94:0/2516922663 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x564a7f3698c0 con 0x564a7f441b70 2024-02-17T12:01:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff728ff9640 1 -- 172.21.15.94:0/2516922663 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7300035f0 con 0x564a7f441b70 2024-02-17T12:01:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff728ff9640 1 -- 172.21.15.94:0/2516922663 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff73003c040 con 0x564a7f441b70 2024-02-17T12:01:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff74a5e7740 10 client.25825 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:01:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff74a5e7740 10 client.25825 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:01:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.577+0000 7ff74a5e7740 20 client.25825 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a7f4c3040 0x564a7f4c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff74a5e7740 4 client.25825 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:01:14.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff74a5e7740 1 -- 172.21.15.94:0/2516922663 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564a7f134890 con 0x564a7f4c3040 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff7457aa640 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a7f4c3040 0x564a7f4c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff728ff9640 10 client.25825 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff7457aa640 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a7f4c3040 0x564a7f4c5430 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4ead67657a66bf server_cookie=3333661e9a1dd822 in_seq=0 out_seq=0 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff728ff9640 1 -- 172.21.15.94:0/2516922663 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564a7f134890 con 0x564a7f4c3040 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff728ff9640 10 client.25825 handle_command_reply: tid=0 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff74a5e7740 1 client.25825 shutdown 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff74a5e7740 1 -- 172.21.15.94:0/2516922663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 msgr2=0x564a7f458450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f458450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x564a7f448d50 tx=0x7ff7300092d0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff74a5e7740 1 -- 172.21.15.94:0/2516922663 shutdown_connections 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a7f4c3040 0x564a7f4c5430 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff728ff9640 0 client.25825 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a7f44a750 0x564a7f4553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a7f441b70 0x564a7f458450 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff74a5e7740 1 --2- 172.21.15.94:0/2516922663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a7f441250 0x564a7f457f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.578+0000 7ff74a5e7740 1 -- 172.21.15.94:0/2516922663 >> 172.21.15.94:0/2516922663 conn(0x564a7f43b980 msgr2=0x564a7f473ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff74a5e7740 1 -- 172.21.15.94:0/2516922663 shutdown_connections 2024-02-17T12:01:14.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff74a5e7740 1 -- 172.21.15.94:0/2516922663 wait complete. 2024-02-17T12:01:14.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff74a5e7740 20 client.25825 trim_cache size 0 max 16384 2024-02-17T12:01:14.587 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:01:14.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff747a0e640 1 -- 172.21.15.94:0/97937280 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff72c0a6640 msgr2=0x7ff72c0a8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff747a0e640 1 --2- 172.21.15.94:0/97937280 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff72c0a6640 0x7ff72c0a8b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff738005fd0 tx=0x7ff738012630 comp rx=0 tx=0).stop 2024-02-17T12:01:14.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff747a0e640 1 -- 172.21.15.94:0/97937280 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff740106500 msgr2=0x7ff740197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff747a0e640 1 --2- 172.21.15.94:0/97937280 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff740106500 0x7ff740197d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff730000c00 tx=0x7ff73002cd80 comp rx=0 tx=0).stop 2024-02-17T12:01:14.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff747a0e640 1 -- 172.21.15.94:0/97937280 shutdown_connections 2024-02-17T12:01:14.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff747a0e640 1 --2- 172.21.15.94:0/97937280 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff72c0a6640 0x7ff72c0a8b00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff747a0e640 1 --2- 172.21.15.94:0/97937280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74019c010 0x7ff74019c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff747a0e640 1 --2- 172.21.15.94:0/97937280 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff740106500 0x7ff740197d80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff747a0e640 1 --2- 172.21.15.94:0/97937280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff7400ff8f0 0x7ff740197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff747a0e640 1 -- 172.21.15.94:0/97937280 >> 172.21.15.94:0/97937280 conn(0x7ff7400765a0 msgr2=0x7ff740104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff747a0e640 1 -- 172.21.15.94:0/97937280 shutdown_connections 2024-02-17T12:01:14.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.579+0000 7ff747a0e640 1 -- 172.21.15.94:0/97937280 wait complete. 2024-02-17T12:01:14.589 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:01:14.589 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:01:14.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.724+0000 7fd6b352c640 1 -- 172.21.15.94:0/1680152665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6ac101d40 msgr2=0x7fd6ac105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.724+0000 7fd6b352c640 1 --2- 172.21.15.94:0/1680152665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6ac101d40 0x7fd6ac105d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009900 tx=0x7fd69c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:01:14.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.725+0000 7fd6b352c640 1 -- 172.21.15.94:0/1680152665 shutdown_connections 2024-02-17T12:01:14.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.725+0000 7fd6b352c640 1 --2- 172.21.15.94:0/1680152665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ac106460 0x7fd6ac10e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.725+0000 7fd6b352c640 1 --2- 172.21.15.94:0/1680152665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6ac101d40 0x7fd6ac105d80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.725+0000 7fd6b352c640 1 --2- 172.21.15.94:0/1680152665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6ac101390 0x7fd6ac101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.725+0000 7fd6b352c640 1 -- 172.21.15.94:0/1680152665 >> 172.21.15.94:0/1680152665 conn(0x7fd6ac0fcbb0 msgr2=0x7fd6ac0fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.725+0000 7fd6b352c640 1 -- 172.21.15.94:0/1680152665 shutdown_connections 2024-02-17T12:01:14.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.725+0000 7fd6b352c640 1 -- 172.21.15.94:0/1680152665 wait complete. 2024-02-17T12:01:14.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.726+0000 7fd6b352c640 1 Processor -- start 2024-02-17T12:01:14.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.726+0000 7fd6b352c640 1 -- start start 2024-02-17T12:01:14.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.726+0000 7fd6b352c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ac101390 0x7fd6ac19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.726+0000 7fd6b352c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6ac106460 0x7fd6ac19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.726+0000 7fd6b352c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6ac1997b0 0x7fd6ac199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b352c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac19a280 con 0x7fd6ac106460 2024-02-17T12:01:14.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b352c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac19a3f0 con 0x7fd6ac101390 2024-02-17T12:01:14.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b352c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac19a5b0 con 0x7fd6ac1997b0 2024-02-17T12:01:14.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b12c8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ac101390 0x7fd6ac19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b12c8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ac101390 0x7fd6ac19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41756/0 (socket says 172.21.15.94:41756) 2024-02-17T12:01:14.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b12c8640 1 -- 172.21.15.94:0/3188866066 learned_addr learned my addr 172.21.15.94:0/3188866066 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b0ac7640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6ac106460 0x7fd6ac19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b1ac9640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6ac1997b0 0x7fd6ac199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b12c8640 1 -- 172.21.15.94:0/3188866066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6ac1997b0 msgr2=0x7fd6ac199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b12c8640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6ac1997b0 0x7fd6ac199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b12c8640 1 -- 172.21.15.94:0/3188866066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6ac106460 msgr2=0x7fd6ac19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b12c8640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6ac106460 0x7fd6ac19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b12c8640 1 -- 172.21.15.94:0/3188866066 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c009560 con 0x7fd6ac101390 2024-02-17T12:01:14.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b1ac9640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6ac1997b0 0x7fd6ac199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:01:14.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b12c8640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ac101390 0x7fd6ac19f6a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800c920 tx=0x7fd6a800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6a27fc640 1 -- 172.21.15.94:0/3188866066 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8007c70 con 0x7fd6ac101390 2024-02-17T12:01:14.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6a27fc640 1 -- 172.21.15.94:0/3188866066 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a8007dd0 con 0x7fd6ac101390 2024-02-17T12:01:14.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b352c640 1 -- 172.21.15.94:0/3188866066 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac1a6500 con 0x7fd6ac101390 2024-02-17T12:01:14.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6a27fc640 1 -- 172.21.15.94:0/3188866066 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8018860 con 0x7fd6ac101390 2024-02-17T12:01:14.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.727+0000 7fd6b352c640 1 -- 172.21.15.94:0/3188866066 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac1a69e0 con 0x7fd6ac101390 2024-02-17T12:01:14.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.731+0000 7fd6a27fc640 1 -- 172.21.15.94:0/3188866066 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd6a800fac0 con 0x7fd6ac101390 2024-02-17T12:01:14.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.731+0000 7fd6b61dd740 1 Processor -- start 2024-02-17T12:01:14.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.731+0000 7fd6b61dd740 1 -- start start 2024-02-17T12:01:14.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.731+0000 7fd6b61dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842dda60 0x564a842dde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.731+0000 7fd6b61dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.731+0000 7fd6b61dd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842e6f60 0x564a842e9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.731+0000 7fd6b61dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a84290960 con 0x564a842de380 2024-02-17T12:01:14.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.731+0000 7fd6b61dd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a84244310 con 0x564a842e6f60 2024-02-17T12:01:14.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.731+0000 7fd6b61dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a84241840 con 0x564a842dda60 2024-02-17T12:01:14.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6b12c8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6b1ac9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842dda60 0x564a842dde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6b0ac7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842e6f60 0x564a842e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6b1ac9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842dda60 0x564a842dde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60450/0 (socket says 172.21.15.94:60450) 2024-02-17T12:01:14.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6b1ac9640 1 -- 172.21.15.94:0/934987638 learned_addr learned my addr 172.21.15.94:0/934987638 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6a27fc640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd6980a22b0 0x7fd6980a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6b12c8640 1 -- 172.21.15.94:0/934987638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842dda60 msgr2=0x564a842dde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6b12c8640 1 --2- 172.21.15.94:0/934987638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842dda60 0x564a842dde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6b12c8640 1 -- 172.21.15.94:0/934987638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842e6f60 msgr2=0x564a842e9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6b12c8640 1 --2- 172.21.15.94:0/934987638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842e6f60 0x564a842e9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6a27fc640 1 -- 172.21.15.94:0/3188866066 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd6a80ce290 con 0x7fd6ac101390 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6b1ac9640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd6980a22b0 0x7fd6980a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6b12c8640 1 -- 172.21.15.94:0/934987638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a841ff060 con 0x564a842de380 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.732+0000 7fd6b12c8640 1 --2- 172.21.15.94:0/934987638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842e6a20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80cb970 tx=0x7fd6a800fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9c2d79046bda17 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.733+0000 7fd6b0ac7640 1 --2- 172.21.15.94:0/934987638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842e6f60 0x564a842e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.733+0000 7fd6977fe640 1 -- 172.21.15.94:0/934987638 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a809a750 con 0x564a842de380 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.733+0000 7fd6b1ac9640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd6980a22b0 0x7fd6980a4770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009fd0 tx=0x7fd6a4014040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.733+0000 7fd6977fe640 1 -- 172.21.15.94:0/934987638 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a809a570 con 0x564a842de380 2024-02-17T12:01:14.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.733+0000 7fd6977fe640 1 -- 172.21.15.94:0/934987638 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a80a33b0 con 0x564a842de380 2024-02-17T12:01:14.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.735+0000 7fd6b61dd740 1 -- 172.21.15.94:0/934987638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 msgr2=0x564a842e6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.735+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/934987638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842e6a20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd6a80cb970 tx=0x7fd6a800fec0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.735+0000 7fd6b61dd740 1 -- 172.21.15.94:0/934987638 shutdown_connections 2024-02-17T12:01:14.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.735+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/934987638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842e6f60 0x564a842e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.735+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/934987638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842e6a20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.735+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/934987638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842dda60 0x564a842dde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.735+0000 7fd6b61dd740 1 -- 172.21.15.94:0/934987638 >> 172.21.15.94:0/934987638 conn(0x564a842d8190 msgr2=0x564a842d8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.735+0000 7fd6b61dd740 1 -- 172.21.15.94:0/934987638 shutdown_connections 2024-02-17T12:01:14.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.735+0000 7fd6b61dd740 1 -- 172.21.15.94:0/934987638 wait complete. 2024-02-17T12:01:14.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.735+0000 7fd6b61dd740 1 -- start start 2024-02-17T12:01:14.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.736+0000 7fd6b61dd740 1 Processor -- start 2024-02-17T12:01:14.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.736+0000 7fd6b61dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842dda60 0x564a842e3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.736+0000 7fd6b61dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842e44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.736+0000 7fd6b61dd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842e6f60 0x564a842e4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.736+0000 7fd6b61dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8423d020 con 0x564a842de380 2024-02-17T12:01:14.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.736+0000 7fd6b61dd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a83f10430 con 0x564a842e6f60 2024-02-17T12:01:14.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.736+0000 7fd6b61dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a83ecce00 con 0x564a842dda60 2024-02-17T12:01:14.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.736+0000 7fd6b1ac9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842e6f60 0x564a842e4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.736+0000 7fd6b0ac7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842e44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.736+0000 7fd6b1ac9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842e6f60 0x564a842e4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41766/0 (socket says 172.21.15.94:41766) 2024-02-17T12:01:14.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.736+0000 7fd6b1ac9640 1 -- 172.21.15.94:0/2416527412 learned_addr learned my addr 172.21.15.94:0/2416527412 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.736+0000 7fd6b61dd740 10 client.0 _ll_register_callbacks cb 0x564a841e4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:14.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.737+0000 7fd6b12c8640 1 --2- 172.21.15.94:0/2416527412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842dda60 0x564a842e3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.737+0000 7fd6b1ac9640 1 -- 172.21.15.94:0/2416527412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842dda60 msgr2=0x564a842e3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.737+0000 7fd6b1ac9640 1 --2- 172.21.15.94:0/2416527412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842dda60 0x564a842e3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.737+0000 7fd6b1ac9640 1 -- 172.21.15.94:0/2416527412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 msgr2=0x564a842e44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.737+0000 7fd6b1ac9640 1 --2- 172.21.15.94:0/2416527412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842e44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.737+0000 7fd6b1ac9640 1 -- 172.21.15.94:0/2416527412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac1a69e0 con 0x564a842e6f60 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.737+0000 7fd6b1ac9640 1 --2- 172.21.15.94:0/2416527412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842e6f60 0x564a842e4a00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40119e0 tx=0x7fd6a4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e395d92977a36b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.737+0000 7fd694ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.737+0000 7fd694ff9640 1 -- 172.21.15.94:0/2416527412 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a401be80 con 0x564a842e6f60 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.737+0000 7fd6b61dd740 1 -- 172.21.15.94:0/2416527412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a83f3c7e0 con 0x564a842e6f60 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.737+0000 7fd694ff9640 1 -- 172.21.15.94:0/2416527412 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd6a4026400 con 0x564a842e6f60 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.738+0000 7fd694ff9640 1 -- 172.21.15.94:0/2416527412 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6a40265c0 con 0x564a842e6f60 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.738+0000 7fd6b61dd740 10 client.25790 fetch_fsmap learned FSMap version 21 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.738+0000 7fd6b61dd740 1 -- 172.21.15.94:0/2416527412 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x564a8423b6a0 con 0x564a842e6f60 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.738+0000 7fd694ff9640 1 -- 172.21.15.94:0/2416527412 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a4033020 con 0x564a842e6f60 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.738+0000 7fd694ff9640 1 -- 172.21.15.94:0/2416527412 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd6a402e850 con 0x564a842e6f60 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.738+0000 7fd6b61dd740 10 client.25790 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:01:14.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.738+0000 7fd6b61dd740 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:14.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.738+0000 7fd6b61dd740 20 client.25790 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:14.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.738+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/2416527412 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a842f70f0 0x564a842f9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.738+0000 7fd6b61dd740 4 client.25790 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:14.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.738+0000 7fd6b61dd740 1 -- 172.21.15.94:0/2416527412 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a83f3c7e0 con 0x564a842f70f0 2024-02-17T12:01:14.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.739+0000 7fd6b12c8640 1 --2- 172.21.15.94:0/2416527412 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a842f70f0 0x564a842f9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.739+0000 7fd6b12c8640 1 --2- 172.21.15.94:0/2416527412 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a842f70f0 0x564a842f9550 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e4d8091ab5d351 server_cookie=307f11788aac6b60 in_seq=0 out_seq=0 2024-02-17T12:01:14.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.739+0000 7fd694ff9640 10 client.25790 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:14.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.740+0000 7fd694ff9640 1 -- 172.21.15.94:0/2416527412 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a83f3c7e0 con 0x564a842f70f0 2024-02-17T12:01:14.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.740+0000 7fd694ff9640 10 client.25790 handle_command_reply: tid=0 2024-02-17T12:01:14.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.740+0000 7fd6b61dd740 1 client.25790 shutdown 2024-02-17T12:01:14.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.740+0000 7fd6b61dd740 1 -- 172.21.15.94:0/2416527412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842e6f60 msgr2=0x564a842e4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.740+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/2416527412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842e6f60 0x564a842e4a00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd6a40119e0 tx=0x7fd6a4004a40 comp rx=0 tx=0).stop 2024-02-17T12:01:14.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.740+0000 7fd6b61dd740 1 -- 172.21.15.94:0/2416527412 shutdown_connections 2024-02-17T12:01:14.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.740+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/2416527412 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a842f70f0 0x564a842f9550 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.740+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/2416527412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842e6f60 0x564a842e4a00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.740+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/2416527412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842e44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.740+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/2416527412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842dda60 0x564a842e3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.740+0000 7fd6b61dd740 1 -- 172.21.15.94:0/2416527412 >> 172.21.15.94:0/2416527412 conn(0x564a842d8190 msgr2=0x564a842d9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.740+0000 7fd694ff9640 0 client.25790 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:14.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.741+0000 7fd6b61dd740 1 -- 172.21.15.94:0/2416527412 shutdown_connections 2024-02-17T12:01:14.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.741+0000 7fd6b61dd740 1 -- 172.21.15.94:0/2416527412 wait complete. 2024-02-17T12:01:14.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.741+0000 7fd6b61dd740 20 client.25790 trim_cache size 0 max 16384 2024-02-17T12:01:14.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.759+0000 7fd6b61dd740 1 Processor -- start 2024-02-17T12:01:14.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.759+0000 7fd6b61dd740 1 -- start start 2024-02-17T12:01:14.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.759+0000 7fd6b61dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842dda60 0x564a842e66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.759+0000 7fd6b61dd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842de380 0x564a842edd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b61dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842e6f60 0x564a842ee290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b0ac7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842de380 0x564a842edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b0ac7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842de380 0x564a842edd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41782/0 (socket says 172.21.15.94:41782) 2024-02-17T12:01:14.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b61dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a841c2130 con 0x564a842dda60 2024-02-17T12:01:14.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b61dd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a842d7cf0 con 0x564a842de380 2024-02-17T12:01:14.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b61dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a842d91c0 con 0x564a842e6f60 2024-02-17T12:01:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b12c8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842dda60 0x564a842e66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b12c8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842dda60 0x564a842e66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55098/0 (socket says 172.21.15.94:55098) 2024-02-17T12:01:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b12c8640 1 -- 172.21.15.94:0/2488751650 learned_addr learned my addr 172.21.15.94:0/2488751650 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b1ac9640 1 --2- 172.21.15.94:0/2488751650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842e6f60 0x564a842ee290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b1ac9640 1 -- 172.21.15.94:0/2488751650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842de380 msgr2=0x564a842edd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b1ac9640 1 --2- 172.21.15.94:0/2488751650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842de380 0x564a842edd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b1ac9640 1 -- 172.21.15.94:0/2488751650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842dda60 msgr2=0x564a842e66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.760+0000 7fd6b1ac9640 1 --2- 172.21.15.94:0/2488751650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842dda60 0x564a842e66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.761+0000 7fd6b1ac9640 1 -- 172.21.15.94:0/2488751650 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a842cb0d0 con 0x564a842e6f60 2024-02-17T12:01:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.761+0000 7fd6b1ac9640 1 --2- 172.21.15.94:0/2488751650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842e6f60 0x564a842ee290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd6a402ebe0 tx=0x7fd6a401b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3229980fc90273a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.761+0000 7fd696ffd640 1 -- 172.21.15.94:0/2488751650 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4012450 con 0x564a842e6f60 2024-02-17T12:01:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.761+0000 7fd696ffd640 1 -- 172.21.15.94:0/2488751650 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6a4033070 con 0x564a842e6f60 2024-02-17T12:01:14.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.761+0000 7fd696ffd640 1 -- 172.21.15.94:0/2488751650 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a400f7e0 con 0x564a842e6f60 2024-02-17T12:01:14.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.762+0000 7fd6b61dd740 1 -- 172.21.15.94:0/2488751650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842e6f60 msgr2=0x564a842ee290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.762+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/2488751650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842e6f60 0x564a842ee290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd6a402ebe0 tx=0x7fd6a401b8b0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.763+0000 7fd6b61dd740 1 -- 172.21.15.94:0/2488751650 shutdown_connections 2024-02-17T12:01:14.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.763+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/2488751650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842e6f60 0x564a842ee290 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.763+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/2488751650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842de380 0x564a842edd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.763+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/2488751650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842dda60 0x564a842e66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.763+0000 7fd6b61dd740 1 -- 172.21.15.94:0/2488751650 >> 172.21.15.94:0/2488751650 conn(0x564a842d8190 msgr2=0x564a842fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.763+0000 7fd6b61dd740 1 -- 172.21.15.94:0/2488751650 shutdown_connections 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.763+0000 7fd6b61dd740 1 -- 172.21.15.94:0/2488751650 wait complete. 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.763+0000 7fd6b61dd740 1 -- start start 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.763+0000 7fd6b61dd740 1 Processor -- start 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.764+0000 7fd6b61dd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842dda60 0x564a842ec950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.764+0000 7fd6b61dd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842ece90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.764+0000 7fd6b61dd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842e6f60 0x564a842ed3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.764+0000 7fd6b61dd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8434d720 con 0x564a842de380 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.764+0000 7fd6b61dd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a8434d890 con 0x564a842dda60 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.764+0000 7fd6b61dd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a842cf2d0 con 0x564a842e6f60 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.764+0000 7fd6b61dd740 10 client.0 _ll_register_callbacks cb 0x7fd6a80d1b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.765+0000 7fd6b0ac7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842ece90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.765+0000 7fd6b0ac7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842ece90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55112/0 (socket says 172.21.15.94:55112) 2024-02-17T12:01:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.765+0000 7fd6b0ac7640 1 -- 172.21.15.94:0/1730050078 learned_addr learned my addr 172.21.15.94:0/1730050078 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.765+0000 7fd6b1ac9640 1 --2- 172.21.15.94:0/1730050078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842e6f60 0x564a842ed3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.765+0000 7fd6b12c8640 1 --2- 172.21.15.94:0/1730050078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842dda60 0x564a842ec950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.765+0000 7fd6b0ac7640 1 -- 172.21.15.94:0/1730050078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842e6f60 msgr2=0x564a842ed3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.765+0000 7fd6b0ac7640 1 --2- 172.21.15.94:0/1730050078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842e6f60 0x564a842ed3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.765+0000 7fd6b0ac7640 1 -- 172.21.15.94:0/1730050078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842dda60 msgr2=0x564a842ec950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.765+0000 7fd6b0ac7640 1 --2- 172.21.15.94:0/1730050078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842dda60 0x564a842ec950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.765+0000 7fd6b0ac7640 1 -- 172.21.15.94:0/1730050078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a8423b6a0 con 0x564a842de380 2024-02-17T12:01:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.765+0000 7fd6b0ac7640 1 --2- 172.21.15.94:0/1730050078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842ece90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009900 tx=0x7fd69c004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f596bbbd1faba27 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.765+0000 7fd694ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:01:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.766+0000 7fd694ff9640 1 -- 172.21.15.94:0/1730050078 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c045070 con 0x564a842de380 2024-02-17T12:01:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.766+0000 7fd694ff9640 1 -- 172.21.15.94:0/1730050078 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd69c0042d0 con 0x564a842de380 2024-02-17T12:01:14.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.766+0000 7fd6b61dd740 1 -- 172.21.15.94:0/1730050078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6a809a570 con 0x564a842de380 2024-02-17T12:01:14.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.767+0000 7fd694ff9640 1 -- 172.21.15.94:0/1730050078 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd69c004490 con 0x564a842de380 2024-02-17T12:01:14.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.767+0000 7fd6b61dd740 10 client.16356 fetch_fsmap learned FSMap version 21 2024-02-17T12:01:14.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.767+0000 7fd6b61dd740 1 -- 172.21.15.94:0/1730050078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=4+}) v3 -- 0x564a841d7c20 con 0x564a842de380 2024-02-17T12:01:14.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.767+0000 7fd694ff9640 1 -- 172.21.15.94:0/1730050078 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd69c0052d0 con 0x564a842de380 2024-02-17T12:01:14.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.767+0000 7fd694ff9640 1 -- 172.21.15.94:0/1730050078 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 21) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd69c03c040 con 0x564a842de380 2024-02-17T12:01:14.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.767+0000 7fd6b61dd740 10 client.16356 fetch_fsmap finished waiting for FSMap version 21 2024-02-17T12:01:14.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.767+0000 7fd6b61dd740 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:14.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.767+0000 7fd6b61dd740 20 client.16356 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:14.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.767+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/1730050078 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a842d0c00 0x564a842d3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:14.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.767+0000 7fd6b61dd740 4 client.16356 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:01:14.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.767+0000 7fd6b61dd740 1 -- 172.21.15.94:0/1730050078 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fd6a809a570 con 0x564a842d0c00 2024-02-17T12:01:14.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.767+0000 7fd6b12c8640 1 --2- 172.21.15.94:0/1730050078 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a842d0c00 0x564a842d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:14.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd6b12c8640 1 --2- 172.21.15.94:0/1730050078 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a842d0c00 0x564a842d3020 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24677883f8950acc server_cookie=edf2d6398bd51f55 in_seq=0 out_seq=0 2024-02-17T12:01:14.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd694ff9640 10 client.16356 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:14.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd694ff9640 1 -- 172.21.15.94:0/1730050078 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fd6a809a570 con 0x564a842d0c00 2024-02-17T12:01:14.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd694ff9640 10 client.16356 handle_command_reply: tid=0 2024-02-17T12:01:14.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd6b61dd740 1 client.16356 shutdown 2024-02-17T12:01:14.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd6b61dd740 1 -- 172.21.15.94:0/1730050078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 msgr2=0x564a842ece90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/1730050078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842ece90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009900 tx=0x7fd69c004c00 comp rx=0 tx=0).stop 2024-02-17T12:01:14.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd6b61dd740 1 -- 172.21.15.94:0/1730050078 shutdown_connections 2024-02-17T12:01:14.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/1730050078 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564a842d0c00 0x564a842d3020 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/1730050078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a842e6f60 0x564a842ed3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/1730050078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a842de380 0x564a842ece90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd6b61dd740 1 --2- 172.21.15.94:0/1730050078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a842dda60 0x564a842ec950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd694ff9640 0 client.16356 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:14.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd6b61dd740 1 -- 172.21.15.94:0/1730050078 >> 172.21.15.94:0/1730050078 conn(0x564a842d8190 msgr2=0x564a842df2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.769+0000 7fd6b61dd740 1 -- 172.21.15.94:0/1730050078 shutdown_connections 2024-02-17T12:01:14.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.770+0000 7fd6b61dd740 1 -- 172.21.15.94:0/1730050078 wait complete. 2024-02-17T12:01:14.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.770+0000 7fd6b61dd740 20 client.16356 trim_cache size 0 max 16384 2024-02-17T12:01:14.773 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:01:14.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.771+0000 7fd6b352c640 1 -- 172.21.15.94:0/3188866066 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd6980a22b0 msgr2=0x7fd6980a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.771+0000 7fd6b352c640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd6980a22b0 0x7fd6980a4770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009fd0 tx=0x7fd6a4014040 comp rx=0 tx=0).stop 2024-02-17T12:01:14.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.771+0000 7fd6b352c640 1 -- 172.21.15.94:0/3188866066 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ac101390 msgr2=0x7fd6ac19f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:14.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.771+0000 7fd6b352c640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ac101390 0x7fd6ac19f6a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800c920 tx=0x7fd6a800cde0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.771+0000 7fd6b352c640 1 -- 172.21.15.94:0/3188866066 shutdown_connections 2024-02-17T12:01:14.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.771+0000 7fd6b352c640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd6980a22b0 0x7fd6980a4770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.771+0000 7fd6b352c640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6ac1997b0 0x7fd6ac199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.771+0000 7fd6b352c640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6ac106460 0x7fd6ac19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.771+0000 7fd6b352c640 1 --2- 172.21.15.94:0/3188866066 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6ac101390 0x7fd6ac19f6a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:14.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.771+0000 7fd6b352c640 1 -- 172.21.15.94:0/3188866066 >> 172.21.15.94:0/3188866066 conn(0x7fd6ac0fcbb0 msgr2=0x7fd6ac0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:14.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.771+0000 7fd6b352c640 1 -- 172.21.15.94:0/3188866066 shutdown_connections 2024-02-17T12:01:14.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:14.771+0000 7fd6b352c640 1 -- 172.21.15.94:0/3188866066 wait complete. 2024-02-17T12:01:14.870 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:14 smithi189 podman[71414]: 2024-02-17 12:01:14.312777832 +0000 UTC m=+0.023498744 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:01:14.870 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:14 smithi189 podman[71414]: 2024-02-17 12:01:14.436987802 +0000 UTC m=+0.147708721 container create 7a5d228de56b83698faddf5d61a8a4da8df53ad4d628d7250ca23898b8177af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f, org.label-schema.build-date=20240214, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-17T12:01:15.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:14 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2375235235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:01:15.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:14 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2375235235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:01:15.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:14 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2375235235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:01:15.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 175.0 s, 3035.0 tps, lat 32.832 ms stddev 36.617 2024-02-17T12:01:15.489 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:15 smithi189 podman[71414]: 2024-02-17 12:01:15.197673307 +0000 UTC m=+0.908394217 container init 7a5d228de56b83698faddf5d61a8a4da8df53ad4d628d7250ca23898b8177af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-17T12:01:15.490 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:15 smithi189 podman[71414]: 2024-02-17 12:01:15.200491932 +0000 UTC m=+0.911212845 container start 7a5d228de56b83698faddf5d61a8a4da8df53ad4d628d7250ca23898b8177af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-02-17T12:01:15.490 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:15 smithi189 ceph-mds[71434]: -- 172.21.15.189:0/987275378 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557bbe1c01a0 con 0x557bbe196800 2024-02-17T12:01:15.490 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:15 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[71429]: starting mds.f at 2024-02-17T12:01:15.490 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:15 smithi189 bash[71414]: 7a5d228de56b83698faddf5d61a8a4da8df53ad4d628d7250ca23898b8177af6 2024-02-17T12:01:15.490 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:01:15 smithi189 systemd[1]: Started Ceph mds.f for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T12:01:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.498+0000 7f46fded0640 1 -- 172.21.15.94:0/102217416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f8103f70 msgr2=0x7f46f8107fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.498+0000 7f46fded0640 1 --2- 172.21.15.94:0/102217416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f8103f70 0x7f46f8107fb0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009930 tx=0x7f46e802bc30 comp rx=0 tx=0).stop 2024-02-17T12:01:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.499+0000 7f46fded0640 1 -- 172.21.15.94:0/102217416 shutdown_connections 2024-02-17T12:01:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.499+0000 7f46fded0640 1 --2- 172.21.15.94:0/102217416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46f8108690 0x7f46f8110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.499+0000 7f46fded0640 1 --2- 172.21.15.94:0/102217416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f8103f70 0x7f46f8107fb0 secure :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009930 tx=0x7f46e802bc30 comp rx=0 tx=0).stop 2024-02-17T12:01:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.499+0000 7f46fded0640 1 --2- 172.21.15.94:0/102217416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46f81035c0 0x7f46f81039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.499+0000 7f46fded0640 1 -- 172.21.15.94:0/102217416 >> 172.21.15.94:0/102217416 conn(0x7f46f80fee40 msgr2=0x7f46f8101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.499+0000 7f46fded0640 1 -- 172.21.15.94:0/102217416 shutdown_connections 2024-02-17T12:01:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.500+0000 7f46fded0640 1 -- 172.21.15.94:0/102217416 wait complete. 2024-02-17T12:01:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.500+0000 7f46fded0640 1 Processor -- start 2024-02-17T12:01:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.501+0000 7f46fded0640 1 -- start start 2024-02-17T12:01:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.501+0000 7f46fded0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f81035c0 0x7f46f8110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.501+0000 7f46fded0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46f8108690 0x7f46f810d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.501+0000 7f46fded0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46f810d7f0 0x7f46f810dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.501+0000 7f46fded0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f81b2c60 con 0x7f46f81035c0 2024-02-17T12:01:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.501+0000 7f46fded0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f81b2dd0 con 0x7f46f810d7f0 2024-02-17T12:01:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.501+0000 7f46fded0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f81b2f90 con 0x7f46f8108690 2024-02-17T12:01:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.501+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46f810d7f0 0x7f46f810dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.501+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46f810d7f0 0x7f46f810dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41802/0 (socket says 172.21.15.94:41802) 2024-02-17T12:01:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.501+0000 7f46f7fff640 1 -- 172.21.15.94:0/1610990374 learned_addr learned my addr 172.21.15.94:0/1610990374 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.502+0000 7f46f6ffd640 1 --2- 172.21.15.94:0/1610990374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46f8108690 0x7f46f810d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.502+0000 7f46f7fff640 1 -- 172.21.15.94:0/1610990374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46f8108690 msgr2=0x7f46f810d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.502+0000 7f46f7fff640 1 --2- 172.21.15.94:0/1610990374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46f8108690 0x7f46f810d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.502+0000 7f46f7fff640 1 -- 172.21.15.94:0/1610990374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f81035c0 msgr2=0x7f46f8110170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.502+0000 7f46f7fff640 1 --2- 172.21.15.94:0/1610990374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f81035c0 0x7f46f8110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.502+0000 7f46f7fff640 1 -- 172.21.15.94:0/1610990374 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e8009560 con 0x7f46f810d7f0 2024-02-17T12:01:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.502+0000 7f46f7fff640 1 --2- 172.21.15.94:0/1610990374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46f810d7f0 0x7f46f810dc30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f46f000eb10 tx=0x7f46f000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.503+0000 7f46f4ff9640 1 -- 172.21.15.94:0/1610990374 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0004b20 con 0x7f46f810d7f0 2024-02-17T12:01:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.503+0000 7f46f4ff9640 1 -- 172.21.15.94:0/1610990374 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46f0004c80 con 0x7f46f810d7f0 2024-02-17T12:01:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.503+0000 7f46fded0640 1 -- 172.21.15.94:0/1610990374 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f81b3270 con 0x7f46f810d7f0 2024-02-17T12:01:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.503+0000 7f46fded0640 1 -- 172.21.15.94:0/1610990374 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f81b3750 con 0x7f46f810d7f0 2024-02-17T12:01:15.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.503+0000 7f46f4ff9640 1 -- 172.21.15.94:0/1610990374 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0010650 con 0x7f46f810d7f0 2024-02-17T12:01:15.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.507+0000 7f46f4ff9640 1 -- 172.21.15.94:0/1610990374 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f46f0005020 con 0x7f46f810d7f0 2024-02-17T12:01:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.507+0000 7f46f4ff9640 1 --2- 172.21.15.94:0/1610990374 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f46e00a2430 0x7f46e00a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.508+0000 7f46f77fe640 1 --2- 172.21.15.94:0/1610990374 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f46e00a2430 0x7f46e00a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.508+0000 7f46f4ff9640 1 -- 172.21.15.94:0/1610990374 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f46f0014070 con 0x7f46f810d7f0 2024-02-17T12:01:15.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.508+0000 7f4700b76740 1 Processor -- start 2024-02-17T12:01:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.509+0000 7f46f77fe640 1 --2- 172.21.15.94:0/1610990374 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f46e00a2430 0x7f46e00a48f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46ec005fd0 tx=0x7f46ec005c70 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:15.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.509+0000 7f4700b76740 1 -- start start 2024-02-17T12:01:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.510+0000 7f4700b76740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c60ad0 0x55edf1c5f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.510+0000 7f4700b76740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.510+0000 7f4700b76740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c6b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.510+0000 7f4700b76740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edf1bce9f0 con 0x55edf1c692e0 2024-02-17T12:01:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.510+0000 7f4700b76740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edf1bcbf20 con 0x55edf1c5fd60 2024-02-17T12:01:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.510+0000 7f4700b76740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edf1bc7700 con 0x55edf1c60ad0 2024-02-17T12:01:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.510+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c6b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.510+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c6b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55148/0 (socket says 172.21.15.94:55148) 2024-02-17T12:01:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.510+0000 7f46f7fff640 1 -- 172.21.15.94:0/3526369099 learned_addr learned my addr 172.21.15.94:0/3526369099 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:15.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.510+0000 7f46f77fe640 1 --2- 172.21.15.94:0/3526369099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c60ad0 0x55edf1c5f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.511+0000 7f46f77fe640 1 -- 172.21.15.94:0/3526369099 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 msgr2=0x55edf1c601e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:01:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.511+0000 7f46f77fe640 1 --2- 172.21.15.94:0/3526369099 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c601e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.511+0000 7f46f77fe640 1 -- 172.21.15.94:0/3526369099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 msgr2=0x55edf1c6b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.511+0000 7f46f77fe640 1 --2- 172.21.15.94:0/3526369099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c6b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.511+0000 7f46f77fe640 1 -- 172.21.15.94:0/3526369099 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edf1b5d7b0 con 0x55edf1c60ad0 2024-02-17T12:01:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.512+0000 7f46f77fe640 1 --2- 172.21.15.94:0/3526369099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c60ad0 0x55edf1c5f820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f46ec004350 tx=0x7f46ec010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38d43d4a08b752fb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.512+0000 7f46ddffb640 1 -- 172.21.15.94:0/3526369099 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec022070 con 0x55edf1c60ad0 2024-02-17T12:01:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.512+0000 7f46ddffb640 1 -- 172.21.15.94:0/3526369099 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46ec017070 con 0x55edf1c60ad0 2024-02-17T12:01:15.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.512+0000 7f46ddffb640 1 -- 172.21.15.94:0/3526369099 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec022070 con 0x55edf1c60ad0 2024-02-17T12:01:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.516+0000 7f4700b76740 1 -- 172.21.15.94:0/3526369099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c60ad0 msgr2=0x55edf1c5f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:15.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.517+0000 7f4700b76740 1 --2- 172.21.15.94:0/3526369099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c60ad0 0x55edf1c5f820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f46ec004350 tx=0x7f46ec010000 comp rx=0 tx=0).stop 2024-02-17T12:01:15.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.517+0000 7f4700b76740 1 -- 172.21.15.94:0/3526369099 shutdown_connections 2024-02-17T12:01:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.517+0000 7f4700b76740 1 --2- 172.21.15.94:0/3526369099 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c6b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.517+0000 7f4700b76740 1 --2- 172.21.15.94:0/3526369099 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.517+0000 7f4700b76740 1 --2- 172.21.15.94:0/3526369099 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c60ad0 0x55edf1c5f820 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f46ec004350 tx=0x7f46ec010000 comp rx=0 tx=0).stop 2024-02-17T12:01:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.517+0000 7f4700b76740 1 -- 172.21.15.94:0/3526369099 >> 172.21.15.94:0/3526369099 conn(0x55edf1c27550 msgr2=0x55edf1c5aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.518+0000 7f4700b76740 1 -- 172.21.15.94:0/3526369099 shutdown_connections 2024-02-17T12:01:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.518+0000 7f4700b76740 1 -- 172.21.15.94:0/3526369099 wait complete. 2024-02-17T12:01:15.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.519+0000 7f4700b76740 1 -- start start 2024-02-17T12:01:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.519+0000 7f4700b76740 1 Processor -- start 2024-02-17T12:01:15.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.520+0000 7f4700b76740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.520+0000 7f4700b76740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c692e0 0x55edf1c668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.520+0000 7f4700b76740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c66e20 0x55edf1c751c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.520+0000 7f46f77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.520+0000 7f46f77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c663a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41804/0 (socket says 172.21.15.94:41804) 2024-02-17T12:01:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.520+0000 7f46f77fe640 1 -- 172.21.15.94:0/2590042952 learned_addr learned my addr 172.21.15.94:0/2590042952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.520+0000 7f4700b76740 1 -- 172.21.15.94:0/2590042952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edf1b6cbf0 con 0x55edf1c66e20 2024-02-17T12:01:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.520+0000 7f4700b76740 1 -- 172.21.15.94:0/2590042952 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edf189b250 con 0x55edf1c5fd60 2024-02-17T12:01:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.520+0000 7f4700b76740 1 -- 172.21.15.94:0/2590042952 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edf1b4d200 con 0x55edf1c692e0 2024-02-17T12:01:15.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.520+0000 7f46f6ffd640 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c692e0 0x55edf1c668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.520+0000 7f46f7fff640 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c66e20 0x55edf1c751c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.521+0000 7f4700b76740 10 client.0 _ll_register_callbacks cb 0x55edf1bbbd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.522+0000 7f46f6ffd640 1 -- 172.21.15.94:0/2590042952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 msgr2=0x55edf1c663a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.522+0000 7f46f6ffd640 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c663a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.522+0000 7f46f6ffd640 1 -- 172.21.15.94:0/2590042952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c66e20 msgr2=0x55edf1c751c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.522+0000 7f46f6ffd640 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c66e20 0x55edf1c751c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.522+0000 7f46f6ffd640 1 -- 172.21.15.94:0/2590042952 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ec004030 con 0x55edf1c692e0 2024-02-17T12:01:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.522+0000 7f46f7fff640 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c66e20 0x55edf1c751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.522+0000 7f46f77fe640 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c663a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.522+0000 7f4700b76740 1 -- 172.21.15.94:0/2590042952 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55edf18c7600 con 0x55edf1c692e0 2024-02-17T12:01:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.522+0000 7f46f6ffd640 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c692e0 0x55edf1c668e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f46f81044c0 tx=0x7f46e802ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c4fcba9e44b389c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.522+0000 7f46d6ffd640 10 client.25861 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:01:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.523+0000 7f46d6ffd640 1 -- 172.21.15.94:0/2590042952 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e8004450 con 0x55edf1c692e0 2024-02-17T12:01:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.523+0000 7f46d6ffd640 1 -- 172.21.15.94:0/2590042952 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f46e802cc00 con 0x55edf1c692e0 2024-02-17T12:01:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.523+0000 7f46d6ffd640 1 -- 172.21.15.94:0/2590042952 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46e802c5d0 con 0x55edf1c692e0 2024-02-17T12:01:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.524+0000 7f4700b76740 10 client.25861 fetch_fsmap learned FSMap version 22 2024-02-17T12:01:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.524+0000 7f4700b76740 1 -- 172.21.15.94:0/2590042952 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=22}) v3 -- 0x55edf1bc5d80 con 0x55edf1c692e0 2024-02-17T12:01:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.525+0000 7f46d6ffd640 1 -- 172.21.15.94:0/2590042952 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 22) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f46e803c040 con 0x55edf1c692e0 2024-02-17T12:01:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.525+0000 7f4700b76740 10 client.25861 fetch_fsmap finished waiting for FSMap version 22 2024-02-17T12:01:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.525+0000 7f4700b76740 10 client.25861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.525+0000 7f4700b76740 20 client.25861 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.525+0000 7f4700b76740 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55edf1c799a0 0x55edf1c7bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.525+0000 7f4700b76740 4 client.25861 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.525+0000 7f4700b76740 1 -- 172.21.15.94:0/2590042952 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55edf19181b0 con 0x55edf1c799a0 2024-02-17T12:01:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.525+0000 7f46f77fe640 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55edf1c799a0 0x55edf1c7bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.526+0000 7f46d6ffd640 10 client.25861 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.526+0000 7f46f77fe640 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55edf1c799a0 0x55edf1c7bd90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52353912ff98936a server_cookie=d59e8290dca55c86 in_seq=0 out_seq=0 2024-02-17T12:01:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.527+0000 7f46d6ffd640 1 -- 172.21.15.94:0/2590042952 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55edf19181b0 con 0x55edf1c799a0 2024-02-17T12:01:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.527+0000 7f46d6ffd640 10 client.25861 handle_command_reply: tid=0 2024-02-17T12:01:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.527+0000 7f4700b76740 1 client.25861 shutdown 2024-02-17T12:01:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.528+0000 7f4700b76740 1 -- 172.21.15.94:0/2590042952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c692e0 msgr2=0x55edf1c668e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.529+0000 7f4700b76740 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c692e0 0x55edf1c668e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f46f81044c0 tx=0x7f46e802ca10 comp rx=0 tx=0).stop 2024-02-17T12:01:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.529+0000 7f4700b76740 1 -- 172.21.15.94:0/2590042952 shutdown_connections 2024-02-17T12:01:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.529+0000 7f4700b76740 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55edf1c799a0 0x55edf1c7bd90 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.530+0000 7f4700b76740 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c66e20 0x55edf1c751c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.530+0000 7f4700b76740 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c692e0 0x55edf1c668e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.530+0000 7f4700b76740 1 --2- 172.21.15.94:0/2590042952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c663a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.530+0000 7f4700b76740 1 -- 172.21.15.94:0/2590042952 >> 172.21.15.94:0/2590042952 conn(0x55edf1c27550 msgr2=0x55edf1c5c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.530+0000 7f46d6ffd640 0 client.25861 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.530+0000 7f4700b76740 1 -- 172.21.15.94:0/2590042952 shutdown_connections 2024-02-17T12:01:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.530+0000 7f4700b76740 1 -- 172.21.15.94:0/2590042952 wait complete. 2024-02-17T12:01:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.530+0000 7f4700b76740 20 client.25861 trim_cache size 0 max 16384 2024-02-17T12:01:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.562+0000 7f4700b76740 1 Processor -- start 2024-02-17T12:01:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.562+0000 7f4700b76740 1 -- start start 2024-02-17T12:01:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.562+0000 7f4700b76740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c70100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.562+0000 7f4700b76740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c66e20 0x55edf1c70640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.562+0000 7f4700b76740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c5a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.562+0000 7f4700b76740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edf19181b0 con 0x55edf1c692e0 2024-02-17T12:01:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.562+0000 7f4700b76740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edf1c5a310 con 0x55edf1c5fd60 2024-02-17T12:01:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.562+0000 7f4700b76740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edf1c76690 con 0x55edf1c66e20 2024-02-17T12:01:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.562+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c5a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.562+0000 7f46f77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c70100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.562+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c5a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55164/0 (socket says 172.21.15.94:55164) 2024-02-17T12:01:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.562+0000 7f46f77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c70100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41810/0 (socket says 172.21.15.94:41810) 2024-02-17T12:01:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.562+0000 7f46f7fff640 1 -- 172.21.15.94:0/3723144665 learned_addr learned my addr 172.21.15.94:0/3723144665 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.563+0000 7f46f6ffd640 1 --2- 172.21.15.94:0/3723144665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c66e20 0x55edf1c70640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.564+0000 7f46f77fe640 1 -- 172.21.15.94:0/3723144665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c66e20 msgr2=0x55edf1c70640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.564+0000 7f46f77fe640 1 --2- 172.21.15.94:0/3723144665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c66e20 0x55edf1c70640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.564+0000 7f46f77fe640 1 -- 172.21.15.94:0/3723144665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 msgr2=0x55edf1c5a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.564+0000 7f46f77fe640 1 --2- 172.21.15.94:0/3723144665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c5a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.564+0000 7f46f77fe640 1 -- 172.21.15.94:0/3723144665 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55edf1b627b0 con 0x55edf1c5fd60 2024-02-17T12:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.564+0000 7f46f77fe640 1 --2- 172.21.15.94:0/3723144665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c70100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f46ec003f90 tx=0x7f46ec004680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5683bd366be8b30 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.564+0000 7f46dd7fa640 1 -- 172.21.15.94:0/3723144665 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec035030 con 0x55edf1c5fd60 2024-02-17T12:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.565+0000 7f46dd7fa640 1 -- 172.21.15.94:0/3723144665 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46ec003b70 con 0x55edf1c5fd60 2024-02-17T12:01:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.565+0000 7f46dd7fa640 1 -- 172.21.15.94:0/3723144665 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ec022070 con 0x55edf1c5fd60 2024-02-17T12:01:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.565+0000 7f4700b76740 1 -- 172.21.15.94:0/3723144665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 msgr2=0x55edf1c70100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.565+0000 7f4700b76740 1 --2- 172.21.15.94:0/3723144665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c70100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f46ec003f90 tx=0x7f46ec004680 comp rx=0 tx=0).stop 2024-02-17T12:01:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.566+0000 7f4700b76740 1 -- 172.21.15.94:0/3723144665 shutdown_connections 2024-02-17T12:01:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.566+0000 7f4700b76740 1 --2- 172.21.15.94:0/3723144665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c5a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.566+0000 7f4700b76740 1 --2- 172.21.15.94:0/3723144665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c66e20 0x55edf1c70640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.566+0000 7f4700b76740 1 --2- 172.21.15.94:0/3723144665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c5fd60 0x55edf1c70100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.566+0000 7f4700b76740 1 -- 172.21.15.94:0/3723144665 >> 172.21.15.94:0/3723144665 conn(0x55edf1c27550 msgr2=0x55edf1c74bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.567+0000 7f4700b76740 1 -- 172.21.15.94:0/3723144665 shutdown_connections 2024-02-17T12:01:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.567+0000 7f4700b76740 1 -- 172.21.15.94:0/3723144665 wait complete. 2024-02-17T12:01:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.567+0000 7f4700b76740 1 -- start start 2024-02-17T12:01:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.568+0000 7f4700b76740 1 Processor -- start 2024-02-17T12:01:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.568+0000 7f4700b76740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c5fd60 0x55edf1c70110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.568+0000 7f4700b76740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c66e20 0x55edf1c62610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.568+0000 7f4700b76740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c62b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.568+0000 7f4700b76740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edf1c70650 con 0x55edf1c692e0 2024-02-17T12:01:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.568+0000 7f4700b76740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edf1c707c0 con 0x55edf1c66e20 2024-02-17T12:01:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.568+0000 7f4700b76740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55edf1c70980 con 0x55edf1c5fd60 2024-02-17T12:01:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.568+0000 7f4700b76740 10 client.0 _ll_register_callbacks cb 0x7f46ec00b2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.568+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c62b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.568+0000 7f46f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c62b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55174/0 (socket says 172.21.15.94:55174) 2024-02-17T12:01:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.568+0000 7f46f7fff640 1 -- 172.21.15.94:0/1904403091 learned_addr learned my addr 172.21.15.94:0/1904403091 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.568+0000 7f46f77fe640 1 --2- 172.21.15.94:0/1904403091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c5fd60 0x55edf1c70110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f46f6ffd640 1 --2- 172.21.15.94:0/1904403091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c66e20 0x55edf1c62610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f46f7fff640 1 -- 172.21.15.94:0/1904403091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c5fd60 msgr2=0x55edf1c70110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f46f7fff640 1 --2- 172.21.15.94:0/1904403091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c5fd60 0x55edf1c70110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f46f7fff640 1 -- 172.21.15.94:0/1904403091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c66e20 msgr2=0x55edf1c62610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f46f7fff640 1 --2- 172.21.15.94:0/1904403091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c66e20 0x55edf1c62610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f46f7fff640 1 -- 172.21.15.94:0/1904403091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55edf1b5d7b0 con 0x55edf1c692e0 2024-02-17T12:01:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f46f7fff640 1 --2- 172.21.15.94:0/1904403091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c62b50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f46f00979e0 tx=0x7f46f009b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da30c796883f254d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f46d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:01:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f46d6ffd640 1 -- 172.21.15.94:0/1904403091 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f00b9530 con 0x55edf1c692e0 2024-02-17T12:01:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f46d6ffd640 1 -- 172.21.15.94:0/1904403091 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f46f00b7070 con 0x55edf1c692e0 2024-02-17T12:01:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f4700b76740 1 -- 172.21.15.94:0/1904403091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46ec017070 con 0x55edf1c692e0 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f46d6ffd640 1 -- 172.21.15.94:0/1904403091 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46f00a3320 con 0x55edf1c692e0 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f4700b76740 10 client.16377 fetch_fsmap learned FSMap version 22 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.569+0000 7f4700b76740 1 -- 172.21.15.94:0/1904403091 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55edf1b88540 con 0x55edf1c692e0 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.570+0000 7f46d6ffd640 1 -- 172.21.15.94:0/1904403091 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46ec017070 con 0x55edf1c692e0 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.570+0000 7f46d6ffd640 1 -- 172.21.15.94:0/1904403091 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 22) v1 ==== 7307+0+0 (secure 0 0 0) 0x7f46f001f060 con 0x55edf1c692e0 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.570+0000 7f4700b76740 10 client.16377 fetch_fsmap finished waiting for FSMap version 22 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.570+0000 7f4700b76740 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.570+0000 7f4700b76740 20 client.16377 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.570+0000 7f4700b76740 1 --2- 172.21.15.94:0/1904403091 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55edf1cd0e50 0x55edf1cd3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.570+0000 7f4700b76740 4 client.16377 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.570+0000 7f4700b76740 1 -- 172.21.15.94:0/1904403091 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55edf1904550 con 0x55edf1cd0e50 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.570+0000 7f46f77fe640 1 --2- 172.21.15.94:0/1904403091 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55edf1cd0e50 0x55edf1cd3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.571+0000 7f46d6ffd640 10 client.16377 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.571+0000 7f46f77fe640 1 --2- 172.21.15.94:0/1904403091 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55edf1cd0e50 0x55edf1cd3240 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8684ef6e68f516f8 server_cookie=cdbce427215d4bcc in_seq=0 out_seq=0 2024-02-17T12:01:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.646+0000 7f46d6ffd640 1 -- 172.21.15.94:0/1904403091 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55edf1904550 con 0x55edf1cd0e50 2024-02-17T12:01:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.646+0000 7f46d6ffd640 10 client.16377 handle_command_reply: tid=0 2024-02-17T12:01:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.646+0000 7f4700b76740 1 client.16377 shutdown 2024-02-17T12:01:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.646+0000 7f4700b76740 1 -- 172.21.15.94:0/1904403091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 msgr2=0x55edf1c62b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.646+0000 7f4700b76740 1 --2- 172.21.15.94:0/1904403091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c62b50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f46f00979e0 tx=0x7f46f009b850 comp rx=0 tx=0).stop 2024-02-17T12:01:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.646+0000 7f4700b76740 1 -- 172.21.15.94:0/1904403091 shutdown_connections 2024-02-17T12:01:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.646+0000 7f4700b76740 1 --2- 172.21.15.94:0/1904403091 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55edf1cd0e50 0x55edf1cd3240 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.646+0000 7f4700b76740 1 --2- 172.21.15.94:0/1904403091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55edf1c692e0 0x55edf1c62b50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.646+0000 7f4700b76740 1 --2- 172.21.15.94:0/1904403091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55edf1c66e20 0x55edf1c62610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.646+0000 7f4700b76740 1 --2- 172.21.15.94:0/1904403091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55edf1c5fd60 0x55edf1c70110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.646+0000 7f4700b76740 1 -- 172.21.15.94:0/1904403091 >> 172.21.15.94:0/1904403091 conn(0x55edf1c27550 msgr2=0x55edf1c74bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.646+0000 7f46d6ffd640 0 client.16377 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.647+0000 7f4700b76740 1 -- 172.21.15.94:0/1904403091 shutdown_connections 2024-02-17T12:01:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.647+0000 7f4700b76740 1 -- 172.21.15.94:0/1904403091 wait complete. 2024-02-17T12:01:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.647+0000 7f4700b76740 20 client.16377 trim_cache size 0 max 16384 2024-02-17T12:01:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:01:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:01:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:01:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:01:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.647+0000 7f46fded0640 1 -- 172.21.15.94:0/1610990374 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f46e00a2430 msgr2=0x7f46e00a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.647+0000 7f46fded0640 1 --2- 172.21.15.94:0/1610990374 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f46e00a2430 0x7f46e00a48f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f46ec005fd0 tx=0x7f46ec005c70 comp rx=0 tx=0).stop 2024-02-17T12:01:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.647+0000 7f46fded0640 1 -- 172.21.15.94:0/1610990374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46f810d7f0 msgr2=0x7f46f810dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.647+0000 7f46fded0640 1 --2- 172.21.15.94:0/1610990374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46f810d7f0 0x7f46f810dc30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f46f000eb10 tx=0x7f46f000c2d0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.648+0000 7f46fded0640 1 -- 172.21.15.94:0/1610990374 shutdown_connections 2024-02-17T12:01:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.648+0000 7f46fded0640 1 --2- 172.21.15.94:0/1610990374 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f46e00a2430 0x7f46e00a48f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.648+0000 7f46fded0640 1 --2- 172.21.15.94:0/1610990374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f46f810d7f0 0x7f46f810dc30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.648+0000 7f46fded0640 1 --2- 172.21.15.94:0/1610990374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f46f8108690 0x7f46f810d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.648+0000 7f46fded0640 1 --2- 172.21.15.94:0/1610990374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f46f81035c0 0x7f46f8110170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.648+0000 7f46fded0640 1 -- 172.21.15.94:0/1610990374 >> 172.21.15.94:0/1610990374 conn(0x7f46f80fee40 msgr2=0x7f46f80ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.649+0000 7f46fded0640 1 -- 172.21.15.94:0/1610990374 shutdown_connections 2024-02-17T12:01:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:15.649+0000 7f46fded0640 1 -- 172.21.15.94:0/1610990374 wait complete. 2024-02-17T12:01:15.787 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:01:15.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:01:15.787 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:01:15.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.990+0000 7fe502e23640 1 -- 172.21.15.94:0/3772441400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4fc10d730 msgr2=0x7fe4fc10db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:15.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.990+0000 7fe502e23640 1 --2- 172.21.15.94:0/3772441400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4fc10d730 0x7fe4fc10db10 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe4f001a940 tx=0x7fe4f003cc90 comp rx=0 tx=0).stop 2024-02-17T12:01:15.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.990+0000 7fe502e23640 1 -- 172.21.15.94:0/3772441400 shutdown_connections 2024-02-17T12:01:15.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.990+0000 7fe502e23640 1 --2- 172.21.15.94:0/3772441400 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4fc1080e0 0x7fe4fc1145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.990+0000 7fe502e23640 1 --2- 172.21.15.94:0/3772441400 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4fc107720 0x7fe4fc107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.990+0000 7fe502e23640 1 --2- 172.21.15.94:0/3772441400 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4fc10d730 0x7fe4fc10db10 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.990+0000 7fe502e23640 1 -- 172.21.15.94:0/3772441400 >> 172.21.15.94:0/3772441400 conn(0x7fe4fc0756c0 msgr2=0x7fe4fc077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.990+0000 7fe502e23640 1 -- 172.21.15.94:0/3772441400 shutdown_connections 2024-02-17T12:01:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.990+0000 7fe502e23640 1 -- 172.21.15.94:0/3772441400 wait complete. 2024-02-17T12:01:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.991+0000 7fe502e23640 1 Processor -- start 2024-02-17T12:01:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.991+0000 7fe502e23640 1 -- start start 2024-02-17T12:01:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.992+0000 7fe502e23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4fc107720 0x7fe4fc114320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.992+0000 7fe502e23640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4fc1080e0 0x7fe4fc1103b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.992+0000 7fe502e23640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4fc10d730 0x7fe4fc1108f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.992+0000 7fe502e23640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc110f40 con 0x7fe4fc107720 2024-02-17T12:01:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.992+0000 7fe502e23640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc1110b0 con 0x7fe4fc10d730 2024-02-17T12:01:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.992+0000 7fe502e23640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4fc111270 con 0x7fe4fc1080e0 2024-02-17T12:01:15.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.992+0000 7fe4fbfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4fc1080e0 0x7fe4fc1103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.992+0000 7fe4fbfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4fc1080e0 0x7fe4fc1103b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60548/0 (socket says 172.21.15.94:60548) 2024-02-17T12:01:15.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.992+0000 7fe4fbfff640 1 -- 172.21.15.94:0/519833327 learned_addr learned my addr 172.21.15.94:0/519833327 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.992+0000 7fe5013c0640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4fc10d730 0x7fe4fc1108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.993+0000 7fe500bbf640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4fc107720 0x7fe4fc114320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.993+0000 7fe4fbfff640 1 -- 172.21.15.94:0/519833327 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4fc10d730 msgr2=0x7fe4fc1108f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.993+0000 7fe4fbfff640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4fc10d730 0x7fe4fc1108f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.993+0000 7fe4fbfff640 1 -- 172.21.15.94:0/519833327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4fc107720 msgr2=0x7fe4fc114320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.993+0000 7fe4fbfff640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4fc107720 0x7fe4fc114320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.993+0000 7fe4fbfff640 1 -- 172.21.15.94:0/519833327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f001a570 con 0x7fe4fc1080e0 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.993+0000 7fe500bbf640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4fc107720 0x7fe4fc114320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.993+0000 7fe4fbfff640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4fc1080e0 0x7fe4fc1103b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00b4a0 tx=0x7fe4ec00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.994+0000 7fe4f9ffb640 1 -- 172.21.15.94:0/519833327 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec010610 con 0x7fe4fc1080e0 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.994+0000 7fe502e23640 1 -- 172.21.15.94:0/519833327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4fc06ad40 con 0x7fe4fc1080e0 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.994+0000 7fe502e23640 1 -- 172.21.15.94:0/519833327 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc06b2a0 con 0x7fe4fc1080e0 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.995+0000 7fe4f9ffb640 1 -- 172.21.15.94:0/519833327 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4ec010c50 con 0x7fe4fc1080e0 2024-02-17T12:01:15.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.995+0000 7fe4f9ffb640 1 -- 172.21.15.94:0/519833327 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec017e20 con 0x7fe4fc1080e0 2024-02-17T12:01:15.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.998+0000 7fe4f9ffb640 1 -- 172.21.15.94:0/519833327 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe4ec00f360 con 0x7fe4fc1080e0 2024-02-17T12:01:15.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.999+0000 7fe4f9ffb640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe4e40a2370 0x7fe4e40a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:15.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:15.999+0000 7fe4f9ffb640 1 -- 172.21.15.94:0/519833327 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe4ec0ce630 con 0x7fe4fc1080e0 2024-02-17T12:01:16.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.000+0000 7fe500bbf640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe4e40a2370 0x7fe4e40a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.000+0000 7fe5059f9740 1 Processor -- start 2024-02-17T12:01:16.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.000+0000 7fe5059f9740 1 -- start start 2024-02-17T12:01:16.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe5059f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7380 0x55eb60ec7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe500bbf640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe4e40a2370 0x7fe4e40a4830 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0000c00 tx=0x7fe4f0004530 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:16.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe5059f9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7d30 0x55eb60ed02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe5059f9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ed08b0 0x55eb60ed2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe5059f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb60e35f90 con 0x55eb60ec7380 2024-02-17T12:01:16.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe5059f9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb60e334c0 con 0x55eb60ed08b0 2024-02-17T12:01:16.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe5059f9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb60e2eca0 con 0x55eb60ec7d30 2024-02-17T12:01:16.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe500bbf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7380 0x55eb60ec7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe500bbf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7380 0x55eb60ec7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55202/0 (socket says 172.21.15.94:55202) 2024-02-17T12:01:16.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe500bbf640 1 -- 172.21.15.94:0/2856467486 learned_addr learned my addr 172.21.15.94:0/2856467486 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:16.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe5013c0640 1 --2- 172.21.15.94:0/2856467486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ed08b0 0x55eb60ed2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe500bbf640 1 -- 172.21.15.94:0/2856467486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7d30 msgr2=0x55eb60ed02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe500bbf640 1 --2- 172.21.15.94:0/2856467486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7d30 0x55eb60ed02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe500bbf640 1 -- 172.21.15.94:0/2856467486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ed08b0 msgr2=0x55eb60ed2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe500bbf640 1 --2- 172.21.15.94:0/2856467486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ed08b0 0x55eb60ed2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.001+0000 7fe500bbf640 1 -- 172.21.15.94:0/2856467486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb60df10b0 con 0x55eb60ec7380 2024-02-17T12:01:16.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.002+0000 7fe500bbf640 1 --2- 172.21.15.94:0/2856467486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7380 0x55eb60ec7760 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0004800 tx=0x7fe4f000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9510f337c35e5aa3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:16.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.002+0000 7fe4e2ffd640 1 -- 172.21.15.94:0/2856467486 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f003d980 con 0x55eb60ec7380 2024-02-17T12:01:16.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.002+0000 7fe4e2ffd640 1 -- 172.21.15.94:0/2856467486 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4f0014070 con 0x55eb60ec7380 2024-02-17T12:01:16.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.002+0000 7fe4e2ffd640 1 -- 172.21.15.94:0/2856467486 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f00038e0 con 0x55eb60ec7380 2024-02-17T12:01:16.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.003+0000 7fe5059f9740 1 -- 172.21.15.94:0/2856467486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7380 msgr2=0x55eb60ec7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.003+0000 7fe5059f9740 1 --2- 172.21.15.94:0/2856467486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7380 0x55eb60ec7760 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0004800 tx=0x7fe4f000d000 comp rx=0 tx=0).stop 2024-02-17T12:01:16.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.003+0000 7fe5059f9740 1 -- 172.21.15.94:0/2856467486 shutdown_connections 2024-02-17T12:01:16.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.003+0000 7fe5059f9740 1 --2- 172.21.15.94:0/2856467486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ed08b0 0x55eb60ed2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.003+0000 7fe5059f9740 1 --2- 172.21.15.94:0/2856467486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7d30 0x55eb60ed02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.003+0000 7fe5059f9740 1 --2- 172.21.15.94:0/2856467486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7380 0x55eb60ec7760 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.003+0000 7fe5059f9740 1 -- 172.21.15.94:0/2856467486 >> 172.21.15.94:0/2856467486 conn(0x55eb60ec1840 msgr2=0x55eb60ec1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.004+0000 7fe5059f9740 1 -- 172.21.15.94:0/2856467486 shutdown_connections 2024-02-17T12:01:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.004+0000 7fe5059f9740 1 -- 172.21.15.94:0/2856467486 wait complete. 2024-02-17T12:01:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.004+0000 7fe5059f9740 1 -- start start 2024-02-17T12:01:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.004+0000 7fe5059f9740 1 Processor -- start 2024-02-17T12:01:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.005+0000 7fe5059f9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7380 0x55eb60ecd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.005+0000 7fe5059f9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec7d30 0x55eb60ecdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.005+0000 7fe5059f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ed08b0 0x55eb60ece2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.005+0000 7fe500bbf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7380 0x55eb60ecd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.005+0000 7fe500bbf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7380 0x55eb60ecd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60568/0 (socket says 172.21.15.94:60568) 2024-02-17T12:01:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.005+0000 7fe500bbf640 1 -- 172.21.15.94:0/1261551996 learned_addr learned my addr 172.21.15.94:0/1261551996 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.005+0000 7fe4fbfff640 1 --2- 172.21.15.94:0/1261551996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec7d30 0x55eb60ecdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.005+0000 7fe5013c0640 1 --2- 172.21.15.94:0/1261551996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ed08b0 0x55eb60ece2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.005+0000 7fe5059f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb60dd3f40 con 0x55eb60ed08b0 2024-02-17T12:01:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.005+0000 7fe5059f9740 1 -- 172.21.15.94:0/1261551996 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb60b024b0 con 0x55eb60ec7d30 2024-02-17T12:01:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.005+0000 7fe5059f9740 1 -- 172.21.15.94:0/1261551996 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb60db4670 con 0x55eb60ec7380 2024-02-17T12:01:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.006+0000 7fe5059f9740 10 client.0 _ll_register_callbacks cb 0x55eb60e21f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.006+0000 7fe4fbfff640 1 -- 172.21.15.94:0/1261551996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7380 msgr2=0x55eb60ecd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.006+0000 7fe4fbfff640 1 --2- 172.21.15.94:0/1261551996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7380 0x55eb60ecd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.006+0000 7fe4fbfff640 1 -- 172.21.15.94:0/1261551996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ed08b0 msgr2=0x55eb60ece2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.006+0000 7fe4fbfff640 1 --2- 172.21.15.94:0/1261551996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ed08b0 0x55eb60ece2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.006+0000 7fe4fbfff640 1 -- 172.21.15.94:0/1261551996 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4fc114ef0 con 0x55eb60ec7d30 2024-02-17T12:01:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.006+0000 7fe4fbfff640 1 --2- 172.21.15.94:0/1261551996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec7d30 0x55eb60ecdd90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0cbe20 tx=0x7fe4ec09b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=445271cd70125a95 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.006+0000 7fe4dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:16.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.006+0000 7fe4dbfff640 1 -- 172.21.15.94:0/1261551996 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0a3860 con 0x55eb60ec7d30 2024-02-17T12:01:16.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.006+0000 7fe4dbfff640 1 -- 172.21.15.94:0/1261551996 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe4ec0cbf90 con 0x55eb60ec7d30 2024-02-17T12:01:16.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.006+0000 7fe5059f9740 1 -- 172.21.15.94:0/1261551996 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb60e08f40 con 0x55eb60ec7d30 2024-02-17T12:01:16.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.007+0000 7fe4dbfff640 1 -- 172.21.15.94:0/1261551996 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4ec0cc150 con 0x55eb60ec7d30 2024-02-17T12:01:16.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.007+0000 7fe5059f9740 10 client.25826 fetch_fsmap learned FSMap version 22 2024-02-17T12:01:16.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.007+0000 7fe5059f9740 1 -- 172.21.15.94:0/1261551996 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55eb60e2d320 con 0x55eb60ec7d30 2024-02-17T12:01:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.007+0000 7fe4dbfff640 1 -- 172.21.15.94:0/1261551996 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55eb60e08f40 con 0x55eb60ec7d30 2024-02-17T12:01:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.007+0000 7fe4dbfff640 1 -- 172.21.15.94:0/1261551996 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 22) v1 ==== 7307+0+0 (secure 0 0 0) 0x7fe4ec09a510 con 0x55eb60ec7d30 2024-02-17T12:01:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.007+0000 7fe5059f9740 10 client.25826 fetch_fsmap finished waiting for FSMap version 22 2024-02-17T12:01:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.007+0000 7fe5059f9740 10 client.25826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.007+0000 7fe5059f9740 20 client.25826 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.008+0000 7fe5059f9740 1 --2- 172.21.15.94:0/1261551996 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55eb60ee0ee0 0x55eb60ee3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.008+0000 7fe5059f9740 4 client.25826 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.008+0000 7fe5059f9740 1 -- 172.21.15.94:0/1261551996 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb60e25310 con 0x55eb60ee0ee0 2024-02-17T12:01:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.008+0000 7fe500bbf640 1 --2- 172.21.15.94:0/1261551996 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55eb60ee0ee0 0x55eb60ee3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.008+0000 7fe500bbf640 1 --2- 172.21.15.94:0/1261551996 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55eb60ee0ee0 0x55eb60ee3300 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75ca0be870812bc6 server_cookie=4553de4c7500b553 in_seq=0 out_seq=0 2024-02-17T12:01:16.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.008+0000 7fe4dbfff640 10 client.25826 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe4dbfff640 1 -- 172.21.15.94:0/1261551996 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb60e25310 con 0x55eb60ee0ee0 2024-02-17T12:01:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe4dbfff640 10 client.25826 handle_command_reply: tid=0 2024-02-17T12:01:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe5059f9740 1 client.25826 shutdown 2024-02-17T12:01:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe5059f9740 1 -- 172.21.15.94:0/1261551996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec7d30 msgr2=0x55eb60ecdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe5059f9740 1 --2- 172.21.15.94:0/1261551996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec7d30 0x55eb60ecdd90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec0cbe20 tx=0x7fe4ec09b7c0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe5059f9740 1 -- 172.21.15.94:0/1261551996 shutdown_connections 2024-02-17T12:01:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe5059f9740 1 --2- 172.21.15.94:0/1261551996 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55eb60ee0ee0 0x55eb60ee3300 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe5059f9740 1 --2- 172.21.15.94:0/1261551996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ed08b0 0x55eb60ece2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe5059f9740 1 --2- 172.21.15.94:0/1261551996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec7d30 0x55eb60ecdd90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe4dbfff640 0 client.25826 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe5059f9740 1 --2- 172.21.15.94:0/1261551996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7380 0x55eb60ecd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe5059f9740 1 -- 172.21.15.94:0/1261551996 >> 172.21.15.94:0/1261551996 conn(0x55eb60ec1840 msgr2=0x55eb60ec37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:16.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe5059f9740 1 -- 172.21.15.94:0/1261551996 shutdown_connections 2024-02-17T12:01:16.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe5059f9740 1 -- 172.21.15.94:0/1261551996 wait complete. 2024-02-17T12:01:16.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.009+0000 7fe5059f9740 20 client.25826 trim_cache size 0 max 16384 2024-02-17T12:01:16.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.030+0000 7fe5059f9740 1 Processor -- start 2024-02-17T12:01:16.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.030+0000 7fe5059f9740 1 -- start start 2024-02-17T12:01:16.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.030+0000 7fe5059f9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec1840 0x55eb60ece0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.030+0000 7fe5059f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7380 0x55eb60ed7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.030+0000 7fe5059f9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7d30 0x55eb60ed7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.030+0000 7fe5059f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb60ed3700 con 0x55eb60ec7380 2024-02-17T12:01:16.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.031+0000 7fe5059f9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb60ecbda0 con 0x55eb60ec1840 2024-02-17T12:01:16.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.031+0000 7fe5059f9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb60ee7e90 con 0x55eb60ec7d30 2024-02-17T12:01:16.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.031+0000 7fe500bbf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec1840 0x55eb60ece0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.031+0000 7fe500bbf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec1840 0x55eb60ece0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41882/0 (socket says 172.21.15.94:41882) 2024-02-17T12:01:16.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.031+0000 7fe500bbf640 1 -- 172.21.15.94:0/148793645 learned_addr learned my addr 172.21.15.94:0/148793645 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:16.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.031+0000 7fe500bbf640 1 -- 172.21.15.94:0/148793645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7d30 msgr2=0x55eb60ed7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.031+0000 7fe500bbf640 1 --2- 172.21.15.94:0/148793645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7d30 0x55eb60ed7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.031+0000 7fe500bbf640 1 -- 172.21.15.94:0/148793645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7380 msgr2=0x55eb60ed7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.031+0000 7fe500bbf640 1 --2- 172.21.15.94:0/148793645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7380 0x55eb60ed7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.031+0000 7fe500bbf640 1 -- 172.21.15.94:0/148793645 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb60dc9c00 con 0x55eb60ec1840 2024-02-17T12:01:16.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.031+0000 7fe500bbf640 1 --2- 172.21.15.94:0/148793645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec1840 0x55eb60ece0f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe4f003d380 tx=0x7fe4f00058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e94beef94b40406a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:16.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.032+0000 7fe4e27fc640 1 -- 172.21.15.94:0/148793645 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0005250 con 0x55eb60ec1840 2024-02-17T12:01:16.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.032+0000 7fe4e27fc640 1 -- 172.21.15.94:0/148793645 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4f0014040 con 0x55eb60ec1840 2024-02-17T12:01:16.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.032+0000 7fe4e27fc640 1 -- 172.21.15.94:0/148793645 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0003ec0 con 0x55eb60ec1840 2024-02-17T12:01:16.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.034+0000 7fe5059f9740 1 -- 172.21.15.94:0/148793645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec1840 msgr2=0x55eb60ece0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.034+0000 7fe5059f9740 1 --2- 172.21.15.94:0/148793645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec1840 0x55eb60ece0f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe4f003d380 tx=0x7fe4f00058b0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.034+0000 7fe5059f9740 1 -- 172.21.15.94:0/148793645 shutdown_connections 2024-02-17T12:01:16.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.034+0000 7fe5059f9740 1 --2- 172.21.15.94:0/148793645 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec7d30 0x55eb60ed7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.034+0000 7fe5059f9740 1 --2- 172.21.15.94:0/148793645 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7380 0x55eb60ed7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.034+0000 7fe5059f9740 1 --2- 172.21.15.94:0/148793645 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec1840 0x55eb60ece0f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.034+0000 7fe5059f9740 1 -- 172.21.15.94:0/148793645 >> 172.21.15.94:0/148793645 conn(0x55eb60ec8480 msgr2=0x55eb60ec0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:16.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.034+0000 7fe5059f9740 1 -- 172.21.15.94:0/148793645 shutdown_connections 2024-02-17T12:01:16.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.035+0000 7fe5059f9740 1 -- 172.21.15.94:0/148793645 wait complete. 2024-02-17T12:01:16.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.035+0000 7fe5059f9740 1 -- start start 2024-02-17T12:01:16.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.035+0000 7fe5059f9740 1 Processor -- start 2024-02-17T12:01:16.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe5059f9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec1840 0x55eb60f31b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe5059f9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec7380 0x55eb60f32070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe5059f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7d30 0x55eb60ed6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe5059f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb60ed7560 con 0x55eb60ec7d30 2024-02-17T12:01:16.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe5059f9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb60ed76d0 con 0x55eb60ec7380 2024-02-17T12:01:16.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe500bbf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec1840 0x55eb60f31b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe5059f9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb60ed7890 con 0x55eb60ec1840 2024-02-17T12:01:16.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe500bbf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec1840 0x55eb60f31b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60586/0 (socket says 172.21.15.94:60586) 2024-02-17T12:01:16.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe500bbf640 1 -- 172.21.15.94:0/850901503 learned_addr learned my addr 172.21.15.94:0/850901503 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:16.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe4fbfff640 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec7380 0x55eb60f32070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe5013c0640 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7d30 0x55eb60ed6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe5059f9740 10 client.0 _ll_register_callbacks cb 0x7fe4f001a790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe4fbfff640 1 -- 172.21.15.94:0/850901503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec1840 msgr2=0x55eb60f31b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe4fbfff640 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec1840 0x55eb60f31b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe4fbfff640 1 -- 172.21.15.94:0/850901503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7d30 msgr2=0x55eb60ed6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe4fbfff640 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7d30 0x55eb60ed6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe4fbfff640 1 -- 172.21.15.94:0/850901503 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb60df10b0 con 0x55eb60ec7380 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.036+0000 7fe500bbf640 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec1840 0x55eb60f31b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.037+0000 7fe4fbfff640 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec7380 0x55eb60f32070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55eb60ec2530 tx=0x7fe4ec097360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1264e9fef51af37 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.037+0000 7fe4dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.037+0000 7fe4dbfff640 1 -- 172.21.15.94:0/850901503 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec0a3690 con 0x55eb60ec7380 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.037+0000 7fe4dbfff640 1 -- 172.21.15.94:0/850901503 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe4ec0a3860 con 0x55eb60ec7380 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.037+0000 7fe5059f9740 1 -- 172.21.15.94:0/850901503 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb60e54900 con 0x55eb60ec7380 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.037+0000 7fe4dbfff640 1 -- 172.21.15.94:0/850901503 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4ec017780 con 0x55eb60ec7380 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.037+0000 7fe5059f9740 10 client.25832 fetch_fsmap learned FSMap version 22 2024-02-17T12:01:16.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.037+0000 7fe5059f9740 1 -- 172.21.15.94:0/850901503 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=4+}) v3 -- 0x55eb60def910 con 0x55eb60ec7380 2024-02-17T12:01:16.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.038+0000 7fe4dbfff640 1 -- 172.21.15.94:0/850901503 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe4ec09ae90 con 0x55eb60ec7380 2024-02-17T12:01:16.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.038+0000 7fe4dbfff640 1 -- 172.21.15.94:0/850901503 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 22) v1 ==== 7307+0+0 (secure 0 0 0) 0x7fe4ec081070 con 0x55eb60ec7380 2024-02-17T12:01:16.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.038+0000 7fe5059f9740 10 client.25832 fetch_fsmap finished waiting for FSMap version 22 2024-02-17T12:01:16.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.038+0000 7fe5059f9740 10 client.25832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:16.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.038+0000 7fe5059f9740 20 client.25832 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:16.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.038+0000 7fe5059f9740 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55eb60ed08b0 0x55eb60f3b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.038+0000 7fe5059f9740 4 client.25832 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:01:16.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.038+0000 7fe5059f9740 1 -- 172.21.15.94:0/850901503 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55eb60e54900 con 0x55eb60ed08b0 2024-02-17T12:01:16.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.038+0000 7fe500bbf640 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55eb60ed08b0 0x55eb60f3b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.039+0000 7fe4dbfff640 10 client.25832 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:16.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.039+0000 7fe500bbf640 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55eb60ed08b0 0x55eb60f3b540 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe38a6a879bdad4 server_cookie=2bf410ca99252a6b in_seq=0 out_seq=0 2024-02-17T12:01:16.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.040+0000 7fe4dbfff640 1 -- 172.21.15.94:0/850901503 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55eb60e54900 con 0x55eb60ed08b0 2024-02-17T12:01:16.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.040+0000 7fe4dbfff640 10 client.25832 handle_command_reply: tid=0 2024-02-17T12:01:16.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.040+0000 7fe5059f9740 1 client.25832 shutdown 2024-02-17T12:01:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.040+0000 7fe5059f9740 1 -- 172.21.15.94:0/850901503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec7380 msgr2=0x55eb60f32070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.040+0000 7fe5059f9740 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec7380 0x55eb60f32070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x55eb60ec2530 tx=0x7fe4ec097360 comp rx=0 tx=0).stop 2024-02-17T12:01:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.041+0000 7fe5059f9740 1 -- 172.21.15.94:0/850901503 shutdown_connections 2024-02-17T12:01:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.041+0000 7fe5059f9740 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55eb60ed08b0 0x55eb60f3b540 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.041+0000 7fe5059f9740 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55eb60ec7d30 0x55eb60ed6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.041+0000 7fe5059f9740 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55eb60ec7380 0x55eb60f32070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.041+0000 7fe5059f9740 1 --2- 172.21.15.94:0/850901503 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55eb60ec1840 0x55eb60f31b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.041+0000 7fe5059f9740 1 -- 172.21.15.94:0/850901503 >> 172.21.15.94:0/850901503 conn(0x55eb60ec8480 msgr2=0x55eb60ec08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.042+0000 7fe4dbfff640 0 client.25832 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.042+0000 7fe5059f9740 1 -- 172.21.15.94:0/850901503 shutdown_connections 2024-02-17T12:01:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.042+0000 7fe5059f9740 1 -- 172.21.15.94:0/850901503 wait complete. 2024-02-17T12:01:16.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.042+0000 7fe5059f9740 20 client.25832 trim_cache size 0 max 16384 2024-02-17T12:01:16.043 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:01:16.043 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:01:16.043 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "0b7e3ea9-ba97-49b3-b9de-e209d24525f1", 2024-02-17T12:01:16.043 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:01:16.043 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:01:16.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.043+0000 7fe502e23640 1 -- 172.21.15.94:0/519833327 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe4e40a2370 msgr2=0x7fe4e40a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.043+0000 7fe502e23640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe4e40a2370 0x7fe4e40a4830 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0000c00 tx=0x7fe4f0004530 comp rx=0 tx=0).stop 2024-02-17T12:01:16.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.043+0000 7fe502e23640 1 -- 172.21.15.94:0/519833327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4fc1080e0 msgr2=0x7fe4fc1103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.043+0000 7fe502e23640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4fc1080e0 0x7fe4fc1103b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec00b4a0 tx=0x7fe4ec00b960 comp rx=0 tx=0).stop 2024-02-17T12:01:16.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.043+0000 7fe502e23640 1 -- 172.21.15.94:0/519833327 shutdown_connections 2024-02-17T12:01:16.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.043+0000 7fe502e23640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe4e40a2370 0x7fe4e40a4830 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.043+0000 7fe502e23640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe4fc10d730 0x7fe4fc1108f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.043+0000 7fe502e23640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4fc1080e0 0x7fe4fc1103b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.043+0000 7fe502e23640 1 --2- 172.21.15.94:0/519833327 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe4fc107720 0x7fe4fc114320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.043+0000 7fe502e23640 1 -- 172.21.15.94:0/519833327 >> 172.21.15.94:0/519833327 conn(0x7fe4fc0756c0 msgr2=0x7fe4fc075e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:16.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.043+0000 7fe502e23640 1 -- 172.21.15.94:0/519833327 shutdown_connections 2024-02-17T12:01:16.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.043+0000 7fe502e23640 1 -- 172.21.15.94:0/519833327 wait complete. 2024-02-17T12:01:16.061 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:01:16.061 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:01:16.104 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:15 smithi094 ceph-mon[32499]: pgmap v97: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 MiB/s wr, 1.46k op/s 2024-02-17T12:01:16.104 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:15 smithi094 ceph-mon[32499]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-02-17T12:01:16.104 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:15 smithi094 ceph-mon[32499]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-02-17T12:01:16.104 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:15 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:01:16.150 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:15 smithi189 ceph-mon[36588]: pgmap v97: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 MiB/s wr, 1.46k op/s 2024-02-17T12:01:16.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:15 smithi189 ceph-mon[36588]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-02-17T12:01:16.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:15 smithi189 ceph-mon[36588]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-02-17T12:01:16.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:15 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:01:16.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:15 smithi125 ceph-mon[36819]: pgmap v97: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 39 MiB/s wr, 1.46k op/s 2024-02-17T12:01:16.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:15 smithi125 ceph-mon[36819]: MDS daemon mds.f is removed because it is dead or otherwise unavailable. 2024-02-17T12:01:16.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:15 smithi125 ceph-mon[36819]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-02-17T12:01:16.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:15 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:01:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.243+0000 7f41c9b77640 1 -- 172.21.15.94:0/327206139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41c41064d0 msgr2=0x7f41c410e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.243+0000 7f41c9b77640 1 --2- 172.21.15.94:0/327206139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41c41064d0 0x7f41c410e080 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009ca0 tx=0x7f41bc02c120 comp rx=0 tx=0).stop 2024-02-17T12:01:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.243+0000 7f41c9b77640 1 -- 172.21.15.94:0/327206139 shutdown_connections 2024-02-17T12:01:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.243+0000 7f41c9b77640 1 --2- 172.21.15.94:0/327206139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41c41064d0 0x7f41c410e080 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.243+0000 7f41c9b77640 1 --2- 172.21.15.94:0/327206139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41c406a410 0x7f41c4105df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.243+0000 7f41c9b77640 1 --2- 172.21.15.94:0/327206139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c4069a60 0x7f41c4069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.243+0000 7f41c9b77640 1 -- 172.21.15.94:0/327206139 >> 172.21.15.94:0/327206139 conn(0x7f41c4077170 msgr2=0x7f41c4077580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:16.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.246+0000 7f41c9b77640 1 -- 172.21.15.94:0/327206139 shutdown_connections 2024-02-17T12:01:16.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.246+0000 7f41c9b77640 1 -- 172.21.15.94:0/327206139 wait complete. 2024-02-17T12:01:16.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.247+0000 7f41c9b77640 1 Processor -- start 2024-02-17T12:01:16.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.247+0000 7f41c9b77640 1 -- start start 2024-02-17T12:01:16.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.247+0000 7f41c9b77640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c4069a60 0x7f41c419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.247+0000 7f41c9b77640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41c41064d0 0x7f41c419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.247+0000 7f41c9b77640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41c41997c0 0x7f41c4199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.247+0000 7f41c9b77640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c419a290 con 0x7f41c4069a60 2024-02-17T12:01:16.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.247+0000 7f41c9b77640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c419a400 con 0x7f41c41997c0 2024-02-17T12:01:16.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.247+0000 7f41c9b77640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c419a5c0 con 0x7f41c41064d0 2024-02-17T12:01:16.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.248+0000 7f41c3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41c41997c0 0x7f41c4199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.248+0000 7f41c3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41c41997c0 0x7f41c4199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41920/0 (socket says 172.21.15.94:41920) 2024-02-17T12:01:16.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.248+0000 7f41c3fff640 1 -- 172.21.15.94:0/2803027127 learned_addr learned my addr 172.21.15.94:0/2803027127 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:16.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.248+0000 7f41c2ffd640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41c41064d0 0x7f41c419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.248+0000 7f41c37fe640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c4069a60 0x7f41c419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.248+0000 7f41c3fff640 1 -- 172.21.15.94:0/2803027127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41c41064d0 msgr2=0x7f41c419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.248+0000 7f41c3fff640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41c41064d0 0x7f41c419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.248+0000 7f41c3fff640 1 -- 172.21.15.94:0/2803027127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c4069a60 msgr2=0x7f41c419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.248+0000 7f41c3fff640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c4069a60 0x7f41c419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.248+0000 7f41c3fff640 1 -- 172.21.15.94:0/2803027127 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc009900 con 0x7f41c41997c0 2024-02-17T12:01:16.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.248+0000 7f41c37fe640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c4069a60 0x7f41c419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:01:16.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.248+0000 7f41c3fff640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41c41997c0 0x7f41c4199c40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009dc0 tx=0x7f41bc0329d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:16.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.249+0000 7f41c0ff9640 1 -- 172.21.15.94:0/2803027127 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc034850 con 0x7f41c41997c0 2024-02-17T12:01:16.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.249+0000 7f41c9b77640 1 -- 172.21.15.94:0/2803027127 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c406aba0 con 0x7f41c41997c0 2024-02-17T12:01:16.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.249+0000 7f41c9b77640 1 -- 172.21.15.94:0/2803027127 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c406b080 con 0x7f41c41997c0 2024-02-17T12:01:16.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.249+0000 7f41c0ff9640 1 -- 172.21.15.94:0/2803027127 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41bc032bc0 con 0x7f41c41997c0 2024-02-17T12:01:16.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.249+0000 7f41c0ff9640 1 -- 172.21.15.94:0/2803027127 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc004860 con 0x7f41c41997c0 2024-02-17T12:01:16.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.253+0000 7f41c0ff9640 1 -- 172.21.15.94:0/2803027127 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f41bc03b050 con 0x7f41c41997c0 2024-02-17T12:01:16.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41cc827740 1 Processor -- start 2024-02-17T12:01:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41cc827740 1 -- start start 2024-02-17T12:01:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41c0ff9640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f41ac0a2400 0x7f41ac0a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41cc827740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd6444a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41cc827740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd644a70 0x559dcd655a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41cc827740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd655fc0 0x559dcd6583b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41cc827740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcd5b3280 con 0x559dcd6440c0 2024-02-17T12:01:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41cc827740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcd5b07b0 con 0x559dcd655fc0 2024-02-17T12:01:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41cc827740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcd5abf90 con 0x559dcd644a70 2024-02-17T12:01:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41c0ff9640 1 -- 172.21.15.94:0/2803027127 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f41bc0f07a0 con 0x7f41c41997c0 2024-02-17T12:01:16.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd6444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd6444a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55280/0 (socket says 172.21.15.94:55280) 2024-02-17T12:01:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41c37fe640 1 -- 172.21.15.94:0/2163922620 learned_addr learned my addr 172.21.15.94:0/2163922620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41c2ffd640 1 --2- 172.21.15.94:0/2163922620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd644a70 0x559dcd655a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41c3fff640 1 --2- 172.21.15.94:0/2163922620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd655fc0 0x559dcd6583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.254+0000 7f41c2ffd640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f41ac0a2400 0x7f41ac0a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.255+0000 7f41c37fe640 1 -- 172.21.15.94:0/2163922620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd644a70 msgr2=0x559dcd655a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.255+0000 7f41c37fe640 1 --2- 172.21.15.94:0/2163922620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd644a70 0x559dcd655a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.255+0000 7f41c37fe640 1 -- 172.21.15.94:0/2163922620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd655fc0 msgr2=0x559dcd6583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.255+0000 7f41c37fe640 1 --2- 172.21.15.94:0/2163922620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd655fc0 0x559dcd6583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.255+0000 7f41c37fe640 1 -- 172.21.15.94:0/2163922620 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dcd56a570 con 0x559dcd6440c0 2024-02-17T12:01:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.255+0000 7f41c3fff640 1 --2- 172.21.15.94:0/2163922620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd655fc0 0x559dcd6583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.255+0000 7f41c37fe640 1 --2- 172.21.15.94:0/2163922620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd6444a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f41b400aa10 tx=0x7f41b400ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9d4e4fc48cf8b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:16.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.255+0000 7f41a9ffb640 1 -- 172.21.15.94:0/2163922620 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b400cc00 con 0x559dcd6440c0 2024-02-17T12:01:16.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.256+0000 7f41a9ffb640 1 -- 172.21.15.94:0/2163922620 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41b400cd60 con 0x559dcd6440c0 2024-02-17T12:01:16.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.256+0000 7f41a9ffb640 1 -- 172.21.15.94:0/2163922620 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b4018800 con 0x559dcd6440c0 2024-02-17T12:01:16.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.257+0000 7f41cc827740 1 -- 172.21.15.94:0/2163922620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 msgr2=0x559dcd6444a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.257+0000 7f41cc827740 1 --2- 172.21.15.94:0/2163922620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd6444a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f41b400aa10 tx=0x7f41b400ae10 comp rx=0 tx=0).stop 2024-02-17T12:01:16.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.257+0000 7f41c2ffd640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f41ac0a2400 0x7f41ac0a48c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x559dcd63fc70 tx=0x7f41b8009110 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:16.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.257+0000 7f41cc827740 1 -- 172.21.15.94:0/2163922620 shutdown_connections 2024-02-17T12:01:16.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.257+0000 7f41cc827740 1 --2- 172.21.15.94:0/2163922620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd655fc0 0x559dcd6583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.257+0000 7f41cc827740 1 --2- 172.21.15.94:0/2163922620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd644a70 0x559dcd655a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.257+0000 7f41cc827740 1 --2- 172.21.15.94:0/2163922620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd6444a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.257+0000 7f41cc827740 1 -- 172.21.15.94:0/2163922620 >> 172.21.15.94:0/2163922620 conn(0x559dcd63e990 msgr2=0x559dcd63ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:16.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.258+0000 7f41cc827740 1 -- 172.21.15.94:0/2163922620 shutdown_connections 2024-02-17T12:01:16.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.258+0000 7f41cc827740 1 -- 172.21.15.94:0/2163922620 wait complete. 2024-02-17T12:01:16.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.259+0000 7f41cc827740 1 -- start start 2024-02-17T12:01:16.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.259+0000 7f41cc827740 1 Processor -- start 2024-02-17T12:01:16.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.259+0000 7f41cc827740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd64f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.259+0000 7f41cc827740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd644a70 0x559dcd64fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.259+0000 7f41cc827740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd655fc0 0x559dcd6553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41cc827740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcd27f430 con 0x559dcd6440c0 2024-02-17T12:01:16.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41cc827740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcd23be00 con 0x559dcd655fc0 2024-02-17T12:01:16.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41cc827740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcd530fd0 con 0x559dcd644a70 2024-02-17T12:01:16.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41c2ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd644a70 0x559dcd64fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41c2ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd644a70 0x559dcd64fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60614/0 (socket says 172.21.15.94:60614) 2024-02-17T12:01:16.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd64f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41c2ffd640 1 -- 172.21.15.94:0/4153592505 learned_addr learned my addr 172.21.15.94:0/4153592505 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:16.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41cc827740 10 client.0 _ll_register_callbacks cb 0x559dcd586fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41c3fff640 1 --2- 172.21.15.94:0/4153592505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd655fc0 0x559dcd6553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41c2ffd640 1 -- 172.21.15.94:0/4153592505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd655fc0 msgr2=0x559dcd6553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41c2ffd640 1 --2- 172.21.15.94:0/4153592505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd655fc0 0x559dcd6553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41c2ffd640 1 -- 172.21.15.94:0/4153592505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 msgr2=0x559dcd64f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41c2ffd640 1 --2- 172.21.15.94:0/4153592505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd64f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41c2ffd640 1 -- 172.21.15.94:0/4153592505 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b400a640 con 0x559dcd644a70 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41c3fff640 1 --2- 172.21.15.94:0/4153592505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd655fc0 0x559dcd6553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41c2ffd640 1 --2- 172.21.15.94:0/4153592505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd644a70 0x559dcd64fc10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009ad0 tx=0x7f41b80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=727f13859fbab2c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.260+0000 7f41a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.261+0000 7f41a2ffd640 1 -- 172.21.15.94:0/4153592505 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8028de0 con 0x559dcd644a70 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.261+0000 7f41a2ffd640 1 -- 172.21.15.94:0/4153592505 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f41b802d040 con 0x559dcd644a70 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.261+0000 7f41cc827740 1 -- 172.21.15.94:0/4153592505 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559dcd2ab7e0 con 0x559dcd644a70 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.261+0000 7f41a2ffd640 1 -- 172.21.15.94:0/4153592505 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41b8026db0 con 0x559dcd644a70 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.261+0000 7f41cc827740 10 client.25885 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:16.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.261+0000 7f41cc827740 1 -- 172.21.15.94:0/4153592505 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559dcd5aa610 con 0x559dcd644a70 2024-02-17T12:01:16.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.262+0000 7f41a2ffd640 1 -- 172.21.15.94:0/4153592505 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41b8006020 con 0x559dcd644a70 2024-02-17T12:01:16.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.262+0000 7f41a2ffd640 1 -- 172.21.15.94:0/4153592505 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f41b8033460 con 0x559dcd644a70 2024-02-17T12:01:16.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.262+0000 7f41cc827740 10 client.25885 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:16.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.262+0000 7f41cc827740 10 client.25885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:16.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.262+0000 7f41cc827740 20 client.25885 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:16.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.262+0000 7f41cc827740 1 --2- 172.21.15.94:0/4153592505 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559dcd66e770 0x559dcd670bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.262+0000 7f41cc827740 4 client.25885 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:16.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.262+0000 7f41cc827740 1 -- 172.21.15.94:0/4153592505 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559dcd2ab7e0 con 0x559dcd66e770 2024-02-17T12:01:16.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.263+0000 7f41c37fe640 1 --2- 172.21.15.94:0/4153592505 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559dcd66e770 0x559dcd670bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.266+0000 7f41a2ffd640 10 client.25885 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:16.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.266+0000 7f41c37fe640 1 --2- 172.21.15.94:0/4153592505 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559dcd66e770 0x559dcd670bf0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=860e009fea60634d server_cookie=9249e60e9a184d4a in_seq=0 out_seq=0 2024-02-17T12:01:16.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.266+0000 7f41a2ffd640 1 -- 172.21.15.94:0/4153592505 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559dcd2ab7e0 con 0x559dcd66e770 2024-02-17T12:01:16.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.266+0000 7f41a2ffd640 10 client.25885 handle_command_reply: tid=0 2024-02-17T12:01:16.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.267+0000 7f41cc827740 1 client.25885 shutdown 2024-02-17T12:01:16.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.268+0000 7f41cc827740 1 -- 172.21.15.94:0/4153592505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd644a70 msgr2=0x559dcd64fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.268+0000 7f41cc827740 1 --2- 172.21.15.94:0/4153592505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd644a70 0x559dcd64fc10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009ad0 tx=0x7f41b80046c0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.268+0000 7f41cc827740 1 -- 172.21.15.94:0/4153592505 shutdown_connections 2024-02-17T12:01:16.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.268+0000 7f41cc827740 1 --2- 172.21.15.94:0/4153592505 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559dcd66e770 0x559dcd670bf0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.268+0000 7f41cc827740 1 --2- 172.21.15.94:0/4153592505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd655fc0 0x559dcd6553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.268+0000 7f41cc827740 1 --2- 172.21.15.94:0/4153592505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd644a70 0x559dcd64fc10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.268+0000 7f41cc827740 1 --2- 172.21.15.94:0/4153592505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd64f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.268+0000 7f41cc827740 1 -- 172.21.15.94:0/4153592505 >> 172.21.15.94:0/4153592505 conn(0x559dcd63e990 msgr2=0x559dcd640730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:16.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.268+0000 7f41a2ffd640 0 client.25885 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:16.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.268+0000 7f41cc827740 1 -- 172.21.15.94:0/4153592505 shutdown_connections 2024-02-17T12:01:16.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.268+0000 7f41cc827740 1 -- 172.21.15.94:0/4153592505 wait complete. 2024-02-17T12:01:16.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.268+0000 7f41cc827740 20 client.25885 trim_cache size 0 max 16384 2024-02-17T12:01:16.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.293+0000 7f41cc827740 1 Processor -- start 2024-02-17T12:01:16.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.293+0000 7f41cc827740 1 -- start start 2024-02-17T12:01:16.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.294+0000 7f41cc827740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd647d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.294+0000 7f41cc827740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd644a70 0x559dcd648270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.294+0000 7f41cc827740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd655fc0 0x559dcd6573b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.294+0000 7f41cc827740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcd63f810 con 0x559dcd6440c0 2024-02-17T12:01:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.294+0000 7f41cc827740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcd63a2e0 con 0x559dcd644a70 2024-02-17T12:01:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.294+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd647d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.294+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd647d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55300/0 (socket says 172.21.15.94:55300) 2024-02-17T12:01:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.294+0000 7f41c37fe640 1 -- 172.21.15.94:0/2067076679 learned_addr learned my addr 172.21.15.94:0/2067076679 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.294+0000 7f41cc827740 1 -- 172.21.15.94:0/2067076679 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcd649020 con 0x559dcd655fc0 2024-02-17T12:01:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.294+0000 7f41c2ffd640 1 --2- 172.21.15.94:0/2067076679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd644a70 0x559dcd648270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.294+0000 7f41c3fff640 1 --2- 172.21.15.94:0/2067076679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd655fc0 0x559dcd6573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.295+0000 7f41c2ffd640 1 -- 172.21.15.94:0/2067076679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd655fc0 msgr2=0x559dcd6573b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.295+0000 7f41c2ffd640 1 --2- 172.21.15.94:0/2067076679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd655fc0 0x559dcd6573b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.295+0000 7f41c2ffd640 1 -- 172.21.15.94:0/2067076679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 msgr2=0x559dcd647d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.295+0000 7f41c2ffd640 1 --2- 172.21.15.94:0/2067076679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd647d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.295+0000 7f41c2ffd640 1 -- 172.21.15.94:0/2067076679 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559dcd546ac0 con 0x559dcd644a70 2024-02-17T12:01:16.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.295+0000 7f41c2ffd640 1 --2- 172.21.15.94:0/2067076679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd644a70 0x559dcd648270 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f41b8027c40 tx=0x7f41b8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8276a5286d05125 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:16.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.295+0000 7f41a97fa640 1 -- 172.21.15.94:0/2067076679 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8026db0 con 0x559dcd644a70 2024-02-17T12:01:16.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.295+0000 7f41a97fa640 1 -- 172.21.15.94:0/2067076679 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41b8033a00 con 0x559dcd644a70 2024-02-17T12:01:16.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.295+0000 7f41a97fa640 1 -- 172.21.15.94:0/2067076679 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8033660 con 0x559dcd644a70 2024-02-17T12:01:16.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.296+0000 7f41cc827740 1 -- 172.21.15.94:0/2067076679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd644a70 msgr2=0x559dcd648270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.296+0000 7f41cc827740 1 --2- 172.21.15.94:0/2067076679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd644a70 0x559dcd648270 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f41b8027c40 tx=0x7f41b8000bc0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.297+0000 7f41cc827740 1 -- 172.21.15.94:0/2067076679 shutdown_connections 2024-02-17T12:01:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.297+0000 7f41cc827740 1 --2- 172.21.15.94:0/2067076679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd655fc0 0x559dcd6573b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.297+0000 7f41cc827740 1 --2- 172.21.15.94:0/2067076679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd644a70 0x559dcd648270 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f41b8027c40 tx=0x7f41b8000bc0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.297+0000 7f41cc827740 1 --2- 172.21.15.94:0/2067076679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd6440c0 0x559dcd647d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.297+0000 7f41cc827740 1 -- 172.21.15.94:0/2067076679 >> 172.21.15.94:0/2067076679 conn(0x559dcd63e990 msgr2=0x559dcd6493a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.297+0000 7f41cc827740 1 -- 172.21.15.94:0/2067076679 shutdown_connections 2024-02-17T12:01:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.297+0000 7f41cc827740 1 -- 172.21.15.94:0/2067076679 wait complete. 2024-02-17T12:01:16.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41cc827740 1 -- start start 2024-02-17T12:01:16.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41cc827740 1 Processor -- start 2024-02-17T12:01:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41cc827740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd6440c0 0x559dcd654180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41cc827740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd655fc0 0x559dcd6546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41cc827740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd6bee40 0x559dcd6bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41cc827740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcd6bf800 con 0x559dcd655fc0 2024-02-17T12:01:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41cc827740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcd6bf970 con 0x559dcd6440c0 2024-02-17T12:01:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41cc827740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559dcd6bfb30 con 0x559dcd6bee40 2024-02-17T12:01:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41cc827740 10 client.0 _ll_register_callbacks cb 0x7f41b400a1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd6440c0 0x559dcd654180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41c2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd655fc0 0x559dcd6546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41c37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd6440c0 0x559dcd654180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41958/0 (socket says 172.21.15.94:41958) 2024-02-17T12:01:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41c37fe640 1 -- 172.21.15.94:0/3140220102 learned_addr learned my addr 172.21.15.94:0/3140220102 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:16.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.298+0000 7f41c3fff640 1 --2- 172.21.15.94:0/3140220102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd6bee40 0x559dcd6bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.299+0000 7f41c37fe640 1 -- 172.21.15.94:0/3140220102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd6bee40 msgr2=0x559dcd6bf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.299+0000 7f41c37fe640 1 --2- 172.21.15.94:0/3140220102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd6bee40 0x559dcd6bf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.299+0000 7f41c37fe640 1 -- 172.21.15.94:0/3140220102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd655fc0 msgr2=0x559dcd6546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.299+0000 7f41c37fe640 1 --2- 172.21.15.94:0/3140220102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd655fc0 0x559dcd6546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.299+0000 7f41c37fe640 1 -- 172.21.15.94:0/3140220102 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559dcd5aa610 con 0x559dcd6440c0 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.299+0000 7f41c37fe640 1 --2- 172.21.15.94:0/3140220102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd6440c0 0x559dcd654180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f41b400a930 tx=0x7f41b4011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b0ac7c9355bf544 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.299+0000 7f41a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.299+0000 7f41a2ffd640 1 -- 172.21.15.94:0/3140220102 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b401f040 con 0x559dcd6440c0 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.299+0000 7f41cc827740 1 -- 172.21.15.94:0/3140220102 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41b400cd60 con 0x559dcd6440c0 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.299+0000 7f41a2ffd640 1 -- 172.21.15.94:0/3140220102 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f41b4027830 con 0x559dcd6440c0 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.299+0000 7f41a2ffd640 1 -- 172.21.15.94:0/3140220102 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41b40113f0 con 0x559dcd6440c0 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.299+0000 7f41cc827740 10 client.25850 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.299+0000 7f41cc827740 1 -- 172.21.15.94:0/3140220102 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559dcd56c6a0 con 0x559dcd6440c0 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.300+0000 7f41a2ffd640 1 -- 172.21.15.94:0/3140220102 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41b400cd60 con 0x559dcd6440c0 2024-02-17T12:01:16.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.300+0000 7f41a2ffd640 1 -- 172.21.15.94:0/3140220102 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f41b401a750 con 0x559dcd6440c0 2024-02-17T12:01:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.300+0000 7f41cc827740 10 client.25850 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.300+0000 7f41cc827740 10 client.25850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.300+0000 7f41cc827740 20 client.25850 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.300+0000 7f41cc827740 1 --2- 172.21.15.94:0/3140220102 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559dcd6c71d0 0x559dcd6c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.300+0000 7f41cc827740 4 client.25850 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:01:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.300+0000 7f41cc827740 1 -- 172.21.15.94:0/3140220102 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559dcd2e8720 con 0x559dcd6c71d0 2024-02-17T12:01:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.300+0000 7f41c2ffd640 1 --2- 172.21.15.94:0/3140220102 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559dcd6c71d0 0x559dcd6c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.301+0000 7f41a2ffd640 10 client.25850 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.301+0000 7f41c2ffd640 1 --2- 172.21.15.94:0/3140220102 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559dcd6c71d0 0x559dcd6c95f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b39ee9123d4df25 server_cookie=ec2167d4d9cdc6eb in_seq=0 out_seq=0 2024-02-17T12:01:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.301+0000 7f41a2ffd640 1 -- 172.21.15.94:0/3140220102 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559dcd2e8720 con 0x559dcd6c71d0 2024-02-17T12:01:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.301+0000 7f41a2ffd640 10 client.25850 handle_command_reply: tid=0 2024-02-17T12:01:16.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.301+0000 7f41cc827740 1 client.25850 shutdown 2024-02-17T12:01:16.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.302+0000 7f41cc827740 1 -- 172.21.15.94:0/3140220102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd6440c0 msgr2=0x559dcd654180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.302+0000 7f41cc827740 1 --2- 172.21.15.94:0/3140220102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd6440c0 0x559dcd654180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f41b400a930 tx=0x7f41b4011f90 comp rx=0 tx=0).stop 2024-02-17T12:01:16.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.302+0000 7f41cc827740 1 -- 172.21.15.94:0/3140220102 shutdown_connections 2024-02-17T12:01:16.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.302+0000 7f41cc827740 1 --2- 172.21.15.94:0/3140220102 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559dcd6c71d0 0x559dcd6c95f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.302+0000 7f41cc827740 1 --2- 172.21.15.94:0/3140220102 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559dcd6bee40 0x559dcd6bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.302+0000 7f41cc827740 1 --2- 172.21.15.94:0/3140220102 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559dcd655fc0 0x559dcd6546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.302+0000 7f41cc827740 1 --2- 172.21.15.94:0/3140220102 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559dcd6440c0 0x559dcd654180 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.302+0000 7f41cc827740 1 -- 172.21.15.94:0/3140220102 >> 172.21.15.94:0/3140220102 conn(0x559dcd63e990 msgr2=0x559dcd63d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:16.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.302+0000 7f41a2ffd640 0 client.25850 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:16.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.302+0000 7f41cc827740 1 -- 172.21.15.94:0/3140220102 shutdown_connections 2024-02-17T12:01:16.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.302+0000 7f41cc827740 1 -- 172.21.15.94:0/3140220102 wait complete. 2024-02-17T12:01:16.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.302+0000 7f41cc827740 20 client.25850 trim_cache size 0 max 16384 2024-02-17T12:01:16.305 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:01:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:01:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:01:16.305 INFO:teuthology.orchestra.run.smithi094.stdout: "0b7e3ea9-ba97-49b3-b9de-e209d24525f1": { 2024-02-17T12:01:16.306 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:01:16.306 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "0b7e3ea9-ba97-49b3-b9de-e209d24525f1", 2024-02-17T12:01:16.306 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:01:16.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:01:16.306 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:01:16.306 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:01:16.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.303+0000 7f41c9b77640 1 -- 172.21.15.94:0/2803027127 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f41ac0a2400 msgr2=0x7f41ac0a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.303+0000 7f41c9b77640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f41ac0a2400 0x7f41ac0a48c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x559dcd63fc70 tx=0x7f41b8009110 comp rx=0 tx=0).stop 2024-02-17T12:01:16.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.303+0000 7f41c9b77640 1 -- 172.21.15.94:0/2803027127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41c41997c0 msgr2=0x7f41c4199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:16.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.303+0000 7f41c9b77640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41c41997c0 0x7f41c4199c40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f41bc009dc0 tx=0x7f41bc0329d0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.303+0000 7f41c9b77640 1 -- 172.21.15.94:0/2803027127 shutdown_connections 2024-02-17T12:01:16.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.303+0000 7f41c9b77640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f41ac0a2400 0x7f41ac0a48c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.303+0000 7f41c9b77640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41c41997c0 0x7f41c4199c40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.303+0000 7f41c9b77640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41c41064d0 0x7f41c419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.303+0000 7f41c9b77640 1 --2- 172.21.15.94:0/2803027127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f41c4069a60 0x7f41c419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:16.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.303+0000 7f41c9b77640 1 -- 172.21.15.94:0/2803027127 >> 172.21.15.94:0/2803027127 conn(0x7f41c4077170 msgr2=0x7f41c40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:16.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.304+0000 7f41c9b77640 1 -- 172.21.15.94:0/2803027127 shutdown_connections 2024-02-17T12:01:16.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:16.304+0000 7f41c9b77640 1 -- 172.21.15.94:0/2803027127 wait complete. 2024-02-17T12:01:16.319 INFO:tasks.cephfs.filesystem:scrub status for tag:0b7e3ea9-ba97-49b3-b9de-e209d24525f1 - {'path': '/', 'tag': '0b7e3ea9-ba97-49b3-b9de-e209d24525f1', 'options': 'recursive,force'} 2024-02-17T12:01:17.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:17 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:01:17.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:17 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:01:17.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:17 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:01:17.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:17 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.189:6834/667623913,v1:172.21.15.189:6835/667623913] up:boot 2024-02-17T12:01:17.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:17 smithi094 ceph-mon[32499]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T12:01:17.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:17 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T12:01:17.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:17 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:01:17.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:17 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:01:17.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:17 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:01:17.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:17 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:01:17.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:17 smithi094 ceph-mon[32499]: pgmap v98: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 39 MiB/s wr, 1.37k op/s 2024-02-17T12:01:17.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:17 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:01:17.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:17 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:01:17.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:17 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:01:17.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:17 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.189:6834/667623913,v1:172.21.15.189:6835/667623913] up:boot 2024-02-17T12:01:17.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:17 smithi189 ceph-mon[36588]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T12:01:17.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:17 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T12:01:17.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:17 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:01:17.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:17 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:01:17.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:17 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:01:17.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:17 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:01:17.652 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:17 smithi189 ceph-mon[36588]: pgmap v98: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 39 MiB/s wr, 1.37k op/s 2024-02-17T12:01:17.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:17 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:01:17.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:17 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:01:17.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:17 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:01:17.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:17 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.189:6834/667623913,v1:172.21.15.189:6835/667623913] up:boot 2024-02-17T12:01:17.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:17 smithi125 ceph-mon[36819]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T12:01:17.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:17 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-17T12:01:17.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:17 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:01:17.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:17 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:01:17.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:17 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:01:17.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:17 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:01:17.685 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:17 smithi125 ceph-mon[36819]: pgmap v98: 97 pgs: 97 active+clean; 2.2 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 39 MiB/s wr, 1.37k op/s 2024-02-17T12:01:18.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:18 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:01:18.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:18 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:01:18.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:18 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:01:19.651 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:19 smithi189 ceph-mon[36588]: pgmap v99: 97 pgs: 97 active+clean; 2.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 60 MiB/s wr, 1.86k op/s 2024-02-17T12:01:19.684 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:19 smithi125 ceph-mon[36819]: pgmap v99: 97 pgs: 97 active+clean; 2.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 60 MiB/s wr, 1.86k op/s 2024-02-17T12:01:19.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:19 smithi094 ceph-mon[32499]: pgmap v99: 97 pgs: 97 active+clean; 2.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 60 MiB/s wr, 1.86k op/s 2024-02-17T12:01:20.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 180.0 s, 3328.2 tps, lat 29.898 ms stddev 38.542 2024-02-17T12:01:22.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:21 smithi189 ceph-mon[36588]: pgmap v100: 97 pgs: 97 active+clean; 2.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 27 MiB/s wr, 820 op/s 2024-02-17T12:01:22.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:21 smithi125 ceph-mon[36819]: pgmap v100: 97 pgs: 97 active+clean; 2.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 27 MiB/s wr, 820 op/s 2024-02-17T12:01:22.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:21 smithi094 ceph-mon[32499]: pgmap v100: 97 pgs: 97 active+clean; 2.3 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 160 KiB/s rd, 27 MiB/s wr, 820 op/s 2024-02-17T12:01:24.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:23 smithi189 ceph-mon[36588]: pgmap v101: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 42 MiB/s wr, 1.12k op/s 2024-02-17T12:01:24.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:23 smithi125 ceph-mon[36819]: pgmap v101: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 42 MiB/s wr, 1.12k op/s 2024-02-17T12:01:24.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:23 smithi094 ceph-mon[32499]: pgmap v101: 97 pgs: 97 active+clean; 2.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 42 MiB/s wr, 1.12k op/s 2024-02-17T12:01:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 185.0 s, 3817.4 tps, lat 26.284 ms stddev 32.567 2024-02-17T12:01:26.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:25 smithi189 ceph-mon[36588]: pgmap v102: 97 pgs: 97 active+clean; 2.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 43 MiB/s wr, 1.01k op/s 2024-02-17T12:01:26.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:25 smithi125 ceph-mon[36819]: pgmap v102: 97 pgs: 97 active+clean; 2.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 43 MiB/s wr, 1.01k op/s 2024-02-17T12:01:26.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:25 smithi094 ceph-mon[32499]: pgmap v102: 97 pgs: 97 active+clean; 2.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 43 MiB/s wr, 1.01k op/s 2024-02-17T12:01:28.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:27 smithi189 ceph-mon[36588]: pgmap v103: 97 pgs: 97 active+clean; 2.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 42 MiB/s wr, 946 op/s 2024-02-17T12:01:28.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:27 smithi125 ceph-mon[36819]: pgmap v103: 97 pgs: 97 active+clean; 2.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 42 MiB/s wr, 946 op/s 2024-02-17T12:01:28.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:27 smithi094 ceph-mon[32499]: pgmap v103: 97 pgs: 97 active+clean; 2.4 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 42 MiB/s wr, 946 op/s 2024-02-17T12:01:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.786+0000 7f0fe24c0640 1 -- 172.21.15.94:0/3082067693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0fdc100150 msgr2=0x7f0fdc10c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.786+0000 7f0fe24c0640 1 --2- 172.21.15.94:0/3082067693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0fdc100150 0x7f0fdc10c660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4009be0 tx=0x7f0fd402bef0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.786+0000 7f0fe24c0640 1 -- 172.21.15.94:0/3082067693 shutdown_connections 2024-02-17T12:01:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.786+0000 7f0fe24c0640 1 --2- 172.21.15.94:0/3082067693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0fdc100150 0x7f0fdc10c660 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.786+0000 7f0fe24c0640 1 --2- 172.21.15.94:0/3082067693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fdc0ff790 0x7f0fdc0ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.786+0000 7f0fe24c0640 1 --2- 172.21.15.94:0/3082067693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0fdc1057a0 0x7f0fdc105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.786+0000 7f0fe24c0640 1 -- 172.21.15.94:0/3082067693 >> 172.21.15.94:0/3082067693 conn(0x7f0fdc076610 msgr2=0x7f0fdc0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.787+0000 7f0fe24c0640 1 -- 172.21.15.94:0/3082067693 shutdown_connections 2024-02-17T12:01:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.787+0000 7f0fe24c0640 1 -- 172.21.15.94:0/3082067693 wait complete. 2024-02-17T12:01:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.788+0000 7f0fe24c0640 1 Processor -- start 2024-02-17T12:01:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.788+0000 7f0fe24c0640 1 -- start start 2024-02-17T12:01:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.788+0000 7f0fe24c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0fdc0ff790 0x7f0fdc1a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.788+0000 7f0fe24c0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0fdc100150 0x7f0fdc1a3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.788+0000 7f0fe24c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fdc1057a0 0x7f0fdc10b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.788+0000 7f0fe24c0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc10bb20 con 0x7f0fdc1057a0 2024-02-17T12:01:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.788+0000 7f0fe24c0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc10bc90 con 0x7f0fdc0ff790 2024-02-17T12:01:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.788+0000 7f0fe24c0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fdc10be50 con 0x7f0fdc100150 2024-02-17T12:01:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.788+0000 7f0fe0a5d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fdc1057a0 0x7f0fdc10b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.788+0000 7f0fe0a5d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fdc1057a0 0x7f0fdc10b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33316/0 (socket says 172.21.15.94:33316) 2024-02-17T12:01:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.788+0000 7f0fe0a5d640 1 -- 172.21.15.94:0/1374635502 learned_addr learned my addr 172.21.15.94:0/1374635502 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.788+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0fdc100150 0x7f0fdc1a3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.789+0000 7f0fdbfff640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0fdc0ff790 0x7f0fdc1a3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.789+0000 7f0fe0a5d640 1 -- 172.21.15.94:0/1374635502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0fdc100150 msgr2=0x7f0fdc1a3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.789+0000 7f0fe0a5d640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0fdc100150 0x7f0fdc1a3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.789+0000 7f0fe0a5d640 1 -- 172.21.15.94:0/1374635502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0fdc0ff790 msgr2=0x7f0fdc1a3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.789+0000 7f0fe0a5d640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0fdc0ff790 0x7f0fdc1a3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.789+0000 7f0fe0a5d640 1 -- 172.21.15.94:0/1374635502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc4009560 con 0x7f0fdc1057a0 2024-02-17T12:01:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.789+0000 7f0fe0a5d640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fdc1057a0 0x7f0fdc10b480 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4036040 tx=0x7f0fd40333e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.790+0000 7f0fd9ffb640 1 -- 172.21.15.94:0/1374635502 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4003b70 con 0x7f0fdc1057a0 2024-02-17T12:01:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.790+0000 7f0fd9ffb640 1 -- 172.21.15.94:0/1374635502 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fd4033810 con 0x7f0fdc1057a0 2024-02-17T12:01:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.790+0000 7f0fd9ffb640 1 -- 172.21.15.94:0/1374635502 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd403d850 con 0x7f0fdc1057a0 2024-02-17T12:01:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.790+0000 7f0fe24c0640 1 -- 172.21.15.94:0/1374635502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd4009840 con 0x7f0fdc1057a0 2024-02-17T12:01:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.790+0000 7f0fe24c0640 1 -- 172.21.15.94:0/1374635502 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fdc1aa7e0 con 0x7f0fdc1057a0 2024-02-17T12:01:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.794+0000 7f0fe516f740 1 Processor -- start 2024-02-17T12:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.794+0000 7f0fe516f740 1 -- start start 2024-02-17T12:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.794+0000 7f0fe516f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c1ad0 0x5559213c0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.794+0000 7f0fe516f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c0d60 0x5559213c11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.794+0000 7f0fe516f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.794+0000 7f0fe516f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55592132f9f0 con 0x5559213c1ad0 2024-02-17T12:01:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.794+0000 7f0fe516f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55592132cf20 con 0x5559213c0d60 2024-02-17T12:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.794+0000 7f0fe516f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555921328700 con 0x5559213ca2e0 2024-02-17T12:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.795+0000 7f0fd9ffb640 1 -- 172.21.15.94:0/1374635502 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0fd4003cd0 con 0x7f0fdc1057a0 2024-02-17T12:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.795+0000 7f0fdbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c0d60 0x5559213c11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.795+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.795+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44450/0 (socket says 172.21.15.94:44450) 2024-02-17T12:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.795+0000 7f0fd37fe640 1 -- 172.21.15.94:0/499810710 learned_addr learned my addr 172.21.15.94:0/499810710 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.795+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/499810710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c1ad0 0x5559213c0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.795+0000 7f0fd37fe640 1 -- 172.21.15.94:0/499810710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c0d60 msgr2=0x5559213c11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.795+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/499810710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c0d60 0x5559213c11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.795+0000 7f0fd37fe640 1 -- 172.21.15.94:0/499810710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c1ad0 msgr2=0x5559213c0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.795+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/499810710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c1ad0 0x5559213c0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.795+0000 7f0fd37fe640 1 -- 172.21.15.94:0/499810710 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559212be7b0 con 0x5559213ca2e0 2024-02-17T12:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.795+0000 7f0fd9ffb640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0fc80a24b0 0x7f0fc80a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.796+0000 7f0fd9ffb640 1 -- 172.21.15.94:0/1374635502 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0fd40f0600 con 0x7f0fdc1057a0 2024-02-17T12:01:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.796+0000 7f0fdbfff640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0fc80a24b0 0x7f0fc80a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.796+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/499810710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213cc6d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4011950 tx=0x7f0fc4005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3dfdb7d0c092ed4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.796+0000 7f0fd27fc640 1 -- 172.21.15.94:0/499810710 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4004020 con 0x5559213ca2e0 2024-02-17T12:01:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.796+0000 7f0fd27fc640 1 -- 172.21.15.94:0/499810710 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fc4005500 con 0x5559213ca2e0 2024-02-17T12:01:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.796+0000 7f0fd27fc640 1 -- 172.21.15.94:0/499810710 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4004b90 con 0x5559213ca2e0 2024-02-17T12:01:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.796+0000 7f0fdbfff640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0fc80a24b0 0x7f0fc80a4970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5559213c2230 tx=0x7f0fcc005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.797+0000 7f0fe516f740 1 -- 172.21.15.94:0/499810710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 msgr2=0x5559213cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.797+0000 7f0fe516f740 1 --2- 172.21.15.94:0/499810710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213cc6d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4011950 tx=0x7f0fc4005a00 comp rx=0 tx=0).stop 2024-02-17T12:01:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.798+0000 7f0fe516f740 1 -- 172.21.15.94:0/499810710 shutdown_connections 2024-02-17T12:01:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.798+0000 7f0fe516f740 1 --2- 172.21.15.94:0/499810710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213cc6d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.798+0000 7f0fe516f740 1 --2- 172.21.15.94:0/499810710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c0d60 0x5559213c11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.798+0000 7f0fe516f740 1 --2- 172.21.15.94:0/499810710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c1ad0 0x5559213c0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.798+0000 7f0fe516f740 1 -- 172.21.15.94:0/499810710 >> 172.21.15.94:0/499810710 conn(0x555921388550 msgr2=0x5559213bba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.799+0000 7f0fe516f740 1 -- 172.21.15.94:0/499810710 shutdown_connections 2024-02-17T12:01:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.799+0000 7f0fe516f740 1 -- 172.21.15.94:0/499810710 wait complete. 2024-02-17T12:01:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.799+0000 7f0fe516f740 1 -- start start 2024-02-17T12:01:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.800+0000 7f0fe516f740 1 Processor -- start 2024-02-17T12:01:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.800+0000 7f0fe516f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c0d60 0x5559213c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.800+0000 7f0fe516f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c1ad0 0x5559213c7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.800+0000 7f0fe516f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.800+0000 7f0fe516f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559212cdbf0 con 0x5559213c0d60 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.800+0000 7f0fe516f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555920ffc250 con 0x5559213c1ad0 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.800+0000 7f0fe516f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559212ae200 con 0x5559213ca2e0 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.800+0000 7f0fe516f740 10 client.0 _ll_register_callbacks cb 0x55592131cd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.800+0000 7f0fdbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c0d60 0x5559213c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.800+0000 7f0fdbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c0d60 0x5559213c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33330/0 (socket says 172.21.15.94:33330) 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.800+0000 7f0fdbfff640 1 -- 172.21.15.94:0/3743513098 learned_addr learned my addr 172.21.15.94:0/3743513098 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.800+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/3743513098 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c1ad0 0x5559213c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.801+0000 7f0fe0a5d640 1 --2- 172.21.15.94:0/3743513098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.801+0000 7f0fd37fe640 1 -- 172.21.15.94:0/3743513098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 msgr2=0x5559213c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.801+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/3743513098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.801+0000 7f0fd37fe640 1 -- 172.21.15.94:0/3743513098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c0d60 msgr2=0x5559213c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.801+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/3743513098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c0d60 0x5559213c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.801+0000 7f0fd37fe640 1 -- 172.21.15.94:0/3743513098 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc4019040 con 0x5559213c1ad0 2024-02-17T12:01:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.801+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/3743513098 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c1ad0 0x5559213c7900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009530 tx=0x7f0fc4009dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1037f0bc505c5e47 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.802+0000 7f0fbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.802+0000 7f0fbb7fe640 1 -- 172.21.15.94:0/3743513098 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4024070 con 0x5559213c1ad0 2024-02-17T12:01:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.802+0000 7f0fbb7fe640 1 -- 172.21.15.94:0/3743513098 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0fc4000e40 con 0x5559213c1ad0 2024-02-17T12:01:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.802+0000 7f0fe516f740 1 -- 172.21.15.94:0/3743513098 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555921028600 con 0x5559213c1ad0 2024-02-17T12:01:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.802+0000 7f0fbb7fe640 1 -- 172.21.15.94:0/3743513098 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fc40046c0 con 0x5559213c1ad0 2024-02-17T12:01:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.802+0000 7f0fe516f740 10 client.25862 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.803+0000 7f0fe516f740 1 -- 172.21.15.94:0/3743513098 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555921326d80 con 0x5559213c1ad0 2024-02-17T12:01:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.803+0000 7f0fbb7fe640 1 -- 172.21.15.94:0/3743513098 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555921028600 con 0x5559213c1ad0 2024-02-17T12:01:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.803+0000 7f0fbb7fe640 1 -- 172.21.15.94:0/3743513098 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0fc4033420 con 0x5559213c1ad0 2024-02-17T12:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.803+0000 7f0fe516f740 10 client.25862 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.803+0000 7f0fe516f740 10 client.25862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.803+0000 7f0fe516f740 20 client.25862 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.803+0000 7f0fe516f740 1 --2- 172.21.15.94:0/3743513098 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5559213eb6e0 0x5559213edad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.803+0000 7f0fe516f740 4 client.25862 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.803+0000 7f0fe516f740 1 -- 172.21.15.94:0/3743513098 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559210791b0 con 0x5559213eb6e0 2024-02-17T12:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.803+0000 7f0fdbfff640 1 --2- 172.21.15.94:0/3743513098 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5559213eb6e0 0x5559213edad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.804+0000 7f0fbb7fe640 10 client.25862 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.804+0000 7f0fdbfff640 1 --2- 172.21.15.94:0/3743513098 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5559213eb6e0 0x5559213edad0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4be3358d3511c9 server_cookie=f2d3e4ec1f68bb6f in_seq=0 out_seq=0 2024-02-17T12:01:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.805+0000 7f0fbb7fe640 1 -- 172.21.15.94:0/3743513098 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5559210791b0 con 0x5559213eb6e0 2024-02-17T12:01:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.805+0000 7f0fbb7fe640 10 client.25862 handle_command_reply: tid=0 2024-02-17T12:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.805+0000 7f0fe516f740 1 client.25862 shutdown 2024-02-17T12:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.806+0000 7f0fe516f740 1 -- 172.21.15.94:0/3743513098 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c1ad0 msgr2=0x5559213c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.806+0000 7f0fe516f740 1 --2- 172.21.15.94:0/3743513098 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c1ad0 0x5559213c7900 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009530 tx=0x7f0fc4009dd0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.806+0000 7f0fe516f740 1 -- 172.21.15.94:0/3743513098 shutdown_connections 2024-02-17T12:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.806+0000 7f0fe516f740 1 --2- 172.21.15.94:0/3743513098 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5559213eb6e0 0x5559213edad0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.806+0000 7f0fe516f740 1 --2- 172.21.15.94:0/3743513098 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.806+0000 7f0fe516f740 1 --2- 172.21.15.94:0/3743513098 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c1ad0 0x5559213c7900 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.806+0000 7f0fe516f740 1 --2- 172.21.15.94:0/3743513098 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c0d60 0x5559213c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.806+0000 7f0fe516f740 1 -- 172.21.15.94:0/3743513098 >> 172.21.15.94:0/3743513098 conn(0x555921388550 msgr2=0x5559213bd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.806+0000 7f0fbb7fe640 0 client.25862 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.806+0000 7f0fe516f740 1 -- 172.21.15.94:0/3743513098 shutdown_connections 2024-02-17T12:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.806+0000 7f0fe516f740 1 -- 172.21.15.94:0/3743513098 wait complete. 2024-02-17T12:01:28.806 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.806+0000 7f0fe516f740 20 client.25862 trim_cache size 0 max 16384 2024-02-17T12:01:28.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.827+0000 7f0fe516f740 1 Processor -- start 2024-02-17T12:01:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.827+0000 7f0fe516f740 1 -- start start 2024-02-17T12:01:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fe516f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213c0d60 0x5559213c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fe516f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c1ad0 0x5559213c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fe516f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213ca2e0 0x5559213c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fe516f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559210791e0 con 0x5559213ca2e0 2024-02-17T12:01:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fe516f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555921028630 con 0x5559213c1ad0 2024-02-17T12:01:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fe516f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559213c8390 con 0x5559213c0d60 2024-02-17T12:01:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fe0a5d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213ca2e0 0x5559213c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fe0a5d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213ca2e0 0x5559213c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33342/0 (socket says 172.21.15.94:33342) 2024-02-17T12:01:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fe0a5d640 1 -- 172.21.15.94:0/658551966 learned_addr learned my addr 172.21.15.94:0/658551966 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c1ad0 0x5559213c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fdbfff640 1 --2- 172.21.15.94:0/658551966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213c0d60 0x5559213c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fd37fe640 1 -- 172.21.15.94:0/658551966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213c0d60 msgr2=0x5559213c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/658551966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213c0d60 0x5559213c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fd37fe640 1 -- 172.21.15.94:0/658551966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213ca2e0 msgr2=0x5559213c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/658551966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213ca2e0 0x5559213c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fd37fe640 1 -- 172.21.15.94:0/658551966 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559212c37b0 con 0x5559213c1ad0 2024-02-17T12:01:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fdbfff640 1 --2- 172.21.15.94:0/658551966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213c0d60 0x5559213c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.828+0000 7f0fe0a5d640 1 --2- 172.21.15.94:0/658551966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213ca2e0 0x5559213c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.829+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/658551966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c1ad0 0x5559213c5c90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5559213c8990 tx=0x7f0fc4033bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22b020a827a00f71 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.829+0000 7f0fd1ffb640 1 -- 172.21.15.94:0/658551966 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4024040 con 0x5559213c1ad0 2024-02-17T12:01:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.829+0000 7f0fd1ffb640 1 -- 172.21.15.94:0/658551966 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fc4005970 con 0x5559213c1ad0 2024-02-17T12:01:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.829+0000 7f0fd1ffb640 1 -- 172.21.15.94:0/658551966 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4004660 con 0x5559213c1ad0 2024-02-17T12:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.830+0000 7f0fe516f740 1 -- 172.21.15.94:0/658551966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c1ad0 msgr2=0x5559213c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.830+0000 7f0fe516f740 1 --2- 172.21.15.94:0/658551966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c1ad0 0x5559213c5c90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5559213c8990 tx=0x7f0fc4033bc0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.830+0000 7f0fe516f740 1 -- 172.21.15.94:0/658551966 shutdown_connections 2024-02-17T12:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.830+0000 7f0fe516f740 1 --2- 172.21.15.94:0/658551966 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213ca2e0 0x5559213c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.830+0000 7f0fe516f740 1 --2- 172.21.15.94:0/658551966 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c1ad0 0x5559213c5c90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.830+0000 7f0fe516f740 1 --2- 172.21.15.94:0/658551966 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213c0d60 0x5559213c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.830+0000 7f0fe516f740 1 -- 172.21.15.94:0/658551966 >> 172.21.15.94:0/658551966 conn(0x555921388550 msgr2=0x5559213bb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.830+0000 7f0fe516f740 1 -- 172.21.15.94:0/658551966 shutdown_connections 2024-02-17T12:01:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.831+0000 7f0fe516f740 1 -- 172.21.15.94:0/658551966 wait complete. 2024-02-17T12:01:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.831+0000 7f0fe516f740 1 -- start start 2024-02-17T12:01:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.831+0000 7f0fe516f740 1 Processor -- start 2024-02-17T12:01:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.832+0000 7f0fe516f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c0d60 0x5559213e1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.832+0000 7f0fe516f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c1ad0 0x5559213e22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.832+0000 7f0fe516f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213e0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.832+0000 7f0fe516f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559213e0830 con 0x5559213c1ad0 2024-02-17T12:01:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.832+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c1ad0 0x5559213e22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.832+0000 7f0fdbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c0d60 0x5559213e1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.832+0000 7f0fe516f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559213e09a0 con 0x5559213c0d60 2024-02-17T12:01:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.832+0000 7f0fe516f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559213e0b60 con 0x5559213ca2e0 2024-02-17T12:01:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.832+0000 7f0fd37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c1ad0 0x5559213e22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33358/0 (socket says 172.21.15.94:33358) 2024-02-17T12:01:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.832+0000 7f0fdbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c0d60 0x5559213e1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53068/0 (socket says 172.21.15.94:53068) 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.832+0000 7f0fd37fe640 1 -- 172.21.15.94:0/1398344011 learned_addr learned my addr 172.21.15.94:0/1398344011 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.832+0000 7f0fe516f740 10 client.0 _ll_register_callbacks cb 0x7f0fc4009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.832+0000 7f0fe0a5d640 1 --2- 172.21.15.94:0/1398344011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213e0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.833+0000 7f0fdbfff640 1 -- 172.21.15.94:0/1398344011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 msgr2=0x5559213e0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.833+0000 7f0fdbfff640 1 --2- 172.21.15.94:0/1398344011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213e0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.833+0000 7f0fdbfff640 1 -- 172.21.15.94:0/1398344011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c1ad0 msgr2=0x5559213e22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.833+0000 7f0fdbfff640 1 --2- 172.21.15.94:0/1398344011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c1ad0 0x5559213e22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.833+0000 7f0fdbfff640 1 -- 172.21.15.94:0/1398344011 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fcc0041c0 con 0x5559213c0d60 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.833+0000 7f0fdbfff640 1 --2- 172.21.15.94:0/1398344011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c0d60 0x5559213e1db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc005850 tx=0x7f0fcc016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6c01bd481e9129 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.833+0000 7f0fbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.833+0000 7f0fbb7fe640 1 -- 172.21.15.94:0/1398344011 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fcc024040 con 0x5559213c0d60 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.833+0000 7f0fe516f740 1 -- 172.21.15.94:0/1398344011 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fc4005500 con 0x5559213c0d60 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.833+0000 7f0fbb7fe640 1 -- 172.21.15.94:0/1398344011 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0fcc003ac0 con 0x5559213c0d60 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.833+0000 7f0fbb7fe640 1 -- 172.21.15.94:0/1398344011 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fcc003c80 con 0x5559213c0d60 2024-02-17T12:01:28.835 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.833+0000 7f0fe516f740 10 client.25868 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.833+0000 7f0fe516f740 1 -- 172.21.15.94:0/1398344011 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f0fdc1aa7e0 con 0x5559213c0d60 2024-02-17T12:01:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.834+0000 7f0fbb7fe640 1 -- 172.21.15.94:0/1398344011 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0fcc03e020 con 0x5559213c0d60 2024-02-17T12:01:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.834+0000 7f0fbb7fe640 1 -- 172.21.15.94:0/1398344011 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0fcc01f530 con 0x5559213c0d60 2024-02-17T12:01:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.834+0000 7f0fe516f740 10 client.25868 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.834+0000 7f0fe516f740 10 client.25868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.834+0000 7f0fe516f740 20 client.25868 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.834+0000 7f0fe516f740 1 --2- 172.21.15.94:0/1398344011 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555921442cf0 0x555921445110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.834+0000 7f0fe516f740 4 client.25868 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:01:28.836 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.834+0000 7f0fe516f740 1 -- 172.21.15.94:0/1398344011 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555921065550 con 0x555921442cf0 2024-02-17T12:01:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.835+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/1398344011 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555921442cf0 0x555921445110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.835+0000 7f0fd37fe640 1 --2- 172.21.15.94:0/1398344011 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555921442cf0 0x555921445110 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709fc06bcda69efd server_cookie=1db48033f3c9d6c in_seq=0 out_seq=0 2024-02-17T12:01:28.837 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.835+0000 7f0fbb7fe640 10 client.25868 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.864+0000 7f0fbb7fe640 1 -- 172.21.15.94:0/1398344011 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555921065550 con 0x555921442cf0 2024-02-17T12:01:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.864+0000 7f0fbb7fe640 10 client.25868 handle_command_reply: tid=0 2024-02-17T12:01:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.864+0000 7f0fe516f740 1 client.25868 shutdown 2024-02-17T12:01:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.864+0000 7f0fe516f740 1 -- 172.21.15.94:0/1398344011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c0d60 msgr2=0x5559213e1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.864+0000 7f0fe516f740 1 --2- 172.21.15.94:0/1398344011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c0d60 0x5559213e1db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc005850 tx=0x7f0fcc016000 comp rx=0 tx=0).stop 2024-02-17T12:01:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.864+0000 7f0fe516f740 1 -- 172.21.15.94:0/1398344011 shutdown_connections 2024-02-17T12:01:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.864+0000 7f0fe516f740 1 --2- 172.21.15.94:0/1398344011 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555921442cf0 0x555921445110 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.864+0000 7f0fe516f740 1 --2- 172.21.15.94:0/1398344011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559213ca2e0 0x5559213e0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.864+0000 7f0fe516f740 1 --2- 172.21.15.94:0/1398344011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559213c1ad0 0x5559213e22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.864+0000 7f0fe516f740 1 --2- 172.21.15.94:0/1398344011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559213c0d60 0x5559213e1db0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.864+0000 7f0fe516f740 1 -- 172.21.15.94:0/1398344011 >> 172.21.15.94:0/1398344011 conn(0x555921388550 msgr2=0x5559213c5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.864+0000 7f0fbb7fe640 0 client.25868 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.864+0000 7f0fe516f740 1 -- 172.21.15.94:0/1398344011 shutdown_connections 2024-02-17T12:01:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.865+0000 7f0fe516f740 1 -- 172.21.15.94:0/1398344011 wait complete. 2024-02-17T12:01:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.865+0000 7f0fe516f740 20 client.25868 trim_cache size 0 max 16384 2024-02-17T12:01:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:01:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:01:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:01:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:01:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.865+0000 7f0fe24c0640 1 -- 172.21.15.94:0/1374635502 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0fc80a24b0 msgr2=0x7f0fc80a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:28.866 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.865+0000 7f0fe24c0640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0fc80a24b0 0x7f0fc80a4970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5559213c2230 tx=0x7f0fcc005f90 comp rx=0 tx=0).stop 2024-02-17T12:01:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.865+0000 7f0fe24c0640 1 -- 172.21.15.94:0/1374635502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fdc1057a0 msgr2=0x7f0fdc10b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.865+0000 7f0fe24c0640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fdc1057a0 0x7f0fdc10b480 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4036040 tx=0x7f0fd40333e0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.865+0000 7f0fe24c0640 1 -- 172.21.15.94:0/1374635502 shutdown_connections 2024-02-17T12:01:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.865+0000 7f0fe24c0640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0fc80a24b0 0x7f0fc80a4970 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.865+0000 7f0fe24c0640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0fdc1057a0 0x7f0fdc10b480 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.865+0000 7f0fe24c0640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0fdc100150 0x7f0fdc1a3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.865+0000 7f0fe24c0640 1 --2- 172.21.15.94:0/1374635502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0fdc0ff790 0x7f0fdc1a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.865+0000 7f0fe24c0640 1 -- 172.21.15.94:0/1374635502 >> 172.21.15.94:0/1374635502 conn(0x7f0fdc076610 msgr2=0x7f0fdc102730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.866+0000 7f0fe24c0640 1 -- 172.21.15.94:0/1374635502 shutdown_connections 2024-02-17T12:01:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:28.866+0000 7f0fe24c0640 1 -- 172.21.15.94:0/1374635502 wait complete. 2024-02-17T12:01:30.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:29 smithi189 ceph-mon[36588]: pgmap v104: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 58 MiB/s wr, 1.37k op/s 2024-02-17T12:01:30.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:29 smithi125 ceph-mon[36819]: pgmap v104: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 58 MiB/s wr, 1.37k op/s 2024-02-17T12:01:30.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:29 smithi094 ceph-mon[32499]: pgmap v104: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 58 MiB/s wr, 1.37k op/s 2024-02-17T12:01:30.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 190.0 s, 3928.8 tps, lat 25.555 ms stddev 30.431 2024-02-17T12:01:32.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:31 smithi189 ceph-mon[36588]: pgmap v105: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 886 op/s 2024-02-17T12:01:32.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:31 smithi125 ceph-mon[36819]: pgmap v105: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 886 op/s 2024-02-17T12:01:32.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:31 smithi094 ceph-mon[32499]: pgmap v105: 97 pgs: 97 active+clean; 2.5 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 37 MiB/s wr, 886 op/s 2024-02-17T12:01:34.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:33 smithi189 ceph-mon[36588]: pgmap v106: 97 pgs: 97 active+clean; 2.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 57 MiB/s wr, 1.68k op/s 2024-02-17T12:01:34.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:33 smithi125 ceph-mon[36819]: pgmap v106: 97 pgs: 97 active+clean; 2.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 57 MiB/s wr, 1.68k op/s 2024-02-17T12:01:34.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:33 smithi094 ceph-mon[32499]: pgmap v106: 97 pgs: 97 active+clean; 2.7 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 57 MiB/s wr, 1.68k op/s 2024-02-17T12:01:35.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 195.0 s, 3359.8 tps, lat 29.771 ms stddev 71.334 2024-02-17T12:01:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:35 smithi189 ceph-mon[36588]: pgmap v107: 97 pgs: 97 active+clean; 2.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 MiB/s wr, 1.94k op/s 2024-02-17T12:01:36.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:35 smithi125 ceph-mon[36819]: pgmap v107: 97 pgs: 97 active+clean; 2.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 MiB/s wr, 1.94k op/s 2024-02-17T12:01:36.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:35 smithi094 ceph-mon[32499]: pgmap v107: 97 pgs: 97 active+clean; 2.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 MiB/s wr, 1.94k op/s 2024-02-17T12:01:38.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:37 smithi189 ceph-mon[36588]: pgmap v108: 97 pgs: 97 active+clean; 2.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 51 MiB/s wr, 1.78k op/s 2024-02-17T12:01:38.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:37 smithi125 ceph-mon[36819]: pgmap v108: 97 pgs: 97 active+clean; 2.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 51 MiB/s wr, 1.78k op/s 2024-02-17T12:01:38.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:37 smithi094 ceph-mon[32499]: pgmap v108: 97 pgs: 97 active+clean; 2.8 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 51 MiB/s wr, 1.78k op/s 2024-02-17T12:01:40.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:39 smithi189 ceph-mon[36588]: pgmap v109: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 58 MiB/s wr, 2.12k op/s 2024-02-17T12:01:40.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:39 smithi125 ceph-mon[36819]: pgmap v109: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 58 MiB/s wr, 2.12k op/s 2024-02-17T12:01:40.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:39 smithi094 ceph-mon[32499]: pgmap v109: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 58 MiB/s wr, 2.12k op/s 2024-02-17T12:01:40.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 200.0 s, 3866.2 tps, lat 25.835 ms stddev 30.068 2024-02-17T12:01:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.984+0000 7f75f9cae640 1 -- 172.21.15.94:0/1041737243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f40ff810 msgr2=0x7f75f40ffc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.984+0000 7f75f9cae640 1 --2- 172.21.15.94:0/1041737243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f40ff810 0x7f75f40ffc90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009860 tx=0x7f75e802bb70 comp rx=0 tx=0).stop 2024-02-17T12:01:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.984+0000 7f75f9cae640 1 -- 172.21.15.94:0/1041737243 shutdown_connections 2024-02-17T12:01:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.984+0000 7f75f9cae640 1 --2- 172.21.15.94:0/1041737243 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75f41001d0 0x7f75f410c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.984+0000 7f75f9cae640 1 --2- 172.21.15.94:0/1041737243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f40ff810 0x7f75f40ffc90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.984+0000 7f75f9cae640 1 --2- 172.21.15.94:0/1041737243 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75f41057d0 0x7f75f4105bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:41.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.984+0000 7f75f9cae640 1 -- 172.21.15.94:0/1041737243 >> 172.21.15.94:0/1041737243 conn(0x7f75f40765a0 msgr2=0x7f75f40fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.984+0000 7f75f9cae640 1 -- 172.21.15.94:0/1041737243 shutdown_connections 2024-02-17T12:01:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.984+0000 7f75f9cae640 1 -- 172.21.15.94:0/1041737243 wait complete. 2024-02-17T12:01:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.985+0000 7f75f9cae640 1 Processor -- start 2024-02-17T12:01:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.985+0000 7f75f9cae640 1 -- start start 2024-02-17T12:01:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f9cae640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75f40ff810 0x7f75f4072a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f9cae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f41001d0 0x7f75f410def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f9cae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75f41057d0 0x7f75f410e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75f41057d0 0x7f75f410e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75f41057d0 0x7f75f410e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57314/0 (socket says 172.21.15.94:57314) 2024-02-17T12:01:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f3fff640 1 -- 172.21.15.94:0/3496897915 learned_addr learned my addr 172.21.15.94:0/3496897915 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f9cae640 1 -- 172.21.15.94:0/3496897915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f4073210 con 0x7f75f41057d0 2024-02-17T12:01:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f9cae640 1 -- 172.21.15.94:0/3496897915 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f410e970 con 0x7f75f40ff810 2024-02-17T12:01:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f9cae640 1 -- 172.21.15.94:0/3496897915 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f410eb00 con 0x7f75f41001d0 2024-02-17T12:01:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f2ffd640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f41001d0 0x7f75f410def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f37fe640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75f40ff810 0x7f75f4072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f3fff640 1 -- 172.21.15.94:0/3496897915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f41001d0 msgr2=0x7f75f410def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f3fff640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f41001d0 0x7f75f410def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f3fff640 1 -- 172.21.15.94:0/3496897915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75f40ff810 msgr2=0x7f75f4072a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f3fff640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75f40ff810 0x7f75f4072a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f3fff640 1 -- 172.21.15.94:0/3496897915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec009900 con 0x7f75f41057d0 2024-02-17T12:01:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.986+0000 7f75f3fff640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75f41057d0 0x7f75f410e430 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00efc0 tx=0x7f75ec00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.987+0000 7f75f0ff9640 1 -- 172.21.15.94:0/3496897915 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec003970 con 0x7f75f41057d0 2024-02-17T12:01:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.987+0000 7f75f0ff9640 1 -- 172.21.15.94:0/3496897915 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75ec004020 con 0x7f75f41057d0 2024-02-17T12:01:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.987+0000 7f75f0ff9640 1 -- 172.21.15.94:0/3496897915 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec00b9b0 con 0x7f75f41057d0 2024-02-17T12:01:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.987+0000 7f75f9cae640 1 -- 172.21.15.94:0/3496897915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e8009490 con 0x7f75f41057d0 2024-02-17T12:01:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.987+0000 7f75f9cae640 1 -- 172.21.15.94:0/3496897915 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f41ae9d0 con 0x7f75f41057d0 2024-02-17T12:01:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.989+0000 7f75fc883740 1 Processor -- start 2024-02-17T12:01:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.991+0000 7f75f0ff9640 1 -- 172.21.15.94:0/3496897915 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f75ec00a040 con 0x7f75f41057d0 2024-02-17T12:01:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.991+0000 7f75fc883740 1 -- start start 2024-02-17T12:01:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.991+0000 7f75fc883740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da334f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.991+0000 7f75fc883740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334f9d00 0x55da334fa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.991+0000 7f75fc883740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.991+0000 7f75fc883740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da334689f0 con 0x55da334faa70 2024-02-17T12:01:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.991+0000 7f75fc883740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da33465f20 con 0x55da334f9d00 2024-02-17T12:01:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.991+0000 7f75fc883740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da33461700 con 0x55da3350ba90 2024-02-17T12:01:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.991+0000 7f75f2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da334f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.991+0000 7f75f2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da334f97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57328/0 (socket says 172.21.15.94:57328) 2024-02-17T12:01:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.992+0000 7f75f2ffd640 1 -- 172.21.15.94:0/3608634942 learned_addr learned my addr 172.21.15.94:0/3608634942 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.992+0000 7f75f2ffd640 1 --2- 172.21.15.94:0/3608634942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.992+0000 7f75f37fe640 1 --2- 172.21.15.94:0/3608634942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334f9d00 0x55da334fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.992+0000 7f75f2ffd640 1 -- 172.21.15.94:0/3608634942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334f9d00 msgr2=0x55da334fa180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.992+0000 7f75f2ffd640 1 --2- 172.21.15.94:0/3608634942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334f9d00 0x55da334fa180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.992+0000 7f75f2ffd640 1 -- 172.21.15.94:0/3608634942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 msgr2=0x55da334f97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.992+0000 7f75f2ffd640 1 --2- 172.21.15.94:0/3608634942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da334f97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.992+0000 7f75f2ffd640 1 -- 172.21.15.94:0/3608634942 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da333f77b0 con 0x55da3350ba90 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.992+0000 7f75f0ff9640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f75dc0a23d0 0x7f75dc0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.992+0000 7f75f2ffd640 1 --2- 172.21.15.94:0/3608634942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350de80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009860 tx=0x7f75e802bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cb30cc24b4a4b59 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.993+0000 7f75f0ff9640 1 -- 172.21.15.94:0/3496897915 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f75ec0ce090 con 0x7f75f41057d0 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.993+0000 7f75f37fe640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f75dc0a23d0 0x7f75dc0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.993+0000 7f75d9ffb640 1 -- 172.21.15.94:0/3608634942 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8034640 con 0x55da3350ba90 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.993+0000 7f75d9ffb640 1 -- 172.21.15.94:0/3608634942 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75e80044b0 con 0x55da3350ba90 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.993+0000 7f75d9ffb640 1 -- 172.21.15.94:0/3608634942 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e802c930 con 0x55da3350ba90 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.993+0000 7f75fc883740 1 -- 172.21.15.94:0/3608634942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 msgr2=0x55da3350de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.993+0000 7f75f37fe640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f75dc0a23d0 0x7f75dc0a4890 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55da334fb1d0 tx=0x7f75e400b410 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.993+0000 7f75fc883740 1 --2- 172.21.15.94:0/3608634942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350de80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009860 tx=0x7f75e802bb70 comp rx=0 tx=0).stop 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.994+0000 7f75fc883740 1 -- 172.21.15.94:0/3608634942 shutdown_connections 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.994+0000 7f75fc883740 1 --2- 172.21.15.94:0/3608634942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350de80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.994+0000 7f75fc883740 1 --2- 172.21.15.94:0/3608634942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334f9d00 0x55da334fa180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.994+0000 7f75fc883740 1 --2- 172.21.15.94:0/3608634942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da334f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.994+0000 7f75fc883740 1 -- 172.21.15.94:0/3608634942 >> 172.21.15.94:0/3608634942 conn(0x55da334c1550 msgr2=0x55da334f49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.994+0000 7f75fc883740 1 -- 172.21.15.94:0/3608634942 shutdown_connections 2024-02-17T12:01:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.994+0000 7f75fc883740 1 -- 172.21.15.94:0/3608634942 wait complete. 2024-02-17T12:01:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.994+0000 7f75fc883740 1 -- start start 2024-02-17T12:01:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.994+0000 7f75fc883740 1 Processor -- start 2024-02-17T12:01:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75fc883740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334f9d00 0x55da335053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75fc883740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da33505900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75fc883740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75fc883740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da33406bf0 con 0x55da334faa70 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75fc883740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da33135250 con 0x55da334f9d00 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75fc883740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da333e7200 con 0x55da3350ba90 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75fc883740 10 client.0 _ll_register_callbacks cb 0x55da33455d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75f3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75f3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:38410/0 (socket says 172.21.15.94:38410) 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75f3fff640 1 -- 172.21.15.94:0/2124553818 learned_addr learned my addr 172.21.15.94:0/2124553818 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75f2ffd640 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da33505900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75f37fe640 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334f9d00 0x55da335053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75f3fff640 1 -- 172.21.15.94:0/2124553818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334f9d00 msgr2=0x55da335053c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75f3fff640 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334f9d00 0x55da335053c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75f3fff640 1 -- 172.21.15.94:0/2124553818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 msgr2=0x55da33505900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75f3fff640 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da33505900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75f3fff640 1 -- 172.21.15.94:0/2124553818 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f4069490 con 0x55da3350ba90 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75f37fe640 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334f9d00 0x55da335053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75f3fff640 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350b130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0097f0 tx=0x7f75ec09b1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf509014e72d9d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.995+0000 7f75d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75d2ffd640 1 -- 172.21.15.94:0/2124553818 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0a31b0 con 0x55da3350ba90 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75d2ffd640 1 -- 172.21.15.94:0/2124553818 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f75ec0cb9f0 con 0x55da3350ba90 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75fc883740 1 -- 172.21.15.94:0/2124553818 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da33161600 con 0x55da3350ba90 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75d2ffd640 1 -- 172.21.15.94:0/2124553818 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75ec0cbbf0 con 0x55da3350ba90 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75fc883740 10 client.25921 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75fc883740 1 -- 172.21.15.94:0/2124553818 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55da3345fd80 con 0x55da3350ba90 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75d2ffd640 1 -- 172.21.15.94:0/2124553818 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75ec0b7080 con 0x55da3350ba90 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75d2ffd640 1 -- 172.21.15.94:0/2124553818 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f75ec099e60 con 0x55da3350ba90 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75fc883740 10 client.25921 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75fc883740 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75fc883740 20 client.25921 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75fc883740 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55da335246b0 0x55da33526aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75fc883740 4 client.25921 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75fc883740 1 -- 172.21.15.94:0/2124553818 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da331b21b0 con 0x55da335246b0 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.996+0000 7f75f37fe640 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55da335246b0 0x55da33526aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.997+0000 7f75f37fe640 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55da335246b0 0x55da33526aa0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0bd9a7d5ce04762 server_cookie=732a04f94ab2daff in_seq=0 out_seq=0 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.997+0000 7f75d2ffd640 10 client.25921 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.997+0000 7f75d2ffd640 1 -- 172.21.15.94:0/2124553818 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55da331b21b0 con 0x55da335246b0 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.997+0000 7f75d2ffd640 10 client.25921 handle_command_reply: tid=0 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.997+0000 7f75fc883740 1 client.25921 shutdown 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.998+0000 7f75fc883740 1 -- 172.21.15.94:0/2124553818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 msgr2=0x55da3350b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.998+0000 7f75fc883740 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350b130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0097f0 tx=0x7f75ec09b1d0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.998+0000 7f75fc883740 1 -- 172.21.15.94:0/2124553818 shutdown_connections 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.998+0000 7f75fc883740 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55da335246b0 0x55da33526aa0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.001 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.998+0000 7f75fc883740 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350b130 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.998+0000 7f75fc883740 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da33505900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.998+0000 7f75fc883740 1 --2- 172.21.15.94:0/2124553818 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334f9d00 0x55da335053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.998+0000 7f75fc883740 1 -- 172.21.15.94:0/2124553818 >> 172.21.15.94:0/2124553818 conn(0x55da334c1550 msgr2=0x55da334f6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.998+0000 7f75d2ffd640 0 client.25921 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.998+0000 7f75fc883740 1 -- 172.21.15.94:0/2124553818 shutdown_connections 2024-02-17T12:01:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.999+0000 7f75fc883740 1 -- 172.21.15.94:0/2124553818 wait complete. 2024-02-17T12:01:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:41.999+0000 7f75fc883740 20 client.25921 trim_cache size 0 max 16384 2024-02-17T12:01:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75fc883740 1 Processor -- start 2024-02-17T12:01:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75fc883740 1 -- start start 2024-02-17T12:01:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75fc883740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334f9d00 0x55da335058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75fc883740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334faa70 0x55da3350b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75fc883740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75fc883740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da334fb060 con 0x55da334f9d00 2024-02-17T12:01:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75fc883740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da334f57f0 con 0x55da334faa70 2024-02-17T12:01:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75fc883740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da335015e0 con 0x55da3350ba90 2024-02-17T12:01:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75f37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334f9d00 0x55da335058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75f37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334f9d00 0x55da335058c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57346/0 (socket says 172.21.15.94:57346) 2024-02-17T12:01:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75f37fe640 1 -- 172.21.15.94:0/1940814163 learned_addr learned my addr 172.21.15.94:0/1940814163 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75f3fff640 1 --2- 172.21.15.94:0/1940814163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75f2ffd640 1 --2- 172.21.15.94:0/1940814163 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334faa70 0x55da3350b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75f37fe640 1 -- 172.21.15.94:0/1940814163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 msgr2=0x55da3350ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75f37fe640 1 --2- 172.21.15.94:0/1940814163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75f37fe640 1 -- 172.21.15.94:0/1940814163 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334faa70 msgr2=0x55da3350b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75f37fe640 1 --2- 172.21.15.94:0/1940814163 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334faa70 0x55da3350b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75f37fe640 1 -- 172.21.15.94:0/1940814163 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da333fc7b0 con 0x55da334f9d00 2024-02-17T12:01:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.018+0000 7f75f37fe640 1 --2- 172.21.15.94:0/1940814163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334f9d00 0x55da335058c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f75e4005d00 tx=0x7f75e4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ff4b46f7548fd4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.019+0000 7f75d97fa640 1 -- 172.21.15.94:0/1940814163 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4027070 con 0x55da334f9d00 2024-02-17T12:01:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.019+0000 7f75d97fa640 1 -- 172.21.15.94:0/1940814163 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75e4010cb0 con 0x55da334f9d00 2024-02-17T12:01:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.019+0000 7f75d97fa640 1 -- 172.21.15.94:0/1940814163 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4020960 con 0x55da334f9d00 2024-02-17T12:01:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.020+0000 7f75fc883740 1 -- 172.21.15.94:0/1940814163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334f9d00 msgr2=0x55da335058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.020+0000 7f75fc883740 1 --2- 172.21.15.94:0/1940814163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334f9d00 0x55da335058c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f75e4005d00 tx=0x7f75e4015000 comp rx=0 tx=0).stop 2024-02-17T12:01:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.020+0000 7f75fc883740 1 -- 172.21.15.94:0/1940814163 shutdown_connections 2024-02-17T12:01:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.020+0000 7f75fc883740 1 --2- 172.21.15.94:0/1940814163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da3350ba90 0x55da3350ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.020+0000 7f75fc883740 1 --2- 172.21.15.94:0/1940814163 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da334faa70 0x55da3350b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.020+0000 7f75fc883740 1 --2- 172.21.15.94:0/1940814163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334f9d00 0x55da335058c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.020+0000 7f75fc883740 1 -- 172.21.15.94:0/1940814163 >> 172.21.15.94:0/1940814163 conn(0x55da334c1550 msgr2=0x55da335051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.020+0000 7f75fc883740 1 -- 172.21.15.94:0/1940814163 shutdown_connections 2024-02-17T12:01:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.021+0000 7f75fc883740 1 -- 172.21.15.94:0/1940814163 wait complete. 2024-02-17T12:01:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.021+0000 7f75fc883740 1 -- start start 2024-02-17T12:01:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75fc883740 1 Processor -- start 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75fc883740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da334f9d00 0x55da33508040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75fc883740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da33508580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75fc883740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da3350ba90 0x55da33508ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75fc883740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3357ab30 con 0x55da334faa70 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75fc883740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3357ac70 con 0x55da3350ba90 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75fc883740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3357ade0 con 0x55da334f9d00 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75fc883740 10 client.0 _ll_register_callbacks cb 0x7f75e80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75f2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da33508580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75f37fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da334f9d00 0x55da33508040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75f2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da33508580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57350/0 (socket says 172.21.15.94:57350) 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75f37fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da334f9d00 0x55da33508040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:38438/0 (socket says 172.21.15.94:38438) 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75f2ffd640 1 -- 172.21.15.94:0/4132796515 learned_addr learned my addr 172.21.15.94:0/4132796515 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.022+0000 7f75f3fff640 1 --2- 172.21.15.94:0/4132796515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da3350ba90 0x55da33508ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75f37fe640 1 -- 172.21.15.94:0/4132796515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da3350ba90 msgr2=0x55da33508ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75f37fe640 1 --2- 172.21.15.94:0/4132796515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da3350ba90 0x55da33508ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75f37fe640 1 -- 172.21.15.94:0/4132796515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 msgr2=0x55da33508580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75f37fe640 1 --2- 172.21.15.94:0/4132796515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da33508580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75f37fe640 1 -- 172.21.15.94:0/4132796515 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e4018070 con 0x55da334f9d00 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75f2ffd640 1 --2- 172.21.15.94:0/4132796515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da33508580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75f37fe640 1 --2- 172.21.15.94:0/4132796515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da334f9d00 0x55da33508040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f75e400ba80 tx=0x7f75e4010ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ad57e14c13cf750 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75d2ffd640 1 -- 172.21.15.94:0/4132796515 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e4027050 con 0x55da334f9d00 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75fc883740 1 -- 172.21.15.94:0/4132796515 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75e80044b0 con 0x55da334f9d00 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75d2ffd640 1 -- 172.21.15.94:0/4132796515 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f75e40275c0 con 0x55da334f9d00 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75d2ffd640 1 -- 172.21.15.94:0/4132796515 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75e4010ee0 con 0x55da334f9d00 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75fc883740 10 client.25927 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.023+0000 7f75fc883740 1 -- 172.21.15.94:0/4132796515 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55da33422540 con 0x55da334f9d00 2024-02-17T12:01:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.024+0000 7f75d2ffd640 1 -- 172.21.15.94:0/4132796515 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f75e403f080 con 0x55da334f9d00 2024-02-17T12:01:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.024+0000 7f75d2ffd640 1 -- 172.21.15.94:0/4132796515 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f75e40315f0 con 0x55da334f9d00 2024-02-17T12:01:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.024+0000 7f75fc883740 10 client.25927 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.024+0000 7f75fc883740 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.024+0000 7f75fc883740 20 client.25927 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.024+0000 7f75fc883740 1 --2- 172.21.15.94:0/4132796515 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55da3357c610 0x55da3357ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:42.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.024+0000 7f75fc883740 4 client.25927 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:01:42.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.024+0000 7f75fc883740 1 -- 172.21.15.94:0/4132796515 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55da3319e550 con 0x55da3357c610 2024-02-17T12:01:42.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.024+0000 7f75f2ffd640 1 --2- 172.21.15.94:0/4132796515 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55da3357c610 0x55da3357ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:42.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.025+0000 7f75d2ffd640 10 client.25927 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:42.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.025+0000 7f75f2ffd640 1 --2- 172.21.15.94:0/4132796515 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55da3357c610 0x55da3357ea30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5753c4401ba4daac server_cookie=c1204eea5bbbe43a in_seq=0 out_seq=0 2024-02-17T12:01:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.122+0000 7f75d2ffd640 1 -- 172.21.15.94:0/4132796515 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55da3319e550 con 0x55da3357c610 2024-02-17T12:01:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.122+0000 7f75d2ffd640 10 client.25927 handle_command_reply: tid=0 2024-02-17T12:01:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75fc883740 1 client.25927 shutdown 2024-02-17T12:01:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75fc883740 1 -- 172.21.15.94:0/4132796515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da334f9d00 msgr2=0x55da33508040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75fc883740 1 --2- 172.21.15.94:0/4132796515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da334f9d00 0x55da33508040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f75e400ba80 tx=0x7f75e4010ab0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75fc883740 1 -- 172.21.15.94:0/4132796515 shutdown_connections 2024-02-17T12:01:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75fc883740 1 --2- 172.21.15.94:0/4132796515 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55da3357c610 0x55da3357ea30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75fc883740 1 --2- 172.21.15.94:0/4132796515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55da3350ba90 0x55da33508ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75d2ffd640 0 client.25927 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75fc883740 1 --2- 172.21.15.94:0/4132796515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55da334faa70 0x55da33508580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75fc883740 1 --2- 172.21.15.94:0/4132796515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55da334f9d00 0x55da33508040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75fc883740 1 -- 172.21.15.94:0/4132796515 >> 172.21.15.94:0/4132796515 conn(0x55da334c1550 msgr2=0x55da334ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75fc883740 1 -- 172.21.15.94:0/4132796515 shutdown_connections 2024-02-17T12:01:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75fc883740 1 -- 172.21.15.94:0/4132796515 wait complete. 2024-02-17T12:01:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75fc883740 20 client.25927 trim_cache size 0 max 16384 2024-02-17T12:01:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:01:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:01:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:01:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:01:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75f9cae640 1 -- 172.21.15.94:0/3496897915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f75dc0a23d0 msgr2=0x7f75dc0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75f9cae640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f75dc0a23d0 0x7f75dc0a4890 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55da334fb1d0 tx=0x7f75e400b410 comp rx=0 tx=0).stop 2024-02-17T12:01:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75f9cae640 1 -- 172.21.15.94:0/3496897915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75f41057d0 msgr2=0x7f75f410e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.123+0000 7f75f9cae640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75f41057d0 0x7f75f410e430 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00efc0 tx=0x7f75ec00c800 comp rx=0 tx=0).stop 2024-02-17T12:01:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.124+0000 7f75f9cae640 1 -- 172.21.15.94:0/3496897915 shutdown_connections 2024-02-17T12:01:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.124+0000 7f75f9cae640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f75dc0a23d0 0x7f75dc0a4890 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.124+0000 7f75f9cae640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75f41057d0 0x7f75f410e430 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.124+0000 7f75f9cae640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f41001d0 0x7f75f410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.124+0000 7f75f9cae640 1 --2- 172.21.15.94:0/3496897915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75f40ff810 0x7f75f4072a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.124+0000 7f75f9cae640 1 -- 172.21.15.94:0/3496897915 >> 172.21.15.94:0/3496897915 conn(0x7f75f40765a0 msgr2=0x7f75f4102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.126+0000 7f75f9cae640 1 -- 172.21.15.94:0/3496897915 shutdown_connections 2024-02-17T12:01:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:42.126+0000 7f75f9cae640 1 -- 172.21.15.94:0/3496897915 wait complete. 2024-02-17T12:01:42.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:41 smithi125 ceph-mon[36819]: pgmap v110: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 42 MiB/s wr, 1.69k op/s 2024-02-17T12:01:42.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:41 smithi094 ceph-mon[32499]: pgmap v110: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 42 MiB/s wr, 1.69k op/s 2024-02-17T12:01:42.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:41 smithi189 ceph-mon[36588]: pgmap v110: 97 pgs: 97 active+clean; 2.9 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 42 MiB/s wr, 1.69k op/s 2024-02-17T12:01:44.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:43 smithi125 ceph-mon[36819]: pgmap v111: 97 pgs: 97 active+clean; 2.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 43 MiB/s wr, 1.82k op/s 2024-02-17T12:01:44.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:43 smithi094 ceph-mon[32499]: pgmap v111: 97 pgs: 97 active+clean; 2.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 43 MiB/s wr, 1.82k op/s 2024-02-17T12:01:44.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:43 smithi189 ceph-mon[36588]: pgmap v111: 97 pgs: 97 active+clean; 2.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 43 MiB/s wr, 1.82k op/s 2024-02-17T12:01:45.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 205.0 s, 3689.3 tps, lat 27.156 ms stddev 34.387 2024-02-17T12:01:46.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:45 smithi125 ceph-mon[36819]: pgmap v112: 97 pgs: 97 active+clean; 2.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 27 MiB/s wr, 1.24k op/s 2024-02-17T12:01:46.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:45 smithi094 ceph-mon[32499]: pgmap v112: 97 pgs: 97 active+clean; 2.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 27 MiB/s wr, 1.24k op/s 2024-02-17T12:01:46.320 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:01:46.321 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:01:46.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:45 smithi189 ceph-mon[36588]: pgmap v112: 97 pgs: 97 active+clean; 2.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 27 MiB/s wr, 1.24k op/s 2024-02-17T12:01:46.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.481+0000 7fb601d9b640 1 -- 172.21.15.94:0/3398375948 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec005220 con 0x7fb604101d10 2024-02-17T12:01:46.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.481+0000 7fb608e64640 1 -- 172.21.15.94:0/3398375948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb604101d10 msgr2=0x7fb604105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:46.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.481+0000 7fb608e64640 1 --2- 172.21.15.94:0/3398375948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb604101d10 0x7fb604105d50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009930 tx=0x7fb5ec02bc30 comp rx=0 tx=0).stop 2024-02-17T12:01:46.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.482+0000 7fb608e64640 1 -- 172.21.15.94:0/3398375948 shutdown_connections 2024-02-17T12:01:46.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.482+0000 7fb608e64640 1 --2- 172.21.15.94:0/3398375948 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb604106430 0x7fb60410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.482+0000 7fb608e64640 1 --2- 172.21.15.94:0/3398375948 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb604101d10 0x7fb604105d50 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.482+0000 7fb608e64640 1 --2- 172.21.15.94:0/3398375948 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb604101360 0x7fb604101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.482+0000 7fb608e64640 1 -- 172.21.15.94:0/3398375948 >> 172.21.15.94:0/3398375948 conn(0x7fb6040fcc20 msgr2=0x7fb6040ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:46.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.483+0000 7fb608e64640 1 -- 172.21.15.94:0/3398375948 shutdown_connections 2024-02-17T12:01:46.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.483+0000 7fb608e64640 1 -- 172.21.15.94:0/3398375948 wait complete. 2024-02-17T12:01:46.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.484+0000 7fb608e64640 1 Processor -- start 2024-02-17T12:01:46.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.484+0000 7fb608e64640 1 -- start start 2024-02-17T12:01:46.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.484+0000 7fb608e64640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb604101360 0x7fb60419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.484+0000 7fb608e64640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb604106430 0x7fb60419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.484+0000 7fb608e64640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb604199910 0x7fb604199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.484+0000 7fb608e64640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60419a300 con 0x7fb604101360 2024-02-17T12:01:46.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.484+0000 7fb608e64640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60419a470 con 0x7fb604199910 2024-02-17T12:01:46.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.484+0000 7fb608e64640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60419a630 con 0x7fb604106430 2024-02-17T12:01:46.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.484+0000 7fb602d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb604199910 0x7fb604199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.484+0000 7fb602d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb604199910 0x7fb604199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55236/0 (socket says 172.21.15.94:55236) 2024-02-17T12:01:46.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.484+0000 7fb602d9d640 1 -- 172.21.15.94:0/1802069994 learned_addr learned my addr 172.21.15.94:0/1802069994 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:46.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.484+0000 7fb60259c640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb604101360 0x7fb60419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.485+0000 7fb602d9d640 1 -- 172.21.15.94:0/1802069994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb604106430 msgr2=0x7fb60419fc50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:01:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.485+0000 7fb602d9d640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb604106430 0x7fb60419fc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.485+0000 7fb602d9d640 1 -- 172.21.15.94:0/1802069994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb604101360 msgr2=0x7fb60419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.485+0000 7fb602d9d640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb604101360 0x7fb60419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.485+0000 7fb602d9d640 1 -- 172.21.15.94:0/1802069994 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc009840 con 0x7fb604199910 2024-02-17T12:01:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.485+0000 7fb60259c640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb604101360 0x7fb60419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:01:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.486+0000 7fb602d9d640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb604199910 0x7fb604199dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00efd0 tx=0x7fb5fc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:46.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.486+0000 7fb5fbfff640 1 -- 172.21.15.94:0/1802069994 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc003b70 con 0x7fb604199910 2024-02-17T12:01:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.486+0000 7fb5fbfff640 1 -- 172.21.15.94:0/1802069994 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5fc003cd0 con 0x7fb604199910 2024-02-17T12:01:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.486+0000 7fb5fbfff640 1 -- 172.21.15.94:0/1802069994 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc010960 con 0x7fb604199910 2024-02-17T12:01:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.486+0000 7fb608e64640 1 -- 172.21.15.94:0/1802069994 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ec009560 con 0x7fb604199910 2024-02-17T12:01:46.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.486+0000 7fb608e64640 1 -- 172.21.15.94:0/1802069994 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6041a6900 con 0x7fb604199910 2024-02-17T12:01:46.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.489+0000 7fb5fbfff640 1 -- 172.21.15.94:0/1802069994 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb5fc004590 con 0x7fb604199910 2024-02-17T12:01:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.490+0000 7fb60ba39740 1 Processor -- start 2024-02-17T12:01:46.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.490+0000 7fb60ba39740 1 -- start start 2024-02-17T12:01:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb60ba39740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af5ac0 0x561cb0af5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.490 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb60ba39740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af63e0 0x561cb0afea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb60ba39740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0afefc0 0x561cb0b013b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb60ba39740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb0a5c280 con 0x561cb0afefc0 2024-02-17T12:01:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb60ba39740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb0a597b0 con 0x561cb0af5ac0 2024-02-17T12:01:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb60ba39740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb0a54f90 con 0x561cb0af63e0 2024-02-17T12:01:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb5fbfff640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb5f00a2400 0x7fb5f00a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb5f9d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af5ac0 0x561cb0af5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb60259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af63e0 0x561cb0afea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb60259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af63e0 0x561cb0afea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39186/0 (socket says 172.21.15.94:39186) 2024-02-17T12:01:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb5f9d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af5ac0 0x561cb0af5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55240/0 (socket says 172.21.15.94:55240) 2024-02-17T12:01:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb60259c640 1 -- 172.21.15.94:0/3474085373 learned_addr learned my addr 172.21.15.94:0/3474085373 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb602d9d640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb5f00a2400 0x7fb5f00a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.491+0000 7fb5fbfff640 1 -- 172.21.15.94:0/1802069994 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb5fc014070 con 0x7fb604199910 2024-02-17T12:01:46.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.492+0000 7fb60259c640 1 -- 172.21.15.94:0/3474085373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af5ac0 msgr2=0x561cb0af5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:46.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.492+0000 7fb60259c640 1 --2- 172.21.15.94:0/3474085373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af5ac0 0x561cb0af5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.492+0000 7fb60259c640 1 -- 172.21.15.94:0/3474085373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0afefc0 msgr2=0x561cb0b013b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:46.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.492+0000 7fb60259c640 1 --2- 172.21.15.94:0/3474085373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0afefc0 0x561cb0b013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.492+0000 7fb60259c640 1 -- 172.21.15.94:0/3474085373 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cb0a13570 con 0x561cb0af63e0 2024-02-17T12:01:46.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.492+0000 7fb5f9d9b640 1 --2- 172.21.15.94:0/3474085373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af5ac0 0x561cb0af5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:01:46.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.492+0000 7fb602d9d640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb5f00a2400 0x7fb5f00a48c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc005c40 tx=0x7fb5fc004a40 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:46.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.492+0000 7fb60259c640 1 --2- 172.21.15.94:0/3474085373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af63e0 0x561cb0afea80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009cf0 tx=0x7fb5f40117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7dfc5ff290654b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:46.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.492+0000 7fb5f8d99640 1 -- 172.21.15.94:0/3474085373 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4012450 con 0x561cb0af63e0 2024-02-17T12:01:46.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.492+0000 7fb5f8d99640 1 -- 172.21.15.94:0/3474085373 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5f4012a90 con 0x561cb0af63e0 2024-02-17T12:01:46.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.493+0000 7fb5f8d99640 1 -- 172.21.15.94:0/3474085373 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400fbd0 con 0x561cb0af63e0 2024-02-17T12:01:46.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.493+0000 7fb60ba39740 1 -- 172.21.15.94:0/3474085373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af63e0 msgr2=0x561cb0afea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:46.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.493+0000 7fb60ba39740 1 --2- 172.21.15.94:0/3474085373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af63e0 0x561cb0afea80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4009cf0 tx=0x7fb5f40117b0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.493+0000 7fb60ba39740 1 -- 172.21.15.94:0/3474085373 shutdown_connections 2024-02-17T12:01:46.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.493+0000 7fb60ba39740 1 --2- 172.21.15.94:0/3474085373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0afefc0 0x561cb0b013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.493+0000 7fb60ba39740 1 --2- 172.21.15.94:0/3474085373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af63e0 0x561cb0afea80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.493+0000 7fb60ba39740 1 --2- 172.21.15.94:0/3474085373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af5ac0 0x561cb0af5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.493+0000 7fb60ba39740 1 -- 172.21.15.94:0/3474085373 >> 172.21.15.94:0/3474085373 conn(0x561cb0af01a0 msgr2=0x561cb0af0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:46.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.494+0000 7fb60ba39740 1 -- 172.21.15.94:0/3474085373 shutdown_connections 2024-02-17T12:01:46.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.494+0000 7fb60ba39740 1 -- 172.21.15.94:0/3474085373 wait complete. 2024-02-17T12:01:46.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.494+0000 7fb60ba39740 1 -- start start 2024-02-17T12:01:46.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.494+0000 7fb60ba39740 1 Processor -- start 2024-02-17T12:01:46.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.494+0000 7fb60ba39740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 0x561cb0afe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.494+0000 7fb60ba39740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0af63e0 0x561cb0afbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.494+0000 7fb60ba39740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0afefc0 0x561cb0afc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.494+0000 7fb60ba39740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb0728430 con 0x561cb0af63e0 2024-02-17T12:01:46.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.494+0000 7fb60ba39740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb06e4e00 con 0x561cb0afefc0 2024-02-17T12:01:46.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.494+0000 7fb60ba39740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb09d9fd0 con 0x561cb0af5ac0 2024-02-17T12:01:46.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.494+0000 7fb60ba39740 10 client.0 _ll_register_callbacks cb 0x561cb0a2ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.495+0000 7fb5f9d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 0x561cb0afe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.495+0000 7fb5f9d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 0x561cb0afe910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39190/0 (socket says 172.21.15.94:39190) 2024-02-17T12:01:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.495+0000 7fb5f9d9b640 1 -- 172.21.15.94:0/1237308176 learned_addr learned my addr 172.21.15.94:0/1237308176 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.495+0000 7fb5f9d9b640 1 --2- 172.21.15.94:0/1237308176 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0afefc0 0x561cb0afc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.495+0000 7fb5f9d9b640 1 -- 172.21.15.94:0/1237308176 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0afefc0 msgr2=0x561cb0afc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.495+0000 7fb5f9d9b640 1 --2- 172.21.15.94:0/1237308176 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0afefc0 0x561cb0afc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.495+0000 7fb5f9d9b640 1 -- 172.21.15.94:0/1237308176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0af63e0 msgr2=0x561cb0afbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.495+0000 7fb5f9d9b640 1 --2- 172.21.15.94:0/1237308176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0af63e0 0x561cb0afbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.496+0000 7fb5f9d9b640 1 -- 172.21.15.94:0/1237308176 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f4009920 con 0x561cb0af5ac0 2024-02-17T12:01:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.496+0000 7fb5f9d9b640 1 --2- 172.21.15.94:0/1237308176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 0x561cb0afe910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec036040 tx=0x7fb5ec02bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b9a9c8ef8cfc27c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb5e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:01:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/1237308176 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec004b00 con 0x561cb0af5ac0 2024-02-17T12:01:46.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb60ba39740 1 -- 172.21.15.94:0/1237308176 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cb07547e0 con 0x561cb0af5ac0 2024-02-17T12:01:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/1237308176 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb5ec005000 con 0x561cb0af5ac0 2024-02-17T12:01:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/1237308176 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5ec02c910 con 0x561cb0af5ac0 2024-02-17T12:01:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb60ba39740 10 client.25936 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb60ba39740 1 -- 172.21.15.94:0/1237308176 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561cb0a53610 con 0x561cb0af5ac0 2024-02-17T12:01:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/1237308176 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561cb07547e0 con 0x561cb0af5ac0 2024-02-17T12:01:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/1237308176 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb5ec003710 con 0x561cb0af5ac0 2024-02-17T12:01:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb60ba39740 10 client.25936 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb60ba39740 10 client.25936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb60ba39740 20 client.25936 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb60ba39740 1 --2- 172.21.15.94:0/1237308176 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561cb0b0efb0 0x561cb0b113c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb60ba39740 4 client.25936 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.497+0000 7fb60ba39740 1 -- 172.21.15.94:0/1237308176 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cb0791720 con 0x561cb0b0efb0 2024-02-17T12:01:46.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.498+0000 7fb60259c640 1 --2- 172.21.15.94:0/1237308176 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561cb0b0efb0 0x561cb0b113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.498+0000 7fb5e9ffb640 10 client.25936 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.498+0000 7fb60259c640 1 --2- 172.21.15.94:0/1237308176 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561cb0b0efb0 0x561cb0b113c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac0bfc19dd7aeb2f server_cookie=c831d8c0605686fc in_seq=0 out_seq=0 2024-02-17T12:01:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.499+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/1237308176 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561cb0791720 con 0x561cb0b0efb0 2024-02-17T12:01:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.499+0000 7fb5e9ffb640 10 client.25936 handle_command_reply: tid=0 2024-02-17T12:01:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.499+0000 7fb60ba39740 1 client.25936 shutdown 2024-02-17T12:01:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.499+0000 7fb60ba39740 1 -- 172.21.15.94:0/1237308176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 msgr2=0x561cb0afe910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.499+0000 7fb60ba39740 1 --2- 172.21.15.94:0/1237308176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 0x561cb0afe910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec036040 tx=0x7fb5ec02bc30 comp rx=0 tx=0).stop 2024-02-17T12:01:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.499+0000 7fb60ba39740 1 -- 172.21.15.94:0/1237308176 shutdown_connections 2024-02-17T12:01:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.499+0000 7fb60ba39740 1 --2- 172.21.15.94:0/1237308176 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561cb0b0efb0 0x561cb0b113c0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.499+0000 7fb60ba39740 1 --2- 172.21.15.94:0/1237308176 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0afefc0 0x561cb0afc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.499+0000 7fb60ba39740 1 --2- 172.21.15.94:0/1237308176 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0af63e0 0x561cb0afbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.499+0000 7fb5e9ffb640 0 client.25936 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.499+0000 7fb60ba39740 1 --2- 172.21.15.94:0/1237308176 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 0x561cb0afe910 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.499+0000 7fb60ba39740 1 -- 172.21.15.94:0/1237308176 >> 172.21.15.94:0/1237308176 conn(0x561cb0af01a0 msgr2=0x561cb0af1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.499+0000 7fb60ba39740 1 -- 172.21.15.94:0/1237308176 shutdown_connections 2024-02-17T12:01:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.500+0000 7fb60ba39740 1 -- 172.21.15.94:0/1237308176 wait complete. 2024-02-17T12:01:46.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.500+0000 7fb60ba39740 20 client.25936 trim_cache size 0 max 16384 2024-02-17T12:01:46.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.527+0000 7fb60ba39740 1 Processor -- start 2024-02-17T12:01:46.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.527+0000 7fb60ba39740 1 -- start start 2024-02-17T12:01:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.527+0000 7fb60ba39740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 0x561cb0af67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb60ba39740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0af63e0 0x561cb0afe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb60ba39740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0afefc0 0x561cb0afe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb60ba39740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb07547e0 con 0x561cb0af63e0 2024-02-17T12:01:46.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb60ba39740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb0af5410 con 0x561cb0afefc0 2024-02-17T12:01:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb60ba39740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb0b0add0 con 0x561cb0af5ac0 2024-02-17T12:01:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb5f9d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 0x561cb0af67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb5f9d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 0x561cb0af67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39200/0 (socket says 172.21.15.94:39200) 2024-02-17T12:01:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb5f9d9b640 1 -- 172.21.15.94:0/3011962573 learned_addr learned my addr 172.21.15.94:0/3011962573 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb5f9d9b640 1 --2- 172.21.15.94:0/3011962573 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0afefc0 0x561cb0afe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb5f9d9b640 1 -- 172.21.15.94:0/3011962573 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0afefc0 msgr2=0x561cb0afe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb5f9d9b640 1 --2- 172.21.15.94:0/3011962573 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0afefc0 0x561cb0afe910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb5f9d9b640 1 -- 172.21.15.94:0/3011962573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0af63e0 msgr2=0x561cb0afe3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb5f9d9b640 1 --2- 172.21.15.94:0/3011962573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0af63e0 0x561cb0afe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb5f9d9b640 1 -- 172.21.15.94:0/3011962573 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cb09efac0 con 0x561cb0af5ac0 2024-02-17T12:01:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.528+0000 7fb5f9d9b640 1 --2- 172.21.15.94:0/3011962573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 0x561cb0af67c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec003c70 tx=0x7fb5ec02df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3341d8cf8337d5c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:46.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.529+0000 7fb5ebfff640 1 -- 172.21.15.94:0/3011962573 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec004e10 con 0x561cb0af5ac0 2024-02-17T12:01:46.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.529+0000 7fb5ebfff640 1 -- 172.21.15.94:0/3011962573 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5ec004460 con 0x561cb0af5ac0 2024-02-17T12:01:46.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.529+0000 7fb5ebfff640 1 -- 172.21.15.94:0/3011962573 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec005330 con 0x561cb0af5ac0 2024-02-17T12:01:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.529+0000 7fb60ba39740 1 -- 172.21.15.94:0/3011962573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 msgr2=0x561cb0af67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.529+0000 7fb60ba39740 1 --2- 172.21.15.94:0/3011962573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 0x561cb0af67c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec003c70 tx=0x7fb5ec02df90 comp rx=0 tx=0).stop 2024-02-17T12:01:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 -- 172.21.15.94:0/3011962573 shutdown_connections 2024-02-17T12:01:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 --2- 172.21.15.94:0/3011962573 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0afefc0 0x561cb0afe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 --2- 172.21.15.94:0/3011962573 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0af63e0 0x561cb0afe3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 --2- 172.21.15.94:0/3011962573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0af5ac0 0x561cb0af67c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 -- 172.21.15.94:0/3011962573 >> 172.21.15.94:0/3011962573 conn(0x561cb0af01a0 msgr2=0x561cb0b15d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 -- 172.21.15.94:0/3011962573 shutdown_connections 2024-02-17T12:01:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 -- 172.21.15.94:0/3011962573 wait complete. 2024-02-17T12:01:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 -- start start 2024-02-17T12:01:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 Processor -- start 2024-02-17T12:01:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0af5ac0 0x561cb0b062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af63e0 0x561cb0b02670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0afefc0 0x561cb0b02c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb0b03310 con 0x561cb0af5ac0 2024-02-17T12:01:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb0b03480 con 0x561cb0af63e0 2024-02-17T12:01:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cb0b654b0 con 0x561cb0afefc0 2024-02-17T12:01:46.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60ba39740 10 client.0 _ll_register_callbacks cb 0x7fb5f40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af63e0 0x561cb0b02670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af63e0 0x561cb0b02670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55274/0 (socket says 172.21.15.94:55274) 2024-02-17T12:01:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.530+0000 7fb60259c640 1 -- 172.21.15.94:0/2982049485 learned_addr learned my addr 172.21.15.94:0/2982049485 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.531+0000 7fb5f9d9b640 1 --2- 172.21.15.94:0/2982049485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0af5ac0 0x561cb0b062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.531+0000 7fb5f9d9b640 1 --2- 172.21.15.94:0/2982049485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0afefc0 0x561cb0b02c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.531+0000 7fb60259c640 1 -- 172.21.15.94:0/2982049485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0afefc0 msgr2=0x561cb0b02c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.531+0000 7fb60259c640 1 --2- 172.21.15.94:0/2982049485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0afefc0 0x561cb0b02c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.531+0000 7fb60259c640 1 -- 172.21.15.94:0/2982049485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0af5ac0 msgr2=0x561cb0b062f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:46.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.531+0000 7fb60259c640 1 --2- 172.21.15.94:0/2982049485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0af5ac0 0x561cb0b062f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.531+0000 7fb60259c640 1 -- 172.21.15.94:0/2982049485 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cb0a53610 con 0x561cb0af63e0 2024-02-17T12:01:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.531+0000 7fb60259c640 1 --2- 172.21.15.94:0/2982049485 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af63e0 0x561cb0b02670 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4000c00 tx=0x7fb5f40058b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=346e59369f8d6655 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.531+0000 7fb5e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.531+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/2982049485 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4012450 con 0x561cb0af63e0 2024-02-17T12:01:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.531+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/2982049485 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb5f40125b0 con 0x561cb0af63e0 2024-02-17T12:01:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.532+0000 7fb60ba39740 1 -- 172.21.15.94:0/2982049485 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5f4012a90 con 0x561cb0af63e0 2024-02-17T12:01:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.532+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/2982049485 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5f4012770 con 0x561cb0af63e0 2024-02-17T12:01:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.532+0000 7fb60ba39740 10 client.25901 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.532+0000 7fb60ba39740 1 -- 172.21.15.94:0/2982049485 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561cb0a156a0 con 0x561cb0af63e0 2024-02-17T12:01:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.532+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/2982049485 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5f4012a90 con 0x561cb0af63e0 2024-02-17T12:01:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.533+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/2982049485 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb5f4022040 con 0x561cb0af63e0 2024-02-17T12:01:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.533+0000 7fb60ba39740 10 client.25901 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:46.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.533+0000 7fb60ba39740 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:46.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.533+0000 7fb60ba39740 20 client.25901 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:46.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.533+0000 7fb60ba39740 1 --2- 172.21.15.94:0/2982049485 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561cb0b0efb0 0x561cb0b68d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.533+0000 7fb60ba39740 4 client.25901 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:01:46.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.533+0000 7fb60ba39740 1 -- 172.21.15.94:0/2982049485 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561cb07e0890 con 0x561cb0b0efb0 2024-02-17T12:01:46.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.533+0000 7fb5f9d9b640 1 --2- 172.21.15.94:0/2982049485 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561cb0b0efb0 0x561cb0b68d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.533+0000 7fb5f9d9b640 1 --2- 172.21.15.94:0/2982049485 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561cb0b0efb0 0x561cb0b68d00 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e95480c1b064eac server_cookie=cc5899926835bdf2 in_seq=0 out_seq=0 2024-02-17T12:01:46.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.534+0000 7fb5e9ffb640 10 client.25901 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:46.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.534+0000 7fb5e9ffb640 1 -- 172.21.15.94:0/2982049485 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561cb07e0890 con 0x561cb0b0efb0 2024-02-17T12:01:46.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.534+0000 7fb5e9ffb640 10 client.25901 handle_command_reply: tid=0 2024-02-17T12:01:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.534+0000 7fb60ba39740 1 client.25901 shutdown 2024-02-17T12:01:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.534+0000 7fb60ba39740 1 -- 172.21.15.94:0/2982049485 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af63e0 msgr2=0x561cb0b02670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.534+0000 7fb60ba39740 1 --2- 172.21.15.94:0/2982049485 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af63e0 0x561cb0b02670 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb5f4000c00 tx=0x7fb5f40058b0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.534+0000 7fb60ba39740 1 -- 172.21.15.94:0/2982049485 shutdown_connections 2024-02-17T12:01:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.534+0000 7fb60ba39740 1 --2- 172.21.15.94:0/2982049485 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561cb0b0efb0 0x561cb0b68d00 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.534+0000 7fb60ba39740 1 --2- 172.21.15.94:0/2982049485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561cb0afefc0 0x561cb0b02c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.534+0000 7fb5e9ffb640 0 client.25901 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.534+0000 7fb60ba39740 1 --2- 172.21.15.94:0/2982049485 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561cb0af63e0 0x561cb0b02670 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.534+0000 7fb60ba39740 1 --2- 172.21.15.94:0/2982049485 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561cb0af5ac0 0x561cb0b062f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.534+0000 7fb60ba39740 1 -- 172.21.15.94:0/2982049485 >> 172.21.15.94:0/2982049485 conn(0x561cb0af01a0 msgr2=0x561cb0afa4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.535+0000 7fb60ba39740 1 -- 172.21.15.94:0/2982049485 shutdown_connections 2024-02-17T12:01:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.535+0000 7fb60ba39740 1 -- 172.21.15.94:0/2982049485 wait complete. 2024-02-17T12:01:46.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.535+0000 7fb60ba39740 20 client.25901 trim_cache size 0 max 16384 2024-02-17T12:01:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:01:46.539 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:01:46.539 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:01:46.539 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:01:46.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.535+0000 7fb608e64640 1 -- 172.21.15.94:0/1802069994 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb5f00a2400 msgr2=0x7fb5f00a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:46.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.535+0000 7fb608e64640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb5f00a2400 0x7fb5f00a48c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc005c40 tx=0x7fb5fc004a40 comp rx=0 tx=0).stop 2024-02-17T12:01:46.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.535+0000 7fb608e64640 1 -- 172.21.15.94:0/1802069994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb604199910 msgr2=0x7fb604199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:46.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.535+0000 7fb608e64640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb604199910 0x7fb604199dc0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00efd0 tx=0x7fb5fc00c6a0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.536+0000 7fb608e64640 1 -- 172.21.15.94:0/1802069994 shutdown_connections 2024-02-17T12:01:46.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.536+0000 7fb608e64640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb5f00a2400 0x7fb5f00a48c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.536+0000 7fb608e64640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb604199910 0x7fb604199dc0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.536+0000 7fb608e64640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb604106430 0x7fb60419fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.536+0000 7fb608e64640 1 --2- 172.21.15.94:0/1802069994 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb604101360 0x7fb60419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.536+0000 7fb608e64640 1 -- 172.21.15.94:0/1802069994 >> 172.21.15.94:0/1802069994 conn(0x7fb6040fcc20 msgr2=0x7fb6040fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:46.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.537+0000 7fb608e64640 1 -- 172.21.15.94:0/1802069994 shutdown_connections 2024-02-17T12:01:46.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.537+0000 7fb608e64640 1 -- 172.21.15.94:0/1802069994 wait complete. 2024-02-17T12:01:46.554 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:01:46.555 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:01:46.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.702+0000 7f839b64d640 1 -- 172.21.15.94:0/2776747661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f839410d760 msgr2=0x7f839410db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:46.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.702+0000 7f839b64d640 1 --2- 172.21.15.94:0/2776747661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f839410d760 0x7f839410db40 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8384009930 tx=0x7f838402bc30 comp rx=0 tx=0).stop 2024-02-17T12:01:46.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.703+0000 7f839b64d640 1 -- 172.21.15.94:0/2776747661 shutdown_connections 2024-02-17T12:01:46.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.703+0000 7f839b64d640 1 --2- 172.21.15.94:0/2776747661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8394108110 0x7f8394114620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.703+0000 7f839b64d640 1 --2- 172.21.15.94:0/2776747661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8394107750 0x7f8394107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.703+0000 7f839b64d640 1 --2- 172.21.15.94:0/2776747661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f839410d760 0x7f839410db40 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.703+0000 7f839b64d640 1 -- 172.21.15.94:0/2776747661 >> 172.21.15.94:0/2776747661 conn(0x7f83940756d0 msgr2=0x7f8394077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:46.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.703+0000 7f839b64d640 1 -- 172.21.15.94:0/2776747661 shutdown_connections 2024-02-17T12:01:46.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.703+0000 7f839b64d640 1 -- 172.21.15.94:0/2776747661 wait complete. 2024-02-17T12:01:46.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.704+0000 7f839b64d640 1 Processor -- start 2024-02-17T12:01:46.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.704+0000 7f839b64d640 1 -- start start 2024-02-17T12:01:46.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.704+0000 7f839b64d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8394107750 0x7f839419dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.704+0000 7f839b64d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8394108110 0x7f839419e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.704+0000 7f839b64d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839410d760 0x7f83941a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.704+0000 7f839b64d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83941a2ca0 con 0x7f8394107750 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.704+0000 7f839b64d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83941a2e10 con 0x7f839410d760 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.704+0000 7f839b64d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83941a2fd0 con 0x7f8394108110 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.705+0000 7f8399bea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839410d760 0x7f83941a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.705+0000 7f8399bea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839410d760 0x7f83941a26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55284/0 (socket says 172.21.15.94:55284) 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.705+0000 7f8399bea640 1 -- 172.21.15.94:0/3768775058 learned_addr learned my addr 172.21.15.94:0/3768775058 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.705+0000 7f8399bea640 1 -- 172.21.15.94:0/3768775058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8394108110 msgr2=0x7f839419e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.705+0000 7f8399bea640 1 --2- 172.21.15.94:0/3768775058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8394108110 0x7f839419e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.705+0000 7f8399bea640 1 -- 172.21.15.94:0/3768775058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8394107750 msgr2=0x7f839419dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.705+0000 7f8399bea640 1 --2- 172.21.15.94:0/3768775058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8394107750 0x7f839419dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.705+0000 7f8399bea640 1 -- 172.21.15.94:0/3768775058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8390009490 con 0x7f839410d760 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.705+0000 7f8399bea640 1 --2- 172.21.15.94:0/3768775058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839410d760 0x7f83941a26a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f838c00eb10 tx=0x7f838c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:46.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.706+0000 7f838a7fc640 1 -- 172.21.15.94:0/3768775058 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c004b20 con 0x7f839410d760 2024-02-17T12:01:46.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.706+0000 7f839b64d640 1 -- 172.21.15.94:0/3768775058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8384009560 con 0x7f839410d760 2024-02-17T12:01:46.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.706+0000 7f838a7fc640 1 -- 172.21.15.94:0/3768775058 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f838c004c80 con 0x7f839410d760 2024-02-17T12:01:46.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.706+0000 7f838a7fc640 1 -- 172.21.15.94:0/3768775058 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c010670 con 0x7f839410d760 2024-02-17T12:01:46.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.706+0000 7f839b64d640 1 -- 172.21.15.94:0/3768775058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83941aa950 con 0x7f839410d760 2024-02-17T12:01:46.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.709+0000 7f839b64d640 1 -- 172.21.15.94:0/3768775058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8394067f70 con 0x7f839410d760 2024-02-17T12:01:46.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.709+0000 7f838a7fc640 1 -- 172.21.15.94:0/3768775058 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f838c005020 con 0x7f839410d760 2024-02-17T12:01:46.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.710+0000 7f838a7fc640 1 --2- 172.21.15.94:0/3768775058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83800a22f0 0x7f83800a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:46.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.710+0000 7f838a7fc640 1 -- 172.21.15.94:0/3768775058 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f838c014070 con 0x7f839410d760 2024-02-17T12:01:46.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.711+0000 7f83993e9640 1 --2- 172.21.15.94:0/3768775058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83800a22f0 0x7f83800a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:46.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.711+0000 7f83993e9640 1 --2- 172.21.15.94:0/3768775058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83800a22f0 0x7f83800a47b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8384036040 tx=0x7f8384000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:46.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:46.714+0000 7f838a7fc640 1 -- 172.21.15.94:0/3768775058 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f838c0973b0 con 0x7f839410d760 2024-02-17T12:01:47.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.062+0000 7f839b64d640 1 -- 172.21.15.94:0/3768775058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f839410fd20 con 0x7f839410d760 2024-02-17T12:01:47.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.063+0000 7f838a7fc640 1 -- 172.21.15.94:0/3768775058 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11344 (secure 0 0 0) 0x7f838c09aa70 con 0x7f839410d760 2024-02-17T12:01:47.063 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:01:47.063 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:01:15.200235+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24418":{"gid":24418,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:01:47.064 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 23 2024-02-17T12:01:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.066+0000 7f839b64d640 1 -- 172.21.15.94:0/3768775058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83800a22f0 msgr2=0x7f83800a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.066+0000 7f839b64d640 1 --2- 172.21.15.94:0/3768775058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83800a22f0 0x7f83800a47b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8384036040 tx=0x7f8384000bc0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.066+0000 7f839b64d640 1 -- 172.21.15.94:0/3768775058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839410d760 msgr2=0x7f83941a26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.066+0000 7f839b64d640 1 --2- 172.21.15.94:0/3768775058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839410d760 0x7f83941a26a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f838c00eb10 tx=0x7f838c00c2d0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.067+0000 7f839b64d640 1 -- 172.21.15.94:0/3768775058 shutdown_connections 2024-02-17T12:01:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.067+0000 7f839b64d640 1 --2- 172.21.15.94:0/3768775058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83800a22f0 0x7f83800a47b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.067+0000 7f839b64d640 1 --2- 172.21.15.94:0/3768775058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839410d760 0x7f83941a26a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.067+0000 7f839b64d640 1 --2- 172.21.15.94:0/3768775058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8394108110 0x7f839419e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.067+0000 7f839b64d640 1 --2- 172.21.15.94:0/3768775058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8394107750 0x7f839419dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.067+0000 7f839b64d640 1 -- 172.21.15.94:0/3768775058 >> 172.21.15.94:0/3768775058 conn(0x7f83940756d0 msgr2=0x7f8394075dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.067+0000 7f839b64d640 1 -- 172.21.15.94:0/3768775058 shutdown_connections 2024-02-17T12:01:47.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.067+0000 7f839b64d640 1 -- 172.21.15.94:0/3768775058 wait complete. 2024-02-17T12:01:47.081 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:01:47.082 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:01:47.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.231+0000 7efde4f4f640 1 -- 172.21.15.94:0/3032403311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efde0108660 msgr2=0x7efde0110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.231+0000 7efde4f4f640 1 --2- 172.21.15.94:0/3032403311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efde0108660 0x7efde0110230 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009cd0 tx=0x7efdd802c0e0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.231+0000 7efde4f4f640 1 -- 172.21.15.94:0/3032403311 shutdown_connections 2024-02-17T12:01:47.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.231+0000 7efde4f4f640 1 --2- 172.21.15.94:0/3032403311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efde0108660 0x7efde0110230 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.231+0000 7efde4f4f640 1 --2- 172.21.15.94:0/3032403311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efde0103f40 0x7efde0107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.231+0000 7efde4f4f640 1 --2- 172.21.15.94:0/3032403311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efde0103590 0x7efde0103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.231+0000 7efde4f4f640 1 -- 172.21.15.94:0/3032403311 >> 172.21.15.94:0/3032403311 conn(0x7efde00fedf0 msgr2=0x7efde0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.231+0000 7efde4f4f640 1 -- 172.21.15.94:0/3032403311 shutdown_connections 2024-02-17T12:01:47.231 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.231+0000 7efde4f4f640 1 -- 172.21.15.94:0/3032403311 wait complete. 2024-02-17T12:01:47.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.232+0000 7efde4f4f640 1 Processor -- start 2024-02-17T12:01:47.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.232+0000 7efde4f4f640 1 -- start start 2024-02-17T12:01:47.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efde4f4f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efde0103590 0x7efde0075690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efde4f4f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efde0108660 0x7efde0075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efde4f4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efde0076110 0x7efde01aa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efde4f4f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde0079870 con 0x7efde0103590 2024-02-17T12:01:47.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efde4f4f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde01aa540 con 0x7efde0076110 2024-02-17T12:01:47.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efde4f4f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde01aa700 con 0x7efde0108660 2024-02-17T12:01:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efdded9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efde0076110 0x7efde01aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efdde59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efde0103590 0x7efde0075690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efdded9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efde0076110 0x7efde01aa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55310/0 (socket says 172.21.15.94:55310) 2024-02-17T12:01:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efdddd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efde0108660 0x7efde0075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efdded9d640 1 -- 172.21.15.94:0/1168667670 learned_addr learned my addr 172.21.15.94:0/1168667670 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efdded9d640 1 -- 172.21.15.94:0/1168667670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efde0108660 msgr2=0x7efde0075bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efdded9d640 1 --2- 172.21.15.94:0/1168667670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efde0108660 0x7efde0075bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efdded9d640 1 -- 172.21.15.94:0/1168667670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efde0103590 msgr2=0x7efde0075690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efdded9d640 1 --2- 172.21.15.94:0/1168667670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efde0103590 0x7efde0075690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.233+0000 7efdded9d640 1 -- 172.21.15.94:0/1168667670 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd8009900 con 0x7efde0076110 2024-02-17T12:01:47.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.234+0000 7efdded9d640 1 --2- 172.21.15.94:0/1168667670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efde0076110 0x7efde01aa000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009ca0 tx=0x7efdd8032570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.234+0000 7efdcf7fe640 1 -- 172.21.15.94:0/1168667670 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd80328b0 con 0x7efde0076110 2024-02-17T12:01:47.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.235+0000 7efdcf7fe640 1 -- 172.21.15.94:0/1168667670 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efdd8003e50 con 0x7efde0076110 2024-02-17T12:01:47.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.235+0000 7efde4f4f640 1 -- 172.21.15.94:0/1168667670 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde01aa980 con 0x7efde0076110 2024-02-17T12:01:47.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.235+0000 7efdcf7fe640 1 -- 172.21.15.94:0/1168667670 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8004770 con 0x7efde0076110 2024-02-17T12:01:47.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.235+0000 7efde4f4f640 1 -- 172.21.15.94:0/1168667670 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde01aae30 con 0x7efde0076110 2024-02-17T12:01:47.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.237+0000 7efdcf7fe640 1 -- 172.21.15.94:0/1168667670 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7efdd800a040 con 0x7efde0076110 2024-02-17T12:01:47.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.238+0000 7efdcf7fe640 1 --2- 172.21.15.94:0/1168667670 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efdc80a2380 0x7efdc80a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.238+0000 7efdde59c640 1 --2- 172.21.15.94:0/1168667670 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efdc80a2380 0x7efdc80a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.238+0000 7efdcf7fe640 1 -- 172.21.15.94:0/1168667670 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7efdd80f0730 con 0x7efde0076110 2024-02-17T12:01:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.238+0000 7efde7c01740 1 Processor -- start 2024-02-17T12:01:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.238+0000 7efde7c01740 1 -- start start 2024-02-17T12:01:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efde7c01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312250 0x55c7e8312630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efde7c01740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e8312b70 0x55c7e831b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efde7c01740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e831db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efde7c01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e82cd960 con 0x55c7e8312250 2024-02-17T12:01:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efde7c01740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e8281310 con 0x55c7e8312b70 2024-02-17T12:01:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efde7c01740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e827e840 con 0x55c7e831b750 2024-02-17T12:01:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdde59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312250 0x55c7e8312630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdde59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312250 0x55c7e8312630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46558/0 (socket says 172.21.15.94:46558) 2024-02-17T12:01:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdde59c640 1 -- 172.21.15.94:0/572303626 learned_addr learned my addr 172.21.15.94:0/572303626 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdddd9b640 1 --2- 172.21.15.94:0/572303626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e8312b70 0x55c7e831b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdded9d640 1 --2- 172.21.15.94:0/572303626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e831db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdde59c640 1 --2- 172.21.15.94:0/1168667670 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efdc80a2380 0x7efdc80a4840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009c20 tx=0x7efdd0011810 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdde59c640 1 -- 172.21.15.94:0/572303626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 msgr2=0x55c7e831db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdde59c640 1 --2- 172.21.15.94:0/572303626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e831db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdde59c640 1 -- 172.21.15.94:0/572303626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e8312b70 msgr2=0x55c7e831b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdde59c640 1 --2- 172.21.15.94:0/572303626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e8312b70 0x55c7e831b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdde59c640 1 -- 172.21.15.94:0/572303626 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7e823c060 con 0x55c7e8312250 2024-02-17T12:01:47.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdded9d640 1 --2- 172.21.15.94:0/572303626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e831db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:47.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdddd9b640 1 --2- 172.21.15.94:0/572303626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e8312b70 0x55c7e831b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:47.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdde59c640 1 --2- 172.21.15.94:0/572303626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312250 0x55c7e8312630 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efdd001d4f0 tx=0x7efdd001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf1fbee71dea041 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.239+0000 7efdccff9640 1 -- 172.21.15.94:0/572303626 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd000f680 con 0x55c7e8312250 2024-02-17T12:01:47.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.240+0000 7efdccff9640 1 -- 172.21.15.94:0/572303626 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efdd000fcc0 con 0x55c7e8312250 2024-02-17T12:01:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.240+0000 7efdccff9640 1 -- 172.21.15.94:0/572303626 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0026e10 con 0x55c7e8312250 2024-02-17T12:01:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.241+0000 7efde7c01740 1 -- 172.21.15.94:0/572303626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312250 msgr2=0x55c7e8312630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.241+0000 7efde7c01740 1 --2- 172.21.15.94:0/572303626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312250 0x55c7e8312630 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efdd001d4f0 tx=0x7efdd001c000 comp rx=0 tx=0).stop 2024-02-17T12:01:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.241+0000 7efde7c01740 1 -- 172.21.15.94:0/572303626 shutdown_connections 2024-02-17T12:01:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.241+0000 7efde7c01740 1 --2- 172.21.15.94:0/572303626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e831db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.241+0000 7efde7c01740 1 --2- 172.21.15.94:0/572303626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e8312b70 0x55c7e831b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.241+0000 7efde7c01740 1 --2- 172.21.15.94:0/572303626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312250 0x55c7e8312630 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.241+0000 7efde7c01740 1 -- 172.21.15.94:0/572303626 >> 172.21.15.94:0/572303626 conn(0x55c7e830c980 msgr2=0x55c7e830cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.241+0000 7efde7c01740 1 -- 172.21.15.94:0/572303626 shutdown_connections 2024-02-17T12:01:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.241+0000 7efde7c01740 1 -- 172.21.15.94:0/572303626 wait complete. 2024-02-17T12:01:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.242+0000 7efde7c01740 1 -- start start 2024-02-17T12:01:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.242+0000 7efde7c01740 1 Processor -- start 2024-02-17T12:01:47.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.242+0000 7efde7c01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 0x55c7e8318870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.242+0000 7efde7c01740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e8318db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.242+0000 7efde7c01740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e83192f0 0x55c7e83276b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.242+0000 7efde7c01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e827a020 con 0x55c7e8312b70 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.242+0000 7efde7c01740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e7f4d430 con 0x55c7e83192f0 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.242+0000 7efde7c01740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e7f09e00 con 0x55c7e831b750 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.242+0000 7efde7c01740 10 client.0 _ll_register_callbacks cb 0x55c7e8221cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.242+0000 7efdddd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e8318db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.242+0000 7efdddd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e8318db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39274/0 (socket says 172.21.15.94:39274) 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.242+0000 7efdddd9b640 1 -- 172.21.15.94:0/2445213548 learned_addr learned my addr 172.21.15.94:0/2445213548 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.242+0000 7efdded9d640 1 --2- 172.21.15.94:0/2445213548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e83192f0 0x55c7e83276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.243+0000 7efdded9d640 1 -- 172.21.15.94:0/2445213548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 msgr2=0x55c7e8318db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.243+0000 7efdded9d640 1 --2- 172.21.15.94:0/2445213548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e8318db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.243+0000 7efdded9d640 1 -- 172.21.15.94:0/2445213548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 msgr2=0x55c7e8318870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.243+0000 7efdded9d640 1 --2- 172.21.15.94:0/2445213548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 0x55c7e8318870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.243+0000 7efdded9d640 1 -- 172.21.15.94:0/2445213548 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd0009e30 con 0x55c7e83192f0 2024-02-17T12:01:47.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.243+0000 7efdded9d640 1 --2- 172.21.15.94:0/2445213548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e83192f0 0x55c7e83276b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efdd80edf50 tx=0x7efdd80bccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15410f666fb0c1f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.244+0000 7efdc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:47.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.244+0000 7efdc5ffb640 1 -- 172.21.15.94:0/2445213548 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd80c13f0 con 0x55c7e83192f0 2024-02-17T12:01:47.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.244+0000 7efdc5ffb640 1 -- 172.21.15.94:0/2445213548 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7efdd80c5a40 con 0x55c7e83192f0 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.244+0000 7efde7c01740 1 -- 172.21.15.94:0/2445213548 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7e7f797e0 con 0x55c7e83192f0 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.245+0000 7efdc5ffb640 1 -- 172.21.15.94:0/2445213548 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdd80b9910 con 0x55c7e83192f0 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.245+0000 7efde7c01740 10 client.25919 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.245+0000 7efde7c01740 1 -- 172.21.15.94:0/2445213548 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c7e82786a0 con 0x55c7e83192f0 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.245+0000 7efdc5ffb640 1 -- 172.21.15.94:0/2445213548 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c7e7f797e0 con 0x55c7e83192f0 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.245+0000 7efdc5ffb640 1 -- 172.21.15.94:0/2445213548 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efdd8050020 con 0x55c7e83192f0 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.245+0000 7efde7c01740 10 client.25919 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.245+0000 7efde7c01740 10 client.25919 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.245+0000 7efde7c01740 20 client.25919 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.245+0000 7efde7c01740 1 --2- 172.21.15.94:0/2445213548 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c7e832be70 0x55c7e832e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.245+0000 7efde7c01740 4 client.25919 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.245+0000 7efde7c01740 1 -- 172.21.15.94:0/2445213548 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7e7fb6720 con 0x55c7e832be70 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.246+0000 7efdde59c640 1 --2- 172.21.15.94:0/2445213548 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c7e832be70 0x55c7e832e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.246+0000 7efdc5ffb640 10 client.25919 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:01:47.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.246+0000 7efdde59c640 1 --2- 172.21.15.94:0/2445213548 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c7e832be70 0x55c7e832e260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54a4e9375b042df7 server_cookie=c70714e227bf8f5 in_seq=0 out_seq=0 2024-02-17T12:01:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.247+0000 7efdc5ffb640 1 -- 172.21.15.94:0/2445213548 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c7e7fb6720 con 0x55c7e832be70 2024-02-17T12:01:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.247+0000 7efdc5ffb640 10 client.25919 handle_command_reply: tid=0 2024-02-17T12:01:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.247+0000 7efde7c01740 1 client.25919 shutdown 2024-02-17T12:01:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.248+0000 7efde7c01740 1 -- 172.21.15.94:0/2445213548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e83192f0 msgr2=0x55c7e83276b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.248+0000 7efde7c01740 1 --2- 172.21.15.94:0/2445213548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e83192f0 0x55c7e83276b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efdd80edf50 tx=0x7efdd80bccc0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.248+0000 7efde7c01740 1 -- 172.21.15.94:0/2445213548 shutdown_connections 2024-02-17T12:01:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.248+0000 7efde7c01740 1 --2- 172.21.15.94:0/2445213548 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c7e832be70 0x55c7e832e260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.248+0000 7efde7c01740 1 --2- 172.21.15.94:0/2445213548 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e83192f0 0x55c7e83276b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.248+0000 7efde7c01740 1 --2- 172.21.15.94:0/2445213548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e8318db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.248+0000 7efde7c01740 1 --2- 172.21.15.94:0/2445213548 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 0x55c7e8318870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.248+0000 7efde7c01740 1 -- 172.21.15.94:0/2445213548 >> 172.21.15.94:0/2445213548 conn(0x55c7e830c980 msgr2=0x55c7e830e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.248+0000 7efdc5ffb640 0 client.25919 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:01:47.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.248+0000 7efde7c01740 1 -- 172.21.15.94:0/2445213548 shutdown_connections 2024-02-17T12:01:47.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.248+0000 7efde7c01740 1 -- 172.21.15.94:0/2445213548 wait complete. 2024-02-17T12:01:47.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.248+0000 7efde7c01740 20 client.25919 trim_cache size 0 max 16384 2024-02-17T12:01:47.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.269+0000 7efde7c01740 1 Processor -- start 2024-02-17T12:01:47.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.269+0000 7efde7c01740 1 -- start start 2024-02-17T12:01:47.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.269+0000 7efde7c01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 0x55c7e8313bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.269+0000 7efde7c01740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e83192f0 0x55c7e83270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.269+0000 7efde7c01740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e831b750 0x55c7e83275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.269+0000 7efde7c01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e81ff130 con 0x55c7e8312b70 2024-02-17T12:01:47.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.269+0000 7efde7c01740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e8312870 con 0x55c7e831b750 2024-02-17T12:01:47.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.269+0000 7efde7c01740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e83082b0 con 0x55c7e83192f0 2024-02-17T12:01:47.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.269+0000 7efdddd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e83192f0 0x55c7e83270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.269+0000 7efdddd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e83192f0 0x55c7e83270b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39276/0 (socket says 172.21.15.94:39276) 2024-02-17T12:01:47.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.269+0000 7efdddd9b640 1 -- 172.21.15.94:0/2611656068 learned_addr learned my addr 172.21.15.94:0/2611656068 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.270+0000 7efdde59c640 1 --2- 172.21.15.94:0/2611656068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 0x55c7e8313bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.270+0000 7efdded9d640 1 --2- 172.21.15.94:0/2611656068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e831b750 0x55c7e83275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.270+0000 7efdddd9b640 1 -- 172.21.15.94:0/2611656068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e831b750 msgr2=0x55c7e83275f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.270+0000 7efdddd9b640 1 --2- 172.21.15.94:0/2611656068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e831b750 0x55c7e83275f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.270+0000 7efdddd9b640 1 -- 172.21.15.94:0/2611656068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 msgr2=0x55c7e8313bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.270+0000 7efdddd9b640 1 --2- 172.21.15.94:0/2611656068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 0x55c7e8313bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.270+0000 7efdddd9b640 1 -- 172.21.15.94:0/2611656068 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7e8214c20 con 0x55c7e83192f0 2024-02-17T12:01:47.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.270+0000 7efdddd9b640 1 --2- 172.21.15.94:0/2611656068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e83192f0 0x55c7e83270b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009d60 tx=0x7efdd4010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6f52aedf7d98f5e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.271+0000 7efdc7fff640 1 -- 172.21.15.94:0/2611656068 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd401c5f0 con 0x55c7e83192f0 2024-02-17T12:01:47.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.271+0000 7efdc7fff640 1 -- 172.21.15.94:0/2611656068 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efdd4018040 con 0x55c7e83192f0 2024-02-17T12:01:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.271+0000 7efdc7fff640 1 -- 172.21.15.94:0/2611656068 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd401b9f0 con 0x55c7e83192f0 2024-02-17T12:01:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.272+0000 7efde7c01740 1 -- 172.21.15.94:0/2611656068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e83192f0 msgr2=0x55c7e83270b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.272+0000 7efde7c01740 1 --2- 172.21.15.94:0/2611656068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e83192f0 0x55c7e83270b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009d60 tx=0x7efdd4010800 comp rx=0 tx=0).stop 2024-02-17T12:01:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.272+0000 7efde7c01740 1 -- 172.21.15.94:0/2611656068 shutdown_connections 2024-02-17T12:01:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.272+0000 7efde7c01740 1 --2- 172.21.15.94:0/2611656068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e831b750 0x55c7e83275f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.272+0000 7efde7c01740 1 --2- 172.21.15.94:0/2611656068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e83192f0 0x55c7e83270b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.272+0000 7efde7c01740 1 --2- 172.21.15.94:0/2611656068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 0x55c7e8313bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.272+0000 7efde7c01740 1 -- 172.21.15.94:0/2611656068 >> 172.21.15.94:0/2611656068 conn(0x55c7e830c980 msgr2=0x55c7e8332bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.272+0000 7efde7c01740 1 -- 172.21.15.94:0/2611656068 shutdown_connections 2024-02-17T12:01:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.272+0000 7efde7c01740 1 -- 172.21.15.94:0/2611656068 wait complete. 2024-02-17T12:01:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.272+0000 7efde7c01740 1 -- start start 2024-02-17T12:01:47.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efde7c01740 1 Processor -- start 2024-02-17T12:01:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efde7c01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 0x55c7e8322870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efde7c01740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e83192f0 0x55c7e8320c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efde7c01740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e83211a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efde7c01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e83217b0 con 0x55c7e8312b70 2024-02-17T12:01:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efde7c01740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e8321920 con 0x55c7e83192f0 2024-02-17T12:01:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efde7c01740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7e8321ae0 con 0x55c7e831b750 2024-02-17T12:01:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efde7c01740 10 client.0 _ll_register_callbacks cb 0x7efdd0009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efdde59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 0x55c7e8322870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efdde59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 0x55c7e8322870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46584/0 (socket says 172.21.15.94:46584) 2024-02-17T12:01:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efdde59c640 1 -- 172.21.15.94:0/2484532210 learned_addr learned my addr 172.21.15.94:0/2484532210 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efdddd9b640 1 --2- 172.21.15.94:0/2484532210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e83192f0 0x55c7e8320c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efdded9d640 1 --2- 172.21.15.94:0/2484532210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e83211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efdddd9b640 1 -- 172.21.15.94:0/2484532210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 msgr2=0x55c7e83211a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efdddd9b640 1 --2- 172.21.15.94:0/2484532210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e83211a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efdddd9b640 1 -- 172.21.15.94:0/2484532210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 msgr2=0x55c7e8322870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efdddd9b640 1 --2- 172.21.15.94:0/2484532210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 0x55c7e8322870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.273+0000 7efdddd9b640 1 -- 172.21.15.94:0/2484532210 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd4009a40 con 0x55c7e83192f0 2024-02-17T12:01:47.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.274+0000 7efdddd9b640 1 --2- 172.21.15.94:0/2484532210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e83192f0 0x55c7e8320c60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55c7e830c340 tx=0x7efdd4005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da32e3161ced9850 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.274+0000 7efdc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:47.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.274+0000 7efdc5ffb640 1 -- 172.21.15.94:0/2484532210 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd4004440 con 0x55c7e83192f0 2024-02-17T12:01:47.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.274+0000 7efdc5ffb640 1 -- 172.21.15.94:0/2484532210 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7efdd40046c0 con 0x55c7e83192f0 2024-02-17T12:01:47.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.274+0000 7efde7c01740 1 -- 172.21.15.94:0/2484532210 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efdd000fcc0 con 0x55c7e83192f0 2024-02-17T12:01:47.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.274+0000 7efdc5ffb640 1 -- 172.21.15.94:0/2484532210 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdd401b390 con 0x55c7e83192f0 2024-02-17T12:01:47.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.274+0000 7efde7c01740 10 client.25925 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:47.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.275+0000 7efde7c01740 1 -- 172.21.15.94:0/2484532210 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c7e823a8c0 con 0x55c7e83192f0 2024-02-17T12:01:47.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.275+0000 7efdc5ffb640 1 -- 172.21.15.94:0/2484532210 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efdd4004d60 con 0x55c7e83192f0 2024-02-17T12:01:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.276+0000 7efdc5ffb640 1 -- 172.21.15.94:0/2484532210 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efdd40035f0 con 0x55c7e83192f0 2024-02-17T12:01:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.276+0000 7efde7c01740 10 client.25925 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.276+0000 7efde7c01740 10 client.25925 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:01:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.276+0000 7efde7c01740 20 client.25925 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.276+0000 7efde7c01740 1 --2- 172.21.15.94:0/2484532210 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c7e8383760 0x55c7e8385b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.276+0000 7efde7c01740 4 client.25925 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:01:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.276+0000 7efde7c01740 1 -- 172.21.15.94:0/2484532210 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7efdd000fcc0 con 0x55c7e8383760 2024-02-17T12:01:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.277+0000 7efdde59c640 1 --2- 172.21.15.94:0/2484532210 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c7e8383760 0x55c7e8385b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efdde59c640 1 --2- 172.21.15.94:0/2484532210 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c7e8383760 0x55c7e8385b50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72b82ee1693caf9a server_cookie=4ff47811361cc086 in_seq=0 out_seq=0 2024-02-17T12:01:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efdc5ffb640 10 client.25925 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:01:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efdc5ffb640 1 -- 172.21.15.94:0/2484532210 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7efdd000fcc0 con 0x55c7e8383760 2024-02-17T12:01:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efdc5ffb640 10 client.25925 handle_command_reply: tid=0 2024-02-17T12:01:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efde7c01740 1 client.25925 shutdown 2024-02-17T12:01:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efde7c01740 1 -- 172.21.15.94:0/2484532210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e83192f0 msgr2=0x55c7e8320c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efde7c01740 1 --2- 172.21.15.94:0/2484532210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e83192f0 0x55c7e8320c60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55c7e830c340 tx=0x7efdd4005750 comp rx=0 tx=0).stop 2024-02-17T12:01:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efde7c01740 1 -- 172.21.15.94:0/2484532210 shutdown_connections 2024-02-17T12:01:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efde7c01740 1 --2- 172.21.15.94:0/2484532210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7e831b750 0x55c7e83211a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efde7c01740 1 --2- 172.21.15.94:0/2484532210 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55c7e8383760 0x55c7e8385b50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efde7c01740 1 --2- 172.21.15.94:0/2484532210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c7e83192f0 0x55c7e8320c60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efde7c01740 1 --2- 172.21.15.94:0/2484532210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c7e8312b70 0x55c7e8322870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efde7c01740 1 -- 172.21.15.94:0/2484532210 >> 172.21.15.94:0/2484532210 conn(0x55c7e830c980 msgr2=0x55c7e8313bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efdc5ffb640 0 client.25925 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:01:47.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efde7c01740 1 -- 172.21.15.94:0/2484532210 shutdown_connections 2024-02-17T12:01:47.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.278+0000 7efde7c01740 1 -- 172.21.15.94:0/2484532210 wait complete. 2024-02-17T12:01:47.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.279+0000 7efde7c01740 20 client.25925 trim_cache size 0 max 16384 2024-02-17T12:01:47.282 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:01:47.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.279+0000 7efde4f4f640 1 -- 172.21.15.94:0/1168667670 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efdc80a2380 msgr2=0x7efdc80a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.279+0000 7efde4f4f640 1 --2- 172.21.15.94:0/1168667670 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efdc80a2380 0x7efdc80a4840 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efdd0009c20 tx=0x7efdd0011810 comp rx=0 tx=0).stop 2024-02-17T12:01:47.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.279+0000 7efde4f4f640 1 -- 172.21.15.94:0/1168667670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efde0076110 msgr2=0x7efde01aa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.279+0000 7efde4f4f640 1 --2- 172.21.15.94:0/1168667670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efde0076110 0x7efde01aa000 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009ca0 tx=0x7efdd8032570 comp rx=0 tx=0).stop 2024-02-17T12:01:47.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.279+0000 7efde4f4f640 1 -- 172.21.15.94:0/1168667670 shutdown_connections 2024-02-17T12:01:47.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.279+0000 7efde4f4f640 1 --2- 172.21.15.94:0/1168667670 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efdc80a2380 0x7efdc80a4840 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.279+0000 7efde4f4f640 1 --2- 172.21.15.94:0/1168667670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efde0076110 0x7efde01aa000 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.279+0000 7efde4f4f640 1 --2- 172.21.15.94:0/1168667670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efde0108660 0x7efde0075bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.279+0000 7efde4f4f640 1 --2- 172.21.15.94:0/1168667670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efde0103590 0x7efde0075690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.279+0000 7efde4f4f640 1 -- 172.21.15.94:0/1168667670 >> 172.21.15.94:0/1168667670 conn(0x7efde00fedf0 msgr2=0x7efde01001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.279+0000 7efde4f4f640 1 -- 172.21.15.94:0/1168667670 shutdown_connections 2024-02-17T12:01:47.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.279+0000 7efde4f4f640 1 -- 172.21.15.94:0/1168667670 wait complete. 2024-02-17T12:01:47.290 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:46 smithi094 ceph-mon[32499]: pgmap v113: 97 pgs: 97 active+clean; 2.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 11 MiB/s wr, 671 op/s 2024-02-17T12:01:47.292 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:01:47.292 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:01:47.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:46 smithi189 ceph-mon[36588]: pgmap v113: 97 pgs: 97 active+clean; 2.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 11 MiB/s wr, 671 op/s 2024-02-17T12:01:47.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:46 smithi125 ceph-mon[36819]: pgmap v113: 97 pgs: 97 active+clean; 2.9 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 11 MiB/s wr, 671 op/s 2024-02-17T12:01:47.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.441+0000 7fdf95e00640 1 -- 172.21.15.94:0/3396265086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf90107710 msgr2=0x7fdf90107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.441+0000 7fdf95e00640 1 --2- 172.21.15.94:0/3396265086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf90107710 0x7fdf90107b90 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009930 tx=0x7fdf8402bc30 comp rx=0 tx=0).stop 2024-02-17T12:01:47.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.441+0000 7fdf95e00640 1 -- 172.21.15.94:0/3396265086 shutdown_connections 2024-02-17T12:01:47.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.441+0000 7fdf95e00640 1 --2- 172.21.15.94:0/3396265086 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf901080d0 0x7fdf901145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.441+0000 7fdf95e00640 1 --2- 172.21.15.94:0/3396265086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf90107710 0x7fdf90107b90 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.441+0000 7fdf95e00640 1 --2- 172.21.15.94:0/3396265086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf9010d720 0x7fdf9010db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.441+0000 7fdf95e00640 1 -- 172.21.15.94:0/3396265086 >> 172.21.15.94:0/3396265086 conn(0x7fdf90075620 msgr2=0x7fdf90077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.441+0000 7fdf95e00640 1 -- 172.21.15.94:0/3396265086 shutdown_connections 2024-02-17T12:01:47.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.441+0000 7fdf95e00640 1 -- 172.21.15.94:0/3396265086 wait complete. 2024-02-17T12:01:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.442+0000 7fdf95e00640 1 Processor -- start 2024-02-17T12:01:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.442+0000 7fdf95e00640 1 -- start start 2024-02-17T12:01:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.442+0000 7fdf95e00640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf90107710 0x7fdf9019dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.442+0000 7fdf95e00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf901080d0 0x7fdf9019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.442+0000 7fdf95e00640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf9010d720 0x7fdf901a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.442+0000 7fdf95e00640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf901a2d10 con 0x7fdf901080d0 2024-02-17T12:01:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.442+0000 7fdf95e00640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf901a2e50 con 0x7fdf90107710 2024-02-17T12:01:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.442+0000 7fdf95e00640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf901a3010 con 0x7fdf9010d720 2024-02-17T12:01:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.442+0000 7fdf8f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf90107710 0x7fdf9019dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.442+0000 7fdf8f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf90107710 0x7fdf9019dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55366/0 (socket says 172.21.15.94:55366) 2024-02-17T12:01:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.442+0000 7fdf8f7fe640 1 -- 172.21.15.94:0/2170502561 learned_addr learned my addr 172.21.15.94:0/2170502561 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.443+0000 7fdf8f7fe640 1 -- 172.21.15.94:0/2170502561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf9010d720 msgr2=0x7fdf901a2710 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-17T12:01:47.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.443+0000 7fdf8f7fe640 1 --2- 172.21.15.94:0/2170502561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf9010d720 0x7fdf901a2710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.443+0000 7fdf8f7fe640 1 -- 172.21.15.94:0/2170502561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf901080d0 msgr2=0x7fdf9019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.443+0000 7fdf8f7fe640 1 --2- 172.21.15.94:0/2170502561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf901080d0 0x7fdf9019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.443+0000 7fdf8f7fe640 1 -- 172.21.15.94:0/2170502561 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf84009560 con 0x7fdf90107710 2024-02-17T12:01:47.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.443+0000 7fdf8f7fe640 1 --2- 172.21.15.94:0/2170502561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf90107710 0x7fdf9019dea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000d7a0 tx=0x7fdf8000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.444+0000 7fdf8cff9640 1 -- 172.21.15.94:0/2170502561 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80009c30 con 0x7fdf90107710 2024-02-17T12:01:47.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.444+0000 7fdf8cff9640 1 -- 172.21.15.94:0/2170502561 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf8000deb0 con 0x7fdf90107710 2024-02-17T12:01:47.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.444+0000 7fdf8cff9640 1 -- 172.21.15.94:0/2170502561 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80004ea0 con 0x7fdf90107710 2024-02-17T12:01:47.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.444+0000 7fdf95e00640 1 -- 172.21.15.94:0/2170502561 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf901a32f0 con 0x7fdf90107710 2024-02-17T12:01:47.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.444+0000 7fdf95e00640 1 -- 172.21.15.94:0/2170502561 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9006ad80 con 0x7fdf90107710 2024-02-17T12:01:47.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.447+0000 7fdf8cff9640 1 -- 172.21.15.94:0/2170502561 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fdf80005020 con 0x7fdf90107710 2024-02-17T12:01:47.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.447+0000 7fdf989cd740 1 Processor -- start 2024-02-17T12:01:47.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.447+0000 7fdf989cd740 1 -- start start 2024-02-17T12:01:47.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.447+0000 7fdf989cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d843630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.447+0000 7fdf989cd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 0x55931d84c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.447+0000 7fdf989cd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d84eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.447+0000 7fdf989cd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931d7fe960 con 0x55931d843b70 2024-02-17T12:01:47.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.447+0000 7fdf989cd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931d7b2310 con 0x55931d84c750 2024-02-17T12:01:47.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.447+0000 7fdf989cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931d7af840 con 0x55931d843250 2024-02-17T12:01:47.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.448+0000 7fdf8effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d84eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.448+0000 7fdf8effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d84eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55382/0 (socket says 172.21.15.94:55382) 2024-02-17T12:01:47.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.448+0000 7fdf8effd640 1 -- 172.21.15.94:0/2815441909 learned_addr learned my addr 172.21.15.94:0/2815441909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.448+0000 7fdf8f7fe640 1 --2- 172.21.15.94:0/2815441909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 0x55931d84c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.448+0000 7fdf8effd640 1 -- 172.21.15.94:0/2815441909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 msgr2=0x55931d843630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.448+0000 7fdf8effd640 1 --2- 172.21.15.94:0/2815441909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d843630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.448+0000 7fdf8effd640 1 -- 172.21.15.94:0/2815441909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 msgr2=0x55931d84c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.448+0000 7fdf8effd640 1 --2- 172.21.15.94:0/2815441909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 0x55931d84c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.448+0000 7fdf8effd640 1 -- 172.21.15.94:0/2815441909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55931d76d060 con 0x55931d84c750 2024-02-17T12:01:47.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.449+0000 7fdf8effd640 1 --2- 172.21.15.94:0/2815441909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d84eb40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf84005c20 tx=0x7fdf8402c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bca28f19b641a86 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.449+0000 7fdf8cff9640 1 --2- 172.21.15.94:0/2170502561 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdf780a22b0 0x7fdf780a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.449+0000 7fdf75ffb640 1 -- 172.21.15.94:0/2815441909 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf84004740 con 0x55931d84c750 2024-02-17T12:01:47.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.449+0000 7fdf75ffb640 1 -- 172.21.15.94:0/2815441909 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf840048a0 con 0x55931d84c750 2024-02-17T12:01:47.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.449+0000 7fdf75ffb640 1 -- 172.21.15.94:0/2815441909 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf84005050 con 0x55931d84c750 2024-02-17T12:01:47.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.449+0000 7fdf8ffff640 1 --2- 172.21.15.94:0/2170502561 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdf780a22b0 0x7fdf780a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.449+0000 7fdf8cff9640 1 -- 172.21.15.94:0/2170502561 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fdf800cdac0 con 0x7fdf90107710 2024-02-17T12:01:47.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.450+0000 7fdf989cd740 1 -- 172.21.15.94:0/2815441909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 msgr2=0x55931d84eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.450+0000 7fdf989cd740 1 --2- 172.21.15.94:0/2815441909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d84eb40 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf84005c20 tx=0x7fdf8402c5d0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.450+0000 7fdf989cd740 1 -- 172.21.15.94:0/2815441909 shutdown_connections 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.450+0000 7fdf989cd740 1 --2- 172.21.15.94:0/2815441909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d84eb40 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.450+0000 7fdf989cd740 1 --2- 172.21.15.94:0/2815441909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 0x55931d84c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.450+0000 7fdf989cd740 1 --2- 172.21.15.94:0/2815441909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d843630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.450+0000 7fdf989cd740 1 -- 172.21.15.94:0/2815441909 >> 172.21.15.94:0/2815441909 conn(0x55931d83d980 msgr2=0x55931d83dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.450+0000 7fdf989cd740 1 -- 172.21.15.94:0/2815441909 shutdown_connections 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.450+0000 7fdf8ffff640 1 --2- 172.21.15.94:0/2170502561 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdf780a22b0 0x7fdf780a4770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005c40 tx=0x7fdf88005bd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.450+0000 7fdf989cd740 1 -- 172.21.15.94:0/2815441909 wait complete. 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.450+0000 7fdf989cd740 1 -- start start 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf989cd740 1 Processor -- start 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf989cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d849720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf989cd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d843b70 0x55931d849c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf989cd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d84c750 0x55931d84a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf989cd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931d7ab020 con 0x55931d84c750 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf989cd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931d47e430 con 0x55931d843b70 2024-02-17T12:01:47.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf989cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931d43ae00 con 0x55931d843250 2024-02-17T12:01:47.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf989cd740 10 client.0 _ll_register_callbacks cb 0x55931d752cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:47.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf8effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d843b70 0x55931d849c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf8effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d843b70 0x55931d849c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55390/0 (socket says 172.21.15.94:55390) 2024-02-17T12:01:47.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf8effd640 1 -- 172.21.15.94:0/1512001475 learned_addr learned my addr 172.21.15.94:0/1512001475 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf8f7fe640 1 --2- 172.21.15.94:0/1512001475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d849720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf8ffff640 1 --2- 172.21.15.94:0/1512001475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d84c750 0x55931d84a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf8effd640 1 -- 172.21.15.94:0/1512001475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 msgr2=0x55931d849720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf8effd640 1 --2- 172.21.15.94:0/1512001475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d849720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf8effd640 1 -- 172.21.15.94:0/1512001475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d84c750 msgr2=0x55931d84a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf8effd640 1 --2- 172.21.15.94:0/1512001475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d84c750 0x55931d84a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.451+0000 7fdf8effd640 1 -- 172.21.15.94:0/1512001475 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf90114ee0 con 0x55931d843b70 2024-02-17T12:01:47.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.452+0000 7fdf8effd640 1 --2- 172.21.15.94:0/1512001475 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d843b70 0x55931d849c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdf8402c360 tx=0x7fdf8402c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62140fb89dd9a1ae server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.452+0000 7fdf6effd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:47.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.452+0000 7fdf6effd640 1 -- 172.21.15.94:0/1512001475 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf84004740 con 0x55931d843b70 2024-02-17T12:01:47.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.452+0000 7fdf6effd640 1 -- 172.21.15.94:0/1512001475 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fdf84004ff0 con 0x55931d843b70 2024-02-17T12:01:47.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.453+0000 7fdf989cd740 1 -- 172.21.15.94:0/1512001475 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55931d4aa7e0 con 0x55931d843b70 2024-02-17T12:01:47.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.453+0000 7fdf6effd640 1 -- 172.21.15.94:0/1512001475 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf84004500 con 0x55931d843b70 2024-02-17T12:01:47.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.453+0000 7fdf989cd740 10 client.25937 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:47.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.453+0000 7fdf989cd740 1 -- 172.21.15.94:0/1512001475 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55931d7a96a0 con 0x55931d843b70 2024-02-17T12:01:47.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.453+0000 7fdf6effd640 1 -- 172.21.15.94:0/1512001475 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55931d4aa7e0 con 0x55931d843b70 2024-02-17T12:01:47.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.453+0000 7fdf6effd640 1 -- 172.21.15.94:0/1512001475 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdf8403c070 con 0x55931d843b70 2024-02-17T12:01:47.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.454+0000 7fdf989cd740 10 client.25937 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:47.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.454+0000 7fdf989cd740 10 client.25937 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:01:47.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.454+0000 7fdf989cd740 20 client.25937 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:47.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.454+0000 7fdf989cd740 1 --2- 172.21.15.94:0/1512001475 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55931d85c760 0x55931d85eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.454+0000 7fdf989cd740 4 client.25937 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:47.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.454+0000 7fdf989cd740 1 -- 172.21.15.94:0/1512001475 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55931d4e7720 con 0x55931d85c760 2024-02-17T12:01:47.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.454+0000 7fdf8f7fe640 1 --2- 172.21.15.94:0/1512001475 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55931d85c760 0x55931d85eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.454+0000 7fdf8f7fe640 1 --2- 172.21.15.94:0/1512001475 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55931d85c760 0x55931d85eb80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bfe19b3869a799d8 server_cookie=f409c57b8ae514a8 in_seq=0 out_seq=0 2024-02-17T12:01:47.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.454+0000 7fdf6effd640 10 client.25937 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:01:47.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf6effd640 1 -- 172.21.15.94:0/1512001475 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55931d4e7720 con 0x55931d85c760 2024-02-17T12:01:47.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf6effd640 10 client.25937 handle_command_reply: tid=0 2024-02-17T12:01:47.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf989cd740 1 client.25937 shutdown 2024-02-17T12:01:47.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf989cd740 1 -- 172.21.15.94:0/1512001475 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d843b70 msgr2=0x55931d849c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf989cd740 1 --2- 172.21.15.94:0/1512001475 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d843b70 0x55931d849c60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdf8402c360 tx=0x7fdf8402c740 comp rx=0 tx=0).stop 2024-02-17T12:01:47.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf989cd740 1 -- 172.21.15.94:0/1512001475 shutdown_connections 2024-02-17T12:01:47.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf989cd740 1 --2- 172.21.15.94:0/1512001475 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55931d85c760 0x55931d85eb80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf989cd740 1 --2- 172.21.15.94:0/1512001475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d84c750 0x55931d84a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf989cd740 1 --2- 172.21.15.94:0/1512001475 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d843b70 0x55931d849c60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf989cd740 1 --2- 172.21.15.94:0/1512001475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d849720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf6effd640 0 client.25937 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:01:47.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf989cd740 1 -- 172.21.15.94:0/1512001475 >> 172.21.15.94:0/1512001475 conn(0x55931d83d980 msgr2=0x55931d83f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf989cd740 1 -- 172.21.15.94:0/1512001475 shutdown_connections 2024-02-17T12:01:47.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf989cd740 1 -- 172.21.15.94:0/1512001475 wait complete. 2024-02-17T12:01:47.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.455+0000 7fdf989cd740 20 client.25937 trim_cache size 0 max 16384 2024-02-17T12:01:47.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.474+0000 7fdf989cd740 1 Processor -- start 2024-02-17T12:01:47.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.474+0000 7fdf989cd740 1 -- start start 2024-02-17T12:01:47.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.474+0000 7fdf989cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d84be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.474+0000 7fdf989cd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 0x55931d849fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.474+0000 7fdf989cd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d8469a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.474+0000 7fdf989cd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931d730130 con 0x55931d843b70 2024-02-17T12:01:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.474+0000 7fdf989cd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931d8395c0 con 0x55931d84c750 2024-02-17T12:01:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.475+0000 7fdf989cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931d83e980 con 0x55931d843250 2024-02-17T12:01:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.475+0000 7fdf8effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 0x55931d849fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.475+0000 7fdf8effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 0x55931d849fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46620/0 (socket says 172.21.15.94:46620) 2024-02-17T12:01:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.475+0000 7fdf8effd640 1 -- 172.21.15.94:0/1978122685 learned_addr learned my addr 172.21.15.94:0/1978122685 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.475+0000 7fdf8f7fe640 1 --2- 172.21.15.94:0/1978122685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d84be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.475+0000 7fdf8ffff640 1 --2- 172.21.15.94:0/1978122685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d8469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.475+0000 7fdf8effd640 1 -- 172.21.15.94:0/1978122685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 msgr2=0x55931d84be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.475+0000 7fdf8effd640 1 --2- 172.21.15.94:0/1978122685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d84be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.475+0000 7fdf8effd640 1 -- 172.21.15.94:0/1978122685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 msgr2=0x55931d8469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.475+0000 7fdf8effd640 1 --2- 172.21.15.94:0/1978122685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d8469a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.475+0000 7fdf8effd640 1 -- 172.21.15.94:0/1978122685 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55931d745c20 con 0x55931d843b70 2024-02-17T12:01:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.476+0000 7fdf8effd640 1 --2- 172.21.15.94:0/1978122685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 0x55931d849fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdf8402c170 tx=0x7fdf8402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ced2c1c3a21b510 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.476+0000 7fdf757fa640 1 -- 172.21.15.94:0/1978122685 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8402caf0 con 0x55931d843b70 2024-02-17T12:01:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.476+0000 7fdf757fa640 1 -- 172.21.15.94:0/1978122685 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf8402cc50 con 0x55931d843b70 2024-02-17T12:01:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.476+0000 7fdf757fa640 1 -- 172.21.15.94:0/1978122685 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf84003ab0 con 0x55931d843b70 2024-02-17T12:01:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.476+0000 7fdf989cd740 1 -- 172.21.15.94:0/1978122685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 msgr2=0x55931d849fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.476+0000 7fdf989cd740 1 --2- 172.21.15.94:0/1978122685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 0x55931d849fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdf8402c170 tx=0x7fdf8402c5d0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.477+0000 7fdf989cd740 1 -- 172.21.15.94:0/1978122685 shutdown_connections 2024-02-17T12:01:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.477+0000 7fdf989cd740 1 --2- 172.21.15.94:0/1978122685 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d8469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.477+0000 7fdf989cd740 1 --2- 172.21.15.94:0/1978122685 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 0x55931d849fc0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.477+0000 7fdf989cd740 1 --2- 172.21.15.94:0/1978122685 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d84be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.477+0000 7fdf989cd740 1 -- 172.21.15.94:0/1978122685 >> 172.21.15.94:0/1978122685 conn(0x55931d83d980 msgr2=0x55931d863500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.477+0000 7fdf989cd740 1 -- 172.21.15.94:0/1978122685 shutdown_connections 2024-02-17T12:01:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.477+0000 7fdf989cd740 1 -- 172.21.15.94:0/1978122685 wait complete. 2024-02-17T12:01:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.477+0000 7fdf989cd740 1 -- start start 2024-02-17T12:01:47.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.477+0000 7fdf989cd740 1 Processor -- start 2024-02-17T12:01:47.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf989cd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d8522c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf989cd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 0x55931d852800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf989cd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d852d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf989cd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931d8b2e00 con 0x55931d843b70 2024-02-17T12:01:47.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf989cd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931d8b2f70 con 0x55931d84c750 2024-02-17T12:01:47.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf989cd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931d8b3130 con 0x55931d843250 2024-02-17T12:01:47.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf989cd740 10 client.0 _ll_register_callbacks cb 0x7fdf880093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:47.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf8f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d8522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf8ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d852d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf8f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d8522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39362/0 (socket says 172.21.15.94:39362) 2024-02-17T12:01:47.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf8f7fe640 1 -- 172.21.15.94:0/3165204626 learned_addr learned my addr 172.21.15.94:0/3165204626 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf8ffff640 1 -- 172.21.15.94:0/3165204626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 msgr2=0x55931d8522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf8ffff640 1 --2- 172.21.15.94:0/3165204626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d8522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf8ffff640 1 -- 172.21.15.94:0/3165204626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 msgr2=0x55931d852800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf8ffff640 1 --2- 172.21.15.94:0/3165204626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 0x55931d852800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.479+0000 7fdf8ffff640 1 -- 172.21.15.94:0/3165204626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55931d7a96a0 con 0x55931d84c750 2024-02-17T12:01:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.480+0000 7fdf8ffff640 1 --2- 172.21.15.94:0/3165204626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d852d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005fd0 tx=0x7fdf88004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f4a85372db750e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.480+0000 7fdf6effd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.480+0000 7fdf6effd640 1 -- 172.21.15.94:0/3165204626 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800ed30 con 0x55931d84c750 2024-02-17T12:01:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.480+0000 7fdf989cd740 1 -- 172.21.15.94:0/3165204626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf840048a0 con 0x55931d84c750 2024-02-17T12:01:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.480+0000 7fdf6effd640 1 -- 172.21.15.94:0/3165204626 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fdf88028640 con 0x55931d84c750 2024-02-17T12:01:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.480+0000 7fdf6effd640 1 -- 172.21.15.94:0/3165204626 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf88028800 con 0x55931d84c750 2024-02-17T12:01:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.480+0000 7fdf989cd740 10 client.25943 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:47.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.480+0000 7fdf989cd740 1 -- 172.21.15.94:0/3165204626 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55931d76b8c0 con 0x55931d84c750 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.480+0000 7fdf6effd640 1 -- 172.21.15.94:0/3165204626 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf840048a0 con 0x55931d84c750 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.481+0000 7fdf6effd640 1 -- 172.21.15.94:0/3165204626 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdf88018070 con 0x55931d84c750 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.481+0000 7fdf989cd740 10 client.25943 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.481+0000 7fdf989cd740 10 client.25943 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.481+0000 7fdf989cd740 20 client.25943 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.481+0000 7fdf989cd740 1 --2- 172.21.15.94:0/3165204626 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55931d8b4a20 0x55931d8b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.481+0000 7fdf989cd740 4 client.25943 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.481+0000 7fdf989cd740 1 -- 172.21.15.94:0/3165204626 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55931d536890 con 0x55931d8b4a20 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.481+0000 7fdf8f7fe640 1 --2- 172.21.15.94:0/3165204626 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55931d8b4a20 0x55931d8b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.481+0000 7fdf6effd640 10 client.25943 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.481+0000 7fdf8f7fe640 1 --2- 172.21.15.94:0/3165204626 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55931d8b4a20 0x55931d8b6e60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=13515dc838830aed server_cookie=7cda7d076524c2fb in_seq=0 out_seq=0 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf6effd640 1 -- 172.21.15.94:0/3165204626 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55931d536890 con 0x55931d8b4a20 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf6effd640 10 client.25943 handle_command_reply: tid=0 2024-02-17T12:01:47.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf989cd740 1 client.25943 shutdown 2024-02-17T12:01:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf989cd740 1 -- 172.21.15.94:0/3165204626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 msgr2=0x55931d852d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf989cd740 1 --2- 172.21.15.94:0/3165204626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d852d40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005fd0 tx=0x7fdf88004a40 comp rx=0 tx=0).stop 2024-02-17T12:01:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf989cd740 1 -- 172.21.15.94:0/3165204626 shutdown_connections 2024-02-17T12:01:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf989cd740 1 --2- 172.21.15.94:0/3165204626 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55931d8b4a20 0x55931d8b6e60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf989cd740 1 --2- 172.21.15.94:0/3165204626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55931d84c750 0x55931d852d40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf989cd740 1 --2- 172.21.15.94:0/3165204626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55931d843b70 0x55931d852800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf6effd640 0 client.25943 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:01:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf989cd740 1 --2- 172.21.15.94:0/3165204626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d843250 0x55931d8522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf989cd740 1 -- 172.21.15.94:0/3165204626 >> 172.21.15.94:0/3165204626 conn(0x55931d83d980 msgr2=0x55931d83c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf989cd740 1 -- 172.21.15.94:0/3165204626 shutdown_connections 2024-02-17T12:01:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf989cd740 1 -- 172.21.15.94:0/3165204626 wait complete. 2024-02-17T12:01:47.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.482+0000 7fdf989cd740 20 client.25943 trim_cache size 0 max 16384 2024-02-17T12:01:47.487 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:01:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.483+0000 7fdf95e00640 1 -- 172.21.15.94:0/2170502561 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdf780a22b0 msgr2=0x7fdf780a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.483+0000 7fdf95e00640 1 --2- 172.21.15.94:0/2170502561 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdf780a22b0 0x7fdf780a4770 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdf88005c40 tx=0x7fdf88005bd0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.483+0000 7fdf95e00640 1 -- 172.21.15.94:0/2170502561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf90107710 msgr2=0x7fdf9019dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.483+0000 7fdf95e00640 1 --2- 172.21.15.94:0/2170502561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf90107710 0x7fdf9019dea0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000d7a0 tx=0x7fdf8000dc60 comp rx=0 tx=0).stop 2024-02-17T12:01:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.483+0000 7fdf95e00640 1 -- 172.21.15.94:0/2170502561 shutdown_connections 2024-02-17T12:01:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.483+0000 7fdf95e00640 1 --2- 172.21.15.94:0/2170502561 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdf780a22b0 0x7fdf780a4770 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.483+0000 7fdf95e00640 1 --2- 172.21.15.94:0/2170502561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf9010d720 0x7fdf901a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.483+0000 7fdf95e00640 1 --2- 172.21.15.94:0/2170502561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdf901080d0 0x7fdf9019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.483+0000 7fdf95e00640 1 --2- 172.21.15.94:0/2170502561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdf90107710 0x7fdf9019dea0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.483+0000 7fdf95e00640 1 -- 172.21.15.94:0/2170502561 >> 172.21.15.94:0/2170502561 conn(0x7fdf90075620 msgr2=0x7fdf90075e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.483+0000 7fdf95e00640 1 -- 172.21.15.94:0/2170502561 shutdown_connections 2024-02-17T12:01:47.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.483+0000 7fdf95e00640 1 -- 172.21.15.94:0/2170502561 wait complete. 2024-02-17T12:01:47.495 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:01:47.495 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:01:47.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.650+0000 7fe688f3a640 1 -- 172.21.15.94:0/1390677611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe684100c50 msgr2=0x7fe684104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.650+0000 7fe688f3a640 1 --2- 172.21.15.94:0/1390677611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe684100c50 0x7fe684104c90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009930 tx=0x7fe66c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:01:47.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.650+0000 7fe688f3a640 1 -- 172.21.15.94:0/1390677611 shutdown_connections 2024-02-17T12:01:47.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.650+0000 7fe688f3a640 1 --2- 172.21.15.94:0/1390677611 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe684105370 0x7fe68410cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.650+0000 7fe688f3a640 1 --2- 172.21.15.94:0/1390677611 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe684100c50 0x7fe684104c90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.650+0000 7fe688f3a640 1 --2- 172.21.15.94:0/1390677611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6841002a0 0x7fe684100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.650+0000 7fe688f3a640 1 -- 172.21.15.94:0/1390677611 >> 172.21.15.94:0/1390677611 conn(0x7fe684076400 msgr2=0x7fe684076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.650+0000 7fe688f3a640 1 -- 172.21.15.94:0/1390677611 shutdown_connections 2024-02-17T12:01:47.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.650+0000 7fe688f3a640 1 -- 172.21.15.94:0/1390677611 wait complete. 2024-02-17T12:01:47.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.651+0000 7fe688f3a640 1 Processor -- start 2024-02-17T12:01:47.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.651+0000 7fe688f3a640 1 -- start start 2024-02-17T12:01:47.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe688f3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6841002a0 0x7fe684101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe688f3a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe684105370 0x7fe684102270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe688f3a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6841027b0 0x7fe6841a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe688f3a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6841a5fb0 con 0x7fe6841002a0 2024-02-17T12:01:47.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe688f3a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6841a6120 con 0x7fe6841027b0 2024-02-17T12:01:47.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe688f3a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6841a62e0 con 0x7fe684105370 2024-02-17T12:01:47.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe68259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6841002a0 0x7fe684101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe68259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6841002a0 0x7fe684101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46660/0 (socket says 172.21.15.94:46660) 2024-02-17T12:01:47.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe68259c640 1 -- 172.21.15.94:0/1708199026 learned_addr learned my addr 172.21.15.94:0/1708199026 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe67bfff640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe684105370 0x7fe684102270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe682d9d640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6841027b0 0x7fe6841a59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe68259c640 1 -- 172.21.15.94:0/1708199026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe684105370 msgr2=0x7fe684102270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe68259c640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe684105370 0x7fe684102270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe68259c640 1 -- 172.21.15.94:0/1708199026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6841027b0 msgr2=0x7fe6841a59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe68259c640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6841027b0 0x7fe6841a59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe68259c640 1 -- 172.21.15.94:0/1708199026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c009560 con 0x7fe6841002a0 2024-02-17T12:01:47.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.652+0000 7fe682d9d640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6841027b0 0x7fe6841a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:47.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.653+0000 7fe68259c640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6841002a0 0x7fe684101d30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe67400e9f0 tx=0x7fe67400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.653+0000 7fe67b7fe640 1 -- 172.21.15.94:0/1708199026 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674009700 con 0x7fe6841002a0 2024-02-17T12:01:47.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.653+0000 7fe67b7fe640 1 -- 172.21.15.94:0/1708199026 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe67400cd80 con 0x7fe6841002a0 2024-02-17T12:01:47.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.654+0000 7fe67b7fe640 1 -- 172.21.15.94:0/1708199026 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674018920 con 0x7fe6841002a0 2024-02-17T12:01:47.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.654+0000 7fe688f3a640 1 -- 172.21.15.94:0/1708199026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6841a65c0 con 0x7fe6841002a0 2024-02-17T12:01:47.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.654+0000 7fe688f3a640 1 -- 172.21.15.94:0/1708199026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6841a6aa0 con 0x7fe6841002a0 2024-02-17T12:01:47.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.656+0000 7fe68bbe5740 1 Processor -- start 2024-02-17T12:01:47.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.657+0000 7fe68bbe5740 1 -- start start 2024-02-17T12:01:47.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.657+0000 7fe68bbe5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec7fa60 0x56212ec7fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.657+0000 7fe68bbe5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec80380 0x56212ec88a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.657+0000 7fe68bbe5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec88f60 0x56212ec8b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.657+0000 7fe68bbe5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212ec32960 con 0x56212ec88f60 2024-02-17T12:01:47.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.657+0000 7fe68bbe5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212ebe6310 con 0x56212ec80380 2024-02-17T12:01:47.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.657+0000 7fe68bbe5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212ebe3840 con 0x56212ec7fa60 2024-02-17T12:01:47.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.657+0000 7fe67b7fe640 1 -- 172.21.15.94:0/1708199026 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe674018a80 con 0x7fe6841002a0 2024-02-17T12:01:47.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.658+0000 7fe682d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec7fa60 0x56212ec7fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.658+0000 7fe682d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec7fa60 0x56212ec7fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39386/0 (socket says 172.21.15.94:39386) 2024-02-17T12:01:47.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.658+0000 7fe682d9d640 1 -- 172.21.15.94:0/3694471862 learned_addr learned my addr 172.21.15.94:0/3694471862 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.658+0000 7fe67bfff640 1 --2- 172.21.15.94:0/3694471862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec88f60 0x56212ec8b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.658+0000 7fe67b7fe640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe6700a2430 0x7fe6700a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.658+0000 7fe682d9d640 1 -- 172.21.15.94:0/3694471862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec80380 msgr2=0x56212ec88a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.658+0000 7fe68259c640 1 --2- 172.21.15.94:0/3694471862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec80380 0x56212ec88a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.658+0000 7fe682d9d640 1 --2- 172.21.15.94:0/3694471862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec80380 0x56212ec88a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.658+0000 7fe682d9d640 1 -- 172.21.15.94:0/3694471862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec88f60 msgr2=0x56212ec8b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.658+0000 7fe682d9d640 1 --2- 172.21.15.94:0/3694471862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec88f60 0x56212ec8b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.658+0000 7fe682d9d640 1 -- 172.21.15.94:0/3694471862 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56212eba1060 con 0x56212ec7fa60 2024-02-17T12:01:47.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.659+0000 7fe67b7fe640 1 -- 172.21.15.94:0/1708199026 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe674014070 con 0x7fe6841002a0 2024-02-17T12:01:47.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.659+0000 7fe682d9d640 1 --2- 172.21.15.94:0/3694471862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec7fa60 0x56212ec7fe40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe67c010710 tx=0x7fe67c010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54c74f157f163a46 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.659+0000 7fe682d9d640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe6700a2430 0x7fe6700a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.659+0000 7fe678ff9640 1 -- 172.21.15.94:0/3694471862 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c020040 con 0x56212ec7fa60 2024-02-17T12:01:47.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.659+0000 7fe678ff9640 1 -- 172.21.15.94:0/3694471862 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe67c00ed70 con 0x56212ec7fa60 2024-02-17T12:01:47.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.659+0000 7fe678ff9640 1 -- 172.21.15.94:0/3694471862 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c01c9d0 con 0x56212ec7fa60 2024-02-17T12:01:47.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.660+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3694471862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec7fa60 msgr2=0x56212ec7fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.660+0000 7fe682d9d640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe6700a2430 0x7fe6700a48f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe67c01b000 tx=0x7fe67c0113e0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.660+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3694471862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec7fa60 0x56212ec7fe40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe67c010710 tx=0x7fe67c010bd0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.660+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3694471862 shutdown_connections 2024-02-17T12:01:47.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.660+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3694471862 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec88f60 0x56212ec8b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.660+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3694471862 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec80380 0x56212ec88a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.660+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3694471862 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec7fa60 0x56212ec7fe40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.660+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3694471862 >> 172.21.15.94:0/3694471862 conn(0x56212ec7a190 msgr2=0x56212ec7a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.660+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3694471862 shutdown_connections 2024-02-17T12:01:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.660+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3694471862 wait complete. 2024-02-17T12:01:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.660+0000 7fe68bbe5740 1 -- start start 2024-02-17T12:01:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68bbe5740 1 Processor -- start 2024-02-17T12:01:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68bbe5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec7fa60 0x56212ec86090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68bbe5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec80380 0x56212ec865d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68bbe5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec88f60 0x56212ec86b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68bbe5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212ebdf020 con 0x56212ec88f60 2024-02-17T12:01:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68bbe5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212e8b2430 con 0x56212ec7fa60 2024-02-17T12:01:47.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68bbe5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212e86ee00 con 0x56212ec80380 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68bbe5740 10 client.0 _ll_register_callbacks cb 0x56212eb86cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec80380 0x56212ec865d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec7fa60 0x56212ec86090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec80380 0x56212ec865d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39392/0 (socket says 172.21.15.94:39392) 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe67bfff640 1 -- 172.21.15.94:0/3293586300 learned_addr learned my addr 172.21.15.94:0/3293586300 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe682d9d640 1 --2- 172.21.15.94:0/3293586300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec88f60 0x56212ec86b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68259c640 1 -- 172.21.15.94:0/3293586300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec80380 msgr2=0x56212ec865d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68259c640 1 --2- 172.21.15.94:0/3293586300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec80380 0x56212ec865d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68259c640 1 -- 172.21.15.94:0/3293586300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec88f60 msgr2=0x56212ec86b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68259c640 1 --2- 172.21.15.94:0/3293586300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec88f60 0x56212ec86b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.661+0000 7fe68259c640 1 -- 172.21.15.94:0/3293586300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c010370 con 0x56212ec7fa60 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe68259c640 1 --2- 172.21.15.94:0/3293586300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec7fa60 0x56212ec86090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe6740cba30 tx=0x7fe67409ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d81871c2f95b831 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:47.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe669ffb640 1 -- 172.21.15.94:0/3293586300 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6740973f0 con 0x56212ec7fa60 2024-02-17T12:01:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe669ffb640 1 -- 172.21.15.94:0/3293586300 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe674097550 con 0x56212ec7fa60 2024-02-17T12:01:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3293586300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56212e8de7e0 con 0x56212ec7fa60 2024-02-17T12:01:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe669ffb640 1 -- 172.21.15.94:0/3293586300 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe6740d00e0 con 0x56212ec7fa60 2024-02-17T12:01:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe68bbe5740 10 client.25952 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3293586300 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56212ebdd6a0 con 0x56212ec7fa60 2024-02-17T12:01:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe669ffb640 1 -- 172.21.15.94:0/3293586300 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56212e8de7e0 con 0x56212ec7fa60 2024-02-17T12:01:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe669ffb640 1 -- 172.21.15.94:0/3293586300 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe6740aed20 con 0x56212ec7fa60 2024-02-17T12:01:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe68bbe5740 10 client.25952 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe68bbe5740 10 client.25952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe68bbe5740 20 client.25952 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3293586300 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56212ec90c20 0x56212ec93010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe68bbe5740 4 client.25952 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:47.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.662+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3293586300 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56212e91b720 con 0x56212ec90c20 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.663+0000 7fe67bfff640 1 --2- 172.21.15.94:0/3293586300 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56212ec90c20 0x56212ec93010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.663+0000 7fe669ffb640 10 client.25952 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.663+0000 7fe67bfff640 1 --2- 172.21.15.94:0/3293586300 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56212ec90c20 0x56212ec93010 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5a3edc7e413e1ea server_cookie=f40029341d255173 in_seq=0 out_seq=0 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe669ffb640 1 -- 172.21.15.94:0/3293586300 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56212e91b720 con 0x56212ec90c20 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe669ffb640 10 client.25952 handle_command_reply: tid=0 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe68bbe5740 1 client.25952 shutdown 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3293586300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec7fa60 msgr2=0x56212ec86090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3293586300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec7fa60 0x56212ec86090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe6740cba30 tx=0x7fe67409ad80 comp rx=0 tx=0).stop 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3293586300 shutdown_connections 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3293586300 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56212ec90c20 0x56212ec93010 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3293586300 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec88f60 0x56212ec86b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3293586300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec80380 0x56212ec865d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3293586300 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec7fa60 0x56212ec86090 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3293586300 >> 172.21.15.94:0/3293586300 conn(0x56212ec7a190 msgr2=0x56212ec7b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe669ffb640 0 client.25952 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3293586300 shutdown_connections 2024-02-17T12:01:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3293586300 wait complete. 2024-02-17T12:01:47.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.664+0000 7fe68bbe5740 20 client.25952 trim_cache size 0 max 16384 2024-02-17T12:01:47.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.684+0000 7fe68bbe5740 1 Processor -- start 2024-02-17T12:01:47.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68bbe5740 1 -- start start 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68bbe5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec7fa60 0x56212ec887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68bbe5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec80380 0x56212ec75310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68bbe5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec88f60 0x56212ec75850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68bbe5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212eb64130 con 0x56212ec7fa60 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec7fa60 0x56212ec887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec80380 0x56212ec75310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec80380 0x56212ec75310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39408/0 (socket says 172.21.15.94:39408) 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec7fa60 0x56212ec887c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46686/0 (socket says 172.21.15.94:46686) 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe67bfff640 1 -- 172.21.15.94:0/2409114589 learned_addr learned my addr 172.21.15.94:0/2409114589 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68bbe5740 1 -- 172.21.15.94:0/2409114589 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212ec7b3c0 con 0x56212ec88f60 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68bbe5740 1 -- 172.21.15.94:0/2409114589 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212ec8dc70 con 0x56212ec80380 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe682d9d640 1 --2- 172.21.15.94:0/2409114589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec88f60 0x56212ec75850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68259c640 1 -- 172.21.15.94:0/2409114589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec80380 msgr2=0x56212ec75310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68259c640 1 --2- 172.21.15.94:0/2409114589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec80380 0x56212ec75310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68259c640 1 -- 172.21.15.94:0/2409114589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec88f60 msgr2=0x56212ec75850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68259c640 1 --2- 172.21.15.94:0/2409114589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec88f60 0x56212ec75850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe68259c640 1 -- 172.21.15.94:0/2409114589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56212eb79c20 con 0x56212ec7fa60 2024-02-17T12:01:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe67bfff640 1 --2- 172.21.15.94:0/2409114589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec80380 0x56212ec75310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.685+0000 7fe682d9d640 1 --2- 172.21.15.94:0/2409114589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec88f60 0x56212ec75850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:01:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.686+0000 7fe68259c640 1 --2- 172.21.15.94:0/2409114589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec7fa60 0x56212ec887c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe67400e9c0 tx=0x7fe6740ce280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dded9783bb39cc4a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.686+0000 7fe66bfff640 1 -- 172.21.15.94:0/2409114589 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67409a8c0 con 0x56212ec7fa60 2024-02-17T12:01:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.686+0000 7fe66bfff640 1 -- 172.21.15.94:0/2409114589 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe674010b80 con 0x56212ec7fa60 2024-02-17T12:01:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.686+0000 7fe66bfff640 1 -- 172.21.15.94:0/2409114589 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67409a1d0 con 0x56212ec7fa60 2024-02-17T12:01:47.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.687+0000 7fe68bbe5740 1 -- 172.21.15.94:0/2409114589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec7fa60 msgr2=0x56212ec887c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.687+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/2409114589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec7fa60 0x56212ec887c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe67400e9c0 tx=0x7fe6740ce280 comp rx=0 tx=0).stop 2024-02-17T12:01:47.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.687+0000 7fe68bbe5740 1 -- 172.21.15.94:0/2409114589 shutdown_connections 2024-02-17T12:01:47.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.687+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/2409114589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec88f60 0x56212ec75850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.687+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/2409114589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec80380 0x56212ec75310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.687+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/2409114589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec7fa60 0x56212ec887c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.687+0000 7fe68bbe5740 1 -- 172.21.15.94:0/2409114589 >> 172.21.15.94:0/2409114589 conn(0x56212ec7a190 msgr2=0x56212ec81380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.688+0000 7fe68bbe5740 1 -- 172.21.15.94:0/2409114589 shutdown_connections 2024-02-17T12:01:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.688+0000 7fe68bbe5740 1 -- 172.21.15.94:0/2409114589 wait complete. 2024-02-17T12:01:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.688+0000 7fe68bbe5740 1 -- start start 2024-02-17T12:01:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.688+0000 7fe68bbe5740 1 Processor -- start 2024-02-17T12:01:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.689+0000 7fe68bbe5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec7fa60 0x56212ec75270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.689+0000 7fe68bbe5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec80380 0x56212ec757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.689+0000 7fe68bbe5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec88f60 0x56212ec73750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.689+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec80380 0x56212ec757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.689+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec80380 0x56212ec757b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46690/0 (socket says 172.21.15.94:46690) 2024-02-17T12:01:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.689+0000 7fe67bfff640 1 -- 172.21.15.94:0/3634547610 learned_addr learned my addr 172.21.15.94:0/3634547610 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.689+0000 7fe68259c640 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec7fa60 0x56212ec75270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.689+0000 7fe682d9d640 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec88f60 0x56212ec73750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.689+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3634547610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212ec73e50 con 0x56212ec80380 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.689+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3634547610 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212ec73fc0 con 0x56212ec7fa60 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.689+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3634547610 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56212ec74180 con 0x56212ec88f60 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.690+0000 7fe68bbe5740 10 client.0 _ll_register_callbacks cb 0x7fe67c009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.690+0000 7fe68259c640 1 -- 172.21.15.94:0/3634547610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec88f60 msgr2=0x56212ec73750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.690+0000 7fe68259c640 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec88f60 0x56212ec73750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.690+0000 7fe68259c640 1 -- 172.21.15.94:0/3634547610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec80380 msgr2=0x56212ec757b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.690+0000 7fe68259c640 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec80380 0x56212ec757b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.690+0000 7fe68259c640 1 -- 172.21.15.94:0/3634547610 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68410d750 con 0x56212ec7fa60 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.690+0000 7fe67bfff640 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec80380 0x56212ec757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.690+0000 7fe68259c640 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec7fa60 0x56212ec75270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe67409aa80 tx=0x7fe67409aeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae76e89f476b9cc1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.690+0000 7fe669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.690+0000 7fe669ffb640 1 -- 172.21.15.94:0/3634547610 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674099fb0 con 0x56212ec7fa60 2024-02-17T12:01:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.690+0000 7fe669ffb640 1 -- 172.21.15.94:0/3634547610 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe6740b9200 con 0x56212ec7fa60 2024-02-17T12:01:47.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.691+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3634547610 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe67c00ed70 con 0x56212ec7fa60 2024-02-17T12:01:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.691+0000 7fe669ffb640 1 -- 172.21.15.94:0/3634547610 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe67409a3e0 con 0x56212ec7fa60 2024-02-17T12:01:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.691+0000 7fe68bbe5740 10 client.25958 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.691+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3634547610 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x56212ebdd6a0 con 0x56212ec7fa60 2024-02-17T12:01:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.692+0000 7fe669ffb640 1 -- 172.21.15.94:0/3634547610 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe67409a8c0 con 0x56212ec7fa60 2024-02-17T12:01:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.692+0000 7fe669ffb640 1 -- 172.21.15.94:0/3634547610 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe6740b3c00 con 0x56212ec7fa60 2024-02-17T12:01:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.692+0000 7fe68bbe5740 10 client.25958 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.692+0000 7fe68bbe5740 10 client.25958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.692+0000 7fe68bbe5740 20 client.25958 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.692+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56212ece7f00 0x56212ecea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.692+0000 7fe68bbe5740 4 client.25958 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:01:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.692+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3634547610 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe67c00ed70 con 0x56212ece7f00 2024-02-17T12:01:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.692+0000 7fe67bfff640 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56212ece7f00 0x56212ecea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.692+0000 7fe669ffb640 10 client.25958 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe67bfff640 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56212ece7f00 0x56212ecea2f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c7d4af3b994a089 server_cookie=f04e545b8bd2fd94 in_seq=0 out_seq=0 2024-02-17T12:01:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe669ffb640 1 -- 172.21.15.94:0/3634547610 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe67c00ed70 con 0x56212ece7f00 2024-02-17T12:01:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe669ffb640 10 client.25958 handle_command_reply: tid=0 2024-02-17T12:01:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe68bbe5740 1 client.25958 shutdown 2024-02-17T12:01:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3634547610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec7fa60 msgr2=0x56212ec75270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec7fa60 0x56212ec75270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe67409aa80 tx=0x7fe67409aeb0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3634547610 shutdown_connections 2024-02-17T12:01:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56212ece7f00 0x56212ecea2f0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56212ec88f60 0x56212ec73750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe669ffb640 0 client.25958 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:47.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56212ec80380 0x56212ec757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe68bbe5740 1 --2- 172.21.15.94:0/3634547610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56212ec7fa60 0x56212ec75270 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3634547610 >> 172.21.15.94:0/3634547610 conn(0x56212ec7a190 msgr2=0x56212ec702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3634547610 shutdown_connections 2024-02-17T12:01:47.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe68bbe5740 1 -- 172.21.15.94:0/3634547610 wait complete. 2024-02-17T12:01:47.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.693+0000 7fe68bbe5740 20 client.25958 trim_cache size 0 max 16384 2024-02-17T12:01:47.697 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:01:47.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.695+0000 7fe688f3a640 1 -- 172.21.15.94:0/1708199026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe6700a2430 msgr2=0x7fe6700a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.695+0000 7fe688f3a640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe6700a2430 0x7fe6700a48f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe67c01b000 tx=0x7fe67c0113e0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.695+0000 7fe688f3a640 1 -- 172.21.15.94:0/1708199026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6841002a0 msgr2=0x7fe684101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:47.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.695+0000 7fe688f3a640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6841002a0 0x7fe684101d30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe67400e9f0 tx=0x7fe67400eeb0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.695+0000 7fe688f3a640 1 -- 172.21.15.94:0/1708199026 shutdown_connections 2024-02-17T12:01:47.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.695+0000 7fe688f3a640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe6700a2430 0x7fe6700a48f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.695+0000 7fe688f3a640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6841027b0 0x7fe6841a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.695+0000 7fe688f3a640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe684105370 0x7fe684102270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.695+0000 7fe688f3a640 1 --2- 172.21.15.94:0/1708199026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6841002a0 0x7fe684101d30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:47.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.695+0000 7fe688f3a640 1 -- 172.21.15.94:0/1708199026 >> 172.21.15.94:0/1708199026 conn(0x7fe684076400 msgr2=0x7fe6841040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:47.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.696+0000 7fe688f3a640 1 -- 172.21.15.94:0/1708199026 shutdown_connections 2024-02-17T12:01:47.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:47.696+0000 7fe688f3a640 1 -- 172.21.15.94:0/1708199026 wait complete. 2024-02-17T12:01:48.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:47 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3768775058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:01:48.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:47 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3768775058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:01:48.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:47 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3768775058' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:01:48.710 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:01:48.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:01:48.711 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:01:48.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.865+0000 7f280cd56640 1 -- 172.21.15.94:0/3068043784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2808100150 msgr2=0x7f280810c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:48.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.865+0000 7f280cd56640 1 --2- 172.21.15.94:0/3068043784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2808100150 0x7f280810c660 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2800009cd0 tx=0x7f280002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.866+0000 7f280cd56640 1 -- 172.21.15.94:0/3068043784 shutdown_connections 2024-02-17T12:01:48.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.866+0000 7f280cd56640 1 --2- 172.21.15.94:0/3068043784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2808100150 0x7f280810c660 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.866+0000 7f280cd56640 1 --2- 172.21.15.94:0/3068043784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28080ff790 0x7f28080ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.866+0000 7f280cd56640 1 --2- 172.21.15.94:0/3068043784 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28081057a0 0x7f2808105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.866+0000 7f280cd56640 1 -- 172.21.15.94:0/3068043784 >> 172.21.15.94:0/3068043784 conn(0x7f28080765a0 msgr2=0x7f28080fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:48.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.866+0000 7f280cd56640 1 -- 172.21.15.94:0/3068043784 shutdown_connections 2024-02-17T12:01:48.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.866+0000 7f280cd56640 1 -- 172.21.15.94:0/3068043784 wait complete. 2024-02-17T12:01:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.867+0000 7f280cd56640 1 Processor -- start 2024-02-17T12:01:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.871+0000 7f280cd56640 1 -- start start 2024-02-17T12:01:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.871+0000 7f280cd56640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28080ff790 0x7f280819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.871+0000 7f280cd56640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2808100150 0x7f280819e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.871+0000 7f280cd56640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28081057a0 0x7f28081a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.871+0000 7f280cd56640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28081a2cc0 con 0x7f28081057a0 2024-02-17T12:01:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.871+0000 7f280cd56640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28081a2e00 con 0x7f28080ff790 2024-02-17T12:01:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.871+0000 7f280cd56640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28081a2fc0 con 0x7f2808100150 2024-02-17T12:01:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.871+0000 7f280659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28080ff790 0x7f280819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.871+0000 7f280659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28080ff790 0x7f280819de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55500/0 (socket says 172.21.15.94:55500) 2024-02-17T12:01:48.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.872+0000 7f280659c640 1 -- 172.21.15.94:0/3449138742 learned_addr learned my addr 172.21.15.94:0/3449138742 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.872+0000 7f2805d9b640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2808100150 0x7f280819e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.872+0000 7f2806d9d640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28081057a0 0x7f28081a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.872+0000 7f280659c640 1 -- 172.21.15.94:0/3449138742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2808100150 msgr2=0x7f280819e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.872+0000 7f280659c640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2808100150 0x7f280819e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.872+0000 7f280659c640 1 -- 172.21.15.94:0/3449138742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28081057a0 msgr2=0x7f28081a26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.872+0000 7f280659c640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28081057a0 0x7f28081a26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.872+0000 7f280659c640 1 -- 172.21.15.94:0/3449138742 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800009900 con 0x7f28080ff790 2024-02-17T12:01:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.872+0000 7f280659c640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28080ff790 0x7f280819de00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27f800d980 tx=0x7f27f800de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.874+0000 7f27f77fe640 1 -- 172.21.15.94:0/3449138742 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8009c30 con 0x7f28080ff790 2024-02-17T12:01:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.874+0000 7f27f77fe640 1 -- 172.21.15.94:0/3449138742 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27f8009d90 con 0x7f28080ff790 2024-02-17T12:01:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.874+0000 7f27f77fe640 1 -- 172.21.15.94:0/3449138742 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800f720 con 0x7f28080ff790 2024-02-17T12:01:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.874+0000 7f280cd56640 1 -- 172.21.15.94:0/3449138742 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28081a32a0 con 0x7f28080ff790 2024-02-17T12:01:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.874+0000 7f280cd56640 1 -- 172.21.15.94:0/3449138742 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280806ad80 con 0x7f28080ff790 2024-02-17T12:01:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.876+0000 7f2806d9d640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28081057a0 0x7f28081a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:01:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.878+0000 7f280fa08740 1 Processor -- start 2024-02-17T12:01:48.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.880+0000 7f280fa08740 1 -- start start 2024-02-17T12:01:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.884+0000 7f280fa08740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4380 0x5614fc2c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.884+0000 7f280fa08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2c4d30 0x5614fc2cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.884+0000 7f280fa08740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2cd8b0 0x5614fc2cfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.884+0000 7f280fa08740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614fc232f90 con 0x5614fc2cd8b0 2024-02-17T12:01:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.884+0000 7f280fa08740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614fc2304c0 con 0x5614fc2c4380 2024-02-17T12:01:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.884+0000 7f280fa08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614fc22bca0 con 0x5614fc2c4d30 2024-02-17T12:01:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.885+0000 7f27f77fe640 1 -- 172.21.15.94:0/3449138742 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f27f8010460 con 0x7f28080ff790 2024-02-17T12:01:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.885+0000 7f280659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2c4d30 0x5614fc2cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:48.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.885+0000 7f280659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2c4d30 0x5614fc2cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39438/0 (socket says 172.21.15.94:39438) 2024-02-17T12:01:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.885+0000 7f280659c640 1 -- 172.21.15.94:0/4234225923 learned_addr learned my addr 172.21.15.94:0/4234225923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.886+0000 7f280659c640 1 -- 172.21.15.94:0/4234225923 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4380 msgr2=0x5614fc2c4760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:01:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.886+0000 7f280659c640 1 --2- 172.21.15.94:0/4234225923 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4380 0x5614fc2c4760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.886+0000 7f280659c640 1 -- 172.21.15.94:0/4234225923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2cd8b0 msgr2=0x5614fc2cfd00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:01:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.886+0000 7f280659c640 1 --2- 172.21.15.94:0/4234225923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2cd8b0 0x5614fc2cfd00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.886+0000 7f280659c640 1 -- 172.21.15.94:0/4234225923 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614fc1ee0b0 con 0x5614fc2c4d30 2024-02-17T12:01:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.886+0000 7f280659c640 1 --2- 172.21.15.94:0/4234225923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2c4d30 0x5614fc2cd2e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f27f80cb040 tx=0x7f27f80d3e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7f6bb54cf3223ff server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.887+0000 7f27f77fe640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f27f00a2400 0x7f27f00a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.887+0000 7f2806d9d640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f27f00a2400 0x7f27f00a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.887+0000 7f27f77fe640 1 -- 172.21.15.94:0/3449138742 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f27f80cda20 con 0x7f28080ff790 2024-02-17T12:01:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.887+0000 7f27f4ff9640 1 -- 172.21.15.94:0/4234225923 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f80d5d50 con 0x5614fc2c4d30 2024-02-17T12:01:48.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.887+0000 7f27f4ff9640 1 -- 172.21.15.94:0/4234225923 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27f80d8900 con 0x5614fc2c4d30 2024-02-17T12:01:48.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.888+0000 7f280fa08740 1 -- 172.21.15.94:0/4234225923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2c4d30 msgr2=0x5614fc2cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:48.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.888+0000 7f280fa08740 1 --2- 172.21.15.94:0/4234225923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2c4d30 0x5614fc2cd2e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f27f80cb040 tx=0x7f27f80d3e40 comp rx=0 tx=0).stop 2024-02-17T12:01:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.892+0000 7f280fa08740 1 -- 172.21.15.94:0/4234225923 shutdown_connections 2024-02-17T12:01:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.892+0000 7f280fa08740 1 --2- 172.21.15.94:0/4234225923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2cd8b0 0x5614fc2cfd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.892+0000 7f280fa08740 1 --2- 172.21.15.94:0/4234225923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2c4d30 0x5614fc2cd2e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.892+0000 7f280fa08740 1 --2- 172.21.15.94:0/4234225923 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4380 0x5614fc2c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.892+0000 7f280fa08740 1 -- 172.21.15.94:0/4234225923 >> 172.21.15.94:0/4234225923 conn(0x5614fc2be840 msgr2=0x5614fc2bec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.894+0000 7f2806d9d640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f27f00a2400 0x7f27f00a48c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2800006010 tx=0x7f2800005d80 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.894+0000 7f280fa08740 1 -- 172.21.15.94:0/4234225923 shutdown_connections 2024-02-17T12:01:48.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.895+0000 7f280fa08740 1 -- 172.21.15.94:0/4234225923 wait complete. 2024-02-17T12:01:48.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.895+0000 7f280fa08740 1 -- start start 2024-02-17T12:01:48.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.896+0000 7f280fa08740 1 Processor -- start 2024-02-17T12:01:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.896+0000 7f280fa08740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.896+0000 7f280fa08740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4d30 0x5614fc2cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.896+0000 7f280fa08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2cd8b0 0x5614fc2cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.896+0000 7f280fa08740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614fc1d0f40 con 0x5614fc2c4380 2024-02-17T12:01:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.896+0000 7f280fa08740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614fbeff4b0 con 0x5614fc2c4d30 2024-02-17T12:01:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.896+0000 7f280fa08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614fc1b1670 con 0x5614fc2cd8b0 2024-02-17T12:01:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.896+0000 7f280fa08740 10 client.0 _ll_register_callbacks cb 0x5614fc21ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:48.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.896+0000 7f2805d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4d30 0x5614fc2cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.896+0000 7f2805d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4d30 0x5614fc2cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55508/0 (socket says 172.21.15.94:55508) 2024-02-17T12:01:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.896+0000 7f2805d9b640 1 -- 172.21.15.94:0/1562751954 learned_addr learned my addr 172.21.15.94:0/1562751954 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.897+0000 7f2805d9b640 1 -- 172.21.15.94:0/1562751954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2cd8b0 msgr2=0x5614fc2cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.897+0000 7f2805d9b640 1 --2- 172.21.15.94:0/1562751954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2cd8b0 0x5614fc2cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.897+0000 7f2805d9b640 1 -- 172.21.15.94:0/1562751954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 msgr2=0x5614fc2ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.897+0000 7f2805d9b640 1 --2- 172.21.15.94:0/1562751954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.897+0000 7f2805d9b640 1 -- 172.21.15.94:0/1562751954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280806ad80 con 0x5614fc2c4d30 2024-02-17T12:01:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.897+0000 7f2805d9b640 1 --2- 172.21.15.94:0/1562751954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4d30 0x5614fc2cacf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00e6a0 tx=0x7f27fc00eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50a271e0ef3c6759 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.897+0000 7f27edffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:01:48.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.897+0000 7f27edffb640 1 -- 172.21.15.94:0/1562751954 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc013070 con 0x5614fc2c4d30 2024-02-17T12:01:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.898+0000 7f27edffb640 1 -- 172.21.15.94:0/1562751954 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f27fc00cb80 con 0x5614fc2c4d30 2024-02-17T12:01:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.898+0000 7f280fa08740 1 -- 172.21.15.94:0/1562751954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614fc205f40 con 0x5614fc2c4d30 2024-02-17T12:01:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.898+0000 7f27edffb640 1 -- 172.21.15.94:0/1562751954 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27fc00cd40 con 0x5614fc2c4d30 2024-02-17T12:01:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.898+0000 7f280fa08740 10 client.25967 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.898+0000 7f280fa08740 1 -- 172.21.15.94:0/1562751954 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5614fc22a320 con 0x5614fc2c4d30 2024-02-17T12:01:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.898+0000 7f27edffb640 1 -- 172.21.15.94:0/1562751954 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5614fc205f40 con 0x5614fc2c4d30 2024-02-17T12:01:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.898+0000 7f27edffb640 1 -- 172.21.15.94:0/1562751954 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f27fc01f070 con 0x5614fc2c4d30 2024-02-17T12:01:48.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.898+0000 7f280fa08740 10 client.25967 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.898+0000 7f280fa08740 10 client.25967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.898+0000 7f280fa08740 20 client.25967 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.898+0000 7f280fa08740 1 --2- 172.21.15.94:0/1562751954 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614fc2dde90 0x5614fc2e02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.898+0000 7f280fa08740 4 client.25967 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.898+0000 7f280fa08740 1 -- 172.21.15.94:0/1562751954 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614fc222310 con 0x5614fc2dde90 2024-02-17T12:01:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.899+0000 7f280659c640 1 --2- 172.21.15.94:0/1562751954 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614fc2dde90 0x5614fc2e02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.899+0000 7f280659c640 1 --2- 172.21.15.94:0/1562751954 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614fc2dde90 0x5614fc2e02b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a12fc73e0cf60f server_cookie=fa0145f8eb30778e in_seq=0 out_seq=0 2024-02-17T12:01:48.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.899+0000 7f27edffb640 10 client.25967 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.900+0000 7f27edffb640 1 -- 172.21.15.94:0/1562751954 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5614fc222310 con 0x5614fc2dde90 2024-02-17T12:01:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.900+0000 7f27edffb640 10 client.25967 handle_command_reply: tid=0 2024-02-17T12:01:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.900+0000 7f280fa08740 1 client.25967 shutdown 2024-02-17T12:01:48.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.900+0000 7f280fa08740 1 -- 172.21.15.94:0/1562751954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4d30 msgr2=0x5614fc2cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:48.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.900+0000 7f280fa08740 1 --2- 172.21.15.94:0/1562751954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4d30 0x5614fc2cacf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00e6a0 tx=0x7f27fc00eb60 comp rx=0 tx=0).stop 2024-02-17T12:01:48.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.900+0000 7f280fa08740 1 -- 172.21.15.94:0/1562751954 shutdown_connections 2024-02-17T12:01:48.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.900+0000 7f280fa08740 1 --2- 172.21.15.94:0/1562751954 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614fc2dde90 0x5614fc2e02b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.900+0000 7f280fa08740 1 --2- 172.21.15.94:0/1562751954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2cd8b0 0x5614fc2cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.900+0000 7f280fa08740 1 --2- 172.21.15.94:0/1562751954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4d30 0x5614fc2cacf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.900+0000 7f27edffb640 0 client.25967 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:48.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.900+0000 7f280fa08740 1 --2- 172.21.15.94:0/1562751954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.900+0000 7f280fa08740 1 -- 172.21.15.94:0/1562751954 >> 172.21.15.94:0/1562751954 conn(0x5614fc2be840 msgr2=0x5614fc2c07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:48.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.900+0000 7f280fa08740 1 -- 172.21.15.94:0/1562751954 shutdown_connections 2024-02-17T12:01:48.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.901+0000 7f280fa08740 1 -- 172.21.15.94:0/1562751954 wait complete. 2024-02-17T12:01:48.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.901+0000 7f280fa08740 20 client.25967 trim_cache size 0 max 16384 2024-02-17T12:01:48.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.919+0000 7f280fa08740 1 Processor -- start 2024-02-17T12:01:48.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.919+0000 7f280fa08740 1 -- start start 2024-02-17T12:01:48.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.919+0000 7f280fa08740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280fa08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2c4d30 0x5614fc2d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280fa08740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2cd8b0 0x5614fc2d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:48.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2c0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46732/0 (socket says 172.21.15.94:46732) 2024-02-17T12:01:48.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280659c640 1 -- 172.21.15.94:0/433527786 learned_addr learned my addr 172.21.15.94:0/433527786 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:48.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280fa08740 1 -- 172.21.15.94:0/433527786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614fc2da810 con 0x5614fc2c4380 2024-02-17T12:01:48.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280fa08740 1 -- 172.21.15.94:0/433527786 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614fc2b8da0 con 0x5614fc2cd8b0 2024-02-17T12:01:48.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280fa08740 1 -- 172.21.15.94:0/433527786 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614fc2d05e0 con 0x5614fc2c4d30 2024-02-17T12:01:48.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280659c640 1 -- 172.21.15.94:0/433527786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2c4d30 msgr2=0x5614fc2d4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:48.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280659c640 1 --2- 172.21.15.94:0/433527786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2c4d30 0x5614fc2d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280659c640 1 -- 172.21.15.94:0/433527786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2cd8b0 msgr2=0x5614fc2d4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:48.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f2806d9d640 1 --2- 172.21.15.94:0/433527786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2cd8b0 0x5614fc2d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:48.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280659c640 1 --2- 172.21.15.94:0/433527786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2cd8b0 0x5614fc2d4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280659c640 1 -- 172.21.15.94:0/433527786 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614fc1c6c00 con 0x5614fc2c4380 2024-02-17T12:01:48.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f2806d9d640 1 --2- 172.21.15.94:0/433527786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2cd8b0 0x5614fc2d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:01:48.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f280659c640 1 --2- 172.21.15.94:0/433527786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2c0580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009610 tx=0x7f27f8010e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f9087be1549e67 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:48.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f27effff640 1 -- 172.21.15.94:0/433527786 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f800b660 con 0x5614fc2c4380 2024-02-17T12:01:48.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f27effff640 1 -- 172.21.15.94:0/433527786 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27f80cdc30 con 0x5614fc2c4380 2024-02-17T12:01:48.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.920+0000 7f27effff640 1 -- 172.21.15.94:0/433527786 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f809ab20 con 0x5614fc2c4380 2024-02-17T12:01:48.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.921+0000 7f280fa08740 1 -- 172.21.15.94:0/433527786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 msgr2=0x5614fc2c0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:48.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.921+0000 7f280fa08740 1 --2- 172.21.15.94:0/433527786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2c0580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009610 tx=0x7f27f8010e30 comp rx=0 tx=0).stop 2024-02-17T12:01:48.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.921+0000 7f280fa08740 1 -- 172.21.15.94:0/433527786 shutdown_connections 2024-02-17T12:01:48.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.921+0000 7f280fa08740 1 --2- 172.21.15.94:0/433527786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2cd8b0 0x5614fc2d4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.921+0000 7f280fa08740 1 --2- 172.21.15.94:0/433527786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2c4d30 0x5614fc2d4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.921+0000 7f280fa08740 1 --2- 172.21.15.94:0/433527786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2c0580 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.921+0000 7f280fa08740 1 -- 172.21.15.94:0/433527786 >> 172.21.15.94:0/433527786 conn(0x5614fc2be840 msgr2=0x5614fc2bfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:48.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.921+0000 7f280fa08740 1 -- 172.21.15.94:0/433527786 shutdown_connections 2024-02-17T12:01:48.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.921+0000 7f280fa08740 1 -- 172.21.15.94:0/433527786 wait complete. 2024-02-17T12:01:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.922+0000 7f280fa08740 1 -- start start 2024-02-17T12:01:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.922+0000 7f280fa08740 1 Processor -- start 2024-02-17T12:01:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.922+0000 7f280fa08740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.922+0000 7f280fa08740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4d30 0x5614fc2d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.922+0000 7f280659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:48.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.922+0000 7f280fa08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2cd8b0 0x5614fc32e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.922+0000 7f280659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2d3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46734/0 (socket says 172.21.15.94:46734) 2024-02-17T12:01:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.922+0000 7f280659c640 1 -- 172.21.15.94:0/3956856541 learned_addr learned my addr 172.21.15.94:0/3956856541 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.922+0000 7f280fa08740 1 -- 172.21.15.94:0/3956856541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614fc32e940 con 0x5614fc2c4380 2024-02-17T12:01:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.922+0000 7f280fa08740 1 -- 172.21.15.94:0/3956856541 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614fc32eab0 con 0x5614fc2c4d30 2024-02-17T12:01:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.922+0000 7f280fa08740 1 -- 172.21.15.94:0/3956856541 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614fc32ec70 con 0x5614fc2cd8b0 2024-02-17T12:01:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.922+0000 7f280fa08740 10 client.0 _ll_register_callbacks cb 0x7f27f80d0f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:48.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f2805d9b640 1 --2- 172.21.15.94:0/3956856541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4d30 0x5614fc2d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f2806d9d640 1 --2- 172.21.15.94:0/3956856541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2cd8b0 0x5614fc32e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f280659c640 1 -- 172.21.15.94:0/3956856541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2cd8b0 msgr2=0x5614fc32e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f280659c640 1 --2- 172.21.15.94:0/3956856541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2cd8b0 0x5614fc32e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f280659c640 1 -- 172.21.15.94:0/3956856541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4d30 msgr2=0x5614fc2d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f280659c640 1 --2- 172.21.15.94:0/3956856541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4d30 0x5614fc2d4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f280659c640 1 -- 172.21.15.94:0/3956856541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614fc1ee0b0 con 0x5614fc2c4380 2024-02-17T12:01:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f280659c640 1 --2- 172.21.15.94:0/3956856541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2d3b40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f27f8012040 tx=0x7f27f80d7550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a83a97b89dc457 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:48.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f27edffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:01:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f27edffb640 1 -- 172.21.15.94:0/3956856541 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f809a1f0 con 0x5614fc2c4380 2024-02-17T12:01:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f27edffb640 1 -- 172.21.15.94:0/3956856541 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f27f809ab20 con 0x5614fc2c4380 2024-02-17T12:01:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f280fa08740 1 -- 172.21.15.94:0/3956856541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614fc251900 con 0x5614fc2c4380 2024-02-17T12:01:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f27edffb640 1 -- 172.21.15.94:0/3956856541 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27f8099f60 con 0x5614fc2c4380 2024-02-17T12:01:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f280fa08740 10 client.16503 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.923+0000 7f280fa08740 1 -- 172.21.15.94:0/3956856541 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5614fc1ec910 con 0x5614fc2c4380 2024-02-17T12:01:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.924+0000 7f27edffb640 1 -- 172.21.15.94:0/3956856541 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f27f80a6080 con 0x5614fc2c4380 2024-02-17T12:01:48.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.924+0000 7f27edffb640 1 -- 172.21.15.94:0/3956856541 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f27f8021d80 con 0x5614fc2c4380 2024-02-17T12:01:48.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.924+0000 7f280fa08740 10 client.16503 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:48.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.924+0000 7f280fa08740 10 client.16503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:48.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.924+0000 7f280fa08740 20 client.16503 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:48.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.924+0000 7f280fa08740 1 --2- 172.21.15.94:0/3956856541 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614fc335f20 0x5614fc338340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:48.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.924+0000 7f280fa08740 4 client.16503 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:01:48.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.924+0000 7f280fa08740 1 -- 172.21.15.94:0/3956856541 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5614fc251900 con 0x5614fc335f20 2024-02-17T12:01:48.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.924+0000 7f2805d9b640 1 --2- 172.21.15.94:0/3956856541 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614fc335f20 0x5614fc338340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.925+0000 7f27edffb640 10 client.16503 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.925+0000 7f2805d9b640 1 --2- 172.21.15.94:0/3956856541 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614fc335f20 0x5614fc338340 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=846997f34f49989b server_cookie=fbf0869ee86e1b86 in_seq=0 out_seq=0 2024-02-17T12:01:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.925+0000 7f27edffb640 1 -- 172.21.15.94:0/3956856541 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5614fc251900 con 0x5614fc335f20 2024-02-17T12:01:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.925+0000 7f27edffb640 10 client.16503 handle_command_reply: tid=0 2024-02-17T12:01:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.925+0000 7f280fa08740 1 client.16503 shutdown 2024-02-17T12:01:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.926+0000 7f280fa08740 1 -- 172.21.15.94:0/3956856541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 msgr2=0x5614fc2d3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:48.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.926+0000 7f280fa08740 1 --2- 172.21.15.94:0/3956856541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2d3b40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f27f8012040 tx=0x7f27f80d7550 comp rx=0 tx=0).stop 2024-02-17T12:01:48.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.926+0000 7f280fa08740 1 -- 172.21.15.94:0/3956856541 shutdown_connections 2024-02-17T12:01:48.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.926+0000 7f280fa08740 1 --2- 172.21.15.94:0/3956856541 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614fc335f20 0x5614fc338340 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.926+0000 7f280fa08740 1 --2- 172.21.15.94:0/3956856541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614fc2cd8b0 0x5614fc32e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.926+0000 7f280fa08740 1 --2- 172.21.15.94:0/3956856541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614fc2c4d30 0x5614fc2d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.926+0000 7f27edffb640 0 client.16503 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:48.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.926+0000 7f280fa08740 1 --2- 172.21.15.94:0/3956856541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614fc2c4380 0x5614fc2d3b40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.926+0000 7f280fa08740 1 -- 172.21.15.94:0/3956856541 >> 172.21.15.94:0/3956856541 conn(0x5614fc2be840 msgr2=0x5614fc2c0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:48.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.926+0000 7f280fa08740 1 -- 172.21.15.94:0/3956856541 shutdown_connections 2024-02-17T12:01:48.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.926+0000 7f280fa08740 1 -- 172.21.15.94:0/3956856541 wait complete. 2024-02-17T12:01:48.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.926+0000 7f280fa08740 20 client.16503 trim_cache size 0 max 16384 2024-02-17T12:01:48.938 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:01:48.938 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:01:48.939 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "5cd6c72e-1d9a-41c8-829b-e2f1e18e0df6", 2024-02-17T12:01:48.939 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:01:48.939 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:01:48.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.928+0000 7f280cd56640 1 -- 172.21.15.94:0/3449138742 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f27f00a2400 msgr2=0x7f27f00a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:48.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.929+0000 7f280cd56640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f27f00a2400 0x7f27f00a48c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2800006010 tx=0x7f2800005d80 comp rx=0 tx=0).stop 2024-02-17T12:01:48.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.929+0000 7f280cd56640 1 -- 172.21.15.94:0/3449138742 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28080ff790 msgr2=0x7f280819de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:48.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.929+0000 7f280cd56640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28080ff790 0x7f280819de00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27f800d980 tx=0x7f27f800de40 comp rx=0 tx=0).stop 2024-02-17T12:01:48.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.929+0000 7f280cd56640 1 -- 172.21.15.94:0/3449138742 shutdown_connections 2024-02-17T12:01:48.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.929+0000 7f280cd56640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f27f00a2400 0x7f27f00a48c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.929+0000 7f280cd56640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f28081057a0 0x7f28081a26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.929+0000 7f280cd56640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2808100150 0x7f280819e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.929+0000 7f280cd56640 1 --2- 172.21.15.94:0/3449138742 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f28080ff790 0x7f280819de00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:48.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.929+0000 7f280cd56640 1 -- 172.21.15.94:0/3449138742 >> 172.21.15.94:0/3449138742 conn(0x7f28080765a0 msgr2=0x7f2808102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:48.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.929+0000 7f280cd56640 1 -- 172.21.15.94:0/3449138742 shutdown_connections 2024-02-17T12:01:48.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:48.929+0000 7f280cd56640 1 -- 172.21.15.94:0/3449138742 wait complete. 2024-02-17T12:01:48.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:01:48.944 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.112+0000 7f845d28a640 1 -- 172.21.15.94:0/48400892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84581142d0 msgr2=0x7f84581166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.112+0000 7f845d28a640 1 --2- 172.21.15.94:0/48400892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84581142d0 0x7f84581166c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f8450009be0 tx=0x7f8450034700 comp rx=0 tx=0).stop 2024-02-17T12:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.112+0000 7f845d28a640 1 -- 172.21.15.94:0/48400892 shutdown_connections 2024-02-17T12:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.112+0000 7f845d28a640 1 --2- 172.21.15.94:0/48400892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84581142d0 0x7f84581166c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.112+0000 7f845d28a640 1 --2- 172.21.15.94:0/48400892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8458108aa0 0x7f8458113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.112+0000 7f845d28a640 1 --2- 172.21.15.94:0/48400892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8458108180 0x7f8458108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.112+0000 7f845d28a640 1 -- 172.21.15.94:0/48400892 >> 172.21.15.94:0/48400892 conn(0x7f845806c6d0 msgr2=0x7f845806cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:49.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.112+0000 7f845d28a640 1 -- 172.21.15.94:0/48400892 shutdown_connections 2024-02-17T12:01:49.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.112+0000 7f845d28a640 1 -- 172.21.15.94:0/48400892 wait complete. 2024-02-17T12:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.113+0000 7f845d28a640 1 Processor -- start 2024-02-17T12:01:49.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.113+0000 7f845d28a640 1 -- start start 2024-02-17T12:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f845d28a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8458108180 0x7f845819de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f845d28a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8458108aa0 0x7f845819e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f845d28a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84581142d0 0x7f84581a26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f845d28a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84581a2cf0 con 0x7f84581142d0 2024-02-17T12:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f845d28a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84581a2e60 con 0x7f8458108180 2024-02-17T12:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f845d28a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84581a3020 con 0x7f8458108aa0 2024-02-17T12:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f8456ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8458108180 0x7f845819de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f84577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84581142d0 0x7f84581a26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f84577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84581142d0 0x7f84581a26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46756/0 (socket says 172.21.15.94:46756) 2024-02-17T12:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f8456ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8458108180 0x7f845819de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55558/0 (socket says 172.21.15.94:55558) 2024-02-17T12:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f84577fe640 1 -- 172.21.15.94:0/3843467670 learned_addr learned my addr 172.21.15.94:0/3843467670 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f84567fc640 1 --2- 172.21.15.94:0/3843467670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8458108aa0 0x7f845819e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f8456ffd640 1 -- 172.21.15.94:0/3843467670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8458108aa0 msgr2=0x7f845819e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f8456ffd640 1 --2- 172.21.15.94:0/3843467670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8458108aa0 0x7f845819e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f8456ffd640 1 -- 172.21.15.94:0/3843467670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84581142d0 msgr2=0x7f84581a26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f8456ffd640 1 --2- 172.21.15.94:0/3843467670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84581142d0 0x7f84581a26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f8456ffd640 1 -- 172.21.15.94:0/3843467670 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8450009840 con 0x7f8458108180 2024-02-17T12:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.114+0000 7f8456ffd640 1 --2- 172.21.15.94:0/3843467670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8458108180 0x7f845819de30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f844c00d720 tx=0x7f844c00dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.115+0000 7f843ffff640 1 -- 172.21.15.94:0/3843467670 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c009c30 con 0x7f8458108180 2024-02-17T12:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.115+0000 7f843ffff640 1 -- 172.21.15.94:0/3843467670 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f844c004510 con 0x7f8458108180 2024-02-17T12:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.115+0000 7f845d28a640 1 -- 172.21.15.94:0/3843467670 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84581a3300 con 0x7f8458108180 2024-02-17T12:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.115+0000 7f843ffff640 1 -- 172.21.15.94:0/3843467670 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c003e40 con 0x7f8458108180 2024-02-17T12:01:49.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.115+0000 7f845d28a640 1 -- 172.21.15.94:0/3843467670 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84581aab80 con 0x7f8458108180 2024-02-17T12:01:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.118+0000 7f845fe56740 1 Processor -- start 2024-02-17T12:01:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.118+0000 7f845fe56740 1 -- start start 2024-02-17T12:01:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.118+0000 7f845fe56740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c662b0 0x5577e4c66690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.118+0000 7f843ffff640 1 -- 172.21.15.94:0/3843467670 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f844c003fa0 con 0x7f8458108180 2024-02-17T12:01:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.118+0000 7f845fe56740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c66bd0 0x5577e4c6f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.118+0000 7f845fe56740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 0x5577e4c71d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.118+0000 7f845fe56740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e4bd5280 con 0x5577e4c66bd0 2024-02-17T12:01:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.118+0000 7f845fe56740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e4bd27b0 con 0x5577e4c6f9a0 2024-02-17T12:01:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.118+0000 7f845fe56740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e4bcdf90 con 0x5577e4c662b0 2024-02-17T12:01:49.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.118+0000 7f8456ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c66bd0 0x5577e4c6f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.118+0000 7f8456ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c66bd0 0x5577e4c6f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46768/0 (socket says 172.21.15.94:46768) 2024-02-17T12:01:49.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.118+0000 7f8456ffd640 1 -- 172.21.15.94:0/1082212854 learned_addr learned my addr 172.21.15.94:0/1082212854 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:49.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.118+0000 7f84577fe640 1 --2- 172.21.15.94:0/1082212854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c662b0 0x5577e4c66690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.119+0000 7f84567fc640 1 --2- 172.21.15.94:0/1082212854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 0x5577e4c71d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.119+0000 7f84577fe640 1 -- 172.21.15.94:0/1082212854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 msgr2=0x5577e4c71d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.119+0000 7f84577fe640 1 --2- 172.21.15.94:0/1082212854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 0x5577e4c71d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.119+0000 7f84577fe640 1 -- 172.21.15.94:0/1082212854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c66bd0 msgr2=0x5577e4c6f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.119+0000 7f84577fe640 1 --2- 172.21.15.94:0/1082212854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c66bd0 0x5577e4c6f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.119+0000 7f84577fe640 1 -- 172.21.15.94:0/1082212854 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577e4b8c570 con 0x5577e4c662b0 2024-02-17T12:01:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.119+0000 7f84577fe640 1 --2- 172.21.15.94:0/1082212854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c662b0 0x5577e4c66690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8450006010 tx=0x7f8450005ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2da3985868400532 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.119+0000 7f845d28a640 1 -- 172.21.15.94:0/1082212854 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845003a920 con 0x5577e4c662b0 2024-02-17T12:01:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.119+0000 7f843ffff640 1 --2- 172.21.15.94:0/3843467670 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f84400a22b0 0x7f84400a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.119+0000 7f845d28a640 1 -- 172.21.15.94:0/1082212854 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8450003e50 con 0x5577e4c662b0 2024-02-17T12:01:49.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.119+0000 7f845d28a640 1 -- 172.21.15.94:0/1082212854 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84500045c0 con 0x5577e4c662b0 2024-02-17T12:01:49.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.120+0000 7f8456ffd640 1 --2- 172.21.15.94:0/3843467670 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f84400a22b0 0x7f84400a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.120+0000 7f845fe56740 1 -- 172.21.15.94:0/1082212854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c662b0 msgr2=0x5577e4c66690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:49.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.120+0000 7f845fe56740 1 --2- 172.21.15.94:0/1082212854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c662b0 0x5577e4c66690 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8450006010 tx=0x7f8450005ec0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.121+0000 7f845fe56740 1 -- 172.21.15.94:0/1082212854 shutdown_connections 2024-02-17T12:01:49.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.121+0000 7f843ffff640 1 -- 172.21.15.94:0/3843467670 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f844c0cd730 con 0x7f8458108180 2024-02-17T12:01:49.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.121+0000 7f845fe56740 1 --2- 172.21.15.94:0/1082212854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 0x5577e4c71d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.121+0000 7f845fe56740 1 --2- 172.21.15.94:0/1082212854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c66bd0 0x5577e4c6f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.121+0000 7f845fe56740 1 --2- 172.21.15.94:0/1082212854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c662b0 0x5577e4c66690 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.121+0000 7f845fe56740 1 -- 172.21.15.94:0/1082212854 >> 172.21.15.94:0/1082212854 conn(0x5577e4c60b80 msgr2=0x5577e4c60f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:49.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.121+0000 7f845fe56740 1 -- 172.21.15.94:0/1082212854 shutdown_connections 2024-02-17T12:01:49.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.121+0000 7f845fe56740 1 -- 172.21.15.94:0/1082212854 wait complete. 2024-02-17T12:01:49.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.121+0000 7f8456ffd640 1 --2- 172.21.15.94:0/3843467670 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f84400a22b0 0x7f84400a4770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f844c096620 tx=0x7f844c0d3550 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:49.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.121+0000 7f845fe56740 1 -- start start 2024-02-17T12:01:49.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.121+0000 7f845fe56740 1 Processor -- start 2024-02-17T12:01:49.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.122+0000 7f845fe56740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c662b0 0x5577e4c6f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.122+0000 7f845fe56740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c66bd0 0x5577e4c6c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.122+0000 7f845fe56740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 0x5577e4c6ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.122+0000 7f84577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c662b0 0x5577e4c6f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.122+0000 7f84577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c662b0 0x5577e4c6f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46782/0 (socket says 172.21.15.94:46782) 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.122+0000 7f84567fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c66bd0 0x5577e4c6c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.122+0000 7f84577fe640 1 -- 172.21.15.94:0/1972716952 learned_addr learned my addr 172.21.15.94:0/1972716952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.122+0000 7f845fe56740 1 -- 172.21.15.94:0/1972716952 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e48a1430 con 0x5577e4c662b0 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.122+0000 7f845fe56740 1 -- 172.21.15.94:0/1972716952 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e485de00 con 0x5577e4c6f9a0 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.122+0000 7f845fe56740 1 -- 172.21.15.94:0/1972716952 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e4b52fd0 con 0x5577e4c66bd0 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.123+0000 7f84577fe640 1 --2- 172.21.15.94:0/1972716952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 0x5577e4c6ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.123+0000 7f84567fc640 1 -- 172.21.15.94:0/1972716952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 msgr2=0x5577e4c6ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.123+0000 7f84567fc640 1 --2- 172.21.15.94:0/1972716952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 0x5577e4c6ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.123+0000 7f84567fc640 1 -- 172.21.15.94:0/1972716952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c662b0 msgr2=0x5577e4c6f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.123+0000 7f84567fc640 1 --2- 172.21.15.94:0/1972716952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c662b0 0x5577e4c6f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.123+0000 7f84567fc640 1 -- 172.21.15.94:0/1972716952 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8458117100 con 0x5577e4c66bd0 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.123+0000 7f84567fc640 1 --2- 172.21.15.94:0/1972716952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c66bd0 0x5577e4c6c7c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84581160f0 tx=0x7f8448012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1f4b2e7a84bf7bd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.123+0000 7f843aff5640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:01:49.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.123+0000 7f845fe56740 10 client.0 _ll_register_callbacks cb 0x5577e4ba8fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.124+0000 7f845fe56740 1 -- 172.21.15.94:0/1972716952 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577e48cd7e0 con 0x5577e4c66bd0 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.124+0000 7f843aff5640 1 -- 172.21.15.94:0/1972716952 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448017070 con 0x5577e4c66bd0 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.124+0000 7f843aff5640 1 -- 172.21.15.94:0/1972716952 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8448004020 con 0x5577e4c66bd0 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.124+0000 7f843aff5640 1 -- 172.21.15.94:0/1972716952 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84480041e0 con 0x5577e4c66bd0 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.124+0000 7f845fe56740 10 client.26014 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.124+0000 7f845fe56740 1 -- 172.21.15.94:0/1972716952 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=23}) v3 -- 0x5577e4bcc610 con 0x5577e4c66bd0 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.125+0000 7f843aff5640 1 -- 172.21.15.94:0/1972716952 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8448004340 con 0x5577e4c66bd0 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.125+0000 7f845fe56740 10 client.26014 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.125+0000 7f845fe56740 10 client.26014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.125+0000 7f845fe56740 20 client.26014 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.125+0000 7f845fe56740 1 --2- 172.21.15.94:0/1972716952 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5577e4c7f450 0x5577e4c818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.125+0000 7f845fe56740 4 client.26014 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.125+0000 7f845fe56740 1 -- 172.21.15.94:0/1972716952 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577e490a720 con 0x5577e4c7f450 2024-02-17T12:01:49.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.125+0000 7f84577fe640 1 --2- 172.21.15.94:0/1972716952 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5577e4c7f450 0x5577e4c818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.125+0000 7f843aff5640 10 client.26014 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.125+0000 7f84577fe640 1 --2- 172.21.15.94:0/1972716952 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5577e4c7f450 0x5577e4c818c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dda277e5b454ce51 server_cookie=974e127f300afe1a in_seq=0 out_seq=0 2024-02-17T12:01:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.126+0000 7f843aff5640 1 -- 172.21.15.94:0/1972716952 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5577e490a720 con 0x5577e4c7f450 2024-02-17T12:01:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.126+0000 7f843aff5640 10 client.26014 handle_command_reply: tid=0 2024-02-17T12:01:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.126+0000 7f845fe56740 1 client.26014 shutdown 2024-02-17T12:01:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.127+0000 7f845fe56740 1 -- 172.21.15.94:0/1972716952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c66bd0 msgr2=0x5577e4c6c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.127+0000 7f845fe56740 1 --2- 172.21.15.94:0/1972716952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c66bd0 0x5577e4c6c7c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f84581160f0 tx=0x7f8448012820 comp rx=0 tx=0).stop 2024-02-17T12:01:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.127+0000 7f845fe56740 1 -- 172.21.15.94:0/1972716952 shutdown_connections 2024-02-17T12:01:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.127+0000 7f845fe56740 1 --2- 172.21.15.94:0/1972716952 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5577e4c7f450 0x5577e4c818c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.127+0000 7f845fe56740 1 --2- 172.21.15.94:0/1972716952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 0x5577e4c6ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.127+0000 7f843aff5640 0 client.26014 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.127+0000 7f845fe56740 1 --2- 172.21.15.94:0/1972716952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c66bd0 0x5577e4c6c7c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.127+0000 7f845fe56740 1 --2- 172.21.15.94:0/1972716952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c662b0 0x5577e4c6f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.127+0000 7f845fe56740 1 -- 172.21.15.94:0/1972716952 >> 172.21.15.94:0/1972716952 conn(0x5577e4c60b80 msgr2=0x5577e4c62920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:49.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.127+0000 7f845fe56740 1 -- 172.21.15.94:0/1972716952 shutdown_connections 2024-02-17T12:01:49.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.127+0000 7f845fe56740 1 -- 172.21.15.94:0/1972716952 wait complete. 2024-02-17T12:01:49.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.127+0000 7f845fe56740 20 client.26014 trim_cache size 0 max 16384 2024-02-17T12:01:49.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.151+0000 7f845fe56740 1 Processor -- start 2024-02-17T12:01:49.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.152+0000 7f845fe56740 1 -- start start 2024-02-17T12:01:49.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.152+0000 7f845fe56740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c662b0 0x5577e4c6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.152+0000 7f845fe56740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c66bd0 0x5577e4c6eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.152+0000 7f845fe56740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c6f9a0 0x5577e4c6f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.152+0000 7f84577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c662b0 0x5577e4c6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.152+0000 7f845fe56740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e4c7f250 con 0x5577e4c662b0 2024-02-17T12:01:49.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.152+0000 7f845fe56740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e4c86b10 con 0x5577e4c66bd0 2024-02-17T12:01:49.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.152+0000 7f845fe56740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e4c653a0 con 0x5577e4c6f9a0 2024-02-17T12:01:49.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.152+0000 7f84577fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c662b0 0x5577e4c6c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46784/0 (socket says 172.21.15.94:46784) 2024-02-17T12:01:49.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.152+0000 7f84577fe640 1 -- 172.21.15.94:0/3300874329 learned_addr learned my addr 172.21.15.94:0/3300874329 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:49.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.152+0000 7f84567fc640 1 --2- 172.21.15.94:0/3300874329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c66bd0 0x5577e4c6eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.153+0000 7f84577fe640 1 --2- 172.21.15.94:0/3300874329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c6f9a0 0x5577e4c6f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.153+0000 7f84577fe640 1 -- 172.21.15.94:0/3300874329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c6f9a0 msgr2=0x5577e4c6f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:49.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.153+0000 7f84577fe640 1 --2- 172.21.15.94:0/3300874329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c6f9a0 0x5577e4c6f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.153+0000 7f84577fe640 1 -- 172.21.15.94:0/3300874329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c66bd0 msgr2=0x5577e4c6eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:49.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.153+0000 7f84577fe640 1 --2- 172.21.15.94:0/3300874329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c66bd0 0x5577e4c6eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.153+0000 7f84577fe640 1 -- 172.21.15.94:0/3300874329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577e4b68ac0 con 0x5577e4c662b0 2024-02-17T12:01:49.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.153+0000 7f84577fe640 1 --2- 172.21.15.94:0/3300874329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c662b0 0x5577e4c6c760 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8450006010 tx=0x7f845003aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35153642b2e6b21e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:49.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.153+0000 7f843cff9640 1 -- 172.21.15.94:0/3300874329 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450003c70 con 0x5577e4c662b0 2024-02-17T12:01:49.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.153+0000 7f843cff9640 1 -- 172.21.15.94:0/3300874329 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8450055070 con 0x5577e4c662b0 2024-02-17T12:01:49.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.153+0000 7f843cff9640 1 -- 172.21.15.94:0/3300874329 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450004500 con 0x5577e4c662b0 2024-02-17T12:01:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.154+0000 7f845fe56740 1 -- 172.21.15.94:0/3300874329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c662b0 msgr2=0x5577e4c6c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.154+0000 7f845fe56740 1 --2- 172.21.15.94:0/3300874329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c662b0 0x5577e4c6c760 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f8450006010 tx=0x7f845003aea0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.154+0000 7f845fe56740 1 -- 172.21.15.94:0/3300874329 shutdown_connections 2024-02-17T12:01:49.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.154+0000 7f845fe56740 1 --2- 172.21.15.94:0/3300874329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c6f9a0 0x5577e4c6f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.154+0000 7f845fe56740 1 --2- 172.21.15.94:0/3300874329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c66bd0 0x5577e4c6eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.154+0000 7f845fe56740 1 --2- 172.21.15.94:0/3300874329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c662b0 0x5577e4c6c760 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.154+0000 7f845fe56740 1 -- 172.21.15.94:0/3300874329 >> 172.21.15.94:0/3300874329 conn(0x5577e4c60b80 msgr2=0x5577e4c6c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.154+0000 7f845fe56740 1 -- 172.21.15.94:0/3300874329 shutdown_connections 2024-02-17T12:01:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.154+0000 7f845fe56740 1 -- 172.21.15.94:0/3300874329 wait complete. 2024-02-17T12:01:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.155+0000 7f845fe56740 1 -- start start 2024-02-17T12:01:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.155+0000 7f845fe56740 1 Processor -- start 2024-02-17T12:01:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.155+0000 7f845fe56740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c662b0 0x5577e4ccfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.155+0000 7f845fe56740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c66bd0 0x5577e4cd0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.155+0000 7f845fe56740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 0x5577e4c738e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.155+0000 7f845fe56740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e4c73ea0 con 0x5577e4c66bd0 2024-02-17T12:01:49.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.155+0000 7f845fe56740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e4c74010 con 0x5577e4c6f9a0 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.155+0000 7f845fe56740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577e4c741d0 con 0x5577e4c662b0 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.155+0000 7f845fe56740 10 client.0 _ll_register_callbacks cb 0x7f8448005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.155+0000 7f84577fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c662b0 0x5577e4ccfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f84577fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c662b0 0x5577e4ccfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39504/0 (socket says 172.21.15.94:39504) 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f84577fe640 1 -- 172.21.15.94:0/1238469103 learned_addr learned my addr 172.21.15.94:0/1238469103 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f84567fc640 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c66bd0 0x5577e4cd0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f84577fe640 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 0x5577e4c738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f84567fc640 1 -- 172.21.15.94:0/1238469103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c662b0 msgr2=0x5577e4ccfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f84567fc640 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c662b0 0x5577e4ccfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f84567fc640 1 -- 172.21.15.94:0/1238469103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 msgr2=0x5577e4c738e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f84567fc640 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 0x5577e4c738e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f84567fc640 1 -- 172.21.15.94:0/1238469103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8450057040 con 0x5577e4c66bd0 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f84577fe640 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 0x5577e4c738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:01:49.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f84567fc640 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c66bd0 0x5577e4cd0330 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8448000c00 tx=0x7f84480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18e169b88a7e55c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f843aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f843aff5640 1 -- 172.21.15.94:0/1238469103 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8448011bd0 con 0x5577e4c66bd0 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f843aff5640 1 -- 172.21.15.94:0/1238469103 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8448000e40 con 0x5577e4c66bd0 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f845fe56740 1 -- 172.21.15.94:0/1238469103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8450003e50 con 0x5577e4c66bd0 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f843aff5640 1 -- 172.21.15.94:0/1238469103 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8448011d30 con 0x5577e4c66bd0 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f845fe56740 10 client.16521 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.156+0000 7f845fe56740 1 -- 172.21.15.94:0/1238469103 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5577e4b8e6a0 con 0x5577e4c66bd0 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.157+0000 7f843aff5640 1 -- 172.21.15.94:0/1238469103 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8450003e50 con 0x5577e4c66bd0 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.157+0000 7f843aff5640 1 -- 172.21.15.94:0/1238469103 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f844802d070 con 0x5577e4c66bd0 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.157+0000 7f845fe56740 10 client.16521 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.157+0000 7f845fe56740 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.157+0000 7f845fe56740 20 client.16521 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.157+0000 7f845fe56740 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5577e4cd7bf0 0x5577e4cda010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:49.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.157+0000 7f845fe56740 4 client.16521 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:01:49.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.157+0000 7f845fe56740 1 -- 172.21.15.94:0/1238469103 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5577e4959890 con 0x5577e4cd7bf0 2024-02-17T12:01:49.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.157+0000 7f84577fe640 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5577e4cd7bf0 0x5577e4cda010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:49.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.157+0000 7f843aff5640 10 client.16521 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:49.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.157+0000 7f84577fe640 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5577e4cd7bf0 0x5577e4cda010 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ffc8b247a67924 server_cookie=839660d97d099de4 in_seq=0 out_seq=0 2024-02-17T12:01:49.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f843aff5640 1 -- 172.21.15.94:0/1238469103 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5577e4959890 con 0x5577e4cd7bf0 2024-02-17T12:01:49.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f843aff5640 10 client.16521 handle_command_reply: tid=0 2024-02-17T12:01:49.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f845fe56740 1 client.16521 shutdown 2024-02-17T12:01:49.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f845fe56740 1 -- 172.21.15.94:0/1238469103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c66bd0 msgr2=0x5577e4cd0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:49.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f845fe56740 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c66bd0 0x5577e4cd0330 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f8448000c00 tx=0x7f84480046c0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f845fe56740 1 -- 172.21.15.94:0/1238469103 shutdown_connections 2024-02-17T12:01:49.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f845fe56740 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5577e4cd7bf0 0x5577e4cda010 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f843aff5640 0 client.16521 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:49.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f845fe56740 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5577e4c6f9a0 0x5577e4c738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f845fe56740 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5577e4c66bd0 0x5577e4cd0330 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f845fe56740 1 --2- 172.21.15.94:0/1238469103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5577e4c662b0 0x5577e4ccfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f845fe56740 1 -- 172.21.15.94:0/1238469103 >> 172.21.15.94:0/1238469103 conn(0x5577e4c60b80 msgr2=0x5577e4c6b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:49.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f845fe56740 1 -- 172.21.15.94:0/1238469103 shutdown_connections 2024-02-17T12:01:49.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f845fe56740 1 -- 172.21.15.94:0/1238469103 wait complete. 2024-02-17T12:01:49.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.158+0000 7f845fe56740 20 client.16521 trim_cache size 0 max 16384 2024-02-17T12:01:49.164 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:01:49.164 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:01:49.165 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:01:49.165 INFO:teuthology.orchestra.run.smithi094.stdout: "5cd6c72e-1d9a-41c8-829b-e2f1e18e0df6": { 2024-02-17T12:01:49.165 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:01:49.165 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "5cd6c72e-1d9a-41c8-829b-e2f1e18e0df6", 2024-02-17T12:01:49.165 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:01:49.165 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:01:49.165 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:01:49.165 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:01:49.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.160+0000 7f843dffb640 1 -- 172.21.15.94:0/3843467670 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f84400a22b0 msgr2=0x7f84400a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:49.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.160+0000 7f843dffb640 1 --2- 172.21.15.94:0/3843467670 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f84400a22b0 0x7f84400a4770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f844c096620 tx=0x7f844c0d3550 comp rx=0 tx=0).stop 2024-02-17T12:01:49.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.160+0000 7f843dffb640 1 -- 172.21.15.94:0/3843467670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8458108180 msgr2=0x7f845819de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:49.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.160+0000 7f843dffb640 1 --2- 172.21.15.94:0/3843467670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8458108180 0x7f845819de30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f844c00d720 tx=0x7f844c00dbe0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.161+0000 7f843dffb640 1 -- 172.21.15.94:0/3843467670 shutdown_connections 2024-02-17T12:01:49.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.161+0000 7f843dffb640 1 --2- 172.21.15.94:0/3843467670 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f84400a22b0 0x7f84400a4770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.161+0000 7f843dffb640 1 --2- 172.21.15.94:0/3843467670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84581142d0 0x7f84581a26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.161+0000 7f843dffb640 1 --2- 172.21.15.94:0/3843467670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8458108aa0 0x7f845819e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.161+0000 7f843dffb640 1 --2- 172.21.15.94:0/3843467670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8458108180 0x7f845819de30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:49.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.161+0000 7f843dffb640 1 -- 172.21.15.94:0/3843467670 >> 172.21.15.94:0/3843467670 conn(0x7f845806c6d0 msgr2=0x7f845806fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:49.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.161+0000 7f843dffb640 1 -- 172.21.15.94:0/3843467670 shutdown_connections 2024-02-17T12:01:49.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:01:49.161+0000 7f843dffb640 1 -- 172.21.15.94:0/3843467670 wait complete. 2024-02-17T12:01:49.173 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:48 smithi094 ceph-mon[32499]: pgmap v114: 97 pgs: 97 active+clean; 2.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 16 MiB/s wr, 995 op/s 2024-02-17T12:01:49.174 INFO:tasks.cephfs.filesystem:scrub status for tag:5cd6c72e-1d9a-41c8-829b-e2f1e18e0df6 - {'path': '/', 'tag': '5cd6c72e-1d9a-41c8-829b-e2f1e18e0df6', 'options': 'recursive,force'} 2024-02-17T12:01:49.302 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:48 smithi189 ceph-mon[36588]: pgmap v114: 97 pgs: 97 active+clean; 2.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 16 MiB/s wr, 995 op/s 2024-02-17T12:01:49.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:48 smithi125 ceph-mon[36819]: pgmap v114: 97 pgs: 97 active+clean; 2.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 16 MiB/s wr, 995 op/s 2024-02-17T12:01:50.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 210.0 s, 3658.1 tps, lat 27.321 ms stddev 30.907 2024-02-17T12:01:50.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:49 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:01:50.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:49 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:01:50.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:49 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:01:50.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:49 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:01:50.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:49 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:01:50.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:49 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:01:50.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:49 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:01:50.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:49 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:01:50.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:49 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:01:50.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:49 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:01:50.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:49 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:01:50.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:49 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:01:51.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:50 smithi189 ceph-mon[36588]: pgmap v115: 97 pgs: 97 active+clean; 2.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.9 MiB/s wr, 659 op/s 2024-02-17T12:01:51.434 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:50 smithi125 ceph-mon[36819]: pgmap v115: 97 pgs: 97 active+clean; 2.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.9 MiB/s wr, 659 op/s 2024-02-17T12:01:51.455 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:50 smithi094 ceph-mon[32499]: pgmap v115: 97 pgs: 97 active+clean; 2.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 9.9 MiB/s wr, 659 op/s 2024-02-17T12:01:52.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:52 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:01:52.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:52 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:01:52.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:52 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:01:53.705 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:53 smithi094 ceph-mon[32499]: pgmap v116: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 12 MiB/s wr, 887 op/s 2024-02-17T12:01:53.901 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:53 smithi189 ceph-mon[36588]: pgmap v116: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 12 MiB/s wr, 887 op/s 2024-02-17T12:01:53.934 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:53 smithi125 ceph-mon[36819]: pgmap v116: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 12 MiB/s wr, 887 op/s 2024-02-17T12:01:55.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 215.0 s, 3780.0 tps, lat 26.402 ms stddev 28.031 2024-02-17T12:01:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.262+0000 7fc4d159a640 1 -- 172.21.15.94:0/1528389929 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc032340 con 0x7fc4d4108810 2024-02-17T12:01:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.263+0000 7fc4d89ef640 1 -- 172.21.15.94:0/1528389929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4d4108810 msgr2=0x7fc4d4108c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.263+0000 7fc4d89ef640 1 --2- 172.21.15.94:0/1528389929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4d4108810 0x7fc4d4108c90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009860 tx=0x7fc4bc02bb70 comp rx=0 tx=0).stop 2024-02-17T12:01:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.263+0000 7fc4d89ef640 1 -- 172.21.15.94:0/1528389929 shutdown_connections 2024-02-17T12:01:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.263+0000 7fc4d89ef640 1 --2- 172.21.15.94:0/1528389929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d4116180 0x7fc4d4118590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.263+0000 7fc4d89ef640 1 --2- 172.21.15.94:0/1528389929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4d4108810 0x7fc4d4108c90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.263+0000 7fc4d89ef640 1 --2- 172.21.15.94:0/1528389929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4d410b830 0x7fc4d41082d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.263+0000 7fc4d89ef640 1 -- 172.21.15.94:0/1528389929 >> 172.21.15.94:0/1528389929 conn(0x7fc4d406c220 msgr2=0x7fc4d406c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.263+0000 7fc4d89ef640 1 -- 172.21.15.94:0/1528389929 shutdown_connections 2024-02-17T12:01:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.263+0000 7fc4d89ef640 1 -- 172.21.15.94:0/1528389929 wait complete. 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.264+0000 7fc4d89ef640 1 Processor -- start 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.264+0000 7fc4d89ef640 1 -- start start 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.264+0000 7fc4d89ef640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d4108810 0x7fc4d41a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.264+0000 7fc4d89ef640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4d410b830 0x7fc4d41a2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.264+0000 7fc4d89ef640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4d4116180 0x7fc4d41a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.264+0000 7fc4d89ef640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d41a6f80 con 0x7fc4d4108810 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.264+0000 7fc4d89ef640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d41a70f0 con 0x7fc4d4116180 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.264+0000 7fc4d89ef640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d41a72b0 con 0x7fc4d410b830 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.264+0000 7fc4d259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d4108810 0x7fc4d41a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.264+0000 7fc4d259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d4108810 0x7fc4d41a20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41342/0 (socket says 172.21.15.94:41342) 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.264+0000 7fc4d259c640 1 -- 172.21.15.94:0/3022817592 learned_addr learned my addr 172.21.15.94:0/3022817592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4d2d9d640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4d4116180 0x7fc4d41a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4d1d9b640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4d410b830 0x7fc4d41a2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4d259c640 1 -- 172.21.15.94:0/3022817592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4d410b830 msgr2=0x7fc4d41a2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4d259c640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4d410b830 0x7fc4d41a2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4d259c640 1 -- 172.21.15.94:0/3022817592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4d4116180 msgr2=0x7fc4d41a6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4d259c640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4d4116180 0x7fc4d41a6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4d259c640 1 -- 172.21.15.94:0/3022817592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc009560 con 0x7fc4d4108810 2024-02-17T12:01:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4d1d9b640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4d410b830 0x7fc4d41a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:01:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4d259c640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d4108810 0x7fc4d41a20c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400eb90 tx=0x7fc4c400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4cb7fe640 1 -- 172.21.15.94:0/3022817592 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40099b0 con 0x7fc4d4108810 2024-02-17T12:01:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4d89ef640 1 -- 172.21.15.94:0/3022817592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc009490 con 0x7fc4d4108810 2024-02-17T12:01:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4cb7fe640 1 -- 172.21.15.94:0/3022817592 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4c400ee90 con 0x7fc4d4108810 2024-02-17T12:01:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4cb7fe640 1 -- 172.21.15.94:0/3022817592 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40107a0 con 0x7fc4d4108810 2024-02-17T12:01:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.265+0000 7fc4d89ef640 1 -- 172.21.15.94:0/3022817592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d41aec30 con 0x7fc4d4108810 2024-02-17T12:01:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.269+0000 7fc4cb7fe640 1 -- 172.21.15.94:0/3022817592 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc4c4005020 con 0x7fc4d4108810 2024-02-17T12:01:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.269+0000 7fc4cb7fe640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc4c00a2350 0x7fc4c00a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.270+0000 7fc4db5bd740 1 Processor -- start 2024-02-17T12:01:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.270+0000 7fc4cb7fe640 1 -- 172.21.15.94:0/3022817592 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc4c4014070 con 0x7fc4d4108810 2024-02-17T12:01:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.270+0000 7fc4d1d9b640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc4c00a2350 0x7fc4c00a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.270+0000 7fc4db5bd740 1 -- start start 2024-02-17T12:01:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.270+0000 7fc4db5bd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe2ad0 0x556902fe1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.270+0000 7fc4db5bd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe1d60 0x556902fe21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4db5bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902fed6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4db5bd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556902f509f0 con 0x556902fe1d60 2024-02-17T12:01:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4db5bd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556902f4df20 con 0x556902fe2ad0 2024-02-17T12:01:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4db5bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556902f49700 con 0x556902feb2e0 2024-02-17T12:01:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4d259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe2ad0 0x556902fe1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4d259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe2ad0 0x556902fe1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:43674/0 (socket says 172.21.15.94:43674) 2024-02-17T12:01:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4d259c640 1 -- 172.21.15.94:0/3689297045 learned_addr learned my addr 172.21.15.94:0/3689297045 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4d1d9b640 1 --2- 172.21.15.94:0/3689297045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe1d60 0x556902fe21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4d2d9d640 1 --2- 172.21.15.94:0/3689297045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902fed6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4d1d9b640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc4c00a2350 0x7fc4c00a4810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc02c180 tx=0x7fc4bc035dd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4d2d9d640 1 -- 172.21.15.94:0/3689297045 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe2ad0 msgr2=0x556902fe1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4d2d9d640 1 --2- 172.21.15.94:0/3689297045 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe2ad0 0x556902fe1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4d2d9d640 1 -- 172.21.15.94:0/3689297045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe1d60 msgr2=0x556902fe21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4d2d9d640 1 --2- 172.21.15.94:0/3689297045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe1d60 0x556902fe21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.271+0000 7fc4d2d9d640 1 -- 172.21.15.94:0/3689297045 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556902edf7b0 con 0x556902feb2e0 2024-02-17T12:01:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.272+0000 7fc4d2d9d640 1 --2- 172.21.15.94:0/3689297045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902fed6d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc4d406a590 tx=0x7fc4cc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86c89f039f5f174f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.272+0000 7fc4c8ff9640 1 -- 172.21.15.94:0/3689297045 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc005220 con 0x556902feb2e0 2024-02-17T12:01:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.272+0000 7fc4c8ff9640 1 -- 172.21.15.94:0/3689297045 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4cc004c80 con 0x556902feb2e0 2024-02-17T12:01:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.272+0000 7fc4c8ff9640 1 -- 172.21.15.94:0/3689297045 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc019810 con 0x556902feb2e0 2024-02-17T12:01:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.273+0000 7fc4db5bd740 1 -- 172.21.15.94:0/3689297045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 msgr2=0x556902fed6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.273+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/3689297045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902fed6d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc4d406a590 tx=0x7fc4cc011810 comp rx=0 tx=0).stop 2024-02-17T12:01:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.273+0000 7fc4db5bd740 1 -- 172.21.15.94:0/3689297045 shutdown_connections 2024-02-17T12:01:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.273+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/3689297045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902fed6d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.273+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/3689297045 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe1d60 0x556902fe21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.273+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/3689297045 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe2ad0 0x556902fe1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.273+0000 7fc4db5bd740 1 -- 172.21.15.94:0/3689297045 >> 172.21.15.94:0/3689297045 conn(0x556902fa9550 msgr2=0x556902fdca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.273+0000 7fc4db5bd740 1 -- 172.21.15.94:0/3689297045 shutdown_connections 2024-02-17T12:01:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.273+0000 7fc4db5bd740 1 -- 172.21.15.94:0/3689297045 wait complete. 2024-02-17T12:01:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.273+0000 7fc4db5bd740 1 -- start start 2024-02-17T12:01:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.273+0000 7fc4db5bd740 1 Processor -- start 2024-02-17T12:01:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4db5bd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe1d60 0x556902fe82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4db5bd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe2ad0 0x556902fe8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4db5bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902fe8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4db5bd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556902eeebf0 con 0x556902fe2ad0 2024-02-17T12:01:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4db5bd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556902c1d250 con 0x556902fe1d60 2024-02-17T12:01:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4db5bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556902ecf200 con 0x556902feb2e0 2024-02-17T12:01:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4db5bd740 10 client.0 _ll_register_callbacks cb 0x556902f3dd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4d1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe2ad0 0x556902fe8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4d259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe1d60 0x556902fe82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4d1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe2ad0 0x556902fe8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41362/0 (socket says 172.21.15.94:41362) 2024-02-17T12:01:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4d1d9b640 1 -- 172.21.15.94:0/2996580331 learned_addr learned my addr 172.21.15.94:0/2996580331 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4d2d9d640 1 --2- 172.21.15.94:0/2996580331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902fe8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4d1d9b640 1 -- 172.21.15.94:0/2996580331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 msgr2=0x556902fe8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4d1d9b640 1 --2- 172.21.15.94:0/2996580331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902fe8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4d1d9b640 1 -- 172.21.15.94:0/2996580331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe1d60 msgr2=0x556902fe82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4d1d9b640 1 --2- 172.21.15.94:0/2996580331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe1d60 0x556902fe82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4d1d9b640 1 -- 172.21.15.94:0/2996580331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4cc009a20 con 0x556902fe2ad0 2024-02-17T12:01:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4d1d9b640 1 --2- 172.21.15.94:0/2996580331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe2ad0 0x556902fe8800 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009860 tx=0x7fc4bc02bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4213b8e733c840 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:01:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4b9ffb640 1 -- 172.21.15.94:0/2996580331 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc034a50 con 0x556902fe2ad0 2024-02-17T12:01:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4b9ffb640 1 -- 172.21.15.94:0/2996580331 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc4bc005240 con 0x556902fe2ad0 2024-02-17T12:01:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.274+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2996580331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556902c49600 con 0x556902fe2ad0 2024-02-17T12:01:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.275+0000 7fc4b9ffb640 1 -- 172.21.15.94:0/2996580331 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4bc005400 con 0x556902fe2ad0 2024-02-17T12:01:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.276+0000 7fc4db5bd740 10 client.16533 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.276+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2996580331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556902f47d80 con 0x556902fe2ad0 2024-02-17T12:01:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.276+0000 7fc4b9ffb640 1 -- 172.21.15.94:0/2996580331 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556902c49600 con 0x556902fe2ad0 2024-02-17T12:01:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.276+0000 7fc4b9ffb640 1 -- 172.21.15.94:0/2996580331 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc4bc0049e0 con 0x556902fe2ad0 2024-02-17T12:01:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.277+0000 7fc4db5bd740 10 client.16533 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.277+0000 7fc4db5bd740 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.277+0000 7fc4db5bd740 20 client.16533 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.277+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/2996580331 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556902ffb550 0x556902ffd970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.277+0000 7fc4db5bd740 4 client.16533 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:01:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.277+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2996580331 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556902c9a1b0 con 0x556902ffb550 2024-02-17T12:01:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.277+0000 7fc4d259c640 1 --2- 172.21.15.94:0/2996580331 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556902ffb550 0x556902ffd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.277+0000 7fc4b9ffb640 10 client.16533 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.277+0000 7fc4d259c640 1 --2- 172.21.15.94:0/2996580331 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556902ffb550 0x556902ffd970 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5de61a83e44f07c server_cookie=14f532aa87ce3bef in_seq=0 out_seq=0 2024-02-17T12:01:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.278+0000 7fc4b9ffb640 1 -- 172.21.15.94:0/2996580331 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556902c9a1b0 con 0x556902ffb550 2024-02-17T12:01:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.278+0000 7fc4b9ffb640 10 client.16533 handle_command_reply: tid=0 2024-02-17T12:01:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.278+0000 7fc4db5bd740 1 client.16533 shutdown 2024-02-17T12:01:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.279+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2996580331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe2ad0 msgr2=0x556902fe8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.279+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/2996580331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe2ad0 0x556902fe8800 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009860 tx=0x7fc4bc02bb70 comp rx=0 tx=0).stop 2024-02-17T12:01:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.279+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2996580331 shutdown_connections 2024-02-17T12:01:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.279+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/2996580331 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556902ffb550 0x556902ffd970 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.279+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/2996580331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902fe8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.279+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/2996580331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe2ad0 0x556902fe8800 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.279+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/2996580331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe1d60 0x556902fe82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.279+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2996580331 >> 172.21.15.94:0/2996580331 conn(0x556902fa9550 msgr2=0x556902fde7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.279+0000 7fc4b9ffb640 0 client.16533 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.279+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2996580331 shutdown_connections 2024-02-17T12:01:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.279+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2996580331 wait complete. 2024-02-17T12:01:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.279+0000 7fc4db5bd740 20 client.16533 trim_cache size 0 max 16384 2024-02-17T12:01:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.299+0000 7fc4db5bd740 1 Processor -- start 2024-02-17T12:01:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.299+0000 7fc4db5bd740 1 -- start start 2024-02-17T12:01:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.299+0000 7fc4db5bd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe1d60 0x556902fe8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.299+0000 7fc4db5bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902fe2ad0 0x556902fe8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.299+0000 7fc4db5bd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902feb2e0 0x556902fe6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.299+0000 7fc4db5bd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556902fdc310 con 0x556902fe1d60 2024-02-17T12:01:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.299+0000 7fc4db5bd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556902ffdd30 con 0x556902feb2e0 2024-02-17T12:01:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.299+0000 7fc4db5bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556902fe25a0 con 0x556902fe2ad0 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.299+0000 7fc4d259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe1d60 0x556902fe8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.299+0000 7fc4d259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe1d60 0x556902fe8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41372/0 (socket says 172.21.15.94:41372) 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.299+0000 7fc4d259c640 1 -- 172.21.15.94:0/3148496700 learned_addr learned my addr 172.21.15.94:0/3148496700 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.299+0000 7fc4d2d9d640 1 --2- 172.21.15.94:0/3148496700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902feb2e0 0x556902fe6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.300+0000 7fc4d1d9b640 1 --2- 172.21.15.94:0/3148496700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902fe2ad0 0x556902fe8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.300+0000 7fc4d259c640 1 -- 172.21.15.94:0/3148496700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902fe2ad0 msgr2=0x556902fe8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.300+0000 7fc4d259c640 1 --2- 172.21.15.94:0/3148496700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902fe2ad0 0x556902fe8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.300+0000 7fc4d259c640 1 -- 172.21.15.94:0/3148496700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902feb2e0 msgr2=0x556902fe6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.300+0000 7fc4d259c640 1 --2- 172.21.15.94:0/3148496700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902feb2e0 0x556902fe6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.300+0000 7fc4d259c640 1 -- 172.21.15.94:0/3148496700 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556902ee47b0 con 0x556902fe1d60 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.300+0000 7fc4d1d9b640 1 --2- 172.21.15.94:0/3148496700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902fe2ad0 0x556902fe8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.300+0000 7fc4d259c640 1 --2- 172.21.15.94:0/3148496700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe1d60 0x556902fe8830 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc4c409b270 tx=0x7fc4c400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69543f100ea11ee9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.300+0000 7fc4bbfff640 1 -- 172.21.15.94:0/3148496700 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4004d20 con 0x556902fe1d60 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.300+0000 7fc4bbfff640 1 -- 172.21.15.94:0/3148496700 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4c40995e0 con 0x556902fe1d60 2024-02-17T12:01:55.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.300+0000 7fc4bbfff640 1 -- 172.21.15.94:0/3148496700 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c409a730 con 0x556902fe1d60 2024-02-17T12:01:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.301+0000 7fc4db5bd740 1 -- 172.21.15.94:0/3148496700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe1d60 msgr2=0x556902fe8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.301+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/3148496700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe1d60 0x556902fe8830 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc4c409b270 tx=0x7fc4c400cf90 comp rx=0 tx=0).stop 2024-02-17T12:01:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.301+0000 7fc4db5bd740 1 -- 172.21.15.94:0/3148496700 shutdown_connections 2024-02-17T12:01:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.301+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/3148496700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902feb2e0 0x556902fe6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.301+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/3148496700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902fe2ad0 0x556902fe8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.301+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/3148496700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe1d60 0x556902fe8830 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.301+0000 7fc4db5bd740 1 -- 172.21.15.94:0/3148496700 >> 172.21.15.94:0/3148496700 conn(0x556902fa9550 msgr2=0x556902fe3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.302+0000 7fc4db5bd740 1 -- 172.21.15.94:0/3148496700 shutdown_connections 2024-02-17T12:01:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.302+0000 7fc4db5bd740 1 -- 172.21.15.94:0/3148496700 wait complete. 2024-02-17T12:01:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.302+0000 7fc4db5bd740 1 -- start start 2024-02-17T12:01:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.302+0000 7fc4db5bd740 1 Processor -- start 2024-02-17T12:01:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.303+0000 7fc4db5bd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe1d60 0x556902ff1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.303+0000 7fc4db5bd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe2ad0 0x556902ff2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.303+0000 7fc4db5bd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902ff03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.303+0000 7fc4db5bd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556902ff0ae0 con 0x556902fe2ad0 2024-02-17T12:01:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.303+0000 7fc4db5bd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556902ff0c50 con 0x556902fe1d60 2024-02-17T12:01:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.303+0000 7fc4db5bd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556902ff0d90 con 0x556902feb2e0 2024-02-17T12:01:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.303+0000 7fc4db5bd740 10 client.0 _ll_register_callbacks cb 0x7fc4cc009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:01:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.303+0000 7fc4d2d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902ff03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.303+0000 7fc4d2d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902ff03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56574/0 (socket says 172.21.15.94:56574) 2024-02-17T12:01:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.303+0000 7fc4d2d9d640 1 -- 172.21.15.94:0/2505432475 learned_addr learned my addr 172.21.15.94:0/2505432475 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:01:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.304+0000 7fc4d1d9b640 1 --2- 172.21.15.94:0/2505432475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe2ad0 0x556902ff2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.304+0000 7fc4d2d9d640 1 -- 172.21.15.94:0/2505432475 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe1d60 msgr2=0x556902ff1e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:01:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.304+0000 7fc4d2d9d640 1 --2- 172.21.15.94:0/2505432475 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe1d60 0x556902ff1e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.304+0000 7fc4d2d9d640 1 -- 172.21.15.94:0/2505432475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe2ad0 msgr2=0x556902ff2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:01:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.304+0000 7fc4d2d9d640 1 --2- 172.21.15.94:0/2505432475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe2ad0 0x556902ff2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.304+0000 7fc4d2d9d640 1 -- 172.21.15.94:0/2505432475 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d41aec30 con 0x556902feb2e0 2024-02-17T12:01:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.304+0000 7fc4d1d9b640 1 --2- 172.21.15.94:0/2505432475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe2ad0 0x556902ff2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:01:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.304+0000 7fc4d2d9d640 1 --2- 172.21.15.94:0/2505432475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902ff03e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc4d406a590 tx=0x7fc4cc0054b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c446eef10c38d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:01:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.304+0000 7fc4b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:01:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.305+0000 7fc4b9ffb640 1 -- 172.21.15.94:0/2505432475 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc003a00 con 0x556902feb2e0 2024-02-17T12:01:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.305+0000 7fc4b9ffb640 1 -- 172.21.15.94:0/2505432475 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc4cc005620 con 0x556902feb2e0 2024-02-17T12:01:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.305+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2505432475 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4cc004c80 con 0x556902feb2e0 2024-02-17T12:01:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.306+0000 7fc4b9ffb640 1 -- 172.21.15.94:0/2505432475 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4cc003b60 con 0x556902feb2e0 2024-02-17T12:01:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.306+0000 7fc4db5bd740 10 client.26038 fetch_fsmap learned FSMap version 23 2024-02-17T12:01:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.306+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2505432475 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x556902f0a540 con 0x556902feb2e0 2024-02-17T12:01:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.306+0000 7fc4b9ffb640 1 -- 172.21.15.94:0/2505432475 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc4cc004c80 con 0x556902feb2e0 2024-02-17T12:01:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.307+0000 7fc4b9ffb640 1 -- 172.21.15.94:0/2505432475 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc4cc021040 con 0x556902feb2e0 2024-02-17T12:01:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.307+0000 7fc4db5bd740 10 client.26038 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:01:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.307+0000 7fc4db5bd740 10 client.26038 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:01:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.307+0000 7fc4db5bd740 20 client.26038 populate_metadata read hostname 'smithi094' 2024-02-17T12:01:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.307+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/2505432475 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556903048a90 0x55690304aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:01:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.307+0000 7fc4db5bd740 4 client.26038 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:01:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.307+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2505432475 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556902c86550 con 0x556903048a90 2024-02-17T12:01:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.307+0000 7fc4d259c640 1 --2- 172.21.15.94:0/2505432475 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556903048a90 0x55690304aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:01:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.307+0000 7fc4d259c640 1 --2- 172.21.15.94:0/2505432475 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556903048a90 0x55690304aeb0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f899b8a385eedae6 server_cookie=69f5cfcb6a64e842 in_seq=0 out_seq=0 2024-02-17T12:01:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.307+0000 7fc4b9ffb640 10 client.26038 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.331+0000 7fc4b9ffb640 1 -- 172.21.15.94:0/2505432475 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556902c86550 con 0x556903048a90 2024-02-17T12:01:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.331+0000 7fc4b9ffb640 10 client.26038 handle_command_reply: tid=0 2024-02-17T12:01:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.331+0000 7fc4db5bd740 1 client.26038 shutdown 2024-02-17T12:01:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.332+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2505432475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 msgr2=0x556902ff03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.332+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/2505432475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902ff03e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc4d406a590 tx=0x7fc4cc0054b0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.333+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2505432475 shutdown_connections 2024-02-17T12:01:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.333+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/2505432475 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556903048a90 0x55690304aeb0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.333+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/2505432475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556902feb2e0 0x556902ff03e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.333+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/2505432475 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556902fe2ad0 0x556902ff2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.333+0000 7fc4db5bd740 1 --2- 172.21.15.94:0/2505432475 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556902fe1d60 0x556902ff1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.333+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2505432475 >> 172.21.15.94:0/2505432475 conn(0x556902fa9550 msgr2=0x556902fdb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.333+0000 7fc4b9ffb640 0 client.26038 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:01:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.333+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2505432475 shutdown_connections 2024-02-17T12:01:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.333+0000 7fc4db5bd740 1 -- 172.21.15.94:0/2505432475 wait complete. 2024-02-17T12:01:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.333+0000 7fc4db5bd740 20 client.26038 trim_cache size 0 max 16384 2024-02-17T12:01:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:01:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:01:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:01:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:01:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.334+0000 7fc4d89ef640 1 -- 172.21.15.94:0/3022817592 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc4c00a2350 msgr2=0x7fc4c00a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.334+0000 7fc4d89ef640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc4c00a2350 0x7fc4c00a4810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc02c180 tx=0x7fc4bc035dd0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.334+0000 7fc4d89ef640 1 -- 172.21.15.94:0/3022817592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d4108810 msgr2=0x7fc4d41a20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:01:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.334+0000 7fc4d89ef640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d4108810 0x7fc4d41a20c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400eb90 tx=0x7fc4c400c6a0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.334+0000 7fc4d89ef640 1 -- 172.21.15.94:0/3022817592 shutdown_connections 2024-02-17T12:01:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.334+0000 7fc4d89ef640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc4c00a2350 0x7fc4c00a4810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.334+0000 7fc4d89ef640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc4d4116180 0x7fc4d41a6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.334+0000 7fc4d89ef640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc4d410b830 0x7fc4d41a2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.334+0000 7fc4d89ef640 1 --2- 172.21.15.94:0/3022817592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc4d4108810 0x7fc4d41a20c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:01:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.334+0000 7fc4d89ef640 1 -- 172.21.15.94:0/3022817592 >> 172.21.15.94:0/3022817592 conn(0x7fc4d406c220 msgr2=0x7fc4d4116580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:01:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.334+0000 7fc4d89ef640 1 -- 172.21.15.94:0/3022817592 shutdown_connections 2024-02-17T12:01:55.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:01:55.334+0000 7fc4d89ef640 1 -- 172.21.15.94:0/3022817592 wait complete. 2024-02-17T12:01:56.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:55 smithi189 ceph-mon[36588]: pgmap v117: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 13 MiB/s wr, 977 op/s 2024-02-17T12:01:56.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:55 smithi125 ceph-mon[36819]: pgmap v117: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 13 MiB/s wr, 977 op/s 2024-02-17T12:01:56.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:55 smithi094 ceph-mon[32499]: pgmap v117: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 13 MiB/s wr, 977 op/s 2024-02-17T12:01:58.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:57 smithi189 ceph-mon[36588]: pgmap v118: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.7 MiB/s wr, 766 op/s 2024-02-17T12:01:58.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:57 smithi125 ceph-mon[36819]: pgmap v118: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.7 MiB/s wr, 766 op/s 2024-02-17T12:01:58.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:57 smithi094 ceph-mon[32499]: pgmap v118: 97 pgs: 97 active+clean; 3.0 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.7 MiB/s wr, 766 op/s 2024-02-17T12:02:00.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:01:59 smithi189 ceph-mon[36588]: pgmap v119: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-02-17T12:02:00.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:01:59 smithi125 ceph-mon[36819]: pgmap v119: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-02-17T12:02:00.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:01:59 smithi094 ceph-mon[32499]: pgmap v119: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 14 MiB/s wr, 1.15k op/s 2024-02-17T12:02:00.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 220.0 s, 3318.8 tps, lat 30.140 ms stddev 32.299 2024-02-17T12:02:02.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:01 smithi189 ceph-mon[36588]: pgmap v120: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.2 MiB/s wr, 828 op/s 2024-02-17T12:02:02.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:01 smithi125 ceph-mon[36819]: pgmap v120: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.2 MiB/s wr, 828 op/s 2024-02-17T12:02:02.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:01 smithi094 ceph-mon[32499]: pgmap v120: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 9.2 MiB/s wr, 828 op/s 2024-02-17T12:02:04.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:03 smithi189 ceph-mon[36588]: pgmap v121: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 11 MiB/s wr, 986 op/s 2024-02-17T12:02:04.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:03 smithi125 ceph-mon[36819]: pgmap v121: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 11 MiB/s wr, 986 op/s 2024-02-17T12:02:04.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:03 smithi094 ceph-mon[32499]: pgmap v121: 97 pgs: 97 active+clean; 3.0 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 11 MiB/s wr, 986 op/s 2024-02-17T12:02:05.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 225.0 s, 467.8 tps, lat 161.208 ms stddev 175.876 2024-02-17T12:02:06.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:05 smithi189 ceph-mon[36588]: pgmap v122: 97 pgs: 97 active+clean; 3.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 9.1 MiB/s wr, 881 op/s 2024-02-17T12:02:06.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:05 smithi125 ceph-mon[36819]: pgmap v122: 97 pgs: 97 active+clean; 3.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 9.1 MiB/s wr, 881 op/s 2024-02-17T12:02:06.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:05 smithi094 ceph-mon[32499]: pgmap v122: 97 pgs: 97 active+clean; 3.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 9.1 MiB/s wr, 881 op/s 2024-02-17T12:02:08.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:07 smithi189 ceph-mon[36588]: pgmap v123: 97 pgs: 97 active+clean; 3.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 666 op/s 2024-02-17T12:02:08.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:07 smithi125 ceph-mon[36819]: pgmap v123: 97 pgs: 97 active+clean; 3.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 666 op/s 2024-02-17T12:02:08.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:07 smithi094 ceph-mon[32499]: pgmap v123: 97 pgs: 97 active+clean; 3.0 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 666 op/s 2024-02-17T12:02:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.468+0000 7f889b99a640 1 -- 172.21.15.94:0/3217834902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88941086a0 msgr2=0x7f8894110270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.468+0000 7f889b99a640 1 --2- 172.21.15.94:0/3217834902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88941086a0 0x7f8894110270 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f888c009cd0 tx=0x7f888c02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.468+0000 7f889b99a640 1 -- 172.21.15.94:0/3217834902 shutdown_connections 2024-02-17T12:02:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.468+0000 7f889b99a640 1 --2- 172.21.15.94:0/3217834902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88941086a0 0x7f8894110270 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.468+0000 7f889b99a640 1 --2- 172.21.15.94:0/3217834902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8894103f80 0x7f8894107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.468+0000 7f889b99a640 1 --2- 172.21.15.94:0/3217834902 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88941035d0 0x7f88941039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.468+0000 7f889b99a640 1 -- 172.21.15.94:0/3217834902 >> 172.21.15.94:0/3217834902 conn(0x7f88940fee30 msgr2=0x7f8894101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.468+0000 7f889b99a640 1 -- 172.21.15.94:0/3217834902 shutdown_connections 2024-02-17T12:02:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.468+0000 7f889b99a640 1 -- 172.21.15.94:0/3217834902 wait complete. 2024-02-17T12:02:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f889b99a640 1 Processor -- start 2024-02-17T12:02:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f889b99a640 1 -- start start 2024-02-17T12:02:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f889b99a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88941035d0 0x7f8894072680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f889b99a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88941086a0 0x7f8894072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f889b99a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f889406c6f0 0x7f889406cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f889b99a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889406d1a0 con 0x7f88941035d0 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f889b99a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889406d310 con 0x7f88941086a0 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f889b99a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889406d4d0 con 0x7f889406c6f0 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f8899f37640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f889406c6f0 0x7f889406cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f8898f35640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88941086a0 0x7f8894072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f8899736640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88941035d0 0x7f8894072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f8899f37640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f889406c6f0 0x7f889406cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44416/0 (socket says 172.21.15.94:44416) 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f8898f35640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88941086a0 0x7f8894072bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:47038/0 (socket says 172.21.15.94:47038) 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.469+0000 7f8899f37640 1 -- 172.21.15.94:0/419810976 learned_addr learned my addr 172.21.15.94:0/419810976 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.470+0000 7f8899f37640 1 -- 172.21.15.94:0/419810976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88941086a0 msgr2=0x7f8894072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.470+0000 7f8899f37640 1 --2- 172.21.15.94:0/419810976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88941086a0 0x7f8894072bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.470+0000 7f8899f37640 1 -- 172.21.15.94:0/419810976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88941035d0 msgr2=0x7f8894072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.470+0000 7f8899f37640 1 --2- 172.21.15.94:0/419810976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88941035d0 0x7f8894072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.471 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.470+0000 7f8899f37640 1 -- 172.21.15.94:0/419810976 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c009900 con 0x7f889406c6f0 2024-02-17T12:02:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.470+0000 7f8898f35640 1 --2- 172.21.15.94:0/419810976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88941086a0 0x7f8894072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.470+0000 7f8899f37640 1 --2- 172.21.15.94:0/419810976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f889406c6f0 0x7f889406cba0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f888c005e30 tx=0x7f888c0325e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.470+0000 7f888a7fc640 1 -- 172.21.15.94:0/419810976 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c034e20 con 0x7f889406c6f0 2024-02-17T12:02:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.470+0000 7f889b99a640 1 -- 172.21.15.94:0/419810976 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88940755c0 con 0x7f889406c6f0 2024-02-17T12:02:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.470+0000 7f888a7fc640 1 -- 172.21.15.94:0/419810976 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f888c003e50 con 0x7f889406c6f0 2024-02-17T12:02:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.470+0000 7f888a7fc640 1 -- 172.21.15.94:0/419810976 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c004750 con 0x7f889406c6f0 2024-02-17T12:02:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.470+0000 7f889b99a640 1 -- 172.21.15.94:0/419810976 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8894075b20 con 0x7f889406c6f0 2024-02-17T12:02:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.474+0000 7f888a7fc640 1 -- 172.21.15.94:0/419810976 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f888c00a040 con 0x7f889406c6f0 2024-02-17T12:02:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f889e573740 1 Processor -- start 2024-02-17T12:02:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f889e573740 1 -- start start 2024-02-17T12:02:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f888a7fc640 1 --2- 172.21.15.94:0/419810976 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f88800a22d0 0x7f88800a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f889e573740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc57b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f889e573740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc57c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f889e573740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 0x5572cc5876d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f889e573740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572cc4ea9f0 con 0x5572cc5852e0 2024-02-17T12:02:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f889e573740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572cc4e7f20 con 0x5572cc57cad0 2024-02-17T12:02:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f889e573740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572cc4e3700 con 0x5572cc57bd60 2024-02-17T12:02:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f888a7fc640 1 -- 172.21.15.94:0/419810976 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f888c0f08c0 con 0x7f889406c6f0 2024-02-17T12:02:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f8898f35640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc57c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f8899f37640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 0x5572cc5876d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f8898f35640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc57c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44430/0 (socket says 172.21.15.94:44430) 2024-02-17T12:02:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f8898f35640 1 -- 172.21.15.94:0/4024189873 learned_addr learned my addr 172.21.15.94:0/4024189873 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f8899736640 1 --2- 172.21.15.94:0/4024189873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc57b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.475+0000 7f8898f35640 1 --2- 172.21.15.94:0/419810976 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f88800a22d0 0x7f88800a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.476+0000 7f8899f37640 1 -- 172.21.15.94:0/4024189873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 msgr2=0x5572cc57c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.476+0000 7f8899f37640 1 --2- 172.21.15.94:0/4024189873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc57c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.476+0000 7f8899f37640 1 -- 172.21.15.94:0/4024189873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 msgr2=0x5572cc57b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.476+0000 7f8899f37640 1 --2- 172.21.15.94:0/4024189873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc57b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.476+0000 7f8899f37640 1 -- 172.21.15.94:0/4024189873 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572cc4797b0 con 0x5572cc5852e0 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.476+0000 7f8899f37640 1 --2- 172.21.15.94:0/4024189873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 0x5572cc5876d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f888c0ee110 tx=0x7f888c0bcef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661da709b10dd63c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.476+0000 7f8898f35640 1 --2- 172.21.15.94:0/419810976 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f88800a22d0 0x7f88800a4790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f88840058c0 tx=0x7f8884009b70 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.476+0000 7f887f7fe640 1 -- 172.21.15.94:0/4024189873 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c04f2b0 con 0x5572cc5852e0 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.477+0000 7f887f7fe640 1 -- 172.21.15.94:0/4024189873 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f888c046e30 con 0x5572cc5852e0 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.477+0000 7f887f7fe640 1 -- 172.21.15.94:0/4024189873 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c0c1500 con 0x5572cc5852e0 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.477+0000 7f889e573740 1 -- 172.21.15.94:0/4024189873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 msgr2=0x5572cc5876d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.477+0000 7f889e573740 1 --2- 172.21.15.94:0/4024189873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 0x5572cc5876d0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f888c0ee110 tx=0x7f888c0bcef0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.477+0000 7f889e573740 1 -- 172.21.15.94:0/4024189873 shutdown_connections 2024-02-17T12:02:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.477+0000 7f889e573740 1 --2- 172.21.15.94:0/4024189873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 0x5572cc5876d0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.477+0000 7f889e573740 1 --2- 172.21.15.94:0/4024189873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc57c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.477+0000 7f889e573740 1 --2- 172.21.15.94:0/4024189873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc57b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.477+0000 7f889e573740 1 -- 172.21.15.94:0/4024189873 >> 172.21.15.94:0/4024189873 conn(0x5572cc543550 msgr2=0x5572cc576a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.477+0000 7f889e573740 1 -- 172.21.15.94:0/4024189873 shutdown_connections 2024-02-17T12:02:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.477+0000 7f889e573740 1 -- 172.21.15.94:0/4024189873 wait complete. 2024-02-17T12:02:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.478+0000 7f889e573740 1 -- start start 2024-02-17T12:02:08.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.478+0000 7f889e573740 1 Processor -- start 2024-02-17T12:02:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.478+0000 7f889e573740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc582320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.478+0000 7f889e573740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc582860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.478+0000 7f889e573740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 0x5572cc582da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.478+0000 7f889e573740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572cc488bf0 con 0x5572cc5852e0 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.478+0000 7f889e573740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572cc1b7250 con 0x5572cc57cad0 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.478+0000 7f889e573740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572cc469200 con 0x5572cc57bd60 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.479+0000 7f889e573740 10 client.0 _ll_register_callbacks cb 0x5572cc4d7d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.479+0000 7f8898f35640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc582860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.479+0000 7f8899736640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc582320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.479+0000 7f8899736640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc582320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44438/0 (socket says 172.21.15.94:44438) 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.479+0000 7f8899736640 1 -- 172.21.15.94:0/109401845 learned_addr learned my addr 172.21.15.94:0/109401845 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.479+0000 7f8898f35640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc582860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:47066/0 (socket says 172.21.15.94:47066) 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.479+0000 7f8899f37640 1 --2- 172.21.15.94:0/109401845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 0x5572cc582da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.479+0000 7f8898f35640 1 -- 172.21.15.94:0/109401845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 msgr2=0x5572cc582320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.479+0000 7f8898f35640 1 --2- 172.21.15.94:0/109401845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc582320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.479+0000 7f8898f35640 1 -- 172.21.15.94:0/109401845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 msgr2=0x5572cc582da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.479+0000 7f8898f35640 1 --2- 172.21.15.94:0/109401845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 0x5572cc582da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.479+0000 7f8898f35640 1 -- 172.21.15.94:0/109401845 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8894075b20 con 0x5572cc57cad0 2024-02-17T12:02:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.480+0000 7f8898f35640 1 --2- 172.21.15.94:0/109401845 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc582860 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5572cc57d230 tx=0x7f8884005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f3ab8b1f390481 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.480+0000 7f887cff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:02:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.480+0000 7f887cff9640 1 -- 172.21.15.94:0/109401845 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888400ebc0 con 0x5572cc57cad0 2024-02-17T12:02:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.480+0000 7f887cff9640 1 -- 172.21.15.94:0/109401845 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f888400ed20 con 0x5572cc57cad0 2024-02-17T12:02:08.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.481+0000 7f889e573740 1 -- 172.21.15.94:0/109401845 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572cc1e3600 con 0x5572cc57cad0 2024-02-17T12:02:08.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.481+0000 7f887cff9640 1 -- 172.21.15.94:0/109401845 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8884004030 con 0x5572cc57cad0 2024-02-17T12:02:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.481+0000 7f889e573740 10 client.26015 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.481+0000 7f889e573740 1 -- 172.21.15.94:0/109401845 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5572cc4e1d80 con 0x5572cc57cad0 2024-02-17T12:02:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.482+0000 7f887cff9640 1 -- 172.21.15.94:0/109401845 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8884024070 con 0x5572cc57cad0 2024-02-17T12:02:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.482+0000 7f887cff9640 1 -- 172.21.15.94:0/109401845 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8884028070 con 0x5572cc57cad0 2024-02-17T12:02:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.482+0000 7f889e573740 10 client.26015 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.482+0000 7f889e573740 10 client.26015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.483+0000 7f889e573740 20 client.26015 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.483+0000 7f889e573740 1 --2- 172.21.15.94:0/109401845 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5572cc595450 0x5572cc597890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.483+0000 7f889e573740 4 client.26015 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.483+0000 7f889e573740 1 -- 172.21.15.94:0/109401845 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572cc1e3600 con 0x5572cc595450 2024-02-17T12:02:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.483+0000 7f8899736640 1 --2- 172.21.15.94:0/109401845 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5572cc595450 0x5572cc597890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.483+0000 7f8899736640 1 --2- 172.21.15.94:0/109401845 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5572cc595450 0x5572cc597890 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=350f8f11ccb64aee server_cookie=554df5fb8d6a2b4c in_seq=0 out_seq=0 2024-02-17T12:02:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.483+0000 7f887cff9640 10 client.26015 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.484+0000 7f887cff9640 1 -- 172.21.15.94:0/109401845 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5572cc1e3600 con 0x5572cc595450 2024-02-17T12:02:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.484+0000 7f887cff9640 10 client.26015 handle_command_reply: tid=0 2024-02-17T12:02:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.484+0000 7f889e573740 1 client.26015 shutdown 2024-02-17T12:02:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.485+0000 7f889e573740 1 -- 172.21.15.94:0/109401845 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 msgr2=0x5572cc582860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.485+0000 7f889e573740 1 --2- 172.21.15.94:0/109401845 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc582860 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5572cc57d230 tx=0x7f8884005850 comp rx=0 tx=0).stop 2024-02-17T12:02:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.485+0000 7f889e573740 1 -- 172.21.15.94:0/109401845 shutdown_connections 2024-02-17T12:02:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.485+0000 7f889e573740 1 --2- 172.21.15.94:0/109401845 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5572cc595450 0x5572cc597890 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.485+0000 7f889e573740 1 --2- 172.21.15.94:0/109401845 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 0x5572cc582da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.485+0000 7f889e573740 1 --2- 172.21.15.94:0/109401845 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc582860 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.485+0000 7f889e573740 1 --2- 172.21.15.94:0/109401845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc582320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.485+0000 7f889e573740 1 -- 172.21.15.94:0/109401845 >> 172.21.15.94:0/109401845 conn(0x5572cc543550 msgr2=0x5572cc5787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.485+0000 7f887cff9640 0 client.26015 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.485+0000 7f889e573740 1 -- 172.21.15.94:0/109401845 shutdown_connections 2024-02-17T12:02:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.485+0000 7f889e573740 1 -- 172.21.15.94:0/109401845 wait complete. 2024-02-17T12:02:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.485+0000 7f889e573740 20 client.26015 trim_cache size 0 max 16384 2024-02-17T12:02:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.507+0000 7f889e573740 1 Processor -- start 2024-02-17T12:02:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.507+0000 7f889e573740 1 -- start start 2024-02-17T12:02:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.507+0000 7f889e573740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc584a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.507+0000 7f889e573740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc582bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.507+0000 7f8898f35640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc582bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f8898f35640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc582bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:47076/0 (socket says 172.21.15.94:47076) 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f8899736640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc584a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f8899736640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc584a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44448/0 (socket says 172.21.15.94:44448) 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f889e573740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 0x5572cc580bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f8898f35640 1 -- 172.21.15.94:0/1683418594 learned_addr learned my addr 172.21.15.94:0/1683418594 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f889e573740 1 -- 172.21.15.94:0/1683418594 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572cc577a50 con 0x5572cc5852e0 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f889e573740 1 -- 172.21.15.94:0/1683418594 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572cc57dd10 con 0x5572cc57cad0 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f889e573740 1 -- 172.21.15.94:0/1683418594 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572cc57d0c0 con 0x5572cc57bd60 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f8899f37640 1 --2- 172.21.15.94:0/1683418594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 0x5572cc580bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f8898f35640 1 -- 172.21.15.94:0/1683418594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 msgr2=0x5572cc584a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f8898f35640 1 --2- 172.21.15.94:0/1683418594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc584a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f8898f35640 1 -- 172.21.15.94:0/1683418594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 msgr2=0x5572cc580bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f8898f35640 1 --2- 172.21.15.94:0/1683418594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 0x5572cc580bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f8898f35640 1 -- 172.21.15.94:0/1683418594 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572cc47e7b0 con 0x5572cc57cad0 2024-02-17T12:02:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.508+0000 7f8898f35640 1 --2- 172.21.15.94:0/1683418594 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc582bc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8884005750 tx=0x7f8884005b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9edb16f088146328 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.509+0000 7f887effd640 1 -- 172.21.15.94:0/1683418594 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884037070 con 0x5572cc57cad0 2024-02-17T12:02:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.509+0000 7f887effd640 1 -- 172.21.15.94:0/1683418594 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8884024040 con 0x5572cc57cad0 2024-02-17T12:02:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.509+0000 7f887effd640 1 -- 172.21.15.94:0/1683418594 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884004b30 con 0x5572cc57cad0 2024-02-17T12:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.510+0000 7f889e573740 1 -- 172.21.15.94:0/1683418594 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 msgr2=0x5572cc582bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.510+0000 7f889e573740 1 --2- 172.21.15.94:0/1683418594 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc582bc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8884005750 tx=0x7f8884005b90 comp rx=0 tx=0).stop 2024-02-17T12:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.510+0000 7f889e573740 1 -- 172.21.15.94:0/1683418594 shutdown_connections 2024-02-17T12:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.510+0000 7f889e573740 1 --2- 172.21.15.94:0/1683418594 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc5852e0 0x5572cc580bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.510+0000 7f889e573740 1 --2- 172.21.15.94:0/1683418594 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57cad0 0x5572cc582bc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.510+0000 7f889e573740 1 --2- 172.21.15.94:0/1683418594 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc57bd60 0x5572cc584a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.510+0000 7f889e573740 1 -- 172.21.15.94:0/1683418594 >> 172.21.15.94:0/1683418594 conn(0x5572cc543550 msgr2=0x5572cc576580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.510+0000 7f889e573740 1 -- 172.21.15.94:0/1683418594 shutdown_connections 2024-02-17T12:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.510+0000 7f889e573740 1 -- 172.21.15.94:0/1683418594 wait complete. 2024-02-17T12:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.510+0000 7f889e573740 1 -- start start 2024-02-17T12:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.510+0000 7f889e573740 1 Processor -- start 2024-02-17T12:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.511+0000 7f889e573740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57bd60 0x5572cc58c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.511+0000 7f889e573740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc57cad0 0x5572cc58a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.511+0000 7f889e573740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc5852e0 0x5572cc58aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.511+0000 7f889e573740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572cc58b050 con 0x5572cc57cad0 2024-02-17T12:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.511+0000 7f889e573740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572cc58b1c0 con 0x5572cc57bd60 2024-02-17T12:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.511+0000 7f889e573740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572cc58b380 con 0x5572cc5852e0 2024-02-17T12:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.511+0000 7f889e573740 10 client.0 _ll_register_callbacks cb 0x7f888c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.511+0000 7f8899736640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57bd60 0x5572cc58c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.511+0000 7f8899736640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57bd60 0x5572cc58c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:47078/0 (socket says 172.21.15.94:47078) 2024-02-17T12:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.511+0000 7f8899736640 1 -- 172.21.15.94:0/3810494991 learned_addr learned my addr 172.21.15.94:0/3810494991 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.511+0000 7f8898f35640 1 --2- 172.21.15.94:0/3810494991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc57cad0 0x5572cc58a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.511+0000 7f8899f37640 1 --2- 172.21.15.94:0/3810494991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc5852e0 0x5572cc58aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.511+0000 7f8899736640 1 -- 172.21.15.94:0/3810494991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc5852e0 msgr2=0x5572cc58aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.512+0000 7f8899736640 1 --2- 172.21.15.94:0/3810494991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc5852e0 0x5572cc58aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.512+0000 7f8899736640 1 -- 172.21.15.94:0/3810494991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc57cad0 msgr2=0x5572cc58a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.512+0000 7f8899736640 1 --2- 172.21.15.94:0/3810494991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc57cad0 0x5572cc58a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.512+0000 7f8899736640 1 -- 172.21.15.94:0/3810494991 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572cc4e1d80 con 0x5572cc57bd60 2024-02-17T12:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.512+0000 7f8899736640 1 --2- 172.21.15.94:0/3810494991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57bd60 0x5572cc58c190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f88900058c0 tx=0x7f8890005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=812b8df7203ebe70 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.512+0000 7f887cff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.512+0000 7f887cff9640 1 -- 172.21.15.94:0/3810494991 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890019040 con 0x5572cc57bd60 2024-02-17T12:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.512+0000 7f887cff9640 1 -- 172.21.15.94:0/3810494991 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f88900263a0 con 0x5572cc57bd60 2024-02-17T12:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.513+0000 7f889e573740 1 -- 172.21.15.94:0/3810494991 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f888c046e30 con 0x5572cc57bd60 2024-02-17T12:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.513+0000 7f887cff9640 1 -- 172.21.15.94:0/3810494991 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8890009b00 con 0x5572cc57bd60 2024-02-17T12:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.513+0000 7f889e573740 10 client.26021 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.513+0000 7f889e573740 1 -- 172.21.15.94:0/3810494991 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5572cc4a4540 con 0x5572cc57bd60 2024-02-17T12:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.513+0000 7f887cff9640 1 -- 172.21.15.94:0/3810494991 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f889001a040 con 0x5572cc57bd60 2024-02-17T12:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.513+0000 7f887cff9640 1 -- 172.21.15.94:0/3810494991 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8890014020 con 0x5572cc57bd60 2024-02-17T12:02:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.514+0000 7f889e573740 10 client.26021 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.514+0000 7f889e573740 10 client.26021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.514+0000 7f889e573740 20 client.26021 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.514+0000 7f889e573740 1 --2- 172.21.15.94:0/3810494991 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5572cc5ed100 0x5572cc5ef520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.514+0000 7f889e573740 4 client.26021 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.514+0000 7f889e573740 1 -- 172.21.15.94:0/3810494991 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f888c046e30 con 0x5572cc5ed100 2024-02-17T12:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.514+0000 7f8898f35640 1 --2- 172.21.15.94:0/3810494991 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5572cc5ed100 0x5572cc5ef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.514+0000 7f887cff9640 10 client.26021 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.514+0000 7f8898f35640 1 --2- 172.21.15.94:0/3810494991 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5572cc5ed100 0x5572cc5ef520 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d6a8333e2004f3 server_cookie=e5672e071c64a078 in_seq=0 out_seq=0 2024-02-17T12:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.530+0000 7f887cff9640 1 -- 172.21.15.94:0/3810494991 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f888c046e30 con 0x5572cc5ed100 2024-02-17T12:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.530+0000 7f887cff9640 10 client.26021 handle_command_reply: tid=0 2024-02-17T12:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.530+0000 7f889e573740 1 client.26021 shutdown 2024-02-17T12:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.531+0000 7f889e573740 1 -- 172.21.15.94:0/3810494991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57bd60 msgr2=0x5572cc58c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.531+0000 7f889e573740 1 --2- 172.21.15.94:0/3810494991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57bd60 0x5572cc58c190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f88900058c0 tx=0x7f8890005f90 comp rx=0 tx=0).stop 2024-02-17T12:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.531+0000 7f889e573740 1 -- 172.21.15.94:0/3810494991 shutdown_connections 2024-02-17T12:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.531+0000 7f889e573740 1 --2- 172.21.15.94:0/3810494991 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5572cc5ed100 0x5572cc5ef520 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.531+0000 7f889e573740 1 --2- 172.21.15.94:0/3810494991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572cc5852e0 0x5572cc58aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.531+0000 7f887cff9640 0 client.26021 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.531+0000 7f889e573740 1 --2- 172.21.15.94:0/3810494991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5572cc57cad0 0x5572cc58a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.531+0000 7f889e573740 1 --2- 172.21.15.94:0/3810494991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5572cc57bd60 0x5572cc58c190 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f88900058c0 tx=0x7f8890005f90 comp rx=0 tx=0).stop 2024-02-17T12:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.531+0000 7f889e573740 1 -- 172.21.15.94:0/3810494991 >> 172.21.15.94:0/3810494991 conn(0x5572cc543550 msgr2=0x5572cc580b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.532+0000 7f889e573740 1 -- 172.21.15.94:0/3810494991 shutdown_connections 2024-02-17T12:02:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.532+0000 7f889e573740 1 -- 172.21.15.94:0/3810494991 wait complete. 2024-02-17T12:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.532+0000 7f889e573740 20 client.26021 trim_cache size 0 max 16384 2024-02-17T12:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:02:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.533+0000 7f889b99a640 1 -- 172.21.15.94:0/419810976 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f88800a22d0 msgr2=0x7f88800a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.533+0000 7f889b99a640 1 --2- 172.21.15.94:0/419810976 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f88800a22d0 0x7f88800a4790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f88840058c0 tx=0x7f8884009b70 comp rx=0 tx=0).stop 2024-02-17T12:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.533+0000 7f889b99a640 1 -- 172.21.15.94:0/419810976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f889406c6f0 msgr2=0x7f889406cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.533+0000 7f889b99a640 1 --2- 172.21.15.94:0/419810976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f889406c6f0 0x7f889406cba0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f888c005e30 tx=0x7f888c0325e0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.533+0000 7f889b99a640 1 -- 172.21.15.94:0/419810976 shutdown_connections 2024-02-17T12:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.533+0000 7f889b99a640 1 --2- 172.21.15.94:0/419810976 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f88800a22d0 0x7f88800a4790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.533+0000 7f889b99a640 1 --2- 172.21.15.94:0/419810976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f889406c6f0 0x7f889406cba0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.533+0000 7f889b99a640 1 --2- 172.21.15.94:0/419810976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88941086a0 0x7f8894072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.533+0000 7f889b99a640 1 --2- 172.21.15.94:0/419810976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88941035d0 0x7f8894072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.533+0000 7f889b99a640 1 -- 172.21.15.94:0/419810976 >> 172.21.15.94:0/419810976 conn(0x7f88940fee30 msgr2=0x7f88940ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.533+0000 7f889b99a640 1 -- 172.21.15.94:0/419810976 shutdown_connections 2024-02-17T12:02:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:08.533+0000 7f889b99a640 1 -- 172.21.15.94:0/419810976 wait complete. 2024-02-17T12:02:10.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:09 smithi189 ceph-mon[36588]: pgmap v124: 97 pgs: 97 active+clean; 3.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 1.88k op/s 2024-02-17T12:02:10.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:09 smithi125 ceph-mon[36819]: pgmap v124: 97 pgs: 97 active+clean; 3.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 1.88k op/s 2024-02-17T12:02:10.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:09 smithi094 ceph-mon[32499]: pgmap v124: 97 pgs: 97 active+clean; 3.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 35 MiB/s wr, 1.88k op/s 2024-02-17T12:02:10.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 230.0 s, 2928.5 tps, lat 42.541 ms stddev 118.392 2024-02-17T12:02:12.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:11 smithi189 ceph-mon[36588]: pgmap v125: 97 pgs: 97 active+clean; 3.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 1.49k op/s 2024-02-17T12:02:12.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:11 smithi125 ceph-mon[36819]: pgmap v125: 97 pgs: 97 active+clean; 3.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 1.49k op/s 2024-02-17T12:02:12.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:11 smithi094 ceph-mon[32499]: pgmap v125: 97 pgs: 97 active+clean; 3.1 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 MiB/s wr, 1.49k op/s 2024-02-17T12:02:14.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:13 smithi189 ceph-mon[36588]: pgmap v126: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 1.62k op/s 2024-02-17T12:02:14.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:13 smithi125 ceph-mon[36819]: pgmap v126: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 1.62k op/s 2024-02-17T12:02:14.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:13 smithi094 ceph-mon[32499]: pgmap v126: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 1.62k op/s 2024-02-17T12:02:15.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 235.0 s, 3711.6 tps, lat 26.743 ms stddev 30.233 2024-02-17T12:02:16.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:15 smithi125 ceph-mon[36819]: pgmap v127: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 1.64k op/s 2024-02-17T12:02:16.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:15 smithi094 ceph-mon[32499]: pgmap v127: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 1.64k op/s 2024-02-17T12:02:16.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:15 smithi189 ceph-mon[36588]: pgmap v127: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 32 MiB/s wr, 1.64k op/s 2024-02-17T12:02:17.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:16 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:02:17.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:16 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:02:17.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:16 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:02:18.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:17 smithi125 ceph-mon[36819]: pgmap v128: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 MiB/s wr, 1.52k op/s 2024-02-17T12:02:18.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:17 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:02:18.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:17 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:02:18.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:17 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:02:18.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:17 smithi094 ceph-mon[32499]: pgmap v128: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 MiB/s wr, 1.52k op/s 2024-02-17T12:02:18.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:17 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:02:18.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:17 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:02:18.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:17 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:02:18.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:17 smithi189 ceph-mon[36588]: pgmap v128: 97 pgs: 97 active+clean; 3.1 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 31 MiB/s wr, 1.52k op/s 2024-02-17T12:02:18.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:17 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:02:18.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:17 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:02:18.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:17 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:02:19.176 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:02:19.176 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:02:19.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.327+0000 7f3eef98c640 1 -- 172.21.15.94:0/4156412301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee80ff770 msgr2=0x7f3ee80ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:19.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.327+0000 7f3eef98c640 1 --2- 172.21.15.94:0/4156412301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee80ff770 0x7f3ee80ffbf0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3ed800aa40 tx=0x7f3ed802cf50 comp rx=0 tx=0).stop 2024-02-17T12:02:19.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.327+0000 7f3eef98c640 1 -- 172.21.15.94:0/4156412301 shutdown_connections 2024-02-17T12:02:19.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.327+0000 7f3eef98c640 1 --2- 172.21.15.94:0/4156412301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee8100130 0x7f3ee810c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.327+0000 7f3eef98c640 1 --2- 172.21.15.94:0/4156412301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee80ff770 0x7f3ee80ffbf0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.327+0000 7f3eef98c640 1 --2- 172.21.15.94:0/4156412301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ee8105780 0x7f3ee8105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.327+0000 7f3eef98c640 1 -- 172.21.15.94:0/4156412301 >> 172.21.15.94:0/4156412301 conn(0x7f3ee80765b0 msgr2=0x7f3ee80fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:19.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.328+0000 7f3eef98c640 1 -- 172.21.15.94:0/4156412301 shutdown_connections 2024-02-17T12:02:19.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.328+0000 7f3eef98c640 1 -- 172.21.15.94:0/4156412301 wait complete. 2024-02-17T12:02:19.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.328+0000 7f3eef98c640 1 Processor -- start 2024-02-17T12:02:19.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.328+0000 7f3eef98c640 1 -- start start 2024-02-17T12:02:19.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.328+0000 7f3eef98c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee80ff770 0x7f3ee8197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.328+0000 7f3eef98c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ee8100130 0x7f3ee8197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.328+0000 7f3eef98c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee8105780 0x7f3ee819bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.328+0000 7f3eef98c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee819c5c0 con 0x7f3ee80ff770 2024-02-17T12:02:19.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.329+0000 7f3eef98c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee819c730 con 0x7f3ee8100130 2024-02-17T12:02:19.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.329+0000 7f3eef98c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee819c8f0 con 0x7f3ee8105780 2024-02-17T12:02:19.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.329+0000 7f3eedf29640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee8105780 0x7f3ee819bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.329+0000 7f3eedf29640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee8105780 0x7f3ee819bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55516/0 (socket says 172.21.15.94:55516) 2024-02-17T12:02:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.329+0000 7f3eedf29640 1 -- 172.21.15.94:0/1252556715 learned_addr learned my addr 172.21.15.94:0/1252556715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.329+0000 7f3eedf29640 1 -- 172.21.15.94:0/1252556715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ee8100130 msgr2=0x7f3ee8197c40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-17T12:02:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.329+0000 7f3eedf29640 1 --2- 172.21.15.94:0/1252556715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ee8100130 0x7f3ee8197c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.329+0000 7f3eedf29640 1 -- 172.21.15.94:0/1252556715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee80ff770 msgr2=0x7f3ee8197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.329+0000 7f3eedf29640 1 --2- 172.21.15.94:0/1252556715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee80ff770 0x7f3ee8197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.329+0000 7f3eedf29640 1 -- 172.21.15.94:0/1252556715 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed800a670 con 0x7f3ee8105780 2024-02-17T12:02:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.329+0000 7f3eedf29640 1 --2- 172.21.15.94:0/1252556715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee8105780 0x7f3ee819bfc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000eb10 tx=0x7f3ee000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.330+0000 7f3ede7fc640 1 -- 172.21.15.94:0/1252556715 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0004b20 con 0x7f3ee8105780 2024-02-17T12:02:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.330+0000 7f3ede7fc640 1 -- 172.21.15.94:0/1252556715 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ee0004c80 con 0x7f3ee8105780 2024-02-17T12:02:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.330+0000 7f3ede7fc640 1 -- 172.21.15.94:0/1252556715 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0010650 con 0x7f3ee8105780 2024-02-17T12:02:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.330+0000 7f3eef98c640 1 -- 172.21.15.94:0/1252556715 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee819cbd0 con 0x7f3ee8105780 2024-02-17T12:02:19.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.330+0000 7f3eef98c640 1 -- 172.21.15.94:0/1252556715 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee8195420 con 0x7f3ee8105780 2024-02-17T12:02:19.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.333+0000 7f3ede7fc640 1 -- 172.21.15.94:0/1252556715 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3ee0005020 con 0x7f3ee8105780 2024-02-17T12:02:19.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.334+0000 7f3ede7fc640 1 --2- 172.21.15.94:0/1252556715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3ed40a2400 0x7f3ed40a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.334+0000 7f3ede7fc640 1 -- 172.21.15.94:0/1252556715 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3ee0014070 con 0x7f3ee8105780 2024-02-17T12:02:19.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3eed728640 1 --2- 172.21.15.94:0/1252556715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3ed40a2400 0x7f3ed40a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3ef2565740 1 Processor -- start 2024-02-17T12:02:19.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3ef2565740 1 -- start start 2024-02-17T12:02:19.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3eed728640 1 --2- 172.21.15.94:0/1252556715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3ed40a2400 0x7f3ed40a48c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400a930 tx=0x7f3ee4005c00 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:19.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3ef2565740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b182b0 0x5643c1b18690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3ef2565740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b18bd0 0x5643c1b21270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3ef2565740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b217b0 0x5643c1b23ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3ef2565740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c1a87280 con 0x5643c1b217b0 2024-02-17T12:02:19.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3ef2565740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c1a847b0 con 0x5643c1b18bd0 2024-02-17T12:02:19.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3ef2565740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c1a7ff90 con 0x5643c1b182b0 2024-02-17T12:02:19.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3eed728640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b182b0 0x5643c1b18690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3eed728640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b182b0 0x5643c1b18690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55518/0 (socket says 172.21.15.94:55518) 2024-02-17T12:02:19.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3eed728640 1 -- 172.21.15.94:0/3655611899 learned_addr learned my addr 172.21.15.94:0/3655611899 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:19.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.335+0000 7f3eedf29640 1 --2- 172.21.15.94:0/3655611899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b217b0 0x5643c1b23ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.336+0000 7f3eecf27640 1 --2- 172.21.15.94:0/3655611899 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b18bd0 0x5643c1b21270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.336+0000 7f3eed728640 1 -- 172.21.15.94:0/3655611899 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b18bd0 msgr2=0x5643c1b21270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:19.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.336+0000 7f3eed728640 1 --2- 172.21.15.94:0/3655611899 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b18bd0 0x5643c1b21270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.336+0000 7f3eed728640 1 -- 172.21.15.94:0/3655611899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b217b0 msgr2=0x5643c1b23ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:19.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.336+0000 7f3eed728640 1 --2- 172.21.15.94:0/3655611899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b217b0 0x5643c1b23ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.336+0000 7f3eed728640 1 -- 172.21.15.94:0/3655611899 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643c1a3e570 con 0x5643c1b182b0 2024-02-17T12:02:19.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.336+0000 7f3eedf29640 1 --2- 172.21.15.94:0/3655611899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b217b0 0x5643c1b23ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:19.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.336+0000 7f3eed728640 1 --2- 172.21.15.94:0/3655611899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b182b0 0x5643c1b18690 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400ccc0 tx=0x7f3ee400f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5298ee53a22af9f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:19.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.336+0000 7f3ed37fe640 1 -- 172.21.15.94:0/3655611899 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4005220 con 0x5643c1b182b0 2024-02-17T12:02:19.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.336+0000 7f3ed37fe640 1 -- 172.21.15.94:0/3655611899 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ee4004c80 con 0x5643c1b182b0 2024-02-17T12:02:19.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.336+0000 7f3ed37fe640 1 -- 172.21.15.94:0/3655611899 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee401c740 con 0x5643c1b182b0 2024-02-17T12:02:19.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.337+0000 7f3ef2565740 1 -- 172.21.15.94:0/3655611899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b182b0 msgr2=0x5643c1b18690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:19.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.337+0000 7f3ef2565740 1 --2- 172.21.15.94:0/3655611899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b182b0 0x5643c1b18690 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400ccc0 tx=0x7f3ee400f000 comp rx=0 tx=0).stop 2024-02-17T12:02:19.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.337+0000 7f3ef2565740 1 -- 172.21.15.94:0/3655611899 shutdown_connections 2024-02-17T12:02:19.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.337+0000 7f3ef2565740 1 --2- 172.21.15.94:0/3655611899 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b217b0 0x5643c1b23ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.337+0000 7f3ef2565740 1 --2- 172.21.15.94:0/3655611899 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b18bd0 0x5643c1b21270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.337+0000 7f3ef2565740 1 --2- 172.21.15.94:0/3655611899 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b182b0 0x5643c1b18690 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.337+0000 7f3ef2565740 1 -- 172.21.15.94:0/3655611899 >> 172.21.15.94:0/3655611899 conn(0x5643c1b12990 msgr2=0x5643c1b12d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:19.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.337+0000 7f3ef2565740 1 -- 172.21.15.94:0/3655611899 shutdown_connections 2024-02-17T12:02:19.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.337+0000 7f3ef2565740 1 -- 172.21.15.94:0/3655611899 wait complete. 2024-02-17T12:02:19.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.338+0000 7f3ef2565740 1 -- start start 2024-02-17T12:02:19.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.338+0000 7f3ef2565740 1 Processor -- start 2024-02-17T12:02:19.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.339+0000 7f3ef2565740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b182b0 0x5643c1b21160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.339+0000 7f3ef2565740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b18bd0 0x5643c1b1e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.339+0000 7f3ef2565740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b217b0 0x5643c1b1ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.339+0000 7f3ef2565740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c1753430 con 0x5643c1b217b0 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.339+0000 7f3ef2565740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c170fe00 con 0x5643c1b182b0 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.339+0000 7f3ef2565740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c1a04fd0 con 0x5643c1b18bd0 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.339+0000 7f3ef2565740 10 client.0 _ll_register_callbacks cb 0x5643c1a5afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.339+0000 7f3eecf27640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b18bd0 0x5643c1b1e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.339+0000 7f3eecf27640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b18bd0 0x5643c1b1e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55524/0 (socket says 172.21.15.94:55524) 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.339+0000 7f3eed728640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b182b0 0x5643c1b21160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.339+0000 7f3eecf27640 1 -- 172.21.15.94:0/1067463234 learned_addr learned my addr 172.21.15.94:0/1067463234 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.340+0000 7f3eedf29640 1 --2- 172.21.15.94:0/1067463234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b217b0 0x5643c1b1ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.340+0000 7f3eed728640 1 -- 172.21.15.94:0/1067463234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b18bd0 msgr2=0x5643c1b1e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.340+0000 7f3eed728640 1 --2- 172.21.15.94:0/1067463234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b18bd0 0x5643c1b1e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.340+0000 7f3eed728640 1 -- 172.21.15.94:0/1067463234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b217b0 msgr2=0x5643c1b1ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.340+0000 7f3eed728640 1 --2- 172.21.15.94:0/1067463234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b217b0 0x5643c1b1ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.340+0000 7f3eed728640 1 -- 172.21.15.94:0/1067463234 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee400c970 con 0x5643c1b182b0 2024-02-17T12:02:19.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.340+0000 7f3eed728640 1 --2- 172.21.15.94:0/1067463234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b182b0 0x5643c1b21160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400afd0 tx=0x7f3ee4004c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=485e247517e03d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.340+0000 7f3ed0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.340+0000 7f3ed0ff9640 1 -- 172.21.15.94:0/1067463234 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4004220 con 0x5643c1b182b0 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.340+0000 7f3ef2565740 1 -- 172.21.15.94:0/1067463234 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643c177f7e0 con 0x5643c1b182b0 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.341+0000 7f3ed0ff9640 1 -- 172.21.15.94:0/1067463234 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3ee4017040 con 0x5643c1b182b0 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.341+0000 7f3ed0ff9640 1 -- 172.21.15.94:0/1067463234 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ee40054d0 con 0x5643c1b182b0 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.341+0000 7f3ef2565740 10 client.26027 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.341+0000 7f3ef2565740 1 -- 172.21.15.94:0/1067463234 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5643c1a7e610 con 0x5643c1b182b0 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.341+0000 7f3ed0ff9640 1 -- 172.21.15.94:0/1067463234 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ee403b080 con 0x5643c1b182b0 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.341+0000 7f3ed0ff9640 1 -- 172.21.15.94:0/1067463234 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3ee40035f0 con 0x5643c1b182b0 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.341+0000 7f3ef2565740 10 client.26027 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.341+0000 7f3ef2565740 10 client.26027 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.341+0000 7f3ef2565740 20 client.26027 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.341+0000 7f3ef2565740 1 --2- 172.21.15.94:0/1067463234 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5643c1b31550 0x5643c1b33940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.341+0000 7f3ef2565740 4 client.26027 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:19.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.341+0000 7f3ef2565740 1 -- 172.21.15.94:0/1067463234 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643c17bc720 con 0x5643c1b31550 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.341+0000 7f3eecf27640 1 --2- 172.21.15.94:0/1067463234 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5643c1b31550 0x5643c1b33940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.342+0000 7f3eecf27640 1 --2- 172.21.15.94:0/1067463234 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5643c1b31550 0x5643c1b33940 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=769e82ed66d3232 server_cookie=1cecbcb200fe00c6 in_seq=0 out_seq=0 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.342+0000 7f3ed0ff9640 10 client.26027 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.342+0000 7f3ed0ff9640 1 -- 172.21.15.94:0/1067463234 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5643c17bc720 con 0x5643c1b31550 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.342+0000 7f3ed0ff9640 10 client.26027 handle_command_reply: tid=0 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.342+0000 7f3ef2565740 1 client.26027 shutdown 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.343+0000 7f3ef2565740 1 -- 172.21.15.94:0/1067463234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b182b0 msgr2=0x5643c1b21160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.343+0000 7f3ef2565740 1 --2- 172.21.15.94:0/1067463234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b182b0 0x5643c1b21160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400afd0 tx=0x7f3ee4004c10 comp rx=0 tx=0).stop 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.343+0000 7f3ef2565740 1 -- 172.21.15.94:0/1067463234 shutdown_connections 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.343+0000 7f3ef2565740 1 --2- 172.21.15.94:0/1067463234 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5643c1b31550 0x5643c1b33940 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.343+0000 7f3ef2565740 1 --2- 172.21.15.94:0/1067463234 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b217b0 0x5643c1b1ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.343+0000 7f3ef2565740 1 --2- 172.21.15.94:0/1067463234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b18bd0 0x5643c1b1e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.343+0000 7f3ef2565740 1 --2- 172.21.15.94:0/1067463234 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b182b0 0x5643c1b21160 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.343+0000 7f3ef2565740 1 -- 172.21.15.94:0/1067463234 >> 172.21.15.94:0/1067463234 conn(0x5643c1b12990 msgr2=0x5643c1b14730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:19.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.343+0000 7f3ed0ff9640 0 client.26027 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:19.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.344+0000 7f3ef2565740 1 -- 172.21.15.94:0/1067463234 shutdown_connections 2024-02-17T12:02:19.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.344+0000 7f3ef2565740 1 -- 172.21.15.94:0/1067463234 wait complete. 2024-02-17T12:02:19.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.344+0000 7f3ef2565740 20 client.26027 trim_cache size 0 max 16384 2024-02-17T12:02:19.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.362+0000 7f3ef2565740 1 Processor -- start 2024-02-17T12:02:19.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.362+0000 7f3ef2565740 1 -- start start 2024-02-17T12:02:19.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.362+0000 7f3ef2565740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b182b0 0x5643c1b20c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.362+0000 7f3ef2565740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b18bd0 0x5643c1b21160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.362+0000 7f3ef2565740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b217b0 0x5643c1b19810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.362+0000 7f3ef2565740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c1b139e0 con 0x5643c1b182b0 2024-02-17T12:02:19.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.362+0000 7f3ef2565740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c1b1e2c0 con 0x5643c1b217b0 2024-02-17T12:02:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.362+0000 7f3ef2565740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c1b2dcd0 con 0x5643c1b18bd0 2024-02-17T12:02:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3eed728640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b182b0 0x5643c1b20c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3eed728640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b182b0 0x5643c1b20c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41572/0 (socket says 172.21.15.94:41572) 2024-02-17T12:02:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3eed728640 1 -- 172.21.15.94:0/2747592884 learned_addr learned my addr 172.21.15.94:0/2747592884 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3eecf27640 1 --2- 172.21.15.94:0/2747592884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b18bd0 0x5643c1b21160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3eedf29640 1 --2- 172.21.15.94:0/2747592884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b217b0 0x5643c1b19810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3eed728640 1 -- 172.21.15.94:0/2747592884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b18bd0 msgr2=0x5643c1b21160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3eed728640 1 --2- 172.21.15.94:0/2747592884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b18bd0 0x5643c1b21160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3eed728640 1 -- 172.21.15.94:0/2747592884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b217b0 msgr2=0x5643c1b19810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3eed728640 1 --2- 172.21.15.94:0/2747592884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b217b0 0x5643c1b19810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3eed728640 1 -- 172.21.15.94:0/2747592884 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643c1a1aac0 con 0x5643c1b182b0 2024-02-17T12:02:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3eed728640 1 --2- 172.21.15.94:0/2747592884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b182b0 0x5643c1b20c20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400c740 tx=0x7f3ee400ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a56718d7919f0b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:19.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3ed2ffd640 1 -- 172.21.15.94:0/2747592884 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee400cb90 con 0x5643c1b182b0 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3ed2ffd640 1 -- 172.21.15.94:0/2747592884 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ee4013070 con 0x5643c1b182b0 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.363+0000 7f3ed2ffd640 1 -- 172.21.15.94:0/2747592884 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4024d50 con 0x5643c1b182b0 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.364+0000 7f3ef2565740 1 -- 172.21.15.94:0/2747592884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b182b0 msgr2=0x5643c1b20c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.364+0000 7f3ef2565740 1 --2- 172.21.15.94:0/2747592884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b182b0 0x5643c1b20c20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400c740 tx=0x7f3ee400ff20 comp rx=0 tx=0).stop 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.364+0000 7f3ef2565740 1 -- 172.21.15.94:0/2747592884 shutdown_connections 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.364+0000 7f3ef2565740 1 --2- 172.21.15.94:0/2747592884 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b217b0 0x5643c1b19810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.364+0000 7f3ef2565740 1 --2- 172.21.15.94:0/2747592884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b18bd0 0x5643c1b21160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.364+0000 7f3ef2565740 1 --2- 172.21.15.94:0/2747592884 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b182b0 0x5643c1b20c20 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.364+0000 7f3ef2565740 1 -- 172.21.15.94:0/2747592884 >> 172.21.15.94:0/2747592884 conn(0x5643c1b12990 msgr2=0x5643c1b38270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.364+0000 7f3ef2565740 1 -- 172.21.15.94:0/2747592884 shutdown_connections 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.364+0000 7f3ef2565740 1 -- 172.21.15.94:0/2747592884 wait complete. 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.364+0000 7f3ef2565740 1 -- start start 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.364+0000 7f3ef2565740 1 Processor -- start 2024-02-17T12:02:19.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3ef2565740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b182b0 0x5643c1b28a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3ef2565740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b18bd0 0x5643c1b24e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3ef2565740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b217b0 0x5643c1b25340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3ef2565740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c1b25a40 con 0x5643c1b18bd0 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3ef2565740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c1b25bb0 con 0x5643c1b217b0 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3ef2565740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c1b879f0 con 0x5643c1b182b0 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3ef2565740 10 client.0 _ll_register_callbacks cb 0x7f3ee400a540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3eecf27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b18bd0 0x5643c1b24e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3eecf27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b18bd0 0x5643c1b24e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41588/0 (socket says 172.21.15.94:41588) 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3eecf27640 1 -- 172.21.15.94:0/2332947435 learned_addr learned my addr 172.21.15.94:0/2332947435 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3eedf29640 1 --2- 172.21.15.94:0/2332947435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b217b0 0x5643c1b25340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3eed728640 1 --2- 172.21.15.94:0/2332947435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b182b0 0x5643c1b28a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3eecf27640 1 -- 172.21.15.94:0/2332947435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b182b0 msgr2=0x5643c1b28a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3eecf27640 1 --2- 172.21.15.94:0/2332947435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b182b0 0x5643c1b28a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3eecf27640 1 -- 172.21.15.94:0/2332947435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b217b0 msgr2=0x5643c1b25340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3eecf27640 1 --2- 172.21.15.94:0/2332947435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b217b0 0x5643c1b25340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3eecf27640 1 -- 172.21.15.94:0/2332947435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643c1a7e610 con 0x5643c1b18bd0 2024-02-17T12:02:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3eecf27640 1 --2- 172.21.15.94:0/2332947435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b18bd0 0x5643c1b24e00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8000c00 tx=0x7f3ed8035720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fea31529108386 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.365+0000 7f3ed0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:02:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ed0ff9640 1 -- 172.21.15.94:0/2332947435 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8004510 con 0x5643c1b18bd0 2024-02-17T12:02:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ed0ff9640 1 -- 172.21.15.94:0/2332947435 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3ed8044040 con 0x5643c1b18bd0 2024-02-17T12:02:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ef2565740 1 -- 172.21.15.94:0/2332947435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ee4004c80 con 0x5643c1b18bd0 2024-02-17T12:02:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ed0ff9640 1 -- 172.21.15.94:0/2332947435 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ed80046c0 con 0x5643c1b18bd0 2024-02-17T12:02:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ef2565740 10 client.16569 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ef2565740 1 -- 172.21.15.94:0/2332947435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5643c1a406a0 con 0x5643c1b18bd0 2024-02-17T12:02:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ed0ff9640 1 -- 172.21.15.94:0/2332947435 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3ed8048070 con 0x5643c1b18bd0 2024-02-17T12:02:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ed0ff9640 1 -- 172.21.15.94:0/2332947435 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3ed804c030 con 0x5643c1b18bd0 2024-02-17T12:02:19.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ef2565740 10 client.16569 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ef2565740 10 client.16569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ef2565740 20 client.16569 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ef2565740 1 --2- 172.21.15.94:0/2332947435 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5643c1b89110 0x5643c1b8b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ef2565740 4 client.16569 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3ef2565740 1 -- 172.21.15.94:0/2332947435 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f3ee4004c80 con 0x5643c1b89110 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.366+0000 7f3eed728640 1 --2- 172.21.15.94:0/2332947435 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5643c1b89110 0x5643c1b8b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ed0ff9640 10 client.16569 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3eed728640 1 --2- 172.21.15.94:0/2332947435 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5643c1b89110 0x5643c1b8b530 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5dc6f63debc2d3c server_cookie=3ce870578cae4337 in_seq=0 out_seq=0 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ed0ff9640 1 -- 172.21.15.94:0/2332947435 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7f3ee4004c80 con 0x5643c1b89110 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ed0ff9640 10 client.16569 handle_command_reply: tid=0 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ef2565740 1 client.16569 shutdown 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ef2565740 1 -- 172.21.15.94:0/2332947435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b18bd0 msgr2=0x5643c1b24e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ef2565740 1 --2- 172.21.15.94:0/2332947435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b18bd0 0x5643c1b24e00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8000c00 tx=0x7f3ed8035720 comp rx=0 tx=0).stop 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ef2565740 1 -- 172.21.15.94:0/2332947435 shutdown_connections 2024-02-17T12:02:19.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ef2565740 1 --2- 172.21.15.94:0/2332947435 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5643c1b89110 0x5643c1b8b530 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ed0ff9640 0 client.16569 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:19.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ef2565740 1 --2- 172.21.15.94:0/2332947435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5643c1b217b0 0x5643c1b25340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ef2565740 1 --2- 172.21.15.94:0/2332947435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5643c1b18bd0 0x5643c1b24e00 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ef2565740 1 --2- 172.21.15.94:0/2332947435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5643c1b182b0 0x5643c1b28a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ef2565740 1 -- 172.21.15.94:0/2332947435 >> 172.21.15.94:0/2332947435 conn(0x5643c1b12990 msgr2=0x5643c1b19810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:19.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ef2565740 1 -- 172.21.15.94:0/2332947435 shutdown_connections 2024-02-17T12:02:19.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ef2565740 1 -- 172.21.15.94:0/2332947435 wait complete. 2024-02-17T12:02:19.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.367+0000 7f3ef2565740 20 client.16569 trim_cache size 0 max 16384 2024-02-17T12:02:19.374 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:02:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:02:19.375 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:02:19.375 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:02:19.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.368+0000 7f3eef98c640 1 -- 172.21.15.94:0/1252556715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3ed40a2400 msgr2=0x7f3ed40a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:19.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.368+0000 7f3eef98c640 1 --2- 172.21.15.94:0/1252556715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3ed40a2400 0x7f3ed40a48c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3ee400a930 tx=0x7f3ee4005c00 comp rx=0 tx=0).stop 2024-02-17T12:02:19.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.368+0000 7f3eef98c640 1 -- 172.21.15.94:0/1252556715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee8105780 msgr2=0x7f3ee819bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:19.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.368+0000 7f3eef98c640 1 --2- 172.21.15.94:0/1252556715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee8105780 0x7f3ee819bfc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000eb10 tx=0x7f3ee000c2d0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.368+0000 7f3eef98c640 1 -- 172.21.15.94:0/1252556715 shutdown_connections 2024-02-17T12:02:19.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.368+0000 7f3eef98c640 1 --2- 172.21.15.94:0/1252556715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3ed40a2400 0x7f3ed40a48c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.368+0000 7f3eef98c640 1 --2- 172.21.15.94:0/1252556715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ee8105780 0x7f3ee819bfc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.368+0000 7f3eef98c640 1 --2- 172.21.15.94:0/1252556715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ee8100130 0x7f3ee8197c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.368+0000 7f3eef98c640 1 --2- 172.21.15.94:0/1252556715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ee80ff770 0x7f3ee8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.368+0000 7f3eef98c640 1 -- 172.21.15.94:0/1252556715 >> 172.21.15.94:0/1252556715 conn(0x7f3ee80765b0 msgr2=0x7f3ee8102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:19.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.368+0000 7f3eef98c640 1 -- 172.21.15.94:0/1252556715 shutdown_connections 2024-02-17T12:02:19.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.368+0000 7f3eef98c640 1 -- 172.21.15.94:0/1252556715 wait complete. 2024-02-17T12:02:19.380 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:02:19.380 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:02:19.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.533+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/1008146346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b68101d30 msgr2=0x7f7b68105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:19.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.533+0000 7f7b6fcb5640 1 --2- 172.21.15.94:0/1008146346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b68101d30 0x7f7b68105d70 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009930 tx=0x7f7b5802bc30 comp rx=0 tx=0).stop 2024-02-17T12:02:19.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.533+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/1008146346 shutdown_connections 2024-02-17T12:02:19.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.533+0000 7f7b6fcb5640 1 --2- 172.21.15.94:0/1008146346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b68106450 0x7f7b6810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.533+0000 7f7b6fcb5640 1 --2- 172.21.15.94:0/1008146346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b68101d30 0x7f7b68105d70 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.533+0000 7f7b6fcb5640 1 --2- 172.21.15.94:0/1008146346 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b68101380 0x7f7b68101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.534+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/1008146346 >> 172.21.15.94:0/1008146346 conn(0x7f7b68078580 msgr2=0x7f7b68078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:19.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.534+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/1008146346 shutdown_connections 2024-02-17T12:02:19.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.534+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/1008146346 wait complete. 2024-02-17T12:02:19.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.535+0000 7f7b6fcb5640 1 Processor -- start 2024-02-17T12:02:19.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.536+0000 7f7b6fcb5640 1 -- start start 2024-02-17T12:02:19.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.536+0000 7f7b6fcb5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b68101380 0x7f7b6819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.536+0000 7f7b6fcb5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b68106450 0x7f7b6819e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.536+0000 7f7b6fcb5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b681a2700 0x7f7b681a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.536+0000 7f7b6fcb5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b681a30f0 con 0x7f7b68106450 2024-02-17T12:02:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.536+0000 7f7b6fcb5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b681a3260 con 0x7f7b681a2700 2024-02-17T12:02:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.536+0000 7f7b6fcb5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b681a33d0 con 0x7f7b68101380 2024-02-17T12:02:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.536+0000 7f7b6da51640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b68101380 0x7f7b6819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.536+0000 7f7b6da51640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b68101380 0x7f7b6819de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55566/0 (socket says 172.21.15.94:55566) 2024-02-17T12:02:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.536+0000 7f7b6da51640 1 -- 172.21.15.94:0/144506717 learned_addr learned my addr 172.21.15.94:0/144506717 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.536+0000 7f7b6d250640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b68106450 0x7f7b6819e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.536+0000 7f7b6e252640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b681a2700 0x7f7b681a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.537+0000 7f7b6da51640 1 -- 172.21.15.94:0/144506717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b681a2700 msgr2=0x7f7b681a2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.537+0000 7f7b6da51640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b681a2700 0x7f7b681a2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.537+0000 7f7b6da51640 1 -- 172.21.15.94:0/144506717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b68106450 msgr2=0x7f7b6819e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.537+0000 7f7b6da51640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b68106450 0x7f7b6819e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.537+0000 7f7b6da51640 1 -- 172.21.15.94:0/144506717 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b58009560 con 0x7f7b68101380 2024-02-17T12:02:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.538+0000 7f7b6da51640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b68101380 0x7f7b6819de40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400d950 tx=0x7f7b6400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.538+0000 7f7b5effd640 1 -- 172.21.15.94:0/144506717 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b64009c30 con 0x7f7b68101380 2024-02-17T12:02:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.538+0000 7f7b5effd640 1 -- 172.21.15.94:0/144506717 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b64009d90 con 0x7f7b68101380 2024-02-17T12:02:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.538+0000 7f7b5effd640 1 -- 172.21.15.94:0/144506717 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400f7e0 con 0x7f7b68101380 2024-02-17T12:02:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.538+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/144506717 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b681aa9b0 con 0x7f7b68101380 2024-02-17T12:02:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.538+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/144506717 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b681aade0 con 0x7f7b68101380 2024-02-17T12:02:19.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.540+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/144506717 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b680676e0 con 0x7f7b68101380 2024-02-17T12:02:19.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.541+0000 7f7b5effd640 1 -- 172.21.15.94:0/144506717 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7b64010ad0 con 0x7f7b68101380 2024-02-17T12:02:19.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.542+0000 7f7b5effd640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7b540a2210 0x7f7b540a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:19.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.542+0000 7f7b5effd640 1 -- 172.21.15.94:0/144506717 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7b640cdd10 con 0x7f7b68101380 2024-02-17T12:02:19.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.543+0000 7f7b6d250640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7b540a2210 0x7f7b540a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:19.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.544+0000 7f7b6d250640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7b540a2210 0x7f7b540a46d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7b5802c170 tx=0x7f7b58000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:19.546 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.546+0000 7f7b5effd640 1 -- 172.21.15.94:0/144506717 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b640d3020 con 0x7f7b68101380 2024-02-17T12:02:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.909+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/144506717 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b68109740 con 0x7f7b68101380 2024-02-17T12:02:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.910+0000 7f7b5effd640 1 -- 172.21.15.94:0/144506717 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11344 (secure 0 0 0) 0x7f7b640969e0 con 0x7f7b68101380 2024-02-17T12:02:19.910 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:02:19.910 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:01:15.200235+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24418":{"gid":24418,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:02:19.910 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 23 2024-02-17T12:02:19.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.912+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/144506717 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7b540a2210 msgr2=0x7f7b540a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:19.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.912+0000 7f7b6fcb5640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7b540a2210 0x7f7b540a46d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7b5802c170 tx=0x7f7b58000bc0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.912+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/144506717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b68101380 msgr2=0x7f7b6819de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:19.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.912+0000 7f7b6fcb5640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b68101380 0x7f7b6819de40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7b6400d950 tx=0x7f7b6400de10 comp rx=0 tx=0).stop 2024-02-17T12:02:19.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.912+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/144506717 shutdown_connections 2024-02-17T12:02:19.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.912+0000 7f7b6fcb5640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7b540a2210 0x7f7b540a46d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.912+0000 7f7b6fcb5640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7b681a2700 0x7f7b681a2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.912+0000 7f7b6fcb5640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7b68106450 0x7f7b6819e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.912+0000 7f7b6fcb5640 1 --2- 172.21.15.94:0/144506717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b68101380 0x7f7b6819de40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:19.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.913+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/144506717 >> 172.21.15.94:0/144506717 conn(0x7f7b68078580 msgr2=0x7f7b680ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:19.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.913+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/144506717 shutdown_connections 2024-02-17T12:02:19.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:19.913+0000 7f7b6fcb5640 1 -- 172.21.15.94:0/144506717 wait complete. 2024-02-17T12:02:19.927 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:02:19.927 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:02:20.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.091+0000 7febce223640 1 -- 172.21.15.94:0/1539864056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc8103c40 msgr2=0x7febc81040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.091+0000 7febce223640 1 --2- 172.21.15.94:0/1539864056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc8103c40 0x7febc81040c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7febc0009cd0 tx=0x7febc002c120 comp rx=0 tx=0).stop 2024-02-17T12:02:20.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.091+0000 7febce223640 1 -- 172.21.15.94:0/1539864056 shutdown_connections 2024-02-17T12:02:20.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.091+0000 7febce223640 1 --2- 172.21.15.94:0/1539864056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc8103c40 0x7febc81040c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.091+0000 7febce223640 1 --2- 172.21.15.94:0/1539864056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc80688c0 0x7febc8068d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.091+0000 7febce223640 1 --2- 172.21.15.94:0/1539864056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc81060d0 0x7febc8068380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.091+0000 7febce223640 1 -- 172.21.15.94:0/1539864056 >> 172.21.15.94:0/1539864056 conn(0x7febc8077180 msgr2=0x7febc8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.098+0000 7febce223640 1 -- 172.21.15.94:0/1539864056 shutdown_connections 2024-02-17T12:02:20.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.099+0000 7febce223640 1 -- 172.21.15.94:0/1539864056 wait complete. 2024-02-17T12:02:20.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.100+0000 7febce223640 1 Processor -- start 2024-02-17T12:02:20.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.101+0000 7febce223640 1 -- start start 2024-02-17T12:02:20.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.101+0000 7febce223640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc80688c0 0x7febc8071ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.101+0000 7febce223640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc8103c40 0x7febc8072010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.101+0000 7febce223640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc81060d0 0x7febc8109570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.101+0000 7febce223640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc8109b70 con 0x7febc80688c0 2024-02-17T12:02:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.101+0000 7febce223640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc8109ce0 con 0x7febc8103c40 2024-02-17T12:02:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.101+0000 7febce223640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc8109ea0 con 0x7febc81060d0 2024-02-17T12:02:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.103+0000 7febc7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc81060d0 0x7febc8109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.103+0000 7febc7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc81060d0 0x7febc8109570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55582/0 (socket says 172.21.15.94:55582) 2024-02-17T12:02:20.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.103+0000 7febc7fff640 1 -- 172.21.15.94:0/1271216220 learned_addr learned my addr 172.21.15.94:0/1271216220 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.105+0000 7febc77fe640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc80688c0 0x7febc8071ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.106+0000 7febc7fff640 1 -- 172.21.15.94:0/1271216220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc8103c40 msgr2=0x7febc8072010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.104+0000 7febc6ffd640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc8103c40 0x7febc8072010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.107+0000 7febc7fff640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc8103c40 0x7febc8072010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.107+0000 7febc7fff640 1 -- 172.21.15.94:0/1271216220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc80688c0 msgr2=0x7febc8071ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.107+0000 7febc7fff640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc80688c0 0x7febc8071ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.107+0000 7febc7fff640 1 -- 172.21.15.94:0/1271216220 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc0009900 con 0x7febc81060d0 2024-02-17T12:02:20.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.110+0000 7febc7fff640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc81060d0 0x7febc8109570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7febc0037040 tx=0x7febc0032a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.113+0000 7febc4ff9640 1 -- 172.21.15.94:0/1271216220 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc0034850 con 0x7febc81060d0 2024-02-17T12:02:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.113+0000 7febce223640 1 -- 172.21.15.94:0/1271216220 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc810a180 con 0x7febc81060d0 2024-02-17T12:02:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.113+0000 7febce223640 1 -- 172.21.15.94:0/1271216220 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc8111a40 con 0x7febc81060d0 2024-02-17T12:02:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.113+0000 7febc4ff9640 1 -- 172.21.15.94:0/1271216220 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febc0003e50 con 0x7febc81060d0 2024-02-17T12:02:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.113+0000 7febc4ff9640 1 -- 172.21.15.94:0/1271216220 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc00406b0 con 0x7febc81060d0 2024-02-17T12:02:20.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.113+0000 7febc6ffd640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc8103c40 0x7febc8072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:02:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.123+0000 7febd0df6740 1 Processor -- start 2024-02-17T12:02:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.126+0000 7febc4ff9640 1 -- 172.21.15.94:0/1271216220 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7febc000a040 con 0x7febc81060d0 2024-02-17T12:02:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.126+0000 7febd0df6740 1 -- start start 2024-02-17T12:02:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.126+0000 7febd0df6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 0x55f255928630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.126+0000 7febd0df6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 0x55f255931210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.126+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 0x55f255928630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.126+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 0x55f255928630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41628/0 (socket says 172.21.15.94:41628) 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.127+0000 7febc77fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 0x55f255931210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.127+0000 7febc77fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 0x55f255931210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55586/0 (socket says 172.21.15.94:55586) 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.127+0000 7febc6ffd640 1 -- 172.21.15.94:0/584395866 learned_addr learned my addr 172.21.15.94:0/584395866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.127+0000 7febd0df6740 1 --2- 172.21.15.94:0/584395866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255931750 0x55f255933b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.127+0000 7febd0df6740 1 -- 172.21.15.94:0/584395866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2558e3960 con 0x55f255928250 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.127+0000 7febd0df6740 1 -- 172.21.15.94:0/584395866 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f255897310 con 0x55f255931750 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.127+0000 7febd0df6740 1 -- 172.21.15.94:0/584395866 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f255894840 con 0x55f255928b70 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.127+0000 7febc4ff9640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7febb00a23c0 0x7febb00a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.128+0000 7febc4ff9640 1 -- 172.21.15.94:0/1271216220 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7febc00f0db0 con 0x7febc81060d0 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.128+0000 7febc77fe640 1 -- 172.21.15.94:0/584395866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255931750 msgr2=0x55f255933b40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.128+0000 7febc77fe640 1 --2- 172.21.15.94:0/584395866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255931750 0x55f255933b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.128+0000 7febc77fe640 1 -- 172.21.15.94:0/584395866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 msgr2=0x55f255928630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.128+0000 7febc77fe640 1 --2- 172.21.15.94:0/584395866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 0x55f255928630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.128+0000 7febc77fe640 1 -- 172.21.15.94:0/584395866 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f255852060 con 0x55f255928b70 2024-02-17T12:02:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.129+0000 7febc7fff640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7febb00a23c0 0x7febb00a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.130+0000 7febc7fff640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7febb00a23c0 0x7febb00a4880 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7febc0032b00 tx=0x7febc002c120 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.130+0000 7febc77fe640 1 --2- 172.21.15.94:0/584395866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 0x55f255931210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7febbc009980 tx=0x7febbc009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d6740d2b605ca57 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.130+0000 7febadffb640 1 -- 172.21.15.94:0/584395866 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc00ea80 con 0x55f255928b70 2024-02-17T12:02:20.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.130+0000 7febadffb640 1 -- 172.21.15.94:0/584395866 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febbc00ebe0 con 0x55f255928b70 2024-02-17T12:02:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.131+0000 7febadffb640 1 -- 172.21.15.94:0/584395866 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc0346d0 con 0x55f255928b70 2024-02-17T12:02:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.131+0000 7febd0df6740 1 -- 172.21.15.94:0/584395866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 msgr2=0x55f255931210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.131+0000 7febd0df6740 1 --2- 172.21.15.94:0/584395866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 0x55f255931210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7febbc009980 tx=0x7febbc009e40 comp rx=0 tx=0).stop 2024-02-17T12:02:20.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.131+0000 7febd0df6740 1 -- 172.21.15.94:0/584395866 shutdown_connections 2024-02-17T12:02:20.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.131+0000 7febd0df6740 1 --2- 172.21.15.94:0/584395866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255931750 0x55f255933b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.131+0000 7febd0df6740 1 --2- 172.21.15.94:0/584395866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 0x55f255931210 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.131+0000 7febd0df6740 1 --2- 172.21.15.94:0/584395866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 0x55f255928630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.131+0000 7febd0df6740 1 -- 172.21.15.94:0/584395866 >> 172.21.15.94:0/584395866 conn(0x55f255922980 msgr2=0x55f255922d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.132+0000 7febd0df6740 1 -- 172.21.15.94:0/584395866 shutdown_connections 2024-02-17T12:02:20.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.132+0000 7febd0df6740 1 -- 172.21.15.94:0/584395866 wait complete. 2024-02-17T12:02:20.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.132+0000 7febd0df6740 1 -- start start 2024-02-17T12:02:20.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.132+0000 7febd0df6740 1 Processor -- start 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.133+0000 7febd0df6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 0x55f25592e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.133+0000 7febd0df6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928b70 0x55f25592eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.134+0000 7febd0df6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255931750 0x55f25592f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.134+0000 7febd0df6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f255890020 con 0x55f255928250 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.134+0000 7febd0df6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f255563430 con 0x55f255928b70 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.134+0000 7febd0df6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f25551fe00 con 0x55f255931750 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.134+0000 7febc77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928b70 0x55f25592eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.134+0000 7febc77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928b70 0x55f25592eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46674/0 (socket says 172.21.15.94:46674) 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.134+0000 7febc77fe640 1 -- 172.21.15.94:0/181985351 learned_addr learned my addr 172.21.15.94:0/181985351 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.134+0000 7febc6ffd640 1 --2- 172.21.15.94:0/181985351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255931750 0x55f25592f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.134+0000 7febd0df6740 10 client.0 _ll_register_callbacks cb 0x55f255837cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.135+0000 7febc6ffd640 1 -- 172.21.15.94:0/181985351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928b70 msgr2=0x55f25592eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.135+0000 7febc6ffd640 1 --2- 172.21.15.94:0/181985351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928b70 0x55f25592eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.135+0000 7febc6ffd640 1 -- 172.21.15.94:0/181985351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 msgr2=0x55f25592e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.135+0000 7febc6ffd640 1 --2- 172.21.15.94:0/181985351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 0x55f25592e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.135+0000 7febc6ffd640 1 -- 172.21.15.94:0/181985351 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febbc0095e0 con 0x55f255931750 2024-02-17T12:02:20.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.135+0000 7febc6ffd640 1 --2- 172.21.15.94:0/181985351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255931750 0x55f25592f060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7febb8006ce0 tx=0x7febb8010960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec2ab845a0a898f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.136+0000 7feba6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:02:20.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.136+0000 7feba6ffd640 1 -- 172.21.15.94:0/181985351 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8012600 con 0x55f255931750 2024-02-17T12:02:20.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.136+0000 7feba6ffd640 1 -- 172.21.15.94:0/181985351 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7febb8012760 con 0x55f255931750 2024-02-17T12:02:20.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.137+0000 7febd0df6740 1 -- 172.21.15.94:0/181985351 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f25558f7e0 con 0x55f255931750 2024-02-17T12:02:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.137+0000 7feba6ffd640 1 -- 172.21.15.94:0/181985351 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7febb8012940 con 0x55f255931750 2024-02-17T12:02:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.137+0000 7febd0df6740 10 client.26089 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.137+0000 7febd0df6740 1 -- 172.21.15.94:0/181985351 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f25588e6a0 con 0x55f255931750 2024-02-17T12:02:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.138+0000 7feba6ffd640 1 -- 172.21.15.94:0/181985351 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f25558f7e0 con 0x55f255931750 2024-02-17T12:02:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.138+0000 7feba6ffd640 1 -- 172.21.15.94:0/181985351 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7febb801d430 con 0x55f255931750 2024-02-17T12:02:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.138+0000 7febd0df6740 10 client.26089 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:20.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.138+0000 7febd0df6740 10 client.26089 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.138+0000 7febd0df6740 20 client.26089 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.138+0000 7febd0df6740 1 --2- 172.21.15.94:0/181985351 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55f255941990 0x55f255943db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.138+0000 7febd0df6740 4 client.26089 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.138+0000 7febd0df6740 1 -- 172.21.15.94:0/181985351 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2555cc720 con 0x55f255941990 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.138+0000 7febc77fe640 1 --2- 172.21.15.94:0/181985351 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55f255941990 0x55f255943db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.139+0000 7feba6ffd640 10 client.26089 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.139+0000 7febc77fe640 1 --2- 172.21.15.94:0/181985351 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55f255941990 0x55f255943db0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e78bce3578fc7a4 server_cookie=38e5a1a3f4186311 in_seq=0 out_seq=0 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.140+0000 7feba6ffd640 1 -- 172.21.15.94:0/181985351 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f2555cc720 con 0x55f255941990 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.140+0000 7feba6ffd640 10 client.26089 handle_command_reply: tid=0 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.140+0000 7febd0df6740 1 client.26089 shutdown 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.140+0000 7febd0df6740 1 -- 172.21.15.94:0/181985351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255931750 msgr2=0x55f25592f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.140+0000 7febd0df6740 1 --2- 172.21.15.94:0/181985351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255931750 0x55f25592f060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7febb8006ce0 tx=0x7febb8010960 comp rx=0 tx=0).stop 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.140+0000 7febd0df6740 1 -- 172.21.15.94:0/181985351 shutdown_connections 2024-02-17T12:02:20.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.140+0000 7febd0df6740 1 --2- 172.21.15.94:0/181985351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255931750 0x55f25592f060 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.140+0000 7febd0df6740 1 --2- 172.21.15.94:0/181985351 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55f255941990 0x55f255943db0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.140+0000 7febd0df6740 1 --2- 172.21.15.94:0/181985351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928b70 0x55f25592eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.140+0000 7febd0df6740 1 --2- 172.21.15.94:0/181985351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 0x55f25592e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.140+0000 7febd0df6740 1 -- 172.21.15.94:0/181985351 >> 172.21.15.94:0/181985351 conn(0x55f255922980 msgr2=0x55f2559246d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.140+0000 7feba6ffd640 0 client.26089 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:02:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.141+0000 7febd0df6740 1 -- 172.21.15.94:0/181985351 shutdown_connections 2024-02-17T12:02:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.141+0000 7febd0df6740 1 -- 172.21.15.94:0/181985351 wait complete. 2024-02-17T12:02:20.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.141+0000 7febd0df6740 20 client.26089 trim_cache size 0 max 16384 2024-02-17T12:02:20.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.161+0000 7febd0df6740 1 Processor -- start 2024-02-17T12:02:20.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.161+0000 7febd0df6740 1 -- start start 2024-02-17T12:02:20.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febd0df6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928250 0x55f255930d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febd0df6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 0x55f25592d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febd0df6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255931750 0x55f25592ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febd0df6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f255815130 con 0x55f255931750 2024-02-17T12:02:20.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febd0df6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f255927eb0 con 0x55f255928250 2024-02-17T12:02:20.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febd0df6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f255948fe0 con 0x55f255928b70 2024-02-17T12:02:20.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febc77fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 0x55f25592d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febc77fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 0x55f25592d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55608/0 (socket says 172.21.15.94:55608) 2024-02-17T12:02:20.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febc77fe640 1 -- 172.21.15.94:0/2145560795 learned_addr learned my addr 172.21.15.94:0/2145560795 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febc6ffd640 1 --2- 172.21.15.94:0/2145560795 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928250 0x55f255930d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febc6ffd640 1 --2- 172.21.15.94:0/2145560795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255931750 0x55f25592ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febc6ffd640 1 -- 172.21.15.94:0/2145560795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 msgr2=0x55f25592d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febc6ffd640 1 --2- 172.21.15.94:0/2145560795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 0x55f25592d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febc6ffd640 1 -- 172.21.15.94:0/2145560795 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928250 msgr2=0x55f255930d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febc6ffd640 1 --2- 172.21.15.94:0/2145560795 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928250 0x55f255930d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febc6ffd640 1 -- 172.21.15.94:0/2145560795 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f25582ac20 con 0x55f255931750 2024-02-17T12:02:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.162+0000 7febc77fe640 1 --2- 172.21.15.94:0/2145560795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 0x55f25592d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.163+0000 7febc6ffd640 1 --2- 172.21.15.94:0/2145560795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255931750 0x55f25592ee80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7febb800e600 tx=0x7febb8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a264c0d8804e1f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.163+0000 7febad7fa640 1 -- 172.21.15.94:0/2145560795 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb800a070 con 0x55f255931750 2024-02-17T12:02:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.163+0000 7febad7fa640 1 -- 172.21.15.94:0/2145560795 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febb802e070 con 0x55f255931750 2024-02-17T12:02:20.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.163+0000 7febad7fa640 1 -- 172.21.15.94:0/2145560795 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8012d40 con 0x55f255931750 2024-02-17T12:02:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.164+0000 7febd0df6740 1 -- 172.21.15.94:0/2145560795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255931750 msgr2=0x55f25592ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.164+0000 7febd0df6740 1 --2- 172.21.15.94:0/2145560795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255931750 0x55f25592ee80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7febb800e600 tx=0x7febb8000bc0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.164+0000 7febd0df6740 1 -- 172.21.15.94:0/2145560795 shutdown_connections 2024-02-17T12:02:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.164+0000 7febd0df6740 1 --2- 172.21.15.94:0/2145560795 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255931750 0x55f25592ee80 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.164+0000 7febd0df6740 1 --2- 172.21.15.94:0/2145560795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255928b70 0x55f25592d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.164+0000 7febd0df6740 1 --2- 172.21.15.94:0/2145560795 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928250 0x55f255930d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.164+0000 7febd0df6740 1 -- 172.21.15.94:0/2145560795 >> 172.21.15.94:0/2145560795 conn(0x55f255922980 msgr2=0x55f25592ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.164+0000 7febd0df6740 1 -- 172.21.15.94:0/2145560795 shutdown_connections 2024-02-17T12:02:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.164+0000 7febd0df6740 1 -- 172.21.15.94:0/2145560795 wait complete. 2024-02-17T12:02:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.164+0000 7febd0df6740 1 -- start start 2024-02-17T12:02:20.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.165+0000 7febd0df6740 1 Processor -- start 2024-02-17T12:02:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.165+0000 7febd0df6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 0x55f255936950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.165+0000 7febd0df6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928b70 0x55f255936e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.165+0000 7febd0df6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255931750 0x55f255990ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.165+0000 7febd0df6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2559913e0 con 0x55f255928250 2024-02-17T12:02:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.165+0000 7febd0df6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f255991550 con 0x55f255928b70 2024-02-17T12:02:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.165+0000 7febd0df6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f255991710 con 0x55f255931750 2024-02-17T12:02:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.165+0000 7febd0df6740 10 client.0 _ll_register_callbacks cb 0x7febbc0093d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.166+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255931750 0x55f255990ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.166+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 0x55f255936950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.166+0000 7febc6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 0x55f255936950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41668/0 (socket says 172.21.15.94:41668) 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.166+0000 7febc6ffd640 1 -- 172.21.15.94:0/2756930910 learned_addr learned my addr 172.21.15.94:0/2756930910 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.166+0000 7febc77fe640 1 --2- 172.21.15.94:0/2756930910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928b70 0x55f255936e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.166+0000 7febc6ffd640 1 -- 172.21.15.94:0/2756930910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928b70 msgr2=0x55f255936e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.166+0000 7febc6ffd640 1 --2- 172.21.15.94:0/2756930910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928b70 0x55f255936e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.166+0000 7febc6ffd640 1 -- 172.21.15.94:0/2756930910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 msgr2=0x55f255936950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.166+0000 7febc6ffd640 1 --2- 172.21.15.94:0/2756930910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 0x55f255936950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.166+0000 7febc6ffd640 1 -- 172.21.15.94:0/2756930910 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f255852060 con 0x55f255931750 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.166+0000 7febc6ffd640 1 --2- 172.21.15.94:0/2756930910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255931750 0x55f255990ea0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7febb8010220 tx=0x7febb8010460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdf50dc9eee7be24 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7feba6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7feba6ffd640 1 -- 172.21.15.94:0/2756930910 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8004450 con 0x55f255931750 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7feba6ffd640 1 -- 172.21.15.94:0/2756930910 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7febb80045b0 con 0x55f255931750 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7febd0df6740 1 -- 172.21.15.94:0/2756930910 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febbc00ebe0 con 0x55f255931750 2024-02-17T12:02:20.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7feba6ffd640 1 -- 172.21.15.94:0/2756930910 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7febb8005990 con 0x55f255931750 2024-02-17T12:02:20.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7febd0df6740 10 client.26095 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:20.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7febd0df6740 1 -- 172.21.15.94:0/2756930910 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f25588e6a0 con 0x55f255931750 2024-02-17T12:02:20.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7feba6ffd640 1 -- 172.21.15.94:0/2756930910 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7febbc00ebe0 con 0x55f255931750 2024-02-17T12:02:20.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7feba6ffd640 1 -- 172.21.15.94:0/2756930910 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7febb8012960 con 0x55f255931750 2024-02-17T12:02:20.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7febd0df6740 10 client.26095 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:20.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7febd0df6740 10 client.26095 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:02:20.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7febd0df6740 20 client.26095 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:20.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7febd0df6740 1 --2- 172.21.15.94:0/2756930910 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55f255998b20 0x55f25599af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7febd0df6740 4 client.26095 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:02:20.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7febd0df6740 1 -- 172.21.15.94:0/2756930910 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f25561b890 con 0x55f255998b20 2024-02-17T12:02:20.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.167+0000 7febc77fe640 1 --2- 172.21.15.94:0/2756930910 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55f255998b20 0x55f25599af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.168+0000 7febc77fe640 1 --2- 172.21.15.94:0/2756930910 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55f255998b20 0x55f25599af10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be9cb27948a29455 server_cookie=5b31bf38513c93f9 in_seq=0 out_seq=0 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.168+0000 7feba6ffd640 10 client.26095 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.168+0000 7feba6ffd640 1 -- 172.21.15.94:0/2756930910 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f25561b890 con 0x55f255998b20 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.168+0000 7feba6ffd640 10 client.26095 handle_command_reply: tid=0 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.169+0000 7febd0df6740 1 client.26095 shutdown 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febd0df6740 1 -- 172.21.15.94:0/2756930910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255931750 msgr2=0x55f255990ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febd0df6740 1 --2- 172.21.15.94:0/2756930910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255931750 0x55f255990ea0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7febb8010220 tx=0x7febb8010460 comp rx=0 tx=0).stop 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febd0df6740 1 -- 172.21.15.94:0/2756930910 shutdown_connections 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febd0df6740 1 --2- 172.21.15.94:0/2756930910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f255931750 0x55f255990ea0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febd0df6740 1 --2- 172.21.15.94:0/2756930910 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55f255998b20 0x55f25599af10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febd0df6740 1 --2- 172.21.15.94:0/2756930910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f255928b70 0x55f255936e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febd0df6740 1 --2- 172.21.15.94:0/2756930910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f255928250 0x55f255936950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febd0df6740 1 -- 172.21.15.94:0/2756930910 >> 172.21.15.94:0/2756930910 conn(0x55f255922980 msgr2=0x55f255921b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7feba6ffd640 0 client.26095 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:02:20.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febd0df6740 1 -- 172.21.15.94:0/2756930910 shutdown_connections 2024-02-17T12:02:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febd0df6740 1 -- 172.21.15.94:0/2756930910 wait complete. 2024-02-17T12:02:20.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febd0df6740 20 client.26095 trim_cache size 0 max 16384 2024-02-17T12:02:20.173 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:02:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febce223640 1 -- 172.21.15.94:0/1271216220 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7febb00a23c0 msgr2=0x7febb00a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febce223640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7febb00a23c0 0x7febb00a4880 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7febc0032b00 tx=0x7febc002c120 comp rx=0 tx=0).stop 2024-02-17T12:02:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febce223640 1 -- 172.21.15.94:0/1271216220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc81060d0 msgr2=0x7febc8109570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.170+0000 7febce223640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc81060d0 0x7febc8109570 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7febc0037040 tx=0x7febc0032a90 comp rx=0 tx=0).stop 2024-02-17T12:02:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.171+0000 7febce223640 1 -- 172.21.15.94:0/1271216220 shutdown_connections 2024-02-17T12:02:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.171+0000 7febce223640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7febb00a23c0 0x7febb00a4880 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.171+0000 7febce223640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7febc81060d0 0x7febc8109570 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.171+0000 7febce223640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7febc8103c40 0x7febc8072010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.171+0000 7febce223640 1 --2- 172.21.15.94:0/1271216220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7febc80688c0 0x7febc8071ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.171+0000 7febce223640 1 -- 172.21.15.94:0/1271216220 >> 172.21.15.94:0/1271216220 conn(0x7febc8077180 msgr2=0x7febc8101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.171+0000 7febce223640 1 -- 172.21.15.94:0/1271216220 shutdown_connections 2024-02-17T12:02:20.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.171+0000 7febce223640 1 -- 172.21.15.94:0/1271216220 wait complete. 2024-02-17T12:02:20.184 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:19 smithi094 ceph-mon[32499]: pgmap v129: 97 pgs: 97 active+clean; 3.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 MiB/s wr, 1.82k op/s 2024-02-17T12:02:20.184 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:19 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/144506717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:02:20.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:19 smithi125 ceph-mon[36819]: pgmap v129: 97 pgs: 97 active+clean; 3.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 MiB/s wr, 1.82k op/s 2024-02-17T12:02:20.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:19 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/144506717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:02:20.185 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:02:20.185 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:02:20.201 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:19 smithi189 ceph-mon[36588]: pgmap v129: 97 pgs: 97 active+clean; 3.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 MiB/s wr, 1.82k op/s 2024-02-17T12:02:20.202 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:19 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/144506717' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:02:20.242 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 240.0 s, 2852.2 tps, lat 35.308 ms stddev 40.871 2024-02-17T12:02:20.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.335+0000 7fe232da0640 1 -- 172.21.15.94:0/584356153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe22c100570 msgr2=0x7fe22c104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.335+0000 7fe232da0640 1 --2- 172.21.15.94:0/584356153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe22c100570 0x7fe22c104310 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe21c009930 tx=0x7fe21c02bc20 comp rx=0 tx=0).stop 2024-02-17T12:02:20.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.335+0000 7fe232da0640 1 -- 172.21.15.94:0/584356153 shutdown_connections 2024-02-17T12:02:20.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.335+0000 7fe232da0640 1 --2- 172.21.15.94:0/584356153 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe22c104a40 0x7fe22c10c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.335+0000 7fe232da0640 1 --2- 172.21.15.94:0/584356153 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe22c100570 0x7fe22c104310 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.335+0000 7fe232da0640 1 --2- 172.21.15.94:0/584356153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22c0ffbc0 0x7fe22c0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.335+0000 7fe232da0640 1 -- 172.21.15.94:0/584356153 >> 172.21.15.94:0/584356153 conn(0x7fe22c0752a0 msgr2=0x7fe22c0756b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.336+0000 7fe232da0640 1 -- 172.21.15.94:0/584356153 shutdown_connections 2024-02-17T12:02:20.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.336+0000 7fe232da0640 1 -- 172.21.15.94:0/584356153 wait complete. 2024-02-17T12:02:20.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.336+0000 7fe232da0640 1 Processor -- start 2024-02-17T12:02:20.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.337+0000 7fe232da0640 1 -- start start 2024-02-17T12:02:20.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.337+0000 7fe232da0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22c0ffbc0 0x7fe22c1a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.337+0000 7fe232da0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe22c104a40 0x7fe22c1a3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.337+0000 7fe232da0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe22c19deb0 0x7fe22c19e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.337+0000 7fe232da0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c19e900 con 0x7fe22c19deb0 2024-02-17T12:02:20.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.337+0000 7fe232da0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c19ea70 con 0x7fe22c104a40 2024-02-17T12:02:20.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.337+0000 7fe232da0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c19ec30 con 0x7fe22c0ffbc0 2024-02-17T12:02:20.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.337+0000 7fe231d9e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22c0ffbc0 0x7fe22c1a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.337+0000 7fe231d9e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22c0ffbc0 0x7fe22c1a3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55620/0 (socket says 172.21.15.94:55620) 2024-02-17T12:02:20.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.337+0000 7fe231d9e640 1 -- 172.21.15.94:0/3095266846 learned_addr learned my addr 172.21.15.94:0/3095266846 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.337+0000 7fe23259f640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe22c19deb0 0x7fe22c19e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.338+0000 7fe23159d640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe22c104a40 0x7fe22c1a3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.338+0000 7fe231d9e640 1 -- 172.21.15.94:0/3095266846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe22c104a40 msgr2=0x7fe22c1a3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.338+0000 7fe231d9e640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe22c104a40 0x7fe22c1a3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.338+0000 7fe231d9e640 1 -- 172.21.15.94:0/3095266846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe22c19deb0 msgr2=0x7fe22c19e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.338+0000 7fe231d9e640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe22c19deb0 0x7fe22c19e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.338+0000 7fe231d9e640 1 -- 172.21.15.94:0/3095266846 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe224009840 con 0x7fe22c0ffbc0 2024-02-17T12:02:20.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.338+0000 7fe23159d640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe22c104a40 0x7fe22c1a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:02:20.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.338+0000 7fe231d9e640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22c0ffbc0 0x7fe22c1a3960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c970 tx=0x7fe22800cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.338+0000 7fe222ffd640 1 -- 172.21.15.94:0/3095266846 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228007b50 con 0x7fe22c0ffbc0 2024-02-17T12:02:20.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.339+0000 7fe232da0640 1 -- 172.21.15.94:0/3095266846 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21c009560 con 0x7fe22c0ffbc0 2024-02-17T12:02:20.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.339+0000 7fe222ffd640 1 -- 172.21.15.94:0/3095266846 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe228007cb0 con 0x7fe22c0ffbc0 2024-02-17T12:02:20.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.339+0000 7fe232da0640 1 -- 172.21.15.94:0/3095266846 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22c1aab40 con 0x7fe22c0ffbc0 2024-02-17T12:02:20.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.339+0000 7fe222ffd640 1 -- 172.21.15.94:0/3095266846 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22800f740 con 0x7fe22c0ffbc0 2024-02-17T12:02:20.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.341+0000 7fe222ffd640 1 -- 172.21.15.94:0/3095266846 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe228018480 con 0x7fe22c0ffbc0 2024-02-17T12:02:20.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.342+0000 7fe236bce740 1 Processor -- start 2024-02-17T12:02:20.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.342+0000 7fe236bce740 1 -- start start 2024-02-17T12:02:20.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.342+0000 7fe222ffd640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe2180a2230 0x7fe2180a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.342+0000 7fe236bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513440 0x561fd9513820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.342+0000 7fe236bce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513d60 0x561fd951c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.342+0000 7fe236bce740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd951c940 0x561fd951ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.342+0000 7fe236bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd94ce960 con 0x561fd9513440 2024-02-17T12:02:20.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.342+0000 7fe236bce740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd9482310 con 0x561fd951c940 2024-02-17T12:02:20.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.342+0000 7fe236bce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd947f840 con 0x561fd9513d60 2024-02-17T12:02:20.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.342+0000 7fe23259f640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe2180a2230 0x7fe2180a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.343+0000 7fe23159d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513d60 0x561fd951c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.343+0000 7fe231d9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513440 0x561fd9513820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.343+0000 7fe23259f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd951c940 0x561fd951ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.343+0000 7fe23159d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513d60 0x561fd951c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55622/0 (socket says 172.21.15.94:55622) 2024-02-17T12:02:20.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.343+0000 7fe23159d640 1 -- 172.21.15.94:0/3357614863 learned_addr learned my addr 172.21.15.94:0/3357614863 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.343+0000 7fe222ffd640 1 -- 172.21.15.94:0/3095266846 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe228013070 con 0x7fe22c0ffbc0 2024-02-17T12:02:20.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.343+0000 7fe23159d640 1 -- 172.21.15.94:0/3357614863 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd951c940 msgr2=0x561fd951ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.343+0000 7fe23159d640 1 --2- 172.21.15.94:0/3357614863 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd951c940 0x561fd951ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.343+0000 7fe23159d640 1 -- 172.21.15.94:0/3357614863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513440 msgr2=0x561fd9513820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.343+0000 7fe23159d640 1 --2- 172.21.15.94:0/3357614863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513440 0x561fd9513820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.343+0000 7fe23159d640 1 -- 172.21.15.94:0/3357614863 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fd943d060 con 0x561fd9513d60 2024-02-17T12:02:20.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.344+0000 7fe23259f640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe2180a2230 0x7fe2180a46f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe224005fd0 tx=0x7fe2240139f0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.345+0000 7fe23159d640 1 --2- 172.21.15.94:0/3357614863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513d60 0x561fd951c400 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe21c036040 tx=0x7fe21c02c640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40749c77cdc7a9c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.345+0000 7fe217fff640 1 -- 172.21.15.94:0/3357614863 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c0049c0 con 0x561fd9513d60 2024-02-17T12:02:20.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.345+0000 7fe217fff640 1 -- 172.21.15.94:0/3357614863 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe21c004020 con 0x561fd9513d60 2024-02-17T12:02:20.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.345+0000 7fe217fff640 1 -- 172.21.15.94:0/3357614863 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c005220 con 0x561fd9513d60 2024-02-17T12:02:20.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.346+0000 7fe236bce740 1 -- 172.21.15.94:0/3357614863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513d60 msgr2=0x561fd951c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.346+0000 7fe236bce740 1 --2- 172.21.15.94:0/3357614863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513d60 0x561fd951c400 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe21c036040 tx=0x7fe21c02c640 comp rx=0 tx=0).stop 2024-02-17T12:02:20.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.346+0000 7fe236bce740 1 -- 172.21.15.94:0/3357614863 shutdown_connections 2024-02-17T12:02:20.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.346+0000 7fe236bce740 1 --2- 172.21.15.94:0/3357614863 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd951c940 0x561fd951ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.346+0000 7fe236bce740 1 --2- 172.21.15.94:0/3357614863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513d60 0x561fd951c400 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.346+0000 7fe236bce740 1 --2- 172.21.15.94:0/3357614863 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513440 0x561fd9513820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.346+0000 7fe236bce740 1 -- 172.21.15.94:0/3357614863 >> 172.21.15.94:0/3357614863 conn(0x561fd950db70 msgr2=0x561fd950df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.346+0000 7fe236bce740 1 -- 172.21.15.94:0/3357614863 shutdown_connections 2024-02-17T12:02:20.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.346+0000 7fe236bce740 1 -- 172.21.15.94:0/3357614863 wait complete. 2024-02-17T12:02:20.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.346+0000 7fe236bce740 1 -- start start 2024-02-17T12:02:20.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.347+0000 7fe236bce740 1 Processor -- start 2024-02-17T12:02:20.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.347+0000 7fe236bce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513440 0x561fd95199b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.347+0000 7fe236bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513d60 0x561fd9519ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.347+0000 7fe236bce740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd951c940 0x561fd951a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.347+0000 7fe236bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd947b020 con 0x561fd9513d60 2024-02-17T12:02:20.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.347+0000 7fe236bce740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd914e430 con 0x561fd951c940 2024-02-17T12:02:20.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.347+0000 7fe236bce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd910ae00 con 0x561fd9513440 2024-02-17T12:02:20.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.347+0000 7fe23259f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd951c940 0x561fd951a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe23259f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd951c940 0x561fd951a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46728/0 (socket says 172.21.15.94:46728) 2024-02-17T12:02:20.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe23259f640 1 -- 172.21.15.94:0/1686790087 learned_addr learned my addr 172.21.15.94:0/1686790087 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe23159d640 1 --2- 172.21.15.94:0/1686790087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513d60 0x561fd9519ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe231d9e640 1 --2- 172.21.15.94:0/1686790087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513440 0x561fd95199b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe236bce740 10 client.0 _ll_register_callbacks cb 0x561fd9422cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:20.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe23259f640 1 -- 172.21.15.94:0/1686790087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513440 msgr2=0x561fd95199b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe23259f640 1 --2- 172.21.15.94:0/1686790087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513440 0x561fd95199b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe23259f640 1 -- 172.21.15.94:0/1686790087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513d60 msgr2=0x561fd9519ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe23259f640 1 --2- 172.21.15.94:0/1686790087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513d60 0x561fd9519ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe23259f640 1 -- 172.21.15.94:0/1686790087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22c067a50 con 0x561fd951c940 2024-02-17T12:02:20.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe23259f640 1 --2- 172.21.15.94:0/1686790087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd951c940 0x561fd951a430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe224003bf0 tx=0x7fe22401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41ff92f1dc13589a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe2157fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:02:20.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe2157fa640 1 -- 172.21.15.94:0/1686790087 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224004680 con 0x561fd951c940 2024-02-17T12:02:20.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe2157fa640 1 -- 172.21.15.94:0/1686790087 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe224025030 con 0x561fd951c940 2024-02-17T12:02:20.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe236bce740 1 -- 172.21.15.94:0/1686790087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fd917a7e0 con 0x561fd951c940 2024-02-17T12:02:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe2157fa640 1 -- 172.21.15.94:0/1686790087 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2240047e0 con 0x561fd951c940 2024-02-17T12:02:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe236bce740 10 client.26057 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.348+0000 7fe236bce740 1 -- 172.21.15.94:0/1686790087 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x561fd94796a0 con 0x561fd951c940 2024-02-17T12:02:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.349+0000 7fe2157fa640 1 -- 172.21.15.94:0/1686790087 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561fd917a7e0 con 0x561fd951c940 2024-02-17T12:02:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.349+0000 7fe2157fa640 1 -- 172.21.15.94:0/1686790087 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe22401f4b0 con 0x561fd951c940 2024-02-17T12:02:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.349+0000 7fe236bce740 10 client.26057 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:20.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.349+0000 7fe236bce740 10 client.26057 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:02:20.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.349+0000 7fe236bce740 20 client.26057 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:20.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.349+0000 7fe236bce740 1 --2- 172.21.15.94:0/1686790087 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x561fd952cd50 0x561fd952f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.349+0000 7fe236bce740 4 client.26057 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:20.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.349+0000 7fe236bce740 1 -- 172.21.15.94:0/1686790087 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fd91b7720 con 0x561fd952cd50 2024-02-17T12:02:20.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.349+0000 7fe231d9e640 1 --2- 172.21.15.94:0/1686790087 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x561fd952cd50 0x561fd952f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.350+0000 7fe2157fa640 10 client.26057 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:02:20.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.350+0000 7fe231d9e640 1 --2- 172.21.15.94:0/1686790087 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x561fd952cd50 0x561fd952f170 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da89daecb6e4002a server_cookie=21b3bc3937463958 in_seq=0 out_seq=0 2024-02-17T12:02:20.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.350+0000 7fe2157fa640 1 -- 172.21.15.94:0/1686790087 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561fd91b7720 con 0x561fd952cd50 2024-02-17T12:02:20.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.350+0000 7fe2157fa640 10 client.26057 handle_command_reply: tid=0 2024-02-17T12:02:20.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.350+0000 7fe236bce740 1 client.26057 shutdown 2024-02-17T12:02:20.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.351+0000 7fe236bce740 1 -- 172.21.15.94:0/1686790087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd951c940 msgr2=0x561fd951a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.351+0000 7fe236bce740 1 --2- 172.21.15.94:0/1686790087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd951c940 0x561fd951a430 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe224003bf0 tx=0x7fe22401d000 comp rx=0 tx=0).stop 2024-02-17T12:02:20.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.351+0000 7fe236bce740 1 -- 172.21.15.94:0/1686790087 shutdown_connections 2024-02-17T12:02:20.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.351+0000 7fe236bce740 1 --2- 172.21.15.94:0/1686790087 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x561fd952cd50 0x561fd952f170 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.351+0000 7fe236bce740 1 --2- 172.21.15.94:0/1686790087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd951c940 0x561fd951a430 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.351+0000 7fe2157fa640 0 client.26057 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:02:20.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.351+0000 7fe236bce740 1 --2- 172.21.15.94:0/1686790087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513d60 0x561fd9519ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.351+0000 7fe236bce740 1 --2- 172.21.15.94:0/1686790087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513440 0x561fd95199b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.351+0000 7fe236bce740 1 -- 172.21.15.94:0/1686790087 >> 172.21.15.94:0/1686790087 conn(0x561fd950db70 msgr2=0x561fd950f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.352+0000 7fe236bce740 1 -- 172.21.15.94:0/1686790087 shutdown_connections 2024-02-17T12:02:20.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.352+0000 7fe236bce740 1 -- 172.21.15.94:0/1686790087 wait complete. 2024-02-17T12:02:20.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.352+0000 7fe236bce740 20 client.26057 trim_cache size 0 max 16384 2024-02-17T12:02:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe236bce740 1 Processor -- start 2024-02-17T12:02:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe236bce740 1 -- start start 2024-02-17T12:02:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe236bce740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513440 0x561fd951c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe236bce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513d60 0x561fd9516c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe236bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd951c940 0x561fd9517170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe236bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd9400130 con 0x561fd951c940 2024-02-17T12:02:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe236bce740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd9502740 con 0x561fd9513440 2024-02-17T12:02:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe236bce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd950d450 con 0x561fd9513d60 2024-02-17T12:02:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe231d9e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513440 0x561fd951c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe231d9e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513440 0x561fd951c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46736/0 (socket says 172.21.15.94:46736) 2024-02-17T12:02:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe231d9e640 1 -- 172.21.15.94:0/730475030 learned_addr learned my addr 172.21.15.94:0/730475030 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe23159d640 1 --2- 172.21.15.94:0/730475030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513d60 0x561fd9516c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe23259f640 1 --2- 172.21.15.94:0/730475030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd951c940 0x561fd9517170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe231d9e640 1 -- 172.21.15.94:0/730475030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513d60 msgr2=0x561fd9516c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe231d9e640 1 --2- 172.21.15.94:0/730475030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513d60 0x561fd9516c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe231d9e640 1 -- 172.21.15.94:0/730475030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd951c940 msgr2=0x561fd9517170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe231d9e640 1 --2- 172.21.15.94:0/730475030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd951c940 0x561fd9517170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe231d9e640 1 -- 172.21.15.94:0/730475030 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fd9415c20 con 0x561fd9513440 2024-02-17T12:02:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.369+0000 7fe23159d640 1 --2- 172.21.15.94:0/730475030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513d60 0x561fd9516c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.370+0000 7fe231d9e640 1 --2- 172.21.15.94:0/730475030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513440 0x561fd951c0e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe22809ab50 tx=0x7fe22801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=684f68e5c9a37614 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.370+0000 7fe2177fe640 1 -- 172.21.15.94:0/730475030 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22809ae50 con 0x561fd9513440 2024-02-17T12:02:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.370+0000 7fe2177fe640 1 -- 172.21.15.94:0/730475030 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe22809efa0 con 0x561fd9513440 2024-02-17T12:02:20.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.370+0000 7fe2177fe640 1 -- 172.21.15.94:0/730475030 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22809fec0 con 0x561fd9513440 2024-02-17T12:02:20.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.371+0000 7fe236bce740 1 -- 172.21.15.94:0/730475030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513440 msgr2=0x561fd951c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.371+0000 7fe236bce740 1 --2- 172.21.15.94:0/730475030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513440 0x561fd951c0e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe22809ab50 tx=0x7fe22801c3b0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.371+0000 7fe236bce740 1 -- 172.21.15.94:0/730475030 shutdown_connections 2024-02-17T12:02:20.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.371+0000 7fe236bce740 1 --2- 172.21.15.94:0/730475030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd951c940 0x561fd9517170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.371+0000 7fe236bce740 1 --2- 172.21.15.94:0/730475030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd9513d60 0x561fd9516c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.371+0000 7fe236bce740 1 --2- 172.21.15.94:0/730475030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513440 0x561fd951c0e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.371+0000 7fe236bce740 1 -- 172.21.15.94:0/730475030 >> 172.21.15.94:0/730475030 conn(0x561fd950db70 msgr2=0x561fd9518280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.371+0000 7fe236bce740 1 -- 172.21.15.94:0/730475030 shutdown_connections 2024-02-17T12:02:20.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.371+0000 7fe236bce740 1 -- 172.21.15.94:0/730475030 wait complete. 2024-02-17T12:02:20.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.372+0000 7fe236bce740 1 -- start start 2024-02-17T12:02:20.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.372+0000 7fe236bce740 1 Processor -- start 2024-02-17T12:02:20.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.372+0000 7fe236bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513440 0x561fd9523b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe236bce740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513d60 0x561fd9521f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe236bce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd951c940 0x561fd95224b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe236bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd9522bb0 con 0x561fd9513440 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe236bce740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd9522d20 con 0x561fd9513d60 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe236bce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fd95832a0 con 0x561fd951c940 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe236bce740 10 client.0 _ll_register_callbacks cb 0x7fe2240093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe23159d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513d60 0x561fd9521f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe23159d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513d60 0x561fd9521f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46738/0 (socket says 172.21.15.94:46738) 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe231d9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513440 0x561fd9523b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe23159d640 1 -- 172.21.15.94:0/234804380 learned_addr learned my addr 172.21.15.94:0/234804380 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe23259f640 1 --2- 172.21.15.94:0/234804380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd951c940 0x561fd95224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe23159d640 1 -- 172.21.15.94:0/234804380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd951c940 msgr2=0x561fd95224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe23159d640 1 --2- 172.21.15.94:0/234804380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd951c940 0x561fd95224b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe23159d640 1 -- 172.21.15.94:0/234804380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513440 msgr2=0x561fd9523b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe23159d640 1 --2- 172.21.15.94:0/234804380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513440 0x561fd9523b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe23159d640 1 -- 172.21.15.94:0/234804380 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fd943d060 con 0x561fd9513d60 2024-02-17T12:02:20.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe23159d640 1 --2- 172.21.15.94:0/234804380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513d60 0x561fd9521f70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe21c02f020 tx=0x7fe21c02c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ab91e5bb40da0b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe2157fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:02:20.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe2157fa640 1 -- 172.21.15.94:0/234804380 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c04b040 con 0x561fd9513d60 2024-02-17T12:02:20.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.373+0000 7fe2157fa640 1 -- 172.21.15.94:0/234804380 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fe21c004ab0 con 0x561fd9513d60 2024-02-17T12:02:20.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.374+0000 7fe236bce740 1 -- 172.21.15.94:0/234804380 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe21c004020 con 0x561fd9513d60 2024-02-17T12:02:20.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.374+0000 7fe2157fa640 1 -- 172.21.15.94:0/234804380 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe21c02cbe0 con 0x561fd9513d60 2024-02-17T12:02:20.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.374+0000 7fe236bce740 10 client.26063 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:20.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.374+0000 7fe236bce740 1 -- 172.21.15.94:0/234804380 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fe22c1aab40 con 0x561fd9513d60 2024-02-17T12:02:20.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.374+0000 7fe2157fa640 1 -- 172.21.15.94:0/234804380 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe21c004020 con 0x561fd9513d60 2024-02-17T12:02:20.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.374+0000 7fe2157fa640 1 -- 172.21.15.94:0/234804380 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe21c057070 con 0x561fd9513d60 2024-02-17T12:02:20.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.375+0000 7fe236bce740 10 client.26063 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:20.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.375+0000 7fe236bce740 10 client.26063 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:02:20.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.375+0000 7fe236bce740 20 client.26063 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:20.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.375+0000 7fe236bce740 1 --2- 172.21.15.94:0/234804380 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x561fd952cd50 0x561fd9586690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.375+0000 7fe236bce740 4 client.26063 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:02:20.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.375+0000 7fe236bce740 1 -- 172.21.15.94:0/234804380 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561fd9206890 con 0x561fd952cd50 2024-02-17T12:02:20.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.375+0000 7fe231d9e640 1 --2- 172.21.15.94:0/234804380 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x561fd952cd50 0x561fd9586690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.375+0000 7fe2157fa640 10 client.26063 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:02:20.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.375+0000 7fe231d9e640 1 --2- 172.21.15.94:0/234804380 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x561fd952cd50 0x561fd9586690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9a67a2f7aa8ff90 server_cookie=fabeaec80e92b126 in_seq=0 out_seq=0 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe2157fa640 1 -- 172.21.15.94:0/234804380 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561fd9206890 con 0x561fd952cd50 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe2157fa640 10 client.26063 handle_command_reply: tid=0 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe236bce740 1 client.26063 shutdown 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe236bce740 1 -- 172.21.15.94:0/234804380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513d60 msgr2=0x561fd9521f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe236bce740 1 --2- 172.21.15.94:0/234804380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513d60 0x561fd9521f70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe21c02f020 tx=0x7fe21c02c730 comp rx=0 tx=0).stop 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe236bce740 1 -- 172.21.15.94:0/234804380 shutdown_connections 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe236bce740 1 --2- 172.21.15.94:0/234804380 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x561fd952cd50 0x561fd9586690 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe236bce740 1 --2- 172.21.15.94:0/234804380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561fd951c940 0x561fd95224b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe236bce740 1 --2- 172.21.15.94:0/234804380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561fd9513d60 0x561fd9521f70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe2157fa640 0 client.26063 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe236bce740 1 --2- 172.21.15.94:0/234804380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561fd9513440 0x561fd9523b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe236bce740 1 -- 172.21.15.94:0/234804380 >> 172.21.15.94:0/234804380 conn(0x561fd950db70 msgr2=0x561fd9519950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe236bce740 1 -- 172.21.15.94:0/234804380 shutdown_connections 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe236bce740 1 -- 172.21.15.94:0/234804380 wait complete. 2024-02-17T12:02:20.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe236bce740 20 client.26063 trim_cache size 0 max 16384 2024-02-17T12:02:20.380 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:02:20.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.376+0000 7fe232da0640 1 -- 172.21.15.94:0/3095266846 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe2180a2230 msgr2=0x7fe2180a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.377+0000 7fe232da0640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe2180a2230 0x7fe2180a46f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe224005fd0 tx=0x7fe2240139f0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.377+0000 7fe232da0640 1 -- 172.21.15.94:0/3095266846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22c0ffbc0 msgr2=0x7fe22c1a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.377+0000 7fe232da0640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22c0ffbc0 0x7fe22c1a3960 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c970 tx=0x7fe22800cd70 comp rx=0 tx=0).stop 2024-02-17T12:02:20.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.377+0000 7fe232da0640 1 -- 172.21.15.94:0/3095266846 shutdown_connections 2024-02-17T12:02:20.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.377+0000 7fe232da0640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe2180a2230 0x7fe2180a46f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.377+0000 7fe232da0640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe22c19deb0 0x7fe22c19e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.377+0000 7fe232da0640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe22c104a40 0x7fe22c1a3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.377+0000 7fe232da0640 1 --2- 172.21.15.94:0/3095266846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe22c0ffbc0 0x7fe22c1a3960 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.377+0000 7fe232da0640 1 -- 172.21.15.94:0/3095266846 >> 172.21.15.94:0/3095266846 conn(0x7fe22c0752a0 msgr2=0x7fe22c10ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.377+0000 7fe232da0640 1 -- 172.21.15.94:0/3095266846 shutdown_connections 2024-02-17T12:02:20.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.377+0000 7fe232da0640 1 -- 172.21.15.94:0/3095266846 wait complete. 2024-02-17T12:02:20.392 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:02:20.392 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:02:20.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.553+0000 7f1f4c13e640 1 -- 172.21.15.94:0/2978876707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f4410d720 msgr2=0x7f1f4410db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.553+0000 7f1f4c13e640 1 --2- 172.21.15.94:0/2978876707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f4410d720 0x7f1f4410db00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009900 tx=0x7f1f4002bc40 comp rx=0 tx=0).stop 2024-02-17T12:02:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.554+0000 7f1f4c13e640 1 -- 172.21.15.94:0/2978876707 shutdown_connections 2024-02-17T12:02:20.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.554+0000 7f1f4c13e640 1 --2- 172.21.15.94:0/2978876707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f441080d0 0x7f1f441145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.554+0000 7f1f4c13e640 1 --2- 172.21.15.94:0/2978876707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f44107710 0x7f1f44107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.554+0000 7f1f4c13e640 1 --2- 172.21.15.94:0/2978876707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f4410d720 0x7f1f4410db00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.555+0000 7f1f4c13e640 1 -- 172.21.15.94:0/2978876707 >> 172.21.15.94:0/2978876707 conn(0x7f1f44075690 msgr2=0x7f1f44077ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.555+0000 7f1f4c13e640 1 -- 172.21.15.94:0/2978876707 shutdown_connections 2024-02-17T12:02:20.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.555+0000 7f1f4c13e640 1 -- 172.21.15.94:0/2978876707 wait complete. 2024-02-17T12:02:20.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.556+0000 7f1f4c13e640 1 Processor -- start 2024-02-17T12:02:20.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.556+0000 7f1f4c13e640 1 -- start start 2024-02-17T12:02:20.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.557+0000 7f1f4c13e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44107710 0x7f1f4419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.557+0000 7f1f4c13e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f441080d0 0x7f1f4419e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.557+0000 7f1f4c13e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f4410d720 0x7f1f441a2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.557+0000 7f1f4c13e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f441a2b00 con 0x7f1f4410d720 2024-02-17T12:02:20.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.557+0000 7f1f4c13e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f441a2c70 con 0x7f1f441080d0 2024-02-17T12:02:20.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.557+0000 7f1f4c13e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f441a2e30 con 0x7f1f44107710 2024-02-17T12:02:20.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.557+0000 7f1f4a6db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f4410d720 0x7f1f441a2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.557+0000 7f1f4a6db640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f4410d720 0x7f1f441a2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41746/0 (socket says 172.21.15.94:41746) 2024-02-17T12:02:20.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.557+0000 7f1f4a6db640 1 -- 172.21.15.94:0/1961933457 learned_addr learned my addr 172.21.15.94:0/1961933457 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.558+0000 7f1f49eda640 1 --2- 172.21.15.94:0/1961933457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44107710 0x7f1f4419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.558+0000 7f1f49eda640 1 -- 172.21.15.94:0/1961933457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f441080d0 msgr2=0x7f1f4419e1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-17T12:02:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.558+0000 7f1f49eda640 1 --2- 172.21.15.94:0/1961933457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f441080d0 0x7f1f4419e1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.558+0000 7f1f49eda640 1 -- 172.21.15.94:0/1961933457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f4410d720 msgr2=0x7f1f441a2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.558+0000 7f1f49eda640 1 --2- 172.21.15.94:0/1961933457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f4410d720 0x7f1f441a2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.558+0000 7f1f49eda640 1 -- 172.21.15.94:0/1961933457 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f40009560 con 0x7f1f44107710 2024-02-17T12:02:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.559+0000 7f1f49eda640 1 --2- 172.21.15.94:0/1961933457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44107710 0x7f1f4419dc60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f40036040 tx=0x7f1f400042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.559+0000 7f1f3affd640 1 -- 172.21.15.94:0/1961933457 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f400044d0 con 0x7f1f44107710 2024-02-17T12:02:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.559+0000 7f1f4c13e640 1 -- 172.21.15.94:0/1961933457 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f441a30b0 con 0x7f1f44107710 2024-02-17T12:02:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.559+0000 7f1f4c13e640 1 -- 172.21.15.94:0/1961933457 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f441aa990 con 0x7f1f44107710 2024-02-17T12:02:20.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.560+0000 7f1f3affd640 1 -- 172.21.15.94:0/1961933457 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f400054e0 con 0x7f1f44107710 2024-02-17T12:02:20.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.560+0000 7f1f3affd640 1 -- 172.21.15.94:0/1961933457 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f4003e450 con 0x7f1f44107710 2024-02-17T12:02:20.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.563+0000 7f1f4eded740 1 Processor -- start 2024-02-17T12:02:20.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.563+0000 7f1f4eded740 1 -- start start 2024-02-17T12:02:20.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4eded740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4250 0x5563e4dd4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4eded740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4dd4b70 0x5563e4ddd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4eded740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4ddd750 0x5563e4ddfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4eded740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e4d8f960 con 0x5563e4ddd750 2024-02-17T12:02:20.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4eded740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e4d43310 con 0x5563e4dd4250 2024-02-17T12:02:20.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4eded740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e4d40840 con 0x5563e4dd4b70 2024-02-17T12:02:20.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4a6db640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4250 0x5563e4dd4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4a6db640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4250 0x5563e4dd4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46760/0 (socket says 172.21.15.94:46760) 2024-02-17T12:02:20.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4a6db640 1 -- 172.21.15.94:0/1919262508 learned_addr learned my addr 172.21.15.94:0/1919262508 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4a6db640 1 -- 172.21.15.94:0/1919262508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4dd4b70 msgr2=0x5563e4ddd210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:02:20.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4a6db640 1 --2- 172.21.15.94:0/1919262508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4dd4b70 0x5563e4ddd210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4a6db640 1 -- 172.21.15.94:0/1919262508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4ddd750 msgr2=0x5563e4ddfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4a6db640 1 --2- 172.21.15.94:0/1919262508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4ddd750 0x5563e4ddfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.564+0000 7f1f4a6db640 1 -- 172.21.15.94:0/1919262508 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563e4cfe060 con 0x5563e4dd4250 2024-02-17T12:02:20.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.565+0000 7f1f4a6db640 1 --2- 172.21.15.94:0/1919262508 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4250 0x5563e4dd4630 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c007ef0 tx=0x7f1f3c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=128f5ca239d8432a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.565+0000 7f1f2ffff640 1 -- 172.21.15.94:0/1919262508 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c010840 con 0x5563e4dd4250 2024-02-17T12:02:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.565+0000 7f1f2ffff640 1 -- 172.21.15.94:0/1919262508 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f3c010e80 con 0x5563e4dd4250 2024-02-17T12:02:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.565+0000 7f1f2ffff640 1 -- 172.21.15.94:0/1919262508 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c0173a0 con 0x5563e4dd4250 2024-02-17T12:02:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.566+0000 7f1f3affd640 1 -- 172.21.15.94:0/1961933457 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1f4003e5d0 con 0x7f1f44107710 2024-02-17T12:02:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.566+0000 7f1f4eded740 1 -- 172.21.15.94:0/1919262508 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4250 msgr2=0x5563e4dd4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.566+0000 7f1f4eded740 1 --2- 172.21.15.94:0/1919262508 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4250 0x5563e4dd4630 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c007ef0 tx=0x7f1f3c00f790 comp rx=0 tx=0).stop 2024-02-17T12:02:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.566+0000 7f1f4eded740 1 -- 172.21.15.94:0/1919262508 shutdown_connections 2024-02-17T12:02:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.566+0000 7f1f4eded740 1 --2- 172.21.15.94:0/1919262508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4ddd750 0x5563e4ddfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.566+0000 7f1f4eded740 1 --2- 172.21.15.94:0/1919262508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4dd4b70 0x5563e4ddd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.566+0000 7f1f4eded740 1 --2- 172.21.15.94:0/1919262508 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4250 0x5563e4dd4630 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.567+0000 7f1f4eded740 1 -- 172.21.15.94:0/1919262508 >> 172.21.15.94:0/1919262508 conn(0x5563e4dce980 msgr2=0x5563e4dced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.567+0000 7f1f4eded740 1 -- 172.21.15.94:0/1919262508 shutdown_connections 2024-02-17T12:02:20.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.567+0000 7f1f4eded740 1 -- 172.21.15.94:0/1919262508 wait complete. 2024-02-17T12:02:20.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.567+0000 7f1f3affd640 1 --2- 172.21.15.94:0/1961933457 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1f300a22d0 0x7f1f300a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.567+0000 7f1f4eded740 1 -- start start 2024-02-17T12:02:20.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.567+0000 7f1f3affd640 1 -- 172.21.15.94:0/1961933457 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1f400f0ad0 con 0x7f1f44107710 2024-02-17T12:02:20.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.568+0000 7f1f4eded740 1 Processor -- start 2024-02-17T12:02:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.568+0000 7f1f496d9640 1 --2- 172.21.15.94:0/1961933457 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1f300a22d0 0x7f1f300a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.568+0000 7f1f496d9640 1 --2- 172.21.15.94:0/1961933457 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1f300a22d0 0x7f1f300a4790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x5563e4dcfb10 tx=0x7f1f34005ad0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f4eded740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4dd4250 0x5563e4dda790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f4eded740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 0x5563e4ddacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f4eded740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4ddd750 0x5563e4ddb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f496d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 0x5563e4ddacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f496d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 0x5563e4ddacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41774/0 (socket says 172.21.15.94:41774) 2024-02-17T12:02:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f49eda640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4dd4250 0x5563e4dda790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f49eda640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4dd4250 0x5563e4dda790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55666/0 (socket says 172.21.15.94:55666) 2024-02-17T12:02:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f496d9640 1 -- 172.21.15.94:0/478839231 learned_addr learned my addr 172.21.15.94:0/478839231 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f4eded740 1 -- 172.21.15.94:0/478839231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e4d3c020 con 0x5563e4dd4b70 2024-02-17T12:02:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f4eded740 1 -- 172.21.15.94:0/478839231 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e4a0f430 con 0x5563e4ddd750 2024-02-17T12:02:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f4eded740 1 -- 172.21.15.94:0/478839231 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e49cbe00 con 0x5563e4dd4250 2024-02-17T12:02:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f4eded740 10 client.0 _ll_register_callbacks cb 0x5563e4ce3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f4a6db640 1 --2- 172.21.15.94:0/478839231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4ddd750 0x5563e4ddb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f496d9640 1 -- 172.21.15.94:0/478839231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4dd4250 msgr2=0x5563e4dda790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f496d9640 1 --2- 172.21.15.94:0/478839231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4dd4250 0x5563e4dda790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f496d9640 1 -- 172.21.15.94:0/478839231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4ddd750 msgr2=0x5563e4ddb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f496d9640 1 --2- 172.21.15.94:0/478839231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4ddd750 0x5563e4ddb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f496d9640 1 -- 172.21.15.94:0/478839231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f3c007b50 con 0x5563e4dd4b70 2024-02-17T12:02:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f496d9640 1 --2- 172.21.15.94:0/478839231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 0x5563e4ddacd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400f480 tx=0x7f1f340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a9edea77df8639 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.569+0000 7f1f2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:02:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.570+0000 7f1f2d7fa640 1 -- 172.21.15.94:0/478839231 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3401c800 con 0x5563e4dd4b70 2024-02-17T12:02:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.570+0000 7f1f4eded740 1 -- 172.21.15.94:0/478839231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563e4a78720 con 0x5563e4dd4b70 2024-02-17T12:02:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.570+0000 7f1f2d7fa640 1 -- 172.21.15.94:0/478839231 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1f34017040 con 0x5563e4dd4b70 2024-02-17T12:02:20.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.570+0000 7f1f2d7fa640 1 -- 172.21.15.94:0/478839231 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f3401c960 con 0x5563e4dd4b70 2024-02-17T12:02:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.570+0000 7f1f4eded740 10 client.16614 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.570+0000 7f1f4eded740 1 -- 172.21.15.94:0/478839231 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5563e4d3a6a0 con 0x5563e4dd4b70 2024-02-17T12:02:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.571+0000 7f1f2d7fa640 1 -- 172.21.15.94:0/478839231 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f340243b0 con 0x5563e4dd4b70 2024-02-17T12:02:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.571+0000 7f1f2d7fa640 1 -- 172.21.15.94:0/478839231 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1f3400e3b0 con 0x5563e4dd4b70 2024-02-17T12:02:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.571+0000 7f1f4eded740 10 client.16614 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.571+0000 7f1f4eded740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.571+0000 7f1f4eded740 20 client.16614 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.571+0000 7f1f4eded740 1 --2- 172.21.15.94:0/478839231 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5563e4ded9e0 0x5563e4defe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.571+0000 7f1f4eded740 4 client.16614 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.571+0000 7f1f4eded740 1 -- 172.21.15.94:0/478839231 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563e4a78720 con 0x5563e4ded9e0 2024-02-17T12:02:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.572+0000 7f1f49eda640 1 --2- 172.21.15.94:0/478839231 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5563e4ded9e0 0x5563e4defe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.573+0000 7f1f2d7fa640 10 client.16614 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.573+0000 7f1f49eda640 1 --2- 172.21.15.94:0/478839231 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5563e4ded9e0 0x5563e4defe00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c90b0b8c441d592d server_cookie=2a27c24307e71ee8 in_seq=0 out_seq=0 2024-02-17T12:02:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.573+0000 7f1f2d7fa640 1 -- 172.21.15.94:0/478839231 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5563e4a78720 con 0x5563e4ded9e0 2024-02-17T12:02:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.573+0000 7f1f2d7fa640 10 client.16614 handle_command_reply: tid=0 2024-02-17T12:02:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.573+0000 7f1f4eded740 1 client.16614 shutdown 2024-02-17T12:02:20.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.573+0000 7f1f4eded740 1 -- 172.21.15.94:0/478839231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 msgr2=0x5563e4ddacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.573+0000 7f1f4eded740 1 --2- 172.21.15.94:0/478839231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 0x5563e4ddacd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1f3400f480 tx=0x7f1f340046c0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.574+0000 7f1f4eded740 1 -- 172.21.15.94:0/478839231 shutdown_connections 2024-02-17T12:02:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.574+0000 7f1f4eded740 1 --2- 172.21.15.94:0/478839231 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5563e4ded9e0 0x5563e4defe00 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.574+0000 7f1f4eded740 1 --2- 172.21.15.94:0/478839231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4ddd750 0x5563e4ddb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.574+0000 7f1f2d7fa640 0 client.16614 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.574+0000 7f1f4eded740 1 --2- 172.21.15.94:0/478839231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 0x5563e4ddacd0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.574+0000 7f1f4eded740 1 --2- 172.21.15.94:0/478839231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4dd4250 0x5563e4dda790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.574+0000 7f1f4eded740 1 -- 172.21.15.94:0/478839231 >> 172.21.15.94:0/478839231 conn(0x5563e4dce980 msgr2=0x5563e4dd06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.574+0000 7f1f4eded740 1 -- 172.21.15.94:0/478839231 shutdown_connections 2024-02-17T12:02:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.574+0000 7f1f4eded740 1 -- 172.21.15.94:0/478839231 wait complete. 2024-02-17T12:02:20.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.574+0000 7f1f4eded740 20 client.16614 trim_cache size 0 max 16384 2024-02-17T12:02:20.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.591+0000 7f1f4eded740 1 Processor -- start 2024-02-17T12:02:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.591+0000 7f1f4eded740 1 -- start start 2024-02-17T12:02:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.591+0000 7f1f4eded740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4250 0x5563e4ddb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.591+0000 7f1f4eded740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 0x5563e4dd9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.591+0000 7f1f4eded740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4ddd750 0x5563e4dd95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.591+0000 7f1f4eded740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e4cc1130 con 0x5563e4dd4b70 2024-02-17T12:02:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.591+0000 7f1f4eded740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e4dec250 con 0x5563e4dd4250 2024-02-17T12:02:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.591+0000 7f1f4eded740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e4dd63e0 con 0x5563e4ddd750 2024-02-17T12:02:20.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.592+0000 7f1f496d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 0x5563e4dd9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.592+0000 7f1f4a6db640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4ddd750 0x5563e4dd95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.592+0000 7f1f496d9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 0x5563e4dd9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41784/0 (socket says 172.21.15.94:41784) 2024-02-17T12:02:20.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.592+0000 7f1f496d9640 1 -- 172.21.15.94:0/3572922350 learned_addr learned my addr 172.21.15.94:0/3572922350 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.593+0000 7f1f496d9640 1 -- 172.21.15.94:0/3572922350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4ddd750 msgr2=0x5563e4dd95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.593+0000 7f1f496d9640 1 --2- 172.21.15.94:0/3572922350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4ddd750 0x5563e4dd95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.593+0000 7f1f496d9640 1 -- 172.21.15.94:0/3572922350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4250 msgr2=0x5563e4ddb030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:02:20.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.593+0000 7f1f496d9640 1 --2- 172.21.15.94:0/3572922350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4250 0x5563e4ddb030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.593+0000 7f1f496d9640 1 -- 172.21.15.94:0/3572922350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563e4cd6c20 con 0x5563e4dd4b70 2024-02-17T12:02:20.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.593+0000 7f1f496d9640 1 --2- 172.21.15.94:0/3572922350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 0x5563e4dd9060 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1f34011040 tx=0x7f1f3400a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad91d331cb48b0d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.594+0000 7f1f2f7fe640 1 -- 172.21.15.94:0/3572922350 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400e950 con 0x5563e4dd4b70 2024-02-17T12:02:20.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.594+0000 7f1f2f7fe640 1 -- 172.21.15.94:0/3572922350 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f34005040 con 0x5563e4dd4b70 2024-02-17T12:02:20.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.594+0000 7f1f2f7fe640 1 -- 172.21.15.94:0/3572922350 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34004970 con 0x5563e4dd4b70 2024-02-17T12:02:20.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.595+0000 7f1f4eded740 1 -- 172.21.15.94:0/3572922350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 msgr2=0x5563e4dd9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.595+0000 7f1f4eded740 1 --2- 172.21.15.94:0/3572922350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 0x5563e4dd9060 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1f34011040 tx=0x7f1f3400a450 comp rx=0 tx=0).stop 2024-02-17T12:02:20.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.595+0000 7f1f4eded740 1 -- 172.21.15.94:0/3572922350 shutdown_connections 2024-02-17T12:02:20.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.595+0000 7f1f4eded740 1 --2- 172.21.15.94:0/3572922350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4ddd750 0x5563e4dd95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.595+0000 7f1f4eded740 1 --2- 172.21.15.94:0/3572922350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4b70 0x5563e4dd9060 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.595+0000 7f1f4eded740 1 --2- 172.21.15.94:0/3572922350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4250 0x5563e4ddb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.595+0000 7f1f4eded740 1 -- 172.21.15.94:0/3572922350 >> 172.21.15.94:0/3572922350 conn(0x5563e4dce980 msgr2=0x5563e4dc9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.595+0000 7f1f4eded740 1 -- 172.21.15.94:0/3572922350 shutdown_connections 2024-02-17T12:02:20.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.595+0000 7f1f4eded740 1 -- 172.21.15.94:0/3572922350 wait complete. 2024-02-17T12:02:20.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.595+0000 7f1f4eded740 1 -- start start 2024-02-17T12:02:20.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.595+0000 7f1f4eded740 1 Processor -- start 2024-02-17T12:02:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f4eded740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4250 0x5563e4de4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f4eded740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4b70 0x5563e4de2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f4eded740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4ddd750 0x5563e4de31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f4eded740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e4de36f0 con 0x5563e4dd4250 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f4eded740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e4de3860 con 0x5563e4dd4b70 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f4eded740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563e4de3a20 con 0x5563e4ddd750 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f4eded740 10 client.0 _ll_register_callbacks cb 0x7f1f3c007920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f49eda640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4250 0x5563e4de4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f49eda640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4250 0x5563e4de4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41786/0 (socket says 172.21.15.94:41786) 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f49eda640 1 -- 172.21.15.94:0/2539977753 learned_addr learned my addr 172.21.15.94:0/2539977753 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f496d9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4b70 0x5563e4de2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f4a6db640 1 --2- 172.21.15.94:0/2539977753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4ddd750 0x5563e4de31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f49eda640 1 -- 172.21.15.94:0/2539977753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4ddd750 msgr2=0x5563e4de31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f49eda640 1 --2- 172.21.15.94:0/2539977753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4ddd750 0x5563e4de31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f49eda640 1 -- 172.21.15.94:0/2539977753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4b70 msgr2=0x5563e4de2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f49eda640 1 --2- 172.21.15.94:0/2539977753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4b70 0x5563e4de2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f49eda640 1 -- 172.21.15.94:0/2539977753 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563e4d3a6a0 con 0x5563e4dd4250 2024-02-17T12:02:20.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.596+0000 7f1f496d9640 1 --2- 172.21.15.94:0/2539977753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4b70 0x5563e4de2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:20.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.597+0000 7f1f49eda640 1 --2- 172.21.15.94:0/2539977753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4250 0x5563e4de4810 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009900 tx=0x7f1f400b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e5b1f13e76178d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:20.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.597+0000 7f1f2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:02:20.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.597+0000 7f1f2d7fa640 1 -- 172.21.15.94:0/2539977753 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f400d5070 con 0x5563e4dd4250 2024-02-17T12:02:20.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.597+0000 7f1f4eded740 1 -- 172.21.15.94:0/2539977753 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563e4a3b7e0 con 0x5563e4dd4250 2024-02-17T12:02:20.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.597+0000 7f1f2d7fa640 1 -- 172.21.15.94:0/2539977753 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1f400bd560 con 0x5563e4dd4250 2024-02-17T12:02:20.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.597+0000 7f1f2d7fa640 1 -- 172.21.15.94:0/2539977753 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f400c16b0 con 0x5563e4dd4250 2024-02-17T12:02:20.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.597+0000 7f1f4eded740 10 client.16620 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:20.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.597+0000 7f1f4eded740 1 -- 172.21.15.94:0/2539977753 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5563e4cfc8c0 con 0x5563e4dd4250 2024-02-17T12:02:20.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.598+0000 7f1f2d7fa640 1 -- 172.21.15.94:0/2539977753 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5563e4a3b7e0 con 0x5563e4dd4250 2024-02-17T12:02:20.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.598+0000 7f1f2d7fa640 1 -- 172.21.15.94:0/2539977753 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1f4003c070 con 0x5563e4dd4250 2024-02-17T12:02:20.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.599+0000 7f1f4eded740 10 client.16620 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:20.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.599+0000 7f1f4eded740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:20.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.599+0000 7f1f4eded740 20 client.16620 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.599+0000 7f1f4eded740 1 --2- 172.21.15.94:0/2539977753 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5563e4ded9e0 0x5563e4e47240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.599+0000 7f1f4eded740 4 client.16620 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.599+0000 7f1f4eded740 1 -- 172.21.15.94:0/2539977753 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5563e4ac7890 con 0x5563e4ded9e0 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.599+0000 7f1f496d9640 1 --2- 172.21.15.94:0/2539977753 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5563e4ded9e0 0x5563e4e47240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.599+0000 7f1f2d7fa640 10 client.16620 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.600+0000 7f1f496d9640 1 --2- 172.21.15.94:0/2539977753 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5563e4ded9e0 0x5563e4e47240 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b05b7f374c8eeea2 server_cookie=b0afab167756ba6e in_seq=0 out_seq=0 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.600+0000 7f1f2d7fa640 1 -- 172.21.15.94:0/2539977753 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5563e4ac7890 con 0x5563e4ded9e0 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.600+0000 7f1f2d7fa640 10 client.16620 handle_command_reply: tid=0 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.600+0000 7f1f4eded740 1 client.16620 shutdown 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.601+0000 7f1f4eded740 1 -- 172.21.15.94:0/2539977753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4250 msgr2=0x5563e4de4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.601+0000 7f1f4eded740 1 --2- 172.21.15.94:0/2539977753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4250 0x5563e4de4810 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1f40009900 tx=0x7f1f400b9040 comp rx=0 tx=0).stop 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.601+0000 7f1f4eded740 1 -- 172.21.15.94:0/2539977753 shutdown_connections 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.601+0000 7f1f4eded740 1 --2- 172.21.15.94:0/2539977753 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5563e4ded9e0 0x5563e4e47240 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.601+0000 7f1f4eded740 1 --2- 172.21.15.94:0/2539977753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5563e4ddd750 0x5563e4de31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.601+0000 7f1f4eded740 1 --2- 172.21.15.94:0/2539977753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5563e4dd4b70 0x5563e4de2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.601+0000 7f1f4eded740 1 --2- 172.21.15.94:0/2539977753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5563e4dd4250 0x5563e4de4810 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.601+0000 7f1f4eded740 1 -- 172.21.15.94:0/2539977753 >> 172.21.15.94:0/2539977753 conn(0x5563e4dce980 msgr2=0x5563e4dd9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.601+0000 7f1f2d7fa640 0 client.16620 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:20.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.601+0000 7f1f4eded740 1 -- 172.21.15.94:0/2539977753 shutdown_connections 2024-02-17T12:02:20.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.601+0000 7f1f4eded740 1 -- 172.21.15.94:0/2539977753 wait complete. 2024-02-17T12:02:20.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.601+0000 7f1f4eded740 20 client.16620 trim_cache size 0 max 16384 2024-02-17T12:02:20.606 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:02:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.602+0000 7f1f4c13e640 1 -- 172.21.15.94:0/1961933457 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1f300a22d0 msgr2=0x7f1f300a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.602+0000 7f1f4c13e640 1 --2- 172.21.15.94:0/1961933457 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1f300a22d0 0x7f1f300a4790 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x5563e4dcfb10 tx=0x7f1f34005ad0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.602+0000 7f1f4c13e640 1 -- 172.21.15.94:0/1961933457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44107710 msgr2=0x7f1f4419dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.602+0000 7f1f4c13e640 1 --2- 172.21.15.94:0/1961933457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44107710 0x7f1f4419dc60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f40036040 tx=0x7f1f400042e0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.602+0000 7f1f4c13e640 1 -- 172.21.15.94:0/1961933457 shutdown_connections 2024-02-17T12:02:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.602+0000 7f1f4c13e640 1 --2- 172.21.15.94:0/1961933457 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1f300a22d0 0x7f1f300a4790 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.602+0000 7f1f4c13e640 1 --2- 172.21.15.94:0/1961933457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1f4410d720 0x7f1f441a2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.602+0000 7f1f4c13e640 1 --2- 172.21.15.94:0/1961933457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1f441080d0 0x7f1f4419e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.602+0000 7f1f4c13e640 1 --2- 172.21.15.94:0/1961933457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f44107710 0x7f1f4419dc60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.602+0000 7f1f4c13e640 1 -- 172.21.15.94:0/1961933457 >> 172.21.15.94:0/1961933457 conn(0x7f1f44075690 msgr2=0x7f1f44075b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.602+0000 7f1f4c13e640 1 -- 172.21.15.94:0/1961933457 shutdown_connections 2024-02-17T12:02:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:20.602+0000 7f1f4c13e640 1 -- 172.21.15.94:0/1961933457 wait complete. 2024-02-17T12:02:21.617 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:02:21.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:02:21.617 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:02:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.677+0000 7eff03b5c640 1 -- 172.21.15.94:0/2676941220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efefc107770 msgr2=0x7efefc107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.677+0000 7eff03b5c640 1 --2- 172.21.15.94:0/2676941220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efefc107770 0x7efefc107bf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efef8009900 tx=0x7efef802bc30 comp rx=0 tx=0).stop 2024-02-17T12:02:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.677+0000 7eff03b5c640 1 -- 172.21.15.94:0/2676941220 shutdown_connections 2024-02-17T12:02:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.677+0000 7eff03b5c640 1 --2- 172.21.15.94:0/2676941220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efefc108130 0x7efefc114640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.677+0000 7eff03b5c640 1 --2- 172.21.15.94:0/2676941220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efefc107770 0x7efefc107bf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.677+0000 7eff03b5c640 1 --2- 172.21.15.94:0/2676941220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc10d780 0x7efefc10db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.677+0000 7eff03b5c640 1 -- 172.21.15.94:0/2676941220 >> 172.21.15.94:0/2676941220 conn(0x7efefc0756f0 msgr2=0x7efefc077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:21.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.677+0000 7eff03b5c640 1 -- 172.21.15.94:0/2676941220 shutdown_connections 2024-02-17T12:02:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.677+0000 7eff03b5c640 1 -- 172.21.15.94:0/2676941220 wait complete. 2024-02-17T12:02:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.678+0000 7eff03b5c640 1 Processor -- start 2024-02-17T12:02:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.678+0000 7eff03b5c640 1 -- start start 2024-02-17T12:02:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.678+0000 7eff03b5c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efefc107770 0x7efefc19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.678+0000 7eff03b5c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc108130 0x7efefc19e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.678+0000 7eff03b5c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efefc10d780 0x7efefc1a2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.678+0000 7eff03b5c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc1a2c10 con 0x7efefc108130 2024-02-17T12:02:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.678+0000 7eff03b5c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc1a2d50 con 0x7efefc107770 2024-02-17T12:02:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.678+0000 7eff03b5c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc1a2f10 con 0x7efefc10d780 2024-02-17T12:02:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.678+0000 7eff020f9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efefc10d780 0x7efefc1a2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.678+0000 7eff020f9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efefc10d780 0x7efefc1a2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55708/0 (socket says 172.21.15.94:55708) 2024-02-17T12:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.679+0000 7eff020f9640 1 -- 172.21.15.94:0/1642376727 learned_addr learned my addr 172.21.15.94:0/1642376727 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.679+0000 7eff018f8640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efefc107770 0x7efefc19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.679+0000 7eff010f7640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc108130 0x7efefc19e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.679+0000 7eff020f9640 1 -- 172.21.15.94:0/1642376727 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efefc107770 msgr2=0x7efefc19dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.679+0000 7eff020f9640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efefc107770 0x7efefc19dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.679+0000 7eff020f9640 1 -- 172.21.15.94:0/1642376727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc108130 msgr2=0x7efefc19e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.679+0000 7eff020f9640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc108130 0x7efefc19e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.679+0000 7eff020f9640 1 -- 172.21.15.94:0/1642376727 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef8009560 con 0x7efefc10d780 2024-02-17T12:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.679+0000 7eff010f7640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc108130 0x7efefc19e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.679+0000 7eff020f9640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efefc10d780 0x7efefc1a2610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efeec00e9a0 tx=0x7efeec00eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.680+0000 7efef2ffd640 1 -- 172.21.15.94:0/1642376727 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec00ccc0 con 0x7efefc10d780 2024-02-17T12:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.680+0000 7eff03b5c640 1 -- 172.21.15.94:0/1642376727 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efefc1a31f0 con 0x7efefc10d780 2024-02-17T12:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.680+0000 7eff03b5c640 1 -- 172.21.15.94:0/1642376727 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc079bb0 con 0x7efefc10d780 2024-02-17T12:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.680+0000 7efef2ffd640 1 -- 172.21.15.94:0/1642376727 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efeec00ce20 con 0x7efefc10d780 2024-02-17T12:02:21.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.680+0000 7efef2ffd640 1 -- 172.21.15.94:0/1642376727 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec010690 con 0x7efefc10d780 2024-02-17T12:02:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.686+0000 7eff06805740 1 Processor -- start 2024-02-17T12:02:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.686+0000 7efef2ffd640 1 -- 172.21.15.94:0/1642376727 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7efeec004ea0 con 0x7efefc10d780 2024-02-17T12:02:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.687+0000 7eff06805740 1 -- start start 2024-02-17T12:02:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.687+0000 7eff06805740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e031ad0 0x563b5e030820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.687+0000 7eff06805740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e030d60 0x563b5e0311e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.687+0000 7eff06805740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e03a2e0 0x563b5e03c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.687+0000 7eff06805740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5df9f9f0 con 0x563b5e03a2e0 2024-02-17T12:02:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.687+0000 7eff06805740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5df9cf20 con 0x563b5e031ad0 2024-02-17T12:02:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.687+0000 7eff06805740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5df98700 con 0x563b5e030d60 2024-02-17T12:02:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.688+0000 7eff018f8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e030d60 0x563b5e0311e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.688+0000 7eff018f8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e030d60 0x563b5e0311e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55714/0 (socket says 172.21.15.94:55714) 2024-02-17T12:02:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.688+0000 7eff018f8640 1 -- 172.21.15.94:0/2540573671 learned_addr learned my addr 172.21.15.94:0/2540573671 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.688+0000 7eff010f7640 1 --2- 172.21.15.94:0/2540573671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e031ad0 0x563b5e030820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.688+0000 7eff010f7640 1 --2- 172.21.15.94:0/2540573671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e03a2e0 0x563b5e03c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.688+0000 7eff018f8640 1 -- 172.21.15.94:0/2540573671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e031ad0 msgr2=0x563b5e030820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.688+0000 7eff018f8640 1 --2- 172.21.15.94:0/2540573671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e031ad0 0x563b5e030820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.688+0000 7eff018f8640 1 -- 172.21.15.94:0/2540573671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e03a2e0 msgr2=0x563b5e03c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.688+0000 7eff018f8640 1 --2- 172.21.15.94:0/2540573671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e03a2e0 0x563b5e03c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.688+0000 7eff018f8640 1 -- 172.21.15.94:0/2540573671 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b5df2e7b0 con 0x563b5e030d60 2024-02-17T12:02:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.688+0000 7eff010f7640 1 --2- 172.21.15.94:0/2540573671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e031ad0 0x563b5e030820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:02:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.689+0000 7eff018f8640 1 --2- 172.21.15.94:0/2540573671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e030d60 0x563b5e0311e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efef4009fd0 tx=0x7efef40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26d09beff8198e7d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.689+0000 7efef2ffd640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efee80a2480 0x7efee80a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.689+0000 7efee7fff640 1 -- 172.21.15.94:0/2540573671 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4003c30 con 0x563b5e030d60 2024-02-17T12:02:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.689+0000 7efee7fff640 1 -- 172.21.15.94:0/2540573671 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efef4003d90 con 0x563b5e030d60 2024-02-17T12:02:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.689+0000 7efee7fff640 1 -- 172.21.15.94:0/2540573671 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef400fb50 con 0x563b5e030d60 2024-02-17T12:02:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.689+0000 7efef2ffd640 1 -- 172.21.15.94:0/1642376727 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7efeec014070 con 0x7efefc10d780 2024-02-17T12:02:21.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.690+0000 7eff010f7640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efee80a2480 0x7efee80a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.691+0000 7eff06805740 1 -- 172.21.15.94:0/2540573671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e030d60 msgr2=0x563b5e0311e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.691+0000 7eff06805740 1 --2- 172.21.15.94:0/2540573671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e030d60 0x563b5e0311e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efef4009fd0 tx=0x7efef40118d0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.691+0000 7eff06805740 1 -- 172.21.15.94:0/2540573671 shutdown_connections 2024-02-17T12:02:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.691+0000 7eff06805740 1 --2- 172.21.15.94:0/2540573671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e03a2e0 0x563b5e03c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.691+0000 7eff06805740 1 --2- 172.21.15.94:0/2540573671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e030d60 0x563b5e0311e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.691+0000 7eff06805740 1 --2- 172.21.15.94:0/2540573671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e031ad0 0x563b5e030820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.691+0000 7eff06805740 1 -- 172.21.15.94:0/2540573671 >> 172.21.15.94:0/2540573671 conn(0x563b5dff8550 msgr2=0x563b5e02ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.692+0000 7eff06805740 1 -- 172.21.15.94:0/2540573671 shutdown_connections 2024-02-17T12:02:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.692+0000 7eff06805740 1 -- 172.21.15.94:0/2540573671 wait complete. 2024-02-17T12:02:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.693+0000 7eff06805740 1 -- start start 2024-02-17T12:02:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.693+0000 7eff06805740 1 Processor -- start 2024-02-17T12:02:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.694+0000 7eff010f7640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efee80a2480 0x7efee80a4940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x563b5e031800 tx=0x7efef8005fd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.695+0000 7eff06805740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e030d60 0x563b5e037290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.695+0000 7eff06805740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e031ad0 0x563b5e0377d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.695+0000 7eff06805740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e03a2e0 0x563b5e037d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.695+0000 7eff06805740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5df3dbf0 con 0x563b5e031ad0 2024-02-17T12:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.695+0000 7eff018f8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e030d60 0x563b5e037290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.695+0000 7eff020f9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e03a2e0 0x563b5e037d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.695+0000 7eff018f8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e030d60 0x563b5e037290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46832/0 (socket says 172.21.15.94:46832) 2024-02-17T12:02:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.695+0000 7eff018f8640 1 -- 172.21.15.94:0/1508696197 learned_addr learned my addr 172.21.15.94:0/1508696197 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.695+0000 7eff06805740 1 -- 172.21.15.94:0/1508696197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5dc6c250 con 0x563b5e030d60 2024-02-17T12:02:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.695+0000 7eff06805740 1 -- 172.21.15.94:0/1508696197 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5df1e200 con 0x563b5e03a2e0 2024-02-17T12:02:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.695+0000 7eff06805740 10 client.0 _ll_register_callbacks cb 0x563b5df8cd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.695+0000 7eff010f7640 1 --2- 172.21.15.94:0/1508696197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e031ad0 0x563b5e0377d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.696+0000 7eff018f8640 1 -- 172.21.15.94:0/1508696197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e03a2e0 msgr2=0x563b5e037d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.696+0000 7eff018f8640 1 --2- 172.21.15.94:0/1508696197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e03a2e0 0x563b5e037d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.696+0000 7eff018f8640 1 -- 172.21.15.94:0/1508696197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e031ad0 msgr2=0x563b5e0377d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.696+0000 7eff018f8640 1 --2- 172.21.15.94:0/1508696197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e031ad0 0x563b5e0377d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.696+0000 7eff018f8640 1 -- 172.21.15.94:0/1508696197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef4009c00 con 0x563b5e030d60 2024-02-17T12:02:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.696+0000 7eff010f7640 1 --2- 172.21.15.94:0/1508696197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e031ad0 0x563b5e0377d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:02:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.697+0000 7eff018f8640 1 --2- 172.21.15.94:0/1508696197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e030d60 0x563b5e037290 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efef4004f70 tx=0x7efef4009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e764cf1b7922aff0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.697+0000 7efee57fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:02:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.697+0000 7efee57fa640 1 -- 172.21.15.94:0/1508696197 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef400fbe0 con 0x563b5e030d60 2024-02-17T12:02:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.697+0000 7efee57fa640 1 -- 172.21.15.94:0/1508696197 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7efef400fd40 con 0x563b5e030d60 2024-02-17T12:02:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.697+0000 7eff06805740 1 -- 172.21.15.94:0/1508696197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b5dc98600 con 0x563b5e030d60 2024-02-17T12:02:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.697+0000 7efee57fa640 1 -- 172.21.15.94:0/1508696197 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efef4004050 con 0x563b5e030d60 2024-02-17T12:02:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.697+0000 7eff06805740 10 client.26087 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.697+0000 7eff06805740 1 -- 172.21.15.94:0/1508696197 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563b5df96d80 con 0x563b5e030d60 2024-02-17T12:02:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.698+0000 7efee57fa640 1 -- 172.21.15.94:0/1508696197 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563b5dc98600 con 0x563b5e030d60 2024-02-17T12:02:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.698+0000 7efee57fa640 1 -- 172.21.15.94:0/1508696197 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efef40055c0 con 0x563b5e030d60 2024-02-17T12:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.698+0000 7eff06805740 10 client.26087 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.699+0000 7eff06805740 10 client.26087 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.699+0000 7eff06805740 20 client.26087 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.699+0000 7eff06805740 1 --2- 172.21.15.94:0/1508696197 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563b5e04a3d0 0x563b5e04c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.699+0000 7eff06805740 4 client.26087 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.699+0000 7eff06805740 1 -- 172.21.15.94:0/1508696197 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b5dce91b0 con 0x563b5e04a3d0 2024-02-17T12:02:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.699+0000 7eff010f7640 1 --2- 172.21.15.94:0/1508696197 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563b5e04a3d0 0x563b5e04c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.700+0000 7eff010f7640 1 --2- 172.21.15.94:0/1508696197 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563b5e04a3d0 0x563b5e04c7f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cdc96b238e24cf3 server_cookie=d150a182c3f82010 in_seq=0 out_seq=0 2024-02-17T12:02:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.700+0000 7efee57fa640 10 client.26087 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.700+0000 7efee57fa640 1 -- 172.21.15.94:0/1508696197 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b5dce91b0 con 0x563b5e04a3d0 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.700+0000 7efee57fa640 10 client.26087 handle_command_reply: tid=0 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.701+0000 7eff06805740 1 client.26087 shutdown 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.701+0000 7eff06805740 1 -- 172.21.15.94:0/1508696197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e030d60 msgr2=0x563b5e037290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.701+0000 7eff06805740 1 --2- 172.21.15.94:0/1508696197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e030d60 0x563b5e037290 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efef4004f70 tx=0x7efef4009300 comp rx=0 tx=0).stop 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.701+0000 7eff06805740 1 -- 172.21.15.94:0/1508696197 shutdown_connections 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.701+0000 7eff06805740 1 --2- 172.21.15.94:0/1508696197 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563b5e04a3d0 0x563b5e04c7f0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.701+0000 7eff06805740 1 --2- 172.21.15.94:0/1508696197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e03a2e0 0x563b5e037d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.701+0000 7eff06805740 1 --2- 172.21.15.94:0/1508696197 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e031ad0 0x563b5e0377d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.701+0000 7eff06805740 1 --2- 172.21.15.94:0/1508696197 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e030d60 0x563b5e037290 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.701+0000 7eff06805740 1 -- 172.21.15.94:0/1508696197 >> 172.21.15.94:0/1508696197 conn(0x563b5dff8550 msgr2=0x563b5e02d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.702+0000 7efee57fa640 0 client.26087 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.702+0000 7eff06805740 1 -- 172.21.15.94:0/1508696197 shutdown_connections 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.702+0000 7eff06805740 1 -- 172.21.15.94:0/1508696197 wait complete. 2024-02-17T12:02:21.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.702+0000 7eff06805740 20 client.26087 trim_cache size 0 max 16384 2024-02-17T12:02:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.725+0000 7eff06805740 1 Processor -- start 2024-02-17T12:02:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff06805740 1 -- start start 2024-02-17T12:02:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff06805740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e030d60 0x563b5e0399c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff06805740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e031ad0 0x563b5e041160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff06805740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e03a2e0 0x563b5e0416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff06805740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5dc98600 con 0x563b5e031ad0 2024-02-17T12:02:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff06805740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5e02c850 con 0x563b5e03a2e0 2024-02-17T12:02:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff06805740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5e0315a0 con 0x563b5e030d60 2024-02-17T12:02:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff010f7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e031ad0 0x563b5e041160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff010f7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e031ad0 0x563b5e041160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41844/0 (socket says 172.21.15.94:41844) 2024-02-17T12:02:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff010f7640 1 -- 172.21.15.94:0/3210433056 learned_addr learned my addr 172.21.15.94:0/3210433056 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff018f8640 1 --2- 172.21.15.94:0/3210433056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e030d60 0x563b5e0399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff020f9640 1 --2- 172.21.15.94:0/3210433056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e03a2e0 0x563b5e0416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff010f7640 1 -- 172.21.15.94:0/3210433056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e030d60 msgr2=0x563b5e0399c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff010f7640 1 --2- 172.21.15.94:0/3210433056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e030d60 0x563b5e0399c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff010f7640 1 -- 172.21.15.94:0/3210433056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e03a2e0 msgr2=0x563b5e0416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.726+0000 7eff010f7640 1 --2- 172.21.15.94:0/3210433056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e03a2e0 0x563b5e0416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.727+0000 7eff010f7640 1 -- 172.21.15.94:0/3210433056 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b5df337b0 con 0x563b5e031ad0 2024-02-17T12:02:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.727+0000 7eff020f9640 1 --2- 172.21.15.94:0/3210433056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e03a2e0 0x563b5e0416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:02:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.727+0000 7eff010f7640 1 --2- 172.21.15.94:0/3210433056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e031ad0 0x563b5e041160 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7efef8009900 tx=0x7efef803a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5aabeb7a2f67333 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.727+0000 7efee77fe640 1 -- 172.21.15.94:0/3210433056 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef8053ea0 con 0x563b5e031ad0 2024-02-17T12:02:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.727+0000 7efee77fe640 1 -- 172.21.15.94:0/3210433056 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efef8053630 con 0x563b5e031ad0 2024-02-17T12:02:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.727+0000 7efee77fe640 1 -- 172.21.15.94:0/3210433056 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef802dea0 con 0x563b5e031ad0 2024-02-17T12:02:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.728+0000 7eff06805740 1 -- 172.21.15.94:0/3210433056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e031ad0 msgr2=0x563b5e041160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.728+0000 7eff06805740 1 --2- 172.21.15.94:0/3210433056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e031ad0 0x563b5e041160 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7efef8009900 tx=0x7efef803a000 comp rx=0 tx=0).stop 2024-02-17T12:02:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.728+0000 7eff06805740 1 -- 172.21.15.94:0/3210433056 shutdown_connections 2024-02-17T12:02:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.728+0000 7eff06805740 1 --2- 172.21.15.94:0/3210433056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e03a2e0 0x563b5e0416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.728+0000 7eff06805740 1 --2- 172.21.15.94:0/3210433056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e031ad0 0x563b5e041160 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.729+0000 7eff06805740 1 --2- 172.21.15.94:0/3210433056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e030d60 0x563b5e0399c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.729+0000 7eff06805740 1 -- 172.21.15.94:0/3210433056 >> 172.21.15.94:0/3210433056 conn(0x563b5dff8550 msgr2=0x563b5e030280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.729+0000 7eff06805740 1 -- 172.21.15.94:0/3210433056 shutdown_connections 2024-02-17T12:02:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.729+0000 7eff06805740 1 -- 172.21.15.94:0/3210433056 wait complete. 2024-02-17T12:02:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.729+0000 7eff06805740 1 -- start start 2024-02-17T12:02:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.730+0000 7eff06805740 1 Processor -- start 2024-02-17T12:02:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.730+0000 7eff06805740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e030d60 0x563b5e041480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.731+0000 7eff06805740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e031ad0 0x563b5e03d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.731+0000 7eff018f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e030d60 0x563b5e041480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.731+0000 7eff018f8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e030d60 0x563b5e041480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41850/0 (socket says 172.21.15.94:41850) 2024-02-17T12:02:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.731+0000 7eff018f8640 1 -- 172.21.15.94:0/853945128 learned_addr learned my addr 172.21.15.94:0/853945128 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.731+0000 7eff010f7640 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e031ad0 0x563b5e03d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.731+0000 7eff06805740 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e03a2e0 0x563b5e03dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.731+0000 7eff06805740 1 -- 172.21.15.94:0/853945128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5e03e440 con 0x563b5e030d60 2024-02-17T12:02:21.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.731+0000 7eff06805740 1 -- 172.21.15.94:0/853945128 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5e03e5b0 con 0x563b5e031ad0 2024-02-17T12:02:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.731+0000 7eff020f9640 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e03a2e0 0x563b5e03dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.732+0000 7eff06805740 1 -- 172.21.15.94:0/853945128 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b5e0a08b0 con 0x563b5e03a2e0 2024-02-17T12:02:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.732+0000 7eff06805740 10 client.0 _ll_register_callbacks cb 0x7efef4009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.732+0000 7eff018f8640 1 -- 172.21.15.94:0/853945128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e03a2e0 msgr2=0x563b5e03dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.732+0000 7eff018f8640 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e03a2e0 0x563b5e03dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.732+0000 7eff018f8640 1 -- 172.21.15.94:0/853945128 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e031ad0 msgr2=0x563b5e03d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.732+0000 7eff018f8640 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e031ad0 0x563b5e03d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.732+0000 7eff018f8640 1 -- 172.21.15.94:0/853945128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc114f40 con 0x563b5e030d60 2024-02-17T12:02:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.732+0000 7eff06805740 1 -- 172.21.15.94:0/853945128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efef4003d90 con 0x563b5e030d60 2024-02-17T12:02:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.732+0000 7eff018f8640 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e030d60 0x563b5e041480 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7efef4011910 tx=0x7efef4004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce873534e356ff1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.732+0000 7efee57fa640 10 client.16635 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:02:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.733+0000 7efee57fa640 1 -- 172.21.15.94:0/853945128 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef40094a0 con 0x563b5e030d60 2024-02-17T12:02:21.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.733+0000 7efee57fa640 1 -- 172.21.15.94:0/853945128 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7efef400f4e0 con 0x563b5e030d60 2024-02-17T12:02:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.733+0000 7efee57fa640 1 -- 172.21.15.94:0/853945128 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7efef4023840 con 0x563b5e030d60 2024-02-17T12:02:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.733+0000 7eff06805740 10 client.16635 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.733+0000 7eff06805740 1 -- 172.21.15.94:0/853945128 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({fsmap=23}) v3 -- 0x563b5df59540 con 0x563b5e030d60 2024-02-17T12:02:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.734+0000 7efee57fa640 1 -- 172.21.15.94:0/853945128 <== mon.0 v2:172.21.15.94:3300/0 4 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7efef40239a0 con 0x563b5e030d60 2024-02-17T12:02:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.734+0000 7eff06805740 10 client.16635 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.734+0000 7eff06805740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.734+0000 7eff06805740 20 client.16635 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.734+0000 7eff06805740 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563b5e0a1950 0x563b5e0a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.734+0000 7eff06805740 4 client.16635 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:02:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.734+0000 7eff06805740 1 -- 172.21.15.94:0/853945128 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563b5dcd5550 con 0x563b5e0a1950 2024-02-17T12:02:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.734+0000 7eff010f7640 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563b5e0a1950 0x563b5e0a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.735+0000 7eff010f7640 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563b5e0a1950 0x563b5e0a3d70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59055330751d3f6e server_cookie=28f661d36b482ef4 in_seq=0 out_seq=0 2024-02-17T12:02:21.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.735+0000 7efee57fa640 10 client.16635 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.756+0000 7efee57fa640 1 -- 172.21.15.94:0/853945128 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563b5dcd5550 con 0x563b5e0a1950 2024-02-17T12:02:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.757+0000 7efee57fa640 10 client.16635 handle_command_reply: tid=0 2024-02-17T12:02:21.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.757+0000 7eff06805740 1 client.16635 shutdown 2024-02-17T12:02:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.757+0000 7eff06805740 1 -- 172.21.15.94:0/853945128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e030d60 msgr2=0x563b5e041480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.758+0000 7eff06805740 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e030d60 0x563b5e041480 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7efef4011910 tx=0x7efef4004cd0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.758+0000 7eff06805740 1 -- 172.21.15.94:0/853945128 shutdown_connections 2024-02-17T12:02:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.758+0000 7eff06805740 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563b5e0a1950 0x563b5e0a3d70 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.758+0000 7eff06805740 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563b5e03a2e0 0x563b5e03dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.758+0000 7eff06805740 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563b5e031ad0 0x563b5e03d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.758+0000 7eff06805740 1 --2- 172.21.15.94:0/853945128 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563b5e030d60 0x563b5e041480 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.758+0000 7eff06805740 1 -- 172.21.15.94:0/853945128 >> 172.21.15.94:0/853945128 conn(0x563b5dff8550 msgr2=0x563b5e030030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.758+0000 7efee57fa640 0 client.16635 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.758+0000 7eff06805740 1 -- 172.21.15.94:0/853945128 shutdown_connections 2024-02-17T12:02:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.758+0000 7eff06805740 1 -- 172.21.15.94:0/853945128 wait complete. 2024-02-17T12:02:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.759+0000 7eff06805740 20 client.16635 trim_cache size 0 max 16384 2024-02-17T12:02:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:02:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:02:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:02:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:02:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.760+0000 7eff03b5c640 1 -- 172.21.15.94:0/1642376727 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efee80a2480 msgr2=0x7efee80a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.760+0000 7eff03b5c640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efee80a2480 0x7efee80a4940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x563b5e031800 tx=0x7efef8005fd0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.760+0000 7eff03b5c640 1 -- 172.21.15.94:0/1642376727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efefc10d780 msgr2=0x7efefc1a2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.760+0000 7eff03b5c640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efefc10d780 0x7efefc1a2610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efeec00e9a0 tx=0x7efeec00eda0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.760+0000 7eff03b5c640 1 -- 172.21.15.94:0/1642376727 shutdown_connections 2024-02-17T12:02:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.760+0000 7eff03b5c640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efee80a2480 0x7efee80a4940 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.760+0000 7eff03b5c640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efefc10d780 0x7efefc1a2610 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.760+0000 7eff03b5c640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efefc108130 0x7efefc19e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.760+0000 7eff03b5c640 1 --2- 172.21.15.94:0/1642376727 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efefc107770 0x7efefc19dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.760+0000 7eff03b5c640 1 -- 172.21.15.94:0/1642376727 >> 172.21.15.94:0/1642376727 conn(0x7efefc0756f0 msgr2=0x7efefc075c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.761+0000 7eff03b5c640 1 -- 172.21.15.94:0/1642376727 shutdown_connections 2024-02-17T12:02:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:21.761+0000 7eff03b5c640 1 -- 172.21.15.94:0/1642376727 wait complete. 2024-02-17T12:02:21.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.808+0000 7fb325b92640 1 -- 172.21.15.94:0/3727636846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3180a5970 msgr2=0x7fb3180ab9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.808+0000 7fb325b92640 1 --2- 172.21.15.94:0/3727636846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3180a5970 0x7fb3180ab9d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb314009900 tx=0x7fb31402bc30 comp rx=0 tx=0).stop 2024-02-17T12:02:21.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.808+0000 7fb325b92640 1 -- 172.21.15.94:0/3727636846 shutdown_connections 2024-02-17T12:02:21.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.808+0000 7fb325b92640 1 --2- 172.21.15.94:0/3727636846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3180ac100 0x7fb3180b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.808+0000 7fb325b92640 1 --2- 172.21.15.94:0/3727636846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3180a5970 0x7fb3180ab9d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.808+0000 7fb325b92640 1 --2- 172.21.15.94:0/3727636846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3180a4fc0 0x7fb3180a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.808+0000 7fb325b92640 1 -- 172.21.15.94:0/3727636846 >> 172.21.15.94:0/3727636846 conn(0x7fb31801a640 msgr2=0x7fb31801aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:21.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.808+0000 7fb325b92640 1 -- 172.21.15.94:0/3727636846 shutdown_connections 2024-02-17T12:02:21.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.808+0000 7fb325b92640 1 -- 172.21.15.94:0/3727636846 wait complete. 2024-02-17T12:02:21.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.808+0000 7fb325b92640 1 Processor -- start 2024-02-17T12:02:21.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.809+0000 7fb325b92640 1 -- start start 2024-02-17T12:02:21.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.809+0000 7fb325b92640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3180a4fc0 0x7fb318149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.809+0000 7fb325b92640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3180ac100 0x7fb318149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.809+0000 7fb325b92640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3180b2d00 0x7fb3180b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.809+0000 7fb325b92640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180b37b0 con 0x7fb3180b2d00 2024-02-17T12:02:21.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.809+0000 7fb325b92640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180b3920 con 0x7fb3180a4fc0 2024-02-17T12:02:21.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.809+0000 7fb325b92640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180b3a90 con 0x7fb3180ac100 2024-02-17T12:02:21.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.809+0000 7fb324b90640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3180a4fc0 0x7fb318149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.809+0000 7fb324b90640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3180a4fc0 0x7fb318149680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46870/0 (socket says 172.21.15.94:46870) 2024-02-17T12:02:21.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.809+0000 7fb324b90640 1 -- 172.21.15.94:0/3824864858 learned_addr learned my addr 172.21.15.94:0/3824864858 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:21.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.809+0000 7fb325391640 1 --2- 172.21.15.94:0/3824864858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3180b2d00 0x7fb3180b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.810+0000 7fb324b90640 1 -- 172.21.15.94:0/3824864858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3180ac100 msgr2=0x7fb318149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.810+0000 7fb324b90640 1 --2- 172.21.15.94:0/3824864858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3180ac100 0x7fb318149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.810+0000 7fb324b90640 1 -- 172.21.15.94:0/3824864858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3180b2d00 msgr2=0x7fb3180b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.810+0000 7fb324b90640 1 --2- 172.21.15.94:0/3824864858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3180b2d00 0x7fb3180b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.810+0000 7fb324b90640 1 -- 172.21.15.94:0/3824864858 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32004f750 con 0x7fb3180a4fc0 2024-02-17T12:02:21.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.810+0000 7fb324b90640 1 --2- 172.21.15.94:0/3824864858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3180a4fc0 0x7fb318149680 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb320064670 tx=0x7fb320072e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:21.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.810+0000 7fb31dffb640 1 -- 172.21.15.94:0/3824864858 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320070c90 con 0x7fb3180a4fc0 2024-02-17T12:02:21.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.810+0000 7fb325b92640 1 -- 172.21.15.94:0/3824864858 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb314009560 con 0x7fb3180a4fc0 2024-02-17T12:02:21.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.810+0000 7fb31dffb640 1 -- 172.21.15.94:0/3824864858 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb320070df0 con 0x7fb3180a4fc0 2024-02-17T12:02:21.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.810+0000 7fb31dffb640 1 -- 172.21.15.94:0/3824864858 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32007c820 con 0x7fb3180a4fc0 2024-02-17T12:02:21.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.810+0000 7fb325b92640 1 -- 172.21.15.94:0/3824864858 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb318150830 con 0x7fb3180a4fc0 2024-02-17T12:02:21.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.813+0000 7fb31dffb640 1 -- 172.21.15.94:0/3824864858 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb3200853f0 con 0x7fb3180a4fc0 2024-02-17T12:02:21.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.814+0000 7fb3299c6740 1 Processor -- start 2024-02-17T12:02:21.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.815+0000 7fb3299c6740 1 -- start start 2024-02-17T12:02:21.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.815+0000 7fb3299c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab847c550 0x563ab847c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.815+0000 7fb3299c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847cf00 0x563ab84854b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.815+0000 7fb3299c6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab8485a80 0x563ab8487ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.815+0000 7fb3299c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab83eaf90 con 0x563ab847c550 2024-02-17T12:02:21.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.815+0000 7fb3299c6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab83e84c0 con 0x563ab8485a80 2024-02-17T12:02:21.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.815+0000 7fb3299c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab83e3ca0 con 0x563ab847cf00 2024-02-17T12:02:21.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.815+0000 7fb31dffb640 1 --2- 172.21.15.94:0/3824864858 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb3080a2330 0x7fb3080a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.815+0000 7fb31ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab8485a80 0x563ab8487ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.815+0000 7fb324b90640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847cf00 0x563ab84854b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.815+0000 7fb31dffb640 1 -- 172.21.15.94:0/3824864858 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb320131510 con 0x7fb3180a4fc0 2024-02-17T12:02:21.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.816+0000 7fb31ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab8485a80 0x563ab8487ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46878/0 (socket says 172.21.15.94:46878) 2024-02-17T12:02:21.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.816+0000 7fb324b90640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847cf00 0x563ab84854b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55772/0 (socket says 172.21.15.94:55772) 2024-02-17T12:02:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.816+0000 7fb31ffff640 1 -- 172.21.15.94:0/2935988804 learned_addr learned my addr 172.21.15.94:0/2935988804 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.816+0000 7fb324b90640 1 -- 172.21.15.94:0/2935988804 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab8485a80 msgr2=0x563ab8487ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.816+0000 7fb324b90640 1 --2- 172.21.15.94:0/2935988804 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab8485a80 0x563ab8487ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.816+0000 7fb324b90640 1 -- 172.21.15.94:0/2935988804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab847c550 msgr2=0x563ab847c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.816+0000 7fb324b90640 1 --2- 172.21.15.94:0/2935988804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab847c550 0x563ab847c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.816+0000 7fb324b90640 1 -- 172.21.15.94:0/2935988804 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab83a60b0 con 0x563ab847cf00 2024-02-17T12:02:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.816+0000 7fb325391640 1 --2- 172.21.15.94:0/3824864858 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb3080a2330 0x7fb3080a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.817+0000 7fb324b90640 1 --2- 172.21.15.94:0/2935988804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847cf00 0x563ab84854b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb32012ebf0 tx=0x7fb3200fdd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=234baa1febedf31b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:21.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.818+0000 7fb325391640 1 --2- 172.21.15.94:0/3824864858 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb3080a2330 0x7fb3080a47f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb310009710 tx=0x7fb310013040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.818+0000 7fb306ffd640 1 -- 172.21.15.94:0/2935988804 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3200fde90 con 0x563ab847cf00 2024-02-17T12:02:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.819+0000 7fb306ffd640 1 -- 172.21.15.94:0/2935988804 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb320101fe0 con 0x563ab847cf00 2024-02-17T12:02:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.819+0000 7fb306ffd640 1 -- 172.21.15.94:0/2935988804 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3201061a0 con 0x563ab847cf00 2024-02-17T12:02:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.820+0000 7fb3299c6740 1 -- 172.21.15.94:0/2935988804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847cf00 msgr2=0x563ab84854b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.820+0000 7fb3299c6740 1 --2- 172.21.15.94:0/2935988804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847cf00 0x563ab84854b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb32012ebf0 tx=0x7fb3200fdd20 comp rx=0 tx=0).stop 2024-02-17T12:02:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.825+0000 7fb3299c6740 1 -- 172.21.15.94:0/2935988804 shutdown_connections 2024-02-17T12:02:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.825+0000 7fb3299c6740 1 --2- 172.21.15.94:0/2935988804 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab8485a80 0x563ab8487ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.826+0000 7fb3299c6740 1 --2- 172.21.15.94:0/2935988804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847cf00 0x563ab84854b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.826+0000 7fb3299c6740 1 --2- 172.21.15.94:0/2935988804 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab847c550 0x563ab847c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.826+0000 7fb3299c6740 1 -- 172.21.15.94:0/2935988804 >> 172.21.15.94:0/2935988804 conn(0x563ab8476a10 msgr2=0x563ab8476df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:21.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.827+0000 7fb3299c6740 1 -- 172.21.15.94:0/2935988804 shutdown_connections 2024-02-17T12:02:21.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.828+0000 7fb3299c6740 1 -- 172.21.15.94:0/2935988804 wait complete. 2024-02-17T12:02:21.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.829+0000 7fb3299c6740 1 -- start start 2024-02-17T12:02:21.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.829+0000 7fb3299c6740 1 Processor -- start 2024-02-17T12:02:21.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.829+0000 7fb3299c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847c550 0x563ab84829c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.829+0000 7fb3299c6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab847cf00 0x563ab8482f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.829+0000 7fb3299c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab8485a80 0x563ab8483440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.829+0000 7fb3299c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab8388f40 con 0x563ab8485a80 2024-02-17T12:02:21.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.829+0000 7fb3299c6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab80b74b0 con 0x563ab847cf00 2024-02-17T12:02:21.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.829+0000 7fb3299c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab8369670 con 0x563ab847c550 2024-02-17T12:02:21.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.829+0000 7fb3299c6740 10 client.0 _ll_register_callbacks cb 0x563ab83d6f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:21.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.830+0000 7fb324b90640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847c550 0x563ab84829c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.830+0000 7fb324b90640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847c550 0x563ab84829c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55774/0 (socket says 172.21.15.94:55774) 2024-02-17T12:02:21.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.830+0000 7fb324b90640 1 -- 172.21.15.94:0/252855700 learned_addr learned my addr 172.21.15.94:0/252855700 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:21.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.831+0000 7fb324b90640 1 -- 172.21.15.94:0/252855700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab847cf00 msgr2=0x563ab8482f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:02:21.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.831+0000 7fb324b90640 1 --2- 172.21.15.94:0/252855700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab847cf00 0x563ab8482f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.831+0000 7fb324b90640 1 -- 172.21.15.94:0/252855700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab8485a80 msgr2=0x563ab8483440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.831+0000 7fb324b90640 1 --2- 172.21.15.94:0/252855700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab8485a80 0x563ab8483440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.831+0000 7fb324b90640 1 -- 172.21.15.94:0/252855700 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb318150830 con 0x563ab847c550 2024-02-17T12:02:21.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.831+0000 7fb324b90640 1 --2- 172.21.15.94:0/252855700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847c550 0x563ab84829c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb320106100 tx=0x7fb320067020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=147a077a5d1a2f91 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.833+0000 7fb2fffff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:02:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.834+0000 7fb2fffff640 1 -- 172.21.15.94:0/252855700 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3200fa500 con 0x563ab847c550 2024-02-17T12:02:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.834+0000 7fb3299c6740 1 -- 172.21.15.94:0/252855700 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ab83bdf40 con 0x563ab847c550 2024-02-17T12:02:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.834+0000 7fb2fffff640 1 -- 172.21.15.94:0/252855700 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb3200fde90 con 0x563ab847c550 2024-02-17T12:02:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.835+0000 7fb2fffff640 1 -- 172.21.15.94:0/252855700 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb32011c400 con 0x563ab847c550 2024-02-17T12:02:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.835+0000 7fb3299c6740 10 client.26149 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:21.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.835+0000 7fb3299c6740 1 -- 172.21.15.94:0/252855700 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563ab83e2320 con 0x563ab847c550 2024-02-17T12:02:21.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.836+0000 7fb2fffff640 1 -- 172.21.15.94:0/252855700 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563ab83bdf40 con 0x563ab847c550 2024-02-17T12:02:21.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.836+0000 7fb2fffff640 1 -- 172.21.15.94:0/252855700 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb32007a020 con 0x563ab847c550 2024-02-17T12:02:21.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.836+0000 7fb3299c6740 10 client.26149 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:21.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.836+0000 7fb3299c6740 10 client.26149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:21.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.836+0000 7fb3299c6740 20 client.26149 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:21.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.836+0000 7fb3299c6740 1 --2- 172.21.15.94:0/252855700 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563ab84b7bc0 0x563ab84b9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.836+0000 7fb3299c6740 4 client.26149 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:21.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.836+0000 7fb3299c6740 1 -- 172.21.15.94:0/252855700 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ab83da310 con 0x563ab84b7bc0 2024-02-17T12:02:21.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.839+0000 7fb31ffff640 1 --2- 172.21.15.94:0/252855700 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563ab84b7bc0 0x563ab84b9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.839+0000 7fb31ffff640 1 --2- 172.21.15.94:0/252855700 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563ab84b7bc0 0x563ab84b9fb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=160b41fa22045887 server_cookie=ee8be266094d6a4f in_seq=0 out_seq=0 2024-02-17T12:02:21.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.839+0000 7fb2fffff640 10 client.26149 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:21.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.840+0000 7fb2fffff640 1 -- 172.21.15.94:0/252855700 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ab83da310 con 0x563ab84b7bc0 2024-02-17T12:02:21.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.840+0000 7fb2fffff640 10 client.26149 handle_command_reply: tid=0 2024-02-17T12:02:21.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.840+0000 7fb3299c6740 1 client.26149 shutdown 2024-02-17T12:02:21.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.840+0000 7fb3299c6740 1 -- 172.21.15.94:0/252855700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847c550 msgr2=0x563ab84829c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.840+0000 7fb3299c6740 1 --2- 172.21.15.94:0/252855700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847c550 0x563ab84829c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb320106100 tx=0x7fb320067020 comp rx=0 tx=0).stop 2024-02-17T12:02:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.841+0000 7fb3299c6740 1 -- 172.21.15.94:0/252855700 shutdown_connections 2024-02-17T12:02:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.841+0000 7fb3299c6740 1 --2- 172.21.15.94:0/252855700 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563ab84b7bc0 0x563ab84b9fb0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.841+0000 7fb3299c6740 1 --2- 172.21.15.94:0/252855700 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab8485a80 0x563ab8483440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.841+0000 7fb3299c6740 1 --2- 172.21.15.94:0/252855700 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab847cf00 0x563ab8482f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.841+0000 7fb3299c6740 1 --2- 172.21.15.94:0/252855700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab847c550 0x563ab84829c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.841+0000 7fb3299c6740 1 -- 172.21.15.94:0/252855700 >> 172.21.15.94:0/252855700 conn(0x563ab8476a10 msgr2=0x563ab8478970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:21.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.841+0000 7fb2fffff640 0 client.26149 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.842+0000 7fb3299c6740 1 -- 172.21.15.94:0/252855700 shutdown_connections 2024-02-17T12:02:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.842+0000 7fb3299c6740 1 -- 172.21.15.94:0/252855700 wait complete. 2024-02-17T12:02:21.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.842+0000 7fb3299c6740 20 client.26149 trim_cache size 0 max 16384 2024-02-17T12:02:21.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.865+0000 7fb3299c6740 1 Processor -- start 2024-02-17T12:02:21.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.865+0000 7fb3299c6740 1 -- start start 2024-02-17T12:02:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb3299c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab847cf00 0x563ab8475b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb3299c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab8485a80 0x563ab848cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb3299c6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab8478ca0 0x563ab848d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb3299c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab8477810 con 0x563ab847cf00 2024-02-17T12:02:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb3299c6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab84888d0 con 0x563ab8478ca0 2024-02-17T12:02:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb3299c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab8472350 con 0x563ab8485a80 2024-02-17T12:02:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb324b90640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab847cf00 0x563ab8475b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb324b90640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab847cf00 0x563ab8475b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41882/0 (socket says 172.21.15.94:41882) 2024-02-17T12:02:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb324b90640 1 -- 172.21.15.94:0/2338343734 learned_addr learned my addr 172.21.15.94:0/2338343734 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb325391640 1 --2- 172.21.15.94:0/2338343734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab8478ca0 0x563ab848d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb31ffff640 1 --2- 172.21.15.94:0/2338343734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab8485a80 0x563ab848cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb324b90640 1 -- 172.21.15.94:0/2338343734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab8485a80 msgr2=0x563ab848cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb324b90640 1 --2- 172.21.15.94:0/2338343734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab8485a80 0x563ab848cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb324b90640 1 -- 172.21.15.94:0/2338343734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab8478ca0 msgr2=0x563ab848d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb324b90640 1 --2- 172.21.15.94:0/2338343734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab8478ca0 0x563ab848d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb324b90640 1 -- 172.21.15.94:0/2338343734 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab837ec00 con 0x563ab847cf00 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb31ffff640 1 --2- 172.21.15.94:0/2338343734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab8485a80 0x563ab848cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb324b90640 1 --2- 172.21.15.94:0/2338343734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab847cf00 0x563ab8475b10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb3200fde20 tx=0x7fb3200fd7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f126072c2aac9b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.866+0000 7fb3067fc640 1 -- 172.21.15.94:0/2338343734 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320101fe0 con 0x563ab847cf00 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.867+0000 7fb3067fc640 1 -- 172.21.15.94:0/2338343734 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb320106630 con 0x563ab847cf00 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.867+0000 7fb3067fc640 1 -- 172.21.15.94:0/2338343734 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320106400 con 0x563ab847cf00 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.867+0000 7fb3299c6740 1 -- 172.21.15.94:0/2338343734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab847cf00 msgr2=0x563ab8475b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.867+0000 7fb3299c6740 1 --2- 172.21.15.94:0/2338343734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab847cf00 0x563ab8475b10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb3200fde20 tx=0x7fb3200fd7f0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.867+0000 7fb3299c6740 1 -- 172.21.15.94:0/2338343734 shutdown_connections 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.867+0000 7fb3299c6740 1 --2- 172.21.15.94:0/2338343734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab8478ca0 0x563ab848d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.867+0000 7fb3299c6740 1 --2- 172.21.15.94:0/2338343734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab8485a80 0x563ab848cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.867+0000 7fb3299c6740 1 --2- 172.21.15.94:0/2338343734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab847cf00 0x563ab8475b10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.867+0000 7fb3299c6740 1 -- 172.21.15.94:0/2338343734 >> 172.21.15.94:0/2338343734 conn(0x563ab8476a10 msgr2=0x563ab8475400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.867+0000 7fb3299c6740 1 -- 172.21.15.94:0/2338343734 shutdown_connections 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.867+0000 7fb3299c6740 1 -- 172.21.15.94:0/2338343734 wait complete. 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb3299c6740 1 -- start start 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb3299c6740 1 Processor -- start 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb3299c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab8478ca0 0x563ab8507350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb3299c6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab847cf00 0x563ab8507890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb3299c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab8485a80 0x563ab848bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb3299c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab848c330 con 0x563ab8478ca0 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb3299c6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab848c4a0 con 0x563ab847cf00 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb3299c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab848c660 con 0x563ab8485a80 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb324b90640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab8478ca0 0x563ab8507350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb324b90640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab8478ca0 0x563ab8507350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41896/0 (socket says 172.21.15.94:41896) 2024-02-17T12:02:21.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb324b90640 1 -- 172.21.15.94:0/569064081 learned_addr learned my addr 172.21.15.94:0/569064081 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb3299c6740 10 client.0 _ll_register_callbacks cb 0x7fb320066920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb31ffff640 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab847cf00 0x563ab8507890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.868+0000 7fb325391640 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab8485a80 0x563ab848bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.869+0000 7fb324b90640 1 -- 172.21.15.94:0/569064081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab8485a80 msgr2=0x563ab848bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.869+0000 7fb324b90640 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab8485a80 0x563ab848bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.869+0000 7fb324b90640 1 -- 172.21.15.94:0/569064081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab847cf00 msgr2=0x563ab8507890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.869+0000 7fb324b90640 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab847cf00 0x563ab8507890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.869+0000 7fb324b90640 1 -- 172.21.15.94:0/569064081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ab83e2320 con 0x563ab8478ca0 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.869+0000 7fb324b90640 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab8478ca0 0x563ab8507350 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb320074e40 tx=0x7fb3200fd880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467c1b8e61faeff1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.869+0000 7fb2fffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.870+0000 7fb2fffff640 1 -- 172.21.15.94:0/569064081 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320101fe0 con 0x563ab8478ca0 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.870+0000 7fb31ffff640 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab847cf00 0x563ab8507890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.870+0000 7fb2fffff640 1 -- 172.21.15.94:0/569064081 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb3200fde90 con 0x563ab8478ca0 2024-02-17T12:02:21.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.870+0000 7fb3299c6740 1 -- 172.21.15.94:0/569064081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563ab8409900 con 0x563ab8478ca0 2024-02-17T12:02:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.870+0000 7fb2fffff640 1 -- 172.21.15.94:0/569064081 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb32011c400 con 0x563ab8478ca0 2024-02-17T12:02:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.870+0000 7fb3299c6740 10 client.16647 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:21.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.870+0000 7fb3299c6740 1 -- 172.21.15.94:0/569064081 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x563ab83a4910 con 0x563ab8478ca0 2024-02-17T12:02:21.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.870+0000 7fb2fffff640 1 -- 172.21.15.94:0/569064081 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb320121080 con 0x563ab8478ca0 2024-02-17T12:02:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.870+0000 7fb2fffff640 1 -- 172.21.15.94:0/569064081 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb320102f00 con 0x563ab8478ca0 2024-02-17T12:02:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.870+0000 7fb3299c6740 10 client.16647 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.870+0000 7fb3299c6740 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.870+0000 7fb3299c6740 20 client.16647 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.870+0000 7fb3299c6740 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563ab850fe00 0x563ab85121f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.871+0000 7fb31ffff640 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563ab850fe00 0x563ab85121f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.871+0000 7fb3299c6740 4 client.16647 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:02:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.871+0000 7fb3299c6740 1 -- 172.21.15.94:0/569064081 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563ab8409900 con 0x563ab850fe00 2024-02-17T12:02:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.872+0000 7fb31ffff640 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563ab850fe00 0x563ab85121f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa98ceaa01ef7752 server_cookie=8fcadf0ae0c97938 in_seq=0 out_seq=0 2024-02-17T12:02:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.872+0000 7fb2fffff640 10 client.16647 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb2fffff640 1 -- 172.21.15.94:0/569064081 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563ab8409900 con 0x563ab850fe00 2024-02-17T12:02:21.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb2fffff640 10 client.16647 handle_command_reply: tid=0 2024-02-17T12:02:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb3299c6740 1 client.16647 shutdown 2024-02-17T12:02:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb3299c6740 1 -- 172.21.15.94:0/569064081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab8478ca0 msgr2=0x563ab8507350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb3299c6740 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab8478ca0 0x563ab8507350 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb320074e40 tx=0x7fb3200fd880 comp rx=0 tx=0).stop 2024-02-17T12:02:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb3299c6740 1 -- 172.21.15.94:0/569064081 shutdown_connections 2024-02-17T12:02:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb3299c6740 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563ab850fe00 0x563ab85121f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb2fffff640 0 client.16647 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb3299c6740 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563ab8485a80 0x563ab848bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb3299c6740 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563ab847cf00 0x563ab8507890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb3299c6740 1 --2- 172.21.15.94:0/569064081 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563ab8478ca0 0x563ab8507350 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb3299c6740 1 -- 172.21.15.94:0/569064081 >> 172.21.15.94:0/569064081 conn(0x563ab8476a10 msgr2=0x563ab8475c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb3299c6740 1 -- 172.21.15.94:0/569064081 shutdown_connections 2024-02-17T12:02:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb3299c6740 1 -- 172.21.15.94:0/569064081 wait complete. 2024-02-17T12:02:21.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.873+0000 7fb3299c6740 20 client.16647 trim_cache size 0 max 16384 2024-02-17T12:02:21.878 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:02:21.878 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:02:21.878 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "84731426-86c7-48e6-848d-e54b5432ab8c", 2024-02-17T12:02:21.878 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:02:21.878 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:02:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.874+0000 7fb325b92640 1 -- 172.21.15.94:0/3824864858 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb3080a2330 msgr2=0x7fb3080a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.874+0000 7fb325b92640 1 --2- 172.21.15.94:0/3824864858 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb3080a2330 0x7fb3080a47f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb310009710 tx=0x7fb310013040 comp rx=0 tx=0).stop 2024-02-17T12:02:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.874+0000 7fb325b92640 1 -- 172.21.15.94:0/3824864858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3180a4fc0 msgr2=0x7fb318149680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.874+0000 7fb325b92640 1 --2- 172.21.15.94:0/3824864858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3180a4fc0 0x7fb318149680 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb320064670 tx=0x7fb320072e00 comp rx=0 tx=0).stop 2024-02-17T12:02:21.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.874+0000 7fb325b92640 1 -- 172.21.15.94:0/3824864858 shutdown_connections 2024-02-17T12:02:21.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.874+0000 7fb325b92640 1 --2- 172.21.15.94:0/3824864858 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb3080a2330 0x7fb3080a47f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.874+0000 7fb325b92640 1 --2- 172.21.15.94:0/3824864858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb3180b2d00 0x7fb3180b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.874+0000 7fb325b92640 1 --2- 172.21.15.94:0/3824864858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3180ac100 0x7fb318149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.874+0000 7fb325b92640 1 --2- 172.21.15.94:0/3824864858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb3180a4fc0 0x7fb318149680 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:21.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.874+0000 7fb325b92640 1 -- 172.21.15.94:0/3824864858 >> 172.21.15.94:0/3824864858 conn(0x7fb31801a640 msgr2=0x7fb3180ad2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:21.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.874+0000 7fb325b92640 1 -- 172.21.15.94:0/3824864858 shutdown_connections 2024-02-17T12:02:21.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:21.874+0000 7fb325b92640 1 -- 172.21.15.94:0/3824864858 wait complete. 2024-02-17T12:02:21.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:02:21.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:02:22.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.098+0000 7f06b12fb640 1 -- 172.21.15.94:0/3217590343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ac100130 msgr2=0x7f06ac10c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.098+0000 7f06b12fb640 1 --2- 172.21.15.94:0/3217590343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ac100130 0x7f06ac10c640 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f06a4009c10 tx=0x7f06a402bef0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.098+0000 7f06b12fb640 1 -- 172.21.15.94:0/3217590343 shutdown_connections 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.098+0000 7f06b12fb640 1 --2- 172.21.15.94:0/3217590343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ac100130 0x7f06ac10c640 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.098+0000 7f06b12fb640 1 --2- 172.21.15.94:0/3217590343 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06ac0ff770 0x7f06ac0ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.098+0000 7f06b12fb640 1 --2- 172.21.15.94:0/3217590343 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06ac105780 0x7f06ac105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.098+0000 7f06b12fb640 1 -- 172.21.15.94:0/3217590343 >> 172.21.15.94:0/3217590343 conn(0x7f06ac0765b0 msgr2=0x7f06ac0fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.098+0000 7f06b12fb640 1 -- 172.21.15.94:0/3217590343 shutdown_connections 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.098+0000 7f06b12fb640 1 -- 172.21.15.94:0/3217590343 wait complete. 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.099+0000 7f06b12fb640 1 Processor -- start 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.099+0000 7f06b12fb640 1 -- start start 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.099+0000 7f06b12fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06ac0ff770 0x7f06ac1154d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.099+0000 7f06b12fb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06ac100130 0x7f06ac115a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.099+0000 7f06b12fb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ac105780 0x7f06ac110630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.099+0000 7f06b12fb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac110dc0 con 0x7f06ac0ff770 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.099+0000 7f06b12fb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac110f30 con 0x7f06ac100130 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.099+0000 7f06b12fb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac1110f0 con 0x7f06ac105780 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.099+0000 7f06a3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06ac100130 0x7f06ac115a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.100+0000 7f06a3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06ac100130 0x7f06ac115a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46934/0 (socket says 172.21.15.94:46934) 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.100+0000 7f06a3fff640 1 -- 172.21.15.94:0/3274709036 learned_addr learned my addr 172.21.15.94:0/3274709036 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.100+0000 7f06ab7fe640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ac105780 0x7f06ac110630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.100+0000 7f06aaffd640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06ac0ff770 0x7f06ac1154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.100+0000 7f06a3fff640 1 -- 172.21.15.94:0/3274709036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ac105780 msgr2=0x7f06ac110630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.100+0000 7f06a3fff640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ac105780 0x7f06ac110630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.100+0000 7f06a3fff640 1 -- 172.21.15.94:0/3274709036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06ac0ff770 msgr2=0x7f06ac1154d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.100+0000 7f06a3fff640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06ac0ff770 0x7f06ac1154d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.100+0000 7f06a3fff640 1 -- 172.21.15.94:0/3274709036 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a4009840 con 0x7f06ac100130 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.100+0000 7f06a3fff640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06ac100130 0x7f06ac115a10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f069c00e9f0 tx=0x7f069c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.100+0000 7f06a8ff9640 1 -- 172.21.15.94:0/3274709036 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c009720 con 0x7f06ac100130 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.101+0000 7f06a8ff9640 1 -- 172.21.15.94:0/3274709036 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f069c005500 con 0x7f06ac100130 2024-02-17T12:02:22.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.101+0000 7f06b12fb640 1 -- 172.21.15.94:0/3274709036 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac1113d0 con 0x7f06ac100130 2024-02-17T12:02:22.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.101+0000 7f06a8ff9640 1 -- 172.21.15.94:0/3274709036 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c010430 con 0x7f06ac100130 2024-02-17T12:02:22.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.101+0000 7f06b12fb640 1 -- 172.21.15.94:0/3274709036 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac11b470 con 0x7f06ac100130 2024-02-17T12:02:22.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.103+0000 7f06a8ff9640 1 -- 172.21.15.94:0/3274709036 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f069c005020 con 0x7f06ac100130 2024-02-17T12:02:22.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.103+0000 7f06b3fad740 1 Processor -- start 2024-02-17T12:02:22.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.103+0000 7f06b3fad740 1 -- start start 2024-02-17T12:02:22.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06b3fad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c30690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06b3fad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 0x55ee98c39270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06b3fad740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c397b0 0x55ee98c3bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06b3fad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee98b9f280 con 0x55ee98c302b0 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06b3fad740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee98b9c7b0 con 0x55ee98c397b0 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06b3fad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee98b97f90 con 0x55ee98c30bd0 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c30690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06aaffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 0x55ee98c39270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06a3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c397b0 0x55ee98c3bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06aaffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 0x55ee98c39270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55818/0 (socket says 172.21.15.94:55818) 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06aaffd640 1 -- 172.21.15.94:0/838952237 learned_addr learned my addr 172.21.15.94:0/838952237 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06a8ff9640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f068c0a22b0 0x7f068c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06ab7fe640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f068c0a22b0 0x7f068c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06a3fff640 1 -- 172.21.15.94:0/838952237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 msgr2=0x55ee98c39270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06a3fff640 1 --2- 172.21.15.94:0/838952237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 0x55ee98c39270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06a3fff640 1 -- 172.21.15.94:0/838952237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 msgr2=0x55ee98c30690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:22.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06a3fff640 1 --2- 172.21.15.94:0/838952237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c30690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06a3fff640 1 -- 172.21.15.94:0/838952237 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee98b56570 con 0x55ee98c397b0 2024-02-17T12:02:22.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06aaffd640 1 --2- 172.21.15.94:0/838952237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 0x55ee98c39270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:22.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.104+0000 7f06a8ff9640 1 -- 172.21.15.94:0/3274709036 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f069c014070 con 0x7f06ac100130 2024-02-17T12:02:22.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.105+0000 7f06ab7fe640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f068c0a22b0 0x7f068c0a4770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f06a4004af0 tx=0x7f06a402bef0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:22.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.106+0000 7f06a3fff640 1 --2- 172.21.15.94:0/838952237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c397b0 0x55ee98c3bba0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f069c0cbb50 tx=0x7f069c021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d01e044f84fdb1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:22.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.106+0000 7f06a17fa640 1 -- 172.21.15.94:0/838952237 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c09b4f0 con 0x55ee98c397b0 2024-02-17T12:02:22.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.106+0000 7f06a17fa640 1 -- 172.21.15.94:0/838952237 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f069c005670 con 0x55ee98c397b0 2024-02-17T12:02:22.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.106+0000 7f06a17fa640 1 -- 172.21.15.94:0/838952237 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c098e90 con 0x55ee98c397b0 2024-02-17T12:02:22.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.107+0000 7f06b3fad740 1 -- 172.21.15.94:0/838952237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c397b0 msgr2=0x55ee98c3bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:22.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.107+0000 7f06b3fad740 1 --2- 172.21.15.94:0/838952237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c397b0 0x55ee98c3bba0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f069c0cbb50 tx=0x7f069c021000 comp rx=0 tx=0).stop 2024-02-17T12:02:22.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.107+0000 7f06b3fad740 1 -- 172.21.15.94:0/838952237 shutdown_connections 2024-02-17T12:02:22.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.107+0000 7f06b3fad740 1 --2- 172.21.15.94:0/838952237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c397b0 0x55ee98c3bba0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.107+0000 7f06b3fad740 1 --2- 172.21.15.94:0/838952237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 0x55ee98c39270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.107+0000 7f06b3fad740 1 --2- 172.21.15.94:0/838952237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c30690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.107+0000 7f06b3fad740 1 -- 172.21.15.94:0/838952237 >> 172.21.15.94:0/838952237 conn(0x55ee98c2a990 msgr2=0x55ee98c2ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:22.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.108+0000 7f06b3fad740 1 -- 172.21.15.94:0/838952237 shutdown_connections 2024-02-17T12:02:22.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.108+0000 7f06b3fad740 1 -- 172.21.15.94:0/838952237 wait complete. 2024-02-17T12:02:22.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.108+0000 7f06b3fad740 1 -- start start 2024-02-17T12:02:22.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.108+0000 7f06b3fad740 1 Processor -- start 2024-02-17T12:02:22.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.109+0000 7f06b3fad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c39200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.109+0000 7f06b3fad740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c30bd0 0x55ee98c36590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.109+0000 7f06b3fad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c397b0 0x55ee98c36cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.109+0000 7f06b3fad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee9886b430 con 0x55ee98c302b0 2024-02-17T12:02:22.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.109+0000 7f06b3fad740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee98827e00 con 0x55ee98c30bd0 2024-02-17T12:02:22.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.109+0000 7f06b3fad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee98b1cfd0 con 0x55ee98c397b0 2024-02-17T12:02:22.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.109+0000 7f06b3fad740 10 client.0 _ll_register_callbacks cb 0x55ee98b72fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.109+0000 7f06aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c39200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.109+0000 7f06aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c39200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41936/0 (socket says 172.21.15.94:41936) 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.109+0000 7f06aaffd640 1 -- 172.21.15.94:0/1264918453 learned_addr learned my addr 172.21.15.94:0/1264918453 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.109+0000 7f06ab7fe640 1 --2- 172.21.15.94:0/1264918453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c397b0 0x55ee98c36cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.110+0000 7f06aaffd640 1 -- 172.21.15.94:0/1264918453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c397b0 msgr2=0x55ee98c36cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.110+0000 7f06aaffd640 1 --2- 172.21.15.94:0/1264918453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c397b0 0x55ee98c36cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.110+0000 7f06aaffd640 1 -- 172.21.15.94:0/1264918453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c30bd0 msgr2=0x55ee98c36590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.110+0000 7f06aaffd640 1 --2- 172.21.15.94:0/1264918453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c30bd0 0x55ee98c36590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.110+0000 7f06aaffd640 1 -- 172.21.15.94:0/1264918453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac11b470 con 0x55ee98c302b0 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.110+0000 7f06aaffd640 1 --2- 172.21.15.94:0/1264918453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c39200 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0694011760 tx=0x7f0694011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e83b5a9180954fc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.110+0000 7f069a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.110+0000 7f069a7fc640 1 -- 172.21.15.94:0/1264918453 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694016070 con 0x55ee98c302b0 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.110+0000 7f069a7fc640 1 -- 172.21.15.94:0/1264918453 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0694004260 con 0x55ee98c302b0 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.110+0000 7f06b3fad740 1 -- 172.21.15.94:0/1264918453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee988977e0 con 0x55ee98c302b0 2024-02-17T12:02:22.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.111+0000 7f069a7fc640 1 -- 172.21.15.94:0/1264918453 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0694004460 con 0x55ee98c302b0 2024-02-17T12:02:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.111+0000 7f06b3fad740 10 client.16659 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.111+0000 7f06b3fad740 1 -- 172.21.15.94:0/1264918453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ee98b96610 con 0x55ee98c302b0 2024-02-17T12:02:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.112+0000 7f069a7fc640 1 -- 172.21.15.94:0/1264918453 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0694027020 con 0x55ee98c302b0 2024-02-17T12:02:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.112+0000 7f069a7fc640 1 -- 172.21.15.94:0/1264918453 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0694003d20 con 0x55ee98c302b0 2024-02-17T12:02:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.112+0000 7f06b3fad740 10 client.16659 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.112+0000 7f06b3fad740 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:22.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.112+0000 7f06b3fad740 20 client.16659 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:22.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.112+0000 7f06b3fad740 1 --2- 172.21.15.94:0/1264918453 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee98c49970 0x55ee98c4bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.112+0000 7f06b3fad740 4 client.16659 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:22.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.112+0000 7f06b3fad740 1 -- 172.21.15.94:0/1264918453 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee988d4720 con 0x55ee98c49970 2024-02-17T12:02:22.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.112+0000 7f06a3fff640 1 --2- 172.21.15.94:0/1264918453 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee98c49970 0x55ee98c4bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.113+0000 7f06a3fff640 1 --2- 172.21.15.94:0/1264918453 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee98c49970 0x55ee98c4bd80 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddec17a4765b772d server_cookie=d8cecd307ef0c7 in_seq=0 out_seq=0 2024-02-17T12:02:22.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.113+0000 7f069a7fc640 10 client.16659 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:22.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.114+0000 7f069a7fc640 1 -- 172.21.15.94:0/1264918453 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee988d4720 con 0x55ee98c49970 2024-02-17T12:02:22.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.114+0000 7f069a7fc640 10 client.16659 handle_command_reply: tid=0 2024-02-17T12:02:22.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.114+0000 7f06b3fad740 1 client.16659 shutdown 2024-02-17T12:02:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.114+0000 7f06b3fad740 1 -- 172.21.15.94:0/1264918453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 msgr2=0x55ee98c39200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:22.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.114+0000 7f06b3fad740 1 --2- 172.21.15.94:0/1264918453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c39200 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0694011760 tx=0x7f0694011c20 comp rx=0 tx=0).stop 2024-02-17T12:02:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.114+0000 7f06b3fad740 1 -- 172.21.15.94:0/1264918453 shutdown_connections 2024-02-17T12:02:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.114+0000 7f06b3fad740 1 --2- 172.21.15.94:0/1264918453 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee98c49970 0x55ee98c4bd80 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.114+0000 7f06b3fad740 1 --2- 172.21.15.94:0/1264918453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c397b0 0x55ee98c36cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.114+0000 7f06b3fad740 1 --2- 172.21.15.94:0/1264918453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c30bd0 0x55ee98c36590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.114+0000 7f069a7fc640 0 client.16659 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.114+0000 7f06b3fad740 1 --2- 172.21.15.94:0/1264918453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c39200 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.114+0000 7f06b3fad740 1 -- 172.21.15.94:0/1264918453 >> 172.21.15.94:0/1264918453 conn(0x55ee98c2a990 msgr2=0x55ee98c2c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.114+0000 7f06b3fad740 1 -- 172.21.15.94:0/1264918453 shutdown_connections 2024-02-17T12:02:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.115+0000 7f06b3fad740 1 -- 172.21.15.94:0/1264918453 wait complete. 2024-02-17T12:02:22.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.115+0000 7f06b3fad740 20 client.16659 trim_cache size 0 max 16384 2024-02-17T12:02:22.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.136+0000 7f06b3fad740 1 Processor -- start 2024-02-17T12:02:22.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.136+0000 7f06b3fad740 1 -- start start 2024-02-17T12:02:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.137+0000 7f06b3fad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c30fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.137+0000 7f06b3fad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 0x55ee98c40600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.137+0000 7f06b3fad740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c397b0 0x55ee98c40b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.137+0000 7f06b3fad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee98c2cd40 con 0x55ee98c302b0 2024-02-17T12:02:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.137+0000 7f06a3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 0x55ee98c40600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.137+0000 7f06a3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 0x55ee98c40600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55834/0 (socket says 172.21.15.94:55834) 2024-02-17T12:02:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.137+0000 7f06a3fff640 1 -- 172.21.15.94:0/2067365397 learned_addr learned my addr 172.21.15.94:0/2067365397 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.137+0000 7f06ab7fe640 1 --2- 172.21.15.94:0/2067365397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c397b0 0x55ee98c40b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.137+0000 7f06aaffd640 1 --2- 172.21.15.94:0/2067365397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c30fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.137+0000 7f06b3fad740 1 -- 172.21.15.94:0/2067365397 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee98c2fcf0 con 0x55ee98c397b0 2024-02-17T12:02:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.137+0000 7f06b3fad740 1 -- 172.21.15.94:0/2067365397 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee98c2a820 con 0x55ee98c30bd0 2024-02-17T12:02:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.138+0000 7f06a3fff640 1 -- 172.21.15.94:0/2067365397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c397b0 msgr2=0x55ee98c40b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:22.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.138+0000 7f06a3fff640 1 --2- 172.21.15.94:0/2067365397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c397b0 0x55ee98c40b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.138+0000 7f06a3fff640 1 -- 172.21.15.94:0/2067365397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 msgr2=0x55ee98c30fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:22.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.138+0000 7f06a3fff640 1 --2- 172.21.15.94:0/2067365397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c30fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.138+0000 7f06a3fff640 1 -- 172.21.15.94:0/2067365397 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee98b32ac0 con 0x55ee98c30bd0 2024-02-17T12:02:22.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.138+0000 7f06a3fff640 1 --2- 172.21.15.94:0/2067365397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 0x55ee98c40600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f069c09aaf0 tx=0x7f069c021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef5ba942181f7abc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:22.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.138+0000 7f06a0ff9640 1 -- 172.21.15.94:0/2067365397 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c09b4f0 con 0x55ee98c30bd0 2024-02-17T12:02:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.139+0000 7f06a0ff9640 1 -- 172.21.15.94:0/2067365397 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f069c09adf0 con 0x55ee98c30bd0 2024-02-17T12:02:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.139+0000 7f06a0ff9640 1 -- 172.21.15.94:0/2067365397 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069c0cbf40 con 0x55ee98c30bd0 2024-02-17T12:02:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.140+0000 7f06b3fad740 1 -- 172.21.15.94:0/2067365397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 msgr2=0x55ee98c40600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.140+0000 7f06b3fad740 1 --2- 172.21.15.94:0/2067365397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 0x55ee98c40600 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f069c09aaf0 tx=0x7f069c021000 comp rx=0 tx=0).stop 2024-02-17T12:02:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.140+0000 7f06b3fad740 1 -- 172.21.15.94:0/2067365397 shutdown_connections 2024-02-17T12:02:22.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.140+0000 7f06b3fad740 1 --2- 172.21.15.94:0/2067365397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c397b0 0x55ee98c40b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.140+0000 7f06b3fad740 1 --2- 172.21.15.94:0/2067365397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c30bd0 0x55ee98c40600 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.140+0000 7f06b3fad740 1 --2- 172.21.15.94:0/2067365397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c302b0 0x55ee98c30fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.140+0000 7f06b3fad740 1 -- 172.21.15.94:0/2067365397 >> 172.21.15.94:0/2067365397 conn(0x55ee98c2a990 msgr2=0x55ee98c50720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:22.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.140+0000 7f06b3fad740 1 -- 172.21.15.94:0/2067365397 shutdown_connections 2024-02-17T12:02:22.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.140+0000 7f06b3fad740 1 -- 172.21.15.94:0/2067365397 wait complete. 2024-02-17T12:02:22.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.140+0000 7f06b3fad740 1 -- start start 2024-02-17T12:02:22.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.141+0000 7f06b3fad740 1 Processor -- start 2024-02-17T12:02:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.141+0000 7f06b3fad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c302b0 0x55ee98c3f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06b3fad740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c30bd0 0x55ee98c3f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06aaffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c302b0 0x55ee98c3f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06aaffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c302b0 0x55ee98c3f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55850/0 (socket says 172.21.15.94:55850) 2024-02-17T12:02:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06aaffd640 1 -- 172.21.15.94:0/3400302504 learned_addr learned my addr 172.21.15.94:0/3400302504 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06b3fad740 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c397b0 0x55ee98c9feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06b3fad740 1 -- 172.21.15.94:0/3400302504 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee98c40050 con 0x55ee98c397b0 2024-02-17T12:02:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06b3fad740 1 -- 172.21.15.94:0/3400302504 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee98c401c0 con 0x55ee98c30bd0 2024-02-17T12:02:22.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06b3fad740 1 -- 172.21.15.94:0/3400302504 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee98ca03f0 con 0x55ee98c302b0 2024-02-17T12:02:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06b3fad740 10 client.0 _ll_register_callbacks cb 0x7f069c097090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06ab7fe640 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c397b0 0x55ee98c9feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06a3fff640 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c30bd0 0x55ee98c3f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06aaffd640 1 -- 172.21.15.94:0/3400302504 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c30bd0 msgr2=0x55ee98c3f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06aaffd640 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c30bd0 0x55ee98c3f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06aaffd640 1 -- 172.21.15.94:0/3400302504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c397b0 msgr2=0x55ee98c9feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06aaffd640 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c397b0 0x55ee98c9feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.142+0000 7f06aaffd640 1 -- 172.21.15.94:0/3400302504 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee98b56570 con 0x55ee98c302b0 2024-02-17T12:02:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.143+0000 7f06aaffd640 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c302b0 0x55ee98c3f350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f06940112f0 tx=0x7f0694011320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d37999d1dae0561d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.143+0000 7f069a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:02:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.143+0000 7f069a7fc640 1 -- 172.21.15.94:0/3400302504 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694016040 con 0x55ee98c302b0 2024-02-17T12:02:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.143+0000 7f069a7fc640 1 -- 172.21.15.94:0/3400302504 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0694000e40 con 0x55ee98c302b0 2024-02-17T12:02:22.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.143+0000 7f06b3fad740 1 -- 172.21.15.94:0/3400302504 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f069c005670 con 0x55ee98c302b0 2024-02-17T12:02:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.143+0000 7f069a7fc640 1 -- 172.21.15.94:0/3400302504 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0694005790 con 0x55ee98c302b0 2024-02-17T12:02:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.144+0000 7f06b3fad740 10 client.26173 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.144+0000 7f06b3fad740 1 -- 172.21.15.94:0/3400302504 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ee98b586a0 con 0x55ee98c302b0 2024-02-17T12:02:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.145+0000 7f069a7fc640 1 -- 172.21.15.94:0/3400302504 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f069c005670 con 0x55ee98c302b0 2024-02-17T12:02:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.145+0000 7f069a7fc640 1 -- 172.21.15.94:0/3400302504 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f06940035f0 con 0x55ee98c302b0 2024-02-17T12:02:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.145+0000 7f06b3fad740 10 client.26173 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:22.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.145+0000 7f06b3fad740 10 client.26173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:22.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.145+0000 7f06b3fad740 20 client.26173 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:22.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.145+0000 7f06b3fad740 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee98ca1bf0 0x55ee98ca4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:22.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.145+0000 7f06b3fad740 4 client.26173 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:02:22.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.145+0000 7f06b3fad740 1 -- 172.21.15.94:0/3400302504 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ee98923890 con 0x55ee98ca1bf0 2024-02-17T12:02:22.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.145+0000 7f06a3fff640 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee98ca1bf0 0x55ee98ca4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f06a3fff640 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee98ca1bf0 0x55ee98ca4010 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a58dad7f5ad1d4c server_cookie=8843b06386d63e94 in_seq=0 out_seq=0 2024-02-17T12:02:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f069a7fc640 10 client.26173 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f069a7fc640 1 -- 172.21.15.94:0/3400302504 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ee98923890 con 0x55ee98ca1bf0 2024-02-17T12:02:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f069a7fc640 10 client.26173 handle_command_reply: tid=0 2024-02-17T12:02:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f06b3fad740 1 client.26173 shutdown 2024-02-17T12:02:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f06b3fad740 1 -- 172.21.15.94:0/3400302504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c302b0 msgr2=0x55ee98c3f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f06b3fad740 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c302b0 0x55ee98c3f350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f06940112f0 tx=0x7f0694011320 comp rx=0 tx=0).stop 2024-02-17T12:02:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f06b3fad740 1 -- 172.21.15.94:0/3400302504 shutdown_connections 2024-02-17T12:02:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f06b3fad740 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee98ca1bf0 0x55ee98ca4010 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f06b3fad740 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee98c397b0 0x55ee98c9feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f06b3fad740 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee98c30bd0 0x55ee98c3f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f06b3fad740 1 --2- 172.21.15.94:0/3400302504 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee98c302b0 0x55ee98c3f350 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f06b3fad740 1 -- 172.21.15.94:0/3400302504 >> 172.21.15.94:0/3400302504 conn(0x55ee98c2a990 msgr2=0x55ee98c34d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:22.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.146+0000 7f069a7fc640 0 client.26173 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:22.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.147+0000 7f06b3fad740 1 -- 172.21.15.94:0/3400302504 shutdown_connections 2024-02-17T12:02:22.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.147+0000 7f06b3fad740 1 -- 172.21.15.94:0/3400302504 wait complete. 2024-02-17T12:02:22.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.147+0000 7f06b3fad740 20 client.26173 trim_cache size 0 max 16384 2024-02-17T12:02:22.148 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:02:22.148 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:02:22.148 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:02:22.149 INFO:teuthology.orchestra.run.smithi094.stdout: "84731426-86c7-48e6-848d-e54b5432ab8c": { 2024-02-17T12:02:22.149 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:02:22.149 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "84731426-86c7-48e6-848d-e54b5432ab8c", 2024-02-17T12:02:22.149 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:02:22.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:02:22.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:02:22.149 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:02:22.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.147+0000 7f06b12fb640 1 -- 172.21.15.94:0/3274709036 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f068c0a22b0 msgr2=0x7f068c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:22.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.147+0000 7f06b12fb640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f068c0a22b0 0x7f068c0a4770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f06a4004af0 tx=0x7f06a402bef0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.147+0000 7f06b12fb640 1 -- 172.21.15.94:0/3274709036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06ac100130 msgr2=0x7f06ac115a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:22.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.147+0000 7f06b12fb640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06ac100130 0x7f06ac115a10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f069c00e9f0 tx=0x7f069c00edf0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.148+0000 7f06b12fb640 1 -- 172.21.15.94:0/3274709036 shutdown_connections 2024-02-17T12:02:22.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.148+0000 7f06b12fb640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f068c0a22b0 0x7f068c0a4770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.148+0000 7f06b12fb640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06ac105780 0x7f06ac110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.148+0000 7f06b12fb640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f06ac100130 0x7f06ac115a10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.148+0000 7f06b12fb640 1 --2- 172.21.15.94:0/3274709036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f06ac0ff770 0x7f06ac1154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:22.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.148+0000 7f06b12fb640 1 -- 172.21.15.94:0/3274709036 >> 172.21.15.94:0/3274709036 conn(0x7f06ac0765b0 msgr2=0x7f06ac1024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:22.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.149+0000 7f06b12fb640 1 -- 172.21.15.94:0/3274709036 shutdown_connections 2024-02-17T12:02:22.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:22.149+0000 7f06b12fb640 1 -- 172.21.15.94:0/3274709036 wait complete. 2024-02-17T12:02:22.162 INFO:tasks.cephfs.filesystem:scrub status for tag:84731426-86c7-48e6-848d-e54b5432ab8c - {'path': '/', 'tag': '84731426-86c7-48e6-848d-e54b5432ab8c', 'options': 'recursive,force'} 2024-02-17T12:02:22.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:21 smithi125 ceph-mon[36819]: pgmap v130: 97 pgs: 97 active+clean; 3.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.2 MiB/s wr, 616 op/s 2024-02-17T12:02:22.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:21 smithi094 ceph-mon[32499]: pgmap v130: 97 pgs: 97 active+clean; 3.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.2 MiB/s wr, 616 op/s 2024-02-17T12:02:22.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:21 smithi189 ceph-mon[36588]: pgmap v130: 97 pgs: 97 active+clean; 3.1 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 7.2 MiB/s wr, 616 op/s 2024-02-17T12:02:23.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:22 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:02:23.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:22 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:02:23.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:22 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:02:23.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:22 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:02:23.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:22 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:02:23.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:22 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:02:23.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:22 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:02:23.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:22 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:02:23.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:22 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:02:23.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:22 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:02:23.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:22 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:02:23.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:22 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:02:23.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:22 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:02:23.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:22 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:02:23.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:22 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:02:24.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:23 smithi125 ceph-mon[36819]: pgmap v131: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 859 op/s 2024-02-17T12:02:24.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:23 smithi094 ceph-mon[32499]: pgmap v131: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 859 op/s 2024-02-17T12:02:24.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:23 smithi189 ceph-mon[36588]: pgmap v131: 97 pgs: 97 active+clean; 3.2 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 11 MiB/s wr, 859 op/s 2024-02-17T12:02:25.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:24 smithi094 ceph-mon[32499]: pgmap v132: 97 pgs: 97 active+clean; 3.2 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 11 MiB/s wr, 909 op/s 2024-02-17T12:02:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 245.0 s, 3384.2 tps, lat 29.287 ms stddev 33.547 2024-02-17T12:02:25.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:24 smithi189 ceph-mon[36588]: pgmap v132: 97 pgs: 97 active+clean; 3.2 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 11 MiB/s wr, 909 op/s 2024-02-17T12:02:25.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:24 smithi125 ceph-mon[36819]: pgmap v132: 97 pgs: 97 active+clean; 3.2 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 11 MiB/s wr, 909 op/s 2024-02-17T12:02:28.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:27 smithi189 ceph-mon[36588]: pgmap v133: 97 pgs: 97 active+clean; 3.2 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 8.8 MiB/s wr, 732 op/s 2024-02-17T12:02:28.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:27 smithi125 ceph-mon[36819]: pgmap v133: 97 pgs: 97 active+clean; 3.2 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 8.8 MiB/s wr, 732 op/s 2024-02-17T12:02:28.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:27 smithi094 ceph-mon[32499]: pgmap v133: 97 pgs: 97 active+clean; 3.2 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 8.8 MiB/s wr, 732 op/s 2024-02-17T12:02:30.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:29 smithi189 ceph-mon[36588]: pgmap v134: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2024-02-17T12:02:30.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:29 smithi125 ceph-mon[36819]: pgmap v134: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2024-02-17T12:02:30.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:29 smithi094 ceph-mon[32499]: pgmap v134: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 13 MiB/s wr, 1.10k op/s 2024-02-17T12:02:30.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 250.0 s, 3253.2 tps, lat 30.929 ms stddev 34.765 2024-02-17T12:02:32.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:31 smithi189 ceph-mon[36588]: pgmap v135: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 9.1 MiB/s wr, 796 op/s 2024-02-17T12:02:32.184 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:31 smithi125 ceph-mon[36819]: pgmap v135: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 9.1 MiB/s wr, 796 op/s 2024-02-17T12:02:32.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:31 smithi094 ceph-mon[32499]: pgmap v135: 97 pgs: 97 active+clean; 3.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 9.1 MiB/s wr, 796 op/s 2024-02-17T12:02:34.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:33 smithi189 ceph-mon[36588]: pgmap v136: 97 pgs: 97 active+clean; 3.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 981 op/s 2024-02-17T12:02:34.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:33 smithi125 ceph-mon[36819]: pgmap v136: 97 pgs: 97 active+clean; 3.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 981 op/s 2024-02-17T12:02:34.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:33 smithi094 ceph-mon[32499]: pgmap v136: 97 pgs: 97 active+clean; 3.2 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 981 op/s 2024-02-17T12:02:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.890+0000 7f14ad365640 1 -- 172.21.15.94:0/2652333057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14a81002d0 msgr2=0x7f14a8100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.890+0000 7f14ad365640 1 --2- 172.21.15.94:0/2652333057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14a81002d0 0x7f14a8100750 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f149c009900 tx=0x7f149c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:02:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.890+0000 7f14ad365640 1 -- 172.21.15.94:0/2652333057 shutdown_connections 2024-02-17T12:02:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.890+0000 7f14ad365640 1 --2- 172.21.15.94:0/2652333057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14a8100c90 0x7f14a810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.890+0000 7f14ad365640 1 --2- 172.21.15.94:0/2652333057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14a81002d0 0x7f14a8100750 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.890+0000 7f14ad365640 1 --2- 172.21.15.94:0/2652333057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a8105020 0x7f14a8105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.890+0000 7f14ad365640 1 -- 172.21.15.94:0/2652333057 >> 172.21.15.94:0/2652333057 conn(0x7f14a8076ed0 msgr2=0x7f14a80772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.890+0000 7f14ad365640 1 -- 172.21.15.94:0/2652333057 shutdown_connections 2024-02-17T12:02:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.890+0000 7f14ad365640 1 -- 172.21.15.94:0/2652333057 wait complete. 2024-02-17T12:02:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.891+0000 7f14ad365640 1 Processor -- start 2024-02-17T12:02:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.891+0000 7f14ad365640 1 -- start start 2024-02-17T12:02:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14ad365640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a81002d0 0x7f14a8197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14ad365640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14a8100c90 0x7f14a8197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14ad365640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14a8105020 0x7f14a819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14ad365640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a819c610 con 0x7f14a8100c90 2024-02-17T12:02:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14ad365640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a819c780 con 0x7f14a8105020 2024-02-17T12:02:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14ad365640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a819c940 con 0x7f14a81002d0 2024-02-17T12:02:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14a6ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a81002d0 0x7f14a8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14a77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14a8105020 0x7f14a819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14a6ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a81002d0 0x7f14a8197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55050/0 (socket says 172.21.15.94:55050) 2024-02-17T12:02:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14a6ffd640 1 -- 172.21.15.94:0/1651291658 learned_addr learned my addr 172.21.15.94:0/1651291658 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14a67fc640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14a8100c90 0x7f14a8197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14a77fe640 1 -- 172.21.15.94:0/1651291658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a81002d0 msgr2=0x7f14a8197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14a77fe640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a81002d0 0x7f14a8197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14a77fe640 1 -- 172.21.15.94:0/1651291658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14a8100c90 msgr2=0x7f14a8197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14a77fe640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14a8100c90 0x7f14a8197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14a77fe640 1 -- 172.21.15.94:0/1651291658 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c009560 con 0x7f14a8105020 2024-02-17T12:02:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14a6ffd640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a81002d0 0x7f14a8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f14a77fe640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14a8105020 0x7f14a819c010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f14a0005fd0 tx=0x7f14a000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.892+0000 7f148ffff640 1 -- 172.21.15.94:0/1651291658 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a000cd80 con 0x7f14a8105020 2024-02-17T12:02:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.893+0000 7f148ffff640 1 -- 172.21.15.94:0/1651291658 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f14a0005020 con 0x7f14a8105020 2024-02-17T12:02:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.893+0000 7f148ffff640 1 -- 172.21.15.94:0/1651291658 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0010690 con 0x7f14a8105020 2024-02-17T12:02:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.893+0000 7f14ad365640 1 -- 172.21.15.94:0/1651291658 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a819cc20 con 0x7f14a8105020 2024-02-17T12:02:34.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.893+0000 7f14ad365640 1 -- 172.21.15.94:0/1651291658 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a8101e00 con 0x7f14a8105020 2024-02-17T12:02:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.895+0000 7f14b0013740 1 Processor -- start 2024-02-17T12:02:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.895+0000 7f14b0013740 1 -- start start 2024-02-17T12:02:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.895+0000 7f14b0013740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555688832ad0 0x555688831820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.895+0000 7f14b0013740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688831d60 0x5556888321e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14b0013740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55568883b2e0 0x55568883d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14b0013740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556887a09f0 con 0x55568883b2e0 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14b0013740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568879df20 con 0x555688831d60 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14b0013740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555688799700 con 0x555688832ad0 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14a67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55568883b2e0 0x55568883d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14a6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688831d60 0x5556888321e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14a67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55568883b2e0 0x55568883d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35744/0 (socket says 172.21.15.94:35744) 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14a67fc640 1 -- 172.21.15.94:0/272401741 learned_addr learned my addr 172.21.15.94:0/272401741 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14a67fc640 1 --2- 172.21.15.94:0/272401741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555688832ad0 0x555688831820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14a6ffd640 1 -- 172.21.15.94:0/272401741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555688832ad0 msgr2=0x555688831820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14a6ffd640 1 --2- 172.21.15.94:0/272401741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555688832ad0 0x555688831820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14a6ffd640 1 -- 172.21.15.94:0/272401741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55568883b2e0 msgr2=0x55568883d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14a6ffd640 1 --2- 172.21.15.94:0/272401741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55568883b2e0 0x55568883d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14a6ffd640 1 -- 172.21.15.94:0/272401741 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55568872f7b0 con 0x555688831d60 2024-02-17T12:02:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14a67fc640 1 --2- 172.21.15.94:0/272401741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55568883b2e0 0x55568883d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f14a6ffd640 1 --2- 172.21.15.94:0/272401741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688831d60 0x5556888321e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1498011510 tx=0x7f14980119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aab07b0725fae14c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f148d7fa640 1 -- 172.21.15.94:0/272401741 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149800f560 con 0x555688831d60 2024-02-17T12:02:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f148d7fa640 1 -- 172.21.15.94:0/272401741 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f149800fba0 con 0x555688831d60 2024-02-17T12:02:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.896+0000 7f148d7fa640 1 -- 172.21.15.94:0/272401741 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498019cd0 con 0x555688831d60 2024-02-17T12:02:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.897+0000 7f14b0013740 1 -- 172.21.15.94:0/272401741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688831d60 msgr2=0x5556888321e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.897+0000 7f14b0013740 1 --2- 172.21.15.94:0/272401741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688831d60 0x5556888321e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1498011510 tx=0x7f14980119d0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.899+0000 7f14b0013740 1 -- 172.21.15.94:0/272401741 shutdown_connections 2024-02-17T12:02:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.899+0000 7f14b0013740 1 --2- 172.21.15.94:0/272401741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55568883b2e0 0x55568883d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.899+0000 7f148ffff640 1 -- 172.21.15.94:0/1651291658 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f14a0005500 con 0x7f14a8105020 2024-02-17T12:02:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.899+0000 7f14b0013740 1 --2- 172.21.15.94:0/272401741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688831d60 0x5556888321e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.899+0000 7f14b0013740 1 --2- 172.21.15.94:0/272401741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555688832ad0 0x555688831820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.899+0000 7f14b0013740 1 -- 172.21.15.94:0/272401741 >> 172.21.15.94:0/272401741 conn(0x5556887f9550 msgr2=0x55568882ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.899+0000 7f14b0013740 1 -- 172.21.15.94:0/272401741 shutdown_connections 2024-02-17T12:02:34.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.899+0000 7f14b0013740 1 -- 172.21.15.94:0/272401741 wait complete. 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.899+0000 7f14b0013740 1 -- start start 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.899+0000 7f14b0013740 1 Processor -- start 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14b0013740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688831d60 0x555688838340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14b0013740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 0x555688838880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14b0013740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55568883b2e0 0x555688838dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14b0013740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568873ebf0 con 0x555688831d60 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14b0013740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568846d250 con 0x555688832ad0 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14b0013740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55568871f200 con 0x55568883b2e0 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14b0013740 10 client.0 _ll_register_callbacks cb 0x55568878dd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f148ffff640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f14900a22e0 0x7f14900a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14a6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 0x555688838880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14a6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 0x555688838880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:39330/0 (socket says 172.21.15.94:39330) 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14a6ffd640 1 -- 172.21.15.94:0/3262430690 learned_addr learned my addr 172.21.15.94:0/3262430690 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14a67fc640 1 --2- 172.21.15.94:0/3262430690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688831d60 0x555688838340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f148ffff640 1 -- 172.21.15.94:0/1651291658 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f14a0014070 con 0x7f14a8105020 2024-02-17T12:02:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14a67fc640 1 --2- 172.21.15.94:0/3262430690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55568883b2e0 0x555688838dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14a77fe640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f14900a22e0 0x7f14900a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14a6ffd640 1 -- 172.21.15.94:0/3262430690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55568883b2e0 msgr2=0x555688838dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14a6ffd640 1 --2- 172.21.15.94:0/3262430690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55568883b2e0 0x555688838dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14a6ffd640 1 -- 172.21.15.94:0/3262430690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688831d60 msgr2=0x555688838340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14a6ffd640 1 --2- 172.21.15.94:0/3262430690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688831d60 0x555688838340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.900+0000 7f14a6ffd640 1 -- 172.21.15.94:0/3262430690 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1498009d10 con 0x555688832ad0 2024-02-17T12:02:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f14a6ffd640 1 --2- 172.21.15.94:0/3262430690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 0x555688838880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f14980196e0 tx=0x7f1498019710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de493d80bba83eca server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f148aff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f14a77fe640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f14900a22e0 0x7f14900a47a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009fd0 tx=0x7f14a00bc040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f148aff5640 1 -- 172.21.15.94:0/3262430690 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149801adf0 con 0x555688832ad0 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f14b0013740 1 -- 172.21.15.94:0/3262430690 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555688499600 con 0x555688832ad0 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f148aff5640 1 -- 172.21.15.94:0/3262430690 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f149800f560 con 0x555688832ad0 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f148aff5640 1 -- 172.21.15.94:0/3262430690 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f149800f720 con 0x555688832ad0 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f14b0013740 10 client.26132 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f14b0013740 1 -- 172.21.15.94:0/3262430690 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555688797d80 con 0x555688832ad0 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f148aff5640 1 -- 172.21.15.94:0/3262430690 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555688499600 con 0x555688832ad0 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f148aff5640 1 -- 172.21.15.94:0/3262430690 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1498004b00 con 0x555688832ad0 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f14b0013740 10 client.26132 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f14b0013740 10 client.26132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f14b0013740 20 client.26132 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f14b0013740 1 --2- 172.21.15.94:0/3262430690 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55568884b5b0 0x55568884d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f14b0013740 4 client.26132 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f14b0013740 1 -- 172.21.15.94:0/3262430690 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556884ea1b0 con 0x55568884b5b0 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.901+0000 7f14a67fc640 1 --2- 172.21.15.94:0/3262430690 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55568884b5b0 0x55568884d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.902+0000 7f148aff5640 10 client.26132 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.902+0000 7f14a67fc640 1 --2- 172.21.15.94:0/3262430690 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55568884b5b0 0x55568884d9d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a823b2bae0f89c server_cookie=5c7d086237703971 in_seq=0 out_seq=0 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.902+0000 7f148aff5640 1 -- 172.21.15.94:0/3262430690 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f149c04da90 con 0x55568884b5b0 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.902+0000 7f148aff5640 10 client.26132 handle_command_reply: tid=0 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.902+0000 7f14b0013740 1 client.26132 shutdown 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.903+0000 7f14b0013740 1 -- 172.21.15.94:0/3262430690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 msgr2=0x555688838880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.903+0000 7f14b0013740 1 --2- 172.21.15.94:0/3262430690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 0x555688838880 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f14980196e0 tx=0x7f1498019710 comp rx=0 tx=0).stop 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.903+0000 7f14b0013740 1 -- 172.21.15.94:0/3262430690 shutdown_connections 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.903+0000 7f14b0013740 1 --2- 172.21.15.94:0/3262430690 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55568884b5b0 0x55568884d9d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.903+0000 7f14b0013740 1 --2- 172.21.15.94:0/3262430690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55568883b2e0 0x555688838dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.903+0000 7f14b0013740 1 --2- 172.21.15.94:0/3262430690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 0x555688838880 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.903+0000 7f148aff5640 0 client.26132 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.903+0000 7f14b0013740 1 --2- 172.21.15.94:0/3262430690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688831d60 0x555688838340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.903+0000 7f14b0013740 1 -- 172.21.15.94:0/3262430690 >> 172.21.15.94:0/3262430690 conn(0x5556887f9550 msgr2=0x55568882e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.903+0000 7f14b0013740 1 -- 172.21.15.94:0/3262430690 shutdown_connections 2024-02-17T12:02:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.903+0000 7f14b0013740 1 -- 172.21.15.94:0/3262430690 wait complete. 2024-02-17T12:02:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.903+0000 7f14b0013740 20 client.26132 trim_cache size 0 max 16384 2024-02-17T12:02:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.918+0000 7f14b0013740 1 Processor -- start 2024-02-17T12:02:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.918+0000 7f14b0013740 1 -- start start 2024-02-17T12:02:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.918+0000 7f14b0013740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555688831d60 0x55568883aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.918+0000 7f14b0013740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688832ad0 0x555688838be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.918+0000 7f14b0013740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55568883b2e0 0x555688842100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.918+0000 7f14b0013740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555688831030 con 0x555688832ad0 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.918+0000 7f14b0013740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555688828310 con 0x55568883b2e0 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.918+0000 7f14b0013740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555688836890 con 0x555688831d60 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.918+0000 7f14a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688832ad0 0x555688838be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.918+0000 7f14a6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688832ad0 0x555688838be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35768/0 (socket says 172.21.15.94:35768) 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.919+0000 7f14a6ffd640 1 -- 172.21.15.94:0/740357921 learned_addr learned my addr 172.21.15.94:0/740357921 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.919+0000 7f14a67fc640 1 --2- 172.21.15.94:0/740357921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55568883b2e0 0x555688842100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.919+0000 7f14a67fc640 1 --2- 172.21.15.94:0/740357921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555688831d60 0x55568883aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.920+0000 7f14a6ffd640 1 -- 172.21.15.94:0/740357921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555688831d60 msgr2=0x55568883aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.920+0000 7f14a6ffd640 1 --2- 172.21.15.94:0/740357921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555688831d60 0x55568883aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.920+0000 7f14a6ffd640 1 -- 172.21.15.94:0/740357921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55568883b2e0 msgr2=0x555688842100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.920+0000 7f14a6ffd640 1 --2- 172.21.15.94:0/740357921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55568883b2e0 0x555688842100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.920+0000 7f14a6ffd640 1 -- 172.21.15.94:0/740357921 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556887347b0 con 0x555688832ad0 2024-02-17T12:02:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.920+0000 7f14a67fc640 1 --2- 172.21.15.94:0/740357921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55568883b2e0 0x555688842100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.920+0000 7f14a6ffd640 1 --2- 172.21.15.94:0/740357921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688832ad0 0x555688838be0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1498011a10 tx=0x7f1498009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516c1981da4e3fb2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.920+0000 7f148cff9640 1 -- 172.21.15.94:0/740357921 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149800f760 con 0x555688832ad0 2024-02-17T12:02:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.920+0000 7f148cff9640 1 -- 172.21.15.94:0/740357921 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1498019e00 con 0x555688832ad0 2024-02-17T12:02:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.920+0000 7f148cff9640 1 -- 172.21.15.94:0/740357921 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14980048e0 con 0x555688832ad0 2024-02-17T12:02:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.921+0000 7f14b0013740 1 -- 172.21.15.94:0/740357921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688832ad0 msgr2=0x555688838be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.921+0000 7f14b0013740 1 --2- 172.21.15.94:0/740357921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688832ad0 0x555688838be0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1498011a10 tx=0x7f1498009c30 comp rx=0 tx=0).stop 2024-02-17T12:02:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.921+0000 7f14b0013740 1 -- 172.21.15.94:0/740357921 shutdown_connections 2024-02-17T12:02:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.921+0000 7f14b0013740 1 --2- 172.21.15.94:0/740357921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55568883b2e0 0x555688842100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.921+0000 7f14b0013740 1 --2- 172.21.15.94:0/740357921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688832ad0 0x555688838be0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.921+0000 7f14b0013740 1 --2- 172.21.15.94:0/740357921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555688831d60 0x55568883aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.921+0000 7f14b0013740 1 -- 172.21.15.94:0/740357921 >> 172.21.15.94:0/740357921 conn(0x5556887f9550 msgr2=0x555688836c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.921+0000 7f14b0013740 1 -- 172.21.15.94:0/740357921 shutdown_connections 2024-02-17T12:02:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.921+0000 7f14b0013740 1 -- 172.21.15.94:0/740357921 wait complete. 2024-02-17T12:02:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14b0013740 1 -- start start 2024-02-17T12:02:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14b0013740 1 Processor -- start 2024-02-17T12:02:34.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14b0013740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688831d60 0x555688834f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14b0013740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 0x5556888354a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14b0013740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55568883b2e0 0x5556888359e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14b0013740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556888a1a90 con 0x555688831d60 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14b0013740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556888a1bd0 con 0x555688832ad0 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14b0013740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556888a1d10 con 0x55568883b2e0 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14b0013740 10 client.0 _ll_register_callbacks cb 0x7f1498009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14a6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 0x5556888354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14a67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688831d60 0x555688834f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14a6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 0x5556888354a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:39344/0 (socket says 172.21.15.94:39344) 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14a6ffd640 1 -- 172.21.15.94:0/1987795288 learned_addr learned my addr 172.21.15.94:0/1987795288 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.922+0000 7f14a67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688831d60 0x555688834f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35774/0 (socket says 172.21.15.94:35774) 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f14a67fc640 1 --2- 172.21.15.94:0/1987795288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55568883b2e0 0x5556888359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f14a6ffd640 1 -- 172.21.15.94:0/1987795288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55568883b2e0 msgr2=0x5556888359e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:34.927 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f14a6ffd640 1 --2- 172.21.15.94:0/1987795288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55568883b2e0 0x5556888359e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f14a6ffd640 1 -- 172.21.15.94:0/1987795288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688831d60 msgr2=0x555688834f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f14a6ffd640 1 --2- 172.21.15.94:0/1987795288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688831d60 0x555688834f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f14a6ffd640 1 -- 172.21.15.94:0/1987795288 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c030070 con 0x555688832ad0 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f14a67fc640 1 --2- 172.21.15.94:0/1987795288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688831d60 0x555688834f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f14a6ffd640 1 --2- 172.21.15.94:0/1987795288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 0x5556888354a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1498011460 tx=0x7f1498025470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=388147dc7d25587f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f148aff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f148aff5640 1 -- 172.21.15.94:0/1987795288 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14980198e0 con 0x555688832ad0 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f14b0013740 1 -- 172.21.15.94:0/1987795288 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f149800fba0 con 0x555688832ad0 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f148aff5640 1 -- 172.21.15.94:0/1987795288 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1498000e40 con 0x555688832ad0 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f148aff5640 1 -- 172.21.15.94:0/1987795288 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f149800f930 con 0x555688832ad0 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f14b0013740 10 client.26138 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f14b0013740 1 -- 172.21.15.94:0/1987795288 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555688797d80 con 0x555688832ad0 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.923+0000 7f148aff5640 1 -- 172.21.15.94:0/1987795288 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f149800fba0 con 0x555688832ad0 2024-02-17T12:02:34.928 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.924+0000 7f148aff5640 1 -- 172.21.15.94:0/1987795288 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f149803e020 con 0x555688832ad0 2024-02-17T12:02:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.924+0000 7f14b0013740 10 client.26138 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.924+0000 7f14b0013740 10 client.26138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.924+0000 7f14b0013740 20 client.26138 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.924+0000 7f14b0013740 1 --2- 172.21.15.94:0/1987795288 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5556888a2f10 0x5556888a5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.924+0000 7f14b0013740 4 client.26138 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:02:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.924+0000 7f14b0013740 1 -- 172.21.15.94:0/1987795288 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5556884d6550 con 0x5556888a2f10 2024-02-17T12:02:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.924+0000 7f14a67fc640 1 --2- 172.21.15.94:0/1987795288 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5556888a2f10 0x5556888a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.924+0000 7f148aff5640 10 client.26138 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.924+0000 7f14a67fc640 1 --2- 172.21.15.94:0/1987795288 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5556888a2f10 0x5556888a5330 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6057cd3147630e3b server_cookie=d700f36e22db14d3 in_seq=0 out_seq=0 2024-02-17T12:02:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f148aff5640 1 -- 172.21.15.94:0/1987795288 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5556884d6550 con 0x5556888a2f10 2024-02-17T12:02:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f148aff5640 10 client.26138 handle_command_reply: tid=0 2024-02-17T12:02:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f14b0013740 1 client.26138 shutdown 2024-02-17T12:02:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f14b0013740 1 -- 172.21.15.94:0/1987795288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 msgr2=0x5556888354a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f14b0013740 1 --2- 172.21.15.94:0/1987795288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 0x5556888354a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1498011460 tx=0x7f1498025470 comp rx=0 tx=0).stop 2024-02-17T12:02:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f14b0013740 1 -- 172.21.15.94:0/1987795288 shutdown_connections 2024-02-17T12:02:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f14b0013740 1 --2- 172.21.15.94:0/1987795288 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5556888a2f10 0x5556888a5330 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f14b0013740 1 --2- 172.21.15.94:0/1987795288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55568883b2e0 0x5556888359e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f148aff5640 0 client.26138 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f14b0013740 1 --2- 172.21.15.94:0/1987795288 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555688832ad0 0x5556888354a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f14b0013740 1 --2- 172.21.15.94:0/1987795288 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555688831d60 0x555688834f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f14b0013740 1 -- 172.21.15.94:0/1987795288 >> 172.21.15.94:0/1987795288 conn(0x5556887f9550 msgr2=0x55568882b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f14b0013740 1 -- 172.21.15.94:0/1987795288 shutdown_connections 2024-02-17T12:02:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f14b0013740 1 -- 172.21.15.94:0/1987795288 wait complete. 2024-02-17T12:02:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.946+0000 7f14b0013740 20 client.26138 trim_cache size 0 max 16384 2024-02-17T12:02:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:02:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:02:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:02:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:02:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.947+0000 7f14ad365640 1 -- 172.21.15.94:0/1651291658 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f14900a22e0 msgr2=0x7f14900a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.947+0000 7f14ad365640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f14900a22e0 0x7f14900a47a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009fd0 tx=0x7f14a00bc040 comp rx=0 tx=0).stop 2024-02-17T12:02:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.947+0000 7f14ad365640 1 -- 172.21.15.94:0/1651291658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14a8105020 msgr2=0x7f14a819c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.947+0000 7f14ad365640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14a8105020 0x7f14a819c010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f14a0005fd0 tx=0x7f14a000ee60 comp rx=0 tx=0).stop 2024-02-17T12:02:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.947+0000 7f14ad365640 1 -- 172.21.15.94:0/1651291658 shutdown_connections 2024-02-17T12:02:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.947+0000 7f14ad365640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f14900a22e0 0x7f14900a47a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.947+0000 7f14ad365640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f14a8105020 0x7f14a819c010 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.947+0000 7f14ad365640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f14a8100c90 0x7f14a8197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.947+0000 7f14ad365640 1 --2- 172.21.15.94:0/1651291658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14a81002d0 0x7f14a8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.947+0000 7f14ad365640 1 -- 172.21.15.94:0/1651291658 >> 172.21.15.94:0/1651291658 conn(0x7f14a8076ed0 msgr2=0x7f14a8103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.947+0000 7f14ad365640 1 -- 172.21.15.94:0/1651291658 shutdown_connections 2024-02-17T12:02:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:34.947+0000 7f14ad365640 1 -- 172.21.15.94:0/1651291658 wait complete. 2024-02-17T12:02:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 255.0 s, 2430.2 tps, lat 40.845 ms stddev 44.625 2024-02-17T12:02:36.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:35 smithi189 ceph-mon[36588]: pgmap v137: 97 pgs: 97 active+clean; 3.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.7 MiB/s wr, 846 op/s 2024-02-17T12:02:36.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:35 smithi125 ceph-mon[36819]: pgmap v137: 97 pgs: 97 active+clean; 3.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.7 MiB/s wr, 846 op/s 2024-02-17T12:02:36.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:35 smithi094 ceph-mon[32499]: pgmap v137: 97 pgs: 97 active+clean; 3.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 8.7 MiB/s wr, 846 op/s 2024-02-17T12:02:38.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:37 smithi189 ceph-mon[36588]: pgmap v138: 97 pgs: 97 active+clean; 3.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.1 MiB/s wr, 664 op/s 2024-02-17T12:02:38.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:37 smithi125 ceph-mon[36819]: pgmap v138: 97 pgs: 97 active+clean; 3.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.1 MiB/s wr, 664 op/s 2024-02-17T12:02:38.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:37 smithi094 ceph-mon[32499]: pgmap v138: 97 pgs: 97 active+clean; 3.2 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.1 MiB/s wr, 664 op/s 2024-02-17T12:02:40.151 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:39 smithi189 ceph-mon[36588]: pgmap v139: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.8 MiB/s wr, 952 op/s 2024-02-17T12:02:40.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:39 smithi125 ceph-mon[36819]: pgmap v139: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.8 MiB/s wr, 952 op/s 2024-02-17T12:02:40.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:39 smithi094 ceph-mon[32499]: pgmap v139: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 9.8 MiB/s wr, 952 op/s 2024-02-17T12:02:40.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 260.0 s, 2628.0 tps, lat 38.391 ms stddev 42.080 2024-02-17T12:02:42.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:41 smithi125 ceph-mon[36819]: pgmap v140: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.6 MiB/s wr, 580 op/s 2024-02-17T12:02:42.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:41 smithi094 ceph-mon[32499]: pgmap v140: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.6 MiB/s wr, 580 op/s 2024-02-17T12:02:42.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:41 smithi189 ceph-mon[36588]: pgmap v140: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 5.6 MiB/s wr, 580 op/s 2024-02-17T12:02:44.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:43 smithi125 ceph-mon[36819]: pgmap v141: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.3 MiB/s wr, 671 op/s 2024-02-17T12:02:44.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:43 smithi094 ceph-mon[32499]: pgmap v141: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.3 MiB/s wr, 671 op/s 2024-02-17T12:02:44.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:43 smithi189 ceph-mon[36588]: pgmap v141: 97 pgs: 97 active+clean; 3.3 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.3 MiB/s wr, 671 op/s 2024-02-17T12:02:45.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 265.0 s, 2789.0 tps, lat 35.781 ms stddev 38.868 2024-02-17T12:02:46.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:45 smithi125 ceph-mon[36819]: pgmap v142: 97 pgs: 97 active+clean; 3.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 683 op/s 2024-02-17T12:02:46.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:45 smithi094 ceph-mon[32499]: pgmap v142: 97 pgs: 97 active+clean; 3.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 683 op/s 2024-02-17T12:02:46.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:45 smithi189 ceph-mon[36588]: pgmap v142: 97 pgs: 97 active+clean; 3.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 6.4 MiB/s wr, 683 op/s 2024-02-17T12:02:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.077+0000 7f84eb963640 1 -- 172.21.15.94:0/431736501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84e4106470 msgr2=0x7f84e410e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.077+0000 7f84eb963640 1 --2- 172.21.15.94:0/431736501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84e4106470 0x7f84e410e040 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009930 tx=0x7f84dc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:02:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.078+0000 7f84eb963640 1 -- 172.21.15.94:0/431736501 shutdown_connections 2024-02-17T12:02:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.078+0000 7f84eb963640 1 --2- 172.21.15.94:0/431736501 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84e4106470 0x7f84e410e040 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.078+0000 7f84eb963640 1 --2- 172.21.15.94:0/431736501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84e4101d50 0x7f84e4105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.078+0000 7f84eb963640 1 --2- 172.21.15.94:0/431736501 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84e41013a0 0x7f84e4101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.078+0000 7f84eb963640 1 -- 172.21.15.94:0/431736501 >> 172.21.15.94:0/431736501 conn(0x7f84e40fcc40 msgr2=0x7f84e40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:48.079 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.078+0000 7f84eb963640 1 -- 172.21.15.94:0/431736501 shutdown_connections 2024-02-17T12:02:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.078+0000 7f84eb963640 1 -- 172.21.15.94:0/431736501 wait complete. 2024-02-17T12:02:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.079+0000 7f84eb963640 1 Processor -- start 2024-02-17T12:02:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.079+0000 7f84eb963640 1 -- start start 2024-02-17T12:02:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.079+0000 7f84eb963640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84e41013a0 0x7f84e419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.079+0000 7f84eb963640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84e4106470 0x7f84e419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.079+0000 7f84eb963640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84e4197600 0x7f84e4197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.079+0000 7f84eb963640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e4197ff0 con 0x7f84e4106470 2024-02-17T12:02:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.079+0000 7f84eb963640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e4198160 con 0x7f84e41013a0 2024-02-17T12:02:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.079+0000 7f84eb963640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e4198320 con 0x7f84e4197600 2024-02-17T12:02:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.079+0000 7f84e8efe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84e4106470 0x7f84e419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.080 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.079+0000 7f84e96ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84e41013a0 0x7f84e419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.079+0000 7f84e8efe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84e4106470 0x7f84e419d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38352/0 (socket says 172.21.15.94:38352) 2024-02-17T12:02:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.079+0000 7f84e8efe640 1 -- 172.21.15.94:0/875806866 learned_addr learned my addr 172.21.15.94:0/875806866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.079+0000 7f84e9f00640 1 --2- 172.21.15.94:0/875806866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84e4197600 0x7f84e4197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.080+0000 7f84e8efe640 1 -- 172.21.15.94:0/875806866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84e4197600 msgr2=0x7f84e4197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.080+0000 7f84e8efe640 1 --2- 172.21.15.94:0/875806866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84e4197600 0x7f84e4197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.080+0000 7f84e8efe640 1 -- 172.21.15.94:0/875806866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84e41013a0 msgr2=0x7f84e419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.080+0000 7f84e8efe640 1 --2- 172.21.15.94:0/875806866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84e41013a0 0x7f84e419d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.080+0000 7f84e8efe640 1 -- 172.21.15.94:0/875806866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc009560 con 0x7f84e4106470 2024-02-17T12:02:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.080+0000 7f84e8efe640 1 --2- 172.21.15.94:0/875806866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84e4106470 0x7f84e419d940 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009f10 tx=0x7f84d400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.081+0000 7f84da7fc640 1 -- 172.21.15.94:0/875806866 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d400ccc0 con 0x7f84e4106470 2024-02-17T12:02:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.081+0000 7f84da7fc640 1 -- 172.21.15.94:0/875806866 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84d400ce20 con 0x7f84e4106470 2024-02-17T12:02:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.081+0000 7f84da7fc640 1 -- 172.21.15.94:0/875806866 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d40047a0 con 0x7f84e4106470 2024-02-17T12:02:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.081+0000 7f84eb963640 1 -- 172.21.15.94:0/875806866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e40733b0 con 0x7f84e4106470 2024-02-17T12:02:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.081+0000 7f84eb963640 1 -- 172.21.15.94:0/875806866 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e4073870 con 0x7f84e4106470 2024-02-17T12:02:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.084+0000 7f84da7fc640 1 -- 172.21.15.94:0/875806866 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f84d4013140 con 0x7f84e4106470 2024-02-17T12:02:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.084+0000 7f84ee60c740 1 Processor -- start 2024-02-17T12:02:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84ee60c740 1 -- start start 2024-02-17T12:02:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84ee60c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124ff6ad0 0x559124ff5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84ee60c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff5d60 0x559124ff61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84ee60c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124fff2e0 0x5591250016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84ee60c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559124f649f0 con 0x559124ff6ad0 2024-02-17T12:02:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84ee60c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559124f61f20 con 0x559124fff2e0 2024-02-17T12:02:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84ee60c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559124f5d700 con 0x559124ff5d60 2024-02-17T12:02:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84e9f00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124ff6ad0 0x559124ff5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84da7fc640 1 --2- 172.21.15.94:0/875806866 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f84d00a2430 0x7f84d00a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84e96ff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff5d60 0x559124ff61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84e9f00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124ff6ad0 0x559124ff5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38362/0 (socket says 172.21.15.94:38362) 2024-02-17T12:02:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84e8efe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124fff2e0 0x5591250016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84e9f00640 1 -- 172.21.15.94:0/707551930 learned_addr learned my addr 172.21.15.94:0/707551930 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:48.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84e9f00640 1 --2- 172.21.15.94:0/875806866 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f84d00a2430 0x7f84d00a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84da7fc640 1 -- 172.21.15.94:0/875806866 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f84d40ce160 con 0x7f84e4106470 2024-02-17T12:02:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84e96ff640 1 -- 172.21.15.94:0/707551930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124fff2e0 msgr2=0x5591250016d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.085+0000 7f84e96ff640 1 --2- 172.21.15.94:0/707551930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124fff2e0 0x5591250016d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.086+0000 7f84e96ff640 1 -- 172.21.15.94:0/707551930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124ff6ad0 msgr2=0x559124ff5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.086+0000 7f84e96ff640 1 --2- 172.21.15.94:0/707551930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124ff6ad0 0x559124ff5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.086+0000 7f84e96ff640 1 -- 172.21.15.94:0/707551930 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559124ef37b0 con 0x559124ff5d60 2024-02-17T12:02:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.086+0000 7f84e96ff640 1 --2- 172.21.15.94:0/707551930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff5d60 0x559124ff61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009fb0 tx=0x7f84e0003e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f0c77f59afe5315 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.087+0000 7f84cf7fe640 1 -- 172.21.15.94:0/707551930 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0004b20 con 0x559124ff5d60 2024-02-17T12:02:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.087+0000 7f84cf7fe640 1 -- 172.21.15.94:0/707551930 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84e0004c80 con 0x559124ff5d60 2024-02-17T12:02:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.088+0000 7f84cf7fe640 1 -- 172.21.15.94:0/707551930 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e00126b0 con 0x559124ff5d60 2024-02-17T12:02:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.088+0000 7f84e9f00640 1 --2- 172.21.15.94:0/875806866 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f84d00a2430 0x7f84d00a48f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f84dc02c780 tx=0x7f84dc02bc30 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.088+0000 7f84ee60c740 1 -- 172.21.15.94:0/707551930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff5d60 msgr2=0x559124ff61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.088+0000 7f84ee60c740 1 --2- 172.21.15.94:0/707551930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff5d60 0x559124ff61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009fb0 tx=0x7f84e0003e00 comp rx=0 tx=0).stop 2024-02-17T12:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.088+0000 7f84ee60c740 1 -- 172.21.15.94:0/707551930 shutdown_connections 2024-02-17T12:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.088+0000 7f84ee60c740 1 --2- 172.21.15.94:0/707551930 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124fff2e0 0x5591250016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.088+0000 7f84ee60c740 1 --2- 172.21.15.94:0/707551930 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff5d60 0x559124ff61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.088+0000 7f84ee60c740 1 --2- 172.21.15.94:0/707551930 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124ff6ad0 0x559124ff5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.088+0000 7f84ee60c740 1 -- 172.21.15.94:0/707551930 >> 172.21.15.94:0/707551930 conn(0x559124fbd550 msgr2=0x559124ff0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.088+0000 7f84ee60c740 1 -- 172.21.15.94:0/707551930 shutdown_connections 2024-02-17T12:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.088+0000 7f84ee60c740 1 -- 172.21.15.94:0/707551930 wait complete. 2024-02-17T12:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.088+0000 7f84ee60c740 1 -- start start 2024-02-17T12:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.088+0000 7f84ee60c740 1 Processor -- start 2024-02-17T12:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.089+0000 7f84ee60c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ffc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.089+0000 7f84ee60c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff6ad0 0x559124ffc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.089+0000 7f84ee60c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124fff2e0 0x559124ffcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84e96ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ffc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84e96ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ffc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:57784/0 (socket says 172.21.15.94:57784) 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84e8efe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff6ad0 0x559124ffc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84e8efe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff6ad0 0x559124ffc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:52322/0 (socket says 172.21.15.94:52322) 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84e9f00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124fff2e0 0x559124ffcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84e9f00640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124fff2e0 0x559124ffcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38366/0 (socket says 172.21.15.94:38366) 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84e96ff640 1 -- 172.21.15.94:0/3705931985 learned_addr learned my addr 172.21.15.94:0/3705931985 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84ee60c740 1 -- 172.21.15.94:0/3705931985 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559124f02bf0 con 0x559124fff2e0 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84ee60c740 1 -- 172.21.15.94:0/3705931985 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559124c31250 con 0x559124ff5d60 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84ee60c740 1 -- 172.21.15.94:0/3705931985 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559124ee3200 con 0x559124ff6ad0 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84ee60c740 10 client.0 _ll_register_callbacks cb 0x559124f51d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84e96ff640 1 -- 172.21.15.94:0/3705931985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff6ad0 msgr2=0x559124ffc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84e96ff640 1 --2- 172.21.15.94:0/3705931985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff6ad0 0x559124ffc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84e96ff640 1 -- 172.21.15.94:0/3705931985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124fff2e0 msgr2=0x559124ffcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84e96ff640 1 --2- 172.21.15.94:0/3705931985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124fff2e0 0x559124ffcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84e96ff640 1 -- 172.21.15.94:0/3705931985 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e0009c10 con 0x559124ff5d60 2024-02-17T12:02:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84e96ff640 1 --2- 172.21.15.94:0/3705931985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ffc290 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f84e0004410 tx=0x7f84e0005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=817e298afdbc738d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.090+0000 7f84ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:02:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ccff9640 1 -- 172.21.15.94:0/3705931985 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0005200 con 0x559124ff5d60 2024-02-17T12:02:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ccff9640 1 -- 172.21.15.94:0/3705931985 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f84e0005360 con 0x559124ff5d60 2024-02-17T12:02:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ee60c740 1 -- 172.21.15.94:0/3705931985 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559124c5d600 con 0x559124ff5d60 2024-02-17T12:02:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ccff9640 1 -- 172.21.15.94:0/3705931985 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84e0005520 con 0x559124ff5d60 2024-02-17T12:02:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ee60c740 10 client.26150 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ee60c740 1 -- 172.21.15.94:0/3705931985 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559124f5bd80 con 0x559124ff5d60 2024-02-17T12:02:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ccff9640 1 -- 172.21.15.94:0/3705931985 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559124c5d600 con 0x559124ff5d60 2024-02-17T12:02:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ccff9640 1 -- 172.21.15.94:0/3705931985 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f84e0018030 con 0x559124ff5d60 2024-02-17T12:02:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ee60c740 10 client.26150 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ee60c740 10 client.26150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ee60c740 20 client.26150 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3705931985 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55912500f560 0x5591250119a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ee60c740 4 client.26150 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.091+0000 7f84ee60c740 1 -- 172.21.15.94:0/3705931985 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559124cae1b0 con 0x55912500f560 2024-02-17T12:02:48.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.092+0000 7f84e8efe640 1 --2- 172.21.15.94:0/3705931985 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55912500f560 0x5591250119a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.092+0000 7f84e8efe640 1 --2- 172.21.15.94:0/3705931985 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55912500f560 0x5591250119a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f40492eee08d80 server_cookie=ed09dcbd10d24ae9 in_seq=0 out_seq=0 2024-02-17T12:02:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.093+0000 7f84ccff9640 10 client.26150 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.093+0000 7f84ccff9640 1 -- 172.21.15.94:0/3705931985 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559124cae1b0 con 0x55912500f560 2024-02-17T12:02:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.093+0000 7f84ccff9640 10 client.26150 handle_command_reply: tid=0 2024-02-17T12:02:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.093+0000 7f84ee60c740 1 client.26150 shutdown 2024-02-17T12:02:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.093+0000 7f84ee60c740 1 -- 172.21.15.94:0/3705931985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 msgr2=0x559124ffc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.094+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3705931985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ffc290 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f84e0004410 tx=0x7f84e0005760 comp rx=0 tx=0).stop 2024-02-17T12:02:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.094+0000 7f84ee60c740 1 -- 172.21.15.94:0/3705931985 shutdown_connections 2024-02-17T12:02:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.094+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3705931985 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55912500f560 0x5591250119a0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.094+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3705931985 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124fff2e0 0x559124ffcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.094+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3705931985 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff6ad0 0x559124ffc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.094+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3705931985 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ffc290 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.094+0000 7f84ee60c740 1 -- 172.21.15.94:0/3705931985 >> 172.21.15.94:0/3705931985 conn(0x559124fbd550 msgr2=0x559124ff27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.094+0000 7f84ccff9640 0 client.26150 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.094+0000 7f84ee60c740 1 -- 172.21.15.94:0/3705931985 shutdown_connections 2024-02-17T12:02:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.094+0000 7f84ee60c740 1 -- 172.21.15.94:0/3705931985 wait complete. 2024-02-17T12:02:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.094+0000 7f84ee60c740 20 client.26150 trim_cache size 0 max 16384 2024-02-17T12:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.116+0000 7f84ee60c740 1 Processor -- start 2024-02-17T12:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.116+0000 7f84ee60c740 1 -- start start 2024-02-17T12:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.116+0000 7f84ee60c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ffe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.116+0000 7f84ee60c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff6ad0 0x5591250061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.116+0000 7f84ee60c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124fff2e0 0x5591250066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.116+0000 7f84ee60c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559124fec180 con 0x559124fff2e0 2024-02-17T12:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.117+0000 7f84ee60c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55912500c170 con 0x559124ff5d60 2024-02-17T12:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.117+0000 7f84ee60c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559125001b80 con 0x559124ff6ad0 2024-02-17T12:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.117+0000 7f84e96ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ffe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.117+0000 7f84e96ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ffe950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:57792/0 (socket says 172.21.15.94:57792) 2024-02-17T12:02:48.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.117+0000 7f84e96ff640 1 -- 172.21.15.94:0/3154908794 learned_addr learned my addr 172.21.15.94:0/3154908794 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.117+0000 7f84e9f00640 1 --2- 172.21.15.94:0/3154908794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124fff2e0 0x5591250066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.117+0000 7f84e8efe640 1 --2- 172.21.15.94:0/3154908794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff6ad0 0x5591250061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.118+0000 7f84e96ff640 1 -- 172.21.15.94:0/3154908794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff6ad0 msgr2=0x5591250061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.118+0000 7f84e96ff640 1 --2- 172.21.15.94:0/3154908794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff6ad0 0x5591250061a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.118+0000 7f84e96ff640 1 -- 172.21.15.94:0/3154908794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124fff2e0 msgr2=0x5591250066e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.118+0000 7f84e96ff640 1 --2- 172.21.15.94:0/3154908794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124fff2e0 0x5591250066e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.118+0000 7f84e96ff640 1 -- 172.21.15.94:0/3154908794 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559124ef87b0 con 0x559124ff5d60 2024-02-17T12:02:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.118+0000 7f84e96ff640 1 --2- 172.21.15.94:0/3154908794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ffe950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f84e0005b30 tx=0x7f84e0005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce04379b35afc79d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.119+0000 7f84ceffd640 1 -- 172.21.15.94:0/3154908794 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e00048f0 con 0x559124ff5d60 2024-02-17T12:02:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.119+0000 7f84ceffd640 1 -- 172.21.15.94:0/3154908794 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84e0025e30 con 0x559124ff5d60 2024-02-17T12:02:48.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.119+0000 7f84ceffd640 1 -- 172.21.15.94:0/3154908794 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0012450 con 0x559124ff5d60 2024-02-17T12:02:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.120+0000 7f84ee60c740 1 -- 172.21.15.94:0/3154908794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 msgr2=0x559124ffe950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.120+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3154908794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ffe950 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f84e0005b30 tx=0x7f84e0005f10 comp rx=0 tx=0).stop 2024-02-17T12:02:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.120+0000 7f84ee60c740 1 -- 172.21.15.94:0/3154908794 shutdown_connections 2024-02-17T12:02:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.120+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3154908794 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124fff2e0 0x5591250066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.120+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3154908794 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124ff6ad0 0x5591250061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.120+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3154908794 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ffe950 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.120+0000 7f84ee60c740 1 -- 172.21.15.94:0/3154908794 >> 172.21.15.94:0/3154908794 conn(0x559124fbd550 msgr2=0x559124ff75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:48.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.120+0000 7f84ee60c740 1 -- 172.21.15.94:0/3154908794 shutdown_connections 2024-02-17T12:02:48.121 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.120+0000 7f84ee60c740 1 -- 172.21.15.94:0/3154908794 wait complete. 2024-02-17T12:02:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.121+0000 7f84ee60c740 1 -- start start 2024-02-17T12:02:48.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.121+0000 7f84ee60c740 1 Processor -- start 2024-02-17T12:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.121+0000 7f84ee60c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ff9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84ee60c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124ff6ad0 0x559125005f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84e96ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ff9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84e96ff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ff9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:57806/0 (socket says 172.21.15.94:57806) 2024-02-17T12:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84ee60c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124fff2e0 0x559125006660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84ee60c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559124ff8690 con 0x559124ff6ad0 2024-02-17T12:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84ee60c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559124ff8800 con 0x559124ff5d60 2024-02-17T12:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84ee60c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559124ff89c0 con 0x559124fff2e0 2024-02-17T12:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84ee60c740 10 client.0 _ll_register_callbacks cb 0x7f84e0009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84e96ff640 1 -- 172.21.15.94:0/3263096569 learned_addr learned my addr 172.21.15.94:0/3263096569 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84e9f00640 1 --2- 172.21.15.94:0/3263096569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124fff2e0 0x559125006660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84e96ff640 1 -- 172.21.15.94:0/3263096569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124fff2e0 msgr2=0x559125006660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:48.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84e96ff640 1 --2- 172.21.15.94:0/3263096569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124fff2e0 0x559125006660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84e96ff640 1 -- 172.21.15.94:0/3263096569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124ff6ad0 msgr2=0x559125005f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84e96ff640 1 --2- 172.21.15.94:0/3263096569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124ff6ad0 0x559125005f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.122+0000 7f84e96ff640 1 -- 172.21.15.94:0/3263096569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559124f5bd80 con 0x559124ff5d60 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.123+0000 7f84e96ff640 1 --2- 172.21.15.94:0/3263096569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ff9f70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f84e0006010 tx=0x7f84e0025880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c1f49c023c931a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.123+0000 7f84ccff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.123+0000 7f84ccff9640 1 -- 172.21.15.94:0/3263096569 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e00048f0 con 0x559124ff5d60 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.123+0000 7f84ccff9640 1 -- 172.21.15.94:0/3263096569 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f84e0025c60 con 0x559124ff5d60 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.123+0000 7f84ee60c740 1 -- 172.21.15.94:0/3263096569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84e0004c80 con 0x559124ff5d60 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.123+0000 7f84ccff9640 1 -- 172.21.15.94:0/3263096569 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84e0012450 con 0x559124ff5d60 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.123+0000 7f84ee60c740 10 client.26156 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.123+0000 7f84ee60c740 1 -- 172.21.15.94:0/3263096569 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559124f1e540 con 0x559124ff5d60 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.124+0000 7f84ccff9640 1 -- 172.21.15.94:0/3263096569 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84e0004c80 con 0x559124ff5d60 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.124+0000 7f84ccff9640 1 -- 172.21.15.94:0/3263096569 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f84e0018070 con 0x559124ff5d60 2024-02-17T12:02:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.124+0000 7f84ee60c740 10 client.26156 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.124+0000 7f84ee60c740 10 client.26156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.124+0000 7f84ee60c740 20 client.26156 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.124+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3263096569 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559125066150 0x559125068570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.124+0000 7f84ee60c740 4 client.26156 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.124+0000 7f84ee60c740 1 -- 172.21.15.94:0/3263096569 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559124c9a550 con 0x559125066150 2024-02-17T12:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.125+0000 7f84e8efe640 1 --2- 172.21.15.94:0/3263096569 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559125066150 0x559125068570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.126+0000 7f84ccff9640 10 client.26156 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.126+0000 7f84e8efe640 1 --2- 172.21.15.94:0/3263096569 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559125066150 0x559125068570 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8179758694421cbb server_cookie=d554f32222fc1924 in_seq=0 out_seq=0 2024-02-17T12:02:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ccff9640 1 -- 172.21.15.94:0/3263096569 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559124c9a550 con 0x559125066150 2024-02-17T12:02:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ccff9640 10 client.26156 handle_command_reply: tid=0 2024-02-17T12:02:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ee60c740 1 client.26156 shutdown 2024-02-17T12:02:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ee60c740 1 -- 172.21.15.94:0/3263096569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 msgr2=0x559124ff9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3263096569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ff9f70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f84e0006010 tx=0x7f84e0025880 comp rx=0 tx=0).stop 2024-02-17T12:02:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ee60c740 1 -- 172.21.15.94:0/3263096569 shutdown_connections 2024-02-17T12:02:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3263096569 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559125066150 0x559125068570 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3263096569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559124fff2e0 0x559125006660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3263096569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559124ff6ad0 0x559125005f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ee60c740 1 --2- 172.21.15.94:0/3263096569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559124ff5d60 0x559124ff9f70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ee60c740 1 -- 172.21.15.94:0/3263096569 >> 172.21.15.94:0/3263096569 conn(0x559124fbd550 msgr2=0x559124ffab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ccff9640 0 client.26156 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ee60c740 1 -- 172.21.15.94:0/3263096569 shutdown_connections 2024-02-17T12:02:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ee60c740 1 -- 172.21.15.94:0/3263096569 wait complete. 2024-02-17T12:02:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.143+0000 7f84ee60c740 20 client.26156 trim_cache size 0 max 16384 2024-02-17T12:02:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:02:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:02:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:02:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:02:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.144+0000 7f84eb963640 1 -- 172.21.15.94:0/875806866 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f84d00a2430 msgr2=0x7f84d00a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.144+0000 7f84eb963640 1 --2- 172.21.15.94:0/875806866 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f84d00a2430 0x7f84d00a48f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f84dc02c780 tx=0x7f84dc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:02:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.144+0000 7f84eb963640 1 -- 172.21.15.94:0/875806866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84e4106470 msgr2=0x7f84e419d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.144+0000 7f84eb963640 1 --2- 172.21.15.94:0/875806866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84e4106470 0x7f84e419d940 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009f10 tx=0x7f84d400ed90 comp rx=0 tx=0).stop 2024-02-17T12:02:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.144+0000 7f84eb963640 1 -- 172.21.15.94:0/875806866 shutdown_connections 2024-02-17T12:02:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.144+0000 7f84eb963640 1 --2- 172.21.15.94:0/875806866 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f84d00a2430 0x7f84d00a48f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.144+0000 7f84eb963640 1 --2- 172.21.15.94:0/875806866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84e4197600 0x7f84e4197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.144+0000 7f84eb963640 1 --2- 172.21.15.94:0/875806866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f84e4106470 0x7f84e419d940 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.144+0000 7f84eb963640 1 --2- 172.21.15.94:0/875806866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f84e41013a0 0x7f84e419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.144+0000 7f84eb963640 1 -- 172.21.15.94:0/875806866 >> 172.21.15.94:0/875806866 conn(0x7f84e40fcc40 msgr2=0x7f84e40fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.144+0000 7f84eb963640 1 -- 172.21.15.94:0/875806866 shutdown_connections 2024-02-17T12:02:48.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:02:48.144+0000 7f84eb963640 1 -- 172.21.15.94:0/875806866 wait complete. 2024-02-17T12:02:48.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:47 smithi125 ceph-mon[36819]: pgmap v143: 97 pgs: 97 active+clean; 3.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 575 op/s 2024-02-17T12:02:48.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:47 smithi094 ceph-mon[32499]: pgmap v143: 97 pgs: 97 active+clean; 3.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 575 op/s 2024-02-17T12:02:48.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:47 smithi189 ceph-mon[36588]: pgmap v143: 97 pgs: 97 active+clean; 3.3 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.6 MiB/s wr, 575 op/s 2024-02-17T12:02:50.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:49 smithi125 ceph-mon[36819]: pgmap v144: 97 pgs: 97 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.5 MiB/s wr, 880 op/s 2024-02-17T12:02:50.203 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:49 smithi189 ceph-mon[36588]: pgmap v144: 97 pgs: 97 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.5 MiB/s wr, 880 op/s 2024-02-17T12:02:50.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:49 smithi094 ceph-mon[32499]: pgmap v144: 97 pgs: 97 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 9.5 MiB/s wr, 880 op/s 2024-02-17T12:02:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 270.0 s, 3619.0 tps, lat 27.628 ms stddev 31.080 2024-02-17T12:02:52.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:02:52.163 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:02:52.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:51 smithi125 ceph-mon[36819]: pgmap v145: 97 pgs: 97 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 592 op/s 2024-02-17T12:02:52.195 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:51 smithi094 ceph-mon[32499]: pgmap v145: 97 pgs: 97 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 592 op/s 2024-02-17T12:02:52.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.301+0000 7f66fe99a640 1 -- 172.21.15.94:0/1180326170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f8100270 msgr2=0x7f66f8100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.301+0000 7f66fe99a640 1 --2- 172.21.15.94:0/1180326170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f8100270 0x7f66f8100650 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009930 tx=0x7f66e802bc30 comp rx=0 tx=0).stop 2024-02-17T12:02:52.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.302+0000 7f66fe99a640 1 -- 172.21.15.94:0/1180326170 shutdown_connections 2024-02-17T12:02:52.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.302+0000 7f66fe99a640 1 --2- 172.21.15.94:0/1180326170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66f8105340 0x7f66f810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.302+0000 7f66fe99a640 1 --2- 172.21.15.94:0/1180326170 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66f8100c20 0x7f66f8104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.302+0000 7f66fe99a640 1 --2- 172.21.15.94:0/1180326170 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f8100270 0x7f66f8100650 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.302+0000 7f66fe99a640 1 -- 172.21.15.94:0/1180326170 >> 172.21.15.94:0/1180326170 conn(0x7f66f8077230 msgr2=0x7f66f8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:52.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.302+0000 7f66fe99a640 1 -- 172.21.15.94:0/1180326170 shutdown_connections 2024-02-17T12:02:52.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.302+0000 7f66fe99a640 1 -- 172.21.15.94:0/1180326170 wait complete. 2024-02-17T12:02:52.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.302+0000 7f66fe99a640 1 Processor -- start 2024-02-17T12:02:52.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.303+0000 7f66fe99a640 1 -- start start 2024-02-17T12:02:52.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.303+0000 7f66fe99a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66f8100270 0x7f66f8196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.303+0000 7f66fe99a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66f8105340 0x7f66f81968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.303+0000 7f66fe99a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f8196ea0 0x7f66f819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.304+0000 7f66fe99a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f819bb30 con 0x7f66f8196ea0 2024-02-17T12:02:52.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.304+0000 7f66fe99a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f819bca0 con 0x7f66f8105340 2024-02-17T12:02:52.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.304+0000 7f66fe99a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f819be60 con 0x7f66f8100270 2024-02-17T12:02:52.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.304+0000 7f66f77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66f8105340 0x7f66f81968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.304+0000 7f66f77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66f8105340 0x7f66f81968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:57828/0 (socket says 172.21.15.94:57828) 2024-02-17T12:02:52.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.304+0000 7f66f77fe640 1 -- 172.21.15.94:0/3053778319 learned_addr learned my addr 172.21.15.94:0/3053778319 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:52.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.305+0000 7f66fcf37640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f8196ea0 0x7f66f819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.307+0000 7f66f7fff640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66f8100270 0x7f66f8196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.307+0000 7f66f77fe640 1 -- 172.21.15.94:0/3053778319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66f8100270 msgr2=0x7f66f8196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.307+0000 7f66f77fe640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66f8100270 0x7f66f8196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.307+0000 7f66f77fe640 1 -- 172.21.15.94:0/3053778319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f8196ea0 msgr2=0x7f66f819b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.307+0000 7f66f77fe640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f8196ea0 0x7f66f819b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.307+0000 7f66f77fe640 1 -- 172.21.15.94:0/3053778319 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e8009560 con 0x7f66f8105340 2024-02-17T12:02:52.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.309+0000 7f66f77fe640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66f8105340 0x7f66f81968c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00e9c0 tx=0x7f66ec00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:52.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.311+0000 7f66f57fa640 1 -- 172.21.15.94:0/3053778319 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec009700 con 0x7f66f8105340 2024-02-17T12:02:52.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.312+0000 7f66f57fa640 1 -- 172.21.15.94:0/3053778319 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66ec00cd80 con 0x7f66f8105340 2024-02-17T12:02:52.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.312+0000 7f66f57fa640 1 -- 172.21.15.94:0/3053778319 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec010430 con 0x7f66f8105340 2024-02-17T12:02:52.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.312+0000 7f66fe99a640 1 -- 172.21.15.94:0/3053778319 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f819c140 con 0x7f66f8105340 2024-02-17T12:02:52.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.312+0000 7f66fe99a640 1 -- 172.21.15.94:0/3053778319 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f819c600 con 0x7f66f8105340 2024-02-17T12:02:52.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.316+0000 7f6701573740 1 Processor -- start 2024-02-17T12:02:52.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.325+0000 7f66f57fa640 1 -- 172.21.15.94:0/3053778319 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f66ec0105d0 con 0x7f66f8105340 2024-02-17T12:02:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.326+0000 7f6701573740 1 -- start start 2024-02-17T12:02:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.326+0000 7f66f57fa640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f66e00a2380 0x7f66e00a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.327+0000 7f66f57fa640 1 -- 172.21.15.94:0/3053778319 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f66ec014070 con 0x7f66f8105340 2024-02-17T12:02:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.327+0000 7f6701573740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba33b0 0x55ee30ba3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.327+0000 7f6701573740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30bac560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.327+0000 7f6701573740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30baee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.327+0000 7f6701573740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee30b12280 con 0x55ee30ba3cd0 2024-02-17T12:02:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.327+0000 7f6701573740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee30b0f7b0 con 0x55ee30ba33b0 2024-02-17T12:02:52.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.327+0000 7f6701573740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee30b0af90 con 0x55ee30bacaa0 2024-02-17T12:02:52.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.331+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba33b0 0x55ee30ba3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.331+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba33b0 0x55ee30ba3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:57838/0 (socket says 172.21.15.94:57838) 2024-02-17T12:02:52.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.331+0000 7f66f7fff640 1 -- 172.21.15.94:0/2561498397 learned_addr learned my addr 172.21.15.94:0/2561498397 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:52.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.331+0000 7f66fcf37640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f66e00a2380 0x7f66e00a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.331+0000 7f66fcf37640 1 --2- 172.21.15.94:0/2561498397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30baee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.331+0000 7f66f77fe640 1 --2- 172.21.15.94:0/2561498397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30bac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.334+0000 7f66fcf37640 1 -- 172.21.15.94:0/2561498397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba33b0 msgr2=0x55ee30ba3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:52.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.334+0000 7f66fcf37640 1 --2- 172.21.15.94:0/2561498397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba33b0 0x55ee30ba3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.334+0000 7f66fcf37640 1 -- 172.21.15.94:0/2561498397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 msgr2=0x55ee30bac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:52.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.334+0000 7f66fcf37640 1 --2- 172.21.15.94:0/2561498397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30bac560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.334+0000 7f66fcf37640 1 -- 172.21.15.94:0/2561498397 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee30ac9570 con 0x55ee30bacaa0 2024-02-17T12:02:52.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.336+0000 7f66fcf37640 1 --2- 172.21.15.94:0/2561498397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30baee90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f66f0005fd0 tx=0x7f66f0010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d809c2937167273a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:52.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.336+0000 7f66fcf37640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f66e00a2380 0x7f66e00a4840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009fd0 tx=0x7f66f0017000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:52.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.345+0000 7f66fe99a640 1 -- 172.21.15.94:0/2561498397 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0012420 con 0x55ee30bacaa0 2024-02-17T12:02:52.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.345+0000 7f66fe99a640 1 -- 172.21.15.94:0/2561498397 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66f00126c0 con 0x55ee30bacaa0 2024-02-17T12:02:52.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.345+0000 7f66fe99a640 1 -- 172.21.15.94:0/2561498397 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0005020 con 0x55ee30bacaa0 2024-02-17T12:02:52.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.348+0000 7f6701573740 1 -- 172.21.15.94:0/2561498397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 msgr2=0x55ee30baee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.348+0000 7f6701573740 1 --2- 172.21.15.94:0/2561498397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30baee90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f66f0005fd0 tx=0x7f66f0010740 comp rx=0 tx=0).stop 2024-02-17T12:02:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.348+0000 7f6701573740 1 -- 172.21.15.94:0/2561498397 shutdown_connections 2024-02-17T12:02:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.348+0000 7f6701573740 1 --2- 172.21.15.94:0/2561498397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30baee90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.348+0000 7f6701573740 1 --2- 172.21.15.94:0/2561498397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30bac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.348+0000 7f6701573740 1 --2- 172.21.15.94:0/2561498397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba33b0 0x55ee30ba3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.348+0000 7f6701573740 1 -- 172.21.15.94:0/2561498397 >> 172.21.15.94:0/2561498397 conn(0x55ee30b9dcd0 msgr2=0x55ee30b9e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:52.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.355+0000 7f6701573740 1 -- 172.21.15.94:0/2561498397 shutdown_connections 2024-02-17T12:02:52.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.355+0000 7f6701573740 1 -- 172.21.15.94:0/2561498397 wait complete. 2024-02-17T12:02:52.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.355+0000 7f6701573740 1 -- start start 2024-02-17T12:02:52.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.356+0000 7f6701573740 1 Processor -- start 2024-02-17T12:02:52.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.356+0000 7f6701573740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30ba33b0 0x55ee30bac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.356+0000 7f6701573740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30ba9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.356+0000 7f6701573740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30bacaa0 0x55ee30ba9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.356+0000 7f6701573740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee307de430 con 0x55ee30ba3cd0 2024-02-17T12:02:52.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.356+0000 7f6701573740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3079ae00 con 0x55ee30bacaa0 2024-02-17T12:02:52.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.356+0000 7f6701573740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee30a8ffd0 con 0x55ee30ba33b0 2024-02-17T12:02:52.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.356+0000 7f6701573740 10 client.0 _ll_register_callbacks cb 0x55ee30ae5fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:52.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.357+0000 7f66f77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30ba9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.357+0000 7f66f77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30ba9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38408/0 (socket says 172.21.15.94:38408) 2024-02-17T12:02:52.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.357+0000 7f66f77fe640 1 -- 172.21.15.94:0/3221187398 learned_addr learned my addr 172.21.15.94:0/3221187398 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:52.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.357+0000 7f66fcf37640 1 --2- 172.21.15.94:0/3221187398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30bacaa0 0x55ee30ba9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.358+0000 7f66f7fff640 1 --2- 172.21.15.94:0/3221187398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30ba33b0 0x55ee30bac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.358+0000 7f66f77fe640 1 -- 172.21.15.94:0/3221187398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30ba33b0 msgr2=0x55ee30bac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:52.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.358+0000 7f66f77fe640 1 --2- 172.21.15.94:0/3221187398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30ba33b0 0x55ee30bac4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.358+0000 7f66f77fe640 1 -- 172.21.15.94:0/3221187398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30bacaa0 msgr2=0x55ee30ba9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:52.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.358+0000 7f66f77fe640 1 --2- 172.21.15.94:0/3221187398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30bacaa0 0x55ee30ba9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.358+0000 7f66f77fe640 1 -- 172.21.15.94:0/3221187398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f001a040 con 0x55ee30ba3cd0 2024-02-17T12:02:52.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.358+0000 7f66f77fe640 1 --2- 172.21.15.94:0/3221187398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30ba9850 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f66ec000c00 tx=0x7f66ec0a0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63c1c034fb9602a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:52.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.358+0000 7f66d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:02:52.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.358+0000 7f66d77fe640 1 -- 172.21.15.94:0/3221187398 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0046c0 con 0x55ee30ba3cd0 2024-02-17T12:02:52.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.358+0000 7f66d77fe640 1 -- 172.21.15.94:0/3221187398 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f66ec0afe40 con 0x55ee30ba3cd0 2024-02-17T12:02:52.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.358+0000 7f6701573740 1 -- 172.21.15.94:0/3221187398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee3080a7e0 con 0x55ee30ba3cd0 2024-02-17T12:02:52.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.359+0000 7f66d77fe640 1 -- 172.21.15.94:0/3221187398 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66ec004820 con 0x55ee30ba3cd0 2024-02-17T12:02:52.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.359+0000 7f6701573740 10 client.16710 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:52.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.359+0000 7f6701573740 1 -- 172.21.15.94:0/3221187398 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ee30b09610 con 0x55ee30ba3cd0 2024-02-17T12:02:52.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.359+0000 7f66d77fe640 1 -- 172.21.15.94:0/3221187398 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ee3080a7e0 con 0x55ee30ba3cd0 2024-02-17T12:02:52.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.359+0000 7f66d77fe640 1 -- 172.21.15.94:0/3221187398 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f66ec01f060 con 0x55ee30ba3cd0 2024-02-17T12:02:52.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.359+0000 7f6701573740 10 client.16710 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:52.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.359+0000 7f6701573740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:52.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.359+0000 7f6701573740 20 client.16710 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:52.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.359+0000 7f6701573740 1 --2- 172.21.15.94:0/3221187398 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee30bcda20 0x55ee30bcfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.359+0000 7f6701573740 4 client.16710 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:52.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.359+0000 7f6701573740 1 -- 172.21.15.94:0/3221187398 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee30847720 con 0x55ee30bcda20 2024-02-17T12:02:52.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.361+0000 7f66f7fff640 1 --2- 172.21.15.94:0/3221187398 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee30bcda20 0x55ee30bcfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.364+0000 7f66f7fff640 1 --2- 172.21.15.94:0/3221187398 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee30bcda20 0x55ee30bcfe40 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59615e9a298fca71 server_cookie=ba7325ef51b5522c in_seq=0 out_seq=0 2024-02-17T12:02:52.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.364+0000 7f66d77fe640 10 client.16710 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:52.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.364+0000 7f66d77fe640 1 -- 172.21.15.94:0/3221187398 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee30847720 con 0x55ee30bcda20 2024-02-17T12:02:52.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.364+0000 7f66d77fe640 10 client.16710 handle_command_reply: tid=0 2024-02-17T12:02:52.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.365+0000 7f6701573740 1 client.16710 shutdown 2024-02-17T12:02:52.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.365+0000 7f6701573740 1 -- 172.21.15.94:0/3221187398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 msgr2=0x55ee30ba9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.365+0000 7f6701573740 1 --2- 172.21.15.94:0/3221187398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30ba9850 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f66ec000c00 tx=0x7f66ec0a0000 comp rx=0 tx=0).stop 2024-02-17T12:02:52.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.365+0000 7f6701573740 1 -- 172.21.15.94:0/3221187398 shutdown_connections 2024-02-17T12:02:52.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.365+0000 7f6701573740 1 --2- 172.21.15.94:0/3221187398 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee30bcda20 0x55ee30bcfe40 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.365+0000 7f6701573740 1 --2- 172.21.15.94:0/3221187398 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30bacaa0 0x55ee30ba9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.365+0000 7f6701573740 1 --2- 172.21.15.94:0/3221187398 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30ba9850 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.365+0000 7f6701573740 1 --2- 172.21.15.94:0/3221187398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30ba33b0 0x55ee30bac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.365+0000 7f6701573740 1 -- 172.21.15.94:0/3221187398 >> 172.21.15.94:0/3221187398 conn(0x55ee30b9dcd0 msgr2=0x55ee30b9fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:52.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.365+0000 7f66d77fe640 0 client.16710 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:52.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.365+0000 7f6701573740 1 -- 172.21.15.94:0/3221187398 shutdown_connections 2024-02-17T12:02:52.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.365+0000 7f6701573740 1 -- 172.21.15.94:0/3221187398 wait complete. 2024-02-17T12:02:52.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.365+0000 7f6701573740 20 client.16710 trim_cache size 0 max 16384 2024-02-17T12:02:52.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.387+0000 7f6701573740 1 Processor -- start 2024-02-17T12:02:52.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.387+0000 7f6701573740 1 -- start start 2024-02-17T12:02:52.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.388+0000 7f6701573740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba33b0 0x55ee30babf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.388+0000 7f6701573740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba3cd0 0x55ee30bac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.388+0000 7f6701573740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30ba8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.388+0000 7f6701573740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee3080a7e0 con 0x55ee30ba33b0 2024-02-17T12:02:52.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.388+0000 7f6701573740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee30ba24a0 con 0x55ee30ba3cd0 2024-02-17T12:02:52.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.388+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba33b0 0x55ee30babf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.388+0000 7f66fcf37640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30ba8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.388+0000 7f66fcf37640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30ba8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:52384/0 (socket says 172.21.15.94:52384) 2024-02-17T12:02:52.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.388+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba33b0 0x55ee30babf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38412/0 (socket says 172.21.15.94:38412) 2024-02-17T12:02:52.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.388+0000 7f66fcf37640 1 -- 172.21.15.94:0/344279235 learned_addr learned my addr 172.21.15.94:0/344279235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:52.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.388+0000 7f6701573740 1 -- 172.21.15.94:0/344279235 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee30bca470 con 0x55ee30bacaa0 2024-02-17T12:02:52.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.389+0000 7f66fcf37640 1 -- 172.21.15.94:0/344279235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba3cd0 msgr2=0x55ee30bac4c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:02:52.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.389+0000 7f66fcf37640 1 --2- 172.21.15.94:0/344279235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba3cd0 0x55ee30bac4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.389+0000 7f66fcf37640 1 -- 172.21.15.94:0/344279235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba33b0 msgr2=0x55ee30babf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:52.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.389+0000 7f66fcf37640 1 --2- 172.21.15.94:0/344279235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba33b0 0x55ee30babf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.389+0000 7f66fcf37640 1 -- 172.21.15.94:0/344279235 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee30aa5ac0 con 0x55ee30bacaa0 2024-02-17T12:02:52.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.389+0000 7f66fcf37640 1 --2- 172.21.15.94:0/344279235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30ba8120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009800 tx=0x7f66f0011d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59ca6e728b09e74c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:52.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.390+0000 7f66ddffb640 1 -- 172.21.15.94:0/344279235 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0012420 con 0x55ee30bacaa0 2024-02-17T12:02:52.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.390+0000 7f66ddffb640 1 -- 172.21.15.94:0/344279235 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66f0011b20 con 0x55ee30bacaa0 2024-02-17T12:02:52.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.390+0000 7f66ddffb640 1 -- 172.21.15.94:0/344279235 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0005340 con 0x55ee30bacaa0 2024-02-17T12:02:52.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.391+0000 7f6701573740 1 -- 172.21.15.94:0/344279235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 msgr2=0x55ee30ba8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.391+0000 7f6701573740 1 --2- 172.21.15.94:0/344279235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30ba8120 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009800 tx=0x7f66f0011d90 comp rx=0 tx=0).stop 2024-02-17T12:02:52.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.391+0000 7f6701573740 1 -- 172.21.15.94:0/344279235 shutdown_connections 2024-02-17T12:02:52.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.391+0000 7f6701573740 1 --2- 172.21.15.94:0/344279235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30ba8120 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.391+0000 7f6701573740 1 --2- 172.21.15.94:0/344279235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba3cd0 0x55ee30bac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.391+0000 7f6701573740 1 --2- 172.21.15.94:0/344279235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba33b0 0x55ee30babf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.391+0000 7f6701573740 1 -- 172.21.15.94:0/344279235 >> 172.21.15.94:0/344279235 conn(0x55ee30b9dcd0 msgr2=0x55ee30ba4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:52.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.391+0000 7f6701573740 1 -- 172.21.15.94:0/344279235 shutdown_connections 2024-02-17T12:02:52.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.391+0000 7f6701573740 1 -- 172.21.15.94:0/344279235 wait complete. 2024-02-17T12:02:52.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f6701573740 1 -- start start 2024-02-17T12:02:52.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f6701573740 1 Processor -- start 2024-02-17T12:02:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f6701573740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba33b0 0x55ee30bb0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f6701573740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30bb07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f6701573740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30bb0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f6701573740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee30c24040 con 0x55ee30ba3cd0 2024-02-17T12:02:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f6701573740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee30c241b0 con 0x55ee30ba33b0 2024-02-17T12:02:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f6701573740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee30c24370 con 0x55ee30bacaa0 2024-02-17T12:02:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f66f77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30bb07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f6701573740 10 client.0 _ll_register_callbacks cb 0x7f66f0009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba33b0 0x55ee30bb0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f66fcf37640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30bb0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f66f7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba33b0 0x55ee30bb0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:57850/0 (socket says 172.21.15.94:57850) 2024-02-17T12:02:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f66fcf37640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30bb0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:52398/0 (socket says 172.21.15.94:52398) 2024-02-17T12:02:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.392+0000 7f66f7fff640 1 -- 172.21.15.94:0/3192850042 learned_addr learned my addr 172.21.15.94:0/3192850042 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.393+0000 7f66fcf37640 1 -- 172.21.15.94:0/3192850042 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba33b0 msgr2=0x55ee30bb0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.393+0000 7f66fcf37640 1 --2- 172.21.15.94:0/3192850042 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba33b0 0x55ee30bb0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.393+0000 7f66fcf37640 1 -- 172.21.15.94:0/3192850042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 msgr2=0x55ee30bb07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.393+0000 7f66fcf37640 1 --2- 172.21.15.94:0/3192850042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30bb07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.393+0000 7f66fcf37640 1 -- 172.21.15.94:0/3192850042 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee30ac9570 con 0x55ee30bacaa0 2024-02-17T12:02:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.393+0000 7f66f77fe640 1 --2- 172.21.15.94:0/3192850042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30bb07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.393+0000 7f66fcf37640 1 --2- 172.21.15.94:0/3192850042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30bb0d10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f66f0012a80 tx=0x7f66f00119c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70999720a4bd0ca3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.393+0000 7f66d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:02:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.393+0000 7f66d77fe640 1 -- 172.21.15.94:0/3192850042 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0012420 con 0x55ee30bacaa0 2024-02-17T12:02:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.393+0000 7f66d77fe640 1 -- 172.21.15.94:0/3192850042 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f66f0011e00 con 0x55ee30bacaa0 2024-02-17T12:02:52.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.393+0000 7f6701573740 1 -- 172.21.15.94:0/3192850042 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66f00126c0 con 0x55ee30bacaa0 2024-02-17T12:02:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.393+0000 7f66d77fe640 1 -- 172.21.15.94:0/3192850042 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66f0004bf0 con 0x55ee30bacaa0 2024-02-17T12:02:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.393+0000 7f6701573740 10 client.26224 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.394+0000 7f6701573740 1 -- 172.21.15.94:0/3192850042 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55ee30acb6a0 con 0x55ee30bacaa0 2024-02-17T12:02:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.394+0000 7f66d77fe640 1 -- 172.21.15.94:0/3192850042 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f66f00126c0 con 0x55ee30bacaa0 2024-02-17T12:02:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.394+0000 7f66d77fe640 1 -- 172.21.15.94:0/3192850042 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f66f000c070 con 0x55ee30bacaa0 2024-02-17T12:02:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.394+0000 7f6701573740 10 client.26224 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.394+0000 7f6701573740 10 client.26224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.394+0000 7f6701573740 20 client.26224 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.395+0000 7f6701573740 1 --2- 172.21.15.94:0/3192850042 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee30c257b0 0x55ee30c27ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.395+0000 7f6701573740 4 client.26224 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:02:52.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.395+0000 7f6701573740 1 -- 172.21.15.94:0/3192850042 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ee30896890 con 0x55ee30c257b0 2024-02-17T12:02:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.395+0000 7f66f7fff640 1 --2- 172.21.15.94:0/3192850042 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee30c257b0 0x55ee30c27ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.395+0000 7f66f7fff640 1 --2- 172.21.15.94:0/3192850042 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee30c257b0 0x55ee30c27ba0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=231f39e021a50213 server_cookie=e70d2e3f087d5a59 in_seq=0 out_seq=0 2024-02-17T12:02:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.395+0000 7f66d77fe640 10 client.26224 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f66d77fe640 1 -- 172.21.15.94:0/3192850042 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ee30896890 con 0x55ee30c257b0 2024-02-17T12:02:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f66d77fe640 10 client.26224 handle_command_reply: tid=0 2024-02-17T12:02:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f6701573740 1 client.26224 shutdown 2024-02-17T12:02:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f6701573740 1 -- 172.21.15.94:0/3192850042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 msgr2=0x55ee30bb0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f6701573740 1 --2- 172.21.15.94:0/3192850042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30bb0d10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f66f0012a80 tx=0x7f66f00119c0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f6701573740 1 -- 172.21.15.94:0/3192850042 shutdown_connections 2024-02-17T12:02:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f6701573740 1 --2- 172.21.15.94:0/3192850042 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ee30c257b0 0x55ee30c27ba0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f6701573740 1 --2- 172.21.15.94:0/3192850042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ee30bacaa0 0x55ee30bb0d10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f6701573740 1 --2- 172.21.15.94:0/3192850042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ee30ba3cd0 0x55ee30bb07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f6701573740 1 --2- 172.21.15.94:0/3192850042 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ee30ba33b0 0x55ee30bb0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f6701573740 1 -- 172.21.15.94:0/3192850042 >> 172.21.15.94:0/3192850042 conn(0x55ee30b9dcd0 msgr2=0x55ee30ba80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:52.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f66d77fe640 0 client.26224 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:52.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f6701573740 1 -- 172.21.15.94:0/3192850042 shutdown_connections 2024-02-17T12:02:52.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.396+0000 7f6701573740 1 -- 172.21.15.94:0/3192850042 wait complete. 2024-02-17T12:02:52.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.397+0000 7f6701573740 20 client.26224 trim_cache size 0 max 16384 2024-02-17T12:02:52.401 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:02:52.401 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:02:52.401 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:02:52.401 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:02:52.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.397+0000 7f66deffd640 1 -- 172.21.15.94:0/3053778319 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f66e00a2380 msgr2=0x7f66e00a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.397+0000 7f66deffd640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f66e00a2380 0x7f66e00a4840 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f66f0009fd0 tx=0x7f66f0017000 comp rx=0 tx=0).stop 2024-02-17T12:02:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.397+0000 7f66deffd640 1 -- 172.21.15.94:0/3053778319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66f8105340 msgr2=0x7f66f81968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.397+0000 7f66deffd640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66f8105340 0x7f66f81968c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00e9c0 tx=0x7f66ec00ee80 comp rx=0 tx=0).stop 2024-02-17T12:02:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.397+0000 7f66deffd640 1 -- 172.21.15.94:0/3053778319 shutdown_connections 2024-02-17T12:02:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.397+0000 7f66deffd640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f66e00a2380 0x7f66e00a4840 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.397+0000 7f66deffd640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f66f8196ea0 0x7f66f819b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.397+0000 7f66deffd640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f66f8105340 0x7f66f81968c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.398+0000 7f66deffd640 1 --2- 172.21.15.94:0/3053778319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f66f8100270 0x7f66f8196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.398+0000 7f66deffd640 1 -- 172.21.15.94:0/3053778319 >> 172.21.15.94:0/3053778319 conn(0x7f66f8077230 msgr2=0x7f66f80fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:52.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.398+0000 7f66deffd640 1 -- 172.21.15.94:0/3053778319 shutdown_connections 2024-02-17T12:02:52.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.398+0000 7f66deffd640 1 -- 172.21.15.94:0/3053778319 wait complete. 2024-02-17T12:02:52.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:51 smithi189 ceph-mon[36588]: pgmap v145: 97 pgs: 97 active+clean; 3.3 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.8 MiB/s wr, 592 op/s 2024-02-17T12:02:52.413 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:02:52.413 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:02:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.600+0000 7faec7eb5640 1 -- 172.21.15.94:0/3125313466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faec0100b00 msgr2=0x7faec0104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.600+0000 7faec7eb5640 1 --2- 172.21.15.94:0/3125313466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faec0100b00 0x7faec0104fc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009930 tx=0x7faeb002bc30 comp rx=0 tx=0).stop 2024-02-17T12:02:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.600+0000 7faec7eb5640 1 -- 172.21.15.94:0/3125313466 shutdown_connections 2024-02-17T12:02:52.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.600+0000 7faec7eb5640 1 --2- 172.21.15.94:0/3125313466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faec01055e0 0x7faec0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.600+0000 7faec7eb5640 1 --2- 172.21.15.94:0/3125313466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faec0100b00 0x7faec0104fc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.600+0000 7faec7eb5640 1 --2- 172.21.15.94:0/3125313466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec01001e0 0x7faec01005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.600+0000 7faec7eb5640 1 -- 172.21.15.94:0/3125313466 >> 172.21.15.94:0/3125313466 conn(0x7faec0077180 msgr2=0x7faec0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:52.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.601+0000 7faec7eb5640 1 -- 172.21.15.94:0/3125313466 shutdown_connections 2024-02-17T12:02:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.601+0000 7faec7eb5640 1 -- 172.21.15.94:0/3125313466 wait complete. 2024-02-17T12:02:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.602+0000 7faec7eb5640 1 Processor -- start 2024-02-17T12:02:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.602+0000 7faec7eb5640 1 -- start start 2024-02-17T12:02:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.602+0000 7faec7eb5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec01001e0 0x7faec01a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.602+0000 7faec7eb5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faec0100b00 0x7faec01a3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.602+0000 7faec7eb5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faec01055e0 0x7faec019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.602+0000 7faec7eb5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec019e110 con 0x7faec01055e0 2024-02-17T12:02:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.602+0000 7faec7eb5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec019e280 con 0x7faec0100b00 2024-02-17T12:02:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.602+0000 7faec7eb5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec019e440 con 0x7faec01001e0 2024-02-17T12:02:52.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.602+0000 7faec6452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faec01055e0 0x7faec019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.602+0000 7faec6452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faec01055e0 0x7faec019dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38434/0 (socket says 172.21.15.94:38434) 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.602+0000 7faec6452640 1 -- 172.21.15.94:0/2465673667 learned_addr learned my addr 172.21.15.94:0/2465673667 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.602+0000 7faec5c51640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec01001e0 0x7faec01a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.602+0000 7faec5450640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faec0100b00 0x7faec01a3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.603+0000 7faec6452640 1 -- 172.21.15.94:0/2465673667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec01001e0 msgr2=0x7faec01a38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.603+0000 7faec6452640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec01001e0 0x7faec01a38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.603+0000 7faec6452640 1 -- 172.21.15.94:0/2465673667 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faec0100b00 msgr2=0x7faec01a3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.603+0000 7faec6452640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faec0100b00 0x7faec01a3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.603+0000 7faec6452640 1 -- 172.21.15.94:0/2465673667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb0009560 con 0x7faec01055e0 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.603+0000 7faec5450640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faec0100b00 0x7faec01a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.603+0000 7faec6452640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faec01055e0 0x7faec019dac0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faeb800eb10 tx=0x7faeb800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.603+0000 7faeb6ffd640 1 -- 172.21.15.94:0/2465673667 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8004b20 con 0x7faec01055e0 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.603+0000 7faec7eb5640 1 -- 172.21.15.94:0/2465673667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec019e720 con 0x7faec01055e0 2024-02-17T12:02:52.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.603+0000 7faec7eb5640 1 -- 172.21.15.94:0/2465673667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec01aa960 con 0x7faec01055e0 2024-02-17T12:02:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.603+0000 7faeb6ffd640 1 -- 172.21.15.94:0/2465673667 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faeb8004c80 con 0x7faec01055e0 2024-02-17T12:02:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.603+0000 7faeb6ffd640 1 -- 172.21.15.94:0/2465673667 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8010650 con 0x7faec01055e0 2024-02-17T12:02:52.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.605+0000 7faec7eb5640 1 -- 172.21.15.94:0/2465673667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea4005290 con 0x7faec01055e0 2024-02-17T12:02:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.606+0000 7faeb6ffd640 1 -- 172.21.15.94:0/2465673667 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7faeb8005020 con 0x7faec01055e0 2024-02-17T12:02:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.607+0000 7faeb6ffd640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7faeac0a2330 0x7faeac0a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:52.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.607+0000 7faeb6ffd640 1 -- 172.21.15.94:0/2465673667 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7faeb8014070 con 0x7faec01055e0 2024-02-17T12:02:52.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.608+0000 7faec5c51640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7faeac0a2330 0x7faeac0a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:52.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.609+0000 7faec5c51640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7faeac0a2330 0x7faeac0a47f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faebc0058c0 tx=0x7faebc00a2d0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:52.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.614+0000 7faeb6ffd640 1 -- 172.21.15.94:0/2465673667 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faeb80971d0 con 0x7faec01055e0 2024-02-17T12:02:52.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.938+0000 7faec7eb5640 1 -- 172.21.15.94:0/2465673667 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faea4005810 con 0x7faec01055e0 2024-02-17T12:02:52.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.939+0000 7faeb6ffd640 1 -- 172.21.15.94:0/2465673667 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+11344 (secure 0 0 0) 0x7faeb809a890 con 0x7faec01055e0 2024-02-17T12:02:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:02:52.939 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:01:15.200235+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24418":{"gid":24418,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1397235152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1397235152},{"type":"v1","addr":"172.21.15.189:6833","nonce":1397235152}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:02:52.939 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 23 2024-02-17T12:02:52.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.942+0000 7faec7eb5640 1 -- 172.21.15.94:0/2465673667 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7faeac0a2330 msgr2=0x7faeac0a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.942+0000 7faec7eb5640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7faeac0a2330 0x7faeac0a47f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faebc0058c0 tx=0x7faebc00a2d0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.942+0000 7faec7eb5640 1 -- 172.21.15.94:0/2465673667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faec01055e0 msgr2=0x7faec019dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:52.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.942+0000 7faec7eb5640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faec01055e0 0x7faec019dac0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7faeb800eb10 tx=0x7faeb800c2d0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.942+0000 7faec7eb5640 1 -- 172.21.15.94:0/2465673667 shutdown_connections 2024-02-17T12:02:52.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.942+0000 7faec7eb5640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7faeac0a2330 0x7faeac0a47f0 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faebc0058c0 tx=0x7faebc00a2d0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.942+0000 7faec7eb5640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faec01055e0 0x7faec019dac0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.942+0000 7faec7eb5640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faec0100b00 0x7faec01a3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.942+0000 7faec7eb5640 1 --2- 172.21.15.94:0/2465673667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec01001e0 0x7faec01a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:52.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.942+0000 7faec7eb5640 1 -- 172.21.15.94:0/2465673667 >> 172.21.15.94:0/2465673667 conn(0x7faec0077180 msgr2=0x7faec0103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:52.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.943+0000 7faec7eb5640 1 -- 172.21.15.94:0/2465673667 shutdown_connections 2024-02-17T12:02:52.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:52.943+0000 7faec7eb5640 1 -- 172.21.15.94:0/2465673667 wait complete. 2024-02-17T12:02:52.956 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:02:52.956 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:02:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.107+0000 7f00fb14b640 1 -- 172.21.15.94:0/1685641456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f40ff890 msgr2=0x7f00f40ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.107+0000 7f00fb14b640 1 --2- 172.21.15.94:0/1685641456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f40ff890 0x7f00f40ffc70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009860 tx=0x7f00e802bb90 comp rx=0 tx=0).stop 2024-02-17T12:02:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.107+0000 7f00fb14b640 1 -- 172.21.15.94:0/1685641456 shutdown_connections 2024-02-17T12:02:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.108+0000 7f00fb14b640 1 --2- 172.21.15.94:0/1685641456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00f4106450 0x7f00f410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.108+0000 7f00fb14b640 1 --2- 172.21.15.94:0/1685641456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f4100240 0x7f00f4105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.108+0000 7f00fb14b640 1 --2- 172.21.15.94:0/1685641456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f40ff890 0x7f00f40ffc70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.108+0000 7f00fb14b640 1 -- 172.21.15.94:0/1685641456 >> 172.21.15.94:0/1685641456 conn(0x7f00f40765a0 msgr2=0x7f00f40fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.108+0000 7f00fb14b640 1 -- 172.21.15.94:0/1685641456 shutdown_connections 2024-02-17T12:02:53.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.108+0000 7f00fb14b640 1 -- 172.21.15.94:0/1685641456 wait complete. 2024-02-17T12:02:53.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.109+0000 7f00fb14b640 1 Processor -- start 2024-02-17T12:02:53.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.109+0000 7f00fb14b640 1 -- start start 2024-02-17T12:02:53.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.109+0000 7f00fb14b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f40ff890 0x7f00f4072800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.109+0000 7f00fb14b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00f4106450 0x7f00f4072d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.109+0000 7f00fb14b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f406b8d0 0x7f00f406bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.109+0000 7f00fb14b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f406c4f0 con 0x7f00f406b8d0 2024-02-17T12:02:53.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.109+0000 7f00fb14b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f406c660 con 0x7f00f4106450 2024-02-17T12:02:53.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.109+0000 7f00fb14b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f4070320 con 0x7f00f40ff890 2024-02-17T12:02:53.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.109+0000 7f00f96e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f406b8d0 0x7f00f406bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.109+0000 7f00f3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00f4106450 0x7f00f4072d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f8ee7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f40ff890 0x7f00f4072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f96e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f406b8d0 0x7f00f406bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38446/0 (socket says 172.21.15.94:38446) 2024-02-17T12:02:53.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f96e8640 1 -- 172.21.15.94:0/379992228 learned_addr learned my addr 172.21.15.94:0/379992228 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f96e8640 1 -- 172.21.15.94:0/379992228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f40ff890 msgr2=0x7f00f4072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f96e8640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f40ff890 0x7f00f4072800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f96e8640 1 -- 172.21.15.94:0/379992228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00f4106450 msgr2=0x7f00f4072d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f96e8640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00f4106450 0x7f00f4072d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f96e8640 1 -- 172.21.15.94:0/379992228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e8009490 con 0x7f00f406b8d0 2024-02-17T12:02:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f8ee7640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f40ff890 0x7f00f4072800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f3fff640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00f4106450 0x7f00f4072d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f96e8640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f406b8d0 0x7f00f406bd60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00eb10 tx=0x7f00ec00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f1ffb640 1 -- 172.21.15.94:0/379992228 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec00ce20 con 0x7f00f406b8d0 2024-02-17T12:02:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f1ffb640 1 -- 172.21.15.94:0/379992228 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00ec005020 con 0x7f00f406b8d0 2024-02-17T12:02:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00f1ffb640 1 -- 172.21.15.94:0/379992228 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec010690 con 0x7f00f406b8d0 2024-02-17T12:02:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00fb14b640 1 -- 172.21.15.94:0/379992228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f4070580 con 0x7f00f406b8d0 2024-02-17T12:02:53.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.110+0000 7f00fb14b640 1 -- 172.21.15.94:0/379992228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f4070a40 con 0x7f00f406b8d0 2024-02-17T12:02:53.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.113+0000 7f00f1ffb640 1 -- 172.21.15.94:0/379992228 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f00ec005500 con 0x7f00f406b8d0 2024-02-17T12:02:53.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.114+0000 7f00f1ffb640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f00e40a22f0 0x7f00e40a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.114+0000 7f00f1ffb640 1 -- 172.21.15.94:0/379992228 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f00ec014070 con 0x7f00f406b8d0 2024-02-17T12:02:53.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.114+0000 7f00f8ee7640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f00e40a22f0 0x7f00e40a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.115+0000 7f00f8ee7640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f00e40a22f0 0x7f00e40a47b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009830 tx=0x7f00e802c0b0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.115+0000 7f00fddf1740 1 Processor -- start 2024-02-17T12:02:53.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.115+0000 7f00fddf1740 1 -- start start 2024-02-17T12:02:53.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.115+0000 7f00fddf1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 0x55973f58f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.115+0000 7f00fddf1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 0x55973f5a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.115+0000 7f00fddf1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f5a0f60 0x55973f5a3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.115+0000 7f00fddf1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973f54a960 con 0x55973f58f060 2024-02-17T12:02:53.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00fddf1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973f4fe310 con 0x55973f5a0f60 2024-02-17T12:02:53.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00fddf1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973f4fb840 con 0x55973f58fa10 2024-02-17T12:02:53.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00f8ee7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 0x55973f58f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00f3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 0x55973f5a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00f8ee7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 0x55973f58f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38458/0 (socket says 172.21.15.94:38458) 2024-02-17T12:02:53.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00f8ee7640 1 -- 172.21.15.94:0/1985451816 learned_addr learned my addr 172.21.15.94:0/1985451816 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00f3fff640 1 --2- 172.21.15.94:0/1985451816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 0x55973f5a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:52430/0 (socket says 172.21.15.94:52430) 2024-02-17T12:02:53.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00f96e8640 1 --2- 172.21.15.94:0/1985451816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f5a0f60 0x55973f5a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00f96e8640 1 -- 172.21.15.94:0/1985451816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 msgr2=0x55973f5a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00f96e8640 1 --2- 172.21.15.94:0/1985451816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 0x55973f5a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00f96e8640 1 -- 172.21.15.94:0/1985451816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 msgr2=0x55973f58f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00f96e8640 1 --2- 172.21.15.94:0/1985451816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 0x55973f58f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00f96e8640 1 -- 172.21.15.94:0/1985451816 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55973f4b9060 con 0x55973f5a0f60 2024-02-17T12:02:53.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.116+0000 7f00f8ee7640 1 --2- 172.21.15.94:0/1985451816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 0x55973f58f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:53.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.117+0000 7f00f96e8640 1 --2- 172.21.15.94:0/1985451816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f5a0f60 0x55973f5a3350 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0cbd00 tx=0x7f00ec005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f55993a5c28a2cc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.117+0000 7f00deffd640 1 -- 172.21.15.94:0/1985451816 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0a3770 con 0x55973f5a0f60 2024-02-17T12:02:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.117+0000 7f00deffd640 1 -- 172.21.15.94:0/1985451816 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00ec0cbfb0 con 0x55973f5a0f60 2024-02-17T12:02:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.118+0000 7f00deffd640 1 -- 172.21.15.94:0/1985451816 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec095b40 con 0x55973f5a0f60 2024-02-17T12:02:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.118+0000 7f00fddf1740 1 -- 172.21.15.94:0/1985451816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f5a0f60 msgr2=0x55973f5a3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.118+0000 7f00fddf1740 1 --2- 172.21.15.94:0/1985451816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f5a0f60 0x55973f5a3350 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0cbd00 tx=0x7f00ec005250 comp rx=0 tx=0).stop 2024-02-17T12:02:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.119+0000 7f00fddf1740 1 -- 172.21.15.94:0/1985451816 shutdown_connections 2024-02-17T12:02:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.119+0000 7f00fddf1740 1 --2- 172.21.15.94:0/1985451816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f5a0f60 0x55973f5a3350 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.119+0000 7f00fddf1740 1 --2- 172.21.15.94:0/1985451816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 0x55973f5a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.119+0000 7f00fddf1740 1 --2- 172.21.15.94:0/1985451816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 0x55973f58f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.119+0000 7f00fddf1740 1 -- 172.21.15.94:0/1985451816 >> 172.21.15.94:0/1985451816 conn(0x55973f589980 msgr2=0x55973f589d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.119+0000 7f00fddf1740 1 -- 172.21.15.94:0/1985451816 shutdown_connections 2024-02-17T12:02:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.119+0000 7f00fddf1740 1 -- 172.21.15.94:0/1985451816 wait complete. 2024-02-17T12:02:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.119+0000 7f00fddf1740 1 -- start start 2024-02-17T12:02:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.119+0000 7f00fddf1740 1 Processor -- start 2024-02-17T12:02:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.120+0000 7f00fddf1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f58f060 0x55973f59e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.120+0000 7f00fddf1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 0x55973f59e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.120+0000 7f00fddf1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f5a0f60 0x55973f59eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.120+0000 7f00fddf1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973f4f7020 con 0x55973f5a0f60 2024-02-17T12:02:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.120+0000 7f00fddf1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973f1ca430 con 0x55973f58f060 2024-02-17T12:02:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.120+0000 7f00fddf1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973f186e00 con 0x55973f58fa10 2024-02-17T12:02:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.120+0000 7f00fddf1740 10 client.0 _ll_register_callbacks cb 0x55973f49ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.120+0000 7f00f8ee7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f58f060 0x55973f59e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.120+0000 7f00f8ee7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f58f060 0x55973f59e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:57886/0 (socket says 172.21.15.94:57886) 2024-02-17T12:02:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.120+0000 7f00f8ee7640 1 -- 172.21.15.94:0/3364147373 learned_addr learned my addr 172.21.15.94:0/3364147373 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.121+0000 7f00f96e8640 1 --2- 172.21.15.94:0/3364147373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f5a0f60 0x55973f59eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.121+0000 7f00f8ee7640 1 -- 172.21.15.94:0/3364147373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 msgr2=0x55973f59e5b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:02:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.121+0000 7f00f8ee7640 1 --2- 172.21.15.94:0/3364147373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 0x55973f59e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.121+0000 7f00f8ee7640 1 -- 172.21.15.94:0/3364147373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f5a0f60 msgr2=0x55973f59eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.121+0000 7f00f8ee7640 1 --2- 172.21.15.94:0/3364147373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f5a0f60 0x55973f59eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.121+0000 7f00f8ee7640 1 -- 172.21.15.94:0/3364147373 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f4070a40 con 0x55973f58f060 2024-02-17T12:02:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.121+0000 7f00f8ee7640 1 --2- 172.21.15.94:0/3364147373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f58f060 0x55973f59e070 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f00e8037040 tx=0x7f00e8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29e656367ab37d88 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.121+0000 7f00d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:02:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.121+0000 7f00d3fff640 1 -- 172.21.15.94:0/3364147373 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e8034a50 con 0x55973f58f060 2024-02-17T12:02:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.122+0000 7f00d3fff640 1 -- 172.21.15.94:0/3364147373 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f00e802c730 con 0x55973f58f060 2024-02-17T12:02:53.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.122+0000 7f00fddf1740 1 -- 172.21.15.94:0/3364147373 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55973f1f67e0 con 0x55973f58f060 2024-02-17T12:02:53.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.122+0000 7f00d3fff640 1 -- 172.21.15.94:0/3364147373 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00e802c930 con 0x55973f58f060 2024-02-17T12:02:53.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.122+0000 7f00fddf1740 10 client.26189 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:53.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.122+0000 7f00fddf1740 1 -- 172.21.15.94:0/3364147373 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55973f4f56a0 con 0x55973f58f060 2024-02-17T12:02:53.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.122+0000 7f00d3fff640 1 -- 172.21.15.94:0/3364147373 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55973f1f67e0 con 0x55973f58f060 2024-02-17T12:02:53.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.122+0000 7f00d3fff640 1 -- 172.21.15.94:0/3364147373 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f00e8053020 con 0x55973f58f060 2024-02-17T12:02:53.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.122+0000 7f00fddf1740 10 client.26189 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:53.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.122+0000 7f00fddf1740 10 client.26189 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:02:53.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.122+0000 7f00fddf1740 20 client.26189 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:53.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.122+0000 7f00fddf1740 1 --2- 172.21.15.94:0/3364147373 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55973f5b14f0 0x55973f5b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.122+0000 7f00fddf1740 4 client.26189 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:53.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.122+0000 7f00fddf1740 1 -- 172.21.15.94:0/3364147373 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55973f233720 con 0x55973f5b14f0 2024-02-17T12:02:53.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.123+0000 7f00f3fff640 1 --2- 172.21.15.94:0/3364147373 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55973f5b14f0 0x55973f5b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.124+0000 7f00f3fff640 1 --2- 172.21.15.94:0/3364147373 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55973f5b14f0 0x55973f5b3910 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68e52a2896d29d0c server_cookie=e1601240cf200175 in_seq=0 out_seq=0 2024-02-17T12:02:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.124+0000 7f00d3fff640 10 client.26189 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:02:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.124+0000 7f00d3fff640 1 -- 172.21.15.94:0/3364147373 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55973f233720 con 0x55973f5b14f0 2024-02-17T12:02:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.124+0000 7f00d3fff640 10 client.26189 handle_command_reply: tid=0 2024-02-17T12:02:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.124+0000 7f00fddf1740 1 client.26189 shutdown 2024-02-17T12:02:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.124+0000 7f00fddf1740 1 -- 172.21.15.94:0/3364147373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f58f060 msgr2=0x55973f59e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.124+0000 7f00fddf1740 1 --2- 172.21.15.94:0/3364147373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f58f060 0x55973f59e070 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f00e8037040 tx=0x7f00e8005f90 comp rx=0 tx=0).stop 2024-02-17T12:02:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.124+0000 7f00fddf1740 1 -- 172.21.15.94:0/3364147373 shutdown_connections 2024-02-17T12:02:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.124+0000 7f00fddf1740 1 --2- 172.21.15.94:0/3364147373 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f5a0f60 0x55973f59eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.124+0000 7f00fddf1740 1 --2- 172.21.15.94:0/3364147373 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 0x55973f59e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.124+0000 7f00fddf1740 1 --2- 172.21.15.94:0/3364147373 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55973f5b14f0 0x55973f5b3910 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.124+0000 7f00fddf1740 1 --2- 172.21.15.94:0/3364147373 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f58f060 0x55973f59e070 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.125+0000 7f00d3fff640 0 client.26189 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:02:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.125+0000 7f00fddf1740 1 -- 172.21.15.94:0/3364147373 >> 172.21.15.94:0/3364147373 conn(0x55973f589980 msgr2=0x55973f58b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.125+0000 7f00fddf1740 1 -- 172.21.15.94:0/3364147373 shutdown_connections 2024-02-17T12:02:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.125+0000 7f00fddf1740 1 -- 172.21.15.94:0/3364147373 wait complete. 2024-02-17T12:02:53.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.125+0000 7f00fddf1740 20 client.26189 trim_cache size 0 max 16384 2024-02-17T12:02:53.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.144+0000 7f00fddf1740 1 Processor -- start 2024-02-17T12:02:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.144+0000 7f00fddf1740 1 -- start start 2024-02-17T12:02:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.144+0000 7f00fddf1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f58f060 0x55973f5ad550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.144+0000 7f00fddf1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58fa10 0x55973f59e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.144+0000 7f00fddf1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f5a0f60 0x55973f588a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.144+0000 7f00fddf1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973f47c130 con 0x55973f58fa10 2024-02-17T12:02:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.144+0000 7f00fddf1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973f5850e0 con 0x55973f58f060 2024-02-17T12:02:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.144+0000 7f00fddf1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973f585370 con 0x55973f5a0f60 2024-02-17T12:02:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.144+0000 7f00f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58fa10 0x55973f59e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.144+0000 7f00f96e8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f5a0f60 0x55973f588a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.144+0000 7f00f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58fa10 0x55973f59e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38484/0 (socket says 172.21.15.94:38484) 2024-02-17T12:02:53.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.144+0000 7f00f3fff640 1 -- 172.21.15.94:0/902852937 learned_addr learned my addr 172.21.15.94:0/902852937 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.144+0000 7f00f8ee7640 1 --2- 172.21.15.94:0/902852937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f58f060 0x55973f5ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.145+0000 7f00f96e8640 1 -- 172.21.15.94:0/902852937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f58f060 msgr2=0x55973f5ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.145+0000 7f00f96e8640 1 --2- 172.21.15.94:0/902852937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f58f060 0x55973f5ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.145+0000 7f00f96e8640 1 -- 172.21.15.94:0/902852937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58fa10 msgr2=0x55973f59e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.145+0000 7f00f96e8640 1 --2- 172.21.15.94:0/902852937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58fa10 0x55973f59e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.145+0000 7f00f96e8640 1 -- 172.21.15.94:0/902852937 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55973f491c20 con 0x55973f5a0f60 2024-02-17T12:02:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.145+0000 7f00f3fff640 1 --2- 172.21.15.94:0/902852937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58fa10 0x55973f59e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.145+0000 7f00f96e8640 1 --2- 172.21.15.94:0/902852937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f5a0f60 0x55973f588a00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55973f58ab10 tx=0x7f00ec09afd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4027c40f64d3a1bd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.145+0000 7f00de7fc640 1 -- 172.21.15.94:0/902852937 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0cc120 con 0x55973f5a0f60 2024-02-17T12:02:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.145+0000 7f00de7fc640 1 -- 172.21.15.94:0/902852937 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00ec0a3770 con 0x55973f5a0f60 2024-02-17T12:02:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.145+0000 7f00de7fc640 1 -- 172.21.15.94:0/902852937 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0a32e0 con 0x55973f5a0f60 2024-02-17T12:02:53.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.146+0000 7f00fddf1740 1 -- 172.21.15.94:0/902852937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f5a0f60 msgr2=0x55973f588a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.146+0000 7f00fddf1740 1 --2- 172.21.15.94:0/902852937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f5a0f60 0x55973f588a00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55973f58ab10 tx=0x7f00ec09afd0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.146+0000 7f00fddf1740 1 -- 172.21.15.94:0/902852937 shutdown_connections 2024-02-17T12:02:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.146+0000 7f00fddf1740 1 --2- 172.21.15.94:0/902852937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f5a0f60 0x55973f588a00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.146+0000 7f00fddf1740 1 --2- 172.21.15.94:0/902852937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58fa10 0x55973f59e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.146+0000 7f00fddf1740 1 --2- 172.21.15.94:0/902852937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f58f060 0x55973f5ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.146+0000 7f00fddf1740 1 -- 172.21.15.94:0/902852937 >> 172.21.15.94:0/902852937 conn(0x55973f589980 msgr2=0x55973f5882f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.147+0000 7f00fddf1740 1 -- 172.21.15.94:0/902852937 shutdown_connections 2024-02-17T12:02:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.147+0000 7f00fddf1740 1 -- 172.21.15.94:0/902852937 wait complete. 2024-02-17T12:02:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00fddf1740 1 -- start start 2024-02-17T12:02:53.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00fddf1740 1 Processor -- start 2024-02-17T12:02:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00fddf1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 0x55973f5a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00fddf1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 0x55973f5a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00fddf1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f5a0f60 0x55973f600b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00fddf1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973f601070 con 0x55973f58f060 2024-02-17T12:02:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00fddf1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973f6011e0 con 0x55973f5a0f60 2024-02-17T12:02:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00fddf1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55973f6013a0 con 0x55973f58fa10 2024-02-17T12:02:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00fddf1740 10 client.0 _ll_register_callbacks cb 0x7f00ec097270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00f8ee7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 0x55973f5a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00f8ee7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 0x55973f5a74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38494/0 (socket says 172.21.15.94:38494) 2024-02-17T12:02:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00f8ee7640 1 -- 172.21.15.94:0/3947826225 learned_addr learned my addr 172.21.15.94:0/3947826225 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00f96e8640 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f5a0f60 0x55973f600b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.148+0000 7f00f3fff640 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 0x55973f5a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00f8ee7640 1 -- 172.21.15.94:0/3947826225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 msgr2=0x55973f5a79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00f8ee7640 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 0x55973f5a79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00f8ee7640 1 -- 172.21.15.94:0/3947826225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f5a0f60 msgr2=0x55973f600b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00f8ee7640 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f5a0f60 0x55973f600b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00f8ee7640 1 -- 172.21.15.94:0/3947826225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e0012050 con 0x55973f58f060 2024-02-17T12:02:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00f96e8640 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f5a0f60 0x55973f600b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00f8ee7640 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 0x55973f5a74b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f00e8005b60 tx=0x7f00e80035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19419db360aacbf2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:02:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00d3fff640 1 -- 172.21.15.94:0/3947826225 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e806a070 con 0x55973f58f060 2024-02-17T12:02:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00d3fff640 1 -- 172.21.15.94:0/3947826225 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f00e802c530 con 0x55973f58f060 2024-02-17T12:02:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00fddf1740 1 -- 172.21.15.94:0/3947826225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00ec0cbfb0 con 0x55973f58f060 2024-02-17T12:02:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00d3fff640 1 -- 172.21.15.94:0/3947826225 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00e802c210 con 0x55973f58f060 2024-02-17T12:02:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00fddf1740 10 client.16740 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00fddf1740 1 -- 172.21.15.94:0/3947826225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55973f4b9060 con 0x55973f58f060 2024-02-17T12:02:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.149+0000 7f00d3fff640 1 -- 172.21.15.94:0/3947826225 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00ec0cbfb0 con 0x55973f58f060 2024-02-17T12:02:53.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.150+0000 7f00d3fff640 1 -- 172.21.15.94:0/3947826225 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f00e806d020 con 0x55973f58f060 2024-02-17T12:02:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.150+0000 7f00fddf1740 10 client.16740 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.150+0000 7f00fddf1740 10 client.16740 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:02:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.150+0000 7f00fddf1740 20 client.16740 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.150+0000 7f00fddf1740 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55973f609b00 0x55973f60bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.150+0000 7f00fddf1740 4 client.16740 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:02:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.150+0000 7f00fddf1740 1 -- 172.21.15.94:0/3947826225 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55973f282890 con 0x55973f609b00 2024-02-17T12:02:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.150+0000 7f00f3fff640 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55973f609b00 0x55973f60bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.150+0000 7f00d3fff640 10 client.16740 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:02:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.150+0000 7f00f3fff640 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55973f609b00 0x55973f60bf20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46ca305ced32e118 server_cookie=ef51e51e15ee5c4a in_seq=0 out_seq=0 2024-02-17T12:02:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.151+0000 7f00d3fff640 1 -- 172.21.15.94:0/3947826225 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55973f282890 con 0x55973f609b00 2024-02-17T12:02:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.151+0000 7f00d3fff640 10 client.16740 handle_command_reply: tid=0 2024-02-17T12:02:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.151+0000 7f00fddf1740 1 client.16740 shutdown 2024-02-17T12:02:53.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.151+0000 7f00fddf1740 1 -- 172.21.15.94:0/3947826225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 msgr2=0x55973f5a74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.151+0000 7f00fddf1740 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 0x55973f5a74b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f00e8005b60 tx=0x7f00e80035f0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.151+0000 7f00fddf1740 1 -- 172.21.15.94:0/3947826225 shutdown_connections 2024-02-17T12:02:53.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.151+0000 7f00fddf1740 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55973f609b00 0x55973f60bf20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.151+0000 7f00fddf1740 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55973f5a0f60 0x55973f600b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.151+0000 7f00fddf1740 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55973f58fa10 0x55973f5a79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.151+0000 7f00fddf1740 1 --2- 172.21.15.94:0/3947826225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55973f58f060 0x55973f5a74b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.151+0000 7f00d3fff640 0 client.16740 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:02:53.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.151+0000 7f00fddf1740 1 -- 172.21.15.94:0/3947826225 >> 172.21.15.94:0/3947826225 conn(0x55973f589980 msgr2=0x55973f588b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.152+0000 7f00fddf1740 1 -- 172.21.15.94:0/3947826225 shutdown_connections 2024-02-17T12:02:53.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.152+0000 7f00fddf1740 1 -- 172.21.15.94:0/3947826225 wait complete. 2024-02-17T12:02:53.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.152+0000 7f00fddf1740 20 client.16740 trim_cache size 0 max 16384 2024-02-17T12:02:53.159 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:02:53.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.153+0000 7f00fb14b640 1 -- 172.21.15.94:0/379992228 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f00e40a22f0 msgr2=0x7f00e40a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.153+0000 7f00fb14b640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f00e40a22f0 0x7f00e40a47b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009830 tx=0x7f00e802c0b0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.153+0000 7f00fb14b640 1 -- 172.21.15.94:0/379992228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f406b8d0 msgr2=0x7f00f406bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.153+0000 7f00fb14b640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f406b8d0 0x7f00f406bd60 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00eb10 tx=0x7f00ec00ef10 comp rx=0 tx=0).stop 2024-02-17T12:02:53.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.153+0000 7f00fb14b640 1 -- 172.21.15.94:0/379992228 shutdown_connections 2024-02-17T12:02:53.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.153+0000 7f00fb14b640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f00e40a22f0 0x7f00e40a47b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.153+0000 7f00fb14b640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f00f406b8d0 0x7f00f406bd60 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.153+0000 7f00fb14b640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f00f4106450 0x7f00f4072d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.153+0000 7f00fb14b640 1 --2- 172.21.15.94:0/379992228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00f40ff890 0x7f00f4072800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.153+0000 7f00fb14b640 1 -- 172.21.15.94:0/379992228 >> 172.21.15.94:0/379992228 conn(0x7f00f40765a0 msgr2=0x7f00f41046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.153+0000 7f00fb14b640 1 -- 172.21.15.94:0/379992228 shutdown_connections 2024-02-17T12:02:53.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.153+0000 7f00fb14b640 1 -- 172.21.15.94:0/379992228 wait complete. 2024-02-17T12:02:53.166 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:02:53.166 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:02:53.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.322+0000 7f513958a640 1 -- 172.21.15.94:0/2032333619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5134101d30 msgr2=0x7f5134105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.322+0000 7f513958a640 1 --2- 172.21.15.94:0/2032333619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5134101d30 0x7f5134105d70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5128009930 tx=0x7f512802bc30 comp rx=0 tx=0).stop 2024-02-17T12:02:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.322+0000 7f513958a640 1 -- 172.21.15.94:0/2032333619 shutdown_connections 2024-02-17T12:02:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.322+0000 7f513958a640 1 --2- 172.21.15.94:0/2032333619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5134106450 0x7f513410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.322+0000 7f513958a640 1 --2- 172.21.15.94:0/2032333619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5134101d30 0x7f5134105d70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.322+0000 7f513958a640 1 --2- 172.21.15.94:0/2032333619 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5134101380 0x7f5134101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.322+0000 7f513958a640 1 -- 172.21.15.94:0/2032333619 >> 172.21.15.94:0/2032333619 conn(0x7f5134078540 msgr2=0x7f5134078950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.322+0000 7f513958a640 1 -- 172.21.15.94:0/2032333619 shutdown_connections 2024-02-17T12:02:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.322+0000 7f513958a640 1 -- 172.21.15.94:0/2032333619 wait complete. 2024-02-17T12:02:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.322+0000 7f513958a640 1 Processor -- start 2024-02-17T12:02:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.322+0000 7f513958a640 1 -- start start 2024-02-17T12:02:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f513958a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5134101380 0x7f5134073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f513958a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5134106450 0x7f51340739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f513958a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5134073f00 0x7f51341a59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f513958a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51341a5fb0 con 0x7f5134073f00 2024-02-17T12:02:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f513958a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51341a6120 con 0x7f5134106450 2024-02-17T12:02:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f513958a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51341a62e0 con 0x7f5134101380 2024-02-17T12:02:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f51327fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5134106450 0x7f51340739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f51327fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5134106450 0x7f51340739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:57922/0 (socket says 172.21.15.94:57922) 2024-02-17T12:02:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f51327fc640 1 -- 172.21.15.94:0/936817591 learned_addr learned my addr 172.21.15.94:0/936817591 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f5132ffd640 1 --2- 172.21.15.94:0/936817591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5134101380 0x7f5134073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f51327fc640 1 -- 172.21.15.94:0/936817591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5134101380 msgr2=0x7f5134073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f51327fc640 1 --2- 172.21.15.94:0/936817591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5134101380 0x7f5134073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f51327fc640 1 -- 172.21.15.94:0/936817591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5134073f00 msgr2=0x7f51341a59e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:02:53.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f51327fc640 1 --2- 172.21.15.94:0/936817591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5134073f00 0x7f51341a59e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.323+0000 7f51327fc640 1 -- 172.21.15.94:0/936817591 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5128009560 con 0x7f5134106450 2024-02-17T12:02:53.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.324+0000 7f51327fc640 1 --2- 172.21.15.94:0/936817591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5134106450 0x7f51340739c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5128009230 tx=0x7f512802c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.324+0000 7f511bfff640 1 -- 172.21.15.94:0/936817591 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128004350 con 0x7f5134106450 2024-02-17T12:02:53.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.325+0000 7f511bfff640 1 -- 172.21.15.94:0/936817591 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51280044b0 con 0x7f5134106450 2024-02-17T12:02:53.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.325+0000 7f511bfff640 1 -- 172.21.15.94:0/936817591 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128003da0 con 0x7f5134106450 2024-02-17T12:02:53.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.325+0000 7f513958a640 1 -- 172.21.15.94:0/936817591 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51341a6560 con 0x7f5134106450 2024-02-17T12:02:53.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.325+0000 7f513958a640 1 -- 172.21.15.94:0/936817591 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51341a6a20 con 0x7f5134106450 2024-02-17T12:02:53.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f511bfff640 1 -- 172.21.15.94:0/936817591 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5128004df0 con 0x7f5134106450 2024-02-17T12:02:53.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f513c233740 1 Processor -- start 2024-02-17T12:02:53.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f513c233740 1 -- start start 2024-02-17T12:02:53.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f513c233740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e18250 0x55cb29e18630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f513c233740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e21210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f513c233740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e21750 0x55cb29e23b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f513c233740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb29dd3960 con 0x55cb29e21750 2024-02-17T12:02:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f513c233740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb29d87310 con 0x55cb29e18250 2024-02-17T12:02:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f513c233740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb29d84840 con 0x55cb29e18b70 2024-02-17T12:02:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f51327fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e21750 0x55cb29e23b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f51337fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e18250 0x55cb29e18630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f5132ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e21210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f51327fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e21750 0x55cb29e23b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38504/0 (socket says 172.21.15.94:38504) 2024-02-17T12:02:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f51337fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e18250 0x55cb29e18630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:57930/0 (socket says 172.21.15.94:57930) 2024-02-17T12:02:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f51327fc640 1 -- 172.21.15.94:0/3089078749 learned_addr learned my addr 172.21.15.94:0/3089078749 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.328+0000 7f5132ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e21210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:52484/0 (socket says 172.21.15.94:52484) 2024-02-17T12:02:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.329+0000 7f511bfff640 1 --2- 172.21.15.94:0/936817591 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f511c0a23b0 0x7f511c0a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.329+0000 7f5132ffd640 1 -- 172.21.15.94:0/3089078749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e18250 msgr2=0x55cb29e18630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.329+0000 7f5132ffd640 1 --2- 172.21.15.94:0/3089078749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e18250 0x55cb29e18630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.329+0000 7f511bfff640 1 -- 172.21.15.94:0/936817591 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f51280f0060 con 0x7f5134106450 2024-02-17T12:02:53.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.329+0000 7f5132ffd640 1 -- 172.21.15.94:0/3089078749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e21750 msgr2=0x55cb29e23b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.329+0000 7f5132ffd640 1 --2- 172.21.15.94:0/3089078749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e21750 0x55cb29e23b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.329+0000 7f5132ffd640 1 -- 172.21.15.94:0/3089078749 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb29d42060 con 0x55cb29e18b70 2024-02-17T12:02:53.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.329+0000 7f51337fe640 1 --2- 172.21.15.94:0/3089078749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e18250 0x55cb29e18630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:53.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.329+0000 7f51327fc640 1 --2- 172.21.15.94:0/3089078749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e21750 0x55cb29e23b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:53.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.329+0000 7f51337fe640 1 --2- 172.21.15.94:0/936817591 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f511c0a23b0 0x7f511c0a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.329+0000 7f5132ffd640 1 --2- 172.21.15.94:0/3089078749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e21210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5124009b70 tx=0x7f5124009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b95da7ac81e288b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.329+0000 7f51197fa640 1 -- 172.21.15.94:0/3089078749 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124011da0 con 0x55cb29e18b70 2024-02-17T12:02:53.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.329+0000 7f51197fa640 1 -- 172.21.15.94:0/3089078749 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5124012930 con 0x55cb29e18b70 2024-02-17T12:02:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.330+0000 7f513c233740 1 -- 172.21.15.94:0/3089078749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 msgr2=0x55cb29e21210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.330+0000 7f513c233740 1 --2- 172.21.15.94:0/3089078749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e21210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5124009b70 tx=0x7f5124009f70 comp rx=0 tx=0).stop 2024-02-17T12:02:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.330+0000 7f51337fe640 1 --2- 172.21.15.94:0/936817591 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f511c0a23b0 0x7f511c0a4870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f512c005fd0 tx=0x7f512c004ab0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.330+0000 7f513c233740 1 -- 172.21.15.94:0/3089078749 shutdown_connections 2024-02-17T12:02:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.330+0000 7f513c233740 1 --2- 172.21.15.94:0/3089078749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e21750 0x55cb29e23b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.330+0000 7f513c233740 1 --2- 172.21.15.94:0/3089078749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e21210 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.330+0000 7f513c233740 1 --2- 172.21.15.94:0/3089078749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e18250 0x55cb29e18630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.330+0000 7f513c233740 1 -- 172.21.15.94:0/3089078749 >> 172.21.15.94:0/3089078749 conn(0x55cb29e12980 msgr2=0x55cb29e12d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.331+0000 7f513c233740 1 -- 172.21.15.94:0/3089078749 shutdown_connections 2024-02-17T12:02:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.331+0000 7f513c233740 1 -- 172.21.15.94:0/3089078749 wait complete. 2024-02-17T12:02:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.331+0000 7f513c233740 1 -- start start 2024-02-17T12:02:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.331+0000 7f513c233740 1 Processor -- start 2024-02-17T12:02:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f513c233740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18250 0x55cb29e1e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f513c233740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e18b70 0x55cb29e1eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f513c233740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e21750 0x55cb29e1f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f513c233740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb29d80020 con 0x55cb29e21750 2024-02-17T12:02:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f513c233740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb29a53430 con 0x55cb29e18b70 2024-02-17T12:02:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f513c233740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb29a0fe00 con 0x55cb29e18250 2024-02-17T12:02:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f5132ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18250 0x55cb29e1e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f5132ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18250 0x55cb29e1e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:52494/0 (socket says 172.21.15.94:52494) 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f5132ffd640 1 -- 172.21.15.94:0/2388681780 learned_addr learned my addr 172.21.15.94:0/2388681780 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f51327fc640 1 --2- 172.21.15.94:0/2388681780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e18b70 0x55cb29e1eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f51337fe640 1 --2- 172.21.15.94:0/2388681780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e21750 0x55cb29e1f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f513c233740 10 client.0 _ll_register_callbacks cb 0x55cb29d27cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f5132ffd640 1 -- 172.21.15.94:0/2388681780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e18b70 msgr2=0x55cb29e1eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f5132ffd640 1 --2- 172.21.15.94:0/2388681780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e18b70 0x55cb29e1eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f5132ffd640 1 -- 172.21.15.94:0/2388681780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e21750 msgr2=0x55cb29e1f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f5132ffd640 1 --2- 172.21.15.94:0/2388681780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e21750 0x55cb29e1f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f5132ffd640 1 -- 172.21.15.94:0/2388681780 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51240097a0 con 0x55cb29e18250 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f5132ffd640 1 --2- 172.21.15.94:0/2388681780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18250 0x55cb29e1e600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f512400f9f0 tx=0x7f5124011e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6b0b8c43da5c57b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.332+0000 7f5116ff5640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.333+0000 7f5116ff5640 1 -- 172.21.15.94:0/2388681780 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124012aa0 con 0x55cb29e18250 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.333+0000 7f5116ff5640 1 -- 172.21.15.94:0/2388681780 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5124012c00 con 0x55cb29e18250 2024-02-17T12:02:53.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.333+0000 7f513c233740 1 -- 172.21.15.94:0/2388681780 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb29a7f7e0 con 0x55cb29e18250 2024-02-17T12:02:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.333+0000 7f5116ff5640 1 -- 172.21.15.94:0/2388681780 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5124012dc0 con 0x55cb29e18250 2024-02-17T12:02:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.333+0000 7f513c233740 10 client.26254 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.333+0000 7f513c233740 1 -- 172.21.15.94:0/2388681780 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55cb29d7e6a0 con 0x55cb29e18250 2024-02-17T12:02:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.333+0000 7f5116ff5640 1 -- 172.21.15.94:0/2388681780 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cb29a7f7e0 con 0x55cb29e18250 2024-02-17T12:02:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.333+0000 7f5116ff5640 1 -- 172.21.15.94:0/2388681780 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f512402a020 con 0x55cb29e18250 2024-02-17T12:02:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.334+0000 7f513c233740 10 client.26254 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.334+0000 7f513c233740 10 client.26254 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:02:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.334+0000 7f513c233740 20 client.26254 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:53.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.334+0000 7f513c233740 1 --2- 172.21.15.94:0/2388681780 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cb29e3a340 0x55cb29e3c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.334+0000 7f513c233740 4 client.26254 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.334+0000 7f513c233740 1 -- 172.21.15.94:0/2388681780 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb29abc720 con 0x55cb29e3a340 2024-02-17T12:02:53.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.334+0000 7f51327fc640 1 --2- 172.21.15.94:0/2388681780 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cb29e3a340 0x55cb29e3c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.335+0000 7f51327fc640 1 --2- 172.21.15.94:0/2388681780 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cb29e3a340 0x55cb29e3c770 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8bf8e61953b3eafe server_cookie=ab71907f2bc77ec2 in_seq=0 out_seq=0 2024-02-17T12:02:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.335+0000 7f5116ff5640 10 client.26254 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:02:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.335+0000 7f5116ff5640 1 -- 172.21.15.94:0/2388681780 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb29abc720 con 0x55cb29e3a340 2024-02-17T12:02:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.335+0000 7f5116ff5640 10 client.26254 handle_command_reply: tid=0 2024-02-17T12:02:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.335+0000 7f513c233740 1 client.26254 shutdown 2024-02-17T12:02:53.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.336+0000 7f513c233740 1 -- 172.21.15.94:0/2388681780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18250 msgr2=0x55cb29e1e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.336+0000 7f513c233740 1 --2- 172.21.15.94:0/2388681780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18250 0x55cb29e1e600 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f512400f9f0 tx=0x7f5124011e50 comp rx=0 tx=0).stop 2024-02-17T12:02:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.336+0000 7f513c233740 1 -- 172.21.15.94:0/2388681780 shutdown_connections 2024-02-17T12:02:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.336+0000 7f513c233740 1 --2- 172.21.15.94:0/2388681780 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cb29e3a340 0x55cb29e3c770 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.336+0000 7f513c233740 1 --2- 172.21.15.94:0/2388681780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e21750 0x55cb29e1f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.336+0000 7f513c233740 1 --2- 172.21.15.94:0/2388681780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e18b70 0x55cb29e1eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.336+0000 7f513c233740 1 --2- 172.21.15.94:0/2388681780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18250 0x55cb29e1e600 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.336+0000 7f513c233740 1 -- 172.21.15.94:0/2388681780 >> 172.21.15.94:0/2388681780 conn(0x55cb29e12980 msgr2=0x55cb29e146d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.336+0000 7f5116ff5640 0 client.26254 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:02:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.336+0000 7f513c233740 1 -- 172.21.15.94:0/2388681780 shutdown_connections 2024-02-17T12:02:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.336+0000 7f513c233740 1 -- 172.21.15.94:0/2388681780 wait complete. 2024-02-17T12:02:53.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.336+0000 7f513c233740 20 client.26254 trim_cache size 0 max 16384 2024-02-17T12:02:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f513c233740 1 Processor -- start 2024-02-17T12:02:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f513c233740 1 -- start start 2024-02-17T12:02:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f513c233740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e18250 0x55cb29e20d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f513c233740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e1dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f513c233740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 0x55cb29e11a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f513c233740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb29d05130 con 0x55cb29e18250 2024-02-17T12:02:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f513c233740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb29e17c90 con 0x55cb29e21750 2024-02-17T12:02:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f513c233740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb29e17eb0 con 0x55cb29e18b70 2024-02-17T12:02:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f5132ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e18250 0x55cb29e20d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f51337fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 0x55cb29e11a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f51327fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e1dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f51337fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 0x55cb29e11a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:57950/0 (socket says 172.21.15.94:57950) 2024-02-17T12:02:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f51337fe640 1 -- 172.21.15.94:0/4243178997 learned_addr learned my addr 172.21.15.94:0/4243178997 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.353+0000 7f5132ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e18250 0x55cb29e20d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38510/0 (socket says 172.21.15.94:38510) 2024-02-17T12:02:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.354+0000 7f51337fe640 1 -- 172.21.15.94:0/4243178997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 msgr2=0x55cb29e1dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.354+0000 7f51337fe640 1 --2- 172.21.15.94:0/4243178997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e1dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.354+0000 7f51337fe640 1 -- 172.21.15.94:0/4243178997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e18250 msgr2=0x55cb29e20d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.354+0000 7f51337fe640 1 --2- 172.21.15.94:0/4243178997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e18250 0x55cb29e20d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.354+0000 7f51337fe640 1 -- 172.21.15.94:0/4243178997 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb29d1ac20 con 0x55cb29e21750 2024-02-17T12:02:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.354+0000 7f51327fc640 1 --2- 172.21.15.94:0/4243178997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e1dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.354+0000 7f5132ffd640 1 --2- 172.21.15.94:0/4243178997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e18250 0x55cb29e20d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:53.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.354+0000 7f51337fe640 1 --2- 172.21.15.94:0/4243178997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 0x55cb29e11a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f512c00e790 tx=0x7f512c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77da06d0b684ae99 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.354+0000 7f5118ff9640 1 -- 172.21.15.94:0/4243178997 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c021050 con 0x55cb29e21750 2024-02-17T12:02:53.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.354+0000 7f5118ff9640 1 -- 172.21.15.94:0/4243178997 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f512c022040 con 0x55cb29e21750 2024-02-17T12:02:53.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.354+0000 7f5118ff9640 1 -- 172.21.15.94:0/4243178997 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c01a660 con 0x55cb29e21750 2024-02-17T12:02:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.356+0000 7f513c233740 1 -- 172.21.15.94:0/4243178997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 msgr2=0x55cb29e11a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.356+0000 7f513c233740 1 --2- 172.21.15.94:0/4243178997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 0x55cb29e11a00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f512c00e790 tx=0x7f512c017000 comp rx=0 tx=0).stop 2024-02-17T12:02:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.356+0000 7f513c233740 1 -- 172.21.15.94:0/4243178997 shutdown_connections 2024-02-17T12:02:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.356+0000 7f513c233740 1 --2- 172.21.15.94:0/4243178997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 0x55cb29e11a00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.356+0000 7f513c233740 1 --2- 172.21.15.94:0/4243178997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e1dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.356+0000 7f513c233740 1 --2- 172.21.15.94:0/4243178997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e18250 0x55cb29e20d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.356+0000 7f513c233740 1 -- 172.21.15.94:0/4243178997 >> 172.21.15.94:0/4243178997 conn(0x55cb29e12980 msgr2=0x55cb29e112f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.356+0000 7f513c233740 1 -- 172.21.15.94:0/4243178997 shutdown_connections 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.356+0000 7f513c233740 1 -- 172.21.15.94:0/4243178997 wait complete. 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f513c233740 1 -- start start 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f513c233740 1 Processor -- start 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f513c233740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e18250 0x55cb29e31140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f513c233740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e2f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f513c233740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 0x55cb29e2fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f513c233740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb29e2ffb0 con 0x55cb29e18250 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f513c233740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb29e30120 con 0x55cb29e21750 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f513c233740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb29e302e0 con 0x55cb29e18b70 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f513c233740 10 client.0 _ll_register_callbacks cb 0x7f5124009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:53.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f51327fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e2f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f51337fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 0x55cb29e2fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f5132ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e18250 0x55cb29e31140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f5132ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e18250 0x55cb29e31140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38512/0 (socket says 172.21.15.94:38512) 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f51337fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 0x55cb29e2fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:57958/0 (socket says 172.21.15.94:57958) 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f5132ffd640 1 -- 172.21.15.94:0/149674828 learned_addr learned my addr 172.21.15.94:0/149674828 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.357+0000 7f51327fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e2f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:52516/0 (socket says 172.21.15.94:52516) 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.358+0000 7f51337fe640 1 -- 172.21.15.94:0/149674828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 msgr2=0x55cb29e2f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.358+0000 7f51337fe640 1 --2- 172.21.15.94:0/149674828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e2f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.358+0000 7f51337fe640 1 -- 172.21.15.94:0/149674828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e18250 msgr2=0x55cb29e31140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.358+0000 7f51337fe640 1 --2- 172.21.15.94:0/149674828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e18250 0x55cb29e31140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.358+0000 7f51337fe640 1 -- 172.21.15.94:0/149674828 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb29d7e6a0 con 0x55cb29e21750 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.358+0000 7f51337fe640 1 --2- 172.21.15.94:0/149674828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 0x55cb29e2fa20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f512c004a80 tx=0x7f512c00ef80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce1ea754c47fffd7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.358+0000 7f5116ff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.358+0000 7f5116ff5640 1 -- 172.21.15.94:0/149674828 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c004d20 con 0x55cb29e21750 2024-02-17T12:02:53.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.358+0000 7f5116ff5640 1 -- 172.21.15.94:0/149674828 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f512c005020 con 0x55cb29e21750 2024-02-17T12:02:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.358+0000 7f513c233740 1 -- 172.21.15.94:0/149674828 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5124012930 con 0x55cb29e21750 2024-02-17T12:02:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.359+0000 7f5116ff5640 1 -- 172.21.15.94:0/149674828 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f512c01cb60 con 0x55cb29e21750 2024-02-17T12:02:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.359+0000 7f513c233740 10 client.26213 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.359+0000 7f513c233740 1 -- 172.21.15.94:0/149674828 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f51341a6560 con 0x55cb29e21750 2024-02-17T12:02:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.359+0000 7f5116ff5640 1 -- 172.21.15.94:0/149674828 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5124012930 con 0x55cb29e21750 2024-02-17T12:02:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.359+0000 7f5116ff5640 1 -- 172.21.15.94:0/149674828 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f512c02f5e0 con 0x55cb29e21750 2024-02-17T12:02:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.359+0000 7f513c233740 10 client.26213 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.359+0000 7f513c233740 10 client.26213 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:02:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.359+0000 7f513c233740 20 client.26213 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.359+0000 7f513c233740 1 --2- 172.21.15.94:0/149674828 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cb29e91df0 0x55cb29e94210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.359+0000 7f513c233740 4 client.26213 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:02:53.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.359+0000 7f513c233740 1 -- 172.21.15.94:0/149674828 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cb29b0b890 con 0x55cb29e91df0 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.359+0000 7f5132ffd640 1 --2- 172.21.15.94:0/149674828 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cb29e91df0 0x55cb29e94210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f5116ff5640 10 client.26213 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f5132ffd640 1 --2- 172.21.15.94:0/149674828 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cb29e91df0 0x55cb29e94210 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=909cdb20117b1721 server_cookie=8e62f38b4b3d266c in_seq=0 out_seq=0 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f5116ff5640 1 -- 172.21.15.94:0/149674828 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cb29b0b890 con 0x55cb29e91df0 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f5116ff5640 10 client.26213 handle_command_reply: tid=0 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f513c233740 1 client.26213 shutdown 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f513c233740 1 -- 172.21.15.94:0/149674828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 msgr2=0x55cb29e2fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f513c233740 1 --2- 172.21.15.94:0/149674828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 0x55cb29e2fa20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f512c004a80 tx=0x7f512c00ef80 comp rx=0 tx=0).stop 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f513c233740 1 -- 172.21.15.94:0/149674828 shutdown_connections 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f513c233740 1 --2- 172.21.15.94:0/149674828 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55cb29e91df0 0x55cb29e94210 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f513c233740 1 --2- 172.21.15.94:0/149674828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb29e21750 0x55cb29e2fa20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f5116ff5640 0 client.26213 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f513c233740 1 --2- 172.21.15.94:0/149674828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb29e18b70 0x55cb29e2f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f513c233740 1 --2- 172.21.15.94:0/149674828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb29e18250 0x55cb29e31140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.360+0000 7f513c233740 1 -- 172.21.15.94:0/149674828 >> 172.21.15.94:0/149674828 conn(0x55cb29e12980 msgr2=0x55cb29e11b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.361+0000 7f513c233740 1 -- 172.21.15.94:0/149674828 shutdown_connections 2024-02-17T12:02:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.361+0000 7f513c233740 1 -- 172.21.15.94:0/149674828 wait complete. 2024-02-17T12:02:53.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.361+0000 7f513c233740 20 client.26213 trim_cache size 0 max 16384 2024-02-17T12:02:53.366 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:02:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.361+0000 7f513958a640 1 -- 172.21.15.94:0/936817591 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f511c0a23b0 msgr2=0x7f511c0a4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.361+0000 7f513958a640 1 --2- 172.21.15.94:0/936817591 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f511c0a23b0 0x7f511c0a4870 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f512c005fd0 tx=0x7f512c004ab0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.361+0000 7f513958a640 1 -- 172.21.15.94:0/936817591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5134106450 msgr2=0x7f51340739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.361+0000 7f513958a640 1 --2- 172.21.15.94:0/936817591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5134106450 0x7f51340739c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5128009230 tx=0x7f512802c680 comp rx=0 tx=0).stop 2024-02-17T12:02:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.362+0000 7f513958a640 1 -- 172.21.15.94:0/936817591 shutdown_connections 2024-02-17T12:02:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.362+0000 7f513958a640 1 --2- 172.21.15.94:0/936817591 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f511c0a23b0 0x7f511c0a4870 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.362+0000 7f513958a640 1 --2- 172.21.15.94:0/936817591 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5134073f00 0x7f51341a59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.362+0000 7f513958a640 1 --2- 172.21.15.94:0/936817591 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5134106450 0x7f51340739c0 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5128009230 tx=0x7f512802c680 comp rx=0 tx=0).stop 2024-02-17T12:02:53.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.362+0000 7f513958a640 1 --2- 172.21.15.94:0/936817591 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5134101380 0x7f5134073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.362+0000 7f513958a640 1 -- 172.21.15.94:0/936817591 >> 172.21.15.94:0/936817591 conn(0x7f5134078540 msgr2=0x7f51340ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.363+0000 7f513958a640 1 -- 172.21.15.94:0/936817591 shutdown_connections 2024-02-17T12:02:53.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.363+0000 7f513958a640 1 -- 172.21.15.94:0/936817591 wait complete. 2024-02-17T12:02:53.378 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:02:53.379 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:02:53.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.534+0000 7f4bcabd0640 1 -- 172.21.15.94:0/1737291626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bc41053f0 msgr2=0x7f4bc410cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.534+0000 7f4bcabd0640 1 --2- 172.21.15.94:0/1737291626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bc41053f0 0x7f4bc410cfa0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009cd0 tx=0x7f4bbc02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.534+0000 7f4bcabd0640 1 -- 172.21.15.94:0/1737291626 shutdown_connections 2024-02-17T12:02:53.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.534+0000 7f4bcabd0640 1 --2- 172.21.15.94:0/1737291626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bc41053f0 0x7f4bc410cfa0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.534+0000 7f4bcabd0640 1 --2- 172.21.15.94:0/1737291626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bc41017e0 0x7f4bc4104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.534+0000 7f4bcabd0640 1 --2- 172.21.15.94:0/1737291626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bc4100e30 0x7f4bc4101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.534+0000 7f4bcabd0640 1 -- 172.21.15.94:0/1737291626 >> 172.21.15.94:0/1737291626 conn(0x7f4bc4075430 msgr2=0x7f4bc4075840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.535+0000 7f4bcabd0640 1 -- 172.21.15.94:0/1737291626 shutdown_connections 2024-02-17T12:02:53.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.535+0000 7f4bcabd0640 1 -- 172.21.15.94:0/1737291626 wait complete. 2024-02-17T12:02:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bcabd0640 1 Processor -- start 2024-02-17T12:02:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bcabd0640 1 -- start start 2024-02-17T12:02:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bcabd0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bc4100e30 0x7f4bc410cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bcabd0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bc41053f0 0x7f4bc4108e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bcabd0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bc4109340 0x7f4bc41097f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bcabd0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc41aa230 con 0x7f4bc41053f0 2024-02-17T12:02:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bcabd0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc41aa3a0 con 0x7f4bc4100e30 2024-02-17T12:02:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bcabd0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc41aa560 con 0x7f4bc4109340 2024-02-17T12:02:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bbbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bc41053f0 0x7f4bc4108e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bbbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bc41053f0 0x7f4bc4108e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38540/0 (socket says 172.21.15.94:38540) 2024-02-17T12:02:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bc916d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bc4109340 0x7f4bc41097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bbbfff640 1 -- 172.21.15.94:0/2741974196 learned_addr learned my addr 172.21.15.94:0/2741974196 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bc896c640 1 --2- 172.21.15.94:0/2741974196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bc4100e30 0x7f4bc410cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bbbfff640 1 -- 172.21.15.94:0/2741974196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bc4109340 msgr2=0x7f4bc41097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bbbfff640 1 --2- 172.21.15.94:0/2741974196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bc4109340 0x7f4bc41097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bbbfff640 1 -- 172.21.15.94:0/2741974196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bc4100e30 msgr2=0x7f4bc410cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bbbfff640 1 --2- 172.21.15.94:0/2741974196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bc4100e30 0x7f4bc410cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.536+0000 7f4bbbfff640 1 -- 172.21.15.94:0/2741974196 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc009900 con 0x7f4bc41053f0 2024-02-17T12:02:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.537+0000 7f4bbbfff640 1 --2- 172.21.15.94:0/2741974196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bc41053f0 0x7f4bc4108e00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400e9f0 tx=0x7f4bb400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.537+0000 7f4bb9ffb640 1 -- 172.21.15.94:0/2741974196 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40096c0 con 0x7f4bc41053f0 2024-02-17T12:02:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.537+0000 7f4bcabd0640 1 -- 172.21.15.94:0/2741974196 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc41aa840 con 0x7f4bc41053f0 2024-02-17T12:02:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.537+0000 7f4bb9ffb640 1 -- 172.21.15.94:0/2741974196 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bb400ccc0 con 0x7f4bc41053f0 2024-02-17T12:02:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.537+0000 7f4bb9ffb640 1 -- 172.21.15.94:0/2741974196 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4010430 con 0x7f4bc41053f0 2024-02-17T12:02:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.537+0000 7f4bcabd0640 1 -- 172.21.15.94:0/2741974196 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc41aad20 con 0x7f4bc41053f0 2024-02-17T12:02:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.540+0000 7f4bb9ffb640 1 -- 172.21.15.94:0/2741974196 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4bb40046c0 con 0x7f4bc41053f0 2024-02-17T12:02:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.540+0000 7f4bcd7a1740 1 Processor -- start 2024-02-17T12:02:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.540+0000 7f4bcd7a1740 1 -- start start 2024-02-17T12:02:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.540+0000 7f4bb9ffb640 1 --2- 172.21.15.94:0/2741974196 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4bac0a22e0 0x7f4bac0a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bcd7a1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680efd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bcd7a1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680efdb70 0x558680f06210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bc916d640 1 --2- 172.21.15.94:0/2741974196 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4bac0a22e0 0x7f4bac0a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bcd7a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680f06750 0x558680f08b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bcd7a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680eb8960 con 0x558680f06750 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bcd7a1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680e6c310 con 0x558680efdb70 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bcd7a1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680e69840 con 0x558680efd250 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bb9ffb640 1 -- 172.21.15.94:0/2741974196 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4bb4014070 con 0x7f4bc41053f0 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bc916d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680f06750 0x558680f08b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bc916d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680f06750 0x558680f08b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38552/0 (socket says 172.21.15.94:38552) 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bc916d640 1 -- 172.21.15.94:0/3046660020 learned_addr learned my addr 172.21.15.94:0/3046660020 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bbbfff640 1 --2- 172.21.15.94:0/3046660020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680efdb70 0x558680f06210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bc896c640 1 --2- 172.21.15.94:0/3046660020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680efd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bc916d640 1 -- 172.21.15.94:0/3046660020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 msgr2=0x558680efd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bc916d640 1 --2- 172.21.15.94:0/3046660020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680efd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bc916d640 1 -- 172.21.15.94:0/3046660020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680efdb70 msgr2=0x558680f06210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bc916d640 1 --2- 172.21.15.94:0/3046660020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680efdb70 0x558680f06210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.541+0000 7f4bc916d640 1 -- 172.21.15.94:0/3046660020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558680e27060 con 0x558680f06750 2024-02-17T12:02:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.542+0000 7f4bc916d640 1 --2- 172.21.15.94:0/2741974196 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4bac0a22e0 0x7f4bac0a47a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc032650 tx=0x7f4bbc032340 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.542+0000 7f4bc916d640 1 --2- 172.21.15.94:0/3046660020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680f06750 0x558680f08b40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc006010 tx=0x7f4bbc048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10134cc685de9a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.543+0000 7f4baaffd640 1 -- 172.21.15.94:0/3046660020 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc048070 con 0x558680f06750 2024-02-17T12:02:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.543+0000 7f4baaffd640 1 -- 172.21.15.94:0/3046660020 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bbc004b40 con 0x558680f06750 2024-02-17T12:02:53.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.543+0000 7f4baaffd640 1 -- 172.21.15.94:0/3046660020 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc004040 con 0x558680f06750 2024-02-17T12:02:53.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.544+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/3046660020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680f06750 msgr2=0x558680f08b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.544+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/3046660020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680f06750 0x558680f08b40 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc006010 tx=0x7f4bbc048000 comp rx=0 tx=0).stop 2024-02-17T12:02:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.544+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/3046660020 shutdown_connections 2024-02-17T12:02:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.544+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/3046660020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680f06750 0x558680f08b40 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.544+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/3046660020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680efdb70 0x558680f06210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.544+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/3046660020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680efd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.544+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/3046660020 >> 172.21.15.94:0/3046660020 conn(0x558680ef7980 msgr2=0x558680ef7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.544+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/3046660020 shutdown_connections 2024-02-17T12:02:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.544+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/3046660020 wait complete. 2024-02-17T12:02:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bcd7a1740 1 -- start start 2024-02-17T12:02:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bcd7a1740 1 Processor -- start 2024-02-17T12:02:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bcd7a1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bcd7a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680efdb70 0x558680f03ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bcd7a1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680f06750 0x558680f04220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bcd7a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680e65020 con 0x558680efdb70 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bcd7a1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680b38430 con 0x558680f06750 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bcd7a1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680af4e00 con 0x558680efd250 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bcd7a1740 10 client.0 _ll_register_callbacks cb 0x558680e0ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bbbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680efdb70 0x558680f03ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bbbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680efdb70 0x558680f03ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:38560/0 (socket says 172.21.15.94:38560) 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bbbfff640 1 -- 172.21.15.94:0/4168569039 learned_addr learned my addr 172.21.15.94:0/4168569039 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bc896c640 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.545+0000 7f4bc916d640 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680f06750 0x558680f04220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4bc896c640 1 -- 172.21.15.94:0/4168569039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680f06750 msgr2=0x558680f04220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4bc896c640 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680f06750 0x558680f04220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4bc896c640 1 -- 172.21.15.94:0/4168569039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680efdb70 msgr2=0x558680f03ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4bc896c640 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680efdb70 0x558680f03ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4bc896c640 1 -- 172.21.15.94:0/4168569039 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc410d7b0 con 0x558680efd250 2024-02-17T12:02:53.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4bbbfff640 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680efdb70 0x558680f03ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:53.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4bc896c640 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f037a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0010870 tx=0x7f4bc0010d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68af677a04a90ae4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4ba3fff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:02:53.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4ba3fff640 1 -- 172.21.15.94:0/4168569039 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0015070 con 0x558680efd250 2024-02-17T12:02:53.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/4168569039 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558680b647e0 con 0x558680efd250 2024-02-17T12:02:53.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4ba3fff640 1 -- 172.21.15.94:0/4168569039 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4bc000fb90 con 0x558680efd250 2024-02-17T12:02:53.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4ba3fff640 1 -- 172.21.15.94:0/4168569039 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bc000fd50 con 0x558680efd250 2024-02-17T12:02:53.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4bcd7a1740 10 client.26272 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:53.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/4168569039 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558680e636a0 con 0x558680efd250 2024-02-17T12:02:53.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.546+0000 7f4ba3fff640 1 -- 172.21.15.94:0/4168569039 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bc0026020 con 0x558680efd250 2024-02-17T12:02:53.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.547+0000 7f4ba3fff640 1 -- 172.21.15.94:0/4168569039 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4bc0018490 con 0x558680efd250 2024-02-17T12:02:53.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.547+0000 7f4bcd7a1740 10 client.26272 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:53.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.547+0000 7f4bcd7a1740 10 client.26272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:53.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.547+0000 7f4bcd7a1740 20 client.26272 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:53.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.547+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558680f16bc0 0x558680f18fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.547+0000 7f4bcd7a1740 4 client.26272 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:53.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.547+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/4168569039 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558680ba1720 con 0x558680f16bc0 2024-02-17T12:02:53.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.548+0000 7f4bbbfff640 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558680f16bc0 0x558680f18fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.548+0000 7f4ba3fff640 10 client.26272 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:53.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.548+0000 7f4bbbfff640 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558680f16bc0 0x558680f18fe0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4329b1683fd6cb8d server_cookie=66cf2d9bbc3b7326 in_seq=0 out_seq=0 2024-02-17T12:02:53.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4ba3fff640 1 -- 172.21.15.94:0/4168569039 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558680ba1720 con 0x558680f16bc0 2024-02-17T12:02:53.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4ba3fff640 10 client.26272 handle_command_reply: tid=0 2024-02-17T12:02:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4bcd7a1740 1 client.26272 shutdown 2024-02-17T12:02:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/4168569039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 msgr2=0x558680f037a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f037a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0010870 tx=0x7f4bc0010d30 comp rx=0 tx=0).stop 2024-02-17T12:02:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/4168569039 shutdown_connections 2024-02-17T12:02:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558680f16bc0 0x558680f18fe0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680f06750 0x558680f04220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680efdb70 0x558680f03ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4ba3fff640 0 client.26272 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/4168569039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f037a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/4168569039 >> 172.21.15.94:0/4168569039 conn(0x558680ef7980 msgr2=0x558680ef96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/4168569039 shutdown_connections 2024-02-17T12:02:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/4168569039 wait complete. 2024-02-17T12:02:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.549+0000 7f4bcd7a1740 20 client.26272 trim_cache size 0 max 16384 2024-02-17T12:02:53.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.565+0000 7f4bcd7a1740 1 Processor -- start 2024-02-17T12:02:53.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.565+0000 7f4bcd7a1740 1 -- start start 2024-02-17T12:02:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bcd7a1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f05ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bcd7a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680efdb70 0x558680f04040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bcd7a1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680f06750 0x558680efea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bcd7a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680dea130 con 0x558680efdb70 2024-02-17T12:02:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bcd7a1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680efc5d0 con 0x558680f06750 2024-02-17T12:02:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bcd7a1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680f1e260 con 0x558680efd250 2024-02-17T12:02:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bc896c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f05ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bc896c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f05ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:52556/0 (socket says 172.21.15.94:52556) 2024-02-17T12:02:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bc896c640 1 -- 172.21.15.94:0/1061235936 learned_addr learned my addr 172.21.15.94:0/1061235936 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bbbfff640 1 --2- 172.21.15.94:0/1061235936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680efdb70 0x558680f04040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bc916d640 1 --2- 172.21.15.94:0/1061235936 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680f06750 0x558680efea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bc896c640 1 -- 172.21.15.94:0/1061235936 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680f06750 msgr2=0x558680efea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bc896c640 1 --2- 172.21.15.94:0/1061235936 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680f06750 0x558680efea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bc896c640 1 -- 172.21.15.94:0/1061235936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680efdb70 msgr2=0x558680f04040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bc896c640 1 --2- 172.21.15.94:0/1061235936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680efdb70 0x558680f04040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bc896c640 1 -- 172.21.15.94:0/1061235936 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558680dffc20 con 0x558680efd250 2024-02-17T12:02:53.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.566+0000 7f4bc896c640 1 --2- 172.21.15.94:0/1061235936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f05ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000a7e0 tx=0x7f4bc0005670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88ffd80a0583e8f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.567+0000 7f4baa7fc640 1 -- 172.21.15.94:0/1061235936 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0015040 con 0x558680efd250 2024-02-17T12:02:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.567+0000 7f4baa7fc640 1 -- 172.21.15.94:0/1061235936 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bc0026050 con 0x558680efd250 2024-02-17T12:02:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.567+0000 7f4baa7fc640 1 -- 172.21.15.94:0/1061235936 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0022680 con 0x558680efd250 2024-02-17T12:02:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.569+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/1061235936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 msgr2=0x558680f05ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.569+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/1061235936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f05ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4bc000a7e0 tx=0x7f4bc0005670 comp rx=0 tx=0).stop 2024-02-17T12:02:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.569+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/1061235936 shutdown_connections 2024-02-17T12:02:53.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.569+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/1061235936 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680f06750 0x558680efea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.569+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/1061235936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680efdb70 0x558680f04040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.569+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/1061235936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f05ed0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.569+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/1061235936 >> 172.21.15.94:0/1061235936 conn(0x558680ef7980 msgr2=0x558680ef2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.569+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/1061235936 shutdown_connections 2024-02-17T12:02:53.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.569+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/1061235936 wait complete. 2024-02-17T12:02:53.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.569+0000 7f4bcd7a1740 1 -- start start 2024-02-17T12:02:53.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.569+0000 7f4bcd7a1740 1 Processor -- start 2024-02-17T12:02:53.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bcd7a1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f0c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bcd7a1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680efdb70 0x558680f0c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bcd7a1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680f06750 0x558680f0ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bcd7a1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680f6d280 con 0x558680f06750 2024-02-17T12:02:53.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bcd7a1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680f6d3f0 con 0x558680efdb70 2024-02-17T12:02:53.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bcd7a1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558680f6d5b0 con 0x558680efd250 2024-02-17T12:02:53.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bcd7a1740 10 client.0 _ll_register_callbacks cb 0x7f4bb4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:53.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bc896c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f0c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bc896c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f0c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:52562/0 (socket says 172.21.15.94:52562) 2024-02-17T12:02:53.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bc896c640 1 -- 172.21.15.94:0/1123903776 learned_addr learned my addr 172.21.15.94:0/1123903776 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:53.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bbbfff640 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680efdb70 0x558680f0c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bc916d640 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680f06750 0x558680f0ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bc896c640 1 -- 172.21.15.94:0/1123903776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680efdb70 msgr2=0x558680f0c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bc896c640 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680efdb70 0x558680f0c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bc896c640 1 -- 172.21.15.94:0/1123903776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680f06750 msgr2=0x558680f0ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bc896c640 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680f06750 0x558680f0ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.571+0000 7f4bc896c640 1 -- 172.21.15.94:0/1123903776 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558680e636a0 con 0x558680efd250 2024-02-17T12:02:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.572+0000 7f4bbbfff640 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680efdb70 0x558680f0c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.572+0000 7f4bc896c640 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f0c230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0018870 tx=0x7f4bc000a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5685607642ebe51a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.572+0000 7f4ba3fff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:02:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.572+0000 7f4ba3fff640 1 -- 172.21.15.94:0/1123903776 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc000fb90 con 0x558680efd250 2024-02-17T12:02:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.572+0000 7f4ba3fff640 1 -- 172.21.15.94:0/1123903776 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4bc0000e40 con 0x558680efd250 2024-02-17T12:02:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.572+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/1123903776 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4bbc004b40 con 0x558680efd250 2024-02-17T12:02:53.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.572+0000 7f4ba3fff640 1 -- 172.21.15.94:0/1123903776 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bc0034450 con 0x558680efd250 2024-02-17T12:02:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.572+0000 7f4bcd7a1740 10 client.26278 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.572+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/1123903776 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x558680e258c0 con 0x558680efd250 2024-02-17T12:02:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.572+0000 7f4ba3fff640 1 -- 172.21.15.94:0/1123903776 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4bc0041020 con 0x558680efd250 2024-02-17T12:02:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.573+0000 7f4ba3fff640 1 -- 172.21.15.94:0/1123903776 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4bc0032070 con 0x558680efd250 2024-02-17T12:02:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.573+0000 7f4bcd7a1740 10 client.26278 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.573+0000 7f4bcd7a1740 10 client.26278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.573+0000 7f4bcd7a1740 20 client.26278 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:53.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.573+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558680f6e9c0 0x558680f70db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.573+0000 7f4bcd7a1740 4 client.26278 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:02:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.573+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/1123903776 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558680bf0890 con 0x558680f6e9c0 2024-02-17T12:02:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.573+0000 7f4bbbfff640 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558680f6e9c0 0x558680f70db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.574+0000 7f4bbbfff640 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558680f6e9c0 0x558680f70db0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cb981e15b17a774 server_cookie=1e4bc2c98696df5 in_seq=0 out_seq=0 2024-02-17T12:02:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.574+0000 7f4ba3fff640 10 client.26278 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.574+0000 7f4ba3fff640 1 -- 172.21.15.94:0/1123903776 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558680bf0890 con 0x558680f6e9c0 2024-02-17T12:02:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.574+0000 7f4ba3fff640 10 client.26278 handle_command_reply: tid=0 2024-02-17T12:02:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.574+0000 7f4bcd7a1740 1 client.26278 shutdown 2024-02-17T12:02:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.575+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/1123903776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 msgr2=0x558680f0c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.575+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f0c230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0018870 tx=0x7f4bc000a420 comp rx=0 tx=0).stop 2024-02-17T12:02:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.575+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/1123903776 shutdown_connections 2024-02-17T12:02:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.575+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558680f6e9c0 0x558680f70db0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.575+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558680f06750 0x558680f0ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.575+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558680efdb70 0x558680f0c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.575+0000 7f4bcd7a1740 1 --2- 172.21.15.94:0/1123903776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558680efd250 0x558680f0c230 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.575+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/1123903776 >> 172.21.15.94:0/1123903776 conn(0x558680ef7980 msgr2=0x558680efea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.575+0000 7f4ba3fff640 0 client.26278 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.575+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/1123903776 shutdown_connections 2024-02-17T12:02:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.575+0000 7f4bcd7a1740 1 -- 172.21.15.94:0/1123903776 wait complete. 2024-02-17T12:02:53.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.575+0000 7f4bcd7a1740 20 client.26278 trim_cache size 0 max 16384 2024-02-17T12:02:53.581 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:02:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.576+0000 7f4bcabd0640 1 -- 172.21.15.94:0/2741974196 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4bac0a22e0 msgr2=0x7f4bac0a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.576+0000 7f4bcabd0640 1 --2- 172.21.15.94:0/2741974196 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4bac0a22e0 0x7f4bac0a47a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc032650 tx=0x7f4bbc032340 comp rx=0 tx=0).stop 2024-02-17T12:02:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.576+0000 7f4bcabd0640 1 -- 172.21.15.94:0/2741974196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bc41053f0 msgr2=0x7f4bc4108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.576+0000 7f4bcabd0640 1 --2- 172.21.15.94:0/2741974196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bc41053f0 0x7f4bc4108e00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400e9f0 tx=0x7f4bb400edf0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.576+0000 7f4bcabd0640 1 -- 172.21.15.94:0/2741974196 shutdown_connections 2024-02-17T12:02:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.576+0000 7f4bcabd0640 1 --2- 172.21.15.94:0/2741974196 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4bac0a22e0 0x7f4bac0a47a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.576+0000 7f4bcabd0640 1 --2- 172.21.15.94:0/2741974196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4bc4109340 0x7f4bc41097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.576+0000 7f4bcabd0640 1 --2- 172.21.15.94:0/2741974196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4bc41053f0 0x7f4bc4108e00 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.576+0000 7f4bcabd0640 1 --2- 172.21.15.94:0/2741974196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4bc4100e30 0x7f4bc410cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:53.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.576+0000 7f4bcabd0640 1 -- 172.21.15.94:0/2741974196 >> 172.21.15.94:0/2741974196 conn(0x7f4bc4075430 msgr2=0x7f4bc40fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:53.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.576+0000 7f4bcabd0640 1 -- 172.21.15.94:0/2741974196 shutdown_connections 2024-02-17T12:02:53.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:53.576+0000 7f4bcabd0640 1 -- 172.21.15.94:0/2741974196 wait complete. 2024-02-17T12:02:54.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:53 smithi125 ceph-mon[36819]: pgmap v146: 97 pgs: 97 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 781 op/s 2024-02-17T12:02:54.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:53 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2465673667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:02:54.205 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:53 smithi094 ceph-mon[32499]: pgmap v146: 97 pgs: 97 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 781 op/s 2024-02-17T12:02:54.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:53 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2465673667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:02:54.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:53 smithi189 ceph-mon[36588]: pgmap v146: 97 pgs: 97 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.9 MiB/s wr, 781 op/s 2024-02-17T12:02:54.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:53 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2465673667' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:02:54.593 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:02:54.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:02:54.593 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:02:54.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.758+0000 7f8d45464640 1 -- 172.21.15.94:0/567272287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d40101370 msgr2=0x7f8d40101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:54.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.758+0000 7f8d45464640 1 --2- 172.21.15.94:0/567272287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d40101370 0x7f8d40101750 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009930 tx=0x7f8d3002bc30 comp rx=0 tx=0).stop 2024-02-17T12:02:54.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.759+0000 7f8d45464640 1 -- 172.21.15.94:0/567272287 shutdown_connections 2024-02-17T12:02:54.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.759+0000 7f8d45464640 1 --2- 172.21.15.94:0/567272287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d40106440 0x7f8d4010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.759+0000 7f8d45464640 1 --2- 172.21.15.94:0/567272287 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d40101d20 0x7f8d40105d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.759+0000 7f8d45464640 1 --2- 172.21.15.94:0/567272287 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d40101370 0x7f8d40101750 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.759+0000 7f8d45464640 1 -- 172.21.15.94:0/567272287 >> 172.21.15.94:0/567272287 conn(0x7f8d400fcc10 msgr2=0x7f8d400ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:54.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.759+0000 7f8d45464640 1 -- 172.21.15.94:0/567272287 shutdown_connections 2024-02-17T12:02:54.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.759+0000 7f8d45464640 1 -- 172.21.15.94:0/567272287 wait complete. 2024-02-17T12:02:54.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.760+0000 7f8d45464640 1 Processor -- start 2024-02-17T12:02:54.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.760+0000 7f8d45464640 1 -- start start 2024-02-17T12:02:54.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.760+0000 7f8d45464640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d40101370 0x7f8d40073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.761+0000 7f8d45464640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d40106440 0x7f8d400739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.761+0000 7f8d3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d40101370 0x7f8d40073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.761+0000 7f8d45464640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d40075120 0x7f8d40073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.761+0000 7f8d45464640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401aa3b0 con 0x7f8d40101370 2024-02-17T12:02:54.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.761+0000 7f8d45464640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401aa520 con 0x7f8d40106440 2024-02-17T12:02:54.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.761+0000 7f8d45464640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401aa6e0 con 0x7f8d40075120 2024-02-17T12:02:54.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.761+0000 7f8d3f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d40075120 0x7f8d40073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.761+0000 7f8d3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d40101370 0x7f8d40073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54370/0 (socket says 172.21.15.94:54370) 2024-02-17T12:02:54.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.761+0000 7f8d3effd640 1 -- 172.21.15.94:0/1603594494 learned_addr learned my addr 172.21.15.94:0/1603594494 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:54.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.761+0000 7f8d3e7fc640 1 --2- 172.21.15.94:0/1603594494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d40106440 0x7f8d400739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.762+0000 7f8d3f7fe640 1 -- 172.21.15.94:0/1603594494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d40106440 msgr2=0x7f8d400739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:54.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.762+0000 7f8d3f7fe640 1 --2- 172.21.15.94:0/1603594494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d40106440 0x7f8d400739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.762+0000 7f8d3f7fe640 1 -- 172.21.15.94:0/1603594494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d40101370 msgr2=0x7f8d40073480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:54.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.762+0000 7f8d3f7fe640 1 --2- 172.21.15.94:0/1603594494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d40101370 0x7f8d40073480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.762+0000 7f8d3f7fe640 1 -- 172.21.15.94:0/1603594494 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d30009560 con 0x7f8d40075120 2024-02-17T12:02:54.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.762+0000 7f8d3f7fe640 1 --2- 172.21.15.94:0/1603594494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d40075120 0x7f8d40073f00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009fd0 tx=0x7f8d3800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:54.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.762+0000 7f8d27fff640 1 -- 172.21.15.94:0/1603594494 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800ce20 con 0x7f8d40075120 2024-02-17T12:02:54.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.763+0000 7f8d45464640 1 -- 172.21.15.94:0/1603594494 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d401aa9c0 con 0x7f8d40075120 2024-02-17T12:02:54.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.763+0000 7f8d45464640 1 -- 172.21.15.94:0/1603594494 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d401aaf20 con 0x7f8d40075120 2024-02-17T12:02:54.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.763+0000 7f8d27fff640 1 -- 172.21.15.94:0/1603594494 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d38005020 con 0x7f8d40075120 2024-02-17T12:02:54.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.763+0000 7f8d27fff640 1 -- 172.21.15.94:0/1603594494 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38010690 con 0x7f8d40075120 2024-02-17T12:02:54.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.765+0000 7f8d27fff640 1 -- 172.21.15.94:0/1603594494 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8d38004a40 con 0x7f8d40075120 2024-02-17T12:02:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.765+0000 7f8d48038740 1 Processor -- start 2024-02-17T12:02:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.765+0000 7f8d48038740 1 -- start start 2024-02-17T12:02:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.765+0000 7f8d48038740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3380 0x55703c2a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.765+0000 7f8d48038740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2a3d30 0x55703c2ac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.765+0000 7f8d48038740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2ac8b0 0x55703c2aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d27fff640 1 --2- 172.21.15.94:0/1603594494 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8d280a22b0 0x7f8d280a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d48038740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703c211f90 con 0x55703c2ac8b0 2024-02-17T12:02:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d48038740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703c20f4c0 con 0x55703c2a3380 2024-02-17T12:02:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d48038740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703c20aca0 con 0x55703c2a3d30 2024-02-17T12:02:54.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d3effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2a3d30 0x55703c2ac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d3f7fe640 1 --2- 172.21.15.94:0/1603594494 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8d280a22b0 0x7f8d280a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d3e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3380 0x55703c2a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d3effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2a3d30 0x55703c2ac2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:36740/0 (socket says 172.21.15.94:36740) 2024-02-17T12:02:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d3e7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3380 0x55703c2a3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:33992/0 (socket says 172.21.15.94:33992) 2024-02-17T12:02:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d3effd640 1 -- 172.21.15.94:0/3107829374 learned_addr learned my addr 172.21.15.94:0/3107829374 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d3e7fc640 1 --2- 172.21.15.94:0/3107829374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2ac8b0 0x55703c2aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d3effd640 1 -- 172.21.15.94:0/3107829374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3380 msgr2=0x55703c2a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:54.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d3effd640 1 --2- 172.21.15.94:0/3107829374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3380 0x55703c2a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d3effd640 1 -- 172.21.15.94:0/3107829374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2ac8b0 msgr2=0x55703c2aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d3effd640 1 --2- 172.21.15.94:0/3107829374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2ac8b0 0x55703c2aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d3effd640 1 -- 172.21.15.94:0/3107829374 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55703c1cd0b0 con 0x55703c2a3d30 2024-02-17T12:02:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.766+0000 7f8d27fff640 1 -- 172.21.15.94:0/1603594494 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8d38014070 con 0x7f8d40075120 2024-02-17T12:02:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.767+0000 7f8d3effd640 1 --2- 172.21.15.94:0/3107829374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2a3d30 0x55703c2ac2e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009230 tx=0x7f8d3002c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=724b5147945ed2a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.767+0000 7f8d3e7fc640 1 --2- 172.21.15.94:0/3107829374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3380 0x55703c2a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:02:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.767+0000 7f8d257fa640 1 -- 172.21.15.94:0/3107829374 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30004880 con 0x55703c2a3d30 2024-02-17T12:02:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.767+0000 7f8d257fa640 1 -- 172.21.15.94:0/3107829374 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d300049e0 con 0x55703c2a3d30 2024-02-17T12:02:54.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.767+0000 7f8d257fa640 1 -- 172.21.15.94:0/3107829374 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d300050e0 con 0x55703c2a3d30 2024-02-17T12:02:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.768+0000 7f8d48038740 1 -- 172.21.15.94:0/3107829374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2a3d30 msgr2=0x55703c2ac2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.768+0000 7f8d3f7fe640 1 --2- 172.21.15.94:0/1603594494 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8d280a22b0 0x7f8d280a4770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8d3809acb0 tx=0x7f8d38014d30 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.768+0000 7f8d48038740 1 --2- 172.21.15.94:0/3107829374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2a3d30 0x55703c2ac2e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009230 tx=0x7f8d3002c660 comp rx=0 tx=0).stop 2024-02-17T12:02:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.768+0000 7f8d48038740 1 -- 172.21.15.94:0/3107829374 shutdown_connections 2024-02-17T12:02:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.768+0000 7f8d48038740 1 --2- 172.21.15.94:0/3107829374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2ac8b0 0x55703c2aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.768+0000 7f8d48038740 1 --2- 172.21.15.94:0/3107829374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2a3d30 0x55703c2ac2e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.768+0000 7f8d48038740 1 --2- 172.21.15.94:0/3107829374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3380 0x55703c2a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.768+0000 7f8d48038740 1 -- 172.21.15.94:0/3107829374 >> 172.21.15.94:0/3107829374 conn(0x55703c29d840 msgr2=0x55703c29dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d48038740 1 -- 172.21.15.94:0/3107829374 shutdown_connections 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d48038740 1 -- 172.21.15.94:0/3107829374 wait complete. 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d48038740 1 -- start start 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d48038740 1 Processor -- start 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d48038740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3380 0x55703c2a9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d48038740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3d30 0x55703c2a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d48038740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 0x55703c2aa290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d48038740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703c1aff40 con 0x55703c2a3d30 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d48038740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703bede4b0 con 0x55703c2a3380 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d48038740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703c190670 con 0x55703c2ac8b0 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d48038740 10 client.0 _ll_register_callbacks cb 0x55703c1fdf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3d30 0x55703c2a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d3effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3d30 0x55703c2a9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54392/0 (socket says 172.21.15.94:54392) 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d3effd640 1 -- 172.21.15.94:0/433023878 learned_addr learned my addr 172.21.15.94:0/433023878 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d3e7fc640 1 --2- 172.21.15.94:0/433023878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 0x55703c2aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.769+0000 7f8d3e7fc640 1 --2- 172.21.15.94:0/433023878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3380 0x55703c2a9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d3effd640 1 -- 172.21.15.94:0/433023878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 msgr2=0x55703c2aa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d3effd640 1 --2- 172.21.15.94:0/433023878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 0x55703c2aa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d3effd640 1 -- 172.21.15.94:0/433023878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3380 msgr2=0x55703c2a9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d3effd640 1 --2- 172.21.15.94:0/433023878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3380 0x55703c2a9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d3effd640 1 -- 172.21.15.94:0/433023878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4010e820 con 0x55703c2a3d30 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d3effd640 1 --2- 172.21.15.94:0/433023878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3d30 0x55703c2a9d50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009900 tx=0x7f8d30005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d31e78819fc794 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d22ff5640 1 -- 172.21.15.94:0/433023878 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30005070 con 0x55703c2a3d30 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d22ff5640 1 -- 172.21.15.94:0/433023878 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8d300051d0 con 0x55703c2a3d30 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d48038740 1 -- 172.21.15.94:0/433023878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55703c1e4f40 con 0x55703c2a3d30 2024-02-17T12:02:54.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d22ff5640 1 -- 172.21.15.94:0/433023878 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d30005390 con 0x55703c2a3d30 2024-02-17T12:02:54.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d48038740 10 client.16785 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:54.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d48038740 1 -- 172.21.15.94:0/433023878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55703c209320 con 0x55703c2a3d30 2024-02-17T12:02:54.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d22ff5640 1 -- 172.21.15.94:0/433023878 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55703c1e4f40 con 0x55703c2a3d30 2024-02-17T12:02:54.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d22ff5640 1 -- 172.21.15.94:0/433023878 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8d3003c050 con 0x55703c2a3d30 2024-02-17T12:02:54.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d48038740 10 client.16785 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:54.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d48038740 10 client.16785 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:54.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d48038740 20 client.16785 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:54.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d48038740 1 --2- 172.21.15.94:0/433023878 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55703c2bc9e0 0x55703c2bedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d48038740 4 client.16785 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:54.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.770+0000 7f8d48038740 1 -- 172.21.15.94:0/433023878 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55703c201310 con 0x55703c2bc9e0 2024-02-17T12:02:54.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.771+0000 7f8d3e7fc640 1 --2- 172.21.15.94:0/433023878 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55703c2bc9e0 0x55703c2bedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.771+0000 7f8d3e7fc640 1 --2- 172.21.15.94:0/433023878 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55703c2bc9e0 0x55703c2bedd0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ee89d1d57e267 server_cookie=324afa1767a7082e in_seq=0 out_seq=0 2024-02-17T12:02:54.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.771+0000 7f8d22ff5640 10 client.16785 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:54.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.772+0000 7f8d22ff5640 1 -- 172.21.15.94:0/433023878 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55703c201310 con 0x55703c2bc9e0 2024-02-17T12:02:54.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.772+0000 7f8d22ff5640 10 client.16785 handle_command_reply: tid=0 2024-02-17T12:02:54.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.772+0000 7f8d48038740 1 client.16785 shutdown 2024-02-17T12:02:54.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.772+0000 7f8d48038740 1 -- 172.21.15.94:0/433023878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3d30 msgr2=0x55703c2a9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:54.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.772+0000 7f8d48038740 1 --2- 172.21.15.94:0/433023878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3d30 0x55703c2a9d50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009900 tx=0x7f8d30005b50 comp rx=0 tx=0).stop 2024-02-17T12:02:54.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.773+0000 7f8d48038740 1 -- 172.21.15.94:0/433023878 shutdown_connections 2024-02-17T12:02:54.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.773+0000 7f8d48038740 1 --2- 172.21.15.94:0/433023878 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55703c2bc9e0 0x55703c2bedd0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.773+0000 7f8d48038740 1 --2- 172.21.15.94:0/433023878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 0x55703c2aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.773+0000 7f8d48038740 1 --2- 172.21.15.94:0/433023878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3d30 0x55703c2a9d50 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.773+0000 7f8d48038740 1 --2- 172.21.15.94:0/433023878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3380 0x55703c2a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.773+0000 7f8d48038740 1 -- 172.21.15.94:0/433023878 >> 172.21.15.94:0/433023878 conn(0x55703c29d840 msgr2=0x55703c29f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:54.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.773+0000 7f8d22ff5640 0 client.16785 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:54.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.773+0000 7f8d48038740 1 -- 172.21.15.94:0/433023878 shutdown_connections 2024-02-17T12:02:54.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.773+0000 7f8d48038740 1 -- 172.21.15.94:0/433023878 wait complete. 2024-02-17T12:02:54.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.773+0000 7f8d48038740 20 client.16785 trim_cache size 0 max 16384 2024-02-17T12:02:54.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.794+0000 7f8d48038740 1 Processor -- start 2024-02-17T12:02:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.794+0000 7f8d48038740 1 -- start start 2024-02-17T12:02:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d48038740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3380 0x55703c2abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d48038740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3d30 0x55703c2aa0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d48038740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 0x55703c2a87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d48038740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703c297da0 con 0x55703c2a3380 2024-02-17T12:02:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d48038740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703c2a7d60 con 0x55703c2a3d30 2024-02-17T12:02:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d48038740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703c2a53b0 con 0x55703c2ac8b0 2024-02-17T12:02:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d3e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3380 0x55703c2abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d3effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3d30 0x55703c2aa0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d3e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3380 0x55703c2abf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54406/0 (socket says 172.21.15.94:54406) 2024-02-17T12:02:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d3effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3d30 0x55703c2aa0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34014/0 (socket says 172.21.15.94:34014) 2024-02-17T12:02:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d3e7fc640 1 -- 172.21.15.94:0/2836183444 learned_addr learned my addr 172.21.15.94:0/2836183444 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:54.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d3e7fc640 1 --2- 172.21.15.94:0/2836183444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 0x55703c2a87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d3effd640 1 -- 172.21.15.94:0/2836183444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 msgr2=0x55703c2a87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:54.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d3effd640 1 --2- 172.21.15.94:0/2836183444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 0x55703c2a87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d3effd640 1 -- 172.21.15.94:0/2836183444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3380 msgr2=0x55703c2abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:54.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d3effd640 1 --2- 172.21.15.94:0/2836183444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3380 0x55703c2abf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d3effd640 1 -- 172.21.15.94:0/2836183444 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55703c1a5c00 con 0x55703c2a3d30 2024-02-17T12:02:54.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d3effd640 1 --2- 172.21.15.94:0/2836183444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3d30 0x55703c2aa0b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009430 tx=0x7f8d30004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9451a4c0a71dce9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:54.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.795+0000 7f8d24ff9640 1 -- 172.21.15.94:0/2836183444 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30005850 con 0x55703c2a3d30 2024-02-17T12:02:54.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.796+0000 7f8d24ff9640 1 -- 172.21.15.94:0/2836183444 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d30053e40 con 0x55703c2a3d30 2024-02-17T12:02:54.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.796+0000 7f8d24ff9640 1 -- 172.21.15.94:0/2836183444 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d300052d0 con 0x55703c2a3d30 2024-02-17T12:02:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.797+0000 7f8d48038740 1 -- 172.21.15.94:0/2836183444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3d30 msgr2=0x55703c2aa0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.797+0000 7f8d48038740 1 --2- 172.21.15.94:0/2836183444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3d30 0x55703c2aa0b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009430 tx=0x7f8d30004610 comp rx=0 tx=0).stop 2024-02-17T12:02:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.797+0000 7f8d48038740 1 -- 172.21.15.94:0/2836183444 shutdown_connections 2024-02-17T12:02:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.797+0000 7f8d48038740 1 --2- 172.21.15.94:0/2836183444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 0x55703c2a87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.797+0000 7f8d48038740 1 --2- 172.21.15.94:0/2836183444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2a3d30 0x55703c2aa0b0 secure :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009430 tx=0x7f8d30004610 comp rx=0 tx=0).stop 2024-02-17T12:02:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.797+0000 7f8d48038740 1 --2- 172.21.15.94:0/2836183444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3380 0x55703c2abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.797+0000 7f8d48038740 1 -- 172.21.15.94:0/2836183444 >> 172.21.15.94:0/2836183444 conn(0x55703c29d840 msgr2=0x55703c2a80e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.798+0000 7f8d48038740 1 -- 172.21.15.94:0/2836183444 shutdown_connections 2024-02-17T12:02:54.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.798+0000 7f8d48038740 1 -- 172.21.15.94:0/2836183444 wait complete. 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.798+0000 7f8d48038740 1 -- start start 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.798+0000 7f8d48038740 1 Processor -- start 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.798+0000 7f8d48038740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3380 0x55703c2b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.798+0000 7f8d48038740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 0x55703c2b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d48038740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2b3210 0x55703c312ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d48038740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703c313540 con 0x55703c2a3380 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d48038740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703c3136b0 con 0x55703c2b3210 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d48038740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55703c313870 con 0x55703c2ac8b0 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d48038740 10 client.0 _ll_register_callbacks cb 0x7f8d380093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d3e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3380 0x55703c2b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d3e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3380 0x55703c2b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54420/0 (socket says 172.21.15.94:54420) 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d3e7fc640 1 -- 172.21.15.94:0/3594370405 learned_addr learned my addr 172.21.15.94:0/3594370405 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d3effd640 1 --2- 172.21.15.94:0/3594370405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 0x55703c2b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d3effd640 1 -- 172.21.15.94:0/3594370405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2b3210 msgr2=0x55703c312ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:54.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d3effd640 1 --2- 172.21.15.94:0/3594370405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2b3210 0x55703c312ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d3effd640 1 -- 172.21.15.94:0/3594370405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3380 msgr2=0x55703c2b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d3effd640 1 --2- 172.21.15.94:0/3594370405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3380 0x55703c2b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.799+0000 7f8d3effd640 1 -- 172.21.15.94:0/3594370405 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55703c209320 con 0x55703c2ac8b0 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.800+0000 7f8d3effd640 1 --2- 172.21.15.94:0/3594370405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 0x55703c2b2bc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55703c2af770 tx=0x7f8d30053bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c5d6e47c27280a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.800+0000 7f8d22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.800+0000 7f8d22ff5640 1 -- 172.21.15.94:0/3594370405 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30005480 con 0x55703c2ac8b0 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.800+0000 7f8d22ff5640 1 -- 172.21.15.94:0/3594370405 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8d30000e40 con 0x55703c2ac8b0 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.800+0000 7f8d48038740 1 -- 172.21.15.94:0/3594370405 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55703c230900 con 0x55703c2ac8b0 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.800+0000 7f8d22ff5640 1 -- 172.21.15.94:0/3594370405 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d300535b0 con 0x55703c2ac8b0 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.800+0000 7f8d48038740 10 client.26296 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.800+0000 7f8d48038740 1 -- 172.21.15.94:0/3594370405 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55703c1cb910 con 0x55703c2ac8b0 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.801+0000 7f8d22ff5640 1 -- 172.21.15.94:0/3594370405 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8d30070080 con 0x55703c2ac8b0 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.801+0000 7f8d22ff5640 1 -- 172.21.15.94:0/3594370405 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8d3006b070 con 0x55703c2ac8b0 2024-02-17T12:02:54.803 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.801+0000 7f8d48038740 10 client.26296 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.801+0000 7f8d48038740 10 client.26296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.801+0000 7f8d48038740 20 client.26296 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.801+0000 7f8d48038740 1 --2- 172.21.15.94:0/3594370405 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55703c314ba0 0x55703c316f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.801+0000 7f8d48038740 4 client.26296 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:02:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.801+0000 7f8d48038740 1 -- 172.21.15.94:0/3594370405 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55703bf0a860 con 0x55703c314ba0 2024-02-17T12:02:54.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.801+0000 7f8d3e7fc640 1 --2- 172.21.15.94:0/3594370405 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55703c314ba0 0x55703c316f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.802+0000 7f8d3e7fc640 1 --2- 172.21.15.94:0/3594370405 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55703c314ba0 0x55703c316f90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd058a569967a34b server_cookie=aff560d56d7dffe5 in_seq=0 out_seq=0 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.802+0000 7f8d22ff5640 10 client.26296 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.802+0000 7f8d22ff5640 1 -- 172.21.15.94:0/3594370405 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55703bf0a860 con 0x55703c314ba0 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.802+0000 7f8d22ff5640 10 client.26296 handle_command_reply: tid=0 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.802+0000 7f8d48038740 1 client.26296 shutdown 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.803+0000 7f8d48038740 1 -- 172.21.15.94:0/3594370405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 msgr2=0x55703c2b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.803+0000 7f8d48038740 1 --2- 172.21.15.94:0/3594370405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 0x55703c2b2bc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55703c2af770 tx=0x7f8d30053bf0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.803+0000 7f8d48038740 1 -- 172.21.15.94:0/3594370405 shutdown_connections 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.803+0000 7f8d48038740 1 --2- 172.21.15.94:0/3594370405 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55703c314ba0 0x55703c316f90 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.803+0000 7f8d48038740 1 --2- 172.21.15.94:0/3594370405 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55703c2b3210 0x55703c312ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.803+0000 7f8d22ff5640 0 client.26296 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.803+0000 7f8d48038740 1 --2- 172.21.15.94:0/3594370405 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55703c2ac8b0 0x55703c2b2bc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.803+0000 7f8d48038740 1 --2- 172.21.15.94:0/3594370405 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55703c2a3380 0x55703c2b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.803+0000 7f8d48038740 1 -- 172.21.15.94:0/3594370405 >> 172.21.15.94:0/3594370405 conn(0x55703c29d840 msgr2=0x55703c29c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:54.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.803+0000 7f8d48038740 1 -- 172.21.15.94:0/3594370405 shutdown_connections 2024-02-17T12:02:54.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.803+0000 7f8d48038740 1 -- 172.21.15.94:0/3594370405 wait complete. 2024-02-17T12:02:54.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.803+0000 7f8d48038740 20 client.26296 trim_cache size 0 max 16384 2024-02-17T12:02:54.807 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:02:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:02:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "f1054b49-6d92-4c44-a6b2-35896defc6bb", 2024-02-17T12:02:54.807 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:02:54.807 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:02:54.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.804+0000 7f8d45464640 1 -- 172.21.15.94:0/1603594494 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8d280a22b0 msgr2=0x7f8d280a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:54.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.804+0000 7f8d45464640 1 --2- 172.21.15.94:0/1603594494 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8d280a22b0 0x7f8d280a4770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8d3809acb0 tx=0x7f8d38014d30 comp rx=0 tx=0).stop 2024-02-17T12:02:54.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.804+0000 7f8d45464640 1 -- 172.21.15.94:0/1603594494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d40075120 msgr2=0x7f8d40073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:54.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.804+0000 7f8d45464640 1 --2- 172.21.15.94:0/1603594494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d40075120 0x7f8d40073f00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009fd0 tx=0x7f8d3800ef10 comp rx=0 tx=0).stop 2024-02-17T12:02:54.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.804+0000 7f8d45464640 1 -- 172.21.15.94:0/1603594494 shutdown_connections 2024-02-17T12:02:54.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.804+0000 7f8d45464640 1 --2- 172.21.15.94:0/1603594494 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8d280a22b0 0x7f8d280a4770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.804+0000 7f8d45464640 1 --2- 172.21.15.94:0/1603594494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d40075120 0x7f8d40073f00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.804+0000 7f8d45464640 1 --2- 172.21.15.94:0/1603594494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8d40106440 0x7f8d400739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.804+0000 7f8d45464640 1 --2- 172.21.15.94:0/1603594494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8d40101370 0x7f8d40073480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.804+0000 7f8d45464640 1 -- 172.21.15.94:0/1603594494 >> 172.21.15.94:0/1603594494 conn(0x7f8d400fcc10 msgr2=0x7f8d400fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:54.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.805+0000 7f8d45464640 1 -- 172.21.15.94:0/1603594494 shutdown_connections 2024-02-17T12:02:54.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.805+0000 7f8d45464640 1 -- 172.21.15.94:0/1603594494 wait complete. 2024-02-17T12:02:54.819 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:02:54.819 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:02:54.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.985+0000 7fc2fbb7a640 1 -- 172.21.15.94:0/21701265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2f4100ab0 msgr2=0x7fc2f4104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:54.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.985+0000 7fc2fbb7a640 1 --2- 172.21.15.94:0/21701265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2f4100ab0 0x7fc2f4104f70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc2e4009900 tx=0x7fc2e402bcd0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.986+0000 7fc2fbb7a640 1 -- 172.21.15.94:0/21701265 shutdown_connections 2024-02-17T12:02:54.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.986+0000 7fc2fbb7a640 1 --2- 172.21.15.94:0/21701265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f41054b0 0x7fc2f4105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.986+0000 7fc2fbb7a640 1 --2- 172.21.15.94:0/21701265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2f4100ab0 0x7fc2f4104f70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.986+0000 7fc2fbb7a640 1 --2- 172.21.15.94:0/21701265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2f4100190 0x7fc2f4100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.986+0000 7fc2fbb7a640 1 -- 172.21.15.94:0/21701265 >> 172.21.15.94:0/21701265 conn(0x7fc2f4077230 msgr2=0x7fc2f4077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:54.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.986+0000 7fc2fbb7a640 1 -- 172.21.15.94:0/21701265 shutdown_connections 2024-02-17T12:02:54.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.987+0000 7fc2fbb7a640 1 -- 172.21.15.94:0/21701265 wait complete. 2024-02-17T12:02:54.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.987+0000 7fc2fbb7a640 1 Processor -- start 2024-02-17T12:02:54.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.987+0000 7fc2fbb7a640 1 -- start start 2024-02-17T12:02:54.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2fbb7a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2f4100190 0x7fc2f419ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2fbb7a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f4100ab0 0x7fc2f419e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2fbb7a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2f41054b0 0x7fc2f41a2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2fbb7a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f41a2c60 con 0x7fc2f4100190 2024-02-17T12:02:54.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2fbb7a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f41a2da0 con 0x7fc2f41054b0 2024-02-17T12:02:54.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2fbb7a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f41a2f60 con 0x7fc2f4100ab0 2024-02-17T12:02:54.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2f9115640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f4100ab0 0x7fc2f419e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2f9115640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f4100ab0 0x7fc2f419e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:36780/0 (socket says 172.21.15.94:36780) 2024-02-17T12:02:54.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2f9115640 1 -- 172.21.15.94:0/1094133494 learned_addr learned my addr 172.21.15.94:0/1094133494 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:54.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2f9916640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2f4100190 0x7fc2f419ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2fa117640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2f41054b0 0x7fc2f41a2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2f9115640 1 -- 172.21.15.94:0/1094133494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2f41054b0 msgr2=0x7fc2f41a2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:54.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2f9115640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2f41054b0 0x7fc2f41a2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2f9115640 1 -- 172.21.15.94:0/1094133494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2f4100190 msgr2=0x7fc2f419ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:54.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2f9115640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2f4100190 0x7fc2f419ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.988+0000 7fc2f9115640 1 -- 172.21.15.94:0/1094133494 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e4009560 con 0x7fc2f4100ab0 2024-02-17T12:02:54.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.989+0000 7fc2f9115640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f4100ab0 0x7fc2f419e330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc2e402c2e0 tx=0x7fc2e402c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:54.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.989+0000 7fc2eaffd640 1 -- 172.21.15.94:0/1094133494 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e402dc60 con 0x7fc2f4100ab0 2024-02-17T12:02:54.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.989+0000 7fc2eaffd640 1 -- 172.21.15.94:0/1094133494 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2e402ddc0 con 0x7fc2f4100ab0 2024-02-17T12:02:54.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.989+0000 7fc2eaffd640 1 -- 172.21.15.94:0/1094133494 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4003bf0 con 0x7fc2f4100ab0 2024-02-17T12:02:54.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.990+0000 7fc2fbb7a640 1 -- 172.21.15.94:0/1094133494 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f41a31e0 con 0x7fc2f4100ab0 2024-02-17T12:02:54.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.990+0000 7fc2fbb7a640 1 -- 172.21.15.94:0/1094133494 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f4106720 con 0x7fc2f4100ab0 2024-02-17T12:02:54.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.995+0000 7fc2fe82e740 1 Processor -- start 2024-02-17T12:02:54.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.995+0000 7fc2eaffd640 1 -- 172.21.15.94:0/1094133494 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc2e4003d50 con 0x7fc2f4100ab0 2024-02-17T12:02:54.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.995+0000 7fc2fe82e740 1 -- start start 2024-02-17T12:02:54.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.995+0000 7fc2fe82e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c342b0 0x560970c34690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.995+0000 7fc2fe82e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c34bd0 0x560970c3d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.995+0000 7fc2fe82e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c3d7b0 0x560970c3fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.995+0000 7fc2fe82e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970ba3280 con 0x560970c342b0 2024-02-17T12:02:54.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.995+0000 7fc2fe82e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970ba07b0 con 0x560970c3d7b0 2024-02-17T12:02:54.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.995+0000 7fc2fe82e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970b9bf90 con 0x560970c34bd0 2024-02-17T12:02:54.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2fa117640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c342b0 0x560970c34690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2fa117640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c342b0 0x560970c34690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54436/0 (socket says 172.21.15.94:54436) 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2fa117640 1 -- 172.21.15.94:0/2034010725 learned_addr learned my addr 172.21.15.94:0/2034010725 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2f9916640 1 --2- 172.21.15.94:0/2034010725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c34bd0 0x560970c3d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2f9115640 1 --2- 172.21.15.94:0/2034010725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c3d7b0 0x560970c3fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2eaffd640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc2e00a2380 0x7fc2e00a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2f9916640 1 -- 172.21.15.94:0/2034010725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c3d7b0 msgr2=0x560970c3fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2f9916640 1 --2- 172.21.15.94:0/2034010725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c3d7b0 0x560970c3fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2f9916640 1 -- 172.21.15.94:0/2034010725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c342b0 msgr2=0x560970c34690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2f9916640 1 --2- 172.21.15.94:0/2034010725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c342b0 0x560970c34690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2f9916640 1 -- 172.21.15.94:0/2034010725 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560970b5a570 con 0x560970c34bd0 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2fa117640 1 --2- 172.21.15.94:0/2034010725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c342b0 0x560970c34690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2f9115640 1 --2- 172.21.15.94:0/2034010725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c3d7b0 0x560970c3fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.996+0000 7fc2eaffd640 1 -- 172.21.15.94:0/1094133494 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc2e40f01c0 con 0x7fc2f4100ab0 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.997+0000 7fc2f9916640 1 --2- 172.21.15.94:0/2034010725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c34bd0 0x560970c3d270 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009c20 tx=0x7fc2f0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad4eb69ee036e5ec server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:54.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.997+0000 7fc2fa117640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc2e00a2380 0x7fc2e00a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.997+0000 7fc2dffff640 1 -- 172.21.15.94:0/2034010725 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0009dd0 con 0x560970c34bd0 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.997+0000 7fc2dffff640 1 -- 172.21.15.94:0/2034010725 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2f0004510 con 0x560970c34bd0 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.997+0000 7fc2dffff640 1 -- 172.21.15.94:0/2034010725 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0004b00 con 0x560970c34bd0 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.998+0000 7fc2fe82e740 1 -- 172.21.15.94:0/2034010725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c34bd0 msgr2=0x560970c3d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.998+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/2034010725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c34bd0 0x560970c3d270 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009c20 tx=0x7fc2f0011810 comp rx=0 tx=0).stop 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.998+0000 7fc2fa117640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc2e00a2380 0x7fc2e00a4840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec005c40 tx=0x7fc2ec009760 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.998+0000 7fc2fe82e740 1 -- 172.21.15.94:0/2034010725 shutdown_connections 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.998+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/2034010725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c3d7b0 0x560970c3fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.998+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/2034010725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c34bd0 0x560970c3d270 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.998+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/2034010725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c342b0 0x560970c34690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.998+0000 7fc2fe82e740 1 -- 172.21.15.94:0/2034010725 >> 172.21.15.94:0/2034010725 conn(0x560970c2e990 msgr2=0x560970c2ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.998+0000 7fc2fe82e740 1 -- 172.21.15.94:0/2034010725 shutdown_connections 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.998+0000 7fc2fe82e740 1 -- 172.21.15.94:0/2034010725 wait complete. 2024-02-17T12:02:55.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.999+0000 7fc2fe82e740 1 -- start start 2024-02-17T12:02:55.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.999+0000 7fc2fe82e740 1 Processor -- start 2024-02-17T12:02:55.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.999+0000 7fc2fe82e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c342b0 0x560970c3d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:55.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:54.999+0000 7fc2fe82e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c34bd0 0x560970c3a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:55.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.000+0000 7fc2fe82e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c3d7b0 0x560970c3ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:55.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.000+0000 7fc2fe82e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56097086f430 con 0x560970c34bd0 2024-02-17T12:02:55.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.000+0000 7fc2fe82e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56097082be00 con 0x560970c342b0 2024-02-17T12:02:55.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.000+0000 7fc2fe82e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970b20fd0 con 0x560970c3d7b0 2024-02-17T12:02:55.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.000+0000 7fc2fe82e740 10 client.0 _ll_register_callbacks cb 0x560970b76fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.000+0000 7fc2f9916640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c342b0 0x560970c3d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.000+0000 7fc2fa117640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c3d7b0 0x560970c3ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.000+0000 7fc2fa117640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c3d7b0 0x560970c3ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:36806/0 (socket says 172.21.15.94:36806) 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.000+0000 7fc2fa117640 1 -- 172.21.15.94:0/3555642158 learned_addr learned my addr 172.21.15.94:0/3555642158 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.001+0000 7fc2f9115640 1 --2- 172.21.15.94:0/3555642158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c34bd0 0x560970c3a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.001+0000 7fc2fa117640 1 -- 172.21.15.94:0/3555642158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c342b0 msgr2=0x560970c3d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.001+0000 7fc2fa117640 1 --2- 172.21.15.94:0/3555642158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c342b0 0x560970c3d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.001+0000 7fc2fa117640 1 -- 172.21.15.94:0/3555642158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c34bd0 msgr2=0x560970c3a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.001+0000 7fc2fa117640 1 --2- 172.21.15.94:0/3555642158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c34bd0 0x560970c3a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.001+0000 7fc2fa117640 1 -- 172.21.15.94:0/3555642158 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f0009850 con 0x560970c3d7b0 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.001+0000 7fc2f9115640 1 --2- 172.21.15.94:0/3555642158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c34bd0 0x560970c3a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.002+0000 7fc2fa117640 1 --2- 172.21.15.94:0/3555642158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c3d7b0 0x560970c3ad30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009c20 tx=0x7fc2ec004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f968b0e42acbe185 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.002+0000 7fc2dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.002+0000 7fc2dd7fa640 1 -- 172.21.15.94:0/3555642158 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00ecf0 con 0x560970c3d7b0 2024-02-17T12:02:55.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.002+0000 7fc2dd7fa640 1 -- 172.21.15.94:0/3555642158 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc2ec010e70 con 0x560970c3d7b0 2024-02-17T12:02:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.002+0000 7fc2fe82e740 1 -- 172.21.15.94:0/3555642158 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56097089b7e0 con 0x560970c3d7b0 2024-02-17T12:02:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.002+0000 7fc2dd7fa640 1 -- 172.21.15.94:0/3555642158 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2ec00ee50 con 0x560970c3d7b0 2024-02-17T12:02:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.002+0000 7fc2fe82e740 10 client.26308 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.002+0000 7fc2fe82e740 1 -- 172.21.15.94:0/3555642158 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560970b9a610 con 0x560970c3d7b0 2024-02-17T12:02:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.003+0000 7fc2dd7fa640 1 -- 172.21.15.94:0/3555642158 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2ec030920 con 0x560970c3d7b0 2024-02-17T12:02:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.003+0000 7fc2dd7fa640 1 -- 172.21.15.94:0/3555642158 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc2ec025020 con 0x560970c3d7b0 2024-02-17T12:02:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.003+0000 7fc2fe82e740 10 client.26308 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.003+0000 7fc2fe82e740 10 client.26308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.003+0000 7fc2fe82e740 20 client.26308 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.003+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/3555642158 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560970c4d930 0x560970c4fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.003+0000 7fc2fe82e740 4 client.26308 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:02:55.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.003+0000 7fc2fe82e740 1 -- 172.21.15.94:0/3555642158 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56097089b7e0 con 0x560970c4d930 2024-02-17T12:02:55.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.003+0000 7fc2f9916640 1 --2- 172.21.15.94:0/3555642158 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560970c4d930 0x560970c4fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.004+0000 7fc2dd7fa640 10 client.26308 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.004+0000 7fc2f9916640 1 --2- 172.21.15.94:0/3555642158 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560970c4d930 0x560970c4fdb0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec9fd2fe2f5805b server_cookie=948041698e8e48dd in_seq=0 out_seq=0 2024-02-17T12:02:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.005+0000 7fc2dd7fa640 1 -- 172.21.15.94:0/3555642158 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56097089b7e0 con 0x560970c4d930 2024-02-17T12:02:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.005+0000 7fc2dd7fa640 10 client.26308 handle_command_reply: tid=0 2024-02-17T12:02:55.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.005+0000 7fc2fe82e740 1 client.26308 shutdown 2024-02-17T12:02:55.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.006+0000 7fc2fe82e740 1 -- 172.21.15.94:0/3555642158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c3d7b0 msgr2=0x560970c3ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:55.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.006+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/3555642158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c3d7b0 0x560970c3ad30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009c20 tx=0x7fc2ec004a40 comp rx=0 tx=0).stop 2024-02-17T12:02:55.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.006+0000 7fc2fe82e740 1 -- 172.21.15.94:0/3555642158 shutdown_connections 2024-02-17T12:02:55.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.006+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/3555642158 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560970c4d930 0x560970c4fdb0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.006+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/3555642158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c3d7b0 0x560970c3ad30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.006+0000 7fc2dd7fa640 0 client.26308 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:55.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.006+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/3555642158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c34bd0 0x560970c3a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.006+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/3555642158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c342b0 0x560970c3d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.006+0000 7fc2fe82e740 1 -- 172.21.15.94:0/3555642158 >> 172.21.15.94:0/3555642158 conn(0x560970c2e990 msgr2=0x560970c30730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:55.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.006+0000 7fc2fe82e740 1 -- 172.21.15.94:0/3555642158 shutdown_connections 2024-02-17T12:02:55.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.006+0000 7fc2fe82e740 1 -- 172.21.15.94:0/3555642158 wait complete. 2024-02-17T12:02:55.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.006+0000 7fc2fe82e740 20 client.26308 trim_cache size 0 max 16384 2024-02-17T12:02:55.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.029+0000 7fc2fe82e740 1 Processor -- start 2024-02-17T12:02:55.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.029+0000 7fc2fe82e740 1 -- start start 2024-02-17T12:02:55.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.029+0000 7fc2fe82e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c342b0 0x560970c3acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:55.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.029+0000 7fc2fe82e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 0x560970c3cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:55.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.029+0000 7fc2fe82e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c3d7b0 0x560970c3d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:55.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.029+0000 7fc2f9115640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 0x560970c3cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:55.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.029+0000 7fc2f9115640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 0x560970c3cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34050/0 (socket says 172.21.15.94:34050) 2024-02-17T12:02:55.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.029+0000 7fc2f9115640 1 -- 172.21.15.94:0/298537333 learned_addr learned my addr 172.21.15.94:0/298537333 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:55.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.029+0000 7fc2fe82e740 1 -- 172.21.15.94:0/298537333 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56097089b7e0 con 0x560970c3d7b0 2024-02-17T12:02:55.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.029+0000 7fc2fe82e740 1 -- 172.21.15.94:0/298537333 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970c2f9e0 con 0x560970c34bd0 2024-02-17T12:02:55.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.029+0000 7fc2fe82e740 1 -- 172.21.15.94:0/298537333 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970c28e20 con 0x560970c342b0 2024-02-17T12:02:55.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.029+0000 7fc2f9916640 1 --2- 172.21.15.94:0/298537333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c342b0 0x560970c3acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:55.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.030+0000 7fc2fa117640 1 --2- 172.21.15.94:0/298537333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c3d7b0 0x560970c3d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:55.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.030+0000 7fc2f9916640 1 -- 172.21.15.94:0/298537333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 msgr2=0x560970c3cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:55.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.030+0000 7fc2f9916640 1 --2- 172.21.15.94:0/298537333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 0x560970c3cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.030+0000 7fc2f9916640 1 -- 172.21.15.94:0/298537333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c3d7b0 msgr2=0x560970c3d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:55.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.030+0000 7fc2f9916640 1 --2- 172.21.15.94:0/298537333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c3d7b0 0x560970c3d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.030+0000 7fc2f9916640 1 -- 172.21.15.94:0/298537333 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560970b36ac0 con 0x560970c342b0 2024-02-17T12:02:55.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.030+0000 7fc2fa117640 1 --2- 172.21.15.94:0/298537333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c3d7b0 0x560970c3d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:02:55.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.030+0000 7fc2f9115640 1 --2- 172.21.15.94:0/298537333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 0x560970c3cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:02:55.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.030+0000 7fc2f9916640 1 --2- 172.21.15.94:0/298537333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c342b0 0x560970c3acd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2f001b040 tx=0x7fc2f0009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14e9a43c1a948472 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:55.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.030+0000 7fc2df7fe640 1 -- 172.21.15.94:0/298537333 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0029070 con 0x560970c342b0 2024-02-17T12:02:55.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.030+0000 7fc2df7fe640 1 -- 172.21.15.94:0/298537333 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2f0005660 con 0x560970c342b0 2024-02-17T12:02:55.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.031+0000 7fc2df7fe640 1 -- 172.21.15.94:0/298537333 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f002a8b0 con 0x560970c342b0 2024-02-17T12:02:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.031+0000 7fc2fe82e740 1 -- 172.21.15.94:0/298537333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c342b0 msgr2=0x560970c3acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.031+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/298537333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c342b0 0x560970c3acd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2f001b040 tx=0x7fc2f0009680 comp rx=0 tx=0).stop 2024-02-17T12:02:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.031+0000 7fc2fe82e740 1 -- 172.21.15.94:0/298537333 shutdown_connections 2024-02-17T12:02:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.031+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/298537333 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c3d7b0 0x560970c3d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.031+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/298537333 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 0x560970c3cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.031+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/298537333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c342b0 0x560970c3acd0 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc2f001b040 tx=0x7fc2f0009680 comp rx=0 tx=0).stop 2024-02-17T12:02:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.031+0000 7fc2fe82e740 1 -- 172.21.15.94:0/298537333 >> 172.21.15.94:0/298537333 conn(0x560970c2e990 msgr2=0x560970c35850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.032+0000 7fc2fe82e740 1 -- 172.21.15.94:0/298537333 shutdown_connections 2024-02-17T12:02:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.032+0000 7fc2fe82e740 1 -- 172.21.15.94:0/298537333 wait complete. 2024-02-17T12:02:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.032+0000 7fc2fe82e740 1 -- start start 2024-02-17T12:02:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.032+0000 7fc2fe82e740 1 Processor -- start 2024-02-17T12:02:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2fe82e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 0x560970c41180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:55.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2fe82e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c3d7b0 0x560970c416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2fe82e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c4d930 0x560970c41c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2fe82e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970ca3f50 con 0x560970c4d930 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2fe82e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970ca40c0 con 0x560970c34bd0 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2fe82e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560970cb5290 con 0x560970c3d7b0 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2fe82e740 10 client.0 _ll_register_callbacks cb 0x7fc2f00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2f9916640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 0x560970c41180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2f9916640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 0x560970c41180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34058/0 (socket says 172.21.15.94:34058) 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2f9916640 1 -- 172.21.15.94:0/2432363911 learned_addr learned my addr 172.21.15.94:0/2432363911 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2fa117640 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c4d930 0x560970c41c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2f9115640 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c3d7b0 0x560970c416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2f9916640 1 -- 172.21.15.94:0/2432363911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c3d7b0 msgr2=0x560970c416c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2f9916640 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c3d7b0 0x560970c416c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2f9916640 1 -- 172.21.15.94:0/2432363911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c4d930 msgr2=0x560970c41c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:02:55.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2f9916640 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c4d930 0x560970c41c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2f9916640 1 -- 172.21.15.94:0/2432363911 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f41a31e0 con 0x560970c34bd0 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.033+0000 7fc2fa117640 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c4d930 0x560970c41c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.034+0000 7fc2f9916640 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 0x560970c41180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0013040 tx=0x7fc2f002a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aa55aec5155b809 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.034+0000 7fc2dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.034+0000 7fc2dd7fa640 1 -- 172.21.15.94:0/2432363911 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0029040 con 0x560970c34bd0 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.034+0000 7fc2dd7fa640 1 -- 172.21.15.94:0/2432363911 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc2f002a7f0 con 0x560970c34bd0 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.034+0000 7fc2fe82e740 1 -- 172.21.15.94:0/2432363911 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc2f0004510 con 0x560970c34bd0 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.034+0000 7fc2dd7fa640 1 -- 172.21.15.94:0/2432363911 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2f002a9b0 con 0x560970c34bd0 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.034+0000 7fc2fe82e740 10 client.26261 fetch_fsmap learned FSMap version 23 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.034+0000 7fc2fe82e740 1 -- 172.21.15.94:0/2432363911 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x560970b5a570 con 0x560970c34bd0 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.035+0000 7fc2dd7fa640 1 -- 172.21.15.94:0/2432363911 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc2f0004510 con 0x560970c34bd0 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.035+0000 7fc2dd7fa640 1 -- 172.21.15.94:0/2432363911 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc2f0021040 con 0x560970c34bd0 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.035+0000 7fc2fe82e740 10 client.26261 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.035+0000 7fc2fe82e740 10 client.26261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:02:55.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.035+0000 7fc2fe82e740 20 client.26261 populate_metadata read hostname 'smithi094' 2024-02-17T12:02:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.035+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560970cb6790 0x560970cb8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:02:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.035+0000 7fc2fe82e740 4 client.26261 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:02:55.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.035+0000 7fc2fe82e740 1 -- 172.21.15.94:0/2432363911 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5609708d8720 con 0x560970cb6790 2024-02-17T12:02:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.035+0000 7fc2f9115640 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560970cb6790 0x560970cb8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:02:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2dd7fa640 10 client.26261 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2f9115640 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560970cb6790 0x560970cb8bb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=965f9adc63e3ff07 server_cookie=e1ccfa56295e189c in_seq=0 out_seq=0 2024-02-17T12:02:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2dd7fa640 1 -- 172.21.15.94:0/2432363911 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5609708d8720 con 0x560970cb6790 2024-02-17T12:02:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2dd7fa640 10 client.26261 handle_command_reply: tid=0 2024-02-17T12:02:55.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2fe82e740 1 client.26261 shutdown 2024-02-17T12:02:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2fe82e740 1 -- 172.21.15.94:0/2432363911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 msgr2=0x560970c41180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 0x560970c41180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0013040 tx=0x7fc2f002a050 comp rx=0 tx=0).stop 2024-02-17T12:02:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2fe82e740 1 -- 172.21.15.94:0/2432363911 shutdown_connections 2024-02-17T12:02:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560970cb6790 0x560970cb8bb0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560970c4d930 0x560970c41c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560970c3d7b0 0x560970c416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2fe82e740 1 --2- 172.21.15.94:0/2432363911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560970c34bd0 0x560970c41180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2fe82e740 1 -- 172.21.15.94:0/2432363911 >> 172.21.15.94:0/2432363911 conn(0x560970c2e990 msgr2=0x560970c2fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2dd7fa640 0 client.26261 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:02:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2fe82e740 1 -- 172.21.15.94:0/2432363911 shutdown_connections 2024-02-17T12:02:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2fe82e740 1 -- 172.21.15.94:0/2432363911 wait complete. 2024-02-17T12:02:55.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.036+0000 7fc2fe82e740 20 client.26261 trim_cache size 0 max 16384 2024-02-17T12:02:55.041 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:02:55.041 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:02:55.041 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:02:55.041 INFO:teuthology.orchestra.run.smithi094.stdout: "f1054b49-6d92-4c44-a6b2-35896defc6bb": { 2024-02-17T12:02:55.041 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:02:55.041 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "f1054b49-6d92-4c44-a6b2-35896defc6bb", 2024-02-17T12:02:55.041 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:02:55.041 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:02:55.041 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:02:55.041 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:02:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.037+0000 7fc2fbb7a640 1 -- 172.21.15.94:0/1094133494 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc2e00a2380 msgr2=0x7fc2e00a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.037+0000 7fc2fbb7a640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc2e00a2380 0x7fc2e00a4840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec005c40 tx=0x7fc2ec009760 comp rx=0 tx=0).stop 2024-02-17T12:02:55.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.037+0000 7fc2fbb7a640 1 -- 172.21.15.94:0/1094133494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f4100ab0 msgr2=0x7fc2f419e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:02:55.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.037+0000 7fc2fbb7a640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f4100ab0 0x7fc2f419e330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc2e402c2e0 tx=0x7fc2e402c740 comp rx=0 tx=0).stop 2024-02-17T12:02:55.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.037+0000 7fc2fbb7a640 1 -- 172.21.15.94:0/1094133494 shutdown_connections 2024-02-17T12:02:55.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.037+0000 7fc2fbb7a640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc2e00a2380 0x7fc2e00a4840 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.037+0000 7fc2fbb7a640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc2f41054b0 0x7fc2f41a2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.037+0000 7fc2fbb7a640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2f4100ab0 0x7fc2f419e330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.037+0000 7fc2fbb7a640 1 --2- 172.21.15.94:0/1094133494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc2f4100190 0x7fc2f419ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:02:55.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.037+0000 7fc2fbb7a640 1 -- 172.21.15.94:0/1094133494 >> 172.21.15.94:0/1094133494 conn(0x7fc2f4077230 msgr2=0x7fc2f4102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:02:55.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.037+0000 7fc2fbb7a640 1 -- 172.21.15.94:0/1094133494 shutdown_connections 2024-02-17T12:02:55.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:02:55.037+0000 7fc2fbb7a640 1 -- 172.21.15.94:0/1094133494 wait complete. 2024-02-17T12:02:55.047 INFO:tasks.cephfs.filesystem:scrub status for tag:f1054b49-6d92-4c44-a6b2-35896defc6bb - {'path': '/', 'tag': 'f1054b49-6d92-4c44-a6b2-35896defc6bb', 'options': 'recursive,force'} 2024-02-17T12:02:55.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 275.0 s, 2816.4 tps, lat 35.623 ms stddev 38.018 2024-02-17T12:02:56.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:55 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:02:56.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:55 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:02:56.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:55 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:02:56.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:55 smithi125 ceph-mon[36819]: pgmap v147: 97 pgs: 97 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.8 MiB/s wr, 826 op/s 2024-02-17T12:02:56.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:55 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:02:56.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:55 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:02:56.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:55 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:02:56.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:55 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:02:56.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:55 smithi094 ceph-mon[32499]: pgmap v147: 97 pgs: 97 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.8 MiB/s wr, 826 op/s 2024-02-17T12:02:56.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:55 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:02:56.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:55 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:02:56.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:55 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:02:56.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:55 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:02:56.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:55 smithi189 ceph-mon[36588]: pgmap v147: 97 pgs: 97 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 9.8 MiB/s wr, 826 op/s 2024-02-17T12:02:56.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:55 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:02:58.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:57 smithi125 ceph-mon[36819]: pgmap v148: 97 pgs: 97 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 629 op/s 2024-02-17T12:02:58.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:57 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:02:58.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:57 smithi094 ceph-mon[32499]: pgmap v148: 97 pgs: 97 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 629 op/s 2024-02-17T12:02:58.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:57 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:02:58.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:57 smithi189 ceph-mon[36588]: pgmap v148: 97 pgs: 97 active+clean; 3.3 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.7 MiB/s wr, 629 op/s 2024-02-17T12:02:58.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:57 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:02:59.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:02:58 smithi189 ceph-mon[36588]: pgmap v149: 97 pgs: 97 active+clean; 3.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 11 MiB/s wr, 1.05k op/s 2024-02-17T12:02:59.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:02:58 smithi125 ceph-mon[36819]: pgmap v149: 97 pgs: 97 active+clean; 3.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 11 MiB/s wr, 1.05k op/s 2024-02-17T12:02:59.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:02:58 smithi094 ceph-mon[32499]: pgmap v149: 97 pgs: 97 active+clean; 3.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 11 MiB/s wr, 1.05k op/s 2024-02-17T12:03:00.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 280.0 s, 3062.8 tps, lat 32.526 ms stddev 35.951 2024-02-17T12:03:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.294+0000 7f7e4318a640 1 -- 172.21.15.94:0/3542941268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e3c10cb60 msgr2=0x7f7e3c10cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.294+0000 7f7e4318a640 1 --2- 172.21.15.94:0/3542941268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e3c10cb60 0x7f7e3c10cf40 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009ca0 tx=0x7f7e3402dfd0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.294+0000 7f7e4318a640 1 -- 172.21.15.94:0/3542941268 shutdown_connections 2024-02-17T12:03:01.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.294+0000 7f7e4318a640 1 --2- 172.21.15.94:0/3542941268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c1086c0 0x7f7e3c108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.294+0000 7f7e4318a640 1 --2- 172.21.15.94:0/3542941268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e3c068ea0 0x7f7e3c108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.294+0000 7f7e4318a640 1 --2- 172.21.15.94:0/3542941268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e3c10cb60 0x7f7e3c10cf40 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.294+0000 7f7e4318a640 1 -- 172.21.15.94:0/3542941268 >> 172.21.15.94:0/3542941268 conn(0x7f7e3c06c6b0 msgr2=0x7f7e3c06cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.294+0000 7f7e4318a640 1 -- 172.21.15.94:0/3542941268 shutdown_connections 2024-02-17T12:03:01.295 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.294+0000 7f7e4318a640 1 -- 172.21.15.94:0/3542941268 wait complete. 2024-02-17T12:03:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.295+0000 7f7e4318a640 1 Processor -- start 2024-02-17T12:03:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.295+0000 7f7e4318a640 1 -- start start 2024-02-17T12:03:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.295+0000 7f7e4318a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e3c068ea0 0x7f7e3c1153a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.295+0000 7f7e4318a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c1086c0 0x7f7e3c1134a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.295+0000 7f7e4318a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e3c10cb60 0x7f7e3c1139e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.295+0000 7f7e40f26640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e3c068ea0 0x7f7e3c1153a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.295+0000 7f7e40f26640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e3c068ea0 0x7f7e3c1153a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54492/0 (socket says 172.21.15.94:54492) 2024-02-17T12:03:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.296+0000 7f7e3bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c1086c0 0x7f7e3c1134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.296+0000 7f7e3bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c1086c0 0x7f7e3c1134a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34070/0 (socket says 172.21.15.94:34070) 2024-02-17T12:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.296+0000 7f7e40f26640 1 -- 172.21.15.94:0/1748070849 learned_addr learned my addr 172.21.15.94:0/1748070849 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.296+0000 7f7e41727640 1 --2- 172.21.15.94:0/1748070849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e3c10cb60 0x7f7e3c1139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.296+0000 7f7e4318a640 1 -- 172.21.15.94:0/1748070849 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c114170 con 0x7f7e3c068ea0 2024-02-17T12:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.296+0000 7f7e4318a640 1 -- 172.21.15.94:0/1748070849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c1142e0 con 0x7f7e3c1086c0 2024-02-17T12:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.296+0000 7f7e4318a640 1 -- 172.21.15.94:0/1748070849 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c1b2aa0 con 0x7f7e3c10cb60 2024-02-17T12:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.296+0000 7f7e3bfff640 1 -- 172.21.15.94:0/1748070849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e3c10cb60 msgr2=0x7f7e3c1139e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.296+0000 7f7e3bfff640 1 --2- 172.21.15.94:0/1748070849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e3c10cb60 0x7f7e3c1139e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.296+0000 7f7e3bfff640 1 -- 172.21.15.94:0/1748070849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e3c068ea0 msgr2=0x7f7e3c1153a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.296+0000 7f7e3bfff640 1 --2- 172.21.15.94:0/1748070849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e3c068ea0 0x7f7e3c1153a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.296+0000 7f7e3bfff640 1 -- 172.21.15.94:0/1748070849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e34009900 con 0x7f7e3c1086c0 2024-02-17T12:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.297+0000 7f7e3bfff640 1 --2- 172.21.15.94:0/1748070849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c1086c0 0x7f7e3c1134a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000b3c0 tx=0x7f7e3000b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.297+0000 7f7e39ffb640 1 -- 172.21.15.94:0/1748070849 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e30011830 con 0x7f7e3c1086c0 2024-02-17T12:03:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.297+0000 7f7e39ffb640 1 -- 172.21.15.94:0/1748070849 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e30011e70 con 0x7f7e3c1086c0 2024-02-17T12:03:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.297+0000 7f7e39ffb640 1 -- 172.21.15.94:0/1748070849 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3000f2e0 con 0x7f7e3c1086c0 2024-02-17T12:03:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.297+0000 7f7e4318a640 1 -- 172.21.15.94:0/1748070849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c1b2d80 con 0x7f7e3c1086c0 2024-02-17T12:03:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.297+0000 7f7e4318a640 1 -- 172.21.15.94:0/1748070849 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c1b3280 con 0x7f7e3c1086c0 2024-02-17T12:03:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.300+0000 7f7e39ffb640 1 -- 172.21.15.94:0/1748070849 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7e30010610 con 0x7f7e3c1086c0 2024-02-17T12:03:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.300+0000 7f7e39ffb640 1 --2- 172.21.15.94:0/1748070849 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7e240a23d0 0x7f7e240a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.301+0000 7f7e39ffb640 1 -- 172.21.15.94:0/1748070849 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7e300cd6d0 con 0x7f7e3c1086c0 2024-02-17T12:03:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.301+0000 7f7e40f26640 1 --2- 172.21.15.94:0/1748070849 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7e240a23d0 0x7f7e240a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.301+0000 7f7e45e3a740 1 Processor -- start 2024-02-17T12:03:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.301+0000 7f7e45e3a740 1 -- start start 2024-02-17T12:03:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e45e3a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0dad0 0x55f400b0c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e45e3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b0cd60 0x55f400b0d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e45e3a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 0x55f400b186d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e45e3a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f400a7b9f0 con 0x55f400b162e0 2024-02-17T12:03:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e45e3a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f400a78f20 con 0x55f400b0dad0 2024-02-17T12:03:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e45e3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f400a74700 con 0x55f400b0cd60 2024-02-17T12:03:01.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e41727640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 0x55f400b186d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e41727640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 0x55f400b186d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54498/0 (socket says 172.21.15.94:54498) 2024-02-17T12:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e41727640 1 -- 172.21.15.94:0/792582453 learned_addr learned my addr 172.21.15.94:0/792582453 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e3bfff640 1 --2- 172.21.15.94:0/792582453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b0cd60 0x55f400b0d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e40f26640 1 --2- 172.21.15.94:0/1748070849 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7e240a23d0 0x7f7e240a4890 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7e34005e30 tx=0x7f7e34005c50 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e40f26640 1 --2- 172.21.15.94:0/792582453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0dad0 0x55f400b0c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e41727640 1 -- 172.21.15.94:0/792582453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b0cd60 msgr2=0x55f400b0d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e41727640 1 --2- 172.21.15.94:0/792582453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b0cd60 0x55f400b0d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e41727640 1 -- 172.21.15.94:0/792582453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0dad0 msgr2=0x55f400b0c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e41727640 1 --2- 172.21.15.94:0/792582453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0dad0 0x55f400b0c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e41727640 1 -- 172.21.15.94:0/792582453 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f400a0a7b0 con 0x55f400b162e0 2024-02-17T12:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.302+0000 7f7e3bfff640 1 --2- 172.21.15.94:0/792582453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b0cd60 0x55f400b0d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.303+0000 7f7e40f26640 1 --2- 172.21.15.94:0/792582453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0dad0 0x55f400b0c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:03:01.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.303+0000 7f7e41727640 1 --2- 172.21.15.94:0/792582453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 0x55f400b186d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009c10 tx=0x7f7e2c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3760684fd73fd33a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.303+0000 7f7e22ffd640 1 -- 172.21.15.94:0/792582453 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c012450 con 0x55f400b162e0 2024-02-17T12:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.303+0000 7f7e22ffd640 1 -- 172.21.15.94:0/792582453 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e2c012a90 con 0x55f400b162e0 2024-02-17T12:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.303+0000 7f7e22ffd640 1 -- 172.21.15.94:0/792582453 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c00fb40 con 0x55f400b162e0 2024-02-17T12:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.304+0000 7f7e45e3a740 1 -- 172.21.15.94:0/792582453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 msgr2=0x55f400b186d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.304+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/792582453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 0x55f400b186d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c009c10 tx=0x7f7e2c0117b0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.304+0000 7f7e45e3a740 1 -- 172.21.15.94:0/792582453 shutdown_connections 2024-02-17T12:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.304+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/792582453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 0x55f400b186d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.304+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/792582453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b0cd60 0x55f400b0d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.304+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/792582453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0dad0 0x55f400b0c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.304+0000 7f7e45e3a740 1 -- 172.21.15.94:0/792582453 >> 172.21.15.94:0/792582453 conn(0x55f400ad4550 msgr2=0x55f400b07a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.304+0000 7f7e45e3a740 1 -- 172.21.15.94:0/792582453 shutdown_connections 2024-02-17T12:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.304+0000 7f7e45e3a740 1 -- 172.21.15.94:0/792582453 wait complete. 2024-02-17T12:03:01.305 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.305+0000 7f7e45e3a740 1 -- start start 2024-02-17T12:03:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.305+0000 7f7e45e3a740 1 Processor -- start 2024-02-17T12:03:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e45e3a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 0x55f400b13360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e45e3a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b0dad0 0x55f400b138a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e45e3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b162e0 0x55f400b13de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e40f26640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 0x55f400b13360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e40f26640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 0x55f400b13360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34082/0 (socket says 172.21.15.94:34082) 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e40f26640 1 -- 172.21.15.94:0/400978730 learned_addr learned my addr 172.21.15.94:0/400978730 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e45e3a740 1 -- 172.21.15.94:0/400978730 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f400a19bf0 con 0x55f400b0dad0 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e45e3a740 1 -- 172.21.15.94:0/400978730 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f400748250 con 0x55f400b0cd60 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e45e3a740 1 -- 172.21.15.94:0/400978730 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4009fa200 con 0x55f400b162e0 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e41727640 1 --2- 172.21.15.94:0/400978730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b162e0 0x55f400b13de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e40f26640 1 -- 172.21.15.94:0/400978730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b162e0 msgr2=0x55f400b13de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e40f26640 1 --2- 172.21.15.94:0/400978730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b162e0 0x55f400b13de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e40f26640 1 -- 172.21.15.94:0/400978730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b0dad0 msgr2=0x55f400b138a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e40f26640 1 --2- 172.21.15.94:0/400978730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b0dad0 0x55f400b138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e40f26640 1 -- 172.21.15.94:0/400978730 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c0098c0 con 0x55f400b0cd60 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.306+0000 7f7e45e3a740 10 client.0 _ll_register_callbacks cb 0x55f400a68d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.307+0000 7f7e40f26640 1 --2- 172.21.15.94:0/400978730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 0x55f400b13360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7e34005c00 tx=0x7f7e340095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53c17bfdb8622def server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:01.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.307+0000 7f7e1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.307+0000 7f7e45e3a740 1 -- 172.21.15.94:0/400978730 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f400774600 con 0x55f400b0cd60 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.307+0000 7f7e1bfff640 1 -- 172.21.15.94:0/400978730 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3402ed80 con 0x55f400b0cd60 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.307+0000 7f7e1bfff640 1 -- 172.21.15.94:0/400978730 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7e34003a30 con 0x55f400b0cd60 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.307+0000 7f7e1bfff640 1 -- 172.21.15.94:0/400978730 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e34003c10 con 0x55f400b0cd60 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.307+0000 7f7e45e3a740 10 client.26270 fetch_fsmap learned FSMap version 23 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.307+0000 7f7e45e3a740 1 -- 172.21.15.94:0/400978730 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({fsmap=23}) v3 -- 0x55f400a72d80 con 0x55f400b0cd60 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.308+0000 7f7e1bfff640 1 -- 172.21.15.94:0/400978730 <== mon.2 v2:172.21.15.125:3300/0 4 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7e340348b0 con 0x55f400b0cd60 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.308+0000 7f7e45e3a740 10 client.26270 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.308+0000 7f7e45e3a740 10 client.26270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.308+0000 7f7e45e3a740 20 client.26270 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.308+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/400978730 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f400b26490 0x55f400b28920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.308+0000 7f7e45e3a740 4 client.26270 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.308+0000 7f7e45e3a740 1 -- 172.21.15.94:0/400978730 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4007c51b0 con 0x55f400b26490 2024-02-17T12:03:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.308+0000 7f7e3bfff640 1 --2- 172.21.15.94:0/400978730 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f400b26490 0x55f400b28920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.308+0000 7f7e1bfff640 10 client.26270 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:01.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.308+0000 7f7e3bfff640 1 --2- 172.21.15.94:0/400978730 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f400b26490 0x55f400b28920 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8cf472cdd7ec51 server_cookie=90fbf148d54e7a5d in_seq=0 out_seq=0 2024-02-17T12:03:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.309+0000 7f7e1bfff640 1 -- 172.21.15.94:0/400978730 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4007c51b0 con 0x55f400b26490 2024-02-17T12:03:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.309+0000 7f7e1bfff640 10 client.26270 handle_command_reply: tid=0 2024-02-17T12:03:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.309+0000 7f7e45e3a740 1 client.26270 shutdown 2024-02-17T12:03:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.309+0000 7f7e45e3a740 1 -- 172.21.15.94:0/400978730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 msgr2=0x55f400b13360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.309+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/400978730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 0x55f400b13360 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7e34005c00 tx=0x7f7e340095c0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.309+0000 7f7e45e3a740 1 -- 172.21.15.94:0/400978730 shutdown_connections 2024-02-17T12:03:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.309+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/400978730 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f400b26490 0x55f400b28920 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.309+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/400978730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b162e0 0x55f400b13de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.309+0000 7f7e1bfff640 0 client.26270 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.309+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/400978730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b0dad0 0x55f400b138a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.309+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/400978730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 0x55f400b13360 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.309+0000 7f7e45e3a740 1 -- 172.21.15.94:0/400978730 >> 172.21.15.94:0/400978730 conn(0x55f400ad4550 msgr2=0x55f400b097a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.310+0000 7f7e45e3a740 1 -- 172.21.15.94:0/400978730 shutdown_connections 2024-02-17T12:03:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.310+0000 7f7e45e3a740 1 -- 172.21.15.94:0/400978730 wait complete. 2024-02-17T12:03:01.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.310+0000 7f7e45e3a740 20 client.26270 trim_cache size 0 max 16384 2024-02-17T12:03:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e45e3a740 1 Processor -- start 2024-02-17T12:03:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e45e3a740 1 -- start start 2024-02-17T12:03:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e45e3a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b0cd60 0x55f400b12c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e45e3a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0dad0 0x55f400b06a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e45e3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b162e0 0x55f400b10620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e45e3a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4007c51b0 con 0x55f400b0cd60 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e45e3a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f400b08c10 con 0x55f400b0dad0 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e45e3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f400b231f0 con 0x55f400b162e0 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e40f26640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b0cd60 0x55f400b12c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e40f26640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b0cd60 0x55f400b12c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54514/0 (socket says 172.21.15.94:54514) 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e3bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0dad0 0x55f400b06a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e40f26640 1 -- 172.21.15.94:0/4169067427 learned_addr learned my addr 172.21.15.94:0/4169067427 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e41727640 1 --2- 172.21.15.94:0/4169067427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b162e0 0x55f400b10620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e40f26640 1 -- 172.21.15.94:0/4169067427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b162e0 msgr2=0x55f400b10620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e40f26640 1 --2- 172.21.15.94:0/4169067427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b162e0 0x55f400b10620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e40f26640 1 -- 172.21.15.94:0/4169067427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0dad0 msgr2=0x55f400b06a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.329+0000 7f7e40f26640 1 --2- 172.21.15.94:0/4169067427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0dad0 0x55f400b06a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.330 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.330+0000 7f7e40f26640 1 -- 172.21.15.94:0/4169067427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f400a0f7b0 con 0x55f400b0cd60 2024-02-17T12:03:01.331 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.330+0000 7f7e3bfff640 1 --2- 172.21.15.94:0/4169067427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0dad0 0x55f400b06a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:01.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.330+0000 7f7e40f26640 1 --2- 172.21.15.94:0/4169067427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b0cd60 0x55f400b12c80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7e3402e5e0 tx=0x7f7e3404d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d17af592793ec47 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:01.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.331+0000 7f7e227fc640 1 -- 172.21.15.94:0/4169067427 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3402eb80 con 0x55f400b0cd60 2024-02-17T12:03:01.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.331+0000 7f7e227fc640 1 -- 172.21.15.94:0/4169067427 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f400774600 con 0x55f400b0cd60 2024-02-17T12:03:01.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.331+0000 7f7e227fc640 1 -- 172.21.15.94:0/4169067427 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e34042b90 con 0x55f400b0cd60 2024-02-17T12:03:01.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.332+0000 7f7e45e3a740 1 -- 172.21.15.94:0/4169067427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b0cd60 msgr2=0x55f400b12c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:01.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.332+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/4169067427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b0cd60 0x55f400b12c80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f7e3402e5e0 tx=0x7f7e3404d000 comp rx=0 tx=0).stop 2024-02-17T12:03:01.332 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.332+0000 7f7e45e3a740 1 -- 172.21.15.94:0/4169067427 shutdown_connections 2024-02-17T12:03:01.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.332+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/4169067427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b162e0 0x55f400b10620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.332+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/4169067427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0dad0 0x55f400b06a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.332+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/4169067427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b0cd60 0x55f400b12c80 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.332+0000 7f7e45e3a740 1 -- 172.21.15.94:0/4169067427 >> 172.21.15.94:0/4169067427 conn(0x55f400ad4550 msgr2=0x55f400b06300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:01.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.332+0000 7f7e45e3a740 1 -- 172.21.15.94:0/4169067427 shutdown_connections 2024-02-17T12:03:01.333 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.332+0000 7f7e45e3a740 1 -- 172.21.15.94:0/4169067427 wait complete. 2024-02-17T12:03:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.333+0000 7f7e45e3a740 1 -- start start 2024-02-17T12:03:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.333+0000 7f7e45e3a740 1 Processor -- start 2024-02-17T12:03:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e45e3a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 0x55f400b1d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e45e3a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b0dad0 0x55f400b1b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e45e3a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 0x55f400b1bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e45e3a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f400b1c480 con 0x55f400b162e0 2024-02-17T12:03:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e45e3a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f400b1c5f0 con 0x55f400b0cd60 2024-02-17T12:03:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e45e3a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f400b7ca50 con 0x55f400b0dad0 2024-02-17T12:03:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e45e3a740 10 client.0 _ll_register_callbacks cb 0x7f7e2c0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e41727640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 0x55f400b1bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e40f26640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 0x55f400b1d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e40f26640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 0x55f400b1d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34098/0 (socket says 172.21.15.94:34098) 2024-02-17T12:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e41727640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 0x55f400b1bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54530/0 (socket says 172.21.15.94:54530) 2024-02-17T12:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e40f26640 1 -- 172.21.15.94:0/304762114 learned_addr learned my addr 172.21.15.94:0/304762114 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e3bfff640 1 --2- 172.21.15.94:0/304762114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b0dad0 0x55f400b1b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e40f26640 1 -- 172.21.15.94:0/304762114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b0dad0 msgr2=0x55f400b1b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e40f26640 1 --2- 172.21.15.94:0/304762114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b0dad0 0x55f400b1b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e40f26640 1 -- 172.21.15.94:0/304762114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 msgr2=0x55f400b1bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e40f26640 1 --2- 172.21.15.94:0/304762114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 0x55f400b1bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e40f26640 1 -- 172.21.15.94:0/304762114 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c1b3280 con 0x55f400b0cd60 2024-02-17T12:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.334+0000 7f7e41727640 1 --2- 172.21.15.94:0/304762114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 0x55f400b1bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.335+0000 7f7e40f26640 1 --2- 172.21.15.94:0/304762114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 0x55f400b1d4a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e34005c00 tx=0x7f7e34042d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3cf3ad5e504f76e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.335+0000 7f7e1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:03:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.335+0000 7f7e1bfff640 1 -- 172.21.15.94:0/304762114 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e340348b0 con 0x55f400b0cd60 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.335+0000 7f7e1bfff640 1 -- 172.21.15.94:0/304762114 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7e3406d460 con 0x55f400b0cd60 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.335+0000 7f7e45e3a740 1 -- 172.21.15.94:0/304762114 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e2c012a90 con 0x55f400b0cd60 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.335+0000 7f7e1bfff640 1 -- 172.21.15.94:0/304762114 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e34034a10 con 0x55f400b0cd60 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.335+0000 7f7e45e3a740 10 client.26276 fetch_fsmap learned FSMap version 23 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.335+0000 7f7e45e3a740 1 -- 172.21.15.94:0/304762114 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f400a72d80 con 0x55f400b0cd60 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.335+0000 7f7e1bfff640 1 -- 172.21.15.94:0/304762114 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7e2c012a90 con 0x55f400b0cd60 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.336+0000 7f7e1bfff640 1 -- 172.21.15.94:0/304762114 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 23) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7e34060420 con 0x55f400b0cd60 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.336+0000 7f7e45e3a740 10 client.26276 fetch_fsmap finished waiting for FSMap version 23 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.336+0000 7f7e45e3a740 10 client.26276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.336+0000 7f7e45e3a740 20 client.26276 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.336+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/304762114 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f400b26490 0x55f400b7fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.336+0000 7f7e45e3a740 4 client.26276 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.336+0000 7f7e45e3a740 1 -- 172.21.15.94:0/304762114 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f4007b1550 con 0x55f400b26490 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.336+0000 7f7e3bfff640 1 --2- 172.21.15.94:0/304762114 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f400b26490 0x55f400b7fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:01.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.336+0000 7f7e1bfff640 10 client.26276 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:01.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:01.337+0000 7f7e3bfff640 1 --2- 172.21.15.94:0/304762114 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f400b26490 0x55f400b7fde0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d83d91660336a7c8 server_cookie=86eb70c97ac80caf in_seq=0 out_seq=0 2024-02-17T12:03:02.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:01 smithi125 ceph-mon[36819]: pgmap v150: 97 pgs: 97 active+clean; 3.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-02-17T12:03:02.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:01 smithi094 ceph-mon[32499]: pgmap v150: 97 pgs: 97 active+clean; 3.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-02-17T12:03:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.385+0000 7f7e1bfff640 1 -- 172.21.15.94:0/304762114 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f4007b1550 con 0x55f400b26490 2024-02-17T12:03:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.385+0000 7f7e1bfff640 10 client.26276 handle_command_reply: tid=0 2024-02-17T12:03:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.385+0000 7f7e45e3a740 1 client.26276 shutdown 2024-02-17T12:03:02.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.386+0000 7f7e45e3a740 1 -- 172.21.15.94:0/304762114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 msgr2=0x55f400b1d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.386+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/304762114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 0x55f400b1d4a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7e34005c00 tx=0x7f7e34042d20 comp rx=0 tx=0).stop 2024-02-17T12:03:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.386+0000 7f7e45e3a740 1 -- 172.21.15.94:0/304762114 shutdown_connections 2024-02-17T12:03:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.386+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/304762114 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f400b26490 0x55f400b7fde0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.386+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/304762114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f400b162e0 0x55f400b1bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.386+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/304762114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f400b0dad0 0x55f400b1b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.386+0000 7f7e45e3a740 1 --2- 172.21.15.94:0/304762114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f400b0cd60 0x55f400b1d4a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:02.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.386+0000 7f7e1bfff640 0 client.26276 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.386+0000 7f7e45e3a740 1 -- 172.21.15.94:0/304762114 >> 172.21.15.94:0/304762114 conn(0x55f400ad4550 msgr2=0x55f400b09630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.386+0000 7f7e45e3a740 1 -- 172.21.15.94:0/304762114 shutdown_connections 2024-02-17T12:03:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.386+0000 7f7e45e3a740 1 -- 172.21.15.94:0/304762114 wait complete. 2024-02-17T12:03:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.386+0000 7f7e45e3a740 20 client.26276 trim_cache size 0 max 16384 2024-02-17T12:03:02.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:03:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:03:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:03:02.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:03:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.387+0000 7f7e4318a640 1 -- 172.21.15.94:0/1748070849 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7e240a23d0 msgr2=0x7f7e240a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.387+0000 7f7e4318a640 1 --2- 172.21.15.94:0/1748070849 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7e240a23d0 0x7f7e240a4890 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7e34005e30 tx=0x7f7e34005c50 comp rx=0 tx=0).stop 2024-02-17T12:03:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.387+0000 7f7e4318a640 1 -- 172.21.15.94:0/1748070849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c1086c0 msgr2=0x7f7e3c1134a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.387+0000 7f7e4318a640 1 --2- 172.21.15.94:0/1748070849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c1086c0 0x7f7e3c1134a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000b3c0 tx=0x7f7e3000b880 comp rx=0 tx=0).stop 2024-02-17T12:03:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.387+0000 7f7e4318a640 1 -- 172.21.15.94:0/1748070849 shutdown_connections 2024-02-17T12:03:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.387+0000 7f7e4318a640 1 --2- 172.21.15.94:0/1748070849 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7e240a23d0 0x7f7e240a4890 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.387+0000 7f7e4318a640 1 --2- 172.21.15.94:0/1748070849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e3c10cb60 0x7f7e3c1139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.387+0000 7f7e4318a640 1 --2- 172.21.15.94:0/1748070849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7e3c1086c0 0x7f7e3c1134a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.387+0000 7f7e4318a640 1 --2- 172.21.15.94:0/1748070849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7e3c068ea0 0x7f7e3c1153a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.387+0000 7f7e4318a640 1 -- 172.21.15.94:0/1748070849 >> 172.21.15.94:0/1748070849 conn(0x7f7e3c06c6b0 msgr2=0x7f7e3c071a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.387+0000 7f7e4318a640 1 -- 172.21.15.94:0/1748070849 shutdown_connections 2024-02-17T12:03:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:02.387+0000 7f7e4318a640 1 -- 172.21.15.94:0/1748070849 wait complete. 2024-02-17T12:03:02.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:01 smithi189 ceph-mon[36588]: pgmap v150: 97 pgs: 97 active+clean; 3.4 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 7.2 MiB/s wr, 745 op/s 2024-02-17T12:03:03.152 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: *** Caught signal (Segmentation fault) ** 2024-02-17T12:03:03.152 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: in thread 7fe1b5ea1700 thread_name:md_log_replay 2024-02-17T12:03:03.153 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:03:03.153 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe1c7262d20] 2024-02-17T12:03:03.153 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 2: /usr/bin/ceph-mds(+0x50481f) [0x55728bbcb81f] 2024-02-17T12:03:03.153 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55728bbcda53] 2024-02-17T12:03:03.153 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 4: (MDLog::_replay_thread()+0x647) [0x55728bc2a277] 2024-02-17T12:03:03.153 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55728bc34111] 2024-02-17T12:03:03.153 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fe1c72581ca] 2024-02-17T12:03:03.153 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 7: clone() 2024-02-17T12:03:03.153 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 2024-02-17T12:03:02.670+0000 7fe1b5ea1700 -1 *** Caught signal (Segmentation fault) ** 2024-02-17T12:03:03.153 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: in thread 7fe1b5ea1700 thread_name:md_log_replay 2024-02-17T12:03:03.153 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 2024-02-17T12:03:03.153 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:03:03.153 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe1c7262d20] 2024-02-17T12:03:03.154 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 2: /usr/bin/ceph-mds(+0x50481f) [0x55728bbcb81f] 2024-02-17T12:03:03.154 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55728bbcda53] 2024-02-17T12:03:03.154 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 4: (MDLog::_replay_thread()+0x647) [0x55728bc2a277] 2024-02-17T12:03:03.154 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55728bc34111] 2024-02-17T12:03:03.154 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fe1c72581ca] 2024-02-17T12:03:03.154 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 7: clone() 2024-02-17T12:03:03.154 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-17T12:03:03.154 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 2024-02-17T12:03:03.154 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 0> 2024-02-17T12:03:02.670+0000 7fe1b5ea1700 -1 *** Caught signal (Segmentation fault) ** 2024-02-17T12:03:03.154 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: in thread 7fe1b5ea1700 thread_name:md_log_replay 2024-02-17T12:03:03.154 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 2024-02-17T12:03:03.154 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:03:03.154 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe1c7262d20] 2024-02-17T12:03:03.155 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 2: /usr/bin/ceph-mds(+0x50481f) [0x55728bbcb81f] 2024-02-17T12:03:03.155 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55728bbcda53] 2024-02-17T12:03:03.155 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 4: (MDLog::_replay_thread()+0x647) [0x55728bc2a277] 2024-02-17T12:03:03.155 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55728bc34111] 2024-02-17T12:03:03.155 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fe1c72581ca] 2024-02-17T12:03:03.155 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 7: clone() 2024-02-17T12:03:03.155 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-17T12:03:03.155 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 2024-02-17T12:03:03.155 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 0> 2024-02-17T12:03:02.670+0000 7fe1b5ea1700 -1 *** Caught signal (Segmentation fault) ** 2024-02-17T12:03:03.155 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: in thread 7fe1b5ea1700 thread_name:md_log_replay 2024-02-17T12:03:03.155 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 2024-02-17T12:03:03.155 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:03:03.155 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe1c7262d20] 2024-02-17T12:03:03.156 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 2: /usr/bin/ceph-mds(+0x50481f) [0x55728bbcb81f] 2024-02-17T12:03:03.156 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55728bbcda53] 2024-02-17T12:03:03.156 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 4: (MDLog::_replay_thread()+0x647) [0x55728bc2a277] 2024-02-17T12:03:03.156 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55728bc34111] 2024-02-17T12:03:03.156 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fe1c72581ca] 2024-02-17T12:03:03.156 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 7: clone() 2024-02-17T12:03:03.156 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-17T12:03:03.156 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[67072]: 2024-02-17T12:03:03.156 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:02 smithi189 podman[71793]: 2024-02-17 12:03:02.728546092 +0000 UTC m=+0.020378614 container died c9250c033e01451b4ec504b338e471cec4cb30d766a0b045ee0213eb2115bd73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:03:03.896 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:03 smithi189 podman[71793]: 2024-02-17 12:03:03.88434878 +0000 UTC m=+1.176181281 container remove c9250c033e01451b4ec504b338e471cec4cb30d766a0b045ee0213eb2115bd73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.build-date=20240214) 2024-02-17T12:03:03.896 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:03 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.l.service: Main process exited, code=exited, status=139/n/a 2024-02-17T12:03:04.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:03 smithi125 ceph-mon[36819]: pgmap v151: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 23 MiB/s wr, 1.31k op/s 2024-02-17T12:03:04.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:03 smithi094 ceph-mon[32499]: pgmap v151: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 23 MiB/s wr, 1.31k op/s 2024-02-17T12:03:04.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:03 smithi189 ceph-mon[36588]: pgmap v151: 97 pgs: 97 active+clean; 3.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 23 MiB/s wr, 1.31k op/s 2024-02-17T12:03:04.402 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:04 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.l.service: Failed with result 'exit-code'. 2024-02-17T12:03:04.402 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:04 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.l.service: Consumed 1.071s CPU time. 2024-02-17T12:03:05.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 285.0 s, 2168.0 tps, lat 46.153 ms stddev 75.883 2024-02-17T12:03:06.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:05 smithi125 ceph-mon[36819]: pgmap v152: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 38 MiB/s wr, 1.64k op/s 2024-02-17T12:03:06.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:05 smithi094 ceph-mon[32499]: pgmap v152: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 38 MiB/s wr, 1.64k op/s 2024-02-17T12:03:06.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:05 smithi189 ceph-mon[36588]: pgmap v152: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 38 MiB/s wr, 1.64k op/s 2024-02-17T12:03:08.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:07 smithi125 ceph-mon[36819]: pgmap v153: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 36 MiB/s wr, 1.50k op/s 2024-02-17T12:03:08.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:07 smithi094 ceph-mon[32499]: pgmap v153: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 36 MiB/s wr, 1.50k op/s 2024-02-17T12:03:08.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:07 smithi189 ceph-mon[36588]: pgmap v153: 97 pgs: 97 active+clean; 3.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 36 MiB/s wr, 1.50k op/s 2024-02-17T12:03:10.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:09 smithi125 ceph-mon[36819]: pgmap v154: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 39 MiB/s wr, 1.78k op/s 2024-02-17T12:03:10.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:09 smithi094 ceph-mon[32499]: pgmap v154: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 39 MiB/s wr, 1.78k op/s 2024-02-17T12:03:10.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 290.0 s, 2512.8 tps, lat 39.701 ms stddev 42.730 2024-02-17T12:03:10.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:09 smithi189 ceph-mon[36588]: pgmap v154: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 39 MiB/s wr, 1.78k op/s 2024-02-17T12:03:12.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:11 smithi125 ceph-mon[36819]: pgmap v155: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 1.36k op/s 2024-02-17T12:03:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:11 smithi094 ceph-mon[32499]: pgmap v155: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 1.36k op/s 2024-02-17T12:03:12.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:11 smithi189 ceph-mon[36588]: pgmap v155: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 36 MiB/s wr, 1.36k op/s 2024-02-17T12:03:14.171 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:13 smithi189 ceph-mon[36588]: pgmap v156: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 1.46k op/s 2024-02-17T12:03:14.171 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:14 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.l.service: Scheduled restart job, restart counter is at 1. 2024-02-17T12:03:14.171 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:14 smithi189 systemd[1]: Stopped Ceph mds.l for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T12:03:14.171 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:14 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.l.service: Consumed 1.071s CPU time. 2024-02-17T12:03:14.171 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:14 smithi189 systemd[1]: Starting Ceph mds.l for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:03:14.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:13 smithi125 ceph-mon[36819]: pgmap v156: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 1.46k op/s 2024-02-17T12:03:14.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:13 smithi094 ceph-mon[32499]: pgmap v156: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 MiB/s wr, 1.46k op/s 2024-02-17T12:03:14.652 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:14 smithi189 podman[71889]: 2024-02-17 12:03:14.299487399 +0000 UTC m=+0.017679665 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:03:14.652 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:14 smithi189 podman[71889]: 2024-02-17 12:03:14.471238932 +0000 UTC m=+0.189431185 container create db9500b09e5939bdbb03030b341ecca04e24acd3c7bb25e897d10c5ad797d9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-17T12:03:14.923 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:14 smithi189 podman[71889]: 2024-02-17 12:03:14.724371874 +0000 UTC m=+0.442564127 container init db9500b09e5939bdbb03030b341ecca04e24acd3c7bb25e897d10c5ad797d9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-17T12:03:14.924 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:14 smithi189 podman[71889]: 2024-02-17 12:03:14.727278818 +0000 UTC m=+0.445471064 container start db9500b09e5939bdbb03030b341ecca04e24acd3c7bb25e897d10c5ad797d9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-17T12:03:14.924 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:14 smithi189 ceph-mds[71909]: -- 172.21.15.189:0/2035836493 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5566796f61a0 con 0x5566796cc800 2024-02-17T12:03:14.924 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:14 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: starting mds.l at 2024-02-17T12:03:14.924 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:14 smithi189 bash[71889]: db9500b09e5939bdbb03030b341ecca04e24acd3c7bb25e897d10c5ad797d9ca 2024-02-17T12:03:14.924 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:03:14 smithi189 systemd[1]: Started Ceph mds.l for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T12:03:15.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:14 smithi125 ceph-mon[36819]: daemon mds.l restarted 2024-02-17T12:03:15.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:14 smithi094 ceph-mon[32499]: daemon mds.l restarted 2024-02-17T12:03:15.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 295.0 s, 2783.6 tps, lat 36.060 ms stddev 41.277 2024-02-17T12:03:15.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:14 smithi189 ceph-mon[36588]: daemon mds.l restarted 2024-02-17T12:03:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.524+0000 7f7c44a9d640 1 -- 172.21.15.94:0/522343784 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c40105020 msgr2=0x7f7c40105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.524+0000 7f7c44a9d640 1 --2- 172.21.15.94:0/522343784 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c40105020 0x7f7c40105400 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009be0 tx=0x7f7c3002bf10 comp rx=0 tx=0).stop 2024-02-17T12:03:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.525+0000 7f7c44a9d640 1 -- 172.21.15.94:0/522343784 shutdown_connections 2024-02-17T12:03:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.525+0000 7f7c44a9d640 1 --2- 172.21.15.94:0/522343784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c40100c90 0x7f7c40068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.525+0000 7f7c44a9d640 1 --2- 172.21.15.94:0/522343784 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c401002d0 0x7f7c40100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.525+0000 7f7c44a9d640 1 --2- 172.21.15.94:0/522343784 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c40105020 0x7f7c40105400 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.525+0000 7f7c44a9d640 1 -- 172.21.15.94:0/522343784 >> 172.21.15.94:0/522343784 conn(0x7f7c40076ed0 msgr2=0x7f7c400772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.525+0000 7f7c44a9d640 1 -- 172.21.15.94:0/522343784 shutdown_connections 2024-02-17T12:03:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.525+0000 7f7c44a9d640 1 -- 172.21.15.94:0/522343784 wait complete. 2024-02-17T12:03:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.526+0000 7f7c44a9d640 1 Processor -- start 2024-02-17T12:03:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.527+0000 7f7c44a9d640 1 -- start start 2024-02-17T12:03:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.527+0000 7f7c44a9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c401002d0 0x7f7c4019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.527+0000 7f7c44a9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c40100c90 0x7f7c4019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.528+0000 7f7c44a9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c40105020 0x7f7c40199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.528+0000 7f7c44a9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c40199eb0 con 0x7f7c401002d0 2024-02-17T12:03:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.528+0000 7f7c44a9d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4019a020 con 0x7f7c40105020 2024-02-17T12:03:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.528+0000 7f7c44a9d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4019a1e0 con 0x7f7c40100c90 2024-02-17T12:03:15.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.528+0000 7f7c3dd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c40100c90 0x7f7c4019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.528+0000 7f7c3dd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c40100c90 0x7f7c4019fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33498/0 (socket says 172.21.15.94:33498) 2024-02-17T12:03:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.528+0000 7f7c3dd9b640 1 -- 172.21.15.94:0/3066319396 learned_addr learned my addr 172.21.15.94:0/3066319396 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.528+0000 7f7c3ed9d640 1 --2- 172.21.15.94:0/3066319396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c40105020 0x7f7c40199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.528+0000 7f7c3dd9b640 1 -- 172.21.15.94:0/3066319396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c40105020 msgr2=0x7f7c40199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.528+0000 7f7c3dd9b640 1 --2- 172.21.15.94:0/3066319396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c40105020 0x7f7c40199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.528+0000 7f7c3dd9b640 1 -- 172.21.15.94:0/3066319396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c401002d0 msgr2=0x7f7c4019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.528+0000 7f7c3dd9b640 1 --2- 172.21.15.94:0/3066319396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c401002d0 0x7f7c4019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.528+0000 7f7c3dd9b640 1 -- 172.21.15.94:0/3066319396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30009840 con 0x7f7c40100c90 2024-02-17T12:03:15.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.529+0000 7f7c3dd9b640 1 --2- 172.21.15.94:0/3066319396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c40100c90 0x7f7c4019fbe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400e9c0 tx=0x7f7c3400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.529+0000 7f7c2f7fe640 1 -- 172.21.15.94:0/3066319396 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3400ccc0 con 0x7f7c40100c90 2024-02-17T12:03:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.529+0000 7f7c44a9d640 1 -- 172.21.15.94:0/3066319396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4019a4c0 con 0x7f7c40100c90 2024-02-17T12:03:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.529+0000 7f7c44a9d640 1 -- 172.21.15.94:0/3066319396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c401a66d0 con 0x7f7c40100c90 2024-02-17T12:03:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.529+0000 7f7c2f7fe640 1 -- 172.21.15.94:0/3066319396 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c3400ce20 con 0x7f7c40100c90 2024-02-17T12:03:15.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.529+0000 7f7c2f7fe640 1 -- 172.21.15.94:0/3066319396 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34010430 con 0x7f7c40100c90 2024-02-17T12:03:15.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.532+0000 7f7c2f7fe640 1 -- 172.21.15.94:0/3066319396 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7c34010590 con 0x7f7c40100c90 2024-02-17T12:03:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.533+0000 7f7c47674740 1 Processor -- start 2024-02-17T12:03:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.533+0000 7f7c2f7fe640 1 --2- 172.21.15.94:0/3066319396 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7c280a22d0 0x7f7c280a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.533+0000 7f7c2f7fe640 1 -- 172.21.15.94:0/3066319396 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7c34014070 con 0x7f7c40100c90 2024-02-17T12:03:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.533+0000 7f7c3ed9d640 1 --2- 172.21.15.94:0/3066319396 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7c280a22d0 0x7f7c280a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.534+0000 7f7c47674740 1 -- start start 2024-02-17T12:03:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.534+0000 7f7c47674740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf8e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.534+0000 7f7c47674740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf8f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.534+0000 7f7c47674740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 0x55acfcf9a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.534+0000 7f7c47674740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acfcefd9f0 con 0x55acfcf8ed60 2024-02-17T12:03:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.534+0000 7f7c47674740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acfcefaf20 con 0x55acfcf8fad0 2024-02-17T12:03:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.534+0000 7f7c47674740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acfcef6700 con 0x55acfcf982e0 2024-02-17T12:03:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.534+0000 7f7c3dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf8f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.534+0000 7f7c3dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf8f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51910/0 (socket says 172.21.15.94:51910) 2024-02-17T12:03:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.534+0000 7f7c3dd9b640 1 -- 172.21.15.94:0/2269782521 learned_addr learned my addr 172.21.15.94:0/2269782521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.535+0000 7f7c3e59c640 1 --2- 172.21.15.94:0/2269782521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf8e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.535+0000 7f7c3ed9d640 1 --2- 172.21.15.94:0/2269782521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 0x55acfcf9a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.535+0000 7f7c3e59c640 1 -- 172.21.15.94:0/2269782521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 msgr2=0x55acfcf9a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.535+0000 7f7c3e59c640 1 --2- 172.21.15.94:0/2269782521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 0x55acfcf9a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.535+0000 7f7c3e59c640 1 -- 172.21.15.94:0/2269782521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 msgr2=0x55acfcf8f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.535+0000 7f7c3e59c640 1 --2- 172.21.15.94:0/2269782521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf8f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.535+0000 7f7c3e59c640 1 -- 172.21.15.94:0/2269782521 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acfce8c7b0 con 0x55acfcf8fad0 2024-02-17T12:03:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.536+0000 7f7c3ed9d640 1 --2- 172.21.15.94:0/3066319396 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7c280a22d0 0x7f7c280a4790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006fd0 tx=0x7f7c38014660 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.536+0000 7f7c3e59c640 1 --2- 172.21.15.94:0/2269782521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf8e820 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7c3002c450 tx=0x7f7c30009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24a0c635dbe754d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.536+0000 7f7c2cff9640 1 -- 172.21.15.94:0/2269782521 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3002c9b0 con 0x55acfcf8fad0 2024-02-17T12:03:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.536+0000 7f7c2cff9640 1 -- 172.21.15.94:0/2269782521 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c300333e0 con 0x55acfcf8fad0 2024-02-17T12:03:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.536+0000 7f7c2cff9640 1 -- 172.21.15.94:0/2269782521 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30004040 con 0x55acfcf8fad0 2024-02-17T12:03:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.537+0000 7f7c47674740 1 -- 172.21.15.94:0/2269782521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 msgr2=0x55acfcf8e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.537+0000 7f7c47674740 1 --2- 172.21.15.94:0/2269782521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf8e820 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7c3002c450 tx=0x7f7c30009330 comp rx=0 tx=0).stop 2024-02-17T12:03:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.538+0000 7f7c47674740 1 -- 172.21.15.94:0/2269782521 shutdown_connections 2024-02-17T12:03:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.538+0000 7f7c47674740 1 --2- 172.21.15.94:0/2269782521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 0x55acfcf9a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.538+0000 7f7c47674740 1 --2- 172.21.15.94:0/2269782521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf8f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.538+0000 7f7c47674740 1 --2- 172.21.15.94:0/2269782521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf8e820 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.538+0000 7f7c47674740 1 -- 172.21.15.94:0/2269782521 >> 172.21.15.94:0/2269782521 conn(0x55acfcf56550 msgr2=0x55acfcf89a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.538+0000 7f7c47674740 1 -- 172.21.15.94:0/2269782521 shutdown_connections 2024-02-17T12:03:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.538+0000 7f7c47674740 1 -- 172.21.15.94:0/2269782521 wait complete. 2024-02-17T12:03:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.538+0000 7f7c47674740 1 -- start start 2024-02-17T12:03:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c47674740 1 Processor -- start 2024-02-17T12:03:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c47674740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf95200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c47674740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf95740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c47674740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 0x55acfcf95c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c47674740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acfce9bbf0 con 0x55acfcf8ed60 2024-02-17T12:03:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c47674740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acfcbca250 con 0x55acfcf8fad0 2024-02-17T12:03:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c47674740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acfce7c200 con 0x55acfcf982e0 2024-02-17T12:03:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c47674740 10 client.0 _ll_register_callbacks cb 0x55acfceead70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c3e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf95200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c3e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf95200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51924/0 (socket says 172.21.15.94:51924) 2024-02-17T12:03:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c3e59c640 1 -- 172.21.15.94:0/1249082001 learned_addr learned my addr 172.21.15.94:0/1249082001 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c3dd9b640 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf95740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c3ed9d640 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 0x55acfcf95c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c3e59c640 1 -- 172.21.15.94:0/1249082001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 msgr2=0x55acfcf95c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c3e59c640 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 0x55acfcf95c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c3e59c640 1 -- 172.21.15.94:0/1249082001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 msgr2=0x55acfcf95740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c3e59c640 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf95740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.539+0000 7f7c3e59c640 1 -- 172.21.15.94:0/1249082001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c40069230 con 0x55acfcf8ed60 2024-02-17T12:03:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.540+0000 7f7c3ed9d640 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 0x55acfcf95c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:03:15.541 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.540+0000 7f7c3dd9b640 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf95740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.540+0000 7f7c3e59c640 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf95200 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7c30036040 tx=0x7f7c30033550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b1651354009e8e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.541+0000 7f7c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:03:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.541+0000 7f7c25ffb640 1 -- 172.21.15.94:0/1249082001 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30033cf0 con 0x55acfcf8ed60 2024-02-17T12:03:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.541+0000 7f7c25ffb640 1 -- 172.21.15.94:0/1249082001 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7c30033e50 con 0x55acfcf8ed60 2024-02-17T12:03:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.541+0000 7f7c47674740 1 -- 172.21.15.94:0/1249082001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acfcbf6600 con 0x55acfcf8ed60 2024-02-17T12:03:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.541+0000 7f7c25ffb640 1 -- 172.21.15.94:0/1249082001 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c30004040 con 0x55acfcf8ed60 2024-02-17T12:03:15.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.541+0000 7f7c47674740 10 client.16833 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.541+0000 7f7c47674740 1 -- 172.21.15.94:0/1249082001 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55acfcef4d80 con 0x55acfcf8ed60 2024-02-17T12:03:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.541+0000 7f7c25ffb640 1 -- 172.21.15.94:0/1249082001 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c3005e080 con 0x55acfcf8ed60 2024-02-17T12:03:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.541+0000 7f7c25ffb640 1 -- 172.21.15.94:0/1249082001 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7c30050030 con 0x55acfcf8ed60 2024-02-17T12:03:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.541+0000 7f7c47674740 10 client.16833 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.541+0000 7f7c47674740 10 client.16833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.542+0000 7f7c47674740 20 client.16833 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.542+0000 7f7c47674740 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55acfcfa8330 0x55acfcfaa770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.542+0000 7f7c47674740 4 client.16833 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.542+0000 7f7c47674740 1 -- 172.21.15.94:0/1249082001 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acfcc471b0 con 0x55acfcfa8330 2024-02-17T12:03:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.542+0000 7f7c3dd9b640 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55acfcfa8330 0x55acfcfaa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.542+0000 7f7c3dd9b640 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55acfcfa8330 0x55acfcfaa770 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5d3cb0dd0245848 server_cookie=35d9892a62132052 in_seq=0 out_seq=0 2024-02-17T12:03:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.542+0000 7f7c25ffb640 10 client.16833 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.543+0000 7f7c25ffb640 1 -- 172.21.15.94:0/1249082001 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55acfcc471b0 con 0x55acfcfa8330 2024-02-17T12:03:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.543+0000 7f7c25ffb640 10 client.16833 handle_command_reply: tid=0 2024-02-17T12:03:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.543+0000 7f7c47674740 1 client.16833 shutdown 2024-02-17T12:03:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.543+0000 7f7c47674740 1 -- 172.21.15.94:0/1249082001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 msgr2=0x55acfcf95200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.543+0000 7f7c47674740 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf95200 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7c30036040 tx=0x7f7c30033550 comp rx=0 tx=0).stop 2024-02-17T12:03:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.543+0000 7f7c47674740 1 -- 172.21.15.94:0/1249082001 shutdown_connections 2024-02-17T12:03:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.543+0000 7f7c47674740 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55acfcfa8330 0x55acfcfaa770 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.543+0000 7f7c25ffb640 0 client.16833 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.543+0000 7f7c47674740 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 0x55acfcf95c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.543+0000 7f7c47674740 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf95740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.543+0000 7f7c47674740 1 --2- 172.21.15.94:0/1249082001 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf95200 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.543+0000 7f7c47674740 1 -- 172.21.15.94:0/1249082001 >> 172.21.15.94:0/1249082001 conn(0x55acfcf56550 msgr2=0x55acfcf8b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.544+0000 7f7c47674740 1 -- 172.21.15.94:0/1249082001 shutdown_connections 2024-02-17T12:03:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.544+0000 7f7c47674740 1 -- 172.21.15.94:0/1249082001 wait complete. 2024-02-17T12:03:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.544+0000 7f7c47674740 20 client.16833 trim_cache size 0 max 16384 2024-02-17T12:03:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.564+0000 7f7c47674740 1 Processor -- start 2024-02-17T12:03:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.564+0000 7f7c47674740 1 -- start start 2024-02-17T12:03:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.565+0000 7f7c47674740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf8ed60 0x55acfcf97930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.565+0000 7f7c47674740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8fad0 0x55acfcf92460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.565+0000 7f7c47674740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf982e0 0x55acfcf929a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.565+0000 7f7c47674740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acfcf85310 con 0x55acfcf8fad0 2024-02-17T12:03:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.565+0000 7f7c47674740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acfcf8aa50 con 0x55acfcf982e0 2024-02-17T12:03:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.565+0000 7f7c47674740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acfcf90e10 con 0x55acfcf8ed60 2024-02-17T12:03:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.565+0000 7f7c3ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf982e0 0x55acfcf929a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.565+0000 7f7c3dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8fad0 0x55acfcf92460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.565+0000 7f7c3dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8fad0 0x55acfcf92460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51932/0 (socket says 172.21.15.94:51932) 2024-02-17T12:03:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.565+0000 7f7c3dd9b640 1 -- 172.21.15.94:0/3225441579 learned_addr learned my addr 172.21.15.94:0/3225441579 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.565+0000 7f7c3e59c640 1 --2- 172.21.15.94:0/3225441579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf8ed60 0x55acfcf97930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.566+0000 7f7c3ed9d640 1 -- 172.21.15.94:0/3225441579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf8ed60 msgr2=0x55acfcf97930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.566+0000 7f7c3ed9d640 1 --2- 172.21.15.94:0/3225441579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf8ed60 0x55acfcf97930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.566+0000 7f7c3ed9d640 1 -- 172.21.15.94:0/3225441579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8fad0 msgr2=0x55acfcf92460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.566+0000 7f7c3ed9d640 1 --2- 172.21.15.94:0/3225441579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8fad0 0x55acfcf92460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.566+0000 7f7c3ed9d640 1 -- 172.21.15.94:0/3225441579 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acfce917b0 con 0x55acfcf982e0 2024-02-17T12:03:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.566+0000 7f7c3ed9d640 1 --2- 172.21.15.94:0/3225441579 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf982e0 0x55acfcf929a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7c38015010 tx=0x7f7c3801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f60a67f8723a05d8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.566+0000 7f7c27fff640 1 -- 172.21.15.94:0/3225441579 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c38027070 con 0x55acfcf982e0 2024-02-17T12:03:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.566+0000 7f7c27fff640 1 -- 172.21.15.94:0/3225441579 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c38005e70 con 0x55acfcf982e0 2024-02-17T12:03:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.566+0000 7f7c27fff640 1 -- 172.21.15.94:0/3225441579 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c380063b0 con 0x55acfcf982e0 2024-02-17T12:03:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.567+0000 7f7c47674740 1 -- 172.21.15.94:0/3225441579 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf982e0 msgr2=0x55acfcf929a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.567+0000 7f7c47674740 1 --2- 172.21.15.94:0/3225441579 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf982e0 0x55acfcf929a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7c38015010 tx=0x7f7c3801a000 comp rx=0 tx=0).stop 2024-02-17T12:03:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.568+0000 7f7c47674740 1 -- 172.21.15.94:0/3225441579 shutdown_connections 2024-02-17T12:03:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.568+0000 7f7c47674740 1 --2- 172.21.15.94:0/3225441579 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf982e0 0x55acfcf929a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.568+0000 7f7c47674740 1 --2- 172.21.15.94:0/3225441579 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8fad0 0x55acfcf92460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.568+0000 7f7c47674740 1 --2- 172.21.15.94:0/3225441579 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf8ed60 0x55acfcf97930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.568+0000 7f7c47674740 1 -- 172.21.15.94:0/3225441579 >> 172.21.15.94:0/3225441579 conn(0x55acfcf56550 msgr2=0x55acfcfaf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.568+0000 7f7c47674740 1 -- 172.21.15.94:0/3225441579 shutdown_connections 2024-02-17T12:03:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.568+0000 7f7c47674740 1 -- 172.21.15.94:0/3225441579 wait complete. 2024-02-17T12:03:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c47674740 1 -- start start 2024-02-17T12:03:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c47674740 1 Processor -- start 2024-02-17T12:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c47674740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf9f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c47674740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf9d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c47674740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 0x55acfcf9df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c47674740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acfcf9e5f0 con 0x55acfcf8ed60 2024-02-17T12:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c47674740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acfcf9e760 con 0x55acfcf8fad0 2024-02-17T12:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c47674740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acfd00f8e0 con 0x55acfcf982e0 2024-02-17T12:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c47674740 10 client.0 _ll_register_callbacks cb 0x7f7c34005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c3e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf9f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c3e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf9f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51946/0 (socket says 172.21.15.94:51946) 2024-02-17T12:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c3e59c640 1 -- 172.21.15.94:0/2550076458 learned_addr learned my addr 172.21.15.94:0/2550076458 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c3dd9b640 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf9d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.569+0000 7f7c3ed9d640 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 0x55acfcf9df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.570+0000 7f7c3e59c640 1 -- 172.21.15.94:0/2550076458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 msgr2=0x55acfcf9df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.570+0000 7f7c3e59c640 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 0x55acfcf9df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.570+0000 7f7c3e59c640 1 -- 172.21.15.94:0/2550076458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 msgr2=0x55acfcf9d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.570+0000 7f7c3e59c640 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf9d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.570+0000 7f7c3e59c640 1 -- 172.21.15.94:0/2550076458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3800e040 con 0x55acfcf8ed60 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.570+0000 7f7c3dd9b640 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf9d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.571+0000 7f7c3e59c640 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf9f550 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7c3002cbe0 tx=0x7f7c30033990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deef2f3c56531227 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.571+0000 7f7c25ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.571+0000 7f7c25ffb640 1 -- 172.21.15.94:0/2550076458 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c30034640 con 0x55acfcf8ed60 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.571+0000 7f7c25ffb640 1 -- 172.21.15.94:0/2550076458 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7c300347a0 con 0x55acfcf8ed60 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.571+0000 7f7c47674740 1 -- 172.21.15.94:0/2550076458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c300333e0 con 0x55acfcf8ed60 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.572+0000 7f7c25ffb640 1 -- 172.21.15.94:0/2550076458 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c30004000 con 0x55acfcf8ed60 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.572+0000 7f7c47674740 10 client.16839 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:15.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.572+0000 7f7c47674740 1 -- 172.21.15.94:0/2550076458 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55acfceb7540 con 0x55acfcf8ed60 2024-02-17T12:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.572+0000 7f7c25ffb640 1 -- 172.21.15.94:0/2550076458 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7c300333e0 con 0x55acfcf8ed60 2024-02-17T12:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.572+0000 7f7c25ffb640 1 -- 172.21.15.94:0/2550076458 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7c30050040 con 0x55acfcf8ed60 2024-02-17T12:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.572+0000 7f7c47674740 10 client.16839 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.572+0000 7f7c47674740 10 client.16839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.572+0000 7f7c47674740 20 client.16839 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.572+0000 7f7c47674740 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55acfcfa8330 0x55acfd012cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.572+0000 7f7c47674740 4 client.16839 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.572+0000 7f7c47674740 1 -- 172.21.15.94:0/2550076458 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55acfcc33550 con 0x55acfcfa8330 2024-02-17T12:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.573+0000 7f7c3dd9b640 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55acfcfa8330 0x55acfd012cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.573+0000 7f7c3dd9b640 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55acfcfa8330 0x55acfd012cb0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89de945640413bd2 server_cookie=94576c759dc7de2b in_seq=0 out_seq=0 2024-02-17T12:03:15.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.573+0000 7f7c25ffb640 10 client.16839 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.601+0000 7f7c25ffb640 1 -- 172.21.15.94:0/2550076458 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55acfcc33550 con 0x55acfcfa8330 2024-02-17T12:03:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.601+0000 7f7c25ffb640 10 client.16839 handle_command_reply: tid=0 2024-02-17T12:03:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.601+0000 7f7c47674740 1 client.16839 shutdown 2024-02-17T12:03:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.602+0000 7f7c47674740 1 -- 172.21.15.94:0/2550076458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 msgr2=0x55acfcf9f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.602+0000 7f7c47674740 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf9f550 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7c3002cbe0 tx=0x7f7c30033990 comp rx=0 tx=0).stop 2024-02-17T12:03:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.602+0000 7f7c47674740 1 -- 172.21.15.94:0/2550076458 shutdown_connections 2024-02-17T12:03:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.602+0000 7f7c47674740 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55acfcfa8330 0x55acfd012cb0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.602+0000 7f7c47674740 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55acfcf982e0 0x55acfcf9df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.602+0000 7f7c25ffb640 0 client.16839 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.602+0000 7f7c47674740 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55acfcf8fad0 0x55acfcf9d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.602+0000 7f7c47674740 1 --2- 172.21.15.94:0/2550076458 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55acfcf8ed60 0x55acfcf9f550 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.602+0000 7f7c47674740 1 -- 172.21.15.94:0/2550076458 >> 172.21.15.94:0/2550076458 conn(0x55acfcf56550 msgr2=0x55acfcf93ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.602+0000 7f7c47674740 1 -- 172.21.15.94:0/2550076458 shutdown_connections 2024-02-17T12:03:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.602+0000 7f7c47674740 1 -- 172.21.15.94:0/2550076458 wait complete. 2024-02-17T12:03:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.602+0000 7f7c47674740 20 client.16839 trim_cache size 0 max 16384 2024-02-17T12:03:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:03:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:03:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:03:15.604 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:03:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.603+0000 7f7c44a9d640 1 -- 172.21.15.94:0/3066319396 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7c280a22d0 msgr2=0x7f7c280a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.603+0000 7f7c44a9d640 1 --2- 172.21.15.94:0/3066319396 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7c280a22d0 0x7f7c280a4790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006fd0 tx=0x7f7c38014660 comp rx=0 tx=0).stop 2024-02-17T12:03:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.603+0000 7f7c44a9d640 1 -- 172.21.15.94:0/3066319396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c40100c90 msgr2=0x7f7c4019fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.603+0000 7f7c44a9d640 1 --2- 172.21.15.94:0/3066319396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c40100c90 0x7f7c4019fbe0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400e9c0 tx=0x7f7c3400ee80 comp rx=0 tx=0).stop 2024-02-17T12:03:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.603+0000 7f7c44a9d640 1 -- 172.21.15.94:0/3066319396 shutdown_connections 2024-02-17T12:03:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.603+0000 7f7c44a9d640 1 --2- 172.21.15.94:0/3066319396 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7c280a22d0 0x7f7c280a4790 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.603+0000 7f7c44a9d640 1 --2- 172.21.15.94:0/3066319396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7c40105020 0x7f7c40199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.603+0000 7f7c44a9d640 1 --2- 172.21.15.94:0/3066319396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7c40100c90 0x7f7c4019fbe0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.603+0000 7f7c44a9d640 1 --2- 172.21.15.94:0/3066319396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7c401002d0 0x7f7c4019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.603+0000 7f7c44a9d640 1 -- 172.21.15.94:0/3066319396 >> 172.21.15.94:0/3066319396 conn(0x7f7c40076ed0 msgr2=0x7f7c400ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.603+0000 7f7c44a9d640 1 -- 172.21.15.94:0/3066319396 shutdown_connections 2024-02-17T12:03:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:15.603+0000 7f7c44a9d640 1 -- 172.21.15.94:0/3066319396 wait complete. 2024-02-17T12:03:16.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:15 smithi125 ceph-mon[36819]: pgmap v157: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 23 MiB/s wr, 1.09k op/s 2024-02-17T12:03:16.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:15 smithi125 ceph-mon[36819]: osdmap e89: 12 total, 12 up, 12 in 2024-02-17T12:03:16.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:15 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.189:6832/39050300,v1:172.21.15.189:6833/39050300] up:boot 2024-02-17T12:03:16.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:15 smithi125 ceph-mon[36819]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-02-17T12:03:16.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:15 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:03:16.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:15 smithi125 ceph-mon[36819]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T12:03:16.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:15 smithi094 ceph-mon[32499]: pgmap v157: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 23 MiB/s wr, 1.09k op/s 2024-02-17T12:03:16.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:15 smithi094 ceph-mon[32499]: osdmap e89: 12 total, 12 up, 12 in 2024-02-17T12:03:16.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:15 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.189:6832/39050300,v1:172.21.15.189:6833/39050300] up:boot 2024-02-17T12:03:16.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:15 smithi094 ceph-mon[32499]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-02-17T12:03:16.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:15 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:03:16.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:15 smithi094 ceph-mon[32499]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T12:03:16.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:15 smithi189 ceph-mon[36588]: pgmap v157: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 23 MiB/s wr, 1.09k op/s 2024-02-17T12:03:16.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:15 smithi189 ceph-mon[36588]: osdmap e89: 12 total, 12 up, 12 in 2024-02-17T12:03:16.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:15 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.189:6832/39050300,v1:172.21.15.189:6833/39050300] up:boot 2024-02-17T12:03:16.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:15 smithi189 ceph-mon[36588]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-02-17T12:03:16.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:15 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:03:16.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:15 smithi189 ceph-mon[36588]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T12:03:18.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:17 smithi125 ceph-mon[36819]: pgmap v159: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 7.1 MiB/s wr, 683 op/s 2024-02-17T12:03:18.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:17 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:03:18.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:17 smithi094 ceph-mon[32499]: pgmap v159: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 7.1 MiB/s wr, 683 op/s 2024-02-17T12:03:18.209 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:17 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:03:18.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:17 smithi189 ceph-mon[36588]: pgmap v159: 97 pgs: 97 active+clean; 3.7 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 7.1 MiB/s wr, 683 op/s 2024-02-17T12:03:18.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:17 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:03:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:18 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:03:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:18 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:03:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:18 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:03:19.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:18 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:03:19.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:18 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:03:19.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:18 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:03:19.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:18 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:03:19.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:18 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:03:19.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:18 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:03:20.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:19 smithi094 ceph-mon[32499]: pgmap v160: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.2 MiB/s wr, 707 op/s 2024-02-17T12:03:20.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 300.0 s, 2713.2 tps, lat 36.816 ms stddev 41.029 2024-02-17T12:03:20.302 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:19 smithi189 ceph-mon[36588]: pgmap v160: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.2 MiB/s wr, 707 op/s 2024-02-17T12:03:20.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:19 smithi125 ceph-mon[36819]: pgmap v160: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.2 MiB/s wr, 707 op/s 2024-02-17T12:03:21.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:20 smithi094 ceph-mon[32499]: pgmap v161: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.2 MiB/s wr, 707 op/s 2024-02-17T12:03:21.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:20 smithi189 ceph-mon[36588]: pgmap v161: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.2 MiB/s wr, 707 op/s 2024-02-17T12:03:21.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:20 smithi125 ceph-mon[36819]: pgmap v161: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 7.2 MiB/s wr, 707 op/s 2024-02-17T12:03:24.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:23 smithi125 ceph-mon[36819]: pgmap v162: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.0 MiB/s wr, 825 op/s 2024-02-17T12:03:24.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:23 smithi094 ceph-mon[32499]: pgmap v162: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.0 MiB/s wr, 825 op/s 2024-02-17T12:03:24.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:23 smithi189 ceph-mon[36588]: pgmap v162: 97 pgs: 97 active+clean; 3.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.0 MiB/s wr, 825 op/s 2024-02-17T12:03:25.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:03:25.048 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:03:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.191+0000 7f44e44c9640 1 -- 172.21.15.94:0/2854501855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc108cb0 msgr2=0x7f44dc113d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.191+0000 7f44e44c9640 1 --2- 172.21.15.94:0/2854501855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc108cb0 0x7f44dc113d90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009900 tx=0x7f44cc02bcd0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.191+0000 7f44e44c9640 1 -- 172.21.15.94:0/2854501855 shutdown_connections 2024-02-17T12:03:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.191+0000 7f44e44c9640 1 --2- 172.21.15.94:0/2854501855 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44dc1142d0 0x7f44dc1166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.191+0000 7f44e44c9640 1 --2- 172.21.15.94:0/2854501855 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc108cb0 0x7f44dc113d90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.191+0000 7f44e44c9640 1 --2- 172.21.15.94:0/2854501855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44dc108390 0x7f44dc108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.191+0000 7f44e44c9640 1 -- 172.21.15.94:0/2854501855 >> 172.21.15.94:0/2854501855 conn(0x7f44dc06c270 msgr2=0x7f44dc06c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.191+0000 7f44e44c9640 1 -- 172.21.15.94:0/2854501855 shutdown_connections 2024-02-17T12:03:25.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.191+0000 7f44e44c9640 1 -- 172.21.15.94:0/2854501855 wait complete. 2024-02-17T12:03:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e44c9640 1 Processor -- start 2024-02-17T12:03:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e44c9640 1 -- start start 2024-02-17T12:03:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e44c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44dc108390 0x7f44dc19ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e44c9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44dc108cb0 0x7f44dc1a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e44c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc1142d0 0x7f44dc1a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e44c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1a4de0 con 0x7f44dc1142d0 2024-02-17T12:03:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e44c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1a4f50 con 0x7f44dc108390 2024-02-17T12:03:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e44c9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1a5110 con 0x7f44dc108cb0 2024-02-17T12:03:25.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e2a66640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc1142d0 0x7f44dc1a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e2265640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44dc108390 0x7f44dc19ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e2a66640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc1142d0 0x7f44dc1a47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35172/0 (socket says 172.21.15.94:35172) 2024-02-17T12:03:25.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e1a64640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44dc108cb0 0x7f44dc1a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e2a66640 1 -- 172.21.15.94:0/3407582068 learned_addr learned my addr 172.21.15.94:0/3407582068 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:25.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.192+0000 7f44e2265640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44dc108390 0x7f44dc19ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59416/0 (socket says 172.21.15.94:59416) 2024-02-17T12:03:25.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.193+0000 7f44e2a66640 1 -- 172.21.15.94:0/3407582068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44dc108cb0 msgr2=0x7f44dc1a04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.193+0000 7f44e2a66640 1 --2- 172.21.15.94:0/3407582068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44dc108cb0 0x7f44dc1a04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.193+0000 7f44e2a66640 1 -- 172.21.15.94:0/3407582068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44dc108390 msgr2=0x7f44dc19ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.193+0000 7f44e2a66640 1 --2- 172.21.15.94:0/3407582068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44dc108390 0x7f44dc19ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.193+0000 7f44e2a66640 1 -- 172.21.15.94:0/3407582068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d4009840 con 0x7f44dc1142d0 2024-02-17T12:03:25.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.193+0000 7f44e2265640 1 --2- 172.21.15.94:0/3407582068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44dc108390 0x7f44dc19ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:25.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.193+0000 7f44e2a66640 1 --2- 172.21.15.94:0/3407582068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc1142d0 0x7f44dc1a47e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f44d400efd0 tx=0x7f44d400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:25.196 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.193+0000 7f44d37fe640 1 -- 172.21.15.94:0/3407582068 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4009b40 con 0x7f44dc1142d0 2024-02-17T12:03:25.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.193+0000 7f44d37fe640 1 -- 172.21.15.94:0/3407582068 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44d4003b70 con 0x7f44dc1142d0 2024-02-17T12:03:25.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.193+0000 7f44d37fe640 1 -- 172.21.15.94:0/3407582068 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4009b40 con 0x7f44dc1142d0 2024-02-17T12:03:25.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.194+0000 7f44e44c9640 1 -- 172.21.15.94:0/3407582068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44cc009560 con 0x7f44dc1142d0 2024-02-17T12:03:25.197 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.194+0000 7f44e44c9640 1 -- 172.21.15.94:0/3407582068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44dc1aca90 con 0x7f44dc1142d0 2024-02-17T12:03:25.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.196+0000 7f44d37fe640 1 -- 172.21.15.94:0/3407582068 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f44d4003ce0 con 0x7f44dc1142d0 2024-02-17T12:03:25.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.197+0000 7f44e717a740 1 Processor -- start 2024-02-17T12:03:25.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.197+0000 7f44e717a740 1 -- start start 2024-02-17T12:03:25.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.197+0000 7f44e717a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba82442b0 0x55aba8244690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.197+0000 7f44e717a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba8244bd0 0x55aba824d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.197+0000 7f44e717a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba824d7b0 0x55aba824fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.197+0000 7f44e717a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba81b3280 con 0x55aba82442b0 2024-02-17T12:03:25.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.197+0000 7f44e717a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba81b07b0 con 0x55aba8244bd0 2024-02-17T12:03:25.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.197+0000 7f44e717a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba81abf90 con 0x55aba824d7b0 2024-02-17T12:03:25.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.197+0000 7f44e1a64640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba82442b0 0x55aba8244690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.197+0000 7f44e2265640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba8244bd0 0x55aba824d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.197+0000 7f44e2265640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba8244bd0 0x55aba824d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59432/0 (socket says 172.21.15.94:59432) 2024-02-17T12:03:25.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.197+0000 7f44e2265640 1 -- 172.21.15.94:0/592442582 learned_addr learned my addr 172.21.15.94:0/592442582 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:25.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.198+0000 7f44d37fe640 1 --2- 172.21.15.94:0/3407582068 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f44c80a22f0 0x7f44c80a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.198+0000 7f44d37fe640 1 -- 172.21.15.94:0/3407582068 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f44d4014070 con 0x7f44dc1142d0 2024-02-17T12:03:25.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.198+0000 7f44e2265640 1 -- 172.21.15.94:0/592442582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba824d7b0 msgr2=0x55aba824fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.198+0000 7f44e2265640 1 --2- 172.21.15.94:0/592442582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba824d7b0 0x55aba824fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.198+0000 7f44e2265640 1 -- 172.21.15.94:0/592442582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba82442b0 msgr2=0x55aba8244690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.198+0000 7f44e2265640 1 --2- 172.21.15.94:0/592442582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba82442b0 0x55aba8244690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.198+0000 7f44e2265640 1 -- 172.21.15.94:0/592442582 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aba816a570 con 0x55aba8244bd0 2024-02-17T12:03:25.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.199+0000 7f44e2265640 1 --2- 172.21.15.94:0/592442582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba8244bd0 0x55aba824d270 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f44d800f4c0 tx=0x7f44d800f980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b0a9b51de8c8b27 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:25.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.199+0000 7f44d0bfe640 1 -- 172.21.15.94:0/592442582 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8005220 con 0x55aba8244bd0 2024-02-17T12:03:25.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.199+0000 7f44e2a66640 1 --2- 172.21.15.94:0/3407582068 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f44c80a22f0 0x7f44c80a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.199+0000 7f44d0bfe640 1 -- 172.21.15.94:0/592442582 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44d80047a0 con 0x55aba8244bd0 2024-02-17T12:03:25.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.199+0000 7f44d0bfe640 1 -- 172.21.15.94:0/592442582 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8010c90 con 0x55aba8244bd0 2024-02-17T12:03:25.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.200+0000 7f44e2a66640 1 --2- 172.21.15.94:0/3407582068 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f44c80a22f0 0x7f44c80a47b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f44d400cba0 tx=0x7f44d400cd50 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:25.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.200+0000 7f44e717a740 1 -- 172.21.15.94:0/592442582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba8244bd0 msgr2=0x55aba824d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.200+0000 7f44e717a740 1 --2- 172.21.15.94:0/592442582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba8244bd0 0x55aba824d270 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f44d800f4c0 tx=0x7f44d800f980 comp rx=0 tx=0).stop 2024-02-17T12:03:25.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.200+0000 7f44e717a740 1 -- 172.21.15.94:0/592442582 shutdown_connections 2024-02-17T12:03:25.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.200+0000 7f44e717a740 1 --2- 172.21.15.94:0/592442582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba824d7b0 0x55aba824fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.200+0000 7f44e717a740 1 --2- 172.21.15.94:0/592442582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba8244bd0 0x55aba824d270 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.200+0000 7f44e717a740 1 --2- 172.21.15.94:0/592442582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba82442b0 0x55aba8244690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.200+0000 7f44e717a740 1 -- 172.21.15.94:0/592442582 >> 172.21.15.94:0/592442582 conn(0x55aba823e990 msgr2=0x55aba823ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.200+0000 7f44e717a740 1 -- 172.21.15.94:0/592442582 shutdown_connections 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.200+0000 7f44e717a740 1 -- 172.21.15.94:0/592442582 wait complete. 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.201+0000 7f44e717a740 1 -- start start 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.201+0000 7f44e717a740 1 Processor -- start 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.201+0000 7f44e717a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba82442b0 0x55aba824a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.201+0000 7f44e717a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba8244bd0 0x55aba824abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.201+0000 7f44e717a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba824d7b0 0x55aba824b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.201+0000 7f44e717a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba7e7f430 con 0x55aba8244bd0 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.201+0000 7f44e717a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba7e3be00 con 0x55aba824d7b0 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.201+0000 7f44e717a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba8130fd0 con 0x55aba82442b0 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.201+0000 7f44e2265640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba8244bd0 0x55aba824abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e2265640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba8244bd0 0x55aba824abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35186/0 (socket says 172.21.15.94:35186) 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e2265640 1 -- 172.21.15.94:0/2164882289 learned_addr learned my addr 172.21.15.94:0/2164882289 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:25.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e1a64640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba82442b0 0x55aba824a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e1a64640 1 --2- 172.21.15.94:0/2164882289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba824d7b0 0x55aba824b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e717a740 10 client.0 _ll_register_callbacks cb 0x55aba8186fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e2265640 1 -- 172.21.15.94:0/2164882289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba82442b0 msgr2=0x55aba824a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e2265640 1 --2- 172.21.15.94:0/2164882289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba82442b0 0x55aba824a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e2265640 1 -- 172.21.15.94:0/2164882289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba824d7b0 msgr2=0x55aba824b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e2265640 1 --2- 172.21.15.94:0/2164882289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba824d7b0 0x55aba824b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e2265640 1 -- 172.21.15.94:0/2164882289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d800acf0 con 0x55aba8244bd0 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e1a64640 1 --2- 172.21.15.94:0/2164882289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba824d7b0 0x55aba824b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e2265640 1 --2- 172.21.15.94:0/2164882289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba8244bd0 0x55aba824abf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f44d801a040 tx=0x7f44d8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4732c05b8d4e416 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44c5ffb640 1 -- 172.21.15.94:0/2164882289 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d80051a0 con 0x55aba8244bd0 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e717a740 1 -- 172.21.15.94:0/2164882289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aba7eab7e0 con 0x55aba8244bd0 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44c5ffb640 1 -- 172.21.15.94:0/2164882289 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f44d8005300 con 0x55aba8244bd0 2024-02-17T12:03:25.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44c5ffb640 1 -- 172.21.15.94:0/2164882289 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44d80054c0 con 0x55aba8244bd0 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e717a740 10 client.16851 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.202+0000 7f44e717a740 1 -- 172.21.15.94:0/2164882289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55aba81aa610 con 0x55aba8244bd0 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.203+0000 7f44c5ffb640 1 -- 172.21.15.94:0/2164882289 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aba7eab7e0 con 0x55aba8244bd0 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.203+0000 7f44c5ffb640 1 -- 172.21.15.94:0/2164882289 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f44d8020040 con 0x55aba8244bd0 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.203+0000 7f44e717a740 10 client.16851 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.203+0000 7f44e717a740 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.204+0000 7f44e717a740 20 client.16851 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.204+0000 7f44e717a740 1 --2- 172.21.15.94:0/2164882289 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aba8261e10 0x55aba8264230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.204+0000 7f44e717a740 4 client.16851 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.204+0000 7f44e717a740 1 -- 172.21.15.94:0/2164882289 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aba7ee8720 con 0x55aba8261e10 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.204+0000 7f44e1a64640 1 --2- 172.21.15.94:0/2164882289 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aba8261e10 0x55aba8264230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.205+0000 7f44c5ffb640 10 client.16851 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.205+0000 7f44e1a64640 1 --2- 172.21.15.94:0/2164882289 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aba8261e10 0x55aba8264230 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=876b9f7337bb27 server_cookie=1591ad39c7c1188b in_seq=0 out_seq=0 2024-02-17T12:03:25.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.205+0000 7f44c5ffb640 1 -- 172.21.15.94:0/2164882289 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aba7ee8720 con 0x55aba8261e10 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.205+0000 7f44c5ffb640 10 client.16851 handle_command_reply: tid=0 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.205+0000 7f44e717a740 1 client.16851 shutdown 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.206+0000 7f44e717a740 1 -- 172.21.15.94:0/2164882289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba8244bd0 msgr2=0x55aba824abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.206+0000 7f44e717a740 1 --2- 172.21.15.94:0/2164882289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba8244bd0 0x55aba824abf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f44d801a040 tx=0x7f44d8004b40 comp rx=0 tx=0).stop 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.206+0000 7f44e717a740 1 -- 172.21.15.94:0/2164882289 shutdown_connections 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.206+0000 7f44e717a740 1 --2- 172.21.15.94:0/2164882289 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aba8261e10 0x55aba8264230 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.206+0000 7f44c5ffb640 0 client.16851 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.206+0000 7f44e717a740 1 --2- 172.21.15.94:0/2164882289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba824d7b0 0x55aba824b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.206+0000 7f44e717a740 1 --2- 172.21.15.94:0/2164882289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba8244bd0 0x55aba824abf0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.206+0000 7f44e717a740 1 --2- 172.21.15.94:0/2164882289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba82442b0 0x55aba824a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.206+0000 7f44e717a740 1 -- 172.21.15.94:0/2164882289 >> 172.21.15.94:0/2164882289 conn(0x55aba823e990 msgr2=0x55aba8240730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.206+0000 7f44e717a740 1 -- 172.21.15.94:0/2164882289 shutdown_connections 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.206+0000 7f44e717a740 1 -- 172.21.15.94:0/2164882289 wait complete. 2024-02-17T12:03:25.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.206+0000 7f44e717a740 20 client.16851 trim_cache size 0 max 16384 2024-02-17T12:03:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.230+0000 7f44e717a740 1 Processor -- start 2024-02-17T12:03:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.230+0000 7f44e717a740 1 -- start start 2024-02-17T12:03:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e717a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba82442b0 0x55aba824cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e717a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba8244bd0 0x55aba8249690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e717a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba824d7b0 0x55aba824af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e717a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba8269460 con 0x55aba824d7b0 2024-02-17T12:03:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e717a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba823f790 con 0x55aba82442b0 2024-02-17T12:03:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e717a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba823a2b0 con 0x55aba8244bd0 2024-02-17T12:03:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e1a64640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba82442b0 0x55aba824cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e1a64640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba82442b0 0x55aba824cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59460/0 (socket says 172.21.15.94:59460) 2024-02-17T12:03:25.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e1a64640 1 -- 172.21.15.94:0/2373809034 learned_addr learned my addr 172.21.15.94:0/2373809034 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e2265640 1 --2- 172.21.15.94:0/2373809034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba8244bd0 0x55aba8249690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e1a64640 1 --2- 172.21.15.94:0/2373809034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba824d7b0 0x55aba824af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e1a64640 1 -- 172.21.15.94:0/2373809034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba8244bd0 msgr2=0x55aba8249690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e1a64640 1 --2- 172.21.15.94:0/2373809034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba8244bd0 0x55aba8249690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e1a64640 1 -- 172.21.15.94:0/2373809034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba824d7b0 msgr2=0x55aba824af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e1a64640 1 --2- 172.21.15.94:0/2373809034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba824d7b0 0x55aba824af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e1a64640 1 -- 172.21.15.94:0/2373809034 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aba8146ac0 con 0x55aba82442b0 2024-02-17T12:03:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.231+0000 7f44e2265640 1 --2- 172.21.15.94:0/2373809034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba8244bd0 0x55aba8249690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.232+0000 7f44e1a64640 1 --2- 172.21.15.94:0/2373809034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba82442b0 0x55aba824cde0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0058c0 tx=0x7f44cc02ca80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdf5f0d043551dd0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.232+0000 7f44c7fff640 1 -- 172.21.15.94:0/2373809034 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc049070 con 0x55aba82442b0 2024-02-17T12:03:25.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.232+0000 7f44c7fff640 1 -- 172.21.15.94:0/2373809034 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44cc0047c0 con 0x55aba82442b0 2024-02-17T12:03:25.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.232+0000 7f44c7fff640 1 -- 172.21.15.94:0/2373809034 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc04c5c0 con 0x55aba82442b0 2024-02-17T12:03:25.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.233+0000 7f44e717a740 1 -- 172.21.15.94:0/2373809034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba82442b0 msgr2=0x55aba824cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.233+0000 7f44e717a740 1 --2- 172.21.15.94:0/2373809034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba82442b0 0x55aba824cde0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0058c0 tx=0x7f44cc02ca80 comp rx=0 tx=0).stop 2024-02-17T12:03:25.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.233+0000 7f44e717a740 1 -- 172.21.15.94:0/2373809034 shutdown_connections 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.233+0000 7f44e717a740 1 --2- 172.21.15.94:0/2373809034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba824d7b0 0x55aba824af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.233+0000 7f44e717a740 1 --2- 172.21.15.94:0/2373809034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba8244bd0 0x55aba8249690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.233+0000 7f44e717a740 1 --2- 172.21.15.94:0/2373809034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba82442b0 0x55aba824cde0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.233+0000 7f44e717a740 1 -- 172.21.15.94:0/2373809034 >> 172.21.15.94:0/2373809034 conn(0x55aba823e990 msgr2=0x55aba8248f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.233+0000 7f44e717a740 1 -- 172.21.15.94:0/2373809034 shutdown_connections 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.233+0000 7f44e717a740 1 -- 172.21.15.94:0/2373809034 wait complete. 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.234+0000 7f44e717a740 1 -- start start 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.234+0000 7f44e717a740 1 Processor -- start 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.234+0000 7f44e717a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba82442b0 0x55aba8258c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.234+0000 7f44e717a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba8244bd0 0x55aba82528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.234+0000 7f44e717a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba824d7b0 0x55aba8252df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.234+0000 7f44e717a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba8259370 con 0x55aba824d7b0 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.234+0000 7f44e717a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba8253330 con 0x55aba8244bd0 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.234+0000 7f44e717a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aba82534f0 con 0x55aba82442b0 2024-02-17T12:03:25.235 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.234+0000 7f44e717a740 10 client.0 _ll_register_callbacks cb 0x7f44d800aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.235+0000 7f44e1a64640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba82442b0 0x55aba8258c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.235+0000 7f44e2265640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba8244bd0 0x55aba82528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.235+0000 7f44e1a64640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba82442b0 0x55aba8258c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57404/0 (socket says 172.21.15.94:57404) 2024-02-17T12:03:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.235+0000 7f44e1a64640 1 -- 172.21.15.94:0/830165806 learned_addr learned my addr 172.21.15.94:0/830165806 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.235+0000 7f44e1a64640 1 -- 172.21.15.94:0/830165806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba8244bd0 msgr2=0x55aba82528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.235+0000 7f44e1a64640 1 --2- 172.21.15.94:0/830165806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba8244bd0 0x55aba82528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.235+0000 7f44e1a64640 1 -- 172.21.15.94:0/830165806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba824d7b0 msgr2=0x55aba8252df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.235+0000 7f44e1a64640 1 --2- 172.21.15.94:0/830165806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba824d7b0 0x55aba8252df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.235+0000 7f44e1a64640 1 -- 172.21.15.94:0/830165806 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aba81aa610 con 0x55aba82442b0 2024-02-17T12:03:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.238+0000 7f44e1a64640 1 --2- 172.21.15.94:0/830165806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba82442b0 0x55aba8258c70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f44cc005ec0 tx=0x7f44cc02c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbecd14a81906cfd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:25.238 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.238+0000 7f44c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:03:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.239+0000 7f44c5ffb640 1 -- 172.21.15.94:0/830165806 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc049070 con 0x55aba82442b0 2024-02-17T12:03:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.239+0000 7f44c5ffb640 1 -- 172.21.15.94:0/830165806 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f44cc04c740 con 0x55aba82442b0 2024-02-17T12:03:25.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.239+0000 7f44e717a740 1 -- 172.21.15.94:0/830165806 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44d80047a0 con 0x55aba82442b0 2024-02-17T12:03:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.242+0000 7f44c5ffb640 1 -- 172.21.15.94:0/830165806 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44cc04c900 con 0x55aba82442b0 2024-02-17T12:03:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.245+0000 7f44e717a740 10 client.26362 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:25.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.245+0000 7f44e717a740 1 -- 172.21.15.94:0/830165806 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f44dc117010 con 0x55aba82442b0 2024-02-17T12:03:25.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 305.0 s, 3171.9 tps, lat 31.565 ms stddev 37.452 2024-02-17T12:03:25.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.249+0000 7f44c5ffb640 1 -- 172.21.15.94:0/830165806 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f44d80047a0 con 0x55aba82442b0 2024-02-17T12:03:25.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.249+0000 7f44c5ffb640 1 -- 172.21.15.94:0/830165806 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f44cc078710 con 0x55aba82442b0 2024-02-17T12:03:25.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.249+0000 7f44e717a740 10 client.26362 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:25.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.249+0000 7f44e717a740 10 client.26362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:25.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.249+0000 7f44e717a740 20 client.26362 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:25.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.249+0000 7f44e717a740 1 --2- 172.21.15.94:0/830165806 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aba82b51b0 0x55aba82b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.249+0000 7f44e717a740 4 client.26362 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:03:25.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.249+0000 7f44e717a740 1 -- 172.21.15.94:0/830165806 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aba7f37890 con 0x55aba82b51b0 2024-02-17T12:03:25.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.251+0000 7f44e2265640 1 --2- 172.21.15.94:0/830165806 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aba82b51b0 0x55aba82b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.259+0000 7f44e2265640 1 --2- 172.21.15.94:0/830165806 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aba82b51b0 0x55aba82b75d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd46270f5c47ac71 server_cookie=3dfcd9e6ce98f4a8 in_seq=0 out_seq=0 2024-02-17T12:03:25.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.259+0000 7f44c5ffb640 10 client.26362 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.260+0000 7f44c5ffb640 1 -- 172.21.15.94:0/830165806 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55aba7f37890 con 0x55aba82b51b0 2024-02-17T12:03:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.260+0000 7f44c5ffb640 10 client.26362 handle_command_reply: tid=0 2024-02-17T12:03:25.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.260+0000 7f44e717a740 1 client.26362 shutdown 2024-02-17T12:03:25.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.264+0000 7f44e717a740 1 -- 172.21.15.94:0/830165806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba82442b0 msgr2=0x55aba8258c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.264+0000 7f44e717a740 1 --2- 172.21.15.94:0/830165806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba82442b0 0x55aba8258c70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f44cc005ec0 tx=0x7f44cc02c480 comp rx=0 tx=0).stop 2024-02-17T12:03:25.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.271+0000 7f44e717a740 1 -- 172.21.15.94:0/830165806 shutdown_connections 2024-02-17T12:03:25.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.271+0000 7f44e717a740 1 --2- 172.21.15.94:0/830165806 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aba82b51b0 0x55aba82b75d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.271+0000 7f44e717a740 1 --2- 172.21.15.94:0/830165806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aba824d7b0 0x55aba8252df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.271+0000 7f44e717a740 1 --2- 172.21.15.94:0/830165806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aba8244bd0 0x55aba82528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.271+0000 7f44e717a740 1 --2- 172.21.15.94:0/830165806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aba82442b0 0x55aba8258c70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.271+0000 7f44e717a740 1 -- 172.21.15.94:0/830165806 >> 172.21.15.94:0/830165806 conn(0x55aba823e990 msgr2=0x55aba823fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:25.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.271+0000 7f44c5ffb640 0 client.26362 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:25.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.271+0000 7f44e717a740 1 -- 172.21.15.94:0/830165806 shutdown_connections 2024-02-17T12:03:25.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.274+0000 7f44e717a740 1 -- 172.21.15.94:0/830165806 wait complete. 2024-02-17T12:03:25.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.277+0000 7f44e717a740 20 client.26362 trim_cache size 0 max 16384 2024-02-17T12:03:25.279 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:03:25.279 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:03:25.279 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:03:25.279 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:03:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.279+0000 7f44e44c9640 1 -- 172.21.15.94:0/3407582068 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f44c80a22f0 msgr2=0x7f44c80a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.279+0000 7f44e44c9640 1 --2- 172.21.15.94:0/3407582068 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f44c80a22f0 0x7f44c80a47b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f44d400cba0 tx=0x7f44d400cd50 comp rx=0 tx=0).stop 2024-02-17T12:03:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.279+0000 7f44e44c9640 1 -- 172.21.15.94:0/3407582068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc1142d0 msgr2=0x7f44dc1a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.279+0000 7f44e44c9640 1 --2- 172.21.15.94:0/3407582068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc1142d0 0x7f44dc1a47e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f44d400efd0 tx=0x7f44d400c6a0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.280+0000 7f44e44c9640 1 -- 172.21.15.94:0/3407582068 shutdown_connections 2024-02-17T12:03:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.280+0000 7f44e44c9640 1 --2- 172.21.15.94:0/3407582068 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f44c80a22f0 0x7f44c80a47b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.280+0000 7f44e44c9640 1 --2- 172.21.15.94:0/3407582068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f44dc1142d0 0x7f44dc1a47e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.280+0000 7f44e44c9640 1 --2- 172.21.15.94:0/3407582068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f44dc108cb0 0x7f44dc1a04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.280+0000 7f44e44c9640 1 --2- 172.21.15.94:0/3407582068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f44dc108390 0x7f44dc19ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.280+0000 7f44e44c9640 1 -- 172.21.15.94:0/3407582068 >> 172.21.15.94:0/3407582068 conn(0x7f44dc06c270 msgr2=0x7f44dc06fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:25.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.280+0000 7f44e44c9640 1 -- 172.21.15.94:0/3407582068 shutdown_connections 2024-02-17T12:03:25.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.280+0000 7f44e44c9640 1 -- 172.21.15.94:0/3407582068 wait complete. 2024-02-17T12:03:25.295 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:03:25.296 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:03:25.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.429+0000 7f6b42de9640 1 -- 172.21.15.94:0/3785710323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b3c1009e0 msgr2=0x7f6b3c10cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.429+0000 7f6b42de9640 1 --2- 172.21.15.94:0/3785710323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b3c1009e0 0x7f6b3c10cea0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009ca0 tx=0x7f6b3402c180 comp rx=0 tx=0).stop 2024-02-17T12:03:25.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.430+0000 7f6b42de9640 1 -- 172.21.15.94:0/3785710323 shutdown_connections 2024-02-17T12:03:25.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.430+0000 7f6b42de9640 1 --2- 172.21.15.94:0/3785710323 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b3c1009e0 0x7f6b3c10cea0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.430+0000 7f6b42de9640 1 --2- 172.21.15.94:0/3785710323 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b3c100040 0x7f6b3c1004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.430+0000 7f6b42de9640 1 --2- 172.21.15.94:0/3785710323 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b3c106100 0x7f6b3c1064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.430+0000 7f6b42de9640 1 -- 172.21.15.94:0/3785710323 >> 172.21.15.94:0/3785710323 conn(0x7f6b3c077110 msgr2=0x7f6b3c077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:25.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.431+0000 7f6b42de9640 1 -- 172.21.15.94:0/3785710323 shutdown_connections 2024-02-17T12:03:25.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.431+0000 7f6b42de9640 1 -- 172.21.15.94:0/3785710323 wait complete. 2024-02-17T12:03:25.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.431+0000 7f6b42de9640 1 Processor -- start 2024-02-17T12:03:25.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.431+0000 7f6b42de9640 1 -- start start 2024-02-17T12:03:25.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b42de9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b3c100040 0x7f6b3c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b42de9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b3c1009e0 0x7f6b3c19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b42de9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b3c106100 0x7f6b3c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b42de9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c199e40 con 0x7f6b3c106100 2024-02-17T12:03:25.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b42de9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c199fb0 con 0x7f6b3c100040 2024-02-17T12:03:25.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b42de9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3c19a170 con 0x7f6b3c1009e0 2024-02-17T12:03:25.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b41386640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b3c106100 0x7f6b3c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b3bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b3c1009e0 0x7f6b3c19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b41386640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b3c106100 0x7f6b3c1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35236/0 (socket says 172.21.15.94:35236) 2024-02-17T12:03:25.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b41386640 1 -- 172.21.15.94:0/125071721 learned_addr learned my addr 172.21.15.94:0/125071721 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:25.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b40b85640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b3c100040 0x7f6b3c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b41386640 1 -- 172.21.15.94:0/125071721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b3c1009e0 msgr2=0x7f6b3c19fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b41386640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b3c1009e0 0x7f6b3c19fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b41386640 1 -- 172.21.15.94:0/125071721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b3c100040 msgr2=0x7f6b3c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b41386640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b3c100040 0x7f6b3c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b41386640 1 -- 172.21.15.94:0/125071721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b34009900 con 0x7f6b3c106100 2024-02-17T12:03:25.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b40b85640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b3c100040 0x7f6b3c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:03:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b3bfff640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b3c1009e0 0x7f6b3c19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b41386640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b3c106100 0x7f6b3c1996b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6b3402c790 tx=0x7f6b34034810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.432+0000 7f6b39ffb640 1 -- 172.21.15.94:0/125071721 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34003d80 con 0x7f6b3c106100 2024-02-17T12:03:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.433+0000 7f6b42de9640 1 -- 172.21.15.94:0/125071721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3c19a3f0 con 0x7f6b3c106100 2024-02-17T12:03:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.433+0000 7f6b39ffb640 1 -- 172.21.15.94:0/125071721 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6b34032c70 con 0x7f6b3c106100 2024-02-17T12:03:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.433+0000 7f6b39ffb640 1 -- 172.21.15.94:0/125071721 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b34004860 con 0x7f6b3c106100 2024-02-17T12:03:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.433+0000 7f6b42de9640 1 -- 172.21.15.94:0/125071721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3c1a6570 con 0x7f6b3c106100 2024-02-17T12:03:25.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.434+0000 7f6b42de9640 1 -- 172.21.15.94:0/125071721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3c101730 con 0x7f6b3c106100 2024-02-17T12:03:25.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.436+0000 7f6b39ffb640 1 -- 172.21.15.94:0/125071721 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6b3400a040 con 0x7f6b3c106100 2024-02-17T12:03:25.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.437+0000 7f6b39ffb640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6b280a2230 0x7f6b280a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.437+0000 7f6b40b85640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6b280a2230 0x7f6b280a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.437+0000 7f6b39ffb640 1 -- 172.21.15.94:0/125071721 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f6b340f0d30 con 0x7f6b3c106100 2024-02-17T12:03:25.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.437+0000 7f6b40b85640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6b280a2230 0x7f6b280a46f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c005fd0 tx=0x7f6b2c005850 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:25.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.443+0000 7f6b39ffb640 1 -- 172.21.15.94:0/125071721 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b340b9aa0 con 0x7f6b3c106100 2024-02-17T12:03:25.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.796+0000 7f6b42de9640 1 -- 172.21.15.94:0/125071721 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b3c19a580 con 0x7f6b3c106100 2024-02-17T12:03:25.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.797+0000 7f6b39ffb640 1 -- 172.21.15.94:0/125071721 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11338 (secure 0 0 0) 0x7f6b340bd160 con 0x7f6b3c106100 2024-02-17T12:03:25.797 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:03:25.798 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:03:14.932679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26335":{"gid":26335,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/39050300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":39050300},{"type":"v1","addr":"172.21.15.189:6833","nonce":39050300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:03:25.798 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-02-17T12:03:25.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.799+0000 7f6b42de9640 1 -- 172.21.15.94:0/125071721 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6b280a2230 msgr2=0x7f6b280a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.799+0000 7f6b42de9640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6b280a2230 0x7f6b280a46f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c005fd0 tx=0x7f6b2c005850 comp rx=0 tx=0).stop 2024-02-17T12:03:25.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.799+0000 7f6b42de9640 1 -- 172.21.15.94:0/125071721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b3c106100 msgr2=0x7f6b3c1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.799+0000 7f6b42de9640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b3c106100 0x7f6b3c1996b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6b3402c790 tx=0x7f6b34034810 comp rx=0 tx=0).stop 2024-02-17T12:03:25.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.800+0000 7f6b42de9640 1 -- 172.21.15.94:0/125071721 shutdown_connections 2024-02-17T12:03:25.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.800+0000 7f6b42de9640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6b280a2230 0x7f6b280a46f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.800+0000 7f6b42de9640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6b3c106100 0x7f6b3c1996b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.800+0000 7f6b42de9640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6b3c1009e0 0x7f6b3c19fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.800+0000 7f6b42de9640 1 --2- 172.21.15.94:0/125071721 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6b3c100040 0x7f6b3c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.800+0000 7f6b42de9640 1 -- 172.21.15.94:0/125071721 >> 172.21.15.94:0/125071721 conn(0x7f6b3c077110 msgr2=0x7f6b3c0fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:25.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.800+0000 7f6b42de9640 1 -- 172.21.15.94:0/125071721 shutdown_connections 2024-02-17T12:03:25.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.800+0000 7f6b42de9640 1 -- 172.21.15.94:0/125071721 wait complete. 2024-02-17T12:03:25.813 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:03:25.813 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:03:25.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.964+0000 7fea28202640 1 -- 172.21.15.94:0/266016983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea201071d0 msgr2=0x7fea201075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.964+0000 7fea28202640 1 --2- 172.21.15.94:0/266016983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea201071d0 0x7fea201075b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009930 tx=0x7fea1c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:03:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.964+0000 7fea28202640 1 -- 172.21.15.94:0/266016983 shutdown_connections 2024-02-17T12:03:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.964+0000 7fea28202640 1 --2- 172.21.15.94:0/266016983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea2006a070 0x7fea2010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.964+0000 7fea28202640 1 --2- 172.21.15.94:0/266016983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea200696f0 0x7fea20069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.964+0000 7fea28202640 1 --2- 172.21.15.94:0/266016983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea201071d0 0x7fea201075b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.964+0000 7fea28202640 1 -- 172.21.15.94:0/266016983 >> 172.21.15.94:0/266016983 conn(0x7fea20077060 msgr2=0x7fea20077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.964+0000 7fea28202640 1 -- 172.21.15.94:0/266016983 shutdown_connections 2024-02-17T12:03:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.964+0000 7fea28202640 1 -- 172.21.15.94:0/266016983 wait complete. 2024-02-17T12:03:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.965+0000 7fea28202640 1 Processor -- start 2024-02-17T12:03:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.965+0000 7fea28202640 1 -- start start 2024-02-17T12:03:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.965+0000 7fea28202640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea200696f0 0x7fea2019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.965+0000 7fea28202640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea2006a070 0x7fea2019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.965+0000 7fea28202640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea201071d0 0x7fea20199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.965+0000 7fea28202640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea20199ee0 con 0x7fea201071d0 2024-02-17T12:03:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.965+0000 7fea28202640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2019a050 con 0x7fea200696f0 2024-02-17T12:03:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.965+0000 7fea28202640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2019a210 con 0x7fea2006a070 2024-02-17T12:03:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.965+0000 7fea2679f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea201071d0 0x7fea20199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.965+0000 7fea2679f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea201071d0 0x7fea20199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35254/0 (socket says 172.21.15.94:35254) 2024-02-17T12:03:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.965+0000 7fea2679f640 1 -- 172.21.15.94:0/2847985273 learned_addr learned my addr 172.21.15.94:0/2847985273 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.965+0000 7fea2579d640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea2006a070 0x7fea2019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.965+0000 7fea25f9e640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea200696f0 0x7fea2019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.966+0000 7fea2679f640 1 -- 172.21.15.94:0/2847985273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea2006a070 msgr2=0x7fea2019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.966+0000 7fea2679f640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea2006a070 0x7fea2019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.966+0000 7fea2679f640 1 -- 172.21.15.94:0/2847985273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea200696f0 msgr2=0x7fea2019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.966+0000 7fea2679f640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea200696f0 0x7fea2019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.966+0000 7fea2679f640 1 -- 172.21.15.94:0/2847985273 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea10009490 con 0x7fea201071d0 2024-02-17T12:03:25.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.966+0000 7fea2579d640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea2006a070 0x7fea2019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:25.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.966+0000 7fea2679f640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea201071d0 0x7fea20199840 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fea1800eb10 tx=0x7fea1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:25.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.966+0000 7fea16ffd640 1 -- 172.21.15.94:0/2847985273 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18004b20 con 0x7fea201071d0 2024-02-17T12:03:25.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.966+0000 7fea28202640 1 -- 172.21.15.94:0/2847985273 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1c009560 con 0x7fea201071d0 2024-02-17T12:03:25.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.966+0000 7fea16ffd640 1 -- 172.21.15.94:0/2847985273 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea18004c80 con 0x7fea201071d0 2024-02-17T12:03:25.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.966+0000 7fea16ffd640 1 -- 172.21.15.94:0/2847985273 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18010670 con 0x7fea201071d0 2024-02-17T12:03:25.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.966+0000 7fea28202640 1 -- 172.21.15.94:0/2847985273 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2019a6e0 con 0x7fea201071d0 2024-02-17T12:03:25.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.969+0000 7fea16ffd640 1 -- 172.21.15.94:0/2847985273 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fea18010890 con 0x7fea201071d0 2024-02-17T12:03:25.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.969+0000 7fea2add5740 1 Processor -- start 2024-02-17T12:03:25.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.969+0000 7fea2add5740 1 -- start start 2024-02-17T12:03:25.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.969+0000 7fea2add5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47250 0x558334a47630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.969+0000 7fea2add5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334a50210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.969+0000 7fea2add5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 0x558334a52b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.969+0000 7fea2add5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334a02960 con 0x558334a47b70 2024-02-17T12:03:25.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.969+0000 7fea2add5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583349b6310 con 0x558334a50750 2024-02-17T12:03:25.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.969+0000 7fea2add5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583349b3840 con 0x558334a47250 2024-02-17T12:03:25.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea16ffd640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fea0c0a2230 0x7fea0c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea25f9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334a50210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea2579d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 0x558334a52b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea25f9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334a50210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35260/0 (socket says 172.21.15.94:35260) 2024-02-17T12:03:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea25f9e640 1 -- 172.21.15.94:0/1100467779 learned_addr learned my addr 172.21.15.94:0/1100467779 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea2579d640 1 --2- 172.21.15.94:0/1100467779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47250 0x558334a47630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea16ffd640 1 -- 172.21.15.94:0/2847985273 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fea18014070 con 0x7fea201071d0 2024-02-17T12:03:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea2679f640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fea0c0a2230 0x7fea0c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea2579d640 1 -- 172.21.15.94:0/1100467779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47250 msgr2=0x558334a47630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea2579d640 1 --2- 172.21.15.94:0/1100467779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47250 0x558334a47630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea2579d640 1 -- 172.21.15.94:0/1100467779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 msgr2=0x558334a50210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea2579d640 1 --2- 172.21.15.94:0/1100467779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334a50210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea2579d640 1 -- 172.21.15.94:0/1100467779 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558334971060 con 0x558334a50750 2024-02-17T12:03:25.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea25f9e640 1 --2- 172.21.15.94:0/1100467779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334a50210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea2679f640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fea0c0a2230 0x7fea0c0a46f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fea180a31d0 tx=0x7fea18009a70 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea2579d640 1 --2- 172.21.15.94:0/1100467779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 0x558334a52b40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fea200fe440 tx=0x7fea10009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5671f9ab4fb64d58 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea0bfff640 1 -- 172.21.15.94:0/1100467779 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1000fc20 con 0x558334a50750 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.970+0000 7fea0bfff640 1 -- 172.21.15.94:0/1100467779 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea10004510 con 0x558334a50750 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.971+0000 7fea0bfff640 1 -- 172.21.15.94:0/1100467779 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea100047a0 con 0x558334a50750 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.971+0000 7fea2add5740 1 -- 172.21.15.94:0/1100467779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 msgr2=0x558334a52b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.971+0000 7fea2add5740 1 --2- 172.21.15.94:0/1100467779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 0x558334a52b40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fea200fe440 tx=0x7fea10009770 comp rx=0 tx=0).stop 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.971+0000 7fea2add5740 1 -- 172.21.15.94:0/1100467779 shutdown_connections 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.971+0000 7fea2add5740 1 --2- 172.21.15.94:0/1100467779 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 0x558334a52b40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.971+0000 7fea2add5740 1 --2- 172.21.15.94:0/1100467779 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334a50210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.971+0000 7fea2add5740 1 --2- 172.21.15.94:0/1100467779 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47250 0x558334a47630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.971+0000 7fea2add5740 1 -- 172.21.15.94:0/1100467779 >> 172.21.15.94:0/1100467779 conn(0x558334a41980 msgr2=0x558334a41d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.971+0000 7fea2add5740 1 -- 172.21.15.94:0/1100467779 shutdown_connections 2024-02-17T12:03:25.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.971+0000 7fea2add5740 1 -- 172.21.15.94:0/1100467779 wait complete. 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.972+0000 7fea2add5740 1 -- start start 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.972+0000 7fea2add5740 1 Processor -- start 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.972+0000 7fea2add5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a47250 0x558334a4d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.972+0000 7fea2add5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334a4dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.972+0000 7fea2add5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a50750 0x558334a4e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.972+0000 7fea2add5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583349af020 con 0x558334a47b70 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.972+0000 7fea2add5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334682430 con 0x558334a47250 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.972+0000 7fea2add5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55833463ee00 con 0x558334a50750 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.972+0000 7fea2add5740 10 client.0 _ll_register_callbacks cb 0x558334956cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.972+0000 7fea2579d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a47250 0x558334a4d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.972+0000 7fea2579d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a47250 0x558334a4d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59532/0 (socket says 172.21.15.94:59532) 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.972+0000 7fea2579d640 1 -- 172.21.15.94:0/3065125679 learned_addr learned my addr 172.21.15.94:0/3065125679 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.973+0000 7fea25f9e640 1 --2- 172.21.15.94:0/3065125679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334a4dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.973+0000 7fea2579d640 1 --2- 172.21.15.94:0/3065125679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a50750 0x558334a4e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.973+0000 7fea2579d640 1 -- 172.21.15.94:0/3065125679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a50750 msgr2=0x558334a4e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.973+0000 7fea2579d640 1 --2- 172.21.15.94:0/3065125679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a50750 0x558334a4e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.973+0000 7fea2579d640 1 -- 172.21.15.94:0/3065125679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 msgr2=0x558334a4dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.973+0000 7fea2579d640 1 --2- 172.21.15.94:0/3065125679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334a4dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.973+0000 7fea2579d640 1 -- 172.21.15.94:0/3065125679 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea10019040 con 0x558334a47250 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.973+0000 7fea2579d640 1 --2- 172.21.15.94:0/3065125679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a47250 0x558334a4d6a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fea10005350 tx=0x7fea100116e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fea75e3b419d870c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.973+0000 7fea097fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.974+0000 7fea097fa640 1 -- 172.21.15.94:0/3065125679 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea100048d0 con 0x558334a47250 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.974+0000 7fea097fa640 1 -- 172.21.15.94:0/3065125679 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fea10000e40 con 0x558334a47250 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.974+0000 7fea2add5740 1 -- 172.21.15.94:0/3065125679 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5583346ae7e0 con 0x558334a47250 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.974+0000 7fea097fa640 1 -- 172.21.15.94:0/3065125679 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea10004a30 con 0x558334a47250 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.974+0000 7fea2add5740 10 client.26336 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.974+0000 7fea2add5740 1 -- 172.21.15.94:0/3065125679 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5583349ad6a0 con 0x558334a47250 2024-02-17T12:03:25.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.975+0000 7fea097fa640 1 -- 172.21.15.94:0/3065125679 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5583346ae7e0 con 0x558334a47250 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.975+0000 7fea097fa640 1 -- 172.21.15.94:0/3065125679 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fea1001d070 con 0x558334a47250 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.975+0000 7fea2add5740 10 client.26336 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.975+0000 7fea2add5740 10 client.26336 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.975+0000 7fea2add5740 20 client.26336 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.975+0000 7fea2add5740 1 --2- 172.21.15.94:0/3065125679 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558334a691f0 0x558334a6b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.975+0000 7fea2add5740 4 client.26336 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.975+0000 7fea2add5740 1 -- 172.21.15.94:0/3065125679 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583346eb720 con 0x558334a691f0 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.976+0000 7fea25f9e640 1 --2- 172.21.15.94:0/3065125679 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558334a691f0 0x558334a6b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.976+0000 7fea097fa640 10 client.26336 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.976+0000 7fea25f9e640 1 --2- 172.21.15.94:0/3065125679 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558334a691f0 0x558334a6b5e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d13dea0f2ad19cb server_cookie=5baeb2b3e5f328ac in_seq=0 out_seq=0 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea097fa640 1 -- 172.21.15.94:0/3065125679 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5583346eb720 con 0x558334a691f0 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea097fa640 10 client.26336 handle_command_reply: tid=0 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea2add5740 1 client.26336 shutdown 2024-02-17T12:03:25.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea2add5740 1 -- 172.21.15.94:0/3065125679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a47250 msgr2=0x558334a4d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:25.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea2add5740 1 --2- 172.21.15.94:0/3065125679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a47250 0x558334a4d6a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fea10005350 tx=0x7fea100116e0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea2add5740 1 -- 172.21.15.94:0/3065125679 shutdown_connections 2024-02-17T12:03:25.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea2add5740 1 --2- 172.21.15.94:0/3065125679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a50750 0x558334a4e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea2add5740 1 --2- 172.21.15.94:0/3065125679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334a4dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea2add5740 1 --2- 172.21.15.94:0/3065125679 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558334a691f0 0x558334a6b5e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea2add5740 1 --2- 172.21.15.94:0/3065125679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a47250 0x558334a4d6a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea097fa640 0 client.26336 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:03:25.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea2add5740 1 -- 172.21.15.94:0/3065125679 >> 172.21.15.94:0/3065125679 conn(0x558334a41980 msgr2=0x558334a436d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:25.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea2add5740 1 -- 172.21.15.94:0/3065125679 shutdown_connections 2024-02-17T12:03:25.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea2add5740 1 -- 172.21.15.94:0/3065125679 wait complete. 2024-02-17T12:03:25.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.977+0000 7fea2add5740 20 client.26336 trim_cache size 0 max 16384 2024-02-17T12:03:25.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.994+0000 7fea2add5740 1 Processor -- start 2024-02-17T12:03:25.994 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.994+0000 7fea2add5740 1 -- start start 2024-02-17T12:03:25.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.994+0000 7fea2add5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47250 0x558334a4fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.994+0000 7fea2add5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47b70 0x558334a57660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.994+0000 7fea2add5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 0x558334a57ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:25.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.994+0000 7fea2add5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334934130 con 0x558334a47250 2024-02-17T12:03:25.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.994+0000 7fea2add5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334a42790 con 0x558334a50750 2024-02-17T12:03:25.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.994+0000 7fea2add5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334a52b40 con 0x558334a47b70 2024-02-17T12:03:25.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.994+0000 7fea25f9e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47b70 0x558334a57660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.994+0000 7fea25f9e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47b70 0x558334a57660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57474/0 (socket says 172.21.15.94:57474) 2024-02-17T12:03:25.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.994+0000 7fea25f9e640 1 -- 172.21.15.94:0/991041748 learned_addr learned my addr 172.21.15.94:0/991041748 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:25.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.995+0000 7fea2579d640 1 --2- 172.21.15.94:0/991041748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47250 0x558334a4fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:25.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.995+0000 7fea25f9e640 1 -- 172.21.15.94:0/991041748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 msgr2=0x558334a57ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.995+0000 7fea25f9e640 1 --2- 172.21.15.94:0/991041748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 0x558334a57ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.995+0000 7fea25f9e640 1 -- 172.21.15.94:0/991041748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47250 msgr2=0x558334a4fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:25.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.995+0000 7fea25f9e640 1 --2- 172.21.15.94:0/991041748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47250 0x558334a4fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:25.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.995+0000 7fea25f9e640 1 -- 172.21.15.94:0/991041748 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558334949c20 con 0x558334a47b70 2024-02-17T12:03:25.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.995+0000 7fea25f9e640 1 --2- 172.21.15.94:0/991041748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47b70 0x558334a57660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea1c036040 tx=0x7fea1c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7dcec4132c59fd1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:25.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.995+0000 7fea0b7fe640 1 -- 172.21.15.94:0/991041748 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0049e0 con 0x558334a47b70 2024-02-17T12:03:25.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.995+0000 7fea0b7fe640 1 -- 172.21.15.94:0/991041748 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea1c048be0 con 0x558334a47b70 2024-02-17T12:03:25.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.995+0000 7fea0b7fe640 1 -- 172.21.15.94:0/991041748 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0483a0 con 0x558334a47b70 2024-02-17T12:03:26.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.996+0000 7fea2add5740 1 -- 172.21.15.94:0/991041748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47b70 msgr2=0x558334a57660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.996+0000 7fea2add5740 1 --2- 172.21.15.94:0/991041748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47b70 0x558334a57660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea1c036040 tx=0x7fea1c009260 comp rx=0 tx=0).stop 2024-02-17T12:03:26.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.996+0000 7fea2add5740 1 -- 172.21.15.94:0/991041748 shutdown_connections 2024-02-17T12:03:26.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.996+0000 7fea2add5740 1 --2- 172.21.15.94:0/991041748 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 0x558334a57ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.996+0000 7fea2add5740 1 --2- 172.21.15.94:0/991041748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47b70 0x558334a57660 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.996+0000 7fea2add5740 1 --2- 172.21.15.94:0/991041748 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47250 0x558334a4fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.996+0000 7fea2add5740 1 -- 172.21.15.94:0/991041748 >> 172.21.15.94:0/991041748 conn(0x558334a41980 msgr2=0x558334a489f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.996+0000 7fea2add5740 1 -- 172.21.15.94:0/991041748 shutdown_connections 2024-02-17T12:03:26.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.996+0000 7fea2add5740 1 -- 172.21.15.94:0/991041748 wait complete. 2024-02-17T12:03:26.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.996+0000 7fea2add5740 1 -- start start 2024-02-17T12:03:26.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.996+0000 7fea2add5740 1 Processor -- start 2024-02-17T12:03:26.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2add5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47250 0x558334ab8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2add5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334ab9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2add5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 0x558334ab3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2add5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334ab3c20 con 0x558334a47b70 2024-02-17T12:03:26.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2add5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334ab3d90 con 0x558334a50750 2024-02-17T12:03:26.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2add5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558334ab3f50 con 0x558334a47250 2024-02-17T12:03:26.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2add5740 10 client.0 _ll_register_callbacks cb 0x7fea10011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:26.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2579d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47250 0x558334ab8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2579d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47250 0x558334ab8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57490/0 (socket says 172.21.15.94:57490) 2024-02-17T12:03:26.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2579d640 1 -- 172.21.15.94:0/1659225319 learned_addr learned my addr 172.21.15.94:0/1659225319 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:26.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea25f9e640 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334ab9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2579d640 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 0x558334ab3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2579d640 1 -- 172.21.15.94:0/1659225319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 msgr2=0x558334ab3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2579d640 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 0x558334ab3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2579d640 1 -- 172.21.15.94:0/1659225319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 msgr2=0x558334ab9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2579d640 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334ab9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2579d640 1 -- 172.21.15.94:0/1659225319 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c030070 con 0x558334a47250 2024-02-17T12:03:26.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea25f9e640 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334ab9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:26.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea2579d640 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47250 0x558334ab8c50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea10011af0 tx=0x7fea10011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9acadd170dc917f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:26.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea097fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:03:26.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.997+0000 7fea097fa640 1 -- 172.21.15.94:0/1659225319 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea10004e60 con 0x558334a47250 2024-02-17T12:03:26.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.998+0000 7fea097fa640 1 -- 172.21.15.94:0/1659225319 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fea10011750 con 0x558334a47250 2024-02-17T12:03:26.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.998+0000 7fea2add5740 1 -- 172.21.15.94:0/1659225319 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea10004510 con 0x558334a47250 2024-02-17T12:03:26.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.998+0000 7fea097fa640 1 -- 172.21.15.94:0/1659225319 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea100056f0 con 0x558334a47250 2024-02-17T12:03:26.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.999+0000 7fea2add5740 10 client.26386 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:26.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.999+0000 7fea2add5740 1 -- 172.21.15.94:0/1659225319 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55833496f8c0 con 0x558334a47250 2024-02-17T12:03:26.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.999+0000 7fea097fa640 1 -- 172.21.15.94:0/1659225319 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea10004510 con 0x558334a47250 2024-02-17T12:03:26.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.999+0000 7fea097fa640 1 -- 172.21.15.94:0/1659225319 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fea1001d050 con 0x558334a47250 2024-02-17T12:03:26.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.999+0000 7fea2add5740 10 client.26386 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:26.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.999+0000 7fea2add5740 10 client.26386 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:03:26.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.999+0000 7fea2add5740 20 client.26386 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:26.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.999+0000 7fea2add5740 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558334ac0710 0x558334ac2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.999+0000 7fea2add5740 4 client.26386 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:03:26.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:25.999+0000 7fea2add5740 1 -- 172.21.15.94:0/1659225319 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55833473a890 con 0x558334ac0710 2024-02-17T12:03:26.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.000+0000 7fea25f9e640 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558334ac0710 0x558334ac2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.000+0000 7fea25f9e640 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558334ac0710 0x558334ac2b00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f47e0cab2ceeb5c server_cookie=4865e91193baa46b in_seq=0 out_seq=0 2024-02-17T12:03:26.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.000+0000 7fea097fa640 10 client.26386 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:03:26.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.000+0000 7fea097fa640 1 -- 172.21.15.94:0/1659225319 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55833473a890 con 0x558334ac0710 2024-02-17T12:03:26.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.000+0000 7fea097fa640 10 client.26386 handle_command_reply: tid=0 2024-02-17T12:03:26.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.000+0000 7fea2add5740 1 client.26386 shutdown 2024-02-17T12:03:26.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.001+0000 7fea2add5740 1 -- 172.21.15.94:0/1659225319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47250 msgr2=0x558334ab8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.001+0000 7fea2add5740 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47250 0x558334ab8c50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea10011af0 tx=0x7fea10011bb0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.001+0000 7fea2add5740 1 -- 172.21.15.94:0/1659225319 shutdown_connections 2024-02-17T12:03:26.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.001+0000 7fea2add5740 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558334ac0710 0x558334ac2b00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.001+0000 7fea2add5740 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558334a50750 0x558334ab3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.001+0000 7fea2add5740 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558334a47b70 0x558334ab9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.001+0000 7fea2add5740 1 --2- 172.21.15.94:0/1659225319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558334a47250 0x558334ab8c50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.001+0000 7fea2add5740 1 -- 172.21.15.94:0/1659225319 >> 172.21.15.94:0/1659225319 conn(0x558334a41980 msgr2=0x558334a402f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.002+0000 7fea097fa640 0 client.26386 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:03:26.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.002+0000 7fea2add5740 1 -- 172.21.15.94:0/1659225319 shutdown_connections 2024-02-17T12:03:26.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.002+0000 7fea2add5740 1 -- 172.21.15.94:0/1659225319 wait complete. 2024-02-17T12:03:26.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.002+0000 7fea2add5740 20 client.26386 trim_cache size 0 max 16384 2024-02-17T12:03:26.007 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:03:26.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.002+0000 7fea28202640 1 -- 172.21.15.94:0/2847985273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fea0c0a2230 msgr2=0x7fea0c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.002+0000 7fea28202640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fea0c0a2230 0x7fea0c0a46f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fea180a31d0 tx=0x7fea18009a70 comp rx=0 tx=0).stop 2024-02-17T12:03:26.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.002+0000 7fea28202640 1 -- 172.21.15.94:0/2847985273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea201071d0 msgr2=0x7fea20199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.002+0000 7fea28202640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea201071d0 0x7fea20199840 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fea1800eb10 tx=0x7fea1800c2d0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.003+0000 7fea28202640 1 -- 172.21.15.94:0/2847985273 shutdown_connections 2024-02-17T12:03:26.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.003+0000 7fea28202640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fea0c0a2230 0x7fea0c0a46f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.003+0000 7fea28202640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea201071d0 0x7fea20199840 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.003+0000 7fea28202640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea2006a070 0x7fea2019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.003+0000 7fea28202640 1 --2- 172.21.15.94:0/2847985273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea200696f0 0x7fea2019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.003+0000 7fea28202640 1 -- 172.21.15.94:0/2847985273 >> 172.21.15.94:0/2847985273 conn(0x7fea20077060 msgr2=0x7fea2010c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.003+0000 7fea28202640 1 -- 172.21.15.94:0/2847985273 shutdown_connections 2024-02-17T12:03:26.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.003+0000 7fea28202640 1 -- 172.21.15.94:0/2847985273 wait complete. 2024-02-17T12:03:26.017 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:03:26.017 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:03:26.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.170+0000 7f3ff73b6640 1 -- 172.21.15.94:0/581336650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ff0105330 msgr2=0x7f3ff010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.170+0000 7f3ff73b6640 1 --2- 172.21.15.94:0/581336650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ff0105330 0x7f3ff010cf00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009cd0 tx=0x7f3fe802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.170+0000 7f3ff73b6640 1 -- 172.21.15.94:0/581336650 shutdown_connections 2024-02-17T12:03:26.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.170+0000 7f3ff73b6640 1 --2- 172.21.15.94:0/581336650 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ff0105330 0x7f3ff010cf00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.170+0000 7f3ff73b6640 1 --2- 172.21.15.94:0/581336650 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff0100c10 0x7f3ff0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.170+0000 7f3ff73b6640 1 --2- 172.21.15.94:0/581336650 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff0100260 0x7f3ff0100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.170+0000 7f3ff73b6640 1 -- 172.21.15.94:0/581336650 >> 172.21.15.94:0/581336650 conn(0x7f3ff0077180 msgr2=0x7f3ff0077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.170+0000 7f3ff73b6640 1 -- 172.21.15.94:0/581336650 shutdown_connections 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.170+0000 7f3ff73b6640 1 -- 172.21.15.94:0/581336650 wait complete. 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.170+0000 7f3ff73b6640 1 Processor -- start 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.170+0000 7f3ff73b6640 1 -- start start 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.170+0000 7f3ff73b6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff0100260 0x7f3ff0198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.170+0000 7f3ff73b6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ff0105330 0x7f3ff0198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff73b6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff0199080 0x7f3ff01a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff73b6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff01a3cd0 con 0x7f3ff0105330 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff73b6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff01a3e40 con 0x7f3ff0199080 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff73b6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff01a4000 con 0x7f3ff0100260 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff4951640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ff0105330 0x7f3ff0198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff5953640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff0199080 0x7f3ff01a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff4951640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ff0105330 0x7f3ff0198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35314/0 (socket says 172.21.15.94:35314) 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff4951640 1 -- 172.21.15.94:0/2037971981 learned_addr learned my addr 172.21.15.94:0/2037971981 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:26.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff5152640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff0100260 0x7f3ff0198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff5953640 1 -- 172.21.15.94:0/2037971981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff0100260 msgr2=0x7f3ff0198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff5953640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff0100260 0x7f3ff0198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff5953640 1 -- 172.21.15.94:0/2037971981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ff0105330 msgr2=0x7f3ff0198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff5953640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ff0105330 0x7f3ff0198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff5953640 1 -- 172.21.15.94:0/2037971981 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe8009900 con 0x7f3ff0199080 2024-02-17T12:03:26.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.171+0000 7f3ff5152640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff0100260 0x7f3ff0198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:03:26.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.172+0000 7f3ff5953640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff0199080 0x7f3ff01a3680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8037040 tx=0x7f3fe8034a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:26.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.172+0000 7f3fe67fc640 1 -- 172.21.15.94:0/2037971981 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80328b0 con 0x7f3ff0199080 2024-02-17T12:03:26.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.172+0000 7f3fe67fc640 1 -- 172.21.15.94:0/2037971981 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fe8003e50 con 0x7f3ff0199080 2024-02-17T12:03:26.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.172+0000 7f3fe67fc640 1 -- 172.21.15.94:0/2037971981 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8004780 con 0x7f3ff0199080 2024-02-17T12:03:26.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.172+0000 7f3ff73b6640 1 -- 172.21.15.94:0/2037971981 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff01a4280 con 0x7f3ff0199080 2024-02-17T12:03:26.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.172+0000 7f3ff73b6640 1 -- 172.21.15.94:0/2037971981 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff01a45f0 con 0x7f3ff0199080 2024-02-17T12:03:26.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.174+0000 7f3fe67fc640 1 -- 172.21.15.94:0/2037971981 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3fe800a040 con 0x7f3ff0199080 2024-02-17T12:03:26.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.175+0000 7f3fe67fc640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3fdc0a22b0 0x7f3fdc0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.175+0000 7f3fe67fc640 1 -- 172.21.15.94:0/2037971981 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3fe80f0960 con 0x7f3ff0199080 2024-02-17T12:03:26.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.175+0000 7f3ff5152640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3fdc0a22b0 0x7f3fdc0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.175+0000 7f3ff9f85740 1 Processor -- start 2024-02-17T12:03:26.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.175+0000 7f3ff9f85740 1 -- start start 2024-02-17T12:03:26.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.175+0000 7f3ff5152640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3fdc0a22b0 0x7f3fdc0a4770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009c20 tx=0x7f3fec005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:26.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.175+0000 7f3ff9f85740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560759654250 0x560759654630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff9f85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654b70 0x56075965d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff9f85740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56075965d750 0x56075965fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff9f85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075960f960 con 0x560759654b70 2024-02-17T12:03:26.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff9f85740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607595c3310 con 0x560759654250 2024-02-17T12:03:26.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff9f85740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607595c0840 con 0x56075965d750 2024-02-17T12:03:26.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff4951640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654b70 0x56075965d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff4951640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654b70 0x56075965d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35320/0 (socket says 172.21.15.94:35320) 2024-02-17T12:03:26.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff4951640 1 -- 172.21.15.94:0/924940783 learned_addr learned my addr 172.21.15.94:0/924940783 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:26.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff5953640 1 --2- 172.21.15.94:0/924940783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56075965d750 0x56075965fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff5152640 1 --2- 172.21.15.94:0/924940783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560759654250 0x560759654630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff4951640 1 -- 172.21.15.94:0/924940783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56075965d750 msgr2=0x56075965fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff4951640 1 --2- 172.21.15.94:0/924940783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56075965d750 0x56075965fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff4951640 1 -- 172.21.15.94:0/924940783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560759654250 msgr2=0x560759654630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff4951640 1 --2- 172.21.15.94:0/924940783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560759654250 0x560759654630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.176+0000 7f3ff4951640 1 -- 172.21.15.94:0/924940783 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56075957e060 con 0x560759654b70 2024-02-17T12:03:26.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.177+0000 7f3ff4951640 1 --2- 172.21.15.94:0/924940783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654b70 0x56075965d210 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0008ba0 tx=0x7f3fe000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3a3b5bc1385adf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:26.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.177+0000 7f3fdb7fe640 1 -- 172.21.15.94:0/924940783 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0008ea0 con 0x560759654b70 2024-02-17T12:03:26.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.177+0000 7f3fdb7fe640 1 -- 172.21.15.94:0/924940783 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fe00047a0 con 0x560759654b70 2024-02-17T12:03:26.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.177+0000 7f3fdb7fe640 1 -- 172.21.15.94:0/924940783 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0021840 con 0x560759654b70 2024-02-17T12:03:26.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.178+0000 7f3ff9f85740 1 -- 172.21.15.94:0/924940783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654b70 msgr2=0x56075965d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.178+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/924940783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654b70 0x56075965d210 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0008ba0 tx=0x7f3fe000e720 comp rx=0 tx=0).stop 2024-02-17T12:03:26.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.178+0000 7f3ff9f85740 1 -- 172.21.15.94:0/924940783 shutdown_connections 2024-02-17T12:03:26.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.178+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/924940783 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56075965d750 0x56075965fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.178+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/924940783 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654b70 0x56075965d210 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.178+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/924940783 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560759654250 0x560759654630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.178+0000 7f3ff9f85740 1 -- 172.21.15.94:0/924940783 >> 172.21.15.94:0/924940783 conn(0x56075964e980 msgr2=0x56075964ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.178+0000 7f3ff9f85740 1 -- 172.21.15.94:0/924940783 shutdown_connections 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.178+0000 7f3ff9f85740 1 -- 172.21.15.94:0/924940783 wait complete. 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.179+0000 7f3ff9f85740 1 -- start start 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.179+0000 7f3ff9f85740 1 Processor -- start 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.179+0000 7f3ff9f85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 0x56075965a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.179+0000 7f3ff9f85740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560759654b70 0x56075965acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.179+0000 7f3ff9f85740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56075965d750 0x56075965b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.179+0000 7f3ff5152640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 0x56075965a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.179+0000 7f3ff9f85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607595bc020 con 0x560759654250 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.179+0000 7f3ff9f85740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075928f430 con 0x56075965d750 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.179+0000 7f3ff9f85740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075924be00 con 0x560759654b70 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff5152640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 0x56075965a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35322/0 (socket says 172.21.15.94:35322) 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff5152640 1 -- 172.21.15.94:0/1180242878 learned_addr learned my addr 172.21.15.94:0/1180242878 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:26.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff9f85740 10 client.0 _ll_register_callbacks cb 0x560759563cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff4951640 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560759654b70 0x56075965acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff5953640 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56075965d750 0x56075965b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff5152640 1 -- 172.21.15.94:0/1180242878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560759654b70 msgr2=0x56075965acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff5152640 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560759654b70 0x56075965acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff5152640 1 -- 172.21.15.94:0/1180242878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56075965d750 msgr2=0x56075965b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff5152640 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56075965d750 0x56075965b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff5152640 1 -- 172.21.15.94:0/1180242878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe0008800 con 0x560759654250 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff4951640 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560759654b70 0x56075965acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff5953640 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56075965d750 0x56075965b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff9f85740 1 -- 172.21.15.94:0/1180242878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607592bb7e0 con 0x560759654250 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff5152640 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 0x56075965a7b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3fec012c40 tx=0x7f3fec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af8205da44b8d9e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3fd8ff9640 10 client.16890 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3fd8ff9640 1 -- 172.21.15.94:0/1180242878 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec029640 con 0x560759654250 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3fd8ff9640 1 -- 172.21.15.94:0/1180242878 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3fec020e40 con 0x560759654250 2024-02-17T12:03:26.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3fd8ff9640 1 -- 172.21.15.94:0/1180242878 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fec012e80 con 0x560759654250 2024-02-17T12:03:26.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff9f85740 10 client.16890 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:26.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3ff9f85740 1 -- 172.21.15.94:0/1180242878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x5607595ba6a0 con 0x560759654250 2024-02-17T12:03:26.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.180+0000 7f3fd8ff9640 1 -- 172.21.15.94:0/1180242878 <== mon.0 v2:172.21.15.94:3300/0 4 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3fec0297a0 con 0x560759654250 2024-02-17T12:03:26.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.181+0000 7f3ff9f85740 10 client.16890 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:26.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.181+0000 7f3ff9f85740 10 client.16890 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:03:26.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.181+0000 7f3ff9f85740 20 client.16890 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:26.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.181+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56075966daa0 0x56075966fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.181+0000 7f3ff9f85740 4 client.16890 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:26.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.181+0000 7f3ff9f85740 1 -- 172.21.15.94:0/1180242878 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607592f8720 con 0x56075966daa0 2024-02-17T12:03:26.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.181+0000 7f3ff4951640 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56075966daa0 0x56075966fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.181+0000 7f3ff4951640 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56075966daa0 0x56075966fec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=460a87a9a27841f0 server_cookie=5454109ee4cf880f in_seq=0 out_seq=0 2024-02-17T12:03:26.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.182+0000 7f3fd8ff9640 10 client.16890 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:03:26.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.182+0000 7f3fd8ff9640 1 -- 172.21.15.94:0/1180242878 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5607592f8720 con 0x56075966daa0 2024-02-17T12:03:26.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.182+0000 7f3fd8ff9640 10 client.16890 handle_command_reply: tid=0 2024-02-17T12:03:26.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.182+0000 7f3ff9f85740 1 client.16890 shutdown 2024-02-17T12:03:26.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.183+0000 7f3ff9f85740 1 -- 172.21.15.94:0/1180242878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 msgr2=0x56075965a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.183+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 0x56075965a7b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3fec012c40 tx=0x7f3fec004730 comp rx=0 tx=0).stop 2024-02-17T12:03:26.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.183+0000 7f3ff9f85740 1 -- 172.21.15.94:0/1180242878 shutdown_connections 2024-02-17T12:03:26.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.183+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56075966daa0 0x56075966fec0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.183+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56075965d750 0x56075965b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.183+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560759654b70 0x56075965acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.183+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/1180242878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 0x56075965a7b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.183+0000 7f3fd8ff9640 0 client.16890 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:03:26.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.183+0000 7f3ff9f85740 1 -- 172.21.15.94:0/1180242878 >> 172.21.15.94:0/1180242878 conn(0x56075964e980 msgr2=0x5607596506d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.183+0000 7f3ff9f85740 1 -- 172.21.15.94:0/1180242878 shutdown_connections 2024-02-17T12:03:26.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.183+0000 7f3ff9f85740 1 -- 172.21.15.94:0/1180242878 wait complete. 2024-02-17T12:03:26.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.183+0000 7f3ff9f85740 20 client.16890 trim_cache size 0 max 16384 2024-02-17T12:03:26.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:25 smithi125 ceph-mon[36819]: pgmap v163: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 8.1 MiB/s wr, 763 op/s 2024-02-17T12:03:26.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:25 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/125071721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:03:26.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff9f85740 1 Processor -- start 2024-02-17T12:03:26.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff9f85740 1 -- start start 2024-02-17T12:03:26.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff9f85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 0x56075965cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff9f85740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560759654b70 0x56075964da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff9f85740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56075965d750 0x560759664600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff9f85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560759541130 con 0x560759654250 2024-02-17T12:03:26.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff9f85740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560759653eb0 con 0x560759654b70 2024-02-17T12:03:26.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff9f85740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56075964f980 con 0x56075965d750 2024-02-17T12:03:26.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff5152640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 0x56075965cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff5152640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 0x56075965cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35328/0 (socket says 172.21.15.94:35328) 2024-02-17T12:03:26.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff4951640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560759654b70 0x56075964da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff5152640 1 -- 172.21.15.94:0/2741722672 learned_addr learned my addr 172.21.15.94:0/2741722672 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:26.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff5953640 1 --2- 172.21.15.94:0/2741722672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56075965d750 0x560759664600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff5152640 1 -- 172.21.15.94:0/2741722672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56075965d750 msgr2=0x560759664600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff5152640 1 --2- 172.21.15.94:0/2741722672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56075965d750 0x560759664600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.200 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff5152640 1 -- 172.21.15.94:0/2741722672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560759654b70 msgr2=0x56075964da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff5152640 1 --2- 172.21.15.94:0/2741722672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560759654b70 0x56075964da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff5152640 1 -- 172.21.15.94:0/2741722672 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560759556c20 con 0x560759654250 2024-02-17T12:03:26.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.198+0000 7f3ff4951640 1 --2- 172.21.15.94:0/2741722672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560759654b70 0x56075964da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:26.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.199+0000 7f3ff5152640 1 --2- 172.21.15.94:0/2741722672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 0x56075965cee0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3fec011ce0 tx=0x7f3fec022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466fe9c0a0f861fe server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:26.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.199+0000 7f3fdaffd640 1 -- 172.21.15.94:0/2741722672 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec029640 con 0x560759654250 2024-02-17T12:03:26.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.199+0000 7f3fdaffd640 1 -- 172.21.15.94:0/2741722672 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5607592bb7e0 con 0x560759654250 2024-02-17T12:03:26.201 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.199+0000 7f3fdaffd640 1 -- 172.21.15.94:0/2741722672 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec004b00 con 0x560759654250 2024-02-17T12:03:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 -- 172.21.15.94:0/2741722672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 msgr2=0x56075965cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/2741722672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 0x56075965cee0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f3fec011ce0 tx=0x7f3fec022040 comp rx=0 tx=0).stop 2024-02-17T12:03:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 -- 172.21.15.94:0/2741722672 shutdown_connections 2024-02-17T12:03:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/2741722672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56075965d750 0x560759664600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/2741722672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560759654b70 0x56075964da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/2741722672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654250 0x56075965cee0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 -- 172.21.15.94:0/2741722672 >> 172.21.15.94:0/2741722672 conn(0x56075964e980 msgr2=0x56075964d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 -- 172.21.15.94:0/2741722672 shutdown_connections 2024-02-17T12:03:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 -- 172.21.15.94:0/2741722672 wait complete. 2024-02-17T12:03:26.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 -- start start 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 Processor -- start 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560759654250 0x5607596b9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654b70 0x5607596ba4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56075965d750 0x5607596c5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607596bac20 con 0x560759654b70 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.200+0000 7f3ff9f85740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607596bad90 con 0x56075965d750 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.201+0000 7f3ff9f85740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607596cdd50 con 0x560759654250 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.201+0000 7f3ff9f85740 10 client.0 _ll_register_callbacks cb 0x7f3fe0008390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.201+0000 7f3ff5953640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56075965d750 0x5607596c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.201+0000 7f3ff5953640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56075965d750 0x5607596c5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59606/0 (socket says 172.21.15.94:59606) 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.201+0000 7f3ff5953640 1 -- 172.21.15.94:0/2984860942 learned_addr learned my addr 172.21.15.94:0/2984860942 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.201+0000 7f3ff4951640 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654b70 0x5607596ba4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.201+0000 7f3ff5152640 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560759654250 0x5607596b9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.201+0000 7f3ff5953640 1 -- 172.21.15.94:0/2984860942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560759654250 msgr2=0x5607596b9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.201+0000 7f3ff5953640 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560759654250 0x5607596b9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.201+0000 7f3ff5953640 1 -- 172.21.15.94:0/2984860942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654b70 msgr2=0x5607596ba4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.201+0000 7f3ff5953640 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654b70 0x5607596ba4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.201+0000 7f3ff5953640 1 -- 172.21.15.94:0/2984860942 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56075957e060 con 0x56075965d750 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.201+0000 7f3ff4951640 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654b70 0x5607596ba4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.202+0000 7f3ff5953640 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56075965d750 0x5607596c5000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009620 tx=0x7f3fe8005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4982b7d52413556 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.202+0000 7f3fd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.202+0000 7f3fd8ff9640 1 -- 172.21.15.94:0/2984860942 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe80c5bf0 con 0x56075965d750 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.202+0000 7f3ff9f85740 1 -- 172.21.15.94:0/2984860942 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fe00047a0 con 0x56075965d750 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.202+0000 7f3fd8ff9640 1 -- 172.21.15.94:0/2984860942 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3fe80ee320 con 0x56075965d750 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.202+0000 7f3fd8ff9640 1 -- 172.21.15.94:0/2984860942 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fe80ee4e0 con 0x56075965d750 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.202+0000 7f3ff9f85740 10 client.26354 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.202+0000 7f3ff9f85740 1 -- 172.21.15.94:0/2984860942 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5607595ba6a0 con 0x56075965d750 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.202+0000 7f3fd8ff9640 1 -- 172.21.15.94:0/2984860942 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3fe80462c0 con 0x56075965d750 2024-02-17T12:03:26.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.202+0000 7f3fd8ff9640 1 -- 172.21.15.94:0/2984860942 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3fe80bd7d0 con 0x56075965d750 2024-02-17T12:03:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.202+0000 7f3ff9f85740 10 client.26354 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.202+0000 7f3ff9f85740 10 client.26354 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:03:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.202+0000 7f3ff9f85740 20 client.26354 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.203+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5607596cf1c0 0x5607596d15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.203+0000 7f3ff9f85740 4 client.26354 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:03:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.203+0000 7f3ff9f85740 1 -- 172.21.15.94:0/2984860942 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f3fe00047a0 con 0x5607596cf1c0 2024-02-17T12:03:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.203+0000 7f3ff5152640 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5607596cf1c0 0x5607596d15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.203+0000 7f3ff5152640 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5607596cf1c0 0x5607596d15b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=647981b73881d334 server_cookie=f93d52fa6d33fdb4 in_seq=0 out_seq=0 2024-02-17T12:03:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.203+0000 7f3fd8ff9640 10 client.26354 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:03:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.204+0000 7f3fd8ff9640 1 -- 172.21.15.94:0/2984860942 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f3fe00047a0 con 0x5607596cf1c0 2024-02-17T12:03:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.204+0000 7f3fd8ff9640 10 client.26354 handle_command_reply: tid=0 2024-02-17T12:03:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.204+0000 7f3ff9f85740 1 client.26354 shutdown 2024-02-17T12:03:26.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.204+0000 7f3ff9f85740 1 -- 172.21.15.94:0/2984860942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56075965d750 msgr2=0x5607596c5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.204+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56075965d750 0x5607596c5000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8009620 tx=0x7f3fe8005df0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.204+0000 7f3ff9f85740 1 -- 172.21.15.94:0/2984860942 shutdown_connections 2024-02-17T12:03:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.204+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5607596cf1c0 0x5607596d15b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.204+0000 7f3fd8ff9640 0 client.26354 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:03:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.205+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56075965d750 0x5607596c5000 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.205+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560759654b70 0x5607596ba4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.205+0000 7f3ff9f85740 1 --2- 172.21.15.94:0/2984860942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560759654250 0x5607596b9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.205+0000 7f3ff9f85740 1 -- 172.21.15.94:0/2984860942 >> 172.21.15.94:0/2984860942 conn(0x56075964e980 msgr2=0x560759650640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.205+0000 7f3ff9f85740 1 -- 172.21.15.94:0/2984860942 shutdown_connections 2024-02-17T12:03:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.205+0000 7f3ff9f85740 1 -- 172.21.15.94:0/2984860942 wait complete. 2024-02-17T12:03:26.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.205+0000 7f3ff9f85740 20 client.26354 trim_cache size 0 max 16384 2024-02-17T12:03:26.210 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:03:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.205+0000 7f3ff73b6640 1 -- 172.21.15.94:0/2037971981 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3fdc0a22b0 msgr2=0x7f3fdc0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.205+0000 7f3ff73b6640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3fdc0a22b0 0x7f3fdc0a4770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009c20 tx=0x7f3fec005f90 comp rx=0 tx=0).stop 2024-02-17T12:03:26.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.205+0000 7f3ff73b6640 1 -- 172.21.15.94:0/2037971981 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff0199080 msgr2=0x7f3ff01a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.205+0000 7f3ff73b6640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff0199080 0x7f3ff01a3680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3fe8037040 tx=0x7f3fe8034a90 comp rx=0 tx=0).stop 2024-02-17T12:03:26.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.206+0000 7f3ff73b6640 1 -- 172.21.15.94:0/2037971981 shutdown_connections 2024-02-17T12:03:26.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.206+0000 7f3ff73b6640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3fdc0a22b0 0x7f3fdc0a4770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.206+0000 7f3ff73b6640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3ff0199080 0x7f3ff01a3680 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.206+0000 7f3ff73b6640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3ff0105330 0x7f3ff0198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.206+0000 7f3ff73b6640 1 --2- 172.21.15.94:0/2037971981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ff0100260 0x7f3ff0198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.206+0000 7f3ff73b6640 1 -- 172.21.15.94:0/2037971981 >> 172.21.15.94:0/2037971981 conn(0x7f3ff0077180 msgr2=0x7f3ff00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.206+0000 7f3ff73b6640 1 -- 172.21.15.94:0/2037971981 shutdown_connections 2024-02-17T12:03:26.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:25 smithi094 ceph-mon[32499]: pgmap v163: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 8.1 MiB/s wr, 763 op/s 2024-02-17T12:03:26.212 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:25 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/125071721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:03:26.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.206+0000 7f3ff73b6640 1 -- 172.21.15.94:0/2037971981 wait complete. 2024-02-17T12:03:26.222 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:03:26.222 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:03:26.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.374+0000 7f87602f1640 1 -- 172.21.15.94:0/137156851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f87580685f0 msgr2=0x7f87580689d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.374+0000 7f87602f1640 1 --2- 172.21.15.94:0/137156851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f87580685f0 0x7f87580689d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8754009930 tx=0x7f875402bc40 comp rx=0 tx=0).stop 2024-02-17T12:03:26.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.374+0000 7f87602f1640 1 -- 172.21.15.94:0/137156851 shutdown_connections 2024-02-17T12:03:26.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.374+0000 7f87602f1640 1 --2- 172.21.15.94:0/137156851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87581064c0 0x7f875810e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.374+0000 7f87602f1640 1 --2- 172.21.15.94:0/137156851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8758068fa0 0x7f8758105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.374+0000 7f87602f1640 1 --2- 172.21.15.94:0/137156851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f87580685f0 0x7f87580689d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.374+0000 7f87602f1640 1 -- 172.21.15.94:0/137156851 >> 172.21.15.94:0/137156851 conn(0x7f8758077180 msgr2=0x7f8758077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.375+0000 7f87602f1640 1 -- 172.21.15.94:0/137156851 shutdown_connections 2024-02-17T12:03:26.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.376+0000 7f87602f1640 1 -- 172.21.15.94:0/137156851 wait complete. 2024-02-17T12:03:26.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.377+0000 7f87602f1640 1 Processor -- start 2024-02-17T12:03:26.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.377+0000 7f87602f1640 1 -- start start 2024-02-17T12:03:26.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.377+0000 7f87602f1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f87580685f0 0x7f8758100040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.377+0000 7f87602f1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87581064c0 0x7f8758100580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.377+0000 7f87602f1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8758100ac0 0x7f87581a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.377+0000 7f87602f1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87581a60a0 con 0x7f8758100ac0 2024-02-17T12:03:26.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.377+0000 7f87602f1640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87581a6210 con 0x7f87580685f0 2024-02-17T12:03:26.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.377+0000 7f87602f1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87581a63d0 con 0x7f87581064c0 2024-02-17T12:03:26.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.377+0000 7f875e08d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f87580685f0 0x7f8758100040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.377+0000 7f875d88c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87581064c0 0x7f8758100580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.377+0000 7f875e08d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f87580685f0 0x7f8758100040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59636/0 (socket says 172.21.15.94:59636) 2024-02-17T12:03:26.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.377+0000 7f875e08d640 1 -- 172.21.15.94:0/171585072 learned_addr learned my addr 172.21.15.94:0/171585072 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:26.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.377+0000 7f875e88e640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8758100ac0 0x7f87581a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.378+0000 7f875e08d640 1 -- 172.21.15.94:0/171585072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87581064c0 msgr2=0x7f8758100580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.378+0000 7f875e08d640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87581064c0 0x7f8758100580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.378+0000 7f875e08d640 1 -- 172.21.15.94:0/171585072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8758100ac0 msgr2=0x7f87581a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.378+0000 7f875e08d640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8758100ac0 0x7f87581a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.378+0000 7f875e08d640 1 -- 172.21.15.94:0/171585072 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8754009560 con 0x7f87580685f0 2024-02-17T12:03:26.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.378+0000 7f875e88e640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8758100ac0 0x7f87581a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:26.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.378+0000 7f875e08d640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f87580685f0 0x7f8758100040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8754005940 tx=0x7f87540042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:26.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.379+0000 7f874f7fe640 1 -- 172.21.15.94:0/171585072 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87540044e0 con 0x7f87580685f0 2024-02-17T12:03:26.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.379+0000 7f874f7fe640 1 -- 172.21.15.94:0/171585072 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f875402cbd0 con 0x7f87580685f0 2024-02-17T12:03:26.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.379+0000 7f87602f1640 1 -- 172.21.15.94:0/171585072 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87581a6650 con 0x7f87580685f0 2024-02-17T12:03:26.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.379+0000 7f874f7fe640 1 -- 172.21.15.94:0/171585072 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754003e20 con 0x7f87580685f0 2024-02-17T12:03:26.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.380+0000 7f87602f1640 1 -- 172.21.15.94:0/171585072 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87581a6b60 con 0x7f87580685f0 2024-02-17T12:03:26.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.382+0000 7f874f7fe640 1 -- 172.21.15.94:0/171585072 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f87540035f0 con 0x7f87580685f0 2024-02-17T12:03:26.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.383+0000 7f8762f9f740 1 Processor -- start 2024-02-17T12:03:26.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.383+0000 7f8762f9f740 1 -- start start 2024-02-17T12:03:26.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.383+0000 7f874f7fe640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f87440a23b0 0x7f87440a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.383+0000 7f8762f9f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574840250 0x562574840630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.383+0000 7f8762f9f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x562574849210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.383+0000 7f8762f9f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574849750 0x56257484bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.383+0000 7f8762f9f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625747fb960 con 0x562574840b70 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.383+0000 7f8762f9f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625747af310 con 0x562574849750 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.383+0000 7f8762f9f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625747ac840 con 0x562574840250 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f875e88e640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f87440a23b0 0x7f87440a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f875e08d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574840250 0x562574840630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f875e88e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574849750 0x56257484bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f875e08d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574840250 0x562574840630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57574/0 (socket says 172.21.15.94:57574) 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f875e08d640 1 -- 172.21.15.94:0/2819794759 learned_addr learned my addr 172.21.15.94:0/2819794759 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f874f7fe640 1 -- 172.21.15.94:0/171585072 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f87540f0310 con 0x7f87580685f0 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f875e08d640 1 -- 172.21.15.94:0/2819794759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574849750 msgr2=0x56257484bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f875e08d640 1 --2- 172.21.15.94:0/2819794759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574849750 0x56257484bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f875e08d640 1 -- 172.21.15.94:0/2819794759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 msgr2=0x562574849210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f875d88c640 1 --2- 172.21.15.94:0/2819794759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x562574849210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f875e08d640 1 --2- 172.21.15.94:0/2819794759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x562574849210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f875e08d640 1 -- 172.21.15.94:0/2819794759 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56257476a060 con 0x562574840250 2024-02-17T12:03:26.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f875e88e640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f87440a23b0 0x7f87440a4870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8750009fd0 tx=0x7f8750009ec0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:26.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.384+0000 7f875d88c640 1 --2- 172.21.15.94:0/2819794759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x562574849210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:03:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.385+0000 7f875e08d640 1 --2- 172.21.15.94:0/2819794759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574840250 0x562574840630 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f87540edb60 tx=0x7f8754003920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=713dee7f6cf3ebc2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.385+0000 7f874cbfe640 1 -- 172.21.15.94:0/2819794759 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87540c0070 con 0x562574840250 2024-02-17T12:03:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.385+0000 7f874cbfe640 1 -- 172.21.15.94:0/2819794759 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87540ef070 con 0x562574840250 2024-02-17T12:03:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.385+0000 7f874cbfe640 1 -- 172.21.15.94:0/2819794759 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87540c0ef0 con 0x562574840250 2024-02-17T12:03:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.386+0000 7f8762f9f740 1 -- 172.21.15.94:0/2819794759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574840250 msgr2=0x562574840630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.386+0000 7f8762f9f740 1 --2- 172.21.15.94:0/2819794759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574840250 0x562574840630 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f87540edb60 tx=0x7f8754003920 comp rx=0 tx=0).stop 2024-02-17T12:03:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.386+0000 7f8762f9f740 1 -- 172.21.15.94:0/2819794759 shutdown_connections 2024-02-17T12:03:26.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.386+0000 7f8762f9f740 1 --2- 172.21.15.94:0/2819794759 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574849750 0x56257484bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.386+0000 7f8762f9f740 1 --2- 172.21.15.94:0/2819794759 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x562574849210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.386+0000 7f8762f9f740 1 --2- 172.21.15.94:0/2819794759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574840250 0x562574840630 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.386+0000 7f8762f9f740 1 -- 172.21.15.94:0/2819794759 >> 172.21.15.94:0/2819794759 conn(0x56257483a980 msgr2=0x56257483ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.387+0000 7f8762f9f740 1 -- 172.21.15.94:0/2819794759 shutdown_connections 2024-02-17T12:03:26.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.387+0000 7f8762f9f740 1 -- 172.21.15.94:0/2819794759 wait complete. 2024-02-17T12:03:26.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.387+0000 7f8762f9f740 1 -- start start 2024-02-17T12:03:26.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.387+0000 7f8762f9f740 1 Processor -- start 2024-02-17T12:03:26.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.387+0000 7f8762f9f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574840250 0x562574846700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.387+0000 7f8762f9f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x562574846c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.387+0000 7f8762f9f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574849750 0x562574847180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.387+0000 7f8762f9f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625747a8020 con 0x562574840b70 2024-02-17T12:03:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.387+0000 7f8762f9f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257447b430 con 0x562574840250 2024-02-17T12:03:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f8762f9f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562574437e00 con 0x562574849750 2024-02-17T12:03:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f875e08d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574840250 0x562574846700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f875e08d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574840250 0x562574846700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59656/0 (socket says 172.21.15.94:59656) 2024-02-17T12:03:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f875d88c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x562574846c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f875e08d640 1 -- 172.21.15.94:0/2333031660 learned_addr learned my addr 172.21.15.94:0/2333031660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f875e88e640 1 --2- 172.21.15.94:0/2333031660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574849750 0x562574847180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f8762f9f740 10 client.0 _ll_register_callbacks cb 0x56257474fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f875e08d640 1 -- 172.21.15.94:0/2333031660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574849750 msgr2=0x562574847180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f875e08d640 1 --2- 172.21.15.94:0/2333031660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574849750 0x562574847180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f875e08d640 1 -- 172.21.15.94:0/2333031660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 msgr2=0x562574846c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f875e08d640 1 --2- 172.21.15.94:0/2333031660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x562574846c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f875e08d640 1 -- 172.21.15.94:0/2333031660 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87581a6b60 con 0x562574840250 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f875e88e640 1 --2- 172.21.15.94:0/2333031660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574849750 0x562574847180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f875e08d640 1 --2- 172.21.15.94:0/2333031660 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574840250 0x562574846700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f87540bcd30 tx=0x7f8754042c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd54a62b4b83be98 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f8741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f8741ffb640 1 -- 172.21.15.94:0/2333031660 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87540c0070 con 0x562574840250 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f8741ffb640 1 -- 172.21.15.94:0/2333031660 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f87540bd4a0 con 0x562574840250 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f8762f9f740 1 -- 172.21.15.94:0/2333031660 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625744a77e0 con 0x562574840250 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f8741ffb640 1 -- 172.21.15.94:0/2333031660 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87540c5540 con 0x562574840250 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f8762f9f740 10 client.26366 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:26.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.388+0000 7f8762f9f740 1 -- 172.21.15.94:0/2333031660 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5625747a66a0 con 0x562574840250 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.389+0000 7f8741ffb640 1 -- 172.21.15.94:0/2333031660 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5625744a77e0 con 0x562574840250 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.389+0000 7f8741ffb640 1 -- 172.21.15.94:0/2333031660 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f875404f020 con 0x562574840250 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.389+0000 7f8762f9f740 10 client.26366 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.389+0000 7f8762f9f740 10 client.26366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.389+0000 7f8762f9f740 20 client.26366 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.389+0000 7f8762f9f740 1 --2- 172.21.15.94:0/2333031660 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562574859290 0x56257485b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.389+0000 7f8762f9f740 4 client.26366 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.389+0000 7f8762f9f740 1 -- 172.21.15.94:0/2333031660 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625744e4720 con 0x562574859290 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.389+0000 7f875d88c640 1 --2- 172.21.15.94:0/2333031660 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562574859290 0x56257485b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.390+0000 7f875d88c640 1 --2- 172.21.15.94:0/2333031660 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562574859290 0x56257485b6b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85c52d0e110f84ce server_cookie=616b2c4780a50f57 in_seq=0 out_seq=0 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.390+0000 7f8741ffb640 10 client.26366 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8741ffb640 1 -- 172.21.15.94:0/2333031660 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5625744e4720 con 0x562574859290 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8741ffb640 10 client.26366 handle_command_reply: tid=0 2024-02-17T12:03:26.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8762f9f740 1 client.26366 shutdown 2024-02-17T12:03:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8762f9f740 1 -- 172.21.15.94:0/2333031660 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574840250 msgr2=0x562574846700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8762f9f740 1 --2- 172.21.15.94:0/2333031660 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574840250 0x562574846700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f87540bcd30 tx=0x7f8754042c20 comp rx=0 tx=0).stop 2024-02-17T12:03:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8762f9f740 1 -- 172.21.15.94:0/2333031660 shutdown_connections 2024-02-17T12:03:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8762f9f740 1 --2- 172.21.15.94:0/2333031660 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562574859290 0x56257485b6b0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8762f9f740 1 --2- 172.21.15.94:0/2333031660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574849750 0x562574847180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8762f9f740 1 --2- 172.21.15.94:0/2333031660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x562574846c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8741ffb640 0 client.26366 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8762f9f740 1 --2- 172.21.15.94:0/2333031660 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574840250 0x562574846700 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8762f9f740 1 -- 172.21.15.94:0/2333031660 >> 172.21.15.94:0/2333031660 conn(0x56257483a980 msgr2=0x56257483c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8762f9f740 1 -- 172.21.15.94:0/2333031660 shutdown_connections 2024-02-17T12:03:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8762f9f740 1 -- 172.21.15.94:0/2333031660 wait complete. 2024-02-17T12:03:26.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.391+0000 7f8762f9f740 20 client.26366 trim_cache size 0 max 16384 2024-02-17T12:03:26.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:25 smithi189 ceph-mon[36588]: pgmap v163: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 8.1 MiB/s wr, 763 op/s 2024-02-17T12:03:26.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:25 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/125071721' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:03:26.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.410+0000 7f8762f9f740 1 Processor -- start 2024-02-17T12:03:26.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f8762f9f740 1 -- start start 2024-02-17T12:03:26.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f8762f9f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840250 0x562574848e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f8762f9f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574840b70 0x562574841a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f8762f9f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574849750 0x562574843960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f8762f9f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257472d130 con 0x562574840250 2024-02-17T12:03:26.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f8762f9f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625748590e0 con 0x562574849750 2024-02-17T12:03:26.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f8762f9f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625744a77e0 con 0x562574840b70 2024-02-17T12:03:26.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f875e08d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840250 0x562574848e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f875e88e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574849750 0x562574843960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f875e88e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574849750 0x562574843960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59672/0 (socket says 172.21.15.94:59672) 2024-02-17T12:03:26.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f875d88c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574840b70 0x562574841a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f875e88e640 1 -- 172.21.15.94:0/1362154401 learned_addr learned my addr 172.21.15.94:0/1362154401 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:26.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f875e08d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840250 0x562574848e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35384/0 (socket says 172.21.15.94:35384) 2024-02-17T12:03:26.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f875d88c640 1 -- 172.21.15.94:0/1362154401 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574849750 msgr2=0x562574843960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f875d88c640 1 --2- 172.21.15.94:0/1362154401 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574849750 0x562574843960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f875d88c640 1 -- 172.21.15.94:0/1362154401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840250 msgr2=0x562574848e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f875d88c640 1 --2- 172.21.15.94:0/1362154401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840250 0x562574848e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.411+0000 7f875d88c640 1 -- 172.21.15.94:0/1362154401 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562574742c20 con 0x562574840b70 2024-02-17T12:03:26.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.412+0000 7f875d88c640 1 --2- 172.21.15.94:0/1362154401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574840b70 0x562574841a90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x562574845f60 tx=0x7f8748008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c01f68649b82f01c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:26.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.412+0000 7f8743fff640 1 -- 172.21.15.94:0/1362154401 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874801c070 con 0x562574840b70 2024-02-17T12:03:26.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.412+0000 7f8743fff640 1 -- 172.21.15.94:0/1362154401 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8748017dd0 con 0x562574840b70 2024-02-17T12:03:26.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.412+0000 7f8743fff640 1 -- 172.21.15.94:0/1362154401 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87480253c0 con 0x562574840b70 2024-02-17T12:03:26.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.413+0000 7f8762f9f740 1 -- 172.21.15.94:0/1362154401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574840b70 msgr2=0x562574841a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.413+0000 7f8762f9f740 1 --2- 172.21.15.94:0/1362154401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574840b70 0x562574841a90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x562574845f60 tx=0x7f8748008040 comp rx=0 tx=0).stop 2024-02-17T12:03:26.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.413+0000 7f8762f9f740 1 -- 172.21.15.94:0/1362154401 shutdown_connections 2024-02-17T12:03:26.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.413+0000 7f8762f9f740 1 --2- 172.21.15.94:0/1362154401 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574849750 0x562574843960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.413+0000 7f8762f9f740 1 --2- 172.21.15.94:0/1362154401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574840b70 0x562574841a90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.413+0000 7f8762f9f740 1 --2- 172.21.15.94:0/1362154401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840250 0x562574848e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.413+0000 7f8762f9f740 1 -- 172.21.15.94:0/1362154401 >> 172.21.15.94:0/1362154401 conn(0x56257483a980 msgr2=0x562574835a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.413+0000 7f8762f9f740 1 -- 172.21.15.94:0/1362154401 shutdown_connections 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.413+0000 7f8762f9f740 1 -- 172.21.15.94:0/1362154401 wait complete. 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f8762f9f740 1 -- start start 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f8762f9f740 1 Processor -- start 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f8762f9f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574840250 0x56257484f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f8762f9f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x56257484f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f8762f9f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574849750 0x5625748b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f875d88c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x56257484f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f875d88c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x56257484f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35400/0 (socket says 172.21.15.94:35400) 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f8762f9f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257484fe10 con 0x562574840b70 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f875d88c640 1 -- 172.21.15.94:0/1102422679 learned_addr learned my addr 172.21.15.94:0/1102422679 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f8762f9f740 1 -- 172.21.15.94:0/1102422679 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257484ff80 con 0x562574840250 2024-02-17T12:03:26.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f8762f9f740 1 -- 172.21.15.94:0/1102422679 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625748b84b0 con 0x562574849750 2024-02-17T12:03:26.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f875e08d640 1 --2- 172.21.15.94:0/1102422679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574840250 0x56257484f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f875e88e640 1 --2- 172.21.15.94:0/1102422679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574849750 0x5625748b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.414+0000 7f8762f9f740 10 client.0 _ll_register_callbacks cb 0x7f875403dfc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.415+0000 7f875d88c640 1 -- 172.21.15.94:0/1102422679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574849750 msgr2=0x5625748b7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.415+0000 7f875d88c640 1 --2- 172.21.15.94:0/1102422679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574849750 0x5625748b7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.415+0000 7f875d88c640 1 -- 172.21.15.94:0/1102422679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574840250 msgr2=0x56257484f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.415+0000 7f875d88c640 1 --2- 172.21.15.94:0/1102422679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574840250 0x56257484f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.415+0000 7f875d88c640 1 -- 172.21.15.94:0/1102422679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874800e040 con 0x562574840b70 2024-02-17T12:03:26.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.416+0000 7f875d88c640 1 --2- 172.21.15.94:0/1102422679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x56257484f650 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8748008010 tx=0x7f87480047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a1af82698ddf9b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.416+0000 7f8741ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:03:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.416+0000 7f8741ffb640 1 -- 172.21.15.94:0/1102422679 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874801c040 con 0x562574840b70 2024-02-17T12:03:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.416+0000 7f8741ffb640 1 -- 172.21.15.94:0/1102422679 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f874801c670 con 0x562574840b70 2024-02-17T12:03:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.417+0000 7f8762f9f740 1 -- 172.21.15.94:0/1102422679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562574846530 con 0x562574840b70 2024-02-17T12:03:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.417+0000 7f8741ffb640 1 -- 172.21.15.94:0/1102422679 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f874801c830 con 0x562574840b70 2024-02-17T12:03:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.417+0000 7f8762f9f740 10 client.16908 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.417+0000 7f8762f9f740 1 -- 172.21.15.94:0/1102422679 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5625747688c0 con 0x562574840b70 2024-02-17T12:03:26.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.418+0000 7f8741ffb640 1 -- 172.21.15.94:0/1102422679 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562574846530 con 0x562574840b70 2024-02-17T12:03:26.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.418+0000 7f8741ffb640 1 -- 172.21.15.94:0/1102422679 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8748026a70 con 0x562574840b70 2024-02-17T12:03:26.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.419+0000 7f8762f9f740 10 client.16908 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:26.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.419+0000 7f8762f9f740 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:26.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.419+0000 7f8762f9f740 20 client.16908 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.419+0000 7f8762f9f740 1 --2- 172.21.15.94:0/1102422679 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5625748b9b50 0x5625748bbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.419+0000 7f8762f9f740 4 client.16908 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:03:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.419+0000 7f8762f9f740 1 -- 172.21.15.94:0/1102422679 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f87540ef070 con 0x5625748b9b50 2024-02-17T12:03:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.419+0000 7f875e08d640 1 --2- 172.21.15.94:0/1102422679 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5625748b9b50 0x5625748bbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.419+0000 7f8741ffb640 10 client.16908 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.419+0000 7f875e08d640 1 --2- 172.21.15.94:0/1102422679 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5625748b9b50 0x5625748bbf70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990d464308cd00e7 server_cookie=c81c6eb1e4e07335 in_seq=0 out_seq=0 2024-02-17T12:03:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.419+0000 7f8741ffb640 1 -- 172.21.15.94:0/1102422679 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f87540ef070 con 0x5625748b9b50 2024-02-17T12:03:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.419+0000 7f8741ffb640 10 client.16908 handle_command_reply: tid=0 2024-02-17T12:03:26.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.420+0000 7f8762f9f740 1 client.16908 shutdown 2024-02-17T12:03:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.421+0000 7f8762f9f740 1 -- 172.21.15.94:0/1102422679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 msgr2=0x56257484f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.421+0000 7f8762f9f740 1 --2- 172.21.15.94:0/1102422679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x56257484f650 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8748008010 tx=0x7f87480047d0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.421+0000 7f8762f9f740 1 -- 172.21.15.94:0/1102422679 shutdown_connections 2024-02-17T12:03:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.421+0000 7f8762f9f740 1 --2- 172.21.15.94:0/1102422679 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5625748b9b50 0x5625748bbf70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.421+0000 7f8762f9f740 1 --2- 172.21.15.94:0/1102422679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562574849750 0x5625748b7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.421+0000 7f8762f9f740 1 --2- 172.21.15.94:0/1102422679 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562574840b70 0x56257484f650 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.421+0000 7f8741ffb640 0 client.16908 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.421+0000 7f8762f9f740 1 --2- 172.21.15.94:0/1102422679 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562574840250 0x56257484f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.421+0000 7f8762f9f740 1 -- 172.21.15.94:0/1102422679 >> 172.21.15.94:0/1102422679 conn(0x56257483a980 msgr2=0x562574841a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.422+0000 7f8762f9f740 1 -- 172.21.15.94:0/1102422679 shutdown_connections 2024-02-17T12:03:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.422+0000 7f8762f9f740 1 -- 172.21.15.94:0/1102422679 wait complete. 2024-02-17T12:03:26.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.422+0000 7f8762f9f740 20 client.16908 trim_cache size 0 max 16384 2024-02-17T12:03:26.423 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:03:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.422+0000 7f87602f1640 1 -- 172.21.15.94:0/171585072 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f87440a23b0 msgr2=0x7f87440a4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.422+0000 7f87602f1640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f87440a23b0 0x7f87440a4870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8750009fd0 tx=0x7f8750009ec0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.422+0000 7f87602f1640 1 -- 172.21.15.94:0/171585072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f87580685f0 msgr2=0x7f8758100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.422+0000 7f87602f1640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f87580685f0 0x7f8758100040 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8754005940 tx=0x7f87540042f0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.422+0000 7f87602f1640 1 -- 172.21.15.94:0/171585072 shutdown_connections 2024-02-17T12:03:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.422+0000 7f87602f1640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f87440a23b0 0x7f87440a4870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.422+0000 7f87602f1640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8758100ac0 0x7f87581a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.422+0000 7f87602f1640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87581064c0 0x7f8758100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.422+0000 7f87602f1640 1 --2- 172.21.15.94:0/171585072 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f87580685f0 0x7f8758100040 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.422+0000 7f87602f1640 1 -- 172.21.15.94:0/171585072 >> 172.21.15.94:0/171585072 conn(0x7f8758077180 msgr2=0x7f87580fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.423+0000 7f87602f1640 1 -- 172.21.15.94:0/171585072 shutdown_connections 2024-02-17T12:03:26.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:26.423+0000 7f87602f1640 1 -- 172.21.15.94:0/171585072 wait complete. 2024-02-17T12:03:27.438 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:03:27.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:03:27.438 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:03:27.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.603+0000 7f0986513640 1 -- 172.21.15.94:0/452043730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f098011a960 msgr2=0x7f098011cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.603+0000 7f0986513640 1 --2- 172.21.15.94:0/452043730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f098011a960 0x7f098011cd50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f097001acb0 tx=0x7f097003d150 comp rx=0 tx=0).stop 2024-02-17T12:03:27.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.603+0000 7f0986513640 1 -- 172.21.15.94:0/452043730 shutdown_connections 2024-02-17T12:03:27.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.603+0000 7f0986513640 1 --2- 172.21.15.94:0/452043730 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f098011a960 0x7f098011cd50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.603+0000 7f0986513640 1 --2- 172.21.15.94:0/452043730 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098010d9d0 0x7f098010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.603+0000 7f0986513640 1 --2- 172.21.15.94:0/452043730 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098010f9a0 0x7f098010d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.603+0000 7f0986513640 1 -- 172.21.15.94:0/452043730 >> 172.21.15.94:0/452043730 conn(0x7f098006c260 msgr2=0x7f098006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:27.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.610+0000 7f0986513640 1 -- 172.21.15.94:0/452043730 shutdown_connections 2024-02-17T12:03:27.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.611+0000 7f0986513640 1 -- 172.21.15.94:0/452043730 wait complete. 2024-02-17T12:03:27.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.616+0000 7f0986513640 1 Processor -- start 2024-02-17T12:03:27.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.618+0000 7f0986513640 1 -- start start 2024-02-17T12:03:27.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.618+0000 7f0986513640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098010d9d0 0x7f0980119ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.618+0000 7f0986513640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f098010f9a0 0x7f0980115f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.618+0000 7f0986513640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098011a960 0x7f09801164c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.619+0000 7f0986513640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0980116a00 con 0x7f098010d9d0 2024-02-17T12:03:27.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.619+0000 7f0986513640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0980116b70 con 0x7f098010f9a0 2024-02-17T12:03:27.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.619+0000 7f0986513640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0980116d30 con 0x7f098011a960 2024-02-17T12:03:27.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.619+0000 7f0984ab0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098011a960 0x7f09801164c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.619+0000 7f0984ab0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098011a960 0x7f09801164c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57632/0 (socket says 172.21.15.94:57632) 2024-02-17T12:03:27.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.619+0000 7f0984ab0640 1 -- 172.21.15.94:0/3119105568 learned_addr learned my addr 172.21.15.94:0/3119105568 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:27.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.619+0000 7f0984ab0640 1 -- 172.21.15.94:0/3119105568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f098010f9a0 msgr2=0x7f0980115f80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-17T12:03:27.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.619+0000 7f097f7fe640 1 --2- 172.21.15.94:0/3119105568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f098010f9a0 0x7f0980115f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.619+0000 7f0984ab0640 1 --2- 172.21.15.94:0/3119105568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f098010f9a0 0x7f0980115f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.620+0000 7f0984ab0640 1 -- 172.21.15.94:0/3119105568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098010d9d0 msgr2=0x7f0980119ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.620+0000 7f0984ab0640 1 --2- 172.21.15.94:0/3119105568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098010d9d0 0x7f0980119ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.620+0000 7f0984ab0640 1 -- 172.21.15.94:0/3119105568 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097001a910 con 0x7f098011a960 2024-02-17T12:03:27.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.620+0000 7f0984ab0640 1 --2- 172.21.15.94:0/3119105568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098011a960 0x7f09801164c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0970048040 tx=0x7f09700436b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.620+0000 7f097d7fa640 1 -- 172.21.15.94:0/3119105568 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097003dd00 con 0x7f098011a960 2024-02-17T12:03:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.620+0000 7f0986513640 1 -- 172.21.15.94:0/3119105568 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09801aa5d0 con 0x7f098011a960 2024-02-17T12:03:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.621+0000 7f0986513640 1 -- 172.21.15.94:0/3119105568 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09801aaa20 con 0x7f098011a960 2024-02-17T12:03:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.621+0000 7f097d7fa640 1 -- 172.21.15.94:0/3119105568 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0970043ec0 con 0x7f098011a960 2024-02-17T12:03:27.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.621+0000 7f097d7fa640 1 -- 172.21.15.94:0/3119105568 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097004eb50 con 0x7f098011a960 2024-02-17T12:03:27.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.624+0000 7f09891c6740 1 Processor -- start 2024-02-17T12:03:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.624+0000 7f097d7fa640 1 -- 172.21.15.94:0/3119105568 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f097004c040 con 0x7f098011a960 2024-02-17T12:03:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.625+0000 7f09891c6740 1 -- start start 2024-02-17T12:03:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.625+0000 7f09891c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214380 0x56294b214760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.625+0000 7f09891c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214d30 0x56294b21d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.625+0000 7f09891c6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b21d8b0 0x56294b21fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.625+0000 7f09891c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294b182f90 con 0x56294b214d30 2024-02-17T12:03:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.625+0000 7f09891c6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294b1804c0 con 0x56294b21d8b0 2024-02-17T12:03:27.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.625+0000 7f09891c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294b17bca0 con 0x56294b214380 2024-02-17T12:03:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.625+0000 7f097f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b21d8b0 0x56294b21fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.625+0000 7f097f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b21d8b0 0x56294b21fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59722/0 (socket says 172.21.15.94:59722) 2024-02-17T12:03:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.625+0000 7f097f7fe640 1 -- 172.21.15.94:0/563846127 learned_addr learned my addr 172.21.15.94:0/563846127 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.625+0000 7f097f7fe640 1 --2- 172.21.15.94:0/563846127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214380 0x56294b214760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.625+0000 7f097d7fa640 1 --2- 172.21.15.94:0/3119105568 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f096c0a2370 0x7f096c0a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.626+0000 7f097f7fe640 1 -- 172.21.15.94:0/563846127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214380 msgr2=0x56294b214760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.626+0000 7f097f7fe640 1 --2- 172.21.15.94:0/563846127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214380 0x56294b214760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.626+0000 7f097f7fe640 1 -- 172.21.15.94:0/563846127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214d30 msgr2=0x56294b21d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.626+0000 7f097f7fe640 1 --2- 172.21.15.94:0/563846127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214d30 0x56294b21d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.626+0000 7f097f7fe640 1 -- 172.21.15.94:0/563846127 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56294b13e0b0 con 0x56294b21d8b0 2024-02-17T12:03:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.626+0000 7f097d7fa640 1 -- 172.21.15.94:0/3119105568 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0970100e00 con 0x7f098011a960 2024-02-17T12:03:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.626+0000 7f097f7fe640 1 --2- 172.21.15.94:0/563846127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b21d8b0 0x56294b21fd00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0978013530 tx=0x7f097800e3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c310affaece3667c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:27.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.626+0000 7f0984ab0640 1 --2- 172.21.15.94:0/3119105568 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f096c0a2370 0x7f096c0a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.626+0000 7f09767fc640 1 -- 172.21.15.94:0/563846127 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09780116a0 con 0x56294b21d8b0 2024-02-17T12:03:27.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.626+0000 7f09767fc640 1 -- 172.21.15.94:0/563846127 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0978011ce0 con 0x56294b21d8b0 2024-02-17T12:03:27.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.626+0000 7f09767fc640 1 -- 172.21.15.94:0/563846127 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978010d30 con 0x56294b21d8b0 2024-02-17T12:03:27.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.627+0000 7f0984ab0640 1 --2- 172.21.15.94:0/3119105568 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f096c0a2370 0x7f096c0a4830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f09700cd590 tx=0x7f097004ecb0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:27.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.627+0000 7f09891c6740 1 -- 172.21.15.94:0/563846127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b21d8b0 msgr2=0x56294b21fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.627+0000 7f09891c6740 1 --2- 172.21.15.94:0/563846127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b21d8b0 0x56294b21fd00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0978013530 tx=0x7f097800e3b0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.628+0000 7f09891c6740 1 -- 172.21.15.94:0/563846127 shutdown_connections 2024-02-17T12:03:27.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.628+0000 7f09891c6740 1 --2- 172.21.15.94:0/563846127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b21d8b0 0x56294b21fd00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.628+0000 7f09891c6740 1 --2- 172.21.15.94:0/563846127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214d30 0x56294b21d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.628+0000 7f09891c6740 1 --2- 172.21.15.94:0/563846127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214380 0x56294b214760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.628+0000 7f09891c6740 1 -- 172.21.15.94:0/563846127 >> 172.21.15.94:0/563846127 conn(0x56294b20e840 msgr2=0x56294b20ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:27.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.628+0000 7f09891c6740 1 -- 172.21.15.94:0/563846127 shutdown_connections 2024-02-17T12:03:27.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.628+0000 7f09891c6740 1 -- 172.21.15.94:0/563846127 wait complete. 2024-02-17T12:03:27.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.629+0000 7f09891c6740 1 -- start start 2024-02-17T12:03:27.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.629+0000 7f09891c6740 1 Processor -- start 2024-02-17T12:03:27.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.629+0000 7f09891c6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b214380 0x56294b21a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.630+0000 7f09891c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214d30 0x56294b21ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.630+0000 7f097ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214d30 0x56294b21ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.630+0000 7f09891c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b21d8b0 0x56294b21b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.630+0000 7f09891c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294b120f40 con 0x56294b214d30 2024-02-17T12:03:27.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.630+0000 7f09891c6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294ae4f4b0 con 0x56294b214380 2024-02-17T12:03:27.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.630+0000 7f09891c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294b101670 con 0x56294b21d8b0 2024-02-17T12:03:27.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.631+0000 7f09891c6740 10 client.0 _ll_register_callbacks cb 0x56294b16ef70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:27.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.631+0000 7f097ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214d30 0x56294b21ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35432/0 (socket says 172.21.15.94:35432) 2024-02-17T12:03:27.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.631+0000 7f097ffff640 1 -- 172.21.15.94:0/1416928459 learned_addr learned my addr 172.21.15.94:0/1416928459 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.631+0000 7f097ffff640 1 -- 172.21.15.94:0/1416928459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b21d8b0 msgr2=0x56294b21b380 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:03:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.631+0000 7f097ffff640 1 --2- 172.21.15.94:0/1416928459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b21d8b0 0x56294b21b380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.631+0000 7f097ffff640 1 -- 172.21.15.94:0/1416928459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b214380 msgr2=0x56294b21a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.631+0000 7f097ffff640 1 --2- 172.21.15.94:0/1416928459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b214380 0x56294b21a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.631+0000 7f097ffff640 1 -- 172.21.15.94:0/1416928459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097800edd0 con 0x56294b214d30 2024-02-17T12:03:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.632+0000 7f097ffff640 1 --2- 172.21.15.94:0/1416928459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214d30 0x56294b21ae40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0968009df0 tx=0x7f096800b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ed0b28af3b9b5b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.632+0000 7f095f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:03:27.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.632+0000 7f095f7fe640 1 -- 172.21.15.94:0/1416928459 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968011420 con 0x56294b214d30 2024-02-17T12:03:27.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.632+0000 7f095f7fe640 1 -- 172.21.15.94:0/1416928459 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0968011580 con 0x56294b214d30 2024-02-17T12:03:27.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.632+0000 7f09891c6740 1 -- 172.21.15.94:0/1416928459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56294b155f40 con 0x56294b214d30 2024-02-17T12:03:27.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.632+0000 7f095f7fe640 1 -- 172.21.15.94:0/1416928459 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0968011740 con 0x56294b214d30 2024-02-17T12:03:27.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.633+0000 7f09891c6740 10 client.16911 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:27.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.633+0000 7f09891c6740 1 -- 172.21.15.94:0/1416928459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56294b17a320 con 0x56294b214d30 2024-02-17T12:03:27.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.633+0000 7f095f7fe640 1 -- 172.21.15.94:0/1416928459 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56294b155f40 con 0x56294b214d30 2024-02-17T12:03:27.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.634+0000 7f095f7fe640 1 -- 172.21.15.94:0/1416928459 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0968019c70 con 0x56294b214d30 2024-02-17T12:03:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.634+0000 7f09891c6740 10 client.16911 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.634+0000 7f09891c6740 10 client.16911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.634+0000 7f09891c6740 20 client.16911 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.634+0000 7f09891c6740 1 --2- 172.21.15.94:0/1416928459 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56294b22e0b0 0x56294b230550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.634+0000 7f09891c6740 4 client.16911 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.634+0000 7f09891c6740 1 -- 172.21.15.94:0/1416928459 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56294b172310 con 0x56294b22e0b0 2024-02-17T12:03:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.634+0000 7f097f7fe640 1 --2- 172.21.15.94:0/1416928459 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56294b22e0b0 0x56294b230550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.635+0000 7f097f7fe640 1 --2- 172.21.15.94:0/1416928459 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56294b22e0b0 0x56294b230550 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=826dc6d4e07c8841 server_cookie=d68247022c023abf in_seq=0 out_seq=0 2024-02-17T12:03:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.635+0000 7f095f7fe640 10 client.16911 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.635+0000 7f095f7fe640 1 -- 172.21.15.94:0/1416928459 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56294b172310 con 0x56294b22e0b0 2024-02-17T12:03:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.635+0000 7f095f7fe640 10 client.16911 handle_command_reply: tid=0 2024-02-17T12:03:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.635+0000 7f09891c6740 1 client.16911 shutdown 2024-02-17T12:03:27.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.636+0000 7f09891c6740 1 -- 172.21.15.94:0/1416928459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214d30 msgr2=0x56294b21ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.636+0000 7f09891c6740 1 --2- 172.21.15.94:0/1416928459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214d30 0x56294b21ae40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f0968009df0 tx=0x7f096800b4b0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.636+0000 7f09891c6740 1 -- 172.21.15.94:0/1416928459 shutdown_connections 2024-02-17T12:03:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.636+0000 7f09891c6740 1 --2- 172.21.15.94:0/1416928459 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56294b22e0b0 0x56294b230550 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.636+0000 7f09891c6740 1 --2- 172.21.15.94:0/1416928459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b21d8b0 0x56294b21b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.636+0000 7f09891c6740 1 --2- 172.21.15.94:0/1416928459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214d30 0x56294b21ae40 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.636+0000 7f09891c6740 1 --2- 172.21.15.94:0/1416928459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b214380 0x56294b21a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.636+0000 7f09891c6740 1 -- 172.21.15.94:0/1416928459 >> 172.21.15.94:0/1416928459 conn(0x56294b20e840 msgr2=0x56294b2107a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.636+0000 7f095f7fe640 0 client.16911 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.636+0000 7f09891c6740 1 -- 172.21.15.94:0/1416928459 shutdown_connections 2024-02-17T12:03:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.636+0000 7f09891c6740 1 -- 172.21.15.94:0/1416928459 wait complete. 2024-02-17T12:03:27.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.636+0000 7f09891c6740 20 client.16911 trim_cache size 0 max 16384 2024-02-17T12:03:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.658+0000 7f09891c6740 1 Processor -- start 2024-02-17T12:03:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.658+0000 7f09891c6740 1 -- start start 2024-02-17T12:03:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.658+0000 7f09891c6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b214380 0x56294b21d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.658+0000 7f09891c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214d30 0x56294b229090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.658+0000 7f09891c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b21d8b0 0x56294b217b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.658+0000 7f09891c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294b216270 con 0x56294b21d8b0 2024-02-17T12:03:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.658+0000 7f09891c6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294b172310 con 0x56294b214380 2024-02-17T12:03:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.658+0000 7f09891c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294b22aa90 con 0x56294b214d30 2024-02-17T12:03:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.658+0000 7f097f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b214380 0x56294b21d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.658+0000 7f097f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b214380 0x56294b21d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59738/0 (socket says 172.21.15.94:59738) 2024-02-17T12:03:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.658+0000 7f097f7fe640 1 -- 172.21.15.94:0/1929328549 learned_addr learned my addr 172.21.15.94:0/1929328549 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.658+0000 7f097f7fe640 1 --2- 172.21.15.94:0/1929328549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b21d8b0 0x56294b217b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.658+0000 7f097ffff640 1 --2- 172.21.15.94:0/1929328549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214d30 0x56294b229090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.659+0000 7f097f7fe640 1 -- 172.21.15.94:0/1929328549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214d30 msgr2=0x56294b229090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.659+0000 7f097f7fe640 1 --2- 172.21.15.94:0/1929328549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214d30 0x56294b229090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.659+0000 7f097f7fe640 1 -- 172.21.15.94:0/1929328549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b21d8b0 msgr2=0x56294b217b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.659+0000 7f097f7fe640 1 --2- 172.21.15.94:0/1929328549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b21d8b0 0x56294b217b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.659+0000 7f097f7fe640 1 -- 172.21.15.94:0/1929328549 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56294b116c00 con 0x56294b214380 2024-02-17T12:03:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.659+0000 7f097f7fe640 1 --2- 172.21.15.94:0/1929328549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b214380 0x56294b21d030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0978013f00 tx=0x7f0978000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31de40b0a559baf8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.659+0000 7f0975ffb640 1 -- 172.21.15.94:0/1929328549 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978005220 con 0x56294b214380 2024-02-17T12:03:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.659+0000 7f0975ffb640 1 -- 172.21.15.94:0/1929328549 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09780105c0 con 0x56294b214380 2024-02-17T12:03:27.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.659+0000 7f0975ffb640 1 -- 172.21.15.94:0/1929328549 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978025680 con 0x56294b214380 2024-02-17T12:03:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.660+0000 7f09891c6740 1 -- 172.21.15.94:0/1929328549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b214380 msgr2=0x56294b21d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.660+0000 7f09891c6740 1 --2- 172.21.15.94:0/1929328549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b214380 0x56294b21d030 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0978013f00 tx=0x7f0978000bc0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.661+0000 7f09891c6740 1 -- 172.21.15.94:0/1929328549 shutdown_connections 2024-02-17T12:03:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.661+0000 7f09891c6740 1 --2- 172.21.15.94:0/1929328549 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b21d8b0 0x56294b217b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.661+0000 7f09891c6740 1 --2- 172.21.15.94:0/1929328549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214d30 0x56294b229090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.661+0000 7f09891c6740 1 --2- 172.21.15.94:0/1929328549 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b214380 0x56294b21d030 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.661+0000 7f09891c6740 1 -- 172.21.15.94:0/1929328549 >> 172.21.15.94:0/1929328549 conn(0x56294b20e840 msgr2=0x56294b228980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.661+0000 7f09891c6740 1 -- 172.21.15.94:0/1929328549 shutdown_connections 2024-02-17T12:03:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.661+0000 7f09891c6740 1 -- 172.21.15.94:0/1929328549 wait complete. 2024-02-17T12:03:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.662+0000 7f09891c6740 1 -- start start 2024-02-17T12:03:27.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.662+0000 7f09891c6740 1 Processor -- start 2024-02-17T12:03:27.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.662+0000 7f09891c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214380 0x56294b235220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f09891c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214d30 0x56294b2335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f09891c6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b21d8b0 0x56294b233b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f09891c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294b234250 con 0x56294b214380 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f09891c6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294b2343c0 con 0x56294b21d8b0 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f09891c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56294b2845f0 con 0x56294b214d30 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f09891c6740 10 client.0 _ll_register_callbacks cb 0x7f097800e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f097ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214d30 0x56294b2335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f097ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214d30 0x56294b2335c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57668/0 (socket says 172.21.15.94:57668) 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f097ffff640 1 -- 172.21.15.94:0/3431643858 learned_addr learned my addr 172.21.15.94:0/3431643858 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f097f7fe640 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b21d8b0 0x56294b233b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f097f7fe640 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214380 0x56294b235220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f097ffff640 1 -- 172.21.15.94:0/3431643858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b21d8b0 msgr2=0x56294b233b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f097ffff640 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b21d8b0 0x56294b233b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f097ffff640 1 -- 172.21.15.94:0/3431643858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214380 msgr2=0x56294b235220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f097ffff640 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214380 0x56294b235220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f097ffff640 1 -- 172.21.15.94:0/3431643858 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56294b13e0b0 con 0x56294b214d30 2024-02-17T12:03:27.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f097f7fe640 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214380 0x56294b235220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f097ffff640 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214d30 0x56294b2335c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x56294b21a160 tx=0x7f0968006bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb58f2580bbcd8fb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f095f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.663+0000 7f095f7fe640 1 -- 172.21.15.94:0/3431643858 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0968010b00 con 0x56294b214d30 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f095f7fe640 1 -- 172.21.15.94:0/3431643858 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0968000e40 con 0x56294b214d30 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f09891c6740 1 -- 172.21.15.94:0/3431643858 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56294b1a1900 con 0x56294b214d30 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f095f7fe640 1 -- 172.21.15.94:0/3431643858 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0968010c60 con 0x56294b214d30 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f09891c6740 10 client.26434 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f09891c6740 1 -- 172.21.15.94:0/3431643858 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56294b13c910 con 0x56294b214d30 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f095f7fe640 1 -- 172.21.15.94:0/3431643858 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56294b1a1900 con 0x56294b214d30 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f095f7fe640 1 -- 172.21.15.94:0/3431643858 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0968011420 con 0x56294b214d30 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f09891c6740 10 client.26434 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f09891c6740 10 client.26434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f09891c6740 20 client.26434 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:27.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f09891c6740 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56294b285da0 0x56294b2881c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f09891c6740 4 client.26434 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:03:27.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f09891c6740 1 -- 172.21.15.94:0/3431643858 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56294ae7b860 con 0x56294b285da0 2024-02-17T12:03:27.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.664+0000 7f097f7fe640 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56294b285da0 0x56294b2881c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.665+0000 7f095f7fe640 10 client.26434 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:27.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.665+0000 7f097f7fe640 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56294b285da0 0x56294b2881c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c6c7c975b1e3b29 server_cookie=d4994487e262d1e0 in_seq=0 out_seq=0 2024-02-17T12:03:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.666+0000 7f095f7fe640 1 -- 172.21.15.94:0/3431643858 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56294ae7b860 con 0x56294b285da0 2024-02-17T12:03:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.666+0000 7f095f7fe640 10 client.26434 handle_command_reply: tid=0 2024-02-17T12:03:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.666+0000 7f09891c6740 1 client.26434 shutdown 2024-02-17T12:03:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.666+0000 7f09891c6740 1 -- 172.21.15.94:0/3431643858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214d30 msgr2=0x56294b2335c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.666+0000 7f09891c6740 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214d30 0x56294b2335c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x56294b21a160 tx=0x7f0968006bb0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.666+0000 7f09891c6740 1 -- 172.21.15.94:0/3431643858 shutdown_connections 2024-02-17T12:03:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.666+0000 7f09891c6740 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56294b285da0 0x56294b2881c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.666+0000 7f09891c6740 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56294b21d8b0 0x56294b233b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.666+0000 7f09891c6740 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56294b214d30 0x56294b2335c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.666+0000 7f09891c6740 1 --2- 172.21.15.94:0/3431643858 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56294b214380 0x56294b235220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.666+0000 7f09891c6740 1 -- 172.21.15.94:0/3431643858 >> 172.21.15.94:0/3431643858 conn(0x56294b20e840 msgr2=0x56294b21b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.666+0000 7f095f7fe640 0 client.26434 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.666+0000 7f09891c6740 1 -- 172.21.15.94:0/3431643858 shutdown_connections 2024-02-17T12:03:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.667+0000 7f09891c6740 1 -- 172.21.15.94:0/3431643858 wait complete. 2024-02-17T12:03:27.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.667+0000 7f09891c6740 20 client.26434 trim_cache size 0 max 16384 2024-02-17T12:03:27.670 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:03:27.670 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:03:27.670 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "da97004e-ccba-4fb1-8d81-e07f3c3a474b", 2024-02-17T12:03:27.670 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:03:27.670 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:03:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.667+0000 7f0986513640 1 -- 172.21.15.94:0/3119105568 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f096c0a2370 msgr2=0x7f096c0a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.667+0000 7f0986513640 1 --2- 172.21.15.94:0/3119105568 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f096c0a2370 0x7f096c0a4830 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f09700cd590 tx=0x7f097004ecb0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.667+0000 7f0986513640 1 -- 172.21.15.94:0/3119105568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098011a960 msgr2=0x7f09801164c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.667+0000 7f0986513640 1 --2- 172.21.15.94:0/3119105568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098011a960 0x7f09801164c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0970048040 tx=0x7f09700436b0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.668+0000 7f0986513640 1 -- 172.21.15.94:0/3119105568 shutdown_connections 2024-02-17T12:03:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.668+0000 7f0986513640 1 --2- 172.21.15.94:0/3119105568 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f096c0a2370 0x7f096c0a4830 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.668+0000 7f0986513640 1 --2- 172.21.15.94:0/3119105568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f098011a960 0x7f09801164c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.668+0000 7f0986513640 1 --2- 172.21.15.94:0/3119105568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f098010f9a0 0x7f0980115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.668+0000 7f0986513640 1 --2- 172.21.15.94:0/3119105568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f098010d9d0 0x7f0980119ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.668+0000 7f0986513640 1 -- 172.21.15.94:0/3119105568 >> 172.21.15.94:0/3119105568 conn(0x7f098006c260 msgr2=0x7f098011c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.668+0000 7f0986513640 1 -- 172.21.15.94:0/3119105568 shutdown_connections 2024-02-17T12:03:27.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.668+0000 7f0986513640 1 -- 172.21.15.94:0/3119105568 wait complete. 2024-02-17T12:03:27.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:03:27.683 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:03:27.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.848+0000 7f05a52ae640 1 -- 172.21.15.94:0/15773876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a00ffca0 msgr2=0x7f05a0103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.848+0000 7f05a52ae640 1 --2- 172.21.15.94:0/15773876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a00ffca0 0x7f05a0103f90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0594009900 tx=0x7f059402bc30 comp rx=0 tx=0).stop 2024-02-17T12:03:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.848+0000 7f05a52ae640 1 -- 172.21.15.94:0/15773876 shutdown_connections 2024-02-17T12:03:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.848+0000 7f05a52ae640 1 --2- 172.21.15.94:0/15773876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05a01044d0 0x7f05a0104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.848+0000 7f05a52ae640 1 --2- 172.21.15.94:0/15773876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a00ffca0 0x7f05a0103f90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.848+0000 7f05a52ae640 1 --2- 172.21.15.94:0/15773876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05a00ff380 0x7f05a00ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.848+0000 7f05a52ae640 1 -- 172.21.15.94:0/15773876 >> 172.21.15.94:0/15773876 conn(0x7f05a0076210 msgr2=0x7f05a00fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.848+0000 7f05a52ae640 1 -- 172.21.15.94:0/15773876 shutdown_connections 2024-02-17T12:03:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.848+0000 7f05a52ae640 1 -- 172.21.15.94:0/15773876 wait complete. 2024-02-17T12:03:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.848+0000 7f05a52ae640 1 Processor -- start 2024-02-17T12:03:27.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.848+0000 7f05a52ae640 1 -- start start 2024-02-17T12:03:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.849+0000 7f05a52ae640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a00ff380 0x7f05a019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.849+0000 7f05a52ae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05a00ffca0 0x7f05a019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.849+0000 7f05a52ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05a01044d0 0x7f05a01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.849+0000 7f05a52ae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0199e70 con 0x7f05a01044d0 2024-02-17T12:03:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.849+0000 7f05a52ae640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0199fe0 con 0x7f05a00ff380 2024-02-17T12:03:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.849+0000 7f05a52ae640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a019a1a0 con 0x7f05a00ffca0 2024-02-17T12:03:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.849+0000 7f059effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a00ff380 0x7f05a019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.849+0000 7f059effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a00ff380 0x7f05a019f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59774/0 (socket says 172.21.15.94:59774) 2024-02-17T12:03:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.849+0000 7f059effd640 1 -- 172.21.15.94:0/2316640922 learned_addr learned my addr 172.21.15.94:0/2316640922 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.849+0000 7f059f7fe640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05a01044d0 0x7f05a01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.849+0000 7f059e7fc640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05a00ffca0 0x7f05a019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.850+0000 7f059effd640 1 -- 172.21.15.94:0/2316640922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05a00ffca0 msgr2=0x7f05a019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.850+0000 7f059effd640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05a00ffca0 0x7f05a019fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.850+0000 7f059effd640 1 -- 172.21.15.94:0/2316640922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05a01044d0 msgr2=0x7f05a01997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.850+0000 7f059effd640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05a01044d0 0x7f05a01997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.850+0000 7f059effd640 1 -- 172.21.15.94:0/2316640922 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0594009560 con 0x7f05a00ff380 2024-02-17T12:03:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.850+0000 7f059effd640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a00ff380 0x7f05a019f660 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f059000d950 tx=0x7f059000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.851+0000 7f0587fff640 1 -- 172.21.15.94:0/2316640922 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590009c30 con 0x7f05a00ff380 2024-02-17T12:03:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.851+0000 7f0587fff640 1 -- 172.21.15.94:0/2316640922 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0590009d90 con 0x7f05a00ff380 2024-02-17T12:03:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.851+0000 7f05a52ae640 1 -- 172.21.15.94:0/2316640922 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a019a480 con 0x7f05a00ff380 2024-02-17T12:03:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.851+0000 7f05a52ae640 1 -- 172.21.15.94:0/2316640922 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a01a6690 con 0x7f05a00ff380 2024-02-17T12:03:27.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.851+0000 7f0587fff640 1 -- 172.21.15.94:0/2316640922 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059000f7e0 con 0x7f05a00ff380 2024-02-17T12:03:27.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.854+0000 7f0587fff640 1 -- 172.21.15.94:0/2316640922 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f059000f940 con 0x7f05a00ff380 2024-02-17T12:03:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.854+0000 7f05a7e84740 1 Processor -- start 2024-02-17T12:03:27.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.854+0000 7f05a7e84740 1 -- start start 2024-02-17T12:03:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f05a7e84740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8185090c0 0x55d8185094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f05a7e84740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d818509a70 0x55d81851aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f05a7e84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d81851afc0 0x55d81851d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f05a7e84740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d818478280 con 0x55d8185090c0 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f05a7e84740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8184757b0 con 0x55d818509a70 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f05a7e84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d818470f90 con 0x55d81851afc0 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f0587fff640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f05880a22f0 0x7f05880a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8185090c0 0x55d8185094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8185090c0 0x55d8185094a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35472/0 (socket says 172.21.15.94:35472) 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f059f7fe640 1 -- 172.21.15.94:0/659888418 learned_addr learned my addr 172.21.15.94:0/659888418 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f059e7fc640 1 --2- 172.21.15.94:0/659888418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d81851afc0 0x55d81851d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f059effd640 1 --2- 172.21.15.94:0/659888418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d818509a70 0x55d81851aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f059f7fe640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f05880a22f0 0x7f05880a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f0587fff640 1 -- 172.21.15.94:0/2316640922 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f05900cdfe0 con 0x7f05a00ff380 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f059effd640 1 -- 172.21.15.94:0/659888418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d81851afc0 msgr2=0x55d81851d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.855+0000 7f059effd640 1 --2- 172.21.15.94:0/659888418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d81851afc0 0x55d81851d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.856+0000 7f059effd640 1 -- 172.21.15.94:0/659888418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8185090c0 msgr2=0x55d8185094a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.856+0000 7f059effd640 1 --2- 172.21.15.94:0/659888418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8185090c0 0x55d8185094a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.856+0000 7f059effd640 1 -- 172.21.15.94:0/659888418 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d81842f570 con 0x55d818509a70 2024-02-17T12:03:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.856+0000 7f059f7fe640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f05880a22f0 0x7f05880a47b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f059800acd0 tx=0x7f0598009a50 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.856+0000 7f059effd640 1 --2- 172.21.15.94:0/659888418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d818509a70 0x55d81851aa80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f05900cb6f0 tx=0x7f0590009f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=318671e39d250c5c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.856+0000 7f05857fa640 1 -- 172.21.15.94:0/659888418 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900ba040 con 0x55d818509a70 2024-02-17T12:03:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.856+0000 7f05857fa640 1 -- 172.21.15.94:0/659888418 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f059009a9c0 con 0x55d818509a70 2024-02-17T12:03:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.856+0000 7f05857fa640 1 -- 172.21.15.94:0/659888418 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900a2cd0 con 0x55d818509a70 2024-02-17T12:03:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.857+0000 7f05a7e84740 1 -- 172.21.15.94:0/659888418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d818509a70 msgr2=0x55d81851aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.857+0000 7f05a7e84740 1 --2- 172.21.15.94:0/659888418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d818509a70 0x55d81851aa80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f05900cb6f0 tx=0x7f0590009f00 comp rx=0 tx=0).stop 2024-02-17T12:03:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.857+0000 7f05a7e84740 1 -- 172.21.15.94:0/659888418 shutdown_connections 2024-02-17T12:03:27.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.857+0000 7f05a7e84740 1 --2- 172.21.15.94:0/659888418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d81851afc0 0x55d81851d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.857+0000 7f05a7e84740 1 --2- 172.21.15.94:0/659888418 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d818509a70 0x55d81851aa80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.857+0000 7f05a7e84740 1 --2- 172.21.15.94:0/659888418 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d8185090c0 0x55d8185094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.857+0000 7f05a7e84740 1 -- 172.21.15.94:0/659888418 >> 172.21.15.94:0/659888418 conn(0x55d818503990 msgr2=0x55d818503d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.857+0000 7f05a7e84740 1 -- 172.21.15.94:0/659888418 shutdown_connections 2024-02-17T12:03:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.857+0000 7f05a7e84740 1 -- 172.21.15.94:0/659888418 wait complete. 2024-02-17T12:03:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.857+0000 7f05a7e84740 1 -- start start 2024-02-17T12:03:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.857+0000 7f05a7e84740 1 Processor -- start 2024-02-17T12:03:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.858+0000 7f05a7e84740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d81851aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.858+0000 7f05a7e84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d818509a70 0x55d818517dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.858+0000 7f05a7e84740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81851afc0 0x55d818518500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.858+0000 7f059e7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d818509a70 0x55d818517dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.858+0000 7f059effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d81851aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.858+0000 7f059effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d81851aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59796/0 (socket says 172.21.15.94:59796) 2024-02-17T12:03:27.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.858+0000 7f059effd640 1 -- 172.21.15.94:0/2543587273 learned_addr learned my addr 172.21.15.94:0/2543587273 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:27.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.858+0000 7f05a7e84740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d818144430 con 0x55d81851afc0 2024-02-17T12:03:27.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.858+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81851afc0 0x55d818518500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.858+0000 7f05a7e84740 1 -- 172.21.15.94:0/2543587273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d818100e00 con 0x55d8185090c0 2024-02-17T12:03:27.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.858+0000 7f05a7e84740 1 -- 172.21.15.94:0/2543587273 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8183f5fd0 con 0x55d818509a70 2024-02-17T12:03:27.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.858+0000 7f05a7e84740 10 client.0 _ll_register_callbacks cb 0x55d81844bfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:27.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.859+0000 7f059effd640 1 -- 172.21.15.94:0/2543587273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d818509a70 msgr2=0x55d818517dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.859+0000 7f059effd640 1 --2- 172.21.15.94:0/2543587273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d818509a70 0x55d818517dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.859+0000 7f059effd640 1 -- 172.21.15.94:0/2543587273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81851afc0 msgr2=0x55d818518500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.859+0000 7f059effd640 1 --2- 172.21.15.94:0/2543587273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81851afc0 0x55d818518500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.859+0000 7f059effd640 1 -- 172.21.15.94:0/2543587273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a01a6690 con 0x55d8185090c0 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.859+0000 7f059effd640 1 --2- 172.21.15.94:0/2543587273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d81851aa30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0590009610 tx=0x7f0590097260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2389f5cdded19d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.859+0000 7f0582ff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f0582ff5640 1 -- 172.21.15.94:0/2543587273 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900ba040 con 0x55d8185090c0 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f05a7e84740 1 -- 172.21.15.94:0/2543587273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8181707e0 con 0x55d8185090c0 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f0582ff5640 1 -- 172.21.15.94:0/2543587273 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f059009eb10 con 0x55d8185090c0 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f0582ff5640 1 -- 172.21.15.94:0/2543587273 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05900cb9a0 con 0x55d8185090c0 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f05a7e84740 10 client.26396 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f05a7e84740 1 -- 172.21.15.94:0/2543587273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d81846f610 con 0x55d8185090c0 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f0582ff5640 1 -- 172.21.15.94:0/2543587273 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05900b2070 con 0x55d8185090c0 2024-02-17T12:03:27.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f0582ff5640 1 -- 172.21.15.94:0/2543587273 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0590023070 con 0x55d8185090c0 2024-02-17T12:03:27.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f05a7e84740 10 client.26396 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:27.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f05a7e84740 10 client.26396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:27.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f05a7e84740 20 client.26396 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:27.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f05a7e84740 1 --2- 172.21.15.94:0/2543587273 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d81850b640 0x55d81850da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f05a7e84740 4 client.26396 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:27.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.860+0000 7f05a7e84740 1 -- 172.21.15.94:0/2543587273 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8181ad720 con 0x55d81850b640 2024-02-17T12:03:27.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.861+0000 7f059e7fc640 1 --2- 172.21.15.94:0/2543587273 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d81850b640 0x55d81850da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.861+0000 7f059e7fc640 1 --2- 172.21.15.94:0/2543587273 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d81850b640 0x55d81850da60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49d81a56edf3747e server_cookie=7168dd22f17a91b1 in_seq=0 out_seq=0 2024-02-17T12:03:27.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.861+0000 7f0582ff5640 10 client.26396 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f0582ff5640 1 -- 172.21.15.94:0/2543587273 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d8181ad720 con 0x55d81850b640 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f0582ff5640 10 client.26396 handle_command_reply: tid=0 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f05a7e84740 1 client.26396 shutdown 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f05a7e84740 1 -- 172.21.15.94:0/2543587273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 msgr2=0x55d81851aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f05a7e84740 1 --2- 172.21.15.94:0/2543587273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d81851aa30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0590009610 tx=0x7f0590097260 comp rx=0 tx=0).stop 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f05a7e84740 1 -- 172.21.15.94:0/2543587273 shutdown_connections 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f05a7e84740 1 --2- 172.21.15.94:0/2543587273 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d81850b640 0x55d81850da60 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f05a7e84740 1 --2- 172.21.15.94:0/2543587273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81851afc0 0x55d818518500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f05a7e84740 1 --2- 172.21.15.94:0/2543587273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d818509a70 0x55d818517dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f05a7e84740 1 --2- 172.21.15.94:0/2543587273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d81851aa30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f05a7e84740 1 -- 172.21.15.94:0/2543587273 >> 172.21.15.94:0/2543587273 conn(0x55d818503990 msgr2=0x55d818505730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f0582ff5640 0 client.26396 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f05a7e84740 1 -- 172.21.15.94:0/2543587273 shutdown_connections 2024-02-17T12:03:27.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f05a7e84740 1 -- 172.21.15.94:0/2543587273 wait complete. 2024-02-17T12:03:27.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.862+0000 7f05a7e84740 20 client.26396 trim_cache size 0 max 16384 2024-02-17T12:03:27.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.883+0000 7f05a7e84740 1 Processor -- start 2024-02-17T12:03:27.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.883+0000 7f05a7e84740 1 -- start start 2024-02-17T12:03:27.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.883+0000 7f05a7e84740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d818517d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.883+0000 7f05a7e84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d818509a70 0x55d81851a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.883+0000 7f05a7e84740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81851afc0 0x55d81851aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.883+0000 7f05a7e84740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8185083a0 con 0x55d81851afc0 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.883+0000 7f05a7e84740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8185031d0 con 0x55d8185090c0 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.883+0000 7f05a7e84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d818504c10 con 0x55d818509a70 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.883+0000 7f059e7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d818509a70 0x55d81851a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.883+0000 7f059e7fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d818509a70 0x55d81851a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57718/0 (socket says 172.21.15.94:57718) 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.883+0000 7f059e7fc640 1 -- 172.21.15.94:0/1362991557 learned_addr learned my addr 172.21.15.94:0/1362991557 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.883+0000 7f059effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d818517d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.884+0000 7f059f7fe640 1 --2- 172.21.15.94:0/1362991557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81851afc0 0x55d81851aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.884+0000 7f059e7fc640 1 -- 172.21.15.94:0/1362991557 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 msgr2=0x55d818517d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.884+0000 7f059e7fc640 1 --2- 172.21.15.94:0/1362991557 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d818517d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.884+0000 7f059e7fc640 1 -- 172.21.15.94:0/1362991557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81851afc0 msgr2=0x55d81851aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.884+0000 7f059e7fc640 1 --2- 172.21.15.94:0/1362991557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81851afc0 0x55d81851aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.884+0000 7f059e7fc640 1 -- 172.21.15.94:0/1362991557 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d81840bac0 con 0x55d818509a70 2024-02-17T12:03:27.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.884+0000 7f059e7fc640 1 --2- 172.21.15.94:0/1362991557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d818509a70 0x55d81851a4f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f059402c240 tx=0x7f0594009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=618599a10e3d77d1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:27.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.884+0000 7f0584ff9640 1 -- 172.21.15.94:0/1362991557 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594045040 con 0x55d818509a70 2024-02-17T12:03:27.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.884+0000 7f0584ff9640 1 -- 172.21.15.94:0/1362991557 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0594047b70 con 0x55d818509a70 2024-02-17T12:03:27.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.884+0000 7f0584ff9640 1 -- 172.21.15.94:0/1362991557 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059402cea0 con 0x55d818509a70 2024-02-17T12:03:27.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.885+0000 7f05a7e84740 1 -- 172.21.15.94:0/1362991557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d818509a70 msgr2=0x55d81851a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.885+0000 7f05a7e84740 1 --2- 172.21.15.94:0/1362991557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d818509a70 0x55d81851a4f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f059402c240 tx=0x7f0594009260 comp rx=0 tx=0).stop 2024-02-17T12:03:27.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.885+0000 7f05a7e84740 1 -- 172.21.15.94:0/1362991557 shutdown_connections 2024-02-17T12:03:27.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.885+0000 7f05a7e84740 1 --2- 172.21.15.94:0/1362991557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d81851afc0 0x55d81851aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.885+0000 7f05a7e84740 1 --2- 172.21.15.94:0/1362991557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d818509a70 0x55d81851a4f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.885+0000 7f05a7e84740 1 --2- 172.21.15.94:0/1362991557 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d818517d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.885+0000 7f05a7e84740 1 -- 172.21.15.94:0/1362991557 >> 172.21.15.94:0/1362991557 conn(0x55d818503990 msgr2=0x55d818517650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:27.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f05a7e84740 1 -- 172.21.15.94:0/1362991557 shutdown_connections 2024-02-17T12:03:27.888 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f05a7e84740 1 -- 172.21.15.94:0/1362991557 wait complete. 2024-02-17T12:03:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f05a7e84740 1 -- start start 2024-02-17T12:03:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f05a7e84740 1 Processor -- start 2024-02-17T12:03:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f05a7e84740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d81852dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f05a7e84740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d818509a70 0x55d81852bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f05a7e84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d81851afc0 0x55d81852c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f05a7e84740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81852e3d0 con 0x55d818509a70 2024-02-17T12:03:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f05a7e84740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81852c9e0 con 0x55d8185090c0 2024-02-17T12:03:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f05a7e84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d81852cba0 con 0x55d81851afc0 2024-02-17T12:03:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f05a7e84740 10 client.0 _ll_register_callbacks cb 0x7f059009a500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:27.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f059effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d81852dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f059e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d818509a70 0x55d81852bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d81851afc0 0x55d81852c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f059e7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d818509a70 0x55d81852bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35506/0 (socket says 172.21.15.94:35506) 2024-02-17T12:03:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f059effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d81852dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59812/0 (socket says 172.21.15.94:59812) 2024-02-17T12:03:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d81851afc0 0x55d81852c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57726/0 (socket says 172.21.15.94:57726) 2024-02-17T12:03:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.886+0000 7f059e7fc640 1 -- 172.21.15.94:0/2667207263 learned_addr learned my addr 172.21.15.94:0/2667207263 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.887+0000 7f059effd640 1 -- 172.21.15.94:0/2667207263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d81851afc0 msgr2=0x55d81852c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.887+0000 7f059effd640 1 --2- 172.21.15.94:0/2667207263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d81851afc0 0x55d81852c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.887+0000 7f059effd640 1 -- 172.21.15.94:0/2667207263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d818509a70 msgr2=0x55d81852bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.887+0000 7f059effd640 1 --2- 172.21.15.94:0/2667207263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d818509a70 0x55d81852bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.887+0000 7f059effd640 1 -- 172.21.15.94:0/2667207263 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a0106700 con 0x55d8185090c0 2024-02-17T12:03:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.887+0000 7f059e7fc640 1 --2- 172.21.15.94:0/2667207263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d818509a70 0x55d81852bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:27.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.887+0000 7f059effd640 1 --2- 172.21.15.94:0/2667207263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d81852dad0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f059009a950 tx=0x7f059009a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbe02c36056f3783 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:27.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.887+0000 7f0582ff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:03:27.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.887+0000 7f0582ff5640 1 -- 172.21.15.94:0/2667207263 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900ba040 con 0x55d8185090c0 2024-02-17T12:03:27.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.887+0000 7f0582ff5640 1 -- 172.21.15.94:0/2667207263 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f059009eb10 con 0x55d8185090c0 2024-02-17T12:03:27.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.888+0000 7f05a7e84740 1 -- 172.21.15.94:0/2667207263 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f059009a9c0 con 0x55d8185090c0 2024-02-17T12:03:27.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.889+0000 7f0582ff5640 1 -- 172.21.15.94:0/2667207263 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05900b3500 con 0x55d8185090c0 2024-02-17T12:03:27.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.889+0000 7f05a7e84740 10 client.26402 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:27.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.889+0000 7f05a7e84740 1 -- 172.21.15.94:0/2667207263 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d8184316a0 con 0x55d8185090c0 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.889+0000 7f0582ff5640 1 -- 172.21.15.94:0/2667207263 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f05900b8080 con 0x55d8185090c0 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.889+0000 7f0582ff5640 1 -- 172.21.15.94:0/2667207263 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0590023040 con 0x55d8185090c0 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.889+0000 7f05a7e84740 10 client.26402 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.889+0000 7f05a7e84740 10 client.26402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.889+0000 7f05a7e84740 20 client.26402 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.889+0000 7f05a7e84740 1 --2- 172.21.15.94:0/2667207263 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d81857a1d0 0x55d81857c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.889+0000 7f05a7e84740 4 client.26402 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.889+0000 7f05a7e84740 1 -- 172.21.15.94:0/2667207263 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f059009a9c0 con 0x55d81857a1d0 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.889+0000 7f059e7fc640 1 --2- 172.21.15.94:0/2667207263 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d81857a1d0 0x55d81857c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f0582ff5640 10 client.26402 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f059e7fc640 1 --2- 172.21.15.94:0/2667207263 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d81857a1d0 0x55d81857c5c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4664b548e1e1cf42 server_cookie=11f889b6e050c7fc in_seq=0 out_seq=0 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f0582ff5640 1 -- 172.21.15.94:0/2667207263 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f059009a9c0 con 0x55d81857a1d0 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f0582ff5640 10 client.26402 handle_command_reply: tid=0 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f05a7e84740 1 client.26402 shutdown 2024-02-17T12:03:27.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f05a7e84740 1 -- 172.21.15.94:0/2667207263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 msgr2=0x55d81852dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f05a7e84740 1 --2- 172.21.15.94:0/2667207263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d81852dad0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f059009a950 tx=0x7f059009a5b0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f05a7e84740 1 -- 172.21.15.94:0/2667207263 shutdown_connections 2024-02-17T12:03:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f05a7e84740 1 --2- 172.21.15.94:0/2667207263 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d81857a1d0 0x55d81857c5c0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f0582ff5640 0 client.26402 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f05a7e84740 1 --2- 172.21.15.94:0/2667207263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d81851afc0 0x55d81852c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f05a7e84740 1 --2- 172.21.15.94:0/2667207263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d818509a70 0x55d81852bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f05a7e84740 1 --2- 172.21.15.94:0/2667207263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d8185090c0 0x55d81852dad0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f05a7e84740 1 -- 172.21.15.94:0/2667207263 >> 172.21.15.94:0/2667207263 conn(0x55d818503990 msgr2=0x55d818516560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f05a7e84740 1 -- 172.21.15.94:0/2667207263 shutdown_connections 2024-02-17T12:03:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f05a7e84740 1 -- 172.21.15.94:0/2667207263 wait complete. 2024-02-17T12:03:27.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.890+0000 7f05a7e84740 20 client.26402 trim_cache size 0 max 16384 2024-02-17T12:03:27.895 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:03:27.896 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:03:27.896 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:03:27.896 INFO:teuthology.orchestra.run.smithi094.stdout: "da97004e-ccba-4fb1-8d81-e07f3c3a474b": { 2024-02-17T12:03:27.896 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:03:27.896 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "da97004e-ccba-4fb1-8d81-e07f3c3a474b", 2024-02-17T12:03:27.896 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:03:27.896 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:03:27.896 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:03:27.896 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:03:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.891+0000 7f05a52ae640 1 -- 172.21.15.94:0/2316640922 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f05880a22f0 msgr2=0x7f05880a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.891+0000 7f05a52ae640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f05880a22f0 0x7f05880a47b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f059800acd0 tx=0x7f0598009a50 comp rx=0 tx=0).stop 2024-02-17T12:03:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.891+0000 7f05a52ae640 1 -- 172.21.15.94:0/2316640922 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a00ff380 msgr2=0x7f05a019f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.891+0000 7f05a52ae640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a00ff380 0x7f05a019f660 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f059000d950 tx=0x7f059000de10 comp rx=0 tx=0).stop 2024-02-17T12:03:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.891+0000 7f05a52ae640 1 -- 172.21.15.94:0/2316640922 shutdown_connections 2024-02-17T12:03:27.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.891+0000 7f05a52ae640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f05880a22f0 0x7f05880a47b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.891+0000 7f05a52ae640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f05a01044d0 0x7f05a01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.891+0000 7f05a52ae640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05a00ffca0 0x7f05a019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.891+0000 7f05a52ae640 1 --2- 172.21.15.94:0/2316640922 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f05a00ff380 0x7f05a019f660 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:27.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.891+0000 7f05a52ae640 1 -- 172.21.15.94:0/2316640922 >> 172.21.15.94:0/2316640922 conn(0x7f05a0076210 msgr2=0x7f05a0101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:27.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.891+0000 7f05a52ae640 1 -- 172.21.15.94:0/2316640922 shutdown_connections 2024-02-17T12:03:27.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:27.891+0000 7f05a52ae640 1 -- 172.21.15.94:0/2316640922 wait complete. 2024-02-17T12:03:27.903 INFO:tasks.cephfs.filesystem:scrub status for tag:da97004e-ccba-4fb1-8d81-e07f3c3a474b - {'path': '/', 'tag': 'da97004e-ccba-4fb1-8d81-e07f3c3a474b', 'options': 'recursive,force'} 2024-02-17T12:03:28.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:27 smithi125 ceph-mon[36819]: pgmap v164: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.8 MiB/s wr, 639 op/s 2024-02-17T12:03:28.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:27 smithi094 ceph-mon[32499]: pgmap v164: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.8 MiB/s wr, 639 op/s 2024-02-17T12:03:28.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:27 smithi189 ceph-mon[36588]: pgmap v164: 97 pgs: 97 active+clean; 3.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.8 MiB/s wr, 639 op/s 2024-02-17T12:03:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.736+0000 7fc6f57e2640 1 -- 172.21.15.94:0/413840718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0101d40 msgr2=0x7fc6f0105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.736+0000 7fc6f57e2640 1 --2- 172.21.15.94:0/413840718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0101d40 0x7fc6f0105d80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009930 tx=0x7fc6e002bc30 comp rx=0 tx=0).stop 2024-02-17T12:03:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.736+0000 7fc6f57e2640 1 -- 172.21.15.94:0/413840718 shutdown_connections 2024-02-17T12:03:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.736+0000 7fc6f57e2640 1 --2- 172.21.15.94:0/413840718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6f0106460 0x7fc6f010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.736+0000 7fc6f57e2640 1 --2- 172.21.15.94:0/413840718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0101d40 0x7fc6f0105d80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.736+0000 7fc6f57e2640 1 --2- 172.21.15.94:0/413840718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6f0101390 0x7fc6f0101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.736+0000 7fc6f57e2640 1 -- 172.21.15.94:0/413840718 >> 172.21.15.94:0/413840718 conn(0x7fc6f00fcc10 msgr2=0x7fc6f00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.736+0000 7fc6f57e2640 1 -- 172.21.15.94:0/413840718 shutdown_connections 2024-02-17T12:03:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.736+0000 7fc6f57e2640 1 -- 172.21.15.94:0/413840718 wait complete. 2024-02-17T12:03:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.737+0000 7fc6f57e2640 1 Processor -- start 2024-02-17T12:03:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.737+0000 7fc6f57e2640 1 -- start start 2024-02-17T12:03:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.737+0000 7fc6f57e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0101390 0x7fc6f010d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6f57e2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6f0106460 0x7fc6f010d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6f57e2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6f01a26a0 0x7fc6f01a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6f57e2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f01a3090 con 0x7fc6f0101390 2024-02-17T12:03:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6f57e2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f01a3200 con 0x7fc6f0106460 2024-02-17T12:03:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6f57e2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f01a3370 con 0x7fc6f01a26a0 2024-02-17T12:03:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6eeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0101390 0x7fc6f010d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6eeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0101390 0x7fc6f010d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35518/0 (socket says 172.21.15.94:35518) 2024-02-17T12:03:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6eeffd640 1 -- 172.21.15.94:0/3714837900 learned_addr learned my addr 172.21.15.94:0/3714837900 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6e67fc640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6f0106460 0x7fc6f010d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6ef7fe640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6f01a26a0 0x7fc6f01a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6eeffd640 1 -- 172.21.15.94:0/3714837900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6f01a26a0 msgr2=0x7fc6f01a2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6eeffd640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6f01a26a0 0x7fc6f01a2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6eeffd640 1 -- 172.21.15.94:0/3714837900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6f0106460 msgr2=0x7fc6f010d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6eeffd640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6f0106460 0x7fc6f010d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.738+0000 7fc6eeffd640 1 -- 172.21.15.94:0/3714837900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e0009560 con 0x7fc6f0101390 2024-02-17T12:03:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.739+0000 7fc6eeffd640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0101390 0x7fc6f010d270 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800e9f0 tx=0x7fc6d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.739+0000 7fc6ecff9640 1 -- 172.21.15.94:0/3714837900 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8009700 con 0x7fc6f0101390 2024-02-17T12:03:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.739+0000 7fc6ecff9640 1 -- 172.21.15.94:0/3714837900 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6d800cd80 con 0x7fc6f0101390 2024-02-17T12:03:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.739+0000 7fc6f57e2640 1 -- 172.21.15.94:0/3714837900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f01aa930 con 0x7fc6f0101390 2024-02-17T12:03:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.739+0000 7fc6f57e2640 1 -- 172.21.15.94:0/3714837900 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f01aadb0 con 0x7fc6f0101390 2024-02-17T12:03:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.740+0000 7fc6ecff9640 1 -- 172.21.15.94:0/3714837900 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8018920 con 0x7fc6f0101390 2024-02-17T12:03:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.742+0000 7fc6ecff9640 1 -- 172.21.15.94:0/3714837900 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc6d8018b60 con 0x7fc6f0101390 2024-02-17T12:03:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.742+0000 7fc6f83bc740 1 Processor -- start 2024-02-17T12:03:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.742+0000 7fc6f83bc740 1 -- start start 2024-02-17T12:03:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.742+0000 7fc6ecff9640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6d00a2430 0x7fc6d00a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6f83bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8ead0 0x559754b8d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6f83bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8dd60 0x559754b8e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6f83bc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b972e0 0x559754b996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6f83bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559754afc9f0 con 0x559754b8dd60 2024-02-17T12:03:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6f83bc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559754af9f20 con 0x559754b972e0 2024-02-17T12:03:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6f83bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559754af5700 con 0x559754b8ead0 2024-02-17T12:03:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6ef7fe640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6d00a2430 0x7fc6d00a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6e67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8dd60 0x559754b8e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6e67fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8dd60 0x559754b8e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35524/0 (socket says 172.21.15.94:35524) 2024-02-17T12:03:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6e67fc640 1 -- 172.21.15.94:0/1666966101 learned_addr learned my addr 172.21.15.94:0/1666966101 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6ef7fe640 1 --2- 172.21.15.94:0/1666966101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b972e0 0x559754b996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6eeffd640 1 --2- 172.21.15.94:0/1666966101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8ead0 0x559754b8d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6ecff9640 1 -- 172.21.15.94:0/3714837900 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc6d8014070 con 0x7fc6f0101390 2024-02-17T12:03:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6e67fc640 1 -- 172.21.15.94:0/1666966101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8ead0 msgr2=0x559754b8d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6e67fc640 1 --2- 172.21.15.94:0/1666966101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8ead0 0x559754b8d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6e67fc640 1 -- 172.21.15.94:0/1666966101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b972e0 msgr2=0x559754b996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6e67fc640 1 --2- 172.21.15.94:0/1666966101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b972e0 0x559754b996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6e67fc640 1 -- 172.21.15.94:0/1666966101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559754a8b7b0 con 0x559754b8dd60 2024-02-17T12:03:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6ef7fe640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6d00a2430 0x7fc6d00a48f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009fd0 tx=0x7fc6e8014bd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.743+0000 7fc6e67fc640 1 --2- 172.21.15.94:0/1666966101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8dd60 0x559754b8e1e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0005ab0 tx=0x7fc6e002c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171c30a49b0a9668 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.744+0000 7fc6e57fa640 1 -- 172.21.15.94:0/1666966101 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0004800 con 0x559754b8dd60 2024-02-17T12:03:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.744+0000 7fc6e57fa640 1 -- 172.21.15.94:0/1666966101 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6e0004960 con 0x559754b8dd60 2024-02-17T12:03:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.744+0000 7fc6e57fa640 1 -- 172.21.15.94:0/1666966101 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00050e0 con 0x559754b8dd60 2024-02-17T12:03:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.746+0000 7fc6f83bc740 1 -- 172.21.15.94:0/1666966101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8dd60 msgr2=0x559754b8e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.746+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/1666966101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8dd60 0x559754b8e1e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0005ab0 tx=0x7fc6e002c680 comp rx=0 tx=0).stop 2024-02-17T12:03:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.746+0000 7fc6f83bc740 1 -- 172.21.15.94:0/1666966101 shutdown_connections 2024-02-17T12:03:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.746+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/1666966101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b972e0 0x559754b996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.746+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/1666966101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8dd60 0x559754b8e1e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.746+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/1666966101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8ead0 0x559754b8d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.746+0000 7fc6f83bc740 1 -- 172.21.15.94:0/1666966101 >> 172.21.15.94:0/1666966101 conn(0x559754b55550 msgr2=0x559754b88a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.746+0000 7fc6f83bc740 1 -- 172.21.15.94:0/1666966101 shutdown_connections 2024-02-17T12:03:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.746+0000 7fc6f83bc740 1 -- 172.21.15.94:0/1666966101 wait complete. 2024-02-17T12:03:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.746+0000 7fc6f83bc740 1 -- start start 2024-02-17T12:03:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.746+0000 7fc6f83bc740 1 Processor -- start 2024-02-17T12:03:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.747+0000 7fc6f83bc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8dd60 0x559754b942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.747+0000 7fc6f83bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8ead0 0x559754b94810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.747+0000 7fc6f83bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b972e0 0x559754b94d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.747+0000 7fc6eeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8dd60 0x559754b942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.747+0000 7fc6eeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8dd60 0x559754b942d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59848/0 (socket says 172.21.15.94:59848) 2024-02-17T12:03:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.747+0000 7fc6eeffd640 1 -- 172.21.15.94:0/629001470 learned_addr learned my addr 172.21.15.94:0/629001470 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.747+0000 7fc6f83bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559754a9abf0 con 0x559754b8ead0 2024-02-17T12:03:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.747+0000 7fc6f83bc740 1 -- 172.21.15.94:0/629001470 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597547c9250 con 0x559754b8dd60 2024-02-17T12:03:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.747+0000 7fc6f83bc740 1 -- 172.21.15.94:0/629001470 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559754a7b200 con 0x559754b972e0 2024-02-17T12:03:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.747+0000 7fc6e67fc640 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8ead0 0x559754b94810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.747+0000 7fc6f83bc740 10 client.0 _ll_register_callbacks cb 0x559754ae9d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.747+0000 7fc6ef7fe640 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b972e0 0x559754b94d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6eeffd640 1 -- 172.21.15.94:0/629001470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b972e0 msgr2=0x559754b94d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6eeffd640 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b972e0 0x559754b94d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6eeffd640 1 -- 172.21.15.94:0/629001470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8ead0 msgr2=0x559754b94810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6eeffd640 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8ead0 0x559754b94810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6eeffd640 1 -- 172.21.15.94:0/629001470 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f010e840 con 0x559754b8dd60 2024-02-17T12:03:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6e67fc640 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8ead0 0x559754b94810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6eeffd640 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8dd60 0x559754b942d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800e320 tx=0x7fc6d801d600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44b3436c17530e49 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:03:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6de7fc640 1 -- 172.21.15.94:0/629001470 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d806b2d0 con 0x559754b8dd60 2024-02-17T12:03:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6de7fc640 1 -- 172.21.15.94:0/629001470 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc6d809ad40 con 0x559754b8dd60 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6f83bc740 1 -- 172.21.15.94:0/629001470 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597547f5600 con 0x559754b8dd60 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6de7fc640 1 -- 172.21.15.94:0/629001470 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6d809ede0 con 0x559754b8dd60 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6f83bc740 10 client.26414 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.748+0000 7fc6f83bc740 1 -- 172.21.15.94:0/629001470 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559754af3d80 con 0x559754b8dd60 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.749+0000 7fc6de7fc640 1 -- 172.21.15.94:0/629001470 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5597547f5600 con 0x559754b8dd60 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.749+0000 7fc6de7fc640 1 -- 172.21.15.94:0/629001470 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc6d80a3030 con 0x559754b8dd60 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.749+0000 7fc6f83bc740 10 client.26414 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.749+0000 7fc6f83bc740 10 client.26414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.749+0000 7fc6f83bc740 20 client.26414 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.749+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559754ba7790 0x559754ba9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.749+0000 7fc6f83bc740 4 client.26414 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.749+0000 7fc6f83bc740 1 -- 172.21.15.94:0/629001470 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597548461b0 con 0x559754ba7790 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.749+0000 7fc6e67fc640 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559754ba7790 0x559754ba9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.749+0000 7fc6e67fc640 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559754ba7790 0x559754ba9bb0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445fea5c58a81f53 server_cookie=47f4b76c069a99a0 in_seq=0 out_seq=0 2024-02-17T12:03:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.749+0000 7fc6de7fc640 10 client.26414 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6de7fc640 1 -- 172.21.15.94:0/629001470 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5597548461b0 con 0x559754ba7790 2024-02-17T12:03:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6de7fc640 10 client.26414 handle_command_reply: tid=0 2024-02-17T12:03:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6f83bc740 1 client.26414 shutdown 2024-02-17T12:03:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6f83bc740 1 -- 172.21.15.94:0/629001470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8dd60 msgr2=0x559754b942d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8dd60 0x559754b942d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800e320 tx=0x7fc6d801d600 comp rx=0 tx=0).stop 2024-02-17T12:03:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6f83bc740 1 -- 172.21.15.94:0/629001470 shutdown_connections 2024-02-17T12:03:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559754ba7790 0x559754ba9bb0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b972e0 0x559754b94d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8ead0 0x559754b94810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6de7fc640 0 client.26414 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/629001470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8dd60 0x559754b942d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6f83bc740 1 -- 172.21.15.94:0/629001470 >> 172.21.15.94:0/629001470 conn(0x559754b55550 msgr2=0x559754b8a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6f83bc740 1 -- 172.21.15.94:0/629001470 shutdown_connections 2024-02-17T12:03:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.750+0000 7fc6f83bc740 1 -- 172.21.15.94:0/629001470 wait complete. 2024-02-17T12:03:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.751+0000 7fc6f83bc740 20 client.26414 trim_cache size 0 max 16384 2024-02-17T12:03:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.769+0000 7fc6f83bc740 1 Processor -- start 2024-02-17T12:03:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.769+0000 7fc6f83bc740 1 -- start start 2024-02-17T12:03:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.770+0000 7fc6f83bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8dd60 0x559754b96a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.770+0000 7fc6f83bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8ead0 0x559754b9e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.770+0000 7fc6f83bc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b972e0 0x559754b9e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.770+0000 7fc6f83bc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559754b952a0 con 0x559754b8ead0 2024-02-17T12:03:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.770+0000 7fc6f83bc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559754b8e5a0 con 0x559754b972e0 2024-02-17T12:03:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.770+0000 7fc6f83bc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559754b89a50 con 0x559754b8dd60 2024-02-17T12:03:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.771+0000 7fc6ef7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b972e0 0x559754b9e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.771+0000 7fc6ef7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b972e0 0x559754b9e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59860/0 (socket says 172.21.15.94:59860) 2024-02-17T12:03:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.771+0000 7fc6ef7fe640 1 -- 172.21.15.94:0/238933910 learned_addr learned my addr 172.21.15.94:0/238933910 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.771+0000 7fc6e67fc640 1 --2- 172.21.15.94:0/238933910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8ead0 0x559754b9e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.772+0000 7fc6ef7fe640 1 -- 172.21.15.94:0/238933910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8dd60 msgr2=0x559754b96a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:03:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.772+0000 7fc6ef7fe640 1 --2- 172.21.15.94:0/238933910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8dd60 0x559754b96a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.772+0000 7fc6ef7fe640 1 -- 172.21.15.94:0/238933910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8ead0 msgr2=0x559754b9e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.772+0000 7fc6ef7fe640 1 --2- 172.21.15.94:0/238933910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8ead0 0x559754b9e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.772+0000 7fc6ef7fe640 1 -- 172.21.15.94:0/238933910 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559754a907b0 con 0x559754b972e0 2024-02-17T12:03:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.772+0000 7fc6ef7fe640 1 --2- 172.21.15.94:0/238933910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b972e0 0x559754b9e670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80143b0 tx=0x7fc6e80096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34f0dd3e8ed7a92b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.772+0000 7fc6e4ff9640 1 -- 172.21.15.94:0/238933910 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8011b40 con 0x559754b972e0 2024-02-17T12:03:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.772+0000 7fc6e4ff9640 1 -- 172.21.15.94:0/238933910 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc6e801a070 con 0x559754b972e0 2024-02-17T12:03:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.772+0000 7fc6e4ff9640 1 -- 172.21.15.94:0/238933910 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80283a0 con 0x559754b972e0 2024-02-17T12:03:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.773+0000 7fc6f83bc740 1 -- 172.21.15.94:0/238933910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b972e0 msgr2=0x559754b9e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.773+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/238933910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b972e0 0x559754b9e670 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80143b0 tx=0x7fc6e80096a0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.773+0000 7fc6f83bc740 1 -- 172.21.15.94:0/238933910 shutdown_connections 2024-02-17T12:03:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.773+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/238933910 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b972e0 0x559754b9e670 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.773+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/238933910 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b8ead0 0x559754b9e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.773+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/238933910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8dd60 0x559754b96a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.773+0000 7fc6f83bc740 1 -- 172.21.15.94:0/238933910 >> 172.21.15.94:0/238933910 conn(0x559754b55550 msgr2=0x559754b8f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.773+0000 7fc6f83bc740 1 -- 172.21.15.94:0/238933910 shutdown_connections 2024-02-17T12:03:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.773+0000 7fc6f83bc740 1 -- 172.21.15.94:0/238933910 wait complete. 2024-02-17T12:03:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.774+0000 7fc6f83bc740 1 -- start start 2024-02-17T12:03:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.775+0000 7fc6f83bc740 1 Processor -- start 2024-02-17T12:03:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.775+0000 7fc6f83bc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8dd60 0x559754b911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.775+0000 7fc6f83bc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8ead0 0x559754b916e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.775+0000 7fc6f83bc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b972e0 0x559754b91c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.775+0000 7fc6e67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8ead0 0x559754b916e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.775+0000 7fc6e67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8ead0 0x559754b916e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:59870/0 (socket says 172.21.15.94:59870) 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.775+0000 7fc6e67fc640 1 -- 172.21.15.94:0/3302096329 learned_addr learned my addr 172.21.15.94:0/3302096329 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.775+0000 7fc6ef7fe640 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b972e0 0x559754b91c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.775+0000 7fc6eeffd640 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8dd60 0x559754b911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.775+0000 7fc6f83bc740 1 -- 172.21.15.94:0/3302096329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559754bfddb0 con 0x559754b972e0 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.775+0000 7fc6f83bc740 1 -- 172.21.15.94:0/3302096329 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559754bfdf20 con 0x559754b8ead0 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.775+0000 7fc6f83bc740 1 -- 172.21.15.94:0/3302096329 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559754bfe0e0 con 0x559754b8dd60 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6f83bc740 10 client.0 _ll_register_callbacks cb 0x7fc6e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6e67fc640 1 -- 172.21.15.94:0/3302096329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8dd60 msgr2=0x559754b911a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6e67fc640 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8dd60 0x559754b911a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6e67fc640 1 -- 172.21.15.94:0/3302096329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b972e0 msgr2=0x559754b91c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6e67fc640 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b972e0 0x559754b91c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6e67fc640 1 -- 172.21.15.94:0/3302096329 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e8014170 con 0x559754b8ead0 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6eeffd640 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8dd60 0x559754b911a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6f83bc740 1 -- 172.21.15.94:0/3302096329 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6e0004960 con 0x559754b8ead0 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6e67fc640 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8ead0 0x559754b916e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0005ab0 tx=0x7fc6e002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dafca10a9378c6e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6de7fc640 10 client.26420 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6de7fc640 1 -- 172.21.15.94:0/3302096329 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0051d70 con 0x559754b8ead0 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6de7fc640 1 -- 172.21.15.94:0/3302096329 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc6e004f070 con 0x559754b8ead0 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6de7fc640 1 -- 172.21.15.94:0/3302096329 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6e0004780 con 0x559754b8ead0 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6f83bc740 10 client.26420 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.776+0000 7fc6f83bc740 1 -- 172.21.15.94:0/3302096329 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x559754ab6540 con 0x559754b8ead0 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.777+0000 7fc6de7fc640 1 -- 172.21.15.94:0/3302096329 <== mon.2 v2:172.21.15.125:3300/0 4 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc6e003c050 con 0x559754b8ead0 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.777+0000 7fc6f83bc740 10 client.26420 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.777+0000 7fc6f83bc740 10 client.26420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.777+0000 7fc6f83bc740 20 client.26420 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.777+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559754bff2f0 0x559754c016e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.777+0000 7fc6f83bc740 4 client.26420 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:03:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.777+0000 7fc6f83bc740 1 -- 172.21.15.94:0/3302096329 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559754832550 con 0x559754bff2f0 2024-02-17T12:03:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.777+0000 7fc6eeffd640 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559754bff2f0 0x559754c016e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.778+0000 7fc6eeffd640 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559754bff2f0 0x559754c016e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f173e8021b59d41 server_cookie=ffae2ed20a5318c0 in_seq=0 out_seq=0 2024-02-17T12:03:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.779+0000 7fc6de7fc640 10 client.26420 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6de7fc640 1 -- 172.21.15.94:0/3302096329 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559754832550 con 0x559754bff2f0 2024-02-17T12:03:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6de7fc640 10 client.26420 handle_command_reply: tid=0 2024-02-17T12:03:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6f83bc740 1 client.26420 shutdown 2024-02-17T12:03:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6f83bc740 1 -- 172.21.15.94:0/3302096329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8ead0 msgr2=0x559754b916e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8ead0 0x559754b916e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0005ab0 tx=0x7fc6e002bc30 comp rx=0 tx=0).stop 2024-02-17T12:03:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6f83bc740 1 -- 172.21.15.94:0/3302096329 shutdown_connections 2024-02-17T12:03:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559754bff2f0 0x559754c016e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559754b972e0 0x559754b91c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559754b8ead0 0x559754b916e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6f83bc740 1 --2- 172.21.15.94:0/3302096329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559754b8dd60 0x559754b911a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6de7fc640 0 client.26420 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6f83bc740 1 -- 172.21.15.94:0/3302096329 >> 172.21.15.94:0/3302096329 conn(0x559754b55550 msgr2=0x559754b92b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6f83bc740 1 -- 172.21.15.94:0/3302096329 shutdown_connections 2024-02-17T12:03:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6f83bc740 1 -- 172.21.15.94:0/3302096329 wait complete. 2024-02-17T12:03:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.801+0000 7fc6f83bc740 20 client.26420 trim_cache size 0 max 16384 2024-02-17T12:03:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:03:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:03:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:03:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:03:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.802+0000 7fc6f57e2640 1 -- 172.21.15.94:0/3714837900 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6d00a2430 msgr2=0x7fc6d00a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.802+0000 7fc6f57e2640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6d00a2430 0x7fc6d00a48f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009fd0 tx=0x7fc6e8014bd0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.802+0000 7fc6f57e2640 1 -- 172.21.15.94:0/3714837900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0101390 msgr2=0x7fc6f010d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.802+0000 7fc6f57e2640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0101390 0x7fc6f010d270 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800e9f0 tx=0x7fc6d800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.802+0000 7fc6f57e2640 1 -- 172.21.15.94:0/3714837900 shutdown_connections 2024-02-17T12:03:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.802+0000 7fc6f57e2640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6d00a2430 0x7fc6d00a48f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.802+0000 7fc6f57e2640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6f01a26a0 0x7fc6f01a2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.802+0000 7fc6f57e2640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc6f0106460 0x7fc6f010d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.802+0000 7fc6f57e2640 1 --2- 172.21.15.94:0/3714837900 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc6f0101390 0x7fc6f010d270 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.802+0000 7fc6f57e2640 1 -- 172.21.15.94:0/3714837900 >> 172.21.15.94:0/3714837900 conn(0x7fc6f00fcc10 msgr2=0x7fc6f00fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.802+0000 7fc6f57e2640 1 -- 172.21.15.94:0/3714837900 shutdown_connections 2024-02-17T12:03:28.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:28.802+0000 7fc6f57e2640 1 -- 172.21.15.94:0/3714837900 wait complete. 2024-02-17T12:03:29.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:28 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:03:29.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:28 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:03:29.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:28 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:03:29.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:28 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:03:29.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:28 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:03:29.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:28 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:03:29.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:28 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:03:29.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:28 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:03:29.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:28 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:03:29.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:28 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:03:29.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:28 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:03:29.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:28 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:03:30.188 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:29 smithi125 ceph-mon[36819]: pgmap v165: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-02-17T12:03:30.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:29 smithi094 ceph-mon[32499]: pgmap v165: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-02-17T12:03:30.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 310.0 s, 2851.8 tps, lat 35.012 ms stddev 40.152 2024-02-17T12:03:30.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:29 smithi189 ceph-mon[36588]: pgmap v165: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-02-17T12:03:32.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:31 smithi125 ceph-mon[36819]: pgmap v166: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 6.8 MiB/s wr, 756 op/s 2024-02-17T12:03:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:31 smithi094 ceph-mon[32499]: pgmap v166: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 6.8 MiB/s wr, 756 op/s 2024-02-17T12:03:32.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:31 smithi189 ceph-mon[36588]: pgmap v166: 97 pgs: 97 active+clean; 3.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 6.8 MiB/s wr, 756 op/s 2024-02-17T12:03:33.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:32 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:03:33.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:32 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:03:33.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:32 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:03:34.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:33 smithi094 ceph-mon[32499]: pgmap v167: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 8.8 MiB/s wr, 948 op/s 2024-02-17T12:03:34.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:33 smithi189 ceph-mon[36588]: pgmap v167: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 8.8 MiB/s wr, 948 op/s 2024-02-17T12:03:34.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:33 smithi125 ceph-mon[36819]: pgmap v167: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 8.8 MiB/s wr, 948 op/s 2024-02-17T12:03:35.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:34 smithi094 ceph-mon[32499]: pgmap v168: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 7.4 MiB/s wr, 859 op/s 2024-02-17T12:03:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 315.0 s, 2950.4 tps, lat 33.959 ms stddev 36.518 2024-02-17T12:03:35.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:34 smithi189 ceph-mon[36588]: pgmap v168: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 7.4 MiB/s wr, 859 op/s 2024-02-17T12:03:35.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:34 smithi125 ceph-mon[36819]: pgmap v168: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 7.4 MiB/s wr, 859 op/s 2024-02-17T12:03:38.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:37 smithi125 ceph-mon[36819]: pgmap v169: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.2 MiB/s wr, 723 op/s 2024-02-17T12:03:38.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:37 smithi094 ceph-mon[32499]: pgmap v169: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.2 MiB/s wr, 723 op/s 2024-02-17T12:03:38.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:37 smithi189 ceph-mon[36588]: pgmap v169: 97 pgs: 97 active+clean; 3.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.2 MiB/s wr, 723 op/s 2024-02-17T12:03:40.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:39 smithi125 ceph-mon[36819]: pgmap v170: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 9.0 MiB/s wr, 1.02k op/s 2024-02-17T12:03:40.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:39 smithi094 ceph-mon[32499]: pgmap v170: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 9.0 MiB/s wr, 1.02k op/s 2024-02-17T12:03:40.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 320.0 s, 3036.8 tps, lat 32.909 ms stddev 36.889 2024-02-17T12:03:40.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:39 smithi189 ceph-mon[36588]: pgmap v170: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 9.0 MiB/s wr, 1.02k op/s 2024-02-17T12:03:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.937+0000 7f803754f640 1 -- 172.21.15.94:0/2609135986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803011a960 msgr2=0x7f803011cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.937+0000 7f803754f640 1 --2- 172.21.15.94:0/2609135986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803011a960 0x7f803011cd50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8028009ca0 tx=0x7f802802c140 comp rx=0 tx=0).stop 2024-02-17T12:03:41.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.937+0000 7f803754f640 1 -- 172.21.15.94:0/2609135986 shutdown_connections 2024-02-17T12:03:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.937+0000 7f803754f640 1 --2- 172.21.15.94:0/2609135986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803011a960 0x7f803011cd50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.937+0000 7f803754f640 1 --2- 172.21.15.94:0/2609135986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f803010fd30 0x7f803010da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.937+0000 7f803754f640 1 --2- 172.21.15.94:0/2609135986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803010f8c0 0x7f803010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.937+0000 7f803754f640 1 -- 172.21.15.94:0/2609135986 >> 172.21.15.94:0/2609135986 conn(0x7f803006c220 msgr2=0x7f803006c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.937+0000 7f803754f640 1 -- 172.21.15.94:0/2609135986 shutdown_connections 2024-02-17T12:03:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.937+0000 7f803754f640 1 -- 172.21.15.94:0/2609135986 wait complete. 2024-02-17T12:03:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.938+0000 7f803754f640 1 Processor -- start 2024-02-17T12:03:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.938+0000 7f803754f640 1 -- start start 2024-02-17T12:03:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.938+0000 7f803754f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803010f8c0 0x7f803011a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.938+0000 7f803754f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803010fd30 0x7f8030114470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.938+0000 7f803754f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f803011a960 0x7f8030114be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f803754f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80301bb110 con 0x7f803011a960 2024-02-17T12:03:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f803754f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80301bb250 con 0x7f803010f8c0 2024-02-17T12:03:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f803754f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80301bb390 con 0x7f803010fd30 2024-02-17T12:03:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f80352eb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803010f8c0 0x7f803011a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f80352eb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803010f8c0 0x7f803011a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:40164/0 (socket says 172.21.15.94:40164) 2024-02-17T12:03:41.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f80352eb640 1 -- 172.21.15.94:0/2186365718 learned_addr learned my addr 172.21.15.94:0/2186365718 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f8035aec640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f803011a960 0x7f8030114be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f8034aea640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803010fd30 0x7f8030114470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f80352eb640 1 -- 172.21.15.94:0/2186365718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803010fd30 msgr2=0x7f8030114470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f80352eb640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803010fd30 0x7f8030114470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f80352eb640 1 -- 172.21.15.94:0/2186365718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f803011a960 msgr2=0x7f8030114be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f80352eb640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f803011a960 0x7f8030114be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f80352eb640 1 -- 172.21.15.94:0/2186365718 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8028009900 con 0x7f803010f8c0 2024-02-17T12:03:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.939+0000 7f80352eb640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803010f8c0 0x7f803011a420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f802c00b480 tx=0x7f802c00b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.940+0000 7f80267fc640 1 -- 172.21.15.94:0/2186365718 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c010870 con 0x7f803010f8c0 2024-02-17T12:03:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.940+0000 7f80267fc640 1 -- 172.21.15.94:0/2186365718 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f802c00f3c0 con 0x7f803010f8c0 2024-02-17T12:03:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.940+0000 7f803754f640 1 -- 172.21.15.94:0/2186365718 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80301bb5f0 con 0x7f803010f8c0 2024-02-17T12:03:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.940+0000 7f80267fc640 1 -- 172.21.15.94:0/2186365718 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c010870 con 0x7f803010f8c0 2024-02-17T12:03:41.942 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.940+0000 7f803754f640 1 -- 172.21.15.94:0/2186365718 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80301bba80 con 0x7f803010f8c0 2024-02-17T12:03:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.943+0000 7f80267fc640 1 -- 172.21.15.94:0/2186365718 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f802c00d360 con 0x7f803010f8c0 2024-02-17T12:03:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.943+0000 7f80267fc640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f801c0a2330 0x7f801c0a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.943+0000 7f80267fc640 1 -- 172.21.15.94:0/2186365718 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f802c0cd620 con 0x7f803010f8c0 2024-02-17T12:03:41.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.943+0000 7f8034aea640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f801c0a2330 0x7f801c0a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.944+0000 7f8034aea640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f801c0a2330 0x7f801c0a47f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8030115c50 tx=0x7f80200090e0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.944+0000 7f803a202740 1 Processor -- start 2024-02-17T12:03:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.944+0000 7f803a202740 1 -- start start 2024-02-17T12:03:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.944+0000 7f803a202740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8d4ad0 0x5645ca8d3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.944+0000 7f803a202740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d3d60 0x5645ca8d41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.944+0000 7f803a202740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8dd2e0 0x5645ca8df6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.944+0000 7f803a202740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645ca8429f0 con 0x5645ca8d3d60 2024-02-17T12:03:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.944+0000 7f803a202740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645ca83ff20 con 0x5645ca8d4ad0 2024-02-17T12:03:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.944+0000 7f803a202740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645ca83b700 con 0x5645ca8dd2e0 2024-02-17T12:03:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.944+0000 7f8034aea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d3d60 0x5645ca8d41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.944+0000 7f8034aea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d3d60 0x5645ca8d41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36306/0 (socket says 172.21.15.94:36306) 2024-02-17T12:03:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.944+0000 7f8034aea640 1 -- 172.21.15.94:0/2388554593 learned_addr learned my addr 172.21.15.94:0/2388554593 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.945+0000 7f80352eb640 1 --2- 172.21.15.94:0/2388554593 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8d4ad0 0x5645ca8d3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.945+0000 7f8035aec640 1 --2- 172.21.15.94:0/2388554593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8dd2e0 0x5645ca8df6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.945+0000 7f80352eb640 1 -- 172.21.15.94:0/2388554593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8dd2e0 msgr2=0x5645ca8df6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.945+0000 7f80352eb640 1 --2- 172.21.15.94:0/2388554593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8dd2e0 0x5645ca8df6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.945+0000 7f80352eb640 1 -- 172.21.15.94:0/2388554593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d3d60 msgr2=0x5645ca8d41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.945+0000 7f80352eb640 1 --2- 172.21.15.94:0/2388554593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d3d60 0x5645ca8d41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.945+0000 7f80352eb640 1 -- 172.21.15.94:0/2388554593 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645ca7d17b0 con 0x5645ca8d4ad0 2024-02-17T12:03:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.945+0000 7f80352eb640 1 --2- 172.21.15.94:0/2388554593 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8d4ad0 0x5645ca8d3820 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f802c0cae70 tx=0x7f802c09b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bf8d7ba4c54e4e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.945+0000 7f801b7fe640 1 -- 172.21.15.94:0/2388554593 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c09f120 con 0x5645ca8d4ad0 2024-02-17T12:03:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.945+0000 7f801b7fe640 1 -- 172.21.15.94:0/2388554593 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f802c0a38e0 con 0x5645ca8d4ad0 2024-02-17T12:03:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.945+0000 7f801b7fe640 1 -- 172.21.15.94:0/2388554593 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c00f730 con 0x5645ca8d4ad0 2024-02-17T12:03:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.946+0000 7f803a202740 1 -- 172.21.15.94:0/2388554593 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8d4ad0 msgr2=0x5645ca8d3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.946+0000 7f803a202740 1 --2- 172.21.15.94:0/2388554593 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8d4ad0 0x5645ca8d3820 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f802c0cae70 tx=0x7f802c09b8d0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.946+0000 7f803a202740 1 -- 172.21.15.94:0/2388554593 shutdown_connections 2024-02-17T12:03:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.946+0000 7f803a202740 1 --2- 172.21.15.94:0/2388554593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8dd2e0 0x5645ca8df6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.946+0000 7f803a202740 1 --2- 172.21.15.94:0/2388554593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d3d60 0x5645ca8d41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.946+0000 7f803a202740 1 --2- 172.21.15.94:0/2388554593 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8d4ad0 0x5645ca8d3820 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.946+0000 7f803a202740 1 -- 172.21.15.94:0/2388554593 >> 172.21.15.94:0/2388554593 conn(0x5645ca89b550 msgr2=0x5645ca8cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.946+0000 7f803a202740 1 -- 172.21.15.94:0/2388554593 shutdown_connections 2024-02-17T12:03:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.947+0000 7f803a202740 1 -- 172.21.15.94:0/2388554593 wait complete. 2024-02-17T12:03:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.947+0000 7f803a202740 1 -- start start 2024-02-17T12:03:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.948+0000 7f803a202740 1 Processor -- start 2024-02-17T12:03:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.948+0000 7f803a202740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8da300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.948+0000 7f803a202740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d4ad0 0x5645ca8da840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.948+0000 7f803a202740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8dd2e0 0x5645ca8dad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f803a202740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645ca7e0bf0 con 0x5645ca8d4ad0 2024-02-17T12:03:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f803a202740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645ca50f250 con 0x5645ca8dd2e0 2024-02-17T12:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f803a202740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645ca7c1200 con 0x5645ca8d3d60 2024-02-17T12:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f803a202740 10 client.0 _ll_register_callbacks cb 0x5645ca82fd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f8035aec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8dd2e0 0x5645ca8dad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f8035aec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8dd2e0 0x5645ca8dad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:40188/0 (socket says 172.21.15.94:40188) 2024-02-17T12:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f8035aec640 1 -- 172.21.15.94:0/3248139709 learned_addr learned my addr 172.21.15.94:0/3248139709 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f80352eb640 1 --2- 172.21.15.94:0/3248139709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8da300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f8035aec640 1 -- 172.21.15.94:0/3248139709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 msgr2=0x5645ca8da300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f8035aec640 1 --2- 172.21.15.94:0/3248139709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8da300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f8035aec640 1 -- 172.21.15.94:0/3248139709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d4ad0 msgr2=0x5645ca8da840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f8035aec640 1 --2- 172.21.15.94:0/3248139709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d4ad0 0x5645ca8da840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f8035aec640 1 -- 172.21.15.94:0/3248139709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80301bba80 con 0x5645ca8dd2e0 2024-02-17T12:03:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f8035aec640 1 --2- 172.21.15.94:0/3248139709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8dd2e0 0x5645ca8dad80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8028009ca0 tx=0x7f8028032940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57a6b3e4a43f8fb4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.949+0000 7f8018ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:03:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.950+0000 7f8018ff9640 1 -- 172.21.15.94:0/3248139709 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028034850 con 0x5645ca8dd2e0 2024-02-17T12:03:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.950+0000 7f8018ff9640 1 -- 172.21.15.94:0/3248139709 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8028003970 con 0x5645ca8dd2e0 2024-02-17T12:03:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.950+0000 7f803a202740 1 -- 172.21.15.94:0/3248139709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5645ca53b600 con 0x5645ca8dd2e0 2024-02-17T12:03:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.950+0000 7f8018ff9640 1 -- 172.21.15.94:0/3248139709 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8028003b70 con 0x5645ca8dd2e0 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.950+0000 7f803a202740 10 client.26432 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.950+0000 7f803a202740 1 -- 172.21.15.94:0/3248139709 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5645ca839d80 con 0x5645ca8dd2e0 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.950+0000 7f8018ff9640 1 -- 172.21.15.94:0/3248139709 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5645ca53b600 con 0x5645ca8dd2e0 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.951+0000 7f8018ff9640 1 -- 172.21.15.94:0/3248139709 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f802803f450 con 0x5645ca8dd2e0 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.951+0000 7f803a202740 10 client.26432 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.951+0000 7f803a202740 10 client.26432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.951+0000 7f803a202740 20 client.26432 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.951+0000 7f803a202740 1 --2- 172.21.15.94:0/3248139709 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5645ca8ed600 0x5645ca8efa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.951+0000 7f803a202740 4 client.26432 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.951+0000 7f803a202740 1 -- 172.21.15.94:0/3248139709 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5645ca58c1b0 con 0x5645ca8ed600 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.951+0000 7f80352eb640 1 --2- 172.21.15.94:0/3248139709 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5645ca8ed600 0x5645ca8efa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.951+0000 7f8018ff9640 10 client.26432 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.951+0000 7f80352eb640 1 --2- 172.21.15.94:0/3248139709 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5645ca8ed600 0x5645ca8efa20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c91c2e320dcc4f3b server_cookie=b98009caa4088126 in_seq=0 out_seq=0 2024-02-17T12:03:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.952+0000 7f8018ff9640 1 -- 172.21.15.94:0/3248139709 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5645ca58c1b0 con 0x5645ca8ed600 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.952+0000 7f8018ff9640 10 client.26432 handle_command_reply: tid=0 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.952+0000 7f803a202740 1 client.26432 shutdown 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.952+0000 7f803a202740 1 -- 172.21.15.94:0/3248139709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8dd2e0 msgr2=0x5645ca8dad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.952+0000 7f803a202740 1 --2- 172.21.15.94:0/3248139709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8dd2e0 0x5645ca8dad80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8028009ca0 tx=0x7f8028032940 comp rx=0 tx=0).stop 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.952+0000 7f803a202740 1 -- 172.21.15.94:0/3248139709 shutdown_connections 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.952+0000 7f803a202740 1 --2- 172.21.15.94:0/3248139709 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5645ca8ed600 0x5645ca8efa20 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.952+0000 7f803a202740 1 --2- 172.21.15.94:0/3248139709 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8dd2e0 0x5645ca8dad80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.952+0000 7f803a202740 1 --2- 172.21.15.94:0/3248139709 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d4ad0 0x5645ca8da840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.952+0000 7f803a202740 1 --2- 172.21.15.94:0/3248139709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8da300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.952+0000 7f8018ff9640 0 client.26432 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.952+0000 7f803a202740 1 -- 172.21.15.94:0/3248139709 >> 172.21.15.94:0/3248139709 conn(0x5645ca89b550 msgr2=0x5645ca8d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.953+0000 7f803a202740 1 -- 172.21.15.94:0/3248139709 shutdown_connections 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.953+0000 7f803a202740 1 -- 172.21.15.94:0/3248139709 wait complete. 2024-02-17T12:03:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.953+0000 7f803a202740 20 client.26432 trim_cache size 0 max 16384 2024-02-17T12:03:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.971+0000 7f803a202740 1 Processor -- start 2024-02-17T12:03:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.971+0000 7f803a202740 1 -- start start 2024-02-17T12:03:41.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.971+0000 7f803a202740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8dca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.971+0000 7f803a202740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d4ad0 0x5645ca8daba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.971+0000 7f803a202740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8dd2e0 0x5645ca8e40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.971+0000 7f803a202740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645ca8d50c0 con 0x5645ca8d4ad0 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f803a202740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645ca8ea0f0 con 0x5645ca8dd2e0 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f803a202740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645ca8cfcb0 con 0x5645ca8d3d60 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f8034aea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d4ad0 0x5645ca8daba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f80352eb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8dca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f80352eb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8dca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:59966/0 (socket says 172.21.15.94:59966) 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f80352eb640 1 -- 172.21.15.94:0/1664196657 learned_addr learned my addr 172.21.15.94:0/1664196657 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f8034aea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d4ad0 0x5645ca8daba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36332/0 (socket says 172.21.15.94:36332) 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f8035aec640 1 --2- 172.21.15.94:0/1664196657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8dd2e0 0x5645ca8e40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f80352eb640 1 -- 172.21.15.94:0/1664196657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8dd2e0 msgr2=0x5645ca8e40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f80352eb640 1 --2- 172.21.15.94:0/1664196657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8dd2e0 0x5645ca8e40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f80352eb640 1 -- 172.21.15.94:0/1664196657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d4ad0 msgr2=0x5645ca8daba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f80352eb640 1 --2- 172.21.15.94:0/1664196657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d4ad0 0x5645ca8daba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f80352eb640 1 -- 172.21.15.94:0/1664196657 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645ca7d67b0 con 0x5645ca8d3d60 2024-02-17T12:03:41.974 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.972+0000 7f8034aea640 1 --2- 172.21.15.94:0/1664196657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d4ad0 0x5645ca8daba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.973+0000 7f80352eb640 1 --2- 172.21.15.94:0/1664196657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8dca30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f802c0caf20 tx=0x7f802c00fd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2b1e01f8d1d9bae server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.974+0000 7f801affd640 1 -- 172.21.15.94:0/1664196657 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c0b7070 con 0x5645ca8d3d60 2024-02-17T12:03:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.974+0000 7f801affd640 1 -- 172.21.15.94:0/1664196657 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f802c097840 con 0x5645ca8d3d60 2024-02-17T12:03:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.974+0000 7f801affd640 1 -- 172.21.15.94:0/1664196657 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c0caf50 con 0x5645ca8d3d60 2024-02-17T12:03:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.974+0000 7f803a202740 1 -- 172.21.15.94:0/1664196657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 msgr2=0x5645ca8dca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:41.975 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.974+0000 7f803a202740 1 --2- 172.21.15.94:0/1664196657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8dca30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f802c0caf20 tx=0x7f802c00fd00 comp rx=0 tx=0).stop 2024-02-17T12:03:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.975+0000 7f803a202740 1 -- 172.21.15.94:0/1664196657 shutdown_connections 2024-02-17T12:03:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.975+0000 7f803a202740 1 --2- 172.21.15.94:0/1664196657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8dd2e0 0x5645ca8e40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.975+0000 7f803a202740 1 --2- 172.21.15.94:0/1664196657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8d4ad0 0x5645ca8daba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.975+0000 7f803a202740 1 --2- 172.21.15.94:0/1664196657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8dca30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.975+0000 7f803a202740 1 -- 172.21.15.94:0/1664196657 >> 172.21.15.94:0/1664196657 conn(0x5645ca89b550 msgr2=0x5645ca8ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.975+0000 7f803a202740 1 -- 172.21.15.94:0/1664196657 shutdown_connections 2024-02-17T12:03:41.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.975+0000 7f803a202740 1 -- 172.21.15.94:0/1664196657 wait complete. 2024-02-17T12:03:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.976+0000 7f803a202740 1 -- start start 2024-02-17T12:03:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.976+0000 7f803a202740 1 Processor -- start 2024-02-17T12:03:41.979 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.976+0000 7f803a202740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.976+0000 7f803a202740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8d4ad0 0x5645ca8e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.976+0000 7f803a202740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8dd2e0 0x5645ca8d6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.976+0000 7f803a202740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645ca8d7120 con 0x5645ca8dd2e0 2024-02-17T12:03:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.976+0000 7f803a202740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645ca8d7290 con 0x5645ca8d4ad0 2024-02-17T12:03:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.976+0000 7f803a202740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645ca8d7450 con 0x5645ca8d3d60 2024-02-17T12:03:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8034aea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8d4ad0 0x5645ca8e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8034aea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8d4ad0 0x5645ca8e45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:40200/0 (socket says 172.21.15.94:40200) 2024-02-17T12:03:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8034aea640 1 -- 172.21.15.94:0/3644648174 learned_addr learned my addr 172.21.15.94:0/3644648174 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f80352eb640 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8035aec640 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8dd2e0 0x5645ca8d6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8034aea640 1 -- 172.21.15.94:0/3644648174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 msgr2=0x5645ca8e40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8034aea640 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8e40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8034aea640 1 -- 172.21.15.94:0/3644648174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8dd2e0 msgr2=0x5645ca8d6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8034aea640 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8dd2e0 0x5645ca8d6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f803a202740 10 client.0 _ll_register_callbacks cb 0x7f802c097470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8034aea640 1 -- 172.21.15.94:0/3644648174 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802001a040 con 0x5645ca8d4ad0 2024-02-17T12:03:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8035aec640 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8dd2e0 0x5645ca8d6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f803a202740 1 -- 172.21.15.94:0/3644648174 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f802c0a38e0 con 0x5645ca8d4ad0 2024-02-17T12:03:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8034aea640 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8d4ad0 0x5645ca8e45f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5645ca8d9b60 tx=0x7f8020015b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=accacc4239278f82 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8018ff9640 10 client.26438 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:03:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8018ff9640 1 -- 172.21.15.94:0/3644648174 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020015e40 con 0x5645ca8d4ad0 2024-02-17T12:03:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8018ff9640 1 -- 172.21.15.94:0/3644648174 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f8020013b30 con 0x5645ca8d4ad0 2024-02-17T12:03:41.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.977+0000 7f8018ff9640 1 -- 172.21.15.94:0/3644648174 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8020013d10 con 0x5645ca8d4ad0 2024-02-17T12:03:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.978+0000 7f803a202740 10 client.26438 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.978+0000 7f803a202740 1 -- 172.21.15.94:0/3644648174 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x5645ca7d17b0 con 0x5645ca8d4ad0 2024-02-17T12:03:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.978+0000 7f8018ff9640 1 -- 172.21.15.94:0/3644648174 <== mon.2 v2:172.21.15.125:3300/0 4 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8020011070 con 0x5645ca8d4ad0 2024-02-17T12:03:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.978+0000 7f803a202740 10 client.26438 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.978+0000 7f803a202740 10 client.26438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.978+0000 7f803a202740 20 client.26438 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.978+0000 7f803a202740 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5645ca944ca0 0x5645ca947090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.978+0000 7f803a202740 4 client.26438 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:03:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.978+0000 7f803a202740 1 -- 172.21.15.94:0/3644648174 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5645ca578550 con 0x5645ca944ca0 2024-02-17T12:03:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.978+0000 7f80352eb640 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5645ca944ca0 0x5645ca947090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.978+0000 7f80352eb640 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5645ca944ca0 0x5645ca947090 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da49f628f25b9340 server_cookie=b3429b5a66c2fcba in_seq=0 out_seq=0 2024-02-17T12:03:41.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.978+0000 7f8018ff9640 10 client.26438 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.991+0000 7f8018ff9640 1 -- 172.21.15.94:0/3644648174 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5645ca578550 con 0x5645ca944ca0 2024-02-17T12:03:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.991+0000 7f8018ff9640 10 client.26438 handle_command_reply: tid=0 2024-02-17T12:03:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.992+0000 7f803a202740 1 client.26438 shutdown 2024-02-17T12:03:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.992+0000 7f803a202740 1 -- 172.21.15.94:0/3644648174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8d4ad0 msgr2=0x5645ca8e45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.992+0000 7f803a202740 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8d4ad0 0x5645ca8e45f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x5645ca8d9b60 tx=0x7f8020015b90 comp rx=0 tx=0).stop 2024-02-17T12:03:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.992+0000 7f803a202740 1 -- 172.21.15.94:0/3644648174 shutdown_connections 2024-02-17T12:03:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.992+0000 7f803a202740 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5645ca944ca0 0x5645ca947090 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.992+0000 7f8018ff9640 0 client.26438 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.992+0000 7f803a202740 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5645ca8dd2e0 0x5645ca8d6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.992+0000 7f803a202740 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5645ca8d4ad0 0x5645ca8e45f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.992+0000 7f803a202740 1 --2- 172.21.15.94:0/3644648174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5645ca8d3d60 0x5645ca8e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.992+0000 7f803a202740 1 -- 172.21.15.94:0/3644648174 >> 172.21.15.94:0/3644648174 conn(0x5645ca89b550 msgr2=0x5645ca8d8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803a202740 1 -- 172.21.15.94:0/3644648174 shutdown_connections 2024-02-17T12:03:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803a202740 1 -- 172.21.15.94:0/3644648174 wait complete. 2024-02-17T12:03:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803a202740 20 client.26438 trim_cache size 0 max 16384 2024-02-17T12:03:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:03:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:03:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:03:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:03:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803754f640 1 -- 172.21.15.94:0/2186365718 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f801c0a2330 msgr2=0x7f801c0a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803754f640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f801c0a2330 0x7f801c0a47f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8030115c50 tx=0x7f80200090e0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803754f640 1 -- 172.21.15.94:0/2186365718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803010f8c0 msgr2=0x7f803011a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803754f640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803010f8c0 0x7f803011a420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f802c00b480 tx=0x7f802c00b940 comp rx=0 tx=0).stop 2024-02-17T12:03:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803754f640 1 -- 172.21.15.94:0/2186365718 shutdown_connections 2024-02-17T12:03:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803754f640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f801c0a2330 0x7f801c0a47f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803754f640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f803011a960 0x7f8030114be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803754f640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803010fd30 0x7f8030114470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803754f640 1 --2- 172.21.15.94:0/2186365718 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f803010f8c0 0x7f803011a420 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803754f640 1 -- 172.21.15.94:0/2186365718 >> 172.21.15.94:0/2186365718 conn(0x7f803006c220 msgr2=0x7f803011b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803754f640 1 -- 172.21.15.94:0/2186365718 shutdown_connections 2024-02-17T12:03:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:41.993+0000 7f803754f640 1 -- 172.21.15.94:0/2186365718 wait complete. 2024-02-17T12:03:42.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:41 smithi125 ceph-mon[36819]: pgmap v171: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 599 op/s 2024-02-17T12:03:42.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:41 smithi094 ceph-mon[32499]: pgmap v171: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 599 op/s 2024-02-17T12:03:42.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:41 smithi189 ceph-mon[36588]: pgmap v171: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.8 MiB/s wr, 599 op/s 2024-02-17T12:03:44.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:43 smithi125 ceph-mon[36819]: pgmap v172: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 637 op/s 2024-02-17T12:03:44.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:43 smithi094 ceph-mon[32499]: pgmap v172: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 637 op/s 2024-02-17T12:03:44.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:43 smithi189 ceph-mon[36588]: pgmap v172: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 6.1 MiB/s wr, 637 op/s 2024-02-17T12:03:45.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 325.0 s, 2714.8 tps, lat 36.809 ms stddev 40.559 2024-02-17T12:03:46.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:45 smithi125 ceph-mon[36819]: pgmap v173: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 707 op/s 2024-02-17T12:03:46.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:45 smithi094 ceph-mon[32499]: pgmap v173: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 707 op/s 2024-02-17T12:03:46.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:45 smithi189 ceph-mon[36588]: pgmap v173: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 707 op/s 2024-02-17T12:03:48.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:47 smithi125 ceph-mon[36819]: pgmap v174: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 MiB/s wr, 596 op/s 2024-02-17T12:03:48.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:47 smithi094 ceph-mon[32499]: pgmap v174: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 MiB/s wr, 596 op/s 2024-02-17T12:03:48.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:47 smithi189 ceph-mon[36588]: pgmap v174: 97 pgs: 97 active+clean; 3.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 MiB/s wr, 596 op/s 2024-02-17T12:03:50.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:49 smithi125 ceph-mon[36819]: pgmap v175: 97 pgs: 97 active+clean; 3.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.5 MiB/s wr, 884 op/s 2024-02-17T12:03:50.203 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:49 smithi189 ceph-mon[36588]: pgmap v175: 97 pgs: 97 active+clean; 3.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.5 MiB/s wr, 884 op/s 2024-02-17T12:03:50.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:49 smithi094 ceph-mon[32499]: pgmap v175: 97 pgs: 97 active+clean; 3.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.5 MiB/s wr, 884 op/s 2024-02-17T12:03:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 330.0 s, 2853.0 tps, lat 35.004 ms stddev 39.207 2024-02-17T12:03:52.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:51 smithi094 ceph-mon[32499]: pgmap v176: 97 pgs: 97 active+clean; 3.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-02-17T12:03:52.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:51 smithi189 ceph-mon[36588]: pgmap v176: 97 pgs: 97 active+clean; 3.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-02-17T12:03:52.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:51 smithi125 ceph-mon[36819]: pgmap v176: 97 pgs: 97 active+clean; 3.8 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-02-17T12:03:54.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:53 smithi094 ceph-mon[32499]: pgmap v177: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.2 MiB/s wr, 633 op/s 2024-02-17T12:03:54.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:53 smithi189 ceph-mon[36588]: pgmap v177: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.2 MiB/s wr, 633 op/s 2024-02-17T12:03:54.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:53 smithi125 ceph-mon[36819]: pgmap v177: 97 pgs: 97 active+clean; 3.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 6.2 MiB/s wr, 633 op/s 2024-02-17T12:03:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.113+0000 7f1028ec9640 1 -- 172.21.15.94:0/4081073842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10240691a0 msgr2=0x7f102410e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.113+0000 7f1028ec9640 1 --2- 172.21.15.94:0/4081073842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10240691a0 0x7f102410e010 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f101c009ca0 tx=0x7f101c02c180 comp rx=0 tx=0).stop 2024-02-17T12:03:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.113+0000 7f1028ec9640 1 -- 172.21.15.94:0/4081073842 shutdown_connections 2024-02-17T12:03:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.113+0000 7f1028ec9640 1 --2- 172.21.15.94:0/4081073842 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f10240691a0 0x7f102410e010 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.113+0000 7f1028ec9640 1 --2- 172.21.15.94:0/4081073842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1024068800 0x7f1024068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.113+0000 7f1028ec9640 1 --2- 172.21.15.94:0/4081073842 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1024107270 0x7f1024107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.114+0000 7f1028ec9640 1 -- 172.21.15.94:0/4081073842 >> 172.21.15.94:0/4081073842 conn(0x7f1024076ed0 msgr2=0x7f10240772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.114+0000 7f1028ec9640 1 -- 172.21.15.94:0/4081073842 shutdown_connections 2024-02-17T12:03:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.114+0000 7f1028ec9640 1 -- 172.21.15.94:0/4081073842 wait complete. 2024-02-17T12:03:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.115+0000 7f1028ec9640 1 Processor -- start 2024-02-17T12:03:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.115+0000 7f1028ec9640 1 -- start start 2024-02-17T12:03:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.115+0000 7f1028ec9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1024068800 0x7f1024072910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.115+0000 7f1028ec9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10240691a0 0x7f1024072e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.115+0000 7f1028ec9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1024107270 0x7f102406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.115+0000 7f1028ec9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102406d370 con 0x7f1024068800 2024-02-17T12:03:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.115+0000 7f1028ec9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102406d4e0 con 0x7f10240691a0 2024-02-17T12:03:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.115+0000 7f1028ec9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102406d6a0 con 0x7f1024107270 2024-02-17T12:03:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.115+0000 7f102259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1024068800 0x7f1024072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.115+0000 7f102259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1024068800 0x7f1024072910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47876/0 (socket says 172.21.15.94:47876) 2024-02-17T12:03:55.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.115+0000 7f102259c640 1 -- 172.21.15.94:0/2044516740 learned_addr learned my addr 172.21.15.94:0/2044516740 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.115+0000 7f1022d9d640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1024107270 0x7f102406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.115+0000 7f1021d9b640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10240691a0 0x7f1024072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.116+0000 7f102259c640 1 -- 172.21.15.94:0/2044516740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1024107270 msgr2=0x7f102406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.116+0000 7f102259c640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1024107270 0x7f102406cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.116+0000 7f102259c640 1 -- 172.21.15.94:0/2044516740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10240691a0 msgr2=0x7f1024072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.116+0000 7f102259c640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10240691a0 0x7f1024072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.116+0000 7f102259c640 1 -- 172.21.15.94:0/2044516740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c009900 con 0x7f1024068800 2024-02-17T12:03:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.116+0000 7f1022d9d640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1024107270 0x7f102406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.116+0000 7f102259c640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1024068800 0x7f1024072910 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f101400e9f0 tx=0x7f101400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.116+0000 7f10137fe640 1 -- 172.21.15.94:0/2044516740 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014009700 con 0x7f1024068800 2024-02-17T12:03:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.116+0000 7f1028ec9640 1 -- 172.21.15.94:0/2044516740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10241ac700 con 0x7f1024068800 2024-02-17T12:03:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.116+0000 7f10137fe640 1 -- 172.21.15.94:0/2044516740 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f101400ccc0 con 0x7f1024068800 2024-02-17T12:03:55.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.116+0000 7f10137fe640 1 -- 172.21.15.94:0/2044516740 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014009700 con 0x7f1024068800 2024-02-17T12:03:55.119 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.116+0000 7f1028ec9640 1 -- 172.21.15.94:0/2044516740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10241acbb0 con 0x7f1024068800 2024-02-17T12:03:55.120 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.120+0000 7f102bb7a740 1 Processor -- start 2024-02-17T12:03:55.122 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.122+0000 7f10137fe640 1 -- 172.21.15.94:0/2044516740 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1014010a30 con 0x7f1024068800 2024-02-17T12:03:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.123+0000 7f10137fe640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f100c0a2400 0x7f100c0a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.123+0000 7f10137fe640 1 -- 172.21.15.94:0/2044516740 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1014014070 con 0x7f1024068800 2024-02-17T12:03:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.124+0000 7f102bb7a740 1 -- start start 2024-02-17T12:03:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.124+0000 7f102bb7a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40c94e0 0x55aff40c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.124+0000 7f102bb7a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40b9f40 0x55aff40ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.124+0000 7f102bb7a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40ba900 0x55aff40d1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.124+0000 7f102bb7a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aff402b9f0 con 0x55aff40ba900 2024-02-17T12:03:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.124+0000 7f102bb7a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aff4028f20 con 0x55aff40c94e0 2024-02-17T12:03:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.124+0000 7f102bb7a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aff4024700 con 0x55aff40b9f40 2024-02-17T12:03:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.124+0000 7f1021d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40b9f40 0x55aff40ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.125+0000 7f1021d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40b9f40 0x55aff40ba3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33486/0 (socket says 172.21.15.94:33486) 2024-02-17T12:03:55.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.125+0000 7f1021d9b640 1 -- 172.21.15.94:0/2418118519 learned_addr learned my addr 172.21.15.94:0/2418118519 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:55.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.127+0000 7f1022d9d640 1 --2- 172.21.15.94:0/2418118519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40ba900 0x55aff40d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.129+0000 7f1022d9d640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f100c0a2400 0x7f100c0a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.129+0000 7f102259c640 1 --2- 172.21.15.94:0/2418118519 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40c94e0 0x55aff40c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.130+0000 7f1022d9d640 1 -- 172.21.15.94:0/2418118519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40b9f40 msgr2=0x55aff40ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.130+0000 7f1022d9d640 1 --2- 172.21.15.94:0/2418118519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40b9f40 0x55aff40ba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.130+0000 7f1022d9d640 1 -- 172.21.15.94:0/2418118519 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40c94e0 msgr2=0x55aff40c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.130+0000 7f1022d9d640 1 --2- 172.21.15.94:0/2418118519 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40c94e0 0x55aff40c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.130+0000 7f1022d9d640 1 -- 172.21.15.94:0/2418118519 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aff3fba7b0 con 0x55aff40ba900 2024-02-17T12:03:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.132+0000 7f1022d9d640 1 --2- 172.21.15.94:0/2418118519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40ba900 0x55aff40d1150 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f101c009bf0 tx=0x7f101c003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600b26cff817113d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:55.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.132+0000 7f1022d9d640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f100c0a2400 0x7f100c0a48c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f101c037040 tx=0x7f101c03c000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:55.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.133+0000 7f1010ff9640 1 -- 172.21.15.94:0/2418118519 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c048070 con 0x55aff40ba900 2024-02-17T12:03:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.137+0000 7f1010ff9640 1 -- 172.21.15.94:0/2418118519 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f101c032c20 con 0x55aff40ba900 2024-02-17T12:03:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.137+0000 7f1010ff9640 1 -- 172.21.15.94:0/2418118519 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c004430 con 0x55aff40ba900 2024-02-17T12:03:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.138+0000 7f102bb7a740 1 -- 172.21.15.94:0/2418118519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40ba900 msgr2=0x55aff40d1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.138+0000 7f102bb7a740 1 --2- 172.21.15.94:0/2418118519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40ba900 0x55aff40d1150 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f101c009bf0 tx=0x7f101c003ca0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.140+0000 7f102bb7a740 1 -- 172.21.15.94:0/2418118519 shutdown_connections 2024-02-17T12:03:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.140+0000 7f102bb7a740 1 --2- 172.21.15.94:0/2418118519 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40ba900 0x55aff40d1150 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.140+0000 7f102bb7a740 1 --2- 172.21.15.94:0/2418118519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40b9f40 0x55aff40ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.140+0000 7f102bb7a740 1 --2- 172.21.15.94:0/2418118519 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40c94e0 0x55aff40c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.140+0000 7f102bb7a740 1 -- 172.21.15.94:0/2418118519 >> 172.21.15.94:0/2418118519 conn(0x55aff4084550 msgr2=0x55aff40b8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:55.140 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.140+0000 7f102bb7a740 1 -- 172.21.15.94:0/2418118519 shutdown_connections 2024-02-17T12:03:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.140+0000 7f102bb7a740 1 -- 172.21.15.94:0/2418118519 wait complete. 2024-02-17T12:03:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.140+0000 7f102bb7a740 1 -- start start 2024-02-17T12:03:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.141+0000 7f102bb7a740 1 Processor -- start 2024-02-17T12:03:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.141+0000 7f102bb7a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40b9f40 0x55aff40bb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.141+0000 7f102bb7a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40ba900 0x55aff40bbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.141+0000 7f102bb7a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40c94e0 0x55aff40bc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.141+0000 7f102bb7a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aff3fc9bf0 con 0x55aff40c94e0 2024-02-17T12:03:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.141+0000 7f102bb7a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aff3cf8250 con 0x55aff40b9f40 2024-02-17T12:03:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.141+0000 7f102bb7a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aff3faa200 con 0x55aff40ba900 2024-02-17T12:03:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.141+0000 7f102bb7a740 10 client.0 _ll_register_callbacks cb 0x55aff4018d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.141+0000 7f1022d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40c94e0 0x55aff40bc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.141+0000 7f1022d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40c94e0 0x55aff40bc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47906/0 (socket says 172.21.15.94:47906) 2024-02-17T12:03:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.141+0000 7f1022d9d640 1 -- 172.21.15.94:0/1383182852 learned_addr learned my addr 172.21.15.94:0/1383182852 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.142+0000 7f102259c640 1 --2- 172.21.15.94:0/1383182852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40b9f40 0x55aff40bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.142+0000 7f1021d9b640 1 --2- 172.21.15.94:0/1383182852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40ba900 0x55aff40bbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.142+0000 7f1022d9d640 1 -- 172.21.15.94:0/1383182852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40ba900 msgr2=0x55aff40bbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.142+0000 7f1022d9d640 1 --2- 172.21.15.94:0/1383182852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40ba900 0x55aff40bbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.142+0000 7f1022d9d640 1 -- 172.21.15.94:0/1383182852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40b9f40 msgr2=0x55aff40bb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.142+0000 7f1022d9d640 1 --2- 172.21.15.94:0/1383182852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40b9f40 0x55aff40bb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.142+0000 7f1022d9d640 1 -- 172.21.15.94:0/1383182852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102410e850 con 0x55aff40c94e0 2024-02-17T12:03:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.143+0000 7f1022d9d640 1 --2- 172.21.15.94:0/1383182852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40c94e0 0x55aff40bc390 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f101c009ca0 tx=0x7f101c003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb8bd35603ce85b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.143+0000 7f1009ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:03:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.143+0000 7f1009ffb640 1 -- 172.21.15.94:0/1383182852 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c048040 con 0x55aff40c94e0 2024-02-17T12:03:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.143+0000 7f102bb7a740 1 -- 172.21.15.94:0/1383182852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aff3d24600 con 0x55aff40c94e0 2024-02-17T12:03:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.144+0000 7f1009ffb640 1 -- 172.21.15.94:0/1383182852 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f101c004150 con 0x55aff40c94e0 2024-02-17T12:03:55.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.144+0000 7f1009ffb640 1 -- 172.21.15.94:0/1383182852 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f101c004310 con 0x55aff40c94e0 2024-02-17T12:03:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.146+0000 7f102bb7a740 10 client.16974 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:55.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.146+0000 7f102bb7a740 1 -- 172.21.15.94:0/1383182852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55aff4022d80 con 0x55aff40c94e0 2024-02-17T12:03:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.149+0000 7f1009ffb640 1 -- 172.21.15.94:0/1383182852 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aff3d24600 con 0x55aff40c94e0 2024-02-17T12:03:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.150+0000 7f1009ffb640 1 -- 172.21.15.94:0/1383182852 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f101c005050 con 0x55aff40c94e0 2024-02-17T12:03:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.150+0000 7f102bb7a740 10 client.16974 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.150+0000 7f102bb7a740 10 client.16974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.150+0000 7f102bb7a740 20 client.16974 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.150+0000 7f102bb7a740 1 --2- 172.21.15.94:0/1383182852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aff40e7400 0x55aff40e9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.150+0000 7f102bb7a740 4 client.16974 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.150+0000 7f102bb7a740 1 -- 172.21.15.94:0/1383182852 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aff3d751b0 con 0x55aff40e7400 2024-02-17T12:03:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.152+0000 7f102259c640 1 --2- 172.21.15.94:0/1383182852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aff40e7400 0x55aff40e9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.152+0000 7f1009ffb640 10 client.16974 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.152+0000 7f102259c640 1 --2- 172.21.15.94:0/1383182852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aff40e7400 0x55aff40e9820 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43a41ec0892eb120 server_cookie=9b7af681ddb2329b in_seq=0 out_seq=0 2024-02-17T12:03:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.153+0000 7f1009ffb640 1 -- 172.21.15.94:0/1383182852 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aff3d751b0 con 0x55aff40e7400 2024-02-17T12:03:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.153+0000 7f1009ffb640 10 client.16974 handle_command_reply: tid=0 2024-02-17T12:03:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.153+0000 7f102bb7a740 1 client.16974 shutdown 2024-02-17T12:03:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.153+0000 7f102bb7a740 1 -- 172.21.15.94:0/1383182852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40c94e0 msgr2=0x55aff40bc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.153+0000 7f102bb7a740 1 --2- 172.21.15.94:0/1383182852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40c94e0 0x55aff40bc390 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f101c009ca0 tx=0x7f101c003a90 comp rx=0 tx=0).stop 2024-02-17T12:03:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.153+0000 7f102bb7a740 1 -- 172.21.15.94:0/1383182852 shutdown_connections 2024-02-17T12:03:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.153+0000 7f102bb7a740 1 --2- 172.21.15.94:0/1383182852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aff40e7400 0x55aff40e9820 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.153+0000 7f102bb7a740 1 --2- 172.21.15.94:0/1383182852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40c94e0 0x55aff40bc390 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.153+0000 7f102bb7a740 1 --2- 172.21.15.94:0/1383182852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40ba900 0x55aff40bbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.153+0000 7f102bb7a740 1 --2- 172.21.15.94:0/1383182852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40b9f40 0x55aff40bb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.153+0000 7f102bb7a740 1 -- 172.21.15.94:0/1383182852 >> 172.21.15.94:0/1383182852 conn(0x55aff4084550 msgr2=0x55aff40b6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.153+0000 7f1009ffb640 0 client.16974 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.154+0000 7f102bb7a740 1 -- 172.21.15.94:0/1383182852 shutdown_connections 2024-02-17T12:03:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.154+0000 7f102bb7a740 1 -- 172.21.15.94:0/1383182852 wait complete. 2024-02-17T12:03:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.154+0000 7f102bb7a740 20 client.16974 trim_cache size 0 max 16384 2024-02-17T12:03:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.176+0000 7f102bb7a740 1 Processor -- start 2024-02-17T12:03:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.176+0000 7f102bb7a740 1 -- start start 2024-02-17T12:03:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.177+0000 7f102bb7a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 0x55aff40cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.177+0000 7f102bb7a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40ba900 0x55aff40ee180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.177+0000 7f102259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 0x55aff40cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.177+0000 7f102bb7a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40c94e0 0x55aff40bf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.177+0000 7f102259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 0x55aff40cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47908/0 (socket says 172.21.15.94:47908) 2024-02-17T12:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.177+0000 7f102bb7a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aff40b8720 con 0x55aff40b9f40 2024-02-17T12:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.177+0000 7f102bb7a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aff40e7200 con 0x55aff40c94e0 2024-02-17T12:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.177+0000 7f102bb7a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aff40b8540 con 0x55aff40ba900 2024-02-17T12:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.177+0000 7f102259c640 1 -- 172.21.15.94:0/2505157071 learned_addr learned my addr 172.21.15.94:0/2505157071 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.178+0000 7f1021d9b640 1 --2- 172.21.15.94:0/2505157071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40ba900 0x55aff40ee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.178+0000 7f1022d9d640 1 --2- 172.21.15.94:0/2505157071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40c94e0 0x55aff40bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.178+0000 7f102259c640 1 -- 172.21.15.94:0/2505157071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40ba900 msgr2=0x55aff40ee180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.178+0000 7f102259c640 1 --2- 172.21.15.94:0/2505157071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40ba900 0x55aff40ee180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.178+0000 7f102259c640 1 -- 172.21.15.94:0/2505157071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40c94e0 msgr2=0x55aff40bf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.178+0000 7f102259c640 1 --2- 172.21.15.94:0/2505157071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40c94e0 0x55aff40bf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.178+0000 7f102259c640 1 -- 172.21.15.94:0/2505157071 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aff3fbf7b0 con 0x55aff40b9f40 2024-02-17T12:03:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.179+0000 7f102259c640 1 --2- 172.21.15.94:0/2505157071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 0x55aff40cb010 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f101400f010 tx=0x7f1014000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c82ef7efe934fb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.179+0000 7f100bfff640 1 -- 172.21.15.94:0/2505157071 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10140a3110 con 0x55aff40b9f40 2024-02-17T12:03:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.179+0000 7f100bfff640 1 -- 172.21.15.94:0/2505157071 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1014098d20 con 0x55aff40b9f40 2024-02-17T12:03:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.180+0000 7f100bfff640 1 -- 172.21.15.94:0/2505157071 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10140b3470 con 0x55aff40b9f40 2024-02-17T12:03:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.180+0000 7f102bb7a740 1 -- 172.21.15.94:0/2505157071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 msgr2=0x55aff40cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.181+0000 7f102bb7a740 1 --2- 172.21.15.94:0/2505157071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 0x55aff40cb010 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f101400f010 tx=0x7f1014000bc0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.181+0000 7f102bb7a740 1 -- 172.21.15.94:0/2505157071 shutdown_connections 2024-02-17T12:03:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.181+0000 7f102bb7a740 1 --2- 172.21.15.94:0/2505157071 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40c94e0 0x55aff40bf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.181+0000 7f102bb7a740 1 --2- 172.21.15.94:0/2505157071 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40ba900 0x55aff40ee180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.181+0000 7f102bb7a740 1 --2- 172.21.15.94:0/2505157071 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 0x55aff40cb010 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.181+0000 7f102bb7a740 1 -- 172.21.15.94:0/2505157071 >> 172.21.15.94:0/2505157071 conn(0x55aff4084550 msgr2=0x55aff40cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.181+0000 7f102bb7a740 1 -- 172.21.15.94:0/2505157071 shutdown_connections 2024-02-17T12:03:55.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.181+0000 7f102bb7a740 1 -- 172.21.15.94:0/2505157071 wait complete. 2024-02-17T12:03:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.182+0000 7f102bb7a740 1 -- start start 2024-02-17T12:03:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.182+0000 7f102bb7a740 1 Processor -- start 2024-02-17T12:03:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.183+0000 7f102bb7a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 0x55aff40bfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.183 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.183+0000 7f102bb7a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40ba900 0x55aff40bdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.183+0000 7f102bb7a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40c94e0 0x55aff40be4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.183+0000 7f102bb7a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aff40beb00 con 0x55aff40b9f40 2024-02-17T12:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.183+0000 7f102bb7a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aff40bec70 con 0x55aff40ba900 2024-02-17T12:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.183+0000 7f102bb7a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aff40bee30 con 0x55aff40c94e0 2024-02-17T12:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.183+0000 7f102bb7a740 10 client.0 _ll_register_callbacks cb 0x7f101c037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.183+0000 7f102259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 0x55aff40bfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.183+0000 7f102259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 0x55aff40bfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47912/0 (socket says 172.21.15.94:47912) 2024-02-17T12:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.183+0000 7f102259c640 1 -- 172.21.15.94:0/3109644162 learned_addr learned my addr 172.21.15.94:0/3109644162 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:55.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.183+0000 7f1022d9d640 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40c94e0 0x55aff40be4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.184+0000 7f102259c640 1 -- 172.21.15.94:0/3109644162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40c94e0 msgr2=0x55aff40be4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.184+0000 7f102259c640 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40c94e0 0x55aff40be4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.184+0000 7f102259c640 1 -- 172.21.15.94:0/3109644162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40ba900 msgr2=0x55aff40bdf80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:03:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.184+0000 7f1021d9b640 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40ba900 0x55aff40bdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.184+0000 7f102259c640 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40ba900 0x55aff40bdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.184+0000 7f102259c640 1 -- 172.21.15.94:0/3109644162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10241acbb0 con 0x55aff40b9f40 2024-02-17T12:03:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.184+0000 7f1021d9b640 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40ba900 0x55aff40bdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:03:55.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.184+0000 7f102259c640 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 0x55aff40bfbb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f101409abe0 tx=0x7f10140972c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0c134dff5fe289 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.184+0000 7f1009ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.184+0000 7f1009ffb640 1 -- 172.21.15.94:0/3109644162 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10140b3410 con 0x55aff40b9f40 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.184+0000 7f1009ffb640 1 -- 172.21.15.94:0/3109644162 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f10140a3110 con 0x55aff40b9f40 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.184+0000 7f102bb7a740 1 -- 172.21.15.94:0/3109644162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f101c032c20 con 0x55aff40b9f40 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.185+0000 7f1009ffb640 1 -- 172.21.15.94:0/3109644162 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10140cbb60 con 0x55aff40b9f40 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.185+0000 7f102bb7a740 10 client.16980 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.185+0000 7f102bb7a740 1 -- 172.21.15.94:0/3109644162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55aff3fe5540 con 0x55aff40b9f40 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.185+0000 7f1009ffb640 1 -- 172.21.15.94:0/3109644162 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f101c032c20 con 0x55aff40b9f40 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.185+0000 7f1009ffb640 1 -- 172.21.15.94:0/3109644162 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f10140998e0 con 0x55aff40b9f40 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.185+0000 7f102bb7a740 10 client.16980 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.185+0000 7f102bb7a740 10 client.16980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.185+0000 7f102bb7a740 20 client.16980 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.185+0000 7f102bb7a740 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aff413eab0 0x55aff4140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:55.187 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.185+0000 7f102bb7a740 4 client.16980 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:03:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.185+0000 7f102bb7a740 1 -- 172.21.15.94:0/3109644162 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55aff3d61550 con 0x55aff413eab0 2024-02-17T12:03:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.186+0000 7f1021d9b640 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aff413eab0 0x55aff4140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.187+0000 7f1021d9b640 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aff413eab0 0x55aff4140ea0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=732d8b1b195ac531 server_cookie=530dd53a161dac8e in_seq=0 out_seq=0 2024-02-17T12:03:55.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.187+0000 7f1009ffb640 10 client.16980 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.206+0000 7f1009ffb640 1 -- 172.21.15.94:0/3109644162 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55aff3d61550 con 0x55aff413eab0 2024-02-17T12:03:55.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.206+0000 7f1009ffb640 10 client.16980 handle_command_reply: tid=0 2024-02-17T12:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.206+0000 7f102bb7a740 1 client.16980 shutdown 2024-02-17T12:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.207+0000 7f102bb7a740 1 -- 172.21.15.94:0/3109644162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 msgr2=0x55aff40bfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.207+0000 7f102bb7a740 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 0x55aff40bfbb0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f101409abe0 tx=0x7f10140972c0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.207+0000 7f102bb7a740 1 -- 172.21.15.94:0/3109644162 shutdown_connections 2024-02-17T12:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.207+0000 7f102bb7a740 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55aff413eab0 0x55aff4140ea0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.207+0000 7f1009ffb640 0 client.16980 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.207+0000 7f102bb7a740 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aff40c94e0 0x55aff40be4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.207+0000 7f102bb7a740 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aff40ba900 0x55aff40bdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.207+0000 7f102bb7a740 1 --2- 172.21.15.94:0/3109644162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aff40b9f40 0x55aff40bfbb0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.207+0000 7f102bb7a740 1 -- 172.21.15.94:0/3109644162 >> 172.21.15.94:0/3109644162 conn(0x55aff4084550 msgr2=0x55aff40ee180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.207+0000 7f102bb7a740 1 -- 172.21.15.94:0/3109644162 shutdown_connections 2024-02-17T12:03:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.207+0000 7f102bb7a740 1 -- 172.21.15.94:0/3109644162 wait complete. 2024-02-17T12:03:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.207+0000 7f102bb7a740 20 client.16980 trim_cache size 0 max 16384 2024-02-17T12:03:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:03:55.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:03:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:03:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:03:55.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.208+0000 7f1028ec9640 1 -- 172.21.15.94:0/2044516740 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f100c0a2400 msgr2=0x7f100c0a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.208+0000 7f1028ec9640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f100c0a2400 0x7f100c0a48c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f101c037040 tx=0x7f101c03c000 comp rx=0 tx=0).stop 2024-02-17T12:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.208+0000 7f1028ec9640 1 -- 172.21.15.94:0/2044516740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1024068800 msgr2=0x7f1024072910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.208+0000 7f1028ec9640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1024068800 0x7f1024072910 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f101400e9f0 tx=0x7f101400eeb0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.208+0000 7f1028ec9640 1 -- 172.21.15.94:0/2044516740 shutdown_connections 2024-02-17T12:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.208+0000 7f1028ec9640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f100c0a2400 0x7f100c0a48c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.208+0000 7f1028ec9640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1024107270 0x7f102406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.208+0000 7f1028ec9640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f10240691a0 0x7f1024072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.208+0000 7f1028ec9640 1 --2- 172.21.15.94:0/2044516740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1024068800 0x7f1024072910 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.208+0000 7f1028ec9640 1 -- 172.21.15.94:0/2044516740 >> 172.21.15.94:0/2044516740 conn(0x7f1024076ed0 msgr2=0x7f10240fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.208+0000 7f1028ec9640 1 -- 172.21.15.94:0/2044516740 shutdown_connections 2024-02-17T12:03:55.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:03:55.208+0000 7f1028ec9640 1 -- 172.21.15.94:0/2044516740 wait complete. 2024-02-17T12:03:55.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 335.0 s, 2659.4 tps, lat 37.451 ms stddev 41.195 2024-02-17T12:03:56.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:55 smithi094 ceph-mon[32499]: pgmap v178: 97 pgs: 97 active+clean; 3.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 MiB/s wr, 849 op/s 2024-02-17T12:03:56.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:55 smithi189 ceph-mon[36588]: pgmap v178: 97 pgs: 97 active+clean; 3.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 MiB/s wr, 849 op/s 2024-02-17T12:03:56.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:55 smithi125 ceph-mon[36819]: pgmap v178: 97 pgs: 97 active+clean; 3.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.6 MiB/s wr, 849 op/s 2024-02-17T12:03:57.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:56 smithi189 ceph-mon[36588]: pgmap v179: 97 pgs: 97 active+clean; 3.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 587 op/s 2024-02-17T12:03:57.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:56 smithi125 ceph-mon[36819]: pgmap v179: 97 pgs: 97 active+clean; 3.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 587 op/s 2024-02-17T12:03:57.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:56 smithi094 ceph-mon[32499]: pgmap v179: 97 pgs: 97 active+clean; 3.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.0 MiB/s wr, 587 op/s 2024-02-17T12:03:57.904 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:03:57.904 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:03:58.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.059+0000 7fd5b5555640 1 -- 172.21.15.94:0/2277246013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b006cb50 msgr2=0x7fd5b006cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.059+0000 7fd5b5555640 1 --2- 172.21.15.94:0/2277246013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b006cb50 0x7fd5b006cf30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009ca0 tx=0x7fd5a802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.059+0000 7fd5b5555640 1 -- 172.21.15.94:0/2277246013 shutdown_connections 2024-02-17T12:03:58.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.059+0000 7fd5b5555640 1 --2- 172.21.15.94:0/2277246013 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5b011a820 0x7fd5b011cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.059+0000 7fd5b5555640 1 --2- 172.21.15.94:0/2277246013 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5b006d470 0x7fd5b011a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.059+0000 7fd5b5555640 1 --2- 172.21.15.94:0/2277246013 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b006cb50 0x7fd5b006cf30 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.059+0000 7fd5b5555640 1 -- 172.21.15.94:0/2277246013 >> 172.21.15.94:0/2277246013 conn(0x7fd5b006c200 msgr2=0x7fd5b010a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.059+0000 7fd5b5555640 1 -- 172.21.15.94:0/2277246013 shutdown_connections 2024-02-17T12:03:58.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.059+0000 7fd5b5555640 1 -- 172.21.15.94:0/2277246013 wait complete. 2024-02-17T12:03:58.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.060+0000 7fd5b5555640 1 Processor -- start 2024-02-17T12:03:58.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.060+0000 7fd5b5555640 1 -- start start 2024-02-17T12:03:58.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.060+0000 7fd5b5555640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5b006cb50 0x7fd5b01a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.060+0000 7fd5b5555640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5b006d470 0x7fd5b01a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.060+0000 7fd5b5555640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b011a820 0x7fd5b01aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.060+0000 7fd5b5555640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b01ab400 con 0x7fd5b011a820 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.060+0000 7fd5b5555640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b01ab570 con 0x7fd5b006cb50 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.060+0000 7fd5b5555640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b01ab730 con 0x7fd5b006d470 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.060+0000 7fd5aeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5b006cb50 0x7fd5b01a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.060+0000 7fd5aeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5b006cb50 0x7fd5b01a6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44636/0 (socket says 172.21.15.94:44636) 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.060+0000 7fd5aeffd640 1 -- 172.21.15.94:0/1126389820 learned_addr learned my addr 172.21.15.94:0/1126389820 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.060+0000 7fd5af7fe640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b011a820 0x7fd5b01aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.060+0000 7fd5ae7fc640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5b006d470 0x7fd5b01a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.061+0000 7fd5aeffd640 1 -- 172.21.15.94:0/1126389820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5b006d470 msgr2=0x7fd5b01a6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.061+0000 7fd5aeffd640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5b006d470 0x7fd5b01a6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.061+0000 7fd5aeffd640 1 -- 172.21.15.94:0/1126389820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b011a820 msgr2=0x7fd5b01aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.061+0000 7fd5aeffd640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b011a820 0x7fd5b01aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.061+0000 7fd5aeffd640 1 -- 172.21.15.94:0/1126389820 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a0009490 con 0x7fd5b006cb50 2024-02-17T12:03:58.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.061+0000 7fd5af7fe640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b011a820 0x7fd5b01aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:58.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.061+0000 7fd5aeffd640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5b006cb50 0x7fd5b01a6540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8005e30 tx=0x7fd5a80325f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.061+0000 7fd597fff640 1 -- 172.21.15.94:0/1126389820 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8034e20 con 0x7fd5b006cb50 2024-02-17T12:03:58.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.062+0000 7fd597fff640 1 -- 172.21.15.94:0/1126389820 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5a8003e50 con 0x7fd5b006cb50 2024-02-17T12:03:58.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.062+0000 7fd5b5555640 1 -- 172.21.15.94:0/1126389820 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a8009900 con 0x7fd5b006cb50 2024-02-17T12:03:58.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.062+0000 7fd5b5555640 1 -- 172.21.15.94:0/1126389820 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b01b30b0 con 0x7fd5b006cb50 2024-02-17T12:03:58.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.063+0000 7fd597fff640 1 -- 172.21.15.94:0/1126389820 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8004750 con 0x7fd5b006cb50 2024-02-17T12:03:58.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.065+0000 7fd5b81fd740 1 Processor -- start 2024-02-17T12:03:58.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.066+0000 7fd5b81fd740 1 -- start start 2024-02-17T12:03:58.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.066+0000 7fd5b81fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04d2b0 0x564aac04d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.066+0000 7fd5b81fd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04dbd0 0x564aac056270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5b81fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac058ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5b81fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aabfbc280 con 0x564aac0567b0 2024-02-17T12:03:58.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5b81fd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aabfb97b0 con 0x564aac04dbd0 2024-02-17T12:03:58.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5b81fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aabfb4f90 con 0x564aac04d2b0 2024-02-17T12:03:58.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd597fff640 1 -- 172.21.15.94:0/1126389820 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd5a800a040 con 0x7fd5b006cb50 2024-02-17T12:03:58.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5ae7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac058ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5ae7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac058ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47942/0 (socket says 172.21.15.94:47942) 2024-02-17T12:03:58.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5af7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04d2b0 0x564aac04d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5ae7fc640 1 -- 172.21.15.94:0/2666711391 learned_addr learned my addr 172.21.15.94:0/2666711391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:58.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5aeffd640 1 --2- 172.21.15.94:0/2666711391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04dbd0 0x564aac056270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5ae7fc640 1 -- 172.21.15.94:0/2666711391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04d2b0 msgr2=0x564aac04d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5ae7fc640 1 --2- 172.21.15.94:0/2666711391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04d2b0 0x564aac04d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5ae7fc640 1 -- 172.21.15.94:0/2666711391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04dbd0 msgr2=0x564aac056270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5ae7fc640 1 --2- 172.21.15.94:0/2666711391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04dbd0 0x564aac056270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5ae7fc640 1 -- 172.21.15.94:0/2666711391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564aabf73570 con 0x564aac0567b0 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5ae7fc640 1 --2- 172.21.15.94:0/2666711391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac058ba0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400bc90 tx=0x7fd5a40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4085e3762a1a6ae4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd597fff640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5980a2380 0x7fd5980a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.067+0000 7fd5957fa640 1 -- 172.21.15.94:0/2666711391 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a401c480 con 0x564aac0567b0 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd5957fa640 1 -- 172.21.15.94:0/2666711391 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5a4015070 con 0x564aac0567b0 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd597fff640 1 -- 172.21.15.94:0/1126389820 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd5a80f0880 con 0x7fd5b006cb50 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd5af7fe640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5980a2380 0x7fd5980a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd5957fa640 1 -- 172.21.15.94:0/2666711391 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4011a20 con 0x564aac0567b0 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd5b81fd740 1 -- 172.21.15.94:0/2666711391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 msgr2=0x564aac058ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/2666711391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac058ba0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd5a400bc90 tx=0x7fd5a40107a0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd5af7fe640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5980a2380 0x7fd5980a4840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0012700 tx=0x7fd5a0005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd5b81fd740 1 -- 172.21.15.94:0/2666711391 shutdown_connections 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/2666711391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac058ba0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/2666711391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04dbd0 0x564aac056270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/2666711391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04d2b0 0x564aac04d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd5b81fd740 1 -- 172.21.15.94:0/2666711391 >> 172.21.15.94:0/2666711391 conn(0x564aac047990 msgr2=0x564aac047d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd5b81fd740 1 -- 172.21.15.94:0/2666711391 shutdown_connections 2024-02-17T12:03:58.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.068+0000 7fd5b81fd740 1 -- 172.21.15.94:0/2666711391 wait complete. 2024-02-17T12:03:58.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.069+0000 7fd5b81fd740 1 -- start start 2024-02-17T12:03:58.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.069+0000 7fd5b81fd740 1 Processor -- start 2024-02-17T12:03:58.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.069+0000 7fd5b81fd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 0x564aac056000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.069+0000 7fd5b81fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04dbd0 0x564aac053390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.069+0000 7fd5b81fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac0538d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.070+0000 7fd5af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac0538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.070+0000 7fd5af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac0538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47952/0 (socket says 172.21.15.94:47952) 2024-02-17T12:03:58.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.070+0000 7fd5af7fe640 1 -- 172.21.15.94:0/1091958888 learned_addr learned my addr 172.21.15.94:0/1091958888 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:58.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.070+0000 7fd5b81fd740 1 -- 172.21.15.94:0/1091958888 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aabc88430 con 0x564aac0567b0 2024-02-17T12:03:58.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.070+0000 7fd5ae7fc640 1 --2- 172.21.15.94:0/1091958888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04dbd0 0x564aac053390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.070+0000 7fd5aeffd640 1 --2- 172.21.15.94:0/1091958888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 0x564aac056000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.070+0000 7fd5b81fd740 1 -- 172.21.15.94:0/1091958888 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aabc44e00 con 0x564aac04d2b0 2024-02-17T12:03:58.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.070+0000 7fd5b81fd740 1 -- 172.21.15.94:0/1091958888 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aabf39fd0 con 0x564aac04dbd0 2024-02-17T12:03:58.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.070+0000 7fd5b81fd740 10 client.0 _ll_register_callbacks cb 0x564aabf8ffd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:58.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.071+0000 7fd5ae7fc640 1 -- 172.21.15.94:0/1091958888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 msgr2=0x564aac056000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.071+0000 7fd5ae7fc640 1 --2- 172.21.15.94:0/1091958888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 0x564aac056000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.071+0000 7fd5ae7fc640 1 -- 172.21.15.94:0/1091958888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 msgr2=0x564aac0538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.071+0000 7fd5ae7fc640 1 --2- 172.21.15.94:0/1091958888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac0538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.071+0000 7fd5ae7fc640 1 -- 172.21.15.94:0/1091958888 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a400b8c0 con 0x564aac04dbd0 2024-02-17T12:03:58.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.071+0000 7fd5ae7fc640 1 --2- 172.21.15.94:0/1091958888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04dbd0 0x564aac053390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4011480 tx=0x7fd5a40052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f3ea410b6109207 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.071+0000 7fd592ff5640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:03:58.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.072+0000 7fd592ff5640 1 -- 172.21.15.94:0/1091958888 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4011720 con 0x564aac04dbd0 2024-02-17T12:03:58.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.072+0000 7fd592ff5640 1 -- 172.21.15.94:0/1091958888 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd5a4017070 con 0x564aac04dbd0 2024-02-17T12:03:58.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.072+0000 7fd5b81fd740 1 -- 172.21.15.94:0/1091958888 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564aabcb47e0 con 0x564aac04dbd0 2024-02-17T12:03:58.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.072+0000 7fd592ff5640 1 -- 172.21.15.94:0/1091958888 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5a4011880 con 0x564aac04dbd0 2024-02-17T12:03:58.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.072+0000 7fd5b81fd740 10 client.26506 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:58.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.072+0000 7fd5b81fd740 1 -- 172.21.15.94:0/1091958888 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564aabfb3610 con 0x564aac04dbd0 2024-02-17T12:03:58.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.073+0000 7fd592ff5640 1 -- 172.21.15.94:0/1091958888 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x564aabcb47e0 con 0x564aac04dbd0 2024-02-17T12:03:58.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.073+0000 7fd592ff5640 1 -- 172.21.15.94:0/1091958888 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd5a4024070 con 0x564aac04dbd0 2024-02-17T12:03:58.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.073+0000 7fd5b81fd740 10 client.26506 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:58.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.073+0000 7fd5b81fd740 10 client.26506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:58.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.073+0000 7fd5b81fd740 20 client.26506 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:58.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.073+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/1091958888 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564aac0668b0 0x564aac068cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.073+0000 7fd5b81fd740 4 client.26506 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:58.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.073+0000 7fd5b81fd740 1 -- 172.21.15.94:0/1091958888 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564aabcf1720 con 0x564aac0668b0 2024-02-17T12:03:58.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.073+0000 7fd5aeffd640 1 --2- 172.21.15.94:0/1091958888 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564aac0668b0 0x564aac068cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.074+0000 7fd592ff5640 10 client.26506 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:58.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.074+0000 7fd5aeffd640 1 --2- 172.21.15.94:0/1091958888 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564aac0668b0 0x564aac068cd0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1528f03c10fca5c server_cookie=9ea39db295009d03 in_seq=0 out_seq=0 2024-02-17T12:03:58.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.075+0000 7fd592ff5640 1 -- 172.21.15.94:0/1091958888 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564aabcf1720 con 0x564aac0668b0 2024-02-17T12:03:58.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.075+0000 7fd592ff5640 10 client.26506 handle_command_reply: tid=0 2024-02-17T12:03:58.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.075+0000 7fd5b81fd740 1 client.26506 shutdown 2024-02-17T12:03:58.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.075+0000 7fd5b81fd740 1 -- 172.21.15.94:0/1091958888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04dbd0 msgr2=0x564aac053390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.075+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/1091958888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04dbd0 0x564aac053390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4011480 tx=0x7fd5a40052f0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.075+0000 7fd5b81fd740 1 -- 172.21.15.94:0/1091958888 shutdown_connections 2024-02-17T12:03:58.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.075+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/1091958888 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564aac0668b0 0x564aac068cd0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.075+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/1091958888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac0538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.075+0000 7fd592ff5640 0 client.26506 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:58.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.075+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/1091958888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04dbd0 0x564aac053390 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.075+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/1091958888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 0x564aac056000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.075+0000 7fd5b81fd740 1 -- 172.21.15.94:0/1091958888 >> 172.21.15.94:0/1091958888 conn(0x564aac047990 msgr2=0x564aac049730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.076+0000 7fd5b81fd740 1 -- 172.21.15.94:0/1091958888 shutdown_connections 2024-02-17T12:03:58.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.076+0000 7fd5b81fd740 1 -- 172.21.15.94:0/1091958888 wait complete. 2024-02-17T12:03:58.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.076+0000 7fd5b81fd740 20 client.26506 trim_cache size 0 max 16384 2024-02-17T12:03:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.102+0000 7fd5b81fd740 1 Processor -- start 2024-02-17T12:03:58.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.102+0000 7fd5b81fd740 1 -- start start 2024-02-17T12:03:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.102+0000 7fd5b81fd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 0x564aac055ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.102+0000 7fd5b81fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04dbd0 0x564aac056000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.102+0000 7fd5b81fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac05d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.103+0000 7fd5aeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 0x564aac055ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.103 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.103+0000 7fd5aeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 0x564aac055ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44662/0 (socket says 172.21.15.94:44662) 2024-02-17T12:03:58.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.103+0000 7fd5aeffd640 1 -- 172.21.15.94:0/3892304860 learned_addr learned my addr 172.21.15.94:0/3892304860 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:58.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.103+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3892304860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aac0546e0 con 0x564aac0567b0 2024-02-17T12:03:58.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.103+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3892304860 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aac0432b0 con 0x564aac04d2b0 2024-02-17T12:03:58.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.103+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3892304860 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aac04c5d0 con 0x564aac04dbd0 2024-02-17T12:03:58.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.104+0000 7fd5af7fe640 1 --2- 172.21.15.94:0/3892304860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac05d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.104+0000 7fd5af7fe640 1 -- 172.21.15.94:0/3892304860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04dbd0 msgr2=0x564aac056000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:03:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.104+0000 7fd5af7fe640 1 --2- 172.21.15.94:0/3892304860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04dbd0 0x564aac056000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.104+0000 7fd5af7fe640 1 -- 172.21.15.94:0/3892304860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 msgr2=0x564aac055ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.104+0000 7fd5af7fe640 1 --2- 172.21.15.94:0/3892304860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 0x564aac055ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.104+0000 7fd5af7fe640 1 -- 172.21.15.94:0/3892304860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564aabf4fac0 con 0x564aac0567b0 2024-02-17T12:03:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.104+0000 7fd5af7fe640 1 --2- 172.21.15.94:0/3892304860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac05d790 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0012370 tx=0x7fd5a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d49e95029c3bd98 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.104+0000 7fd5aeffd640 1 --2- 172.21.15.94:0/3892304860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 0x564aac055ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:03:58.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.106+0000 7fd594ff9640 1 -- 172.21.15.94:0/3892304860 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0025070 con 0x564aac0567b0 2024-02-17T12:03:58.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.106+0000 7fd594ff9640 1 -- 172.21.15.94:0/3892304860 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5a00035f0 con 0x564aac0567b0 2024-02-17T12:03:58.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.106+0000 7fd594ff9640 1 -- 172.21.15.94:0/3892304860 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0017070 con 0x564aac0567b0 2024-02-17T12:03:58.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.107+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3892304860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 msgr2=0x564aac05d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.107+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/3892304860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac05d790 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0012370 tx=0x7fd5a0017000 comp rx=0 tx=0).stop 2024-02-17T12:03:58.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.107+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3892304860 shutdown_connections 2024-02-17T12:03:58.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.107+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/3892304860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac0567b0 0x564aac05d790 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.107+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/3892304860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac04dbd0 0x564aac056000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.107+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/3892304860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 0x564aac055ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.107+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3892304860 >> 172.21.15.94:0/3892304860 conn(0x564aac047990 msgr2=0x564aac052c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.109+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3892304860 shutdown_connections 2024-02-17T12:03:58.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.111+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3892304860 wait complete. 2024-02-17T12:03:58.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.111+0000 7fd5b81fd740 1 -- start start 2024-02-17T12:03:58.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.112+0000 7fd5b81fd740 1 Processor -- start 2024-02-17T12:03:58.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.112+0000 7fd5b81fd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 0x564aac0c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.112+0000 7fd5b81fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac04dbd0 0x564aac0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.112+0000 7fd5b81fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac0567b0 0x564aac05cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.112+0000 7fd5b81fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aac05d460 con 0x564aac04dbd0 2024-02-17T12:03:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.112+0000 7fd5b81fd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aac05d5d0 con 0x564aac04d2b0 2024-02-17T12:03:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.112+0000 7fd5b81fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564aac05d790 con 0x564aac0567b0 2024-02-17T12:03:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.112+0000 7fd5b81fd740 10 client.0 _ll_register_callbacks cb 0x7fd5a400b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.113+0000 7fd5ae7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac04dbd0 0x564aac0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.113+0000 7fd5af7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac0567b0 0x564aac05cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.113+0000 7fd5af7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac0567b0 0x564aac05cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33560/0 (socket says 172.21.15.94:33560) 2024-02-17T12:03:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.113+0000 7fd5af7fe640 1 -- 172.21.15.94:0/3929590933 learned_addr learned my addr 172.21.15.94:0/3929590933 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.113+0000 7fd5ae7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac04dbd0 0x564aac0c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47972/0 (socket says 172.21.15.94:47972) 2024-02-17T12:03:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.113+0000 7fd5af7fe640 1 -- 172.21.15.94:0/3929590933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 msgr2=0x564aac0c85f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:03:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.114+0000 7fd5af7fe640 1 --2- 172.21.15.94:0/3929590933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 0x564aac0c85f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.114+0000 7fd5af7fe640 1 -- 172.21.15.94:0/3929590933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac04dbd0 msgr2=0x564aac0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.114+0000 7fd5af7fe640 1 --2- 172.21.15.94:0/3929590933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac04dbd0 0x564aac0c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.114+0000 7fd5af7fe640 1 -- 172.21.15.94:0/3929590933 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a0012050 con 0x564aac0567b0 2024-02-17T12:03:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.114+0000 7fd5af7fe640 1 --2- 172.21.15.94:0/3929590933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac0567b0 0x564aac05cea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0005d00 tx=0x7fd5a0003fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d256a5fb2404fb6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.114+0000 7fd592ff5640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:03:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.114+0000 7fd592ff5640 1 -- 172.21.15.94:0/3929590933 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0025070 con 0x564aac0567b0 2024-02-17T12:03:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.115+0000 7fd592ff5640 1 -- 172.21.15.94:0/3929590933 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd5a002a470 con 0x564aac0567b0 2024-02-17T12:03:58.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.115+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3929590933 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5a4015070 con 0x564aac0567b0 2024-02-17T12:03:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.115+0000 7fd592ff5640 1 -- 172.21.15.94:0/3929590933 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5a002a630 con 0x564aac0567b0 2024-02-17T12:03:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.115+0000 7fd5b81fd740 10 client.26509 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.115+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3929590933 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x564aabf756a0 con 0x564aac0567b0 2024-02-17T12:03:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.116+0000 7fd592ff5640 1 -- 172.21.15.94:0/3929590933 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5a4015070 con 0x564aac0567b0 2024-02-17T12:03:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.116+0000 7fd592ff5640 1 -- 172.21.15.94:0/3929590933 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd5a0033810 con 0x564aac0567b0 2024-02-17T12:03:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.116+0000 7fd5b81fd740 10 client.26509 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.116+0000 7fd5b81fd740 10 client.26509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:58.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.116+0000 7fd5b81fd740 20 client.26509 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.116+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/3929590933 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564aac0d03e0 0x564aac0d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.116+0000 7fd5b81fd740 4 client.26509 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:03:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.116+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3929590933 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564aabd40890 con 0x564aac0d03e0 2024-02-17T12:03:58.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.117+0000 7fd5aeffd640 1 --2- 172.21.15.94:0/3929590933 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564aac0d03e0 0x564aac0d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.120+0000 7fd5aeffd640 1 --2- 172.21.15.94:0/3929590933 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564aac0d03e0 0x564aac0d2800 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24ae9499f0826b71 server_cookie=43469605912d79b2 in_seq=0 out_seq=0 2024-02-17T12:03:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.120+0000 7fd592ff5640 10 client.26509 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.120+0000 7fd592ff5640 1 -- 172.21.15.94:0/3929590933 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564aabd40890 con 0x564aac0d03e0 2024-02-17T12:03:58.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.120+0000 7fd592ff5640 10 client.26509 handle_command_reply: tid=0 2024-02-17T12:03:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.121+0000 7fd5b81fd740 1 client.26509 shutdown 2024-02-17T12:03:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.121+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3929590933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac0567b0 msgr2=0x564aac05cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.121+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/3929590933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac0567b0 0x564aac05cea0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0005d00 tx=0x7fd5a0003fe0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.121+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3929590933 shutdown_connections 2024-02-17T12:03:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.121+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/3929590933 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564aac0d03e0 0x564aac0d2800 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.121+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/3929590933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564aac0567b0 0x564aac05cea0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.121+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/3929590933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564aac04dbd0 0x564aac0c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.121+0000 7fd5b81fd740 1 --2- 172.21.15.94:0/3929590933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564aac04d2b0 0x564aac0c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.121+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3929590933 >> 172.21.15.94:0/3929590933 conn(0x564aac047990 msgr2=0x564aac046a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.121+0000 7fd592ff5640 0 client.26509 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.121+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3929590933 shutdown_connections 2024-02-17T12:03:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.121+0000 7fd5b81fd740 1 -- 172.21.15.94:0/3929590933 wait complete. 2024-02-17T12:03:58.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.121+0000 7fd5b81fd740 20 client.26509 trim_cache size 0 max 16384 2024-02-17T12:03:58.122 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:03:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:03:58.123 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:03:58.123 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:03:58.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.122+0000 7fd5b5555640 1 -- 172.21.15.94:0/1126389820 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5980a2380 msgr2=0x7fd5980a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.122+0000 7fd5b5555640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5980a2380 0x7fd5980a4840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0012700 tx=0x7fd5a0005f90 comp rx=0 tx=0).stop 2024-02-17T12:03:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.122+0000 7fd5b5555640 1 -- 172.21.15.94:0/1126389820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5b006cb50 msgr2=0x7fd5b01a6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.122+0000 7fd5b5555640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5b006cb50 0x7fd5b01a6540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8005e30 tx=0x7fd5a80325f0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.122+0000 7fd5b5555640 1 -- 172.21.15.94:0/1126389820 shutdown_connections 2024-02-17T12:03:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.122+0000 7fd5b5555640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5980a2380 0x7fd5980a4840 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.122+0000 7fd5b5555640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5b011a820 0x7fd5b01aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.122+0000 7fd5b5555640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5b006d470 0x7fd5b01a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.122+0000 7fd5b5555640 1 --2- 172.21.15.94:0/1126389820 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5b006cb50 0x7fd5b01a6540 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.122+0000 7fd5b5555640 1 -- 172.21.15.94:0/1126389820 >> 172.21.15.94:0/1126389820 conn(0x7fd5b006c200 msgr2=0x7fd5b011bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.122+0000 7fd5b5555640 1 -- 172.21.15.94:0/1126389820 shutdown_connections 2024-02-17T12:03:58.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.122+0000 7fd5b5555640 1 -- 172.21.15.94:0/1126389820 wait complete. 2024-02-17T12:03:58.135 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:03:58.135 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:03:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.279+0000 7fb60726f640 1 -- 172.21.15.94:0/1656197542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb600102150 msgr2=0x7fb600069750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.279+0000 7fb60726f640 1 --2- 172.21.15.94:0/1656197542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb600102150 0x7fb600069750 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009900 tx=0x7fb5fc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:03:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.280+0000 7fb60726f640 1 -- 172.21.15.94:0/1656197542 shutdown_connections 2024-02-17T12:03:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.280+0000 7fb60726f640 1 --2- 172.21.15.94:0/1656197542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb60010da60 0x7fb60010fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.280+0000 7fb60726f640 1 --2- 172.21.15.94:0/1656197542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb600069d20 0x7fb60006a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.280+0000 7fb60726f640 1 --2- 172.21.15.94:0/1656197542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb600102150 0x7fb600069750 secure :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009900 tx=0x7fb5fc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:03:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.280+0000 7fb60726f640 1 -- 172.21.15.94:0/1656197542 >> 172.21.15.94:0/1656197542 conn(0x7fb600077060 msgr2=0x7fb600077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.281+0000 7fb60726f640 1 -- 172.21.15.94:0/1656197542 shutdown_connections 2024-02-17T12:03:58.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.281+0000 7fb60726f640 1 -- 172.21.15.94:0/1656197542 wait complete. 2024-02-17T12:03:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.282+0000 7fb60726f640 1 Processor -- start 2024-02-17T12:03:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.282+0000 7fb60726f640 1 -- start start 2024-02-17T12:03:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.282+0000 7fb60726f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb600069d20 0x7fb60019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.282+0000 7fb60726f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb60010da60 0x7fb60019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.282+0000 7fb60726f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb600199ac0 0x7fb600199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.282+0000 7fb60726f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60019a4b0 con 0x7fb600069d20 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.282+0000 7fb60726f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60019a620 con 0x7fb60010da60 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.282+0000 7fb60726f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60019a7e0 con 0x7fb600199ac0 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.282+0000 7fb60500b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb600069d20 0x7fb60019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.282+0000 7fb60500b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb600069d20 0x7fb60019f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47992/0 (socket says 172.21.15.94:47992) 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.282+0000 7fb60580c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb600199ac0 0x7fb600199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.282+0000 7fb60500b640 1 -- 172.21.15.94:0/3882435995 learned_addr learned my addr 172.21.15.94:0/3882435995 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.282+0000 7fb60480a640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb60010da60 0x7fb60019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.283+0000 7fb60500b640 1 -- 172.21.15.94:0/3882435995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb600199ac0 msgr2=0x7fb600199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.283+0000 7fb60500b640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb600199ac0 0x7fb600199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.283+0000 7fb60500b640 1 -- 172.21.15.94:0/3882435995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb60010da60 msgr2=0x7fb60019fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.283+0000 7fb60500b640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb60010da60 0x7fb60019fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.283+0000 7fb60500b640 1 -- 172.21.15.94:0/3882435995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc009560 con 0x7fb600069d20 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.283+0000 7fb60580c640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb600199ac0 0x7fb600199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:58.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.283+0000 7fb60480a640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb60010da60 0x7fb60019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:03:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.283+0000 7fb60500b640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb600069d20 0x7fb60019f910 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc035000 tx=0x7fb5fc02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.283+0000 7fb5f67fc640 1 -- 172.21.15.94:0/3882435995 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc004350 con 0x7fb600069d20 2024-02-17T12:03:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.283+0000 7fb60726f640 1 -- 172.21.15.94:0/3882435995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6001a67d0 con 0x7fb600069d20 2024-02-17T12:03:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.283+0000 7fb5f67fc640 1 -- 172.21.15.94:0/3882435995 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb5fc0044b0 con 0x7fb600069d20 2024-02-17T12:03:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.283+0000 7fb5f67fc640 1 -- 172.21.15.94:0/3882435995 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc003c90 con 0x7fb600069d20 2024-02-17T12:03:58.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.283+0000 7fb60726f640 1 -- 172.21.15.94:0/3882435995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6001a6c30 con 0x7fb600069d20 2024-02-17T12:03:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.284+0000 7fb60726f640 1 -- 172.21.15.94:0/3882435995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb600100680 con 0x7fb600069d20 2024-02-17T12:03:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.286+0000 7fb5f67fc640 1 -- 172.21.15.94:0/3882435995 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb5fc004df0 con 0x7fb600069d20 2024-02-17T12:03:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.286+0000 7fb5f67fc640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb5ec0a2230 0x7fb5ec0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.287+0000 7fb60480a640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb5ec0a2230 0x7fb5ec0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.287+0000 7fb5f67fc640 1 -- 172.21.15.94:0/3882435995 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb5fc0f0380 con 0x7fb600069d20 2024-02-17T12:03:58.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.287+0000 7fb60480a640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb5ec0a2230 0x7fb5ec0a46f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb6000ff950 tx=0x7fb5f0005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.294+0000 7fb5f67fc640 1 -- 172.21.15.94:0/3882435995 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb5fc0b9110 con 0x7fb600069d20 2024-02-17T12:03:58.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.636+0000 7fb60726f640 1 -- 172.21.15.94:0/3882435995 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6000683c0 con 0x7fb600069d20 2024-02-17T12:03:58.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.637+0000 7fb5f67fc640 1 -- 172.21.15.94:0/3882435995 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11338 (secure 0 0 0) 0x7fb5fc0bc7d0 con 0x7fb600069d20 2024-02-17T12:03:58.637 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:03:58.637 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:03:14.932679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26335":{"gid":26335,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/39050300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":39050300},{"type":"v1","addr":"172.21.15.189:6833","nonce":39050300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:03:58.637 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-02-17T12:03:58.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.639+0000 7fb60726f640 1 -- 172.21.15.94:0/3882435995 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb5ec0a2230 msgr2=0x7fb5ec0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.639+0000 7fb60726f640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb5ec0a2230 0x7fb5ec0a46f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb6000ff950 tx=0x7fb5f0005f90 comp rx=0 tx=0).stop 2024-02-17T12:03:58.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.639+0000 7fb60726f640 1 -- 172.21.15.94:0/3882435995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb600069d20 msgr2=0x7fb60019f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.639+0000 7fb60726f640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb600069d20 0x7fb60019f910 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc035000 tx=0x7fb5fc02c640 comp rx=0 tx=0).stop 2024-02-17T12:03:58.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.639+0000 7fb60726f640 1 -- 172.21.15.94:0/3882435995 shutdown_connections 2024-02-17T12:03:58.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.639+0000 7fb60726f640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb5ec0a2230 0x7fb5ec0a46f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.639+0000 7fb60726f640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb600199ac0 0x7fb600199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.639+0000 7fb60726f640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb60010da60 0x7fb60019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.639+0000 7fb60726f640 1 --2- 172.21.15.94:0/3882435995 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb600069d20 0x7fb60019f910 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.639+0000 7fb60726f640 1 -- 172.21.15.94:0/3882435995 >> 172.21.15.94:0/3882435995 conn(0x7fb600077060 msgr2=0x7fb60010e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.639+0000 7fb60726f640 1 -- 172.21.15.94:0/3882435995 shutdown_connections 2024-02-17T12:03:58.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.639+0000 7fb60726f640 1 -- 172.21.15.94:0/3882435995 wait complete. 2024-02-17T12:03:58.652 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:03:58.652 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:03:58.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.786+0000 7f510deec640 1 -- 172.21.15.94:0/3542036347 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5108100260 msgr2=0x7f5108100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.786+0000 7f510deec640 1 --2- 172.21.15.94:0/3542036347 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5108100260 0x7f5108100640 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009890 tx=0x7f50f802bb70 comp rx=0 tx=0).stop 2024-02-17T12:03:58.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.786+0000 7f510deec640 1 -- 172.21.15.94:0/3542036347 shutdown_connections 2024-02-17T12:03:58.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.786+0000 7f510deec640 1 --2- 172.21.15.94:0/3542036347 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5108105330 0x7f510810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.786+0000 7f510deec640 1 --2- 172.21.15.94:0/3542036347 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5108100c10 0x7f5108104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.786+0000 7f510deec640 1 --2- 172.21.15.94:0/3542036347 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5108100260 0x7f5108100640 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.786+0000 7f510deec640 1 -- 172.21.15.94:0/3542036347 >> 172.21.15.94:0/3542036347 conn(0x7f5108077180 msgr2=0x7f5108077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.786+0000 7f510deec640 1 -- 172.21.15.94:0/3542036347 shutdown_connections 2024-02-17T12:03:58.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.786+0000 7f510deec640 1 -- 172.21.15.94:0/3542036347 wait complete. 2024-02-17T12:03:58.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.787+0000 7f510deec640 1 Processor -- start 2024-02-17T12:03:58.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.787+0000 7f510deec640 1 -- start start 2024-02-17T12:03:58.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f510deec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5108100260 0x7f51081985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f510deec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5108105330 0x7f5108198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f510deec640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5108199110 0x7f51081a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f510deec640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081a3d60 con 0x7f5108100260 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f510deec640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081a3ed0 con 0x7f5108105330 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f510deec640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51081a4090 con 0x7f5108199110 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f51077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5108100260 0x7f51081985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f51077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5108100260 0x7f51081985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48020/0 (socket says 172.21.15.94:48020) 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f51077fe640 1 -- 172.21.15.94:0/4154602506 learned_addr learned my addr 172.21.15.94:0/4154602506 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f5107fff640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5108199110 0x7f51081a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f5106ffd640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5108105330 0x7f5108198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f51077fe640 1 -- 172.21.15.94:0/4154602506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5108199110 msgr2=0x7f51081a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f51077fe640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5108199110 0x7f51081a3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f51077fe640 1 -- 172.21.15.94:0/4154602506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5108105330 msgr2=0x7f5108198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f51077fe640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5108105330 0x7f5108198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.788+0000 7f51077fe640 1 -- 172.21.15.94:0/4154602506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5100009900 con 0x7f5108100260 2024-02-17T12:03:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.789+0000 7f51077fe640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5108100260 0x7f51081985f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f50f8000c00 tx=0x7f50f8004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.789+0000 7f5104ff9640 1 -- 172.21.15.94:0/4154602506 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8034950 con 0x7f5108100260 2024-02-17T12:03:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.789+0000 7f510deec640 1 -- 172.21.15.94:0/4154602506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f8009490 con 0x7f5108100260 2024-02-17T12:03:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.789+0000 7f5104ff9640 1 -- 172.21.15.94:0/4154602506 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50f802c9f0 con 0x7f5108100260 2024-02-17T12:03:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.789+0000 7f5104ff9640 1 -- 172.21.15.94:0/4154602506 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8032600 con 0x7f5108100260 2024-02-17T12:03:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.789+0000 7f510deec640 1 -- 172.21.15.94:0/4154602506 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51081a4540 con 0x7f5108100260 2024-02-17T12:03:58.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.792+0000 7f5104ff9640 1 -- 172.21.15.94:0/4154602506 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f50f803b040 con 0x7f5108100260 2024-02-17T12:03:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.792+0000 7f5110b95740 1 Processor -- start 2024-02-17T12:03:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.792+0000 7f5110b95740 1 -- start start 2024-02-17T12:03:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.792+0000 7f5104ff9640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f50f00a22b0 0x7f50f00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.792+0000 7f5107fff640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f50f00a22b0 0x7f50f00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.792+0000 7f5104ff9640 1 -- 172.21.15.94:0/4154602506 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f50f80efaa0 con 0x7f5108100260 2024-02-17T12:03:58.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f5110b95740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5384cb0 0x555fd5385090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f5110b95740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5385660 0x555fd5385ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f5110b95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5396e80 0x555fd5399270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f5110b95740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd5340960 con 0x555fd5385660 2024-02-17T12:03:58.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f5110b95740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd52f4310 con 0x555fd5384cb0 2024-02-17T12:03:58.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f5110b95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd52f1840 con 0x555fd5396e80 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f5107fff640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f50f00a22b0 0x7f50f00a4770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5100005fd0 tx=0x7f51000095c0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f51077fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5384cb0 0x555fd5385090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f51077fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5384cb0 0x555fd5385090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44720/0 (socket says 172.21.15.94:44720) 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f51077fe640 1 -- 172.21.15.94:0/56643831 learned_addr learned my addr 172.21.15.94:0/56643831 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f5106ffd640 1 --2- 172.21.15.94:0/56643831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5385660 0x555fd5385ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f5107fff640 1 --2- 172.21.15.94:0/56643831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5396e80 0x555fd5399270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f51077fe640 1 -- 172.21.15.94:0/56643831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5396e80 msgr2=0x555fd5399270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f51077fe640 1 --2- 172.21.15.94:0/56643831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5396e80 0x555fd5399270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f51077fe640 1 -- 172.21.15.94:0/56643831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5385660 msgr2=0x555fd5385ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f51077fe640 1 --2- 172.21.15.94:0/56643831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5385660 0x555fd5385ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f51077fe640 1 -- 172.21.15.94:0/56643831 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fd52af060 con 0x555fd5384cb0 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.793+0000 7f51077fe640 1 --2- 172.21.15.94:0/56643831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5384cb0 0x555fd5385090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f50f80bc4c0 tx=0x7f50f80b8900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dec9469b657922d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.794+0000 7f50edffb640 1 -- 172.21.15.94:0/56643831 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80bc530 con 0x555fd5384cb0 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.794+0000 7f50edffb640 1 -- 172.21.15.94:0/56643831 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50f80c0680 con 0x555fd5384cb0 2024-02-17T12:03:58.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.794+0000 7f50edffb640 1 -- 172.21.15.94:0/56643831 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80ed400 con 0x555fd5384cb0 2024-02-17T12:03:58.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.795+0000 7f5110b95740 1 -- 172.21.15.94:0/56643831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5384cb0 msgr2=0x555fd5385090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.795+0000 7f5110b95740 1 --2- 172.21.15.94:0/56643831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5384cb0 0x555fd5385090 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f50f80bc4c0 tx=0x7f50f80b8900 comp rx=0 tx=0).stop 2024-02-17T12:03:58.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.796+0000 7f5110b95740 1 -- 172.21.15.94:0/56643831 shutdown_connections 2024-02-17T12:03:58.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.796+0000 7f5110b95740 1 --2- 172.21.15.94:0/56643831 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5396e80 0x555fd5399270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.796+0000 7f5110b95740 1 --2- 172.21.15.94:0/56643831 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5385660 0x555fd5385ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.796+0000 7f5110b95740 1 --2- 172.21.15.94:0/56643831 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5384cb0 0x555fd5385090 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.796+0000 7f5110b95740 1 -- 172.21.15.94:0/56643831 >> 172.21.15.94:0/56643831 conn(0x555fd537f980 msgr2=0x555fd537fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.796+0000 7f5110b95740 1 -- 172.21.15.94:0/56643831 shutdown_connections 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.796+0000 7f5110b95740 1 -- 172.21.15.94:0/56643831 wait complete. 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f5110b95740 1 -- start start 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f5110b95740 1 Processor -- start 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f5110b95740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 0x555fd5387250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f5110b95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 0x555fd5387790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f5110b95740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5396e80 0x555fd5387cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f5110b95740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd52ed020 con 0x555fd5384cb0 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f5110b95740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4fc0430 con 0x555fd5396e80 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f5110b95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd4f7ce00 con 0x555fd5385660 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f5110b95740 10 client.0 _ll_register_callbacks cb 0x555fd5294cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f51077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 0x555fd5387250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f51077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 0x555fd5387250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48032/0 (socket says 172.21.15.94:48032) 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f51077fe640 1 -- 172.21.15.94:0/3638858657 learned_addr learned my addr 172.21.15.94:0/3638858657 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f5107fff640 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5396e80 0x555fd5387cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f5106ffd640 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 0x555fd5387790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f51077fe640 1 -- 172.21.15.94:0/3638858657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 msgr2=0x555fd5387790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f51077fe640 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 0x555fd5387790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f51077fe640 1 -- 172.21.15.94:0/3638858657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5396e80 msgr2=0x555fd5387cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f51077fe640 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5396e80 0x555fd5387cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.797+0000 7f51077fe640 1 -- 172.21.15.94:0/3638858657 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51081a4540 con 0x555fd5384cb0 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f5106ffd640 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 0x555fd5387790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f51077fe640 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 0x555fd5387250 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009860 tx=0x7f50f80c4d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5e6a03b78d2f8f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f50e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f50e6ffd640 1 -- 172.21.15.94:0/3638858657 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80bc530 con 0x555fd5384cb0 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f50e6ffd640 1 -- 172.21.15.94:0/3638858657 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f50f80b8ba0 con 0x555fd5384cb0 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f5110b95740 1 -- 172.21.15.94:0/3638858657 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fd4fec7e0 con 0x555fd5384cb0 2024-02-17T12:03:58.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f50e6ffd640 1 -- 172.21.15.94:0/3638858657 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50f80b8d60 con 0x555fd5384cb0 2024-02-17T12:03:58.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f5110b95740 10 client.17016 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:58.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f5110b95740 1 -- 172.21.15.94:0/3638858657 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555fd52eb6a0 con 0x555fd5384cb0 2024-02-17T12:03:58.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f50e6ffd640 1 -- 172.21.15.94:0/3638858657 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50f80a1020 con 0x555fd5384cb0 2024-02-17T12:03:58.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f50e6ffd640 1 -- 172.21.15.94:0/3638858657 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f50f80c0160 con 0x555fd5384cb0 2024-02-17T12:03:58.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f5110b95740 10 client.17016 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:58.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f5110b95740 10 client.17016 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:03:58.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f5110b95740 20 client.17016 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:58.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f5110b95740 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555fd53a6b10 0x555fd53a8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f5110b95740 4 client.17016 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:58.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.798+0000 7f5110b95740 1 -- 172.21.15.94:0/3638858657 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fd4fec7e0 con 0x555fd53a6b10 2024-02-17T12:03:58.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.799+0000 7f5106ffd640 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555fd53a6b10 0x555fd53a8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.799+0000 7f50e6ffd640 10 client.17016 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:03:58.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.799+0000 7f5106ffd640 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555fd53a6b10 0x555fd53a8f20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=60bf0e653b9983ad server_cookie=7aa07dca12c4806b in_seq=0 out_seq=0 2024-02-17T12:03:58.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f50e6ffd640 1 -- 172.21.15.94:0/3638858657 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555fd4fec7e0 con 0x555fd53a6b10 2024-02-17T12:03:58.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f50e6ffd640 10 client.17016 handle_command_reply: tid=0 2024-02-17T12:03:58.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f5110b95740 1 client.17016 shutdown 2024-02-17T12:03:58.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f5110b95740 1 -- 172.21.15.94:0/3638858657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 msgr2=0x555fd5387250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f5110b95740 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 0x555fd5387250 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009860 tx=0x7f50f80c4d40 comp rx=0 tx=0).stop 2024-02-17T12:03:58.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f5110b95740 1 -- 172.21.15.94:0/3638858657 shutdown_connections 2024-02-17T12:03:58.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f5110b95740 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555fd53a6b10 0x555fd53a8f20 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f5110b95740 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5396e80 0x555fd5387cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f5110b95740 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 0x555fd5387790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f5110b95740 1 --2- 172.21.15.94:0/3638858657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 0x555fd5387250 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f50e6ffd640 0 client.17016 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:03:58.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f5110b95740 1 -- 172.21.15.94:0/3638858657 >> 172.21.15.94:0/3638858657 conn(0x555fd537f980 msgr2=0x555fd53816d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f5110b95740 1 -- 172.21.15.94:0/3638858657 shutdown_connections 2024-02-17T12:03:58.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f5110b95740 1 -- 172.21.15.94:0/3638858657 wait complete. 2024-02-17T12:03:58.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.800+0000 7f5110b95740 20 client.17016 trim_cache size 0 max 16384 2024-02-17T12:03:58.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f5110b95740 1 Processor -- start 2024-02-17T12:03:58.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f5110b95740 1 -- start start 2024-02-17T12:03:58.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f5110b95740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd537f980 0x555fd5389980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f5110b95740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5384cb0 0x555fd5387af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f5110b95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 0x555fd539da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f5110b95740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd5272130 con 0x555fd537f980 2024-02-17T12:03:58.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f5110b95740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd537b080 con 0x555fd5384cb0 2024-02-17T12:03:58.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f5110b95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd53ae150 con 0x555fd5385660 2024-02-17T12:03:58.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f51077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd537f980 0x555fd5389980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f51077fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd537f980 0x555fd5389980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48038/0 (socket says 172.21.15.94:48038) 2024-02-17T12:03:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f51077fe640 1 -- 172.21.15.94:0/3781951380 learned_addr learned my addr 172.21.15.94:0/3781951380 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f5107fff640 1 --2- 172.21.15.94:0/3781951380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 0x555fd539da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f5106ffd640 1 --2- 172.21.15.94:0/3781951380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5384cb0 0x555fd5387af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f51077fe640 1 -- 172.21.15.94:0/3781951380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 msgr2=0x555fd539da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f51077fe640 1 --2- 172.21.15.94:0/3781951380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 0x555fd539da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f51077fe640 1 -- 172.21.15.94:0/3781951380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5384cb0 msgr2=0x555fd5387af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f51077fe640 1 --2- 172.21.15.94:0/3781951380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5384cb0 0x555fd5387af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.814+0000 7f51077fe640 1 -- 172.21.15.94:0/3781951380 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fd5287c20 con 0x555fd537f980 2024-02-17T12:03:58.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.815+0000 7f51077fe640 1 --2- 172.21.15.94:0/3781951380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd537f980 0x555fd5389980 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f50f80bc230 tx=0x7f50f80bcd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277c68b4138b0bd5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.815+0000 7f50ed7fa640 1 -- 172.21.15.94:0/3781951380 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80bb9d0 con 0x555fd537f980 2024-02-17T12:03:58.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.815+0000 7f50ed7fa640 1 -- 172.21.15.94:0/3781951380 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50f80a1070 con 0x555fd537f980 2024-02-17T12:03:58.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.815+0000 7f50ed7fa640 1 -- 172.21.15.94:0/3781951380 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80c4840 con 0x555fd537f980 2024-02-17T12:03:58.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.816+0000 7f5110b95740 1 -- 172.21.15.94:0/3781951380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd537f980 msgr2=0x555fd5389980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.816+0000 7f5110b95740 1 --2- 172.21.15.94:0/3781951380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd537f980 0x555fd5389980 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f50f80bc230 tx=0x7f50f80bcd50 comp rx=0 tx=0).stop 2024-02-17T12:03:58.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.816+0000 7f5110b95740 1 -- 172.21.15.94:0/3781951380 shutdown_connections 2024-02-17T12:03:58.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.816+0000 7f5110b95740 1 --2- 172.21.15.94:0/3781951380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 0x555fd539da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.816+0000 7f5110b95740 1 --2- 172.21.15.94:0/3781951380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd5384cb0 0x555fd5387af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.816+0000 7f5110b95740 1 --2- 172.21.15.94:0/3781951380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd537f980 0x555fd5389980 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.816+0000 7f5110b95740 1 -- 172.21.15.94:0/3781951380 >> 172.21.15.94:0/3781951380 conn(0x555fd5385db0 msgr2=0x555fd5393190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.816+0000 7f5110b95740 1 -- 172.21.15.94:0/3781951380 shutdown_connections 2024-02-17T12:03:58.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.816+0000 7f5110b95740 1 -- 172.21.15.94:0/3781951380 wait complete. 2024-02-17T12:03:58.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.816+0000 7f5110b95740 1 -- start start 2024-02-17T12:03:58.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.816+0000 7f5110b95740 1 Processor -- start 2024-02-17T12:03:58.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5110b95740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd537f980 0x555fd539a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5110b95740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 0x555fd539aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5110b95740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 0x555fd539af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5110b95740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd53fd040 con 0x555fd5384cb0 2024-02-17T12:03:58.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5110b95740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd53fd180 con 0x555fd537f980 2024-02-17T12:03:58.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5110b95740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd53fd2c0 con 0x555fd5385660 2024-02-17T12:03:58.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5110b95740 10 client.0 _ll_register_callbacks cb 0x7f50f8003f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5106ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 0x555fd539aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5106ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 0x555fd539aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48054/0 (socket says 172.21.15.94:48054) 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5106ffd640 1 -- 172.21.15.94:0/3172302821 learned_addr learned my addr 172.21.15.94:0/3172302821 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f51077fe640 1 --2- 172.21.15.94:0/3172302821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd537f980 0x555fd539a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5107fff640 1 --2- 172.21.15.94:0/3172302821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 0x555fd539af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5106ffd640 1 -- 172.21.15.94:0/3172302821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 msgr2=0x555fd539af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5106ffd640 1 --2- 172.21.15.94:0/3172302821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 0x555fd539af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5106ffd640 1 -- 172.21.15.94:0/3172302821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd537f980 msgr2=0x555fd539a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5106ffd640 1 --2- 172.21.15.94:0/3172302821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd537f980 0x555fd539a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5106ffd640 1 -- 172.21.15.94:0/3172302821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fd52eb6a0 con 0x555fd5384cb0 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5106ffd640 1 --2- 172.21.15.94:0/3172302821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 0x555fd539aa50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009de0 tx=0x7f50fc010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989c71adab674e62 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f50e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f50e6ffd640 1 -- 172.21.15.94:0/3172302821 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc01f930 con 0x555fd5384cb0 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f50e6ffd640 1 -- 172.21.15.94:0/3172302821 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f50fc004030 con 0x555fd5384cb0 2024-02-17T12:03:58.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.817+0000 7f5110b95740 1 -- 172.21.15.94:0/3172302821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50f80c0680 con 0x555fd5384cb0 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.818+0000 7f50e6ffd640 1 -- 172.21.15.94:0/3172302821 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50fc0041f0 con 0x555fd5384cb0 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.818+0000 7f5110b95740 10 client.17022 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.818+0000 7f5110b95740 1 -- 172.21.15.94:0/3172302821 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x555fd52ad8c0 con 0x555fd5384cb0 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.818+0000 7f50e6ffd640 1 -- 172.21.15.94:0/3172302821 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50f80c0680 con 0x555fd5384cb0 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.818+0000 7f50e6ffd640 1 -- 172.21.15.94:0/3172302821 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f50fc027710 con 0x555fd5384cb0 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.818+0000 7f5110b95740 10 client.17022 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.818+0000 7f5110b95740 10 client.17022 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.818+0000 7f5110b95740 20 client.17022 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.818+0000 7f5110b95740 1 --2- 172.21.15.94:0/3172302821 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555fd5396e80 0x555fd5400bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.818+0000 7f5110b95740 4 client.17022 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.818+0000 7f5110b95740 1 -- 172.21.15.94:0/3172302821 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555fd5029720 con 0x555fd5396e80 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.818+0000 7f51077fe640 1 --2- 172.21.15.94:0/3172302821 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555fd5396e80 0x555fd5400bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f51077fe640 1 --2- 172.21.15.94:0/3172302821 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555fd5396e80 0x555fd5400bb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9fd57e93737e80a1 server_cookie=4067ddbbc47e535c in_seq=0 out_seq=0 2024-02-17T12:03:58.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f50e6ffd640 10 client.17022 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f50e6ffd640 1 -- 172.21.15.94:0/3172302821 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555fd5029720 con 0x555fd5396e80 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f50e6ffd640 10 client.17022 handle_command_reply: tid=0 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f5110b95740 1 client.17022 shutdown 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f5110b95740 1 -- 172.21.15.94:0/3172302821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 msgr2=0x555fd539aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f5110b95740 1 --2- 172.21.15.94:0/3172302821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 0x555fd539aa50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009de0 tx=0x7f50fc010a40 comp rx=0 tx=0).stop 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f5110b95740 1 -- 172.21.15.94:0/3172302821 shutdown_connections 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f5110b95740 1 --2- 172.21.15.94:0/3172302821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fd5385660 0x555fd539af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f5110b95740 1 --2- 172.21.15.94:0/3172302821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fd5384cb0 0x555fd539aa50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f5110b95740 1 --2- 172.21.15.94:0/3172302821 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555fd5396e80 0x555fd5400bb0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f5110b95740 1 --2- 172.21.15.94:0/3172302821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fd537f980 0x555fd539a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f50e6ffd640 0 client.17022 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f5110b95740 1 -- 172.21.15.94:0/3172302821 >> 172.21.15.94:0/3172302821 conn(0x555fd5385db0 msgr2=0x555fd537e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f5110b95740 1 -- 172.21.15.94:0/3172302821 shutdown_connections 2024-02-17T12:03:58.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f5110b95740 1 -- 172.21.15.94:0/3172302821 wait complete. 2024-02-17T12:03:58.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.819+0000 7f5110b95740 20 client.17022 trim_cache size 0 max 16384 2024-02-17T12:03:58.826 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:03:58.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.820+0000 7f510deec640 1 -- 172.21.15.94:0/4154602506 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f50f00a22b0 msgr2=0x7f50f00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.820+0000 7f510deec640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f50f00a22b0 0x7f50f00a4770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5100005fd0 tx=0x7f51000095c0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.820+0000 7f510deec640 1 -- 172.21.15.94:0/4154602506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5108100260 msgr2=0x7f51081985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:58.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.820+0000 7f510deec640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5108100260 0x7f51081985f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f50f8000c00 tx=0x7f50f8004530 comp rx=0 tx=0).stop 2024-02-17T12:03:58.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.820+0000 7f510deec640 1 -- 172.21.15.94:0/4154602506 shutdown_connections 2024-02-17T12:03:58.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.820+0000 7f510deec640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f50f00a22b0 0x7f50f00a4770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.820+0000 7f510deec640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5108199110 0x7f51081a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.820+0000 7f510deec640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5108105330 0x7f5108198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.820+0000 7f510deec640 1 --2- 172.21.15.94:0/4154602506 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5108100260 0x7f51081985f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:58.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.820+0000 7f510deec640 1 -- 172.21.15.94:0/4154602506 >> 172.21.15.94:0/4154602506 conn(0x7f5108077180 msgr2=0x7f51080fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:58.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.820+0000 7f510deec640 1 -- 172.21.15.94:0/4154602506 shutdown_connections 2024-02-17T12:03:58.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:58.820+0000 7f510deec640 1 -- 172.21.15.94:0/4154602506 wait complete. 2024-02-17T12:03:58.833 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:03:58.834 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:03:58.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:58 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3882435995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:03:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.000+0000 7f12bf59e640 1 -- 172.21.15.94:0/1549337436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12c010dce0 msgr2=0x7f12c011c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.000+0000 7f12bf59e640 1 --2- 172.21.15.94:0/1549337436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12c010dce0 0x7f12c011c970 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009890 tx=0x7f12b802bb70 comp rx=0 tx=0).stop 2024-02-17T12:03:59.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.007+0000 7f12bf59e640 1 -- 172.21.15.94:0/1549337436 shutdown_connections 2024-02-17T12:03:59.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.007+0000 7f12bf59e640 1 --2- 172.21.15.94:0/1549337436 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12c010dce0 0x7f12c011c970 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.007+0000 7f12bf59e640 1 --2- 172.21.15.94:0/1549337436 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12c010d260 0x7f12c010d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.007+0000 7f12bf59e640 1 --2- 172.21.15.94:0/1549337436 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12c010f6a0 0x7f12c010fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.007+0000 7f12bf59e640 1 -- 172.21.15.94:0/1549337436 >> 172.21.15.94:0/1549337436 conn(0x7f12c006c030 msgr2=0x7f12c006c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:59.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.007+0000 7f12bf59e640 1 -- 172.21.15.94:0/1549337436 shutdown_connections 2024-02-17T12:03:59.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.007+0000 7f12bf59e640 1 -- 172.21.15.94:0/1549337436 wait complete. 2024-02-17T12:03:59.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.008+0000 7f12bf59e640 1 Processor -- start 2024-02-17T12:03:59.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.009+0000 7f12bf59e640 1 -- start start 2024-02-17T12:03:59.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.009+0000 7f12bf59e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12c010d260 0x7f12c01a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.009+0000 7f12bf59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12c010dce0 0x7f12c01a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.009+0000 7f12bf59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12c010f6a0 0x7f12c01aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.009+0000 7f12bf59e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c01ab1d0 con 0x7f12c010dce0 2024-02-17T12:03:59.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.009+0000 7f12bf59e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c01ab340 con 0x7f12c010f6a0 2024-02-17T12:03:59.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.009+0000 7f12bf59e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c01ab500 con 0x7f12c010d260 2024-02-17T12:03:59.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.009+0000 7f12be59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12c010d260 0x7f12c01a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.009+0000 7f12be59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12c010d260 0x7f12c01a63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33692/0 (socket says 172.21.15.94:33692) 2024-02-17T12:03:59.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.009+0000 7f12be59c640 1 -- 172.21.15.94:0/13735366 learned_addr learned my addr 172.21.15.94:0/13735366 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:59.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.009+0000 7f12bed9d640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12c010f6a0 0x7f12c01aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.010+0000 7f12bdd9b640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12c010dce0 0x7f12c01a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.010+0000 7f12be59c640 1 -- 172.21.15.94:0/13735366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12c010f6a0 msgr2=0x7f12c01aabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.010+0000 7f12be59c640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12c010f6a0 0x7f12c01aabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.010+0000 7f12be59c640 1 -- 172.21.15.94:0/13735366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12c010dce0 msgr2=0x7f12c01a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.010+0000 7f12be59c640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12c010dce0 0x7f12c01a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.010+0000 7f12be59c640 1 -- 172.21.15.94:0/13735366 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b8009490 con 0x7f12c010d260 2024-02-17T12:03:59.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.011+0000 7f12be59c640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12c010d260 0x7f12c01a63b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f12a800d950 tx=0x7f12a800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:59.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.011+0000 7f12b77fe640 1 -- 172.21.15.94:0/13735366 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8009c30 con 0x7f12c010d260 2024-02-17T12:03:59.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.011+0000 7f12b77fe640 1 -- 172.21.15.94:0/13735366 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12a8009d90 con 0x7f12c010d260 2024-02-17T12:03:59.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.011+0000 7f12b77fe640 1 -- 172.21.15.94:0/13735366 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a800f7e0 con 0x7f12c010d260 2024-02-17T12:03:59.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.011+0000 7f12bf59e640 1 -- 172.21.15.94:0/13735366 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c01ab7e0 con 0x7f12c010d260 2024-02-17T12:03:59.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.011+0000 7f12bf59e640 1 -- 172.21.15.94:0/13735366 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c01b3060 con 0x7f12c010d260 2024-02-17T12:03:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.016+0000 7f12c73cc740 1 Processor -- start 2024-02-17T12:03:59.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.017+0000 7f12c73cc740 1 -- start start 2024-02-17T12:03:59.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.018+0000 7f12c73cc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194460 0x55ffc4194840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.018+0000 7f12c73cc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc4194d80 0x55ffc419d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.018+0000 7f12c73cc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc419d960 0x55ffc419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.018+0000 7f12c73cc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc414f960 con 0x55ffc4194d80 2024-02-17T12:03:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.018+0000 7f12c73cc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc4103310 con 0x55ffc419d960 2024-02-17T12:03:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.018+0000 7f12c73cc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc4100840 con 0x55ffc4194460 2024-02-17T12:03:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.018+0000 7f12b77fe640 1 -- 172.21.15.94:0/13735366 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f12a8010ad0 con 0x7f12c010d260 2024-02-17T12:03:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.019+0000 7f12be59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc4194d80 0x55ffc419d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.019+0000 7f12be59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc4194d80 0x55ffc419d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48086/0 (socket says 172.21.15.94:48086) 2024-02-17T12:03:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.019+0000 7f12be59c640 1 -- 172.21.15.94:0/2808076873 learned_addr learned my addr 172.21.15.94:0/2808076873 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.019+0000 7f12bdd9b640 1 --2- 172.21.15.94:0/2808076873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc419d960 0x55ffc419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.019+0000 7f12bed9d640 1 --2- 172.21.15.94:0/2808076873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194460 0x55ffc4194840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.019+0000 7f12bdd9b640 1 -- 172.21.15.94:0/2808076873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194460 msgr2=0x55ffc4194840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.019+0000 7f12bdd9b640 1 --2- 172.21.15.94:0/2808076873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194460 0x55ffc4194840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.019+0000 7f12bdd9b640 1 -- 172.21.15.94:0/2808076873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc4194d80 msgr2=0x55ffc419d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.019+0000 7f12bdd9b640 1 --2- 172.21.15.94:0/2808076873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc4194d80 0x55ffc419d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.019+0000 7f12bdd9b640 1 -- 172.21.15.94:0/2808076873 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffc40be060 con 0x55ffc419d960 2024-02-17T12:03:59.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.019+0000 7f12bdd9b640 1 --2- 172.21.15.94:0/2808076873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc419d960 0x55ffc419fd50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f12b0005c40 tx=0x7f12b000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80372405c7618926 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.020+0000 7f12b4ff9640 1 -- 172.21.15.94:0/2808076873 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0003970 con 0x55ffc419d960 2024-02-17T12:03:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.020+0000 7f12b77fe640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12ac0a2330 0x7f12ac0a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.020+0000 7f12b4ff9640 1 -- 172.21.15.94:0/2808076873 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12b0004020 con 0x55ffc419d960 2024-02-17T12:03:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.020+0000 7f12b4ff9640 1 -- 172.21.15.94:0/2808076873 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0011ab0 con 0x55ffc419d960 2024-02-17T12:03:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.020+0000 7f12b77fe640 1 -- 172.21.15.94:0/13735366 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f12a80cde00 con 0x7f12c010d260 2024-02-17T12:03:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.020+0000 7f12bed9d640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12ac0a2330 0x7f12ac0a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.021+0000 7f12c73cc740 1 -- 172.21.15.94:0/2808076873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc419d960 msgr2=0x55ffc419fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.021+0000 7f12c73cc740 1 --2- 172.21.15.94:0/2808076873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc419d960 0x55ffc419fd50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f12b0005c40 tx=0x7f12b000f730 comp rx=0 tx=0).stop 2024-02-17T12:03:59.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.022+0000 7f12c73cc740 1 -- 172.21.15.94:0/2808076873 shutdown_connections 2024-02-17T12:03:59.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.022+0000 7f12c73cc740 1 --2- 172.21.15.94:0/2808076873 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc419d960 0x55ffc419fd50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.022+0000 7f12c73cc740 1 --2- 172.21.15.94:0/2808076873 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc4194d80 0x55ffc419d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.022+0000 7f12c73cc740 1 --2- 172.21.15.94:0/2808076873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194460 0x55ffc4194840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.022+0000 7f12c73cc740 1 -- 172.21.15.94:0/2808076873 >> 172.21.15.94:0/2808076873 conn(0x55ffc418eb90 msgr2=0x55ffc418ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.022+0000 7f12c73cc740 1 -- 172.21.15.94:0/2808076873 shutdown_connections 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.023+0000 7f12c73cc740 1 -- 172.21.15.94:0/2808076873 wait complete. 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.023+0000 7f12bed9d640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12ac0a2330 0x7f12ac0a47f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f12b802c180 tx=0x7f12b80324b0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.023+0000 7f12c73cc740 1 -- start start 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.023+0000 7f12c73cc740 1 Processor -- start 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.023+0000 7f12c73cc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc4194460 0x55ffc419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.023+0000 7f12c73cc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194d80 0x55ffc419aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.023+0000 7f12c73cc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc419d960 0x55ffc419b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.023+0000 7f12c73cc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc40fc020 con 0x55ffc419d960 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.023+0000 7f12c73cc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc3dcf430 con 0x55ffc4194460 2024-02-17T12:03:59.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.023+0000 7f12c73cc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc3d8be00 con 0x55ffc4194d80 2024-02-17T12:03:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.023+0000 7f12c73cc740 10 client.0 _ll_register_callbacks cb 0x55ffc40a3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12be59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc4194460 0x55ffc419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12bdd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194d80 0x55ffc419aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12be59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc4194460 0x55ffc419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44778/0 (socket says 172.21.15.94:44778) 2024-02-17T12:03:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12bdd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194d80 0x55ffc419aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33718/0 (socket says 172.21.15.94:33718) 2024-02-17T12:03:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12be59c640 1 -- 172.21.15.94:0/3898827826 learned_addr learned my addr 172.21.15.94:0/3898827826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12bed9d640 1 --2- 172.21.15.94:0/3898827826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc419d960 0x55ffc419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12be59c640 1 -- 172.21.15.94:0/3898827826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194d80 msgr2=0x55ffc419aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12be59c640 1 --2- 172.21.15.94:0/3898827826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194d80 0x55ffc419aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12be59c640 1 -- 172.21.15.94:0/3898827826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc419d960 msgr2=0x55ffc419b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12be59c640 1 --2- 172.21.15.94:0/3898827826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc419d960 0x55ffc419b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12be59c640 1 -- 172.21.15.94:0/3898827826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b000aa10 con 0x55ffc4194460 2024-02-17T12:03:59.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12bed9d640 1 --2- 172.21.15.94:0/3898827826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc419d960 0x55ffc419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:03:59.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12be59c640 1 --2- 172.21.15.94:0/3898827826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc4194460 0x55ffc419a9a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f12a8010fd0 tx=0x7f12a80d13f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3955d2df7ded268 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:59.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:03:59.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12a5ffb640 1 -- 172.21.15.94:0/3898827826 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a809a550 con 0x55ffc4194460 2024-02-17T12:03:59.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.024+0000 7f12a5ffb640 1 -- 172.21.15.94:0/3898827826 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f12a809ae80 con 0x55ffc4194460 2024-02-17T12:03:59.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.025+0000 7f12c73cc740 1 -- 172.21.15.94:0/3898827826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffc3dfb7e0 con 0x55ffc4194460 2024-02-17T12:03:59.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.025+0000 7f12a5ffb640 1 -- 172.21.15.94:0/3898827826 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12a809a2c0 con 0x55ffc4194460 2024-02-17T12:03:59.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.025+0000 7f12c73cc740 10 client.26501 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:59.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.025+0000 7f12c73cc740 1 -- 172.21.15.94:0/3898827826 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ffc40fa6a0 con 0x55ffc4194460 2024-02-17T12:03:59.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.025+0000 7f12a5ffb640 1 -- 172.21.15.94:0/3898827826 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ffc3dfb7e0 con 0x55ffc4194460 2024-02-17T12:03:59.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.025+0000 7f12a5ffb640 1 -- 172.21.15.94:0/3898827826 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f12a8096df0 con 0x55ffc4194460 2024-02-17T12:03:59.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.025+0000 7f12c73cc740 10 client.26501 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:59.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.025+0000 7f12c73cc740 10 client.26501 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:03:59.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.025+0000 7f12c73cc740 20 client.26501 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:59.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.025+0000 7f12c73cc740 1 --2- 172.21.15.94:0/3898827826 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffc41adea0 0x55ffc41b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.025+0000 7f12c73cc740 4 client.26501 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:59.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.025+0000 7f12c73cc740 1 -- 172.21.15.94:0/3898827826 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffc3e38720 con 0x55ffc41adea0 2024-02-17T12:03:59.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.026+0000 7f12bdd9b640 1 --2- 172.21.15.94:0/3898827826 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffc41adea0 0x55ffc41b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.029+0000 7f12bdd9b640 1 --2- 172.21.15.94:0/3898827826 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffc41adea0 0x55ffc41b02c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=500cbea3a93e1e78 server_cookie=a12ff7ea23be20d in_seq=0 out_seq=0 2024-02-17T12:03:59.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.029+0000 7f12a5ffb640 10 client.26501 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:03:59.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12a5ffb640 1 -- 172.21.15.94:0/3898827826 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffc3e38720 con 0x55ffc41adea0 2024-02-17T12:03:59.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12a5ffb640 10 client.26501 handle_command_reply: tid=0 2024-02-17T12:03:59.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12c73cc740 1 client.26501 shutdown 2024-02-17T12:03:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12c73cc740 1 -- 172.21.15.94:0/3898827826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc4194460 msgr2=0x55ffc419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12c73cc740 1 --2- 172.21.15.94:0/3898827826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc4194460 0x55ffc419a9a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f12a8010fd0 tx=0x7f12a80d13f0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12c73cc740 1 -- 172.21.15.94:0/3898827826 shutdown_connections 2024-02-17T12:03:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12c73cc740 1 --2- 172.21.15.94:0/3898827826 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffc41adea0 0x55ffc41b02c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12c73cc740 1 --2- 172.21.15.94:0/3898827826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc419d960 0x55ffc419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12c73cc740 1 --2- 172.21.15.94:0/3898827826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194d80 0x55ffc419aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12c73cc740 1 --2- 172.21.15.94:0/3898827826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc4194460 0x55ffc419a9a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12c73cc740 1 -- 172.21.15.94:0/3898827826 >> 172.21.15.94:0/3898827826 conn(0x55ffc418eb90 msgr2=0x55ffc41908e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12a5ffb640 0 client.26501 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:03:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12c73cc740 1 -- 172.21.15.94:0/3898827826 shutdown_connections 2024-02-17T12:03:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12c73cc740 1 -- 172.21.15.94:0/3898827826 wait complete. 2024-02-17T12:03:59.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.030+0000 7f12c73cc740 20 client.26501 trim_cache size 0 max 16384 2024-02-17T12:03:59.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.050+0000 7f12c73cc740 1 Processor -- start 2024-02-17T12:03:59.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.050+0000 7f12c73cc740 1 -- start start 2024-02-17T12:03:59.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.050+0000 7f12c73cc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc4194460 0x55ffc419d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.050+0000 7f12c73cc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194d80 0x55ffc419b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.050+0000 7f12be59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc4194460 0x55ffc419d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.050+0000 7f12be59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc4194460 0x55ffc419d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48112/0 (socket says 172.21.15.94:48112) 2024-02-17T12:03:59.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.050+0000 7f12be59c640 1 -- 172.21.15.94:0/2296124822 learned_addr learned my addr 172.21.15.94:0/2296124822 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:59.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.051+0000 7f12c73cc740 1 --2- 172.21.15.94:0/2296124822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc419d960 0x55ffc4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.051+0000 7f12c73cc740 1 -- 172.21.15.94:0/2296124822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc4081130 con 0x55ffc4194460 2024-02-17T12:03:59.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.051+0000 7f12c73cc740 1 -- 172.21.15.94:0/2296124822 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc41b52b0 con 0x55ffc419d960 2024-02-17T12:03:59.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.051+0000 7f12c73cc740 1 -- 172.21.15.94:0/2296124822 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc3dfb7e0 con 0x55ffc4194d80 2024-02-17T12:03:59.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.051+0000 7f12bdd9b640 1 --2- 172.21.15.94:0/2296124822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194d80 0x55ffc419b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.051+0000 7f12be59c640 1 -- 172.21.15.94:0/2296124822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194d80 msgr2=0x55ffc419b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.051+0000 7f12be59c640 1 --2- 172.21.15.94:0/2296124822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194d80 0x55ffc419b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.051+0000 7f12be59c640 1 -- 172.21.15.94:0/2296124822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc419d960 msgr2=0x55ffc4197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.051+0000 7f12be59c640 1 --2- 172.21.15.94:0/2296124822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc419d960 0x55ffc4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.051+0000 7f12be59c640 1 -- 172.21.15.94:0/2296124822 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffc4096c20 con 0x55ffc4194460 2024-02-17T12:03:59.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.051+0000 7f12be59c640 1 --2- 172.21.15.94:0/2296124822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc4194460 0x55ffc419d0d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f12a80a29f0 tx=0x7f12a80d1800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac136382a98948eb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:59.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.051+0000 7f12a7fff640 1 -- 172.21.15.94:0/2296124822 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a809a550 con 0x55ffc4194460 2024-02-17T12:03:59.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.051+0000 7f12a7fff640 1 -- 172.21.15.94:0/2296124822 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12a809a0e0 con 0x55ffc4194460 2024-02-17T12:03:59.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.052+0000 7f12a7fff640 1 -- 172.21.15.94:0/2296124822 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a809ae80 con 0x55ffc4194460 2024-02-17T12:03:59.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.052+0000 7f12c73cc740 1 -- 172.21.15.94:0/2296124822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc4194460 msgr2=0x55ffc419d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.052+0000 7f12c73cc740 1 --2- 172.21.15.94:0/2296124822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc4194460 0x55ffc419d0d0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f12a80a29f0 tx=0x7f12a80d1800 comp rx=0 tx=0).stop 2024-02-17T12:03:59.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.052+0000 7f12c73cc740 1 -- 172.21.15.94:0/2296124822 shutdown_connections 2024-02-17T12:03:59.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.052+0000 7f12c73cc740 1 --2- 172.21.15.94:0/2296124822 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc419d960 0x55ffc4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.052+0000 7f12c73cc740 1 --2- 172.21.15.94:0/2296124822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194d80 0x55ffc419b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.052+0000 7f12c73cc740 1 --2- 172.21.15.94:0/2296124822 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc4194460 0x55ffc419d0d0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.052+0000 7f12c73cc740 1 -- 172.21.15.94:0/2296124822 >> 172.21.15.94:0/2296124822 conn(0x55ffc418eb90 msgr2=0x55ffc4199270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:59.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.052+0000 7f12c73cc740 1 -- 172.21.15.94:0/2296124822 shutdown_connections 2024-02-17T12:03:59.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.052+0000 7f12c73cc740 1 -- 172.21.15.94:0/2296124822 wait complete. 2024-02-17T12:03:59.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.053+0000 7f12c73cc740 1 -- start start 2024-02-17T12:03:59.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.053+0000 7f12c73cc740 1 Processor -- start 2024-02-17T12:03:59.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.053+0000 7f12c73cc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194460 0x55ffc41a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.053+0000 7f12c73cc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc4194d80 0x55ffc41a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.053+0000 7f12c73cc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc419d960 0x55ffc41a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.053+0000 7f12c73cc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc41a3ac0 con 0x55ffc419d960 2024-02-17T12:03:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.053+0000 7f12c73cc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc41a3c30 con 0x55ffc4194d80 2024-02-17T12:03:59.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.053+0000 7f12c73cc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffc42043d0 con 0x55ffc4194460 2024-02-17T12:03:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.053+0000 7f12c73cc740 10 client.0 _ll_register_callbacks cb 0x7f12b000a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.053+0000 7f12be59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194460 0x55ffc41a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.053+0000 7f12be59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194460 0x55ffc41a4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33738/0 (socket says 172.21.15.94:33738) 2024-02-17T12:03:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.053+0000 7f12be59c640 1 -- 172.21.15.94:0/749147124 learned_addr learned my addr 172.21.15.94:0/749147124 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.054+0000 7f12be59c640 1 -- 172.21.15.94:0/749147124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc4194d80 msgr2=0x55ffc41a2eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:03:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.054+0000 7f12be59c640 1 --2- 172.21.15.94:0/749147124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc4194d80 0x55ffc41a2eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.054+0000 7f12be59c640 1 -- 172.21.15.94:0/749147124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc419d960 msgr2=0x55ffc41a33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.054+0000 7f12be59c640 1 --2- 172.21.15.94:0/749147124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc419d960 0x55ffc41a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.054+0000 7f12be59c640 1 -- 172.21.15.94:0/749147124 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffc40fa6a0 con 0x55ffc4194460 2024-02-17T12:03:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.054+0000 7f12be59c640 1 --2- 172.21.15.94:0/749147124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194460 0x55ffc41a4ae0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f12a8010e20 tx=0x7f12a800bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2babedb19a7f06a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.054+0000 7f12a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:03:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.054+0000 7f12a5ffb640 1 -- 172.21.15.94:0/749147124 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a80d1310 con 0x55ffc4194460 2024-02-17T12:03:59.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.054+0000 7f12a5ffb640 1 -- 172.21.15.94:0/749147124 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f12a809ae80 con 0x55ffc4194460 2024-02-17T12:03:59.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.054+0000 7f12c73cc740 1 -- 172.21.15.94:0/749147124 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12b0004020 con 0x55ffc4194460 2024-02-17T12:03:59.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.055+0000 7f12a5ffb640 1 -- 172.21.15.94:0/749147124 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12a809a550 con 0x55ffc4194460 2024-02-17T12:03:59.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.056+0000 7f12c73cc740 10 client.26548 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:59.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.056+0000 7f12c73cc740 1 -- 172.21.15.94:0/749147124 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ffc40bc8c0 con 0x55ffc4194460 2024-02-17T12:03:59.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.056+0000 7f12a5ffb640 1 -- 172.21.15.94:0/749147124 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12a807f080 con 0x55ffc4194460 2024-02-17T12:03:59.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.056+0000 7f12a5ffb640 1 -- 172.21.15.94:0/749147124 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f12a8099640 con 0x55ffc4194460 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.056+0000 7f12c73cc740 10 client.26548 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.056+0000 7f12c73cc740 10 client.26548 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.056+0000 7f12c73cc740 20 client.26548 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.056+0000 7f12c73cc740 1 --2- 172.21.15.94:0/749147124 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffc42055c0 0x55ffc42079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.056+0000 7f12c73cc740 4 client.26548 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.056+0000 7f12c73cc740 1 -- 172.21.15.94:0/749147124 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ffc3e87890 con 0x55ffc42055c0 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.056+0000 7f12bdd9b640 1 --2- 172.21.15.94:0/749147124 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffc42055c0 0x55ffc42079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12bdd9b640 1 --2- 172.21.15.94:0/749147124 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffc42055c0 0x55ffc42079b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87b5f367d628a92a server_cookie=59299b07905d08b9 in_seq=0 out_seq=0 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12a5ffb640 10 client.26548 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12a5ffb640 1 -- 172.21.15.94:0/749147124 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ffc3e87890 con 0x55ffc42055c0 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12a5ffb640 10 client.26548 handle_command_reply: tid=0 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12c73cc740 1 client.26548 shutdown 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12c73cc740 1 -- 172.21.15.94:0/749147124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194460 msgr2=0x55ffc41a4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12c73cc740 1 --2- 172.21.15.94:0/749147124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194460 0x55ffc41a4ae0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f12a8010e20 tx=0x7f12a800bf60 comp rx=0 tx=0).stop 2024-02-17T12:03:59.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12c73cc740 1 -- 172.21.15.94:0/749147124 shutdown_connections 2024-02-17T12:03:59.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12c73cc740 1 --2- 172.21.15.94:0/749147124 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffc42055c0 0x55ffc42079b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12c73cc740 1 --2- 172.21.15.94:0/749147124 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffc419d960 0x55ffc41a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12c73cc740 1 --2- 172.21.15.94:0/749147124 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffc4194d80 0x55ffc41a2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12c73cc740 1 --2- 172.21.15.94:0/749147124 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffc4194460 0x55ffc41a4ae0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12c73cc740 1 -- 172.21.15.94:0/749147124 >> 172.21.15.94:0/749147124 conn(0x55ffc418eb90 msgr2=0x55ffc418ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:59.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.057+0000 7f12a5ffb640 0 client.26548 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:03:59.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12c73cc740 1 -- 172.21.15.94:0/749147124 shutdown_connections 2024-02-17T12:03:59.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12c73cc740 1 -- 172.21.15.94:0/749147124 wait complete. 2024-02-17T12:03:59.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12c73cc740 20 client.26548 trim_cache size 0 max 16384 2024-02-17T12:03:59.065 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:03:59.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12bf59e640 1 -- 172.21.15.94:0/13735366 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12ac0a2330 msgr2=0x7f12ac0a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12bf59e640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12ac0a2330 0x7f12ac0a47f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f12b802c180 tx=0x7f12b80324b0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12bf59e640 1 -- 172.21.15.94:0/13735366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12c010d260 msgr2=0x7f12c01a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12bf59e640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12c010d260 0x7f12c01a63b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f12a800d950 tx=0x7f12a800de10 comp rx=0 tx=0).stop 2024-02-17T12:03:59.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12bf59e640 1 -- 172.21.15.94:0/13735366 shutdown_connections 2024-02-17T12:03:59.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12bf59e640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12ac0a2330 0x7f12ac0a47f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12bf59e640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12c010f6a0 0x7f12c01aabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12bf59e640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12c010dce0 0x7f12c01a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12bf59e640 1 --2- 172.21.15.94:0/13735366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12c010d260 0x7f12c01a63b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12bf59e640 1 -- 172.21.15.94:0/13735366 >> 172.21.15.94:0/13735366 conn(0x7f12c006c030 msgr2=0x7f12c0070500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:59.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12bf59e640 1 -- 172.21.15.94:0/13735366 shutdown_connections 2024-02-17T12:03:59.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.058+0000 7f12bf59e640 1 -- 172.21.15.94:0/13735366 wait complete. 2024-02-17T12:03:59.073 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:03:59.073 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:03:59.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:58 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3882435995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:03:59.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:58 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3882435995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:03:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.211+0000 7fc661d9b640 1 -- 172.21.15.94:0/285948062 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc654032340 con 0x7fc664108660 2024-02-17T12:03:59.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.211+0000 7fc669088640 1 -- 172.21.15.94:0/285948062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664108660 msgr2=0x7fc664110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.211+0000 7fc669088640 1 --2- 172.21.15.94:0/285948062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664108660 0x7fc664110230 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc654009890 tx=0x7fc65402bb70 comp rx=0 tx=0).stop 2024-02-17T12:03:59.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.211+0000 7fc669088640 1 -- 172.21.15.94:0/285948062 shutdown_connections 2024-02-17T12:03:59.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.211+0000 7fc669088640 1 --2- 172.21.15.94:0/285948062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664108660 0x7fc664110230 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.211+0000 7fc669088640 1 --2- 172.21.15.94:0/285948062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc664103f40 0x7fc664107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.211+0000 7fc669088640 1 --2- 172.21.15.94:0/285948062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc664103590 0x7fc664103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.211+0000 7fc669088640 1 -- 172.21.15.94:0/285948062 >> 172.21.15.94:0/285948062 conn(0x7fc6640fedd0 msgr2=0x7fc6641011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:59.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.211+0000 7fc669088640 1 -- 172.21.15.94:0/285948062 shutdown_connections 2024-02-17T12:03:59.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.211+0000 7fc669088640 1 -- 172.21.15.94:0/285948062 wait complete. 2024-02-17T12:03:59.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.212+0000 7fc669088640 1 Processor -- start 2024-02-17T12:03:59.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.212+0000 7fc669088640 1 -- start start 2024-02-17T12:03:59.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc669088640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc664103590 0x7fc66419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc669088640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664108660 0x7fc66419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc669088640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6641997d0 0x7fc664199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc669088640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66419a280 con 0x7fc664103590 2024-02-17T12:03:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc669088640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66419a3f0 con 0x7fc664108660 2024-02-17T12:03:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc669088640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66419a560 con 0x7fc6641997d0 2024-02-17T12:03:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc662d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc664103590 0x7fc66419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc66259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664108660 0x7fc66419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc66359e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6641997d0 0x7fc664199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc66259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664108660 0x7fc66419fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44816/0 (socket says 172.21.15.94:44816) 2024-02-17T12:03:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc66259c640 1 -- 172.21.15.94:0/3641458282 learned_addr learned my addr 172.21.15.94:0/3641458282 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc662d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc664103590 0x7fc66419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48122/0 (socket says 172.21.15.94:48122) 2024-02-17T12:03:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc66359e640 1 -- 172.21.15.94:0/3641458282 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664108660 msgr2=0x7fc66419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc66359e640 1 --2- 172.21.15.94:0/3641458282 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664108660 0x7fc66419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc66359e640 1 -- 172.21.15.94:0/3641458282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc664103590 msgr2=0x7fc66419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc66359e640 1 --2- 172.21.15.94:0/3641458282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc664103590 0x7fc66419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc66359e640 1 -- 172.21.15.94:0/3641458282 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc654009490 con 0x7fc6641997d0 2024-02-17T12:03:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc662d9d640 1 --2- 172.21.15.94:0/3641458282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc664103590 0x7fc66419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc66359e640 1 --2- 172.21.15.94:0/3641458282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6641997d0 0x7fc664199c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc65402c180 tx=0x7fc654004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc64bfff640 1 -- 172.21.15.94:0/3641458282 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc654034950 con 0x7fc6641997d0 2024-02-17T12:03:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc64bfff640 1 -- 172.21.15.94:0/3641458282 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc65402c9f0 con 0x7fc6641997d0 2024-02-17T12:03:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc669088640 1 -- 172.21.15.94:0/3641458282 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6641a64b0 con 0x7fc6641997d0 2024-02-17T12:03:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.213+0000 7fc64bfff640 1 -- 172.21.15.94:0/3641458282 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6540035f0 con 0x7fc6641997d0 2024-02-17T12:03:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.214+0000 7fc669088640 1 -- 172.21.15.94:0/3641458282 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6641a6820 con 0x7fc6641997d0 2024-02-17T12:03:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.217+0000 7fc64bfff640 1 -- 172.21.15.94:0/3641458282 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc65403b040 con 0x7fc6641997d0 2024-02-17T12:03:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.217+0000 7fc64bfff640 1 --2- 172.21.15.94:0/3641458282 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc64c0a2230 0x7fc64c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.218+0000 7fc64bfff640 1 -- 172.21.15.94:0/3641458282 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc6540bc8f0 con 0x7fc6641997d0 2024-02-17T12:03:59.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.218+0000 7fc662d9d640 1 --2- 172.21.15.94:0/3641458282 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc64c0a2230 0x7fc64c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.219+0000 7fc66bc68740 1 Processor -- start 2024-02-17T12:03:59.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.219+0000 7fc662d9d640 1 --2- 172.21.15.94:0/3641458282 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc64c0a2230 0x7fc64c0a46f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc658009fc0 tx=0x7fc658005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:59.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.219+0000 7fc66bc68740 1 -- start start 2024-02-17T12:03:59.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.219+0000 7fc66bc68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 0x55643b4ade40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.219+0000 7fc66bc68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 0x55643b4b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.219+0000 7fc66bc68740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 0x55643b4b9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.219+0000 7fc66bc68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643b460960 con 0x55643b4ada60 2024-02-17T12:03:59.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.219+0000 7fc66bc68740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643b414310 con 0x55643b4b6f60 2024-02-17T12:03:59.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.219+0000 7fc66bc68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643b411840 con 0x55643b4ae380 2024-02-17T12:03:59.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.219+0000 7fc662d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 0x55643b4ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.219+0000 7fc662d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 0x55643b4ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48138/0 (socket says 172.21.15.94:48138) 2024-02-17T12:03:59.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.219+0000 7fc662d9d640 1 -- 172.21.15.94:0/1620118086 learned_addr learned my addr 172.21.15.94:0/1620118086 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:59.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.219+0000 7fc66359e640 1 --2- 172.21.15.94:0/1620118086 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 0x55643b4b9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.220+0000 7fc66259c640 1 --2- 172.21.15.94:0/1620118086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 0x55643b4b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.220+0000 7fc662d9d640 1 -- 172.21.15.94:0/1620118086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 msgr2=0x55643b4b6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.220+0000 7fc662d9d640 1 --2- 172.21.15.94:0/1620118086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 0x55643b4b6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.220+0000 7fc662d9d640 1 -- 172.21.15.94:0/1620118086 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 msgr2=0x55643b4b9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.220+0000 7fc662d9d640 1 --2- 172.21.15.94:0/1620118086 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 0x55643b4b9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.220+0000 7fc662d9d640 1 -- 172.21.15.94:0/1620118086 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55643b3cf060 con 0x55643b4ada60 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.220+0000 7fc662d9d640 1 --2- 172.21.15.94:0/1620118086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 0x55643b4ade40 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc6580042c0 tx=0x7fc658014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1721c0ec757b233e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.220+0000 7fc6497fa640 1 -- 172.21.15.94:0/1620118086 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65800f480 con 0x55643b4ada60 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.220+0000 7fc6497fa640 1 -- 172.21.15.94:0/1620118086 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc658021070 con 0x55643b4ada60 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.220+0000 7fc6497fa640 1 -- 172.21.15.94:0/1620118086 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65801ca50 con 0x55643b4ada60 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.221+0000 7fc66bc68740 1 -- 172.21.15.94:0/1620118086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 msgr2=0x55643b4ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.221+0000 7fc66bc68740 1 --2- 172.21.15.94:0/1620118086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 0x55643b4ade40 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc6580042c0 tx=0x7fc658014000 comp rx=0 tx=0).stop 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.221+0000 7fc66bc68740 1 -- 172.21.15.94:0/1620118086 shutdown_connections 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.221+0000 7fc66bc68740 1 --2- 172.21.15.94:0/1620118086 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 0x55643b4b9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.221+0000 7fc66bc68740 1 --2- 172.21.15.94:0/1620118086 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 0x55643b4b6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.221+0000 7fc66bc68740 1 --2- 172.21.15.94:0/1620118086 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 0x55643b4ade40 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.221+0000 7fc66bc68740 1 -- 172.21.15.94:0/1620118086 >> 172.21.15.94:0/1620118086 conn(0x55643b4a8190 msgr2=0x55643b4a8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:59.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.221+0000 7fc66bc68740 1 -- 172.21.15.94:0/1620118086 shutdown_connections 2024-02-17T12:03:59.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.221+0000 7fc66bc68740 1 -- 172.21.15.94:0/1620118086 wait complete. 2024-02-17T12:03:59.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.221+0000 7fc66bc68740 1 -- start start 2024-02-17T12:03:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.221+0000 7fc66bc68740 1 Processor -- start 2024-02-17T12:03:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc66bc68740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4ada60 0x55643b4b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc66bc68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 0x55643b4b4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc66bc68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4b6f60 0x55643b4b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc66bc68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643b40d020 con 0x55643b4b6f60 2024-02-17T12:03:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc66bc68740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643b0e0430 con 0x55643b4ada60 2024-02-17T12:03:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc66bc68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643b09ce00 con 0x55643b4ae380 2024-02-17T12:03:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc66bc68740 10 client.0 _ll_register_callbacks cb 0x55643b3b4cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc662d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4ada60 0x55643b4b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc66359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4b6f60 0x55643b4b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc662d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4ada60 0x55643b4b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44820/0 (socket says 172.21.15.94:44820) 2024-02-17T12:03:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc662d9d640 1 -- 172.21.15.94:0/1357293030 learned_addr learned my addr 172.21.15.94:0/1357293030 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:59.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc66259c640 1 --2- 172.21.15.94:0/1357293030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 0x55643b4b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc662d9d640 1 -- 172.21.15.94:0/1357293030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 msgr2=0x55643b4b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc662d9d640 1 --2- 172.21.15.94:0/1357293030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 0x55643b4b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc662d9d640 1 -- 172.21.15.94:0/1357293030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4b6f60 msgr2=0x55643b4b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc662d9d640 1 --2- 172.21.15.94:0/1357293030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4b6f60 0x55643b4b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc662d9d640 1 -- 172.21.15.94:0/1357293030 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc658017050 con 0x55643b4ada60 2024-02-17T12:03:59.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc662d9d640 1 --2- 172.21.15.94:0/1357293030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4ada60 0x55643b4b4020 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc65801c670 tx=0x7fc658004460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47472a78054efe66 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:59.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc646ff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:03:59.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.222+0000 7fc646ff5640 1 -- 172.21.15.94:0/1357293030 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658005310 con 0x55643b4ada60 2024-02-17T12:03:59.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc646ff5640 1 -- 172.21.15.94:0/1357293030 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc658005470 con 0x55643b4ada60 2024-02-17T12:03:59.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc66bc68740 1 -- 172.21.15.94:0/1357293030 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55643b10c7e0 con 0x55643b4ada60 2024-02-17T12:03:59.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc646ff5640 1 -- 172.21.15.94:0/1357293030 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc658005630 con 0x55643b4ada60 2024-02-17T12:03:59.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc66bc68740 10 client.26513 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:59.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc66bc68740 1 -- 172.21.15.94:0/1357293030 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55643b40b6a0 con 0x55643b4ada60 2024-02-17T12:03:59.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc646ff5640 1 -- 172.21.15.94:0/1357293030 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc65803c020 con 0x55643b4ada60 2024-02-17T12:03:59.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc646ff5640 1 -- 172.21.15.94:0/1357293030 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc658004c60 con 0x55643b4ada60 2024-02-17T12:03:59.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc66bc68740 10 client.26513 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:59.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc66bc68740 10 client.26513 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:59.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc66bc68740 20 client.26513 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:59.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc66bc68740 1 --2- 172.21.15.94:0/1357293030 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55643b4c7430 0x55643b4c9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc66bc68740 4 client.26513 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:03:59.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc66bc68740 1 -- 172.21.15.94:0/1357293030 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55643b149720 con 0x55643b4c7430 2024-02-17T12:03:59.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.223+0000 7fc66259c640 1 --2- 172.21.15.94:0/1357293030 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55643b4c7430 0x55643b4c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.224+0000 7fc646ff5640 10 client.26513 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:59.230 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.224+0000 7fc66259c640 1 --2- 172.21.15.94:0/1357293030 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55643b4c7430 0x55643b4c9850 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fe1a96e30efee64 server_cookie=49167a2fa106f1cc in_seq=0 out_seq=0 2024-02-17T12:03:59.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.226+0000 7fc646ff5640 1 -- 172.21.15.94:0/1357293030 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55643b149720 con 0x55643b4c7430 2024-02-17T12:03:59.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.226+0000 7fc646ff5640 10 client.26513 handle_command_reply: tid=0 2024-02-17T12:03:59.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.226+0000 7fc66bc68740 1 client.26513 shutdown 2024-02-17T12:03:59.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.226+0000 7fc66bc68740 1 -- 172.21.15.94:0/1357293030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4ada60 msgr2=0x55643b4b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.226+0000 7fc66bc68740 1 --2- 172.21.15.94:0/1357293030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4ada60 0x55643b4b4020 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc65801c670 tx=0x7fc658004460 comp rx=0 tx=0).stop 2024-02-17T12:03:59.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.227+0000 7fc66bc68740 1 -- 172.21.15.94:0/1357293030 shutdown_connections 2024-02-17T12:03:59.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.227+0000 7fc66bc68740 1 --2- 172.21.15.94:0/1357293030 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55643b4c7430 0x55643b4c9850 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.227+0000 7fc66bc68740 1 --2- 172.21.15.94:0/1357293030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4b6f60 0x55643b4b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.227+0000 7fc66bc68740 1 --2- 172.21.15.94:0/1357293030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 0x55643b4b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.227+0000 7fc66bc68740 1 --2- 172.21.15.94:0/1357293030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4ada60 0x55643b4b4020 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.227+0000 7fc66bc68740 1 -- 172.21.15.94:0/1357293030 >> 172.21.15.94:0/1357293030 conn(0x55643b4a8190 msgr2=0x55643b4a9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:59.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.227+0000 7fc646ff5640 0 client.26513 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:59.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.227+0000 7fc66bc68740 1 -- 172.21.15.94:0/1357293030 shutdown_connections 2024-02-17T12:03:59.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.227+0000 7fc66bc68740 1 -- 172.21.15.94:0/1357293030 wait complete. 2024-02-17T12:03:59.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.227+0000 7fc66bc68740 20 client.26513 trim_cache size 0 max 16384 2024-02-17T12:03:59.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.244+0000 7fc66bc68740 1 Processor -- start 2024-02-17T12:03:59.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.244+0000 7fc66bc68740 1 -- start start 2024-02-17T12:03:59.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc66bc68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 0x55643b4b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc66bc68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 0x55643b4bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc66bc68740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 0x55643b4be350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc66bc68740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643b392130 con 0x55643b4ada60 2024-02-17T12:03:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc66bc68740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643b49b330 con 0x55643b4b6f60 2024-02-17T12:03:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc66bc68740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643b4ceac0 con 0x55643b4ae380 2024-02-17T12:03:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc662d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 0x55643b4b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc662d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 0x55643b4b48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48156/0 (socket says 172.21.15.94:48156) 2024-02-17T12:03:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc662d9d640 1 -- 172.21.15.94:0/2625345714 learned_addr learned my addr 172.21.15.94:0/2625345714 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc66259c640 1 --2- 172.21.15.94:0/2625345714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 0x55643b4bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc66359e640 1 --2- 172.21.15.94:0/2625345714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 0x55643b4be350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc662d9d640 1 -- 172.21.15.94:0/2625345714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 msgr2=0x55643b4bde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc662d9d640 1 --2- 172.21.15.94:0/2625345714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 0x55643b4bde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc662d9d640 1 -- 172.21.15.94:0/2625345714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 msgr2=0x55643b4be350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc662d9d640 1 --2- 172.21.15.94:0/2625345714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 0x55643b4be350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc662d9d640 1 -- 172.21.15.94:0/2625345714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55643b3a7c20 con 0x55643b4ada60 2024-02-17T12:03:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.245+0000 7fc66259c640 1 --2- 172.21.15.94:0/2625345714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 0x55643b4bde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.246+0000 7fc662d9d640 1 --2- 172.21.15.94:0/2625345714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 0x55643b4b48c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc658009730 tx=0x7fc65801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303c75c3e1be0b6f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.246+0000 7fc648ff9640 1 -- 172.21.15.94:0/2625345714 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65800fc70 con 0x55643b4ada60 2024-02-17T12:03:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.246+0000 7fc648ff9640 1 -- 172.21.15.94:0/2625345714 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc65803c070 con 0x55643b4ada60 2024-02-17T12:03:59.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.246+0000 7fc648ff9640 1 -- 172.21.15.94:0/2625345714 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65801c990 con 0x55643b4ada60 2024-02-17T12:03:59.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.247+0000 7fc66bc68740 1 -- 172.21.15.94:0/2625345714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 msgr2=0x55643b4b48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.247+0000 7fc66bc68740 1 --2- 172.21.15.94:0/2625345714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 0x55643b4b48c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc658009730 tx=0x7fc65801c560 comp rx=0 tx=0).stop 2024-02-17T12:03:59.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.247+0000 7fc66bc68740 1 -- 172.21.15.94:0/2625345714 shutdown_connections 2024-02-17T12:03:59.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.247+0000 7fc66bc68740 1 --2- 172.21.15.94:0/2625345714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 0x55643b4be350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.247+0000 7fc66bc68740 1 --2- 172.21.15.94:0/2625345714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ae380 0x55643b4bde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.247+0000 7fc66bc68740 1 --2- 172.21.15.94:0/2625345714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ada60 0x55643b4b48c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.247+0000 7fc66bc68740 1 -- 172.21.15.94:0/2625345714 >> 172.21.15.94:0/2625345714 conn(0x55643b4a8190 msgr2=0x55643b4af2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:59.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.247+0000 7fc66bc68740 1 -- 172.21.15.94:0/2625345714 shutdown_connections 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.247+0000 7fc66bc68740 1 -- 172.21.15.94:0/2625345714 wait complete. 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.247+0000 7fc66bc68740 1 -- start start 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.247+0000 7fc66bc68740 1 Processor -- start 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc66bc68740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ada60 0x55643b4bddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc66bc68740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ae380 0x55643b4be2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc66bc68740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 0x55643b4bc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc66259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ae380 0x55643b4be2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc66259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ae380 0x55643b4be2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48168/0 (socket says 172.21.15.94:48168) 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc662d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ada60 0x55643b4bddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc662d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ada60 0x55643b4bddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33786/0 (socket says 172.21.15.94:33786) 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc66259c640 1 -- 172.21.15.94:0/3792970936 learned_addr learned my addr 172.21.15.94:0/3792970936 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc66bc68740 1 -- 172.21.15.94:0/3792970936 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643b4bca70 con 0x55643b4ae380 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc66bc68740 1 -- 172.21.15.94:0/3792970936 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643b4bcbe0 con 0x55643b4b6f60 2024-02-17T12:03:59.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc66bc68740 1 -- 172.21.15.94:0/3792970936 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55643b4bcda0 con 0x55643b4ada60 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc66359e640 1 --2- 172.21.15.94:0/3792970936 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 0x55643b4bc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc66bc68740 10 client.0 _ll_register_callbacks cb 0x7fc658009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc662d9d640 1 -- 172.21.15.94:0/3792970936 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 msgr2=0x55643b4bc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc662d9d640 1 --2- 172.21.15.94:0/3792970936 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 0x55643b4bc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc662d9d640 1 -- 172.21.15.94:0/3792970936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ae380 msgr2=0x55643b4be2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc662d9d640 1 --2- 172.21.15.94:0/3792970936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ae380 0x55643b4be2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc662d9d640 1 -- 172.21.15.94:0/3792970936 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55643b40b6a0 con 0x55643b4ada60 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc66259c640 1 --2- 172.21.15.94:0/3792970936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ae380 0x55643b4be2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc662d9d640 1 --2- 172.21.15.94:0/3792970936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ada60 0x55643b4bddb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc658009e80 tx=0x7fc65800fb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4b392ecc01ad81e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.248+0000 7fc646ff5640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.249+0000 7fc646ff5640 1 -- 172.21.15.94:0/3792970936 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc658005520 con 0x55643b4ada60 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.249+0000 7fc646ff5640 1 -- 172.21.15.94:0/3792970936 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc65803d080 con 0x55643b4ada60 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.249+0000 7fc66bc68740 1 -- 172.21.15.94:0/3792970936 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc658021070 con 0x55643b4ada60 2024-02-17T12:03:59.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.249+0000 7fc646ff5640 1 -- 172.21.15.94:0/3792970936 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc658011d20 con 0x55643b4ada60 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.249+0000 7fc66bc68740 10 client.26566 fetch_fsmap learned FSMap version 25 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.249+0000 7fc66bc68740 1 -- 172.21.15.94:0/3792970936 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55643b3cd8c0 con 0x55643b4ada60 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.249+0000 7fc646ff5640 1 -- 172.21.15.94:0/3792970936 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc658021070 con 0x55643b4ada60 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.250+0000 7fc646ff5640 1 -- 172.21.15.94:0/3792970936 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc6580264c0 con 0x55643b4ada60 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.250+0000 7fc66bc68740 10 client.26566 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.250+0000 7fc66bc68740 10 client.26566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.250+0000 7fc66bc68740 20 client.26566 populate_metadata read hostname 'smithi094' 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.250+0000 7fc66bc68740 1 --2- 172.21.15.94:0/3792970936 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55643b51e730 0x55643b520b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.251+0000 7fc66bc68740 4 client.26566 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.251+0000 7fc66bc68740 1 -- 172.21.15.94:0/3792970936 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55643b198890 con 0x55643b51e730 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.252+0000 7fc66259c640 1 --2- 172.21.15.94:0/3792970936 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55643b51e730 0x55643b520b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.252+0000 7fc646ff5640 10 client.26566 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.252+0000 7fc66259c640 1 --2- 172.21.15.94:0/3792970936 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55643b51e730 0x55643b520b20 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8164790fd13a31d server_cookie=eec3126856d292c1 in_seq=0 out_seq=0 2024-02-17T12:03:59.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.252+0000 7fc646ff5640 1 -- 172.21.15.94:0/3792970936 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55643b198890 con 0x55643b51e730 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.252+0000 7fc646ff5640 10 client.26566 handle_command_reply: tid=0 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.252+0000 7fc66bc68740 1 client.26566 shutdown 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc66bc68740 1 -- 172.21.15.94:0/3792970936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ada60 msgr2=0x55643b4bddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc66bc68740 1 --2- 172.21.15.94:0/3792970936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ada60 0x55643b4bddb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc658009e80 tx=0x7fc65800fb60 comp rx=0 tx=0).stop 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc66bc68740 1 -- 172.21.15.94:0/3792970936 shutdown_connections 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc66bc68740 1 --2- 172.21.15.94:0/3792970936 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55643b51e730 0x55643b520b20 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc66bc68740 1 --2- 172.21.15.94:0/3792970936 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55643b4b6f60 0x55643b4bc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc646ff5640 0 client.26566 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc66bc68740 1 --2- 172.21.15.94:0/3792970936 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55643b4ae380 0x55643b4be2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc66bc68740 1 --2- 172.21.15.94:0/3792970936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55643b4ada60 0x55643b4bddb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc66bc68740 1 -- 172.21.15.94:0/3792970936 >> 172.21.15.94:0/3792970936 conn(0x55643b4a8190 msgr2=0x55643b49e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc66bc68740 1 -- 172.21.15.94:0/3792970936 shutdown_connections 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc66bc68740 1 -- 172.21.15.94:0/3792970936 wait complete. 2024-02-17T12:03:59.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc66bc68740 20 client.26566 trim_cache size 0 max 16384 2024-02-17T12:03:59.259 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:03:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc669088640 1 -- 172.21.15.94:0/3641458282 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc64c0a2230 msgr2=0x7fc64c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc669088640 1 --2- 172.21.15.94:0/3641458282 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc64c0a2230 0x7fc64c0a46f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc658009fc0 tx=0x7fc658005f90 comp rx=0 tx=0).stop 2024-02-17T12:03:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc669088640 1 -- 172.21.15.94:0/3641458282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6641997d0 msgr2=0x7fc664199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:03:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc669088640 1 --2- 172.21.15.94:0/3641458282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6641997d0 0x7fc664199c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc65402c180 tx=0x7fc654004530 comp rx=0 tx=0).stop 2024-02-17T12:03:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc669088640 1 -- 172.21.15.94:0/3641458282 shutdown_connections 2024-02-17T12:03:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc669088640 1 --2- 172.21.15.94:0/3641458282 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc64c0a2230 0x7fc64c0a46f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc669088640 1 --2- 172.21.15.94:0/3641458282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc6641997d0 0x7fc664199c80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc669088640 1 --2- 172.21.15.94:0/3641458282 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc664108660 0x7fc66419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.253+0000 7fc669088640 1 --2- 172.21.15.94:0/3641458282 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc664103590 0x7fc66419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:03:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.254+0000 7fc669088640 1 -- 172.21.15.94:0/3641458282 >> 172.21.15.94:0/3641458282 conn(0x7fc6640fedd0 msgr2=0x7fc6641010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:03:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.254+0000 7fc669088640 1 -- 172.21.15.94:0/3641458282 shutdown_connections 2024-02-17T12:03:59.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:03:59.254+0000 7fc669088640 1 -- 172.21.15.94:0/3641458282 wait complete. 2024-02-17T12:03:59.956 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:03:59 smithi094 ceph-mon[32499]: pgmap v180: 97 pgs: 97 active+clean; 4.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2024-02-17T12:04:00.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:03:59 smithi189 ceph-mon[36588]: pgmap v180: 97 pgs: 97 active+clean; 4.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2024-02-17T12:04:00.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:03:59 smithi125 ceph-mon[36819]: pgmap v180: 97 pgs: 97 active+clean; 4.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 16 MiB/s wr, 1.03k op/s 2024-02-17T12:04:00.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 340.0 s, 2261.6 tps, lat 44.200 ms stddev 49.463 2024-02-17T12:04:00.269 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:04:00.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:04:00.269 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:04:00.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.425+0000 7fc693748640 1 -- 172.21.15.94:0/178972659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc68c108660 msgr2=0x7fc68c110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.425+0000 7fc693748640 1 --2- 172.21.15.94:0/178972659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc68c108660 0x7fc68c110230 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc684009cd0 tx=0x7fc68402c0e0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.425+0000 7fc693748640 1 -- 172.21.15.94:0/178972659 shutdown_connections 2024-02-17T12:04:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.425+0000 7fc693748640 1 --2- 172.21.15.94:0/178972659 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc68c108660 0x7fc68c110230 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.425+0000 7fc693748640 1 --2- 172.21.15.94:0/178972659 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc68c103f40 0x7fc68c107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.425+0000 7fc693748640 1 --2- 172.21.15.94:0/178972659 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c103590 0x7fc68c103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.425+0000 7fc693748640 1 -- 172.21.15.94:0/178972659 >> 172.21.15.94:0/178972659 conn(0x7fc68c0fedf0 msgr2=0x7fc68c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.426+0000 7fc693748640 1 -- 172.21.15.94:0/178972659 shutdown_connections 2024-02-17T12:04:00.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.426+0000 7fc693748640 1 -- 172.21.15.94:0/178972659 wait complete. 2024-02-17T12:04:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.427+0000 7fc693748640 1 Processor -- start 2024-02-17T12:04:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.427+0000 7fc693748640 1 -- start start 2024-02-17T12:04:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.427+0000 7fc693748640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc68c103590 0x7fc68c19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.427+0000 7fc693748640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c108660 0x7fc68c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.427+0000 7fc693748640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc68c1997f0 0x7fc68c199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.427+0000 7fc693748640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c19a2a0 con 0x7fc68c1997f0 2024-02-17T12:04:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.427+0000 7fc693748640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c19a410 con 0x7fc68c103590 2024-02-17T12:04:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.427+0000 7fc693748640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68c19a580 con 0x7fc68c108660 2024-02-17T12:04:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.427+0000 7fc6914e4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc68c103590 0x7fc68c19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.427+0000 7fc6914e4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc68c103590 0x7fc68c19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44856/0 (socket says 172.21.15.94:44856) 2024-02-17T12:04:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.427+0000 7fc6914e4640 1 -- 172.21.15.94:0/3961951915 learned_addr learned my addr 172.21.15.94:0/3961951915 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.427+0000 7fc691ce5640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc68c1997f0 0x7fc68c199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.428+0000 7fc690ce3640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c108660 0x7fc68c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.428+0000 7fc6914e4640 1 -- 172.21.15.94:0/3961951915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c108660 msgr2=0x7fc68c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.428+0000 7fc6914e4640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c108660 0x7fc68c19fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.428+0000 7fc6914e4640 1 -- 172.21.15.94:0/3961951915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc68c1997f0 msgr2=0x7fc68c199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.428+0000 7fc6914e4640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc68c1997f0 0x7fc68c199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.428+0000 7fc6914e4640 1 -- 172.21.15.94:0/3961951915 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc684009900 con 0x7fc68c103590 2024-02-17T12:04:00.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.428+0000 7fc690ce3640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c108660 0x7fc68c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:04:00.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.428+0000 7fc6914e4640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc68c103590 0x7fc68c19f680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc68800e9f0 tx=0x7fc68800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:00.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.428+0000 7fc6827fc640 1 -- 172.21.15.94:0/3961951915 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688009700 con 0x7fc68c103590 2024-02-17T12:04:00.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.428+0000 7fc6827fc640 1 -- 172.21.15.94:0/3961951915 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc68800cd80 con 0x7fc68c103590 2024-02-17T12:04:00.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.428+0000 7fc6827fc640 1 -- 172.21.15.94:0/3961951915 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc688018920 con 0x7fc68c103590 2024-02-17T12:04:00.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.428+0000 7fc693748640 1 -- 172.21.15.94:0/3961951915 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68c1a6530 con 0x7fc68c103590 2024-02-17T12:04:00.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.428+0000 7fc693748640 1 -- 172.21.15.94:0/3961951915 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68c1a69c0 con 0x7fc68c103590 2024-02-17T12:04:00.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.431+0000 7fc6827fc640 1 -- 172.21.15.94:0/3961951915 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc688018a80 con 0x7fc68c103590 2024-02-17T12:04:00.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.432+0000 7fc6963f9740 1 Processor -- start 2024-02-17T12:04:00.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.432+0000 7fc6963f9740 1 -- start start 2024-02-17T12:04:00.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.432+0000 7fc6963f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 0x55e1b09a0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.432+0000 7fc6963f9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.432+0000 7fc6963f9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09b20c0 0x55e1b09b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.432+0000 7fc6963f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b090ef90 con 0x55e1b09a0320 2024-02-17T12:04:00.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.432+0000 7fc6963f9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b090c4c0 con 0x55e1b09a0cd0 2024-02-17T12:04:00.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.432+0000 7fc6963f9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b0907ca0 con 0x55e1b09b20c0 2024-02-17T12:04:00.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.432+0000 7fc6827fc640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6780a2330 0x7fc6780a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.432+0000 7fc691ce5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 0x55e1b09a0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.432+0000 7fc691ce5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 0x55e1b09a0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48204/0 (socket says 172.21.15.94:48204) 2024-02-17T12:04:00.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.432+0000 7fc691ce5640 1 -- 172.21.15.94:0/1195495329 learned_addr learned my addr 172.21.15.94:0/1195495329 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:00.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.433+0000 7fc690ce3640 1 --2- 172.21.15.94:0/1195495329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09b20c0 0x55e1b09b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.433+0000 7fc691ce5640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6780a2330 0x7fc6780a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.433+0000 7fc6827fc640 1 -- 172.21.15.94:0/3961951915 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc688014070 con 0x7fc68c103590 2024-02-17T12:04:00.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.433+0000 7fc691ce5640 1 -- 172.21.15.94:0/1195495329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09b20c0 msgr2=0x55e1b09b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.433+0000 7fc691ce5640 1 --2- 172.21.15.94:0/1195495329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09b20c0 0x55e1b09b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.433+0000 7fc691ce5640 1 -- 172.21.15.94:0/1195495329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 msgr2=0x55e1b09b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.433+0000 7fc6914e4640 1 --2- 172.21.15.94:0/1195495329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.433+0000 7fc691ce5640 1 --2- 172.21.15.94:0/1195495329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.433+0000 7fc691ce5640 1 -- 172.21.15.94:0/1195495329 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1b08ca0b0 con 0x55e1b09a0320 2024-02-17T12:04:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.433+0000 7fc6914e4640 1 --2- 172.21.15.94:0/1195495329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:04:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.434+0000 7fc691ce5640 1 --2- 172.21.15.94:0/1195495329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 0x55e1b09a0700 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc684009430 tx=0x7fc684032610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e7edfbadd3cc87 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.434+0000 7fc691ce5640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6780a2330 0x7fc6780a47f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc684048000 tx=0x7fc68404b000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.435+0000 7fc6777fe640 1 -- 172.21.15.94:0/1195495329 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6840326b0 con 0x55e1b09a0320 2024-02-17T12:04:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.435+0000 7fc6777fe640 1 -- 172.21.15.94:0/1195495329 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc684003e50 con 0x55e1b09a0320 2024-02-17T12:04:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.435+0000 7fc6777fe640 1 -- 172.21.15.94:0/1195495329 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6840043c0 con 0x55e1b09a0320 2024-02-17T12:04:00.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.436+0000 7fc6963f9740 1 -- 172.21.15.94:0/1195495329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 msgr2=0x55e1b09a0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.436+0000 7fc6963f9740 1 --2- 172.21.15.94:0/1195495329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 0x55e1b09a0700 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fc684009430 tx=0x7fc684032610 comp rx=0 tx=0).stop 2024-02-17T12:04:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.436+0000 7fc6963f9740 1 -- 172.21.15.94:0/1195495329 shutdown_connections 2024-02-17T12:04:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.436+0000 7fc6963f9740 1 --2- 172.21.15.94:0/1195495329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09b20c0 0x55e1b09b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.436+0000 7fc6963f9740 1 --2- 172.21.15.94:0/1195495329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.436+0000 7fc6963f9740 1 --2- 172.21.15.94:0/1195495329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 0x55e1b09a0700 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.436+0000 7fc6963f9740 1 -- 172.21.15.94:0/1195495329 >> 172.21.15.94:0/1195495329 conn(0x55e1b099a840 msgr2=0x55e1b099ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.436+0000 7fc6963f9740 1 -- 172.21.15.94:0/1195495329 shutdown_connections 2024-02-17T12:04:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.436+0000 7fc6963f9740 1 -- 172.21.15.94:0/1195495329 wait complete. 2024-02-17T12:04:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.437+0000 7fc6963f9740 1 -- start start 2024-02-17T12:04:00.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.437+0000 7fc6963f9740 1 Processor -- start 2024-02-17T12:04:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.437+0000 7fc6963f9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09a0320 0x55e1b09af150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc6914e4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09a0320 0x55e1b09af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc6914e4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09a0320 0x55e1b09af150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33824/0 (socket says 172.21.15.94:33824) 2024-02-17T12:04:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.437+0000 7fc6963f9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc6914e4640 1 -- 172.21.15.94:0/3641378566 learned_addr learned my addr 172.21.15.94:0/3641378566 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc6963f9740 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09b20c0 0x55e1b09afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc6963f9740 1 -- 172.21.15.94:0/3641378566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b08acf40 con 0x55e1b09b20c0 2024-02-17T12:04:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc6963f9740 1 -- 172.21.15.94:0/3641378566 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b05db4b0 con 0x55e1b09a0cd0 2024-02-17T12:04:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc6963f9740 1 -- 172.21.15.94:0/3641378566 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b088d670 con 0x55e1b09a0320 2024-02-17T12:04:00.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc6963f9740 10 client.0 _ll_register_callbacks cb 0x55e1b08faf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc690ce3640 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc691ce5640 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09b20c0 0x55e1b09afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc690ce3640 1 -- 172.21.15.94:0/3641378566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09a0320 msgr2=0x55e1b09af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc690ce3640 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09a0320 0x55e1b09af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc690ce3640 1 -- 172.21.15.94:0/3641378566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09b20c0 msgr2=0x55e1b09afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc690ce3640 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09b20c0 0x55e1b09afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.438+0000 7fc690ce3640 1 -- 172.21.15.94:0/3641378566 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68c110a40 con 0x55e1b09a0cd0 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.439+0000 7fc691ce5640 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09b20c0 0x55e1b09afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.439+0000 7fc690ce3640 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09af690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc68c109ab0 tx=0x7fc67c013940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c2bbc4045fb33c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.439+0000 7fc674ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.439+0000 7fc674ff9640 1 -- 172.21.15.94:0/3641378566 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc67c004300 con 0x55e1b09a0cd0 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.439+0000 7fc674ff9640 1 -- 172.21.15.94:0/3641378566 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc67c004460 con 0x55e1b09a0cd0 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.439+0000 7fc6963f9740 1 -- 172.21.15.94:0/3641378566 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1b08e1f40 con 0x55e1b09a0cd0 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.439+0000 7fc674ff9640 1 -- 172.21.15.94:0/3641378566 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc67c0035f0 con 0x55e1b09a0cd0 2024-02-17T12:04:00.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.440+0000 7fc6963f9740 10 client.26528 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:00.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.440+0000 7fc6963f9740 1 -- 172.21.15.94:0/3641378566 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e1b0906320 con 0x55e1b09a0cd0 2024-02-17T12:04:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.440+0000 7fc674ff9640 1 -- 172.21.15.94:0/3641378566 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e1b08e1f40 con 0x55e1b09a0cd0 2024-02-17T12:04:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.440+0000 7fc674ff9640 1 -- 172.21.15.94:0/3641378566 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc67c011b30 con 0x55e1b09a0cd0 2024-02-17T12:04:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.440+0000 7fc6963f9740 10 client.26528 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:00.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.440+0000 7fc6963f9740 10 client.26528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.440+0000 7fc6963f9740 20 client.26528 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.440+0000 7fc6963f9740 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e1b09c2640 0x55e1b09c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.440+0000 7fc6963f9740 4 client.26528 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.440+0000 7fc6963f9740 1 -- 172.21.15.94:0/3641378566 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1b08fe310 con 0x55e1b09c2640 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.441+0000 7fc6914e4640 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e1b09c2640 0x55e1b09c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.441+0000 7fc674ff9640 10 client.26528 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.441+0000 7fc6914e4640 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e1b09c2640 0x55e1b09c4a30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eece0d527869f97 server_cookie=94c7725c912ec864 in_seq=0 out_seq=0 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.442+0000 7fc674ff9640 1 -- 172.21.15.94:0/3641378566 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e1b08fe310 con 0x55e1b09c2640 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.442+0000 7fc674ff9640 10 client.26528 handle_command_reply: tid=0 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.442+0000 7fc6963f9740 1 client.26528 shutdown 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.442+0000 7fc6963f9740 1 -- 172.21.15.94:0/3641378566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 msgr2=0x55e1b09af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.442+0000 7fc6963f9740 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09af690 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc68c109ab0 tx=0x7fc67c013940 comp rx=0 tx=0).stop 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.442+0000 7fc6963f9740 1 -- 172.21.15.94:0/3641378566 shutdown_connections 2024-02-17T12:04:00.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.442+0000 7fc6963f9740 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e1b09c2640 0x55e1b09c4a30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.442+0000 7fc6963f9740 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09b20c0 0x55e1b09afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.442+0000 7fc6963f9740 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09af690 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.442+0000 7fc6963f9740 1 --2- 172.21.15.94:0/3641378566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09a0320 0x55e1b09af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.442+0000 7fc6963f9740 1 -- 172.21.15.94:0/3641378566 >> 172.21.15.94:0/3641378566 conn(0x55e1b099a840 msgr2=0x55e1b099c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:00.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.442+0000 7fc674ff9640 0 client.26528 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:00.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.442+0000 7fc6963f9740 1 -- 172.21.15.94:0/3641378566 shutdown_connections 2024-02-17T12:04:00.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.443+0000 7fc6963f9740 1 -- 172.21.15.94:0/3641378566 wait complete. 2024-02-17T12:04:00.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.443+0000 7fc6963f9740 20 client.26528 trim_cache size 0 max 16384 2024-02-17T12:04:00.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.466+0000 7fc6963f9740 1 Processor -- start 2024-02-17T12:04:00.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.466+0000 7fc6963f9740 1 -- start start 2024-02-17T12:04:00.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc6963f9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09a0320 0x55e1b09b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc6963f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0cd0 0x55e1b09af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc6963f9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09b20c0 0x55e1b09ada20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc6963f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b099ba90 con 0x55e1b09a0cd0 2024-02-17T12:04:00.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc6963f9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b09a10b0 con 0x55e1b09b20c0 2024-02-17T12:04:00.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc6963f9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b09c9c40 con 0x55e1b09a0320 2024-02-17T12:04:00.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc6914e4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09a0320 0x55e1b09b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc691ce5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09b20c0 0x55e1b09ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc690ce3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0cd0 0x55e1b09af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc691ce5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09b20c0 0x55e1b09ada20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44876/0 (socket says 172.21.15.94:44876) 2024-02-17T12:04:00.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc6914e4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09a0320 0x55e1b09b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:33828/0 (socket says 172.21.15.94:33828) 2024-02-17T12:04:00.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc691ce5640 1 -- 172.21.15.94:0/659880529 learned_addr learned my addr 172.21.15.94:0/659880529 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:00.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc691ce5640 1 -- 172.21.15.94:0/659880529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09a0320 msgr2=0x55e1b09b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc691ce5640 1 --2- 172.21.15.94:0/659880529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09a0320 0x55e1b09b1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc691ce5640 1 -- 172.21.15.94:0/659880529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0cd0 msgr2=0x55e1b09af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc691ce5640 1 --2- 172.21.15.94:0/659880529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0cd0 0x55e1b09af9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc691ce5640 1 -- 172.21.15.94:0/659880529 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1b08a2c00 con 0x55e1b09b20c0 2024-02-17T12:04:00.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.467+0000 7fc6914e4640 1 --2- 172.21.15.94:0/659880529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09a0320 0x55e1b09b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:00.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.468+0000 7fc691ce5640 1 --2- 172.21.15.94:0/659880529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09b20c0 0x55e1b09ada20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc684009430 tx=0x7fc68402c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237e2002ccd72fed server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:00.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.468+0000 7fc676ffd640 1 -- 172.21.15.94:0/659880529 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6840329c0 con 0x55e1b09b20c0 2024-02-17T12:04:00.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.468+0000 7fc676ffd640 1 -- 172.21.15.94:0/659880529 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc68405c070 con 0x55e1b09b20c0 2024-02-17T12:04:00.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.468+0000 7fc676ffd640 1 -- 172.21.15.94:0/659880529 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684004610 con 0x55e1b09b20c0 2024-02-17T12:04:00.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.469+0000 7fc6963f9740 1 -- 172.21.15.94:0/659880529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09b20c0 msgr2=0x55e1b09ada20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.469+0000 7fc6963f9740 1 --2- 172.21.15.94:0/659880529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09b20c0 0x55e1b09ada20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc684009430 tx=0x7fc68402c760 comp rx=0 tx=0).stop 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.469+0000 7fc6963f9740 1 -- 172.21.15.94:0/659880529 shutdown_connections 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.469+0000 7fc6963f9740 1 --2- 172.21.15.94:0/659880529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09b20c0 0x55e1b09ada20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.469+0000 7fc6963f9740 1 --2- 172.21.15.94:0/659880529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0cd0 0x55e1b09af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.469+0000 7fc6963f9740 1 --2- 172.21.15.94:0/659880529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09a0320 0x55e1b09b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.469+0000 7fc6963f9740 1 -- 172.21.15.94:0/659880529 >> 172.21.15.94:0/659880529 conn(0x55e1b099a840 msgr2=0x55e1b09aa4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.469+0000 7fc6963f9740 1 -- 172.21.15.94:0/659880529 shutdown_connections 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.469+0000 7fc6963f9740 1 -- 172.21.15.94:0/659880529 wait complete. 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.470+0000 7fc6963f9740 1 -- start start 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.470+0000 7fc6963f9740 1 Processor -- start 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6963f9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 0x55e1b09b92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6963f9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09b7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6963f9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09b20c0 0x55e1b09b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6914e4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 0x55e1b09b92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6963f9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b09b8160 con 0x55e1b09a0320 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc690ce3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6914e4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 0x55e1b09b92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48218/0 (socket says 172.21.15.94:48218) 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6914e4640 1 -- 172.21.15.94:0/840083016 learned_addr learned my addr 172.21.15.94:0/840083016 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6963f9740 1 -- 172.21.15.94:0/840083016 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b09b82d0 con 0x55e1b09a0cd0 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6963f9740 1 -- 172.21.15.94:0/840083016 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1b09b8490 con 0x55e1b09b20c0 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6963f9740 10 client.0 _ll_register_callbacks cb 0x7fc67c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc691ce5640 1 --2- 172.21.15.94:0/840083016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09b20c0 0x55e1b09b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6914e4640 1 -- 172.21.15.94:0/840083016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09b20c0 msgr2=0x55e1b09b7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6914e4640 1 --2- 172.21.15.94:0/840083016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09b20c0 0x55e1b09b7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6914e4640 1 -- 172.21.15.94:0/840083016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 msgr2=0x55e1b09b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6914e4640 1 --2- 172.21.15.94:0/840083016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09b7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc6914e4640 1 -- 172.21.15.94:0/840083016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68c1a69c0 con 0x55e1b09a0320 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.471+0000 7fc690ce3640 1 --2- 172.21.15.94:0/840083016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc6914e4640 1 --2- 172.21.15.94:0/840083016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 0x55e1b09b92c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc688000c00 tx=0x7fc6880ce3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9664b2b2008f2343 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:00.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc674ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:04:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc674ff9640 1 -- 172.21.15.94:0/840083016 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6880aeea0 con 0x55e1b09a0320 2024-02-17T12:04:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc6963f9740 1 -- 172.21.15.94:0/840083016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1b092d900 con 0x55e1b09a0320 2024-02-17T12:04:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc674ff9640 1 -- 172.21.15.94:0/840083016 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc6880b3070 con 0x55e1b09a0320 2024-02-17T12:04:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc674ff9640 1 -- 172.21.15.94:0/840083016 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6880a35f0 con 0x55e1b09a0320 2024-02-17T12:04:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc6963f9740 10 client.17073 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc6963f9740 1 -- 172.21.15.94:0/840083016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e1b08ca0b0 con 0x55e1b09a0320 2024-02-17T12:04:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc674ff9640 1 -- 172.21.15.94:0/840083016 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e1b092d900 con 0x55e1b09a0320 2024-02-17T12:04:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc674ff9640 1 -- 172.21.15.94:0/840083016 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc6880b91b0 con 0x55e1b09a0320 2024-02-17T12:04:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc6963f9740 10 client.17073 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc6963f9740 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc6963f9740 20 client.17073 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc6963f9740 1 --2- 172.21.15.94:0/840083016 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e1b0a1a170 0x55e1b0a1c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc6963f9740 4 client.17073 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:04:00.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.472+0000 7fc6963f9740 1 -- 172.21.15.94:0/840083016 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e1b0607860 con 0x55e1b0a1a170 2024-02-17T12:04:00.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.473+0000 7fc690ce3640 1 --2- 172.21.15.94:0/840083016 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e1b0a1a170 0x55e1b0a1c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.473+0000 7fc674ff9640 10 client.17073 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:00.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.473+0000 7fc690ce3640 1 --2- 172.21.15.94:0/840083016 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e1b0a1a170 0x55e1b0a1c590 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4ad2c4a77bc4744 server_cookie=d0f3873004960ab2 in_seq=0 out_seq=0 2024-02-17T12:04:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.474+0000 7fc674ff9640 1 -- 172.21.15.94:0/840083016 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e1b0607860 con 0x55e1b0a1a170 2024-02-17T12:04:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.474+0000 7fc674ff9640 10 client.17073 handle_command_reply: tid=0 2024-02-17T12:04:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.474+0000 7fc6963f9740 1 client.17073 shutdown 2024-02-17T12:04:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.474+0000 7fc6963f9740 1 -- 172.21.15.94:0/840083016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 msgr2=0x55e1b09b92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.474+0000 7fc6963f9740 1 --2- 172.21.15.94:0/840083016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 0x55e1b09b92c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc688000c00 tx=0x7fc6880ce3a0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.474+0000 7fc6963f9740 1 -- 172.21.15.94:0/840083016 shutdown_connections 2024-02-17T12:04:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.474+0000 7fc6963f9740 1 --2- 172.21.15.94:0/840083016 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e1b0a1a170 0x55e1b0a1c590 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.474+0000 7fc6963f9740 1 --2- 172.21.15.94:0/840083016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1b09b20c0 0x55e1b09b7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.474+0000 7fc6963f9740 1 --2- 172.21.15.94:0/840083016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e1b09a0cd0 0x55e1b09b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.474+0000 7fc6963f9740 1 --2- 172.21.15.94:0/840083016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e1b09a0320 0x55e1b09b92c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.474+0000 7fc674ff9640 0 client.17073 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.474+0000 7fc6963f9740 1 -- 172.21.15.94:0/840083016 >> 172.21.15.94:0/840083016 conn(0x55e1b099a840 msgr2=0x55e1b09ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.475+0000 7fc6963f9740 1 -- 172.21.15.94:0/840083016 shutdown_connections 2024-02-17T12:04:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.475+0000 7fc6963f9740 1 -- 172.21.15.94:0/840083016 wait complete. 2024-02-17T12:04:00.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.475+0000 7fc6963f9740 20 client.17073 trim_cache size 0 max 16384 2024-02-17T12:04:00.479 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:04:00.479 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:04:00.479 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "508760f9-0d30-4a2b-9207-aed3211228a9", 2024-02-17T12:04:00.479 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:04:00.479 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:04:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.476+0000 7fc693748640 1 -- 172.21.15.94:0/3961951915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6780a2330 msgr2=0x7fc6780a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.476+0000 7fc693748640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6780a2330 0x7fc6780a47f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc684048000 tx=0x7fc68404b000 comp rx=0 tx=0).stop 2024-02-17T12:04:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.476+0000 7fc693748640 1 -- 172.21.15.94:0/3961951915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc68c103590 msgr2=0x7fc68c19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.476+0000 7fc693748640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc68c103590 0x7fc68c19f680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc68800e9f0 tx=0x7fc68800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.476+0000 7fc693748640 1 -- 172.21.15.94:0/3961951915 shutdown_connections 2024-02-17T12:04:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.476+0000 7fc693748640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc6780a2330 0x7fc6780a47f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.476+0000 7fc693748640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc68c1997f0 0x7fc68c199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.476+0000 7fc693748640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc68c108660 0x7fc68c19fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.476+0000 7fc693748640 1 --2- 172.21.15.94:0/3961951915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc68c103590 0x7fc68c19f680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.476+0000 7fc693748640 1 -- 172.21.15.94:0/3961951915 >> 172.21.15.94:0/3961951915 conn(0x7fc68c0fedf0 msgr2=0x7fc68c1001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:00.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.476+0000 7fc693748640 1 -- 172.21.15.94:0/3961951915 shutdown_connections 2024-02-17T12:04:00.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.476+0000 7fc693748640 1 -- 172.21.15.94:0/3961951915 wait complete. 2024-02-17T12:04:00.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:04:00.491 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:04:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.616+0000 7ff41aee1640 1 -- 172.21.15.94:0/3546415706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff414105340 msgr2=0x7ff41410cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.616+0000 7ff41aee1640 1 --2- 172.21.15.94:0/3546415706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff414105340 0x7ff41410cf10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009cd0 tx=0x7ff40c02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.617+0000 7ff41aee1640 1 -- 172.21.15.94:0/3546415706 shutdown_connections 2024-02-17T12:04:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.617+0000 7ff41aee1640 1 --2- 172.21.15.94:0/3546415706 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff414105340 0x7ff41410cf10 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.617+0000 7ff41aee1640 1 --2- 172.21.15.94:0/3546415706 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff414100c20 0x7ff414104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.617+0000 7ff41aee1640 1 --2- 172.21.15.94:0/3546415706 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff414100270 0x7ff414100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.617+0000 7ff41aee1640 1 -- 172.21.15.94:0/3546415706 >> 172.21.15.94:0/3546415706 conn(0x7ff414077230 msgr2=0x7ff414077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:00.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.617+0000 7ff41aee1640 1 -- 172.21.15.94:0/3546415706 shutdown_connections 2024-02-17T12:04:00.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.617+0000 7ff41aee1640 1 -- 172.21.15.94:0/3546415706 wait complete. 2024-02-17T12:04:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.617+0000 7ff41aee1640 1 Processor -- start 2024-02-17T12:04:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.617+0000 7ff41aee1640 1 -- start start 2024-02-17T12:04:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff41aee1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff414100270 0x7ff414197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff41aee1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff414105340 0x7ff414197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff41aee1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff41419c020 0x7ff41419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff41aee1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41419ca10 con 0x7ff414100270 2024-02-17T12:04:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff41aee1640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41419cb80 con 0x7ff414105340 2024-02-17T12:04:00.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff41aee1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41419ccf0 con 0x7ff41419c020 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff418c7d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff414100270 0x7ff414197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff413fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff414105340 0x7ff414197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff418c7d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff414100270 0x7ff414197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48240/0 (socket says 172.21.15.94:48240) 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff418c7d640 1 -- 172.21.15.94:0/2022943337 learned_addr learned my addr 172.21.15.94:0/2022943337 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff41947e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff41419c020 0x7ff41419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff413fff640 1 -- 172.21.15.94:0/2022943337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff41419c020 msgr2=0x7ff41419c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff413fff640 1 --2- 172.21.15.94:0/2022943337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff41419c020 0x7ff41419c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff413fff640 1 -- 172.21.15.94:0/2022943337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff414100270 msgr2=0x7ff414197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff413fff640 1 --2- 172.21.15.94:0/2022943337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff414100270 0x7ff414197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff413fff640 1 -- 172.21.15.94:0/2022943337 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c009900 con 0x7ff414105340 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.618+0000 7ff413fff640 1 --2- 172.21.15.94:0/2022943337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff414105340 0x7ff414197ca0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff400005fd0 tx=0x7ff40000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.619+0000 7ff411ffb640 1 -- 172.21.15.94:0/2022943337 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400009c00 con 0x7ff414105340 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.619+0000 7ff41aee1640 1 -- 172.21.15.94:0/2022943337 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4141a42d0 con 0x7ff414105340 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.619+0000 7ff411ffb640 1 -- 172.21.15.94:0/2022943337 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff400004510 con 0x7ff414105340 2024-02-17T12:04:00.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.619+0000 7ff411ffb640 1 -- 172.21.15.94:0/2022943337 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400003e80 con 0x7ff414105340 2024-02-17T12:04:00.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.619+0000 7ff41aee1640 1 -- 172.21.15.94:0/2022943337 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4141a4780 con 0x7ff414105340 2024-02-17T12:04:00.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff41db8b740 1 Processor -- start 2024-02-17T12:04:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff411ffb640 1 -- 172.21.15.94:0/2022943337 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff400004000 con 0x7ff414105340 2024-02-17T12:04:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff41db8b740 1 -- start start 2024-02-17T12:04:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff41db8b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd6305690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff41db8b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd6305bd0 0x556fd630e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff41db8b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd630e7b0 0x556fd6310ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff41db8b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd6274280 con 0x556fd6305bd0 2024-02-17T12:04:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff41db8b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd62717b0 con 0x556fd63052b0 2024-02-17T12:04:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff41db8b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd626cf90 con 0x556fd630e7b0 2024-02-17T12:04:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff41947e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd6305690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff41947e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd6305690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44912/0 (socket says 172.21.15.94:44912) 2024-02-17T12:04:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff41947e640 1 -- 172.21.15.94:0/3602129951 learned_addr learned my addr 172.21.15.94:0/3602129951 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff418c7d640 1 --2- 172.21.15.94:0/3602129951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd6305bd0 0x556fd630e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.622+0000 7ff413fff640 1 --2- 172.21.15.94:0/3602129951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd630e7b0 0x556fd6310ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.623+0000 7ff41947e640 1 -- 172.21.15.94:0/3602129951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd630e7b0 msgr2=0x556fd6310ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.623+0000 7ff41947e640 1 --2- 172.21.15.94:0/3602129951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd630e7b0 0x556fd6310ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.623+0000 7ff41947e640 1 -- 172.21.15.94:0/3602129951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd6305bd0 msgr2=0x556fd630e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.623+0000 7ff41947e640 1 --2- 172.21.15.94:0/3602129951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd6305bd0 0x556fd630e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.623+0000 7ff411ffb640 1 --2- 172.21.15.94:0/2022943337 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff4040a2230 0x7ff4040a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.623+0000 7ff41947e640 1 -- 172.21.15.94:0/3602129951 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fd622b570 con 0x556fd63052b0 2024-02-17T12:04:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.623+0000 7ff413fff640 1 --2- 172.21.15.94:0/3602129951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd630e7b0 0x556fd6310ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:04:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.623+0000 7ff411ffb640 1 -- 172.21.15.94:0/2022943337 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff4000cd740 con 0x7ff414105340 2024-02-17T12:04:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.623+0000 7ff41947e640 1 --2- 172.21.15.94:0/2022943337 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff4040a2230 0x7ff4040a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.623+0000 7ff41947e640 1 --2- 172.21.15.94:0/3602129951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd6305690 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff40c02c700 tx=0x7ff40c032d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ebe47cbc1c09358 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.623+0000 7ff3feffd640 1 -- 172.21.15.94:0/3602129951 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c034640 con 0x556fd63052b0 2024-02-17T12:04:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.623+0000 7ff3feffd640 1 -- 172.21.15.94:0/3602129951 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff40c003970 con 0x556fd63052b0 2024-02-17T12:04:00.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.623+0000 7ff3feffd640 1 -- 172.21.15.94:0/3602129951 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c004340 con 0x556fd63052b0 2024-02-17T12:04:00.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.624+0000 7ff41db8b740 1 -- 172.21.15.94:0/3602129951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 msgr2=0x556fd6305690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.624+0000 7ff41db8b740 1 --2- 172.21.15.94:0/3602129951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd6305690 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff40c02c700 tx=0x7ff40c032d70 comp rx=0 tx=0).stop 2024-02-17T12:04:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.624+0000 7ff41947e640 1 --2- 172.21.15.94:0/2022943337 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff4040a2230 0x7ff4040a46f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff40c04b000 tx=0x7ff40c04e000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.624+0000 7ff41db8b740 1 -- 172.21.15.94:0/3602129951 shutdown_connections 2024-02-17T12:04:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.624+0000 7ff41db8b740 1 --2- 172.21.15.94:0/3602129951 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd630e7b0 0x556fd6310ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.624+0000 7ff41db8b740 1 --2- 172.21.15.94:0/3602129951 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd6305bd0 0x556fd630e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.624+0000 7ff41db8b740 1 --2- 172.21.15.94:0/3602129951 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd6305690 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.624+0000 7ff41db8b740 1 -- 172.21.15.94:0/3602129951 >> 172.21.15.94:0/3602129951 conn(0x556fd62ff990 msgr2=0x556fd62ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.624+0000 7ff41db8b740 1 -- 172.21.15.94:0/3602129951 shutdown_connections 2024-02-17T12:04:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.624+0000 7ff41db8b740 1 -- 172.21.15.94:0/3602129951 wait complete. 2024-02-17T12:04:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.624+0000 7ff41db8b740 1 -- start start 2024-02-17T12:04:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.624+0000 7ff41db8b740 1 Processor -- start 2024-02-17T12:04:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.625+0000 7ff41db8b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd630e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.625+0000 7ff41db8b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd6305bd0 0x556fd630b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.625+0000 7ff41db8b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd630e7b0 0x556fd630bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.625+0000 7ff41db8b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd5f40430 con 0x556fd630e7b0 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.625+0000 7ff41db8b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd5efce00 con 0x556fd63052b0 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.625+0000 7ff41db8b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd61f1fd0 con 0x556fd6305bd0 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.625+0000 7ff41db8b740 10 client.0 _ll_register_callbacks cb 0x556fd6247fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.625+0000 7ff418c7d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd630e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.625+0000 7ff41947e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd630e7b0 0x556fd630bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.625+0000 7ff418c7d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd630e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44914/0 (socket says 172.21.15.94:44914) 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.625+0000 7ff418c7d640 1 -- 172.21.15.94:0/2462026409 learned_addr learned my addr 172.21.15.94:0/2462026409 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.625+0000 7ff413fff640 1 --2- 172.21.15.94:0/2462026409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd6305bd0 0x556fd630b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff418c7d640 1 -- 172.21.15.94:0/2462026409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd6305bd0 msgr2=0x556fd630b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff418c7d640 1 --2- 172.21.15.94:0/2462026409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd6305bd0 0x556fd630b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff418c7d640 1 -- 172.21.15.94:0/2462026409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd630e7b0 msgr2=0x556fd630bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff418c7d640 1 --2- 172.21.15.94:0/2462026409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd630e7b0 0x556fd630bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff418c7d640 1 -- 172.21.15.94:0/2462026409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41410d720 con 0x556fd63052b0 2024-02-17T12:04:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff41947e640 1 --2- 172.21.15.94:0/2462026409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd630e7b0 0x556fd630bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff418c7d640 1 --2- 172.21.15.94:0/2462026409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd630e210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff408012790 tx=0x7ff408012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=432268530355ec74 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff3f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:04:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff3f3fff640 1 -- 172.21.15.94:0/2462026409 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408017070 con 0x556fd63052b0 2024-02-17T12:04:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff3f3fff640 1 -- 172.21.15.94:0/2462026409 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff408004050 con 0x556fd63052b0 2024-02-17T12:04:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff41db8b740 1 -- 172.21.15.94:0/2462026409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556fd5f6c7e0 con 0x556fd63052b0 2024-02-17T12:04:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff3f3fff640 1 -- 172.21.15.94:0/2462026409 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff408004210 con 0x556fd63052b0 2024-02-17T12:04:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff41db8b740 10 client.26546 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff41db8b740 1 -- 172.21.15.94:0/2462026409 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556fd626b610 con 0x556fd63052b0 2024-02-17T12:04:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff3f3fff640 1 -- 172.21.15.94:0/2462026409 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556fd5f6c7e0 con 0x556fd63052b0 2024-02-17T12:04:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff3f3fff640 1 -- 172.21.15.94:0/2462026409 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff4080039c0 con 0x556fd63052b0 2024-02-17T12:04:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff41db8b740 10 client.26546 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:00.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff41db8b740 10 client.26546 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff41db8b740 20 client.26546 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2462026409 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556fd6322e50 0x556fd6325270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff41db8b740 4 client.26546 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.626+0000 7ff41db8b740 1 -- 172.21.15.94:0/2462026409 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fd5fa9720 con 0x556fd6322e50 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.627+0000 7ff413fff640 1 --2- 172.21.15.94:0/2462026409 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556fd6322e50 0x556fd6325270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.627+0000 7ff3f3fff640 10 client.26546 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.627+0000 7ff413fff640 1 --2- 172.21.15.94:0/2462026409 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556fd6322e50 0x556fd6325270 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=794355ce82613dd2 server_cookie=620ab3a7d10851ba in_seq=0 out_seq=0 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff3f3fff640 1 -- 172.21.15.94:0/2462026409 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556fd5fa9720 con 0x556fd6322e50 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff3f3fff640 10 client.26546 handle_command_reply: tid=0 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff41db8b740 1 client.26546 shutdown 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff41db8b740 1 -- 172.21.15.94:0/2462026409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 msgr2=0x556fd630e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2462026409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd630e210 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff408012790 tx=0x7ff408012b90 comp rx=0 tx=0).stop 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff41db8b740 1 -- 172.21.15.94:0/2462026409 shutdown_connections 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2462026409 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556fd6322e50 0x556fd6325270 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2462026409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd630e7b0 0x556fd630bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2462026409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd6305bd0 0x556fd630b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2462026409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd630e210 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff3f3fff640 0 client.26546 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff41db8b740 1 -- 172.21.15.94:0/2462026409 >> 172.21.15.94:0/2462026409 conn(0x556fd62ff990 msgr2=0x556fd6301730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff41db8b740 1 -- 172.21.15.94:0/2462026409 shutdown_connections 2024-02-17T12:04:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff41db8b740 1 -- 172.21.15.94:0/2462026409 wait complete. 2024-02-17T12:04:00.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.628+0000 7ff41db8b740 20 client.26546 trim_cache size 0 max 16384 2024-02-17T12:04:00.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.645+0000 7ff41db8b740 1 Processor -- start 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.645+0000 7ff41db8b740 1 -- start start 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.645+0000 7ff41db8b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd630b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.645+0000 7ff41db8b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd6305bd0 0x556fd630dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.645+0000 7ff41db8b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd630e7b0 0x556fd630e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.645+0000 7ff41db8b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd5f6c7e0 con 0x556fd6305bd0 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.645+0000 7ff41db8b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd6315890 con 0x556fd63052b0 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.645+0000 7ff41db8b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd62fb0f0 con 0x556fd630e7b0 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.645+0000 7ff413fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd6305bd0 0x556fd630dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.645+0000 7ff413fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd6305bd0 0x556fd630dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48260/0 (socket says 172.21.15.94:48260) 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.645+0000 7ff413fff640 1 -- 172.21.15.94:0/2495655217 learned_addr learned my addr 172.21.15.94:0/2495655217 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.645+0000 7ff418c7d640 1 --2- 172.21.15.94:0/2495655217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd630b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.645+0000 7ff41947e640 1 --2- 172.21.15.94:0/2495655217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd630e7b0 0x556fd630e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.646+0000 7ff418c7d640 1 -- 172.21.15.94:0/2495655217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd630e7b0 msgr2=0x556fd630e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.646+0000 7ff418c7d640 1 --2- 172.21.15.94:0/2495655217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd630e7b0 0x556fd630e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.646+0000 7ff418c7d640 1 -- 172.21.15.94:0/2495655217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd6305bd0 msgr2=0x556fd630dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.646+0000 7ff418c7d640 1 --2- 172.21.15.94:0/2495655217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd6305bd0 0x556fd630dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.646+0000 7ff418c7d640 1 -- 172.21.15.94:0/2495655217 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fd6207ac0 con 0x556fd63052b0 2024-02-17T12:04:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.646+0000 7ff413fff640 1 --2- 172.21.15.94:0/2495655217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd6305bd0 0x556fd630dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.646+0000 7ff418c7d640 1 --2- 172.21.15.94:0/2495655217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd630b540 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff408003e20 tx=0x7ff40800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c6b039259c55b6d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.646+0000 7ff3fe7fc640 1 -- 172.21.15.94:0/2495655217 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408017040 con 0x556fd63052b0 2024-02-17T12:04:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.646+0000 7ff3fe7fc640 1 -- 172.21.15.94:0/2495655217 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff408003ad0 con 0x556fd63052b0 2024-02-17T12:04:00.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.646+0000 7ff3fe7fc640 1 -- 172.21.15.94:0/2495655217 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff408005300 con 0x556fd63052b0 2024-02-17T12:04:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.647+0000 7ff41db8b740 1 -- 172.21.15.94:0/2495655217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 msgr2=0x556fd630b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.647+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2495655217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd630b540 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff408003e20 tx=0x7ff40800f790 comp rx=0 tx=0).stop 2024-02-17T12:04:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.647+0000 7ff41db8b740 1 -- 172.21.15.94:0/2495655217 shutdown_connections 2024-02-17T12:04:00.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.647+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2495655217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd630e7b0 0x556fd630e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.647+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2495655217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd6305bd0 0x556fd630dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.647+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2495655217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd630b540 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.647+0000 7ff41db8b740 1 -- 172.21.15.94:0/2495655217 >> 172.21.15.94:0/2495655217 conn(0x556fd62ff990 msgr2=0x556fd630ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.647+0000 7ff41db8b740 1 -- 172.21.15.94:0/2495655217 shutdown_connections 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.647+0000 7ff41db8b740 1 -- 172.21.15.94:0/2495655217 wait complete. 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.647+0000 7ff41db8b740 1 -- start start 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff41db8b740 1 Processor -- start 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff41db8b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd63186e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff41db8b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd6305bd0 0x556fd6318c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff41db8b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd630e7b0 0x556fd6319160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff41db8b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd63794a0 con 0x556fd630e7b0 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff41db8b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd6379610 con 0x556fd63052b0 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff41db8b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fd63797d0 con 0x556fd6305bd0 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff41db8b740 10 client.0 _ll_register_callbacks cb 0x7ff408005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:00.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff418c7d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd63186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff418c7d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd63186e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44930/0 (socket says 172.21.15.94:44930) 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff418c7d640 1 -- 172.21.15.94:0/2136576009 learned_addr learned my addr 172.21.15.94:0/2136576009 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff41947e640 1 --2- 172.21.15.94:0/2136576009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd630e7b0 0x556fd6319160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff413fff640 1 --2- 172.21.15.94:0/2136576009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd6305bd0 0x556fd6318c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff418c7d640 1 -- 172.21.15.94:0/2136576009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd6305bd0 msgr2=0x556fd6318c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff418c7d640 1 --2- 172.21.15.94:0/2136576009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd6305bd0 0x556fd6318c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff418c7d640 1 -- 172.21.15.94:0/2136576009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd630e7b0 msgr2=0x556fd6319160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff418c7d640 1 --2- 172.21.15.94:0/2136576009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd630e7b0 0x556fd6319160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.648+0000 7ff418c7d640 1 -- 172.21.15.94:0/2136576009 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4141a4780 con 0x556fd63052b0 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff418c7d640 1 --2- 172.21.15.94:0/2136576009 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd63186e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff408009620 tx=0x7ff408010660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0eb76f9a1f4d349 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff3f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff3f3fff640 1 -- 172.21.15.94:0/2136576009 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40800fdf0 con 0x556fd63052b0 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff3f3fff640 1 -- 172.21.15.94:0/2136576009 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff408000e40 con 0x556fd63052b0 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff41db8b740 1 -- 172.21.15.94:0/2136576009 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff40c003970 con 0x556fd63052b0 2024-02-17T12:04:00.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff3f3fff640 1 -- 172.21.15.94:0/2136576009 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff408005070 con 0x556fd63052b0 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff41db8b740 10 client.26552 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff41db8b740 1 -- 172.21.15.94:0/2136576009 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556fd626b610 con 0x556fd63052b0 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff3f3fff640 1 -- 172.21.15.94:0/2136576009 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff40c003970 con 0x556fd63052b0 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff3f3fff640 1 -- 172.21.15.94:0/2136576009 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff40802e460 con 0x556fd63052b0 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff41db8b740 10 client.26552 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff41db8b740 10 client.26552 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff41db8b740 20 client.26552 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2136576009 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556fd637ac60 0x556fd637d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff41db8b740 4 client.26552 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.649+0000 7ff41db8b740 1 -- 172.21.15.94:0/2136576009 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556fd5ff8890 con 0x556fd637ac60 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff413fff640 1 --2- 172.21.15.94:0/2136576009 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556fd637ac60 0x556fd637d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff3f3fff640 10 client.26552 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff413fff640 1 --2- 172.21.15.94:0/2136576009 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556fd637ac60 0x556fd637d050 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a663b12d1c8ddc2 server_cookie=446339890b210959 in_seq=0 out_seq=0 2024-02-17T12:04:00.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff3f3fff640 1 -- 172.21.15.94:0/2136576009 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556fd5ff8890 con 0x556fd637ac60 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff3f3fff640 10 client.26552 handle_command_reply: tid=0 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff41db8b740 1 client.26552 shutdown 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff41db8b740 1 -- 172.21.15.94:0/2136576009 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 msgr2=0x556fd63186e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2136576009 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd63186e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff408009620 tx=0x7ff408010660 comp rx=0 tx=0).stop 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff41db8b740 1 -- 172.21.15.94:0/2136576009 shutdown_connections 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2136576009 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556fd637ac60 0x556fd637d050 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2136576009 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556fd630e7b0 0x556fd6319160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2136576009 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556fd6305bd0 0x556fd6318c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff3f3fff640 0 client.26552 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff41db8b740 1 --2- 172.21.15.94:0/2136576009 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556fd63052b0 0x556fd63186e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff41db8b740 1 -- 172.21.15.94:0/2136576009 >> 172.21.15.94:0/2136576009 conn(0x556fd62ff990 msgr2=0x556fd62fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff41db8b740 1 -- 172.21.15.94:0/2136576009 shutdown_connections 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.650+0000 7ff41db8b740 1 -- 172.21.15.94:0/2136576009 wait complete. 2024-02-17T12:04:00.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.651+0000 7ff41db8b740 20 client.26552 trim_cache size 0 max 16384 2024-02-17T12:04:00.657 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:04:00.657 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:04:00.657 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:04:00.657 INFO:teuthology.orchestra.run.smithi094.stdout: "508760f9-0d30-4a2b-9207-aed3211228a9": { 2024-02-17T12:04:00.657 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:04:00.657 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "508760f9-0d30-4a2b-9207-aed3211228a9", 2024-02-17T12:04:00.657 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:04:00.657 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:04:00.658 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:04:00.658 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:04:00.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.651+0000 7ff41aee1640 1 -- 172.21.15.94:0/2022943337 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff4040a2230 msgr2=0x7ff4040a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.651+0000 7ff41aee1640 1 --2- 172.21.15.94:0/2022943337 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff4040a2230 0x7ff4040a46f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff40c04b000 tx=0x7ff40c04e000 comp rx=0 tx=0).stop 2024-02-17T12:04:00.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.651+0000 7ff41aee1640 1 -- 172.21.15.94:0/2022943337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff414105340 msgr2=0x7ff414197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:00.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.651+0000 7ff41aee1640 1 --2- 172.21.15.94:0/2022943337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff414105340 0x7ff414197ca0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff400005fd0 tx=0x7ff40000dde0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.651+0000 7ff41aee1640 1 -- 172.21.15.94:0/2022943337 shutdown_connections 2024-02-17T12:04:00.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.651+0000 7ff41aee1640 1 --2- 172.21.15.94:0/2022943337 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff4040a2230 0x7ff4040a46f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.651+0000 7ff41aee1640 1 --2- 172.21.15.94:0/2022943337 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff41419c020 0x7ff41419c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.651+0000 7ff41aee1640 1 --2- 172.21.15.94:0/2022943337 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff414105340 0x7ff414197ca0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.651+0000 7ff41aee1640 1 --2- 172.21.15.94:0/2022943337 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff414100270 0x7ff414197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:00.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.651+0000 7ff41aee1640 1 -- 172.21.15.94:0/2022943337 >> 172.21.15.94:0/2022943337 conn(0x7ff414077230 msgr2=0x7ff414068560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:00.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.651+0000 7ff41aee1640 1 -- 172.21.15.94:0/2022943337 shutdown_connections 2024-02-17T12:04:00.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:00.651+0000 7ff41aee1640 1 -- 172.21.15.94:0/2022943337 wait complete. 2024-02-17T12:04:00.663 INFO:tasks.cephfs.filesystem:scrub status for tag:508760f9-0d30-4a2b-9207-aed3211228a9 - {'path': '/', 'tag': '508760f9-0d30-4a2b-9207-aed3211228a9', 'options': 'recursive,force'} 2024-02-17T12:04:02.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:01 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:04:02.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:01 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:04:02.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:01 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:04:02.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:01 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:04:02.186 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:01 smithi125 ceph-mon[36819]: pgmap v181: 97 pgs: 97 active+clean; 4.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 739 op/s 2024-02-17T12:04:02.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:01 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:04:02.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:01 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:04:02.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:01 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:04:02.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:01 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:04:02.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:01 smithi094 ceph-mon[32499]: pgmap v181: 97 pgs: 97 active+clean; 4.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 739 op/s 2024-02-17T12:04:02.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:01 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:04:02.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:01 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:04:02.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:01 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:04:02.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:01 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:04:02.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:01 smithi189 ceph-mon[36588]: pgmap v181: 97 pgs: 97 active+clean; 4.0 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 13 MiB/s wr, 739 op/s 2024-02-17T12:04:03.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:02 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:04:03.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:02 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:04:03.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:02 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:04:04.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:03 smithi125 ceph-mon[36819]: pgmap v182: 97 pgs: 97 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 14 MiB/s wr, 808 op/s 2024-02-17T12:04:04.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:03 smithi094 ceph-mon[32499]: pgmap v182: 97 pgs: 97 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 14 MiB/s wr, 808 op/s 2024-02-17T12:04:04.401 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:03 smithi189 ceph-mon[36588]: pgmap v182: 97 pgs: 97 active+clean; 4.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 14 MiB/s wr, 808 op/s 2024-02-17T12:04:05.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 345.0 s, 336.0 tps, lat 289.919 ms stddev 253.737 2024-02-17T12:04:06.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:05 smithi125 ceph-mon[36819]: pgmap v183: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-02-17T12:04:06.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:05 smithi094 ceph-mon[32499]: pgmap v183: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-02-17T12:04:06.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:05 smithi189 ceph-mon[36588]: pgmap v183: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 17 MiB/s wr, 1.03k op/s 2024-02-17T12:04:08.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:07 smithi125 ceph-mon[36819]: pgmap v184: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 14 MiB/s wr, 773 op/s 2024-02-17T12:04:08.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:07 smithi094 ceph-mon[32499]: pgmap v184: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 14 MiB/s wr, 773 op/s 2024-02-17T12:04:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.358+0000 7f4514d56640 1 -- 172.21.15.94:0/3579925599 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c004c10 con 0x7f45100ff680 2024-02-17T12:04:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.358+0000 7f4517fbc640 1 -- 172.21.15.94:0/3579925599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45100ff680 msgr2=0x7f45100ffae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.358+0000 7f4517fbc640 1 --2- 172.21.15.94:0/3579925599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45100ff680 0x7f45100ffae0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f450c01a910 tx=0x7f450c03cca0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.359+0000 7f4517fbc640 1 -- 172.21.15.94:0/3579925599 shutdown_connections 2024-02-17T12:04:08.370 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.359+0000 7f4517fbc640 1 --2- 172.21.15.94:0/3579925599 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4510100020 0x7f4510103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.359+0000 7f4517fbc640 1 --2- 172.21.15.94:0/3579925599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45100ff680 0x7f45100ffae0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.359+0000 7f4517fbc640 1 --2- 172.21.15.94:0/3579925599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4510106150 0x7f4510106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.359+0000 7f4517fbc640 1 -- 172.21.15.94:0/3579925599 >> 172.21.15.94:0/3579925599 conn(0x7f4510076610 msgr2=0x7f45100fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.360+0000 7f4517fbc640 1 -- 172.21.15.94:0/3579925599 shutdown_connections 2024-02-17T12:04:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.360+0000 7f4517fbc640 1 -- 172.21.15.94:0/3579925599 wait complete. 2024-02-17T12:04:08.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.361+0000 7f4517fbc640 1 Processor -- start 2024-02-17T12:04:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.361+0000 7f4517fbc640 1 -- start start 2024-02-17T12:04:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4517fbc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45100ff680 0x7f45101b7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4517fbc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4510100020 0x7f45101b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4517fbc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4510106150 0x7f45101bbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.372 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4517fbc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45101bc5e0 con 0x7f45100ff680 2024-02-17T12:04:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4517fbc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45101bc750 con 0x7f4510100020 2024-02-17T12:04:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4517fbc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45101bc910 con 0x7f4510106150 2024-02-17T12:04:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4515d58640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45100ff680 0x7f45101b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4515d58640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45100ff680 0x7f45101b7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39520/0 (socket says 172.21.15.94:39520) 2024-02-17T12:04:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4515d58640 1 -- 172.21.15.94:0/810471212 learned_addr learned my addr 172.21.15.94:0/810471212 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:08.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4515557640 1 --2- 172.21.15.94:0/810471212 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4510100020 0x7f45101b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4515d58640 1 -- 172.21.15.94:0/810471212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4510106150 msgr2=0x7f45101bbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4515d58640 1 --2- 172.21.15.94:0/810471212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4510106150 0x7f45101bbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4515d58640 1 -- 172.21.15.94:0/810471212 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4510100020 msgr2=0x7f45101b7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4515d58640 1 --2- 172.21.15.94:0/810471212 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4510100020 0x7f45101b7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.362+0000 7f4515d58640 1 -- 172.21.15.94:0/810471212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c01a570 con 0x7f45100ff680 2024-02-17T12:04:08.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.363+0000 7f4515d58640 1 --2- 172.21.15.94:0/810471212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45100ff680 0x7f45101b7720 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f450800ba50 tx=0x7f450800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.363+0000 7f4506ffd640 1 -- 172.21.15.94:0/810471212 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450800cc10 con 0x7f45100ff680 2024-02-17T12:04:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.363+0000 7f4506ffd640 1 -- 172.21.15.94:0/810471212 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f450800cd70 con 0x7f45100ff680 2024-02-17T12:04:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.363+0000 7f4506ffd640 1 -- 172.21.15.94:0/810471212 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45080077d0 con 0x7f45100ff680 2024-02-17T12:04:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.365+0000 7f4517fbc640 1 -- 172.21.15.94:0/810471212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45101bcbf0 con 0x7f45100ff680 2024-02-17T12:04:08.375 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.365+0000 7f4517fbc640 1 -- 172.21.15.94:0/810471212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451018ea20 con 0x7f45100ff680 2024-02-17T12:04:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.368+0000 7f451ab92740 1 Processor -- start 2024-02-17T12:04:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.370+0000 7f4506ffd640 1 -- 172.21.15.94:0/810471212 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4508014440 con 0x7f45100ff680 2024-02-17T12:04:08.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.371+0000 7f4506ffd640 1 --2- 172.21.15.94:0/810471212 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f44fc0a2500 0x7f44fc0a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.371+0000 7f4506ffd640 1 -- 172.21.15.94:0/810471212 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f45080cd4f0 con 0x7f45100ff680 2024-02-17T12:04:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.372+0000 7f451ab92740 1 -- start start 2024-02-17T12:04:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.372+0000 7f451ab92740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de42062ad0 0x55de42061820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.372+0000 7f451ab92740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42061d60 0x55de420621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.372+0000 7f451ab92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de4206b2e0 0x55de4206d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.372+0000 7f451ab92740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de41fd09f0 con 0x55de42062ad0 2024-02-17T12:04:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.372+0000 7f451ab92740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de41fcdf20 con 0x55de42061d60 2024-02-17T12:04:08.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.372+0000 7f451ab92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de41fc9700 con 0x55de4206b2e0 2024-02-17T12:04:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.372+0000 7f4515557640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42061d60 0x55de420621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.372+0000 7f4515557640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42061d60 0x55de420621e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56406/0 (socket says 172.21.15.94:56406) 2024-02-17T12:04:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.372+0000 7f4515557640 1 -- 172.21.15.94:0/3980190469 learned_addr learned my addr 172.21.15.94:0/3980190469 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.372+0000 7f4515d58640 1 --2- 172.21.15.94:0/3980190469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de42062ad0 0x55de42061820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.373+0000 7f4515d58640 1 -- 172.21.15.94:0/3980190469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de4206b2e0 msgr2=0x55de4206d6d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:04:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.373+0000 7f4515d58640 1 --2- 172.21.15.94:0/3980190469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de4206b2e0 0x55de4206d6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.373+0000 7f4515d58640 1 -- 172.21.15.94:0/3980190469 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42061d60 msgr2=0x55de420621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:08.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.373+0000 7f4515d58640 1 --2- 172.21.15.94:0/3980190469 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42061d60 0x55de420621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.373+0000 7f4515d58640 1 -- 172.21.15.94:0/3980190469 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de41f5f7b0 con 0x55de42062ad0 2024-02-17T12:04:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.373+0000 7f4515d58640 1 --2- 172.21.15.94:0/3980190469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de42062ad0 0x55de42061820 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f45080cace0 tx=0x7f4508099990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f2d3c1448e4829 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.374+0000 7f44fbfff640 1 -- 172.21.15.94:0/3980190469 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450809e0d0 con 0x55de42062ad0 2024-02-17T12:04:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.374+0000 7f44fbfff640 1 -- 172.21.15.94:0/3980190469 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45080a2720 con 0x55de42062ad0 2024-02-17T12:04:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.374+0000 7f4516559640 1 --2- 172.21.15.94:0/810471212 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f44fc0a2500 0x7f44fc0a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.374+0000 7f44fbfff640 1 -- 172.21.15.94:0/3980190469 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45080a2290 con 0x55de42062ad0 2024-02-17T12:04:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.375+0000 7f451ab92740 1 -- 172.21.15.94:0/3980190469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de42062ad0 msgr2=0x55de42061820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.375+0000 7f451ab92740 1 --2- 172.21.15.94:0/3980190469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de42062ad0 0x55de42061820 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f45080cace0 tx=0x7f4508099990 comp rx=0 tx=0).stop 2024-02-17T12:04:08.385 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.375+0000 7f451ab92740 1 -- 172.21.15.94:0/3980190469 shutdown_connections 2024-02-17T12:04:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.375+0000 7f451ab92740 1 --2- 172.21.15.94:0/3980190469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de4206b2e0 0x55de4206d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.376+0000 7f451ab92740 1 --2- 172.21.15.94:0/3980190469 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42061d60 0x55de420621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.376+0000 7f451ab92740 1 --2- 172.21.15.94:0/3980190469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de42062ad0 0x55de42061820 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.376+0000 7f451ab92740 1 -- 172.21.15.94:0/3980190469 >> 172.21.15.94:0/3980190469 conn(0x55de42029550 msgr2=0x55de4205ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.376+0000 7f4516559640 1 --2- 172.21.15.94:0/810471212 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f44fc0a2500 0x7f44fc0a49c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4500006fd0 tx=0x7f4500006c40 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.380+0000 7f451ab92740 1 -- 172.21.15.94:0/3980190469 shutdown_connections 2024-02-17T12:04:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.380+0000 7f451ab92740 1 -- 172.21.15.94:0/3980190469 wait complete. 2024-02-17T12:04:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.380+0000 7f451ab92740 1 -- start start 2024-02-17T12:04:08.386 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.381+0000 7f451ab92740 1 Processor -- start 2024-02-17T12:04:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.382+0000 7f451ab92740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42061d60 0x55de42068360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f451ab92740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de42062ad0 0x55de420688a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f4515d58640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42061d60 0x55de42068360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f451ab92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de4206b2e0 0x55de42068de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f4515d58640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42061d60 0x55de42068360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56414/0 (socket says 172.21.15.94:56414) 2024-02-17T12:04:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f451ab92740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de41f6ebf0 con 0x55de42062ad0 2024-02-17T12:04:08.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f451ab92740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de41c9d250 con 0x55de42061d60 2024-02-17T12:04:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f451ab92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de41f4f200 con 0x55de4206b2e0 2024-02-17T12:04:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f4515d58640 1 -- 172.21.15.94:0/1415366843 learned_addr learned my addr 172.21.15.94:0/1415366843 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f4515557640 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de42062ad0 0x55de420688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f451ab92740 10 client.0 _ll_register_callbacks cb 0x55de41fbdd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f4516559640 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de4206b2e0 0x55de42068de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f4515d58640 1 -- 172.21.15.94:0/1415366843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de4206b2e0 msgr2=0x55de42068de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f4515d58640 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de4206b2e0 0x55de42068de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f4515d58640 1 -- 172.21.15.94:0/1415366843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de42062ad0 msgr2=0x55de420688a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f4515d58640 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de42062ad0 0x55de420688a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f4515d58640 1 -- 172.21.15.94:0/1415366843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451018ea20 con 0x55de42061d60 2024-02-17T12:04:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f4516559640 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de4206b2e0 0x55de42068de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:04:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f4515d58640 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42061d60 0x55de42068360 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f45080cd460 tx=0x7f450800cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5024694e4c6af4ff server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.383+0000 7f44f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:04:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.384+0000 7f44f97fa640 1 -- 172.21.15.94:0/1415366843 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45080146f0 con 0x55de42061d60 2024-02-17T12:04:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.384+0000 7f451ab92740 1 -- 172.21.15.94:0/1415366843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de41cc9600 con 0x55de42061d60 2024-02-17T12:04:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.384+0000 7f44f97fa640 1 -- 172.21.15.94:0/1415366843 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4508014850 con 0x55de42061d60 2024-02-17T12:04:08.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.384+0000 7f44f97fa640 1 -- 172.21.15.94:0/1415366843 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45080a2290 con 0x55de42061d60 2024-02-17T12:04:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.384+0000 7f451ab92740 10 client.26561 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.384+0000 7f451ab92740 1 -- 172.21.15.94:0/1415366843 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55de41fc7d80 con 0x55de42061d60 2024-02-17T12:04:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.385+0000 7f44f97fa640 1 -- 172.21.15.94:0/1415366843 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55de41cc9600 con 0x55de42061d60 2024-02-17T12:04:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.385+0000 7f44f97fa640 1 -- 172.21.15.94:0/1415366843 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f450809c070 con 0x55de42061d60 2024-02-17T12:04:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.385+0000 7f451ab92740 10 client.26561 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.385+0000 7f451ab92740 10 client.26561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.385+0000 7f451ab92740 20 client.26561 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.385+0000 7f451ab92740 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55de4207b330 0x55de4207d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.385+0000 7f451ab92740 4 client.26561 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:04:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.385+0000 7f451ab92740 1 -- 172.21.15.94:0/1415366843 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de41d1a1b0 con 0x55de4207b330 2024-02-17T12:04:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.385+0000 7f4515557640 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55de4207b330 0x55de4207d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.386+0000 7f44f97fa640 10 client.26561 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.388+0000 7f4515557640 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55de4207b330 0x55de4207d740 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14a2c29b40a22a45 server_cookie=d8ceb82a74026aaa in_seq=0 out_seq=0 2024-02-17T12:04:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.391+0000 7f44f97fa640 1 -- 172.21.15.94:0/1415366843 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55de41d1a1b0 con 0x55de4207b330 2024-02-17T12:04:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.391+0000 7f44f97fa640 10 client.26561 handle_command_reply: tid=0 2024-02-17T12:04:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.391+0000 7f451ab92740 1 client.26561 shutdown 2024-02-17T12:04:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.391+0000 7f451ab92740 1 -- 172.21.15.94:0/1415366843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42061d60 msgr2=0x55de42068360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.391+0000 7f451ab92740 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42061d60 0x55de42068360 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f45080cd460 tx=0x7f450800cf00 comp rx=0 tx=0).stop 2024-02-17T12:04:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.391+0000 7f451ab92740 1 -- 172.21.15.94:0/1415366843 shutdown_connections 2024-02-17T12:04:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.391+0000 7f451ab92740 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55de4207b330 0x55de4207d740 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.391+0000 7f451ab92740 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de4206b2e0 0x55de42068de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.392+0000 7f451ab92740 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de42062ad0 0x55de420688a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.392+0000 7f451ab92740 1 --2- 172.21.15.94:0/1415366843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42061d60 0x55de42068360 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f45080cd460 tx=0x7f450800cf00 comp rx=0 tx=0).stop 2024-02-17T12:04:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.392+0000 7f451ab92740 1 -- 172.21.15.94:0/1415366843 >> 172.21.15.94:0/1415366843 conn(0x55de42029550 msgr2=0x55de4205e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.393+0000 7f44f97fa640 0 client.26561 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.393+0000 7f451ab92740 1 -- 172.21.15.94:0/1415366843 shutdown_connections 2024-02-17T12:04:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.394+0000 7f451ab92740 1 -- 172.21.15.94:0/1415366843 wait complete. 2024-02-17T12:04:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.394+0000 7f451ab92740 20 client.26561 trim_cache size 0 max 16384 2024-02-17T12:04:08.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:07 smithi189 ceph-mon[36588]: pgmap v184: 97 pgs: 97 active+clean; 4.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 14 MiB/s wr, 773 op/s 2024-02-17T12:04:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.416+0000 7f451ab92740 1 Processor -- start 2024-02-17T12:04:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.416+0000 7f451ab92740 1 -- start start 2024-02-17T12:04:08.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.417+0000 7f451ab92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de42062ad0 0x55de42068c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.417+0000 7f4515d58640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de42062ad0 0x55de42068c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.417+0000 7f451ab92740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de4206b2e0 0x55de42065620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.417+0000 7f4515d58640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de42062ad0 0x55de42068c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:37112/0 (socket says 172.21.15.94:37112) 2024-02-17T12:04:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.417+0000 7f4515d58640 1 -- 172.21.15.94:0/1923179145 learned_addr learned my addr 172.21.15.94:0/1923179145 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.417+0000 7f451ab92740 1 --2- 172.21.15.94:0/1923179145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de420618f0 0x55de42065b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.418+0000 7f451ab92740 1 -- 172.21.15.94:0/1923179145 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4206def0 con 0x55de4206b2e0 2024-02-17T12:04:08.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.418+0000 7f451ab92740 1 -- 172.21.15.94:0/1923179145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de42082960 con 0x55de420618f0 2024-02-17T12:04:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.418+0000 7f451ab92740 1 -- 172.21.15.94:0/1923179145 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de42061030 con 0x55de42062ad0 2024-02-17T12:04:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.418+0000 7f4516559640 1 --2- 172.21.15.94:0/1923179145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de420618f0 0x55de42065b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.418+0000 7f4515557640 1 --2- 172.21.15.94:0/1923179145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de4206b2e0 0x55de42065620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.418+0000 7f4516559640 1 -- 172.21.15.94:0/1923179145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de42062ad0 msgr2=0x55de42068c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.418+0000 7f4516559640 1 --2- 172.21.15.94:0/1923179145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de42062ad0 0x55de42068c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.418+0000 7f4516559640 1 -- 172.21.15.94:0/1923179145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de4206b2e0 msgr2=0x55de42065620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.418+0000 7f4516559640 1 --2- 172.21.15.94:0/1923179145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de4206b2e0 0x55de42065620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.418+0000 7f4516559640 1 -- 172.21.15.94:0/1923179145 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de41f647b0 con 0x55de420618f0 2024-02-17T12:04:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.418+0000 7f4515557640 1 --2- 172.21.15.94:0/1923179145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de4206b2e0 0x55de42065620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:04:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.418+0000 7f4516559640 1 --2- 172.21.15.94:0/1923179145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de420618f0 0x55de42065b60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55de42062800 tx=0x7f4500009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8160d5537329733e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.419+0000 7f44fb7fe640 1 -- 172.21.15.94:0/1923179145 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500012a90 con 0x55de420618f0 2024-02-17T12:04:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.419+0000 7f44fb7fe640 1 -- 172.21.15.94:0/1923179145 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4500019050 con 0x55de420618f0 2024-02-17T12:04:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.419+0000 7f44fb7fe640 1 -- 172.21.15.94:0/1923179145 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45000105c0 con 0x55de420618f0 2024-02-17T12:04:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.420+0000 7f451ab92740 1 -- 172.21.15.94:0/1923179145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de420618f0 msgr2=0x55de42065b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.420+0000 7f451ab92740 1 --2- 172.21.15.94:0/1923179145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de420618f0 0x55de42065b60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55de42062800 tx=0x7f4500009f90 comp rx=0 tx=0).stop 2024-02-17T12:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.421+0000 7f451ab92740 1 -- 172.21.15.94:0/1923179145 shutdown_connections 2024-02-17T12:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.421+0000 7f451ab92740 1 --2- 172.21.15.94:0/1923179145 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de420618f0 0x55de42065b60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.421+0000 7f451ab92740 1 --2- 172.21.15.94:0/1923179145 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de4206b2e0 0x55de42065620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.421+0000 7f451ab92740 1 --2- 172.21.15.94:0/1923179145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de42062ad0 0x55de42068c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.431 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.421+0000 7f451ab92740 1 -- 172.21.15.94:0/1923179145 >> 172.21.15.94:0/1923179145 conn(0x55de42029550 msgr2=0x55de42066c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.421+0000 7f451ab92740 1 -- 172.21.15.94:0/1923179145 shutdown_connections 2024-02-17T12:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.421+0000 7f451ab92740 1 -- 172.21.15.94:0/1923179145 wait complete. 2024-02-17T12:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.422+0000 7f451ab92740 1 -- start start 2024-02-17T12:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.422+0000 7f451ab92740 1 Processor -- start 2024-02-17T12:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.422+0000 7f451ab92740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de420618f0 0x55de42070e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.422+0000 7f451ab92740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42062ad0 0x55de420713c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f451ab92740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de4206b2e0 0x55de42071940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f451ab92740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de420d1a00 con 0x55de4206b2e0 2024-02-17T12:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f451ab92740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de420d1b70 con 0x55de42062ad0 2024-02-17T12:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f451ab92740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de420d1d30 con 0x55de420618f0 2024-02-17T12:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f4515d58640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de420618f0 0x55de42070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f451ab92740 10 client.0 _ll_register_callbacks cb 0x7f4508014ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f4515557640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42062ad0 0x55de420713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f4515d58640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de420618f0 0x55de42070e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:37126/0 (socket says 172.21.15.94:37126) 2024-02-17T12:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f4515d58640 1 -- 172.21.15.94:0/2115878880 learned_addr learned my addr 172.21.15.94:0/2115878880 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f4516559640 1 --2- 172.21.15.94:0/2115878880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de4206b2e0 0x55de42071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f4515d58640 1 -- 172.21.15.94:0/2115878880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42062ad0 msgr2=0x55de420713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f4515d58640 1 --2- 172.21.15.94:0/2115878880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42062ad0 0x55de420713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f4515d58640 1 -- 172.21.15.94:0/2115878880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de4206b2e0 msgr2=0x55de42071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f4515d58640 1 --2- 172.21.15.94:0/2115878880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de4206b2e0 0x55de42071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f4515d58640 1 -- 172.21.15.94:0/2115878880 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45000135e0 con 0x55de420618f0 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.423+0000 7f4515557640 1 --2- 172.21.15.94:0/2115878880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42062ad0 0x55de420713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.424+0000 7f4515d58640 1 --2- 172.21.15.94:0/2115878880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de420618f0 0x55de42070e80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f450800cf90 tx=0x7f4508098540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b7910a2e48b0c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.424+0000 7f44f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.424+0000 7f44f97fa640 1 -- 172.21.15.94:0/2115878880 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508009950 con 0x55de420618f0 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.424+0000 7f44f97fa640 1 -- 172.21.15.94:0/2115878880 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f450809e0d0 con 0x55de420618f0 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.424+0000 7f451ab92740 1 -- 172.21.15.94:0/2115878880 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45080a2720 con 0x55de420618f0 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.425+0000 7f44f97fa640 1 -- 172.21.15.94:0/2115878880 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45080b8180 con 0x55de420618f0 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.426+0000 7f451ab92740 10 client.26608 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.426+0000 7f451ab92740 1 -- 172.21.15.94:0/2115878880 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55de41f8a540 con 0x55de420618f0 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.426+0000 7f44f97fa640 1 -- 172.21.15.94:0/2115878880 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f45080a2720 con 0x55de420618f0 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.426+0000 7f44f97fa640 1 -- 172.21.15.94:0/2115878880 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f450809c040 con 0x55de420618f0 2024-02-17T12:04:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.426+0000 7f451ab92740 10 client.26608 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.426+0000 7f451ab92740 10 client.26608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.426+0000 7f451ab92740 20 client.26608 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.426+0000 7f451ab92740 1 --2- 172.21.15.94:0/2115878880 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55de420d3010 0x55de420d5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.426+0000 7f451ab92740 4 client.26608 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:04:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.426+0000 7f451ab92740 1 -- 172.21.15.94:0/2115878880 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55de41d06550 con 0x55de420d3010 2024-02-17T12:04:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.426+0000 7f4515557640 1 --2- 172.21.15.94:0/2115878880 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55de420d3010 0x55de420d5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.427+0000 7f44f97fa640 10 client.26608 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:08.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.427+0000 7f4515557640 1 --2- 172.21.15.94:0/2115878880 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55de420d3010 0x55de420d5400 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ef218a9549d946 server_cookie=b13d0469f4e847a2 in_seq=0 out_seq=0 2024-02-17T12:04:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f44f97fa640 1 -- 172.21.15.94:0/2115878880 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55de41d06550 con 0x55de420d3010 2024-02-17T12:04:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f44f97fa640 10 client.26608 handle_command_reply: tid=0 2024-02-17T12:04:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f451ab92740 1 client.26608 shutdown 2024-02-17T12:04:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f451ab92740 1 -- 172.21.15.94:0/2115878880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de420618f0 msgr2=0x55de42070e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f451ab92740 1 --2- 172.21.15.94:0/2115878880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de420618f0 0x55de42070e80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f450800cf90 tx=0x7f4508098540 comp rx=0 tx=0).stop 2024-02-17T12:04:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f451ab92740 1 -- 172.21.15.94:0/2115878880 shutdown_connections 2024-02-17T12:04:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f451ab92740 1 --2- 172.21.15.94:0/2115878880 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55de420d3010 0x55de420d5400 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f451ab92740 1 --2- 172.21.15.94:0/2115878880 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55de4206b2e0 0x55de42071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f451ab92740 1 --2- 172.21.15.94:0/2115878880 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55de42062ad0 0x55de420713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f44f97fa640 0 client.26608 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f451ab92740 1 --2- 172.21.15.94:0/2115878880 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55de420618f0 0x55de42070e80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f451ab92740 1 -- 172.21.15.94:0/2115878880 >> 172.21.15.94:0/2115878880 conn(0x55de42029550 msgr2=0x55de4205b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f451ab92740 1 -- 172.21.15.94:0/2115878880 shutdown_connections 2024-02-17T12:04:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f451ab92740 1 -- 172.21.15.94:0/2115878880 wait complete. 2024-02-17T12:04:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.451+0000 7f451ab92740 20 client.26608 trim_cache size 0 max 16384 2024-02-17T12:04:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:04:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:04:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:04:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:04:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.452+0000 7f4517fbc640 1 -- 172.21.15.94:0/810471212 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f44fc0a2500 msgr2=0x7f44fc0a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.452+0000 7f4517fbc640 1 --2- 172.21.15.94:0/810471212 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f44fc0a2500 0x7f44fc0a49c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4500006fd0 tx=0x7f4500006c40 comp rx=0 tx=0).stop 2024-02-17T12:04:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.452+0000 7f4517fbc640 1 -- 172.21.15.94:0/810471212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45100ff680 msgr2=0x7f45101b7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.452+0000 7f4517fbc640 1 --2- 172.21.15.94:0/810471212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45100ff680 0x7f45101b7720 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f450800ba50 tx=0x7f450800bf10 comp rx=0 tx=0).stop 2024-02-17T12:04:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.452+0000 7f4517fbc640 1 -- 172.21.15.94:0/810471212 shutdown_connections 2024-02-17T12:04:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.452+0000 7f4517fbc640 1 --2- 172.21.15.94:0/810471212 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f44fc0a2500 0x7f44fc0a49c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.452+0000 7f4517fbc640 1 --2- 172.21.15.94:0/810471212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4510106150 0x7f45101bbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.452+0000 7f4517fbc640 1 --2- 172.21.15.94:0/810471212 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4510100020 0x7f45101b7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.452+0000 7f4517fbc640 1 --2- 172.21.15.94:0/810471212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f45100ff680 0x7f45101b7720 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.452+0000 7f4517fbc640 1 -- 172.21.15.94:0/810471212 >> 172.21.15.94:0/810471212 conn(0x7f4510076610 msgr2=0x7f4510101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.452+0000 7f4517fbc640 1 -- 172.21.15.94:0/810471212 shutdown_connections 2024-02-17T12:04:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:08.452+0000 7f4517fbc640 1 -- 172.21.15.94:0/810471212 wait complete. 2024-02-17T12:04:10.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:09 smithi125 ceph-mon[36819]: pgmap v185: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 40 MiB/s wr, 1.81k op/s 2024-02-17T12:04:10.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:09 smithi094 ceph-mon[32499]: pgmap v185: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 40 MiB/s wr, 1.81k op/s 2024-02-17T12:04:10.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 350.0 s, 1123.4 tps, lat 91.840 ms stddev 202.995 2024-02-17T12:04:10.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:09 smithi189 ceph-mon[36588]: pgmap v185: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 40 MiB/s wr, 1.81k op/s 2024-02-17T12:04:12.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:11 smithi125 ceph-mon[36819]: pgmap v186: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 31 MiB/s wr, 1.37k op/s 2024-02-17T12:04:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:11 smithi094 ceph-mon[32499]: pgmap v186: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 31 MiB/s wr, 1.37k op/s 2024-02-17T12:04:12.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:11 smithi189 ceph-mon[36588]: pgmap v186: 97 pgs: 97 active+clean; 4.2 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 31 MiB/s wr, 1.37k op/s 2024-02-17T12:04:14.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:13 smithi094 ceph-mon[32499]: pgmap v187: 97 pgs: 97 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 34 MiB/s wr, 1.47k op/s 2024-02-17T12:04:14.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:13 smithi189 ceph-mon[36588]: pgmap v187: 97 pgs: 97 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 34 MiB/s wr, 1.47k op/s 2024-02-17T12:04:14.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:13 smithi125 ceph-mon[36819]: pgmap v187: 97 pgs: 97 active+clean; 4.3 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 34 MiB/s wr, 1.47k op/s 2024-02-17T12:04:15.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 355.0 s, 2623.8 tps, lat 38.113 ms stddev 49.834 2024-02-17T12:04:16.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:15 smithi094 ceph-mon[32499]: pgmap v188: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 40 MiB/s wr, 1.70k op/s 2024-02-17T12:04:16.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:15 smithi189 ceph-mon[36588]: pgmap v188: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 40 MiB/s wr, 1.70k op/s 2024-02-17T12:04:16.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:15 smithi125 ceph-mon[36819]: pgmap v188: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 40 MiB/s wr, 1.70k op/s 2024-02-17T12:04:18.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:17 smithi094 ceph-mon[32499]: pgmap v189: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 MiB/s wr, 1.43k op/s 2024-02-17T12:04:18.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:17 smithi189 ceph-mon[36588]: pgmap v189: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 MiB/s wr, 1.43k op/s 2024-02-17T12:04:18.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:17 smithi125 ceph-mon[36819]: pgmap v189: 97 pgs: 97 active+clean; 4.3 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 MiB/s wr, 1.43k op/s 2024-02-17T12:04:19.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:18 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:04:19.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:18 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:04:19.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:18 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:04:20.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:19 smithi094 ceph-mon[32499]: pgmap v190: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 43 MiB/s wr, 1.72k op/s 2024-02-17T12:04:20.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:19 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:04:20.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:19 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:04:20.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:19 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:04:20.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 360.0 s, 2815.6 tps, lat 35.259 ms stddev 41.721 2024-02-17T12:04:20.302 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:19 smithi189 ceph-mon[36588]: pgmap v190: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 43 MiB/s wr, 1.72k op/s 2024-02-17T12:04:20.302 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:19 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:04:20.302 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:19 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:04:20.303 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:19 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:04:20.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:19 smithi125 ceph-mon[36819]: pgmap v190: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 43 MiB/s wr, 1.72k op/s 2024-02-17T12:04:20.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:19 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:04:20.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:19 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:04:20.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:19 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:04:21.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:20 smithi189 ceph-mon[36588]: pgmap v191: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 683 op/s 2024-02-17T12:04:21.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:20 smithi125 ceph-mon[36819]: pgmap v191: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 683 op/s 2024-02-17T12:04:21.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:20 smithi094 ceph-mon[32499]: pgmap v191: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 17 MiB/s wr, 683 op/s 2024-02-17T12:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.576+0000 7fd7c86c3640 1 -- 172.21.15.94:0/291240483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c01086c0 msgr2=0x7fd7c0108b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.576+0000 7fd7c86c3640 1 --2- 172.21.15.94:0/291240483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c01086c0 0x7fd7c0108b40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009900 tx=0x7fd7b002bcd0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.576+0000 7fd7c86c3640 1 -- 172.21.15.94:0/291240483 shutdown_connections 2024-02-17T12:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.576+0000 7fd7c86c3640 1 --2- 172.21.15.94:0/291240483 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7c0116210 0x7fd7c0118600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.576+0000 7fd7c86c3640 1 --2- 172.21.15.94:0/291240483 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c01086c0 0x7fd7c0108b40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.576+0000 7fd7c86c3640 1 --2- 172.21.15.94:0/291240483 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7c010b8b0 0x7fd7c0108180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.576+0000 7fd7c86c3640 1 -- 172.21.15.94:0/291240483 >> 172.21.15.94:0/291240483 conn(0x7fd7c006c6b0 msgr2=0x7fd7c006cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c86c3640 1 -- 172.21.15.94:0/291240483 shutdown_connections 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c86c3640 1 -- 172.21.15.94:0/291240483 wait complete. 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c86c3640 1 Processor -- start 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c86c3640 1 -- start start 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c86c3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c01086c0 0x7fd7c01a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c86c3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7c010b8b0 0x7fd7c01a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c86c3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7c0116210 0x7fd7c01a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c86c3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c01a70b0 con 0x7fd7c01086c0 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c86c3640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c01a7220 con 0x7fd7c0116210 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c86c3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c01a73e0 con 0x7fd7c010b8b0 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c645f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c01086c0 0x7fd7c01a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c645f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c01086c0 0x7fd7c01a21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57050/0 (socket says 172.21.15.94:57050) 2024-02-17T12:04:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c645f640 1 -- 172.21.15.94:0/462622715 learned_addr learned my addr 172.21.15.94:0/462622715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c6c60640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7c0116210 0x7fd7c01a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.577+0000 7fd7c5c5e640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7c010b8b0 0x7fd7c01a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.578+0000 7fd7c6c60640 1 -- 172.21.15.94:0/462622715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7c010b8b0 msgr2=0x7fd7c01a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.578+0000 7fd7c6c60640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7c010b8b0 0x7fd7c01a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.578+0000 7fd7c6c60640 1 -- 172.21.15.94:0/462622715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c01086c0 msgr2=0x7fd7c01a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.578+0000 7fd7c6c60640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c01086c0 0x7fd7c01a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.578+0000 7fd7c6c60640 1 -- 172.21.15.94:0/462622715 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b0009560 con 0x7fd7c0116210 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.578+0000 7fd7c645f640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c01086c0 0x7fd7c01a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.578+0000 7fd7c6c60640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7c0116210 0x7fd7c01a6ab0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd7c006a590 tx=0x7fd7b800dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.578+0000 7fd7b77fe640 1 -- 172.21.15.94:0/462622715 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b800bd40 con 0x7fd7c0116210 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.578+0000 7fd7b77fe640 1 -- 172.21.15.94:0/462622715 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7b8010940 con 0x7fd7c0116210 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.578+0000 7fd7c86c3640 1 -- 172.21.15.94:0/462622715 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c01a76c0 con 0x7fd7c0116210 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.578+0000 7fd7b77fe640 1 -- 172.21.15.94:0/462622715 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b800fa20 con 0x7fd7c0116210 2024-02-17T12:04:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.578+0000 7fd7c86c3640 1 -- 172.21.15.94:0/462622715 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c01aef40 con 0x7fd7c0116210 2024-02-17T12:04:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.582+0000 7fd7b77fe640 1 -- 172.21.15.94:0/462622715 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd7b8010ab0 con 0x7fd7c0116210 2024-02-17T12:04:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.583+0000 7fd7b77fe640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd7ac0a22d0 0x7fd7ac0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.583+0000 7fd7c645f640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd7ac0a22d0 0x7fd7ac0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.584+0000 7fd7b77fe640 1 -- 172.21.15.94:0/462622715 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd7b80ce410 con 0x7fd7c0116210 2024-02-17T12:04:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.584+0000 7fd7cb371740 1 Processor -- start 2024-02-17T12:04:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.584+0000 7fd7c645f640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd7ac0a22d0 0x7fd7ac0a4790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc011550 tx=0x7fd7bc005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:21.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.584+0000 7fd7cb371740 1 -- start start 2024-02-17T12:04:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7cb371740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0901ad0 0x558ed0900820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7cb371740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0900d60 0x558ed09011e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7cb371740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed090a2e0 0x558ed090c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7c645f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0901ad0 0x558ed0900820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7c645f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0901ad0 0x558ed0900820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57358/0 (socket says 172.21.15.94:57358) 2024-02-17T12:04:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7c5c5e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0900d60 0x558ed09011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7c5c5e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0900d60 0x558ed09011e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36732/0 (socket says 172.21.15.94:36732) 2024-02-17T12:04:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7c645f640 1 -- 172.21.15.94:0/4265914362 learned_addr learned my addr 172.21.15.94:0/4265914362 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7c6c60640 1 --2- 172.21.15.94:0/4265914362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed090a2e0 0x558ed090c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7cb371740 1 -- 172.21.15.94:0/4265914362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed086f9f0 con 0x558ed090a2e0 2024-02-17T12:04:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7cb371740 1 -- 172.21.15.94:0/4265914362 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed086cf20 con 0x558ed0900d60 2024-02-17T12:04:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7cb371740 1 -- 172.21.15.94:0/4265914362 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed0868700 con 0x558ed0901ad0 2024-02-17T12:04:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7c645f640 1 -- 172.21.15.94:0/4265914362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0900d60 msgr2=0x558ed09011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7c645f640 1 --2- 172.21.15.94:0/4265914362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0900d60 0x558ed09011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7c645f640 1 -- 172.21.15.94:0/4265914362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed090a2e0 msgr2=0x558ed090c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7c645f640 1 --2- 172.21.15.94:0/4265914362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed090a2e0 0x558ed090c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.585+0000 7fd7c645f640 1 -- 172.21.15.94:0/4265914362 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ed07fe7b0 con 0x558ed0901ad0 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.586+0000 7fd7c645f640 1 --2- 172.21.15.94:0/4265914362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0901ad0 0x558ed0900820 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00f770 tx=0x7fd7bc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d93a09517da524d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.586+0000 7fd7b4ff9640 1 -- 172.21.15.94:0/4265914362 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc01c940 con 0x558ed0901ad0 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.586+0000 7fd7b4ff9640 1 -- 172.21.15.94:0/4265914362 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7bc0263f0 con 0x558ed0901ad0 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.586+0000 7fd7b4ff9640 1 -- 172.21.15.94:0/4265914362 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc00e5b0 con 0x558ed0901ad0 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.586+0000 7fd7cb371740 1 -- 172.21.15.94:0/4265914362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0901ad0 msgr2=0x558ed0900820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.586+0000 7fd7cb371740 1 --2- 172.21.15.94:0/4265914362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0901ad0 0x558ed0900820 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc00f770 tx=0x7fd7bc014000 comp rx=0 tx=0).stop 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.587+0000 7fd7cb371740 1 -- 172.21.15.94:0/4265914362 shutdown_connections 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.587+0000 7fd7cb371740 1 --2- 172.21.15.94:0/4265914362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed090a2e0 0x558ed090c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.587+0000 7fd7cb371740 1 --2- 172.21.15.94:0/4265914362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0900d60 0x558ed09011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.587+0000 7fd7cb371740 1 --2- 172.21.15.94:0/4265914362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0901ad0 0x558ed0900820 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.587+0000 7fd7cb371740 1 -- 172.21.15.94:0/4265914362 >> 172.21.15.94:0/4265914362 conn(0x558ed08c8550 msgr2=0x558ed08fba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.587+0000 7fd7cb371740 1 -- 172.21.15.94:0/4265914362 shutdown_connections 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.587+0000 7fd7cb371740 1 -- 172.21.15.94:0/4265914362 wait complete. 2024-02-17T12:04:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.587+0000 7fd7cb371740 1 -- start start 2024-02-17T12:04:21.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.587+0000 7fd7cb371740 1 Processor -- start 2024-02-17T12:04:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.589+0000 7fd7cb371740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0900d60 0x558ed09072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.589+0000 7fd7cb371740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0901ad0 0x558ed0907830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.589+0000 7fd7cb371740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed090a2e0 0x558ed0907d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.589+0000 7fd7cb371740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed080dbf0 con 0x558ed0901ad0 2024-02-17T12:04:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.589+0000 7fd7cb371740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed053c250 con 0x558ed0900d60 2024-02-17T12:04:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.589+0000 7fd7c6c60640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed090a2e0 0x558ed0907d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.589+0000 7fd7c6c60640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed090a2e0 0x558ed0907d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57374/0 (socket says 172.21.15.94:57374) 2024-02-17T12:04:21.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.589+0000 7fd7c6c60640 1 -- 172.21.15.94:0/3578120838 learned_addr learned my addr 172.21.15.94:0/3578120838 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.589+0000 7fd7cb371740 1 -- 172.21.15.94:0/3578120838 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed07ee200 con 0x558ed090a2e0 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.589+0000 7fd7cb371740 10 client.0 _ll_register_callbacks cb 0x558ed085cd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.590+0000 7fd7c6c60640 1 -- 172.21.15.94:0/3578120838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0900d60 msgr2=0x558ed09072f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.590+0000 7fd7c6c60640 1 --2- 172.21.15.94:0/3578120838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0900d60 0x558ed09072f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.590+0000 7fd7c6c60640 1 -- 172.21.15.94:0/3578120838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0901ad0 msgr2=0x558ed0907830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.590+0000 7fd7c6c60640 1 --2- 172.21.15.94:0/3578120838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0901ad0 0x558ed0907830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.590+0000 7fd7c6c60640 1 -- 172.21.15.94:0/3578120838 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc017050 con 0x558ed090a2e0 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.590+0000 7fd7c6c60640 1 --2- 172.21.15.94:0/3578120838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed090a2e0 0x558ed0907d70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8097760 tx=0x7fd7b809ae50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bbbcd9c1e81e96f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.590+0000 7fd7a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.590+0000 7fd7a9ffb640 1 -- 172.21.15.94:0/3578120838 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b809a1e0 con 0x558ed090a2e0 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.590+0000 7fd7cb371740 1 -- 172.21.15.94:0/3578120838 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ed0568600 con 0x558ed090a2e0 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.590+0000 7fd7a9ffb640 1 -- 172.21.15.94:0/3578120838 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd7b8010e40 con 0x558ed090a2e0 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.590+0000 7fd7a9ffb640 1 -- 172.21.15.94:0/3578120838 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7b80b9320 con 0x558ed090a2e0 2024-02-17T12:04:21.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.591+0000 7fd7cb371740 10 client.26620 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.591+0000 7fd7cb371740 1 -- 172.21.15.94:0/3578120838 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558ed0866d80 con 0x558ed090a2e0 2024-02-17T12:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.591+0000 7fd7a9ffb640 1 -- 172.21.15.94:0/3578120838 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558ed0568600 con 0x558ed090a2e0 2024-02-17T12:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.591+0000 7fd7a9ffb640 1 -- 172.21.15.94:0/3578120838 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd7b800fca0 con 0x558ed090a2e0 2024-02-17T12:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.591+0000 7fd7cb371740 10 client.26620 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.591+0000 7fd7cb371740 10 client.26620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.591+0000 7fd7cb371740 20 client.26620 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.591+0000 7fd7cb371740 1 --2- 172.21.15.94:0/3578120838 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558ed091a440 0x558ed091c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.591+0000 7fd7cb371740 4 client.26620 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.591+0000 7fd7cb371740 1 -- 172.21.15.94:0/3578120838 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ed05b91b0 con 0x558ed091a440 2024-02-17T12:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.591+0000 7fd7c645f640 1 --2- 172.21.15.94:0/3578120838 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558ed091a440 0x558ed091c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.592+0000 7fd7a9ffb640 10 client.26620 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:21.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.592+0000 7fd7c645f640 1 --2- 172.21.15.94:0/3578120838 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558ed091a440 0x558ed091c830 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=650a6a470966bd69 server_cookie=6c40c120ed8d6dd3 in_seq=0 out_seq=0 2024-02-17T12:04:21.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7a9ffb640 1 -- 172.21.15.94:0/3578120838 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ed05b91b0 con 0x558ed091a440 2024-02-17T12:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7a9ffb640 10 client.26620 handle_command_reply: tid=0 2024-02-17T12:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7cb371740 1 client.26620 shutdown 2024-02-17T12:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7cb371740 1 -- 172.21.15.94:0/3578120838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed090a2e0 msgr2=0x558ed0907d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7cb371740 1 --2- 172.21.15.94:0/3578120838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed090a2e0 0x558ed0907d70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8097760 tx=0x7fd7b809ae50 comp rx=0 tx=0).stop 2024-02-17T12:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7cb371740 1 -- 172.21.15.94:0/3578120838 shutdown_connections 2024-02-17T12:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7cb371740 1 --2- 172.21.15.94:0/3578120838 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558ed091a440 0x558ed091c830 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7cb371740 1 --2- 172.21.15.94:0/3578120838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed090a2e0 0x558ed0907d70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7cb371740 1 --2- 172.21.15.94:0/3578120838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0901ad0 0x558ed0907830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7cb371740 1 --2- 172.21.15.94:0/3578120838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0900d60 0x558ed09072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7cb371740 1 -- 172.21.15.94:0/3578120838 >> 172.21.15.94:0/3578120838 conn(0x558ed08c8550 msgr2=0x558ed08fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7a9ffb640 0 client.26620 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7cb371740 1 -- 172.21.15.94:0/3578120838 shutdown_connections 2024-02-17T12:04:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7cb371740 1 -- 172.21.15.94:0/3578120838 wait complete. 2024-02-17T12:04:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.593+0000 7fd7cb371740 20 client.26620 trim_cache size 0 max 16384 2024-02-17T12:04:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.617+0000 7fd7cb371740 1 Processor -- start 2024-02-17T12:04:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.617+0000 7fd7cb371740 1 -- start start 2024-02-17T12:04:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7cb371740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0900d60 0x558ed0909a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7cb371740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0901ad0 0x558ed0907b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7cb371740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed090a2e0 0x558ed0904590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7cb371740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed0917180 con 0x558ed0900d60 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7cb371740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed0916050 con 0x558ed0901ad0 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7cb371740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed09000a0 con 0x558ed090a2e0 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7c645f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0900d60 0x558ed0909a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7c645f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0900d60 0x558ed0909a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57064/0 (socket says 172.21.15.94:57064) 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7c645f640 1 -- 172.21.15.94:0/249366581 learned_addr learned my addr 172.21.15.94:0/249366581 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7c5c5e640 1 --2- 172.21.15.94:0/249366581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0901ad0 0x558ed0907b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7c6c60640 1 --2- 172.21.15.94:0/249366581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed090a2e0 0x558ed0904590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7c645f640 1 -- 172.21.15.94:0/249366581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed090a2e0 msgr2=0x558ed0904590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7c645f640 1 --2- 172.21.15.94:0/249366581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed090a2e0 0x558ed0904590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7c645f640 1 -- 172.21.15.94:0/249366581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0901ad0 msgr2=0x558ed0907b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7c645f640 1 --2- 172.21.15.94:0/249366581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0901ad0 0x558ed0907b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.618+0000 7fd7c645f640 1 -- 172.21.15.94:0/249366581 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ed08037b0 con 0x558ed0900d60 2024-02-17T12:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.619+0000 7fd7c645f640 1 --2- 172.21.15.94:0/249366581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0900d60 0x558ed0909a20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc028040 tx=0x7fd7bc026f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676956594ba4330b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.619+0000 7fd7abfff640 1 -- 172.21.15.94:0/249366581 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc005350 con 0x558ed0900d60 2024-02-17T12:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.619+0000 7fd7abfff640 1 -- 172.21.15.94:0/249366581 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7bc009240 con 0x558ed0900d60 2024-02-17T12:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.619+0000 7fd7abfff640 1 -- 172.21.15.94:0/249366581 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc034530 con 0x558ed0900d60 2024-02-17T12:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.620+0000 7fd7cb371740 1 -- 172.21.15.94:0/249366581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0900d60 msgr2=0x558ed0909a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.620+0000 7fd7cb371740 1 --2- 172.21.15.94:0/249366581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0900d60 0x558ed0909a20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc028040 tx=0x7fd7bc026f20 comp rx=0 tx=0).stop 2024-02-17T12:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.620+0000 7fd7cb371740 1 -- 172.21.15.94:0/249366581 shutdown_connections 2024-02-17T12:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.620+0000 7fd7cb371740 1 --2- 172.21.15.94:0/249366581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed090a2e0 0x558ed0904590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.620+0000 7fd7cb371740 1 --2- 172.21.15.94:0/249366581 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed0901ad0 0x558ed0907b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.620+0000 7fd7cb371740 1 --2- 172.21.15.94:0/249366581 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0900d60 0x558ed0909a20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.620+0000 7fd7cb371740 1 -- 172.21.15.94:0/249366581 >> 172.21.15.94:0/249366581 conn(0x558ed08c8550 msgr2=0x558ed0905be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.620+0000 7fd7cb371740 1 -- 172.21.15.94:0/249366581 shutdown_connections 2024-02-17T12:04:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.620+0000 7fd7cb371740 1 -- 172.21.15.94:0/249366581 wait complete. 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.620+0000 7fd7cb371740 1 -- start start 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.620+0000 7fd7cb371740 1 Processor -- start 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.620+0000 7fd7cb371740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0900d60 0x558ed0910250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7cb371740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0901ad0 0x558ed0910790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7cb371740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed090a2e0 0x558ed0979170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7cb371740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed0910f20 con 0x558ed0901ad0 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7cb371740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed09796e0 con 0x558ed090a2e0 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7cb371740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ed09798a0 con 0x558ed0900d60 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7cb371740 10 client.0 _ll_register_callbacks cb 0x7fd7bc009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7c645f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0900d60 0x558ed0910250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7c645f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0900d60 0x558ed0910250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57406/0 (socket says 172.21.15.94:57406) 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7c645f640 1 -- 172.21.15.94:0/2057954340 learned_addr learned my addr 172.21.15.94:0/2057954340 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7c6c60640 1 --2- 172.21.15.94:0/2057954340 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed090a2e0 0x558ed0979170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7c5c5e640 1 --2- 172.21.15.94:0/2057954340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0901ad0 0x558ed0910790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7c645f640 1 -- 172.21.15.94:0/2057954340 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed090a2e0 msgr2=0x558ed0979170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7c645f640 1 --2- 172.21.15.94:0/2057954340 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed090a2e0 0x558ed0979170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7c645f640 1 -- 172.21.15.94:0/2057954340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0901ad0 msgr2=0x558ed0910790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7c645f640 1 --2- 172.21.15.94:0/2057954340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0901ad0 0x558ed0910790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.621+0000 7fd7c645f640 1 -- 172.21.15.94:0/2057954340 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ed07fe7b0 con 0x558ed0900d60 2024-02-17T12:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.622+0000 7fd7c645f640 1 --2- 172.21.15.94:0/2057954340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0900d60 0x558ed0910250 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc011de0 tx=0x7fd7bc005620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c9ef70c1e613b50 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.622+0000 7fd7a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.622+0000 7fd7a9ffb640 1 -- 172.21.15.94:0/2057954340 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc026970 con 0x558ed0900d60 2024-02-17T12:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.622+0000 7fd7a9ffb640 1 -- 172.21.15.94:0/2057954340 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd7bc026ad0 con 0x558ed0900d60 2024-02-17T12:04:21.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.622+0000 7fd7cb371740 1 -- 172.21.15.94:0/2057954340 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7bc0263f0 con 0x558ed0900d60 2024-02-17T12:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.622+0000 7fd7a9ffb640 1 -- 172.21.15.94:0/2057954340 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7bc026c90 con 0x558ed0900d60 2024-02-17T12:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.622+0000 7fd7cb371740 10 client.26626 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.622+0000 7fd7cb371740 1 -- 172.21.15.94:0/2057954340 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x558ed0829540 con 0x558ed0900d60 2024-02-17T12:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.623+0000 7fd7a9ffb640 1 -- 172.21.15.94:0/2057954340 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7bc034650 con 0x558ed0900d60 2024-02-17T12:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.623+0000 7fd7a9ffb640 1 -- 172.21.15.94:0/2057954340 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd7bc02f040 con 0x558ed0900d60 2024-02-17T12:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.623+0000 7fd7cb371740 10 client.26626 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.623+0000 7fd7cb371740 10 client.26626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.623+0000 7fd7cb371740 20 client.26626 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.623+0000 7fd7cb371740 1 --2- 172.21.15.94:0/2057954340 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558ed097ada0 0x558ed097d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.623+0000 7fd7cb371740 4 client.26626 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.623+0000 7fd7cb371740 1 -- 172.21.15.94:0/2057954340 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fd7bc0263f0 con 0x558ed097ada0 2024-02-17T12:04:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.624+0000 7fd7c5c5e640 1 --2- 172.21.15.94:0/2057954340 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558ed097ada0 0x558ed097d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.624+0000 7fd7c5c5e640 1 --2- 172.21.15.94:0/2057954340 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558ed097ada0 0x558ed097d190 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=264cd7b46499aa9 server_cookie=76d3f8b1d18abb96 in_seq=0 out_seq=0 2024-02-17T12:04:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.624+0000 7fd7a9ffb640 10 client.26626 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.651+0000 7fd7a9ffb640 1 -- 172.21.15.94:0/2057954340 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fd7bc0263f0 con 0x558ed097ada0 2024-02-17T12:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.651+0000 7fd7a9ffb640 10 client.26626 handle_command_reply: tid=0 2024-02-17T12:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.651+0000 7fd7cb371740 1 client.26626 shutdown 2024-02-17T12:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.651+0000 7fd7cb371740 1 -- 172.21.15.94:0/2057954340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0900d60 msgr2=0x558ed0910250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.651+0000 7fd7cb371740 1 --2- 172.21.15.94:0/2057954340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0900d60 0x558ed0910250 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc011de0 tx=0x7fd7bc005620 comp rx=0 tx=0).stop 2024-02-17T12:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.652+0000 7fd7cb371740 1 -- 172.21.15.94:0/2057954340 shutdown_connections 2024-02-17T12:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.652+0000 7fd7cb371740 1 --2- 172.21.15.94:0/2057954340 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558ed097ada0 0x558ed097d190 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.652+0000 7fd7cb371740 1 --2- 172.21.15.94:0/2057954340 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558ed090a2e0 0x558ed0979170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.652+0000 7fd7a9ffb640 0 client.26626 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.652+0000 7fd7cb371740 1 --2- 172.21.15.94:0/2057954340 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558ed0901ad0 0x558ed0910790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.652+0000 7fd7cb371740 1 --2- 172.21.15.94:0/2057954340 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ed0900d60 0x558ed0910250 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.652+0000 7fd7cb371740 1 -- 172.21.15.94:0/2057954340 >> 172.21.15.94:0/2057954340 conn(0x558ed08c8550 msgr2=0x558ed08fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.652+0000 7fd7cb371740 1 -- 172.21.15.94:0/2057954340 shutdown_connections 2024-02-17T12:04:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.652+0000 7fd7cb371740 1 -- 172.21.15.94:0/2057954340 wait complete. 2024-02-17T12:04:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.652+0000 7fd7cb371740 20 client.26626 trim_cache size 0 max 16384 2024-02-17T12:04:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:04:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:04:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:04:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.653+0000 7fd7c86c3640 1 -- 172.21.15.94:0/462622715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd7ac0a22d0 msgr2=0x7fd7ac0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.653+0000 7fd7c86c3640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd7ac0a22d0 0x7fd7ac0a4790 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc011550 tx=0x7fd7bc005f90 comp rx=0 tx=0).stop 2024-02-17T12:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.653+0000 7fd7c86c3640 1 -- 172.21.15.94:0/462622715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7c0116210 msgr2=0x7fd7c01a6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.653+0000 7fd7c86c3640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7c0116210 0x7fd7c01a6ab0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd7c006a590 tx=0x7fd7b800dea0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.653+0000 7fd7c86c3640 1 -- 172.21.15.94:0/462622715 shutdown_connections 2024-02-17T12:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.653+0000 7fd7c86c3640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd7ac0a22d0 0x7fd7ac0a4790 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.653+0000 7fd7c86c3640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd7c0116210 0x7fd7c01a6ab0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.653+0000 7fd7c86c3640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd7c010b8b0 0x7fd7c01a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.653+0000 7fd7c86c3640 1 --2- 172.21.15.94:0/462622715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd7c01086c0 0x7fd7c01a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.653+0000 7fd7c86c3640 1 -- 172.21.15.94:0/462622715 >> 172.21.15.94:0/462622715 conn(0x7fd7c006c6b0 msgr2=0x7fd7c01165f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.653+0000 7fd7c86c3640 1 -- 172.21.15.94:0/462622715 shutdown_connections 2024-02-17T12:04:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:21.653+0000 7fd7c86c3640 1 -- 172.21.15.94:0/462622715 wait complete. 2024-02-17T12:04:24.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:23 smithi125 ceph-mon[36819]: pgmap v192: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 19 MiB/s wr, 780 op/s 2024-02-17T12:04:24.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:23 smithi094 ceph-mon[32499]: pgmap v192: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 19 MiB/s wr, 780 op/s 2024-02-17T12:04:24.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:23 smithi189 ceph-mon[36588]: pgmap v192: 97 pgs: 97 active+clean; 4.4 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 19 MiB/s wr, 780 op/s 2024-02-17T12:04:25.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 365.0 s, 2726.4 tps, lat 36.433 ms stddev 45.020 2024-02-17T12:04:26.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:25 smithi125 ceph-mon[36819]: pgmap v193: 97 pgs: 97 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 21 MiB/s wr, 882 op/s 2024-02-17T12:04:26.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:25 smithi094 ceph-mon[32499]: pgmap v193: 97 pgs: 97 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 21 MiB/s wr, 882 op/s 2024-02-17T12:04:26.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:25 smithi189 ceph-mon[36588]: pgmap v193: 97 pgs: 97 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 21 MiB/s wr, 882 op/s 2024-02-17T12:04:28.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:27 smithi125 ceph-mon[36819]: pgmap v194: 97 pgs: 97 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 592 op/s 2024-02-17T12:04:28.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:27 smithi094 ceph-mon[32499]: pgmap v194: 97 pgs: 97 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 592 op/s 2024-02-17T12:04:28.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:27 smithi189 ceph-mon[36588]: pgmap v194: 97 pgs: 97 active+clean; 4.5 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 14 MiB/s wr, 592 op/s 2024-02-17T12:04:30.185 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:29 smithi125 ceph-mon[36819]: pgmap v195: 97 pgs: 97 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 22 MiB/s wr, 967 op/s 2024-02-17T12:04:30.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:29 smithi094 ceph-mon[32499]: pgmap v195: 97 pgs: 97 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 22 MiB/s wr, 967 op/s 2024-02-17T12:04:30.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 370.0 s, 2048.4 tps, lat 49.299 ms stddev 94.301 2024-02-17T12:04:30.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:29 smithi189 ceph-mon[36588]: pgmap v195: 97 pgs: 97 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 22 MiB/s wr, 967 op/s 2024-02-17T12:04:30.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:04:30.664 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:04:30.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.817+0000 7f595522e640 1 -- 172.21.15.94:0/3869576793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5950100270 msgr2=0x7f5950100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:30.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.817+0000 7f595522e640 1 --2- 172.21.15.94:0/3869576793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5950100270 0x7f5950100650 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5944009930 tx=0x7f594402bc30 comp rx=0 tx=0).stop 2024-02-17T12:04:30.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.818+0000 7f595522e640 1 -- 172.21.15.94:0/3869576793 shutdown_connections 2024-02-17T12:04:30.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.818+0000 7f595522e640 1 --2- 172.21.15.94:0/3869576793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5950105340 0x7f595010cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.818+0000 7f595522e640 1 --2- 172.21.15.94:0/3869576793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5950100c20 0x7f5950104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.818+0000 7f595522e640 1 --2- 172.21.15.94:0/3869576793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5950100270 0x7f5950100650 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.818+0000 7f595522e640 1 -- 172.21.15.94:0/3869576793 >> 172.21.15.94:0/3869576793 conn(0x7f5950077230 msgr2=0x7f5950077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:30.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.818+0000 7f595522e640 1 -- 172.21.15.94:0/3869576793 shutdown_connections 2024-02-17T12:04:30.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.818+0000 7f595522e640 1 -- 172.21.15.94:0/3869576793 wait complete. 2024-02-17T12:04:30.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.819+0000 7f595522e640 1 Processor -- start 2024-02-17T12:04:30.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.819+0000 7f595522e640 1 -- start start 2024-02-17T12:04:30.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.819+0000 7f595522e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5950100270 0x7f595019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.819+0000 7f595522e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5950105340 0x7f595019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.819+0000 7f595522e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5950199800 0x7f5950199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.819+0000 7f595522e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595019a2b0 con 0x7f5950100270 2024-02-17T12:04:30.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.819+0000 7f595522e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595019a420 con 0x7f5950105340 2024-02-17T12:04:30.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.819+0000 7f595522e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595019a590 con 0x7f5950199800 2024-02-17T12:04:30.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.820+0000 7f594ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5950100270 0x7f595019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.820+0000 7f594ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5950100270 0x7f595019f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42488/0 (socket says 172.21.15.94:42488) 2024-02-17T12:04:30.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.820+0000 7f594ed9d640 1 -- 172.21.15.94:0/2408467030 learned_addr learned my addr 172.21.15.94:0/2408467030 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:30.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.820+0000 7f594f59e640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5950199800 0x7f5950199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.820+0000 7f594ed9d640 1 -- 172.21.15.94:0/2408467030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5950199800 msgr2=0x7f5950199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:30.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.820+0000 7f594e59c640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5950105340 0x7f595019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.821+0000 7f594ed9d640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5950199800 0x7f5950199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.821+0000 7f594ed9d640 1 -- 172.21.15.94:0/2408467030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5950105340 msgr2=0x7f595019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:30.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.821+0000 7f594ed9d640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5950105340 0x7f595019fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.821+0000 7f594ed9d640 1 -- 172.21.15.94:0/2408467030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5944009560 con 0x7f5950100270 2024-02-17T12:04:30.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.821+0000 7f594ed9d640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5950100270 0x7f595019f640 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5944000c00 tx=0x7f594402cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:30.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.822+0000 7f5937fff640 1 -- 172.21.15.94:0/2408467030 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944004350 con 0x7f5950100270 2024-02-17T12:04:30.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.822+0000 7f5937fff640 1 -- 172.21.15.94:0/2408467030 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59440044b0 con 0x7f5950100270 2024-02-17T12:04:30.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.823+0000 7f5937fff640 1 -- 172.21.15.94:0/2408467030 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944003d00 con 0x7f5950100270 2024-02-17T12:04:30.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.823+0000 7f595522e640 1 -- 172.21.15.94:0/2408467030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595010d970 con 0x7f5950100270 2024-02-17T12:04:30.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.823+0000 7f595522e640 1 -- 172.21.15.94:0/2408467030 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595010de30 con 0x7f5950100270 2024-02-17T12:04:30.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.826+0000 7f5957e00740 1 Processor -- start 2024-02-17T12:04:30.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.827+0000 7f5957e00740 1 -- start start 2024-02-17T12:04:30.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.827+0000 7f5957e00740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7abfb2b0 0x562b7abfb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.827+0000 7f5957e00740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac04270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.827+0000 7f5957e00740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7ac047b0 0x562b7ac06ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.827+0000 7f5957e00740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7ab6a280 con 0x562b7abfb2b0 2024-02-17T12:04:30.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.827+0000 7f5957e00740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7ab677b0 con 0x562b7ac047b0 2024-02-17T12:04:30.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.827+0000 7f5957e00740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7ab62f90 con 0x562b7abfbbd0 2024-02-17T12:04:30.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.828+0000 7f5937fff640 1 -- 172.21.15.94:0/2408467030 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5944004df0 con 0x7f5950100270 2024-02-17T12:04:30.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.828+0000 7f594e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7ac047b0 0x562b7ac06ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.828+0000 7f594e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7ac047b0 0x562b7ac06ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41950/0 (socket says 172.21.15.94:41950) 2024-02-17T12:04:30.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.828+0000 7f594e59c640 1 -- 172.21.15.94:0/4288441933 learned_addr learned my addr 172.21.15.94:0/4288441933 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:30.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.828+0000 7f594e59c640 1 -- 172.21.15.94:0/4288441933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 msgr2=0x562b7ac04270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:04:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.828+0000 7f594e59c640 1 --2- 172.21.15.94:0/4288441933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac04270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.828+0000 7f594e59c640 1 -- 172.21.15.94:0/4288441933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7abfb2b0 msgr2=0x562b7abfb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:30.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.828+0000 7f594e59c640 1 --2- 172.21.15.94:0/4288441933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7abfb2b0 0x562b7abfb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.828+0000 7f594e59c640 1 -- 172.21.15.94:0/4288441933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b7ab21570 con 0x562b7ac047b0 2024-02-17T12:04:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.828+0000 7f5937fff640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59380a2360 0x7f59380a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.829+0000 7f594e59c640 1 --2- 172.21.15.94:0/4288441933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7ac047b0 0x562b7ac06ba0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5940009b10 tx=0x7f594000e2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=603cced76f6d7a1b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.829+0000 7f5937fff640 1 -- 172.21.15.94:0/2408467030 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f59440f0640 con 0x7f5950100270 2024-02-17T12:04:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.829+0000 7f594f59e640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59380a2360 0x7f59380a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.829+0000 7f59357fa640 1 -- 172.21.15.94:0/4288441933 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594000ecf0 con 0x562b7ac047b0 2024-02-17T12:04:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.829+0000 7f59357fa640 1 -- 172.21.15.94:0/4288441933 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f594000ee50 con 0x562b7ac047b0 2024-02-17T12:04:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.830+0000 7f59357fa640 1 -- 172.21.15.94:0/4288441933 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940004ea0 con 0x562b7ac047b0 2024-02-17T12:04:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.830+0000 7f5957e00740 1 -- 172.21.15.94:0/4288441933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7ac047b0 msgr2=0x562b7ac06ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.830+0000 7f594f59e640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59380a2360 0x7f59380a4820 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5948005fd0 tx=0x7f5948009a70 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.830+0000 7f5957e00740 1 --2- 172.21.15.94:0/4288441933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7ac047b0 0x562b7ac06ba0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5940009b10 tx=0x7f594000e2f0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.830+0000 7f5957e00740 1 -- 172.21.15.94:0/4288441933 shutdown_connections 2024-02-17T12:04:30.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.830+0000 7f5957e00740 1 --2- 172.21.15.94:0/4288441933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7ac047b0 0x562b7ac06ba0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.830+0000 7f5957e00740 1 --2- 172.21.15.94:0/4288441933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac04270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.830+0000 7f5957e00740 1 --2- 172.21.15.94:0/4288441933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7abfb2b0 0x562b7abfb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.830+0000 7f5957e00740 1 -- 172.21.15.94:0/4288441933 >> 172.21.15.94:0/4288441933 conn(0x562b7abf5990 msgr2=0x562b7abf5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.830+0000 7f5957e00740 1 -- 172.21.15.94:0/4288441933 shutdown_connections 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.830+0000 7f5957e00740 1 -- 172.21.15.94:0/4288441933 wait complete. 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.831+0000 7f5957e00740 1 -- start start 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.831+0000 7f5957e00740 1 Processor -- start 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.831+0000 7f5957e00740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7abfb2b0 0x562b7ac01670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.831+0000 7f5957e00740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac01bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.831+0000 7f5957e00740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7ac047b0 0x562b7ac020f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.831+0000 7f5957e00740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7a836430 con 0x562b7ac047b0 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.831+0000 7f5957e00740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7a7f2e00 con 0x562b7abfb2b0 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.831+0000 7f5957e00740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7aae7fd0 con 0x562b7abfbbd0 2024-02-17T12:04:30.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.831+0000 7f5957e00740 10 client.0 _ll_register_callbacks cb 0x562b7ab3dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:30.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.831+0000 7f594ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7abfb2b0 0x562b7ac01670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.831+0000 7f594ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7abfb2b0 0x562b7ac01670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41956/0 (socket says 172.21.15.94:41956) 2024-02-17T12:04:30.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.831+0000 7f594ed9d640 1 -- 172.21.15.94:0/1259441785 learned_addr learned my addr 172.21.15.94:0/1259441785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:30.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.832+0000 7f594ed9d640 1 -- 172.21.15.94:0/1259441785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 msgr2=0x562b7ac01bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:30.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.832+0000 7f594ed9d640 1 --2- 172.21.15.94:0/1259441785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac01bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.832+0000 7f594ed9d640 1 -- 172.21.15.94:0/1259441785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7ac047b0 msgr2=0x562b7ac020f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:30.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.832+0000 7f594ed9d640 1 --2- 172.21.15.94:0/1259441785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7ac047b0 0x562b7ac020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.832+0000 7f594ed9d640 1 -- 172.21.15.94:0/1259441785 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5940009740 con 0x562b7abfb2b0 2024-02-17T12:04:30.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.833+0000 7f594ed9d640 1 --2- 172.21.15.94:0/1259441785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7abfb2b0 0x562b7ac01670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f59440edef0 tx=0x7f5944003e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21e7ccc9eaccf4d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.833+0000 7f5932ff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.833+0000 7f5932ff5640 1 -- 172.21.15.94:0/1259441785 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440bd620 con 0x562b7abfb2b0 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.833+0000 7f5932ff5640 1 -- 172.21.15.94:0/1259441785 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f59440bd8c0 con 0x562b7abfb2b0 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.833+0000 7f5957e00740 1 -- 172.21.15.94:0/1259441785 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b7a8627e0 con 0x562b7abfb2b0 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.834+0000 7f5932ff5640 1 -- 172.21.15.94:0/1259441785 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59440c1120 con 0x562b7abfb2b0 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.834+0000 7f5957e00740 10 client.26591 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.834+0000 7f5957e00740 1 -- 172.21.15.94:0/1259441785 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562b7ab61610 con 0x562b7abfb2b0 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.834+0000 7f5932ff5640 1 -- 172.21.15.94:0/1259441785 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59440d9080 con 0x562b7abfb2b0 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.834+0000 7f5932ff5640 1 -- 172.21.15.94:0/1259441785 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f594403c070 con 0x562b7abfb2b0 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.834+0000 7f5957e00740 10 client.26591 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.834+0000 7f5957e00740 10 client.26591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.834+0000 7f5957e00740 20 client.26591 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.834+0000 7f5957e00740 1 --2- 172.21.15.94:0/1259441785 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b7ac147e0 0x562b7ac16c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.834+0000 7f5957e00740 4 client.26591 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:04:30.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.834+0000 7f5957e00740 1 -- 172.21.15.94:0/1259441785 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b7a89f720 con 0x562b7ac147e0 2024-02-17T12:04:30.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.835+0000 7f594e59c640 1 --2- 172.21.15.94:0/1259441785 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b7ac147e0 0x562b7ac16c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.835+0000 7f594e59c640 1 --2- 172.21.15.94:0/1259441785 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b7ac147e0 0x562b7ac16c00 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4b7ac5e9a4feb8f server_cookie=b7981506e4bb7289 in_seq=0 out_seq=0 2024-02-17T12:04:30.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.835+0000 7f5932ff5640 10 client.26591 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:30.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5932ff5640 1 -- 172.21.15.94:0/1259441785 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b7a89f720 con 0x562b7ac147e0 2024-02-17T12:04:30.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5932ff5640 10 client.26591 handle_command_reply: tid=0 2024-02-17T12:04:30.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5957e00740 1 client.26591 shutdown 2024-02-17T12:04:30.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5957e00740 1 -- 172.21.15.94:0/1259441785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7abfb2b0 msgr2=0x562b7ac01670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:30.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5957e00740 1 --2- 172.21.15.94:0/1259441785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7abfb2b0 0x562b7ac01670 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f59440edef0 tx=0x7f5944003e60 comp rx=0 tx=0).stop 2024-02-17T12:04:30.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5957e00740 1 -- 172.21.15.94:0/1259441785 shutdown_connections 2024-02-17T12:04:30.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5957e00740 1 --2- 172.21.15.94:0/1259441785 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b7ac147e0 0x562b7ac16c00 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5957e00740 1 --2- 172.21.15.94:0/1259441785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7ac047b0 0x562b7ac020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5957e00740 1 --2- 172.21.15.94:0/1259441785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac01bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5957e00740 1 --2- 172.21.15.94:0/1259441785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7abfb2b0 0x562b7ac01670 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5957e00740 1 -- 172.21.15.94:0/1259441785 >> 172.21.15.94:0/1259441785 conn(0x562b7abf5990 msgr2=0x562b7abf7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:30.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5932ff5640 0 client.26591 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:30.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5957e00740 1 -- 172.21.15.94:0/1259441785 shutdown_connections 2024-02-17T12:04:30.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5957e00740 1 -- 172.21.15.94:0/1259441785 wait complete. 2024-02-17T12:04:30.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.836+0000 7f5957e00740 20 client.26591 trim_cache size 0 max 16384 2024-02-17T12:04:30.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.858+0000 7f5957e00740 1 Processor -- start 2024-02-17T12:04:30.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.858+0000 7f5957e00740 1 -- start start 2024-02-17T12:04:30.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.858+0000 7f5957e00740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7abfb2b0 0x562b7ac03da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f5957e00740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac01f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f5957e00740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7ac047b0 0x562b7abfe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f594ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7abfb2b0 0x562b7ac03da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f5957e00740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7abf1120 con 0x562b7abfb2b0 2024-02-17T12:04:30.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f5957e00740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7abf6ba0 con 0x562b7ac047b0 2024-02-17T12:04:30.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f5957e00740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7abf69e0 con 0x562b7abfbbd0 2024-02-17T12:04:30.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f594ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7abfb2b0 0x562b7ac03da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42506/0 (socket says 172.21.15.94:42506) 2024-02-17T12:04:30.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f594ed9d640 1 -- 172.21.15.94:0/2544251757 learned_addr learned my addr 172.21.15.94:0/2544251757 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:30.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f594e59c640 1 --2- 172.21.15.94:0/2544251757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac01f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f594f59e640 1 --2- 172.21.15.94:0/2544251757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7ac047b0 0x562b7abfe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f594ed9d640 1 -- 172.21.15.94:0/2544251757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 msgr2=0x562b7ac01f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:30.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f594ed9d640 1 --2- 172.21.15.94:0/2544251757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac01f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f594ed9d640 1 -- 172.21.15.94:0/2544251757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7ac047b0 msgr2=0x562b7abfe8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:30.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f594ed9d640 1 --2- 172.21.15.94:0/2544251757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7ac047b0 0x562b7abfe8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f594ed9d640 1 -- 172.21.15.94:0/2544251757 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b7aafdac0 con 0x562b7abfb2b0 2024-02-17T12:04:30.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.859+0000 7f594e59c640 1 --2- 172.21.15.94:0/2544251757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac01f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:04:30.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.860+0000 7f594ed9d640 1 --2- 172.21.15.94:0/2544251757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7abfb2b0 0x562b7ac03da0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f59440ee380 tx=0x7f5944003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95938567ba1fbf0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:30.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.860+0000 7f5934ff9640 1 -- 172.21.15.94:0/2544251757 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440c1120 con 0x562b7abfb2b0 2024-02-17T12:04:30.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.860+0000 7f5934ff9640 1 -- 172.21.15.94:0/2544251757 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562b7a8627e0 con 0x562b7abfb2b0 2024-02-17T12:04:30.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.860+0000 7f5934ff9640 1 -- 172.21.15.94:0/2544251757 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440b8f10 con 0x562b7abfb2b0 2024-02-17T12:04:30.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.861+0000 7f5957e00740 1 -- 172.21.15.94:0/2544251757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7abfb2b0 msgr2=0x562b7ac03da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:30.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.861+0000 7f5957e00740 1 --2- 172.21.15.94:0/2544251757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7abfb2b0 0x562b7ac03da0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f59440ee380 tx=0x7f5944003f90 comp rx=0 tx=0).stop 2024-02-17T12:04:30.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.861+0000 7f5957e00740 1 -- 172.21.15.94:0/2544251757 shutdown_connections 2024-02-17T12:04:30.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.861+0000 7f5957e00740 1 --2- 172.21.15.94:0/2544251757 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7ac047b0 0x562b7abfe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.861+0000 7f5957e00740 1 --2- 172.21.15.94:0/2544251757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac01f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.861+0000 7f5957e00740 1 --2- 172.21.15.94:0/2544251757 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7abfb2b0 0x562b7ac03da0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.861+0000 7f5957e00740 1 -- 172.21.15.94:0/2544251757 >> 172.21.15.94:0/2544251757 conn(0x562b7abf5990 msgr2=0x562b7abfc950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:30.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.861+0000 7f5957e00740 1 -- 172.21.15.94:0/2544251757 shutdown_connections 2024-02-17T12:04:30.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.861+0000 7f5957e00740 1 -- 172.21.15.94:0/2544251757 wait complete. 2024-02-17T12:04:30.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.861+0000 7f5957e00740 1 -- start start 2024-02-17T12:04:30.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.861+0000 7f5957e00740 1 Processor -- start 2024-02-17T12:04:30.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.862+0000 7f5957e00740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7abfb2b0 0x562b7ac0bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.862+0000 7f5957e00740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac09e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.862+0000 7f5957e00740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7ac047b0 0x562b7ac0a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.862+0000 7f594ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7abfb2b0 0x562b7ac0bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.862+0000 7f594ed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7abfb2b0 0x562b7ac0bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41986/0 (socket says 172.21.15.94:41986) 2024-02-17T12:04:30.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.862+0000 7f594ed9d640 1 -- 172.21.15.94:0/2752227592 learned_addr learned my addr 172.21.15.94:0/2752227592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:30.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.862+0000 7f5957e00740 1 -- 172.21.15.94:0/2752227592 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7ac0aab0 con 0x562b7ac047b0 2024-02-17T12:04:30.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.862+0000 7f5957e00740 1 -- 172.21.15.94:0/2752227592 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7ac0ac20 con 0x562b7abfb2b0 2024-02-17T12:04:30.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.862+0000 7f5957e00740 1 -- 172.21.15.94:0/2752227592 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b7ac6ad10 con 0x562b7abfbbd0 2024-02-17T12:04:30.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.862+0000 7f5957e00740 10 client.0 _ll_register_callbacks cb 0x7f59400092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:30.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.862+0000 7f594e59c640 1 --2- 172.21.15.94:0/2752227592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac09e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f594f59e640 1 --2- 172.21.15.94:0/2752227592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7ac047b0 0x562b7ac0a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f594e59c640 1 -- 172.21.15.94:0/2752227592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7abfb2b0 msgr2=0x562b7ac0bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:30.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f594e59c640 1 --2- 172.21.15.94:0/2752227592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7abfb2b0 0x562b7ac0bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f594e59c640 1 -- 172.21.15.94:0/2752227592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7ac047b0 msgr2=0x562b7ac0a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f594e59c640 1 --2- 172.21.15.94:0/2752227592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7ac047b0 0x562b7ac0a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f594e59c640 1 -- 172.21.15.94:0/2752227592 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b7ab61610 con 0x562b7abfbbd0 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f594e59c640 1 --2- 172.21.15.94:0/2752227592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac09e50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x562b7ac02c40 tx=0x7f5940004a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c05397d25eff04d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f5932ff5640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f5932ff5640 1 -- 172.21.15.94:0/2752227592 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940020040 con 0x562b7abfbbd0 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f5957e00740 1 -- 172.21.15.94:0/2752227592 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f594000ee50 con 0x562b7abfbbd0 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f5932ff5640 1 -- 172.21.15.94:0/2752227592 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5940015030 con 0x562b7abfbbd0 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f5932ff5640 1 -- 172.21.15.94:0/2752227592 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5940004820 con 0x562b7abfbbd0 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f5957e00740 10 client.26638 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.863+0000 7f5957e00740 1 -- 172.21.15.94:0/2752227592 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562b7ab236a0 con 0x562b7abfbbd0 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.864+0000 7f5932ff5640 1 -- 172.21.15.94:0/2752227592 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f594000ee50 con 0x562b7abfbbd0 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.864+0000 7f5932ff5640 1 -- 172.21.15.94:0/2752227592 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f594000c070 con 0x562b7abfbbd0 2024-02-17T12:04:30.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.864+0000 7f5957e00740 10 client.26638 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:30.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.864+0000 7f5957e00740 10 client.26638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:30.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.864+0000 7f5957e00740 20 client.26638 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:30.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.864+0000 7f5957e00740 1 --2- 172.21.15.94:0/2752227592 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b7ac6c5a0 0x562b7ac6e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:30.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.864+0000 7f5957e00740 4 client.26638 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:04:30.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.864+0000 7f5957e00740 1 -- 172.21.15.94:0/2752227592 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562b7a8ee890 con 0x562b7ac6c5a0 2024-02-17T12:04:30.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.864+0000 7f594ed9d640 1 --2- 172.21.15.94:0/2752227592 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b7ac6c5a0 0x562b7ac6e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.865+0000 7f594ed9d640 1 --2- 172.21.15.94:0/2752227592 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b7ac6c5a0 0x562b7ac6e9c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb6e1a7825626fd8 server_cookie=c3a3bd6fd74829e5 in_seq=0 out_seq=0 2024-02-17T12:04:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.865+0000 7f5932ff5640 10 client.26638 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.865+0000 7f5932ff5640 1 -- 172.21.15.94:0/2752227592 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562b7a8ee890 con 0x562b7ac6c5a0 2024-02-17T12:04:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.865+0000 7f5932ff5640 10 client.26638 handle_command_reply: tid=0 2024-02-17T12:04:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.865+0000 7f5957e00740 1 client.26638 shutdown 2024-02-17T12:04:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.865+0000 7f5957e00740 1 -- 172.21.15.94:0/2752227592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 msgr2=0x562b7ac09e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.865+0000 7f5957e00740 1 --2- 172.21.15.94:0/2752227592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac09e50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x562b7ac02c40 tx=0x7f5940004a70 comp rx=0 tx=0).stop 2024-02-17T12:04:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.866+0000 7f5957e00740 1 -- 172.21.15.94:0/2752227592 shutdown_connections 2024-02-17T12:04:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.866+0000 7f5957e00740 1 --2- 172.21.15.94:0/2752227592 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x562b7ac6c5a0 0x562b7ac6e9c0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.866+0000 7f5957e00740 1 --2- 172.21.15.94:0/2752227592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b7ac047b0 0x562b7ac0a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.866+0000 7f5957e00740 1 --2- 172.21.15.94:0/2752227592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b7abfbbd0 0x562b7ac09e50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.866+0000 7f5957e00740 1 --2- 172.21.15.94:0/2752227592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b7abfb2b0 0x562b7ac0bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.866+0000 7f5932ff5640 0 client.26638 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:30.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.866+0000 7f5957e00740 1 -- 172.21.15.94:0/2752227592 >> 172.21.15.94:0/2752227592 conn(0x562b7abf5990 msgr2=0x562b7abf6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:30.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.866+0000 7f5957e00740 1 -- 172.21.15.94:0/2752227592 shutdown_connections 2024-02-17T12:04:30.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.866+0000 7f5957e00740 1 -- 172.21.15.94:0/2752227592 wait complete. 2024-02-17T12:04:30.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.866+0000 7f5957e00740 20 client.26638 trim_cache size 0 max 16384 2024-02-17T12:04:30.870 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:04:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:04:30.870 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:04:30.870 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:04:30.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.867+0000 7f595522e640 1 -- 172.21.15.94:0/2408467030 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59380a2360 msgr2=0x7f59380a4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:30.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.867+0000 7f595522e640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59380a2360 0x7f59380a4820 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5948005fd0 tx=0x7f5948009a70 comp rx=0 tx=0).stop 2024-02-17T12:04:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.867+0000 7f595522e640 1 -- 172.21.15.94:0/2408467030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5950100270 msgr2=0x7f595019f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.867+0000 7f595522e640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5950100270 0x7f595019f640 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5944000c00 tx=0x7f594402cd80 comp rx=0 tx=0).stop 2024-02-17T12:04:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.867+0000 7f595522e640 1 -- 172.21.15.94:0/2408467030 shutdown_connections 2024-02-17T12:04:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.867+0000 7f595522e640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59380a2360 0x7f59380a4820 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.867+0000 7f595522e640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5950199800 0x7f5950199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.867+0000 7f595522e640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5950105340 0x7f595019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.867+0000 7f595522e640 1 --2- 172.21.15.94:0/2408467030 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5950100270 0x7f595019f640 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.867+0000 7f595522e640 1 -- 172.21.15.94:0/2408467030 >> 172.21.15.94:0/2408467030 conn(0x7f5950077230 msgr2=0x7f59500fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.867+0000 7f595522e640 1 -- 172.21.15.94:0/2408467030 shutdown_connections 2024-02-17T12:04:30.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:30.867+0000 7f595522e640 1 -- 172.21.15.94:0/2408467030 wait complete. 2024-02-17T12:04:30.881 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:04:30.882 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:04:31.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.024+0000 7f1e24e8a640 1 -- 172.21.15.94:0/3339263445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2010d740 msgr2=0x7f1e2010db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.024+0000 7f1e24e8a640 1 --2- 172.21.15.94:0/3339263445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2010d740 0x7f1e2010db20 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009930 tx=0x7f1e1002bc30 comp rx=0 tx=0).stop 2024-02-17T12:04:31.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.024+0000 7f1e24e8a640 1 -- 172.21.15.94:0/3339263445 shutdown_connections 2024-02-17T12:04:31.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.024+0000 7f1e24e8a640 1 --2- 172.21.15.94:0/3339263445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e201080f0 0x7f1e20114600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.024+0000 7f1e24e8a640 1 --2- 172.21.15.94:0/3339263445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e20107730 0x7f1e20107bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.024+0000 7f1e24e8a640 1 --2- 172.21.15.94:0/3339263445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e2010d740 0x7f1e2010db20 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.024+0000 7f1e24e8a640 1 -- 172.21.15.94:0/3339263445 >> 172.21.15.94:0/3339263445 conn(0x7f1e200756d0 msgr2=0x7f1e20077af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.024+0000 7f1e24e8a640 1 -- 172.21.15.94:0/3339263445 shutdown_connections 2024-02-17T12:04:31.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.024+0000 7f1e24e8a640 1 -- 172.21.15.94:0/3339263445 wait complete. 2024-02-17T12:04:31.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.025+0000 7f1e24e8a640 1 Processor -- start 2024-02-17T12:04:31.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.025+0000 7f1e24e8a640 1 -- start start 2024-02-17T12:04:31.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.025+0000 7f1e24e8a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e20107730 0x7f1e2019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e24e8a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e201080f0 0x7f1e2019e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e24e8a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e2010d740 0x7f1e201a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e24e8a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e201a2d10 con 0x7f1e201080f0 2024-02-17T12:04:31.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e24e8a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e201a2e80 con 0x7f1e20107730 2024-02-17T12:04:31.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e24e8a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e201a3040 con 0x7f1e2010d740 2024-02-17T12:04:31.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e1dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e201080f0 0x7f1e2019e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e1dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e201080f0 0x7f1e2019e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42530/0 (socket says 172.21.15.94:42530) 2024-02-17T12:04:31.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e1dd9b640 1 -- 172.21.15.94:0/2070355410 learned_addr learned my addr 172.21.15.94:0/2070355410 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e1ed9d640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e2010d740 0x7f1e201a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e1e59c640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e20107730 0x7f1e2019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e1ed9d640 1 -- 172.21.15.94:0/2070355410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e20107730 msgr2=0x7f1e2019de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e1ed9d640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e20107730 0x7f1e2019de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e1ed9d640 1 -- 172.21.15.94:0/2070355410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e201080f0 msgr2=0x7f1e2019e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e1ed9d640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e201080f0 0x7f1e2019e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.026+0000 7f1e1ed9d640 1 -- 172.21.15.94:0/2070355410 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e10009560 con 0x7f1e2010d740 2024-02-17T12:04:31.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.027+0000 7f1e1ed9d640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e2010d740 0x7f1e201a2710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800da10 tx=0x7f1e1800ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.027+0000 7f1e0f7fe640 1 -- 172.21.15.94:0/2070355410 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1800bd70 con 0x7f1e2010d740 2024-02-17T12:04:31.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.027+0000 7f1e0f7fe640 1 -- 172.21.15.94:0/2070355410 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1e18003e50 con 0x7f1e2010d740 2024-02-17T12:04:31.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.027+0000 7f1e0f7fe640 1 -- 172.21.15.94:0/2070355410 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e180106c0 con 0x7f1e2010d740 2024-02-17T12:04:31.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.027+0000 7f1e24e8a640 1 -- 172.21.15.94:0/2070355410 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e201a3320 con 0x7f1e2010d740 2024-02-17T12:04:31.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.027+0000 7f1e24e8a640 1 -- 172.21.15.94:0/2070355410 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e201aaba0 con 0x7f1e2010d740 2024-02-17T12:04:31.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.029+0000 7f1e24e8a640 1 -- 172.21.15.94:0/2070355410 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e200676e0 con 0x7f1e2010d740 2024-02-17T12:04:31.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.031+0000 7f1e0f7fe640 1 -- 172.21.15.94:0/2070355410 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1e18004000 con 0x7f1e2010d740 2024-02-17T12:04:31.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.031+0000 7f1e0f7fe640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1e080a22b0 0x7f1e080a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.032+0000 7f1e0f7fe640 1 -- 172.21.15.94:0/2070355410 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1e180ce3c0 con 0x7f1e2010d740 2024-02-17T12:04:31.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.032+0000 7f1e1e59c640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1e080a22b0 0x7f1e080a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.032+0000 7f1e1e59c640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1e080a22b0 0x7f1e080a4770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e10005ab0 tx=0x7f1e10005fd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.035+0000 7f1e0f7fe640 1 -- 172.21.15.94:0/2070355410 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1e18096fe0 con 0x7f1e2010d740 2024-02-17T12:04:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.364+0000 7f1e24e8a640 1 -- 172.21.15.94:0/2070355410 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e2010fcd0 con 0x7f1e2010d740 2024-02-17T12:04:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.365+0000 7f1e0f7fe640 1 -- 172.21.15.94:0/2070355410 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11338 (secure 0 0 0) 0x7f1e1809a6a0 con 0x7f1e2010d740 2024-02-17T12:04:31.366 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:04:31.366 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:03:14.932679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26335":{"gid":26335,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/39050300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":39050300},{"type":"v1","addr":"172.21.15.189:6833","nonce":39050300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:04:31.367 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-02-17T12:04:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.368+0000 7f1e24e8a640 1 -- 172.21.15.94:0/2070355410 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1e080a22b0 msgr2=0x7f1e080a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.368+0000 7f1e24e8a640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1e080a22b0 0x7f1e080a4770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1e10005ab0 tx=0x7f1e10005fd0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.368+0000 7f1e24e8a640 1 -- 172.21.15.94:0/2070355410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e2010d740 msgr2=0x7f1e201a2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.368+0000 7f1e24e8a640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e2010d740 0x7f1e201a2710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1e1800da10 tx=0x7f1e1800ded0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.369+0000 7f1e24e8a640 1 -- 172.21.15.94:0/2070355410 shutdown_connections 2024-02-17T12:04:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.369+0000 7f1e24e8a640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1e080a22b0 0x7f1e080a4770 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.369+0000 7f1e24e8a640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e2010d740 0x7f1e201a2710 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.369+0000 7f1e24e8a640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1e201080f0 0x7f1e2019e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.369+0000 7f1e24e8a640 1 --2- 172.21.15.94:0/2070355410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1e20107730 0x7f1e2019de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.369+0000 7f1e24e8a640 1 -- 172.21.15.94:0/2070355410 >> 172.21.15.94:0/2070355410 conn(0x7f1e200756d0 msgr2=0x7f1e20075fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.369+0000 7f1e24e8a640 1 -- 172.21.15.94:0/2070355410 shutdown_connections 2024-02-17T12:04:31.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.369+0000 7f1e24e8a640 1 -- 172.21.15.94:0/2070355410 wait complete. 2024-02-17T12:04:31.384 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:04:31.384 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:04:31.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.549+0000 7f29b142a640 1 -- 172.21.15.94:0/2947691476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac100260 msgr2=0x7f29ac100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.549+0000 7f29b142a640 1 --2- 172.21.15.94:0/2947691476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac100260 0x7f29ac100640 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f299c00fb40 tx=0x7f299c02fdc0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.549+0000 7f29b142a640 1 -- 172.21.15.94:0/2947691476 shutdown_connections 2024-02-17T12:04:31.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.549+0000 7f29b142a640 1 --2- 172.21.15.94:0/2947691476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ac105330 0x7f29ac10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.549+0000 7f29b142a640 1 --2- 172.21.15.94:0/2947691476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29ac100c10 0x7f29ac104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.549+0000 7f29b142a640 1 --2- 172.21.15.94:0/2947691476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac100260 0x7f29ac100640 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.549+0000 7f29b142a640 1 -- 172.21.15.94:0/2947691476 >> 172.21.15.94:0/2947691476 conn(0x7f29ac077180 msgr2=0x7f29ac077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.549+0000 7f29b142a640 1 -- 172.21.15.94:0/2947691476 shutdown_connections 2024-02-17T12:04:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.549+0000 7f29b142a640 1 -- 172.21.15.94:0/2947691476 wait complete. 2024-02-17T12:04:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.550+0000 7f29b142a640 1 Processor -- start 2024-02-17T12:04:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.550+0000 7f29b142a640 1 -- start start 2024-02-17T12:04:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.550+0000 7f29b142a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac100260 0x7f29ac195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.550+0000 7f29b142a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29ac105330 0x7f29ac195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.550+0000 7f29b142a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ac199f10 0x7f29ac19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.550+0000 7f29b142a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac19a900 con 0x7f29ac100260 2024-02-17T12:04:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.550+0000 7f29b142a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac19aa70 con 0x7f29ac199f10 2024-02-17T12:04:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.550+0000 7f29b142a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ac19abe0 con 0x7f29ac105330 2024-02-17T12:04:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.550+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ac199f10 0x7f29ac19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.550+0000 7f29aaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac100260 0x7f29ac195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.550+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ac199f10 0x7f29ac19a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42014/0 (socket says 172.21.15.94:42014) 2024-02-17T12:04:31.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.550+0000 7f29ab7fe640 1 -- 172.21.15.94:0/4076954404 learned_addr learned my addr 172.21.15.94:0/4076954404 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.551+0000 7f29aaffd640 1 -- 172.21.15.94:0/4076954404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29ac105330 msgr2=0x7f29ac195b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:04:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.551+0000 7f29aaffd640 1 --2- 172.21.15.94:0/4076954404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29ac105330 0x7f29ac195b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.551+0000 7f29aaffd640 1 -- 172.21.15.94:0/4076954404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ac199f10 msgr2=0x7f29ac19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.551+0000 7f29aaffd640 1 --2- 172.21.15.94:0/4076954404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ac199f10 0x7f29ac19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.551+0000 7f29aaffd640 1 -- 172.21.15.94:0/4076954404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c00f770 con 0x7f29ac100260 2024-02-17T12:04:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.551+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/4076954404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ac199f10 0x7f29ac19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.551+0000 7f29aaffd640 1 --2- 172.21.15.94:0/4076954404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac100260 0x7f29ac195650 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f299c009230 tx=0x7f299c005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.552+0000 7f29a8ff9640 1 -- 172.21.15.94:0/4076954404 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c030dc0 con 0x7f29ac100260 2024-02-17T12:04:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.552+0000 7f29a8ff9640 1 -- 172.21.15.94:0/4076954404 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f299c00f5f0 con 0x7f29ac100260 2024-02-17T12:04:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.552+0000 7f29a8ff9640 1 -- 172.21.15.94:0/4076954404 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c040b60 con 0x7f29ac100260 2024-02-17T12:04:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.552+0000 7f29b142a640 1 -- 172.21.15.94:0/4076954404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ac1a21c0 con 0x7f29ac100260 2024-02-17T12:04:31.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.552+0000 7f29b142a640 1 -- 172.21.15.94:0/4076954404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ac1a2530 con 0x7f29ac100260 2024-02-17T12:04:31.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.555+0000 7f29a8ff9640 1 -- 172.21.15.94:0/4076954404 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f299c00f110 con 0x7f29ac100260 2024-02-17T12:04:31.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.555+0000 7f29b3ffe740 1 Processor -- start 2024-02-17T12:04:31.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.555+0000 7f29b3ffe740 1 -- start start 2024-02-17T12:04:31.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29b3ffe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1250 0x561ce70b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29a8ff9640 1 --2- 172.21.15.94:0/4076954404 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f29980a2380 0x7f29980a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29b3ffe740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1b70 0x561ce70ba210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29b3ffe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 0x561ce70bcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29b3ffe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce706c960 con 0x561ce70b1250 2024-02-17T12:04:31.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29b3ffe740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce7020310 con 0x561ce70b1b70 2024-02-17T12:04:31.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29b3ffe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce701d840 con 0x561ce70ba750 2024-02-17T12:04:31.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1250 0x561ce70b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1250 0x561ce70b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42554/0 (socket says 172.21.15.94:42554) 2024-02-17T12:04:31.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29ab7fe640 1 -- 172.21.15.94:0/3292966627 learned_addr learned my addr 172.21.15.94:0/3292966627 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29a8ff9640 1 -- 172.21.15.94:0/4076954404 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f299c0efe80 con 0x7f29ac100260 2024-02-17T12:04:31.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29aaffd640 1 --2- 172.21.15.94:0/4076954404 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f29980a2380 0x7f29980a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/3292966627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 0x561ce70bcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29a27fc640 1 --2- 172.21.15.94:0/3292966627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1b70 0x561ce70ba210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.556+0000 7f29ab7fe640 1 -- 172.21.15.94:0/3292966627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 msgr2=0x561ce70bcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.557+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/3292966627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 0x561ce70bcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.557+0000 7f29ab7fe640 1 -- 172.21.15.94:0/3292966627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1b70 msgr2=0x561ce70ba210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.557+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/3292966627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1b70 0x561ce70ba210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.557+0000 7f29ab7fe640 1 -- 172.21.15.94:0/3292966627 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ce6fdb060 con 0x561ce70b1250 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.557+0000 7f29a27fc640 1 --2- 172.21.15.94:0/3292966627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1b70 0x561ce70ba210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.557+0000 7f29aaffd640 1 --2- 172.21.15.94:0/4076954404 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f29980a2380 0x7f29980a4840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f299c0bc670 tx=0x7f299c000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.557+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/3292966627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1250 0x561ce70b1630 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f29a400afc0 tx=0x7f29a4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b176b4a00ac0d979 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.557+0000 7f29a17fa640 1 -- 172.21.15.94:0/3292966627 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4020040 con 0x561ce70b1250 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.557+0000 7f29a17fa640 1 -- 172.21.15.94:0/3292966627 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29a4021040 con 0x561ce70b1250 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.557+0000 7f29a17fa640 1 -- 172.21.15.94:0/3292966627 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a402c780 con 0x561ce70b1250 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.558+0000 7f29b3ffe740 1 -- 172.21.15.94:0/3292966627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1250 msgr2=0x561ce70b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.558+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/3292966627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1250 0x561ce70b1630 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f29a400afc0 tx=0x7f29a4009260 comp rx=0 tx=0).stop 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.558+0000 7f29b3ffe740 1 -- 172.21.15.94:0/3292966627 shutdown_connections 2024-02-17T12:04:31.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.558+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/3292966627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 0x561ce70bcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.558+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/3292966627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1b70 0x561ce70ba210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.558+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/3292966627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1250 0x561ce70b1630 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.558+0000 7f29b3ffe740 1 -- 172.21.15.94:0/3292966627 >> 172.21.15.94:0/3292966627 conn(0x561ce70ab980 msgr2=0x561ce70abd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.558+0000 7f29b3ffe740 1 -- 172.21.15.94:0/3292966627 shutdown_connections 2024-02-17T12:04:31.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.558+0000 7f29b3ffe740 1 -- 172.21.15.94:0/3292966627 wait complete. 2024-02-17T12:04:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.558+0000 7f29b3ffe740 1 -- start start 2024-02-17T12:04:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.558+0000 7f29b3ffe740 1 Processor -- start 2024-02-17T12:04:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29b3ffe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70b1250 0x561ce70b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29b3ffe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29b3ffe740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70ba750 0x561ce70b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70b1250 0x561ce70b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70b1250 0x561ce70b77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56054/0 (socket says 172.21.15.94:56054) 2024-02-17T12:04:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29ab7fe640 1 -- 172.21.15.94:0/1943495979 learned_addr learned my addr 172.21.15.94:0/1943495979 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1943495979 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce7019020 con 0x561ce70b1b70 2024-02-17T12:04:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1943495979 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce6cec430 con 0x561ce70ba750 2024-02-17T12:04:31.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1943495979 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce6ca8e00 con 0x561ce70b1250 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/1943495979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70ba750 0x561ce70b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29ab7fe640 1 -- 172.21.15.94:0/1943495979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70ba750 msgr2=0x561ce70b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/1943495979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70ba750 0x561ce70b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29ab7fe640 1 -- 172.21.15.94:0/1943495979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 msgr2=0x561ce70b7cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/1943495979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70b7cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.559+0000 7f29ab7fe640 1 -- 172.21.15.94:0/1943495979 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a400ad70 con 0x561ce70b1250 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.560+0000 7f29b3ffe740 10 client.0 _ll_register_callbacks cb 0x561ce6fc0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.560+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/1943495979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70b1250 0x561ce70b77b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f29a4005ca0 tx=0x7f29a402c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80a9a93a82a629ee server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.560+0000 7f29927fc640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.560+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1943495979 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ce6d187e0 con 0x561ce70b1250 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.560+0000 7f29927fc640 1 -- 172.21.15.94:0/1943495979 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4020040 con 0x561ce70b1250 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.560+0000 7f29927fc640 1 -- 172.21.15.94:0/1943495979 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f29a402d890 con 0x561ce70b1250 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.561+0000 7f29927fc640 1 -- 172.21.15.94:0/1943495979 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29a402da90 con 0x561ce70b1250 2024-02-17T12:04:31.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.561+0000 7f29b3ffe740 10 client.26653 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.561+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1943495979 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=25}) v3 -- 0x561ce70176a0 con 0x561ce70b1250 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.561+0000 7f29927fc640 1 -- 172.21.15.94:0/1943495979 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f29a402dbf0 con 0x561ce70b1250 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.561+0000 7f29b3ffe740 10 client.26653 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.561+0000 7f29b3ffe740 10 client.26653 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.561+0000 7f29b3ffe740 20 client.26653 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.561+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/1943495979 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x561ce70ca990 0x561ce70cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.561+0000 7f29b3ffe740 4 client.26653 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.561+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1943495979 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ce6d187e0 con 0x561ce70ca990 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.561+0000 7f29a27fc640 1 --2- 172.21.15.94:0/1943495979 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x561ce70ca990 0x561ce70cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.562+0000 7f29927fc640 10 client.26653 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.562+0000 7f29a27fc640 1 --2- 172.21.15.94:0/1943495979 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x561ce70ca990 0x561ce70cce50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=941aafa5f79b1b72 server_cookie=e4271960754d738f in_seq=0 out_seq=0 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.562+0000 7f29927fc640 1 -- 172.21.15.94:0/1943495979 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ce6d187e0 con 0x561ce70ca990 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.562+0000 7f29927fc640 10 client.26653 handle_command_reply: tid=0 2024-02-17T12:04:31.569 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.562+0000 7f29b3ffe740 1 client.26653 shutdown 2024-02-17T12:04:31.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.562+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1943495979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70b1250 msgr2=0x561ce70b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.562+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/1943495979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70b1250 0x561ce70b77b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f29a4005ca0 tx=0x7f29a402c480 comp rx=0 tx=0).stop 2024-02-17T12:04:31.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.563+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1943495979 shutdown_connections 2024-02-17T12:04:31.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.563+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/1943495979 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x561ce70ca990 0x561ce70cce50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.563+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/1943495979 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70ba750 0x561ce70b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.563+0000 7f29927fc640 0 client.26653 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:04:31.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.563+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/1943495979 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.563+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/1943495979 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70b1250 0x561ce70b77b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.563+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1943495979 >> 172.21.15.94:0/1943495979 conn(0x561ce70ab980 msgr2=0x561ce70ad6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.563+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1943495979 shutdown_connections 2024-02-17T12:04:31.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.563+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1943495979 wait complete. 2024-02-17T12:04:31.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.563+0000 7f29b3ffe740 20 client.26653 trim_cache size 0 max 16384 2024-02-17T12:04:31.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.581+0000 7f29b3ffe740 1 Processor -- start 2024-02-17T12:04:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.581+0000 7f29b3ffe740 1 -- start start 2024-02-17T12:04:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.581+0000 7f29b3ffe740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 0x561ce70b9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.582+0000 7f29b3ffe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.582+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 0x561ce70b9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.582+0000 7f29a27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.582+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 0x561ce70b9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42042/0 (socket says 172.21.15.94:42042) 2024-02-17T12:04:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.582+0000 7f29a27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42556/0 (socket says 172.21.15.94:42556) 2024-02-17T12:04:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.582+0000 7f29b3ffe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 0x561ce70b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.582+0000 7f29ab7fe640 1 -- 172.21.15.94:0/1281185811 learned_addr learned my addr 172.21.15.94:0/1281185811 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.582+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1281185811 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce6f9e130 con 0x561ce70b1b70 2024-02-17T12:04:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.582+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1281185811 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce70a72b0 con 0x561ce70b1250 2024-02-17T12:04:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.582+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1281185811 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce70b5d00 con 0x561ce70ba750 2024-02-17T12:04:31.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.582+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/1281185811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 0x561ce70b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.583+0000 7f29ab7fe640 1 -- 172.21.15.94:0/1281185811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 msgr2=0x561ce70b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.583+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/1281185811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 0x561ce70b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.583+0000 7f29ab7fe640 1 -- 172.21.15.94:0/1281185811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 msgr2=0x561ce70b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.583+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/1281185811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.583+0000 7f29ab7fe640 1 -- 172.21.15.94:0/1281185811 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ce6fb3c20 con 0x561ce70b1250 2024-02-17T12:04:31.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.583+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/1281185811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 0x561ce70b9ee0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f29a402c5d0 tx=0x7f29a4033000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fa778adb92c098c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.583+0000 7f29a0ff9640 1 -- 172.21.15.94:0/1281185811 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a402c3e0 con 0x561ce70b1250 2024-02-17T12:04:31.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.583+0000 7f29a0ff9640 1 -- 172.21.15.94:0/1281185811 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29a4063020 con 0x561ce70b1250 2024-02-17T12:04:31.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.583+0000 7f29a0ff9640 1 -- 172.21.15.94:0/1281185811 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a402db20 con 0x561ce70b1250 2024-02-17T12:04:31.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.584+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1281185811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 msgr2=0x561ce70b9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.584+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/1281185811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 0x561ce70b9ee0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f29a402c5d0 tx=0x7f29a4033000 comp rx=0 tx=0).stop 2024-02-17T12:04:31.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.584+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1281185811 shutdown_connections 2024-02-17T12:04:31.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.584+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/1281185811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 0x561ce70b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.584+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/1281185811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.584+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/1281185811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 0x561ce70b9ee0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.584+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1281185811 >> 172.21.15.94:0/1281185811 conn(0x561ce70ab980 msgr2=0x561ce70a6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.584+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1281185811 shutdown_connections 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.584+0000 7f29b3ffe740 1 -- 172.21.15.94:0/1281185811 wait complete. 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.584+0000 7f29b3ffe740 1 -- start start 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29b3ffe740 1 Processor -- start 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29b3ffe740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 0x561ce70c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29b3ffe740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70bfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29b3ffe740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 0x561ce70c0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29b3ffe740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce70c0b60 con 0x561ce70b1b70 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29b3ffe740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce7120f80 con 0x561ce70b1250 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29b3ffe740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce71210c0 con 0x561ce70ba750 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29b3ffe740 10 client.0 _ll_register_callbacks cb 0x7f29a400a620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:31.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 0x561ce70c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29a27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70bfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29ab7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 0x561ce70c1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42058/0 (socket says 172.21.15.94:42058) 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29a27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70bfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42558/0 (socket says 172.21.15.94:42558) 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29a27fc640 1 -- 172.21.15.94:0/479152339 learned_addr learned my addr 172.21.15.94:0/479152339 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/479152339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 0x561ce70c0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29a27fc640 1 -- 172.21.15.94:0/479152339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 msgr2=0x561ce70c0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29a27fc640 1 --2- 172.21.15.94:0/479152339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 0x561ce70c0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29a27fc640 1 -- 172.21.15.94:0/479152339 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 msgr2=0x561ce70c1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29a27fc640 1 --2- 172.21.15.94:0/479152339 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 0x561ce70c1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29a27fc640 1 -- 172.21.15.94:0/479152339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ce70176a0 con 0x561ce70b1b70 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.585+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/479152339 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 0x561ce70c1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.586+0000 7f29a27fc640 1 --2- 172.21.15.94:0/479152339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70bfde0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x561ce70b0d20 tx=0x7f2994009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b024d065c15eaeb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.586+0000 7f29927fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:04:31.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.586+0000 7f29927fc640 1 -- 172.21.15.94:0/479152339 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2994019500 con 0x561ce70b1b70 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.586+0000 7f29927fc640 1 -- 172.21.15.94:0/479152339 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2994016070 con 0x561ce70b1b70 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.586+0000 7f29b3ffe740 1 -- 172.21.15.94:0/479152339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29a4021040 con 0x561ce70b1b70 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.586+0000 7f29927fc640 1 -- 172.21.15.94:0/479152339 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2994019660 con 0x561ce70b1b70 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.586+0000 7f29b3ffe740 10 client.17157 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.586+0000 7f29b3ffe740 1 -- 172.21.15.94:0/479152339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561ce6fd98c0 con 0x561ce70b1b70 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.586+0000 7f29927fc640 1 -- 172.21.15.94:0/479152339 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f299402d080 con 0x561ce70b1b70 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.586+0000 7f29927fc640 1 -- 172.21.15.94:0/479152339 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f29940105d0 con 0x561ce70b1b70 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.587+0000 7f29b3ffe740 10 client.17157 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.587+0000 7f29b3ffe740 10 client.17157 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.587+0000 7f29b3ffe740 20 client.17157 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.587+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/479152339 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x561ce71228a0 0x561ce7124cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.587+0000 7f29b3ffe740 4 client.17157 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.587+0000 7f29b3ffe740 1 -- 172.21.15.94:0/479152339 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561ce6d55720 con 0x561ce71228a0 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.587+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/479152339 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x561ce71228a0 0x561ce7124cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.587+0000 7f29927fc640 10 client.17157 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:04:31.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.587+0000 7f29ab7fe640 1 --2- 172.21.15.94:0/479152339 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x561ce71228a0 0x561ce7124cc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f266394ee0e3c89d server_cookie=be83c67c00aba474 in_seq=0 out_seq=0 2024-02-17T12:04:31.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.589+0000 7f29927fc640 1 -- 172.21.15.94:0/479152339 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561ce6d55720 con 0x561ce71228a0 2024-02-17T12:04:31.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.589+0000 7f29927fc640 10 client.17157 handle_command_reply: tid=0 2024-02-17T12:04:31.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.589+0000 7f29b3ffe740 1 client.17157 shutdown 2024-02-17T12:04:31.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.590+0000 7f29b3ffe740 1 -- 172.21.15.94:0/479152339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 msgr2=0x561ce70bfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.590+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/479152339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70bfde0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x561ce70b0d20 tx=0x7f2994009c80 comp rx=0 tx=0).stop 2024-02-17T12:04:31.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.590+0000 7f29b3ffe740 1 -- 172.21.15.94:0/479152339 shutdown_connections 2024-02-17T12:04:31.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.590+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/479152339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ce70ba750 0x561ce70c0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.590+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/479152339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ce70b1b70 0x561ce70bfde0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.590+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/479152339 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x561ce71228a0 0x561ce7124cc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.590+0000 7f29b3ffe740 1 --2- 172.21.15.94:0/479152339 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ce70b1250 0x561ce70c1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.590+0000 7f29927fc640 0 client.17157 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:04:31.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.590+0000 7f29b3ffe740 1 -- 172.21.15.94:0/479152339 >> 172.21.15.94:0/479152339 conn(0x561ce70ab980 msgr2=0x561ce70b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.590+0000 7f29b3ffe740 1 -- 172.21.15.94:0/479152339 shutdown_connections 2024-02-17T12:04:31.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.590+0000 7f29b3ffe740 1 -- 172.21.15.94:0/479152339 wait complete. 2024-02-17T12:04:31.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.590+0000 7f29b3ffe740 20 client.17157 trim_cache size 0 max 16384 2024-02-17T12:04:31.597 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:04:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.591+0000 7f29b142a640 1 -- 172.21.15.94:0/4076954404 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f29980a2380 msgr2=0x7f29980a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.591+0000 7f29b142a640 1 --2- 172.21.15.94:0/4076954404 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f29980a2380 0x7f29980a4840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f299c0bc670 tx=0x7f299c000bc0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.591+0000 7f29b142a640 1 -- 172.21.15.94:0/4076954404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac100260 msgr2=0x7f29ac195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.591+0000 7f29b142a640 1 --2- 172.21.15.94:0/4076954404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac100260 0x7f29ac195650 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f299c009230 tx=0x7f299c005230 comp rx=0 tx=0).stop 2024-02-17T12:04:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.591+0000 7f29b142a640 1 -- 172.21.15.94:0/4076954404 shutdown_connections 2024-02-17T12:04:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.591+0000 7f29b142a640 1 --2- 172.21.15.94:0/4076954404 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f29980a2380 0x7f29980a4840 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.591+0000 7f29b142a640 1 --2- 172.21.15.94:0/4076954404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f29ac199f10 0x7f29ac19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.591+0000 7f29b142a640 1 --2- 172.21.15.94:0/4076954404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29ac105330 0x7f29ac195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.591+0000 7f29b142a640 1 --2- 172.21.15.94:0/4076954404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f29ac100260 0x7f29ac195650 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.591+0000 7f29b142a640 1 -- 172.21.15.94:0/4076954404 >> 172.21.15.94:0/4076954404 conn(0x7f29ac077180 msgr2=0x7f29ac0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.592+0000 7f29b142a640 1 -- 172.21.15.94:0/4076954404 shutdown_connections 2024-02-17T12:04:31.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.592+0000 7f29b142a640 1 -- 172.21.15.94:0/4076954404 wait complete. 2024-02-17T12:04:31.607 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:04:31.607 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:04:31.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.750+0000 7f3c559f4640 1 -- 172.21.15.94:0/374647618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c50100c10 msgr2=0x7f3c50104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.750+0000 7f3c559f4640 1 --2- 172.21.15.94:0/374647618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c50100c10 0x7f3c50104c50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009930 tx=0x7f3c3802bc30 comp rx=0 tx=0).stop 2024-02-17T12:04:31.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.750+0000 7f3c559f4640 1 -- 172.21.15.94:0/374647618 shutdown_connections 2024-02-17T12:04:31.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.750+0000 7f3c559f4640 1 --2- 172.21.15.94:0/374647618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c50105330 0x7f3c5010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.750+0000 7f3c559f4640 1 --2- 172.21.15.94:0/374647618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c50100c10 0x7f3c50104c50 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.750+0000 7f3c559f4640 1 --2- 172.21.15.94:0/374647618 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c50100260 0x7f3c50100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.750+0000 7f3c559f4640 1 -- 172.21.15.94:0/374647618 >> 172.21.15.94:0/374647618 conn(0x7f3c50077180 msgr2=0x7f3c50077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.750+0000 7f3c559f4640 1 -- 172.21.15.94:0/374647618 shutdown_connections 2024-02-17T12:04:31.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.750+0000 7f3c559f4640 1 -- 172.21.15.94:0/374647618 wait complete. 2024-02-17T12:04:31.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c559f4640 1 Processor -- start 2024-02-17T12:04:31.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c559f4640 1 -- start start 2024-02-17T12:04:31.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c559f4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c50100260 0x7f3c5019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c559f4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c50105330 0x7f3c5019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c559f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c50197630 0x7f3c50197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c559f4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c50198020 con 0x7f3c50197630 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c559f4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c50198190 con 0x7f3c50105330 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c559f4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c50198350 con 0x7f3c50100260 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c4f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c50197630 0x7f3c50197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c4f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c50197630 0x7f3c50197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42584/0 (socket says 172.21.15.94:42584) 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c4f7fe640 1 -- 172.21.15.94:0/4106294635 learned_addr learned my addr 172.21.15.94:0/4106294635 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c4effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c50100260 0x7f3c5019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c47fff640 1 --2- 172.21.15.94:0/4106294635 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c50105330 0x7f3c5019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c4f7fe640 1 -- 172.21.15.94:0/4106294635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c50100260 msgr2=0x7f3c5019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c4f7fe640 1 --2- 172.21.15.94:0/4106294635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c50100260 0x7f3c5019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c4f7fe640 1 -- 172.21.15.94:0/4106294635 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c50105330 msgr2=0x7f3c5019d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c4f7fe640 1 --2- 172.21.15.94:0/4106294635 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c50105330 0x7f3c5019d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.751+0000 7f3c4f7fe640 1 -- 172.21.15.94:0/4106294635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c38009560 con 0x7f3c50197630 2024-02-17T12:04:31.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.752+0000 7f3c4f7fe640 1 --2- 172.21.15.94:0/4106294635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c50197630 0x7f3c50197ae0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800eb10 tx=0x7f3c4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.752+0000 7f3c549f2640 1 -- 172.21.15.94:0/4106294635 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48004b20 con 0x7f3c50197630 2024-02-17T12:04:31.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.752+0000 7f3c559f4640 1 -- 172.21.15.94:0/4106294635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c501a42f0 con 0x7f3c50197630 2024-02-17T12:04:31.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.752+0000 7f3c549f2640 1 -- 172.21.15.94:0/4106294635 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c48004c80 con 0x7f3c50197630 2024-02-17T12:04:31.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.752+0000 7f3c549f2640 1 -- 172.21.15.94:0/4106294635 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48010670 con 0x7f3c50197630 2024-02-17T12:04:31.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.752+0000 7f3c559f4640 1 -- 172.21.15.94:0/4106294635 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c501a4820 con 0x7f3c50197630 2024-02-17T12:04:31.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.756+0000 7f3c549f2640 1 -- 172.21.15.94:0/4106294635 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3c48010890 con 0x7f3c50197630 2024-02-17T12:04:31.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.756+0000 7f3c549f2640 1 --2- 172.21.15.94:0/4106294635 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3c3c0a2230 0x7f3c3c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.756+0000 7f3c549f2640 1 -- 172.21.15.94:0/4106294635 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3c48014070 con 0x7f3c50197630 2024-02-17T12:04:31.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.756+0000 7f3c4effd640 1 --2- 172.21.15.94:0/4106294635 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3c3c0a2230 0x7f3c3c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c4effd640 1 --2- 172.21.15.94:0/4106294635 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3c3c0a2230 0x7f3c3c0a46f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009bd0 tx=0x7f3c40005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c585c9740 1 Processor -- start 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c585c9740 1 -- start start 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c585c9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d28630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c585c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d31210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c585c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 0x55adf8d33b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c585c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf8ce3960 con 0x55adf8d28b70 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c585c9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf8c97310 con 0x55adf8d28250 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c585c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf8c94840 con 0x55adf8d31750 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d31210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c4effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d28630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d31210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42598/0 (socket says 172.21.15.94:42598) 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c4effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d28630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42092/0 (socket says 172.21.15.94:42092) 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c47fff640 1 -- 172.21.15.94:0/1776028251 learned_addr learned my addr 172.21.15.94:0/1776028251 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.757+0000 7f3c4f7fe640 1 --2- 172.21.15.94:0/1776028251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 0x55adf8d33b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.758+0000 7f3c4effd640 1 -- 172.21.15.94:0/1776028251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 msgr2=0x55adf8d33b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.758+0000 7f3c4effd640 1 --2- 172.21.15.94:0/1776028251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 0x55adf8d33b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.758+0000 7f3c4effd640 1 -- 172.21.15.94:0/1776028251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 msgr2=0x55adf8d31210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.758+0000 7f3c4effd640 1 --2- 172.21.15.94:0/1776028251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d31210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.758+0000 7f3c4effd640 1 -- 172.21.15.94:0/1776028251 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adf8c52060 con 0x55adf8d28250 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.758+0000 7f3c4effd640 1 --2- 172.21.15.94:0/1776028251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d28630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009fd0 tx=0x7f3c40014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed00b7eb605671d1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.758+0000 7f3c45ffb640 1 -- 172.21.15.94:0/1776028251 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4000ee80 con 0x55adf8d28250 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.758+0000 7f3c45ffb640 1 -- 172.21.15.94:0/1776028251 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c40021070 con 0x55adf8d28250 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.758+0000 7f3c45ffb640 1 -- 172.21.15.94:0/1776028251 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c400266d0 con 0x55adf8d28250 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.759+0000 7f3c585c9740 1 -- 172.21.15.94:0/1776028251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 msgr2=0x55adf8d28630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.759+0000 7f3c585c9740 1 --2- 172.21.15.94:0/1776028251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d28630 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009fd0 tx=0x7f3c40014000 comp rx=0 tx=0).stop 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.759+0000 7f3c585c9740 1 -- 172.21.15.94:0/1776028251 shutdown_connections 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.759+0000 7f3c585c9740 1 --2- 172.21.15.94:0/1776028251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 0x55adf8d33b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.759+0000 7f3c585c9740 1 --2- 172.21.15.94:0/1776028251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d31210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.759+0000 7f3c585c9740 1 --2- 172.21.15.94:0/1776028251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d28630 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.759+0000 7f3c585c9740 1 -- 172.21.15.94:0/1776028251 >> 172.21.15.94:0/1776028251 conn(0x55adf8d22980 msgr2=0x55adf8d22d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.759+0000 7f3c585c9740 1 -- 172.21.15.94:0/1776028251 shutdown_connections 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.759+0000 7f3c585c9740 1 -- 172.21.15.94:0/1776028251 wait complete. 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.759+0000 7f3c585c9740 1 -- start start 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.759+0000 7f3c585c9740 1 Processor -- start 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c585c9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d2e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c585c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d2ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c585c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 0x55adf8d2f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c585c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf8c90020 con 0x55adf8d28b70 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c585c9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf8963430 con 0x55adf8d28250 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c585c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf891fe00 con 0x55adf8d31750 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c585c9740 10 client.0 _ll_register_callbacks cb 0x55adf8c37cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d2ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d2ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42608/0 (socket says 172.21.15.94:42608) 2024-02-17T12:04:31.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c47fff640 1 -- 172.21.15.94:0/1149116552 learned_addr learned my addr 172.21.15.94:0/1149116552 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c4f7fe640 1 --2- 172.21.15.94:0/1149116552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 0x55adf8d2f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c4effd640 1 --2- 172.21.15.94:0/1149116552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d2e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c47fff640 1 -- 172.21.15.94:0/1149116552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 msgr2=0x55adf8d2f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c47fff640 1 --2- 172.21.15.94:0/1149116552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 0x55adf8d2f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c47fff640 1 -- 172.21.15.94:0/1149116552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 msgr2=0x55adf8d2e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c47fff640 1 --2- 172.21.15.94:0/1149116552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d2e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c47fff640 1 -- 172.21.15.94:0/1149116552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c40017070 con 0x55adf8d28b70 2024-02-17T12:04:31.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c47fff640 1 --2- 172.21.15.94:0/1149116552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d2ed60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000c00 tx=0x7f3c3802ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04f9788318f275a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c2effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:04:31.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c2effd640 1 -- 172.21.15.94:0/1149116552 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38004040 con 0x55adf8d28b70 2024-02-17T12:04:31.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c2effd640 1 -- 172.21.15.94:0/1149116552 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3c380041a0 con 0x55adf8d28b70 2024-02-17T12:04:31.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c585c9740 1 -- 172.21.15.94:0/1149116552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55adf898f7e0 con 0x55adf8d28b70 2024-02-17T12:04:31.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c2effd640 1 -- 172.21.15.94:0/1149116552 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c38004360 con 0x55adf8d28b70 2024-02-17T12:04:31.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c585c9740 10 client.17169 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:31.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.760+0000 7f3c585c9740 1 -- 172.21.15.94:0/1149116552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55adf8c8e6a0 con 0x55adf8d28b70 2024-02-17T12:04:31.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.761+0000 7f3c2effd640 1 -- 172.21.15.94:0/1149116552 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55adf898f7e0 con 0x55adf8d28b70 2024-02-17T12:04:31.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.761+0000 7f3c2effd640 1 -- 172.21.15.94:0/1149116552 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3c3803c040 con 0x55adf8d28b70 2024-02-17T12:04:31.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.761+0000 7f3c585c9740 10 client.17169 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:31.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.761+0000 7f3c585c9740 10 client.17169 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:04:31.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.761+0000 7f3c585c9740 20 client.17169 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:31.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.761+0000 7f3c585c9740 1 --2- 172.21.15.94:0/1149116552 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55adf8d41bb0 0x55adf8d43fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.761+0000 7f3c585c9740 4 client.17169 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:04:31.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.761+0000 7f3c585c9740 1 -- 172.21.15.94:0/1149116552 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adf89cc720 con 0x55adf8d41bb0 2024-02-17T12:04:31.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.761+0000 7f3c4effd640 1 --2- 172.21.15.94:0/1149116552 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55adf8d41bb0 0x55adf8d43fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.761+0000 7f3c2effd640 10 client.17169 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:04:31.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.761+0000 7f3c4effd640 1 --2- 172.21.15.94:0/1149116552 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55adf8d41bb0 0x55adf8d43fd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8701735ec4770ad7 server_cookie=af9873324506e568 in_seq=0 out_seq=0 2024-02-17T12:04:31.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c2effd640 1 -- 172.21.15.94:0/1149116552 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55adf89cc720 con 0x55adf8d41bb0 2024-02-17T12:04:31.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c2effd640 10 client.17169 handle_command_reply: tid=0 2024-02-17T12:04:31.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c585c9740 1 client.17169 shutdown 2024-02-17T12:04:31.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c585c9740 1 -- 172.21.15.94:0/1149116552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 msgr2=0x55adf8d2ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c585c9740 1 --2- 172.21.15.94:0/1149116552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d2ed60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000c00 tx=0x7f3c3802ca10 comp rx=0 tx=0).stop 2024-02-17T12:04:31.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c585c9740 1 -- 172.21.15.94:0/1149116552 shutdown_connections 2024-02-17T12:04:31.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c585c9740 1 --2- 172.21.15.94:0/1149116552 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55adf8d41bb0 0x55adf8d43fd0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c585c9740 1 --2- 172.21.15.94:0/1149116552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 0x55adf8d2f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c2effd640 0 client.17169 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:04:31.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c585c9740 1 --2- 172.21.15.94:0/1149116552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d2ed60 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c585c9740 1 --2- 172.21.15.94:0/1149116552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d2e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c585c9740 1 -- 172.21.15.94:0/1149116552 >> 172.21.15.94:0/1149116552 conn(0x55adf8d22980 msgr2=0x55adf8d246d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c585c9740 1 -- 172.21.15.94:0/1149116552 shutdown_connections 2024-02-17T12:04:31.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c585c9740 1 -- 172.21.15.94:0/1149116552 wait complete. 2024-02-17T12:04:31.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.762+0000 7f3c585c9740 20 client.17169 trim_cache size 0 max 16384 2024-02-17T12:04:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.776+0000 7f3c585c9740 1 Processor -- start 2024-02-17T12:04:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.777+0000 7f3c585c9740 1 -- start start 2024-02-17T12:04:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.777+0000 7f3c585c9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d30f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.777+0000 7f3c585c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d28b70 0x55adf8d2f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.777+0000 7f3c585c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d31750 0x55adf8d2ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.777+0000 7f3c585c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf8c15130 con 0x55adf8d31750 2024-02-17T12:04:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.777+0000 7f3c585c9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf8d23780 con 0x55adf8d28250 2024-02-17T12:04:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.777+0000 7f3c585c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf8d1e370 con 0x55adf8d28b70 2024-02-17T12:04:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c4f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d31750 0x55adf8d2ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c4f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d31750 0x55adf8d2ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42612/0 (socket says 172.21.15.94:42612) 2024-02-17T12:04:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c4f7fe640 1 -- 172.21.15.94:0/2581161769 learned_addr learned my addr 172.21.15.94:0/2581161769 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d28b70 0x55adf8d2f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c4effd640 1 --2- 172.21.15.94:0/2581161769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d30f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c4f7fe640 1 -- 172.21.15.94:0/2581161769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d28b70 msgr2=0x55adf8d2f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c4f7fe640 1 --2- 172.21.15.94:0/2581161769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d28b70 0x55adf8d2f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c4f7fe640 1 -- 172.21.15.94:0/2581161769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 msgr2=0x55adf8d30f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c4f7fe640 1 --2- 172.21.15.94:0/2581161769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d30f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c4f7fe640 1 -- 172.21.15.94:0/2581161769 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adf8c2ac20 con 0x55adf8d31750 2024-02-17T12:04:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c4effd640 1 --2- 172.21.15.94:0/2581161769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d30f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c4f7fe640 1 --2- 172.21.15.94:0/2581161769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d31750 0x55adf8d2ba80 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3c480049d0 tx=0x7f3c4800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fb8730cee436fa server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c457fa640 1 -- 172.21.15.94:0/2581161769 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c480ccfa0 con 0x55adf8d31750 2024-02-17T12:04:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c457fa640 1 -- 172.21.15.94:0/2581161769 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c480cd100 con 0x55adf8d31750 2024-02-17T12:04:31.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.778+0000 7f3c457fa640 1 -- 172.21.15.94:0/2581161769 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c48010da0 con 0x55adf8d31750 2024-02-17T12:04:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.779+0000 7f3c585c9740 1 -- 172.21.15.94:0/2581161769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d31750 msgr2=0x55adf8d2ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.779+0000 7f3c585c9740 1 --2- 172.21.15.94:0/2581161769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d31750 0x55adf8d2ba80 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3c480049d0 tx=0x7f3c4800cee0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.779+0000 7f3c585c9740 1 -- 172.21.15.94:0/2581161769 shutdown_connections 2024-02-17T12:04:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.779+0000 7f3c585c9740 1 --2- 172.21.15.94:0/2581161769 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d31750 0x55adf8d2ba80 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.779+0000 7f3c585c9740 1 --2- 172.21.15.94:0/2581161769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d28b70 0x55adf8d2f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.779+0000 7f3c585c9740 1 --2- 172.21.15.94:0/2581161769 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d30f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.779+0000 7f3c585c9740 1 -- 172.21.15.94:0/2581161769 >> 172.21.15.94:0/2581161769 conn(0x55adf8d22980 msgr2=0x55adf8d2d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.779+0000 7f3c585c9740 1 -- 172.21.15.94:0/2581161769 shutdown_connections 2024-02-17T12:04:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.779+0000 7f3c585c9740 1 -- 172.21.15.94:0/2581161769 wait complete. 2024-02-17T12:04:31.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.779+0000 7f3c585c9740 1 -- start start 2024-02-17T12:04:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.779+0000 7f3c585c9740 1 Processor -- start 2024-02-17T12:04:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c585c9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d371b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c585c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d376f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c585c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 0x55adf8d37c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c585c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf8da09b0 con 0x55adf8d28b70 2024-02-17T12:04:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c585c9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf8da0af0 con 0x55adf8d28250 2024-02-17T12:04:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c585c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf8da0cb0 con 0x55adf8d31750 2024-02-17T12:04:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c585c9740 10 client.0 _ll_register_callbacks cb 0x7f3c400099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c47fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d376f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42620/0 (socket says 172.21.15.94:42620) 2024-02-17T12:04:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c47fff640 1 -- 172.21.15.94:0/546893278 learned_addr learned my addr 172.21.15.94:0/546893278 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c4effd640 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d371b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c4f7fe640 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 0x55adf8d37c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c4effd640 1 -- 172.21.15.94:0/546893278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 msgr2=0x55adf8d37c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c4effd640 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 0x55adf8d37c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c4effd640 1 -- 172.21.15.94:0/546893278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 msgr2=0x55adf8d376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c4effd640 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d376f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c4effd640 1 -- 172.21.15.94:0/546893278 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55adf8c52060 con 0x55adf8d28250 2024-02-17T12:04:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c47fff640 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c4effd640 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d371b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3c40003bf0 tx=0x7f3c40011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6591158a99f7c9f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c2effd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:04:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.780+0000 7f3c2effd640 1 -- 172.21.15.94:0/546893278 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c400038b0 con 0x55adf8d28250 2024-02-17T12:04:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c2effd640 1 -- 172.21.15.94:0/546893278 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3c40003a10 con 0x55adf8d28250 2024-02-17T12:04:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c585c9740 1 -- 172.21.15.94:0/546893278 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c40021070 con 0x55adf8d28250 2024-02-17T12:04:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c2effd640 1 -- 172.21.15.94:0/546893278 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c40033390 con 0x55adf8d28250 2024-02-17T12:04:31.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c585c9740 10 client.26639 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c585c9740 1 -- 172.21.15.94:0/546893278 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f3c501a4820 con 0x55adf8d28250 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c2effd640 1 -- 172.21.15.94:0/546893278 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3c40021070 con 0x55adf8d28250 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c2effd640 1 -- 172.21.15.94:0/546893278 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3c40026470 con 0x55adf8d28250 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c585c9740 10 client.26639 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c585c9740 10 client.26639 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c585c9740 20 client.26639 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c585c9740 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55adf8d41bb0 0x55adf8da44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c585c9740 4 client.26639 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c585c9740 1 -- 172.21.15.94:0/546893278 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55adf8a1b890 con 0x55adf8d41bb0 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.781+0000 7f3c47fff640 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55adf8d41bb0 0x55adf8da44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c47fff640 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55adf8d41bb0 0x55adf8da44d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=46e2b2ff7f21cbaa server_cookie=d64616a8cdde7422 in_seq=0 out_seq=0 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c2effd640 10 client.26639 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c2effd640 1 -- 172.21.15.94:0/546893278 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55adf8a1b890 con 0x55adf8d41bb0 2024-02-17T12:04:31.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c2effd640 10 client.26639 handle_command_reply: tid=0 2024-02-17T12:04:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c585c9740 1 client.26639 shutdown 2024-02-17T12:04:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c585c9740 1 -- 172.21.15.94:0/546893278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 msgr2=0x55adf8d371b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c585c9740 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d371b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3c40003bf0 tx=0x7f3c40011a50 comp rx=0 tx=0).stop 2024-02-17T12:04:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c585c9740 1 -- 172.21.15.94:0/546893278 shutdown_connections 2024-02-17T12:04:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c585c9740 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55adf8d41bb0 0x55adf8da44d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c585c9740 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf8d31750 0x55adf8d37c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c2effd640 0 client.26639 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:04:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c585c9740 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf8d28b70 0x55adf8d376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c585c9740 1 --2- 172.21.15.94:0/546893278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf8d28250 0x55adf8d371b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c585c9740 1 -- 172.21.15.94:0/546893278 >> 172.21.15.94:0/546893278 conn(0x55adf8d22980 msgr2=0x55adf8d23db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c585c9740 1 -- 172.21.15.94:0/546893278 shutdown_connections 2024-02-17T12:04:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c585c9740 1 -- 172.21.15.94:0/546893278 wait complete. 2024-02-17T12:04:31.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.782+0000 7f3c585c9740 20 client.26639 trim_cache size 0 max 16384 2024-02-17T12:04:31.791 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:04:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.783+0000 7f3c559f4640 1 -- 172.21.15.94:0/4106294635 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3c3c0a2230 msgr2=0x7f3c3c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.783+0000 7f3c559f4640 1 --2- 172.21.15.94:0/4106294635 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3c3c0a2230 0x7f3c3c0a46f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3c40009bd0 tx=0x7f3c40005f90 comp rx=0 tx=0).stop 2024-02-17T12:04:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.783+0000 7f3c559f4640 1 -- 172.21.15.94:0/4106294635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c50197630 msgr2=0x7f3c50197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.783+0000 7f3c559f4640 1 --2- 172.21.15.94:0/4106294635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c50197630 0x7f3c50197ae0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800eb10 tx=0x7f3c4800c2d0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.783+0000 7f3c559f4640 1 -- 172.21.15.94:0/4106294635 shutdown_connections 2024-02-17T12:04:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.783+0000 7f3c559f4640 1 --2- 172.21.15.94:0/4106294635 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3c3c0a2230 0x7f3c3c0a46f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.783+0000 7f3c559f4640 1 --2- 172.21.15.94:0/4106294635 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3c50197630 0x7f3c50197ae0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.783+0000 7f3c559f4640 1 --2- 172.21.15.94:0/4106294635 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3c50105330 0x7f3c5019d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.783+0000 7f3c559f4640 1 --2- 172.21.15.94:0/4106294635 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c50100260 0x7f3c5019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.783+0000 7f3c559f4640 1 -- 172.21.15.94:0/4106294635 >> 172.21.15.94:0/4106294635 conn(0x7f3c50077180 msgr2=0x7f3c500fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.783+0000 7f3c559f4640 1 -- 172.21.15.94:0/4106294635 shutdown_connections 2024-02-17T12:04:31.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.783+0000 7f3c559f4640 1 -- 172.21.15.94:0/4106294635 wait complete. 2024-02-17T12:04:31.795 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:04:31.796 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:04:31.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.941+0000 7f86f541c640 1 -- 172.21.15.94:0/3553556217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f0101380 msgr2=0x7f86f0101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.941+0000 7f86f541c640 1 --2- 172.21.15.94:0/3553556217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f0101380 0x7f86f0101760 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009930 tx=0x7f86e002bc30 comp rx=0 tx=0).stop 2024-02-17T12:04:31.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.941+0000 7f86f541c640 1 -- 172.21.15.94:0/3553556217 shutdown_connections 2024-02-17T12:04:31.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.941+0000 7f86f541c640 1 --2- 172.21.15.94:0/3553556217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86f0106450 0x7f86f010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.941+0000 7f86f541c640 1 --2- 172.21.15.94:0/3553556217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f0101d30 0x7f86f0105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.941+0000 7f86f541c640 1 --2- 172.21.15.94:0/3553556217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f0101380 0x7f86f0101760 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.941+0000 7f86f541c640 1 -- 172.21.15.94:0/3553556217 >> 172.21.15.94:0/3553556217 conn(0x7f86f00fcc20 msgr2=0x7f86f00ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.941+0000 7f86f541c640 1 -- 172.21.15.94:0/3553556217 shutdown_connections 2024-02-17T12:04:31.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.941+0000 7f86f541c640 1 -- 172.21.15.94:0/3553556217 wait complete. 2024-02-17T12:04:31.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.941+0000 7f86f541c640 1 Processor -- start 2024-02-17T12:04:31.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86f541c640 1 -- start start 2024-02-17T12:04:31.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86f541c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f0101380 0x7f86f019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86f541c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f0106450 0x7f86f019db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86f541c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86f01976f0 0x7f86f0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86f541c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01981c0 con 0x7f86f0101380 2024-02-17T12:04:31.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86f541c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f0198330 con 0x7f86f0106450 2024-02-17T12:04:31.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86f541c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f01984f0 con 0x7f86f01976f0 2024-02-17T12:04:31.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86eeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f0101380 0x7f86f019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86eeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f0101380 0x7f86f019d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42640/0 (socket says 172.21.15.94:42640) 2024-02-17T12:04:31.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86eeffd640 1 -- 172.21.15.94:0/2662033270 learned_addr learned my addr 172.21.15.94:0/2662033270 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86ef7fe640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86f01976f0 0x7f86f0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86e7fff640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f0106450 0x7f86f019db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86eeffd640 1 -- 172.21.15.94:0/2662033270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86f01976f0 msgr2=0x7f86f0197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86eeffd640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86f01976f0 0x7f86f0197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86eeffd640 1 -- 172.21.15.94:0/2662033270 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f0106450 msgr2=0x7f86f019db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86eeffd640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f0106450 0x7f86f019db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86eeffd640 1 -- 172.21.15.94:0/2662033270 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0009560 con 0x7f86f0101380 2024-02-17T12:04:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86e7fff640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f0106450 0x7f86f019db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:04:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86eeffd640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f0101380 0x7f86f019d5e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f86e0000c00 tx=0x7f86e002c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86ecff9640 1 -- 172.21.15.94:0/2662033270 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00042d0 con 0x7f86f0101380 2024-02-17T12:04:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86ecff9640 1 -- 172.21.15.94:0/2662033270 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86e0004430 con 0x7f86f0101380 2024-02-17T12:04:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86ecff9640 1 -- 172.21.15.94:0/2662033270 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0003c50 con 0x7f86f0101380 2024-02-17T12:04:31.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86f541c640 1 -- 172.21.15.94:0/2662033270 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f01a43e0 con 0x7f86f0101380 2024-02-17T12:04:31.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.942+0000 7f86f541c640 1 -- 172.21.15.94:0/2662033270 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f01a48c0 con 0x7f86f0101380 2024-02-17T12:04:31.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.946+0000 7f86ecff9640 1 -- 172.21.15.94:0/2662033270 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f86e0004df0 con 0x7f86f0101380 2024-02-17T12:04:31.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.946+0000 7f86f7ff2740 1 Processor -- start 2024-02-17T12:04:31.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.946+0000 7f86f7ff2740 1 -- start start 2024-02-17T12:04:31.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86f7ff2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e58f0 0x5634a52e5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86f7ff2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e62a0 0x5634a52d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86f7ff2740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86f7ff2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a5293960 con 0x5634a52e62a0 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86f7ff2740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a5247310 con 0x5634a52d2a50 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86f7ff2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a5244840 con 0x5634a52e58f0 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86eeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e62a0 0x5634a52d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86eeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e62a0 0x5634a52d2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42642/0 (socket says 172.21.15.94:42642) 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86eeffd640 1 -- 172.21.15.94:0/90619909 learned_addr learned my addr 172.21.15.94:0/90619909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86ef7fe640 1 --2- 172.21.15.94:0/90619909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e58f0 0x5634a52e5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86e7fff640 1 --2- 172.21.15.94:0/90619909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86ecff9640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f86dc0a22b0 0x7f86dc0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86eeffd640 1 -- 172.21.15.94:0/90619909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e58f0 msgr2=0x5634a52e5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86eeffd640 1 --2- 172.21.15.94:0/90619909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e58f0 0x5634a52e5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86eeffd640 1 -- 172.21.15.94:0/90619909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 msgr2=0x5634a52d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86eeffd640 1 --2- 172.21.15.94:0/90619909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52d2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86eeffd640 1 -- 172.21.15.94:0/90619909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634a5202060 con 0x5634a52e62a0 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86e7fff640 1 --2- 172.21.15.94:0/90619909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52d2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:31.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86ef7fe640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f86dc0a22b0 0x7f86dc0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.947+0000 7f86ecff9640 1 -- 172.21.15.94:0/2662033270 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f86e00f07c0 con 0x7f86f0101380 2024-02-17T12:04:31.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.948+0000 7f86eeffd640 1 --2- 172.21.15.94:0/90619909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e62a0 0x5634a52d2430 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f86e00edef0 tx=0x7f86e004a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf12468a359626a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.948+0000 7f86e57fa640 1 -- 172.21.15.94:0/90619909 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e003ce80 con 0x5634a52e62a0 2024-02-17T12:04:31.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.948+0000 7f86ef7fe640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f86dc0a22b0 0x7f86dc0a4770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x5634a52d3570 tx=0x7f86e8014040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.948+0000 7f86e57fa640 1 -- 172.21.15.94:0/90619909 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86e00bd020 con 0x5634a52e62a0 2024-02-17T12:04:31.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.948+0000 7f86e57fa640 1 -- 172.21.15.94:0/90619909 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e003d740 con 0x5634a52e62a0 2024-02-17T12:04:31.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.948+0000 7f86f7ff2740 1 -- 172.21.15.94:0/90619909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e62a0 msgr2=0x5634a52d2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.948+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/90619909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e62a0 0x5634a52d2430 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f86e00edef0 tx=0x7f86e004a000 comp rx=0 tx=0).stop 2024-02-17T12:04:31.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 -- 172.21.15.94:0/90619909 shutdown_connections 2024-02-17T12:04:31.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/90619909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52d2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/90619909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e62a0 0x5634a52d2430 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/90619909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e58f0 0x5634a52e5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 -- 172.21.15.94:0/90619909 >> 172.21.15.94:0/90619909 conn(0x5634a52d4ef0 msgr2=0x5634a52d52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 -- 172.21.15.94:0/90619909 shutdown_connections 2024-02-17T12:04:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 -- 172.21.15.94:0/90619909 wait complete. 2024-02-17T12:04:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 -- start start 2024-02-17T12:04:31.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 Processor -- start 2024-02-17T12:04:31.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52d2a50 0x5634a52e97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 0x5634a52e9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52e62a0 0x5634a52ea220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a5240020 con 0x5634a52e58f0 2024-02-17T12:04:31.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a4f13430 con 0x5634a52e62a0 2024-02-17T12:04:31.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a4ecfe00 con 0x5634a52d2a50 2024-02-17T12:04:31.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86f7ff2740 10 client.0 _ll_register_callbacks cb 0x5634a51e7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:31.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86e7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 0x5634a52e9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86e7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 0x5634a52e9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42646/0 (socket says 172.21.15.94:42646) 2024-02-17T12:04:31.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86e7fff640 1 -- 172.21.15.94:0/244235135 learned_addr learned my addr 172.21.15.94:0/244235135 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86eeffd640 1 --2- 172.21.15.94:0/244235135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52d2a50 0x5634a52e97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.949+0000 7f86ef7fe640 1 --2- 172.21.15.94:0/244235135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52e62a0 0x5634a52ea220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86e7fff640 1 -- 172.21.15.94:0/244235135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52d2a50 msgr2=0x5634a52e97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86e7fff640 1 --2- 172.21.15.94:0/244235135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52d2a50 0x5634a52e97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86e7fff640 1 -- 172.21.15.94:0/244235135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52e62a0 msgr2=0x5634a52ea220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86e7fff640 1 --2- 172.21.15.94:0/244235135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52e62a0 0x5634a52ea220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86e7fff640 1 -- 172.21.15.94:0/244235135 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f01a48c0 con 0x5634a52e58f0 2024-02-17T12:04:31.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86e7fff640 1 --2- 172.21.15.94:0/244235135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 0x5634a52e9ce0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f86d80137b0 tx=0x7f86d8013c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe375e17d15dd6a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:04:31.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86d67fc640 1 -- 172.21.15.94:0/244235135 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8011dd0 con 0x5634a52e58f0 2024-02-17T12:04:31.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86d67fc640 1 -- 172.21.15.94:0/244235135 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f86d8004030 con 0x5634a52e58f0 2024-02-17T12:04:31.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86f7ff2740 1 -- 172.21.15.94:0/244235135 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634a4f3f7e0 con 0x5634a52e58f0 2024-02-17T12:04:31.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86d67fc640 1 -- 172.21.15.94:0/244235135 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86d80041f0 con 0x5634a52e58f0 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86f7ff2740 10 client.17187 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86f7ff2740 1 -- 172.21.15.94:0/244235135 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5634a523e6a0 con 0x5634a52e58f0 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86d67fc640 1 -- 172.21.15.94:0/244235135 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86d8027080 con 0x5634a52e58f0 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.950+0000 7f86d67fc640 1 -- 172.21.15.94:0/244235135 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f86d8023550 con 0x5634a52e58f0 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.951+0000 7f86f7ff2740 10 client.17187 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.951+0000 7f86f7ff2740 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.951+0000 7f86f7ff2740 20 client.17187 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.951+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/244235135 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5634a52fa1c0 0x5634a52fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.951+0000 7f86f7ff2740 4 client.17187 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.951+0000 7f86f7ff2740 1 -- 172.21.15.94:0/244235135 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634a4f7c720 con 0x5634a52fa1c0 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.951+0000 7f86eeffd640 1 --2- 172.21.15.94:0/244235135 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5634a52fa1c0 0x5634a52fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.951+0000 7f86d67fc640 10 client.17187 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.951+0000 7f86eeffd640 1 --2- 172.21.15.94:0/244235135 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5634a52fa1c0 0x5634a52fc5e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=654e76a4ef666adb server_cookie=b30e82142ec49ab in_seq=0 out_seq=0 2024-02-17T12:04:31.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86d67fc640 1 -- 172.21.15.94:0/244235135 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634a4f7c720 con 0x5634a52fa1c0 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86d67fc640 10 client.17187 handle_command_reply: tid=0 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86f7ff2740 1 client.17187 shutdown 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86f7ff2740 1 -- 172.21.15.94:0/244235135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 msgr2=0x5634a52e9ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/244235135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 0x5634a52e9ce0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f86d80137b0 tx=0x7f86d8013c70 comp rx=0 tx=0).stop 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86f7ff2740 1 -- 172.21.15.94:0/244235135 shutdown_connections 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/244235135 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5634a52fa1c0 0x5634a52fc5e0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/244235135 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52e62a0 0x5634a52ea220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/244235135 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 0x5634a52e9ce0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86d67fc640 0 client.17187 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/244235135 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52d2a50 0x5634a52e97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86f7ff2740 1 -- 172.21.15.94:0/244235135 >> 172.21.15.94:0/244235135 conn(0x5634a52d4ef0 msgr2=0x5634a52d3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86f7ff2740 1 -- 172.21.15.94:0/244235135 shutdown_connections 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86f7ff2740 1 -- 172.21.15.94:0/244235135 wait complete. 2024-02-17T12:04:31.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.952+0000 7f86f7ff2740 20 client.17187 trim_cache size 0 max 16384 2024-02-17T12:04:31.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.966+0000 7f86f7ff2740 1 Processor -- start 2024-02-17T12:04:31.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.967+0000 7f86f7ff2740 1 -- start start 2024-02-17T12:04:31.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.967+0000 7f86f7ff2740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52ea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.967+0000 7f86f7ff2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e58f0 0x5634a52f0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.967+0000 7f86f7ff2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e62a0 0x5634a52f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.967+0000 7f86f7ff2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a51c5130 con 0x5634a52e62a0 2024-02-17T12:04:31.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.967+0000 7f86f7ff2740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a52ce0d0 con 0x5634a52d2a50 2024-02-17T12:04:31.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.967+0000 7f86f7ff2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a52ce5c0 con 0x5634a52e58f0 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.967+0000 7f86eeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52ea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.967+0000 7f86e7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e58f0 0x5634a52f0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.967+0000 7f86e7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e58f0 0x5634a52f0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56174/0 (socket says 172.21.15.94:56174) 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.967+0000 7f86eeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52ea060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42152/0 (socket says 172.21.15.94:42152) 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.967+0000 7f86e7fff640 1 -- 172.21.15.94:0/776984909 learned_addr learned my addr 172.21.15.94:0/776984909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.967+0000 7f86ef7fe640 1 --2- 172.21.15.94:0/776984909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e62a0 0x5634a52f10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.968+0000 7f86eeffd640 1 -- 172.21.15.94:0/776984909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e58f0 msgr2=0x5634a52f0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.968+0000 7f86eeffd640 1 --2- 172.21.15.94:0/776984909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e58f0 0x5634a52f0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.968+0000 7f86eeffd640 1 -- 172.21.15.94:0/776984909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e62a0 msgr2=0x5634a52f10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.968+0000 7f86eeffd640 1 --2- 172.21.15.94:0/776984909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e62a0 0x5634a52f10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.968+0000 7f86eeffd640 1 -- 172.21.15.94:0/776984909 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634a51dac20 con 0x5634a52d2a50 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.968+0000 7f86e7fff640 1 --2- 172.21.15.94:0/776984909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e58f0 0x5634a52f0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.968+0000 7f86ef7fe640 1 --2- 172.21.15.94:0/776984909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e62a0 0x5634a52f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:04:31.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.968+0000 7f86eeffd640 1 --2- 172.21.15.94:0/776984909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52ea060 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f86e00b9c20 tx=0x7f86e00f6530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e672ec9eec51817b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.968+0000 7f86e4ff9640 1 -- 172.21.15.94:0/776984909 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00bcbb0 con 0x5634a52d2a50 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.968+0000 7f86e4ff9640 1 -- 172.21.15.94:0/776984909 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86e003ce30 con 0x5634a52d2a50 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.968+0000 7f86e4ff9640 1 -- 172.21.15.94:0/776984909 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00bb2f0 con 0x5634a52d2a50 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.969+0000 7f86f7ff2740 1 -- 172.21.15.94:0/776984909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 msgr2=0x5634a52ea060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.969+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/776984909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52ea060 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f86e00b9c20 tx=0x7f86e00f6530 comp rx=0 tx=0).stop 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.969+0000 7f86f7ff2740 1 -- 172.21.15.94:0/776984909 shutdown_connections 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.969+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/776984909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e62a0 0x5634a52f10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.969+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/776984909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e58f0 0x5634a52f0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.969+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/776984909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52ea060 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.969+0000 7f86f7ff2740 1 -- 172.21.15.94:0/776984909 >> 172.21.15.94:0/776984909 conn(0x5634a52d4ef0 msgr2=0x5634a5300f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.969+0000 7f86f7ff2740 1 -- 172.21.15.94:0/776984909 shutdown_connections 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.969+0000 7f86f7ff2740 1 -- 172.21.15.94:0/776984909 wait complete. 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.969+0000 7f86f7ff2740 1 -- start start 2024-02-17T12:04:31.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.969+0000 7f86f7ff2740 1 Processor -- start 2024-02-17T12:04:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86f7ff2740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52ef4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86f7ff2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 0x5634a52efa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86f7ff2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e62a0 0x5634a52eff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86f7ff2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a52f1460 con 0x5634a52e58f0 2024-02-17T12:04:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86f7ff2740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a53506d0 con 0x5634a52d2a50 2024-02-17T12:04:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86f7ff2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634a5350840 con 0x5634a52e62a0 2024-02-17T12:04:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86f7ff2740 10 client.0 _ll_register_callbacks cb 0x7f86e00f6400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86e7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 0x5634a52efa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86e7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 0x5634a52efa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42664/0 (socket says 172.21.15.94:42664) 2024-02-17T12:04:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86eeffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52ef4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86e7fff640 1 -- 172.21.15.94:0/2135485511 learned_addr learned my addr 172.21.15.94:0/2135485511 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86ef7fe640 1 --2- 172.21.15.94:0/2135485511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e62a0 0x5634a52eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86e7fff640 1 -- 172.21.15.94:0/2135485511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e62a0 msgr2=0x5634a52eff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86e7fff640 1 --2- 172.21.15.94:0/2135485511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e62a0 0x5634a52eff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86e7fff640 1 -- 172.21.15.94:0/2135485511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 msgr2=0x5634a52ef4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86e7fff640 1 --2- 172.21.15.94:0/2135485511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52ef4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86e7fff640 1 -- 172.21.15.94:0/2135485511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634a523e6a0 con 0x5634a52e58f0 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86e7fff640 1 --2- 172.21.15.94:0/2135485511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 0x5634a52efa00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f86d8011810 tx=0x7f86d8011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17a2eb6133e9a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86d67fc640 1 -- 172.21.15.94:0/2135485511 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8009720 con 0x5634a52e58f0 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86d67fc640 1 -- 172.21.15.94:0/2135485511 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f86d802a030 con 0x5634a52e58f0 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.970+0000 7f86f7ff2740 1 -- 172.21.15.94:0/2135485511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86e00bd020 con 0x5634a52e58f0 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.971+0000 7f86d67fc640 1 -- 172.21.15.94:0/2135485511 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86d80050b0 con 0x5634a52e58f0 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.971+0000 7f86f7ff2740 10 client.17193 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.971+0000 7f86f7ff2740 1 -- 172.21.15.94:0/2135485511 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5634a5202060 con 0x5634a52e58f0 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.971+0000 7f86d67fc640 1 -- 172.21.15.94:0/2135485511 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86e00bd020 con 0x5634a52e58f0 2024-02-17T12:04:31.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.971+0000 7f86d67fc640 1 -- 172.21.15.94:0/2135485511 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f86d8003bc0 con 0x5634a52e58f0 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.971+0000 7f86f7ff2740 10 client.17193 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.971+0000 7f86f7ff2740 10 client.17193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.971+0000 7f86f7ff2740 20 client.17193 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.971+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/2135485511 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5634a5352070 0x5634a5354490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.971+0000 7f86f7ff2740 4 client.17193 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.971+0000 7f86f7ff2740 1 -- 172.21.15.94:0/2135485511 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5634a4fcb890 con 0x5634a5352070 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.971+0000 7f86eeffd640 1 --2- 172.21.15.94:0/2135485511 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5634a5352070 0x5634a5354490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86d67fc640 10 client.17193 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86eeffd640 1 --2- 172.21.15.94:0/2135485511 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5634a5352070 0x5634a5354490 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48be27465542ba0 server_cookie=20d2838636283bb3 in_seq=0 out_seq=0 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86d67fc640 1 -- 172.21.15.94:0/2135485511 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5634a4fcb890 con 0x5634a5352070 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86d67fc640 10 client.17193 handle_command_reply: tid=0 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f7ff2740 1 client.17193 shutdown 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f7ff2740 1 -- 172.21.15.94:0/2135485511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 msgr2=0x5634a52efa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/2135485511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 0x5634a52efa00 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f86d8011810 tx=0x7f86d8011e80 comp rx=0 tx=0).stop 2024-02-17T12:04:31.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f7ff2740 1 -- 172.21.15.94:0/2135485511 shutdown_connections 2024-02-17T12:04:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/2135485511 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5634a5352070 0x5634a5354490 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/2135485511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5634a52e62a0 0x5634a52eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86d67fc640 0 client.17193 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/2135485511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5634a52e58f0 0x5634a52efa00 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f7ff2740 1 --2- 172.21.15.94:0/2135485511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5634a52d2a50 0x5634a52ef4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f7ff2740 1 -- 172.21.15.94:0/2135485511 >> 172.21.15.94:0/2135485511 conn(0x5634a52d4ef0 msgr2=0x5634a52ea0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f7ff2740 1 -- 172.21.15.94:0/2135485511 shutdown_connections 2024-02-17T12:04:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f7ff2740 1 -- 172.21.15.94:0/2135485511 wait complete. 2024-02-17T12:04:31.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f7ff2740 20 client.17193 trim_cache size 0 max 16384 2024-02-17T12:04:31.979 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:04:31.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f541c640 1 -- 172.21.15.94:0/2662033270 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f86dc0a22b0 msgr2=0x7f86dc0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f541c640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f86dc0a22b0 0x7f86dc0a4770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x5634a52d3570 tx=0x7f86e8014040 comp rx=0 tx=0).stop 2024-02-17T12:04:31.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f541c640 1 -- 172.21.15.94:0/2662033270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f0101380 msgr2=0x7f86f019d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:31.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.972+0000 7f86f541c640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f0101380 0x7f86f019d5e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f86e0000c00 tx=0x7f86e002c5f0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.973+0000 7f86f541c640 1 -- 172.21.15.94:0/2662033270 shutdown_connections 2024-02-17T12:04:31.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.973+0000 7f86f541c640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f86dc0a22b0 0x7f86dc0a4770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.973+0000 7f86f541c640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86f01976f0 0x7f86f0197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.973+0000 7f86f541c640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f0106450 0x7f86f019db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.973+0000 7f86f541c640 1 --2- 172.21.15.94:0/2662033270 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f0101380 0x7f86f019d5e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:31.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.973+0000 7f86f541c640 1 -- 172.21.15.94:0/2662033270 >> 172.21.15.94:0/2662033270 conn(0x7f86f00fcc20 msgr2=0x7f86f00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:31.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.973+0000 7f86f541c640 1 -- 172.21.15.94:0/2662033270 shutdown_connections 2024-02-17T12:04:31.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:31.973+0000 7f86f541c640 1 -- 172.21.15.94:0/2662033270 wait complete. 2024-02-17T12:04:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:31 smithi094 ceph-mon[32499]: pgmap v196: 97 pgs: 97 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 675 op/s 2024-02-17T12:04:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:31 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2070355410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:04:32.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:31 smithi189 ceph-mon[36588]: pgmap v196: 97 pgs: 97 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 675 op/s 2024-02-17T12:04:32.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:31 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2070355410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:04:32.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:31 smithi125 ceph-mon[36819]: pgmap v196: 97 pgs: 97 active+clean; 4.6 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 15 MiB/s wr, 675 op/s 2024-02-17T12:04:32.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:31 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2070355410' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:04:32.986 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:04:32.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:04:32.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:04:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.135+0000 7f39d1f77640 1 -- 172.21.15.94:0/3113472921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39cc0688c0 msgr2=0x7f39cc068d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.135+0000 7f39d1f77640 1 --2- 172.21.15.94:0/3113472921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39cc0688c0 0x7f39cc068d20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009860 tx=0x7f39b402bb70 comp rx=0 tx=0).stop 2024-02-17T12:04:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.136+0000 7f39d1f77640 1 -- 172.21.15.94:0/3113472921 shutdown_connections 2024-02-17T12:04:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.136+0000 7f39d1f77640 1 --2- 172.21.15.94:0/3113472921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc103c40 0x7f39cc1040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.136+0000 7f39d1f77640 1 --2- 172.21.15.94:0/3113472921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39cc0688c0 0x7f39cc068d20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.136+0000 7f39d1f77640 1 --2- 172.21.15.94:0/3113472921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39cc1060d0 0x7f39cc068380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.136+0000 7f39d1f77640 1 -- 172.21.15.94:0/3113472921 >> 172.21.15.94:0/3113472921 conn(0x7f39cc077180 msgr2=0x7f39cc077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.136+0000 7f39d1f77640 1 -- 172.21.15.94:0/3113472921 shutdown_connections 2024-02-17T12:04:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.136+0000 7f39d1f77640 1 -- 172.21.15.94:0/3113472921 wait complete. 2024-02-17T12:04:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.136+0000 7f39d1f77640 1 Processor -- start 2024-02-17T12:04:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.136+0000 7f39d1f77640 1 -- start start 2024-02-17T12:04:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.136+0000 7f39d1f77640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39cc0688c0 0x7f39cc1a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.136+0000 7f39d1f77640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc103c40 0x7f39cc1a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39d1f77640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39cc1060d0 0x7f39cc19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39d1f77640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc19e130 con 0x7f39cc103c40 2024-02-17T12:04:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39d1f77640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc19e2a0 con 0x7f39cc1060d0 2024-02-17T12:04:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39d1f77640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc19e460 con 0x7f39cc0688c0 2024-02-17T12:04:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc103c40 0x7f39cc1a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc103c40 0x7f39cc1a3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42680/0 (socket says 172.21.15.94:42680) 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39c3fff640 1 -- 172.21.15.94:0/609104539 learned_addr learned my addr 172.21.15.94:0/609104539 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39cbfff640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39cc1060d0 0x7f39cc19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39cb7fe640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39cc0688c0 0x7f39cc1a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39c3fff640 1 -- 172.21.15.94:0/609104539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39cc0688c0 msgr2=0x7f39cc1a3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39c3fff640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39cc0688c0 0x7f39cc1a3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39c3fff640 1 -- 172.21.15.94:0/609104539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39cc1060d0 msgr2=0x7f39cc19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39c3fff640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39cc1060d0 0x7f39cc19dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39c3fff640 1 -- 172.21.15.94:0/609104539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c4009840 con 0x7f39cc103c40 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39cbfff640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39cc1060d0 0x7f39cc19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39c3fff640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc103c40 0x7f39cc1a3e70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f39b4037040 tx=0x7f39b40045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39c97fa640 1 -- 172.21.15.94:0/609104539 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b40349d0 con 0x7f39cc103c40 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39d1f77640 1 -- 172.21.15.94:0/609104539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc009560 con 0x7f39cc103c40 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.137+0000 7f39c97fa640 1 -- 172.21.15.94:0/609104539 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39b402c9f0 con 0x7f39cc103c40 2024-02-17T12:04:33.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.138+0000 7f39d1f77640 1 -- 172.21.15.94:0/609104539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b4009490 con 0x7f39cc103c40 2024-02-17T12:04:33.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.138+0000 7f39c97fa640 1 -- 172.21.15.94:0/609104539 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4003e70 con 0x7f39cc103c40 2024-02-17T12:04:33.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.141+0000 7f39d4c29740 1 Processor -- start 2024-02-17T12:04:33.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39d4c29740 1 -- start start 2024-02-17T12:04:33.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39c97fa640 1 -- 172.21.15.94:0/609104539 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f39b403b040 con 0x7f39cc103c40 2024-02-17T12:04:33.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39d4c29740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4a2380 0x55810c4a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39d4c29740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2d30 0x55810c4ab2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39d4c29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4ab8b0 0x55810c4add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39d4c29740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c410f90 con 0x55810c4a2d30 2024-02-17T12:04:33.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39d4c29740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c40e4c0 con 0x55810c4a2380 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39d4c29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c409ca0 con 0x55810c4ab8b0 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39cbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4a2380 0x55810c4a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2d30 0x55810c4ab2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39cbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4a2380 0x55810c4a2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42182/0 (socket says 172.21.15.94:42182) 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39cbfff640 1 -- 172.21.15.94:0/541886736 learned_addr learned my addr 172.21.15.94:0/541886736 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39c3fff640 1 --2- 172.21.15.94:0/541886736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4ab8b0 0x55810c4add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39c97fa640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f39b80a2230 0x7f39b80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39cbfff640 1 -- 172.21.15.94:0/541886736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4ab8b0 msgr2=0x55810c4add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39cbfff640 1 --2- 172.21.15.94:0/541886736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4ab8b0 0x55810c4add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39cbfff640 1 -- 172.21.15.94:0/541886736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2d30 msgr2=0x55810c4ab2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39cbfff640 1 --2- 172.21.15.94:0/541886736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2d30 0x55810c4ab2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39cbfff640 1 -- 172.21.15.94:0/541886736 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55810c3cc0b0 con 0x55810c4a2380 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.142+0000 7f39c3fff640 1 --2- 172.21.15.94:0/541886736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4ab8b0 0x55810c4add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.143+0000 7f39c97fa640 1 -- 172.21.15.94:0/609104539 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f39b40efad0 con 0x7f39cc103c40 2024-02-17T12:04:33.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.143+0000 7f39cbfff640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f39b80a2230 0x7f39b80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.143+0000 7f39cbfff640 1 --2- 172.21.15.94:0/541886736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4a2380 0x55810c4a2760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f39c400e910 tx=0x7f39c400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=527da8fa25cc217a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:33.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.143+0000 7f39c1ffb640 1 -- 172.21.15.94:0/541886736 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4020030 con 0x55810c4a2380 2024-02-17T12:04:33.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.143+0000 7f39c1ffb640 1 -- 172.21.15.94:0/541886736 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39c4011e00 con 0x55810c4a2380 2024-02-17T12:04:33.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.144+0000 7f39c1ffb640 1 -- 172.21.15.94:0/541886736 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4019c40 con 0x55810c4a2380 2024-02-17T12:04:33.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.144+0000 7f39cbfff640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f39b80a2230 0x7f39b80a46f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f39c4020000 tx=0x7f39c400f790 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:33.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.145+0000 7f39d4c29740 1 -- 172.21.15.94:0/541886736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4a2380 msgr2=0x55810c4a2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.145+0000 7f39d4c29740 1 --2- 172.21.15.94:0/541886736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4a2380 0x55810c4a2760 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f39c400e910 tx=0x7f39c400ed10 comp rx=0 tx=0).stop 2024-02-17T12:04:33.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.145+0000 7f39d4c29740 1 -- 172.21.15.94:0/541886736 shutdown_connections 2024-02-17T12:04:33.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.145+0000 7f39d4c29740 1 --2- 172.21.15.94:0/541886736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4ab8b0 0x55810c4add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.145+0000 7f39d4c29740 1 --2- 172.21.15.94:0/541886736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2d30 0x55810c4ab2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.145+0000 7f39d4c29740 1 --2- 172.21.15.94:0/541886736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4a2380 0x55810c4a2760 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.145+0000 7f39d4c29740 1 -- 172.21.15.94:0/541886736 >> 172.21.15.94:0/541886736 conn(0x55810c49c840 msgr2=0x55810c49cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:33.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.145+0000 7f39d4c29740 1 -- 172.21.15.94:0/541886736 shutdown_connections 2024-02-17T12:04:33.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.145+0000 7f39d4c29740 1 -- 172.21.15.94:0/541886736 wait complete. 2024-02-17T12:04:33.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.145+0000 7f39d4c29740 1 -- start start 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.146+0000 7f39d4c29740 1 Processor -- start 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.146+0000 7f39d4c29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4a2380 0x55810c4a87b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.146+0000 7f39d4c29740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2d30 0x55810c4a8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.146+0000 7f39d4c29740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4ab8b0 0x55810c4a9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.146+0000 7f39d4c29740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c3aef40 con 0x55810c4a2d30 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.146+0000 7f39d4c29740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c0dd4b0 con 0x55810c4ab8b0 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.146+0000 7f39d4c29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c38f670 con 0x55810c4a2380 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.146+0000 7f39c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2d30 0x55810c4a8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.146+0000 7f39d4c29740 10 client.0 _ll_register_callbacks cb 0x55810c3fcf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.146+0000 7f39cbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4ab8b0 0x55810c4a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.146+0000 7f39c3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2d30 0x55810c4a8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42702/0 (socket says 172.21.15.94:42702) 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.146+0000 7f39c3fff640 1 -- 172.21.15.94:0/2560749033 learned_addr learned my addr 172.21.15.94:0/2560749033 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.146+0000 7f39cb7fe640 1 --2- 172.21.15.94:0/2560749033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4a2380 0x55810c4a87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39cbfff640 1 -- 172.21.15.94:0/2560749033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4a2380 msgr2=0x55810c4a87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39cbfff640 1 --2- 172.21.15.94:0/2560749033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4a2380 0x55810c4a87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39cbfff640 1 -- 172.21.15.94:0/2560749033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2d30 msgr2=0x55810c4a8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39cbfff640 1 --2- 172.21.15.94:0/2560749033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2d30 0x55810c4a8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39cbfff640 1 -- 172.21.15.94:0/2560749033 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c400e5c0 con 0x55810c4ab8b0 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39cbfff640 1 --2- 172.21.15.94:0/2560749033 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4ab8b0 0x55810c4a9230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f39c400e8e0 tx=0x7f39c4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc0a84b35cf5982c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39aaffd640 1 -- 172.21.15.94:0/2560749033 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4019d50 con 0x55810c4ab8b0 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39d4c29740 1 -- 172.21.15.94:0/2560749033 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55810c3e3f40 con 0x55810c4ab8b0 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39aaffd640 1 -- 172.21.15.94:0/2560749033 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f39c4021070 con 0x55810c4ab8b0 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39aaffd640 1 -- 172.21.15.94:0/2560749033 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39c4019560 con 0x55810c4ab8b0 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39d4c29740 10 client.26669 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39d4c29740 1 -- 172.21.15.94:0/2560749033 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55810c408320 con 0x55810c4ab8b0 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.147+0000 7f39aaffd640 1 -- 172.21.15.94:0/2560749033 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55810c3e3f40 con 0x55810c4ab8b0 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.148+0000 7f39aaffd640 1 -- 172.21.15.94:0/2560749033 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39c4030430 con 0x55810c4ab8b0 2024-02-17T12:04:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.148+0000 7f39d4c29740 10 client.26669 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:33.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.148+0000 7f39d4c29740 10 client.26669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:33.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.148+0000 7f39d4c29740 20 client.26669 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:33.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.148+0000 7f39d4c29740 1 --2- 172.21.15.94:0/2560749033 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55810c4bbdd0 0x55810c4be1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.148+0000 7f39d4c29740 4 client.26669 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:04:33.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.148+0000 7f39d4c29740 1 -- 172.21.15.94:0/2560749033 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55810c400310 con 0x55810c4bbdd0 2024-02-17T12:04:33.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.148+0000 7f39cb7fe640 1 --2- 172.21.15.94:0/2560749033 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55810c4bbdd0 0x55810c4be1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.149+0000 7f39aaffd640 10 client.26669 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:33.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.149+0000 7f39cb7fe640 1 --2- 172.21.15.94:0/2560749033 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55810c4bbdd0 0x55810c4be1f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb9505ba5b05db32 server_cookie=813fe10e0ca3882b in_seq=0 out_seq=0 2024-02-17T12:04:33.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39aaffd640 1 -- 172.21.15.94:0/2560749033 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55810c400310 con 0x55810c4bbdd0 2024-02-17T12:04:33.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39aaffd640 10 client.26669 handle_command_reply: tid=0 2024-02-17T12:04:33.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39d4c29740 1 client.26669 shutdown 2024-02-17T12:04:33.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39d4c29740 1 -- 172.21.15.94:0/2560749033 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4ab8b0 msgr2=0x55810c4a9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39d4c29740 1 --2- 172.21.15.94:0/2560749033 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4ab8b0 0x55810c4a9230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f39c400e8e0 tx=0x7f39c4017000 comp rx=0 tx=0).stop 2024-02-17T12:04:33.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39d4c29740 1 -- 172.21.15.94:0/2560749033 shutdown_connections 2024-02-17T12:04:33.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39d4c29740 1 --2- 172.21.15.94:0/2560749033 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55810c4bbdd0 0x55810c4be1f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39d4c29740 1 --2- 172.21.15.94:0/2560749033 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4ab8b0 0x55810c4a9230 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39d4c29740 1 --2- 172.21.15.94:0/2560749033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2d30 0x55810c4a8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39aaffd640 0 client.26669 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39d4c29740 1 --2- 172.21.15.94:0/2560749033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4a2380 0x55810c4a87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39d4c29740 1 -- 172.21.15.94:0/2560749033 >> 172.21.15.94:0/2560749033 conn(0x55810c49c840 msgr2=0x55810c49e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39d4c29740 1 -- 172.21.15.94:0/2560749033 shutdown_connections 2024-02-17T12:04:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39d4c29740 1 -- 172.21.15.94:0/2560749033 wait complete. 2024-02-17T12:04:33.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.150+0000 7f39d4c29740 20 client.26669 trim_cache size 0 max 16384 2024-02-17T12:04:33.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39d4c29740 1 Processor -- start 2024-02-17T12:04:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39d4c29740 1 -- start start 2024-02-17T12:04:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39d4c29740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2380 0x55810c4aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39d4c29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4a2d30 0x55810c4a9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39d4c29740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4ab8b0 0x55810c4a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39d4c29740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c4bbc00 con 0x55810c4a2380 2024-02-17T12:04:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39d4c29740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c4ae700 con 0x55810c4ab8b0 2024-02-17T12:04:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39d4c29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c4a1410 con 0x55810c4a2d30 2024-02-17T12:04:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2380 0x55810c4aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39cb7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2380 0x55810c4aaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42704/0 (socket says 172.21.15.94:42704) 2024-02-17T12:04:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39cb7fe640 1 -- 172.21.15.94:0/1303421306 learned_addr learned my addr 172.21.15.94:0/1303421306 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39c3fff640 1 --2- 172.21.15.94:0/1303421306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4a2d30 0x55810c4a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39cbfff640 1 --2- 172.21.15.94:0/1303421306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4ab8b0 0x55810c4a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.171+0000 7f39cb7fe640 1 -- 172.21.15.94:0/1303421306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4a2d30 msgr2=0x55810c4a9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.172+0000 7f39cb7fe640 1 --2- 172.21.15.94:0/1303421306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4a2d30 0x55810c4a9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.172+0000 7f39cb7fe640 1 -- 172.21.15.94:0/1303421306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4ab8b0 msgr2=0x55810c4a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.172+0000 7f39cb7fe640 1 --2- 172.21.15.94:0/1303421306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4ab8b0 0x55810c4a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.172+0000 7f39cb7fe640 1 -- 172.21.15.94:0/1303421306 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55810c3a4c00 con 0x55810c4a2380 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.172+0000 7f39cb7fe640 1 --2- 172.21.15.94:0/1303421306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2380 0x55810c4aaee0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f39bc000c00 tx=0x7f39bc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbd1f5ab3b86c94 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.172+0000 7f39c17fa640 1 -- 172.21.15.94:0/1303421306 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc017040 con 0x55810c4a2380 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.172+0000 7f39c17fa640 1 -- 172.21.15.94:0/1303421306 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39bc025070 con 0x55810c4a2380 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.172+0000 7f39c17fa640 1 -- 172.21.15.94:0/1303421306 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc00fbe0 con 0x55810c4a2380 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.173+0000 7f39d4c29740 1 -- 172.21.15.94:0/1303421306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2380 msgr2=0x55810c4aaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.173+0000 7f39d4c29740 1 --2- 172.21.15.94:0/1303421306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2380 0x55810c4aaee0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f39bc000c00 tx=0x7f39bc009260 comp rx=0 tx=0).stop 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.173+0000 7f39d4c29740 1 -- 172.21.15.94:0/1303421306 shutdown_connections 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.173+0000 7f39d4c29740 1 --2- 172.21.15.94:0/1303421306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4ab8b0 0x55810c4a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.173+0000 7f39d4c29740 1 --2- 172.21.15.94:0/1303421306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4a2d30 0x55810c4a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.173+0000 7f39d4c29740 1 --2- 172.21.15.94:0/1303421306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c4a2380 0x55810c4aaee0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.173+0000 7f39d4c29740 1 -- 172.21.15.94:0/1303421306 >> 172.21.15.94:0/1303421306 conn(0x55810c49c840 msgr2=0x55810c4a3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:33.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.174+0000 7f39d4c29740 1 -- 172.21.15.94:0/1303421306 shutdown_connections 2024-02-17T12:04:33.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.174+0000 7f39d4c29740 1 -- 172.21.15.94:0/1303421306 wait complete. 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.174+0000 7f39d4c29740 1 -- start start 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.175+0000 7f39d4c29740 1 Processor -- start 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.175+0000 7f39d4c29740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4a2d30 0x55810c4b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.175+0000 7f39d4c29740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4ab8b0 0x55810c4b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.175+0000 7f39d4c29740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c5102b0 0x55810c5126a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.175+0000 7f39d4c29740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c512be0 con 0x55810c5102b0 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.175+0000 7f39d4c29740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c512d20 con 0x55810c4a2d30 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.175+0000 7f39d4c29740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810c512e60 con 0x55810c4ab8b0 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.175+0000 7f39d4c29740 10 client.0 _ll_register_callbacks cb 0x7f39c400e490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.175+0000 7f39cbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c5102b0 0x55810c5126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.175+0000 7f39cbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c5102b0 0x55810c5126a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42706/0 (socket says 172.21.15.94:42706) 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.175+0000 7f39cbfff640 1 -- 172.21.15.94:0/2882715620 learned_addr learned my addr 172.21.15.94:0/2882715620 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.175+0000 7f39c3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4ab8b0 0x55810c4b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.175+0000 7f39cb7fe640 1 --2- 172.21.15.94:0/2882715620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4a2d30 0x55810c4b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.176+0000 7f39c3fff640 1 -- 172.21.15.94:0/2882715620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4a2d30 msgr2=0x55810c4b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.176+0000 7f39c3fff640 1 --2- 172.21.15.94:0/2882715620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4a2d30 0x55810c4b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.176+0000 7f39c3fff640 1 -- 172.21.15.94:0/2882715620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c5102b0 msgr2=0x55810c5126a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.176+0000 7f39c3fff640 1 --2- 172.21.15.94:0/2882715620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c5102b0 0x55810c5126a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.176+0000 7f39c3fff640 1 -- 172.21.15.94:0/2882715620 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc009d80 con 0x55810c4ab8b0 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.176+0000 7f39cb7fe640 1 --2- 172.21.15.94:0/2882715620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4a2d30 0x55810c4b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.176+0000 7f39c3fff640 1 --2- 172.21.15.94:0/2882715620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4ab8b0 0x55810c4b2110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009830 tx=0x7f39b402cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b75475d136cee5a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.176+0000 7f39aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.176+0000 7f39aaffd640 1 -- 172.21.15.94:0/2882715620 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b40b8040 con 0x55810c4ab8b0 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39aaffd640 1 -- 172.21.15.94:0/2882715620 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f39b40bccc0 con 0x55810c4ab8b0 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39d4c29740 1 -- 172.21.15.94:0/2882715620 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55810c42f900 con 0x55810c4ab8b0 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39aaffd640 1 -- 172.21.15.94:0/2882715620 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39b40b8860 con 0x55810c4ab8b0 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39d4c29740 10 client.26713 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:33.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39d4c29740 1 -- 172.21.15.94:0/2882715620 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55810c3ca910 con 0x55810c4ab8b0 2024-02-17T12:04:33.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39aaffd640 1 -- 172.21.15.94:0/2882715620 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39b40bc390 con 0x55810c4ab8b0 2024-02-17T12:04:33.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39aaffd640 1 -- 172.21.15.94:0/2882715620 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39b40ed400 con 0x55810c4ab8b0 2024-02-17T12:04:33.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39d4c29740 10 client.26713 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:33.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39d4c29740 10 client.26713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:33.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39d4c29740 20 client.26713 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:33.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39d4c29740 1 --2- 172.21.15.94:0/2882715620 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55810c5142d0 0x55810c5166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39d4c29740 4 client.26713 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:04:33.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39d4c29740 1 -- 172.21.15.94:0/2882715620 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55810c42f900 con 0x55810c5142d0 2024-02-17T12:04:33.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.177+0000 7f39cb7fe640 1 --2- 172.21.15.94:0/2882715620 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55810c5142d0 0x55810c5166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.178+0000 7f39aaffd640 10 client.26713 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:33.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.178+0000 7f39cb7fe640 1 --2- 172.21.15.94:0/2882715620 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55810c5142d0 0x55810c5166f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321a8baa112513d5 server_cookie=b8970b52a6780ce in_seq=0 out_seq=0 2024-02-17T12:04:33.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.179+0000 7f39aaffd640 1 -- 172.21.15.94:0/2882715620 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55810c42f900 con 0x55810c5142d0 2024-02-17T12:04:33.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.179+0000 7f39aaffd640 10 client.26713 handle_command_reply: tid=0 2024-02-17T12:04:33.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.179+0000 7f39d4c29740 1 client.26713 shutdown 2024-02-17T12:04:33.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.180+0000 7f39d4c29740 1 -- 172.21.15.94:0/2882715620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4ab8b0 msgr2=0x55810c4b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.180+0000 7f39d4c29740 1 --2- 172.21.15.94:0/2882715620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4ab8b0 0x55810c4b2110 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f39b4009830 tx=0x7f39b402cc20 comp rx=0 tx=0).stop 2024-02-17T12:04:33.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.180+0000 7f39d4c29740 1 -- 172.21.15.94:0/2882715620 shutdown_connections 2024-02-17T12:04:33.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.180+0000 7f39d4c29740 1 --2- 172.21.15.94:0/2882715620 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55810c5142d0 0x55810c5166f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.180+0000 7f39d4c29740 1 --2- 172.21.15.94:0/2882715620 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55810c5102b0 0x55810c5126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.180+0000 7f39d4c29740 1 --2- 172.21.15.94:0/2882715620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55810c4ab8b0 0x55810c4b2110 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.180+0000 7f39d4c29740 1 --2- 172.21.15.94:0/2882715620 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55810c4a2d30 0x55810c4b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.180+0000 7f39d4c29740 1 -- 172.21.15.94:0/2882715620 >> 172.21.15.94:0/2882715620 conn(0x55810c49c840 msgr2=0x55810c49de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:33.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.180+0000 7f39aaffd640 0 client.26713 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:33.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.180+0000 7f39d4c29740 1 -- 172.21.15.94:0/2882715620 shutdown_connections 2024-02-17T12:04:33.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.180+0000 7f39d4c29740 1 -- 172.21.15.94:0/2882715620 wait complete. 2024-02-17T12:04:33.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.180+0000 7f39d4c29740 20 client.26713 trim_cache size 0 max 16384 2024-02-17T12:04:33.184 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:04:33.184 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:04:33.184 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "f76b39fa-4a3a-4919-95bf-bd5c1a429c80", 2024-02-17T12:04:33.184 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:04:33.184 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:04:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.181+0000 7f39d1f77640 1 -- 172.21.15.94:0/609104539 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f39b80a2230 msgr2=0x7f39b80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.181+0000 7f39d1f77640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f39b80a2230 0x7f39b80a46f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f39c4020000 tx=0x7f39c400f790 comp rx=0 tx=0).stop 2024-02-17T12:04:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.181+0000 7f39d1f77640 1 -- 172.21.15.94:0/609104539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc103c40 msgr2=0x7f39cc1a3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.181+0000 7f39d1f77640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc103c40 0x7f39cc1a3e70 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f39b4037040 tx=0x7f39b40045b0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.181+0000 7f39d1f77640 1 -- 172.21.15.94:0/609104539 shutdown_connections 2024-02-17T12:04:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.181+0000 7f39d1f77640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f39b80a2230 0x7f39b80a46f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.181+0000 7f39d1f77640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39cc1060d0 0x7f39cc19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.181+0000 7f39d1f77640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39cc103c40 0x7f39cc1a3e70 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.181+0000 7f39d1f77640 1 --2- 172.21.15.94:0/609104539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39cc0688c0 0x7f39cc1a3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.181+0000 7f39d1f77640 1 -- 172.21.15.94:0/609104539 >> 172.21.15.94:0/609104539 conn(0x7f39cc077180 msgr2=0x7f39cc101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.181+0000 7f39d1f77640 1 -- 172.21.15.94:0/609104539 shutdown_connections 2024-02-17T12:04:33.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.181+0000 7f39d1f77640 1 -- 172.21.15.94:0/609104539 wait complete. 2024-02-17T12:04:33.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:04:33.196 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:04:33.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.369+0000 7f2906fc9640 1 -- 172.21.15.94:0/2104292162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2900108180 msgr2=0x7f29001087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.369+0000 7f2906fc9640 1 --2- 172.21.15.94:0/2104292162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2900108180 0x7f29001087b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009900 tx=0x7f28f402bc80 comp rx=0 tx=0).stop 2024-02-17T12:04:33.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.370+0000 7f2906fc9640 1 -- 172.21.15.94:0/2104292162 shutdown_connections 2024-02-17T12:04:33.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.370+0000 7f2906fc9640 1 --2- 172.21.15.94:0/2104292162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f290010cd50 0x7f2900073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.370+0000 7f2906fc9640 1 --2- 172.21.15.94:0/2104292162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f290010c3b0 0x7f290010c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.370+0000 7f2906fc9640 1 --2- 172.21.15.94:0/2104292162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2900108180 0x7f29001087b0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.370+0000 7f2906fc9640 1 -- 172.21.15.94:0/2104292162 >> 172.21.15.94:0/2104292162 conn(0x7f290006c270 msgr2=0x7f290006c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:33.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.371+0000 7f2906fc9640 1 -- 172.21.15.94:0/2104292162 shutdown_connections 2024-02-17T12:04:33.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.371+0000 7f2906fc9640 1 -- 172.21.15.94:0/2104292162 wait complete. 2024-02-17T12:04:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.372+0000 7f2906fc9640 1 Processor -- start 2024-02-17T12:04:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.372+0000 7f2906fc9640 1 -- start start 2024-02-17T12:04:33.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.372+0000 7f2906fc9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2900108180 0x7f2900083cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.372+0000 7f2906fc9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f290010c3b0 0x7f290007de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.372+0000 7f2906fc9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f290010cd50 0x7f290007e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.372+0000 7f2906fc9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290007eb10 con 0x7f290010cd50 2024-02-17T12:04:33.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.372+0000 7f2906fc9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290007ec80 con 0x7f290010c3b0 2024-02-17T12:04:33.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.372+0000 7f2906fc9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2900136630 con 0x7f2900108180 2024-02-17T12:04:33.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.373+0000 7f2904d65640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2900108180 0x7f2900083cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.373+0000 7f28fffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f290010c3b0 0x7f290007de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.373+0000 7f2904d65640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2900108180 0x7f2900083cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56238/0 (socket says 172.21.15.94:56238) 2024-02-17T12:04:33.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.373+0000 7f2904d65640 1 -- 172.21.15.94:0/3755593529 learned_addr learned my addr 172.21.15.94:0/3755593529 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:33.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.373+0000 7f2905566640 1 --2- 172.21.15.94:0/3755593529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f290010cd50 0x7f290007e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.373+0000 7f2904d65640 1 -- 172.21.15.94:0/3755593529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f290010c3b0 msgr2=0x7f290007de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.373+0000 7f2904d65640 1 --2- 172.21.15.94:0/3755593529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f290010c3b0 0x7f290007de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.373+0000 7f2904d65640 1 -- 172.21.15.94:0/3755593529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f290010cd50 msgr2=0x7f290007e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.373+0000 7f2904d65640 1 --2- 172.21.15.94:0/3755593529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f290010cd50 0x7f290007e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.373+0000 7f2904d65640 1 -- 172.21.15.94:0/3755593529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f4009560 con 0x7f2900108180 2024-02-17T12:04:33.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.374+0000 7f2904d65640 1 --2- 172.21.15.94:0/3755593529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2900108180 0x7f2900083cf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f28f4005ab0 tx=0x7f28f4004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:33.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.374+0000 7f28fdffb640 1 -- 172.21.15.94:0/3755593529 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4004460 con 0x7f2900108180 2024-02-17T12:04:33.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.374+0000 7f2906fc9640 1 -- 172.21.15.94:0/3755593529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29001368b0 con 0x7f2900108180 2024-02-17T12:04:33.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.374+0000 7f28fdffb640 1 -- 172.21.15.94:0/3755593529 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28f40052d0 con 0x7f2900108180 2024-02-17T12:04:33.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.374+0000 7f28fdffb640 1 -- 172.21.15.94:0/3755593529 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4003de0 con 0x7f2900108180 2024-02-17T12:04:33.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.375+0000 7f2906fc9640 1 -- 172.21.15.94:0/3755593529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2900136e10 con 0x7f2900108180 2024-02-17T12:04:33.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.379+0000 7f2909ba0740 1 Processor -- start 2024-02-17T12:04:33.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.381+0000 7f2909ba0740 1 -- start start 2024-02-17T12:04:33.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.381+0000 7f2909ba0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.381+0000 7f2909ba0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.381+0000 7f2909ba0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4bf7b0 0x55ef7c4c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.381+0000 7f2909ba0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7c425280 con 0x55ef7c4b6bd0 2024-02-17T12:04:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.381+0000 7f2909ba0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7c4227b0 con 0x55ef7c4b62b0 2024-02-17T12:04:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.381+0000 7f2909ba0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7c41df90 con 0x55ef7c4bf7b0 2024-02-17T12:04:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.381+0000 7f28fdffb640 1 -- 172.21.15.94:0/3755593529 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f28f4005440 con 0x7f2900108180 2024-02-17T12:04:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.381+0000 7f28fffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4bf7b0 0x55ef7c4c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.381+0000 7f28fffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4bf7b0 0x55ef7c4c1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56254/0 (socket says 172.21.15.94:56254) 2024-02-17T12:04:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.381+0000 7f28fffff640 1 -- 172.21.15.94:0/2122879062 learned_addr learned my addr 172.21.15.94:0/2122879062 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.382+0000 7f28fffff640 1 -- 172.21.15.94:0/2122879062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b62b0 msgr2=0x55ef7c4b6690 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:04:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.382+0000 7f28fffff640 1 --2- 172.21.15.94:0/2122879062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4b6690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.382+0000 7f28fffff640 1 -- 172.21.15.94:0/2122879062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 msgr2=0x55ef7c4bf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.382+0000 7f28fffff640 1 --2- 172.21.15.94:0/2122879062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.382+0000 7f28fffff640 1 -- 172.21.15.94:0/2122879062 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef7c3dc570 con 0x55ef7c4bf7b0 2024-02-17T12:04:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.382+0000 7f28fdffb640 1 --2- 172.21.15.94:0/3755593529 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f28e80a2370 0x7f28e80a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.382+0000 7f28fdffb640 1 -- 172.21.15.94:0/3755593529 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f28f40f0800 con 0x7f2900108180 2024-02-17T12:04:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.383+0000 7f28fffff640 1 --2- 172.21.15.94:0/2122879062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4bf7b0 0x55ef7c4c1ba0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009fd0 tx=0x7f28f80096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc764e4aa4462340 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.383+0000 7f2905566640 1 --2- 172.21.15.94:0/3755593529 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f28e80a2370 0x7f28e80a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.383+0000 7f28e6ffd640 1 -- 172.21.15.94:0/2122879062 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8010040 con 0x55ef7c4bf7b0 2024-02-17T12:04:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.383+0000 7f28e6ffd640 1 -- 172.21.15.94:0/2122879062 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28f8013c70 con 0x55ef7c4bf7b0 2024-02-17T12:04:33.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.383+0000 7f28e6ffd640 1 -- 172.21.15.94:0/2122879062 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8014de0 con 0x55ef7c4bf7b0 2024-02-17T12:04:33.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.386+0000 7f2909ba0740 1 -- 172.21.15.94:0/2122879062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4bf7b0 msgr2=0x55ef7c4c1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.386+0000 7f2909ba0740 1 --2- 172.21.15.94:0/2122879062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4bf7b0 0x55ef7c4c1ba0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009fd0 tx=0x7f28f80096d0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.386+0000 7f2905566640 1 --2- 172.21.15.94:0/3755593529 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f28e80a2370 0x7f28e80a4830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f290006cb70 tx=0x7f28f0009060 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.386+0000 7f2909ba0740 1 -- 172.21.15.94:0/2122879062 shutdown_connections 2024-02-17T12:04:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.386+0000 7f2909ba0740 1 --2- 172.21.15.94:0/2122879062 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4bf7b0 0x55ef7c4c1ba0 secure :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009fd0 tx=0x7f28f80096d0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.386+0000 7f2909ba0740 1 --2- 172.21.15.94:0/2122879062 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4bf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.386+0000 7f2909ba0740 1 --2- 172.21.15.94:0/2122879062 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.386+0000 7f2909ba0740 1 -- 172.21.15.94:0/2122879062 >> 172.21.15.94:0/2122879062 conn(0x55ef7c4b0990 msgr2=0x55ef7c4b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.386+0000 7f2909ba0740 1 -- 172.21.15.94:0/2122879062 shutdown_connections 2024-02-17T12:04:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.386+0000 7f2909ba0740 1 -- 172.21.15.94:0/2122879062 wait complete. 2024-02-17T12:04:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.386+0000 7f2909ba0740 1 -- start start 2024-02-17T12:04:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.386+0000 7f2909ba0740 1 Processor -- start 2024-02-17T12:04:33.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.387+0000 7f2909ba0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4bc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.387+0000 7f2909ba0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4bcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.387+0000 7f2909ba0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4cb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.387+0000 7f2909ba0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7c0f1430 con 0x55ef7c4b6bd0 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.387+0000 7f2909ba0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7c0ade00 con 0x55ef7c4b62b0 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.387+0000 7f2909ba0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7c3a2fd0 con 0x55ef7c4bd0d0 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.387+0000 7f2909ba0740 10 client.0 _ll_register_callbacks cb 0x55ef7c3f8fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.388+0000 7f2904d65640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4bc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.388+0000 7f2904d65640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4bc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42260/0 (socket says 172.21.15.94:42260) 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.388+0000 7f2904d65640 1 -- 172.21.15.94:0/241354134 learned_addr learned my addr 172.21.15.94:0/241354134 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.388+0000 7f2905566640 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.388+0000 7f28fffff640 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4bcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.388+0000 7f2904d65640 1 -- 172.21.15.94:0/241354134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4bd0d0 msgr2=0x55ef7c4cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.388+0000 7f2904d65640 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.388+0000 7f2904d65640 1 -- 172.21.15.94:0/241354134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 msgr2=0x55ef7c4bcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.388+0000 7f2904d65640 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4bcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.388+0000 7f2904d65640 1 -- 172.21.15.94:0/241354134 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f8009d00 con 0x55ef7c4b62b0 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.388+0000 7f2905566640 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.388+0000 7f28fffff640 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4bcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.389+0000 7f2904d65640 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4bc650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f28f40bcf90 tx=0x7f28f40bccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=319684f3db8c00b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.389+0000 7f28dffff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.389+0000 7f28dffff640 1 -- 172.21.15.94:0/241354134 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f40c13e0 con 0x55ef7c4b62b0 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.389+0000 7f2909ba0740 1 -- 172.21.15.94:0/241354134 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef7c11d7e0 con 0x55ef7c4b62b0 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.389+0000 7f28dffff640 1 -- 172.21.15.94:0/241354134 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f28f40c5a30 con 0x55ef7c4b62b0 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.389+0000 7f28dffff640 1 -- 172.21.15.94:0/241354134 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28f40d9070 con 0x55ef7c4b62b0 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.389+0000 7f2909ba0740 10 client.26681 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.389+0000 7f2909ba0740 1 -- 172.21.15.94:0/241354134 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ef7c41c610 con 0x55ef7c4b62b0 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.389+0000 7f28dffff640 1 -- 172.21.15.94:0/241354134 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55ef7c11d7e0 con 0x55ef7c4b62b0 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.390+0000 7f28dffff640 1 -- 172.21.15.94:0/241354134 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f28f403c070 con 0x55ef7c4b62b0 2024-02-17T12:04:33.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.390+0000 7f2909ba0740 10 client.26681 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.390+0000 7f2909ba0740 10 client.26681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.390+0000 7f2909ba0740 20 client.26681 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.390+0000 7f2909ba0740 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ef7c4cfbf0 0x55ef7c4d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.390+0000 7f2909ba0740 4 client.26681 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:04:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.390+0000 7f2909ba0740 1 -- 172.21.15.94:0/241354134 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef7c15a720 con 0x55ef7c4cfbf0 2024-02-17T12:04:33.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.390+0000 7f28fffff640 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ef7c4cfbf0 0x55ef7c4d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.391+0000 7f28fffff640 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ef7c4cfbf0 0x55ef7c4d2000 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72386b5a63401511 server_cookie=49b09e72ae23b6c8 in_seq=0 out_seq=0 2024-02-17T12:04:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.391+0000 7f28dffff640 10 client.26681 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.392+0000 7f28dffff640 1 -- 172.21.15.94:0/241354134 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ef7c15a720 con 0x55ef7c4cfbf0 2024-02-17T12:04:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.392+0000 7f28dffff640 10 client.26681 handle_command_reply: tid=0 2024-02-17T12:04:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.392+0000 7f2909ba0740 1 client.26681 shutdown 2024-02-17T12:04:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.392+0000 7f2909ba0740 1 -- 172.21.15.94:0/241354134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b62b0 msgr2=0x55ef7c4bc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.392+0000 7f2909ba0740 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4bc650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f28f40bcf90 tx=0x7f28f40bccb0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.392+0000 7f2909ba0740 1 -- 172.21.15.94:0/241354134 shutdown_connections 2024-02-17T12:04:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.392+0000 7f2909ba0740 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ef7c4cfbf0 0x55ef7c4d2000 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.392+0000 7f2909ba0740 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.392+0000 7f2909ba0740 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4bcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.393+0000 7f2909ba0740 1 --2- 172.21.15.94:0/241354134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4bc650 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.393+0000 7f2909ba0740 1 -- 172.21.15.94:0/241354134 >> 172.21.15.94:0/241354134 conn(0x55ef7c4b0990 msgr2=0x55ef7c4b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:33.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.393+0000 7f28dffff640 0 client.26681 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:33.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.393+0000 7f2909ba0740 1 -- 172.21.15.94:0/241354134 shutdown_connections 2024-02-17T12:04:33.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.393+0000 7f2909ba0740 1 -- 172.21.15.94:0/241354134 wait complete. 2024-02-17T12:04:33.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.393+0000 7f2909ba0740 20 client.26681 trim_cache size 0 max 16384 2024-02-17T12:04:33.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.417+0000 7f2909ba0740 1 Processor -- start 2024-02-17T12:04:33.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.417+0000 7f2909ba0740 1 -- start start 2024-02-17T12:04:33.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.417+0000 7f2909ba0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.417+0000 7f2909ba0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.417+0000 7f2909ba0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f28fffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f28fffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4b98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42264/0 (socket says 172.21.15.94:42264) 2024-02-17T12:04:33.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f2904d65640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f2904d65640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4af8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56268/0 (socket says 172.21.15.94:56268) 2024-02-17T12:04:33.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f2905566640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f28fffff640 1 -- 172.21.15.94:0/1397026789 learned_addr learned my addr 172.21.15.94:0/1397026789 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:33.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f2909ba0740 1 -- 172.21.15.94:0/1397026789 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7c4b19e0 con 0x55ef7c4bd0d0 2024-02-17T12:04:33.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f2909ba0740 1 -- 172.21.15.94:0/1397026789 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7c4b1c10 con 0x55ef7c4b6bd0 2024-02-17T12:04:33.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f2909ba0740 1 -- 172.21.15.94:0/1397026789 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7c4b1610 con 0x55ef7c4b62b0 2024-02-17T12:04:33.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f28fffff640 1 -- 172.21.15.94:0/1397026789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4b62b0 msgr2=0x55ef7c4af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f28fffff640 1 --2- 172.21.15.94:0/1397026789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4af8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f28fffff640 1 -- 172.21.15.94:0/1397026789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4bd0d0 msgr2=0x55ef7c4b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f28fffff640 1 --2- 172.21.15.94:0/1397026789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4b9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.418+0000 7f28fffff640 1 -- 172.21.15.94:0/1397026789 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef7c3b8ac0 con 0x55ef7c4b6bd0 2024-02-17T12:04:33.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.419+0000 7f2904d65640 1 --2- 172.21.15.94:0/1397026789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:33.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.419+0000 7f28fffff640 1 --2- 172.21.15.94:0/1397026789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4b98f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f28f8000f80 tx=0x7f28f800d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50b18801abeb6e4d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:33.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.419+0000 7f28e67fc640 1 -- 172.21.15.94:0/1397026789 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f802c070 con 0x55ef7c4b6bd0 2024-02-17T12:04:33.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.419+0000 7f28e67fc640 1 -- 172.21.15.94:0/1397026789 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28f8013de0 con 0x55ef7c4b6bd0 2024-02-17T12:04:33.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.419+0000 7f28e67fc640 1 -- 172.21.15.94:0/1397026789 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8023040 con 0x55ef7c4b6bd0 2024-02-17T12:04:33.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.420+0000 7f2909ba0740 1 -- 172.21.15.94:0/1397026789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b6bd0 msgr2=0x55ef7c4b98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.420+0000 7f2909ba0740 1 --2- 172.21.15.94:0/1397026789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4b98f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f28f8000f80 tx=0x7f28f800d040 comp rx=0 tx=0).stop 2024-02-17T12:04:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.420+0000 7f2909ba0740 1 -- 172.21.15.94:0/1397026789 shutdown_connections 2024-02-17T12:04:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.420+0000 7f2909ba0740 1 --2- 172.21.15.94:0/1397026789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.420+0000 7f2909ba0740 1 --2- 172.21.15.94:0/1397026789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4b98f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.420+0000 7f2909ba0740 1 --2- 172.21.15.94:0/1397026789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.420+0000 7f2909ba0740 1 -- 172.21.15.94:0/1397026789 >> 172.21.15.94:0/1397026789 conn(0x55ef7c4b0990 msgr2=0x55ef7c4af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.420+0000 7f2909ba0740 1 -- 172.21.15.94:0/1397026789 shutdown_connections 2024-02-17T12:04:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.420+0000 7f2909ba0740 1 -- 172.21.15.94:0/1397026789 wait complete. 2024-02-17T12:04:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.421+0000 7f2909ba0740 1 -- start start 2024-02-17T12:04:33.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.421+0000 7f2909ba0740 1 Processor -- start 2024-02-17T12:04:33.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.422+0000 7f2909ba0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.422+0000 7f2909ba0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.422+0000 7f2909ba0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.422+0000 7f2909ba0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7c4c59c0 con 0x55ef7c4b6bd0 2024-02-17T12:04:33.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.422+0000 7f2909ba0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7c4c5b30 con 0x55ef7c4bd0d0 2024-02-17T12:04:33.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.422+0000 7f2909ba0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7c5260f0 con 0x55ef7c4b62b0 2024-02-17T12:04:33.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.422+0000 7f2905566640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.422+0000 7f2909ba0740 10 client.0 _ll_register_callbacks cb 0x7f28f8009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:33.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.422+0000 7f2905566640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4c52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42280/0 (socket says 172.21.15.94:42280) 2024-02-17T12:04:33.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.422+0000 7f2904d65640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.422+0000 7f2905566640 1 -- 172.21.15.94:0/3186448634 learned_addr learned my addr 172.21.15.94:0/3186448634 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:33.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.422+0000 7f28fffff640 1 --2- 172.21.15.94:0/3186448634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f2905566640 1 -- 172.21.15.94:0/3186448634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4b62b0 msgr2=0x55ef7c4c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f2905566640 1 --2- 172.21.15.94:0/3186448634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f2905566640 1 -- 172.21.15.94:0/3186448634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 msgr2=0x55ef7c4c4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f2905566640 1 --2- 172.21.15.94:0/3186448634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4c4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f2905566640 1 -- 172.21.15.94:0/3186448634 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef7c41c610 con 0x55ef7c4bd0d0 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f2905566640 1 --2- 172.21.15.94:0/3186448634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4c52c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f28f000ed10 tx=0x7f28f0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4c44c5a73d3cc65 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f28dffff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f28dffff640 1 -- 172.21.15.94:0/3186448634 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f0022460 con 0x55ef7c4bd0d0 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f28dffff640 1 -- 172.21.15.94:0/3186448634 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f28f00225c0 con 0x55ef7c4bd0d0 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f2909ba0740 1 -- 172.21.15.94:0/3186448634 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28f8013c70 con 0x55ef7c4bd0d0 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f28dffff640 1 -- 172.21.15.94:0/3186448634 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28f0005970 con 0x55ef7c4bd0d0 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f2909ba0740 10 client.26687 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f2909ba0740 1 -- 172.21.15.94:0/3186448634 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ef7c3dc570 con 0x55ef7c4bd0d0 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f28dffff640 1 -- 172.21.15.94:0/3186448634 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28f8013c70 con 0x55ef7c4bd0d0 2024-02-17T12:04:33.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f28dffff640 1 -- 172.21.15.94:0/3186448634 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f28f0011070 con 0x55ef7c4bd0d0 2024-02-17T12:04:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f2909ba0740 10 client.26687 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f2909ba0740 10 client.26687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.423+0000 7f2909ba0740 20 client.26687 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.424+0000 7f2909ba0740 1 --2- 172.21.15.94:0/3186448634 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ef7c527860 0x55ef7c529c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.424+0000 7f2909ba0740 4 client.26687 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:04:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.424+0000 7f2909ba0740 1 -- 172.21.15.94:0/3186448634 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ef7c1a9890 con 0x55ef7c527860 2024-02-17T12:04:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.424+0000 7f2904d65640 1 --2- 172.21.15.94:0/3186448634 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ef7c527860 0x55ef7c529c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.424+0000 7f28dffff640 10 client.26687 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:33.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.424+0000 7f2904d65640 1 --2- 172.21.15.94:0/3186448634 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ef7c527860 0x55ef7c529c80 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=810c1e7f011e1da0 server_cookie=ce224b96a29b40ef in_seq=0 out_seq=0 2024-02-17T12:04:33.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.425+0000 7f28dffff640 1 -- 172.21.15.94:0/3186448634 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ef7c1a9890 con 0x55ef7c527860 2024-02-17T12:04:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.425+0000 7f28dffff640 10 client.26687 handle_command_reply: tid=0 2024-02-17T12:04:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.425+0000 7f2909ba0740 1 client.26687 shutdown 2024-02-17T12:04:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.426+0000 7f2909ba0740 1 -- 172.21.15.94:0/3186448634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4bd0d0 msgr2=0x55ef7c4c52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.426+0000 7f2909ba0740 1 --2- 172.21.15.94:0/3186448634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4c52c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f28f000ed10 tx=0x7f28f0004750 comp rx=0 tx=0).stop 2024-02-17T12:04:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.426+0000 7f2909ba0740 1 -- 172.21.15.94:0/3186448634 shutdown_connections 2024-02-17T12:04:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.426+0000 7f2909ba0740 1 --2- 172.21.15.94:0/3186448634 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55ef7c527860 0x55ef7c529c80 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.426+0000 7f2909ba0740 1 --2- 172.21.15.94:0/3186448634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ef7c4bd0d0 0x55ef7c4c52c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.426+0000 7f2909ba0740 1 --2- 172.21.15.94:0/3186448634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ef7c4b6bd0 0x55ef7c4c4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.426+0000 7f2909ba0740 1 --2- 172.21.15.94:0/3186448634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ef7c4b62b0 0x55ef7c4c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.426+0000 7f2909ba0740 1 -- 172.21.15.94:0/3186448634 >> 172.21.15.94:0/3186448634 conn(0x55ef7c4b0990 msgr2=0x55ef7c4b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.426+0000 7f28dffff640 0 client.26687 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.426+0000 7f2909ba0740 1 -- 172.21.15.94:0/3186448634 shutdown_connections 2024-02-17T12:04:33.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.426+0000 7f2909ba0740 1 -- 172.21.15.94:0/3186448634 wait complete. 2024-02-17T12:04:33.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.426+0000 7f2909ba0740 20 client.26687 trim_cache size 0 max 16384 2024-02-17T12:04:33.432 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:04:33.432 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:04:33.432 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:04:33.432 INFO:teuthology.orchestra.run.smithi094.stdout: "f76b39fa-4a3a-4919-95bf-bd5c1a429c80": { 2024-02-17T12:04:33.432 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:04:33.432 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "f76b39fa-4a3a-4919-95bf-bd5c1a429c80", 2024-02-17T12:04:33.432 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:04:33.432 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:04:33.432 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:04:33.433 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:04:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.427+0000 7f2906fc9640 1 -- 172.21.15.94:0/3755593529 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f28e80a2370 msgr2=0x7f28e80a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.427+0000 7f2906fc9640 1 --2- 172.21.15.94:0/3755593529 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f28e80a2370 0x7f28e80a4830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f290006cb70 tx=0x7f28f0009060 comp rx=0 tx=0).stop 2024-02-17T12:04:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.427+0000 7f2906fc9640 1 -- 172.21.15.94:0/3755593529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2900108180 msgr2=0x7f2900083cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.427+0000 7f2906fc9640 1 --2- 172.21.15.94:0/3755593529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2900108180 0x7f2900083cf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f28f4005ab0 tx=0x7f28f4004060 comp rx=0 tx=0).stop 2024-02-17T12:04:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.427+0000 7f2906fc9640 1 -- 172.21.15.94:0/3755593529 shutdown_connections 2024-02-17T12:04:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.427+0000 7f2906fc9640 1 --2- 172.21.15.94:0/3755593529 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f28e80a2370 0x7f28e80a4830 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.427+0000 7f2906fc9640 1 --2- 172.21.15.94:0/3755593529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f290010cd50 0x7f290007e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.427+0000 7f2906fc9640 1 --2- 172.21.15.94:0/3755593529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f290010c3b0 0x7f290007de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.427+0000 7f2906fc9640 1 --2- 172.21.15.94:0/3755593529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2900108180 0x7f2900083cf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:33.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.427+0000 7f2906fc9640 1 -- 172.21.15.94:0/3755593529 >> 172.21.15.94:0/3755593529 conn(0x7f290006c270 msgr2=0x7f290010b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:33.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.427+0000 7f2906fc9640 1 -- 172.21.15.94:0/3755593529 shutdown_connections 2024-02-17T12:04:33.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:04:33.427+0000 7f2906fc9640 1 -- 172.21.15.94:0/3755593529 wait complete. 2024-02-17T12:04:33.443 INFO:tasks.cephfs.filesystem:scrub status for tag:f76b39fa-4a3a-4919-95bf-bd5c1a429c80 - {'path': '/', 'tag': 'f76b39fa-4a3a-4919-95bf-bd5c1a429c80', 'options': 'recursive,force'} 2024-02-17T12:04:34.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:33 smithi094 ceph-mon[32499]: pgmap v197: 97 pgs: 97 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 21 MiB/s wr, 1.10k op/s 2024-02-17T12:04:34.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:33 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:04:34.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:33 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:04:34.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:33 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:04:34.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:33 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:04:34.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:33 smithi189 ceph-mon[36588]: pgmap v197: 97 pgs: 97 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 21 MiB/s wr, 1.10k op/s 2024-02-17T12:04:34.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:33 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:04:34.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:33 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:04:34.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:33 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:04:34.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:33 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:04:34.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:33 smithi125 ceph-mon[36819]: pgmap v197: 97 pgs: 97 active+clean; 4.6 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 21 MiB/s wr, 1.10k op/s 2024-02-17T12:04:34.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:33 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:04:34.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:33 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:04:34.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:33 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:04:34.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:33 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:04:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.770+0000 7f09c8617640 1 -- 172.21.15.94:0/593706196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c0108180 msgr2=0x7f09c01085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.770+0000 7f09c8617640 1 --2- 172.21.15.94:0/593706196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c0108180 0x7f09c01085e0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009900 tx=0x7f09b402bc80 comp rx=0 tx=0).stop 2024-02-17T12:04:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.770+0000 7f09c8617640 1 -- 172.21.15.94:0/593706196 shutdown_connections 2024-02-17T12:04:34.771 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.770+0000 7f09c8617640 1 --2- 172.21.15.94:0/593706196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c0108b20 0x7f09c01149d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.770+0000 7f09c8617640 1 --2- 172.21.15.94:0/593706196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c0108180 0x7f09c01085e0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.770+0000 7f09c8617640 1 --2- 172.21.15.94:0/593706196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c010cb60 0x7f09c010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.770+0000 7f09c8617640 1 -- 172.21.15.94:0/593706196 >> 172.21.15.94:0/593706196 conn(0x7f09c006c6b0 msgr2=0x7f09c006cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.770+0000 7f09c8617640 1 -- 172.21.15.94:0/593706196 shutdown_connections 2024-02-17T12:04:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.770+0000 7f09c8617640 1 -- 172.21.15.94:0/593706196 wait complete. 2024-02-17T12:04:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.771+0000 7f09c8617640 1 Processor -- start 2024-02-17T12:04:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.771+0000 7f09c8617640 1 -- start start 2024-02-17T12:04:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.771+0000 7f09c8617640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c0108180 0x7f09c019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.771+0000 7f09c8617640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c0108b20 0x7f09c019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.771+0000 7f09c8617640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c010cb60 0x7f09c01a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.771+0000 7f09c8617640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01a3d60 con 0x7f09c0108180 2024-02-17T12:04:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.771+0000 7f09c8617640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01a3ed0 con 0x7f09c010cb60 2024-02-17T12:04:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.771+0000 7f09c8617640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01a4090 con 0x7f09c0108b20 2024-02-17T12:04:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.772+0000 7f09c6bb4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c010cb60 0x7f09c01a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.772+0000 7f09c6bb4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c010cb60 0x7f09c01a3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44694/0 (socket says 172.21.15.94:44694) 2024-02-17T12:04:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.772+0000 7f09c6bb4640 1 -- 172.21.15.94:0/490765078 learned_addr learned my addr 172.21.15.94:0/490765078 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.772+0000 7f09c5bb2640 1 --2- 172.21.15.94:0/490765078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c0108b20 0x7f09c019f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.772+0000 7f09c6bb4640 1 -- 172.21.15.94:0/490765078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c0108b20 msgr2=0x7f09c019f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.772+0000 7f09c6bb4640 1 --2- 172.21.15.94:0/490765078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c0108b20 0x7f09c019f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.772+0000 7f09c6bb4640 1 -- 172.21.15.94:0/490765078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c0108180 msgr2=0x7f09c019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.772+0000 7f09c6bb4640 1 --2- 172.21.15.94:0/490765078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c0108180 0x7f09c019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.772+0000 7f09c6bb4640 1 -- 172.21.15.94:0/490765078 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b4009560 con 0x7f09c010cb60 2024-02-17T12:04:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.773+0000 7f09c6bb4640 1 --2- 172.21.15.94:0/490765078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c010cb60 0x7f09c01a3760 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00e970 tx=0x7f09bc00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.773+0000 7f09b37fe640 1 -- 172.21.15.94:0/490765078 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc009720 con 0x7f09c010cb60 2024-02-17T12:04:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.773+0000 7f09c8617640 1 -- 172.21.15.94:0/490765078 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c01a4370 con 0x7f09c010cb60 2024-02-17T12:04:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.773+0000 7f09c8617640 1 -- 172.21.15.94:0/490765078 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c0115c30 con 0x7f09c010cb60 2024-02-17T12:04:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.774+0000 7f09b37fe640 1 -- 172.21.15.94:0/490765078 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09bc005500 con 0x7f09c010cb60 2024-02-17T12:04:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.774+0000 7f09b37fe640 1 -- 172.21.15.94:0/490765078 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc010430 con 0x7f09c010cb60 2024-02-17T12:04:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.776+0000 7f09b37fe640 1 -- 172.21.15.94:0/490765078 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f09bc010590 con 0x7f09c010cb60 2024-02-17T12:04:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.777+0000 7f09b37fe640 1 --2- 172.21.15.94:0/490765078 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09ac0a2350 0x7f09ac0a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.777+0000 7f09b37fe640 1 -- 172.21.15.94:0/490765078 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f09bc014070 con 0x7f09c010cb60 2024-02-17T12:04:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.777+0000 7f09cb1e0740 1 Processor -- start 2024-02-17T12:04:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.778+0000 7f09c5bb2640 1 --2- 172.21.15.94:0/490765078 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09ac0a2350 0x7f09ac0a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.778+0000 7f09cb1e0740 1 -- start start 2024-02-17T12:04:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.778+0000 7f09cb1e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09cb1e0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e2d60 0x556c7f5e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09cb1e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5ec2e0 0x556c7f5ee6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09cb1e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7f5519f0 con 0x556c7f5ec2e0 2024-02-17T12:04:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09cb1e0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7f54ef20 con 0x556c7f5e2d60 2024-02-17T12:04:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09cb1e0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7f54a700 con 0x556c7f5e3ad0 2024-02-17T12:04:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09c5bb2640 1 --2- 172.21.15.94:0/490765078 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09ac0a2350 0x7f09ac0a4810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f09c00727e0 tx=0x7f09b4000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09c63b3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09c63b3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:41000/0 (socket says 172.21.15.94:41000) 2024-02-17T12:04:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09c63b3640 1 -- 172.21.15.94:0/3964264926 learned_addr learned my addr 172.21.15.94:0/3964264926 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:34.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09c6bb4640 1 --2- 172.21.15.94:0/3964264926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5ec2e0 0x556c7f5ee6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09c5bb2640 1 --2- 172.21.15.94:0/3964264926 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e2d60 0x556c7f5e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09c63b3640 1 -- 172.21.15.94:0/3964264926 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e2d60 msgr2=0x556c7f5e31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09c63b3640 1 --2- 172.21.15.94:0/3964264926 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e2d60 0x556c7f5e31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09c63b3640 1 -- 172.21.15.94:0/3964264926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5ec2e0 msgr2=0x556c7f5ee6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09c63b3640 1 --2- 172.21.15.94:0/3964264926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5ec2e0 0x556c7f5ee6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09c63b3640 1 -- 172.21.15.94:0/3964264926 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c7f4e07b0 con 0x556c7f5e3ad0 2024-02-17T12:04:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.779+0000 7f09c5bb2640 1 --2- 172.21.15.94:0/3964264926 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e2d60 0x556c7f5e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:04:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.780+0000 7f09c63b3640 1 --2- 172.21.15.94:0/3964264926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e2820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009ea0 tx=0x7f09b8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1776d15195758fc7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.780+0000 7f09b0bfe640 1 -- 172.21.15.94:0/3964264926 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b800eee0 con 0x556c7f5e3ad0 2024-02-17T12:04:34.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.780+0000 7f09b0bfe640 1 -- 172.21.15.94:0/3964264926 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09b8004d20 con 0x556c7f5e3ad0 2024-02-17T12:04:34.783 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.780+0000 7f09b0bfe640 1 -- 172.21.15.94:0/3964264926 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b80108a0 con 0x556c7f5e3ad0 2024-02-17T12:04:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.780+0000 7f09cb1e0740 1 -- 172.21.15.94:0/3964264926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5e3ad0 msgr2=0x556c7f5e2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.780+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/3964264926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e2820 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f09b8009ea0 tx=0x7f09b8009f90 comp rx=0 tx=0).stop 2024-02-17T12:04:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.780+0000 7f09cb1e0740 1 -- 172.21.15.94:0/3964264926 shutdown_connections 2024-02-17T12:04:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.780+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/3964264926 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5ec2e0 0x556c7f5ee6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.781+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/3964264926 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e2d60 0x556c7f5e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.781+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/3964264926 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e2820 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.781+0000 7f09cb1e0740 1 -- 172.21.15.94:0/3964264926 >> 172.21.15.94:0/3964264926 conn(0x556c7f5aa550 msgr2=0x556c7f5dda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.781+0000 7f09cb1e0740 1 -- 172.21.15.94:0/3964264926 shutdown_connections 2024-02-17T12:04:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.781+0000 7f09cb1e0740 1 -- 172.21.15.94:0/3964264926 wait complete. 2024-02-17T12:04:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.781+0000 7f09cb1e0740 1 -- start start 2024-02-17T12:04:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.781+0000 7f09cb1e0740 1 Processor -- start 2024-02-17T12:04:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.781+0000 7f09cb1e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e2d60 0x556c7f5e9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.781+0000 7f09cb1e0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.781+0000 7f09cb1e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5ec2e0 0x556c7f5e9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.781+0000 7f09cb1e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7f4efbf0 con 0x556c7f5e2d60 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.782+0000 7f09c6bb4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5ec2e0 0x556c7f5e9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.782+0000 7f09c5bb2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.782+0000 7f09c6bb4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5ec2e0 0x556c7f5e9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:41016/0 (socket says 172.21.15.94:41016) 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.782+0000 7f09c6bb4640 1 -- 172.21.15.94:0/1405051885 learned_addr learned my addr 172.21.15.94:0/1405051885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.782+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1405051885 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7f21e250 con 0x556c7f5e3ad0 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.782+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1405051885 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7f4d0200 con 0x556c7f5ec2e0 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.782+0000 7f09c63b3640 1 --2- 172.21.15.94:0/1405051885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e2d60 0x556c7f5e9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.782+0000 7f09cb1e0740 10 client.0 _ll_register_callbacks cb 0x556c7f53ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.782+0000 7f09c6bb4640 1 -- 172.21.15.94:0/1405051885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e3ad0 msgr2=0x556c7f5e9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.782+0000 7f09c6bb4640 1 --2- 172.21.15.94:0/1405051885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.782+0000 7f09c6bb4640 1 -- 172.21.15.94:0/1405051885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e2d60 msgr2=0x556c7f5e9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.782+0000 7f09c6bb4640 1 --2- 172.21.15.94:0/1405051885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e2d60 0x556c7f5e9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.782+0000 7f09c6bb4640 1 -- 172.21.15.94:0/1405051885 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b8009b80 con 0x556c7f5ec2e0 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.783+0000 7f09c63b3640 1 --2- 172.21.15.94:0/1405051885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e2d60 0x556c7f5e9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:34.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.783+0000 7f09c6bb4640 1 --2- 172.21.15.94:0/1405051885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5ec2e0 0x556c7f5e9cb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0982f0 tx=0x7f09bc005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ab05e3cd736ce70 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.783+0000 7f09a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.783+0000 7f09a9ffb640 1 -- 172.21.15.94:0/1405051885 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc098550 con 0x556c7f5ec2e0 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.783+0000 7f09a9ffb640 1 -- 172.21.15.94:0/1405051885 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f09bc0986b0 con 0x556c7f5ec2e0 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.783+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1405051885 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c7f24a600 con 0x556c7f5ec2e0 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.783+0000 7f09a9ffb640 1 -- 172.21.15.94:0/1405051885 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09bc0b9250 con 0x556c7f5ec2e0 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.783+0000 7f09cb1e0740 10 client.26740 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.783+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1405051885 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556c7f548d80 con 0x556c7f5ec2e0 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.784+0000 7f09a9ffb640 1 -- 172.21.15.94:0/1405051885 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09bc0b93b0 con 0x556c7f5ec2e0 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.784+0000 7f09a9ffb640 1 -- 172.21.15.94:0/1405051885 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f09bc01f060 con 0x556c7f5ec2e0 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.784+0000 7f09cb1e0740 10 client.26740 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.784+0000 7f09cb1e0740 10 client.26740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.784+0000 7f09cb1e0740 20 client.26740 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.784+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/1405051885 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556c7f5fc5f0 0x556c7f5feab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.784+0000 7f09cb1e0740 4 client.26740 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:04:34.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.784+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1405051885 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c7f24a600 con 0x556c7f5fc5f0 2024-02-17T12:04:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.785+0000 7f09c63b3640 1 --2- 172.21.15.94:0/1405051885 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556c7f5fc5f0 0x556c7f5feab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.785+0000 7f09a9ffb640 10 client.26740 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.785+0000 7f09c63b3640 1 --2- 172.21.15.94:0/1405051885 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556c7f5fc5f0 0x556c7f5feab0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3678c5490bde1d46 server_cookie=bdc894e3802e441f in_seq=0 out_seq=0 2024-02-17T12:04:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.786+0000 7f09a9ffb640 1 -- 172.21.15.94:0/1405051885 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556c7f24a600 con 0x556c7f5fc5f0 2024-02-17T12:04:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.786+0000 7f09a9ffb640 10 client.26740 handle_command_reply: tid=0 2024-02-17T12:04:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.786+0000 7f09cb1e0740 1 client.26740 shutdown 2024-02-17T12:04:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.786+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1405051885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5ec2e0 msgr2=0x556c7f5e9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:34.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.786+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/1405051885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5ec2e0 0x556c7f5e9cb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0982f0 tx=0x7f09bc005730 comp rx=0 tx=0).stop 2024-02-17T12:04:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.786+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1405051885 shutdown_connections 2024-02-17T12:04:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.786+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/1405051885 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556c7f5fc5f0 0x556c7f5feab0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.786+0000 7f09a9ffb640 0 client.26740 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.786+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/1405051885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5ec2e0 0x556c7f5e9cb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.786+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/1405051885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.786+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/1405051885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e2d60 0x556c7f5e9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.786+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1405051885 >> 172.21.15.94:0/1405051885 conn(0x556c7f5aa550 msgr2=0x556c7f5df7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.787+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1405051885 shutdown_connections 2024-02-17T12:04:34.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.787+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1405051885 wait complete. 2024-02-17T12:04:34.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.788+0000 7f09cb1e0740 20 client.26740 trim_cache size 0 max 16384 2024-02-17T12:04:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.816+0000 7f09cb1e0740 1 Processor -- start 2024-02-17T12:04:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.816+0000 7f09cb1e0740 1 -- start start 2024-02-17T12:04:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.816+0000 7f09cb1e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e2d60 0x556c7f5eb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09cb1e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09cb1e0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5ec2e0 0x556c7f5e8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09cb1e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7f5e4e30 con 0x556c7f5e2d60 2024-02-17T12:04:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09c6bb4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5ec2e0 0x556c7f5e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09c6bb4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5ec2e0 0x556c7f5e8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44728/0 (socket says 172.21.15.94:44728) 2024-02-17T12:04:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09c6bb4640 1 -- 172.21.15.94:0/1524950798 learned_addr learned my addr 172.21.15.94:0/1524950798 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09cb1e0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7f5e9020 con 0x556c7f5ec2e0 2024-02-17T12:04:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1524950798 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7f5fee40 con 0x556c7f5e3ad0 2024-02-17T12:04:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09c5bb2640 1 --2- 172.21.15.94:0/1524950798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09c5bb2640 1 -- 172.21.15.94:0/1524950798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5ec2e0 msgr2=0x556c7f5e8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09c5bb2640 1 --2- 172.21.15.94:0/1524950798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5ec2e0 0x556c7f5e8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09c5bb2640 1 -- 172.21.15.94:0/1524950798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e2d60 msgr2=0x556c7f5eb960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:04:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09c5bb2640 1 --2- 172.21.15.94:0/1524950798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e2d60 0x556c7f5eb960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.817+0000 7f09c5bb2640 1 -- 172.21.15.94:0/1524950798 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c7f4e57b0 con 0x556c7f5e3ad0 2024-02-17T12:04:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.818+0000 7f09c5bb2640 1 --2- 172.21.15.94:0/1524950798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e7b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009230 tx=0x7f09b403e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd20c4d790368fb1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.821+0000 7f09abfff640 1 -- 172.21.15.94:0/1524950798 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4004240 con 0x556c7f5e3ad0 2024-02-17T12:04:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.821+0000 7f09abfff640 1 -- 172.21.15.94:0/1524950798 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09b402cbb0 con 0x556c7f5e3ad0 2024-02-17T12:04:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.821+0000 7f09abfff640 1 -- 172.21.15.94:0/1524950798 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b4003900 con 0x556c7f5e3ad0 2024-02-17T12:04:34.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.822+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1524950798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5e3ad0 msgr2=0x556c7f5e7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.822+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/1524950798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e7b00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009230 tx=0x7f09b403e000 comp rx=0 tx=0).stop 2024-02-17T12:04:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.822+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1524950798 shutdown_connections 2024-02-17T12:04:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.822+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/1524950798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5ec2e0 0x556c7f5e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.822+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/1524950798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5e3ad0 0x556c7f5e7b00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.822+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/1524950798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e2d60 0x556c7f5eb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.822+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1524950798 >> 172.21.15.94:0/1524950798 conn(0x556c7f5aa550 msgr2=0x556c7f5f8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:34.832 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.830+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1524950798 shutdown_connections 2024-02-17T12:04:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.834+0000 7f09cb1e0740 1 -- 172.21.15.94:0/1524950798 wait complete. 2024-02-17T12:04:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.834+0000 7f09cb1e0740 1 -- start start 2024-02-17T12:04:34.834 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.834+0000 7f09cb1e0740 1 Processor -- start 2024-02-17T12:04:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.839+0000 7f09cb1e0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e2d60 0x556c7f5f3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.839+0000 7f09cb1e0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e3ad0 0x556c7f5f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.839+0000 7f09cb1e0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5ec2e0 0x556c7f5f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.839+0000 7f09cb1e0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7f5f2540 con 0x556c7f5e3ad0 2024-02-17T12:04:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.839+0000 7f09cb1e0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7f5f26b0 con 0x556c7f5e2d60 2024-02-17T12:04:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.839+0000 7f09cb1e0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c7f652ba0 con 0x556c7f5ec2e0 2024-02-17T12:04:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.839+0000 7f09cb1e0740 10 client.0 _ll_register_callbacks cb 0x7f09b8009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.840+0000 7f09c63b3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e2d60 0x556c7f5f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.840+0000 7f09c63b3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e2d60 0x556c7f5f3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44738/0 (socket says 172.21.15.94:44738) 2024-02-17T12:04:34.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.840+0000 7f09c63b3640 1 -- 172.21.15.94:0/2789060954 learned_addr learned my addr 172.21.15.94:0/2789060954 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.842+0000 7f09c6bb4640 1 --2- 172.21.15.94:0/2789060954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5ec2e0 0x556c7f5f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.842 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.842+0000 7f09c5bb2640 1 --2- 172.21.15.94:0/2789060954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e3ad0 0x556c7f5f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.843+0000 7f09c6bb4640 1 -- 172.21.15.94:0/2789060954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e2d60 msgr2=0x556c7f5f3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.843+0000 7f09c6bb4640 1 --2- 172.21.15.94:0/2789060954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e2d60 0x556c7f5f3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.843+0000 7f09c6bb4640 1 -- 172.21.15.94:0/2789060954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e3ad0 msgr2=0x556c7f5f18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.843+0000 7f09c6bb4640 1 --2- 172.21.15.94:0/2789060954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e3ad0 0x556c7f5f18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.843+0000 7f09c6bb4640 1 -- 172.21.15.94:0/2789060954 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c0069060 con 0x556c7f5ec2e0 2024-02-17T12:04:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.843+0000 7f09c6bb4640 1 --2- 172.21.15.94:0/2789060954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5ec2e0 0x556c7f5f1e40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00c6a0 tx=0x7f09bc09f050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4c17a8a21a5c6ac server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.845+0000 7f09a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.845+0000 7f09a9ffb640 1 -- 172.21.15.94:0/2789060954 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00cec0 con 0x556c7f5ec2e0 2024-02-17T12:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.846+0000 7f09cb1e0740 1 -- 172.21.15.94:0/2789060954 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09b8004d20 con 0x556c7f5ec2e0 2024-02-17T12:04:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.846+0000 7f09a9ffb640 1 -- 172.21.15.94:0/2789060954 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f09bc09c5e0 con 0x556c7f5ec2e0 2024-02-17T12:04:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.847+0000 7f09a9ffb640 1 -- 172.21.15.94:0/2789060954 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09bc09b840 con 0x556c7f5ec2e0 2024-02-17T12:04:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.850+0000 7f09cb1e0740 10 client.26746 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.850+0000 7f09cb1e0740 1 -- 172.21.15.94:0/2789060954 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556c7f50b540 con 0x556c7f5ec2e0 2024-02-17T12:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.854+0000 7f09a9ffb640 1 -- 172.21.15.94:0/2789060954 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09b8004d20 con 0x556c7f5ec2e0 2024-02-17T12:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.854+0000 7f09a9ffb640 1 -- 172.21.15.94:0/2789060954 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f09bc083070 con 0x556c7f5ec2e0 2024-02-17T12:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.854+0000 7f09cb1e0740 10 client.26746 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.854+0000 7f09cb1e0740 10 client.26746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.854+0000 7f09cb1e0740 20 client.26746 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.854+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/2789060954 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556c7f653e20 0x556c7f656240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.854+0000 7f09cb1e0740 4 client.26746 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:04:34.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.854+0000 7f09cb1e0740 1 -- 172.21.15.94:0/2789060954 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556c7f29b1b0 con 0x556c7f653e20 2024-02-17T12:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.855+0000 7f09c63b3640 1 --2- 172.21.15.94:0/2789060954 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556c7f653e20 0x556c7f656240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.856+0000 7f09c63b3640 1 --2- 172.21.15.94:0/2789060954 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556c7f653e20 0x556c7f656240 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2479fdf6f839771d server_cookie=b3f506f0e7641a72 in_seq=0 out_seq=0 2024-02-17T12:04:34.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.856+0000 7f09a9ffb640 10 client.26746 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.907+0000 7f09a9ffb640 1 -- 172.21.15.94:0/2789060954 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556c7f29b1b0 con 0x556c7f653e20 2024-02-17T12:04:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.908+0000 7f09a9ffb640 10 client.26746 handle_command_reply: tid=0 2024-02-17T12:04:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.908+0000 7f09cb1e0740 1 client.26746 shutdown 2024-02-17T12:04:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.908+0000 7f09cb1e0740 1 -- 172.21.15.94:0/2789060954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5ec2e0 msgr2=0x556c7f5f1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.908+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/2789060954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5ec2e0 0x556c7f5f1e40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00c6a0 tx=0x7f09bc09f050 comp rx=0 tx=0).stop 2024-02-17T12:04:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.909+0000 7f09cb1e0740 1 -- 172.21.15.94:0/2789060954 shutdown_connections 2024-02-17T12:04:34.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.909+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/2789060954 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556c7f653e20 0x556c7f656240 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.909+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/2789060954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556c7f5ec2e0 0x556c7f5f1e40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.909+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/2789060954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556c7f5e3ad0 0x556c7f5f18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.909+0000 7f09cb1e0740 1 --2- 172.21.15.94:0/2789060954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556c7f5e2d60 0x556c7f5f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.909+0000 7f09cb1e0740 1 -- 172.21.15.94:0/2789060954 >> 172.21.15.94:0/2789060954 conn(0x556c7f5aa550 msgr2=0x556c7f5e7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.909+0000 7f09a9ffb640 0 client.26746 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.909+0000 7f09cb1e0740 1 -- 172.21.15.94:0/2789060954 shutdown_connections 2024-02-17T12:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.909+0000 7f09cb1e0740 1 -- 172.21.15.94:0/2789060954 wait complete. 2024-02-17T12:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.909+0000 7f09cb1e0740 20 client.26746 trim_cache size 0 max 16384 2024-02-17T12:04:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:04:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:04:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:04:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:04:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.910+0000 7f09c8617640 1 -- 172.21.15.94:0/490765078 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09ac0a2350 msgr2=0x7f09ac0a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.910+0000 7f09c8617640 1 --2- 172.21.15.94:0/490765078 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09ac0a2350 0x7f09ac0a4810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f09c00727e0 tx=0x7f09b4000bc0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.910+0000 7f09c8617640 1 -- 172.21.15.94:0/490765078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c010cb60 msgr2=0x7f09c01a3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.910+0000 7f09c8617640 1 --2- 172.21.15.94:0/490765078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c010cb60 0x7f09c01a3760 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00e970 tx=0x7f09bc00ed70 comp rx=0 tx=0).stop 2024-02-17T12:04:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.910+0000 7f09c8617640 1 -- 172.21.15.94:0/490765078 shutdown_connections 2024-02-17T12:04:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.910+0000 7f09c8617640 1 --2- 172.21.15.94:0/490765078 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09ac0a2350 0x7f09ac0a4810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.910+0000 7f09c8617640 1 --2- 172.21.15.94:0/490765078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09c010cb60 0x7f09c01a3760 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.910+0000 7f09c8617640 1 --2- 172.21.15.94:0/490765078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09c0108b20 0x7f09c019f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.910+0000 7f09c8617640 1 --2- 172.21.15.94:0/490765078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09c0108180 0x7f09c019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.910+0000 7f09c8617640 1 -- 172.21.15.94:0/490765078 >> 172.21.15.94:0/490765078 conn(0x7f09c006c6b0 msgr2=0x7f09c0070970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.910+0000 7f09c8617640 1 -- 172.21.15.94:0/490765078 shutdown_connections 2024-02-17T12:04:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:34.910+0000 7f09c8617640 1 -- 172.21.15.94:0/490765078 wait complete. 2024-02-17T12:04:35.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 375.0 s, 2446.2 tps, lat 40.577 ms stddev 49.567 2024-02-17T12:04:36.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:35 smithi094 ceph-mon[32499]: pgmap v198: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 34 MiB/s wr, 1.99k op/s 2024-02-17T12:04:36.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:35 smithi189 ceph-mon[36588]: pgmap v198: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 34 MiB/s wr, 1.99k op/s 2024-02-17T12:04:36.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:35 smithi125 ceph-mon[36819]: pgmap v198: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 34 MiB/s wr, 1.99k op/s 2024-02-17T12:04:38.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:37 smithi094 ceph-mon[32499]: pgmap v199: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 30 MiB/s wr, 1.78k op/s 2024-02-17T12:04:38.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:37 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:04:38.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:37 smithi189 ceph-mon[36588]: pgmap v199: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 30 MiB/s wr, 1.78k op/s 2024-02-17T12:04:38.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:37 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:04:38.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:37 smithi125 ceph-mon[36819]: pgmap v199: 97 pgs: 97 active+clean; 4.8 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 30 MiB/s wr, 1.78k op/s 2024-02-17T12:04:38.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:37 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:04:40.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:39 smithi094 ceph-mon[32499]: pgmap v200: 97 pgs: 97 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 34 MiB/s wr, 2.18k op/s 2024-02-17T12:04:40.237 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:39 smithi125 ceph-mon[36819]: pgmap v200: 97 pgs: 97 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 34 MiB/s wr, 2.18k op/s 2024-02-17T12:04:40.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 380.0 s, 3211.0 tps, lat 31.415 ms stddev 40.233 2024-02-17T12:04:40.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:39 smithi189 ceph-mon[36588]: pgmap v200: 97 pgs: 97 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 34 MiB/s wr, 2.18k op/s 2024-02-17T12:04:42.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:41 smithi094 ceph-mon[32499]: pgmap v201: 97 pgs: 97 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 27 MiB/s wr, 1.80k op/s 2024-02-17T12:04:42.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:41 smithi189 ceph-mon[36588]: pgmap v201: 97 pgs: 97 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 27 MiB/s wr, 1.80k op/s 2024-02-17T12:04:42.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:41 smithi125 ceph-mon[36819]: pgmap v201: 97 pgs: 97 active+clean; 4.8 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 27 MiB/s wr, 1.80k op/s 2024-02-17T12:04:44.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:43 smithi094 ceph-mon[32499]: pgmap v202: 97 pgs: 97 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 29 MiB/s wr, 1.89k op/s 2024-02-17T12:04:44.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:43 smithi189 ceph-mon[36588]: pgmap v202: 97 pgs: 97 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 29 MiB/s wr, 1.89k op/s 2024-02-17T12:04:44.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:43 smithi125 ceph-mon[36819]: pgmap v202: 97 pgs: 97 active+clean; 4.8 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 29 MiB/s wr, 1.89k op/s 2024-02-17T12:04:45.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 385.0 s, 2745.0 tps, lat 36.186 ms stddev 40.167 2024-02-17T12:04:45.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:44 smithi189 ceph-mon[36588]: pgmap v203: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 25 MiB/s wr, 1.69k op/s 2024-02-17T12:04:45.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:44 smithi125 ceph-mon[36819]: pgmap v203: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 25 MiB/s wr, 1.69k op/s 2024-02-17T12:04:45.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:44 smithi094 ceph-mon[32499]: pgmap v203: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 25 MiB/s wr, 1.69k op/s 2024-02-17T12:04:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.041+0000 7fc797edf640 1 -- 172.21.15.94:0/1056430480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc79010f8c0 msgr2=0x7fc79010d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.041+0000 7fc797edf640 1 --2- 172.21.15.94:0/1056430480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc79010f8c0 0x7fc79010d4e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc788009ca0 tx=0x7fc78802c100 comp rx=0 tx=0).stop 2024-02-17T12:04:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.042+0000 7fc797edf640 1 -- 172.21.15.94:0/1056430480 shutdown_connections 2024-02-17T12:04:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.042+0000 7fc797edf640 1 --2- 172.21.15.94:0/1056430480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc79011a960 0x7fc79011cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.042+0000 7fc797edf640 1 --2- 172.21.15.94:0/1056430480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc79010fd30 0x7fc79010da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.042+0000 7fc797edf640 1 --2- 172.21.15.94:0/1056430480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc79010f8c0 0x7fc79010d4e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.042+0000 7fc797edf640 1 -- 172.21.15.94:0/1056430480 >> 172.21.15.94:0/1056430480 conn(0x7fc79006c220 msgr2=0x7fc79006c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.043+0000 7fc797edf640 1 -- 172.21.15.94:0/1056430480 shutdown_connections 2024-02-17T12:04:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.043+0000 7fc797edf640 1 -- 172.21.15.94:0/1056430480 wait complete. 2024-02-17T12:04:48.046 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.044+0000 7fc797edf640 1 Processor -- start 2024-02-17T12:04:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.044+0000 7fc797edf640 1 -- start start 2024-02-17T12:04:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.044+0000 7fc797edf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc79010f8c0 0x7fc7901a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.044+0000 7fc797edf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc79010fd30 0x7fc7901a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.044+0000 7fc797edf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc79011a960 0x7fc7901aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.044+0000 7fc797edf640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901ab4c0 con 0x7fc79010f8c0 2024-02-17T12:04:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.044+0000 7fc797edf640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901ab630 con 0x7fc79010fd30 2024-02-17T12:04:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.044+0000 7fc797edf640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901ab7f0 con 0x7fc79011a960 2024-02-17T12:04:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.044+0000 7fc795c7b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc79010f8c0 0x7fc7901a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.044+0000 7fc795c7b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc79010f8c0 0x7fc7901a6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55390/0 (socket says 172.21.15.94:55390) 2024-02-17T12:04:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.044+0000 7fc795c7b640 1 -- 172.21.15.94:0/1371338178 learned_addr learned my addr 172.21.15.94:0/1371338178 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.044+0000 7fc79547a640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc79010fd30 0x7fc7901a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.044+0000 7fc79647c640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc79011a960 0x7fc7901aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.045+0000 7fc79547a640 1 -- 172.21.15.94:0/1371338178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc79011a960 msgr2=0x7fc7901aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.045+0000 7fc79547a640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc79011a960 0x7fc7901aaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.045+0000 7fc79547a640 1 -- 172.21.15.94:0/1371338178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc79010f8c0 msgr2=0x7fc7901a6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.045+0000 7fc79547a640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc79010f8c0 0x7fc7901a6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.045+0000 7fc79547a640 1 -- 172.21.15.94:0/1371338178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc788009900 con 0x7fc79010fd30 2024-02-17T12:04:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.045+0000 7fc79547a640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc79010fd30 0x7fc7901a6b40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc78000b3c0 tx=0x7fc78000b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.046+0000 7fc786ffd640 1 -- 172.21.15.94:0/1371338178 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780011830 con 0x7fc79010fd30 2024-02-17T12:04:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.046+0000 7fc786ffd640 1 -- 172.21.15.94:0/1371338178 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc780011e70 con 0x7fc79010fd30 2024-02-17T12:04:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.046+0000 7fc786ffd640 1 -- 172.21.15.94:0/1371338178 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78000f2e0 con 0x7fc79010fd30 2024-02-17T12:04:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.046+0000 7fc797edf640 1 -- 172.21.15.94:0/1371338178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7901abad0 con 0x7fc79010fd30 2024-02-17T12:04:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.046+0000 7fc797edf640 1 -- 172.21.15.94:0/1371338178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7901b3350 con 0x7fc79010fd30 2024-02-17T12:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.048+0000 7fc786ffd640 1 -- 172.21.15.94:0/1371338178 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc780010610 con 0x7fc79010fd30 2024-02-17T12:04:48.049 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.049+0000 7fc79ab8f740 1 Processor -- start 2024-02-17T12:04:48.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.050+0000 7fc786ffd640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc77c0a24b0 0x7fc77c0a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.050+0000 7fc786ffd640 1 -- 172.21.15.94:0/1371338178 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc7800cd610 con 0x7fc79010fd30 2024-02-17T12:04:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc79647c640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc77c0a24b0 0x7fc77c0a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc79ab8f740 1 -- start start 2024-02-17T12:04:48.051 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc79ab8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e834151ad0 0x55e834150820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc79ab8f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834150d60 0x55e8341511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc79ab8f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e83415a2e0 0x55e83415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc79ab8f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8340bf9f0 con 0x55e83415a2e0 2024-02-17T12:04:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc79ab8f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8340bcf20 con 0x55e834150d60 2024-02-17T12:04:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc79ab8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8340b8700 con 0x55e834151ad0 2024-02-17T12:04:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc79547a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834150d60 0x55e8341511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc79647c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e83415a2e0 0x55e83415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc79547a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834150d60 0x55e8341511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:54808/0 (socket says 172.21.15.94:54808) 2024-02-17T12:04:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc79547a640 1 -- 172.21.15.94:0/3930067555 learned_addr learned my addr 172.21.15.94:0/3930067555 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc795c7b640 1 --2- 172.21.15.94:0/3930067555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e834151ad0 0x55e834150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.051+0000 7fc79647c640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc77c0a24b0 0x7fc77c0a4970 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc79006a590 tx=0x7fc78c005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.052+0000 7fc79647c640 1 -- 172.21.15.94:0/3930067555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e834151ad0 msgr2=0x55e834150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.052+0000 7fc79647c640 1 --2- 172.21.15.94:0/3930067555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e834151ad0 0x55e834150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.052+0000 7fc79647c640 1 -- 172.21.15.94:0/3930067555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834150d60 msgr2=0x55e8341511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.052+0000 7fc79647c640 1 --2- 172.21.15.94:0/3930067555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834150d60 0x55e8341511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.052+0000 7fc79647c640 1 -- 172.21.15.94:0/3930067555 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e83404e7b0 con 0x55e83415a2e0 2024-02-17T12:04:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.052+0000 7fc79547a640 1 --2- 172.21.15.94:0/3930067555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834150d60 0x55e8341511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.052+0000 7fc79647c640 1 --2- 172.21.15.94:0/3930067555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e83415a2e0 0x55e83415c6d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc78c011d50 tx=0x7fc78c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12254396505c64a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.052+0000 7fc77bfff640 1 -- 172.21.15.94:0/3930067555 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c004230 con 0x55e83415a2e0 2024-02-17T12:04:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.052+0000 7fc77bfff640 1 -- 172.21.15.94:0/3930067555 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc78c021070 con 0x55e83415a2e0 2024-02-17T12:04:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.052+0000 7fc77bfff640 1 -- 172.21.15.94:0/3930067555 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c003c60 con 0x55e83415a2e0 2024-02-17T12:04:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.053+0000 7fc79ab8f740 1 -- 172.21.15.94:0/3930067555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e83415a2e0 msgr2=0x55e83415c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.053+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/3930067555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e83415a2e0 0x55e83415c6d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc78c011d50 tx=0x7fc78c014000 comp rx=0 tx=0).stop 2024-02-17T12:04:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.053+0000 7fc79ab8f740 1 -- 172.21.15.94:0/3930067555 shutdown_connections 2024-02-17T12:04:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.053+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/3930067555 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e83415a2e0 0x55e83415c6d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.053+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/3930067555 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834150d60 0x55e8341511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.053+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/3930067555 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e834151ad0 0x55e834150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.053+0000 7fc79ab8f740 1 -- 172.21.15.94:0/3930067555 >> 172.21.15.94:0/3930067555 conn(0x55e834118550 msgr2=0x55e83414ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.054+0000 7fc79ab8f740 1 -- 172.21.15.94:0/3930067555 shutdown_connections 2024-02-17T12:04:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.054+0000 7fc79ab8f740 1 -- 172.21.15.94:0/3930067555 wait complete. 2024-02-17T12:04:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.054+0000 7fc79ab8f740 1 -- start start 2024-02-17T12:04:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.054+0000 7fc79ab8f740 1 Processor -- start 2024-02-17T12:04:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79ab8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e834150d60 0x55e8341572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79ab8f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834151ad0 0x55e834157830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79ab8f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e83415a2e0 0x55e834157d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79ab8f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e83405dbf0 con 0x55e83415a2e0 2024-02-17T12:04:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79ab8f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e833d8c250 con 0x55e834151ad0 2024-02-17T12:04:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc795c7b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e834150d60 0x55e8341572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79647c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e83415a2e0 0x55e834157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79ab8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e83403e200 con 0x55e834150d60 2024-02-17T12:04:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc795c7b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e834150d60 0x55e8341572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:51244/0 (socket says 172.21.15.94:51244) 2024-02-17T12:04:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc795c7b640 1 -- 172.21.15.94:0/4031268120 learned_addr learned my addr 172.21.15.94:0/4031268120 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79547a640 1 --2- 172.21.15.94:0/4031268120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834151ad0 0x55e834157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79ab8f740 10 client.0 _ll_register_callbacks cb 0x55e8340acd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79547a640 1 -- 172.21.15.94:0/4031268120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e834150d60 msgr2=0x55e8341572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79547a640 1 --2- 172.21.15.94:0/4031268120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e834150d60 0x55e8341572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79547a640 1 -- 172.21.15.94:0/4031268120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e83415a2e0 msgr2=0x55e834157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79547a640 1 --2- 172.21.15.94:0/4031268120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e83415a2e0 0x55e834157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79547a640 1 -- 172.21.15.94:0/4031268120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78c017070 con 0x55e834151ad0 2024-02-17T12:04:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79647c640 1 --2- 172.21.15.94:0/4031268120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e83415a2e0 0x55e834157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc79547a640 1 --2- 172.21.15.94:0/4031268120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834151ad0 0x55e834157830 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc780099da0 tx=0x7fc78009a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74e336a00ea36a64 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc7797fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.055+0000 7fc7797fa640 1 -- 172.21.15.94:0/4031268120 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78009e1f0 con 0x55e834151ad0 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc79ab8f740 1 -- 172.21.15.94:0/4031268120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e833db8600 con 0x55e834151ad0 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc7797fa640 1 -- 172.21.15.94:0/4031268120 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc7800a2840 con 0x55e834151ad0 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc7797fa640 1 -- 172.21.15.94:0/4031268120 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7800caf70 con 0x55e834151ad0 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc79ab8f740 10 client.26717 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc79ab8f740 1 -- 172.21.15.94:0/4031268120 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e8340b6d80 con 0x55e834151ad0 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc7797fa640 1 -- 172.21.15.94:0/4031268120 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e833db8600 con 0x55e834151ad0 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc7797fa640 1 -- 172.21.15.94:0/4031268120 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc7800cd110 con 0x55e834151ad0 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc79ab8f740 10 client.26717 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc79ab8f740 10 client.26717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc79ab8f740 20 client.26717 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/4031268120 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e83416a690 0x55e83416cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc79ab8f740 4 client.26717 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:04:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc79ab8f740 1 -- 172.21.15.94:0/4031268120 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e833e091b0 con 0x55e83416a690 2024-02-17T12:04:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.056+0000 7fc795c7b640 1 --2- 172.21.15.94:0/4031268120 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e83416a690 0x55e83416cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.057+0000 7fc7797fa640 10 client.26717 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.057+0000 7fc795c7b640 1 --2- 172.21.15.94:0/4031268120 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e83416a690 0x55e83416cab0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a6a674c15c0bb5 server_cookie=8c7c823352a05909 in_seq=0 out_seq=0 2024-02-17T12:04:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.057+0000 7fc7797fa640 1 -- 172.21.15.94:0/4031268120 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e833e091b0 con 0x55e83416a690 2024-02-17T12:04:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.058+0000 7fc7797fa640 10 client.26717 handle_command_reply: tid=0 2024-02-17T12:04:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.058+0000 7fc79ab8f740 1 client.26717 shutdown 2024-02-17T12:04:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.058+0000 7fc79ab8f740 1 -- 172.21.15.94:0/4031268120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834151ad0 msgr2=0x55e834157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.058+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/4031268120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834151ad0 0x55e834157830 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc780099da0 tx=0x7fc78009a7a0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.058+0000 7fc79ab8f740 1 -- 172.21.15.94:0/4031268120 shutdown_connections 2024-02-17T12:04:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.058+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/4031268120 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e83416a690 0x55e83416cab0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.058+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/4031268120 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e83415a2e0 0x55e834157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.058+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/4031268120 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834151ad0 0x55e834157830 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.058+0000 7fc7797fa640 0 client.26717 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.058+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/4031268120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e834150d60 0x55e8341572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.058+0000 7fc79ab8f740 1 -- 172.21.15.94:0/4031268120 >> 172.21.15.94:0/4031268120 conn(0x55e834118550 msgr2=0x55e83414d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.058+0000 7fc79ab8f740 1 -- 172.21.15.94:0/4031268120 shutdown_connections 2024-02-17T12:04:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.059+0000 7fc79ab8f740 1 -- 172.21.15.94:0/4031268120 wait complete. 2024-02-17T12:04:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.059+0000 7fc79ab8f740 20 client.26717 trim_cache size 0 max 16384 2024-02-17T12:04:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.074+0000 7fc79ab8f740 1 Processor -- start 2024-02-17T12:04:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.074+0000 7fc79ab8f740 1 -- start start 2024-02-17T12:04:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79ab8f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834150d60 0x55e834159a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79ab8f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e834151ad0 0x55e834157b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79ab8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e83415a2e0 0x55e834155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79ab8f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e83415cef0 con 0x55e834151ad0 2024-02-17T12:04:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79ab8f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e834150030 con 0x55e834150d60 2024-02-17T12:04:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79ab8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e834155840 con 0x55e83415a2e0 2024-02-17T12:04:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79647c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e83415a2e0 0x55e834155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79647c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e83415a2e0 0x55e834155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:51260/0 (socket says 172.21.15.94:51260) 2024-02-17T12:04:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79647c640 1 -- 172.21.15.94:0/859744888 learned_addr learned my addr 172.21.15.94:0/859744888 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79647c640 1 -- 172.21.15.94:0/859744888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834150d60 msgr2=0x55e834159a20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:04:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79647c640 1 --2- 172.21.15.94:0/859744888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834150d60 0x55e834159a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79647c640 1 -- 172.21.15.94:0/859744888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e834151ad0 msgr2=0x55e834157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79647c640 1 --2- 172.21.15.94:0/859744888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e834151ad0 0x55e834157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.075+0000 7fc79647c640 1 -- 172.21.15.94:0/859744888 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8340537b0 con 0x55e83415a2e0 2024-02-17T12:04:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.076+0000 7fc79647c640 1 --2- 172.21.15.94:0/859744888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e83415a2e0 0x55e834155bc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc78c011d20 tx=0x7fc78c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d447d9fe216861b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.076+0000 7fc77b7fe640 1 -- 172.21.15.94:0/859744888 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0036f0 con 0x55e83415a2e0 2024-02-17T12:04:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.076+0000 7fc77b7fe640 1 -- 172.21.15.94:0/859744888 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc78c003850 con 0x55e83415a2e0 2024-02-17T12:04:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.076+0000 7fc77b7fe640 1 -- 172.21.15.94:0/859744888 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c01d8f0 con 0x55e83415a2e0 2024-02-17T12:04:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.077+0000 7fc79ab8f740 1 -- 172.21.15.94:0/859744888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e83415a2e0 msgr2=0x55e834155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.077+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/859744888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e83415a2e0 0x55e834155bc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc78c011d20 tx=0x7fc78c005850 comp rx=0 tx=0).stop 2024-02-17T12:04:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.077+0000 7fc79ab8f740 1 -- 172.21.15.94:0/859744888 shutdown_connections 2024-02-17T12:04:48.081 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.077+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/859744888 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e83415a2e0 0x55e834155bc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.077+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/859744888 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e834151ad0 0x55e834157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.077+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/859744888 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834150d60 0x55e834159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.077+0000 7fc79ab8f740 1 -- 172.21.15.94:0/859744888 >> 172.21.15.94:0/859744888 conn(0x55e834118550 msgr2=0x55e834171450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.077+0000 7fc79ab8f740 1 -- 172.21.15.94:0/859744888 shutdown_connections 2024-02-17T12:04:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.077+0000 7fc79ab8f740 1 -- 172.21.15.94:0/859744888 wait complete. 2024-02-17T12:04:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.077+0000 7fc79ab8f740 1 -- start start 2024-02-17T12:04:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.077+0000 7fc79ab8f740 1 Processor -- start 2024-02-17T12:04:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.078+0000 7fc79ab8f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e834150d60 0x55e834161230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.078+0000 7fc79ab8f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834151ad0 0x55e83415f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.078+0000 7fc79ab8f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e83415a2e0 0x55e83415fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.078+0000 7fc79ab8f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e834160050 con 0x55e834150d60 2024-02-17T12:04:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.078+0000 7fc79ab8f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8341601c0 con 0x55e834151ad0 2024-02-17T12:04:48.082 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.078+0000 7fc79ab8f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e834160380 con 0x55e83415a2e0 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.078+0000 7fc79ab8f740 10 client.0 _ll_register_callbacks cb 0x7fc78c0099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.078+0000 7fc79547a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834151ad0 0x55e83415f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.078+0000 7fc79547a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834151ad0 0x55e83415f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:54814/0 (socket says 172.21.15.94:54814) 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.078+0000 7fc79547a640 1 -- 172.21.15.94:0/1650755484 learned_addr learned my addr 172.21.15.94:0/1650755484 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.079+0000 7fc79647c640 1 --2- 172.21.15.94:0/1650755484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e83415a2e0 0x55e83415fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.079+0000 7fc79547a640 1 -- 172.21.15.94:0/1650755484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e83415a2e0 msgr2=0x55e83415fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.079+0000 7fc79547a640 1 --2- 172.21.15.94:0/1650755484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e83415a2e0 0x55e83415fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.079+0000 7fc79547a640 1 -- 172.21.15.94:0/1650755484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e834150d60 msgr2=0x55e834161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.079+0000 7fc79547a640 1 --2- 172.21.15.94:0/1650755484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e834150d60 0x55e834161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.079+0000 7fc79547a640 1 -- 172.21.15.94:0/1650755484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e83404e7b0 con 0x55e834151ad0 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.079+0000 7fc79647c640 1 --2- 172.21.15.94:0/1650755484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e83415a2e0 0x55e83415fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.079+0000 7fc79547a640 1 --2- 172.21.15.94:0/1650755484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834151ad0 0x55e83415f5d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc78001a000 tx=0x7fc7800969c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b570065245a0d2a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.079+0000 7fc7797fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:04:48.083 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.079+0000 7fc7797fa640 1 -- 172.21.15.94:0/1650755484 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78000d040 con 0x55e834151ad0 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.079+0000 7fc7797fa640 1 -- 172.21.15.94:0/1650755484 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fc780009180 con 0x55e834151ad0 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.079+0000 7fc79ab8f740 1 -- 172.21.15.94:0/1650755484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc78c021070 con 0x55e834151ad0 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc7797fa640 1 -- 172.21.15.94:0/1650755484 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc780099be0 con 0x55e834151ad0 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc79ab8f740 10 client.26720 fetch_fsmap learned FSMap version 25 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc79ab8f740 1 -- 172.21.15.94:0/1650755484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e834079540 con 0x55e834151ad0 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc7797fa640 1 -- 172.21.15.94:0/1650755484 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc78c021070 con 0x55e834151ad0 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc7797fa640 1 -- 172.21.15.94:0/1650755484 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc7800caec0 con 0x55e834151ad0 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc79ab8f740 10 client.26720 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc79ab8f740 10 client.26720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc79ab8f740 20 client.26720 populate_metadata read hostname 'smithi094' 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/1650755484 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e8341c1d90 0x55e8341c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc79ab8f740 4 client.26720 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc79ab8f740 1 -- 172.21.15.94:0/1650755484 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e833df5550 con 0x55e8341c1d90 2024-02-17T12:04:48.084 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc795c7b640 1 --2- 172.21.15.94:0/1650755484 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e8341c1d90 0x55e8341c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:04:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc7797fa640 10 client.26720 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:48.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.080+0000 7fc795c7b640 1 --2- 172.21.15.94:0/1650755484 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e8341c1d90 0x55e8341c41b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fdac3e6e77c9a92 server_cookie=5c7f4dfde1b6eae6 in_seq=0 out_seq=0 2024-02-17T12:04:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.099+0000 7fc7797fa640 1 -- 172.21.15.94:0/1650755484 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e833df5550 con 0x55e8341c1d90 2024-02-17T12:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.099+0000 7fc7797fa640 10 client.26720 handle_command_reply: tid=0 2024-02-17T12:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.099+0000 7fc79ab8f740 1 client.26720 shutdown 2024-02-17T12:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.099+0000 7fc79ab8f740 1 -- 172.21.15.94:0/1650755484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834151ad0 msgr2=0x55e83415f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.099+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/1650755484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834151ad0 0x55e83415f5d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc78001a000 tx=0x7fc7800969c0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.099+0000 7fc79ab8f740 1 -- 172.21.15.94:0/1650755484 shutdown_connections 2024-02-17T12:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.099+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/1650755484 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e8341c1d90 0x55e8341c41b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.099+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/1650755484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e83415a2e0 0x55e83415fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.099+0000 7fc7797fa640 0 client.26720 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.099+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/1650755484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e834151ad0 0x55e83415f5d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.099+0000 7fc79ab8f740 1 --2- 172.21.15.94:0/1650755484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e834150d60 0x55e834161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.099+0000 7fc79ab8f740 1 -- 172.21.15.94:0/1650755484 >> 172.21.15.94:0/1650755484 conn(0x55e834118550 msgr2=0x55e834155b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc79ab8f740 1 -- 172.21.15.94:0/1650755484 shutdown_connections 2024-02-17T12:04:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc79ab8f740 1 -- 172.21.15.94:0/1650755484 wait complete. 2024-02-17T12:04:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc79ab8f740 20 client.26720 trim_cache size 0 max 16384 2024-02-17T12:04:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:04:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:04:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:04:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:04:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc797edf640 1 -- 172.21.15.94:0/1371338178 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc77c0a24b0 msgr2=0x7fc77c0a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc797edf640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc77c0a24b0 0x7fc77c0a4970 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc79006a590 tx=0x7fc78c005f90 comp rx=0 tx=0).stop 2024-02-17T12:04:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc797edf640 1 -- 172.21.15.94:0/1371338178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc79010fd30 msgr2=0x7fc7901a6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:04:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc797edf640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc79010fd30 0x7fc7901a6b40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc78000b3c0 tx=0x7fc78000b880 comp rx=0 tx=0).stop 2024-02-17T12:04:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc797edf640 1 -- 172.21.15.94:0/1371338178 shutdown_connections 2024-02-17T12:04:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc797edf640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc77c0a24b0 0x7fc77c0a4970 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc797edf640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc79011a960 0x7fc7901aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc797edf640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc79010fd30 0x7fc7901a6b40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc797edf640 1 --2- 172.21.15.94:0/1371338178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc79010f8c0 0x7fc7901a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc797edf640 1 -- 172.21.15.94:0/1371338178 >> 172.21.15.94:0/1371338178 conn(0x7fc79006c220 msgr2=0x7fc790119390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc797edf640 1 -- 172.21.15.94:0/1371338178 shutdown_connections 2024-02-17T12:04:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:04:48.100+0000 7fc797edf640 1 -- 172.21.15.94:0/1371338178 wait complete. 2024-02-17T12:04:48.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:47 smithi094 ceph-mon[32499]: pgmap v204: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 9.4 MiB/s wr, 698 op/s 2024-02-17T12:04:48.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:47 smithi189 ceph-mon[36588]: pgmap v204: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 9.4 MiB/s wr, 698 op/s 2024-02-17T12:04:48.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:47 smithi125 ceph-mon[36819]: pgmap v204: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 9.4 MiB/s wr, 698 op/s 2024-02-17T12:04:50.203 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:49 smithi189 ceph-mon[36588]: pgmap v205: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 994 op/s 2024-02-17T12:04:50.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:49 smithi094 ceph-mon[32499]: pgmap v205: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 994 op/s 2024-02-17T12:04:50.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 390.0 s, 2424.8 tps, lat 41.492 ms stddev 49.008 2024-02-17T12:04:50.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:49 smithi125 ceph-mon[36819]: pgmap v205: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 14 MiB/s wr, 994 op/s 2024-02-17T12:04:52.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:51 smithi094 ceph-mon[32499]: pgmap v206: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.7 MiB/s wr, 600 op/s 2024-02-17T12:04:52.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:51 smithi189 ceph-mon[36588]: pgmap v206: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.7 MiB/s wr, 600 op/s 2024-02-17T12:04:52.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:51 smithi125 ceph-mon[36819]: pgmap v206: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.7 MiB/s wr, 600 op/s 2024-02-17T12:04:54.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:53 smithi094 ceph-mon[32499]: pgmap v207: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.0 MiB/s wr, 639 op/s 2024-02-17T12:04:54.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:53 smithi189 ceph-mon[36588]: pgmap v207: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.0 MiB/s wr, 639 op/s 2024-02-17T12:04:54.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:53 smithi125 ceph-mon[36819]: pgmap v207: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.0 MiB/s wr, 639 op/s 2024-02-17T12:04:55.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 395.0 s, 2600.6 tps, lat 38.459 ms stddev 46.021 2024-02-17T12:04:56.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:55 smithi094 ceph-mon[32499]: pgmap v208: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.8 MiB/s wr, 782 op/s 2024-02-17T12:04:56.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:55 smithi189 ceph-mon[36588]: pgmap v208: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.8 MiB/s wr, 782 op/s 2024-02-17T12:04:56.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:55 smithi125 ceph-mon[36819]: pgmap v208: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 9.8 MiB/s wr, 782 op/s 2024-02-17T12:04:57.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:57 smithi189 ceph-mon[36588]: pgmap v209: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 7.1 MiB/s wr, 568 op/s 2024-02-17T12:04:57.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:57 smithi125 ceph-mon[36819]: pgmap v209: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 7.1 MiB/s wr, 568 op/s 2024-02-17T12:04:57.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:57 smithi094 ceph-mon[32499]: pgmap v209: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 7.1 MiB/s wr, 568 op/s 2024-02-17T12:05:00.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:04:59 smithi094 ceph-mon[32499]: pgmap v210: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 12 MiB/s wr, 909 op/s 2024-02-17T12:05:00.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 400.0 s, 2965.4 tps, lat 33.747 ms stddev 37.581 2024-02-17T12:05:00.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:04:59 smithi189 ceph-mon[36588]: pgmap v210: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 12 MiB/s wr, 909 op/s 2024-02-17T12:05:00.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:04:59 smithi125 ceph-mon[36819]: pgmap v210: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 12 MiB/s wr, 909 op/s 2024-02-17T12:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.200+0000 7f02cac4e640 1 -- 172.21.15.94:0/2115129520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c4068a70 msgr2=0x7f02c4068e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.200+0000 7f02cac4e640 1 --2- 172.21.15.94:0/2115129520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c4068a70 0x7f02c4068e50 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f02c0009930 tx=0x7f02c002bc30 comp rx=0 tx=0).stop 2024-02-17T12:05:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.200+0000 7f02cac4e640 1 -- 172.21.15.94:0/2115129520 shutdown_connections 2024-02-17T12:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.200+0000 7f02cac4e640 1 --2- 172.21.15.94:0/2115129520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02c41064a0 0x7f02c410e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.200+0000 7f02cac4e640 1 --2- 172.21.15.94:0/2115129520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c4069420 0x7f02c4105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.200+0000 7f02cac4e640 1 --2- 172.21.15.94:0/2115129520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c4068a70 0x7f02c4068e50 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.200+0000 7f02cac4e640 1 -- 172.21.15.94:0/2115129520 >> 172.21.15.94:0/2115129520 conn(0x7f02c4076ed0 msgr2=0x7f02c40772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.200+0000 7f02cac4e640 1 -- 172.21.15.94:0/2115129520 shutdown_connections 2024-02-17T12:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.200+0000 7f02cac4e640 1 -- 172.21.15.94:0/2115129520 wait complete. 2024-02-17T12:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02cac4e640 1 Processor -- start 2024-02-17T12:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02cac4e640 1 -- start start 2024-02-17T12:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02cac4e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02c4068a70 0x7f02c41003a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02cac4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c41064a0 0x7f02c41008e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02cac4e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c4100e20 0x7f02c41a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02cac4e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c41a5ec0 con 0x7f02c41064a0 2024-02-17T12:05:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02cac4e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c41a6030 con 0x7f02c4068a70 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02cac4e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c41a61f0 con 0x7f02c4100e20 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c41064a0 0x7f02c41008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c41064a0 0x7f02c41008e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42150/0 (socket says 172.21.15.94:42150) 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02bbfff640 1 -- 172.21.15.94:0/331976755 learned_addr learned my addr 172.21.15.94:0/331976755 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02c91eb640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c4100e20 0x7f02c41a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02c89ea640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02c4068a70 0x7f02c41003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02bbfff640 1 -- 172.21.15.94:0/331976755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c4100e20 msgr2=0x7f02c41a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02bbfff640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c4100e20 0x7f02c41a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02bbfff640 1 -- 172.21.15.94:0/331976755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02c4068a70 msgr2=0x7f02c41003a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02bbfff640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02c4068a70 0x7f02c41003a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02bbfff640 1 -- 172.21.15.94:0/331976755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c0009560 con 0x7f02c41064a0 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02c91eb640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c4100e20 0x7f02c41a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.201+0000 7f02c89ea640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02c4068a70 0x7f02c41003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:05:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.202+0000 7f02bbfff640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c41064a0 0x7f02c41008e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f02b400e9c0 tx=0x7f02b400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.202+0000 7f02b9ffb640 1 -- 172.21.15.94:0/331976755 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40096c0 con 0x7f02c41064a0 2024-02-17T12:05:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.202+0000 7f02b9ffb640 1 -- 172.21.15.94:0/331976755 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02b4005500 con 0x7f02c41064a0 2024-02-17T12:05:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.202+0000 7f02cac4e640 1 -- 172.21.15.94:0/331976755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c41a64d0 con 0x7f02c41064a0 2024-02-17T12:05:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.202+0000 7f02b9ffb640 1 -- 172.21.15.94:0/331976755 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4010430 con 0x7f02c41064a0 2024-02-17T12:05:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.202+0000 7f02cac4e640 1 -- 172.21.15.94:0/331976755 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c41a6990 con 0x7f02c41064a0 2024-02-17T12:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.204+0000 7f02b9ffb640 1 -- 172.21.15.94:0/331976755 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f02b4005020 con 0x7f02c41064a0 2024-02-17T12:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02cd824740 1 Processor -- start 2024-02-17T12:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02b9ffb640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02ac0a22d0 0x7f02ac0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02cd824740 1 -- start start 2024-02-17T12:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02c91eb640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02ac0a22d0 0x7f02ac0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02b9ffb640 1 -- 172.21.15.94:0/331976755 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f02b4014070 con 0x7f02c41064a0 2024-02-17T12:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02cd824740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9466ad0 0x55f8e9465820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02cd824740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9465d60 0x55f8e94661e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02cd824740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e946f2e0 0x55f8e94716d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02cd824740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8e93d49f0 con 0x55f8e946f2e0 2024-02-17T12:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02cd824740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8e93d1f20 con 0x55f8e9466ad0 2024-02-17T12:05:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02cd824740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8e93cd700 con 0x55f8e9465d60 2024-02-17T12:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02c89ea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9466ad0 0x55f8e9465820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02c89ea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9466ad0 0x55f8e9465820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:35440/0 (socket says 172.21.15.94:35440) 2024-02-17T12:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02c91eb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e946f2e0 0x55f8e94716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02c89ea640 1 -- 172.21.15.94:0/418606615 learned_addr learned my addr 172.21.15.94:0/418606615 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02bbfff640 1 --2- 172.21.15.94:0/418606615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9465d60 0x55f8e94661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02c91eb640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02ac0a22d0 0x7f02ac0a4790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009fd0 tx=0x7f02bc011b70 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02c89ea640 1 -- 172.21.15.94:0/418606615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9465d60 msgr2=0x55f8e94661e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02c89ea640 1 --2- 172.21.15.94:0/418606615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9465d60 0x55f8e94661e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02c89ea640 1 -- 172.21.15.94:0/418606615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e946f2e0 msgr2=0x55f8e94716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02c89ea640 1 --2- 172.21.15.94:0/418606615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e946f2e0 0x55f8e94716d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02c89ea640 1 -- 172.21.15.94:0/418606615 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8e93637b0 con 0x55f8e9466ad0 2024-02-17T12:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.205+0000 7f02c91eb640 1 --2- 172.21.15.94:0/418606615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e946f2e0 0x55f8e94716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:01.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.206+0000 7f02c89ea640 1 --2- 172.21.15.94:0/418606615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9466ad0 0x55f8e9465820 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f02c0000c00 tx=0x7f02c002c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5999ba1490a9a9ee server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.206+0000 7f02aaffd640 1 -- 172.21.15.94:0/418606615 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0004a80 con 0x55f8e9466ad0 2024-02-17T12:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.206+0000 7f02aaffd640 1 -- 172.21.15.94:0/418606615 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02c0004030 con 0x55f8e9466ad0 2024-02-17T12:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.206+0000 7f02aaffd640 1 -- 172.21.15.94:0/418606615 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c00051d0 con 0x55f8e9466ad0 2024-02-17T12:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.206+0000 7f02cd824740 1 -- 172.21.15.94:0/418606615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9466ad0 msgr2=0x55f8e9465820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.206+0000 7f02cd824740 1 --2- 172.21.15.94:0/418606615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9466ad0 0x55f8e9465820 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f02c0000c00 tx=0x7f02c002c650 comp rx=0 tx=0).stop 2024-02-17T12:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 -- 172.21.15.94:0/418606615 shutdown_connections 2024-02-17T12:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 --2- 172.21.15.94:0/418606615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e946f2e0 0x55f8e94716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 --2- 172.21.15.94:0/418606615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9465d60 0x55f8e94661e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 --2- 172.21.15.94:0/418606615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9466ad0 0x55f8e9465820 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 -- 172.21.15.94:0/418606615 >> 172.21.15.94:0/418606615 conn(0x55f8e942d550 msgr2=0x55f8e9460a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 -- 172.21.15.94:0/418606615 shutdown_connections 2024-02-17T12:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 -- 172.21.15.94:0/418606615 wait complete. 2024-02-17T12:05:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 -- start start 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 Processor -- start 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9465d60 0x55f8e946c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9466ad0 0x55f8e946c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e946f2e0 0x55f8e946cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8e9372bf0 con 0x55f8e946f2e0 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8e90a1250 con 0x55f8e9465d60 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8e9353200 con 0x55f8e9466ad0 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02cd824740 10 client.0 _ll_register_callbacks cb 0x55f8e93c1d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02c89ea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9465d60 0x55f8e946c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.207+0000 7f02c91eb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e946f2e0 0x55f8e946cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02c89ea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9465d60 0x55f8e946c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:35456/0 (socket says 172.21.15.94:35456) 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02c91eb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e946f2e0 0x55f8e946cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42174/0 (socket says 172.21.15.94:42174) 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02c89ea640 1 -- 172.21.15.94:0/809983676 learned_addr learned my addr 172.21.15.94:0/809983676 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02bbfff640 1 --2- 172.21.15.94:0/809983676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9466ad0 0x55f8e946c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02c91eb640 1 -- 172.21.15.94:0/809983676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9466ad0 msgr2=0x55f8e946c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02c91eb640 1 --2- 172.21.15.94:0/809983676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9466ad0 0x55f8e946c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02c91eb640 1 -- 172.21.15.94:0/809983676 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9465d60 msgr2=0x55f8e946c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02c91eb640 1 --2- 172.21.15.94:0/809983676 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9465d60 0x55f8e946c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02c91eb640 1 -- 172.21.15.94:0/809983676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c410e860 con 0x55f8e946f2e0 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02c89ea640 1 --2- 172.21.15.94:0/809983676 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9465d60 0x55f8e946c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02c91eb640 1 --2- 172.21.15.94:0/809983676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e946f2e0 0x55f8e946cd10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f02bc01b000 tx=0x7f02bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44f424ecddab00b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02a3fff640 1 -- 172.21.15.94:0/809983676 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc028440 con 0x55f8e946f2e0 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02a3fff640 1 -- 172.21.15.94:0/809983676 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f02bc0285a0 con 0x55f8e946f2e0 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02cd824740 1 -- 172.21.15.94:0/809983676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8e90cd600 con 0x55f8e946f2e0 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02a3fff640 1 -- 172.21.15.94:0/809983676 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02bc028760 con 0x55f8e946f2e0 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02cd824740 10 client.17259 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02cd824740 1 -- 172.21.15.94:0/809983676 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f8e93cbd80 con 0x55f8e946f2e0 2024-02-17T12:05:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.208+0000 7f02a3fff640 1 -- 172.21.15.94:0/809983676 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f8e90cd600 con 0x55f8e946f2e0 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.209+0000 7f02a3fff640 1 -- 172.21.15.94:0/809983676 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02bc01ea10 con 0x55f8e946f2e0 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.209+0000 7f02cd824740 10 client.17259 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.209+0000 7f02cd824740 10 client.17259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.209+0000 7f02cd824740 20 client.17259 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.209+0000 7f02cd824740 1 --2- 172.21.15.94:0/809983676 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f8e947f620 0x55f8e9481a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.209+0000 7f02cd824740 4 client.17259 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.209+0000 7f02cd824740 1 -- 172.21.15.94:0/809983676 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8e911e1b0 con 0x55f8e947f620 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.209+0000 7f02c89ea640 1 --2- 172.21.15.94:0/809983676 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f8e947f620 0x55f8e9481a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.209+0000 7f02a3fff640 10 client.17259 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.209+0000 7f02c89ea640 1 --2- 172.21.15.94:0/809983676 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f8e947f620 0x55f8e9481a40 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceb5f79cde524ee8 server_cookie=f8355362891952b5 in_seq=0 out_seq=0 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02a3fff640 1 -- 172.21.15.94:0/809983676 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f8e911e1b0 con 0x55f8e947f620 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02a3fff640 10 client.17259 handle_command_reply: tid=0 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02cd824740 1 client.17259 shutdown 2024-02-17T12:05:01.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02cd824740 1 -- 172.21.15.94:0/809983676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e946f2e0 msgr2=0x55f8e946cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02cd824740 1 --2- 172.21.15.94:0/809983676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e946f2e0 0x55f8e946cd10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f02bc01b000 tx=0x7f02bc004ab0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02cd824740 1 -- 172.21.15.94:0/809983676 shutdown_connections 2024-02-17T12:05:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02cd824740 1 --2- 172.21.15.94:0/809983676 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f8e947f620 0x55f8e9481a40 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02cd824740 1 --2- 172.21.15.94:0/809983676 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e946f2e0 0x55f8e946cd10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02a3fff640 0 client.17259 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02cd824740 1 --2- 172.21.15.94:0/809983676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9466ad0 0x55f8e946c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02cd824740 1 --2- 172.21.15.94:0/809983676 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9465d60 0x55f8e946c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02cd824740 1 -- 172.21.15.94:0/809983676 >> 172.21.15.94:0/809983676 conn(0x55f8e942d550 msgr2=0x55f8e94627a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02cd824740 1 -- 172.21.15.94:0/809983676 shutdown_connections 2024-02-17T12:05:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02cd824740 1 -- 172.21.15.94:0/809983676 wait complete. 2024-02-17T12:05:01.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.210+0000 7f02cd824740 20 client.17259 trim_cache size 0 max 16384 2024-02-17T12:05:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02cd824740 1 Processor -- start 2024-02-17T12:05:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02cd824740 1 -- start start 2024-02-17T12:05:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02cd824740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9465d60 0x55f8e946e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02cd824740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 0x55f8e946cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02cd824740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e946f2e0 0x55f8e946ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02cd824740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8e911e1b0 con 0x55f8e9466ad0 2024-02-17T12:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02cd824740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8e90cd600 con 0x55f8e9465d60 2024-02-17T12:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02cd824740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8e947c090 con 0x55f8e946f2e0 2024-02-17T12:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 0x55f8e946cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 0x55f8e946cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42180/0 (socket says 172.21.15.94:42180) 2024-02-17T12:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02bbfff640 1 -- 172.21.15.94:0/2890382169 learned_addr learned my addr 172.21.15.94:0/2890382169 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02c89ea640 1 --2- 172.21.15.94:0/2890382169 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9465d60 0x55f8e946e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02c91eb640 1 --2- 172.21.15.94:0/2890382169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e946f2e0 0x55f8e946ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02bbfff640 1 -- 172.21.15.94:0/2890382169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e946f2e0 msgr2=0x55f8e946ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02bbfff640 1 --2- 172.21.15.94:0/2890382169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e946f2e0 0x55f8e946ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02bbfff640 1 -- 172.21.15.94:0/2890382169 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9465d60 msgr2=0x55f8e946e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:01.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02bbfff640 1 --2- 172.21.15.94:0/2890382169 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9465d60 0x55f8e946e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02bbfff640 1 -- 172.21.15.94:0/2890382169 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8e93687b0 con 0x55f8e9466ad0 2024-02-17T12:05:01.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.226+0000 7f02bbfff640 1 --2- 172.21.15.94:0/2890382169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 0x55f8e946cb30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f02b409ae00 tx=0x7f02b400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ffb52893a18648 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:01.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.227+0000 7f02aa7fc640 1 -- 172.21.15.94:0/2890382169 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40b7070 con 0x55f8e9466ad0 2024-02-17T12:05:01.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.227+0000 7f02aa7fc640 1 -- 172.21.15.94:0/2890382169 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02b409ae70 con 0x55f8e9466ad0 2024-02-17T12:05:01.229 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.227+0000 7f02aa7fc640 1 -- 172.21.15.94:0/2890382169 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4099290 con 0x55f8e9466ad0 2024-02-17T12:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.227+0000 7f02cd824740 1 -- 172.21.15.94:0/2890382169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 msgr2=0x55f8e946cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.227+0000 7f02cd824740 1 --2- 172.21.15.94:0/2890382169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 0x55f8e946cb30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f02b409ae00 tx=0x7f02b400cd70 comp rx=0 tx=0).stop 2024-02-17T12:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.227+0000 7f02cd824740 1 -- 172.21.15.94:0/2890382169 shutdown_connections 2024-02-17T12:05:01.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.227+0000 7f02cd824740 1 --2- 172.21.15.94:0/2890382169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e946f2e0 0x55f8e946ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.227+0000 7f02cd824740 1 --2- 172.21.15.94:0/2890382169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 0x55f8e946cb30 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.227+0000 7f02cd824740 1 --2- 172.21.15.94:0/2890382169 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e9465d60 0x55f8e946e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.227+0000 7f02cd824740 1 -- 172.21.15.94:0/2890382169 >> 172.21.15.94:0/2890382169 conn(0x55f8e942d550 msgr2=0x55f8e9467580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.228+0000 7f02cd824740 1 -- 172.21.15.94:0/2890382169 shutdown_connections 2024-02-17T12:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.228+0000 7f02cd824740 1 -- 172.21.15.94:0/2890382169 wait complete. 2024-02-17T12:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.228+0000 7f02cd824740 1 -- start start 2024-02-17T12:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.228+0000 7f02cd824740 1 Processor -- start 2024-02-17T12:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.228+0000 7f02cd824740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9465d60 0x55f8e94764e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.228+0000 7f02cd824740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 0x55f8e9474880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.228+0000 7f02cd824740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e946f2e0 0x55f8e9474e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.228+0000 7f02cd824740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8e9475520 con 0x55f8e9466ad0 2024-02-17T12:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.228+0000 7f02cd824740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8e9475690 con 0x55f8e946f2e0 2024-02-17T12:05:01.233 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.228+0000 7f02cd824740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8e94d5b30 con 0x55f8e9465d60 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.228+0000 7f02cd824740 10 client.0 _ll_register_callbacks cb 0x7f02bc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.228+0000 7f02bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 0x55f8e9474880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 0x55f8e9474880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42186/0 (socket says 172.21.15.94:42186) 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02bbfff640 1 -- 172.21.15.94:0/4114259225 learned_addr learned my addr 172.21.15.94:0/4114259225 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.228+0000 7f02c89ea640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9465d60 0x55f8e94764e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02c91eb640 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e946f2e0 0x55f8e9474e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02bbfff640 1 -- 172.21.15.94:0/4114259225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9465d60 msgr2=0x55f8e94764e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02bbfff640 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9465d60 0x55f8e94764e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02bbfff640 1 -- 172.21.15.94:0/4114259225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e946f2e0 msgr2=0x55f8e9474e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02bbfff640 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e946f2e0 0x55f8e9474e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02bbfff640 1 -- 172.21.15.94:0/4114259225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c41a6990 con 0x55f8e9466ad0 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02c91eb640 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e946f2e0 0x55f8e9474e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02c89ea640 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9465d60 0x55f8e94764e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:01.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02bbfff640 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 0x55f8e9474880 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f02b40972d0 tx=0x7f02b400cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7917f4c7c330de45 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02a3fff640 1 -- 172.21.15.94:0/4114259225 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40b7070 con 0x55f8e9466ad0 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02a3fff640 1 -- 172.21.15.94:0/4114259225 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f02b409efc0 con 0x55f8e9466ad0 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02cd824740 1 -- 172.21.15.94:0/4114259225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02c0004030 con 0x55f8e9466ad0 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02a3fff640 1 -- 172.21.15.94:0/4114259225 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b40cc060 con 0x55f8e9466ad0 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02cd824740 10 client.17265 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.229+0000 7f02cd824740 1 -- 172.21.15.94:0/4114259225 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f8e938e540 con 0x55f8e9466ad0 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.230+0000 7f02a3fff640 1 -- 172.21.15.94:0/4114259225 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02b4098f10 con 0x55f8e9466ad0 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.230+0000 7f02a3fff640 1 -- 172.21.15.94:0/4114259225 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f02b401f060 con 0x55f8e9466ad0 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.230+0000 7f02cd824740 10 client.17265 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.230+0000 7f02cd824740 10 client.17265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.230+0000 7f02cd824740 20 client.17265 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.230+0000 7f02cd824740 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f8e947f620 0x55f8e94d8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:01.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.230+0000 7f02cd824740 4 client.17265 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:05:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.230+0000 7f02cd824740 1 -- 172.21.15.94:0/4114259225 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f02c0004030 con 0x55f8e947f620 2024-02-17T12:05:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.230+0000 7f02c89ea640 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f8e947f620 0x55f8e94d8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.230+0000 7f02c89ea640 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f8e947f620 0x55f8e94d8e80 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc8bcfceabc41f7e server_cookie=5421bd1e9588a336 in_seq=0 out_seq=0 2024-02-17T12:05:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:01.230+0000 7f02a3fff640 10 client.17265 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:02.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:01 smithi094 ceph-mon[32499]: pgmap v211: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 8.0 MiB/s wr, 613 op/s 2024-02-17T12:05:02.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:01 smithi189 ceph-mon[36588]: pgmap v211: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 8.0 MiB/s wr, 613 op/s 2024-02-17T12:05:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02a3fff640 1 -- 172.21.15.94:0/4114259225 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f02c0004030 con 0x55f8e947f620 2024-02-17T12:05:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02a3fff640 10 client.17265 handle_command_reply: tid=0 2024-02-17T12:05:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02cd824740 1 client.17265 shutdown 2024-02-17T12:05:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02cd824740 1 -- 172.21.15.94:0/4114259225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 msgr2=0x55f8e9474880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02cd824740 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 0x55f8e9474880 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f02b40972d0 tx=0x7f02b400cec0 comp rx=0 tx=0).stop 2024-02-17T12:05:02.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02cd824740 1 -- 172.21.15.94:0/4114259225 shutdown_connections 2024-02-17T12:05:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02cd824740 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f8e947f620 0x55f8e94d8e80 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02cd824740 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f8e946f2e0 0x55f8e9474e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02cd824740 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f8e9466ad0 0x55f8e9474880 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02a3fff640 0 client.17265 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02cd824740 1 --2- 172.21.15.94:0/4114259225 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8e9465d60 0x55f8e94764e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02cd824740 1 -- 172.21.15.94:0/4114259225 >> 172.21.15.94:0/4114259225 conn(0x55f8e942d550 msgr2=0x55f8e946aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02cd824740 1 -- 172.21.15.94:0/4114259225 shutdown_connections 2024-02-17T12:05:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02cd824740 1 -- 172.21.15.94:0/4114259225 wait complete. 2024-02-17T12:05:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.415+0000 7f02cd824740 20 client.17265 trim_cache size 0 max 16384 2024-02-17T12:05:02.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:05:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:05:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:05:02.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:05:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.416+0000 7f02cac4e640 1 -- 172.21.15.94:0/331976755 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02ac0a22d0 msgr2=0x7f02ac0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.416+0000 7f02cac4e640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02ac0a22d0 0x7f02ac0a4790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f02bc009fd0 tx=0x7f02bc011b70 comp rx=0 tx=0).stop 2024-02-17T12:05:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.416+0000 7f02cac4e640 1 -- 172.21.15.94:0/331976755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c41064a0 msgr2=0x7f02c41008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.416+0000 7f02cac4e640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c41064a0 0x7f02c41008e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f02b400e9c0 tx=0x7f02b400ee80 comp rx=0 tx=0).stop 2024-02-17T12:05:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.416+0000 7f02cac4e640 1 -- 172.21.15.94:0/331976755 shutdown_connections 2024-02-17T12:05:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.416+0000 7f02cac4e640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02ac0a22d0 0x7f02ac0a4790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.416+0000 7f02cac4e640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f02c4100e20 0x7f02c41a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.416+0000 7f02cac4e640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f02c41064a0 0x7f02c41008e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.416+0000 7f02cac4e640 1 --2- 172.21.15.94:0/331976755 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02c4068a70 0x7f02c41003a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.416+0000 7f02cac4e640 1 -- 172.21.15.94:0/331976755 >> 172.21.15.94:0/331976755 conn(0x7f02c4076ed0 msgr2=0x7f02c40fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.416+0000 7f02cac4e640 1 -- 172.21.15.94:0/331976755 shutdown_connections 2024-02-17T12:05:02.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:02.416+0000 7f02cac4e640 1 -- 172.21.15.94:0/331976755 wait complete. 2024-02-17T12:05:02.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:01 smithi125 ceph-mon[36819]: pgmap v211: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 8.0 MiB/s wr, 613 op/s 2024-02-17T12:05:02.903 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: *** Caught signal (Segmentation fault) ** 2024-02-17T12:05:02.903 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: in thread 7fb7dc2dc700 thread_name:md_log_replay 2024-02-17T12:05:02.903 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:05:02.903 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb7ed69dd20] 2024-02-17T12:05:02.903 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 2: /usr/bin/ceph-mds(+0x504802) [0x556676ce3802] 2024-02-17T12:05:02.903 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556676ce5a53] 2024-02-17T12:05:02.903 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 4: (MDLog::_replay_thread()+0x647) [0x556676d42277] 2024-02-17T12:05:02.903 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556676d4c111] 2024-02-17T12:05:02.903 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fb7ed6931ca] 2024-02-17T12:05:02.903 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 7: clone() 2024-02-17T12:05:02.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 2024-02-17T12:05:02.588+0000 7fb7dc2dc700 -1 *** Caught signal (Segmentation fault) ** 2024-02-17T12:05:02.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: in thread 7fb7dc2dc700 thread_name:md_log_replay 2024-02-17T12:05:02.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 2024-02-17T12:05:02.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:05:02.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb7ed69dd20] 2024-02-17T12:05:02.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 2: /usr/bin/ceph-mds(+0x504802) [0x556676ce3802] 2024-02-17T12:05:02.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556676ce5a53] 2024-02-17T12:05:02.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 4: (MDLog::_replay_thread()+0x647) [0x556676d42277] 2024-02-17T12:05:02.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556676d4c111] 2024-02-17T12:05:02.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fb7ed6931ca] 2024-02-17T12:05:02.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 7: clone() 2024-02-17T12:05:02.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 0> 2024-02-17T12:05:02.588+0000 7fb7dc2dc700 -1 *** Caught signal (Segmentation fault) ** 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: in thread 7fb7dc2dc700 thread_name:md_log_replay 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb7ed69dd20] 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 2: /usr/bin/ceph-mds(+0x504802) [0x556676ce3802] 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556676ce5a53] 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 4: (MDLog::_replay_thread()+0x647) [0x556676d42277] 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556676d4c111] 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fb7ed6931ca] 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 7: clone() 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-17T12:05:02.905 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 2024-02-17T12:05:02.906 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 0> 2024-02-17T12:05:02.588+0000 7fb7dc2dc700 -1 *** Caught signal (Segmentation fault) ** 2024-02-17T12:05:02.906 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: in thread 7fb7dc2dc700 thread_name:md_log_replay 2024-02-17T12:05:02.906 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 2024-02-17T12:05:02.906 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: ceph version 19.0.0-1515-g6822c7b5 (6822c7b54e949006be7a0802650d01723d834d1d) squid (dev) 2024-02-17T12:05:02.906 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb7ed69dd20] 2024-02-17T12:05:02.906 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 2: /usr/bin/ceph-mds(+0x504802) [0x556676ce3802] 2024-02-17T12:05:02.906 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556676ce5a53] 2024-02-17T12:05:02.906 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 4: (MDLog::_replay_thread()+0x647) [0x556676d42277] 2024-02-17T12:05:02.906 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556676d4c111] 2024-02-17T12:05:02.906 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fb7ed6931ca] 2024-02-17T12:05:02.906 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 7: clone() 2024-02-17T12:05:02.906 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-02-17T12:05:02.906 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[71905]: 2024-02-17T12:05:02.907 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:02 smithi189 podman[72258]: 2024-02-17 12:05:02.646829676 +0000 UTC m=+0.019878888 container died db9500b09e5939bdbb03030b341ecca04e24acd3c7bb25e897d10c5ad797d9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-17T12:05:03.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:05:03.444 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:05:03.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.561+0000 7f092e548640 1 -- 172.21.15.94:0/1075428885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0928100270 msgr2=0x7f0928100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.561+0000 7f092e548640 1 --2- 172.21.15.94:0/1075428885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0928100270 0x7f0928100650 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0918009930 tx=0x7f091802bc30 comp rx=0 tx=0).stop 2024-02-17T12:05:03.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.561+0000 7f092e548640 1 -- 172.21.15.94:0/1075428885 shutdown_connections 2024-02-17T12:05:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.561+0000 7f092e548640 1 --2- 172.21.15.94:0/1075428885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0928105340 0x7f092810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.561+0000 7f092e548640 1 --2- 172.21.15.94:0/1075428885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0928100c20 0x7f0928104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.561+0000 7f092e548640 1 --2- 172.21.15.94:0/1075428885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0928100270 0x7f0928100650 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.561+0000 7f092e548640 1 -- 172.21.15.94:0/1075428885 >> 172.21.15.94:0/1075428885 conn(0x7f0928077230 msgr2=0x7f0928077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.561+0000 7f092e548640 1 -- 172.21.15.94:0/1075428885 shutdown_connections 2024-02-17T12:05:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.561+0000 7f092e548640 1 -- 172.21.15.94:0/1075428885 wait complete. 2024-02-17T12:05:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.562+0000 7f092e548640 1 Processor -- start 2024-02-17T12:05:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.562+0000 7f092e548640 1 -- start start 2024-02-17T12:05:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.562+0000 7f092e548640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0928100270 0x7f092819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.562+0000 7f092e548640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0928105340 0x7f092819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.562+0000 7f092e548640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09281975c0 0x7f0928197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.565 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.562+0000 7f092e548640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0928197fb0 con 0x7f09281975c0 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.562+0000 7f092e548640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0928198120 con 0x7f0928100270 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.562+0000 7f092e548640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09281982e0 con 0x7f0928105340 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.562+0000 7f092cae5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09281975c0 0x7f0928197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.562+0000 7f092cae5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09281975c0 0x7f0928197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42190/0 (socket says 172.21.15.94:42190) 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.562+0000 7f092cae5640 1 -- 172.21.15.94:0/3992243142 learned_addr learned my addr 172.21.15.94:0/3992243142 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.562+0000 7f09277fe640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0928105340 0x7f092819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.562+0000 7f0927fff640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0928100270 0x7f092819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.563+0000 7f092cae5640 1 -- 172.21.15.94:0/3992243142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0928105340 msgr2=0x7f092819d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.563+0000 7f092cae5640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0928105340 0x7f092819d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.563+0000 7f092cae5640 1 -- 172.21.15.94:0/3992243142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0928100270 msgr2=0x7f092819d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.563+0000 7f092cae5640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0928100270 0x7f092819d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.563+0000 7f092cae5640 1 -- 172.21.15.94:0/3992243142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0918009560 con 0x7f09281975c0 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.563+0000 7f092cae5640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09281975c0 0x7f0928197a70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f092000eb10 tx=0x7f092000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:03.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.563+0000 7f09257fa640 1 -- 172.21.15.94:0/3992243142 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920004b20 con 0x7f09281975c0 2024-02-17T12:05:03.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.563+0000 7f092e548640 1 -- 172.21.15.94:0/3992243142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09281a42d0 con 0x7f09281975c0 2024-02-17T12:05:03.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.563+0000 7f09257fa640 1 -- 172.21.15.94:0/3992243142 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0920004c80 con 0x7f09281975c0 2024-02-17T12:05:03.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.563+0000 7f09257fa640 1 -- 172.21.15.94:0/3992243142 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0920010670 con 0x7f09281975c0 2024-02-17T12:05:03.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.563+0000 7f092e548640 1 -- 172.21.15.94:0/3992243142 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09281a47e0 con 0x7f09281975c0 2024-02-17T12:05:03.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.565+0000 7f09257fa640 1 -- 172.21.15.94:0/3992243142 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0920005020 con 0x7f09281975c0 2024-02-17T12:05:03.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.565+0000 7f09257fa640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09100a2230 0x7f09100a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.565+0000 7f09311f8740 1 Processor -- start 2024-02-17T12:05:03.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.565+0000 7f09311f8740 1 -- start start 2024-02-17T12:05:03.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f09257fa640 1 -- 172.21.15.94:0/3992243142 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0920014070 con 0x7f09281975c0 2024-02-17T12:05:03.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f0927fff640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09100a2230 0x7f09100a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f09311f8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b12b0 0x5578fc1b1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f09311f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1b1bd0 0x5578fc1ba270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f09311f8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1ba7b0 0x5578fc1bcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f09311f8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578fc120280 con 0x5578fc1ba7b0 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f09311f8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578fc11d7b0 con 0x5578fc1b12b0 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f09311f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578fc118f90 con 0x5578fc1b1bd0 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f092cae5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1ba7b0 0x5578fc1bcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f092cae5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1ba7b0 0x5578fc1bcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42200/0 (socket says 172.21.15.94:42200) 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f09277fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1b1bd0 0x5578fc1ba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f092cae5640 1 -- 172.21.15.94:0/1852792525 learned_addr learned my addr 172.21.15.94:0/1852792525 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f0927fff640 1 --2- 172.21.15.94:0/1852792525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b12b0 0x5578fc1b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f0927fff640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09100a2230 0x7f09100a46f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0918005ab0 tx=0x7f0918004720 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f092cae5640 1 -- 172.21.15.94:0/1852792525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1b1bd0 msgr2=0x5578fc1ba270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f092cae5640 1 --2- 172.21.15.94:0/1852792525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1b1bd0 0x5578fc1ba270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f092cae5640 1 -- 172.21.15.94:0/1852792525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b12b0 msgr2=0x5578fc1b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:03.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f092cae5640 1 --2- 172.21.15.94:0/1852792525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b12b0 0x5578fc1b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f092cae5640 1 -- 172.21.15.94:0/1852792525 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578fc0d7570 con 0x5578fc1ba7b0 2024-02-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f092cae5640 1 --2- 172.21.15.94:0/1852792525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1ba7b0 0x5578fc1bcba0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0920097950 tx=0x7f0920000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64bb92a91a48de5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f090e7fc640 1 -- 172.21.15.94:0/1852792525 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092009af80 con 0x5578fc1ba7b0 2024-02-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f090e7fc640 1 -- 172.21.15.94:0/1852792525 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f092009f0d0 con 0x5578fc1ba7b0 2024-02-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.566+0000 7f090e7fc640 1 -- 172.21.15.94:0/1852792525 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09200cc130 con 0x5578fc1ba7b0 2024-02-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.567+0000 7f09311f8740 1 -- 172.21.15.94:0/1852792525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1ba7b0 msgr2=0x5578fc1bcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.567+0000 7f09311f8740 1 --2- 172.21.15.94:0/1852792525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1ba7b0 0x5578fc1bcba0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0920097950 tx=0x7f0920000f40 comp rx=0 tx=0).stop 2024-02-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.567+0000 7f09311f8740 1 -- 172.21.15.94:0/1852792525 shutdown_connections 2024-02-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.567+0000 7f09311f8740 1 --2- 172.21.15.94:0/1852792525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1ba7b0 0x5578fc1bcba0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.567+0000 7f09311f8740 1 --2- 172.21.15.94:0/1852792525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1b1bd0 0x5578fc1ba270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.567+0000 7f09311f8740 1 --2- 172.21.15.94:0/1852792525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b12b0 0x5578fc1b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.567+0000 7f09311f8740 1 -- 172.21.15.94:0/1852792525 >> 172.21.15.94:0/1852792525 conn(0x5578fc1ab990 msgr2=0x5578fc1abd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:03.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.567+0000 7f09311f8740 1 -- 172.21.15.94:0/1852792525 shutdown_connections 2024-02-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.567+0000 7f09311f8740 1 -- 172.21.15.94:0/1852792525 wait complete. 2024-02-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.567+0000 7f09311f8740 1 -- start start 2024-02-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f09311f8740 1 Processor -- start 2024-02-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f09311f8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 0x5578fc1ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f09311f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1b1bd0 0x5578fc1b7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f09311f8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1ba7b0 0x5578fc1b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f09311f8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578fbdec430 con 0x5578fc1b12b0 2024-02-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f09311f8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578fbda8e00 con 0x5578fc1ba7b0 2024-02-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f09311f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578fc09dfd0 con 0x5578fc1b1bd0 2024-02-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f09311f8740 10 client.0 _ll_register_callbacks cb 0x5578fc0f3fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f0927fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 0x5578fc1ba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f0927fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 0x5578fc1ba0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42212/0 (socket says 172.21.15.94:42212) 2024-02-17T12:05:03.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f0927fff640 1 -- 172.21.15.94:0/671726044 learned_addr learned my addr 172.21.15.94:0/671726044 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f09277fe640 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1b1bd0 0x5578fc1b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f092cae5640 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1ba7b0 0x5578fc1b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f0927fff640 1 -- 172.21.15.94:0/671726044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1b1bd0 msgr2=0x5578fc1b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f0927fff640 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1b1bd0 0x5578fc1b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f0927fff640 1 -- 172.21.15.94:0/671726044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1ba7b0 msgr2=0x5578fc1b79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f0927fff640 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1ba7b0 0x5578fc1b79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f0927fff640 1 -- 172.21.15.94:0/671726044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09281a47e0 con 0x5578fc1b12b0 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f092cae5640 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1ba7b0 0x5578fc1b79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f0927fff640 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 0x5578fc1ba0f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0918035000 tx=0x7f091802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3542826d6df87615 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.568+0000 7f09077fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09077fe640 1 -- 172.21.15.94:0/671726044 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918005000 con 0x5578fc1b12b0 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09077fe640 1 -- 172.21.15.94:0/671726044 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0918005160 con 0x5578fc1b12b0 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09311f8740 1 -- 172.21.15.94:0/671726044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578fbe187e0 con 0x5578fc1b12b0 2024-02-17T12:05:03.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09077fe640 1 -- 172.21.15.94:0/671726044 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0918005320 con 0x5578fc1b12b0 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09311f8740 10 client.17277 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09311f8740 1 -- 172.21.15.94:0/671726044 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5578fc117610 con 0x5578fc1b12b0 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09077fe640 1 -- 172.21.15.94:0/671726044 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5578fbe187e0 con 0x5578fc1b12b0 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09077fe640 1 -- 172.21.15.94:0/671726044 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f09180414f0 con 0x5578fc1b12b0 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09311f8740 10 client.17277 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09311f8740 10 client.17277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09311f8740 20 client.17277 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09311f8740 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5578fc1ca820 0x5578fc1ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09311f8740 4 client.17277 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.569+0000 7f09311f8740 1 -- 172.21.15.94:0/671726044 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578fbe55720 con 0x5578fc1ca820 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.570+0000 7f09277fe640 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5578fc1ca820 0x5578fc1ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.570+0000 7f09077fe640 10 client.17277 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.570+0000 7f09277fe640 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5578fc1ca820 0x5578fc1ccc40 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=790a95a567a4b6be server_cookie=1d78ae3cc60bf484 in_seq=0 out_seq=0 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.570+0000 7f09077fe640 1 -- 172.21.15.94:0/671726044 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5578fbe55720 con 0x5578fc1ca820 2024-02-17T12:05:03.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.570+0000 7f09077fe640 10 client.17277 handle_command_reply: tid=0 2024-02-17T12:05:03.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.570+0000 7f09311f8740 1 client.17277 shutdown 2024-02-17T12:05:03.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.571+0000 7f09311f8740 1 -- 172.21.15.94:0/671726044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 msgr2=0x5578fc1ba0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.571+0000 7f09311f8740 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 0x5578fc1ba0f0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f0918035000 tx=0x7f091802bc30 comp rx=0 tx=0).stop 2024-02-17T12:05:03.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.571+0000 7f09311f8740 1 -- 172.21.15.94:0/671726044 shutdown_connections 2024-02-17T12:05:03.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.571+0000 7f09311f8740 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5578fc1ca820 0x5578fc1ccc40 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.571+0000 7f09311f8740 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1ba7b0 0x5578fc1b79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.571+0000 7f09311f8740 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1b1bd0 0x5578fc1b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.571+0000 7f09311f8740 1 --2- 172.21.15.94:0/671726044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 0x5578fc1ba0f0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.571+0000 7f09077fe640 0 client.17277 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:03.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.571+0000 7f09311f8740 1 -- 172.21.15.94:0/671726044 >> 172.21.15.94:0/671726044 conn(0x5578fc1ab990 msgr2=0x5578fc1ad730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:03.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.571+0000 7f09311f8740 1 -- 172.21.15.94:0/671726044 shutdown_connections 2024-02-17T12:05:03.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.571+0000 7f09311f8740 1 -- 172.21.15.94:0/671726044 wait complete. 2024-02-17T12:05:03.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.571+0000 7f09311f8740 20 client.17277 trim_cache size 0 max 16384 2024-02-17T12:05:03.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.586+0000 7f09311f8740 1 Processor -- start 2024-02-17T12:05:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.586+0000 7f09311f8740 1 -- start start 2024-02-17T12:05:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.586+0000 7f09311f8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b12b0 0x5578fc1b9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.586+0000 7f09311f8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b1bd0 0x5578fc1ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.586+0000 7f09311f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1ba7b0 0x5578fc1c1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.586+0000 7f09311f8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578fc1c73c0 con 0x5578fc1b1bd0 2024-02-17T12:05:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.586+0000 7f09311f8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578fc1aca10 con 0x5578fc1b12b0 2024-02-17T12:05:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.586+0000 7f09311f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578fc1c6570 con 0x5578fc1ba7b0 2024-02-17T12:05:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.586+0000 7f09277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b1bd0 0x5578fc1ba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.586+0000 7f09277fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b1bd0 0x5578fc1ba0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42220/0 (socket says 172.21.15.94:42220) 2024-02-17T12:05:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.586+0000 7f09277fe640 1 -- 172.21.15.94:0/2464902665 learned_addr learned my addr 172.21.15.94:0/2464902665 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.586+0000 7f0927fff640 1 --2- 172.21.15.94:0/2464902665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b12b0 0x5578fc1b9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.587+0000 7f092cae5640 1 --2- 172.21.15.94:0/2464902665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1ba7b0 0x5578fc1c1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.587+0000 7f09277fe640 1 -- 172.21.15.94:0/2464902665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1ba7b0 msgr2=0x5578fc1c1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.587+0000 7f09277fe640 1 --2- 172.21.15.94:0/2464902665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1ba7b0 0x5578fc1c1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.587+0000 7f09277fe640 1 -- 172.21.15.94:0/2464902665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b12b0 msgr2=0x5578fc1b9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.587+0000 7f09277fe640 1 --2- 172.21.15.94:0/2464902665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b12b0 0x5578fc1b9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.587+0000 7f09277fe640 1 -- 172.21.15.94:0/2464902665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578fc0b3ac0 con 0x5578fc1b1bd0 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.587+0000 7f09277fe640 1 --2- 172.21.15.94:0/2464902665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b1bd0 0x5578fc1ba0f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f091c009d10 tx=0x7f091c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5026c75c833b4f0d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.587+0000 7f090dffb640 1 -- 172.21.15.94:0/2464902665 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c017070 con 0x5578fc1b1bd0 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.587+0000 7f090dffb640 1 -- 172.21.15.94:0/2464902665 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f091c018040 con 0x5578fc1b1bd0 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.587+0000 7f090dffb640 1 -- 172.21.15.94:0/2464902665 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c010d00 con 0x5578fc1b1bd0 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.588+0000 7f09311f8740 1 -- 172.21.15.94:0/2464902665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b1bd0 msgr2=0x5578fc1ba0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.588+0000 7f09311f8740 1 --2- 172.21.15.94:0/2464902665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b1bd0 0x5578fc1ba0f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f091c009d10 tx=0x7f091c005f90 comp rx=0 tx=0).stop 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.588+0000 7f09311f8740 1 -- 172.21.15.94:0/2464902665 shutdown_connections 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.588+0000 7f09311f8740 1 --2- 172.21.15.94:0/2464902665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1ba7b0 0x5578fc1c1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.588+0000 7f09311f8740 1 --2- 172.21.15.94:0/2464902665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b1bd0 0x5578fc1ba0f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.588+0000 7f09311f8740 1 --2- 172.21.15.94:0/2464902665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b12b0 0x5578fc1b9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.588+0000 7f09311f8740 1 -- 172.21.15.94:0/2464902665 >> 172.21.15.94:0/2464902665 conn(0x5578fc1ab990 msgr2=0x5578fc1d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:03.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.588+0000 7f09311f8740 1 -- 172.21.15.94:0/2464902665 shutdown_connections 2024-02-17T12:05:03.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.588+0000 7f09311f8740 1 -- 172.21.15.94:0/2464902665 wait complete. 2024-02-17T12:05:03.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.588+0000 7f09311f8740 1 -- start start 2024-02-17T12:05:03.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f09311f8740 1 Processor -- start 2024-02-17T12:05:03.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f09311f8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 0x5578fc1c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f09311f8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b1bd0 0x5578fc1bde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f09311f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1ba7b0 0x5578fc1be470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f09311f8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578fc1beb40 con 0x5578fc1b12b0 2024-02-17T12:05:03.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f09311f8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578fc1becb0 con 0x5578fc1b1bd0 2024-02-17T12:05:03.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f09311f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578fc220d20 con 0x5578fc1ba7b0 2024-02-17T12:05:03.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f09311f8740 10 client.0 _ll_register_callbacks cb 0x7f0918009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:03.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f0927fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 0x5578fc1c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f0927fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 0x5578fc1c1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42236/0 (socket says 172.21.15.94:42236) 2024-02-17T12:05:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f0927fff640 1 -- 172.21.15.94:0/3888379569 learned_addr learned my addr 172.21.15.94:0/3888379569 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f09277fe640 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b1bd0 0x5578fc1bde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f092cae5640 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1ba7b0 0x5578fc1be470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f0927fff640 1 -- 172.21.15.94:0/3888379569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1ba7b0 msgr2=0x5578fc1be470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f0927fff640 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1ba7b0 0x5578fc1be470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f0927fff640 1 -- 172.21.15.94:0/3888379569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b1bd0 msgr2=0x5578fc1bde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f0927fff640 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b1bd0 0x5578fc1bde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f0927fff640 1 -- 172.21.15.94:0/3888379569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c004040 con 0x5578fc1b12b0 2024-02-17T12:05:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.589+0000 7f09277fe640 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b1bd0 0x5578fc1bde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.590+0000 7f0927fff640 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 0x5578fc1c1ae0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f091802c170 tx=0x7f091802c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86bcbf6788ae31d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.590+0000 7f09077fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.590+0000 7f09077fe640 1 -- 172.21.15.94:0/3888379569 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0918041670 con 0x5578fc1b12b0 2024-02-17T12:05:03.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.590+0000 7f09077fe640 1 -- 172.21.15.94:0/3888379569 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f0918041e40 con 0x5578fc1b12b0 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.590+0000 7f09311f8740 1 -- 172.21.15.94:0/3888379569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f092009f0d0 con 0x5578fc1b12b0 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.590+0000 7f09077fe640 1 -- 172.21.15.94:0/3888379569 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0918041bd0 con 0x5578fc1b12b0 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.590+0000 7f09311f8740 10 client.17283 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.590+0000 7f09311f8740 1 -- 172.21.15.94:0/3888379569 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5578fc0d96a0 con 0x5578fc1b12b0 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.590+0000 7f09077fe640 1 -- 172.21.15.94:0/3888379569 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0918067030 con 0x5578fc1b12b0 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.590+0000 7f09077fe640 1 -- 172.21.15.94:0/3888379569 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0918003cc0 con 0x5578fc1b12b0 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.591+0000 7f09311f8740 10 client.17283 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.591+0000 7f09311f8740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.591+0000 7f09311f8740 20 client.17283 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.591+0000 7f09311f8740 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5578fc222270 0x5578fc224690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.591+0000 7f09311f8740 4 client.17283 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.591+0000 7f09311f8740 1 -- 172.21.15.94:0/3888379569 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5578fbea4890 con 0x5578fc222270 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.591+0000 7f09277fe640 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5578fc222270 0x5578fc224690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.591+0000 7f09077fe640 10 client.17283 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:03.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.591+0000 7f09277fe640 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5578fc222270 0x5578fc224690 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7d6013ba680d91b server_cookie=44985122c58d686e in_seq=0 out_seq=0 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.591+0000 7f09077fe640 1 -- 172.21.15.94:0/3888379569 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5578fbea4890 con 0x5578fc222270 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.591+0000 7f09077fe640 10 client.17283 handle_command_reply: tid=0 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.592+0000 7f09311f8740 1 client.17283 shutdown 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.592+0000 7f09311f8740 1 -- 172.21.15.94:0/3888379569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 msgr2=0x5578fc1c1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.592+0000 7f09311f8740 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 0x5578fc1c1ae0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f091802c170 tx=0x7f091802c3b0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.592+0000 7f09311f8740 1 -- 172.21.15.94:0/3888379569 shutdown_connections 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.592+0000 7f09311f8740 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5578fc222270 0x5578fc224690 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.592+0000 7f09311f8740 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5578fc1ba7b0 0x5578fc1be470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.592+0000 7f09311f8740 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5578fc1b1bd0 0x5578fc1bde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.592+0000 7f09311f8740 1 --2- 172.21.15.94:0/3888379569 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5578fc1b12b0 0x5578fc1c1ae0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.592+0000 7f09311f8740 1 -- 172.21.15.94:0/3888379569 >> 172.21.15.94:0/3888379569 conn(0x5578fc1ab990 msgr2=0x5578fc1b5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.592+0000 7f09077fe640 0 client.17283 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.592+0000 7f09311f8740 1 -- 172.21.15.94:0/3888379569 shutdown_connections 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.592+0000 7f09311f8740 1 -- 172.21.15.94:0/3888379569 wait complete. 2024-02-17T12:05:03.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.592+0000 7f09311f8740 20 client.17283 trim_cache size 0 max 16384 2024-02-17T12:05:03.599 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:05:03.599 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:05:03.599 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:05:03.599 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:05:03.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.593+0000 7f092e548640 1 -- 172.21.15.94:0/3992243142 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09100a2230 msgr2=0x7f09100a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.593+0000 7f092e548640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09100a2230 0x7f09100a46f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0918005ab0 tx=0x7f0918004720 comp rx=0 tx=0).stop 2024-02-17T12:05:03.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.593+0000 7f092e548640 1 -- 172.21.15.94:0/3992243142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09281975c0 msgr2=0x7f0928197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.593+0000 7f092e548640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09281975c0 0x7f0928197a70 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f092000eb10 tx=0x7f092000c2d0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.593+0000 7f092e548640 1 -- 172.21.15.94:0/3992243142 shutdown_connections 2024-02-17T12:05:03.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.593+0000 7f092e548640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09100a2230 0x7f09100a46f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.593+0000 7f092e548640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09281975c0 0x7f0928197a70 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.593+0000 7f092e548640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0928105340 0x7f092819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.593+0000 7f092e548640 1 --2- 172.21.15.94:0/3992243142 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0928100270 0x7f092819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.593+0000 7f092e548640 1 -- 172.21.15.94:0/3992243142 >> 172.21.15.94:0/3992243142 conn(0x7f0928077230 msgr2=0x7f0928103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:03.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.593+0000 7f092e548640 1 -- 172.21.15.94:0/3992243142 shutdown_connections 2024-02-17T12:05:03.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.593+0000 7f092e548640 1 -- 172.21.15.94:0/3992243142 wait complete. 2024-02-17T12:05:03.604 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:05:03.605 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:05:03.652 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:03 smithi189 podman[72258]: 2024-02-17 12:05:03.278377374 +0000 UTC m=+0.651426579 container remove db9500b09e5939bdbb03030b341ecca04e24acd3c7bb25e897d10c5ad797d9ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD) 2024-02-17T12:05:03.652 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:03 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.l.service: Main process exited, code=exited, status=139/n/a 2024-02-17T12:05:03.652 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:03 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.l.service: Failed with result 'exit-code'. 2024-02-17T12:05:03.652 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:03 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.l.service: Consumed 1.091s CPU time. 2024-02-17T12:05:03.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.714+0000 7f96dd422640 1 -- 172.21.15.94:0/2316323415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96d80ffa50 msgr2=0x7f96d80ffed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.714+0000 7f96dd422640 1 --2- 172.21.15.94:0/2316323415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96d80ffa50 0x7f96d80ffed0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009930 tx=0x7f96cc02bcd0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.715+0000 7f96dd422640 1 -- 172.21.15.94:0/2316323415 shutdown_connections 2024-02-17T12:05:03.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.715+0000 7f96dd422640 1 --2- 172.21.15.94:0/2316323415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96d8100410 0x7f96d810e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.715+0000 7f96dd422640 1 --2- 172.21.15.94:0/2316323415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96d80ffa50 0x7f96d80ffed0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.715+0000 7f96dd422640 1 --2- 172.21.15.94:0/2316323415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96d8107390 0x7f96d8107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.715+0000 7f96dd422640 1 -- 172.21.15.94:0/2316323415 >> 172.21.15.94:0/2316323415 conn(0x7f96d80753d0 msgr2=0x7f96d80757e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:03.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.715+0000 7f96dd422640 1 -- 172.21.15.94:0/2316323415 shutdown_connections 2024-02-17T12:05:03.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.715+0000 7f96dd422640 1 -- 172.21.15.94:0/2316323415 wait complete. 2024-02-17T12:05:03.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.715+0000 7f96dd422640 1 Processor -- start 2024-02-17T12:05:03.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96dd422640 1 -- start start 2024-02-17T12:05:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96dd422640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96d80ffa50 0x7f96d819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96dd422640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96d8100410 0x7f96d819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96dd422640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96d8107390 0x7f96d81976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96dd422640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d8197e30 con 0x7f96d8107390 2024-02-17T12:05:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96dd422640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d8197fa0 con 0x7f96d8100410 2024-02-17T12:05:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96dd422640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d8198160 con 0x7f96d80ffa50 2024-02-17T12:05:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96d77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96d8107390 0x7f96d81976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96d67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96d8100410 0x7f96d819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96d77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96d8107390 0x7f96d81976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42248/0 (socket says 172.21.15.94:42248) 2024-02-17T12:05:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96d77fe640 1 -- 172.21.15.94:0/1643561006 learned_addr learned my addr 172.21.15.94:0/1643561006 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:03.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96d6ffd640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96d80ffa50 0x7f96d819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96d77fe640 1 -- 172.21.15.94:0/1643561006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96d80ffa50 msgr2=0x7f96d819d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96d77fe640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96d80ffa50 0x7f96d819d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96d77fe640 1 -- 172.21.15.94:0/1643561006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96d8100410 msgr2=0x7f96d819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96d77fe640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96d8100410 0x7f96d819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96d77fe640 1 -- 172.21.15.94:0/1643561006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96cc009560 con 0x7f96d8107390 2024-02-17T12:05:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96d6ffd640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96d80ffa50 0x7f96d819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:05:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.716+0000 7f96d67fc640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96d8100410 0x7f96d819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.717+0000 7f96d77fe640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96d8107390 0x7f96d81976a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f96d000eb10 tx=0x7f96d000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.717+0000 7f96bffff640 1 -- 172.21.15.94:0/1643561006 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0004b20 con 0x7f96d8107390 2024-02-17T12:05:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.717+0000 7f96bffff640 1 -- 172.21.15.94:0/1643561006 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96d0004c80 con 0x7f96d8107390 2024-02-17T12:05:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.717+0000 7f96bffff640 1 -- 172.21.15.94:0/1643561006 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0010650 con 0x7f96d8107390 2024-02-17T12:05:03.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.717+0000 7f96dd422640 1 -- 172.21.15.94:0/1643561006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d8198440 con 0x7f96d8107390 2024-02-17T12:05:03.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.717+0000 7f96dd422640 1 -- 172.21.15.94:0/1643561006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d81a4510 con 0x7f96d8107390 2024-02-17T12:05:03.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.718+0000 7f96dd422640 1 -- 172.21.15.94:0/1643561006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d8069350 con 0x7f96d8107390 2024-02-17T12:05:03.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.720+0000 7f96bffff640 1 -- 172.21.15.94:0/1643561006 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f96d00107d0 con 0x7f96d8107390 2024-02-17T12:05:03.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.720+0000 7f96bffff640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96c00a22b0 0x7f96c00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:03.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.721+0000 7f96d6ffd640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96c00a22b0 0x7f96c00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:03.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.721+0000 7f96bffff640 1 -- 172.21.15.94:0/1643561006 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f96d0014070 con 0x7f96d8107390 2024-02-17T12:05:03.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.721+0000 7f96d6ffd640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96c00a22b0 0x7f96c00a4770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f96c8009c20 tx=0x7f96c8005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:03.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.725+0000 7f96bffff640 1 -- 172.21.15.94:0/1643561006 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96d0097490 con 0x7f96d8107390 2024-02-17T12:05:03.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.982+0000 7f96dd422640 1 -- 172.21.15.94:0/1643561006 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96d8198b80 con 0x7f96d8107390 2024-02-17T12:05:03.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.983+0000 7f96bffff640 1 -- 172.21.15.94:0/1643561006 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11338 (secure 0 0 0) 0x7f96d009ab50 con 0x7f96d8107390 2024-02-17T12:05:03.983 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:05:03.983 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:03:14.932679+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26335":{"gid":26335,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/39050300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":39050300},{"type":"v1","addr":"172.21.15.189:6833","nonce":39050300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:05:03.984 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 25 2024-02-17T12:05:03.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.985+0000 7f96dd422640 1 -- 172.21.15.94:0/1643561006 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96c00a22b0 msgr2=0x7f96c00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.985+0000 7f96dd422640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96c00a22b0 0x7f96c00a4770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f96c8009c20 tx=0x7f96c8005f90 comp rx=0 tx=0).stop 2024-02-17T12:05:03.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.985+0000 7f96dd422640 1 -- 172.21.15.94:0/1643561006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96d8107390 msgr2=0x7f96d81976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:03.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.985+0000 7f96dd422640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96d8107390 0x7f96d81976a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f96d000eb10 tx=0x7f96d000c2d0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.985+0000 7f96dd422640 1 -- 172.21.15.94:0/1643561006 shutdown_connections 2024-02-17T12:05:03.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.985+0000 7f96dd422640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96c00a22b0 0x7f96c00a4770 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.985+0000 7f96dd422640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96d8107390 0x7f96d81976a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.985+0000 7f96dd422640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96d8100410 0x7f96d819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.985+0000 7f96dd422640 1 --2- 172.21.15.94:0/1643561006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96d80ffa50 0x7f96d819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:03.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.985+0000 7f96dd422640 1 -- 172.21.15.94:0/1643561006 >> 172.21.15.94:0/1643561006 conn(0x7f96d80753d0 msgr2=0x7f96d80fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:03.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.985+0000 7f96dd422640 1 -- 172.21.15.94:0/1643561006 shutdown_connections 2024-02-17T12:05:03.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:03.985+0000 7f96dd422640 1 -- 172.21.15.94:0/1643561006 wait complete. 2024-02-17T12:05:03.997 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:05:03.997 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:05:04.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.107+0000 7fdbadc39640 1 -- 172.21.15.94:0/1130093844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba8105330 msgr2=0x7fdba810cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.107+0000 7fdbadc39640 1 --2- 172.21.15.94:0/1130093844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba8105330 0x7fdba810cf00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009cd0 tx=0x7fdba002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.107+0000 7fdbadc39640 1 -- 172.21.15.94:0/1130093844 shutdown_connections 2024-02-17T12:05:04.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.107+0000 7fdbadc39640 1 --2- 172.21.15.94:0/1130093844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba8105330 0x7fdba810cf00 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.107+0000 7fdbadc39640 1 --2- 172.21.15.94:0/1130093844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdba8100c10 0x7fdba8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.107+0000 7fdbadc39640 1 --2- 172.21.15.94:0/1130093844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdba8100260 0x7fdba8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.107+0000 7fdbadc39640 1 -- 172.21.15.94:0/1130093844 >> 172.21.15.94:0/1130093844 conn(0x7fdba8077180 msgr2=0x7fdba8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.107+0000 7fdbadc39640 1 -- 172.21.15.94:0/1130093844 shutdown_connections 2024-02-17T12:05:04.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.107+0000 7fdbadc39640 1 -- 172.21.15.94:0/1130093844 wait complete. 2024-02-17T12:05:04.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.107+0000 7fdbadc39640 1 Processor -- start 2024-02-17T12:05:04.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdbadc39640 1 -- start start 2024-02-17T12:05:04.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdbadc39640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdba8100260 0x7fdba819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdbadc39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdba8105330 0x7fdba819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdbadc39640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba81975d0 0x7fdba8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdbadc39640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba8197fc0 con 0x7fdba81975d0 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdbadc39640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba8198130 con 0x7fdba8105330 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdbadc39640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba81982f0 con 0x7fdba8100260 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdba7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba81975d0 0x7fdba8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdba7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba81975d0 0x7fdba8197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42258/0 (socket says 172.21.15.94:42258) 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdba7fff640 1 -- 172.21.15.94:0/3968594968 learned_addr learned my addr 172.21.15.94:0/3968594968 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdba77fe640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdba8100260 0x7fdba819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdba6ffd640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdba8105330 0x7fdba819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdba77fe640 1 -- 172.21.15.94:0/3968594968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdba8105330 msgr2=0x7fdba819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdba77fe640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdba8105330 0x7fdba819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdba77fe640 1 -- 172.21.15.94:0/3968594968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba81975d0 msgr2=0x7fdba8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdba77fe640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba81975d0 0x7fdba8197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdba77fe640 1 -- 172.21.15.94:0/3968594968 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba0009900 con 0x7fdba8100260 2024-02-17T12:05:04.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdba7fff640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba81975d0 0x7fdba8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:04.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.108+0000 7fdba6ffd640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdba8105330 0x7fdba819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:04.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.109+0000 7fdba77fe640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdba8100260 0x7fdba819d420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800e9f0 tx=0x7fdb9800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.109+0000 7fdba4ff9640 1 -- 172.21.15.94:0/3968594968 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9800ccc0 con 0x7fdba8100260 2024-02-17T12:05:04.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.109+0000 7fdbadc39640 1 -- 172.21.15.94:0/3968594968 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba81a42e0 con 0x7fdba8100260 2024-02-17T12:05:04.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.109+0000 7fdba4ff9640 1 -- 172.21.15.94:0/3968594968 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb9800ce20 con 0x7fdba8100260 2024-02-17T12:05:04.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.109+0000 7fdba4ff9640 1 -- 172.21.15.94:0/3968594968 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98004a00 con 0x7fdba8100260 2024-02-17T12:05:04.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.109+0000 7fdbadc39640 1 -- 172.21.15.94:0/3968594968 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba81a47a0 con 0x7fdba8100260 2024-02-17T12:05:04.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdbb0806740 1 Processor -- start 2024-02-17T12:05:04.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdbb0806740 1 -- start start 2024-02-17T12:05:04.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba4ff9640 1 -- 172.21.15.94:0/3968594968 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fdb980050b0 con 0x7fdba8100260 2024-02-17T12:05:04.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdbb0806740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5c250 0x5559efd5c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdbb0806740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5cb70 0x5559efd65210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdbb0806740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd65750 0x5559efd67b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdbb0806740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559efd17960 con 0x5559efd5c250 2024-02-17T12:05:04.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdbb0806740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559efccb310 con 0x5559efd65750 2024-02-17T12:05:04.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdbb0806740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559efcc8840 con 0x5559efd5cb70 2024-02-17T12:05:04.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5c250 0x5559efd5c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5c250 0x5559efd5c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42270/0 (socket says 172.21.15.94:42270) 2024-02-17T12:05:04.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba7fff640 1 -- 172.21.15.94:0/15776097 learned_addr learned my addr 172.21.15.94:0/15776097 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba77fe640 1 --2- 172.21.15.94:0/15776097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5cb70 0x5559efd65210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba6ffd640 1 --2- 172.21.15.94:0/15776097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd65750 0x5559efd67b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba7fff640 1 -- 172.21.15.94:0/15776097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5cb70 msgr2=0x5559efd65210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba7fff640 1 --2- 172.21.15.94:0/15776097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5cb70 0x5559efd65210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba7fff640 1 -- 172.21.15.94:0/15776097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd65750 msgr2=0x5559efd67b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba7fff640 1 --2- 172.21.15.94:0/15776097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd65750 0x5559efd67b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba7fff640 1 -- 172.21.15.94:0/15776097 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559efc86060 con 0x5559efd5c250 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba6ffd640 1 --2- 172.21.15.94:0/15776097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd65750 0x5559efd67b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba77fe640 1 --2- 172.21.15.94:0/15776097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5cb70 0x5559efd65210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.112+0000 7fdba4ff9640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdb900a2230 0x7fdb900a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdba7fff640 1 --2- 172.21.15.94:0/15776097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5c250 0x5559efd5c630 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdba0034820 tx=0x7fdba0034850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570a98a3955b95 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdba77fe640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdb900a2230 0x7fdb900a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdb8dffb640 1 -- 172.21.15.94:0/15776097 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0032e70 con 0x5559efd5c250 2024-02-17T12:05:04.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdba4ff9640 1 -- 172.21.15.94:0/3968594968 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fdb98014070 con 0x7fdba8100260 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdb8dffb640 1 -- 172.21.15.94:0/15776097 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdba0032340 con 0x5559efd5c250 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdb8dffb640 1 -- 172.21.15.94:0/15776097 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba0003c80 con 0x5559efd5c250 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdbb0806740 1 -- 172.21.15.94:0/15776097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5c250 msgr2=0x5559efd5c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdba77fe640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdb900a2230 0x7fdb900a46f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdb980cb840 tx=0x7fdb98009710 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdbb0806740 1 --2- 172.21.15.94:0/15776097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5c250 0x5559efd5c630 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdba0034820 tx=0x7fdba0034850 comp rx=0 tx=0).stop 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdbb0806740 1 -- 172.21.15.94:0/15776097 shutdown_connections 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdbb0806740 1 --2- 172.21.15.94:0/15776097 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd65750 0x5559efd67b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdbb0806740 1 --2- 172.21.15.94:0/15776097 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5cb70 0x5559efd65210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdbb0806740 1 --2- 172.21.15.94:0/15776097 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5c250 0x5559efd5c630 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.113+0000 7fdbb0806740 1 -- 172.21.15.94:0/15776097 >> 172.21.15.94:0/15776097 conn(0x5559efd56980 msgr2=0x5559efd56d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdbb0806740 1 -- 172.21.15.94:0/15776097 shutdown_connections 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdbb0806740 1 -- 172.21.15.94:0/15776097 wait complete. 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdbb0806740 1 -- start start 2024-02-17T12:05:04.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdbb0806740 1 Processor -- start 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdbb0806740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5c250 0x5559efd627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdbb0806740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 0x5559efd62cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdbb0806740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd65750 0x5559efd63230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdbb0806740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559efcc4020 con 0x5559efd5cb70 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdbb0806740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ef997430 con 0x5559efd65750 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdbb0806740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559ef953e00 con 0x5559efd5c250 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdbb0806740 10 client.0 _ll_register_callbacks cb 0x5559efc6bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdba6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 0x5559efd62cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdba6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 0x5559efd62cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42286/0 (socket says 172.21.15.94:42286) 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdba6ffd640 1 -- 172.21.15.94:0/2268209162 learned_addr learned my addr 172.21.15.94:0/2268209162 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.114+0000 7fdba7fff640 1 --2- 172.21.15.94:0/2268209162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5c250 0x5559efd627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdba7fff640 1 --2- 172.21.15.94:0/2268209162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd65750 0x5559efd63230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdba6ffd640 1 -- 172.21.15.94:0/2268209162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5c250 msgr2=0x5559efd627b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdba6ffd640 1 --2- 172.21.15.94:0/2268209162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5c250 0x5559efd627b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdba6ffd640 1 -- 172.21.15.94:0/2268209162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd65750 msgr2=0x5559efd63230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdba6ffd640 1 --2- 172.21.15.94:0/2268209162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd65750 0x5559efd63230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdba6ffd640 1 -- 172.21.15.94:0/2268209162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba810d710 con 0x5559efd5cb70 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdba6ffd640 1 --2- 172.21.15.94:0/2268209162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 0x5559efd62cf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c012780 tx=0x7fdb9c012c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df59f494182c2605 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdb86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdb86ffd640 1 -- 172.21.15.94:0/2268209162 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c010dc0 con 0x5559efd5cb70 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdb86ffd640 1 -- 172.21.15.94:0/2268209162 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fdb9c004020 con 0x5559efd5cb70 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdbb0806740 1 -- 172.21.15.94:0/2268209162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5559ef9c37e0 con 0x5559efd5cb70 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdb86ffd640 1 -- 172.21.15.94:0/2268209162 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb9c0041e0 con 0x5559efd5cb70 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdbb0806740 10 client.17301 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdbb0806740 1 -- 172.21.15.94:0/2268209162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5559efcc26a0 con 0x5559efd5cb70 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.115+0000 7fdb86ffd640 1 -- 172.21.15.94:0/2268209162 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb9c028080 con 0x5559efd5cb70 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.116+0000 7fdb86ffd640 1 -- 172.21.15.94:0/2268209162 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb9c024410 con 0x5559efd5cb70 2024-02-17T12:05:04.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.116+0000 7fdbb0806740 10 client.17301 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.116+0000 7fdbb0806740 10 client.17301 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.116+0000 7fdbb0806740 20 client.17301 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.116+0000 7fdbb0806740 1 --2- 172.21.15.94:0/2268209162 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5559efd75c70 0x5559efd78090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.116+0000 7fdbb0806740 4 client.17301 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.116+0000 7fdbb0806740 1 -- 172.21.15.94:0/2268209162 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559efa00720 con 0x5559efd75c70 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.116+0000 7fdba7fff640 1 --2- 172.21.15.94:0/2268209162 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5559efd75c70 0x5559efd78090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.116+0000 7fdba7fff640 1 --2- 172.21.15.94:0/2268209162 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5559efd75c70 0x5559efd78090 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9440843f5e551c78 server_cookie=95d04e03e624fe5f in_seq=0 out_seq=0 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.116+0000 7fdb86ffd640 10 client.17301 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdb86ffd640 1 -- 172.21.15.94:0/2268209162 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5559efa00720 con 0x5559efd75c70 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdb86ffd640 10 client.17301 handle_command_reply: tid=0 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdbb0806740 1 client.17301 shutdown 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdbb0806740 1 -- 172.21.15.94:0/2268209162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 msgr2=0x5559efd62cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdbb0806740 1 --2- 172.21.15.94:0/2268209162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 0x5559efd62cf0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c012780 tx=0x7fdb9c012c40 comp rx=0 tx=0).stop 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdbb0806740 1 -- 172.21.15.94:0/2268209162 shutdown_connections 2024-02-17T12:05:04.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdbb0806740 1 --2- 172.21.15.94:0/2268209162 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5559efd75c70 0x5559efd78090 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdbb0806740 1 --2- 172.21.15.94:0/2268209162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd65750 0x5559efd63230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdbb0806740 1 --2- 172.21.15.94:0/2268209162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 0x5559efd62cf0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdb86ffd640 0 client.17301 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:05:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdbb0806740 1 --2- 172.21.15.94:0/2268209162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5c250 0x5559efd627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdbb0806740 1 -- 172.21.15.94:0/2268209162 >> 172.21.15.94:0/2268209162 conn(0x5559efd56980 msgr2=0x5559efd586d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdbb0806740 1 -- 172.21.15.94:0/2268209162 shutdown_connections 2024-02-17T12:05:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdbb0806740 1 -- 172.21.15.94:0/2268209162 wait complete. 2024-02-17T12:05:04.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.117+0000 7fdbb0806740 20 client.17301 trim_cache size 0 max 16384 2024-02-17T12:05:04.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.130+0000 7fdbb0806740 1 Processor -- start 2024-02-17T12:05:04.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.130+0000 7fdbb0806740 1 -- start start 2024-02-17T12:05:04.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdbb0806740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5c250 0x5559efd64ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdbb0806740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd5cb70 0x5559efd63050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdbb0806740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd65750 0x5559efd5fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdbb0806740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559efc49130 con 0x5559efd65750 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdbb0806740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559efd52310 con 0x5559efd5cb70 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdbb0806740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559efd67f30 con 0x5559efd5c250 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdba7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd65750 0x5559efd5fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdba7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd65750 0x5559efd5fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42288/0 (socket says 172.21.15.94:42288) 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdba7fff640 1 -- 172.21.15.94:0/3250022556 learned_addr learned my addr 172.21.15.94:0/3250022556 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdba7fff640 1 --2- 172.21.15.94:0/3250022556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5c250 0x5559efd64ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdba6ffd640 1 --2- 172.21.15.94:0/3250022556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd5cb70 0x5559efd63050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdba7fff640 1 -- 172.21.15.94:0/3250022556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5c250 msgr2=0x5559efd64ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdba7fff640 1 --2- 172.21.15.94:0/3250022556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5c250 0x5559efd64ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdba7fff640 1 -- 172.21.15.94:0/3250022556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd5cb70 msgr2=0x5559efd63050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdba7fff640 1 --2- 172.21.15.94:0/3250022556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd5cb70 0x5559efd63050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdba7fff640 1 -- 172.21.15.94:0/3250022556 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559efc5ec20 con 0x5559efd65750 2024-02-17T12:05:04.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdba7fff640 1 --2- 172.21.15.94:0/3250022556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd65750 0x5559efd5fa10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdba002c6d0 tx=0x7fdba002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5778ec9c2a09db10 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdb8d7fa640 1 -- 172.21.15.94:0/3250022556 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba004d070 con 0x5559efd65750 2024-02-17T12:05:04.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.131+0000 7fdb8d7fa640 1 -- 172.21.15.94:0/3250022556 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdba00573c0 con 0x5559efd65750 2024-02-17T12:05:04.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.132+0000 7fdb8d7fa640 1 -- 172.21.15.94:0/3250022556 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba003f6a0 con 0x5559efd65750 2024-02-17T12:05:04.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.132+0000 7fdbb0806740 1 -- 172.21.15.94:0/3250022556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd65750 msgr2=0x5559efd5fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.132+0000 7fdbb0806740 1 --2- 172.21.15.94:0/3250022556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd65750 0x5559efd5fa10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdba002c6d0 tx=0x7fdba002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.132+0000 7fdbb0806740 1 -- 172.21.15.94:0/3250022556 shutdown_connections 2024-02-17T12:05:04.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.132+0000 7fdbb0806740 1 --2- 172.21.15.94:0/3250022556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd65750 0x5559efd5fa10 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.132+0000 7fdbb0806740 1 --2- 172.21.15.94:0/3250022556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd5cb70 0x5559efd63050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.132+0000 7fdbb0806740 1 --2- 172.21.15.94:0/3250022556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd5c250 0x5559efd64ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.132+0000 7fdbb0806740 1 -- 172.21.15.94:0/3250022556 >> 172.21.15.94:0/3250022556 conn(0x5559efd56980 msgr2=0x5559efd61080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.132+0000 7fdbb0806740 1 -- 172.21.15.94:0/3250022556 shutdown_connections 2024-02-17T12:05:04.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.132+0000 7fdbb0806740 1 -- 172.21.15.94:0/3250022556 wait complete. 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdbb0806740 1 -- start start 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdbb0806740 1 Processor -- start 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdbb0806740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd5c250 0x5559efd6b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdbb0806740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 0x5559efd6b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdbb0806740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd65750 0x5559efdcc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdbb0806740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559efd6bf70 con 0x5559efd5cb70 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdbb0806740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559efd6c0e0 con 0x5559efd5c250 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdbb0806740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559efdcc6f0 con 0x5559efd65750 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdbb0806740 10 client.0 _ll_register_callbacks cb 0x7fdb98005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdba6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 0x5559efd6b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdba6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 0x5559efd6b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42298/0 (socket says 172.21.15.94:42298) 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdba7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd5c250 0x5559efd6b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdba6ffd640 1 -- 172.21.15.94:0/3133364012 learned_addr learned my addr 172.21.15.94:0/3133364012 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdba7fff640 1 --2- 172.21.15.94:0/3133364012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd65750 0x5559efdcc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdba6ffd640 1 -- 172.21.15.94:0/3133364012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd65750 msgr2=0x5559efdcc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdba6ffd640 1 --2- 172.21.15.94:0/3133364012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd65750 0x5559efdcc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdba6ffd640 1 -- 172.21.15.94:0/3133364012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd5c250 msgr2=0x5559efd6b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdba6ffd640 1 --2- 172.21.15.94:0/3133364012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd5c250 0x5559efd6b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdba6ffd640 1 -- 172.21.15.94:0/3133364012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba0050040 con 0x5559efd5cb70 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.133+0000 7fdba7fff640 1 --2- 172.21.15.94:0/3133364012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd5c250 0x5559efd6b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdba6ffd640 1 --2- 172.21.15.94:0/3133364012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 0x5559efd6b7a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x5559efd62010 tx=0x7fdb9c005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5079081d331720 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdb86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdb86ffd640 1 -- 172.21.15.94:0/3133364012 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c024be0 con 0x5559efd5cb70 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdb86ffd640 1 -- 172.21.15.94:0/3133364012 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fdb9c02b050 con 0x5559efd5cb70 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdbb0806740 1 -- 172.21.15.94:0/3133364012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdba0032340 con 0x5559efd5cb70 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdb86ffd640 1 -- 172.21.15.94:0/3133364012 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb9c010ed0 con 0x5559efd5cb70 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdbb0806740 10 client.17307 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdbb0806740 1 -- 172.21.15.94:0/3133364012 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5559efc848c0 con 0x5559efd5cb70 2024-02-17T12:05:04.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdb86ffd640 1 -- 172.21.15.94:0/3133364012 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb9c03b080 con 0x5559efd5cb70 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdb86ffd640 1 -- 172.21.15.94:0/3133364012 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb9c02d070 con 0x5559efd5cb70 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdbb0806740 10 client.17307 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdbb0806740 10 client.17307 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdbb0806740 20 client.17307 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdbb0806740 1 --2- 172.21.15.94:0/3133364012 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5559efdcdeb0 0x5559efdd02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdbb0806740 4 client.17307 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.134+0000 7fdbb0806740 1 -- 172.21.15.94:0/3133364012 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5559efa4f890 con 0x5559efdcdeb0 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdba7fff640 1 --2- 172.21.15.94:0/3133364012 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5559efdcdeb0 0x5559efdd02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdba7fff640 1 --2- 172.21.15.94:0/3133364012 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5559efdcdeb0 0x5559efdd02d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3608ca291dfe2593 server_cookie=3a8ee2fe0b7656c9 in_seq=0 out_seq=0 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdb86ffd640 10 client.17307 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdb86ffd640 1 -- 172.21.15.94:0/3133364012 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5559efa4f890 con 0x5559efdcdeb0 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdb86ffd640 10 client.17307 handle_command_reply: tid=0 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdbb0806740 1 client.17307 shutdown 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdbb0806740 1 -- 172.21.15.94:0/3133364012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 msgr2=0x5559efd6b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdbb0806740 1 --2- 172.21.15.94:0/3133364012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 0x5559efd6b7a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x5559efd62010 tx=0x7fdb9c005990 comp rx=0 tx=0).stop 2024-02-17T12:05:04.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdbb0806740 1 -- 172.21.15.94:0/3133364012 shutdown_connections 2024-02-17T12:05:04.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdbb0806740 1 --2- 172.21.15.94:0/3133364012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5559efd65750 0x5559efdcc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdbb0806740 1 --2- 172.21.15.94:0/3133364012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5559efd5cb70 0x5559efd6b7a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdbb0806740 1 --2- 172.21.15.94:0/3133364012 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5559efdcdeb0 0x5559efdd02d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdbb0806740 1 --2- 172.21.15.94:0/3133364012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5559efd5c250 0x5559efd6b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdb86ffd640 0 client.17307 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:05:04.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdbb0806740 1 -- 172.21.15.94:0/3133364012 >> 172.21.15.94:0/3133364012 conn(0x5559efd56980 msgr2=0x5559efd552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdbb0806740 1 -- 172.21.15.94:0/3133364012 shutdown_connections 2024-02-17T12:05:04.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.135+0000 7fdbb0806740 1 -- 172.21.15.94:0/3133364012 wait complete. 2024-02-17T12:05:04.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.136+0000 7fdbb0806740 20 client.17307 trim_cache size 0 max 16384 2024-02-17T12:05:04.142 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:05:04.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.136+0000 7fdbadc39640 1 -- 172.21.15.94:0/3968594968 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdb900a2230 msgr2=0x7fdb900a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.136+0000 7fdbadc39640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdb900a2230 0x7fdb900a46f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdb980cb840 tx=0x7fdb98009710 comp rx=0 tx=0).stop 2024-02-17T12:05:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.136+0000 7fdbadc39640 1 -- 172.21.15.94:0/3968594968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdba8100260 msgr2=0x7fdba819d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.136+0000 7fdbadc39640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdba8100260 0x7fdba819d420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800e9f0 tx=0x7fdb9800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.136+0000 7fdbadc39640 1 -- 172.21.15.94:0/3968594968 shutdown_connections 2024-02-17T12:05:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.136+0000 7fdbadc39640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdb900a2230 0x7fdb900a46f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.136+0000 7fdbadc39640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdba81975d0 0x7fdba8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.136+0000 7fdbadc39640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdba8105330 0x7fdba819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.136+0000 7fdbadc39640 1 --2- 172.21.15.94:0/3968594968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdba8100260 0x7fdba819d420 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.136+0000 7fdbadc39640 1 -- 172.21.15.94:0/3968594968 >> 172.21.15.94:0/3968594968 conn(0x7fdba8077180 msgr2=0x7fdba8068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.136+0000 7fdbadc39640 1 -- 172.21.15.94:0/3968594968 shutdown_connections 2024-02-17T12:05:04.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.136+0000 7fdbadc39640 1 -- 172.21.15.94:0/3968594968 wait complete. 2024-02-17T12:05:04.148 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:05:04.148 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:05:04.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:03 smithi094 ceph-mon[32499]: pgmap v212: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 8.3 MiB/s wr, 662 op/s 2024-02-17T12:05:04.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.261+0000 7f3865b0d640 1 -- 172.21.15.94:0/3822284129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38600685d0 msgr2=0x7f38600689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.261+0000 7f3865b0d640 1 --2- 172.21.15.94:0/3822284129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38600685d0 0x7f38600689b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3850009930 tx=0x7f385002bc30 comp rx=0 tx=0).stop 2024-02-17T12:05:04.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.261+0000 7f3865b0d640 1 -- 172.21.15.94:0/3822284129 shutdown_connections 2024-02-17T12:05:04.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.261+0000 7f3865b0d640 1 --2- 172.21.15.94:0/3822284129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3860106430 0x7f386010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.261+0000 7f3865b0d640 1 --2- 172.21.15.94:0/3822284129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3860068f80 0x7f3860105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.261+0000 7f3865b0d640 1 --2- 172.21.15.94:0/3822284129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f38600685d0 0x7f38600689b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.261+0000 7f3865b0d640 1 -- 172.21.15.94:0/3822284129 >> 172.21.15.94:0/3822284129 conn(0x7f3860077180 msgr2=0x7f3860077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.261+0000 7f3865b0d640 1 -- 172.21.15.94:0/3822284129 shutdown_connections 2024-02-17T12:05:04.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.261+0000 7f3865b0d640 1 -- 172.21.15.94:0/3822284129 wait complete. 2024-02-17T12:05:04.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f3865b0d640 1 Processor -- start 2024-02-17T12:05:04.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f3865b0d640 1 -- start start 2024-02-17T12:05:04.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f3865b0d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38600685d0 0x7f386019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f3865b0d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3860106430 0x7f386019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f3865b0d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38601975e0 0x7f3860197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f3865b0d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3860197fd0 con 0x7f3860106430 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f3865b0d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3860198140 con 0x7f38600685d0 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f3865b0d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3860198300 con 0x7f38601975e0 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f385effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3860106430 0x7f386019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f385effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3860106430 0x7f386019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42312/0 (socket says 172.21.15.94:42312) 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f385effd640 1 -- 172.21.15.94:0/1603618568 learned_addr learned my addr 172.21.15.94:0/1603618568 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f385ffff640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38601975e0 0x7f3860197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f385f7fe640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38600685d0 0x7f386019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f385effd640 1 -- 172.21.15.94:0/1603618568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38601975e0 msgr2=0x7f3860197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f385effd640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38601975e0 0x7f3860197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f385effd640 1 -- 172.21.15.94:0/1603618568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38600685d0 msgr2=0x7f386019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f385effd640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38600685d0 0x7f386019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f385effd640 1 -- 172.21.15.94:0/1603618568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3854009490 con 0x7f3860106430 2024-02-17T12:05:04.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.262+0000 7f385ffff640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38601975e0 0x7f3860197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:04.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.263+0000 7f385effd640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3860106430 0x7f386019d970 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f385400e9a0 tx=0x7f385400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.263+0000 7f385cff9640 1 -- 172.21.15.94:0/1603618568 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400cd80 con 0x7f3860106430 2024-02-17T12:05:04.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.263+0000 7f3865b0d640 1 -- 172.21.15.94:0/1603618568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3850009560 con 0x7f3860106430 2024-02-17T12:05:04.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.263+0000 7f385cff9640 1 -- 172.21.15.94:0/1603618568 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38540035f0 con 0x7f3860106430 2024-02-17T12:05:04.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.263+0000 7f385cff9640 1 -- 172.21.15.94:0/1603618568 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3854010690 con 0x7f3860106430 2024-02-17T12:05:04.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.263+0000 7f3865b0d640 1 -- 172.21.15.94:0/1603618568 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38601a46c0 con 0x7f3860106430 2024-02-17T12:05:04.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.265+0000 7f385cff9640 1 -- 172.21.15.94:0/1603618568 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3854003ad0 con 0x7f3860106430 2024-02-17T12:05:04.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f38687ba740 1 Processor -- start 2024-02-17T12:05:04.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f38687ba740 1 -- start start 2024-02-17T12:05:04.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f38687ba740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db2fe360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f38687ba740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db2fe8a0 0x5593db30b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f38687ba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db30bb60 0x5593db30df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f38687ba740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593db2b9960 con 0x5593db2fdf80 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f38687ba740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593db26d310 con 0x5593db2fe8a0 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f38687ba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593db26a840 con 0x5593db30bb60 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f385cff9640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f38480a2230 0x7f38480a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f385effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db30bb60 0x5593db30df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f385ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db2fe360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f385effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db30bb60 0x5593db30df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:36602/0 (socket says 172.21.15.94:36602) 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f385effd640 1 -- 172.21.15.94:0/1138490051 learned_addr learned my addr 172.21.15.94:0/1138490051 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f385f7fe640 1 --2- 172.21.15.94:0/1138490051 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db2fe8a0 0x5593db30b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.266+0000 7f385ffff640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f38480a2230 0x7f38480a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.267+0000 7f385cff9640 1 -- 172.21.15.94:0/1603618568 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3854014070 con 0x7f3860106430 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.267+0000 7f385effd640 1 -- 172.21.15.94:0/1138490051 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db2fe8a0 msgr2=0x5593db30b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.267+0000 7f385effd640 1 --2- 172.21.15.94:0/1138490051 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db2fe8a0 0x5593db30b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.267+0000 7f385effd640 1 -- 172.21.15.94:0/1138490051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 msgr2=0x5593db2fe360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.267+0000 7f385effd640 1 --2- 172.21.15.94:0/1138490051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db2fe360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.267+0000 7f385effd640 1 -- 172.21.15.94:0/1138490051 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593db228060 con 0x5593db30bb60 2024-02-17T12:05:04.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.267+0000 7f385effd640 1 --2- 172.21.15.94:0/1138490051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db30bb60 0x5593db30df50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f385400c6a0 tx=0x7f3854003ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6359754bfee37a1f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.267+0000 7f385ffff640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f38480a2230 0x7f38480a46f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f38580099e0 tx=0x7f3858009500 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.267+0000 7f3845ffb640 1 -- 172.21.15.94:0/1138490051 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385409b5b0 con 0x5593db30bb60 2024-02-17T12:05:04.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.267+0000 7f3845ffb640 1 -- 172.21.15.94:0/1138490051 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f385409aeb0 con 0x5593db30bb60 2024-02-17T12:05:04.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.267+0000 7f3845ffb640 1 -- 172.21.15.94:0/1138490051 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385409a300 con 0x5593db30bb60 2024-02-17T12:05:04.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.268+0000 7f38687ba740 1 -- 172.21.15.94:0/1138490051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db30bb60 msgr2=0x5593db30df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.268+0000 7f38687ba740 1 --2- 172.21.15.94:0/1138490051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db30bb60 0x5593db30df50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f385400c6a0 tx=0x7f3854003ed0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.268+0000 7f38687ba740 1 -- 172.21.15.94:0/1138490051 shutdown_connections 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.268+0000 7f38687ba740 1 --2- 172.21.15.94:0/1138490051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db30bb60 0x5593db30df50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.268+0000 7f38687ba740 1 --2- 172.21.15.94:0/1138490051 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db2fe8a0 0x5593db30b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.268+0000 7f38687ba740 1 --2- 172.21.15.94:0/1138490051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db2fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.268+0000 7f38687ba740 1 -- 172.21.15.94:0/1138490051 >> 172.21.15.94:0/1138490051 conn(0x5593db2f8980 msgr2=0x5593db2f8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.268+0000 7f38687ba740 1 -- 172.21.15.94:0/1138490051 shutdown_connections 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.268+0000 7f38687ba740 1 -- 172.21.15.94:0/1138490051 wait complete. 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f38687ba740 1 -- start start 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f38687ba740 1 Processor -- start 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f38687ba740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db308c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f38687ba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db2fe8a0 0x5593db309150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f38687ba740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db30bb60 0x5593db309690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f38687ba740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593db266020 con 0x5593db2fdf80 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f38687ba740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593daf39430 con 0x5593db30bb60 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f38687ba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593daef5e00 con 0x5593db2fe8a0 2024-02-17T12:05:04.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f38687ba740 10 client.0 _ll_register_callbacks cb 0x5593db20dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f385f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db308c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f385effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db2fe8a0 0x5593db309150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f385f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db308c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42344/0 (socket says 172.21.15.94:42344) 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f385ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db30bb60 0x5593db309690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.269+0000 7f385f7fe640 1 -- 172.21.15.94:0/623810003 learned_addr learned my addr 172.21.15.94:0/623810003 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f385f7fe640 1 -- 172.21.15.94:0/623810003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db2fe8a0 msgr2=0x5593db309150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f385f7fe640 1 --2- 172.21.15.94:0/623810003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db2fe8a0 0x5593db309150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f385f7fe640 1 -- 172.21.15.94:0/623810003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db30bb60 msgr2=0x5593db309690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f385f7fe640 1 --2- 172.21.15.94:0/623810003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db30bb60 0x5593db309690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f385f7fe640 1 -- 172.21.15.94:0/623810003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38601a46c0 con 0x5593db2fdf80 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f385effd640 1 --2- 172.21.15.94:0/623810003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db2fe8a0 0x5593db309150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f385ffff640 1 --2- 172.21.15.94:0/623810003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db30bb60 0x5593db309690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f385f7fe640 1 --2- 172.21.15.94:0/623810003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db308c10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3850009900 tx=0x7f385002ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3619abb8c64c0945 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f383effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f383effd640 1 -- 172.21.15.94:0/623810003 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385002c640 con 0x5593db2fdf80 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f383effd640 1 -- 172.21.15.94:0/623810003 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3850004040 con 0x5593db2fdf80 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f38687ba740 1 -- 172.21.15.94:0/623810003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593daf657e0 con 0x5593db2fdf80 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f383effd640 1 -- 172.21.15.94:0/623810003 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3850004240 con 0x5593db2fdf80 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f38687ba740 10 client.17319 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.270+0000 7f38687ba740 1 -- 172.21.15.94:0/623810003 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5593db2646a0 con 0x5593db2fdf80 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.271+0000 7f383effd640 1 -- 172.21.15.94:0/623810003 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5593daf657e0 con 0x5593db2fdf80 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.271+0000 7f383effd640 1 -- 172.21.15.94:0/623810003 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f385003c040 con 0x5593db2fdf80 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.271+0000 7f38687ba740 10 client.17319 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.271+0000 7f38687ba740 10 client.17319 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.271+0000 7f38687ba740 20 client.17319 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.271+0000 7f38687ba740 1 --2- 172.21.15.94:0/623810003 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5593db31c010 0x5593db31e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.271+0000 7f38687ba740 4 client.17319 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:04.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.271+0000 7f38687ba740 1 -- 172.21.15.94:0/623810003 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593dafa2720 con 0x5593db31c010 2024-02-17T12:05:04.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.271+0000 7f385effd640 1 --2- 172.21.15.94:0/623810003 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5593db31c010 0x5593db31e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.271+0000 7f383effd640 10 client.17319 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:05:04.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.271+0000 7f385effd640 1 --2- 172.21.15.94:0/623810003 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5593db31c010 0x5593db31e400 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f76061f6d4018648 server_cookie=10f02c0ecbf8bc5d in_seq=0 out_seq=0 2024-02-17T12:05:04.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f383effd640 1 -- 172.21.15.94:0/623810003 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5593dafa2720 con 0x5593db31c010 2024-02-17T12:05:04.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f383effd640 10 client.17319 handle_command_reply: tid=0 2024-02-17T12:05:04.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f38687ba740 1 client.17319 shutdown 2024-02-17T12:05:04.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f38687ba740 1 -- 172.21.15.94:0/623810003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 msgr2=0x5593db308c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f38687ba740 1 --2- 172.21.15.94:0/623810003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db308c10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f3850009900 tx=0x7f385002ca10 comp rx=0 tx=0).stop 2024-02-17T12:05:04.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f38687ba740 1 -- 172.21.15.94:0/623810003 shutdown_connections 2024-02-17T12:05:04.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f38687ba740 1 --2- 172.21.15.94:0/623810003 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5593db31c010 0x5593db31e400 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f38687ba740 1 --2- 172.21.15.94:0/623810003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db30bb60 0x5593db309690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f38687ba740 1 --2- 172.21.15.94:0/623810003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db2fe8a0 0x5593db309150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f38687ba740 1 --2- 172.21.15.94:0/623810003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db308c10 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f38687ba740 1 -- 172.21.15.94:0/623810003 >> 172.21.15.94:0/623810003 conn(0x5593db2f8980 msgr2=0x5593db2fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f383effd640 0 client.17319 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:05:04.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f38687ba740 1 -- 172.21.15.94:0/623810003 shutdown_connections 2024-02-17T12:05:04.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.272+0000 7f38687ba740 1 -- 172.21.15.94:0/623810003 wait complete. 2024-02-17T12:05:04.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.273+0000 7f38687ba740 20 client.17319 trim_cache size 0 max 16384 2024-02-17T12:05:04.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.284+0000 7f38687ba740 1 Processor -- start 2024-02-17T12:05:04.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.284+0000 7f38687ba740 1 -- start start 2024-02-17T12:05:04.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f38687ba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db2fdf80 0x5593db30b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f38687ba740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fe8a0 0x5593db3094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f38687ba740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db30bb60 0x5593db322d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f38687ba740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593db1eb130 con 0x5593db2fe8a0 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f38687ba740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593db2fd390 con 0x5593db30bb60 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f38687ba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593db2f45c0 con 0x5593db2fdf80 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f385effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fe8a0 0x5593db3094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f385effd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fe8a0 0x5593db3094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42352/0 (socket says 172.21.15.94:42352) 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f385effd640 1 -- 172.21.15.94:0/1657143799 learned_addr learned my addr 172.21.15.94:0/1657143799 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f385ffff640 1 --2- 172.21.15.94:0/1657143799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db30bb60 0x5593db322d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f385f7fe640 1 --2- 172.21.15.94:0/1657143799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db2fdf80 0x5593db30b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f385effd640 1 -- 172.21.15.94:0/1657143799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db2fdf80 msgr2=0x5593db30b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f385effd640 1 --2- 172.21.15.94:0/1657143799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db2fdf80 0x5593db30b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f385effd640 1 -- 172.21.15.94:0/1657143799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db30bb60 msgr2=0x5593db322d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f385effd640 1 --2- 172.21.15.94:0/1657143799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db30bb60 0x5593db322d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f385effd640 1 -- 172.21.15.94:0/1657143799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593db200c20 con 0x5593db2fe8a0 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f385effd640 1 --2- 172.21.15.94:0/1657143799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fe8a0 0x5593db3094b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f38540cc210 tx=0x7f38540a3650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d2e705453160d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f38457fa640 1 -- 172.21.15.94:0/1657143799 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385409b5b0 con 0x5593db2fe8a0 2024-02-17T12:05:04.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f38457fa640 1 -- 172.21.15.94:0/1657143799 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38540cbe90 con 0x5593db2fe8a0 2024-02-17T12:05:04.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.285+0000 7f38457fa640 1 -- 172.21.15.94:0/1657143799 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38540b9390 con 0x5593db2fe8a0 2024-02-17T12:05:04.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.286+0000 7f38687ba740 1 -- 172.21.15.94:0/1657143799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fe8a0 msgr2=0x5593db3094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.286+0000 7f38687ba740 1 --2- 172.21.15.94:0/1657143799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fe8a0 0x5593db3094b0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f38540cc210 tx=0x7f38540a3650 comp rx=0 tx=0).stop 2024-02-17T12:05:04.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.286+0000 7f38687ba740 1 -- 172.21.15.94:0/1657143799 shutdown_connections 2024-02-17T12:05:04.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.286+0000 7f38687ba740 1 --2- 172.21.15.94:0/1657143799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db30bb60 0x5593db322d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.286+0000 7f38687ba740 1 --2- 172.21.15.94:0/1657143799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fe8a0 0x5593db3094b0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.286+0000 7f38687ba740 1 --2- 172.21.15.94:0/1657143799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db2fdf80 0x5593db30b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.286+0000 7f38687ba740 1 -- 172.21.15.94:0/1657143799 >> 172.21.15.94:0/1657143799 conn(0x5593db2f8980 msgr2=0x5593db303f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.286+0000 7f38687ba740 1 -- 172.21.15.94:0/1657143799 shutdown_connections 2024-02-17T12:05:04.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.286+0000 7f38687ba740 1 -- 172.21.15.94:0/1657143799 wait complete. 2024-02-17T12:05:04.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f38687ba740 1 -- start start 2024-02-17T12:05:04.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f38687ba740 1 Processor -- start 2024-02-17T12:05:04.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f38687ba740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db312980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f38687ba740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db2fe8a0 0x5593db312ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f38687ba740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db30bb60 0x5593db310ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.293 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f38687ba740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593db311570 con 0x5593db2fdf80 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f38687ba740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593db3116e0 con 0x5593db2fe8a0 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f38687ba740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593db3118a0 con 0x5593db30bb60 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f38687ba740 10 client.0 _ll_register_callbacks cb 0x7f385409ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f385f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db312980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f385f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db312980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42358/0 (socket says 172.21.15.94:42358) 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f385f7fe640 1 -- 172.21.15.94:0/786942122 learned_addr learned my addr 172.21.15.94:0/786942122 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f385effd640 1 --2- 172.21.15.94:0/786942122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db2fe8a0 0x5593db312ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f385ffff640 1 --2- 172.21.15.94:0/786942122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db30bb60 0x5593db310ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f385f7fe640 1 -- 172.21.15.94:0/786942122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db30bb60 msgr2=0x5593db310ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f385f7fe640 1 --2- 172.21.15.94:0/786942122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db30bb60 0x5593db310ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f385f7fe640 1 -- 172.21.15.94:0/786942122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db2fe8a0 msgr2=0x5593db312ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f385f7fe640 1 --2- 172.21.15.94:0/786942122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db2fe8a0 0x5593db312ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.287+0000 7f385f7fe640 1 -- 172.21.15.94:0/786942122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593db228060 con 0x5593db2fdf80 2024-02-17T12:05:04.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f385f7fe640 1 --2- 172.21.15.94:0/786942122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db312980 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3850004460 tx=0x7f3850004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e43881bee2d6571 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f383effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f383effd640 1 -- 172.21.15.94:0/786942122 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385002c5d0 con 0x5593db2fdf80 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f383effd640 1 -- 172.21.15.94:0/786942122 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3850000e40 con 0x5593db2fdf80 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f38687ba740 1 -- 172.21.15.94:0/786942122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f385409aeb0 con 0x5593db2fdf80 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f383effd640 1 -- 172.21.15.94:0/786942122 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3850003ae0 con 0x5593db2fdf80 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f38687ba740 10 client.17325 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f38687ba740 1 -- 172.21.15.94:0/786942122 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5593db2268c0 con 0x5593db2fdf80 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f383effd640 1 -- 172.21.15.94:0/786942122 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3850062020 con 0x5593db2fdf80 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f383effd640 1 -- 172.21.15.94:0/786942122 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f385003c040 con 0x5593db2fdf80 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f38687ba740 10 client.17325 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f38687ba740 10 client.17325 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f38687ba740 20 client.17325 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f38687ba740 1 --2- 172.21.15.94:0/786942122 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5593db373470 0x5593db375890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f38687ba740 4 client.17325 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:05:04.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f38687ba740 1 -- 172.21.15.94:0/786942122 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5593daff1890 con 0x5593db373470 2024-02-17T12:05:04.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.288+0000 7f385effd640 1 --2- 172.21.15.94:0/786942122 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5593db373470 0x5593db375890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f383effd640 10 client.17325 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:05:04.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f385effd640 1 --2- 172.21.15.94:0/786942122 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5593db373470 0x5593db375890 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6083fb016f99bc88 server_cookie=d510b2672221b1d0 in_seq=0 out_seq=0 2024-02-17T12:05:04.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f383effd640 1 -- 172.21.15.94:0/786942122 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5593daff1890 con 0x5593db373470 2024-02-17T12:05:04.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f383effd640 10 client.17325 handle_command_reply: tid=0 2024-02-17T12:05:04.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f38687ba740 1 client.17325 shutdown 2024-02-17T12:05:04.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f38687ba740 1 -- 172.21.15.94:0/786942122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 msgr2=0x5593db312980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f38687ba740 1 --2- 172.21.15.94:0/786942122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db312980 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3850004460 tx=0x7f3850004490 comp rx=0 tx=0).stop 2024-02-17T12:05:04.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f38687ba740 1 -- 172.21.15.94:0/786942122 shutdown_connections 2024-02-17T12:05:04.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f38687ba740 1 --2- 172.21.15.94:0/786942122 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5593db373470 0x5593db375890 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f38687ba740 1 --2- 172.21.15.94:0/786942122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593db30bb60 0x5593db310ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f383effd640 0 client.17325 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:05:04.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f38687ba740 1 --2- 172.21.15.94:0/786942122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593db2fe8a0 0x5593db312ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f38687ba740 1 --2- 172.21.15.94:0/786942122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593db2fdf80 0x5593db312980 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f38687ba740 1 -- 172.21.15.94:0/786942122 >> 172.21.15.94:0/786942122 conn(0x5593db2f8980 msgr2=0x5593db322d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f38687ba740 1 -- 172.21.15.94:0/786942122 shutdown_connections 2024-02-17T12:05:04.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f38687ba740 1 -- 172.21.15.94:0/786942122 wait complete. 2024-02-17T12:05:04.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.289+0000 7f38687ba740 20 client.17325 trim_cache size 0 max 16384 2024-02-17T12:05:04.297 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:05:04.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.290+0000 7f3865b0d640 1 -- 172.21.15.94:0/1603618568 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f38480a2230 msgr2=0x7f38480a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.290+0000 7f3865b0d640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f38480a2230 0x7f38480a46f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f38580099e0 tx=0x7f3858009500 comp rx=0 tx=0).stop 2024-02-17T12:05:04.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.290+0000 7f3865b0d640 1 -- 172.21.15.94:0/1603618568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3860106430 msgr2=0x7f386019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.290+0000 7f3865b0d640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3860106430 0x7f386019d970 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f385400e9a0 tx=0x7f385400ee60 comp rx=0 tx=0).stop 2024-02-17T12:05:04.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.290+0000 7f3865b0d640 1 -- 172.21.15.94:0/1603618568 shutdown_connections 2024-02-17T12:05:04.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.290+0000 7f3865b0d640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f38480a2230 0x7f38480a46f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.290+0000 7f3865b0d640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38601975e0 0x7f3860197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.290+0000 7f3865b0d640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3860106430 0x7f386019d970 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.290+0000 7f3865b0d640 1 --2- 172.21.15.94:0/1603618568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f38600685d0 0x7f386019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.290+0000 7f3865b0d640 1 -- 172.21.15.94:0/1603618568 >> 172.21.15.94:0/1603618568 conn(0x7f3860077180 msgr2=0x7f38600fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.290+0000 7f3865b0d640 1 -- 172.21.15.94:0/1603618568 shutdown_connections 2024-02-17T12:05:04.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.290+0000 7f3865b0d640 1 -- 172.21.15.94:0/1603618568 wait complete. 2024-02-17T12:05:04.302 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:05:04.302 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:05:04.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:03 smithi189 ceph-mon[36588]: pgmap v212: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 8.3 MiB/s wr, 662 op/s 2024-02-17T12:05:04.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.414+0000 7fd75f5f6640 1 -- 172.21.15.94:0/4006339469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd758106450 msgr2=0x7fd75810e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.414+0000 7fd75f5f6640 1 --2- 172.21.15.94:0/4006339469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd758106450 0x7fd75810e020 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd750009cd0 tx=0x7fd75002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.414+0000 7fd75f5f6640 1 -- 172.21.15.94:0/4006339469 shutdown_connections 2024-02-17T12:05:04.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.414+0000 7fd75f5f6640 1 --2- 172.21.15.94:0/4006339469 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd758106450 0x7fd75810e020 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.414+0000 7fd75f5f6640 1 --2- 172.21.15.94:0/4006339469 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd758101d30 0x7fd758105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.414+0000 7fd75f5f6640 1 --2- 172.21.15.94:0/4006339469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd758101380 0x7fd758101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.414+0000 7fd75f5f6640 1 -- 172.21.15.94:0/4006339469 >> 172.21.15.94:0/4006339469 conn(0x7fd758078580 msgr2=0x7fd758078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.414+0000 7fd75f5f6640 1 -- 172.21.15.94:0/4006339469 shutdown_connections 2024-02-17T12:05:04.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.414+0000 7fd75f5f6640 1 -- 172.21.15.94:0/4006339469 wait complete. 2024-02-17T12:05:04.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75f5f6640 1 Processor -- start 2024-02-17T12:05:04.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75f5f6640 1 -- start start 2024-02-17T12:05:04.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75f5f6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd758101380 0x7fd75819d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75f5f6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd758106450 0x7fd75819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75f5f6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd758197600 0x7fd758197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75f5f6640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758197ff0 con 0x7fd758197600 2024-02-17T12:05:04.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75f5f6640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758198160 con 0x7fd758106450 2024-02-17T12:05:04.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75f5f6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758198320 con 0x7fd758101380 2024-02-17T12:05:04.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75db93640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd758197600 0x7fd758197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75db93640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd758197600 0x7fd758197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42378/0 (socket says 172.21.15.94:42378) 2024-02-17T12:05:04.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75db93640 1 -- 172.21.15.94:0/2735898565 learned_addr learned my addr 172.21.15.94:0/2735898565 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75cb91640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd758106450 0x7fd75819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75d392640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd758101380 0x7fd75819d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75db93640 1 -- 172.21.15.94:0/2735898565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd758101380 msgr2=0x7fd75819d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75db93640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd758101380 0x7fd75819d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75db93640 1 -- 172.21.15.94:0/2735898565 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd758106450 msgr2=0x7fd75819d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75db93640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd758106450 0x7fd75819d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.415+0000 7fd75db93640 1 -- 172.21.15.94:0/2735898565 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd750009900 con 0x7fd758197600 2024-02-17T12:05:04.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.416+0000 7fd75db93640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd758197600 0x7fd758197ab0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd750004af0 tx=0x7fd750034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.416+0000 7fd74e7fc640 1 -- 172.21.15.94:0/2735898565 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7500328c0 con 0x7fd758197600 2024-02-17T12:05:04.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.416+0000 7fd75f5f6640 1 -- 172.21.15.94:0/2735898565 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7581a42c0 con 0x7fd758197600 2024-02-17T12:05:04.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.416+0000 7fd74e7fc640 1 -- 172.21.15.94:0/2735898565 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd750003e50 con 0x7fd758197600 2024-02-17T12:05:04.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.416+0000 7fd74e7fc640 1 -- 172.21.15.94:0/2735898565 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd750004860 con 0x7fd758197600 2024-02-17T12:05:04.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.416+0000 7fd75f5f6640 1 -- 172.21.15.94:0/2735898565 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7581a4740 con 0x7fd758197600 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.418+0000 7fd74e7fc640 1 -- 172.21.15.94:0/2735898565 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd75000a040 con 0x7fd758197600 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.418+0000 7fd7621c5740 1 Processor -- start 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.418+0000 7fd7621c5740 1 -- start start 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.418+0000 7fd74e7fc640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd7440a2230 0x7fd7440a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd7621c5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd7621c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cbb70 0x55e12c9d4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd74e7fc640 1 -- 172.21.15.94:0/2735898565 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd7500f0c80 con 0x7fd758197600 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75cb91640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd7440a2230 0x7fd7440a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd7621c5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9d4750 0x55e12c9d6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd7621c5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c986960 con 0x55e12c9cb250 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75d392640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd7621c5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c93a310 con 0x55e12c9d4750 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd7621c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c937840 con 0x55e12c9cbb70 2024-02-17T12:05:04.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75cb91640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cbb70 0x55e12c9d4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75d392640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42382/0 (socket says 172.21.15.94:42382) 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75d392640 1 -- 172.21.15.94:0/3537625637 learned_addr learned my addr 172.21.15.94:0/3537625637 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75db93640 1 --2- 172.21.15.94:0/3537625637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9d4750 0x55e12c9d6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75d392640 1 -- 172.21.15.94:0/3537625637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cbb70 msgr2=0x55e12c9d4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75d392640 1 --2- 172.21.15.94:0/3537625637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cbb70 0x55e12c9d4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75d392640 1 -- 172.21.15.94:0/3537625637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9d4750 msgr2=0x55e12c9d6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75d392640 1 --2- 172.21.15.94:0/3537625637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9d4750 0x55e12c9d6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75d392640 1 -- 172.21.15.94:0/3537625637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e12c8f5060 con 0x55e12c9cb250 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75cb91640 1 --2- 172.21.15.94:0/3537625637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cbb70 0x55e12c9d4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75cb91640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd7440a2230 0x7fd7440a46f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd758100c00 tx=0x7fd74800baa0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.419+0000 7fd75d392640 1 --2- 172.21.15.94:0/3537625637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9cb630 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd754009d20 tx=0x7fd754011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb3b5325fae09ed server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.420+0000 7fd7437fe640 1 -- 172.21.15.94:0/3537625637 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754009e10 con 0x55e12c9cb250 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.420+0000 7fd7437fe640 1 -- 172.21.15.94:0/3537625637 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7540198f0 con 0x55e12c9cb250 2024-02-17T12:05:04.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.420+0000 7fd7437fe640 1 -- 172.21.15.94:0/3537625637 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754018990 con 0x55e12c9cb250 2024-02-17T12:05:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.420+0000 7fd7621c5740 1 -- 172.21.15.94:0/3537625637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 msgr2=0x55e12c9cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.420+0000 7fd7621c5740 1 --2- 172.21.15.94:0/3537625637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9cb630 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd754009d20 tx=0x7fd754011810 comp rx=0 tx=0).stop 2024-02-17T12:05:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.420+0000 7fd7621c5740 1 -- 172.21.15.94:0/3537625637 shutdown_connections 2024-02-17T12:05:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.420+0000 7fd7621c5740 1 --2- 172.21.15.94:0/3537625637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9d4750 0x55e12c9d6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.420+0000 7fd7621c5740 1 --2- 172.21.15.94:0/3537625637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cbb70 0x55e12c9d4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.420+0000 7fd7621c5740 1 --2- 172.21.15.94:0/3537625637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9cb630 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.420+0000 7fd7621c5740 1 -- 172.21.15.94:0/3537625637 >> 172.21.15.94:0/3537625637 conn(0x55e12c9c5980 msgr2=0x55e12c9c5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.420+0000 7fd7621c5740 1 -- 172.21.15.94:0/3537625637 shutdown_connections 2024-02-17T12:05:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.420+0000 7fd7621c5740 1 -- 172.21.15.94:0/3537625637 wait complete. 2024-02-17T12:05:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd7621c5740 1 -- start start 2024-02-17T12:05:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd7621c5740 1 Processor -- start 2024-02-17T12:05:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd7621c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cb250 0x55e12c9d1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd7621c5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9cbb70 0x55e12c9d1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd7621c5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9d4750 0x55e12c9d2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd7621c5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c933020 con 0x55e12c9d4750 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd7621c5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c606430 con 0x55e12c9cbb70 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd7621c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c5c2e00 con 0x55e12c9cb250 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd7621c5740 10 client.0 _ll_register_callbacks cb 0x55e12c8dacb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd75db93640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9d4750 0x55e12c9d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd75d392640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cb250 0x55e12c9d1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd75cb91640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9cbb70 0x55e12c9d1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd75d392640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cb250 0x55e12c9d1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:36662/0 (socket says 172.21.15.94:36662) 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd75db93640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9d4750 0x55e12c9d2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42386/0 (socket says 172.21.15.94:42386) 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.421+0000 7fd75d392640 1 -- 172.21.15.94:0/3784973785 learned_addr learned my addr 172.21.15.94:0/3784973785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd75d392640 1 -- 172.21.15.94:0/3784973785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9cbb70 msgr2=0x55e12c9d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd75d392640 1 --2- 172.21.15.94:0/3784973785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9cbb70 0x55e12c9d1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd75d392640 1 -- 172.21.15.94:0/3784973785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9d4750 msgr2=0x55e12c9d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd75d392640 1 --2- 172.21.15.94:0/3784973785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9d4750 0x55e12c9d2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd75d392640 1 -- 172.21.15.94:0/3784973785 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd754009950 con 0x55e12c9cb250 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd75cb91640 1 --2- 172.21.15.94:0/3784973785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9cbb70 0x55e12c9d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd75db93640 1 --2- 172.21.15.94:0/3784973785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9d4750 0x55e12c9d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd75d392640 1 --2- 172.21.15.94:0/3784973785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cb250 0x55e12c9d1710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd754019fa0 tx=0x7fd754019e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abf68f48e09a83af server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd740ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd740ff9640 1 -- 172.21.15.94:0/3784973785 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7540196a0 con 0x55e12c9cb250 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd7621c5740 1 -- 172.21.15.94:0/3784973785 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e12c6327e0 con 0x55e12c9cb250 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd740ff9640 1 -- 172.21.15.94:0/3784973785 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd754004050 con 0x55e12c9cb250 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd740ff9640 1 -- 172.21.15.94:0/3784973785 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd754004210 con 0x55e12c9cb250 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd7621c5740 10 client.26854 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd7621c5740 1 -- 172.21.15.94:0/3784973785 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e12c9316a0 con 0x55e12c9cb250 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.422+0000 7fd740ff9640 1 -- 172.21.15.94:0/3784973785 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e12c6327e0 con 0x55e12c9cb250 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.423+0000 7fd740ff9640 1 -- 172.21.15.94:0/3784973785 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd754004b00 con 0x55e12c9cb250 2024-02-17T12:05:04.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.423+0000 7fd7621c5740 10 client.26854 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.423+0000 7fd7621c5740 10 client.26854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.423+0000 7fd7621c5740 20 client.26854 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.423+0000 7fd7621c5740 1 --2- 172.21.15.94:0/3784973785 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e12c9e8eb0 0x55e12c9eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.423+0000 7fd7621c5740 4 client.26854 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.423+0000 7fd7621c5740 1 -- 172.21.15.94:0/3784973785 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e12c66f720 con 0x55e12c9e8eb0 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.423+0000 7fd75cb91640 1 --2- 172.21.15.94:0/3784973785 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e12c9e8eb0 0x55e12c9eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.423+0000 7fd75cb91640 1 --2- 172.21.15.94:0/3784973785 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e12c9e8eb0 0x55e12c9eb2f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e8153ecdf55e0a2 server_cookie=d5e25bc4a867802e in_seq=0 out_seq=0 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.423+0000 7fd740ff9640 10 client.26854 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd740ff9640 1 -- 172.21.15.94:0/3784973785 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e12c66f720 con 0x55e12c9e8eb0 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd740ff9640 10 client.26854 handle_command_reply: tid=0 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd7621c5740 1 client.26854 shutdown 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd7621c5740 1 -- 172.21.15.94:0/3784973785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cb250 msgr2=0x55e12c9d1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd7621c5740 1 --2- 172.21.15.94:0/3784973785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cb250 0x55e12c9d1710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd754019fa0 tx=0x7fd754019e90 comp rx=0 tx=0).stop 2024-02-17T12:05:04.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd7621c5740 1 -- 172.21.15.94:0/3784973785 shutdown_connections 2024-02-17T12:05:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd7621c5740 1 --2- 172.21.15.94:0/3784973785 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e12c9e8eb0 0x55e12c9eb2f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd7621c5740 1 --2- 172.21.15.94:0/3784973785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9d4750 0x55e12c9d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd740ff9640 0 client.26854 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd7621c5740 1 --2- 172.21.15.94:0/3784973785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9cbb70 0x55e12c9d1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd7621c5740 1 --2- 172.21.15.94:0/3784973785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cb250 0x55e12c9d1710 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd7621c5740 1 -- 172.21.15.94:0/3784973785 >> 172.21.15.94:0/3784973785 conn(0x55e12c9c5980 msgr2=0x55e12c9c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd7621c5740 1 -- 172.21.15.94:0/3784973785 shutdown_connections 2024-02-17T12:05:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd7621c5740 1 -- 172.21.15.94:0/3784973785 wait complete. 2024-02-17T12:05:04.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.424+0000 7fd7621c5740 20 client.26854 trim_cache size 0 max 16384 2024-02-17T12:05:04.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:03 smithi125 ceph-mon[36819]: pgmap v212: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 8.3 MiB/s wr, 662 op/s 2024-02-17T12:05:04.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.436+0000 7fd7621c5740 1 Processor -- start 2024-02-17T12:05:04.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.436+0000 7fd7621c5740 1 -- start start 2024-02-17T12:05:04.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.436+0000 7fd7621c5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9d3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.436+0000 7fd7621c5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9cbb70 0x55e12c9cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.436+0000 7fd7621c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9d4750 0x55e12c9ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.436+0000 7fd7621c5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c8b8130 con 0x55e12c9cb250 2024-02-17T12:05:04.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.436+0000 7fd7621c5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c6327e0 con 0x55e12c9cbb70 2024-02-17T12:05:04.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd7621c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c9e8c30 con 0x55e12c9d4750 2024-02-17T12:05:04.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd75d392640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9d3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd75d392640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9d3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35330/0 (socket says 172.21.15.94:35330) 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd75d392640 1 -- 172.21.15.94:0/4253305244 learned_addr learned my addr 172.21.15.94:0/4253305244 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd75db93640 1 --2- 172.21.15.94:0/4253305244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9d4750 0x55e12c9ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd75cb91640 1 --2- 172.21.15.94:0/4253305244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9cbb70 0x55e12c9cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd75d392640 1 -- 172.21.15.94:0/4253305244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9d4750 msgr2=0x55e12c9ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd75d392640 1 --2- 172.21.15.94:0/4253305244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9d4750 0x55e12c9ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd75d392640 1 -- 172.21.15.94:0/4253305244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9cbb70 msgr2=0x55e12c9cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd75d392640 1 --2- 172.21.15.94:0/4253305244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9cbb70 0x55e12c9cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd75d392640 1 -- 172.21.15.94:0/4253305244 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e12c8cdc20 con 0x55e12c9cb250 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd75d392640 1 --2- 172.21.15.94:0/4253305244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9d3e40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd754009250 tx=0x7fd754011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1caaf90bc89e87 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd742ffd640 1 -- 172.21.15.94:0/4253305244 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754004fe0 con 0x55e12c9cb250 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd742ffd640 1 -- 172.21.15.94:0/4253305244 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd754004680 con 0x55e12c9cb250 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.437+0000 7fd742ffd640 1 -- 172.21.15.94:0/4253305244 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754018980 con 0x55e12c9cb250 2024-02-17T12:05:04.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.438+0000 7fd7621c5740 1 -- 172.21.15.94:0/4253305244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 msgr2=0x55e12c9d3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.438+0000 7fd7621c5740 1 --2- 172.21.15.94:0/4253305244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9d3e40 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd754009250 tx=0x7fd754011810 comp rx=0 tx=0).stop 2024-02-17T12:05:04.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.438+0000 7fd7621c5740 1 -- 172.21.15.94:0/4253305244 shutdown_connections 2024-02-17T12:05:04.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.438+0000 7fd7621c5740 1 --2- 172.21.15.94:0/4253305244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9d4750 0x55e12c9ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.438+0000 7fd7621c5740 1 --2- 172.21.15.94:0/4253305244 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9cbb70 0x55e12c9cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.438+0000 7fd7621c5740 1 --2- 172.21.15.94:0/4253305244 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9d3e40 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.438+0000 7fd7621c5740 1 -- 172.21.15.94:0/4253305244 >> 172.21.15.94:0/4253305244 conn(0x55e12c9c5980 msgr2=0x55e12c9c0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.438+0000 7fd7621c5740 1 -- 172.21.15.94:0/4253305244 shutdown_connections 2024-02-17T12:05:04.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.438+0000 7fd7621c5740 1 -- 172.21.15.94:0/4253305244 wait complete. 2024-02-17T12:05:04.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.438+0000 7fd7621c5740 1 -- start start 2024-02-17T12:05:04.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd7621c5740 1 Processor -- start 2024-02-17T12:05:04.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd7621c5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9d9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd7621c5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cbb70 0x55e12c9da470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd7621c5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9d4750 0x55e12c9da9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd7621c5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12c9dbed0 con 0x55e12c9cb250 2024-02-17T12:05:04.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd7621c5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12ca3f420 con 0x55e12c9d4750 2024-02-17T12:05:04.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd7621c5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e12ca3f590 con 0x55e12c9cbb70 2024-02-17T12:05:04.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd7621c5740 10 client.0 _ll_register_callbacks cb 0x7fd754009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:04.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd75d392640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9d9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd75d392640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9d9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35336/0 (socket says 172.21.15.94:35336) 2024-02-17T12:05:04.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd75d392640 1 -- 172.21.15.94:0/2045187378 learned_addr learned my addr 172.21.15.94:0/2045187378 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:04.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd75cb91640 1 --2- 172.21.15.94:0/2045187378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cbb70 0x55e12c9da470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd75db93640 1 --2- 172.21.15.94:0/2045187378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9d4750 0x55e12c9da9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd75d392640 1 -- 172.21.15.94:0/2045187378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cbb70 msgr2=0x55e12c9da470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd75d392640 1 --2- 172.21.15.94:0/2045187378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cbb70 0x55e12c9da470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd75d392640 1 -- 172.21.15.94:0/2045187378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9d4750 msgr2=0x55e12c9da9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd75d392640 1 --2- 172.21.15.94:0/2045187378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9d4750 0x55e12c9da9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd75d392640 1 -- 172.21.15.94:0/2045187378 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e12c9316a0 con 0x55e12c9cb250 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.439+0000 7fd75d392640 1 --2- 172.21.15.94:0/2045187378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9d9f30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd754011d90 tx=0x7fd754019d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f11aa5a4549583f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd740ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd740ff9640 1 -- 172.21.15.94:0/2045187378 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd754019e90 con 0x55e12c9cb250 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd7621c5740 1 -- 172.21.15.94:0/2045187378 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7540198f0 con 0x55e12c9cb250 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd740ff9640 1 -- 172.21.15.94:0/2045187378 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fd754000e40 con 0x55e12c9cb250 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd740ff9640 1 -- 172.21.15.94:0/2045187378 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd754004a00 con 0x55e12c9cb250 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd7621c5740 10 client.17343 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:04.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd7621c5740 1 -- 172.21.15.94:0/2045187378 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e12c8f38c0 con 0x55e12c9cb250 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd740ff9640 1 -- 172.21.15.94:0/2045187378 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd7540198f0 con 0x55e12c9cb250 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd740ff9640 1 -- 172.21.15.94:0/2045187378 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd754018ad0 con 0x55e12c9cb250 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd7621c5740 10 client.17343 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd7621c5740 10 client.17343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd7621c5740 20 client.17343 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd7621c5740 1 --2- 172.21.15.94:0/2045187378 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e12ca408d0 0x55e12ca42cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd7621c5740 4 client.17343 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd7621c5740 1 -- 172.21.15.94:0/2045187378 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e12c6be890 con 0x55e12ca408d0 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.440+0000 7fd75cb91640 1 --2- 172.21.15.94:0/2045187378 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e12ca408d0 0x55e12ca42cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd740ff9640 10 client.17343 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd75cb91640 1 --2- 172.21.15.94:0/2045187378 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e12ca408d0 0x55e12ca42cc0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3572e3723d9a10b9 server_cookie=71325f147f38b5c8 in_seq=0 out_seq=0 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd740ff9640 1 -- 172.21.15.94:0/2045187378 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e12c6be890 con 0x55e12ca408d0 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd740ff9640 10 client.17343 handle_command_reply: tid=0 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd7621c5740 1 client.17343 shutdown 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd7621c5740 1 -- 172.21.15.94:0/2045187378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 msgr2=0x55e12c9d9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd7621c5740 1 --2- 172.21.15.94:0/2045187378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9d9f30 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd754011d90 tx=0x7fd754019d70 comp rx=0 tx=0).stop 2024-02-17T12:05:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd7621c5740 1 -- 172.21.15.94:0/2045187378 shutdown_connections 2024-02-17T12:05:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd7621c5740 1 --2- 172.21.15.94:0/2045187378 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e12ca408d0 0x55e12ca42cc0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd7621c5740 1 --2- 172.21.15.94:0/2045187378 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e12c9d4750 0x55e12c9da9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd740ff9640 0 client.17343 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd7621c5740 1 --2- 172.21.15.94:0/2045187378 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e12c9cbb70 0x55e12c9da470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd7621c5740 1 --2- 172.21.15.94:0/2045187378 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e12c9cb250 0x55e12c9d9f30 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd7621c5740 1 -- 172.21.15.94:0/2045187378 >> 172.21.15.94:0/2045187378 conn(0x55e12c9c5980 msgr2=0x55e12c9cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd7621c5740 1 -- 172.21.15.94:0/2045187378 shutdown_connections 2024-02-17T12:05:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.441+0000 7fd7621c5740 1 -- 172.21.15.94:0/2045187378 wait complete. 2024-02-17T12:05:04.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.442+0000 7fd7621c5740 20 client.17343 trim_cache size 0 max 16384 2024-02-17T12:05:04.448 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:05:04.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.442+0000 7fd75f5f6640 1 -- 172.21.15.94:0/2735898565 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd7440a2230 msgr2=0x7fd7440a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.442+0000 7fd75f5f6640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd7440a2230 0x7fd7440a46f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd758100c00 tx=0x7fd74800baa0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.442+0000 7fd75f5f6640 1 -- 172.21.15.94:0/2735898565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd758197600 msgr2=0x7fd758197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:04.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.442+0000 7fd75f5f6640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd758197600 0x7fd758197ab0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd750004af0 tx=0x7fd750034cc0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.442+0000 7fd75f5f6640 1 -- 172.21.15.94:0/2735898565 shutdown_connections 2024-02-17T12:05:04.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.442+0000 7fd75f5f6640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd7440a2230 0x7fd7440a46f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.442+0000 7fd75f5f6640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd758197600 0x7fd758197ab0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.442+0000 7fd75f5f6640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd758106450 0x7fd75819d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.442+0000 7fd75f5f6640 1 --2- 172.21.15.94:0/2735898565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd758101380 0x7fd75819d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:04.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.442+0000 7fd75f5f6640 1 -- 172.21.15.94:0/2735898565 >> 172.21.15.94:0/2735898565 conn(0x7fd758078580 msgr2=0x7fd7580ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:04.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.442+0000 7fd75f5f6640 1 -- 172.21.15.94:0/2735898565 shutdown_connections 2024-02-17T12:05:04.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:04.442+0000 7fd75f5f6640 1 -- 172.21.15.94:0/2735898565 wait complete. 2024-02-17T12:05:05.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:04 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1643561006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:05:05.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 405.0 s, 386.0 tps, lat 217.410 ms stddev 219.866 2024-02-17T12:05:05.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:04 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1643561006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:05:05.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:04 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1643561006' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:05:05.454 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:05:05.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:05:05.455 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:05:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.574+0000 7ff87898c640 1 -- 172.21.15.94:0/735726957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff874100c10 msgr2=0x7ff874104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.574+0000 7ff87898c640 1 --2- 172.21.15.94:0/735726957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff874100c10 0x7ff874104c50 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009930 tx=0x7ff85c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:05:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.574+0000 7ff87898c640 1 -- 172.21.15.94:0/735726957 shutdown_connections 2024-02-17T12:05:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.574+0000 7ff87898c640 1 --2- 172.21.15.94:0/735726957 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874105330 0x7ff87410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.574+0000 7ff87898c640 1 --2- 172.21.15.94:0/735726957 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff874100c10 0x7ff874104c50 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.574+0000 7ff87898c640 1 --2- 172.21.15.94:0/735726957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff874100260 0x7ff874100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.574+0000 7ff87898c640 1 -- 172.21.15.94:0/735726957 >> 172.21.15.94:0/735726957 conn(0x7ff874077180 msgr2=0x7ff874077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.574+0000 7ff87898c640 1 -- 172.21.15.94:0/735726957 shutdown_connections 2024-02-17T12:05:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.574+0000 7ff87898c640 1 -- 172.21.15.94:0/735726957 wait complete. 2024-02-17T12:05:05.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff87898c640 1 Processor -- start 2024-02-17T12:05:05.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff87898c640 1 -- start start 2024-02-17T12:05:05.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff87898c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff874100260 0x7ff8741966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff87898c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874105330 0x7ff874196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff87898c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff87419af70 0x7ff87419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff87898c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87419b960 con 0x7ff87419af70 2024-02-17T12:05:05.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff87898c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87419bad0 con 0x7ff874105330 2024-02-17T12:05:05.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff87898c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87419bc40 con 0x7ff874100260 2024-02-17T12:05:05.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff872d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff87419af70 0x7ff87419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff872d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff87419af70 0x7ff87419b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35354/0 (socket says 172.21.15.94:35354) 2024-02-17T12:05:05.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff872d9d640 1 -- 172.21.15.94:0/2367212250 learned_addr learned my addr 172.21.15.94:0/2367212250 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:05.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff87259c640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff874100260 0x7ff8741966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff869d9b640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874105330 0x7ff874196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff872d9d640 1 -- 172.21.15.94:0/2367212250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff874100260 msgr2=0x7ff8741966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff872d9d640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff874100260 0x7ff8741966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff872d9d640 1 -- 172.21.15.94:0/2367212250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874105330 msgr2=0x7ff874196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff872d9d640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874105330 0x7ff874196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.575+0000 7ff872d9d640 1 -- 172.21.15.94:0/2367212250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c009560 con 0x7ff87419af70 2024-02-17T12:05:05.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.576+0000 7ff872d9d640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff87419af70 0x7ff87419b420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00ebb0 tx=0x7ff86c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:05.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.576+0000 7ff86bfff640 1 -- 172.21.15.94:0/2367212250 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c013070 con 0x7ff87419af70 2024-02-17T12:05:05.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.576+0000 7ff87898c640 1 -- 172.21.15.94:0/2367212250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8740682e0 con 0x7ff87419af70 2024-02-17T12:05:05.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.576+0000 7ff86bfff640 1 -- 172.21.15.94:0/2367212250 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff86c007ca0 con 0x7ff87419af70 2024-02-17T12:05:05.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.576+0000 7ff86bfff640 1 -- 172.21.15.94:0/2367212250 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c005660 con 0x7ff87419af70 2024-02-17T12:05:05.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.576+0000 7ff87898c640 1 -- 172.21.15.94:0/2367212250 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8740687c0 con 0x7ff87419af70 2024-02-17T12:05:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.578+0000 7ff86bfff640 1 -- 172.21.15.94:0/2367212250 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff86c00e160 con 0x7ff87419af70 2024-02-17T12:05:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.578+0000 7ff87b1fd740 1 Processor -- start 2024-02-17T12:05:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.578+0000 7ff87b1fd740 1 -- start start 2024-02-17T12:05:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.578+0000 7ff86bfff640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff8600a2230 0x7ff8600a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff87b1fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff87b1fd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8cfd30 0x55d07d8d82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff87b1fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8d88b0 0x55d07d8dad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff87b1fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07d83df90 con 0x55d07d8d88b0 2024-02-17T12:05:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff869d9b640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff8600a2230 0x7ff8600a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff86bfff640 1 -- 172.21.15.94:0/2367212250 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff86c0cf420 con 0x7ff87419af70 2024-02-17T12:05:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff87b1fd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07d83b4c0 con 0x55d07d8cfd30 2024-02-17T12:05:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff87b1fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07d836ca0 con 0x55d07d8cf380 2024-02-17T12:05:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff872d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8d88b0 0x55d07d8dad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff872d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8d88b0 0x55d07d8dad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35368/0 (socket says 172.21.15.94:35368) 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff872d9d640 1 -- 172.21.15.94:0/448390885 learned_addr learned my addr 172.21.15.94:0/448390885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff87259c640 1 --2- 172.21.15.94:0/448390885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff869d9b640 1 --2- 172.21.15.94:0/448390885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8cfd30 0x55d07d8d82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff872d9d640 1 -- 172.21.15.94:0/448390885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 msgr2=0x55d07d8cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff872d9d640 1 --2- 172.21.15.94:0/448390885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff872d9d640 1 -- 172.21.15.94:0/448390885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8cfd30 msgr2=0x55d07d8d82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff872d9d640 1 --2- 172.21.15.94:0/448390885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8cfd30 0x55d07d8d82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff872d9d640 1 -- 172.21.15.94:0/448390885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d07d7f90b0 con 0x55d07d8d88b0 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff869d9b640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff8600a2230 0x7ff8600a46f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff85c000c00 tx=0x7ff85c004a60 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff872d9d640 1 --2- 172.21.15.94:0/448390885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8d88b0 0x55d07d8dad00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0a4070 tx=0x7ff86c09b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3498eb8374b917c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff868d99640 1 -- 172.21.15.94:0/448390885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c09e070 con 0x55d07d8d88b0 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff868d99640 1 -- 172.21.15.94:0/448390885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff86c09ff50 con 0x55d07d8d88b0 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.579+0000 7ff868d99640 1 -- 172.21.15.94:0/448390885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c098520 con 0x55d07d8d88b0 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.580+0000 7ff87b1fd740 1 -- 172.21.15.94:0/448390885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8d88b0 msgr2=0x55d07d8dad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.580+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/448390885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8d88b0 0x55d07d8dad00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0a4070 tx=0x7ff86c09b9f0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.580+0000 7ff87b1fd740 1 -- 172.21.15.94:0/448390885 shutdown_connections 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.580+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/448390885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8d88b0 0x55d07d8dad00 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.580+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/448390885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8cfd30 0x55d07d8d82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.580+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/448390885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.580+0000 7ff87b1fd740 1 -- 172.21.15.94:0/448390885 >> 172.21.15.94:0/448390885 conn(0x55d07d8c9840 msgr2=0x55d07d8c9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.580+0000 7ff87b1fd740 1 -- 172.21.15.94:0/448390885 shutdown_connections 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.580+0000 7ff87b1fd740 1 -- 172.21.15.94:0/448390885 wait complete. 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff87b1fd740 1 -- start start 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff87b1fd740 1 Processor -- start 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff87b1fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8d5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff87b1fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8d5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff87b1fd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8d88b0 0x55d07d8d62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff87b1fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07d7dbf40 con 0x55d07d8cfd30 2024-02-17T12:05:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff87b1fd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07d50a4b0 con 0x55d07d8d88b0 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff87b1fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07d7bc670 con 0x55d07d8cf380 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff87b1fd740 10 client.0 _ll_register_callbacks cb 0x55d07d829f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff869d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8d5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff869d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8d5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35374/0 (socket says 172.21.15.94:35374) 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff869d9b640 1 -- 172.21.15.94:0/3475767682 learned_addr learned my addr 172.21.15.94:0/3475767682 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff872d9d640 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8d88b0 0x55d07d8d62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff87259c640 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8d5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff869d9b640 1 -- 172.21.15.94:0/3475767682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 msgr2=0x55d07d8d5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff869d9b640 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8d5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff869d9b640 1 -- 172.21.15.94:0/3475767682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8d88b0 msgr2=0x55d07d8d62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff869d9b640 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8d88b0 0x55d07d8d62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff869d9b640 1 -- 172.21.15.94:0/3475767682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8740687c0 con 0x55d07d8cfd30 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff87259c640 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff869d9b640 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8d5d90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff85c035000 tx=0x7ff85c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a5badc068750e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.581+0000 7ff859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff859ffb640 1 -- 172.21.15.94:0/3475767682 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c004df0 con 0x55d07d8cfd30 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff859ffb640 1 -- 172.21.15.94:0/3475767682 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff85c004f50 con 0x55d07d8cfd30 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff87b1fd740 1 -- 172.21.15.94:0/3475767682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d07d810f40 con 0x55d07d8cfd30 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff859ffb640 1 -- 172.21.15.94:0/3475767682 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff85c005110 con 0x55d07d8cfd30 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff87b1fd740 10 client.17355 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff87b1fd740 1 -- 172.21.15.94:0/3475767682 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d07d835320 con 0x55d07d8cfd30 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff859ffb640 1 -- 172.21.15.94:0/3475767682 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d07d810f40 con 0x55d07d8cfd30 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff859ffb640 1 -- 172.21.15.94:0/3475767682 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff85c03c070 con 0x55d07d8cfd30 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff87b1fd740 10 client.17355 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff87b1fd740 10 client.17355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff87b1fd740 20 client.17355 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d07d8e8e80 0x55d07d8eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff87b1fd740 4 client.17355 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff87b1fd740 1 -- 172.21.15.94:0/3475767682 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d07d82d310 con 0x55d07d8e8e80 2024-02-17T12:05:05.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.582+0000 7ff87259c640 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d07d8e8e80 0x55d07d8eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.583+0000 7ff859ffb640 10 client.17355 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.583+0000 7ff87259c640 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d07d8e8e80 0x55d07d8eb2c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37e8cf17cb45e196 server_cookie=494374c1b240162e in_seq=0 out_seq=0 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.583+0000 7ff859ffb640 1 -- 172.21.15.94:0/3475767682 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d07d82d310 con 0x55d07d8e8e80 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.583+0000 7ff859ffb640 10 client.17355 handle_command_reply: tid=0 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.583+0000 7ff87b1fd740 1 client.17355 shutdown 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.584+0000 7ff87b1fd740 1 -- 172.21.15.94:0/3475767682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 msgr2=0x55d07d8d5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.584+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8d5d90 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff85c035000 tx=0x7ff85c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.584+0000 7ff87b1fd740 1 -- 172.21.15.94:0/3475767682 shutdown_connections 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.584+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d07d8e8e80 0x55d07d8eb2c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.584+0000 7ff859ffb640 0 client.17355 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.584+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8d88b0 0x55d07d8d62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.584+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8d5d90 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.584+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/3475767682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8d5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.584+0000 7ff87b1fd740 1 -- 172.21.15.94:0/3475767682 >> 172.21.15.94:0/3475767682 conn(0x55d07d8c9840 msgr2=0x55d07d8cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:05.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.584+0000 7ff87b1fd740 1 -- 172.21.15.94:0/3475767682 shutdown_connections 2024-02-17T12:05:05.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.584+0000 7ff87b1fd740 1 -- 172.21.15.94:0/3475767682 wait complete. 2024-02-17T12:05:05.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.584+0000 7ff87b1fd740 20 client.17355 trim_cache size 0 max 16384 2024-02-17T12:05:05.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.602+0000 7ff87b1fd740 1 Processor -- start 2024-02-17T12:05:05.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.602+0000 7ff87b1fd740 1 -- start start 2024-02-17T12:05:05.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff87b1fd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8cf380 0x55d07d8d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff87b1fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8d60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff87b1fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8d88b0 0x55d07d8d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff87b1fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07d8e5940 con 0x55d07d8cfd30 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff87b1fd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07d8ca410 con 0x55d07d8cf380 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff87b1fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07d8ca5d0 con 0x55d07d8d88b0 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff869d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8d60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff869d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8d60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35382/0 (socket says 172.21.15.94:35382) 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff869d9b640 1 -- 172.21.15.94:0/4077242440 learned_addr learned my addr 172.21.15.94:0/4077242440 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff872d9d640 1 --2- 172.21.15.94:0/4077242440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8d88b0 0x55d07d8d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff87259c640 1 --2- 172.21.15.94:0/4077242440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8cf380 0x55d07d8d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff869d9b640 1 -- 172.21.15.94:0/4077242440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8d88b0 msgr2=0x55d07d8d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff869d9b640 1 --2- 172.21.15.94:0/4077242440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8d88b0 0x55d07d8d2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff869d9b640 1 -- 172.21.15.94:0/4077242440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8cf380 msgr2=0x55d07d8d7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff869d9b640 1 --2- 172.21.15.94:0/4077242440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8cf380 0x55d07d8d7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.603+0000 7ff869d9b640 1 -- 172.21.15.94:0/4077242440 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d07d7d1c00 con 0x55d07d8cfd30 2024-02-17T12:05:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.604+0000 7ff869d9b640 1 --2- 172.21.15.94:0/4077242440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8d60f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff85c02c170 tx=0x7ff85c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4f4148f35197c4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.604+0000 7ff85bfff640 1 -- 172.21.15.94:0/4077242440 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c02c860 con 0x55d07d8cfd30 2024-02-17T12:05:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.604+0000 7ff85bfff640 1 -- 172.21.15.94:0/4077242440 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff85c003ad0 con 0x55d07d8cfd30 2024-02-17T12:05:05.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.604+0000 7ff85bfff640 1 -- 172.21.15.94:0/4077242440 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c042d30 con 0x55d07d8cfd30 2024-02-17T12:05:05.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.605+0000 7ff87b1fd740 1 -- 172.21.15.94:0/4077242440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 msgr2=0x55d07d8d60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.605+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/4077242440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8d60f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff85c02c170 tx=0x7ff85c005fd0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.605+0000 7ff87b1fd740 1 -- 172.21.15.94:0/4077242440 shutdown_connections 2024-02-17T12:05:05.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.605+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/4077242440 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8d88b0 0x55d07d8d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.605+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/4077242440 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8d60f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.605+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/4077242440 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8cf380 0x55d07d8d7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.605+0000 7ff87b1fd740 1 -- 172.21.15.94:0/4077242440 >> 172.21.15.94:0/4077242440 conn(0x55d07d8c9840 msgr2=0x55d07d8d4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:05.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.605+0000 7ff87b1fd740 1 -- 172.21.15.94:0/4077242440 shutdown_connections 2024-02-17T12:05:05.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.605+0000 7ff87b1fd740 1 -- 172.21.15.94:0/4077242440 wait complete. 2024-02-17T12:05:05.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.605+0000 7ff87b1fd740 1 -- start start 2024-02-17T12:05:05.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.605+0000 7ff87b1fd740 1 Processor -- start 2024-02-17T12:05:05.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.605+0000 7ff87b1fd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8dfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff87b1fd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8ddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff87b1fd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8d88b0 0x55d07d8de4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff87b1fd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07d8deb80 con 0x55d07d8cfd30 2024-02-17T12:05:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff87b1fd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07d8decf0 con 0x55d07d8d88b0 2024-02-17T12:05:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff87b1fd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d07d93f3f0 con 0x55d07d8cf380 2024-02-17T12:05:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff87b1fd740 10 client.0 _ll_register_callbacks cb 0x7ff86c005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff869d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8ddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff869d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8ddfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35396/0 (socket says 172.21.15.94:35396) 2024-02-17T12:05:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff869d9b640 1 -- 172.21.15.94:0/2645280853 learned_addr learned my addr 172.21.15.94:0/2645280853 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff872d9d640 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8d88b0 0x55d07d8de4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff87259c640 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8dfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff869d9b640 1 -- 172.21.15.94:0/2645280853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 msgr2=0x55d07d8dfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff869d9b640 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8dfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff869d9b640 1 -- 172.21.15.94:0/2645280853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8d88b0 msgr2=0x55d07d8de4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff869d9b640 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8d88b0 0x55d07d8de4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff869d9b640 1 -- 172.21.15.94:0/2645280853 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d07d835320 con 0x55d07d8cfd30 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff87259c640 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff869d9b640 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8ddfa0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff85c060000 tx=0x7ff85c003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b37651556beeef0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff859ffb640 1 -- 172.21.15.94:0/2645280853 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85c042440 con 0x55d07d8cfd30 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff859ffb640 1 -- 172.21.15.94:0/2645280853 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ff85c0425a0 con 0x55d07d8cfd30 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff87b1fd740 1 -- 172.21.15.94:0/2645280853 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d07d85c900 con 0x55d07d8cfd30 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff859ffb640 1 -- 172.21.15.94:0/2645280853 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff85c042760 con 0x55d07d8cfd30 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff87b1fd740 10 client.17361 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.606+0000 7ff87b1fd740 1 -- 172.21.15.94:0/2645280853 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d07d7f7910 con 0x55d07d8cfd30 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.607+0000 7ff859ffb640 1 -- 172.21.15.94:0/2645280853 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d07d85c900 con 0x55d07d8cfd30 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.607+0000 7ff859ffb640 1 -- 172.21.15.94:0/2645280853 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff85c03c070 con 0x55d07d8cfd30 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.607+0000 7ff87b1fd740 10 client.17361 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.607+0000 7ff87b1fd740 10 client.17361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.607+0000 7ff87b1fd740 20 client.17361 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:05.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.607+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d07d9405a0 0x55d07d942990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.607+0000 7ff87b1fd740 4 client.17361 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.607+0000 7ff87b1fd740 1 -- 172.21.15.94:0/2645280853 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d07d536860 con 0x55d07d9405a0 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.607+0000 7ff87259c640 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d07d9405a0 0x55d07d942990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.607+0000 7ff859ffb640 10 client.17361 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.607+0000 7ff87259c640 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d07d9405a0 0x55d07d942990 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3388c337616e8cb4 server_cookie=abbd8ca523c5d40c in_seq=0 out_seq=0 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.608+0000 7ff859ffb640 1 -- 172.21.15.94:0/2645280853 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d07d536860 con 0x55d07d9405a0 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.608+0000 7ff859ffb640 10 client.17361 handle_command_reply: tid=0 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.608+0000 7ff87b1fd740 1 client.17361 shutdown 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87b1fd740 1 -- 172.21.15.94:0/2645280853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 msgr2=0x55d07d8ddfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8ddfa0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff85c060000 tx=0x7ff85c003c40 comp rx=0 tx=0).stop 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87b1fd740 1 -- 172.21.15.94:0/2645280853 shutdown_connections 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d07d9405a0 0x55d07d942990 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d07d8d88b0 0x55d07d8de4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d07d8cfd30 0x55d07d8ddfa0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff859ffb640 0 client.17361 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:05.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87b1fd740 1 --2- 172.21.15.94:0/2645280853 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d07d8cf380 0x55d07d8dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87b1fd740 1 -- 172.21.15.94:0/2645280853 >> 172.21.15.94:0/2645280853 conn(0x55d07d8c9840 msgr2=0x55d07d8c8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:05.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87b1fd740 1 -- 172.21.15.94:0/2645280853 shutdown_connections 2024-02-17T12:05:05.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87b1fd740 1 -- 172.21.15.94:0/2645280853 wait complete. 2024-02-17T12:05:05.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87b1fd740 20 client.17361 trim_cache size 0 max 16384 2024-02-17T12:05:05.615 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:05:05.615 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:05:05.615 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "795c9bd8-4280-4426-a2c5-fdfb055f1ff0", 2024-02-17T12:05:05.615 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:05:05.615 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:05:05.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87898c640 1 -- 172.21.15.94:0/2367212250 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff8600a2230 msgr2=0x7ff8600a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87898c640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff8600a2230 0x7ff8600a46f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff85c000c00 tx=0x7ff85c004a60 comp rx=0 tx=0).stop 2024-02-17T12:05:05.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87898c640 1 -- 172.21.15.94:0/2367212250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff87419af70 msgr2=0x7ff87419b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87898c640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff87419af70 0x7ff87419b420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00ebb0 tx=0x7ff86c007590 comp rx=0 tx=0).stop 2024-02-17T12:05:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87898c640 1 -- 172.21.15.94:0/2367212250 shutdown_connections 2024-02-17T12:05:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87898c640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff8600a2230 0x7ff8600a46f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87898c640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff87419af70 0x7ff87419b420 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87898c640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff874105330 0x7ff874196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87898c640 1 --2- 172.21.15.94:0/2367212250 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff874100260 0x7ff8741966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.609+0000 7ff87898c640 1 -- 172.21.15.94:0/2367212250 >> 172.21.15.94:0/2367212250 conn(0x7ff874077180 msgr2=0x7ff8740fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.610+0000 7ff87898c640 1 -- 172.21.15.94:0/2367212250 shutdown_connections 2024-02-17T12:05:05.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.610+0000 7ff87898c640 1 -- 172.21.15.94:0/2367212250 wait complete. 2024-02-17T12:05:05.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:05:05.623 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:05:05.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.773+0000 7f77dc640640 1 -- 172.21.15.94:0/2097451346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77d4101d10 msgr2=0x7f77d4105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.773+0000 7f77dc640640 1 --2- 172.21.15.94:0/2097451346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77d4101d10 0x7f77d4105d50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009930 tx=0x7f77c402bc30 comp rx=0 tx=0).stop 2024-02-17T12:05:05.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.773+0000 7f77dc640640 1 -- 172.21.15.94:0/2097451346 shutdown_connections 2024-02-17T12:05:05.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.773+0000 7f77dc640640 1 --2- 172.21.15.94:0/2097451346 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d4106430 0x7f77d410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.773+0000 7f77dc640640 1 --2- 172.21.15.94:0/2097451346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77d4101d10 0x7f77d4105d50 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.773+0000 7f77dc640640 1 --2- 172.21.15.94:0/2097451346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77d4101360 0x7f77d4101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.773+0000 7f77dc640640 1 -- 172.21.15.94:0/2097451346 >> 172.21.15.94:0/2097451346 conn(0x7f77d40fcc20 msgr2=0x7f77d40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:05.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.773+0000 7f77dc640640 1 -- 172.21.15.94:0/2097451346 shutdown_connections 2024-02-17T12:05:05.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.773+0000 7f77dc640640 1 -- 172.21.15.94:0/2097451346 wait complete. 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dc640640 1 Processor -- start 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dc640640 1 -- start start 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dc640640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d4101360 0x7f77d419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dc640640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77d4106430 0x7f77d419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dc640640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77d4197620 0x7f77d4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dc640640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d4198010 con 0x7f77d4197620 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dc640640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d4198180 con 0x7f77d4101360 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dc640640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d4198340 con 0x7f77d4106430 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dabdd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77d4197620 0x7f77d4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77da3dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d4101360 0x7f77d419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77da3dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d4101360 0x7f77d419d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:35124/0 (socket says 172.21.15.94:35124) 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dabdd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77d4197620 0x7f77d4197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35418/0 (socket says 172.21.15.94:35418) 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77da3dc640 1 -- 172.21.15.94:0/1607727129 learned_addr learned my addr 172.21.15.94:0/1607727129 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77d9bdb640 1 --2- 172.21.15.94:0/1607727129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77d4106430 0x7f77d419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dabdd640 1 -- 172.21.15.94:0/1607727129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77d4106430 msgr2=0x7f77d419d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dabdd640 1 --2- 172.21.15.94:0/1607727129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77d4106430 0x7f77d419d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dabdd640 1 -- 172.21.15.94:0/1607727129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d4101360 msgr2=0x7f77d419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dabdd640 1 --2- 172.21.15.94:0/1607727129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d4101360 0x7f77d419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.774+0000 7f77dabdd640 1 -- 172.21.15.94:0/1607727129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c4009560 con 0x7f77d4197620 2024-02-17T12:05:05.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.775+0000 7f77dabdd640 1 --2- 172.21.15.94:0/1607727129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77d4197620 0x7f77d4197ad0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009fd0 tx=0x7f77cc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:05.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.775+0000 7f77cb7fe640 1 -- 172.21.15.94:0/1607727129 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00ce20 con 0x7f77d4197620 2024-02-17T12:05:05.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.775+0000 7f77cb7fe640 1 -- 172.21.15.94:0/1607727129 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77cc005020 con 0x7f77d4197620 2024-02-17T12:05:05.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.775+0000 7f77dc640640 1 -- 172.21.15.94:0/1607727129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d41a4330 con 0x7f77d4197620 2024-02-17T12:05:05.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.775+0000 7f77cb7fe640 1 -- 172.21.15.94:0/1607727129 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc010690 con 0x7f77d4197620 2024-02-17T12:05:05.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.775+0000 7f77dc640640 1 -- 172.21.15.94:0/1607727129 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d41a47c0 con 0x7f77d4197620 2024-02-17T12:05:05.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.779+0000 7f77cb7fe640 1 -- 172.21.15.94:0/1607727129 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f77cc005500 con 0x7f77d4197620 2024-02-17T12:05:05.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.781+0000 7f77df210740 1 Processor -- start 2024-02-17T12:05:05.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.781+0000 7f77cb7fe640 1 --2- 172.21.15.94:0/1607727129 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f77c00a22b0 0x7f77c00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.781+0000 7f77df210740 1 -- start start 2024-02-17T12:05:05.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.781+0000 7f77cb7fe640 1 -- 172.21.15.94:0/1607727129 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f77cc014070 con 0x7f77d4197620 2024-02-17T12:05:05.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.781+0000 7f77d9bdb640 1 --2- 172.21.15.94:0/1607727129 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f77c00a22b0 0x7f77c00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.782+0000 7f77d9bdb640 1 --2- 172.21.15.94:0/1607727129 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f77c00a22b0 0x7f77c00a4770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f77c402c170 tx=0x7f77c4000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.782+0000 7f77df210740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f576c2b0 0x55e5f576c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.782+0000 7f77df210740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5775270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.782+0000 7f77df210740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f57757b0 0x55e5f5777ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.782+0000 7f77da3dc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f576c2b0 0x55e5f576c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.783+0000 7f77da3dc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f576c2b0 0x55e5f576c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47126/0 (socket says 172.21.15.94:47126) 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.783+0000 7f77da3dc640 1 -- 172.21.15.94:0/2542968402 learned_addr learned my addr 172.21.15.94:0/2542968402 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.783+0000 7f77df210740 1 -- 172.21.15.94:0/2542968402 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f56db280 con 0x55e5f576cbd0 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.783+0000 7f77df210740 1 -- 172.21.15.94:0/2542968402 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f56d87b0 con 0x55e5f57757b0 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.783+0000 7f77df210740 1 -- 172.21.15.94:0/2542968402 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f56d3f90 con 0x55e5f576c2b0 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.783+0000 7f77dabdd640 1 --2- 172.21.15.94:0/2542968402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f57757b0 0x55e5f5777ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.783+0000 7f77d9bdb640 1 --2- 172.21.15.94:0/2542968402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5775270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.783+0000 7f77da3dc640 1 -- 172.21.15.94:0/2542968402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f57757b0 msgr2=0x55e5f5777ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.783+0000 7f77da3dc640 1 --2- 172.21.15.94:0/2542968402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f57757b0 0x55e5f5777ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.783+0000 7f77da3dc640 1 -- 172.21.15.94:0/2542968402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 msgr2=0x55e5f5775270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.783+0000 7f77da3dc640 1 --2- 172.21.15.94:0/2542968402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5775270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.783+0000 7f77da3dc640 1 -- 172.21.15.94:0/2542968402 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5f5692570 con 0x55e5f576c2b0 2024-02-17T12:05:05.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.783+0000 7f77da3dc640 1 --2- 172.21.15.94:0/2542968402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f576c2b0 0x55e5f576c690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009e10 tx=0x7f77d0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8d5a415a0bb2916 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:05.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.784+0000 7f77c8bfe640 1 -- 172.21.15.94:0/2542968402 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0019630 con 0x55e5f576c2b0 2024-02-17T12:05:05.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.784+0000 7f77c8bfe640 1 -- 172.21.15.94:0/2542968402 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77d0019c70 con 0x55e5f576c2b0 2024-02-17T12:05:05.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.784+0000 7f77c8bfe640 1 -- 172.21.15.94:0/2542968402 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d000fd20 con 0x55e5f576c2b0 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.784+0000 7f77df210740 1 -- 172.21.15.94:0/2542968402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f576c2b0 msgr2=0x55e5f576c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.784+0000 7f77df210740 1 --2- 172.21.15.94:0/2542968402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f576c2b0 0x55e5f576c690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009e10 tx=0x7f77d0011810 comp rx=0 tx=0).stop 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.785+0000 7f77df210740 1 -- 172.21.15.94:0/2542968402 shutdown_connections 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.785+0000 7f77df210740 1 --2- 172.21.15.94:0/2542968402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f57757b0 0x55e5f5777ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.785+0000 7f77df210740 1 --2- 172.21.15.94:0/2542968402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5775270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.785+0000 7f77df210740 1 --2- 172.21.15.94:0/2542968402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f576c2b0 0x55e5f576c690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.785+0000 7f77df210740 1 -- 172.21.15.94:0/2542968402 >> 172.21.15.94:0/2542968402 conn(0x55e5f5766990 msgr2=0x55e5f5766d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.785+0000 7f77df210740 1 -- 172.21.15.94:0/2542968402 shutdown_connections 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.785+0000 7f77df210740 1 -- 172.21.15.94:0/2542968402 wait complete. 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.785+0000 7f77df210740 1 -- start start 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.785+0000 7f77df210740 1 Processor -- start 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77df210740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f5772610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77df210740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5772b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77df210740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f57757b0 0x55e5f5773090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77df210740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f53a7430 con 0x55e5f576cbd0 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77da3dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f5772610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77df210740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f5363e00 con 0x55e5f576c2b0 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77df210740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f5658fd0 con 0x55e5f57757b0 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77da3dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f5772610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:35136/0 (socket says 172.21.15.94:35136) 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77da3dc640 1 -- 172.21.15.94:0/2510291482 learned_addr learned my addr 172.21.15.94:0/2510291482 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77df210740 10 client.0 _ll_register_callbacks cb 0x55e5f56aefd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77dabdd640 1 --2- 172.21.15.94:0/2510291482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f57757b0 0x55e5f5773090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77d9bdb640 1 --2- 172.21.15.94:0/2510291482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5772b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77da3dc640 1 -- 172.21.15.94:0/2510291482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f57757b0 msgr2=0x55e5f5773090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77da3dc640 1 --2- 172.21.15.94:0/2510291482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f57757b0 0x55e5f5773090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77da3dc640 1 -- 172.21.15.94:0/2510291482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 msgr2=0x55e5f5772b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77da3dc640 1 --2- 172.21.15.94:0/2510291482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5772b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.786+0000 7f77da3dc640 1 -- 172.21.15.94:0/2510291482 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d0009a70 con 0x55e5f576c2b0 2024-02-17T12:05:05.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.787+0000 7f77da3dc640 1 --2- 172.21.15.94:0/2510291482 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f5772610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f77d0010000 tx=0x7f77d000f6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152adc740d336ce5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:05.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.787+0000 7f77bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:05:05.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.787+0000 7f77bdffb640 1 -- 172.21.15.94:0/2510291482 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0005440 con 0x55e5f576c2b0 2024-02-17T12:05:05.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.787+0000 7f77df210740 1 -- 172.21.15.94:0/2510291482 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e5f53d37e0 con 0x55e5f576c2b0 2024-02-17T12:05:05.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.787+0000 7f77bdffb640 1 -- 172.21.15.94:0/2510291482 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f77d000f4c0 con 0x55e5f576c2b0 2024-02-17T12:05:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.789+0000 7f77bdffb640 1 -- 172.21.15.94:0/2510291482 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77d00055a0 con 0x55e5f576c2b0 2024-02-17T12:05:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.789+0000 7f77df210740 10 client.26846 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.789+0000 7f77df210740 1 -- 172.21.15.94:0/2510291482 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55e5f56d2610 con 0x55e5f576c2b0 2024-02-17T12:05:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.790+0000 7f77bdffb640 1 -- 172.21.15.94:0/2510291482 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e5f53d37e0 con 0x55e5f576c2b0 2024-02-17T12:05:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.790+0000 7f77bdffb640 1 -- 172.21.15.94:0/2510291482 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77d0004cf0 con 0x55e5f576c2b0 2024-02-17T12:05:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.790+0000 7f77df210740 10 client.26846 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.790+0000 7f77df210740 10 client.26846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.790+0000 7f77df210740 20 client.26846 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.790+0000 7f77df210740 1 --2- 172.21.15.94:0/2510291482 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e5f57856f0 0x55e5f5787ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.790+0000 7f77df210740 4 client.26846 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.790+0000 7f77df210740 1 -- 172.21.15.94:0/2510291482 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5f5410720 con 0x55e5f57856f0 2024-02-17T12:05:05.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.790+0000 7f77d9bdb640 1 --2- 172.21.15.94:0/2510291482 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e5f57856f0 0x55e5f5787ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.791+0000 7f77bdffb640 10 client.26846 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:05.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.791+0000 7f77d9bdb640 1 --2- 172.21.15.94:0/2510291482 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e5f57856f0 0x55e5f5787ae0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc32300c951b58fe server_cookie=9cc161a4c4e01508 in_seq=0 out_seq=0 2024-02-17T12:05:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.792+0000 7f77bdffb640 1 -- 172.21.15.94:0/2510291482 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e5f5410720 con 0x55e5f57856f0 2024-02-17T12:05:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.792+0000 7f77bdffb640 10 client.26846 handle_command_reply: tid=0 2024-02-17T12:05:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.792+0000 7f77df210740 1 client.26846 shutdown 2024-02-17T12:05:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.792+0000 7f77df210740 1 -- 172.21.15.94:0/2510291482 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 msgr2=0x55e5f5772610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.792+0000 7f77df210740 1 --2- 172.21.15.94:0/2510291482 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f5772610 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f77d0010000 tx=0x7f77d000f6c0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.792+0000 7f77df210740 1 -- 172.21.15.94:0/2510291482 shutdown_connections 2024-02-17T12:05:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.792+0000 7f77df210740 1 --2- 172.21.15.94:0/2510291482 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e5f57856f0 0x55e5f5787ae0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.792+0000 7f77df210740 1 --2- 172.21.15.94:0/2510291482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f57757b0 0x55e5f5773090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.792+0000 7f77df210740 1 --2- 172.21.15.94:0/2510291482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5772b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.792+0000 7f77df210740 1 --2- 172.21.15.94:0/2510291482 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f5772610 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.792+0000 7f77df210740 1 -- 172.21.15.94:0/2510291482 >> 172.21.15.94:0/2510291482 conn(0x55e5f5766990 msgr2=0x55e5f5768730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.792+0000 7f77bdffb640 0 client.26846 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.792+0000 7f77df210740 1 -- 172.21.15.94:0/2510291482 shutdown_connections 2024-02-17T12:05:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.793+0000 7f77df210740 1 -- 172.21.15.94:0/2510291482 wait complete. 2024-02-17T12:05:05.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.793+0000 7f77df210740 20 client.26846 trim_cache size 0 max 16384 2024-02-17T12:05:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.818+0000 7f77df210740 1 Processor -- start 2024-02-17T12:05:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.818+0000 7f77df210740 1 -- start start 2024-02-17T12:05:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.818+0000 7f77df210740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f5774d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.818+0000 7f77df210740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f576cbd0 0x55e5f577c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.818+0000 7f77df210740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f57757b0 0x55e5f577cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.818+0000 7f77da3dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f5774d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.818+0000 7f77da3dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f5774d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:35144/0 (socket says 172.21.15.94:35144) 2024-02-17T12:05:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.818+0000 7f77df210740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f576bcf0 con 0x55e5f57757b0 2024-02-17T12:05:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.818+0000 7f77df210740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f5760e20 con 0x55e5f576c2b0 2024-02-17T12:05:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.818+0000 7f77df210740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f57679e0 con 0x55e5f576cbd0 2024-02-17T12:05:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.818+0000 7f77da3dc640 1 -- 172.21.15.94:0/2083858651 learned_addr learned my addr 172.21.15.94:0/2083858651 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:05.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.818+0000 7f77d9bdb640 1 --2- 172.21.15.94:0/2083858651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f576cbd0 0x55e5f577c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.818+0000 7f77dabdd640 1 --2- 172.21.15.94:0/2083858651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f57757b0 0x55e5f577cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.819+0000 7f77d9bdb640 1 -- 172.21.15.94:0/2083858651 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 msgr2=0x55e5f5774d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.819+0000 7f77d9bdb640 1 --2- 172.21.15.94:0/2083858651 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f5774d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.819+0000 7f77d9bdb640 1 -- 172.21.15.94:0/2083858651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f57757b0 msgr2=0x55e5f577cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.819+0000 7f77d9bdb640 1 --2- 172.21.15.94:0/2083858651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f57757b0 0x55e5f577cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.819+0000 7f77d9bdb640 1 -- 172.21.15.94:0/2083858651 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5f566eac0 con 0x55e5f576cbd0 2024-02-17T12:05:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.819+0000 7f77da3dc640 1 --2- 172.21.15.94:0/2083858651 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f5774d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.819+0000 7f77d9bdb640 1 --2- 172.21.15.94:0/2083858651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f576cbd0 0x55e5f577c660 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009230 tx=0x7f77c403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69bf15ce5966ade8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.819+0000 7f77bffff640 1 -- 172.21.15.94:0/2083858651 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c402cae0 con 0x55e5f576cbd0 2024-02-17T12:05:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.819+0000 7f77bffff640 1 -- 172.21.15.94:0/2083858651 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77c4052070 con 0x55e5f576cbd0 2024-02-17T12:05:05.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.819+0000 7f77bffff640 1 -- 172.21.15.94:0/2083858651 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4004df0 con 0x55e5f576cbd0 2024-02-17T12:05:05.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.820+0000 7f77df210740 1 -- 172.21.15.94:0/2083858651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f576cbd0 msgr2=0x55e5f577c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.820+0000 7f77df210740 1 --2- 172.21.15.94:0/2083858651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f576cbd0 0x55e5f577c660 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009230 tx=0x7f77c403f000 comp rx=0 tx=0).stop 2024-02-17T12:05:05.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.820+0000 7f77df210740 1 -- 172.21.15.94:0/2083858651 shutdown_connections 2024-02-17T12:05:05.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.820+0000 7f77df210740 1 --2- 172.21.15.94:0/2083858651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f57757b0 0x55e5f577cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.820+0000 7f77df210740 1 --2- 172.21.15.94:0/2083858651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f576cbd0 0x55e5f577c660 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.820+0000 7f77df210740 1 --2- 172.21.15.94:0/2083858651 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f5774d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.820+0000 7f77df210740 1 -- 172.21.15.94:0/2083858651 >> 172.21.15.94:0/2083858651 conn(0x55e5f5766990 msgr2=0x55e5f576d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:05.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.822+0000 7f77df210740 1 -- 172.21.15.94:0/2083858651 shutdown_connections 2024-02-17T12:05:05.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.822+0000 7f77df210740 1 -- 172.21.15.94:0/2083858651 wait complete. 2024-02-17T12:05:05.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.823+0000 7f77df210740 1 -- start start 2024-02-17T12:05:05.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.823+0000 7f77df210740 1 Processor -- start 2024-02-17T12:05:05.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.824+0000 7f77df210740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f577c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.824+0000 7f77df210740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5778cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.824+0000 7f77df210740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f57757b0 0x55e5f5779210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.824+0000 7f77da3dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f577c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.824+0000 7f77d9bdb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5778cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.824+0000 7f77da3dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f577c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:35154/0 (socket says 172.21.15.94:35154) 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.824+0000 7f77d9bdb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5778cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:35442/0 (socket says 172.21.15.94:35442) 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.824+0000 7f77da3dc640 1 -- 172.21.15.94:0/693738815 learned_addr learned my addr 172.21.15.94:0/693738815 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.824+0000 7f77df210740 1 -- 172.21.15.94:0/693738815 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f5779820 con 0x55e5f576cbd0 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.824+0000 7f77dabdd640 1 --2- 172.21.15.94:0/693738815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f57757b0 0x55e5f5779210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.824+0000 7f77df210740 1 -- 172.21.15.94:0/693738815 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f5779990 con 0x55e5f576c2b0 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.825+0000 7f77df210740 1 -- 172.21.15.94:0/693738815 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5f5779b50 con 0x55e5f57757b0 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.825+0000 7f77df210740 10 client.0 _ll_register_callbacks cb 0x7f77d0009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.825+0000 7f77dabdd640 1 -- 172.21.15.94:0/693738815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 msgr2=0x55e5f577c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.825+0000 7f77dabdd640 1 --2- 172.21.15.94:0/693738815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f577c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.825+0000 7f77dabdd640 1 -- 172.21.15.94:0/693738815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 msgr2=0x55e5f5778cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:05.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.825+0000 7f77dabdd640 1 --2- 172.21.15.94:0/693738815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5778cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.825+0000 7f77dabdd640 1 -- 172.21.15.94:0/693738815 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5f56d2610 con 0x55e5f57757b0 2024-02-17T12:05:05.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.825+0000 7f77d9bdb640 1 --2- 172.21.15.94:0/693738815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5778cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:05.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.825+0000 7f77dabdd640 1 --2- 172.21.15.94:0/693738815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f57757b0 0x55e5f5779210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0cbb60 tx=0x7f77cc09ae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d873f52de8078fe server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:05.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.825+0000 7f77bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:05:05.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.825+0000 7f77bdffb640 1 -- 172.21.15.94:0/693738815 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc09d040 con 0x55e5f57757b0 2024-02-17T12:05:05.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.825+0000 7f77bdffb640 1 -- 172.21.15.94:0/693738815 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f77cc0b3b30 con 0x55e5f57757b0 2024-02-17T12:05:05.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.825+0000 7f77df210740 1 -- 172.21.15.94:0/693738815 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77d0019c70 con 0x55e5f57757b0 2024-02-17T12:05:05.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.826+0000 7f77bdffb640 1 -- 172.21.15.94:0/693738815 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77cc0cbff0 con 0x55e5f57757b0 2024-02-17T12:05:05.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.826+0000 7f77df210740 10 client.26890 fetch_fsmap learned FSMap version 25 2024-02-17T12:05:05.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.826+0000 7f77df210740 1 -- 172.21.15.94:0/693738815 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f77d410e810 con 0x55e5f57757b0 2024-02-17T12:05:05.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.826+0000 7f77bdffb640 1 -- 172.21.15.94:0/693738815 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77d0019c70 con 0x55e5f57757b0 2024-02-17T12:05:05.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.826+0000 7f77bdffb640 1 -- 172.21.15.94:0/693738815 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 25) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f77cc0cd040 con 0x55e5f57757b0 2024-02-17T12:05:05.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.826+0000 7f77df210740 10 client.26890 fetch_fsmap finished waiting for FSMap version 25 2024-02-17T12:05:05.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.826+0000 7f77df210740 10 client.26890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:05.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.826+0000 7f77df210740 20 client.26890 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:05.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.826+0000 7f77df210740 1 --2- 172.21.15.94:0/693738815 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e5f57e5dc0 0x55e5f57e81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:05.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.826+0000 7f77df210740 4 client.26890 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.826+0000 7f77df210740 1 -- 172.21.15.94:0/693738815 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e5f545f890 con 0x55e5f57e5dc0 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.826+0000 7f77da3dc640 1 --2- 172.21.15.94:0/693738815 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e5f57e5dc0 0x55e5f57e81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.827+0000 7f77bdffb640 10 client.26890 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.827+0000 7f77da3dc640 1 --2- 172.21.15.94:0/693738815 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e5f57e5dc0 0x55e5f57e81e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbda23746d36b1a6 server_cookie=836e773684f1ab9d in_seq=0 out_seq=0 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.827+0000 7f77bdffb640 1 -- 172.21.15.94:0/693738815 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55e5f545f890 con 0x55e5f57e5dc0 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.827+0000 7f77bdffb640 10 client.26890 handle_command_reply: tid=0 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.827+0000 7f77df210740 1 client.26890 shutdown 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.828+0000 7f77df210740 1 -- 172.21.15.94:0/693738815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f57757b0 msgr2=0x55e5f5779210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.828+0000 7f77df210740 1 --2- 172.21.15.94:0/693738815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f57757b0 0x55e5f5779210 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0cbb60 tx=0x7f77cc09ae00 comp rx=0 tx=0).stop 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.828+0000 7f77df210740 1 -- 172.21.15.94:0/693738815 shutdown_connections 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.828+0000 7f77df210740 1 --2- 172.21.15.94:0/693738815 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e5f57e5dc0 0x55e5f57e81e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.828+0000 7f77df210740 1 --2- 172.21.15.94:0/693738815 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e5f57757b0 0x55e5f5779210 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.828+0000 7f77df210740 1 --2- 172.21.15.94:0/693738815 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e5f576cbd0 0x55e5f5778cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.828+0000 7f77df210740 1 --2- 172.21.15.94:0/693738815 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e5f576c2b0 0x55e5f577c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.828+0000 7f77df210740 1 -- 172.21.15.94:0/693738815 >> 172.21.15.94:0/693738815 conn(0x55e5f5766990 msgr2=0x55e5f57651c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:05.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.828+0000 7f77bdffb640 0 client.26890 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:05.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.828+0000 7f77df210740 1 -- 172.21.15.94:0/693738815 shutdown_connections 2024-02-17T12:05:05.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.828+0000 7f77df210740 1 -- 172.21.15.94:0/693738815 wait complete. 2024-02-17T12:05:05.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.828+0000 7f77df210740 20 client.26890 trim_cache size 0 max 16384 2024-02-17T12:05:05.832 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:05:05.832 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (200 inodes in the stack)", 2024-02-17T12:05:05.832 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:05:05.832 INFO:teuthology.orchestra.run.smithi094.stdout: "795c9bd8-4280-4426-a2c5-fdfb055f1ff0": { 2024-02-17T12:05:05.832 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:05:05.833 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "795c9bd8-4280-4426-a2c5-fdfb055f1ff0", 2024-02-17T12:05:05.833 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:05:05.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:05:05.833 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:05:05.833 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:05:05.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.829+0000 7f77dc640640 1 -- 172.21.15.94:0/1607727129 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f77c00a22b0 msgr2=0x7f77c00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.829+0000 7f77dc640640 1 --2- 172.21.15.94:0/1607727129 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f77c00a22b0 0x7f77c00a4770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f77c402c170 tx=0x7f77c4000bc0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.829+0000 7f77dc640640 1 -- 172.21.15.94:0/1607727129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77d4197620 msgr2=0x7f77d4197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:05.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.829+0000 7f77dc640640 1 --2- 172.21.15.94:0/1607727129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77d4197620 0x7f77d4197ad0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009fd0 tx=0x7f77cc00ef10 comp rx=0 tx=0).stop 2024-02-17T12:05:05.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.829+0000 7f77dc640640 1 -- 172.21.15.94:0/1607727129 shutdown_connections 2024-02-17T12:05:05.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.829+0000 7f77dc640640 1 --2- 172.21.15.94:0/1607727129 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f77c00a22b0 0x7f77c00a4770 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.829+0000 7f77dc640640 1 --2- 172.21.15.94:0/1607727129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77d4197620 0x7f77d4197ad0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.829+0000 7f77dc640640 1 --2- 172.21.15.94:0/1607727129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77d4106430 0x7f77d419d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.829+0000 7f77dc640640 1 --2- 172.21.15.94:0/1607727129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77d4101360 0x7f77d419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:05.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.829+0000 7f77dc640640 1 -- 172.21.15.94:0/1607727129 >> 172.21.15.94:0/1607727129 conn(0x7f77d40fcc20 msgr2=0x7f77d40fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:05.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.829+0000 7f77dc640640 1 -- 172.21.15.94:0/1607727129 shutdown_connections 2024-02-17T12:05:05.835 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:05.829+0000 7f77dc640640 1 -- 172.21.15.94:0/1607727129 wait complete. 2024-02-17T12:05:05.846 INFO:tasks.cephfs.filesystem:scrub status for tag:795c9bd8-4280-4426-a2c5-fdfb055f1ff0 - {'path': '/', 'tag': '795c9bd8-4280-4426-a2c5-fdfb055f1ff0', 'options': 'recursive,force'} 2024-02-17T12:05:06.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:05 smithi094 ceph-mon[32499]: pgmap v213: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 11 MiB/s wr, 895 op/s 2024-02-17T12:05:06.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:05 smithi189 ceph-mon[36588]: pgmap v213: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 11 MiB/s wr, 895 op/s 2024-02-17T12:05:06.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:05 smithi125 ceph-mon[36819]: pgmap v213: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 11 MiB/s wr, 895 op/s 2024-02-17T12:05:07.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:06 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:05:07.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:06 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:05:07.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:06 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:05:07.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:06 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:05:07.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:06 smithi189 ceph-mon[36588]: pgmap v214: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 7.8 MiB/s wr, 661 op/s 2024-02-17T12:05:07.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:06 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:05:07.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:06 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:05:07.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:06 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:05:07.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:06 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:05:07.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:06 smithi125 ceph-mon[36819]: pgmap v214: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 7.8 MiB/s wr, 661 op/s 2024-02-17T12:05:07.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:06 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:05:07.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:06 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:05:07.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:06 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:05:07.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:06 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:05:07.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:06 smithi094 ceph-mon[32499]: pgmap v214: 97 pgs: 97 active+clean; 4.9 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 7.8 MiB/s wr, 661 op/s 2024-02-17T12:05:08.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:07 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:05:08.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:07 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:05:08.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:07 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:05:09.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:08 smithi189 ceph-mon[36588]: pgmap v215: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 31 MiB/s wr, 2.00k op/s 2024-02-17T12:05:09.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:08 smithi125 ceph-mon[36819]: pgmap v215: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 31 MiB/s wr, 2.00k op/s 2024-02-17T12:05:09.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:08 smithi094 ceph-mon[32499]: pgmap v215: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 31 MiB/s wr, 2.00k op/s 2024-02-17T12:05:10.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 410.0 s, 1768.6 tps, lat 65.587 ms stddev 149.156 2024-02-17T12:05:12.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:11 smithi094 ceph-mon[32499]: pgmap v216: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 26 MiB/s wr, 1.65k op/s 2024-02-17T12:05:12.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:11 smithi189 ceph-mon[36588]: pgmap v216: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 26 MiB/s wr, 1.65k op/s 2024-02-17T12:05:12.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:11 smithi125 ceph-mon[36819]: pgmap v216: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 26 MiB/s wr, 1.65k op/s 2024-02-17T12:05:13.948 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:13 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.l.service: Scheduled restart job, restart counter is at 2. 2024-02-17T12:05:13.948 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:13 smithi189 systemd[1]: Stopped Ceph mds.l for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T12:05:13.948 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:13 smithi189 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.l.service: Consumed 1.091s CPU time. 2024-02-17T12:05:13.948 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:13 smithi189 systemd[1]: Starting Ceph mds.l for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:05:14.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:13 smithi094 ceph-mon[32499]: pgmap v217: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 27 MiB/s wr, 1.69k op/s 2024-02-17T12:05:14.215 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:13 smithi189 ceph-mon[36588]: pgmap v217: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 27 MiB/s wr, 1.69k op/s 2024-02-17T12:05:14.215 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:13 smithi189 podman[72348]: 2024-02-17 12:05:13.858489219 +0000 UTC m=+0.021965273 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:05:14.216 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:13 smithi189 podman[72348]: 2024-02-17 12:05:13.982548464 +0000 UTC m=+0.146024504 container create 5dc8e5360759ff9943e845b7629f27b84a5611da9dc02ea1e33e195b0fb5780c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-17T12:05:14.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:13 smithi125 ceph-mon[36819]: pgmap v217: 97 pgs: 97 active+clean; 5.0 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 27 MiB/s wr, 1.69k op/s 2024-02-17T12:05:14.653 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:14 smithi189 podman[72348]: 2024-02-17 12:05:14.215755407 +0000 UTC m=+0.379231459 container init 5dc8e5360759ff9943e845b7629f27b84a5611da9dc02ea1e33e195b0fb5780c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-17T12:05:14.653 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:14 smithi189 podman[72348]: 2024-02-17 12:05:14.220914423 +0000 UTC m=+0.384390459 container start 5dc8e5360759ff9943e845b7629f27b84a5611da9dc02ea1e33e195b0fb5780c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-17T12:05:14.653 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:14 smithi189 ceph-mds[72367]: -- 172.21.15.189:0/3915903466 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a7bd5fe1a0 con 0x55a7bd5d4c00 2024-02-17T12:05:14.653 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:14 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[72363]: starting mds.l at 2024-02-17T12:05:14.653 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:14 smithi189 bash[72348]: 5dc8e5360759ff9943e845b7629f27b84a5611da9dc02ea1e33e195b0fb5780c 2024-02-17T12:05:14.653 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:05:14 smithi189 systemd[1]: Started Ceph mds.l for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T12:05:15.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:14 smithi094 ceph-mon[32499]: daemon mds.l restarted 2024-02-17T12:05:15.243 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 415.0 s, 2428.6 tps, lat 41.258 ms stddev 47.182 2024-02-17T12:05:15.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:14 smithi189 ceph-mon[36588]: daemon mds.l restarted 2024-02-17T12:05:15.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:14 smithi125 ceph-mon[36819]: daemon mds.l restarted 2024-02-17T12:05:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.548+0000 7f207bf2c640 1 -- 172.21.15.94:0/786490703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f20741013b0 msgr2=0x7f2074104cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.548+0000 7f207bf2c640 1 --2- 172.21.15.94:0/786490703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f20741013b0 0x7f2074104cc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2064009930 tx=0x7f206402bc30 comp rx=0 tx=0).stop 2024-02-17T12:05:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.548+0000 7f207bf2c640 1 -- 172.21.15.94:0/786490703 shutdown_connections 2024-02-17T12:05:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.548+0000 7f207bf2c640 1 --2- 172.21.15.94:0/786490703 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f20741053a0 0x7f207410cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.548+0000 7f207bf2c640 1 --2- 172.21.15.94:0/786490703 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f20741013b0 0x7f2074104cc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.548+0000 7f207bf2c640 1 --2- 172.21.15.94:0/786490703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2074100a00 0x7f2074100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.548+0000 7f207bf2c640 1 -- 172.21.15.94:0/786490703 >> 172.21.15.94:0/786490703 conn(0x7f2074076610 msgr2=0x7f20740fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.548+0000 7f207bf2c640 1 -- 172.21.15.94:0/786490703 shutdown_connections 2024-02-17T12:05:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.548+0000 7f207bf2c640 1 -- 172.21.15.94:0/786490703 wait complete. 2024-02-17T12:05:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.549+0000 7f207bf2c640 1 Processor -- start 2024-02-17T12:05:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.549+0000 7f207bf2c640 1 -- start start 2024-02-17T12:05:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f207bf2c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2074100a00 0x7f2074195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f207bf2c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20741053a0 0x7f2074195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f207bf2c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2074199e90 0x7f207419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f207bf2c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207419a880 con 0x7f2074100a00 2024-02-17T12:05:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f207bf2c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207419a9f0 con 0x7f2074199e90 2024-02-17T12:05:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f207bf2c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207419ab60 con 0x7f20741053a0 2024-02-17T12:05:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f2079cc8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2074100a00 0x7f2074195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f2079cc8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2074100a00 0x7f2074195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36074/0 (socket says 172.21.15.94:36074) 2024-02-17T12:05:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f2079cc8640 1 -- 172.21.15.94:0/3291024162 learned_addr learned my addr 172.21.15.94:0/3291024162 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f20794c7640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20741053a0 0x7f2074195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f207a4c9640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2074199e90 0x7f207419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f2079cc8640 1 -- 172.21.15.94:0/3291024162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20741053a0 msgr2=0x7f2074195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f2079cc8640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20741053a0 0x7f2074195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f2079cc8640 1 -- 172.21.15.94:0/3291024162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2074199e90 msgr2=0x7f207419a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f2079cc8640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2074199e90 0x7f207419a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f2079cc8640 1 -- 172.21.15.94:0/3291024162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2064009560 con 0x7f2074100a00 2024-02-17T12:05:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.550+0000 7f207a4c9640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2074199e90 0x7f207419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.551+0000 7f2079cc8640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2074100a00 0x7f2074195620 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f207000d950 tx=0x7f207000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.551+0000 7f206affd640 1 -- 172.21.15.94:0/3291024162 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2070009c30 con 0x7f2074100a00 2024-02-17T12:05:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.551+0000 7f207bf2c640 1 -- 172.21.15.94:0/3291024162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20741a2140 con 0x7f2074100a00 2024-02-17T12:05:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.551+0000 7f206affd640 1 -- 172.21.15.94:0/3291024162 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2070009d90 con 0x7f2074100a00 2024-02-17T12:05:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.551+0000 7f206affd640 1 -- 172.21.15.94:0/3291024162 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207000f7e0 con 0x7f2074100a00 2024-02-17T12:05:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.551+0000 7f207bf2c640 1 -- 172.21.15.94:0/3291024162 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20741a2610 con 0x7f2074100a00 2024-02-17T12:05:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.554+0000 7f206affd640 1 -- 172.21.15.94:0/3291024162 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2070010ad0 con 0x7f2074100a00 2024-02-17T12:05:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.554+0000 7f207ebdb740 1 Processor -- start 2024-02-17T12:05:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f207ebdb740 1 -- start start 2024-02-17T12:05:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f206affd640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f20600a2360 0x7f20600a4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f207ebdb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6bad0 0x559287a6a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f207ebdb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a6ad60 0x559287a6b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f207ebdb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a786f0 0x559287a7aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f207ebdb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592879d99f0 con 0x559287a6bad0 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f207ebdb740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592879d6f20 con 0x559287a786f0 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f207ebdb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592879d2700 con 0x559287a6ad60 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f207a4c9640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f20600a2360 0x7f20600a4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f206affd640 1 -- 172.21.15.94:0/3291024162 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f20700cdc20 con 0x7f2074100a00 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f2079cc8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6bad0 0x559287a6a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f2079cc8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6bad0 0x559287a6a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36080/0 (socket says 172.21.15.94:36080) 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f2079cc8640 1 -- 172.21.15.94:0/3223136065 learned_addr learned my addr 172.21.15.94:0/3223136065 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f207a4c9640 1 --2- 172.21.15.94:0/3223136065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a786f0 0x559287a7aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f20794c7640 1 --2- 172.21.15.94:0/3223136065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a6ad60 0x559287a6b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f2079cc8640 1 -- 172.21.15.94:0/3223136065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a6ad60 msgr2=0x559287a6b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f2079cc8640 1 --2- 172.21.15.94:0/3223136065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a6ad60 0x559287a6b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f2079cc8640 1 -- 172.21.15.94:0/3223136065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a786f0 msgr2=0x559287a7aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f2079cc8640 1 --2- 172.21.15.94:0/3223136065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a786f0 0x559287a7aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.555+0000 7f2079cc8640 1 -- 172.21.15.94:0/3223136065 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592879687b0 con 0x559287a6bad0 2024-02-17T12:05:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.556+0000 7f2079cc8640 1 --2- 172.21.15.94:0/3223136065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6bad0 0x559287a6a820 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f20700cb3c0 tx=0x7f2070000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c500690e2822f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.556+0000 7f207a4c9640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f20600a2360 0x7f20600a4820 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f206c00a780 tx=0x7f206c014580 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.556+0000 7f205ffff640 1 -- 172.21.15.94:0/3223136065 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207009f670 con 0x559287a6bad0 2024-02-17T12:05:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.556+0000 7f205ffff640 1 -- 172.21.15.94:0/3223136065 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f207009b730 con 0x559287a6bad0 2024-02-17T12:05:15.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.556+0000 7f205ffff640 1 -- 172.21.15.94:0/3223136065 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207009b410 con 0x559287a6bad0 2024-02-17T12:05:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.557+0000 7f207ebdb740 1 -- 172.21.15.94:0/3223136065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6bad0 msgr2=0x559287a6a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.557+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3223136065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6bad0 0x559287a6a820 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f20700cb3c0 tx=0x7f2070000bc0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.557+0000 7f207ebdb740 1 -- 172.21.15.94:0/3223136065 shutdown_connections 2024-02-17T12:05:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.557+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3223136065 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a786f0 0x559287a7aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.557+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3223136065 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a6ad60 0x559287a6b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.557+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3223136065 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6bad0 0x559287a6a820 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.557+0000 7f207ebdb740 1 -- 172.21.15.94:0/3223136065 >> 172.21.15.94:0/3223136065 conn(0x559287a32550 msgr2=0x559287a65a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.557+0000 7f207ebdb740 1 -- 172.21.15.94:0/3223136065 shutdown_connections 2024-02-17T12:05:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.557+0000 7f207ebdb740 1 -- 172.21.15.94:0/3223136065 wait complete. 2024-02-17T12:05:15.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.557+0000 7f207ebdb740 1 -- start start 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.557+0000 7f207ebdb740 1 Processor -- start 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.558+0000 7f207ebdb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6ad60 0x559287a75740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.558+0000 7f207ebdb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a6bad0 0x559287a75c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.558+0000 7f207ebdb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a786f0 0x559287a761c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.558+0000 7f207ebdb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559287977bf0 con 0x559287a6ad60 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.558+0000 7f207ebdb740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592876a6250 con 0x559287a786f0 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.558+0000 7f207ebdb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559287958200 con 0x559287a6bad0 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.558+0000 7f20794c7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a6bad0 0x559287a75c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.558+0000 7f20794c7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a6bad0 0x559287a75c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60324/0 (socket says 172.21.15.94:60324) 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.558+0000 7f20794c7640 1 -- 172.21.15.94:0/3877266042 learned_addr learned my addr 172.21.15.94:0/3877266042 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.558+0000 7f207a4c9640 1 --2- 172.21.15.94:0/3877266042 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a786f0 0x559287a761c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.558+0000 7f207ebdb740 10 client.0 _ll_register_callbacks cb 0x5592879c6d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.558+0000 7f2079cc8640 1 --2- 172.21.15.94:0/3877266042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6ad60 0x559287a75740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.559+0000 7f207a4c9640 1 -- 172.21.15.94:0/3877266042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a6bad0 msgr2=0x559287a75c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:15.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.559+0000 7f207a4c9640 1 --2- 172.21.15.94:0/3877266042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a6bad0 0x559287a75c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.559+0000 7f207a4c9640 1 -- 172.21.15.94:0/3877266042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6ad60 msgr2=0x559287a75740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.559+0000 7f207a4c9640 1 --2- 172.21.15.94:0/3877266042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6ad60 0x559287a75740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.559+0000 7f207a4c9640 1 -- 172.21.15.94:0/3877266042 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20741a2610 con 0x559287a786f0 2024-02-17T12:05:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.559+0000 7f207a4c9640 1 --2- 172.21.15.94:0/3877266042 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a786f0 0x559287a761c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x559287a66be0 tx=0x7f206c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aee869817d2e8489 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.560+0000 7f205d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:05:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.560+0000 7f205d7fa640 1 -- 172.21.15.94:0/3877266042 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206c004d20 con 0x559287a786f0 2024-02-17T12:05:15.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.560+0000 7f205d7fa640 1 -- 172.21.15.94:0/3877266042 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f206c012e10 con 0x559287a786f0 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.560+0000 7f207ebdb740 1 -- 172.21.15.94:0/3877266042 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592876d2600 con 0x559287a786f0 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.560+0000 7f205d7fa640 1 -- 172.21.15.94:0/3877266042 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f206c004e80 con 0x559287a786f0 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.560+0000 7f207ebdb740 10 client.26870 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.560+0000 7f207ebdb740 1 -- 172.21.15.94:0/3877266042 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5592879d0d80 con 0x559287a786f0 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.560+0000 7f205d7fa640 1 -- 172.21.15.94:0/3877266042 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5592876d2600 con 0x559287a786f0 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.560+0000 7f205d7fa640 1 -- 172.21.15.94:0/3877266042 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f206c005570 con 0x559287a786f0 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.561+0000 7f207ebdb740 10 client.26870 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.561+0000 7f207ebdb740 10 client.26870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.561+0000 7f207ebdb740 20 client.26870 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.561+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3877266042 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559287a99af0 0x559287a9bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.561+0000 7f207ebdb740 4 client.26870 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.561+0000 7f207ebdb740 1 -- 172.21.15.94:0/3877266042 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592877231b0 con 0x559287a99af0 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.561+0000 7f2079cc8640 1 --2- 172.21.15.94:0/3877266042 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559287a99af0 0x559287a9bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.561+0000 7f205d7fa640 10 client.26870 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.561+0000 7f2079cc8640 1 --2- 172.21.15.94:0/3877266042 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559287a99af0 0x559287a9bf10 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26eac01960530bd server_cookie=be837afa59f28c85 in_seq=0 out_seq=0 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.562+0000 7f205d7fa640 1 -- 172.21.15.94:0/3877266042 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592877231b0 con 0x559287a99af0 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.562+0000 7f205d7fa640 10 client.26870 handle_command_reply: tid=0 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.562+0000 7f207ebdb740 1 client.26870 shutdown 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.563+0000 7f207ebdb740 1 -- 172.21.15.94:0/3877266042 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a786f0 msgr2=0x559287a761c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.563+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3877266042 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a786f0 0x559287a761c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x559287a66be0 tx=0x7f206c004ab0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.563+0000 7f207ebdb740 1 -- 172.21.15.94:0/3877266042 shutdown_connections 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.563+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3877266042 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559287a99af0 0x559287a9bf10 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.563+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3877266042 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a786f0 0x559287a761c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.563+0000 7f205d7fa640 0 client.26870 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.563+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3877266042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a6bad0 0x559287a75c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.563+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3877266042 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6ad60 0x559287a75740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.563+0000 7f207ebdb740 1 -- 172.21.15.94:0/3877266042 >> 172.21.15.94:0/3877266042 conn(0x559287a32550 msgr2=0x559287a677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.563+0000 7f207ebdb740 1 -- 172.21.15.94:0/3877266042 shutdown_connections 2024-02-17T12:05:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.563+0000 7f207ebdb740 1 -- 172.21.15.94:0/3877266042 wait complete. 2024-02-17T12:05:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.563+0000 7f207ebdb740 20 client.26870 trim_cache size 0 max 16384 2024-02-17T12:05:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.581+0000 7f207ebdb740 1 Processor -- start 2024-02-17T12:05:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.581+0000 7f207ebdb740 1 -- start start 2024-02-17T12:05:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f207ebdb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6ad60 0x559287a77e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f207ebdb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6bad0 0x559287a75fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f207ebdb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a786f0 0x559287a729c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f207ebdb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559287a7b300 con 0x559287a6ad60 2024-02-17T12:05:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f207ebdb740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559287aa0f00 con 0x559287a6bad0 2024-02-17T12:05:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f207ebdb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592877231b0 con 0x559287a786f0 2024-02-17T12:05:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f20794c7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6bad0 0x559287a75fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f20794c7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6bad0 0x559287a75fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56674/0 (socket says 172.21.15.94:56674) 2024-02-17T12:05:15.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f20794c7640 1 -- 172.21.15.94:0/3413667261 learned_addr learned my addr 172.21.15.94:0/3413667261 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f2079cc8640 1 --2- 172.21.15.94:0/3413667261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6ad60 0x559287a77e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f20794c7640 1 -- 172.21.15.94:0/3413667261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a786f0 msgr2=0x559287a729c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:05:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f20794c7640 1 --2- 172.21.15.94:0/3413667261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a786f0 0x559287a729c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f20794c7640 1 -- 172.21.15.94:0/3413667261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6ad60 msgr2=0x559287a77e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f20794c7640 1 --2- 172.21.15.94:0/3413667261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6ad60 0x559287a77e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.582+0000 7f20794c7640 1 -- 172.21.15.94:0/3413667261 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55928796d7b0 con 0x559287a6bad0 2024-02-17T12:05:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.583+0000 7f20794c7640 1 --2- 172.21.15.94:0/3413667261 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6bad0 0x559287a75fe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2064009230 tx=0x7f20640049a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb0d8bb3006ace65 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.583+0000 7f205f7fe640 1 -- 172.21.15.94:0/3413667261 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206402c7e0 con 0x559287a6bad0 2024-02-17T12:05:15.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.583+0000 7f205f7fe640 1 -- 172.21.15.94:0/3413667261 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2064004510 con 0x559287a6bad0 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.583+0000 7f205f7fe640 1 -- 172.21.15.94:0/3413667261 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064005300 con 0x559287a6bad0 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.584+0000 7f207ebdb740 1 -- 172.21.15.94:0/3413667261 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6bad0 msgr2=0x559287a75fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.584+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3413667261 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6bad0 0x559287a75fe0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2064009230 tx=0x7f20640049a0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.584+0000 7f207ebdb740 1 -- 172.21.15.94:0/3413667261 shutdown_connections 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.584+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3413667261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a786f0 0x559287a729c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.584+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3413667261 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6bad0 0x559287a75fe0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.584+0000 7f207ebdb740 1 --2- 172.21.15.94:0/3413667261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6ad60 0x559287a77e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.584+0000 7f207ebdb740 1 -- 172.21.15.94:0/3413667261 >> 172.21.15.94:0/3413667261 conn(0x559287a32550 msgr2=0x559287a74010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.584+0000 7f207ebdb740 1 -- 172.21.15.94:0/3413667261 shutdown_connections 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.584+0000 7f207ebdb740 1 -- 172.21.15.94:0/3413667261 wait complete. 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.584+0000 7f207ebdb740 1 -- start start 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f207ebdb740 1 Processor -- start 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f207ebdb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6ad60 0x559287a7de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f207ebdb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6bad0 0x559287a7e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f207ebdb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a786f0 0x559287a7e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f2079cc8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6ad60 0x559287a7de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f20794c7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6bad0 0x559287a7e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f207ebdb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559287a7fdf0 con 0x559287a6bad0 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f207ebdb740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559287af0020 con 0x559287a6ad60 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f2079cc8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6ad60 0x559287a7de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56682/0 (socket says 172.21.15.94:56682) 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f2079cc8640 1 -- 172.21.15.94:0/2846069386 learned_addr learned my addr 172.21.15.94:0/2846069386 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f20794c7640 1 --2- 172.21.15.94:0/2846069386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6bad0 0x559287a7e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:36114/0 (socket says 172.21.15.94:36114) 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f207ebdb740 1 -- 172.21.15.94:0/2846069386 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559287af0190 con 0x559287a786f0 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f207ebdb740 10 client.0 _ll_register_callbacks cb 0x7f207000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.585+0000 7f207a4c9640 1 --2- 172.21.15.94:0/2846069386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a786f0 0x559287a7e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.586+0000 7f2079cc8640 1 -- 172.21.15.94:0/2846069386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a786f0 msgr2=0x559287a7e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.586+0000 7f2079cc8640 1 --2- 172.21.15.94:0/2846069386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a786f0 0x559287a7e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.586+0000 7f2079cc8640 1 -- 172.21.15.94:0/2846069386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6bad0 msgr2=0x559287a7e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.586+0000 7f2079cc8640 1 --2- 172.21.15.94:0/2846069386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6bad0 0x559287a7e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.586+0000 7f2079cc8640 1 -- 172.21.15.94:0/2846069386 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207410d7c0 con 0x559287a6ad60 2024-02-17T12:05:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.586+0000 7f2079cc8640 1 --2- 172.21.15.94:0/2846069386 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6ad60 0x559287a7de50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f207009a3b0 tx=0x7f2070005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e897b8625f3974b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:15.589 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.586+0000 7f205d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:05:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.586+0000 7f205d7fa640 1 -- 172.21.15.94:0/2846069386 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207009a6b0 con 0x559287a6ad60 2024-02-17T12:05:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.586+0000 7f205d7fa640 1 -- 172.21.15.94:0/2846069386 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f207009e750 con 0x559287a6ad60 2024-02-17T12:05:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.586+0000 7f207ebdb740 1 -- 172.21.15.94:0/2846069386 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f207009b730 con 0x559287a6ad60 2024-02-17T12:05:15.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.587+0000 7f205d7fa640 1 -- 172.21.15.94:0/2846069386 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20700a2da0 con 0x559287a6ad60 2024-02-17T12:05:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.587+0000 7f207ebdb740 10 client.26876 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.588+0000 7f207ebdb740 1 -- 172.21.15.94:0/2846069386 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559287993540 con 0x559287a6ad60 2024-02-17T12:05:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.588+0000 7f205d7fa640 1 -- 172.21.15.94:0/2846069386 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f207009f900 con 0x559287a6ad60 2024-02-17T12:05:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.588+0000 7f205d7fa640 1 -- 172.21.15.94:0/2846069386 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f20700cb5e0 con 0x559287a6ad60 2024-02-17T12:05:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.588+0000 7f207ebdb740 10 client.26876 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.588+0000 7f207ebdb740 10 client.26876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.588+0000 7f207ebdb740 20 client.26876 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.588+0000 7f207ebdb740 1 --2- 172.21.15.94:0/2846069386 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559287a99af0 0x559287af34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.588+0000 7f207ebdb740 4 client.26876 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:05:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.588+0000 7f207ebdb740 1 -- 172.21.15.94:0/2846069386 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f207009b730 con 0x559287a99af0 2024-02-17T12:05:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.588+0000 7f20794c7640 1 --2- 172.21.15.94:0/2846069386 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559287a99af0 0x559287af34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.589+0000 7f205d7fa640 10 client.26876 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:15.591 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.589+0000 7f20794c7640 1 --2- 172.21.15.94:0/2846069386 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559287a99af0 0x559287af34f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6dbad6c0541d219 server_cookie=85e6bb8746aa9eea in_seq=0 out_seq=0 2024-02-17T12:05:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f205d7fa640 1 -- 172.21.15.94:0/2846069386 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f207009b730 con 0x559287a99af0 2024-02-17T12:05:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f205d7fa640 10 client.26876 handle_command_reply: tid=0 2024-02-17T12:05:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f207ebdb740 1 client.26876 shutdown 2024-02-17T12:05:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f207ebdb740 1 -- 172.21.15.94:0/2846069386 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6ad60 msgr2=0x559287a7de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f207ebdb740 1 --2- 172.21.15.94:0/2846069386 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6ad60 0x559287a7de50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f207009a3b0 tx=0x7f2070005f90 comp rx=0 tx=0).stop 2024-02-17T12:05:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f207ebdb740 1 -- 172.21.15.94:0/2846069386 shutdown_connections 2024-02-17T12:05:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f207ebdb740 1 --2- 172.21.15.94:0/2846069386 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559287a99af0 0x559287af34f0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f207ebdb740 1 --2- 172.21.15.94:0/2846069386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559287a786f0 0x559287a7e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f207ebdb740 1 --2- 172.21.15.94:0/2846069386 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559287a6bad0 0x559287a7e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f207ebdb740 1 --2- 172.21.15.94:0/2846069386 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559287a6ad60 0x559287a7de50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f207ebdb740 1 -- 172.21.15.94:0/2846069386 >> 172.21.15.94:0/2846069386 conn(0x559287a32550 msgr2=0x559287a66da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f205d7fa640 0 client.26876 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f207ebdb740 1 -- 172.21.15.94:0/2846069386 shutdown_connections 2024-02-17T12:05:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f207ebdb740 1 -- 172.21.15.94:0/2846069386 wait complete. 2024-02-17T12:05:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.615+0000 7f207ebdb740 20 client.26876 trim_cache size 0 max 16384 2024-02-17T12:05:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:05:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:05:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:05:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:05:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.616+0000 7f207bf2c640 1 -- 172.21.15.94:0/3291024162 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f20600a2360 msgr2=0x7f20600a4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.616+0000 7f207bf2c640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f20600a2360 0x7f20600a4820 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f206c00a780 tx=0x7f206c014580 comp rx=0 tx=0).stop 2024-02-17T12:05:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.616+0000 7f207bf2c640 1 -- 172.21.15.94:0/3291024162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2074100a00 msgr2=0x7f2074195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.616+0000 7f207bf2c640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2074100a00 0x7f2074195620 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f207000d950 tx=0x7f207000de10 comp rx=0 tx=0).stop 2024-02-17T12:05:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.616+0000 7f207bf2c640 1 -- 172.21.15.94:0/3291024162 shutdown_connections 2024-02-17T12:05:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.616+0000 7f207bf2c640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f20600a2360 0x7f20600a4820 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.616+0000 7f207bf2c640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2074199e90 0x7f207419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.616+0000 7f207bf2c640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f20741053a0 0x7f2074195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.616+0000 7f207bf2c640 1 --2- 172.21.15.94:0/3291024162 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2074100a00 0x7f2074195620 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.616+0000 7f207bf2c640 1 -- 172.21.15.94:0/3291024162 >> 172.21.15.94:0/3291024162 conn(0x7f2074076610 msgr2=0x7f20740fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.616+0000 7f207bf2c640 1 -- 172.21.15.94:0/3291024162 shutdown_connections 2024-02-17T12:05:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:15.616+0000 7f207bf2c640 1 -- 172.21.15.94:0/3291024162 wait complete. 2024-02-17T12:05:16.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:15 smithi189 ceph-mon[36588]: pgmap v218: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 29 MiB/s wr, 1.88k op/s 2024-02-17T12:05:16.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:15 smithi189 ceph-mon[36588]: osdmap e90: 12 total, 12 up, 12 in 2024-02-17T12:05:16.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:15 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.189:6832/1336939575,v1:172.21.15.189:6833/1336939575] up:boot 2024-02-17T12:05:16.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:15 smithi189 ceph-mon[36588]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-02-17T12:05:16.152 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:15 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:05:16.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:15 smithi189 ceph-mon[36588]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T12:05:16.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:15 smithi094 ceph-mon[32499]: pgmap v218: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 29 MiB/s wr, 1.88k op/s 2024-02-17T12:05:16.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:15 smithi094 ceph-mon[32499]: osdmap e90: 12 total, 12 up, 12 in 2024-02-17T12:05:16.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:15 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.189:6832/1336939575,v1:172.21.15.189:6833/1336939575] up:boot 2024-02-17T12:05:16.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:15 smithi094 ceph-mon[32499]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-02-17T12:05:16.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:15 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:05:16.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:15 smithi094 ceph-mon[32499]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T12:05:16.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:15 smithi125 ceph-mon[36819]: pgmap v218: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 29 MiB/s wr, 1.88k op/s 2024-02-17T12:05:16.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:15 smithi125 ceph-mon[36819]: osdmap e90: 12 total, 12 up, 12 in 2024-02-17T12:05:16.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:15 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.189:6832/1336939575,v1:172.21.15.189:6833/1336939575] up:boot 2024-02-17T12:05:16.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:15 smithi125 ceph-mon[36819]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-02-17T12:05:16.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:15 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:05:16.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:15 smithi125 ceph-mon[36819]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-02-17T12:05:18.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:17 smithi094 ceph-mon[32499]: pgmap v220: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 32 MiB/s wr, 1.93k op/s 2024-02-17T12:05:18.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:17 smithi189 ceph-mon[36588]: pgmap v220: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 32 MiB/s wr, 1.93k op/s 2024-02-17T12:05:18.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:17 smithi125 ceph-mon[36819]: pgmap v220: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 32 MiB/s wr, 1.93k op/s 2024-02-17T12:05:20.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:19 smithi094 ceph-mon[32499]: pgmap v221: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.5 MiB/s wr, 689 op/s 2024-02-17T12:05:20.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:19 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:05:20.226 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:19 smithi189 ceph-mon[36588]: pgmap v221: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.5 MiB/s wr, 689 op/s 2024-02-17T12:05:20.226 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:19 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:05:20.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 420.0 s, 2352.4 tps, lat 41.459 ms stddev 47.172 2024-02-17T12:05:20.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:19 smithi125 ceph-mon[36819]: pgmap v221: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.5 MiB/s wr, 689 op/s 2024-02-17T12:05:20.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:19 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:05:21.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:20 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:05:21.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:20 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:05:21.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:20 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:05:21.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:20 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:05:21.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:20 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:05:21.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:20 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:05:21.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:20 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:05:21.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:20 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:05:21.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:20 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:05:22.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:21 smithi094 ceph-mon[32499]: pgmap v222: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.5 MiB/s wr, 689 op/s 2024-02-17T12:05:22.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:21 smithi189 ceph-mon[36588]: pgmap v222: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.5 MiB/s wr, 689 op/s 2024-02-17T12:05:22.435 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:21 smithi125 ceph-mon[36819]: pgmap v222: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.5 MiB/s wr, 689 op/s 2024-02-17T12:05:24.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:23 smithi189 ceph-mon[36588]: pgmap v223: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.6 MiB/s wr, 691 op/s 2024-02-17T12:05:24.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:23 smithi125 ceph-mon[36819]: pgmap v223: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.6 MiB/s wr, 691 op/s 2024-02-17T12:05:24.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:23 smithi094 ceph-mon[32499]: pgmap v223: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 7.6 MiB/s wr, 691 op/s 2024-02-17T12:05:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 425.0 s, 2723.4 tps, lat 37.637 ms stddev 49.415 2024-02-17T12:05:26.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:25 smithi189 ceph-mon[36588]: pgmap v224: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.0 MiB/s wr, 683 op/s 2024-02-17T12:05:26.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:25 smithi125 ceph-mon[36819]: pgmap v224: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.0 MiB/s wr, 683 op/s 2024-02-17T12:05:26.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:25 smithi094 ceph-mon[32499]: pgmap v224: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 8.0 MiB/s wr, 683 op/s 2024-02-17T12:05:27.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:27 smithi189 ceph-mon[36588]: pgmap v225: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.7 MiB/s wr, 572 op/s 2024-02-17T12:05:27.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:27 smithi125 ceph-mon[36819]: pgmap v225: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.7 MiB/s wr, 572 op/s 2024-02-17T12:05:27.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:27 smithi094 ceph-mon[32499]: pgmap v225: 97 pgs: 97 active+clean; 5.1 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.7 MiB/s wr, 572 op/s 2024-02-17T12:05:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.741+0000 7f22142a7640 1 -- 172.21.15.94:0/3546259299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f220c0ffc10 msgr2=0x7f220c10dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.741+0000 7f22142a7640 1 --2- 172.21.15.94:0/3546259299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f220c0ffc10 0x7f220c10dfc0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2204009ca0 tx=0x7f220402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.741+0000 7f22142a7640 1 -- 172.21.15.94:0/3546259299 shutdown_connections 2024-02-17T12:05:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.741+0000 7f22142a7640 1 --2- 172.21.15.94:0/3546259299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f220c0ffc10 0x7f220c10dfc0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.741+0000 7f22142a7640 1 --2- 172.21.15.94:0/3546259299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f220c0ff250 0x7f220c0ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.741+0000 7f22142a7640 1 --2- 172.21.15.94:0/3546259299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f220c107260 0x7f220c107640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.741+0000 7f22142a7640 1 -- 172.21.15.94:0/3546259299 >> 172.21.15.94:0/3546259299 conn(0x7f220c076210 msgr2=0x7f220c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.741+0000 7f22142a7640 1 -- 172.21.15.94:0/3546259299 shutdown_connections 2024-02-17T12:05:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.742+0000 7f22142a7640 1 -- 172.21.15.94:0/3546259299 wait complete. 2024-02-17T12:05:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.742+0000 7f22142a7640 1 Processor -- start 2024-02-17T12:05:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.743+0000 7f22142a7640 1 -- start start 2024-02-17T12:05:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.743+0000 7f22142a7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f220c0ff250 0x7f220c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.743+0000 7f22142a7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f220c0ffc10 0x7f220c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.743+0000 7f22142a7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f220c107260 0x7f220c19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.743+0000 7f22142a7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c19c470 con 0x7f220c0ffc10 2024-02-17T12:05:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.743+0000 7f22142a7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c19c5b0 con 0x7f220c107260 2024-02-17T12:05:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.743+0000 7f22142a7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c19c770 con 0x7f220c0ff250 2024-02-17T12:05:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.744+0000 7f2212844640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f220c107260 0x7f220c19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.744+0000 7f2212844640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f220c107260 0x7f220c19be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:43570/0 (socket says 172.21.15.94:43570) 2024-02-17T12:05:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.744+0000 7f2212844640 1 -- 172.21.15.94:0/3886754972 learned_addr learned my addr 172.21.15.94:0/3886754972 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.744+0000 7f2212043640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f220c0ff250 0x7f220c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.744+0000 7f2212844640 1 -- 172.21.15.94:0/3886754972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f220c0ff250 msgr2=0x7f220c1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.744+0000 7f2212844640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f220c0ff250 0x7f220c1975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.744+0000 7f2211842640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f220c0ffc10 0x7f220c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.744+0000 7f2212844640 1 -- 172.21.15.94:0/3886754972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f220c0ffc10 msgr2=0x7f220c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.744+0000 7f2212844640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f220c0ffc10 0x7f220c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.744+0000 7f2212844640 1 -- 172.21.15.94:0/3886754972 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2204009900 con 0x7f220c107260 2024-02-17T12:05:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.744+0000 7f2212043640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f220c0ff250 0x7f220c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:05:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.744+0000 7f2211842640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f220c0ffc10 0x7f220c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:05:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.745+0000 7f2212844640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f220c107260 0x7f220c19be70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2204009bf0 tx=0x7f2204034cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.745+0000 7f22037fe640 1 -- 172.21.15.94:0/3886754972 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22040328c0 con 0x7f220c107260 2024-02-17T12:05:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.745+0000 7f22037fe640 1 -- 172.21.15.94:0/3886754972 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2204003e50 con 0x7f220c107260 2024-02-17T12:05:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.745+0000 7f22142a7640 1 -- 172.21.15.94:0/3886754972 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220c19c9f0 con 0x7f220c107260 2024-02-17T12:05:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.745+0000 7f22037fe640 1 -- 172.21.15.94:0/3886754972 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22040047a0 con 0x7f220c107260 2024-02-17T12:05:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.745+0000 7f22142a7640 1 -- 172.21.15.94:0/3886754972 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220c100dd0 con 0x7f220c107260 2024-02-17T12:05:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.748+0000 7f22037fe640 1 -- 172.21.15.94:0/3886754972 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f220400a040 con 0x7f220c107260 2024-02-17T12:05:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.748+0000 7f2216e7f740 1 Processor -- start 2024-02-17T12:05:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.749+0000 7f2216e7f740 1 -- start start 2024-02-17T12:05:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.750+0000 7f2216e7f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543ead0 0x56049543d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.750+0000 7f2216e7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x56049543e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.750+0000 7f2216e7f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x5604954496d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.750+0000 7f2216e7f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604953ac9f0 con 0x56049543ead0 2024-02-17T12:05:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.750+0000 7f2216e7f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604953a9f20 con 0x5604954472e0 2024-02-17T12:05:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.750+0000 7f2216e7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604953a5700 con 0x56049543dd60 2024-02-17T12:05:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.750+0000 7f2212043640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x56049543e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.750+0000 7f2212043640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x56049543e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:50258/0 (socket says 172.21.15.94:50258) 2024-02-17T12:05:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.750+0000 7f2212043640 1 -- 172.21.15.94:0/3303643200 learned_addr learned my addr 172.21.15.94:0/3303643200 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.751+0000 7f2211842640 1 --2- 172.21.15.94:0/3303643200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x5604954496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.751+0000 7f2211842640 1 --2- 172.21.15.94:0/3303643200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543ead0 0x56049543d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.751+0000 7f2212043640 1 -- 172.21.15.94:0/3303643200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 msgr2=0x5604954496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.751+0000 7f2212043640 1 --2- 172.21.15.94:0/3303643200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x5604954496d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.751+0000 7f2212043640 1 -- 172.21.15.94:0/3303643200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543ead0 msgr2=0x56049543d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.751+0000 7f2212043640 1 --2- 172.21.15.94:0/3303643200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543ead0 0x56049543d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.751+0000 7f2212043640 1 -- 172.21.15.94:0/3303643200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56049533b7b0 con 0x56049543dd60 2024-02-17T12:05:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.751+0000 7f2212043640 1 --2- 172.21.15.94:0/3303643200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x56049543e1e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f220800bd40 tx=0x7f220800f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3706f668781b6f73 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.752+0000 7f2200bfe640 1 -- 172.21.15.94:0/3303643200 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220801a660 con 0x56049543dd60 2024-02-17T12:05:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.752+0000 7f2200bfe640 1 -- 172.21.15.94:0/3303643200 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f220800be30 con 0x56049543dd60 2024-02-17T12:05:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.752+0000 7f2200bfe640 1 -- 172.21.15.94:0/3303643200 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208011bb0 con 0x56049543dd60 2024-02-17T12:05:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.752+0000 7f22037fe640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f21f80a67e0 0x7f21f80a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.752+0000 7f2211842640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f21f80a67e0 0x7f21f80a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.752+0000 7f22037fe640 1 -- 172.21.15.94:0/3886754972 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f22040f0db0 con 0x7f220c107260 2024-02-17T12:05:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.753+0000 7f2211842640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f21f80a67e0 0x7f21f80a8ca0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009460 tx=0x7f21fc005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.753+0000 7f2216e7f740 1 -- 172.21.15.94:0/3303643200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 msgr2=0x56049543e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.753+0000 7f2216e7f740 1 --2- 172.21.15.94:0/3303643200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x56049543e1e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f220800bd40 tx=0x7f220800f730 comp rx=0 tx=0).stop 2024-02-17T12:05:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.753+0000 7f2216e7f740 1 -- 172.21.15.94:0/3303643200 shutdown_connections 2024-02-17T12:05:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.753+0000 7f2216e7f740 1 --2- 172.21.15.94:0/3303643200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x5604954496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.753+0000 7f2216e7f740 1 --2- 172.21.15.94:0/3303643200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x56049543e1e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.753+0000 7f2216e7f740 1 --2- 172.21.15.94:0/3303643200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543ead0 0x56049543d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.753+0000 7f2216e7f740 1 -- 172.21.15.94:0/3303643200 >> 172.21.15.94:0/3303643200 conn(0x560495405550 msgr2=0x560495438a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.753+0000 7f2216e7f740 1 -- 172.21.15.94:0/3303643200 shutdown_connections 2024-02-17T12:05:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.753+0000 7f2216e7f740 1 -- 172.21.15.94:0/3303643200 wait complete. 2024-02-17T12:05:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.754+0000 7f2216e7f740 1 -- start start 2024-02-17T12:05:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.754+0000 7f2216e7f740 1 Processor -- start 2024-02-17T12:05:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.754+0000 7f2216e7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x560495444320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.754+0000 7f2216e7f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56049543ead0 0x560495444860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.754+0000 7f2216e7f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604954472e0 0x560495444da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.754+0000 7f2216e7f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049534abf0 con 0x5604954472e0 2024-02-17T12:05:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.754+0000 7f2216e7f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560495079250 con 0x56049543ead0 2024-02-17T12:05:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.754+0000 7f2216e7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56049532b200 con 0x56049543dd60 2024-02-17T12:05:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.754+0000 7f2216e7f740 10 client.0 _ll_register_callbacks cb 0x560495399d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.754+0000 7f2212043640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x560495444320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.754+0000 7f2212043640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x560495444320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:50274/0 (socket says 172.21.15.94:50274) 2024-02-17T12:05:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.754+0000 7f2212043640 1 -- 172.21.15.94:0/471052183 learned_addr learned my addr 172.21.15.94:0/471052183 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.754+0000 7f2211842640 1 --2- 172.21.15.94:0/471052183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56049543ead0 0x560495444860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.755+0000 7f2212844640 1 --2- 172.21.15.94:0/471052183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604954472e0 0x560495444da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.756+0000 7f2212043640 1 -- 172.21.15.94:0/471052183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56049543ead0 msgr2=0x560495444860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.756+0000 7f2212043640 1 --2- 172.21.15.94:0/471052183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56049543ead0 0x560495444860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.756+0000 7f2212043640 1 -- 172.21.15.94:0/471052183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604954472e0 msgr2=0x560495444da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.756+0000 7f2212043640 1 --2- 172.21.15.94:0/471052183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604954472e0 0x560495444da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.756+0000 7f2212043640 1 -- 172.21.15.94:0/471052183 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220800b9a0 con 0x56049543dd60 2024-02-17T12:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.756+0000 7f2212043640 1 --2- 172.21.15.94:0/471052183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x560495444320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2208011730 tx=0x7f2208011b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a4cd679ebf7d07d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.756+0000 7f21f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.756+0000 7f21f5ffb640 1 -- 172.21.15.94:0/471052183 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208005230 con 0x56049543dd60 2024-02-17T12:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.756+0000 7f21f5ffb640 1 -- 172.21.15.94:0/471052183 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f2208005390 con 0x56049543dd60 2024-02-17T12:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.756+0000 7f2216e7f740 1 -- 172.21.15.94:0/471052183 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604950a5600 con 0x56049543dd60 2024-02-17T12:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.757+0000 7f21f5ffb640 1 -- 172.21.15.94:0/471052183 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2208005550 con 0x56049543dd60 2024-02-17T12:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.757+0000 7f2216e7f740 10 client.26923 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.757+0000 7f2216e7f740 1 -- 172.21.15.94:0/471052183 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5604953a3d80 con 0x56049543dd60 2024-02-17T12:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.757+0000 7f21f5ffb640 1 -- 172.21.15.94:0/471052183 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f220801f040 con 0x56049543dd60 2024-02-17T12:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.757+0000 7f21f5ffb640 1 -- 172.21.15.94:0/471052183 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f220801a580 con 0x56049543dd60 2024-02-17T12:05:28.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.757+0000 7f2216e7f740 10 client.26923 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.757+0000 7f2216e7f740 10 client.26923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.757+0000 7f2216e7f740 20 client.26923 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.757+0000 7f2216e7f740 1 --2- 172.21.15.94:0/471052183 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5604954574f0 0x560495459990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.757+0000 7f2216e7f740 4 client.26923 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.757+0000 7f2216e7f740 1 -- 172.21.15.94:0/471052183 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604950a5600 con 0x5604954574f0 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.757+0000 7f2211842640 1 --2- 172.21.15.94:0/471052183 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5604954574f0 0x560495459990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.758+0000 7f21f5ffb640 10 client.26923 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.758+0000 7f2211842640 1 --2- 172.21.15.94:0/471052183 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5604954574f0 0x560495459990 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53646a067d495584 server_cookie=6bc4b7bd50e48a6b in_seq=0 out_seq=0 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f21f5ffb640 1 -- 172.21.15.94:0/471052183 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5604950a5600 con 0x5604954574f0 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f21f5ffb640 10 client.26923 handle_command_reply: tid=0 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f2216e7f740 1 client.26923 shutdown 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f2216e7f740 1 -- 172.21.15.94:0/471052183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 msgr2=0x560495444320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f2216e7f740 1 --2- 172.21.15.94:0/471052183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x560495444320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2208011730 tx=0x7f2208011b60 comp rx=0 tx=0).stop 2024-02-17T12:05:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f2216e7f740 1 -- 172.21.15.94:0/471052183 shutdown_connections 2024-02-17T12:05:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f2216e7f740 1 --2- 172.21.15.94:0/471052183 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5604954574f0 0x560495459990 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f2216e7f740 1 --2- 172.21.15.94:0/471052183 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5604954472e0 0x560495444da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f2216e7f740 1 --2- 172.21.15.94:0/471052183 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56049543ead0 0x560495444860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f21f5ffb640 0 client.26923 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f2216e7f740 1 --2- 172.21.15.94:0/471052183 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x560495444320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f2216e7f740 1 -- 172.21.15.94:0/471052183 >> 172.21.15.94:0/471052183 conn(0x560495405550 msgr2=0x56049543a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f2216e7f740 1 -- 172.21.15.94:0/471052183 shutdown_connections 2024-02-17T12:05:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f2216e7f740 1 -- 172.21.15.94:0/471052183 wait complete. 2024-02-17T12:05:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.759+0000 7f2216e7f740 20 client.26923 trim_cache size 0 max 16384 2024-02-17T12:05:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.782+0000 7f2216e7f740 1 Processor -- start 2024-02-17T12:05:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.783+0000 7f2216e7f740 1 -- start start 2024-02-17T12:05:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.783+0000 7f2216e7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x560495446a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.783+0000 7f2216e7f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543ead0 0x560495442bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.783+0000 7f2216e7f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x560495443130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.783+0000 7f2216e7f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560495434380 con 0x56049543ead0 2024-02-17T12:05:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.783+0000 7f2216e7f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560495454190 con 0x5604954472e0 2024-02-17T12:05:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.783+0000 7f2216e7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560495443670 con 0x56049543dd60 2024-02-17T12:05:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.783+0000 7f2212844640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x560495443130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.783+0000 7f2212844640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x560495443130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:43598/0 (socket says 172.21.15.94:43598) 2024-02-17T12:05:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.783+0000 7f2212844640 1 -- 172.21.15.94:0/1515996453 learned_addr learned my addr 172.21.15.94:0/1515996453 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.783+0000 7f2212043640 1 --2- 172.21.15.94:0/1515996453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x560495446a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.784+0000 7f2212844640 1 -- 172.21.15.94:0/1515996453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 msgr2=0x560495446a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.784+0000 7f2212844640 1 --2- 172.21.15.94:0/1515996453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x560495446a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.784+0000 7f2212844640 1 -- 172.21.15.94:0/1515996453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543ead0 msgr2=0x560495442bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.784+0000 7f2212844640 1 --2- 172.21.15.94:0/1515996453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543ead0 0x560495442bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.784+0000 7f2212844640 1 -- 172.21.15.94:0/1515996453 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604953407b0 con 0x5604954472e0 2024-02-17T12:05:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.784+0000 7f2212844640 1 --2- 172.21.15.94:0/1515996453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x560495443130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2204006010 tx=0x7f22040dd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e277e23ccf73294 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.784+0000 7f21f7fff640 1 -- 172.21.15.94:0/1515996453 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22040c1990 con 0x5604954472e0 2024-02-17T12:05:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.784+0000 7f21f7fff640 1 -- 172.21.15.94:0/1515996453 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f22040c5fe0 con 0x5604954472e0 2024-02-17T12:05:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.784+0000 7f21f7fff640 1 -- 172.21.15.94:0/1515996453 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2204004140 con 0x5604954472e0 2024-02-17T12:05:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.785+0000 7f2216e7f740 1 -- 172.21.15.94:0/1515996453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 msgr2=0x560495443130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.785+0000 7f2216e7f740 1 --2- 172.21.15.94:0/1515996453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x560495443130 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2204006010 tx=0x7f22040dd040 comp rx=0 tx=0).stop 2024-02-17T12:05:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.785+0000 7f2216e7f740 1 -- 172.21.15.94:0/1515996453 shutdown_connections 2024-02-17T12:05:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.785+0000 7f2216e7f740 1 --2- 172.21.15.94:0/1515996453 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x560495443130 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.785+0000 7f2216e7f740 1 --2- 172.21.15.94:0/1515996453 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543ead0 0x560495442bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.786+0000 7f2216e7f740 1 --2- 172.21.15.94:0/1515996453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543dd60 0x560495446a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.786+0000 7f2216e7f740 1 -- 172.21.15.94:0/1515996453 >> 172.21.15.94:0/1515996453 conn(0x560495405550 msgr2=0x560495438580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.786+0000 7f2216e7f740 1 -- 172.21.15.94:0/1515996453 shutdown_connections 2024-02-17T12:05:28.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.786+0000 7f2216e7f740 1 -- 172.21.15.94:0/1515996453 wait complete. 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.786+0000 7f2216e7f740 1 -- start start 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.786+0000 7f2216e7f740 1 Processor -- start 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.787+0000 7f2216e7f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543dd60 0x56049544cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.787+0000 7f2216e7f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543ead0 0x56049544d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.787+0000 7f2216e7f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x56049544d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.787+0000 7f2216e7f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604954adad0 con 0x56049543dd60 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.787+0000 7f2216e7f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604954adc40 con 0x5604954472e0 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.787+0000 7f2216e7f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604954ade00 con 0x56049543ead0 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.787+0000 7f2216e7f740 10 client.0 _ll_register_callbacks cb 0x7f220800b500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.787+0000 7f2212043640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543dd60 0x56049544cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.787+0000 7f2212043640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543dd60 0x56049544cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59854/0 (socket says 172.21.15.94:59854) 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.787+0000 7f2212043640 1 -- 172.21.15.94:0/1582948938 learned_addr learned my addr 172.21.15.94:0/1582948938 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.787+0000 7f2211842640 1 --2- 172.21.15.94:0/1582948938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543ead0 0x56049544d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.787+0000 7f2212844640 1 --2- 172.21.15.94:0/1582948938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x56049544d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.788+0000 7f2211842640 1 -- 172.21.15.94:0/1582948938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 msgr2=0x56049544d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.788+0000 7f2211842640 1 --2- 172.21.15.94:0/1582948938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x56049544d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.788+0000 7f2211842640 1 -- 172.21.15.94:0/1582948938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543dd60 msgr2=0x56049544cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.788+0000 7f2211842640 1 --2- 172.21.15.94:0/1582948938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543dd60 0x56049544cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.788+0000 7f2211842640 1 -- 172.21.15.94:0/1582948938 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220c100dd0 con 0x56049543ead0 2024-02-17T12:05:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.788+0000 7f2211842640 1 --2- 172.21.15.94:0/1582948938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543ead0 0x56049544d1d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5604954458f0 tx=0x7f21fc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7292b0522ea2ab78 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.788+0000 7f21f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:05:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.788+0000 7f21f5ffb640 1 -- 172.21.15.94:0/1582948938 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21fc027040 con 0x56049543ead0 2024-02-17T12:05:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.788+0000 7f21f5ffb640 1 -- 172.21.15.94:0/1582948938 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f21fc029d70 con 0x56049543ead0 2024-02-17T12:05:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.789+0000 7f2216e7f740 1 -- 172.21.15.94:0/1582948938 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f220800be30 con 0x56049543ead0 2024-02-17T12:05:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.789+0000 7f21f5ffb640 1 -- 172.21.15.94:0/1582948938 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21fc003a40 con 0x56049543ead0 2024-02-17T12:05:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.789+0000 7f2216e7f740 10 client.26926 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.789+0000 7f2216e7f740 1 -- 172.21.15.94:0/1582948938 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560495366540 con 0x56049543ead0 2024-02-17T12:05:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.789+0000 7f21f5ffb640 1 -- 172.21.15.94:0/1582948938 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f220800be30 con 0x56049543ead0 2024-02-17T12:05:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.789+0000 7f21f5ffb640 1 -- 172.21.15.94:0/1582948938 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f21fc010510 con 0x56049543ead0 2024-02-17T12:05:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.789+0000 7f2216e7f740 10 client.26926 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.789+0000 7f2216e7f740 10 client.26926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.789+0000 7f2216e7f740 20 client.26926 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.789+0000 7f2216e7f740 1 --2- 172.21.15.94:0/1582948938 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5604954af640 0x5604954b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.789+0000 7f2216e7f740 4 client.26926 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:05:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.789+0000 7f2216e7f740 1 -- 172.21.15.94:0/1582948938 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5604950f61b0 con 0x5604954af640 2024-02-17T12:05:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.790+0000 7f2212043640 1 --2- 172.21.15.94:0/1582948938 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5604954af640 0x5604954b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.790+0000 7f2212043640 1 --2- 172.21.15.94:0/1582948938 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5604954af640 0x5604954b1a30 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce005fa1bd0eacfa server_cookie=808719e5494ffa93 in_seq=0 out_seq=0 2024-02-17T12:05:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.790+0000 7f21f5ffb640 10 client.26926 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.812+0000 7f21f5ffb640 1 -- 172.21.15.94:0/1582948938 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5604950f61b0 con 0x5604954af640 2024-02-17T12:05:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.812+0000 7f21f5ffb640 10 client.26926 handle_command_reply: tid=0 2024-02-17T12:05:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.813+0000 7f2216e7f740 1 client.26926 shutdown 2024-02-17T12:05:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.814+0000 7f2216e7f740 1 -- 172.21.15.94:0/1582948938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543ead0 msgr2=0x56049544d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:28.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.814+0000 7f2216e7f740 1 --2- 172.21.15.94:0/1582948938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543ead0 0x56049544d1d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5604954458f0 tx=0x7f21fc004730 comp rx=0 tx=0).stop 2024-02-17T12:05:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.814+0000 7f2216e7f740 1 -- 172.21.15.94:0/1582948938 shutdown_connections 2024-02-17T12:05:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.814+0000 7f2216e7f740 1 --2- 172.21.15.94:0/1582948938 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5604954af640 0x5604954b1a30 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.814+0000 7f2216e7f740 1 --2- 172.21.15.94:0/1582948938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5604954472e0 0x56049544d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.814+0000 7f21f5ffb640 0 client.26926 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.814+0000 7f2216e7f740 1 --2- 172.21.15.94:0/1582948938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56049543ead0 0x56049544d1d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.814+0000 7f2216e7f740 1 --2- 172.21.15.94:0/1582948938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56049543dd60 0x56049544cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.814+0000 7f2216e7f740 1 -- 172.21.15.94:0/1582948938 >> 172.21.15.94:0/1582948938 conn(0x560495405550 msgr2=0x560495442a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.814+0000 7f2216e7f740 1 -- 172.21.15.94:0/1582948938 shutdown_connections 2024-02-17T12:05:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.815+0000 7f2216e7f740 1 -- 172.21.15.94:0/1582948938 wait complete. 2024-02-17T12:05:28.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.815+0000 7f2216e7f740 20 client.26926 trim_cache size 0 max 16384 2024-02-17T12:05:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:05:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:05:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:05:28.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:05:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.815+0000 7f22142a7640 1 -- 172.21.15.94:0/3886754972 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f21f80a67e0 msgr2=0x7f21f80a8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.815+0000 7f22142a7640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f21f80a67e0 0x7f21f80a8ca0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009460 tx=0x7f21fc005f90 comp rx=0 tx=0).stop 2024-02-17T12:05:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.815+0000 7f22142a7640 1 -- 172.21.15.94:0/3886754972 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f220c107260 msgr2=0x7f220c19be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.815+0000 7f22142a7640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f220c107260 0x7f220c19be70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2204009bf0 tx=0x7f2204034cc0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.816+0000 7f22142a7640 1 -- 172.21.15.94:0/3886754972 shutdown_connections 2024-02-17T12:05:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.816+0000 7f22142a7640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f21f80a67e0 0x7f21f80a8ca0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.816+0000 7f22142a7640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f220c107260 0x7f220c19be70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.816+0000 7f22142a7640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f220c0ffc10 0x7f220c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.816+0000 7f22142a7640 1 --2- 172.21.15.94:0/3886754972 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f220c0ff250 0x7f220c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.816+0000 7f22142a7640 1 -- 172.21.15.94:0/3886754972 >> 172.21.15.94:0/3886754972 conn(0x7f220c076210 msgr2=0x7f220c0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.816+0000 7f22142a7640 1 -- 172.21.15.94:0/3886754972 shutdown_connections 2024-02-17T12:05:28.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:28.816+0000 7f22142a7640 1 -- 172.21.15.94:0/3886754972 wait complete. 2024-02-17T12:05:30.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:29 smithi094 ceph-mon[32499]: pgmap v226: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 11 MiB/s wr, 880 op/s 2024-02-17T12:05:30.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 430.0 s, 2958.0 tps, lat 33.774 ms stddev 39.865 2024-02-17T12:05:30.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:29 smithi189 ceph-mon[36588]: pgmap v226: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 11 MiB/s wr, 880 op/s 2024-02-17T12:05:30.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:29 smithi125 ceph-mon[36819]: pgmap v226: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 11 MiB/s wr, 880 op/s 2024-02-17T12:05:32.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:31 smithi094 ceph-mon[32499]: pgmap v227: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.5 MiB/s wr, 579 op/s 2024-02-17T12:05:32.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:31 smithi189 ceph-mon[36588]: pgmap v227: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.5 MiB/s wr, 579 op/s 2024-02-17T12:05:32.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:31 smithi125 ceph-mon[36819]: pgmap v227: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.5 MiB/s wr, 579 op/s 2024-02-17T12:05:34.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:33 smithi189 ceph-mon[36588]: pgmap v228: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.4 MiB/s wr, 653 op/s 2024-02-17T12:05:34.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:33 smithi125 ceph-mon[36819]: pgmap v228: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.4 MiB/s wr, 653 op/s 2024-02-17T12:05:34.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:33 smithi094 ceph-mon[32499]: pgmap v228: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.4 MiB/s wr, 653 op/s 2024-02-17T12:05:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 435.0 s, 2930.6 tps, lat 34.130 ms stddev 39.612 2024-02-17T12:05:35.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:05:35.847 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:05:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.001+0000 7fb23802c640 1 -- 172.21.15.94:0/545887003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23010fd10 msgr2=0x7fb23010da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.001+0000 7fb23802c640 1 --2- 172.21.15.94:0/545887003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23010fd10 0x7fb23010da00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb220009860 tx=0x7fb22002bb70 comp rx=0 tx=0).stop 2024-02-17T12:05:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.001+0000 7fb23802c640 1 -- 172.21.15.94:0/545887003 shutdown_connections 2024-02-17T12:05:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.001+0000 7fb23802c640 1 --2- 172.21.15.94:0/545887003 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb23011a960 0x7fb23011cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.001+0000 7fb23802c640 1 --2- 172.21.15.94:0/545887003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23010fd10 0x7fb23010da00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.001+0000 7fb23802c640 1 --2- 172.21.15.94:0/545887003 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb23010f8a0 0x7fb23010d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.001+0000 7fb23802c640 1 -- 172.21.15.94:0/545887003 >> 172.21.15.94:0/545887003 conn(0x7fb23006c270 msgr2=0x7fb23006c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.001+0000 7fb23802c640 1 -- 172.21.15.94:0/545887003 shutdown_connections 2024-02-17T12:05:36.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.001+0000 7fb23802c640 1 -- 172.21.15.94:0/545887003 wait complete. 2024-02-17T12:05:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.002+0000 7fb23802c640 1 Processor -- start 2024-02-17T12:05:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.002+0000 7fb23802c640 1 -- start start 2024-02-17T12:05:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb23802c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb23010f8a0 0x7fb2301a6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb23802c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23010fd10 0x7fb2301a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb23802c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb23011a960 0x7fb2301aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb23802c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2301ab5f0 con 0x7fb23010f8a0 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb23802c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2301ab760 con 0x7fb23011a960 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb23802c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2301ab920 con 0x7fb23010fd10 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb235dc8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb23010f8a0 0x7fb2301a6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb2365c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb23011a960 0x7fb2301aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb2365c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb23011a960 0x7fb2301aaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56344/0 (socket says 172.21.15.94:56344) 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb2365c9640 1 -- 172.21.15.94:0/2636772451 learned_addr learned my addr 172.21.15.94:0/2636772451 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb2355c7640 1 --2- 172.21.15.94:0/2636772451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23010fd10 0x7fb2301a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb2365c9640 1 -- 172.21.15.94:0/2636772451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23010fd10 msgr2=0x7fb2301a6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb2365c9640 1 --2- 172.21.15.94:0/2636772451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23010fd10 0x7fb2301a6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb2365c9640 1 -- 172.21.15.94:0/2636772451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb23010f8a0 msgr2=0x7fb2301a6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.003+0000 7fb2365c9640 1 --2- 172.21.15.94:0/2636772451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb23010f8a0 0x7fb2301a6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.004+0000 7fb2365c9640 1 -- 172.21.15.94:0/2636772451 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c009560 con 0x7fb23011a960 2024-02-17T12:05:36.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.004+0000 7fb2365c9640 1 --2- 172.21.15.94:0/2636772451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb23011a960 0x7fb2301aaff0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb22800eb40 tx=0x7fb22800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.004+0000 7fb226ffd640 1 -- 172.21.15.94:0/2636772451 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228004b20 con 0x7fb23011a960 2024-02-17T12:05:36.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.004+0000 7fb226ffd640 1 -- 172.21.15.94:0/2636772451 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb228004c80 con 0x7fb23011a960 2024-02-17T12:05:36.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.004+0000 7fb226ffd640 1 -- 172.21.15.94:0/2636772451 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2280107b0 con 0x7fb23011a960 2024-02-17T12:05:36.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.005+0000 7fb23802c640 1 -- 172.21.15.94:0/2636772451 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb220009490 con 0x7fb23011a960 2024-02-17T12:05:36.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.005+0000 7fb23802c640 1 -- 172.21.15.94:0/2636772451 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2301b32a0 con 0x7fb23011a960 2024-02-17T12:05:36.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.008+0000 7fb226ffd640 1 -- 172.21.15.94:0/2636772451 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb228005020 con 0x7fb23011a960 2024-02-17T12:05:36.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.008+0000 7fb226ffd640 1 --2- 172.21.15.94:0/2636772451 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb21c0a2380 0x7fb21c0a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.008+0000 7fb235dc8640 1 --2- 172.21.15.94:0/2636772451 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb21c0a2380 0x7fb21c0a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.008+0000 7fb226ffd640 1 -- 172.21.15.94:0/2636772451 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb228014070 con 0x7fb23011a960 2024-02-17T12:05:36.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.008+0000 7fb23abfd740 1 Processor -- start 2024-02-17T12:05:36.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.009+0000 7fb23abfd740 1 -- start start 2024-02-17T12:05:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.009+0000 7fb23abfd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbec12b0 0x5596cbec1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.009+0000 7fb23abfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec1bd0 0x5596cbeca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.009+0000 7fb23abfd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbeca7b0 0x5596cbeccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.009+0000 7fb23abfd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cbe30280 con 0x5596cbec12b0 2024-02-17T12:05:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.009+0000 7fb23abfd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cbe2d7b0 con 0x5596cbeca7b0 2024-02-17T12:05:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.009+0000 7fb23abfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cbe28f90 con 0x5596cbec1bd0 2024-02-17T12:05:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.009+0000 7fb2355c7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec1bd0 0x5596cbeca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.009+0000 7fb2355c7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec1bd0 0x5596cbeca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:41878/0 (socket says 172.21.15.94:41878) 2024-02-17T12:05:36.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.009+0000 7fb2355c7640 1 -- 172.21.15.94:0/3655388895 learned_addr learned my addr 172.21.15.94:0/3655388895 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.009+0000 7fb2365c9640 1 --2- 172.21.15.94:0/3655388895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbeca7b0 0x5596cbeccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.010+0000 7fb2355c7640 1 -- 172.21.15.94:0/3655388895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbeca7b0 msgr2=0x5596cbeccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.010+0000 7fb2355c7640 1 --2- 172.21.15.94:0/3655388895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbeca7b0 0x5596cbeccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.010+0000 7fb2355c7640 1 -- 172.21.15.94:0/3655388895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbec12b0 msgr2=0x5596cbec1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.010+0000 7fb2355c7640 1 --2- 172.21.15.94:0/3655388895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbec12b0 0x5596cbec1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.010+0000 7fb2355c7640 1 -- 172.21.15.94:0/3655388895 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596cbde7570 con 0x5596cbec1bd0 2024-02-17T12:05:36.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.010+0000 7fb235dc8640 1 --2- 172.21.15.94:0/2636772451 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb21c0a2380 0x7fb21c0a4840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009530 tx=0x7fb22c011810 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.010+0000 7fb2365c9640 1 --2- 172.21.15.94:0/3655388895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbeca7b0 0x5596cbeccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:05:36.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.010+0000 7fb2355c7640 1 --2- 172.21.15.94:0/3655388895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec1bd0 0x5596cbeca270 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb22002c180 tx=0x7fb2200323c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e73a9ec85635fd21 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.010+0000 7fb21bfff640 1 -- 172.21.15.94:0/3655388895 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220004030 con 0x5596cbec1bd0 2024-02-17T12:05:36.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.010+0000 7fb21bfff640 1 -- 172.21.15.94:0/3655388895 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb220034440 con 0x5596cbec1bd0 2024-02-17T12:05:36.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.010+0000 7fb21bfff640 1 -- 172.21.15.94:0/3655388895 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22002c8f0 con 0x5596cbec1bd0 2024-02-17T12:05:36.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.011+0000 7fb23abfd740 1 -- 172.21.15.94:0/3655388895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec1bd0 msgr2=0x5596cbeca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.011+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3655388895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec1bd0 0x5596cbeca270 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb22002c180 tx=0x7fb2200323c0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.011+0000 7fb23abfd740 1 -- 172.21.15.94:0/3655388895 shutdown_connections 2024-02-17T12:05:36.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.011+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3655388895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbeca7b0 0x5596cbeccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.011+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3655388895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec1bd0 0x5596cbeca270 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.011+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3655388895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbec12b0 0x5596cbec1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.011+0000 7fb23abfd740 1 -- 172.21.15.94:0/3655388895 >> 172.21.15.94:0/3655388895 conn(0x5596cbebb990 msgr2=0x5596cbebbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.011+0000 7fb23abfd740 1 -- 172.21.15.94:0/3655388895 shutdown_connections 2024-02-17T12:05:36.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.011+0000 7fb23abfd740 1 -- 172.21.15.94:0/3655388895 wait complete. 2024-02-17T12:05:36.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.012+0000 7fb23abfd740 1 -- start start 2024-02-17T12:05:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.012+0000 7fb23abfd740 1 Processor -- start 2024-02-17T12:05:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.012+0000 7fb23abfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 0x5596cbeca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.012+0000 7fb23abfd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbec1bd0 0x5596cbec7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.012+0000 7fb23abfd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbeca7b0 0x5596cbec7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.012+0000 7fb23abfd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cbafc430 con 0x5596cbec1bd0 2024-02-17T12:05:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.012+0000 7fb23abfd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cbab8e00 con 0x5596cbeca7b0 2024-02-17T12:05:36.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.012+0000 7fb23abfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cbdadfd0 con 0x5596cbec12b0 2024-02-17T12:05:36.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.012+0000 7fb23abfd740 10 client.0 _ll_register_callbacks cb 0x5596cbe03fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:36.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.012+0000 7fb2355c7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbec1bd0 0x5596cbec7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.012+0000 7fb2355c7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbec1bd0 0x5596cbec7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60892/0 (socket says 172.21.15.94:60892) 2024-02-17T12:05:36.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.012+0000 7fb2355c7640 1 -- 172.21.15.94:0/3098739318 learned_addr learned my addr 172.21.15.94:0/3098739318 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:36.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb2365c9640 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbeca7b0 0x5596cbec7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb235dc8640 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 0x5596cbeca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb2355c7640 1 -- 172.21.15.94:0/3098739318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 msgr2=0x5596cbeca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb2355c7640 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 0x5596cbeca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb2355c7640 1 -- 172.21.15.94:0/3098739318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbeca7b0 msgr2=0x5596cbec7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb2355c7640 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbeca7b0 0x5596cbec7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb2355c7640 1 -- 172.21.15.94:0/3098739318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23011d610 con 0x5596cbec1bd0 2024-02-17T12:05:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb2365c9640 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbeca7b0 0x5596cbec7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb2355c7640 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbec1bd0 0x5596cbec7590 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb22002bbb0 tx=0x7fb220035f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367944404cfb7752 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb2197fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb2197fa640 1 -- 172.21.15.94:0/3098739318 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220004450 con 0x5596cbec1bd0 2024-02-17T12:05:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb23abfd740 1 -- 172.21.15.94:0/3098739318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596cbb287e0 con 0x5596cbec1bd0 2024-02-17T12:05:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb2197fa640 1 -- 172.21.15.94:0/3098739318 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb22002c690 con 0x5596cbec1bd0 2024-02-17T12:05:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb2197fa640 1 -- 172.21.15.94:0/3098739318 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb22002c850 con 0x5596cbec1bd0 2024-02-17T12:05:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb23abfd740 10 client.17421 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb23abfd740 1 -- 172.21.15.94:0/3098739318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5596cbe27610 con 0x5596cbec1bd0 2024-02-17T12:05:36.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.013+0000 7fb2197fa640 1 -- 172.21.15.94:0/3098739318 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb22005c080 con 0x5596cbec1bd0 2024-02-17T12:05:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.014+0000 7fb2197fa640 1 -- 172.21.15.94:0/3098739318 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb220032600 con 0x5596cbec1bd0 2024-02-17T12:05:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.014+0000 7fb23abfd740 10 client.17421 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.014+0000 7fb23abfd740 10 client.17421 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.014+0000 7fb23abfd740 20 client.17421 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.014+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5596cbeda520 0x5596cbedc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.014+0000 7fb23abfd740 4 client.17421 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.014+0000 7fb23abfd740 1 -- 172.21.15.94:0/3098739318 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596cbb65720 con 0x5596cbeda520 2024-02-17T12:05:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.015+0000 7fb235dc8640 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5596cbeda520 0x5596cbedc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.016+0000 7fb235dc8640 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5596cbeda520 0x5596cbedc940 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99110e16f9264033 server_cookie=8e3e6e359073996a in_seq=0 out_seq=0 2024-02-17T12:05:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.016+0000 7fb2197fa640 10 client.17421 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.016+0000 7fb2197fa640 1 -- 172.21.15.94:0/3098739318 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5596cbb65720 con 0x5596cbeda520 2024-02-17T12:05:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.016+0000 7fb2197fa640 10 client.17421 handle_command_reply: tid=0 2024-02-17T12:05:36.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.016+0000 7fb23abfd740 1 client.17421 shutdown 2024-02-17T12:05:36.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.017+0000 7fb23abfd740 1 -- 172.21.15.94:0/3098739318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbec1bd0 msgr2=0x5596cbec7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.017+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbec1bd0 0x5596cbec7590 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb22002bbb0 tx=0x7fb220035f60 comp rx=0 tx=0).stop 2024-02-17T12:05:36.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.018+0000 7fb23abfd740 1 -- 172.21.15.94:0/3098739318 shutdown_connections 2024-02-17T12:05:36.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.018+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5596cbeda520 0x5596cbedc940 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.018+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbeca7b0 0x5596cbec7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.018+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbec1bd0 0x5596cbec7590 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.018+0000 7fb2197fa640 0 client.17421 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.018+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3098739318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 0x5596cbeca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.018+0000 7fb23abfd740 1 -- 172.21.15.94:0/3098739318 >> 172.21.15.94:0/3098739318 conn(0x5596cbebb990 msgr2=0x5596cbebd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.018+0000 7fb23abfd740 1 -- 172.21.15.94:0/3098739318 shutdown_connections 2024-02-17T12:05:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.018+0000 7fb23abfd740 1 -- 172.21.15.94:0/3098739318 wait complete. 2024-02-17T12:05:36.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.018+0000 7fb23abfd740 20 client.17421 trim_cache size 0 max 16384 2024-02-17T12:05:36.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.038+0000 7fb23abfd740 1 Processor -- start 2024-02-17T12:05:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.038+0000 7fb23abfd740 1 -- start start 2024-02-17T12:05:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.038+0000 7fb23abfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 0x5596cbec9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.038+0000 7fb23abfd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbec1bd0 0x5596cbeca200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb23abfd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbeca7b0 0x5596cbec5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb23abfd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cbebb1d0 con 0x5596cbeca7b0 2024-02-17T12:05:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb23abfd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cbebc9e0 con 0x5596cbec1bd0 2024-02-17T12:05:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb23abfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cbebc790 con 0x5596cbec12b0 2024-02-17T12:05:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb2365c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbeca7b0 0x5596cbec5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb2355c7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbec1bd0 0x5596cbeca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb235dc8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 0x5596cbec9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb2365c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbeca7b0 0x5596cbec5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60896/0 (socket says 172.21.15.94:60896) 2024-02-17T12:05:36.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb2365c9640 1 -- 172.21.15.94:0/1617884626 learned_addr learned my addr 172.21.15.94:0/1617884626 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb2355c7640 1 --2- 172.21.15.94:0/1617884626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbec1bd0 0x5596cbeca200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56370/0 (socket says 172.21.15.94:56370) 2024-02-17T12:05:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb235dc8640 1 -- 172.21.15.94:0/1617884626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbec1bd0 msgr2=0x5596cbeca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb235dc8640 1 --2- 172.21.15.94:0/1617884626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbec1bd0 0x5596cbeca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb235dc8640 1 -- 172.21.15.94:0/1617884626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbeca7b0 msgr2=0x5596cbec5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb235dc8640 1 --2- 172.21.15.94:0/1617884626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbeca7b0 0x5596cbec5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb235dc8640 1 -- 172.21.15.94:0/1617884626 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596cbdc3ac0 con 0x5596cbec12b0 2024-02-17T12:05:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb235dc8640 1 --2- 172.21.15.94:0/1617884626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 0x5596cbec9cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5596cbedd220 tx=0x7fb22c011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2508c6bab23b8ab2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb21b7fe640 1 -- 172.21.15.94:0/1617884626 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c023040 con 0x5596cbec12b0 2024-02-17T12:05:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb21b7fe640 1 -- 172.21.15.94:0/1617884626 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb22c026820 con 0x5596cbec12b0 2024-02-17T12:05:36.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.039+0000 7fb21b7fe640 1 -- 172.21.15.94:0/1617884626 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c0047a0 con 0x5596cbec12b0 2024-02-17T12:05:36.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.040+0000 7fb23abfd740 1 -- 172.21.15.94:0/1617884626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 msgr2=0x5596cbec9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.040+0000 7fb23abfd740 1 --2- 172.21.15.94:0/1617884626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 0x5596cbec9cc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5596cbedd220 tx=0x7fb22c011f90 comp rx=0 tx=0).stop 2024-02-17T12:05:36.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.041+0000 7fb23abfd740 1 -- 172.21.15.94:0/1617884626 shutdown_connections 2024-02-17T12:05:36.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.041+0000 7fb23abfd740 1 --2- 172.21.15.94:0/1617884626 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbeca7b0 0x5596cbec5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.041+0000 7fb23abfd740 1 --2- 172.21.15.94:0/1617884626 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbec1bd0 0x5596cbeca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.041+0000 7fb23abfd740 1 --2- 172.21.15.94:0/1617884626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 0x5596cbec9cc0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.041+0000 7fb23abfd740 1 -- 172.21.15.94:0/1617884626 >> 172.21.15.94:0/1617884626 conn(0x5596cbebb990 msgr2=0x5596cbec2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.041+0000 7fb23abfd740 1 -- 172.21.15.94:0/1617884626 shutdown_connections 2024-02-17T12:05:36.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.041+0000 7fb23abfd740 1 -- 172.21.15.94:0/1617884626 wait complete. 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.042+0000 7fb23abfd740 1 -- start start 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.042+0000 7fb23abfd740 1 Processor -- start 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.042+0000 7fb23abfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 0x5596cbed1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.042+0000 7fb23abfd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbec1bd0 0x5596cbecdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.042+0000 7fb23abfd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbeca7b0 0x5596cbece310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.042+0000 7fb23abfd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cbecea10 con 0x5596cbeca7b0 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.042+0000 7fb23abfd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cbeceb80 con 0x5596cbec1bd0 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.042+0000 7fb23abfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596cbf30a20 con 0x5596cbec12b0 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2355c7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbec1bd0 0x5596cbecdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2355c7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbec1bd0 0x5596cbecdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56386/0 (socket says 172.21.15.94:56386) 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2355c7640 1 -- 172.21.15.94:0/3943434681 learned_addr learned my addr 172.21.15.94:0/3943434681 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2365c9640 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbeca7b0 0x5596cbece310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb23abfd740 10 client.0 _ll_register_callbacks cb 0x7fb22c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:36.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb235dc8640 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 0x5596cbed1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2365c9640 1 -- 172.21.15.94:0/3943434681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 msgr2=0x5596cbed1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2365c9640 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 0x5596cbed1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2365c9640 1 -- 172.21.15.94:0/3943434681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbec1bd0 msgr2=0x5596cbecdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2365c9640 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbec1bd0 0x5596cbecdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2365c9640 1 -- 172.21.15.94:0/3943434681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22c00f4e0 con 0x5596cbeca7b0 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2355c7640 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbec1bd0 0x5596cbecdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2365c9640 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbeca7b0 0x5596cbece310 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb228097480 tx=0x7fb22809b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb4bd8e53418679 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2197fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2197fa640 1 -- 172.21.15.94:0/3943434681 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2280a7070 con 0x5596cbeca7b0 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2197fa640 1 -- 172.21.15.94:0/3943434681 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb2280b9030 con 0x5596cbeca7b0 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb23abfd740 1 -- 172.21.15.94:0/3943434681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb220034440 con 0x5596cbeca7b0 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2197fa640 1 -- 172.21.15.94:0/3943434681 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2280a37c0 con 0x5596cbeca7b0 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb23abfd740 10 client.17427 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb23abfd740 1 -- 172.21.15.94:0/3943434681 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5596cbde96a0 con 0x5596cbeca7b0 2024-02-17T12:05:36.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.043+0000 7fb2197fa640 1 -- 172.21.15.94:0/3943434681 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb220034440 con 0x5596cbeca7b0 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.044+0000 7fb2197fa640 1 -- 172.21.15.94:0/3943434681 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb228095b90 con 0x5596cbeca7b0 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.044+0000 7fb23abfd740 10 client.17427 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.044+0000 7fb23abfd740 10 client.17427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.044+0000 7fb23abfd740 20 client.17427 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.044+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5596cbf32290 0x5596cbf346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.044+0000 7fb23abfd740 4 client.17427 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.044+0000 7fb23abfd740 1 -- 172.21.15.94:0/3943434681 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5596cbbb4890 con 0x5596cbf32290 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.044+0000 7fb235dc8640 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5596cbf32290 0x5596cbf346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.044+0000 7fb2197fa640 10 client.17427 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.044+0000 7fb235dc8640 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5596cbf32290 0x5596cbf346b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a98f6fa16998a44d server_cookie=5b33aa504d17d0eb in_seq=0 out_seq=0 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.045+0000 7fb2197fa640 1 -- 172.21.15.94:0/3943434681 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5596cbbb4890 con 0x5596cbf32290 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.045+0000 7fb2197fa640 10 client.17427 handle_command_reply: tid=0 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.045+0000 7fb23abfd740 1 client.17427 shutdown 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.046+0000 7fb23abfd740 1 -- 172.21.15.94:0/3943434681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbeca7b0 msgr2=0x5596cbece310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.046+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbeca7b0 0x5596cbece310 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb228097480 tx=0x7fb22809b7e0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.046+0000 7fb23abfd740 1 -- 172.21.15.94:0/3943434681 shutdown_connections 2024-02-17T12:05:36.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.046+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5596cbf32290 0x5596cbf346b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.046+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5596cbeca7b0 0x5596cbece310 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.046+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5596cbec1bd0 0x5596cbecdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.046+0000 7fb23abfd740 1 --2- 172.21.15.94:0/3943434681 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5596cbec12b0 0x5596cbed1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.046+0000 7fb23abfd740 1 -- 172.21.15.94:0/3943434681 >> 172.21.15.94:0/3943434681 conn(0x5596cbebb990 msgr2=0x5596cbec5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.046+0000 7fb2197fa640 0 client.17427 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:36.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.046+0000 7fb23abfd740 1 -- 172.21.15.94:0/3943434681 shutdown_connections 2024-02-17T12:05:36.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23abfd740 1 -- 172.21.15.94:0/3943434681 wait complete. 2024-02-17T12:05:36.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23abfd740 20 client.17427 trim_cache size 0 max 16384 2024-02-17T12:05:36.052 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:05:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:05:36.053 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:05:36.053 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:05:36.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23802c640 1 -- 172.21.15.94:0/2636772451 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb21c0a2380 msgr2=0x7fb21c0a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23802c640 1 --2- 172.21.15.94:0/2636772451 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb21c0a2380 0x7fb21c0a4840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009530 tx=0x7fb22c011810 comp rx=0 tx=0).stop 2024-02-17T12:05:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23802c640 1 -- 172.21.15.94:0/2636772451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb23011a960 msgr2=0x7fb2301aaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23802c640 1 --2- 172.21.15.94:0/2636772451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb23011a960 0x7fb2301aaff0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb22800eb40 tx=0x7fb22800c2d0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23802c640 1 -- 172.21.15.94:0/2636772451 shutdown_connections 2024-02-17T12:05:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23802c640 1 --2- 172.21.15.94:0/2636772451 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb21c0a2380 0x7fb21c0a4840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23802c640 1 --2- 172.21.15.94:0/2636772451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb23011a960 0x7fb2301aaff0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23802c640 1 --2- 172.21.15.94:0/2636772451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb23010fd10 0x7fb2301a6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23802c640 1 --2- 172.21.15.94:0/2636772451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb23010f8a0 0x7fb2301a6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23802c640 1 -- 172.21.15.94:0/2636772451 >> 172.21.15.94:0/2636772451 conn(0x7fb23006c270 msgr2=0x7fb2301193d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23802c640 1 -- 172.21.15.94:0/2636772451 shutdown_connections 2024-02-17T12:05:36.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.047+0000 7fb23802c640 1 -- 172.21.15.94:0/2636772451 wait complete. 2024-02-17T12:05:36.062 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:05:36.062 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:05:36.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:35 smithi094 ceph-mon[32499]: pgmap v229: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 926 op/s 2024-02-17T12:05:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.215+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/1730079103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fc0069d90 msgr2=0x7f4fc006a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.215+0000 7f4fc6a8d640 1 --2- 172.21.15.94:0/1730079103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fc0069d90 0x7f4fc006a1f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009890 tx=0x7f4fb002bb70 comp rx=0 tx=0).stop 2024-02-17T12:05:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.215+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/1730079103 shutdown_connections 2024-02-17T12:05:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.215+0000 7f4fc6a8d640 1 --2- 172.21.15.94:0/1730079103 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fc0103c60 0x7f4fc01040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.215+0000 7f4fc6a8d640 1 --2- 172.21.15.94:0/1730079103 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fc0069d90 0x7f4fc006a1f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.215+0000 7f4fc6a8d640 1 --2- 172.21.15.94:0/1730079103 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fc01060f0 0x7f4fc0069850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.215+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/1730079103 >> 172.21.15.94:0/1730079103 conn(0x7f4fc0077150 msgr2=0x7f4fc0077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.215+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/1730079103 shutdown_connections 2024-02-17T12:05:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.215+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/1730079103 wait complete. 2024-02-17T12:05:36.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.215+0000 7f4fc6a8d640 1 Processor -- start 2024-02-17T12:05:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.215+0000 7f4fc6a8d640 1 -- start start 2024-02-17T12:05:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.216+0000 7f4fc6a8d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fc0069d90 0x7f4fc01a3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.216+0000 7f4fc6a8d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fc0103c60 0x7f4fc01a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.216+0000 7f4fbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fc0103c60 0x7f4fc01a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.216+0000 7f4fbffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fc0103c60 0x7f4fc01a3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60936/0 (socket says 172.21.15.94:60936) 2024-02-17T12:05:36.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.216+0000 7f4fc4829640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fc0069d90 0x7f4fc01a3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.216+0000 7f4fc6a8d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fc01060f0 0x7f4fc019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.216+0000 7f4fc6a8d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc019e150 con 0x7f4fc0103c60 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.216+0000 7f4fc6a8d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc019e2c0 con 0x7f4fc01060f0 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.216+0000 7f4fc6a8d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc019e480 con 0x7f4fc0069d90 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.216+0000 7f4fbffff640 1 -- 172.21.15.94:0/28411952 learned_addr learned my addr 172.21.15.94:0/28411952 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.217+0000 7f4fc502a640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fc01060f0 0x7f4fc019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.217+0000 7f4fc4829640 1 -- 172.21.15.94:0/28411952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fc01060f0 msgr2=0x7f4fc019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.217+0000 7f4fc4829640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fc01060f0 0x7f4fc019db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.217+0000 7f4fc4829640 1 -- 172.21.15.94:0/28411952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fc0103c60 msgr2=0x7f4fc01a3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.217+0000 7f4fc4829640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fc0103c60 0x7f4fc01a3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.217+0000 7f4fc4829640 1 -- 172.21.15.94:0/28411952 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb0009490 con 0x7f4fc0069d90 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.217+0000 7f4fc502a640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fc01060f0 0x7f4fc019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.217+0000 7f4fc4829640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fc0069d90 0x7f4fc01a3950 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800e9f0 tx=0x7f4fa800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.217+0000 7f4fbdffb640 1 -- 172.21.15.94:0/28411952 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa800ccc0 con 0x7f4fc0069d90 2024-02-17T12:05:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.217+0000 7f4fbdffb640 1 -- 172.21.15.94:0/28411952 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fa800ce20 con 0x7f4fc0069d90 2024-02-17T12:05:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.217+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/28411952 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc019e760 con 0x7f4fc0069d90 2024-02-17T12:05:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.217+0000 7f4fbdffb640 1 -- 172.21.15.94:0/28411952 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8018920 con 0x7f4fc0069d90 2024-02-17T12:05:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.218+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/28411952 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc01aa9f0 con 0x7f4fc0069d90 2024-02-17T12:05:36.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.219+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/28411952 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc0100660 con 0x7f4fc0069d90 2024-02-17T12:05:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.220+0000 7f4fbdffb640 1 -- 172.21.15.94:0/28411952 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4fa8018a80 con 0x7f4fc0069d90 2024-02-17T12:05:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.220+0000 7f4fbdffb640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4fac0a2230 0x7f4fac0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.220+0000 7f4fbdffb640 1 -- 172.21.15.94:0/28411952 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f4fa8014070 con 0x7f4fc0069d90 2024-02-17T12:05:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.220+0000 7f4fbffff640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4fac0a2230 0x7f4fac0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.221+0000 7f4fbffff640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4fac0a2230 0x7f4fac0a46f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0005e00 tx=0x7f4fb0032360 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.224+0000 7f4fbdffb640 1 -- 172.21.15.94:0/28411952 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4fa8097090 con 0x7f4fc0069d90 2024-02-17T12:05:36.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:35 smithi189 ceph-mon[36588]: pgmap v229: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 926 op/s 2024-02-17T12:05:36.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:35 smithi125 ceph-mon[36819]: pgmap v229: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 11 MiB/s wr, 926 op/s 2024-02-17T12:05:36.568 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.568+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/28411952 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fc019eff0 con 0x7f4fc0069d90 2024-02-17T12:05:36.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.573+0000 7f4fbdffb640 1 -- 172.21.15.94:0/28411952 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7f4fa809a750 con 0x7f4fc0069d90 2024-02-17T12:05:36.575 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:05:36.575 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:05:36.575 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:05:36.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.581+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/28411952 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4fac0a2230 msgr2=0x7f4fac0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.581+0000 7f4fc6a8d640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4fac0a2230 0x7f4fac0a46f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0005e00 tx=0x7f4fb0032360 comp rx=0 tx=0).stop 2024-02-17T12:05:36.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.581+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/28411952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fc0069d90 msgr2=0x7f4fc01a3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.581+0000 7f4fc6a8d640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fc0069d90 0x7f4fc01a3950 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4fa800e9f0 tx=0x7f4fa800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.581+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/28411952 shutdown_connections 2024-02-17T12:05:36.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.581+0000 7f4fc6a8d640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4fac0a2230 0x7f4fac0a46f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.581+0000 7f4fc6a8d640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fc01060f0 0x7f4fc019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.581+0000 7f4fc6a8d640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fc0103c60 0x7f4fc01a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.581+0000 7f4fc6a8d640 1 --2- 172.21.15.94:0/28411952 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fc0069d90 0x7f4fc01a3950 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.581+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/28411952 >> 172.21.15.94:0/28411952 conn(0x7f4fc0077150 msgr2=0x7f4fc0100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.582+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/28411952 shutdown_connections 2024-02-17T12:05:36.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.582+0000 7f4fc6a8d640 1 -- 172.21.15.94:0/28411952 wait complete. 2024-02-17T12:05:36.604 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:05:36.604 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:05:36.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.750+0000 7f1d45b7a640 1 -- 172.21.15.94:0/1586699570 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d4010d730 msgr2=0x7f1d4010db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.750+0000 7f1d45b7a640 1 --2- 172.21.15.94:0/1586699570 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d4010d730 0x7f1d4010db10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009900 tx=0x7f1d3802bc30 comp rx=0 tx=0).stop 2024-02-17T12:05:36.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.751+0000 7f1d45b7a640 1 -- 172.21.15.94:0/1586699570 shutdown_connections 2024-02-17T12:05:36.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.751+0000 7f1d45b7a640 1 --2- 172.21.15.94:0/1586699570 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d401080e0 0x7f1d401145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.751+0000 7f1d45b7a640 1 --2- 172.21.15.94:0/1586699570 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d40107720 0x7f1d40107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.751+0000 7f1d45b7a640 1 --2- 172.21.15.94:0/1586699570 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d4010d730 0x7f1d4010db10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.751+0000 7f1d45b7a640 1 -- 172.21.15.94:0/1586699570 >> 172.21.15.94:0/1586699570 conn(0x7f1d400756c0 msgr2=0x7f1d40077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.751+0000 7f1d45b7a640 1 -- 172.21.15.94:0/1586699570 shutdown_connections 2024-02-17T12:05:36.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.751+0000 7f1d45b7a640 1 -- 172.21.15.94:0/1586699570 wait complete. 2024-02-17T12:05:36.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.752+0000 7f1d45b7a640 1 Processor -- start 2024-02-17T12:05:36.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.752+0000 7f1d45b7a640 1 -- start start 2024-02-17T12:05:36.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.752+0000 7f1d45b7a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d40107720 0x7f1d40081600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.752+0000 7f1d45b7a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d401080e0 0x7f1d40081b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.752+0000 7f1d45b7a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4010d730 0x7f1d4007a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.753+0000 7f1d45b7a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4007af70 con 0x7f1d4010d730 2024-02-17T12:05:36.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.753+0000 7f1d45b7a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4007b0e0 con 0x7f1d401080e0 2024-02-17T12:05:36.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.753+0000 7f1d45b7a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4007b2a0 con 0x7f1d40107720 2024-02-17T12:05:36.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.753+0000 7f1d3effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d401080e0 0x7f1d40081b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.753+0000 7f1d3f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d40107720 0x7f1d40081600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.753+0000 7f1d3f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d40107720 0x7f1d40081600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:41936/0 (socket says 172.21.15.94:41936) 2024-02-17T12:05:36.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.753+0000 7f1d3effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d401080e0 0x7f1d40081b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56410/0 (socket says 172.21.15.94:56410) 2024-02-17T12:05:36.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.753+0000 7f1d3f7fe640 1 -- 172.21.15.94:0/958727627 learned_addr learned my addr 172.21.15.94:0/958727627 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:36.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.753+0000 7f1d3effd640 1 -- 172.21.15.94:0/958727627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d40107720 msgr2=0x7f1d40081600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.753+0000 7f1d3effd640 1 --2- 172.21.15.94:0/958727627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d40107720 0x7f1d40081600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.753+0000 7f1d3effd640 1 -- 172.21.15.94:0/958727627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4010d730 msgr2=0x7f1d4007a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.753+0000 7f1d3effd640 1 --2- 172.21.15.94:0/958727627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4010d730 0x7f1d4007a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.753+0000 7f1d3effd640 1 -- 172.21.15.94:0/958727627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d30009840 con 0x7f1d401080e0 2024-02-17T12:05:36.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.754+0000 7f1d3effd640 1 --2- 172.21.15.94:0/958727627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d401080e0 0x7f1d40081b60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000ec80 tx=0x7f1d3000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.754+0000 7f1d3cff9640 1 -- 172.21.15.94:0/958727627 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30003b70 con 0x7f1d401080e0 2024-02-17T12:05:36.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.754+0000 7f1d3cff9640 1 -- 172.21.15.94:0/958727627 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d30003cd0 con 0x7f1d401080e0 2024-02-17T12:05:36.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.755+0000 7f1d45b7a640 1 -- 172.21.15.94:0/958727627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d38009560 con 0x7f1d401080e0 2024-02-17T12:05:36.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.755+0000 7f1d45b7a640 1 -- 172.21.15.94:0/958727627 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4007f3c0 con 0x7f1d401080e0 2024-02-17T12:05:36.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.756+0000 7f1d3cff9640 1 -- 172.21.15.94:0/958727627 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30018940 con 0x7f1d401080e0 2024-02-17T12:05:36.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.762+0000 7f1d4882b740 1 Processor -- start 2024-02-17T12:05:36.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.770+0000 7f1d3cff9640 1 -- 172.21.15.94:0/958727627 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1d30018aa0 con 0x7f1d401080e0 2024-02-17T12:05:36.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.770+0000 7f1d3cff9640 1 --2- 172.21.15.94:0/958727627 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1d280a22e0 0x7f1d280a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.771+0000 7f1d3cff9640 1 -- 172.21.15.94:0/958727627 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1d30014070 con 0x7f1d401080e0 2024-02-17T12:05:36.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.773+0000 7f1d4882b740 1 -- start start 2024-02-17T12:05:36.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.774+0000 7f1d4882b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2d4250 0x558b7a2d4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.774+0000 7f1d4882b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4b70 0x558b7a2dd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.774+0000 7f1d4882b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2dd750 0x558b7a2dfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.774+0000 7f1d4882b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7a28f960 con 0x558b7a2d4250 2024-02-17T12:05:36.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.774+0000 7f1d4882b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7a243310 con 0x558b7a2dd750 2024-02-17T12:05:36.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.774+0000 7f1d4882b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7a240840 con 0x558b7a2d4b70 2024-02-17T12:05:36.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.774+0000 7f1d3effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4b70 0x558b7a2dd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.774+0000 7f1d3effd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4b70 0x558b7a2dd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:41952/0 (socket says 172.21.15.94:41952) 2024-02-17T12:05:36.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.774+0000 7f1d3effd640 1 -- 172.21.15.94:0/3427251937 learned_addr learned my addr 172.21.15.94:0/3427251937 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:36.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.775+0000 7f1d3f7fe640 1 --2- 172.21.15.94:0/3427251937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2d4250 0x558b7a2d4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.775+0000 7f1d3f7fe640 1 -- 172.21.15.94:0/3427251937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4b70 msgr2=0x558b7a2dd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.775+0000 7f1d3f7fe640 1 --2- 172.21.15.94:0/3427251937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4b70 0x558b7a2dd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.775+0000 7f1d3f7fe640 1 -- 172.21.15.94:0/3427251937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2dd750 msgr2=0x558b7a2dfb40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:05:36.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.775+0000 7f1d3f7fe640 1 --2- 172.21.15.94:0/3427251937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2dd750 0x558b7a2dfb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.775+0000 7f1d3f7fe640 1 -- 172.21.15.94:0/3427251937 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b7a1fe060 con 0x558b7a2d4250 2024-02-17T12:05:36.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.776+0000 7f1d3f7fe640 1 --2- 172.21.15.94:0/3427251937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2d4250 0x558b7a2d4630 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1d38005ec0 tx=0x7f1d3802c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=651d3b62afd841ef server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.776+0000 7f1d25ffb640 1 -- 172.21.15.94:0/3427251937 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3802c950 con 0x558b7a2d4250 2024-02-17T12:05:36.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.776+0000 7f1d25ffb640 1 -- 172.21.15.94:0/3427251937 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d3802cab0 con 0x558b7a2d4250 2024-02-17T12:05:36.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.776+0000 7f1d25ffb640 1 -- 172.21.15.94:0/3427251937 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38005160 con 0x558b7a2d4250 2024-02-17T12:05:36.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.781+0000 7f1d3ffff640 1 --2- 172.21.15.94:0/958727627 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1d280a22e0 0x7f1d280a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.782+0000 7f1d4882b740 1 -- 172.21.15.94:0/3427251937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2d4250 msgr2=0x558b7a2d4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.782+0000 7f1d4882b740 1 --2- 172.21.15.94:0/3427251937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2d4250 0x558b7a2d4630 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f1d38005ec0 tx=0x7f1d3802c670 comp rx=0 tx=0).stop 2024-02-17T12:05:36.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.784+0000 7f1d4882b740 1 -- 172.21.15.94:0/3427251937 shutdown_connections 2024-02-17T12:05:36.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.784+0000 7f1d4882b740 1 --2- 172.21.15.94:0/3427251937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2dd750 0x558b7a2dfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.784+0000 7f1d4882b740 1 --2- 172.21.15.94:0/3427251937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4b70 0x558b7a2dd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.784+0000 7f1d4882b740 1 --2- 172.21.15.94:0/3427251937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2d4250 0x558b7a2d4630 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.784+0000 7f1d4882b740 1 -- 172.21.15.94:0/3427251937 >> 172.21.15.94:0/3427251937 conn(0x558b7a2ce980 msgr2=0x558b7a2ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.785+0000 7f1d3ffff640 1 --2- 172.21.15.94:0/958727627 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1d280a22e0 0x7f1d280a47a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x558b7a2cfb10 tx=0x7f1d34009630 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.788+0000 7f1d4882b740 1 -- 172.21.15.94:0/3427251937 shutdown_connections 2024-02-17T12:05:36.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.792+0000 7f1d4882b740 1 -- 172.21.15.94:0/3427251937 wait complete. 2024-02-17T12:05:36.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.792+0000 7f1d4882b740 1 -- start start 2024-02-17T12:05:36.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.793+0000 7f1d4882b740 1 Processor -- start 2024-02-17T12:05:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.795+0000 7f1d4882b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4250 0x558b7a2da7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.795+0000 7f1d4882b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2d4b70 0x558b7a2dacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.795+0000 7f1d4882b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2dd750 0x558b7a2db230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.795+0000 7f1d4882b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7a23c020 con 0x558b7a2dd750 2024-02-17T12:05:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.795+0000 7f1d4882b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b79f0f430 con 0x558b7a2d4b70 2024-02-17T12:05:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.795+0000 7f1d4882b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b79ecbe00 con 0x558b7a2d4250 2024-02-17T12:05:36.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.795+0000 7f1d4882b740 10 client.0 _ll_register_callbacks cb 0x558b7a1e3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.796+0000 7f1d3ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2dd750 0x558b7a2db230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.796+0000 7f1d3ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2dd750 0x558b7a2db230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:60954/0 (socket says 172.21.15.94:60954) 2024-02-17T12:05:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.796+0000 7f1d3ffff640 1 -- 172.21.15.94:0/3889521242 learned_addr learned my addr 172.21.15.94:0/3889521242 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.797+0000 7f1d3f7fe640 1 --2- 172.21.15.94:0/3889521242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4250 0x558b7a2da7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.797+0000 7f1d3effd640 1 --2- 172.21.15.94:0/3889521242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2d4b70 0x558b7a2dacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.797+0000 7f1d3ffff640 1 -- 172.21.15.94:0/3889521242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4250 msgr2=0x558b7a2da7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.797+0000 7f1d3ffff640 1 --2- 172.21.15.94:0/3889521242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4250 0x558b7a2da7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.797+0000 7f1d3ffff640 1 -- 172.21.15.94:0/3889521242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2d4b70 msgr2=0x558b7a2dacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.797+0000 7f1d3ffff640 1 --2- 172.21.15.94:0/3889521242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2d4b70 0x558b7a2dacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.797+0000 7f1d3ffff640 1 -- 172.21.15.94:0/3889521242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d40114ef0 con 0x558b7a2dd750 2024-02-17T12:05:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.797+0000 7f1d3ffff640 1 --2- 172.21.15.94:0/3889521242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2dd750 0x558b7a2db230 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1d340106b0 tx=0x7f1d340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70d288ba590256d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.798+0000 7f1d1effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.798+0000 7f1d1effd640 1 -- 172.21.15.94:0/3889521242 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34005220 con 0x558b7a2dd750 2024-02-17T12:05:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.798+0000 7f1d4882b740 1 -- 172.21.15.94:0/3889521242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b79f3b7e0 con 0x558b7a2dd750 2024-02-17T12:05:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.798+0000 7f1d1effd640 1 -- 172.21.15.94:0/3889521242 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1d34005380 con 0x558b7a2dd750 2024-02-17T12:05:36.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.798+0000 7f1d1effd640 1 -- 172.21.15.94:0/3889521242 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d34005540 con 0x558b7a2dd750 2024-02-17T12:05:36.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.804+0000 7f1d4882b740 10 client.17442 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:36.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.804+0000 7f1d4882b740 1 -- 172.21.15.94:0/3889521242 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558b7a23a6a0 con 0x558b7a2dd750 2024-02-17T12:05:36.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.804+0000 7f1d1effd640 1 -- 172.21.15.94:0/3889521242 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558b79f3b7e0 con 0x558b7a2dd750 2024-02-17T12:05:36.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.804+0000 7f1d1effd640 1 -- 172.21.15.94:0/3889521242 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1d34018070 con 0x558b7a2dd750 2024-02-17T12:05:36.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.805+0000 7f1d4882b740 10 client.17442 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:36.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.805+0000 7f1d4882b740 10 client.17442 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:05:36.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.805+0000 7f1d4882b740 20 client.17442 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:36.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.805+0000 7f1d4882b740 1 --2- 172.21.15.94:0/3889521242 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558b7a2febb0 0x558b7a300fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.805+0000 7f1d4882b740 4 client.17442 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:36.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.805+0000 7f1d4882b740 1 -- 172.21.15.94:0/3889521242 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b79f78720 con 0x558b7a2febb0 2024-02-17T12:05:36.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.806+0000 7f1d3f7fe640 1 --2- 172.21.15.94:0/3889521242 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558b7a2febb0 0x558b7a300fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.811+0000 7f1d3f7fe640 1 --2- 172.21.15.94:0/3889521242 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558b7a2febb0 0x558b7a300fd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b49397b1b9696e8 server_cookie=eed535b52420ec70 in_seq=0 out_seq=0 2024-02-17T12:05:36.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.811+0000 7f1d1effd640 10 client.17442 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:05:36.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.815+0000 7f1d1effd640 1 -- 172.21.15.94:0/3889521242 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b79f78720 con 0x558b7a2febb0 2024-02-17T12:05:36.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.815+0000 7f1d1effd640 10 client.17442 handle_command_reply: tid=0 2024-02-17T12:05:36.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.816+0000 7f1d4882b740 1 client.17442 shutdown 2024-02-17T12:05:36.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.817+0000 7f1d4882b740 1 -- 172.21.15.94:0/3889521242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2dd750 msgr2=0x558b7a2db230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.817+0000 7f1d4882b740 1 --2- 172.21.15.94:0/3889521242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2dd750 0x558b7a2db230 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1d340106b0 tx=0x7f1d340046c0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.818+0000 7f1d4882b740 1 -- 172.21.15.94:0/3889521242 shutdown_connections 2024-02-17T12:05:36.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.818+0000 7f1d4882b740 1 --2- 172.21.15.94:0/3889521242 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2dd750 0x558b7a2db230 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.818+0000 7f1d4882b740 1 --2- 172.21.15.94:0/3889521242 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558b7a2febb0 0x558b7a300fd0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.818+0000 7f1d4882b740 1 --2- 172.21.15.94:0/3889521242 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2d4b70 0x558b7a2dacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.818+0000 7f1d4882b740 1 --2- 172.21.15.94:0/3889521242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4250 0x558b7a2da7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.818+0000 7f1d4882b740 1 -- 172.21.15.94:0/3889521242 >> 172.21.15.94:0/3889521242 conn(0x558b7a2ce980 msgr2=0x558b7a2d06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.818+0000 7f1d1effd640 0 client.17442 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:05:36.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.819+0000 7f1d4882b740 1 -- 172.21.15.94:0/3889521242 shutdown_connections 2024-02-17T12:05:36.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.819+0000 7f1d4882b740 1 -- 172.21.15.94:0/3889521242 wait complete. 2024-02-17T12:05:36.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.819+0000 7f1d4882b740 20 client.17442 trim_cache size 0 max 16384 2024-02-17T12:05:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.837+0000 7f1d4882b740 1 Processor -- start 2024-02-17T12:05:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.837+0000 7f1d4882b740 1 -- start start 2024-02-17T12:05:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d4882b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2d4250 0x558b7a2dcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d4882b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4b70 0x558b7a2d7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d4882b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2dd750 0x558b7a2d7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d4882b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7a1c1130 con 0x558b7a2d4250 2024-02-17T12:05:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d4882b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7a2d3c20 con 0x558b7a2dd750 2024-02-17T12:05:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d4882b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7a2d5630 con 0x558b7a2d4b70 2024-02-17T12:05:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d3ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2dd750 0x558b7a2d7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d3ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2dd750 0x558b7a2d7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56434/0 (socket says 172.21.15.94:56434) 2024-02-17T12:05:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d3ffff640 1 -- 172.21.15.94:0/4153614043 learned_addr learned my addr 172.21.15.94:0/4153614043 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:36.839 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d3f7fe640 1 --2- 172.21.15.94:0/4153614043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2d4250 0x558b7a2dcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d3effd640 1 --2- 172.21.15.94:0/4153614043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4b70 0x558b7a2d7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d3ffff640 1 -- 172.21.15.94:0/4153614043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4b70 msgr2=0x558b7a2d7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d3ffff640 1 --2- 172.21.15.94:0/4153614043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4b70 0x558b7a2d7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.840 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d3ffff640 1 -- 172.21.15.94:0/4153614043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2d4250 msgr2=0x558b7a2dcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d3ffff640 1 --2- 172.21.15.94:0/4153614043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2d4250 0x558b7a2dcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.838+0000 7f1d3ffff640 1 -- 172.21.15.94:0/4153614043 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b7a1d6c20 con 0x558b7a2dd750 2024-02-17T12:05:36.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.839+0000 7f1d3ffff640 1 --2- 172.21.15.94:0/4153614043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2dd750 0x558b7a2d7f90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400e7c0 tx=0x7f1d34009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1246fedd9cf158ed server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.839+0000 7f1d257fa640 1 -- 172.21.15.94:0/4153614043 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3401c6f0 con 0x558b7a2dd750 2024-02-17T12:05:36.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.839+0000 7f1d257fa640 1 -- 172.21.15.94:0/4153614043 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d3402cc20 con 0x558b7a2dd750 2024-02-17T12:05:36.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.839+0000 7f1d257fa640 1 -- 172.21.15.94:0/4153614043 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3402d6a0 con 0x558b7a2dd750 2024-02-17T12:05:36.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.840+0000 7f1d4882b740 1 -- 172.21.15.94:0/4153614043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2dd750 msgr2=0x558b7a2d7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.841+0000 7f1d4882b740 1 --2- 172.21.15.94:0/4153614043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2dd750 0x558b7a2d7f90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400e7c0 tx=0x7f1d34009900 comp rx=0 tx=0).stop 2024-02-17T12:05:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.841+0000 7f1d4882b740 1 -- 172.21.15.94:0/4153614043 shutdown_connections 2024-02-17T12:05:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.841+0000 7f1d4882b740 1 --2- 172.21.15.94:0/4153614043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2dd750 0x558b7a2d7f90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.841+0000 7f1d4882b740 1 --2- 172.21.15.94:0/4153614043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4b70 0x558b7a2d7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.841+0000 7f1d4882b740 1 --2- 172.21.15.94:0/4153614043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2d4250 0x558b7a2dcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.841+0000 7f1d4882b740 1 -- 172.21.15.94:0/4153614043 >> 172.21.15.94:0/4153614043 conn(0x558b7a2ce980 msgr2=0x558b7a2d5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.841+0000 7f1d4882b740 1 -- 172.21.15.94:0/4153614043 shutdown_connections 2024-02-17T12:05:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.841+0000 7f1d4882b740 1 -- 172.21.15.94:0/4153614043 wait complete. 2024-02-17T12:05:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.842+0000 7f1d4882b740 1 -- start start 2024-02-17T12:05:36.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.842+0000 7f1d4882b740 1 Processor -- start 2024-02-17T12:05:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.842+0000 7f1d4882b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4250 0x558b7a34e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d4882b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2d4b70 0x558b7a34e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d4882b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2dd750 0x558b7a2f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d4882b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7a2f4fc0 con 0x558b7a2dd750 2024-02-17T12:05:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d4882b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7a2f5130 con 0x558b7a2d4b70 2024-02-17T12:05:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d3effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2d4b70 0x558b7a34e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d4882b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7a2f52f0 con 0x558b7a2d4250 2024-02-17T12:05:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d4882b740 10 client.0 _ll_register_callbacks cb 0x7f1d34005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:36.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d3effd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2d4b70 0x558b7a34e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56436/0 (socket says 172.21.15.94:56436) 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d3effd640 1 -- 172.21.15.94:0/4178127741 learned_addr learned my addr 172.21.15.94:0/4178127741 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d3f7fe640 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4250 0x558b7a34e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d3ffff640 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2dd750 0x558b7a2f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d3effd640 1 -- 172.21.15.94:0/4178127741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4250 msgr2=0x558b7a34e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d3effd640 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4250 0x558b7a34e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d3effd640 1 -- 172.21.15.94:0/4178127741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2dd750 msgr2=0x558b7a2f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d3effd640 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2dd750 0x558b7a2f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d3effd640 1 -- 172.21.15.94:0/4178127741 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b7a23a6a0 con 0x558b7a2d4b70 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.843+0000 7f1d3ffff640 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2dd750 0x558b7a2f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d3effd640 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2d4b70 0x558b7a34e960 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1d3009a7e0 tx=0x7f1d30096eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3cfb82e454dce93 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d1effd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d1effd640 1 -- 172.21.15.94:0/4178127741 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d300b7070 con 0x558b7a2d4b70 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d1effd640 1 -- 172.21.15.94:0/4178127741 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f1d300b2070 con 0x558b7a2d4b70 2024-02-17T12:05:36.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d4882b740 1 -- 172.21.15.94:0/4178127741 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d3802cab0 con 0x558b7a2d4b70 2024-02-17T12:05:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d1effd640 1 -- 172.21.15.94:0/4178127741 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d300cbcd0 con 0x558b7a2d4b70 2024-02-17T12:05:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d4882b740 10 client.26933 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d4882b740 1 -- 172.21.15.94:0/4178127741 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558b7a1fc8c0 con 0x558b7a2d4b70 2024-02-17T12:05:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d1effd640 1 -- 172.21.15.94:0/4178127741 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d3802cab0 con 0x558b7a2d4b70 2024-02-17T12:05:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d1effd640 1 -- 172.21.15.94:0/4178127741 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1d30096830 con 0x558b7a2d4b70 2024-02-17T12:05:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d4882b740 10 client.26933 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d4882b740 10 client.26933 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:05:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d4882b740 20 client.26933 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d4882b740 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558b7a3574a0 0x558b7a3598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d4882b740 4 client.26933 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:05:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.844+0000 7f1d4882b740 1 -- 172.21.15.94:0/4178127741 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558b79fc7890 con 0x558b7a3574a0 2024-02-17T12:05:36.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.845+0000 7f1d3f7fe640 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558b7a3574a0 0x558b7a3598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:36.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.846+0000 7f1d3f7fe640 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558b7a3574a0 0x558b7a3598c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e81cd524d96b0a46 server_cookie=7f9bf8b6335d1365 in_seq=0 out_seq=0 2024-02-17T12:05:36.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.846+0000 7f1d1effd640 10 client.26933 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:05:36.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.846+0000 7f1d1effd640 1 -- 172.21.15.94:0/4178127741 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558b79fc7890 con 0x558b7a3574a0 2024-02-17T12:05:36.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.846+0000 7f1d1effd640 10 client.26933 handle_command_reply: tid=0 2024-02-17T12:05:36.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.847+0000 7f1d4882b740 1 client.26933 shutdown 2024-02-17T12:05:36.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.847+0000 7f1d4882b740 1 -- 172.21.15.94:0/4178127741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2d4b70 msgr2=0x558b7a34e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.847+0000 7f1d4882b740 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2d4b70 0x558b7a34e960 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1d3009a7e0 tx=0x7f1d30096eb0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.847+0000 7f1d4882b740 1 -- 172.21.15.94:0/4178127741 shutdown_connections 2024-02-17T12:05:36.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.847+0000 7f1d4882b740 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558b7a2dd750 0x558b7a2f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.847+0000 7f1d4882b740 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x558b7a3574a0 0x558b7a3598c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.847+0000 7f1d4882b740 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558b7a2d4b70 0x558b7a34e960 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.847+0000 7f1d4882b740 1 --2- 172.21.15.94:0/4178127741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558b7a2d4250 0x558b7a34e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.847+0000 7f1d4882b740 1 -- 172.21.15.94:0/4178127741 >> 172.21.15.94:0/4178127741 conn(0x558b7a2ce980 msgr2=0x558b7a2da750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.847+0000 7f1d1effd640 0 client.26933 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:05:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.847+0000 7f1d4882b740 1 -- 172.21.15.94:0/4178127741 shutdown_connections 2024-02-17T12:05:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.847+0000 7f1d4882b740 1 -- 172.21.15.94:0/4178127741 wait complete. 2024-02-17T12:05:36.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.848+0000 7f1d4882b740 20 client.26933 trim_cache size 0 max 16384 2024-02-17T12:05:36.851 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:05:36.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.848+0000 7f1d45b7a640 1 -- 172.21.15.94:0/958727627 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1d280a22e0 msgr2=0x7f1d280a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.848+0000 7f1d45b7a640 1 --2- 172.21.15.94:0/958727627 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1d280a22e0 0x7f1d280a47a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x558b7a2cfb10 tx=0x7f1d34009630 comp rx=0 tx=0).stop 2024-02-17T12:05:36.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.848+0000 7f1d45b7a640 1 -- 172.21.15.94:0/958727627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d401080e0 msgr2=0x7f1d40081b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:36.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.848+0000 7f1d45b7a640 1 --2- 172.21.15.94:0/958727627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d401080e0 0x7f1d40081b60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000ec80 tx=0x7f1d3000c6a0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.848+0000 7f1d45b7a640 1 -- 172.21.15.94:0/958727627 shutdown_connections 2024-02-17T12:05:36.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.848+0000 7f1d45b7a640 1 --2- 172.21.15.94:0/958727627 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1d280a22e0 0x7f1d280a47a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.848+0000 7f1d45b7a640 1 --2- 172.21.15.94:0/958727627 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1d4010d730 0x7f1d4007a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.848+0000 7f1d45b7a640 1 --2- 172.21.15.94:0/958727627 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1d401080e0 0x7f1d40081b60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.848+0000 7f1d45b7a640 1 --2- 172.21.15.94:0/958727627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d40107720 0x7f1d40081600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:36.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.848+0000 7f1d45b7a640 1 -- 172.21.15.94:0/958727627 >> 172.21.15.94:0/958727627 conn(0x7f1d400756c0 msgr2=0x7f1d40075ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:36.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.848+0000 7f1d45b7a640 1 -- 172.21.15.94:0/958727627 shutdown_connections 2024-02-17T12:05:36.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:36.849+0000 7f1d45b7a640 1 -- 172.21.15.94:0/958727627 wait complete. 2024-02-17T12:05:36.864 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:05:36.864 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:05:37.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.017+0000 7fb4237fe640 1 -- 172.21.15.94:0/2328683760 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c02cbe0 con 0x7fb424069a60 2024-02-17T12:05:37.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.018+0000 7fb42afd0640 1 -- 172.21.15.94:0/2328683760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb424069a60 msgr2=0x7fb424069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.018+0000 7fb42afd0640 1 --2- 172.21.15.94:0/2328683760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb424069a60 0x7fb424069ec0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009900 tx=0x7fb40c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:05:37.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.018+0000 7fb42afd0640 1 -- 172.21.15.94:0/2328683760 shutdown_connections 2024-02-17T12:05:37.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.018+0000 7fb42afd0640 1 --2- 172.21.15.94:0/2328683760 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb424103b80 0x7fb424104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.018+0000 7fb42afd0640 1 --2- 172.21.15.94:0/2328683760 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb424069a60 0x7fb424069ec0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.018+0000 7fb42afd0640 1 --2- 172.21.15.94:0/2328683760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb424106080 0x7fb424069520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.018+0000 7fb42afd0640 1 -- 172.21.15.94:0/2328683760 >> 172.21.15.94:0/2328683760 conn(0x7fb424077110 msgr2=0x7fb424077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:37.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.018+0000 7fb42afd0640 1 -- 172.21.15.94:0/2328683760 shutdown_connections 2024-02-17T12:05:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.018+0000 7fb42afd0640 1 -- 172.21.15.94:0/2328683760 wait complete. 2024-02-17T12:05:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.019+0000 7fb42afd0640 1 Processor -- start 2024-02-17T12:05:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.019+0000 7fb42afd0640 1 -- start start 2024-02-17T12:05:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.019+0000 7fb42afd0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb424069a60 0x7fb4241a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.019+0000 7fb42afd0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb424103b80 0x7fb4241a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.019+0000 7fb42afd0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb424106080 0x7fb42419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.019+0000 7fb42afd0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42419e140 con 0x7fb424106080 2024-02-17T12:05:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.019+0000 7fb42afd0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42419e2b0 con 0x7fb424069a60 2024-02-17T12:05:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.019+0000 7fb42afd0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42419e470 con 0x7fb424103b80 2024-02-17T12:05:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.019+0000 7fb42956d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb424106080 0x7fb42419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.019+0000 7fb428d6c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb424069a60 0x7fb4241a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.019+0000 7fb428d6c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb424069a60 0x7fb4241a3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56452/0 (socket says 172.21.15.94:56452) 2024-02-17T12:05:37.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.019+0000 7fb42956d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb424106080 0x7fb42419daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32776/0 (socket says 172.21.15.94:32776) 2024-02-17T12:05:37.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.019+0000 7fb428d6c640 1 -- 172.21.15.94:0/925697737 learned_addr learned my addr 172.21.15.94:0/925697737 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:37.022 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.020+0000 7fb423fff640 1 --2- 172.21.15.94:0/925697737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb424103b80 0x7fb4241a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.020+0000 7fb42956d640 1 -- 172.21.15.94:0/925697737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb424103b80 msgr2=0x7fb4241a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.020+0000 7fb42956d640 1 --2- 172.21.15.94:0/925697737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb424103b80 0x7fb4241a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.020+0000 7fb42956d640 1 -- 172.21.15.94:0/925697737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb424069a60 msgr2=0x7fb4241a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.020+0000 7fb42956d640 1 --2- 172.21.15.94:0/925697737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb424069a60 0x7fb4241a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.020+0000 7fb42956d640 1 -- 172.21.15.94:0/925697737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c009560 con 0x7fb424106080 2024-02-17T12:05:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.020+0000 7fb42956d640 1 --2- 172.21.15.94:0/925697737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb424106080 0x7fb42419daf0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb414009fd0 tx=0x7fb41400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.021+0000 7fb421ffb640 1 -- 172.21.15.94:0/925697737 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41400ce20 con 0x7fb424106080 2024-02-17T12:05:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.021+0000 7fb421ffb640 1 -- 172.21.15.94:0/925697737 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb414005020 con 0x7fb424106080 2024-02-17T12:05:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.021+0000 7fb42afd0640 1 -- 172.21.15.94:0/925697737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb42419e750 con 0x7fb424106080 2024-02-17T12:05:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.021+0000 7fb421ffb640 1 -- 172.21.15.94:0/925697737 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb414010690 con 0x7fb424106080 2024-02-17T12:05:37.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.022+0000 7fb42afd0640 1 -- 172.21.15.94:0/925697737 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4241aa9e0 con 0x7fb424106080 2024-02-17T12:05:37.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.024+0000 7fb421ffb640 1 -- 172.21.15.94:0/925697737 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb414005500 con 0x7fb424106080 2024-02-17T12:05:37.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.025+0000 7fb42dba5740 1 Processor -- start 2024-02-17T12:05:37.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.025+0000 7fb42dba5740 1 -- start start 2024-02-17T12:05:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.025+0000 7fb42dba5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c29630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.025+0000 7fb42dba5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29b70 0x557d22c32210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.025+0000 7fb42dba5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 0x557d22c34b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.025+0000 7fb42dba5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d22be4960 con 0x557d22c29250 2024-02-17T12:05:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.025+0000 7fb42dba5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d22b98310 con 0x557d22c29b70 2024-02-17T12:05:37.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.025+0000 7fb42dba5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d22b95840 con 0x557d22c32750 2024-02-17T12:05:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.025+0000 7fb421ffb640 1 --2- 172.21.15.94:0/925697737 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4100a2400 0x7fb4100a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.025+0000 7fb423fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c29630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.025+0000 7fb423fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c29630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32786/0 (socket says 172.21.15.94:32786) 2024-02-17T12:05:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.025+0000 7fb423fff640 1 -- 172.21.15.94:0/3659780838 learned_addr learned my addr 172.21.15.94:0/3659780838 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.025+0000 7fb42956d640 1 --2- 172.21.15.94:0/925697737 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4100a2400 0x7fb4100a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.026+0000 7fb428d6c640 1 --2- 172.21.15.94:0/3659780838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29b70 0x557d22c32210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.026+0000 7fb421ffb640 1 -- 172.21.15.94:0/925697737 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb414014070 con 0x7fb424106080 2024-02-17T12:05:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.026+0000 7fb423fff640 1 --2- 172.21.15.94:0/3659780838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 0x557d22c34b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.026+0000 7fb423fff640 1 -- 172.21.15.94:0/3659780838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 msgr2=0x557d22c34b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.026+0000 7fb423fff640 1 --2- 172.21.15.94:0/3659780838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 0x557d22c34b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.026+0000 7fb423fff640 1 -- 172.21.15.94:0/3659780838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29b70 msgr2=0x557d22c32210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.026+0000 7fb423fff640 1 --2- 172.21.15.94:0/3659780838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29b70 0x557d22c32210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.028 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.026+0000 7fb423fff640 1 -- 172.21.15.94:0/3659780838 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d22b53060 con 0x557d22c29250 2024-02-17T12:05:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.026+0000 7fb423fff640 1 --2- 172.21.15.94:0/3659780838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c29630 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb40c006010 tx=0x7fb40c005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0d5086b60ebb7a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.026+0000 7fb42956d640 1 --2- 172.21.15.94:0/925697737 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4100a2400 0x7fb4100a48c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb414097a90 tx=0x7fb414013700 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.026+0000 7fb41affd640 1 -- 172.21.15.94:0/3659780838 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c02c860 con 0x557d22c29250 2024-02-17T12:05:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.026+0000 7fb41affd640 1 -- 172.21.15.94:0/3659780838 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb40c004510 con 0x557d22c29250 2024-02-17T12:05:37.029 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.026+0000 7fb41affd640 1 -- 172.21.15.94:0/3659780838 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c005400 con 0x557d22c29250 2024-02-17T12:05:37.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.027+0000 7fb42dba5740 1 -- 172.21.15.94:0/3659780838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 msgr2=0x557d22c29630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.027+0000 7fb42dba5740 1 --2- 172.21.15.94:0/3659780838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c29630 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb40c006010 tx=0x7fb40c005e20 comp rx=0 tx=0).stop 2024-02-17T12:05:37.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.027+0000 7fb42dba5740 1 -- 172.21.15.94:0/3659780838 shutdown_connections 2024-02-17T12:05:37.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.027+0000 7fb42dba5740 1 --2- 172.21.15.94:0/3659780838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 0x557d22c34b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.027+0000 7fb42dba5740 1 --2- 172.21.15.94:0/3659780838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29b70 0x557d22c32210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.027+0000 7fb42dba5740 1 --2- 172.21.15.94:0/3659780838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c29630 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.027+0000 7fb42dba5740 1 -- 172.21.15.94:0/3659780838 >> 172.21.15.94:0/3659780838 conn(0x557d22c23980 msgr2=0x557d22c23d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.029+0000 7fb42dba5740 1 -- 172.21.15.94:0/3659780838 shutdown_connections 2024-02-17T12:05:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.029+0000 7fb42dba5740 1 -- 172.21.15.94:0/3659780838 wait complete. 2024-02-17T12:05:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.029+0000 7fb42dba5740 1 -- start start 2024-02-17T12:05:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.029+0000 7fb42dba5740 1 Processor -- start 2024-02-17T12:05:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.029+0000 7fb42dba5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c2f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.029+0000 7fb42dba5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29b70 0x557d22c2fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb42dba5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 0x557d22c30220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb42dba5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d22b91020 con 0x557d22c29250 2024-02-17T12:05:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb42dba5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d22864430 con 0x557d22c29b70 2024-02-17T12:05:37.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb42dba5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d22820e00 con 0x557d22c32750 2024-02-17T12:05:37.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb42dba5740 10 client.0 _ll_register_callbacks cb 0x557d22b38cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:37.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb423fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c2f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb423fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c2f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32790/0 (socket says 172.21.15.94:32790) 2024-02-17T12:05:37.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb428d6c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29b70 0x557d22c2fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb423fff640 1 -- 172.21.15.94:0/1512140914 learned_addr learned my addr 172.21.15.94:0/1512140914 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:37.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb423fff640 1 --2- 172.21.15.94:0/1512140914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 0x557d22c30220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb428d6c640 1 -- 172.21.15.94:0/1512140914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 msgr2=0x557d22c30220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb428d6c640 1 --2- 172.21.15.94:0/1512140914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 0x557d22c30220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb428d6c640 1 -- 172.21.15.94:0/1512140914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 msgr2=0x557d22c2f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb428d6c640 1 --2- 172.21.15.94:0/1512140914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c2f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb428d6c640 1 -- 172.21.15.94:0/1512140914 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb424114f80 con 0x557d22c29b70 2024-02-17T12:05:37.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb423fff640 1 --2- 172.21.15.94:0/1512140914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c2f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:37.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb428d6c640 1 --2- 172.21.15.94:0/1512140914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29b70 0x557d22c2fce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb41c010a20 tx=0x7fb41c010ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7530d2825bd761ad server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb403fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb403fff640 1 -- 172.21.15.94:0/1512140914 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c015070 con 0x557d22c29b70 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb403fff640 1 -- 172.21.15.94:0/1512140914 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb41c00fb90 con 0x557d22c29b70 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.030+0000 7fb42dba5740 1 -- 172.21.15.94:0/1512140914 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d228907e0 con 0x557d22c29b70 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.031+0000 7fb403fff640 1 -- 172.21.15.94:0/1512140914 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb41c00fd50 con 0x557d22c29b70 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.031+0000 7fb42dba5740 10 client.26945 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.031+0000 7fb42dba5740 1 -- 172.21.15.94:0/1512140914 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557d22b8f6a0 con 0x557d22c29b70 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.031+0000 7fb403fff640 1 -- 172.21.15.94:0/1512140914 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557d228907e0 con 0x557d22c29b70 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.031+0000 7fb403fff640 1 -- 172.21.15.94:0/1512140914 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb41c00e440 con 0x557d22c29b70 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.031+0000 7fb42dba5740 10 client.26945 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.031+0000 7fb42dba5740 10 client.26945 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.031+0000 7fb42dba5740 20 client.26945 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.031+0000 7fb42dba5740 1 --2- 172.21.15.94:0/1512140914 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x557d22c42ba0 0x557d22c44fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.031+0000 7fb42dba5740 4 client.26945 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:37.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.031+0000 7fb42dba5740 1 -- 172.21.15.94:0/1512140914 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d228cd720 con 0x557d22c42ba0 2024-02-17T12:05:37.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.031+0000 7fb423fff640 1 --2- 172.21.15.94:0/1512140914 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x557d22c42ba0 0x557d22c44fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.032+0000 7fb423fff640 1 --2- 172.21.15.94:0/1512140914 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x557d22c42ba0 0x557d22c44fc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ecf9181b775acfa9 server_cookie=bf3a10ef7d764512 in_seq=0 out_seq=0 2024-02-17T12:05:37.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.032+0000 7fb403fff640 10 client.26945 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:05:37.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.032+0000 7fb403fff640 1 -- 172.21.15.94:0/1512140914 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d228cd720 con 0x557d22c42ba0 2024-02-17T12:05:37.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.032+0000 7fb403fff640 10 client.26945 handle_command_reply: tid=0 2024-02-17T12:05:37.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.032+0000 7fb42dba5740 1 client.26945 shutdown 2024-02-17T12:05:37.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.033+0000 7fb42dba5740 1 -- 172.21.15.94:0/1512140914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29b70 msgr2=0x557d22c2fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.033+0000 7fb42dba5740 1 --2- 172.21.15.94:0/1512140914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29b70 0x557d22c2fce0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb41c010a20 tx=0x7fb41c010ee0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.033+0000 7fb42dba5740 1 -- 172.21.15.94:0/1512140914 shutdown_connections 2024-02-17T12:05:37.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.033+0000 7fb42dba5740 1 --2- 172.21.15.94:0/1512140914 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x557d22c42ba0 0x557d22c44fc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.037 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.033+0000 7fb403fff640 0 client.26945 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:05:37.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.033+0000 7fb42dba5740 1 --2- 172.21.15.94:0/1512140914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 0x557d22c30220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.033+0000 7fb42dba5740 1 --2- 172.21.15.94:0/1512140914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29b70 0x557d22c2fce0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.033+0000 7fb42dba5740 1 --2- 172.21.15.94:0/1512140914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c2f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.033+0000 7fb42dba5740 1 -- 172.21.15.94:0/1512140914 >> 172.21.15.94:0/1512140914 conn(0x557d22c23980 msgr2=0x557d22c256d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:37.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.033+0000 7fb42dba5740 1 -- 172.21.15.94:0/1512140914 shutdown_connections 2024-02-17T12:05:37.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.033+0000 7fb42dba5740 1 -- 172.21.15.94:0/1512140914 wait complete. 2024-02-17T12:05:37.038 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.033+0000 7fb42dba5740 20 client.26945 trim_cache size 0 max 16384 2024-02-17T12:05:37.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.051+0000 7fb42dba5740 1 Processor -- start 2024-02-17T12:05:37.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.052+0000 7fb42dba5740 1 -- start start 2024-02-17T12:05:37.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.053+0000 7fb42dba5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29250 0x557d22c31ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.053+0000 7fb42dba5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29b70 0x557d22c30040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.053+0000 7fb42dba5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 0x557d22c1ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.053+0000 7fb428d6c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29b70 0x557d22c30040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.053+0000 7fb42dba5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d22b16130 con 0x557d22c29b70 2024-02-17T12:05:37.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.053+0000 7fb428d6c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29b70 0x557d22c30040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32800/0 (socket says 172.21.15.94:32800) 2024-02-17T12:05:37.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.053+0000 7fb42dba5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d22c28eb0 con 0x557d22c29250 2024-02-17T12:05:37.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.053+0000 7fb428d6c640 1 -- 172.21.15.94:0/3504779130 learned_addr learned my addr 172.21.15.94:0/3504779130 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:37.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.053+0000 7fb42dba5740 1 -- 172.21.15.94:0/3504779130 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d22c4a1f0 con 0x557d22c32750 2024-02-17T12:05:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.053+0000 7fb423fff640 1 --2- 172.21.15.94:0/3504779130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29250 0x557d22c31ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.054+0000 7fb428d6c640 1 -- 172.21.15.94:0/3504779130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 msgr2=0x557d22c1ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.054+0000 7fb428d6c640 1 --2- 172.21.15.94:0/3504779130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 0x557d22c1ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.054+0000 7fb428d6c640 1 -- 172.21.15.94:0/3504779130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29250 msgr2=0x557d22c31ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.054+0000 7fb428d6c640 1 --2- 172.21.15.94:0/3504779130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29250 0x557d22c31ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.054+0000 7fb428d6c640 1 -- 172.21.15.94:0/3504779130 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d22b2bc20 con 0x557d22c29b70 2024-02-17T12:05:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.054+0000 7fb428d6c640 1 --2- 172.21.15.94:0/3504779130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29b70 0x557d22c30040 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00eac0 tx=0x7fb41c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d7fd3dff064a15 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.054+0000 7fb41a7fc640 1 -- 172.21.15.94:0/3504779130 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c015070 con 0x557d22c29b70 2024-02-17T12:05:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.054+0000 7fb41a7fc640 1 -- 172.21.15.94:0/3504779130 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb41c016040 con 0x557d22c29b70 2024-02-17T12:05:37.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.054+0000 7fb41a7fc640 1 -- 172.21.15.94:0/3504779130 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c00fdf0 con 0x557d22c29b70 2024-02-17T12:05:37.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.055+0000 7fb42dba5740 1 -- 172.21.15.94:0/3504779130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29b70 msgr2=0x557d22c30040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.055+0000 7fb42dba5740 1 --2- 172.21.15.94:0/3504779130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29b70 0x557d22c30040 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb41c00eac0 tx=0x7fb41c00f7f0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.055+0000 7fb42dba5740 1 -- 172.21.15.94:0/3504779130 shutdown_connections 2024-02-17T12:05:37.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.055+0000 7fb42dba5740 1 --2- 172.21.15.94:0/3504779130 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c32750 0x557d22c1ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.055+0000 7fb42dba5740 1 --2- 172.21.15.94:0/3504779130 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29b70 0x557d22c30040 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.055+0000 7fb42dba5740 1 --2- 172.21.15.94:0/3504779130 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c29250 0x557d22c31ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.055+0000 7fb42dba5740 1 -- 172.21.15.94:0/3504779130 >> 172.21.15.94:0/3504779130 conn(0x557d22c23980 msgr2=0x557d22c29f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:37.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.056+0000 7fb42dba5740 1 -- 172.21.15.94:0/3504779130 shutdown_connections 2024-02-17T12:05:37.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.056+0000 7fb42dba5740 1 -- 172.21.15.94:0/3504779130 wait complete. 2024-02-17T12:05:37.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.056+0000 7fb42dba5740 1 -- start start 2024-02-17T12:05:37.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.056+0000 7fb42dba5740 1 Processor -- start 2024-02-17T12:05:37.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.056+0000 7fb42dba5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c37900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.056+0000 7fb42dba5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c29b70 0x557d22c37e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb42dba5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c32750 0x557d22c92100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb423fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c37900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb423fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c37900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32806/0 (socket says 172.21.15.94:32806) 2024-02-17T12:05:37.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb423fff640 1 -- 172.21.15.94:0/932909204 learned_addr learned my addr 172.21.15.94:0/932909204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:37.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb42dba5740 1 -- 172.21.15.94:0/932909204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d22c92640 con 0x557d22c29250 2024-02-17T12:05:37.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb42dba5740 1 -- 172.21.15.94:0/932909204 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d22c927b0 con 0x557d22c32750 2024-02-17T12:05:37.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb42dba5740 1 -- 172.21.15.94:0/932909204 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d22c92970 con 0x557d22c29b70 2024-02-17T12:05:37.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb423fff640 1 --2- 172.21.15.94:0/932909204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c32750 0x557d22c92100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb42dba5740 10 client.0 _ll_register_callbacks cb 0x7fb40c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:37.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb428d6c640 1 --2- 172.21.15.94:0/932909204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c29b70 0x557d22c37e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb423fff640 1 -- 172.21.15.94:0/932909204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c29b70 msgr2=0x557d22c37e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb423fff640 1 --2- 172.21.15.94:0/932909204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c29b70 0x557d22c37e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb423fff640 1 -- 172.21.15.94:0/932909204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c32750 msgr2=0x557d22c92100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb423fff640 1 --2- 172.21.15.94:0/932909204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c32750 0x557d22c92100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb423fff640 1 -- 172.21.15.94:0/932909204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d22b8f6a0 con 0x557d22c29250 2024-02-17T12:05:37.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb423fff640 1 --2- 172.21.15.94:0/932909204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c37900 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0058c0 tx=0x7fb40c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6f3064a7e9751b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:37.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.057+0000 7fb403fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.058+0000 7fb403fff640 1 -- 172.21.15.94:0/932909204 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c058d00 con 0x557d22c29250 2024-02-17T12:05:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.058+0000 7fb403fff640 1 -- 172.21.15.94:0/932909204 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7fb40c005130 con 0x557d22c29250 2024-02-17T12:05:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.058+0000 7fb42dba5740 1 -- 172.21.15.94:0/932909204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb40c004510 con 0x557d22c29250 2024-02-17T12:05:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.059+0000 7fb403fff640 1 -- 172.21.15.94:0/932909204 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb40c058e60 con 0x557d22c29250 2024-02-17T12:05:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.059+0000 7fb42dba5740 10 client.17466 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.060+0000 7fb42dba5740 1 -- 172.21.15.94:0/932909204 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557d22b518c0 con 0x557d22c29250 2024-02-17T12:05:37.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.060+0000 7fb403fff640 1 -- 172.21.15.94:0/932909204 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb40c005500 con 0x557d22c29250 2024-02-17T12:05:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.060+0000 7fb403fff640 1 -- 172.21.15.94:0/932909204 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb40c03c070 con 0x557d22c29250 2024-02-17T12:05:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.060+0000 7fb42dba5740 10 client.17466 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.060+0000 7fb42dba5740 10 client.17466 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:05:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.060+0000 7fb42dba5740 20 client.17466 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.060+0000 7fb42dba5740 1 --2- 172.21.15.94:0/932909204 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x557d22c42ba0 0x557d22c9bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.060+0000 7fb42dba5740 4 client.17466 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:05:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.060+0000 7fb42dba5740 1 -- 172.21.15.94:0/932909204 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fb40c004510 con 0x557d22c42ba0 2024-02-17T12:05:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.061+0000 7fb428d6c640 1 --2- 172.21.15.94:0/932909204 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x557d22c42ba0 0x557d22c9bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.061+0000 7fb428d6c640 1 --2- 172.21.15.94:0/932909204 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x557d22c42ba0 0x557d22c9bf00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91703b96f8f1df60 server_cookie=da2a6c56c5b7466c in_seq=0 out_seq=0 2024-02-17T12:05:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.061+0000 7fb403fff640 10 client.17466 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:05:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb403fff640 1 -- 172.21.15.94:0/932909204 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fb40c004510 con 0x557d22c42ba0 2024-02-17T12:05:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb403fff640 10 client.17466 handle_command_reply: tid=0 2024-02-17T12:05:37.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb42dba5740 1 client.17466 shutdown 2024-02-17T12:05:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb42dba5740 1 -- 172.21.15.94:0/932909204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 msgr2=0x557d22c37900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb42dba5740 1 --2- 172.21.15.94:0/932909204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c37900 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0058c0 tx=0x7fb40c005850 comp rx=0 tx=0).stop 2024-02-17T12:05:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb42dba5740 1 -- 172.21.15.94:0/932909204 shutdown_connections 2024-02-17T12:05:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb42dba5740 1 --2- 172.21.15.94:0/932909204 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x557d22c42ba0 0x557d22c9bf00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb42dba5740 1 --2- 172.21.15.94:0/932909204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557d22c32750 0x557d22c92100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb42dba5740 1 --2- 172.21.15.94:0/932909204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d22c29b70 0x557d22c37e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb42dba5740 1 --2- 172.21.15.94:0/932909204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557d22c29250 0x557d22c37900 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb42dba5740 1 -- 172.21.15.94:0/932909204 >> 172.21.15.94:0/932909204 conn(0x557d22c23980 msgr2=0x557d22c1ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb403fff640 0 client.17466 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:05:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb42dba5740 1 -- 172.21.15.94:0/932909204 shutdown_connections 2024-02-17T12:05:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb42dba5740 1 -- 172.21.15.94:0/932909204 wait complete. 2024-02-17T12:05:37.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.062+0000 7fb42dba5740 20 client.17466 trim_cache size 0 max 16384 2024-02-17T12:05:37.066 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:05:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.063+0000 7fb42afd0640 1 -- 172.21.15.94:0/925697737 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4100a2400 msgr2=0x7fb4100a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.063+0000 7fb42afd0640 1 --2- 172.21.15.94:0/925697737 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4100a2400 0x7fb4100a48c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb414097a90 tx=0x7fb414013700 comp rx=0 tx=0).stop 2024-02-17T12:05:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.063+0000 7fb42afd0640 1 -- 172.21.15.94:0/925697737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb424106080 msgr2=0x7fb42419daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.063+0000 7fb42afd0640 1 --2- 172.21.15.94:0/925697737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb424106080 0x7fb42419daf0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb414009fd0 tx=0x7fb41400ef10 comp rx=0 tx=0).stop 2024-02-17T12:05:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.063+0000 7fb42afd0640 1 -- 172.21.15.94:0/925697737 shutdown_connections 2024-02-17T12:05:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.063+0000 7fb42afd0640 1 --2- 172.21.15.94:0/925697737 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4100a2400 0x7fb4100a48c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.063+0000 7fb42afd0640 1 --2- 172.21.15.94:0/925697737 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb424106080 0x7fb42419daf0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.063+0000 7fb42afd0640 1 --2- 172.21.15.94:0/925697737 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb424103b80 0x7fb4241a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.063+0000 7fb42afd0640 1 --2- 172.21.15.94:0/925697737 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb424069a60 0x7fb4241a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.063+0000 7fb42afd0640 1 -- 172.21.15.94:0/925697737 >> 172.21.15.94:0/925697737 conn(0x7fb424077110 msgr2=0x7fb4240fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.063+0000 7fb42afd0640 1 -- 172.21.15.94:0/925697737 shutdown_connections 2024-02-17T12:05:37.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.063+0000 7fb42afd0640 1 -- 172.21.15.94:0/925697737 wait complete. 2024-02-17T12:05:37.080 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:05:37.080 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:05:37.206 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:36 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/28411952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:05:37.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:36 smithi094 ceph-mon[32499]: pgmap v230: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.1 MiB/s wr, 692 op/s 2024-02-17T12:05:37.244 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.242+0000 7f96b49f4640 1 -- 172.21.15.94:0/1993101272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96b0101390 msgr2=0x7f96b0101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.243+0000 7f96b49f4640 1 --2- 172.21.15.94:0/1993101272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96b0101390 0x7f96b0101770 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009930 tx=0x7f96a002bc50 comp rx=0 tx=0).stop 2024-02-17T12:05:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.244+0000 7f96b49f4640 1 -- 172.21.15.94:0/1993101272 shutdown_connections 2024-02-17T12:05:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.244+0000 7f96b49f4640 1 --2- 172.21.15.94:0/1993101272 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96b0106460 0x7f96b010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.244+0000 7f96b49f4640 1 --2- 172.21.15.94:0/1993101272 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96b0101d40 0x7f96b0105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.244+0000 7f96b49f4640 1 --2- 172.21.15.94:0/1993101272 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96b0101390 0x7f96b0101770 secure :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009930 tx=0x7f96a002bc50 comp rx=0 tx=0).stop 2024-02-17T12:05:37.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.244+0000 7f96b49f4640 1 -- 172.21.15.94:0/1993101272 >> 172.21.15.94:0/1993101272 conn(0x7f96b00fcc10 msgr2=0x7f96b00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.247+0000 7f96b49f4640 1 -- 172.21.15.94:0/1993101272 shutdown_connections 2024-02-17T12:05:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.248+0000 7f96b49f4640 1 -- 172.21.15.94:0/1993101272 wait complete. 2024-02-17T12:05:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.249+0000 7f96b49f4640 1 Processor -- start 2024-02-17T12:05:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.249+0000 7f96b49f4640 1 -- start start 2024-02-17T12:05:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.249+0000 7f96b49f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96b0106460 0x7f96b01ac660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.249+0000 7f96b49f4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96b01acba0 0x7f96b01a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.249+0000 7f96b49f4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96b01a6d00 0x7f96b01a7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.249+0000 7f96b49f4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b01ad240 con 0x7f96b0106460 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.249+0000 7f96b49f4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b01b3470 con 0x7f96b01a6d00 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.249+0000 7f96b49f4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b01b35b0 con 0x7f96b01acba0 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.250+0000 7f96add9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96b01acba0 0x7f96b01a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.250+0000 7f96add9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96b01acba0 0x7f96b01a67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:42056/0 (socket says 172.21.15.94:42056) 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.250+0000 7f96add9b640 1 -- 172.21.15.94:0/3237660814 learned_addr learned my addr 172.21.15.94:0/3237660814 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.250+0000 7f96ae59c640 1 --2- 172.21.15.94:0/3237660814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96b0106460 0x7f96b01ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.250+0000 7f96add9b640 1 -- 172.21.15.94:0/3237660814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96b01a6d00 msgr2=0x7f96b01a7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.250+0000 7f96add9b640 1 --2- 172.21.15.94:0/3237660814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96b01a6d00 0x7f96b01a7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.250+0000 7f96add9b640 1 -- 172.21.15.94:0/3237660814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96b0106460 msgr2=0x7f96b01ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.250+0000 7f96add9b640 1 --2- 172.21.15.94:0/3237660814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96b0106460 0x7f96b01ac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.250+0000 7f96add9b640 1 -- 172.21.15.94:0/3237660814 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a0009560 con 0x7f96b01acba0 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.251+0000 7f96add9b640 1 --2- 172.21.15.94:0/3237660814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96b01acba0 0x7f96b01a67c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f96a400b4a0 tx=0x7f96a400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.251+0000 7f969f7fe640 1 -- 172.21.15.94:0/3237660814 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4010610 con 0x7f96b01acba0 2024-02-17T12:05:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.251+0000 7f96b49f4640 1 -- 172.21.15.94:0/3237660814 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b01b3890 con 0x7f96b01acba0 2024-02-17T12:05:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.251+0000 7f969f7fe640 1 -- 172.21.15.94:0/3237660814 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96a4010c50 con 0x7f96b01acba0 2024-02-17T12:05:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.251+0000 7f96b49f4640 1 -- 172.21.15.94:0/3237660814 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b01b3da0 con 0x7f96b01acba0 2024-02-17T12:05:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.251+0000 7f969f7fe640 1 -- 172.21.15.94:0/3237660814 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4017e30 con 0x7f96b01acba0 2024-02-17T12:05:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.255+0000 7f96b75c9740 1 Processor -- start 2024-02-17T12:05:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.255+0000 7f969f7fe640 1 -- 172.21.15.94:0/3237660814 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f96a4010770 con 0x7f96b01acba0 2024-02-17T12:05:37.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.255+0000 7f96b75c9740 1 -- start start 2024-02-17T12:05:37.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.255+0000 7f96b75c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838250 0x55f5ba838630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.255+0000 7f96b75c9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838b70 0x55f5ba841210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.255+0000 7f96b75c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 0x55f5ba843b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.255+0000 7f96b75c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ba7f3960 con 0x55f5ba838250 2024-02-17T12:05:37.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.255+0000 7f96b75c9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ba7a7310 con 0x55f5ba838b70 2024-02-17T12:05:37.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.255+0000 7f96b75c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ba7a4840 con 0x55f5ba841750 2024-02-17T12:05:37.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.255+0000 7f96aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838250 0x55f5ba838630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.256+0000 7f96aed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838250 0x55f5ba838630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32822/0 (socket says 172.21.15.94:32822) 2024-02-17T12:05:37.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.256+0000 7f96aed9d640 1 -- 172.21.15.94:0/3432649033 learned_addr learned my addr 172.21.15.94:0/3432649033 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:37.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.256+0000 7f96add9b640 1 --2- 172.21.15.94:0/3432649033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 0x55f5ba843b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.256+0000 7f96aed9d640 1 -- 172.21.15.94:0/3432649033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 msgr2=0x55f5ba843b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.256+0000 7f96aed9d640 1 --2- 172.21.15.94:0/3432649033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 0x55f5ba843b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.256+0000 7f96aed9d640 1 -- 172.21.15.94:0/3432649033 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838b70 msgr2=0x55f5ba841210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:05:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.256+0000 7f96aed9d640 1 --2- 172.21.15.94:0/3432649033 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838b70 0x55f5ba841210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.256+0000 7f96aed9d640 1 -- 172.21.15.94:0/3432649033 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5ba762060 con 0x55f5ba838250 2024-02-17T12:05:37.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.256+0000 7f96aed9d640 1 --2- 172.21.15.94:0/3432649033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838250 0x55f5ba838630 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f96a8007c50 tx=0x7f96a800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292f77cf32083c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.257+0000 7f969cff9640 1 -- 172.21.15.94:0/3432649033 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a800f650 con 0x55f5ba838250 2024-02-17T12:05:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.257+0000 7f969cff9640 1 -- 172.21.15.94:0/3432649033 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96a800fc90 con 0x55f5ba838250 2024-02-17T12:05:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.257+0000 7f969cff9640 1 -- 172.21.15.94:0/3432649033 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a800ec40 con 0x55f5ba838250 2024-02-17T12:05:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.258+0000 7f969f7fe640 1 --2- 172.21.15.94:0/3237660814 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96980a22f0 0x7f96980a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.258+0000 7f969f7fe640 1 -- 172.21.15.94:0/3237660814 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f96a40cd620 con 0x7f96b01acba0 2024-02-17T12:05:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.258+0000 7f96b75c9740 1 -- 172.21.15.94:0/3432649033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838250 msgr2=0x55f5ba838630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.258+0000 7f96b75c9740 1 --2- 172.21.15.94:0/3432649033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838250 0x55f5ba838630 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f96a8007c50 tx=0x7f96a800c6a0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.258+0000 7f96ae59c640 1 --2- 172.21.15.94:0/3237660814 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96980a22f0 0x7f96980a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.259+0000 7f96b75c9740 1 -- 172.21.15.94:0/3432649033 shutdown_connections 2024-02-17T12:05:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.259+0000 7f96b75c9740 1 --2- 172.21.15.94:0/3432649033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 0x55f5ba843b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.259+0000 7f96b75c9740 1 --2- 172.21.15.94:0/3432649033 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838b70 0x55f5ba841210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.259+0000 7f96b75c9740 1 --2- 172.21.15.94:0/3432649033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838250 0x55f5ba838630 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.259+0000 7f96b75c9740 1 -- 172.21.15.94:0/3432649033 >> 172.21.15.94:0/3432649033 conn(0x55f5ba832980 msgr2=0x55f5ba832d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:37.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.259+0000 7f96ae59c640 1 --2- 172.21.15.94:0/3237660814 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96980a22f0 0x7f96980a47b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009230 tx=0x7f96a0000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:37.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.259+0000 7f96b75c9740 1 -- 172.21.15.94:0/3432649033 shutdown_connections 2024-02-17T12:05:37.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.259+0000 7f96b75c9740 1 -- 172.21.15.94:0/3432649033 wait complete. 2024-02-17T12:05:37.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.260+0000 7f96b75c9740 1 -- start start 2024-02-17T12:05:37.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.260+0000 7f96b75c9740 1 Processor -- start 2024-02-17T12:05:37.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.260+0000 7f96b75c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838250 0x55f5ba83e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.260+0000 7f96b75c9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838b70 0x55f5ba83ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.260+0000 7f96b75c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 0x55f5ba83f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.260+0000 7f96b75c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ba7a0020 con 0x55f5ba838250 2024-02-17T12:05:37.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.260+0000 7f96b75c9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ba473430 con 0x55f5ba838b70 2024-02-17T12:05:37.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.260+0000 7f96b75c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ba42fe00 con 0x55f5ba841750 2024-02-17T12:05:37.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.260+0000 7f96b75c9740 10 client.0 _ll_register_callbacks cb 0x55f5ba747cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:37.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.260+0000 7f96add9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838b70 0x55f5ba83ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.261+0000 7f96ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838250 0x55f5ba83e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.261+0000 7f96ae59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838250 0x55f5ba83e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32834/0 (socket says 172.21.15.94:32834) 2024-02-17T12:05:37.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.261+0000 7f96ae59c640 1 -- 172.21.15.94:0/698910456 learned_addr learned my addr 172.21.15.94:0/698910456 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:37.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.261+0000 7f96add9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838b70 0x55f5ba83ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56494/0 (socket says 172.21.15.94:56494) 2024-02-17T12:05:37.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.261+0000 7f96add9b640 1 -- 172.21.15.94:0/698910456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 msgr2=0x55f5ba83f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.261+0000 7f96add9b640 1 --2- 172.21.15.94:0/698910456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 0x55f5ba83f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.261+0000 7f96add9b640 1 -- 172.21.15.94:0/698910456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838250 msgr2=0x55f5ba83e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.261+0000 7f96add9b640 1 --2- 172.21.15.94:0/698910456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838250 0x55f5ba83e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.261+0000 7f96add9b640 1 -- 172.21.15.94:0/698910456 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a8007880 con 0x55f5ba838b70 2024-02-17T12:05:37.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.261+0000 7f96add9b640 1 --2- 172.21.15.94:0/698910456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838b70 0x55f5ba83ecf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f96a40cade0 tx=0x7f96a4099e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad2ecf2a038d1ef9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:37.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.261+0000 7f9695ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:05:37.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.262+0000 7f9695ffb640 1 -- 172.21.15.94:0/698910456 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a409a7b0 con 0x55f5ba838b70 2024-02-17T12:05:37.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.262+0000 7f96b75c9740 1 -- 172.21.15.94:0/698910456 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5ba49f7e0 con 0x55f5ba838b70 2024-02-17T12:05:37.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.262+0000 7f9695ffb640 1 -- 172.21.15.94:0/698910456 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f96a409a0b0 con 0x55f5ba838b70 2024-02-17T12:05:37.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.262+0000 7f9695ffb640 1 -- 172.21.15.94:0/698910456 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96a409e200 con 0x55f5ba838b70 2024-02-17T12:05:37.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.262+0000 7f96b75c9740 10 client.26954 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:37.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.262+0000 7f96b75c9740 1 -- 172.21.15.94:0/698910456 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f5ba79e6a0 con 0x55f5ba838b70 2024-02-17T12:05:37.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.262+0000 7f9695ffb640 1 -- 172.21.15.94:0/698910456 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96a40a2850 con 0x55f5ba838b70 2024-02-17T12:05:37.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.263+0000 7f9695ffb640 1 -- 172.21.15.94:0/698910456 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f96a4099500 con 0x55f5ba838b70 2024-02-17T12:05:37.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.263+0000 7f96b75c9740 10 client.26954 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:37.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.263+0000 7f96b75c9740 10 client.26954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:37.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.263+0000 7f96b75c9740 20 client.26954 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:37.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.263+0000 7f96b75c9740 1 --2- 172.21.15.94:0/698910456 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5ba851a60 0x55f5ba853ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.263+0000 7f96b75c9740 4 client.26954 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.263+0000 7f96b75c9740 1 -- 172.21.15.94:0/698910456 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5ba49f7e0 con 0x55f5ba851a60 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.263+0000 7f96ae59c640 1 --2- 172.21.15.94:0/698910456 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5ba851a60 0x55f5ba853ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.264+0000 7f96ae59c640 1 --2- 172.21.15.94:0/698910456 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5ba851a60 0x55f5ba853ee0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca370c711f4acb93 server_cookie=bb1decace6c5be22 in_seq=0 out_seq=0 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.264+0000 7f9695ffb640 10 client.26954 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.264+0000 7f9695ffb640 1 -- 172.21.15.94:0/698910456 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f5ba49f7e0 con 0x55f5ba851a60 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f9695ffb640 10 client.26954 handle_command_reply: tid=0 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f96b75c9740 1 client.26954 shutdown 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f96b75c9740 1 -- 172.21.15.94:0/698910456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838b70 msgr2=0x55f5ba83ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f96b75c9740 1 --2- 172.21.15.94:0/698910456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838b70 0x55f5ba83ecf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f96a40cade0 tx=0x7f96a4099e80 comp rx=0 tx=0).stop 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f96b75c9740 1 -- 172.21.15.94:0/698910456 shutdown_connections 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f96b75c9740 1 --2- 172.21.15.94:0/698910456 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5ba851a60 0x55f5ba853ee0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f96b75c9740 1 --2- 172.21.15.94:0/698910456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 0x55f5ba83f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f96b75c9740 1 --2- 172.21.15.94:0/698910456 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838b70 0x55f5ba83ecf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f96b75c9740 1 --2- 172.21.15.94:0/698910456 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838250 0x55f5ba83e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f96b75c9740 1 -- 172.21.15.94:0/698910456 >> 172.21.15.94:0/698910456 conn(0x55f5ba832980 msgr2=0x55f5ba8346d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:37.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f9695ffb640 0 client.26954 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:37.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f96b75c9740 1 -- 172.21.15.94:0/698910456 shutdown_connections 2024-02-17T12:05:37.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f96b75c9740 1 -- 172.21.15.94:0/698910456 wait complete. 2024-02-17T12:05:37.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.265+0000 7f96b75c9740 20 client.26954 trim_cache size 0 max 16384 2024-02-17T12:05:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96b75c9740 1 Processor -- start 2024-02-17T12:05:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96b75c9740 1 -- start start 2024-02-17T12:05:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96b75c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba838250 0x55f5ba840ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96b75c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838b70 0x55f5ba82da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96b75c9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba841750 0x55f5ba858880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96b75c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ba725130 con 0x55f5ba838b70 2024-02-17T12:05:37.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96b75c9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ba83f7b0 con 0x55f5ba841750 2024-02-17T12:05:37.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96b75c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ba83cd00 con 0x55f5ba838250 2024-02-17T12:05:37.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96aed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba841750 0x55f5ba858880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96aed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba841750 0x55f5ba858880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56498/0 (socket says 172.21.15.94:56498) 2024-02-17T12:05:37.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96aed9d640 1 -- 172.21.15.94:0/822181567 learned_addr learned my addr 172.21.15.94:0/822181567 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:37.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96ae59c640 1 --2- 172.21.15.94:0/822181567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba838250 0x55f5ba840ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96aed9d640 1 -- 172.21.15.94:0/822181567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba838250 msgr2=0x55f5ba840ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96aed9d640 1 --2- 172.21.15.94:0/822181567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba838250 0x55f5ba840ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.283+0000 7f96aed9d640 1 -- 172.21.15.94:0/822181567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838b70 msgr2=0x55f5ba82da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.284+0000 7f96aed9d640 1 --2- 172.21.15.94:0/822181567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838b70 0x55f5ba82da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.284+0000 7f96aed9d640 1 -- 172.21.15.94:0/822181567 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5ba73ac20 con 0x55f5ba841750 2024-02-17T12:05:37.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.284+0000 7f96aed9d640 1 --2- 172.21.15.94:0/822181567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba841750 0x55f5ba858880 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f96a80077a0 tx=0x7f96a80077d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e310150e5d6ffb17 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:37.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.284+0000 7f9697fff640 1 -- 172.21.15.94:0/822181567 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a801e070 con 0x55f5ba841750 2024-02-17T12:05:37.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.284+0000 7f9697fff640 1 -- 172.21.15.94:0/822181567 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96a80074e0 con 0x55f5ba841750 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.284+0000 7f9697fff640 1 -- 172.21.15.94:0/822181567 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8023be0 con 0x55f5ba841750 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.285+0000 7f96b75c9740 1 -- 172.21.15.94:0/822181567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba841750 msgr2=0x55f5ba858880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.285+0000 7f96b75c9740 1 --2- 172.21.15.94:0/822181567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba841750 0x55f5ba858880 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f96a80077a0 tx=0x7f96a80077d0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.285+0000 7f96b75c9740 1 -- 172.21.15.94:0/822181567 shutdown_connections 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.285+0000 7f96b75c9740 1 --2- 172.21.15.94:0/822181567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba841750 0x55f5ba858880 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.285+0000 7f96b75c9740 1 --2- 172.21.15.94:0/822181567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838b70 0x55f5ba82da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.285+0000 7f96b75c9740 1 --2- 172.21.15.94:0/822181567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba838250 0x55f5ba840ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.285+0000 7f96b75c9740 1 -- 172.21.15.94:0/822181567 >> 172.21.15.94:0/822181567 conn(0x55f5ba832980 msgr2=0x55f5ba832d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.285+0000 7f96b75c9740 1 -- 172.21.15.94:0/822181567 shutdown_connections 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.285+0000 7f96b75c9740 1 -- 172.21.15.94:0/822181567 wait complete. 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.286+0000 7f96b75c9740 1 -- start start 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.286+0000 7f96b75c9740 1 Processor -- start 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.286+0000 7f96b75c9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838250 0x55f5ba848a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.286+0000 7f96b75c9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838b70 0x55f5ba846da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.286+0000 7f96b75c9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 0x55f5ba847300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.286+0000 7f96b75c9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ba847a00 con 0x55f5ba838b70 2024-02-17T12:05:37.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.286+0000 7f96b75c9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ba847b70 con 0x55f5ba838250 2024-02-17T12:05:37.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.286+0000 7f96b75c9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5ba8b9020 con 0x55f5ba841750 2024-02-17T12:05:37.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.286+0000 7f96ae59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838250 0x55f5ba848a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.286+0000 7f96add9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838b70 0x55f5ba846da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.286+0000 7f96b75c9740 10 client.0 _ll_register_callbacks cb 0x7f96a8007400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:37.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.286+0000 7f96ae59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838250 0x55f5ba848a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56508/0 (socket says 172.21.15.94:56508) 2024-02-17T12:05:37.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.286+0000 7f96ae59c640 1 -- 172.21.15.94:0/2844646296 learned_addr learned my addr 172.21.15.94:0/2844646296 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:37.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.287+0000 7f96aed9d640 1 --2- 172.21.15.94:0/2844646296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 0x55f5ba847300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.287+0000 7f96ae59c640 1 -- 172.21.15.94:0/2844646296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 msgr2=0x55f5ba847300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.287+0000 7f96ae59c640 1 --2- 172.21.15.94:0/2844646296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 0x55f5ba847300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.287+0000 7f96ae59c640 1 -- 172.21.15.94:0/2844646296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838b70 msgr2=0x55f5ba846da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:37.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.287+0000 7f96ae59c640 1 --2- 172.21.15.94:0/2844646296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838b70 0x55f5ba846da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.287+0000 7f96ae59c640 1 -- 172.21.15.94:0/2844646296 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5ba762060 con 0x55f5ba838250 2024-02-17T12:05:37.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.287+0000 7f96add9b640 1 --2- 172.21.15.94:0/2844646296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838b70 0x55f5ba846da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:37.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.287+0000 7f96ae59c640 1 --2- 172.21.15.94:0/2844646296 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838250 0x55f5ba848a00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f96a002c820 tx=0x7f96a002bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35c63b5bfbadf70d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:37.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.287+0000 7f9695ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:05:37.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.287+0000 7f9695ffb640 1 -- 172.21.15.94:0/2844646296 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a0054070 con 0x55f5ba838250 2024-02-17T12:05:37.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.287+0000 7f9695ffb640 1 -- 172.21.15.94:0/2844646296 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f96a0047040 con 0x55f5ba838250 2024-02-17T12:05:37.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.287+0000 7f96b75c9740 1 -- 172.21.15.94:0/2844646296 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96a800fc90 con 0x55f5ba838250 2024-02-17T12:05:37.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.287+0000 7f9695ffb640 1 -- 172.21.15.94:0/2844646296 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96a00043d0 con 0x55f5ba838250 2024-02-17T12:05:37.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.289+0000 7f96b75c9740 10 client.26960 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:37.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.289+0000 7f96b75c9740 1 -- 172.21.15.94:0/2844646296 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f5ba7608c0 con 0x55f5ba838250 2024-02-17T12:05:37.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.289+0000 7f9695ffb640 1 -- 172.21.15.94:0/2844646296 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f96a800fc90 con 0x55f5ba838250 2024-02-17T12:05:37.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.289+0000 7f9695ffb640 1 -- 172.21.15.94:0/2844646296 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f96a00038c0 con 0x55f5ba838250 2024-02-17T12:05:37.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.289+0000 7f96b75c9740 10 client.26960 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:37.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.289+0000 7f96b75c9740 10 client.26960 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:37.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.289+0000 7f96b75c9740 20 client.26960 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:37.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.289+0000 7f96b75c9740 1 --2- 172.21.15.94:0/2844646296 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5ba8ba720 0x55f5ba8bcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:37.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.289+0000 7f96b75c9740 4 client.26960 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:05:37.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.289+0000 7f96b75c9740 1 -- 172.21.15.94:0/2844646296 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f5ba4dc720 con 0x55f5ba8ba720 2024-02-17T12:05:37.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.289+0000 7f96add9b640 1 --2- 172.21.15.94:0/2844646296 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5ba8ba720 0x55f5ba8bcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:37.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f96add9b640 1 --2- 172.21.15.94:0/2844646296 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5ba8ba720 0x55f5ba8bcb40 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1fe8b7134f1e56e server_cookie=9ef64c537c43393c in_seq=0 out_seq=0 2024-02-17T12:05:37.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f9695ffb640 10 client.26960 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:37.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f9695ffb640 1 -- 172.21.15.94:0/2844646296 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f5ba4dc720 con 0x55f5ba8ba720 2024-02-17T12:05:37.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f9695ffb640 10 client.26960 handle_command_reply: tid=0 2024-02-17T12:05:37.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f96b75c9740 1 client.26960 shutdown 2024-02-17T12:05:37.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f96b75c9740 1 -- 172.21.15.94:0/2844646296 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838250 msgr2=0x55f5ba848a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f96b75c9740 1 --2- 172.21.15.94:0/2844646296 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838250 0x55f5ba848a00 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f96a002c820 tx=0x7f96a002bc50 comp rx=0 tx=0).stop 2024-02-17T12:05:37.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f96b75c9740 1 -- 172.21.15.94:0/2844646296 shutdown_connections 2024-02-17T12:05:37.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f96b75c9740 1 --2- 172.21.15.94:0/2844646296 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5ba8ba720 0x55f5ba8bcb40 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f96b75c9740 1 --2- 172.21.15.94:0/2844646296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5ba841750 0x55f5ba847300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f9695ffb640 0 client.26960 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:37.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f96b75c9740 1 --2- 172.21.15.94:0/2844646296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5ba838b70 0x55f5ba846da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f96b75c9740 1 --2- 172.21.15.94:0/2844646296 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5ba838250 0x55f5ba848a00 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f96b75c9740 1 -- 172.21.15.94:0/2844646296 >> 172.21.15.94:0/2844646296 conn(0x55f5ba832980 msgr2=0x55f5ba858880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:37.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f96b75c9740 1 -- 172.21.15.94:0/2844646296 shutdown_connections 2024-02-17T12:05:37.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f96b75c9740 1 -- 172.21.15.94:0/2844646296 wait complete. 2024-02-17T12:05:37.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.290+0000 7f96b75c9740 20 client.26960 trim_cache size 0 max 16384 2024-02-17T12:05:37.296 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:05:37.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.291+0000 7f96b49f4640 1 -- 172.21.15.94:0/3237660814 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96980a22f0 msgr2=0x7f96980a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.291+0000 7f96b49f4640 1 --2- 172.21.15.94:0/3237660814 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96980a22f0 0x7f96980a47b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009230 tx=0x7f96a0000bc0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.291+0000 7f96b49f4640 1 -- 172.21.15.94:0/3237660814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96b01acba0 msgr2=0x7f96b01a67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:37.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.291+0000 7f96b49f4640 1 --2- 172.21.15.94:0/3237660814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96b01acba0 0x7f96b01a67c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f96a400b4a0 tx=0x7f96a400b960 comp rx=0 tx=0).stop 2024-02-17T12:05:37.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.291+0000 7f96b49f4640 1 -- 172.21.15.94:0/3237660814 shutdown_connections 2024-02-17T12:05:37.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.291+0000 7f96b49f4640 1 --2- 172.21.15.94:0/3237660814 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f96980a22f0 0x7f96980a47b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.291+0000 7f96b49f4640 1 --2- 172.21.15.94:0/3237660814 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96b01a6d00 0x7f96b01a7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.291+0000 7f96b49f4640 1 --2- 172.21.15.94:0/3237660814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96b01acba0 0x7f96b01a67c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.291+0000 7f96b49f4640 1 --2- 172.21.15.94:0/3237660814 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f96b0106460 0x7f96b01ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:37.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.291+0000 7f96b49f4640 1 -- 172.21.15.94:0/3237660814 >> 172.21.15.94:0/3237660814 conn(0x7f96b00fcc10 msgr2=0x7f96b0074e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:37.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.291+0000 7f96b49f4640 1 -- 172.21.15.94:0/3237660814 shutdown_connections 2024-02-17T12:05:37.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:37.291+0000 7f96b49f4640 1 -- 172.21.15.94:0/3237660814 wait complete. 2024-02-17T12:05:37.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:36 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/28411952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:05:37.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:36 smithi189 ceph-mon[36588]: pgmap v230: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.1 MiB/s wr, 692 op/s 2024-02-17T12:05:37.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:36 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/28411952' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:05:37.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:36 smithi125 ceph-mon[36819]: pgmap v230: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.1 MiB/s wr, 692 op/s 2024-02-17T12:05:38.308 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:05:38.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:05:38.309 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:05:38.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.451+0000 7f7ff6064640 1 -- 172.21.15.94:0/795350163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ff0100040 msgr2=0x7f7ff01004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.451+0000 7f7ff6064640 1 --2- 172.21.15.94:0/795350163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ff0100040 0x7f7ff01004a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009860 tx=0x7f7fe002bc00 comp rx=0 tx=0).stop 2024-02-17T12:05:38.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.453+0000 7f7ff6064640 1 -- 172.21.15.94:0/795350163 shutdown_connections 2024-02-17T12:05:38.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.453+0000 7f7ff6064640 1 --2- 172.21.15.94:0/795350163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ff01009e0 0x7f7ff010cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.453+0000 7f7ff6064640 1 --2- 172.21.15.94:0/795350163 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ff0100040 0x7f7ff01004a0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.453+0000 7f7ff6064640 1 --2- 172.21.15.94:0/795350163 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ff0106100 0x7f7ff01064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.453+0000 7f7ff6064640 1 -- 172.21.15.94:0/795350163 >> 172.21.15.94:0/795350163 conn(0x7f7ff0077110 msgr2=0x7f7ff0077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:38.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.461+0000 7f7ff6064640 1 -- 172.21.15.94:0/795350163 shutdown_connections 2024-02-17T12:05:38.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.461+0000 7f7ff6064640 1 -- 172.21.15.94:0/795350163 wait complete. 2024-02-17T12:05:38.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.462+0000 7f7ff6064640 1 Processor -- start 2024-02-17T12:05:38.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.462+0000 7f7ff6064640 1 -- start start 2024-02-17T12:05:38.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.462+0000 7f7ff6064640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ff0100040 0x7f7ff01929c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.462+0000 7f7ff6064640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ff01009e0 0x7f7ff0192f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.462+0000 7f7ff6064640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ff0106100 0x7f7ff006d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.462+0000 7f7ff6064640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff006dea0 con 0x7f7ff0100040 2024-02-17T12:05:38.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.462+0000 7f7ff6064640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff006e010 con 0x7f7ff0106100 2024-02-17T12:05:38.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.462+0000 7f7ff6064640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff006e1d0 con 0x7f7ff01009e0 2024-02-17T12:05:38.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.463+0000 7f7feffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ff0106100 0x7f7ff006d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.463+0000 7f7feffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ff0106100 0x7f7ff006d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56524/0 (socket says 172.21.15.94:56524) 2024-02-17T12:05:38.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.463+0000 7f7feffff640 1 -- 172.21.15.94:0/1417933104 learned_addr learned my addr 172.21.15.94:0/1417933104 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:38.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.463+0000 7f7fef7fe640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ff0100040 0x7f7ff01929c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.464+0000 7f7feeffd640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ff01009e0 0x7f7ff0192f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.465+0000 7f7feffff640 1 -- 172.21.15.94:0/1417933104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ff01009e0 msgr2=0x7f7ff0192f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.465+0000 7f7feffff640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ff01009e0 0x7f7ff0192f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.465+0000 7f7feffff640 1 -- 172.21.15.94:0/1417933104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ff0100040 msgr2=0x7f7ff01929c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.465+0000 7f7feffff640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ff0100040 0x7f7ff01929c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.465+0000 7f7feffff640 1 -- 172.21.15.94:0/1417933104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe4009490 con 0x7f7ff0106100 2024-02-17T12:05:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.465+0000 7f7feffff640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ff0106100 0x7f7ff006d850 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800cc30 tx=0x7f7fe800a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.466+0000 7f7fecff9640 1 -- 172.21.15.94:0/1417933104 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8003b70 con 0x7f7ff0106100 2024-02-17T12:05:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.466+0000 7f7fecff9640 1 -- 172.21.15.94:0/1417933104 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fe800ae50 con 0x7f7ff0106100 2024-02-17T12:05:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.466+0000 7f7fecff9640 1 -- 172.21.15.94:0/1417933104 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe800f870 con 0x7f7ff0106100 2024-02-17T12:05:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.466+0000 7f7ff6064640 1 -- 172.21.15.94:0/1417933104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe0009490 con 0x7f7ff0106100 2024-02-17T12:05:38.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.466+0000 7f7ff6064640 1 -- 172.21.15.94:0/1417933104 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff01bb870 con 0x7f7ff0106100 2024-02-17T12:05:38.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.470+0000 7f7fecff9640 1 -- 172.21.15.94:0/1417933104 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7fe800f9d0 con 0x7f7ff0106100 2024-02-17T12:05:38.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.470+0000 7f7ff8c39740 1 Processor -- start 2024-02-17T12:05:38.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.470+0000 7f7ff8c39740 1 -- start start 2024-02-17T12:05:38.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.472+0000 7f7fecff9640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7fd80a2460 0x7f7fd80a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.473+0000 7f7fecff9640 1 -- 172.21.15.94:0/1417933104 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7fe80ce550 con 0x7f7ff0106100 2024-02-17T12:05:38.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.474+0000 7f7ff8c39740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b09b90 0x55f518b09f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.474+0000 7f7ff8c39740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b0a540 0x55f518b12af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.474+0000 7f7ff8c39740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b130c0 0x55f518b15510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.474+0000 7f7ff8c39740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f518a6ff90 con 0x55f518b09b90 2024-02-17T12:05:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.474+0000 7f7ff8c39740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f518a6d4c0 con 0x55f518b0a540 2024-02-17T12:05:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.474+0000 7f7ff8c39740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f518a68ca0 con 0x55f518b130c0 2024-02-17T12:05:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.474+0000 7f7feeffd640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7fd80a2460 0x7f7fd80a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.474+0000 7f7feffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b130c0 0x55f518b15510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.474+0000 7f7feffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b130c0 0x55f518b15510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:42140/0 (socket says 172.21.15.94:42140) 2024-02-17T12:05:38.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.474+0000 7f7feffff640 1 -- 172.21.15.94:0/746799550 learned_addr learned my addr 172.21.15.94:0/746799550 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:38.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.474+0000 7f7fef7fe640 1 --2- 172.21.15.94:0/746799550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b09b90 0x55f518b09f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.475+0000 7f7feeffd640 1 --2- 172.21.15.94:0/746799550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b0a540 0x55f518b12af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.475+0000 7f7fef7fe640 1 -- 172.21.15.94:0/746799550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b130c0 msgr2=0x55f518b15510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.475+0000 7f7fef7fe640 1 --2- 172.21.15.94:0/746799550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b130c0 0x55f518b15510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.475+0000 7f7fef7fe640 1 -- 172.21.15.94:0/746799550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b0a540 msgr2=0x55f518b12af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.475+0000 7f7fef7fe640 1 --2- 172.21.15.94:0/746799550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b0a540 0x55f518b12af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.475+0000 7f7fef7fe640 1 -- 172.21.15.94:0/746799550 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f518a2b0b0 con 0x55f518b09b90 2024-02-17T12:05:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.475+0000 7f7fef7fe640 1 --2- 172.21.15.94:0/746799550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b09b90 0x55f518b09f70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009d00 tx=0x7f7fe40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883fc34e8361f77e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.476+0000 7f7feeffd640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7fd80a2460 0x7f7fd80a4920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0037040 tx=0x7f7fe0004060 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.476+0000 7f7fd5ffb640 1 -- 172.21.15.94:0/746799550 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe4012450 con 0x55f518b09b90 2024-02-17T12:05:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.476+0000 7f7fd5ffb640 1 -- 172.21.15.94:0/746799550 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fe4012a90 con 0x55f518b09b90 2024-02-17T12:05:38.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.476+0000 7f7fd5ffb640 1 -- 172.21.15.94:0/746799550 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400fca0 con 0x55f518b09b90 2024-02-17T12:05:38.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.477+0000 7f7ff8c39740 1 -- 172.21.15.94:0/746799550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b09b90 msgr2=0x55f518b09f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.477+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/746799550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b09b90 0x55f518b09f70 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7fe4009d00 tx=0x7f7fe40117b0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.478+0000 7f7ff8c39740 1 -- 172.21.15.94:0/746799550 shutdown_connections 2024-02-17T12:05:38.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.478+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/746799550 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b130c0 0x55f518b15510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.478+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/746799550 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b0a540 0x55f518b12af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.478+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/746799550 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b09b90 0x55f518b09f70 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.478+0000 7f7ff8c39740 1 -- 172.21.15.94:0/746799550 >> 172.21.15.94:0/746799550 conn(0x55f518b04050 msgr2=0x55f518b04430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:38.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.478+0000 7f7ff8c39740 1 -- 172.21.15.94:0/746799550 shutdown_connections 2024-02-17T12:05:38.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.478+0000 7f7ff8c39740 1 -- 172.21.15.94:0/746799550 wait complete. 2024-02-17T12:05:38.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.479+0000 7f7ff8c39740 1 -- start start 2024-02-17T12:05:38.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.479+0000 7f7ff8c39740 1 Processor -- start 2024-02-17T12:05:38.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.479+0000 7f7ff8c39740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b09b90 0x55f518b10060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7ff8c39740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b0a540 0x55f518b105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7ff8c39740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b130c0 0x55f518b10ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b09b90 0x55f518b10060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b09b90 0x55f518b10060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32870/0 (socket says 172.21.15.94:32870) 2024-02-17T12:05:38.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7fef7fe640 1 -- 172.21.15.94:0/2322815351 learned_addr learned my addr 172.21.15.94:0/2322815351 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:38.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7ff8c39740 1 -- 172.21.15.94:0/2322815351 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f518a0df40 con 0x55f518b09b90 2024-02-17T12:05:38.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7ff8c39740 1 -- 172.21.15.94:0/2322815351 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f51873c4b0 con 0x55f518b130c0 2024-02-17T12:05:38.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7feeffd640 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b0a540 0x55f518b105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7ff8c39740 1 -- 172.21.15.94:0/2322815351 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5189ee670 con 0x55f518b0a540 2024-02-17T12:05:38.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7feffff640 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b130c0 0x55f518b10ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7ff8c39740 10 client.0 _ll_register_callbacks cb 0x55f518a5bf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:38.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7feffff640 1 -- 172.21.15.94:0/2322815351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b0a540 msgr2=0x55f518b105a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7feffff640 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b0a540 0x55f518b105a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7feffff640 1 -- 172.21.15.94:0/2322815351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b09b90 msgr2=0x55f518b10060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7feffff640 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b09b90 0x55f518b10060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7feffff640 1 -- 172.21.15.94:0/2322815351 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe4009930 con 0x55f518b130c0 2024-02-17T12:05:38.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.480+0000 7f7fef7fe640 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b09b90 0x55f518b10060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:05:38.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.481+0000 7f7feffff640 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b130c0 0x55f518b10ae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7fe809ffd0 tx=0x7f7fe809b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=983a881cf4949713 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:38.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.481+0000 7f7fceffd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:05:38.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.481+0000 7f7fceffd640 1 -- 172.21.15.94:0/2322815351 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80cc0b0 con 0x55f518b130c0 2024-02-17T12:05:38.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.481+0000 7f7fceffd640 1 -- 172.21.15.94:0/2322815351 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7fe801a0d0 con 0x55f518b130c0 2024-02-17T12:05:38.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.481+0000 7f7ff8c39740 1 -- 172.21.15.94:0/2322815351 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f518a42f40 con 0x55f518b130c0 2024-02-17T12:05:38.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.482+0000 7f7fceffd640 1 -- 172.21.15.94:0/2322815351 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fe809b1c0 con 0x55f518b130c0 2024-02-17T12:05:38.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.482+0000 7f7ff8c39740 10 client.26969 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:38.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.482+0000 7f7ff8c39740 1 -- 172.21.15.94:0/2322815351 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f518a67320 con 0x55f518b130c0 2024-02-17T12:05:38.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.483+0000 7f7fceffd640 1 -- 172.21.15.94:0/2322815351 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fe80a7080 con 0x55f518b130c0 2024-02-17T12:05:38.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.483+0000 7f7fceffd640 1 -- 172.21.15.94:0/2322815351 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7fe8016030 con 0x55f518b130c0 2024-02-17T12:05:38.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.483+0000 7f7ff8c39740 10 client.26969 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:38.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.483+0000 7f7ff8c39740 10 client.26969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:38.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.483+0000 7f7ff8c39740 20 client.26969 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:38.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.483+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f518b234f0 0x55f518b25900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.483+0000 7f7ff8c39740 4 client.26969 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:38.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.483+0000 7f7ff8c39740 1 -- 172.21.15.94:0/2322815351 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f518a42f40 con 0x55f518b234f0 2024-02-17T12:05:38.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.484+0000 7f7fef7fe640 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f518b234f0 0x55f518b25900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.484+0000 7f7fceffd640 10 client.26969 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:38.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.484+0000 7f7fef7fe640 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f518b234f0 0x55f518b25900 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbd8fe1f16c7059b server_cookie=245efdc99f54215a in_seq=0 out_seq=0 2024-02-17T12:05:38.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7fceffd640 1 -- 172.21.15.94:0/2322815351 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f518a42f40 con 0x55f518b234f0 2024-02-17T12:05:38.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7fceffd640 10 client.26969 handle_command_reply: tid=0 2024-02-17T12:05:38.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7ff8c39740 1 client.26969 shutdown 2024-02-17T12:05:38.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7ff8c39740 1 -- 172.21.15.94:0/2322815351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b130c0 msgr2=0x55f518b10ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b130c0 0x55f518b10ae0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7fe809ffd0 tx=0x7f7fe809b7a0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7ff8c39740 1 -- 172.21.15.94:0/2322815351 shutdown_connections 2024-02-17T12:05:38.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f518b234f0 0x55f518b25900 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b130c0 0x55f518b10ae0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b0a540 0x55f518b105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7fceffd640 0 client.26969 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:38.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/2322815351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b09b90 0x55f518b10060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7ff8c39740 1 -- 172.21.15.94:0/2322815351 >> 172.21.15.94:0/2322815351 conn(0x55f518b04050 msgr2=0x55f518b05fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:38.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7ff8c39740 1 -- 172.21.15.94:0/2322815351 shutdown_connections 2024-02-17T12:05:38.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7ff8c39740 1 -- 172.21.15.94:0/2322815351 wait complete. 2024-02-17T12:05:38.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.485+0000 7f7ff8c39740 20 client.26969 trim_cache size 0 max 16384 2024-02-17T12:05:38.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.507+0000 7f7ff8c39740 1 Processor -- start 2024-02-17T12:05:38.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.507+0000 7f7ff8c39740 1 -- start start 2024-02-17T12:05:38.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.507+0000 7f7ff8c39740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b09b90 0x55f518b12790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7ff8c39740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 0x55f518b10900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7ff8c39740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b130c0 0x55f518b0d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7ff8c39740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f518af5da0 con 0x55f518b0a540 2024-02-17T12:05:38.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7ff8c39740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f518b1f000 con 0x55f518b09b90 2024-02-17T12:05:38.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7ff8c39740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f518b08f00 con 0x55f518b130c0 2024-02-17T12:05:38.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7feeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 0x55f518b10900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7feeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 0x55f518b10900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32882/0 (socket says 172.21.15.94:32882) 2024-02-17T12:05:38.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7feeffd640 1 -- 172.21.15.94:0/1525590785 learned_addr learned my addr 172.21.15.94:0/1525590785 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:38.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7feffff640 1 --2- 172.21.15.94:0/1525590785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b130c0 0x55f518b0d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7fef7fe640 1 --2- 172.21.15.94:0/1525590785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b09b90 0x55f518b12790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7feeffd640 1 -- 172.21.15.94:0/1525590785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b130c0 msgr2=0x55f518b0d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7feeffd640 1 --2- 172.21.15.94:0/1525590785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b130c0 0x55f518b0d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7feeffd640 1 -- 172.21.15.94:0/1525590785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b09b90 msgr2=0x55f518b12790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7feeffd640 1 --2- 172.21.15.94:0/1525590785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b09b90 0x55f518b12790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7feeffd640 1 -- 172.21.15.94:0/1525590785 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f518a03c00 con 0x55f518b0a540 2024-02-17T12:05:38.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7fef7fe640 1 --2- 172.21.15.94:0/1525590785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b09b90 0x55f518b12790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:05:38.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7feeffd640 1 --2- 172.21.15.94:0/1525590785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 0x55f518b10900 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009830 tx=0x7f7fe003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0df3554b8d25a21 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:38.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.508+0000 7f7fd57fa640 1 -- 172.21.15.94:0/1525590785 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe00043a0 con 0x55f518b0a540 2024-02-17T12:05:38.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.509+0000 7f7fd57fa640 1 -- 172.21.15.94:0/1525590785 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fe0004500 con 0x55f518b0a540 2024-02-17T12:05:38.511 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.509+0000 7f7fd57fa640 1 -- 172.21.15.94:0/1525590785 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe002c800 con 0x55f518b0a540 2024-02-17T12:05:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.509+0000 7f7ff8c39740 1 -- 172.21.15.94:0/1525590785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 msgr2=0x55f518b10900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.509+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/1525590785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 0x55f518b10900 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009830 tx=0x7f7fe003d000 comp rx=0 tx=0).stop 2024-02-17T12:05:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.509+0000 7f7ff8c39740 1 -- 172.21.15.94:0/1525590785 shutdown_connections 2024-02-17T12:05:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.509+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/1525590785 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b130c0 0x55f518b0d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.509+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/1525590785 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 0x55f518b10900 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.509+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/1525590785 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b09b90 0x55f518b12790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.509+0000 7f7ff8c39740 1 -- 172.21.15.94:0/1525590785 >> 172.21.15.94:0/1525590785 conn(0x55f518b04050 msgr2=0x55f518b0e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.509+0000 7f7ff8c39740 1 -- 172.21.15.94:0/1525590785 shutdown_connections 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.509+0000 7f7ff8c39740 1 -- 172.21.15.94:0/1525590785 wait complete. 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7ff8c39740 1 -- start start 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7ff8c39740 1 Processor -- start 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7ff8c39740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b09b90 0x55f518b19320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7ff8c39740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 0x55f518b19860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7ff8c39740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b130c0 0x55f518b739c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7ff8c39740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f518b73f00 con 0x55f518b0a540 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7ff8c39740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f518b74070 con 0x55f518b130c0 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7ff8c39740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f518b74230 con 0x55f518b09b90 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7ff8c39740 10 client.0 _ll_register_callbacks cb 0x7f7fe4009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7feeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 0x55f518b19860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7feeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 0x55f518b19860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32898/0 (socket says 172.21.15.94:32898) 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7feeffd640 1 -- 172.21.15.94:0/4038081307 learned_addr learned my addr 172.21.15.94:0/4038081307 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7feffff640 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b130c0 0x55f518b739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7fef7fe640 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b09b90 0x55f518b19320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7feeffd640 1 -- 172.21.15.94:0/4038081307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b09b90 msgr2=0x55f518b19320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7feeffd640 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b09b90 0x55f518b19320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7feeffd640 1 -- 172.21.15.94:0/4038081307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b130c0 msgr2=0x55f518b739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7feeffd640 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b130c0 0x55f518b739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.510+0000 7f7feeffd640 1 -- 172.21.15.94:0/4038081307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe0046040 con 0x55f518b0a540 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.511+0000 7f7feffff640 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b130c0 0x55f518b739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.511+0000 7f7feeffd640 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 0x55f518b19860 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009860 tx=0x7f7fe0034960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b75e04e621292e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.511+0000 7f7fceffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.511+0000 7f7fceffd640 1 -- 172.21.15.94:0/4038081307 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe0005240 con 0x55f518b0a540 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.511+0000 7f7fceffd640 1 -- 172.21.15.94:0/4038081307 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f7fe00053a0 con 0x55f518b0a540 2024-02-17T12:05:38.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.512+0000 7f7ff8c39740 1 -- 172.21.15.94:0/4038081307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f518a5f310 con 0x55f518b0a540 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.512+0000 7f7fceffd640 1 -- 172.21.15.94:0/4038081307 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fe0005560 con 0x55f518b0a540 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.512+0000 7f7ff8c39740 10 client.17499 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.512+0000 7f7ff8c39740 1 -- 172.21.15.94:0/4038081307 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f518a29910 con 0x55f518b0a540 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.512+0000 7f7fceffd640 1 -- 172.21.15.94:0/4038081307 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fe002ca80 con 0x55f518b0a540 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.512+0000 7f7fceffd640 1 -- 172.21.15.94:0/4038081307 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7fe0004cd0 con 0x55f518b0a540 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.512+0000 7f7ff8c39740 10 client.17499 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.512+0000 7f7ff8c39740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.512+0000 7f7ff8c39740 20 client.17499 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.512+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f518b7b870 0x55f518b7dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.512+0000 7f7ff8c39740 4 client.17499 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.512+0000 7f7ff8c39740 1 -- 172.21.15.94:0/4038081307 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f518a5f310 con 0x55f518b7b870 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.513+0000 7f7fef7fe640 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f518b7b870 0x55f518b7dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.513+0000 7f7fceffd640 10 client.17499 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:38.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.513+0000 7f7fef7fe640 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f518b7b870 0x55f518b7dc90 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dfa514ba1f678e8 server_cookie=5e461de7a193c45d in_seq=0 out_seq=0 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.514+0000 7f7fceffd640 1 -- 172.21.15.94:0/4038081307 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f518a5f310 con 0x55f518b7b870 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.514+0000 7f7fceffd640 10 client.17499 handle_command_reply: tid=0 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.514+0000 7f7ff8c39740 1 client.17499 shutdown 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.514+0000 7f7ff8c39740 1 -- 172.21.15.94:0/4038081307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 msgr2=0x55f518b19860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.514+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 0x55f518b19860 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009860 tx=0x7f7fe0034960 comp rx=0 tx=0).stop 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.514+0000 7f7ff8c39740 1 -- 172.21.15.94:0/4038081307 shutdown_connections 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.514+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f518b7b870 0x55f518b7dc90 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.514+0000 7f7fceffd640 0 client.17499 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.514+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f518b130c0 0x55f518b739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.514+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f518b0a540 0x55f518b19860 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.514+0000 7f7ff8c39740 1 --2- 172.21.15.94:0/4038081307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f518b09b90 0x55f518b19320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.514+0000 7f7ff8c39740 1 -- 172.21.15.94:0/4038081307 >> 172.21.15.94:0/4038081307 conn(0x55f518b04050 msgr2=0x55f518afa050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.515+0000 7f7ff8c39740 1 -- 172.21.15.94:0/4038081307 shutdown_connections 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.515+0000 7f7ff8c39740 1 -- 172.21.15.94:0/4038081307 wait complete. 2024-02-17T12:05:38.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.515+0000 7f7ff8c39740 20 client.17499 trim_cache size 0 max 16384 2024-02-17T12:05:38.520 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:05:38.520 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:05:38.520 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "4be090d3-12b5-4b72-8fa4-2af4015eaed6", 2024-02-17T12:05:38.520 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:05:38.520 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:05:38.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.516+0000 7f7ff6064640 1 -- 172.21.15.94:0/1417933104 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7fd80a2460 msgr2=0x7f7fd80a4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.516+0000 7f7ff6064640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7fd80a2460 0x7f7fd80a4920 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0037040 tx=0x7f7fe0004060 comp rx=0 tx=0).stop 2024-02-17T12:05:38.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.516+0000 7f7ff6064640 1 -- 172.21.15.94:0/1417933104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ff0106100 msgr2=0x7f7ff006d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.516+0000 7f7ff6064640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ff0106100 0x7f7ff006d850 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800cc30 tx=0x7f7fe800a620 comp rx=0 tx=0).stop 2024-02-17T12:05:38.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.516+0000 7f7ff6064640 1 -- 172.21.15.94:0/1417933104 shutdown_connections 2024-02-17T12:05:38.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.516+0000 7f7ff6064640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7fd80a2460 0x7f7fd80a4920 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.516+0000 7f7ff6064640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7ff0106100 0x7f7ff006d850 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.516+0000 7f7ff6064640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ff01009e0 0x7f7ff0192f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.516+0000 7f7ff6064640 1 --2- 172.21.15.94:0/1417933104 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7ff0100040 0x7f7ff01929c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.516+0000 7f7ff6064640 1 -- 172.21.15.94:0/1417933104 >> 172.21.15.94:0/1417933104 conn(0x7f7ff0077110 msgr2=0x7f7ff00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:38.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.516+0000 7f7ff6064640 1 -- 172.21.15.94:0/1417933104 shutdown_connections 2024-02-17T12:05:38.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.516+0000 7f7ff6064640 1 -- 172.21.15.94:0/1417933104 wait complete. 2024-02-17T12:05:38.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:05:38.531 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:05:38.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.694+0000 7f347f0a3640 1 -- 172.21.15.94:0/474844740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3478100ab0 msgr2=0x7f3478104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.694+0000 7f347f0a3640 1 --2- 172.21.15.94:0/474844740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3478100ab0 0x7f3478104f70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3470009be0 tx=0x7f347002bfc0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.695+0000 7f347f0a3640 1 -- 172.21.15.94:0/474844740 shutdown_connections 2024-02-17T12:05:38.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.695+0000 7f347f0a3640 1 --2- 172.21.15.94:0/474844740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34781054b0 0x7f3478105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.695+0000 7f347f0a3640 1 --2- 172.21.15.94:0/474844740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3478100ab0 0x7f3478104f70 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.695+0000 7f347f0a3640 1 --2- 172.21.15.94:0/474844740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3478100190 0x7f3478100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.695+0000 7f347f0a3640 1 -- 172.21.15.94:0/474844740 >> 172.21.15.94:0/474844740 conn(0x7f3478077230 msgr2=0x7f3478077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:38.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.699+0000 7f347f0a3640 1 -- 172.21.15.94:0/474844740 shutdown_connections 2024-02-17T12:05:38.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.701+0000 7f347f0a3640 1 -- 172.21.15.94:0/474844740 wait complete. 2024-02-17T12:05:38.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.702+0000 7f347f0a3640 1 Processor -- start 2024-02-17T12:05:38.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.703+0000 7f347f0a3640 1 -- start start 2024-02-17T12:05:38.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.703+0000 7f347f0a3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478100190 0x7f34780727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.703+0000 7f347f0a3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3478100ab0 0x7f3478072d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.703+0000 7f347f0a3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f34781054b0 0x7f347806ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.703+0000 7f347f0a3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347806c1a0 con 0x7f3478100ab0 2024-02-17T12:05:38.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.703+0000 7f347f0a3640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347806c310 con 0x7f34781054b0 2024-02-17T12:05:38.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.703+0000 7f347f0a3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347806c4d0 con 0x7f3478100190 2024-02-17T12:05:38.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.704+0000 7f347ce3f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478100190 0x7f34780727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.704+0000 7f347ce3f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478100190 0x7f34780727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:42180/0 (socket says 172.21.15.94:42180) 2024-02-17T12:05:38.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.704+0000 7f347ce3f640 1 -- 172.21.15.94:0/2151407944 learned_addr learned my addr 172.21.15.94:0/2151407944 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:38.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.705+0000 7f3477fff640 1 --2- 172.21.15.94:0/2151407944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3478100ab0 0x7f3478072d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.706+0000 7f347ce3f640 1 -- 172.21.15.94:0/2151407944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f34781054b0 msgr2=0x7f347806ba10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-17T12:05:38.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.706+0000 7f347ce3f640 1 --2- 172.21.15.94:0/2151407944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f34781054b0 0x7f347806ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.706+0000 7f347ce3f640 1 -- 172.21.15.94:0/2151407944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3478100ab0 msgr2=0x7f3478072d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.706+0000 7f347ce3f640 1 --2- 172.21.15.94:0/2151407944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3478100ab0 0x7f3478072d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.706+0000 7f347ce3f640 1 -- 172.21.15.94:0/2151407944 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3470009840 con 0x7f3478100190 2024-02-17T12:05:38.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.706+0000 7f347ce3f640 1 --2- 172.21.15.94:0/2151407944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478100190 0x7f34780727b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f346800d950 tx=0x7f346800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:38.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.707+0000 7f3475ffb640 1 -- 172.21.15.94:0/2151407944 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468009c30 con 0x7f3478100190 2024-02-17T12:05:38.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.708+0000 7f347f0a3640 1 -- 172.21.15.94:0/2151407944 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347806c7b0 con 0x7f3478100190 2024-02-17T12:05:38.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.708+0000 7f347f0a3640 1 -- 172.21.15.94:0/2151407944 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3478070690 con 0x7f3478100190 2024-02-17T12:05:38.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.708+0000 7f3475ffb640 1 -- 172.21.15.94:0/2151407944 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3468009d90 con 0x7f3478100190 2024-02-17T12:05:38.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.708+0000 7f3475ffb640 1 -- 172.21.15.94:0/2151407944 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346800f7e0 con 0x7f3478100190 2024-02-17T12:05:38.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.716+0000 7f3481c71740 1 Processor -- start 2024-02-17T12:05:38.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.716+0000 7f3481c71740 1 -- start start 2024-02-17T12:05:38.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.717+0000 7f3481c71740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab41340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.717+0000 7f3481c71740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab41880 0x55a59ab5b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.717+0000 7f3481c71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab5b7c0 0x55a59ab5dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.717+0000 7f3481c71740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59aab0280 con 0x55a59ab41880 2024-02-17T12:05:38.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.717+0000 7f3481c71740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59aaad7b0 con 0x55a59ab40f60 2024-02-17T12:05:38.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.717+0000 7f3481c71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59aaa8f90 con 0x55a59ab5b7c0 2024-02-17T12:05:38.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.717+0000 7f347d640640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab41340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.717+0000 7f347d640640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab41340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56598/0 (socket says 172.21.15.94:56598) 2024-02-17T12:05:38.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.717+0000 7f347d640640 1 -- 172.21.15.94:0/3762644993 learned_addr learned my addr 172.21.15.94:0/3762644993 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:38.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.718+0000 7f347d640640 1 -- 172.21.15.94:0/3762644993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab5b7c0 msgr2=0x55a59ab5dbb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:05:38.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.718+0000 7f347d640640 1 --2- 172.21.15.94:0/3762644993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab5b7c0 0x55a59ab5dbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.718+0000 7f347d640640 1 -- 172.21.15.94:0/3762644993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab41880 msgr2=0x55a59ab5b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.718+0000 7f347d640640 1 --2- 172.21.15.94:0/3762644993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab41880 0x55a59ab5b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.718+0000 7f347d640640 1 -- 172.21.15.94:0/3762644993 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a59aa67570 con 0x55a59ab40f60 2024-02-17T12:05:38.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.718+0000 7f3475ffb640 1 -- 172.21.15.94:0/2151407944 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f346800f940 con 0x7f3478100190 2024-02-17T12:05:38.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.719+0000 7f3475ffb640 1 --2- 172.21.15.94:0/2151407944 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f34600a2440 0x7f34600a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.719+0000 7f347d640640 1 --2- 172.21.15.94:0/3762644993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab41340 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f346c009720 tx=0x7f346c009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9de1ed513fc1bb8d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:38.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.719+0000 7f3475ffb640 1 -- 172.21.15.94:0/2151407944 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f34680cdad0 con 0x7f3478100190 2024-02-17T12:05:38.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.719+0000 7f345effd640 1 -- 172.21.15.94:0/3762644993 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c00bad0 con 0x55a59ab40f60 2024-02-17T12:05:38.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.719+0000 7f345effd640 1 -- 172.21.15.94:0/3762644993 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f346c009e30 con 0x55a59ab40f60 2024-02-17T12:05:38.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.721+0000 7f3481c71740 1 -- 172.21.15.94:0/3762644993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 msgr2=0x55a59ab41340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.721+0000 7f3481c71740 1 --2- 172.21.15.94:0/3762644993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab41340 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f346c009720 tx=0x7f346c009be0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.721+0000 7f3481c71740 1 -- 172.21.15.94:0/3762644993 shutdown_connections 2024-02-17T12:05:38.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.721+0000 7f3481c71740 1 --2- 172.21.15.94:0/3762644993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab5b7c0 0x55a59ab5dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.721+0000 7f3481c71740 1 --2- 172.21.15.94:0/3762644993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab41880 0x55a59ab5b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.721+0000 7f3481c71740 1 --2- 172.21.15.94:0/3762644993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab41340 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.721+0000 7f3481c71740 1 -- 172.21.15.94:0/3762644993 >> 172.21.15.94:0/3762644993 conn(0x55a59ab3b990 msgr2=0x55a59ab3bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:38.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.721+0000 7f347ce3f640 1 --2- 172.21.15.94:0/2151407944 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f34600a2440 0x7f34600a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.721+0000 7f3481c71740 1 -- 172.21.15.94:0/3762644993 shutdown_connections 2024-02-17T12:05:38.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.722+0000 7f3481c71740 1 -- 172.21.15.94:0/3762644993 wait complete. 2024-02-17T12:05:38.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.722+0000 7f3481c71740 1 -- start start 2024-02-17T12:05:38.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.723+0000 7f3481c71740 1 Processor -- start 2024-02-17T12:05:38.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.723+0000 7f3481c71740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab5b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.723+0000 7f3481c71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab41880 0x55a59ab58450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.723+0000 7f3481c71740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab5b7c0 0x55a59ab58990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.723+0000 7f3481c71740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59a77c430 con 0x55a59ab5b7c0 2024-02-17T12:05:38.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.723+0000 7f3481c71740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59a738e00 con 0x55a59ab40f60 2024-02-17T12:05:38.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.723+0000 7f3481c71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59aa2dfd0 con 0x55a59ab41880 2024-02-17T12:05:38.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.723+0000 7f3481c71740 10 client.0 _ll_register_callbacks cb 0x55a59aa83fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.724+0000 7f347d640640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab5b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.724+0000 7f347d640640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab5b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56608/0 (socket says 172.21.15.94:56608) 2024-02-17T12:05:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.724+0000 7f347d640640 1 -- 172.21.15.94:0/4192008956 learned_addr learned my addr 172.21.15.94:0/4192008956 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.724+0000 7f347d640640 1 --2- 172.21.15.94:0/4192008956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab5b7c0 0x55a59ab58990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.724+0000 7f3477fff640 1 --2- 172.21.15.94:0/4192008956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab41880 0x55a59ab58450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.724+0000 7f347d640640 1 -- 172.21.15.94:0/4192008956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab41880 msgr2=0x55a59ab58450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.724+0000 7f347d640640 1 --2- 172.21.15.94:0/4192008956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab41880 0x55a59ab58450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.724+0000 7f347d640640 1 -- 172.21.15.94:0/4192008956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab5b7c0 msgr2=0x55a59ab58990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.724+0000 7f347d640640 1 --2- 172.21.15.94:0/4192008956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab5b7c0 0x55a59ab58990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.724+0000 7f347d640640 1 -- 172.21.15.94:0/4192008956 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346c009350 con 0x55a59ab40f60 2024-02-17T12:05:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.725+0000 7f347d640640 1 --2- 172.21.15.94:0/4192008956 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab5b0c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f346c0118f0 tx=0x7f346c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da278e494d63c60 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.725+0000 7f347ce3f640 1 --2- 172.21.15.94:0/2151407944 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f34600a2440 0x7f34600a4900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3468010fd0 tx=0x7f3468096040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.725+0000 7f3457fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:05:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.725+0000 7f3457fff640 1 -- 172.21.15.94:0/4192008956 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c00bd10 con 0x55a59ab40f60 2024-02-17T12:05:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.725+0000 7f3457fff640 1 -- 172.21.15.94:0/4192008956 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f346c00be70 con 0x55a59ab40f60 2024-02-17T12:05:38.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.726+0000 7f3481c71740 1 -- 172.21.15.94:0/4192008956 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a59a7a87e0 con 0x55a59ab40f60 2024-02-17T12:05:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.726+0000 7f3457fff640 1 -- 172.21.15.94:0/4192008956 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f346c010800 con 0x55a59ab40f60 2024-02-17T12:05:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.726+0000 7f3481c71740 10 client.26984 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:38.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.726+0000 7f3481c71740 1 -- 172.21.15.94:0/4192008956 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a59aaa7610 con 0x55a59ab40f60 2024-02-17T12:05:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.727+0000 7f3457fff640 1 -- 172.21.15.94:0/4192008956 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a59a7a87e0 con 0x55a59ab40f60 2024-02-17T12:05:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.727+0000 7f3457fff640 1 -- 172.21.15.94:0/4192008956 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f346c01ba70 con 0x55a59ab40f60 2024-02-17T12:05:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.727+0000 7f3481c71740 10 client.26984 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.727+0000 7f3481c71740 10 client.26984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.727+0000 7f3481c71740 20 client.26984 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.727+0000 7f3481c71740 1 --2- 172.21.15.94:0/4192008956 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a59ab6b9a0 0x55a59ab6ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.727+0000 7f3481c71740 4 client.26984 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.727+0000 7f3481c71740 1 -- 172.21.15.94:0/4192008956 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a59a7e5720 con 0x55a59ab6b9a0 2024-02-17T12:05:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.728+0000 7f3477fff640 1 --2- 172.21.15.94:0/4192008956 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a59ab6b9a0 0x55a59ab6ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.728+0000 7f3457fff640 10 client.26984 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:38.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.728+0000 7f3477fff640 1 --2- 172.21.15.94:0/4192008956 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a59ab6b9a0 0x55a59ab6ddc0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1609a4b1ce1109b6 server_cookie=6f0a7153710436c8 in_seq=0 out_seq=0 2024-02-17T12:05:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.729+0000 7f3457fff640 1 -- 172.21.15.94:0/4192008956 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a59a7e5720 con 0x55a59ab6b9a0 2024-02-17T12:05:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.729+0000 7f3457fff640 10 client.26984 handle_command_reply: tid=0 2024-02-17T12:05:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.729+0000 7f3481c71740 1 client.26984 shutdown 2024-02-17T12:05:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.729+0000 7f3481c71740 1 -- 172.21.15.94:0/4192008956 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 msgr2=0x55a59ab5b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.729+0000 7f3481c71740 1 --2- 172.21.15.94:0/4192008956 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab5b0c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f346c0118f0 tx=0x7f346c008040 comp rx=0 tx=0).stop 2024-02-17T12:05:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.730+0000 7f3481c71740 1 -- 172.21.15.94:0/4192008956 shutdown_connections 2024-02-17T12:05:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.730+0000 7f3481c71740 1 --2- 172.21.15.94:0/4192008956 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a59ab6b9a0 0x55a59ab6ddc0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.730+0000 7f3481c71740 1 --2- 172.21.15.94:0/4192008956 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab5b7c0 0x55a59ab58990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.730+0000 7f3481c71740 1 --2- 172.21.15.94:0/4192008956 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab41880 0x55a59ab58450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.730+0000 7f3481c71740 1 --2- 172.21.15.94:0/4192008956 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab5b0c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.730+0000 7f3481c71740 1 -- 172.21.15.94:0/4192008956 >> 172.21.15.94:0/4192008956 conn(0x55a59ab3b990 msgr2=0x55a59ab3d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:38.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.730+0000 7f3457fff640 0 client.26984 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:38.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.730+0000 7f3481c71740 1 -- 172.21.15.94:0/4192008956 shutdown_connections 2024-02-17T12:05:38.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.730+0000 7f3481c71740 1 -- 172.21.15.94:0/4192008956 wait complete. 2024-02-17T12:05:38.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.730+0000 7f3481c71740 20 client.26984 trim_cache size 0 max 16384 2024-02-17T12:05:38.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.755+0000 7f3481c71740 1 Processor -- start 2024-02-17T12:05:38.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.755+0000 7f3481c71740 1 -- start start 2024-02-17T12:05:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.755+0000 7f3481c71740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab5ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.755+0000 7f3481c71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab41880 0x55a59ab5b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f3481c71740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab5b7c0 0x55a59ab58410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f3481c71740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59ab5df60 con 0x55a59ab5b7c0 2024-02-17T12:05:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f3481c71740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59ab72fb0 con 0x55a59ab40f60 2024-02-17T12:05:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f3481c71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59ab41670 con 0x55a59ab41880 2024-02-17T12:05:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f347d640640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab5ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f347d640640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab5ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56614/0 (socket says 172.21.15.94:56614) 2024-02-17T12:05:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f347d640640 1 -- 172.21.15.94:0/3384496302 learned_addr learned my addr 172.21.15.94:0/3384496302 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f3477fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab41880 0x55a59ab5b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f347d640640 1 --2- 172.21.15.94:0/3384496302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab5b7c0 0x55a59ab58410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f347d640640 1 -- 172.21.15.94:0/3384496302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab41880 msgr2=0x55a59ab5b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f347d640640 1 --2- 172.21.15.94:0/3384496302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab41880 0x55a59ab5b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f347d640640 1 -- 172.21.15.94:0/3384496302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab5b7c0 msgr2=0x55a59ab58410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f347d640640 1 --2- 172.21.15.94:0/3384496302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab5b7c0 0x55a59ab58410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f347d640640 1 -- 172.21.15.94:0/3384496302 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a59aa43ac0 con 0x55a59ab40f60 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f347d640640 1 --2- 172.21.15.94:0/3384496302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab5ab80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f346c014040 tx=0x7f346c009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1353f7920df59bae server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f345e7fc640 1 -- 172.21.15.94:0/3384496302 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c006d60 con 0x55a59ab40f60 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f345e7fc640 1 -- 172.21.15.94:0/3384496302 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f346c00b4b0 con 0x55a59ab40f60 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.756+0000 7f345e7fc640 1 -- 172.21.15.94:0/3384496302 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346c0299e0 con 0x55a59ab40f60 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.757+0000 7f3481c71740 1 -- 172.21.15.94:0/3384496302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 msgr2=0x55a59ab5ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.757+0000 7f3481c71740 1 --2- 172.21.15.94:0/3384496302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab5ab80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f346c014040 tx=0x7f346c009be0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.757+0000 7f3481c71740 1 -- 172.21.15.94:0/3384496302 shutdown_connections 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.757+0000 7f3481c71740 1 --2- 172.21.15.94:0/3384496302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab5b7c0 0x55a59ab58410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.758+0000 7f3481c71740 1 --2- 172.21.15.94:0/3384496302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab41880 0x55a59ab5b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.758+0000 7f3481c71740 1 --2- 172.21.15.94:0/3384496302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab40f60 0x55a59ab5ab80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.758+0000 7f3481c71740 1 -- 172.21.15.94:0/3384496302 >> 172.21.15.94:0/3384496302 conn(0x55a59ab3b990 msgr2=0x55a59ab53780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:38.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.759+0000 7f3481c71740 1 -- 172.21.15.94:0/3384496302 shutdown_connections 2024-02-17T12:05:38.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.759+0000 7f3481c71740 1 -- 172.21.15.94:0/3384496302 wait complete. 2024-02-17T12:05:38.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.760+0000 7f3481c71740 1 -- start start 2024-02-17T12:05:38.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.760+0000 7f3481c71740 1 Processor -- start 2024-02-17T12:05:38.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.760+0000 7f3481c71740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab40f60 0x55a59ab553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.760+0000 7f3481c71740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab41880 0x55a59ab55900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.760+0000 7f3481c71740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab5b7c0 0x55a59ab55e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.760+0000 7f3481c71740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59abc2060 con 0x55a59ab41880 2024-02-17T12:05:38.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.760+0000 7f3481c71740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59abc21d0 con 0x55a59ab5b7c0 2024-02-17T12:05:38.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.760+0000 7f3481c71740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a59abc2390 con 0x55a59ab40f60 2024-02-17T12:05:38.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.760+0000 7f3481c71740 10 client.0 _ll_register_callbacks cb 0x7f346c006f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:38.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f3477fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab41880 0x55a59ab55900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f3477fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab41880 0x55a59ab55900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32942/0 (socket says 172.21.15.94:32942) 2024-02-17T12:05:38.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f3477fff640 1 -- 172.21.15.94:0/4268774799 learned_addr learned my addr 172.21.15.94:0/4268774799 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:38.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f347d640640 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab5b7c0 0x55a59ab55e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f347d640640 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab40f60 0x55a59ab553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f3477fff640 1 -- 172.21.15.94:0/4268774799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab40f60 msgr2=0x55a59ab553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f3477fff640 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab40f60 0x55a59ab553c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f3477fff640 1 -- 172.21.15.94:0/4268774799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab5b7c0 msgr2=0x55a59ab55e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:38.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f3477fff640 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab5b7c0 0x55a59ab55e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f3477fff640 1 -- 172.21.15.94:0/4268774799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a59aaa7610 con 0x55a59ab41880 2024-02-17T12:05:38.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f347d640640 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab5b7c0 0x55a59ab55e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:38.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f3477fff640 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab41880 0x55a59ab55900 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f347002c5a0 tx=0x7f3470045d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652ab94e1b2e2a90 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:38.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f3457fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:38.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f3457fff640 1 -- 172.21.15.94:0/4268774799 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470045550 con 0x55a59ab41880 2024-02-17T12:05:38.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f3457fff640 1 -- 172.21.15.94:0/4268774799 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f3470042040 con 0x55a59ab41880 2024-02-17T12:05:38.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.761+0000 7f3481c71740 1 -- 172.21.15.94:0/4268774799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f346c009e30 con 0x55a59ab41880 2024-02-17T12:05:38.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.762+0000 7f3457fff640 1 -- 172.21.15.94:0/4268774799 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34700456b0 con 0x55a59ab41880 2024-02-17T12:05:38.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.762+0000 7f3481c71740 10 client.17514 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:38.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.762+0000 7f3481c71740 1 -- 172.21.15.94:0/4268774799 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a59aa696a0 con 0x55a59ab41880 2024-02-17T12:05:38.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.762+0000 7f3457fff640 1 -- 172.21.15.94:0/4268774799 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f347005c020 con 0x55a59ab41880 2024-02-17T12:05:38.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.762+0000 7f3457fff640 1 -- 172.21.15.94:0/4268774799 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f347003d3f0 con 0x55a59ab41880 2024-02-17T12:05:38.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.762+0000 7f3481c71740 10 client.17514 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:38.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.762+0000 7f3481c71740 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:38.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.762+0000 7f3481c71740 20 client.17514 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:38.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.762+0000 7f3481c71740 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a59abc3b90 0x55a59abc5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:38.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.762+0000 7f3481c71740 4 client.17514 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:05:38.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.762+0000 7f3481c71740 1 -- 172.21.15.94:0/4268774799 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a59a834890 con 0x55a59abc3b90 2024-02-17T12:05:38.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.762+0000 7f347d640640 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a59abc3b90 0x55a59abc5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:38.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3457fff640 10 client.17514 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:38.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f347d640640 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a59abc3b90 0x55a59abc5fb0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd9e53b3549bdc14 server_cookie=7c6105c39a00be7c in_seq=0 out_seq=0 2024-02-17T12:05:38.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3457fff640 1 -- 172.21.15.94:0/4268774799 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a59a834890 con 0x55a59abc3b90 2024-02-17T12:05:38.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3457fff640 10 client.17514 handle_command_reply: tid=0 2024-02-17T12:05:38.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3481c71740 1 client.17514 shutdown 2024-02-17T12:05:38.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3481c71740 1 -- 172.21.15.94:0/4268774799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab41880 msgr2=0x55a59ab55900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3481c71740 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab41880 0x55a59ab55900 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f347002c5a0 tx=0x7f3470045d40 comp rx=0 tx=0).stop 2024-02-17T12:05:38.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3481c71740 1 -- 172.21.15.94:0/4268774799 shutdown_connections 2024-02-17T12:05:38.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3481c71740 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a59abc3b90 0x55a59abc5fb0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3481c71740 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a59ab5b7c0 0x55a59ab55e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3481c71740 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a59ab41880 0x55a59ab55900 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3481c71740 1 --2- 172.21.15.94:0/4268774799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a59ab40f60 0x55a59ab553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3481c71740 1 -- 172.21.15.94:0/4268774799 >> 172.21.15.94:0/4268774799 conn(0x55a59ab3b990 msgr2=0x55a59ab58410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:38.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3457fff640 0 client.17514 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:38.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3481c71740 1 -- 172.21.15.94:0/4268774799 shutdown_connections 2024-02-17T12:05:38.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3481c71740 1 -- 172.21.15.94:0/4268774799 wait complete. 2024-02-17T12:05:38.771 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.763+0000 7f3481c71740 20 client.17514 trim_cache size 0 max 16384 2024-02-17T12:05:38.771 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:05:38.771 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:05:38.771 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:05:38.771 INFO:teuthology.orchestra.run.smithi094.stdout: "4be090d3-12b5-4b72-8fa4-2af4015eaed6": { 2024-02-17T12:05:38.771 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:05:38.771 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "4be090d3-12b5-4b72-8fa4-2af4015eaed6", 2024-02-17T12:05:38.771 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:05:38.772 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:05:38.772 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:05:38.772 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:05:38.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.764+0000 7f347f0a3640 1 -- 172.21.15.94:0/2151407944 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f34600a2440 msgr2=0x7f34600a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.764+0000 7f347f0a3640 1 --2- 172.21.15.94:0/2151407944 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f34600a2440 0x7f34600a4900 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3468010fd0 tx=0x7f3468096040 comp rx=0 tx=0).stop 2024-02-17T12:05:38.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.764+0000 7f347f0a3640 1 -- 172.21.15.94:0/2151407944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478100190 msgr2=0x7f34780727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:38.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.764+0000 7f347f0a3640 1 --2- 172.21.15.94:0/2151407944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478100190 0x7f34780727b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f346800d950 tx=0x7f346800de10 comp rx=0 tx=0).stop 2024-02-17T12:05:38.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.764+0000 7f347f0a3640 1 -- 172.21.15.94:0/2151407944 shutdown_connections 2024-02-17T12:05:38.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.764+0000 7f347f0a3640 1 --2- 172.21.15.94:0/2151407944 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f34600a2440 0x7f34600a4900 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.764+0000 7f347f0a3640 1 --2- 172.21.15.94:0/2151407944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f34781054b0 0x7f347806ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.764+0000 7f347f0a3640 1 --2- 172.21.15.94:0/2151407944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3478100ab0 0x7f3478072d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.764+0000 7f347f0a3640 1 --2- 172.21.15.94:0/2151407944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3478100190 0x7f34780727b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:38.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.764+0000 7f347f0a3640 1 -- 172.21.15.94:0/2151407944 >> 172.21.15.94:0/2151407944 conn(0x7f3478077230 msgr2=0x7f3478102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:38.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.765+0000 7f347f0a3640 1 -- 172.21.15.94:0/2151407944 shutdown_connections 2024-02-17T12:05:38.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:05:38.765+0000 7f347f0a3640 1 -- 172.21.15.94:0/2151407944 wait complete. 2024-02-17T12:05:38.778 INFO:tasks.cephfs.filesystem:scrub status for tag:4be090d3-12b5-4b72-8fa4-2af4015eaed6 - {'path': '/', 'tag': '4be090d3-12b5-4b72-8fa4-2af4015eaed6', 'options': 'recursive,force'} 2024-02-17T12:05:40.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:39 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:05:40.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:39 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:05:40.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:39 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:05:40.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:39 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:05:40.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:39 smithi094 ceph-mon[32499]: pgmap v231: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 982 op/s 2024-02-17T12:05:40.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 440.0 s, 2724.2 tps, lat 36.679 ms stddev 38.864 2024-02-17T12:05:40.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:39 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:05:40.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:39 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:05:40.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:39 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:05:40.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:39 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:05:40.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:39 smithi189 ceph-mon[36588]: pgmap v231: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 982 op/s 2024-02-17T12:05:40.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:39 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:05:40.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:39 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:05:40.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:39 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:05:40.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:39 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:05:40.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:39 smithi125 ceph-mon[36819]: pgmap v231: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 11 MiB/s wr, 982 op/s 2024-02-17T12:05:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.944+0000 7ffb87481640 1 -- 172.21.15.94:0/3257882741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb801002d0 msgr2=0x7ffb801006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.944+0000 7ffb87481640 1 --2- 172.21.15.94:0/3257882741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb801002d0 0x7ffb801006b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009930 tx=0x7ffb7c02bc20 comp rx=0 tx=0).stop 2024-02-17T12:05:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 -- 172.21.15.94:0/3257882741 shutdown_connections 2024-02-17T12:05:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 --2- 172.21.15.94:0/3257882741 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb801053a0 0x7ffb8010cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 --2- 172.21.15.94:0/3257882741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb80100c80 0x7ffb80104cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 --2- 172.21.15.94:0/3257882741 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb801002d0 0x7ffb801006b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 -- 172.21.15.94:0/3257882741 >> 172.21.15.94:0/3257882741 conn(0x7ffb800771e0 msgr2=0x7ffb800775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 -- 172.21.15.94:0/3257882741 shutdown_connections 2024-02-17T12:05:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 -- 172.21.15.94:0/3257882741 wait complete. 2024-02-17T12:05:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 Processor -- start 2024-02-17T12:05:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 -- start start 2024-02-17T12:05:41.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb801002d0 0x7ffb80195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb801053a0 0x7ffb80195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb80199e40 0x7ffb8019a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8019a830 con 0x7ffb80199e40 2024-02-17T12:05:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8019a9a0 con 0x7ffb801053a0 2024-02-17T12:05:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.945+0000 7ffb87481640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8019ab10 con 0x7ffb801002d0 2024-02-17T12:05:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.946+0000 7ffb85a1e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb80199e40 0x7ffb8019a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.946+0000 7ffb85a1e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb80199e40 0x7ffb8019a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32962/0 (socket says 172.21.15.94:32962) 2024-02-17T12:05:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.946+0000 7ffb85a1e640 1 -- 172.21.15.94:0/3454159752 learned_addr learned my addr 172.21.15.94:0/3454159752 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.946+0000 7ffb84a1c640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb801053a0 0x7ffb80195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.946+0000 7ffb8521d640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb801002d0 0x7ffb80195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.946+0000 7ffb85a1e640 1 -- 172.21.15.94:0/3454159752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb801002d0 msgr2=0x7ffb80195580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:41.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.946+0000 7ffb85a1e640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb801002d0 0x7ffb80195580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.946+0000 7ffb85a1e640 1 -- 172.21.15.94:0/3454159752 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb801053a0 msgr2=0x7ffb80195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.946+0000 7ffb85a1e640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb801053a0 0x7ffb80195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.946+0000 7ffb85a1e640 1 -- 172.21.15.94:0/3454159752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c009560 con 0x7ffb80199e40 2024-02-17T12:05:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.946+0000 7ffb84a1c640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb801053a0 0x7ffb80195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.947+0000 7ffb85a1e640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb80199e40 0x7ffb8019a2f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ffb78010f90 tx=0x7ffb7800e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.947+0000 7ffb767fc640 1 -- 172.21.15.94:0/3454159752 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7800bbc0 con 0x7ffb80199e40 2024-02-17T12:05:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.947+0000 7ffb767fc640 1 -- 172.21.15.94:0/3454159752 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb78011040 con 0x7ffb80199e40 2024-02-17T12:05:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.947+0000 7ffb767fc640 1 -- 172.21.15.94:0/3454159752 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78005990 con 0x7ffb80199e40 2024-02-17T12:05:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.947+0000 7ffb87481640 1 -- 172.21.15.94:0/3454159752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb801a20f0 con 0x7ffb80199e40 2024-02-17T12:05:41.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.948+0000 7ffb87481640 1 -- 172.21.15.94:0/3454159752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb801a68d0 con 0x7ffb80199e40 2024-02-17T12:05:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.950+0000 7ffb767fc640 1 -- 172.21.15.94:0/3454159752 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ffb78004a40 con 0x7ffb80199e40 2024-02-17T12:05:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.950+0000 7ffb8a055740 1 Processor -- start 2024-02-17T12:05:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.950+0000 7ffb8a055740 1 -- start start 2024-02-17T12:05:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8a055740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89fad0 0x55a22d89e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8a055740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d89ed60 0x55a22d89f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8a055740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d8a82e0 0x55a22d8aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8521d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d89ed60 0x55a22d89f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb767fc640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ffb6c0a2430 0x7ffb6c0a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8521d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d89ed60 0x55a22d89f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:42224/0 (socket says 172.21.15.94:42224) 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8521d640 1 -- 172.21.15.94:0/3076463076 learned_addr learned my addr 172.21.15.94:0/3076463076 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8a055740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22d80d9f0 con 0x55a22d89fad0 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8a055740 1 -- 172.21.15.94:0/3076463076 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22d80af20 con 0x55a22d8a82e0 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8a055740 1 -- 172.21.15.94:0/3076463076 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22d806700 con 0x55a22d89ed60 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb85a1e640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ffb6c0a2430 0x7ffb6c0a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8521d640 1 -- 172.21.15.94:0/3076463076 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d8a82e0 msgr2=0x55a22d8aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8521d640 1 --2- 172.21.15.94:0/3076463076 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d8a82e0 0x55a22d8aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8521d640 1 -- 172.21.15.94:0/3076463076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89fad0 msgr2=0x55a22d89e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8521d640 1 --2- 172.21.15.94:0/3076463076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89fad0 0x55a22d89e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb8521d640 1 -- 172.21.15.94:0/3076463076 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a22d79c7b0 con 0x55a22d89ed60 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.951+0000 7ffb767fc640 1 -- 172.21.15.94:0/3454159752 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ffb780cde30 con 0x7ffb80199e40 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.952+0000 7ffb8521d640 1 --2- 172.21.15.94:0/3076463076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d89ed60 0x55a22d89f1e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c036040 tx=0x7ffb7c02c670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=647f743fc555c82f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.952+0000 7ffb6b7fe640 1 -- 172.21.15.94:0/3076463076 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c004880 con 0x55a22d89ed60 2024-02-17T12:05:41.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.952+0000 7ffb6b7fe640 1 -- 172.21.15.94:0/3076463076 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb7c0049e0 con 0x55a22d89ed60 2024-02-17T12:05:41.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.952+0000 7ffb6b7fe640 1 -- 172.21.15.94:0/3076463076 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c005150 con 0x55a22d89ed60 2024-02-17T12:05:41.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.954+0000 7ffb8a055740 1 -- 172.21.15.94:0/3076463076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d89ed60 msgr2=0x55a22d89f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.954+0000 7ffb8a055740 1 --2- 172.21.15.94:0/3076463076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d89ed60 0x55a22d89f1e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c036040 tx=0x7ffb7c02c670 comp rx=0 tx=0).stop 2024-02-17T12:05:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.954+0000 7ffb85a1e640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ffb6c0a2430 0x7ffb6c0a48f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ffb780cb620 tx=0x7ffb78096480 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.954+0000 7ffb8a055740 1 -- 172.21.15.94:0/3076463076 shutdown_connections 2024-02-17T12:05:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.954+0000 7ffb8a055740 1 --2- 172.21.15.94:0/3076463076 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d8a82e0 0x55a22d8aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.954+0000 7ffb8a055740 1 --2- 172.21.15.94:0/3076463076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d89ed60 0x55a22d89f1e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.954+0000 7ffb8a055740 1 --2- 172.21.15.94:0/3076463076 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89fad0 0x55a22d89e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.954+0000 7ffb8a055740 1 -- 172.21.15.94:0/3076463076 >> 172.21.15.94:0/3076463076 conn(0x55a22d866550 msgr2=0x55a22d899a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.954+0000 7ffb8a055740 1 -- 172.21.15.94:0/3076463076 shutdown_connections 2024-02-17T12:05:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.954+0000 7ffb8a055740 1 -- 172.21.15.94:0/3076463076 wait complete. 2024-02-17T12:05:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.954+0000 7ffb8a055740 1 -- start start 2024-02-17T12:05:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.954+0000 7ffb8a055740 1 Processor -- start 2024-02-17T12:05:41.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.955+0000 7ffb8a055740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 0x55a22d8a5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.955+0000 7ffb8a055740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89fad0 0x55a22d8a5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.955+0000 7ffb84a1c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 0x55a22d8a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.955+0000 7ffb84a1c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 0x55a22d8a5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32984/0 (socket says 172.21.15.94:32984) 2024-02-17T12:05:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.955+0000 7ffb8521d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89fad0 0x55a22d8a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.955+0000 7ffb8521d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89fad0 0x55a22d8a5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:56642/0 (socket says 172.21.15.94:56642) 2024-02-17T12:05:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.955+0000 7ffb8a055740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 0x55a22d8a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.956+0000 7ffb84a1c640 1 -- 172.21.15.94:0/1218318194 learned_addr learned my addr 172.21.15.94:0/1218318194 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.956+0000 7ffb8a055740 1 -- 172.21.15.94:0/1218318194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22d7abbf0 con 0x55a22d89ed60 2024-02-17T12:05:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.956+0000 7ffb8a055740 1 -- 172.21.15.94:0/1218318194 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22d4da250 con 0x55a22d89fad0 2024-02-17T12:05:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.956+0000 7ffb8a055740 1 -- 172.21.15.94:0/1218318194 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22d78c200 con 0x55a22d8a82e0 2024-02-17T12:05:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.956+0000 7ffb84a1c640 1 --2- 172.21.15.94:0/1218318194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 0x55a22d8a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.956+0000 7ffb84a1c640 1 -- 172.21.15.94:0/1218318194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 msgr2=0x55a22d8a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.956+0000 7ffb84a1c640 1 --2- 172.21.15.94:0/1218318194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 0x55a22d8a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.956+0000 7ffb84a1c640 1 -- 172.21.15.94:0/1218318194 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89fad0 msgr2=0x55a22d8a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.956+0000 7ffb84a1c640 1 --2- 172.21.15.94:0/1218318194 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89fad0 0x55a22d8a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.956+0000 7ffb84a1c640 1 -- 172.21.15.94:0/1218318194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8010d780 con 0x55a22d89ed60 2024-02-17T12:05:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.957+0000 7ffb84a1c640 1 --2- 172.21.15.94:0/1218318194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 0x55a22d8a5420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000b640 tx=0x7ffb7000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34709d3b49039eca server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.957+0000 7ffb68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.957+0000 7ffb8a055740 10 client.0 _ll_register_callbacks cb 0x55a22d7fad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.957+0000 7ffb68ff9640 1 -- 172.21.15.94:0/1218318194 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7001a070 con 0x55a22d89ed60 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.957+0000 7ffb68ff9640 1 -- 172.21.15.94:0/1218318194 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ffb700107a0 con 0x55a22d89ed60 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.957+0000 7ffb8a055740 1 -- 172.21.15.94:0/1218318194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a22d506600 con 0x55a22d89ed60 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.957+0000 7ffb68ff9640 1 -- 172.21.15.94:0/1218318194 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb70004020 con 0x55a22d89ed60 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.957+0000 7ffb8a055740 10 client.17523 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.957+0000 7ffb8a055740 1 -- 172.21.15.94:0/1218318194 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x55a22d804d80 con 0x55a22d89ed60 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.958+0000 7ffb68ff9640 1 -- 172.21.15.94:0/1218318194 <== mon.0 v2:172.21.15.94:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffb70004180 con 0x55a22d89ed60 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.958+0000 7ffb8a055740 10 client.17523 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.958+0000 7ffb8a055740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.958+0000 7ffb8a055740 20 client.17523 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.958+0000 7ffb8a055740 1 --2- 172.21.15.94:0/1218318194 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a22d8b8460 0x55a22d8ba920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.958+0000 7ffb8a055740 4 client.17523 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.958+0000 7ffb8a055740 1 -- 172.21.15.94:0/1218318194 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a22d5571b0 con 0x55a22d8b8460 2024-02-17T12:05:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.958+0000 7ffb8521d640 1 --2- 172.21.15.94:0/1218318194 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a22d8b8460 0x55a22d8ba920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb8521d640 1 --2- 172.21.15.94:0/1218318194 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a22d8b8460 0x55a22d8ba920 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94204ab95794e99b server_cookie=8d00023f43293f62 in_seq=0 out_seq=0 2024-02-17T12:05:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb68ff9640 10 client.17523 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb68ff9640 1 -- 172.21.15.94:0/1218318194 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a22d5571b0 con 0x55a22d8b8460 2024-02-17T12:05:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb68ff9640 10 client.17523 handle_command_reply: tid=0 2024-02-17T12:05:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb8a055740 1 client.17523 shutdown 2024-02-17T12:05:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb8a055740 1 -- 172.21.15.94:0/1218318194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 msgr2=0x55a22d8a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb8a055740 1 --2- 172.21.15.94:0/1218318194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 0x55a22d8a5420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000b640 tx=0x7ffb7000bdd0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb8a055740 1 -- 172.21.15.94:0/1218318194 shutdown_connections 2024-02-17T12:05:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb8a055740 1 --2- 172.21.15.94:0/1218318194 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a22d8b8460 0x55a22d8ba920 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb8a055740 1 --2- 172.21.15.94:0/1218318194 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 0x55a22d8a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb68ff9640 0 client.17523 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb8a055740 1 --2- 172.21.15.94:0/1218318194 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89fad0 0x55a22d8a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb8a055740 1 --2- 172.21.15.94:0/1218318194 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 0x55a22d8a5420 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb8a055740 1 -- 172.21.15.94:0/1218318194 >> 172.21.15.94:0/1218318194 conn(0x55a22d866550 msgr2=0x55a22d89b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb8a055740 1 -- 172.21.15.94:0/1218318194 shutdown_connections 2024-02-17T12:05:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.959+0000 7ffb8a055740 1 -- 172.21.15.94:0/1218318194 wait complete. 2024-02-17T12:05:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.960+0000 7ffb8a055740 20 client.17523 trim_cache size 0 max 16384 2024-02-17T12:05:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.979+0000 7ffb8a055740 1 Processor -- start 2024-02-17T12:05:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.979+0000 7ffb8a055740 1 -- start start 2024-02-17T12:05:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.979+0000 7ffb8a055740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89ed60 0x55a22d8a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.979+0000 7ffb8a055740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89fad0 0x55a22d8a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.979+0000 7ffb8a055740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 0x55a22d8af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.979+0000 7ffb8a055740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22d8a0260 con 0x55a22d89fad0 2024-02-17T12:05:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.979+0000 7ffb8a055740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22d8bfb70 con 0x55a22d89ed60 2024-02-17T12:05:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.979+0000 7ffb8a055740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22d8a3990 con 0x55a22d8a82e0 2024-02-17T12:05:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.980+0000 7ffb8521d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89fad0 0x55a22d8a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.980+0000 7ffb8521d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89fad0 0x55a22d8a5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32988/0 (socket says 172.21.15.94:32988) 2024-02-17T12:05:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.980+0000 7ffb8521d640 1 -- 172.21.15.94:0/4203458248 learned_addr learned my addr 172.21.15.94:0/4203458248 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.980+0000 7ffb84a1c640 1 --2- 172.21.15.94:0/4203458248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89ed60 0x55a22d8a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.980+0000 7ffb84a1c640 1 --2- 172.21.15.94:0/4203458248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 0x55a22d8af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.980+0000 7ffb8521d640 1 -- 172.21.15.94:0/4203458248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 msgr2=0x55a22d8af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.980+0000 7ffb8521d640 1 --2- 172.21.15.94:0/4203458248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 0x55a22d8af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.980+0000 7ffb8521d640 1 -- 172.21.15.94:0/4203458248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89ed60 msgr2=0x55a22d8a7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.980+0000 7ffb8521d640 1 --2- 172.21.15.94:0/4203458248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89ed60 0x55a22d8a7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.980+0000 7ffb8521d640 1 -- 172.21.15.94:0/4203458248 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a22d7a17b0 con 0x55a22d89fad0 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.980+0000 7ffb84a1c640 1 --2- 172.21.15.94:0/4203458248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 0x55a22d8af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.981+0000 7ffb8521d640 1 --2- 172.21.15.94:0/4203458248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89fad0 0x55a22d8a5cc0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c005e00 tx=0x7ffb7c041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85bc54af07c85c4f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.981+0000 7ffb6affd640 1 -- 172.21.15.94:0/4203458248 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c04e040 con 0x55a22d89fad0 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.981+0000 7ffb6affd640 1 -- 172.21.15.94:0/4203458248 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb7c02cbb0 con 0x55a22d89fad0 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.981+0000 7ffb6affd640 1 -- 172.21.15.94:0/4203458248 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0050c0 con 0x55a22d89fad0 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.982+0000 7ffb8a055740 1 -- 172.21.15.94:0/4203458248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89fad0 msgr2=0x55a22d8a5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.982+0000 7ffb8a055740 1 --2- 172.21.15.94:0/4203458248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89fad0 0x55a22d8a5cc0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c005e00 tx=0x7ffb7c041040 comp rx=0 tx=0).stop 2024-02-17T12:05:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.982+0000 7ffb8a055740 1 -- 172.21.15.94:0/4203458248 shutdown_connections 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.982+0000 7ffb8a055740 1 --2- 172.21.15.94:0/4203458248 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 0x55a22d8af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.982+0000 7ffb8a055740 1 --2- 172.21.15.94:0/4203458248 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89fad0 0x55a22d8a5cc0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.982+0000 7ffb8a055740 1 --2- 172.21.15.94:0/4203458248 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89ed60 0x55a22d8a7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.982+0000 7ffb8a055740 1 -- 172.21.15.94:0/4203458248 >> 172.21.15.94:0/4203458248 conn(0x55a22d866550 msgr2=0x55a22d8a3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.982+0000 7ffb8a055740 1 -- 172.21.15.94:0/4203458248 shutdown_connections 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.982+0000 7ffb8a055740 1 -- 172.21.15.94:0/4203458248 wait complete. 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.982+0000 7ffb8a055740 1 -- start start 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb8a055740 1 Processor -- start 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb8a055740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 0x55a22d8a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb8a055740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89fad0 0x55a22d8a25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb8a055740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 0x55a22d8a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb8a055740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22d90ea40 con 0x55a22d89ed60 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb8a055740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22d90eb80 con 0x55a22d89fad0 2024-02-17T12:05:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb8a055740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a22d90ecc0 con 0x55a22d8a82e0 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb8a055740 10 client.0 _ll_register_callbacks cb 0x7ffb7c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb84a1c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 0x55a22d8a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb84a1c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 0x55a22d8a20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33002/0 (socket says 172.21.15.94:33002) 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb84a1c640 1 -- 172.21.15.94:0/1246249362 learned_addr learned my addr 172.21.15.94:0/1246249362 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb8521d640 1 --2- 172.21.15.94:0/1246249362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89fad0 0x55a22d8a25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb84a1c640 1 --2- 172.21.15.94:0/1246249362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 0x55a22d8a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb84a1c640 1 -- 172.21.15.94:0/1246249362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 msgr2=0x55a22d8a2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb84a1c640 1 --2- 172.21.15.94:0/1246249362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 0x55a22d8a2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb84a1c640 1 -- 172.21.15.94:0/1246249362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89fad0 msgr2=0x55a22d8a25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb84a1c640 1 --2- 172.21.15.94:0/1246249362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89fad0 0x55a22d8a25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb84a1c640 1 -- 172.21.15.94:0/1246249362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a22d79c7b0 con 0x55a22d89ed60 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.983+0000 7ffb84a1c640 1 --2- 172.21.15.94:0/1246249362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 0x55a22d8a20a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ffb70010a80 tx=0x7ffb7000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d640d1cb1a40e5f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb68ff9640 1 -- 172.21.15.94:0/1246249362 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7001a040 con 0x55a22d89ed60 2024-02-17T12:05:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb68ff9640 1 -- 172.21.15.94:0/1246249362 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7ffb70026060 con 0x55a22d89ed60 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb8a055740 1 -- 172.21.15.94:0/1246249362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb7c0049e0 con 0x55a22d89ed60 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb68ff9640 1 -- 172.21.15.94:0/1246249362 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb70011640 con 0x55a22d89ed60 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb8a055740 10 client.17529 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb8a055740 1 -- 172.21.15.94:0/1246249362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a22d7c7540 con 0x55a22d89ed60 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb68ff9640 1 -- 172.21.15.94:0/1246249362 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb70025020 con 0x55a22d89ed60 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb68ff9640 1 -- 172.21.15.94:0/1246249362 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ffb7002a040 con 0x55a22d89ed60 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb8a055740 10 client.17529 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb8a055740 10 client.17529 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb8a055740 20 client.17529 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb8a055740 1 --2- 172.21.15.94:0/1246249362 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a22d910360 0x55a22d912750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb8a055740 4 client.17529 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb8a055740 1 -- 172.21.15.94:0/1246249362 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7ffb7c0049e0 con 0x55a22d910360 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.984+0000 7ffb8521d640 1 --2- 172.21.15.94:0/1246249362 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a22d910360 0x55a22d912750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.985+0000 7ffb68ff9640 10 client.17529 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:41.985+0000 7ffb8521d640 1 --2- 172.21.15.94:0/1246249362 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a22d910360 0x55a22d912750 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fab9885d6a6d16d8 server_cookie=be9a6e759f2d20d0 in_seq=0 out_seq=0 2024-02-17T12:05:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.010+0000 7ffb68ff9640 1 -- 172.21.15.94:0/1246249362 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7ffb7c0049e0 con 0x55a22d910360 2024-02-17T12:05:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.010+0000 7ffb68ff9640 10 client.17529 handle_command_reply: tid=0 2024-02-17T12:05:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.010+0000 7ffb8a055740 1 client.17529 shutdown 2024-02-17T12:05:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.011+0000 7ffb8a055740 1 -- 172.21.15.94:0/1246249362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 msgr2=0x55a22d8a20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.011+0000 7ffb8a055740 1 --2- 172.21.15.94:0/1246249362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 0x55a22d8a20a0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ffb70010a80 tx=0x7ffb7000d000 comp rx=0 tx=0).stop 2024-02-17T12:05:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.011+0000 7ffb8a055740 1 -- 172.21.15.94:0/1246249362 shutdown_connections 2024-02-17T12:05:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.011+0000 7ffb8a055740 1 --2- 172.21.15.94:0/1246249362 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55a22d910360 0x55a22d912750 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.011+0000 7ffb8a055740 1 --2- 172.21.15.94:0/1246249362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a22d8a82e0 0x55a22d8a2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.011+0000 7ffb8a055740 1 --2- 172.21.15.94:0/1246249362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a22d89fad0 0x55a22d8a25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.011+0000 7ffb8a055740 1 --2- 172.21.15.94:0/1246249362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a22d89ed60 0x55a22d8a20a0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.011+0000 7ffb8a055740 1 -- 172.21.15.94:0/1246249362 >> 172.21.15.94:0/1246249362 conn(0x55a22d866550 msgr2=0x55a22d898300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.011+0000 7ffb68ff9640 0 client.17529 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.011+0000 7ffb8a055740 1 -- 172.21.15.94:0/1246249362 shutdown_connections 2024-02-17T12:05:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.011+0000 7ffb8a055740 1 -- 172.21.15.94:0/1246249362 wait complete. 2024-02-17T12:05:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.011+0000 7ffb8a055740 20 client.17529 trim_cache size 0 max 16384 2024-02-17T12:05:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:05:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:05:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:05:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:05:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.012+0000 7ffb87481640 1 -- 172.21.15.94:0/3454159752 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ffb6c0a2430 msgr2=0x7ffb6c0a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.012+0000 7ffb87481640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ffb6c0a2430 0x7ffb6c0a48f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ffb780cb620 tx=0x7ffb78096480 comp rx=0 tx=0).stop 2024-02-17T12:05:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.012+0000 7ffb87481640 1 -- 172.21.15.94:0/3454159752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb80199e40 msgr2=0x7ffb8019a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.012+0000 7ffb87481640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb80199e40 0x7ffb8019a2f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ffb78010f90 tx=0x7ffb7800e490 comp rx=0 tx=0).stop 2024-02-17T12:05:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.012+0000 7ffb87481640 1 -- 172.21.15.94:0/3454159752 shutdown_connections 2024-02-17T12:05:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.012+0000 7ffb87481640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ffb6c0a2430 0x7ffb6c0a48f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.012+0000 7ffb87481640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ffb80199e40 0x7ffb8019a2f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.012+0000 7ffb87481640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ffb801053a0 0x7ffb80195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.012+0000 7ffb87481640 1 --2- 172.21.15.94:0/3454159752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb801002d0 0x7ffb80195580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.012+0000 7ffb87481640 1 -- 172.21.15.94:0/3454159752 >> 172.21.15.94:0/3454159752 conn(0x7ffb800771e0 msgr2=0x7ffb800fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.012+0000 7ffb87481640 1 -- 172.21.15.94:0/3454159752 shutdown_connections 2024-02-17T12:05:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:42.012+0000 7ffb87481640 1 -- 172.21.15.94:0/3454159752 wait complete. 2024-02-17T12:05:42.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:41 smithi094 ceph-mon[32499]: pgmap v232: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.9 MiB/s wr, 671 op/s 2024-02-17T12:05:42.237 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:41 smithi125 ceph-mon[36819]: pgmap v232: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.9 MiB/s wr, 671 op/s 2024-02-17T12:05:42.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:41 smithi189 ceph-mon[36588]: pgmap v232: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 6.9 MiB/s wr, 671 op/s 2024-02-17T12:05:43.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:42 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:05:43.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:42 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:05:43.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:42 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:05:44.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:43 smithi189 ceph-mon[36588]: pgmap v233: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.0 MiB/s wr, 781 op/s 2024-02-17T12:05:44.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:43 smithi125 ceph-mon[36819]: pgmap v233: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.0 MiB/s wr, 781 op/s 2024-02-17T12:05:44.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:43 smithi094 ceph-mon[32499]: pgmap v233: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 8.0 MiB/s wr, 781 op/s 2024-02-17T12:05:45.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 445.0 s, 2805.0 tps, lat 35.648 ms stddev 39.631 2024-02-17T12:05:46.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:45 smithi189 ceph-mon[36588]: pgmap v234: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 9.0 MiB/s wr, 980 op/s 2024-02-17T12:05:46.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:45 smithi125 ceph-mon[36819]: pgmap v234: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 9.0 MiB/s wr, 980 op/s 2024-02-17T12:05:46.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:45 smithi094 ceph-mon[32499]: pgmap v234: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 9.0 MiB/s wr, 980 op/s 2024-02-17T12:05:48.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:47 smithi189 ceph-mon[36588]: pgmap v235: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 5.9 MiB/s wr, 673 op/s 2024-02-17T12:05:48.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:47 smithi125 ceph-mon[36819]: pgmap v235: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 5.9 MiB/s wr, 673 op/s 2024-02-17T12:05:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:47 smithi094 ceph-mon[32499]: pgmap v235: 97 pgs: 97 active+clean; 5.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 5.9 MiB/s wr, 673 op/s 2024-02-17T12:05:49.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:48 smithi189 ceph-mon[36588]: pgmap v236: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 8.7 MiB/s wr, 953 op/s 2024-02-17T12:05:49.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:48 smithi125 ceph-mon[36819]: pgmap v236: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 8.7 MiB/s wr, 953 op/s 2024-02-17T12:05:49.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:48 smithi094 ceph-mon[32499]: pgmap v236: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 8.7 MiB/s wr, 953 op/s 2024-02-17T12:05:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 450.0 s, 1999.4 tps, lat 49.995 ms stddev 57.412 2024-02-17T12:05:52.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:51 smithi094 ceph-mon[32499]: pgmap v237: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 5.7 MiB/s wr, 663 op/s 2024-02-17T12:05:52.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:51 smithi189 ceph-mon[36588]: pgmap v237: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 5.7 MiB/s wr, 663 op/s 2024-02-17T12:05:52.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:51 smithi125 ceph-mon[36819]: pgmap v237: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 5.7 MiB/s wr, 663 op/s 2024-02-17T12:05:54.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:53 smithi189 ceph-mon[36588]: pgmap v238: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.9 MiB/s wr, 697 op/s 2024-02-17T12:05:54.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:53 smithi125 ceph-mon[36819]: pgmap v238: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.9 MiB/s wr, 697 op/s 2024-02-17T12:05:54.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:53 smithi094 ceph-mon[32499]: pgmap v238: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 5.9 MiB/s wr, 697 op/s 2024-02-17T12:05:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.128+0000 7f5dc828d640 1 -- 172.21.15.94:0/3962196498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc0106460 msgr2=0x7f5dc010e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.128+0000 7f5dc828d640 1 --2- 172.21.15.94:0/3962196498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc0106460 0x7f5dc010e030 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5db8009cd0 tx=0x7f5db802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.128+0000 7f5dc828d640 1 -- 172.21.15.94:0/3962196498 shutdown_connections 2024-02-17T12:05:55.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.128+0000 7f5dc828d640 1 --2- 172.21.15.94:0/3962196498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc0106460 0x7f5dc010e030 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.128+0000 7f5dc828d640 1 --2- 172.21.15.94:0/3962196498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dc0101d40 0x7f5dc0105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.128+0000 7f5dc828d640 1 --2- 172.21.15.94:0/3962196498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dc0101390 0x7f5dc0101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.128+0000 7f5dc828d640 1 -- 172.21.15.94:0/3962196498 >> 172.21.15.94:0/3962196498 conn(0x7f5dc00fcc10 msgr2=0x7f5dc00ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.128+0000 7f5dc828d640 1 -- 172.21.15.94:0/3962196498 shutdown_connections 2024-02-17T12:05:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.129+0000 7f5dc828d640 1 -- 172.21.15.94:0/3962196498 wait complete. 2024-02-17T12:05:55.135 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.129+0000 7f5dc828d640 1 Processor -- start 2024-02-17T12:05:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.129+0000 7f5dc828d640 1 -- start start 2024-02-17T12:05:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.129+0000 7f5dc828d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dc0101390 0x7f5dc0197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.129+0000 7f5dc828d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dc0106460 0x7f5dc0197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.129+0000 7f5dc828d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc019bfd0 0x7f5dc019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.129+0000 7f5dc828d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc019c9c0 con 0x7f5dc0106460 2024-02-17T12:05:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.129+0000 7f5dc828d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc019cb30 con 0x7f5dc019bfd0 2024-02-17T12:05:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.129+0000 7f5dc828d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc019cca0 con 0x7f5dc0101390 2024-02-17T12:05:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.129+0000 7f5dc5828640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dc0106460 0x7f5dc0197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.129+0000 7f5dc5828640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dc0106460 0x7f5dc0197c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43554/0 (socket says 172.21.15.94:43554) 2024-02-17T12:05:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.129+0000 7f5dc5828640 1 -- 172.21.15.94:0/2860712165 learned_addr learned my addr 172.21.15.94:0/2860712165 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:55.136 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.129+0000 7f5dc682a640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc019bfd0 0x7f5dc019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.130+0000 7f5dc6029640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dc0101390 0x7f5dc0197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.130+0000 7f5dc5828640 1 -- 172.21.15.94:0/2860712165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dc0101390 msgr2=0x7f5dc0197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.130+0000 7f5dc5828640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dc0101390 0x7f5dc0197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.130+0000 7f5dc5828640 1 -- 172.21.15.94:0/2860712165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc019bfd0 msgr2=0x7f5dc019c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.130+0000 7f5dc5828640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc019bfd0 0x7f5dc019c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.130+0000 7f5dc5828640 1 -- 172.21.15.94:0/2860712165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db8009900 con 0x7f5dc0106460 2024-02-17T12:05:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.130+0000 7f5dc682a640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc019bfd0 0x7f5dc019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:05:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.130+0000 7f5dc5828640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dc0106460 0x7f5dc0197c50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e9c0 tx=0x7f5db000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:55.137 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.130+0000 7f5db77fe640 1 -- 172.21.15.94:0/2860712165 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0009700 con 0x7f5dc0106460 2024-02-17T12:05:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.130+0000 7f5dc828d640 1 -- 172.21.15.94:0/2860712165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc01a4280 con 0x7f5dc0106460 2024-02-17T12:05:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.130+0000 7f5db77fe640 1 -- 172.21.15.94:0/2860712165 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5db000cd80 con 0x7f5dc0106460 2024-02-17T12:05:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.130+0000 7f5db77fe640 1 -- 172.21.15.94:0/2860712165 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0010430 con 0x7f5dc0106460 2024-02-17T12:05:55.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.130+0000 7f5dc828d640 1 -- 172.21.15.94:0/2860712165 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc01a4780 con 0x7f5dc0106460 2024-02-17T12:05:55.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.134+0000 7f5dcae5c740 1 Processor -- start 2024-02-17T12:05:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.134+0000 7f5db77fe640 1 -- 172.21.15.94:0/2860712165 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5db0005000 con 0x7f5dc0106460 2024-02-17T12:05:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.134+0000 7f5dcae5c740 1 -- start start 2024-02-17T12:05:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.134+0000 7f5dcae5c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc338aad0 0x556bc3389820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.134+0000 7f5dcae5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc3389d60 0x556bc338a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.134+0000 7f5dcae5c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 0x556bc33956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.134+0000 7f5dcae5c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc32f89f0 con 0x556bc338aad0 2024-02-17T12:05:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.134+0000 7f5dcae5c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc32f5f20 con 0x556bc33932e0 2024-02-17T12:05:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.134+0000 7f5dcae5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc32f1700 con 0x556bc3389d60 2024-02-17T12:05:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.134+0000 7f5dc682a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc338aad0 0x556bc3389820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.134+0000 7f5dc682a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc338aad0 0x556bc3389820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43560/0 (socket says 172.21.15.94:43560) 2024-02-17T12:05:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.134+0000 7f5dc682a640 1 -- 172.21.15.94:0/4132142144 learned_addr learned my addr 172.21.15.94:0/4132142144 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:55.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.134+0000 7f5dc5828640 1 --2- 172.21.15.94:0/4132142144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 0x556bc33956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5dc6029640 1 --2- 172.21.15.94:0/4132142144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc3389d60 0x556bc338a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5db77fe640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5dac0a22d0 0x7f5dac0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5dc682a640 1 -- 172.21.15.94:0/4132142144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc3389d60 msgr2=0x556bc338a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5dc682a640 1 --2- 172.21.15.94:0/4132142144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc3389d60 0x556bc338a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5dc682a640 1 -- 172.21.15.94:0/4132142144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 msgr2=0x556bc33956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5dc682a640 1 --2- 172.21.15.94:0/4132142144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 0x556bc33956d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5dc682a640 1 -- 172.21.15.94:0/4132142144 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bc32877b0 con 0x556bc338aad0 2024-02-17T12:05:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5dc6029640 1 --2- 172.21.15.94:0/4132142144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc3389d60 0x556bc338a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:05:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5dc682a640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5dac0a22d0 0x7f5dac0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5db77fe640 1 -- 172.21.15.94:0/2860712165 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5db0014070 con 0x7f5dc0106460 2024-02-17T12:05:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5dc682a640 1 --2- 172.21.15.94:0/4132142144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc338aad0 0x556bc3389820 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5db802c700 tx=0x7f5db8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a751a84826ba35a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5db4bfe640 1 -- 172.21.15.94:0/4132142144 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8034e50 con 0x556bc338aad0 2024-02-17T12:05:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5db4bfe640 1 -- 172.21.15.94:0/4132142144 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5db8003970 con 0x556bc338aad0 2024-02-17T12:05:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.135+0000 7f5db4bfe640 1 -- 172.21.15.94:0/4132142144 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8004430 con 0x556bc338aad0 2024-02-17T12:05:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.136+0000 7f5dcae5c740 1 -- 172.21.15.94:0/4132142144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc338aad0 msgr2=0x556bc3389820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.136+0000 7f5dc682a640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5dac0a22d0 0x7f5dac0a4790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5db804d000 tx=0x7f5db804a000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.136+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/4132142144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc338aad0 0x556bc3389820 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5db802c700 tx=0x7f5db8005f90 comp rx=0 tx=0).stop 2024-02-17T12:05:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.136+0000 7f5dcae5c740 1 -- 172.21.15.94:0/4132142144 shutdown_connections 2024-02-17T12:05:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.136+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/4132142144 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 0x556bc33956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.136+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/4132142144 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc3389d60 0x556bc338a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.136+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/4132142144 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc338aad0 0x556bc3389820 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.136+0000 7f5dcae5c740 1 -- 172.21.15.94:0/4132142144 >> 172.21.15.94:0/4132142144 conn(0x556bc3351550 msgr2=0x556bc3384a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.136+0000 7f5dcae5c740 1 -- 172.21.15.94:0/4132142144 shutdown_connections 2024-02-17T12:05:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.136+0000 7f5dcae5c740 1 -- 172.21.15.94:0/4132142144 wait complete. 2024-02-17T12:05:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.136+0000 7f5dcae5c740 1 -- start start 2024-02-17T12:05:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.136+0000 7f5dcae5c740 1 Processor -- start 2024-02-17T12:05:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dcae5c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 0x556bc33902d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dcae5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc338aad0 0x556bc3390810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dcae5c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 0x556bc3390d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dcae5c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc3296bf0 con 0x556bc3389d60 2024-02-17T12:05:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dc6029640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 0x556bc33902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dcae5c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc2fc5250 con 0x556bc33932e0 2024-02-17T12:05:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dcae5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc3277200 con 0x556bc338aad0 2024-02-17T12:05:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dc6029640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 0x556bc33902d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43572/0 (socket says 172.21.15.94:43572) 2024-02-17T12:05:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dc6029640 1 -- 172.21.15.94:0/3878311283 learned_addr learned my addr 172.21.15.94:0/3878311283 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dcae5c740 10 client.0 _ll_register_callbacks cb 0x556bc32e5d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dc5828640 1 --2- 172.21.15.94:0/3878311283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc338aad0 0x556bc3390810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dc6029640 1 -- 172.21.15.94:0/3878311283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc338aad0 msgr2=0x556bc3390810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dc6029640 1 --2- 172.21.15.94:0/3878311283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc338aad0 0x556bc3390810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dc6029640 1 -- 172.21.15.94:0/3878311283 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 msgr2=0x556bc3390d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dc6029640 1 --2- 172.21.15.94:0/3878311283 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 0x556bc3390d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dc6029640 1 -- 172.21.15.94:0/3878311283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc010e840 con 0x556bc3389d60 2024-02-17T12:05:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dc6029640 1 --2- 172.21.15.94:0/3878311283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 0x556bc33902d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00baa0 tx=0x7f5dbc00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b20a33e8af85be85 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5da9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5da9ffb640 1 -- 172.21.15.94:0/3878311283 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc01b640 con 0x556bc3389d60 2024-02-17T12:05:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5da9ffb640 1 -- 172.21.15.94:0/3878311283 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5dbc011440 con 0x556bc3389d60 2024-02-17T12:05:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.137+0000 7f5dcae5c740 1 -- 172.21.15.94:0/3878311283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bc2ff1600 con 0x556bc3389d60 2024-02-17T12:05:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.138+0000 7f5da9ffb640 1 -- 172.21.15.94:0/3878311283 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dbc003e70 con 0x556bc3389d60 2024-02-17T12:05:55.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.138+0000 7f5dcae5c740 10 client.17541 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.138+0000 7f5dcae5c740 1 -- 172.21.15.94:0/3878311283 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556bc32efd80 con 0x556bc3389d60 2024-02-17T12:05:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.138+0000 7f5da9ffb640 1 -- 172.21.15.94:0/3878311283 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556bc2ff1600 con 0x556bc3389d60 2024-02-17T12:05:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.138+0000 7f5da9ffb640 1 -- 172.21.15.94:0/3878311283 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5dbc011b00 con 0x556bc3389d60 2024-02-17T12:05:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.138+0000 7f5dcae5c740 10 client.17541 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.138+0000 7f5dcae5c740 10 client.17541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:55.159 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.138+0000 7f5dcae5c740 20 client.17541 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.138+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/3878311283 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556bc33a3690 0x556bc33a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.138+0000 7f5dcae5c740 4 client.17541 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:05:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.138+0000 7f5dcae5c740 1 -- 172.21.15.94:0/3878311283 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bc30421b0 con 0x556bc33a3690 2024-02-17T12:05:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.138+0000 7f5dc5828640 1 --2- 172.21.15.94:0/3878311283 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556bc33a3690 0x556bc33a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.139+0000 7f5da9ffb640 10 client.17541 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.139+0000 7f5dc5828640 1 --2- 172.21.15.94:0/3878311283 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556bc33a3690 0x556bc33a5ad0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1d40369050b0ea9 server_cookie=a24b6300fd76d4f4 in_seq=0 out_seq=0 2024-02-17T12:05:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.139+0000 7f5da9ffb640 1 -- 172.21.15.94:0/3878311283 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556bc30421b0 con 0x556bc33a3690 2024-02-17T12:05:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.139+0000 7f5da9ffb640 10 client.17541 handle_command_reply: tid=0 2024-02-17T12:05:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.140+0000 7f5dcae5c740 1 client.17541 shutdown 2024-02-17T12:05:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.140+0000 7f5dcae5c740 1 -- 172.21.15.94:0/3878311283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 msgr2=0x556bc33902d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:55.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.140+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/3878311283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 0x556bc33902d0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00baa0 tx=0x7f5dbc00bf60 comp rx=0 tx=0).stop 2024-02-17T12:05:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.140+0000 7f5dcae5c740 1 -- 172.21.15.94:0/3878311283 shutdown_connections 2024-02-17T12:05:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.140+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/3878311283 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556bc33a3690 0x556bc33a5ad0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.140+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/3878311283 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 0x556bc3390d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.140+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/3878311283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc338aad0 0x556bc3390810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.140+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/3878311283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 0x556bc33902d0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.140+0000 7f5dcae5c740 1 -- 172.21.15.94:0/3878311283 >> 172.21.15.94:0/3878311283 conn(0x556bc3351550 msgr2=0x556bc33867a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.140+0000 7f5da9ffb640 0 client.17541 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.141+0000 7f5dcae5c740 1 -- 172.21.15.94:0/3878311283 shutdown_connections 2024-02-17T12:05:55.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.141+0000 7f5dcae5c740 1 -- 172.21.15.94:0/3878311283 wait complete. 2024-02-17T12:05:55.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.141+0000 7f5dcae5c740 20 client.17541 trim_cache size 0 max 16384 2024-02-17T12:05:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.157+0000 7f5dcae5c740 1 Processor -- start 2024-02-17T12:05:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.157+0000 7f5dcae5c740 1 -- start start 2024-02-17T12:05:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dcae5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc3389d60 0x556bc3392a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dcae5c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc338aad0 0x556bc3390b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dcae5c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc33932e0 0x556bc338eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dcae5c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc3385ad0 con 0x556bc33932e0 2024-02-17T12:05:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dcae5c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc3389060 con 0x556bc338aad0 2024-02-17T12:05:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dcae5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc33aad30 con 0x556bc3389d60 2024-02-17T12:05:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dc682a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc33932e0 0x556bc338eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dc682a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc33932e0 0x556bc338eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43574/0 (socket says 172.21.15.94:43574) 2024-02-17T12:05:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dc682a640 1 -- 172.21.15.94:0/4263556442 learned_addr learned my addr 172.21.15.94:0/4263556442 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dc5828640 1 --2- 172.21.15.94:0/4263556442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc338aad0 0x556bc3390b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dc682a640 1 -- 172.21.15.94:0/4263556442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc3389d60 msgr2=0x556bc3392a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:05:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dc682a640 1 --2- 172.21.15.94:0/4263556442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc3389d60 0x556bc3392a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dc682a640 1 -- 172.21.15.94:0/4263556442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc338aad0 msgr2=0x556bc3390b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dc682a640 1 --2- 172.21.15.94:0/4263556442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc338aad0 0x556bc3390b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dc682a640 1 -- 172.21.15.94:0/4263556442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bc328c7b0 con 0x556bc33932e0 2024-02-17T12:05:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dc682a640 1 --2- 172.21.15.94:0/4263556442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc33932e0 0x556bc338eba0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5db802c6d0 tx=0x7f5db8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678705fe051a67dc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dabfff640 1 -- 172.21.15.94:0/4263556442 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8034e50 con 0x556bc33932e0 2024-02-17T12:05:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.158+0000 7f5dabfff640 1 -- 172.21.15.94:0/4263556442 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5db8004650 con 0x556bc33932e0 2024-02-17T12:05:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.159+0000 7f5dabfff640 1 -- 172.21.15.94:0/4263556442 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8032910 con 0x556bc33932e0 2024-02-17T12:05:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.159+0000 7f5dcae5c740 1 -- 172.21.15.94:0/4263556442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc33932e0 msgr2=0x556bc338eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.159+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/4263556442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc33932e0 0x556bc338eba0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5db802c6d0 tx=0x7f5db8005ef0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 -- 172.21.15.94:0/4263556442 shutdown_connections 2024-02-17T12:05:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/4263556442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc33932e0 0x556bc338eba0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/4263556442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc338aad0 0x556bc3390b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/4263556442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc3389d60 0x556bc3392a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 -- 172.21.15.94:0/4263556442 >> 172.21.15.94:0/4263556442 conn(0x556bc3351550 msgr2=0x556bc338b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 -- 172.21.15.94:0/4263556442 shutdown_connections 2024-02-17T12:05:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 -- 172.21.15.94:0/4263556442 wait complete. 2024-02-17T12:05:55.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 -- start start 2024-02-17T12:05:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 Processor -- start 2024-02-17T12:05:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 0x556bc33980f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc338aad0 0x556bc3398630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 0x556bc3398bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc33f8d60 con 0x556bc3389d60 2024-02-17T12:05:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc33f8ed0 con 0x556bc33932e0 2024-02-17T12:05:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc33f9090 con 0x556bc338aad0 2024-02-17T12:05:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.160+0000 7f5dcae5c740 10 client.0 _ll_register_callbacks cb 0x7f5db0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:05:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dc6029640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 0x556bc33980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dc6029640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 0x556bc33980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43584/0 (socket says 172.21.15.94:43584) 2024-02-17T12:05:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dc6029640 1 -- 172.21.15.94:0/1195924582 learned_addr learned my addr 172.21.15.94:0/1195924582 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:05:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dc5828640 1 --2- 172.21.15.94:0/1195924582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc338aad0 0x556bc3398630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dc682a640 1 --2- 172.21.15.94:0/1195924582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 0x556bc3398bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dc6029640 1 -- 172.21.15.94:0/1195924582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc338aad0 msgr2=0x556bc3398630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dc6029640 1 --2- 172.21.15.94:0/1195924582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc338aad0 0x556bc3398630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dc6029640 1 -- 172.21.15.94:0/1195924582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 msgr2=0x556bc3398bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:05:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dc6029640 1 --2- 172.21.15.94:0/1195924582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 0x556bc3398bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dc6029640 1 -- 172.21.15.94:0/1195924582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db8051070 con 0x556bc3389d60 2024-02-17T12:05:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dc6029640 1 --2- 172.21.15.94:0/1195924582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 0x556bc33980f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc010c20 tx=0x7f5dbc010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b6b9e7ddf4807f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:05:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5da9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:05:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5da9ffb640 1 -- 172.21.15.94:0/1195924582 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc004340 con 0x556bc3389d60 2024-02-17T12:05:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dcae5c740 1 -- 172.21.15.94:0/1195924582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5db8003970 con 0x556bc3389d60 2024-02-17T12:05:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5da9ffb640 1 -- 172.21.15.94:0/1195924582 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8541+0+0 (secure 0 0 0) 0x7f5dbc005850 con 0x556bc3389d60 2024-02-17T12:05:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5da9ffb640 1 -- 172.21.15.94:0/1195924582 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5dbc0039c0 con 0x556bc3389d60 2024-02-17T12:05:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dcae5c740 10 client.17547 fetch_fsmap learned FSMap version 27 2024-02-17T12:05:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5dcae5c740 1 -- 172.21.15.94:0/1195924582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556bc32b2540 con 0x556bc3389d60 2024-02-17T12:05:55.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.161+0000 7f5da9ffb640 1 -- 172.21.15.94:0/1195924582 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5dbc037080 con 0x556bc3389d60 2024-02-17T12:05:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.162+0000 7f5da9ffb640 1 -- 172.21.15.94:0/1195924582 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5dbc004d00 con 0x556bc3389d60 2024-02-17T12:05:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.162+0000 7f5dcae5c740 10 client.17547 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:05:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.162+0000 7f5dcae5c740 10 client.17547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:05:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.162+0000 7f5dcae5c740 20 client.17547 populate_metadata read hostname 'smithi094' 2024-02-17T12:05:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.162+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/1195924582 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556bc33fa910 0x556bc33fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:05:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.162+0000 7f5dcae5c740 4 client.17547 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:05:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.162+0000 7f5dcae5c740 1 -- 172.21.15.94:0/1195924582 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556bc302e550 con 0x556bc33fa910 2024-02-17T12:05:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.162+0000 7f5dc5828640 1 --2- 172.21.15.94:0/1195924582 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556bc33fa910 0x556bc33fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:05:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.162+0000 7f5dc5828640 1 --2- 172.21.15.94:0/1195924582 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556bc33fa910 0x556bc33fcd30 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6032ca25cc214a42 server_cookie=1ba2b870c16b81aa in_seq=0 out_seq=0 2024-02-17T12:05:55.181 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.162+0000 7f5da9ffb640 10 client.17547 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:55.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 455.0 s, 2050.4 tps, lat 48.147 ms stddev 52.075 2024-02-17T12:05:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.249+0000 7f5da9ffb640 1 -- 172.21.15.94:0/1195924582 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556bc302e550 con 0x556bc33fa910 2024-02-17T12:05:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.249+0000 7f5da9ffb640 10 client.17547 handle_command_reply: tid=0 2024-02-17T12:05:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.249+0000 7f5dcae5c740 1 client.17547 shutdown 2024-02-17T12:05:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.249+0000 7f5dcae5c740 1 -- 172.21.15.94:0/1195924582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 msgr2=0x556bc33980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.249+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/1195924582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 0x556bc33980f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc010c20 tx=0x7f5dbc010c50 comp rx=0 tx=0).stop 2024-02-17T12:05:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dcae5c740 1 -- 172.21.15.94:0/1195924582 shutdown_connections 2024-02-17T12:05:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/1195924582 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556bc33fa910 0x556bc33fcd30 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/1195924582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556bc33932e0 0x556bc3398bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/1195924582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556bc338aad0 0x556bc3398630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dcae5c740 1 --2- 172.21.15.94:0/1195924582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556bc3389d60 0x556bc33980f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5da9ffb640 0 client.17547 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:05:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dcae5c740 1 -- 172.21.15.94:0/1195924582 >> 172.21.15.94:0/1195924582 conn(0x556bc3351550 msgr2=0x556bc338eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dcae5c740 1 -- 172.21.15.94:0/1195924582 shutdown_connections 2024-02-17T12:05:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dcae5c740 1 -- 172.21.15.94:0/1195924582 wait complete. 2024-02-17T12:05:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dcae5c740 20 client.17547 trim_cache size 0 max 16384 2024-02-17T12:05:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:05:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:05:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:05:55.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:05:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dc828d640 1 -- 172.21.15.94:0/2860712165 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5dac0a22d0 msgr2=0x7f5dac0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dc828d640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5dac0a22d0 0x7f5dac0a4790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5db804d000 tx=0x7f5db804a000 comp rx=0 tx=0).stop 2024-02-17T12:05:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dc828d640 1 -- 172.21.15.94:0/2860712165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dc0106460 msgr2=0x7f5dc0197c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:05:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dc828d640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dc0106460 0x7f5dc0197c50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e9c0 tx=0x7f5db000ee80 comp rx=0 tx=0).stop 2024-02-17T12:05:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dc828d640 1 -- 172.21.15.94:0/2860712165 shutdown_connections 2024-02-17T12:05:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dc828d640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5dac0a22d0 0x7f5dac0a4790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dc828d640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5dc019bfd0 0x7f5dc019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dc828d640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5dc0106460 0x7f5dc0197c50 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dc828d640 1 --2- 172.21.15.94:0/2860712165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5dc0101390 0x7f5dc0197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:05:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dc828d640 1 -- 172.21.15.94:0/2860712165 >> 172.21.15.94:0/2860712165 conn(0x7f5dc00fcc10 msgr2=0x7f5dc00feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:05:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dc828d640 1 -- 172.21.15.94:0/2860712165 shutdown_connections 2024-02-17T12:05:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:05:55.250+0000 7f5dc828d640 1 -- 172.21.15.94:0/2860712165 wait complete. 2024-02-17T12:05:56.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:55 smithi189 ceph-mon[36588]: pgmap v239: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.1 MiB/s wr, 822 op/s 2024-02-17T12:05:56.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:55 smithi125 ceph-mon[36819]: pgmap v239: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.1 MiB/s wr, 822 op/s 2024-02-17T12:05:56.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:55 smithi094 ceph-mon[32499]: pgmap v239: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.1 MiB/s wr, 822 op/s 2024-02-17T12:05:58.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:57 smithi189 ceph-mon[36588]: pgmap v240: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.3 MiB/s wr, 547 op/s 2024-02-17T12:05:58.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:57 smithi125 ceph-mon[36819]: pgmap v240: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.3 MiB/s wr, 547 op/s 2024-02-17T12:05:58.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:57 smithi094 ceph-mon[32499]: pgmap v240: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 5.3 MiB/s wr, 547 op/s 2024-02-17T12:05:59.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:05:59 smithi189 ceph-mon[36588]: pgmap v241: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.1 MiB/s wr, 835 op/s 2024-02-17T12:05:59.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:05:59 smithi125 ceph-mon[36819]: pgmap v241: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.1 MiB/s wr, 835 op/s 2024-02-17T12:05:59.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:05:59 smithi094 ceph-mon[32499]: pgmap v241: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 9.1 MiB/s wr, 835 op/s 2024-02-17T12:06:00.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 460.0 s, 1803.6 tps, lat 55.768 ms stddev 59.187 2024-02-17T12:06:02.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:01 smithi189 ceph-mon[36588]: pgmap v242: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 555 op/s 2024-02-17T12:06:02.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:01 smithi125 ceph-mon[36819]: pgmap v242: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 555 op/s 2024-02-17T12:06:02.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:01 smithi094 ceph-mon[32499]: pgmap v242: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.3 MiB/s wr, 555 op/s 2024-02-17T12:06:04.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:03 smithi189 ceph-mon[36588]: pgmap v243: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.7 MiB/s wr, 607 op/s 2024-02-17T12:06:04.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:03 smithi125 ceph-mon[36819]: pgmap v243: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.7 MiB/s wr, 607 op/s 2024-02-17T12:06:04.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:03 smithi094 ceph-mon[32499]: pgmap v243: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 6.7 MiB/s wr, 607 op/s 2024-02-17T12:06:05.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 465.0 s, 287.4 tps, lat 324.505 ms stddev 298.617 2024-02-17T12:06:06.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:05 smithi189 ceph-mon[36588]: pgmap v244: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.9 MiB/s wr, 761 op/s 2024-02-17T12:06:06.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:05 smithi125 ceph-mon[36819]: pgmap v244: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.9 MiB/s wr, 761 op/s 2024-02-17T12:06:06.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:05 smithi094 ceph-mon[32499]: pgmap v244: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 7.9 MiB/s wr, 761 op/s 2024-02-17T12:06:07.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:07 smithi189 ceph-mon[36588]: pgmap v245: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.7 MiB/s wr, 527 op/s 2024-02-17T12:06:07.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:07 smithi125 ceph-mon[36819]: pgmap v245: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.7 MiB/s wr, 527 op/s 2024-02-17T12:06:07.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:07 smithi094 ceph-mon[32499]: pgmap v245: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.7 MiB/s wr, 527 op/s 2024-02-17T12:06:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.382+0000 7f87525e2640 1 -- 172.21.15.94:0/2396839694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f874c10b770 msgr2=0x7f874c10dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.382+0000 7f87525e2640 1 --2- 172.21.15.94:0/2396839694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f874c10b770 0x7f874c10dbf0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8744009ca0 tx=0x7f874402c180 comp rx=0 tx=0).stop 2024-02-17T12:06:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.382+0000 7f87525e2640 1 -- 172.21.15.94:0/2396839694 shutdown_connections 2024-02-17T12:06:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.382+0000 7f87525e2640 1 --2- 172.21.15.94:0/2396839694 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f874c10b770 0x7f874c10dbf0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.382+0000 7f87525e2640 1 --2- 172.21.15.94:0/2396839694 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c102cc0 0x7f874c10b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.382+0000 7f87525e2640 1 --2- 172.21.15.94:0/2396839694 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c1023a0 0x7f874c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.390 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.382+0000 7f87525e2640 1 -- 172.21.15.94:0/2396839694 >> 172.21.15.94:0/2396839694 conn(0x7f874c077290 msgr2=0x7f874c0776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.382+0000 7f87525e2640 1 -- 172.21.15.94:0/2396839694 shutdown_connections 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.382+0000 7f87525e2640 1 -- 172.21.15.94:0/2396839694 wait complete. 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.383+0000 7f87525e2640 1 Processor -- start 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.383+0000 7f87525e2640 1 -- start start 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.383+0000 7f87525e2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c1023a0 0x7f874c10a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.383+0000 7f87525e2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c102cc0 0x7f874c10a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.383+0000 7f87525e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f874c10b770 0x7f874c1a2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.383+0000 7f87525e2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1a2c20 con 0x7f874c10b770 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.383+0000 7f87525e2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1a2d90 con 0x7f874c102cc0 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.383+0000 7f87525e2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c1a2f50 con 0x7f874c1023a0 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.383+0000 7f8750b7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f874c10b770 0x7f874c1a2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f874b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c102cc0 0x7f874c10a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f8750b7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f874c10b770 0x7f874c1a2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41218/0 (socket says 172.21.15.94:41218) 2024-02-17T12:06:08.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f8750b7f640 1 -- 172.21.15.94:0/4226030672 learned_addr learned my addr 172.21.15.94:0/4226030672 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f874bfff640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c1023a0 0x7f874c10a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f874b7fe640 1 -- 172.21.15.94:0/4226030672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c1023a0 msgr2=0x7f874c10a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f874b7fe640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c1023a0 0x7f874c10a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f874b7fe640 1 -- 172.21.15.94:0/4226030672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f874c10b770 msgr2=0x7f874c1a2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f874b7fe640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f874c10b770 0x7f874c1a2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f874b7fe640 1 -- 172.21.15.94:0/4226030672 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8744009900 con 0x7f874c102cc0 2024-02-17T12:06:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f8750b7f640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f874c10b770 0x7f874c1a2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f874b7fe640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c102cc0 0x7f874c10a810 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f873c00e9f0 tx=0x7f873c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f87497fa640 1 -- 172.21.15.94:0/4226030672 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c00cc40 con 0x7f874c102cc0 2024-02-17T12:06:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f87525e2640 1 -- 172.21.15.94:0/4226030672 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c1a3230 con 0x7f874c102cc0 2024-02-17T12:06:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f87497fa640 1 -- 172.21.15.94:0/4226030672 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f873c00cda0 con 0x7f874c102cc0 2024-02-17T12:06:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f87497fa640 1 -- 172.21.15.94:0/4226030672 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873c004a00 con 0x7f874c102cc0 2024-02-17T12:06:08.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.384+0000 7f87525e2640 1 -- 172.21.15.94:0/4226030672 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c1aaa90 con 0x7f874c102cc0 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.387+0000 7f87497fa640 1 -- 172.21.15.94:0/4226030672 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f873c004b60 con 0x7f874c102cc0 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.387+0000 7f87551b5740 1 Processor -- start 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.387+0000 7f87551b5740 1 -- start start 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f87551b5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6acccad0 0x55fd6accb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f87551b5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6accbd60 0x55fd6accc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f87551b5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acd52e0 0x55fd6acd76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f87497fa640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f87340a23a0 0x7f87340a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f87551b5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6ac3a9f0 con 0x55fd6acd52e0 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f87551b5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6ac37f20 con 0x55fd6acccad0 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f87551b5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6ac33700 con 0x55fd6accbd60 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f8750b7f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6acccad0 0x55fd6accb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f8750b7f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6acccad0 0x55fd6accb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46220/0 (socket says 172.21.15.94:46220) 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f8750b7f640 1 -- 172.21.15.94:0/2787641291 learned_addr learned my addr 172.21.15.94:0/2787641291 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:08.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f874bfff640 1 --2- 172.21.15.94:0/2787641291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6accbd60 0x55fd6accc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f874b7fe640 1 --2- 172.21.15.94:0/2787641291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acd52e0 0x55fd6acd76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f8750b7f640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f87340a23a0 0x7f87340a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f87497fa640 1 -- 172.21.15.94:0/4226030672 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f873c014070 con 0x7f874c102cc0 2024-02-17T12:06:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f8750b7f640 1 -- 172.21.15.94:0/2787641291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6accbd60 msgr2=0x55fd6accc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f8750b7f640 1 --2- 172.21.15.94:0/2787641291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6accbd60 0x55fd6accc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f8750b7f640 1 -- 172.21.15.94:0/2787641291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acd52e0 msgr2=0x55fd6acd76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f8750b7f640 1 --2- 172.21.15.94:0/2787641291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acd52e0 0x55fd6acd76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.388+0000 7f8750b7f640 1 -- 172.21.15.94:0/2787641291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd6abc97b0 con 0x55fd6acccad0 2024-02-17T12:06:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.389+0000 7f8750b7f640 1 --2- 172.21.15.94:0/2787641291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6acccad0 0x55fd6accb820 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f87440349e0 tx=0x7f8744034a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64cc286d9bb567ca server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.389+0000 7f87327fc640 1 -- 172.21.15.94:0/2787641291 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744032d60 con 0x55fd6acccad0 2024-02-17T12:06:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.389+0000 7f87327fc640 1 -- 172.21.15.94:0/2787641291 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8744032ec0 con 0x55fd6acccad0 2024-02-17T12:06:08.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.389+0000 7f87327fc640 1 -- 172.21.15.94:0/2787641291 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744042770 con 0x55fd6acccad0 2024-02-17T12:06:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.390+0000 7f8750b7f640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f87340a23a0 0x7f87340a4860 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f874404d000 tx=0x7f8744049000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.390+0000 7f87551b5740 1 -- 172.21.15.94:0/2787641291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6acccad0 msgr2=0x55fd6accb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.390+0000 7f87551b5740 1 --2- 172.21.15.94:0/2787641291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6acccad0 0x55fd6accb820 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f87440349e0 tx=0x7f8744034a10 comp rx=0 tx=0).stop 2024-02-17T12:06:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.390+0000 7f87551b5740 1 -- 172.21.15.94:0/2787641291 shutdown_connections 2024-02-17T12:06:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.390+0000 7f87551b5740 1 --2- 172.21.15.94:0/2787641291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acd52e0 0x55fd6acd76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.390+0000 7f87551b5740 1 --2- 172.21.15.94:0/2787641291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6accbd60 0x55fd6accc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.390+0000 7f87551b5740 1 --2- 172.21.15.94:0/2787641291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6acccad0 0x55fd6accb820 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.390+0000 7f87551b5740 1 -- 172.21.15.94:0/2787641291 >> 172.21.15.94:0/2787641291 conn(0x55fd6ac93550 msgr2=0x55fd6acc6a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.390+0000 7f87551b5740 1 -- 172.21.15.94:0/2787641291 shutdown_connections 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.390+0000 7f87551b5740 1 -- 172.21.15.94:0/2787641291 wait complete. 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.390+0000 7f87551b5740 1 -- start start 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.391+0000 7f87551b5740 1 Processor -- start 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.391+0000 7f87551b5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6accbd60 0x55fd6acd2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.391+0000 7f87551b5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acccad0 0x55fd6acd2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.391+0000 7f87551b5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acd52e0 0x55fd6acd2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.391+0000 7f87551b5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6abd8bf0 con 0x55fd6acccad0 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.391+0000 7f87551b5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6a907250 con 0x55fd6accbd60 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.391+0000 7f874bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6accbd60 0x55fd6acd2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.391+0000 7f87551b5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6abb9200 con 0x55fd6acd52e0 2024-02-17T12:06:08.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.391+0000 7f87551b5740 10 client.0 _ll_register_callbacks cb 0x55fd6ac27d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.391+0000 7f874bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6accbd60 0x55fd6acd2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46228/0 (socket says 172.21.15.94:46228) 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.391+0000 7f874bfff640 1 -- 172.21.15.94:0/631747091 learned_addr learned my addr 172.21.15.94:0/631747091 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f874b7fe640 1 --2- 172.21.15.94:0/631747091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acccad0 0x55fd6acd2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f874bfff640 1 -- 172.21.15.94:0/631747091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acd52e0 msgr2=0x55fd6acd2d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f874bfff640 1 --2- 172.21.15.94:0/631747091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acd52e0 0x55fd6acd2d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f874bfff640 1 -- 172.21.15.94:0/631747091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acccad0 msgr2=0x55fd6acd2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f874bfff640 1 --2- 172.21.15.94:0/631747091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acccad0 0x55fd6acd2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f874bfff640 1 -- 172.21.15.94:0/631747091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c10e870 con 0x55fd6accbd60 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f874bfff640 1 --2- 172.21.15.94:0/631747091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6accbd60 0x55fd6acd2300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f874000bfc0 tx=0x7f87400118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=843aa021b8e5c15c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f872b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f872b7fe640 1 -- 172.21.15.94:0/631747091 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740016070 con 0x55fd6accbd60 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f872b7fe640 1 -- 172.21.15.94:0/631747091 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8740004020 con 0x55fd6accbd60 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f87551b5740 1 -- 172.21.15.94:0/631747091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd6a933600 con 0x55fd6accbd60 2024-02-17T12:06:08.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f872b7fe640 1 -- 172.21.15.94:0/631747091 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87400041e0 con 0x55fd6accbd60 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f87551b5740 10 client.27032 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.392+0000 7f87551b5740 1 -- 172.21.15.94:0/631747091 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fd6ac31d80 con 0x55fd6accbd60 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.393+0000 7f872b7fe640 1 -- 172.21.15.94:0/631747091 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8740026020 con 0x55fd6accbd60 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.394+0000 7f872b7fe640 1 -- 172.21.15.94:0/631747091 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f87400107c0 con 0x55fd6accbd60 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.394+0000 7f87551b5740 10 client.27032 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.394+0000 7f87551b5740 10 client.27032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.394+0000 7f87551b5740 20 client.27032 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.394+0000 7f87551b5740 1 --2- 172.21.15.94:0/631747091 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd6ace5620 0x55fd6ace7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.394+0000 7f87551b5740 4 client.27032 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.394+0000 7f87551b5740 1 -- 172.21.15.94:0/631747091 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd6a9841b0 con 0x55fd6ace5620 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.395+0000 7f874b7fe640 1 --2- 172.21.15.94:0/631747091 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd6ace5620 0x55fd6ace7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.395+0000 7f874b7fe640 1 --2- 172.21.15.94:0/631747091 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd6ace5620 0x55fd6ace7a40 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edb5fded578fb51f server_cookie=fedecc782f0c9be7 in_seq=0 out_seq=0 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.395+0000 7f872b7fe640 10 client.27032 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:08.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.396+0000 7f872b7fe640 1 -- 172.21.15.94:0/631747091 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd6a9841b0 con 0x55fd6ace5620 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.396+0000 7f872b7fe640 10 client.27032 handle_command_reply: tid=0 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.396+0000 7f87551b5740 1 client.27032 shutdown 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.396+0000 7f87551b5740 1 -- 172.21.15.94:0/631747091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6accbd60 msgr2=0x55fd6acd2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.396+0000 7f87551b5740 1 --2- 172.21.15.94:0/631747091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6accbd60 0x55fd6acd2300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f874000bfc0 tx=0x7f87400118d0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.396+0000 7f87551b5740 1 -- 172.21.15.94:0/631747091 shutdown_connections 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.396+0000 7f87551b5740 1 --2- 172.21.15.94:0/631747091 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd6ace5620 0x55fd6ace7a40 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.396+0000 7f87551b5740 1 --2- 172.21.15.94:0/631747091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acd52e0 0x55fd6acd2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.396+0000 7f872b7fe640 0 client.27032 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.397+0000 7f87551b5740 1 --2- 172.21.15.94:0/631747091 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acccad0 0x55fd6acd2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.397+0000 7f87551b5740 1 --2- 172.21.15.94:0/631747091 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6accbd60 0x55fd6acd2300 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.397+0000 7f87551b5740 1 -- 172.21.15.94:0/631747091 >> 172.21.15.94:0/631747091 conn(0x55fd6ac93550 msgr2=0x55fd6acc87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.397+0000 7f87551b5740 1 -- 172.21.15.94:0/631747091 shutdown_connections 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.397+0000 7f87551b5740 1 -- 172.21.15.94:0/631747091 wait complete. 2024-02-17T12:06:08.402 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.397+0000 7f87551b5740 20 client.27032 trim_cache size 0 max 16384 2024-02-17T12:06:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.415+0000 7f87551b5740 1 Processor -- start 2024-02-17T12:06:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.415+0000 7f87551b5740 1 -- start start 2024-02-17T12:06:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.415+0000 7f87551b5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6accbd60 0x55fd6acd0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.415+0000 7f87551b5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acccad0 0x55fd6acd1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.415+0000 7f87551b5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6acd52e0 0x55fd6accf560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.415+0000 7f87551b5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6a9841b0 con 0x55fd6accbd60 2024-02-17T12:06:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.415+0000 7f87551b5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6acc6310 con 0x55fd6acd52e0 2024-02-17T12:06:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.415+0000 7f87551b5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6ace2140 con 0x55fd6acccad0 2024-02-17T12:06:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.415+0000 7f874bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6accbd60 0x55fd6acd0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.415+0000 7f874bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6accbd60 0x55fd6acd0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41248/0 (socket says 172.21.15.94:41248) 2024-02-17T12:06:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.415+0000 7f874bfff640 1 -- 172.21.15.94:0/1422397532 learned_addr learned my addr 172.21.15.94:0/1422397532 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.415+0000 7f874b7fe640 1 --2- 172.21.15.94:0/1422397532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acccad0 0x55fd6acd1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.415+0000 7f8750b7f640 1 --2- 172.21.15.94:0/1422397532 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6acd52e0 0x55fd6accf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.416+0000 7f874bfff640 1 -- 172.21.15.94:0/1422397532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acccad0 msgr2=0x55fd6acd1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.416+0000 7f874bfff640 1 --2- 172.21.15.94:0/1422397532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acccad0 0x55fd6acd1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.416+0000 7f874bfff640 1 -- 172.21.15.94:0/1422397532 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6acd52e0 msgr2=0x55fd6accf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.416+0000 7f874bfff640 1 --2- 172.21.15.94:0/1422397532 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6acd52e0 0x55fd6accf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.416+0000 7f874bfff640 1 -- 172.21.15.94:0/1422397532 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd6abce7b0 con 0x55fd6accbd60 2024-02-17T12:06:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.416+0000 7f874bfff640 1 --2- 172.21.15.94:0/1422397532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6accbd60 0x55fd6acd0bd0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8740003610 tx=0x7f874000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fe86af26802a66 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.416+0000 7f8731ffb640 1 -- 172.21.15.94:0/1422397532 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740016070 con 0x55fd6accbd60 2024-02-17T12:06:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.416+0000 7f8731ffb640 1 -- 172.21.15.94:0/1422397532 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8740026030 con 0x55fd6accbd60 2024-02-17T12:06:08.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.416+0000 7f8731ffb640 1 -- 172.21.15.94:0/1422397532 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740004990 con 0x55fd6accbd60 2024-02-17T12:06:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.417+0000 7f87551b5740 1 -- 172.21.15.94:0/1422397532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6accbd60 msgr2=0x55fd6acd0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.417+0000 7f87551b5740 1 --2- 172.21.15.94:0/1422397532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6accbd60 0x55fd6acd0bd0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f8740003610 tx=0x7f874000b940 comp rx=0 tx=0).stop 2024-02-17T12:06:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.417+0000 7f87551b5740 1 -- 172.21.15.94:0/1422397532 shutdown_connections 2024-02-17T12:06:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.417+0000 7f87551b5740 1 --2- 172.21.15.94:0/1422397532 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6acd52e0 0x55fd6accf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.417+0000 7f87551b5740 1 --2- 172.21.15.94:0/1422397532 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acccad0 0x55fd6acd1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.417+0000 7f87551b5740 1 --2- 172.21.15.94:0/1422397532 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6accbd60 0x55fd6acd0bd0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.417+0000 7f87551b5740 1 -- 172.21.15.94:0/1422397532 >> 172.21.15.94:0/1422397532 conn(0x55fd6ac93550 msgr2=0x55fd6acc6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:08.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.417+0000 7f87551b5740 1 -- 172.21.15.94:0/1422397532 shutdown_connections 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.417+0000 7f87551b5740 1 -- 172.21.15.94:0/1422397532 wait complete. 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.417+0000 7f87551b5740 1 -- start start 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.417+0000 7f87551b5740 1 Processor -- start 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f87551b5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6accbd60 0x55fd6acdaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f87551b5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acccad0 0x55fd6acdafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f87551b5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acd52e0 0x55fd6acdb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f87551b5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6acdca00 con 0x55fd6acd52e0 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f87551b5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6ad3bb80 con 0x55fd6accbd60 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f87551b5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd6ad3bd40 con 0x55fd6acccad0 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f87551b5740 10 client.0 _ll_register_callbacks cb 0x7f8744037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f8750b7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acd52e0 0x55fd6acdb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f8750b7f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acd52e0 0x55fd6acdb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41260/0 (socket says 172.21.15.94:41260) 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f8750b7f640 1 -- 172.21.15.94:0/1202576011 learned_addr learned my addr 172.21.15.94:0/1202576011 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f874bfff640 1 --2- 172.21.15.94:0/1202576011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6accbd60 0x55fd6acdaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f874b7fe640 1 --2- 172.21.15.94:0/1202576011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acccad0 0x55fd6acdafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f8750b7f640 1 -- 172.21.15.94:0/1202576011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acccad0 msgr2=0x55fd6acdafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f8750b7f640 1 --2- 172.21.15.94:0/1202576011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acccad0 0x55fd6acdafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f8750b7f640 1 -- 172.21.15.94:0/1202576011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6accbd60 msgr2=0x55fd6acdaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f8750b7f640 1 --2- 172.21.15.94:0/1202576011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6accbd60 0x55fd6acdaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f8750b7f640 1 -- 172.21.15.94:0/1202576011 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd6ac31d80 con 0x55fd6acd52e0 2024-02-17T12:06:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f8750b7f640 1 --2- 172.21.15.94:0/1202576011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acd52e0 0x55fd6acdb4e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f874402c760 tx=0x7f8744042650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a10630e8b848ee server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f872b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f872b7fe640 1 -- 172.21.15.94:0/1202576011 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744059070 con 0x55fd6acd52e0 2024-02-17T12:06:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f872b7fe640 1 -- 172.21.15.94:0/1202576011 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8744005bd0 con 0x55fd6acd52e0 2024-02-17T12:06:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.418+0000 7f87551b5740 1 -- 172.21.15.94:0/1202576011 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8744032ec0 con 0x55fd6acd52e0 2024-02-17T12:06:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.419+0000 7f872b7fe640 1 -- 172.21.15.94:0/1202576011 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8744042740 con 0x55fd6acd52e0 2024-02-17T12:06:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.419+0000 7f87551b5740 10 client.17565 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:08.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.419+0000 7f87551b5740 1 -- 172.21.15.94:0/1202576011 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fd6abf4540 con 0x55fd6acd52e0 2024-02-17T12:06:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.419+0000 7f872b7fe640 1 -- 172.21.15.94:0/1202576011 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f874406b020 con 0x55fd6acd52e0 2024-02-17T12:06:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.419+0000 7f872b7fe640 1 -- 172.21.15.94:0/1202576011 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f87440042b0 con 0x55fd6acd52e0 2024-02-17T12:06:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.419+0000 7f87551b5740 10 client.17565 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.419+0000 7f87551b5740 10 client.17565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.419+0000 7f87551b5740 20 client.17565 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.419+0000 7f87551b5740 1 --2- 172.21.15.94:0/1202576011 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd6ace5620 0x55fd6ad3f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.419+0000 7f87551b5740 4 client.17565 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:06:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.419+0000 7f87551b5740 1 -- 172.21.15.94:0/1202576011 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd6a970550 con 0x55fd6ace5620 2024-02-17T12:06:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.419+0000 7f874bfff640 1 --2- 172.21.15.94:0/1202576011 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd6ace5620 0x55fd6ad3f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.420+0000 7f874bfff640 1 --2- 172.21.15.94:0/1202576011 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd6ace5620 0x55fd6ad3f300 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=859a69f6a2de1957 server_cookie=aae6d2dc84a285bb in_seq=0 out_seq=0 2024-02-17T12:06:08.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.420+0000 7f872b7fe640 10 client.17565 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.445+0000 7f872b7fe640 1 -- 172.21.15.94:0/1202576011 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd6a970550 con 0x55fd6ace5620 2024-02-17T12:06:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.445+0000 7f872b7fe640 10 client.17565 handle_command_reply: tid=0 2024-02-17T12:06:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.447+0000 7f87551b5740 1 client.17565 shutdown 2024-02-17T12:06:08.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.447+0000 7f87551b5740 1 -- 172.21.15.94:0/1202576011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acd52e0 msgr2=0x55fd6acdb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.447+0000 7f87551b5740 1 --2- 172.21.15.94:0/1202576011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acd52e0 0x55fd6acdb4e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f874402c760 tx=0x7f8744042650 comp rx=0 tx=0).stop 2024-02-17T12:06:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.448+0000 7f87551b5740 1 -- 172.21.15.94:0/1202576011 shutdown_connections 2024-02-17T12:06:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.448+0000 7f87551b5740 1 --2- 172.21.15.94:0/1202576011 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd6ace5620 0x55fd6ad3f300 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.448+0000 7f87551b5740 1 --2- 172.21.15.94:0/1202576011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd6acd52e0 0x55fd6acdb4e0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.448+0000 7f87551b5740 1 --2- 172.21.15.94:0/1202576011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd6acccad0 0x55fd6acdafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.448+0000 7f87551b5740 1 --2- 172.21.15.94:0/1202576011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd6accbd60 0x55fd6acdaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.448+0000 7f87551b5740 1 -- 172.21.15.94:0/1202576011 >> 172.21.15.94:0/1202576011 conn(0x55fd6ac93550 msgr2=0x55fd6acc6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:08.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.448+0000 7f872b7fe640 0 client.17565 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.448+0000 7f87551b5740 1 -- 172.21.15.94:0/1202576011 shutdown_connections 2024-02-17T12:06:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.448+0000 7f87551b5740 1 -- 172.21.15.94:0/1202576011 wait complete. 2024-02-17T12:06:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.449+0000 7f87551b5740 20 client.17565 trim_cache size 0 max 16384 2024-02-17T12:06:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:06:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:06:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:06:08.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:06:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.449+0000 7f87525e2640 1 -- 172.21.15.94:0/4226030672 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f87340a23a0 msgr2=0x7f87340a4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.449+0000 7f87525e2640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f87340a23a0 0x7f87340a4860 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f874404d000 tx=0x7f8744049000 comp rx=0 tx=0).stop 2024-02-17T12:06:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.449+0000 7f87525e2640 1 -- 172.21.15.94:0/4226030672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c102cc0 msgr2=0x7f874c10a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.449+0000 7f87525e2640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c102cc0 0x7f874c10a810 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f873c00e9f0 tx=0x7f873c00eeb0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.449+0000 7f87525e2640 1 -- 172.21.15.94:0/4226030672 shutdown_connections 2024-02-17T12:06:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.449+0000 7f87525e2640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f87340a23a0 0x7f87340a4860 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.449+0000 7f87525e2640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f874c10b770 0x7f874c1a2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.449+0000 7f87525e2640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f874c102cc0 0x7f874c10a810 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.449+0000 7f87525e2640 1 --2- 172.21.15.94:0/4226030672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f874c1023a0 0x7f874c10a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.449+0000 7f87525e2640 1 -- 172.21.15.94:0/4226030672 >> 172.21.15.94:0/4226030672 conn(0x7f874c077290 msgr2=0x7f874c0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.449+0000 7f87525e2640 1 -- 172.21.15.94:0/4226030672 shutdown_connections 2024-02-17T12:06:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:08.449+0000 7f87525e2640 1 -- 172.21.15.94:0/4226030672 wait complete. 2024-02-17T12:06:08.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:06:08.780 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:06:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.891+0000 7fbd7918c640 1 -- 172.21.15.94:0/1253364314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd74101d10 msgr2=0x7fbd74105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.891+0000 7fbd7918c640 1 --2- 172.21.15.94:0/1253364314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd74101d10 0x7fbd74105d50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009930 tx=0x7fbd6802bc30 comp rx=0 tx=0).stop 2024-02-17T12:06:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.891+0000 7fbd7918c640 1 -- 172.21.15.94:0/1253364314 shutdown_connections 2024-02-17T12:06:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.891+0000 7fbd7918c640 1 --2- 172.21.15.94:0/1253364314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd74106430 0x7fbd7410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.891+0000 7fbd7918c640 1 --2- 172.21.15.94:0/1253364314 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd74101d10 0x7fbd74105d50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.891+0000 7fbd7918c640 1 --2- 172.21.15.94:0/1253364314 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd74101360 0x7fbd74101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.891+0000 7fbd7918c640 1 -- 172.21.15.94:0/1253364314 >> 172.21.15.94:0/1253364314 conn(0x7fbd740fcc20 msgr2=0x7fbd740ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.891+0000 7fbd7918c640 1 -- 172.21.15.94:0/1253364314 shutdown_connections 2024-02-17T12:06:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.891+0000 7fbd7918c640 1 -- 172.21.15.94:0/1253364314 wait complete. 2024-02-17T12:06:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.892+0000 7fbd7918c640 1 Processor -- start 2024-02-17T12:06:08.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.892+0000 7fbd7918c640 1 -- start start 2024-02-17T12:06:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.892+0000 7fbd7918c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd74101360 0x7fbd7419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.892+0000 7fbd7918c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd74106430 0x7fbd7419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.892+0000 7fbd7918c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd741976a0 0x7fbd74197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.892+0000 7fbd7918c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd74198170 con 0x7fbd741976a0 2024-02-17T12:06:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.892+0000 7fbd7918c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd741982e0 con 0x7fbd74101360 2024-02-17T12:06:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.892+0000 7fbd7918c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd741984a0 con 0x7fbd74106430 2024-02-17T12:06:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.892+0000 7fbd7359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd741976a0 0x7fbd74197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd72d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd74101360 0x7fbd7419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd7359e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd741976a0 0x7fbd74197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41278/0 (socket says 172.21.15.94:41278) 2024-02-17T12:06:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd7359e640 1 -- 172.21.15.94:0/1710505411 learned_addr learned my addr 172.21.15.94:0/1710505411 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd72d9d640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd74101360 0x7fbd7419d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46264/0 (socket says 172.21.15.94:46264) 2024-02-17T12:06:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd7259c640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd74106430 0x7fbd7419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd7359e640 1 -- 172.21.15.94:0/1710505411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd74106430 msgr2=0x7fbd7419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd7359e640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd74106430 0x7fbd7419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd7359e640 1 -- 172.21.15.94:0/1710505411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd74101360 msgr2=0x7fbd7419d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd7359e640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd74101360 0x7fbd7419d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd7359e640 1 -- 172.21.15.94:0/1710505411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd68009560 con 0x7fbd741976a0 2024-02-17T12:06:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd72d9d640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd74101360 0x7fbd7419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd7359e640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd741976a0 0x7fbd74197b20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00eaa0 tx=0x7fbd6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd5bfff640 1 -- 172.21.15.94:0/1710505411 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00cdb0 con 0x7fbd741976a0 2024-02-17T12:06:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd7918c640 1 -- 172.21.15.94:0/1710505411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd741a43f0 con 0x7fbd741976a0 2024-02-17T12:06:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd5bfff640 1 -- 172.21.15.94:0/1710505411 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd6c005020 con 0x7fbd741976a0 2024-02-17T12:06:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd5bfff640 1 -- 172.21.15.94:0/1710505411 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0116b0 con 0x7fbd741976a0 2024-02-17T12:06:08.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.893+0000 7fbd7918c640 1 -- 172.21.15.94:0/1710505411 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd741a48d0 con 0x7fbd741976a0 2024-02-17T12:06:08.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.896+0000 7fbd5bfff640 1 -- 172.21.15.94:0/1710505411 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fbd6c005520 con 0x7fbd741976a0 2024-02-17T12:06:08.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.896+0000 7fbd7bd65740 1 Processor -- start 2024-02-17T12:06:08.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.896+0000 7fbd7bd65740 1 -- start start 2024-02-17T12:06:08.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.896+0000 7fbd7bd65740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e310532b0 0x557e31053690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.896+0000 7fbd7bd65740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e3105c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.896+0000 7fbd7bd65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 0x557e3105eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.896+0000 7fbd7bd65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e30fc2280 con 0x557e3105c7b0 2024-02-17T12:06:08.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.896+0000 7fbd7bd65740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e30fbf7b0 con 0x557e310532b0 2024-02-17T12:06:08.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.896+0000 7fbd7bd65740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e30fbaf90 con 0x557e31053bd0 2024-02-17T12:06:08.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd7259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 0x557e3105eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd5bfff640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbd5c0a2230 0x7fbd5c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd7259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 0x557e3105eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41288/0 (socket says 172.21.15.94:41288) 2024-02-17T12:06:08.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd7259c640 1 -- 172.21.15.94:0/2523877023 learned_addr learned my addr 172.21.15.94:0/2523877023 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:08.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd72d9d640 1 --2- 172.21.15.94:0/2523877023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e3105c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd7259c640 1 --2- 172.21.15.94:0/2523877023 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e310532b0 0x557e31053690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd7359e640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbd5c0a2230 0x7fbd5c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd7259c640 1 -- 172.21.15.94:0/2523877023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 msgr2=0x557e3105c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd7259c640 1 --2- 172.21.15.94:0/2523877023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e3105c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd5bfff640 1 -- 172.21.15.94:0/1710505411 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbd6c0ce2d0 con 0x7fbd741976a0 2024-02-17T12:06:08.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd7259c640 1 -- 172.21.15.94:0/2523877023 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e310532b0 msgr2=0x557e31053690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd7259c640 1 --2- 172.21.15.94:0/2523877023 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e310532b0 0x557e31053690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd7259c640 1 -- 172.21.15.94:0/2523877023 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e30f79570 con 0x557e3105c7b0 2024-02-17T12:06:08.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd7259c640 1 --2- 172.21.15.94:0/2523877023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 0x557e3105eba0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbd68005ec0 tx=0x7fbd68005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f770dd2fc7d34fd3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:08.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd597fa640 1 -- 172.21.15.94:0/2523877023 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6802c9a0 con 0x557e3105c7b0 2024-02-17T12:06:08.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.897+0000 7fbd597fa640 1 -- 172.21.15.94:0/2523877023 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd6802cb00 con 0x557e3105c7b0 2024-02-17T12:06:08.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.898+0000 7fbd597fa640 1 -- 172.21.15.94:0/2523877023 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68003da0 con 0x557e3105c7b0 2024-02-17T12:06:08.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.898+0000 7fbd7bd65740 1 -- 172.21.15.94:0/2523877023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 msgr2=0x557e3105eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.898+0000 7fbd7359e640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbd5c0a2230 0x7fbd5c0a46f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00cfd0 tx=0x7fbd6c01e6e0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:08.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.898+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/2523877023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 0x557e3105eba0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbd68005ec0 tx=0x7fbd68005fd0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.898+0000 7fbd7bd65740 1 -- 172.21.15.94:0/2523877023 shutdown_connections 2024-02-17T12:06:08.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.898+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/2523877023 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 0x557e3105eba0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.898+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/2523877023 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e3105c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.898+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/2523877023 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e310532b0 0x557e31053690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.898+0000 7fbd7bd65740 1 -- 172.21.15.94:0/2523877023 >> 172.21.15.94:0/2523877023 conn(0x557e3104d990 msgr2=0x557e3104dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.898+0000 7fbd7bd65740 1 -- 172.21.15.94:0/2523877023 shutdown_connections 2024-02-17T12:06:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.898+0000 7fbd7bd65740 1 -- 172.21.15.94:0/2523877023 wait complete. 2024-02-17T12:06:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7bd65740 1 -- start start 2024-02-17T12:06:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7bd65740 1 Processor -- start 2024-02-17T12:06:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7bd65740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e310532b0 0x557e31059620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7bd65740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e31059b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7bd65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 0x557e3105a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7bd65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e30c8e430 con 0x557e3105c7b0 2024-02-17T12:06:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7bd65740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e30c4ae00 con 0x557e310532b0 2024-02-17T12:06:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7bd65740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e30f3ffd0 con 0x557e31053bd0 2024-02-17T12:06:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7bd65740 10 client.0 _ll_register_callbacks cb 0x557e30f95fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 0x557e3105a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 0x557e3105a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41300/0 (socket says 172.21.15.94:41300) 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7259c640 1 -- 172.21.15.94:0/99268299 learned_addr learned my addr 172.21.15.94:0/99268299 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd72d9d640 1 --2- 172.21.15.94:0/99268299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e31059b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7259c640 1 --2- 172.21.15.94:0/99268299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e310532b0 0x557e31059620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7259c640 1 -- 172.21.15.94:0/99268299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 msgr2=0x557e31059b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7259c640 1 --2- 172.21.15.94:0/99268299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e31059b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7259c640 1 -- 172.21.15.94:0/99268299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e310532b0 msgr2=0x557e31059620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7259c640 1 --2- 172.21.15.94:0/99268299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e310532b0 0x557e31059620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.899+0000 7fbd7259c640 1 -- 172.21.15.94:0/99268299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7410e810 con 0x557e3105c7b0 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd7259c640 1 --2- 172.21.15.94:0/99268299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 0x557e3105a0a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbd6802cd90 tx=0x7fbd68003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd58fece6620b651 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd56ff5640 1 -- 172.21.15.94:0/99268299 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68003f30 con 0x557e3105c7b0 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd56ff5640 1 -- 172.21.15.94:0/99268299 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbd68004090 con 0x557e3105c7b0 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd7bd65740 1 -- 172.21.15.94:0/99268299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e30cba7e0 con 0x557e3105c7b0 2024-02-17T12:06:08.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd56ff5640 1 -- 172.21.15.94:0/99268299 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd68004250 con 0x557e3105c7b0 2024-02-17T12:06:08.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd7bd65740 10 client.17577 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:08.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd7bd65740 1 -- 172.21.15.94:0/99268299 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557e30fb9610 con 0x557e3105c7b0 2024-02-17T12:06:08.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd56ff5640 1 -- 172.21.15.94:0/99268299 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557e30cba7e0 con 0x557e3105c7b0 2024-02-17T12:06:08.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd56ff5640 1 -- 172.21.15.94:0/99268299 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbd6804b020 con 0x557e3105c7b0 2024-02-17T12:06:08.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd7bd65740 10 client.17577 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:08.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd7bd65740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:08.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd7bd65740 20 client.17577 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:08.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/99268299 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557e3106c660 0x557e3106ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd7bd65740 4 client.17577 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:08.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.900+0000 7fbd7bd65740 1 -- 172.21.15.94:0/99268299 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e30cf7720 con 0x557e3106c660 2024-02-17T12:06:08.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.901+0000 7fbd72d9d640 1 --2- 172.21.15.94:0/99268299 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557e3106c660 0x557e3106ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.901+0000 7fbd56ff5640 10 client.17577 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:08.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.901+0000 7fbd72d9d640 1 --2- 172.21.15.94:0/99268299 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557e3106c660 0x557e3106ea80 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703f94527b48d2bb server_cookie=62f5713c33e61d18 in_seq=0 out_seq=0 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.901+0000 7fbd56ff5640 1 -- 172.21.15.94:0/99268299 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557e30cf7720 con 0x557e3106c660 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.901+0000 7fbd56ff5640 10 client.17577 handle_command_reply: tid=0 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.901+0000 7fbd7bd65740 1 client.17577 shutdown 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.902+0000 7fbd7bd65740 1 -- 172.21.15.94:0/99268299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 msgr2=0x557e3105a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.902+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/99268299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 0x557e3105a0a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbd6802cd90 tx=0x7fbd68003980 comp rx=0 tx=0).stop 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.902+0000 7fbd7bd65740 1 -- 172.21.15.94:0/99268299 shutdown_connections 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.902+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/99268299 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557e3106c660 0x557e3106ea80 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.902+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/99268299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e3105c7b0 0x557e3105a0a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.902+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/99268299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e31059b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.902+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/99268299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e310532b0 0x557e31059620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.902+0000 7fbd7bd65740 1 -- 172.21.15.94:0/99268299 >> 172.21.15.94:0/99268299 conn(0x557e3104d990 msgr2=0x557e3104f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.902+0000 7fbd56ff5640 0 client.17577 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.902+0000 7fbd7bd65740 1 -- 172.21.15.94:0/99268299 shutdown_connections 2024-02-17T12:06:08.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.902+0000 7fbd7bd65740 1 -- 172.21.15.94:0/99268299 wait complete. 2024-02-17T12:06:08.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.902+0000 7fbd7bd65740 20 client.17577 trim_cache size 0 max 16384 2024-02-17T12:06:08.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.919+0000 7fbd7bd65740 1 Processor -- start 2024-02-17T12:06:08.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.919+0000 7fbd7bd65740 1 -- start start 2024-02-17T12:06:08.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.919+0000 7fbd7bd65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 0x557e3105bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.919+0000 7fbd7bd65740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e31058620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.919+0000 7fbd7bd65740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e3105c7b0 0x557e310568c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.919+0000 7fbd7bd65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e3106c4b0 con 0x557e310532b0 2024-02-17T12:06:08.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.919+0000 7fbd7bd65740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e31055020 con 0x557e3105c7b0 2024-02-17T12:06:08.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.919+0000 7fbd7bd65740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e30cba7e0 con 0x557e31053bd0 2024-02-17T12:06:08.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd7259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 0x557e3105bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd7259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 0x557e3105bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41312/0 (socket says 172.21.15.94:41312) 2024-02-17T12:06:08.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd7259c640 1 -- 172.21.15.94:0/583302278 learned_addr learned my addr 172.21.15.94:0/583302278 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:08.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd72d9d640 1 --2- 172.21.15.94:0/583302278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e31058620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd7259c640 1 --2- 172.21.15.94:0/583302278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e3105c7b0 0x557e310568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd7259c640 1 -- 172.21.15.94:0/583302278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 msgr2=0x557e31058620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd7259c640 1 --2- 172.21.15.94:0/583302278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e31058620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd7259c640 1 -- 172.21.15.94:0/583302278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e3105c7b0 msgr2=0x557e310568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd7259c640 1 --2- 172.21.15.94:0/583302278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e3105c7b0 0x557e310568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd7259c640 1 -- 172.21.15.94:0/583302278 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e30f55ac0 con 0x557e310532b0 2024-02-17T12:06:08.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd7259c640 1 --2- 172.21.15.94:0/583302278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 0x557e3105bd50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbd68005db0 tx=0x7fbd68005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e72a72138ab378a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:08.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd58ff9640 1 -- 172.21.15.94:0/583302278 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68004af0 con 0x557e310532b0 2024-02-17T12:06:08.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd58ff9640 1 -- 172.21.15.94:0/583302278 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd68056030 con 0x557e310532b0 2024-02-17T12:06:08.923 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.920+0000 7fbd58ff9640 1 -- 172.21.15.94:0/583302278 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6804e420 con 0x557e310532b0 2024-02-17T12:06:08.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.921+0000 7fbd7bd65740 1 -- 172.21.15.94:0/583302278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 msgr2=0x557e3105bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.921+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/583302278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 0x557e3105bd50 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbd68005db0 tx=0x7fbd68005fd0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.921+0000 7fbd7bd65740 1 -- 172.21.15.94:0/583302278 shutdown_connections 2024-02-17T12:06:08.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.921+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/583302278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e3105c7b0 0x557e310568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.921+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/583302278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e31058620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.921+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/583302278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 0x557e3105bd50 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.921+0000 7fbd7bd65740 1 -- 172.21.15.94:0/583302278 >> 172.21.15.94:0/583302278 conn(0x557e3104d990 msgr2=0x557e31057f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.921+0000 7fbd7bd65740 1 -- 172.21.15.94:0/583302278 shutdown_connections 2024-02-17T12:06:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.921+0000 7fbd7bd65740 1 -- 172.21.15.94:0/583302278 wait complete. 2024-02-17T12:06:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7bd65740 1 -- start start 2024-02-17T12:06:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7bd65740 1 Processor -- start 2024-02-17T12:06:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7bd65740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 0x557e31062360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7bd65740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e310628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7bd65740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e3105c7b0 0x557e310c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7bd65740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e310630b0 con 0x557e310532b0 2024-02-17T12:06:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7bd65740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e31063220 con 0x557e3105c7b0 2024-02-17T12:06:08.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7bd65740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e310c30f0 con 0x557e31053bd0 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7bd65740 10 client.0 _ll_register_callbacks cb 0x7fbd6802c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 0x557e31062360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 0x557e31062360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41326/0 (socket says 172.21.15.94:41326) 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7259c640 1 -- 172.21.15.94:0/4042461131 learned_addr learned my addr 172.21.15.94:0/4042461131 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd72d9d640 1 --2- 172.21.15.94:0/4042461131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e310628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7259c640 1 --2- 172.21.15.94:0/4042461131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e3105c7b0 0x557e310c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7259c640 1 -- 172.21.15.94:0/4042461131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 msgr2=0x557e310628a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7259c640 1 --2- 172.21.15.94:0/4042461131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e310628a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7259c640 1 -- 172.21.15.94:0/4042461131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e3105c7b0 msgr2=0x557e310c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.922+0000 7fbd7259c640 1 --2- 172.21.15.94:0/4042461131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e3105c7b0 0x557e310c2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd7259c640 1 -- 172.21.15.94:0/4042461131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e30fb9610 con 0x557e310532b0 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd7259c640 1 --2- 172.21.15.94:0/4042461131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 0x557e31062360 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009830 tx=0x7fbd68041b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e35101142b3d0a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd56ff5640 1 -- 172.21.15.94:0/4042461131 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68003870 con 0x557e310532b0 2024-02-17T12:06:08.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd56ff5640 1 -- 172.21.15.94:0/4042461131 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fbd68000e40 con 0x557e310532b0 2024-02-17T12:06:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd7bd65740 1 -- 172.21.15.94:0/4042461131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd6802cb00 con 0x557e310532b0 2024-02-17T12:06:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd56ff5640 1 -- 172.21.15.94:0/4042461131 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd68041bc0 con 0x557e310532b0 2024-02-17T12:06:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd7bd65740 10 client.17583 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd7bd65740 1 -- 172.21.15.94:0/4042461131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557e30f7b6a0 con 0x557e310532b0 2024-02-17T12:06:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd56ff5640 1 -- 172.21.15.94:0/4042461131 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd6807d020 con 0x557e310532b0 2024-02-17T12:06:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd56ff5640 1 -- 172.21.15.94:0/4042461131 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbd6804b070 con 0x557e310532b0 2024-02-17T12:06:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd7bd65740 10 client.17583 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd7bd65740 10 client.17583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd7bd65740 20 client.17583 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/4042461131 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557e310c4460 0x557e310c6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd7bd65740 4 client.17583 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:06:08.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.923+0000 7fbd7bd65740 1 -- 172.21.15.94:0/4042461131 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557e30d46890 con 0x557e310c4460 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.924+0000 7fbd72d9d640 1 --2- 172.21.15.94:0/4042461131 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557e310c4460 0x557e310c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.924+0000 7fbd56ff5640 10 client.17583 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.924+0000 7fbd72d9d640 1 --2- 172.21.15.94:0/4042461131 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557e310c4460 0x557e310c6850 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ce67091866293a server_cookie=b896916de7f41e54 in_seq=0 out_seq=0 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.924+0000 7fbd56ff5640 1 -- 172.21.15.94:0/4042461131 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557e30d46890 con 0x557e310c4460 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.924+0000 7fbd56ff5640 10 client.17583 handle_command_reply: tid=0 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.924+0000 7fbd7bd65740 1 client.17583 shutdown 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7bd65740 1 -- 172.21.15.94:0/4042461131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 msgr2=0x557e31062360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/4042461131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 0x557e31062360 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009830 tx=0x7fbd68041b20 comp rx=0 tx=0).stop 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7bd65740 1 -- 172.21.15.94:0/4042461131 shutdown_connections 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/4042461131 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557e310c4460 0x557e310c6850 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/4042461131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557e3105c7b0 0x557e310c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/4042461131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557e31053bd0 0x557e310628a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7bd65740 1 --2- 172.21.15.94:0/4042461131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557e310532b0 0x557e31062360 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7bd65740 1 -- 172.21.15.94:0/4042461131 >> 172.21.15.94:0/4042461131 conn(0x557e3104d990 msgr2=0x557e3104c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:08.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd56ff5640 0 client.17583 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:08.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7bd65740 1 -- 172.21.15.94:0/4042461131 shutdown_connections 2024-02-17T12:06:08.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7bd65740 1 -- 172.21.15.94:0/4042461131 wait complete. 2024-02-17T12:06:08.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7bd65740 20 client.17583 trim_cache size 0 max 16384 2024-02-17T12:06:08.933 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:06:08.933 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:06:08.933 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:06:08.933 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:06:08.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7918c640 1 -- 172.21.15.94:0/1710505411 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbd5c0a2230 msgr2=0x7fbd5c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7918c640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbd5c0a2230 0x7fbd5c0a46f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00cfd0 tx=0x7fbd6c01e6e0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7918c640 1 -- 172.21.15.94:0/1710505411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd741976a0 msgr2=0x7fbd74197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:08.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7918c640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd741976a0 0x7fbd74197b20 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00eaa0 tx=0x7fbd6c00ef60 comp rx=0 tx=0).stop 2024-02-17T12:06:08.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7918c640 1 -- 172.21.15.94:0/1710505411 shutdown_connections 2024-02-17T12:06:08.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7918c640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbd5c0a2230 0x7fbd5c0a46f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7918c640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbd741976a0 0x7fbd74197b20 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7918c640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbd74106430 0x7fbd7419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7918c640 1 --2- 172.21.15.94:0/1710505411 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbd74101360 0x7fbd7419d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:08.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.925+0000 7fbd7918c640 1 -- 172.21.15.94:0/1710505411 >> 172.21.15.94:0/1710505411 conn(0x7fbd740fcc20 msgr2=0x7fbd740fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:08.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.926+0000 7fbd7918c640 1 -- 172.21.15.94:0/1710505411 shutdown_connections 2024-02-17T12:06:08.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:08.926+0000 7fbd7918c640 1 -- 172.21.15.94:0/1710505411 wait complete. 2024-02-17T12:06:08.938 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:06:08.939 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:06:09.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.047+0000 7fd5fe948640 1 -- 172.21.15.94:0/3569160319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f81013b0 msgr2=0x7fd5f8101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.047+0000 7fd5fe948640 1 --2- 172.21.15.94:0/3569160319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f81013b0 0x7fd5f8101790 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009930 tx=0x7fd5e802bc30 comp rx=0 tx=0).stop 2024-02-17T12:06:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.047+0000 7fd5fe948640 1 -- 172.21.15.94:0/3569160319 shutdown_connections 2024-02-17T12:06:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.047+0000 7fd5fe948640 1 --2- 172.21.15.94:0/3569160319 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5f8106480 0x7fd5f810e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.047+0000 7fd5fe948640 1 --2- 172.21.15.94:0/3569160319 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f8101d60 0x7fd5f8105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.047+0000 7fd5fe948640 1 --2- 172.21.15.94:0/3569160319 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f81013b0 0x7fd5f8101790 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.047+0000 7fd5fe948640 1 -- 172.21.15.94:0/3569160319 >> 172.21.15.94:0/3569160319 conn(0x7fd5f80fcbf0 msgr2=0x7fd5f80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.047+0000 7fd5fe948640 1 -- 172.21.15.94:0/3569160319 shutdown_connections 2024-02-17T12:06:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.047+0000 7fd5fe948640 1 -- 172.21.15.94:0/3569160319 wait complete. 2024-02-17T12:06:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.048+0000 7fd5fe948640 1 Processor -- start 2024-02-17T12:06:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.048+0000 7fd5fe948640 1 -- start start 2024-02-17T12:06:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.048+0000 7fd5fe948640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f81013b0 0x7fd5f819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.048+0000 7fd5fe948640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5f8106480 0x7fd5f819db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.048+0000 7fd5fe948640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f81976b0 0x7fd5f8197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.048+0000 7fd5fe948640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f8198180 con 0x7fd5f81976b0 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.048+0000 7fd5fe948640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f81982f0 con 0x7fd5f81013b0 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.048+0000 7fd5fe948640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f81984b0 con 0x7fd5f8106480 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.048+0000 7fd5fcee5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f81976b0 0x7fd5f8197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.048+0000 7fd5fcee5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f81976b0 0x7fd5f8197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41342/0 (socket says 172.21.15.94:41342) 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.048+0000 7fd5fcee5640 1 -- 172.21.15.94:0/1677831747 learned_addr learned my addr 172.21.15.94:0/1677831747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.048+0000 7fd5f7fff640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f81013b0 0x7fd5f819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.048+0000 7fd5f77fe640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5f8106480 0x7fd5f819db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.049+0000 7fd5fcee5640 1 -- 172.21.15.94:0/1677831747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5f8106480 msgr2=0x7fd5f819db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.049+0000 7fd5fcee5640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5f8106480 0x7fd5f819db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.049+0000 7fd5fcee5640 1 -- 172.21.15.94:0/1677831747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f81013b0 msgr2=0x7fd5f819d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.049+0000 7fd5fcee5640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f81013b0 0x7fd5f819d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.049+0000 7fd5fcee5640 1 -- 172.21.15.94:0/1677831747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e8009560 con 0x7fd5f81976b0 2024-02-17T12:06:09.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.049+0000 7fd5f7fff640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f81013b0 0x7fd5f819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.049+0000 7fd5fcee5640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f81976b0 0x7fd5f8197b30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000eb10 tx=0x7fd5f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.049+0000 7fd5f57fa640 1 -- 172.21.15.94:0/1677831747 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0004b20 con 0x7fd5f81976b0 2024-02-17T12:06:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.049+0000 7fd5f57fa640 1 -- 172.21.15.94:0/1677831747 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5f0004c80 con 0x7fd5f81976b0 2024-02-17T12:06:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.049+0000 7fd5fe948640 1 -- 172.21.15.94:0/1677831747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f81a4450 con 0x7fd5f81976b0 2024-02-17T12:06:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.049+0000 7fd5f57fa640 1 -- 172.21.15.94:0/1677831747 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0010650 con 0x7fd5f81976b0 2024-02-17T12:06:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.049+0000 7fd5fe948640 1 -- 172.21.15.94:0/1677831747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f81a48e0 con 0x7fd5f81976b0 2024-02-17T12:06:09.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.050+0000 7fd5fe948640 1 -- 172.21.15.94:0/1677831747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f8102880 con 0x7fd5f81976b0 2024-02-17T12:06:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.051+0000 7fd5f57fa640 1 -- 172.21.15.94:0/1677831747 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd5f0005020 con 0x7fd5f81976b0 2024-02-17T12:06:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.051+0000 7fd5f57fa640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5e00a2230 0x7fd5e00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.052+0000 7fd5f7fff640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5e00a2230 0x7fd5e00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.052+0000 7fd5f57fa640 1 -- 172.21.15.94:0/1677831747 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fd5f0014070 con 0x7fd5f81976b0 2024-02-17T12:06:09.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.052+0000 7fd5f7fff640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5e00a2230 0x7fd5e00a46f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8005ab0 tx=0x7fd5e8005fd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.056+0000 7fd5f57fa640 1 -- 172.21.15.94:0/1677831747 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd5f0097240 con 0x7fd5f81976b0 2024-02-17T12:06:09.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:09 smithi094 ceph-mon[32499]: osdmap e91: 12 total, 12 up, 12 in 2024-02-17T12:06:09.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:09 smithi094 ceph-mon[32499]: pgmap v247: 97 pgs: 7 active+clean+snaptrim_wait, 85 active+clean, 5 active+clean+snaptrim; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-02-17T12:06:09.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.294+0000 7fd5fe948640 1 -- 172.21.15.94:0/1677831747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5f8109750 con 0x7fd5f81976b0 2024-02-17T12:06:09.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.295+0000 7fd5f57fa640 1 -- 172.21.15.94:0/1677831747 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7fd5f009a900 con 0x7fd5f81976b0 2024-02-17T12:06:09.296 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:06:09.296 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:06:09.297 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:06:09.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.297+0000 7fd5fe948640 1 -- 172.21.15.94:0/1677831747 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5e00a2230 msgr2=0x7fd5e00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.297+0000 7fd5fe948640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5e00a2230 0x7fd5e00a46f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8005ab0 tx=0x7fd5e8005fd0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.297+0000 7fd5fe948640 1 -- 172.21.15.94:0/1677831747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f81976b0 msgr2=0x7fd5f8197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.297+0000 7fd5fe948640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f81976b0 0x7fd5f8197b30 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000eb10 tx=0x7fd5f000c2d0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.297+0000 7fd5fe948640 1 -- 172.21.15.94:0/1677831747 shutdown_connections 2024-02-17T12:06:09.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.297+0000 7fd5fe948640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5e00a2230 0x7fd5e00a46f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.297+0000 7fd5fe948640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5f81976b0 0x7fd5f8197b30 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.297+0000 7fd5fe948640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5f8106480 0x7fd5f819db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.297+0000 7fd5fe948640 1 --2- 172.21.15.94:0/1677831747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5f81013b0 0x7fd5f819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.297+0000 7fd5fe948640 1 -- 172.21.15.94:0/1677831747 >> 172.21.15.94:0/1677831747 conn(0x7fd5f80fcbf0 msgr2=0x7fd5f80fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.297+0000 7fd5fe948640 1 -- 172.21.15.94:0/1677831747 shutdown_connections 2024-02-17T12:06:09.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.297+0000 7fd5fe948640 1 -- 172.21.15.94:0/1677831747 wait complete. 2024-02-17T12:06:09.311 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:06:09.311 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:06:09.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:09 smithi189 ceph-mon[36588]: osdmap e91: 12 total, 12 up, 12 in 2024-02-17T12:06:09.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:09 smithi189 ceph-mon[36588]: pgmap v247: 97 pgs: 7 active+clean+snaptrim_wait, 85 active+clean, 5 active+clean+snaptrim; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-02-17T12:06:09.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.420+0000 7fdb82c9d640 1 -- 172.21.15.94:0/3214525586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb7c1002a0 msgr2=0x7fdb7c100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.420+0000 7fdb82c9d640 1 --2- 172.21.15.94:0/3214525586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb7c1002a0 0x7fdb7c100680 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009930 tx=0x7fdb7002bc30 comp rx=0 tx=0).stop 2024-02-17T12:06:09.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.420+0000 7fdb82c9d640 1 -- 172.21.15.94:0/3214525586 shutdown_connections 2024-02-17T12:06:09.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.420+0000 7fdb82c9d640 1 --2- 172.21.15.94:0/3214525586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb7c105370 0x7fdb7c10cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.420+0000 7fdb82c9d640 1 --2- 172.21.15.94:0/3214525586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb7c100c50 0x7fdb7c104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.420+0000 7fdb82c9d640 1 --2- 172.21.15.94:0/3214525586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb7c1002a0 0x7fdb7c100680 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.420+0000 7fdb82c9d640 1 -- 172.21.15.94:0/3214525586 >> 172.21.15.94:0/3214525586 conn(0x7fdb7c076400 msgr2=0x7fdb7c076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.420+0000 7fdb82c9d640 1 -- 172.21.15.94:0/3214525586 shutdown_connections 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.420+0000 7fdb82c9d640 1 -- 172.21.15.94:0/3214525586 wait complete. 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.420+0000 7fdb82c9d640 1 Processor -- start 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb82c9d640 1 -- start start 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb82c9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb7c1002a0 0x7fdb7c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb82c9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb7c105370 0x7fdb7c19db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb82c9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb7c197680 0x7fdb7c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb82c9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c198150 con 0x7fdb7c197680 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb82c9d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c1982c0 con 0x7fdb7c105370 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb82c9d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb7c198480 con 0x7fdb7c1002a0 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb8123a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb7c197680 0x7fdb7c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb80a39640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb7c1002a0 0x7fdb7c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb8123a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb7c197680 0x7fdb7c197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41364/0 (socket says 172.21.15.94:41364) 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb8123a640 1 -- 172.21.15.94:0/196036701 learned_addr learned my addr 172.21.15.94:0/196036701 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/196036701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb7c105370 0x7fdb7c19db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.426 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb8123a640 1 -- 172.21.15.94:0/196036701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb7c1002a0 msgr2=0x7fdb7c19d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb8123a640 1 --2- 172.21.15.94:0/196036701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb7c1002a0 0x7fdb7c19d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb8123a640 1 -- 172.21.15.94:0/196036701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb7c105370 msgr2=0x7fdb7c19db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb8123a640 1 --2- 172.21.15.94:0/196036701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb7c105370 0x7fdb7c19db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb8123a640 1 -- 172.21.15.94:0/196036701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb70009560 con 0x7fdb7c197680 2024-02-17T12:06:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb8123a640 1 --2- 172.21.15.94:0/196036701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb7c197680 0x7fdb7c197b00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400eb10 tx=0x7fdb7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb79ffb640 1 -- 172.21.15.94:0/196036701 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb74004b20 con 0x7fdb7c197680 2024-02-17T12:06:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb79ffb640 1 -- 172.21.15.94:0/196036701 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb74004c80 con 0x7fdb7c197680 2024-02-17T12:06:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb79ffb640 1 -- 172.21.15.94:0/196036701 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb74010650 con 0x7fdb7c197680 2024-02-17T12:06:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.421+0000 7fdb82c9d640 1 -- 172.21.15.94:0/196036701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb7c1a4420 con 0x7fdb7c197680 2024-02-17T12:06:09.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.422+0000 7fdb82c9d640 1 -- 172.21.15.94:0/196036701 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb7c1a4900 con 0x7fdb7c197680 2024-02-17T12:06:09.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.424+0000 7fdb79ffb640 1 -- 172.21.15.94:0/196036701 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fdb74004df0 con 0x7fdb7c197680 2024-02-17T12:06:09.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb85877740 1 Processor -- start 2024-02-17T12:06:09.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb85877740 1 -- start start 2024-02-17T12:06:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb85877740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d1e250 0x564422d1e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb85877740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d27210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb85877740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d27750 0x564422d29b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb85877740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422cd9960 con 0x564422d27750 2024-02-17T12:06:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb85877740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422c8d310 con 0x564422d1eb70 2024-02-17T12:06:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb85877740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422c8a840 con 0x564422d1e250 2024-02-17T12:06:09.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d27750 0x564422d29b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb79ffb640 1 --2- 172.21.15.94:0/196036701 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdb6c0a2230 0x7fdb6c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d27750 0x564422d29b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41380/0 (socket says 172.21.15.94:41380) 2024-02-17T12:06:09.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb7bfff640 1 -- 172.21.15.94:0/571228932 learned_addr learned my addr 172.21.15.94:0/571228932 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb80a39640 1 --2- 172.21.15.94:0/571228932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d27210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/571228932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d1e250 0x564422d1e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.425+0000 7fdb8123a640 1 --2- 172.21.15.94:0/196036701 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdb6c0a2230 0x7fdb6c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.426+0000 7fdb7bfff640 1 -- 172.21.15.94:0/571228932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d1e250 msgr2=0x564422d1e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.426+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/571228932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d1e250 0x564422d1e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.426+0000 7fdb7bfff640 1 -- 172.21.15.94:0/571228932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 msgr2=0x564422d27210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.426+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/571228932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d27210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.426+0000 7fdb7bfff640 1 -- 172.21.15.94:0/571228932 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564422c48060 con 0x564422d27750 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.426+0000 7fdb80a39640 1 --2- 172.21.15.94:0/571228932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d27210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.426+0000 7fdb79ffb640 1 -- 172.21.15.94:0/196036701 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fdb74014070 con 0x7fdb7c197680 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.426+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/571228932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d27750 0x564422d29b40 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fdb680058c0 tx=0x7fdb68005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f911b4168e49639f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.426+0000 7fdb8123a640 1 --2- 172.21.15.94:0/196036701 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdb6c0a2230 0x7fdb6c0a46f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb740cbda0 tx=0x7fdb74096c10 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.426+0000 7fdb66ffd640 1 -- 172.21.15.94:0/571228932 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68004030 con 0x564422d27750 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.427+0000 7fdb66ffd640 1 -- 172.21.15.94:0/571228932 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb68005020 con 0x564422d27750 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.427+0000 7fdb66ffd640 1 -- 172.21.15.94:0/571228932 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68004cd0 con 0x564422d27750 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.427+0000 7fdb85877740 1 -- 172.21.15.94:0/571228932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d27750 msgr2=0x564422d29b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.427+0000 7fdb85877740 1 --2- 172.21.15.94:0/571228932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d27750 0x564422d29b40 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fdb680058c0 tx=0x7fdb68005850 comp rx=0 tx=0).stop 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.427+0000 7fdb85877740 1 -- 172.21.15.94:0/571228932 shutdown_connections 2024-02-17T12:06:09.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.427+0000 7fdb85877740 1 --2- 172.21.15.94:0/571228932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d27750 0x564422d29b40 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.427+0000 7fdb85877740 1 --2- 172.21.15.94:0/571228932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d27210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.427+0000 7fdb85877740 1 --2- 172.21.15.94:0/571228932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d1e250 0x564422d1e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.427+0000 7fdb85877740 1 -- 172.21.15.94:0/571228932 >> 172.21.15.94:0/571228932 conn(0x564422d18980 msgr2=0x564422d18d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.427+0000 7fdb85877740 1 -- 172.21.15.94:0/571228932 shutdown_connections 2024-02-17T12:06:09.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.427+0000 7fdb85877740 1 -- 172.21.15.94:0/571228932 wait complete. 2024-02-17T12:06:09.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.428+0000 7fdb85877740 1 -- start start 2024-02-17T12:06:09.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.428+0000 7fdb85877740 1 Processor -- start 2024-02-17T12:06:09.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.428+0000 7fdb85877740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d247d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.428+0000 7fdb85877740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d1eb70 0x564422d24d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.428+0000 7fdb85877740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d27750 0x564422d25250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.428+0000 7fdb85877740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422c86020 con 0x564422d1e250 2024-02-17T12:06:09.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.428+0000 7fdb85877740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422959430 con 0x564422d27750 2024-02-17T12:06:09.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.428+0000 7fdb85877740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422915e00 con 0x564422d1eb70 2024-02-17T12:06:09.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.428+0000 7fdb85877740 10 client.0 _ll_register_callbacks cb 0x564422c2dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:09.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.428+0000 7fdb7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.428+0000 7fdb7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d247d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41390/0 (socket says 172.21.15.94:41390) 2024-02-17T12:06:09.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.428+0000 7fdb7bfff640 1 -- 172.21.15.94:0/3650237056 learned_addr learned my addr 172.21.15.94:0/3650237056 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.428+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d27750 0x564422d25250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb80a39640 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d1eb70 0x564422d24d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb7bfff640 1 -- 172.21.15.94:0/3650237056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d1eb70 msgr2=0x564422d24d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d1eb70 0x564422d24d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb7bfff640 1 -- 172.21.15.94:0/3650237056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d27750 msgr2=0x564422d25250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d27750 0x564422d25250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb7bfff640 1 -- 172.21.15.94:0/3650237056 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb68019040 con 0x564422d1e250 2024-02-17T12:06:09.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb80a39640 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d1eb70 0x564422d24d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:06:09.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d247d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fdb68005bc0 tx=0x7fdb68001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7785fe3a1e0f23d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb5bfff640 1 -- 172.21.15.94:0/3650237056 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68004c90 con 0x564422d1e250 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb5bfff640 1 -- 172.21.15.94:0/3650237056 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fdb68004df0 con 0x564422d1e250 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb85877740 1 -- 172.21.15.94:0/3650237056 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644229857e0 con 0x564422d1e250 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb5bfff640 1 -- 172.21.15.94:0/3650237056 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb68004040 con 0x564422d1e250 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb85877740 10 client.17601 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb85877740 1 -- 172.21.15.94:0/3650237056 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564422c846a0 con 0x564422d1e250 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.429+0000 7fdb5bfff640 1 -- 172.21.15.94:0/3650237056 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb6803b080 con 0x564422d1e250 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.430+0000 7fdb5bfff640 1 -- 172.21.15.94:0/3650237056 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb6801d070 con 0x564422d1e250 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.430+0000 7fdb85877740 10 client.17601 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.430+0000 7fdb85877740 10 client.17601 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.430+0000 7fdb85877740 20 client.17601 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.430+0000 7fdb85877740 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564422d3be10 0x564422d3e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.430+0000 7fdb85877740 4 client.17601 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:09.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.430+0000 7fdb85877740 1 -- 172.21.15.94:0/3650237056 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644229c2720 con 0x564422d3be10 2024-02-17T12:06:09.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.430+0000 7fdb80a39640 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564422d3be10 0x564422d3e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.430+0000 7fdb5bfff640 10 client.17601 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:06:09.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.430+0000 7fdb80a39640 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564422d3be10 0x564422d3e230 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d24ea8d30b3497c server_cookie=2fd38abf13268ca in_seq=0 out_seq=0 2024-02-17T12:06:09.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb5bfff640 1 -- 172.21.15.94:0/3650237056 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644229c2720 con 0x564422d3be10 2024-02-17T12:06:09.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb5bfff640 10 client.17601 handle_command_reply: tid=0 2024-02-17T12:06:09.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb85877740 1 client.17601 shutdown 2024-02-17T12:06:09.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb85877740 1 -- 172.21.15.94:0/3650237056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 msgr2=0x564422d247d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb85877740 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d247d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fdb68005bc0 tx=0x7fdb68001e50 comp rx=0 tx=0).stop 2024-02-17T12:06:09.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb85877740 1 -- 172.21.15.94:0/3650237056 shutdown_connections 2024-02-17T12:06:09.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb85877740 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564422d3be10 0x564422d3e230 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb85877740 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d27750 0x564422d25250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb85877740 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d1eb70 0x564422d24d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb5bfff640 0 client.17601 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:06:09.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb85877740 1 --2- 172.21.15.94:0/3650237056 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d247d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb85877740 1 -- 172.21.15.94:0/3650237056 >> 172.21.15.94:0/3650237056 conn(0x564422d18980 msgr2=0x564422d1a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb85877740 1 -- 172.21.15.94:0/3650237056 shutdown_connections 2024-02-17T12:06:09.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb85877740 1 -- 172.21.15.94:0/3650237056 wait complete. 2024-02-17T12:06:09.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.431+0000 7fdb85877740 20 client.17601 trim_cache size 0 max 16384 2024-02-17T12:06:09.446 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:09 smithi125 ceph-mon[36819]: osdmap e91: 12 total, 12 up, 12 in 2024-02-17T12:06:09.446 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:09 smithi125 ceph-mon[36819]: pgmap v247: 97 pgs: 7 active+clean+snaptrim_wait, 85 active+clean, 5 active+clean+snaptrim; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-02-17T12:06:09.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb85877740 1 Processor -- start 2024-02-17T12:06:09.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb85877740 1 -- start start 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb85877740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d26e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb85877740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d1fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb85877740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d27750 0x564422d1ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb85877740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422c0b130 con 0x564422d1e250 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb85877740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422d0aa40 con 0x564422d1eb70 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb85877740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422d38ac0 con 0x564422d27750 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d26e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d26e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41404/0 (socket says 172.21.15.94:41404) 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb7bfff640 1 -- 172.21.15.94:0/1545899787 learned_addr learned my addr 172.21.15.94:0/1545899787 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb80a39640 1 --2- 172.21.15.94:0/1545899787 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d1fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/1545899787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d27750 0x564422d1ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb7bfff640 1 -- 172.21.15.94:0/1545899787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d27750 msgr2=0x564422d1ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/1545899787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d27750 0x564422d1ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb7bfff640 1 -- 172.21.15.94:0/1545899787 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 msgr2=0x564422d1fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/1545899787 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d1fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb7bfff640 1 -- 172.21.15.94:0/1545899787 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564422c20c20 con 0x564422d1e250 2024-02-17T12:06:09.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.444+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/1545899787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d26e90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fdb680059f0 tx=0x7fdb68005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784780c95b941a5e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.445+0000 7fdb667fc640 1 -- 172.21.15.94:0/1545899787 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb68005680 con 0x564422d1e250 2024-02-17T12:06:09.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.445+0000 7fdb667fc640 1 -- 172.21.15.94:0/1545899787 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb6803e030 con 0x564422d1e250 2024-02-17T12:06:09.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.445+0000 7fdb667fc640 1 -- 172.21.15.94:0/1545899787 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb680112c0 con 0x564422d1e250 2024-02-17T12:06:09.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.445+0000 7fdb85877740 1 -- 172.21.15.94:0/1545899787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 msgr2=0x564422d26e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.445+0000 7fdb85877740 1 --2- 172.21.15.94:0/1545899787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d26e90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fdb680059f0 tx=0x7fdb68005f90 comp rx=0 tx=0).stop 2024-02-17T12:06:09.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.445+0000 7fdb85877740 1 -- 172.21.15.94:0/1545899787 shutdown_connections 2024-02-17T12:06:09.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.445+0000 7fdb85877740 1 --2- 172.21.15.94:0/1545899787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d27750 0x564422d1ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.455 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.445+0000 7fdb85877740 1 --2- 172.21.15.94:0/1545899787 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d1fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.445+0000 7fdb85877740 1 --2- 172.21.15.94:0/1545899787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d26e90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.445+0000 7fdb85877740 1 -- 172.21.15.94:0/1545899787 >> 172.21.15.94:0/1545899787 conn(0x564422d18980 msgr2=0x564422d13a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.445+0000 7fdb85877740 1 -- 172.21.15.94:0/1545899787 shutdown_connections 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.445+0000 7fdb85877740 1 -- 172.21.15.94:0/1545899787 wait complete. 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb85877740 1 -- start start 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb85877740 1 Processor -- start 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb85877740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d316b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb85877740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d31bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb85877740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d27750 0x564422d922e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb85877740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422d32350 con 0x564422d1e250 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb85877740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422d324c0 con 0x564422d1eb70 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb85877740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564422d92820 con 0x564422d27750 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb85877740 10 client.0 _ll_register_callbacks cb 0x7fdb68009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:09.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d316b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d316b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41416/0 (socket says 172.21.15.94:41416) 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb7bfff640 1 -- 172.21.15.94:0/2957749647 learned_addr learned my addr 172.21.15.94:0/2957749647 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb80a39640 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d31bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.446+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d27750 0x564422d922e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb80a39640 1 -- 172.21.15.94:0/2957749647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d27750 msgr2=0x564422d922e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb80a39640 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d27750 0x564422d922e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb80a39640 1 -- 172.21.15.94:0/2957749647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 msgr2=0x564422d316b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb80a39640 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d316b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb80a39640 1 -- 172.21.15.94:0/2957749647 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564422c846a0 con 0x564422d1eb70 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d316b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb80a39640 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d31bf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009230 tx=0x7fdb70005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbde028fa97fdfa5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb5bfff640 1 -- 172.21.15.94:0/2957749647 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb70048df0 con 0x564422d1eb70 2024-02-17T12:06:09.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb85877740 1 -- 172.21.15.94:0/2957749647 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564422d24600 con 0x564422d1eb70 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb5bfff640 1 -- 172.21.15.94:0/2957749647 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fdb70045040 con 0x564422d1eb70 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb5bfff640 1 -- 172.21.15.94:0/2957749647 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb70004040 con 0x564422d1eb70 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb85877740 10 client.27080 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb85877740 1 -- 172.21.15.94:0/2957749647 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564422c468c0 con 0x564422d1eb70 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb5bfff640 1 -- 172.21.15.94:0/2957749647 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdb7005e080 con 0x564422d1eb70 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb5bfff640 1 -- 172.21.15.94:0/2957749647 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdb7003c040 con 0x564422d1eb70 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb85877740 10 client.27080 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb85877740 10 client.27080 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb85877740 20 client.27080 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.447+0000 7fdb85877740 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564422d94020 0x564422d96440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb85877740 4 client.27080 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb85877740 1 -- 172.21.15.94:0/2957749647 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fdb68005020 con 0x564422d94020 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564422d94020 0x564422d96440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb5bfff640 10 client.27080 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb7bfff640 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564422d94020 0x564422d96440 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa86c7c44331c6d server_cookie=6feece5a0ea16f13 in_seq=0 out_seq=0 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb5bfff640 1 -- 172.21.15.94:0/2957749647 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fdb68005020 con 0x564422d94020 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb5bfff640 10 client.27080 handle_command_reply: tid=0 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb85877740 1 client.27080 shutdown 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb85877740 1 -- 172.21.15.94:0/2957749647 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 msgr2=0x564422d31bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb85877740 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d31bf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009230 tx=0x7fdb70005db0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb85877740 1 -- 172.21.15.94:0/2957749647 shutdown_connections 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb85877740 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564422d27750 0x564422d922e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb85877740 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x564422d94020 0x564422d96440 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb85877740 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564422d1eb70 0x564422d31bf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb85877740 1 --2- 172.21.15.94:0/2957749647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564422d1e250 0x564422d316b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb5bfff640 0 client.27080 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.448+0000 7fdb85877740 1 -- 172.21.15.94:0/2957749647 >> 172.21.15.94:0/2957749647 conn(0x564422d18980 msgr2=0x564422d1fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb85877740 1 -- 172.21.15.94:0/2957749647 shutdown_connections 2024-02-17T12:06:09.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb85877740 1 -- 172.21.15.94:0/2957749647 wait complete. 2024-02-17T12:06:09.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb85877740 20 client.27080 trim_cache size 0 max 16384 2024-02-17T12:06:09.460 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:06:09.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb82c9d640 1 -- 172.21.15.94:0/196036701 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdb6c0a2230 msgr2=0x7fdb6c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb82c9d640 1 --2- 172.21.15.94:0/196036701 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdb6c0a2230 0x7fdb6c0a46f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb740cbda0 tx=0x7fdb74096c10 comp rx=0 tx=0).stop 2024-02-17T12:06:09.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb82c9d640 1 -- 172.21.15.94:0/196036701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb7c197680 msgr2=0x7fdb7c197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb82c9d640 1 --2- 172.21.15.94:0/196036701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb7c197680 0x7fdb7c197b00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400eb10 tx=0x7fdb7400c2d0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb82c9d640 1 -- 172.21.15.94:0/196036701 shutdown_connections 2024-02-17T12:06:09.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb82c9d640 1 --2- 172.21.15.94:0/196036701 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdb6c0a2230 0x7fdb6c0a46f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb82c9d640 1 --2- 172.21.15.94:0/196036701 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdb7c197680 0x7fdb7c197b00 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb82c9d640 1 --2- 172.21.15.94:0/196036701 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdb7c105370 0x7fdb7c19db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb82c9d640 1 --2- 172.21.15.94:0/196036701 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb7c1002a0 0x7fdb7c19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb82c9d640 1 -- 172.21.15.94:0/196036701 >> 172.21.15.94:0/196036701 conn(0x7fdb7c076400 msgr2=0x7fdb7c104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb82c9d640 1 -- 172.21.15.94:0/196036701 shutdown_connections 2024-02-17T12:06:09.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.449+0000 7fdb82c9d640 1 -- 172.21.15.94:0/196036701 wait complete. 2024-02-17T12:06:09.462 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:06:09.463 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:06:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.573+0000 7f9fded2c640 1 -- 172.21.15.94:0/235578262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd81002a0 msgr2=0x7f9fd8100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.573+0000 7f9fded2c640 1 --2- 172.21.15.94:0/235578262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd81002a0 0x7f9fd8100680 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009930 tx=0x7f9fcc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:06:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.573+0000 7f9fded2c640 1 -- 172.21.15.94:0/235578262 shutdown_connections 2024-02-17T12:06:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.573+0000 7f9fded2c640 1 --2- 172.21.15.94:0/235578262 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fd8105370 0x7f9fd810cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.573+0000 7f9fded2c640 1 --2- 172.21.15.94:0/235578262 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fd8100c50 0x7f9fd8104c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.573+0000 7f9fded2c640 1 --2- 172.21.15.94:0/235578262 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd81002a0 0x7f9fd8100680 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.573+0000 7f9fded2c640 1 -- 172.21.15.94:0/235578262 >> 172.21.15.94:0/235578262 conn(0x7f9fd8076400 msgr2=0x7f9fd8076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.573+0000 7f9fded2c640 1 -- 172.21.15.94:0/235578262 shutdown_connections 2024-02-17T12:06:09.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.573+0000 7f9fded2c640 1 -- 172.21.15.94:0/235578262 wait complete. 2024-02-17T12:06:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.574+0000 7f9fded2c640 1 Processor -- start 2024-02-17T12:06:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.574+0000 7f9fded2c640 1 -- start start 2024-02-17T12:06:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.574+0000 7f9fded2c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd81002a0 0x7f9fd81966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.574+0000 7f9fded2c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fd8105370 0x7f9fd8196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.574+0000 7f9fded2c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fd819af90 0x7f9fd819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.574+0000 7f9fded2c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd819b980 con 0x7f9fd81002a0 2024-02-17T12:06:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.574+0000 7f9fded2c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd819baf0 con 0x7f9fd8105370 2024-02-17T12:06:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.574+0000 7f9fded2c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd819bc60 con 0x7f9fd819af90 2024-02-17T12:06:09.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.574+0000 7f9fdcac8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd81002a0 0x7f9fd81966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.574+0000 7f9fdcac8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd81002a0 0x7f9fd81966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41432/0 (socket says 172.21.15.94:41432) 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.574+0000 7f9fdcac8640 1 -- 172.21.15.94:0/1680683719 learned_addr learned my addr 172.21.15.94:0/1680683719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.574+0000 7f9fd7fff640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fd8105370 0x7f9fd8196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.574+0000 7f9fdd2c9640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fd819af90 0x7f9fd819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.575+0000 7f9fdcac8640 1 -- 172.21.15.94:0/1680683719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fd819af90 msgr2=0x7f9fd819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.575+0000 7f9fdcac8640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fd819af90 0x7f9fd819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.575+0000 7f9fdcac8640 1 -- 172.21.15.94:0/1680683719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fd8105370 msgr2=0x7f9fd8196c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.575+0000 7f9fdcac8640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fd8105370 0x7f9fd8196c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.575+0000 7f9fdcac8640 1 -- 172.21.15.94:0/1680683719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fcc009560 con 0x7f9fd81002a0 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.575+0000 7f9fdcac8640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd81002a0 0x7f9fd81966d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009230 tx=0x7f9fcc02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.575+0000 7f9fd5ffb640 1 -- 172.21.15.94:0/1680683719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc004350 con 0x7f9fd81002a0 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.575+0000 7f9fded2c640 1 -- 172.21.15.94:0/1680683719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd81a3240 con 0x7f9fd81002a0 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.575+0000 7f9fd5ffb640 1 -- 172.21.15.94:0/1680683719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fcc0044b0 con 0x7f9fd81002a0 2024-02-17T12:06:09.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.575+0000 7f9fd5ffb640 1 -- 172.21.15.94:0/1680683719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc003d00 con 0x7f9fd81002a0 2024-02-17T12:06:09.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.575+0000 7f9fded2c640 1 -- 172.21.15.94:0/1680683719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd81a36f0 con 0x7f9fd81002a0 2024-02-17T12:06:09.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.577+0000 7f9fd5ffb640 1 -- 172.21.15.94:0/1680683719 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9fcc004df0 con 0x7f9fd81002a0 2024-02-17T12:06:09.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fe19da740 1 Processor -- start 2024-02-17T12:06:09.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fe19da740 1 -- start start 2024-02-17T12:06:09.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fe19da740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72250 0x562a1ea72630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fe19da740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72b70 0x562a1ea7b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fe19da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 0x562a1ea7db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fd5ffb640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fc80a2230 0x7f9fc80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fe19da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1ea2d960 con 0x562a1ea7b750 2024-02-17T12:06:09.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fe19da740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1e9e1310 con 0x562a1ea72b70 2024-02-17T12:06:09.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fe19da740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1e9de840 con 0x562a1ea72250 2024-02-17T12:06:09.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fd7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 0x562a1ea7db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fdd2c9640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fc80a2230 0x7f9fc80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fd7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 0x562a1ea7db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41436/0 (socket says 172.21.15.94:41436) 2024-02-17T12:06:09.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fd5ffb640 1 -- 172.21.15.94:0/1680683719 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9fcc0f0090 con 0x7f9fd81002a0 2024-02-17T12:06:09.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fd7fff640 1 -- 172.21.15.94:0/4115159191 learned_addr learned my addr 172.21.15.94:0/4115159191 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fdcac8640 1 --2- 172.21.15.94:0/4115159191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72b70 0x562a1ea7b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.578+0000 7f9fdd2c9640 1 --2- 172.21.15.94:0/4115159191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72250 0x562a1ea72630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.579+0000 7f9fd7fff640 1 -- 172.21.15.94:0/4115159191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72250 msgr2=0x562a1ea72630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.579+0000 7f9fd7fff640 1 --2- 172.21.15.94:0/4115159191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72250 0x562a1ea72630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.579+0000 7f9fd7fff640 1 -- 172.21.15.94:0/4115159191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72b70 msgr2=0x562a1ea7b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.579+0000 7f9fd7fff640 1 --2- 172.21.15.94:0/4115159191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72b70 0x562a1ea7b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.579+0000 7f9fd7fff640 1 -- 172.21.15.94:0/4115159191 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a1e99c060 con 0x562a1ea7b750 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.579+0000 7f9fdcac8640 1 --2- 172.21.15.94:0/4115159191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72b70 0x562a1ea7b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.579+0000 7f9fd7fff640 1 --2- 172.21.15.94:0/4115159191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 0x562a1ea7db40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009be0 tx=0x7f9fc40119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0da80d98f3232 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.579+0000 7f9fdd2c9640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fc80a2230 0x7f9fc80a46f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009270 tx=0x7f9fd0009b20 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.579+0000 7f9fc2ffd640 1 -- 172.21.15.94:0/4115159191 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4016070 con 0x562a1ea7b750 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.579+0000 7f9fc2ffd640 1 -- 172.21.15.94:0/4115159191 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc40035f0 con 0x562a1ea7b750 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.579+0000 7f9fc2ffd640 1 -- 172.21.15.94:0/4115159191 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc401a620 con 0x562a1ea7b750 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.580+0000 7f9fe19da740 1 -- 172.21.15.94:0/4115159191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 msgr2=0x562a1ea7db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.580+0000 7f9fe19da740 1 --2- 172.21.15.94:0/4115159191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 0x562a1ea7db40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009be0 tx=0x7f9fc40119a0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.580+0000 7f9fe19da740 1 -- 172.21.15.94:0/4115159191 shutdown_connections 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.580+0000 7f9fe19da740 1 --2- 172.21.15.94:0/4115159191 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 0x562a1ea7db40 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.580+0000 7f9fe19da740 1 --2- 172.21.15.94:0/4115159191 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72b70 0x562a1ea7b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.580+0000 7f9fe19da740 1 --2- 172.21.15.94:0/4115159191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72250 0x562a1ea72630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.580+0000 7f9fe19da740 1 -- 172.21.15.94:0/4115159191 >> 172.21.15.94:0/4115159191 conn(0x562a1ea6c980 msgr2=0x562a1ea6cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.580+0000 7f9fe19da740 1 -- 172.21.15.94:0/4115159191 shutdown_connections 2024-02-17T12:06:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.580+0000 7f9fe19da740 1 -- 172.21.15.94:0/4115159191 wait complete. 2024-02-17T12:06:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.580+0000 7f9fe19da740 1 -- start start 2024-02-17T12:06:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.580+0000 7f9fe19da740 1 Processor -- start 2024-02-17T12:06:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fe19da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea72250 0x562a1ea78760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fe19da740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72b70 0x562a1ea78ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fe19da740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea7b750 0x562a1ea791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fe19da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1e9da020 con 0x562a1ea72250 2024-02-17T12:06:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fe19da740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1e6ad430 con 0x562a1ea7b750 2024-02-17T12:06:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fe19da740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1e669e00 con 0x562a1ea72b70 2024-02-17T12:06:09.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fe19da740 10 client.0 _ll_register_callbacks cb 0x562a1e981cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fdd2c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea7b750 0x562a1ea791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fd7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72b70 0x562a1ea78ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fdcac8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea72250 0x562a1ea78760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fdd2c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea7b750 0x562a1ea791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46422/0 (socket says 172.21.15.94:46422) 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fdd2c9640 1 -- 172.21.15.94:0/2287773273 learned_addr learned my addr 172.21.15.94:0/2287773273 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fdcac8640 1 -- 172.21.15.94:0/2287773273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72b70 msgr2=0x562a1ea78ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fdcac8640 1 --2- 172.21.15.94:0/2287773273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72b70 0x562a1ea78ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fdcac8640 1 -- 172.21.15.94:0/2287773273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea7b750 msgr2=0x562a1ea791e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fdcac8640 1 --2- 172.21.15.94:0/2287773273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea7b750 0x562a1ea791e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fdcac8640 1 -- 172.21.15.94:0/2287773273 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc4009890 con 0x562a1ea72250 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.581+0000 7f9fdd2c9640 1 --2- 172.21.15.94:0/2287773273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea7b750 0x562a1ea791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fdcac8640 1 --2- 172.21.15.94:0/2287773273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea72250 0x562a1ea78760 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0ed880 tx=0x7f9fcc0037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83ed849eadc342a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fb7fff640 1 -- 172.21.15.94:0/2287773273 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0035f0 con 0x562a1ea72250 2024-02-17T12:06:09.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fb7fff640 1 -- 172.21.15.94:0/2287773273 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9fcc0bd220 con 0x562a1ea72250 2024-02-17T12:06:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fe19da740 1 -- 172.21.15.94:0/2287773273 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a1e6d97e0 con 0x562a1ea72250 2024-02-17T12:06:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fb7fff640 1 -- 172.21.15.94:0/2287773273 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fcc0bcb20 con 0x562a1ea72250 2024-02-17T12:06:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fe19da740 10 client.17619 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fe19da740 1 -- 172.21.15.94:0/2287773273 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562a1e9d86a0 con 0x562a1ea72250 2024-02-17T12:06:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fb7fff640 1 -- 172.21.15.94:0/2287773273 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562a1e6d97e0 con 0x562a1ea72250 2024-02-17T12:06:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fb7fff640 1 -- 172.21.15.94:0/2287773273 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9fcc03c070 con 0x562a1ea72250 2024-02-17T12:06:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fe19da740 10 client.17619 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fe19da740 10 client.17619 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:06:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fe19da740 20 client.17619 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fe19da740 1 --2- 172.21.15.94:0/2287773273 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562a1ea8b8e0 0x562a1ea8dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fe19da740 4 client.17619 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.582+0000 7f9fe19da740 1 -- 172.21.15.94:0/2287773273 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a1e716720 con 0x562a1ea8b8e0 2024-02-17T12:06:09.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.583+0000 7f9fd7fff640 1 --2- 172.21.15.94:0/2287773273 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562a1ea8b8e0 0x562a1ea8dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.583+0000 7f9fd7fff640 1 --2- 172.21.15.94:0/2287773273 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562a1ea8b8e0 0x562a1ea8dcd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b5050d6e69583450 server_cookie=c9608a7d9cc99fd4 in_seq=0 out_seq=0 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.583+0000 7f9fb7fff640 10 client.17619 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fb7fff640 1 -- 172.21.15.94:0/2287773273 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a1e716720 con 0x562a1ea8b8e0 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fb7fff640 10 client.17619 handle_command_reply: tid=0 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fe19da740 1 client.17619 shutdown 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fe19da740 1 -- 172.21.15.94:0/2287773273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea72250 msgr2=0x562a1ea78760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fe19da740 1 --2- 172.21.15.94:0/2287773273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea72250 0x562a1ea78760 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0ed880 tx=0x7f9fcc0037d0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fe19da740 1 -- 172.21.15.94:0/2287773273 shutdown_connections 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fe19da740 1 --2- 172.21.15.94:0/2287773273 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562a1ea8b8e0 0x562a1ea8dcd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fb7fff640 0 client.17619 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fe19da740 1 --2- 172.21.15.94:0/2287773273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea7b750 0x562a1ea791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fe19da740 1 --2- 172.21.15.94:0/2287773273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72b70 0x562a1ea78ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fe19da740 1 --2- 172.21.15.94:0/2287773273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea72250 0x562a1ea78760 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fe19da740 1 -- 172.21.15.94:0/2287773273 >> 172.21.15.94:0/2287773273 conn(0x562a1ea6c980 msgr2=0x562a1ea6e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fe19da740 1 -- 172.21.15.94:0/2287773273 shutdown_connections 2024-02-17T12:06:09.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fe19da740 1 -- 172.21.15.94:0/2287773273 wait complete. 2024-02-17T12:06:09.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.584+0000 7f9fe19da740 20 client.17619 trim_cache size 0 max 16384 2024-02-17T12:06:09.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.596+0000 7f9fe19da740 1 Processor -- start 2024-02-17T12:06:09.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.596+0000 7f9fe19da740 1 -- start start 2024-02-17T12:06:09.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.596+0000 7f9fe19da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea72250 0x562a1ea79000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.596+0000 7f9fe19da740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72b70 0x562a1ea6ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.596+0000 7f9fe19da740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea7b750 0x562a1ea82600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.596+0000 7f9fe19da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1e95f130 con 0x562a1ea72250 2024-02-17T12:06:09.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.596+0000 7f9fe19da740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1ea71c90 con 0x562a1ea72b70 2024-02-17T12:06:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.596+0000 7f9fe19da740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1ea71eb0 con 0x562a1ea7b750 2024-02-17T12:06:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fdcac8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea72250 0x562a1ea79000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fdcac8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea72250 0x562a1ea79000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41448/0 (socket says 172.21.15.94:41448) 2024-02-17T12:06:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fdcac8640 1 -- 172.21.15.94:0/3838701646 learned_addr learned my addr 172.21.15.94:0/3838701646 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fdd2c9640 1 --2- 172.21.15.94:0/3838701646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea7b750 0x562a1ea82600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fd7fff640 1 --2- 172.21.15.94:0/3838701646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72b70 0x562a1ea6ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fdcac8640 1 -- 172.21.15.94:0/3838701646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea7b750 msgr2=0x562a1ea82600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fdcac8640 1 --2- 172.21.15.94:0/3838701646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea7b750 0x562a1ea82600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fdcac8640 1 -- 172.21.15.94:0/3838701646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72b70 msgr2=0x562a1ea6ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fdcac8640 1 --2- 172.21.15.94:0/3838701646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72b70 0x562a1ea6ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fdcac8640 1 -- 172.21.15.94:0/3838701646 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a1e974c20 con 0x562a1ea72250 2024-02-17T12:06:09.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fdcac8640 1 --2- 172.21.15.94:0/3838701646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea72250 0x562a1ea79000 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0bc820 tx=0x7f9fcc0bb030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f84bbada868bcb1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fc27fc640 1 -- 172.21.15.94:0/3838701646 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0bc8f0 con 0x562a1ea72250 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fc27fc640 1 -- 172.21.15.94:0/3838701646 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fcc0bc480 con 0x562a1ea72250 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.597+0000 7f9fc27fc640 1 -- 172.21.15.94:0/3838701646 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0c50e0 con 0x562a1ea72250 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.598+0000 7f9fe19da740 1 -- 172.21.15.94:0/3838701646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea72250 msgr2=0x562a1ea79000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.598+0000 7f9fe19da740 1 --2- 172.21.15.94:0/3838701646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea72250 0x562a1ea79000 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0bc820 tx=0x7f9fcc0bb030 comp rx=0 tx=0).stop 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.598+0000 7f9fe19da740 1 -- 172.21.15.94:0/3838701646 shutdown_connections 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.598+0000 7f9fe19da740 1 --2- 172.21.15.94:0/3838701646 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea7b750 0x562a1ea82600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.598+0000 7f9fe19da740 1 --2- 172.21.15.94:0/3838701646 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72b70 0x562a1ea6ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.598+0000 7f9fe19da740 1 --2- 172.21.15.94:0/3838701646 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea72250 0x562a1ea79000 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.598+0000 7f9fe19da740 1 -- 172.21.15.94:0/3838701646 >> 172.21.15.94:0/3838701646 conn(0x562a1ea6c980 msgr2=0x562a1ea6b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.598+0000 7f9fe19da740 1 -- 172.21.15.94:0/3838701646 shutdown_connections 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.598+0000 7f9fe19da740 1 -- 172.21.15.94:0/3838701646 wait complete. 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.598+0000 7f9fe19da740 1 -- start start 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.598+0000 7f9fe19da740 1 Processor -- start 2024-02-17T12:06:09.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fe19da740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72250 0x562a1ea80f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fe19da740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72b70 0x562a1ea814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fe19da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 0x562a1ea81a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fe19da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1ea82ed0 con 0x562a1ea7b750 2024-02-17T12:06:09.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fe19da740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1eae1dc0 con 0x562a1ea72250 2024-02-17T12:06:09.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fe19da740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a1eae1f30 con 0x562a1ea72b70 2024-02-17T12:06:09.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fe19da740 10 client.0 _ll_register_callbacks cb 0x7f9fc40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:09.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fdd2c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 0x562a1ea81a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fdd2c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 0x562a1ea81a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41456/0 (socket says 172.21.15.94:41456) 2024-02-17T12:06:09.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fdcac8640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72250 0x562a1ea80f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fdd2c9640 1 -- 172.21.15.94:0/3655778563 learned_addr learned my addr 172.21.15.94:0/3655778563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fd7fff640 1 --2- 172.21.15.94:0/3655778563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72b70 0x562a1ea814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fdd2c9640 1 -- 172.21.15.94:0/3655778563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72b70 msgr2=0x562a1ea814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fdd2c9640 1 --2- 172.21.15.94:0/3655778563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72b70 0x562a1ea814c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fdd2c9640 1 -- 172.21.15.94:0/3655778563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72250 msgr2=0x562a1ea80f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fdd2c9640 1 --2- 172.21.15.94:0/3655778563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72250 0x562a1ea80f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fdd2c9640 1 -- 172.21.15.94:0/3655778563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a1e9d86a0 con 0x562a1ea7b750 2024-02-17T12:06:09.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fd7fff640 1 --2- 172.21.15.94:0/3655778563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72b70 0x562a1ea814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:09.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fdd2c9640 1 --2- 172.21.15.94:0/3655778563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 0x562a1ea81a00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0003d20 tx=0x7f9fd0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89acea594823d7b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:09.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fb7fff640 1 -- 172.21.15.94:0/3655778563 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd001e640 con 0x562a1ea7b750 2024-02-17T12:06:09.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fb7fff640 1 -- 172.21.15.94:0/3655778563 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f9fd001e7a0 con 0x562a1ea7b750 2024-02-17T12:06:09.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.599+0000 7f9fe19da740 1 -- 172.21.15.94:0/3655778563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fc40035f0 con 0x562a1ea7b750 2024-02-17T12:06:09.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.600+0000 7f9fb7fff640 1 -- 172.21.15.94:0/3655778563 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fd001e960 con 0x562a1ea7b750 2024-02-17T12:06:09.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.600+0000 7f9fe19da740 10 client.17625 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:09.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.600+0000 7f9fe19da740 1 -- 172.21.15.94:0/3655778563 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562a1e99a8c0 con 0x562a1ea7b750 2024-02-17T12:06:09.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.600+0000 7f9fb7fff640 1 -- 172.21.15.94:0/3655778563 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fc40035f0 con 0x562a1ea7b750 2024-02-17T12:06:09.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.600+0000 7f9fb7fff640 1 -- 172.21.15.94:0/3655778563 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9fd0027070 con 0x562a1ea7b750 2024-02-17T12:06:09.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.600+0000 7f9fe19da740 10 client.17625 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:09.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.600+0000 7f9fe19da740 10 client.17625 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:06:09.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.600+0000 7f9fe19da740 20 client.17625 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:09.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.600+0000 7f9fe19da740 1 --2- 172.21.15.94:0/3655778563 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562a1ea8b8e0 0x562a1eae5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.600+0000 7f9fe19da740 4 client.17625 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:06:09.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.600+0000 7f9fe19da740 1 -- 172.21.15.94:0/3655778563 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562a1e765890 con 0x562a1ea8b8e0 2024-02-17T12:06:09.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.600+0000 7f9fdcac8640 1 --2- 172.21.15.94:0/3655778563 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562a1ea8b8e0 0x562a1eae5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fb7fff640 10 client.17625 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:06:09.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fdcac8640 1 --2- 172.21.15.94:0/3655778563 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562a1ea8b8e0 0x562a1eae5870 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae03cdce3f6ca73b server_cookie=de4da0fdfbd5f7f0 in_seq=0 out_seq=0 2024-02-17T12:06:09.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fb7fff640 1 -- 172.21.15.94:0/3655778563 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562a1e765890 con 0x562a1ea8b8e0 2024-02-17T12:06:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fb7fff640 10 client.17625 handle_command_reply: tid=0 2024-02-17T12:06:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fe19da740 1 client.17625 shutdown 2024-02-17T12:06:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fe19da740 1 -- 172.21.15.94:0/3655778563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 msgr2=0x562a1ea81a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fe19da740 1 --2- 172.21.15.94:0/3655778563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 0x562a1ea81a00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0003d20 tx=0x7f9fd0004ab0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fe19da740 1 -- 172.21.15.94:0/3655778563 shutdown_connections 2024-02-17T12:06:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fe19da740 1 --2- 172.21.15.94:0/3655778563 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562a1ea8b8e0 0x562a1eae5870 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fe19da740 1 --2- 172.21.15.94:0/3655778563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562a1ea7b750 0x562a1ea81a00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fb7fff640 0 client.17625 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:06:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fe19da740 1 --2- 172.21.15.94:0/3655778563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a1ea72b70 0x562a1ea814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fe19da740 1 --2- 172.21.15.94:0/3655778563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562a1ea72250 0x562a1ea80f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fe19da740 1 -- 172.21.15.94:0/3655778563 >> 172.21.15.94:0/3655778563 conn(0x562a1ea6c980 msgr2=0x562a1ea6bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fe19da740 1 -- 172.21.15.94:0/3655778563 shutdown_connections 2024-02-17T12:06:09.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fe19da740 1 -- 172.21.15.94:0/3655778563 wait complete. 2024-02-17T12:06:09.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.601+0000 7f9fe19da740 20 client.17625 trim_cache size 0 max 16384 2024-02-17T12:06:09.612 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:06:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.602+0000 7f9fded2c640 1 -- 172.21.15.94:0/1680683719 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fc80a2230 msgr2=0x7f9fc80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.602+0000 7f9fded2c640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fc80a2230 0x7f9fc80a46f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009270 tx=0x7f9fd0009b20 comp rx=0 tx=0).stop 2024-02-17T12:06:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.602+0000 7f9fded2c640 1 -- 172.21.15.94:0/1680683719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd81002a0 msgr2=0x7f9fd81966d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.602+0000 7f9fded2c640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd81002a0 0x7f9fd81966d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009230 tx=0x7f9fcc02c650 comp rx=0 tx=0).stop 2024-02-17T12:06:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.602+0000 7f9fded2c640 1 -- 172.21.15.94:0/1680683719 shutdown_connections 2024-02-17T12:06:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.602+0000 7f9fded2c640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fc80a2230 0x7f9fc80a46f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.602+0000 7f9fded2c640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fd819af90 0x7f9fd819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.602+0000 7f9fded2c640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fd8105370 0x7f9fd8196c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.602+0000 7f9fded2c640 1 --2- 172.21.15.94:0/1680683719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fd81002a0 0x7f9fd81966d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.602+0000 7f9fded2c640 1 -- 172.21.15.94:0/1680683719 >> 172.21.15.94:0/1680683719 conn(0x7f9fd8076400 msgr2=0x7f9fd80fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.602+0000 7f9fded2c640 1 -- 172.21.15.94:0/1680683719 shutdown_connections 2024-02-17T12:06:09.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.602+0000 7f9fded2c640 1 -- 172.21.15.94:0/1680683719 wait complete. 2024-02-17T12:06:09.615 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:06:09.615 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:06:09.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.726+0000 7f8fabbe4640 1 -- 172.21.15.94:0/2411888329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa4101d30 msgr2=0x7f8fa4105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.739 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.726+0000 7f8fabbe4640 1 --2- 172.21.15.94:0/2411888329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa4101d30 0x7f8fa4105d70 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009930 tx=0x7f8f9402bc30 comp rx=0 tx=0).stop 2024-02-17T12:06:09.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.726+0000 7f8fabbe4640 1 -- 172.21.15.94:0/2411888329 shutdown_connections 2024-02-17T12:06:09.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.726+0000 7f8fabbe4640 1 --2- 172.21.15.94:0/2411888329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8fa4106450 0x7f8fa410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.726+0000 7f8fabbe4640 1 --2- 172.21.15.94:0/2411888329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa4101d30 0x7f8fa4105d70 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.726+0000 7f8fabbe4640 1 --2- 172.21.15.94:0/2411888329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fa4101380 0x7f8fa4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.726+0000 7f8fabbe4640 1 -- 172.21.15.94:0/2411888329 >> 172.21.15.94:0/2411888329 conn(0x7f8fa4078580 msgr2=0x7f8fa4078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.726+0000 7f8fabbe4640 1 -- 172.21.15.94:0/2411888329 shutdown_connections 2024-02-17T12:06:09.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.726+0000 7f8fabbe4640 1 -- 172.21.15.94:0/2411888329 wait complete. 2024-02-17T12:06:09.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8fabbe4640 1 Processor -- start 2024-02-17T12:06:09.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8fabbe4640 1 -- start start 2024-02-17T12:06:09.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8fabbe4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fa4101380 0x7f8fa419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8fabbe4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8fa4106450 0x7f8fa419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8fabbe4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa4197610 0x7f8fa4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.741 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8fabbe4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa4198000 con 0x7f8fa4197610 2024-02-17T12:06:09.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8fabbe4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa4198170 con 0x7f8fa4106450 2024-02-17T12:06:09.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8fabbe4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa4198330 con 0x7f8fa4101380 2024-02-17T12:06:09.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8faa181640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa4197610 0x7f8fa4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8fa9980640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fa4101380 0x7f8fa419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8faa181640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa4197610 0x7f8fa4197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41466/0 (socket says 172.21.15.94:41466) 2024-02-17T12:06:09.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8faa181640 1 -- 172.21.15.94:0/1858048895 learned_addr learned my addr 172.21.15.94:0/1858048895 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.742 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8fa917f640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8fa4106450 0x7f8fa419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8faa181640 1 -- 172.21.15.94:0/1858048895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fa4101380 msgr2=0x7f8fa419d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8faa181640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fa4101380 0x7f8fa419d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8faa181640 1 -- 172.21.15.94:0/1858048895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8fa4106450 msgr2=0x7f8fa419d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8faa181640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8fa4106450 0x7f8fa419d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8faa181640 1 -- 172.21.15.94:0/1858048895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f94009560 con 0x7f8fa4197610 2024-02-17T12:06:09.743 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.727+0000 7f8fa917f640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8fa4106450 0x7f8fa419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:06:09.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.728+0000 7f8faa181640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa4197610 0x7f8fa4197ac0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009fd0 tx=0x7f8f9c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.728+0000 7f8f9affd640 1 -- 172.21.15.94:0/1858048895 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00bea0 con 0x7f8fa4197610 2024-02-17T12:06:09.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.728+0000 7f8f9affd640 1 -- 172.21.15.94:0/1858048895 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f9c009940 con 0x7f8fa4197610 2024-02-17T12:06:09.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.728+0000 7f8fabbe4640 1 -- 172.21.15.94:0/1858048895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa41a4320 con 0x7f8fa4197610 2024-02-17T12:06:09.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.728+0000 7f8f9affd640 1 -- 172.21.15.94:0/1858048895 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c0106c0 con 0x7f8fa4197610 2024-02-17T12:06:09.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.728+0000 7f8fabbe4640 1 -- 172.21.15.94:0/1858048895 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa41a47e0 con 0x7f8fa4197610 2024-02-17T12:06:09.744 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.730+0000 7f8f9affd640 1 -- 172.21.15.94:0/1858048895 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8f9c004240 con 0x7f8fa4197610 2024-02-17T12:06:09.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.730+0000 7f8fae7b0740 1 Processor -- start 2024-02-17T12:06:09.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fae7b0740 1 -- start start 2024-02-17T12:06:09.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fae7b0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fd3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fae7b0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3b70 0x563cb8fdc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fae7b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 0x563cb8fdeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fae7b0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8f8e960 con 0x563cb8fd3250 2024-02-17T12:06:09.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fae7b0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8f42310 con 0x563cb8fd3b70 2024-02-17T12:06:09.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fae7b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8f3f840 con 0x563cb8fdc750 2024-02-17T12:06:09.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8f9affd640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8f900a2230 0x7f8f900a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fa917f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fd3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8faa181640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8f900a2230 0x7f8f900a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.745 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fa917f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fd3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41480/0 (socket says 172.21.15.94:41480) 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fa917f640 1 -- 172.21.15.94:0/16730689 learned_addr learned my addr 172.21.15.94:0/16730689 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fa9980640 1 --2- 172.21.15.94:0/16730689 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3b70 0x563cb8fdc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fa917f640 1 --2- 172.21.15.94:0/16730689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 0x563cb8fdeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8f9affd640 1 -- 172.21.15.94:0/1858048895 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8f9c0cddf0 con 0x7f8fa4197610 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fa9980640 1 -- 172.21.15.94:0/16730689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 msgr2=0x563cb8fdeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fa9980640 1 --2- 172.21.15.94:0/16730689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 0x563cb8fdeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fa9980640 1 -- 172.21.15.94:0/16730689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 msgr2=0x563cb8fd3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fa9980640 1 --2- 172.21.15.94:0/16730689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fd3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fa9980640 1 -- 172.21.15.94:0/16730689 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cb8efd060 con 0x563cb8fd3b70 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.731+0000 7f8fa917f640 1 --2- 172.21.15.94:0/16730689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fd3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.732+0000 7f8faa181640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8f900a2230 0x7f8f900a46f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0cbad0 tx=0x7f8f9c0ba040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.732+0000 7f8fa9980640 1 --2- 172.21.15.94:0/16730689 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3b70 0x563cb8fdc210 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0005fd0 tx=0x7f8fa00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=564d0016594ee920 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.732+0000 7f8f8ffff640 1 -- 172.21.15.94:0/16730689 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa001c480 con 0x563cb8fd3b70 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.732+0000 7f8f8ffff640 1 -- 172.21.15.94:0/16730689 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fa0015070 con 0x563cb8fd3b70 2024-02-17T12:06:09.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.732+0000 7f8f8ffff640 1 -- 172.21.15.94:0/16730689 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0011a20 con 0x563cb8fd3b70 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.732+0000 7f8fae7b0740 1 -- 172.21.15.94:0/16730689 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3b70 msgr2=0x563cb8fdc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/16730689 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3b70 0x563cb8fdc210 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0005fd0 tx=0x7f8fa00107a0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 -- 172.21.15.94:0/16730689 shutdown_connections 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/16730689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 0x563cb8fdeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/16730689 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3b70 0x563cb8fdc210 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/16730689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fd3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 -- 172.21.15.94:0/16730689 >> 172.21.15.94:0/16730689 conn(0x563cb8fcd980 msgr2=0x563cb8fcdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 -- 172.21.15.94:0/16730689 shutdown_connections 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 -- 172.21.15.94:0/16730689 wait complete. 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 -- start start 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 Processor -- start 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fd97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fd3b70 0x563cb8fd9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fdc750 0x563cb8fda220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8f3b020 con 0x563cb8fd3250 2024-02-17T12:06:09.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8c0e430 con 0x563cb8fdc750 2024-02-17T12:06:09.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.733+0000 7f8fae7b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8bcae00 con 0x563cb8fd3b70 2024-02-17T12:06:09.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fa917f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fd97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fae7b0740 10 client.0 _ll_register_callbacks cb 0x563cb8ee2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:09.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fa9980640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fd3b70 0x563cb8fd9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fa917f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fd97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41492/0 (socket says 172.21.15.94:41492) 2024-02-17T12:06:09.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fa917f640 1 -- 172.21.15.94:0/1877680871 learned_addr learned my addr 172.21.15.94:0/1877680871 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fa917f640 1 --2- 172.21.15.94:0/1877680871 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fdc750 0x563cb8fda220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fa917f640 1 -- 172.21.15.94:0/1877680871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fd3b70 msgr2=0x563cb8fd9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fa917f640 1 --2- 172.21.15.94:0/1877680871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fd3b70 0x563cb8fd9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fa917f640 1 -- 172.21.15.94:0/1877680871 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fdc750 msgr2=0x563cb8fda220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fa917f640 1 --2- 172.21.15.94:0/1877680871 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fdc750 0x563cb8fda220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fa917f640 1 -- 172.21.15.94:0/1877680871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa000b8c0 con 0x563cb8fd3250 2024-02-17T12:06:09.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fa917f640 1 --2- 172.21.15.94:0/1877680871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fd97a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8f9402c170 tx=0x7f8f94005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e9c7d9e2dbbce0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8f8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:09.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8f8d7fa640 1 -- 172.21.15.94:0/1877680871 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94004ff0 con 0x563cb8fd3250 2024-02-17T12:06:09.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8f8d7fa640 1 -- 172.21.15.94:0/1877680871 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8f94005150 con 0x563cb8fd3250 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fae7b0740 1 -- 172.21.15.94:0/1877680871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563cb8c3a7e0 con 0x563cb8fd3250 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8f8d7fa640 1 -- 172.21.15.94:0/1877680871 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f94005310 con 0x563cb8fd3250 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fae7b0740 10 client.17637 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8fae7b0740 1 -- 172.21.15.94:0/1877680871 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563cb8f396a0 con 0x563cb8fd3250 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.734+0000 7f8f8d7fa640 1 -- 172.21.15.94:0/1877680871 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563cb8c3a7e0 con 0x563cb8fd3250 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.735+0000 7f8f8d7fa640 1 -- 172.21.15.94:0/1877680871 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8f9403c040 con 0x563cb8fd3250 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.735+0000 7f8fae7b0740 10 client.17637 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.735+0000 7f8fae7b0740 10 client.17637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.735+0000 7f8fae7b0740 20 client.17637 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.735+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/1877680871 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563cb8fecb80 0x563cb8feef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.735+0000 7f8fae7b0740 4 client.17637 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.735+0000 7f8fae7b0740 1 -- 172.21.15.94:0/1877680871 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563cb8c77720 con 0x563cb8fecb80 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.735+0000 7f8fa9980640 1 --2- 172.21.15.94:0/1877680871 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563cb8fecb80 0x563cb8feef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.735+0000 7f8fa9980640 1 --2- 172.21.15.94:0/1877680871 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563cb8fecb80 0x563cb8feef70 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb56743be4970e6 server_cookie=e551af38964a4d31 in_seq=0 out_seq=0 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.735+0000 7f8f8d7fa640 10 client.17637 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8f8d7fa640 1 -- 172.21.15.94:0/1877680871 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563cb8c77720 con 0x563cb8fecb80 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8f8d7fa640 10 client.17637 handle_command_reply: tid=0 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8fae7b0740 1 client.17637 shutdown 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8fae7b0740 1 -- 172.21.15.94:0/1877680871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 msgr2=0x563cb8fd97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/1877680871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fd97a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f8f9402c170 tx=0x7f8f94005a20 comp rx=0 tx=0).stop 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8fae7b0740 1 -- 172.21.15.94:0/1877680871 shutdown_connections 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/1877680871 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563cb8fecb80 0x563cb8feef70 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/1877680871 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fdc750 0x563cb8fda220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/1877680871 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fd3b70 0x563cb8fd9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8f8d7fa640 0 client.17637 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/1877680871 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fd97a0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8fae7b0740 1 -- 172.21.15.94:0/1877680871 >> 172.21.15.94:0/1877680871 conn(0x563cb8fcd980 msgr2=0x563cb8fcf6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8fae7b0740 1 -- 172.21.15.94:0/1877680871 shutdown_connections 2024-02-17T12:06:09.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8fae7b0740 1 -- 172.21.15.94:0/1877680871 wait complete. 2024-02-17T12:06:09.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.736+0000 7f8fae7b0740 20 client.17637 trim_cache size 0 max 16384 2024-02-17T12:06:09.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fae7b0740 1 Processor -- start 2024-02-17T12:06:09.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fae7b0740 1 -- start start 2024-02-17T12:06:09.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fae7b0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fdbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fae7b0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3b70 0x563cb8fd8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fae7b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 0x563cb8fd85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fae7b0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8ec0130 con 0x563cb8fd3250 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fae7b0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8fe9680 con 0x563cb8fd3b70 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fae7b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8fd2eb0 con 0x563cb8fdc750 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fa917f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fdbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fa917f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fdbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41496/0 (socket says 172.21.15.94:41496) 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fa917f640 1 -- 172.21.15.94:0/4199087311 learned_addr learned my addr 172.21.15.94:0/4199087311 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fa9980640 1 --2- 172.21.15.94:0/4199087311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3b70 0x563cb8fd8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fa917f640 1 -- 172.21.15.94:0/4199087311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 msgr2=0x563cb8fd85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fa917f640 1 --2- 172.21.15.94:0/4199087311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 0x563cb8fd85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fa917f640 1 -- 172.21.15.94:0/4199087311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3b70 msgr2=0x563cb8fd8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fa917f640 1 --2- 172.21.15.94:0/4199087311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3b70 0x563cb8fd8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fa917f640 1 -- 172.21.15.94:0/4199087311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563cb8ed5c20 con 0x563cb8fd3250 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.748+0000 7f8fa917f640 1 --2- 172.21.15.94:0/4199087311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fdbed0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009850 tx=0x7f8f9402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4788914b900d9ea server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.749+0000 7f8f8f7fe640 1 -- 172.21.15.94:0/4199087311 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94043700 con 0x563cb8fd3250 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.749+0000 7f8f8f7fe640 1 -- 172.21.15.94:0/4199087311 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f94005d50 con 0x563cb8fd3250 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.749+0000 7f8f8f7fe640 1 -- 172.21.15.94:0/4199087311 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94003ac0 con 0x563cb8fd3250 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.749+0000 7f8fae7b0740 1 -- 172.21.15.94:0/4199087311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 msgr2=0x563cb8fdbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.749+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/4199087311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fdbed0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009850 tx=0x7f8f9402bc30 comp rx=0 tx=0).stop 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.749+0000 7f8fae7b0740 1 -- 172.21.15.94:0/4199087311 shutdown_connections 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.749+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/4199087311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 0x563cb8fd85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.749+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/4199087311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3b70 0x563cb8fd8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.749+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/4199087311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3250 0x563cb8fdbed0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.749+0000 7f8fae7b0740 1 -- 172.21.15.94:0/4199087311 >> 172.21.15.94:0/4199087311 conn(0x563cb8fcd980 msgr2=0x563cb8ff38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.749+0000 7f8fae7b0740 1 -- 172.21.15.94:0/4199087311 shutdown_connections 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.749+0000 7f8fae7b0740 1 -- 172.21.15.94:0/4199087311 wait complete. 2024-02-17T12:06:09.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.749+0000 7f8fae7b0740 1 -- start start 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fae7b0740 1 Processor -- start 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fae7b0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3250 0x563cb8fe39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fae7b0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3b70 0x563cb8fe1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fae7b0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 0x563cb8fe2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fae7b0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8fe2a00 con 0x563cb8fd3b70 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fae7b0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb8fe2b70 con 0x563cb8fd3250 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fae7b0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563cb9043060 con 0x563cb8fdc750 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fae7b0740 10 client.0 _ll_register_callbacks cb 0x7f8fa000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fa9980640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3b70 0x563cb8fe1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fa9980640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3b70 0x563cb8fe1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41500/0 (socket says 172.21.15.94:41500) 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fa9980640 1 -- 172.21.15.94:0/2301729566 learned_addr learned my addr 172.21.15.94:0/2301729566 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fa917f640 1 --2- 172.21.15.94:0/2301729566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3250 0x563cb8fe39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fa917f640 1 --2- 172.21.15.94:0/2301729566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 0x563cb8fe2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fa9980640 1 -- 172.21.15.94:0/2301729566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 msgr2=0x563cb8fe2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fa9980640 1 --2- 172.21.15.94:0/2301729566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 0x563cb8fe2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fa9980640 1 -- 172.21.15.94:0/2301729566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3250 msgr2=0x563cb8fe39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fa9980640 1 --2- 172.21.15.94:0/2301729566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3250 0x563cb8fe39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fa9980640 1 -- 172.21.15.94:0/2301729566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563cb8f396a0 con 0x563cb8fd3b70 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8fa9980640 1 --2- 172.21.15.94:0/2301729566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3b70 0x563cb8fe1d60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f8fa001b000 tx=0x7f8fa0011580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98902acd5f95eff0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.750+0000 7f8f8d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8f8d7fa640 1 -- 172.21.15.94:0/2301729566 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa0021040 con 0x563cb8fd3b70 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8f8d7fa640 1 -- 172.21.15.94:0/2301729566 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f8fa0017070 con 0x563cb8fd3b70 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8fae7b0740 1 -- 172.21.15.94:0/2301729566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fa0015070 con 0x563cb8fd3b70 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8f8d7fa640 1 -- 172.21.15.94:0/2301729566 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fa001cea0 con 0x563cb8fd3b70 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8fae7b0740 10 client.17643 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8fae7b0740 1 -- 172.21.15.94:0/2301729566 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563cb8efb8c0 con 0x563cb8fd3b70 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8f8d7fa640 1 -- 172.21.15.94:0/2301729566 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8fa0015070 con 0x563cb8fd3b70 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8f8d7fa640 1 -- 172.21.15.94:0/2301729566 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8fa0011730 con 0x563cb8fd3b70 2024-02-17T12:06:09.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8fae7b0740 10 client.17643 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:09.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8fae7b0740 10 client.17643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:09.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8fae7b0740 20 client.17643 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:09.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/2301729566 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563cb90447d0 0x563cb9046bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:09.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8fae7b0740 4 client.17643 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:06:09.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8fae7b0740 1 -- 172.21.15.94:0/2301729566 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563cb8cc6890 con 0x563cb90447d0 2024-02-17T12:06:09.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.751+0000 7f8fa917f640 1 --2- 172.21.15.94:0/2301729566 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563cb90447d0 0x563cb9046bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:09.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8f8d7fa640 10 client.17643 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:09.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fa917f640 1 --2- 172.21.15.94:0/2301729566 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563cb90447d0 0x563cb9046bf0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da9a765fbb3cc879 server_cookie=b693876b44f3f00b in_seq=0 out_seq=0 2024-02-17T12:06:09.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8f8d7fa640 1 -- 172.21.15.94:0/2301729566 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563cb8cc6890 con 0x563cb90447d0 2024-02-17T12:06:09.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8f8d7fa640 10 client.17643 handle_command_reply: tid=0 2024-02-17T12:06:09.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fae7b0740 1 client.17643 shutdown 2024-02-17T12:06:09.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fae7b0740 1 -- 172.21.15.94:0/2301729566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3b70 msgr2=0x563cb8fe1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/2301729566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3b70 0x563cb8fe1d60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f8fa001b000 tx=0x7f8fa0011580 comp rx=0 tx=0).stop 2024-02-17T12:06:09.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fae7b0740 1 -- 172.21.15.94:0/2301729566 shutdown_connections 2024-02-17T12:06:09.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/2301729566 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563cb90447d0 0x563cb9046bf0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/2301729566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563cb8fdc750 0x563cb8fe2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/2301729566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563cb8fd3b70 0x563cb8fe1d60 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fae7b0740 1 --2- 172.21.15.94:0/2301729566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563cb8fd3250 0x563cb8fe39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8f8d7fa640 0 client.17643 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:09.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fae7b0740 1 -- 172.21.15.94:0/2301729566 >> 172.21.15.94:0/2301729566 conn(0x563cb8fcd980 msgr2=0x563cb8fd8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fae7b0740 1 -- 172.21.15.94:0/2301729566 shutdown_connections 2024-02-17T12:06:09.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fae7b0740 1 -- 172.21.15.94:0/2301729566 wait complete. 2024-02-17T12:06:09.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fae7b0740 20 client.17643 trim_cache size 0 max 16384 2024-02-17T12:06:09.764 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:06:09.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.752+0000 7f8fabbe4640 1 -- 172.21.15.94:0/1858048895 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8f900a2230 msgr2=0x7f8f900a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.753+0000 7f8fabbe4640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8f900a2230 0x7f8f900a46f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c0cbad0 tx=0x7f8f9c0ba040 comp rx=0 tx=0).stop 2024-02-17T12:06:09.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.753+0000 7f8fabbe4640 1 -- 172.21.15.94:0/1858048895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa4197610 msgr2=0x7f8fa4197ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:09.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.753+0000 7f8fabbe4640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa4197610 0x7f8fa4197ac0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c009fd0 tx=0x7f8f9c00b260 comp rx=0 tx=0).stop 2024-02-17T12:06:09.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.753+0000 7f8fabbe4640 1 -- 172.21.15.94:0/1858048895 shutdown_connections 2024-02-17T12:06:09.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.753+0000 7f8fabbe4640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8f900a2230 0x7f8f900a46f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.753+0000 7f8fabbe4640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8fa4197610 0x7f8fa4197ac0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.753+0000 7f8fabbe4640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8fa4106450 0x7f8fa419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.753+0000 7f8fabbe4640 1 --2- 172.21.15.94:0/1858048895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8fa4101380 0x7f8fa419d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:09.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.753+0000 7f8fabbe4640 1 -- 172.21.15.94:0/1858048895 >> 172.21.15.94:0/1858048895 conn(0x7f8fa4078580 msgr2=0x7f8fa40ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:09.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.753+0000 7f8fabbe4640 1 -- 172.21.15.94:0/1858048895 shutdown_connections 2024-02-17T12:06:09.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:09.753+0000 7f8fabbe4640 1 -- 172.21.15.94:0/1858048895 wait complete. 2024-02-17T12:06:10.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 470.0 s, 188.8 tps, lat 543.523 ms stddev 336.672 2024-02-17T12:06:10.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:10 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1677831747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:06:10.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:10 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1677831747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:06:10.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:10 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1677831747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:06:10.767 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:06:10.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:06:10.767 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:06:10.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.926+0000 7f57f4ec3640 1 -- 172.21.15.94:0/3984912780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f0102420 msgr2=0x7f57f00ffb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:10.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.926+0000 7f57f4ec3640 1 --2- 172.21.15.94:0/3984912780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f0102420 0x7f57f00ffb60 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009900 tx=0x7f57d802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.927+0000 7f57f4ec3640 1 -- 172.21.15.94:0/3984912780 shutdown_connections 2024-02-17T12:06:10.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.927+0000 7f57f4ec3640 1 --2- 172.21.15.94:0/3984912780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57f0100180 0x7f57f010fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.927+0000 7f57f4ec3640 1 --2- 172.21.15.94:0/3984912780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f0102420 0x7f57f00ffb60 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.927+0000 7f57f4ec3640 1 --2- 172.21.15.94:0/3984912780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57f0102040 0x7f57f00ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.927+0000 7f57f4ec3640 1 -- 172.21.15.94:0/3984912780 >> 172.21.15.94:0/3984912780 conn(0x7f57f00765a0 msgr2=0x7f57f00fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.927+0000 7f57f4ec3640 1 -- 172.21.15.94:0/3984912780 shutdown_connections 2024-02-17T12:06:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.927+0000 7f57f4ec3640 1 -- 172.21.15.94:0/3984912780 wait complete. 2024-02-17T12:06:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.928+0000 7f57f4ec3640 1 Processor -- start 2024-02-17T12:06:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.928+0000 7f57f4ec3640 1 -- start start 2024-02-17T12:06:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.929+0000 7f57f4ec3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f0100180 0x7f57f01029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.929+0000 7f57f4ec3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57f0102040 0x7f57f010f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.929+0000 7f57f4ec3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57f0102420 0x7f57f0102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.929+0000 7f57f4ec3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f010fe90 con 0x7f57f0100180 2024-02-17T12:06:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.929+0000 7f57f4ec3640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f0110000 con 0x7f57f0102420 2024-02-17T12:06:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.929+0000 7f57f4ec3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f0103440 con 0x7f57f0102040 2024-02-17T12:06:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.929+0000 7f57ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f0100180 0x7f57f01029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.929+0000 7f57ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f0100180 0x7f57f01029c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41520/0 (socket says 172.21.15.94:41520) 2024-02-17T12:06:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.929+0000 7f57ee59c640 1 -- 172.21.15.94:0/3687069929 learned_addr learned my addr 172.21.15.94:0/3687069929 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.929+0000 7f57eed9d640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57f0102420 0x7f57f0102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.929+0000 7f57eed9d640 1 -- 172.21.15.94:0/3687069929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57f0102040 msgr2=0x7f57f010f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.929+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57f0102040 0x7f57f010f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.930+0000 7f57eed9d640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57f0102040 0x7f57f010f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.930+0000 7f57eed9d640 1 -- 172.21.15.94:0/3687069929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f0100180 msgr2=0x7f57f01029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.930+0000 7f57eed9d640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f0100180 0x7f57f01029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.930+0000 7f57eed9d640 1 -- 172.21.15.94:0/3687069929 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d8009560 con 0x7f57f0102420 2024-02-17T12:06:10.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.930+0000 7f57ee59c640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f0100180 0x7f57f01029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.930+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57f0102040 0x7f57f010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:06:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.930+0000 7f57eed9d640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57f0102420 0x7f57f0102f00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009fd0 tx=0x7f57e800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.930+0000 7f57e7fff640 1 -- 172.21.15.94:0/3687069929 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e800ce20 con 0x7f57f0102420 2024-02-17T12:06:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.930+0000 7f57f4ec3640 1 -- 172.21.15.94:0/3687069929 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f01036c0 con 0x7f57f0102420 2024-02-17T12:06:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.931+0000 7f57f4ec3640 1 -- 172.21.15.94:0/3687069929 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f01b3440 con 0x7f57f0102420 2024-02-17T12:06:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.931+0000 7f57e7fff640 1 -- 172.21.15.94:0/3687069929 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57e8005020 con 0x7f57f0102420 2024-02-17T12:06:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.931+0000 7f57e7fff640 1 -- 172.21.15.94:0/3687069929 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8010690 con 0x7f57f0102420 2024-02-17T12:06:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.933+0000 7f57f7b72740 1 Processor -- start 2024-02-17T12:06:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.934+0000 7f57e7fff640 1 -- 172.21.15.94:0/3687069929 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f57e8004a40 con 0x7f57f0102420 2024-02-17T12:06:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.934+0000 7f57f7b72740 1 -- start start 2024-02-17T12:06:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.934+0000 7f57f7b72740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac68760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.934+0000 7f57f7b72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 0x55b41ac712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.934+0000 7f57f7b72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 0x55b41ac73d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.934+0000 7f57f7b72740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41abd6f90 con 0x55b41ac718b0 2024-02-17T12:06:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.934+0000 7f57f7b72740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41abd44c0 con 0x55b41ac68380 2024-02-17T12:06:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.934+0000 7f57f7b72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41abcfca0 con 0x55b41ac68d30 2024-02-17T12:06:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.934+0000 7f57e7fff640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f57dc0a2330 0x7f57dc0a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57ee59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 0x55b41ac712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57ee59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 0x55b41ac712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57924/0 (socket says 172.21.15.94:57924) 2024-02-17T12:06:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57ee59c640 1 -- 172.21.15.94:0/1809896478 learned_addr learned my addr 172.21.15.94:0/1809896478 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57eed9d640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f57dc0a2330 0x7f57dc0a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/1809896478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac68760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/1809896478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 0x55b41ac73d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57e7fff640 1 -- 172.21.15.94:0/3687069929 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f57e8014070 con 0x7f57f0102420 2024-02-17T12:06:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57ee59c640 1 -- 172.21.15.94:0/1809896478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 msgr2=0x55b41ac68760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57ee59c640 1 --2- 172.21.15.94:0/1809896478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac68760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57ee59c640 1 -- 172.21.15.94:0/1809896478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 msgr2=0x55b41ac73d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57ee59c640 1 --2- 172.21.15.94:0/1809896478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 0x55b41ac73d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57ee59c640 1 -- 172.21.15.94:0/1809896478 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b41ab920b0 con 0x55b41ac68d30 2024-02-17T12:06:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/1809896478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 0x55b41ac73d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:06:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.935+0000 7f57eed9d640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f57dc0a2330 0x7f57dc0a47f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f57e809adb0 tx=0x7f57e8090780 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.936+0000 7f57ee59c640 1 --2- 172.21.15.94:0/1809896478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 0x55b41ac712e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f57e0010580 tx=0x7f57e0010a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4835504227b09a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.936+0000 7f57e4d99640 1 -- 172.21.15.94:0/1809896478 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e000bc40 con 0x55b41ac68d30 2024-02-17T12:06:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.936+0000 7f57e4d99640 1 -- 172.21.15.94:0/1809896478 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57e0015070 con 0x55b41ac68d30 2024-02-17T12:06:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.936+0000 7f57e4d99640 1 -- 172.21.15.94:0/1809896478 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0011d80 con 0x55b41ac68d30 2024-02-17T12:06:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.937+0000 7f57f7b72740 1 -- 172.21.15.94:0/1809896478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 msgr2=0x55b41ac712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.937+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1809896478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 0x55b41ac712e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f57e0010580 tx=0x7f57e0010a40 comp rx=0 tx=0).stop 2024-02-17T12:06:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.937+0000 7f57f7b72740 1 -- 172.21.15.94:0/1809896478 shutdown_connections 2024-02-17T12:06:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.937+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1809896478 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 0x55b41ac73d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.937+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1809896478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 0x55b41ac712e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.937+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1809896478 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac68760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.937+0000 7f57f7b72740 1 -- 172.21.15.94:0/1809896478 >> 172.21.15.94:0/1809896478 conn(0x55b41ac62840 msgr2=0x55b41ac62c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57f7b72740 1 -- 172.21.15.94:0/1809896478 shutdown_connections 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57f7b72740 1 -- 172.21.15.94:0/1809896478 wait complete. 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57f7b72740 1 -- start start 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57f7b72740 1 Processor -- start 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57f7b72740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac6e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57f7b72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 0x55b41ac6ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57f7b72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 0x55b41ac6f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57f7b72740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41ab74f40 con 0x55b41ac718b0 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57f7b72740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41a8a34b0 con 0x55b41ac68380 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57f7b72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41ab55670 con 0x55b41ac68d30 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57f7b72740 10 client.0 _ll_register_callbacks cb 0x55b41abc2f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57e5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac6e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57e5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac6e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46532/0 (socket says 172.21.15.94:46532) 2024-02-17T12:06:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57e5d9b640 1 -- 172.21.15.94:0/1839758366 learned_addr learned my addr 172.21.15.94:0/1839758366 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.938+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/1839758366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 0x55b41ac6f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.939+0000 7f57e5d9b640 1 -- 172.21.15.94:0/1839758366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 msgr2=0x55b41ac6ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.939+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/1839758366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 0x55b41ac6ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.939+0000 7f57e5d9b640 1 -- 172.21.15.94:0/1839758366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 msgr2=0x55b41ac6f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.939+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/1839758366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 0x55b41ac6f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.939+0000 7f57e5d9b640 1 -- 172.21.15.94:0/1839758366 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e000bda0 con 0x55b41ac68380 2024-02-17T12:06:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.939+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/1839758366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac6e7e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009900 tx=0x7f57d802bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=980fa7677732c1f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.939+0000 7f57d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:06:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.939+0000 7f57d5ffb640 1 -- 172.21.15.94:0/1839758366 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8004460 con 0x55b41ac68380 2024-02-17T12:06:10.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.939+0000 7f57d5ffb640 1 -- 172.21.15.94:0/1839758366 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f57d8004df0 con 0x55b41ac68380 2024-02-17T12:06:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.939+0000 7f57f7b72740 1 -- 172.21.15.94:0/1839758366 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b41aba9f40 con 0x55b41ac68380 2024-02-17T12:06:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.940+0000 7f57d5ffb640 1 -- 172.21.15.94:0/1839758366 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57d8004fb0 con 0x55b41ac68380 2024-02-17T12:06:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.940+0000 7f57f7b72740 10 client.27125 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.940+0000 7f57f7b72740 1 -- 172.21.15.94:0/1839758366 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b41abce320 con 0x55b41ac68380 2024-02-17T12:06:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.940+0000 7f57d5ffb640 1 -- 172.21.15.94:0/1839758366 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b41aba9f40 con 0x55b41ac68380 2024-02-17T12:06:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.940+0000 7f57d5ffb640 1 -- 172.21.15.94:0/1839758366 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f57d8003c70 con 0x55b41ac68380 2024-02-17T12:06:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.940+0000 7f57f7b72740 10 client.27125 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.940+0000 7f57f7b72740 10 client.27125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.940+0000 7f57f7b72740 20 client.27125 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.940+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1839758366 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b41ac81f50 0x55b41ac84370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.940+0000 7f57f7b72740 4 client.27125 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.940+0000 7f57f7b72740 1 -- 172.21.15.94:0/1839758366 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b41abc6310 con 0x55b41ac81f50 2024-02-17T12:06:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.940+0000 7f57ee59c640 1 --2- 172.21.15.94:0/1839758366 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b41ac81f50 0x55b41ac84370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.942+0000 7f57ee59c640 1 --2- 172.21.15.94:0/1839758366 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b41ac81f50 0x55b41ac84370 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a779321f6dca348 server_cookie=4cb5da8d3b68a90a in_seq=0 out_seq=0 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.942+0000 7f57d5ffb640 10 client.27125 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.942+0000 7f57d5ffb640 1 -- 172.21.15.94:0/1839758366 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b41abc6310 con 0x55b41ac81f50 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.942+0000 7f57d5ffb640 10 client.27125 handle_command_reply: tid=0 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.942+0000 7f57f7b72740 1 client.27125 shutdown 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.943+0000 7f57f7b72740 1 -- 172.21.15.94:0/1839758366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 msgr2=0x55b41ac6e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.943+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1839758366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac6e7e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009900 tx=0x7f57d802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.943+0000 7f57f7b72740 1 -- 172.21.15.94:0/1839758366 shutdown_connections 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.943+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1839758366 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b41ac81f50 0x55b41ac84370 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.943+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1839758366 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 0x55b41ac6f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.943+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1839758366 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 0x55b41ac6ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.943+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1839758366 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac6e7e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.943+0000 7f57f7b72740 1 -- 172.21.15.94:0/1839758366 >> 172.21.15.94:0/1839758366 conn(0x55b41ac62840 msgr2=0x55b41ac647a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.943+0000 7f57d5ffb640 0 client.27125 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.943+0000 7f57f7b72740 1 -- 172.21.15.94:0/1839758366 shutdown_connections 2024-02-17T12:06:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.943+0000 7f57f7b72740 1 -- 172.21.15.94:0/1839758366 wait complete. 2024-02-17T12:06:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.943+0000 7f57f7b72740 20 client.27125 trim_cache size 0 max 16384 2024-02-17T12:06:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.964+0000 7f57f7b72740 1 Processor -- start 2024-02-17T12:06:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.964+0000 7f57f7b72740 1 -- start start 2024-02-17T12:06:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.965+0000 7f57f7b72740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac70f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.965+0000 7f57f7b72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac68d30 0x55b41ac69b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.965+0000 7f57e5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac70f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.965+0000 7f57ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac68d30 0x55b41ac69b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.965+0000 7f57ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac68d30 0x55b41ac69b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41552/0 (socket says 172.21.15.94:41552) 2024-02-17T12:06:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.965+0000 7f57e5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac70f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46534/0 (socket says 172.21.15.94:46534) 2024-02-17T12:06:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.965+0000 7f57ee59c640 1 -- 172.21.15.94:0/2891339007 learned_addr learned my addr 172.21.15.94:0/2891339007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.965+0000 7f57f7b72740 1 --2- 172.21.15.94:0/2891339007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac718b0 0x55b41ac6a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.965+0000 7f57f7b72740 1 -- 172.21.15.94:0/2891339007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41ac7d880 con 0x55b41ac68d30 2024-02-17T12:06:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.965+0000 7f57f7b72740 1 -- 172.21.15.94:0/2891339007 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41ac74700 con 0x55b41ac68380 2024-02-17T12:06:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.965+0000 7f57f7b72740 1 -- 172.21.15.94:0/2891339007 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41ac74160 con 0x55b41ac718b0 2024-02-17T12:06:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.966+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/2891339007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac718b0 0x55b41ac6a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.966+0000 7f57e5d9b640 1 -- 172.21.15.94:0/2891339007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac718b0 msgr2=0x55b41ac6a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.966+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/2891339007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac718b0 0x55b41ac6a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.966+0000 7f57e5d9b640 1 -- 172.21.15.94:0/2891339007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac68d30 msgr2=0x55b41ac69b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.966+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/2891339007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac68d30 0x55b41ac69b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.966+0000 7f57e5d9b640 1 -- 172.21.15.94:0/2891339007 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b41ab6ac00 con 0x55b41ac68380 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.966+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/2891339007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac70f10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f57d802c210 tx=0x7f57d8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2997ad48a3bf97f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.967+0000 7f57d7fff640 1 -- 172.21.15.94:0/2891339007 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d80436f0 con 0x55b41ac68380 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.967+0000 7f57d7fff640 1 -- 172.21.15.94:0/2891339007 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f57d8049030 con 0x55b41ac68380 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.967+0000 7f57d7fff640 1 -- 172.21.15.94:0/2891339007 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d80038e0 con 0x55b41ac68380 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.968+0000 7f57f7b72740 1 -- 172.21.15.94:0/2891339007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 msgr2=0x55b41ac70f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.968+0000 7f57f7b72740 1 --2- 172.21.15.94:0/2891339007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac70f10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f57d802c210 tx=0x7f57d8005d50 comp rx=0 tx=0).stop 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.968+0000 7f57f7b72740 1 -- 172.21.15.94:0/2891339007 shutdown_connections 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.968+0000 7f57f7b72740 1 --2- 172.21.15.94:0/2891339007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac718b0 0x55b41ac6a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.968+0000 7f57f7b72740 1 --2- 172.21.15.94:0/2891339007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac68d30 0x55b41ac69b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.968+0000 7f57f7b72740 1 --2- 172.21.15.94:0/2891339007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac70f10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.968+0000 7f57f7b72740 1 -- 172.21.15.94:0/2891339007 >> 172.21.15.94:0/2891339007 conn(0x55b41ac62840 msgr2=0x55b41ac88cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.968+0000 7f57f7b72740 1 -- 172.21.15.94:0/2891339007 shutdown_connections 2024-02-17T12:06:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.968+0000 7f57f7b72740 1 -- 172.21.15.94:0/2891339007 wait complete. 2024-02-17T12:06:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.968+0000 7f57f7b72740 1 -- start start 2024-02-17T12:06:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.968+0000 7f57f7b72740 1 Processor -- start 2024-02-17T12:06:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.969+0000 7f57f7b72740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac78660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.969+0000 7f57f7b72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 0x55b41ac78ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.969+0000 7f57f7b72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 0x55b41ac76c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.969+0000 7f57e5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac78660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.969+0000 7f57e5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac78660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46538/0 (socket says 172.21.15.94:46538) 2024-02-17T12:06:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.969+0000 7f57ee59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 0x55b41ac78ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.969+0000 7f57e5d9b640 1 -- 172.21.15.94:0/1438954022 learned_addr learned my addr 172.21.15.94:0/1438954022 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.969+0000 7f57f7b72740 1 -- 172.21.15.94:0/1438954022 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41ac771c0 con 0x55b41ac718b0 2024-02-17T12:06:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.969+0000 7f57f7b72740 1 -- 172.21.15.94:0/1438954022 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41ac77330 con 0x55b41ac68380 2024-02-17T12:06:10.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.969+0000 7f57f7b72740 1 -- 172.21.15.94:0/1438954022 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b41ac774f0 con 0x55b41ac68d30 2024-02-17T12:06:10.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.969+0000 7f57f7b72740 10 client.0 _ll_register_callbacks cb 0x7f57e000bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:10.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.969+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/1438954022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 0x55b41ac76c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57e5d9b640 1 -- 172.21.15.94:0/1438954022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 msgr2=0x55b41ac78ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:10.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/1438954022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 0x55b41ac78ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57e5d9b640 1 -- 172.21.15.94:0/1438954022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 msgr2=0x55b41ac76c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:10.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/1438954022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 0x55b41ac76c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57e5d9b640 1 -- 172.21.15.94:0/1438954022 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b41abce320 con 0x55b41ac68380 2024-02-17T12:06:10.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57e5d9b640 1 --2- 172.21.15.94:0/1438954022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac78660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f57d802c2b0 tx=0x7f57d8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e53f53aa2c21ea6d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:10.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:06:10.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57d5ffb640 1 -- 172.21.15.94:0/1438954022 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d805e040 con 0x55b41ac68380 2024-02-17T12:06:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57d5ffb640 1 -- 172.21.15.94:0/1438954022 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f57d8000e40 con 0x55b41ac68380 2024-02-17T12:06:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57f7b72740 1 -- 172.21.15.94:0/1438954022 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b41abf5900 con 0x55b41ac68380 2024-02-17T12:06:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57d5ffb640 1 -- 172.21.15.94:0/1438954022 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57d8043b20 con 0x55b41ac68380 2024-02-17T12:06:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57f7b72740 10 client.27131 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57f7b72740 1 -- 172.21.15.94:0/1438954022 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b41ab90910 con 0x55b41ac68380 2024-02-17T12:06:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.970+0000 7f57d5ffb640 1 -- 172.21.15.94:0/1438954022 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b41abf5900 con 0x55b41ac68380 2024-02-17T12:06:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.971+0000 7f57d5ffb640 1 -- 172.21.15.94:0/1438954022 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f57d8064020 con 0x55b41ac68380 2024-02-17T12:06:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.971+0000 7f57f7b72740 10 client.27131 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.971+0000 7f57f7b72740 10 client.27131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.971+0000 7f57f7b72740 20 client.27131 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.971+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1438954022 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b41acd9290 0x55b41acdb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.971+0000 7f57f7b72740 4 client.27131 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:06:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.971+0000 7f57f7b72740 1 -- 172.21.15.94:0/1438954022 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b41a8cf860 con 0x55b41acd9290 2024-02-17T12:06:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.971+0000 7f57ee59c640 1 --2- 172.21.15.94:0/1438954022 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b41acd9290 0x55b41acdb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.971+0000 7f57d5ffb640 10 client.27131 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.971+0000 7f57ee59c640 1 --2- 172.21.15.94:0/1438954022 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b41acd9290 0x55b41acdb6b0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=847533ae4358a33 server_cookie=4b2dceb52c05b308 in_seq=0 out_seq=0 2024-02-17T12:06:10.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.972+0000 7f57d5ffb640 1 -- 172.21.15.94:0/1438954022 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b41a8cf860 con 0x55b41acd9290 2024-02-17T12:06:10.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.972+0000 7f57d5ffb640 10 client.27131 handle_command_reply: tid=0 2024-02-17T12:06:10.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.972+0000 7f57f7b72740 1 client.27131 shutdown 2024-02-17T12:06:10.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.973+0000 7f57f7b72740 1 -- 172.21.15.94:0/1438954022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 msgr2=0x55b41ac78660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:10.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.973+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1438954022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac78660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f57d802c2b0 tx=0x7f57d8005fd0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.973+0000 7f57f7b72740 1 -- 172.21.15.94:0/1438954022 shutdown_connections 2024-02-17T12:06:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.973+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1438954022 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b41acd9290 0x55b41acdb6b0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.973+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1438954022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b41ac718b0 0x55b41ac76c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.973+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1438954022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b41ac68d30 0x55b41ac78ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.973+0000 7f57f7b72740 1 --2- 172.21.15.94:0/1438954022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b41ac68380 0x55b41ac78660 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.973+0000 7f57f7b72740 1 -- 172.21.15.94:0/1438954022 >> 172.21.15.94:0/1438954022 conn(0x55b41ac62840 msgr2=0x55b41ac69b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.973+0000 7f57d5ffb640 0 client.27131 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.973+0000 7f57f7b72740 1 -- 172.21.15.94:0/1438954022 shutdown_connections 2024-02-17T12:06:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.973+0000 7f57f7b72740 1 -- 172.21.15.94:0/1438954022 wait complete. 2024-02-17T12:06:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.973+0000 7f57f7b72740 20 client.27131 trim_cache size 0 max 16384 2024-02-17T12:06:10.981 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:06:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:06:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "a7425257-1e4d-4980-a9ed-5100c09308ac", 2024-02-17T12:06:10.982 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:06:10.982 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:06:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.975+0000 7f57f4ec3640 1 -- 172.21.15.94:0/3687069929 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f57dc0a2330 msgr2=0x7f57dc0a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.975+0000 7f57f4ec3640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f57dc0a2330 0x7f57dc0a47f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f57e809adb0 tx=0x7f57e8090780 comp rx=0 tx=0).stop 2024-02-17T12:06:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.975+0000 7f57f4ec3640 1 -- 172.21.15.94:0/3687069929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57f0102420 msgr2=0x7f57f0102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.975+0000 7f57f4ec3640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57f0102420 0x7f57f0102f00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f57e8009fd0 tx=0x7f57e800ef10 comp rx=0 tx=0).stop 2024-02-17T12:06:10.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.975+0000 7f57f4ec3640 1 -- 172.21.15.94:0/3687069929 shutdown_connections 2024-02-17T12:06:10.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.975+0000 7f57f4ec3640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f57dc0a2330 0x7f57dc0a47f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.975+0000 7f57f4ec3640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f57f0102420 0x7f57f0102f00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.975+0000 7f57f4ec3640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57f0102040 0x7f57f010f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.975+0000 7f57f4ec3640 1 --2- 172.21.15.94:0/3687069929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f57f0100180 0x7f57f01029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:10.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.975+0000 7f57f4ec3640 1 -- 172.21.15.94:0/3687069929 >> 172.21.15.94:0/3687069929 conn(0x7f57f00765a0 msgr2=0x7f57f00fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:10.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.975+0000 7f57f4ec3640 1 -- 172.21.15.94:0/3687069929 shutdown_connections 2024-02-17T12:06:10.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:10.975+0000 7f57f4ec3640 1 -- 172.21.15.94:0/3687069929 wait complete. 2024-02-17T12:06:10.990 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:06:10.991 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:06:11.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.116+0000 7fb65a8c9640 1 -- 172.21.15.94:0/1175907843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb654100270 msgr2=0x7fb654100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.116+0000 7fb65a8c9640 1 --2- 172.21.15.94:0/1175907843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb654100270 0x7fb654100650 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb644009930 tx=0x7fb64402bc30 comp rx=0 tx=0).stop 2024-02-17T12:06:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.116+0000 7fb65a8c9640 1 -- 172.21.15.94:0/1175907843 shutdown_connections 2024-02-17T12:06:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.116+0000 7fb65a8c9640 1 --2- 172.21.15.94:0/1175907843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb654105340 0x7fb65410cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.116+0000 7fb65a8c9640 1 --2- 172.21.15.94:0/1175907843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb654100c20 0x7fb654104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.116+0000 7fb65a8c9640 1 --2- 172.21.15.94:0/1175907843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb654100270 0x7fb654100650 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.116+0000 7fb65a8c9640 1 -- 172.21.15.94:0/1175907843 >> 172.21.15.94:0/1175907843 conn(0x7fb654077230 msgr2=0x7fb654077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.116+0000 7fb65a8c9640 1 -- 172.21.15.94:0/1175907843 shutdown_connections 2024-02-17T12:06:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.116+0000 7fb65a8c9640 1 -- 172.21.15.94:0/1175907843 wait complete. 2024-02-17T12:06:11.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.117+0000 7fb65a8c9640 1 Processor -- start 2024-02-17T12:06:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.117+0000 7fb65a8c9640 1 -- start start 2024-02-17T12:06:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.117+0000 7fb65a8c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb654100270 0x7fb65419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.117+0000 7fb65a8c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb654105340 0x7fb65419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.117+0000 7fb65a8c9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6541975b0 0x7fb654197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.117+0000 7fb65a8c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb654197fa0 con 0x7fb654105340 2024-02-17T12:06:11.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.117+0000 7fb65a8c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb654198110 con 0x7fb654100270 2024-02-17T12:06:11.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.117+0000 7fb65a8c9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6541982d0 con 0x7fb6541975b0 2024-02-17T12:06:11.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb654105340 0x7fb65419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb654105340 0x7fb65419d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41584/0 (socket says 172.21.15.94:41584) 2024-02-17T12:06:11.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb6537fe640 1 -- 172.21.15.94:0/55036412 learned_addr learned my addr 172.21.15.94:0/55036412 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:11.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb658e66640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6541975b0 0x7fb654197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb653fff640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb654100270 0x7fb65419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb6537fe640 1 -- 172.21.15.94:0/55036412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6541975b0 msgr2=0x7fb654197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb6537fe640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6541975b0 0x7fb654197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb6537fe640 1 -- 172.21.15.94:0/55036412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb654100270 msgr2=0x7fb65419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb6537fe640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb654100270 0x7fb65419d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb6537fe640 1 -- 172.21.15.94:0/55036412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb644009560 con 0x7fb654105340 2024-02-17T12:06:11.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb653fff640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb654100270 0x7fb65419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:06:11.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb6537fe640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb654105340 0x7fb65419d940 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb648005fd0 tx=0x7fb64800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:11.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb6517fa640 1 -- 172.21.15.94:0/55036412 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb648009c30 con 0x7fb654105340 2024-02-17T12:06:11.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb65a8c9640 1 -- 172.21.15.94:0/55036412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6541a42c0 con 0x7fb654105340 2024-02-17T12:06:11.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb6517fa640 1 -- 172.21.15.94:0/55036412 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb648009d90 con 0x7fb654105340 2024-02-17T12:06:11.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb6517fa640 1 -- 172.21.15.94:0/55036412 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb648010460 con 0x7fb654105340 2024-02-17T12:06:11.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.118+0000 7fb65a8c9640 1 -- 172.21.15.94:0/55036412 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6541a4820 con 0x7fb654105340 2024-02-17T12:06:11.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.121+0000 7fb6517fa640 1 -- 172.21.15.94:0/55036412 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb648003f20 con 0x7fb654105340 2024-02-17T12:06:11.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.121+0000 7fb65d570740 1 Processor -- start 2024-02-17T12:06:11.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.121+0000 7fb65d570740 1 -- start start 2024-02-17T12:06:11.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.121+0000 7fb65d570740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6602b0 0x556d7d660690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.121+0000 7fb65d570740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d669270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.121+0000 7fb65d570740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6697b0 0x556d7d66bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.121+0000 7fb65d570740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7d5cf280 con 0x556d7d6697b0 2024-02-17T12:06:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.121+0000 7fb65d570740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7d5cc7b0 con 0x556d7d6602b0 2024-02-17T12:06:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.121+0000 7fb65d570740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7d5c7f90 con 0x556d7d660bd0 2024-02-17T12:06:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb6517fa640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb63c0a2230 0x7fb63c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6697b0 0x556d7d66bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb653fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d669270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb653fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d669270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57972/0 (socket says 172.21.15.94:57972) 2024-02-17T12:06:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb653fff640 1 -- 172.21.15.94:0/1306888576 learned_addr learned my addr 172.21.15.94:0/1306888576 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb658e66640 1 --2- 172.21.15.94:0/1306888576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6602b0 0x556d7d660690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6697b0 0x556d7d66bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41598/0 (socket says 172.21.15.94:41598) 2024-02-17T12:06:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb658e66640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb63c0a2230 0x7fb63c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb6517fa640 1 -- 172.21.15.94:0/55036412 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fb6480cdd80 con 0x7fb654105340 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb653fff640 1 -- 172.21.15.94:0/1306888576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6602b0 msgr2=0x556d7d660690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb653fff640 1 --2- 172.21.15.94:0/1306888576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6602b0 0x556d7d660690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb653fff640 1 -- 172.21.15.94:0/1306888576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6697b0 msgr2=0x556d7d66bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb653fff640 1 --2- 172.21.15.94:0/1306888576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6697b0 0x556d7d66bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb653fff640 1 -- 172.21.15.94:0/1306888576 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7d586570 con 0x556d7d660bd0 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb658e66640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb63c0a2230 0x7fb63c0a46f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009fd0 tx=0x7fb64c009600 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb653fff640 1 --2- 172.21.15.94:0/1306888576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d669270 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb644000c00 tx=0x7fb64402c700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2679e10b7d97a2b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.122+0000 7fb63a7fc640 1 -- 172.21.15.94:0/1306888576 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644004b00 con 0x556d7d660bd0 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.123+0000 7fb63a7fc640 1 -- 172.21.15.94:0/1306888576 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb644004030 con 0x556d7d660bd0 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.123+0000 7fb63a7fc640 1 -- 172.21.15.94:0/1306888576 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644005280 con 0x556d7d660bd0 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.123+0000 7fb65d570740 1 -- 172.21.15.94:0/1306888576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 msgr2=0x556d7d669270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.123+0000 7fb65d570740 1 --2- 172.21.15.94:0/1306888576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d669270 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb644000c00 tx=0x7fb64402c700 comp rx=0 tx=0).stop 2024-02-17T12:06:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.123+0000 7fb65d570740 1 -- 172.21.15.94:0/1306888576 shutdown_connections 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.124+0000 7fb65d570740 1 --2- 172.21.15.94:0/1306888576 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6697b0 0x556d7d66bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.124+0000 7fb65d570740 1 --2- 172.21.15.94:0/1306888576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d669270 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.124+0000 7fb65d570740 1 --2- 172.21.15.94:0/1306888576 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6602b0 0x556d7d660690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.124+0000 7fb65d570740 1 -- 172.21.15.94:0/1306888576 >> 172.21.15.94:0/1306888576 conn(0x556d7d65a990 msgr2=0x556d7d65ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.124+0000 7fb65d570740 1 -- 172.21.15.94:0/1306888576 shutdown_connections 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.124+0000 7fb65d570740 1 -- 172.21.15.94:0/1306888576 wait complete. 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.124+0000 7fb65d570740 1 -- start start 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.124+0000 7fb65d570740 1 Processor -- start 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb65d570740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 0x556d7d669140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb65d570740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d6664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb65d570740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6697b0 0x556d7d666a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb65d570740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7d29b430 con 0x556d7d6602b0 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb65d570740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7d257e00 con 0x556d7d6697b0 2024-02-17T12:06:11.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb65d570740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7d54cfd0 con 0x556d7d660bd0 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb65d570740 10 client.0 _ll_register_callbacks cb 0x556d7d5a2fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 0x556d7d669140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 0x556d7d669140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41608/0 (socket says 172.21.15.94:41608) 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb653fff640 1 -- 172.21.15.94:0/589854063 learned_addr learned my addr 172.21.15.94:0/589854063 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d6664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb658e66640 1 --2- 172.21.15.94:0/589854063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6697b0 0x556d7d666a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb653fff640 1 -- 172.21.15.94:0/589854063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 msgr2=0x556d7d6664d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb653fff640 1 --2- 172.21.15.94:0/589854063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d6664d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb653fff640 1 -- 172.21.15.94:0/589854063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6697b0 msgr2=0x556d7d666a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb653fff640 1 --2- 172.21.15.94:0/589854063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6697b0 0x556d7d666a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb653fff640 1 -- 172.21.15.94:0/589854063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65410d720 con 0x556d7d6602b0 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb658e66640 1 --2- 172.21.15.94:0/589854063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6697b0 0x556d7d666a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb653fff640 1 --2- 172.21.15.94:0/589854063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 0x556d7d669140 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb644005940 tx=0x7fb644005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a3e37de8cca914 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb6337fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb6337fe640 1 -- 172.21.15.94:0/589854063 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64402cbf0 con 0x556d7d6602b0 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb6337fe640 1 -- 172.21.15.94:0/589854063 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fb644005080 con 0x556d7d6602b0 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.125+0000 7fb65d570740 1 -- 172.21.15.94:0/589854063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d7d2c77e0 con 0x556d7d6602b0 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.126+0000 7fb6337fe640 1 -- 172.21.15.94:0/589854063 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb644005280 con 0x556d7d6602b0 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.126+0000 7fb65d570740 10 client.17673 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.126+0000 7fb65d570740 1 -- 172.21.15.94:0/589854063 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556d7d5c6610 con 0x556d7d6602b0 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.126+0000 7fb6337fe640 1 -- 172.21.15.94:0/589854063 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556d7d2c77e0 con 0x556d7d6602b0 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.126+0000 7fb6337fe640 1 -- 172.21.15.94:0/589854063 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb64403c040 con 0x556d7d6602b0 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.126+0000 7fb65d570740 10 client.17673 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.126+0000 7fb65d570740 10 client.17673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.126+0000 7fb65d570740 20 client.17673 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.126+0000 7fb65d570740 1 --2- 172.21.15.94:0/589854063 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556d7d6795a0 0x556d7d67b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.126+0000 7fb65d570740 4 client.17673 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.126+0000 7fb65d570740 1 -- 172.21.15.94:0/589854063 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d7d304720 con 0x556d7d6795a0 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.127+0000 7fb6537fe640 1 --2- 172.21.15.94:0/589854063 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556d7d6795a0 0x556d7d67b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.127+0000 7fb6337fe640 10 client.17673 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.127+0000 7fb6537fe640 1 --2- 172.21.15.94:0/589854063 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556d7d6795a0 0x556d7d67b990 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50137e341f386fc4 server_cookie=de5669247ccb869a in_seq=0 out_seq=0 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb6337fe640 1 -- 172.21.15.94:0/589854063 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d7d304720 con 0x556d7d6795a0 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb6337fe640 10 client.17673 handle_command_reply: tid=0 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb65d570740 1 client.17673 shutdown 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb65d570740 1 -- 172.21.15.94:0/589854063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 msgr2=0x556d7d669140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb65d570740 1 --2- 172.21.15.94:0/589854063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 0x556d7d669140 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb644005940 tx=0x7fb644005bd0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb65d570740 1 -- 172.21.15.94:0/589854063 shutdown_connections 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb65d570740 1 --2- 172.21.15.94:0/589854063 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556d7d6795a0 0x556d7d67b990 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb65d570740 1 --2- 172.21.15.94:0/589854063 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6697b0 0x556d7d666a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb65d570740 1 --2- 172.21.15.94:0/589854063 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d6664d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb65d570740 1 --2- 172.21.15.94:0/589854063 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 0x556d7d669140 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb65d570740 1 -- 172.21.15.94:0/589854063 >> 172.21.15.94:0/589854063 conn(0x556d7d65a990 msgr2=0x556d7d65c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:11.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb6337fe640 0 client.17673 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:11.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb65d570740 1 -- 172.21.15.94:0/589854063 shutdown_connections 2024-02-17T12:06:11.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb65d570740 1 -- 172.21.15.94:0/589854063 wait complete. 2024-02-17T12:06:11.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.128+0000 7fb65d570740 20 client.17673 trim_cache size 0 max 16384 2024-02-17T12:06:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb65d570740 1 Processor -- start 2024-02-17T12:06:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb65d570740 1 -- start start 2024-02-17T12:06:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb65d570740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d65a990 0x556d7d668c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb65d570740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 0x556d7d669140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb65d570740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d660bd0 0x556d7d664de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb65d570740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7d65b790 con 0x556d7d6602b0 2024-02-17T12:06:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb65d570740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7d65f680 con 0x556d7d660bd0 2024-02-17T12:06:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb65d570740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7d680ba0 con 0x556d7d65a990 2024-02-17T12:06:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 0x556d7d669140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb6537fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 0x556d7d669140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41624/0 (socket says 172.21.15.94:41624) 2024-02-17T12:06:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb6537fe640 1 -- 172.21.15.94:0/3362012584 learned_addr learned my addr 172.21.15.94:0/3362012584 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb658e66640 1 --2- 172.21.15.94:0/3362012584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d660bd0 0x556d7d664de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb653fff640 1 --2- 172.21.15.94:0/3362012584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d65a990 0x556d7d668c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb6537fe640 1 -- 172.21.15.94:0/3362012584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d65a990 msgr2=0x556d7d668c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:11.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb6537fe640 1 --2- 172.21.15.94:0/3362012584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d65a990 0x556d7d668c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb6537fe640 1 -- 172.21.15.94:0/3362012584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d660bd0 msgr2=0x556d7d664de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:11.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb6537fe640 1 --2- 172.21.15.94:0/3362012584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d660bd0 0x556d7d664de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.144+0000 7fb6537fe640 1 -- 172.21.15.94:0/3362012584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7d562ac0 con 0x556d7d6602b0 2024-02-17T12:06:11.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.145+0000 7fb6537fe640 1 --2- 172.21.15.94:0/3362012584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 0x556d7d669140 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb648009610 tx=0x7fb6480d0220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a371817bd414550a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:11.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.145+0000 7fb639ffb640 1 -- 172.21.15.94:0/3362012584 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64809ae00 con 0x556d7d6602b0 2024-02-17T12:06:11.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.145+0000 7fb639ffb640 1 -- 172.21.15.94:0/3362012584 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb64809a700 con 0x556d7d6602b0 2024-02-17T12:06:11.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.145+0000 7fb639ffb640 1 -- 172.21.15.94:0/3362012584 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb648098b20 con 0x556d7d6602b0 2024-02-17T12:06:11.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.146+0000 7fb65d570740 1 -- 172.21.15.94:0/3362012584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 msgr2=0x556d7d669140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:11.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.146+0000 7fb65d570740 1 --2- 172.21.15.94:0/3362012584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 0x556d7d669140 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fb648009610 tx=0x7fb6480d0220 comp rx=0 tx=0).stop 2024-02-17T12:06:11.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.146+0000 7fb65d570740 1 -- 172.21.15.94:0/3362012584 shutdown_connections 2024-02-17T12:06:11.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.146+0000 7fb65d570740 1 --2- 172.21.15.94:0/3362012584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d660bd0 0x556d7d664de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.146+0000 7fb65d570740 1 --2- 172.21.15.94:0/3362012584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d6602b0 0x556d7d669140 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.146+0000 7fb65d570740 1 --2- 172.21.15.94:0/3362012584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d65a990 0x556d7d668c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.146+0000 7fb65d570740 1 -- 172.21.15.94:0/3362012584 >> 172.21.15.94:0/3362012584 conn(0x556d7d66bec0 msgr2=0x556d7d661810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:11.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.146+0000 7fb65d570740 1 -- 172.21.15.94:0/3362012584 shutdown_connections 2024-02-17T12:06:11.163 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.146+0000 7fb65d570740 1 -- 172.21.15.94:0/3362012584 wait complete. 2024-02-17T12:06:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.146+0000 7fb65d570740 1 -- start start 2024-02-17T12:06:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.146+0000 7fb65d570740 1 Processor -- start 2024-02-17T12:06:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb65d570740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d65a990 0x556d7d6630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb65d570740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6602b0 0x556d7d6635e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb65d570740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d663b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb65d570740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7d670f00 con 0x556d7d65a990 2024-02-17T12:06:11.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb65d570740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7d6cfa90 con 0x556d7d6602b0 2024-02-17T12:06:11.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb65d570740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7d6cfbd0 con 0x556d7d660bd0 2024-02-17T12:06:11.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb65d570740 10 client.0 _ll_register_callbacks cb 0x7fb644009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:11.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d65a990 0x556d7d6630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb653fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d65a990 0x556d7d6630a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41636/0 (socket says 172.21.15.94:41636) 2024-02-17T12:06:11.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb653fff640 1 -- 172.21.15.94:0/1004676520 learned_addr learned my addr 172.21.15.94:0/1004676520 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:11.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb6537fe640 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6602b0 0x556d7d6635e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb658e66640 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d663b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb653fff640 1 -- 172.21.15.94:0/1004676520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 msgr2=0x556d7d663b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:11.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb653fff640 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d663b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb653fff640 1 -- 172.21.15.94:0/1004676520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6602b0 msgr2=0x556d7d6635e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb653fff640 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6602b0 0x556d7d6635e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb653fff640 1 -- 172.21.15.94:0/1004676520 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6480b8e70 con 0x556d7d65a990 2024-02-17T12:06:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb6537fe640 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6602b0 0x556d7d6635e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb653fff640 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d65a990 0x556d7d6630a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb6440045c0 tx=0x7fb644005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd07f85e587c420 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb6337fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb6337fe640 1 -- 172.21.15.94:0/1004676520 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644003a80 con 0x556d7d65a990 2024-02-17T12:06:11.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb6337fe640 1 -- 172.21.15.94:0/1004676520 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fb644000e40 con 0x556d7d65a990 2024-02-17T12:06:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb65d570740 1 -- 172.21.15.94:0/1004676520 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb644004030 con 0x556d7d65a990 2024-02-17T12:06:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb6337fe640 1 -- 172.21.15.94:0/1004676520 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb644003be0 con 0x556d7d65a990 2024-02-17T12:06:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb65d570740 10 client.17679 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.147+0000 7fb65d570740 1 -- 172.21.15.94:0/1004676520 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556d7d5886a0 con 0x556d7d65a990 2024-02-17T12:06:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb6337fe640 1 -- 172.21.15.94:0/1004676520 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb644004030 con 0x556d7d65a990 2024-02-17T12:06:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb6337fe640 1 -- 172.21.15.94:0/1004676520 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb644034670 con 0x556d7d65a990 2024-02-17T12:06:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb65d570740 10 client.17679 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:11.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb65d570740 10 client.17679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb65d570740 20 client.17679 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb65d570740 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556d7d6697b0 0x556d7d6d3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb65d570740 4 client.17679 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:06:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb65d570740 1 -- 172.21.15.94:0/1004676520 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556d7d353890 con 0x556d7d6697b0 2024-02-17T12:06:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb6537fe640 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556d7d6697b0 0x556d7d6d3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb6337fe640 10 client.17679 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb6537fe640 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556d7d6697b0 0x556d7d6d3000 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7ba539974a68085 server_cookie=2a9160d43293283e in_seq=0 out_seq=0 2024-02-17T12:06:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb6337fe640 1 -- 172.21.15.94:0/1004676520 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x556d7d353890 con 0x556d7d6697b0 2024-02-17T12:06:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb6337fe640 10 client.17679 handle_command_reply: tid=0 2024-02-17T12:06:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.148+0000 7fb65d570740 1 client.17679 shutdown 2024-02-17T12:06:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65d570740 1 -- 172.21.15.94:0/1004676520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d65a990 msgr2=0x556d7d6630a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:11.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65d570740 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d65a990 0x556d7d6630a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb6440045c0 tx=0x7fb644005c90 comp rx=0 tx=0).stop 2024-02-17T12:06:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65d570740 1 -- 172.21.15.94:0/1004676520 shutdown_connections 2024-02-17T12:06:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65d570740 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556d7d6697b0 0x556d7d6d3000 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65d570740 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556d7d660bd0 0x556d7d663b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65d570740 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556d7d6602b0 0x556d7d6635e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb6337fe640 0 client.17679 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65d570740 1 --2- 172.21.15.94:0/1004676520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556d7d65a990 0x556d7d6630a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65d570740 1 -- 172.21.15.94:0/1004676520 >> 172.21.15.94:0/1004676520 conn(0x556d7d66bec0 msgr2=0x556d7d664da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65d570740 1 -- 172.21.15.94:0/1004676520 shutdown_connections 2024-02-17T12:06:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65d570740 1 -- 172.21.15.94:0/1004676520 wait complete. 2024-02-17T12:06:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65d570740 20 client.17679 trim_cache size 0 max 16384 2024-02-17T12:06:11.173 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:06:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:06:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:06:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "a7425257-1e4d-4980-a9ed-5100c09308ac": { 2024-02-17T12:06:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:06:11.173 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "a7425257-1e4d-4980-a9ed-5100c09308ac", 2024-02-17T12:06:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:06:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:06:11.174 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:06:11.174 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:06:11.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65a8c9640 1 -- 172.21.15.94:0/55036412 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb63c0a2230 msgr2=0x7fb63c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65a8c9640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb63c0a2230 0x7fb63c0a46f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009fd0 tx=0x7fb64c009600 comp rx=0 tx=0).stop 2024-02-17T12:06:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65a8c9640 1 -- 172.21.15.94:0/55036412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb654105340 msgr2=0x7fb65419d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65a8c9640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb654105340 0x7fb65419d940 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb648005fd0 tx=0x7fb64800dde0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65a8c9640 1 -- 172.21.15.94:0/55036412 shutdown_connections 2024-02-17T12:06:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65a8c9640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb63c0a2230 0x7fb63c0a46f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65a8c9640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6541975b0 0x7fb654197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65a8c9640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb654105340 0x7fb65419d940 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65a8c9640 1 --2- 172.21.15.94:0/55036412 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb654100270 0x7fb65419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65a8c9640 1 -- 172.21.15.94:0/55036412 >> 172.21.15.94:0/55036412 conn(0x7fb654077230 msgr2=0x7fb654068640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65a8c9640 1 -- 172.21.15.94:0/55036412 shutdown_connections 2024-02-17T12:06:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:11.149+0000 7fb65a8c9640 1 -- 172.21.15.94:0/55036412 wait complete. 2024-02-17T12:06:11.178 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:11 smithi094 ceph-mon[32499]: pgmap v248: 97 pgs: 7 active+clean+snaptrim_wait, 85 active+clean, 5 active+clean+snaptrim; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-02-17T12:06:11.179 INFO:tasks.cephfs.filesystem:scrub status for tag:a7425257-1e4d-4980-a9ed-5100c09308ac - {'path': '/', 'tag': 'a7425257-1e4d-4980-a9ed-5100c09308ac', 'options': 'recursive,force'} 2024-02-17T12:06:11.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:11 smithi189 ceph-mon[36588]: pgmap v248: 97 pgs: 7 active+clean+snaptrim_wait, 85 active+clean, 5 active+clean+snaptrim; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-02-17T12:06:11.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:11 smithi125 ceph-mon[36819]: pgmap v248: 97 pgs: 7 active+clean+snaptrim_wait, 85 active+clean, 5 active+clean+snaptrim; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 3.1 MiB/s wr, 450 op/s 2024-02-17T12:06:12.337 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:12 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:06:12.337 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:12 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:06:12.337 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:12 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:06:12.337 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:12 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:06:12.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:12 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:06:12.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:12 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:06:12.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:12 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:06:12.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:12 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:06:12.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:12 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:06:12.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:12 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:06:12.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:12 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:06:12.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:12 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:06:13.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:13 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:06:13.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:13 smithi189 ceph-mon[36588]: pgmap v249: 97 pgs: 7 active+clean+snaptrim_wait, 85 active+clean, 5 active+clean+snaptrim; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.0 MiB/s wr, 550 op/s 2024-02-17T12:06:13.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:13 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:06:13.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:13 smithi125 ceph-mon[36819]: pgmap v249: 97 pgs: 7 active+clean+snaptrim_wait, 85 active+clean, 5 active+clean+snaptrim; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.0 MiB/s wr, 550 op/s 2024-02-17T12:06:13.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:13 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:06:13.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:13 smithi094 ceph-mon[32499]: pgmap v249: 97 pgs: 7 active+clean+snaptrim_wait, 85 active+clean, 5 active+clean+snaptrim; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 6.0 MiB/s wr, 550 op/s 2024-02-17T12:06:15.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 475.0 s, 629.6 tps, lat 166.012 ms stddev 244.628 2024-02-17T12:06:16.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:15 smithi189 ceph-mon[36588]: pgmap v250: 97 pgs: 97 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 18 MiB/s wr, 1.09k op/s 2024-02-17T12:06:16.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:15 smithi189 ceph-mon[36588]: osdmap e92: 12 total, 12 up, 12 in 2024-02-17T12:06:16.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:15 smithi125 ceph-mon[36819]: pgmap v250: 97 pgs: 97 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 18 MiB/s wr, 1.09k op/s 2024-02-17T12:06:16.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:15 smithi125 ceph-mon[36819]: osdmap e92: 12 total, 12 up, 12 in 2024-02-17T12:06:16.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:15 smithi094 ceph-mon[32499]: pgmap v250: 97 pgs: 97 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 18 MiB/s wr, 1.09k op/s 2024-02-17T12:06:16.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:15 smithi094 ceph-mon[32499]: osdmap e92: 12 total, 12 up, 12 in 2024-02-17T12:06:18.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:17 smithi189 ceph-mon[36588]: pgmap v252: 97 pgs: 97 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 20 MiB/s wr, 1.12k op/s 2024-02-17T12:06:18.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:17 smithi125 ceph-mon[36819]: pgmap v252: 97 pgs: 97 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 20 MiB/s wr, 1.12k op/s 2024-02-17T12:06:18.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:17 smithi094 ceph-mon[32499]: pgmap v252: 97 pgs: 97 active+clean; 5.0 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 20 MiB/s wr, 1.12k op/s 2024-02-17T12:06:19.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:18 smithi189 ceph-mon[36588]: pgmap v253: 97 pgs: 97 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 36 MiB/s wr, 1.76k op/s 2024-02-17T12:06:19.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:18 smithi125 ceph-mon[36819]: pgmap v253: 97 pgs: 97 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 36 MiB/s wr, 1.76k op/s 2024-02-17T12:06:19.456 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:18 smithi094 ceph-mon[32499]: pgmap v253: 97 pgs: 97 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 36 MiB/s wr, 1.76k op/s 2024-02-17T12:06:20.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 480.0 s, 2366.8 tps, lat 42.461 ms stddev 50.032 2024-02-17T12:06:21.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:20 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:06:21.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:20 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:06:21.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:20 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:06:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.566+0000 7fb779915640 1 -- 172.21.15.94:0/3649650996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7741051d0 msgr2=0x7fb77410cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.566+0000 7fb779915640 1 --2- 172.21.15.94:0/3649650996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7741051d0 0x7fb77410cda0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009cd0 tx=0x7fb76c02c0f0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.567+0000 7fb779915640 1 -- 172.21.15.94:0/3649650996 shutdown_connections 2024-02-17T12:06:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.567+0000 7fb779915640 1 --2- 172.21.15.94:0/3649650996 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7741051d0 0x7fb77410cda0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.567+0000 7fb779915640 1 --2- 172.21.15.94:0/3649650996 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb774100ab0 0x7fb774104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.567+0000 7fb779915640 1 --2- 172.21.15.94:0/3649650996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb774100100 0x7fb7741004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.567+0000 7fb779915640 1 -- 172.21.15.94:0/3649650996 >> 172.21.15.94:0/3649650996 conn(0x7fb7740fb980 msgr2=0x7fb7740fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.569+0000 7fb779915640 1 -- 172.21.15.94:0/3649650996 shutdown_connections 2024-02-17T12:06:21.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.569+0000 7fb779915640 1 -- 172.21.15.94:0/3649650996 wait complete. 2024-02-17T12:06:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.569+0000 7fb779915640 1 Processor -- start 2024-02-17T12:06:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb779915640 1 -- start start 2024-02-17T12:06:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb779915640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb774100100 0x7fb774197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb779915640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7741051d0 0x7fb774197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb779915640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb77419bcd0 0x7fb77419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb779915640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77419c6c0 con 0x7fb774100100 2024-02-17T12:06:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb779915640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77419c830 con 0x7fb77419bcd0 2024-02-17T12:06:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb779915640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77419c9a0 con 0x7fb7741051d0 2024-02-17T12:06:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb778913640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb774100100 0x7fb774197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb778913640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb774100100 0x7fb774197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47628/0 (socket says 172.21.15.94:47628) 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb778913640 1 -- 172.21.15.94:0/998983533 learned_addr learned my addr 172.21.15.94:0/998983533 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb773fff640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7741051d0 0x7fb774197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb779114640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb77419bcd0 0x7fb77419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb778913640 1 -- 172.21.15.94:0/998983533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7741051d0 msgr2=0x7fb774197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb778913640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7741051d0 0x7fb774197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb778913640 1 -- 172.21.15.94:0/998983533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb77419bcd0 msgr2=0x7fb77419c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb778913640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb77419bcd0 0x7fb77419c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.570+0000 7fb778913640 1 -- 172.21.15.94:0/998983533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76c009900 con 0x7fb774100100 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.571+0000 7fb778913640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb774100100 0x7fb774197410 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb76800e9f0 tx=0x7fb76800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.571+0000 7fb771ffb640 1 -- 172.21.15.94:0/998983533 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768009720 con 0x7fb774100100 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.571+0000 7fb771ffb640 1 -- 172.21.15.94:0/998983533 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb76800cd80 con 0x7fb774100100 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.571+0000 7fb779915640 1 -- 172.21.15.94:0/998983533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7741a3f80 con 0x7fb774100100 2024-02-17T12:06:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.571+0000 7fb771ffb640 1 -- 172.21.15.94:0/998983533 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768010690 con 0x7fb774100100 2024-02-17T12:06:21.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.571+0000 7fb779915640 1 -- 172.21.15.94:0/998983533 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7741a4340 con 0x7fb774100100 2024-02-17T12:06:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.575+0000 7fb77d827740 1 Processor -- start 2024-02-17T12:06:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.575+0000 7fb77d827740 1 -- start start 2024-02-17T12:06:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.575+0000 7fb77d827740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064638d6b0 0x56064638da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.575+0000 7fb77d827740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064637e130 0x56064637e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.575+0000 7fb77d827740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646395320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.575+0000 7fb77d827740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606462ef9f0 con 0x56064637eaf0 2024-02-17T12:06:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.575+0000 7fb77d827740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606462ecf20 con 0x56064637e130 2024-02-17T12:06:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.575+0000 7fb77d827740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606462e8700 con 0x56064638d6b0 2024-02-17T12:06:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.575+0000 7fb773fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646395320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.575+0000 7fb773fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646395320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47640/0 (socket says 172.21.15.94:47640) 2024-02-17T12:06:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.575+0000 7fb779114640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064638d6b0 0x56064638da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.575+0000 7fb773fff640 1 -- 172.21.15.94:0/2679549077 learned_addr learned my addr 172.21.15.94:0/2679549077 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.575+0000 7fb771ffb640 1 -- 172.21.15.94:0/998983533 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb7680046c0 con 0x7fb774100100 2024-02-17T12:06:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.576+0000 7fb779114640 1 -- 172.21.15.94:0/2679549077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064637e130 msgr2=0x56064637e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.576+0000 7fb779114640 1 --2- 172.21.15.94:0/2679549077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064637e130 0x56064637e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.576+0000 7fb779114640 1 -- 172.21.15.94:0/2679549077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 msgr2=0x560646395320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.576+0000 7fb779114640 1 --2- 172.21.15.94:0/2679549077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646395320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.576+0000 7fb779114640 1 -- 172.21.15.94:0/2679549077 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56064627e7b0 con 0x56064638d6b0 2024-02-17T12:06:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.576+0000 7fb779114640 1 --2- 172.21.15.94:0/2679549077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064638d6b0 0x56064638da90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009bf0 tx=0x7fb76c009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2ae9bdb005e392 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.576+0000 7fb75effd640 1 -- 172.21.15.94:0/2679549077 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c034440 con 0x56064638d6b0 2024-02-17T12:06:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.576+0000 7fb75effd640 1 -- 172.21.15.94:0/2679549077 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb76c0345a0 con 0x56064638d6b0 2024-02-17T12:06:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.576+0000 7fb75effd640 1 -- 172.21.15.94:0/2679549077 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c003bb0 con 0x56064638d6b0 2024-02-17T12:06:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.577+0000 7fb771ffb640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb7640a2320 0x7fb7640a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.577+0000 7fb778913640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb7640a2320 0x7fb7640a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.577+0000 7fb771ffb640 1 -- 172.21.15.94:0/998983533 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb768014070 con 0x7fb774100100 2024-02-17T12:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.577+0000 7fb77d827740 1 -- 172.21.15.94:0/2679549077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064638d6b0 msgr2=0x56064638da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.577+0000 7fb778913640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb7640a2320 0x7fb7640a47e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x56064637de00 tx=0x7fb76801b4f0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.577+0000 7fb77d827740 1 --2- 172.21.15.94:0/2679549077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064638d6b0 0x56064638da90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009bf0 tx=0x7fb76c009540 comp rx=0 tx=0).stop 2024-02-17T12:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.577+0000 7fb77d827740 1 -- 172.21.15.94:0/2679549077 shutdown_connections 2024-02-17T12:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.577+0000 7fb77d827740 1 --2- 172.21.15.94:0/2679549077 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646395320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.577+0000 7fb77d827740 1 --2- 172.21.15.94:0/2679549077 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064637e130 0x56064637e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.577+0000 7fb77d827740 1 --2- 172.21.15.94:0/2679549077 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064638d6b0 0x56064638da90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.577+0000 7fb77d827740 1 -- 172.21.15.94:0/2679549077 >> 172.21.15.94:0/2679549077 conn(0x560646348550 msgr2=0x56064637d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.577+0000 7fb77d827740 1 -- 172.21.15.94:0/2679549077 shutdown_connections 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.577+0000 7fb77d827740 1 -- 172.21.15.94:0/2679549077 wait complete. 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.578+0000 7fb77d827740 1 -- start start 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.578+0000 7fb77d827740 1 Processor -- start 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.578+0000 7fb77d827740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 0x5606463928f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb77d827740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646392e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb77d827740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064638d6b0 0x560646393370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb77d827740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064628dbf0 con 0x56064637eaf0 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb77d827740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560645fbc250 con 0x56064638d6b0 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb77d827740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064626e200 con 0x56064637e130 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb77d827740 10 client.0 _ll_register_callbacks cb 0x5606462dcd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb779114640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 0x5606463928f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb779114640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 0x5606463928f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44174/0 (socket says 172.21.15.94:44174) 2024-02-17T12:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb779114640 1 -- 172.21.15.94:0/1377755963 learned_addr learned my addr 172.21.15.94:0/1377755963 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb773fff640 1 --2- 172.21.15.94:0/1377755963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646392e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb779114640 1 --2- 172.21.15.94:0/1377755963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064638d6b0 0x560646393370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb779114640 1 -- 172.21.15.94:0/1377755963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064638d6b0 msgr2=0x560646393370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb779114640 1 --2- 172.21.15.94:0/1377755963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064638d6b0 0x560646393370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb779114640 1 -- 172.21.15.94:0/1377755963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 msgr2=0x560646392e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb779114640 1 --2- 172.21.15.94:0/1377755963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646392e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb779114640 1 -- 172.21.15.94:0/1377755963 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb774069250 con 0x56064637e130 2024-02-17T12:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb779114640 1 --2- 172.21.15.94:0/1377755963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 0x5606463928f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb76c02c130 tx=0x7fb76c0093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3e1d8667107b66c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.579+0000 7fb753fff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.580+0000 7fb753fff640 1 -- 172.21.15.94:0/1377755963 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c003970 con 0x56064637e130 2024-02-17T12:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.580+0000 7fb753fff640 1 -- 172.21.15.94:0/1377755963 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb76c003ad0 con 0x56064637e130 2024-02-17T12:06:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.580+0000 7fb77d827740 1 -- 172.21.15.94:0/1377755963 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560645fe8600 con 0x56064637e130 2024-02-17T12:06:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.581+0000 7fb753fff640 1 -- 172.21.15.94:0/1377755963 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb76c003c90 con 0x56064637e130 2024-02-17T12:06:21.586 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.582+0000 7fb77d827740 10 client.27184 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.582+0000 7fb77d827740 1 -- 172.21.15.94:0/1377755963 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5606462e6d80 con 0x56064637e130 2024-02-17T12:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.583+0000 7fb753fff640 1 -- 172.21.15.94:0/1377755963 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560645fe8600 con 0x56064637e130 2024-02-17T12:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.583+0000 7fb753fff640 1 -- 172.21.15.94:0/1377755963 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb76c05b070 con 0x56064637e130 2024-02-17T12:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.583+0000 7fb77d827740 10 client.27184 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.583+0000 7fb77d827740 10 client.27184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.583+0000 7fb77d827740 20 client.27184 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.583+0000 7fb77d827740 1 --2- 172.21.15.94:0/1377755963 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5606463a2c90 0x5606463a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.583+0000 7fb77d827740 4 client.27184 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.583+0000 7fb77d827740 1 -- 172.21.15.94:0/1377755963 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606460391b0 con 0x5606463a2c90 2024-02-17T12:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.583+0000 7fb773fff640 1 --2- 172.21.15.94:0/1377755963 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5606463a2c90 0x5606463a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.584+0000 7fb753fff640 10 client.27184 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.584+0000 7fb773fff640 1 --2- 172.21.15.94:0/1377755963 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5606463a2c90 0x5606463a50b0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fb0532b8388014a server_cookie=3ab74e7f4628aee2 in_seq=0 out_seq=0 2024-02-17T12:06:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.584+0000 7fb753fff640 1 -- 172.21.15.94:0/1377755963 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5606460391b0 con 0x5606463a2c90 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.584+0000 7fb753fff640 10 client.27184 handle_command_reply: tid=0 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.585+0000 7fb77d827740 1 client.27184 shutdown 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.585+0000 7fb77d827740 1 -- 172.21.15.94:0/1377755963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 msgr2=0x5606463928f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.585+0000 7fb77d827740 1 --2- 172.21.15.94:0/1377755963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 0x5606463928f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb76c02c130 tx=0x7fb76c0093f0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.585+0000 7fb77d827740 1 -- 172.21.15.94:0/1377755963 shutdown_connections 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.585+0000 7fb77d827740 1 --2- 172.21.15.94:0/1377755963 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5606463a2c90 0x5606463a50b0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.585+0000 7fb77d827740 1 --2- 172.21.15.94:0/1377755963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064638d6b0 0x560646393370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.585+0000 7fb77d827740 1 --2- 172.21.15.94:0/1377755963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646392e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.585+0000 7fb77d827740 1 --2- 172.21.15.94:0/1377755963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 0x5606463928f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.585+0000 7fb753fff640 0 client.27184 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.585+0000 7fb77d827740 1 -- 172.21.15.94:0/1377755963 >> 172.21.15.94:0/1377755963 conn(0x560646348550 msgr2=0x56064637b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.585+0000 7fb77d827740 1 -- 172.21.15.94:0/1377755963 shutdown_connections 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.585+0000 7fb77d827740 1 -- 172.21.15.94:0/1377755963 wait complete. 2024-02-17T12:06:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.585+0000 7fb77d827740 20 client.27184 trim_cache size 0 max 16384 2024-02-17T12:06:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.628+0000 7fb77d827740 1 Processor -- start 2024-02-17T12:06:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.628+0000 7fb77d827740 1 -- start start 2024-02-17T12:06:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.628+0000 7fb77d827740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 0x560646395020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.628+0000 7fb77d827740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646399a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.628+0000 7fb779114640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 0x560646395020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.628+0000 7fb779114640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 0x560646395020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44178/0 (socket says 172.21.15.94:44178) 2024-02-17T12:06:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.628+0000 7fb77d827740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064638d6b0 0x560646399fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.628+0000 7fb77d827740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606463772e0 con 0x56064637eaf0 2024-02-17T12:06:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.628+0000 7fb77d827740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064639fb30 con 0x56064638d6b0 2024-02-17T12:06:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.628+0000 7fb77d827740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56064637c910 con 0x56064637e130 2024-02-17T12:06:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.628+0000 7fb773fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646399a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.628+0000 7fb779114640 1 -- 172.21.15.94:0/1611189915 learned_addr learned my addr 172.21.15.94:0/1611189915 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.629+0000 7fb779114640 1 --2- 172.21.15.94:0/1611189915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064638d6b0 0x560646399fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.629+0000 7fb779114640 1 -- 172.21.15.94:0/1611189915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064638d6b0 msgr2=0x560646399fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.629+0000 7fb779114640 1 --2- 172.21.15.94:0/1611189915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064638d6b0 0x560646399fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.629+0000 7fb779114640 1 -- 172.21.15.94:0/1611189915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 msgr2=0x560646399a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.629+0000 7fb779114640 1 --2- 172.21.15.94:0/1611189915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646399a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.629+0000 7fb779114640 1 -- 172.21.15.94:0/1611189915 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606462837b0 con 0x56064637e130 2024-02-17T12:06:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.629+0000 7fb773fff640 1 --2- 172.21.15.94:0/1611189915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646399a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.629+0000 7fb779114640 1 --2- 172.21.15.94:0/1611189915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 0x560646395020 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb76c047b20 tx=0x7fb76c034ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5835f4691babc26e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.629+0000 7fb75e7fc640 1 -- 172.21.15.94:0/1611189915 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c032d70 con 0x56064637e130 2024-02-17T12:06:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.630+0000 7fb75e7fc640 1 -- 172.21.15.94:0/1611189915 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb76c004b40 con 0x56064637e130 2024-02-17T12:06:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.630+0000 7fb75e7fc640 1 -- 172.21.15.94:0/1611189915 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76c047c40 con 0x56064637e130 2024-02-17T12:06:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.631+0000 7fb77d827740 1 -- 172.21.15.94:0/1611189915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 msgr2=0x560646395020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.631+0000 7fb77d827740 1 --2- 172.21.15.94:0/1611189915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 0x560646395020 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb76c047b20 tx=0x7fb76c034ca0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.631+0000 7fb77d827740 1 -- 172.21.15.94:0/1611189915 shutdown_connections 2024-02-17T12:06:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.631+0000 7fb77d827740 1 --2- 172.21.15.94:0/1611189915 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064638d6b0 0x560646399fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.631+0000 7fb77d827740 1 --2- 172.21.15.94:0/1611189915 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637eaf0 0x560646399a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.631+0000 7fb77d827740 1 --2- 172.21.15.94:0/1611189915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064637e130 0x560646395020 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.631+0000 7fb77d827740 1 -- 172.21.15.94:0/1611189915 >> 172.21.15.94:0/1611189915 conn(0x560646348550 msgr2=0x56064638c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.632+0000 7fb77d827740 1 -- 172.21.15.94:0/1611189915 shutdown_connections 2024-02-17T12:06:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.632+0000 7fb77d827740 1 -- 172.21.15.94:0/1611189915 wait complete. 2024-02-17T12:06:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.632+0000 7fb77d827740 1 -- start start 2024-02-17T12:06:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.632+0000 7fb77d827740 1 Processor -- start 2024-02-17T12:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.633+0000 7fb77d827740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637e130 0x560646399ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.633+0000 7fb77d827740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064637eaf0 0x560646398080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.633+0000 7fb77d827740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064638d6b0 0x5606463985c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb77d827740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560646398bd0 con 0x56064637e130 2024-02-17T12:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb779114640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637e130 0x560646399ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb779114640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637e130 0x560646399ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47650/0 (socket says 172.21.15.94:47650) 2024-02-17T12:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb779114640 1 -- 172.21.15.94:0/497200634 learned_addr learned my addr 172.21.15.94:0/497200634 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb77d827740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560646398d40 con 0x56064637eaf0 2024-02-17T12:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb773fff640 1 --2- 172.21.15.94:0/497200634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064637eaf0 0x560646398080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb77d827740 1 -- 172.21.15.94:0/497200634 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560646398f00 con 0x56064638d6b0 2024-02-17T12:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb77d827740 10 client.0 _ll_register_callbacks cb 0x7fb760005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb779114640 1 --2- 172.21.15.94:0/497200634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064638d6b0 0x5606463985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb773fff640 1 -- 172.21.15.94:0/497200634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064638d6b0 msgr2=0x5606463985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb773fff640 1 --2- 172.21.15.94:0/497200634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064638d6b0 0x5606463985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb773fff640 1 -- 172.21.15.94:0/497200634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637e130 msgr2=0x560646399ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb773fff640 1 --2- 172.21.15.94:0/497200634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637e130 0x560646399ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb773fff640 1 -- 172.21.15.94:0/497200634 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb760009e30 con 0x56064637eaf0 2024-02-17T12:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb773fff640 1 --2- 172.21.15.94:0/497200634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064637eaf0 0x560646398080 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5606463a5960 tx=0x7fb76001d750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=611cf3c5e31e338 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.634+0000 7fb753fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.635+0000 7fb753fff640 1 -- 172.21.15.94:0/497200634 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760004460 con 0x56064637eaf0 2024-02-17T12:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.635+0000 7fb753fff640 1 -- 172.21.15.94:0/497200634 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb760010960 con 0x56064637eaf0 2024-02-17T12:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.635+0000 7fb77d827740 1 -- 172.21.15.94:0/497200634 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb76c0345a0 con 0x56064637eaf0 2024-02-17T12:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.635+0000 7fb753fff640 1 -- 172.21.15.94:0/497200634 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb760010b20 con 0x56064637eaf0 2024-02-17T12:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.635+0000 7fb77d827740 10 client.27164 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.635+0000 7fb77d827740 1 -- 172.21.15.94:0/497200634 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5606462e6d80 con 0x56064637eaf0 2024-02-17T12:06:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.635+0000 7fb753fff640 1 -- 172.21.15.94:0/497200634 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb76c0345a0 con 0x56064637eaf0 2024-02-17T12:06:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.636+0000 7fb753fff640 1 -- 172.21.15.94:0/497200634 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb760010c80 con 0x56064637eaf0 2024-02-17T12:06:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.636+0000 7fb77d827740 10 client.27164 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.636+0000 7fb77d827740 10 client.27164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.636+0000 7fb77d827740 20 client.27164 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.636+0000 7fb77d827740 1 --2- 172.21.15.94:0/497200634 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5606463fa800 0x5606463fcbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.636+0000 7fb77d827740 4 client.27164 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:06:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.636+0000 7fb77d827740 1 -- 172.21.15.94:0/497200634 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560646025550 con 0x5606463fa800 2024-02-17T12:06:21.638 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.637+0000 7fb779114640 1 --2- 172.21.15.94:0/497200634 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5606463fa800 0x5606463fcbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.637+0000 7fb779114640 1 --2- 172.21.15.94:0/497200634 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5606463fa800 0x5606463fcbf0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e1c985bd63b7f22 server_cookie=e21468c84b885bd9 in_seq=0 out_seq=0 2024-02-17T12:06:21.639 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.637+0000 7fb753fff640 10 client.27164 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.659+0000 7fb753fff640 1 -- 172.21.15.94:0/497200634 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560646025550 con 0x5606463fa800 2024-02-17T12:06:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.659+0000 7fb753fff640 10 client.27164 handle_command_reply: tid=0 2024-02-17T12:06:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.659+0000 7fb77d827740 1 client.27164 shutdown 2024-02-17T12:06:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.659+0000 7fb77d827740 1 -- 172.21.15.94:0/497200634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064637eaf0 msgr2=0x560646398080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:21.660 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.659+0000 7fb77d827740 1 --2- 172.21.15.94:0/497200634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064637eaf0 0x560646398080 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x5606463a5960 tx=0x7fb76001d750 comp rx=0 tx=0).stop 2024-02-17T12:06:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.659+0000 7fb77d827740 1 -- 172.21.15.94:0/497200634 shutdown_connections 2024-02-17T12:06:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.659+0000 7fb77d827740 1 --2- 172.21.15.94:0/497200634 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5606463fa800 0x5606463fcbf0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.659+0000 7fb77d827740 1 --2- 172.21.15.94:0/497200634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56064638d6b0 0x5606463985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.659+0000 7fb77d827740 1 --2- 172.21.15.94:0/497200634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56064637eaf0 0x560646398080 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.660+0000 7fb77d827740 1 --2- 172.21.15.94:0/497200634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56064637e130 0x560646399ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.660+0000 7fb77d827740 1 -- 172.21.15.94:0/497200634 >> 172.21.15.94:0/497200634 conn(0x560646348550 msgr2=0x56064637a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.660+0000 7fb753fff640 0 client.27164 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:21.661 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.660+0000 7fb77d827740 1 -- 172.21.15.94:0/497200634 shutdown_connections 2024-02-17T12:06:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.660+0000 7fb77d827740 1 -- 172.21.15.94:0/497200634 wait complete. 2024-02-17T12:06:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.660+0000 7fb77d827740 20 client.27164 trim_cache size 0 max 16384 2024-02-17T12:06:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:06:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:06:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:06:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:06:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.661+0000 7fb779915640 1 -- 172.21.15.94:0/998983533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb7640a2320 msgr2=0x7fb7640a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.661+0000 7fb779915640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb7640a2320 0x7fb7640a47e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x56064637de00 tx=0x7fb76801b4f0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.661+0000 7fb779915640 1 -- 172.21.15.94:0/998983533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb774100100 msgr2=0x7fb774197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.661+0000 7fb779915640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb774100100 0x7fb774197410 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fb76800e9f0 tx=0x7fb76800edf0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.661+0000 7fb779915640 1 -- 172.21.15.94:0/998983533 shutdown_connections 2024-02-17T12:06:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.661+0000 7fb779915640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb7640a2320 0x7fb7640a47e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.661+0000 7fb779915640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb77419bcd0 0x7fb77419c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.661+0000 7fb779915640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7741051d0 0x7fb774197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.661+0000 7fb779915640 1 --2- 172.21.15.94:0/998983533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb774100100 0x7fb774197410 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.661+0000 7fb779915640 1 -- 172.21.15.94:0/998983533 >> 172.21.15.94:0/998983533 conn(0x7fb7740fb980 msgr2=0x7fb7740fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.661+0000 7fb779915640 1 -- 172.21.15.94:0/998983533 shutdown_connections 2024-02-17T12:06:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:21.661+0000 7fb779915640 1 -- 172.21.15.94:0/998983533 wait complete. 2024-02-17T12:06:22.303 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:21 smithi189 ceph-mon[36588]: pgmap v254: 97 pgs: 97 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 36 MiB/s wr, 1.76k op/s 2024-02-17T12:06:22.303 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:21 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:06:22.303 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:21 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:06:22.303 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:21 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:06:22.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:21 smithi125 ceph-mon[36819]: pgmap v254: 97 pgs: 97 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 36 MiB/s wr, 1.76k op/s 2024-02-17T12:06:22.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:21 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:06:22.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:21 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:06:22.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:21 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:06:22.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:21 smithi094 ceph-mon[32499]: pgmap v254: 97 pgs: 97 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 36 MiB/s wr, 1.76k op/s 2024-02-17T12:06:22.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:21 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:06:22.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:21 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:06:22.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:21 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:06:23.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:23 smithi189 ceph-mon[36588]: pgmap v255: 97 pgs: 97 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 33 MiB/s wr, 1.64k op/s 2024-02-17T12:06:23.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:23 smithi125 ceph-mon[36819]: pgmap v255: 97 pgs: 97 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 33 MiB/s wr, 1.64k op/s 2024-02-17T12:06:23.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:23 smithi094 ceph-mon[32499]: pgmap v255: 97 pgs: 97 active+clean; 5.1 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 33 MiB/s wr, 1.64k op/s 2024-02-17T12:06:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 485.0 s, 1860.6 tps, lat 53.360 ms stddev 56.771 2024-02-17T12:06:26.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:25 smithi189 ceph-mon[36588]: pgmap v256: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.20k op/s 2024-02-17T12:06:26.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:25 smithi125 ceph-mon[36819]: pgmap v256: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.20k op/s 2024-02-17T12:06:26.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:25 smithi094 ceph-mon[32499]: pgmap v256: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 22 MiB/s wr, 1.20k op/s 2024-02-17T12:06:28.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:27 smithi189 ceph-mon[36588]: pgmap v257: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-02-17T12:06:28.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:27 smithi125 ceph-mon[36819]: pgmap v257: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-02-17T12:06:28.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:27 smithi094 ceph-mon[32499]: pgmap v257: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 19 MiB/s wr, 1.03k op/s 2024-02-17T12:06:29.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:29 smithi189 ceph-mon[36588]: pgmap v258: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-02-17T12:06:29.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:29 smithi125 ceph-mon[36819]: pgmap v258: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-02-17T12:06:29.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:29 smithi094 ceph-mon[32499]: pgmap v258: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-02-17T12:06:30.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 490.0 s, 1654.0 tps, lat 60.702 ms stddev 68.271 2024-02-17T12:06:32.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:31 smithi189 ceph-mon[36588]: pgmap v259: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.9 MiB/s wr, 620 op/s 2024-02-17T12:06:32.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:31 smithi125 ceph-mon[36819]: pgmap v259: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.9 MiB/s wr, 620 op/s 2024-02-17T12:06:32.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:31 smithi094 ceph-mon[32499]: pgmap v259: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 5.9 MiB/s wr, 620 op/s 2024-02-17T12:06:33.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:32 smithi189 ceph-mon[36588]: pgmap v260: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.3 MiB/s wr, 656 op/s 2024-02-17T12:06:33.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:32 smithi125 ceph-mon[36819]: pgmap v260: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.3 MiB/s wr, 656 op/s 2024-02-17T12:06:33.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:32 smithi094 ceph-mon[32499]: pgmap v260: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 6.3 MiB/s wr, 656 op/s 2024-02-17T12:06:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.802+0000 7f09a9cdf640 1 -- 172.21.15.94:0/3164968568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09a4100c90 msgr2=0x7f09a4068800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.802+0000 7f09a9cdf640 1 --2- 172.21.15.94:0/3164968568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09a4100c90 0x7f09a4068800 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f099c009ca0 tx=0x7f099c02c120 comp rx=0 tx=0).stop 2024-02-17T12:06:34.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.803+0000 7f09a9cdf640 1 -- 172.21.15.94:0/3164968568 shutdown_connections 2024-02-17T12:06:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.803+0000 7f09a9cdf640 1 --2- 172.21.15.94:0/3164968568 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09a4100c90 0x7f09a4068800 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.803+0000 7f09a9cdf640 1 --2- 172.21.15.94:0/3164968568 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09a41002d0 0x7f09a4100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.803+0000 7f09a9cdf640 1 --2- 172.21.15.94:0/3164968568 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09a4105020 0x7f09a4105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.803+0000 7f09a9cdf640 1 -- 172.21.15.94:0/3164968568 >> 172.21.15.94:0/3164968568 conn(0x7f09a4076ed0 msgr2=0x7f09a40772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.803+0000 7f09a9cdf640 1 -- 172.21.15.94:0/3164968568 shutdown_connections 2024-02-17T12:06:34.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.803+0000 7f09a9cdf640 1 -- 172.21.15.94:0/3164968568 wait complete. 2024-02-17T12:06:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.804+0000 7f09a9cdf640 1 Processor -- start 2024-02-17T12:06:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.804+0000 7f09a9cdf640 1 -- start start 2024-02-17T12:06:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.805+0000 7f09a9cdf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09a41002d0 0x7f09a4070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.805+0000 7f09a9cdf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09a4100c90 0x7f09a40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.805+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09a41002d0 0x7f09a4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.805+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09a41002d0 0x7f09a4070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53640/0 (socket says 172.21.15.94:53640) 2024-02-17T12:06:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.805+0000 7f09a37fe640 1 -- 172.21.15.94:0/486862677 learned_addr learned my addr 172.21.15.94:0/486862677 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.805+0000 7f09a9cdf640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09a4105020 0x7f09a4070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.805+0000 7f09a9cdf640 1 -- 172.21.15.94:0/486862677 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a4112270 con 0x7f09a41002d0 2024-02-17T12:06:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.805+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09a4100c90 0x7f09a40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.805+0000 7f09a9cdf640 1 -- 172.21.15.94:0/486862677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41123e0 con 0x7f09a4105020 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.805+0000 7f09a9cdf640 1 -- 172.21.15.94:0/486862677 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a41125a0 con 0x7f09a4100c90 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.805+0000 7f09a3fff640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09a4105020 0x7f09a4070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.806+0000 7f09a3fff640 1 -- 172.21.15.94:0/486862677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09a4100c90 msgr2=0x7f09a40707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.806+0000 7f09a3fff640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09a4100c90 0x7f09a40707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.806+0000 7f09a3fff640 1 -- 172.21.15.94:0/486862677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09a41002d0 msgr2=0x7f09a4070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.806+0000 7f09a3fff640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09a41002d0 0x7f09a4070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.806+0000 7f09a3fff640 1 -- 172.21.15.94:0/486862677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c009900 con 0x7f09a4105020 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.806+0000 7f09a3fff640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09a4105020 0x7f09a4070ce0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f099c009bf0 tx=0x7f099c0347d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.806+0000 7f09a0ff9640 1 -- 172.21.15.94:0/486862677 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c034440 con 0x7f09a4105020 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.807+0000 7f09a0ff9640 1 -- 172.21.15.94:0/486862677 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f099c003970 con 0x7f09a4105020 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.807+0000 7f09a0ff9640 1 -- 172.21.15.94:0/486862677 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c004610 con 0x7f09a4105020 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.807+0000 7f09a9cdf640 1 -- 172.21.15.94:0/486862677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a4112820 con 0x7f09a4105020 2024-02-17T12:06:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.807+0000 7f09a9cdf640 1 -- 172.21.15.94:0/486862677 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a4112c50 con 0x7f09a4105020 2024-02-17T12:06:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.811+0000 7f09ac992740 1 Processor -- start 2024-02-17T12:06:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.812+0000 7f09a0ff9640 1 -- 172.21.15.94:0/486862677 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f099c03b040 con 0x7f09a4105020 2024-02-17T12:06:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.812+0000 7f09ac992740 1 -- start start 2024-02-17T12:06:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09ac992740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 0x563714225820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09ac992740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x5637142261e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09ac992740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 0x5637142316d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09ac992740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637141949f0 con 0x563714226ad0 2024-02-17T12:06:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09ac992740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563714191f20 con 0x56371422f2e0 2024-02-17T12:06:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09ac992740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371418d700 con 0x563714225d60 2024-02-17T12:06:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 0x563714225820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a0ff9640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f098c0a22d0 0x7f098c0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 0x563714225820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53644/0 (socket says 172.21.15.94:53644) 2024-02-17T12:06:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a2ffd640 1 -- 172.21.15.94:0/1124198057 learned_addr learned my addr 172.21.15.94:0/1124198057 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/1124198057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 0x5637142316d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a37fe640 1 --2- 172.21.15.94:0/1124198057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x5637142261e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a3fff640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f098c0a22d0 0x7f098c0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a0ff9640 1 -- 172.21.15.94:0/486862677 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f099c0f0630 con 0x7f09a4105020 2024-02-17T12:06:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a2ffd640 1 -- 172.21.15.94:0/1124198057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 msgr2=0x5637142261e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/1124198057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x5637142261e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a2ffd640 1 -- 172.21.15.94:0/1124198057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 msgr2=0x5637142316d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/1124198057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 0x5637142316d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a2ffd640 1 -- 172.21.15.94:0/1124198057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637141237b0 con 0x563714226ad0 2024-02-17T12:06:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a37fe640 1 --2- 172.21.15.94:0/1124198057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x5637142261e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:06:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.813+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/1124198057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 0x563714225820 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0998012790 tx=0x7f099800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6949c4beaf020ecf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.814+0000 7f0989ffb640 1 -- 172.21.15.94:0/1124198057 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0998012dd0 con 0x563714226ad0 2024-02-17T12:06:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.814+0000 7f0989ffb640 1 -- 172.21.15.94:0/1124198057 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0998012080 con 0x563714226ad0 2024-02-17T12:06:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.814+0000 7f0989ffb640 1 -- 172.21.15.94:0/1124198057 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099800ed30 con 0x563714226ad0 2024-02-17T12:06:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.815+0000 7f09ac992740 1 -- 172.21.15.94:0/1124198057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 msgr2=0x563714225820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.815+0000 7f09a3fff640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f098c0a22d0 0x7f098c0a4790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f099c0b9ac0 tx=0x7f099c035f70 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.815+0000 7f09ac992740 1 --2- 172.21.15.94:0/1124198057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 0x563714225820 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0998012790 tx=0x7f099800b390 comp rx=0 tx=0).stop 2024-02-17T12:06:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.815+0000 7f09ac992740 1 -- 172.21.15.94:0/1124198057 shutdown_connections 2024-02-17T12:06:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.815+0000 7f09ac992740 1 --2- 172.21.15.94:0/1124198057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 0x5637142316d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.815+0000 7f09ac992740 1 --2- 172.21.15.94:0/1124198057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x5637142261e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.815+0000 7f09ac992740 1 --2- 172.21.15.94:0/1124198057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 0x563714225820 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.815+0000 7f09ac992740 1 -- 172.21.15.94:0/1124198057 >> 172.21.15.94:0/1124198057 conn(0x5637141ed550 msgr2=0x563714220a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.815+0000 7f09ac992740 1 -- 172.21.15.94:0/1124198057 shutdown_connections 2024-02-17T12:06:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.815+0000 7f09ac992740 1 -- 172.21.15.94:0/1124198057 wait complete. 2024-02-17T12:06:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.816+0000 7f09ac992740 1 -- start start 2024-02-17T12:06:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.816+0000 7f09ac992740 1 Processor -- start 2024-02-17T12:06:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.816+0000 7f09ac992740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x56371422c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09ac992740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563714226ad0 0x56371422c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09ac992740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56371422f2e0 0x56371422cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09a2ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x56371422c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09a2ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x56371422c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:48314/0 (socket says 172.21.15.94:48314) 2024-02-17T12:06:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09a2ffd640 1 -- 172.21.15.94:0/2290256688 learned_addr learned my addr 172.21.15.94:0/2290256688 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09a37fe640 1 --2- 172.21.15.94:0/2290256688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563714226ad0 0x56371422c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.821 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09ac992740 1 -- 172.21.15.94:0/2290256688 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563714132bf0 con 0x56371422f2e0 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09ac992740 1 -- 172.21.15.94:0/2290256688 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563713e61250 con 0x563714226ad0 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09ac992740 1 -- 172.21.15.94:0/2290256688 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563714113200 con 0x563714225d60 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09a2ffd640 1 -- 172.21.15.94:0/2290256688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563714226ad0 msgr2=0x56371422c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/2290256688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563714226ad0 0x56371422c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09a2ffd640 1 -- 172.21.15.94:0/2290256688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56371422f2e0 msgr2=0x56371422cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/2290256688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56371422f2e0 0x56371422cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.817+0000 7f09a2ffd640 1 -- 172.21.15.94:0/2290256688 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099801a040 con 0x563714225d60 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.818+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/2290256688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x56371422c2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f099800ba80 tx=0x7f099800e900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c1d73abd2cd3f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.818+0000 7f0982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.818+0000 7f09ac992740 10 client.0 _ll_register_callbacks cb 0x563714181d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.818+0000 7f0982ffd640 1 -- 172.21.15.94:0/2290256688 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099800ee70 con 0x563714225d60 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.818+0000 7f0982ffd640 1 -- 172.21.15.94:0/2290256688 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0998005040 con 0x563714225d60 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.818+0000 7f09ac992740 1 -- 172.21.15.94:0/2290256688 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563713e8d600 con 0x563714225d60 2024-02-17T12:06:34.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.818+0000 7f0982ffd640 1 -- 172.21.15.94:0/2290256688 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0998005200 con 0x563714225d60 2024-02-17T12:06:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.818+0000 7f09ac992740 10 client.27199 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.818+0000 7f09ac992740 1 -- 172.21.15.94:0/2290256688 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x56371418bd80 con 0x563714225d60 2024-02-17T12:06:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.819+0000 7f0982ffd640 1 -- 172.21.15.94:0/2290256688 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0998005360 con 0x563714225d60 2024-02-17T12:06:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.819+0000 7f09ac992740 10 client.27199 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.819+0000 7f09ac992740 10 client.27199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.819+0000 7f09ac992740 20 client.27199 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.819+0000 7f09ac992740 1 --2- 172.21.15.94:0/2290256688 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56371423f200 0x563714241660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.819+0000 7f09ac992740 4 client.27199 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.819+0000 7f09ac992740 1 -- 172.21.15.94:0/2290256688 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563713e8d600 con 0x56371423f200 2024-02-17T12:06:34.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.819+0000 7f09a37fe640 1 --2- 172.21.15.94:0/2290256688 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56371423f200 0x563714241660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.820+0000 7f09a37fe640 1 --2- 172.21.15.94:0/2290256688 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56371423f200 0x563714241660 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47a5972af23a0385 server_cookie=3f712143baba2e08 in_seq=0 out_seq=0 2024-02-17T12:06:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.820+0000 7f0982ffd640 10 client.27199 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.820+0000 7f0982ffd640 1 -- 172.21.15.94:0/2290256688 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563713e8d600 con 0x56371423f200 2024-02-17T12:06:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.820+0000 7f0982ffd640 10 client.27199 handle_command_reply: tid=0 2024-02-17T12:06:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.820+0000 7f09ac992740 1 client.27199 shutdown 2024-02-17T12:06:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.820+0000 7f09ac992740 1 -- 172.21.15.94:0/2290256688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 msgr2=0x56371422c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:34.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.820+0000 7f09ac992740 1 --2- 172.21.15.94:0/2290256688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x56371422c2f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f099800ba80 tx=0x7f099800e900 comp rx=0 tx=0).stop 2024-02-17T12:06:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.821+0000 7f09ac992740 1 -- 172.21.15.94:0/2290256688 shutdown_connections 2024-02-17T12:06:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.821+0000 7f09ac992740 1 --2- 172.21.15.94:0/2290256688 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56371423f200 0x563714241660 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.821+0000 7f09ac992740 1 --2- 172.21.15.94:0/2290256688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56371422f2e0 0x56371422cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.821+0000 7f09ac992740 1 --2- 172.21.15.94:0/2290256688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563714226ad0 0x56371422c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.821+0000 7f09ac992740 1 --2- 172.21.15.94:0/2290256688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x56371422c2f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.821+0000 7f09ac992740 1 -- 172.21.15.94:0/2290256688 >> 172.21.15.94:0/2290256688 conn(0x5637141ed550 msgr2=0x5637142227a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.821+0000 7f0982ffd640 0 client.27199 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.821+0000 7f09ac992740 1 -- 172.21.15.94:0/2290256688 shutdown_connections 2024-02-17T12:06:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.822+0000 7f09ac992740 1 -- 172.21.15.94:0/2290256688 wait complete. 2024-02-17T12:06:34.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.822+0000 7f09ac992740 20 client.27199 trim_cache size 0 max 16384 2024-02-17T12:06:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.840+0000 7f09ac992740 1 Processor -- start 2024-02-17T12:06:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09ac992740 1 -- start start 2024-02-17T12:06:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09ac992740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x56371422ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09ac992740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 0x56371422abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09ac992740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 0x56371422b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09ac992740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371423c110 con 0x563714226ad0 2024-02-17T12:06:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09ac992740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371422c0e0 con 0x56371422f2e0 2024-02-17T12:06:34.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09ac992740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56371422b670 con 0x563714225d60 2024-02-17T12:06:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 0x56371422abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 0x56371422abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53662/0 (socket says 172.21.15.94:53662) 2024-02-17T12:06:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09a37fe640 1 -- 172.21.15.94:0/2606939613 learned_addr learned my addr 172.21.15.94:0/2606939613 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/2606939613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x56371422ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/2606939613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 0x56371422b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09a37fe640 1 -- 172.21.15.94:0/2606939613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 msgr2=0x56371422ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09a37fe640 1 --2- 172.21.15.94:0/2606939613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x56371422ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09a37fe640 1 -- 172.21.15.94:0/2606939613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 msgr2=0x56371422b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09a37fe640 1 --2- 172.21.15.94:0/2606939613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 0x56371422b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09a37fe640 1 -- 172.21.15.94:0/2606939613 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637141287b0 con 0x563714226ad0 2024-02-17T12:06:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.841+0000 7f09a37fe640 1 --2- 172.21.15.94:0/2606939613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 0x56371422abc0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0994009fd0 tx=0x7f0994008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d87c729a42d52b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.842+0000 7f09897fa640 1 -- 172.21.15.94:0/2606939613 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994016d20 con 0x563714226ad0 2024-02-17T12:06:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.842+0000 7f09897fa640 1 -- 172.21.15.94:0/2606939613 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f099401c040 con 0x563714226ad0 2024-02-17T12:06:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.842+0000 7f09897fa640 1 -- 172.21.15.94:0/2606939613 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09940155c0 con 0x563714226ad0 2024-02-17T12:06:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.843+0000 7f09ac992740 1 -- 172.21.15.94:0/2606939613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 msgr2=0x56371422abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.843+0000 7f09ac992740 1 --2- 172.21.15.94:0/2606939613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 0x56371422abc0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0994009fd0 tx=0x7f0994008040 comp rx=0 tx=0).stop 2024-02-17T12:06:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.843+0000 7f09ac992740 1 -- 172.21.15.94:0/2606939613 shutdown_connections 2024-02-17T12:06:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.843+0000 7f09ac992740 1 --2- 172.21.15.94:0/2606939613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 0x56371422b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.844+0000 7f09ac992740 1 --2- 172.21.15.94:0/2606939613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714226ad0 0x56371422abc0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.844+0000 7f09ac992740 1 --2- 172.21.15.94:0/2606939613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714225d60 0x56371422ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.844+0000 7f09ac992740 1 -- 172.21.15.94:0/2606939613 >> 172.21.15.94:0/2606939613 conn(0x5637141ed550 msgr2=0x563714227600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.844+0000 7f09ac992740 1 -- 172.21.15.94:0/2606939613 shutdown_connections 2024-02-17T12:06:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.845+0000 7f09ac992740 1 -- 172.21.15.94:0/2606939613 wait complete. 2024-02-17T12:06:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.845+0000 7f09ac992740 1 -- start start 2024-02-17T12:06:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.845+0000 7f09ac992740 1 Processor -- start 2024-02-17T12:06:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.845+0000 7f09ac992740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714225d60 0x5637142360e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.845+0000 7f09ac992740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714226ad0 0x563714236620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.845+0000 7f09ac992740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 0x563714234680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.845+0000 7f09ac992740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563714234d50 con 0x563714225d60 2024-02-17T12:06:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.845+0000 7f09ac992740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563714234ec0 con 0x56371422f2e0 2024-02-17T12:06:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.845+0000 7f09ac992740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563714235080 con 0x563714226ad0 2024-02-17T12:06:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.845+0000 7f09ac992740 10 client.0 _ll_register_callbacks cb 0x7f099800bad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.846+0000 7f09a2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714225d60 0x5637142360e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.846+0000 7f09a2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714225d60 0x5637142360e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53672/0 (socket says 172.21.15.94:53672) 2024-02-17T12:06:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.846+0000 7f09a2ffd640 1 -- 172.21.15.94:0/2435622437 learned_addr learned my addr 172.21.15.94:0/2435622437 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.846+0000 7f09a37fe640 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714226ad0 0x563714236620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.846+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 0x563714234680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.846+0000 7f09a2ffd640 1 -- 172.21.15.94:0/2435622437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714226ad0 msgr2=0x563714236620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.846+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714226ad0 0x563714236620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.846+0000 7f09a2ffd640 1 -- 172.21.15.94:0/2435622437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 msgr2=0x563714234680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.846+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 0x563714234680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.846+0000 7f09a2ffd640 1 -- 172.21.15.94:0/2435622437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0994009cb0 con 0x563714225d60 2024-02-17T12:06:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.846+0000 7f09a37fe640 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714226ad0 0x563714236620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.846+0000 7f09a2ffd640 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714225d60 0x5637142360e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f099802b040 tx=0x7f0998023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a9f6ff328b39c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.846+0000 7f0982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.847+0000 7f0982ffd640 1 -- 172.21.15.94:0/2435622437 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099801f780 con 0x563714225d60 2024-02-17T12:06:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.847+0000 7f0982ffd640 1 -- 172.21.15.94:0/2435622437 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f099801f8e0 con 0x563714225d60 2024-02-17T12:06:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.847+0000 7f09ac992740 1 -- 172.21.15.94:0/2435622437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0998012080 con 0x563714225d60 2024-02-17T12:06:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.847+0000 7f0982ffd640 1 -- 172.21.15.94:0/2435622437 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f099801faa0 con 0x563714225d60 2024-02-17T12:06:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.847+0000 7f09ac992740 10 client.17712 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.848+0000 7f09ac992740 1 -- 172.21.15.94:0/2435622437 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56371414e540 con 0x563714225d60 2024-02-17T12:06:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.848+0000 7f0982ffd640 1 -- 172.21.15.94:0/2435622437 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0998012080 con 0x563714225d60 2024-02-17T12:06:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.848+0000 7f0982ffd640 1 -- 172.21.15.94:0/2435622437 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f099804bb90 con 0x563714225d60 2024-02-17T12:06:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.848+0000 7f09ac992740 10 client.17712 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.848+0000 7f09ac992740 10 client.17712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.848+0000 7f09ac992740 20 client.17712 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.848+0000 7f09ac992740 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563714296630 0x563714298a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.848+0000 7f09ac992740 4 client.17712 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:06:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.848+0000 7f09ac992740 1 -- 172.21.15.94:0/2435622437 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563713ede1b0 con 0x563714296630 2024-02-17T12:06:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.848+0000 7f09a37fe640 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563714296630 0x563714298a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.849+0000 7f09a37fe640 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563714296630 0x563714298a20 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=526caa116e7178b4 server_cookie=218271d96ff79965 in_seq=0 out_seq=0 2024-02-17T12:06:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.849+0000 7f0982ffd640 10 client.17712 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.868+0000 7f0982ffd640 1 -- 172.21.15.94:0/2435622437 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563713ede1b0 con 0x563714296630 2024-02-17T12:06:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.868+0000 7f0982ffd640 10 client.17712 handle_command_reply: tid=0 2024-02-17T12:06:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.868+0000 7f09ac992740 1 client.17712 shutdown 2024-02-17T12:06:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09ac992740 1 -- 172.21.15.94:0/2435622437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714225d60 msgr2=0x5637142360e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09ac992740 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714225d60 0x5637142360e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f099802b040 tx=0x7f0998023000 comp rx=0 tx=0).stop 2024-02-17T12:06:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09ac992740 1 -- 172.21.15.94:0/2435622437 shutdown_connections 2024-02-17T12:06:34.870 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09ac992740 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563714296630 0x563714298a20 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f0982ffd640 0 client.17712 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09ac992740 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56371422f2e0 0x563714234680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09ac992740 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563714226ad0 0x563714236620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09ac992740 1 --2- 172.21.15.94:0/2435622437 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563714225d60 0x5637142360e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09ac992740 1 -- 172.21.15.94:0/2435622437 >> 172.21.15.94:0/2435622437 conn(0x5637141ed550 msgr2=0x56371422aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09ac992740 1 -- 172.21.15.94:0/2435622437 shutdown_connections 2024-02-17T12:06:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09ac992740 1 -- 172.21.15.94:0/2435622437 wait complete. 2024-02-17T12:06:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09ac992740 20 client.17712 trim_cache size 0 max 16384 2024-02-17T12:06:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:06:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:06:34.871 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:06:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:06:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09a9cdf640 1 -- 172.21.15.94:0/486862677 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f098c0a22d0 msgr2=0x7f098c0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09a9cdf640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f098c0a22d0 0x7f098c0a4790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f099c0b9ac0 tx=0x7f099c035f70 comp rx=0 tx=0).stop 2024-02-17T12:06:34.872 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09a9cdf640 1 -- 172.21.15.94:0/486862677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09a4105020 msgr2=0x7f09a4070ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09a9cdf640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09a4105020 0x7f09a4070ce0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f099c009bf0 tx=0x7f099c0347d0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09a9cdf640 1 -- 172.21.15.94:0/486862677 shutdown_connections 2024-02-17T12:06:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09a9cdf640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f098c0a22d0 0x7f098c0a4790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.869+0000 7f09a9cdf640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09a4105020 0x7f09a4070ce0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.870+0000 7f09a9cdf640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09a4100c90 0x7f09a40707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.870+0000 7f09a9cdf640 1 --2- 172.21.15.94:0/486862677 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09a41002d0 0x7f09a4070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.870+0000 7f09a9cdf640 1 -- 172.21.15.94:0/486862677 >> 172.21.15.94:0/486862677 conn(0x7f09a4076ed0 msgr2=0x7f09a40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.870+0000 7f09a9cdf640 1 -- 172.21.15.94:0/486862677 shutdown_connections 2024-02-17T12:06:34.873 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:34.870+0000 7f09a9cdf640 1 -- 172.21.15.94:0/486862677 wait complete. 2024-02-17T12:06:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 495.0 s, 2334.4 tps, lat 42.952 ms stddev 47.936 2024-02-17T12:06:36.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:35 smithi189 ceph-mon[36588]: pgmap v261: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.3 MiB/s wr, 872 op/s 2024-02-17T12:06:36.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:35 smithi125 ceph-mon[36819]: pgmap v261: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.3 MiB/s wr, 872 op/s 2024-02-17T12:06:36.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:35 smithi094 ceph-mon[32499]: pgmap v261: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 8.3 MiB/s wr, 872 op/s 2024-02-17T12:06:38.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:37 smithi189 ceph-mon[36588]: pgmap v262: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-02-17T12:06:38.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:37 smithi125 ceph-mon[36819]: pgmap v262: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-02-17T12:06:38.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:37 smithi094 ceph-mon[32499]: pgmap v262: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 5.7 MiB/s wr, 602 op/s 2024-02-17T12:06:39.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:39 smithi189 ceph-mon[36588]: pgmap v263: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.3 MiB/s wr, 881 op/s 2024-02-17T12:06:39.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:39 smithi125 ceph-mon[36819]: pgmap v263: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.3 MiB/s wr, 881 op/s 2024-02-17T12:06:39.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:39 smithi094 ceph-mon[32499]: pgmap v263: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 8.3 MiB/s wr, 881 op/s 2024-02-17T12:06:40.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 500.0 s, 2222.6 tps, lat 44.999 ms stddev 49.708 2024-02-17T12:06:41.180 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:06:41.181 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:06:41.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.319+0000 7f960700e640 1 -- 172.21.15.94:0/88594389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9600106500 msgr2=0x7f960010e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.319+0000 7f960700e640 1 --2- 172.21.15.94:0/88594389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9600106500 0x7f960010e060 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009ca0 tx=0x7f95f802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.319+0000 7f960700e640 1 -- 172.21.15.94:0/88594389 shutdown_connections 2024-02-17T12:06:41.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.319+0000 7f960700e640 1 --2- 172.21.15.94:0/88594389 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9600106500 0x7f960010e060 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.319+0000 7f960700e640 1 --2- 172.21.15.94:0/88594389 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96001002a0 0x7f9600105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.319+0000 7f960700e640 1 --2- 172.21.15.94:0/88594389 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96000ff8f0 0x7f96000ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.319+0000 7f960700e640 1 -- 172.21.15.94:0/88594389 >> 172.21.15.94:0/88594389 conn(0x7f96000765b0 msgr2=0x7f96000fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:41.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.319+0000 7f960700e640 1 -- 172.21.15.94:0/88594389 shutdown_connections 2024-02-17T12:06:41.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.319+0000 7f960700e640 1 -- 172.21.15.94:0/88594389 wait complete. 2024-02-17T12:06:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f960700e640 1 Processor -- start 2024-02-17T12:06:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f960700e640 1 -- start start 2024-02-17T12:06:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f960700e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96000ff8f0 0x7f96001975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f960700e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9600106500 0x7f9600197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f960700e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f960019be80 0x7f960019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f960700e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960019c870 con 0x7f9600106500 2024-02-17T12:06:41.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f960700e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960019c9e0 con 0x7f96000ff8f0 2024-02-17T12:06:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f960700e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960019cb50 con 0x7f960019be80 2024-02-17T12:06:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f95fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9600106500 0x7f9600197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f95fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9600106500 0x7f9600197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53688/0 (socket says 172.21.15.94:53688) 2024-02-17T12:06:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f95fffff640 1 -- 172.21.15.94:0/899812296 learned_addr learned my addr 172.21.15.94:0/899812296 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f9604daa640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96000ff8f0 0x7f96001975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f96055ab640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f960019be80 0x7f960019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f95fffff640 1 -- 172.21.15.94:0/899812296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f960019be80 msgr2=0x7f960019c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f95fffff640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f960019be80 0x7f960019c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f95fffff640 1 -- 172.21.15.94:0/899812296 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96000ff8f0 msgr2=0x7f96001975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f95fffff640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96000ff8f0 0x7f96001975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f95fffff640 1 -- 172.21.15.94:0/899812296 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f8009900 con 0x7f9600106500 2024-02-17T12:06:41.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f9604daa640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96000ff8f0 0x7f96001975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:41.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.320+0000 7f95fffff640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9600106500 0x7f9600197b00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00d740 tx=0x7f95ec00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:41.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.321+0000 7f95fdffb640 1 -- 172.21.15.94:0/899812296 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec009c30 con 0x7f9600106500 2024-02-17T12:06:41.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.321+0000 7f960700e640 1 -- 172.21.15.94:0/899812296 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96001a4130 con 0x7f9600106500 2024-02-17T12:06:41.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.321+0000 7f95fdffb640 1 -- 172.21.15.94:0/899812296 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95ec00de50 con 0x7f9600106500 2024-02-17T12:06:41.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.321+0000 7f95fdffb640 1 -- 172.21.15.94:0/899812296 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec010460 con 0x7f9600106500 2024-02-17T12:06:41.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.321+0000 7f960700e640 1 -- 172.21.15.94:0/899812296 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96001a4590 con 0x7f9600106500 2024-02-17T12:06:41.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.324+0000 7f9609be5740 1 Processor -- start 2024-02-17T12:06:41.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.324+0000 7f95fdffb640 1 -- 172.21.15.94:0/899812296 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f95ec004f10 con 0x7f9600106500 2024-02-17T12:06:41.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.324+0000 7f9609be5740 1 -- start start 2024-02-17T12:06:41.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.324+0000 7f9609be5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f462b0 0x5565e3f46690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.324+0000 7f9609be5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f46bd0 0x5565e3f4f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.324+0000 7f9609be5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 0x5565e3f51ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.324+0000 7f9609be5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e3eb5280 con 0x5565e3f46bd0 2024-02-17T12:06:41.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.324+0000 7f9609be5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e3eb27b0 con 0x5565e3f462b0 2024-02-17T12:06:41.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.324+0000 7f9609be5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e3eadf90 con 0x5565e3f4f7b0 2024-02-17T12:06:41.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f96055ab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f462b0 0x5565e3f46690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f9604daa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f46bd0 0x5565e3f4f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f96055ab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f462b0 0x5565e3f46690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52440/0 (socket says 172.21.15.94:52440) 2024-02-17T12:06:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f96055ab640 1 -- 172.21.15.94:0/2856656511 learned_addr learned my addr 172.21.15.94:0/2856656511 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f95fffff640 1 --2- 172.21.15.94:0/2856656511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 0x5565e3f51ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f95fdffb640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f95f00a2230 0x7f95f00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f96055ab640 1 -- 172.21.15.94:0/2856656511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 msgr2=0x5565e3f51ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f96055ab640 1 --2- 172.21.15.94:0/2856656511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 0x5565e3f51ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f96055ab640 1 -- 172.21.15.94:0/2856656511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f46bd0 msgr2=0x5565e3f4f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f96055ab640 1 --2- 172.21.15.94:0/2856656511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f46bd0 0x5565e3f4f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f96055ab640 1 -- 172.21.15.94:0/2856656511 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565e3e6c570 con 0x5565e3f462b0 2024-02-17T12:06:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f9604daa640 1 --2- 172.21.15.94:0/2856656511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f46bd0 0x5565e3f4f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f95fdffb640 1 -- 172.21.15.94:0/899812296 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95ec0cd980 con 0x7f9600106500 2024-02-17T12:06:41.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f96055ab640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f95f00a2230 0x7f95f00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f96055ab640 1 --2- 172.21.15.94:0/2856656511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f462b0 0x5565e3f46690 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009ca0 tx=0x7f95f8009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8994f96900b32c05 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.325+0000 7f95eaffd640 1 -- 172.21.15.94:0/2856656511 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8034a90 con 0x5565e3f462b0 2024-02-17T12:06:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.326+0000 7f95eaffd640 1 -- 172.21.15.94:0/2856656511 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95f8003e50 con 0x5565e3f462b0 2024-02-17T12:06:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.326+0000 7f95eaffd640 1 -- 172.21.15.94:0/2856656511 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f80044d0 con 0x5565e3f462b0 2024-02-17T12:06:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.326+0000 7f9609be5740 1 -- 172.21.15.94:0/2856656511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f462b0 msgr2=0x5565e3f46690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.326+0000 7f96055ab640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f95f00a2230 0x7f95f00a46f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95f804b000 tx=0x7f95f804e000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.326+0000 7f9609be5740 1 --2- 172.21.15.94:0/2856656511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f462b0 0x5565e3f46690 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009ca0 tx=0x7f95f8009540 comp rx=0 tx=0).stop 2024-02-17T12:06:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.326+0000 7f9609be5740 1 -- 172.21.15.94:0/2856656511 shutdown_connections 2024-02-17T12:06:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.326+0000 7f9609be5740 1 --2- 172.21.15.94:0/2856656511 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 0x5565e3f51ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.326+0000 7f9609be5740 1 --2- 172.21.15.94:0/2856656511 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f46bd0 0x5565e3f4f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.326+0000 7f9609be5740 1 --2- 172.21.15.94:0/2856656511 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f462b0 0x5565e3f46690 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.326+0000 7f9609be5740 1 -- 172.21.15.94:0/2856656511 >> 172.21.15.94:0/2856656511 conn(0x5565e3f40990 msgr2=0x5565e3f40d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.328+0000 7f9609be5740 1 -- 172.21.15.94:0/2856656511 shutdown_connections 2024-02-17T12:06:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.328+0000 7f9609be5740 1 -- 172.21.15.94:0/2856656511 wait complete. 2024-02-17T12:06:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.328+0000 7f9609be5740 1 -- start start 2024-02-17T12:06:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.328+0000 7f9609be5740 1 Processor -- start 2024-02-17T12:06:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9609be5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f462b0 0x5565e3f594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9609be5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f46bd0 0x5565e3f599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9609be5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 0x5565e3f538b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9609be5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e3b81430 con 0x5565e3f462b0 2024-02-17T12:06:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9609be5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e3b3de00 con 0x5565e3f46bd0 2024-02-17T12:06:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9609be5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e3e32fd0 con 0x5565e3f4f7b0 2024-02-17T12:06:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9604daa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f462b0 0x5565e3f594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f95fffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f46bd0 0x5565e3f599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9604daa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f462b0 0x5565e3f594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53714/0 (socket says 172.21.15.94:53714) 2024-02-17T12:06:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9604daa640 1 -- 172.21.15.94:0/178318328 learned_addr learned my addr 172.21.15.94:0/178318328 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9609be5740 10 client.0 _ll_register_callbacks cb 0x5565e3e88fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f96055ab640 1 --2- 172.21.15.94:0/178318328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 0x5565e3f538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9604daa640 1 -- 172.21.15.94:0/178318328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 msgr2=0x5565e3f538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9604daa640 1 --2- 172.21.15.94:0/178318328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 0x5565e3f538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9604daa640 1 -- 172.21.15.94:0/178318328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f46bd0 msgr2=0x5565e3f599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:41.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9604daa640 1 --2- 172.21.15.94:0/178318328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f46bd0 0x5565e3f599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f9604daa640 1 -- 172.21.15.94:0/178318328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960010e850 con 0x5565e3f462b0 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.329+0000 7f95fffff640 1 --2- 172.21.15.94:0/178318328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f46bd0 0x5565e3f599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.330+0000 7f9604daa640 1 --2- 172.21.15.94:0/178318328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f462b0 0x5565e3f594a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f95f4012d20 tx=0x7f95f400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ce7d3cbfc6d72b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.330+0000 7f95dffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.330+0000 7f95dffff640 1 -- 172.21.15.94:0/178318328 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4017070 con 0x5565e3f462b0 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.330+0000 7f95dffff640 1 -- 172.21.15.94:0/178318328 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95f40120d0 con 0x5565e3f462b0 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.330+0000 7f9609be5740 1 -- 172.21.15.94:0/178318328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565e3bad7e0 con 0x5565e3f462b0 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.330+0000 7f95dffff640 1 -- 172.21.15.94:0/178318328 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95f4012290 con 0x5565e3f462b0 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.330+0000 7f9609be5740 10 client.17724 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.330+0000 7f9609be5740 1 -- 172.21.15.94:0/178318328 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5565e3eac610 con 0x5565e3f462b0 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.330+0000 7f95dffff640 1 -- 172.21.15.94:0/178318328 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5565e3bad7e0 con 0x5565e3f462b0 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.330+0000 7f95dffff640 1 -- 172.21.15.94:0/178318328 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f95f4010660 con 0x5565e3f462b0 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.331+0000 7f9609be5740 10 client.17724 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:41.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.331+0000 7f9609be5740 10 client.17724 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:41.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.331+0000 7f9609be5740 20 client.17724 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:41.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.331+0000 7f9609be5740 1 --2- 172.21.15.94:0/178318328 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5565e3f63dc0 0x5565e3f66220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.331+0000 7f9609be5740 4 client.17724 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:41.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.331+0000 7f9609be5740 1 -- 172.21.15.94:0/178318328 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565e3bea720 con 0x5565e3f63dc0 2024-02-17T12:06:41.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.331+0000 7f95fffff640 1 --2- 172.21.15.94:0/178318328 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5565e3f63dc0 0x5565e3f66220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.331+0000 7f95dffff640 10 client.17724 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:41.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.331+0000 7f95fffff640 1 --2- 172.21.15.94:0/178318328 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5565e3f63dc0 0x5565e3f66220 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9ac349ceab6ebbd server_cookie=c838e4e5fd973a5a in_seq=0 out_seq=0 2024-02-17T12:06:41.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f95dffff640 1 -- 172.21.15.94:0/178318328 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565e3bea720 con 0x5565e3f63dc0 2024-02-17T12:06:41.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f95dffff640 10 client.17724 handle_command_reply: tid=0 2024-02-17T12:06:41.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f9609be5740 1 client.17724 shutdown 2024-02-17T12:06:41.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f9609be5740 1 -- 172.21.15.94:0/178318328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f462b0 msgr2=0x5565e3f594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f9609be5740 1 --2- 172.21.15.94:0/178318328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f462b0 0x5565e3f594a0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f95f4012d20 tx=0x7f95f400f790 comp rx=0 tx=0).stop 2024-02-17T12:06:41.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f9609be5740 1 -- 172.21.15.94:0/178318328 shutdown_connections 2024-02-17T12:06:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f9609be5740 1 --2- 172.21.15.94:0/178318328 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5565e3f63dc0 0x5565e3f66220 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f9609be5740 1 --2- 172.21.15.94:0/178318328 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 0x5565e3f538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f95dffff640 0 client.17724 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f9609be5740 1 --2- 172.21.15.94:0/178318328 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f46bd0 0x5565e3f599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f9609be5740 1 --2- 172.21.15.94:0/178318328 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f462b0 0x5565e3f594a0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f9609be5740 1 -- 172.21.15.94:0/178318328 >> 172.21.15.94:0/178318328 conn(0x5565e3f40990 msgr2=0x5565e3f42730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f9609be5740 1 -- 172.21.15.94:0/178318328 shutdown_connections 2024-02-17T12:06:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f9609be5740 1 -- 172.21.15.94:0/178318328 wait complete. 2024-02-17T12:06:41.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.332+0000 7f9609be5740 20 client.17724 trim_cache size 0 max 16384 2024-02-17T12:06:41.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.366+0000 7f9609be5740 1 Processor -- start 2024-02-17T12:06:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.366+0000 7f9609be5740 1 -- start start 2024-02-17T12:06:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.366+0000 7f9609be5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f462b0 0x5565e3f53140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.366+0000 7f9609be5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f46bd0 0x5565e3f53680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.366+0000 7f9609be5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 0x5565e3f4dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.366+0000 7f9609be5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e3f601f0 con 0x5565e3f462b0 2024-02-17T12:06:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.366+0000 7f9609be5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e3f419e0 con 0x5565e3f46bd0 2024-02-17T12:06:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.366+0000 7f9609be5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e3f45630 con 0x5565e3f4f7b0 2024-02-17T12:06:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.367+0000 7f96055ab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 0x5565e3f4dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.367+0000 7f96055ab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 0x5565e3f4dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:48382/0 (socket says 172.21.15.94:48382) 2024-02-17T12:06:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.367+0000 7f96055ab640 1 -- 172.21.15.94:0/134310020 learned_addr learned my addr 172.21.15.94:0/134310020 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.367+0000 7f9604daa640 1 --2- 172.21.15.94:0/134310020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f462b0 0x5565e3f53140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.367+0000 7f95fffff640 1 --2- 172.21.15.94:0/134310020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f46bd0 0x5565e3f53680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.367+0000 7f9604daa640 1 -- 172.21.15.94:0/134310020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 msgr2=0x5565e3f4dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:41.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.367+0000 7f9604daa640 1 --2- 172.21.15.94:0/134310020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 0x5565e3f4dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.367+0000 7f9604daa640 1 -- 172.21.15.94:0/134310020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f46bd0 msgr2=0x5565e3f53680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:41.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.367+0000 7f9604daa640 1 --2- 172.21.15.94:0/134310020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f46bd0 0x5565e3f53680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.367+0000 7f9604daa640 1 -- 172.21.15.94:0/134310020 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565e3e48ac0 con 0x5565e3f462b0 2024-02-17T12:06:41.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.368+0000 7f9604daa640 1 --2- 172.21.15.94:0/134310020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f462b0 0x5565e3f53140 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f95f4012cf0 tx=0x7f95f4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a2be67367ebfa server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:41.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.369+0000 7f95fffff640 1 --2- 172.21.15.94:0/134310020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f46bd0 0x5565e3f53680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:06:41.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.369+0000 7f95ea7fc640 1 -- 172.21.15.94:0/134310020 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4017040 con 0x5565e3f462b0 2024-02-17T12:06:41.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.369+0000 7f95ea7fc640 1 -- 172.21.15.94:0/134310020 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95f4010be0 con 0x5565e3f462b0 2024-02-17T12:06:41.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.369+0000 7f95ea7fc640 1 -- 172.21.15.94:0/134310020 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4005590 con 0x5565e3f462b0 2024-02-17T12:06:41.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.370+0000 7f9609be5740 1 -- 172.21.15.94:0/134310020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f462b0 msgr2=0x5565e3f53140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.370+0000 7f9609be5740 1 --2- 172.21.15.94:0/134310020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f462b0 0x5565e3f53140 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f95f4012cf0 tx=0x7f95f4012000 comp rx=0 tx=0).stop 2024-02-17T12:06:41.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.370+0000 7f9609be5740 1 -- 172.21.15.94:0/134310020 shutdown_connections 2024-02-17T12:06:41.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.370+0000 7f9609be5740 1 --2- 172.21.15.94:0/134310020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f4f7b0 0x5565e3f4dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.370+0000 7f9609be5740 1 --2- 172.21.15.94:0/134310020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f46bd0 0x5565e3f53680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.370+0000 7f9609be5740 1 --2- 172.21.15.94:0/134310020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f462b0 0x5565e3f53140 secure :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f95f4012cf0 tx=0x7f95f4012000 comp rx=0 tx=0).stop 2024-02-17T12:06:41.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.370+0000 7f9609be5740 1 -- 172.21.15.94:0/134310020 >> 172.21.15.94:0/134310020 conn(0x5565e3f40990 msgr2=0x5565e3f6abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:41.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.370+0000 7f9609be5740 1 -- 172.21.15.94:0/134310020 shutdown_connections 2024-02-17T12:06:41.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.370+0000 7f9609be5740 1 -- 172.21.15.94:0/134310020 wait complete. 2024-02-17T12:06:41.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.371+0000 7f9609be5740 1 -- start start 2024-02-17T12:06:41.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.371+0000 7f9609be5740 1 Processor -- start 2024-02-17T12:06:41.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9609be5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f46bd0 0x5565e3f55f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9609be5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f4f7b0 0x5565e3f56490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9609be5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f569d0 0x5565e3fb6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9609be5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e3fb6860 con 0x5565e3f46bd0 2024-02-17T12:06:41.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9609be5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e3fc7af0 con 0x5565e3f4f7b0 2024-02-17T12:06:41.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9609be5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565e3fc7c30 con 0x5565e3f569d0 2024-02-17T12:06:41.373 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9609be5740 10 client.0 _ll_register_callbacks cb 0x7f95f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9604daa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f46bd0 0x5565e3f55f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9604daa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f46bd0 0x5565e3f55f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53724/0 (socket says 172.21.15.94:53724) 2024-02-17T12:06:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9604daa640 1 -- 172.21.15.94:0/3059972460 learned_addr learned my addr 172.21.15.94:0/3059972460 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f96055ab640 1 --2- 172.21.15.94:0/3059972460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f569d0 0x5565e3fb6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9604daa640 1 -- 172.21.15.94:0/3059972460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f569d0 msgr2=0x5565e3fb6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9604daa640 1 --2- 172.21.15.94:0/3059972460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f569d0 0x5565e3fb6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9604daa640 1 -- 172.21.15.94:0/3059972460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f4f7b0 msgr2=0x5565e3f56490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:06:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9604daa640 1 --2- 172.21.15.94:0/3059972460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f4f7b0 0x5565e3f56490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.372+0000 7f9604daa640 1 -- 172.21.15.94:0/3059972460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565e3eac610 con 0x5565e3f46bd0 2024-02-17T12:06:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.373+0000 7f9604daa640 1 --2- 172.21.15.94:0/3059972460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f46bd0 0x5565e3f55f50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f95f4012cc0 tx=0x7f95f400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae118ed9cd2f98c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.373+0000 7f95dffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.373+0000 7f95dffff640 1 -- 172.21.15.94:0/3059972460 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4017070 con 0x5565e3f46bd0 2024-02-17T12:06:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.373+0000 7f95dffff640 1 -- 172.21.15.94:0/3059972460 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95f4005310 con 0x5565e3f46bd0 2024-02-17T12:06:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.373+0000 7f9609be5740 1 -- 172.21.15.94:0/3059972460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95f8003e50 con 0x5565e3f46bd0 2024-02-17T12:06:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.373+0000 7f95dffff640 1 -- 172.21.15.94:0/3059972460 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95f4010d50 con 0x5565e3f46bd0 2024-02-17T12:06:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.373+0000 7f9609be5740 10 client.17730 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.373+0000 7f9609be5740 1 -- 172.21.15.94:0/3059972460 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5565e3e6e6a0 con 0x5565e3f46bd0 2024-02-17T12:06:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.374+0000 7f95dffff640 1 -- 172.21.15.94:0/3059972460 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95f8003e50 con 0x5565e3f46bd0 2024-02-17T12:06:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.374+0000 7f95dffff640 1 -- 172.21.15.94:0/3059972460 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f95f4035070 con 0x5565e3f46bd0 2024-02-17T12:06:41.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.374+0000 7f9609be5740 10 client.17730 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.374+0000 7f9609be5740 10 client.17730 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.374+0000 7f9609be5740 20 client.17730 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.374+0000 7f9609be5740 1 --2- 172.21.15.94:0/3059972460 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5565e3fc8fa0 0x5565e3fcb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.374+0000 7f9609be5740 4 client.17730 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:06:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.374+0000 7f9609be5740 1 -- 172.21.15.94:0/3059972460 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5565e3c39890 con 0x5565e3fc8fa0 2024-02-17T12:06:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.374+0000 7f95fffff640 1 --2- 172.21.15.94:0/3059972460 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5565e3fc8fa0 0x5565e3fcb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f95dffff640 10 client.17730 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f95fffff640 1 --2- 172.21.15.94:0/3059972460 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5565e3fc8fa0 0x5565e3fcb3c0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e860152656b00e7 server_cookie=d4eea9b3add711b in_seq=0 out_seq=0 2024-02-17T12:06:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f95dffff640 1 -- 172.21.15.94:0/3059972460 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5565e3c39890 con 0x5565e3fc8fa0 2024-02-17T12:06:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f95dffff640 10 client.17730 handle_command_reply: tid=0 2024-02-17T12:06:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f9609be5740 1 client.17730 shutdown 2024-02-17T12:06:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f9609be5740 1 -- 172.21.15.94:0/3059972460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f46bd0 msgr2=0x5565e3f55f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f9609be5740 1 --2- 172.21.15.94:0/3059972460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f46bd0 0x5565e3f55f50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f95f4012cc0 tx=0x7f95f400f790 comp rx=0 tx=0).stop 2024-02-17T12:06:41.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f9609be5740 1 -- 172.21.15.94:0/3059972460 shutdown_connections 2024-02-17T12:06:41.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f9609be5740 1 --2- 172.21.15.94:0/3059972460 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5565e3fc8fa0 0x5565e3fcb3c0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f9609be5740 1 --2- 172.21.15.94:0/3059972460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5565e3f569d0 0x5565e3fb6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f9609be5740 1 --2- 172.21.15.94:0/3059972460 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5565e3f4f7b0 0x5565e3f56490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f9609be5740 1 --2- 172.21.15.94:0/3059972460 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5565e3f46bd0 0x5565e3f55f50 secure :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f95f4012cc0 tx=0x7f95f400f790 comp rx=0 tx=0).stop 2024-02-17T12:06:41.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f9609be5740 1 -- 172.21.15.94:0/3059972460 >> 172.21.15.94:0/3059972460 conn(0x5565e3f40990 msgr2=0x5565e3f53110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:41.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f95dffff640 0 client.17730 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:41.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f9609be5740 1 -- 172.21.15.94:0/3059972460 shutdown_connections 2024-02-17T12:06:41.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f9609be5740 1 -- 172.21.15.94:0/3059972460 wait complete. 2024-02-17T12:06:41.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.375+0000 7f9609be5740 20 client.17730 trim_cache size 0 max 16384 2024-02-17T12:06:41.380 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:06:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:06:41.380 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:06:41.380 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:06:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.376+0000 7f960700e640 1 -- 172.21.15.94:0/899812296 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f95f00a2230 msgr2=0x7f95f00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.376+0000 7f960700e640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f95f00a2230 0x7f95f00a46f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f95f804b000 tx=0x7f95f804e000 comp rx=0 tx=0).stop 2024-02-17T12:06:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.376+0000 7f960700e640 1 -- 172.21.15.94:0/899812296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9600106500 msgr2=0x7f9600197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.376+0000 7f960700e640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9600106500 0x7f9600197b00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00d740 tx=0x7f95ec00dc00 comp rx=0 tx=0).stop 2024-02-17T12:06:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.376+0000 7f960700e640 1 -- 172.21.15.94:0/899812296 shutdown_connections 2024-02-17T12:06:41.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.376+0000 7f960700e640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f95f00a2230 0x7f95f00a46f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.376+0000 7f960700e640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f960019be80 0x7f960019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.376+0000 7f960700e640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9600106500 0x7f9600197b00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.376+0000 7f960700e640 1 --2- 172.21.15.94:0/899812296 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f96000ff8f0 0x7f96001975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.376+0000 7f960700e640 1 -- 172.21.15.94:0/899812296 >> 172.21.15.94:0/899812296 conn(0x7f96000765b0 msgr2=0x7f9600105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:41.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.377+0000 7f960700e640 1 -- 172.21.15.94:0/899812296 shutdown_connections 2024-02-17T12:06:41.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.377+0000 7f960700e640 1 -- 172.21.15.94:0/899812296 wait complete. 2024-02-17T12:06:41.392 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:06:41.393 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:06:41.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.516+0000 7f2b2a062640 1 -- 172.21.15.94:0/910628267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b2410bb50 msgr2=0x7f2b2410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.516+0000 7f2b2a062640 1 --2- 172.21.15.94:0/910628267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b2410bb50 0x7f2b2410bfd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2b18009900 tx=0x7f2b1802bc30 comp rx=0 tx=0).stop 2024-02-17T12:06:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.516+0000 7f2b2a062640 1 -- 172.21.15.94:0/910628267 shutdown_connections 2024-02-17T12:06:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.516+0000 7f2b2a062640 1 --2- 172.21.15.94:0/910628267 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b2410c510 0x7f2b241189d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.516+0000 7f2b2a062640 1 --2- 172.21.15.94:0/910628267 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b2410bb50 0x7f2b2410bfd0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.516+0000 7f2b2a062640 1 --2- 172.21.15.94:0/910628267 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b24111c10 0x7f2b24111ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.519 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.516+0000 7f2b2a062640 1 -- 172.21.15.94:0/910628267 >> 172.21.15.94:0/910628267 conn(0x7f2b2406c1e0 msgr2=0x7f2b2410a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:41.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.518+0000 7f2b2a062640 1 -- 172.21.15.94:0/910628267 shutdown_connections 2024-02-17T12:06:41.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.518+0000 7f2b2a062640 1 -- 172.21.15.94:0/910628267 wait complete. 2024-02-17T12:06:41.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.519+0000 7f2b2a062640 1 Processor -- start 2024-02-17T12:06:41.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.519+0000 7f2b2a062640 1 -- start start 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.519+0000 7f2b2a062640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b2410bb50 0x7f2b241a2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.519+0000 7f2b2a062640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b2410c510 0x7f2b241a2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.519+0000 7f2b2a062640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b24111c10 0x7f2b241a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.519+0000 7f2b2a062640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b241a6fd0 con 0x7f2b2410c510 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.519+0000 7f2b2a062640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b241a7140 con 0x7f2b2410bb50 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.519+0000 7f2b2a062640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b241a7300 con 0x7f2b24111c10 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.519+0000 7f2b22ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b2410c510 0x7f2b241a2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.519+0000 7f2b22ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b2410c510 0x7f2b241a2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53732/0 (socket says 172.21.15.94:53732) 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.519+0000 7f2b22ffd640 1 -- 172.21.15.94:0/3334629395 learned_addr learned my addr 172.21.15.94:0/3334629395 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.519+0000 7f2b237fe640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b2410bb50 0x7f2b241a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.520+0000 7f2b23fff640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b24111c10 0x7f2b241a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.520+0000 7f2b22ffd640 1 -- 172.21.15.94:0/3334629395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b24111c10 msgr2=0x7f2b241a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.520+0000 7f2b22ffd640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b24111c10 0x7f2b241a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.520+0000 7f2b22ffd640 1 -- 172.21.15.94:0/3334629395 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b2410bb50 msgr2=0x7f2b241a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.520+0000 7f2b22ffd640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b2410bb50 0x7f2b241a2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.520+0000 7f2b22ffd640 1 -- 172.21.15.94:0/3334629395 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b18009560 con 0x7f2b2410c510 2024-02-17T12:06:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.520+0000 7f2b22ffd640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b2410c510 0x7f2b241a2650 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2b18005940 tx=0x7f2b1802c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.520+0000 7f2b20ff9640 1 -- 172.21.15.94:0/3334629395 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b180043d0 con 0x7f2b2410c510 2024-02-17T12:06:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.520+0000 7f2b2a062640 1 -- 172.21.15.94:0/3334629395 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b241a7580 con 0x7f2b2410c510 2024-02-17T12:06:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.520+0000 7f2b20ff9640 1 -- 172.21.15.94:0/3334629395 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b180052d0 con 0x7f2b2410c510 2024-02-17T12:06:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.520+0000 7f2b20ff9640 1 -- 172.21.15.94:0/3334629395 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b18003cc0 con 0x7f2b2410c510 2024-02-17T12:06:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.520+0000 7f2b2a062640 1 -- 172.21.15.94:0/3334629395 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b241aee60 con 0x7f2b2410c510 2024-02-17T12:06:41.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.521+0000 7f2b2a062640 1 -- 172.21.15.94:0/3334629395 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b24067f70 con 0x7f2b2410c510 2024-02-17T12:06:41.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.525+0000 7f2b20ff9640 1 -- 172.21.15.94:0/3334629395 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2b18005440 con 0x7f2b2410c510 2024-02-17T12:06:41.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.525+0000 7f2b20ff9640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2b0c0a2230 0x7f2b0c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:41.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.526+0000 7f2b20ff9640 1 -- 172.21.15.94:0/3334629395 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2b180f07b0 con 0x7f2b2410c510 2024-02-17T12:06:41.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.526+0000 7f2b237fe640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2b0c0a2230 0x7f2b0c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:41.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.526+0000 7f2b237fe640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2b0c0a2230 0x7f2b0c0a46f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009bd0 tx=0x7f2b14005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:41.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.528+0000 7f2b20ff9640 1 -- 172.21.15.94:0/3334629395 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b180b9510 con 0x7f2b2410c510 2024-02-17T12:06:41.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.877+0000 7f2b2a062640 1 -- 172.21.15.94:0/3334629395 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b24113f90 con 0x7f2b2410c510 2024-02-17T12:06:41.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.879+0000 7f2b20ff9640 1 -- 172.21.15.94:0/3334629395 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7f2b180bcbd0 con 0x7f2b2410c510 2024-02-17T12:06:41.880 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:06:41.881 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:06:41.881 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:06:41.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.881+0000 7f2b2a062640 1 -- 172.21.15.94:0/3334629395 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2b0c0a2230 msgr2=0x7f2b0c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.881+0000 7f2b2a062640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2b0c0a2230 0x7f2b0c0a46f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009bd0 tx=0x7f2b14005f90 comp rx=0 tx=0).stop 2024-02-17T12:06:41.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.881+0000 7f2b2a062640 1 -- 172.21.15.94:0/3334629395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b2410c510 msgr2=0x7f2b241a2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:41.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.881+0000 7f2b2a062640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b2410c510 0x7f2b241a2650 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2b18005940 tx=0x7f2b1802c700 comp rx=0 tx=0).stop 2024-02-17T12:06:41.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.881+0000 7f2b2a062640 1 -- 172.21.15.94:0/3334629395 shutdown_connections 2024-02-17T12:06:41.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.881+0000 7f2b2a062640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2b0c0a2230 0x7f2b0c0a46f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.881+0000 7f2b2a062640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b24111c10 0x7f2b241a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.881+0000 7f2b2a062640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b2410c510 0x7f2b241a2650 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.881+0000 7f2b2a062640 1 --2- 172.21.15.94:0/3334629395 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b2410bb50 0x7f2b241a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:41.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.881+0000 7f2b2a062640 1 -- 172.21.15.94:0/3334629395 >> 172.21.15.94:0/3334629395 conn(0x7f2b2406c1e0 msgr2=0x7f2b2410a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:41.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.881+0000 7f2b2a062640 1 -- 172.21.15.94:0/3334629395 shutdown_connections 2024-02-17T12:06:41.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:41.881+0000 7f2b2a062640 1 -- 172.21.15.94:0/3334629395 wait complete. 2024-02-17T12:06:41.898 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:06:41.898 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:06:42.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.046+0000 7f0af9de9640 1 -- 172.21.15.94:0/783898034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af4100040 msgr2=0x7f0af41004a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.046+0000 7f0af9de9640 1 --2- 172.21.15.94:0/783898034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af4100040 0x7f0af41004a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009900 tx=0x7f0ae802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.046+0000 7f0af9de9640 1 -- 172.21.15.94:0/783898034 shutdown_connections 2024-02-17T12:06:42.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.046+0000 7f0af9de9640 1 --2- 172.21.15.94:0/783898034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0af41009e0 0x7f0af410cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.046+0000 7f0af9de9640 1 --2- 172.21.15.94:0/783898034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af4100040 0x7f0af41004a0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.046+0000 7f0af9de9640 1 --2- 172.21.15.94:0/783898034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0af4106100 0x7f0af41064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.046+0000 7f0af9de9640 1 -- 172.21.15.94:0/783898034 >> 172.21.15.94:0/783898034 conn(0x7f0af4077110 msgr2=0x7f0af4077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.046+0000 7f0af9de9640 1 -- 172.21.15.94:0/783898034 shutdown_connections 2024-02-17T12:06:42.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.046+0000 7f0af9de9640 1 -- 172.21.15.94:0/783898034 wait complete. 2024-02-17T12:06:42.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.046+0000 7f0af9de9640 1 Processor -- start 2024-02-17T12:06:42.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af9de9640 1 -- start start 2024-02-17T12:06:42.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af9de9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0af4100040 0x7f0af410b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af9de9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0af41009e0 0x7f0af410ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af9de9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af4106100 0x7f0af41a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af9de9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af410c3b0 con 0x7f0af4106100 2024-02-17T12:06:42.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af9de9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41aa050 con 0x7f0af4100040 2024-02-17T12:06:42.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af9de9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41aa1e0 con 0x7f0af41009e0 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af4106100 0x7f0af41a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af4106100 0x7f0af41a9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53764/0 (socket says 172.21.15.94:53764) 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af3fff640 1 -- 172.21.15.94:0/2227826318 learned_addr learned my addr 172.21.15.94:0/2227826318 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af37fe640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0af4100040 0x7f0af410b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0af41009e0 0x7f0af410ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af3fff640 1 -- 172.21.15.94:0/2227826318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0af41009e0 msgr2=0x7f0af410ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af3fff640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0af41009e0 0x7f0af410ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af3fff640 1 -- 172.21.15.94:0/2227826318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0af4100040 msgr2=0x7f0af410b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af3fff640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0af4100040 0x7f0af410b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af3fff640 1 -- 172.21.15.94:0/2227826318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae8009560 con 0x7f0af4106100 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af37fe640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0af4100040 0x7f0af410b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.047+0000 7f0af3fff640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af4106100 0x7f0af41a9b10 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00eb40 tx=0x7f0aec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.048+0000 7f0af0ff9640 1 -- 172.21.15.94:0/2227826318 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec004b20 con 0x7f0af4106100 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.048+0000 7f0af9de9640 1 -- 172.21.15.94:0/2227826318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af41aa4c0 con 0x7f0af4106100 2024-02-17T12:06:42.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.048+0000 7f0af0ff9640 1 -- 172.21.15.94:0/2227826318 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0aec004c80 con 0x7f0af4106100 2024-02-17T12:06:42.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.048+0000 7f0af0ff9640 1 -- 172.21.15.94:0/2227826318 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec010670 con 0x7f0af4106100 2024-02-17T12:06:42.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.048+0000 7f0af9de9640 1 -- 172.21.15.94:0/2227826318 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af41aa9a0 con 0x7f0af4106100 2024-02-17T12:06:42.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.050+0000 7f0af0ff9640 1 -- 172.21.15.94:0/2227826318 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0aec0108f0 con 0x7f0af4106100 2024-02-17T12:06:42.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.050+0000 7f0afc9be740 1 Processor -- start 2024-02-17T12:06:42.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.051+0000 7f0afc9be740 1 -- start start 2024-02-17T12:06:42.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.051+0000 7f0afc9be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33b250 0x555c0e33b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.051+0000 7f0afc9be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33bb70 0x555c0e344210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.051+0000 7f0afc9be740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e344750 0x555c0e346b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.051+0000 7f0afc9be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0e2f6960 con 0x555c0e33bb70 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.051+0000 7f0afc9be740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0e2aa310 con 0x555c0e344750 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.051+0000 7f0afc9be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0e2a7840 con 0x555c0e33b250 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.051+0000 7f0af2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e344750 0x555c0e346b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.051+0000 7f0af37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33bb70 0x555c0e344210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.051+0000 7f0af2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e344750 0x555c0e346b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52522/0 (socket says 172.21.15.94:52522) 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.051+0000 7f0af2ffd640 1 -- 172.21.15.94:0/2731746210 learned_addr learned my addr 172.21.15.94:0/2731746210 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.051+0000 7f0af0ff9640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0adc0a2270 0x7f0adc0a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.051+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/2731746210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33b250 0x555c0e33b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.052+0000 7f0af3fff640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0adc0a2270 0x7f0adc0a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.052+0000 7f0af0ff9640 1 -- 172.21.15.94:0/2227826318 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0aec014070 con 0x7f0af4106100 2024-02-17T12:06:42.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.052+0000 7f0af2ffd640 1 -- 172.21.15.94:0/2731746210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33b250 msgr2=0x555c0e33b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.052+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/2731746210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33b250 0x555c0e33b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.052+0000 7f0af2ffd640 1 -- 172.21.15.94:0/2731746210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33bb70 msgr2=0x555c0e344210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.052+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/2731746210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33bb70 0x555c0e344210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.052+0000 7f0af2ffd640 1 -- 172.21.15.94:0/2731746210 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c0e265060 con 0x555c0e344750 2024-02-17T12:06:42.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.052+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/2731746210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e344750 0x555c0e346b40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009230 tx=0x7f0ae8005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e4aa05051a597c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.052+0000 7f0af3fff640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0adc0a2270 0x7f0adc0a4730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0cbe30 tx=0x7f0aec005830 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.052+0000 7f0ad9ffb640 1 -- 172.21.15.94:0/2731746210 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae80041d0 con 0x555c0e344750 2024-02-17T12:06:42.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.052+0000 7f0ad9ffb640 1 -- 172.21.15.94:0/2731746210 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ae802cd60 con 0x555c0e344750 2024-02-17T12:06:42.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.052+0000 7f0ad9ffb640 1 -- 172.21.15.94:0/2731746210 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8003ca0 con 0x555c0e344750 2024-02-17T12:06:42.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.053+0000 7f0afc9be740 1 -- 172.21.15.94:0/2731746210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e344750 msgr2=0x555c0e346b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.053+0000 7f0afc9be740 1 --2- 172.21.15.94:0/2731746210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e344750 0x555c0e346b40 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009230 tx=0x7f0ae8005b50 comp rx=0 tx=0).stop 2024-02-17T12:06:42.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.053+0000 7f0afc9be740 1 -- 172.21.15.94:0/2731746210 shutdown_connections 2024-02-17T12:06:42.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.053+0000 7f0afc9be740 1 --2- 172.21.15.94:0/2731746210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e344750 0x555c0e346b40 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.053+0000 7f0afc9be740 1 --2- 172.21.15.94:0/2731746210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33bb70 0x555c0e344210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.053+0000 7f0afc9be740 1 --2- 172.21.15.94:0/2731746210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33b250 0x555c0e33b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.053+0000 7f0afc9be740 1 -- 172.21.15.94:0/2731746210 >> 172.21.15.94:0/2731746210 conn(0x555c0e335980 msgr2=0x555c0e335d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.053+0000 7f0afc9be740 1 -- 172.21.15.94:0/2731746210 shutdown_connections 2024-02-17T12:06:42.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.053+0000 7f0afc9be740 1 -- 172.21.15.94:0/2731746210 wait complete. 2024-02-17T12:06:42.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0afc9be740 1 -- start start 2024-02-17T12:06:42.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0afc9be740 1 Processor -- start 2024-02-17T12:06:42.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0afc9be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33b250 0x555c0e3417b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0afc9be740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e33bb70 0x555c0e341cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0afc9be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e344750 0x555c0e342230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0afc9be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0e2a3020 con 0x555c0e33b250 2024-02-17T12:06:42.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0afc9be740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0df76430 con 0x555c0e33bb70 2024-02-17T12:06:42.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0afc9be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0df32e00 con 0x555c0e344750 2024-02-17T12:06:42.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0af2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33b250 0x555c0e3417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0af2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33b250 0x555c0e3417b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53772/0 (socket says 172.21.15.94:53772) 2024-02-17T12:06:42.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0af2ffd640 1 -- 172.21.15.94:0/4021965531 learned_addr learned my addr 172.21.15.94:0/4021965531 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0af37fe640 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e33bb70 0x555c0e341cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0afc9be740 10 client.0 _ll_register_callbacks cb 0x555c0e24acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:42.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.054+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e344750 0x555c0e342230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0af2ffd640 1 -- 172.21.15.94:0/4021965531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e344750 msgr2=0x555c0e342230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e344750 0x555c0e342230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0af2ffd640 1 -- 172.21.15.94:0/4021965531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e33bb70 msgr2=0x555c0e341cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e33bb70 0x555c0e341cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0af2ffd640 1 -- 172.21.15.94:0/4021965531 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af410d7a0 con 0x555c0e33b250 2024-02-17T12:06:42.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0af37fe640 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e33bb70 0x555c0e341cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:06:42.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33b250 0x555c0e3417b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8005c40 tx=0x7f0ae80037e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c5b8cd42bafa1a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:42.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0ad2ffd640 1 -- 172.21.15.94:0/4021965531 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae802cb40 con 0x555c0e33b250 2024-02-17T12:06:42.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0afc9be740 1 -- 172.21.15.94:0/4021965531 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555c0dfa27e0 con 0x555c0e33b250 2024-02-17T12:06:42.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0ad2ffd640 1 -- 172.21.15.94:0/4021965531 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0ae8000e40 con 0x555c0e33b250 2024-02-17T12:06:42.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0ad2ffd640 1 -- 172.21.15.94:0/4021965531 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ae8003a20 con 0x555c0e33b250 2024-02-17T12:06:42.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0afc9be740 10 client.17748 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:42.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.055+0000 7f0afc9be740 1 -- 172.21.15.94:0/4021965531 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555c0e2a16a0 con 0x555c0e33b250 2024-02-17T12:06:42.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.056+0000 7f0ad2ffd640 1 -- 172.21.15.94:0/4021965531 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ae8003b80 con 0x555c0e33b250 2024-02-17T12:06:42.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.056+0000 7f0ad2ffd640 1 -- 172.21.15.94:0/4021965531 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ae804b040 con 0x555c0e33b250 2024-02-17T12:06:42.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.056+0000 7f0afc9be740 10 client.17748 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:42.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.056+0000 7f0afc9be740 10 client.17748 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.056+0000 7f0afc9be740 20 client.17748 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.056+0000 7f0afc9be740 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555c0e3546b0 0x555c0e356b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.056+0000 7f0afc9be740 4 client.17748 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.056+0000 7f0afc9be740 1 -- 172.21.15.94:0/4021965531 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555c0dfa27e0 con 0x555c0e3546b0 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.056+0000 7f0af37fe640 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555c0e3546b0 0x555c0e356b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.057+0000 7f0af37fe640 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555c0e3546b0 0x555c0e356b30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b583c6eb66e4c9f4 server_cookie=5cc64bbd4a15430a in_seq=0 out_seq=0 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.057+0000 7f0ad2ffd640 10 client.17748 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.058+0000 7f0ad2ffd640 1 -- 172.21.15.94:0/4021965531 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555c0dfa27e0 con 0x555c0e3546b0 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.058+0000 7f0ad2ffd640 10 client.17748 handle_command_reply: tid=0 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.058+0000 7f0afc9be740 1 client.17748 shutdown 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.058+0000 7f0afc9be740 1 -- 172.21.15.94:0/4021965531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33b250 msgr2=0x555c0e3417b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.058+0000 7f0afc9be740 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33b250 0x555c0e3417b0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8005c40 tx=0x7f0ae80037e0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.058+0000 7f0afc9be740 1 -- 172.21.15.94:0/4021965531 shutdown_connections 2024-02-17T12:06:42.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.058+0000 7f0afc9be740 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e344750 0x555c0e342230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.058+0000 7f0afc9be740 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555c0e3546b0 0x555c0e356b30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.058+0000 7f0afc9be740 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e33bb70 0x555c0e341cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.058+0000 7f0ad2ffd640 0 client.17748 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:06:42.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.059+0000 7f0afc9be740 1 --2- 172.21.15.94:0/4021965531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33b250 0x555c0e3417b0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.059+0000 7f0afc9be740 1 -- 172.21.15.94:0/4021965531 >> 172.21.15.94:0/4021965531 conn(0x555c0e335980 msgr2=0x555c0e3376d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.059+0000 7f0afc9be740 1 -- 172.21.15.94:0/4021965531 shutdown_connections 2024-02-17T12:06:42.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.059+0000 7f0afc9be740 1 -- 172.21.15.94:0/4021965531 wait complete. 2024-02-17T12:06:42.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.059+0000 7f0afc9be740 20 client.17748 trim_cache size 0 max 16384 2024-02-17T12:06:42.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.073+0000 7f0afc9be740 1 Processor -- start 2024-02-17T12:06:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0afc9be740 1 -- start start 2024-02-17T12:06:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0afc9be740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e33b250 0x555c0e343ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0afc9be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33bb70 0x555c0e342050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0afc9be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e344750 0x555c0e334a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0afc9be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0e228130 con 0x555c0e344750 2024-02-17T12:06:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0afc9be740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0e33a650 con 0x555c0e33b250 2024-02-17T12:06:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0afc9be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0e33d3d0 con 0x555c0e33bb70 2024-02-17T12:06:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0af2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e344750 0x555c0e334a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0af2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e33b250 0x555c0e343ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0af2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e344750 0x555c0e334a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53780/0 (socket says 172.21.15.94:53780) 2024-02-17T12:06:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0af2ffd640 1 -- 172.21.15.94:0/1854846750 learned_addr learned my addr 172.21.15.94:0/1854846750 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0af37fe640 1 --2- 172.21.15.94:0/1854846750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33bb70 0x555c0e342050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0af2ffd640 1 -- 172.21.15.94:0/1854846750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33bb70 msgr2=0x555c0e342050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/1854846750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33bb70 0x555c0e342050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0af2ffd640 1 -- 172.21.15.94:0/1854846750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e33b250 msgr2=0x555c0e343ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/1854846750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e33b250 0x555c0e343ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0af2ffd640 1 -- 172.21.15.94:0/1854846750 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c0e23dc20 con 0x555c0e344750 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.074+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/1854846750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e344750 0x555c0e334a00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0ae805a040 tx=0x7f0ae8009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea2291995ebc54 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.075+0000 7f0ad97fa640 1 -- 172.21.15.94:0/1854846750 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae80035f0 con 0x555c0e344750 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.075+0000 7f0ad97fa640 1 -- 172.21.15.94:0/1854846750 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ae8050dd0 con 0x555c0e344750 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.075+0000 7f0ad97fa640 1 -- 172.21.15.94:0/1854846750 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8003ae0 con 0x555c0e344750 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.075+0000 7f0afc9be740 1 -- 172.21.15.94:0/1854846750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e344750 msgr2=0x555c0e334a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.075+0000 7f0afc9be740 1 --2- 172.21.15.94:0/1854846750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e344750 0x555c0e334a00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0ae805a040 tx=0x7f0ae8009230 comp rx=0 tx=0).stop 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.075+0000 7f0afc9be740 1 -- 172.21.15.94:0/1854846750 shutdown_connections 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.075+0000 7f0afc9be740 1 --2- 172.21.15.94:0/1854846750 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e344750 0x555c0e334a00 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.075+0000 7f0afc9be740 1 --2- 172.21.15.94:0/1854846750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33bb70 0x555c0e342050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.076+0000 7f0afc9be740 1 --2- 172.21.15.94:0/1854846750 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e33b250 0x555c0e343ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.076+0000 7f0afc9be740 1 -- 172.21.15.94:0/1854846750 >> 172.21.15.94:0/1854846750 conn(0x555c0e335980 msgr2=0x555c0e3342f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.076+0000 7f0afc9be740 1 -- 172.21.15.94:0/1854846750 shutdown_connections 2024-02-17T12:06:42.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.076+0000 7f0afc9be740 1 -- 172.21.15.94:0/1854846750 wait complete. 2024-02-17T12:06:42.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.076+0000 7f0afc9be740 1 -- start start 2024-02-17T12:06:42.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.076+0000 7f0afc9be740 1 Processor -- start 2024-02-17T12:06:42.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.076+0000 7f0afc9be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33b250 0x555c0e34b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.076+0000 7f0afc9be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33bb70 0x555c0e349970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.076+0000 7f0afc9be740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e344750 0x555c0e349eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.076+0000 7f0afc9be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0e34bea0 con 0x555c0e33b250 2024-02-17T12:06:42.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.076+0000 7f0afc9be740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0e34a3f0 con 0x555c0e344750 2024-02-17T12:06:42.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.076+0000 7f0afc9be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c0e34a5b0 con 0x555c0e33bb70 2024-02-17T12:06:42.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.076+0000 7f0afc9be740 10 client.0 _ll_register_callbacks cb 0x7f0ae8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:42.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.077+0000 7f0af2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33b250 0x555c0e34b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.077+0000 7f0af37fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33bb70 0x555c0e349970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.077+0000 7f0af2ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33b250 0x555c0e34b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53790/0 (socket says 172.21.15.94:53790) 2024-02-17T12:06:42.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.077+0000 7f0af2ffd640 1 -- 172.21.15.94:0/2911370108 learned_addr learned my addr 172.21.15.94:0/2911370108 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.077+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/2911370108 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e344750 0x555c0e349eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.077+0000 7f0af37fe640 1 -- 172.21.15.94:0/2911370108 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e344750 msgr2=0x555c0e349eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.077+0000 7f0af37fe640 1 --2- 172.21.15.94:0/2911370108 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e344750 0x555c0e349eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.077+0000 7f0af37fe640 1 -- 172.21.15.94:0/2911370108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33b250 msgr2=0x555c0e34b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.077+0000 7f0af37fe640 1 --2- 172.21.15.94:0/2911370108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33b250 0x555c0e34b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.077+0000 7f0af37fe640 1 -- 172.21.15.94:0/2911370108 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c0e2a16a0 con 0x555c0e33bb70 2024-02-17T12:06:42.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.077+0000 7f0af37fe640 1 --2- 172.21.15.94:0/2911370108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33bb70 0x555c0e349970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x555c0e357440 tx=0x7f0ae401e770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8a6ee03cc0fa3c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.077+0000 7f0ad2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:06:42.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.077+0000 7f0ad2ffd640 1 -- 172.21.15.94:0/2911370108 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4003bd0 con 0x555c0e33bb70 2024-02-17T12:06:42.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.078+0000 7f0ad2ffd640 1 -- 172.21.15.94:0/2911370108 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0ae4003d30 con 0x555c0e33bb70 2024-02-17T12:06:42.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.078+0000 7f0afc9be740 1 -- 172.21.15.94:0/2911370108 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ae802cd60 con 0x555c0e33bb70 2024-02-17T12:06:42.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.078+0000 7f0ad2ffd640 1 -- 172.21.15.94:0/2911370108 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ae4005020 con 0x555c0e33bb70 2024-02-17T12:06:42.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.079+0000 7f0afc9be740 10 client.27247 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:42.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.079+0000 7f0afc9be740 1 -- 172.21.15.94:0/2911370108 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555c0e2638c0 con 0x555c0e33bb70 2024-02-17T12:06:42.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.079+0000 7f0ad2ffd640 1 -- 172.21.15.94:0/2911370108 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ae4017040 con 0x555c0e33bb70 2024-02-17T12:06:42.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.079+0000 7f0ad2ffd640 1 -- 172.21.15.94:0/2911370108 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ae4011440 con 0x555c0e33bb70 2024-02-17T12:06:42.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.079+0000 7f0afc9be740 10 client.27247 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:42.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.079+0000 7f0afc9be740 10 client.27247 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:06:42.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.079+0000 7f0afc9be740 20 client.27247 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:42.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.079+0000 7f0afc9be740 1 --2- 172.21.15.94:0/2911370108 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555c0e3546b0 0x555c0e3ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.079+0000 7f0afc9be740 4 client.27247 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:06:42.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.079+0000 7f0afc9be740 1 -- 172.21.15.94:0/2911370108 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f0ae802cd60 con 0x555c0e3546b0 2024-02-17T12:06:42.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.079+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/2911370108 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555c0e3546b0 0x555c0e3ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0ad2ffd640 10 client.27247 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:06:42.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0af2ffd640 1 --2- 172.21.15.94:0/2911370108 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555c0e3546b0 0x555c0e3ae050 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=709632d66770313c server_cookie=9af938e6d42b1248 in_seq=0 out_seq=0 2024-02-17T12:06:42.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0ad2ffd640 1 -- 172.21.15.94:0/2911370108 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f0ae802cd60 con 0x555c0e3546b0 2024-02-17T12:06:42.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0ad2ffd640 10 client.27247 handle_command_reply: tid=0 2024-02-17T12:06:42.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0afc9be740 1 client.27247 shutdown 2024-02-17T12:06:42.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0afc9be740 1 -- 172.21.15.94:0/2911370108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33bb70 msgr2=0x555c0e349970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0afc9be740 1 --2- 172.21.15.94:0/2911370108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33bb70 0x555c0e349970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x555c0e357440 tx=0x7f0ae401e770 comp rx=0 tx=0).stop 2024-02-17T12:06:42.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0afc9be740 1 -- 172.21.15.94:0/2911370108 shutdown_connections 2024-02-17T12:06:42.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0afc9be740 1 --2- 172.21.15.94:0/2911370108 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555c0e3546b0 0x555c0e3ae050 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0ad2ffd640 0 client.27247 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:06:42.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0afc9be740 1 --2- 172.21.15.94:0/2911370108 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555c0e344750 0x555c0e349eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0afc9be740 1 --2- 172.21.15.94:0/2911370108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555c0e33bb70 0x555c0e349970 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0afc9be740 1 --2- 172.21.15.94:0/2911370108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555c0e33b250 0x555c0e34b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.080+0000 7f0afc9be740 1 -- 172.21.15.94:0/2911370108 >> 172.21.15.94:0/2911370108 conn(0x555c0e335980 msgr2=0x555c0e334b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0afc9be740 1 -- 172.21.15.94:0/2911370108 shutdown_connections 2024-02-17T12:06:42.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0afc9be740 1 -- 172.21.15.94:0/2911370108 wait complete. 2024-02-17T12:06:42.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0afc9be740 20 client.27247 trim_cache size 0 max 16384 2024-02-17T12:06:42.086 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:06:42.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0af9de9640 1 -- 172.21.15.94:0/2227826318 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0adc0a2270 msgr2=0x7f0adc0a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0af9de9640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0adc0a2270 0x7f0adc0a4730 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0cbe30 tx=0x7f0aec005830 comp rx=0 tx=0).stop 2024-02-17T12:06:42.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0af9de9640 1 -- 172.21.15.94:0/2227826318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af4106100 msgr2=0x7f0af41a9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0af9de9640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af4106100 0x7f0af41a9b10 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00eb40 tx=0x7f0aec00c2d0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0af9de9640 1 -- 172.21.15.94:0/2227826318 shutdown_connections 2024-02-17T12:06:42.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0af9de9640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0adc0a2270 0x7f0adc0a4730 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0af9de9640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0af4106100 0x7f0af41a9b10 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0af9de9640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0af41009e0 0x7f0af410ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0af9de9640 1 --2- 172.21.15.94:0/2227826318 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0af4100040 0x7f0af410b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0af9de9640 1 -- 172.21.15.94:0/2227826318 >> 172.21.15.94:0/2227826318 conn(0x7f0af4077110 msgr2=0x7f0af41052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0af9de9640 1 -- 172.21.15.94:0/2227826318 shutdown_connections 2024-02-17T12:06:42.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.081+0000 7f0af9de9640 1 -- 172.21.15.94:0/2227826318 wait complete. 2024-02-17T12:06:42.095 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:06:42.096 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:06:42.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:41 smithi094 ceph-mon[32499]: pgmap v264: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-02-17T12:06:42.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:41 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3334629395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:06:42.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.252+0000 7f3512ffd640 1 -- 172.21.15.94:0/2492633846 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350802cc20 con 0x7f351410d9d0 2024-02-17T12:06:42.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.252+0000 7f351a6df640 1 -- 172.21.15.94:0/2492633846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f351410d9d0 msgr2=0x7f351410de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.252+0000 7f351a6df640 1 --2- 172.21.15.94:0/2492633846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f351410d9d0 0x7f351410de50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3508009900 tx=0x7f350802bc80 comp rx=0 tx=0).stop 2024-02-17T12:06:42.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.255+0000 7f351a6df640 1 -- 172.21.15.94:0/2492633846 shutdown_connections 2024-02-17T12:06:42.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.255+0000 7f351a6df640 1 --2- 172.21.15.94:0/2492633846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f351411a960 0x7f351411cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.255+0000 7f351a6df640 1 --2- 172.21.15.94:0/2492633846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f351410d9d0 0x7f351410de50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.255+0000 7f351a6df640 1 --2- 172.21.15.94:0/2492633846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f351410f9a0 0x7f351410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.255+0000 7f351a6df640 1 -- 172.21.15.94:0/2492633846 >> 172.21.15.94:0/2492633846 conn(0x7f351406c260 msgr2=0x7f351406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.256+0000 7f351a6df640 1 -- 172.21.15.94:0/2492633846 shutdown_connections 2024-02-17T12:06:42.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.257+0000 7f351a6df640 1 -- 172.21.15.94:0/2492633846 wait complete. 2024-02-17T12:06:42.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.257+0000 7f351a6df640 1 Processor -- start 2024-02-17T12:06:42.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.257+0000 7f351a6df640 1 -- start start 2024-02-17T12:06:42.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.258+0000 7f351a6df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f351410d9d0 0x7f3514117e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.258+0000 7f351a6df640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f351410f9a0 0x7f3514118340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.258+0000 7f351a6df640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f351411a960 0x7f3514110ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.258+0000 7f351a6df640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3514111780 con 0x7f351410d9d0 2024-02-17T12:06:42.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.258+0000 7f351a6df640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35141118f0 con 0x7f351410f9a0 2024-02-17T12:06:42.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.258+0000 7f351a6df640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3514111ab0 con 0x7f351411a960 2024-02-17T12:06:42.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.258+0000 7f3513fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f351410d9d0 0x7f3514117e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.258+0000 7f3513fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f351410d9d0 0x7f3514117e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53794/0 (socket says 172.21.15.94:53794) 2024-02-17T12:06:42.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.258+0000 7f3513fff640 1 -- 172.21.15.94:0/1642871414 learned_addr learned my addr 172.21.15.94:0/1642871414 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.259+0000 7f3518c7c640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f351411a960 0x7f3514110ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.259+0000 7f3513fff640 1 -- 172.21.15.94:0/1642871414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f351411a960 msgr2=0x7f3514110ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.259+0000 7f35137fe640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f351410f9a0 0x7f3514118340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.259+0000 7f3513fff640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f351411a960 0x7f3514110ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.259+0000 7f3513fff640 1 -- 172.21.15.94:0/1642871414 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f351410f9a0 msgr2=0x7f3514118340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.259+0000 7f3513fff640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f351410f9a0 0x7f3514118340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.259+0000 7f3513fff640 1 -- 172.21.15.94:0/1642871414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3508009560 con 0x7f351410d9d0 2024-02-17T12:06:42.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.259+0000 7f35137fe640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f351410f9a0 0x7f3514118340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:06:42.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.259+0000 7f3513fff640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f351410d9d0 0x7f3514117e00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f350400b450 tx=0x7f350400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.264+0000 7f35117fa640 1 -- 172.21.15.94:0/1642871414 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3504011890 con 0x7f351410d9d0 2024-02-17T12:06:42.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.264+0000 7f351a6df640 1 -- 172.21.15.94:0/1642871414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3514115920 con 0x7f351410d9d0 2024-02-17T12:06:42.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.264+0000 7f35117fa640 1 -- 172.21.15.94:0/1642871414 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35040103e0 con 0x7f351410d9d0 2024-02-17T12:06:42.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.264+0000 7f35117fa640 1 -- 172.21.15.94:0/1642871414 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350400f5a0 con 0x7f351410d9d0 2024-02-17T12:06:42.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.264+0000 7f351a6df640 1 -- 172.21.15.94:0/1642871414 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3514115e30 con 0x7f351410d9d0 2024-02-17T12:06:42.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.267+0000 7f351d392740 1 Processor -- start 2024-02-17T12:06:42.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.268+0000 7f35117fa640 1 -- 172.21.15.94:0/1642871414 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3504010550 con 0x7f351410d9d0 2024-02-17T12:06:42.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.269+0000 7f351d392740 1 -- start start 2024-02-17T12:06:42.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.269+0000 7f351d392740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8250 0x55d5726a8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.269+0000 7f351d392740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726b1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.269+0000 7f351d392740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726b1750 0x55d5726b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.269+0000 7f351d392740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d572663960 con 0x55d5726a8b70 2024-02-17T12:06:42.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.269+0000 7f351d392740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d572617310 con 0x55d5726b1750 2024-02-17T12:06:42.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.269+0000 7f351d392740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d572614840 con 0x55d5726a8250 2024-02-17T12:06:42.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.269+0000 7f3518c7c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8250 0x55d5726a8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.269+0000 7f35137fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726b1750 0x55d5726b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.269+0000 7f35137fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726b1750 0x55d5726b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52574/0 (socket says 172.21.15.94:52574) 2024-02-17T12:06:42.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.269+0000 7f35137fe640 1 -- 172.21.15.94:0/3258986821 learned_addr learned my addr 172.21.15.94:0/3258986821 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.269+0000 7f3513fff640 1 --2- 172.21.15.94:0/3258986821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.270+0000 7f35137fe640 1 -- 172.21.15.94:0/3258986821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8250 msgr2=0x55d5726a8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.270+0000 7f35137fe640 1 --2- 172.21.15.94:0/3258986821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8250 0x55d5726a8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.270+0000 7f35137fe640 1 -- 172.21.15.94:0/3258986821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 msgr2=0x55d5726b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.270+0000 7f35137fe640 1 --2- 172.21.15.94:0/3258986821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.270+0000 7f35137fe640 1 -- 172.21.15.94:0/3258986821 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5725d2060 con 0x55d5726b1750 2024-02-17T12:06:42.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.270+0000 7f35117fa640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f34fc0a22b0 0x7f34fc0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.271+0000 7f35137fe640 1 --2- 172.21.15.94:0/3258986821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726b1750 0x55d5726b3b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3508006010 tx=0x7f3508004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8c96a0a9b7f64fb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.271+0000 7f35117fa640 1 -- 172.21.15.94:0/1642871414 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f35040cde30 con 0x7f351410d9d0 2024-02-17T12:06:42.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.271+0000 7f34fa7fc640 1 -- 172.21.15.94:0/3258986821 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350802c860 con 0x55d5726b1750 2024-02-17T12:06:42.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.271+0000 7f34fa7fc640 1 -- 172.21.15.94:0/3258986821 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3508004510 con 0x55d5726b1750 2024-02-17T12:06:42.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.271+0000 7f34fa7fc640 1 -- 172.21.15.94:0/3258986821 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508005500 con 0x55d5726b1750 2024-02-17T12:06:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.271+0000 7f3518c7c640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f34fc0a22b0 0x7f34fc0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.272+0000 7f351d392740 1 -- 172.21.15.94:0/3258986821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726b1750 msgr2=0x55d5726b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.272+0000 7f351d392740 1 --2- 172.21.15.94:0/3258986821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726b1750 0x55d5726b3b40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3508006010 tx=0x7f3508004770 comp rx=0 tx=0).stop 2024-02-17T12:06:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.272+0000 7f351d392740 1 -- 172.21.15.94:0/3258986821 shutdown_connections 2024-02-17T12:06:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.272+0000 7f351d392740 1 --2- 172.21.15.94:0/3258986821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726b1750 0x55d5726b3b40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.272+0000 7f351d392740 1 --2- 172.21.15.94:0/3258986821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.272+0000 7f351d392740 1 --2- 172.21.15.94:0/3258986821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8250 0x55d5726a8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.272+0000 7f351d392740 1 -- 172.21.15.94:0/3258986821 >> 172.21.15.94:0/3258986821 conn(0x55d5726a2980 msgr2=0x55d5726a2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.272+0000 7f3518c7c640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f34fc0a22b0 0x7f34fc0a4770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f350c007d80 tx=0x7f350c007430 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.273+0000 7f351d392740 1 -- 172.21.15.94:0/3258986821 shutdown_connections 2024-02-17T12:06:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.273+0000 7f351d392740 1 -- 172.21.15.94:0/3258986821 wait complete. 2024-02-17T12:06:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.273+0000 7f351d392740 1 -- start start 2024-02-17T12:06:42.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.274+0000 7f351d392740 1 Processor -- start 2024-02-17T12:06:42.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.274+0000 7f351d392740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726a8250 0x55d5726ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.274+0000 7f351d392740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8b70 0x55d5726aed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.274+0000 7f351d392740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726b1750 0x55d5726af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.274+0000 7f351d392740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d572610020 con 0x55d5726b1750 2024-02-17T12:06:42.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.274+0000 7f351d392740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5722e3430 con 0x55d5726a8250 2024-02-17T12:06:42.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.274+0000 7f351d392740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d57229fe00 con 0x55d5726a8b70 2024-02-17T12:06:42.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.274+0000 7f351d392740 10 client.0 _ll_register_callbacks cb 0x55d5725b7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:42.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.274+0000 7f3513fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726a8250 0x55d5726ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f3513fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726a8250 0x55d5726ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52582/0 (socket says 172.21.15.94:52582) 2024-02-17T12:06:42.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f3513fff640 1 -- 172.21.15.94:0/3482863765 learned_addr learned my addr 172.21.15.94:0/3482863765 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f35137fe640 1 --2- 172.21.15.94:0/3482863765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8b70 0x55d5726aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f3518c7c640 1 --2- 172.21.15.94:0/3482863765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726b1750 0x55d5726af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f3513fff640 1 -- 172.21.15.94:0/3482863765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8b70 msgr2=0x55d5726aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f3513fff640 1 --2- 172.21.15.94:0/3482863765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8b70 0x55d5726aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f3513fff640 1 -- 172.21.15.94:0/3482863765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726b1750 msgr2=0x55d5726af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f3513fff640 1 --2- 172.21.15.94:0/3482863765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726b1750 0x55d5726af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f3513fff640 1 -- 172.21.15.94:0/3482863765 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351411d680 con 0x55d5726a8250 2024-02-17T12:06:42.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f3513fff640 1 --2- 172.21.15.94:0/3482863765 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726a8250 0x55d5726ae7e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f350409a5f0 tx=0x7f350409a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50f7b9b02ba21686 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f34f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:06:42.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f34f37fe640 1 -- 172.21.15.94:0/3482863765 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350409a8f0 con 0x55d5726a8250 2024-02-17T12:06:42.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f351d392740 1 -- 172.21.15.94:0/3482863765 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d57230f7e0 con 0x55d5726a8250 2024-02-17T12:06:42.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.275+0000 7f34f37fe640 1 -- 172.21.15.94:0/3482863765 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f350409ea40 con 0x55d5726a8250 2024-02-17T12:06:42.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.276+0000 7f34f37fe640 1 -- 172.21.15.94:0/3482863765 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35040a3090 con 0x55d5726a8250 2024-02-17T12:06:42.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.276+0000 7f351d392740 10 client.27230 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:42.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.276+0000 7f351d392740 1 -- 172.21.15.94:0/3482863765 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d57260e6a0 con 0x55d5726a8250 2024-02-17T12:06:42.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.276+0000 7f34f37fe640 1 -- 172.21.15.94:0/3482863765 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d57230f7e0 con 0x55d5726a8250 2024-02-17T12:06:42.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.276+0000 7f34f37fe640 1 -- 172.21.15.94:0/3482863765 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f35040b32d0 con 0x55d5726a8250 2024-02-17T12:06:42.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.276+0000 7f351d392740 10 client.27230 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:42.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.276+0000 7f351d392740 10 client.27230 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:06:42.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.276+0000 7f351d392740 20 client.27230 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:42.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.276+0000 7f351d392740 1 --2- 172.21.15.94:0/3482863765 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d5726c1c30 0x55d5726c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.276+0000 7f351d392740 4 client.27230 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:42.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.276+0000 7f351d392740 1 -- 172.21.15.94:0/3482863765 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d57234c720 con 0x55d5726c1c30 2024-02-17T12:06:42.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.277+0000 7f35137fe640 1 --2- 172.21.15.94:0/3482863765 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d5726c1c30 0x55d5726c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.278+0000 7f35137fe640 1 --2- 172.21.15.94:0/3482863765 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d5726c1c30 0x55d5726c4020 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3bffdeefc70c0238 server_cookie=3919b77a2a5376c8 in_seq=0 out_seq=0 2024-02-17T12:06:42.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.278+0000 7f34f37fe640 10 client.27230 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:06:42.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.278+0000 7f34f37fe640 1 -- 172.21.15.94:0/3482863765 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d57234c720 con 0x55d5726c1c30 2024-02-17T12:06:42.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.278+0000 7f34f37fe640 10 client.27230 handle_command_reply: tid=0 2024-02-17T12:06:42.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.278+0000 7f351d392740 1 client.27230 shutdown 2024-02-17T12:06:42.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.279+0000 7f351d392740 1 -- 172.21.15.94:0/3482863765 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726a8250 msgr2=0x55d5726ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.279+0000 7f351d392740 1 --2- 172.21.15.94:0/3482863765 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726a8250 0x55d5726ae7e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f350409a5f0 tx=0x7f350409a250 comp rx=0 tx=0).stop 2024-02-17T12:06:42.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.279+0000 7f351d392740 1 -- 172.21.15.94:0/3482863765 shutdown_connections 2024-02-17T12:06:42.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.279+0000 7f351d392740 1 --2- 172.21.15.94:0/3482863765 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d5726c1c30 0x55d5726c4020 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.279+0000 7f34f37fe640 0 client.27230 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:06:42.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.279+0000 7f351d392740 1 --2- 172.21.15.94:0/3482863765 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726b1750 0x55d5726af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.279+0000 7f351d392740 1 --2- 172.21.15.94:0/3482863765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8b70 0x55d5726aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.279+0000 7f351d392740 1 --2- 172.21.15.94:0/3482863765 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726a8250 0x55d5726ae7e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.279+0000 7f351d392740 1 -- 172.21.15.94:0/3482863765 >> 172.21.15.94:0/3482863765 conn(0x55d5726a2980 msgr2=0x55d5726a46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.279+0000 7f351d392740 1 -- 172.21.15.94:0/3482863765 shutdown_connections 2024-02-17T12:06:42.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.279+0000 7f351d392740 1 -- 172.21.15.94:0/3482863765 wait complete. 2024-02-17T12:06:42.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.279+0000 7f351d392740 20 client.27230 trim_cache size 0 max 16384 2024-02-17T12:06:42.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f351d392740 1 Processor -- start 2024-02-17T12:06:42.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f351d392740 1 -- start start 2024-02-17T12:06:42.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f351d392740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726a8250 0x55d5726af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f351d392740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f351d392740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726b1750 0x55d5726b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f351d392740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d572595130 con 0x55d5726a8b70 2024-02-17T12:06:42.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f351d392740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5726a7c90 con 0x55d5726a8250 2024-02-17T12:06:42.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f35137fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f3513fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726a8250 0x55d5726af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f35137fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726a44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53814/0 (socket says 172.21.15.94:53814) 2024-02-17T12:06:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f35137fe640 1 -- 172.21.15.94:0/2249307714 learned_addr learned my addr 172.21.15.94:0/2249307714 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f351d392740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5726a7650 con 0x55d5726b1750 2024-02-17T12:06:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f3518c7c640 1 --2- 172.21.15.94:0/2249307714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726b1750 0x55d5726b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f35137fe640 1 -- 172.21.15.94:0/2249307714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726b1750 msgr2=0x55d5726b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f35137fe640 1 --2- 172.21.15.94:0/2249307714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726b1750 0x55d5726b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f35137fe640 1 -- 172.21.15.94:0/2249307714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726a8250 msgr2=0x55d5726af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.297+0000 7f35137fe640 1 --2- 172.21.15.94:0/2249307714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726a8250 0x55d5726af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.298+0000 7f35137fe640 1 -- 172.21.15.94:0/2249307714 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5725aac20 con 0x55d5726a8b70 2024-02-17T12:06:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.298+0000 7f35137fe640 1 --2- 172.21.15.94:0/2249307714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726a44c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f3508009900 tx=0x7f350802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e688959188086f2a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.298+0000 7f34f9ffb640 1 -- 172.21.15.94:0/2249307714 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350804b040 con 0x55d5726a8b70 2024-02-17T12:06:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.298+0000 7f34f9ffb640 1 -- 172.21.15.94:0/2249307714 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3508004ae0 con 0x55d5726a8b70 2024-02-17T12:06:42.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.298+0000 7f34f9ffb640 1 -- 172.21.15.94:0/2249307714 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508005420 con 0x55d5726a8b70 2024-02-17T12:06:42.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.299+0000 7f351d392740 1 -- 172.21.15.94:0/2249307714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 msgr2=0x55d5726a44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.299+0000 7f351d392740 1 --2- 172.21.15.94:0/2249307714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726a44c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f3508009900 tx=0x7f350802e040 comp rx=0 tx=0).stop 2024-02-17T12:06:42.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.299+0000 7f351d392740 1 -- 172.21.15.94:0/2249307714 shutdown_connections 2024-02-17T12:06:42.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.299+0000 7f351d392740 1 --2- 172.21.15.94:0/2249307714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726b1750 0x55d5726b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.299+0000 7f351d392740 1 --2- 172.21.15.94:0/2249307714 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726a44c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.299+0000 7f351d392740 1 --2- 172.21.15.94:0/2249307714 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726a8250 0x55d5726af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.300+0000 7f351d392740 1 -- 172.21.15.94:0/2249307714 >> 172.21.15.94:0/2249307714 conn(0x55d5726a2980 msgr2=0x55d5726a3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.300+0000 7f351d392740 1 -- 172.21.15.94:0/2249307714 shutdown_connections 2024-02-17T12:06:42.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.300+0000 7f351d392740 1 -- 172.21.15.94:0/2249307714 wait complete. 2024-02-17T12:06:42.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.300+0000 7f351d392740 1 -- start start 2024-02-17T12:06:42.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f351d392740 1 Processor -- start 2024-02-17T12:06:42.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f351d392740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8250 0x55d5726b7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f351d392740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f351d392740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726b1750 0x55d5726b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f351d392740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d572718130 con 0x55d5726a8b70 2024-02-17T12:06:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f351d392740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d572718270 con 0x55d5726b1750 2024-02-17T12:06:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f351d392740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5727183b0 con 0x55d5726a8250 2024-02-17T12:06:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f35137fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f35137fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726b75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53830/0 (socket says 172.21.15.94:53830) 2024-02-17T12:06:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f35137fe640 1 -- 172.21.15.94:0/3047519220 learned_addr learned my addr 172.21.15.94:0/3047519220 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f3513fff640 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8250 0x55d5726b7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f351d392740 10 client.0 _ll_register_callbacks cb 0x7f3508009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f3518c7c640 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726b1750 0x55d5726b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f35137fe640 1 -- 172.21.15.94:0/3047519220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8250 msgr2=0x55d5726b7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f35137fe640 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8250 0x55d5726b7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f35137fe640 1 -- 172.21.15.94:0/3047519220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726b1750 msgr2=0x55d5726b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f35137fe640 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726b1750 0x55d5726b7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f35137fe640 1 -- 172.21.15.94:0/3047519220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5725d2060 con 0x55d5726a8b70 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.301+0000 7f3518c7c640 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726b1750 0x55d5726b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f35137fe640 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726b75a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3508005ab0 tx=0x7f3508005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21bad1a05981195 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f34f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f34f37fe640 1 -- 172.21.15.94:0/3047519220 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350804b070 con 0x55d5726a8b70 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f34f37fe640 1 -- 172.21.15.94:0/3047519220 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3508000e40 con 0x55d5726a8b70 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f351d392740 1 -- 172.21.15.94:0/3047519220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3508004510 con 0x55d5726a8b70 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f34f37fe640 1 -- 172.21.15.94:0/3047519220 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3508004030 con 0x55d5726a8b70 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f351d392740 10 client.17769 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f351d392740 1 -- 172.21.15.94:0/3047519220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d5725d08c0 con 0x55d5726a8b70 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f34f37fe640 1 -- 172.21.15.94:0/3047519220 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3508069080 con 0x55d5726a8b70 2024-02-17T12:06:42.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f34f37fe640 1 -- 172.21.15.94:0/3047519220 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f350803c040 con 0x55d5726a8b70 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f351d392740 10 client.17769 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f351d392740 10 client.17769 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f351d392740 20 client.17769 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f351d392740 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d572719810 0x55d57271bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f351d392740 4 client.17769 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.302+0000 7f351d392740 1 -- 172.21.15.94:0/3047519220 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d57239b890 con 0x55d572719810 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.303+0000 7f3513fff640 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d572719810 0x55d57271bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.303+0000 7f3513fff640 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d572719810 0x55d57271bc30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df7a16d112fb42c8 server_cookie=ac47cc608a59334d in_seq=0 out_seq=0 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.303+0000 7f34f37fe640 10 client.17769 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.303+0000 7f34f37fe640 1 -- 172.21.15.94:0/3047519220 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d57239b890 con 0x55d572719810 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.303+0000 7f34f37fe640 10 client.17769 handle_command_reply: tid=0 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.303+0000 7f351d392740 1 client.17769 shutdown 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.303+0000 7f351d392740 1 -- 172.21.15.94:0/3047519220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 msgr2=0x55d5726b75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.303+0000 7f351d392740 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726b75a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3508005ab0 tx=0x7f3508005030 comp rx=0 tx=0).stop 2024-02-17T12:06:42.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.304+0000 7f351d392740 1 -- 172.21.15.94:0/3047519220 shutdown_connections 2024-02-17T12:06:42.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.304+0000 7f351d392740 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d572719810 0x55d57271bc30 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.304+0000 7f351d392740 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d5726b1750 0x55d5726b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.304+0000 7f351d392740 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d5726a8b70 0x55d5726b75a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.304+0000 7f351d392740 1 --2- 172.21.15.94:0/3047519220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d5726a8250 0x55d5726b7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.304+0000 7f34f37fe640 0 client.17769 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:06:42.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.304+0000 7f351d392740 1 -- 172.21.15.94:0/3047519220 >> 172.21.15.94:0/3047519220 conn(0x55d5726a2980 msgr2=0x55d5726a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.304+0000 7f351d392740 1 -- 172.21.15.94:0/3047519220 shutdown_connections 2024-02-17T12:06:42.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.304+0000 7f351d392740 1 -- 172.21.15.94:0/3047519220 wait complete. 2024-02-17T12:06:42.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.304+0000 7f351d392740 20 client.17769 trim_cache size 0 max 16384 2024-02-17T12:06:42.310 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:06:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.305+0000 7f351a6df640 1 -- 172.21.15.94:0/1642871414 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f34fc0a22b0 msgr2=0x7f34fc0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.305+0000 7f351a6df640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f34fc0a22b0 0x7f34fc0a4770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f350c007d80 tx=0x7f350c007430 comp rx=0 tx=0).stop 2024-02-17T12:06:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.305+0000 7f351a6df640 1 -- 172.21.15.94:0/1642871414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f351410d9d0 msgr2=0x7f3514117e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.305+0000 7f351a6df640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f351410d9d0 0x7f3514117e00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f350400b450 tx=0x7f350400b910 comp rx=0 tx=0).stop 2024-02-17T12:06:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.305+0000 7f351a6df640 1 -- 172.21.15.94:0/1642871414 shutdown_connections 2024-02-17T12:06:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.305+0000 7f351a6df640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f34fc0a22b0 0x7f34fc0a4770 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.305+0000 7f351a6df640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f351411a960 0x7f3514110ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.305+0000 7f351a6df640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f351410f9a0 0x7f3514118340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.305+0000 7f351a6df640 1 --2- 172.21.15.94:0/1642871414 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f351410d9d0 0x7f3514117e00 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.305+0000 7f351a6df640 1 -- 172.21.15.94:0/1642871414 >> 172.21.15.94:0/1642871414 conn(0x7f351406c260 msgr2=0x7f3514071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.305+0000 7f351a6df640 1 -- 172.21.15.94:0/1642871414 shutdown_connections 2024-02-17T12:06:42.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.306+0000 7f351a6df640 1 -- 172.21.15.94:0/1642871414 wait complete. 2024-02-17T12:06:42.319 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:06:42.319 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:06:42.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:41 smithi189 ceph-mon[36588]: pgmap v264: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-02-17T12:06:42.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:41 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3334629395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:06:42.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:41 smithi125 ceph-mon[36819]: pgmap v264: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.4 MiB/s wr, 569 op/s 2024-02-17T12:06:42.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:41 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3334629395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:06:42.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.455+0000 7f938ef3f640 1 -- 172.21.15.94:0/2316738493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9388108080 msgr2=0x7f9388114500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.455+0000 7f938ef3f640 1 --2- 172.21.15.94:0/2316738493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9388108080 0x7f9388114500 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9380009cd0 tx=0x7f938002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.455+0000 7f938ef3f640 1 -- 172.21.15.94:0/2316738493 shutdown_connections 2024-02-17T12:06:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.455+0000 7f938ef3f640 1 --2- 172.21.15.94:0/2316738493 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9388108080 0x7f9388114500 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.455+0000 7f938ef3f640 1 --2- 172.21.15.94:0/2316738493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93881076c0 0x7f9388107b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.455+0000 7f938ef3f640 1 --2- 172.21.15.94:0/2316738493 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f938810d640 0x7f938810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.455+0000 7f938ef3f640 1 -- 172.21.15.94:0/2316738493 >> 172.21.15.94:0/2316738493 conn(0x7f9388076ed0 msgr2=0x7f9388105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.455+0000 7f938ef3f640 1 -- 172.21.15.94:0/2316738493 shutdown_connections 2024-02-17T12:06:42.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.455+0000 7f938ef3f640 1 -- 172.21.15.94:0/2316738493 wait complete. 2024-02-17T12:06:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.456+0000 7f938ef3f640 1 Processor -- start 2024-02-17T12:06:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.456+0000 7f938ef3f640 1 -- start start 2024-02-17T12:06:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.456+0000 7f938ef3f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93881076c0 0x7f93881a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.456+0000 7f938ef3f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9388108080 0x7f93881a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938ef3f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f938810d640 0x7f938819da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938ef3f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938819e0d0 con 0x7f938810d640 2024-02-17T12:06:42.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938ef3f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938819e240 con 0x7f9388108080 2024-02-17T12:06:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938ef3f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938819e400 con 0x7f93881076c0 2024-02-17T12:06:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938df3d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93881076c0 0x7f93881a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938df3d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93881076c0 0x7f93881a3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:48534/0 (socket says 172.21.15.94:48534) 2024-02-17T12:06:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938df3d640 1 -- 172.21.15.94:0/2005084198 learned_addr learned my addr 172.21.15.94:0/2005084198 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938d73c640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9388108080 0x7f93881a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938e73e640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f938810d640 0x7f938819da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938d73c640 1 -- 172.21.15.94:0/2005084198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93881076c0 msgr2=0x7f93881a3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938d73c640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93881076c0 0x7f93881a3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938d73c640 1 -- 172.21.15.94:0/2005084198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f938810d640 msgr2=0x7f938819da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938d73c640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f938810d640 0x7f938819da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.457+0000 7f938d73c640 1 -- 172.21.15.94:0/2005084198 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9380009900 con 0x7f9388108080 2024-02-17T12:06:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.458+0000 7f938d73c640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9388108080 0x7f93881a3dc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9378005fd0 tx=0x7f937800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.458+0000 7f937effd640 1 -- 172.21.15.94:0/2005084198 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378009c30 con 0x7f9388108080 2024-02-17T12:06:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.458+0000 7f937effd640 1 -- 172.21.15.94:0/2005084198 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9378004510 con 0x7f9388108080 2024-02-17T12:06:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.458+0000 7f937effd640 1 -- 172.21.15.94:0/2005084198 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378003e60 con 0x7f9388108080 2024-02-17T12:06:42.460 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.458+0000 7f938ef3f640 1 -- 172.21.15.94:0/2005084198 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938819e6e0 con 0x7f9388108080 2024-02-17T12:06:42.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.458+0000 7f938ef3f640 1 -- 172.21.15.94:0/2005084198 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93881aa920 con 0x7f9388108080 2024-02-17T12:06:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.461+0000 7f937effd640 1 -- 172.21.15.94:0/2005084198 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9378010460 con 0x7f9388108080 2024-02-17T12:06:42.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.461+0000 7f9392d79740 1 Processor -- start 2024-02-17T12:06:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.461+0000 7f9392d79740 1 -- start start 2024-02-17T12:06:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.462+0000 7f937effd640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f93740a22b0 0x7f93740a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.462+0000 7f937effd640 1 -- 172.21.15.94:0/2005084198 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f93780cdb60 con 0x7f9388108080 2024-02-17T12:06:42.464 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.462+0000 7f9392d79740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 0x55e04fa71820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.462+0000 7f9392d79740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71d60 0x55e04fa7a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.462+0000 7f9392d79740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa7a940 0x55e04fa7cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.462+0000 7f9392d79740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04fa2c960 con 0x55e04fa7a940 2024-02-17T12:06:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.462+0000 7f9392d79740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f9e0310 con 0x55e04fa71440 2024-02-17T12:06:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.462+0000 7f9392d79740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f9dd840 con 0x55e04fa71d60 2024-02-17T12:06:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.463+0000 7f938e73e640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f93740a22b0 0x7f93740a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.463+0000 7f938df3d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 0x55e04fa71820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.463+0000 7f938df3d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 0x55e04fa71820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52612/0 (socket says 172.21.15.94:52612) 2024-02-17T12:06:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.463+0000 7f938df3d640 1 -- 172.21.15.94:0/2446915828 learned_addr learned my addr 172.21.15.94:0/2446915828 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.463+0000 7f938e73e640 1 --2- 172.21.15.94:0/2446915828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa7a940 0x55e04fa7cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.463+0000 7f938df3d640 1 -- 172.21.15.94:0/2446915828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71d60 msgr2=0x55e04fa7a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.463+0000 7f938df3d640 1 --2- 172.21.15.94:0/2446915828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71d60 0x55e04fa7a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.463+0000 7f938df3d640 1 -- 172.21.15.94:0/2446915828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa7a940 msgr2=0x55e04fa7cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.463+0000 7f938df3d640 1 --2- 172.21.15.94:0/2446915828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa7a940 0x55e04fa7cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.463+0000 7f938df3d640 1 -- 172.21.15.94:0/2446915828 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e04f99b060 con 0x55e04fa71440 2024-02-17T12:06:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.463+0000 7f938df3d640 1 --2- 172.21.15.94:0/2446915828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 0x55e04fa71820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f93840045b0 tx=0x7f938400bc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e7bdd5eb215fb62 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.464+0000 7f938e73e640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f93740a22b0 0x7f93740a4770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9380005cc0 tx=0x7f9380005f10 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.464+0000 7f9373fff640 1 -- 172.21.15.94:0/2446915828 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384003dd0 con 0x55e04fa71440 2024-02-17T12:06:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.464+0000 7f9373fff640 1 -- 172.21.15.94:0/2446915828 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f938400be60 con 0x55e04fa71440 2024-02-17T12:06:42.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.464+0000 7f9373fff640 1 -- 172.21.15.94:0/2446915828 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384011860 con 0x55e04fa71440 2024-02-17T12:06:42.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.464+0000 7f9392d79740 1 -- 172.21.15.94:0/2446915828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 msgr2=0x55e04fa71820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.464+0000 7f9392d79740 1 --2- 172.21.15.94:0/2446915828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 0x55e04fa71820 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f93840045b0 tx=0x7f938400bc70 comp rx=0 tx=0).stop 2024-02-17T12:06:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 -- 172.21.15.94:0/2446915828 shutdown_connections 2024-02-17T12:06:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 --2- 172.21.15.94:0/2446915828 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa7a940 0x55e04fa7cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 --2- 172.21.15.94:0/2446915828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71d60 0x55e04fa7a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 --2- 172.21.15.94:0/2446915828 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 0x55e04fa71820 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 -- 172.21.15.94:0/2446915828 >> 172.21.15.94:0/2446915828 conn(0x55e04fa6bb70 msgr2=0x55e04fa6bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 -- 172.21.15.94:0/2446915828 shutdown_connections 2024-02-17T12:06:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 -- 172.21.15.94:0/2446915828 wait complete. 2024-02-17T12:06:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 -- start start 2024-02-17T12:06:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 Processor -- start 2024-02-17T12:06:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71440 0x55e04fa779b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71d60 0x55e04fa77ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa7a940 0x55e04fa78430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f9d9020 con 0x55e04fa71d60 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f6ac430 con 0x55e04fa7a940 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f668e00 con 0x55e04fa71440 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.465+0000 7f9392d79740 10 client.0 _ll_register_callbacks cb 0x55e04f980cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f938d73c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71d60 0x55e04fa77ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f938e73e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa7a940 0x55e04fa78430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f938d73c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71d60 0x55e04fa77ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53862/0 (socket says 172.21.15.94:53862) 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f938e73e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa7a940 0x55e04fa78430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52628/0 (socket says 172.21.15.94:52628) 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f938d73c640 1 -- 172.21.15.94:0/619351393 learned_addr learned my addr 172.21.15.94:0/619351393 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f938df3d640 1 --2- 172.21.15.94:0/619351393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71440 0x55e04fa779b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f938e73e640 1 -- 172.21.15.94:0/619351393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71440 msgr2=0x55e04fa779b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f938e73e640 1 --2- 172.21.15.94:0/619351393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71440 0x55e04fa779b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f938e73e640 1 -- 172.21.15.94:0/619351393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71d60 msgr2=0x55e04fa77ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f938e73e640 1 --2- 172.21.15.94:0/619351393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71d60 0x55e04fa77ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f938e73e640 1 -- 172.21.15.94:0/619351393 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9384004210 con 0x55e04fa7a940 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f938e73e640 1 --2- 172.21.15.94:0/619351393 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa7a940 0x55e04fa78430 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9380037040 tx=0x7f9380005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3bf1da0616ac986 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f93717fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f93717fa640 1 -- 172.21.15.94:0/619351393 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938004a070 con 0x55e04fa7a940 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f93717fa640 1 -- 172.21.15.94:0/619351393 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f93800328c0 con 0x55e04fa7a940 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.466+0000 7f9392d79740 1 -- 172.21.15.94:0/619351393 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e04f6d87e0 con 0x55e04fa7a940 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.467+0000 7f93717fa640 1 -- 172.21.15.94:0/619351393 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9380032a80 con 0x55e04fa7a940 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.467+0000 7f9392d79740 10 client.27248 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.467+0000 7f9392d79740 1 -- 172.21.15.94:0/619351393 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e04f9d76a0 con 0x55e04fa7a940 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.467+0000 7f93717fa640 1 -- 172.21.15.94:0/619351393 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e04f6d87e0 con 0x55e04fa7a940 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.467+0000 7f93717fa640 1 -- 172.21.15.94:0/619351393 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9380004680 con 0x55e04fa7a940 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.467+0000 7f9392d79740 10 client.27248 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.467+0000 7f9392d79740 10 client.27248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:42.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.467+0000 7f9392d79740 20 client.27248 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:42.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.467+0000 7f9392d79740 1 --2- 172.21.15.94:0/619351393 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e04fa8adc0 0x55e04fa8d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.467+0000 7f9392d79740 4 client.27248 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:42.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.467+0000 7f9392d79740 1 -- 172.21.15.94:0/619351393 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e04f715720 con 0x55e04fa8adc0 2024-02-17T12:06:42.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.468+0000 7f938df3d640 1 --2- 172.21.15.94:0/619351393 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e04fa8adc0 0x55e04fa8d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.468+0000 7f93717fa640 10 client.27248 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:42.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.468+0000 7f938df3d640 1 --2- 172.21.15.94:0/619351393 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e04fa8adc0 0x55e04fa8d1e0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7431ffeeac6cc5df server_cookie=1cbc208f130b371c in_seq=0 out_seq=0 2024-02-17T12:06:42.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.469+0000 7f93717fa640 1 -- 172.21.15.94:0/619351393 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e04f715720 con 0x55e04fa8adc0 2024-02-17T12:06:42.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.469+0000 7f93717fa640 10 client.27248 handle_command_reply: tid=0 2024-02-17T12:06:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.469+0000 7f9392d79740 1 client.27248 shutdown 2024-02-17T12:06:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.469+0000 7f9392d79740 1 -- 172.21.15.94:0/619351393 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa7a940 msgr2=0x55e04fa78430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.469+0000 7f9392d79740 1 --2- 172.21.15.94:0/619351393 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa7a940 0x55e04fa78430 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9380037040 tx=0x7f9380005c50 comp rx=0 tx=0).stop 2024-02-17T12:06:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.469+0000 7f9392d79740 1 -- 172.21.15.94:0/619351393 shutdown_connections 2024-02-17T12:06:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.469+0000 7f9392d79740 1 --2- 172.21.15.94:0/619351393 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e04fa8adc0 0x55e04fa8d1e0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.469+0000 7f9392d79740 1 --2- 172.21.15.94:0/619351393 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa7a940 0x55e04fa78430 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.469+0000 7f9392d79740 1 --2- 172.21.15.94:0/619351393 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71d60 0x55e04fa77ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.469+0000 7f9392d79740 1 --2- 172.21.15.94:0/619351393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71440 0x55e04fa779b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.469+0000 7f9392d79740 1 -- 172.21.15.94:0/619351393 >> 172.21.15.94:0/619351393 conn(0x55e04fa6bb70 msgr2=0x55e04fa6d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.469+0000 7f93717fa640 0 client.27248 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.469+0000 7f9392d79740 1 -- 172.21.15.94:0/619351393 shutdown_connections 2024-02-17T12:06:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.470+0000 7f9392d79740 1 -- 172.21.15.94:0/619351393 wait complete. 2024-02-17T12:06:42.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.470+0000 7f9392d79740 20 client.27248 trim_cache size 0 max 16384 2024-02-17T12:06:42.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.490+0000 7f9392d79740 1 Processor -- start 2024-02-17T12:06:42.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.490+0000 7f9392d79740 1 -- start start 2024-02-17T12:06:42.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f9392d79740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71440 0x55e04fa7a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f9392d79740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71d60 0x55e04fa78250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f9392d79740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa7a940 0x55e04fa762c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f9392d79740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04f95e130 con 0x55e04fa71440 2024-02-17T12:06:42.493 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f9392d79740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04fa70e80 con 0x55e04fa7a940 2024-02-17T12:06:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f9392d79740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04fa6cb70 con 0x55e04fa71d60 2024-02-17T12:06:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f938df3d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71440 0x55e04fa7a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f938df3d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71440 0x55e04fa7a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53868/0 (socket says 172.21.15.94:53868) 2024-02-17T12:06:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f938df3d640 1 -- 172.21.15.94:0/2799576574 learned_addr learned my addr 172.21.15.94:0/2799576574 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f938d73c640 1 --2- 172.21.15.94:0/2799576574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71d60 0x55e04fa78250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f938e73e640 1 --2- 172.21.15.94:0/2799576574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa7a940 0x55e04fa762c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f938df3d640 1 -- 172.21.15.94:0/2799576574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71d60 msgr2=0x55e04fa78250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f938df3d640 1 --2- 172.21.15.94:0/2799576574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71d60 0x55e04fa78250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f938df3d640 1 -- 172.21.15.94:0/2799576574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa7a940 msgr2=0x55e04fa762c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f938df3d640 1 --2- 172.21.15.94:0/2799576574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa7a940 0x55e04fa762c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f938df3d640 1 -- 172.21.15.94:0/2799576574 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e04f973c20 con 0x55e04fa71440 2024-02-17T12:06:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.491+0000 7f938df3d640 1 --2- 172.21.15.94:0/2799576574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71440 0x55e04fa7a0e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9384000c00 tx=0x7f9384005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c311ec940f41775d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.492+0000 7f93737fe640 1 -- 172.21.15.94:0/2799576574 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938401e040 con 0x55e04fa71440 2024-02-17T12:06:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.492+0000 7f93737fe640 1 -- 172.21.15.94:0/2799576574 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f938401f040 con 0x55e04fa71440 2024-02-17T12:06:42.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.492+0000 7f93737fe640 1 -- 172.21.15.94:0/2799576574 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938402b710 con 0x55e04fa71440 2024-02-17T12:06:42.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.493+0000 7f9392d79740 1 -- 172.21.15.94:0/2799576574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71440 msgr2=0x55e04fa7a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.493+0000 7f9392d79740 1 --2- 172.21.15.94:0/2799576574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71440 0x55e04fa7a0e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9384000c00 tx=0x7f9384005fb0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.493+0000 7f9392d79740 1 -- 172.21.15.94:0/2799576574 shutdown_connections 2024-02-17T12:06:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.493+0000 7f9392d79740 1 --2- 172.21.15.94:0/2799576574 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa7a940 0x55e04fa762c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.493+0000 7f9392d79740 1 --2- 172.21.15.94:0/2799576574 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa71d60 0x55e04fa78250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.493+0000 7f9392d79740 1 --2- 172.21.15.94:0/2799576574 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71440 0x55e04fa7a0e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.493+0000 7f9392d79740 1 -- 172.21.15.94:0/2799576574 >> 172.21.15.94:0/2799576574 conn(0x55e04fa6bb70 msgr2=0x55e04fa72d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.493+0000 7f9392d79740 1 -- 172.21.15.94:0/2799576574 shutdown_connections 2024-02-17T12:06:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.493+0000 7f9392d79740 1 -- 172.21.15.94:0/2799576574 wait complete. 2024-02-17T12:06:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.493+0000 7f9392d79740 1 -- start start 2024-02-17T12:06:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.493+0000 7f9392d79740 1 Processor -- start 2024-02-17T12:06:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.494+0000 7f9392d79740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 0x55e04fa81a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.494+0000 7f9392d79740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71d60 0x55e04fa7fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.494+0000 7f9392d79740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa7a940 0x55e04fa80320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.494+0000 7f938df3d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 0x55e04fa81a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.494+0000 7f938df3d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 0x55e04fa81a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52642/0 (socket says 172.21.15.94:52642) 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.494+0000 7f938df3d640 1 -- 172.21.15.94:0/1874447101 learned_addr learned my addr 172.21.15.94:0/1874447101 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.494+0000 7f9392d79740 1 -- 172.21.15.94:0/1874447101 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04fa808e0 con 0x55e04fa71d60 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.494+0000 7f9392d79740 1 -- 172.21.15.94:0/1874447101 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04fa80a50 con 0x55e04fa71440 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.494+0000 7f9392d79740 1 -- 172.21.15.94:0/1874447101 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04fa80c10 con 0x55e04fa7a940 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.494+0000 7f9392d79740 10 client.0 _ll_register_callbacks cb 0x7f9384004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.494+0000 7f938e73e640 1 --2- 172.21.15.94:0/1874447101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa7a940 0x55e04fa80320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.495+0000 7f938e73e640 1 -- 172.21.15.94:0/1874447101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 msgr2=0x55e04fa81a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.495+0000 7f938e73e640 1 --2- 172.21.15.94:0/1874447101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 0x55e04fa81a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.495+0000 7f938e73e640 1 -- 172.21.15.94:0/1874447101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71d60 msgr2=0x55e04fa7fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.495+0000 7f938e73e640 1 --2- 172.21.15.94:0/1874447101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71d60 0x55e04fa7fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.495+0000 7f938e73e640 1 -- 172.21.15.94:0/1874447101 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e04f99b060 con 0x55e04fa7a940 2024-02-17T12:06:42.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.495+0000 7f938df3d640 1 --2- 172.21.15.94:0/1874447101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 0x55e04fa81a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.495+0000 7f938e73e640 1 --2- 172.21.15.94:0/1874447101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa7a940 0x55e04fa80320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f938002c630 tx=0x7f9380032f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e80d5a2c69049687 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.495+0000 7f93717fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.495+0000 7f93717fa640 1 -- 172.21.15.94:0/1874447101 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938004a040 con 0x55e04fa7a940 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.495+0000 7f9392d79740 1 -- 172.21.15.94:0/1874447101 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f938400be60 con 0x55e04fa7a940 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.495+0000 7f93717fa640 1 -- 172.21.15.94:0/1874447101 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f938004fbc0 con 0x55e04fa7a940 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.496+0000 7f93717fa640 1 -- 172.21.15.94:0/1874447101 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f938004fd80 con 0x55e04fa7a940 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.496+0000 7f9392d79740 10 client.27274 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.496+0000 7f9392d79740 1 -- 172.21.15.94:0/1874447101 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e04f9998c0 con 0x55e04fa7a940 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.496+0000 7f93717fa640 1 -- 172.21.15.94:0/1874447101 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f938006c080 con 0x55e04fa7a940 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.496+0000 7f93717fa640 1 -- 172.21.15.94:0/1874447101 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9380044440 con 0x55e04fa7a940 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.496+0000 7f9392d79740 10 client.27274 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.496+0000 7f9392d79740 10 client.27274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.496+0000 7f9392d79740 20 client.27274 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:42.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.496+0000 7f9392d79740 1 --2- 172.21.15.94:0/1874447101 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e04fae25f0 0x55e04fae4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.496+0000 7f9392d79740 4 client.27274 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.496+0000 7f9392d79740 1 -- 172.21.15.94:0/1874447101 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e04f764890 con 0x55e04fae25f0 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.496+0000 7f938df3d640 1 --2- 172.21.15.94:0/1874447101 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e04fae25f0 0x55e04fae4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f93717fa640 10 client.27274 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f938df3d640 1 --2- 172.21.15.94:0/1874447101 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e04fae25f0 0x55e04fae4a30 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c64390fbe2c6fdf server_cookie=ddfeae397ec93d52 in_seq=0 out_seq=0 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f93717fa640 1 -- 172.21.15.94:0/1874447101 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e04f764890 con 0x55e04fae25f0 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f93717fa640 10 client.27274 handle_command_reply: tid=0 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f9392d79740 1 client.27274 shutdown 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f9392d79740 1 -- 172.21.15.94:0/1874447101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa7a940 msgr2=0x55e04fa80320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f9392d79740 1 --2- 172.21.15.94:0/1874447101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa7a940 0x55e04fa80320 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f938002c630 tx=0x7f9380032f00 comp rx=0 tx=0).stop 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f9392d79740 1 -- 172.21.15.94:0/1874447101 shutdown_connections 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f9392d79740 1 --2- 172.21.15.94:0/1874447101 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e04fae25f0 0x55e04fae4a30 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f9392d79740 1 --2- 172.21.15.94:0/1874447101 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e04fa7a940 0x55e04fa80320 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f93717fa640 0 client.27274 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f9392d79740 1 --2- 172.21.15.94:0/1874447101 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e04fa71d60 0x55e04fa7fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f9392d79740 1 --2- 172.21.15.94:0/1874447101 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e04fa71440 0x55e04fa81a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f9392d79740 1 -- 172.21.15.94:0/1874447101 >> 172.21.15.94:0/1874447101 conn(0x55e04fa6bb70 msgr2=0x55e04fa761d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f9392d79740 1 -- 172.21.15.94:0/1874447101 shutdown_connections 2024-02-17T12:06:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f9392d79740 1 -- 172.21.15.94:0/1874447101 wait complete. 2024-02-17T12:06:42.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.497+0000 7f9392d79740 20 client.27274 trim_cache size 0 max 16384 2024-02-17T12:06:42.503 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:06:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.498+0000 7f938ef3f640 1 -- 172.21.15.94:0/2005084198 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f93740a22b0 msgr2=0x7f93740a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.498+0000 7f938ef3f640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f93740a22b0 0x7f93740a4770 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9380005cc0 tx=0x7f9380005f10 comp rx=0 tx=0).stop 2024-02-17T12:06:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.498+0000 7f938ef3f640 1 -- 172.21.15.94:0/2005084198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9388108080 msgr2=0x7f93881a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.498+0000 7f938ef3f640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9388108080 0x7f93881a3dc0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f9378005fd0 tx=0x7f937800dde0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.498+0000 7f938ef3f640 1 -- 172.21.15.94:0/2005084198 shutdown_connections 2024-02-17T12:06:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.498+0000 7f938ef3f640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f93740a22b0 0x7f93740a4770 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.498+0000 7f938ef3f640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f938810d640 0x7f938819da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.498+0000 7f938ef3f640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9388108080 0x7f93881a3dc0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.498+0000 7f938ef3f640 1 --2- 172.21.15.94:0/2005084198 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93881076c0 0x7f93881a3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:42.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.498+0000 7f938ef3f640 1 -- 172.21.15.94:0/2005084198 >> 172.21.15.94:0/2005084198 conn(0x7f9388076ed0 msgr2=0x7f9388077490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:42.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.498+0000 7f938ef3f640 1 -- 172.21.15.94:0/2005084198 shutdown_connections 2024-02-17T12:06:42.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:42.498+0000 7f938ef3f640 1 -- 172.21.15.94:0/2005084198 wait complete. 2024-02-17T12:06:43.514 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:06:43.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:06:43.515 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:06:43.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.652+0000 7faeb1481640 1 -- 172.21.15.94:0/3363722883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeac101380 msgr2=0x7faeac101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.652+0000 7faeb1481640 1 --2- 172.21.15.94:0/3363722883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeac101380 0x7faeac101760 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009930 tx=0x7fae9c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:06:43.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 -- 172.21.15.94:0/3363722883 shutdown_connections 2024-02-17T12:06:43.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 --2- 172.21.15.94:0/3363722883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeac106450 0x7faeac10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 --2- 172.21.15.94:0/3363722883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeac101d30 0x7faeac105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 --2- 172.21.15.94:0/3363722883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeac101380 0x7faeac101760 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 -- 172.21.15.94:0/3363722883 >> 172.21.15.94:0/3363722883 conn(0x7faeac078540 msgr2=0x7faeac078950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 -- 172.21.15.94:0/3363722883 shutdown_connections 2024-02-17T12:06:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 -- 172.21.15.94:0/3363722883 wait complete. 2024-02-17T12:06:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 Processor -- start 2024-02-17T12:06:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 -- start start 2024-02-17T12:06:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeac101380 0x7faeac19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeac106450 0x7faeac19daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeac197670 0x7faeac197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac198140 con 0x7faeac106450 2024-02-17T12:06:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac1982b0 con 0x7faeac101380 2024-02-17T12:06:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.653+0000 7faeb1481640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeac198470 con 0x7faeac197670 2024-02-17T12:06:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.654+0000 7faeaaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeac101380 0x7faeac19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.654+0000 7faea3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeac106450 0x7faeac19daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.654+0000 7faeaaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeac101380 0x7faeac19d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52656/0 (socket says 172.21.15.94:52656) 2024-02-17T12:06:43.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.654+0000 7faeaaffd640 1 -- 172.21.15.94:0/1110689480 learned_addr learned my addr 172.21.15.94:0/1110689480 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:43.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.654+0000 7faeab7fe640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeac197670 0x7faeac197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.654+0000 7faeaaffd640 1 -- 172.21.15.94:0/1110689480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeac197670 msgr2=0x7faeac197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.654+0000 7faeaaffd640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeac197670 0x7faeac197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.654+0000 7faeaaffd640 1 -- 172.21.15.94:0/1110689480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeac106450 msgr2=0x7faeac19daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.654+0000 7faeaaffd640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeac106450 0x7faeac19daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.654+0000 7faeaaffd640 1 -- 172.21.15.94:0/1110689480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c009560 con 0x7faeac101380 2024-02-17T12:06:43.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.654+0000 7faeab7fe640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeac197670 0x7faeac197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:06:43.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.654+0000 7faea3fff640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeac106450 0x7faeac19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:43.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.655+0000 7faeaaffd640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeac101380 0x7faeac19d5b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009230 tx=0x7fae9c02c6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:43.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.655+0000 7faea8ff9640 1 -- 172.21.15.94:0/1110689480 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c004350 con 0x7faeac101380 2024-02-17T12:06:43.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.655+0000 7faeb1481640 1 -- 172.21.15.94:0/1110689480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeac073410 con 0x7faeac101380 2024-02-17T12:06:43.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.655+0000 7faeb1481640 1 -- 172.21.15.94:0/1110689480 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac073970 con 0x7faeac101380 2024-02-17T12:06:43.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.655+0000 7faea8ff9640 1 -- 172.21.15.94:0/1110689480 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae9c0044b0 con 0x7faeac101380 2024-02-17T12:06:43.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.655+0000 7faea8ff9640 1 -- 172.21.15.94:0/1110689480 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c003d70 con 0x7faeac101380 2024-02-17T12:06:43.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.658+0000 7faeb4052740 1 Processor -- start 2024-02-17T12:06:43.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.658+0000 7faeb4052740 1 -- start start 2024-02-17T12:06:43.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.658+0000 7faea8ff9640 1 -- 172.21.15.94:0/1110689480 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fae9c004df0 con 0x7faeac101380 2024-02-17T12:06:43.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.658+0000 7faeb4052740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04f380 0x56339d04f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.658+0000 7faeb4052740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d04fd30 0x56339d0582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.658+0000 7faeb4052740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d0588b0 0x56339d05ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.658+0000 7faeb4052740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339cfbdf90 con 0x56339d0588b0 2024-02-17T12:06:43.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.658+0000 7faeb4052740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339cfbb4c0 con 0x56339d04fd30 2024-02-17T12:06:43.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.658+0000 7faeb4052740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339cfb6ca0 con 0x56339d04f380 2024-02-17T12:06:43.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.658+0000 7faeaaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d04fd30 0x56339d0582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.658+0000 7faeaaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d04fd30 0x56339d0582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52668/0 (socket says 172.21.15.94:52668) 2024-02-17T12:06:43.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.658+0000 7faeaaffd640 1 -- 172.21.15.94:0/2215869711 learned_addr learned my addr 172.21.15.94:0/2215869711 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:43.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.658+0000 7faea3fff640 1 --2- 172.21.15.94:0/2215869711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d0588b0 0x56339d05ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.659+0000 7faeab7fe640 1 --2- 172.21.15.94:0/2215869711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04f380 0x56339d04f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.659+0000 7faea8ff9640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fae980a2290 0x7fae980a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.659+0000 7faea3fff640 1 -- 172.21.15.94:0/2215869711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04f380 msgr2=0x56339d04f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:43.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.659+0000 7faea3fff640 1 --2- 172.21.15.94:0/2215869711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04f380 0x56339d04f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.659+0000 7faea3fff640 1 -- 172.21.15.94:0/2215869711 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d04fd30 msgr2=0x56339d0582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:43.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.659+0000 7faea3fff640 1 --2- 172.21.15.94:0/2215869711 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d04fd30 0x56339d0582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.659+0000 7faea3fff640 1 -- 172.21.15.94:0/2215869711 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56339cf790b0 con 0x56339d0588b0 2024-02-17T12:06:43.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.659+0000 7faeab7fe640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fae980a2290 0x7fae980a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.659+0000 7faea8ff9640 1 -- 172.21.15.94:0/1110689480 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fae9c0f00d0 con 0x7faeac101380 2024-02-17T12:06:43.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.660+0000 7faeab7fe640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fae980a2290 0x7fae980a4750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faea4005fd0 tx=0x7faea4004ab0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:43.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.660+0000 7faea3fff640 1 --2- 172.21.15.94:0/2215869711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d0588b0 0x56339d05ad00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fae94005fd0 tx=0x7fae94010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4678de1c39df1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:43.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.660+0000 7faea17fa640 1 -- 172.21.15.94:0/2215869711 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94005220 con 0x56339d0588b0 2024-02-17T12:06:43.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.661+0000 7faea17fa640 1 -- 172.21.15.94:0/2215869711 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae94015070 con 0x56339d0588b0 2024-02-17T12:06:43.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.661+0000 7faea17fa640 1 -- 172.21.15.94:0/2215869711 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94011870 con 0x56339d0588b0 2024-02-17T12:06:43.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 -- 172.21.15.94:0/2215869711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d0588b0 msgr2=0x56339d05ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 --2- 172.21.15.94:0/2215869711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d0588b0 0x56339d05ad00 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fae94005fd0 tx=0x7fae94010860 comp rx=0 tx=0).stop 2024-02-17T12:06:43.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 -- 172.21.15.94:0/2215869711 shutdown_connections 2024-02-17T12:06:43.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 --2- 172.21.15.94:0/2215869711 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d0588b0 0x56339d05ad00 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.667 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 --2- 172.21.15.94:0/2215869711 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d04fd30 0x56339d0582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 --2- 172.21.15.94:0/2215869711 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04f380 0x56339d04f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 -- 172.21.15.94:0/2215869711 >> 172.21.15.94:0/2215869711 conn(0x56339d049840 msgr2=0x56339d049c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 -- 172.21.15.94:0/2215869711 shutdown_connections 2024-02-17T12:06:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 -- 172.21.15.94:0/2215869711 wait complete. 2024-02-17T12:06:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 -- start start 2024-02-17T12:06:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 Processor -- start 2024-02-17T12:06:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04f380 0x56339d055740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04fd30 0x56339d055c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d0588b0 0x56339d0561c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339cf5bf40 con 0x56339d04fd30 2024-02-17T12:06:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339cc8a4b0 con 0x56339d0588b0 2024-02-17T12:06:43.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339cf3c670 con 0x56339d04f380 2024-02-17T12:06:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeb4052740 10 client.0 _ll_register_callbacks cb 0x56339cfa9f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faea3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04fd30 0x56339d055c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faea3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04fd30 0x56339d055c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53912/0 (socket says 172.21.15.94:53912) 2024-02-17T12:06:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faea3fff640 1 -- 172.21.15.94:0/3410827517 learned_addr learned my addr 172.21.15.94:0/3410827517 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.662+0000 7faeab7fe640 1 --2- 172.21.15.94:0/3410827517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d0588b0 0x56339d0561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7faeaaffd640 1 --2- 172.21.15.94:0/3410827517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04f380 0x56339d055740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7faea3fff640 1 -- 172.21.15.94:0/3410827517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04f380 msgr2=0x56339d055740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7faea3fff640 1 --2- 172.21.15.94:0/3410827517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04f380 0x56339d055740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7faea3fff640 1 -- 172.21.15.94:0/3410827517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d0588b0 msgr2=0x56339d0561c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7faea3fff640 1 --2- 172.21.15.94:0/3410827517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d0588b0 0x56339d0561c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7faea3fff640 1 -- 172.21.15.94:0/3410827517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9400bc60 con 0x56339d04fd30 2024-02-17T12:06:43.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7faea3fff640 1 --2- 172.21.15.94:0/3410827517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04fd30 0x56339d055c80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fae940042e0 tx=0x7fae94004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afff0aec6252ef95 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7fae927fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7fae927fc640 1 -- 172.21.15.94:0/3410827517 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94003a00 con 0x56339d04fd30 2024-02-17T12:06:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7faeb4052740 1 -- 172.21.15.94:0/3410827517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56339cf90f40 con 0x56339d04fd30 2024-02-17T12:06:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7fae927fc640 1 -- 172.21.15.94:0/3410827517 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fae94017070 con 0x56339d04fd30 2024-02-17T12:06:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7fae927fc640 1 -- 172.21.15.94:0/3410827517 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae94003b60 con 0x56339d04fd30 2024-02-17T12:06:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7faeb4052740 10 client.17796 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.663+0000 7faeb4052740 1 -- 172.21.15.94:0/3410827517 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56339cfb5320 con 0x56339d04fd30 2024-02-17T12:06:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.664+0000 7fae927fc640 1 -- 172.21.15.94:0/3410827517 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56339cf90f40 con 0x56339d04fd30 2024-02-17T12:06:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.664+0000 7fae927fc640 1 -- 172.21.15.94:0/3410827517 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fae9401b430 con 0x56339d04fd30 2024-02-17T12:06:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.664+0000 7faeb4052740 10 client.17796 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.664+0000 7faeb4052740 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:43.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.664+0000 7faeb4052740 20 client.17796 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.664+0000 7faeb4052740 1 --2- 172.21.15.94:0/3410827517 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56339d068c60 0x56339d06b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.664+0000 7faeb4052740 4 client.17796 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.664+0000 7faeb4052740 1 -- 172.21.15.94:0/3410827517 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56339cfad310 con 0x56339d068c60 2024-02-17T12:06:43.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.665+0000 7faeaaffd640 1 --2- 172.21.15.94:0/3410827517 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56339d068c60 0x56339d06b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.671+0000 7fae927fc640 10 client.17796 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:43.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.671+0000 7faeaaffd640 1 --2- 172.21.15.94:0/3410827517 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56339d068c60 0x56339d06b080 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dc98cbebf48cc6b server_cookie=b05ec188f86fceb4 in_seq=0 out_seq=0 2024-02-17T12:06:43.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.672+0000 7fae927fc640 1 -- 172.21.15.94:0/3410827517 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56339cfad310 con 0x56339d068c60 2024-02-17T12:06:43.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.672+0000 7fae927fc640 10 client.17796 handle_command_reply: tid=0 2024-02-17T12:06:43.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.672+0000 7faeb4052740 1 client.17796 shutdown 2024-02-17T12:06:43.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.684+0000 7faeb4052740 1 -- 172.21.15.94:0/3410827517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04fd30 msgr2=0x56339d055c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.684+0000 7faeb4052740 1 --2- 172.21.15.94:0/3410827517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04fd30 0x56339d055c80 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fae940042e0 tx=0x7fae94004310 comp rx=0 tx=0).stop 2024-02-17T12:06:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.694+0000 7faeb4052740 1 -- 172.21.15.94:0/3410827517 shutdown_connections 2024-02-17T12:06:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.694+0000 7faeb4052740 1 --2- 172.21.15.94:0/3410827517 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56339d068c60 0x56339d06b080 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.694+0000 7faeb4052740 1 --2- 172.21.15.94:0/3410827517 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d0588b0 0x56339d0561c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.694+0000 7faeb4052740 1 --2- 172.21.15.94:0/3410827517 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04fd30 0x56339d055c80 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.694+0000 7faeb4052740 1 --2- 172.21.15.94:0/3410827517 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04f380 0x56339d055740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.694+0000 7faeb4052740 1 -- 172.21.15.94:0/3410827517 >> 172.21.15.94:0/3410827517 conn(0x56339d049840 msgr2=0x56339d04b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.694+0000 7fae927fc640 0 client.17796 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:43.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.704+0000 7faeb4052740 1 -- 172.21.15.94:0/3410827517 shutdown_connections 2024-02-17T12:06:43.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.708+0000 7faeb4052740 1 -- 172.21.15.94:0/3410827517 wait complete. 2024-02-17T12:06:43.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.713+0000 7faeb4052740 20 client.17796 trim_cache size 0 max 16384 2024-02-17T12:06:43.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.747+0000 7faeb4052740 1 Processor -- start 2024-02-17T12:06:43.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.748+0000 7faeb4052740 1 -- start start 2024-02-17T12:06:43.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.749+0000 7faeb4052740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d04f380 0x56339d0548a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.749+0000 7faeb4052740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04fd30 0x56339d055fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.749+0000 7faeb4052740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d0588b0 0x56339d048760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.749+0000 7faeb4052740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339d04a640 con 0x56339d0588b0 2024-02-17T12:06:43.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.749+0000 7faeb4052740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339d04ecd0 con 0x56339d04f380 2024-02-17T12:06:43.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.749+0000 7faeb4052740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339d054de0 con 0x56339d04fd30 2024-02-17T12:06:43.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.749+0000 7faeaaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d04f380 0x56339d0548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.749+0000 7faeaaffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d04f380 0x56339d0548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52696/0 (socket says 172.21.15.94:52696) 2024-02-17T12:06:43.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.749+0000 7faeaaffd640 1 -- 172.21.15.94:0/3533110799 learned_addr learned my addr 172.21.15.94:0/3533110799 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:43.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.750+0000 7faeaaffd640 1 -- 172.21.15.94:0/3533110799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04fd30 msgr2=0x56339d055fe0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:06:43.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.750+0000 7faeaaffd640 1 --2- 172.21.15.94:0/3533110799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04fd30 0x56339d055fe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.750+0000 7faeaaffd640 1 -- 172.21.15.94:0/3533110799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d0588b0 msgr2=0x56339d048760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:06:43.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.750+0000 7faeaaffd640 1 --2- 172.21.15.94:0/3533110799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d0588b0 0x56339d048760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.750+0000 7faeaaffd640 1 -- 172.21.15.94:0/3533110799 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56339d04f1f0 con 0x56339d04f380 2024-02-17T12:06:43.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.750+0000 7faeaaffd640 1 --2- 172.21.15.94:0/3533110799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d04f380 0x56339d0548a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0bc890 tx=0x7fae9c0f3be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c53b7bf273b9192 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:43.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.751+0000 7faea0ff9640 1 -- 172.21.15.94:0/3533110799 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c003d70 con 0x56339d04f380 2024-02-17T12:06:43.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.751+0000 7faea0ff9640 1 -- 172.21.15.94:0/3533110799 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fae9c0bc6d0 con 0x56339d04f380 2024-02-17T12:06:43.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.751+0000 7faea0ff9640 1 -- 172.21.15.94:0/3533110799 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0c0ce0 con 0x56339d04f380 2024-02-17T12:06:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.752+0000 7faeb4052740 1 -- 172.21.15.94:0/3533110799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d04f380 msgr2=0x56339d0548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.752+0000 7faeb4052740 1 --2- 172.21.15.94:0/3533110799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d04f380 0x56339d0548a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fae9c0bc890 tx=0x7fae9c0f3be0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.752+0000 7faeb4052740 1 -- 172.21.15.94:0/3533110799 shutdown_connections 2024-02-17T12:06:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.752+0000 7faeb4052740 1 --2- 172.21.15.94:0/3533110799 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d0588b0 0x56339d048760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.752+0000 7faeb4052740 1 --2- 172.21.15.94:0/3533110799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04fd30 0x56339d055fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.752+0000 7faeb4052740 1 --2- 172.21.15.94:0/3533110799 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d04f380 0x56339d0548a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.753+0000 7faeb4052740 1 -- 172.21.15.94:0/3533110799 >> 172.21.15.94:0/3533110799 conn(0x56339d049840 msgr2=0x56339d048050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:43.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.753+0000 7faeb4052740 1 -- 172.21.15.94:0/3533110799 shutdown_connections 2024-02-17T12:06:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.753+0000 7faeb4052740 1 -- 172.21.15.94:0/3533110799 wait complete. 2024-02-17T12:06:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.753+0000 7faeb4052740 1 -- start start 2024-02-17T12:06:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.753+0000 7faeb4052740 1 Processor -- start 2024-02-17T12:06:43.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeb4052740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04f380 0x56339d05edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeb4052740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04fd30 0x56339d05f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeb4052740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d0588b0 0x56339d0c0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeb4052740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339d0c0e60 con 0x56339d04f380 2024-02-17T12:06:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeb4052740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339d0c0fd0 con 0x56339d0588b0 2024-02-17T12:06:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeb4052740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339d0c1190 con 0x56339d04fd30 2024-02-17T12:06:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeb4052740 10 client.0 _ll_register_callbacks cb 0x7fae9c0f3ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeaaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04f380 0x56339d05edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeaaffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04f380 0x56339d05edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53924/0 (socket says 172.21.15.94:53924) 2024-02-17T12:06:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeaaffd640 1 -- 172.21.15.94:0/2781134021 learned_addr learned my addr 172.21.15.94:0/2781134021 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faea3fff640 1 --2- 172.21.15.94:0/2781134021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04fd30 0x56339d05f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeab7fe640 1 --2- 172.21.15.94:0/2781134021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d0588b0 0x56339d0c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeaaffd640 1 -- 172.21.15.94:0/2781134021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04fd30 msgr2=0x56339d05f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:43.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeaaffd640 1 --2- 172.21.15.94:0/2781134021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04fd30 0x56339d05f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeaaffd640 1 -- 172.21.15.94:0/2781134021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d0588b0 msgr2=0x56339d0c0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeaaffd640 1 --2- 172.21.15.94:0/2781134021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d0588b0 0x56339d0c0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.754+0000 7faeaaffd640 1 -- 172.21.15.94:0/2781134021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeac073970 con 0x56339d04f380 2024-02-17T12:06:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.755+0000 7faeaaffd640 1 --2- 172.21.15.94:0/2781134021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04f380 0x56339d05edd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fae9c003880 tx=0x7fae9c0ed910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c3eee9b9e899d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.755+0000 7fae927fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.755+0000 7fae927fc640 1 -- 172.21.15.94:0/2781134021 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9c0bd290 con 0x56339d04f380 2024-02-17T12:06:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.755+0000 7fae927fc640 1 -- 172.21.15.94:0/2781134021 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fae9c0c5330 con 0x56339d04f380 2024-02-17T12:06:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.755+0000 7faeb4052740 1 -- 172.21.15.94:0/2781134021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56339cfdc900 con 0x56339d04f380 2024-02-17T12:06:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.755+0000 7fae927fc640 1 -- 172.21.15.94:0/2781134021 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae9c0c0ce0 con 0x56339d04f380 2024-02-17T12:06:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.755+0000 7faeb4052740 10 client.17799 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.755+0000 7faeb4052740 1 -- 172.21.15.94:0/2781134021 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56339cf51c00 con 0x56339d04f380 2024-02-17T12:06:43.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.756+0000 7fae927fc640 1 -- 172.21.15.94:0/2781134021 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56339cfdc900 con 0x56339d04f380 2024-02-17T12:06:43.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.757+0000 7fae927fc640 1 -- 172.21.15.94:0/2781134021 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fae9c03c070 con 0x56339d04f380 2024-02-17T12:06:43.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.757+0000 7faeb4052740 10 client.17799 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:43.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.757+0000 7faeb4052740 10 client.17799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:43.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.757+0000 7faeb4052740 20 client.17799 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:43.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.757+0000 7faeb4052740 1 --2- 172.21.15.94:0/2781134021 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56339d0b5c80 0x56339d0b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.757+0000 7faeb4052740 4 client.17799 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:06:43.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.757+0000 7faeb4052740 1 -- 172.21.15.94:0/2781134021 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56339ccb6860 con 0x56339d0b5c80 2024-02-17T12:06:43.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.759+0000 7faea3fff640 1 --2- 172.21.15.94:0/2781134021 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56339d0b5c80 0x56339d0b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.759+0000 7faea3fff640 1 --2- 172.21.15.94:0/2781134021 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56339d0b5c80 0x56339d0b80a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620c167d070223f3 server_cookie=7d5c9e8d21ea063d in_seq=0 out_seq=0 2024-02-17T12:06:43.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.759+0000 7fae927fc640 10 client.17799 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:43.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7fae927fc640 1 -- 172.21.15.94:0/2781134021 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56339ccb6860 con 0x56339d0b5c80 2024-02-17T12:06:43.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7fae927fc640 10 client.17799 handle_command_reply: tid=0 2024-02-17T12:06:43.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7faeb4052740 1 client.17799 shutdown 2024-02-17T12:06:43.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7faeb4052740 1 -- 172.21.15.94:0/2781134021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04f380 msgr2=0x56339d05edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7faeb4052740 1 --2- 172.21.15.94:0/2781134021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04f380 0x56339d05edd0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fae9c003880 tx=0x7fae9c0ed910 comp rx=0 tx=0).stop 2024-02-17T12:06:43.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7faeb4052740 1 -- 172.21.15.94:0/2781134021 shutdown_connections 2024-02-17T12:06:43.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7faeb4052740 1 --2- 172.21.15.94:0/2781134021 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56339d0b5c80 0x56339d0b80a0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7faeb4052740 1 --2- 172.21.15.94:0/2781134021 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56339d0588b0 0x56339d0c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7faeb4052740 1 --2- 172.21.15.94:0/2781134021 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56339d04fd30 0x56339d05f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7faeb4052740 1 --2- 172.21.15.94:0/2781134021 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56339d04f380 0x56339d05edd0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7faeb4052740 1 -- 172.21.15.94:0/2781134021 >> 172.21.15.94:0/2781134021 conn(0x56339d049840 msgr2=0x56339d0488e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:43.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7fae927fc640 0 client.17799 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:43.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7faeb4052740 1 -- 172.21.15.94:0/2781134021 shutdown_connections 2024-02-17T12:06:43.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7faeb4052740 1 -- 172.21.15.94:0/2781134021 wait complete. 2024-02-17T12:06:43.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.761+0000 7faeb4052740 20 client.17799 trim_cache size 0 max 16384 2024-02-17T12:06:43.764 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:06:43.764 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:06:43.764 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "633ffebb-4184-44c2-a39b-5d98327e9264", 2024-02-17T12:06:43.764 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:06:43.764 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:06:43.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.762+0000 7faeb1481640 1 -- 172.21.15.94:0/1110689480 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fae980a2290 msgr2=0x7fae980a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.762+0000 7faeb1481640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fae980a2290 0x7fae980a4750 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7faea4005fd0 tx=0x7faea4004ab0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.762+0000 7faeb1481640 1 -- 172.21.15.94:0/1110689480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeac101380 msgr2=0x7faeac19d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.762+0000 7faeb1481640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeac101380 0x7faeac19d5b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009230 tx=0x7fae9c02c6c0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.763+0000 7faeb1481640 1 -- 172.21.15.94:0/1110689480 shutdown_connections 2024-02-17T12:06:43.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.763+0000 7faeb1481640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fae980a2290 0x7fae980a4750 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.763+0000 7faeb1481640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faeac197670 0x7faeac197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.763+0000 7faeb1481640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7faeac106450 0x7faeac19daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.763+0000 7faeb1481640 1 --2- 172.21.15.94:0/1110689480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7faeac101380 0x7faeac19d5b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.763+0000 7faeb1481640 1 -- 172.21.15.94:0/1110689480 >> 172.21.15.94:0/1110689480 conn(0x7faeac078540 msgr2=0x7faeac0ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:43.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.763+0000 7faeb1481640 1 -- 172.21.15.94:0/1110689480 shutdown_connections 2024-02-17T12:06:43.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.763+0000 7faeb1481640 1 -- 172.21.15.94:0/1110689480 wait complete. 2024-02-17T12:06:43.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:06:43.778 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:06:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.926+0000 7fb43fd0c640 1 -- 172.21.15.94:0/211847370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb438101360 msgr2=0x7fb438101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.926+0000 7fb43fd0c640 1 --2- 172.21.15.94:0/211847370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb438101360 0x7fb438101740 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb434009930 tx=0x7fb43402bc50 comp rx=0 tx=0).stop 2024-02-17T12:06:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.926+0000 7fb43fd0c640 1 -- 172.21.15.94:0/211847370 shutdown_connections 2024-02-17T12:06:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.926+0000 7fb43fd0c640 1 --2- 172.21.15.94:0/211847370 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb438106430 0x7fb43810e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.926+0000 7fb43fd0c640 1 --2- 172.21.15.94:0/211847370 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb438101d10 0x7fb438105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.926+0000 7fb43fd0c640 1 --2- 172.21.15.94:0/211847370 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb438101360 0x7fb438101740 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.926+0000 7fb43fd0c640 1 -- 172.21.15.94:0/211847370 >> 172.21.15.94:0/211847370 conn(0x7fb4380fcc20 msgr2=0x7fb4380ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.926+0000 7fb43fd0c640 1 -- 172.21.15.94:0/211847370 shutdown_connections 2024-02-17T12:06:43.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.926+0000 7fb43fd0c640 1 -- 172.21.15.94:0/211847370 wait complete. 2024-02-17T12:06:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.927+0000 7fb43fd0c640 1 Processor -- start 2024-02-17T12:06:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.927+0000 7fb43fd0c640 1 -- start start 2024-02-17T12:06:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.927+0000 7fb43fd0c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb438101360 0x7fb43819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.927+0000 7fb43fd0c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb438106430 0x7fb43819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.927+0000 7fb43fd0c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb438197650 0x7fb438197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.927+0000 7fb43fd0c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb438198120 con 0x7fb438101360 2024-02-17T12:06:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.927+0000 7fb43fd0c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb438198290 con 0x7fb438197650 2024-02-17T12:06:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.927+0000 7fb43fd0c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb438198450 con 0x7fb438106430 2024-02-17T12:06:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.927+0000 7fb43d2a7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb438106430 0x7fb43819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.927+0000 7fb43d2a7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb438106430 0x7fb43819dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:48622/0 (socket says 172.21.15.94:48622) 2024-02-17T12:06:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.927+0000 7fb43d2a7640 1 -- 172.21.15.94:0/2412931983 learned_addr learned my addr 172.21.15.94:0/2412931983 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:43.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.928+0000 7fb43e2a9640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb438197650 0x7fb438197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.928+0000 7fb43daa8640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb438101360 0x7fb43819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.928+0000 7fb43d2a7640 1 -- 172.21.15.94:0/2412931983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb438197650 msgr2=0x7fb438197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.928+0000 7fb43d2a7640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb438197650 0x7fb438197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.928+0000 7fb43d2a7640 1 -- 172.21.15.94:0/2412931983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb438101360 msgr2=0x7fb43819d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.928+0000 7fb43d2a7640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb438101360 0x7fb43819d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.928+0000 7fb43d2a7640 1 -- 172.21.15.94:0/2412931983 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb434009560 con 0x7fb438106430 2024-02-17T12:06:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.928+0000 7fb43d2a7640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb438106430 0x7fb43819dad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb42800d770 tx=0x7fb42800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.928+0000 7fb42effd640 1 -- 172.21.15.94:0/2412931983 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428009c30 con 0x7fb438106430 2024-02-17T12:06:43.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.928+0000 7fb43fd0c640 1 -- 172.21.15.94:0/2412931983 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4381a43f0 con 0x7fb438106430 2024-02-17T12:06:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.928+0000 7fb42effd640 1 -- 172.21.15.94:0/2412931983 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb42800de80 con 0x7fb438106430 2024-02-17T12:06:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.928+0000 7fb42effd640 1 -- 172.21.15.94:0/2412931983 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42800f6b0 con 0x7fb438106430 2024-02-17T12:06:43.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.928+0000 7fb43fd0c640 1 -- 172.21.15.94:0/2412931983 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4381a4920 con 0x7fb438106430 2024-02-17T12:06:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.931+0000 7fb42effd640 1 -- 172.21.15.94:0/2412931983 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb42800f810 con 0x7fb438106430 2024-02-17T12:06:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.931+0000 7fb4429be740 1 Processor -- start 2024-02-17T12:06:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.932+0000 7fb4429be740 1 -- start start 2024-02-17T12:06:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.932+0000 7fb4429be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c693590c90 0x55c693591070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.932+0000 7fb4429be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693591640 0x55c69359e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.932+0000 7fb4429be740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c69359ebc0 0x55c6935a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.932+0000 7fb4429be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c693500280 con 0x55c693591640 2024-02-17T12:06:43.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.932+0000 7fb4429be740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6934fd7b0 con 0x55c69359ebc0 2024-02-17T12:06:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.932+0000 7fb4429be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6934f8f90 con 0x55c693590c90 2024-02-17T12:06:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.932+0000 7fb43d2a7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c69359ebc0 0x55c6935a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.933+0000 7fb43d2a7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c69359ebc0 0x55c6935a0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52722/0 (socket says 172.21.15.94:52722) 2024-02-17T12:06:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.933+0000 7fb43d2a7640 1 -- 172.21.15.94:0/1831360419 learned_addr learned my addr 172.21.15.94:0/1831360419 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.933+0000 7fb43e2a9640 1 --2- 172.21.15.94:0/1831360419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c693590c90 0x55c693591070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.933+0000 7fb42effd640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4240a2330 0x7fb4240a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.933+0000 7fb43e2a9640 1 -- 172.21.15.94:0/1831360419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c69359ebc0 msgr2=0x55c6935a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.933+0000 7fb43e2a9640 1 --2- 172.21.15.94:0/1831360419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c69359ebc0 0x55c6935a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.933+0000 7fb43e2a9640 1 -- 172.21.15.94:0/1831360419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693591640 msgr2=0x55c69359e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.933+0000 7fb43e2a9640 1 --2- 172.21.15.94:0/1831360419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693591640 0x55c69359e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.933+0000 7fb43e2a9640 1 -- 172.21.15.94:0/1831360419 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6934b7570 con 0x55c693590c90 2024-02-17T12:06:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.933+0000 7fb42effd640 1 -- 172.21.15.94:0/2412931983 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb4280cde10 con 0x7fb438106430 2024-02-17T12:06:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.933+0000 7fb43e2a9640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4240a2330 0x7fb4240a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.933+0000 7fb43e2a9640 1 --2- 172.21.15.94:0/1831360419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c693590c90 0x55c693591070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb430009eb0 tx=0x7fb430010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea2934d6c5f8d91 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.934+0000 7fb423fff640 1 -- 172.21.15.94:0/1831360419 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430025830 con 0x55c693590c90 2024-02-17T12:06:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.934+0000 7fb423fff640 1 -- 172.21.15.94:0/1831360419 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb43001d070 con 0x55c693590c90 2024-02-17T12:06:43.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.934+0000 7fb423fff640 1 -- 172.21.15.94:0/1831360419 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430022b90 con 0x55c693590c90 2024-02-17T12:06:43.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.934+0000 7fb4429be740 1 -- 172.21.15.94:0/1831360419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c693590c90 msgr2=0x55c693591070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.935+0000 7fb4429be740 1 --2- 172.21.15.94:0/1831360419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c693590c90 0x55c693591070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb430009eb0 tx=0x7fb430010740 comp rx=0 tx=0).stop 2024-02-17T12:06:43.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.934+0000 7fb43e2a9640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4240a2330 0x7fb4240a47f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb430011010 tx=0x7fb430028000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:43.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.935+0000 7fb4429be740 1 -- 172.21.15.94:0/1831360419 shutdown_connections 2024-02-17T12:06:43.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.935+0000 7fb4429be740 1 --2- 172.21.15.94:0/1831360419 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c69359ebc0 0x55c6935a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.935+0000 7fb4429be740 1 --2- 172.21.15.94:0/1831360419 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693591640 0x55c69359e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.935+0000 7fb4429be740 1 --2- 172.21.15.94:0/1831360419 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c693590c90 0x55c693591070 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.935+0000 7fb4429be740 1 -- 172.21.15.94:0/1831360419 >> 172.21.15.94:0/1831360419 conn(0x55c69358b990 msgr2=0x55c69358bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:43.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.935+0000 7fb4429be740 1 -- 172.21.15.94:0/1831360419 shutdown_connections 2024-02-17T12:06:43.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.935+0000 7fb4429be740 1 -- 172.21.15.94:0/1831360419 wait complete. 2024-02-17T12:06:43.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.935+0000 7fb4429be740 1 -- start start 2024-02-17T12:06:43.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.935+0000 7fb4429be740 1 Processor -- start 2024-02-17T12:06:43.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb4429be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 0x55c69359e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb4429be740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c693591640 0x55c6935978f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb4429be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c69359ebc0 0x55c693597f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb4429be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6931cc430 con 0x55c693590c90 2024-02-17T12:06:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb43d2a7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c693591640 0x55c6935978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb43e2a9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c69359ebc0 0x55c693597f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb43daa8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 0x55c69359e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb43d2a7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c693591640 0x55c6935978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52736/0 (socket says 172.21.15.94:52736) 2024-02-17T12:06:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb43d2a7640 1 -- 172.21.15.94:0/4123370734 learned_addr learned my addr 172.21.15.94:0/4123370734 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb4429be740 1 -- 172.21.15.94:0/4123370734 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c693188e00 con 0x55c693591640 2024-02-17T12:06:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb4429be740 1 -- 172.21.15.94:0/4123370734 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69347dfd0 con 0x55c69359ebc0 2024-02-17T12:06:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb43d2a7640 1 -- 172.21.15.94:0/4123370734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c69359ebc0 msgr2=0x55c693597f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:43.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb43d2a7640 1 --2- 172.21.15.94:0/4123370734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c69359ebc0 0x55c693597f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb4429be740 10 client.0 _ll_register_callbacks cb 0x55c6934d3fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:43.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb43d2a7640 1 -- 172.21.15.94:0/4123370734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 msgr2=0x55c69359e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:43.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb43d2a7640 1 --2- 172.21.15.94:0/4123370734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 0x55c69359e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb43d2a7640 1 -- 172.21.15.94:0/4123370734 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb430009b10 con 0x55c693591640 2024-02-17T12:06:43.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.936+0000 7fb4429be740 1 -- 172.21.15.94:0/4123370734 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6931f87e0 con 0x55c693591640 2024-02-17T12:06:43.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb43e2a9640 1 --2- 172.21.15.94:0/4123370734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c69359ebc0 0x55c693597f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:43.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb43d2a7640 1 --2- 172.21.15.94:0/4123370734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c693591640 0x55c6935978f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb4280a29d0 tx=0x7fb4280d1330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=609b5067670b9e1c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:43.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb4217fa640 10 client.27284 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:06:43.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb4217fa640 1 -- 172.21.15.94:0/4123370734 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb42809a350 con 0x55c693591640 2024-02-17T12:06:43.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb4217fa640 1 -- 172.21.15.94:0/4123370734 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb42809a810 con 0x55c693591640 2024-02-17T12:06:43.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb4217fa640 1 -- 172.21.15.94:0/4123370734 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb42809a170 con 0x55c693591640 2024-02-17T12:06:43.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb4429be740 10 client.27284 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:43.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb4429be740 1 -- 172.21.15.94:0/4123370734 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x55c6934f7610 con 0x55c693591640 2024-02-17T12:06:43.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb4217fa640 1 -- 172.21.15.94:0/4123370734 <== mon.2 v2:172.21.15.125:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb4280cb740 con 0x55c693591640 2024-02-17T12:06:43.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb4429be740 10 client.27284 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:43.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb4429be740 10 client.27284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:43.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb4429be740 20 client.27284 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:43.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb4429be740 1 --2- 172.21.15.94:0/4123370734 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c6935aa760 0x55c6935acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb4429be740 4 client.27284 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:43.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb4429be740 1 -- 172.21.15.94:0/4123370734 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c693235720 con 0x55c6935aa760 2024-02-17T12:06:43.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.937+0000 7fb43daa8640 1 --2- 172.21.15.94:0/4123370734 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c6935aa760 0x55c6935acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.938+0000 7fb4217fa640 10 client.27284 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:43.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.938+0000 7fb43daa8640 1 --2- 172.21.15.94:0/4123370734 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c6935aa760 0x55c6935acbe0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eebece5a26c37b0b server_cookie=b5b482aae50e0e01 in_seq=0 out_seq=0 2024-02-17T12:06:43.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.939+0000 7fb4217fa640 1 -- 172.21.15.94:0/4123370734 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c693235720 con 0x55c6935aa760 2024-02-17T12:06:43.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.939+0000 7fb4217fa640 10 client.27284 handle_command_reply: tid=0 2024-02-17T12:06:43.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.939+0000 7fb4429be740 1 client.27284 shutdown 2024-02-17T12:06:43.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.939+0000 7fb4429be740 1 -- 172.21.15.94:0/4123370734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c693591640 msgr2=0x55c6935978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.939+0000 7fb4429be740 1 --2- 172.21.15.94:0/4123370734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c693591640 0x55c6935978f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb4280a29d0 tx=0x7fb4280d1330 comp rx=0 tx=0).stop 2024-02-17T12:06:43.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.940+0000 7fb4429be740 1 -- 172.21.15.94:0/4123370734 shutdown_connections 2024-02-17T12:06:43.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.940+0000 7fb4429be740 1 --2- 172.21.15.94:0/4123370734 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c6935aa760 0x55c6935acbe0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.940+0000 7fb4429be740 1 --2- 172.21.15.94:0/4123370734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c69359ebc0 0x55c693597f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.940+0000 7fb4429be740 1 --2- 172.21.15.94:0/4123370734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c693591640 0x55c6935978f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.940+0000 7fb4429be740 1 --2- 172.21.15.94:0/4123370734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 0x55c69359e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.940+0000 7fb4429be740 1 -- 172.21.15.94:0/4123370734 >> 172.21.15.94:0/4123370734 conn(0x55c69358b990 msgr2=0x55c69358d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:43.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.940+0000 7fb4217fa640 0 client.27284 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:43.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.940+0000 7fb4429be740 1 -- 172.21.15.94:0/4123370734 shutdown_connections 2024-02-17T12:06:43.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.940+0000 7fb4429be740 1 -- 172.21.15.94:0/4123370734 wait complete. 2024-02-17T12:06:43.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.940+0000 7fb4429be740 20 client.27284 trim_cache size 0 max 16384 2024-02-17T12:06:43.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.961+0000 7fb4429be740 1 Processor -- start 2024-02-17T12:06:43.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.961+0000 7fb4429be740 1 -- start start 2024-02-17T12:06:43.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.962+0000 7fb4429be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 0x55c69359e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.962+0000 7fb4429be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c693591640 0x55c693592c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.962+0000 7fb4429be740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c69359ebc0 0x55c693593140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.962+0000 7fb4429be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6935872b0 con 0x55c693590c90 2024-02-17T12:06:43.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.962+0000 7fb4429be740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c693592330 con 0x55c69359ebc0 2024-02-17T12:06:43.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.962+0000 7fb4429be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c693597d10 con 0x55c693591640 2024-02-17T12:06:43.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.962+0000 7fb43e2a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c69359ebc0 0x55c693593140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.962+0000 7fb43e2a9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c69359ebc0 0x55c693593140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52738/0 (socket says 172.21.15.94:52738) 2024-02-17T12:06:43.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.962+0000 7fb43e2a9640 1 -- 172.21.15.94:0/4088713290 learned_addr learned my addr 172.21.15.94:0/4088713290 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:43.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.963+0000 7fb43e2a9640 1 -- 172.21.15.94:0/4088713290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c693591640 msgr2=0x55c693592c00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:06:43.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.963+0000 7fb43e2a9640 1 --2- 172.21.15.94:0/4088713290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c693591640 0x55c693592c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.963+0000 7fb43e2a9640 1 -- 172.21.15.94:0/4088713290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 msgr2=0x55c69359e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:43.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.963+0000 7fb43e2a9640 1 --2- 172.21.15.94:0/4088713290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 0x55c69359e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.963+0000 7fb43e2a9640 1 -- 172.21.15.94:0/4088713290 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c693493ac0 con 0x55c69359ebc0 2024-02-17T12:06:43.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.963+0000 7fb43e2a9640 1 --2- 172.21.15.94:0/4088713290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c69359ebc0 0x55c693593140 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb430009eb0 tx=0x7fb43001b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=944e9813168c1229 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:43.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.963+0000 7fb4237fe640 1 -- 172.21.15.94:0/4088713290 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430035050 con 0x55c69359ebc0 2024-02-17T12:06:43.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.964+0000 7fb4237fe640 1 -- 172.21.15.94:0/4088713290 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb430036040 con 0x55c69359ebc0 2024-02-17T12:06:43.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.964+0000 7fb4237fe640 1 -- 172.21.15.94:0/4088713290 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430025b10 con 0x55c69359ebc0 2024-02-17T12:06:43.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.964+0000 7fb4429be740 1 -- 172.21.15.94:0/4088713290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c69359ebc0 msgr2=0x55c693593140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.964+0000 7fb4429be740 1 --2- 172.21.15.94:0/4088713290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c69359ebc0 0x55c693593140 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb430009eb0 tx=0x7fb43001b040 comp rx=0 tx=0).stop 2024-02-17T12:06:43.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.964+0000 7fb4429be740 1 -- 172.21.15.94:0/4088713290 shutdown_connections 2024-02-17T12:06:43.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.964+0000 7fb4429be740 1 --2- 172.21.15.94:0/4088713290 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c69359ebc0 0x55c693593140 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.964+0000 7fb4429be740 1 --2- 172.21.15.94:0/4088713290 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c693591640 0x55c693592c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.964+0000 7fb4429be740 1 --2- 172.21.15.94:0/4088713290 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 0x55c69359e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.964+0000 7fb4429be740 1 -- 172.21.15.94:0/4088713290 >> 172.21.15.94:0/4088713290 conn(0x55c69358b990 msgr2=0x55c6935b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:43.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.964+0000 7fb4429be740 1 -- 172.21.15.94:0/4088713290 shutdown_connections 2024-02-17T12:06:43.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.964+0000 7fb4429be740 1 -- 172.21.15.94:0/4088713290 wait complete. 2024-02-17T12:06:43.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb4429be740 1 -- start start 2024-02-17T12:06:43.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb4429be740 1 Processor -- start 2024-02-17T12:06:43.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb4429be740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 0x55c6935a1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb4429be740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c693591640 0x55c6935a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb4429be740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c69359ebc0 0x55c6935a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb4429be740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6935a7580 con 0x55c693590c90 2024-02-17T12:06:43.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb4429be740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c693600cf0 con 0x55c693591640 2024-02-17T12:06:43.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb4429be740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c693600e30 con 0x55c69359ebc0 2024-02-17T12:06:43.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb4429be740 10 client.0 _ll_register_callbacks cb 0x7fb430009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:43.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb43daa8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 0x55c6935a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb43daa8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 0x55c6935a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53976/0 (socket says 172.21.15.94:53976) 2024-02-17T12:06:43.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb43d2a7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c693591640 0x55c6935a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb43daa8640 1 -- 172.21.15.94:0/2882682599 learned_addr learned my addr 172.21.15.94:0/2882682599 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:43.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.965+0000 7fb43e2a9640 1 --2- 172.21.15.94:0/2882682599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c69359ebc0 0x55c6935a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.966+0000 7fb43daa8640 1 -- 172.21.15.94:0/2882682599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c69359ebc0 msgr2=0x55c6935a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:43.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.966+0000 7fb43daa8640 1 --2- 172.21.15.94:0/2882682599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c69359ebc0 0x55c6935a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.966+0000 7fb43daa8640 1 -- 172.21.15.94:0/2882682599 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c693591640 msgr2=0x55c6935a1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:43.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.966+0000 7fb43daa8640 1 --2- 172.21.15.94:0/2882682599 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c693591640 0x55c6935a1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.966+0000 7fb43daa8640 1 -- 172.21.15.94:0/2882682599 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6934b7570 con 0x55c693590c90 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.966+0000 7fb43daa8640 1 --2- 172.21.15.94:0/2882682599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 0x55c6935a1740 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb434009930 tx=0x7fb434005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691304e79b039571 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.966+0000 7fb4217fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.966+0000 7fb4217fa640 1 -- 172.21.15.94:0/2882682599 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb434045070 con 0x55c693590c90 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.966+0000 7fb4217fa640 1 -- 172.21.15.94:0/2882682599 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb434004990 con 0x55c693590c90 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.966+0000 7fb4429be740 1 -- 172.21.15.94:0/2882682599 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb43001d070 con 0x55c693590c90 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.966+0000 7fb4217fa640 1 -- 172.21.15.94:0/2882682599 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb434047b70 con 0x55c693590c90 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.966+0000 7fb4429be740 10 client.17811 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.966+0000 7fb4429be740 1 -- 172.21.15.94:0/2882682599 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c6934b96a0 con 0x55c693590c90 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.967+0000 7fb4217fa640 1 -- 172.21.15.94:0/2882682599 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb43001d070 con 0x55c693590c90 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.967+0000 7fb4217fa640 1 -- 172.21.15.94:0/2882682599 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb434005200 con 0x55c693590c90 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.967+0000 7fb4429be740 10 client.17811 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.967+0000 7fb4429be740 10 client.17811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:43.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.967+0000 7fb4429be740 20 client.17811 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.967+0000 7fb4429be740 1 --2- 172.21.15.94:0/2882682599 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c693602540 0x55c693604930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.967+0000 7fb4429be740 4 client.17811 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.967+0000 7fb4429be740 1 -- 172.21.15.94:0/2882682599 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c693284890 con 0x55c693602540 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.967+0000 7fb43d2a7640 1 --2- 172.21.15.94:0/2882682599 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c693602540 0x55c693604930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.968+0000 7fb43d2a7640 1 --2- 172.21.15.94:0/2882682599 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c693602540 0x55c693604930 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=645b04b915bbb7a9 server_cookie=af62eee0d2d8f122 in_seq=0 out_seq=0 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.968+0000 7fb4217fa640 10 client.17811 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.968+0000 7fb4217fa640 1 -- 172.21.15.94:0/2882682599 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c693284890 con 0x55c693602540 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.968+0000 7fb4217fa640 10 client.17811 handle_command_reply: tid=0 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.968+0000 7fb4429be740 1 client.17811 shutdown 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb4429be740 1 -- 172.21.15.94:0/2882682599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 msgr2=0x55c6935a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb4429be740 1 --2- 172.21.15.94:0/2882682599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 0x55c6935a1740 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb434009930 tx=0x7fb434005fd0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb4429be740 1 -- 172.21.15.94:0/2882682599 shutdown_connections 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb4429be740 1 --2- 172.21.15.94:0/2882682599 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c693602540 0x55c693604930 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb4429be740 1 --2- 172.21.15.94:0/2882682599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c69359ebc0 0x55c6935a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb4217fa640 0 client.17811 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:43.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb4429be740 1 --2- 172.21.15.94:0/2882682599 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c693591640 0x55c6935a1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb4429be740 1 --2- 172.21.15.94:0/2882682599 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c693590c90 0x55c6935a1740 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb4429be740 1 -- 172.21.15.94:0/2882682599 >> 172.21.15.94:0/2882682599 conn(0x55c69358b990 msgr2=0x55c693592c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:43.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb4429be740 1 -- 172.21.15.94:0/2882682599 shutdown_connections 2024-02-17T12:06:43.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb4429be740 1 -- 172.21.15.94:0/2882682599 wait complete. 2024-02-17T12:06:43.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb4429be740 20 client.17811 trim_cache size 0 max 16384 2024-02-17T12:06:43.975 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:06:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:06:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:06:43.975 INFO:teuthology.orchestra.run.smithi094.stdout: "633ffebb-4184-44c2-a39b-5d98327e9264": { 2024-02-17T12:06:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:06:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "633ffebb-4184-44c2-a39b-5d98327e9264", 2024-02-17T12:06:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:06:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:06:43.976 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:06:43.976 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:06:43.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb43fd0c640 1 -- 172.21.15.94:0/2412931983 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4240a2330 msgr2=0x7fb4240a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb43fd0c640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4240a2330 0x7fb4240a47f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb430011010 tx=0x7fb430028000 comp rx=0 tx=0).stop 2024-02-17T12:06:43.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb43fd0c640 1 -- 172.21.15.94:0/2412931983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb438106430 msgr2=0x7fb43819dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:43.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb43fd0c640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb438106430 0x7fb43819dad0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb42800d770 tx=0x7fb42800dc30 comp rx=0 tx=0).stop 2024-02-17T12:06:43.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb43fd0c640 1 -- 172.21.15.94:0/2412931983 shutdown_connections 2024-02-17T12:06:43.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb43fd0c640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4240a2330 0x7fb4240a47f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb43fd0c640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb438197650 0x7fb438197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.969+0000 7fb43fd0c640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb438106430 0x7fb43819dad0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.970+0000 7fb43fd0c640 1 --2- 172.21.15.94:0/2412931983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb438101360 0x7fb43819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:43.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.970+0000 7fb43fd0c640 1 -- 172.21.15.94:0/2412931983 >> 172.21.15.94:0/2412931983 conn(0x7fb4380fcc20 msgr2=0x7fb4380fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:43.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.970+0000 7fb43fd0c640 1 -- 172.21.15.94:0/2412931983 shutdown_connections 2024-02-17T12:06:43.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:06:43.970+0000 7fb43fd0c640 1 -- 172.21.15.94:0/2412931983 wait complete. 2024-02-17T12:06:43.985 INFO:tasks.cephfs.filesystem:scrub status for tag:633ffebb-4184-44c2-a39b-5d98327e9264 - {'path': '/', 'tag': '633ffebb-4184-44c2-a39b-5d98327e9264', 'options': 'recursive,force'} 2024-02-17T12:06:44.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:43 smithi094 ceph-mon[32499]: pgmap v265: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 580 op/s 2024-02-17T12:06:44.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:43 smithi189 ceph-mon[36588]: pgmap v265: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 580 op/s 2024-02-17T12:06:44.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:43 smithi125 ceph-mon[36819]: pgmap v265: 97 pgs: 97 active+clean; 5.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.4 MiB/s wr, 580 op/s 2024-02-17T12:06:45.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 505.0 s, 1907.6 tps, lat 51.785 ms stddev 53.236 2024-02-17T12:06:45.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:44 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:06:45.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:44 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:06:45.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:44 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:06:45.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:44 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:06:45.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:44 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:06:45.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:44 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:06:45.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:44 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:06:45.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:44 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:06:45.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:44 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:06:45.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:44 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:06:45.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:44 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:06:45.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:44 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:06:46.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:45 smithi189 ceph-mon[36588]: pgmap v266: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.3 MiB/s wr, 792 op/s 2024-02-17T12:06:46.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:45 smithi125 ceph-mon[36819]: pgmap v266: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.3 MiB/s wr, 792 op/s 2024-02-17T12:06:46.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:45 smithi094 ceph-mon[32499]: pgmap v266: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.3 MiB/s wr, 792 op/s 2024-02-17T12:06:47.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:46 smithi189 ceph-mon[36588]: pgmap v267: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.1 MiB/s wr, 537 op/s 2024-02-17T12:06:47.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:46 smithi125 ceph-mon[36819]: pgmap v267: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.1 MiB/s wr, 537 op/s 2024-02-17T12:06:47.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:46 smithi094 ceph-mon[32499]: pgmap v267: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.1 MiB/s wr, 537 op/s 2024-02-17T12:06:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.004+0000 7f12bf155640 1 -- 172.21.15.94:0/1644452513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12b810f8c0 msgr2=0x7f12b810d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.004+0000 7f12bf155640 1 --2- 172.21.15.94:0/1644452513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12b810f8c0 0x7f12b810d4e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009900 tx=0x7f12b402bc60 comp rx=0 tx=0).stop 2024-02-17T12:06:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.008+0000 7f12bf155640 1 -- 172.21.15.94:0/1644452513 shutdown_connections 2024-02-17T12:06:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.008+0000 7f12bf155640 1 --2- 172.21.15.94:0/1644452513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b811a960 0x7f12b811cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.008+0000 7f12bf155640 1 --2- 172.21.15.94:0/1644452513 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12b810fd30 0x7f12b810da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.008+0000 7f12bf155640 1 --2- 172.21.15.94:0/1644452513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12b810f8c0 0x7f12b810d4e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.008+0000 7f12bf155640 1 -- 172.21.15.94:0/1644452513 >> 172.21.15.94:0/1644452513 conn(0x7f12b806c220 msgr2=0x7f12b806c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:48.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.016+0000 7f12bf155640 1 -- 172.21.15.94:0/1644452513 shutdown_connections 2024-02-17T12:06:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.016+0000 7f12bf155640 1 -- 172.21.15.94:0/1644452513 wait complete. 2024-02-17T12:06:48.019 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.017+0000 7f12bf155640 1 Processor -- start 2024-02-17T12:06:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.018+0000 7f12bf155640 1 -- start start 2024-02-17T12:06:48.020 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.019+0000 7f12bf155640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12b810f8c0 0x7f12b8119f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.019+0000 7f12bf155640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12b810fd30 0x7f12b8113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.019+0000 7f12bf155640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b811a960 0x7f12b81144a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.019+0000 7f12bf155640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b81149e0 con 0x7f12b811a960 2024-02-17T12:06:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.019+0000 7f12bf155640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8114b50 con 0x7f12b810f8c0 2024-02-17T12:06:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.019+0000 7f12bf155640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8114d10 con 0x7f12b810fd30 2024-02-17T12:06:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.019+0000 7f12affff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12b810fd30 0x7f12b8113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.019+0000 7f12affff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12b810fd30 0x7f12b8113f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:58652/0 (socket says 172.21.15.94:58652) 2024-02-17T12:06:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.019+0000 7f12affff640 1 -- 172.21.15.94:0/3501600533 learned_addr learned my addr 172.21.15.94:0/3501600533 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.020+0000 7f12affff640 1 -- 172.21.15.94:0/3501600533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12b810f8c0 msgr2=0x7f12b8119f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:06:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.020+0000 7f12affff640 1 --2- 172.21.15.94:0/3501600533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12b810f8c0 0x7f12b8119f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.020+0000 7f12affff640 1 -- 172.21.15.94:0/3501600533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b811a960 msgr2=0x7f12b81144a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.020+0000 7f12affff640 1 --2- 172.21.15.94:0/3501600533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b811a960 0x7f12b81144a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.020+0000 7f12affff640 1 -- 172.21.15.94:0/3501600533 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b4009560 con 0x7f12b810fd30 2024-02-17T12:06:48.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.020+0000 7f12affff640 1 --2- 172.21.15.94:0/3501600533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12b810fd30 0x7f12b8113f60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f12a800c970 tx=0x7f12a800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.020+0000 7f12adffb640 1 -- 172.21.15.94:0/3501600533 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8007bb0 con 0x7f12b810fd30 2024-02-17T12:06:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.021+0000 7f12adffb640 1 -- 172.21.15.94:0/3501600533 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12a8007d10 con 0x7f12b810fd30 2024-02-17T12:06:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.021+0000 7f12bf155640 1 -- 172.21.15.94:0/3501600533 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b81bb600 con 0x7f12b810fd30 2024-02-17T12:06:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.021+0000 7f12adffb640 1 -- 172.21.15.94:0/3501600533 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a800f450 con 0x7f12b810fd30 2024-02-17T12:06:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.022+0000 7f12bf155640 1 -- 172.21.15.94:0/3501600533 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b81bbae0 con 0x7f12b810fd30 2024-02-17T12:06:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.027+0000 7f12adffb640 1 -- 172.21.15.94:0/3501600533 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f12a8005020 con 0x7f12b810fd30 2024-02-17T12:06:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.027+0000 7f12c1dfd740 1 Processor -- start 2024-02-17T12:06:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.027+0000 7f12c1dfd740 1 -- start start 2024-02-17T12:06:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.028+0000 7f12c1dfd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f9ad0 0x55f33b2f8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.028+0000 7f12c1dfd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f8d60 0x55f33b2f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.028+0000 7f12c1dfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b3022e0 0x55f33b3046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.028+0000 7f12c1dfd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33b2679f0 con 0x55f33b2f9ad0 2024-02-17T12:06:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.028+0000 7f12c1dfd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33b264f20 con 0x55f33b2f8d60 2024-02-17T12:06:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.028+0000 7f12c1dfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33b260700 con 0x55f33b3022e0 2024-02-17T12:06:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.028+0000 7f12bcef1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f8d60 0x55f33b2f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.028+0000 7f12bcef1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f8d60 0x55f33b2f91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:47860/0 (socket says 172.21.15.94:47860) 2024-02-17T12:06:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.028+0000 7f12bcef1640 1 -- 172.21.15.94:0/1560552683 learned_addr learned my addr 172.21.15.94:0/1560552683 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.029+0000 7f12bcef1640 1 -- 172.21.15.94:0/1560552683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b3022e0 msgr2=0x55f33b3046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.029+0000 7f12adffb640 1 --2- 172.21.15.94:0/3501600533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12a00a24b0 0x7f12a00a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.029+0000 7f12affff640 1 --2- 172.21.15.94:0/1560552683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b3022e0 0x55f33b3046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.029+0000 7f12bcef1640 1 --2- 172.21.15.94:0/1560552683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b3022e0 0x55f33b3046d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.029+0000 7f12bcef1640 1 -- 172.21.15.94:0/1560552683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f9ad0 msgr2=0x55f33b2f8820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:06:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.030+0000 7f12adffb640 1 -- 172.21.15.94:0/3501600533 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f12a80cd440 con 0x7f12b810fd30 2024-02-17T12:06:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.030+0000 7f12bcef1640 1 --2- 172.21.15.94:0/1560552683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f9ad0 0x55f33b2f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.030+0000 7f12bcef1640 1 -- 172.21.15.94:0/1560552683 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f33b1f67b0 con 0x55f33b2f8d60 2024-02-17T12:06:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.030+0000 7f12bcef1640 1 --2- 172.21.15.94:0/1560552683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f8d60 0x55f33b2f91e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f12b4036040 tx=0x7f12b40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fd0b8b000d816cf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.031+0000 7f129effd640 1 -- 172.21.15.94:0/1560552683 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b40048b0 con 0x55f33b2f8d60 2024-02-17T12:06:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.031+0000 7f129effd640 1 -- 172.21.15.94:0/1560552683 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12b4004a10 con 0x55f33b2f8d60 2024-02-17T12:06:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.031+0000 7f129effd640 1 -- 172.21.15.94:0/1560552683 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4005200 con 0x55f33b2f8d60 2024-02-17T12:06:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.031+0000 7f12bd6f2640 1 --2- 172.21.15.94:0/3501600533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12a00a24b0 0x7f12a00a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.033+0000 7f12c1dfd740 1 -- 172.21.15.94:0/1560552683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f8d60 msgr2=0x55f33b2f91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.033+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/1560552683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f8d60 0x55f33b2f91e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f12b4036040 tx=0x7f12b40046c0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.034+0000 7f12c1dfd740 1 -- 172.21.15.94:0/1560552683 shutdown_connections 2024-02-17T12:06:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.034+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/1560552683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b3022e0 0x55f33b3046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.034+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/1560552683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f8d60 0x55f33b2f91e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.034+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/1560552683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f9ad0 0x55f33b2f8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.034+0000 7f12c1dfd740 1 -- 172.21.15.94:0/1560552683 >> 172.21.15.94:0/1560552683 conn(0x55f33b2c0550 msgr2=0x55f33b2f3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.035+0000 7f12c1dfd740 1 -- 172.21.15.94:0/1560552683 shutdown_connections 2024-02-17T12:06:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.035+0000 7f12bd6f2640 1 --2- 172.21.15.94:0/3501600533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12a00a24b0 0x7f12a00a4970 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f12b806a590 tx=0x7f12b0003ce0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.035+0000 7f12c1dfd740 1 -- 172.21.15.94:0/1560552683 wait complete. 2024-02-17T12:06:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.036+0000 7f12c1dfd740 1 -- start start 2024-02-17T12:06:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.036+0000 7f12c1dfd740 1 Processor -- start 2024-02-17T12:06:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.037+0000 7f12c1dfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b2f8d60 0x55f33b2ff3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.037+0000 7f12c1dfd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f9ad0 0x55f33b2ff920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.037+0000 7f12bcef1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b2f8d60 0x55f33b2ff3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.037+0000 7f12bcef1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b2f8d60 0x55f33b2ff3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:58670/0 (socket says 172.21.15.94:58670) 2024-02-17T12:06:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.037+0000 7f12bcef1640 1 -- 172.21.15.94:0/2490559078 learned_addr learned my addr 172.21.15.94:0/2490559078 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.038+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b3022e0 0x55f33b2ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.038+0000 7f12affff640 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f9ad0 0x55f33b2ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.038+0000 7f12c1dfd740 1 -- 172.21.15.94:0/2490559078 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33b205bf0 con 0x55f33b3022e0 2024-02-17T12:06:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.038+0000 7f12c1dfd740 1 -- 172.21.15.94:0/2490559078 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33af34250 con 0x55f33b2f9ad0 2024-02-17T12:06:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.038+0000 7f12c1dfd740 1 -- 172.21.15.94:0/2490559078 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33b1e6200 con 0x55f33b2f8d60 2024-02-17T12:06:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.038+0000 7f12bd6f2640 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b3022e0 0x55f33b2ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f12bcef1640 1 -- 172.21.15.94:0/2490559078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f9ad0 msgr2=0x55f33b2ff920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f12bcef1640 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f9ad0 0x55f33b2ff920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f12bcef1640 1 -- 172.21.15.94:0/2490559078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b3022e0 msgr2=0x55f33b2ffe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f12bcef1640 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b3022e0 0x55f33b2ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f12bcef1640 1 -- 172.21.15.94:0/2490559078 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b811d640 con 0x55f33b2f8d60 2024-02-17T12:06:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f12bcef1640 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b2f8d60 0x55f33b2ff3e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f12b402c270 tx=0x7f12b4004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acb216f90d40574 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f12c1dfd740 10 client.0 _ll_register_callbacks cb 0x55f33b254d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f12c1dfd740 1 -- 172.21.15.94:0/2490559078 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f33af60600 con 0x55f33b2f8d60 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f1297fff640 10 client.27313 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f1297fff640 1 -- 172.21.15.94:0/2490559078 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b40048b0 con 0x55f33b2f8d60 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f1297fff640 1 -- 172.21.15.94:0/2490559078 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f12b402cc20 con 0x55f33b2f8d60 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f1297fff640 1 -- 172.21.15.94:0/2490559078 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12b4005180 con 0x55f33b2f8d60 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f12c1dfd740 10 client.27313 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.039+0000 7f12c1dfd740 1 -- 172.21.15.94:0/2490559078 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x55f33b25ed80 con 0x55f33b2f8d60 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.040+0000 7f1297fff640 1 -- 172.21.15.94:0/2490559078 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f12b4058070 con 0x55f33b2f8d60 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.040+0000 7f12c1dfd740 10 client.27313 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.040+0000 7f12c1dfd740 10 client.27313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.040+0000 7f12c1dfd740 20 client.27313 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.040+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f33b3123b0 0x55f33b314830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.040+0000 7f12c1dfd740 4 client.27313 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.040+0000 7f12c1dfd740 1 -- 172.21.15.94:0/2490559078 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f33afb11b0 con 0x55f33b3123b0 2024-02-17T12:06:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.041+0000 7f12affff640 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f33b3123b0 0x55f33b314830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.042+0000 7f12affff640 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f33b3123b0 0x55f33b314830 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8778f8e49057360e server_cookie=8d973b8cd132ba8a in_seq=0 out_seq=0 2024-02-17T12:06:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.042+0000 7f1297fff640 10 client.27313 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.042+0000 7f1297fff640 1 -- 172.21.15.94:0/2490559078 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f33afb11b0 con 0x55f33b3123b0 2024-02-17T12:06:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.042+0000 7f1297fff640 10 client.27313 handle_command_reply: tid=0 2024-02-17T12:06:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.042+0000 7f12c1dfd740 1 client.27313 shutdown 2024-02-17T12:06:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.045+0000 7f12c1dfd740 1 -- 172.21.15.94:0/2490559078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b2f8d60 msgr2=0x55f33b2ff3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.045+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b2f8d60 0x55f33b2ff3e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f12b402c270 tx=0x7f12b4004510 comp rx=0 tx=0).stop 2024-02-17T12:06:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.045+0000 7f12c1dfd740 1 -- 172.21.15.94:0/2490559078 shutdown_connections 2024-02-17T12:06:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.045+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f33b3123b0 0x55f33b314830 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.045+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b3022e0 0x55f33b2ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.045+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f9ad0 0x55f33b2ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.045+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/2490559078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b2f8d60 0x55f33b2ff3e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.045+0000 7f12c1dfd740 1 -- 172.21.15.94:0/2490559078 >> 172.21.15.94:0/2490559078 conn(0x55f33b2c0550 msgr2=0x55f33b2f57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.045+0000 7f1297fff640 0 client.27313 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.046+0000 7f12c1dfd740 1 -- 172.21.15.94:0/2490559078 shutdown_connections 2024-02-17T12:06:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.046+0000 7f12c1dfd740 1 -- 172.21.15.94:0/2490559078 wait complete. 2024-02-17T12:06:48.048 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.046+0000 7f12c1dfd740 20 client.27313 trim_cache size 0 max 16384 2024-02-17T12:06:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.065+0000 7f12c1dfd740 1 Processor -- start 2024-02-17T12:06:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.065+0000 7f12c1dfd740 1 -- start start 2024-02-17T12:06:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.065+0000 7f12c1dfd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 0x55f33b301b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.065+0000 7f12c1dfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b2f9ad0 0x55f33b3090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.065+0000 7f12c1dfd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b3022e0 0x55f33b3095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.065+0000 7f12c1dfd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33b3003b0 con 0x55f33b2f8d60 2024-02-17T12:06:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.065+0000 7f12c1dfd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33b2ef380 con 0x55f33b3022e0 2024-02-17T12:06:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.065+0000 7f12c1dfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33b30f1d0 con 0x55f33b2f9ad0 2024-02-17T12:06:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.065+0000 7f12bcef1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 0x55f33b301b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.065+0000 7f12bcef1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 0x55f33b301b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40372/0 (socket says 172.21.15.94:40372) 2024-02-17T12:06:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.065+0000 7f12bcef1640 1 -- 172.21.15.94:0/571867055 learned_addr learned my addr 172.21.15.94:0/571867055 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.066+0000 7f12affff640 1 --2- 172.21.15.94:0/571867055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b2f9ad0 0x55f33b3090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.066+0000 7f12bd6f2640 1 --2- 172.21.15.94:0/571867055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b3022e0 0x55f33b3095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.066+0000 7f12bcef1640 1 -- 172.21.15.94:0/571867055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b2f9ad0 msgr2=0x55f33b3090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.066+0000 7f12bcef1640 1 --2- 172.21.15.94:0/571867055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b2f9ad0 0x55f33b3090a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.066+0000 7f12bcef1640 1 -- 172.21.15.94:0/571867055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b3022e0 msgr2=0x55f33b3095e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.066+0000 7f12bcef1640 1 --2- 172.21.15.94:0/571867055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b3022e0 0x55f33b3095e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.069 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.066+0000 7f12bcef1640 1 -- 172.21.15.94:0/571867055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f33b1fb7b0 con 0x55f33b2f8d60 2024-02-17T12:06:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.066+0000 7f12bcef1640 1 --2- 172.21.15.94:0/571867055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 0x55f33b301b10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f12b4006010 tx=0x7f12b404b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42882c34ea29d00 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.066+0000 7f129e7fc640 1 -- 172.21.15.94:0/571867055 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4005180 con 0x55f33b2f8d60 2024-02-17T12:06:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.066+0000 7f129e7fc640 1 -- 172.21.15.94:0/571867055 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f33af60600 con 0x55f33b2f8d60 2024-02-17T12:06:48.070 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.066+0000 7f129e7fc640 1 -- 172.21.15.94:0/571867055 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4003d30 con 0x55f33b2f8d60 2024-02-17T12:06:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.067+0000 7f12c1dfd740 1 -- 172.21.15.94:0/571867055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 msgr2=0x55f33b301b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.068+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/571867055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 0x55f33b301b10 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f12b4006010 tx=0x7f12b404b000 comp rx=0 tx=0).stop 2024-02-17T12:06:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.068+0000 7f12c1dfd740 1 -- 172.21.15.94:0/571867055 shutdown_connections 2024-02-17T12:06:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.068+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/571867055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b3022e0 0x55f33b3095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.068+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/571867055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b2f9ad0 0x55f33b3090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.068+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/571867055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 0x55f33b301b10 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.068+0000 7f12c1dfd740 1 -- 172.21.15.94:0/571867055 >> 172.21.15.94:0/571867055 conn(0x55f33b2c0550 msgr2=0x55f33b2f3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.068+0000 7f12c1dfd740 1 -- 172.21.15.94:0/571867055 shutdown_connections 2024-02-17T12:06:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.068+0000 7f12c1dfd740 1 -- 172.21.15.94:0/571867055 wait complete. 2024-02-17T12:06:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.068+0000 7f12c1dfd740 1 -- start start 2024-02-17T12:06:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.069+0000 7f12c1dfd740 1 Processor -- start 2024-02-17T12:06:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.069+0000 7f12c1dfd740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 0x55f33b3093e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.069+0000 7f12c1dfd740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f9ad0 0x55f33b2fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.069+0000 7f12c1dfd740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b3022e0 0x55f33b2fc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.069+0000 7f12c1dfd740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33b2fc970 con 0x55f33b2f8d60 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.069+0000 7f12c1dfd740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33b2fcae0 con 0x55f33b2f9ad0 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.069+0000 7f12bcef1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 0x55f33b3093e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12c1dfd740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f33b368950 con 0x55f33b3022e0 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12bcef1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 0x55f33b3093e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40388/0 (socket says 172.21.15.94:40388) 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12bcef1640 1 -- 172.21.15.94:0/3892030341 learned_addr learned my addr 172.21.15.94:0/3892030341 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12bd6f2640 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b3022e0 0x55f33b2fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12affff640 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f9ad0 0x55f33b2fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12c1dfd740 10 client.0 _ll_register_callbacks cb 0x7f12b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12bd6f2640 1 -- 172.21.15.94:0/3892030341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f9ad0 msgr2=0x55f33b2fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12bd6f2640 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f9ad0 0x55f33b2fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12bd6f2640 1 -- 172.21.15.94:0/3892030341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 msgr2=0x55f33b3093e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:06:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12bd6f2640 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 0x55f33b3093e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12bd6f2640 1 -- 172.21.15.94:0/3892030341 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f33b25ed80 con 0x55f33b3022e0 2024-02-17T12:06:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12bcef1640 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 0x55f33b3093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:06:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12bd6f2640 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b3022e0 0x55f33b2fc290 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55f33b2fa1d0 tx=0x7f12b0006c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc085b0c2104852e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:06:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f1297fff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:06:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f1297fff640 1 -- 172.21.15.94:0/3892030341 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0006a20 con 0x55f33b3022e0 2024-02-17T12:06:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f1297fff640 1 -- 172.21.15.94:0/3892030341 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f12b0007e40 con 0x55f33b3022e0 2024-02-17T12:06:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.070+0000 7f12c1dfd740 1 -- 172.21.15.94:0/3892030341 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12b4004a10 con 0x55f33b3022e0 2024-02-17T12:06:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.071+0000 7f1297fff640 1 -- 172.21.15.94:0/3892030341 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12b0014b90 con 0x55f33b3022e0 2024-02-17T12:06:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.071+0000 7f12c1dfd740 10 client.27319 fetch_fsmap learned FSMap version 27 2024-02-17T12:06:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.071+0000 7f12c1dfd740 1 -- 172.21.15.94:0/3892030341 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f33b221540 con 0x55f33b3022e0 2024-02-17T12:06:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.071+0000 7f1297fff640 1 -- 172.21.15.94:0/3892030341 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12b002e020 con 0x55f33b3022e0 2024-02-17T12:06:48.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.071+0000 7f1297fff640 1 -- 172.21.15.94:0/3892030341 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f12b001f070 con 0x55f33b3022e0 2024-02-17T12:06:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.071+0000 7f12c1dfd740 10 client.27319 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:06:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.071+0000 7f12c1dfd740 10 client.27319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:06:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.072+0000 7f12c1dfd740 20 client.27319 populate_metadata read hostname 'smithi094' 2024-02-17T12:06:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.072+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f33b36a030 0x55f33b36c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:06:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.072+0000 7f12c1dfd740 4 client.27319 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:06:48.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.072+0000 7f12c1dfd740 1 -- 172.21.15.94:0/3892030341 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f33af9d550 con 0x55f33b36a030 2024-02-17T12:06:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.072+0000 7f12bcef1640 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f33b36a030 0x55f33b36c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:06:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.073+0000 7f12bcef1640 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f33b36a030 0x55f33b36c450 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4923914a57a22f02 server_cookie=d678616298a4fa74 in_seq=0 out_seq=0 2024-02-17T12:06:48.078 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.073+0000 7f1297fff640 10 client.27319 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.088+0000 7f1297fff640 1 -- 172.21.15.94:0/3892030341 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f33af9d550 con 0x55f33b36a030 2024-02-17T12:06:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.088+0000 7f1297fff640 10 client.27319 handle_command_reply: tid=0 2024-02-17T12:06:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.088+0000 7f12c1dfd740 1 client.27319 shutdown 2024-02-17T12:06:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.088+0000 7f12c1dfd740 1 -- 172.21.15.94:0/3892030341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b3022e0 msgr2=0x55f33b2fc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.088+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b3022e0 0x55f33b2fc290 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55f33b2fa1d0 tx=0x7f12b0006c50 comp rx=0 tx=0).stop 2024-02-17T12:06:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.088+0000 7f12c1dfd740 1 -- 172.21.15.94:0/3892030341 shutdown_connections 2024-02-17T12:06:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.088+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f33b36a030 0x55f33b36c450 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.088+0000 7f1297fff640 0 client.27319 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:06:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.088+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f33b3022e0 0x55f33b2fc290 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.088+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f33b2f9ad0 0x55f33b2fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.088+0000 7f12c1dfd740 1 --2- 172.21.15.94:0/3892030341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f33b2f8d60 0x55f33b3093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.088+0000 7f12c1dfd740 1 -- 172.21.15.94:0/3892030341 >> 172.21.15.94:0/3892030341 conn(0x55f33b2c0550 msgr2=0x55f33b2fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.089+0000 7f12c1dfd740 1 -- 172.21.15.94:0/3892030341 shutdown_connections 2024-02-17T12:06:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.089+0000 7f12c1dfd740 1 -- 172.21.15.94:0/3892030341 wait complete. 2024-02-17T12:06:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.089+0000 7f12c1dfd740 20 client.27319 trim_cache size 0 max 16384 2024-02-17T12:06:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:06:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:06:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:06:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:06:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.089+0000 7f12bf155640 1 -- 172.21.15.94:0/3501600533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12a00a24b0 msgr2=0x7f12a00a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.089+0000 7f12bf155640 1 --2- 172.21.15.94:0/3501600533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12a00a24b0 0x7f12a00a4970 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f12b806a590 tx=0x7f12b0003ce0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.089+0000 7f12bf155640 1 -- 172.21.15.94:0/3501600533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12b810fd30 msgr2=0x7f12b8113f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:06:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.089+0000 7f12bf155640 1 --2- 172.21.15.94:0/3501600533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12b810fd30 0x7f12b8113f60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f12a800c970 tx=0x7f12a800ce30 comp rx=0 tx=0).stop 2024-02-17T12:06:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.090+0000 7f12bf155640 1 -- 172.21.15.94:0/3501600533 shutdown_connections 2024-02-17T12:06:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.090+0000 7f12bf155640 1 --2- 172.21.15.94:0/3501600533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f12a00a24b0 0x7f12a00a4970 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.090+0000 7f12bf155640 1 --2- 172.21.15.94:0/3501600533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f12b811a960 0x7f12b81144a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.090+0000 7f12bf155640 1 --2- 172.21.15.94:0/3501600533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12b810fd30 0x7f12b8113f60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.090+0000 7f12bf155640 1 --2- 172.21.15.94:0/3501600533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f12b810f8c0 0x7f12b8119f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:06:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.090+0000 7f12bf155640 1 -- 172.21.15.94:0/3501600533 >> 172.21.15.94:0/3501600533 conn(0x7f12b806c220 msgr2=0x7f12b811b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:06:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.090+0000 7f12bf155640 1 -- 172.21.15.94:0/3501600533 shutdown_connections 2024-02-17T12:06:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:06:48.090+0000 7f12bf155640 1 -- 172.21.15.94:0/3501600533 wait complete. 2024-02-17T12:06:48.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:47 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:06:48.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:47 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:06:48.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:47 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:06:49.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:48 smithi189 ceph-mon[36588]: pgmap v268: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.7 MiB/s wr, 892 op/s 2024-02-17T12:06:49.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:48 smithi125 ceph-mon[36819]: pgmap v268: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.7 MiB/s wr, 892 op/s 2024-02-17T12:06:49.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:48 smithi094 ceph-mon[32499]: pgmap v268: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.7 MiB/s wr, 892 op/s 2024-02-17T12:06:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 510.0 s, 2092.6 tps, lat 48.314 ms stddev 53.556 2024-02-17T12:06:52.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:51 smithi189 ceph-mon[36588]: pgmap v269: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.1 MiB/s wr, 613 op/s 2024-02-17T12:06:52.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:51 smithi125 ceph-mon[36819]: pgmap v269: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.1 MiB/s wr, 613 op/s 2024-02-17T12:06:52.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:51 smithi094 ceph-mon[32499]: pgmap v269: 97 pgs: 97 active+clean; 5.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.1 MiB/s wr, 613 op/s 2024-02-17T12:06:54.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:53 smithi189 ceph-mon[36588]: pgmap v270: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.2 MiB/s wr, 698 op/s 2024-02-17T12:06:54.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:53 smithi125 ceph-mon[36819]: pgmap v270: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.2 MiB/s wr, 698 op/s 2024-02-17T12:06:54.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:53 smithi094 ceph-mon[32499]: pgmap v270: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.2 MiB/s wr, 698 op/s 2024-02-17T12:06:55.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 515.0 s, 2905.6 tps, lat 34.062 ms stddev 38.403 2024-02-17T12:06:55.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:54 smithi189 ceph-mon[36588]: pgmap v271: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 7.8 MiB/s wr, 894 op/s 2024-02-17T12:06:55.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:54 smithi125 ceph-mon[36819]: pgmap v271: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 7.8 MiB/s wr, 894 op/s 2024-02-17T12:06:55.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:54 smithi094 ceph-mon[32499]: pgmap v271: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 7.8 MiB/s wr, 894 op/s 2024-02-17T12:06:58.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:57 smithi189 ceph-mon[36588]: pgmap v272: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.5 MiB/s wr, 646 op/s 2024-02-17T12:06:58.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:57 smithi125 ceph-mon[36819]: pgmap v272: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.5 MiB/s wr, 646 op/s 2024-02-17T12:06:58.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:57 smithi094 ceph-mon[32499]: pgmap v272: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.5 MiB/s wr, 646 op/s 2024-02-17T12:07:00.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 520.0 s, 1555.4 tps, lat 64.896 ms stddev 71.091 2024-02-17T12:07:00.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:06:59 smithi189 ceph-mon[36588]: pgmap v273: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 8.4 MiB/s wr, 929 op/s 2024-02-17T12:07:00.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:06:59 smithi125 ceph-mon[36819]: pgmap v273: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 8.4 MiB/s wr, 929 op/s 2024-02-17T12:07:00.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:06:59 smithi094 ceph-mon[32499]: pgmap v273: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 8.4 MiB/s wr, 929 op/s 2024-02-17T12:07:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.235+0000 7f9ffbeeb640 1 -- 172.21.15.94:0/3249246118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff4103b90 msgr2=0x7f9ff4103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:01.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.235+0000 7f9ffbeeb640 1 --2- 172.21.15.94:0/3249246118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff4103b90 0x7f9ff4103ff0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009cd0 tx=0x7f9fec02c190 comp rx=0 tx=0).stop 2024-02-17T12:07:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.238+0000 7f9ffbeeb640 1 -- 172.21.15.94:0/3249246118 shutdown_connections 2024-02-17T12:07:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.238+0000 7f9ffbeeb640 1 --2- 172.21.15.94:0/3249246118 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff4103b90 0x7f9ff4103ff0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.238+0000 7f9ffbeeb640 1 --2- 172.21.15.94:0/3249246118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ff4068d40 0x7f9ff4069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.238+0000 7f9ffbeeb640 1 --2- 172.21.15.94:0/3249246118 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ff4106140 0x7f9ff4068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.238+0000 7f9ffbeeb640 1 -- 172.21.15.94:0/3249246118 >> 172.21.15.94:0/3249246118 conn(0x7f9ff4076ed0 msgr2=0x7f9ff40772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:01.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.244+0000 7f9ffbeeb640 1 -- 172.21.15.94:0/3249246118 shutdown_connections 2024-02-17T12:07:01.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.247+0000 7f9ffbeeb640 1 -- 172.21.15.94:0/3249246118 wait complete. 2024-02-17T12:07:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.248+0000 7f9ffbeeb640 1 Processor -- start 2024-02-17T12:07:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.249+0000 7f9ffbeeb640 1 -- start start 2024-02-17T12:07:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.249+0000 7f9ffbeeb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff4068d40 0x7f9ff41a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.249+0000 7f9ffbeeb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ff4103b90 0x7f9ff41a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.249+0000 7f9ffbeeb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ff4106140 0x7f9ff419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.249+0000 7f9ffbeeb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff419e0a0 con 0x7f9ff4103b90 2024-02-17T12:07:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.249+0000 7f9ffbeeb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff419e210 con 0x7f9ff4068d40 2024-02-17T12:07:01.251 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.249+0000 7f9ffbeeb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff419e3d0 con 0x7f9ff4106140 2024-02-17T12:07:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.251+0000 7f9ff9486640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ff4103b90 0x7f9ff41a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.251+0000 7f9ff9486640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ff4103b90 0x7f9ff41a3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46866/0 (socket says 172.21.15.94:46866) 2024-02-17T12:07:01.253 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.251+0000 7f9ff9486640 1 -- 172.21.15.94:0/3421185615 learned_addr learned my addr 172.21.15.94:0/3421185615 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.253+0000 7f9ff9486640 1 -- 172.21.15.94:0/3421185615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ff4106140 msgr2=0x7f9ff419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.253+0000 7f9ff9486640 1 --2- 172.21.15.94:0/3421185615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ff4106140 0x7f9ff419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.253+0000 7f9ff9486640 1 -- 172.21.15.94:0/3421185615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff4068d40 msgr2=0x7f9ff41a38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.253+0000 7f9ff9486640 1 --2- 172.21.15.94:0/3421185615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff4068d40 0x7f9ff41a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.253+0000 7f9ff9486640 1 -- 172.21.15.94:0/3421185615 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fec009900 con 0x7f9ff4103b90 2024-02-17T12:07:01.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.254+0000 7f9ff9486640 1 --2- 172.21.15.94:0/3421185615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ff4103b90 0x7f9ff41a3de0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400c890 tx=0x7f9fe400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.256+0000 7f9feaffd640 1 -- 172.21.15.94:0/3421185615 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4004220 con 0x7f9ff4103b90 2024-02-17T12:07:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.256+0000 7f9ffbeeb640 1 -- 172.21.15.94:0/3421185615 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff419e6b0 con 0x7f9ff4103b90 2024-02-17T12:07:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.256+0000 7f9ffbeeb640 1 -- 172.21.15.94:0/3421185615 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff406cd90 con 0x7f9ff4103b90 2024-02-17T12:07:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.257+0000 7f9feaffd640 1 -- 172.21.15.94:0/3421185615 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fe4007e90 con 0x7f9ff4103b90 2024-02-17T12:07:01.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.257+0000 7f9feaffd640 1 -- 172.21.15.94:0/3421185615 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4004a00 con 0x7f9ff4103b90 2024-02-17T12:07:01.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.262+0000 7f9feaffd640 1 -- 172.21.15.94:0/3421185615 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9fe4004b60 con 0x7f9ff4103b90 2024-02-17T12:07:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.263+0000 7f9ffeb98740 1 Processor -- start 2024-02-17T12:07:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.265+0000 7f9feaffd640 1 --2- 172.21.15.94:0/3421185615 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fe00a24b0 0x7f9fe00a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.266+0000 7f9feaffd640 1 -- 172.21.15.94:0/3421185615 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9fe40cee70 con 0x7f9ff4103b90 2024-02-17T12:07:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.272+0000 7f9ffeb98740 1 -- start start 2024-02-17T12:07:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.272+0000 7f9ffeb98740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d562ad0 0x56363d561820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.272+0000 7f9ffeb98740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d561d60 0x56363d5621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.272+0000 7f9ffeb98740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d56b2e0 0x56363d56d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.272+0000 7f9ffeb98740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363d4d09f0 con 0x56363d56b2e0 2024-02-17T12:07:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.272+0000 7f9ffeb98740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363d4cdf20 con 0x56363d561d60 2024-02-17T12:07:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.272+0000 7f9ffeb98740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363d4c9700 con 0x56363d562ad0 2024-02-17T12:07:01.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.273+0000 7f9ffa488640 1 --2- 172.21.15.94:0/3421185615 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fe00a24b0 0x7f9fe00a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.274+0000 7f9ff9c87640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d562ad0 0x56363d561820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:01.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.274+0000 7f9ff9486640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d561d60 0x56363d5621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.274+0000 7f9ff9c87640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d562ad0 0x56363d561820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:36346/0 (socket says 172.21.15.94:36346) 2024-02-17T12:07:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.274+0000 7f9ff9c87640 1 -- 172.21.15.94:0/2695441107 learned_addr learned my addr 172.21.15.94:0/2695441107 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:01.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.274+0000 7f9ffa488640 1 --2- 172.21.15.94:0/2695441107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d56b2e0 0x56363d56d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.277+0000 7f9ff9c87640 1 -- 172.21.15.94:0/2695441107 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d561d60 msgr2=0x56363d5621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.277+0000 7f9ff9c87640 1 --2- 172.21.15.94:0/2695441107 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d561d60 0x56363d5621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.277+0000 7f9ff9c87640 1 -- 172.21.15.94:0/2695441107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d56b2e0 msgr2=0x56363d56d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:01.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.277+0000 7f9ff9c87640 1 --2- 172.21.15.94:0/2695441107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d56b2e0 0x56363d56d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.277+0000 7f9ff9c87640 1 -- 172.21.15.94:0/2695441107 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56363d45f7b0 con 0x56363d562ad0 2024-02-17T12:07:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.280+0000 7f9ffa488640 1 --2- 172.21.15.94:0/3421185615 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fe00a24b0 0x7f9fe00a4970 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9fec02c7a0 tx=0x7f9fec032490 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.281+0000 7f9ff9c87640 1 --2- 172.21.15.94:0/2695441107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d562ad0 0x56363d561820 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000bae0 tx=0x7f9ff000d2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8402df6ba15caa6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.282+0000 7f9fdffff640 1 -- 172.21.15.94:0/2695441107 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff000dcb0 con 0x56363d562ad0 2024-02-17T12:07:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.283+0000 7f9fdffff640 1 -- 172.21.15.94:0/2695441107 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ff000de10 con 0x56363d562ad0 2024-02-17T12:07:01.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.283+0000 7f9fdffff640 1 -- 172.21.15.94:0/2695441107 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff00106c0 con 0x56363d562ad0 2024-02-17T12:07:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.284+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2695441107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d562ad0 msgr2=0x56363d561820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:01.285 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.284+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/2695441107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d562ad0 0x56363d561820 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000bae0 tx=0x7f9ff000d2e0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.287+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2695441107 shutdown_connections 2024-02-17T12:07:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.287+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/2695441107 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d56b2e0 0x56363d56d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.287+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/2695441107 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d561d60 0x56363d5621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.287+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/2695441107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d562ad0 0x56363d561820 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.287+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2695441107 >> 172.21.15.94:0/2695441107 conn(0x56363d529550 msgr2=0x56363d55ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:01.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.292+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2695441107 shutdown_connections 2024-02-17T12:07:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.295+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2695441107 wait complete. 2024-02-17T12:07:01.297 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.296+0000 7f9ffeb98740 1 -- start start 2024-02-17T12:07:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.296+0000 7f9ffeb98740 1 Processor -- start 2024-02-17T12:07:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.301+0000 7f9ffeb98740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 0x56363d5682d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.301+0000 7f9ffeb98740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d562ad0 0x56363d568810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.301+0000 7f9ffeb98740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d56b2e0 0x56363d568d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.301+0000 7f9ffeb98740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363d46ebf0 con 0x56363d56b2e0 2024-02-17T12:07:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.301+0000 7f9ffeb98740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363d19d250 con 0x56363d562ad0 2024-02-17T12:07:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.301+0000 7f9ffeb98740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363d44f200 con 0x56363d561d60 2024-02-17T12:07:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.301+0000 7f9ffeb98740 10 client.0 _ll_register_callbacks cb 0x56363d4bdd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.302+0000 7f9ff9486640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d562ad0 0x56363d568810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.302+0000 7f9ff9486640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d562ad0 0x56363d568810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:40304/0 (socket says 172.21.15.94:40304) 2024-02-17T12:07:01.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.302+0000 7f9ff9486640 1 -- 172.21.15.94:0/3867217518 learned_addr learned my addr 172.21.15.94:0/3867217518 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.304+0000 7f9ff9486640 1 -- 172.21.15.94:0/3867217518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 msgr2=0x56363d5682d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:07:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.304+0000 7f9ff9486640 1 --2- 172.21.15.94:0/3867217518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 0x56363d5682d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.304+0000 7f9ff9486640 1 -- 172.21.15.94:0/3867217518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d56b2e0 msgr2=0x56363d568d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.304+0000 7f9ff9486640 1 --2- 172.21.15.94:0/3867217518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d56b2e0 0x56363d568d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.304+0000 7f9ff9486640 1 -- 172.21.15.94:0/3867217518 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff000b740 con 0x56363d562ad0 2024-02-17T12:07:01.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.307+0000 7f9ff9486640 1 --2- 172.21.15.94:0/3867217518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d562ad0 0x56363d568810 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4097c20 tx=0x7f9fe409b1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90d8037fe83de31e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:01.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.309+0000 7f9fdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:07:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.311+0000 7f9fdd7fa640 1 -- 172.21.15.94:0/3867217518 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe40a3f60 con 0x56363d562ad0 2024-02-17T12:07:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.311+0000 7f9fdd7fa640 1 -- 172.21.15.94:0/3867217518 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9fe40cc7a0 con 0x56363d562ad0 2024-02-17T12:07:01.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.312+0000 7f9ffeb98740 1 -- 172.21.15.94:0/3867217518 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56363d1c9600 con 0x56363d562ad0 2024-02-17T12:07:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.318+0000 7f9fdd7fa640 1 -- 172.21.15.94:0/3867217518 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fe40cc960 con 0x56363d562ad0 2024-02-17T12:07:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.319+0000 7f9ffeb98740 10 client.27311 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.319+0000 7f9ffeb98740 1 -- 172.21.15.94:0/3867217518 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56363d4c7d80 con 0x56363d562ad0 2024-02-17T12:07:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.322+0000 7f9fdd7fa640 1 -- 172.21.15.94:0/3867217518 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56363d1c9600 con 0x56363d562ad0 2024-02-17T12:07:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.322+0000 7f9fdd7fa640 1 -- 172.21.15.94:0/3867217518 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9fe4016030 con 0x56363d562ad0 2024-02-17T12:07:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.322+0000 7f9ffeb98740 10 client.27311 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.322+0000 7f9ffeb98740 10 client.27311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.322+0000 7f9ffeb98740 20 client.27311 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.322+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/3867217518 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56363d58c6d0 0x56363d58eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.322+0000 7f9ffeb98740 4 client.27311 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.322+0000 7f9ffeb98740 1 -- 172.21.15.94:0/3867217518 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56363d21a1b0 con 0x56363d58c6d0 2024-02-17T12:07:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.326+0000 7f9ff9c87640 1 --2- 172.21.15.94:0/3867217518 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56363d58c6d0 0x56363d58eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.337+0000 7f9ff9c87640 1 --2- 172.21.15.94:0/3867217518 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56363d58c6d0 0x56363d58eaf0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6363f0d5df228c68 server_cookie=c086a57c8d9de176 in_seq=0 out_seq=0 2024-02-17T12:07:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.337+0000 7f9fdd7fa640 10 client.27311 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.340+0000 7f9fdd7fa640 1 -- 172.21.15.94:0/3867217518 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56363d21a1b0 con 0x56363d58c6d0 2024-02-17T12:07:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.340+0000 7f9fdd7fa640 10 client.27311 handle_command_reply: tid=0 2024-02-17T12:07:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.340+0000 7f9ffeb98740 1 client.27311 shutdown 2024-02-17T12:07:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.346+0000 7f9ffeb98740 1 -- 172.21.15.94:0/3867217518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d562ad0 msgr2=0x56363d568810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:01.347 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.346+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/3867217518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d562ad0 0x56363d568810 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4097c20 tx=0x7f9fe409b1e0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.349+0000 7f9ffeb98740 1 -- 172.21.15.94:0/3867217518 shutdown_connections 2024-02-17T12:07:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.349+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/3867217518 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56363d58c6d0 0x56363d58eaf0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.349+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/3867217518 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d56b2e0 0x56363d568d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.349+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/3867217518 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d562ad0 0x56363d568810 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.349+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/3867217518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 0x56363d5682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.349+0000 7f9ffeb98740 1 -- 172.21.15.94:0/3867217518 >> 172.21.15.94:0/3867217518 conn(0x56363d529550 msgr2=0x56363d55e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.351+0000 7f9fdd7fa640 0 client.27311 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.353+0000 7f9ffeb98740 1 -- 172.21.15.94:0/3867217518 shutdown_connections 2024-02-17T12:07:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.353+0000 7f9ffeb98740 1 -- 172.21.15.94:0/3867217518 wait complete. 2024-02-17T12:07:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.357+0000 7f9ffeb98740 20 client.27311 trim_cache size 0 max 16384 2024-02-17T12:07:01.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.387+0000 7f9ffeb98740 1 Processor -- start 2024-02-17T12:07:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.390+0000 7f9ffeb98740 1 -- start start 2024-02-17T12:07:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.390+0000 7f9ffeb98740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 0x56363d568820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.391 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.391+0000 7f9ffeb98740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d562ad0 0x56363d568d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.391+0000 7f9ffeb98740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d56b2e0 0x56363d565530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.391+0000 7f9ff9486640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d562ad0 0x56363d568d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.391+0000 7f9ff9486640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d562ad0 0x56363d568d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46886/0 (socket says 172.21.15.94:46886) 2024-02-17T12:07:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.391+0000 7f9ff9486640 1 -- 172.21.15.94:0/376789031 learned_addr learned my addr 172.21.15.94:0/376789031 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.391+0000 7f9ffeb98740 1 -- 172.21.15.94:0/376789031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363d5625a0 con 0x56363d562ad0 2024-02-17T12:07:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.391+0000 7f9ffeb98740 1 -- 172.21.15.94:0/376789031 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363d589130 con 0x56363d56b2e0 2024-02-17T12:07:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.391+0000 7f9ffeb98740 1 -- 172.21.15.94:0/376789031 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363d56db30 con 0x56363d561d60 2024-02-17T12:07:01.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.393+0000 7f9ffa488640 1 --2- 172.21.15.94:0/376789031 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d56b2e0 0x56363d565530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:01.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.393+0000 7f9ff9c87640 1 --2- 172.21.15.94:0/376789031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 0x56363d568820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.396+0000 7f9ff9486640 1 -- 172.21.15.94:0/376789031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 msgr2=0x56363d568820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.396+0000 7f9ff9486640 1 --2- 172.21.15.94:0/376789031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 0x56363d568820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.396+0000 7f9ff9486640 1 -- 172.21.15.94:0/376789031 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d56b2e0 msgr2=0x56363d565530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.396+0000 7f9ff9486640 1 --2- 172.21.15.94:0/376789031 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d56b2e0 0x56363d565530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.396+0000 7f9ff9486640 1 -- 172.21.15.94:0/376789031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56363d4647b0 con 0x56363d562ad0 2024-02-17T12:07:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.399+0000 7f9ff9486640 1 --2- 172.21.15.94:0/376789031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d562ad0 0x56363d568d60 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4005fd0 tx=0x7f9fe40bb050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28cbc2e20fcb3a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:01.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:01 smithi189 ceph-mon[36588]: pgmap v274: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 574 op/s 2024-02-17T12:07:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.401+0000 7f9fdf7fe640 1 -- 172.21.15.94:0/376789031 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4005290 con 0x56363d562ad0 2024-02-17T12:07:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.401+0000 7f9fdf7fe640 1 -- 172.21.15.94:0/376789031 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fe409b590 con 0x56363d562ad0 2024-02-17T12:07:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.401+0000 7f9fdf7fe640 1 -- 172.21.15.94:0/376789031 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4005070 con 0x56363d562ad0 2024-02-17T12:07:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.404+0000 7f9ffeb98740 1 -- 172.21.15.94:0/376789031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d562ad0 msgr2=0x56363d568d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.404+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/376789031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d562ad0 0x56363d568d60 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4005fd0 tx=0x7f9fe40bb050 comp rx=0 tx=0).stop 2024-02-17T12:07:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.408+0000 7f9ffeb98740 1 -- 172.21.15.94:0/376789031 shutdown_connections 2024-02-17T12:07:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.408+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/376789031 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d56b2e0 0x56363d565530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.408+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/376789031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d562ad0 0x56363d568d60 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.408+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/376789031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 0x56363d568820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.408+0000 7f9ffeb98740 1 -- 172.21.15.94:0/376789031 >> 172.21.15.94:0/376789031 conn(0x56363d529550 msgr2=0x56363d566ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.410+0000 7f9ffeb98740 1 -- 172.21.15.94:0/376789031 shutdown_connections 2024-02-17T12:07:01.411 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.411+0000 7f9ffeb98740 1 -- 172.21.15.94:0/376789031 wait complete. 2024-02-17T12:07:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.411+0000 7f9ffeb98740 1 -- start start 2024-02-17T12:07:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.412+0000 7f9ffeb98740 1 Processor -- start 2024-02-17T12:07:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.412+0000 7f9ffeb98740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 0x56363d572500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.413 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.412+0000 7f9ffeb98740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d562ad0 0x56363d5708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.413+0000 7f9ffeb98740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d56b2e0 0x56363d570de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.413+0000 7f9ffeb98740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363d5714e0 con 0x56363d562ad0 2024-02-17T12:07:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.413+0000 7f9ffeb98740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363d571650 con 0x56363d56b2e0 2024-02-17T12:07:01.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.413+0000 7f9ff9c87640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 0x56363d572500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.413+0000 7f9ff9c87640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 0x56363d572500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:36368/0 (socket says 172.21.15.94:36368) 2024-02-17T12:07:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.413+0000 7f9ff9c87640 1 -- 172.21.15.94:0/2292532673 learned_addr learned my addr 172.21.15.94:0/2292532673 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.413+0000 7f9ffa488640 1 --2- 172.21.15.94:0/2292532673 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d56b2e0 0x56363d570de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.413+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2292532673 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56363d5e2c20 con 0x56363d561d60 2024-02-17T12:07:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.413+0000 7f9ffeb98740 10 client.0 _ll_register_callbacks cb 0x7f9ff000b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.414+0000 7f9ff9c87640 1 -- 172.21.15.94:0/2292532673 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d56b2e0 msgr2=0x56363d570de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.414+0000 7f9ff9c87640 1 --2- 172.21.15.94:0/2292532673 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d56b2e0 0x56363d570de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.414+0000 7f9ff9c87640 1 -- 172.21.15.94:0/2292532673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d562ad0 msgr2=0x56363d5708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.414+0000 7f9ff9c87640 1 --2- 172.21.15.94:0/2292532673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d562ad0 0x56363d5708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.414+0000 7f9ff9c87640 1 -- 172.21.15.94:0/2292532673 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56363d4c7d80 con 0x56363d561d60 2024-02-17T12:07:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.414+0000 7f9ff9c87640 1 --2- 172.21.15.94:0/2292532673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 0x56363d572500 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0005d00 tx=0x7f9ff000ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bff47fd39a2ecbe server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:01.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.415+0000 7f9fdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:07:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.415+0000 7f9fdd7fa640 1 -- 172.21.15.94:0/2292532673 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff001f040 con 0x56363d561d60 2024-02-17T12:07:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.415+0000 7f9fdd7fa640 1 -- 172.21.15.94:0/2292532673 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9ff00049a0 con 0x56363d561d60 2024-02-17T12:07:01.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.415+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2292532673 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ff000de10 con 0x56363d561d60 2024-02-17T12:07:01.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.418+0000 7f9fdd7fa640 1 -- 172.21.15.94:0/2292532673 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ff0024420 con 0x56363d561d60 2024-02-17T12:07:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.421+0000 7f9ffeb98740 10 client.27325 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:01.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.422+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2292532673 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56363d48a540 con 0x56363d561d60 2024-02-17T12:07:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.425+0000 7f9fdd7fa640 1 -- 172.21.15.94:0/2292532673 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ff0014050 con 0x56363d561d60 2024-02-17T12:07:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.425+0000 7f9fdd7fa640 1 -- 172.21.15.94:0/2292532673 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9ff0004ed0 con 0x56363d561d60 2024-02-17T12:07:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.428+0000 7f9ffeb98740 10 client.27325 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.428+0000 7f9ffeb98740 10 client.27325 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.428+0000 7f9ffeb98740 20 client.27325 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.428+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/2292532673 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56363d5e4240 0x56363d5e6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.428+0000 7f9ffeb98740 4 client.27325 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:07:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.428+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2292532673 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f9ff000de10 con 0x56363d5e4240 2024-02-17T12:07:01.430 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.430+0000 7f9ff9486640 1 --2- 172.21.15.94:0/2292532673 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56363d5e4240 0x56363d5e6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.433+0000 7f9ff9486640 1 --2- 172.21.15.94:0/2292532673 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56363d5e4240 0x56363d5e6660 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3664e1d7d8bb8b88 server_cookie=1a610d62c9e7e454 in_seq=0 out_seq=0 2024-02-17T12:07:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:01.433+0000 7f9fdd7fa640 10 client.27325 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:01.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:01 smithi125 ceph-mon[36819]: pgmap v274: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 574 op/s 2024-02-17T12:07:01.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:01 smithi094 ceph-mon[32499]: pgmap v274: 97 pgs: 97 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 5.7 MiB/s wr, 574 op/s 2024-02-17T12:07:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9fdd7fa640 1 -- 172.21.15.94:0/2292532673 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f9ff000de10 con 0x56363d5e4240 2024-02-17T12:07:02.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9fdd7fa640 10 client.27325 handle_command_reply: tid=0 2024-02-17T12:07:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9ffeb98740 1 client.27325 shutdown 2024-02-17T12:07:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2292532673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 msgr2=0x56363d572500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/2292532673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 0x56363d572500 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0005d00 tx=0x7f9ff000ba10 comp rx=0 tx=0).stop 2024-02-17T12:07:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2292532673 shutdown_connections 2024-02-17T12:07:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/2292532673 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56363d5e4240 0x56363d5e6660 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/2292532673 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56363d56b2e0 0x56363d570de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9fdd7fa640 0 client.27325 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/2292532673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56363d562ad0 0x56363d5708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9ffeb98740 1 --2- 172.21.15.94:0/2292532673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56363d561d60 0x56363d572500 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2292532673 >> 172.21.15.94:0/2292532673 conn(0x56363d529550 msgr2=0x56363d55b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2292532673 shutdown_connections 2024-02-17T12:07:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9ffeb98740 1 -- 172.21.15.94:0/2292532673 wait complete. 2024-02-17T12:07:02.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.204+0000 7f9ffeb98740 20 client.27325 trim_cache size 0 max 16384 2024-02-17T12:07:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:07:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:07:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:07:02.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:07:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.205+0000 7f9ffbeeb640 1 -- 172.21.15.94:0/3421185615 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fe00a24b0 msgr2=0x7f9fe00a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.205+0000 7f9ffbeeb640 1 --2- 172.21.15.94:0/3421185615 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fe00a24b0 0x7f9fe00a4970 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9fec02c7a0 tx=0x7f9fec032490 comp rx=0 tx=0).stop 2024-02-17T12:07:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.205+0000 7f9ffbeeb640 1 -- 172.21.15.94:0/3421185615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ff4103b90 msgr2=0x7f9ff41a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.205+0000 7f9ffbeeb640 1 --2- 172.21.15.94:0/3421185615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ff4103b90 0x7f9ff41a3de0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400c890 tx=0x7f9fe400cd50 comp rx=0 tx=0).stop 2024-02-17T12:07:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.205+0000 7f9ffbeeb640 1 -- 172.21.15.94:0/3421185615 shutdown_connections 2024-02-17T12:07:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.205+0000 7f9ffbeeb640 1 --2- 172.21.15.94:0/3421185615 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fe00a24b0 0x7f9fe00a4970 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.205+0000 7f9ffbeeb640 1 --2- 172.21.15.94:0/3421185615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ff4106140 0x7f9ff419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:02.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.205+0000 7f9ffbeeb640 1 --2- 172.21.15.94:0/3421185615 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ff4103b90 0x7f9ff41a3de0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.205+0000 7f9ffbeeb640 1 --2- 172.21.15.94:0/3421185615 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ff4068d40 0x7f9ff41a38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.205+0000 7f9ffbeeb640 1 -- 172.21.15.94:0/3421185615 >> 172.21.15.94:0/3421185615 conn(0x7f9ff4076ed0 msgr2=0x7f9ff40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.205+0000 7f9ffbeeb640 1 -- 172.21.15.94:0/3421185615 shutdown_connections 2024-02-17T12:07:02.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:02.205+0000 7f9ffbeeb640 1 -- 172.21.15.94:0/3421185615 wait complete. 2024-02-17T12:07:03.686 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:03 smithi125 ceph-mon[36819]: osdmap e93: 12 total, 12 up, 12 in 2024-02-17T12:07:03.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:03 smithi125 ceph-mon[36819]: pgmap v276: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.0 MiB/s wr, 666 op/s 2024-02-17T12:07:03.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:03 smithi094 ceph-mon[32499]: osdmap e93: 12 total, 12 up, 12 in 2024-02-17T12:07:03.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:03 smithi094 ceph-mon[32499]: pgmap v276: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.0 MiB/s wr, 666 op/s 2024-02-17T12:07:03.902 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:03 smithi189 ceph-mon[36588]: osdmap e93: 12 total, 12 up, 12 in 2024-02-17T12:07:03.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:03 smithi189 ceph-mon[36588]: pgmap v276: 97 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 93 active+clean; 5.3 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 7.0 MiB/s wr, 666 op/s 2024-02-17T12:07:05.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 525.0 s, 1085.6 tps, lat 91.871 ms stddev 267.182 2024-02-17T12:07:06.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:05 smithi189 ceph-mon[36588]: pgmap v277: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 33 MiB/s wr, 1.93k op/s 2024-02-17T12:07:06.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:05 smithi125 ceph-mon[36819]: pgmap v277: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 33 MiB/s wr, 1.93k op/s 2024-02-17T12:07:06.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:05 smithi094 ceph-mon[32499]: pgmap v277: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 33 MiB/s wr, 1.93k op/s 2024-02-17T12:07:08.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:07 smithi189 ceph-mon[36588]: pgmap v278: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 33 MiB/s wr, 1.93k op/s 2024-02-17T12:07:08.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:07 smithi125 ceph-mon[36819]: pgmap v278: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 33 MiB/s wr, 1.93k op/s 2024-02-17T12:07:08.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:07 smithi094 ceph-mon[32499]: pgmap v278: 97 pgs: 11 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 66 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 33 MiB/s wr, 1.93k op/s 2024-02-17T12:07:10.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 530.0 s, 1915.2 tps, lat 51.941 ms stddev 59.651 2024-02-17T12:07:10.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:09 smithi189 ceph-mon[36588]: pgmap v279: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 32 MiB/s wr, 1.91k op/s 2024-02-17T12:07:10.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:09 smithi125 ceph-mon[36819]: pgmap v279: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 32 MiB/s wr, 1.91k op/s 2024-02-17T12:07:10.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:09 smithi094 ceph-mon[32499]: pgmap v279: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 32 MiB/s wr, 1.91k op/s 2024-02-17T12:07:11.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:11 smithi189 ceph-mon[36588]: osdmap e94: 12 total, 12 up, 12 in 2024-02-17T12:07:11.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:11 smithi189 ceph-mon[36588]: pgmap v281: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 38 MiB/s wr, 2.24k op/s 2024-02-17T12:07:11.686 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:11 smithi125 ceph-mon[36819]: osdmap e94: 12 total, 12 up, 12 in 2024-02-17T12:07:11.686 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:11 smithi125 ceph-mon[36819]: pgmap v281: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 38 MiB/s wr, 2.24k op/s 2024-02-17T12:07:11.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:11 smithi094 ceph-mon[32499]: osdmap e94: 12 total, 12 up, 12 in 2024-02-17T12:07:11.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:11 smithi094 ceph-mon[32499]: pgmap v281: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 38 MiB/s wr, 2.24k op/s 2024-02-17T12:07:13.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:07:13.986 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:07:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.143+0000 7f599644e640 1 -- 172.21.15.94:0/3856249502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5990108660 msgr2=0x7f5990110230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.143+0000 7f599644e640 1 --2- 172.21.15.94:0/3856249502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5990108660 0x7f5990110230 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5988009cd0 tx=0x7f598802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.143+0000 7f599644e640 1 -- 172.21.15.94:0/3856249502 shutdown_connections 2024-02-17T12:07:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.143+0000 7f599644e640 1 --2- 172.21.15.94:0/3856249502 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5990108660 0x7f5990110230 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.143+0000 7f599644e640 1 --2- 172.21.15.94:0/3856249502 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5990103f40 0x7f5990107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.143+0000 7f599644e640 1 --2- 172.21.15.94:0/3856249502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5990103590 0x7f5990103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.143+0000 7f599644e640 1 -- 172.21.15.94:0/3856249502 >> 172.21.15.94:0/3856249502 conn(0x7f59900fee70 msgr2=0x7f5990101290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.143+0000 7f599644e640 1 -- 172.21.15.94:0/3856249502 shutdown_connections 2024-02-17T12:07:14.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.143+0000 7f599644e640 1 -- 172.21.15.94:0/3856249502 wait complete. 2024-02-17T12:07:14.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.144+0000 7f599644e640 1 Processor -- start 2024-02-17T12:07:14.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.144+0000 7f599644e640 1 -- start start 2024-02-17T12:07:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f599644e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5990103590 0x7f5990075690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f599644e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5990108660 0x7f5990075bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f599644e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5990076110 0x7f59901aa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f599644e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5990079820 con 0x7f5990108660 2024-02-17T12:07:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f599644e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59901aa540 con 0x7f5990103590 2024-02-17T12:07:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f599644e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59901aa6d0 con 0x7f5990076110 2024-02-17T12:07:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f598f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5990108660 0x7f5990075bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f598f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5990108660 0x7f5990075bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46428/0 (socket says 172.21.15.94:46428) 2024-02-17T12:07:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f598f7fe640 1 -- 172.21.15.94:0/287567770 learned_addr learned my addr 172.21.15.94:0/287567770 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f598ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5990103590 0x7f5990075690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f59949eb640 1 --2- 172.21.15.94:0/287567770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5990076110 0x7f59901aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f598f7fe640 1 -- 172.21.15.94:0/287567770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5990076110 msgr2=0x7f59901aa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f598f7fe640 1 --2- 172.21.15.94:0/287567770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5990076110 0x7f59901aa000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f598f7fe640 1 -- 172.21.15.94:0/287567770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5990103590 msgr2=0x7f5990075690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f598f7fe640 1 --2- 172.21.15.94:0/287567770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5990103590 0x7f5990075690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.145+0000 7f598f7fe640 1 -- 172.21.15.94:0/287567770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5988009900 con 0x7f5990108660 2024-02-17T12:07:14.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.146+0000 7f598f7fe640 1 --2- 172.21.15.94:0/287567770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5990108660 0x7f5990075bd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f598400ea20 tx=0x7f598400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.146+0000 7f598d7fa640 1 -- 172.21.15.94:0/287567770 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984009700 con 0x7f5990108660 2024-02-17T12:07:14.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.146+0000 7f598d7fa640 1 -- 172.21.15.94:0/287567770 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f598400cd80 con 0x7f5990108660 2024-02-17T12:07:14.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.146+0000 7f599644e640 1 -- 172.21.15.94:0/287567770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59901aa980 con 0x7f5990108660 2024-02-17T12:07:14.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.146+0000 7f598d7fa640 1 -- 172.21.15.94:0/287567770 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5984010430 con 0x7f5990108660 2024-02-17T12:07:14.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.146+0000 7f599644e640 1 -- 172.21.15.94:0/287567770 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59901aae90 con 0x7f5990108660 2024-02-17T12:07:14.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.149+0000 7f598d7fa640 1 -- 172.21.15.94:0/287567770 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f59840105f0 con 0x7f5990108660 2024-02-17T12:07:14.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.149+0000 7f5999029740 1 Processor -- start 2024-02-17T12:07:14.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.149+0000 7f5999029740 1 -- start start 2024-02-17T12:07:14.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.149+0000 7f5999029740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9ddf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.149+0000 7f5999029740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.149+0000 7f5999029740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9de87b0 0x55e0a9deaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.149+0000 7f5999029740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a9d4e280 con 0x55e0a9ddfbd0 2024-02-17T12:07:14.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.149+0000 7f5999029740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a9d4b7b0 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.149+0000 7f5999029740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a9d46f90 con 0x55e0a9de87b0 2024-02-17T12:07:14.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.150+0000 7f598d7fa640 1 --2- 172.21.15.94:0/287567770 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59780a2230 0x7f59780a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.150+0000 7f598f7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9de87b0 0x55e0a9deaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.150+0000 7f59949eb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9ddf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.150+0000 7f598ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.150+0000 7f598ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46440/0 (socket says 172.21.15.94:46440) 2024-02-17T12:07:14.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.150+0000 7f598ffff640 1 -- 172.21.15.94:0/3738887486 learned_addr learned my addr 172.21.15.94:0/3738887486 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:14.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.150+0000 7f598d7fa640 1 -- 172.21.15.94:0/287567770 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5984014070 con 0x7f5990108660 2024-02-17T12:07:14.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.150+0000 7f59949eb640 1 --2- 172.21.15.94:0/287567770 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59780a2230 0x7f59780a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.150+0000 7f598ffff640 1 -- 172.21.15.94:0/3738887486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9de87b0 msgr2=0x55e0a9deaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.150+0000 7f598ffff640 1 --2- 172.21.15.94:0/3738887486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9de87b0 0x55e0a9deaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.150+0000 7f598ffff640 1 -- 172.21.15.94:0/3738887486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddf2b0 msgr2=0x55e0a9ddf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.150+0000 7f598ffff640 1 --2- 172.21.15.94:0/3738887486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9ddf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.150+0000 7f598ffff640 1 -- 172.21.15.94:0/3738887486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0a9d05570 con 0x55e0a9ddfbd0 2024-02-17T12:07:14.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.151+0000 7f598ffff640 1 --2- 172.21.15.94:0/3738887486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de8270 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5980009d20 tx=0x7f5980011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0791056f65f06f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.151+0000 7f59949eb640 1 --2- 172.21.15.94:0/287567770 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59780a2230 0x7f59780a46f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5988005cc0 tx=0x7f5988009360 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.151+0000 7f59767fc640 1 -- 172.21.15.94:0/3738887486 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980019630 con 0x55e0a9ddfbd0 2024-02-17T12:07:14.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.151+0000 7f59767fc640 1 -- 172.21.15.94:0/3738887486 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5980019c70 con 0x55e0a9ddfbd0 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.151+0000 7f59767fc640 1 -- 172.21.15.94:0/3738887486 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598000ec40 con 0x55e0a9ddfbd0 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.152+0000 7f5999029740 1 -- 172.21.15.94:0/3738887486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 msgr2=0x55e0a9de8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.152+0000 7f5999029740 1 --2- 172.21.15.94:0/3738887486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de8270 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f5980009d20 tx=0x7f5980011810 comp rx=0 tx=0).stop 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.152+0000 7f5999029740 1 -- 172.21.15.94:0/3738887486 shutdown_connections 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.152+0000 7f5999029740 1 --2- 172.21.15.94:0/3738887486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9de87b0 0x55e0a9deaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.152+0000 7f5999029740 1 --2- 172.21.15.94:0/3738887486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de8270 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.152+0000 7f5999029740 1 --2- 172.21.15.94:0/3738887486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9ddf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.152+0000 7f5999029740 1 -- 172.21.15.94:0/3738887486 >> 172.21.15.94:0/3738887486 conn(0x55e0a9dd9990 msgr2=0x55e0a9dd9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.152+0000 7f5999029740 1 -- 172.21.15.94:0/3738887486 shutdown_connections 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.152+0000 7f5999029740 1 -- 172.21.15.94:0/3738887486 wait complete. 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.152+0000 7f5999029740 1 -- start start 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.152+0000 7f5999029740 1 Processor -- start 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.153+0000 7f5999029740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9de8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.153+0000 7f5999029740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.153+0000 7f5999029740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9de87b0 0x55e0a9de5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.153+0000 7f5999029740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a9a1a430 con 0x55e0a9ddfbd0 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.153+0000 7f5999029740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a99d6e00 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.153+0000 7f5999029740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a9ccbfd0 con 0x55e0a9de87b0 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.153+0000 7f5999029740 10 client.0 _ll_register_callbacks cb 0x55e0a9d21fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.153+0000 7f598ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9de8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.153+0000 7f598ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9de8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44456/0 (socket says 172.21.15.94:44456) 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.153+0000 7f598ffff640 1 -- 172.21.15.94:0/1815946638 learned_addr learned my addr 172.21.15.94:0/1815946638 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.153+0000 7f598f7fe640 1 --2- 172.21.15.94:0/1815946638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.154+0000 7f598ffff640 1 -- 172.21.15.94:0/1815946638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9de87b0 msgr2=0x55e0a9de5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.154+0000 7f598ffff640 1 --2- 172.21.15.94:0/1815946638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9de87b0 0x55e0a9de5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.154+0000 7f598ffff640 1 -- 172.21.15.94:0/1815946638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 msgr2=0x55e0a9de54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.154+0000 7f598ffff640 1 --2- 172.21.15.94:0/1815946638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.154+0000 7f598ffff640 1 -- 172.21.15.94:0/1815946638 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5980009950 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.154+0000 7f598f7fe640 1 --2- 172.21.15.94:0/1815946638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:14.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.154+0000 7f598ffff640 1 --2- 172.21.15.94:0/1815946638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9de8160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5980004490 tx=0x7f59800044c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab55e985fd57cfac server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.154+0000 7f596f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:07:14.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.154+0000 7f596f7fe640 1 -- 172.21.15.94:0/1815946638 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598000ed90 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.155+0000 7f5999029740 1 -- 172.21.15.94:0/1815946638 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0a9a467e0 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.155+0000 7f596f7fe640 1 -- 172.21.15.94:0/1815946638 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f598000e3b0 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.155+0000 7f596f7fe640 1 -- 172.21.15.94:0/1815946638 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5980009ed0 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.155+0000 7f5999029740 10 client.27329 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:14.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.155+0000 7f5999029740 1 -- 172.21.15.94:0/1815946638 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e0a9d45610 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.155+0000 7f596f7fe640 1 -- 172.21.15.94:0/1815946638 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e0a9a467e0 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.156+0000 7f596f7fe640 1 -- 172.21.15.94:0/1815946638 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59800039f0 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.156+0000 7f5999029740 10 client.27329 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:14.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.156+0000 7f5999029740 10 client.27329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:14.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.156+0000 7f5999029740 20 client.27329 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:14.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.156+0000 7f5999029740 1 --2- 172.21.15.94:0/1815946638 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e0a9df88a0 0x55e0a9dfacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.156+0000 7f5999029740 4 client.27329 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:14.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.156+0000 7f5999029740 1 -- 172.21.15.94:0/1815946638 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0a9a83720 con 0x55e0a9df88a0 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.156+0000 7f598f7fe640 1 --2- 172.21.15.94:0/1815946638 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e0a9df88a0 0x55e0a9dfacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.156+0000 7f598f7fe640 1 --2- 172.21.15.94:0/1815946638 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e0a9df88a0 0x55e0a9dfacb0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21909692b112d68 server_cookie=d1f43b162ad8be35 in_seq=0 out_seq=0 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.156+0000 7f596f7fe640 10 client.27329 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f596f7fe640 1 -- 172.21.15.94:0/1815946638 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0a9a83720 con 0x55e0a9df88a0 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f596f7fe640 10 client.27329 handle_command_reply: tid=0 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f5999029740 1 client.27329 shutdown 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f5999029740 1 -- 172.21.15.94:0/1815946638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddf2b0 msgr2=0x55e0a9de8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f5999029740 1 --2- 172.21.15.94:0/1815946638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9de8160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5980004490 tx=0x7f59800044c0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f5999029740 1 -- 172.21.15.94:0/1815946638 shutdown_connections 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f5999029740 1 --2- 172.21.15.94:0/1815946638 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e0a9df88a0 0x55e0a9dfacb0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f5999029740 1 --2- 172.21.15.94:0/1815946638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9de87b0 0x55e0a9de5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f5999029740 1 --2- 172.21.15.94:0/1815946638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f5999029740 1 --2- 172.21.15.94:0/1815946638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9de8160 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f5999029740 1 -- 172.21.15.94:0/1815946638 >> 172.21.15.94:0/1815946638 conn(0x55e0a9dd9990 msgr2=0x55e0a9ddb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f596f7fe640 0 client.27329 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:14.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f5999029740 1 -- 172.21.15.94:0/1815946638 shutdown_connections 2024-02-17T12:07:14.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f5999029740 1 -- 172.21.15.94:0/1815946638 wait complete. 2024-02-17T12:07:14.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.157+0000 7f5999029740 20 client.27329 trim_cache size 0 max 16384 2024-02-17T12:07:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.176+0000 7f5999029740 1 Processor -- start 2024-02-17T12:07:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.176+0000 7f5999029740 1 -- start start 2024-02-17T12:07:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.176+0000 7f5999029740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9de7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.176+0000 7f5999029740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.176+0000 7f5999029740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9de87b0 0x55e0a9de0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.176+0000 7f5999029740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a9ddaa10 con 0x55e0a9ddfbd0 2024-02-17T12:07:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.176+0000 7f5999029740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a9dd3e20 con 0x55e0a9de87b0 2024-02-17T12:07:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.176+0000 7f5999029740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a9dda790 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.176+0000 7f598f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.177+0000 7f598f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46456/0 (socket says 172.21.15.94:46456) 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.177+0000 7f598f7fe640 1 -- 172.21.15.94:0/36238376 learned_addr learned my addr 172.21.15.94:0/36238376 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.177+0000 7f59949eb640 1 --2- 172.21.15.94:0/36238376 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9de87b0 0x55e0a9de0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.177+0000 7f598ffff640 1 --2- 172.21.15.94:0/36238376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9de7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.177+0000 7f598f7fe640 1 -- 172.21.15.94:0/36238376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9ddf2b0 msgr2=0x55e0a9de7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.177+0000 7f598f7fe640 1 --2- 172.21.15.94:0/36238376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9de7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.177+0000 7f598f7fe640 1 -- 172.21.15.94:0/36238376 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9de87b0 msgr2=0x55e0a9de0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.177+0000 7f598f7fe640 1 --2- 172.21.15.94:0/36238376 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9de87b0 0x55e0a9de0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.177+0000 7f598f7fe640 1 -- 172.21.15.94:0/36238376 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0a9ce1ac0 con 0x55e0a9ddfbd0 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.177+0000 7f598f7fe640 1 --2- 172.21.15.94:0/36238376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de8160 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5984097780 tx=0x7f5984004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b00d019b8d47e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.177+0000 7f5975ffb640 1 -- 172.21.15.94:0/36238376 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598409adb0 con 0x55e0a9ddfbd0 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.177+0000 7f5975ffb640 1 -- 172.21.15.94:0/36238376 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f598409ef00 con 0x55e0a9ddfbd0 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.177+0000 7f5975ffb640 1 -- 172.21.15.94:0/36238376 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59840cbd90 con 0x55e0a9ddfbd0 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.178+0000 7f5999029740 1 -- 172.21.15.94:0/36238376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 msgr2=0x55e0a9de8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.178+0000 7f5999029740 1 --2- 172.21.15.94:0/36238376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de8160 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5984097780 tx=0x7f5984004a50 comp rx=0 tx=0).stop 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.178+0000 7f5999029740 1 -- 172.21.15.94:0/36238376 shutdown_connections 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.178+0000 7f5999029740 1 --2- 172.21.15.94:0/36238376 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9de87b0 0x55e0a9de0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.178+0000 7f5999029740 1 --2- 172.21.15.94:0/36238376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9de8160 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.178+0000 7f5999029740 1 --2- 172.21.15.94:0/36238376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9de7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.178+0000 7f5999029740 1 -- 172.21.15.94:0/36238376 >> 172.21.15.94:0/36238376 conn(0x55e0a9dd9990 msgr2=0x55e0a9dff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.178+0000 7f5999029740 1 -- 172.21.15.94:0/36238376 shutdown_connections 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.178+0000 7f5999029740 1 -- 172.21.15.94:0/36238376 wait complete. 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.178+0000 7f5999029740 1 -- start start 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.179+0000 7f5999029740 1 Processor -- start 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.179+0000 7f5999029740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9debdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.179+0000 7f5999029740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9dec310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.179+0000 7f5999029740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9de87b0 0x55e0a9e46d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.179+0000 7f5999029740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a9e47450 con 0x55e0a9de87b0 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.179+0000 7f5999029740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a9e475c0 con 0x55e0a9ddfbd0 2024-02-17T12:07:14.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.179+0000 7f5999029740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0a9e47780 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.179+0000 7f5999029740 10 client.0 _ll_register_callbacks cb 0x7f5980009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:14.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.179+0000 7f59949eb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9de87b0 0x55e0a9e46d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.179+0000 7f59949eb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9de87b0 0x55e0a9e46d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46470/0 (socket says 172.21.15.94:46470) 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.179+0000 7f59949eb640 1 -- 172.21.15.94:0/2614027782 learned_addr learned my addr 172.21.15.94:0/2614027782 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.179+0000 7f598f7fe640 1 --2- 172.21.15.94:0/2614027782 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9dec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.179+0000 7f598ffff640 1 --2- 172.21.15.94:0/2614027782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9debdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.180+0000 7f598ffff640 1 -- 172.21.15.94:0/2614027782 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddfbd0 msgr2=0x55e0a9dec310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.180+0000 7f598ffff640 1 --2- 172.21.15.94:0/2614027782 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9dec310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.180+0000 7f598ffff640 1 -- 172.21.15.94:0/2614027782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9de87b0 msgr2=0x55e0a9e46d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.180+0000 7f598ffff640 1 --2- 172.21.15.94:0/2614027782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9de87b0 0x55e0a9e46d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.180+0000 7f598ffff640 1 -- 172.21.15.94:0/2614027782 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59901aae90 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.180+0000 7f598ffff640 1 --2- 172.21.15.94:0/2614027782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9debdd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5980009230 tx=0x7f5980003fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b3dbc5b373c0504 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.180+0000 7f596f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.180+0000 7f596f7fe640 1 -- 172.21.15.94:0/2614027782 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598000e3b0 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.180+0000 7f596f7fe640 1 -- 172.21.15.94:0/2614027782 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5980000e40 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.180+0000 7f5999029740 1 -- 172.21.15.94:0/2614027782 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5980019c70 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.180+0000 7f596f7fe640 1 -- 172.21.15.94:0/2614027782 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f598000e510 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.180+0000 7f5999029740 10 client.27340 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.180+0000 7f5999029740 1 -- 172.21.15.94:0/2614027782 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e0a9d076a0 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.181+0000 7f596f7fe640 1 -- 172.21.15.94:0/2614027782 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5980044020 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.181+0000 7f596f7fe640 1 -- 172.21.15.94:0/2614027782 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5980005430 con 0x55e0a9ddf2b0 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.181+0000 7f5999029740 10 client.27340 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.181+0000 7f5999029740 10 client.27340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.181+0000 7f5999029740 20 client.27340 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.181+0000 7f5999029740 1 --2- 172.21.15.94:0/2614027782 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e0a9e4fd90 0x55e0a9e521b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.181+0000 7f5999029740 4 client.27340 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.181+0000 7f5999029740 1 -- 172.21.15.94:0/2614027782 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e0a9ad2890 con 0x55e0a9e4fd90 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.182+0000 7f598f7fe640 1 --2- 172.21.15.94:0/2614027782 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e0a9e4fd90 0x55e0a9e521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.182+0000 7f598f7fe640 1 --2- 172.21.15.94:0/2614027782 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e0a9e4fd90 0x55e0a9e521b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f47c6359d2cf92ef server_cookie=ceadb0e29fbc6d76 in_seq=0 out_seq=0 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.182+0000 7f596f7fe640 10 client.27340 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.182+0000 7f596f7fe640 1 -- 172.21.15.94:0/2614027782 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e0a9ad2890 con 0x55e0a9e4fd90 2024-02-17T12:07:14.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.182+0000 7f596f7fe640 10 client.27340 handle_command_reply: tid=0 2024-02-17T12:07:14.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.182+0000 7f5999029740 1 client.27340 shutdown 2024-02-17T12:07:14.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f5999029740 1 -- 172.21.15.94:0/2614027782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9ddf2b0 msgr2=0x55e0a9debdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f5999029740 1 --2- 172.21.15.94:0/2614027782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9debdd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5980009230 tx=0x7f5980003fa0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f5999029740 1 -- 172.21.15.94:0/2614027782 shutdown_connections 2024-02-17T12:07:14.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f5999029740 1 --2- 172.21.15.94:0/2614027782 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e0a9e4fd90 0x55e0a9e521b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f5999029740 1 --2- 172.21.15.94:0/2614027782 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e0a9de87b0 0x55e0a9e46d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f5999029740 1 --2- 172.21.15.94:0/2614027782 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e0a9ddfbd0 0x55e0a9dec310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f596f7fe640 0 client.27340 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:14.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f5999029740 1 --2- 172.21.15.94:0/2614027782 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e0a9ddf2b0 0x55e0a9debdd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f5999029740 1 -- 172.21.15.94:0/2614027782 >> 172.21.15.94:0/2614027782 conn(0x55e0a9dd9990 msgr2=0x55e0a9de0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f5999029740 1 -- 172.21.15.94:0/2614027782 shutdown_connections 2024-02-17T12:07:14.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f5999029740 1 -- 172.21.15.94:0/2614027782 wait complete. 2024-02-17T12:07:14.188 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f5999029740 20 client.27340 trim_cache size 0 max 16384 2024-02-17T12:07:14.189 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:07:14.189 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:07:14.189 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:07:14.189 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:07:14.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f599644e640 1 -- 172.21.15.94:0/287567770 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59780a2230 msgr2=0x7f59780a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f599644e640 1 --2- 172.21.15.94:0/287567770 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59780a2230 0x7f59780a46f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5988005cc0 tx=0x7f5988009360 comp rx=0 tx=0).stop 2024-02-17T12:07:14.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f599644e640 1 -- 172.21.15.94:0/287567770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5990108660 msgr2=0x7f5990075bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f599644e640 1 --2- 172.21.15.94:0/287567770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5990108660 0x7f5990075bd0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f598400ea20 tx=0x7f598400eee0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f599644e640 1 -- 172.21.15.94:0/287567770 shutdown_connections 2024-02-17T12:07:14.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f599644e640 1 --2- 172.21.15.94:0/287567770 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59780a2230 0x7f59780a46f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f599644e640 1 --2- 172.21.15.94:0/287567770 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5990076110 0x7f59901aa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f599644e640 1 --2- 172.21.15.94:0/287567770 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5990108660 0x7f5990075bd0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f599644e640 1 --2- 172.21.15.94:0/287567770 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5990103590 0x7f5990075690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.183+0000 7f599644e640 1 -- 172.21.15.94:0/287567770 >> 172.21.15.94:0/287567770 conn(0x7f59900fee70 msgr2=0x7f5990101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.184+0000 7f599644e640 1 -- 172.21.15.94:0/287567770 shutdown_connections 2024-02-17T12:07:14.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.184+0000 7f599644e640 1 -- 172.21.15.94:0/287567770 wait complete. 2024-02-17T12:07:14.197 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:07:14.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:07:14.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.339+0000 7fa9f70eb640 1 -- 172.21.15.94:0/3760772131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9f01015e0 msgr2=0x7fa9f010cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.339+0000 7fa9f70eb640 1 --2- 172.21.15.94:0/3760772131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9f01015e0 0x7fa9f010cf90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009890 tx=0x7fa9e802bb70 comp rx=0 tx=0).stop 2024-02-17T12:07:14.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.339+0000 7fa9f70eb640 1 -- 172.21.15.94:0/3760772131 shutdown_connections 2024-02-17T12:07:14.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.339+0000 7fa9f70eb640 1 --2- 172.21.15.94:0/3760772131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9f01015e0 0x7fa9f010cf90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.339+0000 7fa9f70eb640 1 --2- 172.21.15.94:0/3760772131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f0100c20 0x7fa9f01010a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.339+0000 7fa9f70eb640 1 --2- 172.21.15.94:0/3760772131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f01050b0 0x7fa9f0105490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.339+0000 7fa9f70eb640 1 -- 172.21.15.94:0/3760772131 >> 172.21.15.94:0/3760772131 conn(0x7fa9f0075390 msgr2=0x7fa9f00757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.339+0000 7fa9f70eb640 1 -- 172.21.15.94:0/3760772131 shutdown_connections 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.339+0000 7fa9f70eb640 1 -- 172.21.15.94:0/3760772131 wait complete. 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.339+0000 7fa9f70eb640 1 Processor -- start 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.339+0000 7fa9f70eb640 1 -- start start 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f70eb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9f0100c20 0x7fa9f010b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f70eb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f01015e0 0x7fa9f010b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f70eb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f01050b0 0x7fa9f010bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f70eb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f01a9f30 con 0x7fa9f01050b0 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f70eb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f01aa0a0 con 0x7fa9f0100c20 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f70eb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f01aa260 con 0x7fa9f01015e0 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f5688640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f01050b0 0x7fa9f010bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f5688640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f01050b0 0x7fa9f010bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46500/0 (socket says 172.21.15.94:46500) 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f5688640 1 -- 172.21.15.94:0/2233760756 learned_addr learned my addr 172.21.15.94:0/2233760756 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f4e87640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9f0100c20 0x7fa9f010b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9e7fff640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f01015e0 0x7fa9f010b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f5688640 1 -- 172.21.15.94:0/2233760756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f01015e0 msgr2=0x7fa9f010b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f5688640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f01015e0 0x7fa9f010b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f5688640 1 -- 172.21.15.94:0/2233760756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9f0100c20 msgr2=0x7fa9f010b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f5688640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9f0100c20 0x7fa9f010b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f5688640 1 -- 172.21.15.94:0/2233760756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e8009490 con 0x7fa9f01050b0 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9e7fff640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f01015e0 0x7fa9f010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f4e87640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9f0100c20 0x7fa9f010b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f5688640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f01050b0 0x7fa9f010bd40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8037040 tx=0x7fa9e80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9e5ffb640 1 -- 172.21.15.94:0/2233760756 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80349d0 con 0x7fa9f01050b0 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9f70eb640 1 -- 172.21.15.94:0/2233760756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f01aa4e0 con 0x7fa9f01050b0 2024-02-17T12:07:14.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9e5ffb640 1 -- 172.21.15.94:0/2233760756 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa9e802c9f0 con 0x7fa9f01050b0 2024-02-17T12:07:14.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.340+0000 7fa9e5ffb640 1 -- 172.21.15.94:0/2233760756 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8032ed0 con 0x7fa9f01050b0 2024-02-17T12:07:14.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.341+0000 7fa9f70eb640 1 -- 172.21.15.94:0/2233760756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f01aa850 con 0x7fa9f01050b0 2024-02-17T12:07:14.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.341+0000 7fa9f70eb640 1 -- 172.21.15.94:0/2233760756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f00683c0 con 0x7fa9f01050b0 2024-02-17T12:07:14.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.343+0000 7fa9e5ffb640 1 -- 172.21.15.94:0/2233760756 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa9e803b040 con 0x7fa9f01050b0 2024-02-17T12:07:14.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.344+0000 7fa9e5ffb640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa9d80a2220 0x7fa9d80a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.344+0000 7fa9e5ffb640 1 -- 172.21.15.94:0/2233760756 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa9e80efff0 con 0x7fa9f01050b0 2024-02-17T12:07:14.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.344+0000 7fa9f4e87640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa9d80a2220 0x7fa9d80a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.345+0000 7fa9f4e87640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa9d80a2220 0x7fa9d80a46e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009fd0 tx=0x7fa9ec009720 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.348+0000 7fa9e5ffb640 1 -- 172.21.15.94:0/2233760756 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa9e80b8d50 con 0x7fa9f01050b0 2024-02-17T12:07:14.388 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:13 smithi125 ceph-mon[36819]: pgmap v282: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 31 MiB/s wr, 1.84k op/s 2024-02-17T12:07:14.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:13 smithi189 ceph-mon[36588]: pgmap v282: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 31 MiB/s wr, 1.84k op/s 2024-02-17T12:07:14.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:13 smithi094 ceph-mon[32499]: pgmap v282: 97 pgs: 97 active+clean; 5.0 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 31 MiB/s wr, 1.84k op/s 2024-02-17T12:07:14.668 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.667+0000 7fa9f70eb640 1 -- 172.21.15.94:0/2233760756 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9f01033e0 con 0x7fa9f01050b0 2024-02-17T12:07:14.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.671+0000 7fa9e5ffb640 1 -- 172.21.15.94:0/2233760756 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7fa9e804b090 con 0x7fa9f01050b0 2024-02-17T12:07:14.673 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:07:14.673 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:07:14.674 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:07:14.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.683+0000 7fa9f70eb640 1 -- 172.21.15.94:0/2233760756 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa9d80a2220 msgr2=0x7fa9d80a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.683+0000 7fa9f70eb640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa9d80a2220 0x7fa9d80a46e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009fd0 tx=0x7fa9ec009720 comp rx=0 tx=0).stop 2024-02-17T12:07:14.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.683+0000 7fa9f70eb640 1 -- 172.21.15.94:0/2233760756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f01050b0 msgr2=0x7fa9f010bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.683+0000 7fa9f70eb640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f01050b0 0x7fa9f010bd40 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8037040 tx=0x7fa9e80045e0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.683+0000 7fa9f70eb640 1 -- 172.21.15.94:0/2233760756 shutdown_connections 2024-02-17T12:07:14.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.683+0000 7fa9f70eb640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa9d80a2220 0x7fa9d80a46e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.683+0000 7fa9f70eb640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9f01050b0 0x7fa9f010bd40 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.683+0000 7fa9f70eb640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9f01015e0 0x7fa9f010b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.683+0000 7fa9f70eb640 1 --2- 172.21.15.94:0/2233760756 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9f0100c20 0x7fa9f010b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.683+0000 7fa9f70eb640 1 -- 172.21.15.94:0/2233760756 >> 172.21.15.94:0/2233760756 conn(0x7fa9f0075390 msgr2=0x7fa9f010fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.684+0000 7fa9f70eb640 1 -- 172.21.15.94:0/2233760756 shutdown_connections 2024-02-17T12:07:14.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.686+0000 7fa9f70eb640 1 -- 172.21.15.94:0/2233760756 wait complete. 2024-02-17T12:07:14.707 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:07:14.707 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:07:14.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.835+0000 7f514424e640 1 -- 172.21.15.94:0/4119758634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f513c100d20 msgr2=0x7f513c10ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.835+0000 7f514424e640 1 --2- 172.21.15.94:0/4119758634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f513c100d20 0x7f513c10ce70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5134009ca0 tx=0x7f513402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.835+0000 7f514424e640 1 -- 172.21.15.94:0/4119758634 shutdown_connections 2024-02-17T12:07:14.837 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.835+0000 7f514424e640 1 --2- 172.21.15.94:0/4119758634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f513c100d20 0x7f513c10ce70 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.835+0000 7f514424e640 1 --2- 172.21.15.94:0/4119758634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f513c100360 0x7f513c1007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.835+0000 7f514424e640 1 --2- 172.21.15.94:0/4119758634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513c106240 0x7f513c106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.835+0000 7f514424e640 1 -- 172.21.15.94:0/4119758634 >> 172.21.15.94:0/4119758634 conn(0x7f513c077060 msgr2=0x7f513c077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.836+0000 7f514424e640 1 -- 172.21.15.94:0/4119758634 shutdown_connections 2024-02-17T12:07:14.838 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.836+0000 7f514424e640 1 -- 172.21.15.94:0/4119758634 wait complete. 2024-02-17T12:07:14.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f514424e640 1 Processor -- start 2024-02-17T12:07:14.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f514424e640 1 -- start start 2024-02-17T12:07:14.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f514424e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f513c100360 0x7f513c10d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f514424e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f513c100d20 0x7f513c10ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f514424e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513c106240 0x7f513c10e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f514424e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c10f7c0 con 0x7f513c100360 2024-02-17T12:07:14.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f514424e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c1aa0a0 con 0x7f513c106240 2024-02-17T12:07:14.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f514424e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513c1aa1e0 con 0x7f513c100d20 2024-02-17T12:07:14.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f5141fea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f513c100360 0x7f513c10d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f5141fea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f513c100360 0x7f513c10d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42706/0 (socket says 172.21.15.94:42706) 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f5141fea640 1 -- 172.21.15.94:0/3645743114 learned_addr learned my addr 172.21.15.94:0/3645743114 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f51427eb640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513c106240 0x7f513c10e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f51417e9640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f513c100d20 0x7f513c10ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f5141fea640 1 -- 172.21.15.94:0/3645743114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f513c100d20 msgr2=0x7f513c10ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f5141fea640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f513c100d20 0x7f513c10ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f5141fea640 1 -- 172.21.15.94:0/3645743114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513c106240 msgr2=0x7f513c10e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f5141fea640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513c106240 0x7f513c10e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f5141fea640 1 -- 172.21.15.94:0/3645743114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5134009900 con 0x7f513c100360 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f51427eb640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513c106240 0x7f513c10e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.837+0000 7f5141fea640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f513c100360 0x7f513c10d870 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f513800e9f0 tx=0x7f513800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.838+0000 7f5132ffd640 1 -- 172.21.15.94:0/3645743114 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138009700 con 0x7f513c100360 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.838+0000 7f514424e640 1 -- 172.21.15.94:0/3645743114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513c1aa4c0 con 0x7f513c100360 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.838+0000 7f5132ffd640 1 -- 172.21.15.94:0/3645743114 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f513800cd80 con 0x7f513c100360 2024-02-17T12:07:14.843 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.838+0000 7f5132ffd640 1 -- 172.21.15.94:0/3645743114 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138018920 con 0x7f513c100360 2024-02-17T12:07:14.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.838+0000 7f514424e640 1 -- 172.21.15.94:0/3645743114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c1aaa20 con 0x7f513c100360 2024-02-17T12:07:14.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.841+0000 7f5132ffd640 1 -- 172.21.15.94:0/3645743114 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5138018a80 con 0x7f513c100360 2024-02-17T12:07:14.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.841+0000 7f5146e23740 1 Processor -- start 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.841+0000 7f5146e23740 1 -- start start 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.841+0000 7f5132ffd640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f51280a2230 0x7f51280a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5146e23740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9250 0x5575029f9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5146e23740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575029f9b70 0x557502a02210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5146e23740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557502a02750 0x557502a04b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5146e23740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575029b4960 con 0x557502a02750 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5146e23740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502968310 con 0x5575029f9250 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5146e23740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502965840 con 0x5575029f9b70 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5132ffd640 1 -- 172.21.15.94:0/3645743114 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5138014070 con 0x7f513c100360 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f51427eb640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f51280a2230 0x7f51280a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5141fea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9250 0x5575029f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f51427eb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557502a02750 0x557502a04b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f51417e9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575029f9b70 0x557502a02210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5141fea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9250 0x5575029f9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53544/0 (socket says 172.21.15.94:53544) 2024-02-17T12:07:14.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5141fea640 1 -- 172.21.15.94:0/1657957451 learned_addr learned my addr 172.21.15.94:0/1657957451 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:14.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5141fea640 1 -- 172.21.15.94:0/1657957451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575029f9b70 msgr2=0x557502a02210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5141fea640 1 --2- 172.21.15.94:0/1657957451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575029f9b70 0x557502a02210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5141fea640 1 -- 172.21.15.94:0/1657957451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557502a02750 msgr2=0x557502a04b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5141fea640 1 --2- 172.21.15.94:0/1657957451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557502a02750 0x557502a04b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5141fea640 1 -- 172.21.15.94:0/1657957451 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557502923060 con 0x5575029f9250 2024-02-17T12:07:14.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f51417e9640 1 --2- 172.21.15.94:0/1657957451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575029f9b70 0x557502a02210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:14.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f51427eb640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f51280a2230 0x7f51280a46f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5134006010 tx=0x7f5134032e00 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5141fea640 1 --2- 172.21.15.94:0/1657957451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9250 0x5575029f9630 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f513809add0 tx=0x7f513809a980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e658c05e0023ea0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5127fff640 1 -- 172.21.15.94:0/1657957451 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138019450 con 0x5575029f9250 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5127fff640 1 -- 172.21.15.94:0/1657957451 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f513809ae40 con 0x5575029f9250 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.842+0000 7f5127fff640 1 -- 172.21.15.94:0/1657957451 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51380cbd70 con 0x5575029f9250 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.843+0000 7f5146e23740 1 -- 172.21.15.94:0/1657957451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9250 msgr2=0x5575029f9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.843+0000 7f5146e23740 1 --2- 172.21.15.94:0/1657957451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9250 0x5575029f9630 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f513809add0 tx=0x7f513809a980 comp rx=0 tx=0).stop 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.843+0000 7f5146e23740 1 -- 172.21.15.94:0/1657957451 shutdown_connections 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.843+0000 7f5146e23740 1 --2- 172.21.15.94:0/1657957451 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557502a02750 0x557502a04b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.843+0000 7f5146e23740 1 --2- 172.21.15.94:0/1657957451 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575029f9b70 0x557502a02210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.843+0000 7f5146e23740 1 --2- 172.21.15.94:0/1657957451 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9250 0x5575029f9630 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.843+0000 7f5146e23740 1 -- 172.21.15.94:0/1657957451 >> 172.21.15.94:0/1657957451 conn(0x5575029f3980 msgr2=0x5575029f3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.843+0000 7f5146e23740 1 -- 172.21.15.94:0/1657957451 shutdown_connections 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.843+0000 7f5146e23740 1 -- 172.21.15.94:0/1657957451 wait complete. 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.844+0000 7f5146e23740 1 -- start start 2024-02-17T12:07:14.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.844+0000 7f5146e23740 1 Processor -- start 2024-02-17T12:07:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f5146e23740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9250 0x5575029ff6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f5146e23740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575029f9b70 0x5575029ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f5146e23740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557502a02750 0x557502a00170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f5146e23740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502961020 con 0x557502a02750 2024-02-17T12:07:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f5146e23740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502634430 con 0x5575029f9250 2024-02-17T12:07:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f5146e23740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575025f0e00 con 0x5575029f9b70 2024-02-17T12:07:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f51417e9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575029f9b70 0x5575029ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f5141fea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9250 0x5575029ff6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f51417e9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575029f9b70 0x5575029ffc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44768/0 (socket says 172.21.15.94:44768) 2024-02-17T12:07:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f51417e9640 1 -- 172.21.15.94:0/3086579044 learned_addr learned my addr 172.21.15.94:0/3086579044 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f5146e23740 10 client.0 _ll_register_callbacks cb 0x557502908cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f51427eb640 1 --2- 172.21.15.94:0/3086579044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557502a02750 0x557502a00170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f51417e9640 1 -- 172.21.15.94:0/3086579044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9250 msgr2=0x5575029ff6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f51417e9640 1 --2- 172.21.15.94:0/3086579044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9250 0x5575029ff6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f51417e9640 1 -- 172.21.15.94:0/3086579044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557502a02750 msgr2=0x557502a00170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f51417e9640 1 --2- 172.21.15.94:0/3086579044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557502a02750 0x557502a00170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.845+0000 7f51417e9640 1 -- 172.21.15.94:0/3086579044 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f513c1aaa20 con 0x5575029f9b70 2024-02-17T12:07:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.846+0000 7f51417e9640 1 --2- 172.21.15.94:0/3086579044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575029f9b70 0x5575029ffc30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f512c00bad0 tx=0x7f512c00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55b62cfc7faf9501 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.846+0000 7f51257fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:07:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.846+0000 7f51257fa640 1 -- 172.21.15.94:0/3086579044 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512c016070 con 0x5575029f9b70 2024-02-17T12:07:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.846+0000 7f51257fa640 1 -- 172.21.15.94:0/3086579044 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f512c003e20 con 0x5575029f9b70 2024-02-17T12:07:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.846+0000 7f5146e23740 1 -- 172.21.15.94:0/3086579044 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575026607e0 con 0x5575029f9b70 2024-02-17T12:07:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.846+0000 7f51257fa640 1 -- 172.21.15.94:0/3086579044 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f512c011440 con 0x5575029f9b70 2024-02-17T12:07:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.846+0000 7f5146e23740 10 client.27355 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.846+0000 7f5146e23740 1 -- 172.21.15.94:0/3086579044 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55750295f6a0 con 0x5575029f9b70 2024-02-17T12:07:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.846+0000 7f51257fa640 1 -- 172.21.15.94:0/3086579044 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f512c026020 con 0x5575029f9b70 2024-02-17T12:07:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.847+0000 7f51257fa640 1 -- 172.21.15.94:0/3086579044 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f512c01b760 con 0x5575029f9b70 2024-02-17T12:07:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.847+0000 7f5146e23740 10 client.27355 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.847+0000 7f5146e23740 10 client.27355 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:07:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.847+0000 7f5146e23740 20 client.27355 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.847+0000 7f5146e23740 1 --2- 172.21.15.94:0/3086579044 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557502a12b40 0x557502a14fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.847+0000 7f5146e23740 4 client.27355 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.847+0000 7f5146e23740 1 -- 172.21.15.94:0/3086579044 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55750269d720 con 0x557502a12b40 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.847+0000 7f5141fea640 1 --2- 172.21.15.94:0/3086579044 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557502a12b40 0x557502a14fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.848+0000 7f5141fea640 1 --2- 172.21.15.94:0/3086579044 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557502a12b40 0x557502a14fc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=966f6d4e0126e866 server_cookie=e4399904f1f786af in_seq=0 out_seq=0 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.848+0000 7f51257fa640 10 client.27355 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.848+0000 7f51257fa640 1 -- 172.21.15.94:0/3086579044 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55750269d720 con 0x557502a12b40 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.848+0000 7f51257fa640 10 client.27355 handle_command_reply: tid=0 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.848+0000 7f5146e23740 1 client.27355 shutdown 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.848+0000 7f5146e23740 1 -- 172.21.15.94:0/3086579044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575029f9b70 msgr2=0x5575029ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.848+0000 7f5146e23740 1 --2- 172.21.15.94:0/3086579044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575029f9b70 0x5575029ffc30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f512c00bad0 tx=0x7f512c00bf90 comp rx=0 tx=0).stop 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.849+0000 7f5146e23740 1 -- 172.21.15.94:0/3086579044 shutdown_connections 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.849+0000 7f5146e23740 1 --2- 172.21.15.94:0/3086579044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557502a02750 0x557502a00170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.849+0000 7f5146e23740 1 --2- 172.21.15.94:0/3086579044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5575029f9b70 0x5575029ffc30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.849+0000 7f5146e23740 1 --2- 172.21.15.94:0/3086579044 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557502a12b40 0x557502a14fc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.849+0000 7f5146e23740 1 --2- 172.21.15.94:0/3086579044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9250 0x5575029ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.849+0000 7f51257fa640 0 client.27355 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:07:14.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.849+0000 7f5146e23740 1 -- 172.21.15.94:0/3086579044 >> 172.21.15.94:0/3086579044 conn(0x5575029f3980 msgr2=0x5575029f56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.849+0000 7f5146e23740 1 -- 172.21.15.94:0/3086579044 shutdown_connections 2024-02-17T12:07:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.849+0000 7f5146e23740 1 -- 172.21.15.94:0/3086579044 wait complete. 2024-02-17T12:07:14.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.849+0000 7f5146e23740 20 client.27355 trim_cache size 0 max 16384 2024-02-17T12:07:14.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5146e23740 1 Processor -- start 2024-02-17T12:07:14.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5146e23740 1 -- start start 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5146e23740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 0x5575029ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5146e23740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9b70 0x557502a001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5146e23740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557502a02750 0x5575029fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5146e23740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575028e6130 con 0x5575029f9250 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5146e23740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502a0f5c0 con 0x5575029f9b70 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5146e23740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575029f8c90 con 0x557502a02750 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5141fea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 0x5575029ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f51427eb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557502a02750 0x5575029fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5141fea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 0x5575029ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42724/0 (socket says 172.21.15.94:42724) 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5141fea640 1 -- 172.21.15.94:0/2146702723 learned_addr learned my addr 172.21.15.94:0/2146702723 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f51417e9640 1 --2- 172.21.15.94:0/2146702723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9b70 0x557502a001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5141fea640 1 -- 172.21.15.94:0/2146702723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557502a02750 msgr2=0x5575029fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5141fea640 1 --2- 172.21.15.94:0/2146702723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557502a02750 0x5575029fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5141fea640 1 -- 172.21.15.94:0/2146702723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9b70 msgr2=0x557502a001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5141fea640 1 --2- 172.21.15.94:0/2146702723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9b70 0x557502a001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.866+0000 7f5141fea640 1 -- 172.21.15.94:0/2146702723 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575028fbc20 con 0x5575029f9250 2024-02-17T12:07:14.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.867+0000 7f5141fea640 1 --2- 172.21.15.94:0/2146702723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 0x5575029ffc80 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f51380cbba0 tx=0x7f513809aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fe3ce98aac728d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.867+0000 7f51277fe640 1 -- 172.21.15.94:0/2146702723 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5138099260 con 0x5575029f9250 2024-02-17T12:07:14.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.867+0000 7f51277fe640 1 -- 172.21.15.94:0/2146702723 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f51380b7070 con 0x5575029f9250 2024-02-17T12:07:14.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.867+0000 7f51277fe640 1 -- 172.21.15.94:0/2146702723 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51380974b0 con 0x5575029f9250 2024-02-17T12:07:14.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.868+0000 7f5146e23740 1 -- 172.21.15.94:0/2146702723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 msgr2=0x5575029ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.868+0000 7f5146e23740 1 --2- 172.21.15.94:0/2146702723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 0x5575029ffc80 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f51380cbba0 tx=0x7f513809aab0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.868+0000 7f5146e23740 1 -- 172.21.15.94:0/2146702723 shutdown_connections 2024-02-17T12:07:14.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.868+0000 7f5146e23740 1 --2- 172.21.15.94:0/2146702723 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557502a02750 0x5575029fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.868+0000 7f5146e23740 1 --2- 172.21.15.94:0/2146702723 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9b70 0x557502a001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.868+0000 7f5146e23740 1 --2- 172.21.15.94:0/2146702723 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 0x5575029ffc80 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.868+0000 7f5146e23740 1 -- 172.21.15.94:0/2146702723 >> 172.21.15.94:0/2146702723 conn(0x5575029f3980 msgr2=0x5575029faa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.868+0000 7f5146e23740 1 -- 172.21.15.94:0/2146702723 shutdown_connections 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.868+0000 7f5146e23740 1 -- 172.21.15.94:0/2146702723 wait complete. 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.868+0000 7f5146e23740 1 -- start start 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5146e23740 1 Processor -- start 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5146e23740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 0x557502a09b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5146e23740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9b70 0x557502a07eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5146e23740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557502a02750 0x557502a085f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5146e23740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502a08cf0 con 0x5575029f9250 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5146e23740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502a690b0 con 0x5575029f9b70 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5146e23740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557502a691f0 con 0x557502a02750 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5146e23740 10 client.0 _ll_register_callbacks cb 0x7f512c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5141fea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 0x557502a09b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5141fea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 0x557502a09b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42730/0 (socket says 172.21.15.94:42730) 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f51417e9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9b70 0x557502a07eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5141fea640 1 -- 172.21.15.94:0/291495644 learned_addr learned my addr 172.21.15.94:0/291495644 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:14.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f51427eb640 1 --2- 172.21.15.94:0/291495644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557502a02750 0x557502a085f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5141fea640 1 -- 172.21.15.94:0/291495644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557502a02750 msgr2=0x557502a085f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5141fea640 1 --2- 172.21.15.94:0/291495644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557502a02750 0x557502a085f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5141fea640 1 -- 172.21.15.94:0/291495644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9b70 msgr2=0x557502a07eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:14.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5141fea640 1 --2- 172.21.15.94:0/291495644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9b70 0x557502a07eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.869+0000 7f5141fea640 1 -- 172.21.15.94:0/291495644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557502923060 con 0x5575029f9250 2024-02-17T12:07:14.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.870+0000 7f5141fea640 1 --2- 172.21.15.94:0/291495644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 0x557502a09b10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f513800e9c0 tx=0x7f5138009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58a6bbc2f270f56 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:14.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.870+0000 7f51257fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:14.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.871+0000 7f51257fa640 1 -- 172.21.15.94:0/291495644 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51380a3530 con 0x5575029f9250 2024-02-17T12:07:14.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.871+0000 7f51257fa640 1 -- 172.21.15.94:0/291495644 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f51380b3040 con 0x5575029f9250 2024-02-17T12:07:14.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.871+0000 7f5146e23740 1 -- 172.21.15.94:0/291495644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f513809ae40 con 0x5575029f9250 2024-02-17T12:07:14.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.871+0000 7f51257fa640 1 -- 172.21.15.94:0/291495644 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f513809eee0 con 0x5575029f9250 2024-02-17T12:07:14.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.871+0000 7f5146e23740 10 client.17871 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.871+0000 7f5146e23740 1 -- 172.21.15.94:0/291495644 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5575029218c0 con 0x5575029f9250 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.872+0000 7f51257fa640 1 -- 172.21.15.94:0/291495644 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f513809ae40 con 0x5575029f9250 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.872+0000 7f51257fa640 1 -- 172.21.15.94:0/291495644 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f51380b3ac0 con 0x5575029f9250 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.872+0000 7f5146e23740 10 client.17871 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.872+0000 7f5146e23740 10 client.17871 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.872+0000 7f5146e23740 20 client.17871 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.872+0000 7f5146e23740 1 --2- 172.21.15.94:0/291495644 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557502a6a430 0x557502a6c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.872+0000 7f5146e23740 4 client.17871 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.872+0000 7f5146e23740 1 -- 172.21.15.94:0/291495644 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5575026ec890 con 0x557502a6a430 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.872+0000 7f51417e9640 1 --2- 172.21.15.94:0/291495644 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557502a6a430 0x557502a6c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.872+0000 7f51417e9640 1 --2- 172.21.15.94:0/291495644 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557502a6a430 0x557502a6c820 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfbd3dcf6358720f server_cookie=aa99f9339b0a1f92 in_seq=0 out_seq=0 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.872+0000 7f51257fa640 10 client.17871 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.873+0000 7f51257fa640 1 -- 172.21.15.94:0/291495644 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5575026ec890 con 0x557502a6a430 2024-02-17T12:07:14.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.873+0000 7f51257fa640 10 client.17871 handle_command_reply: tid=0 2024-02-17T12:07:14.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.873+0000 7f5146e23740 1 client.17871 shutdown 2024-02-17T12:07:14.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.873+0000 7f5146e23740 1 -- 172.21.15.94:0/291495644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 msgr2=0x557502a09b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.873+0000 7f5146e23740 1 --2- 172.21.15.94:0/291495644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 0x557502a09b10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f513800e9c0 tx=0x7f5138009310 comp rx=0 tx=0).stop 2024-02-17T12:07:14.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.873+0000 7f5146e23740 1 -- 172.21.15.94:0/291495644 shutdown_connections 2024-02-17T12:07:14.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.873+0000 7f5146e23740 1 --2- 172.21.15.94:0/291495644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557502a02750 0x557502a085f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.873+0000 7f5146e23740 1 --2- 172.21.15.94:0/291495644 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557502a6a430 0x557502a6c820 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.873+0000 7f5146e23740 1 --2- 172.21.15.94:0/291495644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5575029f9b70 0x557502a07eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.873+0000 7f5146e23740 1 --2- 172.21.15.94:0/291495644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5575029f9250 0x557502a09b10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.873+0000 7f5146e23740 1 -- 172.21.15.94:0/291495644 >> 172.21.15.94:0/291495644 conn(0x5575029f3980 msgr2=0x5575029f22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.873+0000 7f51257fa640 0 client.17871 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:07:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.874+0000 7f5146e23740 1 -- 172.21.15.94:0/291495644 shutdown_connections 2024-02-17T12:07:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.874+0000 7f5146e23740 1 -- 172.21.15.94:0/291495644 wait complete. 2024-02-17T12:07:14.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.874+0000 7f5146e23740 20 client.17871 trim_cache size 0 max 16384 2024-02-17T12:07:14.883 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:07:14.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.874+0000 7f514424e640 1 -- 172.21.15.94:0/3645743114 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f51280a2230 msgr2=0x7f51280a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.874+0000 7f514424e640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f51280a2230 0x7f51280a46f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5134006010 tx=0x7f5134032e00 comp rx=0 tx=0).stop 2024-02-17T12:07:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.874+0000 7f514424e640 1 -- 172.21.15.94:0/3645743114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f513c100360 msgr2=0x7f513c10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.874+0000 7f514424e640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f513c100360 0x7f513c10d870 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f513800e9f0 tx=0x7f513800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.874+0000 7f514424e640 1 -- 172.21.15.94:0/3645743114 shutdown_connections 2024-02-17T12:07:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.874+0000 7f514424e640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f51280a2230 0x7f51280a46f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.874+0000 7f514424e640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f513c106240 0x7f513c10e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.875+0000 7f514424e640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f513c100d20 0x7f513c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.875+0000 7f514424e640 1 --2- 172.21.15.94:0/3645743114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f513c100360 0x7f513c10d870 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.875+0000 7f514424e640 1 -- 172.21.15.94:0/3645743114 >> 172.21.15.94:0/3645743114 conn(0x7f513c077060 msgr2=0x7f513c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.875+0000 7f514424e640 1 -- 172.21.15.94:0/3645743114 shutdown_connections 2024-02-17T12:07:14.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:14.875+0000 7f514424e640 1 -- 172.21.15.94:0/3645743114 wait complete. 2024-02-17T12:07:14.906 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:07:14.907 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:07:15.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.052+0000 7f7104a23640 1 -- 172.21.15.94:0/3018557364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71001071d0 msgr2=0x7f71001075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.052+0000 7f7104a23640 1 --2- 172.21.15.94:0/3018557364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71001071d0 0x7f71001075b0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009930 tx=0x7f70f002bc30 comp rx=0 tx=0).stop 2024-02-17T12:07:15.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.053+0000 7f7104a23640 1 -- 172.21.15.94:0/3018557364 shutdown_connections 2024-02-17T12:07:15.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.053+0000 7f7104a23640 1 --2- 172.21.15.94:0/3018557364 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f710006a070 0x7f710010e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.053+0000 7f7104a23640 1 --2- 172.21.15.94:0/3018557364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71000696f0 0x7f7100069b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.053+0000 7f7104a23640 1 --2- 172.21.15.94:0/3018557364 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71001071d0 0x7f71001075b0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.053+0000 7f7104a23640 1 -- 172.21.15.94:0/3018557364 >> 172.21.15.94:0/3018557364 conn(0x7f7100077060 msgr2=0x7f7100077470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.053+0000 7f7104a23640 1 -- 172.21.15.94:0/3018557364 shutdown_connections 2024-02-17T12:07:15.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.053+0000 7f7104a23640 1 -- 172.21.15.94:0/3018557364 wait complete. 2024-02-17T12:07:15.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.053+0000 7f7104a23640 1 Processor -- start 2024-02-17T12:07:15.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.054+0000 7f7104a23640 1 -- start start 2024-02-17T12:07:15.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.054+0000 7f7104a23640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71000696f0 0x7f710019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.054+0000 7f7104a23640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710006a070 0x7f710019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.054+0000 7f7104a23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71001071d0 0x7f7100197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.054+0000 7f70f5d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710006a070 0x7f710019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.054+0000 7f70f5d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710006a070 0x7f710019dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44796/0 (socket says 172.21.15.94:44796) 2024-02-17T12:07:15.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.054+0000 7f70f5d9b640 1 -- 172.21.15.94:0/596467137 learned_addr learned my addr 172.21.15.94:0/596467137 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.054+0000 7f70fe59c640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71000696f0 0x7f710019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.054+0000 7f7104a23640 1 -- 172.21.15.94:0/596467137 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7100197dd0 con 0x7f71001071d0 2024-02-17T12:07:15.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.054+0000 7f7104a23640 1 -- 172.21.15.94:0/596467137 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7100197f40 con 0x7f71000696f0 2024-02-17T12:07:15.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.054+0000 7f7104a23640 1 -- 172.21.15.94:0/596467137 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7100198100 con 0x7f710006a070 2024-02-17T12:07:15.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.055+0000 7f70fed9d640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71001071d0 0x7f7100197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.055+0000 7f70fe59c640 1 -- 172.21.15.94:0/596467137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710006a070 msgr2=0x7f710019dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.055+0000 7f70fe59c640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710006a070 0x7f710019dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.055+0000 7f70fe59c640 1 -- 172.21.15.94:0/596467137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71001071d0 msgr2=0x7f7100197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.055+0000 7f70fe59c640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71001071d0 0x7f7100197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.055+0000 7f70fe59c640 1 -- 172.21.15.94:0/596467137 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f0009560 con 0x7f71000696f0 2024-02-17T12:07:15.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.055+0000 7f70fe59c640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71000696f0 0x7f710019d580 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f70f0036040 tx=0x7f70f002c640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.056+0000 7f70f7fff640 1 -- 172.21.15.94:0/596467137 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f00042d0 con 0x7f71000696f0 2024-02-17T12:07:15.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.056+0000 7f7104a23640 1 -- 172.21.15.94:0/596467137 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7100198380 con 0x7f71000696f0 2024-02-17T12:07:15.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.056+0000 7f70f7fff640 1 -- 172.21.15.94:0/596467137 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70f0004430 con 0x7f71000696f0 2024-02-17T12:07:15.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.056+0000 7f70f7fff640 1 -- 172.21.15.94:0/596467137 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0003bf0 con 0x7f71000696f0 2024-02-17T12:07:15.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.056+0000 7f7104a23640 1 -- 172.21.15.94:0/596467137 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71001a4500 con 0x7f71000696f0 2024-02-17T12:07:15.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f70f7fff640 1 -- 172.21.15.94:0/596467137 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f70f0003d50 con 0x7f71000696f0 2024-02-17T12:07:15.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f71075fb740 1 Processor -- start 2024-02-17T12:07:15.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f71075fb740 1 -- start start 2024-02-17T12:07:15.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f71075fb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 0x56261f880630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f71075fb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f889210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f71075fb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 0x56261f88bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f71075fb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261f83b960 con 0x56261f880250 2024-02-17T12:07:15.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f71075fb740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261f7ef310 con 0x56261f889750 2024-02-17T12:07:15.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f71075fb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261f7ec840 con 0x56261f880b70 2024-02-17T12:07:15.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f70fed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 0x56261f880630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f70fe59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f889210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f70fed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 0x56261f880630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42770/0 (socket says 172.21.15.94:42770) 2024-02-17T12:07:15.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f70fe59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f889210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44808/0 (socket says 172.21.15.94:44808) 2024-02-17T12:07:15.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f70fed9d640 1 -- 172.21.15.94:0/3863565666 learned_addr learned my addr 172.21.15.94:0/3863565666 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f70f5d9b640 1 --2- 172.21.15.94:0/3863565666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 0x56261f88bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.059+0000 7f70f7fff640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f70ec0a2230 0x7f70ec0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.060+0000 7f70fe59c640 1 -- 172.21.15.94:0/3863565666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 msgr2=0x56261f88bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.060+0000 7f70fe59c640 1 --2- 172.21.15.94:0/3863565666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 0x56261f88bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.060+0000 7f70fe59c640 1 -- 172.21.15.94:0/3863565666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 msgr2=0x56261f880630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.060+0000 7f70fe59c640 1 --2- 172.21.15.94:0/3863565666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 0x56261f880630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.060+0000 7f70fe59c640 1 -- 172.21.15.94:0/3863565666 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56261f7aa060 con 0x56261f880b70 2024-02-17T12:07:15.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.060+0000 7f70fed9d640 1 --2- 172.21.15.94:0/3863565666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 0x56261f880630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:15.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.060+0000 7f70fed9d640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f70ec0a2230 0x7f70ec0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.060+0000 7f70f7fff640 1 -- 172.21.15.94:0/596467137 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f70f00f06c0 con 0x7f71000696f0 2024-02-17T12:07:15.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.060+0000 7f70fe59c640 1 --2- 172.21.15.94:0/3863565666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f889210 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f70f00edee0 tx=0x7f70f004f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60bdcb3695db4cbb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.060+0000 7f70fed9d640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f70ec0a2230 0x7f70ec0a46f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f70f8013780 tx=0x7f70f8004a40 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.060+0000 7f70f4d99640 1 -- 172.21.15.94:0/3863565666 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f00bccc0 con 0x56261f880b70 2024-02-17T12:07:15.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.060+0000 7f70f4d99640 1 -- 172.21.15.94:0/3863565666 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70f00bcae0 con 0x56261f880b70 2024-02-17T12:07:15.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.060+0000 7f70f4d99640 1 -- 172.21.15.94:0/3863565666 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f003b9b0 con 0x56261f880b70 2024-02-17T12:07:15.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.061+0000 7f71075fb740 1 -- 172.21.15.94:0/3863565666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 msgr2=0x56261f889210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.061+0000 7f71075fb740 1 --2- 172.21.15.94:0/3863565666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f889210 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f70f00edee0 tx=0x7f70f004f000 comp rx=0 tx=0).stop 2024-02-17T12:07:15.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.062+0000 7f71075fb740 1 -- 172.21.15.94:0/3863565666 shutdown_connections 2024-02-17T12:07:15.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.062+0000 7f71075fb740 1 --2- 172.21.15.94:0/3863565666 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 0x56261f88bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.062+0000 7f71075fb740 1 --2- 172.21.15.94:0/3863565666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f889210 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.062+0000 7f71075fb740 1 --2- 172.21.15.94:0/3863565666 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 0x56261f880630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.062+0000 7f71075fb740 1 -- 172.21.15.94:0/3863565666 >> 172.21.15.94:0/3863565666 conn(0x56261f87a980 msgr2=0x56261f87ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.062+0000 7f71075fb740 1 -- 172.21.15.94:0/3863565666 shutdown_connections 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.062+0000 7f71075fb740 1 -- 172.21.15.94:0/3863565666 wait complete. 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.062+0000 7f71075fb740 1 -- start start 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.062+0000 7f71075fb740 1 Processor -- start 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.062+0000 7f71075fb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 0x56261f8866d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.062+0000 7f71075fb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f886c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.062+0000 7f71075fb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 0x56261f887150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.062+0000 7f71075fb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261f7e8020 con 0x56261f880250 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.063+0000 7f71075fb740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261f4bb430 con 0x56261f889750 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.063+0000 7f71075fb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261f477e00 con 0x56261f880b70 2024-02-17T12:07:15.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.063+0000 7f71075fb740 10 client.0 _ll_register_callbacks cb 0x56261f78fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.063+0000 7f70fe59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 0x56261f8866d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.063+0000 7f70f5d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f886c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.063+0000 7f70fe59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 0x56261f8866d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42778/0 (socket says 172.21.15.94:42778) 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.063+0000 7f70fe59c640 1 -- 172.21.15.94:0/3367124631 learned_addr learned my addr 172.21.15.94:0/3367124631 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.063+0000 7f70fed9d640 1 --2- 172.21.15.94:0/3367124631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 0x56261f887150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.063+0000 7f70fe59c640 1 -- 172.21.15.94:0/3367124631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 msgr2=0x56261f886c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.063+0000 7f70fe59c640 1 --2- 172.21.15.94:0/3367124631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f886c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.063+0000 7f70fe59c640 1 -- 172.21.15.94:0/3367124631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 msgr2=0x56261f887150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.063+0000 7f70fe59c640 1 --2- 172.21.15.94:0/3367124631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 0x56261f887150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.063+0000 7f70fe59c640 1 -- 172.21.15.94:0/3367124631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71001a4500 con 0x56261f880250 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.064+0000 7f70e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.064+0000 7f70fe59c640 1 --2- 172.21.15.94:0/3367124631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 0x56261f8866d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f70f00edf90 tx=0x7f70f00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da013c53a6590c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.064+0000 7f70e5ffb640 1 -- 172.21.15.94:0/3367124631 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f00bccc0 con 0x56261f880250 2024-02-17T12:07:15.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.064+0000 7f70e5ffb640 1 -- 172.21.15.94:0/3367124631 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f70f00bd180 con 0x56261f880250 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.064+0000 7f71075fb740 1 -- 172.21.15.94:0/3367124631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56261f4e77e0 con 0x56261f880250 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.064+0000 7f70e5ffb640 1 -- 172.21.15.94:0/3367124631 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70f00bccc0 con 0x56261f880250 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.065+0000 7f71075fb740 10 client.17883 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.065+0000 7f71075fb740 1 -- 172.21.15.94:0/3367124631 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56261f7e66a0 con 0x56261f880250 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.065+0000 7f70e5ffb640 1 -- 172.21.15.94:0/3367124631 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56261f4e77e0 con 0x56261f880250 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.065+0000 7f70e5ffb640 1 -- 172.21.15.94:0/3367124631 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f70f00f8070 con 0x56261f880250 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.065+0000 7f71075fb740 10 client.17883 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.065+0000 7f71075fb740 10 client.17883 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.065+0000 7f71075fb740 20 client.17883 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.065+0000 7f71075fb740 1 --2- 172.21.15.94:0/3367124631 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56261f899330 0x56261f89b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.065+0000 7f71075fb740 4 client.17883 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.065+0000 7f71075fb740 1 -- 172.21.15.94:0/3367124631 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56261f524720 con 0x56261f899330 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.065+0000 7f70f5d9b640 1 --2- 172.21.15.94:0/3367124631 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56261f899330 0x56261f89b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.066+0000 7f70e5ffb640 10 client.17883 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:07:15.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.066+0000 7f70f5d9b640 1 --2- 172.21.15.94:0/3367124631 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56261f899330 0x56261f89b720 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4c496ab39f0cb39 server_cookie=d43457c1200c8af2 in_seq=0 out_seq=0 2024-02-17T12:07:15.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.066+0000 7f70e5ffb640 1 -- 172.21.15.94:0/3367124631 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56261f524720 con 0x56261f899330 2024-02-17T12:07:15.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.066+0000 7f70e5ffb640 10 client.17883 handle_command_reply: tid=0 2024-02-17T12:07:15.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.066+0000 7f71075fb740 1 client.17883 shutdown 2024-02-17T12:07:15.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.067+0000 7f71075fb740 1 -- 172.21.15.94:0/3367124631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 msgr2=0x56261f8866d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.067+0000 7f71075fb740 1 --2- 172.21.15.94:0/3367124631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 0x56261f8866d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f70f00edf90 tx=0x7f70f00035f0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.067+0000 7f71075fb740 1 -- 172.21.15.94:0/3367124631 shutdown_connections 2024-02-17T12:07:15.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.067+0000 7f71075fb740 1 --2- 172.21.15.94:0/3367124631 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56261f899330 0x56261f89b720 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.067+0000 7f71075fb740 1 --2- 172.21.15.94:0/3367124631 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 0x56261f887150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.067+0000 7f70e5ffb640 0 client.17883 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:07:15.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.067+0000 7f71075fb740 1 --2- 172.21.15.94:0/3367124631 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f886c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.067+0000 7f71075fb740 1 --2- 172.21.15.94:0/3367124631 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880250 0x56261f8866d0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.067+0000 7f71075fb740 1 -- 172.21.15.94:0/3367124631 >> 172.21.15.94:0/3367124631 conn(0x56261f87a980 msgr2=0x56261f87c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.067+0000 7f71075fb740 1 -- 172.21.15.94:0/3367124631 shutdown_connections 2024-02-17T12:07:15.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.067+0000 7f71075fb740 1 -- 172.21.15.94:0/3367124631 wait complete. 2024-02-17T12:07:15.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.067+0000 7f71075fb740 20 client.17883 trim_cache size 0 max 16384 2024-02-17T12:07:15.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f71075fb740 1 Processor -- start 2024-02-17T12:07:15.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f71075fb740 1 -- start start 2024-02-17T12:07:15.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f71075fb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880250 0x56261f885830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f71075fb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880b70 0x56261f886f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f71075fb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 0x56261f879a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f71075fb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261f76d130 con 0x56261f880b70 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f71075fb740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261f8962e0 con 0x56261f889750 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f71075fb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261f87b980 con 0x56261f880250 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f70f5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880b70 0x56261f886f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f70fed9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 0x56261f879a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f70f5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880b70 0x56261f886f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42794/0 (socket says 172.21.15.94:42794) 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f70f5d9b640 1 -- 172.21.15.94:0/309141938 learned_addr learned my addr 172.21.15.94:0/309141938 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f70fe59c640 1 --2- 172.21.15.94:0/309141938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880250 0x56261f885830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f70f5d9b640 1 -- 172.21.15.94:0/309141938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880250 msgr2=0x56261f885830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f70f5d9b640 1 --2- 172.21.15.94:0/309141938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880250 0x56261f885830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f70f5d9b640 1 -- 172.21.15.94:0/309141938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 msgr2=0x56261f879a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f70f5d9b640 1 --2- 172.21.15.94:0/309141938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 0x56261f879a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f70f5d9b640 1 -- 172.21.15.94:0/309141938 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56261f782c20 con 0x56261f880b70 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f70fed9d640 1 --2- 172.21.15.94:0/309141938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 0x56261f879a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:15.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.083+0000 7f70f5d9b640 1 --2- 172.21.15.94:0/309141938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880b70 0x56261f886f70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009ce0 tx=0x7f70e8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea22480f38cc43a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.084+0000 7f70e7fff640 1 -- 172.21.15.94:0/309141938 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8017030 con 0x56261f880b70 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.084+0000 7f70e7fff640 1 -- 172.21.15.94:0/309141938 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70e8018040 con 0x56261f880b70 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.084+0000 7f70e7fff640 1 -- 172.21.15.94:0/309141938 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8020360 con 0x56261f880b70 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.084+0000 7f71075fb740 1 -- 172.21.15.94:0/309141938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880b70 msgr2=0x56261f886f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.084+0000 7f71075fb740 1 --2- 172.21.15.94:0/309141938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880b70 0x56261f886f70 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009ce0 tx=0x7f70e8005f90 comp rx=0 tx=0).stop 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.084+0000 7f71075fb740 1 -- 172.21.15.94:0/309141938 shutdown_connections 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.084+0000 7f71075fb740 1 --2- 172.21.15.94:0/309141938 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f889750 0x56261f879a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.084+0000 7f71075fb740 1 --2- 172.21.15.94:0/309141938 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f880b70 0x56261f886f70 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.084+0000 7f71075fb740 1 --2- 172.21.15.94:0/309141938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880250 0x56261f885830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.084+0000 7f71075fb740 1 -- 172.21.15.94:0/309141938 >> 172.21.15.94:0/309141938 conn(0x56261f87a980 msgr2=0x56261f8792f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.084+0000 7f71075fb740 1 -- 172.21.15.94:0/309141938 shutdown_connections 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.084+0000 7f71075fb740 1 -- 172.21.15.94:0/309141938 wait complete. 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f71075fb740 1 -- start start 2024-02-17T12:07:15.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f71075fb740 1 Processor -- start 2024-02-17T12:07:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f71075fb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f880250 0x56261f88ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f71075fb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f88f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f71075fb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f889750 0x56261f88f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f71075fb740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261f8ef7e0 con 0x56261f889750 2024-02-17T12:07:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f71075fb740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261f8ef920 con 0x56261f880250 2024-02-17T12:07:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f71075fb740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261f8efa90 con 0x56261f880b70 2024-02-17T12:07:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f71075fb740 10 client.0 _ll_register_callbacks cb 0x7f70f00f6420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:15.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f70fed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f889750 0x56261f88f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f70fed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f889750 0x56261f88f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42798/0 (socket says 172.21.15.94:42798) 2024-02-17T12:07:15.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f70f5d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f88f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f70fed9d640 1 -- 172.21.15.94:0/2480937059 learned_addr learned my addr 172.21.15.94:0/2480937059 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f70fe59c640 1 --2- 172.21.15.94:0/2480937059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f880250 0x56261f88ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f70fed9d640 1 -- 172.21.15.94:0/2480937059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 msgr2=0x56261f88f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f70fed9d640 1 --2- 172.21.15.94:0/2480937059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f88f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f70fed9d640 1 -- 172.21.15.94:0/2480937059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f880250 msgr2=0x56261f88ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f70fed9d640 1 --2- 172.21.15.94:0/2480937059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f880250 0x56261f88ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.085+0000 7f70fed9d640 1 -- 172.21.15.94:0/2480937059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f80098e0 con 0x56261f889750 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.086+0000 7f70fed9d640 1 --2- 172.21.15.94:0/2480937059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f889750 0x56261f88f910 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f70f8010f20 tx=0x7f70f8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a70f303f101a7ee server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.086+0000 7f70e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.086+0000 7f70e5ffb640 1 -- 172.21.15.94:0/2480937059 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f801f830 con 0x56261f889750 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.086+0000 7f70e5ffb640 1 -- 172.21.15.94:0/2480937059 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f70f801f990 con 0x56261f889750 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.086+0000 7f71075fb740 1 -- 172.21.15.94:0/2480937059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70f00bcae0 con 0x56261f889750 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.087+0000 7f70e5ffb640 1 -- 172.21.15.94:0/2480937059 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70f801fb50 con 0x56261f889750 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.087+0000 7f71075fb740 10 client.17889 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.087+0000 7f71075fb740 1 -- 172.21.15.94:0/2480937059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f70e8009b50 con 0x56261f889750 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.087+0000 7f70e5ffb640 1 -- 172.21.15.94:0/2480937059 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70f00bcae0 con 0x56261f889750 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.087+0000 7f70e5ffb640 1 -- 172.21.15.94:0/2480937059 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f70f801e8b0 con 0x56261f889750 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.087+0000 7f71075fb740 10 client.17889 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.087+0000 7f71075fb740 10 client.17889 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.087+0000 7f71075fb740 20 client.17889 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.087+0000 7f71075fb740 1 --2- 172.21.15.94:0/2480937059 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56261f899330 0x56261f8f3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.087+0000 7f71075fb740 4 client.17889 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.087+0000 7f71075fb740 1 -- 172.21.15.94:0/2480937059 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56261f573890 con 0x56261f899330 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.088+0000 7f70fe59c640 1 --2- 172.21.15.94:0/2480937059 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56261f899330 0x56261f8f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.088+0000 7f70e5ffb640 10 client.17889 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.088+0000 7f70fe59c640 1 --2- 172.21.15.94:0/2480937059 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56261f899330 0x56261f8f3340 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8cdf778512efd16 server_cookie=4347576f745ea057 in_seq=0 out_seq=0 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.089+0000 7f70e5ffb640 1 -- 172.21.15.94:0/2480937059 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56261f573890 con 0x56261f899330 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.089+0000 7f70e5ffb640 10 client.17889 handle_command_reply: tid=0 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.089+0000 7f71075fb740 1 client.17889 shutdown 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.089+0000 7f71075fb740 1 -- 172.21.15.94:0/2480937059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f889750 msgr2=0x56261f88f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.089+0000 7f71075fb740 1 --2- 172.21.15.94:0/2480937059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f889750 0x56261f88f910 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f70f8010f20 tx=0x7f70f8004ab0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.089+0000 7f71075fb740 1 -- 172.21.15.94:0/2480937059 shutdown_connections 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.089+0000 7f71075fb740 1 --2- 172.21.15.94:0/2480937059 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56261f899330 0x56261f8f3340 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.089+0000 7f70e5ffb640 0 client.17889 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.089+0000 7f71075fb740 1 --2- 172.21.15.94:0/2480937059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56261f889750 0x56261f88f910 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.089+0000 7f71075fb740 1 --2- 172.21.15.94:0/2480937059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56261f880b70 0x56261f88f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.089+0000 7f71075fb740 1 --2- 172.21.15.94:0/2480937059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56261f880250 0x56261f88ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.089+0000 7f71075fb740 1 -- 172.21.15.94:0/2480937059 >> 172.21.15.94:0/2480937059 conn(0x56261f87a980 msgr2=0x56261f886fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f71075fb740 1 -- 172.21.15.94:0/2480937059 shutdown_connections 2024-02-17T12:07:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f71075fb740 1 -- 172.21.15.94:0/2480937059 wait complete. 2024-02-17T12:07:15.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f71075fb740 20 client.17889 trim_cache size 0 max 16384 2024-02-17T12:07:15.099 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:07:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f7104a23640 1 -- 172.21.15.94:0/596467137 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f70ec0a2230 msgr2=0x7f70ec0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f7104a23640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f70ec0a2230 0x7f70ec0a46f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f70f8013780 tx=0x7f70f8004a40 comp rx=0 tx=0).stop 2024-02-17T12:07:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f7104a23640 1 -- 172.21.15.94:0/596467137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71000696f0 msgr2=0x7f710019d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f7104a23640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71000696f0 0x7f710019d580 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f70f0036040 tx=0x7f70f002c640 comp rx=0 tx=0).stop 2024-02-17T12:07:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f7104a23640 1 -- 172.21.15.94:0/596467137 shutdown_connections 2024-02-17T12:07:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f7104a23640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f70ec0a2230 0x7f70ec0a46f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f7104a23640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f71001071d0 0x7f7100197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f7104a23640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f710006a070 0x7f710019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f7104a23640 1 --2- 172.21.15.94:0/596467137 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f71000696f0 0x7f710019d580 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f7104a23640 1 -- 172.21.15.94:0/596467137 >> 172.21.15.94:0/596467137 conn(0x7f7100077060 msgr2=0x7f71000ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f7104a23640 1 -- 172.21.15.94:0/596467137 shutdown_connections 2024-02-17T12:07:15.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.090+0000 7f7104a23640 1 -- 172.21.15.94:0/596467137 wait complete. 2024-02-17T12:07:15.106 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:07:15.106 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:07:15.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 535.0 s, 2534.8 tps, lat 39.774 ms stddev 46.086 2024-02-17T12:07:15.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.253+0000 7f88cd6d4640 1 -- 172.21.15.94:0/3174598397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c8100b00 msgr2=0x7f88c8104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.253+0000 7f88cd6d4640 1 --2- 172.21.15.94:0/3174598397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c8100b00 0x7f88c8104fc0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009930 tx=0x7f88bc02bcd0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.253+0000 7f88cd6d4640 1 -- 172.21.15.94:0/3174598397 shutdown_connections 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.253+0000 7f88cd6d4640 1 --2- 172.21.15.94:0/3174598397 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88c81055e0 0x7f88c8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.253+0000 7f88cd6d4640 1 --2- 172.21.15.94:0/3174598397 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c8100b00 0x7f88c8104fc0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.253+0000 7f88cd6d4640 1 --2- 172.21.15.94:0/3174598397 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88c81001e0 0x7f88c81005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.253+0000 7f88cd6d4640 1 -- 172.21.15.94:0/3174598397 >> 172.21.15.94:0/3174598397 conn(0x7f88c80771e0 msgr2=0x7f88c80775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.253+0000 7f88cd6d4640 1 -- 172.21.15.94:0/3174598397 shutdown_connections 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.253+0000 7f88cd6d4640 1 -- 172.21.15.94:0/3174598397 wait complete. 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.254+0000 7f88cd6d4640 1 Processor -- start 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.254+0000 7f88cd6d4640 1 -- start start 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.254+0000 7f88cd6d4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c81001e0 0x7f88c819de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.254+0000 7f88cd6d4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88c8100b00 0x7f88c819e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.254+0000 7f88cd6d4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88c81055e0 0x7f88c81a2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.254+0000 7f88cd6d4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c81a2d10 con 0x7f88c81001e0 2024-02-17T12:07:15.256 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.254+0000 7f88cd6d4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c81a2e80 con 0x7f88c8100b00 2024-02-17T12:07:15.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.254+0000 7f88cd6d4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c81a3040 con 0x7f88c81055e0 2024-02-17T12:07:15.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.254+0000 7f88c6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c81001e0 0x7f88c819de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.254+0000 7f88c6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c81001e0 0x7f88c819de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42812/0 (socket says 172.21.15.94:42812) 2024-02-17T12:07:15.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.254+0000 7f88c6ffd640 1 -- 172.21.15.94:0/3350457661 learned_addr learned my addr 172.21.15.94:0/3350457661 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.254+0000 7f88c67fc640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88c8100b00 0x7f88c819e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.257 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.255+0000 7f88c77fe640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88c81055e0 0x7f88c81a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.255+0000 7f88c6ffd640 1 -- 172.21.15.94:0/3350457661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88c81055e0 msgr2=0x7f88c81a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.255+0000 7f88c6ffd640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88c81055e0 0x7f88c81a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.255+0000 7f88c6ffd640 1 -- 172.21.15.94:0/3350457661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88c8100b00 msgr2=0x7f88c819e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.255+0000 7f88c6ffd640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88c8100b00 0x7f88c819e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.255+0000 7f88c6ffd640 1 -- 172.21.15.94:0/3350457661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c0009840 con 0x7f88c81001e0 2024-02-17T12:07:15.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.255+0000 7f88c67fc640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88c8100b00 0x7f88c819e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:07:15.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.255+0000 7f88c6ffd640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c81001e0 0x7f88c819de50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f88b800e9f0 tx=0x7f88b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.256+0000 7f88affff640 1 -- 172.21.15.94:0/3350457661 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8009700 con 0x7f88c81001e0 2024-02-17T12:07:15.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.256+0000 7f88affff640 1 -- 172.21.15.94:0/3350457661 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f88b800ccc0 con 0x7f88c81001e0 2024-02-17T12:07:15.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.256+0000 7f88cd6d4640 1 -- 172.21.15.94:0/3350457661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc009560 con 0x7f88c81001e0 2024-02-17T12:07:15.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.256+0000 7f88affff640 1 -- 172.21.15.94:0/3350457661 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8018820 con 0x7f88c81001e0 2024-02-17T12:07:15.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.256+0000 7f88cd6d4640 1 -- 172.21.15.94:0/3350457661 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c81aa9c0 con 0x7f88c81001e0 2024-02-17T12:07:15.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.261+0000 7f88d0380740 1 Processor -- start 2024-02-17T12:07:15.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.261+0000 7f88d0380740 1 -- start start 2024-02-17T12:07:15.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.261+0000 7f88d0380740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955fe250 0x5591955fe630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.261+0000 7f88d0380740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591955feb70 0x559195607210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.261+0000 7f88d0380740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195607750 0x559195609b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.261+0000 7f88d0380740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591955b9960 con 0x5591955feb70 2024-02-17T12:07:15.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.261+0000 7f88d0380740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919556d310 con 0x5591955fe250 2024-02-17T12:07:15.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.262+0000 7f88d0380740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919556a840 con 0x559195607750 2024-02-17T12:07:15.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.262+0000 7f88c77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955fe250 0x5591955fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.262+0000 7f88c67fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195607750 0x559195609b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.262+0000 7f88c77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955fe250 0x5591955fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53644/0 (socket says 172.21.15.94:53644) 2024-02-17T12:07:15.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.262+0000 7f88c77fe640 1 -- 172.21.15.94:0/224402875 learned_addr learned my addr 172.21.15.94:0/224402875 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.262+0000 7f88affff640 1 -- 172.21.15.94:0/3350457661 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f88b8010430 con 0x7f88c81001e0 2024-02-17T12:07:15.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.262+0000 7f88c6ffd640 1 --2- 172.21.15.94:0/224402875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591955feb70 0x559195607210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.263+0000 7f88c77fe640 1 -- 172.21.15.94:0/224402875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195607750 msgr2=0x559195609b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.263+0000 7f88affff640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f88b00a2230 0x7f88b00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.263+0000 7f88c77fe640 1 --2- 172.21.15.94:0/224402875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195607750 0x559195609b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.263+0000 7f88c77fe640 1 -- 172.21.15.94:0/224402875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591955feb70 msgr2=0x559195607210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.263+0000 7f88c77fe640 1 --2- 172.21.15.94:0/224402875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591955feb70 0x559195607210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.263+0000 7f88c77fe640 1 -- 172.21.15.94:0/224402875 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559195528060 con 0x5591955fe250 2024-02-17T12:07:15.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.263+0000 7f88c6ffd640 1 --2- 172.21.15.94:0/224402875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591955feb70 0x559195607210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:07:15.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.263+0000 7f88affff640 1 -- 172.21.15.94:0/3350457661 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f88b8014070 con 0x7f88c81001e0 2024-02-17T12:07:15.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.263+0000 7f88c77fe640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f88b00a2230 0x7f88b00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.263+0000 7f88c77fe640 1 --2- 172.21.15.94:0/224402875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955fe250 0x5591955fe630 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f88c0011570 tx=0x7f88c0011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=747b186a29150c5d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.263+0000 7f88ad7fa640 1 -- 172.21.15.94:0/224402875 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c0020030 con 0x5591955fe250 2024-02-17T12:07:15.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.264+0000 7f88ad7fa640 1 -- 172.21.15.94:0/224402875 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f88c00047b0 con 0x5591955fe250 2024-02-17T12:07:15.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.264+0000 7f88ad7fa640 1 -- 172.21.15.94:0/224402875 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c001b690 con 0x5591955fe250 2024-02-17T12:07:15.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.264+0000 7f88c77fe640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f88b00a2230 0x7f88b00a46f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f88c0020000 tx=0x7f88c001bd00 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.265+0000 7f88d0380740 1 -- 172.21.15.94:0/224402875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955fe250 msgr2=0x5591955fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.266+0000 7f88d0380740 1 --2- 172.21.15.94:0/224402875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955fe250 0x5591955fe630 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f88c0011570 tx=0x7f88c0011a30 comp rx=0 tx=0).stop 2024-02-17T12:07:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.266+0000 7f88d0380740 1 -- 172.21.15.94:0/224402875 shutdown_connections 2024-02-17T12:07:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.266+0000 7f88d0380740 1 --2- 172.21.15.94:0/224402875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559195607750 0x559195609b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.266+0000 7f88d0380740 1 --2- 172.21.15.94:0/224402875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591955feb70 0x559195607210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.266+0000 7f88d0380740 1 --2- 172.21.15.94:0/224402875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955fe250 0x5591955fe630 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.266+0000 7f88d0380740 1 -- 172.21.15.94:0/224402875 >> 172.21.15.94:0/224402875 conn(0x5591955f8980 msgr2=0x5591955f8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.267+0000 7f88d0380740 1 -- 172.21.15.94:0/224402875 shutdown_connections 2024-02-17T12:07:15.268 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.267+0000 7f88d0380740 1 -- 172.21.15.94:0/224402875 wait complete. 2024-02-17T12:07:15.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.267+0000 7f88d0380740 1 -- start start 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.267+0000 7f88d0380740 1 Processor -- start 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.267+0000 7f88d0380740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591955fe250 0x5591956047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.267+0000 7f88d0380740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955feb70 0x559195604d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.267+0000 7f88d0380740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559195607750 0x559195605270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.267+0000 7f88d0380740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559195566020 con 0x5591955fe250 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.267+0000 7f88d0380740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559195239430 con 0x559195607750 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.267+0000 7f88d0380740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591951f5e00 con 0x5591955feb70 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.268+0000 7f88d0380740 10 client.0 _ll_register_callbacks cb 0x55919550dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.268+0000 7f88c6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591955fe250 0x5591956047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.268+0000 7f88c67fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955feb70 0x559195604d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.268+0000 7f88c77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559195607750 0x559195605270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.268+0000 7f88c67fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955feb70 0x559195604d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44840/0 (socket says 172.21.15.94:44840) 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.268+0000 7f88c6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591955fe250 0x5591956047f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42830/0 (socket says 172.21.15.94:42830) 2024-02-17T12:07:15.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.268+0000 7f88c67fc640 1 -- 172.21.15.94:0/2383306449 learned_addr learned my addr 172.21.15.94:0/2383306449 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.268+0000 7f88c6ffd640 1 -- 172.21.15.94:0/2383306449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955feb70 msgr2=0x559195604d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.268+0000 7f88c6ffd640 1 --2- 172.21.15.94:0/2383306449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955feb70 0x559195604d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.268+0000 7f88c6ffd640 1 -- 172.21.15.94:0/2383306449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559195607750 msgr2=0x559195605270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.268+0000 7f88c6ffd640 1 --2- 172.21.15.94:0/2383306449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559195607750 0x559195605270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.268+0000 7f88c6ffd640 1 -- 172.21.15.94:0/2383306449 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c0009d80 con 0x5591955fe250 2024-02-17T12:07:15.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.268+0000 7f88c77fe640 1 --2- 172.21.15.94:0/2383306449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559195607750 0x559195605270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:15.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.269+0000 7f88aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:15.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.269+0000 7f88c6ffd640 1 --2- 172.21.15.94:0/2383306449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591955fe250 0x5591956047f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f88b80cbcf0 tx=0x7f88b8010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96324348a2f7f210 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.269+0000 7f88aaff5640 1 -- 172.21.15.94:0/2383306449 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b809f0e0 con 0x5591955fe250 2024-02-17T12:07:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.269+0000 7f88aaff5640 1 -- 172.21.15.94:0/2383306449 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f88b80a3730 con 0x5591955fe250 2024-02-17T12:07:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.269+0000 7f88d0380740 1 -- 172.21.15.94:0/2383306449 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591952657e0 con 0x5591955fe250 2024-02-17T12:07:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.269+0000 7f88aaff5640 1 -- 172.21.15.94:0/2383306449 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88b809a3e0 con 0x5591955fe250 2024-02-17T12:07:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.269+0000 7f88d0380740 10 client.17898 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.269+0000 7f88d0380740 1 -- 172.21.15.94:0/2383306449 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5591955646a0 con 0x5591955fe250 2024-02-17T12:07:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.270+0000 7f88aaff5640 1 -- 172.21.15.94:0/2383306449 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5591952657e0 con 0x5591955fe250 2024-02-17T12:07:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.270+0000 7f88aaff5640 1 -- 172.21.15.94:0/2383306449 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f88b8096ce0 con 0x5591955fe250 2024-02-17T12:07:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.270+0000 7f88d0380740 10 client.17898 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.270+0000 7f88d0380740 10 client.17898 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.270+0000 7f88d0380740 20 client.17898 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:15.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.270+0000 7f88d0380740 1 --2- 172.21.15.94:0/2383306449 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559195617b30 0x559195619f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.270+0000 7f88d0380740 4 client.17898 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.270+0000 7f88d0380740 1 -- 172.21.15.94:0/2383306449 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591952a2720 con 0x559195617b30 2024-02-17T12:07:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.270+0000 7f88c67fc640 1 --2- 172.21.15.94:0/2383306449 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559195617b30 0x559195619f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88c67fc640 1 --2- 172.21.15.94:0/2383306449 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559195617b30 0x559195619f20 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf8bd0dd0a4edc43 server_cookie=85bdd650cd8d797f in_seq=0 out_seq=0 2024-02-17T12:07:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88aaff5640 10 client.17898 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88aaff5640 1 -- 172.21.15.94:0/2383306449 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5591952a2720 con 0x559195617b30 2024-02-17T12:07:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88aaff5640 10 client.17898 handle_command_reply: tid=0 2024-02-17T12:07:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88d0380740 1 client.17898 shutdown 2024-02-17T12:07:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88d0380740 1 -- 172.21.15.94:0/2383306449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591955fe250 msgr2=0x5591956047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88d0380740 1 --2- 172.21.15.94:0/2383306449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591955fe250 0x5591956047f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f88b80cbcf0 tx=0x7f88b8010a10 comp rx=0 tx=0).stop 2024-02-17T12:07:15.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88d0380740 1 -- 172.21.15.94:0/2383306449 shutdown_connections 2024-02-17T12:07:15.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88d0380740 1 --2- 172.21.15.94:0/2383306449 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559195617b30 0x559195619f20 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88aaff5640 0 client.17898 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:15.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88d0380740 1 --2- 172.21.15.94:0/2383306449 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559195607750 0x559195605270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88d0380740 1 --2- 172.21.15.94:0/2383306449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955feb70 0x559195604d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88d0380740 1 --2- 172.21.15.94:0/2383306449 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5591955fe250 0x5591956047f0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88d0380740 1 -- 172.21.15.94:0/2383306449 >> 172.21.15.94:0/2383306449 conn(0x5591955f8980 msgr2=0x5591955fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.271+0000 7f88d0380740 1 -- 172.21.15.94:0/2383306449 shutdown_connections 2024-02-17T12:07:15.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.272+0000 7f88d0380740 1 -- 172.21.15.94:0/2383306449 wait complete. 2024-02-17T12:07:15.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.272+0000 7f88d0380740 20 client.17898 trim_cache size 0 max 16384 2024-02-17T12:07:15.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88d0380740 1 Processor -- start 2024-02-17T12:07:15.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88d0380740 1 -- start start 2024-02-17T12:07:15.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88d0380740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955fe250 0x559195606f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88d0380740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955feb70 0x559195605090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88d0380740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559195607750 0x559195601a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88d0380740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591954eb130 con 0x559195607750 2024-02-17T12:07:15.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88d0380740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591955f9bb0 con 0x5591955fe250 2024-02-17T12:07:15.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88d0380740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591955fdeb0 con 0x5591955feb70 2024-02-17T12:07:15.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559195607750 0x559195601a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88c67fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955feb70 0x559195605090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88c77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559195607750 0x559195601a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42846/0 (socket says 172.21.15.94:42846) 2024-02-17T12:07:15.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88c77fe640 1 -- 172.21.15.94:0/3715344942 learned_addr learned my addr 172.21.15.94:0/3715344942 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88c6ffd640 1 --2- 172.21.15.94:0/3715344942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955fe250 0x559195606f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88c77fe640 1 -- 172.21.15.94:0/3715344942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955feb70 msgr2=0x559195605090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88c77fe640 1 --2- 172.21.15.94:0/3715344942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955feb70 0x559195605090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88c77fe640 1 -- 172.21.15.94:0/3715344942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955fe250 msgr2=0x559195606f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88c77fe640 1 --2- 172.21.15.94:0/3715344942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955fe250 0x559195606f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.287+0000 7f88c77fe640 1 -- 172.21.15.94:0/3715344942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559195500c20 con 0x559195607750 2024-02-17T12:07:15.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.288+0000 7f88c67fc640 1 --2- 172.21.15.94:0/3715344942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955feb70 0x559195605090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:15.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.288+0000 7f88c77fe640 1 --2- 172.21.15.94:0/3715344942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559195607750 0x559195601a50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f88c0019780 tx=0x7f88c0029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0d101535035ee server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.288+0000 7f88acff9640 1 -- 172.21.15.94:0/3715344942 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c0003970 con 0x559195607750 2024-02-17T12:07:15.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.288+0000 7f88acff9640 1 -- 172.21.15.94:0/3715344942 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f88c000f8f0 con 0x559195607750 2024-02-17T12:07:15.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.288+0000 7f88acff9640 1 -- 172.21.15.94:0/3715344942 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c0019900 con 0x559195607750 2024-02-17T12:07:15.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.289+0000 7f88d0380740 1 -- 172.21.15.94:0/3715344942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559195607750 msgr2=0x559195601a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.289+0000 7f88d0380740 1 --2- 172.21.15.94:0/3715344942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559195607750 0x559195601a50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f88c0019780 tx=0x7f88c0029000 comp rx=0 tx=0).stop 2024-02-17T12:07:15.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.290+0000 7f88d0380740 1 -- 172.21.15.94:0/3715344942 shutdown_connections 2024-02-17T12:07:15.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.290+0000 7f88d0380740 1 --2- 172.21.15.94:0/3715344942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559195607750 0x559195601a50 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.290+0000 7f88d0380740 1 --2- 172.21.15.94:0/3715344942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955feb70 0x559195605090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.290+0000 7f88d0380740 1 --2- 172.21.15.94:0/3715344942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955fe250 0x559195606f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.290+0000 7f88d0380740 1 -- 172.21.15.94:0/3715344942 >> 172.21.15.94:0/3715344942 conn(0x5591955f8980 msgr2=0x5591955ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.290+0000 7f88d0380740 1 -- 172.21.15.94:0/3715344942 shutdown_connections 2024-02-17T12:07:15.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.290+0000 7f88d0380740 1 -- 172.21.15.94:0/3715344942 wait complete. 2024-02-17T12:07:15.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.290+0000 7f88d0380740 1 -- start start 2024-02-17T12:07:15.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.290+0000 7f88d0380740 1 Processor -- start 2024-02-17T12:07:15.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88d0380740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955fe250 0x55919560d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88d0380740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955feb70 0x55919560d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88d0380740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559195607750 0x55919560dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88d0380740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919566e210 con 0x559195607750 2024-02-17T12:07:15.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88d0380740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919566e380 con 0x5591955feb70 2024-02-17T12:07:15.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88d0380740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919566e540 con 0x5591955fe250 2024-02-17T12:07:15.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88d0380740 10 client.0 _ll_register_callbacks cb 0x7f88c0009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:15.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88c67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955feb70 0x55919560d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88c6ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955fe250 0x55919560d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88c67fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955feb70 0x55919560d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53670/0 (socket says 172.21.15.94:53670) 2024-02-17T12:07:15.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88c67fc640 1 -- 172.21.15.94:0/4127842542 learned_addr learned my addr 172.21.15.94:0/4127842542 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88c77fe640 1 --2- 172.21.15.94:0/4127842542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559195607750 0x55919560dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88c67fc640 1 -- 172.21.15.94:0/4127842542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955fe250 msgr2=0x55919560d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88c67fc640 1 --2- 172.21.15.94:0/4127842542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955fe250 0x55919560d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88c67fc640 1 -- 172.21.15.94:0/4127842542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559195607750 msgr2=0x55919560dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88c67fc640 1 --2- 172.21.15.94:0/4127842542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559195607750 0x55919560dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.291+0000 7f88c67fc640 1 -- 172.21.15.94:0/4127842542 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c8114f10 con 0x5591955feb70 2024-02-17T12:07:15.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88c67fc640 1 --2- 172.21.15.94:0/4127842542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955feb70 0x55919560d7d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009930 tx=0x7f88bc02d970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc7369880860ee66 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:07:15.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88aaff5640 1 -- 172.21.15.94:0/4127842542 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc044040 con 0x5591955feb70 2024-02-17T12:07:15.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88d0380740 1 -- 172.21.15.94:0/4127842542 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88c00047b0 con 0x5591955feb70 2024-02-17T12:07:15.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88aaff5640 1 -- 172.21.15.94:0/4127842542 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f88bc02dcf0 con 0x5591955feb70 2024-02-17T12:07:15.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88aaff5640 1 -- 172.21.15.94:0/4127842542 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88bc004240 con 0x5591955feb70 2024-02-17T12:07:15.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88d0380740 10 client.27395 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:15.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88d0380740 1 -- 172.21.15.94:0/4127842542 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559195528060 con 0x5591955feb70 2024-02-17T12:07:15.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88aaff5640 1 -- 172.21.15.94:0/4127842542 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f88c00047b0 con 0x5591955feb70 2024-02-17T12:07:15.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88aaff5640 1 -- 172.21.15.94:0/4127842542 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f88bc054020 con 0x5591955feb70 2024-02-17T12:07:15.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88d0380740 10 client.27395 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:15.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88d0380740 10 client.27395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:15.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88d0380740 20 client.27395 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:15.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88d0380740 1 --2- 172.21.15.94:0/4127842542 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559195617b30 0x559195671d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88d0380740 4 client.27395 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:07:15.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.292+0000 7f88d0380740 1 -- 172.21.15.94:0/4127842542 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5591952f1890 con 0x559195617b30 2024-02-17T12:07:15.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88c6ffd640 1 --2- 172.21.15.94:0/4127842542 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559195617b30 0x559195671d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88aaff5640 10 client.27395 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:15.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88c6ffd640 1 --2- 172.21.15.94:0/4127842542 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559195617b30 0x559195671d90 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8adb0f603c677db server_cookie=850567f1b7c2f44c in_seq=0 out_seq=0 2024-02-17T12:07:15.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88aaff5640 1 -- 172.21.15.94:0/4127842542 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5591952f1890 con 0x559195617b30 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88aaff5640 10 client.27395 handle_command_reply: tid=0 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88d0380740 1 client.27395 shutdown 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88d0380740 1 -- 172.21.15.94:0/4127842542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955feb70 msgr2=0x55919560d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88d0380740 1 --2- 172.21.15.94:0/4127842542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955feb70 0x55919560d7d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f88bc009930 tx=0x7f88bc02d970 comp rx=0 tx=0).stop 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88d0380740 1 -- 172.21.15.94:0/4127842542 shutdown_connections 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88d0380740 1 --2- 172.21.15.94:0/4127842542 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559195617b30 0x559195671d90 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88d0380740 1 --2- 172.21.15.94:0/4127842542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559195607750 0x55919560dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88aaff5640 0 client.27395 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88d0380740 1 --2- 172.21.15.94:0/4127842542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5591955feb70 0x55919560d7d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88d0380740 1 --2- 172.21.15.94:0/4127842542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591955fe250 0x55919560d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88d0380740 1 -- 172.21.15.94:0/4127842542 >> 172.21.15.94:0/4127842542 conn(0x5591955f8980 msgr2=0x5591955f72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88d0380740 1 -- 172.21.15.94:0/4127842542 shutdown_connections 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.293+0000 7f88d0380740 1 -- 172.21.15.94:0/4127842542 wait complete. 2024-02-17T12:07:15.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.294+0000 7f88d0380740 20 client.27395 trim_cache size 0 max 16384 2024-02-17T12:07:15.303 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:07:15.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.294+0000 7f88cd6d4640 1 -- 172.21.15.94:0/3350457661 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f88b00a2230 msgr2=0x7f88b00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.294+0000 7f88cd6d4640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f88b00a2230 0x7f88b00a46f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f88c0020000 tx=0x7f88c001bd00 comp rx=0 tx=0).stop 2024-02-17T12:07:15.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.294+0000 7f88cd6d4640 1 -- 172.21.15.94:0/3350457661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c81001e0 msgr2=0x7f88c819de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.294+0000 7f88cd6d4640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c81001e0 0x7f88c819de50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f88b800e9f0 tx=0x7f88b800edf0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.294+0000 7f88cd6d4640 1 -- 172.21.15.94:0/3350457661 shutdown_connections 2024-02-17T12:07:15.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.294+0000 7f88cd6d4640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f88b00a2230 0x7f88b00a46f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.294+0000 7f88cd6d4640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88c81055e0 0x7f88c81a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.294+0000 7f88cd6d4640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f88c8100b00 0x7f88c819e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.294+0000 7f88cd6d4640 1 --2- 172.21.15.94:0/3350457661 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f88c81001e0 0x7f88c819de50 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.294+0000 7f88cd6d4640 1 -- 172.21.15.94:0/3350457661 >> 172.21.15.94:0/3350457661 conn(0x7f88c80771e0 msgr2=0x7f88c8102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.294+0000 7f88cd6d4640 1 -- 172.21.15.94:0/3350457661 shutdown_connections 2024-02-17T12:07:15.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:15.294+0000 7f88cd6d4640 1 -- 172.21.15.94:0/3350457661 wait complete. 2024-02-17T12:07:15.307 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:14 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2233760756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:07:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.359+0000 7fd6facc0640 1 -- 172.21.15.94:0/1437564302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f411a960 msgr2=0x7fd6f411cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.359+0000 7fd6facc0640 1 --2- 172.21.15.94:0/1437564302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f411a960 0x7fd6f411cd50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009ca0 tx=0x7fd6ec02c140 comp rx=0 tx=0).stop 2024-02-17T12:07:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.362+0000 7fd6facc0640 1 -- 172.21.15.94:0/1437564302 shutdown_connections 2024-02-17T12:07:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.362+0000 7fd6facc0640 1 --2- 172.21.15.94:0/1437564302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f411a960 0x7fd6f411cd50 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.362+0000 7fd6facc0640 1 --2- 172.21.15.94:0/1437564302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6f410fd30 0x7fd6f410da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.362+0000 7fd6facc0640 1 --2- 172.21.15.94:0/1437564302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6f410f8c0 0x7fd6f410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.362+0000 7fd6facc0640 1 -- 172.21.15.94:0/1437564302 >> 172.21.15.94:0/1437564302 conn(0x7fd6f406c220 msgr2=0x7fd6f406c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.371+0000 7fd6facc0640 1 -- 172.21.15.94:0/1437564302 shutdown_connections 2024-02-17T12:07:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.372+0000 7fd6facc0640 1 -- 172.21.15.94:0/1437564302 wait complete. 2024-02-17T12:07:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.373+0000 7fd6facc0640 1 Processor -- start 2024-02-17T12:07:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.373+0000 7fd6facc0640 1 -- start start 2024-02-17T12:07:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.374+0000 7fd6facc0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6f410f8c0 0x7fd6f4117d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.374+0000 7fd6facc0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6f410fd30 0x7fd6f4118270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.374+0000 7fd6facc0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f411a960 0x7fd6f4110df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.374+0000 7fd6facc0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f4111580 con 0x7fd6f411a960 2024-02-17T12:07:15.378 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.374+0000 7fd6facc0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f41116f0 con 0x7fd6f410fd30 2024-02-17T12:07:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.374+0000 7fd6facc0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f41118b0 con 0x7fd6f410f8c0 2024-02-17T12:07:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.374+0000 7fd6f3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6f410fd30 0x7fd6f4118270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.374+0000 7fd6f925d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f411a960 0x7fd6f4110df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.374+0000 7fd6f925d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f411a960 0x7fd6f4110df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42868/0 (socket says 172.21.15.94:42868) 2024-02-17T12:07:15.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.374+0000 7fd6f925d640 1 -- 172.21.15.94:0/2838913432 learned_addr learned my addr 172.21.15.94:0/2838913432 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.374+0000 7fd6f8a5c640 1 --2- 172.21.15.94:0/2838913432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6f410f8c0 0x7fd6f4117d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.375+0000 7fd6f925d640 1 -- 172.21.15.94:0/2838913432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6f410f8c0 msgr2=0x7fd6f4117d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.375+0000 7fd6f925d640 1 --2- 172.21.15.94:0/2838913432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6f410f8c0 0x7fd6f4117d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.375+0000 7fd6f925d640 1 -- 172.21.15.94:0/2838913432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6f410fd30 msgr2=0x7fd6f4118270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.375+0000 7fd6f925d640 1 --2- 172.21.15.94:0/2838913432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6f410fd30 0x7fd6f4118270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.375+0000 7fd6f925d640 1 -- 172.21.15.94:0/2838913432 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec009900 con 0x7fd6f411a960 2024-02-17T12:07:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.375+0000 7fd6f925d640 1 --2- 172.21.15.94:0/2838913432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f411a960 0x7fd6f4110df0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009bf0 tx=0x7fd6ec032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.375+0000 7fd6f1ffb640 1 -- 172.21.15.94:0/2838913432 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec034910 con 0x7fd6f411a960 2024-02-17T12:07:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.375+0000 7fd6f1ffb640 1 -- 172.21.15.94:0/2838913432 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6ec03d8d0 con 0x7fd6f411a960 2024-02-17T12:07:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.375+0000 7fd6f1ffb640 1 -- 172.21.15.94:0/2838913432 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec03c950 con 0x7fd6f411a960 2024-02-17T12:07:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.375+0000 7fd6facc0640 1 -- 172.21.15.94:0/2838913432 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f4111b30 con 0x7fd6f411a960 2024-02-17T12:07:15.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.376+0000 7fd6facc0640 1 -- 172.21.15.94:0/2838913432 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f4115a30 con 0x7fd6f411a960 2024-02-17T12:07:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.379+0000 7fd6fd96e740 1 Processor -- start 2024-02-17T12:07:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.380+0000 7fd6f1ffb640 1 -- 172.21.15.94:0/2838913432 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd6ec03b040 con 0x7fd6f411a960 2024-02-17T12:07:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.380+0000 7fd6fd96e740 1 -- start start 2024-02-17T12:07:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.380+0000 7fd6fd96e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a2ad0 0x561a966a1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.380+0000 7fd6fd96e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a1d60 0x561a966a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.380+0000 7fd6fd96e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966ab2e0 0x561a966ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.392 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.380+0000 7fd6fd96e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a966109f0 con 0x561a966ab2e0 2024-02-17T12:07:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.380+0000 7fd6fd96e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9660df20 con 0x561a966a2ad0 2024-02-17T12:07:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.380+0000 7fd6fd96e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a96609700 con 0x561a966a1d60 2024-02-17T12:07:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.380+0000 7fd6f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966ab2e0 0x561a966ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.380+0000 7fd6f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966ab2e0 0x561a966ad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42880/0 (socket says 172.21.15.94:42880) 2024-02-17T12:07:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.380+0000 7fd6f3fff640 1 -- 172.21.15.94:0/1178378040 learned_addr learned my addr 172.21.15.94:0/1178378040 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.380+0000 7fd6f8a5c640 1 --2- 172.21.15.94:0/1178378040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a1d60 0x561a966a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.380+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/1178378040 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a2ad0 0x561a966a1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.381+0000 7fd6f1ffb640 1 --2- 172.21.15.94:0/2838913432 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd6dc0a2500 0x7fd6dc0a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.393 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.381+0000 7fd6f3fff640 1 -- 172.21.15.94:0/1178378040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a1d60 msgr2=0x561a966a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.381+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/1178378040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a1d60 0x561a966a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.381+0000 7fd6f3fff640 1 -- 172.21.15.94:0/1178378040 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a2ad0 msgr2=0x561a966a1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.381+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/1178378040 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a2ad0 0x561a966a1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.381+0000 7fd6f3fff640 1 -- 172.21.15.94:0/1178378040 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a9659f7b0 con 0x561a966ab2e0 2024-02-17T12:07:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.381+0000 7fd6f925d640 1 --2- 172.21.15.94:0/2838913432 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd6dc0a2500 0x7fd6dc0a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.381+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/1178378040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966ab2e0 0x561a966ad6d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40126c0 tx=0x7fd6e400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe5b1975a49b0d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.381+0000 7fd6f1ffb640 1 -- 172.21.15.94:0/2838913432 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd6ec0f07f0 con 0x7fd6f411a960 2024-02-17T12:07:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.381+0000 7fd6daffd640 1 -- 172.21.15.94:0/1178378040 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4012c40 con 0x561a966ab2e0 2024-02-17T12:07:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.381+0000 7fd6daffd640 1 -- 172.21.15.94:0/1178378040 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6e4012da0 con 0x561a966ab2e0 2024-02-17T12:07:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.381+0000 7fd6daffd640 1 -- 172.21.15.94:0/1178378040 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4012c40 con 0x561a966ab2e0 2024-02-17T12:07:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.382+0000 7fd6fd96e740 1 -- 172.21.15.94:0/1178378040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966ab2e0 msgr2=0x561a966ad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.382+0000 7fd6f925d640 1 --2- 172.21.15.94:0/2838913432 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd6dc0a2500 0x7fd6dc0a49c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0bd000 tx=0x7fd6ec0b96d0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.394 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.382+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/1178378040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966ab2e0 0x561a966ad6d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40126c0 tx=0x7fd6e400b6e0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.382+0000 7fd6fd96e740 1 -- 172.21.15.94:0/1178378040 shutdown_connections 2024-02-17T12:07:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.382+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/1178378040 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966ab2e0 0x561a966ad6d0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.382+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/1178378040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a1d60 0x561a966a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.382+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/1178378040 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a2ad0 0x561a966a1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.382+0000 7fd6fd96e740 1 -- 172.21.15.94:0/1178378040 >> 172.21.15.94:0/1178378040 conn(0x561a96669550 msgr2=0x561a9669ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.382+0000 7fd6fd96e740 1 -- 172.21.15.94:0/1178378040 shutdown_connections 2024-02-17T12:07:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.382+0000 7fd6fd96e740 1 -- 172.21.15.94:0/1178378040 wait complete. 2024-02-17T12:07:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.382+0000 7fd6fd96e740 1 -- start start 2024-02-17T12:07:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6fd96e740 1 Processor -- start 2024-02-17T12:07:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6fd96e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a1d60 0x561a966a83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6fd96e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966a2ad0 0x561a966a88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.395 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6fd96e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966ab2e0 0x561a966a8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6fd96e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a965aebf0 con 0x561a966a2ad0 2024-02-17T12:07:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6fd96e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a962dd250 con 0x561a966a1d60 2024-02-17T12:07:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6fd96e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9658f200 con 0x561a966ab2e0 2024-02-17T12:07:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6f8a5c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966a2ad0 0x561a966a88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6f3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a1d60 0x561a966a83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6f8a5c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966a2ad0 0x561a966a88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42892/0 (socket says 172.21.15.94:42892) 2024-02-17T12:07:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6f8a5c640 1 -- 172.21.15.94:0/676319459 learned_addr learned my addr 172.21.15.94:0/676319459 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6fd96e740 10 client.0 _ll_register_callbacks cb 0x561a965fdd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/676319459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966ab2e0 0x561a966a8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6f8a5c640 1 -- 172.21.15.94:0/676319459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966ab2e0 msgr2=0x561a966a8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6f8a5c640 1 --2- 172.21.15.94:0/676319459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966ab2e0 0x561a966a8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6f8a5c640 1 -- 172.21.15.94:0/676319459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a1d60 msgr2=0x561a966a83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6f8a5c640 1 --2- 172.21.15.94:0/676319459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a1d60 0x561a966a83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.383+0000 7fd6f8a5c640 1 -- 172.21.15.94:0/676319459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e401a040 con 0x561a966a2ad0 2024-02-17T12:07:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.384+0000 7fd6f8a5c640 1 --2- 172.21.15.94:0/676319459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966a2ad0 0x561a966a88f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8010820 tx=0x7fd6e8010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4366e190212b2f5c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.384+0000 7fd6d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.384+0000 7fd6d3fff640 1 -- 172.21.15.94:0/676319459 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8015070 con 0x561a966a2ad0 2024-02-17T12:07:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.384+0000 7fd6d3fff640 1 -- 172.21.15.94:0/676319459 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd6e800fb90 con 0x561a966a2ad0 2024-02-17T12:07:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.384+0000 7fd6fd96e740 1 -- 172.21.15.94:0/676319459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a96309600 con 0x561a966a2ad0 2024-02-17T12:07:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.384+0000 7fd6d3fff640 1 -- 172.21.15.94:0/676319459 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6e800fd50 con 0x561a966a2ad0 2024-02-17T12:07:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.384+0000 7fd6fd96e740 10 client.17916 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.384+0000 7fd6fd96e740 1 -- 172.21.15.94:0/676319459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561a96607d80 con 0x561a966a2ad0 2024-02-17T12:07:15.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.385+0000 7fd6d3fff640 1 -- 172.21.15.94:0/676319459 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6e8025020 con 0x561a966a2ad0 2024-02-17T12:07:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.385+0000 7fd6d3fff640 1 -- 172.21.15.94:0/676319459 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd6e800e4c0 con 0x561a966a2ad0 2024-02-17T12:07:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.385+0000 7fd6fd96e740 10 client.17916 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.385+0000 7fd6fd96e740 10 client.17916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.385+0000 7fd6fd96e740 20 client.17916 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.385+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/676319459 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a966bb7a0 0x561a966bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.385+0000 7fd6fd96e740 4 client.17916 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.385+0000 7fd6fd96e740 1 -- 172.21.15.94:0/676319459 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a9635a1b0 con 0x561a966bb7a0 2024-02-17T12:07:15.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.386+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/676319459 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a966bb7a0 0x561a966bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.386+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/676319459 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a966bb7a0 0x561a966bdbe0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=718830d0825a8a78 server_cookie=6ad2776813f271e1 in_seq=0 out_seq=0 2024-02-17T12:07:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.386+0000 7fd6d3fff640 10 client.17916 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6d3fff640 1 -- 172.21.15.94:0/676319459 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561a9635a1b0 con 0x561a966bb7a0 2024-02-17T12:07:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6d3fff640 10 client.17916 handle_command_reply: tid=0 2024-02-17T12:07:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6fd96e740 1 client.17916 shutdown 2024-02-17T12:07:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6fd96e740 1 -- 172.21.15.94:0/676319459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966a2ad0 msgr2=0x561a966a88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/676319459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966a2ad0 0x561a966a88f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd6e8010820 tx=0x7fd6e8010ce0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6fd96e740 1 -- 172.21.15.94:0/676319459 shutdown_connections 2024-02-17T12:07:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/676319459 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a966bb7a0 0x561a966bdbe0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6d3fff640 0 client.17916 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/676319459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966ab2e0 0x561a966a8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/676319459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966a2ad0 0x561a966a88f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/676319459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a1d60 0x561a966a83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6fd96e740 1 -- 172.21.15.94:0/676319459 >> 172.21.15.94:0/676319459 conn(0x561a96669550 msgr2=0x561a9669e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6fd96e740 1 -- 172.21.15.94:0/676319459 shutdown_connections 2024-02-17T12:07:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6fd96e740 1 -- 172.21.15.94:0/676319459 wait complete. 2024-02-17T12:07:15.400 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.387+0000 7fd6fd96e740 20 client.17916 trim_cache size 0 max 16384 2024-02-17T12:07:15.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:14 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2233760756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:07:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.404+0000 7fd6fd96e740 1 Processor -- start 2024-02-17T12:07:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.404+0000 7fd6fd96e740 1 -- start start 2024-02-17T12:07:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.404+0000 7fd6fd96e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a1d60 0x561a966b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.404+0000 7fd6fd96e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a2ad0 0x561a966b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.404+0000 7fd6fd96e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966ab2e0 0x561a966a8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.404+0000 7fd6fd96e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a966a3280 con 0x561a966ab2e0 2024-02-17T12:07:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.404+0000 7fd6fd96e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a966adef0 con 0x561a966a1d60 2024-02-17T12:07:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.404+0000 7fd6fd96e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a9669d850 con 0x561a966a2ad0 2024-02-17T12:07:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.405+0000 7fd6f3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a1d60 0x561a966b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.405+0000 7fd6f8a5c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a2ad0 0x561a966b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.405+0000 7fd6f3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a1d60 0x561a966b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53708/0 (socket says 172.21.15.94:53708) 2024-02-17T12:07:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.405+0000 7fd6f8a5c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a2ad0 0x561a966b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44910/0 (socket says 172.21.15.94:44910) 2024-02-17T12:07:15.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.405+0000 7fd6f3fff640 1 -- 172.21.15.94:0/2509146402 learned_addr learned my addr 172.21.15.94:0/2509146402 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.405+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/2509146402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966ab2e0 0x561a966a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.405+0000 7fd6f8a5c640 1 -- 172.21.15.94:0/2509146402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a1d60 msgr2=0x561a966b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.405+0000 7fd6f8a5c640 1 --2- 172.21.15.94:0/2509146402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a1d60 0x561a966b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.405+0000 7fd6f8a5c640 1 -- 172.21.15.94:0/2509146402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966ab2e0 msgr2=0x561a966a8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.405+0000 7fd6f8a5c640 1 --2- 172.21.15.94:0/2509146402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966ab2e0 0x561a966a8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.405+0000 7fd6f8a5c640 1 -- 172.21.15.94:0/2509146402 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a965a47b0 con 0x561a966a2ad0 2024-02-17T12:07:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.405+0000 7fd6f8a5c640 1 --2- 172.21.15.94:0/2509146402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a2ad0 0x561a966b2640 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800b8f0 tx=0x7fd6e8005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f5dabbb3ff55aff server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.406+0000 7fd6da7fc640 1 -- 172.21.15.94:0/2509146402 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8015020 con 0x561a966a2ad0 2024-02-17T12:07:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.406+0000 7fd6da7fc640 1 -- 172.21.15.94:0/2509146402 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6e8025070 con 0x561a966a2ad0 2024-02-17T12:07:15.409 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.406+0000 7fd6da7fc640 1 -- 172.21.15.94:0/2509146402 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800be70 con 0x561a966a2ad0 2024-02-17T12:07:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.407+0000 7fd6fd96e740 1 -- 172.21.15.94:0/2509146402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a2ad0 msgr2=0x561a966b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.407+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/2509146402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a2ad0 0x561a966b2640 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd6e800b8f0 tx=0x7fd6e8005fa0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.407+0000 7fd6fd96e740 1 -- 172.21.15.94:0/2509146402 shutdown_connections 2024-02-17T12:07:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.407+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/2509146402 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966ab2e0 0x561a966a8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.407+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/2509146402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a2ad0 0x561a966b2640 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.407+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/2509146402 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966a1d60 0x561a966b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.407+0000 7fd6fd96e740 1 -- 172.21.15.94:0/2509146402 >> 172.21.15.94:0/2509146402 conn(0x561a96669550 msgr2=0x561a966a6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.407+0000 7fd6fd96e740 1 -- 172.21.15.94:0/2509146402 shutdown_connections 2024-02-17T12:07:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.407+0000 7fd6fd96e740 1 -- 172.21.15.94:0/2509146402 wait complete. 2024-02-17T12:07:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6fd96e740 1 -- start start 2024-02-17T12:07:15.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6fd96e740 1 Processor -- start 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6fd96e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a1d60 0x561a966b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6fd96e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966a2ad0 0x561a966a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6fd96e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966ab2e0 0x561a966a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6fd96e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a966a5bc0 con 0x561a966a2ad0 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6fd96e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a966a5d30 con 0x561a966ab2e0 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6fd96e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a96711d50 con 0x561a966a1d60 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6f3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a1d60 0x561a966b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6fd96e740 10 client.0 _ll_register_callbacks cb 0x7fd6e400b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6f3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a1d60 0x561a966b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44914/0 (socket says 172.21.15.94:44914) 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6f3fff640 1 -- 172.21.15.94:0/538808798 learned_addr learned my addr 172.21.15.94:0/538808798 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966ab2e0 0x561a966a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.408+0000 7fd6f8a5c640 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966a2ad0 0x561a966a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.409+0000 7fd6f3fff640 1 -- 172.21.15.94:0/538808798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966ab2e0 msgr2=0x561a966a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.409+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966ab2e0 0x561a966a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.415 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.409+0000 7fd6f3fff640 1 -- 172.21.15.94:0/538808798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966a2ad0 msgr2=0x561a966a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.409+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966a2ad0 0x561a966a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.409+0000 7fd6f3fff640 1 -- 172.21.15.94:0/538808798 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561a96607d80 con 0x561a966a1d60 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.409+0000 7fd6f8a5c640 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966a2ad0 0x561a966a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.409+0000 7fd6d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.410+0000 7fd6f3fff640 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a1d60 0x561a966b25a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40126c0 tx=0x7fd6e400b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3efb1380afb9789 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.410+0000 7fd6d3fff640 1 -- 172.21.15.94:0/538808798 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e402b340 con 0x561a966a1d60 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.410+0000 7fd6fd96e740 1 -- 172.21.15.94:0/538808798 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6e4012da0 con 0x561a966a1d60 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.410+0000 7fd6d3fff640 1 -- 172.21.15.94:0/538808798 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd6e4000e40 con 0x561a966a1d60 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.410+0000 7fd6d3fff640 1 -- 172.21.15.94:0/538808798 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6e402b340 con 0x561a966a1d60 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.410+0000 7fd6fd96e740 10 client.27412 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.410+0000 7fd6fd96e740 1 -- 172.21.15.94:0/538808798 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561a965ca540 con 0x561a966a1d60 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.411+0000 7fd6d3fff640 1 -- 172.21.15.94:0/538808798 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6e4012da0 con 0x561a966a1d60 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.411+0000 7fd6d3fff640 1 -- 172.21.15.94:0/538808798 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd6e4010c00 con 0x561a966a1d60 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.411+0000 7fd6fd96e740 10 client.27412 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:15.416 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.411+0000 7fd6fd96e740 10 client.27412 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.411+0000 7fd6fd96e740 20 client.27412 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.411+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a96713490 0x561a967158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.411+0000 7fd6fd96e740 4 client.27412 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:07:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.411+0000 7fd6fd96e740 1 -- 172.21.15.94:0/538808798 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561a96346550 con 0x561a96713490 2024-02-17T12:07:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.411+0000 7fd6f8a5c640 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a96713490 0x561a967158b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.412+0000 7fd6d3fff640 10 client.27412 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:15.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.412+0000 7fd6f8a5c640 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a96713490 0x561a967158b0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2557e18043cc589a server_cookie=127b69fcfee96ebe in_seq=0 out_seq=0 2024-02-17T12:07:15.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:14 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2233760756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:07:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.440+0000 7fd6d3fff640 1 -- 172.21.15.94:0/538808798 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561a96346550 con 0x561a96713490 2024-02-17T12:07:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.440+0000 7fd6d3fff640 10 client.27412 handle_command_reply: tid=0 2024-02-17T12:07:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.440+0000 7fd6fd96e740 1 client.27412 shutdown 2024-02-17T12:07:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.440+0000 7fd6fd96e740 1 -- 172.21.15.94:0/538808798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a1d60 msgr2=0x561a966b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.440+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a1d60 0x561a966b25a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40126c0 tx=0x7fd6e400b6e0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.440+0000 7fd6fd96e740 1 -- 172.21.15.94:0/538808798 shutdown_connections 2024-02-17T12:07:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.441+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a96713490 0x561a967158b0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.441+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a966ab2e0 0x561a966a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.441+0000 7fd6d3fff640 0 client.27412 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.441+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a966a2ad0 0x561a966a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.441+0000 7fd6fd96e740 1 --2- 172.21.15.94:0/538808798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a966a1d60 0x561a966b25a0 secure :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40126c0 tx=0x7fd6e400b6e0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.441+0000 7fd6fd96e740 1 -- 172.21.15.94:0/538808798 >> 172.21.15.94:0/538808798 conn(0x561a96669550 msgr2=0x561a9669b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.441+0000 7fd6fd96e740 1 -- 172.21.15.94:0/538808798 shutdown_connections 2024-02-17T12:07:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.441+0000 7fd6fd96e740 1 -- 172.21.15.94:0/538808798 wait complete. 2024-02-17T12:07:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.442+0000 7fd6fd96e740 20 client.27412 trim_cache size 0 max 16384 2024-02-17T12:07:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:07:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:07:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:07:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:07:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.442+0000 7fd6facc0640 1 -- 172.21.15.94:0/2838913432 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd6dc0a2500 msgr2=0x7fd6dc0a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.442+0000 7fd6facc0640 1 --2- 172.21.15.94:0/2838913432 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd6dc0a2500 0x7fd6dc0a49c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0bd000 tx=0x7fd6ec0b96d0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.442+0000 7fd6facc0640 1 -- 172.21.15.94:0/2838913432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f411a960 msgr2=0x7fd6f4110df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.442+0000 7fd6facc0640 1 --2- 172.21.15.94:0/2838913432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f411a960 0x7fd6f4110df0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009bf0 tx=0x7fd6ec032f20 comp rx=0 tx=0).stop 2024-02-17T12:07:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.442+0000 7fd6facc0640 1 -- 172.21.15.94:0/2838913432 shutdown_connections 2024-02-17T12:07:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.442+0000 7fd6facc0640 1 --2- 172.21.15.94:0/2838913432 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd6dc0a2500 0x7fd6dc0a49c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.442+0000 7fd6facc0640 1 --2- 172.21.15.94:0/2838913432 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd6f411a960 0x7fd6f4110df0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.442+0000 7fd6facc0640 1 --2- 172.21.15.94:0/2838913432 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd6f410fd30 0x7fd6f4118270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.442+0000 7fd6facc0640 1 --2- 172.21.15.94:0/2838913432 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6f410f8c0 0x7fd6f4117d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.442+0000 7fd6facc0640 1 -- 172.21.15.94:0/2838913432 >> 172.21.15.94:0/2838913432 conn(0x7fd6f406c220 msgr2=0x7fd6f4119390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.442+0000 7fd6facc0640 1 -- 172.21.15.94:0/2838913432 shutdown_connections 2024-02-17T12:07:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:15.442+0000 7fd6facc0640 1 -- 172.21.15.94:0/2838913432 wait complete. 2024-02-17T12:07:16.310 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:07:16.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:07:16.311 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:07:16.339 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:15 smithi094 ceph-mon[32499]: pgmap v283: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-02-17T12:07:16.402 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:15 smithi189 ceph-mon[36588]: pgmap v283: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-02-17T12:07:16.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:15 smithi125 ceph-mon[36819]: pgmap v283: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-02-17T12:07:16.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.463+0000 7f9d2249a640 1 -- 172.21.15.94:0/3983660613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c100260 msgr2=0x7f9d1c100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.463+0000 7f9d2249a640 1 --2- 172.21.15.94:0/3983660613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c100260 0x7f9d1c100640 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009930 tx=0x7f9d0c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:07:16.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.463+0000 7f9d2249a640 1 -- 172.21.15.94:0/3983660613 shutdown_connections 2024-02-17T12:07:16.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.463+0000 7f9d2249a640 1 --2- 172.21.15.94:0/3983660613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d1c105330 0x7f9d1c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.463+0000 7f9d2249a640 1 --2- 172.21.15.94:0/3983660613 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d1c100c10 0x7f9d1c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.463+0000 7f9d2249a640 1 --2- 172.21.15.94:0/3983660613 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c100260 0x7f9d1c100640 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.463+0000 7f9d2249a640 1 -- 172.21.15.94:0/3983660613 >> 172.21.15.94:0/3983660613 conn(0x7f9d1c077180 msgr2=0x7f9d1c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:16.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.463+0000 7f9d2249a640 1 -- 172.21.15.94:0/3983660613 shutdown_connections 2024-02-17T12:07:16.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.463+0000 7f9d2249a640 1 -- 172.21.15.94:0/3983660613 wait complete. 2024-02-17T12:07:16.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.464+0000 7f9d2249a640 1 Processor -- start 2024-02-17T12:07:16.466 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.464+0000 7f9d2249a640 1 -- start start 2024-02-17T12:07:16.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.464+0000 7f9d2249a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d1c100260 0x7f9d1c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.464+0000 7f9d2249a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c105330 0x7f9d1c198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.467 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.464+0000 7f9d2249a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d1c199170 0x7f9d1c1a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.464+0000 7f9d2249a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c1a3dc0 con 0x7f9d1c105330 2024-02-17T12:07:16.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.464+0000 7f9d2249a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c1a3f30 con 0x7f9d1c100260 2024-02-17T12:07:16.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.464+0000 7f9d2249a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c1a40f0 con 0x7f9d1c199170 2024-02-17T12:07:16.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.464+0000 7f9d1bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d1c100260 0x7f9d1c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.464+0000 7f9d1bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d1c100260 0x7f9d1c198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53738/0 (socket says 172.21.15.94:53738) 2024-02-17T12:07:16.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.464+0000 7f9d1bfff640 1 -- 172.21.15.94:0/2820807127 learned_addr learned my addr 172.21.15.94:0/2820807127 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:16.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.464+0000 7f9d20a37640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d1c199170 0x7f9d1c1a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.464+0000 7f9d1b7fe640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c105330 0x7f9d1c198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.465+0000 7f9d1bfff640 1 -- 172.21.15.94:0/2820807127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d1c199170 msgr2=0x7f9d1c1a36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.465+0000 7f9d1bfff640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d1c199170 0x7f9d1c1a36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.465+0000 7f9d1bfff640 1 -- 172.21.15.94:0/2820807127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c105330 msgr2=0x7f9d1c198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.465+0000 7f9d1bfff640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c105330 0x7f9d1c198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.468 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.465+0000 7f9d1bfff640 1 -- 172.21.15.94:0/2820807127 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0c009560 con 0x7f9d1c100260 2024-02-17T12:07:16.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.465+0000 7f9d1bfff640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d1c100260 0x7f9d1c198650 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c036040 tx=0x7f9d0c004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:16.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.465+0000 7f9d197fa640 1 -- 172.21.15.94:0/2820807127 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c004250 con 0x7f9d1c100260 2024-02-17T12:07:16.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.465+0000 7f9d2249a640 1 -- 172.21.15.94:0/2820807127 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c1a4370 con 0x7f9d1c100260 2024-02-17T12:07:16.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.465+0000 7f9d197fa640 1 -- 172.21.15.94:0/2820807127 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d0c0043b0 con 0x7f9d1c100260 2024-02-17T12:07:16.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.465+0000 7f9d197fa640 1 -- 172.21.15.94:0/2820807127 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c003b40 con 0x7f9d1c100260 2024-02-17T12:07:16.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.465+0000 7f9d2249a640 1 -- 172.21.15.94:0/2820807127 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c1a48d0 con 0x7f9d1c100260 2024-02-17T12:07:16.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.468+0000 7f9d197fa640 1 -- 172.21.15.94:0/2820807127 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9d0c003ca0 con 0x7f9d1c100260 2024-02-17T12:07:16.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.468+0000 7f9d2506d740 1 Processor -- start 2024-02-17T12:07:16.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.468+0000 7f9d2506d740 1 -- start start 2024-02-17T12:07:16.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.468+0000 7f9d197fa640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9d040a2230 0x7f9d040a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.468+0000 7f9d2506d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.468+0000 7f9d2506d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83d8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.468+0000 7f9d2506d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83d9400 0x5610d83db7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.468+0000 7f9d2506d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d8335f90 con 0x5610d83c6ff0 2024-02-17T12:07:16.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.468+0000 7f9d2506d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d83334c0 con 0x5610d83d9400 2024-02-17T12:07:16.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.468+0000 7f9d20a37640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9d040a2230 0x7f9d040a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.468+0000 7f9d2506d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d832eca0 con 0x5610d83c7910 2024-02-17T12:07:16.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.468+0000 7f9d197fa640 1 -- 172.21.15.94:0/2820807127 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9d0c0f0640 con 0x7f9d1c100260 2024-02-17T12:07:16.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83c73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42924/0 (socket says 172.21.15.94:42924) 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d1b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83d8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d1bfff640 1 -- 172.21.15.94:0/48878338 learned_addr learned my addr 172.21.15.94:0/48878338 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d20a37640 1 --2- 172.21.15.94:0/48878338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83d9400 0x5610d83db7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d1bfff640 1 -- 172.21.15.94:0/48878338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 msgr2=0x5610d83d8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d1bfff640 1 --2- 172.21.15.94:0/48878338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83d8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d1bfff640 1 -- 172.21.15.94:0/48878338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83d9400 msgr2=0x5610d83db7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d1bfff640 1 --2- 172.21.15.94:0/48878338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83d9400 0x5610d83db7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d1bfff640 1 -- 172.21.15.94:0/48878338 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d82f10b0 con 0x5610d83c6ff0 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d1b7fe640 1 --2- 172.21.15.94:0/48878338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83d8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d20a37640 1 --2- 172.21.15.94:0/48878338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83d9400 0x5610d83db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d20a37640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9d040a2230 0x7f9d040a46f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400a7b0 tx=0x7f9d1400abb0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d1bfff640 1 --2- 172.21.15.94:0/48878338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83c73d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0bce00 tx=0x7f9d0c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c049acd5c8c08bf1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:16.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d027fc640 1 -- 172.21.15.94:0/48878338 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0bca60 con 0x5610d83c6ff0 2024-02-17T12:07:16.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d027fc640 1 -- 172.21.15.94:0/48878338 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d0c0bd800 con 0x5610d83c6ff0 2024-02-17T12:07:16.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.469+0000 7f9d027fc640 1 -- 172.21.15.94:0/48878338 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0b9770 con 0x5610d83c6ff0 2024-02-17T12:07:16.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.470+0000 7f9d2506d740 1 -- 172.21.15.94:0/48878338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 msgr2=0x5610d83c73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.470+0000 7f9d2506d740 1 --2- 172.21.15.94:0/48878338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83c73d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0bce00 tx=0x7f9d0c0035f0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.470+0000 7f9d2506d740 1 -- 172.21.15.94:0/48878338 shutdown_connections 2024-02-17T12:07:16.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.470+0000 7f9d2506d740 1 --2- 172.21.15.94:0/48878338 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83d9400 0x5610d83db7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.470+0000 7f9d2506d740 1 --2- 172.21.15.94:0/48878338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83d8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.470+0000 7f9d2506d740 1 --2- 172.21.15.94:0/48878338 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83c73d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.470+0000 7f9d2506d740 1 -- 172.21.15.94:0/48878338 >> 172.21.15.94:0/48878338 conn(0x5610d83c1840 msgr2=0x5610d83c1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:16.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.470+0000 7f9d2506d740 1 -- 172.21.15.94:0/48878338 shutdown_connections 2024-02-17T12:07:16.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.470+0000 7f9d2506d740 1 -- 172.21.15.94:0/48878338 wait complete. 2024-02-17T12:07:16.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.470+0000 7f9d2506d740 1 -- start start 2024-02-17T12:07:16.476 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d2506d740 1 Processor -- start 2024-02-17T12:07:16.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d2506d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83d6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d2506d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83d6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d2506d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83d9400 0x5610d83d6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d2506d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d82d3f40 con 0x5610d83c6ff0 2024-02-17T12:07:16.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d2506d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d80024b0 con 0x5610d83d9400 2024-02-17T12:07:16.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d2506d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d82b4670 con 0x5610d83c7910 2024-02-17T12:07:16.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d2506d740 10 client.0 _ll_register_callbacks cb 0x5610d8321f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:16.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83d6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83d6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42930/0 (socket says 172.21.15.94:42930) 2024-02-17T12:07:16.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d1bfff640 1 -- 172.21.15.94:0/4099449292 learned_addr learned my addr 172.21.15.94:0/4099449292 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:16.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d20a37640 1 --2- 172.21.15.94:0/4099449292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83d9400 0x5610d83d6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d1b7fe640 1 --2- 172.21.15.94:0/4099449292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83d6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.477 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d1bfff640 1 -- 172.21.15.94:0/4099449292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 msgr2=0x5610d83d6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d1bfff640 1 --2- 172.21.15.94:0/4099449292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83d6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d1bfff640 1 -- 172.21.15.94:0/4099449292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83d9400 msgr2=0x5610d83d6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d1bfff640 1 --2- 172.21.15.94:0/4099449292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83d9400 0x5610d83d6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d1bfff640 1 -- 172.21.15.94:0/4099449292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c1a48d0 con 0x5610d83c6ff0 2024-02-17T12:07:16.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9d1bfff640 1 --2- 172.21.15.94:0/4099449292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83d6430 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0c5a60 tx=0x7f9d0c005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92b423b8968526a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:16.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.471+0000 7f9cfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:16.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.472+0000 7f9cfb7fe640 1 -- 172.21.15.94:0/4099449292 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0bca60 con 0x5610d83c6ff0 2024-02-17T12:07:16.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.472+0000 7f9cfb7fe640 1 -- 172.21.15.94:0/4099449292 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9d0c0bd100 con 0x5610d83c6ff0 2024-02-17T12:07:16.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.472+0000 7f9d2506d740 1 -- 172.21.15.94:0/4099449292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610d8308f40 con 0x5610d83c6ff0 2024-02-17T12:07:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.472+0000 7f9cfb7fe640 1 -- 172.21.15.94:0/4099449292 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d0c080020 con 0x5610d83c6ff0 2024-02-17T12:07:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.472+0000 7f9d2506d740 10 client.17934 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.472+0000 7f9d2506d740 1 -- 172.21.15.94:0/4099449292 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5610d832d320 con 0x5610d83c6ff0 2024-02-17T12:07:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.473+0000 7f9cfb7fe640 1 -- 172.21.15.94:0/4099449292 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5610d8308f40 con 0x5610d83c6ff0 2024-02-17T12:07:16.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.473+0000 7f9cfb7fe640 1 -- 172.21.15.94:0/4099449292 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9d0c07dd20 con 0x5610d83c6ff0 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.473+0000 7f9d2506d740 10 client.17934 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.473+0000 7f9d2506d740 10 client.17934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.473+0000 7f9d2506d740 20 client.17934 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.473+0000 7f9d2506d740 1 --2- 172.21.15.94:0/4099449292 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5610d83e9010 0x5610d83eb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.473+0000 7f9d2506d740 4 client.17934 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.473+0000 7f9d2506d740 1 -- 172.21.15.94:0/4099449292 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610d8325310 con 0x5610d83e9010 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.474+0000 7f9d1b7fe640 1 --2- 172.21.15.94:0/4099449292 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5610d83e9010 0x5610d83eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.474+0000 7f9cfb7fe640 10 client.17934 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.474+0000 7f9d1b7fe640 1 --2- 172.21.15.94:0/4099449292 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5610d83e9010 0x5610d83eb430 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2c18f92986f3294 server_cookie=7800bec311d6b0b4 in_seq=0 out_seq=0 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9cfb7fe640 1 -- 172.21.15.94:0/4099449292 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5610d8325310 con 0x5610d83e9010 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9cfb7fe640 10 client.17934 handle_command_reply: tid=0 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9d2506d740 1 client.17934 shutdown 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9d2506d740 1 -- 172.21.15.94:0/4099449292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 msgr2=0x5610d83d6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9d2506d740 1 --2- 172.21.15.94:0/4099449292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83d6430 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0c5a60 tx=0x7f9d0c005be0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9d2506d740 1 -- 172.21.15.94:0/4099449292 shutdown_connections 2024-02-17T12:07:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9d2506d740 1 --2- 172.21.15.94:0/4099449292 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5610d83e9010 0x5610d83eb430 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9d2506d740 1 --2- 172.21.15.94:0/4099449292 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83d9400 0x5610d83d6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9d2506d740 1 --2- 172.21.15.94:0/4099449292 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83d6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9cfb7fe640 0 client.17934 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9d2506d740 1 --2- 172.21.15.94:0/4099449292 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83d6430 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9d2506d740 1 -- 172.21.15.94:0/4099449292 >> 172.21.15.94:0/4099449292 conn(0x5610d83c1840 msgr2=0x5610d83c37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9d2506d740 1 -- 172.21.15.94:0/4099449292 shutdown_connections 2024-02-17T12:07:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9d2506d740 1 -- 172.21.15.94:0/4099449292 wait complete. 2024-02-17T12:07:16.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.475+0000 7f9d2506d740 20 client.17934 trim_cache size 0 max 16384 2024-02-17T12:07:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.494+0000 7f9d2506d740 1 Processor -- start 2024-02-17T12:07:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.494+0000 7f9d2506d740 1 -- start start 2024-02-17T12:07:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d2506d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83c1840 0x5610d83d5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d2506d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83d6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d2506d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d2506d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d83e5f80 con 0x5610d83c6ff0 2024-02-17T12:07:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d2506d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d83c66a0 con 0x5610d83c1840 2024-02-17T12:07:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d2506d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d83c6d80 con 0x5610d83c7910 2024-02-17T12:07:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d1b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83d6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d20a37640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d1bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83c1840 0x5610d83d5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d20a37640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83c35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44950/0 (socket says 172.21.15.94:44950) 2024-02-17T12:07:16.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d1bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83c1840 0x5610d83d5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53746/0 (socket says 172.21.15.94:53746) 2024-02-17T12:07:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d20a37640 1 -- 172.21.15.94:0/2177925733 learned_addr learned my addr 172.21.15.94:0/2177925733 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d1b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83d6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42934/0 (socket says 172.21.15.94:42934) 2024-02-17T12:07:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d20a37640 1 -- 172.21.15.94:0/2177925733 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83c1840 msgr2=0x5610d83d5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d20a37640 1 --2- 172.21.15.94:0/2177925733 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83c1840 0x5610d83d5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d20a37640 1 -- 172.21.15.94:0/2177925733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 msgr2=0x5610d83d6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d20a37640 1 --2- 172.21.15.94:0/2177925733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83d6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d20a37640 1 -- 172.21.15.94:0/2177925733 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d82c9c00 con 0x5610d83c7910 2024-02-17T12:07:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d1b7fe640 1 --2- 172.21.15.94:0/2177925733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83d6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d20a37640 1 --2- 172.21.15.94:0/2177925733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83c35b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400a500 tx=0x7f9d1401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bd1807dbf510334 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:16.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.495+0000 7f9d01ffb640 1 -- 172.21.15.94:0/2177925733 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14015e80 con 0x5610d83c7910 2024-02-17T12:07:16.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.496+0000 7f9d01ffb640 1 -- 172.21.15.94:0/2177925733 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9d1401abe0 con 0x5610d83c7910 2024-02-17T12:07:16.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.496+0000 7f9d01ffb640 1 -- 172.21.15.94:0/2177925733 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1401b400 con 0x5610d83c7910 2024-02-17T12:07:16.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.497+0000 7f9d2506d740 1 -- 172.21.15.94:0/2177925733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 msgr2=0x5610d83c35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.497+0000 7f9d2506d740 1 --2- 172.21.15.94:0/2177925733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83c35b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400a500 tx=0x7f9d1401c000 comp rx=0 tx=0).stop 2024-02-17T12:07:16.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.497+0000 7f9d2506d740 1 -- 172.21.15.94:0/2177925733 shutdown_connections 2024-02-17T12:07:16.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.497+0000 7f9d2506d740 1 --2- 172.21.15.94:0/2177925733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c7910 0x5610d83c35b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.497+0000 7f9d2506d740 1 --2- 172.21.15.94:0/2177925733 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c6ff0 0x5610d83d6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.497+0000 7f9d2506d740 1 --2- 172.21.15.94:0/2177925733 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83c1840 0x5610d83d5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.497+0000 7f9d2506d740 1 -- 172.21.15.94:0/2177925733 >> 172.21.15.94:0/2177925733 conn(0x5610d83c8520 msgr2=0x5610d83c2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:16.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.497+0000 7f9d2506d740 1 -- 172.21.15.94:0/2177925733 shutdown_connections 2024-02-17T12:07:16.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.497+0000 7f9d2506d740 1 -- 172.21.15.94:0/2177925733 wait complete. 2024-02-17T12:07:16.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.497+0000 7f9d2506d740 1 -- start start 2024-02-17T12:07:16.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d2506d740 1 Processor -- start 2024-02-17T12:07:16.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d2506d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c1840 0x5610d83df1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d2506d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c6ff0 0x5610d83df700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d2506d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83c7910 0x5610d8439520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d2506d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d8439a60 con 0x5610d83c1840 2024-02-17T12:07:16.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d2506d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d8439ba0 con 0x5610d83c7910 2024-02-17T12:07:16.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d2506d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d8439d60 con 0x5610d83c6ff0 2024-02-17T12:07:16.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d2506d740 10 client.0 _ll_register_callbacks cb 0x7f9d0c003f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:16.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d1bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c1840 0x5610d83df1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d20a37640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83c7910 0x5610d8439520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d20a37640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83c7910 0x5610d8439520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53762/0 (socket says 172.21.15.94:53762) 2024-02-17T12:07:16.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d20a37640 1 -- 172.21.15.94:0/145545954 learned_addr learned my addr 172.21.15.94:0/145545954 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:16.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d1b7fe640 1 --2- 172.21.15.94:0/145545954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c6ff0 0x5610d83df700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d1bfff640 1 -- 172.21.15.94:0/145545954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c6ff0 msgr2=0x5610d83df700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d1bfff640 1 --2- 172.21.15.94:0/145545954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c6ff0 0x5610d83df700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d1bfff640 1 -- 172.21.15.94:0/145545954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83c7910 msgr2=0x5610d8439520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d1bfff640 1 --2- 172.21.15.94:0/145545954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83c7910 0x5610d8439520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d1bfff640 1 -- 172.21.15.94:0/145545954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d10009ad0 con 0x5610d83c1840 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d1b7fe640 1 --2- 172.21.15.94:0/145545954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c6ff0 0x5610d83df700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9d1bfff640 1 --2- 172.21.15.94:0/145545954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c1840 0x5610d83df1c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0ee350 tx=0x7f9d0c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bada7626cd9ee921 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.498+0000 7f9cfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9cfb7fe640 1 -- 172.21.15.94:0/145545954 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0bcc40 con 0x5610d83c1840 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9cfb7fe640 1 -- 172.21.15.94:0/145545954 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9d0c0c1250 con 0x5610d83c1840 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9d2506d740 1 -- 172.21.15.94:0/145545954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610d8354900 con 0x5610d83c1840 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9cfb7fe640 1 -- 172.21.15.94:0/145545954 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d0c0bca60 con 0x5610d83c1840 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9d2506d740 10 client.17940 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9d2506d740 1 -- 172.21.15.94:0/145545954 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f9d14013040 con 0x5610d83c1840 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9cfb7fe640 1 -- 172.21.15.94:0/145545954 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5610d8354900 con 0x5610d83c1840 2024-02-17T12:07:16.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9cfb7fe640 1 -- 172.21.15.94:0/145545954 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9d0c03c070 con 0x5610d83c1840 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9d2506d740 10 client.17940 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9d2506d740 10 client.17940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9d2506d740 20 client.17940 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9d2506d740 1 --2- 172.21.15.94:0/145545954 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5610d83d9400 0x5610d84431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9d2506d740 4 client.17940 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9d2506d740 1 -- 172.21.15.94:0/145545954 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5610d802e860 con 0x5610d83d9400 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.499+0000 7f9d1b7fe640 1 --2- 172.21.15.94:0/145545954 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5610d83d9400 0x5610d84431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.500+0000 7f9cfb7fe640 10 client.17940 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.500+0000 7f9d1b7fe640 1 --2- 172.21.15.94:0/145545954 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5610d83d9400 0x5610d84431d0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b49863832efbd4f3 server_cookie=2a9a99b16d59c30 in_seq=0 out_seq=0 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9cfb7fe640 1 -- 172.21.15.94:0/145545954 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5610d802e860 con 0x5610d83d9400 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9cfb7fe640 10 client.17940 handle_command_reply: tid=0 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9d2506d740 1 client.17940 shutdown 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9d2506d740 1 -- 172.21.15.94:0/145545954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c1840 msgr2=0x5610d83df1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9d2506d740 1 --2- 172.21.15.94:0/145545954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c1840 0x5610d83df1c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0ee350 tx=0x7f9d0c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:07:16.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9d2506d740 1 -- 172.21.15.94:0/145545954 shutdown_connections 2024-02-17T12:07:16.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9d2506d740 1 --2- 172.21.15.94:0/145545954 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5610d83d9400 0x5610d84431d0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9d2506d740 1 --2- 172.21.15.94:0/145545954 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610d83c7910 0x5610d8439520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9d2506d740 1 --2- 172.21.15.94:0/145545954 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610d83c6ff0 0x5610d83df700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9d2506d740 1 --2- 172.21.15.94:0/145545954 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610d83c1840 0x5610d83df1c0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9d2506d740 1 -- 172.21.15.94:0/145545954 >> 172.21.15.94:0/145545954 conn(0x5610d83c8520 msgr2=0x5610d83c3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:16.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9cfb7fe640 0 client.17940 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:16.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9d2506d740 1 -- 172.21.15.94:0/145545954 shutdown_connections 2024-02-17T12:07:16.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9d2506d740 1 -- 172.21.15.94:0/145545954 wait complete. 2024-02-17T12:07:16.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.501+0000 7f9d2506d740 20 client.17940 trim_cache size 0 max 16384 2024-02-17T12:07:16.507 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:07:16.507 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:07:16.508 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "1b2ce7a8-aed2-4984-babb-763ea2d1a8d7", 2024-02-17T12:07:16.508 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:07:16.508 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:07:16.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.502+0000 7f9d2249a640 1 -- 172.21.15.94:0/2820807127 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9d040a2230 msgr2=0x7f9d040a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.502+0000 7f9d2249a640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9d040a2230 0x7f9d040a46f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400a7b0 tx=0x7f9d1400abb0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.502+0000 7f9d2249a640 1 -- 172.21.15.94:0/2820807127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d1c100260 msgr2=0x7f9d1c198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.502+0000 7f9d2249a640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d1c100260 0x7f9d1c198650 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c036040 tx=0x7f9d0c004970 comp rx=0 tx=0).stop 2024-02-17T12:07:16.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.502+0000 7f9d2249a640 1 -- 172.21.15.94:0/2820807127 shutdown_connections 2024-02-17T12:07:16.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.502+0000 7f9d2249a640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9d040a2230 0x7f9d040a46f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.502+0000 7f9d2249a640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9d1c199170 0x7f9d1c1a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.502+0000 7f9d2249a640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9d1c105330 0x7f9d1c198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.502+0000 7f9d2249a640 1 --2- 172.21.15.94:0/2820807127 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9d1c100260 0x7f9d1c198650 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.502+0000 7f9d2249a640 1 -- 172.21.15.94:0/2820807127 >> 172.21.15.94:0/2820807127 conn(0x7f9d1c077180 msgr2=0x7f9d1c0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:16.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.503+0000 7f9d2249a640 1 -- 172.21.15.94:0/2820807127 shutdown_connections 2024-02-17T12:07:16.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.503+0000 7f9d2249a640 1 -- 172.21.15.94:0/2820807127 wait complete. 2024-02-17T12:07:16.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:07:16.516 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:07:16.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.704+0000 7f23ec57a640 1 -- 172.21.15.94:0/3259273592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23e4101d10 msgr2=0x7f23e4105d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.704+0000 7f23ec57a640 1 --2- 172.21.15.94:0/3259273592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23e4101d10 0x7f23e4105d50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009930 tx=0x7f23d402bc30 comp rx=0 tx=0).stop 2024-02-17T12:07:16.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 -- 172.21.15.94:0/3259273592 shutdown_connections 2024-02-17T12:07:16.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 --2- 172.21.15.94:0/3259273592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23e4106430 0x7f23e410e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 --2- 172.21.15.94:0/3259273592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23e4101d10 0x7f23e4105d50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 --2- 172.21.15.94:0/3259273592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e4101360 0x7f23e4101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 -- 172.21.15.94:0/3259273592 >> 172.21.15.94:0/3259273592 conn(0x7f23e40fcc20 msgr2=0x7f23e40ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:16.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 -- 172.21.15.94:0/3259273592 shutdown_connections 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 -- 172.21.15.94:0/3259273592 wait complete. 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 Processor -- start 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 -- start start 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e4101360 0x7f23e419d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23e4106430 0x7f23e419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23e4197650 0x7f23e4197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e4198120 con 0x7f23e4101360 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e4198290 con 0x7f23e4197650 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ec57a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e4198450 con 0x7f23e4106430 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ea316640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e4101360 0x7f23e419d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.705+0000 7f23ea316640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e4101360 0x7f23e419d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42966/0 (socket says 172.21.15.94:42966) 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23ea316640 1 -- 172.21.15.94:0/683199729 learned_addr learned my addr 172.21.15.94:0/683199729 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23eab17640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23e4197650 0x7f23e4197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23e9b15640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23e4106430 0x7f23e419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23ea316640 1 -- 172.21.15.94:0/683199729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23e4106430 msgr2=0x7f23e419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23ea316640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23e4106430 0x7f23e419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23ea316640 1 -- 172.21.15.94:0/683199729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23e4197650 msgr2=0x7f23e4197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23ea316640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23e4197650 0x7f23e4197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23ea316640 1 -- 172.21.15.94:0/683199729 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23dc009840 con 0x7f23e4101360 2024-02-17T12:07:16.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23eab17640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23e4197650 0x7f23e4197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:16.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23e9b15640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23e4106430 0x7f23e419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:16.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23ea316640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e4101360 0x7f23e419d590 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f23e000d950 tx=0x7f23e000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:16.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23db7fe640 1 -- 172.21.15.94:0/683199729 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e0009c30 con 0x7f23e4101360 2024-02-17T12:07:16.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23ec57a640 1 -- 172.21.15.94:0/683199729 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d4009560 con 0x7f23e4101360 2024-02-17T12:07:16.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23db7fe640 1 -- 172.21.15.94:0/683199729 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23e000de80 con 0x7f23e4101360 2024-02-17T12:07:16.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23db7fe640 1 -- 172.21.15.94:0/683199729 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e000f700 con 0x7f23e4101360 2024-02-17T12:07:16.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.706+0000 7f23ec57a640 1 -- 172.21.15.94:0/683199729 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e41a4790 con 0x7f23e4101360 2024-02-17T12:07:16.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23ef22f740 1 Processor -- start 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23ef22f740 1 -- start start 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23ef22f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c22b0 0x55cc853c2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23ef22f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853c2bd0 0x55cc853cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23ef22f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853cb7b0 0x55cc853cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23ef22f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc85331280 con 0x55cc853cb7b0 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23ef22f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc8532e7b0 con 0x55cc853c22b0 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23ef22f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc85329f90 con 0x55cc853c2bd0 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23db7fe640 1 -- 172.21.15.94:0/683199729 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f23e0010460 con 0x7f23e4101360 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23eab17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c22b0 0x55cc853c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23e9b15640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853cb7b0 0x55cc853cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23eab17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c22b0 0x55cc853c2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53796/0 (socket says 172.21.15.94:53796) 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23e9b15640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853cb7b0 0x55cc853cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42972/0 (socket says 172.21.15.94:42972) 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.710+0000 7f23eab17640 1 -- 172.21.15.94:0/1424413278 learned_addr learned my addr 172.21.15.94:0/1424413278 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:16.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23ea316640 1 --2- 172.21.15.94:0/1424413278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853c2bd0 0x55cc853cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23eab17640 1 -- 172.21.15.94:0/1424413278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853c2bd0 msgr2=0x55cc853cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23eab17640 1 --2- 172.21.15.94:0/1424413278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853c2bd0 0x55cc853cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23eab17640 1 -- 172.21.15.94:0/1424413278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853cb7b0 msgr2=0x55cc853cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23eab17640 1 --2- 172.21.15.94:0/1424413278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853cb7b0 0x55cc853cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23eab17640 1 -- 172.21.15.94:0/1424413278 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc852e8570 con 0x55cc853c22b0 2024-02-17T12:07:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23e9b15640 1 --2- 172.21.15.94:0/1424413278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853cb7b0 0x55cc853cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23db7fe640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f23d00a22b0 0x7f23d00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23eab17640 1 --2- 172.21.15.94:0/1424413278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c22b0 0x55cc853c2690 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00e890 tx=0x7f23dc00ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6dfffa89c967cb1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23ea316640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f23d00a22b0 0x7f23d00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23db7fe640 1 -- 172.21.15.94:0/683199729 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f23e00cdcb0 con 0x7f23e4101360 2024-02-17T12:07:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23d8bfe640 1 -- 172.21.15.94:0/1424413278 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc011ce0 con 0x55cc853c22b0 2024-02-17T12:07:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23d8bfe640 1 -- 172.21.15.94:0/1424413278 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23dc003e50 con 0x55cc853c22b0 2024-02-17T12:07:16.714 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.711+0000 7f23d8bfe640 1 -- 172.21.15.94:0/1424413278 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc019430 con 0x55cc853c22b0 2024-02-17T12:07:16.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.712+0000 7f23ea316640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f23d00a22b0 0x7f23d00a4770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55cc853c1d80 tx=0x7f23e0010e30 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:16.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.712+0000 7f23ef22f740 1 -- 172.21.15.94:0/1424413278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c22b0 msgr2=0x55cc853c2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.712+0000 7f23ef22f740 1 --2- 172.21.15.94:0/1424413278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c22b0 0x55cc853c2690 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00e890 tx=0x7f23dc00ed50 comp rx=0 tx=0).stop 2024-02-17T12:07:16.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.712+0000 7f23ef22f740 1 -- 172.21.15.94:0/1424413278 shutdown_connections 2024-02-17T12:07:16.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.712+0000 7f23ef22f740 1 --2- 172.21.15.94:0/1424413278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853cb7b0 0x55cc853cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.712+0000 7f23ef22f740 1 --2- 172.21.15.94:0/1424413278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853c2bd0 0x55cc853cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.712+0000 7f23ef22f740 1 --2- 172.21.15.94:0/1424413278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c22b0 0x55cc853c2690 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.712+0000 7f23ef22f740 1 -- 172.21.15.94:0/1424413278 >> 172.21.15.94:0/1424413278 conn(0x55cc853bc990 msgr2=0x55cc853bcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:16.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.712+0000 7f23ef22f740 1 -- 172.21.15.94:0/1424413278 shutdown_connections 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.713+0000 7f23ef22f740 1 -- 172.21.15.94:0/1424413278 wait complete. 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.713+0000 7f23ef22f740 1 -- start start 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.713+0000 7f23ef22f740 1 Processor -- start 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.713+0000 7f23ef22f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 0x55cc853cb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.713+0000 7f23ef22f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c2bd0 0x55cc853c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.713+0000 7f23ef22f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853cb7b0 0x55cc853c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.713+0000 7f23ef22f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc84ffd430 con 0x55cc853c22b0 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.713+0000 7f23ef22f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc84fb9e00 con 0x55cc853c2bd0 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.713+0000 7f23ef22f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc852aefd0 con 0x55cc853cb7b0 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.713+0000 7f23ef22f740 10 client.0 _ll_register_callbacks cb 0x55cc85304fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.713+0000 7f23eab17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 0x55cc853cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.713+0000 7f23e9b15640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c2bd0 0x55cc853c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.713+0000 7f23eab17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 0x55cc853cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42988/0 (socket says 172.21.15.94:42988) 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23eab17640 1 -- 172.21.15.94:0/84190491 learned_addr learned my addr 172.21.15.94:0/84190491 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:16.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23eab17640 1 --2- 172.21.15.94:0/84190491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853cb7b0 0x55cc853c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23eab17640 1 -- 172.21.15.94:0/84190491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853cb7b0 msgr2=0x55cc853c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23eab17640 1 --2- 172.21.15.94:0/84190491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853cb7b0 0x55cc853c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23eab17640 1 -- 172.21.15.94:0/84190491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c2bd0 msgr2=0x55cc853c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23eab17640 1 --2- 172.21.15.94:0/84190491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c2bd0 0x55cc853c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23eab17640 1 -- 172.21.15.94:0/84190491 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23dc00e540 con 0x55cc853c22b0 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23eab17640 1 --2- 172.21.15.94:0/84190491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 0x55cc853cb160 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f23dc004660 tx=0x7f23dc004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425624e441c8a08c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23cdffb640 1 -- 172.21.15.94:0/84190491 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc0047a0 con 0x55cc853c22b0 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23cdffb640 1 -- 172.21.15.94:0/84190491 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f23dc0041d0 con 0x55cc853c22b0 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23ef22f740 1 -- 172.21.15.94:0/84190491 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc850297e0 con 0x55cc853c22b0 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23cdffb640 1 -- 172.21.15.94:0/84190491 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23dc004390 con 0x55cc853c22b0 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23ef22f740 10 client.17952 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.714+0000 7f23ef22f740 1 -- 172.21.15.94:0/84190491 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cc85328610 con 0x55cc853c22b0 2024-02-17T12:07:16.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.715+0000 7f23cdffb640 1 -- 172.21.15.94:0/84190491 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cc850297e0 con 0x55cc853c22b0 2024-02-17T12:07:16.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.716+0000 7f23cdffb640 1 -- 172.21.15.94:0/84190491 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f23dc0057f0 con 0x55cc853c22b0 2024-02-17T12:07:16.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.716+0000 7f23ef22f740 10 client.17952 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:16.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.716+0000 7f23ef22f740 10 client.17952 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:16.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.716+0000 7f23ef22f740 20 client.17952 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:16.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.716+0000 7f23ef22f740 1 --2- 172.21.15.94:0/84190491 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cc853e4070 0x55cc853e6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.716+0000 7f23ef22f740 4 client.17952 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:16.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.716+0000 7f23ef22f740 1 -- 172.21.15.94:0/84190491 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc85066720 con 0x55cc853e4070 2024-02-17T12:07:16.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.716+0000 7f23e9b15640 1 --2- 172.21.15.94:0/84190491 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cc853e4070 0x55cc853e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.717+0000 7f23cdffb640 10 client.17952 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:16.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.717+0000 7f23e9b15640 1 --2- 172.21.15.94:0/84190491 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cc853e4070 0x55cc853e6460 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25b4e5a4f7ce7aac server_cookie=cff0d0265623446a in_seq=0 out_seq=0 2024-02-17T12:07:16.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.717+0000 7f23cdffb640 1 -- 172.21.15.94:0/84190491 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc85066720 con 0x55cc853e4070 2024-02-17T12:07:16.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.717+0000 7f23cdffb640 10 client.17952 handle_command_reply: tid=0 2024-02-17T12:07:16.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.718+0000 7f23ef22f740 1 client.17952 shutdown 2024-02-17T12:07:16.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.718+0000 7f23ef22f740 1 -- 172.21.15.94:0/84190491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 msgr2=0x55cc853cb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.718+0000 7f23ef22f740 1 --2- 172.21.15.94:0/84190491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 0x55cc853cb160 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f23dc004660 tx=0x7f23dc004560 comp rx=0 tx=0).stop 2024-02-17T12:07:16.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.718+0000 7f23ef22f740 1 -- 172.21.15.94:0/84190491 shutdown_connections 2024-02-17T12:07:16.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.718+0000 7f23ef22f740 1 --2- 172.21.15.94:0/84190491 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cc853e4070 0x55cc853e6460 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.718+0000 7f23ef22f740 1 --2- 172.21.15.94:0/84190491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853cb7b0 0x55cc853c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.718+0000 7f23ef22f740 1 --2- 172.21.15.94:0/84190491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c2bd0 0x55cc853c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.718+0000 7f23cdffb640 0 client.17952 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:16.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.718+0000 7f23ef22f740 1 --2- 172.21.15.94:0/84190491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 0x55cc853cb160 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.718+0000 7f23ef22f740 1 -- 172.21.15.94:0/84190491 >> 172.21.15.94:0/84190491 conn(0x55cc853bc990 msgr2=0x55cc853be730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:16.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.718+0000 7f23ef22f740 1 -- 172.21.15.94:0/84190491 shutdown_connections 2024-02-17T12:07:16.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.718+0000 7f23ef22f740 1 -- 172.21.15.94:0/84190491 wait complete. 2024-02-17T12:07:16.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.718+0000 7f23ef22f740 20 client.17952 trim_cache size 0 max 16384 2024-02-17T12:07:16.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.737+0000 7f23ef22f740 1 Processor -- start 2024-02-17T12:07:16.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.737+0000 7f23ef22f740 1 -- start start 2024-02-17T12:07:16.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.737+0000 7f23ef22f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853c22b0 0x55cc853c3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.737+0000 7f23ef22f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c2bd0 0x55cc853c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.737+0000 7f23ef22f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853cb7b0 0x55cc853cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.737+0000 7f23ef22f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc853bd9e0 con 0x55cc853c2bd0 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.737+0000 7f23ef22f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc853c82c0 con 0x55cc853cb7b0 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.737+0000 7f23ef22f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc853c6a40 con 0x55cc853c22b0 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.737+0000 7f23e9b15640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c2bd0 0x55cc853c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.737+0000 7f23e9b15640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c2bd0 0x55cc853c3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43000/0 (socket says 172.21.15.94:43000) 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.737+0000 7f23e9b15640 1 -- 172.21.15.94:0/1072321346 learned_addr learned my addr 172.21.15.94:0/1072321346 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.738+0000 7f23eab17640 1 --2- 172.21.15.94:0/1072321346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853c22b0 0x55cc853c3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.738+0000 7f23e9b15640 1 -- 172.21.15.94:0/1072321346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853c22b0 msgr2=0x55cc853c3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.738+0000 7f23e9b15640 1 --2- 172.21.15.94:0/1072321346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853c22b0 0x55cc853c3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.738+0000 7f23e9b15640 1 -- 172.21.15.94:0/1072321346 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853cb7b0 msgr2=0x55cc853cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.738+0000 7f23e9b15640 1 --2- 172.21.15.94:0/1072321346 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853cb7b0 0x55cc853cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.738+0000 7f23e9b15640 1 -- 172.21.15.94:0/1072321346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc852c4ac0 con 0x55cc853c2bd0 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.738+0000 7f23eab17640 1 --2- 172.21.15.94:0/1072321346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853c22b0 0x55cc853c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:07:16.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.738+0000 7f23e9b15640 1 --2- 172.21.15.94:0/1072321346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c2bd0 0x55cc853c3d50 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f23d4005ab0 tx=0x7f23d4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53085cd747ee7890 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.738+0000 7f23cffff640 1 -- 172.21.15.94:0/1072321346 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d4049be0 con 0x55cc853c2bd0 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.738+0000 7f23cffff640 1 -- 172.21.15.94:0/1072321346 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23d4049d40 con 0x55cc853c2bd0 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.738+0000 7f23cffff640 1 -- 172.21.15.94:0/1072321346 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d40496d0 con 0x55cc853c2bd0 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 -- 172.21.15.94:0/1072321346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c2bd0 msgr2=0x55cc853c3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 --2- 172.21.15.94:0/1072321346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c2bd0 0x55cc853c3d50 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f23d4005ab0 tx=0x7f23d4009260 comp rx=0 tx=0).stop 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 -- 172.21.15.94:0/1072321346 shutdown_connections 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 --2- 172.21.15.94:0/1072321346 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853cb7b0 0x55cc853cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 --2- 172.21.15.94:0/1072321346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c2bd0 0x55cc853c3d50 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 --2- 172.21.15.94:0/1072321346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853c22b0 0x55cc853c3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 -- 172.21.15.94:0/1072321346 >> 172.21.15.94:0/1072321346 conn(0x55cc853bc990 msgr2=0x55cc853ead90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 -- 172.21.15.94:0/1072321346 shutdown_connections 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 -- 172.21.15.94:0/1072321346 wait complete. 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 -- start start 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 Processor -- start 2024-02-17T12:07:16.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 0x55cc853d7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c2bd0 0x55cc853d8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853cb7b0 0x55cc8543a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc853d8cc0 con 0x55cc853c22b0 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc8543aa50 con 0x55cc853c2bd0 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc8543ac10 con 0x55cc853cb7b0 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23ef22f740 10 client.0 _ll_register_callbacks cb 0x7f23dc00e490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23eab17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 0x55cc853d7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23eab17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 0x55cc853d7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:43006/0 (socket says 172.21.15.94:43006) 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23eab17640 1 -- 172.21.15.94:0/2878688531 learned_addr learned my addr 172.21.15.94:0/2878688531 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.739+0000 7f23e9b15640 1 --2- 172.21.15.94:0/2878688531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c2bd0 0x55cc853d8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23eab17640 1 --2- 172.21.15.94:0/2878688531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853cb7b0 0x55cc8543a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23eab17640 1 -- 172.21.15.94:0/2878688531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853cb7b0 msgr2=0x55cc8543a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23eab17640 1 --2- 172.21.15.94:0/2878688531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853cb7b0 0x55cc8543a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23eab17640 1 -- 172.21.15.94:0/2878688531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c2bd0 msgr2=0x55cc853d8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23eab17640 1 --2- 172.21.15.94:0/2878688531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c2bd0 0x55cc853d8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23eab17640 1 -- 172.21.15.94:0/2878688531 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d4030070 con 0x55cc853c22b0 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23eab17640 1 --2- 172.21.15.94:0/2878688531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 0x55cc853d7fe0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0045f0 tx=0x7f23dc00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d5c811492f30d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23cdffb640 1 -- 172.21.15.94:0/2878688531 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23dc011810 con 0x55cc853c22b0 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23cdffb640 1 -- 172.21.15.94:0/2878688531 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f23dc003c80 con 0x55cc853c22b0 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23ef22f740 1 -- 172.21.15.94:0/2878688531 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23dc003e50 con 0x55cc853c22b0 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23cdffb640 1 -- 172.21.15.94:0/2878688531 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23dc027870 con 0x55cc853c22b0 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23ef22f740 10 client.17958 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.740+0000 7f23ef22f740 1 -- 172.21.15.94:0/2878688531 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cc852ea6a0 con 0x55cc853c22b0 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.741+0000 7f23cdffb640 1 -- 172.21.15.94:0/2878688531 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f23dc003e50 con 0x55cc853c22b0 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.741+0000 7f23cdffb640 1 -- 172.21.15.94:0/2878688531 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f23dc005040 con 0x55cc853c22b0 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.741+0000 7f23ef22f740 10 client.17958 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.741+0000 7f23ef22f740 10 client.17958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:16.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.742+0000 7f23ef22f740 20 client.17958 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:16.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.742+0000 7f23ef22f740 1 --2- 172.21.15.94:0/2878688531 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cc8543c0e0 0x55cc8543e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:16.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.742+0000 7f23ef22f740 4 client.17958 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:07:16.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.742+0000 7f23ef22f740 1 -- 172.21.15.94:0/2878688531 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cc850b5890 con 0x55cc8543c0e0 2024-02-17T12:07:16.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.742+0000 7f23e9b15640 1 --2- 172.21.15.94:0/2878688531 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cc8543c0e0 0x55cc8543e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:16.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.742+0000 7f23cdffb640 10 client.17958 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:16.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.742+0000 7f23e9b15640 1 --2- 172.21.15.94:0/2878688531 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cc8543c0e0 0x55cc8543e500 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86e10787a3e8d990 server_cookie=cb352e1073f52333 in_seq=0 out_seq=0 2024-02-17T12:07:16.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.742+0000 7f23cdffb640 1 -- 172.21.15.94:0/2878688531 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cc850b5890 con 0x55cc8543c0e0 2024-02-17T12:07:16.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.742+0000 7f23cdffb640 10 client.17958 handle_command_reply: tid=0 2024-02-17T12:07:16.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.742+0000 7f23ef22f740 1 client.17958 shutdown 2024-02-17T12:07:16.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ef22f740 1 -- 172.21.15.94:0/2878688531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 msgr2=0x55cc853d7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ef22f740 1 --2- 172.21.15.94:0/2878688531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 0x55cc853d7fe0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0045f0 tx=0x7f23dc00ed50 comp rx=0 tx=0).stop 2024-02-17T12:07:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ef22f740 1 -- 172.21.15.94:0/2878688531 shutdown_connections 2024-02-17T12:07:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ef22f740 1 --2- 172.21.15.94:0/2878688531 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cc8543c0e0 0x55cc8543e500 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ef22f740 1 --2- 172.21.15.94:0/2878688531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cc853cb7b0 0x55cc8543a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ef22f740 1 --2- 172.21.15.94:0/2878688531 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cc853c2bd0 0x55cc853d8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ef22f740 1 --2- 172.21.15.94:0/2878688531 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cc853c22b0 0x55cc853d7fe0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ef22f740 1 -- 172.21.15.94:0/2878688531 >> 172.21.15.94:0/2878688531 conn(0x55cc853bc990 msgr2=0x55cc853c3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23cdffb640 0 client.17958 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ef22f740 1 -- 172.21.15.94:0/2878688531 shutdown_connections 2024-02-17T12:07:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ef22f740 1 -- 172.21.15.94:0/2878688531 wait complete. 2024-02-17T12:07:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ef22f740 20 client.17958 trim_cache size 0 max 16384 2024-02-17T12:07:16.752 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:07:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:07:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:07:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "1b2ce7a8-aed2-4984-babb-763ea2d1a8d7": { 2024-02-17T12:07:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:07:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "1b2ce7a8-aed2-4984-babb-763ea2d1a8d7", 2024-02-17T12:07:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:07:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:07:16.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:07:16.753 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:07:16.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ec57a640 1 -- 172.21.15.94:0/683199729 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f23d00a22b0 msgr2=0x7f23d00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ec57a640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f23d00a22b0 0x7f23d00a4770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55cc853c1d80 tx=0x7f23e0010e30 comp rx=0 tx=0).stop 2024-02-17T12:07:16.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ec57a640 1 -- 172.21.15.94:0/683199729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e4101360 msgr2=0x7f23e419d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:16.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ec57a640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e4101360 0x7f23e419d590 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f23e000d950 tx=0x7f23e000dd50 comp rx=0 tx=0).stop 2024-02-17T12:07:16.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ec57a640 1 -- 172.21.15.94:0/683199729 shutdown_connections 2024-02-17T12:07:16.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ec57a640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f23d00a22b0 0x7f23d00a4770 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ec57a640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f23e4197650 0x7f23e4197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ec57a640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23e4106430 0x7f23e419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ec57a640 1 --2- 172.21.15.94:0/683199729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f23e4101360 0x7f23e419d590 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ec57a640 1 -- 172.21.15.94:0/683199729 >> 172.21.15.94:0/683199729 conn(0x7f23e40fcc20 msgr2=0x7f23e40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ec57a640 1 -- 172.21.15.94:0/683199729 shutdown_connections 2024-02-17T12:07:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:16.743+0000 7f23ec57a640 1 -- 172.21.15.94:0/683199729 wait complete. 2024-02-17T12:07:16.756 INFO:tasks.cephfs.filesystem:scrub status for tag:1b2ce7a8-aed2-4984-babb-763ea2d1a8d7 - {'path': '/', 'tag': '1b2ce7a8-aed2-4984-babb-763ea2d1a8d7', 'options': 'recursive,force'} 2024-02-17T12:07:17.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:17 smithi189 ceph-mon[36588]: pgmap v284: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-02-17T12:07:17.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:17 smithi125 ceph-mon[36819]: pgmap v284: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-02-17T12:07:17.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:17 smithi094 ceph-mon[32499]: pgmap v284: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 6.0 MiB/s wr, 638 op/s 2024-02-17T12:07:18.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:18 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:07:18.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:18 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:07:18.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:18 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:07:18.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:18 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:07:18.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:18 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:07:18.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:18 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:07:18.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:18 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:07:18.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:18 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:07:18.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:18 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:07:18.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:18 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:07:18.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:18 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:07:18.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:18 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:07:18.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:18 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:07:18.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:18 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:07:18.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:18 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:07:19.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:19 smithi189 ceph-mon[36588]: pgmap v285: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.7 MiB/s wr, 766 op/s 2024-02-17T12:07:19.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:19 smithi125 ceph-mon[36819]: pgmap v285: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.7 MiB/s wr, 766 op/s 2024-02-17T12:07:19.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:19 smithi094 ceph-mon[32499]: pgmap v285: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.7 MiB/s wr, 766 op/s 2024-02-17T12:07:20.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 540.0 s, 1875.6 tps, lat 52.411 ms stddev 67.550 2024-02-17T12:07:22.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:21 smithi189 ceph-mon[36588]: pgmap v286: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 6.3 MiB/s wr, 719 op/s 2024-02-17T12:07:22.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:21 smithi125 ceph-mon[36819]: pgmap v286: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 6.3 MiB/s wr, 719 op/s 2024-02-17T12:07:22.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:21 smithi094 ceph-mon[32499]: pgmap v286: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 6.3 MiB/s wr, 719 op/s 2024-02-17T12:07:23.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:22 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:07:23.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:22 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:07:23.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:22 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:07:23.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:22 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:07:23.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:22 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:07:23.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:22 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:07:23.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:22 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:07:23.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:22 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:07:23.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:22 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:07:23.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:22 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:07:23.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:22 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:07:23.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:22 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:07:24.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:23 smithi189 ceph-mon[36588]: pgmap v287: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.7 MiB/s wr, 658 op/s 2024-02-17T12:07:24.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:23 smithi125 ceph-mon[36819]: pgmap v287: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.7 MiB/s wr, 658 op/s 2024-02-17T12:07:24.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:23 smithi094 ceph-mon[32499]: pgmap v287: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 5.7 MiB/s wr, 658 op/s 2024-02-17T12:07:25.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 545.0 s, 1508.0 tps, lat 66.999 ms stddev 75.994 2024-02-17T12:07:25.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:24 smithi189 ceph-mon[36588]: pgmap v288: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 7.7 MiB/s wr, 875 op/s 2024-02-17T12:07:25.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:25 smithi125 ceph-mon[36819]: pgmap v288: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 7.7 MiB/s wr, 875 op/s 2024-02-17T12:07:25.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:24 smithi094 ceph-mon[32499]: pgmap v288: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 7.7 MiB/s wr, 875 op/s 2024-02-17T12:07:28.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:27 smithi189 ceph-mon[36588]: pgmap v289: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 5.3 MiB/s wr, 620 op/s 2024-02-17T12:07:28.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:27 smithi125 ceph-mon[36819]: pgmap v289: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 5.3 MiB/s wr, 620 op/s 2024-02-17T12:07:28.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:27 smithi094 ceph-mon[32499]: pgmap v289: 97 pgs: 97 active+clean; 5.1 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 5.3 MiB/s wr, 620 op/s 2024-02-17T12:07:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.556+0000 7f02967bc640 1 -- 172.21.15.94:0/775590171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0290107270 msgr2=0x7f0290107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.556+0000 7f02967bc640 1 --2- 172.21.15.94:0/775590171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0290107270 0x7f0290107650 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0280009900 tx=0x7f028002bcd0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.558+0000 7f02967bc640 1 -- 172.21.15.94:0/775590171 shutdown_connections 2024-02-17T12:07:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.558+0000 7f02967bc640 1 --2- 172.21.15.94:0/775590171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02900691a0 0x7f029010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.558+0000 7f02967bc640 1 --2- 172.21.15.94:0/775590171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0290068800 0x7f0290068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.558+0000 7f02967bc640 1 --2- 172.21.15.94:0/775590171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0290107270 0x7f0290107650 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.558+0000 7f02967bc640 1 -- 172.21.15.94:0/775590171 >> 172.21.15.94:0/775590171 conn(0x7f0290076ed0 msgr2=0x7f02900772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.558+0000 7f02967bc640 1 -- 172.21.15.94:0/775590171 shutdown_connections 2024-02-17T12:07:28.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.558+0000 7f02967bc640 1 -- 172.21.15.94:0/775590171 wait complete. 2024-02-17T12:07:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.559+0000 7f02967bc640 1 Processor -- start 2024-02-17T12:07:28.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.559+0000 7f02967bc640 1 -- start start 2024-02-17T12:07:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.559+0000 7f02967bc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0290068800 0x7f0290197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.559+0000 7f02967bc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02900691a0 0x7f0290197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.559+0000 7f02967bc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0290107270 0x7f029019bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.559+0000 7f02967bc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029019c550 con 0x7f0290068800 2024-02-17T12:07:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.559+0000 7f02967bc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029019c6c0 con 0x7f02900691a0 2024-02-17T12:07:28.561 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.559+0000 7f02967bc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029019c880 con 0x7f0290107270 2024-02-17T12:07:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.559+0000 7f028ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0290068800 0x7f0290197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.559+0000 7f028ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0290068800 0x7f0290197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56896/0 (socket says 172.21.15.94:56896) 2024-02-17T12:07:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.559+0000 7f028ffff640 1 -- 172.21.15.94:0/631683600 learned_addr learned my addr 172.21.15.94:0/631683600 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.559+0000 7f0294d59640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0290107270 0x7f029019bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.559+0000 7f028f7fe640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02900691a0 0x7f0290197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.560+0000 7f028ffff640 1 -- 172.21.15.94:0/631683600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0290107270 msgr2=0x7f029019bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.560+0000 7f028ffff640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0290107270 0x7f029019bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.560+0000 7f028ffff640 1 -- 172.21.15.94:0/631683600 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02900691a0 msgr2=0x7f0290197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.560+0000 7f028ffff640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02900691a0 0x7f0290197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.560+0000 7f028ffff640 1 -- 172.21.15.94:0/631683600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0280009560 con 0x7f0290068800 2024-02-17T12:07:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.560+0000 7f028f7fe640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02900691a0 0x7f0290197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.560+0000 7f028ffff640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0290068800 0x7f0290197690 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0280005940 tx=0x7f028002c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.561+0000 7f028d7fa640 1 -- 172.21.15.94:0/631683600 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800043d0 con 0x7f0290068800 2024-02-17T12:07:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.561+0000 7f028d7fa640 1 -- 172.21.15.94:0/631683600 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02800052d0 con 0x7f0290068800 2024-02-17T12:07:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.561+0000 7f028d7fa640 1 -- 172.21.15.94:0/631683600 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280003d30 con 0x7f0290068800 2024-02-17T12:07:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.561+0000 7f02967bc640 1 -- 172.21.15.94:0/631683600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029019cb00 con 0x7f0290068800 2024-02-17T12:07:28.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.561+0000 7f02967bc640 1 -- 172.21.15.94:0/631683600 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02901a43e0 con 0x7f0290068800 2024-02-17T12:07:28.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.563+0000 7f028d7fa640 1 -- 172.21.15.94:0/631683600 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0280005440 con 0x7f0290068800 2024-02-17T12:07:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.563+0000 7f028d7fa640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02780a2350 0x7f02780a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.564+0000 7f028f7fe640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02780a2350 0x7f02780a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.564+0000 7f028d7fa640 1 -- 172.21.15.94:0/631683600 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f02800f0580 con 0x7f0290068800 2024-02-17T12:07:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.564+0000 7f0299394740 1 Processor -- start 2024-02-17T12:07:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.564+0000 7f0299394740 1 -- start start 2024-02-17T12:07:28.565 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.564+0000 7f028f7fe640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02780a2350 0x7f02780a4810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f02900774d0 tx=0x7f028400b390 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.566+0000 7f0299394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a140ad0 0x55645a13f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.566+0000 7f0299394740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a13fd60 0x55645a1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.566+0000 7f0299394740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a14d6f0 0x55645a14fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.566+0000 7f0299394740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645a0ae9f0 con 0x55645a14d6f0 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.566+0000 7f0299394740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645a0abf20 con 0x55645a13fd60 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.566+0000 7f0299394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645a0a7700 con 0x55645a140ad0 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.566+0000 7f0294d59640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a14d6f0 0x55645a14fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.566+0000 7f0294d59640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a14d6f0 0x55645a14fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56908/0 (socket says 172.21.15.94:56908) 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.566+0000 7f028ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a140ad0 0x55645a13f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.566+0000 7f0294d59640 1 -- 172.21.15.94:0/1247542258 learned_addr learned my addr 172.21.15.94:0/1247542258 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.566+0000 7f028f7fe640 1 --2- 172.21.15.94:0/1247542258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a13fd60 0x55645a1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.567+0000 7f028f7fe640 1 -- 172.21.15.94:0/1247542258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a140ad0 msgr2=0x55645a13f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.567+0000 7f028f7fe640 1 --2- 172.21.15.94:0/1247542258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a140ad0 0x55645a13f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.567+0000 7f028f7fe640 1 -- 172.21.15.94:0/1247542258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a14d6f0 msgr2=0x55645a14fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.567+0000 7f028f7fe640 1 --2- 172.21.15.94:0/1247542258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a14d6f0 0x55645a14fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.567+0000 7f028f7fe640 1 -- 172.21.15.94:0/1247542258 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55645a03d7b0 con 0x55645a13fd60 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.567+0000 7f0294d59640 1 --2- 172.21.15.94:0/1247542258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a14d6f0 0x55645a14fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:28.569 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.567+0000 7f028f7fe640 1 --2- 172.21.15.94:0/1247542258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a13fd60 0x55645a1401e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f028400bfd0 tx=0x7f0284013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fd09658cf5059d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.567+0000 7f02767fc640 1 -- 172.21.15.94:0/1247542258 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028401c780 con 0x55645a13fd60 2024-02-17T12:07:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.567+0000 7f02767fc640 1 -- 172.21.15.94:0/1247542258 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0284015070 con 0x55645a13fd60 2024-02-17T12:07:28.570 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.567+0000 7f02767fc640 1 -- 172.21.15.94:0/1247542258 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0284027e00 con 0x55645a13fd60 2024-02-17T12:07:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.568+0000 7f0299394740 1 -- 172.21.15.94:0/1247542258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a13fd60 msgr2=0x55645a1401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.568+0000 7f0299394740 1 --2- 172.21.15.94:0/1247542258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a13fd60 0x55645a1401e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f028400bfd0 tx=0x7f0284013000 comp rx=0 tx=0).stop 2024-02-17T12:07:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.568+0000 7f0299394740 1 -- 172.21.15.94:0/1247542258 shutdown_connections 2024-02-17T12:07:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.568+0000 7f0299394740 1 --2- 172.21.15.94:0/1247542258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a14d6f0 0x55645a14fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.568+0000 7f0299394740 1 --2- 172.21.15.94:0/1247542258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a13fd60 0x55645a1401e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.568+0000 7f0299394740 1 --2- 172.21.15.94:0/1247542258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a140ad0 0x55645a13f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.568+0000 7f0299394740 1 -- 172.21.15.94:0/1247542258 >> 172.21.15.94:0/1247542258 conn(0x55645a107550 msgr2=0x55645a13aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.568+0000 7f0299394740 1 -- 172.21.15.94:0/1247542258 shutdown_connections 2024-02-17T12:07:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.568+0000 7f0299394740 1 -- 172.21.15.94:0/1247542258 wait complete. 2024-02-17T12:07:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.569+0000 7f0299394740 1 -- start start 2024-02-17T12:07:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.569+0000 7f0299394740 1 Processor -- start 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.569+0000 7f0299394740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a13fd60 0x55645a14a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.569+0000 7f0299394740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a140ad0 0x55645a14acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.569+0000 7f0299394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a14d6f0 0x55645a14b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.569+0000 7f0299394740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645a04cbf0 con 0x55645a13fd60 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.569+0000 7f0299394740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556459d7b250 con 0x55645a140ad0 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.569+0000 7f0299394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645a02d200 con 0x55645a14d6f0 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.569+0000 7f0299394740 10 client.0 _ll_register_callbacks cb 0x55645a09bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f028ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a13fd60 0x55645a14a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f028f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a140ad0 0x55645a14acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f028f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a140ad0 0x55645a14acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52764/0 (socket says 172.21.15.94:52764) 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f028f7fe640 1 -- 172.21.15.94:0/2784853846 learned_addr learned my addr 172.21.15.94:0/2784853846 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f028ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a13fd60 0x55645a14a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56916/0 (socket says 172.21.15.94:56916) 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f0294d59640 1 --2- 172.21.15.94:0/2784853846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a14d6f0 0x55645a14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f028f7fe640 1 -- 172.21.15.94:0/2784853846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a14d6f0 msgr2=0x55645a14b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f028f7fe640 1 --2- 172.21.15.94:0/2784853846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a14d6f0 0x55645a14b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f028f7fe640 1 -- 172.21.15.94:0/2784853846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a13fd60 msgr2=0x55645a14a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f028f7fe640 1 --2- 172.21.15.94:0/2784853846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a13fd60 0x55645a14a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f028f7fe640 1 -- 172.21.15.94:0/2784853846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0284011440 con 0x55645a140ad0 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f028ffff640 1 --2- 172.21.15.94:0/2784853846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a13fd60 0x55645a14a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f028f7fe640 1 --2- 172.21.15.94:0/2784853846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a140ad0 0x55645a14acc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0284005940 tx=0x7f0284028d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ab8dd13b2caa01 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f026f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f026f7fe640 1 -- 172.21.15.94:0/2784853846 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028402d070 con 0x55645a140ad0 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f026f7fe640 1 -- 172.21.15.94:0/2784853846 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0284022070 con 0x55645a140ad0 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.570+0000 7f0299394740 1 -- 172.21.15.94:0/2784853846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556459da7600 con 0x55645a140ad0 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.571+0000 7f026f7fe640 1 -- 172.21.15.94:0/2784853846 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0284027480 con 0x55645a140ad0 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.571+0000 7f0299394740 10 client.27458 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.571+0000 7f0299394740 1 -- 172.21.15.94:0/2784853846 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55645a0a5d80 con 0x55645a140ad0 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.571+0000 7f026f7fe640 1 -- 172.21.15.94:0/2784853846 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556459da7600 con 0x55645a140ad0 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.571+0000 7f026f7fe640 1 -- 172.21.15.94:0/2784853846 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f028401c580 con 0x55645a140ad0 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.571+0000 7f0299394740 10 client.27458 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:28.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.571+0000 7f0299394740 10 client.27458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.571+0000 7f0299394740 20 client.27458 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.571+0000 7f0299394740 1 --2- 172.21.15.94:0/2784853846 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55645a15da70 0x55645a15fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.571+0000 7f0299394740 4 client.27458 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.571+0000 7f0299394740 1 -- 172.21.15.94:0/2784853846 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556459df81b0 con 0x55645a15da70 2024-02-17T12:07:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.571+0000 7f028ffff640 1 --2- 172.21.15.94:0/2784853846 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55645a15da70 0x55645a15fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.572+0000 7f026f7fe640 10 client.27458 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:28.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.572+0000 7f028ffff640 1 --2- 172.21.15.94:0/2784853846 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55645a15da70 0x55645a15fe90 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35c877512101aae1 server_cookie=55f47d37f4c138f4 in_seq=0 out_seq=0 2024-02-17T12:07:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.572+0000 7f026f7fe640 1 -- 172.21.15.94:0/2784853846 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556459df81b0 con 0x55645a15da70 2024-02-17T12:07:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.572+0000 7f026f7fe640 10 client.27458 handle_command_reply: tid=0 2024-02-17T12:07:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.572+0000 7f0299394740 1 client.27458 shutdown 2024-02-17T12:07:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.572+0000 7f0299394740 1 -- 172.21.15.94:0/2784853846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a140ad0 msgr2=0x55645a14acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.572+0000 7f0299394740 1 --2- 172.21.15.94:0/2784853846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a140ad0 0x55645a14acc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0284005940 tx=0x7f0284028d80 comp rx=0 tx=0).stop 2024-02-17T12:07:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.573+0000 7f0299394740 1 -- 172.21.15.94:0/2784853846 shutdown_connections 2024-02-17T12:07:28.577 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.573+0000 7f0299394740 1 --2- 172.21.15.94:0/2784853846 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55645a15da70 0x55645a15fe90 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.573+0000 7f0299394740 1 --2- 172.21.15.94:0/2784853846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a14d6f0 0x55645a14b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.573+0000 7f0299394740 1 --2- 172.21.15.94:0/2784853846 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a140ad0 0x55645a14acc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.573+0000 7f026f7fe640 0 client.27458 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.573+0000 7f0299394740 1 --2- 172.21.15.94:0/2784853846 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a13fd60 0x55645a14a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.573+0000 7f0299394740 1 -- 172.21.15.94:0/2784853846 >> 172.21.15.94:0/2784853846 conn(0x55645a107550 msgr2=0x55645a13c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.573+0000 7f0299394740 1 -- 172.21.15.94:0/2784853846 shutdown_connections 2024-02-17T12:07:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.574+0000 7f0299394740 1 -- 172.21.15.94:0/2784853846 wait complete. 2024-02-17T12:07:28.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.574+0000 7f0299394740 20 client.27458 trim_cache size 0 max 16384 2024-02-17T12:07:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.592+0000 7f0299394740 1 Processor -- start 2024-02-17T12:07:28.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.592+0000 7f0299394740 1 -- start start 2024-02-17T12:07:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.592+0000 7f0299394740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a13fd60 0x55645a14ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.592+0000 7f0299394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a140ad0 0x55645a14b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.592+0000 7f0299394740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a14d6f0 0x55645a154510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.592+0000 7f0299394740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556459da7600 con 0x55645a13fd60 2024-02-17T12:07:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.592+0000 7f0299394740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645a13b850 con 0x55645a14d6f0 2024-02-17T12:07:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.592+0000 7f0299394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645a13a310 con 0x55645a140ad0 2024-02-17T12:07:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.592+0000 7f028ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a13fd60 0x55645a14ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.592+0000 7f028ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a13fd60 0x55645a14ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56918/0 (socket says 172.21.15.94:56918) 2024-02-17T12:07:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.592+0000 7f028ffff640 1 -- 172.21.15.94:0/2756660746 learned_addr learned my addr 172.21.15.94:0/2756660746 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.592+0000 7f0294d59640 1 --2- 172.21.15.94:0/2756660746 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a14d6f0 0x55645a154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.593+0000 7f028f7fe640 1 --2- 172.21.15.94:0/2756660746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a140ad0 0x55645a14b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.593+0000 7f0294d59640 1 -- 172.21.15.94:0/2756660746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a140ad0 msgr2=0x55645a14b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:28.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.593+0000 7f0294d59640 1 --2- 172.21.15.94:0/2756660746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a140ad0 0x55645a14b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.593+0000 7f0294d59640 1 -- 172.21.15.94:0/2756660746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a13fd60 msgr2=0x55645a14ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.593+0000 7f0294d59640 1 --2- 172.21.15.94:0/2756660746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a13fd60 0x55645a14ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.593+0000 7f0294d59640 1 -- 172.21.15.94:0/2756660746 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55645a0427b0 con 0x55645a14d6f0 2024-02-17T12:07:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.593+0000 7f0294d59640 1 --2- 172.21.15.94:0/2756660746 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a14d6f0 0x55645a154510 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55645a13bbe0 tx=0x7f0288009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b574c53db4b2853e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.593+0000 7f0275ffb640 1 -- 172.21.15.94:0/2756660746 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288016070 con 0x55645a14d6f0 2024-02-17T12:07:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.594+0000 7f0275ffb640 1 -- 172.21.15.94:0/2756660746 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0288017040 con 0x55645a14d6f0 2024-02-17T12:07:28.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.594+0000 7f0275ffb640 1 -- 172.21.15.94:0/2756660746 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288010930 con 0x55645a14d6f0 2024-02-17T12:07:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.595+0000 7f0299394740 1 -- 172.21.15.94:0/2756660746 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a14d6f0 msgr2=0x55645a154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.595+0000 7f0299394740 1 --2- 172.21.15.94:0/2756660746 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a14d6f0 0x55645a154510 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55645a13bbe0 tx=0x7f0288009f90 comp rx=0 tx=0).stop 2024-02-17T12:07:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.595+0000 7f0299394740 1 -- 172.21.15.94:0/2756660746 shutdown_connections 2024-02-17T12:07:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.595+0000 7f0299394740 1 --2- 172.21.15.94:0/2756660746 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a14d6f0 0x55645a154510 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.595+0000 7f0299394740 1 --2- 172.21.15.94:0/2756660746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a140ad0 0x55645a14b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.595+0000 7f0299394740 1 --2- 172.21.15.94:0/2756660746 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a13fd60 0x55645a14ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.595+0000 7f0299394740 1 -- 172.21.15.94:0/2756660746 >> 172.21.15.94:0/2756660746 conn(0x55645a107550 msgr2=0x55645a13a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.595+0000 7f0299394740 1 -- 172.21.15.94:0/2756660746 shutdown_connections 2024-02-17T12:07:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.595+0000 7f0299394740 1 -- 172.21.15.94:0/2756660746 wait complete. 2024-02-17T12:07:28.597 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.596+0000 7f0299394740 1 -- start start 2024-02-17T12:07:28.598 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.596+0000 7f0299394740 1 Processor -- start 2024-02-17T12:07:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.596+0000 7f0299394740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a13fd60 0x55645a154710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.600 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.596+0000 7f0299394740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a140ad0 0x55645a146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.596+0000 7f0299394740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a14d6f0 0x55645a147530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.596+0000 7f0299394740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645a154e50 con 0x55645a140ad0 2024-02-17T12:07:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.596+0000 7f0299394740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645a147aa0 con 0x55645a13fd60 2024-02-17T12:07:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.596+0000 7f0299394740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645a147c60 con 0x55645a14d6f0 2024-02-17T12:07:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.596+0000 7f0299394740 10 client.0 _ll_register_callbacks cb 0x7f028400bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f028ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a13fd60 0x55645a154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f028f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a140ad0 0x55645a146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f028ffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a13fd60 0x55645a154710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52792/0 (socket says 172.21.15.94:52792) 2024-02-17T12:07:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f028f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a140ad0 0x55645a146ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56926/0 (socket says 172.21.15.94:56926) 2024-02-17T12:07:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f028ffff640 1 -- 172.21.15.94:0/3500610356 learned_addr learned my addr 172.21.15.94:0/3500610356 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f0294d59640 1 --2- 172.21.15.94:0/3500610356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a14d6f0 0x55645a147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f028ffff640 1 -- 172.21.15.94:0/3500610356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a14d6f0 msgr2=0x55645a147530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:28.601 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f028ffff640 1 --2- 172.21.15.94:0/3500610356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a14d6f0 0x55645a147530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f028ffff640 1 -- 172.21.15.94:0/3500610356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a140ad0 msgr2=0x55645a146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f028ffff640 1 --2- 172.21.15.94:0/3500610356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a140ad0 0x55645a146ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f028ffff640 1 -- 172.21.15.94:0/3500610356 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0288009e00 con 0x55645a13fd60 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f028f7fe640 1 --2- 172.21.15.94:0/3500610356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a140ad0 0x55645a146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f028ffff640 1 --2- 172.21.15.94:0/3500610356 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a13fd60 0x55645a154710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f028002de90 tx=0x7f02800bd100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54be270cccf44e48 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f026f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f026f7fe640 1 -- 172.21.15.94:0/3500610356 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800c57e0 con 0x55645a13fd60 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f0299394740 1 -- 172.21.15.94:0/3500610356 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0284015070 con 0x55645a13fd60 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.597+0000 7f026f7fe640 1 -- 172.21.15.94:0/3500610356 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f02800c5350 con 0x55645a13fd60 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.598+0000 7f026f7fe640 1 -- 172.21.15.94:0/3500610356 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02800c5510 con 0x55645a13fd60 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.598+0000 7f0299394740 10 client.27464 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.598+0000 7f0299394740 1 -- 172.21.15.94:0/3500610356 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55645a068540 con 0x55645a13fd60 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.598+0000 7f026f7fe640 1 -- 172.21.15.94:0/3500610356 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0284015070 con 0x55645a13fd60 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.598+0000 7f026f7fe640 1 -- 172.21.15.94:0/3500610356 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f028003c070 con 0x55645a13fd60 2024-02-17T12:07:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.598+0000 7f0299394740 10 client.27464 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.598+0000 7f0299394740 10 client.27464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.598+0000 7f0299394740 20 client.27464 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.598+0000 7f0299394740 1 --2- 172.21.15.94:0/3500610356 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55645a1b5090 0x55645a1b7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.598+0000 7f0299394740 4 client.27464 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:07:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.598+0000 7f0299394740 1 -- 172.21.15.94:0/3500610356 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556459de4550 con 0x55645a1b5090 2024-02-17T12:07:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.598+0000 7f028f7fe640 1 --2- 172.21.15.94:0/3500610356 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55645a1b5090 0x55645a1b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.598+0000 7f026f7fe640 10 client.27464 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:28.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.599+0000 7f028f7fe640 1 --2- 172.21.15.94:0/3500610356 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55645a1b5090 0x55645a1b7480 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a990d1241f004d9 server_cookie=e5c9d3ac33abf18f in_seq=0 out_seq=0 2024-02-17T12:07:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.666+0000 7f026f7fe640 1 -- 172.21.15.94:0/3500610356 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556459de4550 con 0x55645a1b5090 2024-02-17T12:07:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.666+0000 7f026f7fe640 10 client.27464 handle_command_reply: tid=0 2024-02-17T12:07:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.666+0000 7f0299394740 1 client.27464 shutdown 2024-02-17T12:07:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f0299394740 1 -- 172.21.15.94:0/3500610356 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a13fd60 msgr2=0x55645a154710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f0299394740 1 --2- 172.21.15.94:0/3500610356 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a13fd60 0x55645a154710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f028002de90 tx=0x7f02800bd100 comp rx=0 tx=0).stop 2024-02-17T12:07:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f0299394740 1 -- 172.21.15.94:0/3500610356 shutdown_connections 2024-02-17T12:07:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f0299394740 1 --2- 172.21.15.94:0/3500610356 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55645a1b5090 0x55645a1b7480 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f0299394740 1 --2- 172.21.15.94:0/3500610356 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55645a14d6f0 0x55645a147530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f026f7fe640 0 client.27464 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f0299394740 1 --2- 172.21.15.94:0/3500610356 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55645a140ad0 0x55645a146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f0299394740 1 --2- 172.21.15.94:0/3500610356 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55645a13fd60 0x55645a154710 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f0299394740 1 -- 172.21.15.94:0/3500610356 >> 172.21.15.94:0/3500610356 conn(0x55645a107550 msgr2=0x55645a148fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f0299394740 1 -- 172.21.15.94:0/3500610356 shutdown_connections 2024-02-17T12:07:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f0299394740 1 -- 172.21.15.94:0/3500610356 wait complete. 2024-02-17T12:07:28.669 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f0299394740 20 client.27464 trim_cache size 0 max 16384 2024-02-17T12:07:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:07:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:07:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:07:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:07:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f02967bc640 1 -- 172.21.15.94:0/631683600 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02780a2350 msgr2=0x7f02780a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f02967bc640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02780a2350 0x7f02780a4810 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f02900774d0 tx=0x7f028400b390 comp rx=0 tx=0).stop 2024-02-17T12:07:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f02967bc640 1 -- 172.21.15.94:0/631683600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0290068800 msgr2=0x7f0290197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f02967bc640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0290068800 0x7f0290197690 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0280005940 tx=0x7f028002c780 comp rx=0 tx=0).stop 2024-02-17T12:07:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f02967bc640 1 -- 172.21.15.94:0/631683600 shutdown_connections 2024-02-17T12:07:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f02967bc640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f02780a2350 0x7f02780a4810 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f02967bc640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0290107270 0x7f029019bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f02967bc640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f02900691a0 0x7f0290197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f02967bc640 1 --2- 172.21.15.94:0/631683600 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0290068800 0x7f0290197690 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.667+0000 7f02967bc640 1 -- 172.21.15.94:0/631683600 >> 172.21.15.94:0/631683600 conn(0x7f0290076ed0 msgr2=0x7f02900fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.668+0000 7f02967bc640 1 -- 172.21.15.94:0/631683600 shutdown_connections 2024-02-17T12:07:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:28.668+0000 7f02967bc640 1 -- 172.21.15.94:0/631683600 wait complete. 2024-02-17T12:07:29.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:28 smithi189 ceph-mon[36588]: pgmap v290: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 9.5 MiB/s wr, 903 op/s 2024-02-17T12:07:29.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:28 smithi125 ceph-mon[36819]: pgmap v290: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 9.5 MiB/s wr, 903 op/s 2024-02-17T12:07:29.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:28 smithi094 ceph-mon[32499]: pgmap v290: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 9.5 MiB/s wr, 903 op/s 2024-02-17T12:07:30.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 550.0 s, 1951.0 tps, lat 51.504 ms stddev 57.940 2024-02-17T12:07:31.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:31 smithi189 ceph-mon[36588]: pgmap v291: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.5 MiB/s wr, 532 op/s 2024-02-17T12:07:31.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:31 smithi125 ceph-mon[36819]: pgmap v291: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.5 MiB/s wr, 532 op/s 2024-02-17T12:07:31.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:31 smithi094 ceph-mon[32499]: pgmap v291: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.5 MiB/s wr, 532 op/s 2024-02-17T12:07:34.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:33 smithi189 ceph-mon[36588]: pgmap v292: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.6 MiB/s wr, 544 op/s 2024-02-17T12:07:34.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:33 smithi125 ceph-mon[36819]: pgmap v292: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.6 MiB/s wr, 544 op/s 2024-02-17T12:07:34.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:33 smithi094 ceph-mon[32499]: pgmap v292: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 6.6 MiB/s wr, 544 op/s 2024-02-17T12:07:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 555.0 s, 2285.8 tps, lat 43.034 ms stddev 47.625 2024-02-17T12:07:35.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:35 smithi189 ceph-mon[36588]: pgmap v293: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.3 MiB/s wr, 833 op/s 2024-02-17T12:07:35.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:35 smithi125 ceph-mon[36819]: pgmap v293: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.3 MiB/s wr, 833 op/s 2024-02-17T12:07:35.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:35 smithi094 ceph-mon[32499]: pgmap v293: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 9.3 MiB/s wr, 833 op/s 2024-02-17T12:07:37.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:37 smithi189 ceph-mon[36588]: pgmap v294: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.2 MiB/s wr, 603 op/s 2024-02-17T12:07:37.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:37 smithi125 ceph-mon[36819]: pgmap v294: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.2 MiB/s wr, 603 op/s 2024-02-17T12:07:37.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:37 smithi094 ceph-mon[32499]: pgmap v294: 97 pgs: 97 active+clean; 5.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.2 MiB/s wr, 603 op/s 2024-02-17T12:07:39.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:39 smithi189 ceph-mon[36588]: pgmap v295: 97 pgs: 97 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 1.84k op/s 2024-02-17T12:07:39.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:39 smithi125 ceph-mon[36819]: pgmap v295: 97 pgs: 97 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 1.84k op/s 2024-02-17T12:07:39.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:39 smithi094 ceph-mon[32499]: pgmap v295: 97 pgs: 97 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 26 MiB/s wr, 1.84k op/s 2024-02-17T12:07:40.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 560.0 s, 1194.4 tps, lat 84.944 ms stddev 150.524 2024-02-17T12:07:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.786+0000 7ff722242640 1 -- 172.21.15.94:0/1430432572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff71c1035d0 msgr2=0x7ff71c1039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.786+0000 7ff722242640 1 --2- 172.21.15.94:0/1430432572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff71c1035d0 0x7ff71c1039b0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009930 tx=0x7ff70c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:07:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.786+0000 7ff722242640 1 -- 172.21.15.94:0/1430432572 shutdown_connections 2024-02-17T12:07:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.786+0000 7ff722242640 1 --2- 172.21.15.94:0/1430432572 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff71c1086a0 0x7ff71c110270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.786+0000 7ff722242640 1 --2- 172.21.15.94:0/1430432572 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff71c103f80 0x7ff71c107fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.786+0000 7ff722242640 1 --2- 172.21.15.94:0/1430432572 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff71c1035d0 0x7ff71c1039b0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.786+0000 7ff722242640 1 -- 172.21.15.94:0/1430432572 >> 172.21.15.94:0/1430432572 conn(0x7ff71c0fee30 msgr2=0x7ff71c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.786+0000 7ff722242640 1 -- 172.21.15.94:0/1430432572 shutdown_connections 2024-02-17T12:07:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.786+0000 7ff722242640 1 -- 172.21.15.94:0/1430432572 wait complete. 2024-02-17T12:07:41.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.787+0000 7ff722242640 1 Processor -- start 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.787+0000 7ff722242640 1 -- start start 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.787+0000 7ff722242640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff71c1035d0 0x7ff71c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.787+0000 7ff722242640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff71c1086a0 0x7ff71c19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.787+0000 7ff722242640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff71c1997b0 0x7ff71c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.787+0000 7ff722242640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c19a260 con 0x7ff71c1997b0 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.787+0000 7ff722242640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c19a3d0 con 0x7ff71c1086a0 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.787+0000 7ff722242640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c19a540 con 0x7ff71c1035d0 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.787+0000 7ff71b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff71c1086a0 0x7ff71c19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.787+0000 7ff71bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff71c1997b0 0x7ff71c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff71b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff71c1086a0 0x7ff71c19fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:40642/0 (socket says 172.21.15.94:40642) 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff71b7fe640 1 -- 172.21.15.94:0/2528692273 learned_addr learned my addr 172.21.15.94:0/2528692273 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff71bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff71c1997b0 0x7ff71c199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58962/0 (socket says 172.21.15.94:58962) 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff713fff640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff71c1035d0 0x7ff71c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff71b7fe640 1 -- 172.21.15.94:0/2528692273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff71c1035d0 msgr2=0x7ff71c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:41.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff71b7fe640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff71c1035d0 0x7ff71c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff71b7fe640 1 -- 172.21.15.94:0/2528692273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff71c1997b0 msgr2=0x7ff71c199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff71b7fe640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff71c1997b0 0x7ff71c199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff71b7fe640 1 -- 172.21.15.94:0/2528692273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c009560 con 0x7ff71c1086a0 2024-02-17T12:07:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff71bfff640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff71c1997b0 0x7ff71c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff71b7fe640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff71c1086a0 0x7ff71c19fb80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff70400e9c0 tx=0x7ff70400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff7197fa640 1 -- 172.21.15.94:0/2528692273 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704009700 con 0x7ff71c1086a0 2024-02-17T12:07:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff7197fa640 1 -- 172.21.15.94:0/2528692273 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff70400cd80 con 0x7ff71c1086a0 2024-02-17T12:07:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff7197fa640 1 -- 172.21.15.94:0/2528692273 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704010430 con 0x7ff71c1086a0 2024-02-17T12:07:41.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff722242640 1 -- 172.21.15.94:0/2528692273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71c1a64f0 con 0x7ff71c1086a0 2024-02-17T12:07:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.788+0000 7ff722242640 1 -- 172.21.15.94:0/2528692273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71c1a6980 con 0x7ff71c1086a0 2024-02-17T12:07:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.792+0000 7ff724e0c740 1 Processor -- start 2024-02-17T12:07:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.792+0000 7ff724e0c740 1 -- start start 2024-02-17T12:07:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.792+0000 7ff724e0c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc63b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.792+0000 7ff724e0c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc63bd00 0x558bbc63c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.792+0000 7ff724e0c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc645280 0x558bbc647670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.792+0000 7ff724e0c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc5aa9f0 con 0x558bbc63ca70 2024-02-17T12:07:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.792+0000 7ff724e0c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc5a7f20 con 0x558bbc645280 2024-02-17T12:07:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.792+0000 7ff724e0c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc5a3700 con 0x558bbc63bd00 2024-02-17T12:07:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.792+0000 7ff71bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc63b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.793+0000 7ff71bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc63b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:58974/0 (socket says 172.21.15.94:58974) 2024-02-17T12:07:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.793+0000 7ff71bfff640 1 -- 172.21.15.94:0/166103665 learned_addr learned my addr 172.21.15.94:0/166103665 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.793+0000 7ff713fff640 1 --2- 172.21.15.94:0/166103665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc63bd00 0x558bbc63c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.793+0000 7ff71bfff640 1 -- 172.21.15.94:0/166103665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc63bd00 msgr2=0x558bbc63c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.793+0000 7ff71bfff640 1 --2- 172.21.15.94:0/166103665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc63bd00 0x558bbc63c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.793+0000 7ff71bfff640 1 -- 172.21.15.94:0/166103665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc645280 msgr2=0x558bbc647670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:07:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.793+0000 7ff71bfff640 1 --2- 172.21.15.94:0/166103665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc645280 0x558bbc647670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.793+0000 7ff71bfff640 1 -- 172.21.15.94:0/166103665 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bbc5397b0 con 0x558bbc63ca70 2024-02-17T12:07:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.793+0000 7ff71bfff640 1 --2- 172.21.15.94:0/166103665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc63b7c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff714011790 tx=0x7ff714011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f257f611107d36a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.793+0000 7ff711ffb640 1 -- 172.21.15.94:0/166103665 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff714011ea0 con 0x558bbc63ca70 2024-02-17T12:07:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.794+0000 7ff711ffb640 1 -- 172.21.15.94:0/166103665 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff71400fe80 con 0x558bbc63ca70 2024-02-17T12:07:41.795 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.794+0000 7ff711ffb640 1 -- 172.21.15.94:0/166103665 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff714004840 con 0x558bbc63ca70 2024-02-17T12:07:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.794+0000 7ff7197fa640 1 -- 172.21.15.94:0/2528692273 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff704005000 con 0x7ff71c1086a0 2024-02-17T12:07:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.794+0000 7ff724e0c740 1 -- 172.21.15.94:0/166103665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 msgr2=0x558bbc63b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.794+0000 7ff724e0c740 1 --2- 172.21.15.94:0/166103665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc63b7c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7ff714011790 tx=0x7ff714011c50 comp rx=0 tx=0).stop 2024-02-17T12:07:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.794+0000 7ff724e0c740 1 -- 172.21.15.94:0/166103665 shutdown_connections 2024-02-17T12:07:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.794+0000 7ff724e0c740 1 --2- 172.21.15.94:0/166103665 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc645280 0x558bbc647670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.794+0000 7ff724e0c740 1 --2- 172.21.15.94:0/166103665 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc63bd00 0x558bbc63c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.794+0000 7ff724e0c740 1 --2- 172.21.15.94:0/166103665 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc63b7c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.799 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.794+0000 7ff724e0c740 1 -- 172.21.15.94:0/166103665 >> 172.21.15.94:0/166103665 conn(0x558bbc603550 msgr2=0x558bbc6369f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.795+0000 7ff724e0c740 1 -- 172.21.15.94:0/166103665 shutdown_connections 2024-02-17T12:07:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.795+0000 7ff724e0c740 1 -- 172.21.15.94:0/166103665 wait complete. 2024-02-17T12:07:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.795+0000 7ff7197fa640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7080a2260 0x7ff7080a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.795+0000 7ff7197fa640 1 -- 172.21.15.94:0/2528692273 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff704014070 con 0x7ff71c1086a0 2024-02-17T12:07:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.795+0000 7ff713fff640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7080a2260 0x7ff7080a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.795+0000 7ff724e0c740 1 -- start start 2024-02-17T12:07:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.795+0000 7ff724e0c740 1 Processor -- start 2024-02-17T12:07:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.795+0000 7ff713fff640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7080a2260 0x7ff7080a4720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff70c005ab0 tx=0x7ff70c005fd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff724e0c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc6422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff724e0c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc63ca70 0x558bbc642810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff724e0c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc645280 0x558bbc642d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff724e0c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc548bf0 con 0x558bbc645280 2024-02-17T12:07:41.800 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff724e0c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc277250 con 0x558bbc63bd00 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff724e0c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc529200 con 0x558bbc63ca70 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff724e0c740 10 client.0 _ll_register_callbacks cb 0x558bbc597d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff713fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc6422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff713fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc6422d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:40662/0 (socket says 172.21.15.94:40662) 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff713fff640 1 -- 172.21.15.94:0/1094471416 learned_addr learned my addr 172.21.15.94:0/1094471416 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff71b7fe640 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc63ca70 0x558bbc642810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff71bfff640 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc645280 0x558bbc642d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff71b7fe640 1 -- 172.21.15.94:0/1094471416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 msgr2=0x558bbc6422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff71b7fe640 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc6422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff71b7fe640 1 -- 172.21.15.94:0/1094471416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc645280 msgr2=0x558bbc642d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff71b7fe640 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc645280 0x558bbc642d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff71b7fe640 1 -- 172.21.15.94:0/1094471416 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff714011440 con 0x558bbc63ca70 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff713fff640 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc6422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff71b7fe640 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc63ca70 0x558bbc642810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff7040cbcd0 tx=0x7ff70409a990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40978f0fba0591d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:41.801 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.796+0000 7ff6faffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff6faffd640 1 -- 172.21.15.94:0/1094471416 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7040cc150 con 0x558bbc63ca70 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff6faffd640 1 -- 172.21.15.94:0/1094471416 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff70409f0c0 con 0x558bbc63ca70 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff724e0c740 1 -- 172.21.15.94:0/1094471416 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bbc2a3600 con 0x558bbc63ca70 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff6faffd640 1 -- 172.21.15.94:0/1094471416 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7040a3710 con 0x558bbc63ca70 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff724e0c740 10 client.27469 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff724e0c740 1 -- 172.21.15.94:0/1094471416 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558bbc5a1d80 con 0x558bbc63ca70 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff6faffd640 1 -- 172.21.15.94:0/1094471416 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558bbc2a3600 con 0x558bbc63ca70 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff6faffd640 1 -- 172.21.15.94:0/1094471416 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff70401f060 con 0x558bbc63ca70 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff724e0c740 10 client.27469 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff724e0c740 10 client.27469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff724e0c740 20 client.27469 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff724e0c740 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558bbc6555f0 0x558bbc657a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff724e0c740 4 client.27469 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff724e0c740 1 -- 172.21.15.94:0/1094471416 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bbc2f41b0 con 0x558bbc6555f0 2024-02-17T12:07:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.797+0000 7ff713fff640 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558bbc6555f0 0x558bbc657a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.798+0000 7ff6faffd640 10 client.27469 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.798+0000 7ff713fff640 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558bbc6555f0 0x558bbc657a10 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65b09cf1455e6137 server_cookie=e044aa2a76f06651 in_seq=0 out_seq=0 2024-02-17T12:07:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.798+0000 7ff6faffd640 1 -- 172.21.15.94:0/1094471416 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558bbc2f41b0 con 0x558bbc6555f0 2024-02-17T12:07:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.798+0000 7ff6faffd640 10 client.27469 handle_command_reply: tid=0 2024-02-17T12:07:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.799+0000 7ff724e0c740 1 client.27469 shutdown 2024-02-17T12:07:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.799+0000 7ff724e0c740 1 -- 172.21.15.94:0/1094471416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc63ca70 msgr2=0x558bbc642810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.799+0000 7ff724e0c740 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc63ca70 0x558bbc642810 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff7040cbcd0 tx=0x7ff70409a990 comp rx=0 tx=0).stop 2024-02-17T12:07:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.799+0000 7ff724e0c740 1 -- 172.21.15.94:0/1094471416 shutdown_connections 2024-02-17T12:07:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.799+0000 7ff724e0c740 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558bbc6555f0 0x558bbc657a10 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.804 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.799+0000 7ff724e0c740 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc645280 0x558bbc642d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.799+0000 7ff724e0c740 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc63ca70 0x558bbc642810 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.799+0000 7ff6faffd640 0 client.27469 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.799+0000 7ff724e0c740 1 --2- 172.21.15.94:0/1094471416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc6422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.799+0000 7ff724e0c740 1 -- 172.21.15.94:0/1094471416 >> 172.21.15.94:0/1094471416 conn(0x558bbc603550 msgr2=0x558bbc638740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.799+0000 7ff724e0c740 1 -- 172.21.15.94:0/1094471416 shutdown_connections 2024-02-17T12:07:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.799+0000 7ff724e0c740 1 -- 172.21.15.94:0/1094471416 wait complete. 2024-02-17T12:07:41.805 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.799+0000 7ff724e0c740 20 client.27469 trim_cache size 0 max 16384 2024-02-17T12:07:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.816+0000 7ff724e0c740 1 Processor -- start 2024-02-17T12:07:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.816+0000 7ff724e0c740 1 -- start start 2024-02-17T12:07:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff724e0c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc644a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff724e0c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc65c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff724e0c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc645280 0x558bbc640ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff724e0c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc2f41b0 con 0x558bbc63ca70 2024-02-17T12:07:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff724e0c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc2a3600 con 0x558bbc63bd00 2024-02-17T12:07:41.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff724e0c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc6432a0 con 0x558bbc645280 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff713fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc644a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff713fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc644a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:40676/0 (socket says 172.21.15.94:40676) 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff713fff640 1 -- 172.21.15.94:0/2928912207 learned_addr learned my addr 172.21.15.94:0/2928912207 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff71b7fe640 1 --2- 172.21.15.94:0/2928912207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc65c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff71bfff640 1 --2- 172.21.15.94:0/2928912207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc645280 0x558bbc640ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff713fff640 1 -- 172.21.15.94:0/2928912207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc645280 msgr2=0x558bbc640ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff713fff640 1 --2- 172.21.15.94:0/2928912207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc645280 0x558bbc640ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff713fff640 1 -- 172.21.15.94:0/2928912207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 msgr2=0x558bbc65c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff713fff640 1 --2- 172.21.15.94:0/2928912207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc65c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff713fff640 1 -- 172.21.15.94:0/2928912207 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bbc53e7b0 con 0x558bbc63bd00 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff713fff640 1 --2- 172.21.15.94:0/2928912207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc644a00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff70c036040 tx=0x7ff70c045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=529a68736416a737 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.817+0000 7ff7117fa640 1 -- 172.21.15.94:0/2928912207 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c052040 con 0x558bbc63bd00 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.818+0000 7ff7117fa640 1 -- 172.21.15.94:0/2928912207 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff70c02c7e0 con 0x558bbc63bd00 2024-02-17T12:07:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.818+0000 7ff7117fa640 1 -- 172.21.15.94:0/2928912207 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff70c0038e0 con 0x558bbc63bd00 2024-02-17T12:07:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.818+0000 7ff724e0c740 1 -- 172.21.15.94:0/2928912207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 msgr2=0x558bbc644a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.818+0000 7ff724e0c740 1 --2- 172.21.15.94:0/2928912207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc644a00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff70c036040 tx=0x7ff70c045000 comp rx=0 tx=0).stop 2024-02-17T12:07:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.818+0000 7ff724e0c740 1 -- 172.21.15.94:0/2928912207 shutdown_connections 2024-02-17T12:07:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.818+0000 7ff724e0c740 1 --2- 172.21.15.94:0/2928912207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc645280 0x558bbc640ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.818+0000 7ff724e0c740 1 --2- 172.21.15.94:0/2928912207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc65c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.818+0000 7ff724e0c740 1 --2- 172.21.15.94:0/2928912207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc644a00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.818+0000 7ff724e0c740 1 -- 172.21.15.94:0/2928912207 >> 172.21.15.94:0/2928912207 conn(0x558bbc603550 msgr2=0x558bbc63d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.818+0000 7ff724e0c740 1 -- 172.21.15.94:0/2928912207 shutdown_connections 2024-02-17T12:07:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.818+0000 7ff724e0c740 1 -- 172.21.15.94:0/2928912207 wait complete. 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.819+0000 7ff724e0c740 1 -- start start 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.819+0000 7ff724e0c740 1 Processor -- start 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.819+0000 7ff724e0c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc64bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.819+0000 7ff724e0c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc64c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.819+0000 7ff724e0c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc645280 0x558bbc64a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.819+0000 7ff724e0c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc64a8f0 con 0x558bbc63ca70 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.819+0000 7ff724e0c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc64aa60 con 0x558bbc63bd00 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.819+0000 7ff724e0c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bbc64ac20 con 0x558bbc645280 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.819+0000 7ff724e0c740 10 client.0 _ll_register_callbacks cb 0x7ff714009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff71b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc64c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff71b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc64c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59008/0 (socket says 172.21.15.94:59008) 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff71b7fe640 1 -- 172.21.15.94:0/3257992586 learned_addr learned my addr 172.21.15.94:0/3257992586 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff71bfff640 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc645280 0x558bbc64a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff713fff640 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc64bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff71b7fe640 1 -- 172.21.15.94:0/3257992586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc645280 msgr2=0x558bbc64a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff71b7fe640 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc645280 0x558bbc64a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff71b7fe640 1 -- 172.21.15.94:0/3257992586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 msgr2=0x558bbc64bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff71b7fe640 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc64bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff71b7fe640 1 -- 172.21.15.94:0/3257992586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71c110a80 con 0x558bbc63ca70 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff713fff640 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc64bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff71b7fe640 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc64c2d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff704009250 tx=0x7ff70409aa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56bb3c0ce212432 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff6faffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff6faffd640 1 -- 172.21.15.94:0/3257992586 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7040054b0 con 0x558bbc63ca70 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff6faffd640 1 -- 172.21.15.94:0/3257992586 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff70409f0c0 con 0x558bbc63ca70 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.820+0000 7ff724e0c740 1 -- 172.21.15.94:0/3257992586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bbc642100 con 0x558bbc63ca70 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.821+0000 7ff6faffd640 1 -- 172.21.15.94:0/3257992586 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff70409aab0 con 0x558bbc63ca70 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.821+0000 7ff724e0c740 10 client.17994 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.821+0000 7ff724e0c740 1 -- 172.21.15.94:0/3257992586 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558bbc564540 con 0x558bbc63ca70 2024-02-17T12:07:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.821+0000 7ff6faffd640 1 -- 172.21.15.94:0/3257992586 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558bbc642100 con 0x558bbc63ca70 2024-02-17T12:07:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.821+0000 7ff6faffd640 1 -- 172.21.15.94:0/3257992586 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff70401f070 con 0x558bbc63ca70 2024-02-17T12:07:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.821+0000 7ff724e0c740 10 client.17994 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.821+0000 7ff724e0c740 10 client.17994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.821+0000 7ff724e0c740 20 client.17994 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.821+0000 7ff724e0c740 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558bbc6ac970 0x558bbc6aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.821+0000 7ff724e0c740 4 client.17994 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:07:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.821+0000 7ff724e0c740 1 -- 172.21.15.94:0/3257992586 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7ff71400fe80 con 0x558bbc6ac970 2024-02-17T12:07:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.822+0000 7ff713fff640 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558bbc6ac970 0x558bbc6aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.822+0000 7ff6faffd640 10 client.17994 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.822+0000 7ff713fff640 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558bbc6ac970 0x558bbc6aed90 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f394a72afa871f6b server_cookie=1df5fceb0987fef6 in_seq=0 out_seq=0 2024-02-17T12:07:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff6faffd640 1 -- 172.21.15.94:0/3257992586 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7ff71400fe80 con 0x558bbc6ac970 2024-02-17T12:07:41.839 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff6faffd640 10 client.17994 handle_command_reply: tid=0 2024-02-17T12:07:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff724e0c740 1 client.17994 shutdown 2024-02-17T12:07:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff724e0c740 1 -- 172.21.15.94:0/3257992586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 msgr2=0x558bbc64c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff724e0c740 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc64c2d0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff704009250 tx=0x7ff70409aa00 comp rx=0 tx=0).stop 2024-02-17T12:07:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff724e0c740 1 -- 172.21.15.94:0/3257992586 shutdown_connections 2024-02-17T12:07:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff724e0c740 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558bbc6ac970 0x558bbc6aed90 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff724e0c740 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558bbc645280 0x558bbc64a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff724e0c740 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558bbc63ca70 0x558bbc64c2d0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff724e0c740 1 --2- 172.21.15.94:0/3257992586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558bbc63bd00 0x558bbc64bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff724e0c740 1 -- 172.21.15.94:0/3257992586 >> 172.21.15.94:0/3257992586 conn(0x558bbc603550 msgr2=0x558bbc63b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff6faffd640 0 client.17994 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff724e0c740 1 -- 172.21.15.94:0/3257992586 shutdown_connections 2024-02-17T12:07:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff724e0c740 1 -- 172.21.15.94:0/3257992586 wait complete. 2024-02-17T12:07:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.838+0000 7ff724e0c740 20 client.17994 trim_cache size 0 max 16384 2024-02-17T12:07:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:07:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:07:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:07:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:07:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.840+0000 7ff722242640 1 -- 172.21.15.94:0/2528692273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7080a2260 msgr2=0x7ff7080a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.840+0000 7ff722242640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7080a2260 0x7ff7080a4720 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff70c005ab0 tx=0x7ff70c005fd0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.840+0000 7ff722242640 1 -- 172.21.15.94:0/2528692273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff71c1086a0 msgr2=0x7ff71c19fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.840+0000 7ff722242640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff71c1086a0 0x7ff71c19fb80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff70400e9c0 tx=0x7ff70400ee80 comp rx=0 tx=0).stop 2024-02-17T12:07:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.841+0000 7ff722242640 1 -- 172.21.15.94:0/2528692273 shutdown_connections 2024-02-17T12:07:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.841+0000 7ff722242640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7080a2260 0x7ff7080a4720 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff70c005ab0 tx=0x7ff70c005fd0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.841+0000 7ff722242640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff71c1997b0 0x7ff71c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.841+0000 7ff722242640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff71c1086a0 0x7ff71c19fb80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.841+0000 7ff722242640 1 --2- 172.21.15.94:0/2528692273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff71c1035d0 0x7ff71c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.841+0000 7ff722242640 1 -- 172.21.15.94:0/2528692273 >> 172.21.15.94:0/2528692273 conn(0x7ff71c0fee30 msgr2=0x7ff71c101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.841+0000 7ff722242640 1 -- 172.21.15.94:0/2528692273 shutdown_connections 2024-02-17T12:07:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:41.841+0000 7ff722242640 1 -- 172.21.15.94:0/2528692273 wait complete. 2024-02-17T12:07:42.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:41 smithi189 ceph-mon[36588]: pgmap v296: 97 pgs: 97 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2024-02-17T12:07:42.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:41 smithi125 ceph-mon[36819]: pgmap v296: 97 pgs: 97 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2024-02-17T12:07:42.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:41 smithi094 ceph-mon[32499]: pgmap v296: 97 pgs: 97 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2024-02-17T12:07:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:43 smithi189 ceph-mon[36588]: pgmap v297: 97 pgs: 97 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2024-02-17T12:07:43.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:43 smithi125 ceph-mon[36819]: pgmap v297: 97 pgs: 97 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2024-02-17T12:07:43.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:43 smithi094 ceph-mon[32499]: pgmap v297: 97 pgs: 97 active+clean; 5.4 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 1.55k op/s 2024-02-17T12:07:45.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 565.0 s, 1540.2 tps, lat 64.823 ms stddev 65.758 2024-02-17T12:07:46.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:45 smithi189 ceph-mon[36588]: pgmap v298: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 25 MiB/s wr, 1.88k op/s 2024-02-17T12:07:46.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:45 smithi125 ceph-mon[36819]: pgmap v298: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 25 MiB/s wr, 1.88k op/s 2024-02-17T12:07:46.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:45 smithi094 ceph-mon[32499]: pgmap v298: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 25 MiB/s wr, 1.88k op/s 2024-02-17T12:07:46.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:07:46.757 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:07:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.916+0000 7f7977dd7640 1 -- 172.21.15.94:0/400388565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79701064e0 msgr2=0x7f797010e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.916+0000 7f7977dd7640 1 --2- 172.21.15.94:0/400388565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79701064e0 0x7f797010e040 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7960009900 tx=0x7f796002bc30 comp rx=0 tx=0).stop 2024-02-17T12:07:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.916+0000 7f7977dd7640 1 -- 172.21.15.94:0/400388565 shutdown_connections 2024-02-17T12:07:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.916+0000 7f7977dd7640 1 --2- 172.21.15.94:0/400388565 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79701064e0 0x7f797010e040 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.916+0000 7f7977dd7640 1 --2- 172.21.15.94:0/400388565 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7970100280 0x7f7970105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.916+0000 7f7977dd7640 1 --2- 172.21.15.94:0/400388565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79700ff8d0 0x7f79700ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.916+0000 7f7977dd7640 1 -- 172.21.15.94:0/400388565 >> 172.21.15.94:0/400388565 conn(0x7f79700765b0 msgr2=0x7f79700fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.916+0000 7f7977dd7640 1 -- 172.21.15.94:0/400388565 shutdown_connections 2024-02-17T12:07:46.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.917+0000 7f7977dd7640 1 -- 172.21.15.94:0/400388565 wait complete. 2024-02-17T12:07:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.917+0000 7f7977dd7640 1 Processor -- start 2024-02-17T12:07:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.917+0000 7f7977dd7640 1 -- start start 2024-02-17T12:07:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7977dd7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79700ff8d0 0x7f7970197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7977dd7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79701064e0 0x7f7970197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7977dd7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f797019bde0 0x7f797019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7977dd7640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797019c7d0 con 0x7f797019bde0 2024-02-17T12:07:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7977dd7640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797019c940 con 0x7f79701064e0 2024-02-17T12:07:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7977dd7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f797019cab0 con 0x7f79700ff8d0 2024-02-17T12:07:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7975b73640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79700ff8d0 0x7f7970197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7976374640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f797019bde0 0x7f797019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7975b73640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79700ff8d0 0x7f7970197520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55900/0 (socket says 172.21.15.94:55900) 2024-02-17T12:07:46.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7975b73640 1 -- 172.21.15.94:0/1797793470 learned_addr learned my addr 172.21.15.94:0/1797793470 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:46.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7975372640 1 --2- 172.21.15.94:0/1797793470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79701064e0 0x7f7970197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7976374640 1 -- 172.21.15.94:0/1797793470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79700ff8d0 msgr2=0x7f7970197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:46.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7976374640 1 --2- 172.21.15.94:0/1797793470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79700ff8d0 0x7f7970197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7976374640 1 -- 172.21.15.94:0/1797793470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79701064e0 msgr2=0x7f7970197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:46.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7976374640 1 --2- 172.21.15.94:0/1797793470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79701064e0 0x7f7970197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.918+0000 7f7976374640 1 -- 172.21.15.94:0/1797793470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7960009560 con 0x7f797019bde0 2024-02-17T12:07:46.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.919+0000 7f7976374640 1 --2- 172.21.15.94:0/1797793470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f797019bde0 0x7f797019c290 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7960009850 tx=0x7f7960004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:46.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.919+0000 7f796effd640 1 -- 172.21.15.94:0/1797793470 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79600044f0 con 0x7f797019bde0 2024-02-17T12:07:46.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.919+0000 7f796effd640 1 -- 172.21.15.94:0/1797793470 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79600052d0 con 0x7f797019bde0 2024-02-17T12:07:46.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.919+0000 7f7977dd7640 1 -- 172.21.15.94:0/1797793470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7970100770 con 0x7f797019bde0 2024-02-17T12:07:46.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.919+0000 7f7977dd7640 1 -- 172.21.15.94:0/1797793470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7970100c80 con 0x7f797019bde0 2024-02-17T12:07:46.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.919+0000 7f796effd640 1 -- 172.21.15.94:0/1797793470 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960003d80 con 0x7f797019bde0 2024-02-17T12:07:46.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.923+0000 7f797a9af740 1 Processor -- start 2024-02-17T12:07:46.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.923+0000 7f796effd640 1 -- 172.21.15.94:0/1797793470 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f796003e450 con 0x7f797019bde0 2024-02-17T12:07:46.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.924+0000 7f796effd640 1 --2- 172.21.15.94:0/1797793470 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f79580a2380 0x7f79580a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.924+0000 7f797a9af740 1 -- start start 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.924+0000 7f796effd640 1 -- 172.21.15.94:0/1797793470 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f79600f0650 con 0x7f797019bde0 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.924+0000 7f797a9af740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601afe0 0x56370601b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.924+0000 7f797a9af740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 0x56370602ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.924+0000 7f797a9af740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x56370602f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.924+0000 7f797a9af740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563705f8a280 con 0x56370601b900 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.924+0000 7f797a9af740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563705f877b0 con 0x56370601afe0 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.924+0000 7f7975372640 1 --2- 172.21.15.94:0/1797793470 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f79580a2380 0x7f79580a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.924+0000 7f797a9af740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563705f82f90 con 0x56370602cfc0 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7975b73640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601afe0 0x56370601b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7976374640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x56370602f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7975372640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 0x56370602ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7976374640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x56370602f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:55908/0 (socket says 172.21.15.94:55908) 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7975b73640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601afe0 0x56370601b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:58432/0 (socket says 172.21.15.94:58432) 2024-02-17T12:07:46.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7976374640 1 -- 172.21.15.94:0/1301667361 learned_addr learned my addr 172.21.15.94:0/1301667361 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:46.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7976374640 1 -- 172.21.15.94:0/1301667361 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601afe0 msgr2=0x56370601b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:46.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7976374640 1 --2- 172.21.15.94:0/1301667361 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601afe0 0x56370601b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7976374640 1 -- 172.21.15.94:0/1301667361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 msgr2=0x56370602ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:46.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7976374640 1 --2- 172.21.15.94:0/1301667361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 0x56370602ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7976374640 1 -- 172.21.15.94:0/1301667361 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563705f41570 con 0x56370602cfc0 2024-02-17T12:07:46.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7975372640 1 --2- 172.21.15.94:0/1797793470 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f79580a2380 0x7f79580a4840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7970103980 tx=0x7f7964012810 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:46.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7975b73640 1 --2- 172.21.15.94:0/1301667361 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601afe0 0x56370601b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:46.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7976374640 1 --2- 172.21.15.94:0/1301667361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x56370602f3b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f79600bcfe0 tx=0x7f79600bccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff0312f849aded3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:46.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7957fff640 1 -- 172.21.15.94:0/1301667361 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79600c1120 con 0x56370602cfc0 2024-02-17T12:07:46.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7957fff640 1 -- 172.21.15.94:0/1301667361 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79600edf80 con 0x56370602cfc0 2024-02-17T12:07:46.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.925+0000 7f7957fff640 1 -- 172.21.15.94:0/1301667361 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79600c2530 con 0x56370602cfc0 2024-02-17T12:07:46.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.926+0000 7f797a9af740 1 -- 172.21.15.94:0/1301667361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 msgr2=0x56370602f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.926+0000 7f797a9af740 1 --2- 172.21.15.94:0/1301667361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x56370602f3b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f79600bcfe0 tx=0x7f79600bccc0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.926+0000 7f797a9af740 1 -- 172.21.15.94:0/1301667361 shutdown_connections 2024-02-17T12:07:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.926+0000 7f797a9af740 1 --2- 172.21.15.94:0/1301667361 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x56370602f3b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.926+0000 7f797a9af740 1 --2- 172.21.15.94:0/1301667361 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 0x56370602ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.926+0000 7f797a9af740 1 --2- 172.21.15.94:0/1301667361 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601afe0 0x56370601b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.926+0000 7f797a9af740 1 -- 172.21.15.94:0/1301667361 >> 172.21.15.94:0/1301667361 conn(0x563706015990 msgr2=0x563706015d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.927+0000 7f797a9af740 1 -- 172.21.15.94:0/1301667361 shutdown_connections 2024-02-17T12:07:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.927+0000 7f797a9af740 1 -- 172.21.15.94:0/1301667361 wait complete. 2024-02-17T12:07:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.927+0000 7f797a9af740 1 -- start start 2024-02-17T12:07:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.927+0000 7f797a9af740 1 Processor -- start 2024-02-17T12:07:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.927+0000 7f797a9af740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601afe0 0x56370602c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f797a9af740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601b900 0x563706029d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f797a9af740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x56370602a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f797a9af740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563705c56430 con 0x56370601afe0 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f797a9af740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563705c12e00 con 0x56370601b900 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f797a9af740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563705f07fd0 con 0x56370602cfc0 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f7975b73640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601afe0 0x56370602c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f7975b73640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601afe0 0x56370602c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39762/0 (socket says 172.21.15.94:39762) 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f7975b73640 1 -- 172.21.15.94:0/1335490890 learned_addr learned my addr 172.21.15.94:0/1335490890 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f797a9af740 10 client.0 _ll_register_callbacks cb 0x563705f5dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f7975372640 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601b900 0x563706029d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f7976374640 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x56370602a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f7975b73640 1 -- 172.21.15.94:0/1335490890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 msgr2=0x56370602a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f7975b73640 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x56370602a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f7975b73640 1 -- 172.21.15.94:0/1335490890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601b900 msgr2=0x563706029d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f7975b73640 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601b900 0x563706029d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f7975b73640 1 -- 172.21.15.94:0/1335490890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7970100c80 con 0x56370601afe0 2024-02-17T12:07:46.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f7975372640 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601b900 0x563706029d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:46.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f7976374640 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x56370602a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:07:46.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f7975b73640 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601afe0 0x56370602c9a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7968012d20 tx=0x7f796800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b60b7cee80a10e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:46.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f79557fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:46.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.928+0000 7f79557fa640 1 -- 172.21.15.94:0/1335490890 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968017070 con 0x56370601afe0 2024-02-17T12:07:46.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.929+0000 7f797a9af740 1 -- 172.21.15.94:0/1335490890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563705c827e0 con 0x56370601afe0 2024-02-17T12:07:46.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.929+0000 7f79557fa640 1 -- 172.21.15.94:0/1335490890 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f796800fe20 con 0x56370601afe0 2024-02-17T12:07:46.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.930+0000 7f79557fa640 1 -- 172.21.15.94:0/1335490890 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79680122e0 con 0x56370601afe0 2024-02-17T12:07:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.931+0000 7f797a9af740 10 client.18006 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.931+0000 7f797a9af740 1 -- 172.21.15.94:0/1335490890 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563705f81610 con 0x56370601afe0 2024-02-17T12:07:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.931+0000 7f79557fa640 1 -- 172.21.15.94:0/1335490890 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x563705c827e0 con 0x56370601afe0 2024-02-17T12:07:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.931+0000 7f79557fa640 1 -- 172.21.15.94:0/1335490890 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f796801c650 con 0x56370601afe0 2024-02-17T12:07:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.931+0000 7f797a9af740 10 client.18006 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.931+0000 7f797a9af740 10 client.18006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.931+0000 7f797a9af740 20 client.18006 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.931+0000 7f797a9af740 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56370603d050 0x56370603f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.931+0000 7f797a9af740 4 client.18006 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.931+0000 7f797a9af740 1 -- 172.21.15.94:0/1335490890 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563705cbf720 con 0x56370603d050 2024-02-17T12:07:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.932+0000 7f7975372640 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56370603d050 0x56370603f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.932+0000 7f79557fa640 10 client.18006 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:46.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.932+0000 7f7975372640 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56370603d050 0x56370603f470 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d89cc389556350ed server_cookie=8ec96fdd5a7b0c89 in_seq=0 out_seq=0 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.933+0000 7f79557fa640 1 -- 172.21.15.94:0/1335490890 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563705cbf720 con 0x56370603d050 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.933+0000 7f79557fa640 10 client.18006 handle_command_reply: tid=0 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.933+0000 7f797a9af740 1 client.18006 shutdown 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.933+0000 7f797a9af740 1 -- 172.21.15.94:0/1335490890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601afe0 msgr2=0x56370602c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.933+0000 7f797a9af740 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601afe0 0x56370602c9a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f7968012d20 tx=0x7f796800f790 comp rx=0 tx=0).stop 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.933+0000 7f797a9af740 1 -- 172.21.15.94:0/1335490890 shutdown_connections 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.933+0000 7f797a9af740 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56370603d050 0x56370603f470 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.933+0000 7f797a9af740 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x56370602a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.933+0000 7f797a9af740 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601b900 0x563706029d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.933+0000 7f797a9af740 1 --2- 172.21.15.94:0/1335490890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601afe0 0x56370602c9a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.933+0000 7f79557fa640 0 client.18006 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.933+0000 7f797a9af740 1 -- 172.21.15.94:0/1335490890 >> 172.21.15.94:0/1335490890 conn(0x563706015990 msgr2=0x563706017730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.934+0000 7f797a9af740 1 -- 172.21.15.94:0/1335490890 shutdown_connections 2024-02-17T12:07:46.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.934+0000 7f797a9af740 1 -- 172.21.15.94:0/1335490890 wait complete. 2024-02-17T12:07:46.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.934+0000 7f797a9af740 20 client.18006 trim_cache size 0 max 16384 2024-02-17T12:07:46.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.954+0000 7f797a9af740 1 Processor -- start 2024-02-17T12:07:46.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.954+0000 7f797a9af740 1 -- start start 2024-02-17T12:07:46.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.954+0000 7f797a9af740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370601afe0 0x5637060148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.954+0000 7f797a9af740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 0x563706029630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.954+0000 7f797a9af740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370602cfc0 0x563706029b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.954+0000 7f797a9af740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563705cbf720 con 0x56370601b900 2024-02-17T12:07:46.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f797a9af740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563705c827e0 con 0x56370602cfc0 2024-02-17T12:07:46.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f797a9af740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56370601be00 con 0x56370601afe0 2024-02-17T12:07:46.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f7976374640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370602cfc0 0x563706029b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f7976374640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370602cfc0 0x563706029b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:58446/0 (socket says 172.21.15.94:58446) 2024-02-17T12:07:46.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f7976374640 1 -- 172.21.15.94:0/3989628939 learned_addr learned my addr 172.21.15.94:0/3989628939 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:46.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f7975372640 1 --2- 172.21.15.94:0/3989628939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 0x563706029630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f7976374640 1 -- 172.21.15.94:0/3989628939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370601afe0 msgr2=0x5637060148d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:07:46.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f7976374640 1 --2- 172.21.15.94:0/3989628939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370601afe0 0x5637060148d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f7976374640 1 -- 172.21.15.94:0/3989628939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 msgr2=0x563706029630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:46.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f7976374640 1 --2- 172.21.15.94:0/3989628939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 0x563706029630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f7976374640 1 -- 172.21.15.94:0/3989628939 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563705f1dac0 con 0x56370602cfc0 2024-02-17T12:07:46.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f7975372640 1 --2- 172.21.15.94:0/3989628939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 0x563706029630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:07:46.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f7976374640 1 --2- 172.21.15.94:0/3989628939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370602cfc0 0x563706029b70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f79600bcdb0 tx=0x7f79600bb7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=203468522088c209 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:46.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f79577fe640 1 -- 172.21.15.94:0/3989628939 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79600b9580 con 0x56370602cfc0 2024-02-17T12:07:46.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f79577fe640 1 -- 172.21.15.94:0/3989628939 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79600ee0f0 con 0x56370602cfc0 2024-02-17T12:07:46.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.955+0000 7f79577fe640 1 -- 172.21.15.94:0/3989628939 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960005440 con 0x56370602cfc0 2024-02-17T12:07:46.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.956+0000 7f797a9af740 1 -- 172.21.15.94:0/3989628939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370602cfc0 msgr2=0x563706029b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:46.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.956+0000 7f797a9af740 1 --2- 172.21.15.94:0/3989628939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370602cfc0 0x563706029b70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f79600bcdb0 tx=0x7f79600bb7f0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.956+0000 7f797a9af740 1 -- 172.21.15.94:0/3989628939 shutdown_connections 2024-02-17T12:07:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.956+0000 7f797a9af740 1 --2- 172.21.15.94:0/3989628939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370602cfc0 0x563706029b70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.956+0000 7f797a9af740 1 --2- 172.21.15.94:0/3989628939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 0x563706029630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.956+0000 7f797a9af740 1 --2- 172.21.15.94:0/3989628939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370601afe0 0x5637060148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.956+0000 7f797a9af740 1 -- 172.21.15.94:0/3989628939 >> 172.21.15.94:0/3989628939 conn(0x563706015990 msgr2=0x5637060141c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.956+0000 7f797a9af740 1 -- 172.21.15.94:0/3989628939 shutdown_connections 2024-02-17T12:07:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.956+0000 7f797a9af740 1 -- 172.21.15.94:0/3989628939 wait complete. 2024-02-17T12:07:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.956+0000 7f797a9af740 1 -- start start 2024-02-17T12:07:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.957+0000 7f797a9af740 1 Processor -- start 2024-02-17T12:07:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.957+0000 7f797a9af740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601afe0 0x563706034070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.957+0000 7f797a9af740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 0x5637060303f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.957+0000 7f797a9af740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x563706030930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.957+0000 7f797a9af740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563706031030 con 0x56370601b900 2024-02-17T12:07:46.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.957+0000 7f797a9af740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637060311a0 con 0x56370601afe0 2024-02-17T12:07:46.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.957+0000 7f797a9af740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563706093550 con 0x56370602cfc0 2024-02-17T12:07:46.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.957+0000 7f797a9af740 10 client.0 _ll_register_callbacks cb 0x7f7960003ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:46.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.957+0000 7f7975b73640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601afe0 0x563706034070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.957+0000 7f7975b73640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601afe0 0x563706034070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:58454/0 (socket says 172.21.15.94:58454) 2024-02-17T12:07:46.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.957+0000 7f7975b73640 1 -- 172.21.15.94:0/3966146734 learned_addr learned my addr 172.21.15.94:0/3966146734 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:46.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.957+0000 7f7976374640 1 --2- 172.21.15.94:0/3966146734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x563706030930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.958+0000 7f7976374640 1 -- 172.21.15.94:0/3966146734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601afe0 msgr2=0x563706034070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:46.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.958+0000 7f7976374640 1 --2- 172.21.15.94:0/3966146734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601afe0 0x563706034070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.958+0000 7f7976374640 1 -- 172.21.15.94:0/3966146734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 msgr2=0x5637060303f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:46.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.958+0000 7f7976374640 1 --2- 172.21.15.94:0/3966146734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 0x5637060303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.958+0000 7f7976374640 1 -- 172.21.15.94:0/3966146734 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563705f41570 con 0x56370602cfc0 2024-02-17T12:07:46.961 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.958+0000 7f7976374640 1 --2- 172.21.15.94:0/3966146734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x563706030930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f796002c170 tx=0x7f79600b9b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c75f261b545930 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:46.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.959+0000 7f79557fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:07:46.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.959+0000 7f79557fa640 1 -- 172.21.15.94:0/3966146734 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960005650 con 0x56370602cfc0 2024-02-17T12:07:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.959+0000 7f79557fa640 1 -- 172.21.15.94:0/3966146734 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f79600c23d0 con 0x56370602cfc0 2024-02-17T12:07:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.960+0000 7f797a9af740 1 -- 172.21.15.94:0/3966146734 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79600edf80 con 0x56370602cfc0 2024-02-17T12:07:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.960+0000 7f79557fa640 1 -- 172.21.15.94:0/3966146734 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79600c1120 con 0x56370602cfc0 2024-02-17T12:07:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.960+0000 7f797a9af740 10 client.27490 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.960+0000 7f797a9af740 1 -- 172.21.15.94:0/3966146734 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563705f436a0 con 0x56370602cfc0 2024-02-17T12:07:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.960+0000 7f79557fa640 1 -- 172.21.15.94:0/3966146734 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79600edf80 con 0x56370602cfc0 2024-02-17T12:07:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.960+0000 7f79557fa640 1 -- 172.21.15.94:0/3966146734 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f796009e080 con 0x56370602cfc0 2024-02-17T12:07:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.960+0000 7f797a9af740 10 client.27490 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.960+0000 7f797a9af740 10 client.27490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.960+0000 7f797a9af740 20 client.27490 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.960+0000 7f797a9af740 1 --2- 172.21.15.94:0/3966146734 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56370603d050 0x563706096730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:46.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.960+0000 7f797a9af740 4 client.27490 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.960+0000 7f797a9af740 1 -- 172.21.15.94:0/3966146734 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563705d0e890 con 0x56370603d050 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.961+0000 7f7975b73640 1 --2- 172.21.15.94:0/3966146734 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56370603d050 0x563706096730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.961+0000 7f79557fa640 10 client.27490 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.961+0000 7f7975b73640 1 --2- 172.21.15.94:0/3966146734 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56370603d050 0x563706096730 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e263f912997f044 server_cookie=f221d845cf7820bd in_seq=0 out_seq=0 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.961+0000 7f79557fa640 1 -- 172.21.15.94:0/3966146734 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563705d0e890 con 0x56370603d050 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.961+0000 7f79557fa640 10 client.27490 handle_command_reply: tid=0 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.961+0000 7f797a9af740 1 client.27490 shutdown 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.961+0000 7f797a9af740 1 -- 172.21.15.94:0/3966146734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 msgr2=0x563706030930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.961+0000 7f797a9af740 1 --2- 172.21.15.94:0/3966146734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x563706030930 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f796002c170 tx=0x7f79600b9b80 comp rx=0 tx=0).stop 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.962+0000 7f797a9af740 1 -- 172.21.15.94:0/3966146734 shutdown_connections 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.962+0000 7f797a9af740 1 --2- 172.21.15.94:0/3966146734 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56370603d050 0x563706096730 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.962+0000 7f797a9af740 1 --2- 172.21.15.94:0/3966146734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370602cfc0 0x563706030930 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.962+0000 7f797a9af740 1 --2- 172.21.15.94:0/3966146734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56370601b900 0x5637060303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.962+0000 7f797a9af740 1 --2- 172.21.15.94:0/3966146734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56370601afe0 0x563706034070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.962+0000 7f797a9af740 1 -- 172.21.15.94:0/3966146734 >> 172.21.15.94:0/3966146734 conn(0x563706015990 msgr2=0x5637060176a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.962+0000 7f79557fa640 0 client.27490 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.962+0000 7f797a9af740 1 -- 172.21.15.94:0/3966146734 shutdown_connections 2024-02-17T12:07:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.962+0000 7f797a9af740 1 -- 172.21.15.94:0/3966146734 wait complete. 2024-02-17T12:07:46.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.962+0000 7f797a9af740 20 client.27490 trim_cache size 0 max 16384 2024-02-17T12:07:46.966 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:07:46.967 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:07:46.967 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:07:46.967 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:07:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.963+0000 7f7977dd7640 1 -- 172.21.15.94:0/1797793470 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f79580a2380 msgr2=0x7f79580a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.963+0000 7f7977dd7640 1 --2- 172.21.15.94:0/1797793470 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f79580a2380 0x7f79580a4840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7970103980 tx=0x7f7964012810 comp rx=0 tx=0).stop 2024-02-17T12:07:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.963+0000 7f7977dd7640 1 -- 172.21.15.94:0/1797793470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f797019bde0 msgr2=0x7f797019c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.963+0000 7f7977dd7640 1 --2- 172.21.15.94:0/1797793470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f797019bde0 0x7f797019c290 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7960009850 tx=0x7f7960004270 comp rx=0 tx=0).stop 2024-02-17T12:07:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.963+0000 7f7977dd7640 1 -- 172.21.15.94:0/1797793470 shutdown_connections 2024-02-17T12:07:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.963+0000 7f7977dd7640 1 --2- 172.21.15.94:0/1797793470 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f79580a2380 0x7f79580a4840 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.963+0000 7f7977dd7640 1 --2- 172.21.15.94:0/1797793470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f797019bde0 0x7f797019c290 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.963+0000 7f7977dd7640 1 --2- 172.21.15.94:0/1797793470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79701064e0 0x7f7970197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.963+0000 7f7977dd7640 1 --2- 172.21.15.94:0/1797793470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f79700ff8d0 0x7f7970197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:46.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.963+0000 7f7977dd7640 1 -- 172.21.15.94:0/1797793470 >> 172.21.15.94:0/1797793470 conn(0x7f79700765b0 msgr2=0x7f7970104170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.963+0000 7f7977dd7640 1 -- 172.21.15.94:0/1797793470 shutdown_connections 2024-02-17T12:07:46.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:46.963+0000 7f7977dd7640 1 -- 172.21.15.94:0/1797793470 wait complete. 2024-02-17T12:07:46.980 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:07:46.981 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:07:47.045 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:47 smithi094 ceph-mon[32499]: pgmap v299: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 1.57k op/s 2024-02-17T12:07:47.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.142+0000 7fd0ec94b640 1 -- 172.21.15.94:0/1408443561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e8102d10 msgr2=0x7fd0e810b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.142+0000 7fd0ec94b640 1 --2- 172.21.15.94:0/1408443561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e8102d10 0x7fd0e810b1f0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009900 tx=0x7fd0dc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:07:47.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.142+0000 7fd0ec94b640 1 -- 172.21.15.94:0/1408443561 shutdown_connections 2024-02-17T12:07:47.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.142+0000 7fd0ec94b640 1 --2- 172.21.15.94:0/1408443561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0e810b7c0 0x7fd0e810dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.142+0000 7fd0ec94b640 1 --2- 172.21.15.94:0/1408443561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e8102d10 0x7fd0e810b1f0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.142+0000 7fd0ec94b640 1 --2- 172.21.15.94:0/1408443561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e81023f0 0x7fd0e81027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.142+0000 7fd0ec94b640 1 -- 172.21.15.94:0/1408443561 >> 172.21.15.94:0/1408443561 conn(0x7fd0e8077180 msgr2=0x7fd0e8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.142+0000 7fd0ec94b640 1 -- 172.21.15.94:0/1408443561 shutdown_connections 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.142+0000 7fd0ec94b640 1 -- 172.21.15.94:0/1408443561 wait complete. 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.142+0000 7fd0ec94b640 1 Processor -- start 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0ec94b640 1 -- start start 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0ec94b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e81023f0 0x7fd0e819d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0ec94b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e8102d10 0x7fd0e819d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0ec94b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0e810b7c0 0x7fd0e8197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0ec94b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e8197c60 con 0x7fd0e8102d10 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0ec94b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e8197dd0 con 0x7fd0e81023f0 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0ec94b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e8197f90 con 0x7fd0e810b7c0 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0e5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e8102d10 0x7fd0e819d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0e5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e8102d10 0x7fd0e819d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39788/0 (socket says 172.21.15.94:39788) 2024-02-17T12:07:47.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0e5d9b640 1 -- 172.21.15.94:0/2414738904 learned_addr learned my addr 172.21.15.94:0/2414738904 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0e659c640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e81023f0 0x7fd0e819d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0e6d9d640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0e810b7c0 0x7fd0e8197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0e659c640 1 -- 172.21.15.94:0/2414738904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0e810b7c0 msgr2=0x7fd0e8197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0e659c640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0e810b7c0 0x7fd0e8197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0e659c640 1 -- 172.21.15.94:0/2414738904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e8102d10 msgr2=0x7fd0e819d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0e659c640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e8102d10 0x7fd0e819d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0e659c640 1 -- 172.21.15.94:0/2414738904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0dc009560 con 0x7fd0e81023f0 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0e5d9b640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e8102d10 0x7fd0e819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.143+0000 7fd0e659c640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e81023f0 0x7fd0e819d460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800e9f0 tx=0x7fd0d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.144+0000 7fd0d77fe640 1 -- 172.21.15.94:0/2414738904 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d800ccc0 con 0x7fd0e81023f0 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.144+0000 7fd0d77fe640 1 -- 172.21.15.94:0/2414738904 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd0d800ce20 con 0x7fd0e81023f0 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.144+0000 7fd0ec94b640 1 -- 172.21.15.94:0/2414738904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e8198270 con 0x7fd0e81023f0 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.144+0000 7fd0ec94b640 1 -- 172.21.15.94:0/2414738904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e81a4500 con 0x7fd0e81023f0 2024-02-17T12:07:47.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.144+0000 7fd0d77fe640 1 -- 172.21.15.94:0/2414738904 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8018920 con 0x7fd0e81023f0 2024-02-17T12:07:47.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.145+0000 7fd0ec94b640 1 -- 172.21.15.94:0/2414738904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c8005290 con 0x7fd0e81023f0 2024-02-17T12:07:47.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.146+0000 7fd0d77fe640 1 -- 172.21.15.94:0/2414738904 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd0d8010430 con 0x7fd0e81023f0 2024-02-17T12:07:47.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.147+0000 7fd0d77fe640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd0d00a2330 0x7fd0d00a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.147+0000 7fd0d77fe640 1 -- 172.21.15.94:0/2414738904 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd0d8014070 con 0x7fd0e81023f0 2024-02-17T12:07:47.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.147+0000 7fd0e5d9b640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd0d00a2330 0x7fd0d00a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.148+0000 7fd0e5d9b640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd0d00a2330 0x7fd0d00a47f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8198750 tx=0x7fd0dc005a20 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.151+0000 7fd0d77fe640 1 -- 172.21.15.94:0/2414738904 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd0d8052d20 con 0x7fd0e81023f0 2024-02-17T12:07:47.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:47 smithi189 ceph-mon[36588]: pgmap v299: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 1.57k op/s 2024-02-17T12:07:47.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:47 smithi125 ceph-mon[36819]: pgmap v299: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 1.57k op/s 2024-02-17T12:07:47.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.502+0000 7fd0ec94b640 1 -- 172.21.15.94:0/2414738904 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0c8005810 con 0x7fd0e81023f0 2024-02-17T12:07:47.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.503+0000 7fd0d77fe640 1 -- 172.21.15.94:0/2414738904 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7fd0d809ab30 con 0x7fd0e81023f0 2024-02-17T12:07:47.506 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:07:47.506 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:07:47.506 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:07:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.507+0000 7fd0ec94b640 1 -- 172.21.15.94:0/2414738904 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd0d00a2330 msgr2=0x7fd0d00a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.507+0000 7fd0ec94b640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd0d00a2330 0x7fd0d00a47f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8198750 tx=0x7fd0dc005a20 comp rx=0 tx=0).stop 2024-02-17T12:07:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.507+0000 7fd0ec94b640 1 -- 172.21.15.94:0/2414738904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e81023f0 msgr2=0x7fd0e819d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.507+0000 7fd0ec94b640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e81023f0 0x7fd0e819d460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800e9f0 tx=0x7fd0d800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.507+0000 7fd0ec94b640 1 -- 172.21.15.94:0/2414738904 shutdown_connections 2024-02-17T12:07:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.507+0000 7fd0ec94b640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd0d00a2330 0x7fd0d00a47f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.507+0000 7fd0ec94b640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd0e810b7c0 0x7fd0e8197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.507+0000 7fd0ec94b640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd0e8102d10 0x7fd0e819d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.507+0000 7fd0ec94b640 1 --2- 172.21.15.94:0/2414738904 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd0e81023f0 0x7fd0e819d460 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.507+0000 7fd0ec94b640 1 -- 172.21.15.94:0/2414738904 >> 172.21.15.94:0/2414738904 conn(0x7fd0e8077180 msgr2=0x7fd0e80fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.507+0000 7fd0ec94b640 1 -- 172.21.15.94:0/2414738904 shutdown_connections 2024-02-17T12:07:47.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.507+0000 7fd0ec94b640 1 -- 172.21.15.94:0/2414738904 wait complete. 2024-02-17T12:07:47.522 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:07:47.522 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:07:47.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.680+0000 7f39db7fe640 1 -- 172.21.15.94:0/981433902 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc032340 con 0x7f39dc10f9a0 2024-02-17T12:07:47.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.680+0000 7f39e30d4640 1 -- 172.21.15.94:0/981433902 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39dc10f9a0 msgr2=0x7f39dc10d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.680+0000 7f39e30d4640 1 --2- 172.21.15.94:0/981433902 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39dc10f9a0 0x7f39dc10d490 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009860 tx=0x7f39cc02bb90 comp rx=0 tx=0).stop 2024-02-17T12:07:47.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.681+0000 7f39e30d4640 1 -- 172.21.15.94:0/981433902 shutdown_connections 2024-02-17T12:07:47.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.681+0000 7f39e30d4640 1 --2- 172.21.15.94:0/981433902 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39dc11a960 0x7f39dc11cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.681+0000 7f39e30d4640 1 --2- 172.21.15.94:0/981433902 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39dc10d9d0 0x7f39dc10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.681+0000 7f39e30d4640 1 --2- 172.21.15.94:0/981433902 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39dc10f9a0 0x7f39dc10d490 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.681+0000 7f39e30d4640 1 -- 172.21.15.94:0/981433902 >> 172.21.15.94:0/981433902 conn(0x7f39dc06c260 msgr2=0x7f39dc06c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.682+0000 7f39e30d4640 1 -- 172.21.15.94:0/981433902 shutdown_connections 2024-02-17T12:07:47.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.682+0000 7f39e30d4640 1 -- 172.21.15.94:0/981433902 wait complete. 2024-02-17T12:07:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.683+0000 7f39e30d4640 1 Processor -- start 2024-02-17T12:07:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.683+0000 7f39e30d4640 1 -- start start 2024-02-17T12:07:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.683+0000 7f39e30d4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39dc10d9d0 0x7f39dc1a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.683+0000 7f39e30d4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39dc10f9a0 0x7f39dc1a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.683+0000 7f39e30d4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39dc11a960 0x7f39dc1aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.683+0000 7f39e30d4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc1ab4c0 con 0x7f39dc11a960 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.683+0000 7f39e30d4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc1ab630 con 0x7f39dc10f9a0 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.683+0000 7f39e30d4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc1ab7f0 con 0x7f39dc10d9d0 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.683+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39dc10f9a0 0x7f39dc1a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.683+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39dc10f9a0 0x7f39dc1a6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:58490/0 (socket says 172.21.15.94:58490) 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.683+0000 7f39e1671640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39dc11a960 0x7f39dc1aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.683+0000 7f39dbfff640 1 -- 172.21.15.94:0/2481125639 learned_addr learned my addr 172.21.15.94:0/2481125639 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.683+0000 7f39e0e70640 1 --2- 172.21.15.94:0/2481125639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39dc10d9d0 0x7f39dc1a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.684+0000 7f39dbfff640 1 -- 172.21.15.94:0/2481125639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39dc10d9d0 msgr2=0x7f39dc1a6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.684+0000 7f39dbfff640 1 --2- 172.21.15.94:0/2481125639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39dc10d9d0 0x7f39dc1a6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.684+0000 7f39dbfff640 1 -- 172.21.15.94:0/2481125639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39dc11a960 msgr2=0x7f39dc1aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.684+0000 7f39dbfff640 1 --2- 172.21.15.94:0/2481125639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39dc11a960 0x7f39dc1aaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.684+0000 7f39dbfff640 1 -- 172.21.15.94:0/2481125639 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc009490 con 0x7f39dc10f9a0 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.684+0000 7f39dbfff640 1 --2- 172.21.15.94:0/2481125639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39dc10f9a0 0x7f39dc1a6b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f39d400cce0 tx=0x7f39d400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.684+0000 7f39d9ffb640 1 -- 172.21.15.94:0/2481125639 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d4004b20 con 0x7f39dc10f9a0 2024-02-17T12:07:47.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.684+0000 7f39e30d4640 1 -- 172.21.15.94:0/2481125639 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39dc1abad0 con 0x7f39dc10f9a0 2024-02-17T12:07:47.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.684+0000 7f39d9ffb640 1 -- 172.21.15.94:0/2481125639 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39d4004c80 con 0x7f39dc10f9a0 2024-02-17T12:07:47.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.684+0000 7f39d9ffb640 1 -- 172.21.15.94:0/2481125639 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d400f780 con 0x7f39dc10f9a0 2024-02-17T12:07:47.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.684+0000 7f39e30d4640 1 -- 172.21.15.94:0/2481125639 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39dc1b3350 con 0x7f39dc10f9a0 2024-02-17T12:07:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.686+0000 7f39d9ffb640 1 -- 172.21.15.94:0/2481125639 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f39d400f8e0 con 0x7f39dc10f9a0 2024-02-17T12:07:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.687+0000 7f39d9ffb640 1 --2- 172.21.15.94:0/2481125639 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f39c80a2270 0x7f39c80a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.687+0000 7f39e0e70640 1 --2- 172.21.15.94:0/2481125639 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f39c80a2270 0x7f39c80a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.687+0000 7f39d9ffb640 1 -- 172.21.15.94:0/2481125639 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f39d40ce0f0 con 0x7f39dc10f9a0 2024-02-17T12:07:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.687+0000 7f39e5d84740 1 Processor -- start 2024-02-17T12:07:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.687+0000 7f39e5d84740 1 -- start start 2024-02-17T12:07:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e0e70640 1 --2- 172.21.15.94:0/2481125639 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f39c80a2270 0x7f39c80a4730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39cc02c1a0 tx=0x7f39cc000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e5d84740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9250 0x55e7f99d9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e5d84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99d9b70 0x55e7f99e2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e5d84740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99e2750 0x55e7f99e4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e5d84740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f9994960 con 0x55e7f99e2750 2024-02-17T12:07:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e5d84740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f9948310 con 0x55e7f99d9250 2024-02-17T12:07:47.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e5d84740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f9945840 con 0x55e7f99d9b70 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e1671640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99e2750 0x55e7f99e4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e0e70640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9250 0x55e7f99d9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e1671640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99e2750 0x55e7f99e4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39800/0 (socket says 172.21.15.94:39800) 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e1671640 1 -- 172.21.15.94:0/3621391932 learned_addr learned my addr 172.21.15.94:0/3621391932 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e0e70640 1 -- 172.21.15.94:0/3621391932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99d9b70 msgr2=0x55e7f99e2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e0e70640 1 --2- 172.21.15.94:0/3621391932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99d9b70 0x55e7f99e2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e0e70640 1 -- 172.21.15.94:0/3621391932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99e2750 msgr2=0x55e7f99e4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e0e70640 1 --2- 172.21.15.94:0/3621391932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99e2750 0x55e7f99e4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e0e70640 1 -- 172.21.15.94:0/3621391932 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7f9903060 con 0x55e7f99d9250 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e1671640 1 --2- 172.21.15.94:0/3621391932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99e2750 0x55e7f99e4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39e0e70640 1 --2- 172.21.15.94:0/3621391932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9250 0x55e7f99d9630 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0057c0 tx=0x7f39cc03f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=968693bce082d194 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.688+0000 7f39d2ffd640 1 -- 172.21.15.94:0/3621391932 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc034640 con 0x55e7f99d9250 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.689+0000 7f39d2ffd640 1 -- 172.21.15.94:0/3621391932 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39cc02ca10 con 0x55e7f99d9250 2024-02-17T12:07:47.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.689+0000 7f39d2ffd640 1 -- 172.21.15.94:0/3621391932 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc003b20 con 0x55e7f99d9250 2024-02-17T12:07:47.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.689+0000 7f39e5d84740 1 -- 172.21.15.94:0/3621391932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9250 msgr2=0x55e7f99d9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.689+0000 7f39e5d84740 1 --2- 172.21.15.94:0/3621391932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9250 0x55e7f99d9630 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0057c0 tx=0x7f39cc03f000 comp rx=0 tx=0).stop 2024-02-17T12:07:47.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.689+0000 7f39e5d84740 1 -- 172.21.15.94:0/3621391932 shutdown_connections 2024-02-17T12:07:47.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.689+0000 7f39e5d84740 1 --2- 172.21.15.94:0/3621391932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99e2750 0x55e7f99e4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.689+0000 7f39e5d84740 1 --2- 172.21.15.94:0/3621391932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99d9b70 0x55e7f99e2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.689+0000 7f39e5d84740 1 --2- 172.21.15.94:0/3621391932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9250 0x55e7f99d9630 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.689+0000 7f39e5d84740 1 -- 172.21.15.94:0/3621391932 >> 172.21.15.94:0/3621391932 conn(0x55e7f99d3980 msgr2=0x55e7f99d3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.689+0000 7f39e5d84740 1 -- 172.21.15.94:0/3621391932 shutdown_connections 2024-02-17T12:07:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.689+0000 7f39e5d84740 1 -- 172.21.15.94:0/3621391932 wait complete. 2024-02-17T12:07:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.690+0000 7f39e5d84740 1 -- start start 2024-02-17T12:07:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.690+0000 7f39e5d84740 1 Processor -- start 2024-02-17T12:07:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.691+0000 7f39e5d84740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99d9250 0x55e7f99df820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.691+0000 7f39e5d84740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9b70 0x55e7f99dfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.691+0000 7f39e5d84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99e2750 0x55e7f99e02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.691+0000 7f39e0e70640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99d9250 0x55e7f99df820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.691+0000 7f39e0e70640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99d9250 0x55e7f99df820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39810/0 (socket says 172.21.15.94:39810) 2024-02-17T12:07:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.691+0000 7f39e0e70640 1 -- 172.21.15.94:0/2757619422 learned_addr learned my addr 172.21.15.94:0/2757619422 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.691+0000 7f39e5d84740 1 -- 172.21.15.94:0/2757619422 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f9941020 con 0x55e7f99d9250 2024-02-17T12:07:47.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.691+0000 7f39e5d84740 1 -- 172.21.15.94:0/2757619422 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f9614430 con 0x55e7f99d9b70 2024-02-17T12:07:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.691+0000 7f39e5d84740 1 -- 172.21.15.94:0/2757619422 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f95d0e00 con 0x55e7f99e2750 2024-02-17T12:07:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.691+0000 7f39e1671640 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99e2750 0x55e7f99e02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.692+0000 7f39e5d84740 10 client.0 _ll_register_callbacks cb 0x55e7f98e8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:47.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.692+0000 7f39dbfff640 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9b70 0x55e7f99dfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.692+0000 7f39e0e70640 1 -- 172.21.15.94:0/2757619422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99e2750 msgr2=0x55e7f99e02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.692+0000 7f39e0e70640 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99e2750 0x55e7f99e02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.692+0000 7f39e0e70640 1 -- 172.21.15.94:0/2757619422 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9b70 msgr2=0x55e7f99dfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.692+0000 7f39e0e70640 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9b70 0x55e7f99dfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.692+0000 7f39e0e70640 1 -- 172.21.15.94:0/2757619422 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39dc11d680 con 0x55e7f99d9250 2024-02-17T12:07:47.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.692+0000 7f39dbfff640 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9b70 0x55e7f99dfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:07:47.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.692+0000 7f39e0e70640 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99d9250 0x55e7f99df820 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f39cc02bbd0 tx=0x7f39cc005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986ddb48c6fa2600 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.692+0000 7f39bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:47.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.693+0000 7f39bbfff640 1 -- 172.21.15.94:0/2757619422 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc003aa0 con 0x55e7f99d9250 2024-02-17T12:07:47.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.693+0000 7f39bbfff640 1 -- 172.21.15.94:0/2757619422 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f39cc046050 con 0x55e7f99d9250 2024-02-17T12:07:47.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.693+0000 7f39e5d84740 1 -- 172.21.15.94:0/2757619422 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7f96407e0 con 0x55e7f99d9250 2024-02-17T12:07:47.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.693+0000 7f39bbfff640 1 -- 172.21.15.94:0/2757619422 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39cc02cb80 con 0x55e7f99d9250 2024-02-17T12:07:47.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.693+0000 7f39e5d84740 10 client.18024 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:47.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.693+0000 7f39e5d84740 1 -- 172.21.15.94:0/2757619422 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e7f993f6a0 con 0x55e7f99d9250 2024-02-17T12:07:47.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.693+0000 7f39bbfff640 1 -- 172.21.15.94:0/2757619422 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e7f96407e0 con 0x55e7f99d9250 2024-02-17T12:07:47.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.693+0000 7f39bbfff640 1 -- 172.21.15.94:0/2757619422 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39cc03f480 con 0x55e7f99d9250 2024-02-17T12:07:47.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.693+0000 7f39e5d84740 10 client.18024 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:47.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.693+0000 7f39e5d84740 10 client.18024 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:07:47.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.693+0000 7f39e5d84740 20 client.18024 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:47.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.694+0000 7f39e5d84740 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55e7f99f27f0 0x55e7f99f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.694+0000 7f39e5d84740 4 client.18024 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:47.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.694+0000 7f39e5d84740 1 -- 172.21.15.94:0/2757619422 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7f967d720 con 0x55e7f99f27f0 2024-02-17T12:07:47.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.694+0000 7f39dbfff640 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55e7f99f27f0 0x55e7f99f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.694+0000 7f39dbfff640 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55e7f99f27f0 0x55e7f99f4be0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73d4b628d4b561e9 server_cookie=878a5373f6b2d21b in_seq=0 out_seq=0 2024-02-17T12:07:47.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.694+0000 7f39bbfff640 10 client.18024 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:07:47.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.695+0000 7f39bbfff640 1 -- 172.21.15.94:0/2757619422 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e7f967d720 con 0x55e7f99f27f0 2024-02-17T12:07:47.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.695+0000 7f39bbfff640 10 client.18024 handle_command_reply: tid=0 2024-02-17T12:07:47.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.696+0000 7f39e5d84740 1 client.18024 shutdown 2024-02-17T12:07:47.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.696+0000 7f39e5d84740 1 -- 172.21.15.94:0/2757619422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99d9250 msgr2=0x55e7f99df820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.696+0000 7f39e5d84740 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99d9250 0x55e7f99df820 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f39cc02bbd0 tx=0x7f39cc005fd0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.696+0000 7f39e5d84740 1 -- 172.21.15.94:0/2757619422 shutdown_connections 2024-02-17T12:07:47.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.696+0000 7f39e5d84740 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99e2750 0x55e7f99e02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.696+0000 7f39e5d84740 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55e7f99f27f0 0x55e7f99f4be0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.697+0000 7f39e5d84740 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9b70 0x55e7f99dfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.697+0000 7f39bbfff640 0 client.18024 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:07:47.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.697+0000 7f39e5d84740 1 --2- 172.21.15.94:0/2757619422 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99d9250 0x55e7f99df820 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.697+0000 7f39e5d84740 1 -- 172.21.15.94:0/2757619422 >> 172.21.15.94:0/2757619422 conn(0x55e7f99d3980 msgr2=0x55e7f99d56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.697+0000 7f39e5d84740 1 -- 172.21.15.94:0/2757619422 shutdown_connections 2024-02-17T12:07:47.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.697+0000 7f39e5d84740 1 -- 172.21.15.94:0/2757619422 wait complete. 2024-02-17T12:07:47.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.697+0000 7f39e5d84740 20 client.18024 trim_cache size 0 max 16384 2024-02-17T12:07:47.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.714+0000 7f39e5d84740 1 Processor -- start 2024-02-17T12:07:47.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.714+0000 7f39e5d84740 1 -- start start 2024-02-17T12:07:47.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.715+0000 7f39e5d84740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99d9250 0x55e7f99e1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.715+0000 7f39e5d84740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9b70 0x55e7f99cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.715+0000 7f39e5d84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99e2750 0x55e7f99dab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.715+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9b70 0x55e7f99cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.715+0000 7f39dbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9b70 0x55e7f99cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:58512/0 (socket says 172.21.15.94:58512) 2024-02-17T12:07:47.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.715+0000 7f39dbfff640 1 -- 172.21.15.94:0/1535404496 learned_addr learned my addr 172.21.15.94:0/1535404496 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:47.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.715+0000 7f39e5d84740 1 -- 172.21.15.94:0/1535404496 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f98c6130 con 0x55e7f99d9250 2024-02-17T12:07:47.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.715+0000 7f39e5d84740 1 -- 172.21.15.94:0/1535404496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f99e0c80 con 0x55e7f99d9b70 2024-02-17T12:07:47.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.715+0000 7f39e5d84740 1 -- 172.21.15.94:0/1535404496 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f99cf320 con 0x55e7f99e2750 2024-02-17T12:07:47.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.716+0000 7f39e1671640 1 --2- 172.21.15.94:0/1535404496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99e2750 0x55e7f99dab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.716+0000 7f39e1671640 1 -- 172.21.15.94:0/1535404496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9b70 msgr2=0x55e7f99cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.716+0000 7f39e1671640 1 --2- 172.21.15.94:0/1535404496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9b70 0x55e7f99cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.716+0000 7f39e1671640 1 -- 172.21.15.94:0/1535404496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99d9250 msgr2=0x55e7f99e1f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:07:47.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.716+0000 7f39e1671640 1 --2- 172.21.15.94:0/1535404496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99d9250 0x55e7f99e1f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.716+0000 7f39e1671640 1 -- 172.21.15.94:0/1535404496 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7f98dbc20 con 0x55e7f99e2750 2024-02-17T12:07:47.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.716+0000 7f39dbfff640 1 --2- 172.21.15.94:0/1535404496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9b70 0x55e7f99cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:07:47.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.717+0000 7f39e1671640 1 --2- 172.21.15.94:0/1535404496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99e2750 0x55e7f99dab30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f39c4004640 tx=0x7f39c400ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a06a9814eaa4e39 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.717+0000 7f39d27fc640 1 -- 172.21.15.94:0/1535404496 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c401f070 con 0x55e7f99e2750 2024-02-17T12:07:47.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.718+0000 7f39d27fc640 1 -- 172.21.15.94:0/1535404496 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39c4017040 con 0x55e7f99e2750 2024-02-17T12:07:47.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.718+0000 7f39d27fc640 1 -- 172.21.15.94:0/1535404496 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4016070 con 0x55e7f99e2750 2024-02-17T12:07:47.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.719+0000 7f39e5d84740 1 -- 172.21.15.94:0/1535404496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99e2750 msgr2=0x55e7f99dab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.719+0000 7f39e5d84740 1 --2- 172.21.15.94:0/1535404496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99e2750 0x55e7f99dab30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f39c4004640 tx=0x7f39c400ba90 comp rx=0 tx=0).stop 2024-02-17T12:07:47.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.719+0000 7f39e5d84740 1 -- 172.21.15.94:0/1535404496 shutdown_connections 2024-02-17T12:07:47.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.719+0000 7f39e5d84740 1 --2- 172.21.15.94:0/1535404496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99e2750 0x55e7f99dab30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.719+0000 7f39e5d84740 1 --2- 172.21.15.94:0/1535404496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9b70 0x55e7f99cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.720+0000 7f39e5d84740 1 --2- 172.21.15.94:0/1535404496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99d9250 0x55e7f99e1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.720+0000 7f39e5d84740 1 -- 172.21.15.94:0/1535404496 >> 172.21.15.94:0/1535404496 conn(0x55e7f99d3980 msgr2=0x55e7f99d9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.721+0000 7f39e5d84740 1 -- 172.21.15.94:0/1535404496 shutdown_connections 2024-02-17T12:07:47.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.721+0000 7f39e5d84740 1 -- 172.21.15.94:0/1535404496 wait complete. 2024-02-17T12:07:47.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.722+0000 7f39e5d84740 1 -- start start 2024-02-17T12:07:47.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.722+0000 7f39e5d84740 1 Processor -- start 2024-02-17T12:07:47.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.723+0000 7f39e5d84740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9250 0x55e7f99e98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.723+0000 7f39e5d84740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99d9b70 0x55e7f99e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.723+0000 7f39e5d84740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99e2750 0x55e7f99e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.724+0000 7f39e0e70640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9250 0x55e7f99e98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.724+0000 7f39e0e70640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9250 0x55e7f99e98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:58522/0 (socket says 172.21.15.94:58522) 2024-02-17T12:07:47.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.724+0000 7f39e0e70640 1 -- 172.21.15.94:0/786302584 learned_addr learned my addr 172.21.15.94:0/786302584 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:47.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.724+0000 7f39dbfff640 1 --2- 172.21.15.94:0/786302584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99d9b70 0x55e7f99e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.724+0000 7f39e5d84740 1 -- 172.21.15.94:0/786302584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f99e8850 con 0x55e7f99e2750 2024-02-17T12:07:47.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.724+0000 7f39e5d84740 1 -- 172.21.15.94:0/786302584 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f99e89c0 con 0x55e7f99d9250 2024-02-17T12:07:47.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.725+0000 7f39e5d84740 1 -- 172.21.15.94:0/786302584 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7f9a48c60 con 0x55e7f99d9b70 2024-02-17T12:07:47.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.725+0000 7f39e5d84740 10 client.0 _ll_register_callbacks cb 0x7f39cc0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:47.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.725+0000 7f39e0e70640 1 -- 172.21.15.94:0/786302584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99d9b70 msgr2=0x55e7f99e7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.725+0000 7f39e0e70640 1 --2- 172.21.15.94:0/786302584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99d9b70 0x55e7f99e7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.725+0000 7f39e0e70640 1 -- 172.21.15.94:0/786302584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99e2750 msgr2=0x55e7f99e81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.725+0000 7f39e0e70640 1 --2- 172.21.15.94:0/786302584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99e2750 0x55e7f99e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.725+0000 7f39e0e70640 1 -- 172.21.15.94:0/786302584 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c4004320 con 0x55e7f99d9250 2024-02-17T12:07:47.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.725+0000 7f39e0e70640 1 --2- 172.21.15.94:0/786302584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9250 0x55e7f99e98f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f39cc004650 tx=0x7f39cc0035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bdaa6c3708cec2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.725+0000 7f39bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:07:47.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.726+0000 7f39bbfff640 1 -- 172.21.15.94:0/786302584 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc032360 con 0x55e7f99d9250 2024-02-17T12:07:47.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.726+0000 7f39bbfff640 1 -- 172.21.15.94:0/786302584 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f39cc046040 con 0x55e7f99d9250 2024-02-17T12:07:47.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.726+0000 7f39e5d84740 1 -- 172.21.15.94:0/786302584 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39cc02ca10 con 0x55e7f99d9250 2024-02-17T12:07:47.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.726+0000 7f39bbfff640 1 -- 172.21.15.94:0/786302584 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39cc02cb80 con 0x55e7f99d9250 2024-02-17T12:07:47.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.727+0000 7f39e5d84740 10 client.27521 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:47.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.727+0000 7f39e5d84740 1 -- 172.21.15.94:0/786302584 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e7f99018c0 con 0x55e7f99d9250 2024-02-17T12:07:47.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.729+0000 7f39bbfff640 1 -- 172.21.15.94:0/786302584 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f39cc02ca10 con 0x55e7f99d9250 2024-02-17T12:07:47.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.729+0000 7f39bbfff640 1 -- 172.21.15.94:0/786302584 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f39cc032600 con 0x55e7f99d9250 2024-02-17T12:07:47.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.730+0000 7f39e5d84740 10 client.27521 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:47.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.730+0000 7f39e5d84740 10 client.27521 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:07:47.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.730+0000 7f39e5d84740 20 client.27521 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:47.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.730+0000 7f39e5d84740 1 --2- 172.21.15.94:0/786302584 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55e7f99f27f0 0x55e7f9a4bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.730+0000 7f39e5d84740 4 client.27521 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:07:47.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.730+0000 7f39e5d84740 1 -- 172.21.15.94:0/786302584 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e7f96cc890 con 0x55e7f99f27f0 2024-02-17T12:07:47.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.730+0000 7f39dbfff640 1 --2- 172.21.15.94:0/786302584 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55e7f99f27f0 0x55e7f9a4bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.730+0000 7f39bbfff640 10 client.27521 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:07:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.731+0000 7f39dbfff640 1 --2- 172.21.15.94:0/786302584 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55e7f99f27f0 0x55e7f9a4bfe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1b118c91ce0487d server_cookie=75fe70c8cf453661 in_seq=0 out_seq=0 2024-02-17T12:07:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.731+0000 7f39bbfff640 1 -- 172.21.15.94:0/786302584 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e7f96cc890 con 0x55e7f99f27f0 2024-02-17T12:07:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.731+0000 7f39bbfff640 10 client.27521 handle_command_reply: tid=0 2024-02-17T12:07:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.731+0000 7f39e5d84740 1 client.27521 shutdown 2024-02-17T12:07:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.731+0000 7f39e5d84740 1 -- 172.21.15.94:0/786302584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9250 msgr2=0x55e7f99e98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.731+0000 7f39e5d84740 1 --2- 172.21.15.94:0/786302584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9250 0x55e7f99e98f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f39cc004650 tx=0x7f39cc0035f0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.731+0000 7f39e5d84740 1 -- 172.21.15.94:0/786302584 shutdown_connections 2024-02-17T12:07:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.731+0000 7f39e5d84740 1 --2- 172.21.15.94:0/786302584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e7f99e2750 0x55e7f99e81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.732+0000 7f39e5d84740 1 --2- 172.21.15.94:0/786302584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e7f99d9b70 0x55e7f99e7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.732+0000 7f39e5d84740 1 --2- 172.21.15.94:0/786302584 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55e7f99f27f0 0x55e7f9a4bfe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.732+0000 7f39e5d84740 1 --2- 172.21.15.94:0/786302584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e7f99d9250 0x55e7f99e98f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.732+0000 7f39bbfff640 0 client.27521 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:07:47.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.732+0000 7f39e5d84740 1 -- 172.21.15.94:0/786302584 >> 172.21.15.94:0/786302584 conn(0x55e7f99d3980 msgr2=0x55e7f99cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.732+0000 7f39e5d84740 1 -- 172.21.15.94:0/786302584 shutdown_connections 2024-02-17T12:07:47.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.732+0000 7f39e5d84740 1 -- 172.21.15.94:0/786302584 wait complete. 2024-02-17T12:07:47.736 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.733+0000 7f39e5d84740 20 client.27521 trim_cache size 0 max 16384 2024-02-17T12:07:47.736 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:07:47.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.733+0000 7f39e30d4640 1 -- 172.21.15.94:0/2481125639 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f39c80a2270 msgr2=0x7f39c80a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.733+0000 7f39e30d4640 1 --2- 172.21.15.94:0/2481125639 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f39c80a2270 0x7f39c80a4730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39cc02c1a0 tx=0x7f39cc000bc0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.733+0000 7f39e30d4640 1 -- 172.21.15.94:0/2481125639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39dc10f9a0 msgr2=0x7f39dc1a6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.733+0000 7f39e30d4640 1 --2- 172.21.15.94:0/2481125639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39dc10f9a0 0x7f39dc1a6b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f39d400cce0 tx=0x7f39d400a620 comp rx=0 tx=0).stop 2024-02-17T12:07:47.737 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.733+0000 7f39e30d4640 1 -- 172.21.15.94:0/2481125639 shutdown_connections 2024-02-17T12:07:47.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.734+0000 7f39e30d4640 1 --2- 172.21.15.94:0/2481125639 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f39c80a2270 0x7f39c80a4730 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.734+0000 7f39e30d4640 1 --2- 172.21.15.94:0/2481125639 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f39dc11a960 0x7f39dc1aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.734+0000 7f39e30d4640 1 --2- 172.21.15.94:0/2481125639 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f39dc10f9a0 0x7f39dc1a6b40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.734+0000 7f39e30d4640 1 --2- 172.21.15.94:0/2481125639 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39dc10d9d0 0x7f39dc1a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.734+0000 7f39e30d4640 1 -- 172.21.15.94:0/2481125639 >> 172.21.15.94:0/2481125639 conn(0x7f39dc06c260 msgr2=0x7f39dc071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.734+0000 7f39e30d4640 1 -- 172.21.15.94:0/2481125639 shutdown_connections 2024-02-17T12:07:47.738 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.734+0000 7f39e30d4640 1 -- 172.21.15.94:0/2481125639 wait complete. 2024-02-17T12:07:47.752 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:07:47.752 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:07:47.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.913+0000 7f9f4993a640 1 -- 172.21.15.94:0/2582030317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f44112120 msgr2=0x7f9f44114510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.913+0000 7f9f4993a640 1 --2- 172.21.15.94:0/2582030317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f44112120 0x7f9f44114510 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009ca0 tx=0x7f9f3c02c1c0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.914+0000 7f9f4993a640 1 -- 172.21.15.94:0/2582030317 shutdown_connections 2024-02-17T12:07:47.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.914+0000 7f9f4993a640 1 --2- 172.21.15.94:0/2582030317 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f44112120 0x7f9f44114510 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.914+0000 7f9f4993a640 1 --2- 172.21.15.94:0/2582030317 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f44073e40 0x7f9f44111be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.914+0000 7f9f4993a640 1 --2- 172.21.15.94:0/2582030317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f44073520 0x7f9f44073900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.914+0000 7f9f4993a640 1 -- 172.21.15.94:0/2582030317 >> 172.21.15.94:0/2582030317 conn(0x7f9f440fcc20 msgr2=0x7f9f440ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.915+0000 7f9f4993a640 1 -- 172.21.15.94:0/2582030317 shutdown_connections 2024-02-17T12:07:47.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.916+0000 7f9f4993a640 1 -- 172.21.15.94:0/2582030317 wait complete. 2024-02-17T12:07:47.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.917+0000 7f9f4993a640 1 Processor -- start 2024-02-17T12:07:47.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.917+0000 7f9f4993a640 1 -- start start 2024-02-17T12:07:47.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f4993a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f44073520 0x7f9f44111860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f4993a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f44073e40 0x7f9f4410b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f4993a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f44112120 0x7f9f4410be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f4993a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4410c410 con 0x7f9f44112120 2024-02-17T12:07:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f4993a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4410c580 con 0x7f9f44073e40 2024-02-17T12:07:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f4993a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4410c740 con 0x7f9f44073520 2024-02-17T12:07:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f3bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f44073e40 0x7f9f4410b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f3bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f44073e40 0x7f9f4410b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:58538/0 (socket says 172.21.15.94:58538) 2024-02-17T12:07:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f3bfff640 1 -- 172.21.15.94:0/2754083864 learned_addr learned my addr 172.21.15.94:0/2754083864 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f437fe640 1 --2- 172.21.15.94:0/2754083864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f44112120 0x7f9f4410be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f3bfff640 1 -- 172.21.15.94:0/2754083864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f44073520 msgr2=0x7f9f44111860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f3bfff640 1 --2- 172.21.15.94:0/2754083864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f44073520 0x7f9f44111860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f3bfff640 1 -- 172.21.15.94:0/2754083864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f44112120 msgr2=0x7f9f4410be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f3bfff640 1 --2- 172.21.15.94:0/2754083864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f44112120 0x7f9f4410be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.918+0000 7f9f3bfff640 1 -- 172.21.15.94:0/2754083864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c009900 con 0x7f9f44073e40 2024-02-17T12:07:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.919+0000 7f9f3bfff640 1 --2- 172.21.15.94:0/2754083864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f44073e40 0x7f9f4410b8d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400b450 tx=0x7f9f3400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.919+0000 7f9f48938640 1 -- 172.21.15.94:0/2754083864 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34011830 con 0x7f9f44073e40 2024-02-17T12:07:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.919+0000 7f9f48938640 1 -- 172.21.15.94:0/2754083864 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f34011e70 con 0x7f9f44073e40 2024-02-17T12:07:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.919+0000 7f9f48938640 1 -- 172.21.15.94:0/2754083864 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3400f2e0 con 0x7f9f44073e40 2024-02-17T12:07:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.919+0000 7f9f4993a640 1 -- 172.21.15.94:0/2754083864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f441b2f50 con 0x7f9f44073e40 2024-02-17T12:07:47.922 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.919+0000 7f9f4993a640 1 -- 172.21.15.94:0/2754083864 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f441b3320 con 0x7f9f44073e40 2024-02-17T12:07:47.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.924+0000 7f9f4c5e8740 1 Processor -- start 2024-02-17T12:07:47.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.924+0000 7f9f4c5e8740 1 -- start start 2024-02-17T12:07:47.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.925+0000 7f9f4c5e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c35630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.925+0000 7f9f4c5e8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 0x55ffa5c3e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.925+0000 7f9f4c5e8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c3e750 0x55ffa5c40b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.925+0000 7f9f437fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c35630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.925+0000 7f9f437fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c35630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56028/0 (socket says 172.21.15.94:56028) 2024-02-17T12:07:47.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.925+0000 7f9f48938640 1 -- 172.21.15.94:0/2754083864 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9f34010580 con 0x7f9f44073e40 2024-02-17T12:07:47.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.925+0000 7f9f42ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 0x55ffa5c3e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.925+0000 7f9f42ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 0x55ffa5c3e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:58544/0 (socket says 172.21.15.94:58544) 2024-02-17T12:07:47.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.926+0000 7f9f437fe640 1 -- 172.21.15.94:0/106435 learned_addr learned my addr 172.21.15.94:0/106435 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:47.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.926+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/106435 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa5bf0960 con 0x55ffa5c3e750 2024-02-17T12:07:47.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.926+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/106435 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa5ba4310 con 0x55ffa5c35b70 2024-02-17T12:07:47.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.926+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/106435 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa5ba1840 con 0x55ffa5c35250 2024-02-17T12:07:47.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.926+0000 7f9f3bfff640 1 --2- 172.21.15.94:0/106435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c3e750 0x55ffa5c40b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.926+0000 7f9f437fe640 1 -- 172.21.15.94:0/106435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 msgr2=0x55ffa5c3e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.926+0000 7f9f437fe640 1 --2- 172.21.15.94:0/106435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 0x55ffa5c3e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.926+0000 7f9f437fe640 1 -- 172.21.15.94:0/106435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c3e750 msgr2=0x55ffa5c40b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.926+0000 7f9f437fe640 1 --2- 172.21.15.94:0/106435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c3e750 0x55ffa5c40b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.926+0000 7f9f437fe640 1 -- 172.21.15.94:0/106435 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffa5b5f060 con 0x55ffa5c35250 2024-02-17T12:07:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.926+0000 7f9f48938640 1 --2- 172.21.15.94:0/2754083864 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9f300a2440 0x7f9f300a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.926+0000 7f9f42ffd640 1 --2- 172.21.15.94:0/106435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 0x55ffa5c3e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:07:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.927+0000 7f9f48938640 1 -- 172.21.15.94:0/2754083864 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9f340cdb50 con 0x7f9f44073e40 2024-02-17T12:07:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.927+0000 7f9f42ffd640 1 --2- 172.21.15.94:0/2754083864 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9f300a2440 0x7f9f300a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.927+0000 7f9f437fe640 1 --2- 172.21.15.94:0/106435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c35630 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009ca0 tx=0x7f9f3c034db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=516037abc24e75ca server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.927+0000 7f9f39ffb640 1 -- 172.21.15.94:0/106435 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c032540 con 0x55ffa5c35250 2024-02-17T12:07:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.927+0000 7f9f42ffd640 1 --2- 172.21.15.94:0/2754083864 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9f300a2440 0x7f9f300a4900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ffa5c34d20 tx=0x7f9f2c008040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.927+0000 7f9f39ffb640 1 -- 172.21.15.94:0/106435 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c003970 con 0x55ffa5c35250 2024-02-17T12:07:47.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.928+0000 7f9f39ffb640 1 -- 172.21.15.94:0/106435 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c032cc0 con 0x55ffa5c35250 2024-02-17T12:07:47.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.928+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/106435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 msgr2=0x55ffa5c35630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.928+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/106435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c35630 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009ca0 tx=0x7f9f3c034db0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.928+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/106435 shutdown_connections 2024-02-17T12:07:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.928+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/106435 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c3e750 0x55ffa5c40b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.928+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/106435 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 0x55ffa5c3e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.928+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/106435 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c35630 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.928+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/106435 >> 172.21.15.94:0/106435 conn(0x55ffa5c2f980 msgr2=0x55ffa5c2fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.929+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/106435 shutdown_connections 2024-02-17T12:07:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.929+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/106435 wait complete. 2024-02-17T12:07:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.929+0000 7f9f4c5e8740 1 -- start start 2024-02-17T12:07:47.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.929+0000 7f9f4c5e8740 1 Processor -- start 2024-02-17T12:07:47.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.930+0000 7f9f4c5e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c3b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.930+0000 7f9f4c5e8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 0x55ffa5c3bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.930+0000 7f9f4c5e8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c3e750 0x55ffa5c3c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.930+0000 7f9f4c5e8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa5b9d020 con 0x55ffa5c3e750 2024-02-17T12:07:47.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.930+0000 7f9f4c5e8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa5870430 con 0x55ffa5c35b70 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.930+0000 7f9f3bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 0x55ffa5c3bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.930+0000 7f9f3bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 0x55ffa5c3bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:58554/0 (socket says 172.21.15.94:58554) 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.930+0000 7f9f3bfff640 1 -- 172.21.15.94:0/2259195604 learned_addr learned my addr 172.21.15.94:0/2259195604 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.930+0000 7f9f42ffd640 1 --2- 172.21.15.94:0/2259195604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c3b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.930+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/2259195604 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa582ce00 con 0x55ffa5c35250 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.930+0000 7f9f437fe640 1 --2- 172.21.15.94:0/2259195604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c3e750 0x55ffa5c3c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.931+0000 7f9f4c5e8740 10 client.0 _ll_register_callbacks cb 0x55ffa5b44cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.931+0000 7f9f3bfff640 1 -- 172.21.15.94:0/2259195604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 msgr2=0x55ffa5c3b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.931+0000 7f9f3bfff640 1 --2- 172.21.15.94:0/2259195604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c3b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.931+0000 7f9f3bfff640 1 -- 172.21.15.94:0/2259195604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c3e750 msgr2=0x55ffa5c3c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.931+0000 7f9f3bfff640 1 --2- 172.21.15.94:0/2259195604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c3e750 0x55ffa5c3c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.931+0000 7f9f3bfff640 1 -- 172.21.15.94:0/2259195604 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f44114e40 con 0x55ffa5c35b70 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.931+0000 7f9f3bfff640 1 --2- 172.21.15.94:0/2259195604 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 0x55ffa5c3bce0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9f3409a540 tx=0x7f9f3409a440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=636702cc7236e12c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.931+0000 7f9f22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.931+0000 7f9f22ffd640 1 -- 172.21.15.94:0/2259195604 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3409acb0 con 0x55ffa5c35b70 2024-02-17T12:07:47.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.931+0000 7f9f22ffd640 1 -- 172.21.15.94:0/2259195604 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9f3409a5b0 con 0x55ffa5c35b70 2024-02-17T12:07:47.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.931+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/2259195604 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffa589c7e0 con 0x55ffa5c35b70 2024-02-17T12:07:47.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.932+0000 7f9f22ffd640 1 -- 172.21.15.94:0/2259195604 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f3409e700 con 0x55ffa5c35b70 2024-02-17T12:07:47.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.932+0000 7f9f4c5e8740 10 client.27530 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:47.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.932+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/2259195604 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ffa5b9b6a0 con 0x55ffa5c35b70 2024-02-17T12:07:47.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.932+0000 7f9f22ffd640 1 -- 172.21.15.94:0/2259195604 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f340b2080 con 0x55ffa5c35b70 2024-02-17T12:07:47.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.933+0000 7f9f22ffd640 1 -- 172.21.15.94:0/2259195604 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9f340b8d20 con 0x55ffa5c35b70 2024-02-17T12:07:47.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.933+0000 7f9f4c5e8740 10 client.27530 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:47.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.933+0000 7f9f4c5e8740 10 client.27530 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:07:47.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.933+0000 7f9f4c5e8740 20 client.27530 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:47.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.933+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/2259195604 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffa5c4ea70 0x55ffa5c50ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.933+0000 7f9f4c5e8740 4 client.27530 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:47.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.933+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/2259195604 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffa589c7e0 con 0x55ffa5c4ea70 2024-02-17T12:07:47.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.933+0000 7f9f42ffd640 1 --2- 172.21.15.94:0/2259195604 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffa5c4ea70 0x55ffa5c50ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.934+0000 7f9f42ffd640 1 --2- 172.21.15.94:0/2259195604 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffa5c4ea70 0x55ffa5c50ef0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4441e17971a131a server_cookie=304e1259e9f610dd in_seq=0 out_seq=0 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.934+0000 7f9f22ffd640 10 client.27530 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.934+0000 7f9f22ffd640 1 -- 172.21.15.94:0/2259195604 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffa589c7e0 con 0x55ffa5c4ea70 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.934+0000 7f9f22ffd640 10 client.27530 handle_command_reply: tid=0 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.935+0000 7f9f4c5e8740 1 client.27530 shutdown 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.935+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/2259195604 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 msgr2=0x55ffa5c3bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.935+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/2259195604 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 0x55ffa5c3bce0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9f3409a540 tx=0x7f9f3409a440 comp rx=0 tx=0).stop 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.935+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/2259195604 shutdown_connections 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.935+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/2259195604 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffa5c4ea70 0x55ffa5c50ef0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.935+0000 7f9f22ffd640 0 client.27530 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.935+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/2259195604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c3e750 0x55ffa5c3c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.935+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/2259195604 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c35b70 0x55ffa5c3bce0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.935+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/2259195604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c3b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.935+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/2259195604 >> 172.21.15.94:0/2259195604 conn(0x55ffa5c2f980 msgr2=0x55ffa5c316d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.935+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/2259195604 shutdown_connections 2024-02-17T12:07:47.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.935+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/2259195604 wait complete. 2024-02-17T12:07:47.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.935+0000 7f9f4c5e8740 20 client.27530 trim_cache size 0 max 16384 2024-02-17T12:07:47.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.954+0000 7f9f4c5e8740 1 Processor -- start 2024-02-17T12:07:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.955+0000 7f9f4c5e8740 1 -- start start 2024-02-17T12:07:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.955+0000 7f9f4c5e8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c35250 0x55ffa5c3ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.955+0000 7f9f4c5e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35b70 0x55ffa5c38a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.955+0000 7f9f4c5e8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c3e750 0x55ffa5c38f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.955+0000 7f9f4c5e8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa5b22130 con 0x55ffa5c35250 2024-02-17T12:07:47.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.955+0000 7f9f4c5e8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa589c7e0 con 0x55ffa5c3e750 2024-02-17T12:07:47.958 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.955+0000 7f9f4c5e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa5c34af0 con 0x55ffa5c35b70 2024-02-17T12:07:47.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.956+0000 7f9f42ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c35250 0x55ffa5c3ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.956+0000 7f9f42ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c35250 0x55ffa5c3ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39866/0 (socket says 172.21.15.94:39866) 2024-02-17T12:07:47.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.956+0000 7f9f42ffd640 1 -- 172.21.15.94:0/4188375129 learned_addr learned my addr 172.21.15.94:0/4188375129 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:47.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.956+0000 7f9f437fe640 1 --2- 172.21.15.94:0/4188375129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c3e750 0x55ffa5c38f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.956+0000 7f9f3bfff640 1 --2- 172.21.15.94:0/4188375129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35b70 0x55ffa5c38a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.956+0000 7f9f437fe640 1 -- 172.21.15.94:0/4188375129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35b70 msgr2=0x55ffa5c38a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.956+0000 7f9f437fe640 1 --2- 172.21.15.94:0/4188375129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35b70 0x55ffa5c38a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.956+0000 7f9f437fe640 1 -- 172.21.15.94:0/4188375129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c35250 msgr2=0x55ffa5c3ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.956+0000 7f9f437fe640 1 --2- 172.21.15.94:0/4188375129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c35250 0x55ffa5c3ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.956+0000 7f9f437fe640 1 -- 172.21.15.94:0/4188375129 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffa5b37c20 con 0x55ffa5c3e750 2024-02-17T12:07:47.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.956+0000 7f9f42ffd640 1 --2- 172.21.15.94:0/4188375129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c35250 0x55ffa5c3ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:47.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.956+0000 7f9f437fe640 1 --2- 172.21.15.94:0/4188375129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c3e750 0x55ffa5c38f80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c037040 tx=0x7f9f3c036040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa7b13b2c2aac528 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.957+0000 7f9f397fa640 1 -- 172.21.15.94:0/4188375129 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c009540 con 0x55ffa5c3e750 2024-02-17T12:07:47.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.957+0000 7f9f397fa640 1 -- 172.21.15.94:0/4188375129 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c032340 con 0x55ffa5c3e750 2024-02-17T12:07:47.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.957+0000 7f9f397fa640 1 -- 172.21.15.94:0/4188375129 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c032e30 con 0x55ffa5c3e750 2024-02-17T12:07:47.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.957+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/4188375129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c3e750 msgr2=0x55ffa5c38f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.957+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/4188375129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c3e750 0x55ffa5c38f80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c037040 tx=0x7f9f3c036040 comp rx=0 tx=0).stop 2024-02-17T12:07:47.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.958+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/4188375129 shutdown_connections 2024-02-17T12:07:47.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.958+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/4188375129 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c3e750 0x55ffa5c38f80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.958+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/4188375129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35b70 0x55ffa5c38a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.958+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/4188375129 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c35250 0x55ffa5c3ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.958+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/4188375129 >> 172.21.15.94:0/4188375129 conn(0x55ffa5c2f980 msgr2=0x55ffa5c55890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.958+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/4188375129 shutdown_connections 2024-02-17T12:07:47.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.958+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/4188375129 wait complete. 2024-02-17T12:07:47.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.958+0000 7f9f4c5e8740 1 -- start start 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.958+0000 7f9f4c5e8740 1 Processor -- start 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f4c5e8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c9f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f4c5e8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c35b70 0x55ffa5c9f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f4c5e8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c3e750 0x55ffa5c44940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f4c5e8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa5c44f00 con 0x55ffa5c35b70 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f4c5e8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa5c45070 con 0x55ffa5c3e750 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f4c5e8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffa5c45230 con 0x55ffa5c35250 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f4c5e8740 10 client.0 _ll_register_callbacks cb 0x7f9f3c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f437fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c3e750 0x55ffa5c44940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f437fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c3e750 0x55ffa5c44940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:58564/0 (socket says 172.21.15.94:58564) 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f42ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c9f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f437fe640 1 -- 172.21.15.94:0/3863940634 learned_addr learned my addr 172.21.15.94:0/3863940634 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f3bfff640 1 --2- 172.21.15.94:0/3863940634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c35b70 0x55ffa5c9f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f437fe640 1 -- 172.21.15.94:0/3863940634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 msgr2=0x55ffa5c9f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f437fe640 1 --2- 172.21.15.94:0/3863940634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c9f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f437fe640 1 -- 172.21.15.94:0/3863940634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c35b70 msgr2=0x55ffa5c9f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f437fe640 1 --2- 172.21.15.94:0/3863940634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c35b70 0x55ffa5c9f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f437fe640 1 -- 172.21.15.94:0/3863940634 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2c00e040 con 0x55ffa5c3e750 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f437fe640 1 --2- 172.21.15.94:0/3863940634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c3e750 0x55ffa5c44940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c036010 tx=0x7f9f3c009890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d08bdd065a8b9dc9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.959+0000 7f9f22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f22ffd640 1 -- 172.21.15.94:0/3863940634 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3c032d80 con 0x55ffa5c3e750 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f22ffd640 1 -- 172.21.15.94:0/3863940634 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9f3c034440 con 0x55ffa5c3e750 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/3863940634 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f3c003970 con 0x55ffa5c3e750 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f22ffd640 1 -- 172.21.15.94:0/3863940634 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f3c034600 con 0x55ffa5c3e750 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f4c5e8740 10 client.27536 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/3863940634 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ffa5b5f060 con 0x55ffa5c3e750 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f22ffd640 1 -- 172.21.15.94:0/3863940634 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f3c069080 con 0x55ffa5c3e750 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f22ffd640 1 -- 172.21.15.94:0/3863940634 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9f3c065450 con 0x55ffa5c3e750 2024-02-17T12:07:47.965 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f4c5e8740 10 client.27536 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:47.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f4c5e8740 10 client.27536 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:07:47.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f4c5e8740 20 client.27536 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:47.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/3863940634 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffa5ca6ed0 0x55ffa5ca92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:47.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f4c5e8740 4 client.27536 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:07:47.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/3863940634 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ffa58d9720 con 0x55ffa5ca6ed0 2024-02-17T12:07:47.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.960+0000 7f9f42ffd640 1 --2- 172.21.15.94:0/3863940634 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffa5ca6ed0 0x55ffa5ca92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:47.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.961+0000 7f9f22ffd640 10 client.27536 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:07:47.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.961+0000 7f9f42ffd640 1 --2- 172.21.15.94:0/3863940634 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffa5ca6ed0 0x55ffa5ca92f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ccbb42f401679906 server_cookie=1feb9c194afc8e44 in_seq=0 out_seq=0 2024-02-17T12:07:47.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.962+0000 7f9f22ffd640 1 -- 172.21.15.94:0/3863940634 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ffa58d9720 con 0x55ffa5ca6ed0 2024-02-17T12:07:47.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.962+0000 7f9f22ffd640 10 client.27536 handle_command_reply: tid=0 2024-02-17T12:07:47.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.962+0000 7f9f4c5e8740 1 client.27536 shutdown 2024-02-17T12:07:47.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.963+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/3863940634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c3e750 msgr2=0x55ffa5c44940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.963+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/3863940634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c3e750 0x55ffa5c44940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c036010 tx=0x7f9f3c009890 comp rx=0 tx=0).stop 2024-02-17T12:07:47.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/3863940634 shutdown_connections 2024-02-17T12:07:47.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/3863940634 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55ffa5ca6ed0 0x55ffa5ca92f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/3863940634 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55ffa5c3e750 0x55ffa5c44940 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/3863940634 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55ffa5c35b70 0x55ffa5c9f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4c5e8740 1 --2- 172.21.15.94:0/3863940634 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ffa5c35250 0x55ffa5c9f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/3863940634 >> 172.21.15.94:0/3863940634 conn(0x55ffa5c2f980 msgr2=0x55ffa5c3b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f22ffd640 0 client.27536 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:07:47.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/3863940634 shutdown_connections 2024-02-17T12:07:47.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4c5e8740 1 -- 172.21.15.94:0/3863940634 wait complete. 2024-02-17T12:07:47.968 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4c5e8740 20 client.27536 trim_cache size 0 max 16384 2024-02-17T12:07:47.969 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:07:47.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4993a640 1 -- 172.21.15.94:0/2754083864 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9f300a2440 msgr2=0x7f9f300a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4993a640 1 --2- 172.21.15.94:0/2754083864 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9f300a2440 0x7f9f300a4900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ffa5c34d20 tx=0x7f9f2c008040 comp rx=0 tx=0).stop 2024-02-17T12:07:47.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4993a640 1 -- 172.21.15.94:0/2754083864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f44073e40 msgr2=0x7f9f4410b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:47.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4993a640 1 --2- 172.21.15.94:0/2754083864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f44073e40 0x7f9f4410b8d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400b450 tx=0x7f9f3400b910 comp rx=0 tx=0).stop 2024-02-17T12:07:47.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4993a640 1 -- 172.21.15.94:0/2754083864 shutdown_connections 2024-02-17T12:07:47.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4993a640 1 --2- 172.21.15.94:0/2754083864 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9f300a2440 0x7f9f300a4900 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4993a640 1 --2- 172.21.15.94:0/2754083864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f44112120 0x7f9f4410be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4993a640 1 --2- 172.21.15.94:0/2754083864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f44073e40 0x7f9f4410b8d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4993a640 1 --2- 172.21.15.94:0/2754083864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f44073520 0x7f9f44111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:47.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.964+0000 7f9f4993a640 1 -- 172.21.15.94:0/2754083864 >> 172.21.15.94:0/2754083864 conn(0x7f9f440fcc20 msgr2=0x7f9f440fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:47.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.965+0000 7f9f4993a640 1 -- 172.21.15.94:0/2754083864 shutdown_connections 2024-02-17T12:07:47.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:47.965+0000 7f9f4993a640 1 -- 172.21.15.94:0/2754083864 wait complete. 2024-02-17T12:07:47.981 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:07:47.981 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:07:48.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.133+0000 7f1afb6f1640 1 -- 172.21.15.94:0/3833542274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af4100150 msgr2=0x7f1af410c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:48.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.133+0000 7f1afb6f1640 1 --2- 172.21.15.94:0/3833542274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af4100150 0x7f1af410c660 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1aec00ade0 tx=0x7f1aec02cfd0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.133+0000 7f1afb6f1640 1 -- 172.21.15.94:0/3833542274 shutdown_connections 2024-02-17T12:07:48.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.133+0000 7f1afb6f1640 1 --2- 172.21.15.94:0/3833542274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af4100150 0x7f1af410c660 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.137 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.133+0000 7f1afb6f1640 1 --2- 172.21.15.94:0/3833542274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af40ff790 0x7f1af40ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.133+0000 7f1afb6f1640 1 --2- 172.21.15.94:0/3833542274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1af41057a0 0x7f1af4105b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.133+0000 7f1afb6f1640 1 -- 172.21.15.94:0/3833542274 >> 172.21.15.94:0/3833542274 conn(0x7f1af40765a0 msgr2=0x7f1af40fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:48.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1afb6f1640 1 -- 172.21.15.94:0/3833542274 shutdown_connections 2024-02-17T12:07:48.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1afb6f1640 1 -- 172.21.15.94:0/3833542274 wait complete. 2024-02-17T12:07:48.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1afb6f1640 1 Processor -- start 2024-02-17T12:07:48.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1afb6f1640 1 -- start start 2024-02-17T12:07:48.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1afb6f1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1af40ff790 0x7f1af4106700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1afb6f1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af4100150 0x7f1af4106c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1afb6f1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af41057a0 0x7f1af4107180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1afb6f1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af4108650 con 0x7f1af41057a0 2024-02-17T12:07:48.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1afb6f1640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af41aa110 con 0x7f1af40ff790 2024-02-17T12:07:48.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1afb6f1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af41aa250 con 0x7f1af4100150 2024-02-17T12:07:48.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1af9c8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af41057a0 0x7f1af4107180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1af9c8e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af41057a0 0x7f1af4107180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39894/0 (socket says 172.21.15.94:39894) 2024-02-17T12:07:48.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1af9c8e640 1 -- 172.21.15.94:0/638810399 learned_addr learned my addr 172.21.15.94:0/638810399 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:48.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.134+0000 7f1af8c8c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af4100150 0x7f1af4106c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.135+0000 7f1af948d640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1af40ff790 0x7f1af4106700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.135+0000 7f1af9c8e640 1 -- 172.21.15.94:0/638810399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af4100150 msgr2=0x7f1af4106c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:48.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.135+0000 7f1af9c8e640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af4100150 0x7f1af4106c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.135+0000 7f1af9c8e640 1 -- 172.21.15.94:0/638810399 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1af40ff790 msgr2=0x7f1af4106700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:48.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.135+0000 7f1af9c8e640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1af40ff790 0x7f1af4106700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.135+0000 7f1af9c8e640 1 -- 172.21.15.94:0/638810399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec00aa10 con 0x7f1af41057a0 2024-02-17T12:07:48.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.136+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af4100150 0x7f1af4106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:48.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.136+0000 7f1af9c8e640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af41057a0 0x7f1af4107180 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005e30 tx=0x7f1aec035670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:48.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.137+0000 7f1aea7fc640 1 -- 172.21.15.94:0/638810399 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec02de40 con 0x7f1af41057a0 2024-02-17T12:07:48.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.137+0000 7f1aea7fc640 1 -- 172.21.15.94:0/638810399 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1aec003970 con 0x7f1af41057a0 2024-02-17T12:07:48.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.137+0000 7f1aea7fc640 1 -- 172.21.15.94:0/638810399 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec004370 con 0x7f1af41057a0 2024-02-17T12:07:48.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.138+0000 7f1afb6f1640 1 -- 172.21.15.94:0/638810399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af41aa4d0 con 0x7f1af41057a0 2024-02-17T12:07:48.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.138+0000 7f1afb6f1640 1 -- 172.21.15.94:0/638810399 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af41aa930 con 0x7f1af41057a0 2024-02-17T12:07:48.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.141+0000 7f1aea7fc640 1 -- 172.21.15.94:0/638810399 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1aec03c040 con 0x7f1af41057a0 2024-02-17T12:07:48.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.141+0000 7f1afe39f740 1 Processor -- start 2024-02-17T12:07:48.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.141+0000 7f1afe39f740 1 -- start start 2024-02-17T12:07:48.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1afe39f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164cc250 0x55e4164cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1afe39f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164ccb70 0x55e4164d5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1afe39f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164d5750 0x55e4164d7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1afe39f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e416487960 con 0x55e4164ccb70 2024-02-17T12:07:48.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1afe39f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41643b310 con 0x55e4164d5750 2024-02-17T12:07:48.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1afe39f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e416438840 con 0x55e4164cc250 2024-02-17T12:07:48.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1aea7fc640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1ae00a2400 0x7f1ae00a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1af8c8c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164cc250 0x55e4164cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1af8c8c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164cc250 0x55e4164cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56080/0 (socket says 172.21.15.94:56080) 2024-02-17T12:07:48.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1af8c8c640 1 -- 172.21.15.94:0/1751300283 learned_addr learned my addr 172.21.15.94:0/1751300283 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:48.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1af948d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164ccb70 0x55e4164d5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/1751300283 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164d5750 0x55e4164d7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1af9c8e640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1ae00a2400 0x7f1ae00a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1aea7fc640 1 -- 172.21.15.94:0/638810399 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1aec0f0370 con 0x7f1af41057a0 2024-02-17T12:07:48.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1af8c8c640 1 -- 172.21.15.94:0/1751300283 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164d5750 msgr2=0x55e4164d7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:48.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/1751300283 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164d5750 0x55e4164d7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1af8c8c640 1 -- 172.21.15.94:0/1751300283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164ccb70 msgr2=0x55e4164d5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:48.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/1751300283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164ccb70 0x55e4164d5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.142+0000 7f1af8c8c640 1 -- 172.21.15.94:0/1751300283 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4163f6060 con 0x55e4164cc250 2024-02-17T12:07:48.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.143+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/1751300283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164cc250 0x55e4164cc630 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4011b80 tx=0x7f1ae40096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2770084c3aa1f5b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:48.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.145+0000 7f1af9c8e640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1ae00a2400 0x7f1ae00a48c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1aec009da0 tx=0x7f1aec04a000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:48.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.145+0000 7f1adf7fe640 1 -- 172.21.15.94:0/1751300283 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400fc50 con 0x55e4164cc250 2024-02-17T12:07:48.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.145+0000 7f1adf7fe640 1 -- 172.21.15.94:0/1751300283 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ae400fdb0 con 0x55e4164cc250 2024-02-17T12:07:48.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.145+0000 7f1adf7fe640 1 -- 172.21.15.94:0/1751300283 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40049e0 con 0x55e4164cc250 2024-02-17T12:07:48.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.146+0000 7f1afe39f740 1 -- 172.21.15.94:0/1751300283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164cc250 msgr2=0x55e4164cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:48.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.146+0000 7f1afe39f740 1 --2- 172.21.15.94:0/1751300283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164cc250 0x55e4164cc630 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4011b80 tx=0x7f1ae40096c0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.146+0000 7f1afe39f740 1 -- 172.21.15.94:0/1751300283 shutdown_connections 2024-02-17T12:07:48.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.146+0000 7f1afe39f740 1 --2- 172.21.15.94:0/1751300283 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164d5750 0x55e4164d7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.146+0000 7f1afe39f740 1 --2- 172.21.15.94:0/1751300283 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164ccb70 0x55e4164d5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.146+0000 7f1afe39f740 1 --2- 172.21.15.94:0/1751300283 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164cc250 0x55e4164cc630 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.146+0000 7f1afe39f740 1 -- 172.21.15.94:0/1751300283 >> 172.21.15.94:0/1751300283 conn(0x55e4164c6980 msgr2=0x55e4164c6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:48.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.146+0000 7f1afe39f740 1 -- 172.21.15.94:0/1751300283 shutdown_connections 2024-02-17T12:07:48.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.146+0000 7f1afe39f740 1 -- 172.21.15.94:0/1751300283 wait complete. 2024-02-17T12:07:48.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1afe39f740 1 -- start start 2024-02-17T12:07:48.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1afe39f740 1 Processor -- start 2024-02-17T12:07:48.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1afe39f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164cc250 0x55e4164d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1afe39f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164ccb70 0x55e4164d2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1afe39f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164d5750 0x55e4164d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1afe39f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e416434020 con 0x55e4164cc250 2024-02-17T12:07:48.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1afe39f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e416107430 con 0x55e4164d5750 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1afe39f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4160c3e00 con 0x55e4164ccb70 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1afe39f740 10 client.0 _ll_register_callbacks cb 0x55e4163dbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1af8c8c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164cc250 0x55e4164d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1af8c8c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164cc250 0x55e4164d2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39918/0 (socket says 172.21.15.94:39918) 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1af8c8c640 1 -- 172.21.15.94:0/3127716997 learned_addr learned my addr 172.21.15.94:0/3127716997 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164d5750 0x55e4164d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.147+0000 7f1af948d640 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164ccb70 0x55e4164d2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1af948d640 1 -- 172.21.15.94:0/3127716997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164d5750 msgr2=0x55e4164d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1af948d640 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164d5750 0x55e4164d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1af948d640 1 -- 172.21.15.94:0/3127716997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164cc250 msgr2=0x55e4164d2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1af948d640 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164cc250 0x55e4164d2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1af948d640 1 -- 172.21.15.94:0/3127716997 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae4019040 con 0x55e4164ccb70 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164cc250 0x55e4164d2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:48.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1af948d640 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164ccb70 0x55e4164d2c90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1af00126d0 tx=0x7f1af0012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72852ebfc16d8137 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1adcff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1adcff9640 1 -- 172.21.15.94:0/3127716997 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0017070 con 0x55e4164ccb70 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1afe39f740 1 -- 172.21.15.94:0/3127716997 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4161337e0 con 0x55e4164ccb70 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1adcff9640 1 -- 172.21.15.94:0/3127716997 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1af0004030 con 0x55e4164ccb70 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1adcff9640 1 -- 172.21.15.94:0/3127716997 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1af00041f0 con 0x55e4164ccb70 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1afe39f740 10 client.27532 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.148+0000 7f1afe39f740 1 -- 172.21.15.94:0/3127716997 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e4164326a0 con 0x55e4164ccb70 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.149+0000 7f1adcff9640 1 -- 172.21.15.94:0/3127716997 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e4161337e0 con 0x55e4164ccb70 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.149+0000 7f1adcff9640 1 -- 172.21.15.94:0/3127716997 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1af0022070 con 0x55e4164ccb70 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.149+0000 7f1afe39f740 10 client.27532 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.149+0000 7f1afe39f740 10 client.27532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.149+0000 7f1afe39f740 20 client.27532 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.149+0000 7f1afe39f740 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e4164e5b10 0x55e4164e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.149+0000 7f1afe39f740 4 client.27532 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:48.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.149+0000 7f1afe39f740 1 -- 172.21.15.94:0/3127716997 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e416170720 con 0x55e4164e5b10 2024-02-17T12:07:48.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.150+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e4164e5b10 0x55e4164e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.152+0000 7f1adcff9640 10 client.27532 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.152+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e4164e5b10 0x55e4164e7f30 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b719c8779f0be20c server_cookie=20f067938a7f1039 in_seq=0 out_seq=0 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.152+0000 7f1adcff9640 1 -- 172.21.15.94:0/3127716997 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e416170720 con 0x55e4164e5b10 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.152+0000 7f1adcff9640 10 client.27532 handle_command_reply: tid=0 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.152+0000 7f1afe39f740 1 client.27532 shutdown 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.153+0000 7f1afe39f740 1 -- 172.21.15.94:0/3127716997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164ccb70 msgr2=0x55e4164d2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.153+0000 7f1afe39f740 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164ccb70 0x55e4164d2c90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1af00126d0 tx=0x7f1af0012b90 comp rx=0 tx=0).stop 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.153+0000 7f1afe39f740 1 -- 172.21.15.94:0/3127716997 shutdown_connections 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.153+0000 7f1afe39f740 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e4164e5b10 0x55e4164e7f30 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.153+0000 7f1afe39f740 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164d5750 0x55e4164d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.153+0000 7f1afe39f740 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164ccb70 0x55e4164d2c90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.153+0000 7f1afe39f740 1 --2- 172.21.15.94:0/3127716997 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164cc250 0x55e4164d2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.153+0000 7f1afe39f740 1 -- 172.21.15.94:0/3127716997 >> 172.21.15.94:0/3127716997 conn(0x55e4164c6980 msgr2=0x55e4164c86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.153+0000 7f1adcff9640 0 client.27532 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:48.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.153+0000 7f1afe39f740 1 -- 172.21.15.94:0/3127716997 shutdown_connections 2024-02-17T12:07:48.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.153+0000 7f1afe39f740 1 -- 172.21.15.94:0/3127716997 wait complete. 2024-02-17T12:07:48.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.153+0000 7f1afe39f740 20 client.27532 trim_cache size 0 max 16384 2024-02-17T12:07:48.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.169+0000 7f1afe39f740 1 Processor -- start 2024-02-17T12:07:48.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.169+0000 7f1afe39f740 1 -- start start 2024-02-17T12:07:48.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1afe39f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164cc250 0x55e4164d4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1afe39f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164ccb70 0x55e4164d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1afe39f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164d5750 0x55e4164cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1af8c8c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164cc250 0x55e4164d4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1afe39f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4163b9130 con 0x55e4164ccb70 2024-02-17T12:07:48.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1afe39f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4164cbc20 con 0x55e4164cc250 2024-02-17T12:07:48.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1afe39f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4164c64e0 con 0x55e4164d5750 2024-02-17T12:07:48.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1af8c8c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164cc250 0x55e4164d4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:58612/0 (socket says 172.21.15.94:58612) 2024-02-17T12:07:48.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1af8c8c640 1 -- 172.21.15.94:0/1496825211 learned_addr learned my addr 172.21.15.94:0/1496825211 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:48.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1af948d640 1 --2- 172.21.15.94:0/1496825211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164ccb70 0x55e4164d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1af8c8c640 1 -- 172.21.15.94:0/1496825211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164d5750 msgr2=0x55e4164cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:48.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/1496825211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164d5750 0x55e4164cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1af8c8c640 1 -- 172.21.15.94:0/1496825211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164ccb70 msgr2=0x55e4164d2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:48.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/1496825211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164ccb70 0x55e4164d2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1af8c8c640 1 -- 172.21.15.94:0/1496825211 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4163cec20 con 0x55e4164cc250 2024-02-17T12:07:48.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1af948d640 1 --2- 172.21.15.94:0/1496825211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164ccb70 0x55e4164d2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:07:48.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/1496825211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164cc250 0x55e4164d4e80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400fc20 tx=0x7f1ae40055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a367b723ceb4cf90 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:48.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1adeffd640 1 -- 172.21.15.94:0/1496825211 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4005330 con 0x55e4164cc250 2024-02-17T12:07:48.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.170+0000 7f1adeffd640 1 -- 172.21.15.94:0/1496825211 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e416170720 con 0x55e4164cc250 2024-02-17T12:07:48.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.171+0000 7f1adeffd640 1 -- 172.21.15.94:0/1496825211 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae402e380 con 0x55e4164cc250 2024-02-17T12:07:48.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.172+0000 7f1afe39f740 1 -- 172.21.15.94:0/1496825211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164cc250 msgr2=0x55e4164d4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:48.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.172+0000 7f1afe39f740 1 --2- 172.21.15.94:0/1496825211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164cc250 0x55e4164d4e80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400fc20 tx=0x7f1ae40055f0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.172+0000 7f1afe39f740 1 -- 172.21.15.94:0/1496825211 shutdown_connections 2024-02-17T12:07:48.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.172+0000 7f1afe39f740 1 --2- 172.21.15.94:0/1496825211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164d5750 0x55e4164cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.172+0000 7f1afe39f740 1 --2- 172.21.15.94:0/1496825211 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164ccb70 0x55e4164d2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.172+0000 7f1afe39f740 1 --2- 172.21.15.94:0/1496825211 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164cc250 0x55e4164d4e80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.172+0000 7f1afe39f740 1 -- 172.21.15.94:0/1496825211 >> 172.21.15.94:0/1496825211 conn(0x55e4164c6980 msgr2=0x55e4164c1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:48.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.172+0000 7f1afe39f740 1 -- 172.21.15.94:0/1496825211 shutdown_connections 2024-02-17T12:07:48.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.172+0000 7f1afe39f740 1 -- 172.21.15.94:0/1496825211 wait complete. 2024-02-17T12:07:48.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.173+0000 7f1afe39f740 1 -- start start 2024-02-17T12:07:48.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.173+0000 7f1afe39f740 1 Processor -- start 2024-02-17T12:07:48.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1afe39f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164cc250 0x55e4164dcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1afe39f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164ccb70 0x55e4164dae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1afe39f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164d5750 0x55e4164db470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1af8c8c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164cc250 0x55e4164dcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1af8c8c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164cc250 0x55e4164dcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39938/0 (socket says 172.21.15.94:39938) 2024-02-17T12:07:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1af8c8c640 1 -- 172.21.15.94:0/3916981457 learned_addr learned my addr 172.21.15.94:0/3916981457 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1afe39f740 1 -- 172.21.15.94:0/3916981457 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4164dbb40 con 0x55e4164cc250 2024-02-17T12:07:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1afe39f740 1 -- 172.21.15.94:0/3916981457 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4164dbcb0 con 0x55e4164ccb70 2024-02-17T12:07:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1afe39f740 1 -- 172.21.15.94:0/3916981457 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e41653c070 con 0x55e4164d5750 2024-02-17T12:07:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1af948d640 1 --2- 172.21.15.94:0/3916981457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164ccb70 0x55e4164dae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1afe39f740 10 client.0 _ll_register_callbacks cb 0x7f1ae4009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/3916981457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164d5750 0x55e4164db470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1af948d640 1 -- 172.21.15.94:0/3916981457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164d5750 msgr2=0x55e4164db470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:48.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1af948d640 1 --2- 172.21.15.94:0/3916981457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164d5750 0x55e4164db470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1af948d640 1 -- 172.21.15.94:0/3916981457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164cc250 msgr2=0x55e4164dcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1af948d640 1 --2- 172.21.15.94:0/3916981457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164cc250 0x55e4164dcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.174+0000 7f1af948d640 1 -- 172.21.15.94:0/3916981457 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae4027040 con 0x55e4164ccb70 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.175+0000 7f1af948d640 1 --2- 172.21.15.94:0/3916981457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164ccb70 0x55e4164dae40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1af0000c00 tx=0x7f1af0009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=909739c02e46515b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.175+0000 7f1adcff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.175+0000 7f1adcff9640 1 -- 172.21.15.94:0/3916981457 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0017070 con 0x55e4164ccb70 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.175+0000 7f1adcff9640 1 -- 172.21.15.94:0/3916981457 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1af000fbe0 con 0x55e4164ccb70 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.175+0000 7f1afe39f740 1 -- 172.21.15.94:0/3916981457 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ae400fdb0 con 0x55e4164ccb70 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.175+0000 7f1adcff9640 1 -- 172.21.15.94:0/3916981457 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1af000fda0 con 0x55e4164ccb70 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.175+0000 7f1afe39f740 10 client.27554 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.175+0000 7f1afe39f740 1 -- 172.21.15.94:0/3916981457 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e4163f48c0 con 0x55e4164ccb70 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.175+0000 7f1adcff9640 1 -- 172.21.15.94:0/3916981457 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ae400fdb0 con 0x55e4164ccb70 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.175+0000 7f1adcff9640 1 -- 172.21.15.94:0/3916981457 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1af0022040 con 0x55e4164ccb70 2024-02-17T12:07:48.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.176+0000 7f1afe39f740 10 client.27554 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.176+0000 7f1afe39f740 10 client.27554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.176+0000 7f1afe39f740 20 client.27554 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.176+0000 7f1afe39f740 1 --2- 172.21.15.94:0/3916981457 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e41653d7f0 0x55e41653fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.176+0000 7f1afe39f740 4 client.27554 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.176+0000 7f1afe39f740 1 -- 172.21.15.94:0/3916981457 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e4161bf890 con 0x55e41653d7f0 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.176+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/3916981457 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e41653d7f0 0x55e41653fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.176+0000 7f1adcff9640 10 client.27554 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.176+0000 7f1af8c8c640 1 --2- 172.21.15.94:0/3916981457 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e41653d7f0 0x55e41653fc10 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d09bb008f818e412 server_cookie=856aa4baa32161ce in_seq=0 out_seq=0 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.176+0000 7f1adcff9640 1 -- 172.21.15.94:0/3916981457 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e4161bf890 con 0x55e41653d7f0 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.176+0000 7f1adcff9640 10 client.27554 handle_command_reply: tid=0 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.176+0000 7f1afe39f740 1 client.27554 shutdown 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afe39f740 1 -- 172.21.15.94:0/3916981457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164ccb70 msgr2=0x55e4164dae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afe39f740 1 --2- 172.21.15.94:0/3916981457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164ccb70 0x55e4164dae40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1af0000c00 tx=0x7f1af0009da0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.182 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afe39f740 1 -- 172.21.15.94:0/3916981457 shutdown_connections 2024-02-17T12:07:48.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afe39f740 1 --2- 172.21.15.94:0/3916981457 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e41653d7f0 0x55e41653fc10 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afe39f740 1 --2- 172.21.15.94:0/3916981457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e4164d5750 0x55e4164db470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1adcff9640 0 client.27554 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:48.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afe39f740 1 --2- 172.21.15.94:0/3916981457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e4164ccb70 0x55e4164dae40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afe39f740 1 --2- 172.21.15.94:0/3916981457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e4164cc250 0x55e4164dcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afe39f740 1 -- 172.21.15.94:0/3916981457 >> 172.21.15.94:0/3916981457 conn(0x55e4164c6980 msgr2=0x55e4164d0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:48.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afe39f740 1 -- 172.21.15.94:0/3916981457 shutdown_connections 2024-02-17T12:07:48.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afe39f740 1 -- 172.21.15.94:0/3916981457 wait complete. 2024-02-17T12:07:48.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afe39f740 20 client.27554 trim_cache size 0 max 16384 2024-02-17T12:07:48.183 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:07:48.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afb6f1640 1 -- 172.21.15.94:0/638810399 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1ae00a2400 msgr2=0x7f1ae00a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:48.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afb6f1640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1ae00a2400 0x7f1ae00a48c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1aec009da0 tx=0x7f1aec04a000 comp rx=0 tx=0).stop 2024-02-17T12:07:48.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afb6f1640 1 -- 172.21.15.94:0/638810399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af41057a0 msgr2=0x7f1af4107180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:48.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afb6f1640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af41057a0 0x7f1af4107180 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005e30 tx=0x7f1aec035670 comp rx=0 tx=0).stop 2024-02-17T12:07:48.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afb6f1640 1 -- 172.21.15.94:0/638810399 shutdown_connections 2024-02-17T12:07:48.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afb6f1640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1ae00a2400 0x7f1ae00a48c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.177+0000 7f1afb6f1640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1af41057a0 0x7f1af4107180 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.178+0000 7f1afb6f1640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1af4100150 0x7f1af4106c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.178+0000 7f1afb6f1640 1 --2- 172.21.15.94:0/638810399 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1af40ff790 0x7f1af4106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:48.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.178+0000 7f1afb6f1640 1 -- 172.21.15.94:0/638810399 >> 172.21.15.94:0/638810399 conn(0x7f1af40765a0 msgr2=0x7f1af4102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:48.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.178+0000 7f1afb6f1640 1 -- 172.21.15.94:0/638810399 shutdown_connections 2024-02-17T12:07:48.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:48.178+0000 7f1afb6f1640 1 -- 172.21.15.94:0/638810399 wait complete. 2024-02-17T12:07:48.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:48 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2414738904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:07:48.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:48 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2414738904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:07:48.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:48 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2414738904' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:07:49.197 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:07:49.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:07:49.197 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:07:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.352+0000 7fa43b655640 1 -- 172.21.15.94:0/443634380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa43411a960 msgr2=0x7fa43411cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.352+0000 7fa43b655640 1 --2- 172.21.15.94:0/443634380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa43411a960 0x7fa43411cd50 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fa42c01acb0 tx=0x7fa42c03d170 comp rx=0 tx=0).stop 2024-02-17T12:07:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.357+0000 7fa43b655640 1 -- 172.21.15.94:0/443634380 shutdown_connections 2024-02-17T12:07:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.357+0000 7fa43b655640 1 --2- 172.21.15.94:0/443634380 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa43411a960 0x7fa43411cd50 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.357+0000 7fa43b655640 1 --2- 172.21.15.94:0/443634380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa43410d9d0 0x7fa43410de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.357+0000 7fa43b655640 1 --2- 172.21.15.94:0/443634380 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa43410f9a0 0x7fa43410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.357+0000 7fa43b655640 1 -- 172.21.15.94:0/443634380 >> 172.21.15.94:0/443634380 conn(0x7fa43406c260 msgr2=0x7fa43406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:49.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.362+0000 7fa43b655640 1 -- 172.21.15.94:0/443634380 shutdown_connections 2024-02-17T12:07:49.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.362+0000 7fa43b655640 1 -- 172.21.15.94:0/443634380 wait complete. 2024-02-17T12:07:49.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.363+0000 7fa43b655640 1 Processor -- start 2024-02-17T12:07:49.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.363+0000 7fa43b655640 1 -- start start 2024-02-17T12:07:49.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.363+0000 7fa43b655640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa43410d9d0 0x7fa43411a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.363+0000 7fa43b655640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa43410f9a0 0x7fa434116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.363+0000 7fa43b655640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa43411a960 0x7fa4341166a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.363+0000 7fa43b655640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa434116c70 con 0x7fa43411a960 2024-02-17T12:07:49.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.363+0000 7fa43b655640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa434116de0 con 0x7fa43410f9a0 2024-02-17T12:07:49.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.363+0000 7fa43b655640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa434116fa0 con 0x7fa43410d9d0 2024-02-17T12:07:49.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.364+0000 7fa439bf2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa43411a960 0x7fa4341166a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.364+0000 7fa439bf2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa43411a960 0x7fa4341166a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39964/0 (socket says 172.21.15.94:39964) 2024-02-17T12:07:49.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.364+0000 7fa439bf2640 1 -- 172.21.15.94:0/4039606767 learned_addr learned my addr 172.21.15.94:0/4039606767 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:49.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.364+0000 7fa4393f1640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa43410d9d0 0x7fa43411a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.369 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.367+0000 7fa438bf0640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa43410f9a0 0x7fa434116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.367+0000 7fa439bf2640 1 -- 172.21.15.94:0/4039606767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa43410d9d0 msgr2=0x7fa43411a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.367+0000 7fa439bf2640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa43410d9d0 0x7fa43411a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.367+0000 7fa439bf2640 1 -- 172.21.15.94:0/4039606767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa43410f9a0 msgr2=0x7fa434116160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.367+0000 7fa439bf2640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa43410f9a0 0x7fa434116160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.367+0000 7fa439bf2640 1 -- 172.21.15.94:0/4039606767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c01a910 con 0x7fa43411a960 2024-02-17T12:07:49.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.379+0000 7fa439bf2640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa43411a960 0x7fa4341166a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa42c03d780 tx=0x7fa42c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:49.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.382+0000 7fa42a7fc640 1 -- 172.21.15.94:0/4039606767 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c003ea0 con 0x7fa43411a960 2024-02-17T12:07:49.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.382+0000 7fa42a7fc640 1 -- 172.21.15.94:0/4039606767 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa42c043980 con 0x7fa43411a960 2024-02-17T12:07:49.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.382+0000 7fa42a7fc640 1 -- 172.21.15.94:0/4039606767 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c045c10 con 0x7fa43411a960 2024-02-17T12:07:49.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.384+0000 7fa43b655640 1 -- 172.21.15.94:0/4039606767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4341cc6a0 con 0x7fa43411a960 2024-02-17T12:07:49.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.384+0000 7fa43b655640 1 -- 172.21.15.94:0/4039606767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4341ccb80 con 0x7fa43411a960 2024-02-17T12:07:49.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.387+0000 7fa43e22d740 1 Processor -- start 2024-02-17T12:07:49.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.388+0000 7fa43e22d740 1 -- start start 2024-02-17T12:07:49.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.388+0000 7fa43e22d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55380 0x56517ad55760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.388+0000 7fa43e22d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad55d30 0x56517ad5e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.388+0000 7fa43e22d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad5e8b0 0x56517ad60d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.388+0000 7fa43e22d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517acc3f90 con 0x56517ad55d30 2024-02-17T12:07:49.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.388+0000 7fa43e22d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517acc14c0 con 0x56517ad55380 2024-02-17T12:07:49.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.388+0000 7fa43e22d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517acbcca0 con 0x56517ad5e8b0 2024-02-17T12:07:49.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.390+0000 7fa4393f1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad55d30 0x56517ad5e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.390+0000 7fa4393f1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad55d30 0x56517ad5e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:39968/0 (socket says 172.21.15.94:39968) 2024-02-17T12:07:49.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.390+0000 7fa4393f1640 1 -- 172.21.15.94:0/1998204160 learned_addr learned my addr 172.21.15.94:0/1998204160 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:49.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.391+0000 7fa438bf0640 1 --2- 172.21.15.94:0/1998204160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55380 0x56517ad55760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.391+0000 7fa4393f1640 1 -- 172.21.15.94:0/1998204160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad5e8b0 msgr2=0x56517ad60d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.391+0000 7fa4393f1640 1 --2- 172.21.15.94:0/1998204160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad5e8b0 0x56517ad60d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.391+0000 7fa4393f1640 1 -- 172.21.15.94:0/1998204160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55380 msgr2=0x56517ad55760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.391+0000 7fa4393f1640 1 --2- 172.21.15.94:0/1998204160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55380 0x56517ad55760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.391+0000 7fa4393f1640 1 -- 172.21.15.94:0/1998204160 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56517ac7f0b0 con 0x56517ad55d30 2024-02-17T12:07:49.394 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.392+0000 7fa438bf0640 1 --2- 172.21.15.94:0/1998204160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55380 0x56517ad55760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:07:49.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.392+0000 7fa42a7fc640 1 -- 172.21.15.94:0/4039606767 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa42c045440 con 0x7fa43411a960 2024-02-17T12:07:49.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.392+0000 7fa4393f1640 1 --2- 172.21.15.94:0/1998204160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad55d30 0x56517ad5e2e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa43000b820 tx=0x7fa43000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2103e96e8b95ec5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:49.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.393+0000 7fa42a7fc640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa4200a2440 0x7fa4200a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.393+0000 7fa42a7fc640 1 -- 172.21.15.94:0/4039606767 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa42c0f06a0 con 0x7fa43411a960 2024-02-17T12:07:49.395 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.393+0000 7fa438bf0640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa4200a2440 0x7fa4200a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.394+0000 7fa438bf0640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa4200a2440 0x7fa4200a4900 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa424006fd0 tx=0x7fa424008040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:49.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.394+0000 7fa41f7fe640 1 -- 172.21.15.94:0/1998204160 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43000d950 con 0x56517ad55d30 2024-02-17T12:07:49.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.395+0000 7fa41f7fe640 1 -- 172.21.15.94:0/1998204160 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa43000b460 con 0x56517ad55d30 2024-02-17T12:07:49.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.396+0000 7fa43e22d740 1 -- 172.21.15.94:0/1998204160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad55d30 msgr2=0x56517ad5e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.396+0000 7fa43e22d740 1 --2- 172.21.15.94:0/1998204160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad55d30 0x56517ad5e2e0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fa43000b820 tx=0x7fa43000bce0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.396+0000 7fa43e22d740 1 -- 172.21.15.94:0/1998204160 shutdown_connections 2024-02-17T12:07:49.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.396+0000 7fa43e22d740 1 --2- 172.21.15.94:0/1998204160 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad5e8b0 0x56517ad60d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.396+0000 7fa43e22d740 1 --2- 172.21.15.94:0/1998204160 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad55d30 0x56517ad5e2e0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.396+0000 7fa43e22d740 1 --2- 172.21.15.94:0/1998204160 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55380 0x56517ad55760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.396+0000 7fa43e22d740 1 -- 172.21.15.94:0/1998204160 >> 172.21.15.94:0/1998204160 conn(0x56517ad4f840 msgr2=0x56517ad4fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:49.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.397+0000 7fa43e22d740 1 -- 172.21.15.94:0/1998204160 shutdown_connections 2024-02-17T12:07:49.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.397+0000 7fa43e22d740 1 -- 172.21.15.94:0/1998204160 wait complete. 2024-02-17T12:07:49.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.397+0000 7fa43e22d740 1 -- start start 2024-02-17T12:07:49.400 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.397+0000 7fa43e22d740 1 Processor -- start 2024-02-17T12:07:49.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.398+0000 7fa43e22d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55380 0x56517ad5b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.398+0000 7fa43e22d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55d30 0x56517ad5bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.398+0000 7fa43e22d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad5e8b0 0x56517ad5c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.398+0000 7fa43e22d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ac61f40 con 0x56517ad5e8b0 2024-02-17T12:07:49.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.398+0000 7fa43e22d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517a9904b0 con 0x56517ad55380 2024-02-17T12:07:49.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.398+0000 7fa43e22d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ac42670 con 0x56517ad55d30 2024-02-17T12:07:49.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.398+0000 7fa43e22d740 10 client.0 _ll_register_callbacks cb 0x56517acaff70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:49.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.398+0000 7fa4393f1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55380 0x56517ad5b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.399+0000 7fa4393f1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55380 0x56517ad5b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:58642/0 (socket says 172.21.15.94:58642) 2024-02-17T12:07:49.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.399+0000 7fa4393f1640 1 -- 172.21.15.94:0/2808691409 learned_addr learned my addr 172.21.15.94:0/2808691409 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:49.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.399+0000 7fa438bf0640 1 --2- 172.21.15.94:0/2808691409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55d30 0x56517ad5bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.399+0000 7fa439bf2640 1 --2- 172.21.15.94:0/2808691409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad5e8b0 0x56517ad5c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.399+0000 7fa438bf0640 1 -- 172.21.15.94:0/2808691409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55380 msgr2=0x56517ad5b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.399+0000 7fa438bf0640 1 --2- 172.21.15.94:0/2808691409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55380 0x56517ad5b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.399+0000 7fa438bf0640 1 -- 172.21.15.94:0/2808691409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad5e8b0 msgr2=0x56517ad5c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.399+0000 7fa438bf0640 1 --2- 172.21.15.94:0/2808691409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad5e8b0 0x56517ad5c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.399+0000 7fa438bf0640 1 -- 172.21.15.94:0/2808691409 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43000e040 con 0x56517ad55d30 2024-02-17T12:07:49.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:49 smithi189 ceph-mon[36588]: pgmap v300: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 MiB/s wr, 1.83k op/s 2024-02-17T12:07:49.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.400+0000 7fa438bf0640 1 --2- 172.21.15.94:0/2808691409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55d30 0x56517ad5bcd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa424006e10 tx=0x7fa424006da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7028b1813ec3570 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:49.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.400+0000 7fa41cff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:07:49.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.401+0000 7fa41cff9640 1 -- 172.21.15.94:0/2808691409 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424014610 con 0x56517ad55d30 2024-02-17T12:07:49.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.401+0000 7fa43e22d740 1 -- 172.21.15.94:0/2808691409 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56517ac96f40 con 0x56517ad55d30 2024-02-17T12:07:49.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.401+0000 7fa41cff9640 1 -- 172.21.15.94:0/2808691409 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa424014770 con 0x56517ad55d30 2024-02-17T12:07:49.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.401+0000 7fa41cff9640 1 -- 172.21.15.94:0/2808691409 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4240059c0 con 0x56517ad55d30 2024-02-17T12:07:49.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.403+0000 7fa43e22d740 10 client.27541 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:49.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.403+0000 7fa43e22d740 1 -- 172.21.15.94:0/2808691409 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56517acbb320 con 0x56517ad55d30 2024-02-17T12:07:49.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.403+0000 7fa41cff9640 1 -- 172.21.15.94:0/2808691409 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56517ac96f40 con 0x56517ad55d30 2024-02-17T12:07:49.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.404+0000 7fa41cff9640 1 -- 172.21.15.94:0/2808691409 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa424011070 con 0x56517ad55d30 2024-02-17T12:07:49.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.404+0000 7fa43e22d740 10 client.27541 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:49.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.404+0000 7fa43e22d740 10 client.27541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:49.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.404+0000 7fa43e22d740 20 client.27541 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:49.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.404+0000 7fa43e22d740 1 --2- 172.21.15.94:0/2808691409 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56517ad6ede0 0x56517ad71200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.404+0000 7fa43e22d740 4 client.27541 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:49.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.404+0000 7fa43e22d740 1 -- 172.21.15.94:0/2808691409 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56517acb3310 con 0x56517ad6ede0 2024-02-17T12:07:49.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.404+0000 7fa4393f1640 1 --2- 172.21.15.94:0/2808691409 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56517ad6ede0 0x56517ad71200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.404+0000 7fa4393f1640 1 --2- 172.21.15.94:0/2808691409 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56517ad6ede0 0x56517ad71200 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dfb25f14f043ec6 server_cookie=e6b6bb3732a301b0 in_seq=0 out_seq=0 2024-02-17T12:07:49.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.404+0000 7fa41cff9640 10 client.27541 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:49.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.405+0000 7fa41cff9640 1 -- 172.21.15.94:0/2808691409 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56517acb3310 con 0x56517ad6ede0 2024-02-17T12:07:49.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.405+0000 7fa41cff9640 10 client.27541 handle_command_reply: tid=0 2024-02-17T12:07:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.405+0000 7fa43e22d740 1 client.27541 shutdown 2024-02-17T12:07:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.405+0000 7fa43e22d740 1 -- 172.21.15.94:0/2808691409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55d30 msgr2=0x56517ad5bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.405+0000 7fa43e22d740 1 --2- 172.21.15.94:0/2808691409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55d30 0x56517ad5bcd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa424006e10 tx=0x7fa424006da0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.405+0000 7fa43e22d740 1 -- 172.21.15.94:0/2808691409 shutdown_connections 2024-02-17T12:07:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.405+0000 7fa43e22d740 1 --2- 172.21.15.94:0/2808691409 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56517ad6ede0 0x56517ad71200 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.405+0000 7fa43e22d740 1 --2- 172.21.15.94:0/2808691409 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad5e8b0 0x56517ad5c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.405+0000 7fa41cff9640 0 client.27541 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.405+0000 7fa43e22d740 1 --2- 172.21.15.94:0/2808691409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55d30 0x56517ad5bcd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.405+0000 7fa43e22d740 1 --2- 172.21.15.94:0/2808691409 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55380 0x56517ad5b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.405+0000 7fa43e22d740 1 -- 172.21.15.94:0/2808691409 >> 172.21.15.94:0/2808691409 conn(0x56517ad4f840 msgr2=0x56517ad517a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.406+0000 7fa43e22d740 1 -- 172.21.15.94:0/2808691409 shutdown_connections 2024-02-17T12:07:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.406+0000 7fa43e22d740 1 -- 172.21.15.94:0/2808691409 wait complete. 2024-02-17T12:07:49.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.406+0000 7fa43e22d740 20 client.27541 trim_cache size 0 max 16384 2024-02-17T12:07:49.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.427+0000 7fa43e22d740 1 Processor -- start 2024-02-17T12:07:49.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.427+0000 7fa43e22d740 1 -- start start 2024-02-17T12:07:49.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.427+0000 7fa43e22d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55380 0x56517ad5c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.428+0000 7fa43e22d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55d30 0x56517ad4e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.428+0000 7fa4393f1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55380 0x56517ad5c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.428+0000 7fa4393f1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55380 0x56517ad5c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56170/0 (socket says 172.21.15.94:56170) 2024-02-17T12:07:49.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.428+0000 7fa4393f1640 1 -- 172.21.15.94:0/3671739670 learned_addr learned my addr 172.21.15.94:0/3671739670 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:49.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.428+0000 7fa438bf0640 1 --2- 172.21.15.94:0/3671739670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55d30 0x56517ad4e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.428+0000 7fa43e22d740 1 --2- 172.21.15.94:0/3671739670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad5e8b0 0x56517ad65980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.428+0000 7fa43e22d740 1 -- 172.21.15.94:0/3671739670 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ac96f40 con 0x56517ad5e8b0 2024-02-17T12:07:49.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.428+0000 7fa43e22d740 1 -- 172.21.15.94:0/3671739670 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ad4f6d0 con 0x56517ad55d30 2024-02-17T12:07:49.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.428+0000 7fa43e22d740 1 -- 172.21.15.94:0/3671739670 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517ad61120 con 0x56517ad55380 2024-02-17T12:07:49.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.428+0000 7fa439bf2640 1 --2- 172.21.15.94:0/3671739670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad5e8b0 0x56517ad65980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.429+0000 7fa438bf0640 1 -- 172.21.15.94:0/3671739670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55380 msgr2=0x56517ad5c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.429+0000 7fa438bf0640 1 --2- 172.21.15.94:0/3671739670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55380 0x56517ad5c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.429+0000 7fa438bf0640 1 -- 172.21.15.94:0/3671739670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad5e8b0 msgr2=0x56517ad65980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.429+0000 7fa438bf0640 1 --2- 172.21.15.94:0/3671739670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad5e8b0 0x56517ad65980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.429+0000 7fa438bf0640 1 -- 172.21.15.94:0/3671739670 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56517ac57c00 con 0x56517ad55d30 2024-02-17T12:07:49.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.429+0000 7fa438bf0640 1 --2- 172.21.15.94:0/3671739670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55d30 0x56517ad4e760 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa424006aa0 tx=0x7fa42400a7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bac05e186bc75b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:49.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.429+0000 7fa41effd640 1 -- 172.21.15.94:0/3671739670 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42401fea0 con 0x56517ad55d30 2024-02-17T12:07:49.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.429+0000 7fa41effd640 1 -- 172.21.15.94:0/3671739670 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa424018070 con 0x56517ad55d30 2024-02-17T12:07:49.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.429+0000 7fa41effd640 1 -- 172.21.15.94:0/3671739670 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424032430 con 0x56517ad55d30 2024-02-17T12:07:49.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.430+0000 7fa43e22d740 1 -- 172.21.15.94:0/3671739670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55d30 msgr2=0x56517ad4e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.430+0000 7fa43e22d740 1 --2- 172.21.15.94:0/3671739670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55d30 0x56517ad4e760 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa424006aa0 tx=0x7fa42400a7c0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.430+0000 7fa43e22d740 1 -- 172.21.15.94:0/3671739670 shutdown_connections 2024-02-17T12:07:49.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.430+0000 7fa43e22d740 1 --2- 172.21.15.94:0/3671739670 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad5e8b0 0x56517ad65980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.430+0000 7fa43e22d740 1 --2- 172.21.15.94:0/3671739670 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad55d30 0x56517ad4e760 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.430+0000 7fa43e22d740 1 --2- 172.21.15.94:0/3671739670 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55380 0x56517ad5c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.430+0000 7fa43e22d740 1 -- 172.21.15.94:0/3671739670 >> 172.21.15.94:0/3671739670 conn(0x56517ad4f840 msgr2=0x56517ad4e050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:49.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.430+0000 7fa43e22d740 1 -- 172.21.15.94:0/3671739670 shutdown_connections 2024-02-17T12:07:49.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.430+0000 7fa43e22d740 1 -- 172.21.15.94:0/3671739670 wait complete. 2024-02-17T12:07:49.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.430+0000 7fa43e22d740 1 -- start start 2024-02-17T12:07:49.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.431+0000 7fa43e22d740 1 Processor -- start 2024-02-17T12:07:49.435 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.431+0000 7fa43e22d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55380 0x56517ad64e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.431+0000 7fa43e22d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad55d30 0x56517ad65340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.431+0000 7fa43e22d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad5e8b0 0x56517adbe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.431+0000 7fa43e22d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517adbe830 con 0x56517ad55d30 2024-02-17T12:07:49.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.431+0000 7fa43e22d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517adbe9a0 con 0x56517ad5e8b0 2024-02-17T12:07:49.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.431+0000 7fa43e22d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56517adbeb60 con 0x56517ad55380 2024-02-17T12:07:49.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.431+0000 7fa438bf0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad55d30 0x56517ad65340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.431+0000 7fa43e22d740 10 client.0 _ll_register_callbacks cb 0x7fa43000b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:49.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.431+0000 7fa439bf2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad5e8b0 0x56517adbe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.431+0000 7fa438bf0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad55d30 0x56517ad65340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40006/0 (socket says 172.21.15.94:40006) 2024-02-17T12:07:49.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.431+0000 7fa438bf0640 1 -- 172.21.15.94:0/2742228868 learned_addr learned my addr 172.21.15.94:0/2742228868 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:49.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.431+0000 7fa4393f1640 1 --2- 172.21.15.94:0/2742228868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55380 0x56517ad64e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.432+0000 7fa439bf2640 1 -- 172.21.15.94:0/2742228868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55380 msgr2=0x56517ad64e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.432+0000 7fa439bf2640 1 --2- 172.21.15.94:0/2742228868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55380 0x56517ad64e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.432+0000 7fa439bf2640 1 -- 172.21.15.94:0/2742228868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad55d30 msgr2=0x56517ad65340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.432+0000 7fa439bf2640 1 --2- 172.21.15.94:0/2742228868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad55d30 0x56517ad65340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.432+0000 7fa439bf2640 1 -- 172.21.15.94:0/2742228868 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56517acbb320 con 0x56517ad5e8b0 2024-02-17T12:07:49.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.433+0000 7fa439bf2640 1 --2- 172.21.15.94:0/2742228868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad5e8b0 0x56517adbe2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0ede60 tx=0x7fa42c0bcd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c82e9bc0b72fe4f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:49.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.433+0000 7fa41cff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:07:49.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:49 smithi125 ceph-mon[36819]: pgmap v300: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 MiB/s wr, 1.83k op/s 2024-02-17T12:07:49.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.433+0000 7fa41cff9640 1 -- 172.21.15.94:0/2742228868 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42c0ee1d0 con 0x56517ad5e8b0 2024-02-17T12:07:49.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.433+0000 7fa41cff9640 1 -- 172.21.15.94:0/2742228868 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa42c0c1250 con 0x56517ad5e8b0 2024-02-17T12:07:49.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.433+0000 7fa43e22d740 1 -- 172.21.15.94:0/2742228868 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56517ace2900 con 0x56517ad5e8b0 2024-02-17T12:07:49.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.433+0000 7fa41cff9640 1 -- 172.21.15.94:0/2742228868 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa42c0c58a0 con 0x56517ad5e8b0 2024-02-17T12:07:49.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.433+0000 7fa43e22d740 10 client.27566 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:49.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.433+0000 7fa43e22d740 1 -- 172.21.15.94:0/2742228868 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56517ac7d910 con 0x56517ad5e8b0 2024-02-17T12:07:49.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.434+0000 7fa41cff9640 1 -- 172.21.15.94:0/2742228868 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56517ace2900 con 0x56517ad5e8b0 2024-02-17T12:07:49.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.434+0000 7fa41cff9640 1 -- 172.21.15.94:0/2742228868 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa42c04c020 con 0x56517ad5e8b0 2024-02-17T12:07:49.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.434+0000 7fa43e22d740 10 client.27566 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:49.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.434+0000 7fa43e22d740 10 client.27566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:49.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.434+0000 7fa43e22d740 20 client.27566 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:49.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.434+0000 7fa43e22d740 1 --2- 172.21.15.94:0/2742228868 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56517ad6ede0 0x56517adc9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.434+0000 7fa43e22d740 4 client.27566 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:07:49.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.434+0000 7fa43e22d740 1 -- 172.21.15.94:0/2742228868 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56517a9bc860 con 0x56517ad6ede0 2024-02-17T12:07:49.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.434+0000 7fa4393f1640 1 --2- 172.21.15.94:0/2742228868 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56517ad6ede0 0x56517adc9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.435+0000 7fa4393f1640 1 --2- 172.21.15.94:0/2742228868 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56517ad6ede0 0x56517adc9210 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a651b82964deeea4 server_cookie=d44f983219e7821 in_seq=0 out_seq=0 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.435+0000 7fa41cff9640 10 client.27566 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa41cff9640 1 -- 172.21.15.94:0/2742228868 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56517a9bc860 con 0x56517ad6ede0 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa41cff9640 10 client.27566 handle_command_reply: tid=0 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa43e22d740 1 client.27566 shutdown 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa43e22d740 1 -- 172.21.15.94:0/2742228868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad5e8b0 msgr2=0x56517adbe2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa43e22d740 1 --2- 172.21.15.94:0/2742228868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad5e8b0 0x56517adbe2f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0ede60 tx=0x7fa42c0bcd00 comp rx=0 tx=0).stop 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa43e22d740 1 -- 172.21.15.94:0/2742228868 shutdown_connections 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa43e22d740 1 --2- 172.21.15.94:0/2742228868 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56517ad6ede0 0x56517adc9210 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa41cff9640 0 client.27566 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa43e22d740 1 --2- 172.21.15.94:0/2742228868 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56517ad5e8b0 0x56517adbe2f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa43e22d740 1 --2- 172.21.15.94:0/2742228868 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56517ad55d30 0x56517ad65340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa43e22d740 1 --2- 172.21.15.94:0/2742228868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56517ad55380 0x56517ad64e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa43e22d740 1 -- 172.21.15.94:0/2742228868 >> 172.21.15.94:0/2742228868 conn(0x56517ad4f840 msgr2=0x56517ad51660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:49.441 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa43e22d740 1 -- 172.21.15.94:0/2742228868 shutdown_connections 2024-02-17T12:07:49.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.436+0000 7fa43e22d740 1 -- 172.21.15.94:0/2742228868 wait complete. 2024-02-17T12:07:49.442 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.437+0000 7fa43e22d740 20 client.27566 trim_cache size 0 max 16384 2024-02-17T12:07:49.442 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:07:49.442 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:07:49.442 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "45d9d060-0037-4444-a189-cb665e375f98", 2024-02-17T12:07:49.442 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:07:49.442 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:07:49.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.438+0000 7fa43b655640 1 -- 172.21.15.94:0/4039606767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa4200a2440 msgr2=0x7fa4200a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.438+0000 7fa43b655640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa4200a2440 0x7fa4200a4900 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa424006fd0 tx=0x7fa424008040 comp rx=0 tx=0).stop 2024-02-17T12:07:49.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.438+0000 7fa43b655640 1 -- 172.21.15.94:0/4039606767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa43411a960 msgr2=0x7fa4341166a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.438+0000 7fa43b655640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa43411a960 0x7fa4341166a0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fa42c03d780 tx=0x7fa42c003ba0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.439+0000 7fa43b655640 1 -- 172.21.15.94:0/4039606767 shutdown_connections 2024-02-17T12:07:49.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.439+0000 7fa43b655640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa4200a2440 0x7fa4200a4900 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.439+0000 7fa43b655640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa43411a960 0x7fa4341166a0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.439+0000 7fa43b655640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa43410f9a0 0x7fa434116160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.439+0000 7fa43b655640 1 --2- 172.21.15.94:0/4039606767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa43410d9d0 0x7fa43411a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.439+0000 7fa43b655640 1 -- 172.21.15.94:0/4039606767 >> 172.21.15.94:0/4039606767 conn(0x7fa43406c260 msgr2=0x7fa43411c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:49.443 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.439+0000 7fa43b655640 1 -- 172.21.15.94:0/4039606767 shutdown_connections 2024-02-17T12:07:49.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.439+0000 7fa43b655640 1 -- 172.21.15.94:0/4039606767 wait complete. 2024-02-17T12:07:49.453 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:49 smithi094 ceph-mon[32499]: pgmap v300: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 25 MiB/s wr, 1.83k op/s 2024-02-17T12:07:49.455 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:07:49.455 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:07:49.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.607+0000 7f9ea1913640 1 -- 172.21.15.94:0/3360280391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e9c106430 msgr2=0x7f9e9c10e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.607+0000 7f9ea1913640 1 --2- 172.21.15.94:0/3360280391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e9c106430 0x7f9e9c10e000 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009cd0 tx=0x7f9e9402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.607+0000 7f9ea1913640 1 -- 172.21.15.94:0/3360280391 shutdown_connections 2024-02-17T12:07:49.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.607+0000 7f9ea1913640 1 --2- 172.21.15.94:0/3360280391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e9c106430 0x7f9e9c10e000 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.607+0000 7f9ea1913640 1 --2- 172.21.15.94:0/3360280391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e9c101d10 0x7f9e9c105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.607+0000 7f9ea1913640 1 --2- 172.21.15.94:0/3360280391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e9c101360 0x7f9e9c101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.608+0000 7f9ea1913640 1 -- 172.21.15.94:0/3360280391 >> 172.21.15.94:0/3360280391 conn(0x7f9e9c0fcc20 msgr2=0x7f9e9c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:49.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.608+0000 7f9ea1913640 1 -- 172.21.15.94:0/3360280391 shutdown_connections 2024-02-17T12:07:49.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.608+0000 7f9ea1913640 1 -- 172.21.15.94:0/3360280391 wait complete. 2024-02-17T12:07:49.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.608+0000 7f9ea1913640 1 Processor -- start 2024-02-17T12:07:49.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.608+0000 7f9ea1913640 1 -- start start 2024-02-17T12:07:49.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.608+0000 7f9ea1913640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e9c101360 0x7f9e9c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.608+0000 7f9ea1913640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e9c106430 0x7f9e9c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.609+0000 7f9ea1913640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e9c073f00 0x7f9e9c1a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.609+0000 7f9e9affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e9c101360 0x7f9e9c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.609+0000 7f9e9affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e9c101360 0x7f9e9c073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56194/0 (socket says 172.21.15.94:56194) 2024-02-17T12:07:49.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.609+0000 7f9e9affd640 1 -- 172.21.15.94:0/2663719962 learned_addr learned my addr 172.21.15.94:0/2663719962 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:49.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.609+0000 7f9e9b7fe640 1 --2- 172.21.15.94:0/2663719962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e9c073f00 0x7f9e9c1a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.609+0000 7f9ea1913640 1 -- 172.21.15.94:0/2663719962 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c1a5fe0 con 0x7f9e9c073f00 2024-02-17T12:07:49.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.609+0000 7f9ea1913640 1 -- 172.21.15.94:0/2663719962 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c1a6150 con 0x7f9e9c106430 2024-02-17T12:07:49.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.609+0000 7f9ea1913640 1 -- 172.21.15.94:0/2663719962 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9c1a6310 con 0x7f9e9c101360 2024-02-17T12:07:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.609+0000 7f9e9affd640 1 -- 172.21.15.94:0/2663719962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e9c106430 msgr2=0x7f9e9c0739c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-17T12:07:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.609+0000 7f9e9affd640 1 --2- 172.21.15.94:0/2663719962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e9c106430 0x7f9e9c0739c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.609+0000 7f9e9affd640 1 -- 172.21.15.94:0/2663719962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e9c073f00 msgr2=0x7f9e9c1a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.609+0000 7f9e9affd640 1 --2- 172.21.15.94:0/2663719962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e9c073f00 0x7f9e9c1a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.609+0000 7f9e9affd640 1 -- 172.21.15.94:0/2663719962 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e94009900 con 0x7f9e9c101360 2024-02-17T12:07:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.610+0000 7f9e9affd640 1 --2- 172.21.15.94:0/2663719962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e9c101360 0x7f9e9c073480 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00e9f0 tx=0x7f9e8c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.611+0000 7f9ea0911640 1 -- 172.21.15.94:0/2663719962 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c009700 con 0x7f9e9c101360 2024-02-17T12:07:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.611+0000 7f9ea0911640 1 -- 172.21.15.94:0/2663719962 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e8c00cd80 con 0x7f9e9c101360 2024-02-17T12:07:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.611+0000 7f9ea0911640 1 -- 172.21.15.94:0/2663719962 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c018920 con 0x7f9e9c101360 2024-02-17T12:07:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.611+0000 7f9ea1913640 1 -- 172.21.15.94:0/2663719962 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9c1a65f0 con 0x7f9e9c101360 2024-02-17T12:07:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.611+0000 7f9ea1913640 1 -- 172.21.15.94:0/2663719962 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9c1a6ab0 con 0x7f9e9c101360 2024-02-17T12:07:49.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.613+0000 7f9ea0911640 1 -- 172.21.15.94:0/2663719962 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9e8c018a80 con 0x7f9e9c101360 2024-02-17T12:07:49.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9ea45c6740 1 Processor -- start 2024-02-17T12:07:49.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9ea45c6740 1 -- start start 2024-02-17T12:07:49.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9ea45c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f46430c0 0x5561f46434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9ea45c6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4643a70 0x5561f4654a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9ea45c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4654fc0 0x5561f46573b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9ea45c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f45b2280 con 0x5561f4654fc0 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9ea45c6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f45af7b0 con 0x5561f4643a70 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9ea45c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f45aaf90 con 0x5561f46430c0 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9e93fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4654fc0 0x5561f46573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9e9affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4643a70 0x5561f4654a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9e93fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4654fc0 0x5561f46573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40036/0 (socket says 172.21.15.94:40036) 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9e93fff640 1 -- 172.21.15.94:0/3890896136 learned_addr learned my addr 172.21.15.94:0/3890896136 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9ea0911640 1 --2- 172.21.15.94:0/2663719962 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9e7c0a23b0 0x7f9e7c0a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9e9b7fe640 1 --2- 172.21.15.94:0/3890896136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f46430c0 0x5561f46434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9e93fff640 1 -- 172.21.15.94:0/3890896136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f46430c0 msgr2=0x5561f46434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9e93fff640 1 --2- 172.21.15.94:0/3890896136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f46430c0 0x5561f46434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9e93fff640 1 -- 172.21.15.94:0/3890896136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4643a70 msgr2=0x5561f4654a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9e9b7fe640 1 --2- 172.21.15.94:0/2663719962 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9e7c0a23b0 0x7f9e7c0a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9e93fff640 1 --2- 172.21.15.94:0/3890896136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4643a70 0x5561f4654a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9e93fff640 1 -- 172.21.15.94:0/3890896136 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561f4569570 con 0x5561f4654fc0 2024-02-17T12:07:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9e9affd640 1 --2- 172.21.15.94:0/3890896136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4643a70 0x5561f4654a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.614+0000 7f9ea0911640 1 -- 172.21.15.94:0/2663719962 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9e8c014070 con 0x7f9e9c101360 2024-02-17T12:07:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.615+0000 7f9e93fff640 1 --2- 172.21.15.94:0/3890896136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4654fc0 0x5561f46573b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400b980 tx=0x7f9e8400be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519944c904821766 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.615+0000 7f9e9b7fe640 1 --2- 172.21.15.94:0/2663719962 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9e7c0a23b0 0x7f9e7c0a4870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009ca0 tx=0x7f9e9402c0f0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.615+0000 7f9e91ffb640 1 -- 172.21.15.94:0/3890896136 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8400db80 con 0x5561f4654fc0 2024-02-17T12:07:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.615+0000 7f9e91ffb640 1 -- 172.21.15.94:0/3890896136 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e84005500 con 0x5561f4654fc0 2024-02-17T12:07:49.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.615+0000 7f9e91ffb640 1 -- 172.21.15.94:0/3890896136 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e84010460 con 0x5561f4654fc0 2024-02-17T12:07:49.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.616+0000 7f9ea45c6740 1 -- 172.21.15.94:0/3890896136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4654fc0 msgr2=0x5561f46573b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.616+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/3890896136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4654fc0 0x5561f46573b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400b980 tx=0x7f9e8400be40 comp rx=0 tx=0).stop 2024-02-17T12:07:49.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.616+0000 7f9ea45c6740 1 -- 172.21.15.94:0/3890896136 shutdown_connections 2024-02-17T12:07:49.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.616+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/3890896136 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4654fc0 0x5561f46573b0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.616+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/3890896136 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4643a70 0x5561f4654a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.616+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/3890896136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f46430c0 0x5561f46434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.616+0000 7f9ea45c6740 1 -- 172.21.15.94:0/3890896136 >> 172.21.15.94:0/3890896136 conn(0x5561f463d990 msgr2=0x5561f463dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:49.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.616+0000 7f9ea45c6740 1 -- 172.21.15.94:0/3890896136 shutdown_connections 2024-02-17T12:07:49.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.616+0000 7f9ea45c6740 1 -- 172.21.15.94:0/3890896136 wait complete. 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.616+0000 7f9ea45c6740 1 -- start start 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.616+0000 7f9ea45c6740 1 Processor -- start 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.617+0000 7f9ea45c6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f46430c0 0x5561f4654910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.617+0000 7f9e9affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f46430c0 0x5561f4654910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.617+0000 7f9e9affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f46430c0 0x5561f4654910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:58698/0 (socket says 172.21.15.94:58698) 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.617+0000 7f9ea45c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f4643a70 0x5561f4651ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.617+0000 7f9e9affd640 1 -- 172.21.15.94:0/2486853235 learned_addr learned my addr 172.21.15.94:0/2486853235 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.617+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/2486853235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4654fc0 0x5561f4652200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.617+0000 7f9ea45c6740 1 -- 172.21.15.94:0/2486853235 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f427e430 con 0x5561f4654fc0 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.617+0000 7f9ea45c6740 1 -- 172.21.15.94:0/2486853235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f423ae00 con 0x5561f46430c0 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.617+0000 7f9ea45c6740 1 -- 172.21.15.94:0/2486853235 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f452ffd0 con 0x5561f4643a70 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.617+0000 7f9ea45c6740 10 client.0 _ll_register_callbacks cb 0x5561f4585fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9e9affd640 1 -- 172.21.15.94:0/2486853235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f4643a70 msgr2=0x5561f4651ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:07:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9e9affd640 1 --2- 172.21.15.94:0/2486853235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f4643a70 0x5561f4651ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9e9affd640 1 -- 172.21.15.94:0/2486853235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4654fc0 msgr2=0x5561f4652200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9e9affd640 1 --2- 172.21.15.94:0/2486853235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4654fc0 0x5561f4652200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9e9affd640 1 -- 172.21.15.94:0/2486853235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8400b5e0 con 0x5561f46430c0 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9e9affd640 1 --2- 172.21.15.94:0/2486853235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f46430c0 0x5561f4654910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00e9c0 tx=0x7f9e8c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce8ed1c3caa35306 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9e8affd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9e8affd640 1 -- 172.21.15.94:0/2486853235 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c09a390 con 0x5561f46430c0 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9ea45c6740 1 -- 172.21.15.94:0/2486853235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561f42aa7e0 con 0x5561f46430c0 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9e8affd640 1 -- 172.21.15.94:0/2486853235 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9e8c09ad40 con 0x5561f46430c0 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9e8affd640 1 -- 172.21.15.94:0/2486853235 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e8c09a4f0 con 0x5561f46430c0 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9ea45c6740 10 client.27575 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9ea45c6740 1 -- 172.21.15.94:0/2486853235 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5561f45a9610 con 0x5561f46430c0 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.618+0000 7f9e8affd640 1 -- 172.21.15.94:0/2486853235 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5561f42aa7e0 con 0x5561f46430c0 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.619+0000 7f9e8affd640 1 -- 172.21.15.94:0/2486853235 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9e8c099c00 con 0x5561f46430c0 2024-02-17T12:07:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.619+0000 7f9ea45c6740 10 client.27575 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.619+0000 7f9ea45c6740 10 client.27575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.619+0000 7f9ea45c6740 20 client.27575 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.619+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/2486853235 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5561f46650a0 0x5561f4667560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.619+0000 7f9ea45c6740 4 client.27575 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.619+0000 7f9ea45c6740 1 -- 172.21.15.94:0/2486853235 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561f42e7720 con 0x5561f46650a0 2024-02-17T12:07:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.619+0000 7f9e93fff640 1 --2- 172.21.15.94:0/2486853235 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5561f46650a0 0x5561f4667560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.619+0000 7f9e8affd640 10 client.27575 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.619+0000 7f9e93fff640 1 --2- 172.21.15.94:0/2486853235 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5561f46650a0 0x5561f4667560 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe8befaf30259ff server_cookie=ce23201b8f98be62 in_seq=0 out_seq=0 2024-02-17T12:07:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.620+0000 7f9e8affd640 1 -- 172.21.15.94:0/2486853235 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5561f42e7720 con 0x5561f46650a0 2024-02-17T12:07:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.620+0000 7f9e8affd640 10 client.27575 handle_command_reply: tid=0 2024-02-17T12:07:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.620+0000 7f9ea45c6740 1 client.27575 shutdown 2024-02-17T12:07:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.622+0000 7f9ea45c6740 1 -- 172.21.15.94:0/2486853235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f46430c0 msgr2=0x5561f4654910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.622+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/2486853235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f46430c0 0x5561f4654910 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00e9c0 tx=0x7f9e8c009310 comp rx=0 tx=0).stop 2024-02-17T12:07:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.622+0000 7f9ea45c6740 1 -- 172.21.15.94:0/2486853235 shutdown_connections 2024-02-17T12:07:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.622+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/2486853235 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5561f46650a0 0x5561f4667560 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.622+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/2486853235 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4654fc0 0x5561f4652200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.622+0000 7f9e8affd640 0 client.27575 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.622+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/2486853235 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f4643a70 0x5561f4651ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.622+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/2486853235 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f46430c0 0x5561f4654910 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.622+0000 7f9ea45c6740 1 -- 172.21.15.94:0/2486853235 >> 172.21.15.94:0/2486853235 conn(0x5561f463d990 msgr2=0x5561f463f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.622+0000 7f9ea45c6740 1 -- 172.21.15.94:0/2486853235 shutdown_connections 2024-02-17T12:07:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.622+0000 7f9ea45c6740 1 -- 172.21.15.94:0/2486853235 wait complete. 2024-02-17T12:07:49.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.622+0000 7f9ea45c6740 20 client.27575 trim_cache size 0 max 16384 2024-02-17T12:07:49.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9ea45c6740 1 Processor -- start 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9ea45c6740 1 -- start start 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9ea45c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f46430c0 0x5561f4657300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9ea45c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4643a70 0x5561f463dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9ea45c6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4654fc0 0x5561f46543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9ea45c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f4661c10 con 0x5561f4643a70 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9ea45c6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f463e9e0 con 0x5561f4654fc0 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9ea45c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f4657c60 con 0x5561f46430c0 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9e93fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4643a70 0x5561f463dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9e9affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f46430c0 0x5561f4657300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9e93fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4643a70 0x5561f463dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40056/0 (socket says 172.21.15.94:40056) 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9e9affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f46430c0 0x5561f4657300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56226/0 (socket says 172.21.15.94:56226) 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9e93fff640 1 -- 172.21.15.94:0/3472738150 learned_addr learned my addr 172.21.15.94:0/3472738150 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9e9b7fe640 1 --2- 172.21.15.94:0/3472738150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4654fc0 0x5561f46543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9e9affd640 1 -- 172.21.15.94:0/3472738150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4654fc0 msgr2=0x5561f46543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9e9affd640 1 --2- 172.21.15.94:0/3472738150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4654fc0 0x5561f46543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9e9affd640 1 -- 172.21.15.94:0/3472738150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4643a70 msgr2=0x5561f463dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9e9affd640 1 --2- 172.21.15.94:0/3472738150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4643a70 0x5561f463dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9e9affd640 1 -- 172.21.15.94:0/3472738150 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561f4545ac0 con 0x5561f46430c0 2024-02-17T12:07:49.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9e93fff640 1 --2- 172.21.15.94:0/3472738150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4643a70 0x5561f463dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:49.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.641+0000 7f9e9affd640 1 --2- 172.21.15.94:0/3472738150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f46430c0 0x5561f4657300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c098ff0 tx=0x7f9e8c0975c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=721ac7022f9549dd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:49.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.642+0000 7f9e917fa640 1 -- 172.21.15.94:0/3472738150 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c09ee90 con 0x5561f46430c0 2024-02-17T12:07:49.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.642+0000 7f9e917fa640 1 -- 172.21.15.94:0/3472738150 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e8c0a34e0 con 0x5561f46430c0 2024-02-17T12:07:49.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.642+0000 7f9e917fa640 1 -- 172.21.15.94:0/3472738150 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c0b3a70 con 0x5561f46430c0 2024-02-17T12:07:49.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.642+0000 7f9ea45c6740 1 -- 172.21.15.94:0/3472738150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f46430c0 msgr2=0x5561f4657300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.642+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/3472738150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f46430c0 0x5561f4657300 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c098ff0 tx=0x7f9e8c0975c0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.642+0000 7f9ea45c6740 1 -- 172.21.15.94:0/3472738150 shutdown_connections 2024-02-17T12:07:49.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.642+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/3472738150 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4654fc0 0x5561f46543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.642+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/3472738150 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f4643a70 0x5561f463dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.642+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/3472738150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f46430c0 0x5561f4657300 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.642+0000 7f9ea45c6740 1 -- 172.21.15.94:0/3472738150 >> 172.21.15.94:0/3472738150 conn(0x5561f463d990 msgr2=0x5561f4651530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:49.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.642+0000 7f9ea45c6740 1 -- 172.21.15.94:0/3472738150 shutdown_connections 2024-02-17T12:07:49.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.643+0000 7f9ea45c6740 1 -- 172.21.15.94:0/3472738150 wait complete. 2024-02-17T12:07:49.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.643+0000 7f9ea45c6740 1 -- start start 2024-02-17T12:07:49.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.643+0000 7f9ea45c6740 1 Processor -- start 2024-02-17T12:07:49.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.643+0000 7f9ea45c6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f46430c0 0x5561f4658a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.643+0000 7f9ea45c6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f4643a70 0x5561f4658fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.643+0000 7f9ea45c6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4654fc0 0x5561f46594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.643+0000 7f9ea45c6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f46bb720 con 0x5561f46430c0 2024-02-17T12:07:49.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.643+0000 7f9ea45c6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f46bb890 con 0x5561f4654fc0 2024-02-17T12:07:49.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.643+0000 7f9ea45c6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561f46bba50 con 0x5561f4643a70 2024-02-17T12:07:49.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.643+0000 7f9ea45c6740 10 client.0 _ll_register_callbacks cb 0x7f9e84005ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:49.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.643+0000 7f9e9affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f46430c0 0x5561f4658a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.643+0000 7f9e9affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f46430c0 0x5561f4658a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40060/0 (socket says 172.21.15.94:40060) 2024-02-17T12:07:49.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.643+0000 7f9e9affd640 1 -- 172.21.15.94:0/421125193 learned_addr learned my addr 172.21.15.94:0/421125193 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:49.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.644+0000 7f9e93fff640 1 --2- 172.21.15.94:0/421125193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f4643a70 0x5561f4658fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.644+0000 7f9e9affd640 1 -- 172.21.15.94:0/421125193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f4643a70 msgr2=0x5561f4658fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.644+0000 7f9e9affd640 1 --2- 172.21.15.94:0/421125193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f4643a70 0x5561f4658fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.644+0000 7f9e9affd640 1 -- 172.21.15.94:0/421125193 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4654fc0 msgr2=0x5561f46594f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.644+0000 7f9e9affd640 1 --2- 172.21.15.94:0/421125193 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4654fc0 0x5561f46594f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.644+0000 7f9e9affd640 1 -- 172.21.15.94:0/421125193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561f4569570 con 0x5561f46430c0 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.644+0000 7f9e93fff640 1 --2- 172.21.15.94:0/421125193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f4643a70 0x5561f4658fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.644+0000 7f9e9affd640 1 --2- 172.21.15.94:0/421125193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f46430c0 0x5561f4658a70 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0cbbe0 tx=0x7f9e8c0d1a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37c4850faf11fb6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.644+0000 7f9e8affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.645+0000 7f9e8affd640 1 -- 172.21.15.94:0/421125193 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c09a880 con 0x5561f46430c0 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.645+0000 7f9e8affd640 1 -- 172.21.15.94:0/421125193 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9e8c09ee90 con 0x5561f46430c0 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.645+0000 7f9ea45c6740 1 -- 172.21.15.94:0/421125193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e84005500 con 0x5561f46430c0 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.645+0000 7f9e8affd640 1 -- 172.21.15.94:0/421125193 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e8c096a90 con 0x5561f46430c0 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.645+0000 7f9ea45c6740 10 client.18090 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.645+0000 7f9ea45c6740 1 -- 172.21.15.94:0/421125193 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5561f45a9610 con 0x5561f46430c0 2024-02-17T12:07:49.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.646+0000 7f9e8affd640 1 -- 172.21.15.94:0/421125193 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9e8c096bf0 con 0x5561f46430c0 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.646+0000 7f9e8affd640 1 -- 172.21.15.94:0/421125193 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9e8c005040 con 0x5561f46430c0 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.646+0000 7f9ea45c6740 10 client.18090 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.646+0000 7f9ea45c6740 10 client.18090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.646+0000 7f9ea45c6740 20 client.18090 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.646+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/421125193 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5561f46650a0 0x5561f4646290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.646+0000 7f9ea45c6740 4 client.18090 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.646+0000 7f9ea45c6740 1 -- 172.21.15.94:0/421125193 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7f9e84005500 con 0x5561f46650a0 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.646+0000 7f9e93fff640 1 --2- 172.21.15.94:0/421125193 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5561f46650a0 0x5561f4646290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.646+0000 7f9e8affd640 10 client.18090 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9e93fff640 1 --2- 172.21.15.94:0/421125193 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5561f46650a0 0x5561f4646290 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56b9a5ce1183b71a server_cookie=1859b639f22eb565 in_seq=0 out_seq=0 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9e8affd640 1 -- 172.21.15.94:0/421125193 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7f9e84005500 con 0x5561f46650a0 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9e8affd640 10 client.18090 handle_command_reply: tid=0 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9ea45c6740 1 client.18090 shutdown 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9ea45c6740 1 -- 172.21.15.94:0/421125193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f46430c0 msgr2=0x5561f4658a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/421125193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f46430c0 0x5561f4658a70 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c0cbbe0 tx=0x7f9e8c0d1a00 comp rx=0 tx=0).stop 2024-02-17T12:07:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9ea45c6740 1 -- 172.21.15.94:0/421125193 shutdown_connections 2024-02-17T12:07:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/421125193 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5561f46650a0 0x5561f4646290 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/421125193 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5561f4654fc0 0x5561f46594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9e8affd640 0 client.18090 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/421125193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5561f4643a70 0x5561f4658fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9ea45c6740 1 --2- 172.21.15.94:0/421125193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5561f46430c0 0x5561f4658a70 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9ea45c6740 1 -- 172.21.15.94:0/421125193 >> 172.21.15.94:0/421125193 conn(0x5561f463d990 msgr2=0x5561f463dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9ea45c6740 1 -- 172.21.15.94:0/421125193 shutdown_connections 2024-02-17T12:07:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9ea45c6740 1 -- 172.21.15.94:0/421125193 wait complete. 2024-02-17T12:07:49.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.647+0000 7f9ea45c6740 20 client.18090 trim_cache size 0 max 16384 2024-02-17T12:07:49.654 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:07:49.655 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:07:49.655 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:07:49.655 INFO:teuthology.orchestra.run.smithi094.stdout: "45d9d060-0037-4444-a189-cb665e375f98": { 2024-02-17T12:07:49.655 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:07:49.655 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "45d9d060-0037-4444-a189-cb665e375f98", 2024-02-17T12:07:49.655 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:07:49.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:07:49.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:07:49.655 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:07:49.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.648+0000 7f9ea1913640 1 -- 172.21.15.94:0/2663719962 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9e7c0a23b0 msgr2=0x7f9e7c0a4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.648+0000 7f9ea1913640 1 --2- 172.21.15.94:0/2663719962 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9e7c0a23b0 0x7f9e7c0a4870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009ca0 tx=0x7f9e9402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.648+0000 7f9ea1913640 1 -- 172.21.15.94:0/2663719962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e9c101360 msgr2=0x7f9e9c073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:49.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.648+0000 7f9ea1913640 1 --2- 172.21.15.94:0/2663719962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e9c101360 0x7f9e9c073480 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c00e9f0 tx=0x7f9e8c00eeb0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.648+0000 7f9ea1913640 1 -- 172.21.15.94:0/2663719962 shutdown_connections 2024-02-17T12:07:49.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.648+0000 7f9ea1913640 1 --2- 172.21.15.94:0/2663719962 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9e7c0a23b0 0x7f9e7c0a4870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.648+0000 7f9ea1913640 1 --2- 172.21.15.94:0/2663719962 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9e9c073f00 0x7f9e9c1a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.648+0000 7f9ea1913640 1 --2- 172.21.15.94:0/2663719962 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9e9c106430 0x7f9e9c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.648+0000 7f9ea1913640 1 --2- 172.21.15.94:0/2663719962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9e9c101360 0x7f9e9c073480 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:49.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.648+0000 7f9ea1913640 1 -- 172.21.15.94:0/2663719962 >> 172.21.15.94:0/2663719962 conn(0x7f9e9c0fcc20 msgr2=0x7f9e9c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:49.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.648+0000 7f9ea1913640 1 -- 172.21.15.94:0/2663719962 shutdown_connections 2024-02-17T12:07:49.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:07:49.648+0000 7f9ea1913640 1 -- 172.21.15.94:0/2663719962 wait complete. 2024-02-17T12:07:49.664 INFO:tasks.cephfs.filesystem:scrub status for tag:45d9d060-0037-4444-a189-cb665e375f98 - {'path': '/', 'tag': '45d9d060-0037-4444-a189-cb665e375f98', 'options': 'recursive,force'} 2024-02-17T12:07:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 570.0 s, 2076.0 tps, lat 47.870 ms stddev 52.599 2024-02-17T12:07:50.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:50 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:07:50.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:50 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:07:50.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:50 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:07:50.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:50 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:07:50.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:50 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:07:50.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:50 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:07:50.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:50 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:07:50.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:50 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:07:50.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:50 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:07:50.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:50 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:07:50.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:50 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:07:50.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:50 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:07:51.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:51 smithi189 ceph-mon[36588]: pgmap v301: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 MiB/s wr, 599 op/s 2024-02-17T12:07:51.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:51 smithi125 ceph-mon[36819]: pgmap v301: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 MiB/s wr, 599 op/s 2024-02-17T12:07:51.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:51 smithi094 ceph-mon[32499]: pgmap v301: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 MiB/s wr, 599 op/s 2024-02-17T12:07:52.803 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:52 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:07:52.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:52 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:07:52.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:52 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:07:53.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:53 smithi189 ceph-mon[36588]: pgmap v302: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 MiB/s wr, 599 op/s 2024-02-17T12:07:53.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:53 smithi125 ceph-mon[36819]: pgmap v302: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 MiB/s wr, 599 op/s 2024-02-17T12:07:53.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:53 smithi094 ceph-mon[32499]: pgmap v302: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.3 MiB/s wr, 599 op/s 2024-02-17T12:07:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.949+0000 7f21fc498640 1 -- 172.21.15.94:0/1846866106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101d40 msgr2=0x7f21f4105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.949+0000 7f21fc498640 1 --2- 172.21.15.94:0/1846866106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101d40 0x7f21f4105d80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009930 tx=0x7f21e402bc30 comp rx=0 tx=0).stop 2024-02-17T12:07:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.949+0000 7f21fc498640 1 -- 172.21.15.94:0/1846866106 shutdown_connections 2024-02-17T12:07:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.949+0000 7f21fc498640 1 --2- 172.21.15.94:0/1846866106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f4106460 0x7f21f410e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.949+0000 7f21fc498640 1 --2- 172.21.15.94:0/1846866106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101d40 0x7f21f4105d80 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.949+0000 7f21fc498640 1 --2- 172.21.15.94:0/1846866106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21f4101390 0x7f21f4101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.949+0000 7f21fc498640 1 -- 172.21.15.94:0/1846866106 >> 172.21.15.94:0/1846866106 conn(0x7f21f40fcc10 msgr2=0x7f21f40ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.949+0000 7f21fc498640 1 -- 172.21.15.94:0/1846866106 shutdown_connections 2024-02-17T12:07:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.949+0000 7f21fc498640 1 -- 172.21.15.94:0/1846866106 wait complete. 2024-02-17T12:07:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.950+0000 7f21fc498640 1 Processor -- start 2024-02-17T12:07:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.950+0000 7f21fc498640 1 -- start start 2024-02-17T12:07:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21fc498640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101390 0x7f21f4197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21fc498640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21f4106460 0x7f21f4197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21fc498640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f419c020 0x7f21f419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21fc498640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f419ca10 con 0x7f21f4101390 2024-02-17T12:07:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21fc498640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f419cb80 con 0x7f21f419c020 2024-02-17T12:07:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21fc498640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f419ccf0 con 0x7f21f4106460 2024-02-17T12:07:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21fa234640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101390 0x7f21f4197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21fa234640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101390 0x7f21f4197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59696/0 (socket says 172.21.15.94:59696) 2024-02-17T12:07:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21fa234640 1 -- 172.21.15.94:0/3739062297 learned_addr learned my addr 172.21.15.94:0/3739062297 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:54.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21faa35640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f419c020 0x7f21f419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21faa35640 1 -- 172.21.15.94:0/3739062297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21f4106460 msgr2=0x7f21f4197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21faa35640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21f4106460 0x7f21f4197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21faa35640 1 -- 172.21.15.94:0/3739062297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101390 msgr2=0x7f21f4197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.951+0000 7f21faa35640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101390 0x7f21f4197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.952+0000 7f21faa35640 1 -- 172.21.15.94:0/3739062297 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e4009560 con 0x7f21f419c020 2024-02-17T12:07:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.952+0000 7f21fa234640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101390 0x7f21f4197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.952+0000 7f21faa35640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f419c020 0x7f21f419c4d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009fd0 tx=0x7f21ec00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.952+0000 7f21eb7fe640 1 -- 172.21.15.94:0/3739062297 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec00ce20 con 0x7f21f419c020 2024-02-17T12:07:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.952+0000 7f21eb7fe640 1 -- 172.21.15.94:0/3739062297 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21ec005020 con 0x7f21f419c020 2024-02-17T12:07:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.952+0000 7f21eb7fe640 1 -- 172.21.15.94:0/3739062297 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec010690 con 0x7f21f419c020 2024-02-17T12:07:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.952+0000 7f21fc498640 1 -- 172.21.15.94:0/3739062297 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f406cbb0 con 0x7f21f419c020 2024-02-17T12:07:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.952+0000 7f21fc498640 1 -- 172.21.15.94:0/3739062297 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f406d010 con 0x7f21f419c020 2024-02-17T12:07:54.956 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.955+0000 7f21eb7fe640 1 -- 172.21.15.94:0/3739062297 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f21ec005500 con 0x7f21f419c020 2024-02-17T12:07:54.957 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.956+0000 7f21ff060740 1 Processor -- start 2024-02-17T12:07:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.956+0000 7f21ff060740 1 -- start start 2024-02-17T12:07:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.956+0000 7f21eb7fe640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f21e00a2430 0x7f21e00a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.956+0000 7f21ff060740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 0x5573f611a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.956+0000 7f21ff060740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611ad60 0x5573f611b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.956+0000 7f21ff060740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f61242e0 0x5573f61266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.956+0000 7f21ff060740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f60899f0 con 0x5573f611bad0 2024-02-17T12:07:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.956+0000 7f21ff060740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f6086f20 con 0x5573f61242e0 2024-02-17T12:07:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.956+0000 7f21ff060740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f6082700 con 0x5573f611ad60 2024-02-17T12:07:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.956+0000 7f21f9a33640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f21e00a2430 0x7f21e00a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.956+0000 7f21eb7fe640 1 -- 172.21.15.94:0/3739062297 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f21ec014070 con 0x7f21f419c020 2024-02-17T12:07:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21fa234640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 0x5573f611a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21fa234640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 0x5573f611a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59710/0 (socket says 172.21.15.94:59710) 2024-02-17T12:07:54.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21fa234640 1 -- 172.21.15.94:0/3592082761 learned_addr learned my addr 172.21.15.94:0/3592082761 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21f9a33640 1 --2- 172.21.15.94:0/3592082761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611ad60 0x5573f611b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21faa35640 1 --2- 172.21.15.94:0/3592082761 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f61242e0 0x5573f61266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21fa234640 1 -- 172.21.15.94:0/3592082761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611ad60 msgr2=0x5573f611b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21fa234640 1 --2- 172.21.15.94:0/3592082761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611ad60 0x5573f611b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21fa234640 1 -- 172.21.15.94:0/3592082761 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f61242e0 msgr2=0x5573f61266d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21fa234640 1 --2- 172.21.15.94:0/3592082761 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f61242e0 0x5573f61266d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21fa234640 1 -- 172.21.15.94:0/3592082761 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573f60187b0 con 0x5573f611bad0 2024-02-17T12:07:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21faa35640 1 --2- 172.21.15.94:0/3592082761 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f61242e0 0x5573f61266d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:07:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21f9a33640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f21e00a2430 0x7f21e00a48f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f21f4107a00 tx=0x7f21e4004800 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21fa234640 1 --2- 172.21.15.94:0/3592082761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 0x5573f611a820 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f21f0010560 tx=0x7f21f0010a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea2cbb4894d96ef server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21e8bfe640 1 -- 172.21.15.94:0/3592082761 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0011760 con 0x5573f611bad0 2024-02-17T12:07:54.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21e8bfe640 1 -- 172.21.15.94:0/3592082761 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21f0015070 con 0x5573f611bad0 2024-02-17T12:07:54.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.957+0000 7f21e8bfe640 1 -- 172.21.15.94:0/3592082761 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f001cd60 con 0x5573f611bad0 2024-02-17T12:07:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.958+0000 7f21ff060740 1 -- 172.21.15.94:0/3592082761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 msgr2=0x5573f611a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.958+0000 7f21ff060740 1 --2- 172.21.15.94:0/3592082761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 0x5573f611a820 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f21f0010560 tx=0x7f21f0010a20 comp rx=0 tx=0).stop 2024-02-17T12:07:54.965 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.958+0000 7f21ff060740 1 -- 172.21.15.94:0/3592082761 shutdown_connections 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.958+0000 7f21ff060740 1 --2- 172.21.15.94:0/3592082761 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f61242e0 0x5573f61266d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.958+0000 7f21ff060740 1 --2- 172.21.15.94:0/3592082761 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611ad60 0x5573f611b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.958+0000 7f21ff060740 1 --2- 172.21.15.94:0/3592082761 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 0x5573f611a820 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.958+0000 7f21ff060740 1 -- 172.21.15.94:0/3592082761 >> 172.21.15.94:0/3592082761 conn(0x5573f60e2550 msgr2=0x5573f6115a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.958+0000 7f21ff060740 1 -- 172.21.15.94:0/3592082761 shutdown_connections 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.958+0000 7f21ff060740 1 -- 172.21.15.94:0/3592082761 wait complete. 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21ff060740 1 -- start start 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21ff060740 1 Processor -- start 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21ff060740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611ad60 0x5573f61212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21ff060740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 0x5573f6121800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21ff060740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f61242e0 0x5573f6121d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21ff060740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f6027bf0 con 0x5573f611bad0 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21ff060740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f5d56250 con 0x5573f611ad60 2024-02-17T12:07:54.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21ff060740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f6008200 con 0x5573f61242e0 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21ff060740 10 client.0 _ll_register_callbacks cb 0x5573f6076d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21f9a33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 0x5573f6121800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21fa234640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611ad60 0x5573f61212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21fa234640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611ad60 0x5573f61212c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53704/0 (socket says 172.21.15.94:53704) 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21f9a33640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 0x5573f6121800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59722/0 (socket says 172.21.15.94:59722) 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21fa234640 1 -- 172.21.15.94:0/1428014335 learned_addr learned my addr 172.21.15.94:0/1428014335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.959+0000 7f21faa35640 1 --2- 172.21.15.94:0/1428014335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f61242e0 0x5573f6121d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21fa234640 1 -- 172.21.15.94:0/1428014335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f61242e0 msgr2=0x5573f6121d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21fa234640 1 --2- 172.21.15.94:0/1428014335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f61242e0 0x5573f6121d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21fa234640 1 -- 172.21.15.94:0/1428014335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 msgr2=0x5573f6121800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21fa234640 1 --2- 172.21.15.94:0/1428014335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 0x5573f6121800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21fa234640 1 -- 172.21.15.94:0/1428014335 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f000bdf0 con 0x5573f611ad60 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21f9a33640 1 --2- 172.21.15.94:0/1428014335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 0x5573f6121800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:54.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21fa234640 1 --2- 172.21.15.94:0/1428014335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611ad60 0x5573f61212c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f21f0011010 tx=0x7f21f001c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de276226de9fc8a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21ddffb640 1 -- 172.21.15.94:0/1428014335 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f0020040 con 0x5573f611ad60 2024-02-17T12:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21ddffb640 1 -- 172.21.15.94:0/1428014335 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f21f0017070 con 0x5573f611ad60 2024-02-17T12:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21ff060740 1 -- 172.21.15.94:0/1428014335 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573f5d82600 con 0x5573f611ad60 2024-02-17T12:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21ddffb640 1 -- 172.21.15.94:0/1428014335 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21f0005020 con 0x5573f611ad60 2024-02-17T12:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21ff060740 10 client.27590 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21ff060740 1 -- 172.21.15.94:0/1428014335 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5573f6080d80 con 0x5573f611ad60 2024-02-17T12:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.960+0000 7f21ddffb640 1 -- 172.21.15.94:0/1428014335 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21f0032020 con 0x5573f611ad60 2024-02-17T12:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.961+0000 7f21ddffb640 1 -- 172.21.15.94:0/1428014335 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f21f0011560 con 0x5573f611ad60 2024-02-17T12:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.961+0000 7f21ff060740 10 client.27590 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.961+0000 7f21ff060740 10 client.27590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.961+0000 7f21ff060740 20 client.27590 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:54.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.961+0000 7f21ff060740 1 --2- 172.21.15.94:0/1428014335 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5573f61388f0 0x5573f613ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.961+0000 7f21ff060740 4 client.27590 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.961+0000 7f21ff060740 1 -- 172.21.15.94:0/1428014335 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573f5dd31b0 con 0x5573f61388f0 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.961+0000 7f21f9a33640 1 --2- 172.21.15.94:0/1428014335 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5573f61388f0 0x5573f613ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.961+0000 7f21ddffb640 10 client.27590 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.961+0000 7f21f9a33640 1 --2- 172.21.15.94:0/1428014335 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5573f61388f0 0x5573f613ad30 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5491097d29c91279 server_cookie=76fdf9415bf853bc in_seq=0 out_seq=0 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ddffb640 1 -- 172.21.15.94:0/1428014335 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5573f5dd31b0 con 0x5573f61388f0 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ddffb640 10 client.27590 handle_command_reply: tid=0 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ff060740 1 client.27590 shutdown 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ff060740 1 -- 172.21.15.94:0/1428014335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611ad60 msgr2=0x5573f61212c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ff060740 1 --2- 172.21.15.94:0/1428014335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611ad60 0x5573f61212c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f21f0011010 tx=0x7f21f001c7f0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ff060740 1 -- 172.21.15.94:0/1428014335 shutdown_connections 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ff060740 1 --2- 172.21.15.94:0/1428014335 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5573f61388f0 0x5573f613ad30 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ddffb640 0 client.27590 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:54.969 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ff060740 1 --2- 172.21.15.94:0/1428014335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f61242e0 0x5573f6121d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ff060740 1 --2- 172.21.15.94:0/1428014335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f611bad0 0x5573f6121800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ff060740 1 --2- 172.21.15.94:0/1428014335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611ad60 0x5573f61212c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ff060740 1 -- 172.21.15.94:0/1428014335 >> 172.21.15.94:0/1428014335 conn(0x5573f60e2550 msgr2=0x5573f61177a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ff060740 1 -- 172.21.15.94:0/1428014335 shutdown_connections 2024-02-17T12:07:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ff060740 1 -- 172.21.15.94:0/1428014335 wait complete. 2024-02-17T12:07:54.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.962+0000 7f21ff060740 20 client.27590 trim_cache size 0 max 16384 2024-02-17T12:07:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.980+0000 7f21ff060740 1 Processor -- start 2024-02-17T12:07:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.980+0000 7f21ff060740 1 -- start start 2024-02-17T12:07:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.980+0000 7f21ff060740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611ad60 0x5573f611fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.980+0000 7f21ff060740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611bad0 0x5573f61200d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21ff060740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f61242e0 0x5573f6121810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21ff060740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f5dd31b0 con 0x5573f61242e0 2024-02-17T12:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21ff060740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f6116850 con 0x5573f611bad0 2024-02-17T12:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21ff060740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f611ced0 con 0x5573f611ad60 2024-02-17T12:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21f9a33640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611bad0 0x5573f61200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21f9a33640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611bad0 0x5573f61200d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53710/0 (socket says 172.21.15.94:53710) 2024-02-17T12:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21f9a33640 1 -- 172.21.15.94:0/99476872 learned_addr learned my addr 172.21.15.94:0/99476872 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21faa35640 1 --2- 172.21.15.94:0/99476872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f61242e0 0x5573f6121810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21fa234640 1 --2- 172.21.15.94:0/99476872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611ad60 0x5573f611fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21f9a33640 1 -- 172.21.15.94:0/99476872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611ad60 msgr2=0x5573f611fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21f9a33640 1 --2- 172.21.15.94:0/99476872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611ad60 0x5573f611fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21f9a33640 1 -- 172.21.15.94:0/99476872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f61242e0 msgr2=0x5573f6121810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21f9a33640 1 --2- 172.21.15.94:0/99476872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f61242e0 0x5573f6121810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21f9a33640 1 -- 172.21.15.94:0/99476872 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573f601d7b0 con 0x5573f611bad0 2024-02-17T12:07:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21f9a33640 1 --2- 172.21.15.94:0/99476872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611bad0 0x5573f61200d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f21f4068480 tx=0x7f21e4046000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82e3d29f6aa490c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.981+0000 7f21dffff640 1 -- 172.21.15.94:0/99476872 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e4048040 con 0x5573f611bad0 2024-02-17T12:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.982+0000 7f21dffff640 1 -- 172.21.15.94:0/99476872 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21e402cbf0 con 0x5573f611bad0 2024-02-17T12:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.982+0000 7f21dffff640 1 -- 172.21.15.94:0/99476872 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e4005350 con 0x5573f611bad0 2024-02-17T12:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.982+0000 7f21ff060740 1 -- 172.21.15.94:0/99476872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611bad0 msgr2=0x5573f61200d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.982+0000 7f21ff060740 1 --2- 172.21.15.94:0/99476872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611bad0 0x5573f61200d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f21f4068480 tx=0x7f21e4046000 comp rx=0 tx=0).stop 2024-02-17T12:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.982+0000 7f21ff060740 1 -- 172.21.15.94:0/99476872 shutdown_connections 2024-02-17T12:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.982+0000 7f21ff060740 1 --2- 172.21.15.94:0/99476872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f61242e0 0x5573f6121810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.982+0000 7f21ff060740 1 --2- 172.21.15.94:0/99476872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611bad0 0x5573f61200d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.982+0000 7f21ff060740 1 --2- 172.21.15.94:0/99476872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611ad60 0x5573f611fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.982+0000 7f21ff060740 1 -- 172.21.15.94:0/99476872 >> 172.21.15.94:0/99476872 conn(0x5573f60e2550 msgr2=0x5573f613f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.982+0000 7f21ff060740 1 -- 172.21.15.94:0/99476872 shutdown_connections 2024-02-17T12:07:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.982+0000 7f21ff060740 1 -- 172.21.15.94:0/99476872 wait complete. 2024-02-17T12:07:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21ff060740 1 -- start start 2024-02-17T12:07:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21ff060740 1 Processor -- start 2024-02-17T12:07:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21ff060740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611ad60 0x5573f6129bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21ff060740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611bad0 0x5573f612a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21ff060740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f61242e0 0x5573f612a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21ff060740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f618ee60 con 0x5573f61242e0 2024-02-17T12:07:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21ff060740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f618efd0 con 0x5573f611ad60 2024-02-17T12:07:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21ff060740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573f618f190 con 0x5573f611bad0 2024-02-17T12:07:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21ff060740 10 client.0 _ll_register_callbacks cb 0x7f21f000bbe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:07:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21faa35640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f61242e0 0x5573f612a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21faa35640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f61242e0 0x5573f612a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:59738/0 (socket says 172.21.15.94:59738) 2024-02-17T12:07:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21faa35640 1 -- 172.21.15.94:0/2449178190 learned_addr learned my addr 172.21.15.94:0/2449178190 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:07:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21f9a33640 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611bad0 0x5573f612a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21fa234640 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611ad60 0x5573f6129bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21faa35640 1 -- 172.21.15.94:0/2449178190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611bad0 msgr2=0x5573f612a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21faa35640 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611bad0 0x5573f612a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21faa35640 1 -- 172.21.15.94:0/2449178190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611ad60 msgr2=0x5573f6129bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:07:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21faa35640 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611ad60 0x5573f6129bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21faa35640 1 -- 172.21.15.94:0/2449178190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f410e840 con 0x5573f61242e0 2024-02-17T12:07:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.983+0000 7f21f9a33640 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611bad0 0x5573f612a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:07:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.984+0000 7f21faa35640 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f61242e0 0x5573f612a670 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f21ec097ab0 tx=0x7f21ec09ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739329bdb6e7c82 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:07:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.984+0000 7f21ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:07:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.984+0000 7f21ddffb640 1 -- 172.21.15.94:0/2449178190 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec0b0050 con 0x5573f61242e0 2024-02-17T12:07:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.984+0000 7f21ff060740 1 -- 172.21.15.94:0/2449178190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21f0015070 con 0x5573f61242e0 2024-02-17T12:07:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.984+0000 7f21ddffb640 1 -- 172.21.15.94:0/2449178190 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f21ec09b540 con 0x5573f61242e0 2024-02-17T12:07:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.984+0000 7f21ddffb640 1 -- 172.21.15.94:0/2449178190 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21ec0a33f0 con 0x5573f61242e0 2024-02-17T12:07:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.984+0000 7f21ff060740 10 client.18108 fetch_fsmap learned FSMap version 27 2024-02-17T12:07:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.984+0000 7f21ff060740 1 -- 172.21.15.94:0/2449178190 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5573f6043540 con 0x5573f61242e0 2024-02-17T12:07:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.984+0000 7f21ddffb640 1 -- 172.21.15.94:0/2449178190 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21f0015070 con 0x5573f61242e0 2024-02-17T12:07:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.984+0000 7f21ddffb640 1 -- 172.21.15.94:0/2449178190 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f21ec01f070 con 0x5573f61242e0 2024-02-17T12:07:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.984+0000 7f21ff060740 10 client.18108 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:07:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.984+0000 7f21ff060740 10 client.18108 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:07:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.984+0000 7f21ff060740 20 client.18108 populate_metadata read hostname 'smithi094' 2024-02-17T12:07:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.985+0000 7f21ff060740 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5573f61388f0 0x5573f6192a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:07:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.985+0000 7f21ff060740 4 client.18108 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:07:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.985+0000 7f21ff060740 1 -- 172.21.15.94:0/2449178190 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5573f5dbf550 con 0x5573f61388f0 2024-02-17T12:07:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.985+0000 7f21fa234640 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5573f61388f0 0x5573f6192a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:07:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.985+0000 7f21ddffb640 10 client.18108 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:54.992 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:54.985+0000 7f21fa234640 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5573f61388f0 0x5573f6192a30 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa6b54d42f8d6f5 server_cookie=3f9154c43e63d2ce in_seq=0 out_seq=0 2024-02-17T12:07:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ddffb640 1 -- 172.21.15.94:0/2449178190 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5573f5dbf550 con 0x5573f61388f0 2024-02-17T12:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ddffb640 10 client.18108 handle_command_reply: tid=0 2024-02-17T12:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ff060740 1 client.18108 shutdown 2024-02-17T12:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ff060740 1 -- 172.21.15.94:0/2449178190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f61242e0 msgr2=0x5573f612a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ff060740 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f61242e0 0x5573f612a670 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f21ec097ab0 tx=0x7f21ec09ad50 comp rx=0 tx=0).stop 2024-02-17T12:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ff060740 1 -- 172.21.15.94:0/2449178190 shutdown_connections 2024-02-17T12:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ff060740 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5573f61388f0 0x5573f6192a30 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ff060740 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5573f61242e0 0x5573f612a670 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ff060740 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5573f611bad0 0x5573f612a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ff060740 1 --2- 172.21.15.94:0/2449178190 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5573f611ad60 0x5573f6129bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ff060740 1 -- 172.21.15.94:0/2449178190 >> 172.21.15.94:0/2449178190 conn(0x5573f60e2550 msgr2=0x5573f611fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ddffb640 0 client.18108 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:07:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ff060740 1 -- 172.21.15.94:0/2449178190 shutdown_connections 2024-02-17T12:07:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.002+0000 7f21ff060740 1 -- 172.21.15.94:0/2449178190 wait complete. 2024-02-17T12:07:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.003+0000 7f21ff060740 20 client.18108 trim_cache size 0 max 16384 2024-02-17T12:07:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:07:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:07:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:07:55.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:07:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.003+0000 7f21fc498640 1 -- 172.21.15.94:0/3739062297 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f21e00a2430 msgr2=0x7f21e00a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.003+0000 7f21fc498640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f21e00a2430 0x7f21e00a48f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f21f4107a00 tx=0x7f21e4004800 comp rx=0 tx=0).stop 2024-02-17T12:07:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.003+0000 7f21fc498640 1 -- 172.21.15.94:0/3739062297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f419c020 msgr2=0x7f21f419c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:07:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.003+0000 7f21fc498640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f419c020 0x7f21f419c4d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009fd0 tx=0x7f21ec00ef10 comp rx=0 tx=0).stop 2024-02-17T12:07:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.003+0000 7f21fc498640 1 -- 172.21.15.94:0/3739062297 shutdown_connections 2024-02-17T12:07:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.003+0000 7f21fc498640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f21e00a2430 0x7f21e00a48f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.003+0000 7f21fc498640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f21f419c020 0x7f21f419c4d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:55.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.003+0000 7f21fc498640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21f4106460 0x7f21f4197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.003+0000 7f21fc498640 1 --2- 172.21.15.94:0/3739062297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f21f4101390 0x7f21f4197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.003+0000 7f21fc498640 1 -- 172.21.15.94:0/3739062297 >> 172.21.15.94:0/3739062297 conn(0x7f21f40fcc10 msgr2=0x7f21f40fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.003+0000 7f21fc498640 1 -- 172.21.15.94:0/3739062297 shutdown_connections 2024-02-17T12:07:55.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:07:55.003+0000 7f21fc498640 1 -- 172.21.15.94:0/3739062297 wait complete. 2024-02-17T12:07:55.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 575.0 s, 1597.8 tps, lat 63.053 ms stddev 64.821 2024-02-17T12:07:56.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:55 smithi189 ceph-mon[36588]: pgmap v303: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 8.7 MiB/s wr, 949 op/s 2024-02-17T12:07:56.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:55 smithi125 ceph-mon[36819]: pgmap v303: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 8.7 MiB/s wr, 949 op/s 2024-02-17T12:07:56.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:55 smithi094 ceph-mon[32499]: pgmap v303: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 8.7 MiB/s wr, 949 op/s 2024-02-17T12:07:57.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:56 smithi189 ceph-mon[36588]: pgmap v304: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.9 MiB/s wr, 609 op/s 2024-02-17T12:07:57.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:56 smithi125 ceph-mon[36819]: pgmap v304: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.9 MiB/s wr, 609 op/s 2024-02-17T12:07:57.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:56 smithi094 ceph-mon[32499]: pgmap v304: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 4.9 MiB/s wr, 609 op/s 2024-02-17T12:08:00.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 580.0 s, 1565.6 tps, lat 63.772 ms stddev 66.301 2024-02-17T12:08:00.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:07:59 smithi189 ceph-mon[36588]: pgmap v305: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 7.5 MiB/s wr, 890 op/s 2024-02-17T12:08:00.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:07:59 smithi125 ceph-mon[36819]: pgmap v305: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 7.5 MiB/s wr, 890 op/s 2024-02-17T12:08:00.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:07:59 smithi094 ceph-mon[32499]: pgmap v305: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 7.5 MiB/s wr, 890 op/s 2024-02-17T12:08:01.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:00 smithi189 ceph-mon[36588]: pgmap v306: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.0 MiB/s wr, 631 op/s 2024-02-17T12:08:01.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:01 smithi125 ceph-mon[36819]: pgmap v306: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.0 MiB/s wr, 631 op/s 2024-02-17T12:08:01.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:00 smithi094 ceph-mon[32499]: pgmap v306: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.0 MiB/s wr, 631 op/s 2024-02-17T12:08:04.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:03 smithi189 ceph-mon[36588]: pgmap v307: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.0 MiB/s wr, 631 op/s 2024-02-17T12:08:04.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:03 smithi125 ceph-mon[36819]: pgmap v307: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.0 MiB/s wr, 631 op/s 2024-02-17T12:08:04.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:03 smithi094 ceph-mon[32499]: pgmap v307: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.0 MiB/s wr, 631 op/s 2024-02-17T12:08:05.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 585.0 s, 706.2 tps, lat 140.402 ms stddev 207.021 2024-02-17T12:08:05.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:05 smithi189 ceph-mon[36588]: pgmap v308: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 8.4 MiB/s wr, 1.01k op/s 2024-02-17T12:08:05.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:05 smithi125 ceph-mon[36819]: pgmap v308: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 8.4 MiB/s wr, 1.01k op/s 2024-02-17T12:08:05.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:05 smithi094 ceph-mon[32499]: pgmap v308: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 8.4 MiB/s wr, 1.01k op/s 2024-02-17T12:08:07.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:07 smithi189 ceph-mon[36588]: pgmap v309: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 661 op/s 2024-02-17T12:08:07.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:07 smithi125 ceph-mon[36819]: pgmap v309: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 661 op/s 2024-02-17T12:08:07.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:07 smithi094 ceph-mon[32499]: pgmap v309: 97 pgs: 97 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 6.0 MiB/s wr, 661 op/s 2024-02-17T12:08:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.151+0000 7fd2d2b9f640 1 -- 172.21.15.94:0/972761374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2cc111f70 msgr2=0x7fd2cc114360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.151+0000 7fd2d2b9f640 1 --2- 172.21.15.94:0/972761374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2cc111f70 0x7fd2cc114360 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4009ca0 tx=0x7fd2c402c1c0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.151+0000 7fd2d2b9f640 1 -- 172.21.15.94:0/972761374 shutdown_connections 2024-02-17T12:08:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.151+0000 7fd2d2b9f640 1 --2- 172.21.15.94:0/972761374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2cc111f70 0x7fd2cc114360 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.151+0000 7fd2d2b9f640 1 --2- 172.21.15.94:0/972761374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2cc073e40 0x7fd2cc111a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.151+0000 7fd2d2b9f640 1 --2- 172.21.15.94:0/972761374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2cc073520 0x7fd2cc073900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.152+0000 7fd2d2b9f640 1 -- 172.21.15.94:0/972761374 >> 172.21.15.94:0/972761374 conn(0x7fd2cc0fcab0 msgr2=0x7fd2cc0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.152+0000 7fd2d2b9f640 1 -- 172.21.15.94:0/972761374 shutdown_connections 2024-02-17T12:08:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.152+0000 7fd2d2b9f640 1 -- 172.21.15.94:0/972761374 wait complete. 2024-02-17T12:08:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.153+0000 7fd2d2b9f640 1 Processor -- start 2024-02-17T12:08:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.159+0000 7fd2d2b9f640 1 -- start start 2024-02-17T12:08:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.163+0000 7fd2d2b9f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2cc073520 0x7fd2cc1a37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.163+0000 7fd2d2b9f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2cc073e40 0x7fd2cc1a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.163+0000 7fd2d2b9f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2cc111f70 0x7fd2cc19d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.163+0000 7fd2d2b9f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc19dfa0 con 0x7fd2cc111f70 2024-02-17T12:08:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.163+0000 7fd2d2b9f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc19e0e0 con 0x7fd2cc073520 2024-02-17T12:08:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.163+0000 7fd2d2b9f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc19e2a0 con 0x7fd2cc073e40 2024-02-17T12:08:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.164+0000 7fd2d1b9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2cc073520 0x7fd2cc1a37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.164+0000 7fd2d1b9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2cc073520 0x7fd2cc1a37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:39588/0 (socket says 172.21.15.94:39588) 2024-02-17T12:08:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.164+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/1807491273 learned_addr learned my addr 172.21.15.94:0/1807491273 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.164+0000 7fd2d139c640 1 --2- 172.21.15.94:0/1807491273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2cc073e40 0x7fd2cc1a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.164+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/1807491273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2cc073e40 msgr2=0x7fd2cc1a3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.164+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/1807491273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2cc073e40 0x7fd2cc1a3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.164+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/1807491273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2cc111f70 msgr2=0x7fd2cc19d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.164+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/1807491273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2cc111f70 0x7fd2cc19d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.164+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/1807491273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c4009900 con 0x7fd2cc073520 2024-02-17T12:08:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.166+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/1807491273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2cc073520 0x7fd2cc1a37a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800d980 tx=0x7fd2c800dd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.166+0000 7fd2c2ffd640 1 -- 172.21.15.94:0/1807491273 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8009c30 con 0x7fd2cc073520 2024-02-17T12:08:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.166+0000 7fd2c2ffd640 1 -- 172.21.15.94:0/1807491273 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2c800bea0 con 0x7fd2cc073520 2024-02-17T12:08:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.166+0000 7fd2c2ffd640 1 -- 172.21.15.94:0/1807491273 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c800f690 con 0x7fd2cc073520 2024-02-17T12:08:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.167+0000 7fd2d2b9f640 1 -- 172.21.15.94:0/1807491273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc19e580 con 0x7fd2cc073520 2024-02-17T12:08:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.167+0000 7fd2d2b9f640 1 -- 172.21.15.94:0/1807491273 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc06ad80 con 0x7fd2cc073520 2024-02-17T12:08:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.170+0000 7fd2c2ffd640 1 -- 172.21.15.94:0/1807491273 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd2c80109d0 con 0x7fd2cc073520 2024-02-17T12:08:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.173+0000 7fd2d69cb740 1 Processor -- start 2024-02-17T12:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.174+0000 7fd2d69cb740 1 -- start start 2024-02-17T12:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.174+0000 7fd2d69cb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c495c60 0x55e31c4949b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.174+0000 7fd2d69cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c494ef0 0x55e31c495370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.174+0000 7fd2d239e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c495c60 0x55e31c4949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.175+0000 7fd2d239e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c495c60 0x55e31c4949b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:39594/0 (socket says 172.21.15.94:39594) 2024-02-17T12:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.175+0000 7fd2d239e640 1 -- 172.21.15.94:0/301317330 learned_addr learned my addr 172.21.15.94:0/301317330 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.175+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/301317330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c494ef0 0x55e31c495370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.175+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/301317330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c49e470 0x55e31c4a0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.175+0000 7fd2d69cb740 1 -- 172.21.15.94:0/301317330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31c4039f0 con 0x55e31c494ef0 2024-02-17T12:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.175+0000 7fd2d69cb740 1 -- 172.21.15.94:0/301317330 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31c400f20 con 0x55e31c495c60 2024-02-17T12:08:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.175+0000 7fd2d69cb740 1 -- 172.21.15.94:0/301317330 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31c3fc700 con 0x55e31c49e470 2024-02-17T12:08:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.176+0000 7fd2c2ffd640 1 --2- 172.21.15.94:0/1807491273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd2b80a2430 0x7fd2b80a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.176+0000 7fd2c2ffd640 1 -- 172.21.15.94:0/1807491273 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fd2c80cdd60 con 0x7fd2cc073520 2024-02-17T12:08:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.178+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/301317330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c49e470 msgr2=0x55e31c4a0860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:08:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.178+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/301317330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c49e470 0x55e31c4a0860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.178+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/301317330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c495c60 msgr2=0x55e31c4949b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.178+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/301317330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c495c60 0x55e31c4949b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.178+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/301317330 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e31c3927b0 con 0x55e31c494ef0 2024-02-17T12:08:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.178+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/301317330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c494ef0 0x55e31c495370 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80cb3e0 tx=0x7fd2c8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33944553c858487 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.179+0000 7fd2d239e640 1 --2- 172.21.15.94:0/1807491273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd2b80a2430 0x7fd2b80a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.182+0000 7fd2b7fff640 1 -- 172.21.15.94:0/301317330 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c809a700 con 0x55e31c494ef0 2024-02-17T12:08:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.182+0000 7fd2b7fff640 1 -- 172.21.15.94:0/301317330 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2c809e850 con 0x55e31c494ef0 2024-02-17T12:08:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.182+0000 7fd2b7fff640 1 -- 172.21.15.94:0/301317330 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80cbb30 con 0x55e31c494ef0 2024-02-17T12:08:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.184+0000 7fd2d69cb740 1 -- 172.21.15.94:0/301317330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c494ef0 msgr2=0x55e31c495370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.184+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/301317330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c494ef0 0x55e31c495370 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80cb3e0 tx=0x7fd2c8000bc0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.188+0000 7fd2d69cb740 1 -- 172.21.15.94:0/301317330 shutdown_connections 2024-02-17T12:08:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.188+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/301317330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c49e470 0x55e31c4a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.188+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/301317330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c494ef0 0x55e31c495370 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.188+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/301317330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c495c60 0x55e31c4949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.188+0000 7fd2d69cb740 1 -- 172.21.15.94:0/301317330 >> 172.21.15.94:0/301317330 conn(0x55e31c45c550 msgr2=0x55e31c48fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:08.199 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.196+0000 7fd2d69cb740 1 -- 172.21.15.94:0/301317330 shutdown_connections 2024-02-17T12:08:08.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.199+0000 7fd2d239e640 1 --2- 172.21.15.94:0/1807491273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd2b80a2430 0x7fd2b80a48f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd2c402c7d0 tx=0x7fd2c402c6e0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.200+0000 7fd2d69cb740 1 -- 172.21.15.94:0/301317330 wait complete. 2024-02-17T12:08:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.200+0000 7fd2d69cb740 1 -- start start 2024-02-17T12:08:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.200+0000 7fd2d69cb740 1 Processor -- start 2024-02-17T12:08:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.202+0000 7fd2d69cb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c49b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.202+0000 7fd2d69cb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 0x55e31c49ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.203+0000 7fd2d69cb740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 0x55e31c49bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.203+0000 7fd2d1b9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c49b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.203+0000 7fd2d1b9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c49b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:39604/0 (socket says 172.21.15.94:39604) 2024-02-17T12:08:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.203+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/847671860 learned_addr learned my addr 172.21.15.94:0/847671860 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.203+0000 7fd2d69cb740 1 -- 172.21.15.94:0/847671860 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31c3a1bf0 con 0x55e31c49e470 2024-02-17T12:08:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.203+0000 7fd2d69cb740 1 -- 172.21.15.94:0/847671860 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31c0d0250 con 0x55e31c494ef0 2024-02-17T12:08:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.203+0000 7fd2d69cb740 1 -- 172.21.15.94:0/847671860 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31c382200 con 0x55e31c495c60 2024-02-17T12:08:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.203+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/847671860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 msgr2=0x55e31c49ba00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:08:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.203+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/847671860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 0x55e31c49ba00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.203+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/847671860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 msgr2=0x55e31c49bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.203+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/847671860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 0x55e31c49bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.204+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/847671860 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc06ad80 con 0x55e31c494ef0 2024-02-17T12:08:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.204+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/847671860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c49b4c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd2c809a690 tx=0x7fd2c80a2ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51e48cf0a031299e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.204+0000 7fd2d69cb740 10 client.0 _ll_register_callbacks cb 0x55e31c3f0d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.204+0000 7fd2b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:08:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.204+0000 7fd2b57fa640 1 -- 172.21.15.94:0/847671860 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80d0070 con 0x55e31c494ef0 2024-02-17T12:08:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.204+0000 7fd2b57fa640 1 -- 172.21.15.94:0/847671860 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fd2c80ba200 con 0x55e31c494ef0 2024-02-17T12:08:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.204+0000 7fd2d69cb740 1 -- 172.21.15.94:0/847671860 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e31c0fc600 con 0x55e31c494ef0 2024-02-17T12:08:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.209+0000 7fd2b57fa640 1 -- 172.21.15.94:0/847671860 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2c809a780 con 0x55e31c494ef0 2024-02-17T12:08:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.209+0000 7fd2d69cb740 10 client.27605 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.209+0000 7fd2d69cb740 1 -- 172.21.15.94:0/847671860 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x55e31c3fad80 con 0x55e31c494ef0 2024-02-17T12:08:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.212+0000 7fd2b57fa640 1 -- 172.21.15.94:0/847671860 <== mon.2 v2:172.21.15.125:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd2c80cb660 con 0x55e31c494ef0 2024-02-17T12:08:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.212+0000 7fd2d69cb740 10 client.27605 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.212+0000 7fd2d69cb740 10 client.27605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.212+0000 7fd2d69cb740 20 client.27605 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.212+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/847671860 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e31c4adfc0 0x55e31c4b0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.212+0000 7fd2d69cb740 4 client.27605 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.212+0000 7fd2d69cb740 1 -- 172.21.15.94:0/847671860 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e31c0fc600 con 0x55e31c4adfc0 2024-02-17T12:08:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.213+0000 7fd2d139c640 1 --2- 172.21.15.94:0/847671860 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e31c4adfc0 0x55e31c4b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.218+0000 7fd2d139c640 1 --2- 172.21.15.94:0/847671860 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e31c4adfc0 0x55e31c4b0420 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440f58ec06f1c4a1 server_cookie=205c449b11c7ca4c in_seq=0 out_seq=0 2024-02-17T12:08:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.218+0000 7fd2b57fa640 10 client.27605 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.222+0000 7fd2b57fa640 1 -- 172.21.15.94:0/847671860 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e31c0fc600 con 0x55e31c4adfc0 2024-02-17T12:08:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.222+0000 7fd2b57fa640 10 client.27605 handle_command_reply: tid=0 2024-02-17T12:08:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.222+0000 7fd2d69cb740 1 client.27605 shutdown 2024-02-17T12:08:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.231+0000 7fd2d69cb740 1 -- 172.21.15.94:0/847671860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 msgr2=0x55e31c49b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:08.232 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.231+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/847671860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c49b4c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd2c809a690 tx=0x7fd2c80a2ea0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.233+0000 7fd2d69cb740 1 -- 172.21.15.94:0/847671860 shutdown_connections 2024-02-17T12:08:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.233+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/847671860 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e31c4adfc0 0x55e31c4b0420 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.233+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/847671860 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 0x55e31c49bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.233+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/847671860 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 0x55e31c49ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.233+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/847671860 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c49b4c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.233+0000 7fd2d69cb740 1 -- 172.21.15.94:0/847671860 >> 172.21.15.94:0/847671860 conn(0x55e31c45c550 msgr2=0x55e31c491930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.233+0000 7fd2b57fa640 0 client.27605 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.236+0000 7fd2d69cb740 1 -- 172.21.15.94:0/847671860 shutdown_connections 2024-02-17T12:08:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.238+0000 7fd2d69cb740 1 -- 172.21.15.94:0/847671860 wait complete. 2024-02-17T12:08:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.238+0000 7fd2d69cb740 20 client.27605 trim_cache size 0 max 16384 2024-02-17T12:08:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.266+0000 7fd2d69cb740 1 Processor -- start 2024-02-17T12:08:08.267 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.266+0000 7fd2d69cb740 1 -- start start 2024-02-17T12:08:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.266+0000 7fd2d69cb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c49dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.267+0000 7fd2d69cb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 0x55e31c49bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.267+0000 7fd2d1b9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c49dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.268 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.267+0000 7fd2d1b9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c49dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:39618/0 (socket says 172.21.15.94:39618) 2024-02-17T12:08:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.267+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/3343381360 learned_addr learned my addr 172.21.15.94:0/3343381360 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.267+0000 7fd2d139c640 1 --2- 172.21.15.94:0/3343381360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 0x55e31c49bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.267+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/3343381360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 0x55e31c498740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.267+0000 7fd2d69cb740 1 -- 172.21.15.94:0/3343381360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31c4aa0d0 con 0x55e31c49e470 2024-02-17T12:08:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.267+0000 7fd2d69cb740 1 -- 172.21.15.94:0/3343381360 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31c4ab120 con 0x55e31c494ef0 2024-02-17T12:08:08.269 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.267+0000 7fd2d69cb740 1 -- 172.21.15.94:0/3343381360 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31c490be0 con 0x55e31c495c60 2024-02-17T12:08:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.267+0000 7fd2d239e640 1 --2- 172.21.15.94:0/3343381360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 0x55e31c498740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.268+0000 7fd2d239e640 1 -- 172.21.15.94:0/3343381360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 msgr2=0x55e31c49bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.268+0000 7fd2d239e640 1 --2- 172.21.15.94:0/3343381360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 0x55e31c49bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.268+0000 7fd2d239e640 1 -- 172.21.15.94:0/3343381360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 msgr2=0x55e31c49dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.268+0000 7fd2d239e640 1 --2- 172.21.15.94:0/3343381360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c49dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.268+0000 7fd2d239e640 1 -- 172.21.15.94:0/3343381360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e31c3977b0 con 0x55e31c49e470 2024-02-17T12:08:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.268+0000 7fd2d239e640 1 --2- 172.21.15.94:0/3343381360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 0x55e31c498740 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4035fd0 tx=0x7fd2c4009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a3a4a8aa94634 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.269+0000 7fd2b77fe640 1 -- 172.21.15.94:0/3343381360 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4003d80 con 0x55e31c49e470 2024-02-17T12:08:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.269+0000 7fd2b77fe640 1 -- 172.21.15.94:0/3343381360 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2c40325b0 con 0x55e31c49e470 2024-02-17T12:08:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.269+0000 7fd2b77fe640 1 -- 172.21.15.94:0/3343381360 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4004610 con 0x55e31c49e470 2024-02-17T12:08:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.269+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/3343381360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c49dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:08:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.269+0000 7fd2d139c640 1 --2- 172.21.15.94:0/3343381360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 0x55e31c49bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:08:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.270+0000 7fd2d69cb740 1 -- 172.21.15.94:0/3343381360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 msgr2=0x55e31c498740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.271+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/3343381360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 0x55e31c498740 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4035fd0 tx=0x7fd2c4009540 comp rx=0 tx=0).stop 2024-02-17T12:08:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.272+0000 7fd2d69cb740 1 -- 172.21.15.94:0/3343381360 shutdown_connections 2024-02-17T12:08:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.272+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/3343381360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 0x55e31c498740 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.272+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/3343381360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 0x55e31c49bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.272+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/3343381360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c49dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.272+0000 7fd2d69cb740 1 -- 172.21.15.94:0/3343381360 >> 172.21.15.94:0/3343381360 conn(0x55e31c45c550 msgr2=0x55e31c499d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.275+0000 7fd2d69cb740 1 -- 172.21.15.94:0/3343381360 shutdown_connections 2024-02-17T12:08:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.275+0000 7fd2d69cb740 1 -- 172.21.15.94:0/3343381360 wait complete. 2024-02-17T12:08:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.276+0000 7fd2d69cb740 1 -- start start 2024-02-17T12:08:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.276+0000 7fd2d69cb740 1 Processor -- start 2024-02-17T12:08:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.276+0000 7fd2d69cb740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c4a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.277+0000 7fd2d69cb740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 0x55e31c4a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.277+0000 7fd2d1b9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c4a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.277+0000 7fd2d1b9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c4a3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:39628/0 (socket says 172.21.15.94:39628) 2024-02-17T12:08:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.277+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/2261666539 learned_addr learned my addr 172.21.15.94:0/2261666539 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.277+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 0x55e31c4a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.277+0000 7fd2d139c640 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 0x55e31c4a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.277+0000 7fd2d69cb740 1 -- 172.21.15.94:0/2261666539 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31c504680 con 0x55e31c49e470 2024-02-17T12:08:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.277+0000 7fd2d69cb740 1 -- 172.21.15.94:0/2261666539 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31c5047f0 con 0x55e31c494ef0 2024-02-17T12:08:08.279 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.277+0000 7fd2d69cb740 1 -- 172.21.15.94:0/2261666539 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31c5049b0 con 0x55e31c495c60 2024-02-17T12:08:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.278+0000 7fd2d69cb740 10 client.0 _ll_register_callbacks cb 0x7fd2c4037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.278+0000 7fd2d239e640 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 0x55e31c4a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.279+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/2261666539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 msgr2=0x55e31c4a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.279+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 0x55e31c4a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.279+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/2261666539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 msgr2=0x55e31c4a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.279+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 0x55e31c4a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.280+0000 7fd2d1b9d640 1 -- 172.21.15.94:0/2261666539 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc114c90 con 0x55e31c494ef0 2024-02-17T12:08:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.282+0000 7fd2d1b9d640 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c4a3d90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80a2970 tx=0x7fd2c8010d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31fff8a470692f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:08.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.282+0000 7fd2b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:08:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.285+0000 7fd2b57fa640 1 -- 172.21.15.94:0/2261666539 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80b3230 con 0x55e31c494ef0 2024-02-17T12:08:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.285+0000 7fd2d69cb740 1 -- 172.21.15.94:0/2261666539 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2c809e850 con 0x55e31c494ef0 2024-02-17T12:08:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.285+0000 7fd2b57fa640 1 -- 172.21.15.94:0/2261666539 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fd2c809a700 con 0x55e31c494ef0 2024-02-17T12:08:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.285+0000 7fd2b57fa640 1 -- 172.21.15.94:0/2261666539 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2c80d1070 con 0x55e31c494ef0 2024-02-17T12:08:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.287+0000 7fd2d69cb740 10 client.27608 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.287+0000 7fd2d69cb740 1 -- 172.21.15.94:0/2261666539 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e31c3bd540 con 0x55e31c494ef0 2024-02-17T12:08:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.288+0000 7fd2b57fa640 1 -- 172.21.15.94:0/2261666539 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd2c80ac060 con 0x55e31c494ef0 2024-02-17T12:08:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.288+0000 7fd2b57fa640 1 -- 172.21.15.94:0/2261666539 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd2c80cbb30 con 0x55e31c494ef0 2024-02-17T12:08:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.288+0000 7fd2d69cb740 10 client.27608 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.288+0000 7fd2d69cb740 10 client.27608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.288+0000 7fd2d69cb740 20 client.27608 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.288+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e31c505c30 0x55e31c508020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.288+0000 7fd2d69cb740 4 client.27608 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:08:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.288+0000 7fd2d69cb740 1 -- 172.21.15.94:0/2261666539 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fd2c809e850 con 0x55e31c505c30 2024-02-17T12:08:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.288+0000 7fd2d139c640 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e31c505c30 0x55e31c508020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.291+0000 7fd2d139c640 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e31c505c30 0x55e31c508020 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea3326695f74d95d server_cookie=6fff1617e80f384 in_seq=0 out_seq=0 2024-02-17T12:08:08.294 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.292+0000 7fd2b57fa640 10 client.27608 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:08.527 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.525+0000 7fd2b57fa640 1 -- 172.21.15.94:0/2261666539 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fd2c809e850 con 0x55e31c505c30 2024-02-17T12:08:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.525+0000 7fd2b57fa640 10 client.27608 handle_command_reply: tid=0 2024-02-17T12:08:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.525+0000 7fd2d69cb740 1 client.27608 shutdown 2024-02-17T12:08:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.525+0000 7fd2d69cb740 1 -- 172.21.15.94:0/2261666539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 msgr2=0x55e31c4a3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.525+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c4a3d90 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80a2970 tx=0x7fd2c8010d70 comp rx=0 tx=0).stop 2024-02-17T12:08:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.525+0000 7fd2d69cb740 1 -- 172.21.15.94:0/2261666539 shutdown_connections 2024-02-17T12:08:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.525+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55e31c505c30 0x55e31c508020 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.525+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e31c49e470 0x55e31c4a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.525+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e31c495c60 0x55e31c4a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.528 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.525+0000 7fd2d69cb740 1 --2- 172.21.15.94:0/2261666539 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e31c494ef0 0x55e31c4a3d90 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.526+0000 7fd2d69cb740 1 -- 172.21.15.94:0/2261666539 >> 172.21.15.94:0/2261666539 conn(0x55e31c45c550 msgr2=0x55e31c48e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.526+0000 7fd2b57fa640 0 client.27608 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.526+0000 7fd2d69cb740 1 -- 172.21.15.94:0/2261666539 shutdown_connections 2024-02-17T12:08:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.526+0000 7fd2d69cb740 1 -- 172.21.15.94:0/2261666539 wait complete. 2024-02-17T12:08:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.526+0000 7fd2d69cb740 20 client.27608 trim_cache size 0 max 16384 2024-02-17T12:08:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:08:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:08:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:08:08.529 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:08:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.527+0000 7fd2d2b9f640 1 -- 172.21.15.94:0/1807491273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd2b80a2430 msgr2=0x7fd2b80a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.527+0000 7fd2d2b9f640 1 --2- 172.21.15.94:0/1807491273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd2b80a2430 0x7fd2b80a48f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd2c402c7d0 tx=0x7fd2c402c6e0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.527+0000 7fd2d2b9f640 1 -- 172.21.15.94:0/1807491273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2cc073520 msgr2=0x7fd2cc1a37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.527+0000 7fd2d2b9f640 1 --2- 172.21.15.94:0/1807491273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2cc073520 0x7fd2cc1a37a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800d980 tx=0x7fd2c800dd80 comp rx=0 tx=0).stop 2024-02-17T12:08:08.530 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.527+0000 7fd2d2b9f640 1 -- 172.21.15.94:0/1807491273 shutdown_connections 2024-02-17T12:08:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.527+0000 7fd2d2b9f640 1 --2- 172.21.15.94:0/1807491273 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd2b80a2430 0x7fd2b80a48f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.527+0000 7fd2d2b9f640 1 --2- 172.21.15.94:0/1807491273 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd2cc111f70 0x7fd2cc19d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.527+0000 7fd2d2b9f640 1 --2- 172.21.15.94:0/1807491273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2cc073e40 0x7fd2cc1a3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.527+0000 7fd2d2b9f640 1 --2- 172.21.15.94:0/1807491273 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd2cc073520 0x7fd2cc1a37a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.527+0000 7fd2d2b9f640 1 -- 172.21.15.94:0/1807491273 >> 172.21.15.94:0/1807491273 conn(0x7fd2cc0fcab0 msgr2=0x7fd2cc0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.527+0000 7fd2d2b9f640 1 -- 172.21.15.94:0/1807491273 shutdown_connections 2024-02-17T12:08:08.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:08.527+0000 7fd2d2b9f640 1 -- 172.21.15.94:0/1807491273 wait complete. 2024-02-17T12:08:09.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:09 smithi189 ceph-mon[36588]: osdmap e95: 12 total, 12 up, 12 in 2024-02-17T12:08:09.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:09 smithi189 ceph-mon[36588]: pgmap v311: 97 pgs: 28 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 53 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 19 MiB/s wr, 1.73k op/s 2024-02-17T12:08:09.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:09 smithi125 ceph-mon[36819]: osdmap e95: 12 total, 12 up, 12 in 2024-02-17T12:08:09.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:09 smithi125 ceph-mon[36819]: pgmap v311: 97 pgs: 28 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 53 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 19 MiB/s wr, 1.73k op/s 2024-02-17T12:08:09.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:09 smithi094 ceph-mon[32499]: osdmap e95: 12 total, 12 up, 12 in 2024-02-17T12:08:09.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:09 smithi094 ceph-mon[32499]: pgmap v311: 97 pgs: 28 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 53 active+clean; 5.4 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 19 MiB/s wr, 1.73k op/s 2024-02-17T12:08:10.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 590.0 s, 1596.6 tps, lat 62.828 ms stddev 92.135 2024-02-17T12:08:12.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:11 smithi189 ceph-mon[36588]: pgmap v312: 97 pgs: 28 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 53 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 1.82k op/s 2024-02-17T12:08:12.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:11 smithi125 ceph-mon[36819]: pgmap v312: 97 pgs: 28 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 53 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 1.82k op/s 2024-02-17T12:08:12.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:11 smithi094 ceph-mon[32499]: pgmap v312: 97 pgs: 28 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 53 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 1.82k op/s 2024-02-17T12:08:13.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:13 smithi189 ceph-mon[36588]: pgmap v313: 97 pgs: 28 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 53 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 1.82k op/s 2024-02-17T12:08:13.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:13 smithi125 ceph-mon[36819]: pgmap v313: 97 pgs: 28 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 53 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 1.82k op/s 2024-02-17T12:08:13.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:13 smithi094 ceph-mon[32499]: pgmap v313: 97 pgs: 28 active+clean+snaptrim_wait, 16 active+clean+snaptrim, 53 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 1.82k op/s 2024-02-17T12:08:15.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 595.0 s, 1747.6 tps, lat 57.703 ms stddev 65.045 2024-02-17T12:08:16.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:15 smithi189 ceph-mon[36588]: pgmap v314: 97 pgs: 97 active+clean; 5.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 24 MiB/s wr, 1.76k op/s 2024-02-17T12:08:16.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:15 smithi189 ceph-mon[36588]: osdmap e96: 12 total, 12 up, 12 in 2024-02-17T12:08:16.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:15 smithi125 ceph-mon[36819]: pgmap v314: 97 pgs: 97 active+clean; 5.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 24 MiB/s wr, 1.76k op/s 2024-02-17T12:08:16.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:15 smithi125 ceph-mon[36819]: osdmap e96: 12 total, 12 up, 12 in 2024-02-17T12:08:16.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:15 smithi094 ceph-mon[32499]: pgmap v314: 97 pgs: 97 active+clean; 5.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 24 MiB/s wr, 1.76k op/s 2024-02-17T12:08:16.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:15 smithi094 ceph-mon[32499]: osdmap e96: 12 total, 12 up, 12 in 2024-02-17T12:08:17.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:17 smithi189 ceph-mon[36588]: pgmap v316: 97 pgs: 97 active+clean; 5.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 24 MiB/s wr, 1.68k op/s 2024-02-17T12:08:17.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:17 smithi125 ceph-mon[36819]: pgmap v316: 97 pgs: 97 active+clean; 5.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 24 MiB/s wr, 1.68k op/s 2024-02-17T12:08:17.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:17 smithi094 ceph-mon[32499]: pgmap v316: 97 pgs: 97 active+clean; 5.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 24 MiB/s wr, 1.68k op/s 2024-02-17T12:08:19.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:19 smithi189 ceph-mon[36588]: pgmap v317: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 782 op/s 2024-02-17T12:08:19.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:19 smithi125 ceph-mon[36819]: pgmap v317: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 782 op/s 2024-02-17T12:08:19.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:19 smithi094 ceph-mon[32499]: pgmap v317: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 782 op/s 2024-02-17T12:08:19.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:08:19.666 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:08:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.840+0000 7fec8269d640 1 -- 172.21.15.94:0/221132780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec7c11a960 msgr2=0x7fec7c11cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:19.842 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.840+0000 7fec8269d640 1 --2- 172.21.15.94:0/221132780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec7c11a960 0x7fec7c11cd50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009900 tx=0x7fec6c02bc80 comp rx=0 tx=0).stop 2024-02-17T12:08:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.845+0000 7fec8269d640 1 -- 172.21.15.94:0/221132780 shutdown_connections 2024-02-17T12:08:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.845+0000 7fec8269d640 1 --2- 172.21.15.94:0/221132780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec7c11a960 0x7fec7c11cd50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.845+0000 7fec8269d640 1 --2- 172.21.15.94:0/221132780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec7c10fd10 0x7fec7c10da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.845+0000 7fec8269d640 1 --2- 172.21.15.94:0/221132780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec7c10f8a0 0x7fec7c10d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.845+0000 7fec8269d640 1 -- 172.21.15.94:0/221132780 >> 172.21.15.94:0/221132780 conn(0x7fec7c06c270 msgr2=0x7fec7c06c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:19.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.851+0000 7fec8269d640 1 -- 172.21.15.94:0/221132780 shutdown_connections 2024-02-17T12:08:19.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.855+0000 7fec8269d640 1 -- 172.21.15.94:0/221132780 wait complete. 2024-02-17T12:08:19.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.855+0000 7fec8269d640 1 Processor -- start 2024-02-17T12:08:19.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.858+0000 7fec8269d640 1 -- start start 2024-02-17T12:08:19.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.858+0000 7fec8269d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec7c10f8a0 0x7fec7c119d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.858+0000 7fec8269d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec7c10fd10 0x7fec7c11a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.859+0000 7fec8269d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec7c11a960 0x7fec7c112e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.859+0000 7fec8269d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c113620 con 0x7fec7c10f8a0 2024-02-17T12:08:19.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.859+0000 7fec7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec7c10f8a0 0x7fec7c119d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.859+0000 7fec7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec7c10f8a0 0x7fec7c119d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46946/0 (socket says 172.21.15.94:46946) 2024-02-17T12:08:19.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.859+0000 7fec7bfff640 1 -- 172.21.15.94:0/1935546114 learned_addr learned my addr 172.21.15.94:0/1935546114 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:19.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.859+0000 7fec8269d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c113790 con 0x7fec7c11a960 2024-02-17T12:08:19.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.859+0000 7fec8269d640 1 -- 172.21.15.94:0/1935546114 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c113950 con 0x7fec7c10fd10 2024-02-17T12:08:19.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.859+0000 7fec7bfff640 1 -- 172.21.15.94:0/1935546114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec7c10fd10 msgr2=0x7fec7c11a2d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:08:19.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.859+0000 7fec7bfff640 1 --2- 172.21.15.94:0/1935546114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec7c10fd10 0x7fec7c11a2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.859+0000 7fec7bfff640 1 -- 172.21.15.94:0/1935546114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec7c11a960 msgr2=0x7fec7c112e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:08:19.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.859+0000 7fec7bfff640 1 --2- 172.21.15.94:0/1935546114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec7c11a960 0x7fec7c112e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.859+0000 7fec7bfff640 1 -- 172.21.15.94:0/1935546114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c009560 con 0x7fec7c10f8a0 2024-02-17T12:08:19.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.860+0000 7fec7bfff640 1 --2- 172.21.15.94:0/1935546114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec7c10f8a0 0x7fec7c119d90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fec7000e9f0 tx=0x7fec7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:19.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.860+0000 7fec797fa640 1 -- 172.21.15.94:0/1935546114 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70009700 con 0x7fec7c10f8a0 2024-02-17T12:08:19.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.860+0000 7fec797fa640 1 -- 172.21.15.94:0/1935546114 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec7000cd80 con 0x7fec7c10f8a0 2024-02-17T12:08:19.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.860+0000 7fec797fa640 1 -- 172.21.15.94:0/1935546114 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70018880 con 0x7fec7c10f8a0 2024-02-17T12:08:19.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.860+0000 7fec8269d640 1 -- 172.21.15.94:0/1935546114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c113c30 con 0x7fec7c10f8a0 2024-02-17T12:08:19.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.860+0000 7fec8269d640 1 -- 172.21.15.94:0/1935546114 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c117be0 con 0x7fec7c10f8a0 2024-02-17T12:08:19.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.864+0000 7fec797fa640 1 -- 172.21.15.94:0/1935546114 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fec700189e0 con 0x7fec7c10f8a0 2024-02-17T12:08:19.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.865+0000 7fec797fa640 1 --2- 172.21.15.94:0/1935546114 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec640a2440 0x7fec640a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.866+0000 7fec797fa640 1 -- 172.21.15.94:0/1935546114 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fec70014070 con 0x7fec7c10f8a0 2024-02-17T12:08:19.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.868+0000 7fec85274740 1 Processor -- start 2024-02-17T12:08:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.869+0000 7fec85274740 1 -- start start 2024-02-17T12:08:19.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.869+0000 7fec7b7fe640 1 --2- 172.21.15.94:0/1935546114 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec640a2440 0x7fec640a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.870+0000 7fec85274740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b32b0 0x558f037b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.870+0000 7fec85274740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b3bd0 0x558f037bc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.870+0000 7fec85274740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037bc7b0 0x558f037beba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.870+0000 7fec85274740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f03722280 con 0x558f037b32b0 2024-02-17T12:08:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.870+0000 7fec85274740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0371f7b0 con 0x558f037bc7b0 2024-02-17T12:08:19.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.870+0000 7fec85274740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0371af90 con 0x558f037b3bd0 2024-02-17T12:08:19.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.871+0000 7fec80c3a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037bc7b0 0x558f037beba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.871+0000 7fec80c3a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037bc7b0 0x558f037beba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34736/0 (socket says 172.21.15.94:34736) 2024-02-17T12:08:19.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.871+0000 7fec80c3a640 1 -- 172.21.15.94:0/1399535085 learned_addr learned my addr 172.21.15.94:0/1399535085 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:19.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.872+0000 7fec80c3a640 1 -- 172.21.15.94:0/1399535085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b3bd0 msgr2=0x558f037bc270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:08:19.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.872+0000 7fec80c3a640 1 --2- 172.21.15.94:0/1399535085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b3bd0 0x558f037bc270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.872+0000 7fec80c3a640 1 -- 172.21.15.94:0/1399535085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b32b0 msgr2=0x558f037b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:19.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.872+0000 7fec80c3a640 1 --2- 172.21.15.94:0/1399535085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b32b0 0x558f037b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.872+0000 7fec80c3a640 1 -- 172.21.15.94:0/1399535085 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f036d9570 con 0x558f037bc7b0 2024-02-17T12:08:19.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.872+0000 7fec7b7fe640 1 --2- 172.21.15.94:0/1935546114 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec640a2440 0x7fec640a4900 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x558f037b2d80 tx=0x7fec74009500 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:19.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.873+0000 7fec80c3a640 1 --2- 172.21.15.94:0/1399535085 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037bc7b0 0x558f037beba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fec6c02c210 tx=0x7fec6c004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fb4046d684e0462 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:19.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.874+0000 7fec627fc640 1 -- 172.21.15.94:0/1399535085 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c004740 con 0x558f037bc7b0 2024-02-17T12:08:19.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.874+0000 7fec627fc640 1 -- 172.21.15.94:0/1399535085 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec6c0048a0 con 0x558f037bc7b0 2024-02-17T12:08:19.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.875+0000 7fec85274740 1 -- 172.21.15.94:0/1399535085 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037bc7b0 msgr2=0x558f037beba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:19.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.875+0000 7fec85274740 1 --2- 172.21.15.94:0/1399535085 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037bc7b0 0x558f037beba0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fec6c02c210 tx=0x7fec6c004500 comp rx=0 tx=0).stop 2024-02-17T12:08:19.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.875+0000 7fec627fc640 1 -- 172.21.15.94:0/1399535085 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c034a70 con 0x558f037bc7b0 2024-02-17T12:08:19.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.875+0000 7fec85274740 1 -- 172.21.15.94:0/1399535085 shutdown_connections 2024-02-17T12:08:19.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.875+0000 7fec85274740 1 --2- 172.21.15.94:0/1399535085 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037bc7b0 0x558f037beba0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.875+0000 7fec85274740 1 --2- 172.21.15.94:0/1399535085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b3bd0 0x558f037bc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.875+0000 7fec85274740 1 --2- 172.21.15.94:0/1399535085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b32b0 0x558f037b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.875+0000 7fec85274740 1 -- 172.21.15.94:0/1399535085 >> 172.21.15.94:0/1399535085 conn(0x558f037ad990 msgr2=0x558f037add70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:19.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.876+0000 7fec85274740 1 -- 172.21.15.94:0/1399535085 shutdown_connections 2024-02-17T12:08:19.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.876+0000 7fec85274740 1 -- 172.21.15.94:0/1399535085 wait complete. 2024-02-17T12:08:19.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.876+0000 7fec85274740 1 -- start start 2024-02-17T12:08:19.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.877+0000 7fec85274740 1 Processor -- start 2024-02-17T12:08:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.877+0000 7fec85274740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037b32b0 0x558f037bc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.877+0000 7fec85274740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b3bd0 0x558f037b9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.877+0000 7fec85274740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037bc7b0 0x558f037b9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.877+0000 7fec85274740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f033ee430 con 0x558f037b3bd0 2024-02-17T12:08:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.877+0000 7fec85274740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f033aae00 con 0x558f037b32b0 2024-02-17T12:08:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.877+0000 7fec85274740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f0369ffd0 con 0x558f037bc7b0 2024-02-17T12:08:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.877+0000 7fec85274740 10 client.0 _ll_register_callbacks cb 0x558f036f5fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.877+0000 7fec7bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037b32b0 0x558f037bc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.877+0000 7fec7bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037b32b0 0x558f037bc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34744/0 (socket says 172.21.15.94:34744) 2024-02-17T12:08:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.877+0000 7fec7bfff640 1 -- 172.21.15.94:0/673166482 learned_addr learned my addr 172.21.15.94:0/673166482 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.877+0000 7fec80c3a640 1 --2- 172.21.15.94:0/673166482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037bc7b0 0x558f037b9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.878+0000 7fec7b7fe640 1 --2- 172.21.15.94:0/673166482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b3bd0 0x558f037b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.878+0000 7fec7bfff640 1 -- 172.21.15.94:0/673166482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037bc7b0 msgr2=0x558f037b9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.878+0000 7fec7bfff640 1 --2- 172.21.15.94:0/673166482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037bc7b0 0x558f037b9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.878+0000 7fec7bfff640 1 -- 172.21.15.94:0/673166482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b3bd0 msgr2=0x558f037b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.878+0000 7fec7bfff640 1 --2- 172.21.15.94:0/673166482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b3bd0 0x558f037b9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.878+0000 7fec7bfff640 1 -- 172.21.15.94:0/673166482 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6c038040 con 0x558f037b32b0 2024-02-17T12:08:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.878+0000 7fec7bfff640 1 --2- 172.21.15.94:0/673166482 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037b32b0 0x558f037bc210 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fec7009aa80 tx=0x7fec7009ad80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47a972586c7fa730 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.878+0000 7fec5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:08:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.878+0000 7fec5b7fe640 1 -- 172.21.15.94:0/673166482 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7009a3d0 con 0x558f037b32b0 2024-02-17T12:08:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.878+0000 7fec5b7fe640 1 -- 172.21.15.94:0/673166482 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fec700a3520 con 0x558f037b32b0 2024-02-17T12:08:19.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.878+0000 7fec85274740 1 -- 172.21.15.94:0/673166482 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f0341a7e0 con 0x558f037b32b0 2024-02-17T12:08:19.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.881+0000 7fec5b7fe640 1 -- 172.21.15.94:0/673166482 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec7009a530 con 0x558f037b32b0 2024-02-17T12:08:19.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.881+0000 7fec85274740 10 client.27617 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:19.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.881+0000 7fec85274740 1 -- 172.21.15.94:0/673166482 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558f03719610 con 0x558f037b32b0 2024-02-17T12:08:19.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.881+0000 7fec5b7fe640 1 -- 172.21.15.94:0/673166482 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec700a7020 con 0x558f037b32b0 2024-02-17T12:08:19.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.881+0000 7fec5b7fe640 1 -- 172.21.15.94:0/673166482 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fec700b9070 con 0x558f037b32b0 2024-02-17T12:08:19.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.882+0000 7fec85274740 10 client.27617 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:19.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.882+0000 7fec85274740 10 client.27617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:19.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.882+0000 7fec85274740 20 client.27617 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:19.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.882+0000 7fec85274740 1 --2- 172.21.15.94:0/673166482 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558f037ccbb0 0x558f037cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.882+0000 7fec85274740 4 client.27617 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:19.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.882+0000 7fec85274740 1 -- 172.21.15.94:0/673166482 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f03457720 con 0x558f037ccbb0 2024-02-17T12:08:19.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.882+0000 7fec7b7fe640 1 --2- 172.21.15.94:0/673166482 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558f037ccbb0 0x558f037cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.882+0000 7fec5b7fe640 10 client.27617 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:19.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.882+0000 7fec7b7fe640 1 --2- 172.21.15.94:0/673166482 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558f037ccbb0 0x558f037cefd0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d9965ba8d6e2a9 server_cookie=30252030ea3829b in_seq=0 out_seq=0 2024-02-17T12:08:19.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.883+0000 7fec5b7fe640 1 -- 172.21.15.94:0/673166482 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f03457720 con 0x558f037ccbb0 2024-02-17T12:08:19.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.883+0000 7fec5b7fe640 10 client.27617 handle_command_reply: tid=0 2024-02-17T12:08:19.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.883+0000 7fec85274740 1 client.27617 shutdown 2024-02-17T12:08:19.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.883+0000 7fec85274740 1 -- 172.21.15.94:0/673166482 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037b32b0 msgr2=0x558f037bc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:19.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.883+0000 7fec85274740 1 --2- 172.21.15.94:0/673166482 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037b32b0 0x558f037bc210 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fec7009aa80 tx=0x7fec7009ad80 comp rx=0 tx=0).stop 2024-02-17T12:08:19.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.884+0000 7fec85274740 1 -- 172.21.15.94:0/673166482 shutdown_connections 2024-02-17T12:08:19.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.884+0000 7fec85274740 1 --2- 172.21.15.94:0/673166482 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558f037ccbb0 0x558f037cefd0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.884+0000 7fec85274740 1 --2- 172.21.15.94:0/673166482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037bc7b0 0x558f037b9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.884+0000 7fec85274740 1 --2- 172.21.15.94:0/673166482 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b3bd0 0x558f037b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.884+0000 7fec85274740 1 --2- 172.21.15.94:0/673166482 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037b32b0 0x558f037bc210 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.884+0000 7fec85274740 1 -- 172.21.15.94:0/673166482 >> 172.21.15.94:0/673166482 conn(0x558f037ad990 msgr2=0x558f037af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.884+0000 7fec5b7fe640 0 client.27617 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.884+0000 7fec85274740 1 -- 172.21.15.94:0/673166482 shutdown_connections 2024-02-17T12:08:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.884+0000 7fec85274740 1 -- 172.21.15.94:0/673166482 wait complete. 2024-02-17T12:08:19.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.884+0000 7fec85274740 20 client.27617 trim_cache size 0 max 16384 2024-02-17T12:08:19.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.903+0000 7fec85274740 1 Processor -- start 2024-02-17T12:08:19.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.903+0000 7fec85274740 1 -- start start 2024-02-17T12:08:19.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.903+0000 7fec85274740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b32b0 0x558f037bbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.903+0000 7fec85274740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b3bd0 0x558f037d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.903+0000 7fec85274740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037bc7b0 0x558f037c3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec85274740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f037b23a0 con 0x558f037b32b0 2024-02-17T12:08:19.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec85274740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f037aeba0 con 0x558f037bc7b0 2024-02-17T12:08:19.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec85274740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f037b4420 con 0x558f037b3bd0 2024-02-17T12:08:19.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b32b0 0x558f037bbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec7bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b32b0 0x558f037bbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46968/0 (socket says 172.21.15.94:46968) 2024-02-17T12:08:19.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec7bfff640 1 -- 172.21.15.94:0/2171742465 learned_addr learned my addr 172.21.15.94:0/2171742465 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec80c3a640 1 --2- 172.21.15.94:0/2171742465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037bc7b0 0x558f037c3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec7b7fe640 1 --2- 172.21.15.94:0/2171742465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b3bd0 0x558f037d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec7bfff640 1 -- 172.21.15.94:0/2171742465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b3bd0 msgr2=0x558f037d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec7bfff640 1 --2- 172.21.15.94:0/2171742465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b3bd0 0x558f037d3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec7bfff640 1 -- 172.21.15.94:0/2171742465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037bc7b0 msgr2=0x558f037c3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec7bfff640 1 --2- 172.21.15.94:0/2171742465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037bc7b0 0x558f037c3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec7bfff640 1 -- 172.21.15.94:0/2171742465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f036b5ac0 con 0x558f037b32b0 2024-02-17T12:08:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec7b7fe640 1 --2- 172.21.15.94:0/2171742465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b3bd0 0x558f037d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:08:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec7bfff640 1 --2- 172.21.15.94:0/2171742465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b32b0 0x558f037bbd40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fec7000c6a0 tx=0x7fec7009b500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b924f2276cb3e0f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec61ffb640 1 -- 172.21.15.94:0/2171742465 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700cbf70 con 0x558f037b32b0 2024-02-17T12:08:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec61ffb640 1 -- 172.21.15.94:0/2171742465 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec700a7030 con 0x558f037b32b0 2024-02-17T12:08:19.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.904+0000 7fec61ffb640 1 -- 172.21.15.94:0/2171742465 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7009a4c0 con 0x558f037b32b0 2024-02-17T12:08:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.905+0000 7fec85274740 1 -- 172.21.15.94:0/2171742465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b32b0 msgr2=0x558f037bbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.905+0000 7fec85274740 1 --2- 172.21.15.94:0/2171742465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b32b0 0x558f037bbd40 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fec7000c6a0 tx=0x7fec7009b500 comp rx=0 tx=0).stop 2024-02-17T12:08:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.905+0000 7fec85274740 1 -- 172.21.15.94:0/2171742465 shutdown_connections 2024-02-17T12:08:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.905+0000 7fec85274740 1 --2- 172.21.15.94:0/2171742465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037bc7b0 0x558f037c3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.905+0000 7fec85274740 1 --2- 172.21.15.94:0/2171742465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b3bd0 0x558f037d3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.905+0000 7fec85274740 1 --2- 172.21.15.94:0/2171742465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037b32b0 0x558f037bbd40 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.905+0000 7fec85274740 1 -- 172.21.15.94:0/2171742465 >> 172.21.15.94:0/2171742465 conn(0x558f037ad990 msgr2=0x558f037b8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.905+0000 7fec85274740 1 -- 172.21.15.94:0/2171742465 shutdown_connections 2024-02-17T12:08:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.905+0000 7fec85274740 1 -- 172.21.15.94:0/2171742465 wait complete. 2024-02-17T12:08:19.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec85274740 1 -- start start 2024-02-17T12:08:19.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec85274740 1 Processor -- start 2024-02-17T12:08:19.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec85274740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b32b0 0x558f037c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec85274740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037b3bd0 0x558f037c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec85274740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037bc7b0 0x558f037c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec85274740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f037c2440 con 0x558f037bc7b0 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec80c3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037bc7b0 0x558f037c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec7b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037b3bd0 0x558f037c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec80c3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037bc7b0 0x558f037c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:46984/0 (socket says 172.21.15.94:46984) 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec80c3a640 1 -- 172.21.15.94:0/1555088918 learned_addr learned my addr 172.21.15.94:0/1555088918 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec85274740 1 -- 172.21.15.94:0/1555088918 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f037c25b0 con 0x558f037b3bd0 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec85274740 1 -- 172.21.15.94:0/1555088918 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f037c2770 con 0x558f037b32b0 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec85274740 10 client.0 _ll_register_callbacks cb 0x7fec740093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.906+0000 7fec7bfff640 1 --2- 172.21.15.94:0/1555088918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b32b0 0x558f037c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.907+0000 7fec7b7fe640 1 -- 172.21.15.94:0/1555088918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b32b0 msgr2=0x558f037c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.907+0000 7fec7b7fe640 1 --2- 172.21.15.94:0/1555088918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b32b0 0x558f037c36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.907+0000 7fec7b7fe640 1 -- 172.21.15.94:0/1555088918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037bc7b0 msgr2=0x558f037c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.907+0000 7fec7b7fe640 1 --2- 172.21.15.94:0/1555088918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037bc7b0 0x558f037c1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.907+0000 7fec7b7fe640 1 -- 172.21.15.94:0/1555088918 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f03719610 con 0x558f037b3bd0 2024-02-17T12:08:19.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.907+0000 7fec7b7fe640 1 --2- 172.21.15.94:0/1555088918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037b3bd0 0x558f037c3bf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x558f037cf8c0 tx=0x7fec74004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fc01b3e5b79f26 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.907+0000 7fec5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.907+0000 7fec5b7fe640 1 -- 172.21.15.94:0/1555088918 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec740248a0 con 0x558f037b3bd0 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.907+0000 7fec5b7fe640 1 -- 172.21.15.94:0/1555088918 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fec74004b20 con 0x558f037b3bd0 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.907+0000 7fec85274740 1 -- 172.21.15.94:0/1555088918 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec6c0048a0 con 0x558f037b3bd0 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.907+0000 7fec5b7fe640 1 -- 172.21.15.94:0/1555088918 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec74004ce0 con 0x558f037b3bd0 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.907+0000 7fec85274740 10 client.27623 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.907+0000 7fec85274740 1 -- 172.21.15.94:0/1555088918 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558f036db6a0 con 0x558f037b3bd0 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.908+0000 7fec5b7fe640 1 -- 172.21.15.94:0/1555088918 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec6c0048a0 con 0x558f037b3bd0 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.908+0000 7fec5b7fe640 1 -- 172.21.15.94:0/1555088918 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fec74018050 con 0x558f037b3bd0 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.908+0000 7fec85274740 10 client.27623 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.908+0000 7fec85274740 10 client.27623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.908+0000 7fec85274740 20 client.27623 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.908+0000 7fec85274740 1 --2- 172.21.15.94:0/1555088918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558f038243d0 0x558f038267f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:19.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.908+0000 7fec85274740 4 client.27623 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.908+0000 7fec7bfff640 1 --2- 172.21.15.94:0/1555088918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558f038243d0 0x558f038267f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.909+0000 7fec85274740 1 -- 172.21.15.94:0/1555088918 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558f034a6890 con 0x558f038243d0 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.909+0000 7fec7bfff640 1 --2- 172.21.15.94:0/1555088918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558f038243d0 0x558f038267f0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0ed0a2310b82d80 server_cookie=8ed86dd70ac72337 in_seq=0 out_seq=0 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.909+0000 7fec5b7fe640 10 client.27623 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.910+0000 7fec5b7fe640 1 -- 172.21.15.94:0/1555088918 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558f034a6890 con 0x558f038243d0 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.910+0000 7fec5b7fe640 10 client.27623 handle_command_reply: tid=0 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.910+0000 7fec85274740 1 client.27623 shutdown 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.910+0000 7fec85274740 1 -- 172.21.15.94:0/1555088918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037b3bd0 msgr2=0x558f037c3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.911+0000 7fec85274740 1 --2- 172.21.15.94:0/1555088918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037b3bd0 0x558f037c3bf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x558f037cf8c0 tx=0x7fec74004a40 comp rx=0 tx=0).stop 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.911+0000 7fec85274740 1 -- 172.21.15.94:0/1555088918 shutdown_connections 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.911+0000 7fec85274740 1 --2- 172.21.15.94:0/1555088918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558f038243d0 0x558f038267f0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.911+0000 7fec5b7fe640 0 client.27623 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.911+0000 7fec85274740 1 --2- 172.21.15.94:0/1555088918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558f037bc7b0 0x558f037c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.911+0000 7fec85274740 1 --2- 172.21.15.94:0/1555088918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558f037b3bd0 0x558f037c3bf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.917 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.911+0000 7fec85274740 1 --2- 172.21.15.94:0/1555088918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558f037b32b0 0x558f037c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.911+0000 7fec85274740 1 -- 172.21.15.94:0/1555088918 >> 172.21.15.94:0/1555088918 conn(0x558f037ad990 msgr2=0x558f037b8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:19.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.911+0000 7fec85274740 1 -- 172.21.15.94:0/1555088918 shutdown_connections 2024-02-17T12:08:19.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.911+0000 7fec85274740 1 -- 172.21.15.94:0/1555088918 wait complete. 2024-02-17T12:08:19.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.911+0000 7fec85274740 20 client.27623 trim_cache size 0 max 16384 2024-02-17T12:08:19.918 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:08:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:08:19.918 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:08:19.918 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:08:19.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.912+0000 7fec8269d640 1 -- 172.21.15.94:0/1935546114 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec640a2440 msgr2=0x7fec640a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:19.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.912+0000 7fec8269d640 1 --2- 172.21.15.94:0/1935546114 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec640a2440 0x7fec640a4900 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x558f037b2d80 tx=0x7fec74009500 comp rx=0 tx=0).stop 2024-02-17T12:08:19.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.912+0000 7fec8269d640 1 -- 172.21.15.94:0/1935546114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec7c10f8a0 msgr2=0x7fec7c119d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:19.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.912+0000 7fec8269d640 1 --2- 172.21.15.94:0/1935546114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec7c10f8a0 0x7fec7c119d90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fec7000e9f0 tx=0x7fec7000eeb0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.912+0000 7fec8269d640 1 -- 172.21.15.94:0/1935546114 shutdown_connections 2024-02-17T12:08:19.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.912+0000 7fec8269d640 1 --2- 172.21.15.94:0/1935546114 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec640a2440 0x7fec640a4900 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.912+0000 7fec8269d640 1 --2- 172.21.15.94:0/1935546114 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec7c11a960 0x7fec7c112e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.912+0000 7fec8269d640 1 --2- 172.21.15.94:0/1935546114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec7c10fd10 0x7fec7c11a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.912+0000 7fec8269d640 1 --2- 172.21.15.94:0/1935546114 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec7c10f8a0 0x7fec7c119d90 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:19.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.912+0000 7fec8269d640 1 -- 172.21.15.94:0/1935546114 >> 172.21.15.94:0/1935546114 conn(0x7fec7c06c270 msgr2=0x7fec7c11b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:19.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.912+0000 7fec8269d640 1 -- 172.21.15.94:0/1935546114 shutdown_connections 2024-02-17T12:08:19.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:19.912+0000 7fec8269d640 1 -- 172.21.15.94:0/1935546114 wait complete. 2024-02-17T12:08:19.926 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:08:19.926 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:08:20.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.068+0000 7fde53ef4640 1 -- 172.21.15.94:0/727654090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde4c106510 msgr2=0x7fde4c10cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.068+0000 7fde53ef4640 1 --2- 172.21.15.94:0/727654090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde4c106510 0x7fde4c10cf20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fde44009cd0 tx=0x7fde4402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.068+0000 7fde53ef4640 1 -- 172.21.15.94:0/727654090 shutdown_connections 2024-02-17T12:08:20.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.068+0000 7fde53ef4640 1 --2- 172.21.15.94:0/727654090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde4c106510 0x7fde4c10cf20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.068+0000 7fde53ef4640 1 --2- 172.21.15.94:0/727654090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4c101990 0x7fde4c105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.068+0000 7fde53ef4640 1 --2- 172.21.15.94:0/727654090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde4c100fe0 0x7fde4c1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.068+0000 7fde53ef4640 1 -- 172.21.15.94:0/727654090 >> 172.21.15.94:0/727654090 conn(0x7fde4c0753a0 msgr2=0x7fde4c0757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.068+0000 7fde53ef4640 1 -- 172.21.15.94:0/727654090 shutdown_connections 2024-02-17T12:08:20.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.068+0000 7fde53ef4640 1 -- 172.21.15.94:0/727654090 wait complete. 2024-02-17T12:08:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.069+0000 7fde53ef4640 1 Processor -- start 2024-02-17T12:08:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.069+0000 7fde53ef4640 1 -- start start 2024-02-17T12:08:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.069+0000 7fde53ef4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde4c100fe0 0x7fde4c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.069+0000 7fde53ef4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4c106510 0x7fde4c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.069+0000 7fde53ef4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde4c1975e0 0x7fde4c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.069+0000 7fde53ef4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c197fd0 con 0x7fde4c1975e0 2024-02-17T12:08:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.069+0000 7fde53ef4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c198140 con 0x7fde4c106510 2024-02-17T12:08:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.069+0000 7fde53ef4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c198300 con 0x7fde4c100fe0 2024-02-17T12:08:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.069+0000 7fde52491640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde4c1975e0 0x7fde4c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.069+0000 7fde5148f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4c106510 0x7fde4c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.069+0000 7fde5148f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4c106510 0x7fde4c19d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34770/0 (socket says 172.21.15.94:34770) 2024-02-17T12:08:20.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.069+0000 7fde5148f640 1 -- 172.21.15.94:0/1213178984 learned_addr learned my addr 172.21.15.94:0/1213178984 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.069+0000 7fde51c90640 1 --2- 172.21.15.94:0/1213178984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde4c100fe0 0x7fde4c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.070+0000 7fde5148f640 1 -- 172.21.15.94:0/1213178984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde4c100fe0 msgr2=0x7fde4c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.070+0000 7fde5148f640 1 --2- 172.21.15.94:0/1213178984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde4c100fe0 0x7fde4c19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.070+0000 7fde5148f640 1 -- 172.21.15.94:0/1213178984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde4c1975e0 msgr2=0x7fde4c197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.070+0000 7fde5148f640 1 --2- 172.21.15.94:0/1213178984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde4c1975e0 0x7fde4c197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.070+0000 7fde5148f640 1 -- 172.21.15.94:0/1213178984 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde44009900 con 0x7fde4c106510 2024-02-17T12:08:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.070+0000 7fde5148f640 1 --2- 172.21.15.94:0/1213178984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4c106510 0x7fde4c19d970 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00d770 tx=0x7fde3c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.070+0000 7fde42ffd640 1 -- 172.21.15.94:0/1213178984 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c009c30 con 0x7fde4c106510 2024-02-17T12:08:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.070+0000 7fde53ef4640 1 -- 172.21.15.94:0/1213178984 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c102680 con 0x7fde4c106510 2024-02-17T12:08:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.070+0000 7fde42ffd640 1 -- 172.21.15.94:0/1213178984 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fde3c016070 con 0x7fde4c106510 2024-02-17T12:08:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.070+0000 7fde53ef4640 1 -- 172.21.15.94:0/1213178984 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c102be0 con 0x7fde4c106510 2024-02-17T12:08:20.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.070+0000 7fde42ffd640 1 -- 172.21.15.94:0/1213178984 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c01c8a0 con 0x7fde4c106510 2024-02-17T12:08:20.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.071+0000 7fde53ef4640 1 -- 172.21.15.94:0/1213178984 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde30005290 con 0x7fde4c106510 2024-02-17T12:08:20.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.072+0000 7fde42ffd640 1 -- 172.21.15.94:0/1213178984 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fde3c01ca00 con 0x7fde4c106510 2024-02-17T12:08:20.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.073+0000 7fde42ffd640 1 --2- 172.21.15.94:0/1213178984 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fde380a2330 0x7fde380a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.073+0000 7fde42ffd640 1 -- 172.21.15.94:0/1213178984 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fde3c0d1870 con 0x7fde4c106510 2024-02-17T12:08:20.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.073+0000 7fde51c90640 1 --2- 172.21.15.94:0/1213178984 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fde380a2330 0x7fde380a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.073+0000 7fde51c90640 1 --2- 172.21.15.94:0/1213178984 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fde380a2330 0x7fde380a47f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fde48005fd0 tx=0x7fde4800b390 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.077+0000 7fde42ffd640 1 -- 172.21.15.94:0/1213178984 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fde3c09a620 con 0x7fde4c106510 2024-02-17T12:08:20.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 600.0 s, 1856.6 tps, lat 53.169 ms stddev 59.242 2024-02-17T12:08:20.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.400+0000 7fde53ef4640 1 -- 172.21.15.94:0/1213178984 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde300058f0 con 0x7fde4c106510 2024-02-17T12:08:20.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.401+0000 7fde42ffd640 1 -- 172.21.15.94:0/1213178984 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7fde300058f0 con 0x7fde4c106510 2024-02-17T12:08:20.403 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:08:20.403 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:08:20.403 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:08:20.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.404+0000 7fde53ef4640 1 -- 172.21.15.94:0/1213178984 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fde380a2330 msgr2=0x7fde380a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.404+0000 7fde53ef4640 1 --2- 172.21.15.94:0/1213178984 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fde380a2330 0x7fde380a47f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fde48005fd0 tx=0x7fde4800b390 comp rx=0 tx=0).stop 2024-02-17T12:08:20.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.404+0000 7fde53ef4640 1 -- 172.21.15.94:0/1213178984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4c106510 msgr2=0x7fde4c19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.404+0000 7fde53ef4640 1 --2- 172.21.15.94:0/1213178984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4c106510 0x7fde4c19d970 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00d770 tx=0x7fde3c00dc30 comp rx=0 tx=0).stop 2024-02-17T12:08:20.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.404+0000 7fde53ef4640 1 -- 172.21.15.94:0/1213178984 shutdown_connections 2024-02-17T12:08:20.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.404+0000 7fde53ef4640 1 --2- 172.21.15.94:0/1213178984 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fde380a2330 0x7fde380a47f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.404+0000 7fde53ef4640 1 --2- 172.21.15.94:0/1213178984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fde4c1975e0 0x7fde4c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.404+0000 7fde53ef4640 1 --2- 172.21.15.94:0/1213178984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fde4c106510 0x7fde4c19d970 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.404+0000 7fde53ef4640 1 --2- 172.21.15.94:0/1213178984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde4c100fe0 0x7fde4c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.404+0000 7fde53ef4640 1 -- 172.21.15.94:0/1213178984 >> 172.21.15.94:0/1213178984 conn(0x7fde4c0753a0 msgr2=0x7fde4c104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.404+0000 7fde53ef4640 1 -- 172.21.15.94:0/1213178984 shutdown_connections 2024-02-17T12:08:20.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.404+0000 7fde53ef4640 1 -- 172.21.15.94:0/1213178984 wait complete. 2024-02-17T12:08:20.424 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:08:20.425 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:08:20.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.589+0000 7fbbbc593640 1 -- 172.21.15.94:0/2782069977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbb41009e0 msgr2=0x7fbbb410cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.589+0000 7fbbbc593640 1 --2- 172.21.15.94:0/2782069977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbb41009e0 0x7fbbb410cea0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009ca0 tx=0x7fbbac02c180 comp rx=0 tx=0).stop 2024-02-17T12:08:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.589+0000 7fbbbc593640 1 -- 172.21.15.94:0/2782069977 shutdown_connections 2024-02-17T12:08:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.589+0000 7fbbbc593640 1 --2- 172.21.15.94:0/2782069977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbb41009e0 0x7fbbb410cea0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.589+0000 7fbbbc593640 1 --2- 172.21.15.94:0/2782069977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb4100040 0x7fbbb41004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.589+0000 7fbbbc593640 1 --2- 172.21.15.94:0/2782069977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbb4106100 0x7fbbb41064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.589+0000 7fbbbc593640 1 -- 172.21.15.94:0/2782069977 >> 172.21.15.94:0/2782069977 conn(0x7fbbb4077110 msgr2=0x7fbbb4077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.589+0000 7fbbbc593640 1 -- 172.21.15.94:0/2782069977 shutdown_connections 2024-02-17T12:08:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.589+0000 7fbbbc593640 1 -- 172.21.15.94:0/2782069977 wait complete. 2024-02-17T12:08:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.590+0000 7fbbbc593640 1 Processor -- start 2024-02-17T12:08:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.590+0000 7fbbbc593640 1 -- start start 2024-02-17T12:08:20.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.590+0000 7fbbbc593640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbb4100040 0x7fbbb4196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.590+0000 7fbbbc593640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb41009e0 0x7fbbb4196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.590+0000 7fbbbc593640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbb4106100 0x7fbbb4196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.590+0000 7fbbbc593640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb41a5a50 con 0x7fbbb4100040 2024-02-17T12:08:20.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.590+0000 7fbbbc593640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb41a5bc0 con 0x7fbbb4106100 2024-02-17T12:08:20.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.590+0000 7fbbbc593640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb41a5d80 con 0x7fbbb41009e0 2024-02-17T12:08:20.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.590+0000 7fbbba32f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbb4100040 0x7fbbb4196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.590+0000 7fbbb9b2e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb41009e0 0x7fbbb4196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.590+0000 7fbbb9b2e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb41009e0 0x7fbbb4196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57628/0 (socket says 172.21.15.94:57628) 2024-02-17T12:08:20.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.590+0000 7fbbb9b2e640 1 -- 172.21.15.94:0/2305104047 learned_addr learned my addr 172.21.15.94:0/2305104047 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:20.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.591+0000 7fbbb9b2e640 1 -- 172.21.15.94:0/2305104047 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbb4106100 msgr2=0x7fbbb4196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.591+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/2305104047 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbb4106100 0x7fbbb4196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.591+0000 7fbbb9b2e640 1 -- 172.21.15.94:0/2305104047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbb4100040 msgr2=0x7fbbb4196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.591+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/2305104047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbb4100040 0x7fbbb4196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.591+0000 7fbbb9b2e640 1 -- 172.21.15.94:0/2305104047 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac009900 con 0x7fbbb41009e0 2024-02-17T12:08:20.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.591+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/2305104047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb41009e0 0x7fbbb4196850 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbba400d950 tx=0x7fbba400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.592+0000 7fbbab7fe640 1 -- 172.21.15.94:0/2305104047 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4009a30 con 0x7fbbb41009e0 2024-02-17T12:08:20.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.592+0000 7fbbab7fe640 1 -- 172.21.15.94:0/2305104047 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbba4009b90 con 0x7fbbb41009e0 2024-02-17T12:08:20.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.592+0000 7fbbab7fe640 1 -- 172.21.15.94:0/2305104047 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4004a00 con 0x7fbbb41009e0 2024-02-17T12:08:20.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.592+0000 7fbbbc593640 1 -- 172.21.15.94:0/2305104047 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb41a6060 con 0x7fbbb41009e0 2024-02-17T12:08:20.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.592+0000 7fbbbc593640 1 -- 172.21.15.94:0/2305104047 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb41a6520 con 0x7fbbb41009e0 2024-02-17T12:08:20.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.595+0000 7fbbab7fe640 1 -- 172.21.15.94:0/2305104047 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fbba4009d00 con 0x7fbbb41009e0 2024-02-17T12:08:20.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbbf16c740 1 Processor -- start 2024-02-17T12:08:20.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbbf16c740 1 -- start start 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbab7fe640 1 --2- 172.21.15.94:0/2305104047 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbba00a2400 0x7fbba00a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbbf16c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724759250 0x555724759630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbbf16c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759b70 0x555724762210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbbf16c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724762750 0x555724764b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbbf16c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555724714960 con 0x555724759250 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbbf16c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557246c8310 con 0x555724762750 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbbf16c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557246c5840 con 0x555724759b70 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbab7fe640 1 -- 172.21.15.94:0/2305104047 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbba40cdbd0 con 0x7fbbb41009e0 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbbab30640 1 --2- 172.21.15.94:0/2305104047 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbba00a2400 0x7fbba00a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbb9b2e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759b70 0x555724762210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbb9b2e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759b70 0x555724762210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57636/0 (socket says 172.21.15.94:57636) 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbb9b2e640 1 -- 172.21.15.94:0/227958486 learned_addr learned my addr 172.21.15.94:0/227958486 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.596+0000 7fbbba32f640 1 --2- 172.21.15.94:0/227958486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724759250 0x555724759630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.597+0000 7fbbbab30640 1 --2- 172.21.15.94:0/227958486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724762750 0x555724764b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.597+0000 7fbbb9b2e640 1 -- 172.21.15.94:0/227958486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724762750 msgr2=0x555724764b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.597+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/227958486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724762750 0x555724764b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.597+0000 7fbbb9b2e640 1 -- 172.21.15.94:0/227958486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724759250 msgr2=0x555724759630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.597+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/227958486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724759250 0x555724759630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.597+0000 7fbbb9b2e640 1 -- 172.21.15.94:0/227958486 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555724683060 con 0x555724759b70 2024-02-17T12:08:20.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.597+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/227958486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759b70 0x555724762210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbba4096980 tx=0x7fbba4099e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4143740f61e482e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.597+0000 7fbbbab30640 1 --2- 172.21.15.94:0/2305104047 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbba00a2400 0x7fbba00a48c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbbac037040 tx=0x7fbbac005c50 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.597+0000 7fbba8bfe640 1 -- 172.21.15.94:0/227958486 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba409a520 con 0x555724759b70 2024-02-17T12:08:20.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.597+0000 7fbba8bfe640 1 -- 172.21.15.94:0/227958486 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbba409e670 con 0x555724759b70 2024-02-17T12:08:20.600 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.597+0000 7fbba8bfe640 1 -- 172.21.15.94:0/227958486 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4098920 con 0x555724759b70 2024-02-17T12:08:20.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.598+0000 7fbbbf16c740 1 -- 172.21.15.94:0/227958486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759b70 msgr2=0x555724762210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.598+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/227958486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759b70 0x555724762210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbba4096980 tx=0x7fbba4099e80 comp rx=0 tx=0).stop 2024-02-17T12:08:20.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.599+0000 7fbbbf16c740 1 -- 172.21.15.94:0/227958486 shutdown_connections 2024-02-17T12:08:20.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.599+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/227958486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724762750 0x555724764b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.601 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.599+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/227958486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759b70 0x555724762210 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.599+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/227958486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724759250 0x555724759630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.599+0000 7fbbbf16c740 1 -- 172.21.15.94:0/227958486 >> 172.21.15.94:0/227958486 conn(0x555724753980 msgr2=0x555724753d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.599+0000 7fbbbf16c740 1 -- 172.21.15.94:0/227958486 shutdown_connections 2024-02-17T12:08:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.599+0000 7fbbbf16c740 1 -- 172.21.15.94:0/227958486 wait complete. 2024-02-17T12:08:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.600+0000 7fbbbf16c740 1 -- start start 2024-02-17T12:08:20.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.600+0000 7fbbbf16c740 1 Processor -- start 2024-02-17T12:08:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.600+0000 7fbbbf16c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759250 0x55572475f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.600+0000 7fbbbf16c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759b70 0x55572475fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.600+0000 7fbbbf16c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724762750 0x5557247601e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.600+0000 7fbbbf16c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557246c1020 con 0x555724762750 2024-02-17T12:08:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.600+0000 7fbbbf16c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555724394430 con 0x555724759b70 2024-02-17T12:08:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.600+0000 7fbbbf16c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555724350e00 con 0x555724759250 2024-02-17T12:08:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbbf16c740 10 client.0 _ll_register_callbacks cb 0x555724668cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbba32f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759250 0x55572475f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbba32f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759250 0x55572475f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57644/0 (socket says 172.21.15.94:57644) 2024-02-17T12:08:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbbab30640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724762750 0x5557247601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbba32f640 1 -- 172.21.15.94:0/1037439396 learned_addr learned my addr 172.21.15.94:0/1037439396 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/1037439396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759b70 0x55572475fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbba32f640 1 -- 172.21.15.94:0/1037439396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759b70 msgr2=0x55572475fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbba32f640 1 --2- 172.21.15.94:0/1037439396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759b70 0x55572475fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbba32f640 1 -- 172.21.15.94:0/1037439396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724762750 msgr2=0x5557247601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbba32f640 1 --2- 172.21.15.94:0/1037439396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724762750 0x5557247601e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbba32f640 1 -- 172.21.15.94:0/1037439396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb41a6520 con 0x555724759250 2024-02-17T12:08:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/1037439396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759b70 0x55572475fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbba32f640 1 --2- 172.21.15.94:0/1037439396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759250 0x55572475f760 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0009e20 tx=0x7fbbb00117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26212fe001182129 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbb9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:08:20.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbb9dffb640 1 -- 172.21.15.94:0/1037439396 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0012860 con 0x555724759250 2024-02-17T12:08:20.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbb9dffb640 1 -- 172.21.15.94:0/1037439396 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbbb00129c0 con 0x555724759250 2024-02-17T12:08:20.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.601+0000 7fbbbf16c740 1 -- 172.21.15.94:0/1037439396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5557243c07e0 con 0x555724759250 2024-02-17T12:08:20.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.602+0000 7fbb9dffb640 1 -- 172.21.15.94:0/1037439396 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbb0012b80 con 0x555724759250 2024-02-17T12:08:20.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.602+0000 7fbbbf16c740 10 client.27616 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:20.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.602+0000 7fbbbf16c740 1 -- 172.21.15.94:0/1037439396 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5557246bf6a0 con 0x555724759250 2024-02-17T12:08:20.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.602+0000 7fbb9dffb640 1 -- 172.21.15.94:0/1037439396 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbb0027080 con 0x555724759250 2024-02-17T12:08:20.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.602+0000 7fbb9dffb640 1 -- 172.21.15.94:0/1037439396 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbbb000c070 con 0x555724759250 2024-02-17T12:08:20.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.602+0000 7fbbbf16c740 10 client.27616 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:20.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.602+0000 7fbbbf16c740 10 client.27616 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:08:20.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.602+0000 7fbbbf16c740 20 client.27616 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:20.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.602+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/1037439396 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555724772be0 0x555724775000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.602+0000 7fbbbf16c740 4 client.27616 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:20.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.602+0000 7fbbbf16c740 1 -- 172.21.15.94:0/1037439396 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557243fd720 con 0x555724772be0 2024-02-17T12:08:20.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.603+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/1037439396 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555724772be0 0x555724775000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.603+0000 7fbb9dffb640 10 client.27616 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:08:20.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.603+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/1037439396 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555724772be0 0x555724775000 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d584db98b4ca30bc server_cookie=316fc33a7b38635 in_seq=0 out_seq=0 2024-02-17T12:08:20.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbb9dffb640 1 -- 172.21.15.94:0/1037439396 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5557243fd720 con 0x555724772be0 2024-02-17T12:08:20.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbb9dffb640 10 client.27616 handle_command_reply: tid=0 2024-02-17T12:08:20.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbbbf16c740 1 client.27616 shutdown 2024-02-17T12:08:20.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbbbf16c740 1 -- 172.21.15.94:0/1037439396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759250 msgr2=0x55572475f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/1037439396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759250 0x55572475f760 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbbb0009e20 tx=0x7fbbb00117b0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbbbf16c740 1 -- 172.21.15.94:0/1037439396 shutdown_connections 2024-02-17T12:08:20.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/1037439396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724762750 0x5557247601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/1037439396 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555724772be0 0x555724775000 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbb9dffb640 0 client.27616 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:08:20.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/1037439396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759b70 0x55572475fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/1037439396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759250 0x55572475f760 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbbbf16c740 1 -- 172.21.15.94:0/1037439396 >> 172.21.15.94:0/1037439396 conn(0x555724753980 msgr2=0x5557247556d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbbbf16c740 1 -- 172.21.15.94:0/1037439396 shutdown_connections 2024-02-17T12:08:20.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbbbf16c740 1 -- 172.21.15.94:0/1037439396 wait complete. 2024-02-17T12:08:20.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.604+0000 7fbbbf16c740 20 client.27616 trim_cache size 0 max 16384 2024-02-17T12:08:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.626+0000 7fbbbf16c740 1 Processor -- start 2024-02-17T12:08:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.626+0000 7fbbbf16c740 1 -- start start 2024-02-17T12:08:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.626+0000 7fbbbf16c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759250 0x555724761e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.626+0000 7fbbbf16c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759b70 0x555724760000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.626+0000 7fbbbf16c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724762750 0x55572475aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.626+0000 7fbbbf16c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555724646130 con 0x555724762750 2024-02-17T12:08:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.626+0000 7fbbbf16c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557247549b0 con 0x555724759250 2024-02-17T12:08:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.626+0000 7fbbbf16c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55572477a280 con 0x555724759b70 2024-02-17T12:08:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.629+0000 7fbbba32f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759250 0x555724761e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.629+0000 7fbbba32f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759250 0x555724761e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34826/0 (socket says 172.21.15.94:34826) 2024-02-17T12:08:20.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.629+0000 7fbbba32f640 1 -- 172.21.15.94:0/3256331346 learned_addr learned my addr 172.21.15.94:0/3256331346 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:20.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.630+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/3256331346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759b70 0x555724760000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.630+0000 7fbbbab30640 1 --2- 172.21.15.94:0/3256331346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724762750 0x55572475aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.633+0000 7fbbbab30640 1 -- 172.21.15.94:0/3256331346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759b70 msgr2=0x555724760000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.633+0000 7fbbbab30640 1 --2- 172.21.15.94:0/3256331346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759b70 0x555724760000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.633+0000 7fbbbab30640 1 -- 172.21.15.94:0/3256331346 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759250 msgr2=0x555724761e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.633+0000 7fbbbab30640 1 --2- 172.21.15.94:0/3256331346 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759250 0x555724761e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.633+0000 7fbbbab30640 1 -- 172.21.15.94:0/3256331346 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55572465bc20 con 0x555724762750 2024-02-17T12:08:20.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.636+0000 7fbbba32f640 1 --2- 172.21.15.94:0/3256331346 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759250 0x555724761e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:20.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.641+0000 7fbbbab30640 1 --2- 172.21.15.94:0/3256331346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724762750 0x55572475aa90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fbbac02c790 tx=0x7fbbac03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb77c83a7c19a47d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.648+0000 7fbb9ffff640 1 -- 172.21.15.94:0/3256331346 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac05b070 con 0x555724762750 2024-02-17T12:08:20.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.648+0000 7fbb9ffff640 1 -- 172.21.15.94:0/3256331346 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbbac052040 con 0x555724762750 2024-02-17T12:08:20.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.649+0000 7fbbbf16c740 1 -- 172.21.15.94:0/3256331346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724762750 msgr2=0x55572475aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.649+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/3256331346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724762750 0x55572475aa90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fbbac02c790 tx=0x7fbbac03d000 comp rx=0 tx=0).stop 2024-02-17T12:08:20.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.654+0000 7fbbbf16c740 1 -- 172.21.15.94:0/3256331346 shutdown_connections 2024-02-17T12:08:20.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.654+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/3256331346 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724762750 0x55572475aa90 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.654+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/3256331346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724759b70 0x555724760000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.654+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/3256331346 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759250 0x555724761e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.654+0000 7fbbbf16c740 1 -- 172.21.15.94:0/3256331346 >> 172.21.15.94:0/3256331346 conn(0x555724753980 msgr2=0x55572474ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.662+0000 7fbbbf16c740 1 -- 172.21.15.94:0/3256331346 shutdown_connections 2024-02-17T12:08:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.665+0000 7fbbbf16c740 1 -- 172.21.15.94:0/3256331346 wait complete. 2024-02-17T12:08:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.665+0000 7fbbbf16c740 1 -- start start 2024-02-17T12:08:20.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.665+0000 7fbbbf16c740 1 Processor -- start 2024-02-17T12:08:20.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.672+0000 7fbbbf16c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759250 0x5557247683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.672+0000 7fbbbf16c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724759b70 0x5557247688e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.672+0000 7fbbbf16c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724762750 0x5557247c9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.672+0000 7fbbbf16c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557247691b0 con 0x555724759b70 2024-02-17T12:08:20.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.672+0000 7fbbbf16c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557247c9670 con 0x555724759250 2024-02-17T12:08:20.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.672+0000 7fbbbf16c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557247da7c0 con 0x555724762750 2024-02-17T12:08:20.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.673+0000 7fbbbf16c740 10 client.0 _ll_register_callbacks cb 0x7fbba4005240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:20.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.674+0000 7fbbbab30640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724762750 0x5557247c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.674+0000 7fbbbab30640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724762750 0x5557247c9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57660/0 (socket says 172.21.15.94:57660) 2024-02-17T12:08:20.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.674+0000 7fbbbab30640 1 -- 172.21.15.94:0/3915450350 learned_addr learned my addr 172.21.15.94:0/3915450350 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:20.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.674+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/3915450350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724759b70 0x5557247688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.675+0000 7fbbba32f640 1 --2- 172.21.15.94:0/3915450350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759250 0x5557247683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.675+0000 7fbbb9b2e640 1 -- 172.21.15.94:0/3915450350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724762750 msgr2=0x5557247c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.675+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/3915450350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724762750 0x5557247c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.675+0000 7fbbb9b2e640 1 -- 172.21.15.94:0/3915450350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759250 msgr2=0x5557247683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.675+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/3915450350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759250 0x5557247683a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.675+0000 7fbbb9b2e640 1 -- 172.21.15.94:0/3915450350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557246bf6a0 con 0x555724759b70 2024-02-17T12:08:20.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.675+0000 7fbbb9b2e640 1 --2- 172.21.15.94:0/3915450350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724759b70 0x5557247688e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fbba401d000 tx=0x7fbba4098820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f449ea687066789 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.675+0000 7fbb9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:08:20.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.676+0000 7fbb9dffb640 1 -- 172.21.15.94:0/3915450350 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba409a520 con 0x555724759b70 2024-02-17T12:08:20.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.676+0000 7fbb9dffb640 1 -- 172.21.15.94:0/3915450350 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbba405c070 con 0x555724759b70 2024-02-17T12:08:20.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.677+0000 7fbbbf16c740 1 -- 172.21.15.94:0/3915450350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbba409e670 con 0x555724759b70 2024-02-17T12:08:20.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.678+0000 7fbb9dffb640 1 -- 172.21.15.94:0/3915450350 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbba40a2cc0 con 0x555724759b70 2024-02-17T12:08:20.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.679+0000 7fbbbf16c740 10 client.18156 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:20.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.679+0000 7fbbbf16c740 1 -- 172.21.15.94:0/3915450350 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fbbb410d7a0 con 0x555724759b70 2024-02-17T12:08:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.685+0000 7fbb9dffb640 1 -- 172.21.15.94:0/3915450350 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbba409e670 con 0x555724759b70 2024-02-17T12:08:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.686+0000 7fbb9dffb640 1 -- 172.21.15.94:0/3915450350 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbba405f2e0 con 0x555724759b70 2024-02-17T12:08:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.686+0000 7fbbbf16c740 10 client.18156 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.686+0000 7fbbbf16c740 10 client.18156 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:08:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.686+0000 7fbbbf16c740 20 client.18156 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.686+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/3915450350 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555724772be0 0x5557247ddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.686+0000 7fbbbf16c740 4 client.18156 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:08:20.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.686+0000 7fbbbf16c740 1 -- 172.21.15.94:0/3915450350 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55572444c890 con 0x555724772be0 2024-02-17T12:08:20.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.689+0000 7fbbba32f640 1 --2- 172.21.15.94:0/3915450350 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555724772be0 0x5557247ddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.691+0000 7fbbba32f640 1 --2- 172.21.15.94:0/3915450350 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555724772be0 0x5557247ddaa0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50524ef8d63597f7 server_cookie=db70643cf1cba236 in_seq=0 out_seq=0 2024-02-17T12:08:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.691+0000 7fbb9dffb640 10 client.18156 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:08:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.691+0000 7fbb9dffb640 1 -- 172.21.15.94:0/3915450350 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55572444c890 con 0x555724772be0 2024-02-17T12:08:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.691+0000 7fbb9dffb640 10 client.18156 handle_command_reply: tid=0 2024-02-17T12:08:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.691+0000 7fbbbf16c740 1 client.18156 shutdown 2024-02-17T12:08:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.692+0000 7fbbbf16c740 1 -- 172.21.15.94:0/3915450350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724759b70 msgr2=0x5557247688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.692+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/3915450350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724759b70 0x5557247688e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fbba401d000 tx=0x7fbba4098820 comp rx=0 tx=0).stop 2024-02-17T12:08:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.692+0000 7fbbbf16c740 1 -- 172.21.15.94:0/3915450350 shutdown_connections 2024-02-17T12:08:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.692+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/3915450350 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555724762750 0x5557247c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.692+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/3915450350 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555724759b70 0x5557247688e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.692+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/3915450350 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x555724772be0 0x5557247ddaa0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.692+0000 7fbbbf16c740 1 --2- 172.21.15.94:0/3915450350 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555724759250 0x5557247683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.692+0000 7fbbbf16c740 1 -- 172.21.15.94:0/3915450350 >> 172.21.15.94:0/3915450350 conn(0x555724753980 msgr2=0x55572475aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.692+0000 7fbb9dffb640 0 client.18156 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:08:20.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.693+0000 7fbbbf16c740 1 -- 172.21.15.94:0/3915450350 shutdown_connections 2024-02-17T12:08:20.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.693+0000 7fbbbf16c740 1 -- 172.21.15.94:0/3915450350 wait complete. 2024-02-17T12:08:20.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.693+0000 7fbbbf16c740 20 client.18156 trim_cache size 0 max 16384 2024-02-17T12:08:20.695 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:08:20.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.694+0000 7fbbbc593640 1 -- 172.21.15.94:0/2305104047 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbba00a2400 msgr2=0x7fbba00a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.694+0000 7fbbbc593640 1 --2- 172.21.15.94:0/2305104047 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbba00a2400 0x7fbba00a48c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbbac037040 tx=0x7fbbac005c50 comp rx=0 tx=0).stop 2024-02-17T12:08:20.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.694+0000 7fbbbc593640 1 -- 172.21.15.94:0/2305104047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb41009e0 msgr2=0x7fbbb4196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.694+0000 7fbbbc593640 1 --2- 172.21.15.94:0/2305104047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb41009e0 0x7fbbb4196850 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbba400d950 tx=0x7fbba400de10 comp rx=0 tx=0).stop 2024-02-17T12:08:20.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.694+0000 7fbbbc593640 1 -- 172.21.15.94:0/2305104047 shutdown_connections 2024-02-17T12:08:20.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.694+0000 7fbbbc593640 1 --2- 172.21.15.94:0/2305104047 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbba00a2400 0x7fbba00a48c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.695+0000 7fbbbc593640 1 --2- 172.21.15.94:0/2305104047 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbbb4106100 0x7fbbb4196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.695+0000 7fbbbc593640 1 --2- 172.21.15.94:0/2305104047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbb41009e0 0x7fbbb4196850 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.695+0000 7fbbbc593640 1 --2- 172.21.15.94:0/2305104047 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbbb4100040 0x7fbbb4196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.695+0000 7fbbbc593640 1 -- 172.21.15.94:0/2305104047 >> 172.21.15.94:0/2305104047 conn(0x7fbbb4077110 msgr2=0x7fbbb40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.695+0000 7fbbbc593640 1 -- 172.21.15.94:0/2305104047 shutdown_connections 2024-02-17T12:08:20.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.695+0000 7fbbbc593640 1 -- 172.21.15.94:0/2305104047 wait complete. 2024-02-17T12:08:20.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:20 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1213178984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:08:20.709 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:08:20.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:08:20.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.845+0000 7fd5cf2f4640 1 -- 172.21.15.94:0/860146593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8105330 msgr2=0x7fd5c810cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.845+0000 7fd5cf2f4640 1 --2- 172.21.15.94:0/860146593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8105330 0x7fd5c810cf00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400b9f0 tx=0x7fd5c402d930 comp rx=0 tx=0).stop 2024-02-17T12:08:20.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.847+0000 7fd5cf2f4640 1 -- 172.21.15.94:0/860146593 shutdown_connections 2024-02-17T12:08:20.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.847+0000 7fd5cf2f4640 1 --2- 172.21.15.94:0/860146593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8105330 0x7fd5c810cf00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.847+0000 7fd5cf2f4640 1 --2- 172.21.15.94:0/860146593 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5c8100c10 0x7fd5c8104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.847+0000 7fd5cf2f4640 1 --2- 172.21.15.94:0/860146593 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5c8100260 0x7fd5c8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.847+0000 7fd5cf2f4640 1 -- 172.21.15.94:0/860146593 >> 172.21.15.94:0/860146593 conn(0x7fd5c8077180 msgr2=0x7fd5c8077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.853+0000 7fd5cf2f4640 1 -- 172.21.15.94:0/860146593 shutdown_connections 2024-02-17T12:08:20.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.856+0000 7fd5cf2f4640 1 -- 172.21.15.94:0/860146593 wait complete. 2024-02-17T12:08:20.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.857+0000 7fd5cf2f4640 1 Processor -- start 2024-02-17T12:08:20.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.860+0000 7fd5cf2f4640 1 -- start start 2024-02-17T12:08:20.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.860+0000 7fd5cf2f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5c8100260 0x7fd5c80727c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.860+0000 7fd5cf2f4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8105330 0x7fd5c8072d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.860+0000 7fd5cf2f4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5c806b840 0x7fd5c806bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.860+0000 7fd5cf2f4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c806c480 con 0x7fd5c8100260 2024-02-17T12:08:20.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.860+0000 7fd5cf2f4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c806c5f0 con 0x7fd5c806b840 2024-02-17T12:08:20.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.860+0000 7fd5cf2f4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c80702e0 con 0x7fd5c8105330 2024-02-17T12:08:20.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.861+0000 7fd5cc88f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8105330 0x7fd5c8072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.861+0000 7fd5cc88f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8105330 0x7fd5c8072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57678/0 (socket says 172.21.15.94:57678) 2024-02-17T12:08:20.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.861+0000 7fd5cc88f640 1 -- 172.21.15.94:0/2568659763 learned_addr learned my addr 172.21.15.94:0/2568659763 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:20.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.861+0000 7fd5cd891640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5c806b840 0x7fd5c806bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.865+0000 7fd5cd090640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5c8100260 0x7fd5c80727c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.871+0000 7fd5cd891640 1 -- 172.21.15.94:0/2568659763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8105330 msgr2=0x7fd5c8072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.871+0000 7fd5cd891640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8105330 0x7fd5c8072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.871+0000 7fd5cd891640 1 -- 172.21.15.94:0/2568659763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5c8100260 msgr2=0x7fd5c80727c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.871+0000 7fd5cd891640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5c8100260 0x7fd5c80727c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.871+0000 7fd5cd891640 1 -- 172.21.15.94:0/2568659763 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c400b6a0 con 0x7fd5c806b840 2024-02-17T12:08:20.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.872+0000 7fd5cd891640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5c806b840 0x7fd5c806bcf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400b9f0 tx=0x7fd5c4032af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.877+0000 7fd5be7fc640 1 -- 172.21.15.94:0/2568659763 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4032e90 con 0x7fd5c806b840 2024-02-17T12:08:20.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.877+0000 7fd5cf2f4640 1 -- 172.21.15.94:0/2568659763 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c8070560 con 0x7fd5c806b840 2024-02-17T12:08:20.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.877+0000 7fd5cf2f4640 1 -- 172.21.15.94:0/2568659763 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c8070a10 con 0x7fd5c806b840 2024-02-17T12:08:20.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.877+0000 7fd5be7fc640 1 -- 172.21.15.94:0/2568659763 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5c4004ba0 con 0x7fd5c806b840 2024-02-17T12:08:20.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.878+0000 7fd5be7fc640 1 -- 172.21.15.94:0/2568659763 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4033ca0 con 0x7fd5c806b840 2024-02-17T12:08:20.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.882+0000 7fd5d1fa4740 1 Processor -- start 2024-02-17T12:08:20.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.883+0000 7fd5d1fa4740 1 -- start start 2024-02-17T12:08:20.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.883+0000 7fd5be7fc640 1 -- 172.21.15.94:0/2568659763 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd5c4043070 con 0x7fd5c806b840 2024-02-17T12:08:20.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.884+0000 7fd5d1fa4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786250 0x55e06e786630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.884+0000 7fd5d1fa4740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786b70 0x55e06e78f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.884+0000 7fd5d1fa4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78f750 0x55e06e791b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.884+0000 7fd5d1fa4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06e741960 con 0x55e06e786250 2024-02-17T12:08:20.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.884+0000 7fd5d1fa4740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06e6f5310 con 0x55e06e786b70 2024-02-17T12:08:20.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.884+0000 7fd5d1fa4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06e6f2840 con 0x55e06e78f750 2024-02-17T12:08:20.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.884+0000 7fd5cc88f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78f750 0x55e06e791b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.884+0000 7fd5cc88f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786250 0x55e06e786630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.884+0000 7fd5cc88f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786250 0x55e06e786630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47082/0 (socket says 172.21.15.94:47082) 2024-02-17T12:08:20.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.884+0000 7fd5cc88f640 1 -- 172.21.15.94:0/412506316 learned_addr learned my addr 172.21.15.94:0/412506316 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:20.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.885+0000 7fd5be7fc640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5b40a2190 0x7fd5b40a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.885+0000 7fd5cc88f640 1 -- 172.21.15.94:0/412506316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786b70 msgr2=0x55e06e78f210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:08:20.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.885+0000 7fd5cc88f640 1 --2- 172.21.15.94:0/412506316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786b70 0x55e06e78f210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.885+0000 7fd5cc88f640 1 -- 172.21.15.94:0/412506316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786250 msgr2=0x55e06e786630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.885+0000 7fd5cc88f640 1 --2- 172.21.15.94:0/412506316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786250 0x55e06e786630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.885+0000 7fd5cc88f640 1 -- 172.21.15.94:0/412506316 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e06e6b0060 con 0x55e06e78f750 2024-02-17T12:08:20.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.885+0000 7fd5be7fc640 1 -- 172.21.15.94:0/2568659763 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd5c40f0210 con 0x7fd5c806b840 2024-02-17T12:08:20.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.886+0000 7fd5cc88f640 1 --2- 172.21.15.94:0/412506316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78f750 0x55e06e791b40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8013830 tx=0x7fd5b80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6170d19ae61839c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.886+0000 7fd5cd891640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5b40a2190 0x7fd5b40a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.886+0000 7fd5b37fe640 1 -- 172.21.15.94:0/412506316 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8013e70 con 0x55e06e78f750 2024-02-17T12:08:20.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.886+0000 7fd5b37fe640 1 -- 172.21.15.94:0/412506316 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5b8005500 con 0x55e06e78f750 2024-02-17T12:08:20.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.886+0000 7fd5b37fe640 1 -- 172.21.15.94:0/412506316 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8004980 con 0x55e06e78f750 2024-02-17T12:08:20.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.887+0000 7fd5cd891640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5b40a2190 0x7fd5b40a4650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400e090 tx=0x7fd5c4032d70 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.888+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/412506316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78f750 msgr2=0x55e06e791b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.888+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/412506316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78f750 0x55e06e791b40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8013830 tx=0x7fd5b80046c0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.888+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/412506316 shutdown_connections 2024-02-17T12:08:20.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.888+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/412506316 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78f750 0x55e06e791b40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.888+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/412506316 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786b70 0x55e06e78f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.888+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/412506316 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786250 0x55e06e786630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.888+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/412506316 >> 172.21.15.94:0/412506316 conn(0x55e06e780980 msgr2=0x55e06e780d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.892+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/412506316 shutdown_connections 2024-02-17T12:08:20.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.894+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/412506316 wait complete. 2024-02-17T12:08:20.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.894+0000 7fd5d1fa4740 1 -- start start 2024-02-17T12:08:20.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.894+0000 7fd5d1fa4740 1 Processor -- start 2024-02-17T12:08:20.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.895+0000 7fd5d1fa4740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786250 0x55e06e78c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.895+0000 7fd5d1fa4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786b70 0x55e06e78cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.895+0000 7fd5d1fa4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78f750 0x55e06e78d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.895+0000 7fd5d1fa4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06e6ee020 con 0x55e06e786b70 2024-02-17T12:08:20.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.895+0000 7fd5d1fa4740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06e3c1430 con 0x55e06e786250 2024-02-17T12:08:20.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.895+0000 7fd5d1fa4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06e37de00 con 0x55e06e78f750 2024-02-17T12:08:20.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.895+0000 7fd5d1fa4740 10 client.0 _ll_register_callbacks cb 0x55e06e695cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:20.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.895+0000 7fd5cc88f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786250 0x55e06e78c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.895+0000 7fd5cd090640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786b70 0x55e06e78cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.895+0000 7fd5cc88f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78f750 0x55e06e78d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.895+0000 7fd5cd090640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786b70 0x55e06e78cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47084/0 (socket says 172.21.15.94:47084) 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.895+0000 7fd5cd090640 1 -- 172.21.15.94:0/3673682090 learned_addr learned my addr 172.21.15.94:0/3673682090 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.896+0000 7fd5cd090640 1 -- 172.21.15.94:0/3673682090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78f750 msgr2=0x55e06e78d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.896+0000 7fd5cd090640 1 --2- 172.21.15.94:0/3673682090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78f750 0x55e06e78d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.896+0000 7fd5cd090640 1 -- 172.21.15.94:0/3673682090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786250 msgr2=0x55e06e78c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.896+0000 7fd5cd090640 1 --2- 172.21.15.94:0/3673682090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786250 0x55e06e78c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.896+0000 7fd5cd090640 1 -- 172.21.15.94:0/3673682090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b801b040 con 0x55e06e786b70 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.896+0000 7fd5cd090640 1 --2- 172.21.15.94:0/3673682090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786b70 0x55e06e78cc60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0003f30 tx=0x7fd5c00044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a6d4ab810f90b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.896+0000 7fd5b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.897+0000 7fd5b0ff9640 1 -- 172.21.15.94:0/3673682090 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0010980 con 0x55e06e786b70 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.897+0000 7fd5b0ff9640 1 -- 172.21.15.94:0/3673682090 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd5c0010ae0 con 0x55e06e786b70 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.897+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/3673682090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e06e3ed7e0 con 0x55e06e786b70 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.897+0000 7fd5b0ff9640 1 -- 172.21.15.94:0/3673682090 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5c0010ca0 con 0x55e06e786b70 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.897+0000 7fd5d1fa4740 10 client.18168 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:20.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.897+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/3673682090 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e06e6ec6a0 con 0x55e06e786b70 2024-02-17T12:08:20.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.898+0000 7fd5b0ff9640 1 -- 172.21.15.94:0/3673682090 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55e06e3ed7e0 con 0x55e06e786b70 2024-02-17T12:08:20.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.898+0000 7fd5b0ff9640 1 -- 172.21.15.94:0/3673682090 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd5c0021070 con 0x55e06e786b70 2024-02-17T12:08:20.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.898+0000 7fd5d1fa4740 10 client.18168 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.898+0000 7fd5d1fa4740 10 client.18168 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:08:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.898+0000 7fd5d1fa4740 20 client.18168 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.898+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/3673682090 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55e06e79fbb0 0x55e06e7a1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.898+0000 7fd5d1fa4740 4 client.18168 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.898+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/3673682090 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e06e42a720 con 0x55e06e79fbb0 2024-02-17T12:08:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.899+0000 7fd5cc88f640 1 --2- 172.21.15.94:0/3673682090 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55e06e79fbb0 0x55e06e7a1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.900+0000 7fd5b0ff9640 10 client.18168 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:08:20.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.901+0000 7fd5cc88f640 1 --2- 172.21.15.94:0/3673682090 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55e06e79fbb0 0x55e06e7a1fd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1948732d61e1ef3d server_cookie=4a4d4d0f08c84626 in_seq=0 out_seq=0 2024-02-17T12:08:20.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:20 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1213178984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:08:20.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.902+0000 7fd5b0ff9640 1 -- 172.21.15.94:0/3673682090 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e06e42a720 con 0x55e06e79fbb0 2024-02-17T12:08:20.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.902+0000 7fd5b0ff9640 10 client.18168 handle_command_reply: tid=0 2024-02-17T12:08:20.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.903+0000 7fd5d1fa4740 1 client.18168 shutdown 2024-02-17T12:08:20.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.903+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/3673682090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786b70 msgr2=0x55e06e78cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.903+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/3673682090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786b70 0x55e06e78cc60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0003f30 tx=0x7fd5c00044d0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.903+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/3673682090 shutdown_connections 2024-02-17T12:08:20.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.903+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/3673682090 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55e06e79fbb0 0x55e06e7a1fd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.903+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/3673682090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78f750 0x55e06e78d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.903+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/3673682090 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786b70 0x55e06e78cc60 secure :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0003f30 tx=0x7fd5c00044d0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.903+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/3673682090 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786250 0x55e06e78c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.903+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/3673682090 >> 172.21.15.94:0/3673682090 conn(0x55e06e780980 msgr2=0x55e06e7826d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.904+0000 7fd5b0ff9640 0 client.18168 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:08:20.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.904+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/3673682090 shutdown_connections 2024-02-17T12:08:20.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.904+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/3673682090 wait complete. 2024-02-17T12:08:20.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.905+0000 7fd5d1fa4740 20 client.18168 trim_cache size 0 max 16384 2024-02-17T12:08:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.924+0000 7fd5d1fa4740 1 Processor -- start 2024-02-17T12:08:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.924+0000 7fd5d1fa4740 1 -- start start 2024-02-17T12:08:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.924+0000 7fd5d1fa4740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786250 0x55e06e78ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.924+0000 7fd5d1fa4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e78f750 0x55e06e78cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.924+0000 7fd5d1fa4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78aff0 0x55e06e78b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.924+0000 7fd5d1fa4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06e673130 con 0x55e06e78f750 2024-02-17T12:08:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.924+0000 7fd5cc88f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786250 0x55e06e78ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.924+0000 7fd5cc88f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786250 0x55e06e78ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34862/0 (socket says 172.21.15.94:34862) 2024-02-17T12:08:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.924+0000 7fd5cc88f640 1 -- 172.21.15.94:0/4176672115 learned_addr learned my addr 172.21.15.94:0/4176672115 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/4176672115 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06e42a720 con 0x55e06e786250 2024-02-17T12:08:20.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/4176672115 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06e781980 con 0x55e06e78aff0 2024-02-17T12:08:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5cd090640 1 --2- 172.21.15.94:0/4176672115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e78f750 0x55e06e78cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5cc88f640 1 --2- 172.21.15.94:0/4176672115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78aff0 0x55e06e78b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5cd090640 1 -- 172.21.15.94:0/4176672115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78aff0 msgr2=0x55e06e78b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5cd090640 1 --2- 172.21.15.94:0/4176672115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78aff0 0x55e06e78b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5cd090640 1 -- 172.21.15.94:0/4176672115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786250 msgr2=0x55e06e78ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5cd090640 1 --2- 172.21.15.94:0/4176672115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786250 0x55e06e78ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5cd090640 1 -- 172.21.15.94:0/4176672115 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e06e688c20 con 0x55e06e78f750 2024-02-17T12:08:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5cc88f640 1 --2- 172.21.15.94:0/4176672115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78aff0 0x55e06e78b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:08:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5cd090640 1 --2- 172.21.15.94:0/4176672115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e78f750 0x55e06e78cfc0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x55e06e78d7b0 tx=0x7fd5c0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d0d57dc99ef498 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5b2ffd640 1 -- 172.21.15.94:0/4176672115 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c00099c0 con 0x55e06e78f750 2024-02-17T12:08:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5b2ffd640 1 -- 172.21.15.94:0/4176672115 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5c0012450 con 0x55e06e78f750 2024-02-17T12:08:20.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.925+0000 7fd5b2ffd640 1 -- 172.21.15.94:0/4176672115 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0005040 con 0x55e06e78f750 2024-02-17T12:08:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.926+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/4176672115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e78f750 msgr2=0x55e06e78cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.926+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/4176672115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e78f750 0x55e06e78cfc0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x55e06e78d7b0 tx=0x7fd5c0005bd0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.926+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/4176672115 shutdown_connections 2024-02-17T12:08:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.926+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/4176672115 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78aff0 0x55e06e78b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.926+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/4176672115 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e78f750 0x55e06e78cfc0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.926+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/4176672115 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e786250 0x55e06e78ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.926+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/4176672115 >> 172.21.15.94:0/4176672115 conn(0x55e06e780980 msgr2=0x55e06e7a6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.926+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/4176672115 shutdown_connections 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.926+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/4176672115 wait complete. 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.926+0000 7fd5d1fa4740 1 -- start start 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.926+0000 7fd5d1fa4740 1 Processor -- start 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5d1fa4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786250 0x55e06e796820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5d1fa4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78aff0 0x55e06e794bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5d1fa4740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e78f750 0x55e06e795100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5d1fa4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06e795640 con 0x55e06e786250 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5d1fa4740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06e7957b0 con 0x55e06e78f750 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5d1fa4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e06e795920 con 0x55e06e78aff0 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5d1fa4740 10 client.0 _ll_register_callbacks cb 0x7fd5b8009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5cc88f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786250 0x55e06e796820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5cc88f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786250 0x55e06e796820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47100/0 (socket says 172.21.15.94:47100) 2024-02-17T12:08:20.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5cc88f640 1 -- 172.21.15.94:0/1816615117 learned_addr learned my addr 172.21.15.94:0/1816615117 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5cd090640 1 --2- 172.21.15.94:0/1816615117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78aff0 0x55e06e794bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5cc88f640 1 --2- 172.21.15.94:0/1816615117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e78f750 0x55e06e795100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5cc88f640 1 -- 172.21.15.94:0/1816615117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78aff0 msgr2=0x55e06e794bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5cc88f640 1 --2- 172.21.15.94:0/1816615117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78aff0 0x55e06e794bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5cc88f640 1 -- 172.21.15.94:0/1816615117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e78f750 msgr2=0x55e06e795100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5cc88f640 1 --2- 172.21.15.94:0/1816615117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e78f750 0x55e06e795100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5cc88f640 1 -- 172.21.15.94:0/1816615117 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e06e6ec6a0 con 0x55e06e786250 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5cc88f640 1 --2- 172.21.15.94:0/1816615117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786250 0x55e06e796820 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8013fd0 tx=0x7fd5b8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f24d476b990607 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.927+0000 7fd5b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5b0ff9640 1 -- 172.21.15.94:0/1816615117 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8031070 con 0x55e06e786250 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5b0ff9640 1 -- 172.21.15.94:0/1816615117 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd5b8000e40 con 0x55e06e786250 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/1816615117 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5b8005500 con 0x55e06e786250 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5b0ff9640 1 -- 172.21.15.94:0/1816615117 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5b80130d0 con 0x55e06e786250 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5d1fa4740 10 client.18174 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/1816615117 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e06e6ae8c0 con 0x55e06e786250 2024-02-17T12:08:20.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5b0ff9640 1 -- 172.21.15.94:0/1816615117 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5b8005500 con 0x55e06e786250 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5b0ff9640 1 -- 172.21.15.94:0/1816615117 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd5b8020460 con 0x55e06e786250 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5d1fa4740 10 client.18174 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5d1fa4740 10 client.18174 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5d1fa4740 20 client.18174 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/1816615117 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55e06e7f75b0 0x55e06e7f99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5d1fa4740 4 client.18174 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/1816615117 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e06e479890 con 0x55e06e7f75b0 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.928+0000 7fd5cd090640 1 --2- 172.21.15.94:0/1816615117 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55e06e7f75b0 0x55e06e7f99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5b0ff9640 10 client.18174 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5cd090640 1 --2- 172.21.15.94:0/1816615117 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55e06e7f75b0 0x55e06e7f99d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a26d605debf06753 server_cookie=292f36af90a24ac6 in_seq=0 out_seq=0 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5b0ff9640 1 -- 172.21.15.94:0/1816615117 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e06e479890 con 0x55e06e7f75b0 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5b0ff9640 10 client.18174 handle_command_reply: tid=0 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5d1fa4740 1 client.18174 shutdown 2024-02-17T12:08:20.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/1816615117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786250 msgr2=0x55e06e796820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/1816615117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786250 0x55e06e796820 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8013fd0 tx=0x7fd5b8009c30 comp rx=0 tx=0).stop 2024-02-17T12:08:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/1816615117 shutdown_connections 2024-02-17T12:08:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/1816615117 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55e06e7f75b0 0x55e06e7f99d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/1816615117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55e06e78f750 0x55e06e795100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/1816615117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e06e78aff0 0x55e06e794bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5b0ff9640 0 client.18174 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:08:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5d1fa4740 1 --2- 172.21.15.94:0/1816615117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55e06e786250 0x55e06e796820 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/1816615117 >> 172.21.15.94:0/1816615117 conn(0x55e06e780980 msgr2=0x55e06e7a6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/1816615117 shutdown_connections 2024-02-17T12:08:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5d1fa4740 1 -- 172.21.15.94:0/1816615117 wait complete. 2024-02-17T12:08:20.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.929+0000 7fd5d1fa4740 20 client.18174 trim_cache size 0 max 16384 2024-02-17T12:08:20.937 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:08:20.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.930+0000 7fd5cf2f4640 1 -- 172.21.15.94:0/2568659763 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5b40a2190 msgr2=0x7fd5b40a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.930+0000 7fd5cf2f4640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5b40a2190 0x7fd5b40a4650 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400e090 tx=0x7fd5c4032d70 comp rx=0 tx=0).stop 2024-02-17T12:08:20.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.930+0000 7fd5cf2f4640 1 -- 172.21.15.94:0/2568659763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5c806b840 msgr2=0x7fd5c806bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:20.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.930+0000 7fd5cf2f4640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5c806b840 0x7fd5c806bcf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400b9f0 tx=0x7fd5c4032af0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.930+0000 7fd5cf2f4640 1 -- 172.21.15.94:0/2568659763 shutdown_connections 2024-02-17T12:08:20.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.930+0000 7fd5cf2f4640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd5b40a2190 0x7fd5b40a4650 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.930+0000 7fd5cf2f4640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd5c806b840 0x7fd5c806bcf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.930+0000 7fd5cf2f4640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd5c8105330 0x7fd5c8072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.930+0000 7fd5cf2f4640 1 --2- 172.21.15.94:0/2568659763 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd5c8100260 0x7fd5c80727c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:20.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.930+0000 7fd5cf2f4640 1 -- 172.21.15.94:0/2568659763 >> 172.21.15.94:0/2568659763 conn(0x7fd5c8077180 msgr2=0x7fd5c80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:20.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.930+0000 7fd5cf2f4640 1 -- 172.21.15.94:0/2568659763 shutdown_connections 2024-02-17T12:08:20.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:20.930+0000 7fd5cf2f4640 1 -- 172.21.15.94:0/2568659763 wait complete. 2024-02-17T12:08:20.939 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:20 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1213178984' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:08:20.945 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:08:20.945 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:08:21.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.081+0000 7fd3bf8ec640 1 -- 172.21.15.94:0/707654087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3b811a960 msgr2=0x7fd3b811cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.081+0000 7fd3bf8ec640 1 --2- 172.21.15.94:0/707654087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3b811a960 0x7fd3b811cd50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009be0 tx=0x7fd3a802bf10 comp rx=0 tx=0).stop 2024-02-17T12:08:21.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.081+0000 7fd3bf8ec640 1 -- 172.21.15.94:0/707654087 shutdown_connections 2024-02-17T12:08:21.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.081+0000 7fd3bf8ec640 1 --2- 172.21.15.94:0/707654087 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3b811a960 0x7fd3b811cd50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.081+0000 7fd3bf8ec640 1 --2- 172.21.15.94:0/707654087 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b810d9d0 0x7fd3b810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.081+0000 7fd3bf8ec640 1 --2- 172.21.15.94:0/707654087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3b810f9a0 0x7fd3b810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.081+0000 7fd3bf8ec640 1 -- 172.21.15.94:0/707654087 >> 172.21.15.94:0/707654087 conn(0x7fd3b806c260 msgr2=0x7fd3b806c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:21.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.082+0000 7fd3bf8ec640 1 -- 172.21.15.94:0/707654087 shutdown_connections 2024-02-17T12:08:21.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.082+0000 7fd3bf8ec640 1 -- 172.21.15.94:0/707654087 wait complete. 2024-02-17T12:08:21.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.082+0000 7fd3bf8ec640 1 Processor -- start 2024-02-17T12:08:21.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bf8ec640 1 -- start start 2024-02-17T12:08:21.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bf8ec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b810d9d0 0x7fd3b81a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bf8ec640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3b810f9a0 0x7fd3b81a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bf8ec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3b811a960 0x7fd3b81aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bf8ec640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b81ab450 con 0x7fd3b810d9d0 2024-02-17T12:08:21.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bf8ec640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b81ab5c0 con 0x7fd3b811a960 2024-02-17T12:08:21.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bf8ec640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b81ab780 con 0x7fd3b810f9a0 2024-02-17T12:08:21.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bd688640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b810d9d0 0x7fd3b81a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bd688640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b810d9d0 0x7fd3b81a6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47124/0 (socket says 172.21.15.94:47124) 2024-02-17T12:08:21.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bd688640 1 -- 172.21.15.94:0/3981893933 learned_addr learned my addr 172.21.15.94:0/3981893933 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:21.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bde89640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3b811a960 0x7fd3b81aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bce87640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3b810f9a0 0x7fd3b81a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bde89640 1 -- 172.21.15.94:0/3981893933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3b810f9a0 msgr2=0x7fd3b81a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bde89640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3b810f9a0 0x7fd3b81a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bde89640 1 -- 172.21.15.94:0/3981893933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b810d9d0 msgr2=0x7fd3b81a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bde89640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b810d9d0 0x7fd3b81a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.083+0000 7fd3bde89640 1 -- 172.21.15.94:0/3981893933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac009560 con 0x7fd3b811a960 2024-02-17T12:08:21.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.084+0000 7fd3bde89640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3b811a960 0x7fd3b81aae50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8006010 tx=0x7fd3a8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:21.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.084+0000 7fd3b67fc640 1 -- 172.21.15.94:0/3981893933 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8003e40 con 0x7fd3b811a960 2024-02-17T12:08:21.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.084+0000 7fd3b67fc640 1 -- 172.21.15.94:0/3981893933 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3a8004730 con 0x7fd3b811a960 2024-02-17T12:08:21.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.084+0000 7fd3b67fc640 1 -- 172.21.15.94:0/3981893933 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8003e40 con 0x7fd3b811a960 2024-02-17T12:08:21.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.085+0000 7fd3bf8ec640 1 -- 172.21.15.94:0/3981893933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a8009840 con 0x7fd3b811a960 2024-02-17T12:08:21.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.085+0000 7fd3bf8ec640 1 -- 172.21.15.94:0/3981893933 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b81b3100 con 0x7fd3b811a960 2024-02-17T12:08:21.089 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.088+0000 7fd3b67fc640 1 -- 172.21.15.94:0/3981893933 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd3a8034bd0 con 0x7fd3b811a960 2024-02-17T12:08:21.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.088+0000 7fd3b67fc640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd3a00a2320 0x7fd3a00a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.088+0000 7fd3b67fc640 1 -- 172.21.15.94:0/3981893933 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd3a80f0600 con 0x7fd3b811a960 2024-02-17T12:08:21.090 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.088+0000 7fd3bd688640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd3a00a2320 0x7fd3a00a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.089+0000 7fd3bd688640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd3a00a2320 0x7fd3a00a47e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0006fd0 tx=0x7fd3b0008040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.089+0000 7fd3c2595740 1 Processor -- start 2024-02-17T12:08:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.089+0000 7fd3c2595740 1 -- start start 2024-02-17T12:08:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.089+0000 7fd3c2595740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef969250 0x55dcef969630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.089+0000 7fd3c2595740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef972210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.089+0000 7fd3c2595740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef972750 0x55dcef974b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.089+0000 7fd3c2595740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcef924960 con 0x55dcef972750 2024-02-17T12:08:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.089+0000 7fd3c2595740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcef8d8310 con 0x55dcef969b70 2024-02-17T12:08:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.089+0000 7fd3c2595740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcef8d5840 con 0x55dcef969250 2024-02-17T12:08:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd3bde89640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef972750 0x55dcef974b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd3bde89640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef972750 0x55dcef974b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47128/0 (socket says 172.21.15.94:47128) 2024-02-17T12:08:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd3bde89640 1 -- 172.21.15.94:0/415324421 learned_addr learned my addr 172.21.15.94:0/415324421 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd3bce87640 1 --2- 172.21.15.94:0/415324421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef972210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd3bd688640 1 --2- 172.21.15.94:0/415324421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef969250 0x55dcef969630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd3bde89640 1 -- 172.21.15.94:0/415324421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef969250 msgr2=0x55dcef969630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd3bde89640 1 --2- 172.21.15.94:0/415324421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef969250 0x55dcef969630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd3bde89640 1 -- 172.21.15.94:0/415324421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 msgr2=0x55dcef972210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd3bde89640 1 --2- 172.21.15.94:0/415324421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef972210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd3bde89640 1 -- 172.21.15.94:0/415324421 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcef893060 con 0x55dcef972750 2024-02-17T12:08:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd3bde89640 1 --2- 172.21.15.94:0/415324421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef972750 0x55dcef974b40 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80b9a90 tx=0x7fd3a80bcc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4824679580090 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd39f7fe640 1 -- 172.21.15.94:0/415324421 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8003e40 con 0x55dcef972750 2024-02-17T12:08:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd39f7fe640 1 -- 172.21.15.94:0/415324421 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3a80bd0c0 con 0x55dcef972750 2024-02-17T12:08:21.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.090+0000 7fd39f7fe640 1 -- 172.21.15.94:0/415324421 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a80bb4a0 con 0x55dcef972750 2024-02-17T12:08:21.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.091+0000 7fd3c2595740 1 -- 172.21.15.94:0/415324421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef972750 msgr2=0x55dcef974b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.091+0000 7fd3c2595740 1 --2- 172.21.15.94:0/415324421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef972750 0x55dcef974b40 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80b9a90 tx=0x7fd3a80bcc00 comp rx=0 tx=0).stop 2024-02-17T12:08:21.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.091+0000 7fd3c2595740 1 -- 172.21.15.94:0/415324421 shutdown_connections 2024-02-17T12:08:21.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.091+0000 7fd3c2595740 1 --2- 172.21.15.94:0/415324421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef972750 0x55dcef974b40 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.091+0000 7fd3c2595740 1 --2- 172.21.15.94:0/415324421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef972210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.091+0000 7fd3c2595740 1 --2- 172.21.15.94:0/415324421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef969250 0x55dcef969630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.091+0000 7fd3c2595740 1 -- 172.21.15.94:0/415324421 >> 172.21.15.94:0/415324421 conn(0x55dcef963980 msgr2=0x55dcef963d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:21.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.091+0000 7fd3c2595740 1 -- 172.21.15.94:0/415324421 shutdown_connections 2024-02-17T12:08:21.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.091+0000 7fd3c2595740 1 -- 172.21.15.94:0/415324421 wait complete. 2024-02-17T12:08:21.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.092+0000 7fd3c2595740 1 -- start start 2024-02-17T12:08:21.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.092+0000 7fd3c2595740 1 Processor -- start 2024-02-17T12:08:21.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.092+0000 7fd3c2595740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969250 0x55dcef96f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.092+0000 7fd3c2595740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef96fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.092+0000 7fd3c2595740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef972750 0x55dcef970260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3bd688640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969250 0x55dcef96f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3bd688640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969250 0x55dcef96f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47136/0 (socket says 172.21.15.94:47136) 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3bce87640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef96fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3bd688640 1 -- 172.21.15.94:0/931921969 learned_addr learned my addr 172.21.15.94:0/931921969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3c2595740 1 -- 172.21.15.94:0/931921969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcef8d1020 con 0x55dcef969250 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3c2595740 1 -- 172.21.15.94:0/931921969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcef5a4430 con 0x55dcef969b70 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3c2595740 1 -- 172.21.15.94:0/931921969 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcef560e00 con 0x55dcef972750 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3c2595740 10 client.0 _ll_register_callbacks cb 0x55dcef878cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3bde89640 1 --2- 172.21.15.94:0/931921969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef972750 0x55dcef970260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3bce87640 1 -- 172.21.15.94:0/931921969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef972750 msgr2=0x55dcef970260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3bce87640 1 --2- 172.21.15.94:0/931921969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef972750 0x55dcef970260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3bce87640 1 -- 172.21.15.94:0/931921969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969250 msgr2=0x55dcef96f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3bce87640 1 --2- 172.21.15.94:0/931921969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969250 0x55dcef96f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3bce87640 1 -- 172.21.15.94:0/931921969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b81b3100 con 0x55dcef969b70 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3bde89640 1 --2- 172.21.15.94:0/931921969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef972750 0x55dcef970260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3bce87640 1 --2- 172.21.15.94:0/931921969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef96fd20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd3b811ce30 tx=0x7fd3ac012d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19820c306bd4b356 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd39cff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd39cff9640 1 -- 172.21.15.94:0/931921969 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac017070 con 0x55dcef969b70 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd39cff9640 1 -- 172.21.15.94:0/931921969 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd3ac004020 con 0x55dcef969b70 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3c2595740 1 -- 172.21.15.94:0/931921969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcef5d07e0 con 0x55dcef969b70 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd39cff9640 1 -- 172.21.15.94:0/931921969 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3ac0041e0 con 0x55dcef969b70 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3c2595740 10 client.27668 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.093+0000 7fd3c2595740 1 -- 172.21.15.94:0/931921969 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dcef8cf6a0 con 0x55dcef969b70 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.094+0000 7fd39cff9640 1 -- 172.21.15.94:0/931921969 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3ac028080 con 0x55dcef969b70 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.094+0000 7fd39cff9640 1 -- 172.21.15.94:0/931921969 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd3ac0039c0 con 0x55dcef969b70 2024-02-17T12:08:21.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.094+0000 7fd3c2595740 10 client.27668 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.094+0000 7fd3c2595740 10 client.27668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.094+0000 7fd3c2595740 20 client.27668 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.094+0000 7fd3c2595740 1 --2- 172.21.15.94:0/931921969 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55dcef9827a0 0x55dcef984bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.094+0000 7fd3c2595740 4 client.27668 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.094+0000 7fd3c2595740 1 -- 172.21.15.94:0/931921969 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcef60d720 con 0x55dcef9827a0 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.094+0000 7fd3bd688640 1 --2- 172.21.15.94:0/931921969 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55dcef9827a0 0x55dcef984bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.094+0000 7fd3bd688640 1 --2- 172.21.15.94:0/931921969 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55dcef9827a0 0x55dcef984bc0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b848af0c769da8b server_cookie=6264b8e9fbbeded0 in_seq=0 out_seq=0 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.094+0000 7fd39cff9640 10 client.27668 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd39cff9640 1 -- 172.21.15.94:0/931921969 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dcef60d720 con 0x55dcef9827a0 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd39cff9640 10 client.27668 handle_command_reply: tid=0 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd3c2595740 1 client.27668 shutdown 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd3c2595740 1 -- 172.21.15.94:0/931921969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 msgr2=0x55dcef96fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd3c2595740 1 --2- 172.21.15.94:0/931921969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef96fd20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd3b811ce30 tx=0x7fd3ac012d30 comp rx=0 tx=0).stop 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd3c2595740 1 -- 172.21.15.94:0/931921969 shutdown_connections 2024-02-17T12:08:21.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd3c2595740 1 --2- 172.21.15.94:0/931921969 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55dcef9827a0 0x55dcef984bc0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd39cff9640 0 client.27668 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd3c2595740 1 --2- 172.21.15.94:0/931921969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef972750 0x55dcef970260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd3c2595740 1 --2- 172.21.15.94:0/931921969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef96fd20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd3c2595740 1 --2- 172.21.15.94:0/931921969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969250 0x55dcef96f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd3c2595740 1 -- 172.21.15.94:0/931921969 >> 172.21.15.94:0/931921969 conn(0x55dcef963980 msgr2=0x55dcef9656d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd3c2595740 1 -- 172.21.15.94:0/931921969 shutdown_connections 2024-02-17T12:08:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd3c2595740 1 -- 172.21.15.94:0/931921969 wait complete. 2024-02-17T12:08:21.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.095+0000 7fd3c2595740 20 client.27668 trim_cache size 0 max 16384 2024-02-17T12:08:21.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.109+0000 7fd3c2595740 1 Processor -- start 2024-02-17T12:08:21.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3c2595740 1 -- start start 2024-02-17T12:08:21.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3c2595740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969250 0x55dcef971f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3c2595740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef95ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3c2595740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef972750 0x55dcef96ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3c2595740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcef856130 con 0x55dcef969250 2024-02-17T12:08:21.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3c2595740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcef964980 con 0x55dcef969b70 2024-02-17T12:08:21.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3c2595740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcef968c90 con 0x55dcef972750 2024-02-17T12:08:21.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3bd688640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969250 0x55dcef971f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3bce87640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef95ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3bce87640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef95ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34918/0 (socket says 172.21.15.94:34918) 2024-02-17T12:08:21.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3bce87640 1 -- 172.21.15.94:0/1817687866 learned_addr learned my addr 172.21.15.94:0/1817687866 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:21.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3bde89640 1 --2- 172.21.15.94:0/1817687866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef972750 0x55dcef96ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3bce87640 1 -- 172.21.15.94:0/1817687866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef972750 msgr2=0x55dcef96ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3bce87640 1 --2- 172.21.15.94:0/1817687866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef972750 0x55dcef96ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3bce87640 1 -- 172.21.15.94:0/1817687866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969250 msgr2=0x55dcef971f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3bce87640 1 --2- 172.21.15.94:0/1817687866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969250 0x55dcef971f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3bce87640 1 -- 172.21.15.94:0/1817687866 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcef86bc20 con 0x55dcef969b70 2024-02-17T12:08:21.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.110+0000 7fd3bce87640 1 --2- 172.21.15.94:0/1817687866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef95ea10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac003d20 tx=0x7fd3ac0124c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=109bd6760bcab2cc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:21.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.111+0000 7fd39effd640 1 -- 172.21.15.94:0/1817687866 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac017070 con 0x55dcef969b70 2024-02-17T12:08:21.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.111+0000 7fd39effd640 1 -- 172.21.15.94:0/1817687866 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3ac018040 con 0x55dcef969b70 2024-02-17T12:08:21.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.111+0000 7fd39effd640 1 -- 172.21.15.94:0/1817687866 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac00f8f0 con 0x55dcef969b70 2024-02-17T12:08:21.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.111+0000 7fd3c2595740 1 -- 172.21.15.94:0/1817687866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 msgr2=0x55dcef95ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.111+0000 7fd3c2595740 1 --2- 172.21.15.94:0/1817687866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef95ea10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac003d20 tx=0x7fd3ac0124c0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.112+0000 7fd3c2595740 1 -- 172.21.15.94:0/1817687866 shutdown_connections 2024-02-17T12:08:21.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.112+0000 7fd3c2595740 1 --2- 172.21.15.94:0/1817687866 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef972750 0x55dcef96ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.112+0000 7fd3c2595740 1 --2- 172.21.15.94:0/1817687866 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef969b70 0x55dcef95ea10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.112+0000 7fd3c2595740 1 --2- 172.21.15.94:0/1817687866 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969250 0x55dcef971f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.112+0000 7fd3c2595740 1 -- 172.21.15.94:0/1817687866 >> 172.21.15.94:0/1817687866 conn(0x55dcef963980 msgr2=0x55dcef963d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:21.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.112+0000 7fd3c2595740 1 -- 172.21.15.94:0/1817687866 shutdown_connections 2024-02-17T12:08:21.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.112+0000 7fd3c2595740 1 -- 172.21.15.94:0/1817687866 wait complete. 2024-02-17T12:08:21.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.112+0000 7fd3c2595740 1 -- start start 2024-02-17T12:08:21.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.112+0000 7fd3c2595740 1 Processor -- start 2024-02-17T12:08:21.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.113+0000 7fd3c2595740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef969250 0x55dcef978170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.113+0000 7fd3c2595740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969b70 0x55dcef9786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.113+0000 7fd3bce87640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969b70 0x55dcef9786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.113+0000 7fd3bd688640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef969250 0x55dcef978170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.113+0000 7fd3bd688640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef969250 0x55dcef978170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57788/0 (socket says 172.21.15.94:57788) 2024-02-17T12:08:21.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.113+0000 7fd3bce87640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969b70 0x55dcef9786b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47152/0 (socket says 172.21.15.94:47152) 2024-02-17T12:08:21.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.113+0000 7fd3c2595740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef972750 0x55dcef9d8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.113+0000 7fd3bd688640 1 -- 172.21.15.94:0/2325465304 learned_addr learned my addr 172.21.15.94:0/2325465304 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:21.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.113+0000 7fd3c2595740 1 -- 172.21.15.94:0/2325465304 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcef978e40 con 0x55dcef969b70 2024-02-17T12:08:21.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.113+0000 7fd3c2595740 1 -- 172.21.15.94:0/2325465304 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcef978fb0 con 0x55dcef972750 2024-02-17T12:08:21.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.113+0000 7fd3c2595740 1 -- 172.21.15.94:0/2325465304 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcef9d9230 con 0x55dcef969250 2024-02-17T12:08:21.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd3c2595740 10 client.0 _ll_register_callbacks cb 0x7fd3a80048a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:21.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd3bde89640 1 --2- 172.21.15.94:0/2325465304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef972750 0x55dcef9d8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd3bd688640 1 -- 172.21.15.94:0/2325465304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef972750 msgr2=0x55dcef9d8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd3bd688640 1 --2- 172.21.15.94:0/2325465304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef972750 0x55dcef9d8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd3bd688640 1 -- 172.21.15.94:0/2325465304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969b70 msgr2=0x55dcef9786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd3bd688640 1 --2- 172.21.15.94:0/2325465304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969b70 0x55dcef9786b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd3bd688640 1 -- 172.21.15.94:0/2325465304 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dcef8cf6a0 con 0x55dcef969250 2024-02-17T12:08:21.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd3bce87640 1 --2- 172.21.15.94:0/2325465304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969b70 0x55dcef9786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:21.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd3bd688640 1 --2- 172.21.15.94:0/2325465304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef969250 0x55dcef978170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55dcef985510 tx=0x7fd3b00046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dda3c686e00a8a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:21.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd39cff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:08:21.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd39cff9640 1 -- 172.21.15.94:0/2325465304 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b0027040 con 0x55dcef969250 2024-02-17T12:08:21.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd3c2595740 1 -- 172.21.15.94:0/2325465304 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dcef96f610 con 0x55dcef969250 2024-02-17T12:08:21.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd39cff9640 1 -- 172.21.15.94:0/2325465304 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd3b0016b20 con 0x55dcef969250 2024-02-17T12:08:21.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd39cff9640 1 -- 172.21.15.94:0/2325465304 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3b0016ce0 con 0x55dcef969250 2024-02-17T12:08:21.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd3c2595740 10 client.27652 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:21.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.114+0000 7fd3c2595740 1 -- 172.21.15.94:0/2325465304 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dcef8918c0 con 0x55dcef969250 2024-02-17T12:08:21.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.115+0000 7fd39cff9640 1 -- 172.21.15.94:0/2325465304 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd3b0005e50 con 0x55dcef969250 2024-02-17T12:08:21.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.115+0000 7fd39cff9640 1 -- 172.21.15.94:0/2325465304 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fd3b001f040 con 0x55dcef969250 2024-02-17T12:08:21.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.115+0000 7fd3c2595740 10 client.27652 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:21.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.115+0000 7fd3c2595740 10 client.27652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:21.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.115+0000 7fd3c2595740 20 client.27652 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:21.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.115+0000 7fd3c2595740 1 --2- 172.21.15.94:0/2325465304 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55dcef9827a0 0x55dcef9dca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.115+0000 7fd3c2595740 4 client.27652 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:08:21.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.115+0000 7fd3c2595740 1 -- 172.21.15.94:0/2325465304 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dcef96f610 con 0x55dcef9827a0 2024-02-17T12:08:21.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.115+0000 7fd3bce87640 1 --2- 172.21.15.94:0/2325465304 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55dcef9827a0 0x55dcef9dca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.115+0000 7fd39cff9640 10 client.27652 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:21.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.115+0000 7fd3bce87640 1 --2- 172.21.15.94:0/2325465304 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55dcef9827a0 0x55dcef9dca00 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeca702a1decaa9f server_cookie=bf07e20340f1f0a7 in_seq=0 out_seq=0 2024-02-17T12:08:21.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.115+0000 7fd39cff9640 1 -- 172.21.15.94:0/2325465304 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dcef96f610 con 0x55dcef9827a0 2024-02-17T12:08:21.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.115+0000 7fd39cff9640 10 client.27652 handle_command_reply: tid=0 2024-02-17T12:08:21.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.116+0000 7fd3c2595740 1 client.27652 shutdown 2024-02-17T12:08:21.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.116+0000 7fd3c2595740 1 -- 172.21.15.94:0/2325465304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef969250 msgr2=0x55dcef978170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.116+0000 7fd3c2595740 1 --2- 172.21.15.94:0/2325465304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef969250 0x55dcef978170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55dcef985510 tx=0x7fd3b00046e0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.116+0000 7fd3c2595740 1 -- 172.21.15.94:0/2325465304 shutdown_connections 2024-02-17T12:08:21.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.116+0000 7fd3c2595740 1 --2- 172.21.15.94:0/2325465304 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55dcef9827a0 0x55dcef9dca00 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.116+0000 7fd39cff9640 0 client.27652 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:21.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.116+0000 7fd3c2595740 1 --2- 172.21.15.94:0/2325465304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dcef972750 0x55dcef9d8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.116+0000 7fd3c2595740 1 --2- 172.21.15.94:0/2325465304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dcef969b70 0x55dcef9786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.116+0000 7fd3c2595740 1 --2- 172.21.15.94:0/2325465304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcef969250 0x55dcef978170 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.116+0000 7fd3c2595740 1 -- 172.21.15.94:0/2325465304 >> 172.21.15.94:0/2325465304 conn(0x55dcef963980 msgr2=0x55dcef95ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:21.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.116+0000 7fd3c2595740 1 -- 172.21.15.94:0/2325465304 shutdown_connections 2024-02-17T12:08:21.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.116+0000 7fd3c2595740 1 -- 172.21.15.94:0/2325465304 wait complete. 2024-02-17T12:08:21.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.116+0000 7fd3c2595740 20 client.27652 trim_cache size 0 max 16384 2024-02-17T12:08:21.127 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:08:21.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.117+0000 7fd3bf8ec640 1 -- 172.21.15.94:0/3981893933 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd3a00a2320 msgr2=0x7fd3a00a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.117+0000 7fd3bf8ec640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd3a00a2320 0x7fd3a00a47e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0006fd0 tx=0x7fd3b0008040 comp rx=0 tx=0).stop 2024-02-17T12:08:21.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.117+0000 7fd3bf8ec640 1 -- 172.21.15.94:0/3981893933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3b811a960 msgr2=0x7fd3b81aae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.117+0000 7fd3bf8ec640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3b811a960 0x7fd3b81aae50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8006010 tx=0x7fd3a8003c10 comp rx=0 tx=0).stop 2024-02-17T12:08:21.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.117+0000 7fd3bf8ec640 1 -- 172.21.15.94:0/3981893933 shutdown_connections 2024-02-17T12:08:21.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.117+0000 7fd3bf8ec640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd3a00a2320 0x7fd3a00a47e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.117+0000 7fd3bf8ec640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd3b811a960 0x7fd3b81aae50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.117+0000 7fd3bf8ec640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd3b810f9a0 0x7fd3b81a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.117+0000 7fd3bf8ec640 1 --2- 172.21.15.94:0/3981893933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd3b810d9d0 0x7fd3b81a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.117+0000 7fd3bf8ec640 1 -- 172.21.15.94:0/3981893933 >> 172.21.15.94:0/3981893933 conn(0x7fd3b806c260 msgr2=0x7fd3b8071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:21.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.118+0000 7fd3bf8ec640 1 -- 172.21.15.94:0/3981893933 shutdown_connections 2024-02-17T12:08:21.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:21.118+0000 7fd3bf8ec640 1 -- 172.21.15.94:0/3981893933 wait complete. 2024-02-17T12:08:21.644 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.643+0000 7feb8affd640 1 -- 172.21.15.94:0/1035394053 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c02cbd0 con 0x7feb8c1000e0 2024-02-17T12:08:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.643+0000 7feb913ac640 1 -- 172.21.15.94:0/1035394053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb8c1000e0 msgr2=0x7feb8c1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.643+0000 7feb913ac640 1 --2- 172.21.15.94:0/1035394053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb8c1000e0 0x7feb8c1004c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009900 tx=0x7feb7c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:08:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.643+0000 7feb913ac640 1 -- 172.21.15.94:0/1035394053 shutdown_connections 2024-02-17T12:08:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.643+0000 7feb913ac640 1 --2- 172.21.15.94:0/1035394053 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb8c1051b0 0x7feb8c10cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.643+0000 7feb913ac640 1 --2- 172.21.15.94:0/1035394053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb8c100a90 0x7feb8c104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.643+0000 7feb913ac640 1 --2- 172.21.15.94:0/1035394053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb8c1000e0 0x7feb8c1004c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.643+0000 7feb913ac640 1 -- 172.21.15.94:0/1035394053 >> 172.21.15.94:0/1035394053 conn(0x7feb8c0fb9c0 msgr2=0x7feb8c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.643+0000 7feb913ac640 1 -- 172.21.15.94:0/1035394053 shutdown_connections 2024-02-17T12:08:21.645 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.643+0000 7feb913ac640 1 -- 172.21.15.94:0/1035394053 wait complete. 2024-02-17T12:08:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.644+0000 7feb913ac640 1 Processor -- start 2024-02-17T12:08:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.644+0000 7feb913ac640 1 -- start start 2024-02-17T12:08:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.644+0000 7feb913ac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb8c1000e0 0x7feb8c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.644+0000 7feb913ac640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb8c1051b0 0x7feb8c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.644+0000 7feb913ac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb8c199090 0x7feb8c1a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.644+0000 7feb913ac640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1a3ce0 con 0x7feb8c199090 2024-02-17T12:08:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.644+0000 7feb913ac640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1a3e50 con 0x7feb8c1000e0 2024-02-17T12:08:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.644+0000 7feb913ac640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8c1963b0 con 0x7feb8c1051b0 2024-02-17T12:08:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.644+0000 7feb90bab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb8c199090 0x7feb8c1a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.644+0000 7feb8b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb8c1051b0 0x7feb8c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.644+0000 7feb90bab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb8c199090 0x7feb8c1a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47182/0 (socket says 172.21.15.94:47182) 2024-02-17T12:08:21.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.644+0000 7feb90bab640 1 -- 172.21.15.94:0/2702158415 learned_addr learned my addr 172.21.15.94:0/2702158415 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.645+0000 7feb8bfff640 1 --2- 172.21.15.94:0/2702158415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb8c1000e0 0x7feb8c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.645+0000 7feb90bab640 1 -- 172.21.15.94:0/2702158415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb8c1051b0 msgr2=0x7feb8c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.645+0000 7feb90bab640 1 --2- 172.21.15.94:0/2702158415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb8c1051b0 0x7feb8c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.645+0000 7feb90bab640 1 -- 172.21.15.94:0/2702158415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb8c1000e0 msgr2=0x7feb8c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.645+0000 7feb90bab640 1 --2- 172.21.15.94:0/2702158415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb8c1000e0 0x7feb8c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.645+0000 7feb90bab640 1 -- 172.21.15.94:0/2702158415 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb7c009560 con 0x7feb8c199090 2024-02-17T12:08:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.645+0000 7feb90bab640 1 --2- 172.21.15.94:0/2702158415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb8c199090 0x7feb8c1a3640 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7feb8400eaa0 tx=0x7feb8400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.645+0000 7feb897fa640 1 -- 172.21.15.94:0/2702158415 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84004d20 con 0x7feb8c199090 2024-02-17T12:08:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.645+0000 7feb913ac640 1 -- 172.21.15.94:0/2702158415 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb8c196690 con 0x7feb8c199090 2024-02-17T12:08:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.645+0000 7feb913ac640 1 -- 172.21.15.94:0/2702158415 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8c196ba0 con 0x7feb8c199090 2024-02-17T12:08:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.645+0000 7feb897fa640 1 -- 172.21.15.94:0/2702158415 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb84004e80 con 0x7feb8c199090 2024-02-17T12:08:21.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.646+0000 7feb897fa640 1 -- 172.21.15.94:0/2702158415 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb84005a10 con 0x7feb8c199090 2024-02-17T12:08:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.647+0000 7feb897fa640 1 -- 172.21.15.94:0/2702158415 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7feb84011770 con 0x7feb8c199090 2024-02-17T12:08:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.648+0000 7feb897fa640 1 --2- 172.21.15.94:0/2702158415 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feb740a22a0 0x7feb740a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.648+0000 7feb951e7740 1 Processor -- start 2024-02-17T12:08:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb897fa640 1 -- 172.21.15.94:0/2702158415 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7feb840ce190 con 0x7feb8c199090 2024-02-17T12:08:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb951e7740 1 -- start start 2024-02-17T12:08:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb8bfff640 1 --2- 172.21.15.94:0/2702158415 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feb740a22a0 0x7feb740a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb951e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa853c60 0x5652aa8529b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb951e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa852ef0 0x5652aa853370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb951e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa85c470 0x5652aa85e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb951e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652aa7c19f0 con 0x5652aa85c470 2024-02-17T12:08:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb951e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652aa7bef20 con 0x5652aa852ef0 2024-02-17T12:08:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb951e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652aa7ba700 con 0x5652aa853c60 2024-02-17T12:08:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb90bab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa85c470 0x5652aa85e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb90bab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa85c470 0x5652aa85e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47190/0 (socket says 172.21.15.94:47190) 2024-02-17T12:08:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb90bab640 1 -- 172.21.15.94:0/164897909 learned_addr learned my addr 172.21.15.94:0/164897909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb8b7fe640 1 --2- 172.21.15.94:0/164897909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa852ef0 0x5652aa853370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb8bfff640 1 --2- 172.21.15.94:0/2702158415 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feb740a22a0 0x7feb740a4760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb7c005ab0 tx=0x7feb7c005a20 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb90bab640 1 -- 172.21.15.94:0/164897909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa853c60 msgr2=0x5652aa8529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb8bfff640 1 --2- 172.21.15.94:0/164897909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa853c60 0x5652aa8529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb90bab640 1 --2- 172.21.15.94:0/164897909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa853c60 0x5652aa8529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb90bab640 1 -- 172.21.15.94:0/164897909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa852ef0 msgr2=0x5652aa853370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb90bab640 1 --2- 172.21.15.94:0/164897909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa852ef0 0x5652aa853370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb90bab640 1 -- 172.21.15.94:0/164897909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652aa7507b0 con 0x5652aa85c470 2024-02-17T12:08:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.649+0000 7feb8bfff640 1 --2- 172.21.15.94:0/164897909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa853c60 0x5652aa8529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:08:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.650+0000 7feb90bab640 1 --2- 172.21.15.94:0/164897909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa85c470 0x5652aa85e860 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7feb84005780 tx=0x7feb8409ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e9feed21174a82 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.650+0000 7feb727fc640 1 -- 172.21.15.94:0/164897909 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb840974f0 con 0x5652aa85c470 2024-02-17T12:08:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.650+0000 7feb727fc640 1 -- 172.21.15.94:0/164897909 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb840974f0 con 0x5652aa85c470 2024-02-17T12:08:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.650+0000 7feb727fc640 1 -- 172.21.15.94:0/164897909 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb840aec60 con 0x5652aa85c470 2024-02-17T12:08:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.652+0000 7feb951e7740 1 -- 172.21.15.94:0/164897909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa85c470 msgr2=0x5652aa85e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.652+0000 7feb951e7740 1 --2- 172.21.15.94:0/164897909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa85c470 0x5652aa85e860 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7feb84005780 tx=0x7feb8409ab40 comp rx=0 tx=0).stop 2024-02-17T12:08:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.652+0000 7feb951e7740 1 -- 172.21.15.94:0/164897909 shutdown_connections 2024-02-17T12:08:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.652+0000 7feb951e7740 1 --2- 172.21.15.94:0/164897909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa85c470 0x5652aa85e860 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.652+0000 7feb951e7740 1 --2- 172.21.15.94:0/164897909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa852ef0 0x5652aa853370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.652+0000 7feb951e7740 1 --2- 172.21.15.94:0/164897909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa853c60 0x5652aa8529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.652+0000 7feb951e7740 1 -- 172.21.15.94:0/164897909 >> 172.21.15.94:0/164897909 conn(0x5652aa81a550 msgr2=0x5652aa84dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.653+0000 7feb951e7740 1 -- 172.21.15.94:0/164897909 shutdown_connections 2024-02-17T12:08:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.653+0000 7feb951e7740 1 -- 172.21.15.94:0/164897909 wait complete. 2024-02-17T12:08:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.654+0000 7feb951e7740 1 -- start start 2024-02-17T12:08:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.654+0000 7feb951e7740 1 Processor -- start 2024-02-17T12:08:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.654+0000 7feb951e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa859420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.654+0000 7feb951e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa853c60 0x5652aa859960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.654+0000 7feb951e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa85c470 0x5652aa859ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.654+0000 7feb951e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652aa75fbf0 con 0x5652aa85c470 2024-02-17T12:08:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.654+0000 7feb90bab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa85c470 0x5652aa859ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb951e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652aa48e250 con 0x5652aa853c60 2024-02-17T12:08:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb8b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa853c60 0x5652aa859960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb951e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652aa740200 con 0x5652aa852ef0 2024-02-17T12:08:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb8b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa853c60 0x5652aa859960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34966/0 (socket says 172.21.15.94:34966) 2024-02-17T12:08:21.662 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb8b7fe640 1 -- 172.21.15.94:0/3991709044 learned_addr learned my addr 172.21.15.94:0/3991709044 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb90bab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa85c470 0x5652aa859ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47198/0 (socket says 172.21.15.94:47198) 2024-02-17T12:08:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb8bfff640 1 --2- 172.21.15.94:0/3991709044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa859420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb951e7740 10 client.0 _ll_register_callbacks cb 0x5652aa7aed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb8b7fe640 1 -- 172.21.15.94:0/3991709044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 msgr2=0x5652aa859420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb8b7fe640 1 --2- 172.21.15.94:0/3991709044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa859420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb8b7fe640 1 -- 172.21.15.94:0/3991709044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa85c470 msgr2=0x5652aa859ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb8b7fe640 1 --2- 172.21.15.94:0/3991709044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa85c470 0x5652aa859ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb8b7fe640 1 -- 172.21.15.94:0/3991709044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb8c196ba0 con 0x5652aa853c60 2024-02-17T12:08:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb8b7fe640 1 --2- 172.21.15.94:0/3991709044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa853c60 0x5652aa859960 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7feb80011570 tx=0x7feb80011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8843c6be924d45e2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:08:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb6b7fe640 1 -- 172.21.15.94:0/3991709044 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb80013670 con 0x5652aa853c60 2024-02-17T12:08:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb951e7740 1 -- 172.21.15.94:0/3991709044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652aa4ba600 con 0x5652aa853c60 2024-02-17T12:08:21.663 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb6b7fe640 1 -- 172.21.15.94:0/3991709044 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7feb800137d0 con 0x5652aa853c60 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.655+0000 7feb6b7fe640 1 -- 172.21.15.94:0/3991709044 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb800139d0 con 0x5652aa853c60 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.656+0000 7feb951e7740 10 client.27686 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.656+0000 7feb951e7740 1 -- 172.21.15.94:0/3991709044 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5652aa7b8d80 con 0x5652aa853c60 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.656+0000 7feb6b7fe640 1 -- 172.21.15.94:0/3991709044 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5652aa4ba600 con 0x5652aa853c60 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.656+0000 7feb6b7fe640 1 -- 172.21.15.94:0/3991709044 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feb80023070 con 0x5652aa853c60 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.656+0000 7feb951e7740 10 client.27686 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.656+0000 7feb951e7740 10 client.27686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.656+0000 7feb951e7740 20 client.27686 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.656+0000 7feb951e7740 1 --2- 172.21.15.94:0/3991709044 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5652aa86c900 0x5652aa86ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.656+0000 7feb951e7740 4 client.27686 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.656+0000 7feb951e7740 1 -- 172.21.15.94:0/3991709044 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652aa50b1b0 con 0x5652aa86c900 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.656+0000 7feb8bfff640 1 --2- 172.21.15.94:0/3991709044 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5652aa86c900 0x5652aa86ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb8bfff640 1 --2- 172.21.15.94:0/3991709044 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5652aa86c900 0x5652aa86ed20 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bf05868d0b910ba server_cookie=ad402b186a9cbd7c in_seq=0 out_seq=0 2024-02-17T12:08:21.664 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb6b7fe640 10 client.27686 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb6b7fe640 1 -- 172.21.15.94:0/3991709044 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5652aa50b1b0 con 0x5652aa86c900 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb6b7fe640 10 client.27686 handle_command_reply: tid=0 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb951e7740 1 client.27686 shutdown 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb951e7740 1 -- 172.21.15.94:0/3991709044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa853c60 msgr2=0x5652aa859960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb951e7740 1 --2- 172.21.15.94:0/3991709044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa853c60 0x5652aa859960 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7feb80011570 tx=0x7feb80011a30 comp rx=0 tx=0).stop 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb951e7740 1 -- 172.21.15.94:0/3991709044 shutdown_connections 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb951e7740 1 --2- 172.21.15.94:0/3991709044 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5652aa86c900 0x5652aa86ed20 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb951e7740 1 --2- 172.21.15.94:0/3991709044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa85c470 0x5652aa859ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb6b7fe640 0 client.27686 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb951e7740 1 --2- 172.21.15.94:0/3991709044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa853c60 0x5652aa859960 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb951e7740 1 --2- 172.21.15.94:0/3991709044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa859420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.657+0000 7feb951e7740 1 -- 172.21.15.94:0/3991709044 >> 172.21.15.94:0/3991709044 conn(0x5652aa81a550 msgr2=0x5652aa84f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.658+0000 7feb951e7740 1 -- 172.21.15.94:0/3991709044 shutdown_connections 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.658+0000 7feb951e7740 1 -- 172.21.15.94:0/3991709044 wait complete. 2024-02-17T12:08:21.665 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.658+0000 7feb951e7740 20 client.27686 trim_cache size 0 max 16384 2024-02-17T12:08:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb951e7740 1 Processor -- start 2024-02-17T12:08:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb951e7740 1 -- start start 2024-02-17T12:08:21.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb951e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa857cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb951e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa853c60 0x5652aa858230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb951e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa85c470 0x5652aa856680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb951e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652aa84e160 con 0x5652aa853c60 2024-02-17T12:08:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb951e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652aa85a770 con 0x5652aa85c470 2024-02-17T12:08:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb951e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652aa858770 con 0x5652aa852ef0 2024-02-17T12:08:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb8b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa853c60 0x5652aa858230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb8bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa857cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb8b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa853c60 0x5652aa858230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47206/0 (socket says 172.21.15.94:47206) 2024-02-17T12:08:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb8b7fe640 1 -- 172.21.15.94:0/24075035 learned_addr learned my addr 172.21.15.94:0/24075035 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb90bab640 1 --2- 172.21.15.94:0/24075035 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa85c470 0x5652aa856680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb8b7fe640 1 -- 172.21.15.94:0/24075035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 msgr2=0x5652aa857cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb8b7fe640 1 --2- 172.21.15.94:0/24075035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa857cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb8b7fe640 1 -- 172.21.15.94:0/24075035 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa85c470 msgr2=0x5652aa856680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb8b7fe640 1 --2- 172.21.15.94:0/24075035 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa85c470 0x5652aa856680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.676+0000 7feb8b7fe640 1 -- 172.21.15.94:0/24075035 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652aa7557b0 con 0x5652aa853c60 2024-02-17T12:08:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.677+0000 7feb8b7fe640 1 --2- 172.21.15.94:0/24075035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa853c60 0x5652aa858230 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7feb80006e40 tx=0x7feb80006e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294763ea76c48eed server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.677+0000 7feb71ffb640 1 -- 172.21.15.94:0/24075035 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb80013240 con 0x5652aa853c60 2024-02-17T12:08:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.677+0000 7feb71ffb640 1 -- 172.21.15.94:0/24075035 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb80024cd0 con 0x5652aa853c60 2024-02-17T12:08:21.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.677+0000 7feb71ffb640 1 -- 172.21.15.94:0/24075035 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb80014a70 con 0x5652aa853c60 2024-02-17T12:08:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.677+0000 7feb951e7740 1 -- 172.21.15.94:0/24075035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa853c60 msgr2=0x5652aa858230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.677+0000 7feb951e7740 1 --2- 172.21.15.94:0/24075035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa853c60 0x5652aa858230 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7feb80006e40 tx=0x7feb80006e70 comp rx=0 tx=0).stop 2024-02-17T12:08:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.678+0000 7feb951e7740 1 -- 172.21.15.94:0/24075035 shutdown_connections 2024-02-17T12:08:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.678+0000 7feb951e7740 1 --2- 172.21.15.94:0/24075035 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa85c470 0x5652aa856680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.678+0000 7feb951e7740 1 --2- 172.21.15.94:0/24075035 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa853c60 0x5652aa858230 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.678+0000 7feb951e7740 1 --2- 172.21.15.94:0/24075035 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa857cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.678+0000 7feb951e7740 1 -- 172.21.15.94:0/24075035 >> 172.21.15.94:0/24075035 conn(0x5652aa81a550 msgr2=0x5652aa854710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.678+0000 7feb951e7740 1 -- 172.21.15.94:0/24075035 shutdown_connections 2024-02-17T12:08:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.678+0000 7feb951e7740 1 -- 172.21.15.94:0/24075035 wait complete. 2024-02-17T12:08:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.678+0000 7feb951e7740 1 -- start start 2024-02-17T12:08:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.678+0000 7feb951e7740 1 Processor -- start 2024-02-17T12:08:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.679+0000 7feb951e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa863510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.679+0000 7feb951e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa853c60 0x5652aa8618b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.679+0000 7feb8bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa863510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.679+0000 7feb8bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa863510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57850/0 (socket says 172.21.15.94:57850) 2024-02-17T12:08:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.679+0000 7feb8bfff640 1 -- 172.21.15.94:0/480424521 learned_addr learned my addr 172.21.15.94:0/480424521 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.679+0000 7feb951e7740 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa85c470 0x5652aa861df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.679+0000 7feb951e7740 1 -- 172.21.15.94:0/480424521 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652aa862330 con 0x5652aa853c60 2024-02-17T12:08:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.679+0000 7feb951e7740 1 -- 172.21.15.94:0/480424521 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652aa862470 con 0x5652aa85c470 2024-02-17T12:08:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.679+0000 7feb951e7740 1 -- 172.21.15.94:0/480424521 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652aa862630 con 0x5652aa852ef0 2024-02-17T12:08:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.680+0000 7feb951e7740 10 client.0 _ll_register_callbacks cb 0x7feb8409a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.688+0000 7feb8b7fe640 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa853c60 0x5652aa8618b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.692+0000 7feb90bab640 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa85c470 0x5652aa861df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.695+0000 7feb8bfff640 1 -- 172.21.15.94:0/480424521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa85c470 msgr2=0x5652aa861df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.695+0000 7feb8bfff640 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa85c470 0x5652aa861df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.695+0000 7feb8bfff640 1 -- 172.21.15.94:0/480424521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa853c60 msgr2=0x5652aa8618b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.695+0000 7feb8bfff640 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa853c60 0x5652aa8618b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.695+0000 7feb8bfff640 1 -- 172.21.15.94:0/480424521 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652aa7b8d80 con 0x5652aa852ef0 2024-02-17T12:08:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.703+0000 7feb8bfff640 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa863510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feb7c006010 tx=0x7feb7c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a77936b7f00cb22a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.704+0000 7feb6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:08:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.709+0000 7feb6b7fe640 1 -- 172.21.15.94:0/480424521 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c052ea0 con 0x5652aa852ef0 2024-02-17T12:08:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.709+0000 7feb6b7fe640 1 -- 172.21.15.94:0/480424521 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7feb7c047040 con 0x5652aa852ef0 2024-02-17T12:08:21.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.709+0000 7feb951e7740 1 -- 172.21.15.94:0/480424521 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb840974f0 con 0x5652aa852ef0 2024-02-17T12:08:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.713+0000 7feb6b7fe640 1 -- 172.21.15.94:0/480424521 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb7c02dea0 con 0x5652aa852ef0 2024-02-17T12:08:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.715+0000 7feb951e7740 10 client.27667 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.715+0000 7feb951e7740 1 -- 172.21.15.94:0/480424521 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5652aa77b540 con 0x5652aa852ef0 2024-02-17T12:08:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.718+0000 7feb6b7fe640 1 -- 172.21.15.94:0/480424521 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feb840974f0 con 0x5652aa852ef0 2024-02-17T12:08:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.718+0000 7feb6b7fe640 1 -- 172.21.15.94:0/480424521 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feb7c005300 con 0x5652aa852ef0 2024-02-17T12:08:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.718+0000 7feb951e7740 10 client.27667 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.718+0000 7feb951e7740 10 client.27667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.718+0000 7feb951e7740 20 client.27667 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.718+0000 7feb951e7740 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5652aa8c43a0 0x5652aa8c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.718+0000 7feb951e7740 4 client.27667 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:08:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.718+0000 7feb951e7740 1 -- 172.21.15.94:0/480424521 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5652aa4f7550 con 0x5652aa8c43a0 2024-02-17T12:08:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.725+0000 7feb8b7fe640 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5652aa8c43a0 0x5652aa8c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.730+0000 7feb8b7fe640 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5652aa8c43a0 0x5652aa8c67c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d6b6ebd1ab96b85 server_cookie=3ca3ed2ee9c25e16 in_seq=0 out_seq=0 2024-02-17T12:08:21.731 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.730+0000 7feb6b7fe640 10 client.27667 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.788+0000 7feb6b7fe640 1 -- 172.21.15.94:0/480424521 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5652aa4f7550 con 0x5652aa8c43a0 2024-02-17T12:08:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.788+0000 7feb6b7fe640 10 client.27667 handle_command_reply: tid=0 2024-02-17T12:08:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.788+0000 7feb951e7740 1 client.27667 shutdown 2024-02-17T12:08:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.789+0000 7feb951e7740 1 -- 172.21.15.94:0/480424521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 msgr2=0x5652aa863510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.789+0000 7feb951e7740 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa863510 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feb7c006010 tx=0x7feb7c009260 comp rx=0 tx=0).stop 2024-02-17T12:08:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.789+0000 7feb951e7740 1 -- 172.21.15.94:0/480424521 shutdown_connections 2024-02-17T12:08:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.789+0000 7feb951e7740 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5652aa8c43a0 0x5652aa8c67c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.789+0000 7feb951e7740 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5652aa85c470 0x5652aa861df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.789+0000 7feb951e7740 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5652aa853c60 0x5652aa8618b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.789+0000 7feb951e7740 1 --2- 172.21.15.94:0/480424521 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5652aa852ef0 0x5652aa863510 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.789+0000 7feb6b7fe640 0 client.27667 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.789+0000 7feb951e7740 1 -- 172.21.15.94:0/480424521 >> 172.21.15.94:0/480424521 conn(0x5652aa81a550 msgr2=0x5652aa857c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.789+0000 7feb951e7740 1 -- 172.21.15.94:0/480424521 shutdown_connections 2024-02-17T12:08:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.789+0000 7feb951e7740 1 -- 172.21.15.94:0/480424521 wait complete. 2024-02-17T12:08:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.789+0000 7feb951e7740 20 client.27667 trim_cache size 0 max 16384 2024-02-17T12:08:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:08:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:08:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:08:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:08:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.790+0000 7feb913ac640 1 -- 172.21.15.94:0/2702158415 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feb740a22a0 msgr2=0x7feb740a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.790+0000 7feb913ac640 1 --2- 172.21.15.94:0/2702158415 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feb740a22a0 0x7feb740a4760 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb7c005ab0 tx=0x7feb7c005a20 comp rx=0 tx=0).stop 2024-02-17T12:08:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.790+0000 7feb913ac640 1 -- 172.21.15.94:0/2702158415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb8c199090 msgr2=0x7feb8c1a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.790+0000 7feb913ac640 1 --2- 172.21.15.94:0/2702158415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb8c199090 0x7feb8c1a3640 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7feb8400eaa0 tx=0x7feb8400eea0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.791+0000 7feb913ac640 1 -- 172.21.15.94:0/2702158415 shutdown_connections 2024-02-17T12:08:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.791+0000 7feb913ac640 1 --2- 172.21.15.94:0/2702158415 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feb740a22a0 0x7feb740a4760 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.791+0000 7feb913ac640 1 --2- 172.21.15.94:0/2702158415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feb8c199090 0x7feb8c1a3640 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.791+0000 7feb913ac640 1 --2- 172.21.15.94:0/2702158415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb8c1051b0 0x7feb8c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.791+0000 7feb913ac640 1 --2- 172.21.15.94:0/2702158415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feb8c1000e0 0x7feb8c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.791+0000 7feb913ac640 1 -- 172.21.15.94:0/2702158415 >> 172.21.15.94:0/2702158415 conn(0x7feb8c0fb9c0 msgr2=0x7feb8c0fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.791+0000 7feb913ac640 1 -- 172.21.15.94:0/2702158415 shutdown_connections 2024-02-17T12:08:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:21.791+0000 7feb913ac640 1 -- 172.21.15.94:0/2702158415 wait complete. 2024-02-17T12:08:21.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:21 smithi189 ceph-mon[36588]: pgmap v318: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 742 op/s 2024-02-17T12:08:21.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:21 smithi125 ceph-mon[36819]: pgmap v318: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 742 op/s 2024-02-17T12:08:21.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:21 smithi094 ceph-mon[32499]: pgmap v318: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 742 op/s 2024-02-17T12:08:22.131 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:08:22.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:08:22.131 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:08:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.296+0000 7f7fadda0640 1 -- 172.21.15.94:0/2934033334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fa810b620 msgr2=0x7f7fa810da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.296+0000 7f7fadda0640 1 --2- 172.21.15.94:0/2934033334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fa810b620 0x7f7fa810da70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009860 tx=0x7f7fa002bc20 comp rx=0 tx=0).stop 2024-02-17T12:08:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.296+0000 7f7fadda0640 1 -- 172.21.15.94:0/2934033334 shutdown_connections 2024-02-17T12:08:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.296+0000 7f7fadda0640 1 --2- 172.21.15.94:0/2934033334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fa810b620 0x7f7fa810da70 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.296+0000 7f7fadda0640 1 --2- 172.21.15.94:0/2934033334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fa8102b70 0x7f7fa810b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.296+0000 7f7fadda0640 1 --2- 172.21.15.94:0/2934033334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fa8102250 0x7f7fa8102630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.296+0000 7f7fadda0640 1 -- 172.21.15.94:0/2934033334 >> 172.21.15.94:0/2934033334 conn(0x7f7fa80fb9d0 msgr2=0x7f7fa80fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.296+0000 7f7fadda0640 1 -- 172.21.15.94:0/2934033334 shutdown_connections 2024-02-17T12:08:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.296+0000 7f7fadda0640 1 -- 172.21.15.94:0/2934033334 wait complete. 2024-02-17T12:08:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.297+0000 7f7fadda0640 1 Processor -- start 2024-02-17T12:08:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.297+0000 7f7fadda0640 1 -- start start 2024-02-17T12:08:22.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.297+0000 7f7fadda0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fa8102250 0x7f7fa8109270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.297+0000 7f7fadda0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fa8102b70 0x7f7fa81097b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.297+0000 7f7fadda0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fa810b620 0x7f7fa8109cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.297+0000 7f7fadda0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa81a9d60 con 0x7f7fa810b620 2024-02-17T12:08:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.297+0000 7f7fadda0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa81a9ed0 con 0x7f7fa8102250 2024-02-17T12:08:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.297+0000 7f7fadda0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa81aa090 con 0x7f7fa8102b70 2024-02-17T12:08:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.297+0000 7f7fad59f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fa810b620 0x7f7fa8109cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.297+0000 7f7facd9e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fa8102250 0x7f7fa8109270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.297+0000 7f7fad59f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fa810b620 0x7f7fa8109cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47242/0 (socket says 172.21.15.94:47242) 2024-02-17T12:08:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7fad59f640 1 -- 172.21.15.94:0/1290151641 learned_addr learned my addr 172.21.15.94:0/1290151641 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:22.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.297+0000 7f7facd9e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fa8102250 0x7f7fa8109270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:35012/0 (socket says 172.21.15.94:35012) 2024-02-17T12:08:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7f97fff640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fa8102b70 0x7f7fa81097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7facd9e640 1 -- 172.21.15.94:0/1290151641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fa8102b70 msgr2=0x7f7fa81097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7facd9e640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fa8102b70 0x7f7fa81097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7facd9e640 1 -- 172.21.15.94:0/1290151641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fa810b620 msgr2=0x7f7fa8109cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7facd9e640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fa810b620 0x7f7fa8109cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7facd9e640 1 -- 172.21.15.94:0/1290151641 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c009490 con 0x7f7fa8102250 2024-02-17T12:08:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7fad59f640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fa810b620 0x7f7fa8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7facd9e640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fa8102250 0x7f7fa8109270 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800ef60 tx=0x7f7f9800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7fa67fc640 1 -- 172.21.15.94:0/1290151641 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800b570 con 0x7f7fa8102250 2024-02-17T12:08:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7fadda0640 1 -- 172.21.15.94:0/1290151641 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa0009490 con 0x7f7fa8102250 2024-02-17T12:08:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7fa67fc640 1 -- 172.21.15.94:0/1290151641 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f9800bbb0 con 0x7f7fa8102250 2024-02-17T12:08:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7fadda0640 1 -- 172.21.15.94:0/1290151641 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa81aa740 con 0x7f7fa8102250 2024-02-17T12:08:22.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.298+0000 7f7fa67fc640 1 -- 172.21.15.94:0/1290151641 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f98016cd0 con 0x7f7fa8102250 2024-02-17T12:08:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.302+0000 7f7fa67fc640 1 -- 172.21.15.94:0/1290151641 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7f9800a040 con 0x7f7fa8102250 2024-02-17T12:08:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fb1bce740 1 Processor -- start 2024-02-17T12:08:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fb1bce740 1 -- start start 2024-02-17T12:08:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fb1bce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 0x559551030930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fb1bce740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551030f00 0x5595510394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fb1bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551039a80 0x55955103bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fb1bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559550f9ef90 con 0x559551039a80 2024-02-17T12:08:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fb1bce740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559550f9c4c0 con 0x559551030f00 2024-02-17T12:08:22.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fb1bce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559550f97ca0 con 0x559551030550 2024-02-17T12:08:22.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fa67fc640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7f900a2230 0x7f7f900a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7facd9e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551030f00 0x5595510394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551039a80 0x55955103bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fad59f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 0x559551030930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fad59f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 0x559551030930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57868/0 (socket says 172.21.15.94:57868) 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551039a80 0x55955103bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47256/0 (socket says 172.21.15.94:47256) 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fad59f640 1 -- 172.21.15.94:0/3413092671 learned_addr learned my addr 172.21.15.94:0/3413092671 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fad59f640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7f900a2230 0x7f7f900a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.303+0000 7f7fa67fc640 1 -- 172.21.15.94:0/1290151641 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7f980ce000 con 0x7f7fa8102250 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.304+0000 7f7fad59f640 1 -- 172.21.15.94:0/3413092671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551030f00 msgr2=0x5595510394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.304+0000 7f7fad59f640 1 --2- 172.21.15.94:0/3413092671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551030f00 0x5595510394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.304+0000 7f7fad59f640 1 -- 172.21.15.94:0/3413092671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551039a80 msgr2=0x55955103bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.304+0000 7f7fad59f640 1 --2- 172.21.15.94:0/3413092671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551039a80 0x55955103bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.304+0000 7f7fad59f640 1 -- 172.21.15.94:0/3413092671 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559550f5a0b0 con 0x559551030550 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.304+0000 7f7fad59f640 1 --2- 172.21.15.94:0/3413092671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 0x559551030930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0005b60 tx=0x7f7fa00344c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a10816322fb4b2d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:22.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.305+0000 7f7fad59f640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7f900a2230 0x7f7f900a46f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7fa002e000 tx=0x7f7fa0046000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:22.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.305+0000 7f7f96ffd640 1 -- 172.21.15.94:0/3413092671 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0005410 con 0x559551030550 2024-02-17T12:08:22.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.305+0000 7f7f96ffd640 1 -- 172.21.15.94:0/3413092671 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fa0005570 con 0x559551030550 2024-02-17T12:08:22.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.305+0000 7f7f96ffd640 1 -- 172.21.15.94:0/3413092671 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0004da0 con 0x559551030550 2024-02-17T12:08:22.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.306+0000 7f7fb1bce740 1 -- 172.21.15.94:0/3413092671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 msgr2=0x559551030930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.306+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/3413092671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 0x559551030930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0005b60 tx=0x7f7fa00344c0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.310+0000 7f7fb1bce740 1 -- 172.21.15.94:0/3413092671 shutdown_connections 2024-02-17T12:08:22.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.310+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/3413092671 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551039a80 0x55955103bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.310+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/3413092671 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551030f00 0x5595510394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.310+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/3413092671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 0x559551030930 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.310+0000 7f7fb1bce740 1 -- 172.21.15.94:0/3413092671 >> 172.21.15.94:0/3413092671 conn(0x55955102aa10 msgr2=0x55955102adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:22.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.314+0000 7f7fb1bce740 1 -- 172.21.15.94:0/3413092671 shutdown_connections 2024-02-17T12:08:22.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.317+0000 7f7fb1bce740 1 -- 172.21.15.94:0/3413092671 wait complete. 2024-02-17T12:08:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.317+0000 7f7fb1bce740 1 -- start start 2024-02-17T12:08:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.317+0000 7f7fb1bce740 1 Processor -- start 2024-02-17T12:08:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.318+0000 7f7fb1bce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 0x559551036a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.318+0000 7f7fb1bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551030f00 0x559551036f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.318+0000 7f7fb1bce740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551039a80 0x5595510374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.318+0000 7f7fb1bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559550f3cf40 con 0x559551030f00 2024-02-17T12:08:22.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.318+0000 7f7fb1bce740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559550c6b4b0 con 0x559551039a80 2024-02-17T12:08:22.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.318+0000 7f7fb1bce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559550f1d670 con 0x559551030550 2024-02-17T12:08:22.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.318+0000 7f7fb1bce740 10 client.0 _ll_register_callbacks cb 0x559550f8af70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:22.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.319+0000 7f7facd9e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 0x559551036a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.319+0000 7f7fad59f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551039a80 0x5595510374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.319+0000 7f7facd9e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 0x559551036a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57878/0 (socket says 172.21.15.94:57878) 2024-02-17T12:08:22.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.319+0000 7f7facd9e640 1 -- 172.21.15.94:0/2331406131 learned_addr learned my addr 172.21.15.94:0/2331406131 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.322+0000 7f7f97fff640 1 --2- 172.21.15.94:0/2331406131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551030f00 0x559551036f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.323+0000 7f7facd9e640 1 -- 172.21.15.94:0/2331406131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551039a80 msgr2=0x5595510374c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.323+0000 7f7facd9e640 1 --2- 172.21.15.94:0/2331406131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551039a80 0x5595510374c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.323+0000 7f7facd9e640 1 -- 172.21.15.94:0/2331406131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551030f00 msgr2=0x559551036f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.323+0000 7f7facd9e640 1 --2- 172.21.15.94:0/2331406131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551030f00 0x559551036f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.323+0000 7f7facd9e640 1 -- 172.21.15.94:0/2331406131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa8100150 con 0x559551030550 2024-02-17T12:08:22.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.325+0000 7f7facd9e640 1 --2- 172.21.15.94:0/2331406131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 0x559551036a40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800cc90 tx=0x7f7f9806ca20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=769d437642fce5ab server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:22.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.330+0000 7f7f87fff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:08:22.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.330+0000 7f7f87fff640 1 -- 172.21.15.94:0/2331406131 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9809aa60 con 0x559551030550 2024-02-17T12:08:22.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.330+0000 7f7f87fff640 1 -- 172.21.15.94:0/2331406131 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7f9809eb00 con 0x559551030550 2024-02-17T12:08:22.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.331+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2331406131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559550f71f40 con 0x559551030550 2024-02-17T12:08:22.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.333+0000 7f7f87fff640 1 -- 172.21.15.94:0/2331406131 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f980a3150 con 0x559551030550 2024-02-17T12:08:22.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.334+0000 7f7fb1bce740 10 client.27679 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:22.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.334+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2331406131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559550f96320 con 0x559551030550 2024-02-17T12:08:22.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.335+0000 7f7f87fff640 1 -- 172.21.15.94:0/2331406131 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559550f71f40 con 0x559551030550 2024-02-17T12:08:22.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.335+0000 7f7f87fff640 1 -- 172.21.15.94:0/2331406131 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7f98099eb0 con 0x559551030550 2024-02-17T12:08:22.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.335+0000 7f7fb1bce740 10 client.27679 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:22.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.335+0000 7f7fb1bce740 10 client.27679 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:22.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.335+0000 7f7fb1bce740 20 client.27679 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:22.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.335+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/2331406131 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55955105b0d0 0x55955105d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.335+0000 7f7fb1bce740 4 client.27679 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:22.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.335+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2331406131 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559550f8e310 con 0x55955105b0d0 2024-02-17T12:08:22.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.336+0000 7f7f97fff640 1 --2- 172.21.15.94:0/2331406131 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55955105b0d0 0x55955105d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.336+0000 7f7f87fff640 10 client.27679 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:22.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.337+0000 7f7f97fff640 1 --2- 172.21.15.94:0/2331406131 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55955105b0d0 0x55955105d4f0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e7d493e0cb298e8 server_cookie=57243d40bbd5e85 in_seq=0 out_seq=0 2024-02-17T12:08:22.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.337+0000 7f7f87fff640 1 -- 172.21.15.94:0/2331406131 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559550f8e310 con 0x55955105b0d0 2024-02-17T12:08:22.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.337+0000 7f7f87fff640 10 client.27679 handle_command_reply: tid=0 2024-02-17T12:08:22.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.338+0000 7f7fb1bce740 1 client.27679 shutdown 2024-02-17T12:08:22.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.344+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2331406131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 msgr2=0x559551036a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.344+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/2331406131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 0x559551036a40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800cc90 tx=0x7f7f9806ca20 comp rx=0 tx=0).stop 2024-02-17T12:08:22.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.346+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2331406131 shutdown_connections 2024-02-17T12:08:22.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.346+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/2331406131 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55955105b0d0 0x55955105d4f0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.346+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/2331406131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551039a80 0x5595510374c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.346+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/2331406131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551030f00 0x559551036f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.346+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/2331406131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030550 0x559551036a40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.346+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2331406131 >> 172.21.15.94:0/2331406131 conn(0x55955102aa10 msgr2=0x55955102c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:22.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.346+0000 7f7f87fff640 0 client.27679 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:22.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.348+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2331406131 shutdown_connections 2024-02-17T12:08:22.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.349+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2331406131 wait complete. 2024-02-17T12:08:22.350 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.349+0000 7f7fb1bce740 20 client.27679 trim_cache size 0 max 16384 2024-02-17T12:08:22.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.373+0000 7f7fb1bce740 1 Processor -- start 2024-02-17T12:08:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7fb1bce740 1 -- start start 2024-02-17T12:08:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7fb1bce740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551030550 0x559551035ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7fb1bce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030f00 0x5595510372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7fb1bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551039a80 0x559551029b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7fb1bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559550f71f40 con 0x559551039a80 2024-02-17T12:08:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7fb1bce740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559551057b30 con 0x559551030550 2024-02-17T12:08:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7fb1bce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595510360e0 con 0x559551030f00 2024-02-17T12:08:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7fad59f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551039a80 0x559551029b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7fad59f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551039a80 0x559551029b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47266/0 (socket says 172.21.15.94:47266) 2024-02-17T12:08:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7fad59f640 1 -- 172.21.15.94:0/961323231 learned_addr learned my addr 172.21.15.94:0/961323231 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7facd9e640 1 --2- 172.21.15.94:0/961323231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551030550 0x559551035ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7f97fff640 1 --2- 172.21.15.94:0/961323231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030f00 0x5595510372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7facd9e640 1 -- 172.21.15.94:0/961323231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030f00 msgr2=0x5595510372e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7facd9e640 1 --2- 172.21.15.94:0/961323231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030f00 0x5595510372e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7facd9e640 1 -- 172.21.15.94:0/961323231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551039a80 msgr2=0x559551029b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7facd9e640 1 --2- 172.21.15.94:0/961323231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551039a80 0x559551029b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.374+0000 7f7facd9e640 1 -- 172.21.15.94:0/961323231 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559550f32c00 con 0x559551030550 2024-02-17T12:08:22.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.375+0000 7f7facd9e640 1 --2- 172.21.15.94:0/961323231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551030550 0x559551035ba0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f98016880 tx=0x7f7f980a3080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9bd6194b9b091d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.375+0000 7f7f967fc640 1 -- 172.21.15.94:0/961323231 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9809a5a0 con 0x559551030550 2024-02-17T12:08:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.375+0000 7f7f967fc640 1 -- 172.21.15.94:0/961323231 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f98096d00 con 0x559551030550 2024-02-17T12:08:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.375+0000 7f7f967fc640 1 -- 172.21.15.94:0/961323231 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f980970d0 con 0x559551030550 2024-02-17T12:08:22.378 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.377+0000 7f7fb1bce740 1 -- 172.21.15.94:0/961323231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551030550 msgr2=0x559551035ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.377+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/961323231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551030550 0x559551035ba0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f98016880 tx=0x7f7f980a3080 comp rx=0 tx=0).stop 2024-02-17T12:08:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.377+0000 7f7fb1bce740 1 -- 172.21.15.94:0/961323231 shutdown_connections 2024-02-17T12:08:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.377+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/961323231 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551039a80 0x559551029b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.377+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/961323231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030f00 0x5595510372e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.377+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/961323231 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551030550 0x559551035ba0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.377+0000 7f7fb1bce740 1 -- 172.21.15.94:0/961323231 >> 172.21.15.94:0/961323231 conn(0x55955102aa10 msgr2=0x559551029400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.377+0000 7f7fb1bce740 1 -- 172.21.15.94:0/961323231 shutdown_connections 2024-02-17T12:08:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.377+0000 7f7fb1bce740 1 -- 172.21.15.94:0/961323231 wait complete. 2024-02-17T12:08:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.377+0000 7f7fb1bce740 1 -- start start 2024-02-17T12:08:22.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.377+0000 7f7fb1bce740 1 Processor -- start 2024-02-17T12:08:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.378+0000 7f7fb1bce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551030550 0x5595510aa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.378+0000 7f7fb1bce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030f00 0x5595510aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.378+0000 7f7fb1bce740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551039a80 0x55955103fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.378+0000 7f7fb1bce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559551040570 con 0x559551030550 2024-02-17T12:08:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.378+0000 7f7fb1bce740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595510406e0 con 0x559551039a80 2024-02-17T12:08:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.378+0000 7f7fb1bce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595510408a0 con 0x559551030f00 2024-02-17T12:08:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.378+0000 7f7fb1bce740 10 client.0 _ll_register_callbacks cb 0x7f7f98009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.378+0000 7f7fad59f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551039a80 0x55955103fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.378+0000 7f7facd9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551030550 0x5595510aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.378+0000 7f7fad59f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551039a80 0x55955103fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:35036/0 (socket says 172.21.15.94:35036) 2024-02-17T12:08:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.378+0000 7f7fad59f640 1 -- 172.21.15.94:0/2490422663 learned_addr learned my addr 172.21.15.94:0/2490422663 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:22.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.378+0000 7f7f97fff640 1 --2- 172.21.15.94:0/2490422663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030f00 0x5595510aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.379+0000 7f7fad59f640 1 -- 172.21.15.94:0/2490422663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030f00 msgr2=0x5595510aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.379+0000 7f7fad59f640 1 --2- 172.21.15.94:0/2490422663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030f00 0x5595510aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.379+0000 7f7fad59f640 1 -- 172.21.15.94:0/2490422663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551030550 msgr2=0x5595510aa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.379+0000 7f7fad59f640 1 --2- 172.21.15.94:0/2490422663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551030550 0x5595510aa990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.379+0000 7f7fad59f640 1 -- 172.21.15.94:0/2490422663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559550f96320 con 0x559551039a80 2024-02-17T12:08:22.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.380+0000 7f7f87fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:08:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.380+0000 7f7fad59f640 1 --2- 172.21.15.94:0/2490422663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551039a80 0x55955103fe70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0004650 tx=0x7f7fa0004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6cc1f7e4b22a55b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.380+0000 7f7f87fff640 1 -- 172.21.15.94:0/2490422663 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0004370 con 0x559551039a80 2024-02-17T12:08:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.380+0000 7f7f87fff640 1 -- 172.21.15.94:0/2490422663 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7fa0059070 con 0x559551039a80 2024-02-17T12:08:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.380+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2490422663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559550fbd900 con 0x559551039a80 2024-02-17T12:08:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.380+0000 7f7f87fff640 1 -- 172.21.15.94:0/2490422663 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fa0004370 con 0x559551039a80 2024-02-17T12:08:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.380+0000 7f7fb1bce740 10 client.27707 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.380+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2490422663 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559550f58910 con 0x559551039a80 2024-02-17T12:08:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.381+0000 7f7f87fff640 1 -- 172.21.15.94:0/2490422663 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559550fbd900 con 0x559551039a80 2024-02-17T12:08:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.381+0000 7f7f87fff640 1 -- 172.21.15.94:0/2490422663 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7fa0067070 con 0x559551039a80 2024-02-17T12:08:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.381+0000 7f7fb1bce740 10 client.27707 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.381+0000 7f7fb1bce740 10 client.27707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.381+0000 7f7fb1bce740 20 client.27707 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:22.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.381+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/2490422663 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5595510b3640 0x5595510b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.381+0000 7f7fb1bce740 4 client.27707 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:08:22.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.381+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2490422663 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559550c97860 con 0x5595510b3640 2024-02-17T12:08:22.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.381+0000 7f7facd9e640 1 --2- 172.21.15.94:0/2490422663 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5595510b3640 0x5595510b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.382+0000 7f7f87fff640 10 client.27707 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:22.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.382+0000 7f7facd9e640 1 --2- 172.21.15.94:0/2490422663 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5595510b3640 0x5595510b5a60 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d631f1197c7b6d server_cookie=147348b6e7890087 in_seq=0 out_seq=0 2024-02-17T12:08:22.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.383+0000 7f7f87fff640 1 -- 172.21.15.94:0/2490422663 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559550c97860 con 0x5595510b3640 2024-02-17T12:08:22.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.383+0000 7f7f87fff640 10 client.27707 handle_command_reply: tid=0 2024-02-17T12:08:22.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.383+0000 7f7fb1bce740 1 client.27707 shutdown 2024-02-17T12:08:22.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.383+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2490422663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551039a80 msgr2=0x55955103fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.383+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/2490422663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551039a80 0x55955103fe70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0004650 tx=0x7f7fa0004860 comp rx=0 tx=0).stop 2024-02-17T12:08:22.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.384+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2490422663 shutdown_connections 2024-02-17T12:08:22.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.384+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/2490422663 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5595510b3640 0x5595510b5a60 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.384+0000 7f7f87fff640 0 client.27707 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:22.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.384+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/2490422663 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559551039a80 0x55955103fe70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.384+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/2490422663 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559551030f00 0x5595510aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.384+0000 7f7fb1bce740 1 --2- 172.21.15.94:0/2490422663 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559551030550 0x5595510aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.384+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2490422663 >> 172.21.15.94:0/2490422663 conn(0x55955102aa10 msgr2=0x55955102c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:22.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.385+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2490422663 shutdown_connections 2024-02-17T12:08:22.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.385+0000 7f7fb1bce740 1 -- 172.21.15.94:0/2490422663 wait complete. 2024-02-17T12:08:22.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.385+0000 7f7fb1bce740 20 client.27707 trim_cache size 0 max 16384 2024-02-17T12:08:22.387 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:08:22.387 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:08:22.388 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "96b5faab-f3f8-4564-8a42-d2127e680721", 2024-02-17T12:08:22.388 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:08:22.388 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:08:22.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.385+0000 7f7fadda0640 1 -- 172.21.15.94:0/1290151641 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7f900a2230 msgr2=0x7f7f900a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.385+0000 7f7fadda0640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7f900a2230 0x7f7f900a46f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7fa002e000 tx=0x7f7fa0046000 comp rx=0 tx=0).stop 2024-02-17T12:08:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.385+0000 7f7fadda0640 1 -- 172.21.15.94:0/1290151641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fa8102250 msgr2=0x7f7fa8109270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.385+0000 7f7fadda0640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fa8102250 0x7f7fa8109270 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f7f9800ef60 tx=0x7f7f9800ef90 comp rx=0 tx=0).stop 2024-02-17T12:08:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.385+0000 7f7fadda0640 1 -- 172.21.15.94:0/1290151641 shutdown_connections 2024-02-17T12:08:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.385+0000 7f7fadda0640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7f900a2230 0x7f7f900a46f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.385+0000 7f7fadda0640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7fa810b620 0x7f7fa8109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.385+0000 7f7fadda0640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7fa8102b70 0x7f7fa81097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.385+0000 7f7fadda0640 1 --2- 172.21.15.94:0/1290151641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7fa8102250 0x7f7fa8109270 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.385+0000 7f7fadda0640 1 -- 172.21.15.94:0/1290151641 >> 172.21.15.94:0/1290151641 conn(0x7f7fa80fb9d0 msgr2=0x7f7fa8111840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.386+0000 7f7fadda0640 1 -- 172.21.15.94:0/1290151641 shutdown_connections 2024-02-17T12:08:22.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.386+0000 7f7fadda0640 1 -- 172.21.15.94:0/1290151641 wait complete. 2024-02-17T12:08:22.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:08:22.403 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:08:22.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.571+0000 7f3d3c076640 1 -- 172.21.15.94:0/2838870928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d3410f8a0 msgr2=0x7f3d3410d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.571+0000 7f3d3c076640 1 --2- 172.21.15.94:0/2838870928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d3410f8a0 0x7f3d3410d4c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009860 tx=0x7f3d3002bbb0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.572+0000 7f3d39611640 1 --2- 172.21.15.94:0/2838870928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3410fd10 0x7f3d3410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:22.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.573+0000 7f3d3c076640 1 -- 172.21.15.94:0/2838870928 shutdown_connections 2024-02-17T12:08:22.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.573+0000 7f3d3c076640 1 --2- 172.21.15.94:0/2838870928 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d3411a960 0x7f3d3411cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.573+0000 7f3d3c076640 1 --2- 172.21.15.94:0/2838870928 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3410fd10 0x7f3d3410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.573+0000 7f3d3c076640 1 --2- 172.21.15.94:0/2838870928 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d3410f8a0 0x7f3d3410d4c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.577 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.573+0000 7f3d3c076640 1 -- 172.21.15.94:0/2838870928 >> 172.21.15.94:0/2838870928 conn(0x7f3d3406c270 msgr2=0x7f3d3406c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:22.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.577+0000 7f3d3c076640 1 -- 172.21.15.94:0/2838870928 shutdown_connections 2024-02-17T12:08:22.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.579+0000 7f3d3c076640 1 -- 172.21.15.94:0/2838870928 wait complete. 2024-02-17T12:08:22.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.580+0000 7f3d3c076640 1 Processor -- start 2024-02-17T12:08:22.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.581+0000 7f3d3c076640 1 -- start start 2024-02-17T12:08:22.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.581+0000 7f3d3c076640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3410f8a0 0x7f3d34117d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.581+0000 7f3d3c076640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d3410fd10 0x7f3d341182d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.581+0000 7f3d3c076640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d3411a960 0x7f3d34110eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.581+0000 7f3d3c076640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d34111640 con 0x7f3d3410f8a0 2024-02-17T12:08:22.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.581+0000 7f3d3c076640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d341117b0 con 0x7f3d3410fd10 2024-02-17T12:08:22.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.581+0000 7f3d3c076640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d34111970 con 0x7f3d3411a960 2024-02-17T12:08:22.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.583+0000 7f3d39611640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d3410fd10 0x7f3d341182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.583+0000 7f3d39611640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d3410fd10 0x7f3d341182d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:35046/0 (socket says 172.21.15.94:35046) 2024-02-17T12:08:22.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.583+0000 7f3d39611640 1 -- 172.21.15.94:0/1381000499 learned_addr learned my addr 172.21.15.94:0/1381000499 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:22.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.584+0000 7f3d39e12640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3410f8a0 0x7f3d34117d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.584+0000 7f3d3a613640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d3411a960 0x7f3d34110eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.585+0000 7f3d39611640 1 -- 172.21.15.94:0/1381000499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d3411a960 msgr2=0x7f3d34110eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.585+0000 7f3d39611640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d3411a960 0x7f3d34110eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.585+0000 7f3d39611640 1 -- 172.21.15.94:0/1381000499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3410f8a0 msgr2=0x7f3d34117d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.585+0000 7f3d39611640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3410f8a0 0x7f3d34117d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.585+0000 7f3d39611640 1 -- 172.21.15.94:0/1381000499 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c009900 con 0x7f3d3410fd10 2024-02-17T12:08:22.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.588+0000 7f3d39611640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d3410fd10 0x7f3d341182d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00c3f0 tx=0x7f3d2c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:22.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.588+0000 7f3d2affd640 1 -- 172.21.15.94:0/1381000499 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c003970 con 0x7f3d3410fd10 2024-02-17T12:08:22.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.588+0000 7f3d3c076640 1 -- 172.21.15.94:0/1381000499 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d30009490 con 0x7f3d3410fd10 2024-02-17T12:08:22.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.588+0000 7f3d3c076640 1 -- 172.21.15.94:0/1381000499 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d34115a90 con 0x7f3d3410fd10 2024-02-17T12:08:22.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.588+0000 7f3d2affd640 1 -- 172.21.15.94:0/1381000499 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d2c004020 con 0x7f3d3410fd10 2024-02-17T12:08:22.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.589+0000 7f3d2affd640 1 -- 172.21.15.94:0/1381000499 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c00bad0 con 0x7f3d3410fd10 2024-02-17T12:08:22.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.593+0000 7f3d2affd640 1 -- 172.21.15.94:0/1381000499 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3d2c00a040 con 0x7f3d3410fd10 2024-02-17T12:08:22.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.593+0000 7f3d3ec48740 1 Processor -- start 2024-02-17T12:08:22.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.594+0000 7f3d3ec48740 1 -- start start 2024-02-17T12:08:22.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.594+0000 7f3d3ec48740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981af2b0 0x5613981af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.594+0000 7f3d3ec48740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981b8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.594+0000 7f3d3ec48740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981b87b0 0x5613981baba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.594+0000 7f3d3ec48740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139811e280 con 0x5613981b87b0 2024-02-17T12:08:22.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.594+0000 7f3d3ec48740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139811b7b0 con 0x5613981afbd0 2024-02-17T12:08:22.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.594+0000 7f3d3ec48740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561398116f90 con 0x5613981af2b0 2024-02-17T12:08:22.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.594+0000 7f3d2affd640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3d200a2440 0x7f3d200a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.595+0000 7f3d2affd640 1 -- 172.21.15.94:0/1381000499 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d2c0ce650 con 0x7f3d3410fd10 2024-02-17T12:08:22.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.595+0000 7f3d3a613640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981af2b0 0x5613981af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.595+0000 7f3d3a613640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981af2b0 0x5613981af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57946/0 (socket says 172.21.15.94:57946) 2024-02-17T12:08:22.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.595+0000 7f3d3a613640 1 -- 172.21.15.94:0/1892708731 learned_addr learned my addr 172.21.15.94:0/1892708731 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:22.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.595+0000 7f3d3a613640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3d200a2440 0x7f3d200a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.596+0000 7f3d3a613640 1 -- 172.21.15.94:0/1892708731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 msgr2=0x5613981b8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.596+0000 7f3d3a613640 1 --2- 172.21.15.94:0/1892708731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.596+0000 7f3d3a613640 1 -- 172.21.15.94:0/1892708731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981b87b0 msgr2=0x5613981baba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:08:22.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.596+0000 7f3d3a613640 1 --2- 172.21.15.94:0/1892708731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981b87b0 0x5613981baba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.596+0000 7f3d3a613640 1 -- 172.21.15.94:0/1892708731 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613980d5570 con 0x5613981af2b0 2024-02-17T12:08:22.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.599+0000 7f3d3a613640 1 --2- 172.21.15.94:0/1892708731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981af2b0 0x5613981af690 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3d24005d80 tx=0x7f3d24015670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae4030203731092d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:22.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.600+0000 7f3d3a613640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3d200a2440 0x7f3d200a4900 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400e000 tx=0x7f3d2401c000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:22.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.602+0000 7f3d1ffff640 1 -- 172.21.15.94:0/1892708731 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24013d80 con 0x5613981af2b0 2024-02-17T12:08:22.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.602+0000 7f3d1ffff640 1 -- 172.21.15.94:0/1892708731 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d24005510 con 0x5613981af2b0 2024-02-17T12:08:22.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.603+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1892708731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981af2b0 msgr2=0x5613981af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.603+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/1892708731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981af2b0 0x5613981af690 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3d24005d80 tx=0x7f3d24015670 comp rx=0 tx=0).stop 2024-02-17T12:08:22.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.603+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1892708731 shutdown_connections 2024-02-17T12:08:22.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.603+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/1892708731 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981b87b0 0x5613981baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.603+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/1892708731 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981b8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.603+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/1892708731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981af2b0 0x5613981af690 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.604+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1892708731 >> 172.21.15.94:0/1892708731 conn(0x5613981a9990 msgr2=0x5613981a9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:22.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.604+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1892708731 shutdown_connections 2024-02-17T12:08:22.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.604+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1892708731 wait complete. 2024-02-17T12:08:22.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.605+0000 7f3d3ec48740 1 -- start start 2024-02-17T12:08:22.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.605+0000 7f3d3ec48740 1 Processor -- start 2024-02-17T12:08:22.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.606+0000 7f3d3ec48740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.606+0000 7f3d3ec48740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981b5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.606+0000 7f3d3ec48740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 0x5613981b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.606+0000 7f3d3ec48740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561397dea430 con 0x5613981af2b0 2024-02-17T12:08:22.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.606+0000 7f3d3ec48740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561397da6e00 con 0x5613981afbd0 2024-02-17T12:08:22.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.606+0000 7f3d3ec48740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139809bfd0 con 0x5613981b87b0 2024-02-17T12:08:22.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.606+0000 7f3d3ec48740 10 client.0 _ll_register_callbacks cb 0x5613980f1fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:22.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.606+0000 7f3d39e12640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.607+0000 7f3d39e12640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47308/0 (socket says 172.21.15.94:47308) 2024-02-17T12:08:22.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.607+0000 7f3d39e12640 1 -- 172.21.15.94:0/2522278914 learned_addr learned my addr 172.21.15.94:0/2522278914 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:22.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.607+0000 7f3d3a613640 1 --2- 172.21.15.94:0/2522278914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 0x5613981b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.607+0000 7f3d39e12640 1 -- 172.21.15.94:0/2522278914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 msgr2=0x5613981b5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.607+0000 7f3d39e12640 1 --2- 172.21.15.94:0/2522278914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 0x5613981b5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.607+0000 7f3d39e12640 1 -- 172.21.15.94:0/2522278914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 msgr2=0x5613981b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.607+0000 7f3d39611640 1 --2- 172.21.15.94:0/2522278914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.607+0000 7f3d39e12640 1 --2- 172.21.15.94:0/2522278914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.607+0000 7f3d39e12640 1 -- 172.21.15.94:0/2522278914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d24009df0 con 0x5613981af2b0 2024-02-17T12:08:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.607+0000 7f3d39e12640 1 --2- 172.21.15.94:0/2522278914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b8180 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009860 tx=0x7f3d30004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7045251455f7f0d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.607+0000 7f3d1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:08:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d1d7fa640 1 -- 172.21.15.94:0/2522278914 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d30034640 con 0x5613981af2b0 2024-02-17T12:08:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d1d7fa640 1 -- 172.21.15.94:0/2522278914 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d300347a0 con 0x5613981af2b0 2024-02-17T12:08:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d3ec48740 1 -- 172.21.15.94:0/2522278914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561397e167e0 con 0x5613981af2b0 2024-02-17T12:08:22.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d1d7fa640 1 -- 172.21.15.94:0/2522278914 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d30034960 con 0x5613981af2b0 2024-02-17T12:08:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d3ec48740 10 client.18234 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d3ec48740 1 -- 172.21.15.94:0/2522278914 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561398115610 con 0x5613981af2b0 2024-02-17T12:08:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d1d7fa640 1 -- 172.21.15.94:0/2522278914 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561397e167e0 con 0x5613981af2b0 2024-02-17T12:08:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d1d7fa640 1 -- 172.21.15.94:0/2522278914 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3d3002ca10 con 0x5613981af2b0 2024-02-17T12:08:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d3ec48740 10 client.18234 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d3ec48740 10 client.18234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d3ec48740 20 client.18234 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/2522278914 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5613981c89a0 0x5613981cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d3ec48740 4 client.18234 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.608+0000 7f3d3ec48740 1 -- 172.21.15.94:0/2522278914 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561397e53720 con 0x5613981c89a0 2024-02-17T12:08:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.609+0000 7f3d39611640 1 --2- 172.21.15.94:0/2522278914 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5613981c89a0 0x5613981cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.609+0000 7f3d1d7fa640 10 client.18234 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:22.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.609+0000 7f3d39611640 1 --2- 172.21.15.94:0/2522278914 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5613981c89a0 0x5613981cadc0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=435e665ec7ad3e84 server_cookie=a5343c7f7c117201 in_seq=0 out_seq=0 2024-02-17T12:08:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.610+0000 7f3d1d7fa640 1 -- 172.21.15.94:0/2522278914 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561397e53720 con 0x5613981c89a0 2024-02-17T12:08:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.610+0000 7f3d1d7fa640 10 client.18234 handle_command_reply: tid=0 2024-02-17T12:08:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.610+0000 7f3d3ec48740 1 client.18234 shutdown 2024-02-17T12:08:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.610+0000 7f3d3ec48740 1 -- 172.21.15.94:0/2522278914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 msgr2=0x5613981b8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.610+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/2522278914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b8180 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009860 tx=0x7f3d30004270 comp rx=0 tx=0).stop 2024-02-17T12:08:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.610+0000 7f3d3ec48740 1 -- 172.21.15.94:0/2522278914 shutdown_connections 2024-02-17T12:08:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.610+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/2522278914 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5613981c89a0 0x5613981cadc0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.610+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/2522278914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 0x5613981b5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.610+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/2522278914 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.610+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/2522278914 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b8180 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.610+0000 7f3d3ec48740 1 -- 172.21.15.94:0/2522278914 >> 172.21.15.94:0/2522278914 conn(0x5613981a9990 msgr2=0x5613981ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.611+0000 7f3d1d7fa640 0 client.18234 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.611+0000 7f3d3ec48740 1 -- 172.21.15.94:0/2522278914 shutdown_connections 2024-02-17T12:08:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.611+0000 7f3d3ec48740 1 -- 172.21.15.94:0/2522278914 wait complete. 2024-02-17T12:08:22.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.611+0000 7f3d3ec48740 20 client.18234 trim_cache size 0 max 16384 2024-02-17T12:08:22.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.630+0000 7f3d3ec48740 1 Processor -- start 2024-02-17T12:08:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.630+0000 7f3d3ec48740 1 -- start start 2024-02-17T12:08:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d3ec48740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d3ec48740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981b8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d3ec48740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 0x5613981b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d3ec48740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613981a50d0 con 0x5613981af2b0 2024-02-17T12:08:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d3ec48740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613981c45b0 con 0x5613981afbd0 2024-02-17T12:08:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d3ec48740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613981aef10 con 0x5613981b87b0 2024-02-17T12:08:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d39e12640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d39e12640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47316/0 (socket says 172.21.15.94:47316) 2024-02-17T12:08:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d39e12640 1 -- 172.21.15.94:0/3031054291 learned_addr learned my addr 172.21.15.94:0/3031054291 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:22.635 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d39611640 1 --2- 172.21.15.94:0/3031054291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981b8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d3a613640 1 --2- 172.21.15.94:0/3031054291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 0x5613981b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d39e12640 1 -- 172.21.15.94:0/3031054291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 msgr2=0x5613981b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d39e12640 1 --2- 172.21.15.94:0/3031054291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 0x5613981b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d39e12640 1 -- 172.21.15.94:0/3031054291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 msgr2=0x5613981b8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d39e12640 1 --2- 172.21.15.94:0/3031054291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981b8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d39e12640 1 -- 172.21.15.94:0/3031054291 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613980b1ac0 con 0x5613981af2b0 2024-02-17T12:08:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d39611640 1 --2- 172.21.15.94:0/3031054291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981b8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:08:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d39e12640 1 --2- 172.21.15.94:0/3031054291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b7c40 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009830 tx=0x7f3d30005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b9b3337063896f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d1f7fe640 1 -- 172.21.15.94:0/3031054291 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d30034440 con 0x5613981af2b0 2024-02-17T12:08:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d1f7fe640 1 -- 172.21.15.94:0/3031054291 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d3002c550 con 0x5613981af2b0 2024-02-17T12:08:22.636 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.631+0000 7f3d1f7fe640 1 -- 172.21.15.94:0/3031054291 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d30032840 con 0x5613981af2b0 2024-02-17T12:08:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.632+0000 7f3d3ec48740 1 -- 172.21.15.94:0/3031054291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 msgr2=0x5613981b7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.632+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/3031054291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b7c40 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f3d30009830 tx=0x7f3d30005b10 comp rx=0 tx=0).stop 2024-02-17T12:08:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.632+0000 7f3d3ec48740 1 -- 172.21.15.94:0/3031054291 shutdown_connections 2024-02-17T12:08:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.632+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/3031054291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 0x5613981b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.632+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/3031054291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981b8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.632+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/3031054291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b7c40 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.632+0000 7f3d3ec48740 1 -- 172.21.15.94:0/3031054291 >> 172.21.15.94:0/3031054291 conn(0x5613981a9990 msgr2=0x5613981cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.632+0000 7f3d3ec48740 1 -- 172.21.15.94:0/3031054291 shutdown_connections 2024-02-17T12:08:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.632+0000 7f3d3ec48740 1 -- 172.21.15.94:0/3031054291 wait complete. 2024-02-17T12:08:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.632+0000 7f3d3ec48740 1 -- start start 2024-02-17T12:08:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.632+0000 7f3d3ec48740 1 Processor -- start 2024-02-17T12:08:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.633+0000 7f3d3ec48740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.633+0000 7f3d3ec48740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981bf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.634+0000 7f3d3ec48740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 0x5613981b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.634+0000 7f3d39611640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981bf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.634+0000 7f3d39611640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981bf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:35092/0 (socket says 172.21.15.94:35092) 2024-02-17T12:08:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.634+0000 7f3d39611640 1 -- 172.21.15.94:0/1437019623 learned_addr learned my addr 172.21.15.94:0/1437019623 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.634+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1437019623 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613981bff70 con 0x5613981af2b0 2024-02-17T12:08:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.634+0000 7f3d3a613640 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 0x5613981b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.635+0000 7f3d39e12640 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.635+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1437019623 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613981b1cc0 con 0x5613981afbd0 2024-02-17T12:08:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.635+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1437019623 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613981b1e30 con 0x5613981b87b0 2024-02-17T12:08:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.635+0000 7f3d3ec48740 10 client.0 _ll_register_callbacks cb 0x7f3d24009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:22.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.635+0000 7f3d3a613640 1 -- 172.21.15.94:0/1437019623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 msgr2=0x5613981bf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.635+0000 7f3d3a613640 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981bf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.635+0000 7f3d3a613640 1 -- 172.21.15.94:0/1437019623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 msgr2=0x5613981b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.635+0000 7f3d3a613640 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.635+0000 7f3d3a613640 1 -- 172.21.15.94:0/1437019623 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561398115610 con 0x5613981b87b0 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.635+0000 7f3d39e12640 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.635+0000 7f3d39611640 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981bf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.635+0000 7f3d3a613640 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 0x5613981b1780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3d240093a0 tx=0x7f3d24004f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=223a37fdaf7c533 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.635+0000 7f3d1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.636+0000 7f3d1d7fa640 1 -- 172.21.15.94:0/1437019623 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24005030 con 0x5613981b87b0 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.636+0000 7f3d1d7fa640 1 -- 172.21.15.94:0/1437019623 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3d24005190 con 0x5613981b87b0 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.636+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1437019623 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d24005510 con 0x5613981b87b0 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.636+0000 7f3d1d7fa640 1 -- 172.21.15.94:0/1437019623 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d24005370 con 0x5613981b87b0 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.636+0000 7f3d3ec48740 10 client.27700 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:22.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.636+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1437019623 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5613980d76a0 con 0x5613981b87b0 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.636+0000 7f3d1d7fa640 1 -- 172.21.15.94:0/1437019623 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3d2403b080 con 0x5613981b87b0 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.637+0000 7f3d1d7fa640 1 -- 172.21.15.94:0/1437019623 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3d2400c070 con 0x5613981b87b0 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.637+0000 7f3d3ec48740 10 client.27700 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.637+0000 7f3d3ec48740 10 client.27700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.637+0000 7f3d3ec48740 20 client.27700 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.637+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56139821fce0 0x561398222100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.637+0000 7f3d3ec48740 4 client.27700 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.637+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1437019623 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561397ea2890 con 0x56139821fce0 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.637+0000 7f3d39e12640 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56139821fce0 0x561398222100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.637+0000 7f3d1d7fa640 10 client.27700 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.637+0000 7f3d39e12640 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56139821fce0 0x561398222100 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=895b0fa64d56d06b server_cookie=d6f8263e9f50768f in_seq=0 out_seq=0 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d1d7fa640 1 -- 172.21.15.94:0/1437019623 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561397ea2890 con 0x56139821fce0 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d1d7fa640 10 client.27700 handle_command_reply: tid=0 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3ec48740 1 client.27700 shutdown 2024-02-17T12:08:22.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1437019623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 msgr2=0x5613981b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 0x5613981b1780 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3d240093a0 tx=0x7f3d24004f70 comp rx=0 tx=0).stop 2024-02-17T12:08:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1437019623 shutdown_connections 2024-02-17T12:08:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56139821fce0 0x561398222100 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5613981b87b0 0x5613981b1780 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d1d7fa640 0 client.27700 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5613981afbd0 0x5613981bf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3ec48740 1 --2- 172.21.15.94:0/1437019623 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5613981af2b0 0x5613981b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1437019623 >> 172.21.15.94:0/1437019623 conn(0x5613981a9990 msgr2=0x5613981b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1437019623 shutdown_connections 2024-02-17T12:08:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3ec48740 1 -- 172.21.15.94:0/1437019623 wait complete. 2024-02-17T12:08:22.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3ec48740 20 client.27700 trim_cache size 0 max 16384 2024-02-17T12:08:22.646 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:08:22.646 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:08:22.646 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:08:22.646 INFO:teuthology.orchestra.run.smithi094.stdout: "96b5faab-f3f8-4564-8a42-d2127e680721": { 2024-02-17T12:08:22.646 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:08:22.646 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "96b5faab-f3f8-4564-8a42-d2127e680721", 2024-02-17T12:08:22.646 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:08:22.646 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:08:22.646 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:08:22.646 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:08:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3c076640 1 -- 172.21.15.94:0/1381000499 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3d200a2440 msgr2=0x7f3d200a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3c076640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3d200a2440 0x7f3d200a4900 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400e000 tx=0x7f3d2401c000 comp rx=0 tx=0).stop 2024-02-17T12:08:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3c076640 1 -- 172.21.15.94:0/1381000499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d3410fd10 msgr2=0x7f3d341182d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.638+0000 7f3d3c076640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d3410fd10 0x7f3d341182d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00c3f0 tx=0x7f3d2c009f90 comp rx=0 tx=0).stop 2024-02-17T12:08:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.639+0000 7f3d3c076640 1 -- 172.21.15.94:0/1381000499 shutdown_connections 2024-02-17T12:08:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.639+0000 7f3d3c076640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f3d200a2440 0x7f3d200a4900 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.639+0000 7f3d3c076640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d3411a960 0x7f3d34110eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.639+0000 7f3d3c076640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3d3410fd10 0x7f3d341182d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.639+0000 7f3d3c076640 1 --2- 172.21.15.94:0/1381000499 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3d3410f8a0 0x7f3d34117d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.639+0000 7f3d3c076640 1 -- 172.21.15.94:0/1381000499 >> 172.21.15.94:0/1381000499 conn(0x7f3d3406c270 msgr2=0x7f3d341193c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.639+0000 7f3d3c076640 1 -- 172.21.15.94:0/1381000499 shutdown_connections 2024-02-17T12:08:22.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:22.639+0000 7f3d3c076640 1 -- 172.21.15.94:0/1381000499 wait complete. 2024-02-17T12:08:22.651 INFO:tasks.cephfs.filesystem:scrub status for tag:96b5faab-f3f8-4564-8a42-d2127e680721 - {'path': '/', 'tag': '96b5faab-f3f8-4564-8a42-d2127e680721', 'options': 'recursive,force'} 2024-02-17T12:08:22.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:22 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:08:22.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:22 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:08:22.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:22 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:08:22.803 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:22 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:08:22.804 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:22 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:08:22.804 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:22 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:08:22.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:22 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:08:22.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:22 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:08:22.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:22 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:08:23.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:23 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:08:23.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:23 smithi189 ceph-mon[36588]: pgmap v319: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 742 op/s 2024-02-17T12:08:23.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:23 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:08:23.936 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:23 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:08:23.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:23 smithi125 ceph-mon[36819]: pgmap v319: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 742 op/s 2024-02-17T12:08:23.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:23 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:08:23.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:23 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:08:23.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:23 smithi094 ceph-mon[32499]: pgmap v319: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 18 MiB/s wr, 742 op/s 2024-02-17T12:08:23.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:23 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:08:25.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 605.0 s, 1822.4 tps, lat 55.459 ms stddev 63.133 2024-02-17T12:08:25.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:25 smithi189 ceph-mon[36588]: pgmap v320: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 17 MiB/s wr, 770 op/s 2024-02-17T12:08:25.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:25 smithi125 ceph-mon[36819]: pgmap v320: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 17 MiB/s wr, 770 op/s 2024-02-17T12:08:25.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:25 smithi094 ceph-mon[32499]: pgmap v320: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 17 MiB/s wr, 770 op/s 2024-02-17T12:08:26.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:26 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:26.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:26 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:26.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:26 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:26.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:26 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:26.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:26 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:26.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:26 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:26.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:26 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:26.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:26 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:26.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:26 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi189 to 174.1M 2024-02-17T12:08:26.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:26 smithi094 ceph-mon[32499]: Unable to set osd_memory_target on smithi189 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-02-17T12:08:26.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:26 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:26.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:26 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:26.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:26 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:26.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:26 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:26.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:26 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:26.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:26 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:26.656 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:26 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:26.656 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:26 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:26.656 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:26 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi189 to 174.1M 2024-02-17T12:08:26.656 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:26 smithi189 ceph-mon[36588]: Unable to set osd_memory_target on smithi189 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-02-17T12:08:26.686 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:26 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:26.686 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:26 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:26.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:26 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:26.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:26 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:26.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:26 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:26.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:26 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:26.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:26 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:26.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:26 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:26.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:26 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi189 to 174.1M 2024-02-17T12:08:26.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:26 smithi125 ceph-mon[36819]: Unable to set osd_memory_target on smithi189 to 182636339: error parsing value: Value '182636339' is below minimum 939524096 2024-02-17T12:08:27.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:27.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:27.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-02-17T12:08:27.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: Unable to set osd_memory_target on smithi094 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-02-17T12:08:27.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: pgmap v321: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 15 MiB/s wr, 661 op/s 2024-02-17T12:08:27.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:08:27.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:08:27.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:27 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:27.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:27.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:27.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-02-17T12:08:27.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: Unable to set osd_memory_target on smithi094 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-02-17T12:08:27.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: pgmap v321: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 15 MiB/s wr, 661 op/s 2024-02-17T12:08:27.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:08:27.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:08:27.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:27 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:27.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:27.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:27.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi094 to 176.2M 2024-02-17T12:08:27.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: Unable to set osd_memory_target on smithi094 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-02-17T12:08:27.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: pgmap v321: 97 pgs: 97 active+clean; 5.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 15 MiB/s wr, 661 op/s 2024-02-17T12:08:27.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-17T12:08:27.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:08:27.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:08:27.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:27 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:08:28.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:28 smithi189 ceph-mon[36588]: Adjusting osd_memory_target on smithi125 to 174.1M 2024-02-17T12:08:28.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:28 smithi189 ceph-mon[36588]: Unable to set osd_memory_target on smithi125 to 182611968: error parsing value: Value '182611968' is below minimum 939524096 2024-02-17T12:08:28.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:28 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:08:28.686 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:28 smithi125 ceph-mon[36819]: Adjusting osd_memory_target on smithi125 to 174.1M 2024-02-17T12:08:28.686 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:28 smithi125 ceph-mon[36819]: Unable to set osd_memory_target on smithi125 to 182611968: error parsing value: Value '182611968' is below minimum 939524096 2024-02-17T12:08:28.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:28 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:08:28.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:28 smithi094 ceph-mon[32499]: Adjusting osd_memory_target on smithi125 to 174.1M 2024-02-17T12:08:28.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:28 smithi094 ceph-mon[32499]: Unable to set osd_memory_target on smithi125 to 182611968: error parsing value: Value '182611968' is below minimum 939524096 2024-02-17T12:08:28.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:28 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:08:29.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:29 smithi189 ceph-mon[36588]: pgmap v322: 97 pgs: 97 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 19 MiB/s wr, 859 op/s 2024-02-17T12:08:29.686 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:29 smithi125 ceph-mon[36819]: pgmap v322: 97 pgs: 97 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 19 MiB/s wr, 859 op/s 2024-02-17T12:08:29.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:29 smithi094 ceph-mon[32499]: pgmap v322: 97 pgs: 97 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 19 MiB/s wr, 859 op/s 2024-02-17T12:08:30.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 610.0 s, 1981.6 tps, lat 50.520 ms stddev 60.916 2024-02-17T12:08:31.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:31 smithi189 ceph-mon[36588]: pgmap v323: 97 pgs: 97 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 14 MiB/s wr, 686 op/s 2024-02-17T12:08:31.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:31 smithi125 ceph-mon[36819]: pgmap v323: 97 pgs: 97 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 14 MiB/s wr, 686 op/s 2024-02-17T12:08:31.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:31 smithi094 ceph-mon[32499]: pgmap v323: 97 pgs: 97 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 14 MiB/s wr, 686 op/s 2024-02-17T12:08:33.207 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:33 smithi094 ceph-mon[32499]: pgmap v324: 97 pgs: 97 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 12 MiB/s wr, 646 op/s 2024-02-17T12:08:33.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:33 smithi189 ceph-mon[36588]: pgmap v324: 97 pgs: 97 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 12 MiB/s wr, 646 op/s 2024-02-17T12:08:33.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:33 smithi125 ceph-mon[36819]: pgmap v324: 97 pgs: 97 active+clean; 5.3 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 12 MiB/s wr, 646 op/s 2024-02-17T12:08:34.937 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.936+0000 7f8e56857640 1 -- 172.21.15.94:0/2205317682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e5011a960 msgr2=0x7f8e5011cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:34.938 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.936+0000 7f8e56857640 1 --2- 172.21.15.94:0/2205317682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e5011a960 0x7f8e5011cd50 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009900 tx=0x7f8e4402bc80 comp rx=0 tx=0).stop 2024-02-17T12:08:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.939+0000 7f8e56857640 1 -- 172.21.15.94:0/2205317682 shutdown_connections 2024-02-17T12:08:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.939+0000 7f8e56857640 1 --2- 172.21.15.94:0/2205317682 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e5011a960 0x7f8e5011cd50 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.939+0000 7f8e56857640 1 --2- 172.21.15.94:0/2205317682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e5010fd30 0x7f8e5010da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.939+0000 7f8e56857640 1 --2- 172.21.15.94:0/2205317682 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e5010f8c0 0x7f8e5010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.939+0000 7f8e56857640 1 -- 172.21.15.94:0/2205317682 >> 172.21.15.94:0/2205317682 conn(0x7f8e5006c220 msgr2=0x7f8e5006c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.942+0000 7f8e56857640 1 -- 172.21.15.94:0/2205317682 shutdown_connections 2024-02-17T12:08:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.942+0000 7f8e56857640 1 -- 172.21.15.94:0/2205317682 wait complete. 2024-02-17T12:08:34.944 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.943+0000 7f8e56857640 1 Processor -- start 2024-02-17T12:08:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.945+0000 7f8e56857640 1 -- start start 2024-02-17T12:08:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.945+0000 7f8e56857640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e5010f8c0 0x7f8e50119e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.945+0000 7f8e56857640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e5010fd30 0x7f8e5011a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.945+0000 7f8e56857640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e5011a960 0x7f8e50112fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.945+0000 7f8e56857640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e50113760 con 0x7f8e5011a960 2024-02-17T12:08:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.945+0000 7f8e56857640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e501138d0 con 0x7f8e5010fd30 2024-02-17T12:08:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.945+0000 7f8e56857640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e50113a90 con 0x7f8e5010f8c0 2024-02-17T12:08:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.946+0000 7f8e4f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e5010fd30 0x7f8e5011a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.946+0000 7f8e4f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e5010fd30 0x7f8e5011a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46956/0 (socket says 172.21.15.94:46956) 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.946+0000 7f8e4f7fe640 1 -- 172.21.15.94:0/774761793 learned_addr learned my addr 172.21.15.94:0/774761793 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.946+0000 7f8e54df4640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e5011a960 0x7f8e50112fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.946+0000 7f8e4ffff640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e5010f8c0 0x7f8e50119e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.946+0000 7f8e4f7fe640 1 -- 172.21.15.94:0/774761793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e5010f8c0 msgr2=0x7f8e50119e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.946+0000 7f8e4f7fe640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e5010f8c0 0x7f8e50119e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.946+0000 7f8e4f7fe640 1 -- 172.21.15.94:0/774761793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e5011a960 msgr2=0x7f8e50112fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.946+0000 7f8e4f7fe640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e5011a960 0x7f8e50112fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.946+0000 7f8e4f7fe640 1 -- 172.21.15.94:0/774761793 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e44009560 con 0x7f8e5010fd30 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.946+0000 7f8e4ffff640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e5010f8c0 0x7f8e50119e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.947+0000 7f8e4f7fe640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e5010fd30 0x7f8e5011a340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000b9e0 tx=0x7f8e4000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.947+0000 7f8e4d7fa640 1 -- 172.21.15.94:0/774761793 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000cba0 con 0x7f8e5010fd30 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.947+0000 7f8e4d7fa640 1 -- 172.21.15.94:0/774761793 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e4000cd00 con 0x7f8e5010fd30 2024-02-17T12:08:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.947+0000 7f8e4d7fa640 1 -- 172.21.15.94:0/774761793 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40012770 con 0x7f8e5010fd30 2024-02-17T12:08:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.947+0000 7f8e56857640 1 -- 172.21.15.94:0/774761793 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e50117920 con 0x7f8e5010fd30 2024-02-17T12:08:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.947+0000 7f8e56857640 1 -- 172.21.15.94:0/774761793 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e50117de0 con 0x7f8e5010fd30 2024-02-17T12:08:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e4d7fa640 1 -- 172.21.15.94:0/774761793 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8e40012950 con 0x7f8e5010fd30 2024-02-17T12:08:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e5941f740 1 Processor -- start 2024-02-17T12:08:34.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e5941f740 1 -- start start 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e5941f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9bad0 0x563c5ff9a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e5941f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9ad60 0x563c5ff9b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e5941f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e5941f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5ff099f0 con 0x563c5ff9ad60 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e5941f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5ff06f20 con 0x563c5ffa42e0 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e5941f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5ff02700 con 0x563c5ff9bad0 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e54df4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9bad0 0x563c5ff9a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e54df4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9bad0 0x563c5ff9a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:49912/0 (socket says 172.21.15.94:49912) 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e54df4640 1 -- 172.21.15.94:0/1336588008 learned_addr learned my addr 172.21.15.94:0/1336588008 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e4ffff640 1 --2- 172.21.15.94:0/1336588008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9ad60 0x563c5ff9b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.950+0000 7f8e4f7fe640 1 --2- 172.21.15.94:0/1336588008 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.951+0000 7f8e54df4640 1 -- 172.21.15.94:0/1336588008 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 msgr2=0x563c5ffa66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.951+0000 7f8e54df4640 1 --2- 172.21.15.94:0/1336588008 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.951+0000 7f8e54df4640 1 -- 172.21.15.94:0/1336588008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9ad60 msgr2=0x563c5ff9b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:34.953 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.951+0000 7f8e54df4640 1 --2- 172.21.15.94:0/1336588008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9ad60 0x563c5ff9b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.951+0000 7f8e54df4640 1 -- 172.21.15.94:0/1336588008 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c5fe987b0 con 0x563c5ff9bad0 2024-02-17T12:08:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.951+0000 7f8e4ffff640 1 --2- 172.21.15.94:0/1336588008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9ad60 0x563c5ff9b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.951+0000 7f8e4d7fa640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8e380a2320 0x7f8e380a47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.951+0000 7f8e54df4640 1 --2- 172.21.15.94:0/1336588008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9bad0 0x563c5ff9a820 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8e44005ab0 tx=0x7f8e44004800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d9f4ea5d54c707e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.951+0000 7f8e4ffff640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8e380a2320 0x7f8e380a47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.951+0000 7f8e367fc640 1 -- 172.21.15.94:0/1336588008 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44004a30 con 0x563c5ff9bad0 2024-02-17T12:08:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.952+0000 7f8e367fc640 1 -- 172.21.15.94:0/1336588008 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e44004040 con 0x563c5ff9bad0 2024-02-17T12:08:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.952+0000 7f8e367fc640 1 -- 172.21.15.94:0/1336588008 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44005210 con 0x563c5ff9bad0 2024-02-17T12:08:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.952+0000 7f8e4d7fa640 1 -- 172.21.15.94:0/774761793 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8e400cd960 con 0x7f8e5010fd30 2024-02-17T12:08:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.952+0000 7f8e5941f740 1 -- 172.21.15.94:0/1336588008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9bad0 msgr2=0x563c5ff9a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.952+0000 7f8e5941f740 1 --2- 172.21.15.94:0/1336588008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9bad0 0x563c5ff9a820 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8e44005ab0 tx=0x7f8e44004800 comp rx=0 tx=0).stop 2024-02-17T12:08:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.952+0000 7f8e4ffff640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8e380a2320 0x7f8e380a47e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x563c5ff9c230 tx=0x7f8e48008040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.952+0000 7f8e5941f740 1 -- 172.21.15.94:0/1336588008 shutdown_connections 2024-02-17T12:08:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.952+0000 7f8e5941f740 1 --2- 172.21.15.94:0/1336588008 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.958 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.952+0000 7f8e5941f740 1 --2- 172.21.15.94:0/1336588008 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9ad60 0x563c5ff9b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.952+0000 7f8e5941f740 1 --2- 172.21.15.94:0/1336588008 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9bad0 0x563c5ff9a820 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.952+0000 7f8e5941f740 1 -- 172.21.15.94:0/1336588008 >> 172.21.15.94:0/1336588008 conn(0x563c5ff62550 msgr2=0x563c5ff95a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.953+0000 7f8e5941f740 1 -- 172.21.15.94:0/1336588008 shutdown_connections 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.953+0000 7f8e5941f740 1 -- 172.21.15.94:0/1336588008 wait complete. 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.953+0000 7f8e5941f740 1 -- start start 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.953+0000 7f8e5941f740 1 Processor -- start 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e5941f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9ad60 0x563c5ffa1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e5941f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9bad0 0x563c5ffa18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e5941f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e5941f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5fea7bf0 con 0x563c5ff9ad60 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e5941f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5fbd6250 con 0x563c5ffa42e0 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e5941f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5fe88200 con 0x563c5ff9bad0 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e5941f740 10 client.0 _ll_register_callbacks cb 0x563c5fef6d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:34.959 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e54df4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e54df4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46972/0 (socket says 172.21.15.94:46972) 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e54df4640 1 -- 172.21.15.94:0/295537546 learned_addr learned my addr 172.21.15.94:0/295537546 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e4ffff640 1 --2- 172.21.15.94:0/295537546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9ad60 0x563c5ffa1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e4f7fe640 1 --2- 172.21.15.94:0/295537546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9bad0 0x563c5ffa18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e54df4640 1 -- 172.21.15.94:0/295537546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9bad0 msgr2=0x563c5ffa18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e54df4640 1 --2- 172.21.15.94:0/295537546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9bad0 0x563c5ffa18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e54df4640 1 -- 172.21.15.94:0/295537546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9ad60 msgr2=0x563c5ffa1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e54df4640 1 --2- 172.21.15.94:0/295537546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9ad60 0x563c5ffa1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.954+0000 7f8e54df4640 1 -- 172.21.15.94:0/295537546 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5011d640 con 0x563c5ffa42e0 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.955+0000 7f8e54df4640 1 --2- 172.21.15.94:0/295537546 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa1df0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8e4402c290 tx=0x7f8e44005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d0c3559a4d5e00f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.955+0000 7f8e2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.955+0000 7f8e2f7fe640 1 -- 172.21.15.94:0/295537546 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44005190 con 0x563c5ffa42e0 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.955+0000 7f8e2f7fe640 1 -- 172.21.15.94:0/295537546 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8e44004a30 con 0x563c5ffa42e0 2024-02-17T12:08:34.960 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.955+0000 7f8e5941f740 1 -- 172.21.15.94:0/295537546 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563c5fc02600 con 0x563c5ffa42e0 2024-02-17T12:08:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.955+0000 7f8e2f7fe640 1 -- 172.21.15.94:0/295537546 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e4402cc10 con 0x563c5ffa42e0 2024-02-17T12:08:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.955+0000 7f8e5941f740 10 client.27728 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.955+0000 7f8e5941f740 1 -- 172.21.15.94:0/295537546 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563c5ff00d80 con 0x563c5ffa42e0 2024-02-17T12:08:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.955+0000 7f8e2f7fe640 1 -- 172.21.15.94:0/295537546 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e4405f080 con 0x563c5ffa42e0 2024-02-17T12:08:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.956+0000 7f8e2f7fe640 1 -- 172.21.15.94:0/295537546 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8e4403c050 con 0x563c5ffa42e0 2024-02-17T12:08:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.956+0000 7f8e5941f740 10 client.27728 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.956+0000 7f8e5941f740 10 client.27728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.956+0000 7f8e5941f740 20 client.27728 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.956+0000 7f8e5941f740 1 --2- 172.21.15.94:0/295537546 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563c5ffb4370 0x563c5ffb6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.956+0000 7f8e5941f740 4 client.27728 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.956+0000 7f8e5941f740 1 -- 172.21.15.94:0/295537546 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563c5fc531b0 con 0x563c5ffb4370 2024-02-17T12:08:34.961 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.956+0000 7f8e4ffff640 1 --2- 172.21.15.94:0/295537546 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563c5ffb4370 0x563c5ffb6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.957+0000 7f8e2f7fe640 10 client.27728 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.957+0000 7f8e4ffff640 1 --2- 172.21.15.94:0/295537546 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563c5ffb4370 0x563c5ffb6760 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2911b4e0eeae1977 server_cookie=febcdef448b3679c in_seq=0 out_seq=0 2024-02-17T12:08:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.959+0000 7f8e2f7fe640 1 -- 172.21.15.94:0/295537546 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563c5fc531b0 con 0x563c5ffb4370 2024-02-17T12:08:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.959+0000 7f8e2f7fe640 10 client.27728 handle_command_reply: tid=0 2024-02-17T12:08:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.959+0000 7f8e5941f740 1 client.27728 shutdown 2024-02-17T12:08:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.959+0000 7f8e5941f740 1 -- 172.21.15.94:0/295537546 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 msgr2=0x563c5ffa1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.959+0000 7f8e5941f740 1 --2- 172.21.15.94:0/295537546 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa1df0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8e4402c290 tx=0x7f8e44005fd0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.959+0000 7f8e5941f740 1 -- 172.21.15.94:0/295537546 shutdown_connections 2024-02-17T12:08:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.959+0000 7f8e5941f740 1 --2- 172.21.15.94:0/295537546 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563c5ffb4370 0x563c5ffb6760 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.959+0000 7f8e5941f740 1 --2- 172.21.15.94:0/295537546 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa1df0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.959+0000 7f8e5941f740 1 --2- 172.21.15.94:0/295537546 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9bad0 0x563c5ffa18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.959+0000 7f8e5941f740 1 --2- 172.21.15.94:0/295537546 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9ad60 0x563c5ffa1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.960+0000 7f8e5941f740 1 -- 172.21.15.94:0/295537546 >> 172.21.15.94:0/295537546 conn(0x563c5ff62550 msgr2=0x563c5ff977a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.960+0000 7f8e2f7fe640 0 client.27728 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.960+0000 7f8e5941f740 1 -- 172.21.15.94:0/295537546 shutdown_connections 2024-02-17T12:08:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.960+0000 7f8e5941f740 1 -- 172.21.15.94:0/295537546 wait complete. 2024-02-17T12:08:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.960+0000 7f8e5941f740 20 client.27728 trim_cache size 0 max 16384 2024-02-17T12:08:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.978+0000 7f8e5941f740 1 Processor -- start 2024-02-17T12:08:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.978+0000 7f8e5941f740 1 -- start start 2024-02-17T12:08:34.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e5941f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9ad60 0x563c5ffa3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e5941f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9bad0 0x563c5ffa18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e5941f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e4f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9bad0 0x563c5ffa18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e4f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9bad0 0x563c5ffa18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52818/0 (socket says 172.21.15.94:52818) 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e4f7fe640 1 -- 172.21.15.94:0/332535286 learned_addr learned my addr 172.21.15.94:0/332535286 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e5941f740 1 -- 172.21.15.94:0/332535286 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5ffb4170 con 0x563c5ff9bad0 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e4ffff640 1 --2- 172.21.15.94:0/332535286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9ad60 0x563c5ffa3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e5941f740 1 -- 172.21.15.94:0/332535286 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5ff96c80 con 0x563c5ffa42e0 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e5941f740 1 -- 172.21.15.94:0/332535286 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5ff9a030 con 0x563c5ff9ad60 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e54df4640 1 --2- 172.21.15.94:0/332535286 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e4f7fe640 1 -- 172.21.15.94:0/332535286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9ad60 msgr2=0x563c5ffa3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e4f7fe640 1 --2- 172.21.15.94:0/332535286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9ad60 0x563c5ffa3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e4f7fe640 1 -- 172.21.15.94:0/332535286 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 msgr2=0x563c5ffa1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e4f7fe640 1 --2- 172.21.15.94:0/332535286 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e4f7fe640 1 -- 172.21.15.94:0/332535286 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c5fe9d7b0 con 0x563c5ff9bad0 2024-02-17T12:08:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e54df4640 1 --2- 172.21.15.94:0/332535286 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.979+0000 7f8e4f7fe640 1 --2- 172.21.15.94:0/332535286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9bad0 0x563c5ffa18c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8e400cb1e0 tx=0x7f8e40014db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4094bcb6c91adc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.980+0000 7f8e35ffb640 1 -- 172.21.15.94:0/332535286 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40099620 con 0x563c5ff9bad0 2024-02-17T12:08:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.980+0000 7f8e35ffb640 1 -- 172.21.15.94:0/332535286 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e400cb2f0 con 0x563c5ff9bad0 2024-02-17T12:08:34.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.980+0000 7f8e35ffb640 1 -- 172.21.15.94:0/332535286 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e400b8b90 con 0x563c5ff9bad0 2024-02-17T12:08:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.980+0000 7f8e5941f740 1 -- 172.21.15.94:0/332535286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9bad0 msgr2=0x563c5ffa18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.980+0000 7f8e5941f740 1 --2- 172.21.15.94:0/332535286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9bad0 0x563c5ffa18c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f8e400cb1e0 tx=0x7f8e40014db0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.980+0000 7f8e5941f740 1 -- 172.21.15.94:0/332535286 shutdown_connections 2024-02-17T12:08:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.980+0000 7f8e5941f740 1 --2- 172.21.15.94:0/332535286 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ffa42e0 0x563c5ffa1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.980+0000 7f8e5941f740 1 --2- 172.21.15.94:0/332535286 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ff9bad0 0x563c5ffa18c0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.980+0000 7f8e5941f740 1 --2- 172.21.15.94:0/332535286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9ad60 0x563c5ffa3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.981+0000 7f8e5941f740 1 -- 172.21.15.94:0/332535286 >> 172.21.15.94:0/332535286 conn(0x563c5ff62550 msgr2=0x563c5ff9c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.981+0000 7f8e5941f740 1 -- 172.21.15.94:0/332535286 shutdown_connections 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.981+0000 7f8e5941f740 1 -- 172.21.15.94:0/332535286 wait complete. 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.981+0000 7f8e5941f740 1 -- start start 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.981+0000 7f8e5941f740 1 Processor -- start 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.982+0000 7f8e5941f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9ad60 0x563c5ffab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.982+0000 7f8e5941f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ff9bad0 0x563c5ffa9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.982+0000 7f8e5941f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ffa42e0 0x563c5ffa99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.982+0000 7f8e5941f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5ffab9b0 con 0x563c5ffa42e0 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e5941f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5ffa9f00 con 0x563c5ff9bad0 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e4ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9ad60 0x563c5ffab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e5941f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c5ffaa0c0 con 0x563c5ff9ad60 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e5941f740 10 client.0 _ll_register_callbacks cb 0x7f8e44005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e4f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ff9bad0 0x563c5ffa9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e4f7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ff9bad0 0x563c5ffa9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46990/0 (socket says 172.21.15.94:46990) 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e4f7fe640 1 -- 172.21.15.94:0/2498902609 learned_addr learned my addr 172.21.15.94:0/2498902609 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e4ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9ad60 0x563c5ffab0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:49948/0 (socket says 172.21.15.94:49948) 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e4f7fe640 1 -- 172.21.15.94:0/2498902609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9ad60 msgr2=0x563c5ffab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e4f7fe640 1 --2- 172.21.15.94:0/2498902609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9ad60 0x563c5ffab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e4f7fe640 1 -- 172.21.15.94:0/2498902609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ffa42e0 msgr2=0x563c5ffa99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e4f7fe640 1 --2- 172.21.15.94:0/2498902609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ffa42e0 0x563c5ffa99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e4f7fe640 1 -- 172.21.15.94:0/2498902609 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c5ff00d80 con 0x563c5ff9bad0 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e4ffff640 1 --2- 172.21.15.94:0/2498902609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9ad60 0x563c5ffab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e4f7fe640 1 --2- 172.21.15.94:0/2498902609 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ff9bad0 0x563c5ffa9480 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8e40097920 tx=0x7f8e400d30f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5cbec7086f34566 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.983+0000 7f8e2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e2f7fe640 1 -- 172.21.15.94:0/2498902609 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40099620 con 0x563c5ff9bad0 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e2f7fe640 1 -- 172.21.15.94:0/2498902609 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8e400cb460 con 0x563c5ff9bad0 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e5941f740 1 -- 172.21.15.94:0/2498902609 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e44004040 con 0x563c5ff9bad0 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e2f7fe640 1 -- 172.21.15.94:0/2498902609 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e4009e120 con 0x563c5ff9bad0 2024-02-17T12:08:34.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e5941f740 10 client.27734 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e5941f740 1 -- 172.21.15.94:0/2498902609 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f8e50117de0 con 0x563c5ff9bad0 2024-02-17T12:08:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e2f7fe640 1 -- 172.21.15.94:0/2498902609 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e4009e280 con 0x563c5ff9bad0 2024-02-17T12:08:34.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e2f7fe640 1 -- 172.21.15.94:0/2498902609 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8e4009f5b0 con 0x563c5ff9bad0 2024-02-17T12:08:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e5941f740 10 client.27734 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e5941f740 10 client.27734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e5941f740 20 client.27734 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e5941f740 1 --2- 172.21.15.94:0/2498902609 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563c6000b8f0 0x563c6000dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e5941f740 4 client.27734 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:08:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e5941f740 1 -- 172.21.15.94:0/2498902609 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f8e44004040 con 0x563c6000b8f0 2024-02-17T12:08:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.984+0000 7f8e4ffff640 1 --2- 172.21.15.94:0/2498902609 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563c6000b8f0 0x563c6000dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.985+0000 7f8e2f7fe640 10 client.27734 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:34.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:34.985+0000 7f8e4ffff640 1 --2- 172.21.15.94:0/2498902609 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563c6000b8f0 0x563c6000dd10 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2827d726fe9dd2c3 server_cookie=391a2b29e5a7b603 in_seq=0 out_seq=0 2024-02-17T12:08:35.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.004+0000 7f8e2f7fe640 1 -- 172.21.15.94:0/2498902609 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f8e44004040 con 0x563c6000b8f0 2024-02-17T12:08:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.004+0000 7f8e2f7fe640 10 client.27734 handle_command_reply: tid=0 2024-02-17T12:08:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.004+0000 7f8e5941f740 1 client.27734 shutdown 2024-02-17T12:08:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.005+0000 7f8e5941f740 1 -- 172.21.15.94:0/2498902609 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ff9bad0 msgr2=0x563c5ffa9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.005+0000 7f8e5941f740 1 --2- 172.21.15.94:0/2498902609 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ff9bad0 0x563c5ffa9480 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8e40097920 tx=0x7f8e400d30f0 comp rx=0 tx=0).stop 2024-02-17T12:08:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.005+0000 7f8e5941f740 1 -- 172.21.15.94:0/2498902609 shutdown_connections 2024-02-17T12:08:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.005+0000 7f8e5941f740 1 --2- 172.21.15.94:0/2498902609 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563c6000b8f0 0x563c6000dd10 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.005+0000 7f8e5941f740 1 --2- 172.21.15.94:0/2498902609 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563c5ffa42e0 0x563c5ffa99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.005+0000 7f8e5941f740 1 --2- 172.21.15.94:0/2498902609 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563c5ff9bad0 0x563c5ffa9480 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.005+0000 7f8e5941f740 1 --2- 172.21.15.94:0/2498902609 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563c5ff9ad60 0x563c5ffab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.005+0000 7f8e2f7fe640 0 client.27734 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.005+0000 7f8e5941f740 1 -- 172.21.15.94:0/2498902609 >> 172.21.15.94:0/2498902609 conn(0x563c5ff62550 msgr2=0x563c5ff9f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.005+0000 7f8e5941f740 1 -- 172.21.15.94:0/2498902609 shutdown_connections 2024-02-17T12:08:35.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.005+0000 7f8e5941f740 1 -- 172.21.15.94:0/2498902609 wait complete. 2024-02-17T12:08:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.005+0000 7f8e5941f740 20 client.27734 trim_cache size 0 max 16384 2024-02-17T12:08:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:08:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:08:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:08:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:08:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.006+0000 7f8e56857640 1 -- 172.21.15.94:0/774761793 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8e380a2320 msgr2=0x7f8e380a47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.006+0000 7f8e56857640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8e380a2320 0x7f8e380a47e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x563c5ff9c230 tx=0x7f8e48008040 comp rx=0 tx=0).stop 2024-02-17T12:08:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.006+0000 7f8e56857640 1 -- 172.21.15.94:0/774761793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e5010fd30 msgr2=0x7f8e5011a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.006+0000 7f8e56857640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e5010fd30 0x7f8e5011a340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000b9e0 tx=0x7f8e4000bea0 comp rx=0 tx=0).stop 2024-02-17T12:08:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.006+0000 7f8e56857640 1 -- 172.21.15.94:0/774761793 shutdown_connections 2024-02-17T12:08:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.006+0000 7f8e56857640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8e380a2320 0x7f8e380a47e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.006+0000 7f8e56857640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8e5011a960 0x7f8e50112fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.006+0000 7f8e56857640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8e5010fd30 0x7f8e5011a340 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.006+0000 7f8e56857640 1 --2- 172.21.15.94:0/774761793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e5010f8c0 0x7f8e50119e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.006+0000 7f8e56857640 1 -- 172.21.15.94:0/774761793 >> 172.21.15.94:0/774761793 conn(0x7f8e5006c220 msgr2=0x7f8e5011b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:35.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.006+0000 7f8e56857640 1 -- 172.21.15.94:0/774761793 shutdown_connections 2024-02-17T12:08:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:35.006+0000 7f8e56857640 1 -- 172.21.15.94:0/774761793 wait complete. 2024-02-17T12:08:35.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 615.0 s, 1885.4 tps, lat 52.799 ms stddev 61.325 2024-02-17T12:08:35.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:35 smithi189 ceph-mon[36588]: pgmap v325: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 18 MiB/s wr, 938 op/s 2024-02-17T12:08:35.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:35 smithi125 ceph-mon[36819]: pgmap v325: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 18 MiB/s wr, 938 op/s 2024-02-17T12:08:35.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:35 smithi094 ceph-mon[32499]: pgmap v325: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 18 MiB/s wr, 938 op/s 2024-02-17T12:08:37.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:37 smithi189 ceph-mon[36588]: pgmap v326: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 580 op/s 2024-02-17T12:08:37.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:37 smithi125 ceph-mon[36819]: pgmap v326: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 580 op/s 2024-02-17T12:08:37.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:37 smithi094 ceph-mon[32499]: pgmap v326: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 12 MiB/s wr, 580 op/s 2024-02-17T12:08:39.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:39 smithi189 ceph-mon[36588]: pgmap v327: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 898 op/s 2024-02-17T12:08:39.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:39 smithi125 ceph-mon[36819]: pgmap v327: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 898 op/s 2024-02-17T12:08:39.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:39 smithi094 ceph-mon[32499]: pgmap v327: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 17 MiB/s wr, 898 op/s 2024-02-17T12:08:40.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 620.0 s, 1918.8 tps, lat 51.731 ms stddev 62.451 2024-02-17T12:08:41.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:41 smithi189 ceph-mon[36588]: pgmap v328: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 723 op/s 2024-02-17T12:08:41.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:41 smithi125 ceph-mon[36819]: pgmap v328: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 723 op/s 2024-02-17T12:08:41.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:41 smithi094 ceph-mon[32499]: pgmap v328: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 13 MiB/s wr, 723 op/s 2024-02-17T12:08:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:43 smithi189 ceph-mon[36588]: pgmap v329: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 12 MiB/s wr, 652 op/s 2024-02-17T12:08:43.432 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:43 smithi094 ceph-mon[32499]: pgmap v329: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 12 MiB/s wr, 652 op/s 2024-02-17T12:08:43.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:43 smithi125 ceph-mon[36819]: pgmap v329: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 12 MiB/s wr, 652 op/s 2024-02-17T12:08:45.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 625.0 s, 1640.6 tps, lat 61.580 ms stddev 138.780 2024-02-17T12:08:45.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:45 smithi189 ceph-mon[36588]: pgmap v330: 97 pgs: 97 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 33 MiB/s wr, 1.99k op/s 2024-02-17T12:08:45.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:45 smithi125 ceph-mon[36819]: pgmap v330: 97 pgs: 97 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 33 MiB/s wr, 1.99k op/s 2024-02-17T12:08:45.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:45 smithi094 ceph-mon[32499]: pgmap v330: 97 pgs: 97 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 33 MiB/s wr, 1.99k op/s 2024-02-17T12:08:47.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:47 smithi189 ceph-mon[36588]: pgmap v331: 97 pgs: 97 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 MiB/s wr, 1.70k op/s 2024-02-17T12:08:47.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:47 smithi125 ceph-mon[36819]: pgmap v331: 97 pgs: 97 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 MiB/s wr, 1.70k op/s 2024-02-17T12:08:47.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:47 smithi094 ceph-mon[32499]: pgmap v331: 97 pgs: 97 active+clean; 5.6 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 28 MiB/s wr, 1.70k op/s 2024-02-17T12:08:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.149+0000 7f1adb59e640 1 -- 172.21.15.94:0/259462133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1adc10d260 msgr2=0x7f1adc10d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.149+0000 7f1adb59e640 1 --2- 172.21.15.94:0/259462133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1adc10d260 0x7f1adc10d6e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009860 tx=0x7f1acc02bbb0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.151+0000 7f1adb59e640 1 -- 172.21.15.94:0/259462133 shutdown_connections 2024-02-17T12:08:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.151+0000 7f1adb59e640 1 --2- 172.21.15.94:0/259462133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1adc10dd80 0x7f1adc11c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.151+0000 7f1adb59e640 1 --2- 172.21.15.94:0/259462133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1adc10d260 0x7f1adc10d6e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.151+0000 7f1adb59e640 1 --2- 172.21.15.94:0/259462133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1adc10f700 0x7f1adc10fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.151+0000 7f1adb59e640 1 -- 172.21.15.94:0/259462133 >> 172.21.15.94:0/259462133 conn(0x7f1adc06c090 msgr2=0x7f1adc06c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.155+0000 7f1adb59e640 1 -- 172.21.15.94:0/259462133 shutdown_connections 2024-02-17T12:08:48.158 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.156+0000 7f1adb59e640 1 -- 172.21.15.94:0/259462133 wait complete. 2024-02-17T12:08:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.158+0000 7f1adb59e640 1 Processor -- start 2024-02-17T12:08:48.160 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.158+0000 7f1adb59e640 1 -- start start 2024-02-17T12:08:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.159+0000 7f1adb59e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1adc10d260 0x7f1adc11c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.159+0000 7f1adb59e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1adc10dd80 0x7f1adc116600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.161 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.159+0000 7f1adb59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1adc10f700 0x7f1adc116b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.159+0000 7f1ad9d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1adc10dd80 0x7f1adc116600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.159+0000 7f1ad9d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1adc10dd80 0x7f1adc116600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:51668/0 (socket says 172.21.15.94:51668) 2024-02-17T12:08:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.159+0000 7f1ad9d9b640 1 -- 172.21.15.94:0/196848917 learned_addr learned my addr 172.21.15.94:0/196848917 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.159+0000 7f1adad9d640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1adc10f700 0x7f1adc116b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.159+0000 7f1adb59e640 1 -- 172.21.15.94:0/196848917 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc117140 con 0x7f1adc10f700 2024-02-17T12:08:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.160+0000 7f1adb59e640 1 -- 172.21.15.94:0/196848917 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc1172b0 con 0x7f1adc10d260 2024-02-17T12:08:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.160+0000 7f1adb59e640 1 -- 172.21.15.94:0/196848917 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc117470 con 0x7f1adc10dd80 2024-02-17T12:08:48.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.160+0000 7f1ada59c640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1adc10d260 0x7f1adc11c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.163 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.161+0000 7f1ada59c640 1 -- 172.21.15.94:0/196848917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1adc10dd80 msgr2=0x7f1adc116600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.161+0000 7f1ada59c640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1adc10dd80 0x7f1adc116600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.161+0000 7f1ada59c640 1 -- 172.21.15.94:0/196848917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1adc10f700 msgr2=0x7f1adc116b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.161+0000 7f1ada59c640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1adc10f700 0x7f1adc116b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.161+0000 7f1ada59c640 1 -- 172.21.15.94:0/196848917 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc009490 con 0x7f1adc10d260 2024-02-17T12:08:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.161+0000 7f1ad9d9b640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1adc10dd80 0x7f1adc116600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.161+0000 7f1adad9d640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1adc10f700 0x7f1adc116b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.161+0000 7f1ada59c640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1adc10d260 0x7f1adc11c590 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400ef60 tx=0x7f1ad400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.162+0000 7f1ad37fe640 1 -- 172.21.15.94:0/196848917 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400b570 con 0x7f1adc10d260 2024-02-17T12:08:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.162+0000 7f1adb59e640 1 -- 172.21.15.94:0/196848917 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1adc1bb520 con 0x7f1adc10d260 2024-02-17T12:08:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.162+0000 7f1adb59e640 1 -- 172.21.15.94:0/196848917 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1adc1bba50 con 0x7f1adc10d260 2024-02-17T12:08:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.162+0000 7f1ad37fe640 1 -- 172.21.15.94:0/196848917 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ad400bbb0 con 0x7f1adc10d260 2024-02-17T12:08:48.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.162+0000 7f1ad37fe640 1 -- 172.21.15.94:0/196848917 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad4016c70 con 0x7f1adc10d260 2024-02-17T12:08:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.164+0000 7f1ae37d3740 1 Processor -- start 2024-02-17T12:08:48.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.165+0000 7f1ae37d3740 1 -- start start 2024-02-17T12:08:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.165+0000 7f1ae37d3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667fbc60 0x5564667fa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.165+0000 7f1ae37d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667faef0 0x5564667fb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.165+0000 7f1ae37d3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556466804470 0x556466806860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.165+0000 7f1ae37d3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564667699f0 con 0x556466804470 2024-02-17T12:08:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.165+0000 7f1ae37d3740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556466766f20 con 0x5564667fbc60 2024-02-17T12:08:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.165+0000 7f1ae37d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556466762700 con 0x5564667faef0 2024-02-17T12:08:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.165+0000 7f1adad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667fbc60 0x5564667fa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.165+0000 7f1ad9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556466804470 0x556466806860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.165+0000 7f1adad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667fbc60 0x5564667fa9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:49998/0 (socket says 172.21.15.94:49998) 2024-02-17T12:08:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.165+0000 7f1adad9d640 1 -- 172.21.15.94:0/1117992505 learned_addr learned my addr 172.21.15.94:0/1117992505 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.166+0000 7f1adad9d640 1 -- 172.21.15.94:0/1117992505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667faef0 msgr2=0x5564667fb370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:08:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.166+0000 7f1adad9d640 1 --2- 172.21.15.94:0/1117992505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667faef0 0x5564667fb370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.166+0000 7f1adad9d640 1 -- 172.21.15.94:0/1117992505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556466804470 msgr2=0x556466806860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.166+0000 7f1adad9d640 1 --2- 172.21.15.94:0/1117992505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556466804470 0x556466806860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.166+0000 7f1adad9d640 1 -- 172.21.15.94:0/1117992505 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564666f87b0 con 0x5564667fbc60 2024-02-17T12:08:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.166+0000 7f1ad9d9b640 1 --2- 172.21.15.94:0/1117992505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556466804470 0x556466806860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.166+0000 7f1ad37fe640 1 -- 172.21.15.94:0/196848917 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1ad400a040 con 0x7f1adc10d260 2024-02-17T12:08:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.166+0000 7f1adad9d640 1 --2- 172.21.15.94:0/1117992505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667fbc60 0x5564667fa9b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009d70 tx=0x7f1ac400e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b97b4f69b7618f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:48.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.166+0000 7f1ad0ff9640 1 -- 172.21.15.94:0/1117992505 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4009870 con 0x5564667fbc60 2024-02-17T12:08:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.166+0000 7f1ad0ff9640 1 -- 172.21.15.94:0/1117992505 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ac400eec0 con 0x5564667fbc60 2024-02-17T12:08:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.166+0000 7f1ad37fe640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1ac80a2270 0x7f1ac80a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.166+0000 7f1ada59c640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1ac80a2270 0x7f1ac80a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ad37fe640 1 -- 172.21.15.94:0/196848917 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1ad40ce000 con 0x7f1adc10d260 2024-02-17T12:08:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ad0ff9640 1 -- 172.21.15.94:0/1117992505 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40129c0 con 0x5564667fbc60 2024-02-17T12:08:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ae37d3740 1 -- 172.21.15.94:0/1117992505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667fbc60 msgr2=0x5564667fa9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/1117992505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667fbc60 0x5564667fa9b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009d70 tx=0x7f1ac400e5a0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ae37d3740 1 -- 172.21.15.94:0/1117992505 shutdown_connections 2024-02-17T12:08:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/1117992505 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556466804470 0x556466806860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/1117992505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667faef0 0x5564667fb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/1117992505 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667fbc60 0x5564667fa9b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ae37d3740 1 -- 172.21.15.94:0/1117992505 >> 172.21.15.94:0/1117992505 conn(0x5564667c2550 msgr2=0x5564667f5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ae37d3740 1 -- 172.21.15.94:0/1117992505 shutdown_connections 2024-02-17T12:08:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ada59c640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1ac80a2270 0x7f1ac80a4730 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x5564667fc3c0 tx=0x7f1ad40115b0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ae37d3740 1 -- 172.21.15.94:0/1117992505 wait complete. 2024-02-17T12:08:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ae37d3740 1 -- start start 2024-02-17T12:08:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.167+0000 7f1ae37d3740 1 Processor -- start 2024-02-17T12:08:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1ae37d3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564667faef0 0x5564668013a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1ae37d3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667fbc60 0x5564668018e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1ae37d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556466804470 0x556466801e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1ae37d3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556466707bf0 con 0x5564667faef0 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1ae37d3740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556466436250 con 0x5564667fbc60 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1ae37d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564666e8200 con 0x556466804470 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1adad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564667faef0 0x5564668013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1adad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564667faef0 0x5564668013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47068/0 (socket says 172.21.15.94:47068) 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1ae37d3740 10 client.0 _ll_register_callbacks cb 0x556466756d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1adad9d640 1 -- 172.21.15.94:0/802477680 learned_addr learned my addr 172.21.15.94:0/802477680 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1ad9d9b640 1 --2- 172.21.15.94:0/802477680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667fbc60 0x5564668018e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1adad9d640 1 --2- 172.21.15.94:0/802477680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556466804470 0x556466801e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1ad9d9b640 1 -- 172.21.15.94:0/802477680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556466804470 msgr2=0x556466801e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1ad9d9b640 1 --2- 172.21.15.94:0/802477680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556466804470 0x556466801e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1ad9d9b640 1 -- 172.21.15.94:0/802477680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564667faef0 msgr2=0x5564668013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1ad9d9b640 1 --2- 172.21.15.94:0/802477680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564667faef0 0x5564668013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.168+0000 7f1ad9d9b640 1 -- 172.21.15.94:0/802477680 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac40099d0 con 0x5564667fbc60 2024-02-17T12:08:48.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.169+0000 7f1ad9d9b640 1 --2- 172.21.15.94:0/802477680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667fbc60 0x5564668018e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009460 tx=0x7f1acc034650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ba38dcdaed9d2bd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.169+0000 7f1ac1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.169+0000 7f1ac1ffb640 1 -- 172.21.15.94:0/802477680 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc005020 con 0x5564667fbc60 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.169+0000 7f1ac1ffb640 1 -- 172.21.15.94:0/802477680 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1acc005180 con 0x5564667fbc60 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.169+0000 7f1ae37d3740 1 -- 172.21.15.94:0/802477680 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564664b31b0 con 0x5564667fbc60 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.169+0000 7f1ac1ffb640 1 -- 172.21.15.94:0/802477680 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1acc005340 con 0x5564667fbc60 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.169+0000 7f1ae37d3740 10 client.27743 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.169+0000 7f1ae37d3740 1 -- 172.21.15.94:0/802477680 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556466760d80 con 0x5564667fbc60 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.169+0000 7f1ac1ffb640 1 -- 172.21.15.94:0/802477680 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5564664b31b0 con 0x5564667fbc60 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.170+0000 7f1ac1ffb640 1 -- 172.21.15.94:0/802477680 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1acc04d070 con 0x5564667fbc60 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.170+0000 7f1ae37d3740 10 client.27743 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.170+0000 7f1ae37d3740 10 client.27743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.170+0000 7f1ae37d3740 20 client.27743 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.170+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/802477680 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556466814850 0x556466816c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.170+0000 7f1ae37d3740 4 client.27743 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.170+0000 7f1ae37d3740 1 -- 172.21.15.94:0/802477680 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55646649f550 con 0x556466814850 2024-02-17T12:08:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.170+0000 7f1adad9d640 1 --2- 172.21.15.94:0/802477680 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556466814850 0x556466816c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.170+0000 7f1ac1ffb640 10 client.27743 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.170+0000 7f1adad9d640 1 --2- 172.21.15.94:0/802477680 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556466814850 0x556466816c70 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf0d8a5cc7bafac2 server_cookie=ce8cfb06593847bd in_seq=0 out_seq=0 2024-02-17T12:08:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ac1ffb640 1 -- 172.21.15.94:0/802477680 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55646649f550 con 0x556466814850 2024-02-17T12:08:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ac1ffb640 10 client.27743 handle_command_reply: tid=0 2024-02-17T12:08:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ae37d3740 1 client.27743 shutdown 2024-02-17T12:08:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ae37d3740 1 -- 172.21.15.94:0/802477680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667fbc60 msgr2=0x5564668018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/802477680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667fbc60 0x5564668018e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1acc009460 tx=0x7f1acc034650 comp rx=0 tx=0).stop 2024-02-17T12:08:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ae37d3740 1 -- 172.21.15.94:0/802477680 shutdown_connections 2024-02-17T12:08:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/802477680 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556466814850 0x556466816c70 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/802477680 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556466804470 0x556466801e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/802477680 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667fbc60 0x5564668018e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/802477680 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564667faef0 0x5564668013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ae37d3740 1 -- 172.21.15.94:0/802477680 >> 172.21.15.94:0/802477680 conn(0x5564667c2550 msgr2=0x5564667f7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ac1ffb640 0 client.27743 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ae37d3740 1 -- 172.21.15.94:0/802477680 shutdown_connections 2024-02-17T12:08:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.171+0000 7f1ae37d3740 1 -- 172.21.15.94:0/802477680 wait complete. 2024-02-17T12:08:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.172+0000 7f1ae37d3740 20 client.27743 trim_cache size 0 max 16384 2024-02-17T12:08:48.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.186+0000 7f1ae37d3740 1 Processor -- start 2024-02-17T12:08:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.186+0000 7f1ae37d3740 1 -- start start 2024-02-17T12:08:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.187+0000 7f1ae37d3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667faef0 0x55646681b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.187+0000 7f1ae37d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667fbc60 0x5564668018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.187+0000 7f1ae37d3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556466804470 0x556466801e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.187+0000 7f1adad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667faef0 0x55646681b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.187+0000 7f1adad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667faef0 0x55646681b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:50014/0 (socket says 172.21.15.94:50014) 2024-02-17T12:08:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.187+0000 7f1adad9d640 1 -- 172.21.15.94:0/2470196484 learned_addr learned my addr 172.21.15.94:0/2470196484 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.187+0000 7f1ae37d3740 1 -- 172.21.15.94:0/2470196484 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564667f67b0 con 0x556466804470 2024-02-17T12:08:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.187+0000 7f1ae37d3740 1 -- 172.21.15.94:0/2470196484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564668112c0 con 0x5564667faef0 2024-02-17T12:08:48.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.187+0000 7f1ae37d3740 1 -- 172.21.15.94:0/2470196484 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556466802830 con 0x5564667fbc60 2024-02-17T12:08:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.188+0000 7f1ad9d9b640 1 --2- 172.21.15.94:0/2470196484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667fbc60 0x5564668018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.188+0000 7f1adad9d640 1 -- 172.21.15.94:0/2470196484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667fbc60 msgr2=0x5564668018f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.188+0000 7f1adad9d640 1 --2- 172.21.15.94:0/2470196484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667fbc60 0x5564668018f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.188+0000 7f1adad9d640 1 -- 172.21.15.94:0/2470196484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556466804470 msgr2=0x556466801e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.188+0000 7f1adad9d640 1 --2- 172.21.15.94:0/2470196484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556466804470 0x556466801e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.188+0000 7f1adad9d640 1 -- 172.21.15.94:0/2470196484 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564666fd7b0 con 0x5564667faef0 2024-02-17T12:08:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.188+0000 7f1adad9d640 1 --2- 172.21.15.94:0/2470196484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667faef0 0x55646681b5d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4006fd0 tx=0x7f1ac4004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e0d9d5dc7770810 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.189+0000 7f1ac3fff640 1 -- 172.21.15.94:0/2470196484 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40124b0 con 0x5564667faef0 2024-02-17T12:08:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.189+0000 7f1ac3fff640 1 -- 172.21.15.94:0/2470196484 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ac4012cf0 con 0x5564667faef0 2024-02-17T12:08:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.189+0000 7f1ac3fff640 1 -- 172.21.15.94:0/2470196484 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4005e10 con 0x5564667faef0 2024-02-17T12:08:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.190+0000 7f1ae37d3740 1 -- 172.21.15.94:0/2470196484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667faef0 msgr2=0x55646681b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.190+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/2470196484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667faef0 0x55646681b5d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4006fd0 tx=0x7f1ac4004700 comp rx=0 tx=0).stop 2024-02-17T12:08:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.190+0000 7f1ae37d3740 1 -- 172.21.15.94:0/2470196484 shutdown_connections 2024-02-17T12:08:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.190+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/2470196484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556466804470 0x556466801e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.190+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/2470196484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667fbc60 0x5564668018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.190+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/2470196484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5564667faef0 0x55646681b5d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.190+0000 7f1ae37d3740 1 -- 172.21.15.94:0/2470196484 >> 172.21.15.94:0/2470196484 conn(0x5564667c2550 msgr2=0x5564667fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.190+0000 7f1ae37d3740 1 -- 172.21.15.94:0/2470196484 shutdown_connections 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.190+0000 7f1ae37d3740 1 -- 172.21.15.94:0/2470196484 wait complete. 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.190+0000 7f1ae37d3740 1 -- start start 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.190+0000 7f1ae37d3740 1 Processor -- start 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1ae37d3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564667faef0 0x55646680b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1ae37d3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667fbc60 0x55646680b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1ae37d3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556466804470 0x556466809920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1ae37d3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556466809ff0 con 0x5564667faef0 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1ae37d3740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646680a160 con 0x556466804470 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1ae37d3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55646680a320 con 0x5564667fbc60 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1ae37d3740 10 client.0 _ll_register_callbacks cb 0x7f1ac40097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1adad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564667faef0 0x55646680b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1adad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564667faef0 0x55646680b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47088/0 (socket says 172.21.15.94:47088) 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1adad9d640 1 -- 172.21.15.94:0/1908299039 learned_addr learned my addr 172.21.15.94:0/1908299039 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1adad9d640 1 --2- 172.21.15.94:0/1908299039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556466804470 0x556466809920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1ad9d9b640 1 --2- 172.21.15.94:0/1908299039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667fbc60 0x55646680b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1adad9d640 1 -- 172.21.15.94:0/1908299039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667fbc60 msgr2=0x55646680b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1adad9d640 1 --2- 172.21.15.94:0/1908299039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667fbc60 0x55646680b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1adad9d640 1 -- 172.21.15.94:0/1908299039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556466804470 msgr2=0x556466809920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1adad9d640 1 --2- 172.21.15.94:0/1908299039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556466804470 0x556466809920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1adad9d640 1 -- 172.21.15.94:0/1908299039 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac4025040 con 0x5564667faef0 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1adad9d640 1 --2- 172.21.15.94:0/1908299039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564667faef0 0x55646680b380 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40126c0 tx=0x7f1ac4005a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c70adcec1843e32 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.191+0000 7f1ac1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ac1ffb640 1 -- 172.21.15.94:0/1908299039 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac402b770 con 0x5564667faef0 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ae37d3740 1 -- 172.21.15.94:0/1908299039 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556466462600 con 0x5564667faef0 2024-02-17T12:08:48.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ac1ffb640 1 -- 172.21.15.94:0/1908299039 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1ac4004020 con 0x5564667faef0 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ac1ffb640 1 -- 172.21.15.94:0/1908299039 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ac40041e0 con 0x5564667faef0 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ae37d3740 10 client.18270 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ae37d3740 1 -- 172.21.15.94:0/1908299039 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556466723540 con 0x5564667faef0 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ac1ffb640 1 -- 172.21.15.94:0/1908299039 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1ac4043080 con 0x5564667faef0 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ac1ffb640 1 -- 172.21.15.94:0/1908299039 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1ac4011070 con 0x5564667faef0 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ae37d3740 10 client.18270 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ae37d3740 10 client.18270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ae37d3740 20 client.18270 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/1908299039 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55646686bbd0 0x55646686dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ae37d3740 4 client.18270 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ae37d3740 1 -- 172.21.15.94:0/1908299039 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55646640e520 con 0x55646686bbd0 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.192+0000 7f1ad9d9b640 1 --2- 172.21.15.94:0/1908299039 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55646686bbd0 0x55646686dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.193+0000 7f1ac1ffb640 10 client.18270 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:48.197 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.193+0000 7f1ad9d9b640 1 --2- 172.21.15.94:0/1908299039 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55646686bbd0 0x55646686dfc0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d884f5e8a065779 server_cookie=6f772780b1a9e739 in_seq=0 out_seq=0 2024-02-17T12:08:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.257+0000 7f1ac1ffb640 1 -- 172.21.15.94:0/1908299039 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55646640e520 con 0x55646686bbd0 2024-02-17T12:08:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.257+0000 7f1ac1ffb640 10 client.18270 handle_command_reply: tid=0 2024-02-17T12:08:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1ae37d3740 1 client.18270 shutdown 2024-02-17T12:08:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1ae37d3740 1 -- 172.21.15.94:0/1908299039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564667faef0 msgr2=0x55646680b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/1908299039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564667faef0 0x55646680b380 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40126c0 tx=0x7f1ac4005a30 comp rx=0 tx=0).stop 2024-02-17T12:08:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1ae37d3740 1 -- 172.21.15.94:0/1908299039 shutdown_connections 2024-02-17T12:08:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/1908299039 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55646686bbd0 0x55646686dfc0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/1908299039 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556466804470 0x556466809920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1ac1ffb640 0 client.18270 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:48.260 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/1908299039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5564667fbc60 0x55646680b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1ae37d3740 1 --2- 172.21.15.94:0/1908299039 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5564667faef0 0x55646680b380 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1ae37d3740 1 -- 172.21.15.94:0/1908299039 >> 172.21.15.94:0/1908299039 conn(0x5564667c2550 msgr2=0x5564667ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1ae37d3740 1 -- 172.21.15.94:0/1908299039 shutdown_connections 2024-02-17T12:08:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1ae37d3740 1 -- 172.21.15.94:0/1908299039 wait complete. 2024-02-17T12:08:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1ae37d3740 20 client.18270 trim_cache size 0 max 16384 2024-02-17T12:08:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:08:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:08:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:08:48.261 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:08:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1adb59e640 1 -- 172.21.15.94:0/196848917 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1ac80a2270 msgr2=0x7f1ac80a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1adb59e640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1ac80a2270 0x7f1ac80a4730 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x5564667fc3c0 tx=0x7f1ad40115b0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1adb59e640 1 -- 172.21.15.94:0/196848917 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1adc10d260 msgr2=0x7f1adc11c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.258+0000 7f1adb59e640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1adc10d260 0x7f1adc11c590 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400ef60 tx=0x7f1ad400ef90 comp rx=0 tx=0).stop 2024-02-17T12:08:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.259+0000 7f1adb59e640 1 -- 172.21.15.94:0/196848917 shutdown_connections 2024-02-17T12:08:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.259+0000 7f1adb59e640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1ac80a2270 0x7f1ac80a4730 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.259+0000 7f1adb59e640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1adc10f700 0x7f1adc116b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.259+0000 7f1adb59e640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1adc10dd80 0x7f1adc116600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.259+0000 7f1adb59e640 1 --2- 172.21.15.94:0/196848917 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1adc10d260 0x7f1adc11c590 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.259+0000 7f1adb59e640 1 -- 172.21.15.94:0/196848917 >> 172.21.15.94:0/196848917 conn(0x7f1adc06c090 msgr2=0x7f1adc0707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.259+0000 7f1adb59e640 1 -- 172.21.15.94:0/196848917 shutdown_connections 2024-02-17T12:08:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:08:48.259+0000 7f1adb59e640 1 -- 172.21.15.94:0/196848917 wait complete. 2024-02-17T12:08:49.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:49 smithi189 ceph-mon[36588]: pgmap v332: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 33 MiB/s wr, 2.20k op/s 2024-02-17T12:08:49.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:49 smithi125 ceph-mon[36819]: pgmap v332: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 33 MiB/s wr, 2.20k op/s 2024-02-17T12:08:49.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:49 smithi094 ceph-mon[32499]: pgmap v332: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 33 MiB/s wr, 2.20k op/s 2024-02-17T12:08:50.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 630.0 s, 1795.2 tps, lat 55.507 ms stddev 67.797 2024-02-17T12:08:51.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:51 smithi189 ceph-mon[36588]: pgmap v333: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 30 MiB/s wr, 1.97k op/s 2024-02-17T12:08:51.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:51 smithi125 ceph-mon[36819]: pgmap v333: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 30 MiB/s wr, 1.97k op/s 2024-02-17T12:08:51.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:51 smithi094 ceph-mon[32499]: pgmap v333: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 30 MiB/s wr, 1.97k op/s 2024-02-17T12:08:52.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:08:52.653 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:08:52.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.797+0000 7f234c4ae640 1 -- 172.21.15.94:0/4037649939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f234410f8a0 msgr2=0x7f234410d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:52.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.797+0000 7f234c4ae640 1 --2- 172.21.15.94:0/4037649939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f234410f8a0 0x7f234410d4c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f2334009900 tx=0x7f233402bc80 comp rx=0 tx=0).stop 2024-02-17T12:08:52.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.802+0000 7f234c4ae640 1 -- 172.21.15.94:0/4037649939 shutdown_connections 2024-02-17T12:08:52.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.802+0000 7f234c4ae640 1 --2- 172.21.15.94:0/4037649939 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f234411a960 0x7f234411cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.802+0000 7f234c4ae640 1 --2- 172.21.15.94:0/4037649939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f234410fd10 0x7f234410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.802+0000 7f234c4ae640 1 --2- 172.21.15.94:0/4037649939 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f234410f8a0 0x7f234410d4c0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.802+0000 7f234c4ae640 1 -- 172.21.15.94:0/4037649939 >> 172.21.15.94:0/4037649939 conn(0x7f234406c270 msgr2=0x7f234406c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:52.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.803+0000 7f234c4ae640 1 -- 172.21.15.94:0/4037649939 shutdown_connections 2024-02-17T12:08:52.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.804+0000 7f234c4ae640 1 -- 172.21.15.94:0/4037649939 wait complete. 2024-02-17T12:08:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.805+0000 7f234c4ae640 1 Processor -- start 2024-02-17T12:08:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.805+0000 7f234c4ae640 1 -- start start 2024-02-17T12:08:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.805+0000 7f234c4ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f234410f8a0 0x7f234411a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.805+0000 7f234c4ae640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f234410fd10 0x7f2344114100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.805+0000 7f234c4ae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f234411a960 0x7f2344114640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.805+0000 7f234c4ae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2344114c40 con 0x7f234410f8a0 2024-02-17T12:08:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.805+0000 7f234c4ae640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2344114db0 con 0x7f234410fd10 2024-02-17T12:08:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.805+0000 7f234c4ae640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2344114f70 con 0x7f234411a960 2024-02-17T12:08:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.805+0000 7f234aa4b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f234411a960 0x7f2344114640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.805+0000 7f2349a49640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f234410fd10 0x7f2344114100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.805+0000 7f2349a49640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f234410fd10 0x7f2344114100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:50048/0 (socket says 172.21.15.94:50048) 2024-02-17T12:08:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.805+0000 7f2349a49640 1 -- 172.21.15.94:0/1626058977 learned_addr learned my addr 172.21.15.94:0/1626058977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.806+0000 7f234a24a640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f234410f8a0 0x7f234411a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.806+0000 7f2349a49640 1 -- 172.21.15.94:0/1626058977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f234411a960 msgr2=0x7f2344114640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.806+0000 7f2349a49640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f234411a960 0x7f2344114640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.806+0000 7f2349a49640 1 -- 172.21.15.94:0/1626058977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f234410f8a0 msgr2=0x7f234411a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.806+0000 7f2349a49640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f234410f8a0 0x7f234411a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.806+0000 7f2349a49640 1 -- 172.21.15.94:0/1626058977 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2334009560 con 0x7f234410fd10 2024-02-17T12:08:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.806+0000 7f234a24a640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f234410f8a0 0x7f234411a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:08:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.806+0000 7f2349a49640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f234410fd10 0x7f2344114100 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f233c00b9e0 tx=0x7f233c00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.806+0000 7f233b7fe640 1 -- 172.21.15.94:0/1626058977 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c00cb40 con 0x7f234410fd10 2024-02-17T12:08:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.806+0000 7f234c4ae640 1 -- 172.21.15.94:0/1626058977 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234411fa40 con 0x7f234410fd10 2024-02-17T12:08:52.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.806+0000 7f234c4ae640 1 -- 172.21.15.94:0/1626058977 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234411ff50 con 0x7f234410fd10 2024-02-17T12:08:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.807+0000 7f233b7fe640 1 -- 172.21.15.94:0/1626058977 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f233c00cca0 con 0x7f234410fd10 2024-02-17T12:08:52.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.807+0000 7f233b7fe640 1 -- 172.21.15.94:0/1626058977 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c012740 con 0x7f234410fd10 2024-02-17T12:08:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f233b7fe640 1 -- 172.21.15.94:0/1626058977 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f233c0129c0 con 0x7f234410fd10 2024-02-17T12:08:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f234f084740 1 Processor -- start 2024-02-17T12:08:52.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f234f084740 1 -- start start 2024-02-17T12:08:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f234f084740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d4c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f234f084740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d55270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f234f084740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 0x563438d57ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f234f084740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563438cbb280 con 0x563438d4c2b0 2024-02-17T12:08:52.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f234f084740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563438cb87b0 con 0x563438d4cbd0 2024-02-17T12:08:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f234f084740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563438cb3f90 con 0x563438d557b0 2024-02-17T12:08:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f234a24a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d55270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f234a24a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d55270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:50056/0 (socket says 172.21.15.94:50056) 2024-02-17T12:08:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f234a24a640 1 -- 172.21.15.94:0/2008379297 learned_addr learned my addr 172.21.15.94:0/2008379297 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f2349a49640 1 --2- 172.21.15.94:0/2008379297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 0x563438d57ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.810+0000 7f234aa4b640 1 --2- 172.21.15.94:0/2008379297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d4c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.811+0000 7f233b7fe640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f23300a2370 0x7f23300a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.811+0000 7f234aa4b640 1 -- 172.21.15.94:0/2008379297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 msgr2=0x563438d57ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.811+0000 7f234aa4b640 1 --2- 172.21.15.94:0/2008379297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 0x563438d57ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.811+0000 7f234aa4b640 1 -- 172.21.15.94:0/2008379297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 msgr2=0x563438d55270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.811+0000 7f234aa4b640 1 --2- 172.21.15.94:0/2008379297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d55270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.811+0000 7f234aa4b640 1 -- 172.21.15.94:0/2008379297 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563438c72570 con 0x563438d4c2b0 2024-02-17T12:08:52.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.811+0000 7f234aa4b640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f23300a2370 0x7f23300a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.811+0000 7f233b7fe640 1 -- 172.21.15.94:0/1626058977 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f233c0cd800 con 0x7f234410fd10 2024-02-17T12:08:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.811+0000 7f234aa4b640 1 --2- 172.21.15.94:0/2008379297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d4c690 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f234406a5b0 tx=0x7f2340009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b7d2a083ef536 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.811+0000 7f2338bfe640 1 -- 172.21.15.94:0/2008379297 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234000edb0 con 0x563438d4c2b0 2024-02-17T12:08:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.811+0000 7f2338bfe640 1 -- 172.21.15.94:0/2008379297 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2340018070 con 0x563438d4c2b0 2024-02-17T12:08:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.811+0000 7f2338bfe640 1 -- 172.21.15.94:0/2008379297 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23400143f0 con 0x563438d4c2b0 2024-02-17T12:08:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.812+0000 7f234f084740 1 -- 172.21.15.94:0/2008379297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 msgr2=0x563438d4c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.812+0000 7f234f084740 1 --2- 172.21.15.94:0/2008379297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d4c690 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f234406a5b0 tx=0x7f2340009f10 comp rx=0 tx=0).stop 2024-02-17T12:08:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.812+0000 7f234aa4b640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f23300a2370 0x7f23300a4830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f234001e000 tx=0x7f234001b000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.812+0000 7f234f084740 1 -- 172.21.15.94:0/2008379297 shutdown_connections 2024-02-17T12:08:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.812+0000 7f234f084740 1 --2- 172.21.15.94:0/2008379297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 0x563438d57ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.812+0000 7f234f084740 1 --2- 172.21.15.94:0/2008379297 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d55270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.812+0000 7f234f084740 1 --2- 172.21.15.94:0/2008379297 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d4c690 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.812+0000 7f234f084740 1 -- 172.21.15.94:0/2008379297 >> 172.21.15.94:0/2008379297 conn(0x563438d46990 msgr2=0x563438d46d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.812+0000 7f234f084740 1 -- 172.21.15.94:0/2008379297 shutdown_connections 2024-02-17T12:08:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.812+0000 7f234f084740 1 -- 172.21.15.94:0/2008379297 wait complete. 2024-02-17T12:08:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.813+0000 7f234f084740 1 -- start start 2024-02-17T12:08:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.813+0000 7f234f084740 1 Processor -- start 2024-02-17T12:08:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.813+0000 7f234f084740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d55120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.813+0000 7f234f084740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.813+0000 7f234f084740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 0x563438d52a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.813+0000 7f234f084740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563438987430 con 0x563438d4c2b0 2024-02-17T12:08:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.813+0000 7f234f084740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563438943e00 con 0x563438d4cbd0 2024-02-17T12:08:52.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.813+0000 7f234f084740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563438c38fd0 con 0x563438d557b0 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.813+0000 7f234f084740 10 client.0 _ll_register_callbacks cb 0x563438c8efd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.813+0000 7f234a24a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d55120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.813+0000 7f234a24a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d55120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47116/0 (socket says 172.21.15.94:47116) 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.813+0000 7f234a24a640 1 -- 172.21.15.94:0/94304619 learned_addr learned my addr 172.21.15.94:0/94304619 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.813+0000 7f2349a49640 1 --2- 172.21.15.94:0/94304619 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f234a24a640 1 -- 172.21.15.94:0/94304619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 msgr2=0x563438d52a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f234a24a640 1 --2- 172.21.15.94:0/94304619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 0x563438d52a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f234a24a640 1 -- 172.21.15.94:0/94304619 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 msgr2=0x563438d524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f234a24a640 1 --2- 172.21.15.94:0/94304619 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f234a24a640 1 -- 172.21.15.94:0/94304619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23400095b0 con 0x563438d4c2b0 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f2349a49640 1 --2- 172.21.15.94:0/94304619 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f234a24a640 1 --2- 172.21.15.94:0/94304619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d55120 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2334009900 tx=0x7f233402cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506536c6884ac1f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f232dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:08:52.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f232dffb640 1 -- 172.21.15.94:0/94304619 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233402ccf0 con 0x563438d4c2b0 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f232dffb640 1 -- 172.21.15.94:0/94304619 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2334004450 con 0x563438d4c2b0 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f234f084740 1 -- 172.21.15.94:0/94304619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634389b37e0 con 0x563438d4c2b0 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f232dffb640 1 -- 172.21.15.94:0/94304619 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2334004df0 con 0x563438d4c2b0 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f234f084740 10 client.18279 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f234f084740 1 -- 172.21.15.94:0/94304619 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x563438cb2610 con 0x563438d4c2b0 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.814+0000 7f232dffb640 1 -- 172.21.15.94:0/94304619 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2334053080 con 0x563438d4c2b0 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.815+0000 7f232dffb640 1 -- 172.21.15.94:0/94304619 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2334045020 con 0x563438d4c2b0 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.815+0000 7f234f084740 10 client.18279 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.815+0000 7f234f084740 10 client.18279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.815+0000 7f234f084740 20 client.18279 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.815+0000 7f234f084740 1 --2- 172.21.15.94:0/94304619 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563438d65980 0x563438d67dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.815+0000 7f234f084740 4 client.18279 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.815+0000 7f234f084740 1 -- 172.21.15.94:0/94304619 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634389f0720 con 0x563438d65980 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.815+0000 7f2349a49640 1 --2- 172.21.15.94:0/94304619 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563438d65980 0x563438d67dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.824 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.815+0000 7f2349a49640 1 --2- 172.21.15.94:0/94304619 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563438d65980 0x563438d67dc0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94503bd2edd0a7ab server_cookie=df364de7cf93b283 in_seq=0 out_seq=0 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.815+0000 7f232dffb640 10 client.18279 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.816+0000 7f232dffb640 1 -- 172.21.15.94:0/94304619 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634389f0720 con 0x563438d65980 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.816+0000 7f232dffb640 10 client.18279 handle_command_reply: tid=0 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.816+0000 7f234f084740 1 client.18279 shutdown 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.817+0000 7f234f084740 1 -- 172.21.15.94:0/94304619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 msgr2=0x563438d55120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.817+0000 7f234f084740 1 --2- 172.21.15.94:0/94304619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d55120 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f2334009900 tx=0x7f233402cb80 comp rx=0 tx=0).stop 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.817+0000 7f234f084740 1 -- 172.21.15.94:0/94304619 shutdown_connections 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.817+0000 7f234f084740 1 --2- 172.21.15.94:0/94304619 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563438d65980 0x563438d67dc0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.817+0000 7f234f084740 1 --2- 172.21.15.94:0/94304619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 0x563438d52a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.817+0000 7f232dffb640 0 client.18279 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.817+0000 7f234f084740 1 --2- 172.21.15.94:0/94304619 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.817+0000 7f234f084740 1 --2- 172.21.15.94:0/94304619 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d55120 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.817+0000 7f234f084740 1 -- 172.21.15.94:0/94304619 >> 172.21.15.94:0/94304619 conn(0x563438d46990 msgr2=0x563438d48730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.817+0000 7f234f084740 1 -- 172.21.15.94:0/94304619 shutdown_connections 2024-02-17T12:08:52.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.817+0000 7f234f084740 1 -- 172.21.15.94:0/94304619 wait complete. 2024-02-17T12:08:52.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.818+0000 7f234f084740 20 client.18279 trim_cache size 0 max 16384 2024-02-17T12:08:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.837+0000 7f234f084740 1 Processor -- start 2024-02-17T12:08:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.837+0000 7f234f084740 1 -- start start 2024-02-17T12:08:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.837+0000 7f234f084740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d4c2b0 0x563438d67da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.837+0000 7f234f084740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d54be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.837+0000 7f234f084740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d557b0 0x563438d55120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.837+0000 7f234f084740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563438d47790 con 0x563438d557b0 2024-02-17T12:08:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.837+0000 7f234f084740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563438d61d30 con 0x563438d4cbd0 2024-02-17T12:08:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.837+0000 7f234f084740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563438d4bc80 con 0x563438d4c2b0 2024-02-17T12:08:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f2349a49640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d54be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f2349a49640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d54be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:50064/0 (socket says 172.21.15.94:50064) 2024-02-17T12:08:52.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f2349a49640 1 -- 172.21.15.94:0/3200487057 learned_addr learned my addr 172.21.15.94:0/3200487057 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f234a24a640 1 --2- 172.21.15.94:0/3200487057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d4c2b0 0x563438d67da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f234aa4b640 1 --2- 172.21.15.94:0/3200487057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d557b0 0x563438d55120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f2349a49640 1 -- 172.21.15.94:0/3200487057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d4c2b0 msgr2=0x563438d67da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f2349a49640 1 --2- 172.21.15.94:0/3200487057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d4c2b0 0x563438d67da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f2349a49640 1 -- 172.21.15.94:0/3200487057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d557b0 msgr2=0x563438d55120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f2349a49640 1 --2- 172.21.15.94:0/3200487057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d557b0 0x563438d55120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f2349a49640 1 -- 172.21.15.94:0/3200487057 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563438c4eac0 con 0x563438d4cbd0 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f234a24a640 1 --2- 172.21.15.94:0/3200487057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d4c2b0 0x563438d67da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f2349a49640 1 --2- 172.21.15.94:0/3200487057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d54be0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f233c09a2a0 tx=0x7f233c00e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef23e6e6ff8cbb35 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f232ffff640 1 -- 172.21.15.94:0/3200487057 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c07ea90 con 0x563438d4cbd0 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f232ffff640 1 -- 172.21.15.94:0/3200487057 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f233c09e1b0 con 0x563438d4cbd0 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.838+0000 7f232ffff640 1 -- 172.21.15.94:0/3200487057 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c012ed0 con 0x563438d4cbd0 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.839+0000 7f234f084740 1 -- 172.21.15.94:0/3200487057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 msgr2=0x563438d54be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:52.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.839+0000 7f234f084740 1 --2- 172.21.15.94:0/3200487057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d54be0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f233c09a2a0 tx=0x7f233c00e040 comp rx=0 tx=0).stop 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.839+0000 7f234f084740 1 -- 172.21.15.94:0/3200487057 shutdown_connections 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.839+0000 7f234f084740 1 --2- 172.21.15.94:0/3200487057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d557b0 0x563438d55120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.839+0000 7f234f084740 1 --2- 172.21.15.94:0/3200487057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d54be0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.839+0000 7f234f084740 1 --2- 172.21.15.94:0/3200487057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d4c2b0 0x563438d67da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.839+0000 7f234f084740 1 -- 172.21.15.94:0/3200487057 >> 172.21.15.94:0/3200487057 conn(0x563438d46990 msgr2=0x563438d50d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.839+0000 7f234f084740 1 -- 172.21.15.94:0/3200487057 shutdown_connections 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.839+0000 7f234f084740 1 -- 172.21.15.94:0/3200487057 wait complete. 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.840+0000 7f234f084740 1 -- start start 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.840+0000 7f234f084740 1 Processor -- start 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.840+0000 7f234f084740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d4f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.840+0000 7f234f084740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d4f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.840+0000 7f234f084740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 0x563438d4fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.840+0000 7f234a24a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d4f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.840+0000 7f234f084740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x563438dbc000 con 0x563438d4c2b0 2024-02-17T12:08:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.840+0000 7f234a24a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d4f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47146/0 (socket says 172.21.15.94:47146) 2024-02-17T12:08:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.840+0000 7f234a24a640 1 -- 172.21.15.94:0/707521675 learned_addr learned my addr 172.21.15.94:0/707521675 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.840+0000 7f2349a49640 1 --2- 172.21.15.94:0/707521675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d4f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.840+0000 7f234f084740 1 -- 172.21.15.94:0/707521675 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x563438dbc170 con 0x563438d4cbd0 2024-02-17T12:08:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.840+0000 7f234f084740 1 -- 172.21.15.94:0/707521675 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x563438dbc330 con 0x563438d557b0 2024-02-17T12:08:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f234aa4b640 1 --2- 172.21.15.94:0/707521675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 0x563438d4fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f234f084740 10 client.0 _ll_register_callbacks cb 0x7f2340009350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f234a24a640 1 -- 172.21.15.94:0/707521675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 msgr2=0x563438d4fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f234a24a640 1 --2- 172.21.15.94:0/707521675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 0x563438d4fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f234a24a640 1 -- 172.21.15.94:0/707521675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 msgr2=0x563438d4f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f234a24a640 1 --2- 172.21.15.94:0/707521675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d4f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f234a24a640 1 -- 172.21.15.94:0/707521675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234411ff50 con 0x563438d4c2b0 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f234f084740 1 -- 172.21.15.94:0/707521675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2340018070 con 0x563438d4c2b0 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f234a24a640 1 --2- 172.21.15.94:0/707521675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d4f380 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f233402f020 tx=0x7f233402c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da63b99828d02016 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f232dffb640 10 client.18285 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f232dffb640 1 -- 172.21.15.94:0/707521675 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23340037f0 con 0x563438d4c2b0 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f232dffb640 1 -- 172.21.15.94:0/707521675 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2334000e40 con 0x563438d4c2b0 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f232dffb640 1 -- 172.21.15.94:0/707521675 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2334003950 con 0x563438d4c2b0 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f234f084740 10 client.18285 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.841+0000 7f234f084740 1 -- 172.21.15.94:0/707521675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x563438c746a0 con 0x563438d4c2b0 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.842+0000 7f232dffb640 1 -- 172.21.15.94:0/707521675 <== mon.0 v2:172.21.15.94:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2334045040 con 0x563438d4c2b0 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.842+0000 7f234f084740 10 client.18285 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.842+0000 7f234f084740 10 client.18285 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.842+0000 7f234f084740 20 client.18285 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.842+0000 7f234f084740 1 --2- 172.21.15.94:0/707521675 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563438d65980 0x563438dbf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:52.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.842+0000 7f234f084740 4 client.18285 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.842+0000 7f234f084740 1 -- 172.21.15.94:0/707521675 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563438a3f890 con 0x563438d65980 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.842+0000 7f2349a49640 1 --2- 172.21.15.94:0/707521675 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563438d65980 0x563438dbf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.842+0000 7f232dffb640 10 client.18285 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.842+0000 7f2349a49640 1 --2- 172.21.15.94:0/707521675 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563438d65980 0x563438dbf530 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fa3f7342ce5c532 server_cookie=2e7751657d252acd in_seq=0 out_seq=0 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.842+0000 7f232dffb640 1 -- 172.21.15.94:0/707521675 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563438a3f890 con 0x563438d65980 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.842+0000 7f232dffb640 10 client.18285 handle_command_reply: tid=0 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.842+0000 7f234f084740 1 client.18285 shutdown 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.843+0000 7f234f084740 1 -- 172.21.15.94:0/707521675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 msgr2=0x563438d4f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.843+0000 7f234f084740 1 --2- 172.21.15.94:0/707521675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d4f380 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f233402f020 tx=0x7f233402c8c0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.843+0000 7f234f084740 1 -- 172.21.15.94:0/707521675 shutdown_connections 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.843+0000 7f234f084740 1 --2- 172.21.15.94:0/707521675 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x563438d65980 0x563438dbf530 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.843+0000 7f234f084740 1 --2- 172.21.15.94:0/707521675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563438d557b0 0x563438d4fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.843+0000 7f232dffb640 0 client.18285 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.843+0000 7f234f084740 1 --2- 172.21.15.94:0/707521675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x563438d4cbd0 0x563438d4f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.850 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.843+0000 7f234f084740 1 --2- 172.21.15.94:0/707521675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x563438d4c2b0 0x563438d4f380 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.843+0000 7f234f084740 1 -- 172.21.15.94:0/707521675 >> 172.21.15.94:0/707521675 conn(0x563438d46990 msgr2=0x563438d51c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.843+0000 7f234f084740 1 -- 172.21.15.94:0/707521675 shutdown_connections 2024-02-17T12:08:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.843+0000 7f234f084740 1 -- 172.21.15.94:0/707521675 wait complete. 2024-02-17T12:08:52.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.843+0000 7f234f084740 20 client.18285 trim_cache size 0 max 16384 2024-02-17T12:08:52.851 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:08:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:08:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:08:52.852 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:08:52.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.845+0000 7f234c4ae640 1 -- 172.21.15.94:0/1626058977 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f23300a2370 msgr2=0x7f23300a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:52.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.845+0000 7f234c4ae640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f23300a2370 0x7f23300a4830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f234001e000 tx=0x7f234001b000 comp rx=0 tx=0).stop 2024-02-17T12:08:52.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.845+0000 7f234c4ae640 1 -- 172.21.15.94:0/1626058977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f234410fd10 msgr2=0x7f2344114100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:52.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.845+0000 7f234c4ae640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f234410fd10 0x7f2344114100 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f233c00b9e0 tx=0x7f233c00bea0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.845+0000 7f234c4ae640 1 -- 172.21.15.94:0/1626058977 shutdown_connections 2024-02-17T12:08:52.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.845+0000 7f234c4ae640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f23300a2370 0x7f23300a4830 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.845+0000 7f234c4ae640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f234411a960 0x7f2344114640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.845+0000 7f234c4ae640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f234410fd10 0x7f2344114100 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.845+0000 7f234c4ae640 1 --2- 172.21.15.94:0/1626058977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f234410f8a0 0x7f234411a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:52.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.845+0000 7f234c4ae640 1 -- 172.21.15.94:0/1626058977 >> 172.21.15.94:0/1626058977 conn(0x7f234406c270 msgr2=0x7f234410c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:52.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.845+0000 7f234c4ae640 1 -- 172.21.15.94:0/1626058977 shutdown_connections 2024-02-17T12:08:52.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.845+0000 7f234c4ae640 1 -- 172.21.15.94:0/1626058977 wait complete. 2024-02-17T12:08:52.859 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:08:52.859 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:08:53.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.998+0000 7fe6aa84d640 1 -- 172.21.15.94:0/3644815837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a410d9e0 msgr2=0x7fe6a410de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:52.998+0000 7fe6aa84d640 1 --2- 172.21.15.94:0/3644815837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a410d9e0 0x7fe6a410de60 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fe69401a940 tx=0x7fe69403cc70 comp rx=0 tx=0).stop 2024-02-17T12:08:53.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.004+0000 7fe6aa84d640 1 -- 172.21.15.94:0/3644815837 shutdown_connections 2024-02-17T12:08:53.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.004+0000 7fe6aa84d640 1 --2- 172.21.15.94:0/3644815837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6a411a960 0x7fe6a411cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.004+0000 7fe6aa84d640 1 --2- 172.21.15.94:0/3644815837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a410d9e0 0x7fe6a410de60 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.004+0000 7fe6aa84d640 1 --2- 172.21.15.94:0/3644815837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a410f9b0 0x7fe6a410d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.004+0000 7fe6aa84d640 1 -- 172.21.15.94:0/3644815837 >> 172.21.15.94:0/3644815837 conn(0x7fe6a406c270 msgr2=0x7fe6a406c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.009+0000 7fe6aa84d640 1 -- 172.21.15.94:0/3644815837 shutdown_connections 2024-02-17T12:08:53.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.010+0000 7fe6aa84d640 1 -- 172.21.15.94:0/3644815837 wait complete. 2024-02-17T12:08:53.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.012+0000 7fe6aa84d640 1 Processor -- start 2024-02-17T12:08:53.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.016+0000 7fe6aa84d640 1 -- start start 2024-02-17T12:08:53.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.016+0000 7fe6aa84d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6a410d9e0 0x7fe6a4119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.016+0000 7fe6aa84d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a410f9b0 0x7fe6a4116010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.016+0000 7fe6aa84d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a411a960 0x7fe6a4116550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.016+0000 7fe6aa84d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a4116a90 con 0x7fe6a411a960 2024-02-17T12:08:53.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.016+0000 7fe6aa84d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a4116c00 con 0x7fe6a410d9e0 2024-02-17T12:08:53.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.016+0000 7fe6aa84d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a4116dc0 con 0x7fe6a410f9b0 2024-02-17T12:08:53.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.020+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6a410d9e0 0x7fe6a4119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.020+0000 7fe6a3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6a410d9e0 0x7fe6a4119f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:50106/0 (socket says 172.21.15.94:50106) 2024-02-17T12:08:53.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.020+0000 7fe6a3fff640 1 -- 172.21.15.94:0/1269439322 learned_addr learned my addr 172.21.15.94:0/1269439322 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.023+0000 7fe6a3fff640 1 -- 172.21.15.94:0/1269439322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a410f9b0 msgr2=0x7fe6a4116010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.023+0000 7fe6a3fff640 1 --2- 172.21.15.94:0/1269439322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a410f9b0 0x7fe6a4116010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.023+0000 7fe6a3fff640 1 -- 172.21.15.94:0/1269439322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a411a960 msgr2=0x7fe6a4116550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.023+0000 7fe6a3fff640 1 --2- 172.21.15.94:0/1269439322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a411a960 0x7fe6a4116550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.023+0000 7fe6a3fff640 1 -- 172.21.15.94:0/1269439322 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe69401a570 con 0x7fe6a410d9e0 2024-02-17T12:08:53.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.027+0000 7fe6a3fff640 1 --2- 172.21.15.94:0/1269439322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6a410d9e0 0x7fe6a4119f80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe69800b450 tx=0x7fe69800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.033+0000 7fe6a17fa640 1 -- 172.21.15.94:0/1269439322 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698011890 con 0x7fe6a410d9e0 2024-02-17T12:08:53.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.033+0000 7fe6a17fa640 1 -- 172.21.15.94:0/1269439322 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe6980103e0 con 0x7fe6a410d9e0 2024-02-17T12:08:53.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.033+0000 7fe6a17fa640 1 -- 172.21.15.94:0/1269439322 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69800f5a0 con 0x7fe6a410d9e0 2024-02-17T12:08:53.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.034+0000 7fe6aa84d640 1 -- 172.21.15.94:0/1269439322 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a41aa5f0 con 0x7fe6a410d9e0 2024-02-17T12:08:53.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.034+0000 7fe6aa84d640 1 -- 172.21.15.94:0/1269439322 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a41aaa80 con 0x7fe6a410d9e0 2024-02-17T12:08:53.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.036+0000 7fe6aa84d640 1 -- 172.21.15.94:0/1269439322 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a406d510 con 0x7fe6a410d9e0 2024-02-17T12:08:53.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.038+0000 7fe6a17fa640 1 -- 172.21.15.94:0/1269439322 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe6980119f0 con 0x7fe6a410d9e0 2024-02-17T12:08:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.039+0000 7fe6a17fa640 1 --2- 172.21.15.94:0/1269439322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe68c0a2380 0x7fe68c0a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.040+0000 7fe6a37fe640 1 --2- 172.21.15.94:0/1269439322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe68c0a2380 0x7fe68c0a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.040+0000 7fe6a17fa640 1 -- 172.21.15.94:0/1269439322 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe6980cde40 con 0x7fe6a410d9e0 2024-02-17T12:08:53.044 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.043+0000 7fe6a37fe640 1 --2- 172.21.15.94:0/1269439322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe68c0a2380 0x7fe68c0a4840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe6940047e0 tx=0x7fe694000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.048+0000 7fe6a17fa640 1 -- 172.21.15.94:0/1269439322 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe698096ba0 con 0x7fe6a410d9e0 2024-02-17T12:08:53.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.369+0000 7fe6aa84d640 1 -- 172.21.15.94:0/1269439322 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6a4067f70 con 0x7fe6a410d9e0 2024-02-17T12:08:53.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.371+0000 7fe6a17fa640 1 -- 172.21.15.94:0/1269439322 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7fe698015090 con 0x7fe6a410d9e0 2024-02-17T12:08:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:08:53.373 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:08:53.373 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:08:53.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.374+0000 7fe6aa84d640 1 -- 172.21.15.94:0/1269439322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe68c0a2380 msgr2=0x7fe68c0a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.374+0000 7fe6aa84d640 1 --2- 172.21.15.94:0/1269439322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe68c0a2380 0x7fe68c0a4840 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe6940047e0 tx=0x7fe694000bc0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.374+0000 7fe6aa84d640 1 -- 172.21.15.94:0/1269439322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6a410d9e0 msgr2=0x7fe6a4119f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.374+0000 7fe6aa84d640 1 --2- 172.21.15.94:0/1269439322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6a410d9e0 0x7fe6a4119f80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe69800b450 tx=0x7fe69800b910 comp rx=0 tx=0).stop 2024-02-17T12:08:53.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.374+0000 7fe6aa84d640 1 -- 172.21.15.94:0/1269439322 shutdown_connections 2024-02-17T12:08:53.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.374+0000 7fe6aa84d640 1 --2- 172.21.15.94:0/1269439322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe68c0a2380 0x7fe68c0a4840 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.374+0000 7fe6aa84d640 1 --2- 172.21.15.94:0/1269439322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe6a411a960 0x7fe6a4116550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.374+0000 7fe6aa84d640 1 --2- 172.21.15.94:0/1269439322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a410f9b0 0x7fe6a4116010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.374+0000 7fe6aa84d640 1 --2- 172.21.15.94:0/1269439322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe6a410d9e0 0x7fe6a4119f80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.374+0000 7fe6aa84d640 1 -- 172.21.15.94:0/1269439322 >> 172.21.15.94:0/1269439322 conn(0x7fe6a406c270 msgr2=0x7fe6a411b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.374+0000 7fe6aa84d640 1 -- 172.21.15.94:0/1269439322 shutdown_connections 2024-02-17T12:08:53.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.374+0000 7fe6aa84d640 1 -- 172.21.15.94:0/1269439322 wait complete. 2024-02-17T12:08:53.387 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:53 smithi094 ceph-mon[32499]: pgmap v334: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 29 MiB/s wr, 1.92k op/s 2024-02-17T12:08:53.388 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:08:53.389 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:08:53.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:53 smithi189 ceph-mon[36588]: pgmap v334: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 29 MiB/s wr, 1.92k op/s 2024-02-17T12:08:53.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:53 smithi125 ceph-mon[36819]: pgmap v334: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 29 MiB/s wr, 1.92k op/s 2024-02-17T12:08:53.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.523+0000 7f9562e23640 1 -- 172.21.15.94:0/2210684445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955c105020 msgr2=0x7f955c105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.523+0000 7f9562e23640 1 --2- 172.21.15.94:0/2210684445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955c105020 0x7f955c105400 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9550009930 tx=0x7f955002bc30 comp rx=0 tx=0).stop 2024-02-17T12:08:53.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.523+0000 7f9562e23640 1 -- 172.21.15.94:0/2210684445 shutdown_connections 2024-02-17T12:08:53.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.523+0000 7f9562e23640 1 --2- 172.21.15.94:0/2210684445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f955c101550 0x7f955c10cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.523+0000 7f9562e23640 1 --2- 172.21.15.94:0/2210684445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f955c100b90 0x7f955c101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.523+0000 7f9562e23640 1 --2- 172.21.15.94:0/2210684445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955c105020 0x7f955c105400 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.523+0000 7f9562e23640 1 -- 172.21.15.94:0/2210684445 >> 172.21.15.94:0/2210684445 conn(0x7f955c0752e0 msgr2=0x7f955c0756f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.523+0000 7f9562e23640 1 -- 172.21.15.94:0/2210684445 shutdown_connections 2024-02-17T12:08:53.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.523+0000 7f9562e23640 1 -- 172.21.15.94:0/2210684445 wait complete. 2024-02-17T12:08:53.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f9562e23640 1 Processor -- start 2024-02-17T12:08:53.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f9562e23640 1 -- start start 2024-02-17T12:08:53.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f9562e23640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f955c100b90 0x7f955c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f9562e23640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f955c101550 0x7f955c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.527 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f9562e23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955c105020 0x7f955c1a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f9562e23640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c1993d0 con 0x7f955c105020 2024-02-17T12:08:53.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f9562e23640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c1a3bd0 con 0x7f955c101550 2024-02-17T12:08:53.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f9562e23640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c1a3d10 con 0x7f955c100b90 2024-02-17T12:08:53.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f95613c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955c105020 0x7f955c1a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f95613c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955c105020 0x7f955c1a3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47184/0 (socket says 172.21.15.94:47184) 2024-02-17T12:08:53.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f95613c0640 1 -- 172.21.15.94:0/3636512 learned_addr learned my addr 172.21.15.94:0/3636512 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f9560bbf640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f955c100b90 0x7f955c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f955bfff640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f955c101550 0x7f955c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f95613c0640 1 -- 172.21.15.94:0/3636512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f955c100b90 msgr2=0x7f955c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f95613c0640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f955c100b90 0x7f955c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f95613c0640 1 -- 172.21.15.94:0/3636512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f955c101550 msgr2=0x7f955c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f95613c0640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f955c101550 0x7f955c198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.528 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f95613c0640 1 -- 172.21.15.94:0/3636512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9550009560 con 0x7f955c105020 2024-02-17T12:08:53.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.524+0000 7f95613c0640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955c105020 0x7f955c1a3690 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f955400db20 tx=0x7f955400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.525+0000 7f9559ffb640 1 -- 172.21.15.94:0/3636512 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955400be40 con 0x7f955c105020 2024-02-17T12:08:53.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.525+0000 7f9559ffb640 1 -- 172.21.15.94:0/3636512 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9554009940 con 0x7f955c105020 2024-02-17T12:08:53.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.525+0000 7f9562e23640 1 -- 172.21.15.94:0/3636512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955c1a3f70 con 0x7f955c105020 2024-02-17T12:08:53.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.525+0000 7f9559ffb640 1 -- 172.21.15.94:0/3636512 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95540106c0 con 0x7f955c105020 2024-02-17T12:08:53.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.525+0000 7f9562e23640 1 -- 172.21.15.94:0/3636512 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c1a4400 con 0x7f955c105020 2024-02-17T12:08:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.527+0000 7f9559ffb640 1 -- 172.21.15.94:0/3636512 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9554004240 con 0x7f955c105020 2024-02-17T12:08:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.527+0000 7f95659f8740 1 Processor -- start 2024-02-17T12:08:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.527+0000 7f95659f8740 1 -- start start 2024-02-17T12:08:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f95659f8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426faaf250 0x56426faaf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f95659f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 0x56426fab8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f9559ffb640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f954c0a2230 0x7f954c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f95659f8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426fab8750 0x56426fabab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f95659f8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426fa6a960 con 0x56426faaf250 2024-02-17T12:08:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f95659f8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426fa1e310 con 0x56426fab8750 2024-02-17T12:08:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f95659f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426fa1b840 con 0x56426faafb70 2024-02-17T12:08:53.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f955bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426faaf250 0x56426faaf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f955bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426faaf250 0x56426faaf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47192/0 (socket says 172.21.15.94:47192) 2024-02-17T12:08:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f955bfff640 1 -- 172.21.15.94:0/393040602 learned_addr learned my addr 172.21.15.94:0/393040602 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f9560bbf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 0x56426fab8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f955bfff640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f954c0a2230 0x7f954c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f95613c0640 1 --2- 172.21.15.94:0/393040602 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426fab8750 0x56426fabab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f9559ffb640 1 -- 172.21.15.94:0/3636512 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f95540cddd0 con 0x7f955c105020 2024-02-17T12:08:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f9560bbf640 1 -- 172.21.15.94:0/393040602 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426fab8750 msgr2=0x56426fabab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f9560bbf640 1 --2- 172.21.15.94:0/393040602 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426fab8750 0x56426fabab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f9560bbf640 1 -- 172.21.15.94:0/393040602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426faaf250 msgr2=0x56426faaf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f9560bbf640 1 --2- 172.21.15.94:0/393040602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426faaf250 0x56426faaf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f9560bbf640 1 -- 172.21.15.94:0/393040602 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56426f9d9060 con 0x56426faafb70 2024-02-17T12:08:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f955bfff640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f954c0a2230 0x7f954c0a46f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9548014930 tx=0x7f9548005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f9560bbf640 1 --2- 172.21.15.94:0/393040602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 0x56426fab8210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f955002c170 tx=0x7f9550005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d81708ac4041649 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f9546ffd640 1 -- 172.21.15.94:0/393040602 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955002c950 con 0x56426faafb70 2024-02-17T12:08:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.528+0000 7f9546ffd640 1 -- 172.21.15.94:0/393040602 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f955002cab0 con 0x56426faafb70 2024-02-17T12:08:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.529+0000 7f9546ffd640 1 -- 172.21.15.94:0/393040602 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550005050 con 0x56426faafb70 2024-02-17T12:08:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.529+0000 7f95659f8740 1 -- 172.21.15.94:0/393040602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 msgr2=0x56426fab8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.529+0000 7f95659f8740 1 --2- 172.21.15.94:0/393040602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 0x56426fab8210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f955002c170 tx=0x7f9550005c00 comp rx=0 tx=0).stop 2024-02-17T12:08:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.529+0000 7f95659f8740 1 -- 172.21.15.94:0/393040602 shutdown_connections 2024-02-17T12:08:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.529+0000 7f95659f8740 1 --2- 172.21.15.94:0/393040602 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426fab8750 0x56426fabab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.529+0000 7f95659f8740 1 --2- 172.21.15.94:0/393040602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 0x56426fab8210 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.529+0000 7f95659f8740 1 --2- 172.21.15.94:0/393040602 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426faaf250 0x56426faaf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.529+0000 7f95659f8740 1 -- 172.21.15.94:0/393040602 >> 172.21.15.94:0/393040602 conn(0x56426faa9980 msgr2=0x56426faa9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.529+0000 7f95659f8740 1 -- 172.21.15.94:0/393040602 shutdown_connections 2024-02-17T12:08:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.529+0000 7f95659f8740 1 -- 172.21.15.94:0/393040602 wait complete. 2024-02-17T12:08:53.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.529+0000 7f95659f8740 1 -- start start 2024-02-17T12:08:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f95659f8740 1 Processor -- start 2024-02-17T12:08:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f95659f8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426faaf250 0x56426fab5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f95659f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 0x56426fab5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f95659f8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426fab8750 0x56426fab61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f95659f8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426fa17020 con 0x56426faaf250 2024-02-17T12:08:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f95659f8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426f6ea430 con 0x56426fab8750 2024-02-17T12:08:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f95659f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426f6a6e00 con 0x56426faafb70 2024-02-17T12:08:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f95659f8740 10 client.0 _ll_register_callbacks cb 0x56426f9becb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f9560bbf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426faaf250 0x56426fab5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f9560bbf640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426faaf250 0x56426fab5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47208/0 (socket says 172.21.15.94:47208) 2024-02-17T12:08:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f9560bbf640 1 -- 172.21.15.94:0/4196901384 learned_addr learned my addr 172.21.15.94:0/4196901384 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f955bfff640 1 --2- 172.21.15.94:0/4196901384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 0x56426fab5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f95613c0640 1 --2- 172.21.15.94:0/4196901384 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426fab8750 0x56426fab61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f9560bbf640 1 -- 172.21.15.94:0/4196901384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 msgr2=0x56426fab5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f9560bbf640 1 --2- 172.21.15.94:0/4196901384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 0x56426fab5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f9560bbf640 1 -- 172.21.15.94:0/4196901384 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426fab8750 msgr2=0x56426fab61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f9560bbf640 1 --2- 172.21.15.94:0/4196901384 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426fab8750 0x56426fab61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f9560bbf640 1 -- 172.21.15.94:0/4196901384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c10d800 con 0x56426faaf250 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f9560bbf640 1 --2- 172.21.15.94:0/4196901384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426faaf250 0x56426fab5760 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f95500092a0 tx=0x7f9550005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7fd620878b317 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.530+0000 7f953bfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f953bfff640 1 -- 172.21.15.94:0/4196901384 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550051070 con 0x56426faaf250 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f953bfff640 1 -- 172.21.15.94:0/4196901384 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9550005420 con 0x56426faaf250 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f95659f8740 1 -- 172.21.15.94:0/4196901384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56426f7167e0 con 0x56426faaf250 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f953bfff640 1 -- 172.21.15.94:0/4196901384 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f955002c950 con 0x56426faaf250 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f95659f8740 10 client.18300 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:53.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f95659f8740 1 -- 172.21.15.94:0/4196901384 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56426fa156a0 con 0x56426faaf250 2024-02-17T12:08:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f953bfff640 1 -- 172.21.15.94:0/4196901384 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56426f7167e0 con 0x56426faaf250 2024-02-17T12:08:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f953bfff640 1 -- 172.21.15.94:0/4196901384 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9550054020 con 0x56426faaf250 2024-02-17T12:08:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f95659f8740 10 client.18300 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f95659f8740 10 client.18300 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:08:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f95659f8740 20 client.18300 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f95659f8740 1 --2- 172.21.15.94:0/4196901384 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x56426fac8830 0x56426facac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f95659f8740 4 client.18300 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f95659f8740 1 -- 172.21.15.94:0/4196901384 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56426f753720 con 0x56426fac8830 2024-02-17T12:08:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.531+0000 7f955bfff640 1 --2- 172.21.15.94:0/4196901384 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x56426fac8830 0x56426facac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f955bfff640 1 --2- 172.21.15.94:0/4196901384 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x56426fac8830 0x56426facac50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a89e1b7625c40c42 server_cookie=569e8224266bab80 in_seq=0 out_seq=0 2024-02-17T12:08:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f953bfff640 10 client.18300 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:08:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f953bfff640 1 -- 172.21.15.94:0/4196901384 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56426f753720 con 0x56426fac8830 2024-02-17T12:08:53.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f953bfff640 10 client.18300 handle_command_reply: tid=0 2024-02-17T12:08:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f95659f8740 1 client.18300 shutdown 2024-02-17T12:08:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f95659f8740 1 -- 172.21.15.94:0/4196901384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426faaf250 msgr2=0x56426fab5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f95659f8740 1 --2- 172.21.15.94:0/4196901384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426faaf250 0x56426fab5760 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f95500092a0 tx=0x7f9550005020 comp rx=0 tx=0).stop 2024-02-17T12:08:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f95659f8740 1 -- 172.21.15.94:0/4196901384 shutdown_connections 2024-02-17T12:08:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f95659f8740 1 --2- 172.21.15.94:0/4196901384 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x56426fac8830 0x56426facac50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f95659f8740 1 --2- 172.21.15.94:0/4196901384 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426fab8750 0x56426fab61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f953bfff640 0 client.18300 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:08:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f95659f8740 1 --2- 172.21.15.94:0/4196901384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 0x56426fab5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f95659f8740 1 --2- 172.21.15.94:0/4196901384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426faaf250 0x56426fab5760 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f95659f8740 1 -- 172.21.15.94:0/4196901384 >> 172.21.15.94:0/4196901384 conn(0x56426faa9980 msgr2=0x56426faab6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f95659f8740 1 -- 172.21.15.94:0/4196901384 shutdown_connections 2024-02-17T12:08:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.532+0000 7f95659f8740 1 -- 172.21.15.94:0/4196901384 wait complete. 2024-02-17T12:08:53.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.533+0000 7f95659f8740 20 client.18300 trim_cache size 0 max 16384 2024-02-17T12:08:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f95659f8740 1 Processor -- start 2024-02-17T12:08:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f95659f8740 1 -- start start 2024-02-17T12:08:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f95659f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faaf250 0x56426fab7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f95659f8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426faafb70 0x56426fab4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f95659f8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 0x56426fab6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f95659f8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426f99c130 con 0x56426fab8750 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f95659f8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426fac85e0 con 0x56426faafb70 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f95659f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426faa52b0 con 0x56426faaf250 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f95613c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 0x56426fab6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f95613c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 0x56426fab6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47222/0 (socket says 172.21.15.94:47222) 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f95613c0640 1 -- 172.21.15.94:0/405581536 learned_addr learned my addr 172.21.15.94:0/405581536 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f955bfff640 1 --2- 172.21.15.94:0/405581536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426faafb70 0x56426fab4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f95613c0640 1 -- 172.21.15.94:0/405581536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faaf250 msgr2=0x56426fab7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.544+0000 7f95613c0640 1 --2- 172.21.15.94:0/405581536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faaf250 0x56426fab7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95613c0640 1 -- 172.21.15.94:0/405581536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426faafb70 msgr2=0x56426fab4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95613c0640 1 --2- 172.21.15.94:0/405581536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426faafb70 0x56426fab4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95613c0640 1 -- 172.21.15.94:0/405581536 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56426f9b1c20 con 0x56426fab8750 2024-02-17T12:08:53.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95613c0640 1 --2- 172.21.15.94:0/405581536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 0x56426fab6000 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f955409a420 tx=0x7f955409b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b906d66a7bb54544 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95467fc640 1 -- 172.21.15.94:0/405581536 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95540b3210 con 0x56426fab8750 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95467fc640 1 -- 172.21.15.94:0/405581536 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f95540b8e90 con 0x56426fab8750 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95467fc640 1 -- 172.21.15.94:0/405581536 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554096d90 con 0x56426fab8750 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95659f8740 1 -- 172.21.15.94:0/405581536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 msgr2=0x56426fab6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95659f8740 1 --2- 172.21.15.94:0/405581536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 0x56426fab6000 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f955409a420 tx=0x7f955409b910 comp rx=0 tx=0).stop 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95659f8740 1 -- 172.21.15.94:0/405581536 shutdown_connections 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95659f8740 1 --2- 172.21.15.94:0/405581536 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 0x56426fab6000 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95659f8740 1 --2- 172.21.15.94:0/405581536 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426faafb70 0x56426fab4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95659f8740 1 --2- 172.21.15.94:0/405581536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faaf250 0x56426fab7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95659f8740 1 -- 172.21.15.94:0/405581536 >> 172.21.15.94:0/405581536 conn(0x56426faa9980 msgr2=0x56426fab4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95659f8740 1 -- 172.21.15.94:0/405581536 shutdown_connections 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.545+0000 7f95659f8740 1 -- 172.21.15.94:0/405581536 wait complete. 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f95659f8740 1 -- start start 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f95659f8740 1 Processor -- start 2024-02-17T12:08:53.555 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f95659f8740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426faaf250 0x56426fabe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f95659f8740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 0x56426fabed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f95659f8740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 0x56426fb1ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f95659f8740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426fb1f2c0 con 0x56426fab8750 2024-02-17T12:08:53.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f95659f8740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426fb1f400 con 0x56426faaf250 2024-02-17T12:08:53.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f95659f8740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56426fb1f540 con 0x56426faafb70 2024-02-17T12:08:53.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f95659f8740 10 client.0 _ll_register_callbacks cb 0x7f9550009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:53.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f95613c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 0x56426fb1ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f95613c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 0x56426fb1ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47238/0 (socket says 172.21.15.94:47238) 2024-02-17T12:08:53.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f95613c0640 1 -- 172.21.15.94:0/3451354029 learned_addr learned my addr 172.21.15.94:0/3451354029 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f9560bbf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426faaf250 0x56426fabe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.546+0000 7f955bfff640 1 --2- 172.21.15.94:0/3451354029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 0x56426fabed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95613c0640 1 -- 172.21.15.94:0/3451354029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 msgr2=0x56426fabed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.556 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95613c0640 1 --2- 172.21.15.94:0/3451354029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 0x56426fabed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95613c0640 1 -- 172.21.15.94:0/3451354029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426faaf250 msgr2=0x56426fabe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95613c0640 1 --2- 172.21.15.94:0/3451354029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426faaf250 0x56426fabe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95613c0640 1 -- 172.21.15.94:0/3451354029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c1a4400 con 0x56426fab8750 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f9560bbf640 1 --2- 172.21.15.94:0/3451354029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426faaf250 0x56426fabe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95613c0640 1 --2- 172.21.15.94:0/3451354029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 0x56426fb1ed80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f95540850b0 tx=0x7f9554009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba3ab8f5e1bbca3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f953bfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f953bfff640 1 -- 172.21.15.94:0/3451354029 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554014030 con 0x56426fab8750 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f953bfff640 1 -- 172.21.15.94:0/3451354029 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9554015070 con 0x56426fab8750 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95659f8740 1 -- 172.21.15.94:0/3451354029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f955002cab0 con 0x56426fab8750 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f953bfff640 1 -- 172.21.15.94:0/3451354029 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95540a2ec0 con 0x56426fab8750 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95659f8740 10 client.18306 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95659f8740 1 -- 172.21.15.94:0/3451354029 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56426f9d78c0 con 0x56426fab8750 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f953bfff640 1 -- 172.21.15.94:0/3451354029 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f955002cab0 con 0x56426fab8750 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f953bfff640 1 -- 172.21.15.94:0/3451354029 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f95540cbc60 con 0x56426fab8750 2024-02-17T12:08:53.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95659f8740 10 client.18306 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95659f8740 10 client.18306 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95659f8740 20 client.18306 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95659f8740 1 --2- 172.21.15.94:0/3451354029 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x56426fb20880 0x56426fb22c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95659f8740 4 client.18306 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.547+0000 7f95659f8740 1 -- 172.21.15.94:0/3451354029 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56426f7a2890 con 0x56426fb20880 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f9560bbf640 1 --2- 172.21.15.94:0/3451354029 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x56426fb20880 0x56426fb22c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f953bfff640 10 client.18306 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f9560bbf640 1 --2- 172.21.15.94:0/3451354029 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x56426fb20880 0x56426fb22c70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=831f4495e724a5c1 server_cookie=1d0bf70d84e1b1d9 in_seq=0 out_seq=0 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f953bfff640 1 -- 172.21.15.94:0/3451354029 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56426f7a2890 con 0x56426fb20880 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f953bfff640 10 client.18306 handle_command_reply: tid=0 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f95659f8740 1 client.18306 shutdown 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f95659f8740 1 -- 172.21.15.94:0/3451354029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 msgr2=0x56426fb1ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f95659f8740 1 --2- 172.21.15.94:0/3451354029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 0x56426fb1ed80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f95540850b0 tx=0x7f9554009320 comp rx=0 tx=0).stop 2024-02-17T12:08:53.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f95659f8740 1 -- 172.21.15.94:0/3451354029 shutdown_connections 2024-02-17T12:08:53.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f95659f8740 1 --2- 172.21.15.94:0/3451354029 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56426fab8750 0x56426fb1ed80 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f95659f8740 1 --2- 172.21.15.94:0/3451354029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56426faafb70 0x56426fabed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f95659f8740 1 --2- 172.21.15.94:0/3451354029 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x56426fb20880 0x56426fb22c70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f953bfff640 0 client.18306 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:08:53.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f95659f8740 1 --2- 172.21.15.94:0/3451354029 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56426faaf250 0x56426fabe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f95659f8740 1 -- 172.21.15.94:0/3451354029 >> 172.21.15.94:0/3451354029 conn(0x56426faa9980 msgr2=0x56426faa82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f95659f8740 1 -- 172.21.15.94:0/3451354029 shutdown_connections 2024-02-17T12:08:53.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f95659f8740 1 -- 172.21.15.94:0/3451354029 wait complete. 2024-02-17T12:08:53.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.548+0000 7f95659f8740 20 client.18306 trim_cache size 0 max 16384 2024-02-17T12:08:53.559 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:08:53.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.549+0000 7f9562e23640 1 -- 172.21.15.94:0/3636512 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f954c0a2230 msgr2=0x7f954c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.549+0000 7f9562e23640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f954c0a2230 0x7f954c0a46f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9548014930 tx=0x7f9548005f90 comp rx=0 tx=0).stop 2024-02-17T12:08:53.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.549+0000 7f9562e23640 1 -- 172.21.15.94:0/3636512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955c105020 msgr2=0x7f955c1a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.549+0000 7f9562e23640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955c105020 0x7f955c1a3690 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f955400db20 tx=0x7f955400b260 comp rx=0 tx=0).stop 2024-02-17T12:08:53.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.549+0000 7f9562e23640 1 -- 172.21.15.94:0/3636512 shutdown_connections 2024-02-17T12:08:53.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.549+0000 7f9562e23640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f954c0a2230 0x7f954c0a46f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.549+0000 7f9562e23640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f955c105020 0x7f955c1a3690 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.549+0000 7f9562e23640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f955c101550 0x7f955c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.549+0000 7f9562e23640 1 --2- 172.21.15.94:0/3636512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f955c100b90 0x7f955c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.549+0000 7f9562e23640 1 -- 172.21.15.94:0/3636512 >> 172.21.15.94:0/3636512 conn(0x7f955c0752e0 msgr2=0x7f955c0fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.549+0000 7f9562e23640 1 -- 172.21.15.94:0/3636512 shutdown_connections 2024-02-17T12:08:53.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.549+0000 7f9562e23640 1 -- 172.21.15.94:0/3636512 wait complete. 2024-02-17T12:08:53.562 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:08:53.562 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:08:53.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.715+0000 7feca32d4640 1 -- 172.21.15.94:0/1026946117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec9c10d730 msgr2=0x7fec9c10db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.715+0000 7feca32d4640 1 --2- 172.21.15.94:0/1026946117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec9c10d730 0x7fec9c10db10 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fec90009cd0 tx=0x7fec9002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.715+0000 7feca32d4640 1 -- 172.21.15.94:0/1026946117 shutdown_connections 2024-02-17T12:08:53.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.715+0000 7feca32d4640 1 --2- 172.21.15.94:0/1026946117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec9c1080e0 0x7fec9c1145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.715+0000 7feca32d4640 1 --2- 172.21.15.94:0/1026946117 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec9c107720 0x7fec9c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.715+0000 7feca32d4640 1 --2- 172.21.15.94:0/1026946117 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec9c10d730 0x7fec9c10db10 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.715+0000 7feca32d4640 1 -- 172.21.15.94:0/1026946117 >> 172.21.15.94:0/1026946117 conn(0x7fec9c0756c0 msgr2=0x7fec9c077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.715+0000 7feca32d4640 1 -- 172.21.15.94:0/1026946117 shutdown_connections 2024-02-17T12:08:53.717 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.715+0000 7feca32d4640 1 -- 172.21.15.94:0/1026946117 wait complete. 2024-02-17T12:08:53.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca32d4640 1 Processor -- start 2024-02-17T12:08:53.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca32d4640 1 -- start start 2024-02-17T12:08:53.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca32d4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec9c107720 0x7fec9c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca32d4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec9c1080e0 0x7fec9c19e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca32d4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec9c10d730 0x7fec9c1a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca32d4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c1a2ca0 con 0x7fec9c1080e0 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca32d4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c1a2e10 con 0x7fec9c10d730 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca32d4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9c1a2fd0 con 0x7fec9c107720 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca086f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec9c1080e0 0x7fec9c19e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca086f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec9c1080e0 0x7fec9c19e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47268/0 (socket says 172.21.15.94:47268) 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca086f640 1 -- 172.21.15.94:0/3723169184 learned_addr learned my addr 172.21.15.94:0/3723169184 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca1070640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec9c107720 0x7fec9c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca1871640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec9c10d730 0x7fec9c1a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca086f640 1 -- 172.21.15.94:0/3723169184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec9c107720 msgr2=0x7fec9c19dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca086f640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec9c107720 0x7fec9c19dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.716+0000 7feca086f640 1 -- 172.21.15.94:0/3723169184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec9c10d730 msgr2=0x7fec9c1a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.717+0000 7feca086f640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec9c10d730 0x7fec9c1a26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.720 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.717+0000 7feca086f640 1 -- 172.21.15.94:0/3723169184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec90009900 con 0x7fec9c1080e0 2024-02-17T12:08:53.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.717+0000 7feca1070640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec9c107720 0x7fec9c19dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:53.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.717+0000 7feca1871640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec9c10d730 0x7fec9c1a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:53.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.717+0000 7feca086f640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec9c1080e0 0x7fec9c19e320 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fec8800e9f0 tx=0x7fec8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.717+0000 7fec9a7fc640 1 -- 172.21.15.94:0/3723169184 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88009700 con 0x7fec9c1080e0 2024-02-17T12:08:53.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.717+0000 7fec9a7fc640 1 -- 172.21.15.94:0/3723169184 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec8800cd80 con 0x7fec9c1080e0 2024-02-17T12:08:53.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.717+0000 7fec9a7fc640 1 -- 172.21.15.94:0/3723169184 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec88018920 con 0x7fec9c1080e0 2024-02-17T12:08:53.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.717+0000 7feca32d4640 1 -- 172.21.15.94:0/3723169184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9c1a32b0 con 0x7fec9c1080e0 2024-02-17T12:08:53.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.717+0000 7feca32d4640 1 -- 172.21.15.94:0/3723169184 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c1aab30 con 0x7fec9c1080e0 2024-02-17T12:08:53.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.720+0000 7fec9a7fc640 1 -- 172.21.15.94:0/3723169184 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fec88018a80 con 0x7fec9c1080e0 2024-02-17T12:08:53.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.720+0000 7feca5f86740 1 Processor -- start 2024-02-17T12:08:53.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.720+0000 7feca5f86740 1 -- start start 2024-02-17T12:08:53.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.720+0000 7feca5f86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 0x564a06073630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.720+0000 7feca5f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a06073b70 0x564a0607c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca5f86740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a0607c750 0x564a0607eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca5f86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0602e960 con 0x564a06073250 2024-02-17T12:08:53.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca5f86740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a05fe2310 con 0x564a0607c750 2024-02-17T12:08:53.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca5f86740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a05fdf840 con 0x564a06073b70 2024-02-17T12:08:53.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca1871640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 0x564a06073630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca1070640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a06073b70 0x564a0607c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca1871640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 0x564a06073630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47274/0 (socket says 172.21.15.94:47274) 2024-02-17T12:08:53.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca1871640 1 -- 172.21.15.94:0/1043383674 learned_addr learned my addr 172.21.15.94:0/1043383674 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca086f640 1 --2- 172.21.15.94:0/1043383674 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a0607c750 0x564a0607eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca1871640 1 -- 172.21.15.94:0/1043383674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a06073b70 msgr2=0x564a0607c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca1871640 1 --2- 172.21.15.94:0/1043383674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a06073b70 0x564a0607c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca1871640 1 -- 172.21.15.94:0/1043383674 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a0607c750 msgr2=0x564a0607eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca1871640 1 --2- 172.21.15.94:0/1043383674 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a0607c750 0x564a0607eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca1871640 1 -- 172.21.15.94:0/1043383674 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a05f9d060 con 0x564a06073250 2024-02-17T12:08:53.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7fec9a7fc640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec8c0a2380 0x7fec8c0a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca1070640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec8c0a2380 0x7fec8c0a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7fec9a7fc640 1 -- 172.21.15.94:0/3723169184 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fec88014070 con 0x7fec9c1080e0 2024-02-17T12:08:53.724 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7feca1871640 1 --2- 172.21.15.94:0/1043383674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 0x564a06073630 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fec94011440 tx=0x7fec94011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c8e2786df6cff5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.721+0000 7fec877fe640 1 -- 172.21.15.94:0/1043383674 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94004020 con 0x564a06073250 2024-02-17T12:08:53.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.722+0000 7fec877fe640 1 -- 172.21.15.94:0/1043383674 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec94004180 con 0x564a06073250 2024-02-17T12:08:53.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.722+0000 7fec877fe640 1 -- 172.21.15.94:0/1043383674 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9400f820 con 0x564a06073250 2024-02-17T12:08:53.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.722+0000 7feca5f86740 1 -- 172.21.15.94:0/1043383674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 msgr2=0x564a06073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.722+0000 7feca1070640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec8c0a2380 0x7fec8c0a4840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fec90004af0 tx=0x7fec90005c50 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.722+0000 7feca5f86740 1 --2- 172.21.15.94:0/1043383674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 0x564a06073630 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fec94011440 tx=0x7fec94011900 comp rx=0 tx=0).stop 2024-02-17T12:08:53.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.723+0000 7feca5f86740 1 -- 172.21.15.94:0/1043383674 shutdown_connections 2024-02-17T12:08:53.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.723+0000 7feca5f86740 1 --2- 172.21.15.94:0/1043383674 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a0607c750 0x564a0607eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.723+0000 7feca5f86740 1 --2- 172.21.15.94:0/1043383674 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a06073b70 0x564a0607c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.723+0000 7feca5f86740 1 --2- 172.21.15.94:0/1043383674 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 0x564a06073630 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.723+0000 7feca5f86740 1 -- 172.21.15.94:0/1043383674 >> 172.21.15.94:0/1043383674 conn(0x564a0606d980 msgr2=0x564a0606dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.723+0000 7feca5f86740 1 -- 172.21.15.94:0/1043383674 shutdown_connections 2024-02-17T12:08:53.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.723+0000 7feca5f86740 1 -- 172.21.15.94:0/1043383674 wait complete. 2024-02-17T12:08:53.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.723+0000 7feca5f86740 1 -- start start 2024-02-17T12:08:53.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.723+0000 7feca5f86740 1 Processor -- start 2024-02-17T12:08:53.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.724+0000 7feca5f86740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073250 0x564a06079710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.724+0000 7feca5f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a06073b70 0x564a06079c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.724+0000 7feca1070640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073250 0x564a06079710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.724+0000 7feca1070640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073250 0x564a06079710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:50186/0 (socket says 172.21.15.94:50186) 2024-02-17T12:08:53.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.724+0000 7feca1070640 1 -- 172.21.15.94:0/3981754601 learned_addr learned my addr 172.21.15.94:0/3981754601 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.728 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.724+0000 7feca086f640 1 --2- 172.21.15.94:0/3981754601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a06073b70 0x564a06079c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.724+0000 7feca5f86740 1 --2- 172.21.15.94:0/3981754601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0607c750 0x564a0607a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.724+0000 7feca5f86740 1 -- 172.21.15.94:0/3981754601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a05fdb020 con 0x564a0607c750 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.724+0000 7feca5f86740 1 -- 172.21.15.94:0/3981754601 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a05cae430 con 0x564a06073250 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.724+0000 7feca5f86740 1 -- 172.21.15.94:0/3981754601 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a05c6ae00 con 0x564a06073b70 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.725+0000 7feca086f640 1 -- 172.21.15.94:0/3981754601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073250 msgr2=0x564a06079710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.725+0000 7feca086f640 1 --2- 172.21.15.94:0/3981754601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073250 0x564a06079710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.725+0000 7feca086f640 1 -- 172.21.15.94:0/3981754601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0607c750 msgr2=0x564a0607a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.725+0000 7feca086f640 1 --2- 172.21.15.94:0/3981754601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0607c750 0x564a0607a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.725+0000 7feca086f640 1 -- 172.21.15.94:0/3981754601 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec94009cc0 con 0x564a06073b70 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.724+0000 7feca5f86740 10 client.0 _ll_register_callbacks cb 0x564a05f82cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.725+0000 7feca086f640 1 --2- 172.21.15.94:0/3981754601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a06073b70 0x564a06079c50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fec880cba30 tx=0x7fec8809ac40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad0b2e133dd1c92e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.725+0000 7feca5f86740 1 -- 172.21.15.94:0/3981754601 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a05cda7e0 con 0x564a06073b70 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.725+0000 7fec84ff9640 10 client.27766 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:08:53.729 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.725+0000 7fec84ff9640 1 -- 172.21.15.94:0/3981754601 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8809ee50 con 0x564a06073b70 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.725+0000 7fec84ff9640 1 -- 172.21.15.94:0/3981754601 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fec880a34a0 con 0x564a06073b70 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.726+0000 7fec84ff9640 1 -- 172.21.15.94:0/3981754601 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec8809a200 con 0x564a06073b70 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.726+0000 7feca5f86740 10 client.27766 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.726+0000 7feca5f86740 1 -- 172.21.15.94:0/3981754601 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x564a05fd96a0 con 0x564a06073b70 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.726+0000 7fec84ff9640 1 -- 172.21.15.94:0/3981754601 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fec8809a360 con 0x564a06073b70 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.726+0000 7feca5f86740 10 client.27766 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.726+0000 7feca5f86740 10 client.27766 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.726+0000 7feca5f86740 20 client.27766 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.726+0000 7feca5f86740 1 --2- 172.21.15.94:0/3981754601 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a0608ae60 0x564a0608d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.726+0000 7feca5f86740 4 client.27766 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.726+0000 7feca5f86740 1 -- 172.21.15.94:0/3981754601 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a05cda7e0 con 0x564a0608ae60 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.726+0000 7feca1070640 1 --2- 172.21.15.94:0/3981754601 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a0608ae60 0x564a0608d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.727+0000 7fec84ff9640 10 client.27766 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:08:53.730 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.727+0000 7feca1070640 1 --2- 172.21.15.94:0/3981754601 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a0608ae60 0x564a0608d320 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f9f361b1cb6188a server_cookie=d9cb85461a0b09c6 in_seq=0 out_seq=0 2024-02-17T12:08:53.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.727+0000 7fec84ff9640 1 -- 172.21.15.94:0/3981754601 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a05cda7e0 con 0x564a0608ae60 2024-02-17T12:08:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.727+0000 7fec84ff9640 10 client.27766 handle_command_reply: tid=0 2024-02-17T12:08:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.727+0000 7feca5f86740 1 client.27766 shutdown 2024-02-17T12:08:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.728+0000 7feca5f86740 1 -- 172.21.15.94:0/3981754601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a06073b70 msgr2=0x564a06079c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.728+0000 7feca5f86740 1 --2- 172.21.15.94:0/3981754601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a06073b70 0x564a06079c50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fec880cba30 tx=0x7fec8809ac40 comp rx=0 tx=0).stop 2024-02-17T12:08:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.728+0000 7feca5f86740 1 -- 172.21.15.94:0/3981754601 shutdown_connections 2024-02-17T12:08:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.728+0000 7feca5f86740 1 --2- 172.21.15.94:0/3981754601 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a0608ae60 0x564a0608d320 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.728+0000 7feca5f86740 1 --2- 172.21.15.94:0/3981754601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a0607c750 0x564a0607a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.728+0000 7feca5f86740 1 --2- 172.21.15.94:0/3981754601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a06073b70 0x564a06079c50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.728+0000 7feca5f86740 1 --2- 172.21.15.94:0/3981754601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073250 0x564a06079710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.728+0000 7feca5f86740 1 -- 172.21.15.94:0/3981754601 >> 172.21.15.94:0/3981754601 conn(0x564a0606d980 msgr2=0x564a0606f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.728+0000 7fec84ff9640 0 client.27766 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:08:53.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.728+0000 7feca5f86740 1 -- 172.21.15.94:0/3981754601 shutdown_connections 2024-02-17T12:08:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.728+0000 7feca5f86740 1 -- 172.21.15.94:0/3981754601 wait complete. 2024-02-17T12:08:53.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.728+0000 7feca5f86740 20 client.27766 trim_cache size 0 max 16384 2024-02-17T12:08:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.743+0000 7feca5f86740 1 Processor -- start 2024-02-17T12:08:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.743+0000 7feca5f86740 1 -- start start 2024-02-17T12:08:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.743+0000 7feca5f86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 0x564a0607be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.743+0000 7feca5f86740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073b70 0x564a06074a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.743+0000 7feca5f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a0607c750 0x564a06074f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.743+0000 7feca1070640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 0x564a0607be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.743+0000 7feca1070640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 0x564a0607be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47300/0 (socket says 172.21.15.94:47300) 2024-02-17T12:08:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.743+0000 7feca1070640 1 -- 172.21.15.94:0/4034053259 learned_addr learned my addr 172.21.15.94:0/4034053259 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.743+0000 7feca5f86740 1 -- 172.21.15.94:0/4034053259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a05f60130 con 0x564a06073250 2024-02-17T12:08:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.743+0000 7feca5f86740 1 -- 172.21.15.94:0/4034053259 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0609a5f0 con 0x564a06073b70 2024-02-17T12:08:53.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.743+0000 7feca5f86740 1 -- 172.21.15.94:0/4034053259 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a0606fc90 con 0x564a0607c750 2024-02-17T12:08:53.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.743+0000 7feca1871640 1 --2- 172.21.15.94:0/4034053259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a0607c750 0x564a06074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.743+0000 7feca086f640 1 --2- 172.21.15.94:0/4034053259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073b70 0x564a06074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.744+0000 7feca1070640 1 -- 172.21.15.94:0/4034053259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a0607c750 msgr2=0x564a06074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.744+0000 7feca1070640 1 --2- 172.21.15.94:0/4034053259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a0607c750 0x564a06074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.744+0000 7feca1070640 1 -- 172.21.15.94:0/4034053259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073b70 msgr2=0x564a06074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.744+0000 7feca1070640 1 --2- 172.21.15.94:0/4034053259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073b70 0x564a06074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.744+0000 7feca1070640 1 -- 172.21.15.94:0/4034053259 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a05f75c20 con 0x564a06073250 2024-02-17T12:08:53.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.744+0000 7feca1070640 1 --2- 172.21.15.94:0/4034053259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 0x564a0607be40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fec9002c700 tx=0x7fec90042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3a3fc30d7991b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.744+0000 7fec86ffd640 1 -- 172.21.15.94:0/4034053259 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec90052070 con 0x564a06073250 2024-02-17T12:08:53.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.744+0000 7fec86ffd640 1 -- 172.21.15.94:0/4034053259 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec90044070 con 0x564a06073250 2024-02-17T12:08:53.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.744+0000 7fec86ffd640 1 -- 172.21.15.94:0/4034053259 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec90003e70 con 0x564a06073250 2024-02-17T12:08:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.745+0000 7feca5f86740 1 -- 172.21.15.94:0/4034053259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 msgr2=0x564a0607be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.745+0000 7feca5f86740 1 --2- 172.21.15.94:0/4034053259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 0x564a0607be40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fec9002c700 tx=0x7fec90042000 comp rx=0 tx=0).stop 2024-02-17T12:08:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.746+0000 7feca5f86740 1 -- 172.21.15.94:0/4034053259 shutdown_connections 2024-02-17T12:08:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.746+0000 7feca5f86740 1 --2- 172.21.15.94:0/4034053259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a0607c750 0x564a06074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.746+0000 7feca5f86740 1 --2- 172.21.15.94:0/4034053259 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073b70 0x564a06074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.746+0000 7feca5f86740 1 --2- 172.21.15.94:0/4034053259 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073250 0x564a0607be40 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.746+0000 7feca5f86740 1 -- 172.21.15.94:0/4034053259 >> 172.21.15.94:0/4034053259 conn(0x564a0606d980 msgr2=0x564a06091cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.746+0000 7feca5f86740 1 -- 172.21.15.94:0/4034053259 shutdown_connections 2024-02-17T12:08:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.746+0000 7feca5f86740 1 -- 172.21.15.94:0/4034053259 wait complete. 2024-02-17T12:08:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.746+0000 7feca5f86740 1 -- start start 2024-02-17T12:08:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.746+0000 7feca5f86740 1 Processor -- start 2024-02-17T12:08:53.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.747+0000 7feca5f86740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073250 0x564a06083af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.747+0000 7feca5f86740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073b70 0x564a06081e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.747+0000 7feca5f86740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a0607c750 0x564a060825b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.747+0000 7feca5f86740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a06082cb0 con 0x564a06073b70 2024-02-17T12:08:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.747+0000 7feca5f86740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a060e33f0 con 0x564a06073250 2024-02-17T12:08:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.747+0000 7feca1070640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073250 0x564a06083af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.747+0000 7feca1070640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073250 0x564a06083af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:50208/0 (socket says 172.21.15.94:50208) 2024-02-17T12:08:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.747+0000 7feca1070640 1 -- 172.21.15.94:0/2543778123 learned_addr learned my addr 172.21.15.94:0/2543778123 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.747+0000 7feca1871640 1 --2- 172.21.15.94:0/2543778123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a0607c750 0x564a060825b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.747+0000 7feca5f86740 1 -- 172.21.15.94:0/2543778123 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a060e3530 con 0x564a0607c750 2024-02-17T12:08:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.748+0000 7feca5f86740 10 client.0 _ll_register_callbacks cb 0x7fec94009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.748+0000 7feca086f640 1 --2- 172.21.15.94:0/2543778123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073b70 0x564a06081e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.748+0000 7feca1871640 1 -- 172.21.15.94:0/2543778123 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073250 msgr2=0x564a06083af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.748+0000 7feca1871640 1 --2- 172.21.15.94:0/2543778123 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073250 0x564a06083af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.748+0000 7feca1871640 1 -- 172.21.15.94:0/2543778123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073b70 msgr2=0x564a06081e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.748+0000 7feca1871640 1 --2- 172.21.15.94:0/2543778123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073b70 0x564a06081e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.748+0000 7feca1871640 1 -- 172.21.15.94:0/2543778123 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c114ef0 con 0x564a0607c750 2024-02-17T12:08:53.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.748+0000 7feca1871640 1 --2- 172.21.15.94:0/2543778123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a0607c750 0x564a060825b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fec94009250 tx=0x7fec94005660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45f832663b8a11da server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.748+0000 7fec84ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:08:53.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.748+0000 7fec84ff9640 1 -- 172.21.15.94:0/2543778123 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9400f750 con 0x564a0607c750 2024-02-17T12:08:53.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.748+0000 7fec84ff9640 1 -- 172.21.15.94:0/2543778123 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fec9400f8b0 con 0x564a0607c750 2024-02-17T12:08:53.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.748+0000 7feca5f86740 1 -- 172.21.15.94:0/2543778123 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec94004180 con 0x564a0607c750 2024-02-17T12:08:53.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.749+0000 7fec84ff9640 1 -- 172.21.15.94:0/2543778123 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec9400fa70 con 0x564a0607c750 2024-02-17T12:08:53.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.749+0000 7feca5f86740 10 client.27772 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:53.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.749+0000 7feca5f86740 1 -- 172.21.15.94:0/2543778123 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564a05f9b8c0 con 0x564a0607c750 2024-02-17T12:08:53.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.749+0000 7fec84ff9640 1 -- 172.21.15.94:0/2543778123 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec94036080 con 0x564a0607c750 2024-02-17T12:08:53.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.749+0000 7fec84ff9640 1 -- 172.21.15.94:0/2543778123 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fec94028020 con 0x564a0607c750 2024-02-17T12:08:53.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.749+0000 7feca5f86740 10 client.27772 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:53.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.749+0000 7feca5f86740 10 client.27772 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:08:53.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.750+0000 7feca5f86740 20 client.27772 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:53.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.750+0000 7feca5f86740 1 --2- 172.21.15.94:0/2543778123 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a060e4af0 0x564a060e6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.750+0000 7feca5f86740 4 client.27772 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:08:53.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.750+0000 7feca5f86740 1 -- 172.21.15.94:0/2543778123 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564a05d17720 con 0x564a060e4af0 2024-02-17T12:08:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.750+0000 7feca1070640 1 --2- 172.21.15.94:0/2543778123 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a060e4af0 0x564a060e6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.750+0000 7fec84ff9640 10 client.27772 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:08:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.750+0000 7feca1070640 1 --2- 172.21.15.94:0/2543778123 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a060e4af0 0x564a060e6ee0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72f40eb81e162758 server_cookie=d89c6205a360d66a in_seq=0 out_seq=0 2024-02-17T12:08:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.750+0000 7fec84ff9640 1 -- 172.21.15.94:0/2543778123 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564a05d17720 con 0x564a060e4af0 2024-02-17T12:08:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.750+0000 7fec84ff9640 10 client.27772 handle_command_reply: tid=0 2024-02-17T12:08:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.750+0000 7feca5f86740 1 client.27772 shutdown 2024-02-17T12:08:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca5f86740 1 -- 172.21.15.94:0/2543778123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a0607c750 msgr2=0x564a060825b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca5f86740 1 --2- 172.21.15.94:0/2543778123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a0607c750 0x564a060825b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fec94009250 tx=0x7fec94005660 comp rx=0 tx=0).stop 2024-02-17T12:08:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca5f86740 1 -- 172.21.15.94:0/2543778123 shutdown_connections 2024-02-17T12:08:53.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca5f86740 1 --2- 172.21.15.94:0/2543778123 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x564a060e4af0 0x564a060e6ee0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca5f86740 1 --2- 172.21.15.94:0/2543778123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564a0607c750 0x564a060825b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7fec84ff9640 0 client.27772 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:08:53.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca5f86740 1 --2- 172.21.15.94:0/2543778123 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564a06073b70 0x564a06081e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca5f86740 1 --2- 172.21.15.94:0/2543778123 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564a06073250 0x564a06083af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca5f86740 1 -- 172.21.15.94:0/2543778123 >> 172.21.15.94:0/2543778123 conn(0x564a0606d980 msgr2=0x564a06074a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca5f86740 1 -- 172.21.15.94:0/2543778123 shutdown_connections 2024-02-17T12:08:53.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca5f86740 1 -- 172.21.15.94:0/2543778123 wait complete. 2024-02-17T12:08:53.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca5f86740 20 client.27772 trim_cache size 0 max 16384 2024-02-17T12:08:53.759 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:08:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca32d4640 1 -- 172.21.15.94:0/3723169184 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec8c0a2380 msgr2=0x7fec8c0a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca32d4640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec8c0a2380 0x7fec8c0a4840 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fec90004af0 tx=0x7fec90005c50 comp rx=0 tx=0).stop 2024-02-17T12:08:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca32d4640 1 -- 172.21.15.94:0/3723169184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec9c1080e0 msgr2=0x7fec9c19e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.751+0000 7feca32d4640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec9c1080e0 0x7fec9c19e320 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fec8800e9f0 tx=0x7fec8800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.752+0000 7feca32d4640 1 -- 172.21.15.94:0/3723169184 shutdown_connections 2024-02-17T12:08:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.752+0000 7feca32d4640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fec8c0a2380 0x7fec8c0a4840 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.752+0000 7feca32d4640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fec9c10d730 0x7fec9c1a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.752+0000 7feca32d4640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fec9c1080e0 0x7fec9c19e320 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.752+0000 7feca32d4640 1 --2- 172.21.15.94:0/3723169184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fec9c107720 0x7fec9c19dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.752+0000 7feca32d4640 1 -- 172.21.15.94:0/3723169184 >> 172.21.15.94:0/3723169184 conn(0x7fec9c0756c0 msgr2=0x7fec9c075e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.752+0000 7feca32d4640 1 -- 172.21.15.94:0/3723169184 shutdown_connections 2024-02-17T12:08:53.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.752+0000 7feca32d4640 1 -- 172.21.15.94:0/3723169184 wait complete. 2024-02-17T12:08:53.770 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:08:53.770 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:08:53.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.915+0000 7fe2589a1640 1 -- 172.21.15.94:0/2020707044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2541009e0 msgr2=0x7fe25410cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.915+0000 7fe2589a1640 1 --2- 172.21.15.94:0/2020707044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2541009e0 0x7fe25410cea0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe24c009ca0 tx=0x7fe24c02c180 comp rx=0 tx=0).stop 2024-02-17T12:08:53.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.915+0000 7fe2589a1640 1 -- 172.21.15.94:0/2020707044 shutdown_connections 2024-02-17T12:08:53.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.915+0000 7fe2589a1640 1 --2- 172.21.15.94:0/2020707044 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2541009e0 0x7fe25410cea0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.915+0000 7fe2589a1640 1 --2- 172.21.15.94:0/2020707044 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe254100040 0x7fe2541004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.915+0000 7fe2589a1640 1 --2- 172.21.15.94:0/2020707044 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe254106100 0x7fe2541064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.915+0000 7fe2589a1640 1 -- 172.21.15.94:0/2020707044 >> 172.21.15.94:0/2020707044 conn(0x7fe254077110 msgr2=0x7fe254077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.919 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.915+0000 7fe2589a1640 1 -- 172.21.15.94:0/2020707044 shutdown_connections 2024-02-17T12:08:53.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.915+0000 7fe2589a1640 1 -- 172.21.15.94:0/2020707044 wait complete. 2024-02-17T12:08:53.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.916+0000 7fe2589a1640 1 Processor -- start 2024-02-17T12:08:53.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.916+0000 7fe2589a1640 1 -- start start 2024-02-17T12:08:53.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.916+0000 7fe2589a1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe254100040 0x7fe254101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.916+0000 7fe2589a1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2541009e0 0x7fe254102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.916+0000 7fe2589a1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe254106100 0x7fe254102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.916+0000 7fe2589a1640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2541a5a00 con 0x7fe2541009e0 2024-02-17T12:08:53.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.916+0000 7fe2589a1640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2541a5b70 con 0x7fe254106100 2024-02-17T12:08:53.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.916+0000 7fe2589a1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2541a5d30 con 0x7fe254100040 2024-02-17T12:08:53.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.916+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2541009e0 0x7fe254102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.916+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2541009e0 0x7fe254102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47324/0 (socket says 172.21.15.94:47324) 2024-02-17T12:08:53.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.916+0000 7fe251d9b640 1 -- 172.21.15.94:0/3242187608 learned_addr learned my addr 172.21.15.94:0/3242187608 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.916+0000 7fe252d9d640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe254106100 0x7fe254102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.916+0000 7fe25259c640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe254100040 0x7fe254101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.917+0000 7fe251d9b640 1 -- 172.21.15.94:0/3242187608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe254100040 msgr2=0x7fe254101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.917+0000 7fe251d9b640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe254100040 0x7fe254101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.917+0000 7fe251d9b640 1 -- 172.21.15.94:0/3242187608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe254106100 msgr2=0x7fe254102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.917+0000 7fe251d9b640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe254106100 0x7fe254102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.917+0000 7fe251d9b640 1 -- 172.21.15.94:0/3242187608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c009900 con 0x7fe2541009e0 2024-02-17T12:08:53.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.917+0000 7fe251d9b640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2541009e0 0x7fe254102230 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe24800e9f0 tx=0x7fe24800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.917+0000 7fe2437fe640 1 -- 172.21.15.94:0/3242187608 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248009700 con 0x7fe2541009e0 2024-02-17T12:08:53.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.917+0000 7fe2437fe640 1 -- 172.21.15.94:0/3242187608 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe24800cd80 con 0x7fe2541009e0 2024-02-17T12:08:53.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.917+0000 7fe2437fe640 1 -- 172.21.15.94:0/3242187608 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe248018920 con 0x7fe2541009e0 2024-02-17T12:08:53.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.917+0000 7fe2589a1640 1 -- 172.21.15.94:0/3242187608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2541a5fb0 con 0x7fe2541009e0 2024-02-17T12:08:53.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.917+0000 7fe2589a1640 1 -- 172.21.15.94:0/3242187608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2541a6440 con 0x7fe2541009e0 2024-02-17T12:08:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.919+0000 7fe2437fe640 1 -- 172.21.15.94:0/3242187608 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe248018a80 con 0x7fe2541009e0 2024-02-17T12:08:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.920+0000 7fe2437fe640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe23c0a22e0 0x7fe23c0a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.920+0000 7fe2437fe640 1 -- 172.21.15.94:0/3242187608 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe248014070 con 0x7fe2541009e0 2024-02-17T12:08:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.920+0000 7fe25259c640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe23c0a22e0 0x7fe23c0a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.920+0000 7fe25b578740 1 Processor -- start 2024-02-17T12:08:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.920+0000 7fe25b578740 1 -- start start 2024-02-17T12:08:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.920+0000 7fe25259c640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe23c0a22e0 0x7fe23c0a47a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe244005fd0 tx=0x7fe24400b510 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.920+0000 7fe25b578740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324250 0x56010c324630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.920+0000 7fe25b578740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324b70 0x56010c32d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.920+0000 7fe25b578740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c32d750 0x56010c32fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe25b578740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c2df960 con 0x56010c32d750 2024-02-17T12:08:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe25b578740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c293310 con 0x56010c324b70 2024-02-17T12:08:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe25b578740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c290840 con 0x56010c324250 2024-02-17T12:08:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe252d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c32d750 0x56010c32fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe252d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c32d750 0x56010c32fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47332/0 (socket says 172.21.15.94:47332) 2024-02-17T12:08:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe252d9d640 1 -- 172.21.15.94:0/808061289 learned_addr learned my addr 172.21.15.94:0/808061289 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe251d9b640 1 --2- 172.21.15.94:0/808061289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324b70 0x56010c32d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe25259c640 1 --2- 172.21.15.94:0/808061289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324250 0x56010c324630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe252d9d640 1 -- 172.21.15.94:0/808061289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324250 msgr2=0x56010c324630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe252d9d640 1 --2- 172.21.15.94:0/808061289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324250 0x56010c324630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe252d9d640 1 -- 172.21.15.94:0/808061289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324b70 msgr2=0x56010c32d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe252d9d640 1 --2- 172.21.15.94:0/808061289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324b70 0x56010c32d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe252d9d640 1 -- 172.21.15.94:0/808061289 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010c24e060 con 0x56010c32d750 2024-02-17T12:08:53.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe251d9b640 1 --2- 172.21.15.94:0/808061289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324b70 0x56010c32d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:08:53.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe252d9d640 1 --2- 172.21.15.94:0/808061289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c32d750 0x56010c32fb40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe24c02c790 tx=0x7fe24c005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49dbffa76d6bef47 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe240ff9640 1 -- 172.21.15.94:0/808061289 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c034440 con 0x56010c32d750 2024-02-17T12:08:53.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe240ff9640 1 -- 172.21.15.94:0/808061289 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe24c003970 con 0x56010c32d750 2024-02-17T12:08:53.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.921+0000 7fe240ff9640 1 -- 172.21.15.94:0/808061289 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c032dd0 con 0x56010c32d750 2024-02-17T12:08:53.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.922+0000 7fe25b578740 1 -- 172.21.15.94:0/808061289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c32d750 msgr2=0x56010c32fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.922+0000 7fe25b578740 1 --2- 172.21.15.94:0/808061289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c32d750 0x56010c32fb40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe24c02c790 tx=0x7fe24c005f80 comp rx=0 tx=0).stop 2024-02-17T12:08:53.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.922+0000 7fe25b578740 1 -- 172.21.15.94:0/808061289 shutdown_connections 2024-02-17T12:08:53.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.922+0000 7fe25b578740 1 --2- 172.21.15.94:0/808061289 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c32d750 0x56010c32fb40 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.922+0000 7fe25b578740 1 --2- 172.21.15.94:0/808061289 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324b70 0x56010c32d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.922+0000 7fe25b578740 1 --2- 172.21.15.94:0/808061289 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324250 0x56010c324630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.922+0000 7fe25b578740 1 -- 172.21.15.94:0/808061289 >> 172.21.15.94:0/808061289 conn(0x56010c31e980 msgr2=0x56010c31ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.922+0000 7fe25b578740 1 -- 172.21.15.94:0/808061289 shutdown_connections 2024-02-17T12:08:53.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.922+0000 7fe25b578740 1 -- 172.21.15.94:0/808061289 wait complete. 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25b578740 1 -- start start 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25b578740 1 Processor -- start 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25b578740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324250 0x56010c32a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25b578740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324b70 0x56010c32ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25b578740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c32d750 0x56010c32b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25b578740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c28c020 con 0x56010c324250 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25b578740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010bf5f430 con 0x56010c32d750 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25b578740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010bf1be00 con 0x56010c324b70 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324250 0x56010c32a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324250 0x56010c32a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47346/0 (socket says 172.21.15.94:47346) 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25259c640 1 -- 172.21.15.94:0/3973171708 learned_addr learned my addr 172.21.15.94:0/3973171708 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25b578740 10 client.0 _ll_register_callbacks cb 0x56010c233cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe252d9d640 1 --2- 172.21.15.94:0/3973171708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c32d750 0x56010c32b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe251d9b640 1 --2- 172.21.15.94:0/3973171708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324b70 0x56010c32ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25259c640 1 -- 172.21.15.94:0/3973171708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324b70 msgr2=0x56010c32ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25259c640 1 --2- 172.21.15.94:0/3973171708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324b70 0x56010c32ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25259c640 1 -- 172.21.15.94:0/3973171708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c32d750 msgr2=0x56010c32b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.923+0000 7fe25259c640 1 --2- 172.21.15.94:0/3973171708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c32d750 0x56010c32b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe25259c640 1 -- 172.21.15.94:0/3973171708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25410d7a0 con 0x56010c324250 2024-02-17T12:08:53.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe25259c640 1 --2- 172.21.15.94:0/3973171708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324250 0x56010c32a730 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe244011470 tx=0x7fe2440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3111381f7eb30d3c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:08:53.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe239ffb640 1 -- 172.21.15.94:0/3973171708 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24401f840 con 0x56010c324250 2024-02-17T12:08:53.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe239ffb640 1 -- 172.21.15.94:0/3973171708 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe24401f9a0 con 0x56010c324250 2024-02-17T12:08:53.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe25b578740 1 -- 172.21.15.94:0/3973171708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56010bf8b7e0 con 0x56010c324250 2024-02-17T12:08:53.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe239ffb640 1 -- 172.21.15.94:0/3973171708 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe24401fba0 con 0x56010c324250 2024-02-17T12:08:53.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe25b578740 10 client.18333 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:53.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe25b578740 1 -- 172.21.15.94:0/3973171708 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56010c28a6a0 con 0x56010c324250 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe239ffb640 1 -- 172.21.15.94:0/3973171708 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56010bf8b7e0 con 0x56010c324250 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe239ffb640 1 -- 172.21.15.94:0/3973171708 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe244025020 con 0x56010c324250 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe25b578740 10 client.18333 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe25b578740 10 client.18333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe25b578740 20 client.18333 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe25b578740 1 --2- 172.21.15.94:0/3973171708 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56010c33daf0 0x56010c33ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe25b578740 4 client.18333 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.924+0000 7fe25b578740 1 -- 172.21.15.94:0/3973171708 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56010bfc8720 con 0x56010c33daf0 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.925+0000 7fe251d9b640 1 --2- 172.21.15.94:0/3973171708 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56010c33daf0 0x56010c33ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.925+0000 7fe239ffb640 10 client.18333 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.925+0000 7fe251d9b640 1 --2- 172.21.15.94:0/3973171708 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56010c33daf0 0x56010c33ff20 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c14ad844ac50e80f server_cookie=11368844617ec49c in_seq=0 out_seq=0 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe239ffb640 1 -- 172.21.15.94:0/3973171708 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56010bfc8720 con 0x56010c33daf0 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe239ffb640 10 client.18333 handle_command_reply: tid=0 2024-02-17T12:08:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe25b578740 1 client.18333 shutdown 2024-02-17T12:08:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe25b578740 1 -- 172.21.15.94:0/3973171708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324250 msgr2=0x56010c32a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe25b578740 1 --2- 172.21.15.94:0/3973171708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324250 0x56010c32a730 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe244011470 tx=0x7fe2440046c0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe25b578740 1 -- 172.21.15.94:0/3973171708 shutdown_connections 2024-02-17T12:08:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe25b578740 1 --2- 172.21.15.94:0/3973171708 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56010c33daf0 0x56010c33ff20 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe25b578740 1 --2- 172.21.15.94:0/3973171708 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c32d750 0x56010c32b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe239ffb640 0 client.18333 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe25b578740 1 --2- 172.21.15.94:0/3973171708 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324b70 0x56010c32ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe25b578740 1 --2- 172.21.15.94:0/3973171708 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324250 0x56010c32a730 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe25b578740 1 -- 172.21.15.94:0/3973171708 >> 172.21.15.94:0/3973171708 conn(0x56010c31e980 msgr2=0x56010c3206d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe25b578740 1 -- 172.21.15.94:0/3973171708 shutdown_connections 2024-02-17T12:08:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe25b578740 1 -- 172.21.15.94:0/3973171708 wait complete. 2024-02-17T12:08:53.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.926+0000 7fe25b578740 20 client.18333 trim_cache size 0 max 16384 2024-02-17T12:08:53.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.941+0000 7fe25b578740 1 Processor -- start 2024-02-17T12:08:53.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.941+0000 7fe25b578740 1 -- start start 2024-02-17T12:08:53.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.941+0000 7fe25b578740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324250 0x56010c31da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.941+0000 7fe25b578740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324b70 0x56010c3345a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.941+0000 7fe25b578740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c32d750 0x56010c334ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.941+0000 7fe25b578740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c211130 con 0x56010c324b70 2024-02-17T12:08:53.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.941+0000 7fe25b578740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c345170 con 0x56010c324250 2024-02-17T12:08:53.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.941+0000 7fe25b578740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c31e7b0 con 0x56010c32d750 2024-02-17T12:08:53.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.941+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324b70 0x56010c3345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.941+0000 7fe25259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324250 0x56010c31da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.941+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324b70 0x56010c3345a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47362/0 (socket says 172.21.15.94:47362) 2024-02-17T12:08:53.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.941+0000 7fe251d9b640 1 -- 172.21.15.94:0/532300934 learned_addr learned my addr 172.21.15.94:0/532300934 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.942+0000 7fe252d9d640 1 --2- 172.21.15.94:0/532300934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c32d750 0x56010c334ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.942+0000 7fe25259c640 1 -- 172.21.15.94:0/532300934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c32d750 msgr2=0x56010c334ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.942+0000 7fe25259c640 1 --2- 172.21.15.94:0/532300934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c32d750 0x56010c334ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.942+0000 7fe25259c640 1 -- 172.21.15.94:0/532300934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324b70 msgr2=0x56010c3345a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.942+0000 7fe25259c640 1 --2- 172.21.15.94:0/532300934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324b70 0x56010c3345a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.942+0000 7fe25259c640 1 -- 172.21.15.94:0/532300934 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010c226c20 con 0x56010c324250 2024-02-17T12:08:53.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.942+0000 7fe251d9b640 1 --2- 172.21.15.94:0/532300934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324b70 0x56010c3345a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:53.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.942+0000 7fe25259c640 1 --2- 172.21.15.94:0/532300934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324250 0x56010c31da00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe24400f730 tx=0x7fe24400b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b70b3157a3918b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.943+0000 7fe23bfff640 1 -- 172.21.15.94:0/532300934 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe244005240 con 0x56010c324250 2024-02-17T12:08:53.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.943+0000 7fe23bfff640 1 -- 172.21.15.94:0/532300934 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe244005bf0 con 0x56010c324250 2024-02-17T12:08:53.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.943+0000 7fe23bfff640 1 -- 172.21.15.94:0/532300934 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24401f8a0 con 0x56010c324250 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.944+0000 7fe25b578740 1 -- 172.21.15.94:0/532300934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324250 msgr2=0x56010c31da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.944+0000 7fe25b578740 1 --2- 172.21.15.94:0/532300934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324250 0x56010c31da00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe24400f730 tx=0x7fe24400b820 comp rx=0 tx=0).stop 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.944+0000 7fe25b578740 1 -- 172.21.15.94:0/532300934 shutdown_connections 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.944+0000 7fe25b578740 1 --2- 172.21.15.94:0/532300934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c32d750 0x56010c334ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.944+0000 7fe25b578740 1 --2- 172.21.15.94:0/532300934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c324b70 0x56010c3345a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.944+0000 7fe25b578740 1 --2- 172.21.15.94:0/532300934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324250 0x56010c31da00 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.944+0000 7fe25b578740 1 -- 172.21.15.94:0/532300934 >> 172.21.15.94:0/532300934 conn(0x56010c31e980 msgr2=0x56010c31d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.944+0000 7fe25b578740 1 -- 172.21.15.94:0/532300934 shutdown_connections 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.944+0000 7fe25b578740 1 -- 172.21.15.94:0/532300934 wait complete. 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.944+0000 7fe25b578740 1 -- start start 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.944+0000 7fe25b578740 1 Processor -- start 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.945+0000 7fe25b578740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324250 0x56010c333140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.945+0000 7fe25b578740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324b70 0x56010c333680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.945+0000 7fe25b578740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c32d750 0x56010c333bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.945+0000 7fe25b578740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c394140 con 0x56010c32d750 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.945+0000 7fe25b578740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c3942b0 con 0x56010c324250 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.945+0000 7fe25b578740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010c394470 con 0x56010c324b70 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.945+0000 7fe25b578740 10 client.0 _ll_register_callbacks cb 0x7fe248005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.945+0000 7fe252d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c32d750 0x56010c333bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.945+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324b70 0x56010c333680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.945+0000 7fe251d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324b70 0x56010c333680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:51916/0 (socket says 172.21.15.94:51916) 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.945+0000 7fe251d9b640 1 -- 172.21.15.94:0/4116549911 learned_addr learned my addr 172.21.15.94:0/4116549911 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.945+0000 7fe252d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c32d750 0x56010c333bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47368/0 (socket says 172.21.15.94:47368) 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.946+0000 7fe251d9b640 1 -- 172.21.15.94:0/4116549911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324250 msgr2=0x56010c333140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.946+0000 7fe251d9b640 1 --2- 172.21.15.94:0/4116549911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324250 0x56010c333140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.946+0000 7fe251d9b640 1 -- 172.21.15.94:0/4116549911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c32d750 msgr2=0x56010c333bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.946+0000 7fe251d9b640 1 --2- 172.21.15.94:0/4116549911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c32d750 0x56010c333bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.946+0000 7fe251d9b640 1 -- 172.21.15.94:0/4116549911 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2541a6440 con 0x56010c324b70 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.946+0000 7fe251d9b640 1 --2- 172.21.15.94:0/4116549911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324b70 0x56010c333680 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe248005f60 tx=0x7fe24800e940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a757ab043bf2be8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:53.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.946+0000 7fe239ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:08:53.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.946+0000 7fe239ffb640 1 -- 172.21.15.94:0/4116549911 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2480b9220 con 0x56010c324b70 2024-02-17T12:08:53.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.946+0000 7fe25b578740 1 -- 172.21.15.94:0/4116549911 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe24c003970 con 0x56010c324b70 2024-02-17T12:08:53.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.946+0000 7fe239ffb640 1 -- 172.21.15.94:0/4116549911 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe2480aee40 con 0x56010c324b70 2024-02-17T12:08:53.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.947+0000 7fe239ffb640 1 -- 172.21.15.94:0/4116549911 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe2480b9380 con 0x56010c324b70 2024-02-17T12:08:53.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.947+0000 7fe25b578740 10 client.27790 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:53.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.947+0000 7fe25b578740 1 -- 172.21.15.94:0/4116549911 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56010c28a6a0 con 0x56010c324b70 2024-02-17T12:08:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.947+0000 7fe239ffb640 1 -- 172.21.15.94:0/4116549911 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe2480cbdd0 con 0x56010c324b70 2024-02-17T12:08:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.947+0000 7fe239ffb640 1 -- 172.21.15.94:0/4116549911 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe248096b40 con 0x56010c324b70 2024-02-17T12:08:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.948+0000 7fe25b578740 10 client.27790 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.948+0000 7fe25b578740 10 client.27790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.948+0000 7fe25b578740 20 client.27790 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:53.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.948+0000 7fe25b578740 1 --2- 172.21.15.94:0/4116549911 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56010c395c70 0x56010c398090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.948+0000 7fe25b578740 4 client.27790 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:08:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.948+0000 7fe25b578740 1 -- 172.21.15.94:0/4116549911 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe24c003970 con 0x56010c395c70 2024-02-17T12:08:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.949+0000 7fe25259c640 1 --2- 172.21.15.94:0/4116549911 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56010c395c70 0x56010c398090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.950+0000 7fe239ffb640 10 client.27790 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.950+0000 7fe25259c640 1 --2- 172.21.15.94:0/4116549911 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56010c395c70 0x56010c398090 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39a8e03b95900224 server_cookie=111c77e583956c0b in_seq=0 out_seq=0 2024-02-17T12:08:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe239ffb640 1 -- 172.21.15.94:0/4116549911 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe24c003970 con 0x56010c395c70 2024-02-17T12:08:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe239ffb640 10 client.27790 handle_command_reply: tid=0 2024-02-17T12:08:53.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe25b578740 1 client.27790 shutdown 2024-02-17T12:08:53.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe25b578740 1 -- 172.21.15.94:0/4116549911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324b70 msgr2=0x56010c333680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe25b578740 1 --2- 172.21.15.94:0/4116549911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324b70 0x56010c333680 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe248005f60 tx=0x7fe24800e940 comp rx=0 tx=0).stop 2024-02-17T12:08:53.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe25b578740 1 -- 172.21.15.94:0/4116549911 shutdown_connections 2024-02-17T12:08:53.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe25b578740 1 --2- 172.21.15.94:0/4116549911 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56010c395c70 0x56010c398090 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe25b578740 1 --2- 172.21.15.94:0/4116549911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56010c32d750 0x56010c333bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe25b578740 1 --2- 172.21.15.94:0/4116549911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56010c324b70 0x56010c333680 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe25b578740 1 --2- 172.21.15.94:0/4116549911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56010c324250 0x56010c333140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe25b578740 1 -- 172.21.15.94:0/4116549911 >> 172.21.15.94:0/4116549911 conn(0x56010c31e980 msgr2=0x56010c31db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe239ffb640 0 client.27790 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:53.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe25b578740 1 -- 172.21.15.94:0/4116549911 shutdown_connections 2024-02-17T12:08:53.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.951+0000 7fe25b578740 1 -- 172.21.15.94:0/4116549911 wait complete. 2024-02-17T12:08:53.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.952+0000 7fe25b578740 20 client.27790 trim_cache size 0 max 16384 2024-02-17T12:08:53.955 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:08:53.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.952+0000 7fe2589a1640 1 -- 172.21.15.94:0/3242187608 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe23c0a22e0 msgr2=0x7fe23c0a47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.952+0000 7fe2589a1640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe23c0a22e0 0x7fe23c0a47a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe244005fd0 tx=0x7fe24400b510 comp rx=0 tx=0).stop 2024-02-17T12:08:53.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.952+0000 7fe2589a1640 1 -- 172.21.15.94:0/3242187608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2541009e0 msgr2=0x7fe254102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:53.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.952+0000 7fe2589a1640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2541009e0 0x7fe254102230 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe24800e9f0 tx=0x7fe24800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.953+0000 7fe2589a1640 1 -- 172.21.15.94:0/3242187608 shutdown_connections 2024-02-17T12:08:53.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.953+0000 7fe2589a1640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe23c0a22e0 0x7fe23c0a47a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.953+0000 7fe2589a1640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe254106100 0x7fe254102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.953+0000 7fe2589a1640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe2541009e0 0x7fe254102230 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.953+0000 7fe2589a1640 1 --2- 172.21.15.94:0/3242187608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe254100040 0x7fe254101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:53.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.953+0000 7fe2589a1640 1 -- 172.21.15.94:0/3242187608 >> 172.21.15.94:0/3242187608 conn(0x7fe254077110 msgr2=0x7fe2540ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:53.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.953+0000 7fe2589a1640 1 -- 172.21.15.94:0/3242187608 shutdown_connections 2024-02-17T12:08:53.957 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:53.953+0000 7fe2589a1640 1 -- 172.21.15.94:0/3242187608 wait complete. 2024-02-17T12:08:54.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:54 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1269439322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:08:54.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:54 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1269439322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:08:54.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:54 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1269439322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:08:54.968 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:08:54.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:08:54.969 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:08:55.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.125+0000 7f59b5d9b640 1 -- 172.21.15.94:0/4190894068 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a802e880 con 0x7f59b8106100 2024-02-17T12:08:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.126+0000 7f59bd0c3640 1 -- 172.21.15.94:0/4190894068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b8106100 msgr2=0x7f59b81064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.126+0000 7f59bd0c3640 1 --2- 172.21.15.94:0/4190894068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b8106100 0x7f59b81064e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f59a800d910 tx=0x7f59a802dc80 comp rx=0 tx=0).stop 2024-02-17T12:08:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.126+0000 7f59bd0c3640 1 -- 172.21.15.94:0/4190894068 shutdown_connections 2024-02-17T12:08:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.126+0000 7f59bd0c3640 1 --2- 172.21.15.94:0/4190894068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b81009e0 0x7f59b810cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.126+0000 7f59bd0c3640 1 --2- 172.21.15.94:0/4190894068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59b8100040 0x7f59b81004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.126+0000 7f59bd0c3640 1 --2- 172.21.15.94:0/4190894068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b8106100 0x7f59b81064e0 secure :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f59a800d910 tx=0x7f59a802dc80 comp rx=0 tx=0).stop 2024-02-17T12:08:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.126+0000 7f59bd0c3640 1 -- 172.21.15.94:0/4190894068 >> 172.21.15.94:0/4190894068 conn(0x7f59b8077110 msgr2=0x7f59b8077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.126+0000 7f59bd0c3640 1 -- 172.21.15.94:0/4190894068 shutdown_connections 2024-02-17T12:08:55.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.126+0000 7f59bd0c3640 1 -- 172.21.15.94:0/4190894068 wait complete. 2024-02-17T12:08:55.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.127+0000 7f59bd0c3640 1 Processor -- start 2024-02-17T12:08:55.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.127+0000 7f59bd0c3640 1 -- start start 2024-02-17T12:08:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59bd0c3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b8100040 0x7f59b819b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59bd0c3640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59b81009e0 0x7f59b819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59bd0c3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b8195770 0x7f59b8195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59bd0c3640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8196240 con 0x7f59b8100040 2024-02-17T12:08:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59bd0c3640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b81963b0 con 0x7f59b81009e0 2024-02-17T12:08:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59bd0c3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8196570 con 0x7f59b8195770 2024-02-17T12:08:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59b759e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b8195770 0x7f59b8195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59b759e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b8195770 0x7f59b8195bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:45762/0 (socket says 172.21.15.94:45762) 2024-02-17T12:08:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59b759e640 1 -- 172.21.15.94:0/3251230552 learned_addr learned my addr 172.21.15.94:0/3251230552 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59b6d9d640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b8100040 0x7f59b819b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59b759e640 1 -- 172.21.15.94:0/3251230552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59b81009e0 msgr2=0x7f59b819bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59b759e640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59b81009e0 0x7f59b819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59b759e640 1 -- 172.21.15.94:0/3251230552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b8100040 msgr2=0x7f59b819b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59b759e640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b8100040 0x7f59b819b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.128+0000 7f59b759e640 1 -- 172.21.15.94:0/3251230552 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a800d570 con 0x7f59b8195770 2024-02-17T12:08:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.129+0000 7f59b6d9d640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b8100040 0x7f59b819b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.129+0000 7f59b759e640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b8195770 0x7f59b8195bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f59b000b970 tx=0x7f59b000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.129+0000 7f599ffff640 1 -- 172.21.15.94:0/3251230552 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0018440 con 0x7f59b8195770 2024-02-17T12:08:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.129+0000 7f59bd0c3640 1 -- 172.21.15.94:0/3251230552 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b81a2510 con 0x7f59b8195770 2024-02-17T12:08:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.129+0000 7f59bd0c3640 1 -- 172.21.15.94:0/3251230552 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b81a29f0 con 0x7f59b8195770 2024-02-17T12:08:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.129+0000 7f599ffff640 1 -- 172.21.15.94:0/3251230552 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59b0018a80 con 0x7f59b8195770 2024-02-17T12:08:55.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.129+0000 7f599ffff640 1 -- 172.21.15.94:0/3251230552 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0007c20 con 0x7f59b8195770 2024-02-17T12:08:55.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.133+0000 7f599ffff640 1 -- 172.21.15.94:0/3251230552 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f59b0018bf0 con 0x7f59b8195770 2024-02-17T12:08:55.135 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.134+0000 7f59bfd72740 1 Processor -- start 2024-02-17T12:08:55.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.134+0000 7f59bfd72740 1 -- start start 2024-02-17T12:08:55.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.135+0000 7f59bfd72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198b8320 0x557a198b8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.135+0000 7f59bfd72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.135+0000 7f59bfd72740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198ca0c0 0x557a198cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.135+0000 7f59bfd72740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a19826f90 con 0x557a198b8cd0 2024-02-17T12:08:55.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.135+0000 7f59bfd72740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a198244c0 con 0x557a198ca0c0 2024-02-17T12:08:55.136 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.135+0000 7f59bfd72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a1981fca0 con 0x557a198b8320 2024-02-17T12:08:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.135+0000 7f59b6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.135+0000 7f59b6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55478/0 (socket says 172.21.15.94:55478) 2024-02-17T12:08:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.135+0000 7f59b6d9d640 1 -- 172.21.15.94:0/584757989 learned_addr learned my addr 172.21.15.94:0/584757989 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.135+0000 7f599ffff640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59a00a2400 0x7f59a00a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.135+0000 7f59b759e640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59a00a2400 0x7f59a00a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.135+0000 7f59b659c640 1 --2- 172.21.15.94:0/584757989 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198ca0c0 0x557a198cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.135+0000 7f59b659c640 1 --2- 172.21.15.94:0/584757989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198b8320 0x557a198b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.136+0000 7f599ffff640 1 -- 172.21.15.94:0/3251230552 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f59b00d1bb0 con 0x7f59b8195770 2024-02-17T12:08:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.136+0000 7f59b659c640 1 -- 172.21.15.94:0/584757989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198b8320 msgr2=0x557a198b8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.136+0000 7f59b659c640 1 --2- 172.21.15.94:0/584757989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198b8320 0x557a198b8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.136+0000 7f59b659c640 1 -- 172.21.15.94:0/584757989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 msgr2=0x557a198c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.136+0000 7f59b659c640 1 --2- 172.21.15.94:0/584757989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.136+0000 7f59b659c640 1 -- 172.21.15.94:0/584757989 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a197e20b0 con 0x557a198ca0c0 2024-02-17T12:08:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.136+0000 7f59b759e640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59a00a2400 0x7f59a00a48c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f59b009a3a0 tx=0x7f59b009a330 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.136+0000 7f59b659c640 1 --2- 172.21.15.94:0/584757989 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198ca0c0 0x557a198cc4d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f59b8196fc0 tx=0x7f59ac009d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb553ea6c5b65e9f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.136+0000 7f599d7fa640 1 -- 172.21.15.94:0/584757989 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac007120 con 0x557a198ca0c0 2024-02-17T12:08:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.136+0000 7f599d7fa640 1 -- 172.21.15.94:0/584757989 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59ac008920 con 0x557a198ca0c0 2024-02-17T12:08:55.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.136+0000 7f599d7fa640 1 -- 172.21.15.94:0/584757989 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac004b90 con 0x557a198ca0c0 2024-02-17T12:08:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.137+0000 7f59bfd72740 1 -- 172.21.15.94:0/584757989 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198ca0c0 msgr2=0x557a198cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.137+0000 7f59bfd72740 1 --2- 172.21.15.94:0/584757989 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198ca0c0 0x557a198cc4d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f59b8196fc0 tx=0x7f59ac009d40 comp rx=0 tx=0).stop 2024-02-17T12:08:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.137+0000 7f59bfd72740 1 -- 172.21.15.94:0/584757989 shutdown_connections 2024-02-17T12:08:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.137+0000 7f59bfd72740 1 --2- 172.21.15.94:0/584757989 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198ca0c0 0x557a198cc4d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.137+0000 7f59bfd72740 1 --2- 172.21.15.94:0/584757989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.137+0000 7f59bfd72740 1 --2- 172.21.15.94:0/584757989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198b8320 0x557a198b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.137+0000 7f59bfd72740 1 -- 172.21.15.94:0/584757989 >> 172.21.15.94:0/584757989 conn(0x557a198b2840 msgr2=0x557a198b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:55.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.137+0000 7f59bfd72740 1 -- 172.21.15.94:0/584757989 shutdown_connections 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.137+0000 7f59bfd72740 1 -- 172.21.15.94:0/584757989 wait complete. 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59bfd72740 1 -- start start 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59bfd72740 1 Processor -- start 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59bfd72740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8320 0x557a198c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59bfd72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59bfd72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 0x557a198c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59bfd72740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a197c4f40 con 0x557a198b8cd0 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59bfd72740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a194f34b0 con 0x557a198b8320 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59bfd72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a197a5670 con 0x557a198ca0c0 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59bfd72740 10 client.0 _ll_register_callbacks cb 0x557a19812f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59b6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59b659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8320 0x557a198c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59b6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55488/0 (socket says 172.21.15.94:55488) 2024-02-17T12:08:55.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59b6d9d640 1 -- 172.21.15.94:0/338491134 learned_addr learned my addr 172.21.15.94:0/338491134 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.138+0000 7f59b659c640 1 --2- 172.21.15.94:0/338491134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 0x557a198c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.139+0000 7f59b6d9d640 1 -- 172.21.15.94:0/338491134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 msgr2=0x557a198c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.139+0000 7f59b6d9d640 1 --2- 172.21.15.94:0/338491134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 0x557a198c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.139+0000 7f59b6d9d640 1 -- 172.21.15.94:0/338491134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8320 msgr2=0x557a198c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.139+0000 7f59b6d9d640 1 --2- 172.21.15.94:0/338491134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8320 0x557a198c7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.139+0000 7f59b6d9d640 1 -- 172.21.15.94:0/338491134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ac001d20 con 0x557a198b8cd0 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.139+0000 7f59b6d9d640 1 --2- 172.21.15.94:0/338491134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198c75d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009230 tx=0x7f59a8005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12613929105ad55b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.139+0000 7f599aff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.139+0000 7f599aff5640 1 -- 172.21.15.94:0/338491134 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8005180 con 0x557a198b8cd0 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.139+0000 7f59bfd72740 1 -- 172.21.15.94:0/338491134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a197f9f40 con 0x557a198b8cd0 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.139+0000 7f599aff5640 1 -- 172.21.15.94:0/338491134 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f59a80052e0 con 0x557a198b8cd0 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.140+0000 7f599aff5640 1 -- 172.21.15.94:0/338491134 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59a8004030 con 0x557a198b8cd0 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.140+0000 7f59bfd72740 10 client.18351 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.140+0000 7f59bfd72740 1 -- 172.21.15.94:0/338491134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557a1981e320 con 0x557a198b8cd0 2024-02-17T12:08:55.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.140+0000 7f599aff5640 1 -- 172.21.15.94:0/338491134 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557a197f9f40 con 0x557a198b8cd0 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.140+0000 7f599aff5640 1 -- 172.21.15.94:0/338491134 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59a80035f0 con 0x557a198b8cd0 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.140+0000 7f59bfd72740 10 client.18351 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.140+0000 7f59bfd72740 10 client.18351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.140+0000 7f59bfd72740 20 client.18351 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.140+0000 7f59bfd72740 1 --2- 172.21.15.94:0/338491134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557a198da770 0x557a198dcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.140+0000 7f59bfd72740 4 client.18351 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.140+0000 7f59bfd72740 1 -- 172.21.15.94:0/338491134 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a19816310 con 0x557a198da770 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.140+0000 7f59b659c640 1 --2- 172.21.15.94:0/338491134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557a198da770 0x557a198dcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.140+0000 7f599aff5640 10 client.18351 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f59b659c640 1 --2- 172.21.15.94:0/338491134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557a198da770 0x557a198dcb90 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86fd417a1db8b146 server_cookie=e40811aa0b62d52e in_seq=0 out_seq=0 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f599aff5640 1 -- 172.21.15.94:0/338491134 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a19816310 con 0x557a198da770 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f599aff5640 10 client.18351 handle_command_reply: tid=0 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f59bfd72740 1 client.18351 shutdown 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f59bfd72740 1 -- 172.21.15.94:0/338491134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 msgr2=0x557a198c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f59bfd72740 1 --2- 172.21.15.94:0/338491134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198c75d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009230 tx=0x7f59a8005e50 comp rx=0 tx=0).stop 2024-02-17T12:08:55.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f59bfd72740 1 -- 172.21.15.94:0/338491134 shutdown_connections 2024-02-17T12:08:55.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f59bfd72740 1 --2- 172.21.15.94:0/338491134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557a198da770 0x557a198dcb90 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f59bfd72740 1 --2- 172.21.15.94:0/338491134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 0x557a198c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f59bfd72740 1 --2- 172.21.15.94:0/338491134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198c75d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f59bfd72740 1 --2- 172.21.15.94:0/338491134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8320 0x557a198c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f59bfd72740 1 -- 172.21.15.94:0/338491134 >> 172.21.15.94:0/338491134 conn(0x557a198b2840 msgr2=0x557a198b47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:55.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f599aff5640 0 client.18351 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:55.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f59bfd72740 1 -- 172.21.15.94:0/338491134 shutdown_connections 2024-02-17T12:08:55.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.141+0000 7f59bfd72740 1 -- 172.21.15.94:0/338491134 wait complete. 2024-02-17T12:08:55.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.142+0000 7f59bfd72740 20 client.18351 trim_cache size 0 max 16384 2024-02-17T12:08:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.161+0000 7f59bfd72740 1 Processor -- start 2024-02-17T12:08:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.161+0000 7f59bfd72740 1 -- start start 2024-02-17T12:08:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.162+0000 7f59bfd72740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8320 0x557a198c7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.162+0000 7f59bfd72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.162+0000 7f59bfd72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 0x557a198d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.162+0000 7f59bfd72740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a198d6190 con 0x557a198b8cd0 2024-02-17T12:08:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.162+0000 7f59bfd72740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a198b7410 con 0x557a198b8320 2024-02-17T12:08:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.162+0000 7f59bfd72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a198b2370 con 0x557a198ca0c0 2024-02-17T12:08:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.162+0000 7f59b6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.162+0000 7f59b6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55500/0 (socket says 172.21.15.94:55500) 2024-02-17T12:08:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.162+0000 7f59b6d9d640 1 -- 172.21.15.94:0/3362496121 learned_addr learned my addr 172.21.15.94:0/3362496121 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.162+0000 7f59b659c640 1 --2- 172.21.15.94:0/3362496121 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8320 0x557a198c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.163+0000 7f59b659c640 1 --2- 172.21.15.94:0/3362496121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 0x557a198d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.163+0000 7f59b659c640 1 -- 172.21.15.94:0/3362496121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 msgr2=0x557a198d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.163+0000 7f59b659c640 1 --2- 172.21.15.94:0/3362496121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 0x557a198d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.163+0000 7f59b659c640 1 -- 172.21.15.94:0/3362496121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 msgr2=0x557a198b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.163+0000 7f59b659c640 1 --2- 172.21.15.94:0/3362496121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.163+0000 7f59b659c640 1 -- 172.21.15.94:0/3362496121 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a197bac00 con 0x557a198b8320 2024-02-17T12:08:55.166 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.163+0000 7f59b6d9d640 1 --2- 172.21.15.94:0/3362496121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.163+0000 7f59b659c640 1 --2- 172.21.15.94:0/3362496121 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8320 0x557a198c7930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f59ac004870 tx=0x7f59ac001640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b07b97a142543b67 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.164+0000 7f599cff9640 1 -- 172.21.15.94:0/3362496121 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac02c070 con 0x557a198b8320 2024-02-17T12:08:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.164+0000 7f599cff9640 1 -- 172.21.15.94:0/3362496121 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f59ac007600 con 0x557a198b8320 2024-02-17T12:08:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.164+0000 7f599cff9640 1 -- 172.21.15.94:0/3362496121 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac01f040 con 0x557a198b8320 2024-02-17T12:08:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.165+0000 7f59bfd72740 1 -- 172.21.15.94:0/3362496121 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8320 msgr2=0x557a198c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.165+0000 7f59bfd72740 1 --2- 172.21.15.94:0/3362496121 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8320 0x557a198c7930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f59ac004870 tx=0x7f59ac001640 comp rx=0 tx=0).stop 2024-02-17T12:08:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.165+0000 7f59bfd72740 1 -- 172.21.15.94:0/3362496121 shutdown_connections 2024-02-17T12:08:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.165+0000 7f59bfd72740 1 --2- 172.21.15.94:0/3362496121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 0x557a198d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.165+0000 7f59bfd72740 1 --2- 172.21.15.94:0/3362496121 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8cd0 0x557a198b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.165+0000 7f59bfd72740 1 --2- 172.21.15.94:0/3362496121 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8320 0x557a198c7930 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.165+0000 7f59bfd72740 1 -- 172.21.15.94:0/3362496121 >> 172.21.15.94:0/3362496121 conn(0x557a198b2840 msgr2=0x557a198b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.165+0000 7f59bfd72740 1 -- 172.21.15.94:0/3362496121 shutdown_connections 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.165+0000 7f59bfd72740 1 -- 172.21.15.94:0/3362496121 wait complete. 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.165+0000 7f59bfd72740 1 -- start start 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.165+0000 7f59bfd72740 1 Processor -- start 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59bfd72740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8320 0x557a198bc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59bfd72740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8cd0 0x557a198bcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59bfd72740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 0x557a198d0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59bfd72740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a198d0b40 con 0x557a198b8320 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59bfd72740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a198d0cb0 con 0x557a198b8cd0 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59bfd72740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a198d0e70 con 0x557a198ca0c0 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59bfd72740 10 client.0 _ll_register_callbacks cb 0x7f59ac0014e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59b6d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8cd0 0x557a198bcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59b659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8320 0x557a198bc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59b6d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8cd0 0x557a198bcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:54584/0 (socket says 172.21.15.94:54584) 2024-02-17T12:08:55.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59b6d9d640 1 -- 172.21.15.94:0/3727094793 learned_addr learned my addr 172.21.15.94:0/3727094793 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59b659c640 1 --2- 172.21.15.94:0/3727094793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 0x557a198d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59b6d9d640 1 -- 172.21.15.94:0/3727094793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 msgr2=0x557a198d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59b6d9d640 1 --2- 172.21.15.94:0/3727094793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 0x557a198d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59b6d9d640 1 -- 172.21.15.94:0/3727094793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8320 msgr2=0x557a198bc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59b6d9d640 1 --2- 172.21.15.94:0/3727094793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8320 0x557a198bc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59b6d9d640 1 -- 172.21.15.94:0/3727094793 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a1981e320 con 0x557a198b8cd0 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.166+0000 7f59b6d9d640 1 --2- 172.21.15.94:0/3727094793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8cd0 0x557a198bcaa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009230 tx=0x7f59a802e420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c80b2489872da2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f599aff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f599aff5640 1 -- 172.21.15.94:0/3727094793 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8003b70 con 0x557a198b8cd0 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f599aff5640 1 -- 172.21.15.94:0/3727094793 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f59a8000e40 con 0x557a198b8cd0 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f59bfd72740 1 -- 172.21.15.94:0/3727094793 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a19845900 con 0x557a198b8cd0 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f599aff5640 1 -- 172.21.15.94:0/3727094793 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59a8003cd0 con 0x557a198b8cd0 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f59bfd72740 10 client.27830 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:55.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f59bfd72740 1 -- 172.21.15.94:0/3727094793 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f59ac022040 con 0x557a198b8cd0 2024-02-17T12:08:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f599aff5640 1 -- 172.21.15.94:0/3727094793 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557a19845900 con 0x557a198b8cd0 2024-02-17T12:08:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f599aff5640 1 -- 172.21.15.94:0/3727094793 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f59a802e740 con 0x557a198b8cd0 2024-02-17T12:08:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f59bfd72740 10 client.27830 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f59bfd72740 10 client.27830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f59bfd72740 20 client.27830 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f59bfd72740 1 --2- 172.21.15.94:0/3727094793 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557a198c1140 0x557a198c1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f59bfd72740 4 client.27830 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:08:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.167+0000 7f59bfd72740 1 -- 172.21.15.94:0/3727094793 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557a1951f860 con 0x557a198c1140 2024-02-17T12:08:55.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.168+0000 7f59b659c640 1 --2- 172.21.15.94:0/3727094793 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557a198c1140 0x557a198c1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.168+0000 7f599aff5640 10 client.27830 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:55.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.168+0000 7f59b659c640 1 --2- 172.21.15.94:0/3727094793 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557a198c1140 0x557a198c1550 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e42ae8796960da27 server_cookie=977aac7a8fbaf1d5 in_seq=0 out_seq=0 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.169+0000 7f599aff5640 1 -- 172.21.15.94:0/3727094793 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x7f59ac01d080 con 0x557a198c1140 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.169+0000 7f599aff5640 10 client.27830 handle_command_reply: tid=0 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.169+0000 7f59bfd72740 1 client.27830 shutdown 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.170+0000 7f59bfd72740 1 -- 172.21.15.94:0/3727094793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8cd0 msgr2=0x557a198bcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.170+0000 7f59bfd72740 1 --2- 172.21.15.94:0/3727094793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8cd0 0x557a198bcaa0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009230 tx=0x7f59a802e420 comp rx=0 tx=0).stop 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.170+0000 7f59bfd72740 1 -- 172.21.15.94:0/3727094793 shutdown_connections 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.170+0000 7f59bfd72740 1 --2- 172.21.15.94:0/3727094793 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557a198c1140 0x557a198c1550 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.170+0000 7f59bfd72740 1 --2- 172.21.15.94:0/3727094793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a198ca0c0 0x557a198d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.170+0000 7f59bfd72740 1 --2- 172.21.15.94:0/3727094793 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557a198b8cd0 0x557a198bcaa0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.170+0000 7f59bfd72740 1 --2- 172.21.15.94:0/3727094793 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557a198b8320 0x557a198bc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.170+0000 7f59bfd72740 1 -- 172.21.15.94:0/3727094793 >> 172.21.15.94:0/3727094793 conn(0x557a198b2840 msgr2=0x557a198b18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.170+0000 7f599aff5640 0 client.27830 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.170+0000 7f59bfd72740 1 -- 172.21.15.94:0/3727094793 shutdown_connections 2024-02-17T12:08:55.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.170+0000 7f59bfd72740 1 -- 172.21.15.94:0/3727094793 wait complete. 2024-02-17T12:08:55.175 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.170+0000 7f59bfd72740 20 client.27830 trim_cache size 0 max 16384 2024-02-17T12:08:55.175 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:08:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:08:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "f715a9d4-960d-409c-b3f9-b68728740e51", 2024-02-17T12:08:55.175 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:08:55.175 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:08:55.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.172+0000 7f59bd0c3640 1 -- 172.21.15.94:0/3251230552 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59a00a2400 msgr2=0x7f59a00a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.173+0000 7f59bd0c3640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59a00a2400 0x7f59a00a48c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f59b009a3a0 tx=0x7f59b009a330 comp rx=0 tx=0).stop 2024-02-17T12:08:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.173+0000 7f59bd0c3640 1 -- 172.21.15.94:0/3251230552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b8195770 msgr2=0x7f59b8195bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.173+0000 7f59bd0c3640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b8195770 0x7f59b8195bf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f59b000b970 tx=0x7f59b000be30 comp rx=0 tx=0).stop 2024-02-17T12:08:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.173+0000 7f59bd0c3640 1 -- 172.21.15.94:0/3251230552 shutdown_connections 2024-02-17T12:08:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.173+0000 7f59bd0c3640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f59a00a2400 0x7f59a00a48c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.173+0000 7f59bd0c3640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59b8195770 0x7f59b8195bf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.173+0000 7f59bd0c3640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f59b81009e0 0x7f59b819bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.173+0000 7f59bd0c3640 1 --2- 172.21.15.94:0/3251230552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f59b8100040 0x7f59b819b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.173+0000 7f59bd0c3640 1 -- 172.21.15.94:0/3251230552 >> 172.21.15.94:0/3251230552 conn(0x7f59b8077110 msgr2=0x7f59b80feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.173+0000 7f59bd0c3640 1 -- 172.21.15.94:0/3251230552 shutdown_connections 2024-02-17T12:08:55.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.173+0000 7f59bd0c3640 1 -- 172.21.15.94:0/3251230552 wait complete. 2024-02-17T12:08:55.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:08:55.188 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:08:55.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 635.0 s, 1729.9 tps, lat 58.070 ms stddev 72.322 2024-02-17T12:08:55.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.363+0000 7fcf8f756640 1 -- 172.21.15.94:0/2475233156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf8811a960 msgr2=0x7fcf8811cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.363+0000 7fcf8f756640 1 --2- 172.21.15.94:0/2475233156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf8811a960 0x7fcf8811cd50 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fcf78009860 tx=0x7fcf7802bbb0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.363+0000 7fcf8f756640 1 -- 172.21.15.94:0/2475233156 shutdown_connections 2024-02-17T12:08:55.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.363+0000 7fcf8f756640 1 --2- 172.21.15.94:0/2475233156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf8811a960 0x7fcf8811cd50 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.363+0000 7fcf8f756640 1 --2- 172.21.15.94:0/2475233156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf8810fd10 0x7fcf8810da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.363+0000 7fcf8f756640 1 --2- 172.21.15.94:0/2475233156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf8810f8a0 0x7fcf8810d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.368 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.363+0000 7fcf8f756640 1 -- 172.21.15.94:0/2475233156 >> 172.21.15.94:0/2475233156 conn(0x7fcf8806c270 msgr2=0x7fcf8806c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:55.372 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.370+0000 7fcf8f756640 1 -- 172.21.15.94:0/2475233156 shutdown_connections 2024-02-17T12:08:55.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.371+0000 7fcf8f756640 1 -- 172.21.15.94:0/2475233156 wait complete. 2024-02-17T12:08:55.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.372+0000 7fcf8f756640 1 Processor -- start 2024-02-17T12:08:55.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.377+0000 7fcf8f756640 1 -- start start 2024-02-17T12:08:55.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.377+0000 7fcf8f756640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf8810f8a0 0x7fcf88113d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.377+0000 7fcf8f756640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf8810fd10 0x7fcf881142a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.377+0000 7fcf8f756640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf8811a960 0x7fcf88118620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.377+0000 7fcf8f756640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf88118c20 con 0x7fcf8811a960 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.377+0000 7fcf8f756640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf88118d90 con 0x7fcf8810fd10 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.377+0000 7fcf8f756640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf88118f50 con 0x7fcf8810f8a0 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.378+0000 7fcf8d4f2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf8810f8a0 0x7fcf88113d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.378+0000 7fcf8d4f2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf8810f8a0 0x7fcf88113d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:45810/0 (socket says 172.21.15.94:45810) 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.378+0000 7fcf8d4f2640 1 -- 172.21.15.94:0/2320126255 learned_addr learned my addr 172.21.15.94:0/2320126255 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.378+0000 7fcf8dcf3640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf8811a960 0x7fcf88118620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.378+0000 7fcf8ccf1640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf8810fd10 0x7fcf881142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.380+0000 7fcf8ccf1640 1 -- 172.21.15.94:0/2320126255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf8810f8a0 msgr2=0x7fcf88113d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.380+0000 7fcf8ccf1640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf8810f8a0 0x7fcf88113d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.380+0000 7fcf8ccf1640 1 -- 172.21.15.94:0/2320126255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf8811a960 msgr2=0x7fcf88118620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.380+0000 7fcf8ccf1640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf8811a960 0x7fcf88118620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.380+0000 7fcf8ccf1640 1 -- 172.21.15.94:0/2320126255 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf78009490 con 0x7fcf8810fd10 2024-02-17T12:08:55.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.382+0000 7fcf8ccf1640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf8810fd10 0x7fcf881142a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009fd0 tx=0x7fcf8400b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:55.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.386+0000 7fcf7e7fc640 1 -- 172.21.15.94:0/2320126255 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84011630 con 0x7fcf8810fd10 2024-02-17T12:08:55.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.386+0000 7fcf7e7fc640 1 -- 172.21.15.94:0/2320126255 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf84011c70 con 0x7fcf8810fd10 2024-02-17T12:08:55.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.386+0000 7fcf7e7fc640 1 -- 172.21.15.94:0/2320126255 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84010dd0 con 0x7fcf8810fd10 2024-02-17T12:08:55.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.388+0000 7fcf8f756640 1 -- 172.21.15.94:0/2320126255 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf881191d0 con 0x7fcf8810fd10 2024-02-17T12:08:55.393 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.392+0000 7fcf8f756640 1 -- 172.21.15.94:0/2320126255 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8811fe70 con 0x7fcf8810fd10 2024-02-17T12:08:55.397 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.395+0000 7fcf92409740 1 Processor -- start 2024-02-17T12:08:55.398 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.396+0000 7fcf92409740 1 -- start start 2024-02-17T12:08:55.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.396+0000 7fcf92409740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e12b0 0x5589024e1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.396+0000 7fcf92409740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589024e1bd0 0x5589024ea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.396+0000 7fcf92409740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024ea7b0 0x5589024ecba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.396+0000 7fcf92409740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902450280 con 0x5589024e1bd0 2024-02-17T12:08:55.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:55 smithi189 ceph-mon[36588]: pgmap v335: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 32 MiB/s wr, 2.19k op/s 2024-02-17T12:08:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.396+0000 7fcf92409740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890244d7b0 con 0x5589024ea7b0 2024-02-17T12:08:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.396+0000 7fcf92409740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902448f90 con 0x5589024e12b0 2024-02-17T12:08:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.397+0000 7fcf8dcf3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e12b0 0x5589024e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.397+0000 7fcf8dcf3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e12b0 0x5589024e1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:45814/0 (socket says 172.21.15.94:45814) 2024-02-17T12:08:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.397+0000 7fcf8dcf3640 1 -- 172.21.15.94:0/1172816768 learned_addr learned my addr 172.21.15.94:0/1172816768 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.397+0000 7fcf8dcf3640 1 -- 172.21.15.94:0/1172816768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024ea7b0 msgr2=0x5589024ecba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:08:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.397+0000 7fcf8dcf3640 1 --2- 172.21.15.94:0/1172816768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024ea7b0 0x5589024ecba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.397+0000 7fcf8dcf3640 1 -- 172.21.15.94:0/1172816768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589024e1bd0 msgr2=0x5589024ea270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.397+0000 7fcf8dcf3640 1 --2- 172.21.15.94:0/1172816768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589024e1bd0 0x5589024ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.397+0000 7fcf8dcf3640 1 -- 172.21.15.94:0/1172816768 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558902407570 con 0x5589024e12b0 2024-02-17T12:08:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.398+0000 7fcf8dcf3640 1 --2- 172.21.15.94:0/1172816768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e12b0 0x5589024e1690 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcf78032340 tx=0x7fcf78032370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=157d8d41f593f2c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.398+0000 7fcf737fe640 1 -- 172.21.15.94:0/1172816768 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf78004040 con 0x5589024e12b0 2024-02-17T12:08:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.398+0000 7fcf737fe640 1 -- 172.21.15.94:0/1172816768 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf78034440 con 0x5589024e12b0 2024-02-17T12:08:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.399+0000 7fcf7e7fc640 1 -- 172.21.15.94:0/2320126255 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fcf84010380 con 0x7fcf8810fd10 2024-02-17T12:08:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.399+0000 7fcf92409740 1 -- 172.21.15.94:0/1172816768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e12b0 msgr2=0x5589024e1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.399+0000 7fcf92409740 1 --2- 172.21.15.94:0/1172816768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e12b0 0x5589024e1690 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcf78032340 tx=0x7fcf78032370 comp rx=0 tx=0).stop 2024-02-17T12:08:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.399+0000 7fcf7e7fc640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcf740a2440 0x7fcf740a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.400+0000 7fcf7e7fc640 1 -- 172.21.15.94:0/2320126255 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcf840efa90 con 0x7fcf8810fd10 2024-02-17T12:08:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.400+0000 7fcf8dcf3640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcf740a2440 0x7fcf740a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.401+0000 7fcf92409740 1 -- 172.21.15.94:0/1172816768 shutdown_connections 2024-02-17T12:08:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.401+0000 7fcf92409740 1 --2- 172.21.15.94:0/1172816768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024ea7b0 0x5589024ecba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.409 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.401+0000 7fcf92409740 1 --2- 172.21.15.94:0/1172816768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589024e1bd0 0x5589024ea270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.401+0000 7fcf92409740 1 --2- 172.21.15.94:0/1172816768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e12b0 0x5589024e1690 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.401+0000 7fcf92409740 1 -- 172.21.15.94:0/1172816768 >> 172.21.15.94:0/1172816768 conn(0x5589024db990 msgr2=0x5589024dbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.401+0000 7fcf92409740 1 -- 172.21.15.94:0/1172816768 shutdown_connections 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.401+0000 7fcf8dcf3640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcf740a2440 0x7fcf740a4900 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcf7802cd60 tx=0x7fcf7802ccf0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.401+0000 7fcf92409740 1 -- 172.21.15.94:0/1172816768 wait complete. 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.402+0000 7fcf92409740 1 -- start start 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.402+0000 7fcf92409740 1 Processor -- start 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.402+0000 7fcf92409740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e12b0 0x5589024ea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf92409740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e1bd0 0x5589024e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf92409740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589024ea7b0 0x5589024e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf92409740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55890211c430 con 0x5589024ea7b0 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf92409740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589020d8e00 con 0x5589024e12b0 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf92409740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589023cdfd0 con 0x5589024e1bd0 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf92409740 10 client.0 _ll_register_callbacks cb 0x558902423fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:55.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf8d4f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e12b0 0x5589024ea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf8d4f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e12b0 0x5589024ea120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:54618/0 (socket says 172.21.15.94:54618) 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf8d4f2640 1 -- 172.21.15.94:0/2801015148 learned_addr learned my addr 172.21.15.94:0/2801015148 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf8dcf3640 1 --2- 172.21.15.94:0/2801015148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589024ea7b0 0x5589024e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf8d4f2640 1 -- 172.21.15.94:0/2801015148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e1bd0 msgr2=0x5589024e74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf8d4f2640 1 --2- 172.21.15.94:0/2801015148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e1bd0 0x5589024e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf8d4f2640 1 -- 172.21.15.94:0/2801015148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589024ea7b0 msgr2=0x5589024e7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf8d4f2640 1 --2- 172.21.15.94:0/2801015148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589024ea7b0 0x5589024e7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.403+0000 7fcf8d4f2640 1 -- 172.21.15.94:0/2801015148 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8811d610 con 0x5589024e12b0 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.404+0000 7fcf8d4f2640 1 --2- 172.21.15.94:0/2801015148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e12b0 0x5589024ea120 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000fb60 tx=0x7fcf8000c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4141229731888e2e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.404+0000 7fcf70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.404+0000 7fcf70ff9640 1 -- 172.21.15.94:0/2801015148 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80014070 con 0x5589024e12b0 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.404+0000 7fcf92409740 1 -- 172.21.15.94:0/2801015148 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5589021487e0 con 0x5589024e12b0 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.404+0000 7fcf70ff9640 1 -- 172.21.15.94:0/2801015148 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcf80003b80 con 0x5589024e12b0 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.404+0000 7fcf70ff9640 1 -- 172.21.15.94:0/2801015148 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf8000ce70 con 0x5589024e12b0 2024-02-17T12:08:55.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.404+0000 7fcf92409740 10 client.27836 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.404+0000 7fcf92409740 1 -- 172.21.15.94:0/2801015148 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558902447610 con 0x5589024e12b0 2024-02-17T12:08:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.405+0000 7fcf70ff9640 1 -- 172.21.15.94:0/2801015148 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5589021487e0 con 0x5589024e12b0 2024-02-17T12:08:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.405+0000 7fcf70ff9640 1 -- 172.21.15.94:0/2801015148 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcf8000d6b0 con 0x5589024e12b0 2024-02-17T12:08:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.405+0000 7fcf92409740 10 client.27836 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.405+0000 7fcf92409740 10 client.27836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.405+0000 7fcf92409740 20 client.27836 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.405+0000 7fcf92409740 1 --2- 172.21.15.94:0/2801015148 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5589024faa90 0x5589024fceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.405+0000 7fcf92409740 4 client.27836 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:08:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.405+0000 7fcf92409740 1 -- 172.21.15.94:0/2801015148 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558902185720 con 0x5589024faa90 2024-02-17T12:08:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.405+0000 7fcf8ccf1640 1 --2- 172.21.15.94:0/2801015148 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5589024faa90 0x5589024fceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.406+0000 7fcf8ccf1640 1 --2- 172.21.15.94:0/2801015148 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5589024faa90 0x5589024fceb0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d5a31eae3d7e27 server_cookie=13683e1cf3eb20f1 in_seq=0 out_seq=0 2024-02-17T12:08:55.412 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.406+0000 7fcf70ff9640 10 client.27836 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:55.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.407+0000 7fcf70ff9640 1 -- 172.21.15.94:0/2801015148 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558902185720 con 0x5589024faa90 2024-02-17T12:08:55.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.407+0000 7fcf70ff9640 10 client.27836 handle_command_reply: tid=0 2024-02-17T12:08:55.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.407+0000 7fcf92409740 1 client.27836 shutdown 2024-02-17T12:08:55.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.407+0000 7fcf92409740 1 -- 172.21.15.94:0/2801015148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e12b0 msgr2=0x5589024ea120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.407+0000 7fcf92409740 1 --2- 172.21.15.94:0/2801015148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e12b0 0x5589024ea120 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000fb60 tx=0x7fcf8000c740 comp rx=0 tx=0).stop 2024-02-17T12:08:55.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.407+0000 7fcf92409740 1 -- 172.21.15.94:0/2801015148 shutdown_connections 2024-02-17T12:08:55.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.407+0000 7fcf92409740 1 --2- 172.21.15.94:0/2801015148 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5589024faa90 0x5589024fceb0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.407+0000 7fcf92409740 1 --2- 172.21.15.94:0/2801015148 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5589024ea7b0 0x5589024e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.407+0000 7fcf92409740 1 --2- 172.21.15.94:0/2801015148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e1bd0 0x5589024e74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.407+0000 7fcf92409740 1 --2- 172.21.15.94:0/2801015148 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e12b0 0x5589024ea120 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000fb60 tx=0x7fcf8000c740 comp rx=0 tx=0).stop 2024-02-17T12:08:55.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.407+0000 7fcf92409740 1 -- 172.21.15.94:0/2801015148 >> 172.21.15.94:0/2801015148 conn(0x5589024db990 msgr2=0x5589024dd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:55.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.408+0000 7fcf70ff9640 0 client.27836 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:55.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.408+0000 7fcf92409740 1 -- 172.21.15.94:0/2801015148 shutdown_connections 2024-02-17T12:08:55.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.408+0000 7fcf92409740 1 -- 172.21.15.94:0/2801015148 wait complete. 2024-02-17T12:08:55.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.408+0000 7fcf92409740 20 client.27836 trim_cache size 0 max 16384 2024-02-17T12:08:55.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:55 smithi125 ceph-mon[36819]: pgmap v335: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 32 MiB/s wr, 2.19k op/s 2024-02-17T12:08:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.436+0000 7fcf92409740 1 Processor -- start 2024-02-17T12:08:55.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.436+0000 7fcf92409740 1 -- start start 2024-02-17T12:08:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.436+0000 7fcf92409740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e1bd0 0x558902501830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.436+0000 7fcf92409740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024ea7b0 0x5589024e9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.436+0000 7fcf92409740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 0x5589024ea120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.436+0000 7fcf92409740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589024f6630 con 0x558902501d70 2024-02-17T12:08:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.436+0000 7fcf92409740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589024ed480 con 0x5589024ea7b0 2024-02-17T12:08:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.436+0000 7fcf92409740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589024e0c80 con 0x5589024e1bd0 2024-02-17T12:08:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8dcf3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 0x5589024ea120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8d4f2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e1bd0 0x558902501830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8ccf1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024ea7b0 0x5589024e9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8dcf3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 0x5589024ea120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55536/0 (socket says 172.21.15.94:55536) 2024-02-17T12:08:55.439 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8d4f2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e1bd0 0x558902501830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:45824/0 (socket says 172.21.15.94:45824) 2024-02-17T12:08:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8dcf3640 1 -- 172.21.15.94:0/93692241 learned_addr learned my addr 172.21.15.94:0/93692241 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8ccf1640 1 -- 172.21.15.94:0/93692241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e1bd0 msgr2=0x558902501830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8ccf1640 1 --2- 172.21.15.94:0/93692241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e1bd0 0x558902501830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8ccf1640 1 -- 172.21.15.94:0/93692241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 msgr2=0x5589024ea120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8ccf1640 1 --2- 172.21.15.94:0/93692241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 0x5589024ea120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8ccf1640 1 -- 172.21.15.94:0/93692241 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589023e3ac0 con 0x5589024ea7b0 2024-02-17T12:08:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8dcf3640 1 --2- 172.21.15.94:0/93692241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 0x5589024ea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8d4f2640 1 --2- 172.21.15.94:0/93692241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e1bd0 0x558902501830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf8ccf1640 1 --2- 172.21.15.94:0/93692241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024ea7b0 0x5589024e9be0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcf840ed250 tx=0x7fcf840107f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e08f0b389c1ae04f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf72ffd640 1 -- 172.21.15.94:0/93692241 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84011de0 con 0x5589024ea7b0 2024-02-17T12:08:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf72ffd640 1 -- 172.21.15.94:0/93692241 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf840bcbf0 con 0x5589024ea7b0 2024-02-17T12:08:55.440 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.437+0000 7fcf72ffd640 1 -- 172.21.15.94:0/93692241 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf840b8b60 con 0x5589024ea7b0 2024-02-17T12:08:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.438+0000 7fcf92409740 1 -- 172.21.15.94:0/93692241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024ea7b0 msgr2=0x5589024e9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.438+0000 7fcf92409740 1 --2- 172.21.15.94:0/93692241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024ea7b0 0x5589024e9be0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcf840ed250 tx=0x7fcf840107f0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.438+0000 7fcf92409740 1 -- 172.21.15.94:0/93692241 shutdown_connections 2024-02-17T12:08:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.438+0000 7fcf92409740 1 --2- 172.21.15.94:0/93692241 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 0x5589024ea120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.438+0000 7fcf92409740 1 --2- 172.21.15.94:0/93692241 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024ea7b0 0x5589024e9be0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.438+0000 7fcf92409740 1 --2- 172.21.15.94:0/93692241 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024e1bd0 0x558902501830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.438+0000 7fcf92409740 1 -- 172.21.15.94:0/93692241 >> 172.21.15.94:0/93692241 conn(0x5589024db990 msgr2=0x5589024dbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.438+0000 7fcf92409740 1 -- 172.21.15.94:0/93692241 shutdown_connections 2024-02-17T12:08:55.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.438+0000 7fcf92409740 1 -- 172.21.15.94:0/93692241 wait complete. 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.438+0000 7fcf92409740 1 -- start start 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.438+0000 7fcf92409740 1 Processor -- start 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf92409740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e1bd0 0x5589024ee4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf92409740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024ea7b0 0x5589024ee9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf92409740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 0x558902550fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf92409740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589024ef2b0 con 0x558902501d70 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf92409740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589025514f0 con 0x5589024e1bd0 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf92409740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558902551630 con 0x5589024ea7b0 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf92409740 10 client.0 _ll_register_callbacks cb 0x7fcf78037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf8d4f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e1bd0 0x5589024ee4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf8dcf3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 0x558902550fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf8ccf1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024ea7b0 0x5589024ee9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf8dcf3640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 0x558902550fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55542/0 (socket says 172.21.15.94:55542) 2024-02-17T12:08:55.445 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf8dcf3640 1 -- 172.21.15.94:0/3037288716 learned_addr learned my addr 172.21.15.94:0/3037288716 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf8d4f2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e1bd0 0x5589024ee4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:54634/0 (socket says 172.21.15.94:54634) 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf8d4f2640 1 -- 172.21.15.94:0/3037288716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024ea7b0 msgr2=0x5589024ee9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf8d4f2640 1 --2- 172.21.15.94:0/3037288716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024ea7b0 0x5589024ee9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf8d4f2640 1 -- 172.21.15.94:0/3037288716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 msgr2=0x558902550fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf8d4f2640 1 --2- 172.21.15.94:0/3037288716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 0x558902550fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf8d4f2640 1 -- 172.21.15.94:0/3037288716 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558902447610 con 0x5589024e1bd0 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf8dcf3640 1 --2- 172.21.15.94:0/3037288716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 0x558902550fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.439+0000 7fcf8ccf1640 1 --2- 172.21.15.94:0/3037288716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024ea7b0 0x5589024ee9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.440+0000 7fcf8d4f2640 1 --2- 172.21.15.94:0/3037288716 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e1bd0 0x5589024ee4a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000dac0 tx=0x7fcf8000d610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25dc11198e5da801 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.440+0000 7fcf70ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.440+0000 7fcf70ff9640 1 -- 172.21.15.94:0/3037288716 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80014070 con 0x5589024e1bd0 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.440+0000 7fcf92409740 1 -- 172.21.15.94:0/3037288716 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf78034440 con 0x5589024e1bd0 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.440+0000 7fcf70ff9640 1 -- 172.21.15.94:0/3037288716 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcf8000d800 con 0x5589024e1bd0 2024-02-17T12:08:55.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.440+0000 7fcf70ff9640 1 -- 172.21.15.94:0/3037288716 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf80005ce0 con 0x5589024e1bd0 2024-02-17T12:08:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.440+0000 7fcf92409740 10 client.27842 fetch_fsmap learned FSMap version 27 2024-02-17T12:08:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.440+0000 7fcf92409740 1 -- 172.21.15.94:0/3037288716 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558902407570 con 0x5589024e1bd0 2024-02-17T12:08:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.441+0000 7fcf70ff9640 1 -- 172.21.15.94:0/3037288716 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf80037080 con 0x5589024e1bd0 2024-02-17T12:08:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.441+0000 7fcf70ff9640 1 -- 172.21.15.94:0/3037288716 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcf80019700 con 0x5589024e1bd0 2024-02-17T12:08:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.441+0000 7fcf92409740 10 client.27842 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:08:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.441+0000 7fcf92409740 10 client.27842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:08:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.442+0000 7fcf92409740 20 client.27842 populate_metadata read hostname 'smithi094' 2024-02-17T12:08:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.442+0000 7fcf92409740 1 --2- 172.21.15.94:0/3037288716 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558902552c70 0x558902555090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:08:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.442+0000 7fcf92409740 4 client.27842 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:08:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.442+0000 7fcf92409740 1 -- 172.21.15.94:0/3037288716 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5589021d4890 con 0x558902552c70 2024-02-17T12:08:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.442+0000 7fcf8ccf1640 1 --2- 172.21.15.94:0/3037288716 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558902552c70 0x558902555090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:08:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf70ff9640 10 client.27842 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:55.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf8ccf1640 1 --2- 172.21.15.94:0/3037288716 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558902552c70 0x558902555090 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbf2e5671f9d1466 server_cookie=c2d900cf5e278130 in_seq=0 out_seq=0 2024-02-17T12:08:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf70ff9640 1 -- 172.21.15.94:0/3037288716 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5589021d4890 con 0x558902552c70 2024-02-17T12:08:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf70ff9640 10 client.27842 handle_command_reply: tid=0 2024-02-17T12:08:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf92409740 1 client.27842 shutdown 2024-02-17T12:08:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf92409740 1 -- 172.21.15.94:0/3037288716 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e1bd0 msgr2=0x5589024ee4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf92409740 1 --2- 172.21.15.94:0/3037288716 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e1bd0 0x5589024ee4a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000dac0 tx=0x7fcf8000d610 comp rx=0 tx=0).stop 2024-02-17T12:08:55.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf92409740 1 -- 172.21.15.94:0/3037288716 shutdown_connections 2024-02-17T12:08:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf92409740 1 --2- 172.21.15.94:0/3037288716 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558902552c70 0x558902555090 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf92409740 1 --2- 172.21.15.94:0/3037288716 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558902501d70 0x558902550fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf70ff9640 0 client.27842 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:08:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf92409740 1 --2- 172.21.15.94:0/3037288716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5589024ea7b0 0x5589024ee9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf92409740 1 --2- 172.21.15.94:0/3037288716 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5589024e1bd0 0x5589024ee4a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.443+0000 7fcf92409740 1 -- 172.21.15.94:0/3037288716 >> 172.21.15.94:0/3037288716 conn(0x5589024db990 msgr2=0x5589024e2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf92409740 1 -- 172.21.15.94:0/3037288716 shutdown_connections 2024-02-17T12:08:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf92409740 1 -- 172.21.15.94:0/3037288716 wait complete. 2024-02-17T12:08:55.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf92409740 20 client.27842 trim_cache size 0 max 16384 2024-02-17T12:08:55.450 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:08:55.451 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:08:55.451 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:08:55.451 INFO:teuthology.orchestra.run.smithi094.stdout: "f715a9d4-960d-409c-b3f9-b68728740e51": { 2024-02-17T12:08:55.451 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:08:55.451 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "f715a9d4-960d-409c-b3f9-b68728740e51", 2024-02-17T12:08:55.451 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:08:55.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:08:55.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:08:55.451 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:08:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf8f756640 1 -- 172.21.15.94:0/2320126255 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcf740a2440 msgr2=0x7fcf740a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf8f756640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcf740a2440 0x7fcf740a4900 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcf7802cd60 tx=0x7fcf7802ccf0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf8f756640 1 -- 172.21.15.94:0/2320126255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf8810fd10 msgr2=0x7fcf881142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:08:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf8f756640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf8810fd10 0x7fcf881142a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009fd0 tx=0x7fcf8400b870 comp rx=0 tx=0).stop 2024-02-17T12:08:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf8f756640 1 -- 172.21.15.94:0/2320126255 shutdown_connections 2024-02-17T12:08:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf8f756640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcf740a2440 0x7fcf740a4900 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf8f756640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf8811a960 0x7fcf88118620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf8f756640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf8810fd10 0x7fcf881142a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf8f756640 1 --2- 172.21.15.94:0/2320126255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf8810f8a0 0x7fcf88113d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:08:55.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf8f756640 1 -- 172.21.15.94:0/2320126255 >> 172.21.15.94:0/2320126255 conn(0x7fcf8806c270 msgr2=0x7fcf8811b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:08:55.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf8f756640 1 -- 172.21.15.94:0/2320126255 shutdown_connections 2024-02-17T12:08:55.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:08:55.444+0000 7fcf8f756640 1 -- 172.21.15.94:0/2320126255 wait complete. 2024-02-17T12:08:55.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:55 smithi094 ceph-mon[32499]: pgmap v335: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 32 MiB/s wr, 2.19k op/s 2024-02-17T12:08:55.459 INFO:tasks.cephfs.filesystem:scrub status for tag:f715a9d4-960d-409c-b3f9-b68728740e51 - {'path': '/', 'tag': 'f715a9d4-960d-409c-b3f9-b68728740e51', 'options': 'recursive,force'} 2024-02-17T12:08:56.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:56 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:08:56.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:56 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:08:56.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:56 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:08:56.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:56 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:08:56.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:56 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:08:56.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:56 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:08:56.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:56 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:08:56.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:56 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:08:56.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:56 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:08:56.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:56 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:08:56.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:56 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:08:56.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:56 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:08:57.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:57 smithi189 ceph-mon[36588]: pgmap v336: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 11 MiB/s wr, 852 op/s 2024-02-17T12:08:57.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:57 smithi125 ceph-mon[36819]: pgmap v336: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 11 MiB/s wr, 852 op/s 2024-02-17T12:08:57.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:57 smithi094 ceph-mon[32499]: pgmap v336: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 11 MiB/s wr, 852 op/s 2024-02-17T12:08:58.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:58 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:08:58.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:58 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:08:58.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:58 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:08:59.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:08:59 smithi189 ceph-mon[36588]: pgmap v337: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-02-17T12:08:59.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:08:59 smithi125 ceph-mon[36819]: pgmap v337: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-02-17T12:08:59.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:08:59 smithi094 ceph-mon[32499]: pgmap v337: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 14 MiB/s wr, 1.17k op/s 2024-02-17T12:09:00.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 640.0 s, 2303.7 tps, lat 42.859 ms stddev 57.983 2024-02-17T12:09:01.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:01 smithi189 ceph-mon[36588]: pgmap v338: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.2 MiB/s wr, 720 op/s 2024-02-17T12:09:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.418+0000 7f714740e640 1 -- 172.21.15.94:0/1868489268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f714010fd30 msgr2=0x7f714010da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:01.420 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.418+0000 7f714740e640 1 --2- 172.21.15.94:0/1868489268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f714010fd30 0x7f714010da20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f713c009900 tx=0x7f713c02bc80 comp rx=0 tx=0).stop 2024-02-17T12:09:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.418+0000 7f714740e640 1 -- 172.21.15.94:0/1868489268 shutdown_connections 2024-02-17T12:09:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.418+0000 7f714740e640 1 --2- 172.21.15.94:0/1868489268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714011a960 0x7f714011cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.418+0000 7f714740e640 1 --2- 172.21.15.94:0/1868489268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f714010fd30 0x7f714010da20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.418+0000 7f714740e640 1 --2- 172.21.15.94:0/1868489268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f714010f8c0 0x7f714010d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.421 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.418+0000 7f714740e640 1 -- 172.21.15.94:0/1868489268 >> 172.21.15.94:0/1868489268 conn(0x7f714006c220 msgr2=0x7f714006c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:01.422 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.419+0000 7f714740e640 1 -- 172.21.15.94:0/1868489268 shutdown_connections 2024-02-17T12:09:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.421+0000 7f714740e640 1 -- 172.21.15.94:0/1868489268 wait complete. 2024-02-17T12:09:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.422+0000 7f714740e640 1 Processor -- start 2024-02-17T12:09:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.426+0000 7f714740e640 1 -- start start 2024-02-17T12:09:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.426+0000 7f714740e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714010f8c0 0x7f7140117d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.426+0000 7f714740e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f714010fd30 0x7f71401182d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.426+0000 7f714740e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f714011a960 0x7f7140110eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.426+0000 7f714740e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7140111640 con 0x7f714010f8c0 2024-02-17T12:09:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.426+0000 7f714740e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71401117b0 con 0x7f714010fd30 2024-02-17T12:09:01.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.426+0000 7f714740e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7140111970 con 0x7f714011a960 2024-02-17T12:09:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.430+0000 7f71451aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714010f8c0 0x7f7140117d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.430+0000 7f71451aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714010f8c0 0x7f7140117d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55564/0 (socket says 172.21.15.94:55564) 2024-02-17T12:09:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.430+0000 7f71451aa640 1 -- 172.21.15.94:0/2684446604 learned_addr learned my addr 172.21.15.94:0/2684446604 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.430+0000 7f71459ab640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f714011a960 0x7f7140110eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.433 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.430+0000 7f71449a9640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f714010fd30 0x7f71401182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.434 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.432+0000 7f71451aa640 1 -- 172.21.15.94:0/2684446604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f714011a960 msgr2=0x7f7140110eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.432+0000 7f71451aa640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f714011a960 0x7f7140110eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.432+0000 7f71451aa640 1 -- 172.21.15.94:0/2684446604 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f714010fd30 msgr2=0x7f71401182d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.432+0000 7f71451aa640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f714010fd30 0x7f71401182d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.432+0000 7f71451aa640 1 -- 172.21.15.94:0/2684446604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713c009560 con 0x7f714010f8c0 2024-02-17T12:09:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.432+0000 7f71451aa640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714010f8c0 0x7f7140117d90 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f713000b450 tx=0x7f713000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.432+0000 7f71367fc640 1 -- 172.21.15.94:0/2684446604 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7130011890 con 0x7f714010f8c0 2024-02-17T12:09:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.433+0000 7f71367fc640 1 -- 172.21.15.94:0/2684446604 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f71300103e0 con 0x7f714010f8c0 2024-02-17T12:09:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.433+0000 7f71367fc640 1 -- 172.21.15.94:0/2684446604 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713000f5a0 con 0x7f714010f8c0 2024-02-17T12:09:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.433+0000 7f714740e640 1 -- 172.21.15.94:0/2684446604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7140111c50 con 0x7f714010f8c0 2024-02-17T12:09:01.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.433+0000 7f714740e640 1 -- 172.21.15.94:0/2684446604 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7140115bb0 con 0x7f714010f8c0 2024-02-17T12:09:01.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:01 smithi125 ceph-mon[36819]: pgmap v338: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.2 MiB/s wr, 720 op/s 2024-02-17T12:09:01.439 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.437+0000 7f7149fe5740 1 Processor -- start 2024-02-17T12:09:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.438+0000 7f7149fe5740 1 -- start start 2024-02-17T12:09:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.439+0000 7f7149fe5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x55855309e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.439+0000 7f7149fe5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x55855309f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.439+0000 7f7149fe5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585530a82e0 0x5585530aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.439+0000 7f7149fe5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855300d9f0 con 0x5585530a82e0 2024-02-17T12:09:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.439+0000 7f7149fe5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855300af20 con 0x55855309ed60 2024-02-17T12:09:01.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.439+0000 7f7149fe5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558553006700 con 0x55855309fad0 2024-02-17T12:09:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.442+0000 7f71367fc640 1 -- 172.21.15.94:0/2684446604 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7130010550 con 0x7f714010f8c0 2024-02-17T12:09:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.442+0000 7f71459ab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x55855309e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.442+0000 7f71459ab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x55855309e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:45864/0 (socket says 172.21.15.94:45864) 2024-02-17T12:09:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.442+0000 7f71459ab640 1 -- 172.21.15.94:0/2551287525 learned_addr learned my addr 172.21.15.94:0/2551287525 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.442+0000 7f71449a9640 1 --2- 172.21.15.94:0/2551287525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585530a82e0 0x5585530aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.443+0000 7f71459ab640 1 -- 172.21.15.94:0/2551287525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 msgr2=0x55855309f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.443+0000 7f71459ab640 1 --2- 172.21.15.94:0/2551287525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x55855309f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.443+0000 7f71459ab640 1 -- 172.21.15.94:0/2551287525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585530a82e0 msgr2=0x5585530aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:01.446 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.443+0000 7f71459ab640 1 --2- 172.21.15.94:0/2551287525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585530a82e0 0x5585530aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.443+0000 7f71459ab640 1 -- 172.21.15.94:0/2551287525 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558552f9c7b0 con 0x55855309fad0 2024-02-17T12:09:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.444+0000 7f71459ab640 1 --2- 172.21.15.94:0/2551287525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x55855309e820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7138009f90 tx=0x7f71380118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a33e2c9380cb11c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.444+0000 7f71367fc640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f712c0a24b0 0x7f712c0a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.444+0000 7f712b7fe640 1 -- 172.21.15.94:0/2551287525 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138004b20 con 0x55855309fad0 2024-02-17T12:09:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.444+0000 7f712b7fe640 1 -- 172.21.15.94:0/2551287525 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7138004c80 con 0x55855309fad0 2024-02-17T12:09:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.444+0000 7f71367fc640 1 -- 172.21.15.94:0/2684446604 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f71300cdc10 con 0x7f714010f8c0 2024-02-17T12:09:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.445+0000 7f7149fe5740 1 -- 172.21.15.94:0/2551287525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 msgr2=0x55855309e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:01.447 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.445+0000 7f7149fe5740 1 --2- 172.21.15.94:0/2551287525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x55855309e820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7138009f90 tx=0x7f71380118d0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.448+0000 7f71451aa640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f712c0a24b0 0x7f712c0a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.448+0000 7f7149fe5740 1 -- 172.21.15.94:0/2551287525 shutdown_connections 2024-02-17T12:09:01.450 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.448+0000 7f7149fe5740 1 --2- 172.21.15.94:0/2551287525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585530a82e0 0x5585530aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.448+0000 7f7149fe5740 1 --2- 172.21.15.94:0/2551287525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x55855309f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.448+0000 7f7149fe5740 1 --2- 172.21.15.94:0/2551287525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x55855309e820 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.448+0000 7f7149fe5740 1 -- 172.21.15.94:0/2551287525 >> 172.21.15.94:0/2551287525 conn(0x558553066550 msgr2=0x558553099a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:01.451 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.448+0000 7f71451aa640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f712c0a24b0 0x7f712c0a4970 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x5585530a0230 tx=0x7f7130010fe0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:01.453 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.452+0000 7f7149fe5740 1 -- 172.21.15.94:0/2551287525 shutdown_connections 2024-02-17T12:09:01.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:01 smithi094 ceph-mon[32499]: pgmap v338: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 9.2 MiB/s wr, 720 op/s 2024-02-17T12:09:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.456+0000 7f7149fe5740 1 -- 172.21.15.94:0/2551287525 wait complete. 2024-02-17T12:09:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.456+0000 7f7149fe5740 1 -- start start 2024-02-17T12:09:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.456+0000 7f7149fe5740 1 Processor -- start 2024-02-17T12:09:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.457+0000 7f7149fe5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x5585530a5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.457+0000 7f7149fe5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x5585530a5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.457+0000 7f7149fe5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585530a82e0 0x5585530a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.457+0000 7f7149fe5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558552fabbf0 con 0x5585530a82e0 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.457+0000 7f7149fe5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558552cda250 con 0x55855309ed60 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.457+0000 7f7149fe5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558552f8c200 con 0x55855309fad0 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.457+0000 7f7149fe5740 10 client.0 _ll_register_callbacks cb 0x558552ffad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.458+0000 7f71459ab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x5585530a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.458+0000 7f71459ab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x5585530a5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:54662/0 (socket says 172.21.15.94:54662) 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.458+0000 7f71459ab640 1 -- 172.21.15.94:0/12270852 learned_addr learned my addr 172.21.15.94:0/12270852 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.458+0000 7f71459ab640 1 --2- 172.21.15.94:0/12270852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585530a82e0 0x5585530a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.458+0000 7f71459ab640 1 -- 172.21.15.94:0/12270852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 msgr2=0x5585530a5750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.458+0000 7f71459ab640 1 --2- 172.21.15.94:0/12270852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x5585530a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.458+0000 7f71459ab640 1 -- 172.21.15.94:0/12270852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585530a82e0 msgr2=0x5585530a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.458+0000 7f71459ab640 1 --2- 172.21.15.94:0/12270852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585530a82e0 0x5585530a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.459+0000 7f71459ab640 1 -- 172.21.15.94:0/12270852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7138009c40 con 0x55855309ed60 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.460+0000 7f71459ab640 1 --2- 172.21.15.94:0/12270852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x5585530a5210 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7138004640 tx=0x7f71380044a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea840c4a5c796fba server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:01.463 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.460+0000 7f7128ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:09:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.464+0000 7f7128ff9640 1 -- 172.21.15.94:0/12270852 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71380040a0 con 0x55855309ed60 2024-02-17T12:09:01.465 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.464+0000 7f7149fe5740 1 -- 172.21.15.94:0/12270852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558552d06600 con 0x55855309ed60 2024-02-17T12:09:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.464+0000 7f7128ff9640 1 -- 172.21.15.94:0/12270852 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7138005240 con 0x55855309ed60 2024-02-17T12:09:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.464+0000 7f7128ff9640 1 -- 172.21.15.94:0/12270852 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7138005400 con 0x55855309ed60 2024-02-17T12:09:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.465+0000 7f7149fe5740 10 client.27851 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:01.466 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.465+0000 7f7149fe5740 1 -- 172.21.15.94:0/12270852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558553004d80 con 0x55855309ed60 2024-02-17T12:09:01.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.467+0000 7f7128ff9640 1 -- 172.21.15.94:0/12270852 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558552d06600 con 0x55855309ed60 2024-02-17T12:09:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.468+0000 7f7128ff9640 1 -- 172.21.15.94:0/12270852 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f713800fad0 con 0x55855309ed60 2024-02-17T12:09:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.468+0000 7f7149fe5740 10 client.27851 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.468+0000 7f7149fe5740 10 client.27851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.468+0000 7f7149fe5740 20 client.27851 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.468+0000 7f7149fe5740 1 --2- 172.21.15.94:0/12270852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5585530da640 0x5585530dca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.468+0000 7f7149fe5740 4 client.27851 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:01.470 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.468+0000 7f7149fe5740 1 -- 172.21.15.94:0/12270852 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558552d571b0 con 0x5585530da640 2024-02-17T12:09:01.472 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.470+0000 7f71449a9640 1 --2- 172.21.15.94:0/12270852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5585530da640 0x5585530dca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.490+0000 7f71449a9640 1 --2- 172.21.15.94:0/12270852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5585530da640 0x5585530dca60 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f3d34f4b9f341f server_cookie=a1467dac30b20a56 in_seq=0 out_seq=0 2024-02-17T12:09:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.491+0000 7f7128ff9640 10 client.27851 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.491+0000 7f7128ff9640 1 -- 172.21.15.94:0/12270852 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558552d571b0 con 0x5585530da640 2024-02-17T12:09:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.491+0000 7f7128ff9640 10 client.27851 handle_command_reply: tid=0 2024-02-17T12:09:01.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.491+0000 7f7149fe5740 1 client.27851 shutdown 2024-02-17T12:09:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.496+0000 7f7149fe5740 1 -- 172.21.15.94:0/12270852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 msgr2=0x5585530a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:01.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.496+0000 7f7149fe5740 1 --2- 172.21.15.94:0/12270852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x5585530a5210 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7138004640 tx=0x7f71380044a0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.499+0000 7f7149fe5740 1 -- 172.21.15.94:0/12270852 shutdown_connections 2024-02-17T12:09:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.499+0000 7f7149fe5740 1 --2- 172.21.15.94:0/12270852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5585530da640 0x5585530dca60 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.499+0000 7f7149fe5740 1 --2- 172.21.15.94:0/12270852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5585530a82e0 0x5585530a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.499+0000 7f7149fe5740 1 --2- 172.21.15.94:0/12270852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x5585530a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.499+0000 7f7149fe5740 1 --2- 172.21.15.94:0/12270852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x5585530a5210 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.500+0000 7f7149fe5740 1 -- 172.21.15.94:0/12270852 >> 172.21.15.94:0/12270852 conn(0x558553066550 msgr2=0x55855309b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:01.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.500+0000 7f7128ff9640 0 client.27851 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.503+0000 7f7149fe5740 1 -- 172.21.15.94:0/12270852 shutdown_connections 2024-02-17T12:09:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.505+0000 7f7149fe5740 1 -- 172.21.15.94:0/12270852 wait complete. 2024-02-17T12:09:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.506+0000 7f7149fe5740 20 client.27851 trim_cache size 0 max 16384 2024-02-17T12:09:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.532+0000 7f7149fe5740 1 Processor -- start 2024-02-17T12:09:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.532+0000 7f7149fe5740 1 -- start start 2024-02-17T12:09:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.532+0000 7f7149fe5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855309ed60 0x5585530a4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.533+0000 7f7149fe5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x5585530a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.533+0000 7f71459ab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855309ed60 0x5585530a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.533+0000 7f71459ab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855309ed60 0x5585530a4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:55598/0 (socket says 172.21.15.94:55598) 2024-02-17T12:09:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.533+0000 7f71459ab640 1 -- 172.21.15.94:0/598331196 learned_addr learned my addr 172.21.15.94:0/598331196 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.533+0000 7f7149fe5740 1 --2- 172.21.15.94:0/598331196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5585530a82e0 0x55855309b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.533+0000 7f7149fe5740 1 -- 172.21.15.94:0/598331196 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558553099310 con 0x55855309ed60 2024-02-17T12:09:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.533+0000 7f7149fe5740 1 -- 172.21.15.94:0/598331196 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558552d06600 con 0x5585530a82e0 2024-02-17T12:09:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.533+0000 7f7149fe5740 1 -- 172.21.15.94:0/598331196 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855309ea60 con 0x55855309fad0 2024-02-17T12:09:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.534+0000 7f71449a9640 1 --2- 172.21.15.94:0/598331196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x5585530a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.535+0000 7f71449a9640 1 -- 172.21.15.94:0/598331196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5585530a82e0 msgr2=0x55855309b540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:09:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.535+0000 7f71449a9640 1 --2- 172.21.15.94:0/598331196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5585530a82e0 0x55855309b540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.535+0000 7f71449a9640 1 -- 172.21.15.94:0/598331196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855309ed60 msgr2=0x5585530a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.535+0000 7f71449a9640 1 --2- 172.21.15.94:0/598331196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855309ed60 0x5585530a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.535+0000 7f71449a9640 1 -- 172.21.15.94:0/598331196 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558552fa17b0 con 0x55855309fad0 2024-02-17T12:09:01.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.538+0000 7f71449a9640 1 --2- 172.21.15.94:0/598331196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x5585530a5ab0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f713c02c1c0 tx=0x7f713c005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba1bb969fc097c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.539+0000 7f712affd640 1 -- 172.21.15.94:0/598331196 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c045070 con 0x55855309fad0 2024-02-17T12:09:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.539+0000 7f712affd640 1 -- 172.21.15.94:0/598331196 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f713c004df0 con 0x55855309fad0 2024-02-17T12:09:01.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.539+0000 7f712affd640 1 -- 172.21.15.94:0/598331196 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c005220 con 0x55855309fad0 2024-02-17T12:09:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.540+0000 7f7149fe5740 1 -- 172.21.15.94:0/598331196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 msgr2=0x5585530a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:01.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.540+0000 7f7149fe5740 1 --2- 172.21.15.94:0/598331196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x5585530a5ab0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f713c02c1c0 tx=0x7f713c005850 comp rx=0 tx=0).stop 2024-02-17T12:09:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.550+0000 7f7149fe5740 1 -- 172.21.15.94:0/598331196 shutdown_connections 2024-02-17T12:09:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.550+0000 7f7149fe5740 1 --2- 172.21.15.94:0/598331196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5585530a82e0 0x55855309b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.550+0000 7f7149fe5740 1 --2- 172.21.15.94:0/598331196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55855309fad0 0x5585530a5ab0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.550+0000 7f7149fe5740 1 --2- 172.21.15.94:0/598331196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855309ed60 0x5585530a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.550+0000 7f7149fe5740 1 -- 172.21.15.94:0/598331196 >> 172.21.15.94:0/598331196 conn(0x558553066550 msgr2=0x55855309ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:01.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.557+0000 7f7149fe5740 1 -- 172.21.15.94:0/598331196 shutdown_connections 2024-02-17T12:09:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.558+0000 7f7149fe5740 1 -- 172.21.15.94:0/598331196 wait complete. 2024-02-17T12:09:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.558+0000 7f7149fe5740 1 -- start start 2024-02-17T12:09:01.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.559+0000 7f7149fe5740 1 Processor -- start 2024-02-17T12:09:01.563 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.562+0000 7f7149fe5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x5585530adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.562+0000 7f7149fe5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855309fad0 0x5585530ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.562+0000 7f7149fe5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585530a82e0 0x558553130b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.562+0000 7f7149fe5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585530aedd0 con 0x55855309fad0 2024-02-17T12:09:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.562+0000 7f7149fe5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585531310b0 con 0x55855309ed60 2024-02-17T12:09:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.562+0000 7f7149fe5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585531311f0 con 0x5585530a82e0 2024-02-17T12:09:01.564 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.562+0000 7f7149fe5740 10 client.0 _ll_register_callbacks cb 0x7f7138009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.566+0000 7f71459ab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x5585530adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.566+0000 7f71459ab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585530a82e0 0x558553130b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.567 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.566+0000 7f71459ab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x5585530adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:54678/0 (socket says 172.21.15.94:54678) 2024-02-17T12:09:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.566+0000 7f71459ab640 1 -- 172.21.15.94:0/1617646729 learned_addr learned my addr 172.21.15.94:0/1617646729 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:01.568 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.567+0000 7f71449a9640 1 --2- 172.21.15.94:0/1617646729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855309fad0 0x5585530ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.570+0000 7f71459ab640 1 -- 172.21.15.94:0/1617646729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585530a82e0 msgr2=0x558553130b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.570+0000 7f71459ab640 1 --2- 172.21.15.94:0/1617646729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585530a82e0 0x558553130b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.570+0000 7f71459ab640 1 -- 172.21.15.94:0/1617646729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855309fad0 msgr2=0x5585530ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.570+0000 7f71459ab640 1 --2- 172.21.15.94:0/1617646729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855309fad0 0x5585530ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:01.572 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.570+0000 7f71459ab640 1 -- 172.21.15.94:0/1617646729 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714011d640 con 0x55855309ed60 2024-02-17T12:09:01.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.574+0000 7f71459ab640 1 --2- 172.21.15.94:0/1617646729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x5585530adfe0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7138009270 tx=0x7f7138000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e102e7ad877b73a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.575+0000 7f7128ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:09:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.575+0000 7f7128ff9640 1 -- 172.21.15.94:0/1617646729 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138004e70 con 0x55855309ed60 2024-02-17T12:09:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.575+0000 7f7128ff9640 1 -- 172.21.15.94:0/1617646729 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7138004a90 con 0x55855309ed60 2024-02-17T12:09:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.579+0000 7f7149fe5740 1 -- 172.21.15.94:0/1617646729 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7138004c80 con 0x55855309ed60 2024-02-17T12:09:01.588 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.584+0000 7f7128ff9640 1 -- 172.21.15.94:0/1617646729 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7138028850 con 0x55855309ed60 2024-02-17T12:09:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.588+0000 7f7149fe5740 10 client.27854 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:01.590 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.588+0000 7f7149fe5740 1 -- 172.21.15.94:0/1617646729 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558552fc7540 con 0x55855309ed60 2024-02-17T12:09:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.590+0000 7f7128ff9640 1 -- 172.21.15.94:0/1617646729 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7138004c80 con 0x55855309ed60 2024-02-17T12:09:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.590+0000 7f7128ff9640 1 -- 172.21.15.94:0/1617646729 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f713800f930 con 0x55855309ed60 2024-02-17T12:09:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.592+0000 7f7149fe5740 10 client.27854 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.592+0000 7f7149fe5740 10 client.27854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.592+0000 7f7149fe5740 20 client.27854 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:01.594 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.592+0000 7f7149fe5740 1 --2- 172.21.15.94:0/1617646729 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5585531325f0 0x5585531349e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.592+0000 7f7149fe5740 4 client.27854 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:09:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.592+0000 7f7149fe5740 1 -- 172.21.15.94:0/1617646729 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558552d43550 con 0x5585531325f0 2024-02-17T12:09:01.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.593+0000 7f71449a9640 1 --2- 172.21.15.94:0/1617646729 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5585531325f0 0x5585531349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.602+0000 7f71449a9640 1 --2- 172.21.15.94:0/1617646729 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5585531325f0 0x5585531349e0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac4a8f92f5087769 server_cookie=e3df14fd863d85a7 in_seq=0 out_seq=0 2024-02-17T12:09:01.605 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:01.602+0000 7f7128ff9640 10 client.27854 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.071+0000 7f7128ff9640 1 -- 172.21.15.94:0/1617646729 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558552d43550 con 0x5585531325f0 2024-02-17T12:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.071+0000 7f7128ff9640 10 client.27854 handle_command_reply: tid=0 2024-02-17T12:09:02.073 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.071+0000 7f7149fe5740 1 client.27854 shutdown 2024-02-17T12:09:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.071+0000 7f7149fe5740 1 -- 172.21.15.94:0/1617646729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 msgr2=0x5585530adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.071+0000 7f7149fe5740 1 --2- 172.21.15.94:0/1617646729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x5585530adfe0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f7138009270 tx=0x7f7138000f40 comp rx=0 tx=0).stop 2024-02-17T12:09:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.071+0000 7f7149fe5740 1 -- 172.21.15.94:0/1617646729 shutdown_connections 2024-02-17T12:09:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.071+0000 7f7149fe5740 1 --2- 172.21.15.94:0/1617646729 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5585531325f0 0x5585531349e0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.071+0000 7f7149fe5740 1 --2- 172.21.15.94:0/1617646729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585530a82e0 0x558553130b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.071+0000 7f7149fe5740 1 --2- 172.21.15.94:0/1617646729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55855309fad0 0x5585530ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.071+0000 7f7149fe5740 1 --2- 172.21.15.94:0/1617646729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55855309ed60 0x5585530adfe0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.071+0000 7f7149fe5740 1 -- 172.21.15.94:0/1617646729 >> 172.21.15.94:0/1617646729 conn(0x558553066550 msgr2=0x558553098b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.071+0000 7f7128ff9640 0 client.27854 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.072+0000 7f7149fe5740 1 -- 172.21.15.94:0/1617646729 shutdown_connections 2024-02-17T12:09:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.072+0000 7f7149fe5740 1 -- 172.21.15.94:0/1617646729 wait complete. 2024-02-17T12:09:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.072+0000 7f7149fe5740 20 client.27854 trim_cache size 0 max 16384 2024-02-17T12:09:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:09:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:09:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:09:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.073+0000 7f714740e640 1 -- 172.21.15.94:0/2684446604 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f712c0a24b0 msgr2=0x7f712c0a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.073+0000 7f714740e640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f712c0a24b0 0x7f712c0a4970 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x5585530a0230 tx=0x7f7130010fe0 comp rx=0 tx=0).stop 2024-02-17T12:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.073+0000 7f714740e640 1 -- 172.21.15.94:0/2684446604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714010f8c0 msgr2=0x7f7140117d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.073+0000 7f714740e640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714010f8c0 0x7f7140117d90 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f713000b450 tx=0x7f713000b910 comp rx=0 tx=0).stop 2024-02-17T12:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.073+0000 7f714740e640 1 -- 172.21.15.94:0/2684446604 shutdown_connections 2024-02-17T12:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.073+0000 7f714740e640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f712c0a24b0 0x7f712c0a4970 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.073+0000 7f714740e640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f714011a960 0x7f7140110eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.073+0000 7f714740e640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f714010fd30 0x7f71401182d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.073+0000 7f714740e640 1 --2- 172.21.15.94:0/2684446604 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f714010f8c0 0x7f7140117d90 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.073+0000 7f714740e640 1 -- 172.21.15.94:0/2684446604 >> 172.21.15.94:0/2684446604 conn(0x7f714006c220 msgr2=0x7f7140119440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.073+0000 7f714740e640 1 -- 172.21.15.94:0/2684446604 shutdown_connections 2024-02-17T12:09:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:02.073+0000 7f714740e640 1 -- 172.21.15.94:0/2684446604 wait complete. 2024-02-17T12:09:03.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:03 smithi125 ceph-mon[36819]: osdmap e97: 12 total, 12 up, 12 in 2024-02-17T12:09:03.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:03 smithi125 ceph-mon[36819]: pgmap v340: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 9.5 MiB/s wr, 761 op/s 2024-02-17T12:09:03.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:03 smithi094 ceph-mon[32499]: osdmap e97: 12 total, 12 up, 12 in 2024-02-17T12:09:03.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:03 smithi094 ceph-mon[32499]: pgmap v340: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 9.5 MiB/s wr, 761 op/s 2024-02-17T12:09:03.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:03 smithi189 ceph-mon[36588]: osdmap e97: 12 total, 12 up, 12 in 2024-02-17T12:09:03.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:03 smithi189 ceph-mon[36588]: pgmap v340: 97 pgs: 97 active+clean; 5.7 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 9.5 MiB/s wr, 761 op/s 2024-02-17T12:09:05.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 645.0 s, 1047.8 tps, lat 96.157 ms stddev 172.067 2024-02-17T12:09:05.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:05 smithi189 ceph-mon[36588]: pgmap v341: 97 pgs: 20 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 66 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 20 MiB/s wr, 1.84k op/s 2024-02-17T12:09:05.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:05 smithi125 ceph-mon[36819]: pgmap v341: 97 pgs: 20 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 66 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 20 MiB/s wr, 1.84k op/s 2024-02-17T12:09:05.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:05 smithi094 ceph-mon[32499]: pgmap v341: 97 pgs: 20 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 66 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 20 MiB/s wr, 1.84k op/s 2024-02-17T12:09:07.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:07 smithi189 ceph-mon[36588]: pgmap v342: 97 pgs: 20 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 66 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 20 MiB/s wr, 1.84k op/s 2024-02-17T12:09:07.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:07 smithi125 ceph-mon[36819]: pgmap v342: 97 pgs: 20 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 66 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 20 MiB/s wr, 1.84k op/s 2024-02-17T12:09:07.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:07 smithi094 ceph-mon[32499]: pgmap v342: 97 pgs: 20 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 66 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 20 MiB/s wr, 1.84k op/s 2024-02-17T12:09:09.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:09 smithi189 ceph-mon[36588]: pgmap v343: 97 pgs: 97 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 19 MiB/s wr, 1.71k op/s 2024-02-17T12:09:09.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:09 smithi125 ceph-mon[36819]: pgmap v343: 97 pgs: 97 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 19 MiB/s wr, 1.71k op/s 2024-02-17T12:09:09.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:09 smithi094 ceph-mon[32499]: pgmap v343: 97 pgs: 97 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 19 MiB/s wr, 1.71k op/s 2024-02-17T12:09:10.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 650.0 s, 1789.2 tps, lat 55.785 ms stddev 63.448 2024-02-17T12:09:11.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:11 smithi189 ceph-mon[36588]: osdmap e98: 12 total, 12 up, 12 in 2024-02-17T12:09:11.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:11 smithi189 ceph-mon[36588]: pgmap v345: 97 pgs: 97 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 23 MiB/s wr, 2.01k op/s 2024-02-17T12:09:11.686 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:11 smithi125 ceph-mon[36819]: osdmap e98: 12 total, 12 up, 12 in 2024-02-17T12:09:11.686 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:11 smithi125 ceph-mon[36819]: pgmap v345: 97 pgs: 97 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 23 MiB/s wr, 2.01k op/s 2024-02-17T12:09:11.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:11 smithi094 ceph-mon[32499]: osdmap e98: 12 total, 12 up, 12 in 2024-02-17T12:09:11.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:11 smithi094 ceph-mon[32499]: pgmap v345: 97 pgs: 97 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 23 MiB/s wr, 2.01k op/s 2024-02-17T12:09:13.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:13 smithi189 ceph-mon[36588]: pgmap v346: 97 pgs: 97 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-02-17T12:09:13.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:13 smithi125 ceph-mon[36819]: pgmap v346: 97 pgs: 97 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-02-17T12:09:13.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:13 smithi094 ceph-mon[32499]: pgmap v346: 97 pgs: 97 active+clean; 5.3 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 19 MiB/s wr, 1.72k op/s 2024-02-17T12:09:15.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.200+0000 7f324f93e640 1 -- 172.21.15.94:0/1875736696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3248106500 msgr2=0x7f324810e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.200+0000 7f324f93e640 1 --2- 172.21.15.94:0/1875736696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3248106500 0x7f324810e060 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3240009ca0 tx=0x7f324002c100 comp rx=0 tx=0).stop 2024-02-17T12:09:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.200+0000 7f324f93e640 1 -- 172.21.15.94:0/1875736696 shutdown_connections 2024-02-17T12:09:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.200+0000 7f324f93e640 1 --2- 172.21.15.94:0/1875736696 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3248106500 0x7f324810e060 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.200+0000 7f324f93e640 1 --2- 172.21.15.94:0/1875736696 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f32481002a0 0x7f3248105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.200+0000 7f324f93e640 1 --2- 172.21.15.94:0/1875736696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32480ff8f0 0x7f32480ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.200+0000 7f324f93e640 1 -- 172.21.15.94:0/1875736696 >> 172.21.15.94:0/1875736696 conn(0x7f3248076610 msgr2=0x7f32480fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.200+0000 7f324f93e640 1 -- 172.21.15.94:0/1875736696 shutdown_connections 2024-02-17T12:09:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.200+0000 7f324f93e640 1 -- 172.21.15.94:0/1875736696 wait complete. 2024-02-17T12:09:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.201+0000 7f324f93e640 1 Processor -- start 2024-02-17T12:09:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.201+0000 7f324f93e640 1 -- start start 2024-02-17T12:09:15.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.201+0000 7f324f93e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32480ff8f0 0x7f3248197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.201+0000 7f324f93e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3248106500 0x7f3248197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.201+0000 7f324f93e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f324819bee0 0x7f324819c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.201+0000 7f324f93e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324819c8d0 con 0x7f3248106500 2024-02-17T12:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.201+0000 7f324f93e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324819ca40 con 0x7f324819bee0 2024-02-17T12:09:15.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.201+0000 7f324f93e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324819cbb0 con 0x7f32480ff8f0 2024-02-17T12:09:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.201+0000 7f324dedb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f324819bee0 0x7f324819c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.201+0000 7f324dedb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f324819bee0 0x7f324819c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44236/0 (socket says 172.21.15.94:44236) 2024-02-17T12:09:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.201+0000 7f324dedb640 1 -- 172.21.15.94:0/2673380571 learned_addr learned my addr 172.21.15.94:0/2673380571 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.201+0000 7f324d6da640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32480ff8f0 0x7f3248197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.202+0000 7f324ced9640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3248106500 0x7f3248197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.202+0000 7f324dedb640 1 -- 172.21.15.94:0/2673380571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32480ff8f0 msgr2=0x7f3248197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:15.205 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.202+0000 7f324dedb640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32480ff8f0 0x7f3248197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.202+0000 7f324dedb640 1 -- 172.21.15.94:0/2673380571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3248106500 msgr2=0x7f3248197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.202+0000 7f324dedb640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3248106500 0x7f3248197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.202+0000 7f324dedb640 1 -- 172.21.15.94:0/2673380571 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3240009900 con 0x7f324819bee0 2024-02-17T12:09:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.202+0000 7f324dedb640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f324819bee0 0x7f324819c390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3240037040 tx=0x7f3240032570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.203+0000 7f323e7fc640 1 -- 172.21.15.94:0/2673380571 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3240034640 con 0x7f324819bee0 2024-02-17T12:09:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.203+0000 7f323e7fc640 1 -- 172.21.15.94:0/2673380571 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3240034e00 con 0x7f324819bee0 2024-02-17T12:09:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.203+0000 7f324f93e640 1 -- 172.21.15.94:0/2673380571 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3248100790 con 0x7f324819bee0 2024-02-17T12:09:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.203+0000 7f324f93e640 1 -- 172.21.15.94:0/2673380571 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3248100c40 con 0x7f324819bee0 2024-02-17T12:09:15.206 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.203+0000 7f323e7fc640 1 -- 172.21.15.94:0/2673380571 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32400047a0 con 0x7f324819bee0 2024-02-17T12:09:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.205+0000 7f323e7fc640 1 -- 172.21.15.94:0/2673380571 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f324000a040 con 0x7f324819bee0 2024-02-17T12:09:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.206+0000 7f32525ee740 1 Processor -- start 2024-02-17T12:09:15.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.206+0000 7f32525ee740 1 -- start start 2024-02-17T12:09:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.206+0000 7f32525ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb779ad0 0x5588cb778820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.206+0000 7f32525ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb778d60 0x5588cb7791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.206+0000 7f32525ee740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb7822e0 0x5588cb7846d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.206+0000 7f32525ee740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cb6e79f0 con 0x5588cb778d60 2024-02-17T12:09:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.206+0000 7f32525ee740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cb6e4f20 con 0x5588cb7822e0 2024-02-17T12:09:15.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.206+0000 7f32525ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cb6e0700 con 0x5588cb779ad0 2024-02-17T12:09:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f324ced9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb779ad0 0x5588cb778820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f324d6da640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb778d60 0x5588cb7791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f324ced9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb7822e0 0x5588cb7846d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f324d6da640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb778d60 0x5588cb7791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:49024/0 (socket says 172.21.15.94:49024) 2024-02-17T12:09:15.209 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f324d6da640 1 -- 172.21.15.94:0/2042216959 learned_addr learned my addr 172.21.15.94:0/2042216959 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f324ced9640 1 -- 172.21.15.94:0/2042216959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb779ad0 msgr2=0x5588cb778820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:15.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f324ced9640 1 --2- 172.21.15.94:0/2042216959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb779ad0 0x5588cb778820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f324ced9640 1 -- 172.21.15.94:0/2042216959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb778d60 msgr2=0x5588cb7791e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f324ced9640 1 --2- 172.21.15.94:0/2042216959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb778d60 0x5588cb7791e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f324ced9640 1 -- 172.21.15.94:0/2042216959 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588cb6767b0 con 0x5588cb7822e0 2024-02-17T12:09:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f323e7fc640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f32340a24b0 0x7f32340a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f324d6da640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f32340a24b0 0x7f32340a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f323e7fc640 1 -- 172.21.15.94:0/2673380571 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f32400f0a70 con 0x7f324819bee0 2024-02-17T12:09:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.207+0000 7f324ced9640 1 --2- 172.21.15.94:0/2042216959 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb7822e0 0x5588cb7846d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f32480680f0 tx=0x7f3238004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f720faa74ff35e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.208+0000 7f32337fe640 1 -- 172.21.15.94:0/2042216959 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32380095e0 con 0x5588cb7822e0 2024-02-17T12:09:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.208+0000 7f32337fe640 1 -- 172.21.15.94:0/2042216959 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f323800f920 con 0x5588cb7822e0 2024-02-17T12:09:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.208+0000 7f32337fe640 1 -- 172.21.15.94:0/2042216959 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323800ea10 con 0x5588cb7822e0 2024-02-17T12:09:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.208+0000 7f324d6da640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f32340a24b0 0x7f32340a4970 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x5588cb77a230 tx=0x7f3244005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:15.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.209+0000 7f32525ee740 1 -- 172.21.15.94:0/2042216959 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb7822e0 msgr2=0x5588cb7846d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.209+0000 7f32525ee740 1 --2- 172.21.15.94:0/2042216959 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb7822e0 0x5588cb7846d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f32480680f0 tx=0x7f3238004730 comp rx=0 tx=0).stop 2024-02-17T12:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.209+0000 7f32525ee740 1 -- 172.21.15.94:0/2042216959 shutdown_connections 2024-02-17T12:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.209+0000 7f32525ee740 1 --2- 172.21.15.94:0/2042216959 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb7822e0 0x5588cb7846d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.209+0000 7f32525ee740 1 --2- 172.21.15.94:0/2042216959 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb778d60 0x5588cb7791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.209+0000 7f32525ee740 1 --2- 172.21.15.94:0/2042216959 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb779ad0 0x5588cb778820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.209+0000 7f32525ee740 1 -- 172.21.15.94:0/2042216959 >> 172.21.15.94:0/2042216959 conn(0x5588cb740550 msgr2=0x5588cb773a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.209+0000 7f32525ee740 1 -- 172.21.15.94:0/2042216959 shutdown_connections 2024-02-17T12:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.209+0000 7f32525ee740 1 -- 172.21.15.94:0/2042216959 wait complete. 2024-02-17T12:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.209+0000 7f32525ee740 1 -- start start 2024-02-17T12:09:15.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.209+0000 7f32525ee740 1 Processor -- start 2024-02-17T12:09:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f32525ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 0x5588cb77f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f32525ee740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 0x5588cb77f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f32525ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb7822e0 0x5588cb77fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f32525ee740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cb685bf0 con 0x5588cb7822e0 2024-02-17T12:09:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f32525ee740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cb3b4250 con 0x5588cb779ad0 2024-02-17T12:09:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f32525ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cb666200 con 0x5588cb778d60 2024-02-17T12:09:15.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f32525ee740 10 client.0 _ll_register_callbacks cb 0x5588cb6d4d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f324d6da640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 0x5588cb77f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f324d6da640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 0x5588cb77f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:41560/0 (socket says 172.21.15.94:41560) 2024-02-17T12:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f324d6da640 1 -- 172.21.15.94:0/2164918444 learned_addr learned my addr 172.21.15.94:0/2164918444 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f324dedb640 1 --2- 172.21.15.94:0/2164918444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb7822e0 0x5588cb77fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f324d6da640 1 -- 172.21.15.94:0/2164918444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 msgr2=0x5588cb77f840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f324d6da640 1 --2- 172.21.15.94:0/2164918444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 0x5588cb77f840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f324d6da640 1 -- 172.21.15.94:0/2164918444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb7822e0 msgr2=0x5588cb77fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f324d6da640 1 --2- 172.21.15.94:0/2164918444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb7822e0 0x5588cb77fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.210+0000 7f324d6da640 1 -- 172.21.15.94:0/2164918444 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323801b040 con 0x5588cb778d60 2024-02-17T12:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.211+0000 7f324d6da640 1 --2- 172.21.15.94:0/2164918444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 0x5588cb77f300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f324401e9d0 tx=0x7f3244004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6408b82ae0c4aaa3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.211+0000 7f3230ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.211+0000 7f3230ff9640 1 -- 172.21.15.94:0/2164918444 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244023070 con 0x5588cb778d60 2024-02-17T12:09:15.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.211+0000 7f32525ee740 1 -- 172.21.15.94:0/2164918444 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588cb3e0600 con 0x5588cb778d60 2024-02-17T12:09:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.211+0000 7f3230ff9640 1 -- 172.21.15.94:0/2164918444 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3244029420 con 0x5588cb778d60 2024-02-17T12:09:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.211+0000 7f3230ff9640 1 -- 172.21.15.94:0/2164918444 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3244017070 con 0x5588cb778d60 2024-02-17T12:09:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.211+0000 7f32525ee740 10 client.27844 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.211+0000 7f32525ee740 1 -- 172.21.15.94:0/2164918444 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5588cb6ded80 con 0x5588cb778d60 2024-02-17T12:09:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.212+0000 7f3230ff9640 1 -- 172.21.15.94:0/2164918444 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5588cb3e0600 con 0x5588cb778d60 2024-02-17T12:09:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.212+0000 7f3230ff9640 1 -- 172.21.15.94:0/2164918444 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f324401c600 con 0x5588cb778d60 2024-02-17T12:09:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.212+0000 7f32525ee740 10 client.27844 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:15.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.212+0000 7f32525ee740 10 client.27844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.212+0000 7f32525ee740 20 client.27844 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.212+0000 7f32525ee740 1 --2- 172.21.15.94:0/2164918444 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5588cb792800 0x5588cb794c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.212+0000 7f32525ee740 4 client.27844 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.212+0000 7f32525ee740 1 -- 172.21.15.94:0/2164918444 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588cb4311b0 con 0x5588cb792800 2024-02-17T12:09:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.212+0000 7f324ced9640 1 --2- 172.21.15.94:0/2164918444 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5588cb792800 0x5588cb794c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.212+0000 7f324ced9640 1 --2- 172.21.15.94:0/2164918444 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5588cb792800 0x5588cb794c40 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38148aa1b8e1b5a7 server_cookie=db8a1feda447767b in_seq=0 out_seq=0 2024-02-17T12:09:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.212+0000 7f3230ff9640 10 client.27844 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f3230ff9640 1 -- 172.21.15.94:0/2164918444 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588cb4311b0 con 0x5588cb792800 2024-02-17T12:09:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f3230ff9640 10 client.27844 handle_command_reply: tid=0 2024-02-17T12:09:15.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f32525ee740 1 client.27844 shutdown 2024-02-17T12:09:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f32525ee740 1 -- 172.21.15.94:0/2164918444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 msgr2=0x5588cb77f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f32525ee740 1 --2- 172.21.15.94:0/2164918444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 0x5588cb77f300 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f324401e9d0 tx=0x7f3244004730 comp rx=0 tx=0).stop 2024-02-17T12:09:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f32525ee740 1 -- 172.21.15.94:0/2164918444 shutdown_connections 2024-02-17T12:09:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f32525ee740 1 --2- 172.21.15.94:0/2164918444 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5588cb792800 0x5588cb794c40 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f32525ee740 1 --2- 172.21.15.94:0/2164918444 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb7822e0 0x5588cb77fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f3230ff9640 0 client.27844 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f32525ee740 1 --2- 172.21.15.94:0/2164918444 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 0x5588cb77f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f32525ee740 1 --2- 172.21.15.94:0/2164918444 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 0x5588cb77f300 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f32525ee740 1 -- 172.21.15.94:0/2164918444 >> 172.21.15.94:0/2164918444 conn(0x5588cb740550 msgr2=0x5588cb7757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f32525ee740 1 -- 172.21.15.94:0/2164918444 shutdown_connections 2024-02-17T12:09:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.213+0000 7f32525ee740 1 -- 172.21.15.94:0/2164918444 wait complete. 2024-02-17T12:09:15.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.214+0000 7f32525ee740 20 client.27844 trim_cache size 0 max 16384 2024-02-17T12:09:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.233+0000 7f32525ee740 1 Processor -- start 2024-02-17T12:09:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.233+0000 7f32525ee740 1 -- start start 2024-02-17T12:09:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.233+0000 7f32525ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 0x5588cb781a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.233+0000 7f32525ee740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 0x5588cb77c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.235 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.233+0000 7f32525ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb7822e0 0x5588cb77cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.233+0000 7f32525ee740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cb77af50 con 0x5588cb7822e0 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.233+0000 7f32525ee740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cb784ef0 con 0x5588cb779ad0 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.233+0000 7f32525ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cb790f40 con 0x5588cb778d60 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.233+0000 7f324d6da640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 0x5588cb781a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.233+0000 7f324d6da640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 0x5588cb781a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:41574/0 (socket says 172.21.15.94:41574) 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.233+0000 7f324d6da640 1 -- 172.21.15.94:0/247718140 learned_addr learned my addr 172.21.15.94:0/247718140 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.233+0000 7f324ced9640 1 --2- 172.21.15.94:0/247718140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 0x5588cb77c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.234+0000 7f324dedb640 1 --2- 172.21.15.94:0/247718140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb7822e0 0x5588cb77cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.234+0000 7f324d6da640 1 -- 172.21.15.94:0/247718140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 msgr2=0x5588cb77c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.234+0000 7f324d6da640 1 --2- 172.21.15.94:0/247718140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 0x5588cb77c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.234+0000 7f324d6da640 1 -- 172.21.15.94:0/247718140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb7822e0 msgr2=0x5588cb77cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.234+0000 7f324d6da640 1 --2- 172.21.15.94:0/247718140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb7822e0 0x5588cb77cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.234+0000 7f324d6da640 1 -- 172.21.15.94:0/247718140 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588cb67b7b0 con 0x5588cb778d60 2024-02-17T12:09:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.234+0000 7f324dedb640 1 --2- 172.21.15.94:0/247718140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb7822e0 0x5588cb77cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:09:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.234+0000 7f324d6da640 1 --2- 172.21.15.94:0/247718140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 0x5588cb781a30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f324401ceb0 tx=0x7f324401cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f14a2cc6345e0cdf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.234+0000 7f3232ffd640 1 -- 172.21.15.94:0/247718140 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3244023070 con 0x5588cb778d60 2024-02-17T12:09:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.234+0000 7f3232ffd640 1 -- 172.21.15.94:0/247718140 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3244009b50 con 0x5588cb778d60 2024-02-17T12:09:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.234+0000 7f3232ffd640 1 -- 172.21.15.94:0/247718140 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32440319a0 con 0x5588cb778d60 2024-02-17T12:09:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.236+0000 7f32525ee740 1 -- 172.21.15.94:0/247718140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 msgr2=0x5588cb781a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.236+0000 7f32525ee740 1 --2- 172.21.15.94:0/247718140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 0x5588cb781a30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f324401ceb0 tx=0x7f324401cee0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.236+0000 7f32525ee740 1 -- 172.21.15.94:0/247718140 shutdown_connections 2024-02-17T12:09:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.236+0000 7f32525ee740 1 --2- 172.21.15.94:0/247718140 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb7822e0 0x5588cb77cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.236+0000 7f32525ee740 1 --2- 172.21.15.94:0/247718140 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 0x5588cb77c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.236+0000 7f32525ee740 1 --2- 172.21.15.94:0/247718140 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb778d60 0x5588cb781a30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.236+0000 7f32525ee740 1 -- 172.21.15.94:0/247718140 >> 172.21.15.94:0/247718140 conn(0x5588cb740550 msgr2=0x5588cb77dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.236+0000 7f32525ee740 1 -- 172.21.15.94:0/247718140 shutdown_connections 2024-02-17T12:09:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.236+0000 7f32525ee740 1 -- 172.21.15.94:0/247718140 wait complete. 2024-02-17T12:09:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.237+0000 7f32525ee740 1 -- start start 2024-02-17T12:09:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.237+0000 7f32525ee740 1 Processor -- start 2024-02-17T12:09:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.237+0000 7f32525ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb778d60 0x5588cb787f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f32525ee740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 0x5588cb788450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f32525ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb7822e0 0x5588cb7f1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f324ced9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 0x5588cb788450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f324ced9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 0x5588cb788450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44256/0 (socket says 172.21.15.94:44256) 2024-02-17T12:09:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f324d6da640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb778d60 0x5588cb787f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f324ced9640 1 -- 172.21.15.94:0/928144852 learned_addr learned my addr 172.21.15.94:0/928144852 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f32525ee740 1 -- 172.21.15.94:0/928144852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cb788d30 con 0x5588cb778d60 2024-02-17T12:09:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f32525ee740 1 -- 172.21.15.94:0/928144852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cb7f1a80 con 0x5588cb779ad0 2024-02-17T12:09:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f32525ee740 1 -- 172.21.15.94:0/928144852 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cb7f1bc0 con 0x5588cb7822e0 2024-02-17T12:09:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f32525ee740 10 client.0 _ll_register_callbacks cb 0x7f3238009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f324dedb640 1 --2- 172.21.15.94:0/928144852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb7822e0 0x5588cb7f1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f324ced9640 1 -- 172.21.15.94:0/928144852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb7822e0 msgr2=0x5588cb7f1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f324ced9640 1 --2- 172.21.15.94:0/928144852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb7822e0 0x5588cb7f1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f324ced9640 1 -- 172.21.15.94:0/928144852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb778d60 msgr2=0x5588cb787f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f324ced9640 1 --2- 172.21.15.94:0/928144852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb778d60 0x5588cb787f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f324ced9640 1 -- 172.21.15.94:0/928144852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588cb6ded80 con 0x5588cb779ad0 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.238+0000 7f324d6da640 1 --2- 172.21.15.94:0/928144852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb778d60 0x5588cb787f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f32525ee740 1 -- 172.21.15.94:0/928144852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f323800f920 con 0x5588cb779ad0 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f324ced9640 1 --2- 172.21.15.94:0/928144852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 0x5588cb788450 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f323800e660 tx=0x7f3238005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae349c77474920ed server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f3230ff9640 10 client.27869 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f3230ff9640 1 -- 172.21.15.94:0/928144852 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238009bc0 con 0x5588cb779ad0 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f3230ff9640 1 -- 172.21.15.94:0/928144852 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3238000e40 con 0x5588cb779ad0 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f3230ff9640 1 -- 172.21.15.94:0/928144852 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f323800f5e0 con 0x5588cb779ad0 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f32525ee740 10 client.27869 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f32525ee740 1 -- 172.21.15.94:0/928144852 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x5588cb6a1540 con 0x5588cb779ad0 2024-02-17T12:09:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f3230ff9640 1 -- 172.21.15.94:0/928144852 <== mon.2 v2:172.21.15.125:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f323800fa90 con 0x5588cb779ad0 2024-02-17T12:09:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f32525ee740 10 client.27869 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f32525ee740 10 client.27869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f32525ee740 20 client.27869 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f32525ee740 1 --2- 172.21.15.94:0/928144852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5588cb7f2fa0 0x5588cb7f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f32525ee740 4 client.27869 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:09:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.239+0000 7f32525ee740 1 -- 172.21.15.94:0/928144852 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5588cb41d550 con 0x5588cb7f2fa0 2024-02-17T12:09:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.240+0000 7f324d6da640 1 --2- 172.21.15.94:0/928144852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5588cb7f2fa0 0x5588cb7f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.240+0000 7f3230ff9640 10 client.27869 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:15.244 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.240+0000 7f324d6da640 1 --2- 172.21.15.94:0/928144852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5588cb7f2fa0 0x5588cb7f53c0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f30d7f522a17f86 server_cookie=177494fdeff16522 in_seq=0 out_seq=0 2024-02-17T12:09:15.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 655.0 s, 1737.4 tps, lat 57.984 ms stddev 64.705 2024-02-17T12:09:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.311+0000 7f3230ff9640 1 -- 172.21.15.94:0/928144852 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5588cb41d550 con 0x5588cb7f2fa0 2024-02-17T12:09:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.311+0000 7f3230ff9640 10 client.27869 handle_command_reply: tid=0 2024-02-17T12:09:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.311+0000 7f32525ee740 1 client.27869 shutdown 2024-02-17T12:09:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.312+0000 7f32525ee740 1 -- 172.21.15.94:0/928144852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 msgr2=0x5588cb788450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.312+0000 7f32525ee740 1 --2- 172.21.15.94:0/928144852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 0x5588cb788450 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f323800e660 tx=0x7f3238005850 comp rx=0 tx=0).stop 2024-02-17T12:09:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.312+0000 7f32525ee740 1 -- 172.21.15.94:0/928144852 shutdown_connections 2024-02-17T12:09:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.312+0000 7f32525ee740 1 --2- 172.21.15.94:0/928144852 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5588cb7f2fa0 0x5588cb7f53c0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.312+0000 7f32525ee740 1 --2- 172.21.15.94:0/928144852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5588cb7822e0 0x5588cb7f1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.312+0000 7f32525ee740 1 --2- 172.21.15.94:0/928144852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5588cb779ad0 0x5588cb788450 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.312+0000 7f32525ee740 1 --2- 172.21.15.94:0/928144852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5588cb778d60 0x5588cb787f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.312+0000 7f32525ee740 1 -- 172.21.15.94:0/928144852 >> 172.21.15.94:0/928144852 conn(0x5588cb740550 msgr2=0x5588cb772300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.313+0000 7f3230ff9640 0 client.27869 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.313+0000 7f32525ee740 1 -- 172.21.15.94:0/928144852 shutdown_connections 2024-02-17T12:09:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.313+0000 7f32525ee740 1 -- 172.21.15.94:0/928144852 wait complete. 2024-02-17T12:09:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.313+0000 7f32525ee740 20 client.27869 trim_cache size 0 max 16384 2024-02-17T12:09:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:09:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:09:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:09:15.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:09:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.313+0000 7f324f93e640 1 -- 172.21.15.94:0/2673380571 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f32340a24b0 msgr2=0x7f32340a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.313+0000 7f324f93e640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f32340a24b0 0x7f32340a4970 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x5588cb77a230 tx=0x7f3244005f90 comp rx=0 tx=0).stop 2024-02-17T12:09:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.313+0000 7f324f93e640 1 -- 172.21.15.94:0/2673380571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f324819bee0 msgr2=0x7f324819c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.313+0000 7f324f93e640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f324819bee0 0x7f324819c390 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f3240037040 tx=0x7f3240032570 comp rx=0 tx=0).stop 2024-02-17T12:09:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.313+0000 7f324f93e640 1 -- 172.21.15.94:0/2673380571 shutdown_connections 2024-02-17T12:09:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.313+0000 7f324f93e640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f32340a24b0 0x7f32340a4970 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.314+0000 7f324f93e640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f324819bee0 0x7f324819c390 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.314+0000 7f324f93e640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3248106500 0x7f3248197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.314+0000 7f324f93e640 1 --2- 172.21.15.94:0/2673380571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32480ff8f0 0x7f3248197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.314+0000 7f324f93e640 1 -- 172.21.15.94:0/2673380571 >> 172.21.15.94:0/2673380571 conn(0x7f3248076610 msgr2=0x7f3248104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.314+0000 7f324f93e640 1 -- 172.21.15.94:0/2673380571 shutdown_connections 2024-02-17T12:09:15.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:15.314+0000 7f324f93e640 1 -- 172.21.15.94:0/2673380571 wait complete. 2024-02-17T12:09:15.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:15 smithi189 ceph-mon[36588]: pgmap v347: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.8 MiB/s wr, 619 op/s 2024-02-17T12:09:15.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:15 smithi125 ceph-mon[36819]: pgmap v347: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.8 MiB/s wr, 619 op/s 2024-02-17T12:09:15.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:15 smithi094 ceph-mon[32499]: pgmap v347: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.8 MiB/s wr, 619 op/s 2024-02-17T12:09:17.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:17 smithi189 ceph-mon[36588]: pgmap v348: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.8 MiB/s wr, 619 op/s 2024-02-17T12:09:17.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:17 smithi125 ceph-mon[36819]: pgmap v348: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.8 MiB/s wr, 619 op/s 2024-02-17T12:09:17.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:17 smithi094 ceph-mon[32499]: pgmap v348: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.8 MiB/s wr, 619 op/s 2024-02-17T12:09:19.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:19 smithi189 ceph-mon[36588]: pgmap v349: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.0 MiB/s wr, 716 op/s 2024-02-17T12:09:19.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:19 smithi125 ceph-mon[36819]: pgmap v349: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.0 MiB/s wr, 716 op/s 2024-02-17T12:09:19.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:19 smithi094 ceph-mon[32499]: pgmap v349: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 9.0 MiB/s wr, 716 op/s 2024-02-17T12:09:20.244 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 660.0 s, 1978.4 tps, lat 50.181 ms stddev 61.446 2024-02-17T12:09:21.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:21 smithi189 ceph-mon[36588]: pgmap v350: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 632 op/s 2024-02-17T12:09:21.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:21 smithi125 ceph-mon[36819]: pgmap v350: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 632 op/s 2024-02-17T12:09:21.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:21 smithi094 ceph-mon[32499]: pgmap v350: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 7.3 MiB/s wr, 632 op/s 2024-02-17T12:09:23.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:23 smithi189 ceph-mon[36588]: pgmap v351: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.5 MiB/s wr, 562 op/s 2024-02-17T12:09:23.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:23 smithi125 ceph-mon[36819]: pgmap v351: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.5 MiB/s wr, 562 op/s 2024-02-17T12:09:23.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:23 smithi094 ceph-mon[32499]: pgmap v351: 97 pgs: 97 active+clean; 5.4 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 6.5 MiB/s wr, 562 op/s 2024-02-17T12:09:25.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 665.0 s, 2119.8 tps, lat 47.344 ms stddev 55.779 2024-02-17T12:09:25.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:25 smithi189 ceph-mon[36588]: pgmap v352: 97 pgs: 97 active+clean; 5.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 10 MiB/s wr, 827 op/s 2024-02-17T12:09:25.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:25 smithi125 ceph-mon[36819]: pgmap v352: 97 pgs: 97 active+clean; 5.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 10 MiB/s wr, 827 op/s 2024-02-17T12:09:25.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:25 smithi094 ceph-mon[32499]: pgmap v352: 97 pgs: 97 active+clean; 5.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 10 MiB/s wr, 827 op/s 2024-02-17T12:09:25.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:09:25.460 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:09:25.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.603+0000 7f2d8e715640 1 -- 172.21.15.94:0/1593901491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d8810dce0 msgr2=0x7f2d8811c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:25.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.603+0000 7f2d8e715640 1 --2- 172.21.15.94:0/1593901491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d8810dce0 0x7f2d8811c970 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009930 tx=0x7f2d7002bc60 comp rx=0 tx=0).stop 2024-02-17T12:09:25.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.603+0000 7f2d8e715640 1 -- 172.21.15.94:0/1593901491 shutdown_connections 2024-02-17T12:09:25.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.604+0000 7f2d8e715640 1 --2- 172.21.15.94:0/1593901491 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d8810dce0 0x7f2d8811c970 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.604+0000 7f2d8e715640 1 --2- 172.21.15.94:0/1593901491 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8810d260 0x7f2d8810d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.604+0000 7f2d8e715640 1 --2- 172.21.15.94:0/1593901491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8810f6a0 0x7f2d8810fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.604+0000 7f2d8e715640 1 -- 172.21.15.94:0/1593901491 >> 172.21.15.94:0/1593901491 conn(0x7f2d8806c030 msgr2=0x7f2d8806c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:25.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.604+0000 7f2d8e715640 1 -- 172.21.15.94:0/1593901491 shutdown_connections 2024-02-17T12:09:25.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.604+0000 7f2d8e715640 1 -- 172.21.15.94:0/1593901491 wait complete. 2024-02-17T12:09:25.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.605+0000 7f2d8e715640 1 Processor -- start 2024-02-17T12:09:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.605+0000 7f2d8e715640 1 -- start start 2024-02-17T12:09:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.605+0000 7f2d8e715640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d8810d260 0x7f2d8811c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.605+0000 7f2d8e715640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8810dce0 0x7f2d8811c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.605+0000 7f2d8e715640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8810f6a0 0x7f2d88116690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.605+0000 7f2d8e715640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d88116d90 con 0x7f2d8810dce0 2024-02-17T12:09:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.605+0000 7f2d8e715640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d88116f00 con 0x7f2d8810d260 2024-02-17T12:09:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.605+0000 7f2d8e715640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d881170c0 con 0x7f2d8810f6a0 2024-02-17T12:09:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.606+0000 7f2d8df14640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8810f6a0 0x7f2d88116690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.606+0000 7f2d8df14640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8810f6a0 0x7f2d88116690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:50674/0 (socket says 172.21.15.94:50674) 2024-02-17T12:09:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.606+0000 7f2d8df14640 1 -- 172.21.15.94:0/3454154826 learned_addr learned my addr 172.21.15.94:0/3454154826 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.606+0000 7f2d8df14640 1 -- 172.21.15.94:0/3454154826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d8810d260 msgr2=0x7f2d8811c3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:09:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.606+0000 7f2d8df14640 1 --2- 172.21.15.94:0/3454154826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d8810d260 0x7f2d8811c3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.606+0000 7f2d8df14640 1 -- 172.21.15.94:0/3454154826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8810dce0 msgr2=0x7f2d8811c930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:09:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.606+0000 7f2d8df14640 1 --2- 172.21.15.94:0/3454154826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8810dce0 0x7f2d8811c930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.606+0000 7f2d8df14640 1 -- 172.21.15.94:0/3454154826 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d70009560 con 0x7f2d8810f6a0 2024-02-17T12:09:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.607+0000 7f2d8df14640 1 --2- 172.21.15.94:0/3454154826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8810f6a0 0x7f2d88116690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2d7002c1f0 tx=0x7f2d7002eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.607+0000 7f2d867fc640 1 -- 172.21.15.94:0/3454154826 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7002e070 con 0x7f2d8810f6a0 2024-02-17T12:09:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.607+0000 7f2d867fc640 1 -- 172.21.15.94:0/3454154826 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d7002e1d0 con 0x7f2d8810f6a0 2024-02-17T12:09:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.607+0000 7f2d867fc640 1 -- 172.21.15.94:0/3454154826 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7002e070 con 0x7f2d8810f6a0 2024-02-17T12:09:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.607+0000 7f2d8e715640 1 -- 172.21.15.94:0/3454154826 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d881bb370 con 0x7f2d8810f6a0 2024-02-17T12:09:25.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.608+0000 7f2d8e715640 1 -- 172.21.15.94:0/3454154826 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d881bb6e0 con 0x7f2d8810f6a0 2024-02-17T12:09:25.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.611+0000 7f2d867fc640 1 -- 172.21.15.94:0/3454154826 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2d7002c600 con 0x7f2d8810f6a0 2024-02-17T12:09:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.611+0000 7f2d867fc640 1 --2- 172.21.15.94:0/3454154826 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2d740a2330 0x7f2d740a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.611+0000 7f2d92624740 1 Processor -- start 2024-02-17T12:09:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.611+0000 7f2d92624740 1 -- start start 2024-02-17T12:09:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.611+0000 7f2d867fc640 1 -- 172.21.15.94:0/3454154826 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2d700efb80 con 0x7f2d8810f6a0 2024-02-17T12:09:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.611+0000 7f2d8d713640 1 --2- 172.21.15.94:0/3454154826 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2d740a2330 0x7f2d740a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d92624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e521fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d92624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e521fbdf0 0x564e52204490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d8d713640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e521fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d92624740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e522049d0 0x564e52206dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d8d713640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e521fb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32862/0 (socket says 172.21.15.94:32862) 2024-02-17T12:09:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d92624740 1 -- 172.21.15.94:0/1654329147 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5216a280 con 0x564e521fb4d0 2024-02-17T12:09:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d8d713640 1 -- 172.21.15.94:0/1654329147 learned_addr learned my addr 172.21.15.94:0/1654329147 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d92624740 1 -- 172.21.15.94:0/1654329147 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e521677b0 con 0x564e522049d0 2024-02-17T12:09:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d92624740 1 -- 172.21.15.94:0/1654329147 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e52162f90 con 0x564e521fbdf0 2024-02-17T12:09:25.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d8cf12640 1 --2- 172.21.15.94:0/1654329147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e521fbdf0 0x564e52204490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d8d713640 1 --2- 172.21.15.94:0/3454154826 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2d740a2330 0x7f2d740a47f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2d78007fd0 tx=0x7f2d780138e0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d8cf12640 1 -- 172.21.15.94:0/1654329147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e522049d0 msgr2=0x564e52206dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d8cf12640 1 --2- 172.21.15.94:0/1654329147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e522049d0 0x564e52206dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d8cf12640 1 -- 172.21.15.94:0/1654329147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 msgr2=0x564e521fb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d8cf12640 1 --2- 172.21.15.94:0/1654329147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e521fb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d8cf12640 1 -- 172.21.15.94:0/1654329147 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e52121570 con 0x564e521fbdf0 2024-02-17T12:09:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.612+0000 7f2d8cf12640 1 --2- 172.21.15.94:0/1654329147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e521fbdf0 0x564e52204490 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2d88117cb0 tx=0x7f2d8000adc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b56dc24d9c2a9a31 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.613+0000 7f2d7f7fe640 1 -- 172.21.15.94:0/1654329147 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8000cd00 con 0x564e521fbdf0 2024-02-17T12:09:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.613+0000 7f2d7f7fe640 1 -- 172.21.15.94:0/1654329147 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d8000ce60 con 0x564e521fbdf0 2024-02-17T12:09:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.613+0000 7f2d7f7fe640 1 -- 172.21.15.94:0/1654329147 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d80018830 con 0x564e521fbdf0 2024-02-17T12:09:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.614+0000 7f2d92624740 1 -- 172.21.15.94:0/1654329147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e521fbdf0 msgr2=0x564e52204490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.614+0000 7f2d92624740 1 --2- 172.21.15.94:0/1654329147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e521fbdf0 0x564e52204490 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2d88117cb0 tx=0x7f2d8000adc0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.614+0000 7f2d92624740 1 -- 172.21.15.94:0/1654329147 shutdown_connections 2024-02-17T12:09:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.614+0000 7f2d92624740 1 --2- 172.21.15.94:0/1654329147 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e522049d0 0x564e52206dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.614+0000 7f2d92624740 1 --2- 172.21.15.94:0/1654329147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e521fbdf0 0x564e52204490 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.614+0000 7f2d92624740 1 --2- 172.21.15.94:0/1654329147 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e521fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.614+0000 7f2d92624740 1 -- 172.21.15.94:0/1654329147 >> 172.21.15.94:0/1654329147 conn(0x564e521f5c00 msgr2=0x564e521f5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.614+0000 7f2d92624740 1 -- 172.21.15.94:0/1654329147 shutdown_connections 2024-02-17T12:09:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.614+0000 7f2d92624740 1 -- 172.21.15.94:0/1654329147 wait complete. 2024-02-17T12:09:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.614+0000 7f2d92624740 1 -- start start 2024-02-17T12:09:25.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.614+0000 7f2d92624740 1 Processor -- start 2024-02-17T12:09:25.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d92624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e52204470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d92624740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 0x564e52201800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d92624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 0x564e52201ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d92624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e51e36430 con 0x564e521fb4d0 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d92624740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e51df2e00 con 0x564e521fbdf0 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d92624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e520e7fd0 con 0x564e522049d0 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d92624740 10 client.0 _ll_register_callbacks cb 0x564e5213dfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d8d713640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e52204470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d8d713640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e52204470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32876/0 (socket says 172.21.15.94:32876) 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d8d713640 1 -- 172.21.15.94:0/3488284053 learned_addr learned my addr 172.21.15.94:0/3488284053 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d8cf12640 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 0x564e52201800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d8df14640 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 0x564e52201ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d8d713640 1 -- 172.21.15.94:0/3488284053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 msgr2=0x564e52201ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d8d713640 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 0x564e52201ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d8d713640 1 -- 172.21.15.94:0/3488284053 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 msgr2=0x564e52201800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d8d713640 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 0x564e52201800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d8d713640 1 -- 172.21.15.94:0/3488284053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8000a590 con 0x564e521fb4d0 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d8df14640 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 0x564e52201ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d8d713640 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e52204470 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2d78010fd0 tx=0x7f2d78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3334cda5d2801bcf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d7cff9640 1 -- 172.21.15.94:0/3488284053 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78004d20 con 0x564e521fb4d0 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d7cff9640 1 -- 172.21.15.94:0/3488284053 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2d78003970 con 0x564e521fb4d0 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.615+0000 7f2d92624740 1 -- 172.21.15.94:0/3488284053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e51e627e0 con 0x564e521fb4d0 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d7cff9640 1 -- 172.21.15.94:0/3488284053 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d78004e80 con 0x564e521fb4d0 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d92624740 10 client.18405 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d92624740 1 -- 172.21.15.94:0/3488284053 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564e52161610 con 0x564e521fb4d0 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d7cff9640 1 -- 172.21.15.94:0/3488284053 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d78031020 con 0x564e521fb4d0 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d7cff9640 1 -- 172.21.15.94:0/3488284053 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2d78005140 con 0x564e521fb4d0 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d92624740 10 client.18405 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d92624740 10 client.18405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:25.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d92624740 20 client.18405 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d92624740 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564e52214cb0 0x564e522170d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d92624740 4 client.18405 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d92624740 1 -- 172.21.15.94:0/3488284053 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e51e9f720 con 0x564e52214cb0 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d8cf12640 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564e52214cb0 0x564e522170d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d7cff9640 10 client.18405 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.616+0000 7f2d8cf12640 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564e52214cb0 0x564e522170d0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54dbf06b139b9d92 server_cookie=aaeb2220ba713b67 in_seq=0 out_seq=0 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d7cff9640 1 -- 172.21.15.94:0/3488284053 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564e51e9f720 con 0x564e52214cb0 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d7cff9640 10 client.18405 handle_command_reply: tid=0 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d92624740 1 client.18405 shutdown 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d92624740 1 -- 172.21.15.94:0/3488284053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 msgr2=0x564e52204470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d92624740 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e52204470 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f2d78010fd0 tx=0x7f2d78004a40 comp rx=0 tx=0).stop 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d92624740 1 -- 172.21.15.94:0/3488284053 shutdown_connections 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d92624740 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564e52214cb0 0x564e522170d0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d92624740 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 0x564e52201ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d7cff9640 0 client.18405 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:25.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d92624740 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 0x564e52201800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d92624740 1 --2- 172.21.15.94:0/3488284053 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e52204470 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d92624740 1 -- 172.21.15.94:0/3488284053 >> 172.21.15.94:0/3488284053 conn(0x564e521f5c00 msgr2=0x564e521f7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:25.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d92624740 1 -- 172.21.15.94:0/3488284053 shutdown_connections 2024-02-17T12:09:25.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.617+0000 7f2d92624740 1 -- 172.21.15.94:0/3488284053 wait complete. 2024-02-17T12:09:25.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.618+0000 7f2d92624740 20 client.18405 trim_cache size 0 max 16384 2024-02-17T12:09:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d92624740 1 Processor -- start 2024-02-17T12:09:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d92624740 1 -- start start 2024-02-17T12:09:25.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d92624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e52201ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d92624740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 0x564e5220b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d92624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 0x564e5220be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d92624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e52202c60 con 0x564e521fb4d0 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d92624740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e521fb170 con 0x564e521fbdf0 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d92624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e522027c0 con 0x564e522049d0 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d8d713640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e52201ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d8d713640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e52201ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32884/0 (socket says 172.21.15.94:32884) 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d8d713640 1 -- 172.21.15.94:0/3238502837 learned_addr learned my addr 172.21.15.94:0/3238502837 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d8df14640 1 --2- 172.21.15.94:0/3238502837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 0x564e5220be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d8cf12640 1 --2- 172.21.15.94:0/3238502837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 0x564e5220b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d8d713640 1 -- 172.21.15.94:0/3238502837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 msgr2=0x564e5220be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d8d713640 1 --2- 172.21.15.94:0/3238502837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 0x564e5220be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d8d713640 1 -- 172.21.15.94:0/3238502837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 msgr2=0x564e5220b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:25.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d8d713640 1 --2- 172.21.15.94:0/3238502837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 0x564e5220b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.633+0000 7f2d8d713640 1 -- 172.21.15.94:0/3238502837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e520fdac0 con 0x564e521fb4d0 2024-02-17T12:09:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.634+0000 7f2d8d713640 1 --2- 172.21.15.94:0/3238502837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e52201ed0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2d78013de0 tx=0x7f2d78015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59812b71789305b7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.634+0000 7f2d7effd640 1 -- 172.21.15.94:0/3238502837 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78007d70 con 0x564e521fb4d0 2024-02-17T12:09:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.634+0000 7f2d7effd640 1 -- 172.21.15.94:0/3238502837 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d78031070 con 0x564e521fb4d0 2024-02-17T12:09:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.634+0000 7f2d7effd640 1 -- 172.21.15.94:0/3238502837 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78005280 con 0x564e521fb4d0 2024-02-17T12:09:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.634+0000 7f2d92624740 1 -- 172.21.15.94:0/3238502837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 msgr2=0x564e52201ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.634+0000 7f2d92624740 1 --2- 172.21.15.94:0/3238502837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e52201ed0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2d78013de0 tx=0x7f2d78015040 comp rx=0 tx=0).stop 2024-02-17T12:09:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.634+0000 7f2d92624740 1 -- 172.21.15.94:0/3238502837 shutdown_connections 2024-02-17T12:09:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.634+0000 7f2d92624740 1 --2- 172.21.15.94:0/3238502837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 0x564e5220be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.634+0000 7f2d92624740 1 --2- 172.21.15.94:0/3238502837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 0x564e5220b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.634+0000 7f2d92624740 1 --2- 172.21.15.94:0/3238502837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e52201ed0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.634+0000 7f2d92624740 1 -- 172.21.15.94:0/3238502837 >> 172.21.15.94:0/3238502837 conn(0x564e521f5c00 msgr2=0x564e521fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:25.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.634+0000 7f2d92624740 1 -- 172.21.15.94:0/3238502837 shutdown_connections 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.634+0000 7f2d92624740 1 -- 172.21.15.94:0/3238502837 wait complete. 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d92624740 1 -- start start 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d92624740 1 Processor -- start 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d92624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e5220ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d92624740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 0x564e5220b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d92624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 0x564e522652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d92624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e52265820 con 0x564e521fb4d0 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d92624740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e52265990 con 0x564e521fbdf0 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d92624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e52265b50 con 0x564e522049d0 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d92624740 10 client.0 _ll_register_callbacks cb 0x7f2d80005ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d8d713640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e5220ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d8d713640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e5220ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32900/0 (socket says 172.21.15.94:32900) 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d8d713640 1 -- 172.21.15.94:0/2725705719 learned_addr learned my addr 172.21.15.94:0/2725705719 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:25.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d8df14640 1 --2- 172.21.15.94:0/2725705719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 0x564e522652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.635+0000 7f2d8cf12640 1 --2- 172.21.15.94:0/2725705719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 0x564e5220b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d8d713640 1 -- 172.21.15.94:0/2725705719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 msgr2=0x564e522652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d8d713640 1 --2- 172.21.15.94:0/2725705719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 0x564e522652e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d8d713640 1 -- 172.21.15.94:0/2725705719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 msgr2=0x564e5220b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d8d713640 1 --2- 172.21.15.94:0/2725705719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 0x564e5220b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d8d713640 1 -- 172.21.15.94:0/2725705719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564e52161610 con 0x564e521fb4d0 2024-02-17T12:09:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d8d713640 1 --2- 172.21.15.94:0/2725705719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e5220ac60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2d78005b40 tx=0x7f2d78007640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caaea69e75c2a5b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d7cff9640 1 -- 172.21.15.94:0/2725705719 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78007c40 con 0x564e521fb4d0 2024-02-17T12:09:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d7cff9640 1 -- 172.21.15.94:0/2725705719 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2d78032080 con 0x564e521fb4d0 2024-02-17T12:09:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d92624740 1 -- 172.21.15.94:0/2725705719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d8000ce60 con 0x564e521fb4d0 2024-02-17T12:09:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d7cff9640 1 -- 172.21.15.94:0/2725705719 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d780052d0 con 0x564e521fb4d0 2024-02-17T12:09:25.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d92624740 10 client.18411 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d92624740 1 -- 172.21.15.94:0/2725705719 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564e521236a0 con 0x564e521fb4d0 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d7cff9640 1 -- 172.21.15.94:0/2725705719 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2d8000ce60 con 0x564e521fb4d0 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d7cff9640 1 -- 172.21.15.94:0/2725705719 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2d7803d030 con 0x564e521fb4d0 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d92624740 10 client.18411 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d92624740 10 client.18411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d92624740 20 client.18411 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d92624740 1 --2- 172.21.15.94:0/2725705719 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564e52214cb0 0x564e5226f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d92624740 4 client.18411 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.636+0000 7f2d92624740 1 -- 172.21.15.94:0/2725705719 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e51eee890 con 0x564e52214cb0 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d8cf12640 1 --2- 172.21.15.94:0/2725705719 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564e52214cb0 0x564e5226f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d8cf12640 1 --2- 172.21.15.94:0/2725705719 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564e52214cb0 0x564e5226f080 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c8fec6f07be0bc server_cookie=757d9381b8c9ea57 in_seq=0 out_seq=0 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d7cff9640 10 client.18411 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d7cff9640 1 -- 172.21.15.94:0/2725705719 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x564e51eee890 con 0x564e52214cb0 2024-02-17T12:09:25.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d7cff9640 10 client.18411 handle_command_reply: tid=0 2024-02-17T12:09:25.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d92624740 1 client.18411 shutdown 2024-02-17T12:09:25.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d92624740 1 -- 172.21.15.94:0/2725705719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 msgr2=0x564e5220ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:25.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d92624740 1 --2- 172.21.15.94:0/2725705719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e5220ac60 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2d78005b40 tx=0x7f2d78007640 comp rx=0 tx=0).stop 2024-02-17T12:09:25.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d92624740 1 -- 172.21.15.94:0/2725705719 shutdown_connections 2024-02-17T12:09:25.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d92624740 1 --2- 172.21.15.94:0/2725705719 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x564e52214cb0 0x564e5226f080 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d92624740 1 --2- 172.21.15.94:0/2725705719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x564e522049d0 0x564e522652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d92624740 1 --2- 172.21.15.94:0/2725705719 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x564e521fbdf0 0x564e5220b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d7cff9640 0 client.18411 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:25.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d92624740 1 --2- 172.21.15.94:0/2725705719 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x564e521fb4d0 0x564e5220ac60 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.637+0000 7f2d92624740 1 -- 172.21.15.94:0/2725705719 >> 172.21.15.94:0/2725705719 conn(0x564e521f5c00 msgr2=0x564e52201030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:25.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d92624740 1 -- 172.21.15.94:0/2725705719 shutdown_connections 2024-02-17T12:09:25.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d92624740 1 -- 172.21.15.94:0/2725705719 wait complete. 2024-02-17T12:09:25.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d92624740 20 client.18411 trim_cache size 0 max 16384 2024-02-17T12:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:09:25.647 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:09:25.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d8e715640 1 -- 172.21.15.94:0/3454154826 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2d740a2330 msgr2=0x7f2d740a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:25.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d8e715640 1 --2- 172.21.15.94:0/3454154826 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2d740a2330 0x7f2d740a47f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2d78007fd0 tx=0x7f2d780138e0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d8e715640 1 -- 172.21.15.94:0/3454154826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8810f6a0 msgr2=0x7f2d88116690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:25.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d8e715640 1 --2- 172.21.15.94:0/3454154826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8810f6a0 0x7f2d88116690 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2d7002c1f0 tx=0x7f2d7002eb80 comp rx=0 tx=0).stop 2024-02-17T12:09:25.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d8e715640 1 -- 172.21.15.94:0/3454154826 shutdown_connections 2024-02-17T12:09:25.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d8e715640 1 --2- 172.21.15.94:0/3454154826 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2d740a2330 0x7f2d740a47f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d8e715640 1 --2- 172.21.15.94:0/3454154826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2d8810f6a0 0x7f2d88116690 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d8e715640 1 --2- 172.21.15.94:0/3454154826 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2d8810dce0 0x7f2d8811c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d8e715640 1 --2- 172.21.15.94:0/3454154826 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2d8810d260 0x7f2d8811c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d8e715640 1 -- 172.21.15.94:0/3454154826 >> 172.21.15.94:0/3454154826 conn(0x7f2d8806c030 msgr2=0x7f2d880706b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:25.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d8e715640 1 -- 172.21.15.94:0/3454154826 shutdown_connections 2024-02-17T12:09:25.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.638+0000 7f2d8e715640 1 -- 172.21.15.94:0/3454154826 wait complete. 2024-02-17T12:09:25.651 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:09:25.651 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:09:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.768+0000 7f6de5ed4640 1 -- 172.21.15.94:0/2004347616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6de0105380 msgr2=0x7f6de010cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:25.772 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.768+0000 7f6de5ed4640 1 --2- 172.21.15.94:0/2004347616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6de0105380 0x7f6de010cf50 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6dd8009cd0 tx=0x7f6dd802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.768+0000 7f6de5ed4640 1 -- 172.21.15.94:0/2004347616 shutdown_connections 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.768+0000 7f6de5ed4640 1 --2- 172.21.15.94:0/2004347616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6de0105380 0x7f6de010cf50 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.768+0000 7f6de5ed4640 1 --2- 172.21.15.94:0/2004347616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6de0100c60 0x7f6de0104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.768+0000 7f6de5ed4640 1 --2- 172.21.15.94:0/2004347616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6de01002b0 0x7f6de0100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.768+0000 7f6de5ed4640 1 -- 172.21.15.94:0/2004347616 >> 172.21.15.94:0/2004347616 conn(0x7f6de0077120 msgr2=0x7f6de0077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.769+0000 7f6de5ed4640 1 -- 172.21.15.94:0/2004347616 shutdown_connections 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.769+0000 7f6de5ed4640 1 -- 172.21.15.94:0/2004347616 wait complete. 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.769+0000 7f6de5ed4640 1 Processor -- start 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.769+0000 7f6de5ed4640 1 -- start start 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.769+0000 7f6de5ed4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6de01002b0 0x7f6de019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.769+0000 7f6de5ed4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6de0105380 0x7f6de019d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.769+0000 7f6de5ed4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6de01975c0 0x7f6de0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.769+0000 7f6de5ed4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de0197fb0 con 0x7f6de01002b0 2024-02-17T12:09:25.773 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.769+0000 7f6de5ed4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de0198120 con 0x7f6de0105380 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.769+0000 7f6de5ed4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de01982e0 con 0x7f6de01975c0 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6de01002b0 0x7f6de019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6de01002b0 0x7f6de019d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32914/0 (socket says 172.21.15.94:32914) 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddf7fe640 1 -- 172.21.15.94:0/4280759476 learned_addr learned my addr 172.21.15.94:0/4280759476 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddeffd640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6de0105380 0x7f6de019d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddffff640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6de01975c0 0x7f6de0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddf7fe640 1 -- 172.21.15.94:0/4280759476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6de01975c0 msgr2=0x7f6de0197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddf7fe640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6de01975c0 0x7f6de0197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddf7fe640 1 -- 172.21.15.94:0/4280759476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6de0105380 msgr2=0x7f6de019d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddf7fe640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6de0105380 0x7f6de019d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddf7fe640 1 -- 172.21.15.94:0/4280759476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd8009900 con 0x7f6de01002b0 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddf7fe640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6de01002b0 0x7f6de019d410 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000e9f0 tx=0x7f6dd000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddcff9640 1 -- 172.21.15.94:0/4280759476 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0009720 con 0x7f6de01002b0 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6de5ed4640 1 -- 172.21.15.94:0/4280759476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de01a42d0 con 0x7f6de01002b0 2024-02-17T12:09:25.774 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddcff9640 1 -- 172.21.15.94:0/4280759476 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6dd000cd80 con 0x7f6de01002b0 2024-02-17T12:09:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6ddcff9640 1 -- 172.21.15.94:0/4280759476 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0010690 con 0x7f6de01002b0 2024-02-17T12:09:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.770+0000 7f6de5ed4640 1 -- 172.21.15.94:0/4280759476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de01a47e0 con 0x7f6de01002b0 2024-02-17T12:09:25.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.771+0000 7f6de5ed4640 1 -- 172.21.15.94:0/4280759476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de00676e0 con 0x7f6de01002b0 2024-02-17T12:09:25.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.774+0000 7f6ddcff9640 1 -- 172.21.15.94:0/4280759476 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6dd00046c0 con 0x7f6de01002b0 2024-02-17T12:09:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.775+0000 7f6ddcff9640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6dc80a2230 0x7f6dc80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.775+0000 7f6ddeffd640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6dc80a2230 0x7f6dc80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:25.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.775+0000 7f6ddcff9640 1 -- 172.21.15.94:0/4280759476 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6dd0014070 con 0x7f6de01002b0 2024-02-17T12:09:25.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.776+0000 7f6ddeffd640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6dc80a2230 0x7f6dc80a46f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6de00ffb10 tx=0x7f6dd400b410 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:25.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:25.785+0000 7f6ddcff9640 1 -- 172.21.15.94:0/4280759476 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6dd00970c0 con 0x7f6de01002b0 2024-02-17T12:09:26.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.091+0000 7f6de5ed4640 1 -- 172.21.15.94:0/4280759476 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6de01086a0 con 0x7f6de01002b0 2024-02-17T12:09:26.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.092+0000 7f6ddcff9640 1 -- 172.21.15.94:0/4280759476 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7f6de01086a0 con 0x7f6de01002b0 2024-02-17T12:09:26.093 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:09:26.093 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:09:26.094 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:09:26.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.095+0000 7f6de5ed4640 1 -- 172.21.15.94:0/4280759476 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6dc80a2230 msgr2=0x7f6dc80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.095+0000 7f6de5ed4640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6dc80a2230 0x7f6dc80a46f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6de00ffb10 tx=0x7f6dd400b410 comp rx=0 tx=0).stop 2024-02-17T12:09:26.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.095+0000 7f6de5ed4640 1 -- 172.21.15.94:0/4280759476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6de01002b0 msgr2=0x7f6de019d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.095+0000 7f6de5ed4640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6de01002b0 0x7f6de019d410 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000e9f0 tx=0x7f6dd000edf0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.095+0000 7f6de5ed4640 1 -- 172.21.15.94:0/4280759476 shutdown_connections 2024-02-17T12:09:26.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.095+0000 7f6de5ed4640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6dc80a2230 0x7f6dc80a46f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.095+0000 7f6de5ed4640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6de01975c0 0x7f6de0197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.095+0000 7f6de5ed4640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6de0105380 0x7f6de019d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.095+0000 7f6de5ed4640 1 --2- 172.21.15.94:0/4280759476 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6de01002b0 0x7f6de019d410 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.095+0000 7f6de5ed4640 1 -- 172.21.15.94:0/4280759476 >> 172.21.15.94:0/4280759476 conn(0x7f6de0077120 msgr2=0x7f6de00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.095+0000 7f6de5ed4640 1 -- 172.21.15.94:0/4280759476 shutdown_connections 2024-02-17T12:09:26.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.095+0000 7f6de5ed4640 1 -- 172.21.15.94:0/4280759476 wait complete. 2024-02-17T12:09:26.109 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:09:26.109 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:09:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.253+0000 7f78276ae640 1 -- 172.21.15.94:0/3078373341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f782010f9a0 msgr2=0x7f782010d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.253+0000 7f78276ae640 1 --2- 172.21.15.94:0/3078373341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f782010f9a0 0x7f782010d490 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f781c009900 tx=0x7f781c02bc80 comp rx=0 tx=0).stop 2024-02-17T12:09:26.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.255+0000 7f78276ae640 1 -- 172.21.15.94:0/3078373341 shutdown_connections 2024-02-17T12:09:26.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.255+0000 7f78276ae640 1 --2- 172.21.15.94:0/3078373341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f782011a960 0x7f782011cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.255+0000 7f78276ae640 1 --2- 172.21.15.94:0/3078373341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f782010d9d0 0x7f782010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.255+0000 7f78276ae640 1 --2- 172.21.15.94:0/3078373341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f782010f9a0 0x7f782010d490 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.255+0000 7f78276ae640 1 -- 172.21.15.94:0/3078373341 >> 172.21.15.94:0/3078373341 conn(0x7f782006c260 msgr2=0x7f782006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.255+0000 7f78276ae640 1 -- 172.21.15.94:0/3078373341 shutdown_connections 2024-02-17T12:09:26.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.255+0000 7f78276ae640 1 -- 172.21.15.94:0/3078373341 wait complete. 2024-02-17T12:09:26.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.256+0000 7f78276ae640 1 Processor -- start 2024-02-17T12:09:26.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.256+0000 7f78276ae640 1 -- start start 2024-02-17T12:09:26.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.256+0000 7f78276ae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f782010d9d0 0x7f7820117c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.257+0000 7f78276ae640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f782010f9a0 0x7f7820118140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.257+0000 7f78276ae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f782011a960 0x7f7820110c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.257+0000 7f7824c49640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f782010f9a0 0x7f7820118140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.257+0000 7f7824c49640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f782010f9a0 0x7f7820118140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:48880/0 (socket says 172.21.15.94:48880) 2024-02-17T12:09:26.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.257+0000 7f7824c49640 1 -- 172.21.15.94:0/301688383 learned_addr learned my addr 172.21.15.94:0/301688383 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.257+0000 7f78276ae640 1 -- 172.21.15.94:0/301688383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7820111320 con 0x7f782010d9d0 2024-02-17T12:09:26.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.257+0000 7f78276ae640 1 -- 172.21.15.94:0/301688383 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7820111490 con 0x7f782010f9a0 2024-02-17T12:09:26.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.257+0000 7f78276ae640 1 -- 172.21.15.94:0/301688383 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7820111650 con 0x7f782011a960 2024-02-17T12:09:26.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.257+0000 7f7825c4b640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f782011a960 0x7f7820110c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.257+0000 7f782544a640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f782010d9d0 0x7f7820117c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.258+0000 7f782544a640 1 -- 172.21.15.94:0/301688383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f782011a960 msgr2=0x7f7820110c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.258+0000 7f782544a640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f782011a960 0x7f7820110c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.258+0000 7f782544a640 1 -- 172.21.15.94:0/301688383 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f782010f9a0 msgr2=0x7f7820118140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.258+0000 7f782544a640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f782010f9a0 0x7f7820118140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.258+0000 7f782544a640 1 -- 172.21.15.94:0/301688383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c009560 con 0x7f782010d9d0 2024-02-17T12:09:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.258+0000 7f782544a640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f782010d9d0 0x7f7820117c00 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f781c009850 tx=0x7f781c004b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.259+0000 7f78167fc640 1 -- 172.21.15.94:0/301688383 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c0055c0 con 0x7f782010d9d0 2024-02-17T12:09:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.259+0000 7f78167fc640 1 -- 172.21.15.94:0/301688383 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f781c02cb00 con 0x7f782010d9d0 2024-02-17T12:09:26.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.259+0000 7f7824c49640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f782010f9a0 0x7f7820118140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:09:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.259+0000 7f78167fc640 1 -- 172.21.15.94:0/301688383 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c034c80 con 0x7f782010d9d0 2024-02-17T12:09:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.259+0000 7f78276ae640 1 -- 172.21.15.94:0/301688383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78201118d0 con 0x7f782010d9d0 2024-02-17T12:09:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.259+0000 7f78276ae640 1 -- 172.21.15.94:0/301688383 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7820115900 con 0x7f782010d9d0 2024-02-17T12:09:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.262+0000 7f782a282740 1 Processor -- start 2024-02-17T12:09:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.263+0000 7f78167fc640 1 -- 172.21.15.94:0/301688383 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f781c034440 con 0x7f782010d9d0 2024-02-17T12:09:26.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.263+0000 7f782a282740 1 -- start start 2024-02-17T12:09:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.263+0000 7f782a282740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceeb8250 0x559cceeb8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.263+0000 7f782a282740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceeb8b70 0x559cceec1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.263+0000 7f782a282740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceec1750 0x559cceec3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.263+0000 7f782a282740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccee73960 con 0x559cceeb8b70 2024-02-17T12:09:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.263+0000 7f782a282740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccee27310 con 0x559cceec1750 2024-02-17T12:09:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.263+0000 7f782a282740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccee24840 con 0x559cceeb8250 2024-02-17T12:09:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.264+0000 7f7825c4b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceeb8250 0x559cceeb8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.264+0000 7f7825c4b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceeb8250 0x559cceeb8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:50766/0 (socket says 172.21.15.94:50766) 2024-02-17T12:09:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.264+0000 7f7825c4b640 1 -- 172.21.15.94:0/1303076973 learned_addr learned my addr 172.21.15.94:0/1303076973 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.264+0000 7f782544a640 1 --2- 172.21.15.94:0/1303076973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceeb8b70 0x559cceec1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.264+0000 7f78167fc640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f780c0a2370 0x7f780c0a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.265+0000 7f78167fc640 1 -- 172.21.15.94:0/301688383 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f781c0f0080 con 0x7f782010d9d0 2024-02-17T12:09:26.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.265+0000 7f7825c4b640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f780c0a2370 0x7f780c0a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.265+0000 7f7824c49640 1 --2- 172.21.15.94:0/1303076973 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceec1750 0x559cceec3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.266+0000 7f782544a640 1 -- 172.21.15.94:0/1303076973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceeb8250 msgr2=0x559cceeb8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.266+0000 7f782544a640 1 --2- 172.21.15.94:0/1303076973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceeb8250 0x559cceeb8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.266+0000 7f782544a640 1 -- 172.21.15.94:0/1303076973 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceec1750 msgr2=0x559cceec3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.266+0000 7f782544a640 1 --2- 172.21.15.94:0/1303076973 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceec1750 0x559cceec3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.266+0000 7f782544a640 1 -- 172.21.15.94:0/1303076973 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ccede2060 con 0x559cceeb8b70 2024-02-17T12:09:26.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.266+0000 7f782544a640 1 --2- 172.21.15.94:0/1303076973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceeb8b70 0x559cceec1210 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f781c0ed840 tx=0x7f781c02bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3f021f664e75f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.267+0000 7f7825c4b640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f780c0a2370 0x7f780c0a4830 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f78180139f0 tx=0x7f7818009c60 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.269 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.267+0000 7f780b7fe640 1 -- 172.21.15.94:0/1303076973 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c0bd1e0 con 0x559cceeb8b70 2024-02-17T12:09:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.268+0000 7f780b7fe640 1 -- 172.21.15.94:0/1303076973 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f781c0bcae0 con 0x559cceeb8b70 2024-02-17T12:09:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.269+0000 7f782a282740 1 -- 172.21.15.94:0/1303076973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceeb8b70 msgr2=0x559cceec1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.269+0000 7f782a282740 1 --2- 172.21.15.94:0/1303076973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceeb8b70 0x559cceec1210 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f781c0ed840 tx=0x7f781c02bc80 comp rx=0 tx=0).stop 2024-02-17T12:09:26.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.271+0000 7f782a282740 1 -- 172.21.15.94:0/1303076973 shutdown_connections 2024-02-17T12:09:26.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.271+0000 7f782a282740 1 --2- 172.21.15.94:0/1303076973 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceec1750 0x559cceec3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.271+0000 7f782a282740 1 --2- 172.21.15.94:0/1303076973 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceeb8b70 0x559cceec1210 secure :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f781c0ed840 tx=0x7f781c02bc80 comp rx=0 tx=0).stop 2024-02-17T12:09:26.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.271+0000 7f782a282740 1 --2- 172.21.15.94:0/1303076973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceeb8250 0x559cceeb8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.271+0000 7f782a282740 1 -- 172.21.15.94:0/1303076973 >> 172.21.15.94:0/1303076973 conn(0x559cceeb2980 msgr2=0x559cceeb2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.271+0000 7f782a282740 1 -- 172.21.15.94:0/1303076973 shutdown_connections 2024-02-17T12:09:26.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.271+0000 7f782a282740 1 -- 172.21.15.94:0/1303076973 wait complete. 2024-02-17T12:09:26.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f782a282740 1 -- start start 2024-02-17T12:09:26.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f782a282740 1 Processor -- start 2024-02-17T12:09:26.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f782a282740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceeb8250 0x559cceebe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f782a282740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceec1750 0x559cceebecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f782a282740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceebf1f0 0x559cceecd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f782a282740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccee20020 con 0x559cceec1750 2024-02-17T12:09:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f782a282740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cceaf3430 con 0x559cceeb8250 2024-02-17T12:09:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f782a282740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cceaafe00 con 0x559cceebf1f0 2024-02-17T12:09:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f782a282740 10 client.0 _ll_register_callbacks cb 0x559ccedc7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f7824c49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceec1750 0x559cceebecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f7824c49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceec1750 0x559cceebecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32928/0 (socket says 172.21.15.94:32928) 2024-02-17T12:09:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f7824c49640 1 -- 172.21.15.94:0/1111071472 learned_addr learned my addr 172.21.15.94:0/1111071472 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f782544a640 1 --2- 172.21.15.94:0/1111071472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceeb8250 0x559cceebe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.272+0000 7f7825c4b640 1 --2- 172.21.15.94:0/1111071472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceebf1f0 0x559cceecd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.273+0000 7f7824c49640 1 -- 172.21.15.94:0/1111071472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceebf1f0 msgr2=0x559cceecd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.273+0000 7f7824c49640 1 --2- 172.21.15.94:0/1111071472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceebf1f0 0x559cceecd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.273+0000 7f7824c49640 1 -- 172.21.15.94:0/1111071472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceeb8250 msgr2=0x559cceebe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.273+0000 7f7824c49640 1 --2- 172.21.15.94:0/1111071472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceeb8250 0x559cceebe770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.273+0000 7f7824c49640 1 -- 172.21.15.94:0/1111071472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7820115900 con 0x559cceec1750 2024-02-17T12:09:26.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.273+0000 7f7824c49640 1 --2- 172.21.15.94:0/1111071472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceec1750 0x559cceebecb0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f781000e500 tx=0x7f781000e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61970cab8127186c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.273+0000 7f7808ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:26.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.274+0000 7f7808ff9640 1 -- 172.21.15.94:0/1111071472 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810012600 con 0x559cceec1750 2024-02-17T12:09:26.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.274+0000 7f7808ff9640 1 -- 172.21.15.94:0/1111071472 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7810012760 con 0x559cceec1750 2024-02-17T12:09:26.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.274+0000 7f782a282740 1 -- 172.21.15.94:0/1111071472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559cceb1f7e0 con 0x559cceec1750 2024-02-17T12:09:26.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.274+0000 7f7808ff9640 1 -- 172.21.15.94:0/1111071472 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7810012920 con 0x559cceec1750 2024-02-17T12:09:26.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.274+0000 7f782a282740 10 client.18426 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:26.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.274+0000 7f782a282740 1 -- 172.21.15.94:0/1111071472 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559ccee1e6a0 con 0x559cceec1750 2024-02-17T12:09:26.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.275+0000 7f7808ff9640 1 -- 172.21.15.94:0/1111071472 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559cceb1f7e0 con 0x559cceec1750 2024-02-17T12:09:26.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.275+0000 7f7808ff9640 1 -- 172.21.15.94:0/1111071472 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7810011070 con 0x559cceec1750 2024-02-17T12:09:26.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.275+0000 7f782a282740 10 client.18426 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:26.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.275+0000 7f782a282740 10 client.18426 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:09:26.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.275+0000 7f782a282740 20 client.18426 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:26.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.275+0000 7f782a282740 1 --2- 172.21.15.94:0/1111071472 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x559cceed1c30 0x559cceed4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.275+0000 7f782a282740 4 client.18426 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:26.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.275+0000 7f782a282740 1 -- 172.21.15.94:0/1111071472 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cceb5c720 con 0x559cceed1c30 2024-02-17T12:09:26.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.275+0000 7f782544a640 1 --2- 172.21.15.94:0/1111071472 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x559cceed1c30 0x559cceed4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.276+0000 7f782544a640 1 --2- 172.21.15.94:0/1111071472 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x559cceed1c30 0x559cceed4090 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f786b48cfb5827cb server_cookie=16827417431dee09 in_seq=0 out_seq=0 2024-02-17T12:09:26.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.276+0000 7f7808ff9640 10 client.18426 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:09:26.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.276+0000 7f7808ff9640 1 -- 172.21.15.94:0/1111071472 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559cceb5c720 con 0x559cceed1c30 2024-02-17T12:09:26.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.276+0000 7f7808ff9640 10 client.18426 handle_command_reply: tid=0 2024-02-17T12:09:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.277+0000 7f782a282740 1 client.18426 shutdown 2024-02-17T12:09:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.277+0000 7f782a282740 1 -- 172.21.15.94:0/1111071472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceec1750 msgr2=0x559cceebecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.277+0000 7f782a282740 1 --2- 172.21.15.94:0/1111071472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceec1750 0x559cceebecb0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f781000e500 tx=0x7f781000e900 comp rx=0 tx=0).stop 2024-02-17T12:09:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.277+0000 7f782a282740 1 -- 172.21.15.94:0/1111071472 shutdown_connections 2024-02-17T12:09:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.277+0000 7f782a282740 1 --2- 172.21.15.94:0/1111071472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceebf1f0 0x559cceecd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.277+0000 7f782a282740 1 --2- 172.21.15.94:0/1111071472 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceec1750 0x559cceebecb0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.277+0000 7f782a282740 1 --2- 172.21.15.94:0/1111071472 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x559cceed1c30 0x559cceed4090 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.277+0000 7f782a282740 1 --2- 172.21.15.94:0/1111071472 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceeb8250 0x559cceebe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.277+0000 7f782a282740 1 -- 172.21.15.94:0/1111071472 >> 172.21.15.94:0/1111071472 conn(0x559cceeb2980 msgr2=0x559cceeb46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.277+0000 7f7808ff9640 0 client.18426 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:09:26.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.277+0000 7f782a282740 1 -- 172.21.15.94:0/1111071472 shutdown_connections 2024-02-17T12:09:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.277+0000 7f782a282740 1 -- 172.21.15.94:0/1111071472 wait complete. 2024-02-17T12:09:26.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.278+0000 7f782a282740 20 client.18426 trim_cache size 0 max 16384 2024-02-17T12:09:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.293+0000 7f782a282740 1 Processor -- start 2024-02-17T12:09:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.293+0000 7f782a282740 1 -- start start 2024-02-17T12:09:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.293+0000 7f782a282740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceeb8250 0x559cceec83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.293+0000 7f782a282740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 0x559cceec8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.293+0000 7f782a282740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceec1750 0x559cceebd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.293+0000 7f782a282740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cceda5130 con 0x559cceebf1f0 2024-02-17T12:09:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.293+0000 7f782a282740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cceb1f7e0 con 0x559cceeb8250 2024-02-17T12:09:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.293+0000 7f782a282740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cceed0400 con 0x559cceec1750 2024-02-17T12:09:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.293+0000 7f7824c49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 0x559cceec8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.293+0000 7f7824c49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 0x559cceec8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32932/0 (socket says 172.21.15.94:32932) 2024-02-17T12:09:26.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.293+0000 7f7824c49640 1 -- 172.21.15.94:0/629282690 learned_addr learned my addr 172.21.15.94:0/629282690 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.294+0000 7f7825c4b640 1 --2- 172.21.15.94:0/629282690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceec1750 0x559cceebd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.294+0000 7f782544a640 1 --2- 172.21.15.94:0/629282690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceeb8250 0x559cceec83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.294+0000 7f7824c49640 1 -- 172.21.15.94:0/629282690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceec1750 msgr2=0x559cceebd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.294+0000 7f7824c49640 1 --2- 172.21.15.94:0/629282690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceec1750 0x559cceebd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.294+0000 7f7824c49640 1 -- 172.21.15.94:0/629282690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceeb8250 msgr2=0x559cceec83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.294+0000 7f7824c49640 1 --2- 172.21.15.94:0/629282690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceeb8250 0x559cceec83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.294+0000 7f7824c49640 1 -- 172.21.15.94:0/629282690 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ccedbac20 con 0x559cceebf1f0 2024-02-17T12:09:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.294+0000 7f782544a640 1 --2- 172.21.15.94:0/629282690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceeb8250 0x559cceec83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:09:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.294+0000 7f7824c49640 1 --2- 172.21.15.94:0/629282690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 0x559cceec8900 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7810006f50 tx=0x7f781000c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884058944fefae15 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.294+0000 7f780affd640 1 -- 172.21.15.94:0/629282690 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781000ccd0 con 0x559cceebf1f0 2024-02-17T12:09:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.294+0000 7f780affd640 1 -- 172.21.15.94:0/629282690 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f781001d3b0 con 0x559cceebf1f0 2024-02-17T12:09:26.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.294+0000 7f780affd640 1 -- 172.21.15.94:0/629282690 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810014b20 con 0x559cceebf1f0 2024-02-17T12:09:26.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.295+0000 7f782a282740 1 -- 172.21.15.94:0/629282690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 msgr2=0x559cceec8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.295+0000 7f782a282740 1 --2- 172.21.15.94:0/629282690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 0x559cceec8900 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7810006f50 tx=0x7f781000c3d0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.295+0000 7f782a282740 1 -- 172.21.15.94:0/629282690 shutdown_connections 2024-02-17T12:09:26.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.295+0000 7f782a282740 1 --2- 172.21.15.94:0/629282690 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceec1750 0x559cceebd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.295+0000 7f782a282740 1 --2- 172.21.15.94:0/629282690 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 0x559cceec8900 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.295+0000 7f782a282740 1 --2- 172.21.15.94:0/629282690 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceeb8250 0x559cceec83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.295+0000 7f782a282740 1 -- 172.21.15.94:0/629282690 >> 172.21.15.94:0/629282690 conn(0x559cceeb2980 msgr2=0x559cceeb9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.295+0000 7f782a282740 1 -- 172.21.15.94:0/629282690 shutdown_connections 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.295+0000 7f782a282740 1 -- 172.21.15.94:0/629282690 wait complete. 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.296+0000 7f782a282740 1 -- start start 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.296+0000 7f782a282740 1 Processor -- start 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.296+0000 7f782a282740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceeb8250 0x559cceec6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.296+0000 7f782a282740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 0x559cceec74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.296+0000 7f782a282740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceec1750 0x559cceec7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.296+0000 7f782a282740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccef281d0 con 0x559cceebf1f0 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.296+0000 7f782a282740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccef28340 con 0x559cceec1750 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.296+0000 7f782a282740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ccef28500 con 0x559cceeb8250 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.296+0000 7f782a282740 10 client.0 _ll_register_callbacks cb 0x7f781000c1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:26.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.296+0000 7f7824c49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 0x559cceec74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.296+0000 7f7824c49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 0x559cceec74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32946/0 (socket says 172.21.15.94:32946) 2024-02-17T12:09:26.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.296+0000 7f7824c49640 1 -- 172.21.15.94:0/4141979089 learned_addr learned my addr 172.21.15.94:0/4141979089 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.296+0000 7f7825c4b640 1 --2- 172.21.15.94:0/4141979089 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceec1750 0x559cceec7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f7824c49640 1 -- 172.21.15.94:0/4141979089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceeb8250 msgr2=0x559cceec6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f7824c49640 1 --2- 172.21.15.94:0/4141979089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceeb8250 0x559cceec6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f7824c49640 1 -- 172.21.15.94:0/4141979089 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceec1750 msgr2=0x559cceec7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f7824c49640 1 --2- 172.21.15.94:0/4141979089 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceec1750 0x559cceec7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f7824c49640 1 -- 172.21.15.94:0/4141979089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559ccee1e6a0 con 0x559cceebf1f0 2024-02-17T12:09:26.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f7824c49640 1 --2- 172.21.15.94:0/4141979089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 0x559cceec74e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7810014950 tx=0x7f7810014980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c118989de7380c95 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f7808ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:26.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f7808ff9640 1 -- 172.21.15.94:0/4141979089 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781000ccd0 con 0x559cceebf1f0 2024-02-17T12:09:26.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f7808ff9640 1 -- 172.21.15.94:0/4141979089 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f781001d8f0 con 0x559cceebf1f0 2024-02-17T12:09:26.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f782a282740 1 -- 172.21.15.94:0/4141979089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f781c0bcae0 con 0x559cceebf1f0 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f7808ff9640 1 -- 172.21.15.94:0/4141979089 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7810014b20 con 0x559cceebf1f0 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f782a282740 10 client.18432 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f782a282740 1 -- 172.21.15.94:0/4141979089 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559ccede08c0 con 0x559cceebf1f0 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.297+0000 7f7808ff9640 1 -- 172.21.15.94:0/4141979089 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f781c0bcae0 con 0x559cceebf1f0 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.298+0000 7f7808ff9640 1 -- 172.21.15.94:0/4141979089 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7810011040 con 0x559cceebf1f0 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.298+0000 7f782a282740 10 client.18432 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.298+0000 7f782a282740 10 client.18432 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.298+0000 7f782a282740 20 client.18432 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.298+0000 7f782a282740 1 --2- 172.21.15.94:0/4141979089 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x559ccef299f0 0x559ccef2be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.298+0000 7f782a282740 4 client.18432 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.298+0000 7f782a282740 1 -- 172.21.15.94:0/4141979089 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559ccebab890 con 0x559ccef299f0 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.298+0000 7f782544a640 1 --2- 172.21.15.94:0/4141979089 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x559ccef299f0 0x559ccef2be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.299+0000 7f7808ff9640 10 client.18432 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:09:26.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.299+0000 7f782544a640 1 --2- 172.21.15.94:0/4141979089 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x559ccef299f0 0x559ccef2be10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=124336742c4b6dea server_cookie=48beac4f194b8d0d in_seq=0 out_seq=0 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.299+0000 7f7808ff9640 1 -- 172.21.15.94:0/4141979089 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559ccebab890 con 0x559ccef299f0 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.299+0000 7f7808ff9640 10 client.18432 handle_command_reply: tid=0 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.299+0000 7f782a282740 1 client.18432 shutdown 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f782a282740 1 -- 172.21.15.94:0/4141979089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 msgr2=0x559cceec74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f782a282740 1 --2- 172.21.15.94:0/4141979089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 0x559cceec74e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7810014950 tx=0x7f7810014980 comp rx=0 tx=0).stop 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f782a282740 1 -- 172.21.15.94:0/4141979089 shutdown_connections 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f782a282740 1 --2- 172.21.15.94:0/4141979089 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x559ccef299f0 0x559ccef2be10 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f782a282740 1 --2- 172.21.15.94:0/4141979089 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559cceec1750 0x559cceec7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f7808ff9640 0 client.18432 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f782a282740 1 --2- 172.21.15.94:0/4141979089 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559cceebf1f0 0x559cceec74e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f782a282740 1 --2- 172.21.15.94:0/4141979089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cceeb8250 0x559cceec6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f782a282740 1 -- 172.21.15.94:0/4141979089 >> 172.21.15.94:0/4141979089 conn(0x559cceeb2980 msgr2=0x559cceebcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f782a282740 1 -- 172.21.15.94:0/4141979089 shutdown_connections 2024-02-17T12:09:26.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f782a282740 1 -- 172.21.15.94:0/4141979089 wait complete. 2024-02-17T12:09:26.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f782a282740 20 client.18432 trim_cache size 0 max 16384 2024-02-17T12:09:26.307 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:09:26.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f78276ae640 1 -- 172.21.15.94:0/301688383 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f780c0a2370 msgr2=0x7f780c0a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f78276ae640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f780c0a2370 0x7f780c0a4830 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f78180139f0 tx=0x7f7818009c60 comp rx=0 tx=0).stop 2024-02-17T12:09:26.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f78276ae640 1 -- 172.21.15.94:0/301688383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f782010d9d0 msgr2=0x7f7820117c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.300+0000 7f78276ae640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f782010d9d0 0x7f7820117c00 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f781c009850 tx=0x7f781c004b70 comp rx=0 tx=0).stop 2024-02-17T12:09:26.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.301+0000 7f78276ae640 1 -- 172.21.15.94:0/301688383 shutdown_connections 2024-02-17T12:09:26.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.301+0000 7f78276ae640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f780c0a2370 0x7f780c0a4830 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.301+0000 7f78276ae640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f782011a960 0x7f7820110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.301+0000 7f78276ae640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f782010f9a0 0x7f7820118140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.301+0000 7f78276ae640 1 --2- 172.21.15.94:0/301688383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f782010d9d0 0x7f7820117c00 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.301+0000 7f78276ae640 1 -- 172.21.15.94:0/301688383 >> 172.21.15.94:0/301688383 conn(0x7f782006c260 msgr2=0x7f782011b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.301+0000 7f78276ae640 1 -- 172.21.15.94:0/301688383 shutdown_connections 2024-02-17T12:09:26.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.301+0000 7f78276ae640 1 -- 172.21.15.94:0/301688383 wait complete. 2024-02-17T12:09:26.315 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:09:26.316 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:09:26.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:26 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/4280759476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:09:26.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:26 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/4280759476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:09:26.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:26 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/4280759476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:09:26.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.468+0000 7f9f83c49640 1 -- 172.21.15.94:0/2644068202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f7c11a960 msgr2=0x7f9f7c11cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.468+0000 7f9f83c49640 1 --2- 172.21.15.94:0/2644068202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f7c11a960 0x7f9f7c11cd50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9f7401acb0 tx=0x7f9f7403d170 comp rx=0 tx=0).stop 2024-02-17T12:09:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.472+0000 7f9f83c49640 1 -- 172.21.15.94:0/2644068202 shutdown_connections 2024-02-17T12:09:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.472+0000 7f9f83c49640 1 --2- 172.21.15.94:0/2644068202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f7c11a960 0x7f9f7c11cd50 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.472+0000 7f9f83c49640 1 --2- 172.21.15.94:0/2644068202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f7c10d9d0 0x7f9f7c10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.472+0000 7f9f83c49640 1 --2- 172.21.15.94:0/2644068202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f7c10f9a0 0x7f9f7c10d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.472+0000 7f9f83c49640 1 -- 172.21.15.94:0/2644068202 >> 172.21.15.94:0/2644068202 conn(0x7f9f7c06c260 msgr2=0x7f9f7c06c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.474+0000 7f9f83c49640 1 -- 172.21.15.94:0/2644068202 shutdown_connections 2024-02-17T12:09:26.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.477+0000 7f9f83c49640 1 -- 172.21.15.94:0/2644068202 wait complete. 2024-02-17T12:09:26.479 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.478+0000 7f9f83c49640 1 Processor -- start 2024-02-17T12:09:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.478+0000 7f9f83c49640 1 -- start start 2024-02-17T12:09:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.478+0000 7f9f83c49640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f7c10d9d0 0x7f9f7c11a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.478+0000 7f9f83c49640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f7c10f9a0 0x7f9f7c116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.478+0000 7f9f83c49640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f7c11a960 0x7f9f7c116640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.478+0000 7f9f83c49640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c116c40 con 0x7f9f7c10f9a0 2024-02-17T12:09:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.478+0000 7f9f83c49640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c116db0 con 0x7f9f7c11a960 2024-02-17T12:09:26.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.478+0000 7f9f83c49640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7c116f70 con 0x7f9f7c10d9d0 2024-02-17T12:09:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.479+0000 7f9f819e5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f7c10d9d0 0x7f9f7c11a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.479+0000 7f9f819e5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f7c10d9d0 0x7f9f7c11a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:50816/0 (socket says 172.21.15.94:50816) 2024-02-17T12:09:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.479+0000 7f9f819e5640 1 -- 172.21.15.94:0/2409044963 learned_addr learned my addr 172.21.15.94:0/2409044963 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.479+0000 7f9f811e4640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f7c10f9a0 0x7f9f7c116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.479+0000 7f9f821e6640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f7c11a960 0x7f9f7c116640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.479+0000 7f9f819e5640 1 -- 172.21.15.94:0/2409044963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f7c11a960 msgr2=0x7f9f7c116640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.480+0000 7f9f819e5640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f7c11a960 0x7f9f7c116640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.480+0000 7f9f819e5640 1 -- 172.21.15.94:0/2409044963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f7c10f9a0 msgr2=0x7f9f7c116100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.480+0000 7f9f819e5640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f7c10f9a0 0x7f9f7c116100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.480+0000 7f9f819e5640 1 -- 172.21.15.94:0/2409044963 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f7401a910 con 0x7f9f7c10d9d0 2024-02-17T12:09:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.480+0000 7f9f819e5640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f7c10d9d0 0x7f9f7c11a070 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800b450 tx=0x7f9f7800b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.480+0000 7f9f821e6640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f7c11a960 0x7f9f7c116640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:09:26.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.484+0000 7f9f72ffd640 1 -- 172.21.15.94:0/2409044963 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78011890 con 0x7f9f7c10d9d0 2024-02-17T12:09:26.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.484+0000 7f9f72ffd640 1 -- 172.21.15.94:0/2409044963 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f780103e0 con 0x7f9f7c10d9d0 2024-02-17T12:09:26.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.484+0000 7f9f72ffd640 1 -- 172.21.15.94:0/2409044963 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7800f580 con 0x7f9f7c10d9d0 2024-02-17T12:09:26.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.484+0000 7f9f83c49640 1 -- 172.21.15.94:0/2409044963 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f7c1aa690 con 0x7f9f7c10d9d0 2024-02-17T12:09:26.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.485+0000 7f9f83c49640 1 -- 172.21.15.94:0/2409044963 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f7c1aabd0 con 0x7f9f7c10d9d0 2024-02-17T12:09:26.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.489+0000 7f9f86816740 1 Processor -- start 2024-02-17T12:09:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.490+0000 7f9f86816740 1 -- start start 2024-02-17T12:09:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.490+0000 7f9f86816740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926ca630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.490+0000 7f9f86816740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926cab70 0x55d7926d3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.490+0000 7f9f86816740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926d3750 0x55d7926d5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.490+0000 7f9f86816740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d792685960 con 0x55d7926cab70 2024-02-17T12:09:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.490+0000 7f9f86816740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d792639310 con 0x55d7926d3750 2024-02-17T12:09:26.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.490+0000 7f9f86816740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d792636840 con 0x55d7926ca250 2024-02-17T12:09:26.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.493+0000 7f9f821e6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.493+0000 7f9f821e6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:50820/0 (socket says 172.21.15.94:50820) 2024-02-17T12:09:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.493+0000 7f9f819e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926cab70 0x55d7926d3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.493+0000 7f9f819e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926cab70 0x55d7926d3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32972/0 (socket says 172.21.15.94:32972) 2024-02-17T12:09:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.493+0000 7f9f819e5640 1 -- 172.21.15.94:0/1129998299 learned_addr learned my addr 172.21.15.94:0/1129998299 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.493+0000 7f9f811e4640 1 --2- 172.21.15.94:0/1129998299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926d3750 0x55d7926d5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.494+0000 7f9f72ffd640 1 -- 172.21.15.94:0/2409044963 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9f78010550 con 0x7f9f7c10d9d0 2024-02-17T12:09:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.494+0000 7f9f72ffd640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9f640a2440 0x7f9f640a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.495+0000 7f9f72ffd640 1 -- 172.21.15.94:0/2409044963 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9f780cfc30 con 0x7f9f7c10d9d0 2024-02-17T12:09:26.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.495+0000 7f9f811e4640 1 -- 172.21.15.94:0/1129998299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 msgr2=0x55d7926ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.495+0000 7f9f811e4640 1 --2- 172.21.15.94:0/1129998299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.495+0000 7f9f811e4640 1 -- 172.21.15.94:0/1129998299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926cab70 msgr2=0x55d7926d3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.495+0000 7f9f811e4640 1 --2- 172.21.15.94:0/1129998299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926cab70 0x55d7926d3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.495+0000 7f9f811e4640 1 -- 172.21.15.94:0/1129998299 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7925f4060 con 0x55d7926d3750 2024-02-17T12:09:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.496+0000 7f9f821e6640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9f640a2440 0x7f9f640a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.496+0000 7f9f811e4640 1 --2- 172.21.15.94:0/1129998299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926d3750 0x55d7926d5b40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c117950 tx=0x7f9f6c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3c36c0576c6cfac server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.496+0000 7f9f63fff640 1 -- 172.21.15.94:0/1129998299 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c009d70 con 0x55d7926d3750 2024-02-17T12:09:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.496+0000 7f9f63fff640 1 -- 172.21.15.94:0/1129998299 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f6c004510 con 0x55d7926d3750 2024-02-17T12:09:26.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.496+0000 7f9f63fff640 1 -- 172.21.15.94:0/1129998299 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c004ad0 con 0x55d7926d3750 2024-02-17T12:09:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.497+0000 7f9f86816740 1 -- 172.21.15.94:0/1129998299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926d3750 msgr2=0x55d7926d5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.497+0000 7f9f821e6640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9f640a2440 0x7f9f640a4900 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f7403d780 tx=0x7f9f7401a5d0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.497+0000 7f9f86816740 1 --2- 172.21.15.94:0/1129998299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926d3750 0x55d7926d5b40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c117950 tx=0x7f9f6c011810 comp rx=0 tx=0).stop 2024-02-17T12:09:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.497+0000 7f9f86816740 1 -- 172.21.15.94:0/1129998299 shutdown_connections 2024-02-17T12:09:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.497+0000 7f9f86816740 1 --2- 172.21.15.94:0/1129998299 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926d3750 0x55d7926d5b40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.497+0000 7f9f86816740 1 --2- 172.21.15.94:0/1129998299 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926cab70 0x55d7926d3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.497+0000 7f9f86816740 1 --2- 172.21.15.94:0/1129998299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.497+0000 7f9f86816740 1 -- 172.21.15.94:0/1129998299 >> 172.21.15.94:0/1129998299 conn(0x55d7926c4980 msgr2=0x55d7926c4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.498+0000 7f9f86816740 1 -- 172.21.15.94:0/1129998299 shutdown_connections 2024-02-17T12:09:26.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.498+0000 7f9f86816740 1 -- 172.21.15.94:0/1129998299 wait complete. 2024-02-17T12:09:26.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.498+0000 7f9f86816740 1 -- start start 2024-02-17T12:09:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.498+0000 7f9f86816740 1 Processor -- start 2024-02-17T12:09:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.499+0000 7f9f86816740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.499+0000 7f9f86816740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926cab70 0x55d7926d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.499+0000 7f9f86816740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926d3750 0x55d7926d11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.499+0000 7f9f811e4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926cab70 0x55d7926d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.499+0000 7f9f819e5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.499+0000 7f9f811e4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926cab70 0x55d7926d0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:48946/0 (socket says 172.21.15.94:48946) 2024-02-17T12:09:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.499+0000 7f9f819e5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926d0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:50836/0 (socket says 172.21.15.94:50836) 2024-02-17T12:09:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.499+0000 7f9f821e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926d3750 0x55d7926d11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.499+0000 7f9f821e6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926d3750 0x55d7926d11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:32974/0 (socket says 172.21.15.94:32974) 2024-02-17T12:09:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.499+0000 7f9f821e6640 1 -- 172.21.15.94:0/466562265 learned_addr learned my addr 172.21.15.94:0/466562265 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.499+0000 7f9f86816740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d792632020 con 0x55d7926d3750 2024-02-17T12:09:26.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.499+0000 7f9f86816740 1 -- 172.21.15.94:0/466562265 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d792305430 con 0x55d7926cab70 2024-02-17T12:09:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.499+0000 7f9f86816740 1 -- 172.21.15.94:0/466562265 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7922c1e00 con 0x55d7926ca250 2024-02-17T12:09:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.500+0000 7f9f86816740 10 client.0 _ll_register_callbacks cb 0x55d7925d9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.500+0000 7f9f821e6640 1 -- 172.21.15.94:0/466562265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 msgr2=0x55d7926d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.500+0000 7f9f821e6640 1 --2- 172.21.15.94:0/466562265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926d0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.500+0000 7f9f821e6640 1 -- 172.21.15.94:0/466562265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926cab70 msgr2=0x55d7926d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.500+0000 7f9f821e6640 1 --2- 172.21.15.94:0/466562265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926cab70 0x55d7926d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.500+0000 7f9f821e6640 1 -- 172.21.15.94:0/466562265 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6c009850 con 0x55d7926d3750 2024-02-17T12:09:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.501+0000 7f9f821e6640 1 --2- 172.21.15.94:0/466562265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926d3750 0x55d7926d11c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f9f74047000 tx=0x7f9f7403f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d6fca81570ae31 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.501+0000 7f9f617fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.501+0000 7f9f617fa640 1 -- 172.21.15.94:0/466562265 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f74004250 con 0x55d7926d3750 2024-02-17T12:09:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.501+0000 7f9f617fa640 1 -- 172.21.15.94:0/466562265 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9f74056030 con 0x55d7926d3750 2024-02-17T12:09:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.501+0000 7f9f86816740 1 -- 172.21.15.94:0/466562265 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7923317e0 con 0x55d7926d3750 2024-02-17T12:09:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.501+0000 7f9f617fa640 1 -- 172.21.15.94:0/466562265 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f740043b0 con 0x55d7926d3750 2024-02-17T12:09:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.502+0000 7f9f86816740 10 client.18444 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.502+0000 7f9f86816740 1 -- 172.21.15.94:0/466562265 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d7926306a0 con 0x55d7926d3750 2024-02-17T12:09:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.502+0000 7f9f617fa640 1 -- 172.21.15.94:0/466562265 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d7923317e0 con 0x55d7926d3750 2024-02-17T12:09:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.502+0000 7f9f617fa640 1 -- 172.21.15.94:0/466562265 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9f7403dd20 con 0x55d7926d3750 2024-02-17T12:09:26.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.503+0000 7f9f86816740 10 client.18444 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.503+0000 7f9f86816740 10 client.18444 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:09:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.503+0000 7f9f86816740 20 client.18444 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.503+0000 7f9f86816740 1 --2- 172.21.15.94:0/466562265 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d7926e3b60 0x55d7926e5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.503+0000 7f9f86816740 4 client.18444 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.503+0000 7f9f86816740 1 -- 172.21.15.94:0/466562265 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d79236e720 con 0x55d7926e3b60 2024-02-17T12:09:26.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.503+0000 7f9f819e5640 1 --2- 172.21.15.94:0/466562265 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d7926e3b60 0x55d7926e5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.504+0000 7f9f617fa640 10 client.18444 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:09:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.504+0000 7f9f819e5640 1 --2- 172.21.15.94:0/466562265 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d7926e3b60 0x55d7926e5f80 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2b8fc73d2a847e9d server_cookie=87f79ced82e33c2a in_seq=0 out_seq=0 2024-02-17T12:09:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.505+0000 7f9f617fa640 1 -- 172.21.15.94:0/466562265 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d79236e720 con 0x55d7926e3b60 2024-02-17T12:09:26.507 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.505+0000 7f9f617fa640 10 client.18444 handle_command_reply: tid=0 2024-02-17T12:09:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.505+0000 7f9f86816740 1 client.18444 shutdown 2024-02-17T12:09:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.505+0000 7f9f86816740 1 -- 172.21.15.94:0/466562265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926d3750 msgr2=0x55d7926d11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.505+0000 7f9f86816740 1 --2- 172.21.15.94:0/466562265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926d3750 0x55d7926d11c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f9f74047000 tx=0x7f9f7403f040 comp rx=0 tx=0).stop 2024-02-17T12:09:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.505+0000 7f9f86816740 1 -- 172.21.15.94:0/466562265 shutdown_connections 2024-02-17T12:09:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.506+0000 7f9f86816740 1 --2- 172.21.15.94:0/466562265 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d7926e3b60 0x55d7926e5f80 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.506+0000 7f9f86816740 1 --2- 172.21.15.94:0/466562265 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926d3750 0x55d7926d11c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.506+0000 7f9f617fa640 0 client.18444 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:09:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.506+0000 7f9f86816740 1 --2- 172.21.15.94:0/466562265 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926cab70 0x55d7926d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.506+0000 7f9f86816740 1 --2- 172.21.15.94:0/466562265 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926d0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.506+0000 7f9f86816740 1 -- 172.21.15.94:0/466562265 >> 172.21.15.94:0/466562265 conn(0x55d7926c4980 msgr2=0x55d7926c66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.506+0000 7f9f86816740 1 -- 172.21.15.94:0/466562265 shutdown_connections 2024-02-17T12:09:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.506+0000 7f9f86816740 1 -- 172.21.15.94:0/466562265 wait complete. 2024-02-17T12:09:26.508 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.506+0000 7f9f86816740 20 client.18444 trim_cache size 0 max 16384 2024-02-17T12:09:26.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.521+0000 7f9f86816740 1 Processor -- start 2024-02-17T12:09:26.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.521+0000 7f9f86816740 1 -- start start 2024-02-17T12:09:26.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.521+0000 7f9f86816740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.521+0000 7f9f86816740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926cab70 0x55d7926c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.521+0000 7f9f86816740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926d3750 0x55d7926da630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.524 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.521+0000 7f9f819e5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.521+0000 7f9f819e5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926d0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:50844/0 (socket says 172.21.15.94:50844) 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.521+0000 7f9f819e5640 1 -- 172.21.15.94:0/3038299983 learned_addr learned my addr 172.21.15.94:0/3038299983 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.521+0000 7f9f811e4640 1 --2- 172.21.15.94:0/3038299983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926cab70 0x55d7926c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.521+0000 7f9f86816740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7925b7130 con 0x55d7926d3750 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.521+0000 7f9f86816740 1 -- 172.21.15.94:0/3038299983 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7926c5980 con 0x55d7926cab70 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.521+0000 7f9f821e6640 1 --2- 172.21.15.94:0/3038299983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926d3750 0x55d7926da630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.521+0000 7f9f86816740 1 -- 172.21.15.94:0/3038299983 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7926c9eb0 con 0x55d7926ca250 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.522+0000 7f9f819e5640 1 -- 172.21.15.94:0/3038299983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926cab70 msgr2=0x55d7926c3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.522+0000 7f9f819e5640 1 --2- 172.21.15.94:0/3038299983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926cab70 0x55d7926c3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.522+0000 7f9f819e5640 1 -- 172.21.15.94:0/3038299983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926d3750 msgr2=0x55d7926da630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.522+0000 7f9f819e5640 1 --2- 172.21.15.94:0/3038299983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926d3750 0x55d7926da630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.522+0000 7f9f819e5640 1 -- 172.21.15.94:0/3038299983 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7925ccc20 con 0x55d7926ca250 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.522+0000 7f9f819e5640 1 --2- 172.21.15.94:0/3038299983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926d0fe0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009fd0 tx=0x7f9f780d34b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74fc3d34643d4eeb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.523+0000 7f9f637fe640 1 -- 172.21.15.94:0/3038299983 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f780b4070 con 0x55d7926ca250 2024-02-17T12:09:26.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.523+0000 7f9f637fe640 1 -- 172.21.15.94:0/3038299983 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f7809c690 con 0x55d7926ca250 2024-02-17T12:09:26.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.523+0000 7f9f637fe640 1 -- 172.21.15.94:0/3038299983 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7809aa80 con 0x55d7926ca250 2024-02-17T12:09:26.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.524+0000 7f9f86816740 1 -- 172.21.15.94:0/3038299983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 msgr2=0x55d7926d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.524+0000 7f9f86816740 1 --2- 172.21.15.94:0/3038299983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926d0fe0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009fd0 tx=0x7f9f780d34b0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.524+0000 7f9f86816740 1 -- 172.21.15.94:0/3038299983 shutdown_connections 2024-02-17T12:09:26.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.524+0000 7f9f86816740 1 --2- 172.21.15.94:0/3038299983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926d3750 0x55d7926da630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.524+0000 7f9f86816740 1 --2- 172.21.15.94:0/3038299983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926cab70 0x55d7926c3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.524+0000 7f9f86816740 1 --2- 172.21.15.94:0/3038299983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926ca250 0x55d7926d0fe0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.524+0000 7f9f86816740 1 -- 172.21.15.94:0/3038299983 >> 172.21.15.94:0/3038299983 conn(0x55d7926c4980 msgr2=0x55d7926c32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.524+0000 7f9f86816740 1 -- 172.21.15.94:0/3038299983 shutdown_connections 2024-02-17T12:09:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.524+0000 7f9f86816740 1 -- 172.21.15.94:0/3038299983 wait complete. 2024-02-17T12:09:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.524+0000 7f9f86816740 1 -- start start 2024-02-17T12:09:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.525+0000 7f9f86816740 1 Processor -- start 2024-02-17T12:09:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.525+0000 7f9f86816740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926ca250 0x55d792733440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.525+0000 7f9f86816740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926cab70 0x55d792733980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.525+0000 7f9f86816740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926d3750 0x55d7926d9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.525+0000 7f9f86816740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7926d9f20 con 0x55d7926cab70 2024-02-17T12:09:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.525+0000 7f9f811e4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926cab70 0x55d792733980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.525+0000 7f9f811e4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926cab70 0x55d792733980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33004/0 (socket says 172.21.15.94:33004) 2024-02-17T12:09:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.525+0000 7f9f811e4640 1 -- 172.21.15.94:0/2150499726 learned_addr learned my addr 172.21.15.94:0/2150499726 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.525+0000 7f9f86816740 1 -- 172.21.15.94:0/2150499726 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7926da090 con 0x55d7926ca250 2024-02-17T12:09:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.525+0000 7f9f86816740 1 -- 172.21.15.94:0/2150499726 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7926da250 con 0x55d7926d3750 2024-02-17T12:09:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.525+0000 7f9f821e6640 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926d3750 0x55d7926d9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.525+0000 7f9f86816740 10 client.0 _ll_register_callbacks cb 0x7f9f6c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.525+0000 7f9f819e5640 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926ca250 0x55d792733440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f811e4640 1 -- 172.21.15.94:0/2150499726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926d3750 msgr2=0x55d7926d9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f811e4640 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926d3750 0x55d7926d9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f811e4640 1 -- 172.21.15.94:0/2150499726 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926ca250 msgr2=0x55d792733440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f811e4640 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926ca250 0x55d792733440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f811e4640 1 -- 172.21.15.94:0/2150499726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f7c1aabd0 con 0x55d7926cab70 2024-02-17T12:09:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f821e6640 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926d3750 0x55d7926d9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f819e5640 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926ca250 0x55d792733440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f811e4640 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926cab70 0x55d792733980 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c005730 tx=0x7f9f6c019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22e56e397de3bcf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f617fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f617fa640 1 -- 172.21.15.94:0/2150499726 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c019b00 con 0x55d7926cab70 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f86816740 1 -- 172.21.15.94:0/2150499726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f6c004510 con 0x55d7926cab70 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f617fa640 1 -- 172.21.15.94:0/2150499726 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9f6c019c60 con 0x55d7926cab70 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f617fa640 1 -- 172.21.15.94:0/2150499726 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f6c019e20 con 0x55d7926cab70 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f86816740 10 client.18450 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f86816740 1 -- 172.21.15.94:0/2150499726 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d7925f28c0 con 0x55d7926cab70 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.526+0000 7f9f617fa640 1 -- 172.21.15.94:0/2150499726 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9f6c004510 con 0x55d7926cab70 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.527+0000 7f9f617fa640 1 -- 172.21.15.94:0/2150499726 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9f6c034070 con 0x55d7926cab70 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.527+0000 7f9f86816740 10 client.18450 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.527+0000 7f9f86816740 10 client.18450 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.527+0000 7f9f86816740 20 client.18450 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.527+0000 7f9f86816740 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d7926e3b60 0x55d79273e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.527+0000 7f9f86816740 4 client.18450 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:09:26.533 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.527+0000 7f9f86816740 1 -- 172.21.15.94:0/2150499726 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d7923bd890 con 0x55d7926e3b60 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.527+0000 7f9f819e5640 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d7926e3b60 0x55d79273e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.527+0000 7f9f819e5640 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d7926e3b60 0x55d79273e2d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e181a21c857b6d2 server_cookie=d559c5959c12090c in_seq=0 out_seq=0 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.527+0000 7f9f617fa640 10 client.18450 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f617fa640 1 -- 172.21.15.94:0/2150499726 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d7923bd890 con 0x55d7926e3b60 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f617fa640 10 client.18450 handle_command_reply: tid=0 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f86816740 1 client.18450 shutdown 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f86816740 1 -- 172.21.15.94:0/2150499726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926cab70 msgr2=0x55d792733980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f86816740 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926cab70 0x55d792733980 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c005730 tx=0x7f9f6c019780 comp rx=0 tx=0).stop 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f86816740 1 -- 172.21.15.94:0/2150499726 shutdown_connections 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f86816740 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55d7926e3b60 0x55d79273e2d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f86816740 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7926d3750 0x55d7926d9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f86816740 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d7926cab70 0x55d792733980 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f86816740 1 --2- 172.21.15.94:0/2150499726 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d7926ca250 0x55d792733440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f86816740 1 -- 172.21.15.94:0/2150499726 >> 172.21.15.94:0/2150499726 conn(0x55d7926c4980 msgr2=0x55d7926c3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f617fa640 0 client.18450 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:09:26.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f86816740 1 -- 172.21.15.94:0/2150499726 shutdown_connections 2024-02-17T12:09:26.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f86816740 1 -- 172.21.15.94:0/2150499726 wait complete. 2024-02-17T12:09:26.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.528+0000 7f9f86816740 20 client.18450 trim_cache size 0 max 16384 2024-02-17T12:09:26.535 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:09:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.529+0000 7f9f83c49640 1 -- 172.21.15.94:0/2409044963 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9f640a2440 msgr2=0x7f9f640a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.529+0000 7f9f83c49640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9f640a2440 0x7f9f640a4900 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f7403d780 tx=0x7f9f7401a5d0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.529+0000 7f9f83c49640 1 -- 172.21.15.94:0/2409044963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f7c10d9d0 msgr2=0x7f9f7c11a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.529+0000 7f9f83c49640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f7c10d9d0 0x7f9f7c11a070 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800b450 tx=0x7f9f7800b910 comp rx=0 tx=0).stop 2024-02-17T12:09:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.529+0000 7f9f83c49640 1 -- 172.21.15.94:0/2409044963 shutdown_connections 2024-02-17T12:09:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.529+0000 7f9f83c49640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9f640a2440 0x7f9f640a4900 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.529+0000 7f9f83c49640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9f7c11a960 0x7f9f7c116640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.529+0000 7f9f83c49640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9f7c10f9a0 0x7f9f7c116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.529+0000 7f9f83c49640 1 --2- 172.21.15.94:0/2409044963 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f7c10d9d0 0x7f9f7c11a070 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.529+0000 7f9f83c49640 1 -- 172.21.15.94:0/2409044963 >> 172.21.15.94:0/2409044963 conn(0x7f9f7c06c260 msgr2=0x7f9f7c11c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.529+0000 7f9f83c49640 1 -- 172.21.15.94:0/2409044963 shutdown_connections 2024-02-17T12:09:26.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.529+0000 7f9f83c49640 1 -- 172.21.15.94:0/2409044963 wait complete. 2024-02-17T12:09:26.542 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:09:26.543 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:09:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.673+0000 7fe7ac125640 1 -- 172.21.15.94:0/3978777789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a40ff890 msgr2=0x7fe7a40ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.673+0000 7fe7ac125640 1 --2- 172.21.15.94:0/3978777789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a40ff890 0x7fe7a40ffc70 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009930 tx=0x7fe7a002bc30 comp rx=0 tx=0).stop 2024-02-17T12:09:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.673+0000 7fe7ac125640 1 -- 172.21.15.94:0/3978777789 shutdown_connections 2024-02-17T12:09:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.673+0000 7fe7ac125640 1 --2- 172.21.15.94:0/3978777789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7a4106450 0x7fe7a410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.673+0000 7fe7ac125640 1 --2- 172.21.15.94:0/3978777789 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7a4100240 0x7fe7a4105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.673+0000 7fe7ac125640 1 --2- 172.21.15.94:0/3978777789 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a40ff890 0x7fe7a40ffc70 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.673+0000 7fe7ac125640 1 -- 172.21.15.94:0/3978777789 >> 172.21.15.94:0/3978777789 conn(0x7fe7a40765a0 msgr2=0x7fe7a40fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.673+0000 7fe7ac125640 1 -- 172.21.15.94:0/3978777789 shutdown_connections 2024-02-17T12:09:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.673+0000 7fe7ac125640 1 -- 172.21.15.94:0/3978777789 wait complete. 2024-02-17T12:09:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.673+0000 7fe7ac125640 1 Processor -- start 2024-02-17T12:09:26.677 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7ac125640 1 -- start start 2024-02-17T12:09:26.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7ac125640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a40ff890 0x7fe7a419d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7ac125640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7a4106450 0x7fe7a419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7ac125640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7a4197640 0x7fe7a4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7ac125640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a4198110 con 0x7fe7a40ff890 2024-02-17T12:09:26.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7ac125640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a4198280 con 0x7fe7a4106450 2024-02-17T12:09:26.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7ac125640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a4198440 con 0x7fe7a4197640 2024-02-17T12:09:26.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7a9ec1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a40ff890 0x7fe7a419d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7a9ec1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a40ff890 0x7fe7a419d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33020/0 (socket says 172.21.15.94:33020) 2024-02-17T12:09:26.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7a9ec1640 1 -- 172.21.15.94:0/2843541139 learned_addr learned my addr 172.21.15.94:0/2843541139 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7a96c0640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7a4106450 0x7fe7a419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7a4197640 0x7fe7a4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7a9ec1640 1 -- 172.21.15.94:0/2843541139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7a4197640 msgr2=0x7fe7a4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7a9ec1640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7a4197640 0x7fe7a4197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7a9ec1640 1 -- 172.21.15.94:0/2843541139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7a4106450 msgr2=0x7fe7a419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7a9ec1640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7a4106450 0x7fe7a419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7a9ec1640 1 -- 172.21.15.94:0/2843541139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a0009560 con 0x7fe7a40ff890 2024-02-17T12:09:26.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7a9ec1640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a40ff890 0x7fe7a419d580 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009230 tx=0x7fe7a002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe79affd640 1 -- 172.21.15.94:0/2843541139 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0004350 con 0x7fe7a40ff890 2024-02-17T12:09:26.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe79affd640 1 -- 172.21.15.94:0/2843541139 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe7a00044b0 con 0x7fe7a40ff890 2024-02-17T12:09:26.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7ac125640 1 -- 172.21.15.94:0/2843541139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a41a4380 con 0x7fe7a40ff890 2024-02-17T12:09:26.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe79affd640 1 -- 172.21.15.94:0/2843541139 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0003d00 con 0x7fe7a40ff890 2024-02-17T12:09:26.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.674+0000 7fe7ac125640 1 -- 172.21.15.94:0/2843541139 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a41a4860 con 0x7fe7a40ff890 2024-02-17T12:09:26.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe7aedd5740 1 Processor -- start 2024-02-17T12:09:26.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe7aedd5740 1 -- start start 2024-02-17T12:09:26.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe79affd640 1 -- 172.21.15.94:0/2843541139 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe7a0004df0 con 0x7fe7a40ff890 2024-02-17T12:09:26.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe7aedd5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68bf8f0 0x561ad68bfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe7aedd5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68c02a0 0x561ad68ac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe7aedd5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68aca50 0x561ad68aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe7aedd5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad686d960 con 0x561ad68aca50 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe7aedd5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad6821310 con 0x561ad68bf8f0 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe7aedd5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad681e840 con 0x561ad68c02a0 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe7a96c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68aca50 0x561ad68aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe7aa6c2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68bf8f0 0x561ad68bfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe7aa6c2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68bf8f0 0x561ad68bfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:49004/0 (socket says 172.21.15.94:49004) 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe7aa6c2640 1 -- 172.21.15.94:0/2026686901 learned_addr learned my addr 172.21.15.94:0/2026686901 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.677+0000 7fe7a9ec1640 1 --2- 172.21.15.94:0/2026686901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68c02a0 0x561ad68ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.678+0000 7fe7aa6c2640 1 -- 172.21.15.94:0/2026686901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68c02a0 msgr2=0x561ad68ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.678+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/2026686901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68c02a0 0x561ad68ac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.678+0000 7fe7aa6c2640 1 -- 172.21.15.94:0/2026686901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68aca50 msgr2=0x561ad68aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.678+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/2026686901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68aca50 0x561ad68aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.678+0000 7fe7aa6c2640 1 -- 172.21.15.94:0/2026686901 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ad67dc060 con 0x561ad68bf8f0 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.678+0000 7fe7a96c0640 1 --2- 172.21.15.94:0/2026686901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68aca50 0x561ad68aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:26.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.678+0000 7fe79affd640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe7900a2230 0x7fe7900a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.678+0000 7fe7a9ec1640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe7900a2230 0x7fe7900a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.678+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/2026686901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68bf8f0 0x561ad68bfcd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe79c009f40 tx=0x7fe79c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46f89db4bc924312 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.678+0000 7fe79affd640 1 -- 172.21.15.94:0/2843541139 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe7a00f0050 con 0x7fe7a40ff890 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.678+0000 7fe78ffff640 1 -- 172.21.15.94:0/2026686901 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c009a90 con 0x561ad68bf8f0 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.678+0000 7fe78ffff640 1 -- 172.21.15.94:0/2026686901 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe79c015070 con 0x561ad68bf8f0 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.678+0000 7fe78ffff640 1 -- 172.21.15.94:0/2026686901 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c011870 con 0x561ad68bf8f0 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.679+0000 7fe7aedd5740 1 -- 172.21.15.94:0/2026686901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68bf8f0 msgr2=0x561ad68bfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.679+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/2026686901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68bf8f0 0x561ad68bfcd0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe79c009f40 tx=0x7fe79c010860 comp rx=0 tx=0).stop 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.679+0000 7fe7a9ec1640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe7900a2230 0x7fe7900a46f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00ed920 tx=0x7fe7a00037d0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.679+0000 7fe7aedd5740 1 -- 172.21.15.94:0/2026686901 shutdown_connections 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.679+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/2026686901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68aca50 0x561ad68aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.679+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/2026686901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68c02a0 0x561ad68ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.679+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/2026686901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68bf8f0 0x561ad68bfcd0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.679+0000 7fe7aedd5740 1 -- 172.21.15.94:0/2026686901 >> 172.21.15.94:0/2026686901 conn(0x561ad68aeef0 msgr2=0x561ad68af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.679+0000 7fe7aedd5740 1 -- 172.21.15.94:0/2026686901 shutdown_connections 2024-02-17T12:09:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.679+0000 7fe7aedd5740 1 -- 172.21.15.94:0/2026686901 wait complete. 2024-02-17T12:09:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.679+0000 7fe7aedd5740 1 -- start start 2024-02-17T12:09:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.679+0000 7fe7aedd5740 1 Processor -- start 2024-02-17T12:09:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7aedd5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68aca50 0x561ad68c3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7aedd5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 0x561ad68c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7aedd5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68c02a0 0x561ad68c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7aedd5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad681a020 con 0x561ad68bf8f0 2024-02-17T12:09:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7aedd5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad64ed430 con 0x561ad68c02a0 2024-02-17T12:09:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7aedd5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad64a9e00 con 0x561ad68aca50 2024-02-17T12:09:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7aedd5740 10 client.0 _ll_register_callbacks cb 0x561ad67c1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:26.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7a96c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 0x561ad68c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7a96c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 0x561ad68c3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33048/0 (socket says 172.21.15.94:33048) 2024-02-17T12:09:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7a96c0640 1 -- 172.21.15.94:0/2663059898 learned_addr learned my addr 172.21.15.94:0/2663059898 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68c02a0 0x561ad68c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68aca50 0x561ad68c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7a96c0640 1 -- 172.21.15.94:0/2663059898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68aca50 msgr2=0x561ad68c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7a96c0640 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68aca50 0x561ad68c3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7a96c0640 1 -- 172.21.15.94:0/2663059898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68c02a0 msgr2=0x561ad68c43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7a96c0640 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68c02a0 0x561ad68c43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7a96c0640 1 -- 172.21.15.94:0/2663059898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c009bf0 con 0x561ad68bf8f0 2024-02-17T12:09:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68c02a0 0x561ad68c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe7a96c0640 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 0x561ad68c3e60 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe794012700 tx=0x7fe794012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbbc7a0456fe98e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.680+0000 7fe78d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe78d7fa640 1 -- 172.21.15.94:0/2663059898 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794017070 con 0x561ad68bf8f0 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe7aedd5740 1 -- 172.21.15.94:0/2663059898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ad65197e0 con 0x561ad68bf8f0 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe78d7fa640 1 -- 172.21.15.94:0/2663059898 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe794004030 con 0x561ad68bf8f0 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe78d7fa640 1 -- 172.21.15.94:0/2663059898 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe794004230 con 0x561ad68bf8f0 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe7aedd5740 10 client.18462 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe7aedd5740 1 -- 172.21.15.94:0/2663059898 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561ad68186a0 con 0x561ad68bf8f0 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe78d7fa640 1 -- 172.21.15.94:0/2663059898 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe794028080 con 0x561ad68bf8f0 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe78d7fa640 1 -- 172.21.15.94:0/2663059898 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe794003980 con 0x561ad68bf8f0 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe7aedd5740 10 client.18462 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe7aedd5740 10 client.18462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe7aedd5740 20 client.18462 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561ad68d4530 0x561ad68d6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe7aedd5740 4 client.18462 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:26.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe7aedd5740 1 -- 172.21.15.94:0/2663059898 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ad6556720 con 0x561ad68d4530 2024-02-17T12:09:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.681+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561ad68d4530 0x561ad68d6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.682+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561ad68d4530 0x561ad68d6950 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6059cf06cee7d674 server_cookie=db2d79a42067c85d in_seq=0 out_seq=0 2024-02-17T12:09:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.682+0000 7fe78d7fa640 10 client.18462 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.682+0000 7fe78d7fa640 1 -- 172.21.15.94:0/2663059898 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ad6556720 con 0x561ad68d4530 2024-02-17T12:09:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.682+0000 7fe78d7fa640 10 client.18462 handle_command_reply: tid=0 2024-02-17T12:09:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.682+0000 7fe7aedd5740 1 client.18462 shutdown 2024-02-17T12:09:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.683+0000 7fe7aedd5740 1 -- 172.21.15.94:0/2663059898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 msgr2=0x561ad68c3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.683+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 0x561ad68c3e60 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fe794012700 tx=0x7fe794012bc0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.683+0000 7fe7aedd5740 1 -- 172.21.15.94:0/2663059898 shutdown_connections 2024-02-17T12:09:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.683+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561ad68d4530 0x561ad68d6950 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.683+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68c02a0 0x561ad68c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.691 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.683+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 0x561ad68c3e60 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.683+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/2663059898 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68aca50 0x561ad68c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.683+0000 7fe7aedd5740 1 -- 172.21.15.94:0/2663059898 >> 172.21.15.94:0/2663059898 conn(0x561ad68aeef0 msgr2=0x561ad68addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.683+0000 7fe78d7fa640 0 client.18462 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.683+0000 7fe7aedd5740 1 -- 172.21.15.94:0/2663059898 shutdown_connections 2024-02-17T12:09:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.683+0000 7fe7aedd5740 1 -- 172.21.15.94:0/2663059898 wait complete. 2024-02-17T12:09:26.692 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.683+0000 7fe7aedd5740 20 client.18462 trim_cache size 0 max 16384 2024-02-17T12:09:26.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aedd5740 1 Processor -- start 2024-02-17T12:09:26.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aedd5740 1 -- start start 2024-02-17T12:09:26.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aedd5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68aca50 0x561ad68c6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aedd5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68bf8f0 0x561ad68c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aedd5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68c02a0 0x561ad68db2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aedd5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad679f130 con 0x561ad68c02a0 2024-02-17T12:09:26.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aedd5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad68a85c0 con 0x561ad68bf8f0 2024-02-17T12:09:26.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aedd5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad68ad2e0 con 0x561ad68aca50 2024-02-17T12:09:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aa6c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68c02a0 0x561ad68db2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aa6c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68c02a0 0x561ad68db2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33052/0 (socket says 172.21.15.94:33052) 2024-02-17T12:09:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aa6c2640 1 -- 172.21.15.94:0/1554756094 learned_addr learned my addr 172.21.15.94:0/1554756094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/1554756094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68aca50 0x561ad68c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7a96c0640 1 --2- 172.21.15.94:0/1554756094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68bf8f0 0x561ad68c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aa6c2640 1 -- 172.21.15.94:0/1554756094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68aca50 msgr2=0x561ad68c6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/1554756094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68aca50 0x561ad68c6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aa6c2640 1 -- 172.21.15.94:0/1554756094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68bf8f0 msgr2=0x561ad68c41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.697+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/1554756094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68bf8f0 0x561ad68c41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.698+0000 7fe7aa6c2640 1 -- 172.21.15.94:0/1554756094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ad67b4c20 con 0x561ad68c02a0 2024-02-17T12:09:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.698+0000 7fe7a96c0640 1 --2- 172.21.15.94:0/1554756094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68bf8f0 0x561ad68c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:09:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.698+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/1554756094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68c02a0 0x561ad68db2b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe79c004890 tx=0x7fe79c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd87b3493a28a32 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.698+0000 7fe78f7fe640 1 -- 172.21.15.94:0/1554756094 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c02b3d0 con 0x561ad68c02a0 2024-02-17T12:09:26.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.698+0000 7fe78f7fe640 1 -- 172.21.15.94:0/1554756094 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe79c011920 con 0x561ad68c02a0 2024-02-17T12:09:26.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.698+0000 7fe78f7fe640 1 -- 172.21.15.94:0/1554756094 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c01b8b0 con 0x561ad68c02a0 2024-02-17T12:09:26.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.699+0000 7fe7aedd5740 1 -- 172.21.15.94:0/1554756094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68c02a0 msgr2=0x561ad68db2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.699+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/1554756094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68c02a0 0x561ad68db2b0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fe79c004890 tx=0x7fe79c010860 comp rx=0 tx=0).stop 2024-02-17T12:09:26.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.699+0000 7fe7aedd5740 1 -- 172.21.15.94:0/1554756094 shutdown_connections 2024-02-17T12:09:26.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.699+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/1554756094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68c02a0 0x561ad68db2b0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.699+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/1554756094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68bf8f0 0x561ad68c41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.699+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/1554756094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68aca50 0x561ad68c6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.699+0000 7fe7aedd5740 1 -- 172.21.15.94:0/1554756094 >> 172.21.15.94:0/1554756094 conn(0x561ad68aeef0 msgr2=0x561ad68bdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.699+0000 7fe7aedd5740 1 -- 172.21.15.94:0/1554756094 shutdown_connections 2024-02-17T12:09:26.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.699+0000 7fe7aedd5740 1 -- 172.21.15.94:0/1554756094 wait complete. 2024-02-17T12:09:26.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.699+0000 7fe7aedd5740 1 -- start start 2024-02-17T12:09:26.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.699+0000 7fe7aedd5740 1 Processor -- start 2024-02-17T12:09:26.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7aedd5740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68aca50 0x561ad68cb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7aedd5740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 0x561ad68c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7aedd5740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68c02a0 0x561ad68c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7aedd5740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad68ca520 con 0x561ad68bf8f0 2024-02-17T12:09:26.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7aedd5740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad692aa40 con 0x561ad68aca50 2024-02-17T12:09:26.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7aedd5740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ad692ab80 con 0x561ad68c02a0 2024-02-17T12:09:26.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7aedd5740 10 client.0 _ll_register_callbacks cb 0x7fe79c0099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:26.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7a96c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 0x561ad68c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7a96c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 0x561ad68c96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33056/0 (socket says 172.21.15.94:33056) 2024-02-17T12:09:26.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7a96c0640 1 -- 172.21.15.94:0/1110995624 learned_addr learned my addr 172.21.15.94:0/1110995624 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:26.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68aca50 0x561ad68cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68c02a0 0x561ad68c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7a96c0640 1 -- 172.21.15.94:0/1110995624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68c02a0 msgr2=0x561ad68c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7a96c0640 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68c02a0 0x561ad68c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7a96c0640 1 -- 172.21.15.94:0/1110995624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68aca50 msgr2=0x561ad68cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:26.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7a96c0640 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68aca50 0x561ad68cb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7a96c0640 1 -- 172.21.15.94:0/1110995624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c023040 con 0x561ad68bf8f0 2024-02-17T12:09:26.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68c02a0 0x561ad68c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:09:26.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe7a96c0640 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 0x561ad68c96e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x561ad68c4ea0 tx=0x7fe7940096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914d1a2bf4b0754e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.700+0000 7fe78d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe78d7fa640 1 -- 172.21.15.94:0/1110995624 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794017040 con 0x561ad68bf8f0 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe78d7fa640 1 -- 172.21.15.94:0/1110995624 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe794000e40 con 0x561ad68bf8f0 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe7aedd5740 1 -- 172.21.15.94:0/1110995624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561ad68af870 con 0x561ad68bf8f0 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe78d7fa640 1 -- 172.21.15.94:0/1110995624 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7940052f0 con 0x561ad68bf8f0 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe7aedd5740 10 client.18468 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe7aedd5740 1 -- 172.21.15.94:0/1110995624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561ad67da8c0 con 0x561ad68bf8f0 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe78d7fa640 1 -- 172.21.15.94:0/1110995624 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561ad68af870 con 0x561ad68bf8f0 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe78d7fa640 1 -- 172.21.15.94:0/1110995624 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fe794033070 con 0x561ad68bf8f0 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe7aedd5740 10 client.18468 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe7aedd5740 10 client.18468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe7aedd5740 20 client.18468 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561ad692c2e0 0x561ad692e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:26.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe7aedd5740 4 client.18468 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:09:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe7aedd5740 1 -- 172.21.15.94:0/1110995624 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe79c015070 con 0x561ad692c2e0 2024-02-17T12:09:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.701+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561ad692c2e0 0x561ad692e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe7aa6c2640 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561ad692c2e0 0x561ad692e700 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228638b3a9d0e519 server_cookie=51120ea3acb887d in_seq=0 out_seq=0 2024-02-17T12:09:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe78d7fa640 10 client.18468 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe78d7fa640 1 -- 172.21.15.94:0/1110995624 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe79c015070 con 0x561ad692c2e0 2024-02-17T12:09:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe78d7fa640 10 client.18468 handle_command_reply: tid=0 2024-02-17T12:09:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe7aedd5740 1 client.18468 shutdown 2024-02-17T12:09:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe7aedd5740 1 -- 172.21.15.94:0/1110995624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 msgr2=0x561ad68c96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 0x561ad68c96e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x561ad68c4ea0 tx=0x7fe7940096e0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe7aedd5740 1 -- 172.21.15.94:0/1110995624 shutdown_connections 2024-02-17T12:09:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561ad692c2e0 0x561ad692e700 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561ad68c02a0 0x561ad68c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561ad68bf8f0 0x561ad68c96e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe78d7fa640 0 client.18468 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:26.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe7aedd5740 1 --2- 172.21.15.94:0/1110995624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561ad68aca50 0x561ad68cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe7aedd5740 1 -- 172.21.15.94:0/1110995624 >> 172.21.15.94:0/1110995624 conn(0x561ad68aeef0 msgr2=0x561ad68db2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe7aedd5740 1 -- 172.21.15.94:0/1110995624 shutdown_connections 2024-02-17T12:09:26.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe7aedd5740 1 -- 172.21.15.94:0/1110995624 wait complete. 2024-02-17T12:09:26.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.702+0000 7fe7aedd5740 20 client.18468 trim_cache size 0 max 16384 2024-02-17T12:09:26.711 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:09:26.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.703+0000 7fe7ac125640 1 -- 172.21.15.94:0/2843541139 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe7900a2230 msgr2=0x7fe7900a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.703+0000 7fe7ac125640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe7900a2230 0x7fe7900a46f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00ed920 tx=0x7fe7a00037d0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.703+0000 7fe7ac125640 1 -- 172.21.15.94:0/2843541139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a40ff890 msgr2=0x7fe7a419d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:26.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.703+0000 7fe7ac125640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a40ff890 0x7fe7a419d580 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009230 tx=0x7fe7a002c650 comp rx=0 tx=0).stop 2024-02-17T12:09:26.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.703+0000 7fe7ac125640 1 -- 172.21.15.94:0/2843541139 shutdown_connections 2024-02-17T12:09:26.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.703+0000 7fe7ac125640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe7900a2230 0x7fe7900a46f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.703+0000 7fe7ac125640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe7a4197640 0x7fe7a4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.703+0000 7fe7ac125640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe7a4106450 0x7fe7a419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.703+0000 7fe7ac125640 1 --2- 172.21.15.94:0/2843541139 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe7a40ff890 0x7fe7a419d580 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:26.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.703+0000 7fe7ac125640 1 -- 172.21.15.94:0/2843541139 >> 172.21.15.94:0/2843541139 conn(0x7fe7a40765a0 msgr2=0x7fe7a41046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:26.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.703+0000 7fe7ac125640 1 -- 172.21.15.94:0/2843541139 shutdown_connections 2024-02-17T12:09:26.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:26.703+0000 7fe7ac125640 1 -- 172.21.15.94:0/2843541139 wait complete. 2024-02-17T12:09:27.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:27 smithi189 ceph-mon[36588]: pgmap v353: 97 pgs: 97 active+clean; 5.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.3 MiB/s wr, 574 op/s 2024-02-17T12:09:27.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:27 smithi125 ceph-mon[36819]: pgmap v353: 97 pgs: 97 active+clean; 5.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.3 MiB/s wr, 574 op/s 2024-02-17T12:09:27.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:27 smithi094 ceph-mon[32499]: pgmap v353: 97 pgs: 97 active+clean; 5.4 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.3 MiB/s wr, 574 op/s 2024-02-17T12:09:27.717 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:09:27.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:09:27.718 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:09:27.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.908+0000 7f1680a4e640 1 -- 172.21.15.94:0/2336249586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c0ff8f0 msgr2=0x7f167c0ffcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:27.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.908+0000 7f1680a4e640 1 --2- 172.21.15.94:0/2336249586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c0ff8f0 0x7f167c0ffcd0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f166c009900 tx=0x7f166c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:09:27.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.908+0000 7f1680a4e640 1 -- 172.21.15.94:0/2336249586 shutdown_connections 2024-02-17T12:09:27.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.908+0000 7f1680a4e640 1 --2- 172.21.15.94:0/2336249586 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f167c106500 0x7f167c10e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.910 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.908+0000 7f1680a4e640 1 --2- 172.21.15.94:0/2336249586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f167c1002a0 0x7f167c105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.908+0000 7f1680a4e640 1 --2- 172.21.15.94:0/2336249586 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c0ff8f0 0x7f167c0ffcd0 secure :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f166c009900 tx=0x7f166c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:09:27.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.908+0000 7f1680a4e640 1 -- 172.21.15.94:0/2336249586 >> 172.21.15.94:0/2336249586 conn(0x7f167c0765a0 msgr2=0x7f167c0fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:27.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.909+0000 7f1680a4e640 1 -- 172.21.15.94:0/2336249586 shutdown_connections 2024-02-17T12:09:27.911 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.909+0000 7f1680a4e640 1 -- 172.21.15.94:0/2336249586 wait complete. 2024-02-17T12:09:27.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.910+0000 7f1680a4e640 1 Processor -- start 2024-02-17T12:09:27.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.910+0000 7f1680a4e640 1 -- start start 2024-02-17T12:09:27.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.910+0000 7f1680a4e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f167c106500 0x7f167c19d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.910+0000 7f1680a4e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f167c19dba0 0x7f167c1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.910+0000 7f1680a4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c197d10 0x7f167c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.910+0000 7f1680a4e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c19e240 con 0x7f167c197d10 2024-02-17T12:09:27.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.910+0000 7f1680a4e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c1a44b0 con 0x7f167c106500 2024-02-17T12:09:27.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.910+0000 7f1680a4e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c1a4620 con 0x7f167c19dba0 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f167ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c197d10 0x7f167c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f167ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c197d10 0x7f167c198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33074/0 (socket says 172.21.15.94:33074) 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f1673fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f167c19dba0 0x7f167c1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f167ad9d640 1 -- 172.21.15.94:0/1768373675 learned_addr learned my addr 172.21.15.94:0/1768373675 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f167a59c640 1 --2- 172.21.15.94:0/1768373675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f167c106500 0x7f167c19d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f167ad9d640 1 -- 172.21.15.94:0/1768373675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f167c19dba0 msgr2=0x7f167c1977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f167ad9d640 1 --2- 172.21.15.94:0/1768373675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f167c19dba0 0x7f167c1977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f167ad9d640 1 -- 172.21.15.94:0/1768373675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f167c106500 msgr2=0x7f167c19d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f167ad9d640 1 --2- 172.21.15.94:0/1768373675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f167c106500 0x7f167c19d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f167ad9d640 1 -- 172.21.15.94:0/1768373675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c009560 con 0x7f167c197d10 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f167ad9d640 1 --2- 172.21.15.94:0/1768373675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c197d10 0x7f167c198190 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f167400eaa0 tx=0x7f167400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f16737fe640 1 -- 172.21.15.94:0/1768373675 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167400cdb0 con 0x7f167c197d10 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f16737fe640 1 -- 172.21.15.94:0/1768373675 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1674005020 con 0x7f167c197d10 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.911+0000 7f16737fe640 1 -- 172.21.15.94:0/1768373675 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16740116b0 con 0x7f167c197d10 2024-02-17T12:09:27.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.912+0000 7f1680a4e640 1 -- 172.21.15.94:0/1768373675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167c1a4900 con 0x7f167c197d10 2024-02-17T12:09:27.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.912+0000 7f1680a4e640 1 -- 172.21.15.94:0/1768373675 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167c1a4d60 con 0x7f167c197d10 2024-02-17T12:09:27.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.914+0000 7f16737fe640 1 -- 172.21.15.94:0/1768373675 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1674005520 con 0x7f167c197d10 2024-02-17T12:09:27.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.914+0000 7f16737fe640 1 --2- 172.21.15.94:0/1768373675 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f16680a2230 0x7f16680a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.914+0000 7f167a59c640 1 --2- 172.21.15.94:0/1768373675 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f16680a2230 0x7f16680a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.914+0000 7f16737fe640 1 -- 172.21.15.94:0/1768373675 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f16740ce260 con 0x7f167c197d10 2024-02-17T12:09:27.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.916+0000 7f167a59c640 1 --2- 172.21.15.94:0/1768373675 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f16680a2230 0x7f16680a46f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f166c009230 tx=0x7f166c005e80 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:27.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.916+0000 7f1683624740 1 Processor -- start 2024-02-17T12:09:27.918 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.916+0000 7f1683624740 1 -- start start 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f1683624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 0x55f84cf13760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f1683624740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 0x55f84cf1c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f1683624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 0x55f84cf1ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f1683624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84ce81f90 con 0x55f84cf1c8b0 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f1683624740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84ce7f4c0 con 0x55f84cf13d30 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f1683624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84ce7aca0 con 0x55f84cf13380 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f167ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 0x55f84cf1ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f167ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 0x55f84cf1ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33082/0 (socket says 172.21.15.94:33082) 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f167ad9d640 1 -- 172.21.15.94:0/1312483374 learned_addr learned my addr 172.21.15.94:0/1312483374 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f1673fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 0x55f84cf1c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f167a59c640 1 --2- 172.21.15.94:0/1312483374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 0x55f84cf13760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f167ad9d640 1 -- 172.21.15.94:0/1312483374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 msgr2=0x55f84cf13760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f167ad9d640 1 --2- 172.21.15.94:0/1312483374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 0x55f84cf13760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.920 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f167ad9d640 1 -- 172.21.15.94:0/1312483374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 msgr2=0x55f84cf1c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:27.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f167ad9d640 1 --2- 172.21.15.94:0/1312483374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 0x55f84cf1c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f167ad9d640 1 -- 172.21.15.94:0/1312483374 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f84ce3d0b0 con 0x55f84cf1c8b0 2024-02-17T12:09:27.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.917+0000 7f1673fff640 1 --2- 172.21.15.94:0/1312483374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 0x55f84cf1c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:27.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.918+0000 7f167ad9d640 1 --2- 172.21.15.94:0/1312483374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 0x55f84cf1ed00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1674097690 tx=0x7f167409ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48b7b87b08b65c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:27.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.918+0000 7f1670ff9640 1 -- 172.21.15.94:0/1312483374 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16740ce070 con 0x55f84cf1c8b0 2024-02-17T12:09:27.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.918+0000 7f1670ff9640 1 -- 172.21.15.94:0/1312483374 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f16740cbbf0 con 0x55f84cf1c8b0 2024-02-17T12:09:27.921 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.918+0000 7f1670ff9640 1 -- 172.21.15.94:0/1312483374 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167409a370 con 0x55f84cf1c8b0 2024-02-17T12:09:27.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.918+0000 7f1683624740 1 -- 172.21.15.94:0/1312483374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 msgr2=0x55f84cf1ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:27.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.918+0000 7f1683624740 1 --2- 172.21.15.94:0/1312483374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 0x55f84cf1ed00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1674097690 tx=0x7f167409ed60 comp rx=0 tx=0).stop 2024-02-17T12:09:27.925 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 -- 172.21.15.94:0/1312483374 shutdown_connections 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 --2- 172.21.15.94:0/1312483374 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 0x55f84cf1ed00 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 --2- 172.21.15.94:0/1312483374 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 0x55f84cf1c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 --2- 172.21.15.94:0/1312483374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 0x55f84cf13760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 -- 172.21.15.94:0/1312483374 >> 172.21.15.94:0/1312483374 conn(0x55f84cf0d840 msgr2=0x55f84cf0dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 -- 172.21.15.94:0/1312483374 shutdown_connections 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 -- 172.21.15.94:0/1312483374 wait complete. 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 -- start start 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 Processor -- start 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf13380 0x55f84cf19880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 0x55f84cf19dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf1c8b0 0x55f84cf1a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84ce1ff40 con 0x55f84cf13380 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84cb4e4b0 con 0x55f84cf13d30 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84ce00670 con 0x55f84cf1c8b0 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1683624740 10 client.0 _ll_register_callbacks cb 0x55f84ce6df70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:27.926 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f167a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf13380 0x55f84cf19880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f167a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf13380 0x55f84cf19880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33088/0 (socket says 172.21.15.94:33088) 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f167a59c640 1 -- 172.21.15.94:0/280979068 learned_addr learned my addr 172.21.15.94:0/280979068 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.919+0000 7f1673fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 0x55f84cf19dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f167ad9d640 1 --2- 172.21.15.94:0/280979068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf1c8b0 0x55f84cf1a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f167a59c640 1 -- 172.21.15.94:0/280979068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf1c8b0 msgr2=0x55f84cf1a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f167a59c640 1 --2- 172.21.15.94:0/280979068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf1c8b0 0x55f84cf1a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f167a59c640 1 -- 172.21.15.94:0/280979068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 msgr2=0x55f84cf19dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f167a59c640 1 --2- 172.21.15.94:0/280979068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 0x55f84cf19dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f167a59c640 1 -- 172.21.15.94:0/280979068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167c1a4d60 con 0x55f84cf13380 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f167a59c640 1 --2- 172.21.15.94:0/280979068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf13380 0x55f84cf19880 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f166c009900 tx=0x7f166c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17167cf52e9f3af server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f1661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f1661ffb640 1 -- 172.21.15.94:0/280979068 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166c0044d0 con 0x55f84cf13380 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f1661ffb640 1 -- 172.21.15.94:0/280979068 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f166c047040 con 0x55f84cf13380 2024-02-17T12:09:27.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f1683624740 1 -- 172.21.15.94:0/280979068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f84ce54f40 con 0x55f84cf13380 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f1661ffb640 1 -- 172.21.15.94:0/280979068 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f166c004df0 con 0x55f84cf13380 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f1683624740 10 client.18480 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f1683624740 1 -- 172.21.15.94:0/280979068 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f84ce79320 con 0x55f84cf13380 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.920+0000 7f1661ffb640 1 -- 172.21.15.94:0/280979068 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f84ce54f40 con 0x55f84cf13380 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.921+0000 7f1661ffb640 1 -- 172.21.15.94:0/280979068 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f166c003bc0 con 0x55f84cf13380 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.921+0000 7f1683624740 10 client.18480 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.921+0000 7f1683624740 10 client.18480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.921+0000 7f1683624740 20 client.18480 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.921+0000 7f1683624740 1 --2- 172.21.15.94:0/280979068 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f84cf2cea0 0x55f84cf2f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.921+0000 7f1683624740 4 client.18480 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.921+0000 7f1683624740 1 -- 172.21.15.94:0/280979068 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f84ce71310 con 0x55f84cf2cea0 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.921+0000 7f1673fff640 1 --2- 172.21.15.94:0/280979068 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f84cf2cea0 0x55f84cf2f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.922+0000 7f1673fff640 1 --2- 172.21.15.94:0/280979068 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f84cf2cea0 0x55f84cf2f2c0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac67dadb7b0653fc server_cookie=491812c872935908 in_seq=0 out_seq=0 2024-02-17T12:09:27.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.922+0000 7f1661ffb640 10 client.18480 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.922+0000 7f1661ffb640 1 -- 172.21.15.94:0/280979068 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f84ce71310 con 0x55f84cf2cea0 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.922+0000 7f1661ffb640 10 client.18480 handle_command_reply: tid=0 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.922+0000 7f1683624740 1 client.18480 shutdown 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.922+0000 7f1683624740 1 -- 172.21.15.94:0/280979068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf13380 msgr2=0x55f84cf19880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.922+0000 7f1683624740 1 --2- 172.21.15.94:0/280979068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf13380 0x55f84cf19880 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f166c009900 tx=0x7f166c005fd0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.923+0000 7f1683624740 1 -- 172.21.15.94:0/280979068 shutdown_connections 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.923+0000 7f1683624740 1 --2- 172.21.15.94:0/280979068 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f84cf2cea0 0x55f84cf2f2c0 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.923+0000 7f1683624740 1 --2- 172.21.15.94:0/280979068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf1c8b0 0x55f84cf1a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.923+0000 7f1661ffb640 0 client.18480 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.923+0000 7f1683624740 1 --2- 172.21.15.94:0/280979068 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 0x55f84cf19dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.923+0000 7f1683624740 1 --2- 172.21.15.94:0/280979068 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf13380 0x55f84cf19880 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.923+0000 7f1683624740 1 -- 172.21.15.94:0/280979068 >> 172.21.15.94:0/280979068 conn(0x55f84cf0d840 msgr2=0x55f84cf0f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.923+0000 7f1683624740 1 -- 172.21.15.94:0/280979068 shutdown_connections 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.923+0000 7f1683624740 1 -- 172.21.15.94:0/280979068 wait complete. 2024-02-17T12:09:27.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.924+0000 7f1683624740 20 client.18480 trim_cache size 0 max 16384 2024-02-17T12:09:27.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.948+0000 7f1683624740 1 Processor -- start 2024-02-17T12:09:27.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.948+0000 7f1683624740 1 -- start start 2024-02-17T12:09:27.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.948+0000 7f1683624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 0x55f84cf1a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.948+0000 7f1683624740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 0x55f84cf238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.948+0000 7f1683624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 0x55f84cf23e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.948+0000 7f1683624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84cf29970 con 0x55f84cf1c8b0 2024-02-17T12:09:27.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.948+0000 7f1683624740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84cf0e410 con 0x55f84cf13d30 2024-02-17T12:09:27.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.948+0000 7f1683624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84cf126e0 con 0x55f84cf13380 2024-02-17T12:09:27.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.949+0000 7f167ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 0x55f84cf23e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.949+0000 7f167ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 0x55f84cf23e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33090/0 (socket says 172.21.15.94:33090) 2024-02-17T12:09:27.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.949+0000 7f167ad9d640 1 -- 172.21.15.94:0/967645031 learned_addr learned my addr 172.21.15.94:0/967645031 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:27.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.952+0000 7f1673fff640 1 --2- 172.21.15.94:0/967645031 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 0x55f84cf238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.952+0000 7f167a59c640 1 --2- 172.21.15.94:0/967645031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 0x55f84cf1a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.953+0000 7f167ad9d640 1 -- 172.21.15.94:0/967645031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 msgr2=0x55f84cf1a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:27.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.953+0000 7f167ad9d640 1 --2- 172.21.15.94:0/967645031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 0x55f84cf1a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.953+0000 7f167ad9d640 1 -- 172.21.15.94:0/967645031 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 msgr2=0x55f84cf238e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:27.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.953+0000 7f167ad9d640 1 --2- 172.21.15.94:0/967645031 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 0x55f84cf238e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.953+0000 7f167ad9d640 1 -- 172.21.15.94:0/967645031 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f84cf131f0 con 0x55f84cf1c8b0 2024-02-17T12:09:27.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.959+0000 7f167ad9d640 1 --2- 172.21.15.94:0/967645031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 0x55f84cf23e20 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1674097120 tx=0x7f167409bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb10c1a145f0bc52 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:27.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.961+0000 7f1663fff640 1 -- 172.21.15.94:0/967645031 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16740ce040 con 0x55f84cf1c8b0 2024-02-17T12:09:27.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.961+0000 7f1663fff640 1 -- 172.21.15.94:0/967645031 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1674097330 con 0x55f84cf1c8b0 2024-02-17T12:09:27.963 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.961+0000 7f1663fff640 1 -- 172.21.15.94:0/967645031 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167409a370 con 0x55f84cf1c8b0 2024-02-17T12:09:27.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.962+0000 7f1683624740 1 -- 172.21.15.94:0/967645031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 msgr2=0x55f84cf23e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:27.964 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.962+0000 7f1683624740 1 --2- 172.21.15.94:0/967645031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 0x55f84cf23e20 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1674097120 tx=0x7f167409bf40 comp rx=0 tx=0).stop 2024-02-17T12:09:27.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.968+0000 7f1683624740 1 -- 172.21.15.94:0/967645031 shutdown_connections 2024-02-17T12:09:27.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.968+0000 7f1683624740 1 --2- 172.21.15.94:0/967645031 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf1c8b0 0x55f84cf23e20 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.968+0000 7f1683624740 1 --2- 172.21.15.94:0/967645031 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf13d30 0x55f84cf238e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.968+0000 7f1683624740 1 --2- 172.21.15.94:0/967645031 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 0x55f84cf1a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.968+0000 7f1683624740 1 -- 172.21.15.94:0/967645031 >> 172.21.15.94:0/967645031 conn(0x55f84cf0d840 msgr2=0x55f84cf0ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:27.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.973+0000 7f1683624740 1 -- 172.21.15.94:0/967645031 shutdown_connections 2024-02-17T12:09:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.976+0000 7f1683624740 1 -- 172.21.15.94:0/967645031 wait complete. 2024-02-17T12:09:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.977+0000 7f1683624740 1 -- start start 2024-02-17T12:09:27.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.977+0000 7f1683624740 1 Processor -- start 2024-02-17T12:09:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.977+0000 7f1683624740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 0x55f84cf22a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.977+0000 7f1683624740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf13d30 0x55f84cf22f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.977+0000 7f1683624740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf1c8b0 0x55f84cf23480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.977+0000 7f1683624740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84cf94b30 con 0x55f84cf13d30 2024-02-17T12:09:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.977+0000 7f1683624740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84cf94c70 con 0x55f84cf1c8b0 2024-02-17T12:09:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.977+0000 7f1683624740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f84cf94db0 con 0x55f84cf13380 2024-02-17T12:09:27.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.977+0000 7f1683624740 10 client.0 _ll_register_callbacks cb 0x7f167409aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.978+0000 7f167a59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 0x55f84cf22a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.978+0000 7f167a59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 0x55f84cf22a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:50962/0 (socket says 172.21.15.94:50962) 2024-02-17T12:09:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.978+0000 7f167a59c640 1 -- 172.21.15.94:0/4049179178 learned_addr learned my addr 172.21.15.94:0/4049179178 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:27.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.978+0000 7f167ad9d640 1 --2- 172.21.15.94:0/4049179178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf1c8b0 0x55f84cf23480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.980+0000 7f167ad9d640 1 -- 172.21.15.94:0/4049179178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 msgr2=0x55f84cf22a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.980+0000 7f167ad9d640 1 --2- 172.21.15.94:0/4049179178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 0x55f84cf22a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.980+0000 7f167ad9d640 1 -- 172.21.15.94:0/4049179178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf13d30 msgr2=0x55f84cf22f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:09:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.980+0000 7f167ad9d640 1 --2- 172.21.15.94:0/4049179178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf13d30 0x55f84cf22f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.980+0000 7f167ad9d640 1 -- 172.21.15.94:0/4049179178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f84ce3d0b0 con 0x55f84cf1c8b0 2024-02-17T12:09:27.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.982+0000 7f167ad9d640 1 --2- 172.21.15.94:0/4049179178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf1c8b0 0x55f84cf23480 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1674097690 tx=0x7f16740053c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8592fe50d4d6531 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:27.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.982+0000 7f1661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:09:27.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.982+0000 7f1661ffb640 1 -- 172.21.15.94:0/4049179178 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16740ce040 con 0x55f84cf1c8b0 2024-02-17T12:09:27.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.983+0000 7f1683624740 1 -- 172.21.15.94:0/4049179178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f84cea0900 con 0x55f84cf1c8b0 2024-02-17T12:09:27.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.983+0000 7f1661ffb640 1 -- 172.21.15.94:0/4049179178 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f167409a320 con 0x55f84cf1c8b0 2024-02-17T12:09:27.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.983+0000 7f1661ffb640 1 -- 172.21.15.94:0/4049179178 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1674079ae0 con 0x55f84cf1c8b0 2024-02-17T12:09:27.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.984+0000 7f1683624740 10 client.27941 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:27.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.984+0000 7f1683624740 1 -- 172.21.15.94:0/4049179178 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f84ce15c00 con 0x55f84cf1c8b0 2024-02-17T12:09:27.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.984+0000 7f1661ffb640 1 -- 172.21.15.94:0/4049179178 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f84cea0900 con 0x55f84cf1c8b0 2024-02-17T12:09:27.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.985+0000 7f1661ffb640 1 -- 172.21.15.94:0/4049179178 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1674018030 con 0x55f84cf1c8b0 2024-02-17T12:09:27.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.986+0000 7f1683624740 10 client.27941 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:27.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.986+0000 7f1683624740 10 client.27941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:27.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.986+0000 7f1683624740 20 client.27941 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:27.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.986+0000 7f1683624740 1 --2- 172.21.15.94:0/4049179178 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f84cf2cea0 0x55f84cf97ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:27.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.986+0000 7f1683624740 4 client.27941 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:09:27.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.986+0000 7f1683624740 1 -- 172.21.15.94:0/4049179178 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f84cb7a860 con 0x55f84cf2cea0 2024-02-17T12:09:27.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.986+0000 7f167a59c640 1 --2- 172.21.15.94:0/4049179178 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f84cf2cea0 0x55f84cf97ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:27.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.987+0000 7f167a59c640 1 --2- 172.21.15.94:0/4049179178 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f84cf2cea0 0x55f84cf97ff0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ad2878a9c38c35a server_cookie=f8046cd674fd0a0b in_seq=0 out_seq=0 2024-02-17T12:09:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.987+0000 7f1661ffb640 10 client.27941 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.989+0000 7f1661ffb640 1 -- 172.21.15.94:0/4049179178 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f84cb7a860 con 0x55f84cf2cea0 2024-02-17T12:09:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.989+0000 7f1661ffb640 10 client.27941 handle_command_reply: tid=0 2024-02-17T12:09:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.989+0000 7f1683624740 1 client.27941 shutdown 2024-02-17T12:09:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.992+0000 7f1683624740 1 -- 172.21.15.94:0/4049179178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf1c8b0 msgr2=0x55f84cf23480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.992+0000 7f1683624740 1 --2- 172.21.15.94:0/4049179178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf1c8b0 0x55f84cf23480 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1674097690 tx=0x7f16740053c0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.992+0000 7f1683624740 1 -- 172.21.15.94:0/4049179178 shutdown_connections 2024-02-17T12:09:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.992+0000 7f1683624740 1 --2- 172.21.15.94:0/4049179178 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f84cf2cea0 0x55f84cf97ff0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.992+0000 7f1683624740 1 --2- 172.21.15.94:0/4049179178 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f84cf1c8b0 0x55f84cf23480 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.992+0000 7f1683624740 1 --2- 172.21.15.94:0/4049179178 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f84cf13d30 0x55f84cf22f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.992+0000 7f1683624740 1 --2- 172.21.15.94:0/4049179178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f84cf13380 0x55f84cf22a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.992+0000 7f1683624740 1 -- 172.21.15.94:0/4049179178 >> 172.21.15.94:0/4049179178 conn(0x55f84cf0d840 msgr2=0x55f84cf0c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:27.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.993+0000 7f1661ffb640 0 client.27941 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:27.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.995+0000 7f1683624740 1 -- 172.21.15.94:0/4049179178 shutdown_connections 2024-02-17T12:09:27.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.995+0000 7f1683624740 1 -- 172.21.15.94:0/4049179178 wait complete. 2024-02-17T12:09:28.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:27.996+0000 7f1683624740 20 client.27941 trim_cache size 0 max 16384 2024-02-17T12:09:28.005 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:09:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:09:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "c18658ac-6cf9-47a8-b47a-2a7af136e959", 2024-02-17T12:09:28.005 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:09:28.005 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:09:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.004+0000 7f1680a4e640 1 -- 172.21.15.94:0/1768373675 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f16680a2230 msgr2=0x7f16680a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.004+0000 7f1680a4e640 1 --2- 172.21.15.94:0/1768373675 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f16680a2230 0x7f16680a46f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f166c009230 tx=0x7f166c005e80 comp rx=0 tx=0).stop 2024-02-17T12:09:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.004+0000 7f1680a4e640 1 -- 172.21.15.94:0/1768373675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c197d10 msgr2=0x7f167c198190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.004+0000 7f1680a4e640 1 --2- 172.21.15.94:0/1768373675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c197d10 0x7f167c198190 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f167400eaa0 tx=0x7f167400ef60 comp rx=0 tx=0).stop 2024-02-17T12:09:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.004+0000 7f1680a4e640 1 -- 172.21.15.94:0/1768373675 shutdown_connections 2024-02-17T12:09:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.004+0000 7f1680a4e640 1 --2- 172.21.15.94:0/1768373675 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f16680a2230 0x7f16680a46f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.004+0000 7f1680a4e640 1 --2- 172.21.15.94:0/1768373675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f167c197d10 0x7f167c198190 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.004+0000 7f1680a4e640 1 --2- 172.21.15.94:0/1768373675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f167c19dba0 0x7f167c1977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.004+0000 7f1680a4e640 1 --2- 172.21.15.94:0/1768373675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f167c106500 0x7f167c19d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.004+0000 7f1680a4e640 1 -- 172.21.15.94:0/1768373675 >> 172.21.15.94:0/1768373675 conn(0x7f167c0765a0 msgr2=0x7f167c0fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.008+0000 7f1680a4e640 1 -- 172.21.15.94:0/1768373675 shutdown_connections 2024-02-17T12:09:28.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.010+0000 7f1680a4e640 1 -- 172.21.15.94:0/1768373675 wait complete. 2024-02-17T12:09:28.036 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:09:28.036 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:09:28.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.197+0000 7f5681765640 1 -- 172.21.15.94:0/178660396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f567c10de10 msgr2=0x7f567c11c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.197+0000 7f5681765640 1 --2- 172.21.15.94:0/178660396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f567c10de10 0x7f567c11c970 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f566c009900 tx=0x7f566c02bc80 comp rx=0 tx=0).stop 2024-02-17T12:09:28.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.198+0000 7f5681765640 1 -- 172.21.15.94:0/178660396 shutdown_connections 2024-02-17T12:09:28.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.198+0000 7f5681765640 1 --2- 172.21.15.94:0/178660396 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f567c10de10 0x7f567c11c970 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.198+0000 7f5681765640 1 --2- 172.21.15.94:0/178660396 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f567c10d260 0x7f567c10d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.198+0000 7f5681765640 1 --2- 172.21.15.94:0/178660396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f567c10f790 0x7f567c10fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.199 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.198+0000 7f5681765640 1 -- 172.21.15.94:0/178660396 >> 172.21.15.94:0/178660396 conn(0x7f567c06c060 msgr2=0x7f567c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:28.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.200+0000 7f5681765640 1 -- 172.21.15.94:0/178660396 shutdown_connections 2024-02-17T12:09:28.202 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.201+0000 7f5681765640 1 -- 172.21.15.94:0/178660396 wait complete. 2024-02-17T12:09:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.201+0000 7f5681765640 1 Processor -- start 2024-02-17T12:09:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.201+0000 7f5681765640 1 -- start start 2024-02-17T12:09:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f5681765640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f567c10d260 0x7f567c11c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f5681765640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f567c10de10 0x7f567c11c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f5681765640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f567c10f790 0x7f567c115550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.203 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f5681765640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c115ce0 con 0x7f567c10de10 2024-02-17T12:09:28.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f5681765640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c115e50 con 0x7f567c10d260 2024-02-17T12:09:28.204 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f5681765640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c116010 con 0x7f567c10f790 2024-02-17T12:09:28.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f567affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f567c10d260 0x7f567c11c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f567affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f567c10d260 0x7f567c11c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:49102/0 (socket says 172.21.15.94:49102) 2024-02-17T12:09:28.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f567affd640 1 -- 172.21.15.94:0/1754449112 learned_addr learned my addr 172.21.15.94:0/1754449112 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:28.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f567b7fe640 1 --2- 172.21.15.94:0/1754449112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f567c10f790 0x7f567c115550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f567affd640 1 -- 172.21.15.94:0/1754449112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f567c10f790 msgr2=0x7f567c115550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f567affd640 1 --2- 172.21.15.94:0/1754449112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f567c10f790 0x7f567c115550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f567affd640 1 -- 172.21.15.94:0/1754449112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f567c10de10 msgr2=0x7f567c11c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f567affd640 1 --2- 172.21.15.94:0/1754449112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f567c10de10 0x7f567c11c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.202+0000 7f567affd640 1 -- 172.21.15.94:0/1754449112 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c009560 con 0x7f567c10d260 2024-02-17T12:09:28.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.203+0000 7f567affd640 1 --2- 172.21.15.94:0/1754449112 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f567c10d260 0x7f567c11c360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f567400ba50 tx=0x7f567400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:28.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.203+0000 7f5663fff640 1 -- 172.21.15.94:0/1754449112 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567400cc10 con 0x7f567c10d260 2024-02-17T12:09:28.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.203+0000 7f5663fff640 1 -- 172.21.15.94:0/1754449112 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f567400cd70 con 0x7f567c10d260 2024-02-17T12:09:28.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.203+0000 7f5663fff640 1 -- 172.21.15.94:0/1754449112 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56740077d0 con 0x7f567c10d260 2024-02-17T12:09:28.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.203+0000 7f5681765640 1 -- 172.21.15.94:0/1754449112 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567c119e80 con 0x7f567c10d260 2024-02-17T12:09:28.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.203+0000 7f5681765640 1 -- 172.21.15.94:0/1754449112 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567c11a340 con 0x7f567c10d260 2024-02-17T12:09:28.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.206+0000 7f5663fff640 1 -- 172.21.15.94:0/1754449112 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5674014440 con 0x7f567c10d260 2024-02-17T12:09:28.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f5684415740 1 Processor -- start 2024-02-17T12:09:28.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f5684415740 1 -- start start 2024-02-17T12:09:28.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f5684415740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 0x55c207b38870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f5684415740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38db0 0x55c207b41450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f5663fff640 1 --2- 172.21.15.94:0/1754449112 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f56640a22f0 0x7f56640a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f5684415740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207b43d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f5684415740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c207aa7280 con 0x55c207b41990 2024-02-17T12:09:28.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f5684415740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c207aa47b0 con 0x55c207b38490 2024-02-17T12:09:28.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f5684415740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c207a9ff90 con 0x55c207b38db0 2024-02-17T12:09:28.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f5663fff640 1 -- 172.21.15.94:0/1754449112 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f56740cd8e0 con 0x7f567c10d260 2024-02-17T12:09:28.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f567b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 0x55c207b38870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f567b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 0x55c207b38870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:49118/0 (socket says 172.21.15.94:49118) 2024-02-17T12:09:28.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f567b7fe640 1 -- 172.21.15.94:0/3441120920 learned_addr learned my addr 172.21.15.94:0/3441120920 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:28.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f567a7fc640 1 --2- 172.21.15.94:0/1754449112 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f56640a22f0 0x7f56640a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f567b7fe640 1 --2- 172.21.15.94:0/3441120920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207b43d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.207+0000 7f567affd640 1 --2- 172.21.15.94:0/3441120920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38db0 0x55c207b41450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.208+0000 7f567b7fe640 1 -- 172.21.15.94:0/3441120920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38db0 msgr2=0x55c207b41450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.208+0000 7f567b7fe640 1 --2- 172.21.15.94:0/3441120920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38db0 0x55c207b41450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.208+0000 7f567b7fe640 1 -- 172.21.15.94:0/3441120920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 msgr2=0x55c207b43d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.208+0000 7f567b7fe640 1 --2- 172.21.15.94:0/3441120920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207b43d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.208+0000 7f567b7fe640 1 -- 172.21.15.94:0/3441120920 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c207a5e570 con 0x55c207b38490 2024-02-17T12:09:28.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.208+0000 7f567b7fe640 1 --2- 172.21.15.94:0/3441120920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 0x55c207b38870 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f566c009900 tx=0x7f566c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1443b8a60310a86c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:28.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.208+0000 7f56617fa640 1 -- 172.21.15.94:0/3441120920 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c02c9b0 con 0x55c207b38490 2024-02-17T12:09:28.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.208+0000 7f56617fa640 1 -- 172.21.15.94:0/3441120920 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f566c004510 con 0x55c207b38490 2024-02-17T12:09:28.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.208+0000 7f56617fa640 1 -- 172.21.15.94:0/3441120920 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c004df0 con 0x55c207b38490 2024-02-17T12:09:28.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.209+0000 7f5684415740 1 -- 172.21.15.94:0/3441120920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 msgr2=0x55c207b38870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.209+0000 7f5684415740 1 --2- 172.21.15.94:0/3441120920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 0x55c207b38870 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f566c009900 tx=0x7f566c005fd0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.209+0000 7f5684415740 1 -- 172.21.15.94:0/3441120920 shutdown_connections 2024-02-17T12:09:28.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.209+0000 7f5684415740 1 --2- 172.21.15.94:0/3441120920 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207b43d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.209+0000 7f5684415740 1 --2- 172.21.15.94:0/3441120920 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38db0 0x55c207b41450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.209+0000 7f5684415740 1 --2- 172.21.15.94:0/3441120920 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 0x55c207b38870 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.209+0000 7f5684415740 1 -- 172.21.15.94:0/3441120920 >> 172.21.15.94:0/3441120920 conn(0x55c207b32bc0 msgr2=0x55c207b32fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:28.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.210+0000 7f5684415740 1 -- 172.21.15.94:0/3441120920 shutdown_connections 2024-02-17T12:09:28.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.210+0000 7f5684415740 1 -- 172.21.15.94:0/3441120920 wait complete. 2024-02-17T12:09:28.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.210+0000 7f5684415740 1 -- start start 2024-02-17T12:09:28.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.211+0000 7f5684415740 1 Processor -- start 2024-02-17T12:09:28.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.211+0000 7f567a7fc640 1 --2- 172.21.15.94:0/1754449112 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f56640a22f0 0x7f56640a47b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f567c116ba0 tx=0x7f5670006c40 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:28.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.211+0000 7f5684415740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b38490 0x55c207b41420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.211+0000 7f5684415740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38db0 0x55c207b3e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.211+0000 7f5684415740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b41990 0x55c207b3eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.211+0000 7f5684415740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c207773430 con 0x55c207b38490 2024-02-17T12:09:28.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.211+0000 7f5684415740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c20772fe00 con 0x55c207b38db0 2024-02-17T12:09:28.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.211+0000 7f5684415740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c207a24fd0 con 0x55c207b41990 2024-02-17T12:09:28.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.211+0000 7f5684415740 10 client.0 _ll_register_callbacks cb 0x55c207a7afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:28.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.211+0000 7f567affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b38490 0x55c207b41420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.211+0000 7f567a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38db0 0x55c207b3e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.211+0000 7f567affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b38490 0x55c207b41420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33130/0 (socket says 172.21.15.94:33130) 2024-02-17T12:09:28.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.211+0000 7f567affd640 1 -- 172.21.15.94:0/862011311 learned_addr learned my addr 172.21.15.94:0/862011311 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:28.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.212+0000 7f567b7fe640 1 --2- 172.21.15.94:0/862011311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b41990 0x55c207b3eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.212+0000 7f567affd640 1 -- 172.21.15.94:0/862011311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b41990 msgr2=0x55c207b3eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.212+0000 7f567affd640 1 --2- 172.21.15.94:0/862011311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b41990 0x55c207b3eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.212+0000 7f567affd640 1 -- 172.21.15.94:0/862011311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38db0 msgr2=0x55c207b3e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.212+0000 7f567affd640 1 --2- 172.21.15.94:0/862011311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38db0 0x55c207b3e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.212+0000 7f567affd640 1 -- 172.21.15.94:0/862011311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567c11d470 con 0x55c207b38490 2024-02-17T12:09:28.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.212+0000 7f567affd640 1 --2- 172.21.15.94:0/862011311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b38490 0x55c207b41420 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f567400ba20 tx=0x7f5674014ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6249676879240ac5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:28.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.212+0000 7f565eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:28.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.212+0000 7f565eff5640 1 -- 172.21.15.94:0/862011311 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56740a2b40 con 0x55c207b38490 2024-02-17T12:09:28.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.212+0000 7f565eff5640 1 -- 172.21.15.94:0/862011311 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f567409e150 con 0x55c207b38490 2024-02-17T12:09:28.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.212+0000 7f5684415740 1 -- 172.21.15.94:0/862011311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c20779f7e0 con 0x55c207b38490 2024-02-17T12:09:28.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.214+0000 7f565eff5640 1 -- 172.21.15.94:0/862011311 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f567409e310 con 0x55c207b38490 2024-02-17T12:09:28.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.214+0000 7f5684415740 10 client.18489 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:28.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.214+0000 7f5684415740 1 -- 172.21.15.94:0/862011311 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c207a9e610 con 0x55c207b38490 2024-02-17T12:09:28.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.214+0000 7f565eff5640 1 -- 172.21.15.94:0/862011311 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56740b2030 con 0x55c207b38490 2024-02-17T12:09:28.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.214+0000 7f565eff5640 1 -- 172.21.15.94:0/862011311 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f56740998a0 con 0x55c207b38490 2024-02-17T12:09:28.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.214+0000 7f5684415740 10 client.18489 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:28.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.214+0000 7f5684415740 10 client.18489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:28.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.214+0000 7f5684415740 20 client.18489 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:28.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.214+0000 7f5684415740 1 --2- 172.21.15.94:0/862011311 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c207b51b30 0x55c207b53f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.214+0000 7f5684415740 4 client.18489 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:28.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.214+0000 7f5684415740 1 -- 172.21.15.94:0/862011311 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c2077dc720 con 0x55c207b51b30 2024-02-17T12:09:28.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.214+0000 7f567a7fc640 1 --2- 172.21.15.94:0/862011311 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c207b51b30 0x55c207b53f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.215+0000 7f565eff5640 10 client.18489 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:28.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.215+0000 7f567a7fc640 1 --2- 172.21.15.94:0/862011311 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c207b51b30 0x55c207b53f40 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6c43ce1bfe50e96 server_cookie=818f4d5c4804c108 in_seq=0 out_seq=0 2024-02-17T12:09:28.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.215+0000 7f565eff5640 1 -- 172.21.15.94:0/862011311 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c2077dc720 con 0x55c207b51b30 2024-02-17T12:09:28.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.215+0000 7f565eff5640 10 client.18489 handle_command_reply: tid=0 2024-02-17T12:09:28.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.215+0000 7f5684415740 1 client.18489 shutdown 2024-02-17T12:09:28.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.216+0000 7f5684415740 1 -- 172.21.15.94:0/862011311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b38490 msgr2=0x55c207b41420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.216+0000 7f5684415740 1 --2- 172.21.15.94:0/862011311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b38490 0x55c207b41420 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f567400ba20 tx=0x7f5674014ab0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.216+0000 7f5684415740 1 -- 172.21.15.94:0/862011311 shutdown_connections 2024-02-17T12:09:28.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.216+0000 7f5684415740 1 --2- 172.21.15.94:0/862011311 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c207b51b30 0x55c207b53f40 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.216+0000 7f5684415740 1 --2- 172.21.15.94:0/862011311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b41990 0x55c207b3eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.216+0000 7f5684415740 1 --2- 172.21.15.94:0/862011311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38db0 0x55c207b3e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.216+0000 7f565eff5640 0 client.18489 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:28.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.216+0000 7f5684415740 1 --2- 172.21.15.94:0/862011311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b38490 0x55c207b41420 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.216+0000 7f5684415740 1 -- 172.21.15.94:0/862011311 >> 172.21.15.94:0/862011311 conn(0x55c207b32bc0 msgr2=0x55c207b34910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:28.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.216+0000 7f5684415740 1 -- 172.21.15.94:0/862011311 shutdown_connections 2024-02-17T12:09:28.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.216+0000 7f5684415740 1 -- 172.21.15.94:0/862011311 wait complete. 2024-02-17T12:09:28.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.216+0000 7f5684415740 20 client.18489 trim_cache size 0 max 16384 2024-02-17T12:09:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f5684415740 1 Processor -- start 2024-02-17T12:09:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f5684415740 1 -- start start 2024-02-17T12:09:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f5684415740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 0x55c207b588e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f5684415740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38db0 0x55c207b40ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f5684415740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207b41420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f5684415740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c207b3a440 con 0x55c207b41990 2024-02-17T12:09:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f5684415740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c207b2e570 con 0x55c207b38490 2024-02-17T12:09:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f5684415740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c207b37c30 con 0x55c207b38db0 2024-02-17T12:09:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f567b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207b41420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f567affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 0x55c207b588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f567b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207b41420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33138/0 (socket says 172.21.15.94:33138) 2024-02-17T12:09:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f567affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 0x55c207b588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:49142/0 (socket says 172.21.15.94:49142) 2024-02-17T12:09:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f567b7fe640 1 -- 172.21.15.94:0/155175991 learned_addr learned my addr 172.21.15.94:0/155175991 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.236+0000 7f567a7fc640 1 --2- 172.21.15.94:0/155175991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38db0 0x55c207b40ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.237+0000 7f567b7fe640 1 -- 172.21.15.94:0/155175991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38db0 msgr2=0x55c207b40ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.237+0000 7f567b7fe640 1 --2- 172.21.15.94:0/155175991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38db0 0x55c207b40ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.237+0000 7f567b7fe640 1 -- 172.21.15.94:0/155175991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 msgr2=0x55c207b588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.237+0000 7f567b7fe640 1 --2- 172.21.15.94:0/155175991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 0x55c207b588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.237+0000 7f567b7fe640 1 -- 172.21.15.94:0/155175991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c207a3aac0 con 0x55c207b41990 2024-02-17T12:09:28.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.237+0000 7f567affd640 1 --2- 172.21.15.94:0/155175991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 0x55c207b588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:28.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.237+0000 7f567b7fe640 1 --2- 172.21.15.94:0/155175991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207b41420 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f566c02c880 tx=0x7f566c02bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7cf1459f95a5b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:28.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.237+0000 7f5660ff9640 1 -- 172.21.15.94:0/155175991 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c02cbb0 con 0x55c207b41990 2024-02-17T12:09:28.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.237+0000 7f5660ff9640 1 -- 172.21.15.94:0/155175991 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f566c043960 con 0x55c207b41990 2024-02-17T12:09:28.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.237+0000 7f5660ff9640 1 -- 172.21.15.94:0/155175991 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c0052b0 con 0x55c207b41990 2024-02-17T12:09:28.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.239+0000 7f5684415740 1 -- 172.21.15.94:0/155175991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 msgr2=0x55c207b41420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.239+0000 7f5684415740 1 --2- 172.21.15.94:0/155175991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207b41420 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f566c02c880 tx=0x7f566c02bc80 comp rx=0 tx=0).stop 2024-02-17T12:09:28.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.239+0000 7f5684415740 1 -- 172.21.15.94:0/155175991 shutdown_connections 2024-02-17T12:09:28.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.239+0000 7f5684415740 1 --2- 172.21.15.94:0/155175991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207b41420 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.239+0000 7f5684415740 1 --2- 172.21.15.94:0/155175991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38db0 0x55c207b40ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.239+0000 7f5684415740 1 --2- 172.21.15.94:0/155175991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38490 0x55c207b588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.239+0000 7f5684415740 1 -- 172.21.15.94:0/155175991 >> 172.21.15.94:0/155175991 conn(0x55c207b32bc0 msgr2=0x55c207b32fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:28.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.239+0000 7f5684415740 1 -- 172.21.15.94:0/155175991 shutdown_connections 2024-02-17T12:09:28.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.239+0000 7f5684415740 1 -- 172.21.15.94:0/155175991 wait complete. 2024-02-17T12:09:28.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.239+0000 7f5684415740 1 -- start start 2024-02-17T12:09:28.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.239+0000 7f5684415740 1 Processor -- start 2024-02-17T12:09:28.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.240+0000 7f5684415740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38490 0x55c207b474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.240+0000 7f5684415740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38db0 0x55c207b47a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.240+0000 7f5684415740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207ba8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.240+0000 7f5684415740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c207b481c0 con 0x55c207b41990 2024-02-17T12:09:28.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.240+0000 7f5684415740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c207b48330 con 0x55c207b38db0 2024-02-17T12:09:28.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.240+0000 7f5684415740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c207ba85b0 con 0x55c207b38490 2024-02-17T12:09:28.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.240+0000 7f5684415740 10 client.0 _ll_register_callbacks cb 0x7f566c004800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:28.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.240+0000 7f567a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38db0 0x55c207b47a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.240+0000 7f567affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38490 0x55c207b474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.240+0000 7f567affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38490 0x55c207b474e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:51030/0 (socket says 172.21.15.94:51030) 2024-02-17T12:09:28.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.240+0000 7f567affd640 1 -- 172.21.15.94:0/3980617326 learned_addr learned my addr 172.21.15.94:0/3980617326 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:28.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.240+0000 7f567a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38db0 0x55c207b47a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:49152/0 (socket says 172.21.15.94:49152) 2024-02-17T12:09:28.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.240+0000 7f567b7fe640 1 --2- 172.21.15.94:0/3980617326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207ba8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f567a7fc640 1 -- 172.21.15.94:0/3980617326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38490 msgr2=0x55c207b474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f567a7fc640 1 --2- 172.21.15.94:0/3980617326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38490 0x55c207b474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f567a7fc640 1 -- 172.21.15.94:0/3980617326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 msgr2=0x55c207ba8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f567a7fc640 1 --2- 172.21.15.94:0/3980617326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207ba8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f567a7fc640 1 -- 172.21.15.94:0/3980617326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c207a9e610 con 0x55c207b38db0 2024-02-17T12:09:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f567affd640 1 --2- 172.21.15.94:0/3980617326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38490 0x55c207b474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f567a7fc640 1 --2- 172.21.15.94:0/3980617326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38db0 0x55c207b47a20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55c207b41960 tx=0x7f56700047e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98e99da73f36b150 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f565eff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:09:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f565eff5640 1 -- 172.21.15.94:0/3980617326 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567001a040 con 0x55c207b38db0 2024-02-17T12:09:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f565eff5640 1 -- 172.21.15.94:0/3980617326 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5670005a00 con 0x55c207b38db0 2024-02-17T12:09:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f5684415740 1 -- 172.21.15.94:0/3980617326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f566c004510 con 0x55c207b38db0 2024-02-17T12:09:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f565eff5640 1 -- 172.21.15.94:0/3980617326 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5670005bc0 con 0x55c207b38db0 2024-02-17T12:09:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f5684415740 10 client.27959 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f5684415740 1 -- 172.21.15.94:0/3980617326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c207a5e570 con 0x55c207b38db0 2024-02-17T12:09:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f565eff5640 1 -- 172.21.15.94:0/3980617326 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f566c004510 con 0x55c207b38db0 2024-02-17T12:09:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.241+0000 7f565eff5640 1 -- 172.21.15.94:0/3980617326 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5670033070 con 0x55c207b38db0 2024-02-17T12:09:28.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.242+0000 7f5684415740 10 client.27959 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:28.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.242+0000 7f5684415740 10 client.27959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:28.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.242+0000 7f5684415740 20 client.27959 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:28.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.242+0000 7f5684415740 1 --2- 172.21.15.94:0/3980617326 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c207ba9e70 0x55c207bac290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.242+0000 7f5684415740 4 client.27959 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:09:28.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.242+0000 7f5684415740 1 -- 172.21.15.94:0/3980617326 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c20782b890 con 0x55c207ba9e70 2024-02-17T12:09:28.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.242+0000 7f567affd640 1 --2- 172.21.15.94:0/3980617326 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c207ba9e70 0x55c207bac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.242+0000 7f565eff5640 10 client.27959 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:28.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.242+0000 7f567affd640 1 --2- 172.21.15.94:0/3980617326 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c207ba9e70 0x55c207bac290 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5287c31f5418e0dc server_cookie=9b9ea3cba9af2824 in_seq=0 out_seq=0 2024-02-17T12:09:28.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f565eff5640 1 -- 172.21.15.94:0/3980617326 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c20782b890 con 0x55c207ba9e70 2024-02-17T12:09:28.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f565eff5640 10 client.27959 handle_command_reply: tid=0 2024-02-17T12:09:28.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f5684415740 1 client.27959 shutdown 2024-02-17T12:09:28.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f5684415740 1 -- 172.21.15.94:0/3980617326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38db0 msgr2=0x55c207b47a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f5684415740 1 --2- 172.21.15.94:0/3980617326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38db0 0x55c207b47a20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55c207b41960 tx=0x7f56700047e0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f5684415740 1 -- 172.21.15.94:0/3980617326 shutdown_connections 2024-02-17T12:09:28.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f5684415740 1 --2- 172.21.15.94:0/3980617326 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c207ba9e70 0x55c207bac290 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f5684415740 1 --2- 172.21.15.94:0/3980617326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c207b41990 0x55c207ba8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f5684415740 1 --2- 172.21.15.94:0/3980617326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c207b38db0 0x55c207b47a20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f5684415740 1 --2- 172.21.15.94:0/3980617326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c207b38490 0x55c207b474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f5684415740 1 -- 172.21.15.94:0/3980617326 >> 172.21.15.94:0/3980617326 conn(0x55c207b32bc0 msgr2=0x55c207b588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:28.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f565eff5640 0 client.27959 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:28.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f5684415740 1 -- 172.21.15.94:0/3980617326 shutdown_connections 2024-02-17T12:09:28.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f5684415740 1 -- 172.21.15.94:0/3980617326 wait complete. 2024-02-17T12:09:28.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.243+0000 7f5684415740 20 client.27959 trim_cache size 0 max 16384 2024-02-17T12:09:28.252 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:09:28.252 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:09:28.252 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:09:28.252 INFO:teuthology.orchestra.run.smithi094.stdout: "c18658ac-6cf9-47a8-b47a-2a7af136e959": { 2024-02-17T12:09:28.252 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:09:28.252 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "c18658ac-6cf9-47a8-b47a-2a7af136e959", 2024-02-17T12:09:28.252 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:09:28.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:09:28.252 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:09:28.253 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:09:28.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.244+0000 7f5681765640 1 -- 172.21.15.94:0/1754449112 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f56640a22f0 msgr2=0x7f56640a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.244+0000 7f5681765640 1 --2- 172.21.15.94:0/1754449112 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f56640a22f0 0x7f56640a47b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f567c116ba0 tx=0x7f5670006c40 comp rx=0 tx=0).stop 2024-02-17T12:09:28.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.244+0000 7f5681765640 1 -- 172.21.15.94:0/1754449112 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f567c10d260 msgr2=0x7f567c11c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.244+0000 7f5681765640 1 --2- 172.21.15.94:0/1754449112 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f567c10d260 0x7f567c11c360 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f567400ba50 tx=0x7f567400bf10 comp rx=0 tx=0).stop 2024-02-17T12:09:28.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.244+0000 7f5681765640 1 -- 172.21.15.94:0/1754449112 shutdown_connections 2024-02-17T12:09:28.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.244+0000 7f5681765640 1 --2- 172.21.15.94:0/1754449112 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f56640a22f0 0x7f56640a47b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.244+0000 7f5681765640 1 --2- 172.21.15.94:0/1754449112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f567c10f790 0x7f567c115550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.244+0000 7f5681765640 1 --2- 172.21.15.94:0/1754449112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f567c10de10 0x7f567c11c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.244+0000 7f5681765640 1 --2- 172.21.15.94:0/1754449112 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f567c10d260 0x7f567c11c360 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.244+0000 7f5681765640 1 -- 172.21.15.94:0/1754449112 >> 172.21.15.94:0/1754449112 conn(0x7f567c06c060 msgr2=0x7f567c070780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:28.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.244+0000 7f5681765640 1 -- 172.21.15.94:0/1754449112 shutdown_connections 2024-02-17T12:09:28.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:28.244+0000 7f5681765640 1 -- 172.21.15.94:0/1754449112 wait complete. 2024-02-17T12:09:28.258 INFO:tasks.cephfs.filesystem:scrub status for tag:c18658ac-6cf9-47a8-b47a-2a7af136e959 - {'path': '/', 'tag': 'c18658ac-6cf9-47a8-b47a-2a7af136e959', 'options': 'recursive,force'} 2024-02-17T12:09:28.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:28 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:09:28.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:28 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:09:28.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:28 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:09:28.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:28 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:09:28.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:28 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:09:28.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:28 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:09:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.472+0000 7f0106cfb640 1 -- 172.21.15.94:0/2746732561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01001083d0 msgr2=0x7f01001087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.472+0000 7f0106cfb640 1 --2- 172.21.15.94:0/2746732561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01001083d0 0x7f01001087b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009900 tx=0x7f00f002bc50 comp rx=0 tx=0).stop 2024-02-17T12:09:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.473+0000 7f0106cfb640 1 -- 172.21.15.94:0/2746732561 shutdown_connections 2024-02-17T12:09:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.473+0000 7f0106cfb640 1 --2- 172.21.15.94:0/2746732561 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01001142d0 0x7f01001166c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.473+0000 7f0106cfb640 1 --2- 172.21.15.94:0/2746732561 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0100108cf0 0x7f0100113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.473+0000 7f0106cfb640 1 --2- 172.21.15.94:0/2746732561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f01001083d0 0x7f01001087b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.473+0000 7f0106cfb640 1 -- 172.21.15.94:0/2746732561 >> 172.21.15.94:0/2746732561 conn(0x7f010006c220 msgr2=0x7f010006c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.473+0000 7f0106cfb640 1 -- 172.21.15.94:0/2746732561 shutdown_connections 2024-02-17T12:09:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.473+0000 7f0106cfb640 1 -- 172.21.15.94:0/2746732561 wait complete. 2024-02-17T12:09:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.473+0000 7f0106cfb640 1 Processor -- start 2024-02-17T12:09:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.473+0000 7f0106cfb640 1 -- start start 2024-02-17T12:09:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.473+0000 7f0106cfb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01001083d0 0x7f01001a0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.473+0000 7f0106cfb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0100108cf0 0x7f01001a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.473+0000 7f0106cfb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01001142d0 0x7f01001a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.473+0000 7f0106cfb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001a4ec0 con 0x7f01001083d0 2024-02-17T12:09:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.474+0000 7f0106cfb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001a5030 con 0x7f01001142d0 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.474+0000 7f0106cfb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01001a51f0 con 0x7f0100108cf0 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.474+0000 7f0104a97640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01001083d0 0x7f01001a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.474+0000 7f0104a97640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01001083d0 0x7f01001a0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33156/0 (socket says 172.21.15.94:33156) 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.474+0000 7f0104a97640 1 -- 172.21.15.94:0/3159330251 learned_addr learned my addr 172.21.15.94:0/3159330251 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.474+0000 7f00fffff640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0100108cf0 0x7f01001a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.474+0000 7f0105298640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01001142d0 0x7f01001a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.474+0000 7f00fffff640 1 -- 172.21.15.94:0/3159330251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01001142d0 msgr2=0x7f01001a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.474+0000 7f00fffff640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01001142d0 0x7f01001a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.474+0000 7f00fffff640 1 -- 172.21.15.94:0/3159330251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01001083d0 msgr2=0x7f01001a0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.474+0000 7f00fffff640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01001083d0 0x7f01001a0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.474+0000 7f00fffff640 1 -- 172.21.15.94:0/3159330251 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f0009560 con 0x7f0100108cf0 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.474+0000 7f00fffff640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0100108cf0 0x7f01001a0540 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009fd0 tx=0x7f00e800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.475+0000 7f00fdffb640 1 -- 172.21.15.94:0/3159330251 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e8011830 con 0x7f0100108cf0 2024-02-17T12:09:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.475+0000 7f00fdffb640 1 -- 172.21.15.94:0/3159330251 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00e8011e70 con 0x7f0100108cf0 2024-02-17T12:09:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.475+0000 7f00fdffb640 1 -- 172.21.15.94:0/3159330251 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e800f2e0 con 0x7f0100108cf0 2024-02-17T12:09:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.475+0000 7f0106cfb640 1 -- 172.21.15.94:0/3159330251 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01001a54d0 con 0x7f0100108cf0 2024-02-17T12:09:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.475+0000 7f0106cfb640 1 -- 172.21.15.94:0/3159330251 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01001acd50 con 0x7f0100108cf0 2024-02-17T12:09:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.478+0000 7f00fdffb640 1 -- 172.21.15.94:0/3159330251 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f00e8010680 con 0x7f0100108cf0 2024-02-17T12:09:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.478+0000 7f01099ad740 1 Processor -- start 2024-02-17T12:09:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f01099ad740 1 -- start start 2024-02-17T12:09:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f01099ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55930721dad0 0x55930721c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f01099ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x55930721d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f01099ad740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593072262e0 0x5593072286d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f01099ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930718b9f0 con 0x55930721dad0 2024-02-17T12:09:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f01099ad740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559307188f20 con 0x5593072262e0 2024-02-17T12:09:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f01099ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559307184700 con 0x55930721cd60 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f00fdffb640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f00ec0a2430 0x7f00ec0a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f0105298640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55930721dad0 0x55930721c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f0104a97640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x55930721d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f0104a97640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x55930721d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:51052/0 (socket says 172.21.15.94:51052) 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f0104a97640 1 -- 172.21.15.94:0/2682630566 learned_addr learned my addr 172.21.15.94:0/2682630566 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f00fffff640 1 --2- 172.21.15.94:0/2682630566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593072262e0 0x5593072286d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f00fdffb640 1 -- 172.21.15.94:0/3159330251 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f00e80cd980 con 0x7f0100108cf0 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f0104a97640 1 -- 172.21.15.94:0/2682630566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593072262e0 msgr2=0x5593072286d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f0104a97640 1 --2- 172.21.15.94:0/2682630566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593072262e0 0x5593072286d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f0104a97640 1 -- 172.21.15.94:0/2682630566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55930721dad0 msgr2=0x55930721c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f0104a97640 1 --2- 172.21.15.94:0/2682630566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55930721dad0 0x55930721c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f0105298640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f00ec0a2430 0x7f00ec0a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.479+0000 7f0104a97640 1 -- 172.21.15.94:0/2682630566 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55930711a7b0 con 0x55930721cd60 2024-02-17T12:09:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.480+0000 7f0104a97640 1 --2- 172.21.15.94:0/2682630566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x55930721d1e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f00f0004930 tx=0x7f00f0004960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b89b71f568425c04 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.480+0000 7f00f6ffd640 1 -- 172.21.15.94:0/2682630566 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f002c830 con 0x55930721cd60 2024-02-17T12:09:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.480+0000 7f00f6ffd640 1 -- 172.21.15.94:0/2682630566 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f002c990 con 0x55930721cd60 2024-02-17T12:09:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.480+0000 7f00f6ffd640 1 -- 172.21.15.94:0/2682630566 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0005560 con 0x55930721cd60 2024-02-17T12:09:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.481+0000 7f01099ad740 1 -- 172.21.15.94:0/2682630566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 msgr2=0x55930721d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.481+0000 7f01099ad740 1 --2- 172.21.15.94:0/2682630566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x55930721d1e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f00f0004930 tx=0x7f00f0004960 comp rx=0 tx=0).stop 2024-02-17T12:09:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.481+0000 7f0105298640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f00ec0a2430 0x7f00ec0a48f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009900 tx=0x7f00f8009650 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.481+0000 7f01099ad740 1 -- 172.21.15.94:0/2682630566 shutdown_connections 2024-02-17T12:09:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.481+0000 7f01099ad740 1 --2- 172.21.15.94:0/2682630566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5593072262e0 0x5593072286d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.481+0000 7f01099ad740 1 --2- 172.21.15.94:0/2682630566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x55930721d1e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.481+0000 7f01099ad740 1 --2- 172.21.15.94:0/2682630566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55930721dad0 0x55930721c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.481+0000 7f01099ad740 1 -- 172.21.15.94:0/2682630566 >> 172.21.15.94:0/2682630566 conn(0x5593071e4550 msgr2=0x559307217a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.481+0000 7f01099ad740 1 -- 172.21.15.94:0/2682630566 shutdown_connections 2024-02-17T12:09:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.481+0000 7f01099ad740 1 -- 172.21.15.94:0/2682630566 wait complete. 2024-02-17T12:09:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.481+0000 7f01099ad740 1 -- start start 2024-02-17T12:09:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.482+0000 7f01099ad740 1 Processor -- start 2024-02-17T12:09:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.483+0000 7f01099ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x559307225d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.483+0000 7f01099ad740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721dad0 0x55930721f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.483+0000 7f01099ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x55930721f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.483+0000 7f01099ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559307129bf0 con 0x5593072262e0 2024-02-17T12:09:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.483+0000 7f01099ad740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559306e58250 con 0x55930721dad0 2024-02-17T12:09:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.483+0000 7f01099ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930710a200 con 0x55930721cd60 2024-02-17T12:09:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.483+0000 7f01099ad740 10 client.0 _ll_register_callbacks cb 0x559307178d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.483+0000 7f0104a97640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x559307225d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.483+0000 7f0104a97640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x559307225d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:51054/0 (socket says 172.21.15.94:51054) 2024-02-17T12:09:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.483+0000 7f0104a97640 1 -- 172.21.15.94:0/963491025 learned_addr learned my addr 172.21.15.94:0/963491025 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.483+0000 7f00fffff640 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721dad0 0x55930721f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.483+0000 7f0105298640 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x55930721f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f0104a97640 1 -- 172.21.15.94:0/963491025 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721dad0 msgr2=0x55930721f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f0104a97640 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721dad0 0x55930721f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f0104a97640 1 -- 172.21.15.94:0/963491025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 msgr2=0x55930721f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f0104a97640 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x55930721f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f0104a97640 1 -- 172.21.15.94:0/963491025 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0100117050 con 0x55930721cd60 2024-02-17T12:09:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f00fffff640 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721dad0 0x55930721f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f0104a97640 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x559307225d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f00f002bc90 tx=0x7f00f0005510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63bf2021a2b21cbe server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f00dffff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f00dffff640 1 -- 172.21.15.94:0/963491025 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f002c830 con 0x55930721cd60 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f01099ad740 1 -- 172.21.15.94:0/963491025 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559306e84600 con 0x55930721cd60 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f00dffff640 1 -- 172.21.15.94:0/963491025 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f00f0050070 con 0x55930721cd60 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f00dffff640 1 -- 172.21.15.94:0/963491025 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f0004f00 con 0x55930721cd60 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f01099ad740 10 client.27964 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f01099ad740 1 -- 172.21.15.94:0/963491025 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559307182d80 con 0x55930721cd60 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.484+0000 7f00dffff640 1 -- 172.21.15.94:0/963491025 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559306e84600 con 0x55930721cd60 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.485+0000 7f00dffff640 1 -- 172.21.15.94:0/963491025 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f00f003c050 con 0x55930721cd60 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.485+0000 7f01099ad740 10 client.27964 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.485+0000 7f01099ad740 10 client.27964 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.485+0000 7f01099ad740 20 client.27964 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.485+0000 7f01099ad740 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559307236110 0x559307238500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.485+0000 7f01099ad740 4 client.27964 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.485+0000 7f01099ad740 1 -- 172.21.15.94:0/963491025 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559306ed51b0 con 0x559307236110 2024-02-17T12:09:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.486+0000 7f00fffff640 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559307236110 0x559307238500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.486+0000 7f00fffff640 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559307236110 0x559307238500 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2b550a050ade256 server_cookie=de730eac2dc7f808 in_seq=0 out_seq=0 2024-02-17T12:09:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.486+0000 7f00dffff640 10 client.27964 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.486+0000 7f00dffff640 1 -- 172.21.15.94:0/963491025 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559306ed51b0 con 0x559307236110 2024-02-17T12:09:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.486+0000 7f00dffff640 10 client.27964 handle_command_reply: tid=0 2024-02-17T12:09:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.486+0000 7f01099ad740 1 client.27964 shutdown 2024-02-17T12:09:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.487+0000 7f01099ad740 1 -- 172.21.15.94:0/963491025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 msgr2=0x559307225d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.487+0000 7f01099ad740 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x559307225d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f00f002bc90 tx=0x7f00f0005510 comp rx=0 tx=0).stop 2024-02-17T12:09:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.487+0000 7f01099ad740 1 -- 172.21.15.94:0/963491025 shutdown_connections 2024-02-17T12:09:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.487+0000 7f01099ad740 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559307236110 0x559307238500 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.487+0000 7f01099ad740 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x55930721f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.487+0000 7f01099ad740 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721dad0 0x55930721f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.487+0000 7f01099ad740 1 --2- 172.21.15.94:0/963491025 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x559307225d90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.487+0000 7f00dffff640 0 client.27964 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.487+0000 7f01099ad740 1 -- 172.21.15.94:0/963491025 >> 172.21.15.94:0/963491025 conn(0x5593071e4550 msgr2=0x5593072197a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.487+0000 7f01099ad740 1 -- 172.21.15.94:0/963491025 shutdown_connections 2024-02-17T12:09:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.487+0000 7f01099ad740 1 -- 172.21.15.94:0/963491025 wait complete. 2024-02-17T12:09:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.487+0000 7f01099ad740 20 client.27964 trim_cache size 0 max 16384 2024-02-17T12:09:28.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.503+0000 7f01099ad740 1 Processor -- start 2024-02-17T12:09:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.503+0000 7f01099ad740 1 -- start start 2024-02-17T12:09:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.503+0000 7f01099ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x55930721c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.503+0000 7f01099ad740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721dad0 0x55930721c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.503+0000 7f01099ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x5593072256e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f01099ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559306ed51b0 con 0x5593072262e0 2024-02-17T12:09:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f01099ad740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559306e84600 con 0x55930721dad0 2024-02-17T12:09:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f01099ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930723d6a0 con 0x55930721cd60 2024-02-17T12:09:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f0105298640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x5593072256e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f0105298640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x5593072256e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33188/0 (socket says 172.21.15.94:33188) 2024-02-17T12:09:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f0105298640 1 -- 172.21.15.94:0/3385937285 learned_addr learned my addr 172.21.15.94:0/3385937285 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f00fffff640 1 --2- 172.21.15.94:0/3385937285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721dad0 0x55930721c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f0104a97640 1 --2- 172.21.15.94:0/3385937285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x55930721c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f00fffff640 1 -- 172.21.15.94:0/3385937285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 msgr2=0x55930721c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f00fffff640 1 --2- 172.21.15.94:0/3385937285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x55930721c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f00fffff640 1 -- 172.21.15.94:0/3385937285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 msgr2=0x5593072256e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f00fffff640 1 --2- 172.21.15.94:0/3385937285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x5593072256e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f00fffff640 1 -- 172.21.15.94:0/3385937285 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55930711f7b0 con 0x55930721dad0 2024-02-17T12:09:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f0105298640 1 --2- 172.21.15.94:0/3385937285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x5593072256e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.504+0000 7f00fffff640 1 --2- 172.21.15.94:0/3385937285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721dad0 0x55930721c7c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f00e80cb1a0 tx=0x7f00e80146e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78e4bdf97f575adf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.505+0000 7f00f67fc640 1 -- 172.21.15.94:0/3385937285 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e809e590 con 0x55930721dad0 2024-02-17T12:09:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.505+0000 7f00f67fc640 1 -- 172.21.15.94:0/3385937285 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00e80a2be0 con 0x55930721dad0 2024-02-17T12:09:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.505+0000 7f00f67fc640 1 -- 172.21.15.94:0/3385937285 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e800dd40 con 0x55930721dad0 2024-02-17T12:09:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.505+0000 7f01099ad740 1 -- 172.21.15.94:0/3385937285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721dad0 msgr2=0x55930721c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.505+0000 7f01099ad740 1 --2- 172.21.15.94:0/3385937285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721dad0 0x55930721c7c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f00e80cb1a0 tx=0x7f00e80146e0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.505+0000 7f01099ad740 1 -- 172.21.15.94:0/3385937285 shutdown_connections 2024-02-17T12:09:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.505+0000 7f01099ad740 1 --2- 172.21.15.94:0/3385937285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x5593072256e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.505+0000 7f01099ad740 1 --2- 172.21.15.94:0/3385937285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721dad0 0x55930721c7c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.505+0000 7f01099ad740 1 --2- 172.21.15.94:0/3385937285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721cd60 0x55930721c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.505+0000 7f01099ad740 1 -- 172.21.15.94:0/3385937285 >> 172.21.15.94:0/3385937285 conn(0x5593071e4550 msgr2=0x5593072291a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.505+0000 7f01099ad740 1 -- 172.21.15.94:0/3385937285 shutdown_connections 2024-02-17T12:09:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.505+0000 7f01099ad740 1 -- 172.21.15.94:0/3385937285 wait complete. 2024-02-17T12:09:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f01099ad740 1 -- start start 2024-02-17T12:09:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f01099ad740 1 Processor -- start 2024-02-17T12:09:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f01099ad740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721cd60 0x55930722bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f01099ad740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721dad0 0x5593072321c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f01099ad740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x559307232900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f01099ad740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559307233000 con 0x5593072262e0 2024-02-17T12:09:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f01099ad740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930722a0e0 con 0x55930721cd60 2024-02-17T12:09:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f01099ad740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55930722a2a0 con 0x55930721dad0 2024-02-17T12:09:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f01099ad740 10 client.0 _ll_register_callbacks cb 0x7f00f00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f0104a97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721cd60 0x55930722bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f0105298640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x559307232900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f0104a97640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721cd60 0x55930722bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:49192/0 (socket says 172.21.15.94:49192) 2024-02-17T12:09:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f0104a97640 1 -- 172.21.15.94:0/1927978787 learned_addr learned my addr 172.21.15.94:0/1927978787 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.506+0000 7f00fffff640 1 --2- 172.21.15.94:0/1927978787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721dad0 0x5593072321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.507+0000 7f0104a97640 1 -- 172.21.15.94:0/1927978787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721dad0 msgr2=0x5593072321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.507+0000 7f0104a97640 1 --2- 172.21.15.94:0/1927978787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721dad0 0x5593072321c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.507+0000 7f0104a97640 1 -- 172.21.15.94:0/1927978787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 msgr2=0x559307232900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.507+0000 7f0104a97640 1 --2- 172.21.15.94:0/1927978787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x559307232900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.507+0000 7f0104a97640 1 -- 172.21.15.94:0/1927978787 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f8013050 con 0x55930721cd60 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.508+0000 7f0104a97640 1 --2- 172.21.15.94:0/1927978787 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721cd60 0x55930722bba0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f00f0006010 tx=0x7f00f003fec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f55eea937f7d16f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.508+0000 7f00dffff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.508+0000 7f00dffff640 1 -- 172.21.15.94:0/1927978787 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f002c5f0 con 0x55930721cd60 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.508+0000 7f00dffff640 1 -- 172.21.15.94:0/1927978787 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f00f0050040 con 0x55930721cd60 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.508+0000 7f01099ad740 1 -- 172.21.15.94:0/1927978787 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00f002c990 con 0x55930721cd60 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.508+0000 7f00dffff640 1 -- 172.21.15.94:0/1927978787 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f0005000 con 0x55930721cd60 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.508+0000 7f01099ad740 10 client.27974 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.508+0000 7f01099ad740 1 -- 172.21.15.94:0/1927978787 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f01001acd50 con 0x55930721cd60 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.508+0000 7f00dffff640 1 -- 172.21.15.94:0/1927978787 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00f006b080 con 0x55930721cd60 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.509+0000 7f00dffff640 1 -- 172.21.15.94:0/1927978787 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f00f003c040 con 0x55930721cd60 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.509+0000 7f01099ad740 10 client.27974 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.509+0000 7f01099ad740 10 client.27974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.509+0000 7f01099ad740 20 client.27974 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.509+0000 7f01099ad740 1 --2- 172.21.15.94:0/1927978787 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55930728da50 0x55930728fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.509+0000 7f01099ad740 4 client.27974 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:09:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.509+0000 7f01099ad740 1 -- 172.21.15.94:0/1927978787 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f00f002c990 con 0x55930728da50 2024-02-17T12:09:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.509+0000 7f00fffff640 1 --2- 172.21.15.94:0/1927978787 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55930728da50 0x55930728fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.509+0000 7f00dffff640 10 client.27974 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.509+0000 7f00fffff640 1 --2- 172.21.15.94:0/1927978787 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55930728da50 0x55930728fe70 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a205e58bda33d97 server_cookie=a26b5f6364dbef68 in_seq=0 out_seq=0 2024-02-17T12:09:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f00dffff640 1 -- 172.21.15.94:0/1927978787 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f00f002c990 con 0x55930728da50 2024-02-17T12:09:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f00dffff640 10 client.27974 handle_command_reply: tid=0 2024-02-17T12:09:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f01099ad740 1 client.27974 shutdown 2024-02-17T12:09:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f01099ad740 1 -- 172.21.15.94:0/1927978787 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721cd60 msgr2=0x55930722bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f01099ad740 1 --2- 172.21.15.94:0/1927978787 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721cd60 0x55930722bba0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f00f0006010 tx=0x7f00f003fec0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f01099ad740 1 -- 172.21.15.94:0/1927978787 shutdown_connections 2024-02-17T12:09:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f01099ad740 1 --2- 172.21.15.94:0/1927978787 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55930728da50 0x55930728fe70 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f01099ad740 1 --2- 172.21.15.94:0/1927978787 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5593072262e0 0x559307232900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f00dffff640 0 client.27974 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f01099ad740 1 --2- 172.21.15.94:0/1927978787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55930721dad0 0x5593072321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f01099ad740 1 --2- 172.21.15.94:0/1927978787 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55930721cd60 0x55930722bba0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f01099ad740 1 -- 172.21.15.94:0/1927978787 >> 172.21.15.94:0/1927978787 conn(0x5593071e4550 msgr2=0x55930721c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f01099ad740 1 -- 172.21.15.94:0/1927978787 shutdown_connections 2024-02-17T12:09:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f01099ad740 1 -- 172.21.15.94:0/1927978787 wait complete. 2024-02-17T12:09:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.531+0000 7f01099ad740 20 client.27974 trim_cache size 0 max 16384 2024-02-17T12:09:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:09:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:09:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:09:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:09:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.532+0000 7f0106cfb640 1 -- 172.21.15.94:0/3159330251 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f00ec0a2430 msgr2=0x7f00ec0a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.532+0000 7f0106cfb640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f00ec0a2430 0x7f00ec0a48f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f00f8009900 tx=0x7f00f8009650 comp rx=0 tx=0).stop 2024-02-17T12:09:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.532+0000 7f0106cfb640 1 -- 172.21.15.94:0/3159330251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0100108cf0 msgr2=0x7f01001a0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.532+0000 7f0106cfb640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0100108cf0 0x7f01001a0540 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f00e8009fd0 tx=0x7f00e800b870 comp rx=0 tx=0).stop 2024-02-17T12:09:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.532+0000 7f0106cfb640 1 -- 172.21.15.94:0/3159330251 shutdown_connections 2024-02-17T12:09:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.532+0000 7f0106cfb640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f00ec0a2430 0x7f00ec0a48f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.532+0000 7f0106cfb640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f01001142d0 0x7f01001a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.532+0000 7f0106cfb640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0100108cf0 0x7f01001a0540 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.532+0000 7f0106cfb640 1 --2- 172.21.15.94:0/3159330251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f01001083d0 0x7f01001a0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.532+0000 7f0106cfb640 1 -- 172.21.15.94:0/3159330251 >> 172.21.15.94:0/3159330251 conn(0x7f010006c220 msgr2=0x7f010006fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.532+0000 7f0106cfb640 1 -- 172.21.15.94:0/3159330251 shutdown_connections 2024-02-17T12:09:28.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:28.532+0000 7f0106cfb640 1 -- 172.21.15.94:0/3159330251 wait complete. 2024-02-17T12:09:28.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:28 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:09:28.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:28 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:09:28.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:28 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:09:29.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:29 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:09:29.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:29 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:09:29.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:29 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:09:29.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:29 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:09:29.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:29 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:09:29.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:29 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:09:29.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:29 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:09:29.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:29 smithi125 ceph-mon[36819]: pgmap v354: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 9.4 MiB/s wr, 852 op/s 2024-02-17T12:09:29.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:29 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:09:29.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:29 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:09:29.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:29 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:09:29.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:29 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:09:29.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:29 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:09:29.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:29 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:09:29.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:29 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:09:29.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:29 smithi094 ceph-mon[32499]: pgmap v354: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 9.4 MiB/s wr, 852 op/s 2024-02-17T12:09:29.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:29 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:09:29.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:29 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:09:29.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:29 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:09:29.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:29 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:09:29.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:29 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:09:29.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:29 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:09:29.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:29 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:09:29.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:29 smithi189 ceph-mon[36588]: pgmap v354: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 9.4 MiB/s wr, 852 op/s 2024-02-17T12:09:30.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 670.0 s, 1836.6 tps, lat 54.739 ms stddev 62.637 2024-02-17T12:09:31.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:31 smithi189 ceph-mon[36588]: pgmap v355: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.8 MiB/s wr, 626 op/s 2024-02-17T12:09:31.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:31 smithi125 ceph-mon[36819]: pgmap v355: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.8 MiB/s wr, 626 op/s 2024-02-17T12:09:31.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:31 smithi094 ceph-mon[32499]: pgmap v355: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.8 MiB/s wr, 626 op/s 2024-02-17T12:09:32.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:32 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:09:32.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:32 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:09:32.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:32 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:09:33.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:33 smithi189 ceph-mon[36588]: pgmap v356: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.6 MiB/s wr, 600 op/s 2024-02-17T12:09:33.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:33 smithi125 ceph-mon[36819]: pgmap v356: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.6 MiB/s wr, 600 op/s 2024-02-17T12:09:33.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:33 smithi094 ceph-mon[32499]: pgmap v356: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 6.6 MiB/s wr, 600 op/s 2024-02-17T12:09:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 675.0 s, 1803.8 tps, lat 54.588 ms stddev 67.677 2024-02-17T12:09:35.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:35 smithi189 ceph-mon[36588]: pgmap v357: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 11 MiB/s wr, 898 op/s 2024-02-17T12:09:35.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:35 smithi125 ceph-mon[36819]: pgmap v357: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 11 MiB/s wr, 898 op/s 2024-02-17T12:09:35.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:35 smithi094 ceph-mon[32499]: pgmap v357: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 11 MiB/s wr, 898 op/s 2024-02-17T12:09:37.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:37 smithi189 ceph-mon[36588]: pgmap v358: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.3 MiB/s wr, 633 op/s 2024-02-17T12:09:37.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:37 smithi125 ceph-mon[36819]: pgmap v358: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.3 MiB/s wr, 633 op/s 2024-02-17T12:09:37.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:37 smithi094 ceph-mon[32499]: pgmap v358: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 7.3 MiB/s wr, 633 op/s 2024-02-17T12:09:39.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:39 smithi189 ceph-mon[36588]: pgmap v359: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 10 MiB/s wr, 931 op/s 2024-02-17T12:09:39.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:39 smithi125 ceph-mon[36819]: pgmap v359: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 10 MiB/s wr, 931 op/s 2024-02-17T12:09:39.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:39 smithi094 ceph-mon[32499]: pgmap v359: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 10 MiB/s wr, 931 op/s 2024-02-17T12:09:40.247 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 680.0 s, 1667.6 tps, lat 60.748 ms stddev 66.447 2024-02-17T12:09:41.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:41 smithi189 ceph-mon[36588]: pgmap v360: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.4 MiB/s wr, 682 op/s 2024-02-17T12:09:41.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:41 smithi125 ceph-mon[36819]: pgmap v360: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.4 MiB/s wr, 682 op/s 2024-02-17T12:09:41.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:41 smithi094 ceph-mon[32499]: pgmap v360: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 8.4 MiB/s wr, 682 op/s 2024-02-17T12:09:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.667+0000 7f8757fe2640 1 -- 172.21.15.94:0/1036632705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8750108110 msgr2=0x7f8750114620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:41.668 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.667+0000 7f8757fe2640 1 --2- 172.21.15.94:0/1036632705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8750108110 0x7f8750114620 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8740009930 tx=0x7f874002bc30 comp rx=0 tx=0).stop 2024-02-17T12:09:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.667+0000 7f8757fe2640 1 -- 172.21.15.94:0/1036632705 shutdown_connections 2024-02-17T12:09:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.667+0000 7f8757fe2640 1 --2- 172.21.15.94:0/1036632705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8750108110 0x7f8750114620 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.667+0000 7f8757fe2640 1 --2- 172.21.15.94:0/1036632705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750107750 0x7f8750107bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.667+0000 7f8757fe2640 1 --2- 172.21.15.94:0/1036632705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f875010d760 0x7f875010db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.667+0000 7f8757fe2640 1 -- 172.21.15.94:0/1036632705 >> 172.21.15.94:0/1036632705 conn(0x7f87500756f0 msgr2=0x7f8750077b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.667+0000 7f8757fe2640 1 -- 172.21.15.94:0/1036632705 shutdown_connections 2024-02-17T12:09:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.668+0000 7f8757fe2640 1 -- 172.21.15.94:0/1036632705 wait complete. 2024-02-17T12:09:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.668+0000 7f8757fe2640 1 Processor -- start 2024-02-17T12:09:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.668+0000 7f8757fe2640 1 -- start start 2024-02-17T12:09:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.668+0000 7f8757fe2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750107750 0x7f8750072710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.669+0000 7f8757fe2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8750108110 0x7f8750072c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.669+0000 7f8755d7e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750107750 0x7f8750072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.669+0000 7f8755d7e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750107750 0x7f8750072710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:32812/0 (socket says 172.21.15.94:32812) 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.669+0000 7f8755d7e640 1 -- 172.21.15.94:0/3324783989 learned_addr learned my addr 172.21.15.94:0/3324783989 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.669+0000 7f8757fe2640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f875010d760 0x7f875006b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.669+0000 7f8757fe2640 1 -- 172.21.15.94:0/3324783989 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875006bf20 con 0x7f875010d760 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.669+0000 7f8757fe2640 1 -- 172.21.15.94:0/3324783989 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875006c090 con 0x7f8750107750 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.669+0000 7f8757fe2640 1 -- 172.21.15.94:0/3324783989 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875006c250 con 0x7f8750108110 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.670+0000 7f875657f640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f875010d760 0x7f875006b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.670+0000 7f875557d640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8750108110 0x7f8750072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.670+0000 7f8755d7e640 1 -- 172.21.15.94:0/3324783989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8750108110 msgr2=0x7f8750072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.670+0000 7f8755d7e640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8750108110 0x7f8750072c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.670+0000 7f8755d7e640 1 -- 172.21.15.94:0/3324783989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f875010d760 msgr2=0x7f875006b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.670+0000 7f8755d7e640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f875010d760 0x7f875006b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.670+0000 7f8755d7e640 1 -- 172.21.15.94:0/3324783989 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8740009560 con 0x7f8750107750 2024-02-17T12:09:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.670+0000 7f875557d640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8750108110 0x7f8750072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:09:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.670+0000 7f8755d7e640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750107750 0x7f8750072710 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f874c00e9f0 tx=0x7f874c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.671+0000 7f8746ffd640 1 -- 172.21.15.94:0/3324783989 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c00cd80 con 0x7f8750107750 2024-02-17T12:09:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.671+0000 7f8757fe2640 1 -- 172.21.15.94:0/3324783989 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875006c530 con 0x7f8750107750 2024-02-17T12:09:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.671+0000 7f8757fe2640 1 -- 172.21.15.94:0/3324783989 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8750070410 con 0x7f8750107750 2024-02-17T12:09:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.671+0000 7f8746ffd640 1 -- 172.21.15.94:0/3324783989 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f874c010910 con 0x7f8750107750 2024-02-17T12:09:41.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.672+0000 7f8746ffd640 1 -- 172.21.15.94:0/3324783989 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c0188e0 con 0x7f8750107750 2024-02-17T12:09:41.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.674+0000 7f8746ffd640 1 -- 172.21.15.94:0/3324783989 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f874c018ae0 con 0x7f8750107750 2024-02-17T12:09:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.674+0000 7f8746ffd640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f873c0a2430 0x7f873c0a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.675+0000 7f8746ffd640 1 -- 172.21.15.94:0/3324783989 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f874c014070 con 0x7f8750107750 2024-02-17T12:09:41.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.675+0000 7f875abb1740 1 Processor -- start 2024-02-17T12:09:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.675+0000 7f875557d640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f873c0a2430 0x7f873c0a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.675+0000 7f875abb1740 1 -- start start 2024-02-17T12:09:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.675+0000 7f875abb1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.675+0000 7f875abb1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8ced60 0x55cb4e8cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.675+0000 7f875abb1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8da6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.675+0000 7f875abb1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4e83d9f0 con 0x55cb4e8cfad0 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.675+0000 7f875abb1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4e83af20 con 0x55cb4e8ced60 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.675+0000 7f875abb1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4e836700 con 0x55cb4e8d82e0 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f8755d7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f875657f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8da6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f875557d640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f873c0a2430 0x7f873c0a48f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f875006cf50 tx=0x7f87480073c0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f875657f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8da6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:37518/0 (socket says 172.21.15.94:37518) 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f8755d7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44774/0 (socket says 172.21.15.94:44774) 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f875657f640 1 -- 172.21.15.94:0/1244789947 learned_addr learned my addr 172.21.15.94:0/1244789947 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f875557d640 1 --2- 172.21.15.94:0/1244789947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8ced60 0x55cb4e8cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f875657f640 1 -- 172.21.15.94:0/1244789947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8ced60 msgr2=0x55cb4e8cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f875657f640 1 --2- 172.21.15.94:0/1244789947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8ced60 0x55cb4e8cf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.684 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f875657f640 1 -- 172.21.15.94:0/1244789947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8cfad0 msgr2=0x55cb4e8ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f875657f640 1 --2- 172.21.15.94:0/1244789947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f875657f640 1 -- 172.21.15.94:0/1244789947 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb4e7cc7b0 con 0x55cb4e8d82e0 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f875657f640 1 --2- 172.21.15.94:0/1244789947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8da6d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8740009430 tx=0x7f874002c5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=394b05a0f391c19b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.676+0000 7f873bfff640 1 -- 172.21.15.94:0/1244789947 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740004940 con 0x55cb4e8d82e0 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.677+0000 7f873bfff640 1 -- 172.21.15.94:0/1244789947 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8740004aa0 con 0x55cb4e8d82e0 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.677+0000 7f873bfff640 1 -- 172.21.15.94:0/1244789947 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740005210 con 0x55cb4e8d82e0 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.677+0000 7f875abb1740 1 -- 172.21.15.94:0/1244789947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8d82e0 msgr2=0x55cb4e8da6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.677+0000 7f875abb1740 1 --2- 172.21.15.94:0/1244789947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8da6d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8740009430 tx=0x7f874002c5d0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.677+0000 7f875abb1740 1 -- 172.21.15.94:0/1244789947 shutdown_connections 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.677+0000 7f875abb1740 1 --2- 172.21.15.94:0/1244789947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8da6d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.677+0000 7f875abb1740 1 --2- 172.21.15.94:0/1244789947 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8ced60 0x55cb4e8cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.677+0000 7f875abb1740 1 --2- 172.21.15.94:0/1244789947 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.677+0000 7f875abb1740 1 -- 172.21.15.94:0/1244789947 >> 172.21.15.94:0/1244789947 conn(0x55cb4e896550 msgr2=0x55cb4e8c9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:41.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.677+0000 7f875abb1740 1 -- 172.21.15.94:0/1244789947 shutdown_connections 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.677+0000 7f875abb1740 1 -- 172.21.15.94:0/1244789947 wait complete. 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.678+0000 7f875abb1740 1 -- start start 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.678+0000 7f875abb1740 1 Processor -- start 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.678+0000 7f875abb1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8ced60 0x55cb4e8d51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.678+0000 7f875abb1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8d56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f875abb1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8d5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f8755d7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8ced60 0x55cb4e8d51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f875abb1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4e7dbbf0 con 0x55cb4e8ced60 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f8755d7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8ced60 0x55cb4e8d51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44784/0 (socket says 172.21.15.94:44784) 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f8755d7e640 1 -- 172.21.15.94:0/2459307187 learned_addr learned my addr 172.21.15.94:0/2459307187 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f875abb1740 1 -- 172.21.15.94:0/2459307187 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4e50a250 con 0x55cb4e8cfad0 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f875abb1740 1 -- 172.21.15.94:0/2459307187 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4e7bc200 con 0x55cb4e8d82e0 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f875557d640 1 --2- 172.21.15.94:0/2459307187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8d56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f875657f640 1 --2- 172.21.15.94:0/2459307187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8d5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f875abb1740 10 client.0 _ll_register_callbacks cb 0x55cb4e82ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:41.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f875657f640 1 -- 172.21.15.94:0/2459307187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8cfad0 msgr2=0x55cb4e8d56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f875657f640 1 --2- 172.21.15.94:0/2459307187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8d56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f875657f640 1 -- 172.21.15.94:0/2459307187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8ced60 msgr2=0x55cb4e8d51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f875657f640 1 --2- 172.21.15.94:0/2459307187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8ced60 0x55cb4e8d51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f875657f640 1 -- 172.21.15.94:0/2459307187 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8750114f20 con 0x55cb4e8d82e0 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.679+0000 7f875657f640 1 --2- 172.21.15.94:0/2459307187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8d5c30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8740005940 tx=0x7f87400042c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=987135317ef41cb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.680+0000 7f87397fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.680+0000 7f87397fa640 1 -- 172.21.15.94:0/2459307187 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874002cbf0 con 0x55cb4e8d82e0 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.680+0000 7f87397fa640 1 -- 172.21.15.94:0/2459307187 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8740004f90 con 0x55cb4e8d82e0 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.680+0000 7f875abb1740 1 -- 172.21.15.94:0/2459307187 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb4e536600 con 0x55cb4e8d82e0 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.680+0000 7f87397fa640 1 -- 172.21.15.94:0/2459307187 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8740005170 con 0x55cb4e8d82e0 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.680+0000 7f875abb1740 10 client.27979 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.680+0000 7f875abb1740 1 -- 172.21.15.94:0/2459307187 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cb4e834d80 con 0x55cb4e8d82e0 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.681+0000 7f87397fa640 1 -- 172.21.15.94:0/2459307187 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87400035f0 con 0x55cb4e8d82e0 2024-02-17T12:09:41.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.681+0000 7f87397fa640 1 -- 172.21.15.94:0/2459307187 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f874003c040 con 0x55cb4e8d82e0 2024-02-17T12:09:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.681+0000 7f875abb1740 10 client.27979 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.681+0000 7f875abb1740 10 client.27979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.681+0000 7f875abb1740 20 client.27979 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.681+0000 7f875abb1740 1 --2- 172.21.15.94:0/2459307187 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cb4e8e81d0 0x55cb4e8ea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.681+0000 7f875abb1740 4 client.27979 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.681+0000 7f875abb1740 1 -- 172.21.15.94:0/2459307187 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb4e536600 con 0x55cb4e8e81d0 2024-02-17T12:09:41.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.682+0000 7f8755d7e640 1 --2- 172.21.15.94:0/2459307187 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cb4e8e81d0 0x55cb4e8ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.682+0000 7f87397fa640 10 client.27979 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.682+0000 7f8755d7e640 1 --2- 172.21.15.94:0/2459307187 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cb4e8e81d0 0x55cb4e8ea650 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e794aa0403fcf0d6 server_cookie=1e77aab20af11001 in_seq=0 out_seq=0 2024-02-17T12:09:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f87397fa640 1 -- 172.21.15.94:0/2459307187 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f874c07ae90 con 0x55cb4e8e81d0 2024-02-17T12:09:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f87397fa640 10 client.27979 handle_command_reply: tid=0 2024-02-17T12:09:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f875abb1740 1 client.27979 shutdown 2024-02-17T12:09:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f875abb1740 1 -- 172.21.15.94:0/2459307187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8d82e0 msgr2=0x55cb4e8d5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f875abb1740 1 --2- 172.21.15.94:0/2459307187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8d5c30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8740005940 tx=0x7f87400042c0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f875abb1740 1 -- 172.21.15.94:0/2459307187 shutdown_connections 2024-02-17T12:09:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f875abb1740 1 --2- 172.21.15.94:0/2459307187 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cb4e8e81d0 0x55cb4e8ea650 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f875abb1740 1 --2- 172.21.15.94:0/2459307187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8d5c30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f87397fa640 0 client.27979 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f875abb1740 1 --2- 172.21.15.94:0/2459307187 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8d56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f875abb1740 1 --2- 172.21.15.94:0/2459307187 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8ced60 0x55cb4e8d51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f875abb1740 1 -- 172.21.15.94:0/2459307187 >> 172.21.15.94:0/2459307187 conn(0x55cb4e896550 msgr2=0x55cb4e8cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f875abb1740 1 -- 172.21.15.94:0/2459307187 shutdown_connections 2024-02-17T12:09:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.683+0000 7f875abb1740 1 -- 172.21.15.94:0/2459307187 wait complete. 2024-02-17T12:09:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.684+0000 7f875abb1740 20 client.27979 trim_cache size 0 max 16384 2024-02-17T12:09:41.702 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.700+0000 7f875abb1740 1 Processor -- start 2024-02-17T12:09:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.700+0000 7f875abb1740 1 -- start start 2024-02-17T12:09:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.700+0000 7f875abb1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8ced60 0x55cb4e8d78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.700+0000 7f875abb1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.700+0000 7f875abb1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f875abb1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4e8c9f90 con 0x55cb4e8d82e0 2024-02-17T12:09:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f875abb1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4e8ce030 con 0x55cb4e8ced60 2024-02-17T12:09:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f875abb1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4e8ce8f0 con 0x55cb4e8cfad0 2024-02-17T12:09:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f8755d7e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8ced60 0x55cb4e8d78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f8755d7e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8ced60 0x55cb4e8d78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:32836/0 (socket says 172.21.15.94:32836) 2024-02-17T12:09:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f8755d7e640 1 -- 172.21.15.94:0/2358017133 learned_addr learned my addr 172.21.15.94:0/2358017133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f875657f640 1 --2- 172.21.15.94:0/2358017133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f875557d640 1 --2- 172.21.15.94:0/2358017133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.703 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f875557d640 1 -- 172.21.15.94:0/2358017133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8ced60 msgr2=0x55cb4e8d78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f875557d640 1 --2- 172.21.15.94:0/2358017133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8ced60 0x55cb4e8d78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f875557d640 1 -- 172.21.15.94:0/2358017133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8d82e0 msgr2=0x55cb4e8d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f875557d640 1 --2- 172.21.15.94:0/2358017133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f875557d640 1 -- 172.21.15.94:0/2358017133 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb4e7d17b0 con 0x55cb4e8cfad0 2024-02-17T12:09:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.701+0000 7f875557d640 1 --2- 172.21.15.94:0/2358017133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8eefb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f87480077a0 tx=0x7f874800f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12429c63dd48b54a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.702+0000 7f873b7fe640 1 -- 172.21.15.94:0/2358017133 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874801d800 con 0x55cb4e8cfad0 2024-02-17T12:09:41.704 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.702+0000 7f873b7fe640 1 -- 172.21.15.94:0/2358017133 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f874801de40 con 0x55cb4e8cfad0 2024-02-17T12:09:41.705 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.702+0000 7f873b7fe640 1 -- 172.21.15.94:0/2358017133 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748026e00 con 0x55cb4e8cfad0 2024-02-17T12:09:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.703+0000 7f875abb1740 1 -- 172.21.15.94:0/2358017133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8cfad0 msgr2=0x55cb4e8eefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.703+0000 7f875abb1740 1 --2- 172.21.15.94:0/2358017133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8eefb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f87480077a0 tx=0x7f874800f000 comp rx=0 tx=0).stop 2024-02-17T12:09:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.703+0000 7f875abb1740 1 -- 172.21.15.94:0/2358017133 shutdown_connections 2024-02-17T12:09:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.703+0000 7f875abb1740 1 --2- 172.21.15.94:0/2358017133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8d82e0 0x55cb4e8d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.703+0000 7f875abb1740 1 --2- 172.21.15.94:0/2358017133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8eefb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.703+0000 7f875abb1740 1 --2- 172.21.15.94:0/2358017133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8ced60 0x55cb4e8d78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.703+0000 7f875abb1740 1 -- 172.21.15.94:0/2358017133 >> 172.21.15.94:0/2358017133 conn(0x55cb4e896550 msgr2=0x55cb4e8d3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.704+0000 7f875abb1740 1 -- 172.21.15.94:0/2358017133 shutdown_connections 2024-02-17T12:09:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.704+0000 7f875abb1740 1 -- 172.21.15.94:0/2358017133 wait complete. 2024-02-17T12:09:41.706 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.704+0000 7f875abb1740 1 -- start start 2024-02-17T12:09:41.707 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.704+0000 7f875abb1740 1 Processor -- start 2024-02-17T12:09:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f875abb1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8ced60 0x55cb4e8de460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.711 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f875abb1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8de9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f875abb1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8d82e0 0x55cb4e938780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f8755d7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8ced60 0x55cb4e8de460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f875abb1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4e938cc0 con 0x55cb4e8ced60 2024-02-17T12:09:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f8755d7e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8ced60 0x55cb4e8de460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44810/0 (socket says 172.21.15.94:44810) 2024-02-17T12:09:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f8755d7e640 1 -- 172.21.15.94:0/2550767515 learned_addr learned my addr 172.21.15.94:0/2550767515 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f875abb1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4e938e00 con 0x55cb4e8d82e0 2024-02-17T12:09:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f875557d640 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8de9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f875abb1740 1 -- 172.21.15.94:0/2550767515 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb4e938fc0 con 0x55cb4e8cfad0 2024-02-17T12:09:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f875657f640 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8d82e0 0x55cb4e938780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f875abb1740 10 client.0 _ll_register_callbacks cb 0x7f874c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f8755d7e640 1 -- 172.21.15.94:0/2550767515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8cfad0 msgr2=0x55cb4e8de9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:41.712 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f8755d7e640 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8de9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f8755d7e640 1 -- 172.21.15.94:0/2550767515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8d82e0 msgr2=0x55cb4e938780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f8755d7e640 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8d82e0 0x55cb4e938780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f8755d7e640 1 -- 172.21.15.94:0/2550767515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874800f440 con 0x55cb4e8ced60 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f875557d640 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8de9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.705+0000 7f875657f640 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8d82e0 0x55cb4e938780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.706+0000 7f8755d7e640 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8ced60 0x55cb4e8de460 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f874c0ce560 tx=0x7f874c0ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc749af1e5021293 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.706+0000 7f87397fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.706+0000 7f87397fa640 1 -- 172.21.15.94:0/2550767515 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c09f090 con 0x55cb4e8ced60 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.706+0000 7f87397fa640 1 -- 172.21.15.94:0/2550767515 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f874c0b3040 con 0x55cb4e8ced60 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.706+0000 7f875abb1740 1 -- 172.21.15.94:0/2550767515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8740004aa0 con 0x55cb4e8ced60 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.706+0000 7f87397fa640 1 -- 172.21.15.94:0/2550767515 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f874c0a36e0 con 0x55cb4e8ced60 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.706+0000 7f875abb1740 10 client.18525 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.706+0000 7f875abb1740 1 -- 172.21.15.94:0/2550767515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cb4e7f7540 con 0x55cb4e8ced60 2024-02-17T12:09:41.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.706+0000 7f87397fa640 1 -- 172.21.15.94:0/2550767515 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8740004aa0 con 0x55cb4e8ced60 2024-02-17T12:09:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.707+0000 7f87397fa640 1 -- 172.21.15.94:0/2550767515 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f874c0b92c0 con 0x55cb4e8ced60 2024-02-17T12:09:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.707+0000 7f875abb1740 10 client.18525 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.707+0000 7f875abb1740 10 client.18525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.707+0000 7f875abb1740 20 client.18525 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.707+0000 7f875abb1740 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cb4e8e81d0 0x55cb4e9428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.707+0000 7f875abb1740 4 client.18525 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:09:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.707+0000 7f875abb1740 1 -- 172.21.15.94:0/2550767515 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cb4e5871b0 con 0x55cb4e8e81d0 2024-02-17T12:09:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.707+0000 7f875557d640 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cb4e8e81d0 0x55cb4e9428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.707+0000 7f875557d640 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cb4e8e81d0 0x55cb4e9428a0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6992500be342fc01 server_cookie=f71a6df473ebb5c7 in_seq=0 out_seq=0 2024-02-17T12:09:41.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.707+0000 7f87397fa640 10 client.18525 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f87397fa640 1 -- 172.21.15.94:0/2550767515 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cb4e5871b0 con 0x55cb4e8e81d0 2024-02-17T12:09:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f87397fa640 10 client.18525 handle_command_reply: tid=0 2024-02-17T12:09:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f875abb1740 1 client.18525 shutdown 2024-02-17T12:09:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f875abb1740 1 -- 172.21.15.94:0/2550767515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8ced60 msgr2=0x55cb4e8de460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:41.732 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f875abb1740 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8ced60 0x55cb4e8de460 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f874c0ce560 tx=0x7f874c0ce040 comp rx=0 tx=0).stop 2024-02-17T12:09:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f875abb1740 1 -- 172.21.15.94:0/2550767515 shutdown_connections 2024-02-17T12:09:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f875abb1740 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cb4e8e81d0 0x55cb4e9428a0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f875abb1740 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cb4e8d82e0 0x55cb4e938780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f875abb1740 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb4e8cfad0 0x55cb4e8de9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f87397fa640 0 client.18525 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f875abb1740 1 --2- 172.21.15.94:0/2550767515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cb4e8ced60 0x55cb4e8de460 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f875abb1740 1 -- 172.21.15.94:0/2550767515 >> 172.21.15.94:0/2550767515 conn(0x55cb4e896550 msgr2=0x55cb4e8eefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f875abb1740 1 -- 172.21.15.94:0/2550767515 shutdown_connections 2024-02-17T12:09:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.730+0000 7f875abb1740 1 -- 172.21.15.94:0/2550767515 wait complete. 2024-02-17T12:09:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.731+0000 7f875abb1740 20 client.18525 trim_cache size 0 max 16384 2024-02-17T12:09:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:09:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:09:41.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:09:41.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:09:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.732+0000 7f8757fe2640 1 -- 172.21.15.94:0/3324783989 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f873c0a2430 msgr2=0x7f873c0a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.732+0000 7f8757fe2640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f873c0a2430 0x7f873c0a48f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f875006cf50 tx=0x7f87480073c0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.732+0000 7f8757fe2640 1 -- 172.21.15.94:0/3324783989 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750107750 msgr2=0x7f8750072710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.732+0000 7f8757fe2640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750107750 0x7f8750072710 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f874c00e9f0 tx=0x7f874c00eeb0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.733+0000 7f8757fe2640 1 -- 172.21.15.94:0/3324783989 shutdown_connections 2024-02-17T12:09:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.733+0000 7f8757fe2640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f873c0a2430 0x7f873c0a48f0 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f875006cf50 tx=0x7f87480073c0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.733+0000 7f8757fe2640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f875010d760 0x7f875006b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.733+0000 7f8757fe2640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8750108110 0x7f8750072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.733+0000 7f8757fe2640 1 --2- 172.21.15.94:0/3324783989 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8750107750 0x7f8750072710 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.733+0000 7f8757fe2640 1 -- 172.21.15.94:0/3324783989 >> 172.21.15.94:0/3324783989 conn(0x7f87500756f0 msgr2=0x7f8750075bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.733+0000 7f8757fe2640 1 -- 172.21.15.94:0/3324783989 shutdown_connections 2024-02-17T12:09:41.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:41.733+0000 7f8757fe2640 1 -- 172.21.15.94:0/3324783989 wait complete. 2024-02-17T12:09:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:43 smithi189 ceph-mon[36588]: pgmap v361: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.6 MiB/s wr, 623 op/s 2024-02-17T12:09:43.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:43 smithi125 ceph-mon[36819]: pgmap v361: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.6 MiB/s wr, 623 op/s 2024-02-17T12:09:43.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:43 smithi094 ceph-mon[32499]: pgmap v361: 97 pgs: 97 active+clean; 5.5 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 7.6 MiB/s wr, 623 op/s 2024-02-17T12:09:45.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 685.0 s, 1255.4 tps, lat 79.746 ms stddev 144.531 2024-02-17T12:09:45.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:45 smithi189 ceph-mon[36588]: pgmap v362: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 34 MiB/s wr, 1.77k op/s 2024-02-17T12:09:45.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:45 smithi125 ceph-mon[36819]: pgmap v362: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 34 MiB/s wr, 1.77k op/s 2024-02-17T12:09:45.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:45 smithi094 ceph-mon[32499]: pgmap v362: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 34 MiB/s wr, 1.77k op/s 2024-02-17T12:09:47.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:47 smithi189 ceph-mon[36588]: pgmap v363: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 MiB/s wr, 1.47k op/s 2024-02-17T12:09:47.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:47 smithi125 ceph-mon[36819]: pgmap v363: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 MiB/s wr, 1.47k op/s 2024-02-17T12:09:47.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:47 smithi094 ceph-mon[32499]: pgmap v363: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 30 MiB/s wr, 1.47k op/s 2024-02-17T12:09:49.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:49 smithi189 ceph-mon[36588]: pgmap v364: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 MiB/s wr, 1.68k op/s 2024-02-17T12:09:49.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:49 smithi125 ceph-mon[36819]: pgmap v364: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 MiB/s wr, 1.68k op/s 2024-02-17T12:09:49.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:49 smithi094 ceph-mon[32499]: pgmap v364: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 32 MiB/s wr, 1.68k op/s 2024-02-17T12:09:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 690.0 s, 1708.8 tps, lat 58.326 ms stddev 63.874 2024-02-17T12:09:51.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:51 smithi189 ceph-mon[36588]: pgmap v365: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 MiB/s wr, 1.41k op/s 2024-02-17T12:09:51.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:51 smithi125 ceph-mon[36819]: pgmap v365: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 MiB/s wr, 1.41k op/s 2024-02-17T12:09:51.491 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:51 smithi094 ceph-mon[32499]: pgmap v365: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 MiB/s wr, 1.41k op/s 2024-02-17T12:09:53.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:53 smithi189 ceph-mon[36588]: pgmap v366: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 MiB/s wr, 1.39k op/s 2024-02-17T12:09:53.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:53 smithi125 ceph-mon[36819]: pgmap v366: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 MiB/s wr, 1.39k op/s 2024-02-17T12:09:53.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:53 smithi094 ceph-mon[32499]: pgmap v366: 97 pgs: 97 active+clean; 5.8 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 29 MiB/s wr, 1.39k op/s 2024-02-17T12:09:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.891+0000 7f472a1c9640 1 -- 172.21.15.94:0/3174486651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472411a960 msgr2=0x7f472411cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:54.893 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.891+0000 7f472a1c9640 1 --2- 172.21.15.94:0/3174486651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472411a960 0x7f472411cd50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f471c009be0 tx=0x7f471c02bf10 comp rx=0 tx=0).stop 2024-02-17T12:09:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.893+0000 7f472a1c9640 1 -- 172.21.15.94:0/3174486651 shutdown_connections 2024-02-17T12:09:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.893+0000 7f472a1c9640 1 --2- 172.21.15.94:0/3174486651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472411a960 0x7f472411cd50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.895 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.893+0000 7f472a1c9640 1 --2- 172.21.15.94:0/3174486651 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f472410fd30 0x7f472410da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.893+0000 7f472a1c9640 1 --2- 172.21.15.94:0/3174486651 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f472410f8c0 0x7f472410d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.893+0000 7f472a1c9640 1 -- 172.21.15.94:0/3174486651 >> 172.21.15.94:0/3174486651 conn(0x7f472406c220 msgr2=0x7f472406c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:54.896 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.894+0000 7f472a1c9640 1 -- 172.21.15.94:0/3174486651 shutdown_connections 2024-02-17T12:09:54.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.897+0000 7f472a1c9640 1 -- 172.21.15.94:0/3174486651 wait complete. 2024-02-17T12:09:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.897+0000 7f472a1c9640 1 Processor -- start 2024-02-17T12:09:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.898+0000 7f472a1c9640 1 -- start start 2024-02-17T12:09:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.898+0000 7f472a1c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f472410f8c0 0x7f4724117d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.898+0000 7f472a1c9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472410fd30 0x7f4724118270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.898+0000 7f472a1c9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f472411a960 0x7f4724110df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.898+0000 7f472a1c9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4724111580 con 0x7f472410f8c0 2024-02-17T12:09:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.898+0000 7f472a1c9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47241116f0 con 0x7f472411a960 2024-02-17T12:09:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.898+0000 7f472a1c9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47241118b0 con 0x7f472410fd30 2024-02-17T12:09:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.899+0000 7f4723fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f472411a960 0x7f4724110df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.899+0000 7f4722ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472410fd30 0x7f4724118270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.899+0000 7f4722ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472410fd30 0x7f4724118270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47128/0 (socket says 172.21.15.94:47128) 2024-02-17T12:09:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.899+0000 7f4722ffd640 1 -- 172.21.15.94:0/856002384 learned_addr learned my addr 172.21.15.94:0/856002384 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.899+0000 7f47237fe640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f472410f8c0 0x7f4724117d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.899+0000 7f4722ffd640 1 -- 172.21.15.94:0/856002384 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f472411a960 msgr2=0x7f4724110df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.899+0000 7f4722ffd640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f472411a960 0x7f4724110df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.899+0000 7f4722ffd640 1 -- 172.21.15.94:0/856002384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f472410f8c0 msgr2=0x7f4724117d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.899+0000 7f4722ffd640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f472410f8c0 0x7f4724117d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.899+0000 7f4722ffd640 1 -- 172.21.15.94:0/856002384 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c009840 con 0x7f472410fd30 2024-02-17T12:09:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.899+0000 7f47237fe640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f472410f8c0 0x7f4724117d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:09:54.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.899+0000 7f4722ffd640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472410fd30 0x7f4724118270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f471400e9f0 tx=0x7f471400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.900+0000 7f4720ff9640 1 -- 172.21.15.94:0/856002384 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714009600 con 0x7f472410fd30 2024-02-17T12:09:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.900+0000 7f4720ff9640 1 -- 172.21.15.94:0/856002384 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f471400ccc0 con 0x7f472410fd30 2024-02-17T12:09:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.901+0000 7f4720ff9640 1 -- 172.21.15.94:0/856002384 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714018750 con 0x7f472410fd30 2024-02-17T12:09:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.901+0000 7f472a1c9640 1 -- 172.21.15.94:0/856002384 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4724111b90 con 0x7f472410fd30 2024-02-17T12:09:54.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.901+0000 7f472a1c9640 1 -- 172.21.15.94:0/856002384 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4724115a30 con 0x7f472410fd30 2024-02-17T12:09:54.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.905+0000 7f4720ff9640 1 -- 172.21.15.94:0/856002384 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f47140188b0 con 0x7f472410fd30 2024-02-17T12:09:54.909 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.906+0000 7f4720ff9640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f470c0a2480 0x7f470c0a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.906+0000 7f4720ff9640 1 -- 172.21.15.94:0/856002384 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4714014070 con 0x7f472410fd30 2024-02-17T12:09:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f472cda0740 1 Processor -- start 2024-02-17T12:09:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f472cda0740 1 -- start start 2024-02-17T12:09:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f47237fe640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f470c0a2480 0x7f470c0a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f472cda0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a7f9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f472cda0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a7fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f472cda0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 0x55fd8a8056d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f472cda0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8a7689f0 con 0x55fd8a7faad0 2024-02-17T12:09:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f472cda0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8a765f20 con 0x55fd8a8032e0 2024-02-17T12:09:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f472cda0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8a761700 con 0x55fd8a7f9d60 2024-02-17T12:09:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f4722ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a7fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f4722ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a7fa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47136/0 (socket says 172.21.15.94:47136) 2024-02-17T12:09:54.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f4722ffd640 1 -- 172.21.15.94:0/2408205037 learned_addr learned my addr 172.21.15.94:0/2408205037 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f4723fff640 1 --2- 172.21.15.94:0/2408205037 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 0x55fd8a8056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.907+0000 7f47237fe640 1 --2- 172.21.15.94:0/2408205037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a7f9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.908+0000 7f4722ffd640 1 -- 172.21.15.94:0/2408205037 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 msgr2=0x55fd8a8056d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.908+0000 7f4722ffd640 1 --2- 172.21.15.94:0/2408205037 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 0x55fd8a8056d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.908+0000 7f4722ffd640 1 -- 172.21.15.94:0/2408205037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 msgr2=0x55fd8a7f9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.908+0000 7f4722ffd640 1 --2- 172.21.15.94:0/2408205037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a7f9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.908+0000 7f4722ffd640 1 -- 172.21.15.94:0/2408205037 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd8a6f77b0 con 0x55fd8a7f9d60 2024-02-17T12:09:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.908+0000 7f47237fe640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f470c0a2480 0x7f470c0a4940 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4718006fd0 tx=0x7f4718009d90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:54.911 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.908+0000 7f4722ffd640 1 --2- 172.21.15.94:0/2408205037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a7fa1e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4714097650 tx=0x7f471409a880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e3d610ca68a726 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.908+0000 7f4709ffb640 1 -- 172.21.15.94:0/2408205037 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140cd030 con 0x55fd8a7f9d60 2024-02-17T12:09:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.909+0000 7f4709ffb640 1 -- 172.21.15.94:0/2408205037 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f471409ac80 con 0x55fd8a7f9d60 2024-02-17T12:09:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.910+0000 7f472cda0740 1 -- 172.21.15.94:0/2408205037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 msgr2=0x55fd8a7fa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.910+0000 7f472cda0740 1 --2- 172.21.15.94:0/2408205037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a7fa1e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4714097650 tx=0x7f471409a880 comp rx=0 tx=0).stop 2024-02-17T12:09:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.910+0000 7f472cda0740 1 -- 172.21.15.94:0/2408205037 shutdown_connections 2024-02-17T12:09:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.910+0000 7f472cda0740 1 --2- 172.21.15.94:0/2408205037 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 0x55fd8a8056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.910+0000 7f472cda0740 1 --2- 172.21.15.94:0/2408205037 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a7fa1e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.910+0000 7f472cda0740 1 --2- 172.21.15.94:0/2408205037 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a7f9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.910+0000 7f472cda0740 1 -- 172.21.15.94:0/2408205037 >> 172.21.15.94:0/2408205037 conn(0x55fd8a7c1550 msgr2=0x55fd8a7f4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.910+0000 7f472cda0740 1 -- 172.21.15.94:0/2408205037 shutdown_connections 2024-02-17T12:09:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.911+0000 7f472cda0740 1 -- 172.21.15.94:0/2408205037 wait complete. 2024-02-17T12:09:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.911+0000 7f472cda0740 1 -- start start 2024-02-17T12:09:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.911+0000 7f472cda0740 1 Processor -- start 2024-02-17T12:09:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.912+0000 7f472cda0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a800310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.912+0000 7f472cda0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a800850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.912+0000 7f472cda0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 0x55fd8a800d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.912+0000 7f472cda0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8a706bf0 con 0x55fd8a7faad0 2024-02-17T12:09:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.912+0000 7f472cda0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8a435250 con 0x55fd8a8032e0 2024-02-17T12:09:54.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.912+0000 7f472cda0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8a6e7200 con 0x55fd8a7f9d60 2024-02-17T12:09:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.912+0000 7f472cda0740 10 client.0 _ll_register_callbacks cb 0x55fd8a755d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.912+0000 7f4723fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 0x55fd8a800d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.912+0000 7f4722ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a800850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.912+0000 7f4723fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 0x55fd8a800d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44282/0 (socket says 172.21.15.94:44282) 2024-02-17T12:09:54.915 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.912+0000 7f4723fff640 1 -- 172.21.15.94:0/1298904752 learned_addr learned my addr 172.21.15.94:0/1298904752 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.913+0000 7f4722ffd640 1 -- 172.21.15.94:0/1298904752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 msgr2=0x55fd8a800310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.913+0000 7f4722ffd640 1 --2- 172.21.15.94:0/1298904752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a800310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.913+0000 7f4722ffd640 1 -- 172.21.15.94:0/1298904752 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 msgr2=0x55fd8a800d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.913+0000 7f4722ffd640 1 --2- 172.21.15.94:0/1298904752 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 0x55fd8a800d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.913+0000 7f4722ffd640 1 -- 172.21.15.94:0/1298904752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4724115a30 con 0x55fd8a7faad0 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.913+0000 7f4722ffd640 1 --2- 172.21.15.94:0/1298904752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a800850 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4714097650 tx=0x7f47140968b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01549f44b034694 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.913+0000 7f4702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.913+0000 7f4702ffd640 1 -- 172.21.15.94:0/1298904752 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140b9070 con 0x55fd8a7faad0 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.913+0000 7f472cda0740 1 -- 172.21.15.94:0/1298904752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd8a461600 con 0x55fd8a7faad0 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.913+0000 7f4702ffd640 1 -- 172.21.15.94:0/1298904752 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f471409edd0 con 0x55fd8a7faad0 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.913+0000 7f4702ffd640 1 -- 172.21.15.94:0/1298904752 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47140cd030 con 0x55fd8a7faad0 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.913+0000 7f472cda0740 10 client.18537 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.913+0000 7f472cda0740 1 -- 172.21.15.94:0/1298904752 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fd8a75fd80 con 0x55fd8a7faad0 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.914+0000 7f4702ffd640 1 -- 172.21.15.94:0/1298904752 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55fd8a461600 con 0x55fd8a7faad0 2024-02-17T12:09:54.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.914+0000 7f4702ffd640 1 -- 172.21.15.94:0/1298904752 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f47140a9070 con 0x55fd8a7faad0 2024-02-17T12:09:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.914+0000 7f472cda0740 10 client.18537 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.914+0000 7f472cda0740 10 client.18537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.914+0000 7f472cda0740 20 client.18537 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.914+0000 7f472cda0740 1 --2- 172.21.15.94:0/1298904752 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd8a813200 0x55fd8a8155f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.914+0000 7f472cda0740 4 client.18537 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.914+0000 7f472cda0740 1 -- 172.21.15.94:0/1298904752 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd8a4b21b0 con 0x55fd8a813200 2024-02-17T12:09:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.915+0000 7f47237fe640 1 --2- 172.21.15.94:0/1298904752 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd8a813200 0x55fd8a8155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.918 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.915+0000 7f4702ffd640 10 client.18537 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.916+0000 7f47237fe640 1 --2- 172.21.15.94:0/1298904752 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd8a813200 0x55fd8a8155f0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a1aa7f9afabe5e2 server_cookie=fb798ddfe4c1bf2 in_seq=0 out_seq=0 2024-02-17T12:09:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.916+0000 7f4702ffd640 1 -- 172.21.15.94:0/1298904752 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fd8a4b21b0 con 0x55fd8a813200 2024-02-17T12:09:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.916+0000 7f4702ffd640 10 client.18537 handle_command_reply: tid=0 2024-02-17T12:09:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.917+0000 7f472cda0740 1 client.18537 shutdown 2024-02-17T12:09:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.917+0000 7f472cda0740 1 -- 172.21.15.94:0/1298904752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 msgr2=0x55fd8a800850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:54.919 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.917+0000 7f472cda0740 1 --2- 172.21.15.94:0/1298904752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a800850 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4714097650 tx=0x7f47140968b0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.919+0000 7f472cda0740 1 -- 172.21.15.94:0/1298904752 shutdown_connections 2024-02-17T12:09:54.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.919+0000 7f472cda0740 1 --2- 172.21.15.94:0/1298904752 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd8a813200 0x55fd8a8155f0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.919+0000 7f472cda0740 1 --2- 172.21.15.94:0/1298904752 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 0x55fd8a800d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.919+0000 7f472cda0740 1 --2- 172.21.15.94:0/1298904752 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a800850 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.919+0000 7f472cda0740 1 --2- 172.21.15.94:0/1298904752 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a800310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.919+0000 7f472cda0740 1 -- 172.21.15.94:0/1298904752 >> 172.21.15.94:0/1298904752 conn(0x55fd8a7c1550 msgr2=0x55fd8a7f67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.919+0000 7f4702ffd640 0 client.18537 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.920+0000 7f472cda0740 1 -- 172.21.15.94:0/1298904752 shutdown_connections 2024-02-17T12:09:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.920+0000 7f472cda0740 1 -- 172.21.15.94:0/1298904752 wait complete. 2024-02-17T12:09:54.922 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.920+0000 7f472cda0740 20 client.18537 trim_cache size 0 max 16384 2024-02-17T12:09:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.944+0000 7f472cda0740 1 Processor -- start 2024-02-17T12:09:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.944+0000 7f472cda0740 1 -- start start 2024-02-17T12:09:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.944+0000 7f472cda0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a802a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.944+0000 7f472cda0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a7faad0 0x55fd8a800bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.944+0000 7f472cda0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a8032e0 0x55fd8a7fd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.944+0000 7f472cda0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8a810200 con 0x55fd8a8032e0 2024-02-17T12:09:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.944+0000 7f472cda0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8a7ef050 con 0x55fd8a7faad0 2024-02-17T12:09:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.944+0000 7f472cda0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8a805c30 con 0x55fd8a7f9d60 2024-02-17T12:09:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f4722ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a7faad0 0x55fd8a800bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f4722ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a7faad0 0x55fd8a800bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44286/0 (socket says 172.21.15.94:44286) 2024-02-17T12:09:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f4722ffd640 1 -- 172.21.15.94:0/1889816563 learned_addr learned my addr 172.21.15.94:0/1889816563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f47237fe640 1 --2- 172.21.15.94:0/1889816563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a802a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f4723fff640 1 --2- 172.21.15.94:0/1889816563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a8032e0 0x55fd8a7fd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f4722ffd640 1 -- 172.21.15.94:0/1889816563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 msgr2=0x55fd8a802a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f4722ffd640 1 --2- 172.21.15.94:0/1889816563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a802a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f4722ffd640 1 -- 172.21.15.94:0/1889816563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a8032e0 msgr2=0x55fd8a7fd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f4722ffd640 1 --2- 172.21.15.94:0/1889816563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a8032e0 0x55fd8a7fd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f4722ffd640 1 -- 172.21.15.94:0/1889816563 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd8a6fc7b0 con 0x55fd8a7faad0 2024-02-17T12:09:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f4722ffd640 1 --2- 172.21.15.94:0/1889816563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a7faad0 0x55fd8a800bb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f471409feb0 tx=0x7f4714096920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=461e964f6bc37001 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f47097fa640 1 -- 172.21.15.94:0/1889816563 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140b9040 con 0x55fd8a7faad0 2024-02-17T12:09:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f47097fa640 1 -- 172.21.15.94:0/1889816563 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f471409fcf0 con 0x55fd8a7faad0 2024-02-17T12:09:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.945+0000 7f47097fa640 1 -- 172.21.15.94:0/1889816563 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471409edd0 con 0x55fd8a7faad0 2024-02-17T12:09:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.946+0000 7f472cda0740 1 -- 172.21.15.94:0/1889816563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a7faad0 msgr2=0x55fd8a800bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.946+0000 7f472cda0740 1 --2- 172.21.15.94:0/1889816563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a7faad0 0x55fd8a800bb0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f471409feb0 tx=0x7f4714096920 comp rx=0 tx=0).stop 2024-02-17T12:09:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.947+0000 7f472cda0740 1 -- 172.21.15.94:0/1889816563 shutdown_connections 2024-02-17T12:09:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.947+0000 7f472cda0740 1 --2- 172.21.15.94:0/1889816563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a8032e0 0x55fd8a7fd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.947+0000 7f472cda0740 1 --2- 172.21.15.94:0/1889816563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a7faad0 0x55fd8a800bb0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.947+0000 7f472cda0740 1 --2- 172.21.15.94:0/1889816563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a802a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.947+0000 7f472cda0740 1 -- 172.21.15.94:0/1889816563 >> 172.21.15.94:0/1889816563 conn(0x55fd8a7c1550 msgr2=0x55fd8a7fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.948+0000 7f472cda0740 1 -- 172.21.15.94:0/1889816563 shutdown_connections 2024-02-17T12:09:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.948+0000 7f472cda0740 1 -- 172.21.15.94:0/1889816563 wait complete. 2024-02-17T12:09:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.948+0000 7f472cda0740 1 -- start start 2024-02-17T12:09:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.949+0000 7f472cda0740 1 Processor -- start 2024-02-17T12:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.949+0000 7f472cda0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a80a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.949+0000 7f472cda0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a8087c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.949+0000 7f472cda0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 0x55fd8a808d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.949+0000 7f472cda0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8a809310 con 0x55fd8a7faad0 2024-02-17T12:09:54.951 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.949+0000 7f4722ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a8087c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.949+0000 7f4722ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a8087c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41082/0 (socket says 172.21.15.94:41082) 2024-02-17T12:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.949+0000 7f47237fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a80a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.949+0000 7f4722ffd640 1 -- 172.21.15.94:0/2999284762 learned_addr learned my addr 172.21.15.94:0/2999284762 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.949+0000 7f4723fff640 1 --2- 172.21.15.94:0/2999284762 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 0x55fd8a808d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.949+0000 7f472cda0740 1 -- 172.21.15.94:0/2999284762 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8a809480 con 0x55fd8a8032e0 2024-02-17T12:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.949+0000 7f472cda0740 1 -- 172.21.15.94:0/2999284762 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd8a809640 con 0x55fd8a7f9d60 2024-02-17T12:09:54.952 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.949+0000 7f472cda0740 10 client.0 _ll_register_callbacks cb 0x7f4714096800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.950+0000 7f47237fe640 1 -- 172.21.15.94:0/2999284762 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 msgr2=0x55fd8a808d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.950+0000 7f47237fe640 1 --2- 172.21.15.94:0/2999284762 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 0x55fd8a808d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.950+0000 7f47237fe640 1 -- 172.21.15.94:0/2999284762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 msgr2=0x55fd8a8087c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.950+0000 7f47237fe640 1 --2- 172.21.15.94:0/2999284762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a8087c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.950+0000 7f47237fe640 1 -- 172.21.15.94:0/2999284762 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd8a75fd80 con 0x55fd8a7f9d60 2024-02-17T12:09:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.950+0000 7f47237fe640 1 --2- 172.21.15.94:0/2999284762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a80a420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f471800d530 tx=0x7f47180046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f4ab1ed84a5806 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.950+0000 7f4702ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:09:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.950+0000 7f4702ffd640 1 -- 172.21.15.94:0/2999284762 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4718029070 con 0x55fd8a7f9d60 2024-02-17T12:09:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.951+0000 7f4702ffd640 1 -- 172.21.15.94:0/2999284762 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f471801c040 con 0x55fd8a7f9d60 2024-02-17T12:09:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.951+0000 7f472cda0740 1 -- 172.21.15.94:0/2999284762 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f471409ac80 con 0x55fd8a7f9d60 2024-02-17T12:09:54.954 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.951+0000 7f4702ffd640 1 -- 172.21.15.94:0/2999284762 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4718024880 con 0x55fd8a7f9d60 2024-02-17T12:09:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.951+0000 7f472cda0740 10 client.28000 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.951+0000 7f472cda0740 1 -- 172.21.15.94:0/2999284762 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fd8a722540 con 0x55fd8a7f9d60 2024-02-17T12:09:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.951+0000 7f4702ffd640 1 -- 172.21.15.94:0/2999284762 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f471409ac80 con 0x55fd8a7f9d60 2024-02-17T12:09:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.951+0000 7f4702ffd640 1 -- 172.21.15.94:0/2999284762 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4718035070 con 0x55fd8a7f9d60 2024-02-17T12:09:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.951+0000 7f472cda0740 10 client.28000 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.951+0000 7f472cda0740 10 client.28000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.951+0000 7f472cda0740 20 client.28000 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.951+0000 7f472cda0740 1 --2- 172.21.15.94:0/2999284762 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd8a86adb0 0x55fd8a86d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.951+0000 7f472cda0740 4 client.28000 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:09:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.951+0000 7f472cda0740 1 -- 172.21.15.94:0/2999284762 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fd8a49e550 con 0x55fd8a86adb0 2024-02-17T12:09:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.952+0000 7f4722ffd640 1 --2- 172.21.15.94:0/2999284762 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd8a86adb0 0x55fd8a86d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.952+0000 7f4702ffd640 10 client.28000 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:54.955 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.952+0000 7f4722ffd640 1 --2- 172.21.15.94:0/2999284762 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd8a86adb0 0x55fd8a86d1d0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea48d738ff3423ee server_cookie=7043d2f4288f91ee in_seq=0 out_seq=0 2024-02-17T12:09:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.978+0000 7f4702ffd640 1 -- 172.21.15.94:0/2999284762 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fd8a49e550 con 0x55fd8a86adb0 2024-02-17T12:09:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.978+0000 7f4702ffd640 10 client.28000 handle_command_reply: tid=0 2024-02-17T12:09:54.980 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.978+0000 7f472cda0740 1 client.28000 shutdown 2024-02-17T12:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.979+0000 7f472cda0740 1 -- 172.21.15.94:0/2999284762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 msgr2=0x55fd8a80a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.979+0000 7f472cda0740 1 --2- 172.21.15.94:0/2999284762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a80a420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f471800d530 tx=0x7f47180046e0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.979+0000 7f472cda0740 1 -- 172.21.15.94:0/2999284762 shutdown_connections 2024-02-17T12:09:54.981 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.979+0000 7f472cda0740 1 --2- 172.21.15.94:0/2999284762 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55fd8a86adb0 0x55fd8a86d1d0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.980+0000 7f472cda0740 1 --2- 172.21.15.94:0/2999284762 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fd8a8032e0 0x55fd8a808d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.980+0000 7f472cda0740 1 --2- 172.21.15.94:0/2999284762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fd8a7faad0 0x55fd8a8087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.980+0000 7f472cda0740 1 --2- 172.21.15.94:0/2999284762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fd8a7f9d60 0x55fd8a80a420 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.980+0000 7f4702ffd640 0 client.28000 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.980+0000 7f472cda0740 1 -- 172.21.15.94:0/2999284762 >> 172.21.15.94:0/2999284762 conn(0x55fd8a7c1550 msgr2=0x55fd8a7f5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.980+0000 7f472cda0740 1 -- 172.21.15.94:0/2999284762 shutdown_connections 2024-02-17T12:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.980+0000 7f472cda0740 1 -- 172.21.15.94:0/2999284762 wait complete. 2024-02-17T12:09:54.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.980+0000 7f472cda0740 20 client.28000 trim_cache size 0 max 16384 2024-02-17T12:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.981+0000 7f472a1c9640 1 -- 172.21.15.94:0/856002384 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f470c0a2480 msgr2=0x7f470c0a4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.981+0000 7f472a1c9640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f470c0a2480 0x7f470c0a4940 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4718006fd0 tx=0x7f4718009d90 comp rx=0 tx=0).stop 2024-02-17T12:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.981+0000 7f472a1c9640 1 -- 172.21.15.94:0/856002384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472410fd30 msgr2=0x7f4724118270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:54.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.981+0000 7f472a1c9640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472410fd30 0x7f4724118270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f471400e9f0 tx=0x7f471400eeb0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.982+0000 7f472a1c9640 1 -- 172.21.15.94:0/856002384 shutdown_connections 2024-02-17T12:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.982+0000 7f472a1c9640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f470c0a2480 0x7f470c0a4940 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.982+0000 7f472a1c9640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f472411a960 0x7f4724110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.982+0000 7f472a1c9640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f472410fd30 0x7f4724118270 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.982+0000 7f472a1c9640 1 --2- 172.21.15.94:0/856002384 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f472410f8c0 0x7f4724117d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:54.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.983+0000 7f472a1c9640 1 -- 172.21.15.94:0/856002384 >> 172.21.15.94:0/856002384 conn(0x7f472406c220 msgr2=0x7f472411b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.983+0000 7f472a1c9640 1 -- 172.21.15.94:0/856002384 shutdown_connections 2024-02-17T12:09:54.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:09:54.983+0000 7f472a1c9640 1 -- 172.21.15.94:0/856002384 wait complete. 2024-02-17T12:09:55.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 695.0 s, 1954.6 tps, lat 51.372 ms stddev 64.119 2024-02-17T12:09:55.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:55 smithi189 ceph-mon[36588]: pgmap v367: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 1.64k op/s 2024-02-17T12:09:55.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:55 smithi125 ceph-mon[36819]: pgmap v367: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 1.64k op/s 2024-02-17T12:09:55.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:55 smithi094 ceph-mon[32499]: pgmap v367: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 32 MiB/s wr, 1.64k op/s 2024-02-17T12:09:57.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:57 smithi189 ceph-mon[36588]: pgmap v368: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 499 op/s 2024-02-17T12:09:57.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:57 smithi125 ceph-mon[36819]: pgmap v368: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 499 op/s 2024-02-17T12:09:57.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:57 smithi094 ceph-mon[32499]: pgmap v368: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.0 MiB/s wr, 499 op/s 2024-02-17T12:09:58.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:09:58.260 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:09:58.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.417+0000 7f35bde39640 1 -- 172.21.15.94:0/497558993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b8100c20 msgr2=0x7f35b8104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:58.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.417+0000 7f35bde39640 1 --2- 172.21.15.94:0/497558993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b8100c20 0x7f35b8104c60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009930 tx=0x7f35ac02bc30 comp rx=0 tx=0).stop 2024-02-17T12:09:58.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.417+0000 7f35bde39640 1 -- 172.21.15.94:0/497558993 shutdown_connections 2024-02-17T12:09:58.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.417+0000 7f35bde39640 1 --2- 172.21.15.94:0/497558993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35b8105340 0x7f35b810cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.417+0000 7f35bde39640 1 --2- 172.21.15.94:0/497558993 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b8100c20 0x7f35b8104c60 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.417+0000 7f35bde39640 1 --2- 172.21.15.94:0/497558993 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b8100270 0x7f35b8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.417+0000 7f35bde39640 1 -- 172.21.15.94:0/497558993 >> 172.21.15.94:0/497558993 conn(0x7f35b8077230 msgr2=0x7f35b8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:58.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.417+0000 7f35bde39640 1 -- 172.21.15.94:0/497558993 shutdown_connections 2024-02-17T12:09:58.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.417+0000 7f35bde39640 1 -- 172.21.15.94:0/497558993 wait complete. 2024-02-17T12:09:58.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.417+0000 7f35bde39640 1 Processor -- start 2024-02-17T12:09:58.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.418+0000 7f35bde39640 1 -- start start 2024-02-17T12:09:58.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.418+0000 7f35bde39640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b8100270 0x7f35b8196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.418+0000 7f35bde39640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b8105340 0x7f35b8196890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.418+0000 7f35bde39640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35b8196e70 0x7f35b819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.418+0000 7f35bde39640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b819bb00 con 0x7f35b8100270 2024-02-17T12:09:58.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.418+0000 7f35bde39640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b819bc70 con 0x7f35b8105340 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.418+0000 7f35bde39640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b819be30 con 0x7f35b8196e70 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.418+0000 7f35b7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35b8196e70 0x7f35b819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.418+0000 7f35b7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35b8196e70 0x7f35b819b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47190/0 (socket says 172.21.15.94:47190) 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.418+0000 7f35b7fff640 1 -- 172.21.15.94:0/3566429311 learned_addr learned my addr 172.21.15.94:0/3566429311 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.418+0000 7f35b77fe640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b8100270 0x7f35b8196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.419+0000 7f35b7fff640 1 -- 172.21.15.94:0/3566429311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b8105340 msgr2=0x7f35b8196890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.419+0000 7f35b7fff640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b8105340 0x7f35b8196890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.419+0000 7f35b7fff640 1 -- 172.21.15.94:0/3566429311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b8100270 msgr2=0x7f35b8196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.419+0000 7f35b7fff640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b8100270 0x7f35b8196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.419+0000 7f35b7fff640 1 -- 172.21.15.94:0/3566429311 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac009560 con 0x7f35b8196e70 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.419+0000 7f35b77fe640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b8100270 0x7f35b8196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.419+0000 7f35b7fff640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35b8196e70 0x7f35b819b3a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f35b000eb10 tx=0x7f35b000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.419+0000 7f35b4ff9640 1 -- 172.21.15.94:0/3566429311 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0004b20 con 0x7f35b8196e70 2024-02-17T12:09:58.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.419+0000 7f35b4ff9640 1 -- 172.21.15.94:0/3566429311 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35b0004c80 con 0x7f35b8196e70 2024-02-17T12:09:58.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.419+0000 7f35b4ff9640 1 -- 172.21.15.94:0/3566429311 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b0010720 con 0x7f35b8196e70 2024-02-17T12:09:58.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.419+0000 7f35bde39640 1 -- 172.21.15.94:0/3566429311 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b819c110 con 0x7f35b8196e70 2024-02-17T12:09:58.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.420+0000 7f35bde39640 1 -- 172.21.15.94:0/3566429311 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b819c5d0 con 0x7f35b8196e70 2024-02-17T12:09:58.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.423+0000 7f35c0a08740 1 Processor -- start 2024-02-17T12:09:58.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.423+0000 7f35b4ff9640 1 -- 172.21.15.94:0/3566429311 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f35b0005020 con 0x7f35b8196e70 2024-02-17T12:09:58.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.423+0000 7f35c0a08740 1 -- start start 2024-02-17T12:09:58.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.423+0000 7f35c0a08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef412b0 0x561b1ef41690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.423+0000 7f35c0a08740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef41bd0 0x561b1ef4a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.423+0000 7f35c0a08740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef4cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.423+0000 7f35c0a08740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1eeb0280 con 0x561b1ef4a7b0 2024-02-17T12:09:58.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.423+0000 7f35c0a08740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1eead7b0 con 0x561b1ef41bd0 2024-02-17T12:09:58.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.423+0000 7f35c0a08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1eea8f90 con 0x561b1ef412b0 2024-02-17T12:09:58.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.423+0000 7f35b6ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef412b0 0x561b1ef41690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.423+0000 7f35b6ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef412b0 0x561b1ef41690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47204/0 (socket says 172.21.15.94:47204) 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.423+0000 7f35b6ffd640 1 -- 172.21.15.94:0/3161064901 learned_addr learned my addr 172.21.15.94:0/3161064901 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.423+0000 7f35b77fe640 1 --2- 172.21.15.94:0/3161064901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef41bd0 0x561b1ef4a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.424+0000 7f35b6ffd640 1 --2- 172.21.15.94:0/3161064901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef4cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.424+0000 7f35b77fe640 1 -- 172.21.15.94:0/3161064901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef412b0 msgr2=0x561b1ef41690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.424+0000 7f35b77fe640 1 --2- 172.21.15.94:0/3161064901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef412b0 0x561b1ef41690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.424+0000 7f35b77fe640 1 -- 172.21.15.94:0/3161064901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 msgr2=0x561b1ef4cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.424+0000 7f35b77fe640 1 --2- 172.21.15.94:0/3161064901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef4cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.424+0000 7f35b77fe640 1 -- 172.21.15.94:0/3161064901 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b1ee67570 con 0x561b1ef41bd0 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.424+0000 7f35b4ff9640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35a00a2400 0x7f35a00a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.424+0000 7f35b4ff9640 1 -- 172.21.15.94:0/3566429311 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f35b0014070 con 0x7f35b8196e70 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.424+0000 7f35b6ffd640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35a00a2400 0x7f35a00a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.424+0000 7f35b77fe640 1 --2- 172.21.15.94:0/3161064901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef41bd0 0x561b1ef4a270 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009c30 tx=0x7f35a80117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e120e1eae72b4ee server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.424+0000 7f359dffb640 1 -- 172.21.15.94:0/3161064901 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8012450 con 0x561b1ef41bd0 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.424+0000 7f359dffb640 1 -- 172.21.15.94:0/3161064901 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35a8012a90 con 0x561b1ef41bd0 2024-02-17T12:09:58.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.424+0000 7f359dffb640 1 -- 172.21.15.94:0/3161064901 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a800f980 con 0x561b1ef41bd0 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.425+0000 7f35c0a08740 1 -- 172.21.15.94:0/3161064901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef41bd0 msgr2=0x561b1ef4a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.425+0000 7f35b6ffd640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35a00a2400 0x7f35a00a48c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x561b1ef3cb70 tx=0x7f35ac000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.425+0000 7f35c0a08740 1 --2- 172.21.15.94:0/3161064901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef41bd0 0x561b1ef4a270 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009c30 tx=0x7f35a80117b0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.425+0000 7f35c0a08740 1 -- 172.21.15.94:0/3161064901 shutdown_connections 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.425+0000 7f35c0a08740 1 --2- 172.21.15.94:0/3161064901 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef4cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.425+0000 7f35c0a08740 1 --2- 172.21.15.94:0/3161064901 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef41bd0 0x561b1ef4a270 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.425+0000 7f35c0a08740 1 --2- 172.21.15.94:0/3161064901 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef412b0 0x561b1ef41690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.425+0000 7f35c0a08740 1 -- 172.21.15.94:0/3161064901 >> 172.21.15.94:0/3161064901 conn(0x561b1ef3b990 msgr2=0x561b1ef3bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.425+0000 7f35c0a08740 1 -- 172.21.15.94:0/3161064901 shutdown_connections 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.425+0000 7f35c0a08740 1 -- 172.21.15.94:0/3161064901 wait complete. 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.425+0000 7f35c0a08740 1 -- start start 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35c0a08740 1 Processor -- start 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35c0a08740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef412b0 0x561b1ef4a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35c0a08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef41bd0 0x561b1ef47510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.431 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35c0a08740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef47b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35c0a08740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1eb7c430 con 0x561b1ef4a7b0 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35c0a08740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1eb38e00 con 0x561b1ef412b0 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35c0a08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1ee2dfd0 con 0x561b1ef41bd0 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35c0a08740 10 client.0 _ll_register_callbacks cb 0x561b1ee83fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35b77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef412b0 0x561b1ef4a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35b77fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef412b0 0x561b1ef4a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44320/0 (socket says 172.21.15.94:44320) 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35b77fe640 1 -- 172.21.15.94:0/1048627369 learned_addr learned my addr 172.21.15.94:0/1048627369 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35b6ffd640 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef41bd0 0x561b1ef47510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35b7fff640 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef47b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35b77fe640 1 -- 172.21.15.94:0/1048627369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef41bd0 msgr2=0x561b1ef47510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35b77fe640 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef41bd0 0x561b1ef47510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35b77fe640 1 -- 172.21.15.94:0/1048627369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 msgr2=0x561b1ef47b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35b77fe640 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef47b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35b77fe640 1 -- 172.21.15.94:0/1048627369 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a8009860 con 0x561b1ef412b0 2024-02-17T12:09:58.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.426+0000 7f35b7fff640 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef47b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:58.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.427+0000 7f35b77fe640 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef412b0 0x561b1ef4a180 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f35a800f520 tx=0x7f35a80124d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=798f0757542f29 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:58.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.427+0000 7f3596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:09:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.427+0000 7f3596ffd640 1 -- 172.21.15.94:0/1048627369 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a80128d0 con 0x561b1ef412b0 2024-02-17T12:09:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.427+0000 7f3596ffd640 1 -- 172.21.15.94:0/1048627369 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f35a8005040 con 0x561b1ef412b0 2024-02-17T12:09:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.427+0000 7f35c0a08740 1 -- 172.21.15.94:0/1048627369 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b1eba87e0 con 0x561b1ef412b0 2024-02-17T12:09:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.427+0000 7f3596ffd640 1 -- 172.21.15.94:0/1048627369 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35a8005200 con 0x561b1ef412b0 2024-02-17T12:09:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.427+0000 7f35c0a08740 10 client.28010 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.427+0000 7f35c0a08740 1 -- 172.21.15.94:0/1048627369 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561b1eea7610 con 0x561b1ef412b0 2024-02-17T12:09:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.428+0000 7f3596ffd640 1 -- 172.21.15.94:0/1048627369 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35a8032030 con 0x561b1ef412b0 2024-02-17T12:09:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.428+0000 7f3596ffd640 1 -- 172.21.15.94:0/1048627369 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f35a802c060 con 0x561b1ef412b0 2024-02-17T12:09:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.428+0000 7f35c0a08740 10 client.28010 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.428+0000 7f35c0a08740 10 client.28010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.428+0000 7f35c0a08740 20 client.28010 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.428+0000 7f35c0a08740 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561b1ef6b9e0 0x561b1ef6ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.436 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.428+0000 7f35c0a08740 4 client.28010 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.428+0000 7f35c0a08740 1 -- 172.21.15.94:0/1048627369 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b1ebe5720 con 0x561b1ef6b9e0 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.428+0000 7f35b6ffd640 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561b1ef6b9e0 0x561b1ef6ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.429+0000 7f3596ffd640 10 client.28010 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.429+0000 7f35b6ffd640 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561b1ef6b9e0 0x561b1ef6ddd0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a29e0f9be3bb53a server_cookie=8e44e7b86280c24f in_seq=0 out_seq=0 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f3596ffd640 1 -- 172.21.15.94:0/1048627369 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561b1ebe5720 con 0x561b1ef6b9e0 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f3596ffd640 10 client.28010 handle_command_reply: tid=0 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f35c0a08740 1 client.28010 shutdown 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f35c0a08740 1 -- 172.21.15.94:0/1048627369 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef412b0 msgr2=0x561b1ef4a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f35c0a08740 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef412b0 0x561b1ef4a180 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f35a800f520 tx=0x7f35a80124d0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f35c0a08740 1 -- 172.21.15.94:0/1048627369 shutdown_connections 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f35c0a08740 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561b1ef6b9e0 0x561b1ef6ddd0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f35c0a08740 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef47b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f35c0a08740 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef41bd0 0x561b1ef47510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.437 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f35c0a08740 1 --2- 172.21.15.94:0/1048627369 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef412b0 0x561b1ef4a180 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f35c0a08740 1 -- 172.21.15.94:0/1048627369 >> 172.21.15.94:0/1048627369 conn(0x561b1ef3b990 msgr2=0x561b1ef3d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f3596ffd640 0 client.28010 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f35c0a08740 1 -- 172.21.15.94:0/1048627369 shutdown_connections 2024-02-17T12:09:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f35c0a08740 1 -- 172.21.15.94:0/1048627369 wait complete. 2024-02-17T12:09:58.438 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.430+0000 7f35c0a08740 20 client.28010 trim_cache size 0 max 16384 2024-02-17T12:09:58.454 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.447+0000 7f35c0a08740 1 Processor -- start 2024-02-17T12:09:58.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.456+0000 7f35c0a08740 1 -- start start 2024-02-17T12:09:58.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.456+0000 7f35c0a08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef412b0 0x561b1ef72770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.456+0000 7f35c0a08740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef41bd0 0x561b1ef49c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.456+0000 7f35c0a08740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef4a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.456+0000 7f35c0a08740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1ef35e20 con 0x561b1ef4a7b0 2024-02-17T12:09:58.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.456+0000 7f35c0a08740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1ef3b7f0 con 0x561b1ef41bd0 2024-02-17T12:09:58.459 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.456+0000 7f35c0a08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1ef675d0 con 0x561b1ef412b0 2024-02-17T12:09:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.460+0000 7f35b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef4a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.460+0000 7f35b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef4a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41116/0 (socket says 172.21.15.94:41116) 2024-02-17T12:09:58.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.460+0000 7f35b7fff640 1 -- 172.21.15.94:0/2609895079 learned_addr learned my addr 172.21.15.94:0/2609895079 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:58.465 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.462+0000 7f35b6ffd640 1 --2- 172.21.15.94:0/2609895079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef41bd0 0x561b1ef49c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.462+0000 7f35b77fe640 1 --2- 172.21.15.94:0/2609895079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef412b0 0x561b1ef72770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.464+0000 7f35b7fff640 1 -- 172.21.15.94:0/2609895079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef412b0 msgr2=0x561b1ef72770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:58.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.464+0000 7f35b7fff640 1 --2- 172.21.15.94:0/2609895079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef412b0 0x561b1ef72770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.464+0000 7f35b7fff640 1 -- 172.21.15.94:0/2609895079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef41bd0 msgr2=0x561b1ef49c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:58.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.464+0000 7f35b7fff640 1 --2- 172.21.15.94:0/2609895079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef41bd0 0x561b1ef49c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.469 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.464+0000 7f35b7fff640 1 -- 172.21.15.94:0/2609895079 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b1ee43ac0 con 0x561b1ef4a7b0 2024-02-17T12:09:58.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.468+0000 7f35b6ffd640 1 --2- 172.21.15.94:0/2609895079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef41bd0 0x561b1ef49c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:09:58.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.469+0000 7f35b7fff640 1 --2- 172.21.15.94:0/2609895079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef4a180 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f35b00979d0 tx=0x7f35b009a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d08c92683798a1f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:58.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.475+0000 7f359d7fa640 1 -- 172.21.15.94:0/2609895079 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b009f150 con 0x561b1ef4a7b0 2024-02-17T12:09:58.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.475+0000 7f359d7fa640 1 -- 172.21.15.94:0/2609895079 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35b00a37a0 con 0x561b1ef4a7b0 2024-02-17T12:09:58.478 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.475+0000 7f359d7fa640 1 -- 172.21.15.94:0/2609895079 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b009b4c0 con 0x561b1ef4a7b0 2024-02-17T12:09:58.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.478+0000 7f35c0a08740 1 -- 172.21.15.94:0/2609895079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 msgr2=0x561b1ef4a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:58.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.478+0000 7f35c0a08740 1 --2- 172.21.15.94:0/2609895079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef4a180 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f35b00979d0 tx=0x7f35b009a960 comp rx=0 tx=0).stop 2024-02-17T12:09:58.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.483+0000 7f35c0a08740 1 -- 172.21.15.94:0/2609895079 shutdown_connections 2024-02-17T12:09:58.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.483+0000 7f35c0a08740 1 --2- 172.21.15.94:0/2609895079 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef4a7b0 0x561b1ef4a180 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.483+0000 7f35c0a08740 1 --2- 172.21.15.94:0/2609895079 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef41bd0 0x561b1ef49c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.483+0000 7f35c0a08740 1 --2- 172.21.15.94:0/2609895079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef412b0 0x561b1ef72770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.483+0000 7f35c0a08740 1 -- 172.21.15.94:0/2609895079 >> 172.21.15.94:0/2609895079 conn(0x561b1ef3b990 msgr2=0x561b1ef428b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:58.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.484+0000 7f35c0a08740 1 -- 172.21.15.94:0/2609895079 shutdown_connections 2024-02-17T12:09:58.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.487+0000 7f35c0a08740 1 -- 172.21.15.94:0/2609895079 wait complete. 2024-02-17T12:09:58.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.488+0000 7f35c0a08740 1 -- start start 2024-02-17T12:09:58.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.488+0000 7f35c0a08740 1 Processor -- start 2024-02-17T12:09:58.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.488+0000 7f35c0a08740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef412b0 0x561b1ef5eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.489+0000 7f35c0a08740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef41bd0 0x561b1ef5f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.489+0000 7f35c0a08740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef4a7b0 0x561b1efb9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.489+0000 7f35c0a08740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1efba570 con 0x561b1ef412b0 2024-02-17T12:09:58.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.489+0000 7f35c0a08740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1efba6e0 con 0x561b1ef4a7b0 2024-02-17T12:09:58.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.489+0000 7f35c0a08740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b1efba8a0 con 0x561b1ef41bd0 2024-02-17T12:09:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.489+0000 7f35b6ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef41bd0 0x561b1ef5f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.489+0000 7f35b6ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef41bd0 0x561b1ef5f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47214/0 (socket says 172.21.15.94:47214) 2024-02-17T12:09:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.489+0000 7f35b6ffd640 1 -- 172.21.15.94:0/1818430688 learned_addr learned my addr 172.21.15.94:0/1818430688 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.489+0000 7f35c0a08740 10 client.0 _ll_register_callbacks cb 0x7f35a80095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.489+0000 7f35b7fff640 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef4a7b0 0x561b1efb9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.489+0000 7f35b77fe640 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef412b0 0x561b1ef5eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.490+0000 7f35b6ffd640 1 -- 172.21.15.94:0/1818430688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef4a7b0 msgr2=0x561b1efb9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.490+0000 7f35b6ffd640 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef4a7b0 0x561b1efb9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.490+0000 7f35b6ffd640 1 -- 172.21.15.94:0/1818430688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef412b0 msgr2=0x561b1ef5eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.490+0000 7f35b6ffd640 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef412b0 0x561b1ef5eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.490+0000 7f35b6ffd640 1 -- 172.21.15.94:0/1818430688 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b819c5d0 con 0x561b1ef41bd0 2024-02-17T12:09:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.490+0000 7f35b77fe640 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef412b0 0x561b1ef5eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:09:58.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.490+0000 7f35b7fff640 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef4a7b0 0x561b1efb9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:09:58.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.490+0000 7f35b6ffd640 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef41bd0 0x561b1ef5f1e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f35ac005fd0 tx=0x7f35ac009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60ebc8acabd9f306 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:58.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.491+0000 7f3596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:09:58.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.491+0000 7f3596ffd640 1 -- 172.21.15.94:0/1818430688 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac056070 con 0x561b1ef41bd0 2024-02-17T12:09:58.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.491+0000 7f35c0a08740 1 -- 172.21.15.94:0/1818430688 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f35a8012a90 con 0x561b1ef41bd0 2024-02-17T12:09:58.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.491+0000 7f3596ffd640 1 -- 172.21.15.94:0/1818430688 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f35ac057b00 con 0x561b1ef41bd0 2024-02-17T12:09:58.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.491+0000 7f3596ffd640 1 -- 172.21.15.94:0/1818430688 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35ac04a040 con 0x561b1ef41bd0 2024-02-17T12:09:58.494 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.491+0000 7f35c0a08740 10 client.28018 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:58.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.491+0000 7f35c0a08740 1 -- 172.21.15.94:0/1818430688 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561b1ee696a0 con 0x561b1ef41bd0 2024-02-17T12:09:58.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.492+0000 7f3596ffd640 1 -- 172.21.15.94:0/1818430688 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35a8012a90 con 0x561b1ef41bd0 2024-02-17T12:09:58.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.492+0000 7f3596ffd640 1 -- 172.21.15.94:0/1818430688 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f35ac004df0 con 0x561b1ef41bd0 2024-02-17T12:09:58.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.492+0000 7f35c0a08740 10 client.28018 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:58.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.492+0000 7f35c0a08740 10 client.28018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:58.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.492+0000 7f35c0a08740 20 client.28018 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:58.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.492+0000 7f35c0a08740 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561b1efc2fe0 0x561b1efc5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.492+0000 7f35c0a08740 4 client.28018 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:09:58.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.492+0000 7f35c0a08740 1 -- 172.21.15.94:0/1818430688 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561b1ec34890 con 0x561b1efc2fe0 2024-02-17T12:09:58.495 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.492+0000 7f35b77fe640 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561b1efc2fe0 0x561b1efc5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f35b77fe640 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561b1efc2fe0 0x561b1efc5400 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21dba10eace79fb8 server_cookie=76fe92c72ab44679 in_seq=0 out_seq=0 2024-02-17T12:09:58.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f3596ffd640 10 client.28018 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:58.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f3596ffd640 1 -- 172.21.15.94:0/1818430688 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561b1ec34890 con 0x561b1efc2fe0 2024-02-17T12:09:58.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f3596ffd640 10 client.28018 handle_command_reply: tid=0 2024-02-17T12:09:58.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f35c0a08740 1 client.28018 shutdown 2024-02-17T12:09:58.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f35c0a08740 1 -- 172.21.15.94:0/1818430688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef41bd0 msgr2=0x561b1ef5f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:58.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f35c0a08740 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef41bd0 0x561b1ef5f1e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f35ac005fd0 tx=0x7f35ac009260 comp rx=0 tx=0).stop 2024-02-17T12:09:58.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f35c0a08740 1 -- 172.21.15.94:0/1818430688 shutdown_connections 2024-02-17T12:09:58.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f35c0a08740 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561b1efc2fe0 0x561b1efc5400 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f35c0a08740 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561b1ef4a7b0 0x561b1efb9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f35c0a08740 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561b1ef41bd0 0x561b1ef5f1e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f35c0a08740 1 --2- 172.21.15.94:0/1818430688 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561b1ef412b0 0x561b1ef5eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f35c0a08740 1 -- 172.21.15.94:0/1818430688 >> 172.21.15.94:0/1818430688 conn(0x561b1ef3b990 msgr2=0x561b1ef72770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:58.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.493+0000 7f3596ffd640 0 client.28018 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:58.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.494+0000 7f35c0a08740 1 -- 172.21.15.94:0/1818430688 shutdown_connections 2024-02-17T12:09:58.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.494+0000 7f35c0a08740 1 -- 172.21.15.94:0/1818430688 wait complete. 2024-02-17T12:09:58.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.494+0000 7f35c0a08740 20 client.28018 trim_cache size 0 max 16384 2024-02-17T12:09:58.498 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:09:58.498 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:09:58.498 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:09:58.498 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:09:58.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.494+0000 7f35bde39640 1 -- 172.21.15.94:0/3566429311 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35a00a2400 msgr2=0x7f35a00a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:58.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.494+0000 7f35bde39640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35a00a2400 0x7f35a00a48c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x561b1ef3cb70 tx=0x7f35ac000bc0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.494+0000 7f35bde39640 1 -- 172.21.15.94:0/3566429311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35b8196e70 msgr2=0x7f35b819b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.494+0000 7f35bde39640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35b8196e70 0x7f35b819b3a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f35b000eb10 tx=0x7f35b000c2d0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.494+0000 7f35bde39640 1 -- 172.21.15.94:0/3566429311 shutdown_connections 2024-02-17T12:09:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.494+0000 7f35bde39640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35a00a2400 0x7f35a00a48c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.494+0000 7f35bde39640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35b8196e70 0x7f35b819b3a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.494+0000 7f35bde39640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35b8105340 0x7f35b8196890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.494+0000 7f35bde39640 1 --2- 172.21.15.94:0/3566429311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35b8100270 0x7f35b8196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.494+0000 7f35bde39640 1 -- 172.21.15.94:0/3566429311 >> 172.21.15.94:0/3566429311 conn(0x7f35b8077230 msgr2=0x7f35b80fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.495+0000 7f35bde39640 1 -- 172.21.15.94:0/3566429311 shutdown_connections 2024-02-17T12:09:58.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.495+0000 7f35bde39640 1 -- 172.21.15.94:0/3566429311 wait complete. 2024-02-17T12:09:58.509 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:09:58.509 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:09:58.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.625+0000 7fbc94d9a640 1 -- 172.21.15.94:0/1287203306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc90105060 msgr2=0x7fbc90105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:58.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.625+0000 7fbc94d9a640 1 --2- 172.21.15.94:0/1287203306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc90105060 0x7fbc90105440 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009930 tx=0x7fbc8002bc30 comp rx=0 tx=0).stop 2024-02-17T12:09:58.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.625+0000 7fbc94d9a640 1 -- 172.21.15.94:0/1287203306 shutdown_connections 2024-02-17T12:09:58.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.625+0000 7fbc94d9a640 1 --2- 172.21.15.94:0/1287203306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc90100d90 0x7fbc9010ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.625+0000 7fbc94d9a640 1 --2- 172.21.15.94:0/1287203306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc901003d0 0x7fbc90100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.625+0000 7fbc94d9a640 1 --2- 172.21.15.94:0/1287203306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc90105060 0x7fbc90105440 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.625+0000 7fbc94d9a640 1 -- 172.21.15.94:0/1287203306 >> 172.21.15.94:0/1287203306 conn(0x7fbc90075e80 msgr2=0x7fbc90076290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:58.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.625+0000 7fbc94d9a640 1 -- 172.21.15.94:0/1287203306 shutdown_connections 2024-02-17T12:09:58.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.625+0000 7fbc94d9a640 1 -- 172.21.15.94:0/1287203306 wait complete. 2024-02-17T12:09:58.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc94d9a640 1 Processor -- start 2024-02-17T12:09:58.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc94d9a640 1 -- start start 2024-02-17T12:09:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc94d9a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc901003d0 0x7fbc9019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc94d9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc90100d90 0x7fbc9019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc94d9a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc90105060 0x7fbc90199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc94d9a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc90199ee0 con 0x7fbc90100d90 2024-02-17T12:09:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc94d9a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9019a050 con 0x7fbc90105060 2024-02-17T12:09:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc94d9a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9019a210 con 0x7fbc901003d0 2024-02-17T12:09:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc85d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc90100d90 0x7fbc9019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc85d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc90100d90 0x7fbc9019fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41134/0 (socket says 172.21.15.94:41134) 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc85d9b640 1 -- 172.21.15.94:0/2683133293 learned_addr learned my addr 172.21.15.94:0/2683133293 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc8ed9d640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc90105060 0x7fbc90199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc8e59c640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc901003d0 0x7fbc9019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc85d9b640 1 -- 172.21.15.94:0/2683133293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc901003d0 msgr2=0x7fbc9019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc85d9b640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc901003d0 0x7fbc9019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc85d9b640 1 -- 172.21.15.94:0/2683133293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc90105060 msgr2=0x7fbc90199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc85d9b640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc90105060 0x7fbc90199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.626+0000 7fbc85d9b640 1 -- 172.21.15.94:0/2683133293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc80009560 con 0x7fbc90100d90 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.627+0000 7fbc85d9b640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc90100d90 0x7fbc9019fc10 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800e9f0 tx=0x7fbc7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.627+0000 7fbc87fff640 1 -- 172.21.15.94:0/2683133293 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78009700 con 0x7fbc90100d90 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.627+0000 7fbc94d9a640 1 -- 172.21.15.94:0/2683133293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9019a4f0 con 0x7fbc90100d90 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.627+0000 7fbc87fff640 1 -- 172.21.15.94:0/2683133293 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbc7800cd80 con 0x7fbc90100d90 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.627+0000 7fbc87fff640 1 -- 172.21.15.94:0/2683133293 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78018920 con 0x7fbc90100d90 2024-02-17T12:09:58.631 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.627+0000 7fbc94d9a640 1 -- 172.21.15.94:0/2683133293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc901a6700 con 0x7fbc90100d90 2024-02-17T12:09:58.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.628+0000 7fbc94d9a640 1 -- 172.21.15.94:0/2683133293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc901017a0 con 0x7fbc90100d90 2024-02-17T12:09:58.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.629+0000 7fbc87fff640 1 -- 172.21.15.94:0/2683133293 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fbc78018a80 con 0x7fbc90100d90 2024-02-17T12:09:58.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.630+0000 7fbc87fff640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbc7c0a2230 0x7fbc7c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:58.632 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.630+0000 7fbc8e59c640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbc7c0a2230 0x7fbc7c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:58.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.630+0000 7fbc87fff640 1 -- 172.21.15.94:0/2683133293 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbc78014070 con 0x7fbc90100d90 2024-02-17T12:09:58.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.631+0000 7fbc8e59c640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbc7c0a2230 0x7fbc7c0a46f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009900 tx=0x7fbc80005a20 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:58.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.635+0000 7fbc87fff640 1 -- 172.21.15.94:0/2683133293 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc78052d20 con 0x7fbc90100d90 2024-02-17T12:09:58.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.982+0000 7fbc94d9a640 1 -- 172.21.15.94:0/2683133293 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc9010f0f0 con 0x7fbc90100d90 2024-02-17T12:09:58.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.983+0000 7fbc87fff640 1 -- 172.21.15.94:0/2683133293 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7fbc78097190 con 0x7fbc90100d90 2024-02-17T12:09:58.985 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:09:58.985 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:09:58.985 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:09:58.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.986+0000 7fbc94d9a640 1 -- 172.21.15.94:0/2683133293 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbc7c0a2230 msgr2=0x7fbc7c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:58.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.986+0000 7fbc94d9a640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbc7c0a2230 0x7fbc7c0a46f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009900 tx=0x7fbc80005a20 comp rx=0 tx=0).stop 2024-02-17T12:09:58.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.986+0000 7fbc94d9a640 1 -- 172.21.15.94:0/2683133293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc90100d90 msgr2=0x7fbc9019fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:58.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.986+0000 7fbc94d9a640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc90100d90 0x7fbc9019fc10 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fbc7800e9f0 tx=0x7fbc7800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.986+0000 7fbc94d9a640 1 -- 172.21.15.94:0/2683133293 shutdown_connections 2024-02-17T12:09:58.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.986+0000 7fbc94d9a640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbc7c0a2230 0x7fbc7c0a46f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.986+0000 7fbc94d9a640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbc90105060 0x7fbc90199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.987+0000 7fbc94d9a640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbc90100d90 0x7fbc9019fc10 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.987+0000 7fbc94d9a640 1 --2- 172.21.15.94:0/2683133293 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc901003d0 0x7fbc9019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:58.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.987+0000 7fbc94d9a640 1 -- 172.21.15.94:0/2683133293 >> 172.21.15.94:0/2683133293 conn(0x7fbc90075e80 msgr2=0x7fbc900fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:58.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.987+0000 7fbc94d9a640 1 -- 172.21.15.94:0/2683133293 shutdown_connections 2024-02-17T12:09:58.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:58.987+0000 7fbc94d9a640 1 -- 172.21.15.94:0/2683133293 wait complete. 2024-02-17T12:09:59.004 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:09:59.004 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:09:59.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.171+0000 7fdae60d0640 1 -- 172.21.15.94:0/1363003108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdae010d9d0 msgr2=0x7fdae010de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.171+0000 7fdae60d0640 1 --2- 172.21.15.94:0/1363003108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdae010d9d0 0x7fdae010de50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009ca0 tx=0x7fdad802c120 comp rx=0 tx=0).stop 2024-02-17T12:09:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.171+0000 7fdae60d0640 1 -- 172.21.15.94:0/1363003108 shutdown_connections 2024-02-17T12:09:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.171+0000 7fdae60d0640 1 --2- 172.21.15.94:0/1363003108 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdae011a960 0x7fdae011cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.171+0000 7fdae60d0640 1 --2- 172.21.15.94:0/1363003108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdae010d9d0 0x7fdae010de50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.171+0000 7fdae60d0640 1 --2- 172.21.15.94:0/1363003108 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae010f9a0 0x7fdae010d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.171+0000 7fdae60d0640 1 -- 172.21.15.94:0/1363003108 >> 172.21.15.94:0/1363003108 conn(0x7fdae006c260 msgr2=0x7fdae006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.171+0000 7fdae60d0640 1 -- 172.21.15.94:0/1363003108 shutdown_connections 2024-02-17T12:09:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.171+0000 7fdae60d0640 1 -- 172.21.15.94:0/1363003108 wait complete. 2024-02-17T12:09:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.172+0000 7fdae60d0640 1 Processor -- start 2024-02-17T12:09:59.174 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.172+0000 7fdae60d0640 1 -- start start 2024-02-17T12:09:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.172+0000 7fdae60d0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdae010d9d0 0x7fdae01a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.172+0000 7fdae60d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdae010f9a0 0x7fdae01a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.172+0000 7fdae60d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae011a960 0x7fdae01aae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.172+0000 7fdae60d0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae01ab490 con 0x7fdae010f9a0 2024-02-17T12:09:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.172+0000 7fdae60d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae01ab600 con 0x7fdae011a960 2024-02-17T12:09:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.172+0000 7fdae60d0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae01ab7c0 con 0x7fdae010d9d0 2024-02-17T12:09:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.173+0000 7fdadffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae011a960 0x7fdae01aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.173+0000 7fdadeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdae010f9a0 0x7fdae01a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.173+0000 7fdadffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae011a960 0x7fdae01aae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44366/0 (socket says 172.21.15.94:44366) 2024-02-17T12:09:59.176 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.173+0000 7fdadffff640 1 -- 172.21.15.94:0/2232394597 learned_addr learned my addr 172.21.15.94:0/2232394597 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.173+0000 7fdadf7fe640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdae010d9d0 0x7fdae01a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.173+0000 7fdadffff640 1 -- 172.21.15.94:0/2232394597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdae010d9d0 msgr2=0x7fdae01a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.173+0000 7fdadffff640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdae010d9d0 0x7fdae01a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.173+0000 7fdadffff640 1 -- 172.21.15.94:0/2232394597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdae010f9a0 msgr2=0x7fdae01a6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.173+0000 7fdadffff640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdae010f9a0 0x7fdae01a6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.173+0000 7fdadffff640 1 -- 172.21.15.94:0/2232394597 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad8009900 con 0x7fdae011a960 2024-02-17T12:09:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.173+0000 7fdadeffd640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdae010f9a0 0x7fdae01a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.173+0000 7fdadffff640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae011a960 0x7fdae01aae90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdad400d750 tx=0x7fdad400dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.174+0000 7fdadcff9640 1 -- 172.21.15.94:0/2232394597 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad4009bd0 con 0x7fdae011a960 2024-02-17T12:09:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.174+0000 7fdadcff9640 1 -- 172.21.15.94:0/2232394597 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdad400de60 con 0x7fdae011a960 2024-02-17T12:09:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.174+0000 7fdadcff9640 1 -- 172.21.15.94:0/2232394597 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad4004e70 con 0x7fdae011a960 2024-02-17T12:09:59.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.174+0000 7fdae60d0640 1 -- 172.21.15.94:0/2232394597 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae01abaa0 con 0x7fdae011a960 2024-02-17T12:09:59.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.174+0000 7fdae60d0640 1 -- 172.21.15.94:0/2232394597 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae01b3320 con 0x7fdae011a960 2024-02-17T12:09:59.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.177+0000 7fdadcff9640 1 -- 172.21.15.94:0/2232394597 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fdad4010460 con 0x7fdae011a960 2024-02-17T12:09:59.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.178+0000 7fdadcff9640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdac80a23c0 0x7fdac80a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.178+0000 7fdadcff9640 1 -- 172.21.15.94:0/2232394597 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdad40cd7c0 con 0x7fdae011a960 2024-02-17T12:09:59.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.179+0000 7fdadf7fe640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdac80a23c0 0x7fdac80a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.179+0000 7fdae8d77740 1 Processor -- start 2024-02-17T12:09:59.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.180+0000 7fdadf7fe640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdac80a23c0 0x7fdac80a4880 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009bd0 tx=0x7fdad0005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.180+0000 7fdae8d77740 1 -- start start 2024-02-17T12:09:59.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.180+0000 7fdae8d77740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ec250 0x5632ec0ec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdae8d77740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ecb70 0x5632ec0f5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdae8d77740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0f5750 0x5632ec0f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdadf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ec250 0x5632ec0ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdae8d77740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ec0a7960 con 0x5632ec0ec250 2024-02-17T12:09:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdadf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ec250 0x5632ec0ec630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41168/0 (socket says 172.21.15.94:41168) 2024-02-17T12:09:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdadf7fe640 1 -- 172.21.15.94:0/1558698415 learned_addr learned my addr 172.21.15.94:0/1558698415 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdae8d77740 1 -- 172.21.15.94:0/1558698415 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ec05b310 con 0x5632ec0ecb70 2024-02-17T12:09:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdae8d77740 1 -- 172.21.15.94:0/1558698415 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ec058840 con 0x5632ec0f5750 2024-02-17T12:09:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdadeffd640 1 --2- 172.21.15.94:0/1558698415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ecb70 0x5632ec0f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdadffff640 1 --2- 172.21.15.94:0/1558698415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0f5750 0x5632ec0f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdadf7fe640 1 -- 172.21.15.94:0/1558698415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0f5750 msgr2=0x5632ec0f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdadf7fe640 1 --2- 172.21.15.94:0/1558698415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0f5750 0x5632ec0f7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdadf7fe640 1 -- 172.21.15.94:0/1558698415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ecb70 msgr2=0x5632ec0f5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdadf7fe640 1 --2- 172.21.15.94:0/1558698415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ecb70 0x5632ec0f5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.181+0000 7fdadf7fe640 1 -- 172.21.15.94:0/1558698415 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632ec016060 con 0x5632ec0ec250 2024-02-17T12:09:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.182+0000 7fdadf7fe640 1 --2- 172.21.15.94:0/1558698415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ec250 0x5632ec0ec630 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fdad000e630 tx=0x7fdad0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa194f693ef7c3ac server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.182+0000 7fdadeffd640 1 --2- 172.21.15.94:0/1558698415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ecb70 0x5632ec0f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:09:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.182+0000 7fdadffff640 1 --2- 172.21.15.94:0/1558698415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0f5750 0x5632ec0f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:09:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.182+0000 7fdac5ffb640 1 -- 172.21.15.94:0/1558698415 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0004020 con 0x5632ec0ec250 2024-02-17T12:09:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.182+0000 7fdac5ffb640 1 -- 172.21.15.94:0/1558698415 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdad0021070 con 0x5632ec0ec250 2024-02-17T12:09:59.185 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.182+0000 7fdac5ffb640 1 -- 172.21.15.94:0/1558698415 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad001c650 con 0x5632ec0ec250 2024-02-17T12:09:59.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.183+0000 7fdae8d77740 1 -- 172.21.15.94:0/1558698415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ec250 msgr2=0x5632ec0ec630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.186 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.183+0000 7fdae8d77740 1 --2- 172.21.15.94:0/1558698415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ec250 0x5632ec0ec630 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fdad000e630 tx=0x7fdad0014000 comp rx=0 tx=0).stop 2024-02-17T12:09:59.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.184+0000 7fdae8d77740 1 -- 172.21.15.94:0/1558698415 shutdown_connections 2024-02-17T12:09:59.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.184+0000 7fdae8d77740 1 --2- 172.21.15.94:0/1558698415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0f5750 0x5632ec0f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.184+0000 7fdae8d77740 1 --2- 172.21.15.94:0/1558698415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ecb70 0x5632ec0f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.184+0000 7fdae8d77740 1 --2- 172.21.15.94:0/1558698415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ec250 0x5632ec0ec630 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.184+0000 7fdae8d77740 1 -- 172.21.15.94:0/1558698415 >> 172.21.15.94:0/1558698415 conn(0x5632ec0e6980 msgr2=0x5632ec0e6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.184+0000 7fdae8d77740 1 -- 172.21.15.94:0/1558698415 shutdown_connections 2024-02-17T12:09:59.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.184+0000 7fdae8d77740 1 -- 172.21.15.94:0/1558698415 wait complete. 2024-02-17T12:09:59.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.185+0000 7fdae8d77740 1 -- start start 2024-02-17T12:09:59.187 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.185+0000 7fdae8d77740 1 Processor -- start 2024-02-17T12:09:59.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.185+0000 7fdae8d77740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ec250 0x5632ec0f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.185+0000 7fdae8d77740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ecb70 0x5632ec0f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.185+0000 7fdae8d77740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0f5750 0x5632ec0f3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.189 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.185+0000 7fdae8d77740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ec054020 con 0x5632ec0ecb70 2024-02-17T12:09:59.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdae8d77740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ebd27430 con 0x5632ec0f5750 2024-02-17T12:09:59.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdae8d77740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ebce3e00 con 0x5632ec0ec250 2024-02-17T12:09:59.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdadeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ecb70 0x5632ec0f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdadf7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ec250 0x5632ec0f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdadeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ecb70 0x5632ec0f2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41180/0 (socket says 172.21.15.94:41180) 2024-02-17T12:09:59.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdadf7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ec250 0x5632ec0f26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47276/0 (socket says 172.21.15.94:47276) 2024-02-17T12:09:59.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdadeffd640 1 -- 172.21.15.94:0/2776558929 learned_addr learned my addr 172.21.15.94:0/2776558929 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdae8d77740 10 client.0 _ll_register_callbacks cb 0x5632ebffbcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:59.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdadffff640 1 --2- 172.21.15.94:0/2776558929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0f5750 0x5632ec0f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdadeffd640 1 -- 172.21.15.94:0/2776558929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ec250 msgr2=0x5632ec0f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdadeffd640 1 --2- 172.21.15.94:0/2776558929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ec250 0x5632ec0f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.190 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdadeffd640 1 -- 172.21.15.94:0/2776558929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0f5750 msgr2=0x5632ec0f3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdadeffd640 1 --2- 172.21.15.94:0/2776558929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0f5750 0x5632ec0f3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdadeffd640 1 -- 172.21.15.94:0/2776558929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad0017070 con 0x5632ec0ecb70 2024-02-17T12:09:59.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdadeffd640 1 --2- 172.21.15.94:0/2776558929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ecb70 0x5632ec0f2c10 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009ca0 tx=0x7fdad8032ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47731d448f73d4b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdabeffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:59.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.186+0000 7fdabeffd640 1 -- 172.21.15.94:0/2776558929 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad8034850 con 0x5632ec0ecb70 2024-02-17T12:09:59.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.187+0000 7fdabeffd640 1 -- 172.21.15.94:0/2776558929 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdad8045070 con 0x5632ec0ecb70 2024-02-17T12:09:59.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.187+0000 7fdae8d77740 1 -- 172.21.15.94:0/2776558929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632ebd537e0 con 0x5632ec0ecb70 2024-02-17T12:09:59.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.187+0000 7fdabeffd640 1 -- 172.21.15.94:0/2776558929 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdad8003970 con 0x5632ec0ecb70 2024-02-17T12:09:59.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.187+0000 7fdae8d77740 10 client.18579 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:59.191 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.187+0000 7fdae8d77740 1 -- 172.21.15.94:0/2776558929 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5632ec0526a0 con 0x5632ec0ecb70 2024-02-17T12:09:59.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.189+0000 7fdabeffd640 1 -- 172.21.15.94:0/2776558929 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdad8056080 con 0x5632ec0ecb70 2024-02-17T12:09:59.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.189+0000 7fdabeffd640 1 -- 172.21.15.94:0/2776558929 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdad804a480 con 0x5632ec0ecb70 2024-02-17T12:09:59.193 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.189+0000 7fdae8d77740 10 client.18579 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.189+0000 7fdae8d77740 10 client.18579 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.189+0000 7fdae8d77740 20 client.18579 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.189+0000 7fdae8d77740 1 --2- 172.21.15.94:0/2776558929 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5632ec105ae0 0x5632ec107f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.189+0000 7fdae8d77740 4 client.18579 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.189+0000 7fdae8d77740 1 -- 172.21.15.94:0/2776558929 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632ebd90720 con 0x5632ec105ae0 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.189+0000 7fdadf7fe640 1 --2- 172.21.15.94:0/2776558929 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5632ec105ae0 0x5632ec107f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.190+0000 7fdabeffd640 10 client.18579 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.190+0000 7fdadf7fe640 1 --2- 172.21.15.94:0/2776558929 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5632ec105ae0 0x5632ec107f20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68c782bee1953bef server_cookie=4a44e0b9b6e934e6 in_seq=0 out_seq=0 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.190+0000 7fdabeffd640 1 -- 172.21.15.94:0/2776558929 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5632ebd90720 con 0x5632ec105ae0 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.190+0000 7fdabeffd640 10 client.18579 handle_command_reply: tid=0 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.190+0000 7fdae8d77740 1 client.18579 shutdown 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.190+0000 7fdae8d77740 1 -- 172.21.15.94:0/2776558929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ecb70 msgr2=0x5632ec0f2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.190+0000 7fdae8d77740 1 --2- 172.21.15.94:0/2776558929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ecb70 0x5632ec0f2c10 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009ca0 tx=0x7fdad8032ef0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.194 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.190+0000 7fdae8d77740 1 -- 172.21.15.94:0/2776558929 shutdown_connections 2024-02-17T12:09:59.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.190+0000 7fdae8d77740 1 --2- 172.21.15.94:0/2776558929 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5632ec105ae0 0x5632ec107f20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.191+0000 7fdabeffd640 0 client.18579 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:09:59.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.191+0000 7fdae8d77740 1 --2- 172.21.15.94:0/2776558929 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0f5750 0x5632ec0f3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.191+0000 7fdae8d77740 1 --2- 172.21.15.94:0/2776558929 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0ecb70 0x5632ec0f2c10 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.191+0000 7fdae8d77740 1 --2- 172.21.15.94:0/2776558929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ec250 0x5632ec0f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.191+0000 7fdae8d77740 1 -- 172.21.15.94:0/2776558929 >> 172.21.15.94:0/2776558929 conn(0x5632ec0e6980 msgr2=0x5632ec0e86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.191+0000 7fdae8d77740 1 -- 172.21.15.94:0/2776558929 shutdown_connections 2024-02-17T12:09:59.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.191+0000 7fdae8d77740 1 -- 172.21.15.94:0/2776558929 wait complete. 2024-02-17T12:09:59.195 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.191+0000 7fdae8d77740 20 client.18579 trim_cache size 0 max 16384 2024-02-17T12:09:59.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.207+0000 7fdae8d77740 1 Processor -- start 2024-02-17T12:09:59.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.207+0000 7fdae8d77740 1 -- start start 2024-02-17T12:09:59.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.207+0000 7fdae8d77740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ec250 0x5632ec0f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.207+0000 7fdae8d77740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ecb70 0x5632ec0e5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.207+0000 7fdadf7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ec250 0x5632ec0f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.207+0000 7fdadf7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ec250 0x5632ec0f2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44380/0 (socket says 172.21.15.94:44380) 2024-02-17T12:09:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.207+0000 7fdadf7fe640 1 -- 172.21.15.94:0/3402662434 learned_addr learned my addr 172.21.15.94:0/3402662434 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.207+0000 7fdae8d77740 1 --2- 172.21.15.94:0/3402662434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0f5750 0x5632ec0fc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.207+0000 7fdadeffd640 1 --2- 172.21.15.94:0/3402662434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ecb70 0x5632ec0e5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.208+0000 7fdae8d77740 1 -- 172.21.15.94:0/3402662434 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ebfd9130 con 0x5632ec0f5750 2024-02-17T12:09:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.208+0000 7fdae8d77740 1 -- 172.21.15.94:0/3402662434 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ec10d170 con 0x5632ec0ec250 2024-02-17T12:09:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.208+0000 7fdae8d77740 1 -- 172.21.15.94:0/3402662434 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ec0eb340 con 0x5632ec0ecb70 2024-02-17T12:09:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.208+0000 7fdadffff640 1 --2- 172.21.15.94:0/3402662434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0f5750 0x5632ec0fc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.208+0000 7fdadf7fe640 1 -- 172.21.15.94:0/3402662434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ecb70 msgr2=0x5632ec0e5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.208+0000 7fdadf7fe640 1 --2- 172.21.15.94:0/3402662434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ecb70 0x5632ec0e5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.208+0000 7fdadf7fe640 1 -- 172.21.15.94:0/3402662434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0f5750 msgr2=0x5632ec0fc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.208+0000 7fdadf7fe640 1 --2- 172.21.15.94:0/3402662434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0f5750 0x5632ec0fc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.208+0000 7fdadf7fe640 1 -- 172.21.15.94:0/3402662434 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632ebfeec20 con 0x5632ec0ec250 2024-02-17T12:09:59.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.208+0000 7fdadf7fe640 1 --2- 172.21.15.94:0/3402662434 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ec250 0x5632ec0f2f70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdad000eaf0 tx=0x7fdad0003d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8c5926e030e6f1c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.208+0000 7fdac57fa640 1 -- 172.21.15.94:0/3402662434 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad0038030 con 0x5632ec0ec250 2024-02-17T12:09:59.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.209+0000 7fdac57fa640 1 -- 172.21.15.94:0/3402662434 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdad002b040 con 0x5632ec0ec250 2024-02-17T12:09:59.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.209+0000 7fdac57fa640 1 -- 172.21.15.94:0/3402662434 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad00332d0 con 0x5632ec0ec250 2024-02-17T12:09:59.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.209+0000 7fdae8d77740 1 -- 172.21.15.94:0/3402662434 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ec250 msgr2=0x5632ec0f2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.209+0000 7fdae8d77740 1 --2- 172.21.15.94:0/3402662434 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ec250 0x5632ec0f2f70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdad000eaf0 tx=0x7fdad0003d50 comp rx=0 tx=0).stop 2024-02-17T12:09:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.209+0000 7fdae8d77740 1 -- 172.21.15.94:0/3402662434 shutdown_connections 2024-02-17T12:09:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.209+0000 7fdae8d77740 1 --2- 172.21.15.94:0/3402662434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0f5750 0x5632ec0fc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.209+0000 7fdae8d77740 1 --2- 172.21.15.94:0/3402662434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ecb70 0x5632ec0e5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.209+0000 7fdae8d77740 1 --2- 172.21.15.94:0/3402662434 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ec250 0x5632ec0f2f70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.209+0000 7fdae8d77740 1 -- 172.21.15.94:0/3402662434 >> 172.21.15.94:0/3402662434 conn(0x5632ec0e6980 msgr2=0x5632ec0e52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.209+0000 7fdae8d77740 1 -- 172.21.15.94:0/3402662434 shutdown_connections 2024-02-17T12:09:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.209+0000 7fdae8d77740 1 -- 172.21.15.94:0/3402662434 wait complete. 2024-02-17T12:09:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.210+0000 7fdae8d77740 1 -- start start 2024-02-17T12:09:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.210+0000 7fdae8d77740 1 Processor -- start 2024-02-17T12:09:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.210+0000 7fdae8d77740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ec250 0x5632ec0fb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.210+0000 7fdae8d77740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ecb70 0x5632ec0fb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.210+0000 7fdae8d77740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0f5750 0x5632ec15c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdadf7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ec250 0x5632ec0fb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdadf7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ec250 0x5632ec0fb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44384/0 (socket says 172.21.15.94:44384) 2024-02-17T12:09:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdae8d77740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ec0fc000 con 0x5632ec0f5750 2024-02-17T12:09:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdae8d77740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ec0fc170 con 0x5632ec0ec250 2024-02-17T12:09:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdae8d77740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632ec15c5c0 con 0x5632ec0ecb70 2024-02-17T12:09:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdadf7fe640 1 -- 172.21.15.94:0/3642652212 learned_addr learned my addr 172.21.15.94:0/3642652212 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdadffff640 1 --2- 172.21.15.94:0/3642652212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0f5750 0x5632ec15c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdae8d77740 10 client.0 _ll_register_callbacks cb 0x7fdad0009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdadeffd640 1 --2- 172.21.15.94:0/3642652212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ecb70 0x5632ec0fb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdadffff640 1 -- 172.21.15.94:0/3642652212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ecb70 msgr2=0x5632ec0fb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdadffff640 1 --2- 172.21.15.94:0/3642652212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ecb70 0x5632ec0fb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdadffff640 1 -- 172.21.15.94:0/3642652212 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ec250 msgr2=0x5632ec0fb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdadffff640 1 --2- 172.21.15.94:0/3642652212 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ec250 0x5632ec0fb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdadffff640 1 -- 172.21.15.94:0/3642652212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632ec016060 con 0x5632ec0f5750 2024-02-17T12:09:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.211+0000 7fdadffff640 1 --2- 172.21.15.94:0/3642652212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0f5750 0x5632ec15c050 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fdad40cae70 tx=0x7fdad40a3aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7818dc797003ab0e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.212+0000 7fdabeffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.212+0000 7fdabeffd640 1 -- 172.21.15.94:0/3642652212 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad4097960 con 0x5632ec0f5750 2024-02-17T12:09:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.212+0000 7fdabeffd640 1 -- 172.21.15.94:0/3642652212 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdad4097ac0 con 0x5632ec0f5750 2024-02-17T12:09:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.212+0000 7fdae8d77740 1 -- 172.21.15.94:0/3642652212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdad0021070 con 0x5632ec0f5750 2024-02-17T12:09:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.212+0000 7fdabeffd640 1 -- 172.21.15.94:0/3642652212 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdad40cb0f0 con 0x5632ec0f5750 2024-02-17T12:09:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.213+0000 7fdae8d77740 10 client.18585 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:59.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.213+0000 7fdae8d77740 1 -- 172.21.15.94:0/3642652212 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5632ec0148c0 con 0x5632ec0f5750 2024-02-17T12:09:59.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.213+0000 7fdabeffd640 1 -- 172.21.15.94:0/3642652212 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdad40cb250 con 0x5632ec0f5750 2024-02-17T12:09:59.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.213+0000 7fdabeffd640 1 -- 172.21.15.94:0/3642652212 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdad40a02b0 con 0x5632ec0f5750 2024-02-17T12:09:59.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.213+0000 7fdae8d77740 10 client.18585 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:59.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.213+0000 7fdae8d77740 10 client.18585 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:09:59.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.213+0000 7fdae8d77740 20 client.18585 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:59.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.213+0000 7fdae8d77740 1 --2- 172.21.15.94:0/3642652212 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5632ec15dc90 0x5632ec1600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.214+0000 7fdae8d77740 4 client.18585 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:09:59.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.214+0000 7fdae8d77740 1 -- 172.21.15.94:0/3642652212 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fdad0021070 con 0x5632ec15dc90 2024-02-17T12:09:59.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.214+0000 7fdadf7fe640 1 --2- 172.21.15.94:0/3642652212 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5632ec15dc90 0x5632ec1600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.215+0000 7fdabeffd640 10 client.18585 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.215+0000 7fdadf7fe640 1 --2- 172.21.15.94:0/3642652212 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5632ec15dc90 0x5632ec1600b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94bb4e615742f0ca server_cookie=a987343b30d20523 in_seq=0 out_seq=0 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.215+0000 7fdabeffd640 1 -- 172.21.15.94:0/3642652212 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fdad0021070 con 0x5632ec15dc90 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.215+0000 7fdabeffd640 10 client.18585 handle_command_reply: tid=0 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.215+0000 7fdae8d77740 1 client.18585 shutdown 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.215+0000 7fdae8d77740 1 -- 172.21.15.94:0/3642652212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0f5750 msgr2=0x5632ec15c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.215+0000 7fdae8d77740 1 --2- 172.21.15.94:0/3642652212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0f5750 0x5632ec15c050 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fdad40cae70 tx=0x7fdad40a3aa0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.215+0000 7fdae8d77740 1 -- 172.21.15.94:0/3642652212 shutdown_connections 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.215+0000 7fdae8d77740 1 --2- 172.21.15.94:0/3642652212 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5632ec0f5750 0x5632ec15c050 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae8d77740 1 --2- 172.21.15.94:0/3642652212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632ec0ecb70 0x5632ec0fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae8d77740 1 --2- 172.21.15.94:0/3642652212 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5632ec15dc90 0x5632ec1600b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae8d77740 1 --2- 172.21.15.94:0/3642652212 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5632ec0ec250 0x5632ec0fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdabeffd640 0 client.18585 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae8d77740 1 -- 172.21.15.94:0/3642652212 >> 172.21.15.94:0/3642652212 conn(0x5632ec0e6980 msgr2=0x5632ec0e5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.221 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae8d77740 1 -- 172.21.15.94:0/3642652212 shutdown_connections 2024-02-17T12:09:59.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae8d77740 1 -- 172.21.15.94:0/3642652212 wait complete. 2024-02-17T12:09:59.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae8d77740 20 client.18585 trim_cache size 0 max 16384 2024-02-17T12:09:59.222 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:09:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae60d0640 1 -- 172.21.15.94:0/2232394597 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdac80a23c0 msgr2=0x7fdac80a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae60d0640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdac80a23c0 0x7fdac80a4880 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdad0009bd0 tx=0x7fdad0005f90 comp rx=0 tx=0).stop 2024-02-17T12:09:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae60d0640 1 -- 172.21.15.94:0/2232394597 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae011a960 msgr2=0x7fdae01aae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae60d0640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae011a960 0x7fdae01aae90 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdad400d750 tx=0x7fdad400dc10 comp rx=0 tx=0).stop 2024-02-17T12:09:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae60d0640 1 -- 172.21.15.94:0/2232394597 shutdown_connections 2024-02-17T12:09:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae60d0640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdac80a23c0 0x7fdac80a4880 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae60d0640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdae011a960 0x7fdae01aae90 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae60d0640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdae010f9a0 0x7fdae01a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae60d0640 1 --2- 172.21.15.94:0/2232394597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdae010d9d0 0x7fdae01a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.216+0000 7fdae60d0640 1 -- 172.21.15.94:0/2232394597 >> 172.21.15.94:0/2232394597 conn(0x7fdae006c260 msgr2=0x7fdae0071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.217+0000 7fdae60d0640 1 -- 172.21.15.94:0/2232394597 shutdown_connections 2024-02-17T12:09:59.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.217+0000 7fdae60d0640 1 -- 172.21.15.94:0/2232394597 wait complete. 2024-02-17T12:09:59.232 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:09:59.233 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:09:59.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:59 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2683133293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:09:59.263 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:09:59 smithi094 ceph-mon[32499]: pgmap v369: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.8 MiB/s wr, 748 op/s 2024-02-17T12:09:59.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.379+0000 7ff80a98c640 1 -- 172.21.15.94:0/1579017156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff804100bf0 msgr2=0x7ff804101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.379+0000 7ff80a98c640 1 --2- 172.21.15.94:0/1579017156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff804100bf0 0x7ff804101070 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009930 tx=0x7ff7f802bc30 comp rx=0 tx=0).stop 2024-02-17T12:09:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.379+0000 7ff80a98c640 1 -- 172.21.15.94:0/1579017156 shutdown_connections 2024-02-17T12:09:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.379+0000 7ff80a98c640 1 --2- 172.21.15.94:0/1579017156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff8041015b0 0x7ff80410cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.379+0000 7ff80a98c640 1 --2- 172.21.15.94:0/1579017156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff804100bf0 0x7ff804101070 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.379+0000 7ff80a98c640 1 --2- 172.21.15.94:0/1579017156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff804105080 0x7ff804105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.379+0000 7ff80a98c640 1 -- 172.21.15.94:0/1579017156 >> 172.21.15.94:0/1579017156 conn(0x7ff8040761d0 msgr2=0x7ff8040fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.379+0000 7ff80a98c640 1 -- 172.21.15.94:0/1579017156 shutdown_connections 2024-02-17T12:09:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.379+0000 7ff80a98c640 1 -- 172.21.15.94:0/1579017156 wait complete. 2024-02-17T12:09:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.380+0000 7ff80a98c640 1 Processor -- start 2024-02-17T12:09:59.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.380+0000 7ff80a98c640 1 -- start start 2024-02-17T12:09:59.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.380+0000 7ff80a98c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff804100bf0 0x7ff80410ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.380+0000 7ff80a98c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8041015b0 0x7ff804108d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.380+0000 7ff80a98c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff804105080 0x7ff8041092c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.380+0000 7ff80a98c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff804109a50 con 0x7ff804100bf0 2024-02-17T12:09:59.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.380+0000 7ff80a98c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff804109bc0 con 0x7ff804105080 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.380+0000 7ff80a98c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8041aa330 con 0x7ff8041015b0 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.380+0000 7ff803fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff804100bf0 0x7ff80410ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.380+0000 7ff803fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff804100bf0 0x7ff80410ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41230/0 (socket says 172.21.15.94:41230) 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.380+0000 7ff803fff640 1 -- 172.21.15.94:0/360993362 learned_addr learned my addr 172.21.15.94:0/360993362 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.380+0000 7ff808f29640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff804105080 0x7ff8041092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.381+0000 7ff8037fe640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8041015b0 0x7ff804108d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.381+0000 7ff803fff640 1 -- 172.21.15.94:0/360993362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8041015b0 msgr2=0x7ff804108d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.381+0000 7ff803fff640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8041015b0 0x7ff804108d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.381+0000 7ff803fff640 1 -- 172.21.15.94:0/360993362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff804105080 msgr2=0x7ff8041092c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.381+0000 7ff803fff640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff804105080 0x7ff8041092c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.381+0000 7ff803fff640 1 -- 172.21.15.94:0/360993362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc009840 con 0x7ff804100bf0 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.381+0000 7ff808f29640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff804105080 0x7ff8041092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.381+0000 7ff803fff640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff804100bf0 0x7ff80410ac00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400d770 tx=0x7ff7f400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.381+0000 7ff8017fa640 1 -- 172.21.15.94:0/360993362 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4005220 con 0x7ff804100bf0 2024-02-17T12:09:59.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.381+0000 7ff80a98c640 1 -- 172.21.15.94:0/360993362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f8009560 con 0x7ff804100bf0 2024-02-17T12:09:59.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.381+0000 7ff8017fa640 1 -- 172.21.15.94:0/360993362 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7f4009d00 con 0x7ff804100bf0 2024-02-17T12:09:59.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.381+0000 7ff8017fa640 1 -- 172.21.15.94:0/360993362 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4004e00 con 0x7ff804100bf0 2024-02-17T12:09:59.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.381+0000 7ff80a98c640 1 -- 172.21.15.94:0/360993362 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8041aa960 con 0x7ff804100bf0 2024-02-17T12:09:59.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.384+0000 7ff80d564740 1 Processor -- start 2024-02-17T12:09:59.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.385+0000 7ff80d564740 1 -- start start 2024-02-17T12:09:59.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.385+0000 7ff80d564740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b597fd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.385+0000 7ff80d564740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b59806210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.385+0000 7ff80d564740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b59808b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.385+0000 7ff80d564740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b597b8960 con 0x562b597fdb70 2024-02-17T12:09:59.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.385+0000 7ff80d564740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5976c310 con 0x562b59806750 2024-02-17T12:09:59.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.385+0000 7ff80d564740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b59769840 con 0x562b597fd250 2024-02-17T12:09:59.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.385+0000 7ff808f29640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b597fd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.385+0000 7ff8037fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b59808b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.385+0000 7ff808f29640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b597fd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47342/0 (socket says 172.21.15.94:47342) 2024-02-17T12:09:59.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.385+0000 7ff808f29640 1 -- 172.21.15.94:0/551317105 learned_addr learned my addr 172.21.15.94:0/551317105 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.385+0000 7ff8037fe640 1 --2- 172.21.15.94:0/551317105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b59808b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44402/0 (socket says 172.21.15.94:44402) 2024-02-17T12:09:59.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.386+0000 7ff8037fe640 1 -- 172.21.15.94:0/551317105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 msgr2=0x562b597fd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.386+0000 7ff8037fe640 1 --2- 172.21.15.94:0/551317105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b597fd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.386+0000 7ff8037fe640 1 -- 172.21.15.94:0/551317105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 msgr2=0x562b59806210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:09:59.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.386+0000 7ff8037fe640 1 --2- 172.21.15.94:0/551317105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b59806210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.386+0000 7ff8037fe640 1 -- 172.21.15.94:0/551317105 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b59727060 con 0x562b59806750 2024-02-17T12:09:59.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.386+0000 7ff808f29640 1 --2- 172.21.15.94:0/551317105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b597fd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:59.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.386+0000 7ff8017fa640 1 -- 172.21.15.94:0/360993362 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff7f4009820 con 0x7ff804100bf0 2024-02-17T12:09:59.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.386+0000 7ff8037fe640 1 --2- 172.21.15.94:0/551317105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b59808b40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009230 tx=0x7ff7f802c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d21abe706b43b7d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.386+0000 7ff7ea7fc640 1 -- 172.21.15.94:0/551317105 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8004a00 con 0x562b59806750 2024-02-17T12:09:59.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.386+0000 7ff7ea7fc640 1 -- 172.21.15.94:0/551317105 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7f8004020 con 0x562b59806750 2024-02-17T12:09:59.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.386+0000 7ff7ea7fc640 1 -- 172.21.15.94:0/551317105 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f80052b0 con 0x562b59806750 2024-02-17T12:09:59.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.386+0000 7ff8017fa640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7ec0a2230 0x7ff7ec0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.387+0000 7ff808f29640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7ec0a2230 0x7ff7ec0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.387+0000 7ff8017fa640 1 -- 172.21.15.94:0/360993362 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff7f40cdb30 con 0x7ff804100bf0 2024-02-17T12:09:59.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.387+0000 7ff808f29640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7ec0a2230 0x7ff7ec0a46f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00ece0 tx=0x7ff7fc009500 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.401 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.387+0000 7ff80d564740 1 -- 172.21.15.94:0/551317105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 msgr2=0x562b59808b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.387+0000 7ff80d564740 1 --2- 172.21.15.94:0/551317105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b59808b40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8009230 tx=0x7ff7f802c5d0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.387+0000 7ff80d564740 1 -- 172.21.15.94:0/551317105 shutdown_connections 2024-02-17T12:09:59.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.387+0000 7ff80d564740 1 --2- 172.21.15.94:0/551317105 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b59808b40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.387+0000 7ff80d564740 1 --2- 172.21.15.94:0/551317105 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b59806210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.387+0000 7ff80d564740 1 --2- 172.21.15.94:0/551317105 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b597fd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.387+0000 7ff80d564740 1 -- 172.21.15.94:0/551317105 >> 172.21.15.94:0/551317105 conn(0x562b597f7980 msgr2=0x562b597f7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.387+0000 7ff80d564740 1 -- 172.21.15.94:0/551317105 shutdown_connections 2024-02-17T12:09:59.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.387+0000 7ff80d564740 1 -- 172.21.15.94:0/551317105 wait complete. 2024-02-17T12:09:59.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff80d564740 1 -- start start 2024-02-17T12:09:59.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff80d564740 1 Processor -- start 2024-02-17T12:09:59.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff80d564740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fd250 0x562b59803770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff80d564740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fdb70 0x562b59803cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff80d564740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b598041f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff80d564740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b59765020 con 0x562b597fd250 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff80d564740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b59438430 con 0x562b59806750 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff80d564740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b593f4e00 con 0x562b597fdb70 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff80d564740 10 client.0 _ll_register_callbacks cb 0x562b5970ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff808f29640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b598041f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff808f29640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b598041f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44412/0 (socket says 172.21.15.94:44412) 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff808f29640 1 -- 172.21.15.94:0/4245857261 learned_addr learned my addr 172.21.15.94:0/4245857261 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff803fff640 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fd250 0x562b59803770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.388+0000 7ff8037fe640 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fdb70 0x562b59803cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff808f29640 1 -- 172.21.15.94:0/4245857261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fdb70 msgr2=0x562b59803cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff808f29640 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fdb70 0x562b59803cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff808f29640 1 -- 172.21.15.94:0/4245857261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fd250 msgr2=0x562b59803770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff808f29640 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fd250 0x562b59803770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff808f29640 1 -- 172.21.15.94:0/4245857261 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80410d810 con 0x562b59806750 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff803fff640 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fd250 0x562b59803770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff808f29640 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b598041f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc01e420 tx=0x7ff7fc004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21c4d3defb510166 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff7e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff7e37fe640 1 -- 172.21.15.94:0/4245857261 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7fc023070 con 0x562b59806750 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff7e37fe640 1 -- 172.21.15.94:0/4245857261 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff7fc01ea10 con 0x562b59806750 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff80d564740 1 -- 172.21.15.94:0/4245857261 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b594647e0 con 0x562b59806750 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff7e37fe640 1 -- 172.21.15.94:0/4245857261 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7fc017070 con 0x562b59806750 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff80d564740 10 client.28049 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff80d564740 1 -- 172.21.15.94:0/4245857261 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562b597636a0 con 0x562b59806750 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.389+0000 7ff7e37fe640 1 -- 172.21.15.94:0/4245857261 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562b594647e0 con 0x562b59806750 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.390+0000 7ff7e37fe640 1 -- 172.21.15.94:0/4245857261 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff7fc029420 con 0x562b59806750 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.390+0000 7ff80d564740 10 client.28049 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.390+0000 7ff80d564740 10 client.28049 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:09:59.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.390+0000 7ff80d564740 20 client.28049 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.390+0000 7ff80d564740 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562b59816b40 0x562b59818f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.390+0000 7ff80d564740 4 client.28049 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.390+0000 7ff80d564740 1 -- 172.21.15.94:0/4245857261 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b594a1720 con 0x562b59816b40 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.391+0000 7ff803fff640 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562b59816b40 0x562b59818f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.391+0000 7ff803fff640 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562b59816b40 0x562b59818f30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0ca7174b297324d server_cookie=bc834898ac1c15c9 in_seq=0 out_seq=0 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.391+0000 7ff7e37fe640 10 client.28049 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.392+0000 7ff7e37fe640 1 -- 172.21.15.94:0/4245857261 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b594a1720 con 0x562b59816b40 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.392+0000 7ff7e37fe640 10 client.28049 handle_command_reply: tid=0 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.392+0000 7ff80d564740 1 client.28049 shutdown 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.392+0000 7ff80d564740 1 -- 172.21.15.94:0/4245857261 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 msgr2=0x562b598041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.392+0000 7ff80d564740 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b598041f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc01e420 tx=0x7ff7fc004ab0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.392+0000 7ff80d564740 1 -- 172.21.15.94:0/4245857261 shutdown_connections 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.392+0000 7ff80d564740 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562b59816b40 0x562b59818f30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.392+0000 7ff80d564740 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b598041f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.392+0000 7ff7e37fe640 0 client.28049 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:09:59.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.392+0000 7ff80d564740 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fdb70 0x562b59803cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.392+0000 7ff80d564740 1 --2- 172.21.15.94:0/4245857261 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fd250 0x562b59803770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.392+0000 7ff80d564740 1 -- 172.21.15.94:0/4245857261 >> 172.21.15.94:0/4245857261 conn(0x562b597f7980 msgr2=0x562b597f96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.392+0000 7ff80d564740 1 -- 172.21.15.94:0/4245857261 shutdown_connections 2024-02-17T12:09:59.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.394+0000 7ff80d564740 1 -- 172.21.15.94:0/4245857261 wait complete. 2024-02-17T12:09:59.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.394+0000 7ff80d564740 20 client.28049 trim_cache size 0 max 16384 2024-02-17T12:09:59.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:59 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2683133293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:09:59.406 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:09:59 smithi189 ceph-mon[36588]: pgmap v369: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.8 MiB/s wr, 748 op/s 2024-02-17T12:09:59.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.410+0000 7ff80d564740 1 Processor -- start 2024-02-17T12:09:59.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.410+0000 7ff80d564740 1 -- start start 2024-02-17T12:09:59.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.410+0000 7ff80d564740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b59805ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.410+0000 7ff80d564740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b59802040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.410+0000 7ff80d564740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b59802580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.410+0000 7ff80d564740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b596ea130 con 0x562b597fdb70 2024-02-17T12:09:59.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.410+0000 7ff80d564740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b597fceb0 con 0x562b59806750 2024-02-17T12:09:59.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.410+0000 7ff80d564740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b59803590 con 0x562b597fd250 2024-02-17T12:09:59.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff8037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b59802040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff8037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b59802040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41250/0 (socket says 172.21.15.94:41250) 2024-02-17T12:09:59.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff8037fe640 1 -- 172.21.15.94:0/1341667094 learned_addr learned my addr 172.21.15.94:0/1341667094 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff808f29640 1 --2- 172.21.15.94:0/1341667094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b59802580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff803fff640 1 --2- 172.21.15.94:0/1341667094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b59805ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff8037fe640 1 -- 172.21.15.94:0/1341667094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 msgr2=0x562b59805ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff8037fe640 1 --2- 172.21.15.94:0/1341667094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b59805ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff8037fe640 1 -- 172.21.15.94:0/1341667094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 msgr2=0x562b59802580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff8037fe640 1 --2- 172.21.15.94:0/1341667094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b59802580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff8037fe640 1 -- 172.21.15.94:0/1341667094 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b596ffc20 con 0x562b597fdb70 2024-02-17T12:09:59.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff808f29640 1 --2- 172.21.15.94:0/1341667094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b59802580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:59.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff8037fe640 1 --2- 172.21.15.94:0/1341667094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b59802040 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff7f802bc70 tx=0x7ff7f802c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6aaebde641e2471 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff7e9ffb640 1 -- 172.21.15.94:0/1341667094 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8004290 con 0x562b597fdb70 2024-02-17T12:09:59.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff7e9ffb640 1 -- 172.21.15.94:0/1341667094 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7f80043f0 con 0x562b597fdb70 2024-02-17T12:09:59.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.411+0000 7ff7e9ffb640 1 -- 172.21.15.94:0/1341667094 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8005230 con 0x562b597fdb70 2024-02-17T12:09:59.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.412+0000 7ff80d564740 1 -- 172.21.15.94:0/1341667094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 msgr2=0x562b59802040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.412+0000 7ff80d564740 1 --2- 172.21.15.94:0/1341667094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b59802040 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7ff7f802bc70 tx=0x7ff7f802c360 comp rx=0 tx=0).stop 2024-02-17T12:09:59.418 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.412+0000 7ff80d564740 1 -- 172.21.15.94:0/1341667094 shutdown_connections 2024-02-17T12:09:59.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.412+0000 7ff80d564740 1 --2- 172.21.15.94:0/1341667094 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b59802580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.412+0000 7ff80d564740 1 --2- 172.21.15.94:0/1341667094 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b59802040 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.412+0000 7ff80d564740 1 --2- 172.21.15.94:0/1341667094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b59805ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.412+0000 7ff80d564740 1 -- 172.21.15.94:0/1341667094 >> 172.21.15.94:0/1341667094 conn(0x562b597f7980 msgr2=0x562b5981d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.412+0000 7ff80d564740 1 -- 172.21.15.94:0/1341667094 shutdown_connections 2024-02-17T12:09:59.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.412+0000 7ff80d564740 1 -- 172.21.15.94:0/1341667094 wait complete. 2024-02-17T12:09:59.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.412+0000 7ff80d564740 1 -- start start 2024-02-17T12:09:59.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.413+0000 7ff80d564740 1 Processor -- start 2024-02-17T12:09:59.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.413+0000 7ff80d564740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b5980d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.413+0000 7ff80d564740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b5980bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.413+0000 7ff80d564740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b5980c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.413+0000 7ff80d564740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5980c8f0 con 0x562b597fdb70 2024-02-17T12:09:59.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.413+0000 7ff80d564740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5980ca60 con 0x562b59806750 2024-02-17T12:09:59.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.413+0000 7ff80d564740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b5986d010 con 0x562b597fd250 2024-02-17T12:09:59.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.413+0000 7ff80d564740 10 client.0 _ll_register_callbacks cb 0x7ff7fc0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:59.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.414+0000 7ff803fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b5980d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.414+0000 7ff808f29640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b5980c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.414+0000 7ff8037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b5980bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.414+0000 7ff803fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b5980d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47374/0 (socket says 172.21.15.94:47374) 2024-02-17T12:09:59.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.414+0000 7ff803fff640 1 -- 172.21.15.94:0/1087193883 learned_addr learned my addr 172.21.15.94:0/1087193883 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.414+0000 7ff8037fe640 1 -- 172.21.15.94:0/1087193883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 msgr2=0x562b5980d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.414+0000 7ff8037fe640 1 --2- 172.21.15.94:0/1087193883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b5980d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.414+0000 7ff8037fe640 1 -- 172.21.15.94:0/1087193883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 msgr2=0x562b5980c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.414+0000 7ff8037fe640 1 --2- 172.21.15.94:0/1087193883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b5980c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.414+0000 7ff8037fe640 1 -- 172.21.15.94:0/1087193883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b597636a0 con 0x562b597fdb70 2024-02-17T12:09:59.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.415+0000 7ff8037fe640 1 --2- 172.21.15.94:0/1087193883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b5980bcb0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8005ec0 tx=0x7ff7f8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aace59fe77b5fb76 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.415+0000 7ff7e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:09:59.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.415+0000 7ff7e37fe640 1 -- 172.21.15.94:0/1087193883 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8004af0 con 0x562b597fdb70 2024-02-17T12:09:59.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.415+0000 7ff7e37fe640 1 -- 172.21.15.94:0/1087193883 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff7f80050a0 con 0x562b597fdb70 2024-02-17T12:09:59.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.415+0000 7ff80d564740 1 -- 172.21.15.94:0/1087193883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7f8004020 con 0x562b597fdb70 2024-02-17T12:09:59.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.416+0000 7ff7e37fe640 1 -- 172.21.15.94:0/1087193883 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7f8005260 con 0x562b597fdb70 2024-02-17T12:09:59.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.416+0000 7ff80d564740 10 client.18600 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:59.427 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.416+0000 7ff80d564740 1 -- 172.21.15.94:0/1087193883 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562b59727060 con 0x562b597fdb70 2024-02-17T12:09:59.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.416+0000 7ff7e37fe640 1 -- 172.21.15.94:0/1087193883 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7f8004020 con 0x562b597fdb70 2024-02-17T12:09:59.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.416+0000 7ff7e37fe640 1 -- 172.21.15.94:0/1087193883 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff7f803c040 con 0x562b597fdb70 2024-02-17T12:09:59.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.416+0000 7ff80d564740 10 client.18600 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:59.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.416+0000 7ff80d564740 10 client.18600 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:09:59.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.416+0000 7ff80d564740 20 client.18600 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:59.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.416+0000 7ff80d564740 1 --2- 172.21.15.94:0/1087193883 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562b59816b40 0x562b59870330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.416+0000 7ff80d564740 4 client.18600 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:09:59.428 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.416+0000 7ff80d564740 1 -- 172.21.15.94:0/1087193883 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562b594f0890 con 0x562b59816b40 2024-02-17T12:09:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.417+0000 7ff803fff640 1 --2- 172.21.15.94:0/1087193883 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562b59816b40 0x562b59870330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.417+0000 7ff7e37fe640 10 client.18600 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:09:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.417+0000 7ff803fff640 1 --2- 172.21.15.94:0/1087193883 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562b59816b40 0x562b59870330 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3ff8415c8773cc7 server_cookie=56be01cacdd8c541 in_seq=0 out_seq=0 2024-02-17T12:09:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.417+0000 7ff7e37fe640 1 -- 172.21.15.94:0/1087193883 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562b594f0890 con 0x562b59816b40 2024-02-17T12:09:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.417+0000 7ff7e37fe640 10 client.18600 handle_command_reply: tid=0 2024-02-17T12:09:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.417+0000 7ff80d564740 1 client.18600 shutdown 2024-02-17T12:09:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80d564740 1 -- 172.21.15.94:0/1087193883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 msgr2=0x562b5980bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80d564740 1 --2- 172.21.15.94:0/1087193883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b5980bcb0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8005ec0 tx=0x7ff7f8005fd0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80d564740 1 -- 172.21.15.94:0/1087193883 shutdown_connections 2024-02-17T12:09:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80d564740 1 --2- 172.21.15.94:0/1087193883 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x562b59816b40 0x562b59870330 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80d564740 1 --2- 172.21.15.94:0/1087193883 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562b59806750 0x562b5980c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff7e37fe640 0 client.18600 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:09:59.429 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80d564740 1 --2- 172.21.15.94:0/1087193883 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562b597fdb70 0x562b5980bcb0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80d564740 1 --2- 172.21.15.94:0/1087193883 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b597fd250 0x562b5980d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80d564740 1 -- 172.21.15.94:0/1087193883 >> 172.21.15.94:0/1087193883 conn(0x562b597f7980 msgr2=0x562b59801fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80d564740 1 -- 172.21.15.94:0/1087193883 shutdown_connections 2024-02-17T12:09:59.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80d564740 1 -- 172.21.15.94:0/1087193883 wait complete. 2024-02-17T12:09:59.430 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80d564740 20 client.18600 trim_cache size 0 max 16384 2024-02-17T12:09:59.431 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:09:59.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80a98c640 1 -- 172.21.15.94:0/360993362 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7ec0a2230 msgr2=0x7ff7ec0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.432 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80a98c640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7ec0a2230 0x7ff7ec0a46f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc00ece0 tx=0x7ff7fc009500 comp rx=0 tx=0).stop 2024-02-17T12:09:59.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80a98c640 1 -- 172.21.15.94:0/360993362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff804100bf0 msgr2=0x7ff80410ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80a98c640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff804100bf0 0x7ff80410ac00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7ff7f400d770 tx=0x7ff7f400db70 comp rx=0 tx=0).stop 2024-02-17T12:09:59.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80a98c640 1 -- 172.21.15.94:0/360993362 shutdown_connections 2024-02-17T12:09:59.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80a98c640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7ec0a2230 0x7ff7ec0a46f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.433 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80a98c640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff804105080 0x7ff8041092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80a98c640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8041015b0 0x7ff804108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80a98c640 1 --2- 172.21.15.94:0/360993362 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff804100bf0 0x7ff80410ac00 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.418+0000 7ff80a98c640 1 -- 172.21.15.94:0/360993362 >> 172.21.15.94:0/360993362 conn(0x7ff8040761d0 msgr2=0x7ff80410b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.419+0000 7ff80a98c640 1 -- 172.21.15.94:0/360993362 shutdown_connections 2024-02-17T12:09:59.434 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.419+0000 7ff80a98c640 1 -- 172.21.15.94:0/360993362 wait complete. 2024-02-17T12:09:59.435 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:09:59.435 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:09:59.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:59 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2683133293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:09:59.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:09:59 smithi125 ceph-mon[36819]: pgmap v369: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 7.8 MiB/s wr, 748 op/s 2024-02-17T12:09:59.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.640+0000 7f8f34023640 1 -- 172.21.15.94:0/3433148821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f2c108da0 msgr2=0x7f8f2c10c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.640+0000 7f8f34023640 1 --2- 172.21.15.94:0/3433148821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f2c108da0 0x7f8f2c10c7d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8f24008760 tx=0x7f8f2400aea0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.640+0000 7f8f34023640 1 -- 172.21.15.94:0/3433148821 shutdown_connections 2024-02-17T12:09:59.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.640+0000 7f8f34023640 1 --2- 172.21.15.94:0/3433148821 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f2c10cf00 0x7f8f2c114a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.640+0000 7f8f34023640 1 --2- 172.21.15.94:0/3433148821 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f2c108da0 0x7f8f2c10c7d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.640+0000 7f8f34023640 1 --2- 172.21.15.94:0/3433148821 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f2c1083f0 0x7f8f2c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.640+0000 7f8f34023640 1 -- 172.21.15.94:0/3433148821 >> 172.21.15.94:0/3433148821 conn(0x7f8f2c06c6c0 msgr2=0x7f8f2c06cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.641+0000 7f8f34023640 1 -- 172.21.15.94:0/3433148821 shutdown_connections 2024-02-17T12:09:59.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.644+0000 7f8f34023640 1 -- 172.21.15.94:0/3433148821 wait complete. 2024-02-17T12:09:59.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.644+0000 7f8f34023640 1 Processor -- start 2024-02-17T12:09:59.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.644+0000 7f8f34023640 1 -- start start 2024-02-17T12:09:59.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f34023640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f2c1083f0 0x7f8f2c1a22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f34023640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f2c10cf00 0x7f8f2c1a2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f34023640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f2c1a6b50 0x7f8f2c1a7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f34023640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c1a7540 con 0x7f8f2c10cf00 2024-02-17T12:09:59.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f34023640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c1a76b0 con 0x7f8f2c1a6b50 2024-02-17T12:09:59.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f34023640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c1a7820 con 0x7f8f2c1083f0 2024-02-17T12:09:59.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f315be640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f2c10cf00 0x7f8f2c1a2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f315be640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f2c10cf00 0x7f8f2c1a2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41278/0 (socket says 172.21.15.94:41278) 2024-02-17T12:09:59.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f315be640 1 -- 172.21.15.94:0/3749433867 learned_addr learned my addr 172.21.15.94:0/3749433867 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f31dbf640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f2c1083f0 0x7f8f2c1a22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f325c0640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f2c1a6b50 0x7f8f2c1a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f31dbf640 1 -- 172.21.15.94:0/3749433867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f2c1a6b50 msgr2=0x7f8f2c1a7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f31dbf640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f2c1a6b50 0x7f8f2c1a7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f31dbf640 1 -- 172.21.15.94:0/3749433867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f2c10cf00 msgr2=0x7f8f2c1a2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f31dbf640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f2c10cf00 0x7f8f2c1a2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f31dbf640 1 -- 172.21.15.94:0/3749433867 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f240083c0 con 0x7f8f2c1083f0 2024-02-17T12:09:59.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.645+0000 7f8f315be640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f2c10cf00 0x7f8f2c1a2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:09:59.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.646+0000 7f8f31dbf640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f2c1083f0 0x7f8f2c1a22e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00d6d0 tx=0x7f8f1c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.646+0000 7f8f22ffd640 1 -- 172.21.15.94:0/3749433867 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c009a20 con 0x7f8f2c1083f0 2024-02-17T12:09:59.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.646+0000 7f8f34023640 1 -- 172.21.15.94:0/3749433867 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2c1aee00 con 0x7f8f2c1083f0 2024-02-17T12:09:59.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.646+0000 7f8f34023640 1 -- 172.21.15.94:0/3749433867 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2c1af270 con 0x7f8f2c1083f0 2024-02-17T12:09:59.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.646+0000 7f8f22ffd640 1 -- 172.21.15.94:0/3749433867 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f1c009b80 con 0x7f8f2c1083f0 2024-02-17T12:09:59.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.646+0000 7f8f22ffd640 1 -- 172.21.15.94:0/3749433867 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c00f620 con 0x7f8f2c1083f0 2024-02-17T12:09:59.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.650+0000 7f8f22ffd640 1 -- 172.21.15.94:0/3749433867 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8f1c010460 con 0x7f8f2c1083f0 2024-02-17T12:09:59.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.650+0000 7f8f36bf7740 1 Processor -- start 2024-02-17T12:09:59.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.650+0000 7f8f36bf7740 1 -- start start 2024-02-17T12:09:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.650+0000 7f8f36bf7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac918ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.650+0000 7f8f36bf7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9193a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.650+0000 7f8f36bf7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 0x55bac9196350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.650+0000 7f8f36bf7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac913d960 con 0x55bac9193f60 2024-02-17T12:09:59.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.650+0000 7f8f36bf7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac90f1310 con 0x55bac918aa60 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.650+0000 7f8f36bf7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac90ee840 con 0x55bac918b380 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f22ffd640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8f180a2370 0x7f8f180a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f325c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac918ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f325c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac918ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44444/0 (socket says 172.21.15.94:44444) 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f315be640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 0x55bac9196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f325c0640 1 -- 172.21.15.94:0/1308376230 learned_addr learned my addr 172.21.15.94:0/1308376230 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f31dbf640 1 --2- 172.21.15.94:0/1308376230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9193a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f325c0640 1 -- 172.21.15.94:0/1308376230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 msgr2=0x55bac9193a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f325c0640 1 --2- 172.21.15.94:0/1308376230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9193a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f325c0640 1 -- 172.21.15.94:0/1308376230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 msgr2=0x55bac9196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f325c0640 1 --2- 172.21.15.94:0/1308376230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 0x55bac9196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f325c0640 1 -- 172.21.15.94:0/1308376230 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bac90ac060 con 0x55bac918aa60 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f325c0640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8f180a2370 0x7f8f180a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f325c0640 1 --2- 172.21.15.94:0/1308376230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac918ae40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800b860 tx=0x7f8f2800bd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b266109fcc08864 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f22ffd640 1 -- 172.21.15.94:0/3749433867 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8f1c0cddb0 con 0x7f8f2c1083f0 2024-02-17T12:09:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.651+0000 7f8f17fff640 1 -- 172.21.15.94:0/1308376230 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f28010040 con 0x55bac918aa60 2024-02-17T12:09:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.652+0000 7f8f17fff640 1 -- 172.21.15.94:0/1308376230 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f28027070 con 0x55bac918aa60 2024-02-17T12:09:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.652+0000 7f8f17fff640 1 -- 172.21.15.94:0/1308376230 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f280225b0 con 0x55bac918aa60 2024-02-17T12:09:59.655 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.652+0000 7f8f325c0640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8f180a2370 0x7f8f180a4830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8f28022000 tx=0x7f8f28023000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.652+0000 7f8f36bf7740 1 -- 172.21.15.94:0/1308376230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 msgr2=0x55bac918ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.652+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/1308376230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac918ae40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8f2800b860 tx=0x7f8f2800bd20 comp rx=0 tx=0).stop 2024-02-17T12:09:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.653+0000 7f8f36bf7740 1 -- 172.21.15.94:0/1308376230 shutdown_connections 2024-02-17T12:09:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.653+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/1308376230 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 0x55bac9196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.653+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/1308376230 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9193a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.653+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/1308376230 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac918ae40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.653+0000 7f8f36bf7740 1 -- 172.21.15.94:0/1308376230 >> 172.21.15.94:0/1308376230 conn(0x55bac9185190 msgr2=0x55bac9185570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.656 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.654+0000 7f8f36bf7740 1 -- 172.21.15.94:0/1308376230 shutdown_connections 2024-02-17T12:09:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.654+0000 7f8f36bf7740 1 -- 172.21.15.94:0/1308376230 wait complete. 2024-02-17T12:09:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.655+0000 7f8f36bf7740 1 -- start start 2024-02-17T12:09:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.655+0000 7f8f36bf7740 1 Processor -- start 2024-02-17T12:09:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.655+0000 7f8f36bf7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac9190f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.655+0000 7f8f36bf7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9191450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.656+0000 7f8f36bf7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 0x55bac9191990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.656+0000 7f8f36bf7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac90ea020 con 0x55bac9193f60 2024-02-17T12:09:59.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.656+0000 7f8f36bf7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac8dbd430 con 0x55bac918aa60 2024-02-17T12:09:59.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.656+0000 7f8f36bf7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac8d79e00 con 0x55bac918b380 2024-02-17T12:09:59.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.656+0000 7f8f36bf7740 10 client.0 _ll_register_callbacks cb 0x55bac9091cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:59.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.656+0000 7f8f315be640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9191450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.656+0000 7f8f315be640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9191450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47414/0 (socket says 172.21.15.94:47414) 2024-02-17T12:09:59.658 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.656+0000 7f8f315be640 1 -- 172.21.15.94:0/618133771 learned_addr learned my addr 172.21.15.94:0/618133771 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.656+0000 7f8f325c0640 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 0x55bac9191990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.656+0000 7f8f31dbf640 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac9190f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.656+0000 7f8f315be640 1 -- 172.21.15.94:0/618133771 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 msgr2=0x55bac9190f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.656+0000 7f8f315be640 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac9190f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.657+0000 7f8f315be640 1 -- 172.21.15.94:0/618133771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 msgr2=0x55bac9191990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.657+0000 7f8f315be640 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 0x55bac9191990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.657+0000 7f8f315be640 1 -- 172.21.15.94:0/618133771 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f280042c0 con 0x55bac918b380 2024-02-17T12:09:59.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.657+0000 7f8f31dbf640 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac9190f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:09:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.657+0000 7f8f315be640 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9191450 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8f240086b0 tx=0x7f8f2402d850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fb51aaaf6ffeaaa server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.657+0000 7f8f157fa640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:09:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.657+0000 7f8f157fa640 1 -- 172.21.15.94:0/618133771 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24003970 con 0x55bac918b380 2024-02-17T12:09:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.657+0000 7f8f36bf7740 1 -- 172.21.15.94:0/618133771 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bac8de97e0 con 0x55bac918b380 2024-02-17T12:09:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.657+0000 7f8f157fa640 1 -- 172.21.15.94:0/618133771 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8f2403d040 con 0x55bac918b380 2024-02-17T12:09:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.658+0000 7f8f157fa640 1 -- 172.21.15.94:0/618133771 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f24003ad0 con 0x55bac918b380 2024-02-17T12:09:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.658+0000 7f8f36bf7740 10 client.28072 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.658+0000 7f8f36bf7740 1 -- 172.21.15.94:0/618133771 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bac90e86a0 con 0x55bac918b380 2024-02-17T12:09:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.658+0000 7f8f157fa640 1 -- 172.21.15.94:0/618133771 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55bac8de97e0 con 0x55bac918b380 2024-02-17T12:09:59.661 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.659+0000 7f8f157fa640 1 -- 172.21.15.94:0/618133771 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8f24004410 con 0x55bac918b380 2024-02-17T12:09:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.659+0000 7f8f36bf7740 10 client.28072 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.659+0000 7f8f36bf7740 10 client.28072 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.659+0000 7f8f36bf7740 20 client.28072 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.659+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55bac91a4440 0x55bac91a6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.659+0000 7f8f36bf7740 4 client.28072 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:09:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.659+0000 7f8f36bf7740 1 -- 172.21.15.94:0/618133771 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bac8e26720 con 0x55bac91a4440 2024-02-17T12:09:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.659+0000 7f8f31dbf640 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55bac91a4440 0x55bac91a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.659+0000 7f8f31dbf640 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55bac91a4440 0x55bac91a6860 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f47f599857acbc83 server_cookie=70b9d3f4ad2f1e0c in_seq=0 out_seq=0 2024-02-17T12:09:59.662 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.659+0000 7f8f157fa640 10 client.28072 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.660+0000 7f8f157fa640 1 -- 172.21.15.94:0/618133771 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bac8e26720 con 0x55bac91a4440 2024-02-17T12:09:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.660+0000 7f8f157fa640 10 client.28072 handle_command_reply: tid=0 2024-02-17T12:09:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.660+0000 7f8f36bf7740 1 client.28072 shutdown 2024-02-17T12:09:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.661+0000 7f8f36bf7740 1 -- 172.21.15.94:0/618133771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 msgr2=0x55bac9191450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.663 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.661+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9191450 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8f240086b0 tx=0x7f8f2402d850 comp rx=0 tx=0).stop 2024-02-17T12:09:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.661+0000 7f8f36bf7740 1 -- 172.21.15.94:0/618133771 shutdown_connections 2024-02-17T12:09:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.661+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55bac91a4440 0x55bac91a6860 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.661+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 0x55bac9191990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.661+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9191450 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.661+0000 7f8f157fa640 0 client.28072 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.661+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/618133771 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac9190f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.661+0000 7f8f36bf7740 1 -- 172.21.15.94:0/618133771 >> 172.21.15.94:0/618133771 conn(0x55bac9185190 msgr2=0x55bac9186650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.661+0000 7f8f36bf7740 1 -- 172.21.15.94:0/618133771 shutdown_connections 2024-02-17T12:09:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.661+0000 7f8f36bf7740 1 -- 172.21.15.94:0/618133771 wait complete. 2024-02-17T12:09:59.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.662+0000 7f8f36bf7740 20 client.28072 trim_cache size 0 max 16384 2024-02-17T12:09:59.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.681+0000 7f8f36bf7740 1 Processor -- start 2024-02-17T12:09:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.681+0000 7f8f36bf7740 1 -- start start 2024-02-17T12:09:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.681+0000 7f8f36bf7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918aa60 0x55bac9193640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.681+0000 7f8f36bf7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac918b380 0x55bac919ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f36bf7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac9193f60 0x55bac919b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f36bf7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac906f130 con 0x55bac918b380 2024-02-17T12:09:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f36bf7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac91785c0 con 0x55bac9193f60 2024-02-17T12:09:59.685 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f36bf7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac918a4a0 con 0x55bac918aa60 2024-02-17T12:09:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f315be640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac918b380 0x55bac919ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f315be640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac918b380 0x55bac919ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41314/0 (socket says 172.21.15.94:41314) 2024-02-17T12:09:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f315be640 1 -- 172.21.15.94:0/897775459 learned_addr learned my addr 172.21.15.94:0/897775459 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f325c0640 1 --2- 172.21.15.94:0/897775459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac9193f60 0x55bac919b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f31dbf640 1 --2- 172.21.15.94:0/897775459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918aa60 0x55bac9193640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f315be640 1 -- 172.21.15.94:0/897775459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918aa60 msgr2=0x55bac9193640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f315be640 1 --2- 172.21.15.94:0/897775459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918aa60 0x55bac9193640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f315be640 1 -- 172.21.15.94:0/897775459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac9193f60 msgr2=0x55bac919b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f315be640 1 --2- 172.21.15.94:0/897775459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac9193f60 0x55bac919b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.682+0000 7f8f315be640 1 -- 172.21.15.94:0/897775459 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bac9084c20 con 0x55bac918b380 2024-02-17T12:09:59.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.684+0000 7f8f315be640 1 --2- 172.21.15.94:0/897775459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac918b380 0x55bac919ae70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8f24005c40 tx=0x7f8f24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ad7a3a27b30871 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.684+0000 7f8f177fe640 1 -- 172.21.15.94:0/897775459 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24038450 con 0x55bac918b380 2024-02-17T12:09:59.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.684+0000 7f8f177fe640 1 -- 172.21.15.94:0/897775459 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8f2402de60 con 0x55bac918b380 2024-02-17T12:09:59.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.684+0000 7f8f177fe640 1 -- 172.21.15.94:0/897775459 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24044440 con 0x55bac918b380 2024-02-17T12:09:59.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.685+0000 7f8f36bf7740 1 -- 172.21.15.94:0/897775459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac918b380 msgr2=0x55bac919ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.685+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/897775459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac918b380 0x55bac919ae70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8f24005c40 tx=0x7f8f24004a40 comp rx=0 tx=0).stop 2024-02-17T12:09:59.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.685+0000 7f8f36bf7740 1 -- 172.21.15.94:0/897775459 shutdown_connections 2024-02-17T12:09:59.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.685+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/897775459 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac9193f60 0x55bac919b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.685+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/897775459 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac918b380 0x55bac919ae70 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.685+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/897775459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918aa60 0x55bac9193640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.685+0000 7f8f36bf7740 1 -- 172.21.15.94:0/897775459 >> 172.21.15.94:0/897775459 conn(0x55bac9185190 msgr2=0x55bac9177a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.685+0000 7f8f36bf7740 1 -- 172.21.15.94:0/897775459 shutdown_connections 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.685+0000 7f8f36bf7740 1 -- 172.21.15.94:0/897775459 wait complete. 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.686+0000 7f8f36bf7740 1 -- start start 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.686+0000 7f8f36bf7740 1 Processor -- start 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f36bf7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac9199870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f36bf7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9199db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f36bf7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 0x55bac919a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f36bf7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac91fa970 con 0x55bac9193f60 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f31dbf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac9199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f31dbf640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac9199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44462/0 (socket says 172.21.15.94:44462) 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f31dbf640 1 -- 172.21.15.94:0/956227664 learned_addr learned my addr 172.21.15.94:0/956227664 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f36bf7740 1 -- 172.21.15.94:0/956227664 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac91faab0 con 0x55bac918aa60 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f36bf7740 1 -- 172.21.15.94:0/956227664 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bac91fabf0 con 0x55bac918b380 2024-02-17T12:09:59.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f36bf7740 10 client.0 _ll_register_callbacks cb 0x7f8f28004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f325c0640 1 --2- 172.21.15.94:0/956227664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 0x55bac919a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f315be640 1 --2- 172.21.15.94:0/956227664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f31dbf640 1 -- 172.21.15.94:0/956227664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 msgr2=0x55bac9199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f31dbf640 1 --2- 172.21.15.94:0/956227664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f31dbf640 1 -- 172.21.15.94:0/956227664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 msgr2=0x55bac919a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.687+0000 7f8f31dbf640 1 --2- 172.21.15.94:0/956227664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 0x55bac919a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f31dbf640 1 -- 172.21.15.94:0/956227664 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bac90e86a0 con 0x55bac918aa60 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f31dbf640 1 --2- 172.21.15.94:0/956227664 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac9199870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c010d80 tx=0x7f8f1c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=182aef7b74f5c6a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f157fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f157fa640 1 -- 172.21.15.94:0/956227664 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0b3120 con 0x55bac918aa60 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f157fa640 1 -- 172.21.15.94:0/956227664 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8f1c009cf0 con 0x55bac918aa60 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f36bf7740 1 -- 172.21.15.94:0/956227664 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bac9190d40 con 0x55bac918aa60 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f157fa640 1 -- 172.21.15.94:0/956227664 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f1c0cb6e0 con 0x55bac918aa60 2024-02-17T12:09:59.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f36bf7740 10 client.28070 fetch_fsmap learned FSMap version 27 2024-02-17T12:09:59.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f36bf7740 1 -- 172.21.15.94:0/956227664 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55bac90aa8c0 con 0x55bac918aa60 2024-02-17T12:09:59.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f157fa640 1 -- 172.21.15.94:0/956227664 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55bac9190d40 con 0x55bac918aa60 2024-02-17T12:09:59.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f157fa640 1 -- 172.21.15.94:0/956227664 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8f1c09aa10 con 0x55bac918aa60 2024-02-17T12:09:59.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f36bf7740 10 client.28070 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:09:59.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f36bf7740 10 client.28070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:09:59.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f36bf7740 20 client.28070 populate_metadata read hostname 'smithi094' 2024-02-17T12:09:59.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.688+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/956227664 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55bac91fc2f0 0x55bac91fe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:09:59.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.689+0000 7f8f36bf7740 4 client.28070 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:09:59.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.689+0000 7f8f36bf7740 1 -- 172.21.15.94:0/956227664 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f8f28027070 con 0x55bac91fc2f0 2024-02-17T12:09:59.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.689+0000 7f8f315be640 1 --2- 172.21.15.94:0/956227664 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55bac91fc2f0 0x55bac91fe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:09:59.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.690+0000 7f8f315be640 1 --2- 172.21.15.94:0/956227664 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55bac91fc2f0 0x55bac91fe710 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde8eb6a15ce5c47 server_cookie=e2acc137ecd2fb30 in_seq=0 out_seq=0 2024-02-17T12:09:59.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.690+0000 7f8f157fa640 10 client.28070 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:59.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.691+0000 7f8f157fa640 1 -- 172.21.15.94:0/956227664 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f8f28027070 con 0x55bac91fc2f0 2024-02-17T12:09:59.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.691+0000 7f8f157fa640 10 client.28070 handle_command_reply: tid=0 2024-02-17T12:09:59.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.691+0000 7f8f36bf7740 1 client.28070 shutdown 2024-02-17T12:09:59.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.691+0000 7f8f36bf7740 1 -- 172.21.15.94:0/956227664 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 msgr2=0x55bac9199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.691+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/956227664 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac9199870 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c010d80 tx=0x7f8f1c000bc0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.691+0000 7f8f36bf7740 1 -- 172.21.15.94:0/956227664 shutdown_connections 2024-02-17T12:09:59.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.691+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/956227664 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55bac91fc2f0 0x55bac91fe710 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.691+0000 7f8f157fa640 0 client.28070 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:09:59.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.691+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/956227664 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55bac9193f60 0x55bac919a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.691+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/956227664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bac918b380 0x55bac9199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.691+0000 7f8f36bf7740 1 --2- 172.21.15.94:0/956227664 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55bac918aa60 0x55bac9199870 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.691+0000 7f8f36bf7740 1 -- 172.21.15.94:0/956227664 >> 172.21.15.94:0/956227664 conn(0x55bac9185190 msgr2=0x55bac918f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f36bf7740 1 -- 172.21.15.94:0/956227664 shutdown_connections 2024-02-17T12:09:59.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f36bf7740 1 -- 172.21.15.94:0/956227664 wait complete. 2024-02-17T12:09:59.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f36bf7740 20 client.28070 trim_cache size 0 max 16384 2024-02-17T12:09:59.701 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:09:59.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f34023640 1 -- 172.21.15.94:0/3749433867 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8f180a2370 msgr2=0x7f8f180a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f34023640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8f180a2370 0x7f8f180a4830 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8f28022000 tx=0x7f8f28023000 comp rx=0 tx=0).stop 2024-02-17T12:09:59.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f34023640 1 -- 172.21.15.94:0/3749433867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f2c1083f0 msgr2=0x7f8f2c1a22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:09:59.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f34023640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f2c1083f0 0x7f8f2c1a22e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00d6d0 tx=0x7f8f1c00db90 comp rx=0 tx=0).stop 2024-02-17T12:09:59.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f34023640 1 -- 172.21.15.94:0/3749433867 shutdown_connections 2024-02-17T12:09:59.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f34023640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f8f180a2370 0x7f8f180a4830 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f34023640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8f2c1a6b50 0x7f8f2c1a7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f34023640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f8f2c10cf00 0x7f8f2c1a2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f34023640 1 --2- 172.21.15.94:0/3749433867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8f2c1083f0 0x7f8f2c1a22e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:09:59.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f34023640 1 -- 172.21.15.94:0/3749433867 >> 172.21.15.94:0/3749433867 conn(0x7f8f2c06c6c0 msgr2=0x7f8f2c071060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:09:59.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.692+0000 7f8f34023640 1 -- 172.21.15.94:0/3749433867 shutdown_connections 2024-02-17T12:09:59.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:09:59.693+0000 7f8f34023640 1 -- 172.21.15.94:0/3749433867 wait complete. 2024-02-17T12:10:00.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 700.0 s, 2072.8 tps, lat 47.881 ms stddev 53.233 2024-02-17T12:10:00.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:00 smithi189 ceph-mon[36588]: overall HEALTH_OK 2024-02-17T12:10:00.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:00 smithi125 ceph-mon[36819]: overall HEALTH_OK 2024-02-17T12:10:00.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:00 smithi094 ceph-mon[32499]: overall HEALTH_OK 2024-02-17T12:10:00.709 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:10:00.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:10:00.709 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:10:00.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.868+0000 7f73010f4640 1 -- 172.21.15.94:0/416355809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72fc10d9d0 msgr2=0x7f72fc10de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:00.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.868+0000 7f73010f4640 1 --2- 172.21.15.94:0/416355809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72fc10d9d0 0x7f72fc10de50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009ca0 tx=0x7f72f402c140 comp rx=0 tx=0).stop 2024-02-17T12:10:00.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.872+0000 7f73010f4640 1 -- 172.21.15.94:0/416355809 shutdown_connections 2024-02-17T12:10:00.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.872+0000 7f73010f4640 1 --2- 172.21.15.94:0/416355809 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72fc11a960 0x7f72fc11cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.872+0000 7f73010f4640 1 --2- 172.21.15.94:0/416355809 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72fc10d9d0 0x7f72fc10de50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.872+0000 7f73010f4640 1 --2- 172.21.15.94:0/416355809 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72fc10f9a0 0x7f72fc10d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.872+0000 7f73010f4640 1 -- 172.21.15.94:0/416355809 >> 172.21.15.94:0/416355809 conn(0x7f72fc06c260 msgr2=0x7f72fc06c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:00.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.877+0000 7f73010f4640 1 -- 172.21.15.94:0/416355809 shutdown_connections 2024-02-17T12:10:00.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.877+0000 7f73010f4640 1 -- 172.21.15.94:0/416355809 wait complete. 2024-02-17T12:10:00.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.878+0000 7f73010f4640 1 Processor -- start 2024-02-17T12:10:00.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.879+0000 7f73010f4640 1 -- start start 2024-02-17T12:10:00.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.879+0000 7f73010f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72fc10d9d0 0x7f72fc119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.879+0000 7f73010f4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72fc10f9a0 0x7f72fc114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.881 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.879+0000 7f73010f4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72fc11a960 0x7f72fc1145a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.879+0000 7f73010f4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc114ae0 con 0x7f72fc10d9d0 2024-02-17T12:10:00.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.879+0000 7f73010f4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc114c50 con 0x7f72fc10f9a0 2024-02-17T12:10:00.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.879+0000 7f73010f4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc114e10 con 0x7f72fc11a960 2024-02-17T12:10:00.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.880+0000 7f72fa59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72fc10f9a0 0x7f72fc114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:00.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.880+0000 7f72fa59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72fc10f9a0 0x7f72fc114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44480/0 (socket says 172.21.15.94:44480) 2024-02-17T12:10:00.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.880+0000 7f72fa59c640 1 -- 172.21.15.94:0/1494665964 learned_addr learned my addr 172.21.15.94:0/1494665964 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:00.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.880+0000 7f72fad9d640 1 --2- 172.21.15.94:0/1494665964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72fc10d9d0 0x7f72fc119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:00.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.881+0000 7f72fa59c640 1 -- 172.21.15.94:0/1494665964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72fc11a960 msgr2=0x7f72fc1145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:00.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.881+0000 7f72fa59c640 1 --2- 172.21.15.94:0/1494665964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72fc11a960 0x7f72fc1145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.881+0000 7f72fa59c640 1 -- 172.21.15.94:0/1494665964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72fc10d9d0 msgr2=0x7f72fc119fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:00.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.881+0000 7f72fa59c640 1 --2- 172.21.15.94:0/1494665964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72fc10d9d0 0x7f72fc119fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.881+0000 7f72fa59c640 1 -- 172.21.15.94:0/1494665964 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f4009900 con 0x7f72fc10f9a0 2024-02-17T12:10:00.884 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.881+0000 7f72fa59c640 1 --2- 172.21.15.94:0/1494665964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72fc10f9a0 0x7f72fc114060 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f72f402c750 tx=0x7f72f4032550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:00.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.883+0000 7f72e3fff640 1 -- 172.21.15.94:0/1494665964 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f4034850 con 0x7f72fc10f9a0 2024-02-17T12:10:00.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.883+0000 7f72e3fff640 1 -- 172.21.15.94:0/1494665964 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f72f4003e50 con 0x7f72fc10f9a0 2024-02-17T12:10:00.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.883+0000 7f72e3fff640 1 -- 172.21.15.94:0/1494665964 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f40406b0 con 0x7f72fc10f9a0 2024-02-17T12:10:00.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.883+0000 7f73010f4640 1 -- 172.21.15.94:0/1494665964 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72fc1bb610 con 0x7f72fc10f9a0 2024-02-17T12:10:00.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.883+0000 7f73010f4640 1 -- 172.21.15.94:0/1494665964 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72fc1bbaa0 con 0x7f72fc10f9a0 2024-02-17T12:10:00.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.887+0000 7f7303d9c740 1 Processor -- start 2024-02-17T12:10:00.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.887+0000 7f72e3fff640 1 -- 172.21.15.94:0/1494665964 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f72f400a040 con 0x7f72fc10f9a0 2024-02-17T12:10:00.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.888+0000 7f7303d9c740 1 -- start start 2024-02-17T12:10:00.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.888+0000 7f72e3fff640 1 --2- 172.21.15.94:0/1494665964 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f72e40a23c0 0x7f72e40a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.888+0000 7f72e3fff640 1 -- 172.21.15.94:0/1494665964 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f72f40f0e50 con 0x7f72fc10f9a0 2024-02-17T12:10:00.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.889+0000 7f7303d9c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a8f380 0x561c33a8f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.889+0000 7f7303d9c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8fd30 0x561c33a982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.889+0000 7f7303d9c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a988b0 0x561c33a9ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.889+0000 7f7303d9c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c339fdf90 con 0x561c33a8f380 2024-02-17T12:10:00.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.889+0000 7f7303d9c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c339fb4c0 con 0x561c33a988b0 2024-02-17T12:10:00.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.889+0000 7f7303d9c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c339f6ca0 con 0x561c33a8fd30 2024-02-17T12:10:00.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.889+0000 7f72fa59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8fd30 0x561c33a982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:00.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.889+0000 7f72fb59e640 1 --2- 172.21.15.94:0/1494665964 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f72e40a23c0 0x7f72e40a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:00.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.889+0000 7f72fa59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8fd30 0x561c33a982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47468/0 (socket says 172.21.15.94:47468) 2024-02-17T12:10:00.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.889+0000 7f72fa59c640 1 -- 172.21.15.94:0/3627861359 learned_addr learned my addr 172.21.15.94:0/3627861359 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:00.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.889+0000 7f72fb59e640 1 --2- 172.21.15.94:0/3627861359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a988b0 0x561c33a9ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:00.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.890+0000 7f72fa59c640 1 -- 172.21.15.94:0/3627861359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a988b0 msgr2=0x561c33a9ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:00.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.890+0000 7f72fa59c640 1 --2- 172.21.15.94:0/3627861359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a988b0 0x561c33a9ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.890+0000 7f72fa59c640 1 -- 172.21.15.94:0/3627861359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a8f380 msgr2=0x561c33a8f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:00.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.890+0000 7f72fa59c640 1 --2- 172.21.15.94:0/3627861359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a8f380 0x561c33a8f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.890+0000 7f72fa59c640 1 -- 172.21.15.94:0/3627861359 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c339b90b0 con 0x561c33a8fd30 2024-02-17T12:10:00.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.890+0000 7f72fb59e640 1 --2- 172.21.15.94:0/1494665964 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f72e40a23c0 0x7f72e40a4880 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f72f00143a0 tx=0x7f72f0009660 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:00.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.890+0000 7f72fa59c640 1 --2- 172.21.15.94:0/3627861359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8fd30 0x561c33a982e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f72f40ba2e0 tx=0x7f72f4050000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f264dd202e531605 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:00.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.891+0000 7f72e17fa640 1 -- 172.21.15.94:0/3627861359 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f40c1a60 con 0x561c33a8fd30 2024-02-17T12:10:00.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.891+0000 7f72e17fa640 1 -- 172.21.15.94:0/3627861359 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f72f40c60b0 con 0x561c33a8fd30 2024-02-17T12:10:00.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.892+0000 7f72e17fa640 1 -- 172.21.15.94:0/3627861359 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f40ba140 con 0x561c33a8fd30 2024-02-17T12:10:00.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.892+0000 7f7303d9c740 1 -- 172.21.15.94:0/3627861359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8fd30 msgr2=0x561c33a982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:00.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.892+0000 7f7303d9c740 1 --2- 172.21.15.94:0/3627861359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8fd30 0x561c33a982e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f72f40ba2e0 tx=0x7f72f4050000 comp rx=0 tx=0).stop 2024-02-17T12:10:00.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.892+0000 7f7303d9c740 1 -- 172.21.15.94:0/3627861359 shutdown_connections 2024-02-17T12:10:00.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.892+0000 7f7303d9c740 1 --2- 172.21.15.94:0/3627861359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a988b0 0x561c33a9ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.893+0000 7f7303d9c740 1 --2- 172.21.15.94:0/3627861359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8fd30 0x561c33a982e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.893+0000 7f7303d9c740 1 --2- 172.21.15.94:0/3627861359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a8f380 0x561c33a8f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.893+0000 7f7303d9c740 1 -- 172.21.15.94:0/3627861359 >> 172.21.15.94:0/3627861359 conn(0x561c33a89840 msgr2=0x561c33a89c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:00.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.893+0000 7f7303d9c740 1 -- 172.21.15.94:0/3627861359 shutdown_connections 2024-02-17T12:10:00.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.893+0000 7f7303d9c740 1 -- 172.21.15.94:0/3627861359 wait complete. 2024-02-17T12:10:00.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.893+0000 7f7303d9c740 1 -- start start 2024-02-17T12:10:00.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.894+0000 7f7303d9c740 1 Processor -- start 2024-02-17T12:10:00.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.894+0000 7f7303d9c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a8f380 0x561c33a958c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.894+0000 7f7303d9c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8fd30 0x561c33a95e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.894+0000 7f72fad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a8f380 0x561c33a958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.894+0000 7f72fad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a8f380 0x561c33a958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44502/0 (socket says 172.21.15.94:44502) 2024-02-17T12:10:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.894+0000 7f7303d9c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a988b0 0x561c33a96340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.894+0000 7f72fad9d640 1 -- 172.21.15.94:0/2900146949 learned_addr learned my addr 172.21.15.94:0/2900146949 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.894+0000 7f7303d9c740 1 -- 172.21.15.94:0/2900146949 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c3399bf40 con 0x561c33a988b0 2024-02-17T12:10:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.894+0000 7f7303d9c740 1 -- 172.21.15.94:0/2900146949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c336ca4b0 con 0x561c33a8f380 2024-02-17T12:10:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.894+0000 7f7303d9c740 1 -- 172.21.15.94:0/2900146949 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c3397c670 con 0x561c33a8fd30 2024-02-17T12:10:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.894+0000 7f72fa59c640 1 --2- 172.21.15.94:0/2900146949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8fd30 0x561c33a95e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.895+0000 7f7303d9c740 10 client.0 _ll_register_callbacks cb 0x561c339e9f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.895+0000 7f72fad9d640 1 -- 172.21.15.94:0/2900146949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8fd30 msgr2=0x561c33a95e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.895+0000 7f72fad9d640 1 --2- 172.21.15.94:0/2900146949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8fd30 0x561c33a95e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.895+0000 7f72fad9d640 1 -- 172.21.15.94:0/2900146949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a988b0 msgr2=0x561c33a96340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.895+0000 7f72fad9d640 1 --2- 172.21.15.94:0/2900146949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a988b0 0x561c33a96340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.895+0000 7f72fad9d640 1 -- 172.21.15.94:0/2900146949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72fc1bbaa0 con 0x561c33a8f380 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.895+0000 7f72fad9d640 1 --2- 172.21.15.94:0/2900146949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a8f380 0x561c33a958c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00e450 tx=0x7f72ec00e910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eddb1525dd3e141 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.895+0000 7f7303d9c740 1 -- 172.21.15.94:0/2900146949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c339d0f40 con 0x561c33a8f380 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.895+0000 7f72deff5640 10 client.28082 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.895+0000 7f72deff5640 1 -- 172.21.15.94:0/2900146949 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec013680 con 0x561c33a8f380 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.896+0000 7f72deff5640 1 -- 172.21.15.94:0/2900146949 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f72ec0137e0 con 0x561c33a8f380 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.896+0000 7f72deff5640 1 -- 172.21.15.94:0/2900146949 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72ec013a00 con 0x561c33a8f380 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.896+0000 7f7303d9c740 10 client.28082 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.896+0000 7f7303d9c740 1 -- 172.21.15.94:0/2900146949 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x561c339f5320 con 0x561c33a8f380 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.896+0000 7f72deff5640 1 -- 172.21.15.94:0/2900146949 <== mon.2 v2:172.21.15.125:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f72ec011070 con 0x561c33a8f380 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.896+0000 7f7303d9c740 10 client.28082 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.896+0000 7f7303d9c740 10 client.28082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.896+0000 7f7303d9c740 20 client.28082 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.896+0000 7f7303d9c740 1 --2- 172.21.15.94:0/2900146949 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561c33aa8cc0 0x561c33aab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.897+0000 7f7303d9c740 4 client.28082 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:10:00.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.897+0000 7f7303d9c740 1 -- 172.21.15.94:0/2900146949 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c339d0f40 con 0x561c33aa8cc0 2024-02-17T12:10:00.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.897+0000 7f72fa59c640 1 --2- 172.21.15.94:0/2900146949 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561c33aa8cc0 0x561c33aab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:00.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.898+0000 7f72fa59c640 1 --2- 172.21.15.94:0/2900146949 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561c33aa8cc0 0x561c33aab120 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e1793cd23a0d77 server_cookie=834d8dba23f377fe in_seq=0 out_seq=0 2024-02-17T12:10:00.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.898+0000 7f72deff5640 10 client.28082 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:00.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.899+0000 7f72deff5640 1 -- 172.21.15.94:0/2900146949 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561c339d0f40 con 0x561c33aa8cc0 2024-02-17T12:10:00.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.899+0000 7f72deff5640 10 client.28082 handle_command_reply: tid=0 2024-02-17T12:10:00.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.899+0000 7f7303d9c740 1 client.28082 shutdown 2024-02-17T12:10:00.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.900+0000 7f7303d9c740 1 -- 172.21.15.94:0/2900146949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a8f380 msgr2=0x561c33a958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:00.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.900+0000 7f7303d9c740 1 --2- 172.21.15.94:0/2900146949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a8f380 0x561c33a958c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00e450 tx=0x7f72ec00e910 comp rx=0 tx=0).stop 2024-02-17T12:10:00.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.901+0000 7f7303d9c740 1 -- 172.21.15.94:0/2900146949 shutdown_connections 2024-02-17T12:10:00.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.901+0000 7f7303d9c740 1 --2- 172.21.15.94:0/2900146949 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561c33aa8cc0 0x561c33aab120 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.901+0000 7f7303d9c740 1 --2- 172.21.15.94:0/2900146949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a988b0 0x561c33a96340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.901+0000 7f7303d9c740 1 --2- 172.21.15.94:0/2900146949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8fd30 0x561c33a95e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.901+0000 7f7303d9c740 1 --2- 172.21.15.94:0/2900146949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a8f380 0x561c33a958c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.901+0000 7f7303d9c740 1 -- 172.21.15.94:0/2900146949 >> 172.21.15.94:0/2900146949 conn(0x561c33a89840 msgr2=0x561c33a8b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.901+0000 7f72deff5640 0 client.28082 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.901+0000 7f7303d9c740 1 -- 172.21.15.94:0/2900146949 shutdown_connections 2024-02-17T12:10:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.901+0000 7f7303d9c740 1 -- 172.21.15.94:0/2900146949 wait complete. 2024-02-17T12:10:00.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.902+0000 7f7303d9c740 20 client.28082 trim_cache size 0 max 16384 2024-02-17T12:10:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.931+0000 7f7303d9c740 1 Processor -- start 2024-02-17T12:10:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.931+0000 7f7303d9c740 1 -- start start 2024-02-17T12:10:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.931+0000 7f7303d9c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8f380 0x561c33a92b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.931+0000 7f7303d9c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a8fd30 0x561c33a930c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.931+0000 7f7303d9c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a988b0 0x561c33a95880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.931+0000 7f7303d9c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c339d0f40 con 0x561c33a988b0 2024-02-17T12:10:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.931+0000 7f7303d9c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c33a8a640 con 0x561c33a8fd30 2024-02-17T12:10:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.931+0000 7f7303d9c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c33aa5850 con 0x561c33a8f380 2024-02-17T12:10:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.931+0000 7f72fb59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a988b0 0x561c33a95880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.931+0000 7f72fb59e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a988b0 0x561c33a95880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41348/0 (socket says 172.21.15.94:41348) 2024-02-17T12:10:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.931+0000 7f72fb59e640 1 -- 172.21.15.94:0/2745462641 learned_addr learned my addr 172.21.15.94:0/2745462641 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.931+0000 7f72fa59c640 1 --2- 172.21.15.94:0/2745462641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a8fd30 0x561c33a930c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.932+0000 7f72fb59e640 1 -- 172.21.15.94:0/2745462641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8f380 msgr2=0x561c33a92b80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:10:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.932+0000 7f72fb59e640 1 --2- 172.21.15.94:0/2745462641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8f380 0x561c33a92b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.932+0000 7f72fb59e640 1 -- 172.21.15.94:0/2745462641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a8fd30 msgr2=0x561c33a930c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.932+0000 7f72fb59e640 1 --2- 172.21.15.94:0/2745462641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a8fd30 0x561c33a930c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.932+0000 7f72fb59e640 1 -- 172.21.15.94:0/2745462641 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c33991c00 con 0x561c33a988b0 2024-02-17T12:10:00.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.932+0000 7f72fb59e640 1 --2- 172.21.15.94:0/2745462641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a988b0 0x561c33a95880 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009b70 tx=0x7f72f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806255d9ee5d6c9a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:00.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.932+0000 7f72e0ff9640 1 -- 172.21.15.94:0/2745462641 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f0004020 con 0x561c33a988b0 2024-02-17T12:10:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.932+0000 7f72e0ff9640 1 -- 172.21.15.94:0/2745462641 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f72f000c050 con 0x561c33a988b0 2024-02-17T12:10:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.932+0000 7f72e0ff9640 1 -- 172.21.15.94:0/2745462641 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f0021680 con 0x561c33a988b0 2024-02-17T12:10:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.933+0000 7f7303d9c740 1 -- 172.21.15.94:0/2745462641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a988b0 msgr2=0x561c33a95880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.933+0000 7f7303d9c740 1 --2- 172.21.15.94:0/2745462641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a988b0 0x561c33a95880 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009b70 tx=0x7f72f0017000 comp rx=0 tx=0).stop 2024-02-17T12:10:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.934+0000 7f7303d9c740 1 -- 172.21.15.94:0/2745462641 shutdown_connections 2024-02-17T12:10:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.934+0000 7f7303d9c740 1 --2- 172.21.15.94:0/2745462641 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a988b0 0x561c33a95880 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.934+0000 7f7303d9c740 1 --2- 172.21.15.94:0/2745462641 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a8fd30 0x561c33a930c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.934+0000 7f7303d9c740 1 --2- 172.21.15.94:0/2745462641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8f380 0x561c33a92b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.934+0000 7f7303d9c740 1 -- 172.21.15.94:0/2745462641 >> 172.21.15.94:0/2745462641 conn(0x561c33a89840 msgr2=0x561c33a941d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.934+0000 7f7303d9c740 1 -- 172.21.15.94:0/2745462641 shutdown_connections 2024-02-17T12:10:00.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.934+0000 7f7303d9c740 1 -- 172.21.15.94:0/2745462641 wait complete. 2024-02-17T12:10:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.934+0000 7f7303d9c740 1 -- start start 2024-02-17T12:10:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.934+0000 7f7303d9c740 1 Processor -- start 2024-02-17T12:10:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f7303d9c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8f380 0x561c33a9eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f7303d9c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a8fd30 0x561c33a9efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f7303d9c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a988b0 0x561c33af9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f7303d9c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c33af98b0 con 0x561c33a8fd30 2024-02-17T12:10:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f72fa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a8fd30 0x561c33a9efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f7303d9c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c33af99f0 con 0x561c33a988b0 2024-02-17T12:10:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f7303d9c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c33af9bb0 con 0x561c33a8f380 2024-02-17T12:10:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f72fa59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a8fd30 0x561c33a9efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41364/0 (socket says 172.21.15.94:41364) 2024-02-17T12:10:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f72fa59c640 1 -- 172.21.15.94:0/722017806 learned_addr learned my addr 172.21.15.94:0/722017806 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f7303d9c740 10 client.0 _ll_register_callbacks cb 0x7f72f4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f72fa59c640 1 -- 172.21.15.94:0/722017806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8f380 msgr2=0x561c33a9eaa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:10:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f72fa59c640 1 --2- 172.21.15.94:0/722017806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8f380 0x561c33a9eaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f72fa59c640 1 -- 172.21.15.94:0/722017806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a988b0 msgr2=0x561c33af9370 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:10:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f72fa59c640 1 --2- 172.21.15.94:0/722017806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a988b0 0x561c33af9370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.935+0000 7f72fa59c640 1 -- 172.21.15.94:0/722017806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f00099b0 con 0x561c33a8fd30 2024-02-17T12:10:00.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.936+0000 7f72fa59c640 1 --2- 172.21.15.94:0/722017806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a8fd30 0x561c33a9efe0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f72f40eeb60 tx=0x7f72f402c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd694ddac34ab3d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.936+0000 7f72deff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.936+0000 7f72deff5640 1 -- 172.21.15.94:0/722017806 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f40c1a60 con 0x561c33a8fd30 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.936+0000 7f72deff5640 1 -- 172.21.15.94:0/722017806 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f72f40b9ff0 con 0x561c33a8fd30 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.936+0000 7f7303d9c740 1 -- 172.21.15.94:0/722017806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c33a1c900 con 0x561c33a8fd30 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.936+0000 7f72deff5640 1 -- 172.21.15.94:0/722017806 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72f4004290 con 0x561c33a8fd30 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.936+0000 7f7303d9c740 10 client.18627 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.936+0000 7f7303d9c740 1 -- 172.21.15.94:0/722017806 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561c339b7910 con 0x561c33a8fd30 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.936+0000 7f72deff5640 1 -- 172.21.15.94:0/722017806 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f72f40043f0 con 0x561c33a8fd30 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.937+0000 7f72deff5640 1 -- 172.21.15.94:0/722017806 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f72f40c5c20 con 0x561c33a8fd30 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.937+0000 7f7303d9c740 10 client.18627 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.937+0000 7f7303d9c740 10 client.18627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.937+0000 7f7303d9c740 20 client.18627 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.937+0000 7f7303d9c740 1 --2- 172.21.15.94:0/722017806 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561c33b01130 0x561c33b03520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:00.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.937+0000 7f7303d9c740 4 client.18627 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:10:00.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.937+0000 7f7303d9c740 1 -- 172.21.15.94:0/722017806 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561c33a1c900 con 0x561c33b01130 2024-02-17T12:10:00.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.937+0000 7f72fad9d640 1 --2- 172.21.15.94:0/722017806 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561c33b01130 0x561c33b03520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:00.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.938+0000 7f72fad9d640 1 --2- 172.21.15.94:0/722017806 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561c33b01130 0x561c33b03520 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9825fca9eb486ec0 server_cookie=6da58c1c688ece74 in_seq=0 out_seq=0 2024-02-17T12:10:00.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.938+0000 7f72deff5640 10 client.18627 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:00.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.940+0000 7f72deff5640 1 -- 172.21.15.94:0/722017806 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561c33a1c900 con 0x561c33b01130 2024-02-17T12:10:00.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.940+0000 7f72deff5640 10 client.18627 handle_command_reply: tid=0 2024-02-17T12:10:00.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.940+0000 7f7303d9c740 1 client.18627 shutdown 2024-02-17T12:10:00.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.941+0000 7f7303d9c740 1 -- 172.21.15.94:0/722017806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a8fd30 msgr2=0x561c33a9efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:00.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.941+0000 7f7303d9c740 1 --2- 172.21.15.94:0/722017806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a8fd30 0x561c33a9efe0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f72f40eeb60 tx=0x7f72f402c140 comp rx=0 tx=0).stop 2024-02-17T12:10:00.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.941+0000 7f7303d9c740 1 -- 172.21.15.94:0/722017806 shutdown_connections 2024-02-17T12:10:00.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.941+0000 7f7303d9c740 1 --2- 172.21.15.94:0/722017806 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561c33b01130 0x561c33b03520 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.941+0000 7f72deff5640 0 client.18627 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:00.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.941+0000 7f7303d9c740 1 --2- 172.21.15.94:0/722017806 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561c33a988b0 0x561c33af9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.941+0000 7f7303d9c740 1 --2- 172.21.15.94:0/722017806 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561c33a8fd30 0x561c33a9efe0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.941+0000 7f7303d9c740 1 --2- 172.21.15.94:0/722017806 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561c33a8f380 0x561c33a9eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.941+0000 7f7303d9c740 1 -- 172.21.15.94:0/722017806 >> 172.21.15.94:0/722017806 conn(0x561c33a89840 msgr2=0x561c33a888e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:00.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.941+0000 7f7303d9c740 1 -- 172.21.15.94:0/722017806 shutdown_connections 2024-02-17T12:10:00.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.941+0000 7f7303d9c740 1 -- 172.21.15.94:0/722017806 wait complete. 2024-02-17T12:10:00.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.941+0000 7f7303d9c740 20 client.18627 trim_cache size 0 max 16384 2024-02-17T12:10:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:10:00.945 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:10:00.945 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "116a2d58-e860-4d34-a89f-34003d2756dc", 2024-02-17T12:10:00.945 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:10:00.945 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:10:00.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.943+0000 7f73010f4640 1 -- 172.21.15.94:0/1494665964 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f72e40a23c0 msgr2=0x7f72e40a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:00.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.943+0000 7f73010f4640 1 --2- 172.21.15.94:0/1494665964 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f72e40a23c0 0x7f72e40a4880 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f72f00143a0 tx=0x7f72f0009660 comp rx=0 tx=0).stop 2024-02-17T12:10:00.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.943+0000 7f73010f4640 1 -- 172.21.15.94:0/1494665964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72fc10f9a0 msgr2=0x7f72fc114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.943+0000 7f73010f4640 1 --2- 172.21.15.94:0/1494665964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72fc10f9a0 0x7f72fc114060 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f72f402c750 tx=0x7f72f4032550 comp rx=0 tx=0).stop 2024-02-17T12:10:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.943+0000 7f73010f4640 1 -- 172.21.15.94:0/1494665964 shutdown_connections 2024-02-17T12:10:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.943+0000 7f73010f4640 1 --2- 172.21.15.94:0/1494665964 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f72e40a23c0 0x7f72e40a4880 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.943+0000 7f73010f4640 1 --2- 172.21.15.94:0/1494665964 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f72fc11a960 0x7f72fc1145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.943+0000 7f73010f4640 1 --2- 172.21.15.94:0/1494665964 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f72fc10f9a0 0x7f72fc114060 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.943+0000 7f73010f4640 1 --2- 172.21.15.94:0/1494665964 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f72fc10d9d0 0x7f72fc119fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.943+0000 7f73010f4640 1 -- 172.21.15.94:0/1494665964 >> 172.21.15.94:0/1494665964 conn(0x7f72fc06c260 msgr2=0x7f72fc11c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.943+0000 7f73010f4640 1 -- 172.21.15.94:0/1494665964 shutdown_connections 2024-02-17T12:10:00.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:00.943+0000 7f73010f4640 1 -- 172.21.15.94:0/1494665964 wait complete. 2024-02-17T12:10:00.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:10:00.959 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:10:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.111+0000 7f4d56c95640 1 -- 172.21.15.94:0/2860060566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d50101d40 msgr2=0x7f4d50105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.111+0000 7f4d56c95640 1 --2- 172.21.15.94:0/2860060566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d50101d40 0x7f4d50105d80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009890 tx=0x7f4d3c02bb70 comp rx=0 tx=0).stop 2024-02-17T12:10:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.111+0000 7f4d56c95640 1 -- 172.21.15.94:0/2860060566 shutdown_connections 2024-02-17T12:10:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.111+0000 7f4d56c95640 1 --2- 172.21.15.94:0/2860060566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d50106460 0x7f4d5010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.111+0000 7f4d56c95640 1 --2- 172.21.15.94:0/2860060566 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d50101d40 0x7f4d50105d80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.111+0000 7f4d56c95640 1 --2- 172.21.15.94:0/2860060566 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d50101390 0x7f4d50101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.111+0000 7f4d56c95640 1 -- 172.21.15.94:0/2860060566 >> 172.21.15.94:0/2860060566 conn(0x7f4d500fcc50 msgr2=0x7f4d500ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:01.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.111+0000 7f4d56c95640 1 -- 172.21.15.94:0/2860060566 shutdown_connections 2024-02-17T12:10:01.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.111+0000 7f4d56c95640 1 -- 172.21.15.94:0/2860060566 wait complete. 2024-02-17T12:10:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.112+0000 7f4d56c95640 1 Processor -- start 2024-02-17T12:10:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.112+0000 7f4d56c95640 1 -- start start 2024-02-17T12:10:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.112+0000 7f4d56c95640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d50101390 0x7f4d5019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.112+0000 7f4d56c95640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d50106460 0x7f4d5019d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.112+0000 7f4d56c95640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d50197620 0x7f4d50197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.113+0000 7f4d56c95640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d50198010 con 0x7f4d50197620 2024-02-17T12:10:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.113+0000 7f4d56c95640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d50198180 con 0x7f4d50101390 2024-02-17T12:10:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.113+0000 7f4d56c95640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d50198340 con 0x7f4d50106460 2024-02-17T12:10:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.113+0000 7f4d54a31640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d50101390 0x7f4d5019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.113+0000 7f4d54a31640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d50101390 0x7f4d5019d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:44526/0 (socket says 172.21.15.94:44526) 2024-02-17T12:10:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.113+0000 7f4d54a31640 1 -- 172.21.15.94:0/291326058 learned_addr learned my addr 172.21.15.94:0/291326058 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:01.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.113+0000 7f4d4ffff640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d50106460 0x7f4d5019d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.113+0000 7f4d55232640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d50197620 0x7f4d50197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.113+0000 7f4d54a31640 1 -- 172.21.15.94:0/291326058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d50106460 msgr2=0x7f4d5019d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:01.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.113+0000 7f4d54a31640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d50106460 0x7f4d5019d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.113+0000 7f4d54a31640 1 -- 172.21.15.94:0/291326058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d50197620 msgr2=0x7f4d50197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:01.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.113+0000 7f4d54a31640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d50197620 0x7f4d50197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.113+0000 7f4d54a31640 1 -- 172.21.15.94:0/291326058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d44009560 con 0x7f4d50101390 2024-02-17T12:10:01.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.114+0000 7f4d54a31640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d50101390 0x7f4d5019d470 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400e9c0 tx=0x7f4d4400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:01.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.114+0000 7f4d4dffb640 1 -- 172.21.15.94:0/291326058 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4400cde0 con 0x7f4d50101390 2024-02-17T12:10:01.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.114+0000 7f4d4dffb640 1 -- 172.21.15.94:0/291326058 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d440035f0 con 0x7f4d50101390 2024-02-17T12:10:01.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.114+0000 7f4d4dffb640 1 -- 172.21.15.94:0/291326058 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44010670 con 0x7f4d50101390 2024-02-17T12:10:01.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.114+0000 7f4d56c95640 1 -- 172.21.15.94:0/291326058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c009490 con 0x7f4d50101390 2024-02-17T12:10:01.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.115+0000 7f4d56c95640 1 -- 172.21.15.94:0/291326058 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d500736b0 con 0x7f4d50101390 2024-02-17T12:10:01.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.117+0000 7f4d4dffb640 1 -- 172.21.15.94:0/291326058 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4d44003ad0 con 0x7f4d50101390 2024-02-17T12:10:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.118+0000 7f4d5986a740 1 Processor -- start 2024-02-17T12:10:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.118+0000 7f4d5986a740 1 -- start start 2024-02-17T12:10:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.118+0000 7f4d4dffb640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4d400a66c0 0x7f4d400a8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.118+0000 7f4d5986a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.118+0000 7f4d5986a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497cfbd0 0x560e497d8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.118+0000 7f4d5986a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497d87b0 0x560e497daba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.118+0000 7f4d5986a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4973e280 con 0x560e497cf2b0 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.118+0000 7f4d4dffb640 1 -- 172.21.15.94:0/291326058 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4d44014070 con 0x7f4d50101390 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.118+0000 7f4d5986a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4973b7b0 con 0x560e497cfbd0 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.118+0000 7f4d5986a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e49736f90 con 0x560e497d87b0 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d55232640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4d400a66c0 0x7f4d400a8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d54a31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d54a31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41394/0 (socket says 172.21.15.94:41394) 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d54a31640 1 -- 172.21.15.94:0/1801325055 learned_addr learned my addr 172.21.15.94:0/1801325055 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d4ffff640 1 --2- 172.21.15.94:0/1801325055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497cfbd0 0x560e497d8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d55232640 1 --2- 172.21.15.94:0/1801325055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497d87b0 0x560e497daba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d54a31640 1 -- 172.21.15.94:0/1801325055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497d87b0 msgr2=0x560e497daba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d54a31640 1 --2- 172.21.15.94:0/1801325055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497d87b0 0x560e497daba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d54a31640 1 -- 172.21.15.94:0/1801325055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497cfbd0 msgr2=0x560e497d8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d54a31640 1 --2- 172.21.15.94:0/1801325055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497cfbd0 0x560e497d8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d54a31640 1 -- 172.21.15.94:0/1801325055 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e496f5570 con 0x560e497cf2b0 2024-02-17T12:10:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d55232640 1 --2- 172.21.15.94:0/1801325055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497d87b0 0x560e497daba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:10:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d54a31640 1 --2- 172.21.15.94:0/1801325055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497cf690 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4d440a30e0 tx=0x7f4d44003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3108f8a6c3d60adf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d55232640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4d400a66c0 0x7f4d400a8b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x560e497cab70 tx=0x7f4d48009a90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d3affd640 1 -- 172.21.15.94:0/1801325055 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4409efc0 con 0x560e497cf2b0 2024-02-17T12:10:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.119+0000 7f4d3affd640 1 -- 172.21.15.94:0/1801325055 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d440a3610 con 0x560e497cf2b0 2024-02-17T12:10:01.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.120+0000 7f4d3affd640 1 -- 172.21.15.94:0/1801325055 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4409a520 con 0x560e497cf2b0 2024-02-17T12:10:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.120+0000 7f4d5986a740 1 -- 172.21.15.94:0/1801325055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 msgr2=0x560e497cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.120+0000 7f4d5986a740 1 --2- 172.21.15.94:0/1801325055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497cf690 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f4d440a30e0 tx=0x7f4d44003760 comp rx=0 tx=0).stop 2024-02-17T12:10:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.120+0000 7f4d5986a740 1 -- 172.21.15.94:0/1801325055 shutdown_connections 2024-02-17T12:10:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.120+0000 7f4d5986a740 1 --2- 172.21.15.94:0/1801325055 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497d87b0 0x560e497daba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.120+0000 7f4d5986a740 1 --2- 172.21.15.94:0/1801325055 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497cfbd0 0x560e497d8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.120+0000 7f4d5986a740 1 --2- 172.21.15.94:0/1801325055 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497cf690 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.120+0000 7f4d5986a740 1 -- 172.21.15.94:0/1801325055 >> 172.21.15.94:0/1801325055 conn(0x560e497c9990 msgr2=0x560e497c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:01.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.120+0000 7f4d5986a740 1 -- 172.21.15.94:0/1801325055 shutdown_connections 2024-02-17T12:10:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.120+0000 7f4d5986a740 1 -- 172.21.15.94:0/1801325055 wait complete. 2024-02-17T12:10:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.121+0000 7f4d5986a740 1 -- start start 2024-02-17T12:10:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.121+0000 7f4d5986a740 1 Processor -- start 2024-02-17T12:10:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.121+0000 7f4d5986a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497d5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.121+0000 7f4d5986a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 0x560e497d5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.121+0000 7f4d5986a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497d87b0 0x560e497d6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.121+0000 7f4d5986a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4940a430 con 0x560e497cf2b0 2024-02-17T12:10:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.121+0000 7f4d5986a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e493c6e00 con 0x560e497d87b0 2024-02-17T12:10:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.121+0000 7f4d5986a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e496bbfd0 con 0x560e497cfbd0 2024-02-17T12:10:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.121+0000 7f4d54a31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497d5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.122+0000 7f4d54a31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497d5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41408/0 (socket says 172.21.15.94:41408) 2024-02-17T12:10:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.122+0000 7f4d54a31640 1 -- 172.21.15.94:0/11183509 learned_addr learned my addr 172.21.15.94:0/11183509 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:01.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.122+0000 7f4d5986a740 10 client.0 _ll_register_callbacks cb 0x560e49711fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.122+0000 7f4d4ffff640 1 --2- 172.21.15.94:0/11183509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 0x560e497d5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.122+0000 7f4d55232640 1 --2- 172.21.15.94:0/11183509 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497d87b0 0x560e497d6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.122+0000 7f4d54a31640 1 -- 172.21.15.94:0/11183509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 msgr2=0x560e497d5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.122+0000 7f4d54a31640 1 --2- 172.21.15.94:0/11183509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 0x560e497d5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.122+0000 7f4d54a31640 1 -- 172.21.15.94:0/11183509 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497d87b0 msgr2=0x560e497d6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.122+0000 7f4d54a31640 1 --2- 172.21.15.94:0/11183509 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497d87b0 0x560e497d6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.122+0000 7f4d54a31640 1 -- 172.21.15.94:0/11183509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d500736b0 con 0x560e497cf2b0 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.122+0000 7f4d54a31640 1 --2- 172.21.15.94:0/11183509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497d5610 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f4d440ce5e0 tx=0x7f4d44003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16de9db08736d7e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.122+0000 7f4d2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.123+0000 7f4d2ffff640 1 -- 172.21.15.94:0/11183509 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44099170 con 0x560e497cf2b0 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.123+0000 7f4d2ffff640 1 -- 172.21.15.94:0/11183509 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4d440b3ba0 con 0x560e497cf2b0 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.123+0000 7f4d5986a740 1 -- 172.21.15.94:0/11183509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e494367e0 con 0x560e497cf2b0 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.123+0000 7f4d2ffff640 1 -- 172.21.15.94:0/11183509 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d440b3d60 con 0x560e497cf2b0 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.123+0000 7f4d5986a740 10 client.18639 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:01.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.123+0000 7f4d5986a740 1 -- 172.21.15.94:0/11183509 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560e49735610 con 0x560e497cf2b0 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.123+0000 7f4d2ffff640 1 -- 172.21.15.94:0/11183509 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d440a9080 con 0x560e497cf2b0 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.124+0000 7f4d2ffff640 1 -- 172.21.15.94:0/11183509 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4d4401f060 con 0x560e497cf2b0 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.124+0000 7f4d5986a740 10 client.18639 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.124+0000 7f4d5986a740 10 client.18639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.124+0000 7f4d5986a740 20 client.18639 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.124+0000 7f4d5986a740 1 --2- 172.21.15.94:0/11183509 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e497e85a0 0x560e497ea990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.124+0000 7f4d5986a740 4 client.18639 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.124+0000 7f4d5986a740 1 -- 172.21.15.94:0/11183509 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e49473720 con 0x560e497e85a0 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.124+0000 7f4d4ffff640 1 --2- 172.21.15.94:0/11183509 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e497e85a0 0x560e497ea990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.124+0000 7f4d2ffff640 10 client.18639 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.124+0000 7f4d4ffff640 1 --2- 172.21.15.94:0/11183509 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e497e85a0 0x560e497ea990 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88708c9e087fd72b server_cookie=887b1a18f1f2adc4 in_seq=0 out_seq=0 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d2ffff640 1 -- 172.21.15.94:0/11183509 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560e49473720 con 0x560e497e85a0 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d2ffff640 10 client.18639 handle_command_reply: tid=0 2024-02-17T12:10:01.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d5986a740 1 client.18639 shutdown 2024-02-17T12:10:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d5986a740 1 -- 172.21.15.94:0/11183509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 msgr2=0x560e497d5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d5986a740 1 --2- 172.21.15.94:0/11183509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497d5610 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f4d440ce5e0 tx=0x7f4d44003ed0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d5986a740 1 -- 172.21.15.94:0/11183509 shutdown_connections 2024-02-17T12:10:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d5986a740 1 --2- 172.21.15.94:0/11183509 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e497e85a0 0x560e497ea990 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d5986a740 1 --2- 172.21.15.94:0/11183509 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497d87b0 0x560e497d6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d5986a740 1 --2- 172.21.15.94:0/11183509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 0x560e497d5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d5986a740 1 --2- 172.21.15.94:0/11183509 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497d5610 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d5986a740 1 -- 172.21.15.94:0/11183509 >> 172.21.15.94:0/11183509 conn(0x560e497c9990 msgr2=0x560e497cb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d2ffff640 0 client.18639 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d5986a740 1 -- 172.21.15.94:0/11183509 shutdown_connections 2024-02-17T12:10:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d5986a740 1 -- 172.21.15.94:0/11183509 wait complete. 2024-02-17T12:10:01.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.125+0000 7f4d5986a740 20 client.18639 trim_cache size 0 max 16384 2024-02-17T12:10:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.142+0000 7f4d5986a740 1 Processor -- start 2024-02-17T12:10:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.142+0000 7f4d5986a740 1 -- start start 2024-02-17T12:10:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d5986a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497d7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d5986a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 0x560e497df600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d5986a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497d87b0 0x560e497dfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d5986a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e497ca9e0 con 0x560e497cf2b0 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d5986a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e497d1240 con 0x560e497d87b0 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d5986a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e497daf20 con 0x560e497cfbd0 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d54a31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497d7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d54a31640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497d7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41416/0 (socket says 172.21.15.94:41416) 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d54a31640 1 -- 172.21.15.94:0/355646341 learned_addr learned my addr 172.21.15.94:0/355646341 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d55232640 1 --2- 172.21.15.94:0/355646341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497d87b0 0x560e497dfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d4ffff640 1 --2- 172.21.15.94:0/355646341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 0x560e497df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d54a31640 1 -- 172.21.15.94:0/355646341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 msgr2=0x560e497df600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d54a31640 1 --2- 172.21.15.94:0/355646341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 0x560e497df600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d54a31640 1 -- 172.21.15.94:0/355646341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497d87b0 msgr2=0x560e497dfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d54a31640 1 --2- 172.21.15.94:0/355646341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497d87b0 0x560e497dfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d54a31640 1 -- 172.21.15.94:0/355646341 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e496d1ac0 con 0x560e497cf2b0 2024-02-17T12:10:01.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d54a31640 1 --2- 172.21.15.94:0/355646341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497d7d40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f4d4409ab70 tx=0x7f4d440b94f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1549c4ebe71807b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d3a7fc640 1 -- 172.21.15.94:0/355646341 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4409b570 con 0x560e497cf2b0 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d3a7fc640 1 -- 172.21.15.94:0/355646341 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x560e494367e0 con 0x560e497cf2b0 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.143+0000 7f4d3a7fc640 1 -- 172.21.15.94:0/355646341 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44084030 con 0x560e497cf2b0 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.144+0000 7f4d5986a740 1 -- 172.21.15.94:0/355646341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 msgr2=0x560e497d7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.144+0000 7f4d5986a740 1 --2- 172.21.15.94:0/355646341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497d7d40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f4d4409ab70 tx=0x7f4d440b94f0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.144+0000 7f4d5986a740 1 -- 172.21.15.94:0/355646341 shutdown_connections 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.144+0000 7f4d5986a740 1 --2- 172.21.15.94:0/355646341 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497d87b0 0x560e497dfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.144+0000 7f4d5986a740 1 --2- 172.21.15.94:0/355646341 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 0x560e497df600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.144+0000 7f4d5986a740 1 --2- 172.21.15.94:0/355646341 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497cf2b0 0x560e497d7d40 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.144+0000 7f4d5986a740 1 -- 172.21.15.94:0/355646341 >> 172.21.15.94:0/355646341 conn(0x560e497c9990 msgr2=0x560e497ef280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.144+0000 7f4d5986a740 1 -- 172.21.15.94:0/355646341 shutdown_connections 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.144+0000 7f4d5986a740 1 -- 172.21.15.94:0/355646341 wait complete. 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d5986a740 1 -- start start 2024-02-17T12:10:01.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d5986a740 1 Processor -- start 2024-02-17T12:10:01.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d5986a740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497cf2b0 0x560e4983f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d5986a740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 0x560e4983fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d5986a740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497d87b0 0x560e4983b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d5986a740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4983b8a0 con 0x560e497d87b0 2024-02-17T12:10:01.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d5986a740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4983ba10 con 0x560e497cf2b0 2024-02-17T12:10:01.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d5986a740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e4983bbd0 con 0x560e497cfbd0 2024-02-17T12:10:01.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d5986a740 10 client.0 _ll_register_callbacks cb 0x7f4d440969f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:01.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d55232640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497d87b0 0x560e4983b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d55232640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497d87b0 0x560e4983b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41430/0 (socket says 172.21.15.94:41430) 2024-02-17T12:10:01.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d55232640 1 -- 172.21.15.94:0/3891183877 learned_addr learned my addr 172.21.15.94:0/3891183877 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:01.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d54a31640 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497cf2b0 0x560e4983f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.145+0000 7f4d4ffff640 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 0x560e4983fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d54a31640 1 -- 172.21.15.94:0/3891183877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 msgr2=0x560e4983fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:01.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d54a31640 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 0x560e4983fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d54a31640 1 -- 172.21.15.94:0/3891183877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497d87b0 msgr2=0x560e4983b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:01.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d54a31640 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497d87b0 0x560e4983b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d54a31640 1 -- 172.21.15.94:0/3891183877 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560e49735610 con 0x560e497cf2b0 2024-02-17T12:10:01.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d55232640 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497d87b0 0x560e4983b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:01.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d54a31640 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497cf2b0 0x560e4983f7e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4d4409ab70 tx=0x7f4d44003760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b38784aa3e125e21 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:01.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:10:01.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d2ffff640 1 -- 172.21.15.94:0/3891183877 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4409a9b0 con 0x560e497cf2b0 2024-02-17T12:10:01.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d2ffff640 1 -- 172.21.15.94:0/3891183877 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4d44099170 con 0x560e497cf2b0 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d5986a740 1 -- 172.21.15.94:0/3891183877 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d440a3610 con 0x560e497cf2b0 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d2ffff640 1 -- 172.21.15.94:0/3891183877 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d4409b570 con 0x560e497cf2b0 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d5986a740 10 client.28103 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d5986a740 1 -- 172.21.15.94:0/3891183877 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560e496f76a0 con 0x560e497cf2b0 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.146+0000 7f4d2ffff640 1 -- 172.21.15.94:0/3891183877 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d44089080 con 0x560e497cf2b0 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.147+0000 7f4d2ffff640 1 -- 172.21.15.94:0/3891183877 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4d44085b30 con 0x560e497cf2b0 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.147+0000 7f4d5986a740 10 client.28103 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.147+0000 7f4d5986a740 10 client.28103 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.147+0000 7f4d5986a740 20 client.28103 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.147+0000 7f4d5986a740 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e498482b0 0x560e4984a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.147+0000 7f4d5986a740 4 client.28103 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.147+0000 7f4d5986a740 1 -- 172.21.15.94:0/3891183877 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560e494c2890 con 0x560e498482b0 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.147+0000 7f4d4ffff640 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e498482b0 0x560e4984a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.147+0000 7f4d2ffff640 10 client.28103 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:01.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.147+0000 7f4d4ffff640 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e498482b0 0x560e4984a6a0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8715e94542b6130d server_cookie=6763f40788c612f7 in_seq=0 out_seq=0 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.147+0000 7f4d2ffff640 1 -- 172.21.15.94:0/3891183877 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560e494c2890 con 0x560e498482b0 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.147+0000 7f4d2ffff640 10 client.28103 handle_command_reply: tid=0 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.147+0000 7f4d5986a740 1 client.28103 shutdown 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d5986a740 1 -- 172.21.15.94:0/3891183877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497cf2b0 msgr2=0x560e4983f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d5986a740 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497cf2b0 0x560e4983f7e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4d4409ab70 tx=0x7f4d44003760 comp rx=0 tx=0).stop 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d5986a740 1 -- 172.21.15.94:0/3891183877 shutdown_connections 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d5986a740 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560e498482b0 0x560e4984a6a0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d5986a740 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x560e497d87b0 0x560e4983b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d2ffff640 0 client.28103 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d5986a740 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e497cfbd0 0x560e4983fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d5986a740 1 --2- 172.21.15.94:0/3891183877 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x560e497cf2b0 0x560e4983f7e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d5986a740 1 -- 172.21.15.94:0/3891183877 >> 172.21.15.94:0/3891183877 conn(0x560e497c9990 msgr2=0x560e497c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d5986a740 1 -- 172.21.15.94:0/3891183877 shutdown_connections 2024-02-17T12:10:01.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d5986a740 1 -- 172.21.15.94:0/3891183877 wait complete. 2024-02-17T12:10:01.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d5986a740 20 client.28103 trim_cache size 0 max 16384 2024-02-17T12:10:01.156 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:10:01.156 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:10:01.156 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:10:01.156 INFO:teuthology.orchestra.run.smithi094.stdout: "116a2d58-e860-4d34-a89f-34003d2756dc": { 2024-02-17T12:10:01.156 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:10:01.156 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "116a2d58-e860-4d34-a89f-34003d2756dc", 2024-02-17T12:10:01.156 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:10:01.156 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:10:01.157 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:10:01.157 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:10:01.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d56c95640 1 -- 172.21.15.94:0/291326058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4d400a66c0 msgr2=0x7f4d400a8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:01.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d56c95640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4d400a66c0 0x7f4d400a8b80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x560e497cab70 tx=0x7f4d48009a90 comp rx=0 tx=0).stop 2024-02-17T12:10:01.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d56c95640 1 -- 172.21.15.94:0/291326058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d50101390 msgr2=0x7f4d5019d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:01.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d56c95640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d50101390 0x7f4d5019d470 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400e9c0 tx=0x7f4d4400ee80 comp rx=0 tx=0).stop 2024-02-17T12:10:01.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d56c95640 1 -- 172.21.15.94:0/291326058 shutdown_connections 2024-02-17T12:10:01.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d56c95640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4d400a66c0 0x7f4d400a8b80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d56c95640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d50197620 0x7f4d50197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d56c95640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d50106460 0x7f4d5019d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.148+0000 7f4d56c95640 1 --2- 172.21.15.94:0/291326058 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d50101390 0x7f4d5019d470 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:01.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.149+0000 7f4d56c95640 1 -- 172.21.15.94:0/291326058 >> 172.21.15.94:0/291326058 conn(0x7f4d500fcc50 msgr2=0x7f4d500fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:01.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.149+0000 7f4d56c95640 1 -- 172.21.15.94:0/291326058 shutdown_connections 2024-02-17T12:10:01.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:01.149+0000 7f4d56c95640 1 -- 172.21.15.94:0/291326058 wait complete. 2024-02-17T12:10:01.163 INFO:tasks.cephfs.filesystem:scrub status for tag:116a2d58-e860-4d34-a89f-34003d2756dc - {'path': '/', 'tag': '116a2d58-e860-4d34-a89f-34003d2756dc', 'options': 'recursive,force'} 2024-02-17T12:10:01.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:01 smithi189 ceph-mon[36588]: pgmap v370: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.8 MiB/s wr, 560 op/s 2024-02-17T12:10:01.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:01 smithi125 ceph-mon[36819]: pgmap v370: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.8 MiB/s wr, 560 op/s 2024-02-17T12:10:01.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:01 smithi094 ceph-mon[32499]: pgmap v370: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.8 MiB/s wr, 560 op/s 2024-02-17T12:10:02.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:02 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:10:02.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:02 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:10:02.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:02 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:10:02.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:02 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:10:02.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:02 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:10:02.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:02 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:10:02.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:02 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:10:02.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:02 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:10:02.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:02 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:10:02.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:02 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:10:02.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:02 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:10:02.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:02 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:10:03.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:03 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:10:03.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:03 smithi189 ceph-mon[36588]: pgmap v371: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.6 MiB/s wr, 532 op/s 2024-02-17T12:10:03.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:03 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:10:03.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:03 smithi125 ceph-mon[36819]: pgmap v371: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.6 MiB/s wr, 532 op/s 2024-02-17T12:10:03.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:03 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:10:03.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:03 smithi094 ceph-mon[32499]: pgmap v371: 97 pgs: 97 active+clean; 5.8 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 5.6 MiB/s wr, 532 op/s 2024-02-17T12:10:05.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 705.0 s, 1180.2 tps, lat 83.924 ms stddev 147.588 2024-02-17T12:10:05.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:05 smithi189 ceph-mon[36588]: pgmap v372: 97 pgs: 97 active+clean; 5.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 20 MiB/s wr, 1.93k op/s 2024-02-17T12:10:05.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:05 smithi125 ceph-mon[36819]: pgmap v372: 97 pgs: 97 active+clean; 5.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 20 MiB/s wr, 1.93k op/s 2024-02-17T12:10:05.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:05 smithi094 ceph-mon[32499]: pgmap v372: 97 pgs: 97 active+clean; 5.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 20 MiB/s wr, 1.93k op/s 2024-02-17T12:10:07.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:07 smithi189 ceph-mon[36588]: pgmap v373: 97 pgs: 97 active+clean; 5.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 18 MiB/s wr, 1.67k op/s 2024-02-17T12:10:07.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:07 smithi125 ceph-mon[36819]: pgmap v373: 97 pgs: 97 active+clean; 5.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 18 MiB/s wr, 1.67k op/s 2024-02-17T12:10:07.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:07 smithi094 ceph-mon[32499]: pgmap v373: 97 pgs: 97 active+clean; 5.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 18 MiB/s wr, 1.67k op/s 2024-02-17T12:10:08.133 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.125+0000 7f76f6a9d640 1 -- 172.21.15.94:0/1772122022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f0103b90 msgr2=0x7f76f0103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:08.134 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.125+0000 7f76f6a9d640 1 --2- 172.21.15.94:0/1772122022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f0103b90 0x7f76f0103ff0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f76d8009890 tx=0x7f76d802bc10 comp rx=0 tx=0).stop 2024-02-17T12:10:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.135+0000 7f76f6a9d640 1 -- 172.21.15.94:0/1772122022 shutdown_connections 2024-02-17T12:10:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.135+0000 7f76f6a9d640 1 --2- 172.21.15.94:0/1772122022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f0103b90 0x7f76f0103ff0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.135+0000 7f76f6a9d640 1 --2- 172.21.15.94:0/1772122022 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f0068d40 0x7f76f0069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.135+0000 7f76f6a9d640 1 --2- 172.21.15.94:0/1772122022 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0106140 0x7f76f0068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.138 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.135+0000 7f76f6a9d640 1 -- 172.21.15.94:0/1772122022 >> 172.21.15.94:0/1772122022 conn(0x7f76f0076ed0 msgr2=0x7f76f00772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:08.139 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.137+0000 7f76f6a9d640 1 -- 172.21.15.94:0/1772122022 shutdown_connections 2024-02-17T12:10:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.141+0000 7f76f6a9d640 1 -- 172.21.15.94:0/1772122022 wait complete. 2024-02-17T12:10:08.143 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.141+0000 7f76f6a9d640 1 Processor -- start 2024-02-17T12:10:08.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.141+0000 7f76f6a9d640 1 -- start start 2024-02-17T12:10:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.149+0000 7f76f6a9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f0068d40 0x7f76f0071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.149+0000 7f76f6a9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0103b90 0x7f76f0071e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.149+0000 7f76f6a9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f0106140 0x7f76f010a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.153 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.149+0000 7f76f6a9d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f010afa0 con 0x7f76f0106140 2024-02-17T12:10:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.149+0000 7f76f6a9d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f010b110 con 0x7f76f0103b90 2024-02-17T12:10:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.149+0000 7f76f6a9d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f010b2d0 con 0x7f76f0068d40 2024-02-17T12:10:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.150+0000 7f76f4839640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f0068d40 0x7f76f0071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.150+0000 7f76f4839640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f0068d40 0x7f76f0071930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60290/0 (socket says 172.21.15.94:60290) 2024-02-17T12:10:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.150+0000 7f76f4839640 1 -- 172.21.15.94:0/2341970984 learned_addr learned my addr 172.21.15.94:0/2341970984 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.150+0000 7f76effff640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0103b90 0x7f76f0071e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.151+0000 7f76f503a640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f0106140 0x7f76f010a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.152+0000 7f76effff640 1 -- 172.21.15.94:0/2341970984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f0068d40 msgr2=0x7f76f0071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.152+0000 7f76effff640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f0068d40 0x7f76f0071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.152+0000 7f76effff640 1 -- 172.21.15.94:0/2341970984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f0106140 msgr2=0x7f76f010a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.152+0000 7f76effff640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f0106140 0x7f76f010a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.152+0000 7f76effff640 1 -- 172.21.15.94:0/2341970984 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d8009490 con 0x7f76f0103b90 2024-02-17T12:10:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.159+0000 7f76effff640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0103b90 0x7f76f0071e70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f76e000b3f0 tx=0x7f76e000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.163+0000 7f76edffb640 1 -- 172.21.15.94:0/2341970984 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0011830 con 0x7f76f0103b90 2024-02-17T12:10:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.163+0000 7f76edffb640 1 -- 172.21.15.94:0/2341970984 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76e0011e70 con 0x7f76f0103b90 2024-02-17T12:10:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.163+0000 7f76edffb640 1 -- 172.21.15.94:0/2341970984 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e000f2e0 con 0x7f76f0103b90 2024-02-17T12:10:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.165+0000 7f76f6a9d640 1 -- 172.21.15.94:0/2341970984 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f010b5b0 con 0x7f76f0103b90 2024-02-17T12:10:08.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.165+0000 7f76f6a9d640 1 -- 172.21.15.94:0/2341970984 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f0111c70 con 0x7f76f0103b90 2024-02-17T12:10:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.175+0000 7f76f967b740 1 Processor -- start 2024-02-17T12:10:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.180+0000 7f76f967b740 1 -- start start 2024-02-17T12:10:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.180+0000 7f76f967b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 0x561a51343820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.180+0000 7f76f967b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a51343d60 0x561a513441e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.180+0000 7f76f967b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a5134d2e0 0x561a5134f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.180+0000 7f76f967b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a512b29f0 con 0x561a51344ad0 2024-02-17T12:10:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.180+0000 7f76f967b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a512aff20 con 0x561a5134d2e0 2024-02-17T12:10:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.180+0000 7f76f967b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a512ab700 con 0x561a51343d60 2024-02-17T12:10:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.181+0000 7f76f503a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 0x561a51343820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.181+0000 7f76f503a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 0x561a51343820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:44734/0 (socket says 172.21.15.94:44734) 2024-02-17T12:10:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.181+0000 7f76f503a640 1 -- 172.21.15.94:0/1363755345 learned_addr learned my addr 172.21.15.94:0/1363755345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.181+0000 7f76f4839640 1 --2- 172.21.15.94:0/1363755345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a51343d60 0x561a513441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.183+0000 7f76edffb640 1 -- 172.21.15.94:0/2341970984 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f76e0010610 con 0x7f76f0103b90 2024-02-17T12:10:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.183+0000 7f76edffb640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f76dc0a2430 0x7f76dc0a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.184+0000 7f76edffb640 1 -- 172.21.15.94:0/2341970984 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f76e00cd590 con 0x7f76f0103b90 2024-02-17T12:10:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.185+0000 7f76f4839640 1 -- 172.21.15.94:0/1363755345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a5134d2e0 msgr2=0x561a5134f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.185+0000 7f76f4839640 1 --2- 172.21.15.94:0/1363755345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a5134d2e0 0x561a5134f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.185+0000 7f76f4839640 1 -- 172.21.15.94:0/1363755345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 msgr2=0x561a51343820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.185+0000 7f76f4839640 1 --2- 172.21.15.94:0/1363755345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 0x561a51343820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.185+0000 7f76f4839640 1 -- 172.21.15.94:0/1363755345 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a512417b0 con 0x561a51343d60 2024-02-17T12:10:08.189 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.186+0000 7f76f4839640 1 --2- 172.21.15.94:0/1363755345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a51343d60 0x561a513441e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009f90 tx=0x7f76e80108a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb208474bb30f5f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:08.191 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.187+0000 7f76f503a640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f76dc0a2430 0x7f76dc0a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.190+0000 7f76e6ffd640 1 -- 172.21.15.94:0/1363755345 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e8003b70 con 0x561a51343d60 2024-02-17T12:10:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.190+0000 7f76e6ffd640 1 -- 172.21.15.94:0/1363755345 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76e8003cd0 con 0x561a51343d60 2024-02-17T12:10:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.190+0000 7f76e6ffd640 1 -- 172.21.15.94:0/1363755345 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e8004610 con 0x561a51343d60 2024-02-17T12:10:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.191+0000 7f76f967b740 1 -- 172.21.15.94:0/1363755345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a51343d60 msgr2=0x561a513441e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.191+0000 7f76f967b740 1 --2- 172.21.15.94:0/1363755345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a51343d60 0x561a513441e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009f90 tx=0x7f76e80108a0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.200+0000 7f76f503a640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f76dc0a2430 0x7f76dc0a48f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f76d8034fa0 tx=0x7f76d802bc10 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.202+0000 7f76f967b740 1 -- 172.21.15.94:0/1363755345 shutdown_connections 2024-02-17T12:10:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.202+0000 7f76f967b740 1 --2- 172.21.15.94:0/1363755345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a5134d2e0 0x561a5134f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.202+0000 7f76f967b740 1 --2- 172.21.15.94:0/1363755345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a51343d60 0x561a513441e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.202+0000 7f76f967b740 1 --2- 172.21.15.94:0/1363755345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 0x561a51343820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.202+0000 7f76f967b740 1 -- 172.21.15.94:0/1363755345 >> 172.21.15.94:0/1363755345 conn(0x561a5130b550 msgr2=0x561a5133ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.205+0000 7f76f967b740 1 -- 172.21.15.94:0/1363755345 shutdown_connections 2024-02-17T12:10:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.208+0000 7f76f967b740 1 -- 172.21.15.94:0/1363755345 wait complete. 2024-02-17T12:10:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.208+0000 7f76f967b740 1 -- start start 2024-02-17T12:10:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.208+0000 7f76f967b740 1 Processor -- start 2024-02-17T12:10:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.211+0000 7f76f967b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51343d60 0x561a5134a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.211+0000 7f76f967b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 0x561a5134a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.211+0000 7f76f967b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a5134ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.211+0000 7f76f967b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a51250bf0 con 0x561a51344ad0 2024-02-17T12:10:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.211+0000 7f76f967b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a50f7f250 con 0x561a51343d60 2024-02-17T12:10:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.211+0000 7f76f967b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a51231200 con 0x561a5134d2e0 2024-02-17T12:10:08.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.211+0000 7f76f967b740 10 client.0 _ll_register_callbacks cb 0x561a5129fd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.212+0000 7f76f4839640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51343d60 0x561a5134a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.212+0000 7f76f4839640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51343d60 0x561a5134a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:49294/0 (socket says 172.21.15.94:49294) 2024-02-17T12:10:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.212+0000 7f76f4839640 1 -- 172.21.15.94:0/1146136564 learned_addr learned my addr 172.21.15.94:0/1146136564 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:08.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.212+0000 7f76f503a640 1 --2- 172.21.15.94:0/1146136564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a5134ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.218+0000 7f76effff640 1 --2- 172.21.15.94:0/1146136564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 0x561a5134a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.219+0000 7f76f4839640 1 -- 172.21.15.94:0/1146136564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 msgr2=0x561a5134ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.219+0000 7f76f4839640 1 --2- 172.21.15.94:0/1146136564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a5134ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.219+0000 7f76f4839640 1 -- 172.21.15.94:0/1146136564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 msgr2=0x561a5134a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.219+0000 7f76f4839640 1 --2- 172.21.15.94:0/1146136564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 0x561a5134a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.219+0000 7f76f4839640 1 -- 172.21.15.94:0/1146136564 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e8009bc0 con 0x561a51343d60 2024-02-17T12:10:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.220+0000 7f76f4839640 1 --2- 172.21.15.94:0/1146136564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51343d60 0x561a5134a380 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f76e8004ca0 tx=0x7f76e8004cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e205aad9bc146959 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.220+0000 7f76cffff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:10:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.221+0000 7f76cffff640 1 -- 172.21.15.94:0/1146136564 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e8004590 con 0x561a51343d60 2024-02-17T12:10:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.221+0000 7f76f967b740 1 -- 172.21.15.94:0/1146136564 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a50fab600 con 0x561a51343d60 2024-02-17T12:10:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.221+0000 7f76cffff640 1 -- 172.21.15.94:0/1146136564 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f76e801f040 con 0x561a51343d60 2024-02-17T12:10:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.222+0000 7f76cffff640 1 -- 172.21.15.94:0/1146136564 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76e8003f80 con 0x561a51343d60 2024-02-17T12:10:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.222+0000 7f76f967b740 10 client.28112 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.222+0000 7f76f967b740 1 -- 172.21.15.94:0/1146136564 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561a512a9d80 con 0x561a51343d60 2024-02-17T12:10:08.226 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.222+0000 7f76cffff640 1 -- 172.21.15.94:0/1146136564 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561a50fab600 con 0x561a51343d60 2024-02-17T12:10:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.222+0000 7f76cffff640 1 -- 172.21.15.94:0/1146136564 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f76e80053d0 con 0x561a51343d60 2024-02-17T12:10:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.222+0000 7f76f967b740 10 client.28112 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.222+0000 7f76f967b740 10 client.28112 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.223+0000 7f76f967b740 20 client.28112 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.223+0000 7f76f967b740 1 --2- 172.21.15.94:0/1146136564 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a5135d640 0x561a5135fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.227 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.223+0000 7f76f967b740 4 client.28112 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:10:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.223+0000 7f76f967b740 1 -- 172.21.15.94:0/1146136564 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a50ffc1b0 con 0x561a5135d640 2024-02-17T12:10:08.228 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.224+0000 7f76effff640 1 --2- 172.21.15.94:0/1146136564 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a5135d640 0x561a5135fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.228+0000 7f76effff640 1 --2- 172.21.15.94:0/1146136564 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a5135d640 0x561a5135fa30 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10717cdc53b33fc2 server_cookie=ccf5e80014d072ba in_seq=0 out_seq=0 2024-02-17T12:10:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.228+0000 7f76cffff640 10 client.28112 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.231+0000 7f76cffff640 1 -- 172.21.15.94:0/1146136564 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f76e009a720 con 0x561a5135d640 2024-02-17T12:10:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.231+0000 7f76cffff640 10 client.28112 handle_command_reply: tid=0 2024-02-17T12:10:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.231+0000 7f76f967b740 1 client.28112 shutdown 2024-02-17T12:10:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.235+0000 7f76f967b740 1 -- 172.21.15.94:0/1146136564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51343d60 msgr2=0x561a5134a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:08.239 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.235+0000 7f76f967b740 1 --2- 172.21.15.94:0/1146136564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51343d60 0x561a5134a380 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f76e8004ca0 tx=0x7f76e8004cd0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.236+0000 7f76f967b740 1 -- 172.21.15.94:0/1146136564 shutdown_connections 2024-02-17T12:10:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.236+0000 7f76f967b740 1 --2- 172.21.15.94:0/1146136564 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a5135d640 0x561a5135fa30 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.236+0000 7f76f967b740 1 --2- 172.21.15.94:0/1146136564 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a5134ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.236+0000 7f76f967b740 1 --2- 172.21.15.94:0/1146136564 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 0x561a5134a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.236+0000 7f76f967b740 1 --2- 172.21.15.94:0/1146136564 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51343d60 0x561a5134a380 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.236+0000 7f76f967b740 1 -- 172.21.15.94:0/1146136564 >> 172.21.15.94:0/1146136564 conn(0x561a5130b550 msgr2=0x561a513407a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:08.240 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.236+0000 7f76cffff640 0 client.28112 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.238+0000 7f76f967b740 1 -- 172.21.15.94:0/1146136564 shutdown_connections 2024-02-17T12:10:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.242+0000 7f76f967b740 1 -- 172.21.15.94:0/1146136564 wait complete. 2024-02-17T12:10:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.242+0000 7f76f967b740 20 client.28112 trim_cache size 0 max 16384 2024-02-17T12:10:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.268+0000 7f76f967b740 1 Processor -- start 2024-02-17T12:10:08.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.269+0000 7f76f967b740 1 -- start start 2024-02-17T12:10:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.269+0000 7f76f967b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51343d60 0x561a5134cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.269+0000 7f76f967b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 0x561a5134ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.269+0000 7f76f967b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a51354100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.269+0000 7f76f967b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a51343030 con 0x561a51344ad0 2024-02-17T12:10:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.270+0000 7f76f967b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a51364c30 con 0x561a51343d60 2024-02-17T12:10:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.270+0000 7f76f503a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a51354100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.270+0000 7f76f503a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a51354100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60312/0 (socket says 172.21.15.94:60312) 2024-02-17T12:10:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.270+0000 7f76f503a640 1 -- 172.21.15.94:0/2501120736 learned_addr learned my addr 172.21.15.94:0/2501120736 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:08.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.270+0000 7f76f967b740 1 -- 172.21.15.94:0/2501120736 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a513488f0 con 0x561a5134d2e0 2024-02-17T12:10:08.273 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.271+0000 7f76f503a640 1 -- 172.21.15.94:0/2501120736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51343d60 msgr2=0x561a5134cab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:10:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.271+0000 7f76f503a640 1 --2- 172.21.15.94:0/2501120736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51343d60 0x561a5134cab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.271+0000 7f76f503a640 1 -- 172.21.15.94:0/2501120736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 msgr2=0x561a5134ac20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:10:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.271+0000 7f76f503a640 1 --2- 172.21.15.94:0/2501120736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 0x561a5134ac20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.271+0000 7f76f503a640 1 -- 172.21.15.94:0/2501120736 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a512467b0 con 0x561a5134d2e0 2024-02-17T12:10:08.274 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.272+0000 7f76f503a640 1 --2- 172.21.15.94:0/2501120736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a51354100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f76d8037040 tx=0x7f76d8034bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b02ed524e4a2eb5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.274+0000 7f76e67fc640 1 -- 172.21.15.94:0/2501120736 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8053070 con 0x561a5134d2e0 2024-02-17T12:10:08.276 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.274+0000 7f76e67fc640 1 -- 172.21.15.94:0/2501120736 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76d80051d0 con 0x561a5134d2e0 2024-02-17T12:10:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.275+0000 7f76f967b740 1 -- 172.21.15.94:0/2501120736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 msgr2=0x561a51354100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:08.277 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.275+0000 7f76f967b740 1 --2- 172.21.15.94:0/2501120736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a51354100 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f76d8037040 tx=0x7f76d8034bf0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.279+0000 7f76f967b740 1 -- 172.21.15.94:0/2501120736 shutdown_connections 2024-02-17T12:10:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.279+0000 7f76f967b740 1 --2- 172.21.15.94:0/2501120736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a51354100 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.279+0000 7f76f967b740 1 --2- 172.21.15.94:0/2501120736 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51344ad0 0x561a5134ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.279+0000 7f76f967b740 1 --2- 172.21.15.94:0/2501120736 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51343d60 0x561a5134cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.280 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.279+0000 7f76f967b740 1 -- 172.21.15.94:0/2501120736 >> 172.21.15.94:0/2501120736 conn(0x561a5130b550 msgr2=0x561a513456a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:08.284 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.283+0000 7f76f967b740 1 -- 172.21.15.94:0/2501120736 shutdown_connections 2024-02-17T12:10:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.285+0000 7f76f967b740 1 -- 172.21.15.94:0/2501120736 wait complete. 2024-02-17T12:10:08.287 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.286+0000 7f76f967b740 1 -- start start 2024-02-17T12:10:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.286+0000 7f76f967b740 1 Processor -- start 2024-02-17T12:10:08.288 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.287+0000 7f76f967b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51343d60 0x561a51347150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.287+0000 7f76f967b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51344ad0 0x561a51347690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.287+0000 7f76f967b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a51347bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.289 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.287+0000 7f76f967b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a513b3b20 con 0x561a51343d60 2024-02-17T12:10:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.287+0000 7f76effff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51344ad0 0x561a51347690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.287+0000 7f76effff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51344ad0 0x561a51347690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:49302/0 (socket says 172.21.15.94:49302) 2024-02-17T12:10:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.287+0000 7f76effff640 1 -- 172.21.15.94:0/611434533 learned_addr learned my addr 172.21.15.94:0/611434533 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.288+0000 7f76f503a640 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a51347bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.288+0000 7f76f967b740 1 -- 172.21.15.94:0/611434533 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a513b3c90 con 0x561a51344ad0 2024-02-17T12:10:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.288+0000 7f76f967b740 1 -- 172.21.15.94:0/611434533 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a513b3e50 con 0x561a5134d2e0 2024-02-17T12:10:08.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.288+0000 7f76f967b740 10 client.0 _ll_register_callbacks cb 0x7f76e8009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.288+0000 7f76f4839640 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51343d60 0x561a51347150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.289+0000 7f76effff640 1 -- 172.21.15.94:0/611434533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 msgr2=0x561a51347bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.289+0000 7f76effff640 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a51347bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.289+0000 7f76effff640 1 -- 172.21.15.94:0/611434533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51343d60 msgr2=0x561a51347150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.289+0000 7f76effff640 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51343d60 0x561a51347150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.289+0000 7f76effff640 1 -- 172.21.15.94:0/611434533 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f01048b0 con 0x561a51344ad0 2024-02-17T12:10:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.289+0000 7f76f4839640 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51343d60 0x561a51347150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:08.291 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.289+0000 7f76effff640 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51344ad0 0x561a51347690 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f76e0099fb0 tx=0x7f76e009a830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c573607d2024ab server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.289+0000 7f76cffff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:10:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.290+0000 7f76cffff640 1 -- 172.21.15.94:0/611434533 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e00a27c0 con 0x561a51344ad0 2024-02-17T12:10:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.290+0000 7f76f967b740 1 -- 172.21.15.94:0/611434533 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76e8003cd0 con 0x561a51344ad0 2024-02-17T12:10:08.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.290+0000 7f76cffff640 1 -- 172.21.15.94:0/611434533 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f76e00caef0 con 0x561a51344ad0 2024-02-17T12:10:08.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.291+0000 7f76cffff640 1 -- 172.21.15.94:0/611434533 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76e00cb0b0 con 0x561a51344ad0 2024-02-17T12:10:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.294+0000 7f76f967b740 10 client.28115 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.294+0000 7f76f967b740 1 -- 172.21.15.94:0/611434533 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561a5126c540 con 0x561a51344ad0 2024-02-17T12:10:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.297+0000 7f76cffff640 1 -- 172.21.15.94:0/611434533 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f76e8003cd0 con 0x561a51344ad0 2024-02-17T12:10:08.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.297+0000 7f76cffff640 1 -- 172.21.15.94:0/611434533 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f76e00b8270 con 0x561a51344ad0 2024-02-17T12:10:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.300+0000 7f76f967b740 10 client.28115 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.300+0000 7f76f967b740 10 client.28115 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.301+0000 7f76f967b740 20 client.28115 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.301+0000 7f76f967b740 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a513b5550 0x561a513b7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.301+0000 7f76f967b740 4 client.28115 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:10:08.303 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.301+0000 7f76f967b740 1 -- 172.21.15.94:0/611434533 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561a50fe8550 con 0x561a513b5550 2024-02-17T12:10:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.302+0000 7f76f4839640 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a513b5550 0x561a513b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.306+0000 7f76f4839640 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a513b5550 0x561a513b7970 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c57ec4c47558c95f server_cookie=ac692993a6af7ee in_seq=0 out_seq=0 2024-02-17T12:10:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.306+0000 7f76cffff640 10 client.28115 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.646+0000 7f76cffff640 1 -- 172.21.15.94:0/611434533 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561a50fe8550 con 0x561a513b5550 2024-02-17T12:10:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.646+0000 7f76cffff640 10 client.28115 handle_command_reply: tid=0 2024-02-17T12:10:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.646+0000 7f76f967b740 1 client.28115 shutdown 2024-02-17T12:10:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.646+0000 7f76f967b740 1 -- 172.21.15.94:0/611434533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51344ad0 msgr2=0x561a51347690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.646+0000 7f76f967b740 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51344ad0 0x561a51347690 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f76e0099fb0 tx=0x7f76e009a830 comp rx=0 tx=0).stop 2024-02-17T12:10:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.647+0000 7f76f967b740 1 -- 172.21.15.94:0/611434533 shutdown_connections 2024-02-17T12:10:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.647+0000 7f76f967b740 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a513b5550 0x561a513b7970 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.647+0000 7f76f967b740 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a5134d2e0 0x561a51347bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.647+0000 7f76f967b740 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a51344ad0 0x561a51347690 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.647+0000 7f76f967b740 1 --2- 172.21.15.94:0/611434533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a51343d60 0x561a51347150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.647+0000 7f76f967b740 1 -- 172.21.15.94:0/611434533 >> 172.21.15.94:0/611434533 conn(0x561a5130b550 msgr2=0x561a51348bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.647+0000 7f76cffff640 0 client.28115 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.647+0000 7f76f967b740 1 -- 172.21.15.94:0/611434533 shutdown_connections 2024-02-17T12:10:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.647+0000 7f76f967b740 1 -- 172.21.15.94:0/611434533 wait complete. 2024-02-17T12:10:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.647+0000 7f76f967b740 20 client.28115 trim_cache size 0 max 16384 2024-02-17T12:10:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:10:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:10:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:10:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:10:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.648+0000 7f76f6a9d640 1 -- 172.21.15.94:0/2341970984 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f76dc0a2430 msgr2=0x7f76dc0a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.648+0000 7f76f6a9d640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f76dc0a2430 0x7f76dc0a48f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f76d8034fa0 tx=0x7f76d802bc10 comp rx=0 tx=0).stop 2024-02-17T12:10:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.648+0000 7f76f6a9d640 1 -- 172.21.15.94:0/2341970984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0103b90 msgr2=0x7f76f0071e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.648+0000 7f76f6a9d640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0103b90 0x7f76f0071e70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f76e000b3f0 tx=0x7f76e000b8b0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.648+0000 7f76f6a9d640 1 -- 172.21.15.94:0/2341970984 shutdown_connections 2024-02-17T12:10:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.648+0000 7f76f6a9d640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f76dc0a2430 0x7f76dc0a48f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.648+0000 7f76f6a9d640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f76f0106140 0x7f76f010a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.648+0000 7f76f6a9d640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f76f0103b90 0x7f76f0071e70 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.648+0000 7f76f6a9d640 1 --2- 172.21.15.94:0/2341970984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76f0068d40 0x7f76f0071930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.648+0000 7f76f6a9d640 1 -- 172.21.15.94:0/2341970984 >> 172.21.15.94:0/2341970984 conn(0x7f76f0076ed0 msgr2=0x7f76f00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.648+0000 7f76f6a9d640 1 -- 172.21.15.94:0/2341970984 shutdown_connections 2024-02-17T12:10:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:08.648+0000 7f76f6a9d640 1 -- 172.21.15.94:0/2341970984 wait complete. 2024-02-17T12:10:09.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:09 smithi189 ceph-mon[36588]: osdmap e99: 12 total, 12 up, 12 in 2024-02-17T12:10:09.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:09 smithi189 ceph-mon[36588]: pgmap v375: 97 pgs: 31 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 51 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 20 MiB/s wr, 1.96k op/s 2024-02-17T12:10:09.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:09 smithi125 ceph-mon[36819]: osdmap e99: 12 total, 12 up, 12 in 2024-02-17T12:10:09.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:09 smithi125 ceph-mon[36819]: pgmap v375: 97 pgs: 31 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 51 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 20 MiB/s wr, 1.96k op/s 2024-02-17T12:10:09.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:09 smithi094 ceph-mon[32499]: osdmap e99: 12 total, 12 up, 12 in 2024-02-17T12:10:09.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:09 smithi094 ceph-mon[32499]: pgmap v375: 97 pgs: 31 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 51 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 20 MiB/s wr, 1.96k op/s 2024-02-17T12:10:10.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 710.0 s, 1243.8 tps, lat 81.405 ms stddev 107.915 2024-02-17T12:10:11.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:11 smithi189 ceph-mon[36588]: pgmap v376: 97 pgs: 31 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 51 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 20 MiB/s wr, 1.98k op/s 2024-02-17T12:10:11.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:11 smithi125 ceph-mon[36819]: pgmap v376: 97 pgs: 31 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 51 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 20 MiB/s wr, 1.98k op/s 2024-02-17T12:10:11.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:11 smithi094 ceph-mon[32499]: pgmap v376: 97 pgs: 31 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 51 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 20 MiB/s wr, 1.98k op/s 2024-02-17T12:10:13.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:13 smithi189 ceph-mon[36588]: pgmap v377: 97 pgs: 31 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 51 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 20 MiB/s wr, 1.98k op/s 2024-02-17T12:10:13.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:13 smithi125 ceph-mon[36819]: pgmap v377: 97 pgs: 31 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 51 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 20 MiB/s wr, 1.98k op/s 2024-02-17T12:10:13.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:13 smithi094 ceph-mon[32499]: pgmap v377: 97 pgs: 31 active+clean+snaptrim_wait, 15 active+clean+snaptrim, 51 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 20 MiB/s wr, 1.98k op/s 2024-02-17T12:10:15.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 715.0 s, 2249.8 tps, lat 44.668 ms stddev 56.364 2024-02-17T12:10:15.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:15 smithi189 ceph-mon[36588]: pgmap v378: 97 pgs: 97 active+clean; 5.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.9 MiB/s wr, 621 op/s 2024-02-17T12:10:15.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:15 smithi125 ceph-mon[36819]: pgmap v378: 97 pgs: 97 active+clean; 5.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.9 MiB/s wr, 621 op/s 2024-02-17T12:10:15.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:15 smithi094 ceph-mon[32499]: pgmap v378: 97 pgs: 97 active+clean; 5.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 5.9 MiB/s wr, 621 op/s 2024-02-17T12:10:16.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:16 smithi189 ceph-mon[36588]: osdmap e100: 12 total, 12 up, 12 in 2024-02-17T12:10:16.686 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:16 smithi125 ceph-mon[36819]: osdmap e100: 12 total, 12 up, 12 in 2024-02-17T12:10:16.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:16 smithi094 ceph-mon[32499]: osdmap e100: 12 total, 12 up, 12 in 2024-02-17T12:10:17.588 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:17 smithi125 ceph-mon[36819]: pgmap v380: 97 pgs: 97 active+clean; 5.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.6 MiB/s wr, 698 op/s 2024-02-17T12:10:17.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:17 smithi189 ceph-mon[36588]: pgmap v380: 97 pgs: 97 active+clean; 5.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.6 MiB/s wr, 698 op/s 2024-02-17T12:10:17.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:17 smithi094 ceph-mon[32499]: pgmap v380: 97 pgs: 97 active+clean; 5.3 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 6.6 MiB/s wr, 698 op/s 2024-02-17T12:10:19.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:19 smithi189 ceph-mon[36588]: pgmap v381: 97 pgs: 97 active+clean; 5.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-02-17T12:10:19.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:19 smithi125 ceph-mon[36819]: pgmap v381: 97 pgs: 97 active+clean; 5.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-02-17T12:10:19.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:19 smithi094 ceph-mon[32499]: pgmap v381: 97 pgs: 97 active+clean; 5.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.9 MiB/s wr, 697 op/s 2024-02-17T12:10:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 720.0 s, 1665.8 tps, lat 59.032 ms stddev 63.891 2024-02-17T12:10:21.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:21 smithi189 ceph-mon[36588]: pgmap v382: 97 pgs: 97 active+clean; 5.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.6 MiB/s wr, 676 op/s 2024-02-17T12:10:21.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:21 smithi125 ceph-mon[36819]: pgmap v382: 97 pgs: 97 active+clean; 5.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.6 MiB/s wr, 676 op/s 2024-02-17T12:10:21.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:21 smithi094 ceph-mon[32499]: pgmap v382: 97 pgs: 97 active+clean; 5.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.6 MiB/s wr, 676 op/s 2024-02-17T12:10:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.772+0000 7f8374525640 1 -- 172.21.15.94:0/2054547450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f836c068800 msgr2=0x7f836c068c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.772+0000 7f8374525640 1 --2- 172.21.15.94:0/2054547450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f836c068800 0x7f836c068c60 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f835c009900 tx=0x7f835c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:10:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.773+0000 7f8374525640 1 -- 172.21.15.94:0/2054547450 shutdown_connections 2024-02-17T12:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.773+0000 7f8374525640 1 --2- 172.21.15.94:0/2054547450 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f836c0691a0 0x7f836c10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.773+0000 7f8374525640 1 --2- 172.21.15.94:0/2054547450 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f836c068800 0x7f836c068c60 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.773+0000 7f8374525640 1 --2- 172.21.15.94:0/2054547450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f836c107290 0x7f836c107670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.773+0000 7f8374525640 1 -- 172.21.15.94:0/2054547450 >> 172.21.15.94:0/2054547450 conn(0x7f836c076ed0 msgr2=0x7f836c0772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.773+0000 7f8374525640 1 -- 172.21.15.94:0/2054547450 shutdown_connections 2024-02-17T12:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.773+0000 7f8374525640 1 -- 172.21.15.94:0/2054547450 wait complete. 2024-02-17T12:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.773+0000 7f8374525640 1 Processor -- start 2024-02-17T12:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.773+0000 7f8374525640 1 -- start start 2024-02-17T12:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f8374525640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f836c068800 0x7f836c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f8374525640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f836c0691a0 0x7f836c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f8374525640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f836c107290 0x7f836c19bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f8374525640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c19c540 con 0x7f836c107290 2024-02-17T12:10:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f8374525640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c19c6b0 con 0x7f836c068800 2024-02-17T12:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f8374525640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836c19c870 con 0x7f836c0691a0 2024-02-17T12:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f83722c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f836c068800 0x7f836c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f83722c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f836c068800 0x7f836c197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:50868/0 (socket says 172.21.15.94:50868) 2024-02-17T12:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f83722c1640 1 -- 172.21.15.94:0/3030149416 learned_addr learned my addr 172.21.15.94:0/3030149416 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f8371ac0640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f836c0691a0 0x7f836c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f8372ac2640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f836c107290 0x7f836c19bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f83722c1640 1 -- 172.21.15.94:0/3030149416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f836c0691a0 msgr2=0x7f836c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f83722c1640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f836c0691a0 0x7f836c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f83722c1640 1 -- 172.21.15.94:0/3030149416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f836c107290 msgr2=0x7f836c19bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f83722c1640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f836c107290 0x7f836c19bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f83722c1640 1 -- 172.21.15.94:0/3030149416 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c009560 con 0x7f836c068800 2024-02-17T12:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f83722c1640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f836c068800 0x7f836c197680 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f836800c970 tx=0x7f836800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.774+0000 7f83637fe640 1 -- 172.21.15.94:0/3030149416 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8368007bb0 con 0x7f836c068800 2024-02-17T12:10:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.775+0000 7f8374525640 1 -- 172.21.15.94:0/3030149416 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836c19cb50 con 0x7f836c068800 2024-02-17T12:10:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.775+0000 7f83637fe640 1 -- 172.21.15.94:0/3030149416 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8368007d10 con 0x7f836c068800 2024-02-17T12:10:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.775+0000 7f83637fe640 1 -- 172.21.15.94:0/3030149416 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836800f450 con 0x7f836c068800 2024-02-17T12:10:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.775+0000 7f8374525640 1 -- 172.21.15.94:0/3030149416 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836c1a43d0 con 0x7f836c068800 2024-02-17T12:10:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.777+0000 7f83637fe640 1 -- 172.21.15.94:0/3030149416 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8368004f70 con 0x7f836c068800 2024-02-17T12:10:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.778+0000 7f83771d2740 1 Processor -- start 2024-02-17T12:10:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.778+0000 7f83771d2740 1 -- start start 2024-02-17T12:10:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.778+0000 7f83771d2740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea05ad0 0x559b0ea04820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.778+0000 7f83771d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea04d60 0x559b0ea051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.778+0000 7f83771d2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea1f2f0 0x559b0ea216e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.779+0000 7f83771d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0e9739f0 con 0x559b0ea04d60 2024-02-17T12:10:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.779+0000 7f83771d2740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0e970f20 con 0x559b0ea05ad0 2024-02-17T12:10:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.779+0000 7f83771d2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0e96c700 con 0x559b0ea1f2f0 2024-02-17T12:10:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.779+0000 7f8372ac2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea05ad0 0x559b0ea04820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.779+0000 7f8372ac2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea05ad0 0x559b0ea04820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:50884/0 (socket says 172.21.15.94:50884) 2024-02-17T12:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.779+0000 7f8372ac2640 1 -- 172.21.15.94:0/3469042188 learned_addr learned my addr 172.21.15.94:0/3469042188 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.779+0000 7f83722c1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea04d60 0x559b0ea051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.779+0000 7f8372ac2640 1 -- 172.21.15.94:0/3469042188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea1f2f0 msgr2=0x559b0ea216e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.779+0000 7f8372ac2640 1 --2- 172.21.15.94:0/3469042188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea1f2f0 0x559b0ea216e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.779+0000 7f8372ac2640 1 -- 172.21.15.94:0/3469042188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea04d60 msgr2=0x559b0ea051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.779+0000 7f8372ac2640 1 --2- 172.21.15.94:0/3469042188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea04d60 0x559b0ea051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.779+0000 7f8372ac2640 1 -- 172.21.15.94:0/3469042188 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b0e9027b0 con 0x559b0ea05ad0 2024-02-17T12:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.779+0000 7f83637fe640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83580a22d0 0x7f83580a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.780+0000 7f8372ac2640 1 --2- 172.21.15.94:0/3469042188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea05ad0 0x559b0ea04820 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8364009fc0 tx=0x7f83640118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81f8be36db3f301d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.780+0000 7f83637fe640 1 -- 172.21.15.94:0/3030149416 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f83680cddf0 con 0x7f836c068800 2024-02-17T12:10:21.785 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.780+0000 7f83722c1640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83580a22d0 0x7f83580a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.780+0000 7f8360bfe640 1 -- 172.21.15.94:0/3469042188 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364004d20 con 0x559b0ea05ad0 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.780+0000 7f8360bfe640 1 -- 172.21.15.94:0/3469042188 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8364004e80 con 0x559b0ea05ad0 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.780+0000 7f8360bfe640 1 -- 172.21.15.94:0/3469042188 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364005280 con 0x559b0ea05ad0 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.781+0000 7f83771d2740 1 -- 172.21.15.94:0/3469042188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea05ad0 msgr2=0x559b0ea04820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.781+0000 7f83722c1640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83580a22d0 0x7f83580a4790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x559b0ea06230 tx=0x7f83680d1860 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.781+0000 7f83771d2740 1 --2- 172.21.15.94:0/3469042188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea05ad0 0x559b0ea04820 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8364009fc0 tx=0x7f83640118d0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.781+0000 7f83771d2740 1 -- 172.21.15.94:0/3469042188 shutdown_connections 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.781+0000 7f83771d2740 1 --2- 172.21.15.94:0/3469042188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea1f2f0 0x559b0ea216e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.781+0000 7f83771d2740 1 --2- 172.21.15.94:0/3469042188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea04d60 0x559b0ea051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.781+0000 7f83771d2740 1 --2- 172.21.15.94:0/3469042188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea05ad0 0x559b0ea04820 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.781+0000 7f83771d2740 1 -- 172.21.15.94:0/3469042188 >> 172.21.15.94:0/3469042188 conn(0x559b0e9cc550 msgr2=0x559b0e9ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.781+0000 7f83771d2740 1 -- 172.21.15.94:0/3469042188 shutdown_connections 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.781+0000 7f83771d2740 1 -- 172.21.15.94:0/3469042188 wait complete. 2024-02-17T12:10:21.786 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.781+0000 7f83771d2740 1 -- start start 2024-02-17T12:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f83771d2740 1 Processor -- start 2024-02-17T12:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f83771d2740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea04d60 0x559b0ea1c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f83771d2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea1c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f83771d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea1f2f0 0x559b0ea1cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f83771d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0e911bf0 con 0x559b0ea1f2f0 2024-02-17T12:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f83771d2740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0e640250 con 0x559b0ea04d60 2024-02-17T12:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f83771d2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0e8f2200 con 0x559b0ea05ad0 2024-02-17T12:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f83771d2740 10 client.0 _ll_register_callbacks cb 0x559b0e960d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f8372ac2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea04d60 0x559b0ea1c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f8371ac0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea1c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.787 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f8372ac2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea1f2f0 0x559b0ea1cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f8371ac0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea1c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:36992/0 (socket says 172.21.15.94:36992) 2024-02-17T12:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f8371ac0640 1 -- 172.21.15.94:0/2094909834 learned_addr learned my addr 172.21.15.94:0/2094909834 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f8371ac0640 1 -- 172.21.15.94:0/2094909834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea04d60 msgr2=0x559b0ea1c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f8371ac0640 1 --2- 172.21.15.94:0/2094909834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea04d60 0x559b0ea1c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f8371ac0640 1 -- 172.21.15.94:0/2094909834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea1f2f0 msgr2=0x559b0ea1cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f8371ac0640 1 --2- 172.21.15.94:0/2094909834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea1f2f0 0x559b0ea1cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f8371ac0640 1 -- 172.21.15.94:0/2094909834 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8364009c70 con 0x559b0ea05ad0 2024-02-17T12:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.782+0000 7f8372ac2640 1 --2- 172.21.15.94:0/2094909834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea1f2f0 0x559b0ea1cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.783+0000 7f8371ac0640 1 --2- 172.21.15.94:0/2094909834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea1c880 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f835c009230 tx=0x7f835c02ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97810a66d3a21db9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.783+0000 7f8355ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.783+0000 7f8355ffb640 1 -- 172.21.15.94:0/2094909834 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c004040 con 0x559b0ea05ad0 2024-02-17T12:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.783+0000 7f83771d2740 1 -- 172.21.15.94:0/2094909834 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b0e6bd1b0 con 0x559b0ea05ad0 2024-02-17T12:10:21.788 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.783+0000 7f8355ffb640 1 -- 172.21.15.94:0/2094909834 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f835c0041a0 con 0x559b0ea05ad0 2024-02-17T12:10:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.783+0000 7f8355ffb640 1 -- 172.21.15.94:0/2094909834 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f835c004360 con 0x559b0ea05ad0 2024-02-17T12:10:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.783+0000 7f83771d2740 10 client.28126 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.783+0000 7f83771d2740 1 -- 172.21.15.94:0/2094909834 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559b0e96ad80 con 0x559b0ea05ad0 2024-02-17T12:10:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.783+0000 7f8355ffb640 1 -- 172.21.15.94:0/2094909834 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559b0e6bd1b0 con 0x559b0ea05ad0 2024-02-17T12:10:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.783+0000 7f8355ffb640 1 -- 172.21.15.94:0/2094909834 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f835c03c050 con 0x559b0ea05ad0 2024-02-17T12:10:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.784+0000 7f83771d2740 10 client.28126 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.784+0000 7f83771d2740 10 client.28126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.784+0000 7f83771d2740 20 client.28126 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.784+0000 7f83771d2740 1 --2- 172.21.15.94:0/2094909834 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559b0ea339a0 0x559b0ea35dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.784+0000 7f83771d2740 4 client.28126 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:10:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.784+0000 7f83771d2740 1 -- 172.21.15.94:0/2094909834 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b0e6a9550 con 0x559b0ea339a0 2024-02-17T12:10:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.784+0000 7f8372ac2640 1 --2- 172.21.15.94:0/2094909834 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559b0ea339a0 0x559b0ea35dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.785+0000 7f8355ffb640 10 client.28126 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.785+0000 7f8372ac2640 1 --2- 172.21.15.94:0/2094909834 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559b0ea339a0 0x559b0ea35dc0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d45e7d2d3d61e1fb server_cookie=30b04a81c916c4d2 in_seq=0 out_seq=0 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.785+0000 7f8355ffb640 1 -- 172.21.15.94:0/2094909834 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b0e6a9550 con 0x559b0ea339a0 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.785+0000 7f8355ffb640 10 client.28126 handle_command_reply: tid=0 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.785+0000 7f83771d2740 1 client.28126 shutdown 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.786+0000 7f83771d2740 1 -- 172.21.15.94:0/2094909834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 msgr2=0x559b0ea1c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.786+0000 7f83771d2740 1 --2- 172.21.15.94:0/2094909834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea1c880 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f835c009230 tx=0x7f835c02ca10 comp rx=0 tx=0).stop 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.786+0000 7f83771d2740 1 -- 172.21.15.94:0/2094909834 shutdown_connections 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.786+0000 7f83771d2740 1 --2- 172.21.15.94:0/2094909834 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559b0ea339a0 0x559b0ea35dc0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.786+0000 7f8355ffb640 0 client.28126 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.786+0000 7f83771d2740 1 --2- 172.21.15.94:0/2094909834 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea1f2f0 0x559b0ea1cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.786+0000 7f83771d2740 1 --2- 172.21.15.94:0/2094909834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea1c880 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.786+0000 7f83771d2740 1 --2- 172.21.15.94:0/2094909834 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea04d60 0x559b0ea1c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.786+0000 7f83771d2740 1 -- 172.21.15.94:0/2094909834 >> 172.21.15.94:0/2094909834 conn(0x559b0e9cc550 msgr2=0x559b0ea017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.786+0000 7f83771d2740 1 -- 172.21.15.94:0/2094909834 shutdown_connections 2024-02-17T12:10:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.786+0000 7f83771d2740 1 -- 172.21.15.94:0/2094909834 wait complete. 2024-02-17T12:10:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.786+0000 7f83771d2740 20 client.28126 trim_cache size 0 max 16384 2024-02-17T12:10:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f83771d2740 1 Processor -- start 2024-02-17T12:10:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f83771d2740 1 -- start start 2024-02-17T12:10:21.807 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f83771d2740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea04d60 0x559b0ea1ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f83771d2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea1cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f83771d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea1f2f0 0x559b0ea26160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f83771d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0ea00620 con 0x559b0ea1f2f0 2024-02-17T12:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f83771d2740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0e9faa20 con 0x559b0ea04d60 2024-02-17T12:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f83771d2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0ea17d20 con 0x559b0ea05ad0 2024-02-17T12:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f8372ac2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea1f2f0 0x559b0ea26160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f8372ac2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea1f2f0 0x559b0ea26160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48986/0 (socket says 172.21.15.94:48986) 2024-02-17T12:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f8372ac2640 1 -- 172.21.15.94:0/1241037100 learned_addr learned my addr 172.21.15.94:0/1241037100 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f8372ac2640 1 --2- 172.21.15.94:0/1241037100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea04d60 0x559b0ea1ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f8371ac0640 1 --2- 172.21.15.94:0/1241037100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea1cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f8372ac2640 1 -- 172.21.15.94:0/1241037100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 msgr2=0x559b0ea1cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:21.808 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f8372ac2640 1 --2- 172.21.15.94:0/1241037100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea1cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f8372ac2640 1 -- 172.21.15.94:0/1241037100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea04d60 msgr2=0x559b0ea1ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f8372ac2640 1 --2- 172.21.15.94:0/1241037100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea04d60 0x559b0ea1ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.804+0000 7f8372ac2640 1 -- 172.21.15.94:0/1241037100 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b0e9077b0 con 0x559b0ea1f2f0 2024-02-17T12:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.805+0000 7f8372ac2640 1 --2- 172.21.15.94:0/1241037100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea1f2f0 0x559b0ea26160 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f8364014040 tx=0x7f8364005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b8fb503e1492a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.805+0000 7f8357fff640 1 -- 172.21.15.94:0/1241037100 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8364004000 con 0x559b0ea1f2f0 2024-02-17T12:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.805+0000 7f8357fff640 1 -- 172.21.15.94:0/1241037100 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8364033070 con 0x559b0ea1f2f0 2024-02-17T12:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.805+0000 7f8357fff640 1 -- 172.21.15.94:0/1241037100 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836400fcb0 con 0x559b0ea1f2f0 2024-02-17T12:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.806+0000 7f83771d2740 1 -- 172.21.15.94:0/1241037100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea1f2f0 msgr2=0x559b0ea26160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.806+0000 7f83771d2740 1 --2- 172.21.15.94:0/1241037100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea1f2f0 0x559b0ea26160 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f8364014040 tx=0x7f8364005fd0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.806+0000 7f83771d2740 1 -- 172.21.15.94:0/1241037100 shutdown_connections 2024-02-17T12:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.806+0000 7f83771d2740 1 --2- 172.21.15.94:0/1241037100 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea1f2f0 0x559b0ea26160 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.806+0000 7f83771d2740 1 --2- 172.21.15.94:0/1241037100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea1cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.809 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.806+0000 7f83771d2740 1 --2- 172.21.15.94:0/1241037100 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea04d60 0x559b0ea1ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.806+0000 7f83771d2740 1 -- 172.21.15.94:0/1241037100 >> 172.21.15.94:0/1241037100 conn(0x559b0e9cc550 msgr2=0x559b0ea1ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.806+0000 7f83771d2740 1 -- 172.21.15.94:0/1241037100 shutdown_connections 2024-02-17T12:10:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.806+0000 7f83771d2740 1 -- 172.21.15.94:0/1241037100 wait complete. 2024-02-17T12:10:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.806+0000 7f83771d2740 1 -- start start 2024-02-17T12:10:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.806+0000 7f83771d2740 1 Processor -- start 2024-02-17T12:10:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.807+0000 7f83771d2740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea04d60 0x559b0ea19020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.807+0000 7f83771d2740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea19560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.807+0000 7f83771d2740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea1f2f0 0x559b0ea19aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.807+0000 7f83771d2740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0ea89ea0 con 0x559b0ea04d60 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.807+0000 7f83771d2740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0ea89fe0 con 0x559b0ea1f2f0 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.807+0000 7f83771d2740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0ea8a120 con 0x559b0ea05ad0 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.807+0000 7f83771d2740 10 client.0 _ll_register_callbacks cb 0x7f8364009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.807+0000 7f8372ac2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea1f2f0 0x559b0ea19aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.807+0000 7f8372ac2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea1f2f0 0x559b0ea19aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:50906/0 (socket says 172.21.15.94:50906) 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.807+0000 7f8372ac2640 1 -- 172.21.15.94:0/527070773 learned_addr learned my addr 172.21.15.94:0/527070773 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.808+0000 7f8371ac0640 1 --2- 172.21.15.94:0/527070773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea19560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.808+0000 7f8372ac2640 1 -- 172.21.15.94:0/527070773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 msgr2=0x559b0ea19560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.808+0000 7f8372ac2640 1 --2- 172.21.15.94:0/527070773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea19560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.808+0000 7f8372ac2640 1 -- 172.21.15.94:0/527070773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea04d60 msgr2=0x559b0ea19020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.808+0000 7f8372ac2640 1 --2- 172.21.15.94:0/527070773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea04d60 0x559b0ea19020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.808+0000 7f8372ac2640 1 -- 172.21.15.94:0/527070773 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b0e9027b0 con 0x559b0ea1f2f0 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.808+0000 7f8372ac2640 1 --2- 172.21.15.94:0/527070773 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea1f2f0 0x559b0ea19aa0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8364004e50 tx=0x7f83640094f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7298724122b3510 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.808+0000 7f8355ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.808+0000 7f8355ffb640 1 -- 172.21.15.94:0/527070773 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f836400fe40 con 0x559b0ea1f2f0 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.808+0000 7f8355ffb640 1 -- 172.21.15.94:0/527070773 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8364004000 con 0x559b0ea1f2f0 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.808+0000 7f83771d2740 1 -- 172.21.15.94:0/527070773 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b0e66c600 con 0x559b0ea1f2f0 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.809+0000 7f8355ffb640 1 -- 172.21.15.94:0/527070773 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83640041c0 con 0x559b0ea1f2f0 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.809+0000 7f83771d2740 10 client.28133 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.809+0000 7f83771d2740 1 -- 172.21.15.94:0/527070773 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559b0e92d540 con 0x559b0ea1f2f0 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.809+0000 7f8355ffb640 1 -- 172.21.15.94:0/527070773 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559b0e66c600 con 0x559b0ea1f2f0 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.809+0000 7f8355ffb640 1 -- 172.21.15.94:0/527070773 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8364021040 con 0x559b0ea1f2f0 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.809+0000 7f83771d2740 10 client.28133 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.809+0000 7f83771d2740 10 client.28133 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.809+0000 7f83771d2740 20 client.28133 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.809+0000 7f83771d2740 1 --2- 172.21.15.94:0/527070773 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559b0ea339a0 0x559b0ea2d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.809+0000 7f83771d2740 4 client.28133 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:10:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.809+0000 7f83771d2740 1 -- 172.21.15.94:0/527070773 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559b0e618520 con 0x559b0ea339a0 2024-02-17T12:10:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.809+0000 7f8371ac0640 1 --2- 172.21.15.94:0/527070773 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559b0ea339a0 0x559b0ea2d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.809+0000 7f8355ffb640 10 client.28133 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.810+0000 7f8371ac0640 1 --2- 172.21.15.94:0/527070773 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559b0ea339a0 0x559b0ea2d7d0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b15144ffda872c server_cookie=3fee42b5e7032f7c in_seq=0 out_seq=0 2024-02-17T12:10:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.898+0000 7f8355ffb640 1 -- 172.21.15.94:0/527070773 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559b0e618520 con 0x559b0ea339a0 2024-02-17T12:10:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.898+0000 7f8355ffb640 10 client.28133 handle_command_reply: tid=0 2024-02-17T12:10:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.898+0000 7f83771d2740 1 client.28133 shutdown 2024-02-17T12:10:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.899+0000 7f83771d2740 1 -- 172.21.15.94:0/527070773 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea1f2f0 msgr2=0x559b0ea19aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.899+0000 7f83771d2740 1 --2- 172.21.15.94:0/527070773 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea1f2f0 0x559b0ea19aa0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8364004e50 tx=0x7f83640094f0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.899+0000 7f83771d2740 1 -- 172.21.15.94:0/527070773 shutdown_connections 2024-02-17T12:10:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.899+0000 7f83771d2740 1 --2- 172.21.15.94:0/527070773 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559b0ea339a0 0x559b0ea2d7d0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f83771d2740 1 --2- 172.21.15.94:0/527070773 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559b0ea1f2f0 0x559b0ea19aa0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f83771d2740 1 --2- 172.21.15.94:0/527070773 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559b0ea05ad0 0x559b0ea19560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f83771d2740 1 --2- 172.21.15.94:0/527070773 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559b0ea04d60 0x559b0ea19020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f8355ffb640 0 client.28133 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f83771d2740 1 -- 172.21.15.94:0/527070773 >> 172.21.15.94:0/527070773 conn(0x559b0e9cc550 msgr2=0x559b0e9fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f83771d2740 1 -- 172.21.15.94:0/527070773 shutdown_connections 2024-02-17T12:10:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f83771d2740 1 -- 172.21.15.94:0/527070773 wait complete. 2024-02-17T12:10:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f83771d2740 20 client.28133 trim_cache size 0 max 16384 2024-02-17T12:10:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:10:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:10:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:10:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:10:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f8374525640 1 -- 172.21.15.94:0/3030149416 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83580a22d0 msgr2=0x7f83580a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f8374525640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83580a22d0 0x7f83580a4790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x559b0ea06230 tx=0x7f83680d1860 comp rx=0 tx=0).stop 2024-02-17T12:10:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f8374525640 1 -- 172.21.15.94:0/3030149416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f836c068800 msgr2=0x7f836c197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f8374525640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f836c068800 0x7f836c197680 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f836800c970 tx=0x7f836800ce30 comp rx=0 tx=0).stop 2024-02-17T12:10:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f8374525640 1 -- 172.21.15.94:0/3030149416 shutdown_connections 2024-02-17T12:10:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f8374525640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83580a22d0 0x7f83580a4790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f8374525640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f836c107290 0x7f836c19bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f8374525640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f836c0691a0 0x7f836c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f8374525640 1 --2- 172.21.15.94:0/3030149416 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f836c068800 0x7f836c197680 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f8374525640 1 -- 172.21.15.94:0/3030149416 >> 172.21.15.94:0/3030149416 conn(0x7f836c076ed0 msgr2=0x7f836c0fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f8374525640 1 -- 172.21.15.94:0/3030149416 shutdown_connections 2024-02-17T12:10:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:21.900+0000 7f8374525640 1 -- 172.21.15.94:0/3030149416 wait complete. 2024-02-17T12:10:23.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:23 smithi189 ceph-mon[36588]: pgmap v383: 97 pgs: 97 active+clean; 5.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.6 MiB/s wr, 676 op/s 2024-02-17T12:10:23.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:23 smithi125 ceph-mon[36819]: pgmap v383: 97 pgs: 97 active+clean; 5.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.6 MiB/s wr, 676 op/s 2024-02-17T12:10:23.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:23 smithi094 ceph-mon[32499]: pgmap v383: 97 pgs: 97 active+clean; 5.3 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 6.6 MiB/s wr, 676 op/s 2024-02-17T12:10:25.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 725.0 s, 1404.2 tps, lat 71.912 ms stddev 73.949 2024-02-17T12:10:25.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:25 smithi189 ceph-mon[36588]: pgmap v384: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.3 MiB/s wr, 673 op/s 2024-02-17T12:10:25.436 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:25 smithi125 ceph-mon[36819]: pgmap v384: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.3 MiB/s wr, 673 op/s 2024-02-17T12:10:25.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:25 smithi094 ceph-mon[32499]: pgmap v384: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 6.3 MiB/s wr, 673 op/s 2024-02-17T12:10:27.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:27 smithi189 ceph-mon[36588]: pgmap v385: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-02-17T12:10:27.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:27 smithi125 ceph-mon[36819]: pgmap v385: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-02-17T12:10:27.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:27 smithi094 ceph-mon[32499]: pgmap v385: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-02-17T12:10:29.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:29 smithi189 ceph-mon[36588]: pgmap v386: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.2 MiB/s wr, 752 op/s 2024-02-17T12:10:29.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:29 smithi125 ceph-mon[36819]: pgmap v386: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.2 MiB/s wr, 752 op/s 2024-02-17T12:10:29.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:29 smithi094 ceph-mon[32499]: pgmap v386: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 7.2 MiB/s wr, 752 op/s 2024-02-17T12:10:30.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 730.0 s, 1417.0 tps, lat 70.153 ms stddev 78.973 2024-02-17T12:10:30.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:30 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:10:30.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:30 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:30.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:30 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:30.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:30 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:30.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:30 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:30.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:30 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:10:30.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:30 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:30.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:30 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:30.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:30 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:30.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:30 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:30.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:30 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:10:30.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:30 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:30.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:30 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:30.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:30 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:30.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:30 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:31.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:10:31.164 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:10:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.311+0000 7fa109f27640 1 -- 172.21.15.94:0/1246071844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0fc004740 msgr2=0x7fa0fc004b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.311+0000 7fa109f27640 1 --2- 172.21.15.94:0/1246071844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0fc004740 0x7fa0fc004b20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa104066e50 tx=0x7fa104096e70 comp rx=0 tx=0).stop 2024-02-17T12:10:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.311+0000 7fa109f27640 1 -- 172.21.15.94:0/1246071844 shutdown_connections 2024-02-17T12:10:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.311+0000 7fa109f27640 1 --2- 172.21.15.94:0/1246071844 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0fc0a2a00 0x7fa0fc0a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.311+0000 7fa109f27640 1 --2- 172.21.15.94:0/1246071844 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0fc005060 0x7fa0fc0a24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.311+0000 7fa109f27640 1 --2- 172.21.15.94:0/1246071844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0fc004740 0x7fa0fc004b20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.311+0000 7fa109f27640 1 -- 172.21.15.94:0/1246071844 >> 172.21.15.94:0/1246071844 conn(0x7fa0fc00a740 msgr2=0x7fa0fc00ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.311+0000 7fa109f27640 1 -- 172.21.15.94:0/1246071844 shutdown_connections 2024-02-17T12:10:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.311+0000 7fa109f27640 1 -- 172.21.15.94:0/1246071844 wait complete. 2024-02-17T12:10:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.311+0000 7fa109f27640 1 Processor -- start 2024-02-17T12:10:31.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa109f27640 1 -- start start 2024-02-17T12:10:31.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa109f27640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0fc004740 0x7fa0fc097130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa109f27640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0fc005060 0x7fa0fc097670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa109f27640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0fc0a2a00 0x7fa0fc097bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa109f27640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc13eac0 con 0x7fa0fc004740 2024-02-17T12:10:31.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa109f27640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc12e1d0 con 0x7fa0fc005060 2024-02-17T12:10:31.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa109f27640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc12e390 con 0x7fa0fc0a2a00 2024-02-17T12:10:31.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa102ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0fc005060 0x7fa0fc097670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa1037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0fc004740 0x7fa0fc097130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa102ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0fc005060 0x7fa0fc097670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41754/0 (socket says 172.21.15.94:41754) 2024-02-17T12:10:31.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa102ffd640 1 -- 172.21.15.94:0/3404415870 learned_addr learned my addr 172.21.15.94:0/3404415870 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:31.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa103fff640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0fc0a2a00 0x7fa0fc097bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa102ffd640 1 -- 172.21.15.94:0/3404415870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0fc0a2a00 msgr2=0x7fa0fc097bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa102ffd640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0fc0a2a00 0x7fa0fc097bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa102ffd640 1 -- 172.21.15.94:0/3404415870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0fc004740 msgr2=0x7fa0fc097130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa102ffd640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0fc004740 0x7fa0fc097130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa102ffd640 1 -- 172.21.15.94:0/3404415870 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10404f750 con 0x7fa0fc005060 2024-02-17T12:10:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.312+0000 7fa1037fe640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0fc004740 0x7fa0fc097130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.313+0000 7fa102ffd640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0fc005060 0x7fa0fc097670 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000d950 tx=0x7fa0f000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.313+0000 7fa100ff9640 1 -- 172.21.15.94:0/3404415870 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0009c30 con 0x7fa0fc005060 2024-02-17T12:10:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.313+0000 7fa109f27640 1 -- 172.21.15.94:0/3404415870 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc12e670 con 0x7fa0fc005060 2024-02-17T12:10:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.313+0000 7fa109f27640 1 -- 172.21.15.94:0/3404415870 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc12ebd0 con 0x7fa0fc005060 2024-02-17T12:10:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.313+0000 7fa100ff9640 1 -- 172.21.15.94:0/3404415870 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0f0009d90 con 0x7fa0fc005060 2024-02-17T12:10:31.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.315+0000 7fa100ff9640 1 -- 172.21.15.94:0/3404415870 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f000f7e0 con 0x7fa0fc005060 2024-02-17T12:10:31.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.316+0000 7fa100ff9640 1 -- 172.21.15.94:0/3404415870 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa0f000f9c0 con 0x7fa0fc005060 2024-02-17T12:10:31.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.316+0000 7fa10cbce740 1 Processor -- start 2024-02-17T12:10:31.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.316+0000 7fa10cbce740 1 -- start start 2024-02-17T12:10:31.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa10cbce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117490 0x558fc2117870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa10cbce740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2117db0 0x558fc2120450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa10cbce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2120990 0x558fc2122d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa10cbce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc2086280 con 0x558fc2117490 2024-02-17T12:10:31.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa10cbce740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc20837b0 con 0x558fc2117db0 2024-02-17T12:10:31.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa10cbce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc207ef90 con 0x558fc2120990 2024-02-17T12:10:31.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa100ff9640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa0f40a2330 0x7fa0f40a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa103fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117490 0x558fc2117870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa103fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117490 0x558fc2117870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40672/0 (socket says 172.21.15.94:40672) 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa103fff640 1 -- 172.21.15.94:0/3264098468 learned_addr learned my addr 172.21.15.94:0/3264098468 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa102ffd640 1 --2- 172.21.15.94:0/3264098468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2120990 0x558fc2122d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa1037fe640 1 --2- 172.21.15.94:0/3264098468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2117db0 0x558fc2120450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa103fff640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa0f40a2330 0x7fa0f40a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa100ff9640 1 -- 172.21.15.94:0/3404415870 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa0f00cdfa0 con 0x7fa0fc005060 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.317+0000 7fa103fff640 1 -- 172.21.15.94:0/3264098468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2120990 msgr2=0x558fc2122d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.318+0000 7fa103fff640 1 --2- 172.21.15.94:0/3264098468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2120990 0x558fc2122d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.318+0000 7fa103fff640 1 -- 172.21.15.94:0/3264098468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2117db0 msgr2=0x558fc2120450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.318+0000 7fa103fff640 1 --2- 172.21.15.94:0/3264098468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2117db0 0x558fc2120450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.318+0000 7fa103fff640 1 -- 172.21.15.94:0/3264098468 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fc203d570 con 0x558fc2117490 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.318+0000 7fa103fff640 1 --2- 172.21.15.94:0/3264098468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117490 0x558fc2117870 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800bd20 tx=0x7fa0f80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=957421f0a0f245dd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.318+0000 7fa103fff640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa0f40a2330 0x7fa0f40a47f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8003ec0 tx=0x7fa0f801d000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.318+0000 7fa0edffb640 1 -- 172.21.15.94:0/3264098468 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8005230 con 0x558fc2117490 2024-02-17T12:10:31.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.318+0000 7fa0edffb640 1 -- 172.21.15.94:0/3264098468 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0f8004c80 con 0x558fc2117490 2024-02-17T12:10:31.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.318+0000 7fa0edffb640 1 -- 172.21.15.94:0/3264098468 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80107a0 con 0x558fc2117490 2024-02-17T12:10:31.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.319+0000 7fa10cbce740 1 -- 172.21.15.94:0/3264098468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117490 msgr2=0x558fc2117870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:31.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.319+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3264098468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117490 0x558fc2117870 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800bd20 tx=0x7fa0f80039c0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.319+0000 7fa10cbce740 1 -- 172.21.15.94:0/3264098468 shutdown_connections 2024-02-17T12:10:31.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.319+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3264098468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2120990 0x558fc2122d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.319+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3264098468 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2117db0 0x558fc2120450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.319+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3264098468 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117490 0x558fc2117870 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.319+0000 7fa10cbce740 1 -- 172.21.15.94:0/3264098468 >> 172.21.15.94:0/3264098468 conn(0x558fc2111bc0 msgr2=0x558fc2111fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:31.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.319+0000 7fa10cbce740 1 -- 172.21.15.94:0/3264098468 shutdown_connections 2024-02-17T12:10:31.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.319+0000 7fa10cbce740 1 -- 172.21.15.94:0/3264098468 wait complete. 2024-02-17T12:10:31.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.319+0000 7fa10cbce740 1 -- start start 2024-02-17T12:10:31.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.320+0000 7fa10cbce740 1 Processor -- start 2024-02-17T12:10:31.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.320+0000 7fa10cbce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117490 0x558fc21203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.320+0000 7fa10cbce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117db0 0x558fc211d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.320+0000 7fa10cbce740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2120990 0x558fc211de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.320+0000 7fa10cbce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc1d52430 con 0x558fc2117db0 2024-02-17T12:10:31.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.320+0000 7fa10cbce740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc1d0ee00 con 0x558fc2120990 2024-02-17T12:10:31.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.320+0000 7fa10cbce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc2003fd0 con 0x558fc2117490 2024-02-17T12:10:31.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.320+0000 7fa10cbce740 10 client.0 _ll_register_callbacks cb 0x558fc2059fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:31.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.320+0000 7fa102ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117db0 0x558fc211d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.320+0000 7fa102ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117db0 0x558fc211d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40684/0 (socket says 172.21.15.94:40684) 2024-02-17T12:10:31.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.320+0000 7fa103fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2120990 0x558fc211de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.320+0000 7fa1037fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117490 0x558fc21203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.320+0000 7fa102ffd640 1 -- 172.21.15.94:0/3365177893 learned_addr learned my addr 172.21.15.94:0/3365177893 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:31.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.321+0000 7fa103fff640 1 -- 172.21.15.94:0/3365177893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117490 msgr2=0x558fc21203c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:31.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.321+0000 7fa103fff640 1 --2- 172.21.15.94:0/3365177893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117490 0x558fc21203c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.321+0000 7fa103fff640 1 -- 172.21.15.94:0/3365177893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117db0 msgr2=0x558fc211d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:31.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.321+0000 7fa103fff640 1 --2- 172.21.15.94:0/3365177893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117db0 0x558fc211d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.321+0000 7fa103fff640 1 -- 172.21.15.94:0/3365177893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f800b900 con 0x558fc2120990 2024-02-17T12:10:31.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.321+0000 7fa102ffd640 1 --2- 172.21.15.94:0/3365177893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117db0 0x558fc211d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:31.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.321+0000 7fa0e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:10:31.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.321+0000 7fa103fff640 1 --2- 172.21.15.94:0/3365177893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2120990 0x558fc211de90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80056f0 tx=0x7fa0f800b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db24b85122f26d56 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:31.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.321+0000 7fa0e2ffd640 1 -- 172.21.15.94:0/3365177893 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8005030 con 0x558fc2120990 2024-02-17T12:10:31.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.321+0000 7fa10cbce740 1 -- 172.21.15.94:0/3365177893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fc1d7e7e0 con 0x558fc2120990 2024-02-17T12:10:31.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.321+0000 7fa0e2ffd640 1 -- 172.21.15.94:0/3365177893 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa0f8005190 con 0x558fc2120990 2024-02-17T12:10:31.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.321+0000 7fa0e2ffd640 1 -- 172.21.15.94:0/3365177893 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0f8005350 con 0x558fc2120990 2024-02-17T12:10:31.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.322+0000 7fa10cbce740 10 client.28145 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:31.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.322+0000 7fa10cbce740 1 -- 172.21.15.94:0/3365177893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558fc207d610 con 0x558fc2120990 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.322+0000 7fa0e2ffd640 1 -- 172.21.15.94:0/3365177893 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558fc1d7e7e0 con 0x558fc2120990 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.322+0000 7fa0e2ffd640 1 -- 172.21.15.94:0/3365177893 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa0f80294a0 con 0x558fc2120990 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.322+0000 7fa10cbce740 10 client.28145 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.322+0000 7fa10cbce740 10 client.28145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.322+0000 7fa10cbce740 20 client.28145 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.322+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3365177893 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558fc2130ac0 0x558fc2132ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.322+0000 7fa10cbce740 4 client.28145 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.322+0000 7fa10cbce740 1 -- 172.21.15.94:0/3365177893 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fc1dbb720 con 0x558fc2130ac0 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.323+0000 7fa1037fe640 1 --2- 172.21.15.94:0/3365177893 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558fc2130ac0 0x558fc2132ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.323+0000 7fa0e2ffd640 10 client.28145 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.323+0000 7fa1037fe640 1 --2- 172.21.15.94:0/3365177893 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558fc2130ac0 0x558fc2132ee0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f06da9379f8f17 server_cookie=c6e8f01a32361209 in_seq=0 out_seq=0 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa0e2ffd640 1 -- 172.21.15.94:0/3365177893 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558fc1dbb720 con 0x558fc2130ac0 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa0e2ffd640 10 client.28145 handle_command_reply: tid=0 2024-02-17T12:10:31.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa10cbce740 1 client.28145 shutdown 2024-02-17T12:10:31.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa10cbce740 1 -- 172.21.15.94:0/3365177893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2120990 msgr2=0x558fc211de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:31.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3365177893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2120990 0x558fc211de90 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80056f0 tx=0x7fa0f800b7c0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa10cbce740 1 -- 172.21.15.94:0/3365177893 shutdown_connections 2024-02-17T12:10:31.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3365177893 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558fc2130ac0 0x558fc2132ee0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa0e2ffd640 0 client.28145 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:31.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3365177893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2120990 0x558fc211de90 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3365177893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117db0 0x558fc211d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3365177893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117490 0x558fc21203c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa10cbce740 1 -- 172.21.15.94:0/3365177893 >> 172.21.15.94:0/3365177893 conn(0x558fc2111bc0 msgr2=0x558fc2113910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:31.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa10cbce740 1 -- 172.21.15.94:0/3365177893 shutdown_connections 2024-02-17T12:10:31.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa10cbce740 1 -- 172.21.15.94:0/3365177893 wait complete. 2024-02-17T12:10:31.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.324+0000 7fa10cbce740 20 client.28145 trim_cache size 0 max 16384 2024-02-17T12:10:31.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.344+0000 7fa10cbce740 1 Processor -- start 2024-02-17T12:10:31.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.344+0000 7fa10cbce740 1 -- start start 2024-02-17T12:10:31.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.344+0000 7fa10cbce740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2117490 0x558fc211fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.344+0000 7fa10cbce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117db0 0x558fc21203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.344+0000 7fa10cbce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2120990 0x558fc211c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.344+0000 7fa10cbce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc212d5f0 con 0x558fc2120990 2024-02-17T12:10:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.344+0000 7fa10cbce740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc211e490 con 0x558fc2117490 2024-02-17T12:10:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.344+0000 7fa10cbce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc211eaa0 con 0x558fc2117db0 2024-02-17T12:10:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa1037fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2117490 0x558fc211fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa1037fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2117490 0x558fc211fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41778/0 (socket says 172.21.15.94:41778) 2024-02-17T12:10:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa1037fe640 1 -- 172.21.15.94:0/3422390542 learned_addr learned my addr 172.21.15.94:0/3422390542 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa102ffd640 1 --2- 172.21.15.94:0/3422390542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117db0 0x558fc21203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa103fff640 1 --2- 172.21.15.94:0/3422390542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2120990 0x558fc211c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa102ffd640 1 -- 172.21.15.94:0/3422390542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2117490 msgr2=0x558fc211fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa102ffd640 1 --2- 172.21.15.94:0/3422390542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2117490 0x558fc211fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa102ffd640 1 -- 172.21.15.94:0/3422390542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2120990 msgr2=0x558fc211c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa102ffd640 1 --2- 172.21.15.94:0/3422390542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2120990 0x558fc211c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa102ffd640 1 -- 172.21.15.94:0/3422390542 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fc2019ac0 con 0x558fc2117db0 2024-02-17T12:10:31.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa1037fe640 1 --2- 172.21.15.94:0/3422390542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2117490 0x558fc211fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:31.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa102ffd640 1 --2- 172.21.15.94:0/3422390542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117db0 0x558fc21203c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa0f009a6a0 tx=0x7fa0f0085040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4243e60b66596abf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:31.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa0ed7fa640 1 -- 172.21.15.94:0/3422390542 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f009a300 con 0x558fc2117db0 2024-02-17T12:10:31.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa0ed7fa640 1 -- 172.21.15.94:0/3422390542 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0f009a9a0 con 0x558fc2117db0 2024-02-17T12:10:31.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.345+0000 7fa0ed7fa640 1 -- 172.21.15.94:0/3422390542 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f009ac00 con 0x558fc2117db0 2024-02-17T12:10:31.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.346+0000 7fa10cbce740 1 -- 172.21.15.94:0/3422390542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117db0 msgr2=0x558fc21203c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:31.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.346+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3422390542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117db0 0x558fc21203c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa0f009a6a0 tx=0x7fa0f0085040 comp rx=0 tx=0).stop 2024-02-17T12:10:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.347+0000 7fa10cbce740 1 -- 172.21.15.94:0/3422390542 shutdown_connections 2024-02-17T12:10:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.347+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3422390542 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2120990 0x558fc211c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.347+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3422390542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117db0 0x558fc21203c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.347+0000 7fa10cbce740 1 --2- 172.21.15.94:0/3422390542 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2117490 0x558fc211fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.347+0000 7fa10cbce740 1 -- 172.21.15.94:0/3422390542 >> 172.21.15.94:0/3422390542 conn(0x558fc2111bc0 msgr2=0x558fc210c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:31.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.347+0000 7fa10cbce740 1 -- 172.21.15.94:0/3422390542 shutdown_connections 2024-02-17T12:10:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.347+0000 7fa10cbce740 1 -- 172.21.15.94:0/3422390542 wait complete. 2024-02-17T12:10:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.347+0000 7fa10cbce740 1 -- start start 2024-02-17T12:10:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa10cbce740 1 Processor -- start 2024-02-17T12:10:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa10cbce740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117490 0x558fc21241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa10cbce740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117db0 0x558fc2124710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa10cbce740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2120990 0x558fc2124c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa10cbce740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc2186fa0 con 0x558fc2117490 2024-02-17T12:10:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa10cbce740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc21870e0 con 0x558fc2120990 2024-02-17T12:10:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa10cbce740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fc2187220 con 0x558fc2117db0 2024-02-17T12:10:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa10cbce740 10 client.0 _ll_register_callbacks cb 0x7fa0f8004170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa1037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117490 0x558fc21241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa102ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117db0 0x558fc2124710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa1037fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117490 0x558fc21241d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40698/0 (socket says 172.21.15.94:40698) 2024-02-17T12:10:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa103fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2120990 0x558fc2124c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa1037fe640 1 -- 172.21.15.94:0/4042854223 learned_addr learned my addr 172.21.15.94:0/4042854223 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.348+0000 7fa102ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117db0 0x558fc2124710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47680/0 (socket says 172.21.15.94:47680) 2024-02-17T12:10:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa103fff640 1 -- 172.21.15.94:0/4042854223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117db0 msgr2=0x558fc2124710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa103fff640 1 --2- 172.21.15.94:0/4042854223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117db0 0x558fc2124710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa103fff640 1 -- 172.21.15.94:0/4042854223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117490 msgr2=0x558fc21241d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa103fff640 1 --2- 172.21.15.94:0/4042854223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117490 0x558fc21241d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa103fff640 1 -- 172.21.15.94:0/4042854223 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc0a55e0 con 0x558fc2120990 2024-02-17T12:10:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa102ffd640 1 --2- 172.21.15.94:0/4042854223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117db0 0x558fc2124710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa103fff640 1 --2- 172.21.15.94:0/4042854223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2120990 0x558fc2124c50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8005880 tx=0x7fa0f8029650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b142f424807f0c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:31.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa0e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa0e2ffd640 1 -- 172.21.15.94:0/4042854223 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f80047f0 con 0x558fc2120990 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa0e2ffd640 1 -- 172.21.15.94:0/4042854223 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa0f8004950 con 0x558fc2120990 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa10cbce740 1 -- 172.21.15.94:0/4042854223 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0f8004c80 con 0x558fc2120990 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa0e2ffd640 1 -- 172.21.15.94:0/4042854223 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0f8004b10 con 0x558fc2120990 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa10cbce740 10 client.28151 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.349+0000 7fa10cbce740 1 -- 172.21.15.94:0/4042854223 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fa0fc12ebd0 con 0x558fc2120990 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.350+0000 7fa0e2ffd640 1 -- 172.21.15.94:0/4042854223 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa0f8004c80 con 0x558fc2120990 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.350+0000 7fa0e2ffd640 1 -- 172.21.15.94:0/4042854223 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa0f80107c0 con 0x558fc2120990 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.350+0000 7fa10cbce740 10 client.28151 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.350+0000 7fa10cbce740 10 client.28151 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.350+0000 7fa10cbce740 20 client.28151 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.350+0000 7fa10cbce740 1 --2- 172.21.15.94:0/4042854223 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558fc21886c0 0x558fc218aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.350+0000 7fa10cbce740 4 client.28151 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:10:31.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.350+0000 7fa10cbce740 1 -- 172.21.15.94:0/4042854223 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558fc1e0a890 con 0x558fc21886c0 2024-02-17T12:10:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.350+0000 7fa1037fe640 1 --2- 172.21.15.94:0/4042854223 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558fc21886c0 0x558fc218aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.350+0000 7fa0e2ffd640 10 client.28151 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.350+0000 7fa1037fe640 1 --2- 172.21.15.94:0/4042854223 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558fc21886c0 0x558fc218aab0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6efada2a725b505 server_cookie=393d7fff84477c5e in_seq=0 out_seq=0 2024-02-17T12:10:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa0e2ffd640 1 -- 172.21.15.94:0/4042854223 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558fc1e0a890 con 0x558fc21886c0 2024-02-17T12:10:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa0e2ffd640 10 client.28151 handle_command_reply: tid=0 2024-02-17T12:10:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa10cbce740 1 client.28151 shutdown 2024-02-17T12:10:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa10cbce740 1 -- 172.21.15.94:0/4042854223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2120990 msgr2=0x558fc2124c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa10cbce740 1 --2- 172.21.15.94:0/4042854223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2120990 0x558fc2124c50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8005880 tx=0x7fa0f8029650 comp rx=0 tx=0).stop 2024-02-17T12:10:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa10cbce740 1 -- 172.21.15.94:0/4042854223 shutdown_connections 2024-02-17T12:10:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa10cbce740 1 --2- 172.21.15.94:0/4042854223 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558fc21886c0 0x558fc218aab0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa10cbce740 1 --2- 172.21.15.94:0/4042854223 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558fc2120990 0x558fc2124c50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa0e2ffd640 0 client.28151 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:31.364 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa10cbce740 1 --2- 172.21.15.94:0/4042854223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558fc2117db0 0x558fc2124710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa10cbce740 1 --2- 172.21.15.94:0/4042854223 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558fc2117490 0x558fc21241d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa10cbce740 1 -- 172.21.15.94:0/4042854223 >> 172.21.15.94:0/4042854223 conn(0x558fc2111bc0 msgr2=0x558fc211cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa10cbce740 1 -- 172.21.15.94:0/4042854223 shutdown_connections 2024-02-17T12:10:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa10cbce740 1 -- 172.21.15.94:0/4042854223 wait complete. 2024-02-17T12:10:31.365 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.351+0000 7fa10cbce740 20 client.28151 trim_cache size 0 max 16384 2024-02-17T12:10:31.365 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:10:31.365 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:10:31.365 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:10:31.365 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:10:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.352+0000 7fa109f27640 1 -- 172.21.15.94:0/3404415870 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa0f40a2330 msgr2=0x7fa0f40a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:31.366 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.352+0000 7fa109f27640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa0f40a2330 0x7fa0f40a47f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8003ec0 tx=0x7fa0f801d000 comp rx=0 tx=0).stop 2024-02-17T12:10:31.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.352+0000 7fa109f27640 1 -- 172.21.15.94:0/3404415870 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0fc005060 msgr2=0x7fa0fc097670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:31.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.352+0000 7fa109f27640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0fc005060 0x7fa0fc097670 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000d950 tx=0x7fa0f000de10 comp rx=0 tx=0).stop 2024-02-17T12:10:31.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.352+0000 7fa109f27640 1 -- 172.21.15.94:0/3404415870 shutdown_connections 2024-02-17T12:10:31.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.352+0000 7fa109f27640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa0f40a2330 0x7fa0f40a47f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.352+0000 7fa109f27640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0fc0a2a00 0x7fa0fc097bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.352+0000 7fa109f27640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa0fc005060 0x7fa0fc097670 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.352+0000 7fa109f27640 1 --2- 172.21.15.94:0/3404415870 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa0fc004740 0x7fa0fc097130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.352+0000 7fa109f27640 1 -- 172.21.15.94:0/3404415870 >> 172.21.15.94:0/3404415870 conn(0x7fa0fc00a740 msgr2=0x7fa0fc0a3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:31.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.353+0000 7fa109f27640 1 -- 172.21.15.94:0/3404415870 shutdown_connections 2024-02-17T12:10:31.367 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.354+0000 7fa109f27640 1 -- 172.21.15.94:0/3404415870 wait complete. 2024-02-17T12:10:31.367 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:31 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:10:31.367 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:31 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:10:31.368 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:31 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:31.368 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:31 smithi094 ceph-mon[32499]: pgmap v387: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.1 MiB/s wr, 537 op/s 2024-02-17T12:10:31.370 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:10:31.370 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:10:31.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:31 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:10:31.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:31 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:10:31.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:31 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:31.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:31 smithi189 ceph-mon[36588]: pgmap v387: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.1 MiB/s wr, 537 op/s 2024-02-17T12:10:31.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:31 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:10:31.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:31 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:10:31.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:31 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:10:31.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:31 smithi125 ceph-mon[36819]: pgmap v387: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.1 MiB/s wr, 537 op/s 2024-02-17T12:10:31.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.520+0000 7efd218d4640 1 -- 172.21.15.94:0/2048742869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd1c10f9b0 msgr2=0x7efd1c10d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:31.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.520+0000 7efd218d4640 1 --2- 172.21.15.94:0/2048742869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd1c10f9b0 0x7efd1c10d4a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7efd14009cd0 tx=0x7efd1402c100 comp rx=0 tx=0).stop 2024-02-17T12:10:31.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.527+0000 7efd218d4640 1 -- 172.21.15.94:0/2048742869 shutdown_connections 2024-02-17T12:10:31.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.527+0000 7efd218d4640 1 --2- 172.21.15.94:0/2048742869 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd1c11a960 0x7efd1c11cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.527+0000 7efd218d4640 1 --2- 172.21.15.94:0/2048742869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c10d9e0 0x7efd1c10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.527+0000 7efd218d4640 1 --2- 172.21.15.94:0/2048742869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd1c10f9b0 0x7efd1c10d4a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.529 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.527+0000 7efd218d4640 1 -- 172.21.15.94:0/2048742869 >> 172.21.15.94:0/2048742869 conn(0x7efd1c06c270 msgr2=0x7efd1c06c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:31.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.532+0000 7efd218d4640 1 -- 172.21.15.94:0/2048742869 shutdown_connections 2024-02-17T12:10:31.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.536+0000 7efd218d4640 1 -- 172.21.15.94:0/2048742869 wait complete. 2024-02-17T12:10:31.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.536+0000 7efd218d4640 1 Processor -- start 2024-02-17T12:10:31.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.536+0000 7efd218d4640 1 -- start start 2024-02-17T12:10:31.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.537+0000 7efd218d4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd1c10d9e0 0x7efd1c11a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.537+0000 7efd218d4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd1c10f9b0 0x7efd1c114480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.537+0000 7efd218d4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c11a960 0x7efd1c114bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.537+0000 7efd218d4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c1bb080 con 0x7efd1c10f9b0 2024-02-17T12:10:31.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.537+0000 7efd218d4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c1bb1c0 con 0x7efd1c10d9e0 2024-02-17T12:10:31.540 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.537+0000 7efd218d4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c1bb300 con 0x7efd1c11a960 2024-02-17T12:10:31.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.537+0000 7efd1b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c11a960 0x7efd1c114bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.537+0000 7efd1b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c11a960 0x7efd1c114bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47706/0 (socket says 172.21.15.94:47706) 2024-02-17T12:10:31.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.537+0000 7efd1b7fe640 1 -- 172.21.15.94:0/625866767 learned_addr learned my addr 172.21.15.94:0/625866767 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:31.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.538+0000 7efd1b7fe640 1 -- 172.21.15.94:0/625866767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd1c10d9e0 msgr2=0x7efd1c11a3c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:10:31.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.538+0000 7efd1b7fe640 1 --2- 172.21.15.94:0/625866767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd1c10d9e0 0x7efd1c11a3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.538+0000 7efd1b7fe640 1 -- 172.21.15.94:0/625866767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd1c10f9b0 msgr2=0x7efd1c114480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:31.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.538+0000 7efd1b7fe640 1 --2- 172.21.15.94:0/625866767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd1c10f9b0 0x7efd1c114480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.538+0000 7efd1b7fe640 1 -- 172.21.15.94:0/625866767 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd14009900 con 0x7efd1c11a960 2024-02-17T12:10:31.541 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.538+0000 7efd1b7fe640 1 --2- 172.21.15.94:0/625866767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c11a960 0x7efd1c114bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efd1000b3f0 tx=0x7efd1000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:31.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.541+0000 7efd208d2640 1 -- 172.21.15.94:0/625866767 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd10011830 con 0x7efd1c11a960 2024-02-17T12:10:31.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.541+0000 7efd208d2640 1 -- 172.21.15.94:0/625866767 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7efd10011e70 con 0x7efd1c11a960 2024-02-17T12:10:31.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.541+0000 7efd208d2640 1 -- 172.21.15.94:0/625866767 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1000f2e0 con 0x7efd1c11a960 2024-02-17T12:10:31.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.544+0000 7efd218d4640 1 -- 172.21.15.94:0/625866767 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1c1bb500 con 0x7efd1c11a960 2024-02-17T12:10:31.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.544+0000 7efd218d4640 1 -- 172.21.15.94:0/625866767 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c1bb930 con 0x7efd1c11a960 2024-02-17T12:10:31.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.545+0000 7efd218d4640 1 -- 172.21.15.94:0/625866767 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1c06d510 con 0x7efd1c11a960 2024-02-17T12:10:31.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.549+0000 7efd208d2640 1 -- 172.21.15.94:0/625866767 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7efd10010380 con 0x7efd1c11a960 2024-02-17T12:10:31.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.549+0000 7efd208d2640 1 --2- 172.21.15.94:0/625866767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efd040a2440 0x7efd040a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:31.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.550+0000 7efd208d2640 1 -- 172.21.15.94:0/625866767 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7efd100cd710 con 0x7efd1c11a960 2024-02-17T12:10:31.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.550+0000 7efd1affd640 1 --2- 172.21.15.94:0/625866767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efd040a2440 0x7efd040a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:31.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.551+0000 7efd1affd640 1 --2- 172.21.15.94:0/625866767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efd040a2440 0x7efd040a4900 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd14006010 tx=0x7efd14009540 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:31.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.556+0000 7efd208d2640 1 -- 172.21.15.94:0/625866767 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efd10096490 con 0x7efd1c11a960 2024-02-17T12:10:31.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.894+0000 7efd218d4640 1 -- 172.21.15.94:0/625866767 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd1c115960 con 0x7efd1c11a960 2024-02-17T12:10:31.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.897+0000 7efd208d2640 1 -- 172.21.15.94:0/625866767 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7efd10015090 con 0x7efd1c11a960 2024-02-17T12:10:31.899 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:10:31.900 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:10:31.900 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:10:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.901+0000 7efd218d4640 1 -- 172.21.15.94:0/625866767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efd040a2440 msgr2=0x7efd040a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.901+0000 7efd218d4640 1 --2- 172.21.15.94:0/625866767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efd040a2440 0x7efd040a4900 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd14006010 tx=0x7efd14009540 comp rx=0 tx=0).stop 2024-02-17T12:10:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.901+0000 7efd218d4640 1 -- 172.21.15.94:0/625866767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c11a960 msgr2=0x7efd1c114bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.901+0000 7efd218d4640 1 --2- 172.21.15.94:0/625866767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c11a960 0x7efd1c114bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efd1000b3f0 tx=0x7efd1000b8b0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.901+0000 7efd218d4640 1 -- 172.21.15.94:0/625866767 shutdown_connections 2024-02-17T12:10:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.901+0000 7efd218d4640 1 --2- 172.21.15.94:0/625866767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7efd040a2440 0x7efd040a4900 secure :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7efd14006010 tx=0x7efd14009540 comp rx=0 tx=0).stop 2024-02-17T12:10:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.901+0000 7efd218d4640 1 --2- 172.21.15.94:0/625866767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd1c11a960 0x7efd1c114bf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.901+0000 7efd218d4640 1 --2- 172.21.15.94:0/625866767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7efd1c10f9b0 0x7efd1c114480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.901+0000 7efd218d4640 1 --2- 172.21.15.94:0/625866767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7efd1c10d9e0 0x7efd1c11a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:31.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.901+0000 7efd218d4640 1 -- 172.21.15.94:0/625866767 >> 172.21.15.94:0/625866767 conn(0x7efd1c06c270 msgr2=0x7efd1c071c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:31.904 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.902+0000 7efd218d4640 1 -- 172.21.15.94:0/625866767 shutdown_connections 2024-02-17T12:10:31.905 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:31.903+0000 7efd218d4640 1 -- 172.21.15.94:0/625866767 wait complete. 2024-02-17T12:10:31.924 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:10:31.924 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:10:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.051+0000 7f9ff0c8f640 1 -- 172.21.15.94:0/839221193 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fec1017e0 msgr2=0x7f9fec104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.051+0000 7f9ff0c8f640 1 --2- 172.21.15.94:0/839221193 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fec1017e0 0x7f9fec104d10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009930 tx=0x7f9fe002bc30 comp rx=0 tx=0).stop 2024-02-17T12:10:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.051+0000 7f9ff0c8f640 1 -- 172.21.15.94:0/839221193 shutdown_connections 2024-02-17T12:10:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.051+0000 7f9ff0c8f640 1 --2- 172.21.15.94:0/839221193 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fec1053f0 0x7f9fec10cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.051+0000 7f9ff0c8f640 1 --2- 172.21.15.94:0/839221193 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fec1017e0 0x7f9fec104d10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.051+0000 7f9ff0c8f640 1 --2- 172.21.15.94:0/839221193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fec100e30 0x7f9fec101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.051+0000 7f9ff0c8f640 1 -- 172.21.15.94:0/839221193 >> 172.21.15.94:0/839221193 conn(0x7f9fec075430 msgr2=0x7f9fec075840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.051+0000 7f9ff0c8f640 1 -- 172.21.15.94:0/839221193 shutdown_connections 2024-02-17T12:10:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.051+0000 7f9ff0c8f640 1 -- 172.21.15.94:0/839221193 wait complete. 2024-02-17T12:10:32.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.052+0000 7f9ff0c8f640 1 Processor -- start 2024-02-17T12:10:32.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.052+0000 7f9ff0c8f640 1 -- start start 2024-02-17T12:10:32.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9ff0c8f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fec100e30 0x7f9fec19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9ff0c8f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fec1053f0 0x7f9fec19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9ff0c8f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fec1998b0 0x7f9fec199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9ff0c8f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec19a2a0 con 0x7f9fec100e30 2024-02-17T12:10:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9ff0c8f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec19a410 con 0x7f9fec1998b0 2024-02-17T12:10:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9ff0c8f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fec19a5d0 con 0x7f9fec1053f0 2024-02-17T12:10:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9fea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fec100e30 0x7f9fec19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9fea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fec100e30 0x7f9fec19f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40746/0 (socket says 172.21.15.94:40746) 2024-02-17T12:10:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9fead9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fec1998b0 0x7f9fec199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9fea59c640 1 -- 172.21.15.94:0/1279009768 learned_addr learned my addr 172.21.15.94:0/1279009768 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9fead9d640 1 -- 172.21.15.94:0/1279009768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fec1053f0 msgr2=0x7f9fec19fbf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-17T12:10:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9fead9d640 1 --2- 172.21.15.94:0/1279009768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fec1053f0 0x7f9fec19fbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9fead9d640 1 -- 172.21.15.94:0/1279009768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fec100e30 msgr2=0x7f9fec19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9fead9d640 1 --2- 172.21.15.94:0/1279009768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fec100e30 0x7f9fec19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9fead9d640 1 -- 172.21.15.94:0/1279009768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe4009840 con 0x7f9fec1998b0 2024-02-17T12:10:32.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.053+0000 7f9fead9d640 1 --2- 172.21.15.94:0/1279009768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fec1998b0 0x7f9fec199d60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400ec50 tx=0x7f9fe400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.054+0000 7f9fdb7fe640 1 -- 172.21.15.94:0/1279009768 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4003b70 con 0x7f9fec1998b0 2024-02-17T12:10:32.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.054+0000 7f9fdb7fe640 1 -- 172.21.15.94:0/1279009768 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fe400ce70 con 0x7f9fec1998b0 2024-02-17T12:10:32.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.054+0000 7f9fdb7fe640 1 -- 172.21.15.94:0/1279009768 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4010830 con 0x7f9fec1998b0 2024-02-17T12:10:32.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.054+0000 7f9ff0c8f640 1 -- 172.21.15.94:0/1279009768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe0009560 con 0x7f9fec1998b0 2024-02-17T12:10:32.058 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.054+0000 7f9ff0c8f640 1 -- 172.21.15.94:0/1279009768 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fec1a6840 con 0x7f9fec1998b0 2024-02-17T12:10:32.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.056+0000 7f9fdb7fe640 1 -- 172.21.15.94:0/1279009768 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9fe4004020 con 0x7f9fec1998b0 2024-02-17T12:10:32.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.056+0000 7f9fdb7fe640 1 --2- 172.21.15.94:0/1279009768 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fd40a2230 0x7f9fd40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.057+0000 7f9fea59c640 1 --2- 172.21.15.94:0/1279009768 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fd40a2230 0x7f9fd40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.057+0000 7f9fdb7fe640 1 -- 172.21.15.94:0/1279009768 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9fe4014070 con 0x7f9fec1998b0 2024-02-17T12:10:32.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.057+0000 7f9fea59c640 1 --2- 172.21.15.94:0/1279009768 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fd40a2230 0x7f9fd40a46f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9fec104f50 tx=0x7f9fdc005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.057+0000 7f9ff385d740 1 Processor -- start 2024-02-17T12:10:32.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.057+0000 7f9ff385d740 1 -- start start 2024-02-17T12:10:32.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9ff385d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6eb250 0x557dff6eb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9ff385d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6ebb70 0x557dff6f4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9ff385d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 0x557dff6f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9ff385d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff6a6960 con 0x557dff6ebb70 2024-02-17T12:10:32.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9ff385d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff65a310 con 0x557dff6eb250 2024-02-17T12:10:32.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9ff385d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff657840 con 0x557dff6f4750 2024-02-17T12:10:32.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9fe9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6ebb70 0x557dff6f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9fe9d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6ebb70 0x557dff6f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40762/0 (socket says 172.21.15.94:40762) 2024-02-17T12:10:32.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9fe9d9b640 1 -- 172.21.15.94:0/3237045856 learned_addr learned my addr 172.21.15.94:0/3237045856 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9fead9d640 1 --2- 172.21.15.94:0/3237045856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 0x557dff6f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9fea59c640 1 --2- 172.21.15.94:0/3237045856 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6eb250 0x557dff6eb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9fead9d640 1 -- 172.21.15.94:0/3237045856 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6eb250 msgr2=0x557dff6eb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9fead9d640 1 --2- 172.21.15.94:0/3237045856 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6eb250 0x557dff6eb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9fead9d640 1 -- 172.21.15.94:0/3237045856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6ebb70 msgr2=0x557dff6f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9fead9d640 1 --2- 172.21.15.94:0/3237045856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6ebb70 0x557dff6f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9fead9d640 1 -- 172.21.15.94:0/3237045856 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dff615060 con 0x557dff6f4750 2024-02-17T12:10:32.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.058+0000 7f9fe9d9b640 1 --2- 172.21.15.94:0/3237045856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6ebb70 0x557dff6f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:32.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.059+0000 7f9fead9d640 1 --2- 172.21.15.94:0/3237045856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 0x557dff6f6b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40cbfd0 tx=0x7f9fe409b100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4511623ddc83cae server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.059+0000 7f9fd8ff9640 1 -- 172.21.15.94:0/3237045856 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe409b970 con 0x557dff6f4750 2024-02-17T12:10:32.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.059+0000 7f9fd8ff9640 1 -- 172.21.15.94:0/3237045856 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fe409b270 con 0x557dff6f4750 2024-02-17T12:10:32.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.059+0000 7f9fd8ff9640 1 -- 172.21.15.94:0/3237045856 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4004430 con 0x557dff6f4750 2024-02-17T12:10:32.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.060+0000 7f9ff385d740 1 -- 172.21.15.94:0/3237045856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 msgr2=0x557dff6f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.060+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3237045856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 0x557dff6f6b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40cbfd0 tx=0x7f9fe409b100 comp rx=0 tx=0).stop 2024-02-17T12:10:32.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.061+0000 7f9ff385d740 1 -- 172.21.15.94:0/3237045856 shutdown_connections 2024-02-17T12:10:32.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.061+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3237045856 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 0x557dff6f6b40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.061+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3237045856 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6ebb70 0x557dff6f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.061+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3237045856 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6eb250 0x557dff6eb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.061+0000 7f9ff385d740 1 -- 172.21.15.94:0/3237045856 >> 172.21.15.94:0/3237045856 conn(0x557dff6e5980 msgr2=0x557dff6e5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.061+0000 7f9ff385d740 1 -- 172.21.15.94:0/3237045856 shutdown_connections 2024-02-17T12:10:32.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.061+0000 7f9ff385d740 1 -- 172.21.15.94:0/3237045856 wait complete. 2024-02-17T12:10:32.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.062+0000 7f9ff385d740 1 -- start start 2024-02-17T12:10:32.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.062+0000 7f9ff385d740 1 Processor -- start 2024-02-17T12:10:32.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.062+0000 7f9ff385d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 0x557dff6f1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.062+0000 7f9ff385d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6ebb70 0x557dff6f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.062+0000 7f9ff385d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 0x557dff6f2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.062+0000 7f9ff385d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff653020 con 0x557dff6eb250 2024-02-17T12:10:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.062+0000 7f9fea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 0x557dff6f1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.062+0000 7f9fea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 0x557dff6f1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40770/0 (socket says 172.21.15.94:40770) 2024-02-17T12:10:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.062+0000 7f9fea59c640 1 -- 172.21.15.94:0/3064295776 learned_addr learned my addr 172.21.15.94:0/3064295776 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.062+0000 7f9ff385d740 1 -- 172.21.15.94:0/3064295776 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff326430 con 0x557dff6ebb70 2024-02-17T12:10:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.062+0000 7f9fe9d9b640 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6ebb70 0x557dff6f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.062+0000 7f9ff385d740 1 -- 172.21.15.94:0/3064295776 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff2e2e00 con 0x557dff6f4750 2024-02-17T12:10:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9ff385d740 10 client.0 _ll_register_callbacks cb 0x557dff5facb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9fead9d640 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 0x557dff6f2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9fea59c640 1 -- 172.21.15.94:0/3064295776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 msgr2=0x557dff6f2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9fea59c640 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 0x557dff6f2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9fea59c640 1 -- 172.21.15.94:0/3064295776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6ebb70 msgr2=0x557dff6f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9fea59c640 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6ebb70 0x557dff6f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9fea59c640 1 -- 172.21.15.94:0/3064295776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fec1a6840 con 0x557dff6eb250 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9fe9d9b640 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6ebb70 0x557dff6f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9fea59c640 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 0x557dff6f1890 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc01b900 tx=0x7f9fdc013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31e0db267b5b624 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9fd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9fd1ffb640 1 -- 172.21.15.94:0/3064295776 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc021040 con 0x557dff6eb250 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9ff385d740 1 -- 172.21.15.94:0/3064295776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557dff3527e0 con 0x557dff6eb250 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9fd1ffb640 1 -- 172.21.15.94:0/3064295776 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9fdc00ed40 con 0x557dff6eb250 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9fd1ffb640 1 -- 172.21.15.94:0/3064295776 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fdc01be80 con 0x557dff6eb250 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9ff385d740 10 client.18705 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.063+0000 7f9ff385d740 1 -- 172.21.15.94:0/3064295776 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557dff6516a0 con 0x557dff6eb250 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.064+0000 7f9fd1ffb640 1 -- 172.21.15.94:0/3064295776 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fdc033020 con 0x557dff6eb250 2024-02-17T12:10:32.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.064+0000 7f9fd1ffb640 1 -- 172.21.15.94:0/3064295776 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9fdc02e5e0 con 0x557dff6eb250 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.064+0000 7f9ff385d740 10 client.18705 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.064+0000 7f9ff385d740 10 client.18705 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.064+0000 7f9ff385d740 20 client.18705 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.064+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557dff7049c0 0x557dff706e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.064+0000 7f9ff385d740 4 client.18705 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.064+0000 7f9ff385d740 1 -- 172.21.15.94:0/3064295776 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557dff38f720 con 0x557dff7049c0 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.065+0000 7f9fe9d9b640 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557dff7049c0 0x557dff706e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.065+0000 7f9fe9d9b640 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557dff7049c0 0x557dff706e00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3857b6772bbaa223 server_cookie=c3d748cac67e0368 in_seq=0 out_seq=0 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.065+0000 7f9fd1ffb640 10 client.18705 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.066+0000 7f9fd1ffb640 1 -- 172.21.15.94:0/3064295776 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557dff38f720 con 0x557dff7049c0 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.066+0000 7f9fd1ffb640 10 client.18705 handle_command_reply: tid=0 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.066+0000 7f9ff385d740 1 client.18705 shutdown 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.066+0000 7f9ff385d740 1 -- 172.21.15.94:0/3064295776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 msgr2=0x557dff6f1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.066+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 0x557dff6f1890 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc01b900 tx=0x7f9fdc013040 comp rx=0 tx=0).stop 2024-02-17T12:10:32.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.066+0000 7f9ff385d740 1 -- 172.21.15.94:0/3064295776 shutdown_connections 2024-02-17T12:10:32.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.066+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 0x557dff6f2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.066+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557dff7049c0 0x557dff706e00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.066+0000 7f9fd1ffb640 0 client.18705 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:10:32.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.066+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6ebb70 0x557dff6f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.066+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3064295776 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 0x557dff6f1890 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.066+0000 7f9ff385d740 1 -- 172.21.15.94:0/3064295776 >> 172.21.15.94:0/3064295776 conn(0x557dff6e5980 msgr2=0x557dff6e76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.067+0000 7f9ff385d740 1 -- 172.21.15.94:0/3064295776 shutdown_connections 2024-02-17T12:10:32.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.067+0000 7f9ff385d740 1 -- 172.21.15.94:0/3064295776 wait complete. 2024-02-17T12:10:32.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.067+0000 7f9ff385d740 20 client.18705 trim_cache size 0 max 16384 2024-02-17T12:10:32.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.084+0000 7f9ff385d740 1 Processor -- start 2024-02-17T12:10:32.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.084+0000 7f9ff385d740 1 -- start start 2024-02-17T12:10:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.084+0000 7f9ff385d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6eb250 0x557dff6f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.084+0000 7f9ff385d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6ebb70 0x557dff6f2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.085+0000 7f9fea59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6eb250 0x557dff6f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.085+0000 7f9fea59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6eb250 0x557dff6f3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41890/0 (socket says 172.21.15.94:41890) 2024-02-17T12:10:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.085+0000 7f9fea59c640 1 -- 172.21.15.94:0/3724934934 learned_addr learned my addr 172.21.15.94:0/3724934934 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.085+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3724934934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6f4750 0x557dff6eeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.085+0000 7f9fe9d9b640 1 --2- 172.21.15.94:0/3724934934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6ebb70 0x557dff6f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.085+0000 7f9ff385d740 1 -- 172.21.15.94:0/3724934934 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff5d8130 con 0x557dff6f4750 2024-02-17T12:10:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.085+0000 7f9ff385d740 1 -- 172.21.15.94:0/3724934934 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff701710 con 0x557dff6eb250 2024-02-17T12:10:32.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.085+0000 7f9ff385d740 1 -- 172.21.15.94:0/3724934934 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff6e6bb0 con 0x557dff6ebb70 2024-02-17T12:10:32.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.089+0000 7f9fea59c640 1 -- 172.21.15.94:0/3724934934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6ebb70 msgr2=0x557dff6f2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.089+0000 7f9fea59c640 1 --2- 172.21.15.94:0/3724934934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6ebb70 0x557dff6f2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.089+0000 7f9fea59c640 1 -- 172.21.15.94:0/3724934934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6f4750 msgr2=0x557dff6eeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.089+0000 7f9fea59c640 1 --2- 172.21.15.94:0/3724934934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6f4750 0x557dff6eeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.089+0000 7f9fea59c640 1 -- 172.21.15.94:0/3724934934 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557dff5edc20 con 0x557dff6eb250 2024-02-17T12:10:32.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.089+0000 7f9fea59c640 1 --2- 172.21.15.94:0/3724934934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6eb250 0x557dff6f3fc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc016040 tx=0x7f9fdc01b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc485017ee20f6c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.093+0000 7f9fd3fff640 1 -- 172.21.15.94:0/3724934934 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc00e780 con 0x557dff6eb250 2024-02-17T12:10:32.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.093+0000 7f9fd3fff640 1 -- 172.21.15.94:0/3724934934 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fdc033030 con 0x557dff6eb250 2024-02-17T12:10:32.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.093+0000 7f9fd3fff640 1 -- 172.21.15.94:0/3724934934 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc005290 con 0x557dff6eb250 2024-02-17T12:10:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.094+0000 7f9ff385d740 1 -- 172.21.15.94:0/3724934934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6eb250 msgr2=0x557dff6f3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.094+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3724934934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6eb250 0x557dff6f3fc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc016040 tx=0x7f9fdc01b7a0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.098+0000 7f9ff385d740 1 -- 172.21.15.94:0/3724934934 shutdown_connections 2024-02-17T12:10:32.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.098+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3724934934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6f4750 0x557dff6eeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.098+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3724934934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6ebb70 0x557dff6f2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.098+0000 7f9ff385d740 1 --2- 172.21.15.94:0/3724934934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6eb250 0x557dff6f3fc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.098+0000 7f9ff385d740 1 -- 172.21.15.94:0/3724934934 >> 172.21.15.94:0/3724934934 conn(0x557dff6e5980 msgr2=0x557dff6ecaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.105+0000 7f9ff385d740 1 -- 172.21.15.94:0/3724934934 shutdown_connections 2024-02-17T12:10:32.110 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.108+0000 7f9ff385d740 1 -- 172.21.15.94:0/3724934934 wait complete. 2024-02-17T12:10:32.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.109+0000 7f9ff385d740 1 -- start start 2024-02-17T12:10:32.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.109+0000 7f9ff385d740 1 Processor -- start 2024-02-17T12:10:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.117+0000 7f9ff385d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 0x557dff6fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.117+0000 7f9ff385d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6ebb70 0x557dff6fad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.117+0000 7f9fea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 0x557dff6fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.118+0000 7f9fea59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 0x557dff6fa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40800/0 (socket says 172.21.15.94:40800) 2024-02-17T12:10:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.118+0000 7f9fea59c640 1 -- 172.21.15.94:0/1724397351 learned_addr learned my addr 172.21.15.94:0/1724397351 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.118+0000 7f9ff385d740 1 --2- 172.21.15.94:0/1724397351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 0x557dff754f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.118+0000 7f9ff385d740 1 -- 172.21.15.94:0/1724397351 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff755440 con 0x557dff6eb250 2024-02-17T12:10:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.118+0000 7f9ff385d740 1 -- 172.21.15.94:0/1724397351 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff7555b0 con 0x557dff6ebb70 2024-02-17T12:10:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.118+0000 7f9ff385d740 1 -- 172.21.15.94:0/1724397351 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557dff755770 con 0x557dff6f4750 2024-02-17T12:10:32.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.118+0000 7f9ff385d740 10 client.0 _ll_register_callbacks cb 0x7f9fe4004620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:32.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.121+0000 7f9fe9d9b640 1 --2- 172.21.15.94:0/1724397351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6ebb70 0x557dff6fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.122+0000 7f9fea59c640 1 -- 172.21.15.94:0/1724397351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 msgr2=0x557dff754f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:10:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.122+0000 7f9fea59c640 1 --2- 172.21.15.94:0/1724397351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 0x557dff754f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.122+0000 7f9fea59c640 1 -- 172.21.15.94:0/1724397351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6ebb70 msgr2=0x557dff6fad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.122+0000 7f9fea59c640 1 --2- 172.21.15.94:0/1724397351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6ebb70 0x557dff6fad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.122+0000 7f9fea59c640 1 -- 172.21.15.94:0/1724397351 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557dff6516a0 con 0x557dff6eb250 2024-02-17T12:10:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.122+0000 7f9fea59c640 1 --2- 172.21.15.94:0/1724397351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 0x557dff6fa7d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc02e9c0 tx=0x7f9fdc02ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48deedbd24465fd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.123+0000 7f9fd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:10:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.124+0000 7f9fd1ffb640 1 -- 172.21.15.94:0/1724397351 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc021030 con 0x557dff6eb250 2024-02-17T12:10:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.124+0000 7f9ff385d740 1 -- 172.21.15.94:0/1724397351 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fe409b270 con 0x557dff6eb250 2024-02-17T12:10:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.124+0000 7f9fd1ffb640 1 -- 172.21.15.94:0/1724397351 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9fdc034080 con 0x557dff6eb250 2024-02-17T12:10:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.124+0000 7f9fd1ffb640 1 -- 172.21.15.94:0/1724397351 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fdc00e780 con 0x557dff6eb250 2024-02-17T12:10:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.124+0000 7f9ff385d740 10 client.18708 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:32.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.124+0000 7f9ff385d740 1 -- 172.21.15.94:0/1724397351 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557dff6138c0 con 0x557dff6eb250 2024-02-17T12:10:32.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.126+0000 7f9fd1ffb640 1 -- 172.21.15.94:0/1724397351 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9fe409b270 con 0x557dff6eb250 2024-02-17T12:10:32.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.126+0000 7f9fd1ffb640 1 -- 172.21.15.94:0/1724397351 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9fdc0048f0 con 0x557dff6eb250 2024-02-17T12:10:32.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.128+0000 7f9ff385d740 10 client.18708 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:32.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.128+0000 7f9ff385d740 10 client.18708 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:10:32.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.128+0000 7f9ff385d740 20 client.18708 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:32.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.128+0000 7f9ff385d740 1 --2- 172.21.15.94:0/1724397351 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557dff75cb80 0x557dff75ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.128+0000 7f9ff385d740 4 client.18708 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:10:32.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.128+0000 7f9ff385d740 1 -- 172.21.15.94:0/1724397351 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557dff3de890 con 0x557dff75cb80 2024-02-17T12:10:32.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.129+0000 7f9fe9d9b640 1 --2- 172.21.15.94:0/1724397351 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557dff75cb80 0x557dff75ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.137+0000 7f9fe9d9b640 1 --2- 172.21.15.94:0/1724397351 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557dff75cb80 0x557dff75ef70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7125f72d22a3938 server_cookie=56b71ebcc5aaf92e in_seq=0 out_seq=0 2024-02-17T12:10:32.139 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.138+0000 7f9fd1ffb640 10 client.18708 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:10:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.138+0000 7f9fd1ffb640 1 -- 172.21.15.94:0/1724397351 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557dff3de890 con 0x557dff75cb80 2024-02-17T12:10:32.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.139+0000 7f9fd1ffb640 10 client.18708 handle_command_reply: tid=0 2024-02-17T12:10:32.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.140+0000 7f9ff385d740 1 client.18708 shutdown 2024-02-17T12:10:32.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.146+0000 7f9ff385d740 1 -- 172.21.15.94:0/1724397351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 msgr2=0x557dff6fa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.146+0000 7f9ff385d740 1 --2- 172.21.15.94:0/1724397351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 0x557dff6fa7d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc02e9c0 tx=0x7f9fdc02ed70 comp rx=0 tx=0).stop 2024-02-17T12:10:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.148+0000 7f9ff385d740 1 -- 172.21.15.94:0/1724397351 shutdown_connections 2024-02-17T12:10:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.148+0000 7f9ff385d740 1 --2- 172.21.15.94:0/1724397351 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557dff6f4750 0x557dff754f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.148+0000 7f9ff385d740 1 --2- 172.21.15.94:0/1724397351 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x557dff75cb80 0x557dff75ef70 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.148+0000 7f9ff385d740 1 --2- 172.21.15.94:0/1724397351 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557dff6ebb70 0x557dff6fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.148+0000 7f9ff385d740 1 --2- 172.21.15.94:0/1724397351 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557dff6eb250 0x557dff6fa7d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.148+0000 7f9ff385d740 1 -- 172.21.15.94:0/1724397351 >> 172.21.15.94:0/1724397351 conn(0x557dff6e5980 msgr2=0x557dff6e42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.149+0000 7f9fd1ffb640 0 client.18708 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:10:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.149+0000 7f9ff385d740 1 -- 172.21.15.94:0/1724397351 shutdown_connections 2024-02-17T12:10:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.149+0000 7f9ff385d740 1 -- 172.21.15.94:0/1724397351 wait complete. 2024-02-17T12:10:32.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.150+0000 7f9ff385d740 20 client.18708 trim_cache size 0 max 16384 2024-02-17T12:10:32.151 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:10:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.150+0000 7f9ff0c8f640 1 -- 172.21.15.94:0/1279009768 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fd40a2230 msgr2=0x7f9fd40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.150+0000 7f9ff0c8f640 1 --2- 172.21.15.94:0/1279009768 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fd40a2230 0x7f9fd40a46f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9fec104f50 tx=0x7f9fdc005f90 comp rx=0 tx=0).stop 2024-02-17T12:10:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.150+0000 7f9ff0c8f640 1 -- 172.21.15.94:0/1279009768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fec1998b0 msgr2=0x7f9fec199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.150+0000 7f9ff0c8f640 1 --2- 172.21.15.94:0/1279009768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fec1998b0 0x7f9fec199d60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400ec50 tx=0x7f9fe400c6a0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.151+0000 7f9ff0c8f640 1 -- 172.21.15.94:0/1279009768 shutdown_connections 2024-02-17T12:10:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.151+0000 7f9ff0c8f640 1 --2- 172.21.15.94:0/1279009768 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9fd40a2230 0x7f9fd40a46f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.151+0000 7f9ff0c8f640 1 --2- 172.21.15.94:0/1279009768 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9fec1998b0 0x7f9fec199d60 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.151+0000 7f9ff0c8f640 1 --2- 172.21.15.94:0/1279009768 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fec1053f0 0x7f9fec19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.151+0000 7f9ff0c8f640 1 --2- 172.21.15.94:0/1279009768 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9fec100e30 0x7f9fec19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.151+0000 7f9ff0c8f640 1 -- 172.21.15.94:0/1279009768 >> 172.21.15.94:0/1279009768 conn(0x7f9fec075430 msgr2=0x7f9fec0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.151+0000 7f9ff0c8f640 1 -- 172.21.15.94:0/1279009768 shutdown_connections 2024-02-17T12:10:32.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.151+0000 7f9ff0c8f640 1 -- 172.21.15.94:0/1279009768 wait complete. 2024-02-17T12:10:32.167 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:10:32.167 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:10:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.326+0000 7fea6760f640 1 -- 172.21.15.94:0/724449285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea6010d9d0 msgr2=0x7fea6010de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.326+0000 7fea6760f640 1 --2- 172.21.15.94:0/724449285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea6010d9d0 0x7fea6010de50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fea54009860 tx=0x7fea5402bb70 comp rx=0 tx=0).stop 2024-02-17T12:10:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.332+0000 7fea6760f640 1 -- 172.21.15.94:0/724449285 shutdown_connections 2024-02-17T12:10:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.332+0000 7fea6760f640 1 --2- 172.21.15.94:0/724449285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea6011a960 0x7fea6011cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.332+0000 7fea6760f640 1 --2- 172.21.15.94:0/724449285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea6010d9d0 0x7fea6010de50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.332+0000 7fea6760f640 1 --2- 172.21.15.94:0/724449285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea6010f9a0 0x7fea6010d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.332+0000 7fea6760f640 1 -- 172.21.15.94:0/724449285 >> 172.21.15.94:0/724449285 conn(0x7fea6006c260 msgr2=0x7fea6006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.334+0000 7fea6760f640 1 -- 172.21.15.94:0/724449285 shutdown_connections 2024-02-17T12:10:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.334+0000 7fea6760f640 1 -- 172.21.15.94:0/724449285 wait complete. 2024-02-17T12:10:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.335+0000 7fea6760f640 1 Processor -- start 2024-02-17T12:10:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.335+0000 7fea6760f640 1 -- start start 2024-02-17T12:10:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.335+0000 7fea6760f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea6010d9d0 0x7fea601a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.335+0000 7fea6760f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea6010f9a0 0x7fea601a6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.335+0000 7fea6760f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea6011a960 0x7fea601aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.335+0000 7fea6760f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea601ab4a0 con 0x7fea6010d9d0 2024-02-17T12:10:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.335+0000 7fea6760f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea601ab610 con 0x7fea6011a960 2024-02-17T12:10:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.335+0000 7fea6760f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea601ab7d0 con 0x7fea6010f9a0 2024-02-17T12:10:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.336+0000 7fea65bac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea6011a960 0x7fea601aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.336+0000 7fea65bac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea6011a960 0x7fea601aaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41926/0 (socket says 172.21.15.94:41926) 2024-02-17T12:10:32.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.336+0000 7fea65bac640 1 -- 172.21.15.94:0/251575987 learned_addr learned my addr 172.21.15.94:0/251575987 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.339+0000 7fea65bac640 1 -- 172.21.15.94:0/251575987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea6010f9a0 msgr2=0x7fea601a6bc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-17T12:10:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.339+0000 7fea65bac640 1 --2- 172.21.15.94:0/251575987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea6010f9a0 0x7fea601a6bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.339+0000 7fea65bac640 1 -- 172.21.15.94:0/251575987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea6010d9d0 msgr2=0x7fea601a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.339+0000 7fea65bac640 1 --2- 172.21.15.94:0/251575987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea6010d9d0 0x7fea601a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.339+0000 7fea65bac640 1 -- 172.21.15.94:0/251575987 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea54009490 con 0x7fea6011a960 2024-02-17T12:10:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.339+0000 7fea65bac640 1 --2- 172.21.15.94:0/251575987 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea6011a960 0x7fea601aaea0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fea5800e970 tx=0x7fea5800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.339+0000 7fea5e7fc640 1 -- 172.21.15.94:0/251575987 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5800ccc0 con 0x7fea6011a960 2024-02-17T12:10:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.339+0000 7fea5e7fc640 1 -- 172.21.15.94:0/251575987 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea5800ce20 con 0x7fea6011a960 2024-02-17T12:10:32.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.339+0000 7fea6760f640 1 -- 172.21.15.94:0/251575987 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea601abab0 con 0x7fea6011a960 2024-02-17T12:10:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.340+0000 7fea5e7fc640 1 -- 172.21.15.94:0/251575987 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58010430 con 0x7fea6011a960 2024-02-17T12:10:32.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.340+0000 7fea6760f640 1 -- 172.21.15.94:0/251575987 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea601b3330 con 0x7fea6011a960 2024-02-17T12:10:32.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.341+0000 7fea5e7fc640 1 -- 172.21.15.94:0/251575987 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fea58005020 con 0x7fea6011a960 2024-02-17T12:10:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.342+0000 7fea5e7fc640 1 --2- 172.21.15.94:0/251575987 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fea480a2370 0x7fea480a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.342+0000 7fea5e7fc640 1 -- 172.21.15.94:0/251575987 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fea58014070 con 0x7fea6011a960 2024-02-17T12:10:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.342+0000 7fea653ab640 1 --2- 172.21.15.94:0/251575987 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fea480a2370 0x7fea480a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.343+0000 7fea653ab640 1 --2- 172.21.15.94:0/251575987 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fea480a2370 0x7fea480a4830 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fea50005fd0 tx=0x7fea50009710 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.344+0000 7fea6a1db740 1 Processor -- start 2024-02-17T12:10:32.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.344+0000 7fea6a1db740 1 -- start start 2024-02-17T12:10:32.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.344+0000 7fea6a1db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1e9250 0x5640ea1e9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.344+0000 7fea6a1db740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9b70 0x5640ea1f2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea6a1db740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1f2750 0x5640ea1f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea6a1db740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ea1a4960 con 0x5640ea1f2750 2024-02-17T12:10:32.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea6a1db740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ea158310 con 0x5640ea1e9b70 2024-02-17T12:10:32.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea6a1db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ea155840 con 0x5640ea1e9250 2024-02-17T12:10:32.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea653ab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1e9250 0x5640ea1e9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea653ab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1e9250 0x5640ea1e9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47788/0 (socket says 172.21.15.94:47788) 2024-02-17T12:10:32.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea65bac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1f2750 0x5640ea1f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea653ab640 1 -- 172.21.15.94:0/150653237 learned_addr learned my addr 172.21.15.94:0/150653237 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea64baa640 1 --2- 172.21.15.94:0/150653237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9b70 0x5640ea1f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea653ab640 1 -- 172.21.15.94:0/150653237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9b70 msgr2=0x5640ea1f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea653ab640 1 --2- 172.21.15.94:0/150653237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9b70 0x5640ea1f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea653ab640 1 -- 172.21.15.94:0/150653237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1f2750 msgr2=0x5640ea1f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea653ab640 1 --2- 172.21.15.94:0/150653237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1f2750 0x5640ea1f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.345+0000 7fea653ab640 1 -- 172.21.15.94:0/150653237 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640ea113060 con 0x5640ea1e9250 2024-02-17T12:10:32.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.346+0000 7fea653ab640 1 --2- 172.21.15.94:0/150653237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1e9250 0x5640ea1e9630 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fea500114a0 tx=0x7fea50011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1b8c1168a39ab34 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.346+0000 7fea477fe640 1 -- 172.21.15.94:0/150653237 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5001e640 con 0x5640ea1e9250 2024-02-17T12:10:32.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.346+0000 7fea477fe640 1 -- 172.21.15.94:0/150653237 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea5001ec80 con 0x5640ea1e9250 2024-02-17T12:10:32.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.346+0000 7fea477fe640 1 -- 172.21.15.94:0/150653237 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50010cc0 con 0x5640ea1e9250 2024-02-17T12:10:32.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.347+0000 7fea6a1db740 1 -- 172.21.15.94:0/150653237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1e9250 msgr2=0x5640ea1e9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.347+0000 7fea6a1db740 1 --2- 172.21.15.94:0/150653237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1e9250 0x5640ea1e9630 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fea500114a0 tx=0x7fea50011000 comp rx=0 tx=0).stop 2024-02-17T12:10:32.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.347+0000 7fea6a1db740 1 -- 172.21.15.94:0/150653237 shutdown_connections 2024-02-17T12:10:32.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.347+0000 7fea6a1db740 1 --2- 172.21.15.94:0/150653237 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1f2750 0x5640ea1f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.347+0000 7fea6a1db740 1 --2- 172.21.15.94:0/150653237 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9b70 0x5640ea1f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.347+0000 7fea6a1db740 1 --2- 172.21.15.94:0/150653237 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1e9250 0x5640ea1e9630 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.347+0000 7fea6a1db740 1 -- 172.21.15.94:0/150653237 >> 172.21.15.94:0/150653237 conn(0x5640ea1e3980 msgr2=0x5640ea1e3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.347+0000 7fea6a1db740 1 -- 172.21.15.94:0/150653237 shutdown_connections 2024-02-17T12:10:32.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.347+0000 7fea6a1db740 1 -- 172.21.15.94:0/150653237 wait complete. 2024-02-17T12:10:32.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.347+0000 7fea6a1db740 1 -- start start 2024-02-17T12:10:32.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.347+0000 7fea6a1db740 1 Processor -- start 2024-02-17T12:10:32.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.348+0000 7fea6a1db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1e9250 0x5640ea1ef730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.348+0000 7fea6a1db740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9b70 0x5640ea1efc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.348+0000 7fea6a1db740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1f2750 0x5640ea1f01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.348+0000 7fea6a1db740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ea151020 con 0x5640ea1e9b70 2024-02-17T12:10:32.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.348+0000 7fea6a1db740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e9e24430 con 0x5640ea1f2750 2024-02-17T12:10:32.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.348+0000 7fea6a1db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e9de0e00 con 0x5640ea1e9250 2024-02-17T12:10:32.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.348+0000 7fea6a1db740 10 client.0 _ll_register_callbacks cb 0x5640ea0f8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:32.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.348+0000 7fea64baa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9b70 0x5640ea1efc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.348+0000 7fea653ab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1e9250 0x5640ea1ef730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.348+0000 7fea64baa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9b70 0x5640ea1efc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40840/0 (socket says 172.21.15.94:40840) 2024-02-17T12:10:32.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.348+0000 7fea64baa640 1 -- 172.21.15.94:0/1895501786 learned_addr learned my addr 172.21.15.94:0/1895501786 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.348+0000 7fea65bac640 1 --2- 172.21.15.94:0/1895501786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1f2750 0x5640ea1f01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea653ab640 1 -- 172.21.15.94:0/1895501786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1f2750 msgr2=0x5640ea1f01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea653ab640 1 --2- 172.21.15.94:0/1895501786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1f2750 0x5640ea1f01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea653ab640 1 -- 172.21.15.94:0/1895501786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9b70 msgr2=0x5640ea1efc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea653ab640 1 --2- 172.21.15.94:0/1895501786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9b70 0x5640ea1efc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea653ab640 1 -- 172.21.15.94:0/1895501786 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea50009d30 con 0x5640ea1e9250 2024-02-17T12:10:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea653ab640 1 --2- 172.21.15.94:0/1895501786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1e9250 0x5640ea1ef730 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fea50013040 tx=0x7fea50010f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b235d62c102b30b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:10:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea44ff9640 1 -- 172.21.15.94:0/1895501786 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5001e440 con 0x5640ea1e9250 2024-02-17T12:10:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea6a1db740 1 -- 172.21.15.94:0/1895501786 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640e9e507e0 con 0x5640ea1e9250 2024-02-17T12:10:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea44ff9640 1 -- 172.21.15.94:0/1895501786 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fea5001e5a0 con 0x5640ea1e9250 2024-02-17T12:10:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea44ff9640 1 -- 172.21.15.94:0/1895501786 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea5001e760 con 0x5640ea1e9250 2024-02-17T12:10:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea6a1db740 10 client.28174 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea6a1db740 1 -- 172.21.15.94:0/1895501786 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5640ea14f6a0 con 0x5640ea1e9250 2024-02-17T12:10:32.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.349+0000 7fea44ff9640 1 -- 172.21.15.94:0/1895501786 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5640e9e507e0 con 0x5640ea1e9250 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.350+0000 7fea44ff9640 1 -- 172.21.15.94:0/1895501786 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fea50031c80 con 0x5640ea1e9250 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.350+0000 7fea6a1db740 10 client.28174 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.350+0000 7fea6a1db740 10 client.28174 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.350+0000 7fea6a1db740 20 client.28174 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.350+0000 7fea6a1db740 1 --2- 172.21.15.94:0/1895501786 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5640ea202a90 0x5640ea204eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.350+0000 7fea6a1db740 4 client.28174 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.350+0000 7fea6a1db740 1 -- 172.21.15.94:0/1895501786 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640e9e8d720 con 0x5640ea202a90 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.350+0000 7fea64baa640 1 --2- 172.21.15.94:0/1895501786 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5640ea202a90 0x5640ea204eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.350+0000 7fea64baa640 1 --2- 172.21.15.94:0/1895501786 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5640ea202a90 0x5640ea204eb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c6d5b397d4afccbc server_cookie=5b5b3ce5188a6f26 in_seq=0 out_seq=0 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.350+0000 7fea44ff9640 10 client.28174 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea44ff9640 1 -- 172.21.15.94:0/1895501786 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5640e9e8d720 con 0x5640ea202a90 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea44ff9640 10 client.28174 handle_command_reply: tid=0 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea6a1db740 1 client.28174 shutdown 2024-02-17T12:10:32.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea6a1db740 1 -- 172.21.15.94:0/1895501786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1e9250 msgr2=0x5640ea1ef730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea6a1db740 1 --2- 172.21.15.94:0/1895501786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1e9250 0x5640ea1ef730 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fea50013040 tx=0x7fea50010f60 comp rx=0 tx=0).stop 2024-02-17T12:10:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea6a1db740 1 -- 172.21.15.94:0/1895501786 shutdown_connections 2024-02-17T12:10:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea6a1db740 1 --2- 172.21.15.94:0/1895501786 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5640ea202a90 0x5640ea204eb0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea6a1db740 1 --2- 172.21.15.94:0/1895501786 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1f2750 0x5640ea1f01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea6a1db740 1 --2- 172.21.15.94:0/1895501786 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9b70 0x5640ea1efc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea44ff9640 0 client.28174 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:10:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea6a1db740 1 --2- 172.21.15.94:0/1895501786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1e9250 0x5640ea1ef730 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea6a1db740 1 -- 172.21.15.94:0/1895501786 >> 172.21.15.94:0/1895501786 conn(0x5640ea1e3980 msgr2=0x5640ea1e56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea6a1db740 1 -- 172.21.15.94:0/1895501786 shutdown_connections 2024-02-17T12:10:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.351+0000 7fea6a1db740 1 -- 172.21.15.94:0/1895501786 wait complete. 2024-02-17T12:10:32.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.352+0000 7fea6a1db740 20 client.28174 trim_cache size 0 max 16384 2024-02-17T12:10:32.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.367+0000 7fea6a1db740 1 Processor -- start 2024-02-17T12:10:32.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.367+0000 7fea6a1db740 1 -- start start 2024-02-17T12:10:32.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.367+0000 7fea6a1db740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9250 0x5640ea1f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.367+0000 7fea6a1db740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9b70 0x5640ea1efc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.367+0000 7fea6a1db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1f2750 0x5640ea1f01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.367+0000 7fea6a1db740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ea0d6130 con 0x5640ea1e9b70 2024-02-17T12:10:32.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.367+0000 7fea6a1db740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ea20a0e0 con 0x5640ea1e9250 2024-02-17T12:10:32.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.367+0000 7fea64baa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9b70 0x5640ea1efc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.370 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.367+0000 7fea64baa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9b70 0x5640ea1efc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40850/0 (socket says 172.21.15.94:40850) 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.367+0000 7fea64baa640 1 -- 172.21.15.94:0/2621176202 learned_addr learned my addr 172.21.15.94:0/2621176202 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.367+0000 7fea6a1db740 1 -- 172.21.15.94:0/2621176202 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ea1df080 con 0x5640ea1f2750 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.367+0000 7fea653ab640 1 --2- 172.21.15.94:0/2621176202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9250 0x5640ea1f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.367+0000 7fea65bac640 1 --2- 172.21.15.94:0/2621176202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1f2750 0x5640ea1f01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.368+0000 7fea64baa640 1 -- 172.21.15.94:0/2621176202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1f2750 msgr2=0x5640ea1f01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.368+0000 7fea64baa640 1 --2- 172.21.15.94:0/2621176202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1f2750 0x5640ea1f01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.368+0000 7fea64baa640 1 -- 172.21.15.94:0/2621176202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9250 msgr2=0x5640ea1f1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.368+0000 7fea64baa640 1 --2- 172.21.15.94:0/2621176202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9250 0x5640ea1f1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.368+0000 7fea64baa640 1 -- 172.21.15.94:0/2621176202 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640ea0ebc20 con 0x5640ea1e9b70 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.368+0000 7fea653ab640 1 --2- 172.21.15.94:0/2621176202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9250 0x5640ea1f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.368+0000 7fea64baa640 1 --2- 172.21.15.94:0/2621176202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9b70 0x5640ea1efc80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fea54000c00 tx=0x7fea5402c090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511bfc6ac44311ba server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.368+0000 7fea46ffd640 1 -- 172.21.15.94:0/2621176202 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54004040 con 0x5640ea1e9b70 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.368+0000 7fea46ffd640 1 -- 172.21.15.94:0/2621176202 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea54034440 con 0x5640ea1e9b70 2024-02-17T12:10:32.371 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.368+0000 7fea46ffd640 1 -- 172.21.15.94:0/2621176202 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea540055a0 con 0x5640ea1e9b70 2024-02-17T12:10:32.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.369+0000 7fea6a1db740 1 -- 172.21.15.94:0/2621176202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9b70 msgr2=0x5640ea1efc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.369+0000 7fea6a1db740 1 --2- 172.21.15.94:0/2621176202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9b70 0x5640ea1efc80 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fea54000c00 tx=0x7fea5402c090 comp rx=0 tx=0).stop 2024-02-17T12:10:32.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.369+0000 7fea6a1db740 1 -- 172.21.15.94:0/2621176202 shutdown_connections 2024-02-17T12:10:32.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.369+0000 7fea6a1db740 1 --2- 172.21.15.94:0/2621176202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1f2750 0x5640ea1f01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.374 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.369+0000 7fea6a1db740 1 --2- 172.21.15.94:0/2621176202 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9b70 0x5640ea1efc80 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.369+0000 7fea6a1db740 1 --2- 172.21.15.94:0/2621176202 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9250 0x5640ea1f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.369+0000 7fea6a1db740 1 -- 172.21.15.94:0/2621176202 >> 172.21.15.94:0/2621176202 conn(0x5640ea1e3980 msgr2=0x5640ea1ea9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.369+0000 7fea6a1db740 1 -- 172.21.15.94:0/2621176202 shutdown_connections 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.369+0000 7fea6a1db740 1 -- 172.21.15.94:0/2621176202 wait complete. 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.369+0000 7fea6a1db740 1 -- start start 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.369+0000 7fea6a1db740 1 Processor -- start 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.370+0000 7fea6a1db740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9250 0x5640ea1f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.370+0000 7fea6a1db740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9b70 0x5640ea1f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.370+0000 7fea6a1db740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1f2750 0x5640ea1f85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.370+0000 7fea6a1db740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ea1f8c90 con 0x5640ea1e9250 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.370+0000 7fea653ab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9250 0x5640ea1f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.370+0000 7fea6a1db740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ea258fc0 con 0x5640ea1e9b70 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.370+0000 7fea6a1db740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640ea259100 con 0x5640ea1f2750 2024-02-17T12:10:32.375 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.370+0000 7fea64baa640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9b70 0x5640ea1f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.370+0000 7fea65bac640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1f2750 0x5640ea1f85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.370+0000 7fea65bac640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1f2750 0x5640ea1f85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47802/0 (socket says 172.21.15.94:47802) 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.370+0000 7fea65bac640 1 -- 172.21.15.94:0/661604643 learned_addr learned my addr 172.21.15.94:0/661604643 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.371+0000 7fea6a1db740 10 client.0 _ll_register_callbacks cb 0x7fea50009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.371+0000 7fea653ab640 1 -- 172.21.15.94:0/661604643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1f2750 msgr2=0x5640ea1f85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.371+0000 7fea653ab640 1 --2- 172.21.15.94:0/661604643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1f2750 0x5640ea1f85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.371+0000 7fea653ab640 1 -- 172.21.15.94:0/661604643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9b70 msgr2=0x5640ea1f7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.371+0000 7fea653ab640 1 --2- 172.21.15.94:0/661604643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9b70 0x5640ea1f7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.371+0000 7fea653ab640 1 -- 172.21.15.94:0/661604643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640ea14f6a0 con 0x5640ea1e9250 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.371+0000 7fea653ab640 1 --2- 172.21.15.94:0/661604643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9250 0x5640ea1f9ae0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fea5000ebf0 tx=0x7fea500107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef089e738196af36 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.371+0000 7fea44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.371+0000 7fea44ff9640 1 -- 172.21.15.94:0/661604643 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea500103d0 con 0x5640ea1e9250 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.371+0000 7fea44ff9640 1 -- 172.21.15.94:0/661604643 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fea50009480 con 0x5640ea1e9250 2024-02-17T12:10:32.376 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.371+0000 7fea6a1db740 1 -- 172.21.15.94:0/661604643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea5001ec80 con 0x5640ea1e9250 2024-02-17T12:10:32.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.372+0000 7fea44ff9640 1 -- 172.21.15.94:0/661604643 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea5001e440 con 0x5640ea1e9250 2024-02-17T12:10:32.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.372+0000 7fea6a1db740 10 client.18723 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:32.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.372+0000 7fea6a1db740 1 -- 172.21.15.94:0/661604643 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fea6011d680 con 0x5640ea1e9250 2024-02-17T12:10:32.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.372+0000 7fea44ff9640 1 -- 172.21.15.94:0/661604643 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fea5002c070 con 0x5640ea1e9250 2024-02-17T12:10:32.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.372+0000 7fea44ff9640 1 -- 172.21.15.94:0/661604643 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fea50028ae0 con 0x5640ea1e9250 2024-02-17T12:10:32.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.372+0000 7fea6a1db740 10 client.18723 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:32.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.372+0000 7fea6a1db740 10 client.18723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:10:32.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.372+0000 7fea6a1db740 20 client.18723 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:32.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.372+0000 7fea6a1db740 1 --2- 172.21.15.94:0/661604643 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5640ea25a4e0 0x5640ea25c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.372+0000 7fea6a1db740 4 client.18723 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:10:32.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.372+0000 7fea6a1db740 1 -- 172.21.15.94:0/661604643 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fea5001ec80 con 0x5640ea25a4e0 2024-02-17T12:10:32.377 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.373+0000 7fea64baa640 1 --2- 172.21.15.94:0/661604643 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5640ea25a4e0 0x5640ea25c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.373+0000 7fea64baa640 1 --2- 172.21.15.94:0/661604643 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5640ea25a4e0 0x5640ea25c8d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b227e4d7e9c9593 server_cookie=9bc62fb371b50fb7 in_seq=0 out_seq=0 2024-02-17T12:10:32.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.373+0000 7fea44ff9640 10 client.18723 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:10:32.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.373+0000 7fea44ff9640 1 -- 172.21.15.94:0/661604643 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fea5001ec80 con 0x5640ea25a4e0 2024-02-17T12:10:32.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.373+0000 7fea44ff9640 10 client.18723 handle_command_reply: tid=0 2024-02-17T12:10:32.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.374+0000 7fea6a1db740 1 client.18723 shutdown 2024-02-17T12:10:32.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.374+0000 7fea6a1db740 1 -- 172.21.15.94:0/661604643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9250 msgr2=0x5640ea1f9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.379 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.374+0000 7fea6a1db740 1 --2- 172.21.15.94:0/661604643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9250 0x5640ea1f9ae0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fea5000ebf0 tx=0x7fea500107a0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.374+0000 7fea6a1db740 1 -- 172.21.15.94:0/661604643 shutdown_connections 2024-02-17T12:10:32.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.374+0000 7fea6a1db740 1 --2- 172.21.15.94:0/661604643 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5640ea25a4e0 0x5640ea25c8d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.374+0000 7fea6a1db740 1 --2- 172.21.15.94:0/661604643 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5640ea1f2750 0x5640ea1f85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.374+0000 7fea6a1db740 1 --2- 172.21.15.94:0/661604643 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5640ea1e9b70 0x5640ea1f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.374+0000 7fea6a1db740 1 --2- 172.21.15.94:0/661604643 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5640ea1e9250 0x5640ea1f9ae0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.374+0000 7fea6a1db740 1 -- 172.21.15.94:0/661604643 >> 172.21.15.94:0/661604643 conn(0x5640ea1e3980 msgr2=0x5640ea1e4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.374+0000 7fea44ff9640 0 client.18723 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:10:32.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.374+0000 7fea6a1db740 1 -- 172.21.15.94:0/661604643 shutdown_connections 2024-02-17T12:10:32.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.374+0000 7fea6a1db740 1 -- 172.21.15.94:0/661604643 wait complete. 2024-02-17T12:10:32.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.374+0000 7fea6a1db740 20 client.18723 trim_cache size 0 max 16384 2024-02-17T12:10:32.380 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:10:32.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.375+0000 7fea6760f640 1 -- 172.21.15.94:0/251575987 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fea480a2370 msgr2=0x7fea480a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.375+0000 7fea6760f640 1 --2- 172.21.15.94:0/251575987 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fea480a2370 0x7fea480a4830 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fea50005fd0 tx=0x7fea50009710 comp rx=0 tx=0).stop 2024-02-17T12:10:32.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.375+0000 7fea6760f640 1 -- 172.21.15.94:0/251575987 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea6011a960 msgr2=0x7fea601aaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.375+0000 7fea6760f640 1 --2- 172.21.15.94:0/251575987 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea6011a960 0x7fea601aaea0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fea5800e970 tx=0x7fea5800ee30 comp rx=0 tx=0).stop 2024-02-17T12:10:32.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.375+0000 7fea6760f640 1 -- 172.21.15.94:0/251575987 shutdown_connections 2024-02-17T12:10:32.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.375+0000 7fea6760f640 1 --2- 172.21.15.94:0/251575987 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fea480a2370 0x7fea480a4830 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.375+0000 7fea6760f640 1 --2- 172.21.15.94:0/251575987 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fea6011a960 0x7fea601aaea0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.375+0000 7fea6760f640 1 --2- 172.21.15.94:0/251575987 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea6010f9a0 0x7fea601a6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.375+0000 7fea6760f640 1 --2- 172.21.15.94:0/251575987 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fea6010d9d0 0x7fea601a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.375+0000 7fea6760f640 1 -- 172.21.15.94:0/251575987 >> 172.21.15.94:0/251575987 conn(0x7fea6006c260 msgr2=0x7fea60071c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.375+0000 7fea6760f640 1 -- 172.21.15.94:0/251575987 shutdown_connections 2024-02-17T12:10:32.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.375+0000 7fea6760f640 1 -- 172.21.15.94:0/251575987 wait complete. 2024-02-17T12:10:32.389 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:10:32.390 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:10:32.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.559+0000 7fb7c2039640 1 -- 172.21.15.94:0/487373823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc11a960 msgr2=0x7fb7bc11cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.559+0000 7fb7c2039640 1 --2- 172.21.15.94:0/487373823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc11a960 0x7fb7bc11cd50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb7b401acb0 tx=0x7fb7b403d170 comp rx=0 tx=0).stop 2024-02-17T12:10:32.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.563+0000 7fb7c2039640 1 -- 172.21.15.94:0/487373823 shutdown_connections 2024-02-17T12:10:32.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.563+0000 7fb7c2039640 1 --2- 172.21.15.94:0/487373823 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc11a960 0x7fb7bc11cd50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.563+0000 7fb7c2039640 1 --2- 172.21.15.94:0/487373823 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7bc10d9d0 0x7fb7bc10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.563+0000 7fb7c2039640 1 --2- 172.21.15.94:0/487373823 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7bc10f9a0 0x7fb7bc10d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.563+0000 7fb7c2039640 1 -- 172.21.15.94:0/487373823 >> 172.21.15.94:0/487373823 conn(0x7fb7bc06c260 msgr2=0x7fb7bc06c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.565+0000 7fb7c2039640 1 -- 172.21.15.94:0/487373823 shutdown_connections 2024-02-17T12:10:32.570 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.568+0000 7fb7c2039640 1 -- 172.21.15.94:0/487373823 wait complete. 2024-02-17T12:10:32.571 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.569+0000 7fb7c2039640 1 Processor -- start 2024-02-17T12:10:32.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.570+0000 7fb7c2039640 1 -- start start 2024-02-17T12:10:32.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.575+0000 7fb7c2039640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7bc10d9d0 0x7fb7bc116190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.575+0000 7fb7c2039640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc10f9a0 0x7fb7bc1166d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.575+0000 7fb7c2039640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7bc11a960 0x7fb7bc1a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.575+0000 7fb7c2039640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc1a2c20 con 0x7fb7bc11a960 2024-02-17T12:10:32.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.575+0000 7fb7c2039640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc1a2d90 con 0x7fb7bc10d9d0 2024-02-17T12:10:32.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.575+0000 7fb7c2039640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7bc1a2f50 con 0x7fb7bc10f9a0 2024-02-17T12:10:32.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.575+0000 7fb7bb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7bc10d9d0 0x7fb7bc116190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.576+0000 7fb7bb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7bc10d9d0 0x7fb7bc116190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41976/0 (socket says 172.21.15.94:41976) 2024-02-17T12:10:32.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.576+0000 7fb7bb7fe640 1 -- 172.21.15.94:0/3724747916 learned_addr learned my addr 172.21.15.94:0/3724747916 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.579 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.576+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7bc11a960 0x7fb7bc1a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.576+0000 7fb7b3fff640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc10f9a0 0x7fb7bc1166d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.576+0000 7fb7bb7fe640 1 -- 172.21.15.94:0/3724747916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc10f9a0 msgr2=0x7fb7bc1166d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.576+0000 7fb7bb7fe640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc10f9a0 0x7fb7bc1166d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.576+0000 7fb7bb7fe640 1 -- 172.21.15.94:0/3724747916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7bc11a960 msgr2=0x7fb7bc1a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.576+0000 7fb7bb7fe640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7bc11a960 0x7fb7bc1a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.576+0000 7fb7bb7fe640 1 -- 172.21.15.94:0/3724747916 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b401a910 con 0x7fb7bc10d9d0 2024-02-17T12:10:32.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.577+0000 7fb7bb7fe640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7bc10d9d0 0x7fb7bc116190 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00c920 tx=0x7fb7ac00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.577+0000 7fb7b97fa640 1 -- 172.21.15.94:0/3724747916 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac007c70 con 0x7fb7bc10d9d0 2024-02-17T12:10:32.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.577+0000 7fb7c2039640 1 -- 172.21.15.94:0/3724747916 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7bc1a3230 con 0x7fb7bc10d9d0 2024-02-17T12:10:32.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.577+0000 7fb7c2039640 1 -- 172.21.15.94:0/3724747916 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7bc1aa9d0 con 0x7fb7bc10d9d0 2024-02-17T12:10:32.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.577+0000 7fb7b97fa640 1 -- 172.21.15.94:0/3724747916 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7ac007dd0 con 0x7fb7bc10d9d0 2024-02-17T12:10:32.580 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.577+0000 7fb7b97fa640 1 -- 172.21.15.94:0/3724747916 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac018860 con 0x7fb7bc10d9d0 2024-02-17T12:10:32.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.581+0000 7fb7c4c09740 1 Processor -- start 2024-02-17T12:10:32.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.581+0000 7fb7c4c09740 1 -- start start 2024-02-17T12:10:32.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.581+0000 7fb7c4c09740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 0x55c580693630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.581+0000 7fb7c4c09740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693b70 0x55c58069c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.581+0000 7fb7c4c09740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c58069c750 0x55c58069eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.581+0000 7fb7c4c09740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58064e960 con 0x55c580693250 2024-02-17T12:10:32.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.581+0000 7fb7c4c09740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c580602310 con 0x55c58069c750 2024-02-17T12:10:32.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.581+0000 7fb7c4c09740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5805ff840 con 0x55c580693b70 2024-02-17T12:10:32.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.582+0000 7fb7b97fa640 1 -- 172.21.15.94:0/3724747916 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb7ac0189c0 con 0x7fb7bc10d9d0 2024-02-17T12:10:32.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.582+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 0x55c580693630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.582+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 0x55c580693630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40894/0 (socket says 172.21.15.94:40894) 2024-02-17T12:10:32.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.582+0000 7fb7bbfff640 1 -- 172.21.15.94:0/785329520 learned_addr learned my addr 172.21.15.94:0/785329520 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.582+0000 7fb7b3fff640 1 --2- 172.21.15.94:0/785329520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c58069c750 0x55c58069eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.582+0000 7fb7bbfff640 1 -- 172.21.15.94:0/785329520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693b70 msgr2=0x55c58069c210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:10:32.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.582+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/785329520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693b70 0x55c58069c210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.582+0000 7fb7bbfff640 1 -- 172.21.15.94:0/785329520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c58069c750 msgr2=0x55c58069eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.582+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/785329520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c58069c750 0x55c58069eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.582+0000 7fb7bbfff640 1 -- 172.21.15.94:0/785329520 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5805bd060 con 0x55c580693250 2024-02-17T12:10:32.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.582+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/785329520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 0x55c580693630 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4047000 tx=0x7fb7b401a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e10e7085731066 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.582+0000 7fb7b97fa640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb79c0a22f0 0x7fb79c0a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.583+0000 7fb7b97fa640 1 -- 172.21.15.94:0/3724747916 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb7ac0ce160 con 0x7fb7bc10d9d0 2024-02-17T12:10:32.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.583+0000 7fb7bb7fe640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb79c0a22f0 0x7fb79c0a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.583+0000 7fb7b1ffb640 1 -- 172.21.15.94:0/785329520 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4043b50 con 0x55c580693250 2024-02-17T12:10:32.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.583+0000 7fb7b1ffb640 1 -- 172.21.15.94:0/785329520 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7b4043cb0 con 0x55c580693250 2024-02-17T12:10:32.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.583+0000 7fb7b1ffb640 1 -- 172.21.15.94:0/785329520 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4051cc0 con 0x55c580693250 2024-02-17T12:10:32.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.584+0000 7fb7c4c09740 1 -- 172.21.15.94:0/785329520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 msgr2=0x55c580693630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.584+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/785329520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 0x55c580693630 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb7b4047000 tx=0x7fb7b401a880 comp rx=0 tx=0).stop 2024-02-17T12:10:32.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.584+0000 7fb7bb7fe640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb79c0a22f0 0x7fb79c0a47b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55c580692d20 tx=0x7fb7ac096e60 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.584+0000 7fb7c4c09740 1 -- 172.21.15.94:0/785329520 shutdown_connections 2024-02-17T12:10:32.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.584+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/785329520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c58069c750 0x55c58069eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.584+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/785329520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693b70 0x55c58069c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.584+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/785329520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 0x55c580693630 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.584+0000 7fb7c4c09740 1 -- 172.21.15.94:0/785329520 >> 172.21.15.94:0/785329520 conn(0x55c58068d980 msgr2=0x55c58068dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.584+0000 7fb7c4c09740 1 -- 172.21.15.94:0/785329520 shutdown_connections 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.584+0000 7fb7c4c09740 1 -- 172.21.15.94:0/785329520 wait complete. 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.585+0000 7fb7c4c09740 1 -- start start 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.585+0000 7fb7c4c09740 1 Processor -- start 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.585+0000 7fb7c4c09740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693250 0x55c580699740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7c4c09740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693b70 0x55c580699c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7c4c09740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c58069c750 0x55c58069a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7c4c09740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5805fb020 con 0x55c58069c750 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7c4c09740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5802ce430 con 0x55c580693b70 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7c4c09740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c58028ae00 con 0x55c580693250 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7c4c09740 10 client.0 _ll_register_callbacks cb 0x55c5805a2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7b3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693b70 0x55c580699c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7b3fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693b70 0x55c580699c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:41988/0 (socket says 172.21.15.94:41988) 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7b3fff640 1 -- 172.21.15.94:0/3173251006 learned_addr learned my addr 172.21.15.94:0/3173251006 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/3173251006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c58069c750 0x55c58069a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/3173251006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693250 0x55c580699740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7b3fff640 1 -- 172.21.15.94:0/3173251006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693250 msgr2=0x55c580699740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7b3fff640 1 --2- 172.21.15.94:0/3173251006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693250 0x55c580699740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7b3fff640 1 -- 172.21.15.94:0/3173251006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c58069c750 msgr2=0x55c58069a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7b3fff640 1 --2- 172.21.15.94:0/3173251006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c58069c750 0x55c58069a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7b3fff640 1 -- 172.21.15.94:0/3173251006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7bc11d680 con 0x55c580693b70 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7b3fff640 1 --2- 172.21.15.94:0/3173251006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693b70 0x55c580699c80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400e280 tx=0x7fb7a400e740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d408ecc2dbbfa35a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.586+0000 7fb7aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.587+0000 7fb7aaffd640 1 -- 172.21.15.94:0/3173251006 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a4012600 con 0x55c580693b70 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.587+0000 7fb7aaffd640 1 -- 172.21.15.94:0/3173251006 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb7a4012760 con 0x55c580693b70 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.587+0000 7fb7c4c09740 1 -- 172.21.15.94:0/3173251006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5802fa7e0 con 0x55c580693b70 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.587+0000 7fb7aaffd640 1 -- 172.21.15.94:0/3173251006 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7a4012920 con 0x55c580693b70 2024-02-17T12:10:32.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.587+0000 7fb7c4c09740 10 client.28196 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:32.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.587+0000 7fb7c4c09740 1 -- 172.21.15.94:0/3173251006 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c5805f96a0 con 0x55c580693b70 2024-02-17T12:10:32.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.587+0000 7fb7aaffd640 1 -- 172.21.15.94:0/3173251006 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7a4014400 con 0x55c580693b70 2024-02-17T12:10:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.588+0000 7fb7aaffd640 1 -- 172.21.15.94:0/3173251006 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb7a4011070 con 0x55c580693b70 2024-02-17T12:10:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.588+0000 7fb7c4c09740 10 client.28196 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.588+0000 7fb7c4c09740 10 client.28196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.588+0000 7fb7c4c09740 20 client.28196 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.588+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/3173251006 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c5806acc10 0x55c5806af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.588+0000 7fb7c4c09740 4 client.28196 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:10:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.588+0000 7fb7c4c09740 1 -- 172.21.15.94:0/3173251006 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5802fa7e0 con 0x55c5806acc10 2024-02-17T12:10:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.588+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/3173251006 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c5806acc10 0x55c5806af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.589+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/3173251006 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c5806acc10 0x55c5806af050 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0148bc7497af482 server_cookie=b2bd99a0b06e4e6d in_seq=0 out_seq=0 2024-02-17T12:10:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.589+0000 7fb7aaffd640 10 client.28196 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:32.594 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.589+0000 7fb7aaffd640 1 -- 172.21.15.94:0/3173251006 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5802fa7e0 con 0x55c5806acc10 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.589+0000 7fb7aaffd640 10 client.28196 handle_command_reply: tid=0 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.589+0000 7fb7c4c09740 1 client.28196 shutdown 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.589+0000 7fb7c4c09740 1 -- 172.21.15.94:0/3173251006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693b70 msgr2=0x55c580699c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.589+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/3173251006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693b70 0x55c580699c80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb7a400e280 tx=0x7fb7a400e740 comp rx=0 tx=0).stop 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.589+0000 7fb7c4c09740 1 -- 172.21.15.94:0/3173251006 shutdown_connections 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.590+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/3173251006 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c5806acc10 0x55c5806af050 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.590+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/3173251006 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c58069c750 0x55c58069a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.590+0000 7fb7aaffd640 0 client.28196 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.590+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/3173251006 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693b70 0x55c580699c80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.590+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/3173251006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693250 0x55c580699740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.590+0000 7fb7c4c09740 1 -- 172.21.15.94:0/3173251006 >> 172.21.15.94:0/3173251006 conn(0x55c58068d980 msgr2=0x55c58068f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.590+0000 7fb7c4c09740 1 -- 172.21.15.94:0/3173251006 shutdown_connections 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.590+0000 7fb7c4c09740 1 -- 172.21.15.94:0/3173251006 wait complete. 2024-02-17T12:10:32.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.590+0000 7fb7c4c09740 20 client.28196 trim_cache size 0 max 16384 2024-02-17T12:10:32.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.604+0000 7fb7c4c09740 1 Processor -- start 2024-02-17T12:10:32.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.604+0000 7fb7c4c09740 1 -- start start 2024-02-17T12:10:32.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.605+0000 7fb7c4c09740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 0x55c58069be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.605+0000 7fb7c4c09740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693b70 0x55c5806af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.605+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 0x55c58069be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.605+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 0x55c58069be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40904/0 (socket says 172.21.15.94:40904) 2024-02-17T12:10:32.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.605+0000 7fb7bbfff640 1 -- 172.21.15.94:0/559873445 learned_addr learned my addr 172.21.15.94:0/559873445 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.605+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/559873445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c58069c750 0x55c580699fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.605+0000 7fb7c4c09740 1 -- 172.21.15.94:0/559873445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c580580130 con 0x55c580693250 2024-02-17T12:10:32.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.605+0000 7fb7c4c09740 1 -- 172.21.15.94:0/559873445 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c580692570 con 0x55c580693b70 2024-02-17T12:10:32.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.605+0000 7fb7c4c09740 1 -- 172.21.15.94:0/559873445 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c580694610 con 0x55c58069c750 2024-02-17T12:10:32.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.606+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/559873445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c58069c750 0x55c580699fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.606+0000 7fb7b3fff640 1 --2- 172.21.15.94:0/559873445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693b70 0x55c5806af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.606+0000 7fb7bbfff640 1 -- 172.21.15.94:0/559873445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c58069c750 msgr2=0x55c580699fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.606+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/559873445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c58069c750 0x55c580699fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.606+0000 7fb7bbfff640 1 -- 172.21.15.94:0/559873445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693b70 msgr2=0x55c5806af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.606+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/559873445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693b70 0x55c5806af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.606+0000 7fb7bbfff640 1 -- 172.21.15.94:0/559873445 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c580595c20 con 0x55c580693250 2024-02-17T12:10:32.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.606+0000 7fb7b3fff640 1 --2- 172.21.15.94:0/559873445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693b70 0x55c5806af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:10:32.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.606+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/559873445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 0x55c58069be70 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb7b403d700 tx=0x7fb7b401a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f4db4f7b01bcb0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.606+0000 7fb7b17fa640 1 -- 172.21.15.94:0/559873445 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4061070 con 0x55c580693250 2024-02-17T12:10:32.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.606+0000 7fb7b17fa640 1 -- 172.21.15.94:0/559873445 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7b40519d0 con 0x55c580693250 2024-02-17T12:10:32.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.606+0000 7fb7b17fa640 1 -- 172.21.15.94:0/559873445 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4064470 con 0x55c580693250 2024-02-17T12:10:32.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.607+0000 7fb7c4c09740 1 -- 172.21.15.94:0/559873445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 msgr2=0x55c58069be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.607+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/559873445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 0x55c58069be70 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb7b403d700 tx=0x7fb7b401a880 comp rx=0 tx=0).stop 2024-02-17T12:10:32.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.607+0000 7fb7c4c09740 1 -- 172.21.15.94:0/559873445 shutdown_connections 2024-02-17T12:10:32.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.607+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/559873445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c58069c750 0x55c580699fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.607+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/559873445 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693b70 0x55c5806af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.607+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/559873445 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c580693250 0x55c58069be70 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.607+0000 7fb7c4c09740 1 -- 172.21.15.94:0/559873445 >> 172.21.15.94:0/559873445 conn(0x55c58068d980 msgr2=0x55c580694ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.607+0000 7fb7c4c09740 1 -- 172.21.15.94:0/559873445 shutdown_connections 2024-02-17T12:10:32.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.607+0000 7fb7c4c09740 1 -- 172.21.15.94:0/559873445 wait complete. 2024-02-17T12:10:32.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.607+0000 7fb7c4c09740 1 -- start start 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.608+0000 7fb7c4c09740 1 Processor -- start 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.608+0000 7fb7c4c09740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693250 0x55c5806a2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.608+0000 7fb7c4c09740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693b70 0x55c5806a2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.608+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693250 0x55c5806a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.608+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693250 0x55c5806a2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42002/0 (socket says 172.21.15.94:42002) 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.608+0000 7fb7b3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693b70 0x55c5806a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.608+0000 7fb7b3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693b70 0x55c5806a2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47862/0 (socket says 172.21.15.94:47862) 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.608+0000 7fb7c4c09740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c58069c750 0x55c5806a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.608+0000 7fb7bbfff640 1 -- 172.21.15.94:0/4240848622 learned_addr learned my addr 172.21.15.94:0/4240848622 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7c4c09740 1 -- 172.21.15.94:0/4240848622 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c580703180 con 0x55c58069c750 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7c4c09740 1 -- 172.21.15.94:0/4240848622 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5807032c0 con 0x55c580693250 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7c4c09740 1 -- 172.21.15.94:0/4240848622 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c580703430 con 0x55c580693b70 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7c4c09740 10 client.0 _ll_register_callbacks cb 0x7fb7bc116e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:32.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/4240848622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c58069c750 0x55c5806a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7bbfff640 1 -- 172.21.15.94:0/4240848622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693b70 msgr2=0x55c5806a2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/4240848622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693b70 0x55c5806a2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7bbfff640 1 -- 172.21.15.94:0/4240848622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c58069c750 msgr2=0x55c5806a2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/4240848622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c58069c750 0x55c5806a2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7bbfff640 1 -- 172.21.15.94:0/4240848622 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5805bd060 con 0x55c580693250 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7b3fff640 1 --2- 172.21.15.94:0/4240848622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693b70 0x55c5806a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7bbfff640 1 --2- 172.21.15.94:0/4240848622 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693250 0x55c5806a2100 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb7b403d1b0 tx=0x7fb7b4000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9c092a57cb88430 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7aaffd640 1 -- 172.21.15.94:0/4240848622 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b4061070 con 0x55c580693250 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.609+0000 7fb7aaffd640 1 -- 172.21.15.94:0/4240848622 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb7b4064d90 con 0x55c580693250 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.610+0000 7fb7c4c09740 1 -- 172.21.15.94:0/4240848622 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5806890f0 con 0x55c580693250 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.610+0000 7fb7aaffd640 1 -- 172.21.15.94:0/4240848622 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7b40434b0 con 0x55c580693250 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.610+0000 7fb7c4c09740 10 client.28199 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:32.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.610+0000 7fb7c4c09740 1 -- 172.21.15.94:0/4240848622 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c5805bb8c0 con 0x55c580693250 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.610+0000 7fb7aaffd640 1 -- 172.21.15.94:0/4240848622 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c5806890f0 con 0x55c580693250 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.610+0000 7fb7aaffd640 1 -- 172.21.15.94:0/4240848622 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb7b4058070 con 0x55c580693250 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.610+0000 7fb7c4c09740 10 client.28199 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.610+0000 7fb7c4c09740 10 client.28199 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.610+0000 7fb7c4c09740 20 client.28199 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.610+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/4240848622 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c5806acc10 0x55c580706700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.610+0000 7fb7c4c09740 4 client.28199 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.610+0000 7fb7c4c09740 1 -- 172.21.15.94:0/4240848622 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fb7b4043cb0 con 0x55c5806acc10 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7b3fff640 1 --2- 172.21.15.94:0/4240848622 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c5806acc10 0x55c580706700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7aaffd640 10 client.28199 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7b3fff640 1 --2- 172.21.15.94:0/4240848622 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c5806acc10 0x55c580706700 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbb89adafa9e462c server_cookie=3197f21b16d082f9 in_seq=0 out_seq=0 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7aaffd640 1 -- 172.21.15.94:0/4240848622 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fb7b4043cb0 con 0x55c5806acc10 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7aaffd640 10 client.28199 handle_command_reply: tid=0 2024-02-17T12:10:32.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7c4c09740 1 client.28199 shutdown 2024-02-17T12:10:32.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7c4c09740 1 -- 172.21.15.94:0/4240848622 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693250 msgr2=0x55c5806a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/4240848622 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693250 0x55c5806a2100 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb7b403d1b0 tx=0x7fb7b4000f40 comp rx=0 tx=0).stop 2024-02-17T12:10:32.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7c4c09740 1 -- 172.21.15.94:0/4240848622 shutdown_connections 2024-02-17T12:10:32.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/4240848622 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c5806acc10 0x55c580706700 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/4240848622 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c58069c750 0x55c5806a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7aaffd640 0 client.28199 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:32.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/4240848622 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c580693b70 0x55c5806a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7c4c09740 1 --2- 172.21.15.94:0/4240848622 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c580693250 0x55c5806a2100 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.611+0000 7fb7c4c09740 1 -- 172.21.15.94:0/4240848622 >> 172.21.15.94:0/4240848622 conn(0x55c58068d980 msgr2=0x55c58068c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.612+0000 7fb7c4c09740 1 -- 172.21.15.94:0/4240848622 shutdown_connections 2024-02-17T12:10:32.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.612+0000 7fb7c4c09740 1 -- 172.21.15.94:0/4240848622 wait complete. 2024-02-17T12:10:32.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.612+0000 7fb7c4c09740 20 client.28199 trim_cache size 0 max 16384 2024-02-17T12:10:32.619 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:10:32.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.612+0000 7fb7c2039640 1 -- 172.21.15.94:0/3724747916 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb79c0a22f0 msgr2=0x7fb79c0a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.612+0000 7fb7c2039640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb79c0a22f0 0x7fb79c0a47b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55c580692d20 tx=0x7fb7ac096e60 comp rx=0 tx=0).stop 2024-02-17T12:10:32.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.612+0000 7fb7c2039640 1 -- 172.21.15.94:0/3724747916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7bc10d9d0 msgr2=0x7fb7bc116190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:32.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.612+0000 7fb7c2039640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7bc10d9d0 0x7fb7bc116190 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac00c920 tx=0x7fb7ac00cde0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.612+0000 7fb7c2039640 1 -- 172.21.15.94:0/3724747916 shutdown_connections 2024-02-17T12:10:32.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.612+0000 7fb7c2039640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb79c0a22f0 0x7fb79c0a47b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.612+0000 7fb7c2039640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb7bc11a960 0x7fb7bc1a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.612+0000 7fb7c2039640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7bc10f9a0 0x7fb7bc1166d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.612+0000 7fb7c2039640 1 --2- 172.21.15.94:0/3724747916 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb7bc10d9d0 0x7fb7bc116190 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:32.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.612+0000 7fb7c2039640 1 -- 172.21.15.94:0/3724747916 >> 172.21.15.94:0/3724747916 conn(0x7fb7bc06c260 msgr2=0x7fb7bc11bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:32.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.613+0000 7fb7c2039640 1 -- 172.21.15.94:0/3724747916 shutdown_connections 2024-02-17T12:10:32.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:32.613+0000 7fb7c2039640 1 -- 172.21.15.94:0/3724747916 wait complete. 2024-02-17T12:10:32.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:32 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/625866767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:10:32.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:32 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/625866767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:10:32.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:32 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/625866767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:10:33.628 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:10:33.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:10:33.629 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:10:33.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.780+0000 7f2df96f4640 1 -- 172.21.15.94:0/2921466949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2df41083f0 msgr2=0x7f2df41087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:33.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.780+0000 7f2df96f4640 1 --2- 172.21.15.94:0/2921466949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2df41083f0 0x7f2df41087d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009900 tx=0x7f2ddc02bc60 comp rx=0 tx=0).stop 2024-02-17T12:10:33.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.786+0000 7f2df96f4640 1 -- 172.21.15.94:0/2921466949 shutdown_connections 2024-02-17T12:10:33.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.786+0000 7f2df96f4640 1 --2- 172.21.15.94:0/2921466949 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2df410cf00 0x7f2df4114a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.786+0000 7f2df96f4640 1 --2- 172.21.15.94:0/2921466949 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df4108da0 0x7f2df410c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.786+0000 7f2df96f4640 1 --2- 172.21.15.94:0/2921466949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2df41083f0 0x7f2df41087d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.786+0000 7f2df96f4640 1 -- 172.21.15.94:0/2921466949 >> 172.21.15.94:0/2921466949 conn(0x7f2df406c6c0 msgr2=0x7f2df406cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:33.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.793+0000 7f2df96f4640 1 -- 172.21.15.94:0/2921466949 shutdown_connections 2024-02-17T12:10:33.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.794+0000 7f2df96f4640 1 -- 172.21.15.94:0/2921466949 wait complete. 2024-02-17T12:10:33.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.795+0000 7f2df96f4640 1 Processor -- start 2024-02-17T12:10:33.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.795+0000 7f2df96f4640 1 -- start start 2024-02-17T12:10:33.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.796+0000 7f2df96f4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2df41083f0 0x7f2df41a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.796+0000 7f2df96f4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2df410cf00 0x7f2df41a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.796+0000 7f2df96f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df419dcf0 0x7f2df419e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.796+0000 7f2df96f4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df419e900 con 0x7f2df419dcf0 2024-02-17T12:10:33.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.796+0000 7f2df96f4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df419ea70 con 0x7f2df410cf00 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.796+0000 7f2df96f4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df41a2740 con 0x7f2df41083f0 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.796+0000 7f2df27fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2df410cf00 0x7f2df41a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.796+0000 7f2df27fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2df410cf00 0x7f2df41a5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42022/0 (socket says 172.21.15.94:42022) 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.796+0000 7f2df27fc640 1 -- 172.21.15.94:0/3475015322 learned_addr learned my addr 172.21.15.94:0/3475015322 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.796+0000 7f2df37fe640 1 --2- 172.21.15.94:0/3475015322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df419dcf0 0x7f2df419e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.797+0000 7f2df27fc640 1 -- 172.21.15.94:0/3475015322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2df41083f0 msgr2=0x7f2df41a4c20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.797+0000 7f2df27fc640 1 --2- 172.21.15.94:0/3475015322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2df41083f0 0x7f2df41a4c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.797+0000 7f2df27fc640 1 -- 172.21.15.94:0/3475015322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df419dcf0 msgr2=0x7f2df419e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.797+0000 7f2df27fc640 1 --2- 172.21.15.94:0/3475015322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df419dcf0 0x7f2df419e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.797+0000 7f2df27fc640 1 -- 172.21.15.94:0/3475015322 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc009560 con 0x7f2df410cf00 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.797+0000 7f2df27fc640 1 --2- 172.21.15.94:0/3475015322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2df410cf00 0x7f2df41a5160 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2de400b990 tx=0x7f2de400be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.798+0000 7f2debfff640 1 -- 172.21.15.94:0/3475015322 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de400cba0 con 0x7f2df410cf00 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.798+0000 7f2df96f4640 1 -- 172.21.15.94:0/3475015322 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df41a2a20 con 0x7f2df410cf00 2024-02-17T12:10:33.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.798+0000 7f2df96f4640 1 -- 172.21.15.94:0/3475015322 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df41a2f30 con 0x7f2df410cf00 2024-02-17T12:10:33.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.798+0000 7f2debfff640 1 -- 172.21.15.94:0/3475015322 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2de400cd00 con 0x7f2df410cf00 2024-02-17T12:10:33.802 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.798+0000 7f2debfff640 1 -- 172.21.15.94:0/3475015322 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4012720 con 0x7f2df410cf00 2024-02-17T12:10:33.804 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.803+0000 7f2dfc3a6740 1 Processor -- start 2024-02-17T12:10:33.805 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.803+0000 7f2dfc3a6740 1 -- start start 2024-02-17T12:10:33.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.804+0000 7f2dfc3a6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926cdb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.804+0000 7f2dfc3a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdbd30 0x555926ce42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.804+0000 7f2dfc3a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926ce48b0 0x555926ce6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.804+0000 7f2debfff640 1 -- 172.21.15.94:0/3475015322 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2de40129c0 con 0x7f2df410cf00 2024-02-17T12:10:33.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.804+0000 7f2dfc3a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555926c49f90 con 0x555926ce48b0 2024-02-17T12:10:33.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.804+0000 7f2dfc3a6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555926c474c0 con 0x555926cdb380 2024-02-17T12:10:33.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.804+0000 7f2dfc3a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555926c42ca0 con 0x555926cdbd30 2024-02-17T12:10:33.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.804+0000 7f2df27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926ce48b0 0x555926ce6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.805+0000 7f2df27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926ce48b0 0x555926ce6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40944/0 (socket says 172.21.15.94:40944) 2024-02-17T12:10:33.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.805+0000 7f2df27fc640 1 -- 172.21.15.94:0/3709353891 learned_addr learned my addr 172.21.15.94:0/3709353891 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:33.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.805+0000 7f2df37fe640 1 --2- 172.21.15.94:0/3709353891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926cdb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.805+0000 7f2debfff640 1 --2- 172.21.15.94:0/3475015322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2de00a2430 0x7f2de00a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.807+0000 7f2df37fe640 1 --2- 172.21.15.94:0/3475015322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2de00a2430 0x7f2de00a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.808+0000 7f2df37fe640 1 -- 172.21.15.94:0/3709353891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdbd30 msgr2=0x555926ce42e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:10:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.808+0000 7f2df37fe640 1 --2- 172.21.15.94:0/3709353891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdbd30 0x555926ce42e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.808+0000 7f2df37fe640 1 -- 172.21.15.94:0/3709353891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926ce48b0 msgr2=0x555926ce6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:33.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.808+0000 7f2df37fe640 1 --2- 172.21.15.94:0/3709353891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926ce48b0 0x555926ce6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.808+0000 7f2df37fe640 1 -- 172.21.15.94:0/3709353891 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555926c050b0 con 0x555926cdb380 2024-02-17T12:10:33.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.808+0000 7f2debfff640 1 -- 172.21.15.94:0/3475015322 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2de40cd780 con 0x7f2df410cf00 2024-02-17T12:10:33.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.808+0000 7f2df37fe640 1 --2- 172.21.15.94:0/3709353891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926cdb760 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00bca0 tx=0x7f2dec0039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2265dd3211a32d09 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:33.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.809+0000 7f2df37fe640 1 --2- 172.21.15.94:0/3475015322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2de00a2430 0x7f2de00a48f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2dec01b000 tx=0x7f2dec01d000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:33.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.809+0000 7f2de97fa640 1 -- 172.21.15.94:0/3709353891 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dec005220 con 0x555926cdb380 2024-02-17T12:10:33.811 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.809+0000 7f2de97fa640 1 -- 172.21.15.94:0/3709353891 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2dec012e80 con 0x555926cdb380 2024-02-17T12:10:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.810+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3709353891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 msgr2=0x555926cdb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.810+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3709353891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926cdb760 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2dec00bca0 tx=0x7f2dec0039c0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.812+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3709353891 shutdown_connections 2024-02-17T12:10:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.812+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3709353891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926ce48b0 0x555926ce6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.812+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3709353891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdbd30 0x555926ce42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.817 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.812+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3709353891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926cdb760 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.812+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3709353891 >> 172.21.15.94:0/3709353891 conn(0x555926cd5840 msgr2=0x555926cd5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:33.818 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.816+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3709353891 shutdown_connections 2024-02-17T12:10:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.816+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3709353891 wait complete. 2024-02-17T12:10:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.816+0000 7f2dfc3a6740 1 -- start start 2024-02-17T12:10:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.817+0000 7f2dfc3a6740 1 Processor -- start 2024-02-17T12:10:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.817+0000 7f2dfc3a6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926ce1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.817+0000 7f2dfc3a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926cdbd30 0x555926ce1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.817+0000 7f2dfc3a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926ce48b0 0x555926ce21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.817+0000 7f2dfc3a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555926be7f40 con 0x555926cdbd30 2024-02-17T12:10:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.817+0000 7f2dfc3a6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559269164b0 con 0x555926cdb380 2024-02-17T12:10:33.819 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.817+0000 7f2dfc3a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555926bc8670 con 0x555926ce48b0 2024-02-17T12:10:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.817+0000 7f2dfc3a6740 10 client.0 _ll_register_callbacks cb 0x555926c35f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.817+0000 7f2df2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926ce1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.817+0000 7f2df2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926ce1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42034/0 (socket says 172.21.15.94:42034) 2024-02-17T12:10:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.817+0000 7f2df2ffd640 1 -- 172.21.15.94:0/3506404043 learned_addr learned my addr 172.21.15.94:0/3506404043 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.818+0000 7f2df27fc640 1 --2- 172.21.15.94:0/3506404043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926cdbd30 0x555926ce1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.818+0000 7f2df2ffd640 1 -- 172.21.15.94:0/3506404043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926ce48b0 msgr2=0x555926ce21a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:10:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.818+0000 7f2df2ffd640 1 --2- 172.21.15.94:0/3506404043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926ce48b0 0x555926ce21a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.818+0000 7f2df2ffd640 1 -- 172.21.15.94:0/3506404043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926cdbd30 msgr2=0x555926ce1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.818+0000 7f2df2ffd640 1 --2- 172.21.15.94:0/3506404043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926cdbd30 0x555926ce1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.820 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.818+0000 7f2df2ffd640 1 -- 172.21.15.94:0/3506404043 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dec00b880 con 0x555926cdb380 2024-02-17T12:10:33.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.819+0000 7f2df2ffd640 1 --2- 172.21.15.94:0/3506404043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926ce1720 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc000c00 tx=0x7f2ddc02ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4e9032507faa50c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:33.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.819+0000 7f2dda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:10:33.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.819+0000 7f2dda7fc640 1 -- 172.21.15.94:0/3506404043 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc02c600 con 0x555926cdb380 2024-02-17T12:10:33.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.819+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3506404043 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555926c1cf40 con 0x555926cdb380 2024-02-17T12:10:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.819+0000 7f2dda7fc640 1 -- 172.21.15.94:0/3506404043 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2ddc004040 con 0x555926cdb380 2024-02-17T12:10:33.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.819+0000 7f2dda7fc640 1 -- 172.21.15.94:0/3506404043 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ddc004220 con 0x555926cdb380 2024-02-17T12:10:33.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.823+0000 7f2dfc3a6740 10 client.28211 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:33.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.823+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3506404043 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555926c41320 con 0x555926cdb380 2024-02-17T12:10:33.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.824+0000 7f2dda7fc640 1 -- 172.21.15.94:0/3506404043 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555926c1cf40 con 0x555926cdb380 2024-02-17T12:10:33.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.825+0000 7f2dda7fc640 1 -- 172.21.15.94:0/3506404043 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2ddc03c070 con 0x555926cdb380 2024-02-17T12:10:33.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.825+0000 7f2dfc3a6740 10 client.28211 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:33.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.825+0000 7f2dfc3a6740 10 client.28211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:33.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.825+0000 7f2dfc3a6740 20 client.28211 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:33.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.825+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3506404043 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555926cf4ea0 0x555926cf72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.825+0000 7f2dfc3a6740 4 client.28211 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:10:33.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.825+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3506404043 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555926c39310 con 0x555926cf4ea0 2024-02-17T12:10:33.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.826+0000 7f2df27fc640 1 --2- 172.21.15.94:0/3506404043 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555926cf4ea0 0x555926cf72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.827+0000 7f2df27fc640 1 --2- 172.21.15.94:0/3506404043 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555926cf4ea0 0x555926cf72c0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=833d52ad510abf2b server_cookie=17f42984ddd369ab in_seq=0 out_seq=0 2024-02-17T12:10:33.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.827+0000 7f2dda7fc640 10 client.28211 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:33.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.828+0000 7f2dda7fc640 1 -- 172.21.15.94:0/3506404043 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555926c39310 con 0x555926cf4ea0 2024-02-17T12:10:33.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.828+0000 7f2dda7fc640 10 client.28211 handle_command_reply: tid=0 2024-02-17T12:10:33.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.828+0000 7f2dfc3a6740 1 client.28211 shutdown 2024-02-17T12:10:33.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.829+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3506404043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 msgr2=0x555926ce1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:33.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.829+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3506404043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926ce1720 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc000c00 tx=0x7f2ddc02ca40 comp rx=0 tx=0).stop 2024-02-17T12:10:33.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.829+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3506404043 shutdown_connections 2024-02-17T12:10:33.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.829+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3506404043 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555926cf4ea0 0x555926cf72c0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.829+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3506404043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926ce48b0 0x555926ce21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.829+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3506404043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926cdbd30 0x555926ce1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.829+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3506404043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926ce1720 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.829+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3506404043 >> 172.21.15.94:0/3506404043 conn(0x555926cd5840 msgr2=0x555926cd77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:33.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.829+0000 7f2dda7fc640 0 client.28211 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:33.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.830+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3506404043 shutdown_connections 2024-02-17T12:10:33.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.830+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3506404043 wait complete. 2024-02-17T12:10:33.832 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.830+0000 7f2dfc3a6740 20 client.28211 trim_cache size 0 max 16384 2024-02-17T12:10:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.852+0000 7f2dfc3a6740 1 Processor -- start 2024-02-17T12:10:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.852+0000 7f2dfc3a6740 1 -- start start 2024-02-17T12:10:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2dfc3a6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926ce3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2dfc3a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdbd30 0x555926ce0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2dfc3a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926ce48b0 0x555926ce1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2dfc3a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555926c39340 con 0x555926ce48b0 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2dfc3a6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555926ce4680 con 0x555926cdb380 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2dfc3a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555926cfc520 con 0x555926cdbd30 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2df37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926ce48b0 0x555926ce1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2df2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926ce3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2df2ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926ce3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42036/0 (socket says 172.21.15.94:42036) 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2df37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926ce48b0 0x555926ce1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40960/0 (socket says 172.21.15.94:40960) 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2df2ffd640 1 -- 172.21.15.94:0/789726141 learned_addr learned my addr 172.21.15.94:0/789726141 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2df27fc640 1 --2- 172.21.15.94:0/789726141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdbd30 0x555926ce0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2df2ffd640 1 -- 172.21.15.94:0/789726141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdbd30 msgr2=0x555926ce0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2df2ffd640 1 --2- 172.21.15.94:0/789726141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdbd30 0x555926ce0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2df2ffd640 1 -- 172.21.15.94:0/789726141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926ce48b0 msgr2=0x555926ce1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2df2ffd640 1 --2- 172.21.15.94:0/789726141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926ce48b0 0x555926ce1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2df2ffd640 1 -- 172.21.15.94:0/789726141 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555926bddc00 con 0x555926cdb380 2024-02-17T12:10:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.853+0000 7f2df37fe640 1 --2- 172.21.15.94:0/789726141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926ce48b0 0x555926ce1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:33.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.854+0000 7f2df2ffd640 1 --2- 172.21.15.94:0/789726141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926ce3e50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc006010 tx=0x7f2ddc005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=715bc06a37fd9929 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:33.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.854+0000 7f2de8ff9640 1 -- 172.21.15.94:0/789726141 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc003a70 con 0x555926cdb380 2024-02-17T12:10:33.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.854+0000 7f2de8ff9640 1 -- 172.21.15.94:0/789726141 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ddc02c600 con 0x555926cdb380 2024-02-17T12:10:33.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.854+0000 7f2de8ff9640 1 -- 172.21.15.94:0/789726141 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc005470 con 0x555926cdb380 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.855+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/789726141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 msgr2=0x555926ce3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.855+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/789726141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926ce3e50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc006010 tx=0x7f2ddc005a20 comp rx=0 tx=0).stop 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.855+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/789726141 shutdown_connections 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.855+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/789726141 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926ce48b0 0x555926ce1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.855+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/789726141 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdbd30 0x555926ce0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.855+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/789726141 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926cdb380 0x555926ce3e50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.855+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/789726141 >> 172.21.15.94:0/789726141 conn(0x555926cd5840 msgr2=0x555926cdfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.855+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/789726141 shutdown_connections 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.855+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/789726141 wait complete. 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.855+0000 7f2dfc3a6740 1 -- start start 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2dfc3a6740 1 Processor -- start 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2dfc3a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdb380 0x555926cebc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2dfc3a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926cdbd30 0x555926cea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.861 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2dfc3a6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926ce48b0 0x555926cea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2df27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926cdbd30 0x555926cea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2df2ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdb380 0x555926cebc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2df27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926cdbd30 0x555926cea010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40968/0 (socket says 172.21.15.94:40968) 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2df2ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdb380 0x555926cebc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:47934/0 (socket says 172.21.15.94:47934) 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2dfc3a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555926ceab60 con 0x555926cdbd30 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2dfc3a6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555926ceacd0 con 0x555926ce48b0 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2dfc3a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555926ceae90 con 0x555926cdb380 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2df27fc640 1 -- 172.21.15.94:0/3565020464 learned_addr learned my addr 172.21.15.94:0/3565020464 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2dfc3a6740 10 client.0 _ll_register_callbacks cb 0x7f2dec004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2df37fe640 1 --2- 172.21.15.94:0/3565020464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926ce48b0 0x555926cea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2df27fc640 1 -- 172.21.15.94:0/3565020464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdb380 msgr2=0x555926cebc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2df27fc640 1 --2- 172.21.15.94:0/3565020464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdb380 0x555926cebc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2df27fc640 1 -- 172.21.15.94:0/3565020464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926ce48b0 msgr2=0x555926cea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2df27fc640 1 --2- 172.21.15.94:0/3565020464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926ce48b0 0x555926cea550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2df27fc640 1 -- 172.21.15.94:0/3565020464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dec021070 con 0x555926cdbd30 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.856+0000 7f2df2ffd640 1 --2- 172.21.15.94:0/3565020464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdb380 0x555926cebc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2df27fc640 1 --2- 172.21.15.94:0/3565020464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926cdbd30 0x555926cea010 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2de40966f0 tx=0x7f2de4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abac9218eeb07b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dda7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dda7fc640 1 -- 172.21.15.94:0/3565020464 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de409e170 con 0x555926cdbd30 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dda7fc640 1 -- 172.21.15.94:0/3565020464 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2de409e330 con 0x555926cdbd30 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3565020464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555926c68900 con 0x555926cdbd30 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dda7fc640 1 -- 172.21.15.94:0/3565020464 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2de40a2980 con 0x555926cdbd30 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dfc3a6740 10 client.18753 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3565020464 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555926c41320 con 0x555926cdbd30 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dda7fc640 1 -- 172.21.15.94:0/3565020464 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2de40a6080 con 0x555926cdbd30 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dda7fc640 1 -- 172.21.15.94:0/3565020464 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f2de40a2bd0 con 0x555926cdbd30 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dfc3a6740 10 client.18753 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dfc3a6740 10 client.18753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dfc3a6740 20 client.18753 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3565020464 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555926d4cb70 0x555926d4ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:33.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dfc3a6740 4 client.18753 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:10:33.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.857+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3565020464 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x555926942860 con 0x555926d4cb70 2024-02-17T12:10:33.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.858+0000 7f2df2ffd640 1 --2- 172.21.15.94:0/3565020464 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555926d4cb70 0x555926d4ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:33.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.858+0000 7f2dda7fc640 10 client.18753 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:33.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.858+0000 7f2df2ffd640 1 --2- 172.21.15.94:0/3565020464 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555926d4cb70 0x555926d4ef90 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae7dde90d950fa9 server_cookie=ea64e2b2612ea1aa in_seq=0 out_seq=0 2024-02-17T12:10:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.859+0000 7f2dda7fc640 1 -- 172.21.15.94:0/3565020464 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555926942860 con 0x555926d4cb70 2024-02-17T12:10:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.859+0000 7f2dda7fc640 10 client.18753 handle_command_reply: tid=0 2024-02-17T12:10:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.859+0000 7f2dfc3a6740 1 client.18753 shutdown 2024-02-17T12:10:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.859+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3565020464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926cdbd30 msgr2=0x555926cea010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.859+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3565020464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926cdbd30 0x555926cea010 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2de40966f0 tx=0x7f2de4004b50 comp rx=0 tx=0).stop 2024-02-17T12:10:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.859+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3565020464 shutdown_connections 2024-02-17T12:10:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.859+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3565020464 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555926d4cb70 0x555926d4ef90 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.859+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3565020464 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555926ce48b0 0x555926cea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.859+0000 7f2dda7fc640 0 client.18753 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:33.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.859+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3565020464 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555926cdbd30 0x555926cea010 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.859+0000 7f2dfc3a6740 1 --2- 172.21.15.94:0/3565020464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555926cdb380 0x555926cebc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.859+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3565020464 >> 172.21.15.94:0/3565020464 conn(0x555926cd5840 msgr2=0x555926ce2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:33.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.860+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3565020464 shutdown_connections 2024-02-17T12:10:33.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.860+0000 7f2dfc3a6740 1 -- 172.21.15.94:0/3565020464 wait complete. 2024-02-17T12:10:33.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.860+0000 7f2dfc3a6740 20 client.18753 trim_cache size 0 max 16384 2024-02-17T12:10:33.866 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:10:33.866 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:10:33.866 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "c27ac994-2794-44cb-9c7e-86c6c8d4b286", 2024-02-17T12:10:33.866 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:10:33.866 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:10:33.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.860+0000 7f2df96f4640 1 -- 172.21.15.94:0/3475015322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2de00a2430 msgr2=0x7f2de00a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:33.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.860+0000 7f2df96f4640 1 --2- 172.21.15.94:0/3475015322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2de00a2430 0x7f2de00a48f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2dec01b000 tx=0x7f2dec01d000 comp rx=0 tx=0).stop 2024-02-17T12:10:33.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.860+0000 7f2df96f4640 1 -- 172.21.15.94:0/3475015322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2df410cf00 msgr2=0x7f2df41a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:33.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.860+0000 7f2df96f4640 1 --2- 172.21.15.94:0/3475015322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2df410cf00 0x7f2df41a5160 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2de400b990 tx=0x7f2de400be50 comp rx=0 tx=0).stop 2024-02-17T12:10:33.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.860+0000 7f2df96f4640 1 -- 172.21.15.94:0/3475015322 shutdown_connections 2024-02-17T12:10:33.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.860+0000 7f2df96f4640 1 --2- 172.21.15.94:0/3475015322 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2de00a2430 0x7f2de00a48f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.860+0000 7f2df96f4640 1 --2- 172.21.15.94:0/3475015322 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2df419dcf0 0x7f2df419e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.860+0000 7f2df96f4640 1 --2- 172.21.15.94:0/3475015322 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2df410cf00 0x7f2df41a5160 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.860+0000 7f2df96f4640 1 --2- 172.21.15.94:0/3475015322 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2df41083f0 0x7f2df41a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:33.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.860+0000 7f2df96f4640 1 -- 172.21.15.94:0/3475015322 >> 172.21.15.94:0/3475015322 conn(0x7f2df406c6c0 msgr2=0x7f2df4070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:33.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.861+0000 7f2df96f4640 1 -- 172.21.15.94:0/3475015322 shutdown_connections 2024-02-17T12:10:33.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:33.861+0000 7f2df96f4640 1 -- 172.21.15.94:0/3475015322 wait complete. 2024-02-17T12:10:33.877 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:33 smithi094 ceph-mon[32499]: pgmap v388: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 4.8 MiB/s wr, 509 op/s 2024-02-17T12:10:33.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:10:33.878 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:10:33.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:33 smithi189 ceph-mon[36588]: pgmap v388: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 4.8 MiB/s wr, 509 op/s 2024-02-17T12:10:33.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:33 smithi125 ceph-mon[36819]: pgmap v388: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 4.8 MiB/s wr, 509 op/s 2024-02-17T12:10:34.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.049+0000 7fc468e4e640 1 -- 172.21.15.94:0/1269367541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc46410f8a0 msgr2=0x7fc46410d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:34.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.049+0000 7fc468e4e640 1 --2- 172.21.15.94:0/1269367541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc46410f8a0 0x7fc46410d4c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc45401a910 tx=0x7fc45403cca0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.055+0000 7fc468e4e640 1 -- 172.21.15.94:0/1269367541 shutdown_connections 2024-02-17T12:10:34.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.055+0000 7fc468e4e640 1 --2- 172.21.15.94:0/1269367541 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc46411a960 0x7fc46411cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.055+0000 7fc468e4e640 1 --2- 172.21.15.94:0/1269367541 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46410fd10 0x7fc46410da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.055+0000 7fc468e4e640 1 --2- 172.21.15.94:0/1269367541 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc46410f8a0 0x7fc46410d4c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.055+0000 7fc468e4e640 1 -- 172.21.15.94:0/1269367541 >> 172.21.15.94:0/1269367541 conn(0x7fc46406c270 msgr2=0x7fc46406c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:34.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.059+0000 7fc468e4e640 1 -- 172.21.15.94:0/1269367541 shutdown_connections 2024-02-17T12:10:34.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.061+0000 7fc468e4e640 1 -- 172.21.15.94:0/1269367541 wait complete. 2024-02-17T12:10:34.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.062+0000 7fc468e4e640 1 Processor -- start 2024-02-17T12:10:34.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.063+0000 7fc468e4e640 1 -- start start 2024-02-17T12:10:34.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.063+0000 7fc468e4e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46410f8a0 0x7fc46411a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.063+0000 7fc468e4e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc46410fd10 0x7fc4641160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.063+0000 7fc468e4e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc46411a960 0x7fc464116630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.063+0000 7fc468e4e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc464116c30 con 0x7fc46410f8a0 2024-02-17T12:10:34.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.063+0000 7fc468e4e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc464116da0 con 0x7fc46411a960 2024-02-17T12:10:34.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.063+0000 7fc468e4e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc464116f60 con 0x7fc46410fd10 2024-02-17T12:10:34.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.063+0000 7fc462d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc46411a960 0x7fc464116630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.063+0000 7fc462d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc46411a960 0x7fc464116630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42048/0 (socket says 172.21.15.94:42048) 2024-02-17T12:10:34.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.063+0000 7fc462d9d640 1 -- 172.21.15.94:0/1099478977 learned_addr learned my addr 172.21.15.94:0/1099478977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:34.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.064+0000 7fc46259c640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46410f8a0 0x7fc46411a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.064+0000 7fc462d9d640 1 -- 172.21.15.94:0/1099478977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc46410fd10 msgr2=0x7fc4641160f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:10:34.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.064+0000 7fc462d9d640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc46410fd10 0x7fc4641160f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.064+0000 7fc462d9d640 1 -- 172.21.15.94:0/1099478977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46410f8a0 msgr2=0x7fc46411a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:34.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.064+0000 7fc462d9d640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46410f8a0 0x7fc46411a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.064+0000 7fc462d9d640 1 -- 172.21.15.94:0/1099478977 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45401a570 con 0x7fc46411a960 2024-02-17T12:10:34.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.064+0000 7fc46259c640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46410f8a0 0x7fc46411a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:10:34.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.064+0000 7fc462d9d640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc46411a960 0x7fc464116630 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00b9e0 tx=0x7fc45c00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:34.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.064+0000 7fc4537fe640 1 -- 172.21.15.94:0/1099478977 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c00cac0 con 0x7fc46411a960 2024-02-17T12:10:34.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.064+0000 7fc468e4e640 1 -- 172.21.15.94:0/1099478977 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4641aa680 con 0x7fc46411a960 2024-02-17T12:10:34.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.064+0000 7fc468e4e640 1 -- 172.21.15.94:0/1099478977 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4641aab60 con 0x7fc46411a960 2024-02-17T12:10:34.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.064+0000 7fc4537fe640 1 -- 172.21.15.94:0/1099478977 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc45c00cc20 con 0x7fc46411a960 2024-02-17T12:10:34.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.064+0000 7fc4537fe640 1 -- 172.21.15.94:0/1099478977 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0346a0 con 0x7fc46411a960 2024-02-17T12:10:34.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.067+0000 7fc4537fe640 1 -- 172.21.15.94:0/1099478977 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc45c036bd0 con 0x7fc46411a960 2024-02-17T12:10:34.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.068+0000 7fc4537fe640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc44c0a2370 0x7fc44c0a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.068+0000 7fc4537fe640 1 -- 172.21.15.94:0/1099478977 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc45c0ef7a0 con 0x7fc46411a960 2024-02-17T12:10:34.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.069+0000 7fc46259c640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc44c0a2370 0x7fc44c0a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.070+0000 7fc46ba1e740 1 Processor -- start 2024-02-17T12:10:34.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.070+0000 7fc46259c640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc44c0a2370 0x7fc44c0a4830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc454000c00 tx=0x7fc45403d1c0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:34.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.070+0000 7fc46ba1e740 1 -- start start 2024-02-17T12:10:34.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.070+0000 7fc46ba1e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e61690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.070+0000 7fc46ba1e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e6a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.070+0000 7fc46ba1e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e6a7b0 0x561042e6cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.071+0000 7fc46259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e61690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.071+0000 7fc46259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e61690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42060/0 (socket says 172.21.15.94:42060) 2024-02-17T12:10:34.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.071+0000 7fc461d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e6a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.071+0000 7fc461d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e6a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41002/0 (socket says 172.21.15.94:41002) 2024-02-17T12:10:34.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.071+0000 7fc46259c640 1 -- 172.21.15.94:0/1579729616 learned_addr learned my addr 172.21.15.94:0/1579729616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:34.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.071+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1579729616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561042dd0280 con 0x561042e61bd0 2024-02-17T12:10:34.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.071+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1579729616 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561042dcd7b0 con 0x561042e612b0 2024-02-17T12:10:34.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.071+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1579729616 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561042dc8f90 con 0x561042e6a7b0 2024-02-17T12:10:34.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.072+0000 7fc461d9b640 1 -- 172.21.15.94:0/1579729616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e6a7b0 msgr2=0x561042e6cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:34.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.072+0000 7fc461d9b640 1 --2- 172.21.15.94:0/1579729616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e6a7b0 0x561042e6cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.074 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.072+0000 7fc461d9b640 1 -- 172.21.15.94:0/1579729616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 msgr2=0x561042e61690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:34.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.072+0000 7fc461d9b640 1 --2- 172.21.15.94:0/1579729616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e61690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.072+0000 7fc461d9b640 1 -- 172.21.15.94:0/1579729616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561042d87570 con 0x561042e61bd0 2024-02-17T12:10:34.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.073+0000 7fc461d9b640 1 --2- 172.21.15.94:0/1579729616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e6a270 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc464117940 tx=0x7fc458009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941a9d776e480e81 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:34.075 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.074+0000 7fc450ff9640 1 -- 172.21.15.94:0/1579729616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800b890 con 0x561042e61bd0 2024-02-17T12:10:34.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.074+0000 7fc450ff9640 1 -- 172.21.15.94:0/1579729616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc458011420 con 0x561042e61bd0 2024-02-17T12:10:34.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.074+0000 7fc450ff9640 1 -- 172.21.15.94:0/1579729616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4580105c0 con 0x561042e61bd0 2024-02-17T12:10:34.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.075+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1579729616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 msgr2=0x561042e6a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:34.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.075+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/1579729616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e6a270 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fc464117940 tx=0x7fc458009b40 comp rx=0 tx=0).stop 2024-02-17T12:10:34.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.075+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1579729616 shutdown_connections 2024-02-17T12:10:34.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.075+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/1579729616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e6a7b0 0x561042e6cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.075+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/1579729616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e6a270 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.075+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/1579729616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e61690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.075+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1579729616 >> 172.21.15.94:0/1579729616 conn(0x561042e5b990 msgr2=0x561042e5bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:34.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.076+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1579729616 shutdown_connections 2024-02-17T12:10:34.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.077+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1579729616 wait complete. 2024-02-17T12:10:34.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.077+0000 7fc46ba1e740 1 -- start start 2024-02-17T12:10:34.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.077+0000 7fc46ba1e740 1 Processor -- start 2024-02-17T12:10:34.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.078+0000 7fc46ba1e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e6a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.078+0000 7fc46ba1e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e675c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.078+0000 7fc46259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e6a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.078+0000 7fc46259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e6a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42072/0 (socket says 172.21.15.94:42072) 2024-02-17T12:10:34.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.078+0000 7fc46259c640 1 -- 172.21.15.94:0/1933254134 learned_addr learned my addr 172.21.15.94:0/1933254134 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:34.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.079+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e6a7b0 0x561042e67d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.079+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1933254134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561042a9c430 con 0x561042e61bd0 2024-02-17T12:10:34.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.079+0000 7fc461d9b640 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e675c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.079+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1933254134 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561042a58e00 con 0x561042e612b0 2024-02-17T12:10:34.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.079+0000 7fc462d9d640 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e6a7b0 0x561042e67d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.079+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1933254134 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561042d4dfd0 con 0x561042e6a7b0 2024-02-17T12:10:34.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.079+0000 7fc46ba1e740 10 client.0 _ll_register_callbacks cb 0x561042da3fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:34.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.079+0000 7fc461d9b640 1 -- 172.21.15.94:0/1933254134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e6a7b0 msgr2=0x561042e67d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:34.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.079+0000 7fc461d9b640 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e6a7b0 0x561042e67d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.079+0000 7fc461d9b640 1 -- 172.21.15.94:0/1933254134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 msgr2=0x561042e6a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:34.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.079+0000 7fc461d9b640 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e6a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.079+0000 7fc461d9b640 1 -- 172.21.15.94:0/1933254134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc458009310 con 0x561042e61bd0 2024-02-17T12:10:34.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.079+0000 7fc46259c640 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e6a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:10:34.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.080+0000 7fc461d9b640 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e675c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc458004810 tx=0x7fc458011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43586ea5b857344 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:34.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.080+0000 7fc449ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.080+0000 7fc449ffb640 1 -- 172.21.15.94:0/1933254134 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800baa0 con 0x561042e61bd0 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.080+0000 7fc449ffb640 1 -- 172.21.15.94:0/1933254134 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc45800bc00 con 0x561042e61bd0 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.081+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1933254134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561042ac87e0 con 0x561042e61bd0 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.081+0000 7fc449ffb640 1 -- 172.21.15.94:0/1933254134 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4580119f0 con 0x561042e61bd0 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.081+0000 7fc46ba1e740 10 client.18762 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.081+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1933254134 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561042dc7610 con 0x561042e61bd0 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.081+0000 7fc449ffb640 1 -- 172.21.15.94:0/1933254134 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc458030080 con 0x561042e61bd0 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.081+0000 7fc449ffb640 1 -- 172.21.15.94:0/1933254134 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc45801a550 con 0x561042e61bd0 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.081+0000 7fc46ba1e740 10 client.18762 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.081+0000 7fc46ba1e740 10 client.18762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.081+0000 7fc46ba1e740 20 client.18762 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.081+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561042e7a8e0 0x561042e7cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.081+0000 7fc46ba1e740 4 client.18762 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:10:34.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.081+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1933254134 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561042b05720 con 0x561042e7a8e0 2024-02-17T12:10:34.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.082+0000 7fc46259c640 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561042e7a8e0 0x561042e7cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.082+0000 7fc46259c640 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561042e7a8e0 0x561042e7cd20 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74bc0511f4f36d30 server_cookie=fa7bd0bcc39ee0b4 in_seq=0 out_seq=0 2024-02-17T12:10:34.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.082+0000 7fc449ffb640 10 client.18762 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:34.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc449ffb640 1 -- 172.21.15.94:0/1933254134 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561042b05720 con 0x561042e7a8e0 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc449ffb640 10 client.18762 handle_command_reply: tid=0 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc46ba1e740 1 client.18762 shutdown 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1933254134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 msgr2=0x561042e675c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e675c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fc458004810 tx=0x7fc458011900 comp rx=0 tx=0).stop 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1933254134 shutdown_connections 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561042e7a8e0 0x561042e7cd20 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc449ffb640 0 client.18762 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e6a7b0 0x561042e67d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e675c0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/1933254134 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e6a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1933254134 >> 172.21.15.94:0/1933254134 conn(0x561042e5b990 msgr2=0x561042e5d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1933254134 shutdown_connections 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc46ba1e740 1 -- 172.21.15.94:0/1933254134 wait complete. 2024-02-17T12:10:34.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.083+0000 7fc46ba1e740 20 client.18762 trim_cache size 0 max 16384 2024-02-17T12:10:34.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.103+0000 7fc46ba1e740 1 Processor -- start 2024-02-17T12:10:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.103+0000 7fc46ba1e740 1 -- start start 2024-02-17T12:10:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.103+0000 7fc46ba1e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e612b0 0x561042e665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.103+0000 7fc46ba1e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.103+0000 7fc46ba1e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e6a7b0 0x561042e71c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.103+0000 7fc46ba1e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561042e60630 con 0x561042e61bd0 2024-02-17T12:10:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.103+0000 7fc46ba1e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561042e68910 con 0x561042e6a7b0 2024-02-17T12:10:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.103+0000 7fc46ba1e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561042e6d480 con 0x561042e612b0 2024-02-17T12:10:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.104+0000 7fc462d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e6a7b0 0x561042e71c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.104+0000 7fc462d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e6a7b0 0x561042e71c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42088/0 (socket says 172.21.15.94:42088) 2024-02-17T12:10:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.104+0000 7fc462d9d640 1 -- 172.21.15.94:0/584798477 learned_addr learned my addr 172.21.15.94:0/584798477 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.104+0000 7fc461d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.104+0000 7fc46259c640 1 --2- 172.21.15.94:0/584798477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e612b0 0x561042e665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.104+0000 7fc462d9d640 1 -- 172.21.15.94:0/584798477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e612b0 msgr2=0x561042e665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.104+0000 7fc462d9d640 1 --2- 172.21.15.94:0/584798477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e612b0 0x561042e665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.104+0000 7fc462d9d640 1 -- 172.21.15.94:0/584798477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 msgr2=0x561042e716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.104+0000 7fc462d9d640 1 --2- 172.21.15.94:0/584798477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e716f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.104+0000 7fc462d9d640 1 -- 172.21.15.94:0/584798477 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561042d63ac0 con 0x561042e6a7b0 2024-02-17T12:10:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.104+0000 7fc462d9d640 1 --2- 172.21.15.94:0/584798477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e6a7b0 0x561042e71c30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0bbf00 tx=0x7fc45c0bc2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0bb296a7c4e8cd2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.105+0000 7fc44bfff640 1 -- 172.21.15.94:0/584798477 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0c0170 con 0x561042e6a7b0 2024-02-17T12:10:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.105+0000 7fc44bfff640 1 -- 172.21.15.94:0/584798477 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc45c0ba500 con 0x561042e6a7b0 2024-02-17T12:10:34.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.105+0000 7fc44bfff640 1 -- 172.21.15.94:0/584798477 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c0bb8b0 con 0x561042e6a7b0 2024-02-17T12:10:34.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.106+0000 7fc46ba1e740 1 -- 172.21.15.94:0/584798477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e6a7b0 msgr2=0x561042e71c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:34.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.106+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/584798477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e6a7b0 0x561042e71c30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0bbf00 tx=0x7fc45c0bc2c0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.106+0000 7fc46ba1e740 1 -- 172.21.15.94:0/584798477 shutdown_connections 2024-02-17T12:10:34.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.106+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/584798477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e6a7b0 0x561042e71c30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.106+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/584798477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e61bd0 0x561042e716f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.106+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/584798477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e612b0 0x561042e665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.106+0000 7fc46ba1e740 1 -- 172.21.15.94:0/584798477 >> 172.21.15.94:0/584798477 conn(0x561042e5b990 msgr2=0x561042e65e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:34.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.106+0000 7fc46ba1e740 1 -- 172.21.15.94:0/584798477 shutdown_connections 2024-02-17T12:10:34.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.106+0000 7fc46ba1e740 1 -- 172.21.15.94:0/584798477 wait complete. 2024-02-17T12:10:34.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.106+0000 7fc46ba1e740 1 -- start start 2024-02-17T12:10:34.109 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.106+0000 7fc46ba1e740 1 Processor -- start 2024-02-17T12:10:34.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46ba1e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e709d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46ba1e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e61bd0 0x561042e70f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46ba1e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e6a7b0 0x561042ecae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46ba1e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561042ecb390 con 0x561042e6a7b0 2024-02-17T12:10:34.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46ba1e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561042ecb500 con 0x561042e612b0 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46ba1e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561042ecb6c0 con 0x561042e61bd0 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46ba1e740 10 client.0 _ll_register_callbacks cb 0x7fc458006ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e709d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42090/0 (socket says 172.21.15.94:42090) 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46259c640 1 -- 172.21.15.94:0/513740034 learned_addr learned my addr 172.21.15.94:0/513740034 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc462d9d640 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e6a7b0 0x561042ecae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc461d9b640 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e61bd0 0x561042e70f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46259c640 1 -- 172.21.15.94:0/513740034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e61bd0 msgr2=0x561042e70f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46259c640 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e61bd0 0x561042e70f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46259c640 1 -- 172.21.15.94:0/513740034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e6a7b0 msgr2=0x561042ecae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46259c640 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e6a7b0 0x561042ecae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46259c640 1 -- 172.21.15.94:0/513740034 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4641aab60 con 0x561042e612b0 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc461d9b640 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e61bd0 0x561042e70f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:10:34.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc46259c640 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e709d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc454006010 tx=0x7fc45401a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6e5b34ca929a82e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.107+0000 7fc449ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:10:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc449ffb640 1 -- 172.21.15.94:0/513740034 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454005100 con 0x561042e612b0 2024-02-17T12:10:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc46ba1e740 1 -- 172.21.15.94:0/513740034 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc458011420 con 0x561042e612b0 2024-02-17T12:10:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc449ffb640 1 -- 172.21.15.94:0/513740034 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc454052e40 con 0x561042e612b0 2024-02-17T12:10:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc449ffb640 1 -- 172.21.15.94:0/513740034 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc454005260 con 0x561042e612b0 2024-02-17T12:10:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc46ba1e740 10 client.28235 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc46ba1e740 1 -- 172.21.15.94:0/513740034 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561042d896a0 con 0x561042e612b0 2024-02-17T12:10:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc449ffb640 1 -- 172.21.15.94:0/513740034 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc458011420 con 0x561042e612b0 2024-02-17T12:10:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc449ffb640 1 -- 172.21.15.94:0/513740034 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc454010510 con 0x561042e612b0 2024-02-17T12:10:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc46ba1e740 10 client.28235 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc46ba1e740 10 client.28235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:34.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc46ba1e740 20 client.28235 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561042e7a8e0 0x561042ed4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc46ba1e740 4 client.28235 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:10:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc46ba1e740 1 -- 172.21.15.94:0/513740034 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561042b54890 con 0x561042e7a8e0 2024-02-17T12:10:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.108+0000 7fc461d9b640 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561042e7a8e0 0x561042ed4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc449ffb640 10 client.28235 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc461d9b640 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561042e7a8e0 0x561042ed4fa0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26297b20c1ffe356 server_cookie=eb846eadd8eecc04 in_seq=0 out_seq=0 2024-02-17T12:10:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc449ffb640 1 -- 172.21.15.94:0/513740034 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x561042b54890 con 0x561042e7a8e0 2024-02-17T12:10:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc449ffb640 10 client.28235 handle_command_reply: tid=0 2024-02-17T12:10:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc46ba1e740 1 client.28235 shutdown 2024-02-17T12:10:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc46ba1e740 1 -- 172.21.15.94:0/513740034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 msgr2=0x561042e709d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:34.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e709d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc454006010 tx=0x7fc45401a270 comp rx=0 tx=0).stop 2024-02-17T12:10:34.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc46ba1e740 1 -- 172.21.15.94:0/513740034 shutdown_connections 2024-02-17T12:10:34.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561042e7a8e0 0x561042ed4fa0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561042e6a7b0 0x561042ecae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc449ffb640 0 client.28235 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:34.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561042e61bd0 0x561042e70f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc46ba1e740 1 --2- 172.21.15.94:0/513740034 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561042e612b0 0x561042e709d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc46ba1e740 1 -- 172.21.15.94:0/513740034 >> 172.21.15.94:0/513740034 conn(0x561042e5b990 msgr2=0x561042e5a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:34.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc46ba1e740 1 -- 172.21.15.94:0/513740034 shutdown_connections 2024-02-17T12:10:34.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc46ba1e740 1 -- 172.21.15.94:0/513740034 wait complete. 2024-02-17T12:10:34.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.109+0000 7fc46ba1e740 20 client.28235 trim_cache size 0 max 16384 2024-02-17T12:10:34.117 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:10:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (30 inodes in the stack)", 2024-02-17T12:10:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:10:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "c27ac994-2794-44cb-9c7e-86c6c8d4b286": { 2024-02-17T12:10:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:10:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "c27ac994-2794-44cb-9c7e-86c6c8d4b286", 2024-02-17T12:10:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:10:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:10:34.118 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:10:34.118 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:10:34.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.110+0000 7fc468e4e640 1 -- 172.21.15.94:0/1099478977 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc44c0a2370 msgr2=0x7fc44c0a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:34.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.110+0000 7fc468e4e640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc44c0a2370 0x7fc44c0a4830 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc454000c00 tx=0x7fc45403d1c0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.110+0000 7fc468e4e640 1 -- 172.21.15.94:0/1099478977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc46411a960 msgr2=0x7fc464116630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:34.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.110+0000 7fc468e4e640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc46411a960 0x7fc464116630 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00b9e0 tx=0x7fc45c00bea0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.110+0000 7fc468e4e640 1 -- 172.21.15.94:0/1099478977 shutdown_connections 2024-02-17T12:10:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.110+0000 7fc468e4e640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc44c0a2370 0x7fc44c0a4830 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.110+0000 7fc468e4e640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc46411a960 0x7fc464116630 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.110+0000 7fc468e4e640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc46410fd10 0x7fc4641160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.110+0000 7fc468e4e640 1 --2- 172.21.15.94:0/1099478977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc46410f8a0 0x7fc46411a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.110+0000 7fc468e4e640 1 -- 172.21.15.94:0/1099478977 >> 172.21.15.94:0/1099478977 conn(0x7fc46406c270 msgr2=0x7fc46411b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.110+0000 7fc468e4e640 1 -- 172.21.15.94:0/1099478977 shutdown_connections 2024-02-17T12:10:34.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:10:34.110+0000 7fc468e4e640 1 -- 172.21.15.94:0/1099478977 wait complete. 2024-02-17T12:10:34.127 INFO:tasks.cephfs.filesystem:scrub status for tag:c27ac994-2794-44cb-9c7e-86c6c8d4b286 - {'path': '/', 'tag': 'c27ac994-2794-44cb-9c7e-86c6c8d4b286', 'options': 'recursive,force'} 2024-02-17T12:10:34.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:34 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:10:34.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:34 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:10:34.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:34 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:10:34.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:34 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:10:34.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:34 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:10:34.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:34 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:10:34.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:34 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:10:34.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:34 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:10:34.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:34 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:10:34.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:34 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:10:34.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:34 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:10:34.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:34 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:10:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.006+0000 7f0642664640 1 -- 172.21.15.94:0/446237689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063c100c90 msgr2=0x7f063c104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:35.008 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.006+0000 7f0642664640 1 --2- 172.21.15.94:0/446237689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063c100c90 0x7f063c104cd0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f0630009930 tx=0x7f063002bc30 comp rx=0 tx=0).stop 2024-02-17T12:10:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.007+0000 7f0642664640 1 -- 172.21.15.94:0/446237689 shutdown_connections 2024-02-17T12:10:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.007+0000 7f0642664640 1 --2- 172.21.15.94:0/446237689 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f063c1053b0 0x7f063c10cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.007+0000 7f0642664640 1 --2- 172.21.15.94:0/446237689 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063c100c90 0x7f063c104cd0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.007+0000 7f0642664640 1 --2- 172.21.15.94:0/446237689 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f063c1002e0 0x7f063c1006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.007+0000 7f0642664640 1 -- 172.21.15.94:0/446237689 >> 172.21.15.94:0/446237689 conn(0x7f063c077290 msgr2=0x7f063c0776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.007+0000 7f0642664640 1 -- 172.21.15.94:0/446237689 shutdown_connections 2024-02-17T12:10:35.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.008+0000 7f0642664640 1 -- 172.21.15.94:0/446237689 wait complete. 2024-02-17T12:10:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.009+0000 7f0642664640 1 Processor -- start 2024-02-17T12:10:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.009+0000 7f0642664640 1 -- start start 2024-02-17T12:10:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.009+0000 7f0642664640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f063c1002e0 0x7f063c106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.009+0000 7f0642664640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063c1053b0 0x7f063c1074e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.009+0000 7f0642664640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f063c107a20 0x7f063c1a9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.009+0000 7f0642664640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c10b180 con 0x7f063c1053b0 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.009+0000 7f0642664640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c1aa4a0 con 0x7f063c107a20 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.009+0000 7f0642664640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063c1aa660 con 0x7f063c1002e0 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.009+0000 7f063b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063c1053b0 0x7f063c1074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.009+0000 7f063b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063c1053b0 0x7f063c1074e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47832/0 (socket says 172.21.15.94:47832) 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.009+0000 7f063b7fe640 1 -- 172.21.15.94:0/2087277095 learned_addr learned my addr 172.21.15.94:0/2087277095 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.009+0000 7f0640c01640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f063c107a20 0x7f063c1a9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.009+0000 7f063bfff640 1 --2- 172.21.15.94:0/2087277095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f063c1002e0 0x7f063c106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.010+0000 7f063b7fe640 1 -- 172.21.15.94:0/2087277095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f063c1002e0 msgr2=0x7f063c106fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.010+0000 7f063b7fe640 1 --2- 172.21.15.94:0/2087277095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f063c1002e0 0x7f063c106fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.010+0000 7f063b7fe640 1 -- 172.21.15.94:0/2087277095 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f063c107a20 msgr2=0x7f063c1a9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.010+0000 7f063b7fe640 1 --2- 172.21.15.94:0/2087277095 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f063c107a20 0x7f063c1a9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.010+0000 7f063b7fe640 1 -- 172.21.15.94:0/2087277095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0630009560 con 0x7f063c1053b0 2024-02-17T12:10:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.010+0000 7f063b7fe640 1 --2- 172.21.15.94:0/2087277095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063c1053b0 0x7f063c1074e0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0630009230 tx=0x7f0630004c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.010+0000 7f06397fa640 1 -- 172.21.15.94:0/2087277095 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06300043d0 con 0x7f063c1053b0 2024-02-17T12:10:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.010+0000 7f0642664640 1 -- 172.21.15.94:0/2087277095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063c1aa8e0 con 0x7f063c1053b0 2024-02-17T12:10:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.010+0000 7f06397fa640 1 -- 172.21.15.94:0/2087277095 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06300052d0 con 0x7f063c1053b0 2024-02-17T12:10:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.010+0000 7f06397fa640 1 -- 172.21.15.94:0/2087277095 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0630003d80 con 0x7f063c1053b0 2024-02-17T12:10:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.010+0000 7f0642664640 1 -- 172.21.15.94:0/2087277095 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063c1aae40 con 0x7f063c1053b0 2024-02-17T12:10:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.012+0000 7f06397fa640 1 -- 172.21.15.94:0/2087277095 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0630005440 con 0x7f063c1053b0 2024-02-17T12:10:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.013+0000 7f0645237740 1 Processor -- start 2024-02-17T12:10:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.013+0000 7f06397fa640 1 --2- 172.21.15.94:0/2087277095 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f06240a2350 0x7f06240a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.013+0000 7f0645237740 1 -- start start 2024-02-17T12:10:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.013+0000 7f0645237740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a38623ad0 0x555a38622820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.013+0000 7f0645237740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a38622d60 0x555a386231e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.013+0000 7f0640c01640 1 --2- 172.21.15.94:0/2087277095 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f06240a2350 0x7f06240a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.013+0000 7f0645237740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a3862c2e0 0x555a3862e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.013+0000 7f0645237740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a385919f0 con 0x555a38622d60 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.013+0000 7f0645237740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3858ef20 con 0x555a3862c2e0 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.013+0000 7f0645237740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3858a700 con 0x555a38623ad0 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f06397fa640 1 -- 172.21.15.94:0/2087277095 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f06300f0160 con 0x7f063c1053b0 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f063bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a38623ad0 0x555a38622820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f0640c01640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a3862c2e0 0x555a3862e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f063bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a38623ad0 0x555a38622820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57118/0 (socket says 172.21.15.94:57118) 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f063b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a38622d60 0x555a386231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f063bfff640 1 -- 172.21.15.94:0/3049090675 learned_addr learned my addr 172.21.15.94:0/3049090675 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f063bfff640 1 -- 172.21.15.94:0/3049090675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a3862c2e0 msgr2=0x555a3862e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f063bfff640 1 --2- 172.21.15.94:0/3049090675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a3862c2e0 0x555a3862e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.017 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f063bfff640 1 -- 172.21.15.94:0/3049090675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a38622d60 msgr2=0x555a386231e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f063bfff640 1 --2- 172.21.15.94:0/3049090675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a38622d60 0x555a386231e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f063bfff640 1 -- 172.21.15.94:0/3049090675 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a385207b0 con 0x555a38623ad0 2024-02-17T12:10:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f0640c01640 1 --2- 172.21.15.94:0/2087277095 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f06240a2350 0x7f06240a4810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0634005fd0 tx=0x7f0634014b50 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f063bfff640 1 --2- 172.21.15.94:0/3049090675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a38623ad0 0x555a38622820 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f062c009d20 tx=0x7f062c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e12603fb8910369 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f06227fc640 1 -- 172.21.15.94:0/3049090675 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c009e10 con 0x555a38623ad0 2024-02-17T12:10:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f06227fc640 1 -- 172.21.15.94:0/3049090675 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f062c0198f0 con 0x555a38623ad0 2024-02-17T12:10:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.014+0000 7f06227fc640 1 -- 172.21.15.94:0/3049090675 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c018990 con 0x555a38623ad0 2024-02-17T12:10:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.015+0000 7f0645237740 1 -- 172.21.15.94:0/3049090675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a38623ad0 msgr2=0x555a38622820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.015+0000 7f0645237740 1 --2- 172.21.15.94:0/3049090675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a38623ad0 0x555a38622820 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f062c009d20 tx=0x7f062c011810 comp rx=0 tx=0).stop 2024-02-17T12:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.015+0000 7f0645237740 1 -- 172.21.15.94:0/3049090675 shutdown_connections 2024-02-17T12:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.015+0000 7f0645237740 1 --2- 172.21.15.94:0/3049090675 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a3862c2e0 0x555a3862e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.015+0000 7f0645237740 1 --2- 172.21.15.94:0/3049090675 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a38622d60 0x555a386231e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.015+0000 7f0645237740 1 --2- 172.21.15.94:0/3049090675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a38623ad0 0x555a38622820 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.015+0000 7f0645237740 1 -- 172.21.15.94:0/3049090675 >> 172.21.15.94:0/3049090675 conn(0x555a385ea550 msgr2=0x555a3861da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.015+0000 7f0645237740 1 -- 172.21.15.94:0/3049090675 shutdown_connections 2024-02-17T12:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.015+0000 7f0645237740 1 -- 172.21.15.94:0/3049090675 wait complete. 2024-02-17T12:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0645237740 1 -- start start 2024-02-17T12:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0645237740 1 Processor -- start 2024-02-17T12:10:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0645237740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a38622d60 0x555a386292d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0645237740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38629810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0645237740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a3862c2e0 0x555a38629d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0645237740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3852fbf0 con 0x555a3862c2e0 2024-02-17T12:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0645237740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3825e250 con 0x555a38623ad0 2024-02-17T12:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0645237740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a38510200 con 0x555a38622d60 2024-02-17T12:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0645237740 10 client.0 _ll_register_callbacks cb 0x555a3857ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0640c01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a3862c2e0 0x555a38629d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f063bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a38622d60 0x555a386292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0640c01640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a3862c2e0 0x555a38629d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47844/0 (socket says 172.21.15.94:47844) 2024-02-17T12:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0640c01640 1 -- 172.21.15.94:0/1183621084 learned_addr learned my addr 172.21.15.94:0/1183621084 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f063b7fe640 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38629810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.023 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0640c01640 1 -- 172.21.15.94:0/1183621084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a38622d60 msgr2=0x555a386292d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0640c01640 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a38622d60 0x555a386292d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0640c01640 1 -- 172.21.15.94:0/1183621084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 msgr2=0x555a38629810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0640c01640 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38629810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f0640c01640 1 -- 172.21.15.94:0/1183621084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c009950 con 0x555a3862c2e0 2024-02-17T12:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f063bfff640 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a38622d60 0x555a386292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.016+0000 7f063b7fe640 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38629810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.017+0000 7f0640c01640 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a3862c2e0 0x555a38629d50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0634009a80 tx=0x7f0634004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd8a2c7bbefbfb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.017+0000 7f061b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.017+0000 7f061b7fe640 1 -- 172.21.15.94:0/1183621084 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063401fea0 con 0x555a3862c2e0 2024-02-17T12:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.017+0000 7f061b7fe640 1 -- 172.21.15.94:0/1183621084 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0634026070 con 0x555a3862c2e0 2024-02-17T12:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.017+0000 7f0645237740 1 -- 172.21.15.94:0/1183621084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a3828a600 con 0x555a3862c2e0 2024-02-17T12:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.017+0000 7f061b7fe640 1 -- 172.21.15.94:0/1183621084 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0634003b70 con 0x555a3862c2e0 2024-02-17T12:10:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.017+0000 7f0645237740 10 client.18780 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.017+0000 7f0645237740 1 -- 172.21.15.94:0/1183621084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555a38588d80 con 0x555a3862c2e0 2024-02-17T12:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.017+0000 7f061b7fe640 1 -- 172.21.15.94:0/1183621084 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555a3828a600 con 0x555a3862c2e0 2024-02-17T12:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.018+0000 7f061b7fe640 1 -- 172.21.15.94:0/1183621084 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0634030070 con 0x555a3862c2e0 2024-02-17T12:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.018+0000 7f0645237740 10 client.18780 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.018+0000 7f0645237740 10 client.18780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.018+0000 7f0645237740 20 client.18780 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.018+0000 7f0645237740 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a3863c6b0 0x555a3863ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.018+0000 7f0645237740 4 client.18780 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.018+0000 7f0645237740 1 -- 172.21.15.94:0/1183621084 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a382db1b0 con 0x555a3863c6b0 2024-02-17T12:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.018+0000 7f063bfff640 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a3863c6b0 0x555a3863ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.019+0000 7f063bfff640 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a3863c6b0 0x555a3863ead0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdeb9c0676f643f9 server_cookie=81e8388ae2118854 in_seq=0 out_seq=0 2024-02-17T12:10:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.019+0000 7f061b7fe640 10 client.18780 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.025+0000 7f061b7fe640 1 -- 172.21.15.94:0/1183621084 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a382db1b0 con 0x555a3863c6b0 2024-02-17T12:10:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.025+0000 7f061b7fe640 10 client.18780 handle_command_reply: tid=0 2024-02-17T12:10:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.025+0000 7f0645237740 1 client.18780 shutdown 2024-02-17T12:10:35.036 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.035+0000 7f0645237740 1 -- 172.21.15.94:0/1183621084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a3862c2e0 msgr2=0x555a38629d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:35.037 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.035+0000 7f0645237740 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a3862c2e0 0x555a38629d50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0634009a80 tx=0x7f0634004ab0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.043+0000 7f0645237740 1 -- 172.21.15.94:0/1183621084 shutdown_connections 2024-02-17T12:10:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.043+0000 7f0645237740 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a3863c6b0 0x555a3863ead0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.043+0000 7f0645237740 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a3862c2e0 0x555a38629d50 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.043+0000 7f0645237740 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38629810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.043+0000 7f0645237740 1 --2- 172.21.15.94:0/1183621084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a38622d60 0x555a386292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.045 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.043+0000 7f0645237740 1 -- 172.21.15.94:0/1183621084 >> 172.21.15.94:0/1183621084 conn(0x555a385ea550 msgr2=0x555a3861f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.044+0000 7f061b7fe640 0 client.18780 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:35.050 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.048+0000 7f0645237740 1 -- 172.21.15.94:0/1183621084 shutdown_connections 2024-02-17T12:10:35.056 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.052+0000 7f0645237740 1 -- 172.21.15.94:0/1183621084 wait complete. 2024-02-17T12:10:35.059 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.055+0000 7f0645237740 20 client.18780 trim_cache size 0 max 16384 2024-02-17T12:10:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.083+0000 7f0645237740 1 Processor -- start 2024-02-17T12:10:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.083+0000 7f0645237740 1 -- start start 2024-02-17T12:10:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.084+0000 7f0645237740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a38622d60 0x555a38629820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.084+0000 7f0645237740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38629d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.084+0000 7f0645237740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a3862c2e0 0x555a38627ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.084+0000 7f0645237740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a3861ea80 con 0x555a38622d60 2024-02-17T12:10:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.084+0000 7f0645237740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a38639150 con 0x555a38623ad0 2024-02-17T12:10:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.084+0000 7f0645237740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a386240c0 con 0x555a3862c2e0 2024-02-17T12:10:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.084+0000 7f0640c01640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a3862c2e0 0x555a38627ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.084+0000 7f0640c01640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a3862c2e0 0x555a38627ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57132/0 (socket says 172.21.15.94:57132) 2024-02-17T12:10:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.084+0000 7f0640c01640 1 -- 172.21.15.94:0/4196569556 learned_addr learned my addr 172.21.15.94:0/4196569556 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.085+0000 7f063bfff640 1 --2- 172.21.15.94:0/4196569556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a38622d60 0x555a38629820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.085+0000 7f063b7fe640 1 --2- 172.21.15.94:0/4196569556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38629d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.085+0000 7f063b7fe640 1 -- 172.21.15.94:0/4196569556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a3862c2e0 msgr2=0x555a38627ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.085+0000 7f063b7fe640 1 --2- 172.21.15.94:0/4196569556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a3862c2e0 0x555a38627ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.085+0000 7f063b7fe640 1 -- 172.21.15.94:0/4196569556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a38622d60 msgr2=0x555a38629820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.085+0000 7f063b7fe640 1 --2- 172.21.15.94:0/4196569556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a38622d60 0x555a38629820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.085+0000 7f063b7fe640 1 -- 172.21.15.94:0/4196569556 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a385257b0 con 0x555a38623ad0 2024-02-17T12:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.086+0000 7f063b7fe640 1 --2- 172.21.15.94:0/4196569556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38629d60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f06300ed920 tx=0x7f0630009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f7be05bbd9eb303 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.086+0000 7f0621ffb640 1 -- 172.21.15.94:0/4196569556 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06300bd2c0 con 0x555a38623ad0 2024-02-17T12:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.086+0000 7f0621ffb640 1 -- 172.21.15.94:0/4196569556 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06300bcbc0 con 0x555a38623ad0 2024-02-17T12:10:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.086+0000 7f0621ffb640 1 -- 172.21.15.94:0/4196569556 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06300c5130 con 0x555a38623ad0 2024-02-17T12:10:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.087+0000 7f0645237740 1 -- 172.21.15.94:0/4196569556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 msgr2=0x555a38629d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.087+0000 7f0645237740 1 --2- 172.21.15.94:0/4196569556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38629d60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f06300ed920 tx=0x7f0630009260 comp rx=0 tx=0).stop 2024-02-17T12:10:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.088+0000 7f0645237740 1 -- 172.21.15.94:0/4196569556 shutdown_connections 2024-02-17T12:10:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.088+0000 7f0645237740 1 --2- 172.21.15.94:0/4196569556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a3862c2e0 0x555a38627ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.088+0000 7f0645237740 1 --2- 172.21.15.94:0/4196569556 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38629d60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.088+0000 7f0645237740 1 --2- 172.21.15.94:0/4196569556 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a38622d60 0x555a38629820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.088+0000 7f0645237740 1 -- 172.21.15.94:0/4196569556 >> 172.21.15.94:0/4196569556 conn(0x555a385ea550 msgr2=0x555a38624580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.088+0000 7f0645237740 1 -- 172.21.15.94:0/4196569556 shutdown_connections 2024-02-17T12:10:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.088+0000 7f0645237740 1 -- 172.21.15.94:0/4196569556 wait complete. 2024-02-17T12:10:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.089+0000 7f0645237740 1 -- start start 2024-02-17T12:10:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.089+0000 7f0645237740 1 Processor -- start 2024-02-17T12:10:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.089+0000 7f0645237740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38630de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.089+0000 7f0645237740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a3862c2e0 0x555a38631320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.090+0000 7f0645237740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a386318b0 0x555a38691bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.090+0000 7f0645237740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a386921f0 con 0x555a3862c2e0 2024-02-17T12:10:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.090+0000 7f0645237740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a38692360 con 0x555a38623ad0 2024-02-17T12:10:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.090+0000 7f0645237740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a386a3530 con 0x555a386318b0 2024-02-17T12:10:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.090+0000 7f0645237740 10 client.0 _ll_register_callbacks cb 0x7f062c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.090+0000 7f0640c01640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a386318b0 0x555a38691bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.090+0000 7f0640c01640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a386318b0 0x555a38691bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57142/0 (socket says 172.21.15.94:57142) 2024-02-17T12:10:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.090+0000 7f0640c01640 1 -- 172.21.15.94:0/876138268 learned_addr learned my addr 172.21.15.94:0/876138268 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.090+0000 7f063b7fe640 1 --2- 172.21.15.94:0/876138268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a3862c2e0 0x555a38631320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.090+0000 7f063bfff640 1 --2- 172.21.15.94:0/876138268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38630de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.091+0000 7f063b7fe640 1 -- 172.21.15.94:0/876138268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a386318b0 msgr2=0x555a38691bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.091+0000 7f063b7fe640 1 --2- 172.21.15.94:0/876138268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a386318b0 0x555a38691bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.091+0000 7f063b7fe640 1 -- 172.21.15.94:0/876138268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 msgr2=0x555a38630de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.091+0000 7f063b7fe640 1 --2- 172.21.15.94:0/876138268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38630de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.091+0000 7f063b7fe640 1 -- 172.21.15.94:0/876138268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063c1aae40 con 0x555a3862c2e0 2024-02-17T12:10:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.091+0000 7f063b7fe640 1 --2- 172.21.15.94:0/876138268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a3862c2e0 0x555a38631320 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f06300ed950 tx=0x7f06300bb0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b415b493a9b0730c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.091+0000 7f061b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:10:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.092+0000 7f061b7fe640 1 -- 172.21.15.94:0/876138268 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063009f070 con 0x555a3862c2e0 2024-02-17T12:10:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.092+0000 7f0645237740 1 -- 172.21.15.94:0/876138268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f062c0198f0 con 0x555a3862c2e0 2024-02-17T12:10:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.092+0000 7f061b7fe640 1 -- 172.21.15.94:0/876138268 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f06300c5360 con 0x555a3862c2e0 2024-02-17T12:10:35.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.099+0000 7f061b7fe640 1 -- 172.21.15.94:0/876138268 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06300c0d10 con 0x555a3862c2e0 2024-02-17T12:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.100+0000 7f0645237740 10 client.18786 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.100+0000 7f0645237740 1 -- 172.21.15.94:0/876138268 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555a3854b540 con 0x555a3862c2e0 2024-02-17T12:10:35.102 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.101+0000 7f061b7fe640 1 -- 172.21.15.94:0/876138268 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f062c0198f0 con 0x555a3862c2e0 2024-02-17T12:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.101+0000 7f061b7fe640 1 -- 172.21.15.94:0/876138268 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f06300a2030 con 0x555a3862c2e0 2024-02-17T12:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.101+0000 7f0645237740 10 client.18786 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.101+0000 7f0645237740 10 client.18786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.101+0000 7f0645237740 20 client.18786 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.101+0000 7f0645237740 1 --2- 172.21.15.94:0/876138268 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a386a4810 0x555a386a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.101+0000 7f0645237740 4 client.18786 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:10:35.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.101+0000 7f0645237740 1 -- 172.21.15.94:0/876138268 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555a382c7550 con 0x555a386a4810 2024-02-17T12:10:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.101+0000 7f063bfff640 1 --2- 172.21.15.94:0/876138268 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a386a4810 0x555a386a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.102+0000 7f063bfff640 1 --2- 172.21.15.94:0/876138268 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a386a4810 0x555a386a6c30 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7548918e09773c0 server_cookie=b381ec30df6e1e7c in_seq=0 out_seq=0 2024-02-17T12:10:35.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.102+0000 7f061b7fe640 10 client.18786 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f061b7fe640 1 -- 172.21.15.94:0/876138268 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555a382c7550 con 0x555a386a4810 2024-02-17T12:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f061b7fe640 10 client.18786 handle_command_reply: tid=0 2024-02-17T12:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f0645237740 1 client.18786 shutdown 2024-02-17T12:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f0645237740 1 -- 172.21.15.94:0/876138268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a3862c2e0 msgr2=0x555a38631320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f0645237740 1 --2- 172.21.15.94:0/876138268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a3862c2e0 0x555a38631320 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f06300ed950 tx=0x7f06300bb0d0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f0645237740 1 -- 172.21.15.94:0/876138268 shutdown_connections 2024-02-17T12:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f0645237740 1 --2- 172.21.15.94:0/876138268 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x555a386a4810 0x555a386a6c30 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f0645237740 1 --2- 172.21.15.94:0/876138268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555a386318b0 0x555a38691bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f0645237740 1 --2- 172.21.15.94:0/876138268 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555a3862c2e0 0x555a38631320 secure :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f06300ed950 tx=0x7f06300bb0d0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f0645237740 1 --2- 172.21.15.94:0/876138268 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555a38623ad0 0x555a38630de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f0645237740 1 -- 172.21.15.94:0/876138268 >> 172.21.15.94:0/876138268 conn(0x555a385ea550 msgr2=0x555a3861c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f061b7fe640 0 client.18786 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f0645237740 1 -- 172.21.15.94:0/876138268 shutdown_connections 2024-02-17T12:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f0645237740 1 -- 172.21.15.94:0/876138268 wait complete. 2024-02-17T12:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.144+0000 7f0645237740 20 client.18786 trim_cache size 0 max 16384 2024-02-17T12:10:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:10:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:10:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:10:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:10:35.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.145+0000 7f0642664640 1 -- 172.21.15.94:0/2087277095 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f06240a2350 msgr2=0x7f06240a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.145+0000 7f0642664640 1 --2- 172.21.15.94:0/2087277095 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f06240a2350 0x7f06240a4810 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0634005fd0 tx=0x7f0634014b50 comp rx=0 tx=0).stop 2024-02-17T12:10:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.145+0000 7f0642664640 1 -- 172.21.15.94:0/2087277095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063c1053b0 msgr2=0x7f063c1074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.145+0000 7f0642664640 1 --2- 172.21.15.94:0/2087277095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063c1053b0 0x7f063c1074e0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0630009230 tx=0x7f0630004c00 comp rx=0 tx=0).stop 2024-02-17T12:10:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.145+0000 7f0642664640 1 -- 172.21.15.94:0/2087277095 shutdown_connections 2024-02-17T12:10:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.145+0000 7f0642664640 1 --2- 172.21.15.94:0/2087277095 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f06240a2350 0x7f06240a4810 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.145+0000 7f0642664640 1 --2- 172.21.15.94:0/2087277095 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f063c107a20 0x7f063c1a9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.145+0000 7f0642664640 1 --2- 172.21.15.94:0/2087277095 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f063c1053b0 0x7f063c1074e0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.145+0000 7f0642664640 1 --2- 172.21.15.94:0/2087277095 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f063c1002e0 0x7f063c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.145+0000 7f0642664640 1 -- 172.21.15.94:0/2087277095 >> 172.21.15.94:0/2087277095 conn(0x7f063c077290 msgr2=0x7f063c1066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.145+0000 7f0642664640 1 -- 172.21.15.94:0/2087277095 shutdown_connections 2024-02-17T12:10:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:35.145+0000 7f0642664640 1 -- 172.21.15.94:0/2087277095 wait complete. 2024-02-17T12:10:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 735.0 s, 1324.2 tps, lat 75.843 ms stddev 81.035 2024-02-17T12:10:35.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:35 smithi189 ceph-mon[36588]: pgmap v389: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.5 MiB/s wr, 806 op/s 2024-02-17T12:10:35.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:35 smithi125 ceph-mon[36819]: pgmap v389: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.5 MiB/s wr, 806 op/s 2024-02-17T12:10:35.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:35 smithi094 ceph-mon[32499]: pgmap v389: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.5 MiB/s wr, 806 op/s 2024-02-17T12:10:37.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:37 smithi189 ceph-mon[36588]: pgmap v390: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.1 MiB/s wr, 542 op/s 2024-02-17T12:10:37.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:37 smithi125 ceph-mon[36819]: pgmap v390: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.1 MiB/s wr, 542 op/s 2024-02-17T12:10:37.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:37 smithi094 ceph-mon[32499]: pgmap v390: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 5.1 MiB/s wr, 542 op/s 2024-02-17T12:10:38.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:38 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:10:38.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:38 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:10:38.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:38 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:10:39.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:39 smithi189 ceph-mon[36588]: pgmap v391: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 6.9 MiB/s wr, 817 op/s 2024-02-17T12:10:39.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:39 smithi125 ceph-mon[36819]: pgmap v391: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 6.9 MiB/s wr, 817 op/s 2024-02-17T12:10:39.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:39 smithi094 ceph-mon[32499]: pgmap v391: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 6.9 MiB/s wr, 817 op/s 2024-02-17T12:10:40.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 740.0 s, 1519.2 tps, lat 64.842 ms stddev 68.577 2024-02-17T12:10:41.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:41 smithi189 ceph-mon[36588]: pgmap v392: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.2 MiB/s wr, 723 op/s 2024-02-17T12:10:41.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:41 smithi125 ceph-mon[36819]: pgmap v392: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.2 MiB/s wr, 723 op/s 2024-02-17T12:10:41.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:41 smithi094 ceph-mon[32499]: pgmap v392: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 6.2 MiB/s wr, 723 op/s 2024-02-17T12:10:43.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:43 smithi189 ceph-mon[36588]: pgmap v393: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.6 MiB/s wr, 670 op/s 2024-02-17T12:10:43.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:43 smithi125 ceph-mon[36819]: pgmap v393: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.6 MiB/s wr, 670 op/s 2024-02-17T12:10:43.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:43 smithi094 ceph-mon[32499]: pgmap v393: 97 pgs: 97 active+clean; 5.4 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.6 MiB/s wr, 670 op/s 2024-02-17T12:10:45.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 745.0 s, 967.2 tps, lat 104.886 ms stddev 202.273 2024-02-17T12:10:45.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:45 smithi189 ceph-mon[36588]: pgmap v394: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 22 MiB/s wr, 1.96k op/s 2024-02-17T12:10:45.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:45 smithi125 ceph-mon[36819]: pgmap v394: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 22 MiB/s wr, 1.96k op/s 2024-02-17T12:10:45.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:45 smithi094 ceph-mon[32499]: pgmap v394: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 22 MiB/s wr, 1.96k op/s 2024-02-17T12:10:47.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:47 smithi189 ceph-mon[36588]: pgmap v395: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 20 MiB/s wr, 1.66k op/s 2024-02-17T12:10:47.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:47 smithi125 ceph-mon[36819]: pgmap v395: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 20 MiB/s wr, 1.66k op/s 2024-02-17T12:10:47.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:47 smithi094 ceph-mon[32499]: pgmap v395: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 20 MiB/s wr, 1.66k op/s 2024-02-17T12:10:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.255+0000 7fc05ca62640 1 -- 172.21.15.94:0/395572383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0581002a0 msgr2=0x7fc058105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:48.257 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.255+0000 7fc05ca62640 1 --2- 172.21.15.94:0/395572383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0581002a0 0x7fc058105dd0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009900 tx=0x7fc04c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:10:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.256+0000 7fc05ca62640 1 -- 172.21.15.94:0/395572383 shutdown_connections 2024-02-17T12:10:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.256+0000 7fc05ca62640 1 --2- 172.21.15.94:0/395572383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc058106500 0x7fc05810e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.256+0000 7fc05ca62640 1 --2- 172.21.15.94:0/395572383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc0581002a0 0x7fc058105dd0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.256+0000 7fc05ca62640 1 --2- 172.21.15.94:0/395572383 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0580ff8f0 0x7fc0580ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.258 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.256+0000 7fc05ca62640 1 -- 172.21.15.94:0/395572383 >> 172.21.15.94:0/395572383 conn(0x7fc058076610 msgr2=0x7fc0580fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.258+0000 7fc05ca62640 1 -- 172.21.15.94:0/395572383 shutdown_connections 2024-02-17T12:10:48.259 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.258+0000 7fc05ca62640 1 -- 172.21.15.94:0/395572383 wait complete. 2024-02-17T12:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc05ca62640 1 Processor -- start 2024-02-17T12:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc05ca62640 1 -- start start 2024-02-17T12:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc05ca62640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0580ff8f0 0x7fc058100860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc05ca62640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc058106500 0x7fc058100da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc05ca62640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0581012e0 0x7fc0581a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc05ca62640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0581a5ef0 con 0x7fc058106500 2024-02-17T12:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc05ca62640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0581a6060 con 0x7fc0581012e0 2024-02-17T12:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc05ca62640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0581a6220 con 0x7fc0580ff8f0 2024-02-17T12:10:48.262 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc055d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc058106500 0x7fc058100da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc055d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc058106500 0x7fc058100da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53618/0 (socket says 172.21.15.94:53618) 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc055d9b640 1 -- 172.21.15.94:0/327634629 learned_addr learned my addr 172.21.15.94:0/327634629 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc05659c640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0580ff8f0 0x7fc058100860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc056d9d640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0581012e0 0x7fc0581a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc055d9b640 1 -- 172.21.15.94:0/327634629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0580ff8f0 msgr2=0x7fc058100860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc055d9b640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0580ff8f0 0x7fc058100860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc055d9b640 1 -- 172.21.15.94:0/327634629 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0581012e0 msgr2=0x7fc0581a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc055d9b640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0581012e0 0x7fc0581a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.259+0000 7fc055d9b640 1 -- 172.21.15.94:0/327634629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c009560 con 0x7fc058106500 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.260+0000 7fc056d9d640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0581012e0 0x7fc0581a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.260+0000 7fc055d9b640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc058106500 0x7fc058100da0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc04c036040 tx=0x7fc04c02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.260+0000 7fc0477fe640 1 -- 172.21.15.94:0/327634629 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c004350 con 0x7fc058106500 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.260+0000 7fc05ca62640 1 -- 172.21.15.94:0/327634629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0581a64a0 con 0x7fc058106500 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.260+0000 7fc05ca62640 1 -- 172.21.15.94:0/327634629 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0581a6a00 con 0x7fc058106500 2024-02-17T12:10:48.263 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.260+0000 7fc0477fe640 1 -- 172.21.15.94:0/327634629 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc04c0044b0 con 0x7fc058106500 2024-02-17T12:10:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.260+0000 7fc0477fe640 1 -- 172.21.15.94:0/327634629 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c003da0 con 0x7fc058106500 2024-02-17T12:10:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.262+0000 7fc0477fe640 1 -- 172.21.15.94:0/327634629 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc04c004df0 con 0x7fc058106500 2024-02-17T12:10:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.263+0000 7fc05f631740 1 Processor -- start 2024-02-17T12:10:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.269+0000 7fc0477fe640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc0400a2300 0x7fc0400a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.269+0000 7fc0477fe640 1 -- 172.21.15.94:0/327634629 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc04c0f0030 con 0x7fc058106500 2024-02-17T12:10:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.270+0000 7fc05f631740 1 -- start start 2024-02-17T12:10:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.270+0000 7fc05f631740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531bd2e0 0x55b5531bc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.270+0000 7fc05f631740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531bc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.270+0000 7fc05f631740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531c5af0 0x55b5531c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.270+0000 7fc05f631740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5531229f0 con 0x55b5531bd2e0 2024-02-17T12:10:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.270+0000 7fc05f631740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55311ff20 con 0x55b5531c5af0 2024-02-17T12:10:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.270+0000 7fc05f631740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55311b700 con 0x55b5531bc570 2024-02-17T12:10:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.273+0000 7fc056d9d640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc0400a2300 0x7fc0400a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.275+0000 7fc055d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531bc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.275+0000 7fc055d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531bc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57294/0 (socket says 172.21.15.94:57294) 2024-02-17T12:10:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.275+0000 7fc055d9b640 1 -- 172.21.15.94:0/1473819941 learned_addr learned my addr 172.21.15.94:0/1473819941 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:48.278 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.277+0000 7fc056d9d640 1 --2- 172.21.15.94:0/1473819941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531c5af0 0x55b5531c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.280+0000 7fc055d9b640 1 -- 172.21.15.94:0/1473819941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531c5af0 msgr2=0x55b5531c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.280+0000 7fc055d9b640 1 --2- 172.21.15.94:0/1473819941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531c5af0 0x55b5531c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.280+0000 7fc055d9b640 1 -- 172.21.15.94:0/1473819941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531bd2e0 msgr2=0x55b5531bc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.280+0000 7fc055d9b640 1 --2- 172.21.15.94:0/1473819941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531bd2e0 0x55b5531bc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.280+0000 7fc055d9b640 1 -- 172.21.15.94:0/1473819941 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5530b17b0 con 0x55b5531bc570 2024-02-17T12:10:48.286 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.285+0000 7fc056d9d640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc0400a2300 0x7fc0400a47c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc050005fd0 tx=0x7fc0500135c0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:48.290 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.288+0000 7fc055d9b640 1 --2- 172.21.15.94:0/1473819941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531bc9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0b9550 tx=0x7fc04c0bc4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c3c44b4c0964300 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.290+0000 7fc044ff9640 1 -- 172.21.15.94:0/1473819941 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c0c0cd0 con 0x55b5531bc570 2024-02-17T12:10:48.292 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.290+0000 7fc044ff9640 1 -- 172.21.15.94:0/1473819941 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc04c0ede90 con 0x55b5531bc570 2024-02-17T12:10:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.291+0000 7fc05f631740 1 -- 172.21.15.94:0/1473819941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 msgr2=0x55b5531bc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:48.293 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.291+0000 7fc05f631740 1 --2- 172.21.15.94:0/1473819941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531bc9f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0b9550 tx=0x7fc04c0bc4e0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.292+0000 7fc05f631740 1 -- 172.21.15.94:0/1473819941 shutdown_connections 2024-02-17T12:10:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.292+0000 7fc05f631740 1 --2- 172.21.15.94:0/1473819941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531c5af0 0x55b5531c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.292+0000 7fc05f631740 1 --2- 172.21.15.94:0/1473819941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531bc9f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.292+0000 7fc05f631740 1 --2- 172.21.15.94:0/1473819941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531bd2e0 0x55b5531bc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.296 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.292+0000 7fc05f631740 1 -- 172.21.15.94:0/1473819941 >> 172.21.15.94:0/1473819941 conn(0x55b55317b550 msgr2=0x55b5531b7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:48.302 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.300+0000 7fc05f631740 1 -- 172.21.15.94:0/1473819941 shutdown_connections 2024-02-17T12:10:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.304+0000 7fc05f631740 1 -- 172.21.15.94:0/1473819941 wait complete. 2024-02-17T12:10:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.304+0000 7fc05f631740 1 -- start start 2024-02-17T12:10:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.304+0000 7fc05f631740 1 Processor -- start 2024-02-17T12:10:48.310 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.308+0000 7fc05f631740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531c2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.308+0000 7fc05f631740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531bd2e0 0x55b5531c3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.308+0000 7fc05f631740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531c5af0 0x55b5531c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.308+0000 7fc05f631740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5530c0bf0 con 0x55b5531c5af0 2024-02-17T12:10:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.308+0000 7fc05f631740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b552def250 con 0x55b5531bd2e0 2024-02-17T12:10:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.308+0000 7fc05f631740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5530a1200 con 0x55b5531bc570 2024-02-17T12:10:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.308+0000 7fc05f631740 10 client.0 _ll_register_callbacks cb 0x55b55310fd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.309+0000 7fc056d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531c5af0 0x55b5531c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.309+0000 7fc056d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531c5af0 0x55b5531c36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:53644/0 (socket says 172.21.15.94:53644) 2024-02-17T12:10:48.311 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.309+0000 7fc056d9d640 1 -- 172.21.15.94:0/2500292188 learned_addr learned my addr 172.21.15.94:0/2500292188 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:48.312 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.311+0000 7fc05659c640 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531c2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.312+0000 7fc056d9d640 1 -- 172.21.15.94:0/2500292188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 msgr2=0x55b5531c2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.312+0000 7fc056d9d640 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531c2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.312+0000 7fc055d9b640 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531bd2e0 0x55b5531c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.312+0000 7fc056d9d640 1 -- 172.21.15.94:0/2500292188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531bd2e0 msgr2=0x55b5531c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.312+0000 7fc056d9d640 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531bd2e0 0x55b5531c3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.312+0000 7fc056d9d640 1 -- 172.21.15.94:0/2500292188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0581a6a00 con 0x55b5531c5af0 2024-02-17T12:10:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.312+0000 7fc055d9b640 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531bd2e0 0x55b5531c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:10:48.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.313+0000 7fc056d9d640 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531c5af0 0x55b5531c36c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc050003bd0 tx=0x7fc05001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40517e484b69194 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.314+0000 7fc03dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:10:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.314+0000 7fc03dffb640 1 -- 172.21.15.94:0/2500292188 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050013170 con 0x55b5531c5af0 2024-02-17T12:10:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.314+0000 7fc05f631740 1 -- 172.21.15.94:0/2500292188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b552e6c1b0 con 0x55b5531c5af0 2024-02-17T12:10:48.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.314+0000 7fc03dffb640 1 -- 172.21.15.94:0/2500292188 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc050025030 con 0x55b5531c5af0 2024-02-17T12:10:48.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.315+0000 7fc03dffb640 1 -- 172.21.15.94:0/2500292188 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0500132d0 con 0x55b5531c5af0 2024-02-17T12:10:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.319+0000 7fc05f631740 10 client.18795 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:48.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.319+0000 7fc05f631740 1 -- 172.21.15.94:0/2500292188 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b553119d80 con 0x55b5531c5af0 2024-02-17T12:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.321+0000 7fc03dffb640 1 -- 172.21.15.94:0/2500292188 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b552e6c1b0 con 0x55b5531c5af0 2024-02-17T12:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.321+0000 7fc03dffb640 1 -- 172.21.15.94:0/2500292188 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc0500215b0 con 0x55b5531c5af0 2024-02-17T12:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.321+0000 7fc05f631740 10 client.18795 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.321+0000 7fc05f631740 10 client.18795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.321+0000 7fc05f631740 20 client.18795 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.321+0000 7fc05f631740 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b5531ef6a0 0x55b5531f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.321+0000 7fc05f631740 4 client.18795 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:10:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.321+0000 7fc05f631740 1 -- 172.21.15.94:0/2500292188 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b552e58550 con 0x55b5531ef6a0 2024-02-17T12:10:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.324+0000 7fc05659c640 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b5531ef6a0 0x55b5531f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.333+0000 7fc05659c640 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b5531ef6a0 0x55b5531f1ac0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60e355415658936a server_cookie=355adc3aeaf69415 in_seq=0 out_seq=0 2024-02-17T12:10:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.333+0000 7fc03dffb640 10 client.18795 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.334+0000 7fc03dffb640 1 -- 172.21.15.94:0/2500292188 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b552e58550 con 0x55b5531ef6a0 2024-02-17T12:10:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.334+0000 7fc03dffb640 10 client.18795 handle_command_reply: tid=0 2024-02-17T12:10:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.334+0000 7fc05f631740 1 client.18795 shutdown 2024-02-17T12:10:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.344+0000 7fc05f631740 1 -- 172.21.15.94:0/2500292188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531c5af0 msgr2=0x55b5531c36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:48.346 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.344+0000 7fc05f631740 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531c5af0 0x55b5531c36c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc050003bd0 tx=0x7fc05001d000 comp rx=0 tx=0).stop 2024-02-17T12:10:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.348+0000 7fc05f631740 1 -- 172.21.15.94:0/2500292188 shutdown_connections 2024-02-17T12:10:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.348+0000 7fc05f631740 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b5531ef6a0 0x55b5531f1ac0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.348+0000 7fc05f631740 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531c5af0 0x55b5531c36c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.348+0000 7fc05f631740 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531bd2e0 0x55b5531c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.348+0000 7fc05f631740 1 --2- 172.21.15.94:0/2500292188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531c2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.349+0000 7fc05f631740 1 -- 172.21.15.94:0/2500292188 >> 172.21.15.94:0/2500292188 conn(0x55b55317b550 msgr2=0x55b5531b8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.349+0000 7fc03dffb640 0 client.18795 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.349+0000 7fc05f631740 1 -- 172.21.15.94:0/2500292188 shutdown_connections 2024-02-17T12:10:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.351+0000 7fc05f631740 1 -- 172.21.15.94:0/2500292188 wait complete. 2024-02-17T12:10:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.351+0000 7fc05f631740 20 client.18795 trim_cache size 0 max 16384 2024-02-17T12:10:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.377+0000 7fc05f631740 1 Processor -- start 2024-02-17T12:10:48.379 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.377+0000 7fc05f631740 1 -- start start 2024-02-17T12:10:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.377+0000 7fc05f631740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.380 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.378+0000 7fc05f631740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531bd2e0 0x55b5531b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.378+0000 7fc05f631740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531c5af0 0x55b5531b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.378+0000 7fc055d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531bd2e0 0x55b5531b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.378+0000 7fc055d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531bd2e0 0x55b5531b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:48508/0 (socket says 172.21.15.94:48508) 2024-02-17T12:10:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.378+0000 7fc055d9b640 1 -- 172.21.15.94:0/441907981 learned_addr learned my addr 172.21.15.94:0/441907981 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.378+0000 7fc05f631740 1 -- 172.21.15.94:0/441907981 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5531be600 con 0x55b5531c5af0 2024-02-17T12:10:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.378+0000 7fc05f631740 1 -- 172.21.15.94:0/441907981 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b552e58550 con 0x55b5531bd2e0 2024-02-17T12:10:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.378+0000 7fc05f631740 1 -- 172.21.15.94:0/441907981 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b552e6c1b0 con 0x55b5531bc570 2024-02-17T12:10:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.378+0000 7fc056d9d640 1 --2- 172.21.15.94:0/441907981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531c5af0 0x55b5531b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.379+0000 7fc056d9d640 1 -- 172.21.15.94:0/441907981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 msgr2=0x55b5531c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.379+0000 7fc056d9d640 1 --2- 172.21.15.94:0/441907981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.379+0000 7fc056d9d640 1 -- 172.21.15.94:0/441907981 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531bd2e0 msgr2=0x55b5531b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.379+0000 7fc056d9d640 1 --2- 172.21.15.94:0/441907981 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531bd2e0 0x55b5531b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.381 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.379+0000 7fc056d9d640 1 -- 172.21.15.94:0/441907981 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5530b67b0 con 0x55b5531c5af0 2024-02-17T12:10:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.379+0000 7fc056d9d640 1 --2- 172.21.15.94:0/441907981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531c5af0 0x55b5531b2be0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc050000f80 tx=0x7fc050014fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c146011cb32264 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.379+0000 7fc03ffff640 1 -- 172.21.15.94:0/441907981 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050013cc0 con 0x55b5531c5af0 2024-02-17T12:10:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.379+0000 7fc03ffff640 1 -- 172.21.15.94:0/441907981 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc050005b70 con 0x55b5531c5af0 2024-02-17T12:10:48.382 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.379+0000 7fc03ffff640 1 -- 172.21.15.94:0/441907981 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050004c30 con 0x55b5531c5af0 2024-02-17T12:10:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.380+0000 7fc05f631740 1 -- 172.21.15.94:0/441907981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531c5af0 msgr2=0x55b5531b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.380+0000 7fc05f631740 1 --2- 172.21.15.94:0/441907981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531c5af0 0x55b5531b2be0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc050000f80 tx=0x7fc050014fd0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.381+0000 7fc05f631740 1 -- 172.21.15.94:0/441907981 shutdown_connections 2024-02-17T12:10:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.381+0000 7fc05f631740 1 --2- 172.21.15.94:0/441907981 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531c5af0 0x55b5531b2be0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.381+0000 7fc05f631740 1 --2- 172.21.15.94:0/441907981 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531bd2e0 0x55b5531b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.381+0000 7fc05f631740 1 --2- 172.21.15.94:0/441907981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.383 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.381+0000 7fc05f631740 1 -- 172.21.15.94:0/441907981 >> 172.21.15.94:0/441907981 conn(0x55b55317b550 msgr2=0x55b5531b6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.381+0000 7fc05f631740 1 -- 172.21.15.94:0/441907981 shutdown_connections 2024-02-17T12:10:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.381+0000 7fc05f631740 1 -- 172.21.15.94:0/441907981 wait complete. 2024-02-17T12:10:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.382+0000 7fc05f631740 1 -- start start 2024-02-17T12:10:48.384 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.382+0000 7fc05f631740 1 Processor -- start 2024-02-17T12:10:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.382+0000 7fc05f631740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.382+0000 7fc05f631740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531c5af0 0x55b5531af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.382+0000 7fc05f631740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531afc10 0x55b553245ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc05f631740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5532460e0 con 0x55b5531afc10 2024-02-17T12:10:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc05f631740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b553246220 con 0x55b5531c5af0 2024-02-17T12:10:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc05f631740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b553246360 con 0x55b5531bc570 2024-02-17T12:10:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc05f631740 10 client.0 _ll_register_callbacks cb 0x7fc04c003f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:10:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc05659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc05659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531af190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57328/0 (socket says 172.21.15.94:57328) 2024-02-17T12:10:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc05659c640 1 -- 172.21.15.94:0/1210351258 learned_addr learned my addr 172.21.15.94:0/1210351258 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:10:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc055d9b640 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531c5af0 0x55b5531af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc056d9d640 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531afc10 0x55b553245ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.387 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc055d9b640 1 -- 172.21.15.94:0/1210351258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 msgr2=0x55b5531af190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc055d9b640 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531af190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc055d9b640 1 -- 172.21.15.94:0/1210351258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531afc10 msgr2=0x55b553245ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:10:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc055d9b640 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531afc10 0x55b553245ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc055d9b640 1 -- 172.21.15.94:0/1210351258 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b553119d80 con 0x55b5531c5af0 2024-02-17T12:10:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc056d9d640 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531afc10 0x55b553245ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:10:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc055d9b640 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531c5af0 0x55b5531af6d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0bc880 tx=0x7fc04c0bc950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b1ac88d2a5fd5dd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:10:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.383+0000 7fc03dffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:10:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc03dffb640 1 -- 172.21.15.94:0/1210351258 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c0b9350 con 0x55b5531c5af0 2024-02-17T12:10:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc03dffb640 1 -- 172.21.15.94:0/1210351258 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc04c0c0cd0 con 0x55b5531c5af0 2024-02-17T12:10:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc05f631740 1 -- 172.21.15.94:0/1210351258 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b552dc7520 con 0x55b5531c5af0 2024-02-17T12:10:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc03dffb640 1 -- 172.21.15.94:0/1210351258 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc04c0c5290 con 0x55b5531c5af0 2024-02-17T12:10:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc05f631740 10 client.28268 fetch_fsmap learned FSMap version 27 2024-02-17T12:10:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc05f631740 1 -- 172.21.15.94:0/1210351258 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b5530dc540 con 0x55b5531c5af0 2024-02-17T12:10:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc03dffb640 1 -- 172.21.15.94:0/1210351258 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b552dc7520 con 0x55b5531c5af0 2024-02-17T12:10:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc03dffb640 1 -- 172.21.15.94:0/1210351258 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc04c03c070 con 0x55b5531c5af0 2024-02-17T12:10:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc05f631740 10 client.28268 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:10:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc05f631740 10 client.28268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:10:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc05f631740 20 client.28268 populate_metadata read hostname 'smithi094' 2024-02-17T12:10:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc05f631740 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b553247660 0x55b553249a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:10:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc05f631740 4 client.28268 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:10:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc05f631740 1 -- 172.21.15.94:0/1210351258 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b55324a330 con 0x55b553247660 2024-02-17T12:10:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.384+0000 7fc05659c640 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b553247660 0x55b553249a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:10:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.385+0000 7fc03dffb640 10 client.28268 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.385+0000 7fc05659c640 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b553247660 0x55b553249a80 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff02029dbdef7362 server_cookie=dd31b3f5df785a7d in_seq=0 out_seq=0 2024-02-17T12:10:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc03dffb640 1 -- 172.21.15.94:0/1210351258 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b55324a330 con 0x55b553247660 2024-02-17T12:10:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc03dffb640 10 client.28268 handle_command_reply: tid=0 2024-02-17T12:10:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc05f631740 1 client.28268 shutdown 2024-02-17T12:10:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc05f631740 1 -- 172.21.15.94:0/1210351258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531c5af0 msgr2=0x55b5531af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc05f631740 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531c5af0 0x55b5531af6d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc04c0bc880 tx=0x7fc04c0bc950 comp rx=0 tx=0).stop 2024-02-17T12:10:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc05f631740 1 -- 172.21.15.94:0/1210351258 shutdown_connections 2024-02-17T12:10:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc05f631740 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b553247660 0x55b553249a80 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc05f631740 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b5531afc10 0x55b553245ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc03dffb640 0 client.28268 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:10:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc05f631740 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b5531c5af0 0x55b5531af6d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc05f631740 1 --2- 172.21.15.94:0/1210351258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b5531bc570 0x55b5531af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc05f631740 1 -- 172.21.15.94:0/1210351258 >> 172.21.15.94:0/1210351258 conn(0x55b55317b550 msgr2=0x55b5531c13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc05f631740 1 -- 172.21.15.94:0/1210351258 shutdown_connections 2024-02-17T12:10:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc05f631740 1 -- 172.21.15.94:0/1210351258 wait complete. 2024-02-17T12:10:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.402+0000 7fc05f631740 20 client.28268 trim_cache size 0 max 16384 2024-02-17T12:10:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:10:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:10:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:10:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:10:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.403+0000 7fc05ca62640 1 -- 172.21.15.94:0/327634629 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc0400a2300 msgr2=0x7fc0400a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.403+0000 7fc05ca62640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc0400a2300 0x7fc0400a47c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc050005fd0 tx=0x7fc0500135c0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.403+0000 7fc05ca62640 1 -- 172.21.15.94:0/327634629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc058106500 msgr2=0x7fc058100da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:10:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.403+0000 7fc05ca62640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc058106500 0x7fc058100da0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc04c036040 tx=0x7fc04c02c680 comp rx=0 tx=0).stop 2024-02-17T12:10:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.403+0000 7fc05ca62640 1 -- 172.21.15.94:0/327634629 shutdown_connections 2024-02-17T12:10:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.403+0000 7fc05ca62640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc0400a2300 0x7fc0400a47c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.403+0000 7fc05ca62640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0581012e0 0x7fc0581a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.403+0000 7fc05ca62640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc058106500 0x7fc058100da0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.403+0000 7fc05ca62640 1 --2- 172.21.15.94:0/327634629 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0580ff8f0 0x7fc058100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:10:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.403+0000 7fc05ca62640 1 -- 172.21.15.94:0/327634629 >> 172.21.15.94:0/327634629 conn(0x7fc058076610 msgr2=0x7fc0580fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:10:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.403+0000 7fc05ca62640 1 -- 172.21.15.94:0/327634629 shutdown_connections 2024-02-17T12:10:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:10:48.403+0000 7fc05ca62640 1 -- 172.21.15.94:0/327634629 wait complete. 2024-02-17T12:10:49.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:49 smithi189 ceph-mon[36588]: pgmap v396: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-02-17T12:10:49.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:49 smithi125 ceph-mon[36819]: pgmap v396: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-02-17T12:10:49.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:49 smithi094 ceph-mon[32499]: pgmap v396: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 21 MiB/s wr, 1.84k op/s 2024-02-17T12:10:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 750.0 s, 1291.6 tps, lat 76.591 ms stddev 84.923 2024-02-17T12:10:51.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:51 smithi189 ceph-mon[36588]: pgmap v397: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 1.62k op/s 2024-02-17T12:10:51.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:51 smithi125 ceph-mon[36819]: pgmap v397: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 1.62k op/s 2024-02-17T12:10:51.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:51 smithi094 ceph-mon[32499]: pgmap v397: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 MiB/s wr, 1.62k op/s 2024-02-17T12:10:53.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:53 smithi189 ceph-mon[36588]: pgmap v398: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-02-17T12:10:53.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:53 smithi125 ceph-mon[36819]: pgmap v398: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-02-17T12:10:53.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:53 smithi094 ceph-mon[32499]: pgmap v398: 97 pgs: 97 active+clean; 5.6 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 MiB/s wr, 1.52k op/s 2024-02-17T12:10:55.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 755.0 s, 2148.0 tps, lat 47.022 ms stddev 57.946 2024-02-17T12:10:55.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:55 smithi189 ceph-mon[36588]: pgmap v399: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 21 MiB/s wr, 1.78k op/s 2024-02-17T12:10:55.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:55 smithi125 ceph-mon[36819]: pgmap v399: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 21 MiB/s wr, 1.78k op/s 2024-02-17T12:10:55.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:55 smithi094 ceph-mon[32499]: pgmap v399: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 21 MiB/s wr, 1.78k op/s 2024-02-17T12:10:57.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:57 smithi189 ceph-mon[36588]: pgmap v400: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.5 MiB/s wr, 497 op/s 2024-02-17T12:10:57.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:57 smithi125 ceph-mon[36819]: pgmap v400: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.5 MiB/s wr, 497 op/s 2024-02-17T12:10:57.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:57 smithi094 ceph-mon[32499]: pgmap v400: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 4.5 MiB/s wr, 497 op/s 2024-02-17T12:10:59.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:10:59 smithi189 ceph-mon[36588]: pgmap v401: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.7 MiB/s wr, 707 op/s 2024-02-17T12:10:59.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:10:59 smithi125 ceph-mon[36819]: pgmap v401: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.7 MiB/s wr, 707 op/s 2024-02-17T12:10:59.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:10:59 smithi094 ceph-mon[32499]: pgmap v401: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.7 MiB/s wr, 707 op/s 2024-02-17T12:11:00.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 760.0 s, 1452.2 tps, lat 68.765 ms stddev 78.370 2024-02-17T12:11:01.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:01 smithi189 ceph-mon[36588]: pgmap v402: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.5 MiB/s wr, 562 op/s 2024-02-17T12:11:01.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:01 smithi125 ceph-mon[36819]: pgmap v402: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.5 MiB/s wr, 562 op/s 2024-02-17T12:11:01.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:01 smithi094 ceph-mon[32499]: pgmap v402: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.5 MiB/s wr, 562 op/s 2024-02-17T12:11:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.503+0000 7f6f628d0640 1 -- 172.21.15.94:0/1672439132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5c102cc0 msgr2=0x7f6f5c10b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.503+0000 7f6f628d0640 1 --2- 172.21.15.94:0/1672439132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5c102cc0 0x7f6f5c10b1a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f6f50009900 tx=0x7f6f5002bcd0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.503+0000 7f6f628d0640 1 -- 172.21.15.94:0/1672439132 shutdown_connections 2024-02-17T12:11:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.503+0000 7f6f628d0640 1 --2- 172.21.15.94:0/1672439132 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f5c10b770 0x7f6f5c10dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.503+0000 7f6f628d0640 1 --2- 172.21.15.94:0/1672439132 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5c102cc0 0x7f6f5c10b1a0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.503+0000 7f6f628d0640 1 --2- 172.21.15.94:0/1672439132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f5c1023a0 0x7f6f5c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.503+0000 7f6f628d0640 1 -- 172.21.15.94:0/1672439132 >> 172.21.15.94:0/1672439132 conn(0x7f6f5c077290 msgr2=0x7f6f5c0776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.503+0000 7f6f628d0640 1 -- 172.21.15.94:0/1672439132 shutdown_connections 2024-02-17T12:11:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.503+0000 7f6f628d0640 1 -- 172.21.15.94:0/1672439132 wait complete. 2024-02-17T12:11:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.503+0000 7f6f628d0640 1 Processor -- start 2024-02-17T12:11:01.506 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.503+0000 7f6f628d0640 1 -- start start 2024-02-17T12:11:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f628d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f5c1023a0 0x7f6f5c0ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f628d0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f5c102cc0 0x7f6f5c100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f628d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5c10b770 0x7f6f5c100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f628d0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c1a5a90 con 0x7f6f5c10b770 2024-02-17T12:11:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f628d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c1a5c00 con 0x7f6f5c1023a0 2024-02-17T12:11:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f628d0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c1a5dc0 con 0x7f6f5c102cc0 2024-02-17T12:11:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f60e6d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5c10b770 0x7f6f5c100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f5b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f5c102cc0 0x7f6f5c100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f60e6d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5c10b770 0x7f6f5c100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42204/0 (socket says 172.21.15.94:42204) 2024-02-17T12:11:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f60e6d640 1 -- 172.21.15.94:0/1450047667 learned_addr learned my addr 172.21.15.94:0/1450047667 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f5c102cc0 0x7f6f5c100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:42040/0 (socket says 172.21.15.94:42040) 2024-02-17T12:11:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f5bfff640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f5c1023a0 0x7f6f5c0ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f5b7fe640 1 -- 172.21.15.94:0/1450047667 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f5c1023a0 msgr2=0x7f6f5c0ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f5c1023a0 0x7f6f5c0ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f5b7fe640 1 -- 172.21.15.94:0/1450047667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5c10b770 msgr2=0x7f6f5c100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5c10b770 0x7f6f5c100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.504+0000 7f6f5b7fe640 1 -- 172.21.15.94:0/1450047667 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f50009560 con 0x7f6f5c102cc0 2024-02-17T12:11:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.505+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f5c102cc0 0x7f6f5c100530 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6f50036040 tx=0x7f6f5002c720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.505+0000 7f6f597fa640 1 -- 172.21.15.94:0/1450047667 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f50004350 con 0x7f6f5c102cc0 2024-02-17T12:11:01.509 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.505+0000 7f6f597fa640 1 -- 172.21.15.94:0/1450047667 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f500044b0 con 0x7f6f5c102cc0 2024-02-17T12:11:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.505+0000 7f6f628d0640 1 -- 172.21.15.94:0/1450047667 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5c1a6040 con 0x7f6f5c102cc0 2024-02-17T12:11:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.505+0000 7f6f597fa640 1 -- 172.21.15.94:0/1450047667 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f50003d30 con 0x7f6f5c102cc0 2024-02-17T12:11:01.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.505+0000 7f6f628d0640 1 -- 172.21.15.94:0/1450047667 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5c1a6550 con 0x7f6f5c102cc0 2024-02-17T12:11:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.507+0000 7f6f65576740 1 Processor -- start 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.507+0000 7f6f65576740 1 -- start start 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.507+0000 7f6f597fa640 1 -- 172.21.15.94:0/1450047667 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6f5003e450 con 0x7f6f5c102cc0 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f65576740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f2e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f65576740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f2ed60 0x561a67f2f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f65576740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f382e0 0x561a67f3a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f65576740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67e9d9f0 con 0x561a67f2fad0 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f65576740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67e9af20 con 0x561a67f382e0 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f65576740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67e96700 con 0x561a67f2ed60 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f60e6d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f2e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f60e6d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f2e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42218/0 (socket says 172.21.15.94:42218) 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f5bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f2ed60 0x561a67f2f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f60e6d640 1 -- 172.21.15.94:0/3612371624 learned_addr learned my addr 172.21.15.94:0/3612371624 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/3612371624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f382e0 0x561a67f3a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f60e6d640 1 -- 172.21.15.94:0/3612371624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f2ed60 msgr2=0x561a67f2f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f60e6d640 1 --2- 172.21.15.94:0/3612371624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f2ed60 0x561a67f2f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f60e6d640 1 -- 172.21.15.94:0/3612371624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f382e0 msgr2=0x561a67f3a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f60e6d640 1 --2- 172.21.15.94:0/3612371624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f382e0 0x561a67f3a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f597fa640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6f440a2300 0x7f6f440a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f60e6d640 1 -- 172.21.15.94:0/3612371624 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a67e2c7b0 con 0x561a67f2fad0 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f5bfff640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6f440a2300 0x7f6f440a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f597fa640 1 -- 172.21.15.94:0/1450047667 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6f500f0580 con 0x7f6f5c102cc0 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.508+0000 7f6f60e6d640 1 --2- 172.21.15.94:0/3612371624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f2e820 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f6f54011470 tx=0x7f6f54011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a632ab65f91e4d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.509+0000 7f6f427fc640 1 -- 172.21.15.94:0/3612371624 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f54004b20 con 0x561a67f2fad0 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.509+0000 7f6f427fc640 1 -- 172.21.15.94:0/3612371624 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f54004c80 con 0x561a67f2fad0 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.509+0000 7f6f427fc640 1 -- 172.21.15.94:0/3612371624 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f540053a0 con 0x561a67f2fad0 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f5bfff640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6f440a2300 0x7f6f440a47c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x561a67f30230 tx=0x7f6f4c005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 -- 172.21.15.94:0/3612371624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 msgr2=0x561a67f2e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 --2- 172.21.15.94:0/3612371624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f2e820 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f6f54011470 tx=0x7f6f54011930 comp rx=0 tx=0).stop 2024-02-17T12:11:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 -- 172.21.15.94:0/3612371624 shutdown_connections 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 --2- 172.21.15.94:0/3612371624 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f382e0 0x561a67f3a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 --2- 172.21.15.94:0/3612371624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f2ed60 0x561a67f2f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 --2- 172.21.15.94:0/3612371624 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f2e820 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 -- 172.21.15.94:0/3612371624 >> 172.21.15.94:0/3612371624 conn(0x561a67ef6550 msgr2=0x561a67f29a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 -- 172.21.15.94:0/3612371624 shutdown_connections 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 -- 172.21.15.94:0/3612371624 wait complete. 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 -- start start 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 Processor -- start 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f2ed60 0x561a67f35230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f35770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f382e0 0x561a67f35cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67e3bbf0 con 0x561a67f2fad0 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67b6a250 con 0x561a67f2ed60 2024-02-17T12:11:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67e1c200 con 0x561a67f382e0 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.510+0000 7f6f65576740 10 client.0 _ll_register_callbacks cb 0x561a67e8ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f5b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f35770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f5b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f35770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42230/0 (socket says 172.21.15.94:42230) 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f60e6d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f382e0 0x561a67f35cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f5b7fe640 1 -- 172.21.15.94:0/4069955302 learned_addr learned my addr 172.21.15.94:0/4069955302 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f5bfff640 1 --2- 172.21.15.94:0/4069955302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f2ed60 0x561a67f35230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f5b7fe640 1 -- 172.21.15.94:0/4069955302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f382e0 msgr2=0x561a67f35cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/4069955302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f382e0 0x561a67f35cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f5b7fe640 1 -- 172.21.15.94:0/4069955302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f2ed60 msgr2=0x561a67f35230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/4069955302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f2ed60 0x561a67f35230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f5b7fe640 1 -- 172.21.15.94:0/4069955302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f54009cc0 con 0x561a67f2fad0 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/4069955302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f35770 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f6f5002c210 tx=0x7f6f50005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c23fa150b77829 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f3b7fe640 1 -- 172.21.15.94:0/4069955302 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f500bce10 con 0x561a67f2fad0 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f3b7fe640 1 -- 172.21.15.94:0/4069955302 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6f500035f0 con 0x561a67f2fad0 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f65576740 1 -- 172.21.15.94:0/4069955302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a67b96600 con 0x561a67f2fad0 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f3b7fe640 1 -- 172.21.15.94:0/4069955302 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f500bcb80 con 0x561a67f2fad0 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f65576740 10 client.18813 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f65576740 1 -- 172.21.15.94:0/4069955302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561a67e94d80 con 0x561a67f2fad0 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.511+0000 7f6f3b7fe640 1 -- 172.21.15.94:0/4069955302 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561a67b96600 con 0x561a67f2fad0 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.512+0000 7f6f3b7fe640 1 -- 172.21.15.94:0/4069955302 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6f5003c070 con 0x561a67f2fad0 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.512+0000 7f6f65576740 10 client.18813 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.512+0000 7f6f65576740 10 client.18813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.512+0000 7f6f65576740 20 client.18813 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.512+0000 7f6f65576740 1 --2- 172.21.15.94:0/4069955302 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a67f48090 0x561a67f4a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.512+0000 7f6f65576740 4 client.18813 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.512+0000 7f6f65576740 1 -- 172.21.15.94:0/4069955302 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a67be71b0 con 0x561a67f48090 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.512+0000 7f6f5bfff640 1 --2- 172.21.15.94:0/4069955302 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a67f48090 0x561a67f4a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.512+0000 7f6f3b7fe640 10 client.18813 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.512+0000 7f6f5bfff640 1 --2- 172.21.15.94:0/4069955302 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a67f48090 0x561a67f4a480 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d729c1260febb42d server_cookie=afc92433bdad92e3 in_seq=0 out_seq=0 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f3b7fe640 1 -- 172.21.15.94:0/4069955302 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561a67be71b0 con 0x561a67f48090 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f3b7fe640 10 client.18813 handle_command_reply: tid=0 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f65576740 1 client.18813 shutdown 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f65576740 1 -- 172.21.15.94:0/4069955302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 msgr2=0x561a67f35770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f65576740 1 --2- 172.21.15.94:0/4069955302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f35770 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f6f5002c210 tx=0x7f6f50005c90 comp rx=0 tx=0).stop 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f65576740 1 -- 172.21.15.94:0/4069955302 shutdown_connections 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f65576740 1 --2- 172.21.15.94:0/4069955302 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a67f48090 0x561a67f4a480 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f65576740 1 --2- 172.21.15.94:0/4069955302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f382e0 0x561a67f35cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f3b7fe640 0 client.18813 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f65576740 1 --2- 172.21.15.94:0/4069955302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f35770 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f65576740 1 --2- 172.21.15.94:0/4069955302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f2ed60 0x561a67f35230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f65576740 1 -- 172.21.15.94:0/4069955302 >> 172.21.15.94:0/4069955302 conn(0x561a67ef6550 msgr2=0x561a67f2b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f65576740 1 -- 172.21.15.94:0/4069955302 shutdown_connections 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f65576740 1 -- 172.21.15.94:0/4069955302 wait complete. 2024-02-17T12:11:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.513+0000 7f6f65576740 20 client.18813 trim_cache size 0 max 16384 2024-02-17T12:11:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.527+0000 7f6f65576740 1 Processor -- start 2024-02-17T12:11:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.527+0000 7f6f65576740 1 -- start start 2024-02-17T12:11:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.527+0000 7f6f65576740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f2ed60 0x561a67f35ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.527+0000 7f6f65576740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f2b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.527+0000 7f6f65576740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f382e0 0x561a67f3f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.527+0000 7f6f65576740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67b96600 con 0x561a67f2fad0 2024-02-17T12:11:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.527+0000 7f6f65576740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67f300c0 con 0x561a67f382e0 2024-02-17T12:11:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.527+0000 7f6f65576740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67f2f5a0 con 0x561a67f2ed60 2024-02-17T12:11:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.527+0000 7f6f5b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f2b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.527+0000 7f6f5b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f2b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42232/0 (socket says 172.21.15.94:42232) 2024-02-17T12:11:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.527+0000 7f6f5b7fe640 1 -- 172.21.15.94:0/4001600991 learned_addr learned my addr 172.21.15.94:0/4001600991 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.527+0000 7f6f60e6d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f382e0 0x561a67f3f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.527+0000 7f6f5bfff640 1 --2- 172.21.15.94:0/4001600991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f2ed60 0x561a67f35ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.528+0000 7f6f5b7fe640 1 -- 172.21.15.94:0/4001600991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f2ed60 msgr2=0x561a67f35ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.528+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/4001600991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f2ed60 0x561a67f35ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.528+0000 7f6f5b7fe640 1 -- 172.21.15.94:0/4001600991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f382e0 msgr2=0x561a67f3f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.528+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/4001600991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f382e0 0x561a67f3f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.528+0000 7f6f5b7fe640 1 -- 172.21.15.94:0/4001600991 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a67e317b0 con 0x561a67f2fad0 2024-02-17T12:11:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.528+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/4001600991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f2b540 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6f500bcd40 tx=0x7f6f5002bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a00fe3abe61b4372 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.528+0000 7f6f41ffb640 1 -- 172.21.15.94:0/4001600991 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f50003e90 con 0x561a67f2fad0 2024-02-17T12:11:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.528+0000 7f6f41ffb640 1 -- 172.21.15.94:0/4001600991 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f500c55e0 con 0x561a67f2fad0 2024-02-17T12:11:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.528+0000 7f6f41ffb640 1 -- 172.21.15.94:0/4001600991 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f500b7e10 con 0x561a67f2fad0 2024-02-17T12:11:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.529+0000 7f6f65576740 1 -- 172.21.15.94:0/4001600991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 msgr2=0x561a67f2b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.529+0000 7f6f65576740 1 --2- 172.21.15.94:0/4001600991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f2b540 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f6f500bcd40 tx=0x7f6f5002bcd0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.529+0000 7f6f65576740 1 -- 172.21.15.94:0/4001600991 shutdown_connections 2024-02-17T12:11:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.529+0000 7f6f65576740 1 --2- 172.21.15.94:0/4001600991 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f382e0 0x561a67f3f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.529+0000 7f6f65576740 1 --2- 172.21.15.94:0/4001600991 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2fad0 0x561a67f2b540 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.529+0000 7f6f65576740 1 --2- 172.21.15.94:0/4001600991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f2ed60 0x561a67f35ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.529+0000 7f6f65576740 1 -- 172.21.15.94:0/4001600991 >> 172.21.15.94:0/4001600991 conn(0x561a67ef6550 msgr2=0x561a67f2ae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.529+0000 7f6f65576740 1 -- 172.21.15.94:0/4001600991 shutdown_connections 2024-02-17T12:11:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.529+0000 7f6f65576740 1 -- 172.21.15.94:0/4001600991 wait complete. 2024-02-17T12:11:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.529+0000 7f6f65576740 1 -- start start 2024-02-17T12:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.529+0000 7f6f65576740 1 Processor -- start 2024-02-17T12:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f65576740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2ed60 0x561a67f3dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f65576740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f2fad0 0x561a67f3e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f65576740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f382e0 0x561a67f3e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f65576740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67f9e630 con 0x561a67f2ed60 2024-02-17T12:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f65576740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67f9e7a0 con 0x561a67f2fad0 2024-02-17T12:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f65576740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a67f9e960 con 0x561a67f382e0 2024-02-17T12:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f65576740 10 client.0 _ll_register_callbacks cb 0x7f6f50009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f5bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2ed60 0x561a67f3dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f5bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2ed60 0x561a67f3dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42242/0 (socket says 172.21.15.94:42242) 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f5bfff640 1 -- 172.21.15.94:0/71415410 learned_addr learned my addr 172.21.15.94:0/71415410 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f2fad0 0x561a67f3e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f60e6d640 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f382e0 0x561a67f3e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f5bfff640 1 -- 172.21.15.94:0/71415410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f382e0 msgr2=0x561a67f3e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f5bfff640 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f382e0 0x561a67f3e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f5bfff640 1 -- 172.21.15.94:0/71415410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f2fad0 msgr2=0x561a67f3e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f5bfff640 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f2fad0 0x561a67f3e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f5bfff640 1 -- 172.21.15.94:0/71415410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f500d4030 con 0x561a67f2ed60 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f2fad0 0x561a67f3e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f60e6d640 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f382e0 0x561a67f3e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f5bfff640 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2ed60 0x561a67f3dd20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009460 tx=0x7f6f4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b23123d6aa64e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.530+0000 7f6f3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f3b7fe640 1 -- 172.21.15.94:0/71415410 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c025040 con 0x561a67f2ed60 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f3b7fe640 1 -- 172.21.15.94:0/71415410 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6f4c02a7a0 con 0x561a67f2ed60 2024-02-17T12:11:01.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f65576740 1 -- 172.21.15.94:0/71415410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6f54004c80 con 0x561a67f2ed60 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f3b7fe640 1 -- 172.21.15.94:0/71415410 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f4c017070 con 0x561a67f2ed60 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f65576740 10 client.18819 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f65576740 1 -- 172.21.15.94:0/71415410 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561a67e57540 con 0x561a67f2ed60 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f3b7fe640 1 -- 172.21.15.94:0/71415410 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6f4c03e020 con 0x561a67f2ed60 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f3b7fe640 1 -- 172.21.15.94:0/71415410 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6f4c0047a0 con 0x561a67f2ed60 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f65576740 10 client.18819 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f65576740 10 client.18819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f65576740 20 client.18819 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f65576740 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a67fa01b0 0x561a67fa25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f65576740 4 client.18819 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f65576740 1 -- 172.21.15.94:0/71415410 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561a67bd3550 con 0x561a67fa01b0 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.531+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a67fa01b0 0x561a67fa25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.532+0000 7f6f3b7fe640 10 client.18819 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:01.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.532+0000 7f6f5b7fe640 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a67fa01b0 0x561a67fa25d0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff766036eb77cd13 server_cookie=9e5497230dcd48b7 in_seq=0 out_seq=0 2024-02-17T12:11:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f3b7fe640 1 -- 172.21.15.94:0/71415410 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561a67bd3550 con 0x561a67fa01b0 2024-02-17T12:11:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f3b7fe640 10 client.18819 handle_command_reply: tid=0 2024-02-17T12:11:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f65576740 1 client.18819 shutdown 2024-02-17T12:11:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f65576740 1 -- 172.21.15.94:0/71415410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2ed60 msgr2=0x561a67f3dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f65576740 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2ed60 0x561a67f3dd20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c009460 tx=0x7f6f4c0046c0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f65576740 1 -- 172.21.15.94:0/71415410 shutdown_connections 2024-02-17T12:11:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f65576740 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x561a67fa01b0 0x561a67fa25d0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f65576740 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x561a67f382e0 0x561a67f3e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f65576740 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561a67f2fad0 0x561a67f3e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f3b7fe640 0 client.18819 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f65576740 1 --2- 172.21.15.94:0/71415410 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561a67f2ed60 0x561a67f3dd20 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f65576740 1 -- 172.21.15.94:0/71415410 >> 172.21.15.94:0/71415410 conn(0x561a67ef6550 msgr2=0x561a67f2aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f65576740 1 -- 172.21.15.94:0/71415410 shutdown_connections 2024-02-17T12:11:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f65576740 1 -- 172.21.15.94:0/71415410 wait complete. 2024-02-17T12:11:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.549+0000 7f6f65576740 20 client.18819 trim_cache size 0 max 16384 2024-02-17T12:11:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:11:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:11:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:11:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:11:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.550+0000 7f6f628d0640 1 -- 172.21.15.94:0/1450047667 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6f440a2300 msgr2=0x7f6f440a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.550+0000 7f6f628d0640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6f440a2300 0x7f6f440a47c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x561a67f30230 tx=0x7f6f4c005f90 comp rx=0 tx=0).stop 2024-02-17T12:11:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.550+0000 7f6f628d0640 1 -- 172.21.15.94:0/1450047667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f5c102cc0 msgr2=0x7f6f5c100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.550+0000 7f6f628d0640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f5c102cc0 0x7f6f5c100530 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6f50036040 tx=0x7f6f5002c720 comp rx=0 tx=0).stop 2024-02-17T12:11:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.550+0000 7f6f628d0640 1 -- 172.21.15.94:0/1450047667 shutdown_connections 2024-02-17T12:11:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.550+0000 7f6f628d0640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6f440a2300 0x7f6f440a47c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.550+0000 7f6f628d0640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6f5c10b770 0x7f6f5c100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.550+0000 7f6f628d0640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f5c102cc0 0x7f6f5c100530 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.550+0000 7f6f628d0640 1 --2- 172.21.15.94:0/1450047667 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6f5c1023a0 0x7f6f5c0ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.550+0000 7f6f628d0640 1 -- 172.21.15.94:0/1450047667 >> 172.21.15.94:0/1450047667 conn(0x7f6f5c077290 msgr2=0x7f6f5c0fd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.550+0000 7f6f628d0640 1 -- 172.21.15.94:0/1450047667 shutdown_connections 2024-02-17T12:11:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:01.550+0000 7f6f628d0640 1 -- 172.21.15.94:0/1450047667 wait complete. 2024-02-17T12:11:03.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:03 smithi125 ceph-mon[36819]: osdmap e101: 12 total, 12 up, 12 in 2024-02-17T12:11:03.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:03 smithi125 ceph-mon[36819]: pgmap v404: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 618 op/s 2024-02-17T12:11:03.707 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:03 smithi094 ceph-mon[32499]: osdmap e101: 12 total, 12 up, 12 in 2024-02-17T12:11:03.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:03 smithi094 ceph-mon[32499]: pgmap v404: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 618 op/s 2024-02-17T12:11:03.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:03 smithi189 ceph-mon[36588]: osdmap e101: 12 total, 12 up, 12 in 2024-02-17T12:11:03.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:03 smithi189 ceph-mon[36588]: pgmap v404: 97 pgs: 97 active+clean; 5.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 6.1 MiB/s wr, 618 op/s 2024-02-17T12:11:04.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:11:04.129 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:11:04.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.301+0000 7f8011702640 1 -- 172.21.15.94:0/3378298474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1002d0 msgr2=0x7f800c100730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.301+0000 7f8011702640 1 --2- 172.21.15.94:0/3378298474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1002d0 0x7f800c100730 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8000009900 tx=0x7f800002bc40 comp rx=0 tx=0).stop 2024-02-17T12:11:04.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.301+0000 7f8011702640 1 -- 172.21.15.94:0/3378298474 shutdown_connections 2024-02-17T12:11:04.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.301+0000 7f8011702640 1 --2- 172.21.15.94:0/3378298474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c100c70 0x7f800c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.301+0000 7f8011702640 1 --2- 172.21.15.94:0/3378298474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1002d0 0x7f800c100730 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.301+0000 7f8011702640 1 --2- 172.21.15.94:0/3378298474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f800c1060a0 0x7f800c106480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.301+0000 7f8011702640 1 -- 172.21.15.94:0/3378298474 >> 172.21.15.94:0/3378298474 conn(0x7f800c076e10 msgr2=0x7f800c077220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:04.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.301+0000 7f8011702640 1 -- 172.21.15.94:0/3378298474 shutdown_connections 2024-02-17T12:11:04.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.301+0000 7f8011702640 1 -- 172.21.15.94:0/3378298474 wait complete. 2024-02-17T12:11:04.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.302+0000 7f8011702640 1 Processor -- start 2024-02-17T12:11:04.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.302+0000 7f8011702640 1 -- start start 2024-02-17T12:11:04.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.302+0000 7f8011702640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c1002d0 0x7f800c19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.302+0000 7f8011702640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f800c100c70 0x7f800c19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.302+0000 7f8011702640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1060a0 0x7f800c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.302+0000 7f8011702640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c199ee0 con 0x7f800c1002d0 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.302+0000 7f8011702640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c19a050 con 0x7f800c1060a0 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.302+0000 7f8011702640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c19a210 con 0x7f800c100c70 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.302+0000 7f800affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c1002d0 0x7f800c19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.302+0000 7f800affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c1002d0 0x7f800c19f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42266/0 (socket says 172.21.15.94:42266) 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.302+0000 7f800affd640 1 -- 172.21.15.94:0/3043749533 learned_addr learned my addr 172.21.15.94:0/3043749533 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.302+0000 7f800a7fc640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f800c100c70 0x7f800c19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.302+0000 7f800b7fe640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1060a0 0x7f800c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.303+0000 7f800a7fc640 1 -- 172.21.15.94:0/3043749533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1060a0 msgr2=0x7f800c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.303+0000 7f800a7fc640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1060a0 0x7f800c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.303+0000 7f800a7fc640 1 -- 172.21.15.94:0/3043749533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c1002d0 msgr2=0x7f800c19f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.303+0000 7f800a7fc640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c1002d0 0x7f800c19f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.303+0000 7f800a7fc640 1 -- 172.21.15.94:0/3043749533 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8000009560 con 0x7f800c100c70 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.303+0000 7f800a7fc640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f800c100c70 0x7f800c19fbd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f800002c180 tx=0x7f800002cd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:04.307 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.303+0000 7f7ff3fff640 1 -- 172.21.15.94:0/3043749533 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000004350 con 0x7f800c100c70 2024-02-17T12:11:04.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.304+0000 7f7ff3fff640 1 -- 172.21.15.94:0/3043749533 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80000044b0 con 0x7f800c100c70 2024-02-17T12:11:04.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.304+0000 7f7ff3fff640 1 -- 172.21.15.94:0/3043749533 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000003c60 con 0x7f800c100c70 2024-02-17T12:11:04.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.304+0000 7f8011702640 1 -- 172.21.15.94:0/3043749533 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c19a490 con 0x7f800c100c70 2024-02-17T12:11:04.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.304+0000 7f8011702640 1 -- 172.21.15.94:0/3043749533 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c1a6610 con 0x7f800c100c70 2024-02-17T12:11:04.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.306+0000 7f80143b4740 1 Processor -- start 2024-02-17T12:11:04.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f80143b4740 1 -- start start 2024-02-17T12:11:04.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f80143b4740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 0x558229e2fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f80143b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e303e0 0x558229e38a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f80143b4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e38fc0 0x558229e3b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f80143b4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558229d96280 con 0x558229e38fc0 2024-02-17T12:11:04.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f80143b4740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558229d937b0 con 0x558229e2fac0 2024-02-17T12:11:04.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f80143b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558229d8ef90 con 0x558229e303e0 2024-02-17T12:11:04.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f7ff3fff640 1 -- 172.21.15.94:0/3043749533 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f800003e450 con 0x7f800c100c70 2024-02-17T12:11:04.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f800b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 0x558229e2fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f800b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 0x558229e2fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52754/0 (socket says 172.21.15.94:52754) 2024-02-17T12:11:04.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f800b7fe640 1 -- 172.21.15.94:0/3602265707 learned_addr learned my addr 172.21.15.94:0/3602265707 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:04.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f800affd640 1 --2- 172.21.15.94:0/3602265707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e303e0 0x558229e38a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f800a7fc640 1 --2- 172.21.15.94:0/3602265707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e38fc0 0x558229e3b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f800b7fe640 1 -- 172.21.15.94:0/3602265707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e303e0 msgr2=0x558229e38a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:04.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f800b7fe640 1 --2- 172.21.15.94:0/3602265707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e303e0 0x558229e38a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f800b7fe640 1 -- 172.21.15.94:0/3602265707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e38fc0 msgr2=0x558229e3b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:04.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f800b7fe640 1 --2- 172.21.15.94:0/3602265707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e38fc0 0x558229e3b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f800b7fe640 1 -- 172.21.15.94:0/3602265707 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558229d4d570 con 0x558229e2fac0 2024-02-17T12:11:04.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.307+0000 7f800a7fc640 1 --2- 172.21.15.94:0/3602265707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e38fc0 0x558229e3b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:11:04.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.308+0000 7f7ff3fff640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7ff40a22b0 0x7f7ff40a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.308+0000 7f800affd640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7ff40a22b0 0x7f7ff40a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.308+0000 7f7ff3fff640 1 -- 172.21.15.94:0/3043749533 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f80000f0620 con 0x7f800c100c70 2024-02-17T12:11:04.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.308+0000 7f800b7fe640 1 --2- 172.21.15.94:0/3602265707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 0x558229e2fea0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8004010370 tx=0x7f8004010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0fb0baf5f8f9615 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:04.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.308+0000 7f7ff17fa640 1 -- 172.21.15.94:0/3602265707 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004004d20 con 0x558229e2fac0 2024-02-17T12:11:04.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.308+0000 7f7ff17fa640 1 -- 172.21.15.94:0/3602265707 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8004004e80 con 0x558229e2fac0 2024-02-17T12:11:04.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.308+0000 7f7ff17fa640 1 -- 172.21.15.94:0/3602265707 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004005500 con 0x558229e2fac0 2024-02-17T12:11:04.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.309+0000 7f80143b4740 1 -- 172.21.15.94:0/3602265707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 msgr2=0x558229e2fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.309+0000 7f80143b4740 1 --2- 172.21.15.94:0/3602265707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 0x558229e2fea0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8004010370 tx=0x7f8004010830 comp rx=0 tx=0).stop 2024-02-17T12:11:04.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.309+0000 7f800affd640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7ff40a22b0 0x7f7ff40a4770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009640 tx=0x7f7ffc005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:04.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.309+0000 7f80143b4740 1 -- 172.21.15.94:0/3602265707 shutdown_connections 2024-02-17T12:11:04.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.309+0000 7f80143b4740 1 --2- 172.21.15.94:0/3602265707 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e38fc0 0x558229e3b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.309+0000 7f80143b4740 1 --2- 172.21.15.94:0/3602265707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e303e0 0x558229e38a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.309+0000 7f80143b4740 1 --2- 172.21.15.94:0/3602265707 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 0x558229e2fea0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.309+0000 7f80143b4740 1 -- 172.21.15.94:0/3602265707 >> 172.21.15.94:0/3602265707 conn(0x558229e2a1a0 msgr2=0x558229e2a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.310+0000 7f80143b4740 1 -- 172.21.15.94:0/3602265707 shutdown_connections 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.310+0000 7f80143b4740 1 -- 172.21.15.94:0/3602265707 wait complete. 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.310+0000 7f80143b4740 1 -- start start 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.310+0000 7f80143b4740 1 Processor -- start 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f80143b4740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 0x558229e38a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f80143b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e303e0 0x558229e35e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f80143b4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e38fc0 0x558229e36540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f80143b4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558229a62430 con 0x558229e38fc0 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f80143b4740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558229a1ee00 con 0x558229e2fac0 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f80143b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558229d13fd0 con 0x558229e303e0 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f800affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 0x558229e38a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f800affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 0x558229e38a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52768/0 (socket says 172.21.15.94:52768) 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f800affd640 1 -- 172.21.15.94:0/3990444710 learned_addr learned my addr 172.21.15.94:0/3990444710 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f80143b4740 10 client.0 _ll_register_callbacks cb 0x558229d69fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:04.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f800a7fc640 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e303e0 0x558229e35e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f800b7fe640 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e38fc0 0x558229e36540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f800affd640 1 -- 172.21.15.94:0/3990444710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e303e0 msgr2=0x558229e35e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f800affd640 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e303e0 0x558229e35e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f800affd640 1 -- 172.21.15.94:0/3990444710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e38fc0 msgr2=0x558229e36540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f800affd640 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e38fc0 0x558229e36540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f800affd640 1 -- 172.21.15.94:0/3990444710 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8004009c50 con 0x558229e2fac0 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f800b7fe640 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e38fc0 0x558229e36540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f800affd640 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 0x558229e38a70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc01e9d0 tx=0x7f7ffc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cc7d3ccb3269670 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.311+0000 7f7feeff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f7feeff5640 1 -- 172.21.15.94:0/3990444710 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc023070 con 0x558229e2fac0 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f80143b4740 1 -- 172.21.15.94:0/3990444710 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558229acb720 con 0x558229e2fac0 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f7feeff5640 1 -- 172.21.15.94:0/3990444710 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f7ffc005020 con 0x558229e2fac0 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f7feeff5640 1 -- 172.21.15.94:0/3990444710 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ffc017070 con 0x558229e2fac0 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f80143b4740 10 client.28298 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f80143b4740 1 -- 172.21.15.94:0/3990444710 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558229d8d610 con 0x558229e2fac0 2024-02-17T12:11:04.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f7feeff5640 1 -- 172.21.15.94:0/3990444710 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ffc035020 con 0x558229e2fac0 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f7feeff5640 1 -- 172.21.15.94:0/3990444710 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f7ffc029420 con 0x558229e2fac0 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f80143b4740 10 client.28298 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f80143b4740 10 client.28298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f80143b4740 20 client.28298 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f80143b4740 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558229e49180 0x558229e4b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f80143b4740 4 client.28298 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f80143b4740 1 -- 172.21.15.94:0/3990444710 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558229b1a890 con 0x558229e49180 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.312+0000 7f800a7fc640 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558229e49180 0x558229e4b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.313+0000 7f7feeff5640 10 client.28298 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.313+0000 7f800a7fc640 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558229e49180 0x558229e4b570 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a44f6c9da38cb3c2 server_cookie=6fc5f8281d37ee75 in_seq=0 out_seq=0 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.313+0000 7f7feeff5640 1 -- 172.21.15.94:0/3990444710 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558229b1a890 con 0x558229e49180 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.313+0000 7f7feeff5640 10 client.28298 handle_command_reply: tid=0 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.313+0000 7f80143b4740 1 client.28298 shutdown 2024-02-17T12:11:04.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.314+0000 7f80143b4740 1 -- 172.21.15.94:0/3990444710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 msgr2=0x558229e38a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.314+0000 7f80143b4740 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 0x558229e38a70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc01e9d0 tx=0x7f7ffc004730 comp rx=0 tx=0).stop 2024-02-17T12:11:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.314+0000 7f80143b4740 1 -- 172.21.15.94:0/3990444710 shutdown_connections 2024-02-17T12:11:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.314+0000 7f80143b4740 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558229e49180 0x558229e4b570 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.314+0000 7f7feeff5640 0 client.28298 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.314+0000 7f80143b4740 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e38fc0 0x558229e36540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.314+0000 7f80143b4740 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e303e0 0x558229e35e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.314+0000 7f80143b4740 1 --2- 172.21.15.94:0/3990444710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e2fac0 0x558229e38a70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.314+0000 7f80143b4740 1 -- 172.21.15.94:0/3990444710 >> 172.21.15.94:0/3990444710 conn(0x558229e2a1a0 msgr2=0x558229e2bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.314+0000 7f80143b4740 1 -- 172.21.15.94:0/3990444710 shutdown_connections 2024-02-17T12:11:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.314+0000 7f80143b4740 1 -- 172.21.15.94:0/3990444710 wait complete. 2024-02-17T12:11:04.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.314+0000 7f80143b4740 20 client.28298 trim_cache size 0 max 16384 2024-02-17T12:11:04.339 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.335+0000 7f80143b4740 1 Processor -- start 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.335+0000 7f80143b4740 1 -- start start 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.335+0000 7f80143b4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e2fac0 0x558229e35a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.335+0000 7f80143b4740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e303e0 0x558229e38530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.335+0000 7f80143b4740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e38fc0 0x558229e38a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.335+0000 7f80143b4740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558229b1a890 con 0x558229e2fac0 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.335+0000 7f80143b4740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558229e45d20 con 0x558229e303e0 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.335+0000 7f80143b4740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558229e2f500 con 0x558229e38fc0 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f800affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e2fac0 0x558229e35a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f800b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e38fc0 0x558229e38a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f800affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e2fac0 0x558229e35a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42288/0 (socket says 172.21.15.94:42288) 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f800affd640 1 -- 172.21.15.94:0/3723204220 learned_addr learned my addr 172.21.15.94:0/3723204220 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f800a7fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e303e0 0x558229e38530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f800affd640 1 -- 172.21.15.94:0/3723204220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e38fc0 msgr2=0x558229e38a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:04.340 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f800affd640 1 --2- 172.21.15.94:0/3723204220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e38fc0 0x558229e38a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f800affd640 1 -- 172.21.15.94:0/3723204220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e303e0 msgr2=0x558229e38530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f800affd640 1 --2- 172.21.15.94:0/3723204220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e303e0 0x558229e38530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f800affd640 1 -- 172.21.15.94:0/3723204220 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558229d29ac0 con 0x558229e2fac0 2024-02-17T12:11:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f800a7fc640 1 --2- 172.21.15.94:0/3723204220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e303e0 0x558229e38530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f800b7fe640 1 --2- 172.21.15.94:0/3723204220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e38fc0 0x558229e38a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f800affd640 1 --2- 172.21.15.94:0/3723204220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e2fac0 0x558229e35a70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc01ef10 tx=0x7f7ffc012db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ba56ac1039614e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f7ff0ff9640 1 -- 172.21.15.94:0/3723204220 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc023040 con 0x558229e2fac0 2024-02-17T12:11:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f7ff0ff9640 1 -- 172.21.15.94:0/3723204220 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ffc035070 con 0x558229e2fac0 2024-02-17T12:11:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.336+0000 7f7ff0ff9640 1 -- 172.21.15.94:0/3723204220 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc005450 con 0x558229e2fac0 2024-02-17T12:11:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.337+0000 7f80143b4740 1 -- 172.21.15.94:0/3723204220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e2fac0 msgr2=0x558229e35a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.337+0000 7f80143b4740 1 --2- 172.21.15.94:0/3723204220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e2fac0 0x558229e35a70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc01ef10 tx=0x7f7ffc012db0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.337+0000 7f80143b4740 1 -- 172.21.15.94:0/3723204220 shutdown_connections 2024-02-17T12:11:04.341 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.337+0000 7f80143b4740 1 --2- 172.21.15.94:0/3723204220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e38fc0 0x558229e38a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.337+0000 7f80143b4740 1 --2- 172.21.15.94:0/3723204220 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e303e0 0x558229e38530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.337+0000 7f80143b4740 1 --2- 172.21.15.94:0/3723204220 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e2fac0 0x558229e35a70 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.337+0000 7f80143b4740 1 -- 172.21.15.94:0/3723204220 >> 172.21.15.94:0/3723204220 conn(0x558229e2a1a0 msgr2=0x558229e35360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:04.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.337+0000 7f80143b4740 1 -- 172.21.15.94:0/3723204220 shutdown_connections 2024-02-17T12:11:04.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.337+0000 7f80143b4740 1 -- 172.21.15.94:0/3723204220 wait complete. 2024-02-17T12:11:04.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.338+0000 7f80143b4740 1 -- start start 2024-02-17T12:11:04.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.338+0000 7f80143b4740 1 Processor -- start 2024-02-17T12:11:04.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.338+0000 7f80143b4740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e2fac0 0x558229e40020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.338+0000 7f80143b4740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e303e0 0x558229e3d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.339+0000 7f800affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e2fac0 0x558229e40020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.339+0000 7f800affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e2fac0 0x558229e40020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:42304/0 (socket says 172.21.15.94:42304) 2024-02-17T12:11:04.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.339+0000 7f800affd640 1 -- 172.21.15.94:0/2493920481 learned_addr learned my addr 172.21.15.94:0/2493920481 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:04.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.339+0000 7f80143b4740 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e38fc0 0x558229e3d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.339+0000 7f80143b4740 1 -- 172.21.15.94:0/2493920481 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558229e3de30 con 0x558229e2fac0 2024-02-17T12:11:04.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.339+0000 7f80143b4740 1 -- 172.21.15.94:0/2493920481 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558229e3dfa0 con 0x558229e303e0 2024-02-17T12:11:04.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.339+0000 7f80143b4740 1 -- 172.21.15.94:0/2493920481 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558229e3e160 con 0x558229e38fc0 2024-02-17T12:11:04.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.339+0000 7f800a7fc640 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e303e0 0x558229e3d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.339+0000 7f80143b4740 10 client.0 _ll_register_callbacks cb 0x7f8004009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:04.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.339+0000 7f800b7fe640 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e38fc0 0x558229e3d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.339+0000 7f800a7fc640 1 -- 172.21.15.94:0/2493920481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e38fc0 msgr2=0x558229e3d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:04.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.340+0000 7f800a7fc640 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e38fc0 0x558229e3d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.340+0000 7f800a7fc640 1 -- 172.21.15.94:0/2493920481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e2fac0 msgr2=0x558229e40020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.340+0000 7f800a7fc640 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e2fac0 0x558229e40020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.340+0000 7f800a7fc640 1 -- 172.21.15.94:0/2493920481 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c1a6610 con 0x558229e303e0 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.340+0000 7f800a7fc640 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e303e0 0x558229e3d3b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f80000ede10 tx=0x7f800002bc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f60cccfaac9b8014 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.340+0000 7f7feeff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.340+0000 7f7feeff5640 1 -- 172.21.15.94:0/2493920481 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000c5850 con 0x558229e303e0 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.340+0000 7f7feeff5640 1 -- 172.21.15.94:0/2493920481 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f80000bb1e0 con 0x558229e303e0 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.340+0000 7f80143b4740 1 -- 172.21.15.94:0/2493920481 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558229a8e7e0 con 0x558229e303e0 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.340+0000 7f7feeff5640 1 -- 172.21.15.94:0/2493920481 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80000bb440 con 0x558229e303e0 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.340+0000 7f80143b4740 10 client.28304 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.340+0000 7f80143b4740 1 -- 172.21.15.94:0/2493920481 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558229d4d570 con 0x558229e303e0 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.341+0000 7f7feeff5640 1 -- 172.21.15.94:0/2493920481 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f80000db020 con 0x558229e303e0 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.341+0000 7f7feeff5640 1 -- 172.21.15.94:0/2493920481 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f800003c070 con 0x558229e303e0 2024-02-17T12:11:04.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.341+0000 7f80143b4740 10 client.28304 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.341+0000 7f80143b4740 10 client.28304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.341+0000 7f80143b4740 20 client.28304 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.341+0000 7f80143b4740 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558229ea1820 0x558229ea3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.341+0000 7f80143b4740 4 client.28304 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.341+0000 7f80143b4740 1 -- 172.21.15.94:0/2493920481 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558229a7d4e0 con 0x558229ea1820 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.341+0000 7f800affd640 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558229ea1820 0x558229ea3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.341+0000 7f7feeff5640 10 client.28304 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.341+0000 7f800affd640 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558229ea1820 0x558229ea3c40 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8ea5dfce1db7398 server_cookie=cc9d7ced3c76c5e1 in_seq=0 out_seq=0 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f7feeff5640 1 -- 172.21.15.94:0/2493920481 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558229a7d4e0 con 0x558229ea1820 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f7feeff5640 10 client.28304 handle_command_reply: tid=0 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f80143b4740 1 client.28304 shutdown 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f80143b4740 1 -- 172.21.15.94:0/2493920481 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e303e0 msgr2=0x558229e3d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f80143b4740 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e303e0 0x558229e3d3b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f80000ede10 tx=0x7f800002bc40 comp rx=0 tx=0).stop 2024-02-17T12:11:04.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f80143b4740 1 -- 172.21.15.94:0/2493920481 shutdown_connections 2024-02-17T12:11:04.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f80143b4740 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558229ea1820 0x558229ea3c40 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f80143b4740 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558229e38fc0 0x558229e3d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f80143b4740 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558229e303e0 0x558229e3d3b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f80143b4740 1 --2- 172.21.15.94:0/2493920481 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558229e2fac0 0x558229e40020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f80143b4740 1 -- 172.21.15.94:0/2493920481 >> 172.21.15.94:0/2493920481 conn(0x558229e2a1a0 msgr2=0x558229e35bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:04.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f7feeff5640 0 client.28304 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:04.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f80143b4740 1 -- 172.21.15.94:0/2493920481 shutdown_connections 2024-02-17T12:11:04.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f80143b4740 1 -- 172.21.15.94:0/2493920481 wait complete. 2024-02-17T12:11:04.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f80143b4740 20 client.28304 trim_cache size 0 max 16384 2024-02-17T12:11:04.349 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:11:04.350 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:11:04.350 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:11:04.350 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:11:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f8011702640 1 -- 172.21.15.94:0/3043749533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7ff40a22b0 msgr2=0x7f7ff40a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f8011702640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7ff40a22b0 0x7f7ff40a4770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009640 tx=0x7f7ffc005f90 comp rx=0 tx=0).stop 2024-02-17T12:11:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f8011702640 1 -- 172.21.15.94:0/3043749533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f800c100c70 msgr2=0x7f800c19fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.342+0000 7f8011702640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f800c100c70 0x7f800c19fbd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f800002c180 tx=0x7f800002cd90 comp rx=0 tx=0).stop 2024-02-17T12:11:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.343+0000 7f8011702640 1 -- 172.21.15.94:0/3043749533 shutdown_connections 2024-02-17T12:11:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.343+0000 7f8011702640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f7ff40a22b0 0x7f7ff40a4770 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.343+0000 7f8011702640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f800c1060a0 0x7f800c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.343+0000 7f8011702640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f800c100c70 0x7f800c19fbd0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.343+0000 7f8011702640 1 --2- 172.21.15.94:0/3043749533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f800c1002d0 0x7f800c19f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.343+0000 7f8011702640 1 -- 172.21.15.94:0/3043749533 >> 172.21.15.94:0/3043749533 conn(0x7f800c076e10 msgr2=0x7f800c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.343+0000 7f8011702640 1 -- 172.21.15.94:0/3043749533 shutdown_connections 2024-02-17T12:11:04.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.343+0000 7f8011702640 1 -- 172.21.15.94:0/3043749533 wait complete. 2024-02-17T12:11:04.359 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:11:04.359 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:11:04.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.528+0000 7f4373a40640 1 -- 172.21.15.94:0/831673329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f436c1142d0 msgr2=0x7f436c1166c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.528+0000 7f4373a40640 1 --2- 172.21.15.94:0/831673329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f436c1142d0 0x7f436c1166c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4364009ca0 tx=0x7f436402c120 comp rx=0 tx=0).stop 2024-02-17T12:11:04.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.528+0000 7f4373a40640 1 -- 172.21.15.94:0/831673329 shutdown_connections 2024-02-17T12:11:04.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.528+0000 7f4373a40640 1 --2- 172.21.15.94:0/831673329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f436c1142d0 0x7f436c1166c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.530 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.528+0000 7f4373a40640 1 --2- 172.21.15.94:0/831673329 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f436c108aa0 0x7f436c113d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.528+0000 7f4373a40640 1 --2- 172.21.15.94:0/831673329 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f436c108180 0x7f436c108560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.528+0000 7f4373a40640 1 -- 172.21.15.94:0/831673329 >> 172.21.15.94:0/831673329 conn(0x7f436c06c6d0 msgr2=0x7f436c06cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:04.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.528+0000 7f4373a40640 1 -- 172.21.15.94:0/831673329 shutdown_connections 2024-02-17T12:11:04.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.528+0000 7f4373a40640 1 -- 172.21.15.94:0/831673329 wait complete. 2024-02-17T12:11:04.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.529+0000 7f4373a40640 1 Processor -- start 2024-02-17T12:11:04.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.529+0000 7f4373a40640 1 -- start start 2024-02-17T12:11:04.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4373a40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f436c108180 0x7f436c1ac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4373a40640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f436c108aa0 0x7f436c1ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.534 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4373a40640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f436c1142d0 0x7f436c112ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4373a40640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c113510 con 0x7f436c108180 2024-02-17T12:11:04.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4373a40640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c113680 con 0x7f436c108aa0 2024-02-17T12:11:04.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4373a40640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c113840 con 0x7f436c1142d0 2024-02-17T12:11:04.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4371fdd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f436c1142d0 0x7f436c112ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4371fdd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f436c1142d0 0x7f436c112ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:59986/0 (socket says 172.21.15.94:59986) 2024-02-17T12:11:04.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4371fdd640 1 -- 172.21.15.94:0/352203758 learned_addr learned my addr 172.21.15.94:0/352203758 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:04.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4370fdb640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f436c108aa0 0x7f436c1ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f43717dc640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f436c108180 0x7f436c1ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4371fdd640 1 -- 172.21.15.94:0/352203758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f436c108aa0 msgr2=0x7f436c1ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4371fdd640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f436c108aa0 0x7f436c1ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.535 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4371fdd640 1 -- 172.21.15.94:0/352203758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f436c108180 msgr2=0x7f436c1ac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4371fdd640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f436c108180 0x7f436c1ac370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.530+0000 7f4371fdd640 1 -- 172.21.15.94:0/352203758 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4364009900 con 0x7f436c1142d0 2024-02-17T12:11:04.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.531+0000 7f4371fdd640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f436c1142d0 0x7f436c112ec0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4364037040 tx=0x7f4364032970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:04.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.531+0000 7f43627fc640 1 -- 172.21.15.94:0/352203758 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364034640 con 0x7f436c1142d0 2024-02-17T12:11:04.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.531+0000 7f43627fc640 1 -- 172.21.15.94:0/352203758 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4364003e50 con 0x7f436c1142d0 2024-02-17T12:11:04.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.531+0000 7f4373a40640 1 -- 172.21.15.94:0/352203758 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436c113ac0 con 0x7f436c1142d0 2024-02-17T12:11:04.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.531+0000 7f4373a40640 1 -- 172.21.15.94:0/352203758 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c1b33a0 con 0x7f436c1142d0 2024-02-17T12:11:04.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.532+0000 7f43627fc640 1 -- 172.21.15.94:0/352203758 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43640047d0 con 0x7f436c1142d0 2024-02-17T12:11:04.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.533+0000 7f4373a40640 1 -- 172.21.15.94:0/352203758 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436c067f70 con 0x7f436c1142d0 2024-02-17T12:11:04.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.535+0000 7f43627fc640 1 -- 172.21.15.94:0/352203758 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f436400a040 con 0x7f436c1142d0 2024-02-17T12:11:04.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.535+0000 7f43627fc640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43580a22f0 0x7f43580a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:04.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.536+0000 7f43627fc640 1 -- 172.21.15.94:0/352203758 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f43640f0bf0 con 0x7f436c1142d0 2024-02-17T12:11:04.537 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.536+0000 7f43717dc640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43580a22f0 0x7f43580a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:04.538 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.536+0000 7f43717dc640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43580a22f0 0x7f43580a47b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4368006fd0 tx=0x7f4368008040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:04.542 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.540+0000 7f43627fc640 1 -- 172.21.15.94:0/352203758 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43640b98c0 con 0x7f436c1142d0 2024-02-17T12:11:04.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.894+0000 7f4373a40640 1 -- 172.21.15.94:0/352203758 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f436c10d240 con 0x7f436c1142d0 2024-02-17T12:11:04.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.896+0000 7f43627fc640 1 -- 172.21.15.94:0/352203758 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7f43640bcf80 con 0x7f436c1142d0 2024-02-17T12:11:04.898 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:11:04.898 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:11:04.898 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:11:04.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.898+0000 7f4373a40640 1 -- 172.21.15.94:0/352203758 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43580a22f0 msgr2=0x7f43580a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.898+0000 7f4373a40640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43580a22f0 0x7f43580a47b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4368006fd0 tx=0x7f4368008040 comp rx=0 tx=0).stop 2024-02-17T12:11:04.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.898+0000 7f4373a40640 1 -- 172.21.15.94:0/352203758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f436c1142d0 msgr2=0x7f436c112ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:04.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.898+0000 7f4373a40640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f436c1142d0 0x7f436c112ec0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4364037040 tx=0x7f4364032970 comp rx=0 tx=0).stop 2024-02-17T12:11:04.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.898+0000 7f4373a40640 1 -- 172.21.15.94:0/352203758 shutdown_connections 2024-02-17T12:11:04.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.898+0000 7f4373a40640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43580a22f0 0x7f43580a47b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.898+0000 7f4373a40640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f436c1142d0 0x7f436c112ec0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.898+0000 7f4373a40640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f436c108aa0 0x7f436c1ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.898+0000 7f4373a40640 1 --2- 172.21.15.94:0/352203758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f436c108180 0x7f436c1ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:04.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.898+0000 7f4373a40640 1 -- 172.21.15.94:0/352203758 >> 172.21.15.94:0/352203758 conn(0x7f436c06c6d0 msgr2=0x7f436c071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:04.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.898+0000 7f4373a40640 1 -- 172.21.15.94:0/352203758 shutdown_connections 2024-02-17T12:11:04.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:04.898+0000 7f4373a40640 1 -- 172.21.15.94:0/352203758 wait complete. 2024-02-17T12:11:04.912 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:11:04.912 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:11:05.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.084+0000 7fdca2b81640 1 -- 172.21.15.94:0/1889110592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c10f9a0 msgr2=0x7fdc9c10d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.084+0000 7fdca2b81640 1 --2- 172.21.15.94:0/1889110592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c10f9a0 0x7fdc9c10d490 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009900 tx=0x7fdc8c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:11:05.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.088+0000 7fdca2b81640 1 -- 172.21.15.94:0/1889110592 shutdown_connections 2024-02-17T12:11:05.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.088+0000 7fdca2b81640 1 --2- 172.21.15.94:0/1889110592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc9c11a960 0x7fdc9c11cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.088+0000 7fdca2b81640 1 --2- 172.21.15.94:0/1889110592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc9c10d9d0 0x7fdc9c10de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.088+0000 7fdca2b81640 1 --2- 172.21.15.94:0/1889110592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c10f9a0 0x7fdc9c10d490 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.088+0000 7fdca2b81640 1 -- 172.21.15.94:0/1889110592 >> 172.21.15.94:0/1889110592 conn(0x7fdc9c06c260 msgr2=0x7fdc9c06c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.092+0000 7fdca2b81640 1 -- 172.21.15.94:0/1889110592 shutdown_connections 2024-02-17T12:11:05.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.092+0000 7fdca2b81640 1 -- 172.21.15.94:0/1889110592 wait complete. 2024-02-17T12:11:05.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.093+0000 7fdca2b81640 1 Processor -- start 2024-02-17T12:11:05.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.093+0000 7fdca2b81640 1 -- start start 2024-02-17T12:11:05.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.093+0000 7fdca2b81640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc9c10d9d0 0x7fdc9c119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.093+0000 7fdca2b81640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc9c10f9a0 0x7fdc9c113fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.093+0000 7fdca2b81640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c11a960 0x7fdc9c1144f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.093+0000 7fdca2b81640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c114a30 con 0x7fdc9c10d9d0 2024-02-17T12:11:05.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.093+0000 7fdca2b81640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c114ba0 con 0x7fdc9c10f9a0 2024-02-17T12:11:05.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.093+0000 7fdca2b81640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9c114d60 con 0x7fdc9c11a960 2024-02-17T12:11:05.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.094+0000 7fdc9bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc9c10f9a0 0x7fdc9c113fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.094+0000 7fdc9bfff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc9c10f9a0 0x7fdc9c113fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45928/0 (socket says 172.21.15.94:45928) 2024-02-17T12:11:05.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.094+0000 7fdc9bfff640 1 -- 172.21.15.94:0/4239469326 learned_addr learned my addr 172.21.15.94:0/4239469326 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.094+0000 7fdca111e640 1 --2- 172.21.15.94:0/4239469326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c11a960 0x7fdc9c1144f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.094+0000 7fdc9bfff640 1 -- 172.21.15.94:0/4239469326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c11a960 msgr2=0x7fdc9c1144f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.094+0000 7fdc9bfff640 1 --2- 172.21.15.94:0/4239469326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c11a960 0x7fdc9c1144f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.094+0000 7fdc9bfff640 1 -- 172.21.15.94:0/4239469326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc9c10d9d0 msgr2=0x7fdc9c119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.094+0000 7fdc9bfff640 1 --2- 172.21.15.94:0/4239469326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc9c10d9d0 0x7fdc9c119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.094+0000 7fdc9bfff640 1 -- 172.21.15.94:0/4239469326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c009560 con 0x7fdc9c10f9a0 2024-02-17T12:11:05.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.095+0000 7fdc9bfff640 1 --2- 172.21.15.94:0/4239469326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc9c10f9a0 0x7fdc9c113fb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000c920 tx=0x7fdc9000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.096+0000 7fdc99ffb640 1 -- 172.21.15.94:0/4239469326 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90007c70 con 0x7fdc9c10f9a0 2024-02-17T12:11:05.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.096+0000 7fdca2b81640 1 -- 172.21.15.94:0/4239469326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9c1bb4e0 con 0x7fdc9c10f9a0 2024-02-17T12:11:05.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.096+0000 7fdca2b81640 1 -- 172.21.15.94:0/4239469326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9c1bba10 con 0x7fdc9c10f9a0 2024-02-17T12:11:05.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.096+0000 7fdc99ffb640 1 -- 172.21.15.94:0/4239469326 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc90007dd0 con 0x7fdc9c10f9a0 2024-02-17T12:11:05.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.097+0000 7fdc99ffb640 1 -- 172.21.15.94:0/4239469326 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90007c70 con 0x7fdc9c10f9a0 2024-02-17T12:11:05.101 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.099+0000 7fdca5832740 1 Processor -- start 2024-02-17T12:11:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.100+0000 7fdca5832740 1 -- start start 2024-02-17T12:11:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.100+0000 7fdca5832740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac88b250 0x5610ac88b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.100+0000 7fdca5832740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88bb70 0x5610ac894210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.100+0000 7fdca5832740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac894750 0x5610ac896b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.104 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.100+0000 7fdca5832740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ac846960 con 0x5610ac88bb70 2024-02-17T12:11:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.100+0000 7fdca5832740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ac7fa310 con 0x5610ac894750 2024-02-17T12:11:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.100+0000 7fdca5832740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ac7f7840 con 0x5610ac88b250 2024-02-17T12:11:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.101+0000 7fdca091d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88bb70 0x5610ac894210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.101+0000 7fdca111e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac88b250 0x5610ac88b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.101+0000 7fdca091d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88bb70 0x5610ac894210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47990/0 (socket says 172.21.15.94:47990) 2024-02-17T12:11:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.101+0000 7fdca091d640 1 -- 172.21.15.94:0/1130496584 learned_addr learned my addr 172.21.15.94:0/1130496584 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.101+0000 7fdca091d640 1 -- 172.21.15.94:0/1130496584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac88b250 msgr2=0x5610ac88b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.101+0000 7fdca091d640 1 --2- 172.21.15.94:0/1130496584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac88b250 0x5610ac88b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.101+0000 7fdca091d640 1 -- 172.21.15.94:0/1130496584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac894750 msgr2=0x5610ac896b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:11:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.101+0000 7fdca091d640 1 --2- 172.21.15.94:0/1130496584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac894750 0x5610ac896b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.101+0000 7fdca091d640 1 -- 172.21.15.94:0/1130496584 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610ac7b5060 con 0x5610ac88bb70 2024-02-17T12:11:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.102+0000 7fdca091d640 1 --2- 172.21.15.94:0/1130496584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88bb70 0x5610ac894210 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009230 tx=0x7fdc8c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5467987f6928072 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.105 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.102+0000 7fdc99ffb640 1 -- 172.21.15.94:0/4239469326 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fdc9000f450 con 0x7fdc9c10f9a0 2024-02-17T12:11:05.106 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.102+0000 7fdc99ffb640 1 --2- 172.21.15.94:0/4239469326 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdc840a22c0 0x7fdc840a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.103+0000 7fdc99ffb640 1 -- 172.21.15.94:0/4239469326 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fdc9009b330 con 0x7fdc9c10f9a0 2024-02-17T12:11:05.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.103+0000 7fdc82ffd640 1 -- 172.21.15.94:0/1130496584 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c004900 con 0x5610ac88bb70 2024-02-17T12:11:05.107 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.103+0000 7fdc82ffd640 1 -- 172.21.15.94:0/1130496584 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc8c004a60 con 0x5610ac88bb70 2024-02-17T12:11:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.103+0000 7fdc82ffd640 1 -- 172.21.15.94:0/1130496584 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c005150 con 0x5610ac88bb70 2024-02-17T12:11:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.104+0000 7fdca111e640 1 --2- 172.21.15.94:0/4239469326 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdc840a22c0 0x7fdc840a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.104+0000 7fdca5832740 1 -- 172.21.15.94:0/1130496584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88bb70 msgr2=0x5610ac894210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.104+0000 7fdca5832740 1 --2- 172.21.15.94:0/1130496584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88bb70 0x5610ac894210 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009230 tx=0x7fdc8c004790 comp rx=0 tx=0).stop 2024-02-17T12:11:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.104+0000 7fdca5832740 1 -- 172.21.15.94:0/1130496584 shutdown_connections 2024-02-17T12:11:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.104+0000 7fdca5832740 1 --2- 172.21.15.94:0/1130496584 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac894750 0x5610ac896b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.104+0000 7fdca5832740 1 --2- 172.21.15.94:0/1130496584 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88bb70 0x5610ac894210 secure :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009230 tx=0x7fdc8c004790 comp rx=0 tx=0).stop 2024-02-17T12:11:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.104+0000 7fdca5832740 1 --2- 172.21.15.94:0/1130496584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac88b250 0x5610ac88b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.105+0000 7fdca5832740 1 -- 172.21.15.94:0/1130496584 >> 172.21.15.94:0/1130496584 conn(0x5610ac885980 msgr2=0x5610ac885d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.105+0000 7fdca5832740 1 -- 172.21.15.94:0/1130496584 shutdown_connections 2024-02-17T12:11:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.105+0000 7fdca5832740 1 -- 172.21.15.94:0/1130496584 wait complete. 2024-02-17T12:11:05.108 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.106+0000 7fdca5832740 1 -- start start 2024-02-17T12:11:05.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.106+0000 7fdca5832740 1 Processor -- start 2024-02-17T12:11:05.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.106+0000 7fdca5832740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88b250 0x5610ac8917d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.106+0000 7fdca5832740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac891d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.106+0000 7fdca5832740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac892250 0x5610ac8a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.106+0000 7fdca5832740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ac7f3020 con 0x5610ac88b250 2024-02-17T12:11:05.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.106+0000 7fdca5832740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ac4c6430 con 0x5610ac892250 2024-02-17T12:11:05.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.106+0000 7fdca5832740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ac482e00 con 0x5610ac894750 2024-02-17T12:11:05.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.107+0000 7fdca5832740 10 client.0 _ll_register_callbacks cb 0x5610ac79acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:05.111 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.107+0000 7fdca091d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88b250 0x5610ac8917d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.107+0000 7fdca091d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88b250 0x5610ac8917d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48004/0 (socket says 172.21.15.94:48004) 2024-02-17T12:11:05.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.107+0000 7fdca091d640 1 -- 172.21.15.94:0/3227309256 learned_addr learned my addr 172.21.15.94:0/3227309256 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.107+0000 7fdc9bfff640 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac891d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.107+0000 7fdca111e640 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac892250 0x5610ac8a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.107+0000 7fdc9bfff640 1 -- 172.21.15.94:0/3227309256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac892250 msgr2=0x5610ac8a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.107+0000 7fdc9bfff640 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac892250 0x5610ac8a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.107+0000 7fdc9bfff640 1 -- 172.21.15.94:0/3227309256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88b250 msgr2=0x5610ac8917d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.107+0000 7fdc9bfff640 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88b250 0x5610ac8917d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.108+0000 7fdc9bfff640 1 -- 172.21.15.94:0/3227309256 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9c11d680 con 0x5610ac894750 2024-02-17T12:11:05.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.108+0000 7fdca111e640 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac892250 0x5610ac8a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:11:05.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.108+0000 7fdca111e640 1 --2- 172.21.15.94:0/4239469326 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdc840a22c0 0x7fdc840a4780 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400ad00 tx=0x7fdc9400a310 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.108+0000 7fdc7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:11:05.112 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.108+0000 7fdc9bfff640 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac891d10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdc900cb990 tx=0x7fdc9009ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772f9d6dd1548e4c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.108+0000 7fdc7bfff640 1 -- 172.21.15.94:0/3227309256 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc900a3140 con 0x5610ac894750 2024-02-17T12:11:05.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.108+0000 7fdca5832740 1 -- 172.21.15.94:0/3227309256 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610ac4f27e0 con 0x5610ac894750 2024-02-17T12:11:05.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.108+0000 7fdc7bfff640 1 -- 172.21.15.94:0/3227309256 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fdc9009b330 con 0x5610ac894750 2024-02-17T12:11:05.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.110+0000 7fdc7bfff640 1 -- 172.21.15.94:0/3227309256 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc900a3140 con 0x5610ac894750 2024-02-17T12:11:05.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.110+0000 7fdca5832740 10 client.28291 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:05.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.110+0000 7fdca5832740 1 -- 172.21.15.94:0/3227309256 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5610ac7f16a0 con 0x5610ac894750 2024-02-17T12:11:05.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.110+0000 7fdc7bfff640 1 -- 172.21.15.94:0/3227309256 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5610ac4f27e0 con 0x5610ac894750 2024-02-17T12:11:05.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.111+0000 7fdc7bfff640 1 -- 172.21.15.94:0/3227309256 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdc90016020 con 0x5610ac894750 2024-02-17T12:11:05.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.111+0000 7fdca5832740 10 client.28291 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:05.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.111+0000 7fdca5832740 10 client.28291 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:11:05.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.111+0000 7fdca5832740 20 client.28291 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:05.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.111+0000 7fdca5832740 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5610ac8a5040 0x5610ac8a7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.111+0000 7fdca5832740 4 client.28291 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:05.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.111+0000 7fdca5832740 1 -- 172.21.15.94:0/3227309256 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610ac52f720 con 0x5610ac8a5040 2024-02-17T12:11:05.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.112+0000 7fdca091d640 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5610ac8a5040 0x5610ac8a7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.113+0000 7fdc7bfff640 10 client.28291 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:11:05.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.113+0000 7fdca091d640 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5610ac8a5040 0x5610ac8a7480 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e74125e52f5645c server_cookie=65f029dadcd89977 in_seq=0 out_seq=0 2024-02-17T12:11:05.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.113+0000 7fdc7bfff640 1 -- 172.21.15.94:0/3227309256 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5610ac52f720 con 0x5610ac8a5040 2024-02-17T12:11:05.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.113+0000 7fdc7bfff640 10 client.28291 handle_command_reply: tid=0 2024-02-17T12:11:05.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.114+0000 7fdca5832740 1 client.28291 shutdown 2024-02-17T12:11:05.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.114+0000 7fdca5832740 1 -- 172.21.15.94:0/3227309256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 msgr2=0x5610ac891d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.114+0000 7fdca5832740 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac891d10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fdc900cb990 tx=0x7fdc9009ed80 comp rx=0 tx=0).stop 2024-02-17T12:11:05.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.114+0000 7fdca5832740 1 -- 172.21.15.94:0/3227309256 shutdown_connections 2024-02-17T12:11:05.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.114+0000 7fdca5832740 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5610ac8a5040 0x5610ac8a7480 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.114+0000 7fdca5832740 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac892250 0x5610ac8a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.114+0000 7fdca5832740 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac891d10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.114+0000 7fdca5832740 1 --2- 172.21.15.94:0/3227309256 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88b250 0x5610ac8917d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.114+0000 7fdca5832740 1 -- 172.21.15.94:0/3227309256 >> 172.21.15.94:0/3227309256 conn(0x5610ac885980 msgr2=0x5610ac8876d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.114+0000 7fdc7bfff640 0 client.28291 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:11:05.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.114+0000 7fdca5832740 1 -- 172.21.15.94:0/3227309256 shutdown_connections 2024-02-17T12:11:05.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.114+0000 7fdca5832740 1 -- 172.21.15.94:0/3227309256 wait complete. 2024-02-17T12:11:05.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.114+0000 7fdca5832740 20 client.28291 trim_cache size 0 max 16384 2024-02-17T12:11:05.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.129+0000 7fdca5832740 1 Processor -- start 2024-02-17T12:11:05.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.129+0000 7fdca5832740 1 -- start start 2024-02-17T12:11:05.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.129+0000 7fdca5832740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac88b250 0x5610ac89b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.129+0000 7fdca5832740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac892250 0x5610ac89bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.129+0000 7fdca5832740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac8a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.129+0000 7fdca5832740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ac778130 con 0x5610ac892250 2024-02-17T12:11:05.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.129+0000 7fdca5832740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ac877a40 con 0x5610ac88b250 2024-02-17T12:11:05.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.129+0000 7fdca5832740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ac88aeb0 con 0x5610ac894750 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.129+0000 7fdca111e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac8a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.129+0000 7fdca091d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac88b250 0x5610ac89b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.129+0000 7fdca111e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac8a00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60032/0 (socket says 172.21.15.94:60032) 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.129+0000 7fdca111e640 1 -- 172.21.15.94:0/4003924320 learned_addr learned my addr 172.21.15.94:0/4003924320 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.129+0000 7fdc9bfff640 1 --2- 172.21.15.94:0/4003924320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac892250 0x5610ac89bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.130+0000 7fdca091d640 1 -- 172.21.15.94:0/4003924320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 msgr2=0x5610ac8a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.130+0000 7fdca091d640 1 --2- 172.21.15.94:0/4003924320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac8a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.130+0000 7fdca091d640 1 -- 172.21.15.94:0/4003924320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac892250 msgr2=0x5610ac89bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.130+0000 7fdca091d640 1 --2- 172.21.15.94:0/4003924320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac892250 0x5610ac89bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.130+0000 7fdca091d640 1 -- 172.21.15.94:0/4003924320 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610ac78dc20 con 0x5610ac88b250 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.130+0000 7fdca091d640 1 --2- 172.21.15.94:0/4003924320 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac88b250 0x5610ac89b630 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009850 tx=0x7fdc8c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ca4406634808e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.130+0000 7fdc827fc640 1 -- 172.21.15.94:0/4003924320 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c004160 con 0x5610ac88b250 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.130+0000 7fdc827fc640 1 -- 172.21.15.94:0/4003924320 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc8c051b70 con 0x5610ac88b250 2024-02-17T12:11:05.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.130+0000 7fdc827fc640 1 -- 172.21.15.94:0/4003924320 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c0516b0 con 0x5610ac88b250 2024-02-17T12:11:05.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.131+0000 7fdca5832740 1 -- 172.21.15.94:0/4003924320 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac88b250 msgr2=0x5610ac89b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.131+0000 7fdca5832740 1 --2- 172.21.15.94:0/4003924320 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac88b250 0x5610ac89b630 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009850 tx=0x7fdc8c005fd0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.132+0000 7fdca5832740 1 -- 172.21.15.94:0/4003924320 shutdown_connections 2024-02-17T12:11:05.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.132+0000 7fdca5832740 1 --2- 172.21.15.94:0/4003924320 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac8a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.132+0000 7fdca5832740 1 --2- 172.21.15.94:0/4003924320 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac892250 0x5610ac89bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.132+0000 7fdca5832740 1 --2- 172.21.15.94:0/4003924320 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac88b250 0x5610ac89b630 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.132+0000 7fdca5832740 1 -- 172.21.15.94:0/4003924320 >> 172.21.15.94:0/4003924320 conn(0x5610ac885980 msgr2=0x5610ac891790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.132+0000 7fdca5832740 1 -- 172.21.15.94:0/4003924320 shutdown_connections 2024-02-17T12:11:05.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.132+0000 7fdca5832740 1 -- 172.21.15.94:0/4003924320 wait complete. 2024-02-17T12:11:05.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.133+0000 7fdca5832740 1 -- start start 2024-02-17T12:11:05.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.133+0000 7fdca5832740 1 Processor -- start 2024-02-17T12:11:05.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.133+0000 7fdca5832740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88b250 0x5610ac89ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.133+0000 7fdca5832740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac892250 0x5610ac899e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.133+0000 7fdca5832740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac89a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.133+0000 7fdca5832740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ac89aa80 con 0x5610ac88b250 2024-02-17T12:11:05.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.133+0000 7fdca5832740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ac89abf0 con 0x5610ac892250 2024-02-17T12:11:05.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.133+0000 7fdca5832740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610ac8fb5b0 con 0x5610ac894750 2024-02-17T12:11:05.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.133+0000 7fdca5832740 10 client.0 _ll_register_callbacks cb 0x7fdc8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:05.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.133+0000 7fdca091d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88b250 0x5610ac89ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdca091d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88b250 0x5610ac89ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48026/0 (socket says 172.21.15.94:48026) 2024-02-17T12:11:05.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdca091d640 1 -- 172.21.15.94:0/3369170059 learned_addr learned my addr 172.21.15.94:0/3369170059 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdc9bfff640 1 --2- 172.21.15.94:0/3369170059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac892250 0x5610ac899e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdca111e640 1 --2- 172.21.15.94:0/3369170059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac89a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdca091d640 1 -- 172.21.15.94:0/3369170059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 msgr2=0x5610ac89a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdca091d640 1 --2- 172.21.15.94:0/3369170059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac89a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdca091d640 1 -- 172.21.15.94:0/3369170059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac892250 msgr2=0x5610ac899e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdca091d640 1 --2- 172.21.15.94:0/3369170059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac892250 0x5610ac899e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdca091d640 1 -- 172.21.15.94:0/3369170059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610ac7b5060 con 0x5610ac88b250 2024-02-17T12:11:05.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdca091d640 1 --2- 172.21.15.94:0/3369170059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88b250 0x5610ac89ba30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c053040 tx=0x7fdc8c02c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16eec4e14fc4a6dc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdc7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:05.142 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdc7bfff640 1 -- 172.21.15.94:0/3369170059 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c02c9e0 con 0x5610ac88b250 2024-02-17T12:11:05.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdc7bfff640 1 -- 172.21.15.94:0/3369170059 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fdc8c02cb40 con 0x5610ac88b250 2024-02-17T12:11:05.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdca5832740 1 -- 172.21.15.94:0/3369170059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc8c004a60 con 0x5610ac88b250 2024-02-17T12:11:05.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdc7bfff640 1 -- 172.21.15.94:0/3369170059 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc8c051310 con 0x5610ac88b250 2024-02-17T12:11:05.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdca5832740 10 client.18852 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:05.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.134+0000 7fdca5832740 1 -- 172.21.15.94:0/3369170059 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5610ac7b38c0 con 0x5610ac88b250 2024-02-17T12:11:05.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.135+0000 7fdc7bfff640 1 -- 172.21.15.94:0/3369170059 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdc8c066020 con 0x5610ac88b250 2024-02-17T12:11:05.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.135+0000 7fdc7bfff640 1 -- 172.21.15.94:0/3369170059 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdc8c03c070 con 0x5610ac88b250 2024-02-17T12:11:05.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.135+0000 7fdca5832740 10 client.18852 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:05.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.135+0000 7fdca5832740 10 client.18852 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:11:05.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.135+0000 7fdca5832740 20 client.18852 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:05.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.135+0000 7fdca5832740 1 --2- 172.21.15.94:0/3369170059 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5610ac8fc910 0x5610ac8fed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.135+0000 7fdca5832740 4 client.18852 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.135+0000 7fdca5832740 1 -- 172.21.15.94:0/3369170059 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fdc8c004a60 con 0x5610ac8fc910 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.135+0000 7fdc9bfff640 1 --2- 172.21.15.94:0/3369170059 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5610ac8fc910 0x5610ac8fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.135+0000 7fdc7bfff640 10 client.18852 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.135+0000 7fdc9bfff640 1 --2- 172.21.15.94:0/3369170059 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5610ac8fc910 0x5610ac8fed30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bd1dca1ae90b0bfb server_cookie=5f7e6b89cb941c71 in_seq=0 out_seq=0 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdc7bfff640 1 -- 172.21.15.94:0/3369170059 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fdc8c004a60 con 0x5610ac8fc910 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdc7bfff640 10 client.18852 handle_command_reply: tid=0 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdca5832740 1 client.18852 shutdown 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdca5832740 1 -- 172.21.15.94:0/3369170059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88b250 msgr2=0x5610ac89ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdca5832740 1 --2- 172.21.15.94:0/3369170059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88b250 0x5610ac89ba30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c053040 tx=0x7fdc8c02c3b0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdca5832740 1 -- 172.21.15.94:0/3369170059 shutdown_connections 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdca5832740 1 --2- 172.21.15.94:0/3369170059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5610ac894750 0x5610ac89a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdca5832740 1 --2- 172.21.15.94:0/3369170059 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5610ac8fc910 0x5610ac8fed30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdca5832740 1 --2- 172.21.15.94:0/3369170059 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5610ac892250 0x5610ac899e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdca5832740 1 --2- 172.21.15.94:0/3369170059 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5610ac88b250 0x5610ac89ba30 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdc7bfff640 0 client.18852 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:11:05.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdca5832740 1 -- 172.21.15.94:0/3369170059 >> 172.21.15.94:0/3369170059 conn(0x5610ac885980 msgr2=0x5610ac8a00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdca5832740 1 -- 172.21.15.94:0/3369170059 shutdown_connections 2024-02-17T12:11:05.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdca5832740 1 -- 172.21.15.94:0/3369170059 wait complete. 2024-02-17T12:11:05.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.136+0000 7fdca5832740 20 client.18852 trim_cache size 0 max 16384 2024-02-17T12:11:05.145 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:11:05.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.137+0000 7fdca2b81640 1 -- 172.21.15.94:0/4239469326 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdc840a22c0 msgr2=0x7fdc840a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.137+0000 7fdca2b81640 1 --2- 172.21.15.94:0/4239469326 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdc840a22c0 0x7fdc840a4780 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400ad00 tx=0x7fdc9400a310 comp rx=0 tx=0).stop 2024-02-17T12:11:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.137+0000 7fdca2b81640 1 -- 172.21.15.94:0/4239469326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc9c10f9a0 msgr2=0x7fdc9c113fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.137+0000 7fdca2b81640 1 --2- 172.21.15.94:0/4239469326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc9c10f9a0 0x7fdc9c113fb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdc9000c920 tx=0x7fdc9000cde0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.137+0000 7fdca2b81640 1 -- 172.21.15.94:0/4239469326 shutdown_connections 2024-02-17T12:11:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.137+0000 7fdca2b81640 1 --2- 172.21.15.94:0/4239469326 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdc840a22c0 0x7fdc840a4780 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.137+0000 7fdca2b81640 1 --2- 172.21.15.94:0/4239469326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdc9c11a960 0x7fdc9c1144f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.137+0000 7fdca2b81640 1 --2- 172.21.15.94:0/4239469326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdc9c10f9a0 0x7fdc9c113fb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.137+0000 7fdca2b81640 1 --2- 172.21.15.94:0/4239469326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdc9c10d9d0 0x7fdc9c119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.137+0000 7fdca2b81640 1 -- 172.21.15.94:0/4239469326 >> 172.21.15.94:0/4239469326 conn(0x7fdc9c06c260 msgr2=0x7fdc9c11bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.137+0000 7fdca2b81640 1 -- 172.21.15.94:0/4239469326 shutdown_connections 2024-02-17T12:11:05.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.137+0000 7fdca2b81640 1 -- 172.21.15.94:0/4239469326 wait complete. 2024-02-17T12:11:05.152 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:11:05.152 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:11:05.179 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:04 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/352203758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:11:05.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 765.0 s, 651.4 tps, lat 153.308 ms stddev 209.734 2024-02-17T12:11:05.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.294+0000 7ff00aaae640 1 -- 172.21.15.94:0/2129851992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0040ff890 msgr2=0x7ff0040ffc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.294+0000 7ff00aaae640 1 --2- 172.21.15.94:0/2129851992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0040ff890 0x7ff0040ffc70 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7feff8009930 tx=0x7feff802bc30 comp rx=0 tx=0).stop 2024-02-17T12:11:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.294+0000 7ff00aaae640 1 -- 172.21.15.94:0/2129851992 shutdown_connections 2024-02-17T12:11:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.294+0000 7ff00aaae640 1 --2- 172.21.15.94:0/2129851992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff004106450 0x7ff00410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.294+0000 7ff00aaae640 1 --2- 172.21.15.94:0/2129851992 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff004100240 0x7ff004105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.294+0000 7ff00aaae640 1 --2- 172.21.15.94:0/2129851992 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff0040ff890 0x7ff0040ffc70 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.294+0000 7ff00aaae640 1 -- 172.21.15.94:0/2129851992 >> 172.21.15.94:0/2129851992 conn(0x7ff0040765a0 msgr2=0x7ff0040fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.294+0000 7ff00aaae640 1 -- 172.21.15.94:0/2129851992 shutdown_connections 2024-02-17T12:11:05.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.294+0000 7ff00aaae640 1 -- 172.21.15.94:0/2129851992 wait complete. 2024-02-17T12:11:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.295+0000 7ff00aaae640 1 Processor -- start 2024-02-17T12:11:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.296+0000 7ff00aaae640 1 -- start start 2024-02-17T12:11:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.296+0000 7ff00aaae640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0040ff890 0x7ff004102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.296+0000 7ff00aaae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff004106450 0x7ff0041007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.296+0000 7ff00aaae640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff004100cf0 0x7ff0041011a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.296+0000 7ff00aaae640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff004073210 con 0x7ff004100cf0 2024-02-17T12:11:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.296+0000 7ff00aaae640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0041ae940 con 0x7ff0040ff890 2024-02-17T12:11:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.296+0000 7ff00aaae640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0041aead0 con 0x7ff004106450 2024-02-17T12:11:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff00884a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0040ff890 0x7ff004102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff003fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff004106450 0x7ff0041007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff00904b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff004100cf0 0x7ff0041011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff00884a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0040ff890 0x7ff004102220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45966/0 (socket says 172.21.15.94:45966) 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff003fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff004106450 0x7ff0041007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60066/0 (socket says 172.21.15.94:60066) 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff00884a640 1 -- 172.21.15.94:0/2285822026 learned_addr learned my addr 172.21.15.94:0/2285822026 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff00884a640 1 -- 172.21.15.94:0/2285822026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff004106450 msgr2=0x7ff0041007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff00884a640 1 --2- 172.21.15.94:0/2285822026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff004106450 0x7ff0041007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff00884a640 1 -- 172.21.15.94:0/2285822026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff004100cf0 msgr2=0x7ff0041011a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff00884a640 1 --2- 172.21.15.94:0/2285822026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff004100cf0 0x7ff0041011a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff00884a640 1 -- 172.21.15.94:0/2285822026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff8009560 con 0x7ff0040ff890 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff00904b640 1 --2- 172.21.15.94:0/2285822026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff004100cf0 0x7ff0041011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff00884a640 1 --2- 172.21.15.94:0/2285822026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0040ff890 0x7ff004102220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7feff8009430 tx=0x7feff802c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff001ffb640 1 -- 172.21.15.94:0/2285822026 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8004350 con 0x7ff0040ff890 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff001ffb640 1 -- 172.21.15.94:0/2285822026 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feff80044b0 con 0x7ff0040ff890 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff00aaae640 1 -- 172.21.15.94:0/2285822026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0041aed20 con 0x7ff0040ff890 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.297+0000 7ff001ffb640 1 -- 172.21.15.94:0/2285822026 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8003d30 con 0x7ff0040ff890 2024-02-17T12:11:05.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.298+0000 7ff00aaae640 1 -- 172.21.15.94:0/2285822026 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0041af1d0 con 0x7ff0040ff890 2024-02-17T12:11:05.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.300+0000 7ff001ffb640 1 -- 172.21.15.94:0/2285822026 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7feff803e450 con 0x7ff0040ff890 2024-02-17T12:11:05.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.300+0000 7ff001ffb640 1 --2- 172.21.15.94:0/2285822026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feff40a2230 0x7feff40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.300+0000 7ff00d684740 1 Processor -- start 2024-02-17T12:11:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.301+0000 7ff001ffb640 1 -- 172.21.15.94:0/2285822026 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7feff80efff0 con 0x7ff0040ff890 2024-02-17T12:11:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.301+0000 7ff00d684740 1 -- start start 2024-02-17T12:11:05.303 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.301+0000 7ff003fff640 1 --2- 172.21.15.94:0/2285822026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feff40a2230 0x7feff40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.301+0000 7ff00d684740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a8a60 0x55a43c1a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.301+0000 7ff00d684740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a9380 0x55a43c1b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.301+0000 7ff00d684740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.301+0000 7ff00d684740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c15b960 con 0x55a43c1a9380 2024-02-17T12:11:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.301+0000 7ff00d684740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c10f310 con 0x55a43c1b1f60 2024-02-17T12:11:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.301+0000 7ff00d684740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c10c840 con 0x55a43c1a8a60 2024-02-17T12:11:05.304 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.301+0000 7ff00904b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.301+0000 7ff00904b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45974/0 (socket says 172.21.15.94:45974) 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.301+0000 7ff00904b640 1 -- 172.21.15.94:0/3938294308 learned_addr learned my addr 172.21.15.94:0/3938294308 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.302+0000 7ff003fff640 1 --2- 172.21.15.94:0/3938294308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a9380 0x55a43c1b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.302+0000 7ff00884a640 1 --2- 172.21.15.94:0/3938294308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a8a60 0x55a43c1a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.302+0000 7ff003fff640 1 --2- 172.21.15.94:0/2285822026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feff40a2230 0x7feff40a46f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff004101f70 tx=0x7feff0011440 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.302+0000 7ff00904b640 1 -- 172.21.15.94:0/3938294308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a8a60 msgr2=0x55a43c1a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.302+0000 7ff00904b640 1 --2- 172.21.15.94:0/3938294308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a8a60 0x55a43c1a8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.302+0000 7ff00904b640 1 -- 172.21.15.94:0/3938294308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a9380 msgr2=0x55a43c1b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.302+0000 7ff00904b640 1 --2- 172.21.15.94:0/3938294308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a9380 0x55a43c1b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.302+0000 7ff00904b640 1 -- 172.21.15.94:0/3938294308 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a43c0ca060 con 0x55a43c1b1f60 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.302+0000 7ff00904b640 1 --2- 172.21.15.94:0/3938294308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b4350 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7feffc011790 tx=0x7feffc011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751bcd0d93f8eec5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.302+0000 7fefeeffd640 1 -- 172.21.15.94:0/3938294308 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc011ea0 con 0x55a43c1b1f60 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.302+0000 7fefeeffd640 1 -- 172.21.15.94:0/3938294308 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feffc00fe80 con 0x55a43c1b1f60 2024-02-17T12:11:05.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.302+0000 7fefeeffd640 1 -- 172.21.15.94:0/3938294308 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc004800 con 0x55a43c1b1f60 2024-02-17T12:11:05.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.303+0000 7ff00d684740 1 -- 172.21.15.94:0/3938294308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 msgr2=0x55a43c1b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.303+0000 7ff00d684740 1 --2- 172.21.15.94:0/3938294308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b4350 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7feffc011790 tx=0x7feffc011c50 comp rx=0 tx=0).stop 2024-02-17T12:11:05.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.303+0000 7ff00d684740 1 -- 172.21.15.94:0/3938294308 shutdown_connections 2024-02-17T12:11:05.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.303+0000 7ff00d684740 1 --2- 172.21.15.94:0/3938294308 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b4350 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.303+0000 7ff00d684740 1 --2- 172.21.15.94:0/3938294308 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a9380 0x55a43c1b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.303+0000 7ff00d684740 1 --2- 172.21.15.94:0/3938294308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a8a60 0x55a43c1a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.303+0000 7ff00d684740 1 -- 172.21.15.94:0/3938294308 >> 172.21.15.94:0/3938294308 conn(0x55a43c1a3190 msgr2=0x55a43c1a3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.303+0000 7ff00d684740 1 -- 172.21.15.94:0/3938294308 shutdown_connections 2024-02-17T12:11:05.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.303+0000 7ff00d684740 1 -- 172.21.15.94:0/3938294308 wait complete. 2024-02-17T12:11:05.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.303+0000 7ff00d684740 1 -- start start 2024-02-17T12:11:05.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.303+0000 7ff00d684740 1 Processor -- start 2024-02-17T12:11:05.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00d684740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1a8a60 0x55a43c1aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00d684740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00d684740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1b1f60 0x55a43c1af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00d684740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c108020 con 0x55a43c1b1f60 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00d684740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43bddb430 con 0x55a43c1a8a60 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00d684740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43bd97e00 con 0x55a43c1a9380 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00d684740 10 client.0 _ll_register_callbacks cb 0x55a43c0afcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff003fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00884a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1a8a60 0x55a43c1aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00884a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1a8a60 0x55a43c1aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:45978/0 (socket says 172.21.15.94:45978) 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff003fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1af430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60080/0 (socket says 172.21.15.94:60080) 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00884a640 1 -- 172.21.15.94:0/346014583 learned_addr learned my addr 172.21.15.94:0/346014583 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00904b640 1 --2- 172.21.15.94:0/346014583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1b1f60 0x55a43c1af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00884a640 1 -- 172.21.15.94:0/346014583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 msgr2=0x55a43c1af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00884a640 1 --2- 172.21.15.94:0/346014583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00884a640 1 -- 172.21.15.94:0/346014583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1b1f60 msgr2=0x55a43c1af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00884a640 1 --2- 172.21.15.94:0/346014583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1b1f60 0x55a43c1af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00884a640 1 -- 172.21.15.94:0/346014583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feffc011440 con 0x55a43c1a8a60 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff003fff640 1 --2- 172.21.15.94:0/346014583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7ff00884a640 1 --2- 172.21.15.94:0/346014583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1a8a60 0x55a43c1aeef0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feff80edcd0 tx=0x7feff80b8e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=854e1680fe18d892 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.304+0000 7fefe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.305+0000 7fefe3fff640 1 -- 172.21.15.94:0/346014583 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff80bca80 con 0x55a43c1a8a60 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.305+0000 7fefe3fff640 1 -- 172.21.15.94:0/346014583 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7feff80c0bd0 con 0x55a43c1a8a60 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.305+0000 7ff00d684740 1 -- 172.21.15.94:0/346014583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a43be077e0 con 0x55a43c1a8a60 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.305+0000 7fefe3fff640 1 -- 172.21.15.94:0/346014583 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feff80c5220 con 0x55a43c1a8a60 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.305+0000 7ff00d684740 10 client.28331 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.305+0000 7ff00d684740 1 -- 172.21.15.94:0/346014583 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a43c1066a0 con 0x55a43c1a8a60 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.305+0000 7fefe3fff640 1 -- 172.21.15.94:0/346014583 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55a43be077e0 con 0x55a43c1a8a60 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.306+0000 7fefe3fff640 1 -- 172.21.15.94:0/346014583 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feff803c070 con 0x55a43c1a8a60 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.306+0000 7ff00d684740 10 client.28331 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.306+0000 7ff00d684740 10 client.28331 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:11:05.311 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.306+0000 7ff00d684740 20 client.28331 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:05.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.306+0000 7ff00d684740 1 --2- 172.21.15.94:0/346014583 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55a43c1c2190 0x55a43c1c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.306+0000 7ff00d684740 4 client.28331 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:05.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.306+0000 7ff00d684740 1 -- 172.21.15.94:0/346014583 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a43be44720 con 0x55a43c1c2190 2024-02-17T12:11:05.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.306+0000 7ff003fff640 1 --2- 172.21.15.94:0/346014583 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55a43c1c2190 0x55a43c1c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.307+0000 7ff003fff640 1 --2- 172.21.15.94:0/346014583 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55a43c1c2190 0x55a43c1c45a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9946489b45acb9 server_cookie=57c4d67718a64406 in_seq=0 out_seq=0 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.307+0000 7fefe3fff640 10 client.28331 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.307+0000 7fefe3fff640 1 -- 172.21.15.94:0/346014583 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a43be44720 con 0x55a43c1c2190 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.307+0000 7fefe3fff640 10 client.28331 handle_command_reply: tid=0 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.308+0000 7ff00d684740 1 client.28331 shutdown 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.308+0000 7ff00d684740 1 -- 172.21.15.94:0/346014583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1a8a60 msgr2=0x55a43c1aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.308+0000 7ff00d684740 1 --2- 172.21.15.94:0/346014583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1a8a60 0x55a43c1aeef0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feff80edcd0 tx=0x7feff80b8e50 comp rx=0 tx=0).stop 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.308+0000 7ff00d684740 1 -- 172.21.15.94:0/346014583 shutdown_connections 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.308+0000 7ff00d684740 1 --2- 172.21.15.94:0/346014583 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55a43c1c2190 0x55a43c1c45a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.308+0000 7ff00d684740 1 --2- 172.21.15.94:0/346014583 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1b1f60 0x55a43c1af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.308+0000 7fefe3fff640 0 client.28331 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.308+0000 7ff00d684740 1 --2- 172.21.15.94:0/346014583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.308+0000 7ff00d684740 1 --2- 172.21.15.94:0/346014583 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1a8a60 0x55a43c1aeef0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.308+0000 7ff00d684740 1 -- 172.21.15.94:0/346014583 >> 172.21.15.94:0/346014583 conn(0x55a43c1a3190 msgr2=0x55a43c1a4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.308+0000 7ff00d684740 1 -- 172.21.15.94:0/346014583 shutdown_connections 2024-02-17T12:11:05.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.308+0000 7ff00d684740 1 -- 172.21.15.94:0/346014583 wait complete. 2024-02-17T12:11:05.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.309+0000 7ff00d684740 20 client.28331 trim_cache size 0 max 16384 2024-02-17T12:11:05.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.323+0000 7ff00d684740 1 Processor -- start 2024-02-17T12:11:05.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.323+0000 7ff00d684740 1 -- start start 2024-02-17T12:11:05.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.323+0000 7ff00d684740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 0x55a43c1af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.323+0000 7ff00d684740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.323+0000 7ff00d684740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.323+0000 7ff00d684740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c08d130 con 0x55a43c1a8a60 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.323+0000 7ff00d684740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c1a7de0 con 0x55a43c1b1f60 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.323+0000 7ff00d684740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c1a86c0 con 0x55a43c1a9380 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.324+0000 7ff00884a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 0x55a43c1af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.324+0000 7ff00884a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 0x55a43c1af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48056/0 (socket says 172.21.15.94:48056) 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.324+0000 7ff00884a640 1 -- 172.21.15.94:0/4114239043 learned_addr learned my addr 172.21.15.94:0/4114239043 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.324+0000 7ff003fff640 1 --2- 172.21.15.94:0/4114239043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.324+0000 7ff00904b640 1 --2- 172.21.15.94:0/4114239043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.324+0000 7ff00884a640 1 -- 172.21.15.94:0/4114239043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 msgr2=0x55a43c1b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.324+0000 7ff00884a640 1 --2- 172.21.15.94:0/4114239043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.324+0000 7ff00884a640 1 -- 172.21.15.94:0/4114239043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 msgr2=0x55a43c1b9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.324+0000 7ff00884a640 1 --2- 172.21.15.94:0/4114239043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.324+0000 7ff00884a640 1 -- 172.21.15.94:0/4114239043 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a43c0a2c20 con 0x55a43c1a8a60 2024-02-17T12:11:05.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.324+0000 7ff003fff640 1 --2- 172.21.15.94:0/4114239043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:11:05.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.324+0000 7ff00884a640 1 --2- 172.21.15.94:0/4114239043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 0x55a43c1af790 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7feff80bad30 tx=0x7feff80057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679a4f31e924f89 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.325+0000 7fefee7fc640 1 -- 172.21.15.94:0/4114239043 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff80bbed0 con 0x55a43c1a8a60 2024-02-17T12:11:05.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.325+0000 7fefee7fc640 1 -- 172.21.15.94:0/4114239043 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feff80bc850 con 0x55a43c1a8a60 2024-02-17T12:11:05.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.325+0000 7fefee7fc640 1 -- 172.21.15.94:0/4114239043 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff80bcb10 con 0x55a43c1a8a60 2024-02-17T12:11:05.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.326+0000 7ff00d684740 1 -- 172.21.15.94:0/4114239043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 msgr2=0x55a43c1af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.326+0000 7ff00d684740 1 --2- 172.21.15.94:0/4114239043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 0x55a43c1af790 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7feff80bad30 tx=0x7feff80057e0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.326+0000 7ff00d684740 1 -- 172.21.15.94:0/4114239043 shutdown_connections 2024-02-17T12:11:05.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.326+0000 7ff00d684740 1 --2- 172.21.15.94:0/4114239043 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.326+0000 7ff00d684740 1 --2- 172.21.15.94:0/4114239043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.326+0000 7ff00d684740 1 --2- 172.21.15.94:0/4114239043 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 0x55a43c1af790 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.326+0000 7ff00d684740 1 -- 172.21.15.94:0/4114239043 >> 172.21.15.94:0/4114239043 conn(0x55a43c1a3190 msgr2=0x55a43c195a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.326+0000 7ff00d684740 1 -- 172.21.15.94:0/4114239043 shutdown_connections 2024-02-17T12:11:05.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.326+0000 7ff00d684740 1 -- 172.21.15.94:0/4114239043 wait complete. 2024-02-17T12:11:05.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.326+0000 7ff00d684740 1 -- start start 2024-02-17T12:11:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.326+0000 7ff00d684740 1 Processor -- start 2024-02-17T12:11:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff00d684740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 0x55a43c1b9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff00d684740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff00d684740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff00884a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 0x55a43c1b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff00d684740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c1b7ea0 con 0x55a43c1a8a60 2024-02-17T12:11:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff00d684740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c1b8010 con 0x55a43c1b1f60 2024-02-17T12:11:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff00d684740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43c1b81d0 con 0x55a43c1a9380 2024-02-17T12:11:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff00884a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 0x55a43c1b9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48066/0 (socket says 172.21.15.94:48066) 2024-02-17T12:11:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff00884a640 1 -- 172.21.15.94:0/776834215 learned_addr learned my addr 172.21.15.94:0/776834215 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff003fff640 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff00d684740 10 client.0 _ll_register_callbacks cb 0x7feffc009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff00904b640 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff003fff640 1 -- 172.21.15.94:0/776834215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 msgr2=0x55a43c1b7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff003fff640 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff003fff640 1 -- 172.21.15.94:0/776834215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 msgr2=0x55a43c1b9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff003fff640 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 0x55a43c1b9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff003fff640 1 -- 172.21.15.94:0/776834215 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a43c1066a0 con 0x55a43c1a9380 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff00884a640 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 0x55a43c1b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7ff003fff640 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1b7420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55a43c1c4ef0 tx=0x7feff0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1415ae58e6a6970 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7fefe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.327+0000 7fefe3fff640 1 -- 172.21.15.94:0/776834215 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0023040 con 0x55a43c1a9380 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7fefe3fff640 1 -- 172.21.15.94:0/776834215 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7feff00283d0 con 0x55a43c1a9380 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7ff00d684740 1 -- 172.21.15.94:0/776834215 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feffc00fe80 con 0x55a43c1a9380 2024-02-17T12:11:05.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7fefe3fff640 1 -- 172.21.15.94:0/776834215 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feff0028590 con 0x55a43c1a9380 2024-02-17T12:11:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7ff00d684740 10 client.28315 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7ff00d684740 1 -- 172.21.15.94:0/776834215 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a43c0c88c0 con 0x55a43c1a9380 2024-02-17T12:11:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7fefe3fff640 1 -- 172.21.15.94:0/776834215 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feff0034020 con 0x55a43c1a9380 2024-02-17T12:11:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7fefe3fff640 1 -- 172.21.15.94:0/776834215 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feff000e3b0 con 0x55a43c1a9380 2024-02-17T12:11:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7ff00d684740 10 client.28315 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7ff00d684740 10 client.28315 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:11:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7ff00d684740 20 client.28315 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7ff00d684740 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55a43c1c2190 0x55a43c21bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7ff00d684740 4 client.28315 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:11:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7ff00d684740 1 -- 172.21.15.94:0/776834215 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a43be93890 con 0x55a43c1c2190 2024-02-17T12:11:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.328+0000 7ff00884a640 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55a43c1c2190 0x55a43c21bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.329+0000 7fefe3fff640 10 client.28315 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:11:05.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.329+0000 7ff00884a640 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55a43c1c2190 0x55a43c21bd10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=26a02e1521e2bbfd server_cookie=98c3ab57493fb609 in_seq=0 out_seq=0 2024-02-17T12:11:05.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.329+0000 7fefe3fff640 1 -- 172.21.15.94:0/776834215 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a43be93890 con 0x55a43c1c2190 2024-02-17T12:11:05.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.329+0000 7fefe3fff640 10 client.28315 handle_command_reply: tid=0 2024-02-17T12:11:05.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.329+0000 7ff00d684740 1 client.28315 shutdown 2024-02-17T12:11:05.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.329+0000 7ff00d684740 1 -- 172.21.15.94:0/776834215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 msgr2=0x55a43c1b7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.329+0000 7ff00d684740 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1b7420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x55a43c1c4ef0 tx=0x7feff0004730 comp rx=0 tx=0).stop 2024-02-17T12:11:05.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.330+0000 7ff00d684740 1 -- 172.21.15.94:0/776834215 shutdown_connections 2024-02-17T12:11:05.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.330+0000 7ff00d684740 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55a43c1c2190 0x55a43c21bd10 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.330+0000 7fefe3fff640 0 client.28315 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:11:05.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.330+0000 7ff00d684740 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55a43c1b1f60 0x55a43c1b7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.330+0000 7ff00d684740 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a43c1a9380 0x55a43c1b7420 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.330+0000 7ff00d684740 1 --2- 172.21.15.94:0/776834215 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55a43c1a8a60 0x55a43c1b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.330+0000 7ff00d684740 1 -- 172.21.15.94:0/776834215 >> 172.21.15.94:0/776834215 conn(0x55a43c1a3190 msgr2=0x55a43c1aa200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.330+0000 7ff00d684740 1 -- 172.21.15.94:0/776834215 shutdown_connections 2024-02-17T12:11:05.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.330+0000 7ff00d684740 1 -- 172.21.15.94:0/776834215 wait complete. 2024-02-17T12:11:05.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.331+0000 7ff00d684740 20 client.28315 trim_cache size 0 max 16384 2024-02-17T12:11:05.336 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:11:05.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.332+0000 7ff00aaae640 1 -- 172.21.15.94:0/2285822026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feff40a2230 msgr2=0x7feff40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.332+0000 7ff00aaae640 1 --2- 172.21.15.94:0/2285822026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feff40a2230 0x7feff40a46f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff004101f70 tx=0x7feff0011440 comp rx=0 tx=0).stop 2024-02-17T12:11:05.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.332+0000 7ff00aaae640 1 -- 172.21.15.94:0/2285822026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0040ff890 msgr2=0x7ff004102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.332+0000 7ff00aaae640 1 --2- 172.21.15.94:0/2285822026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0040ff890 0x7ff004102220 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7feff8009430 tx=0x7feff802c650 comp rx=0 tx=0).stop 2024-02-17T12:11:05.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.332+0000 7ff00aaae640 1 -- 172.21.15.94:0/2285822026 shutdown_connections 2024-02-17T12:11:05.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.332+0000 7ff00aaae640 1 --2- 172.21.15.94:0/2285822026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7feff40a2230 0x7feff40a46f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.332+0000 7ff00aaae640 1 --2- 172.21.15.94:0/2285822026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff004100cf0 0x7ff0041011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.332+0000 7ff00aaae640 1 --2- 172.21.15.94:0/2285822026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff004106450 0x7ff0041007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.332+0000 7ff00aaae640 1 --2- 172.21.15.94:0/2285822026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff0040ff890 0x7ff004102220 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.332+0000 7ff00aaae640 1 -- 172.21.15.94:0/2285822026 >> 172.21.15.94:0/2285822026 conn(0x7ff0040765a0 msgr2=0x7ff004104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.332+0000 7ff00aaae640 1 -- 172.21.15.94:0/2285822026 shutdown_connections 2024-02-17T12:11:05.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.332+0000 7ff00aaae640 1 -- 172.21.15.94:0/2285822026 wait complete. 2024-02-17T12:11:05.346 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:11:05.346 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:11:05.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:04 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/352203758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:11:05.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:04 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/352203758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:11:05.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.541+0000 7f4a8078c640 1 -- 172.21.15.94:0/1484565693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a781057a0 msgr2=0x7f4a78105b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.541+0000 7f4a8078c640 1 --2- 172.21.15.94:0/1484565693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a781057a0 0x7f4a78105b80 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400aa40 tx=0x7f4a7402be40 comp rx=0 tx=0).stop 2024-02-17T12:11:05.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.541+0000 7f4a8078c640 1 -- 172.21.15.94:0/1484565693 shutdown_connections 2024-02-17T12:11:05.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.541+0000 7f4a8078c640 1 --2- 172.21.15.94:0/1484565693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a78100150 0x7f4a7810c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.541+0000 7f4a8078c640 1 --2- 172.21.15.94:0/1484565693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a780ff790 0x7f4a780ffc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.541+0000 7f4a8078c640 1 --2- 172.21.15.94:0/1484565693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a781057a0 0x7f4a78105b80 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.541+0000 7f4a8078c640 1 -- 172.21.15.94:0/1484565693 >> 172.21.15.94:0/1484565693 conn(0x7f4a780765a0 msgr2=0x7f4a780fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.541+0000 7f4a8078c640 1 -- 172.21.15.94:0/1484565693 shutdown_connections 2024-02-17T12:11:05.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.541+0000 7f4a8078c640 1 -- 172.21.15.94:0/1484565693 wait complete. 2024-02-17T12:11:05.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.542+0000 7f4a8078c640 1 Processor -- start 2024-02-17T12:11:05.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.542+0000 7f4a8078c640 1 -- start start 2024-02-17T12:11:05.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.542+0000 7f4a8078c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a780ff790 0x7f4a78195600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.543+0000 7f4a8078c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a78100150 0x7f4a78195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.543+0000 7f4a8078c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a781057a0 0x7f4a78199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.543+0000 7f4a7ed29640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a781057a0 0x7f4a78199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.543+0000 7f4a7ed29640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a781057a0 0x7f4a78199e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46014/0 (socket says 172.21.15.94:46014) 2024-02-17T12:11:05.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.543+0000 7f4a7ed29640 1 -- 172.21.15.94:0/3043827767 learned_addr learned my addr 172.21.15.94:0/3043827767 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.544+0000 7f4a8078c640 1 -- 172.21.15.94:0/3043827767 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7819a470 con 0x7f4a780ff790 2024-02-17T12:11:05.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.544+0000 7f4a8078c640 1 -- 172.21.15.94:0/3043827767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7819a5e0 con 0x7f4a781057a0 2024-02-17T12:11:05.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.544+0000 7f4a8078c640 1 -- 172.21.15.94:0/3043827767 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7819a7a0 con 0x7f4a78100150 2024-02-17T12:11:05.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.544+0000 7f4a7e528640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a780ff790 0x7f4a78195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.545+0000 7f4a7ed29640 1 -- 172.21.15.94:0/3043827767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a78100150 msgr2=0x7f4a78195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.545+0000 7f4a7ed29640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a78100150 0x7f4a78195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.545+0000 7f4a7ed29640 1 -- 172.21.15.94:0/3043827767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a780ff790 msgr2=0x7f4a78195600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.545+0000 7f4a7ed29640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a780ff790 0x7f4a78195600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.545+0000 7f4a7ed29640 1 -- 172.21.15.94:0/3043827767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a68009490 con 0x7f4a781057a0 2024-02-17T12:11:05.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.545+0000 7f4a7e528640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a780ff790 0x7f4a78195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:11:05.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.546+0000 7f4a7ed29640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a781057a0 0x7f4a78199e70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000da90 tx=0x7f4a7000df50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.546+0000 7f4a6f7fe640 1 -- 172.21.15.94:0/3043827767 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7000bc30 con 0x7f4a781057a0 2024-02-17T12:11:05.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.546+0000 7f4a6f7fe640 1 -- 172.21.15.94:0/3043827767 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a7000bd90 con 0x7f4a781057a0 2024-02-17T12:11:05.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.546+0000 7f4a6f7fe640 1 -- 172.21.15.94:0/3043827767 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a700173f0 con 0x7f4a781057a0 2024-02-17T12:11:05.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.546+0000 7f4a8078c640 1 -- 172.21.15.94:0/3043827767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a7400a670 con 0x7f4a781057a0 2024-02-17T12:11:05.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.546+0000 7f4a8078c640 1 -- 172.21.15.94:0/3043827767 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a781aa930 con 0x7f4a781057a0 2024-02-17T12:11:05.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.549+0000 7f4a6f7fe640 1 -- 172.21.15.94:0/3043827767 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4a70005220 con 0x7f4a781057a0 2024-02-17T12:11:05.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.549+0000 7f4a83364740 1 Processor -- start 2024-02-17T12:11:05.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.549+0000 7f4a83364740 1 -- start start 2024-02-17T12:11:05.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.549+0000 7f4a83364740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a1142250 0x5607a1142630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.549+0000 7f4a83364740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a114b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.549+0000 7f4a83364740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a114b750 0x5607a114db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.549+0000 7f4a83364740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607a10fd960 con 0x5607a1142b70 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.549+0000 7f4a83364740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607a10b1310 con 0x5607a114b750 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.549+0000 7f4a83364740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607a10ae840 con 0x5607a1142250 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.549+0000 7f4a6f7fe640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4a640a22b0 0x7f4a640a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.549+0000 7f4a7e528640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a114b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.549+0000 7f4a7dd27640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a114b750 0x5607a114db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.549+0000 7f4a7e528640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a114b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48084/0 (socket says 172.21.15.94:48084) 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a7e528640 1 -- 172.21.15.94:0/65953209 learned_addr learned my addr 172.21.15.94:0/65953209 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/65953209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a1142250 0x5607a1142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a7ed29640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4a640a22b0 0x7f4a640a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a7dd27640 1 -- 172.21.15.94:0/65953209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a1142250 msgr2=0x5607a1142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/65953209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a1142250 0x5607a1142630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a7dd27640 1 -- 172.21.15.94:0/65953209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 msgr2=0x5607a114b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/65953209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a114b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a7dd27640 1 -- 172.21.15.94:0/65953209 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607a106c060 con 0x5607a114b750 2024-02-17T12:11:05.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a6f7fe640 1 -- 172.21.15.94:0/3043827767 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f4a700cdfd0 con 0x7f4a781057a0 2024-02-17T12:11:05.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/65953209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a114b750 0x5607a114db40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4a68005d00 tx=0x7f4a68009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c589ba2114120ba7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a7ed29640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4a640a22b0 0x7f4a640a4770 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4a70096e40 tx=0x7f4a7006c9a0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a6cff9640 1 -- 172.21.15.94:0/65953209 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68013e80 con 0x5607a114b750 2024-02-17T12:11:05.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a6cff9640 1 -- 172.21.15.94:0/65953209 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a68019070 con 0x5607a114b750 2024-02-17T12:11:05.554 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.550+0000 7f4a6cff9640 1 -- 172.21.15.94:0/65953209 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a680047a0 con 0x5607a114b750 2024-02-17T12:11:05.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.551+0000 7f4a83364740 1 -- 172.21.15.94:0/65953209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a114b750 msgr2=0x5607a114db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.551+0000 7f4a83364740 1 --2- 172.21.15.94:0/65953209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a114b750 0x5607a114db40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4a68005d00 tx=0x7f4a68009760 comp rx=0 tx=0).stop 2024-02-17T12:11:05.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.551+0000 7f4a83364740 1 -- 172.21.15.94:0/65953209 shutdown_connections 2024-02-17T12:11:05.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.551+0000 7f4a83364740 1 --2- 172.21.15.94:0/65953209 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a114b750 0x5607a114db40 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.551+0000 7f4a83364740 1 --2- 172.21.15.94:0/65953209 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a114b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.551+0000 7f4a83364740 1 --2- 172.21.15.94:0/65953209 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a1142250 0x5607a1142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.551+0000 7f4a83364740 1 -- 172.21.15.94:0/65953209 >> 172.21.15.94:0/65953209 conn(0x5607a113c980 msgr2=0x5607a113cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.551+0000 7f4a83364740 1 -- 172.21.15.94:0/65953209 shutdown_connections 2024-02-17T12:11:05.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.551+0000 7f4a83364740 1 -- 172.21.15.94:0/65953209 wait complete. 2024-02-17T12:11:05.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.551+0000 7f4a83364740 1 -- start start 2024-02-17T12:11:05.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.551+0000 7f4a83364740 1 Processor -- start 2024-02-17T12:11:05.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.552+0000 7f4a83364740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142250 0x5607a1148760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.552+0000 7f4a83364740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a1142b70 0x5607a1148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.552+0000 7f4a83364740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a114b750 0x5607a11491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.552+0000 7f4a7e528640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a1142b70 0x5607a1148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.552+0000 7f4a7e528640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a1142b70 0x5607a1148ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46030/0 (socket says 172.21.15.94:46030) 2024-02-17T12:11:05.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.552+0000 7f4a7e528640 1 -- 172.21.15.94:0/2582584092 learned_addr learned my addr 172.21.15.94:0/2582584092 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.552+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/2582584092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a114b750 0x5607a11491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.553+0000 7f4a83364740 1 -- 172.21.15.94:0/2582584092 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607a10aa020 con 0x5607a1142250 2024-02-17T12:11:05.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.553+0000 7f4a83364740 1 -- 172.21.15.94:0/2582584092 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607a0d7d430 con 0x5607a1142b70 2024-02-17T12:11:05.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.553+0000 7f4a83364740 1 -- 172.21.15.94:0/2582584092 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607a0d39e00 con 0x5607a114b750 2024-02-17T12:11:05.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.553+0000 7f4a83364740 10 client.0 _ll_register_callbacks cb 0x5607a1051cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:05.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.553+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/2582584092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142250 0x5607a1148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.554+0000 7f4a7dd27640 1 -- 172.21.15.94:0/2582584092 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a1142b70 msgr2=0x5607a1148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.554+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/2582584092 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a1142b70 0x5607a1148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.559 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.554+0000 7f4a7dd27640 1 -- 172.21.15.94:0/2582584092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142250 msgr2=0x5607a1148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.554+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/2582584092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142250 0x5607a1148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.554+0000 7f4a7dd27640 1 -- 172.21.15.94:0/2582584092 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a68013610 con 0x5607a114b750 2024-02-17T12:11:05.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.554+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/2582584092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a114b750 0x5607a11491e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4a68001eb0 tx=0x7f4a68005430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc427e7a11f45f46 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.554+0000 7f4a61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:11:05.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.554+0000 7f4a61ffb640 1 -- 172.21.15.94:0/2582584092 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68004020 con 0x5607a114b750 2024-02-17T12:11:05.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.554+0000 7f4a83364740 1 -- 172.21.15.94:0/2582584092 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5607a0da97e0 con 0x5607a114b750 2024-02-17T12:11:05.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.554+0000 7f4a61ffb640 1 -- 172.21.15.94:0/2582584092 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f4a68000e40 con 0x5607a114b750 2024-02-17T12:11:05.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.555+0000 7f4a61ffb640 1 -- 172.21.15.94:0/2582584092 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a68004390 con 0x5607a114b750 2024-02-17T12:11:05.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.555+0000 7f4a83364740 10 client.28324 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:05.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.555+0000 7f4a83364740 1 -- 172.21.15.94:0/2582584092 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5607a10a86a0 con 0x5607a114b750 2024-02-17T12:11:05.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.555+0000 7f4a61ffb640 1 -- 172.21.15.94:0/2582584092 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a68014e90 con 0x5607a114b750 2024-02-17T12:11:05.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.555+0000 7f4a61ffb640 1 -- 172.21.15.94:0/2582584092 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4a6800e8e0 con 0x5607a114b750 2024-02-17T12:11:05.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.555+0000 7f4a83364740 10 client.28324 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:05.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.555+0000 7f4a83364740 10 client.28324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:05.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.555+0000 7f4a83364740 20 client.28324 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:05.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.555+0000 7f4a83364740 1 --2- 172.21.15.94:0/2582584092 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5607a115ba80 0x5607a115dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.555+0000 7f4a83364740 4 client.28324 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:05.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.555+0000 7f4a83364740 1 -- 172.21.15.94:0/2582584092 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5607a0da97e0 con 0x5607a115ba80 2024-02-17T12:11:05.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.555+0000 7f4a7e528640 1 --2- 172.21.15.94:0/2582584092 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5607a115ba80 0x5607a115dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.556+0000 7f4a61ffb640 10 client.28324 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:05.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.556+0000 7f4a7e528640 1 --2- 172.21.15.94:0/2582584092 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5607a115ba80 0x5607a115dec0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe924c31f34b73b7 server_cookie=2689ec8937e9256e in_seq=0 out_seq=0 2024-02-17T12:11:05.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.556+0000 7f4a61ffb640 1 -- 172.21.15.94:0/2582584092 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5607a0da97e0 con 0x5607a115ba80 2024-02-17T12:11:05.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.556+0000 7f4a61ffb640 10 client.28324 handle_command_reply: tid=0 2024-02-17T12:11:05.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.556+0000 7f4a83364740 1 client.28324 shutdown 2024-02-17T12:11:05.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.557+0000 7f4a83364740 1 -- 172.21.15.94:0/2582584092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a114b750 msgr2=0x5607a11491e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.562 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.557+0000 7f4a83364740 1 --2- 172.21.15.94:0/2582584092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a114b750 0x5607a11491e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4a68001eb0 tx=0x7f4a68005430 comp rx=0 tx=0).stop 2024-02-17T12:11:05.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.557+0000 7f4a83364740 1 -- 172.21.15.94:0/2582584092 shutdown_connections 2024-02-17T12:11:05.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.557+0000 7f4a83364740 1 --2- 172.21.15.94:0/2582584092 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5607a115ba80 0x5607a115dec0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.557+0000 7f4a61ffb640 0 client.28324 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:05.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.557+0000 7f4a83364740 1 --2- 172.21.15.94:0/2582584092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a114b750 0x5607a11491e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.557+0000 7f4a83364740 1 --2- 172.21.15.94:0/2582584092 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a1142b70 0x5607a1148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.557+0000 7f4a83364740 1 --2- 172.21.15.94:0/2582584092 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142250 0x5607a1148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.557+0000 7f4a83364740 1 -- 172.21.15.94:0/2582584092 >> 172.21.15.94:0/2582584092 conn(0x5607a113c980 msgr2=0x5607a113e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.557+0000 7f4a83364740 1 -- 172.21.15.94:0/2582584092 shutdown_connections 2024-02-17T12:11:05.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.557+0000 7f4a83364740 1 -- 172.21.15.94:0/2582584092 wait complete. 2024-02-17T12:11:05.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.557+0000 7f4a83364740 20 client.28324 trim_cache size 0 max 16384 2024-02-17T12:11:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.570+0000 7f4a83364740 1 Processor -- start 2024-02-17T12:11:05.573 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a83364740 1 -- start start 2024-02-17T12:11:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a83364740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a1142250 0x5607a114ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a83364740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a1147030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a83364740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a114b750 0x5607a1147570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a83364740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607a102f130 con 0x5607a1142b70 2024-02-17T12:11:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a83364740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607a1147ae0 con 0x5607a114b750 2024-02-17T12:11:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a83364740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607a113da00 con 0x5607a1142250 2024-02-17T12:11:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a7e528640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a1147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a7e528640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a1147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48108/0 (socket says 172.21.15.94:48108) 2024-02-17T12:11:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a7e528640 1 -- 172.21.15.94:0/3180753490 learned_addr learned my addr 172.21.15.94:0/3180753490 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/3180753490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a1142250 0x5607a114ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/3180753490 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a114b750 0x5607a1147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a7e528640 1 -- 172.21.15.94:0/3180753490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a1142250 msgr2=0x5607a114ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.574 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a7e528640 1 --2- 172.21.15.94:0/3180753490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a1142250 0x5607a114ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a7e528640 1 -- 172.21.15.94:0/3180753490 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a114b750 msgr2=0x5607a1147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a7e528640 1 --2- 172.21.15.94:0/3180753490 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a114b750 0x5607a1147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a7e528640 1 -- 172.21.15.94:0/3180753490 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607a1044c20 con 0x5607a1142b70 2024-02-17T12:11:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.571+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/3180753490 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a114b750 0x5607a1147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:11:05.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.572+0000 7f4a7e528640 1 --2- 172.21.15.94:0/3180753490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a1147030 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4a7402f020 tx=0x7f4a74005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394ce361e8959e67 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.573+0000 7f4a63fff640 1 -- 172.21.15.94:0/3180753490 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74047b70 con 0x5607a1142b70 2024-02-17T12:11:05.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.573+0000 7f4a63fff640 1 -- 172.21.15.94:0/3180753490 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a74047cd0 con 0x5607a1142b70 2024-02-17T12:11:05.576 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.573+0000 7f4a63fff640 1 -- 172.21.15.94:0/3180753490 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74047b70 con 0x5607a1142b70 2024-02-17T12:11:05.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.574+0000 7f4a83364740 1 -- 172.21.15.94:0/3180753490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 msgr2=0x5607a1147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.574+0000 7f4a83364740 1 --2- 172.21.15.94:0/3180753490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a1147030 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f4a7402f020 tx=0x7f4a74005bb0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.574+0000 7f4a83364740 1 -- 172.21.15.94:0/3180753490 shutdown_connections 2024-02-17T12:11:05.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.574+0000 7f4a83364740 1 --2- 172.21.15.94:0/3180753490 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a114b750 0x5607a1147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.574+0000 7f4a83364740 1 --2- 172.21.15.94:0/3180753490 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a1147030 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.574+0000 7f4a83364740 1 --2- 172.21.15.94:0/3180753490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a1142250 0x5607a114ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.574+0000 7f4a83364740 1 -- 172.21.15.94:0/3180753490 >> 172.21.15.94:0/3180753490 conn(0x5607a113c980 msgr2=0x5607a1143a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a83364740 1 -- 172.21.15.94:0/3180753490 shutdown_connections 2024-02-17T12:11:05.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a83364740 1 -- 172.21.15.94:0/3180753490 wait complete. 2024-02-17T12:11:05.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a83364740 1 -- start start 2024-02-17T12:11:05.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a83364740 1 Processor -- start 2024-02-17T12:11:05.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a83364740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a1142250 0x5607a1151110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a83364740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a1151650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a83364740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a114b750 0x5607a1151b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a83364740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607a11b1fb0 con 0x5607a1142b70 2024-02-17T12:11:05.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a83364740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607a11b20f0 con 0x5607a1142250 2024-02-17T12:11:05.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a83364740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607a11b22b0 con 0x5607a114b750 2024-02-17T12:11:05.583 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a7dd27640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a1142250 0x5607a1151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a7dd27640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a1142250 0x5607a1151110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46046/0 (socket says 172.21.15.94:46046) 2024-02-17T12:11:05.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a7dd27640 1 -- 172.21.15.94:0/2487031112 learned_addr learned my addr 172.21.15.94:0/2487031112 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:05.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.575+0000 7f4a83364740 10 client.0 _ll_register_callbacks cb 0x7f4a68009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:05.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a7e528640 1 --2- 172.21.15.94:0/2487031112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a1151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/2487031112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a114b750 0x5607a1151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a7dd27640 1 -- 172.21.15.94:0/2487031112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a114b750 msgr2=0x5607a1151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/2487031112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a114b750 0x5607a1151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a7dd27640 1 -- 172.21.15.94:0/2487031112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 msgr2=0x5607a1151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/2487031112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a1151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a7dd27640 1 -- 172.21.15.94:0/2487031112 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a7810cf60 con 0x5607a1142250 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a7dd27640 1 --2- 172.21.15.94:0/2487031112 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a1142250 0x5607a1151110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4a68009300 tx=0x7f4a68005430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1ee2add4eca803b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a61ffb640 1 -- 172.21.15.94:0/2487031112 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68015ea0 con 0x5607a1142250 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a83364740 1 -- 172.21.15.94:0/2487031112 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a68019070 con 0x5607a1142250 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a61ffb640 1 -- 172.21.15.94:0/2487031112 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f4a68009e60 con 0x5607a1142250 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a61ffb640 1 -- 172.21.15.94:0/2487031112 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a680045a0 con 0x5607a1142250 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a83364740 10 client.28352 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.576+0000 7f4a83364740 1 -- 172.21.15.94:0/2487031112 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5607a106a8c0 con 0x5607a1142250 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.577+0000 7f4a61ffb640 1 -- 172.21.15.94:0/2487031112 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4a68019070 con 0x5607a1142250 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.577+0000 7f4a61ffb640 1 -- 172.21.15.94:0/2487031112 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4a68004c10 con 0x5607a1142250 2024-02-17T12:11:05.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.577+0000 7f4a83364740 10 client.28352 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.577+0000 7f4a83364740 10 client.28352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.577+0000 7f4a83364740 20 client.28352 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.577+0000 7f4a83364740 1 --2- 172.21.15.94:0/2487031112 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5607a115ba80 0x5607a11b5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.577+0000 7f4a83364740 4 client.28352 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:11:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.577+0000 7f4a83364740 1 -- 172.21.15.94:0/2487031112 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5607a0de6720 con 0x5607a115ba80 2024-02-17T12:11:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.577+0000 7f4a7e528640 1 --2- 172.21.15.94:0/2487031112 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5607a115ba80 0x5607a11b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.577+0000 7f4a61ffb640 10 client.28352 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.577+0000 7f4a7e528640 1 --2- 172.21.15.94:0/2487031112 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5607a115ba80 0x5607a11b5790 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3291d69f3efc5fc7 server_cookie=c99af7961380b2a5 in_seq=0 out_seq=0 2024-02-17T12:11:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a61ffb640 1 -- 172.21.15.94:0/2487031112 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5607a0de6720 con 0x5607a115ba80 2024-02-17T12:11:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a61ffb640 10 client.28352 handle_command_reply: tid=0 2024-02-17T12:11:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a83364740 1 client.28352 shutdown 2024-02-17T12:11:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a83364740 1 -- 172.21.15.94:0/2487031112 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a1142250 msgr2=0x5607a1151110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a83364740 1 --2- 172.21.15.94:0/2487031112 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a1142250 0x5607a1151110 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4a68009300 tx=0x7f4a68005430 comp rx=0 tx=0).stop 2024-02-17T12:11:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a83364740 1 -- 172.21.15.94:0/2487031112 shutdown_connections 2024-02-17T12:11:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a83364740 1 --2- 172.21.15.94:0/2487031112 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5607a115ba80 0x5607a11b5790 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a83364740 1 --2- 172.21.15.94:0/2487031112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607a114b750 0x5607a1151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a61ffb640 0 client.28352 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a83364740 1 --2- 172.21.15.94:0/2487031112 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607a1142b70 0x5607a1151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a83364740 1 --2- 172.21.15.94:0/2487031112 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607a1142250 0x5607a1151110 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a83364740 1 -- 172.21.15.94:0/2487031112 >> 172.21.15.94:0/2487031112 conn(0x5607a113c980 msgr2=0x5607a1146fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a83364740 1 -- 172.21.15.94:0/2487031112 shutdown_connections 2024-02-17T12:11:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a83364740 1 -- 172.21.15.94:0/2487031112 wait complete. 2024-02-17T12:11:05.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.578+0000 7f4a83364740 20 client.28352 trim_cache size 0 max 16384 2024-02-17T12:11:05.588 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:11:05.588 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.579+0000 7f4a8078c640 1 -- 172.21.15.94:0/3043827767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4a640a22b0 msgr2=0x7f4a640a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.579+0000 7f4a8078c640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4a640a22b0 0x7f4a640a4770 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4a70096e40 tx=0x7f4a7006c9a0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.579+0000 7f4a8078c640 1 -- 172.21.15.94:0/3043827767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a781057a0 msgr2=0x7f4a78199e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.579+0000 7f4a8078c640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a781057a0 0x7f4a78199e70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000da90 tx=0x7f4a7000df50 comp rx=0 tx=0).stop 2024-02-17T12:11:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.579+0000 7f4a8078c640 1 -- 172.21.15.94:0/3043827767 shutdown_connections 2024-02-17T12:11:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.579+0000 7f4a8078c640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4a640a22b0 0x7f4a640a4770 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.579+0000 7f4a8078c640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4a781057a0 0x7f4a78199e70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.579+0000 7f4a8078c640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a78100150 0x7f4a78195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.579+0000 7f4a8078c640 1 --2- 172.21.15.94:0/3043827767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4a780ff790 0x7f4a78195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.579+0000 7f4a8078c640 1 -- 172.21.15.94:0/3043827767 >> 172.21.15.94:0/3043827767 conn(0x7f4a780765a0 msgr2=0x7f4a78102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.579+0000 7f4a8078c640 1 -- 172.21.15.94:0/3043827767 shutdown_connections 2024-02-17T12:11:05.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:05.579+0000 7f4a8078c640 1 -- 172.21.15.94:0/3043827767 wait complete. 2024-02-17T12:11:06.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:05 smithi094 ceph-mon[32499]: pgmap v405: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-02-17T12:11:06.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:05 smithi189 ceph-mon[36588]: pgmap v405: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-02-17T12:11:06.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:05 smithi125 ceph-mon[36819]: pgmap v405: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-02-17T12:11:06.595 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:11:06.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:11:06.595 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:11:06.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.744+0000 7f67e77fe640 1 -- 172.21.15.94:0/3005471251 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0032ad0 con 0x7f67e810d730 2024-02-17T12:11:06.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.745+0000 7f67eec40640 1 -- 172.21.15.94:0/3005471251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e810d730 msgr2=0x7f67e810db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:06.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.745+0000 7f67eec40640 1 --2- 172.21.15.94:0/3005471251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e810d730 0x7f67e810db10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009cd0 tx=0x7f67e002c100 comp rx=0 tx=0).stop 2024-02-17T12:11:06.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.745+0000 7f67eec40640 1 -- 172.21.15.94:0/3005471251 shutdown_connections 2024-02-17T12:11:06.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.745+0000 7f67eec40640 1 --2- 172.21.15.94:0/3005471251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e81080e0 0x7f67e81145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.745+0000 7f67eec40640 1 --2- 172.21.15.94:0/3005471251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67e8107720 0x7f67e8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.745+0000 7f67eec40640 1 --2- 172.21.15.94:0/3005471251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e810d730 0x7f67e810db10 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.745+0000 7f67eec40640 1 -- 172.21.15.94:0/3005471251 >> 172.21.15.94:0/3005471251 conn(0x7f67e80756c0 msgr2=0x7f67e8077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:06.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.745+0000 7f67eec40640 1 -- 172.21.15.94:0/3005471251 shutdown_connections 2024-02-17T12:11:06.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.745+0000 7f67eec40640 1 -- 172.21.15.94:0/3005471251 wait complete. 2024-02-17T12:11:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.746+0000 7f67eec40640 1 Processor -- start 2024-02-17T12:11:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.746+0000 7f67eec40640 1 -- start start 2024-02-17T12:11:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.746+0000 7f67eec40640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e8107720 0x7f67e819de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.746+0000 7f67eec40640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e81080e0 0x7f67e819e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.746+0000 7f67eec40640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67e810d730 0x7f67e81a2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.746+0000 7f67eec40640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81a2d30 con 0x7f67e8107720 2024-02-17T12:11:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.746+0000 7f67eec40640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81a2ea0 con 0x7f67e81080e0 2024-02-17T12:11:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.746+0000 7f67eec40640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81a3060 con 0x7f67e810d730 2024-02-17T12:11:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.746+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e81080e0 0x7f67e819e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.746+0000 7f67ec9dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e8107720 0x7f67e819de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.746+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e81080e0 0x7f67e819e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46068/0 (socket says 172.21.15.94:46068) 2024-02-17T12:11:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.746+0000 7f67e7fff640 1 -- 172.21.15.94:0/1439946466 learned_addr learned my addr 172.21.15.94:0/1439946466 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:06.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.746+0000 7f67ed1dd640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67e810d730 0x7f67e81a2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.747+0000 7f67e7fff640 1 -- 172.21.15.94:0/1439946466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67e810d730 msgr2=0x7f67e81a2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.747+0000 7f67e7fff640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67e810d730 0x7f67e81a2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.747+0000 7f67e7fff640 1 -- 172.21.15.94:0/1439946466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e8107720 msgr2=0x7f67e819de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.747+0000 7f67e7fff640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e8107720 0x7f67e819de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.747+0000 7f67e7fff640 1 -- 172.21.15.94:0/1439946466 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e0009900 con 0x7f67e81080e0 2024-02-17T12:11:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.747+0000 7f67ec9dc640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e8107720 0x7f67e819de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:06.750 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.747+0000 7f67e7fff640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e81080e0 0x7f67e819e3b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f67d800b450 tx=0x7f67d800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:06.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.747+0000 7f67e5ffb640 1 -- 172.21.15.94:0/1439946466 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8011890 con 0x7f67e81080e0 2024-02-17T12:11:06.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.747+0000 7f67e5ffb640 1 -- 172.21.15.94:0/1439946466 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67d80103e0 con 0x7f67e81080e0 2024-02-17T12:11:06.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.747+0000 7f67e5ffb640 1 -- 172.21.15.94:0/1439946466 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d800f5a0 con 0x7f67e81080e0 2024-02-17T12:11:06.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.747+0000 7f67eec40640 1 -- 172.21.15.94:0/1439946466 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e81a3340 con 0x7f67e81080e0 2024-02-17T12:11:06.751 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.748+0000 7f67eec40640 1 -- 172.21.15.94:0/1439946466 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e81aabc0 con 0x7f67e81080e0 2024-02-17T12:11:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.750+0000 7f67f180f740 1 Processor -- start 2024-02-17T12:11:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.750+0000 7f67f180f740 1 -- start start 2024-02-17T12:11:06.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.750+0000 7f67f180f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520c380 0x5556c520c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.750+0000 7f67e5ffb640 1 -- 172.21.15.94:0/1439946466 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f67d8010550 con 0x7f67e81080e0 2024-02-17T12:11:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.750+0000 7f67f180f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520cd30 0x5556c52152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.750+0000 7f67f180f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c52158b0 0x5556c5217d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.750+0000 7f67f180f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c517af90 con 0x5556c52158b0 2024-02-17T12:11:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.750+0000 7f67f180f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c51784c0 con 0x5556c520cd30 2024-02-17T12:11:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.750+0000 7f67f180f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c5173ca0 con 0x5556c520c380 2024-02-17T12:11:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.751+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c52158b0 0x5556c5217d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.751+0000 7f67ec9dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520cd30 0x5556c52152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.751+0000 7f67ec9dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520cd30 0x5556c52152e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46076/0 (socket says 172.21.15.94:46076) 2024-02-17T12:11:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.751+0000 7f67ec9dc640 1 -- 172.21.15.94:0/868105535 learned_addr learned my addr 172.21.15.94:0/868105535 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:06.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.751+0000 7f67ed1dd640 1 --2- 172.21.15.94:0/868105535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520c380 0x5556c520c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.751+0000 7f67e5ffb640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f67d00a22b0 0x7f67d00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.751+0000 7f67e5ffb640 1 -- 172.21.15.94:0/1439946466 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f67d80cdd50 con 0x7f67e81080e0 2024-02-17T12:11:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.752+0000 7f67e7fff640 1 -- 172.21.15.94:0/868105535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520c380 msgr2=0x5556c520c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.752+0000 7f67e7fff640 1 --2- 172.21.15.94:0/868105535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520c380 0x5556c520c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.752+0000 7f67e7fff640 1 -- 172.21.15.94:0/868105535 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520cd30 msgr2=0x5556c52152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.752+0000 7f67e7fff640 1 --2- 172.21.15.94:0/868105535 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520cd30 0x5556c52152e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.752+0000 7f67e7fff640 1 -- 172.21.15.94:0/868105535 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556c51360b0 con 0x5556c52158b0 2024-02-17T12:11:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.752+0000 7f67ed1dd640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f67d00a22b0 0x7f67d00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.753+0000 7f67e7fff640 1 --2- 172.21.15.94:0/868105535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c52158b0 0x5556c5217d00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f67d80cb540 tx=0x7f67d8010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712fa05e16d63184 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:06.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.754+0000 7f67ceffd640 1 -- 172.21.15.94:0/868105535 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d809a140 con 0x5556c52158b0 2024-02-17T12:11:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.754+0000 7f67ceffd640 1 -- 172.21.15.94:0/868105535 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67d809aee0 con 0x5556c52158b0 2024-02-17T12:11:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.754+0000 7f67ceffd640 1 -- 172.21.15.94:0/868105535 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d80cb6b0 con 0x5556c52158b0 2024-02-17T12:11:06.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.754+0000 7f67ed1dd640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f67d00a22b0 0x7f67d00a4770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00bd40 tx=0x7f67dc0046c0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.754+0000 7f67f180f740 1 -- 172.21.15.94:0/868105535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c52158b0 msgr2=0x5556c5217d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.754+0000 7f67f180f740 1 --2- 172.21.15.94:0/868105535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c52158b0 0x5556c5217d00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f67d80cb540 tx=0x7f67d8010d00 comp rx=0 tx=0).stop 2024-02-17T12:11:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.755+0000 7f67f180f740 1 -- 172.21.15.94:0/868105535 shutdown_connections 2024-02-17T12:11:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.755+0000 7f67f180f740 1 --2- 172.21.15.94:0/868105535 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c52158b0 0x5556c5217d00 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.755+0000 7f67f180f740 1 --2- 172.21.15.94:0/868105535 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520cd30 0x5556c52152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.755+0000 7f67f180f740 1 --2- 172.21.15.94:0/868105535 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520c380 0x5556c520c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.755+0000 7f67f180f740 1 -- 172.21.15.94:0/868105535 >> 172.21.15.94:0/868105535 conn(0x5556c5206840 msgr2=0x5556c5206c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.755+0000 7f67f180f740 1 -- 172.21.15.94:0/868105535 shutdown_connections 2024-02-17T12:11:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.755+0000 7f67f180f740 1 -- 172.21.15.94:0/868105535 wait complete. 2024-02-17T12:11:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.755+0000 7f67f180f740 1 -- start start 2024-02-17T12:11:06.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.755+0000 7f67f180f740 1 Processor -- start 2024-02-17T12:11:06.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67f180f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520c380 0x5556c52129a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67f180f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c520cd30 0x5556c5212ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67f180f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c5213420 0x5556c5221990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67f180f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c5118f40 con 0x5556c520cd30 2024-02-17T12:11:06.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67f180f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c4e474b0 con 0x5556c520c380 2024-02-17T12:11:06.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67f180f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c50f9670 con 0x5556c5213420 2024-02-17T12:11:06.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67f180f740 10 client.0 _ll_register_callbacks cb 0x5556c5166f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:06.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67ec9dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520c380 0x5556c52129a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67e7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c520cd30 0x5556c5212ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67ec9dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520c380 0x5556c52129a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46086/0 (socket says 172.21.15.94:46086) 2024-02-17T12:11:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67ec9dc640 1 -- 172.21.15.94:0/249464893 learned_addr learned my addr 172.21.15.94:0/249464893 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67ed1dd640 1 --2- 172.21.15.94:0/249464893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c5213420 0x5556c5221990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67ec9dc640 1 -- 172.21.15.94:0/249464893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c5213420 msgr2=0x5556c5221990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67ec9dc640 1 --2- 172.21.15.94:0/249464893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c5213420 0x5556c5221990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67ec9dc640 1 -- 172.21.15.94:0/249464893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c520cd30 msgr2=0x5556c5212ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67ec9dc640 1 --2- 172.21.15.94:0/249464893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c520cd30 0x5556c5212ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67ec9dc640 1 -- 172.21.15.94:0/249464893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e81aabc0 con 0x5556c520c380 2024-02-17T12:11:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67ec9dc640 1 --2- 172.21.15.94:0/249464893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520c380 0x5556c52129a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f67e0005cc0 tx=0x7f67e0034670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57234e1ac12dc4c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.756+0000 7f67c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:11:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.757+0000 7f67c7fff640 1 -- 172.21.15.94:0/249464893 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0003b70 con 0x5556c520c380 2024-02-17T12:11:06.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.757+0000 7f67c7fff640 1 -- 172.21.15.94:0/249464893 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f67e0046070 con 0x5556c520c380 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.757+0000 7f67f180f740 1 -- 172.21.15.94:0/249464893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556c514df40 con 0x5556c520c380 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.757+0000 7f67c7fff640 1 -- 172.21.15.94:0/249464893 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67e0003cd0 con 0x5556c520c380 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.757+0000 7f67f180f740 10 client.28364 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.757+0000 7f67f180f740 1 -- 172.21.15.94:0/249464893 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5556c5172320 con 0x5556c520c380 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.757+0000 7f67c7fff640 1 -- 172.21.15.94:0/249464893 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67e0057080 con 0x5556c520c380 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.757+0000 7f67c7fff640 1 -- 172.21.15.94:0/249464893 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f67e00538e0 con 0x5556c520c380 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.757+0000 7f67f180f740 10 client.28364 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.757+0000 7f67f180f740 10 client.28364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.758+0000 7f67f180f740 20 client.28364 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.758+0000 7f67f180f740 1 --2- 172.21.15.94:0/249464893 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5556c5226400 0x5556c52287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.758+0000 7f67f180f740 4 client.28364 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.758+0000 7f67f180f740 1 -- 172.21.15.94:0/249464893 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556c516a310 con 0x5556c5226400 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.758+0000 7f67e7fff640 1 --2- 172.21.15.94:0/249464893 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5556c5226400 0x5556c52287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.762 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.758+0000 7f67c7fff640 10 client.28364 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:06.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.759+0000 7f67e7fff640 1 --2- 172.21.15.94:0/249464893 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5556c5226400 0x5556c52287f0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=680d8c080294454a server_cookie=fada8bdcd8c25855 in_seq=0 out_seq=0 2024-02-17T12:11:06.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67c7fff640 1 -- 172.21.15.94:0/249464893 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5556c516a310 con 0x5556c5226400 2024-02-17T12:11:06.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67c7fff640 10 client.28364 handle_command_reply: tid=0 2024-02-17T12:11:06.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67f180f740 1 client.28364 shutdown 2024-02-17T12:11:06.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67f180f740 1 -- 172.21.15.94:0/249464893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520c380 msgr2=0x5556c52129a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:06.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67f180f740 1 --2- 172.21.15.94:0/249464893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520c380 0x5556c52129a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f67e0005cc0 tx=0x7f67e0034670 comp rx=0 tx=0).stop 2024-02-17T12:11:06.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67f180f740 1 -- 172.21.15.94:0/249464893 shutdown_connections 2024-02-17T12:11:06.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67f180f740 1 --2- 172.21.15.94:0/249464893 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5556c5226400 0x5556c52287f0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.764 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67c7fff640 0 client.28364 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:06.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67f180f740 1 --2- 172.21.15.94:0/249464893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c5213420 0x5556c5221990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67f180f740 1 --2- 172.21.15.94:0/249464893 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c520cd30 0x5556c5212ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67f180f740 1 --2- 172.21.15.94:0/249464893 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520c380 0x5556c52129a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67f180f740 1 -- 172.21.15.94:0/249464893 >> 172.21.15.94:0/249464893 conn(0x5556c5206840 msgr2=0x5556c52087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:06.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67f180f740 1 -- 172.21.15.94:0/249464893 shutdown_connections 2024-02-17T12:11:06.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67f180f740 1 -- 172.21.15.94:0/249464893 wait complete. 2024-02-17T12:11:06.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.760+0000 7f67f180f740 20 client.28364 trim_cache size 0 max 16384 2024-02-17T12:11:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.780+0000 7f67f180f740 1 Processor -- start 2024-02-17T12:11:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.780+0000 7f67f180f740 1 -- start start 2024-02-17T12:11:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.780+0000 7f67f180f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c520c380 0x5556c520fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.780+0000 7f67f180f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520cd30 0x5556c52101a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.780+0000 7f67f180f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c5213420 0x5556c5212230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.780+0000 7f67f180f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c5222f00 con 0x5556c520c380 2024-02-17T12:11:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.780+0000 7f67f180f740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c52261b0 con 0x5556c5213420 2024-02-17T12:11:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.780+0000 7f67f180f740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c5202110 con 0x5556c520cd30 2024-02-17T12:11:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.780+0000 7f67ec9dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c520c380 0x5556c520fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.780+0000 7f67ec9dc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c520c380 0x5556c520fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48168/0 (socket says 172.21.15.94:48168) 2024-02-17T12:11:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.780+0000 7f67ec9dc640 1 -- 172.21.15.94:0/1801142266 learned_addr learned my addr 172.21.15.94:0/1801142266 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.780+0000 7f67e7fff640 1 --2- 172.21.15.94:0/1801142266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520cd30 0x5556c52101a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.780+0000 7f67ed1dd640 1 --2- 172.21.15.94:0/1801142266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c5213420 0x5556c5212230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.781+0000 7f67ec9dc640 1 -- 172.21.15.94:0/1801142266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520cd30 msgr2=0x5556c52101a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.781+0000 7f67ec9dc640 1 --2- 172.21.15.94:0/1801142266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520cd30 0x5556c52101a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.781+0000 7f67ec9dc640 1 -- 172.21.15.94:0/1801142266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c5213420 msgr2=0x5556c5212230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.781+0000 7f67ec9dc640 1 --2- 172.21.15.94:0/1801142266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c5213420 0x5556c5212230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.781+0000 7f67ec9dc640 1 -- 172.21.15.94:0/1801142266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556c510ec00 con 0x5556c520c380 2024-02-17T12:11:06.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.781+0000 7f67ec9dc640 1 --2- 172.21.15.94:0/1801142266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c520c380 0x5556c520fc60 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f67e002c640 tx=0x7f67e002c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dc1c52c8206c2f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:06.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.781+0000 7f67ce7fc640 1 -- 172.21.15.94:0/1801142266 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0032e00 con 0x5556c520c380 2024-02-17T12:11:06.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.782+0000 7f67ce7fc640 1 -- 172.21.15.94:0/1801142266 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5556c514df40 con 0x5556c520c380 2024-02-17T12:11:06.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.782+0000 7f67ce7fc640 1 -- 172.21.15.94:0/1801142266 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e00045c0 con 0x5556c520c380 2024-02-17T12:11:06.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.782+0000 7f67f180f740 1 -- 172.21.15.94:0/1801142266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c520c380 msgr2=0x5556c520fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:06.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.782+0000 7f67f180f740 1 --2- 172.21.15.94:0/1801142266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c520c380 0x5556c520fc60 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f67e002c640 tx=0x7f67e002c100 comp rx=0 tx=0).stop 2024-02-17T12:11:06.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.782+0000 7f67f180f740 1 -- 172.21.15.94:0/1801142266 shutdown_connections 2024-02-17T12:11:06.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.782+0000 7f67f180f740 1 --2- 172.21.15.94:0/1801142266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c5213420 0x5556c5212230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.782+0000 7f67f180f740 1 --2- 172.21.15.94:0/1801142266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520cd30 0x5556c52101a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.782+0000 7f67f180f740 1 --2- 172.21.15.94:0/1801142266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c520c380 0x5556c520fc60 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.782+0000 7f67f180f740 1 -- 172.21.15.94:0/1801142266 >> 172.21.15.94:0/1801142266 conn(0x5556c5206840 msgr2=0x5556c52112b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.783+0000 7f67f180f740 1 -- 172.21.15.94:0/1801142266 shutdown_connections 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.783+0000 7f67f180f740 1 -- 172.21.15.94:0/1801142266 wait complete. 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.783+0000 7f67f180f740 1 -- start start 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.783+0000 7f67f180f740 1 Processor -- start 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.783+0000 7f67f180f740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520c380 0x5556c521ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.783+0000 7f67f180f740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520cd30 0x5556c521bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.783+0000 7f67f180f740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c5213420 0x5556c527c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.783+0000 7f67f180f740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c527ce40 con 0x5556c5213420 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.783+0000 7f67ed1dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c5213420 0x5556c527c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.783+0000 7f67ec9dc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520c380 0x5556c521ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.783+0000 7f67ed1dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c5213420 0x5556c527c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48182/0 (socket says 172.21.15.94:48182) 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.783+0000 7f67ed1dd640 1 -- 172.21.15.94:0/3525196918 learned_addr learned my addr 172.21.15.94:0/3525196918 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.783+0000 7f67e7fff640 1 --2- 172.21.15.94:0/3525196918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520cd30 0x5556c521bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67f180f740 1 -- 172.21.15.94:0/3525196918 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c527cf80 con 0x5556c520c380 2024-02-17T12:11:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67f180f740 1 -- 172.21.15.94:0/3525196918 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556c527d0c0 con 0x5556c520cd30 2024-02-17T12:11:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67f180f740 10 client.0 _ll_register_callbacks cb 0x7f67d80046e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67e7fff640 1 -- 172.21.15.94:0/3525196918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520c380 msgr2=0x5556c521ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67e7fff640 1 --2- 172.21.15.94:0/3525196918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520c380 0x5556c521ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67e7fff640 1 -- 172.21.15.94:0/3525196918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c5213420 msgr2=0x5556c527c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67e7fff640 1 --2- 172.21.15.94:0/3525196918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c5213420 0x5556c527c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67e7fff640 1 -- 172.21.15.94:0/3525196918 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556c5172320 con 0x5556c520cd30 2024-02-17T12:11:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67ed1dd640 1 --2- 172.21.15.94:0/3525196918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c5213420 0x5556c527c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67e7fff640 1 --2- 172.21.15.94:0/3525196918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520cd30 0x5556c521bf50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f67d8098a90 tx=0x7f67d80ba040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e7de7927def284 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:11:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67c7fff640 1 -- 172.21.15.94:0/3525196918 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d809aee0 con 0x5556c520cd30 2024-02-17T12:11:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67f180f740 1 -- 172.21.15.94:0/3525196918 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556c5199900 con 0x5556c520cd30 2024-02-17T12:11:06.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67c7fff640 1 -- 172.21.15.94:0/3525196918 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f67d809a7e0 con 0x5556c520cd30 2024-02-17T12:11:06.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67c7fff640 1 -- 172.21.15.94:0/3525196918 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67d80a2f80 con 0x5556c520cd30 2024-02-17T12:11:06.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67f180f740 10 client.28345 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:06.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.784+0000 7f67f180f740 1 -- 172.21.15.94:0/3525196918 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5556c5134910 con 0x5556c520cd30 2024-02-17T12:11:06.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.785+0000 7f67c7fff640 1 -- 172.21.15.94:0/3525196918 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67d807f080 con 0x5556c520cd30 2024-02-17T12:11:06.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.785+0000 7f67c7fff640 1 -- 172.21.15.94:0/3525196918 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f67d8083d20 con 0x5556c520cd30 2024-02-17T12:11:06.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.785+0000 7f67f180f740 10 client.28345 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:06.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.785+0000 7f67f180f740 10 client.28345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:06.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.785+0000 7f67f180f740 20 client.28345 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:06.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.785+0000 7f67f180f740 1 --2- 172.21.15.94:0/3525196918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5556c5226400 0x5556c52805a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.785+0000 7f67f180f740 4 client.28345 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:11:06.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.785+0000 7f67f180f740 1 -- 172.21.15.94:0/3525196918 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5556c4e73860 con 0x5556c5226400 2024-02-17T12:11:06.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.785+0000 7f67ec9dc640 1 --2- 172.21.15.94:0/3525196918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5556c5226400 0x5556c52805a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.792 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.785+0000 7f67c7fff640 10 client.28345 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.785+0000 7f67ec9dc640 1 --2- 172.21.15.94:0/3525196918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5556c5226400 0x5556c52805a0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7256832ad222a8f5 server_cookie=a5b221178c6f4e72 in_seq=0 out_seq=0 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.786+0000 7f67c7fff640 1 -- 172.21.15.94:0/3525196918 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5556c4e73860 con 0x5556c5226400 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.786+0000 7f67c7fff640 10 client.28345 handle_command_reply: tid=0 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.786+0000 7f67f180f740 1 client.28345 shutdown 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.786+0000 7f67f180f740 1 -- 172.21.15.94:0/3525196918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520cd30 msgr2=0x5556c521bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.786+0000 7f67f180f740 1 --2- 172.21.15.94:0/3525196918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520cd30 0x5556c521bf50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f67d8098a90 tx=0x7f67d80ba040 comp rx=0 tx=0).stop 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.786+0000 7f67f180f740 1 -- 172.21.15.94:0/3525196918 shutdown_connections 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.786+0000 7f67f180f740 1 --2- 172.21.15.94:0/3525196918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5556c5226400 0x5556c52805a0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.786+0000 7f67f180f740 1 --2- 172.21.15.94:0/3525196918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5556c5213420 0x5556c527c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.786+0000 7f67f180f740 1 --2- 172.21.15.94:0/3525196918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556c520cd30 0x5556c521bf50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.786+0000 7f67f180f740 1 --2- 172.21.15.94:0/3525196918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5556c520c380 0x5556c521ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.786+0000 7f67c7fff640 0 client.28345 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.786+0000 7f67f180f740 1 -- 172.21.15.94:0/3525196918 >> 172.21.15.94:0/3525196918 conn(0x5556c5206840 msgr2=0x5556c5207df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:06.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.787+0000 7f67f180f740 1 -- 172.21.15.94:0/3525196918 shutdown_connections 2024-02-17T12:11:06.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.787+0000 7f67f180f740 1 -- 172.21.15.94:0/3525196918 wait complete. 2024-02-17T12:11:06.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.787+0000 7f67f180f740 20 client.28345 trim_cache size 0 max 16384 2024-02-17T12:11:06.794 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:11:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:11:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "fbd78fa5-a1d7-43d4-82d6-1d8ee98274e5", 2024-02-17T12:11:06.794 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:11:06.794 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:11:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.789+0000 7f67eec40640 1 -- 172.21.15.94:0/1439946466 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f67d00a22b0 msgr2=0x7f67d00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.789+0000 7f67eec40640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f67d00a22b0 0x7f67d00a4770 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00bd40 tx=0x7f67dc0046c0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.789+0000 7f67eec40640 1 -- 172.21.15.94:0/1439946466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e81080e0 msgr2=0x7f67e819e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.789+0000 7f67eec40640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e81080e0 0x7f67e819e3b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f67d800b450 tx=0x7f67d800b910 comp rx=0 tx=0).stop 2024-02-17T12:11:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.789+0000 7f67eec40640 1 -- 172.21.15.94:0/1439946466 shutdown_connections 2024-02-17T12:11:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.789+0000 7f67eec40640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f67d00a22b0 0x7f67d00a4770 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.789+0000 7f67eec40640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f67e810d730 0x7f67e81a2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.789+0000 7f67eec40640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f67e81080e0 0x7f67e819e3b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.789+0000 7f67eec40640 1 --2- 172.21.15.94:0/1439946466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f67e8107720 0x7f67e819de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.789+0000 7f67eec40640 1 -- 172.21.15.94:0/1439946466 >> 172.21.15.94:0/1439946466 conn(0x7f67e80756c0 msgr2=0x7f67e8075ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:06.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.789+0000 7f67eec40640 1 -- 172.21.15.94:0/1439946466 shutdown_connections 2024-02-17T12:11:06.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.789+0000 7f67eec40640 1 -- 172.21.15.94:0/1439946466 wait complete. 2024-02-17T12:11:06.805 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:11:06.805 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:11:06.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.965+0000 7f6e0dfcc640 1 -- 172.21.15.94:0/2167071196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e0810dce0 msgr2=0x7f6e0811c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:06.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.965+0000 7f6e0dfcc640 1 --2- 172.21.15.94:0/2167071196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e0810dce0 0x7f6e0811c970 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009cd0 tx=0x7f6e0002c120 comp rx=0 tx=0).stop 2024-02-17T12:11:06.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.969+0000 7f6e0dfcc640 1 -- 172.21.15.94:0/2167071196 shutdown_connections 2024-02-17T12:11:06.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.969+0000 7f6e0dfcc640 1 --2- 172.21.15.94:0/2167071196 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e0810dce0 0x7f6e0811c970 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.969+0000 7f6e0dfcc640 1 --2- 172.21.15.94:0/2167071196 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e0810d260 0x7f6e0810d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.969+0000 7f6e0dfcc640 1 --2- 172.21.15.94:0/2167071196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e0810f6a0 0x7f6e0810fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.969+0000 7f6e0dfcc640 1 -- 172.21.15.94:0/2167071196 >> 172.21.15.94:0/2167071196 conn(0x7f6e0806c030 msgr2=0x7f6e0806c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:06.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.978+0000 7f6e0dfcc640 1 -- 172.21.15.94:0/2167071196 shutdown_connections 2024-02-17T12:11:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.981+0000 7f6e0dfcc640 1 -- 172.21.15.94:0/2167071196 wait complete. 2024-02-17T12:11:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.981+0000 7f6e0dfcc640 1 Processor -- start 2024-02-17T12:11:06.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.982+0000 7f6e0dfcc640 1 -- start start 2024-02-17T12:11:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.982+0000 7f6e0dfcc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e0810d260 0x7f6e081a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.982+0000 7f6e0dfcc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e0810dce0 0x7f6e081a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.982+0000 7f6e0dfcc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e0810f6a0 0x7f6e081aacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.982+0000 7f6e0dfcc640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e081ab2f0 con 0x7f6e0810dce0 2024-02-17T12:11:06.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.982+0000 7f6e0dfcc640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e081ab460 con 0x7f6e0810d260 2024-02-17T12:11:06.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.982+0000 7f6e0dfcc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e081ab620 con 0x7f6e0810f6a0 2024-02-17T12:11:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.983+0000 7f6e0d7cb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e0810f6a0 0x7f6e081aacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.984+0000 7f6e0cfca640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e0810d260 0x7f6e081a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.984+0000 7f6e0cfca640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e0810d260 0x7f6e081a64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46132/0 (socket says 172.21.15.94:46132) 2024-02-17T12:11:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.984+0000 7f6e0cfca640 1 -- 172.21.15.94:0/3519800514 learned_addr learned my addr 172.21.15.94:0/3519800514 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:06.986 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.984+0000 7f6e07fff640 1 --2- 172.21.15.94:0/3519800514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e0810dce0 0x7f6e081a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:06.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.985+0000 7f6e0d7cb640 1 -- 172.21.15.94:0/3519800514 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e0810d260 msgr2=0x7f6e081a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:06.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.985+0000 7f6e0d7cb640 1 --2- 172.21.15.94:0/3519800514 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e0810d260 0x7f6e081a64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.985+0000 7f6e0d7cb640 1 -- 172.21.15.94:0/3519800514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e0810dce0 msgr2=0x7f6e081a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:06.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.985+0000 7f6e0d7cb640 1 --2- 172.21.15.94:0/3519800514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e0810dce0 0x7f6e081a6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:06.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.986+0000 7f6e0d7cb640 1 -- 172.21.15.94:0/3519800514 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e00009900 con 0x7f6e0810f6a0 2024-02-17T12:11:06.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.993+0000 7f6e0d7cb640 1 --2- 172.21.15.94:0/3519800514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e0810f6a0 0x7f6e081aacf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6e00037040 tx=0x7f6e00034e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.994+0000 7f6e05ffb640 1 -- 172.21.15.94:0/3519800514 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00034850 con 0x7f6e0810f6a0 2024-02-17T12:11:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.994+0000 7f6e05ffb640 1 -- 172.21.15.94:0/3519800514 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6e00003e50 con 0x7f6e0810f6a0 2024-02-17T12:11:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.994+0000 7f6e05ffb640 1 -- 172.21.15.94:0/3519800514 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00040450 con 0x7f6e0810f6a0 2024-02-17T12:11:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.994+0000 7f6e0dfcc640 1 -- 172.21.15.94:0/3519800514 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e081ab8a0 con 0x7f6e0810f6a0 2024-02-17T12:11:06.996 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.995+0000 7f6e0dfcc640 1 -- 172.21.15.94:0/3519800514 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e081b3020 con 0x7f6e0810f6a0 2024-02-17T12:11:07.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.998+0000 7f6e11e01740 1 Processor -- start 2024-02-17T12:11:07.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.999+0000 7f6e05ffb640 1 -- 172.21.15.94:0/3519800514 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6e0000a040 con 0x7f6e0810f6a0 2024-02-17T12:11:07.003 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:06.999+0000 7f6e11e01740 1 -- start start 2024-02-17T12:11:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.000+0000 7f6e11e01740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b850e4d0 0x5601b850e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.000+0000 7f6e11e01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 0x5601b8517490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.000+0000 7f6e11e01740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b85179d0 0x5601b8519dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.000+0000 7f6e11e01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b847d280 con 0x5601b850edf0 2024-02-17T12:11:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.000+0000 7f6e11e01740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b847a7b0 con 0x5601b85179d0 2024-02-17T12:11:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.000+0000 7f6e11e01740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b8475f90 con 0x5601b850e4d0 2024-02-17T12:11:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.000+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b850e4d0 0x5601b850e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.000+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b850e4d0 0x5601b850e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60214/0 (socket says 172.21.15.94:60214) 2024-02-17T12:11:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.000+0000 7f6e07fff640 1 -- 172.21.15.94:0/2421113802 learned_addr learned my addr 172.21.15.94:0/2421113802 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.000+0000 7f6e07fff640 1 --2- 172.21.15.94:0/2421113802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b85179d0 0x5601b8519dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.000+0000 7f6e05ffb640 1 --2- 172.21.15.94:0/3519800514 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6df40a23c0 0x7f6df40a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.001+0000 7f6e05ffb640 1 -- 172.21.15.94:0/3519800514 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f6e000f0ce0 con 0x7f6e0810f6a0 2024-02-17T12:11:07.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.001+0000 7f6e0d7cb640 1 --2- 172.21.15.94:0/3519800514 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6df40a23c0 0x7f6df40a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.001+0000 7f6e07fff640 1 -- 172.21.15.94:0/2421113802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b85179d0 msgr2=0x5601b8519dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.001+0000 7f6e07fff640 1 --2- 172.21.15.94:0/2421113802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b85179d0 0x5601b8519dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.001+0000 7f6e07fff640 1 -- 172.21.15.94:0/2421113802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 msgr2=0x5601b8517490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:11:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.001+0000 7f6e07fff640 1 --2- 172.21.15.94:0/2421113802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 0x5601b8517490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.001+0000 7f6e07fff640 1 -- 172.21.15.94:0/2421113802 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601b8434570 con 0x5601b850e4d0 2024-02-17T12:11:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.001+0000 7f6e07fff640 1 --2- 172.21.15.94:0/2421113802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b850e4d0 0x5601b850e8b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6df8006aa0 tx=0x7f6df8009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e22748e3f9ab7805 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.002+0000 7f6e0d7cb640 1 --2- 172.21.15.94:0/3519800514 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6df40a23c0 0x7f6df40a4880 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6e000098d0 tx=0x7f6e00034f10 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.002+0000 7f6dfeffd640 1 -- 172.21.15.94:0/2421113802 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8015760 con 0x5601b850e4d0 2024-02-17T12:11:07.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.002+0000 7f6dfeffd640 1 -- 172.21.15.94:0/2421113802 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6df8015da0 con 0x5601b850e4d0 2024-02-17T12:11:07.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.003+0000 7f6e11e01740 1 -- 172.21.15.94:0/2421113802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b850e4d0 msgr2=0x5601b850e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:07.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.003+0000 7f6e11e01740 1 --2- 172.21.15.94:0/2421113802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b850e4d0 0x5601b850e8b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6df8006aa0 tx=0x7f6df8009510 comp rx=0 tx=0).stop 2024-02-17T12:11:07.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.004+0000 7f6e11e01740 1 -- 172.21.15.94:0/2421113802 shutdown_connections 2024-02-17T12:11:07.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.004+0000 7f6e11e01740 1 --2- 172.21.15.94:0/2421113802 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b85179d0 0x5601b8519dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.004+0000 7f6e11e01740 1 --2- 172.21.15.94:0/2421113802 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 0x5601b8517490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.004+0000 7f6e11e01740 1 --2- 172.21.15.94:0/2421113802 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b850e4d0 0x5601b850e8b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.004+0000 7f6e11e01740 1 -- 172.21.15.94:0/2421113802 >> 172.21.15.94:0/2421113802 conn(0x5601b8508c00 msgr2=0x5601b8508fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:07.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.005+0000 7f6e11e01740 1 -- 172.21.15.94:0/2421113802 shutdown_connections 2024-02-17T12:11:07.009 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.005+0000 7f6e11e01740 1 -- 172.21.15.94:0/2421113802 wait complete. 2024-02-17T12:11:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.006+0000 7f6e11e01740 1 -- start start 2024-02-17T12:11:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.006+0000 7f6e11e01740 1 Processor -- start 2024-02-17T12:11:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.006+0000 7f6e11e01740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 0x5601b8517230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.006+0000 7f6e11e01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 0x5601b8514630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.006+0000 7f6e11e01740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 0x5601b8514b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.006+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 0x5601b8517230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.006+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 0x5601b8517230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46150/0 (socket says 172.21.15.94:46150) 2024-02-17T12:11:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.006+0000 7f6e07fff640 1 -- 172.21.15.94:0/1914640705 learned_addr learned my addr 172.21.15.94:0/1914640705 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.006+0000 7f6e11e01740 1 -- 172.21.15.94:0/1914640705 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b8149430 con 0x5601b850edf0 2024-02-17T12:11:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.007+0000 7f6e11e01740 1 -- 172.21.15.94:0/1914640705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b8105e00 con 0x5601b850e4d0 2024-02-17T12:11:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.007+0000 7f6e11e01740 1 -- 172.21.15.94:0/1914640705 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b83fafd0 con 0x5601b85179d0 2024-02-17T12:11:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.007+0000 7f6e07fff640 1 --2- 172.21.15.94:0/1914640705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 0x5601b8514b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:07.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.007+0000 7f6e11e01740 10 client.0 _ll_register_callbacks cb 0x5601b8450fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:07.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.007+0000 7f6e07fff640 1 -- 172.21.15.94:0/1914640705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 msgr2=0x5601b8514b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:07.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.007+0000 7f6e07fff640 1 --2- 172.21.15.94:0/1914640705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 0x5601b8514b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.007+0000 7f6e07fff640 1 -- 172.21.15.94:0/1914640705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 msgr2=0x5601b8514630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:11:07.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.007+0000 7f6e07fff640 1 --2- 172.21.15.94:0/1914640705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 0x5601b8514630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.007+0000 7f6e07fff640 1 -- 172.21.15.94:0/1914640705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df8017440 con 0x5601b850e4d0 2024-02-17T12:11:07.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.007+0000 7f6e07fff640 1 --2- 172.21.15.94:0/1914640705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 0x5601b8517230 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6df80091f0 tx=0x7f6df803a9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f11e1aae1a8cf5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:07.011 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.007+0000 7f6e11e01740 1 -- 172.21.15.94:0/1914640705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601b81757e0 con 0x5601b850e4d0 2024-02-17T12:11:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.008+0000 7f6de7fff640 10 client.28382 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:11:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.008+0000 7f6de7fff640 1 -- 172.21.15.94:0/1914640705 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df804c070 con 0x5601b850e4d0 2024-02-17T12:11:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.008+0000 7f6de7fff640 1 -- 172.21.15.94:0/1914640705 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f6df8015970 con 0x5601b850e4d0 2024-02-17T12:11:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.008+0000 7f6de7fff640 1 -- 172.21.15.94:0/1914640705 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6df8015b30 con 0x5601b850e4d0 2024-02-17T12:11:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.008+0000 7f6e11e01740 10 client.28382 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.008+0000 7f6e11e01740 1 -- 172.21.15.94:0/1914640705 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x5601b8474610 con 0x5601b850e4d0 2024-02-17T12:11:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.009+0000 7f6de7fff640 1 -- 172.21.15.94:0/1914640705 <== mon.2 v2:172.21.15.125:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6df804f020 con 0x5601b850e4d0 2024-02-17T12:11:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.009+0000 7f6e11e01740 10 client.28382 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.009+0000 7f6e11e01740 10 client.28382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.009+0000 7f6e11e01740 20 client.28382 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.009+0000 7f6e11e01740 1 --2- 172.21.15.94:0/1914640705 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601b8527880 0x5601b8529ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.009+0000 7f6e11e01740 4 client.28382 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:07.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.009+0000 7f6e11e01740 1 -- 172.21.15.94:0/1914640705 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601b81757e0 con 0x5601b8527880 2024-02-17T12:11:07.013 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.009+0000 7f6e0cfca640 1 --2- 172.21.15.94:0/1914640705 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601b8527880 0x5601b8529ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.010+0000 7f6e0cfca640 1 --2- 172.21.15.94:0/1914640705 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601b8527880 0x5601b8529ca0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec140fc781e36a76 server_cookie=5a18678e080cfc0f in_seq=0 out_seq=0 2024-02-17T12:11:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6de7fff640 10 client.28382 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6de7fff640 1 -- 172.21.15.94:0/1914640705 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5601b81757e0 con 0x5601b8527880 2024-02-17T12:11:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6de7fff640 10 client.28382 handle_command_reply: tid=0 2024-02-17T12:11:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6e11e01740 1 client.28382 shutdown 2024-02-17T12:11:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6e11e01740 1 -- 172.21.15.94:0/1914640705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 msgr2=0x5601b8517230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6e11e01740 1 --2- 172.21.15.94:0/1914640705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 0x5601b8517230 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6df80091f0 tx=0x7f6df803a9f0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6e11e01740 1 -- 172.21.15.94:0/1914640705 shutdown_connections 2024-02-17T12:11:07.014 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6e11e01740 1 --2- 172.21.15.94:0/1914640705 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601b8527880 0x5601b8529ca0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6e11e01740 1 --2- 172.21.15.94:0/1914640705 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 0x5601b8514b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6e11e01740 1 --2- 172.21.15.94:0/1914640705 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 0x5601b8514630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6e11e01740 1 --2- 172.21.15.94:0/1914640705 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 0x5601b8517230 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6de7fff640 0 client.28382 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6e11e01740 1 -- 172.21.15.94:0/1914640705 >> 172.21.15.94:0/1914640705 conn(0x5601b8508c00 msgr2=0x5601b850a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6e11e01740 1 -- 172.21.15.94:0/1914640705 shutdown_connections 2024-02-17T12:11:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.011+0000 7f6e11e01740 1 -- 172.21.15.94:0/1914640705 wait complete. 2024-02-17T12:11:07.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.012+0000 7f6e11e01740 20 client.28382 trim_cache size 0 max 16384 2024-02-17T12:11:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e11e01740 1 Processor -- start 2024-02-17T12:11:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e11e01740 1 -- start start 2024-02-17T12:11:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e11e01740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 0x5601b8518dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e11e01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 0x5601b8519300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e11e01740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 0x5601b8515d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e11e01740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b850e130 con 0x5601b850edf0 2024-02-17T12:11:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e11e01740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b84fa660 con 0x5601b850e4d0 2024-02-17T12:11:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e11e01740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b8512b80 con 0x5601b85179d0 2024-02-17T12:11:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 0x5601b8518dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e0cfca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 0x5601b8519300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:07.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 0x5601b8518dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:46162/0 (socket says 172.21.15.94:46162) 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e07fff640 1 -- 172.21.15.94:0/500787496 learned_addr learned my addr 172.21.15.94:0/500787496 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e07fff640 1 --2- 172.21.15.94:0/500787496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 0x5601b8515d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e07fff640 1 -- 172.21.15.94:0/500787496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 msgr2=0x5601b8515d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e07fff640 1 --2- 172.21.15.94:0/500787496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 0x5601b8515d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e07fff640 1 -- 172.21.15.94:0/500787496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 msgr2=0x5601b8519300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e07fff640 1 --2- 172.21.15.94:0/500787496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 0x5601b8519300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e07fff640 1 -- 172.21.15.94:0/500787496 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601b8410ac0 con 0x5601b850e4d0 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.030+0000 7f6e0cfca640 1 --2- 172.21.15.94:0/500787496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 0x5601b8519300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.031+0000 7f6e07fff640 1 --2- 172.21.15.94:0/500787496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 0x5601b8518dc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6df8004670 tx=0x7f6df8007000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d607fb83f7b9f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.031+0000 7f6dfe7fc640 1 -- 172.21.15.94:0/500787496 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df804c040 con 0x5601b850e4d0 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.031+0000 7f6dfe7fc640 1 -- 172.21.15.94:0/500787496 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6df804d040 con 0x5601b850e4d0 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.031+0000 7f6dfe7fc640 1 -- 172.21.15.94:0/500787496 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df8045c70 con 0x5601b850e4d0 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.032+0000 7f6e11e01740 1 -- 172.21.15.94:0/500787496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 msgr2=0x5601b8518dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:07.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.032+0000 7f6e11e01740 1 --2- 172.21.15.94:0/500787496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 0x5601b8518dc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6df8004670 tx=0x7f6df8007000 comp rx=0 tx=0).stop 2024-02-17T12:11:07.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.032+0000 7f6e11e01740 1 -- 172.21.15.94:0/500787496 shutdown_connections 2024-02-17T12:11:07.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.032+0000 7f6e11e01740 1 --2- 172.21.15.94:0/500787496 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 0x5601b8515d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.032+0000 7f6e11e01740 1 --2- 172.21.15.94:0/500787496 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850edf0 0x5601b8519300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.032+0000 7f6e11e01740 1 --2- 172.21.15.94:0/500787496 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850e4d0 0x5601b8518dc0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.032+0000 7f6e11e01740 1 -- 172.21.15.94:0/500787496 >> 172.21.15.94:0/500787496 conn(0x5601b8508c00 msgr2=0x5601b8512f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:07.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.032+0000 7f6e11e01740 1 -- 172.21.15.94:0/500787496 shutdown_connections 2024-02-17T12:11:07.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.032+0000 7f6e11e01740 1 -- 172.21.15.94:0/500787496 wait complete. 2024-02-17T12:11:07.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.032+0000 7f6e11e01740 1 -- start start 2024-02-17T12:11:07.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.032+0000 7f6e11e01740 1 Processor -- start 2024-02-17T12:11:07.039 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.032+0000 7f6e11e01740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850e4d0 0x5601b8511340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e11e01740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850edf0 0x5601b8511880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850e4d0 0x5601b8511340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850e4d0 0x5601b8511340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48226/0 (socket says 172.21.15.94:48226) 2024-02-17T12:11:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e11e01740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 0x5601b8511dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e07fff640 1 -- 172.21.15.94:0/3624629007 learned_addr learned my addr 172.21.15.94:0/3624629007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e11e01740 1 -- 172.21.15.94:0/3624629007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b857dd20 con 0x5601b850e4d0 2024-02-17T12:11:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e11e01740 1 -- 172.21.15.94:0/3624629007 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b857de90 con 0x5601b850edf0 2024-02-17T12:11:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e0cfca640 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850edf0 0x5601b8511880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e11e01740 1 -- 172.21.15.94:0/3624629007 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601b857e050 con 0x5601b85179d0 2024-02-17T12:11:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e11e01740 10 client.0 _ll_register_callbacks cb 0x7f6df8009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e07fff640 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 0x5601b8511dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:07.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e0cfca640 1 -- 172.21.15.94:0/3624629007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 msgr2=0x5601b8511dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e0cfca640 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 0x5601b8511dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e0cfca640 1 -- 172.21.15.94:0/3624629007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850e4d0 msgr2=0x5601b8511340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e0cfca640 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850e4d0 0x5601b8511340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e0cfca640 1 -- 172.21.15.94:0/3624629007 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6df000e040 con 0x5601b850edf0 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e07fff640 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 0x5601b8511dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e11e01740 1 -- 172.21.15.94:0/3624629007 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6df8015da0 con 0x5601b850edf0 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6e0cfca640 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850edf0 0x5601b8511880 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x5601b852a550 tx=0x7f6df0016410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=761bc5835a1b23e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6de7fff640 10 client.28388 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.033+0000 7f6de7fff640 1 -- 172.21.15.94:0/3624629007 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df001b070 con 0x5601b850edf0 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.034+0000 7f6de7fff640 1 -- 172.21.15.94:0/3624629007 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f6df00146f0 con 0x5601b850edf0 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.034+0000 7f6de7fff640 1 -- 172.21.15.94:0/3624629007 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6df0014930 con 0x5601b850edf0 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.034+0000 7f6e11e01740 10 client.28388 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.034+0000 7f6e11e01740 1 -- 172.21.15.94:0/3624629007 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x5601b8474610 con 0x5601b850edf0 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.034+0000 7f6de7fff640 1 -- 172.21.15.94:0/3624629007 <== mon.2 v2:172.21.15.125:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6df0011040 con 0x5601b850edf0 2024-02-17T12:11:07.041 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.034+0000 7f6e11e01740 10 client.28388 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.034+0000 7f6e11e01740 10 client.28388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.034+0000 7f6e11e01740 20 client.28388 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.034+0000 7f6e11e01740 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601b857f6b0 0x5601b8581aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.034+0000 7f6e11e01740 4 client.28388 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.034+0000 7f6e11e01740 1 -- 172.21.15.94:0/3624629007 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5601b81b2720 con 0x5601b857f6b0 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.035+0000 7f6e07fff640 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601b857f6b0 0x5601b8581aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.035+0000 7f6de7fff640 10 client.28388 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.035+0000 7f6e07fff640 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601b857f6b0 0x5601b8581aa0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e747f6d5ae205996 server_cookie=f4fc7c9f84c57209 in_seq=0 out_seq=0 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.035+0000 7f6de7fff640 1 -- 172.21.15.94:0/3624629007 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5601b81b2720 con 0x5601b857f6b0 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.035+0000 7f6de7fff640 10 client.28388 handle_command_reply: tid=0 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.036+0000 7f6e11e01740 1 client.28388 shutdown 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.036+0000 7f6e11e01740 1 -- 172.21.15.94:0/3624629007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850edf0 msgr2=0x5601b8511880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.036+0000 7f6e11e01740 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850edf0 0x5601b8511880 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x5601b852a550 tx=0x7f6df0016410 comp rx=0 tx=0).stop 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.036+0000 7f6e11e01740 1 -- 172.21.15.94:0/3624629007 shutdown_connections 2024-02-17T12:11:07.042 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.036+0000 7f6e11e01740 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5601b857f6b0 0x5601b8581aa0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.036+0000 7f6e11e01740 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5601b85179d0 0x5601b8511dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.036+0000 7f6de7fff640 0 client.28388 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:07.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.036+0000 7f6e11e01740 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5601b850edf0 0x5601b8511880 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.036+0000 7f6e11e01740 1 --2- 172.21.15.94:0/3624629007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5601b850e4d0 0x5601b8511340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.036+0000 7f6e11e01740 1 -- 172.21.15.94:0/3624629007 >> 172.21.15.94:0/3624629007 conn(0x5601b8508c00 msgr2=0x5601b85074d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:07.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.036+0000 7f6e11e01740 1 -- 172.21.15.94:0/3624629007 shutdown_connections 2024-02-17T12:11:07.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.036+0000 7f6e11e01740 1 -- 172.21.15.94:0/3624629007 wait complete. 2024-02-17T12:11:07.043 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.036+0000 7f6e11e01740 20 client.28388 trim_cache size 0 max 16384 2024-02-17T12:11:07.043 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:11:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:11:07.043 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:11:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: "fbd78fa5-a1d7-43d4-82d6-1d8ee98274e5": { 2024-02-17T12:11:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:11:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "fbd78fa5-a1d7-43d4-82d6-1d8ee98274e5", 2024-02-17T12:11:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:11:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:11:07.044 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:11:07.044 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:11:07.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.037+0000 7f6e0dfcc640 1 -- 172.21.15.94:0/3519800514 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6df40a23c0 msgr2=0x7f6df40a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:07.045 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.037+0000 7f6e0dfcc640 1 --2- 172.21.15.94:0/3519800514 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6df40a23c0 0x7f6df40a4880 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6e000098d0 tx=0x7f6e00034f10 comp rx=0 tx=0).stop 2024-02-17T12:11:07.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.037+0000 7f6e0dfcc640 1 -- 172.21.15.94:0/3519800514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e0810f6a0 msgr2=0x7f6e081aacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:07.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.037+0000 7f6e0dfcc640 1 --2- 172.21.15.94:0/3519800514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e0810f6a0 0x7f6e081aacf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6e00037040 tx=0x7f6e00034e20 comp rx=0 tx=0).stop 2024-02-17T12:11:07.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.037+0000 7f6e0dfcc640 1 -- 172.21.15.94:0/3519800514 shutdown_connections 2024-02-17T12:11:07.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.037+0000 7f6e0dfcc640 1 --2- 172.21.15.94:0/3519800514 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6df40a23c0 0x7f6df40a4880 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.037+0000 7f6e0dfcc640 1 --2- 172.21.15.94:0/3519800514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e0810f6a0 0x7f6e081aacf0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.037+0000 7f6e0dfcc640 1 --2- 172.21.15.94:0/3519800514 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6e0810dce0 0x7f6e081a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.037+0000 7f6e0dfcc640 1 --2- 172.21.15.94:0/3519800514 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6e0810d260 0x7f6e081a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:07.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.037+0000 7f6e0dfcc640 1 -- 172.21.15.94:0/3519800514 >> 172.21.15.94:0/3519800514 conn(0x7f6e0806c030 msgr2=0x7f6e080705b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:07.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.037+0000 7f6e0dfcc640 1 -- 172.21.15.94:0/3519800514 shutdown_connections 2024-02-17T12:11:07.046 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:07.037+0000 7f6e0dfcc640 1 -- 172.21.15.94:0/3519800514 wait complete. 2024-02-17T12:11:07.052 INFO:tasks.cephfs.filesystem:scrub status for tag:fbd78fa5-a1d7-43d4-82d6-1d8ee98274e5 - {'path': '/', 'tag': 'fbd78fa5-a1d7-43d4-82d6-1d8ee98274e5', 'options': 'recursive,force'} 2024-02-17T12:11:07.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:07 smithi189 ceph-mon[36588]: pgmap v406: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-02-17T12:11:07.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:07 smithi125 ceph-mon[36819]: pgmap v406: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-02-17T12:11:07.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:07 smithi094 ceph-mon[32499]: pgmap v406: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 1.46k op/s 2024-02-17T12:11:08.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:08 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:11:08.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:08 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:11:08.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:08 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:11:08.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:08 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:11:08.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:08 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:11:08.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:08 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:11:08.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:08 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:11:08.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:08 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:11:08.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:08 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:11:08.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:08 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:11:08.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:08 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:11:08.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:08 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:11:08.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:08 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:11:08.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:08 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:11:08.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:08 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:11:09.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:09 smithi189 ceph-mon[36588]: pgmap v407: 97 pgs: 97 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 15 MiB/s wr, 1.65k op/s 2024-02-17T12:11:09.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:09 smithi125 ceph-mon[36819]: pgmap v407: 97 pgs: 97 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 15 MiB/s wr, 1.65k op/s 2024-02-17T12:11:09.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:09 smithi094 ceph-mon[32499]: pgmap v407: 97 pgs: 97 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 15 MiB/s wr, 1.65k op/s 2024-02-17T12:11:10.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 770.0 s, 1659.2 tps, lat 60.935 ms stddev 70.675 2024-02-17T12:11:11.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:11 smithi189 ceph-mon[36588]: osdmap e102: 12 total, 12 up, 12 in 2024-02-17T12:11:11.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:11 smithi189 ceph-mon[36588]: pgmap v409: 97 pgs: 97 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 18 MiB/s wr, 1.95k op/s 2024-02-17T12:11:11.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:11 smithi125 ceph-mon[36819]: osdmap e102: 12 total, 12 up, 12 in 2024-02-17T12:11:11.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:11 smithi125 ceph-mon[36819]: pgmap v409: 97 pgs: 97 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 18 MiB/s wr, 1.95k op/s 2024-02-17T12:11:11.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:11 smithi094 ceph-mon[32499]: osdmap e102: 12 total, 12 up, 12 in 2024-02-17T12:11:11.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:11 smithi094 ceph-mon[32499]: pgmap v409: 97 pgs: 97 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 18 MiB/s wr, 1.95k op/s 2024-02-17T12:11:13.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:13 smithi189 ceph-mon[36588]: pgmap v410: 97 pgs: 97 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 15 MiB/s wr, 1.68k op/s 2024-02-17T12:11:13.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:13 smithi125 ceph-mon[36819]: pgmap v410: 97 pgs: 97 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 15 MiB/s wr, 1.68k op/s 2024-02-17T12:11:13.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:13 smithi094 ceph-mon[32499]: pgmap v410: 97 pgs: 97 active+clean; 5.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 15 MiB/s wr, 1.68k op/s 2024-02-17T12:11:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.670+0000 7fc039b3a640 1 -- 172.21.15.94:0/4000007430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc034102d10 msgr2=0x7fc03410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.670+0000 7fc039b3a640 1 --2- 172.21.15.94:0/4000007430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc034102d10 0x7fc03410b1f0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fc028009900 tx=0x7fc02802bd60 comp rx=0 tx=0).stop 2024-02-17T12:11:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.670+0000 7fc039b3a640 1 -- 172.21.15.94:0/4000007430 shutdown_connections 2024-02-17T12:11:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.670+0000 7fc039b3a640 1 --2- 172.21.15.94:0/4000007430 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc03410b730 0x7fc03410dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.670+0000 7fc039b3a640 1 --2- 172.21.15.94:0/4000007430 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc034102d10 0x7fc03410b1f0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.670+0000 7fc039b3a640 1 --2- 172.21.15.94:0/4000007430 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0341023f0 0x7fc0341027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.670+0000 7fc039b3a640 1 -- 172.21.15.94:0/4000007430 >> 172.21.15.94:0/4000007430 conn(0x7fc0340771b0 msgr2=0x7fc0340775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.671+0000 7fc039b3a640 1 -- 172.21.15.94:0/4000007430 shutdown_connections 2024-02-17T12:11:14.673 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.671+0000 7fc039b3a640 1 -- 172.21.15.94:0/4000007430 wait complete. 2024-02-17T12:11:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.671+0000 7fc039b3a640 1 Processor -- start 2024-02-17T12:11:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.672+0000 7fc039b3a640 1 -- start start 2024-02-17T12:11:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.672+0000 7fc039b3a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0341023f0 0x7fc03419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.672+0000 7fc039b3a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc034102d10 0x7fc03419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.672+0000 7fc039b3a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc03410b730 0x7fc034199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.672+0000 7fc039b3a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc034199e30 con 0x7fc034102d10 2024-02-17T12:11:14.674 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.672+0000 7fc039b3a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc034199fa0 con 0x7fc0341023f0 2024-02-17T12:11:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.672+0000 7fc039b3a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03419a160 con 0x7fc03410b730 2024-02-17T12:11:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.672+0000 7fc0337fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0341023f0 0x7fc03419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.672+0000 7fc0337fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0341023f0 0x7fc03419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34666/0 (socket says 172.21.15.94:34666) 2024-02-17T12:11:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.672+0000 7fc0337fe640 1 -- 172.21.15.94:0/3566671525 learned_addr learned my addr 172.21.15.94:0/3566671525 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:14.675 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.672+0000 7fc033fff640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc03410b730 0x7fc034199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.673+0000 7fc032ffd640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc034102d10 0x7fc03419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.673+0000 7fc0337fe640 1 -- 172.21.15.94:0/3566671525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc03410b730 msgr2=0x7fc034199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.673+0000 7fc0337fe640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc03410b730 0x7fc034199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.673+0000 7fc0337fe640 1 -- 172.21.15.94:0/3566671525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc034102d10 msgr2=0x7fc03419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.673+0000 7fc0337fe640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc034102d10 0x7fc03419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.673+0000 7fc0337fe640 1 -- 172.21.15.94:0/3566671525 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc028009560 con 0x7fc0341023f0 2024-02-17T12:11:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.673+0000 7fc0337fe640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0341023f0 0x7fc03419f620 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc02400d950 tx=0x7fc02400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.674+0000 7fc030ff9640 1 -- 172.21.15.94:0/3566671525 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc024009c30 con 0x7fc0341023f0 2024-02-17T12:11:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.674+0000 7fc030ff9640 1 -- 172.21.15.94:0/3566671525 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc024009d90 con 0x7fc0341023f0 2024-02-17T12:11:14.676 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.674+0000 7fc039b3a640 1 -- 172.21.15.94:0/3566671525 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03419a440 con 0x7fc0341023f0 2024-02-17T12:11:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.674+0000 7fc030ff9640 1 -- 172.21.15.94:0/3566671525 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02400f7e0 con 0x7fc0341023f0 2024-02-17T12:11:14.677 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.674+0000 7fc039b3a640 1 -- 172.21.15.94:0/3566671525 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0341a6650 con 0x7fc0341023f0 2024-02-17T12:11:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.676+0000 7fc030ff9640 1 -- 172.21.15.94:0/3566671525 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fc024010ad0 con 0x7fc0341023f0 2024-02-17T12:11:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.677+0000 7fc030ff9640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc01c0a23d0 0x7fc01c0a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.677+0000 7fc030ff9640 1 -- 172.21.15.94:0/3566671525 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc0240ce070 con 0x7fc0341023f0 2024-02-17T12:11:14.678 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.677+0000 7fc032ffd640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc01c0a23d0 0x7fc01c0a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.677+0000 7fc03c7e3740 1 Processor -- start 2024-02-17T12:11:14.679 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.677+0000 7fc03c7e3740 1 -- start start 2024-02-17T12:11:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.678+0000 7fc03c7e3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 0x5622c5c14820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.678+0000 7fc03c7e3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c14d60 0x5622c5c151e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.678+0000 7fc03c7e3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c1e2e0 0x5622c5c206d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.678+0000 7fc0337fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 0x5622c5c14820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.678+0000 7fc0337fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 0x5622c5c14820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:48692/0 (socket says 172.21.15.94:48692) 2024-02-17T12:11:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.678+0000 7fc0337fe640 1 -- 172.21.15.94:0/174960355 learned_addr learned my addr 172.21.15.94:0/174960355 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.678+0000 7fc03c7e3740 1 -- 172.21.15.94:0/174960355 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c5b839f0 con 0x5622c5c14d60 2024-02-17T12:11:14.680 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.678+0000 7fc03c7e3740 1 -- 172.21.15.94:0/174960355 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c5b80f20 con 0x5622c5c1e2e0 2024-02-17T12:11:14.681 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.678+0000 7fc03c7e3740 1 -- 172.21.15.94:0/174960355 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c5b7c700 con 0x5622c5c15ad0 2024-02-17T12:11:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.678+0000 7fc032ffd640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc01c0a23d0 0x7fc01c0a4890 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc03419aae0 tx=0x7fc028004740 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.678+0000 7fc0337fe640 1 -- 172.21.15.94:0/174960355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c1e2e0 msgr2=0x5622c5c206d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:11:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.678+0000 7fc0337fe640 1 --2- 172.21.15.94:0/174960355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c1e2e0 0x5622c5c206d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.678+0000 7fc0337fe640 1 -- 172.21.15.94:0/174960355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c14d60 msgr2=0x5622c5c151e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.679+0000 7fc0337fe640 1 --2- 172.21.15.94:0/174960355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c14d60 0x5622c5c151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.679+0000 7fc0337fe640 1 -- 172.21.15.94:0/174960355 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622c5b127b0 con 0x5622c5c15ad0 2024-02-17T12:11:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.679+0000 7fc0337fe640 1 --2- 172.21.15.94:0/174960355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 0x5622c5c14820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc0240a2c30 tx=0x7fc02409b0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b23a97f980ce316 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.679+0000 7fc019ffb640 1 -- 172.21.15.94:0/174960355 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240ba050 con 0x5622c5c15ad0 2024-02-17T12:11:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.679+0000 7fc019ffb640 1 -- 172.21.15.94:0/174960355 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0240a3160 con 0x5622c5c15ad0 2024-02-17T12:11:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.680+0000 7fc019ffb640 1 -- 172.21.15.94:0/174960355 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240a2f30 con 0x5622c5c15ad0 2024-02-17T12:11:14.682 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.680+0000 7fc03c7e3740 1 -- 172.21.15.94:0/174960355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 msgr2=0x5622c5c14820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.680+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/174960355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 0x5622c5c14820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc0240a2c30 tx=0x7fc02409b0c0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.680+0000 7fc03c7e3740 1 -- 172.21.15.94:0/174960355 shutdown_connections 2024-02-17T12:11:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.680+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/174960355 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c1e2e0 0x5622c5c206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.680+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/174960355 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c14d60 0x5622c5c151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.680+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/174960355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 0x5622c5c14820 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.680+0000 7fc03c7e3740 1 -- 172.21.15.94:0/174960355 >> 172.21.15.94:0/174960355 conn(0x5622c5bdc550 msgr2=0x5622c5c0fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.680+0000 7fc03c7e3740 1 -- 172.21.15.94:0/174960355 shutdown_connections 2024-02-17T12:11:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.680+0000 7fc03c7e3740 1 -- 172.21.15.94:0/174960355 wait complete. 2024-02-17T12:11:14.683 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.680+0000 7fc03c7e3740 1 -- start start 2024-02-17T12:11:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.680+0000 7fc03c7e3740 1 Processor -- start 2024-02-17T12:11:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc03c7e3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 0x5622c5c1b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc03c7e3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c15ad0 0x5622c5c1b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc03c7e3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c1e2e0 0x5622c5c1bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.685 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc0337fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 0x5622c5c1b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc0337fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 0x5622c5c1b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:48700/0 (socket says 172.21.15.94:48700) 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc03c7e3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c5b21bf0 con 0x5622c5c1e2e0 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc0337fe640 1 -- 172.21.15.94:0/1788119195 learned_addr learned my addr 172.21.15.94:0/1788119195 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc03c7e3740 1 -- 172.21.15.94:0/1788119195 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c5850250 con 0x5622c5c15ad0 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc033fff640 1 --2- 172.21.15.94:0/1788119195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c1e2e0 0x5622c5c1bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc03c7e3740 1 -- 172.21.15.94:0/1788119195 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c5b02200 con 0x5622c5c14d60 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc032ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c15ad0 0x5622c5c1b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc03c7e3740 10 client.0 _ll_register_callbacks cb 0x5622c5b70d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc032ffd640 1 -- 172.21.15.94:0/1788119195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 msgr2=0x5622c5c1b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc032ffd640 1 --2- 172.21.15.94:0/1788119195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 0x5622c5c1b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc032ffd640 1 -- 172.21.15.94:0/1788119195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c1e2e0 msgr2=0x5622c5c1bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc032ffd640 1 --2- 172.21.15.94:0/1788119195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c1e2e0 0x5622c5c1bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc032ffd640 1 -- 172.21.15.94:0/1788119195 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0341a6650 con 0x5622c5c15ad0 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.681+0000 7fc0337fe640 1 --2- 172.21.15.94:0/1788119195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 0x5622c5c1b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:14.686 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc032ffd640 1 --2- 172.21.15.94:0/1788119195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c15ad0 0x5622c5c1b7c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc028009230 tx=0x7fc028004c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=691c9d7006da58a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc012ffd640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc012ffd640 1 -- 172.21.15.94:0/1788119195 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028005200 con 0x5622c5c15ad0 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc012ffd640 1 -- 172.21.15.94:0/1788119195 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc028046050 con 0x5622c5c15ad0 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc03c7e3740 1 -- 172.21.15.94:0/1788119195 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5622c587c600 con 0x5622c5c15ad0 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc012ffd640 1 -- 172.21.15.94:0/1788119195 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc02802cd20 con 0x5622c5c15ad0 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc03c7e3740 10 client.28397 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc03c7e3740 1 -- 172.21.15.94:0/1788119195 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5622c5b7ad80 con 0x5622c5c15ad0 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc012ffd640 1 -- 172.21.15.94:0/1788119195 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5622c587c600 con 0x5622c5c15ad0 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc012ffd640 1 -- 172.21.15.94:0/1788119195 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc02803c040 con 0x5622c5c15ad0 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc03c7e3740 10 client.28397 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc03c7e3740 10 client.28397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc03c7e3740 20 client.28397 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/1788119195 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5622c5c2e6f0 0x5622c5c30b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc03c7e3740 4 client.28397 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc03c7e3740 1 -- 172.21.15.94:0/1788119195 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622c58cd1b0 con 0x5622c5c2e6f0 2024-02-17T12:11:14.687 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.682+0000 7fc0337fe640 1 --2- 172.21.15.94:0/1788119195 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5622c5c2e6f0 0x5622c5c30b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.683+0000 7fc012ffd640 10 client.28397 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.683+0000 7fc0337fe640 1 --2- 172.21.15.94:0/1788119195 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5622c5c2e6f0 0x5622c5c30b10 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7980cbc707838166 server_cookie=5aec2a4f21ec4d58 in_seq=0 out_seq=0 2024-02-17T12:11:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.683+0000 7fc012ffd640 1 -- 172.21.15.94:0/1788119195 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5622c58cd1b0 con 0x5622c5c2e6f0 2024-02-17T12:11:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.683+0000 7fc012ffd640 10 client.28397 handle_command_reply: tid=0 2024-02-17T12:11:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.683+0000 7fc03c7e3740 1 client.28397 shutdown 2024-02-17T12:11:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.684+0000 7fc03c7e3740 1 -- 172.21.15.94:0/1788119195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c15ad0 msgr2=0x5622c5c1b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.684+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/1788119195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c15ad0 0x5622c5c1b7c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc028009230 tx=0x7fc028004c10 comp rx=0 tx=0).stop 2024-02-17T12:11:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.684+0000 7fc03c7e3740 1 -- 172.21.15.94:0/1788119195 shutdown_connections 2024-02-17T12:11:14.688 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.684+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/1788119195 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5622c5c2e6f0 0x5622c5c30b10 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.684+0000 7fc012ffd640 0 client.28397 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.684+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/1788119195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c1e2e0 0x5622c5c1bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.684+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/1788119195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c15ad0 0x5622c5c1b7c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.684+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/1788119195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 0x5622c5c1b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.684+0000 7fc03c7e3740 1 -- 172.21.15.94:0/1788119195 >> 172.21.15.94:0/1788119195 conn(0x5622c5bdc550 msgr2=0x5622c5c117a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.684+0000 7fc03c7e3740 1 -- 172.21.15.94:0/1788119195 shutdown_connections 2024-02-17T12:11:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.684+0000 7fc03c7e3740 1 -- 172.21.15.94:0/1788119195 wait complete. 2024-02-17T12:11:14.689 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.684+0000 7fc03c7e3740 20 client.28397 trim_cache size 0 max 16384 2024-02-17T12:11:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc03c7e3740 1 Processor -- start 2024-02-17T12:11:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc03c7e3740 1 -- start start 2024-02-17T12:11:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc03c7e3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c14d60 0x5622c5c1d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc03c7e3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 0x5622c5c1bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc03c7e3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c1e2e0 0x5622c5c19b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc03c7e3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c5c0b180 con 0x5622c5c14d60 2024-02-17T12:11:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc03c7e3740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c5c10a50 con 0x5622c5c1e2e0 2024-02-17T12:11:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc03c7e3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c5c160c0 con 0x5622c5c15ad0 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc032ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 0x5622c5c1bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc032ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 0x5622c5c1bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:48714/0 (socket says 172.21.15.94:48714) 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc032ffd640 1 -- 172.21.15.94:0/4091053484 learned_addr learned my addr 172.21.15.94:0/4091053484 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc0337fe640 1 --2- 172.21.15.94:0/4091053484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c14d60 0x5622c5c1d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc033fff640 1 --2- 172.21.15.94:0/4091053484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c1e2e0 0x5622c5c19b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc032ffd640 1 -- 172.21.15.94:0/4091053484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c1e2e0 msgr2=0x5622c5c19b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc032ffd640 1 --2- 172.21.15.94:0/4091053484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c1e2e0 0x5622c5c19b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc032ffd640 1 -- 172.21.15.94:0/4091053484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c14d60 msgr2=0x5622c5c1d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc032ffd640 1 --2- 172.21.15.94:0/4091053484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c14d60 0x5622c5c1d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc032ffd640 1 -- 172.21.15.94:0/4091053484 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622c5b177b0 con 0x5622c5c15ad0 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.705+0000 7fc0337fe640 1 --2- 172.21.15.94:0/4091053484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c14d60 0x5622c5c1d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.706+0000 7fc032ffd640 1 --2- 172.21.15.94:0/4091053484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 0x5622c5c1bb20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc02802c2a0 tx=0x7fc028005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d644718c75e72463 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.706+0000 7fc0197fa640 1 -- 172.21.15.94:0/4091053484 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028042070 con 0x5622c5c15ad0 2024-02-17T12:11:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.706+0000 7fc0197fa640 1 -- 172.21.15.94:0/4091053484 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0280054e0 con 0x5622c5c15ad0 2024-02-17T12:11:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.706+0000 7fc0197fa640 1 -- 172.21.15.94:0/4091053484 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc028034600 con 0x5622c5c15ad0 2024-02-17T12:11:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.707+0000 7fc03c7e3740 1 -- 172.21.15.94:0/4091053484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 msgr2=0x5622c5c1bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.707+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/4091053484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 0x5622c5c1bb20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc02802c2a0 tx=0x7fc028005e40 comp rx=0 tx=0).stop 2024-02-17T12:11:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.707+0000 7fc03c7e3740 1 -- 172.21.15.94:0/4091053484 shutdown_connections 2024-02-17T12:11:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.707+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/4091053484 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c1e2e0 0x5622c5c19b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.707+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/4091053484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c15ad0 0x5622c5c1bb20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.707+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/4091053484 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c14d60 0x5622c5c1d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.707+0000 7fc03c7e3740 1 -- 172.21.15.94:0/4091053484 >> 172.21.15.94:0/4091053484 conn(0x5622c5bdc550 msgr2=0x5622c5c35470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.707+0000 7fc03c7e3740 1 -- 172.21.15.94:0/4091053484 shutdown_connections 2024-02-17T12:11:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.707+0000 7fc03c7e3740 1 -- 172.21.15.94:0/4091053484 wait complete. 2024-02-17T12:11:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.707+0000 7fc03c7e3740 1 -- start start 2024-02-17T12:11:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc03c7e3740 1 Processor -- start 2024-02-17T12:11:14.713 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc03c7e3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 0x5622c5c22b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc03c7e3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c15ad0 0x5622c5c230d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc03c7e3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c1e2e0 0x5622c5c23610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc03c7e3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c5c25b40 con 0x5622c5c1e2e0 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc03c7e3740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c5c83bf0 con 0x5622c5c15ad0 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc03c7e3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c5c83d80 con 0x5622c5c14d60 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc03c7e3740 10 client.0 _ll_register_callbacks cb 0x7fc024009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc0337fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 0x5622c5c22b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc0337fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 0x5622c5c22b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:48730/0 (socket says 172.21.15.94:48730) 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc0337fe640 1 -- 172.21.15.94:0/540809907 learned_addr learned my addr 172.21.15.94:0/540809907 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc033fff640 1 --2- 172.21.15.94:0/540809907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c1e2e0 0x5622c5c23610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc032ffd640 1 --2- 172.21.15.94:0/540809907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c15ad0 0x5622c5c230d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc0337fe640 1 -- 172.21.15.94:0/540809907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c15ad0 msgr2=0x5622c5c230d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc0337fe640 1 --2- 172.21.15.94:0/540809907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c15ad0 0x5622c5c230d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc0337fe640 1 -- 172.21.15.94:0/540809907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c1e2e0 msgr2=0x5622c5c23610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc0337fe640 1 --2- 172.21.15.94:0/540809907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c1e2e0 0x5622c5c23610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.708+0000 7fc0337fe640 1 -- 172.21.15.94:0/540809907 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622c5b127b0 con 0x5622c5c14d60 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.709+0000 7fc0337fe640 1 --2- 172.21.15.94:0/540809907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 0x5622c5c22b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc02409a950 tx=0x7fc02409a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5ed34835872ee41 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.709+0000 7fc012ffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.709+0000 7fc012ffd640 1 -- 172.21.15.94:0/540809907 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0240ba040 con 0x5622c5c14d60 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.709+0000 7fc03c7e3740 1 -- 172.21.15.94:0/540809907 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc0240a3160 con 0x5622c5c14d60 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.709+0000 7fc012ffd640 1 -- 172.21.15.94:0/540809907 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fc0240a2cd0 con 0x5622c5c14d60 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.709+0000 7fc012ffd640 1 -- 172.21.15.94:0/540809907 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0240a2e90 con 0x5622c5c14d60 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.709+0000 7fc03c7e3740 10 client.28372 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.709+0000 7fc03c7e3740 1 -- 172.21.15.94:0/540809907 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5622c5b7ad80 con 0x5622c5c14d60 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.709+0000 7fc012ffd640 1 -- 172.21.15.94:0/540809907 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0240a3160 con 0x5622c5c14d60 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.710+0000 7fc012ffd640 1 -- 172.21.15.94:0/540809907 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fc024023070 con 0x5622c5c14d60 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.711+0000 7fc03c7e3740 10 client.28372 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.711+0000 7fc03c7e3740 10 client.28372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.711+0000 7fc03c7e3740 20 client.28372 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.711+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/540809907 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5622c5c2e6f0 0x5622c5c87340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.711+0000 7fc03c7e3740 4 client.28372 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:11:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.711+0000 7fc03c7e3740 1 -- 172.21.15.94:0/540809907 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5622c58b9550 con 0x5622c5c2e6f0 2024-02-17T12:11:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.711+0000 7fc032ffd640 1 --2- 172.21.15.94:0/540809907 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5622c5c2e6f0 0x5622c5c87340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.712+0000 7fc012ffd640 10 client.28372 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.712+0000 7fc032ffd640 1 --2- 172.21.15.94:0/540809907 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5622c5c2e6f0 0x5622c5c87340 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=708212f1954cc9f5 server_cookie=9394b93a43c8dd56 in_seq=0 out_seq=0 2024-02-17T12:11:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc012ffd640 1 -- 172.21.15.94:0/540809907 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5622c58b9550 con 0x5622c5c2e6f0 2024-02-17T12:11:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc012ffd640 10 client.28372 handle_command_reply: tid=0 2024-02-17T12:11:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc03c7e3740 1 client.28372 shutdown 2024-02-17T12:11:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc03c7e3740 1 -- 172.21.15.94:0/540809907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 msgr2=0x5622c5c22b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/540809907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 0x5622c5c22b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc02409a950 tx=0x7fc02409a8c0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc03c7e3740 1 -- 172.21.15.94:0/540809907 shutdown_connections 2024-02-17T12:11:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/540809907 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5622c5c2e6f0 0x5622c5c87340 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/540809907 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5622c5c1e2e0 0x5622c5c23610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/540809907 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5622c5c15ad0 0x5622c5c230d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc03c7e3740 1 --2- 172.21.15.94:0/540809907 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5622c5c14d60 0x5622c5c22b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc03c7e3740 1 -- 172.21.15.94:0/540809907 >> 172.21.15.94:0/540809907 conn(0x5622c5bdc550 msgr2=0x5622c5c19750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc012ffd640 0 client.28372 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc03c7e3740 1 -- 172.21.15.94:0/540809907 shutdown_connections 2024-02-17T12:11:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc03c7e3740 1 -- 172.21.15.94:0/540809907 wait complete. 2024-02-17T12:11:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.731+0000 7fc03c7e3740 20 client.28372 trim_cache size 0 max 16384 2024-02-17T12:11:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:11:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:11:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:11:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:11:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.732+0000 7fc039b3a640 1 -- 172.21.15.94:0/3566671525 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc01c0a23d0 msgr2=0x7fc01c0a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.732+0000 7fc039b3a640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc01c0a23d0 0x7fc01c0a4890 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc03419aae0 tx=0x7fc028004740 comp rx=0 tx=0).stop 2024-02-17T12:11:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.732+0000 7fc039b3a640 1 -- 172.21.15.94:0/3566671525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0341023f0 msgr2=0x7fc03419f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.732+0000 7fc039b3a640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0341023f0 0x7fc03419f620 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc02400d950 tx=0x7fc02400de10 comp rx=0 tx=0).stop 2024-02-17T12:11:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.732+0000 7fc039b3a640 1 -- 172.21.15.94:0/3566671525 shutdown_connections 2024-02-17T12:11:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.732+0000 7fc039b3a640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fc01c0a23d0 0x7fc01c0a4890 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.736 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.732+0000 7fc039b3a640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc03410b730 0x7fc034199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.732+0000 7fc039b3a640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fc034102d10 0x7fc03419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.732+0000 7fc039b3a640 1 --2- 172.21.15.94:0/3566671525 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fc0341023f0 0x7fc03419f620 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.732+0000 7fc039b3a640 1 -- 172.21.15.94:0/3566671525 >> 172.21.15.94:0/3566671525 conn(0x7fc0340771b0 msgr2=0x7fc034107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.732+0000 7fc039b3a640 1 -- 172.21.15.94:0/3566671525 shutdown_connections 2024-02-17T12:11:14.737 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:14.732+0000 7fc039b3a640 1 -- 172.21.15.94:0/3566671525 wait complete. 2024-02-17T12:11:15.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 775.0 s, 1849.6 tps, lat 53.573 ms stddev 59.556 2024-02-17T12:11:15.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:15 smithi189 ceph-mon[36588]: pgmap v411: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.7 MiB/s wr, 800 op/s 2024-02-17T12:11:15.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:15 smithi125 ceph-mon[36819]: pgmap v411: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.7 MiB/s wr, 800 op/s 2024-02-17T12:11:15.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:15 smithi094 ceph-mon[32499]: pgmap v411: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.7 MiB/s wr, 800 op/s 2024-02-17T12:11:17.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:17 smithi189 ceph-mon[36588]: pgmap v412: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.9 MiB/s wr, 831 op/s 2024-02-17T12:11:17.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:17 smithi125 ceph-mon[36819]: pgmap v412: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.9 MiB/s wr, 831 op/s 2024-02-17T12:11:17.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:17 smithi094 ceph-mon[32499]: pgmap v412: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 6.9 MiB/s wr, 831 op/s 2024-02-17T12:11:19.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:19 smithi189 ceph-mon[36588]: pgmap v413: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.8 MiB/s wr, 640 op/s 2024-02-17T12:11:19.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:19 smithi125 ceph-mon[36819]: pgmap v413: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.8 MiB/s wr, 640 op/s 2024-02-17T12:11:19.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:19 smithi094 ceph-mon[32499]: pgmap v413: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 5.8 MiB/s wr, 640 op/s 2024-02-17T12:11:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 780.0 s, 1357.6 tps, lat 74.185 ms stddev 79.113 2024-02-17T12:11:21.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:21 smithi189 ceph-mon[36588]: pgmap v414: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-02-17T12:11:21.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:21 smithi125 ceph-mon[36819]: pgmap v414: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-02-17T12:11:21.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:21 smithi094 ceph-mon[32499]: pgmap v414: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.4 MiB/s wr, 577 op/s 2024-02-17T12:11:23.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:23 smithi189 ceph-mon[36588]: pgmap v415: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.8 MiB/s wr, 514 op/s 2024-02-17T12:11:23.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:23 smithi125 ceph-mon[36819]: pgmap v415: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.8 MiB/s wr, 514 op/s 2024-02-17T12:11:23.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:23 smithi094 ceph-mon[32499]: pgmap v415: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 4.8 MiB/s wr, 514 op/s 2024-02-17T12:11:25.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 785.0 s, 1459.6 tps, lat 68.460 ms stddev 76.999 2024-02-17T12:11:25.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:25 smithi189 ceph-mon[36588]: pgmap v416: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.2 MiB/s wr, 778 op/s 2024-02-17T12:11:25.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:25 smithi125 ceph-mon[36819]: pgmap v416: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.2 MiB/s wr, 778 op/s 2024-02-17T12:11:25.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:25 smithi094 ceph-mon[32499]: pgmap v416: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 7.2 MiB/s wr, 778 op/s 2024-02-17T12:11:27.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:27 smithi189 ceph-mon[36588]: pgmap v417: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.2 MiB/s wr, 578 op/s 2024-02-17T12:11:27.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:27 smithi125 ceph-mon[36819]: pgmap v417: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.2 MiB/s wr, 578 op/s 2024-02-17T12:11:27.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:27 smithi094 ceph-mon[32499]: pgmap v417: 97 pgs: 97 active+clean; 5.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.2 MiB/s wr, 578 op/s 2024-02-17T12:11:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.839+0000 7f31016fb640 1 -- 172.21.15.94:0/665614344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30fc101390 msgr2=0x7f30fc101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.839+0000 7f31016fb640 1 --2- 172.21.15.94:0/665614344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30fc101390 0x7f30fc101770 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009900 tx=0x7f30ec02bc30 comp rx=0 tx=0).stop 2024-02-17T12:11:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 -- 172.21.15.94:0/665614344 shutdown_connections 2024-02-17T12:11:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 --2- 172.21.15.94:0/665614344 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30fc106460 0x7f30fc10e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 --2- 172.21.15.94:0/665614344 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30fc101d40 0x7f30fc105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 --2- 172.21.15.94:0/665614344 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30fc101390 0x7f30fc101770 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 -- 172.21.15.94:0/665614344 >> 172.21.15.94:0/665614344 conn(0x7f30fc0fcc10 msgr2=0x7f30fc0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 -- 172.21.15.94:0/665614344 shutdown_connections 2024-02-17T12:11:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 -- 172.21.15.94:0/665614344 wait complete. 2024-02-17T12:11:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 Processor -- start 2024-02-17T12:11:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 -- start start 2024-02-17T12:11:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30fc101390 0x7f30fc1976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30fc106460 0x7f30fc197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30fc19bf80 0x7f30fc19c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc19c970 con 0x7f30fc106460 2024-02-17T12:11:27.845 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc19cae0 con 0x7f30fc19bf80 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.840+0000 7f31016fb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc19cc50 con 0x7f30fc101390 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30fc106460 0x7f30fc197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30fc106460 0x7f30fc197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52668/0 (socket says 172.21.15.94:52668) 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30f3fff640 1 -- 172.21.15.94:0/166921909 learned_addr learned my addr 172.21.15.94:0/166921909 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30fb7fe640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30fc19bf80 0x7f30fc19c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30faffd640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30fc101390 0x7f30fc1976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30f3fff640 1 -- 172.21.15.94:0/166921909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30fc101390 msgr2=0x7f30fc1976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30f3fff640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30fc101390 0x7f30fc1976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30f3fff640 1 -- 172.21.15.94:0/166921909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30fc19bf80 msgr2=0x7f30fc19c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30f3fff640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30fc19bf80 0x7f30fc19c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30f3fff640 1 -- 172.21.15.94:0/166921909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec009560 con 0x7f30fc106460 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30f3fff640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30fc106460 0x7f30fc197c00 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f30e400e9c0 tx=0x7f30e400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30f8ff9640 1 -- 172.21.15.94:0/166921909 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e4009700 con 0x7f30fc106460 2024-02-17T12:11:27.846 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f31016fb640 1 -- 172.21.15.94:0/166921909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30fc1a4230 con 0x7f30fc106460 2024-02-17T12:11:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30f8ff9640 1 -- 172.21.15.94:0/166921909 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30e400cd80 con 0x7f30fc106460 2024-02-17T12:11:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f30f8ff9640 1 -- 172.21.15.94:0/166921909 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e4010430 con 0x7f30fc106460 2024-02-17T12:11:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.841+0000 7f31016fb640 1 -- 172.21.15.94:0/166921909 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc1a4730 con 0x7f30fc106460 2024-02-17T12:11:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.844+0000 7f30f8ff9640 1 -- 172.21.15.94:0/166921909 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f30e4005000 con 0x7f30fc106460 2024-02-17T12:11:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.844+0000 7f31043ab740 1 Processor -- start 2024-02-17T12:11:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f31043ab740 1 -- start start 2024-02-17T12:11:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f31043ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5bc1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f31043ab740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bc1d60 0x5608a5bc21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f31043ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bcb2e0 0x5608a5bcd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f31043ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5b309f0 con 0x5608a5bc2ad0 2024-02-17T12:11:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f31043ab740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5b2df20 con 0x5608a5bc1d60 2024-02-17T12:11:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f31043ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5b29700 con 0x5608a5bcb2e0 2024-02-17T12:11:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f30faffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bc1d60 0x5608a5bc21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f30faffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bc1d60 0x5608a5bc21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:38288/0 (socket says 172.21.15.94:38288) 2024-02-17T12:11:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f30faffd640 1 -- 172.21.15.94:0/1367420122 learned_addr learned my addr 172.21.15.94:0/1367420122 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f30f3fff640 1 --2- 172.21.15.94:0/1367420122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bcb2e0 0x5608a5bcd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f30f8ff9640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f30e80a22d0 0x7f30e80a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f30fb7fe640 1 --2- 172.21.15.94:0/1367420122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5bc1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f30faffd640 1 -- 172.21.15.94:0/1367420122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bcb2e0 msgr2=0x5608a5bcd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f30faffd640 1 --2- 172.21.15.94:0/1367420122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bcb2e0 0x5608a5bcd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f30faffd640 1 -- 172.21.15.94:0/1367420122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 msgr2=0x5608a5bc1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f30faffd640 1 --2- 172.21.15.94:0/1367420122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5bc1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f30faffd640 1 -- 172.21.15.94:0/1367420122 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608a5abf7b0 con 0x5608a5bc1d60 2024-02-17T12:11:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.845+0000 7f30fb7fe640 1 --2- 172.21.15.94:0/1367420122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5bc1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:11:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.846+0000 7f30f8ff9640 1 -- 172.21.15.94:0/166921909 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f30e4014070 con 0x7f30fc106460 2024-02-17T12:11:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.846+0000 7f30fb7fe640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f30e80a22d0 0x7f30e80a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.846+0000 7f30faffd640 1 --2- 172.21.15.94:0/1367420122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bc1d60 0x5608a5bc21e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009900 tx=0x7f30ec02c5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6985ea188781e911 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.846+0000 7f30f17fa640 1 -- 172.21.15.94:0/1367420122 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec02cad0 con 0x5608a5bc1d60 2024-02-17T12:11:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.846+0000 7f30fb7fe640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f30e80a22d0 0x7f30e80a4790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f30f4005fd0 tx=0x7f30f4004a40 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.846+0000 7f30f17fa640 1 -- 172.21.15.94:0/1367420122 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30ec02cc30 con 0x5608a5bc1d60 2024-02-17T12:11:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.846+0000 7f30f17fa640 1 -- 172.21.15.94:0/1367420122 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec005190 con 0x5608a5bc1d60 2024-02-17T12:11:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.847+0000 7f31043ab740 1 -- 172.21.15.94:0/1367420122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bc1d60 msgr2=0x5608a5bc21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.847+0000 7f31043ab740 1 --2- 172.21.15.94:0/1367420122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bc1d60 0x5608a5bc21e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f30ec009900 tx=0x7f30ec02c5d0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.847+0000 7f31043ab740 1 -- 172.21.15.94:0/1367420122 shutdown_connections 2024-02-17T12:11:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.847+0000 7f31043ab740 1 --2- 172.21.15.94:0/1367420122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bcb2e0 0x5608a5bcd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.847+0000 7f31043ab740 1 --2- 172.21.15.94:0/1367420122 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bc1d60 0x5608a5bc21e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.847+0000 7f31043ab740 1 --2- 172.21.15.94:0/1367420122 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5bc1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.847+0000 7f31043ab740 1 -- 172.21.15.94:0/1367420122 >> 172.21.15.94:0/1367420122 conn(0x5608a5b89550 msgr2=0x5608a5bbca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.847+0000 7f31043ab740 1 -- 172.21.15.94:0/1367420122 shutdown_connections 2024-02-17T12:11:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.847+0000 7f31043ab740 1 -- 172.21.15.94:0/1367420122 wait complete. 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.847+0000 7f31043ab740 1 -- start start 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.847+0000 7f31043ab740 1 Processor -- start 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.847+0000 7f31043ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5bc8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f31043ab740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bc2ad0 0x5608a5bc88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f31043ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bcb2e0 0x5608a5bc8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f30faffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5bc8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f31043ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5acebf0 con 0x5608a5bcb2e0 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f31043ab740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a57fd250 con 0x5608a5bc2ad0 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f31043ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5aaf200 con 0x5608a5bc1d60 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f30faffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5bc8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:48910/0 (socket says 172.21.15.94:48910) 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f30faffd640 1 -- 172.21.15.94:0/748550529 learned_addr learned my addr 172.21.15.94:0/748550529 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f31043ab740 10 client.0 _ll_register_callbacks cb 0x5608a5b1dd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f30fb7fe640 1 --2- 172.21.15.94:0/748550529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bcb2e0 0x5608a5bc8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f30faffd640 1 -- 172.21.15.94:0/748550529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bc2ad0 msgr2=0x5608a5bc88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f30faffd640 1 --2- 172.21.15.94:0/748550529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bc2ad0 0x5608a5bc88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f30faffd640 1 -- 172.21.15.94:0/748550529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bcb2e0 msgr2=0x5608a5bc8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f30faffd640 1 --2- 172.21.15.94:0/748550529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bcb2e0 0x5608a5bc8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.848+0000 7f30faffd640 1 -- 172.21.15.94:0/748550529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc10e840 con 0x5608a5bc1d60 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.849+0000 7f30fb7fe640 1 --2- 172.21.15.94:0/748550529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bcb2e0 0x5608a5bc8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.849+0000 7f30faffd640 1 --2- 172.21.15.94:0/748550529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5bc8360 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f30ec02c170 tx=0x7f30ec02cb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b6d4f18dd165b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.849+0000 7f30e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.849+0000 7f31043ab740 1 -- 172.21.15.94:0/748550529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608a587a1b0 con 0x5608a5bc1d60 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.849+0000 7f30e27fc640 1 -- 172.21.15.94:0/748550529 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ec005280 con 0x5608a5bc1d60 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.849+0000 7f30e27fc640 1 -- 172.21.15.94:0/748550529 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f30ec0053e0 con 0x5608a5bc1d60 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.849+0000 7f30e27fc640 1 -- 172.21.15.94:0/748550529 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30ec0055e0 con 0x5608a5bc1d60 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.849+0000 7f31043ab740 10 client.28384 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.849+0000 7f31043ab740 1 -- 172.21.15.94:0/748550529 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x5608a5b27d80 con 0x5608a5bc1d60 2024-02-17T12:11:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.850+0000 7f30e27fc640 1 -- 172.21.15.94:0/748550529 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f30ec03c040 con 0x5608a5bc1d60 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.850+0000 7f31043ab740 10 client.28384 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.850+0000 7f31043ab740 10 client.28384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.850+0000 7f31043ab740 20 client.28384 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.850+0000 7f31043ab740 1 --2- 172.21.15.94:0/748550529 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5608a5bdf450 0x5608a5be1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.850+0000 7f31043ab740 4 client.28384 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.850+0000 7f31043ab740 1 -- 172.21.15.94:0/748550529 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608a5866550 con 0x5608a5bdf450 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.850+0000 7f30f3fff640 1 --2- 172.21.15.94:0/748550529 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5608a5bdf450 0x5608a5be1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.850+0000 7f30e27fc640 10 client.28384 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.850+0000 7f30f3fff640 1 --2- 172.21.15.94:0/748550529 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5608a5bdf450 0x5608a5be1860 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6cee82338acfa5 server_cookie=e05bfe616dc6e34f in_seq=0 out_seq=0 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f30e27fc640 1 -- 172.21.15.94:0/748550529 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5608a5866550 con 0x5608a5bdf450 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f30e27fc640 10 client.28384 handle_command_reply: tid=0 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f31043ab740 1 client.28384 shutdown 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f31043ab740 1 -- 172.21.15.94:0/748550529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 msgr2=0x5608a5bc8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:27.859 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f31043ab740 1 --2- 172.21.15.94:0/748550529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5bc8360 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f30ec02c170 tx=0x7f30ec02cb60 comp rx=0 tx=0).stop 2024-02-17T12:11:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f31043ab740 1 -- 172.21.15.94:0/748550529 shutdown_connections 2024-02-17T12:11:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f31043ab740 1 --2- 172.21.15.94:0/748550529 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5608a5bdf450 0x5608a5be1860 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f31043ab740 1 --2- 172.21.15.94:0/748550529 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bcb2e0 0x5608a5bc8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f30e27fc640 0 client.28384 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f31043ab740 1 --2- 172.21.15.94:0/748550529 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bc2ad0 0x5608a5bc88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f31043ab740 1 --2- 172.21.15.94:0/748550529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5bc8360 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f31043ab740 1 -- 172.21.15.94:0/748550529 >> 172.21.15.94:0/748550529 conn(0x5608a5b89550 msgr2=0x5608a5bbe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f31043ab740 1 -- 172.21.15.94:0/748550529 shutdown_connections 2024-02-17T12:11:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f31043ab740 1 -- 172.21.15.94:0/748550529 wait complete. 2024-02-17T12:11:27.860 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.851+0000 7f31043ab740 20 client.28384 trim_cache size 0 max 16384 2024-02-17T12:11:27.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.873+0000 7f31043ab740 1 Processor -- start 2024-02-17T12:11:27.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.873+0000 7f31043ab740 1 -- start start 2024-02-17T12:11:27.878 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.874+0000 7f31043ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5bbe520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.874+0000 7f31043ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5bd6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.874+0000 7f31043ab740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bcb2e0 0x5608a5bd69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.874+0000 7f31043ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5bc1680 con 0x5608a5bc2ad0 2024-02-17T12:11:27.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.874+0000 7f31043ab740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5866550 con 0x5608a5bcb2e0 2024-02-17T12:11:27.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.874+0000 7f31043ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5bbd620 con 0x5608a5bc1d60 2024-02-17T12:11:27.879 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.878+0000 7f30fb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bcb2e0 0x5608a5bd69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.882+0000 7f30fb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bcb2e0 0x5608a5bd69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:38310/0 (socket says 172.21.15.94:38310) 2024-02-17T12:11:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.882+0000 7f30fb7fe640 1 -- 172.21.15.94:0/3959658477 learned_addr learned my addr 172.21.15.94:0/3959658477 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.882+0000 7f30f3fff640 1 --2- 172.21.15.94:0/3959658477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5bd6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.884 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.882+0000 7f30faffd640 1 --2- 172.21.15.94:0/3959658477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5bbe520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.885+0000 7f30fb7fe640 1 -- 172.21.15.94:0/3959658477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 msgr2=0x5608a5bbe520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.885+0000 7f30fb7fe640 1 --2- 172.21.15.94:0/3959658477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5bbe520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.885+0000 7f30fb7fe640 1 -- 172.21.15.94:0/3959658477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 msgr2=0x5608a5bd6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.885+0000 7f30fb7fe640 1 --2- 172.21.15.94:0/3959658477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5bd6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.887 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.885+0000 7f30fb7fe640 1 -- 172.21.15.94:0/3959658477 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608a5ac47b0 con 0x5608a5bcb2e0 2024-02-17T12:11:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.888+0000 7f30fb7fe640 1 --2- 172.21.15.94:0/3959658477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bcb2e0 0x5608a5bd69c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f30f4012770 tx=0x7f30f4015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=584551c47ce29171 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.889+0000 7f30f0ff9640 1 -- 172.21.15.94:0/3959658477 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4004d20 con 0x5608a5bcb2e0 2024-02-17T12:11:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.889+0000 7f30f0ff9640 1 -- 172.21.15.94:0/3959658477 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30f4004e80 con 0x5608a5bcb2e0 2024-02-17T12:11:27.891 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.889+0000 7f30f0ff9640 1 -- 172.21.15.94:0/3959658477 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4005810 con 0x5608a5bcb2e0 2024-02-17T12:11:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.892+0000 7f31043ab740 1 -- 172.21.15.94:0/3959658477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bcb2e0 msgr2=0x5608a5bd69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.892+0000 7f31043ab740 1 --2- 172.21.15.94:0/3959658477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bcb2e0 0x5608a5bd69c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f30f4012770 tx=0x7f30f4015000 comp rx=0 tx=0).stop 2024-02-17T12:11:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.899+0000 7f31043ab740 1 -- 172.21.15.94:0/3959658477 shutdown_connections 2024-02-17T12:11:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.899+0000 7f31043ab740 1 --2- 172.21.15.94:0/3959658477 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bcb2e0 0x5608a5bd69c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.899+0000 7f31043ab740 1 --2- 172.21.15.94:0/3959658477 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5bd6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.899+0000 7f31043ab740 1 --2- 172.21.15.94:0/3959658477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5bbe520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.899+0000 7f31043ab740 1 -- 172.21.15.94:0/3959658477 >> 172.21.15.94:0/3959658477 conn(0x5608a5b89550 msgr2=0x5608a5bbde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.901+0000 7f31043ab740 1 -- 172.21.15.94:0/3959658477 shutdown_connections 2024-02-17T12:11:27.905 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.904+0000 7f31043ab740 1 -- 172.21.15.94:0/3959658477 wait complete. 2024-02-17T12:11:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.904+0000 7f31043ab740 1 -- start start 2024-02-17T12:11:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.904+0000 7f31043ab740 1 Processor -- start 2024-02-17T12:11:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.904+0000 7f31043ab740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5c41040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.904+0000 7f31043ab740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5c41580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.906 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.904+0000 7f31043ab740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bcb2e0 0x5608a5bd5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.904+0000 7f31043ab740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5bd6060 con 0x5608a5bc2ad0 2024-02-17T12:11:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.904+0000 7f31043ab740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5bd61d0 con 0x5608a5bcb2e0 2024-02-17T12:11:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.904+0000 7f31043ab740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608a5bd6390 con 0x5608a5bc1d60 2024-02-17T12:11:27.907 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.904+0000 7f31043ab740 10 client.0 _ll_register_callbacks cb 0x7f30f40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.906+0000 7f30fb7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bcb2e0 0x5608a5bd5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.906+0000 7f30f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5c41580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.906+0000 7f30f3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5c41580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:52702/0 (socket says 172.21.15.94:52702) 2024-02-17T12:11:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.906+0000 7f30f3fff640 1 -- 172.21.15.94:0/1685063691 learned_addr learned my addr 172.21.15.94:0/1685063691 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:27.908 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.906+0000 7f30faffd640 1 --2- 172.21.15.94:0/1685063691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5c41040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.908+0000 7f30f3fff640 1 -- 172.21.15.94:0/1685063691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 msgr2=0x5608a5c41040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.908+0000 7f30f3fff640 1 --2- 172.21.15.94:0/1685063691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5c41040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.908+0000 7f30f3fff640 1 -- 172.21.15.94:0/1685063691 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bcb2e0 msgr2=0x5608a5bd5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.908+0000 7f30f3fff640 1 --2- 172.21.15.94:0/1685063691 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bcb2e0 0x5608a5bd5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:27.910 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.908+0000 7f30f3fff640 1 -- 172.21.15.94:0/1685063691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f4012450 con 0x5608a5bc2ad0 2024-02-17T12:11:27.912 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.910+0000 7f30f3fff640 1 --2- 172.21.15.94:0/1685063691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5c41580 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f30e4000c00 tx=0x7f30e40049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8cbeeb9a62b7ae server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.911+0000 7f30e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:27.913 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.911+0000 7f30e27fc640 1 -- 172.21.15.94:0/1685063691 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e409f080 con 0x5608a5bc2ad0 2024-02-17T12:11:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.911+0000 7f31043ab740 1 -- 172.21.15.94:0/1685063691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608a57d5520 con 0x5608a5bc2ad0 2024-02-17T12:11:27.914 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.911+0000 7f30e27fc640 1 -- 172.21.15.94:0/1685063691 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f30e40a36d0 con 0x5608a5bc2ad0 2024-02-17T12:11:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.915+0000 7f30e27fc640 1 -- 172.21.15.94:0/1685063691 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30e40cbf10 con 0x5608a5bc2ad0 2024-02-17T12:11:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.919+0000 7f31043ab740 10 client.18942 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.919+0000 7f31043ab740 1 -- 172.21.15.94:0/1685063691 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5608a5aea540 con 0x5608a5bc2ad0 2024-02-17T12:11:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.924+0000 7f30e27fc640 1 -- 172.21.15.94:0/1685063691 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5608a57d5520 con 0x5608a5bc2ad0 2024-02-17T12:11:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.924+0000 7f30e27fc640 1 -- 172.21.15.94:0/1685063691 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f30e401f060 con 0x5608a5bc2ad0 2024-02-17T12:11:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.924+0000 7f31043ab740 10 client.18942 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.924+0000 7f31043ab740 10 client.18942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.924+0000 7f31043ab740 20 client.18942 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.924+0000 7f31043ab740 1 --2- 172.21.15.94:0/1685063691 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5608a5c49050 0x5608a5c4b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.924+0000 7f31043ab740 4 client.18942 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:11:27.926 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.924+0000 7f31043ab740 1 -- 172.21.15.94:0/1685063691 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5608a5c4bd20 con 0x5608a5c49050 2024-02-17T12:11:27.929 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.925+0000 7f30faffd640 1 --2- 172.21.15.94:0/1685063691 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5608a5c49050 0x5608a5c4b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:27.931 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.929+0000 7f30faffd640 1 --2- 172.21.15.94:0/1685063691 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5608a5c49050 0x5608a5c4b470 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c556ae8666605fa4 server_cookie=94b8f98ea52e72c9 in_seq=0 out_seq=0 2024-02-17T12:11:27.935 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.932+0000 7f30e27fc640 10 client.18942 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.999+0000 7f30e27fc640 1 -- 172.21.15.94:0/1685063691 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5608a5c4bd20 con 0x5608a5c49050 2024-02-17T12:11:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.999+0000 7f30e27fc640 10 client.18942 handle_command_reply: tid=0 2024-02-17T12:11:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:27.999+0000 7f31043ab740 1 client.18942 shutdown 2024-02-17T12:11:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31043ab740 1 -- 172.21.15.94:0/1685063691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 msgr2=0x5608a5c41580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31043ab740 1 --2- 172.21.15.94:0/1685063691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5c41580 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f30e4000c00 tx=0x7f30e40049a0 comp rx=0 tx=0).stop 2024-02-17T12:11:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31043ab740 1 -- 172.21.15.94:0/1685063691 shutdown_connections 2024-02-17T12:11:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31043ab740 1 --2- 172.21.15.94:0/1685063691 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5608a5c49050 0x5608a5c4b470 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31043ab740 1 --2- 172.21.15.94:0/1685063691 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5608a5bcb2e0 0x5608a5bd5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31043ab740 1 --2- 172.21.15.94:0/1685063691 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5608a5bc2ad0 0x5608a5c41580 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f30e27fc640 0 client.18942 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31043ab740 1 --2- 172.21.15.94:0/1685063691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5608a5bc1d60 0x5608a5c41040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31043ab740 1 -- 172.21.15.94:0/1685063691 >> 172.21.15.94:0/1685063691 conn(0x5608a5b89550 msgr2=0x5608a5bbbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31043ab740 1 -- 172.21.15.94:0/1685063691 shutdown_connections 2024-02-17T12:11:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31043ab740 1 -- 172.21.15.94:0/1685063691 wait complete. 2024-02-17T12:11:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31043ab740 20 client.18942 trim_cache size 0 max 16384 2024-02-17T12:11:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:11:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:11:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:11:28.004 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:11:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31016fb640 1 -- 172.21.15.94:0/166921909 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f30e80a22d0 msgr2=0x7f30e80a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31016fb640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f30e80a22d0 0x7f30e80a4790 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f30f4005fd0 tx=0x7f30f4004a40 comp rx=0 tx=0).stop 2024-02-17T12:11:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31016fb640 1 -- 172.21.15.94:0/166921909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30fc106460 msgr2=0x7f30fc197c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31016fb640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30fc106460 0x7f30fc197c00 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f30e400e9c0 tx=0x7f30e400ee80 comp rx=0 tx=0).stop 2024-02-17T12:11:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31016fb640 1 -- 172.21.15.94:0/166921909 shutdown_connections 2024-02-17T12:11:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31016fb640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f30e80a22d0 0x7f30e80a4790 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31016fb640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f30fc19bf80 0x7f30fc19c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31016fb640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f30fc106460 0x7f30fc197c00 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.000+0000 7f31016fb640 1 --2- 172.21.15.94:0/166921909 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30fc101390 0x7f30fc1976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.001+0000 7f31016fb640 1 -- 172.21.15.94:0/166921909 >> 172.21.15.94:0/166921909 conn(0x7f30fc0fcc10 msgr2=0x7f30fc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.001+0000 7f31016fb640 1 -- 172.21.15.94:0/166921909 shutdown_connections 2024-02-17T12:11:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:28.001+0000 7f31016fb640 1 -- 172.21.15.94:0/166921909 wait complete. 2024-02-17T12:11:29.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:29 smithi189 ceph-mon[36588]: pgmap v418: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.4 MiB/s wr, 727 op/s 2024-02-17T12:11:29.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:29 smithi125 ceph-mon[36819]: pgmap v418: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.4 MiB/s wr, 727 op/s 2024-02-17T12:11:29.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:29 smithi094 ceph-mon[32499]: pgmap v418: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.4 MiB/s wr, 727 op/s 2024-02-17T12:11:30.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 790.0 s, 1746.6 tps, lat 57.301 ms stddev 69.957 2024-02-17T12:11:31.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:30 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:11:31.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:30 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:11:31.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:30 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:11:32.153 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:31 smithi189 ceph-mon[36588]: pgmap v419: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.3 MiB/s wr, 595 op/s 2024-02-17T12:11:32.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:31 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:11:32.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:31 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:11:32.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:31 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:11:32.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:31 smithi125 ceph-mon[36819]: pgmap v419: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.3 MiB/s wr, 595 op/s 2024-02-17T12:11:32.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:31 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:11:32.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:31 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:11:32.187 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:31 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:11:32.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:31 smithi094 ceph-mon[32499]: pgmap v419: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.3 MiB/s wr, 595 op/s 2024-02-17T12:11:32.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:31 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:11:32.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:31 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:11:32.208 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:31 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:11:33.417 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:33 smithi189 ceph-mon[36588]: pgmap v420: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.0 MiB/s wr, 554 op/s 2024-02-17T12:11:33.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:33 smithi125 ceph-mon[36819]: pgmap v420: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.0 MiB/s wr, 554 op/s 2024-02-17T12:11:33.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:33 smithi094 ceph-mon[32499]: pgmap v420: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.0 MiB/s wr, 554 op/s 2024-02-17T12:11:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 795.0 s, 1943.8 tps, lat 51.120 ms stddev 63.335 2024-02-17T12:11:35.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:35 smithi189 ceph-mon[36588]: pgmap v421: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.1 MiB/s wr, 801 op/s 2024-02-17T12:11:35.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:35 smithi125 ceph-mon[36819]: pgmap v421: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.1 MiB/s wr, 801 op/s 2024-02-17T12:11:35.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:35 smithi094 ceph-mon[32499]: pgmap v421: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 9.1 MiB/s wr, 801 op/s 2024-02-17T12:11:37.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:11:37.053 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:11:37.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.208+0000 7fa77334b640 1 -- 172.21.15.94:0/1450126618 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c102280 msgr2=0x7fa76c0ff680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.208+0000 7fa77334b640 1 --2- 172.21.15.94:0/1450126618 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c102280 0x7fa76c0ff680 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa768009930 tx=0x7fa76802bc50 comp rx=0 tx=0).stop 2024-02-17T12:11:37.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.208+0000 7fa77334b640 1 -- 172.21.15.94:0/1450126618 shutdown_connections 2024-02-17T12:11:37.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.208+0000 7fa77334b640 1 --2- 172.21.15.94:0/1450126618 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c10de00 0x7fa76c1101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.208+0000 7fa77334b640 1 --2- 172.21.15.94:0/1450126618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76c0ffbc0 0x7fa76c100020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.208+0000 7fa77334b640 1 --2- 172.21.15.94:0/1450126618 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c102280 0x7fa76c0ff680 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.208+0000 7fa77334b640 1 -- 172.21.15.94:0/1450126618 >> 172.21.15.94:0/1450126618 conn(0x7fa76c0765a0 msgr2=0x7fa76c0fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.208+0000 7fa77334b640 1 -- 172.21.15.94:0/1450126618 shutdown_connections 2024-02-17T12:11:37.211 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.208+0000 7fa77334b640 1 -- 172.21.15.94:0/1450126618 wait complete. 2024-02-17T12:11:37.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.209+0000 7fa77334b640 1 Processor -- start 2024-02-17T12:11:37.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.209+0000 7fa77334b640 1 -- start start 2024-02-17T12:11:37.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.209+0000 7fa77334b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c0ffbc0 0x7fa76c10bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.209+0000 7fa77334b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102280 0x7fa76c10c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.209+0000 7fa77334b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76c10de00 0x7fa76c1a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.209+0000 7fa77334b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c10cb20 con 0x7fa76c102280 2024-02-17T12:11:37.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.209+0000 7fa77334b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c10cc90 con 0x7fa76c0ffbc0 2024-02-17T12:11:37.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.209+0000 7fa77334b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c1aa4e0 con 0x7fa76c10de00 2024-02-17T12:11:37.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.209+0000 7fa7710e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c0ffbc0 0x7fa76c10bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.209+0000 7fa7710e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c0ffbc0 0x7fa76c10bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52712/0 (socket says 172.21.15.94:52712) 2024-02-17T12:11:37.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.210+0000 7fa7710e7640 1 -- 172.21.15.94:0/2326592463 learned_addr learned my addr 172.21.15.94:0/2326592463 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:37.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.210+0000 7fa7708e6640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102280 0x7fa76c10c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.210+0000 7fa7718e8640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76c10de00 0x7fa76c1a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.210+0000 7fa7710e7640 1 -- 172.21.15.94:0/2326592463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76c10de00 msgr2=0x7fa76c1a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.210+0000 7fa7710e7640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76c10de00 0x7fa76c1a9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.210+0000 7fa7710e7640 1 -- 172.21.15.94:0/2326592463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102280 msgr2=0x7fa76c10c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.210+0000 7fa7710e7640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102280 0x7fa76c10c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.210+0000 7fa7710e7640 1 -- 172.21.15.94:0/2326592463 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768009560 con 0x7fa76c0ffbc0 2024-02-17T12:11:37.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.210+0000 7fa7708e6640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102280 0x7fa76c10c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:37.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.210+0000 7fa7710e7640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c0ffbc0 0x7fa76c10bdd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa7680058c0 tx=0x7fa76802cda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.210+0000 7fa7627fc640 1 -- 172.21.15.94:0/2326592463 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768004350 con 0x7fa76c0ffbc0 2024-02-17T12:11:37.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.211+0000 7fa77334b640 1 -- 172.21.15.94:0/2326592463 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76c1aa760 con 0x7fa76c0ffbc0 2024-02-17T12:11:37.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.211+0000 7fa7627fc640 1 -- 172.21.15.94:0/2326592463 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7680044b0 con 0x7fa76c0ffbc0 2024-02-17T12:11:37.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.211+0000 7fa7627fc640 1 -- 172.21.15.94:0/2326592463 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768003cb0 con 0x7fa76c0ffbc0 2024-02-17T12:11:37.215 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.211+0000 7fa77334b640 1 -- 172.21.15.94:0/2326592463 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76c1aac70 con 0x7fa76c0ffbc0 2024-02-17T12:11:37.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.214+0000 7fa7627fc640 1 -- 172.21.15.94:0/2326592463 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa768004df0 con 0x7fa76c0ffbc0 2024-02-17T12:11:37.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.215+0000 7fa775ff0740 1 Processor -- start 2024-02-17T12:11:37.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.215+0000 7fa775ff0740 1 -- start start 2024-02-17T12:11:37.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.215+0000 7fa7627fc640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa7580a22f0 0x7fa7580a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.215+0000 7fa775ff0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962f2b0 0x55cab962f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.215+0000 7fa775ff0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab9638270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.215+0000 7fa775ff0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab96387b0 0x55cab963aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.215+0000 7fa775ff0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab959e280 con 0x55cab96387b0 2024-02-17T12:11:37.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.215+0000 7fa775ff0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab959b7b0 con 0x55cab962f2b0 2024-02-17T12:11:37.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.215+0000 7fa775ff0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab9596f90 con 0x55cab962fbd0 2024-02-17T12:11:37.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.215+0000 7fa7627fc640 1 -- 172.21.15.94:0/2326592463 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa7680f0860 con 0x7fa76c0ffbc0 2024-02-17T12:11:37.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.215+0000 7fa7710e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962f2b0 0x55cab962f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.216+0000 7fa7708e6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab9638270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.216+0000 7fa7710e7640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962f2b0 0x55cab962f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52726/0 (socket says 172.21.15.94:52726) 2024-02-17T12:11:37.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.216+0000 7fa7710e7640 1 -- 172.21.15.94:0/2598500051 learned_addr learned my addr 172.21.15.94:0/2598500051 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:37.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.216+0000 7fa7718e8640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa7580a22f0 0x7fa7580a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.216+0000 7fa7718e8640 1 --2- 172.21.15.94:0/2598500051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab96387b0 0x55cab963aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.216+0000 7fa7708e6640 1 -- 172.21.15.94:0/2598500051 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962f2b0 msgr2=0x55cab962f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.216+0000 7fa7708e6640 1 --2- 172.21.15.94:0/2598500051 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962f2b0 0x55cab962f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.216+0000 7fa7708e6640 1 -- 172.21.15.94:0/2598500051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab96387b0 msgr2=0x55cab963aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.216+0000 7fa7708e6640 1 --2- 172.21.15.94:0/2598500051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab96387b0 0x55cab963aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.216+0000 7fa7708e6640 1 -- 172.21.15.94:0/2598500051 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cab9555570 con 0x55cab962fbd0 2024-02-17T12:11:37.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.216+0000 7fa7718e8640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa7580a22f0 0x7fa7580a47b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa764009fd0 tx=0x7fa764009ec0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.216+0000 7fa7708e6640 1 --2- 172.21.15.94:0/2598500051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab9638270 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00bfb0 tx=0x7fa75c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=223414f109c6070c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.217+0000 7fa7577fe640 1 -- 172.21.15.94:0/2598500051 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c004220 con 0x55cab962fbd0 2024-02-17T12:11:37.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.217+0000 7fa7577fe640 1 -- 172.21.15.94:0/2598500051 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa75c015070 con 0x55cab962fbd0 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.217+0000 7fa7577fe640 1 -- 172.21.15.94:0/2598500051 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0117c0 con 0x55cab962fbd0 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.218+0000 7fa775ff0740 1 -- 172.21.15.94:0/2598500051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 msgr2=0x55cab9638270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.218+0000 7fa775ff0740 1 --2- 172.21.15.94:0/2598500051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab9638270 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00bfb0 tx=0x7fa75c0107a0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.218+0000 7fa775ff0740 1 -- 172.21.15.94:0/2598500051 shutdown_connections 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.218+0000 7fa775ff0740 1 --2- 172.21.15.94:0/2598500051 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab96387b0 0x55cab963aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.218+0000 7fa775ff0740 1 --2- 172.21.15.94:0/2598500051 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab9638270 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.218+0000 7fa775ff0740 1 --2- 172.21.15.94:0/2598500051 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962f2b0 0x55cab962f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.218+0000 7fa775ff0740 1 -- 172.21.15.94:0/2598500051 >> 172.21.15.94:0/2598500051 conn(0x55cab9629990 msgr2=0x55cab9629d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.218+0000 7fa775ff0740 1 -- 172.21.15.94:0/2598500051 shutdown_connections 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.218+0000 7fa775ff0740 1 -- 172.21.15.94:0/2598500051 wait complete. 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.218+0000 7fa775ff0740 1 -- start start 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.218+0000 7fa775ff0740 1 Processor -- start 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa775ff0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 0x55cab9638250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa775ff0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962fbd0 0x55cab96355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa775ff0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab96387b0 0x55cab9635cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa775ff0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab926a430 con 0x55cab962f2b0 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa775ff0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab9226e00 con 0x55cab962fbd0 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa775ff0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab951bfd0 con 0x55cab96387b0 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa775ff0740 10 client.0 _ll_register_callbacks cb 0x55cab9571fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa7710e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 0x55cab9638250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa7710e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 0x55cab9638250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54148/0 (socket says 172.21.15.94:54148) 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa7708e6640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962fbd0 0x55cab96355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa7710e7640 1 -- 172.21.15.94:0/1156480932 learned_addr learned my addr 172.21.15.94:0/1156480932 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa7718e8640 1 --2- 172.21.15.94:0/1156480932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab96387b0 0x55cab9635cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa7710e7640 1 -- 172.21.15.94:0/1156480932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab96387b0 msgr2=0x55cab9635cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa7710e7640 1 --2- 172.21.15.94:0/1156480932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab96387b0 0x55cab9635cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa7710e7640 1 -- 172.21.15.94:0/1156480932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962fbd0 msgr2=0x55cab96355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa7710e7640 1 --2- 172.21.15.94:0/1156480932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962fbd0 0x55cab96355e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa7710e7640 1 -- 172.21.15.94:0/1156480932 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c00bc10 con 0x55cab962f2b0 2024-02-17T12:11:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.219+0000 7fa7708e6640 1 --2- 172.21.15.94:0/1156480932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962fbd0 0x55cab96355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa7710e7640 1 --2- 172.21.15.94:0/1156480932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 0x55cab9638250 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa76802c260 tx=0x7fa768003720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dedb2ca494d534 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa754ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa754ff9640 1 -- 172.21.15.94:0/1156480932 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76803c720 con 0x55cab962f2b0 2024-02-17T12:11:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa754ff9640 1 -- 172.21.15.94:0/1156480932 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa7680bd2c0 con 0x55cab962f2b0 2024-02-17T12:11:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa775ff0740 1 -- 172.21.15.94:0/1156480932 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cab92967e0 con 0x55cab962f2b0 2024-02-17T12:11:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa754ff9640 1 -- 172.21.15.94:0/1156480932 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7680c1410 con 0x55cab962f2b0 2024-02-17T12:11:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa775ff0740 10 client.18951 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa775ff0740 1 -- 172.21.15.94:0/1156480932 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cab9595610 con 0x55cab962f2b0 2024-02-17T12:11:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa754ff9640 1 -- 172.21.15.94:0/1156480932 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55cab92967e0 con 0x55cab962f2b0 2024-02-17T12:11:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa754ff9640 1 -- 172.21.15.94:0/1156480932 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa76803c070 con 0x55cab962f2b0 2024-02-17T12:11:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa775ff0740 10 client.18951 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:37.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa775ff0740 10 client.18951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa775ff0740 20 client.18951 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa775ff0740 1 --2- 172.21.15.94:0/1156480932 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cab96487f0 0x55cab964abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa775ff0740 4 client.18951 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa775ff0740 1 -- 172.21.15.94:0/1156480932 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cab92d3720 con 0x55cab96487f0 2024-02-17T12:11:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.220+0000 7fa7708e6640 1 --2- 172.21.15.94:0/1156480932 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cab96487f0 0x55cab964abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.221+0000 7fa754ff9640 10 client.18951 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.221+0000 7fa7708e6640 1 --2- 172.21.15.94:0/1156480932 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cab96487f0 0x55cab964abe0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641b7c048069d5bb server_cookie=9c8422bc5bfaec14 in_seq=0 out_seq=0 2024-02-17T12:11:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.221+0000 7fa754ff9640 1 -- 172.21.15.94:0/1156480932 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cab92d3720 con 0x55cab96487f0 2024-02-17T12:11:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.221+0000 7fa754ff9640 10 client.18951 handle_command_reply: tid=0 2024-02-17T12:11:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.221+0000 7fa775ff0740 1 client.18951 shutdown 2024-02-17T12:11:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.221+0000 7fa775ff0740 1 -- 172.21.15.94:0/1156480932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 msgr2=0x55cab9638250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.221+0000 7fa775ff0740 1 --2- 172.21.15.94:0/1156480932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 0x55cab9638250 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa76802c260 tx=0x7fa768003720 comp rx=0 tx=0).stop 2024-02-17T12:11:37.228 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.222+0000 7fa775ff0740 1 -- 172.21.15.94:0/1156480932 shutdown_connections 2024-02-17T12:11:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.222+0000 7fa775ff0740 1 --2- 172.21.15.94:0/1156480932 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cab96487f0 0x55cab964abe0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.222+0000 7fa775ff0740 1 --2- 172.21.15.94:0/1156480932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab96387b0 0x55cab9635cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.222+0000 7fa754ff9640 0 client.18951 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.222+0000 7fa775ff0740 1 --2- 172.21.15.94:0/1156480932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962fbd0 0x55cab96355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.222+0000 7fa775ff0740 1 --2- 172.21.15.94:0/1156480932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 0x55cab9638250 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.222+0000 7fa775ff0740 1 -- 172.21.15.94:0/1156480932 >> 172.21.15.94:0/1156480932 conn(0x55cab9629990 msgr2=0x55cab962b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.222+0000 7fa775ff0740 1 -- 172.21.15.94:0/1156480932 shutdown_connections 2024-02-17T12:11:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.222+0000 7fa775ff0740 1 -- 172.21.15.94:0/1156480932 wait complete. 2024-02-17T12:11:37.229 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.222+0000 7fa775ff0740 20 client.18951 trim_cache size 0 max 16384 2024-02-17T12:11:37.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.239+0000 7fa775ff0740 1 Processor -- start 2024-02-17T12:11:37.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.239+0000 7fa775ff0740 1 -- start start 2024-02-17T12:11:37.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.239+0000 7fa775ff0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962f2b0 0x55cab9637d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.239+0000 7fa775ff0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab9638250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.239+0000 7fa775ff0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab96387b0 0x55cab9634e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.239+0000 7fa775ff0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab96365a0 con 0x55cab96387b0 2024-02-17T12:11:37.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.239+0000 7fa775ff0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab9623e20 con 0x55cab962f2b0 2024-02-17T12:11:37.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.239+0000 7fa775ff0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab962ef10 con 0x55cab962fbd0 2024-02-17T12:11:37.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.239+0000 7fa7718e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab96387b0 0x55cab9634e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.246 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.239+0000 7fa7718e8640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab96387b0 0x55cab9634e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54156/0 (socket says 172.21.15.94:54156) 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.239+0000 7fa7718e8640 1 -- 172.21.15.94:0/4084927246 learned_addr learned my addr 172.21.15.94:0/4084927246 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.239+0000 7fa7708e6640 1 --2- 172.21.15.94:0/4084927246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab9638250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.240+0000 7fa7710e7640 1 --2- 172.21.15.94:0/4084927246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962f2b0 0x55cab9637d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.240+0000 7fa7718e8640 1 -- 172.21.15.94:0/4084927246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 msgr2=0x55cab9638250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.240+0000 7fa7718e8640 1 --2- 172.21.15.94:0/4084927246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab9638250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.240+0000 7fa7718e8640 1 -- 172.21.15.94:0/4084927246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962f2b0 msgr2=0x55cab9637d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.240+0000 7fa7718e8640 1 --2- 172.21.15.94:0/4084927246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962f2b0 0x55cab9637d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.240+0000 7fa7718e8640 1 -- 172.21.15.94:0/4084927246 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cab9531ac0 con 0x55cab96387b0 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.240+0000 7fa7710e7640 1 --2- 172.21.15.94:0/4084927246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962f2b0 0x55cab9637d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.240+0000 7fa7718e8640 1 --2- 172.21.15.94:0/4084927246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab96387b0 0x55cab9634e70 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fa764008ea0 tx=0x7fa76401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a8f7d7d1adefdd server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.240+0000 7fa756ffd640 1 -- 172.21.15.94:0/4084927246 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764029070 con 0x55cab96387b0 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.240+0000 7fa756ffd640 1 -- 172.21.15.94:0/4084927246 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa76401ae00 con 0x55cab96387b0 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.240+0000 7fa756ffd640 1 -- 172.21.15.94:0/4084927246 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764019350 con 0x55cab96387b0 2024-02-17T12:11:37.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.241+0000 7fa775ff0740 1 -- 172.21.15.94:0/4084927246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab96387b0 msgr2=0x55cab9634e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.241+0000 7fa775ff0740 1 --2- 172.21.15.94:0/4084927246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab96387b0 0x55cab9634e70 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fa764008ea0 tx=0x7fa76401d000 comp rx=0 tx=0).stop 2024-02-17T12:11:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.241+0000 7fa775ff0740 1 -- 172.21.15.94:0/4084927246 shutdown_connections 2024-02-17T12:11:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.241+0000 7fa775ff0740 1 --2- 172.21.15.94:0/4084927246 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab96387b0 0x55cab9634e70 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.241+0000 7fa775ff0740 1 --2- 172.21.15.94:0/4084927246 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab9638250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.241+0000 7fa775ff0740 1 --2- 172.21.15.94:0/4084927246 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab962f2b0 0x55cab9637d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.241+0000 7fa775ff0740 1 -- 172.21.15.94:0/4084927246 >> 172.21.15.94:0/4084927246 conn(0x55cab9629990 msgr2=0x55cab9633eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.241+0000 7fa775ff0740 1 -- 172.21.15.94:0/4084927246 shutdown_connections 2024-02-17T12:11:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.241+0000 7fa775ff0740 1 -- 172.21.15.94:0/4084927246 wait complete. 2024-02-17T12:11:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.241+0000 7fa775ff0740 1 -- start start 2024-02-17T12:11:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.241+0000 7fa775ff0740 1 Processor -- start 2024-02-17T12:11:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa775ff0740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 0x55cab963c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa775ff0740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab963c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa775ff0740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab96387b0 0x55cab963caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa775ff0740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab969ec90 con 0x55cab962f2b0 2024-02-17T12:11:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa775ff0740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab969edd0 con 0x55cab96387b0 2024-02-17T12:11:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa775ff0740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cab969ef10 con 0x55cab962fbd0 2024-02-17T12:11:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa775ff0740 10 client.0 _ll_register_callbacks cb 0x7fa75c00bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa7710e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 0x55cab963c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa7710e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 0x55cab963c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54168/0 (socket says 172.21.15.94:54168) 2024-02-17T12:11:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa7710e7640 1 -- 172.21.15.94:0/1267624791 learned_addr learned my addr 172.21.15.94:0/1267624791 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa7708e6640 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab963c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa7718e8640 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab96387b0 0x55cab963caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa7710e7640 1 -- 172.21.15.94:0/1267624791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 msgr2=0x55cab963c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa7710e7640 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab963c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa7710e7640 1 -- 172.21.15.94:0/1267624791 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab96387b0 msgr2=0x55cab963caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa7710e7640 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab96387b0 0x55cab963caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa7710e7640 1 -- 172.21.15.94:0/1267624791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa764013040 con 0x55cab962f2b0 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.242+0000 7fa7708e6640 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab963c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa7710e7640 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 0x55cab963c070 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fa7680b9720 tx=0x7fa7680035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bc4bdbef408c06 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa754ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa754ff9640 1 -- 172.21.15.94:0/1267624791 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680bd2c0 con 0x55cab962f2b0 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa754ff9640 1 -- 172.21.15.94:0/1267624791 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa7680c1410 con 0x55cab962f2b0 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa775ff0740 1 -- 172.21.15.94:0/1267624791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa75c015070 con 0x55cab962f2b0 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa754ff9640 1 -- 172.21.15.94:0/1267624791 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7680c5a60 con 0x55cab962f2b0 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa775ff0740 10 client.18957 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa775ff0740 1 -- 172.21.15.94:0/1267624791 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55cab95576a0 con 0x55cab962f2b0 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa754ff9640 1 -- 172.21.15.94:0/1267624791 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa75c015070 con 0x55cab962f2b0 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa754ff9640 1 -- 172.21.15.94:0/1267624791 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa7680a6070 con 0x55cab962f2b0 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa775ff0740 10 client.18957 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa775ff0740 10 client.18957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:37.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa775ff0740 20 client.18957 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa775ff0740 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cab96a0200 0x55cab96a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa775ff0740 4 client.18957 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:11:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa775ff0740 1 -- 172.21.15.94:0/1267624791 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cab9322890 con 0x55cab96a0200 2024-02-17T12:11:37.251 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.243+0000 7fa7708e6640 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cab96a0200 0x55cab96a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.244+0000 7fa754ff9640 10 client.18957 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.244+0000 7fa7708e6640 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cab96a0200 0x55cab96a25f0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7461969b9470dc4 server_cookie=c0de270f6e374da8 in_seq=0 out_seq=0 2024-02-17T12:11:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa754ff9640 1 -- 172.21.15.94:0/1267624791 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cab9322890 con 0x55cab96a0200 2024-02-17T12:11:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa754ff9640 10 client.18957 handle_command_reply: tid=0 2024-02-17T12:11:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa775ff0740 1 client.18957 shutdown 2024-02-17T12:11:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa775ff0740 1 -- 172.21.15.94:0/1267624791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 msgr2=0x55cab963c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa775ff0740 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 0x55cab963c070 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fa7680b9720 tx=0x7fa7680035f0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa775ff0740 1 -- 172.21.15.94:0/1267624791 shutdown_connections 2024-02-17T12:11:37.252 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa775ff0740 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55cab96a0200 0x55cab96a25f0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa775ff0740 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55cab96387b0 0x55cab963caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa775ff0740 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cab962fbd0 0x55cab963c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa775ff0740 1 --2- 172.21.15.94:0/1267624791 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55cab962f2b0 0x55cab963c070 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa775ff0740 1 -- 172.21.15.94:0/1267624791 >> 172.21.15.94:0/1267624791 conn(0x55cab9629990 msgr2=0x55cab9634e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa754ff9640 0 client.18957 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa775ff0740 1 -- 172.21.15.94:0/1267624791 shutdown_connections 2024-02-17T12:11:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa775ff0740 1 -- 172.21.15.94:0/1267624791 wait complete. 2024-02-17T12:11:37.253 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa775ff0740 20 client.18957 trim_cache size 0 max 16384 2024-02-17T12:11:37.253 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:11:37.253 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:11:37.253 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:11:37.254 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:11:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.245+0000 7fa77334b640 1 -- 172.21.15.94:0/2326592463 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa7580a22f0 msgr2=0x7fa7580a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.246+0000 7fa77334b640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa7580a22f0 0x7fa7580a47b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa764009fd0 tx=0x7fa764009ec0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.246+0000 7fa77334b640 1 -- 172.21.15.94:0/2326592463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c0ffbc0 msgr2=0x7fa76c10bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.246+0000 7fa77334b640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c0ffbc0 0x7fa76c10bdd0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa7680058c0 tx=0x7fa76802cda0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.246+0000 7fa77334b640 1 -- 172.21.15.94:0/2326592463 shutdown_connections 2024-02-17T12:11:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.246+0000 7fa77334b640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa7580a22f0 0x7fa7580a47b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.254 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.246+0000 7fa77334b640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa76c10de00 0x7fa76c1a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.246+0000 7fa77334b640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa76c102280 0x7fa76c10c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.246+0000 7fa77334b640 1 --2- 172.21.15.94:0/2326592463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa76c0ffbc0 0x7fa76c10bdd0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.246+0000 7fa77334b640 1 -- 172.21.15.94:0/2326592463 >> 172.21.15.94:0/2326592463 conn(0x7fa76c0765a0 msgr2=0x7fa76c10f230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.246+0000 7fa77334b640 1 -- 172.21.15.94:0/2326592463 shutdown_connections 2024-02-17T12:11:37.255 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.246+0000 7fa77334b640 1 -- 172.21.15.94:0/2326592463 wait complete. 2024-02-17T12:11:37.262 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:11:37.262 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:11:37.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.398+0000 7fa8c754b640 1 -- 172.21.15.94:0/1736782978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8c01064a0 msgr2=0x7fa8c010e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.398+0000 7fa8c754b640 1 --2- 172.21.15.94:0/1736782978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8c01064a0 0x7fa8c010e070 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009c10 tx=0x7fa8b802bf10 comp rx=0 tx=0).stop 2024-02-17T12:11:37.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.398+0000 7fa8c754b640 1 -- 172.21.15.94:0/1736782978 shutdown_connections 2024-02-17T12:11:37.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.398+0000 7fa8c754b640 1 --2- 172.21.15.94:0/1736782978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8c01064a0 0x7fa8c010e070 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.398+0000 7fa8c754b640 1 --2- 172.21.15.94:0/1736782978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8c006a1e0 0x7fa8c0105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.398+0000 7fa8c754b640 1 --2- 172.21.15.94:0/1736782978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8c0069830 0x7fa8c0069c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.398+0000 7fa8c754b640 1 -- 172.21.15.94:0/1736782978 >> 172.21.15.94:0/1736782978 conn(0x7fa8c0077120 msgr2=0x7fa8c0077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.399+0000 7fa8c754b640 1 -- 172.21.15.94:0/1736782978 shutdown_connections 2024-02-17T12:11:37.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.399+0000 7fa8c754b640 1 -- 172.21.15.94:0/1736782978 wait complete. 2024-02-17T12:11:37.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.399+0000 7fa8c754b640 1 Processor -- start 2024-02-17T12:11:37.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.399+0000 7fa8c754b640 1 -- start start 2024-02-17T12:11:37.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.399+0000 7fa8c754b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8c0069830 0x7fa8c019d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.399+0000 7fa8c754b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8c01064a0 0x7fa8c019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c754b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8c01976a0 0x7fa8c0197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c754b640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0198170 con 0x7fa8c0069830 2024-02-17T12:11:37.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c754b640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c01982e0 con 0x7fa8c01064a0 2024-02-17T12:11:37.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c754b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c01984a0 con 0x7fa8c01976a0 2024-02-17T12:11:37.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c52e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8c0069830 0x7fa8c019d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c52e7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8c0069830 0x7fa8c019d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54186/0 (socket says 172.21.15.94:54186) 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c52e7640 1 -- 172.21.15.94:0/206297585 learned_addr learned my addr 172.21.15.94:0/206297585 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c4ae6640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8c01064a0 0x7fa8c019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c5ae8640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8c01976a0 0x7fa8c0197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c52e7640 1 -- 172.21.15.94:0/206297585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8c01976a0 msgr2=0x7fa8c0197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c52e7640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8c01976a0 0x7fa8c0197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c52e7640 1 -- 172.21.15.94:0/206297585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8c01064a0 msgr2=0x7fa8c019dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c52e7640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8c01064a0 0x7fa8c019dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c52e7640 1 -- 172.21.15.94:0/206297585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b8009840 con 0x7fa8c0069830 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c5ae8640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8c01976a0 0x7fa8c0197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c52e7640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8c0069830 0x7fa8c019d590 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00b450 tx=0x7fa8bc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8b67fc640 1 -- 172.21.15.94:0/206297585 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc011890 con 0x7fa8c0069830 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8b67fc640 1 -- 172.21.15.94:0/206297585 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa8bc0103e0 con 0x7fa8c0069830 2024-02-17T12:11:37.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c754b640 1 -- 172.21.15.94:0/206297585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c01a43f0 con 0x7fa8c0069830 2024-02-17T12:11:37.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8b67fc640 1 -- 172.21.15.94:0/206297585 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc00f5a0 con 0x7fa8c0069830 2024-02-17T12:11:37.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.400+0000 7fa8c754b640 1 -- 172.21.15.94:0/206297585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c01a48d0 con 0x7fa8c0069830 2024-02-17T12:11:37.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.402+0000 7fa8c754b640 1 -- 172.21.15.94:0/206297585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c01028a0 con 0x7fa8c0069830 2024-02-17T12:11:37.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.403+0000 7fa8b67fc640 1 -- 172.21.15.94:0/206297585 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa8bc010550 con 0x7fa8c0069830 2024-02-17T12:11:37.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.404+0000 7fa8b67fc640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa8ac0a2230 0x7fa8ac0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.404+0000 7fa8c4ae6640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa8ac0a2230 0x7fa8ac0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.404+0000 7fa8b67fc640 1 -- 172.21.15.94:0/206297585 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa8bc0cddb0 con 0x7fa8c0069830 2024-02-17T12:11:37.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.404+0000 7fa8c4ae6640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa8ac0a2230 0x7fa8ac0a46f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0198ea0 tx=0x7fa8b000b410 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.409+0000 7fa8b67fc640 1 -- 172.21.15.94:0/206297585 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa8bc096b10 con 0x7fa8c0069830 2024-02-17T12:11:37.412 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:37 smithi189 ceph-mon[36588]: pgmap v422: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 570 op/s 2024-02-17T12:11:37.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:37 smithi125 ceph-mon[36819]: pgmap v422: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 570 op/s 2024-02-17T12:11:37.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:37 smithi094 ceph-mon[32499]: pgmap v422: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 7.0 MiB/s wr, 570 op/s 2024-02-17T12:11:37.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.696+0000 7fa8c754b640 1 -- 172.21.15.94:0/206297585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8c01096a0 con 0x7fa8c0069830 2024-02-17T12:11:37.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.697+0000 7fa8b67fc640 1 -- 172.21.15.94:0/206297585 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7fa8bc09a1d0 con 0x7fa8c0069830 2024-02-17T12:11:37.699 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:11:37.699 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:11:37.700 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:11:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.700+0000 7fa8c754b640 1 -- 172.21.15.94:0/206297585 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa8ac0a2230 msgr2=0x7fa8ac0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.700+0000 7fa8c754b640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa8ac0a2230 0x7fa8ac0a46f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0198ea0 tx=0x7fa8b000b410 comp rx=0 tx=0).stop 2024-02-17T12:11:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.700+0000 7fa8c754b640 1 -- 172.21.15.94:0/206297585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8c0069830 msgr2=0x7fa8c019d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.700+0000 7fa8c754b640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8c0069830 0x7fa8c019d590 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00b450 tx=0x7fa8bc00b910 comp rx=0 tx=0).stop 2024-02-17T12:11:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.700+0000 7fa8c754b640 1 -- 172.21.15.94:0/206297585 shutdown_connections 2024-02-17T12:11:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.700+0000 7fa8c754b640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa8ac0a2230 0x7fa8ac0a46f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.700+0000 7fa8c754b640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8c01976a0 0x7fa8c0197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.700+0000 7fa8c754b640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa8c01064a0 0x7fa8c019dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.700+0000 7fa8c754b640 1 --2- 172.21.15.94:0/206297585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa8c0069830 0x7fa8c019d590 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.700+0000 7fa8c754b640 1 -- 172.21.15.94:0/206297585 >> 172.21.15.94:0/206297585 conn(0x7fa8c0077120 msgr2=0x7fa8c00fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.701+0000 7fa8c754b640 1 -- 172.21.15.94:0/206297585 shutdown_connections 2024-02-17T12:11:37.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.701+0000 7fa8c754b640 1 -- 172.21.15.94:0/206297585 wait complete. 2024-02-17T12:11:37.715 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:11:37.716 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:11:37.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.861+0000 7fcf1018c640 1 -- 172.21.15.94:0/3219653457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf08100280 msgr2=0x7fcf08105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.861+0000 7fcf1018c640 1 --2- 172.21.15.94:0/3219653457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf08100280 0x7fcf08105db0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009930 tx=0x7fcef802bc30 comp rx=0 tx=0).stop 2024-02-17T12:11:37.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.861+0000 7fcf1018c640 1 -- 172.21.15.94:0/3219653457 shutdown_connections 2024-02-17T12:11:37.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.861+0000 7fcf1018c640 1 --2- 172.21.15.94:0/3219653457 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf081064e0 0x7fcf0810e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.861+0000 7fcf1018c640 1 --2- 172.21.15.94:0/3219653457 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf08100280 0x7fcf08105db0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.861+0000 7fcf1018c640 1 --2- 172.21.15.94:0/3219653457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf080ff8d0 0x7fcf080ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.861+0000 7fcf1018c640 1 -- 172.21.15.94:0/3219653457 >> 172.21.15.94:0/3219653457 conn(0x7fcf080765a0 msgr2=0x7fcf080fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.861+0000 7fcf1018c640 1 -- 172.21.15.94:0/3219653457 shutdown_connections 2024-02-17T12:11:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.861+0000 7fcf1018c640 1 -- 172.21.15.94:0/3219653457 wait complete. 2024-02-17T12:11:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf1018c640 1 Processor -- start 2024-02-17T12:11:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf1018c640 1 -- start start 2024-02-17T12:11:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf1018c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf080ff8d0 0x7fcf081a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf1018c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf081064e0 0x7fcf081a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf1018c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf0810d110 0x7fcf0810d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf1018c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0810db90 con 0x7fcf080ff8d0 2024-02-17T12:11:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf1018c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0810dd00 con 0x7fcf081064e0 2024-02-17T12:11:37.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf1018c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0810dec0 con 0x7fcf0810d110 2024-02-17T12:11:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf0df28640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf080ff8d0 0x7fcf081a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf0d727640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf081064e0 0x7fcf081a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf0d727640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf081064e0 0x7fcf081a4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52776/0 (socket says 172.21.15.94:52776) 2024-02-17T12:11:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf0d727640 1 -- 172.21.15.94:0/2687981758 learned_addr learned my addr 172.21.15.94:0/2687981758 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf0e729640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf0810d110 0x7fcf0810d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf0d727640 1 -- 172.21.15.94:0/2687981758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf0810d110 msgr2=0x7fcf0810d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf0d727640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf0810d110 0x7fcf0810d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf0d727640 1 -- 172.21.15.94:0/2687981758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf080ff8d0 msgr2=0x7fcf081a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf0d727640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf080ff8d0 0x7fcf081a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf0d727640 1 -- 172.21.15.94:0/2687981758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef8009560 con 0x7fcf081064e0 2024-02-17T12:11:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf0df28640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf080ff8d0 0x7fcf081a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.862+0000 7fcf0d727640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf081064e0 0x7fcf081a4090 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcef8000c00 tx=0x7fcef8005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.863+0000 7fcefeffd640 1 -- 172.21.15.94:0/2687981758 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8004350 con 0x7fcf081064e0 2024-02-17T12:11:37.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.863+0000 7fcf1018c640 1 -- 172.21.15.94:0/2687981758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf081aaa00 con 0x7fcf081064e0 2024-02-17T12:11:37.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.863+0000 7fcefeffd640 1 -- 172.21.15.94:0/2687981758 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcef80044b0 con 0x7fcf081064e0 2024-02-17T12:11:37.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.863+0000 7fcefeffd640 1 -- 172.21.15.94:0/2687981758 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef8003cd0 con 0x7fcf081064e0 2024-02-17T12:11:37.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.863+0000 7fcf1018c640 1 -- 172.21.15.94:0/2687981758 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf081aaf30 con 0x7fcf081064e0 2024-02-17T12:11:37.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcefeffd640 1 -- 172.21.15.94:0/2687981758 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fcef8004df0 con 0x7fcf081064e0 2024-02-17T12:11:37.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcf12d64740 1 Processor -- start 2024-02-17T12:11:37.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcf12d64740 1 -- start start 2024-02-17T12:11:37.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcf12d64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 0x55fa4e57e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcf12d64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 0x55fa4e587210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcf12d64740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e587750 0x55fa4e589b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcf12d64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa4e539960 con 0x55fa4e57e250 2024-02-17T12:11:37.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcf12d64740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa4e4ed310 con 0x55fa4e587750 2024-02-17T12:11:37.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcf12d64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa4e4ea840 con 0x55fa4e57eb70 2024-02-17T12:11:37.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcf0e729640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 0x55fa4e57e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcf0e729640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 0x55fa4e57e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54224/0 (socket says 172.21.15.94:54224) 2024-02-17T12:11:37.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcf0e729640 1 -- 172.21.15.94:0/850651085 learned_addr learned my addr 172.21.15.94:0/850651085 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:37.871 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcf0d727640 1 --2- 172.21.15.94:0/850651085 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e587750 0x55fa4e589b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.866+0000 7fcf0df28640 1 --2- 172.21.15.94:0/850651085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 0x55fa4e587210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.867+0000 7fcefeffd640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcef40a2230 0x7fcef40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.867+0000 7fcf0e729640 1 -- 172.21.15.94:0/850651085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 msgr2=0x55fa4e587210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.867+0000 7fcf0e729640 1 --2- 172.21.15.94:0/850651085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 0x55fa4e587210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.867+0000 7fcf0e729640 1 -- 172.21.15.94:0/850651085 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e587750 msgr2=0x55fa4e589b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.867+0000 7fcf0e729640 1 --2- 172.21.15.94:0/850651085 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e587750 0x55fa4e589b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.867+0000 7fcf0e729640 1 -- 172.21.15.94:0/850651085 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa4e4a8060 con 0x55fa4e57e250 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.867+0000 7fcefeffd640 1 -- 172.21.15.94:0/2687981758 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcef80f0160 con 0x7fcf081064e0 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.867+0000 7fcf0e729640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcef40a2230 0x7fcef40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.867+0000 7fcf0e729640 1 --2- 172.21.15.94:0/850651085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 0x55fa4e57e630 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009f40 tx=0x7fcf00010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ce500e169db8d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.867+0000 7fcef3fff640 1 -- 172.21.15.94:0/850651085 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00003b30 con 0x55fa4e57e250 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.867+0000 7fcef3fff640 1 -- 172.21.15.94:0/850651085 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf00015070 con 0x55fa4e57e250 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.867+0000 7fcef3fff640 1 -- 172.21.15.94:0/850651085 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0001e860 con 0x55fa4e57e250 2024-02-17T12:11:37.872 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.868+0000 7fcf12d64740 1 -- 172.21.15.94:0/850651085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 msgr2=0x55fa4e57e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.868+0000 7fcf12d64740 1 --2- 172.21.15.94:0/850651085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 0x55fa4e57e630 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fcf00009f40 tx=0x7fcf00010860 comp rx=0 tx=0).stop 2024-02-17T12:11:37.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.868+0000 7fcf0e729640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcef40a2230 0x7fcef40a46f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf0001f000 tx=0x7fcf00020000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.868+0000 7fcf12d64740 1 -- 172.21.15.94:0/850651085 shutdown_connections 2024-02-17T12:11:37.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.868+0000 7fcf12d64740 1 --2- 172.21.15.94:0/850651085 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e587750 0x55fa4e589b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.868+0000 7fcf12d64740 1 --2- 172.21.15.94:0/850651085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 0x55fa4e587210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.868+0000 7fcf12d64740 1 --2- 172.21.15.94:0/850651085 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 0x55fa4e57e630 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.868+0000 7fcf12d64740 1 -- 172.21.15.94:0/850651085 >> 172.21.15.94:0/850651085 conn(0x55fa4e578980 msgr2=0x55fa4e578d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.868+0000 7fcf12d64740 1 -- 172.21.15.94:0/850651085 shutdown_connections 2024-02-17T12:11:37.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.868+0000 7fcf12d64740 1 -- 172.21.15.94:0/850651085 wait complete. 2024-02-17T12:11:37.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.869+0000 7fcf12d64740 1 -- start start 2024-02-17T12:11:37.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.869+0000 7fcf12d64740 1 Processor -- start 2024-02-17T12:11:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.869+0000 7fcf12d64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 0x55fa4e5847a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.869+0000 7fcf12d64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 0x55fa4e584ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.869+0000 7fcf12d64740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e587750 0x55fa4e585220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.869+0000 7fcf12d64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa4e4e6020 con 0x55fa4e57e250 2024-02-17T12:11:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.869+0000 7fcf12d64740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa4e1b9430 con 0x55fa4e587750 2024-02-17T12:11:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.869+0000 7fcf12d64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa4e175e00 con 0x55fa4e57eb70 2024-02-17T12:11:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.869+0000 7fcf0df28640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 0x55fa4e5847a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.869+0000 7fcf0df28640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 0x55fa4e5847a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54240/0 (socket says 172.21.15.94:54240) 2024-02-17T12:11:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.869+0000 7fcf0df28640 1 -- 172.21.15.94:0/3437963224 learned_addr learned my addr 172.21.15.94:0/3437963224 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.869+0000 7fcf0e729640 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e587750 0x55fa4e585220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcf0d727640 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 0x55fa4e584ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.869+0000 7fcf12d64740 10 client.0 _ll_register_callbacks cb 0x55fa4e48dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcf0df28640 1 -- 172.21.15.94:0/3437963224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 msgr2=0x55fa4e584ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcf0df28640 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 0x55fa4e584ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcf0df28640 1 -- 172.21.15.94:0/3437963224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e587750 msgr2=0x55fa4e585220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcf0df28640 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e587750 0x55fa4e585220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcf0df28640 1 -- 172.21.15.94:0/3437963224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf00009bf0 con 0x55fa4e57e250 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcf0e729640 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e587750 0x55fa4e585220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcf0df28640 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 0x55fa4e5847a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fcf04012730 tx=0x7fcf04012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164513f5c92eff9b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcef17fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcef17fa640 1 -- 172.21.15.94:0/3437963224 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf04010e80 con 0x55fa4e57e250 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcef17fa640 1 -- 172.21.15.94:0/3437963224 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcf0401c3f0 con 0x55fa4e57e250 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcf12d64740 1 -- 172.21.15.94:0/3437963224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa4e1e57e0 con 0x55fa4e57e250 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcef17fa640 1 -- 172.21.15.94:0/3437963224 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf0401c5b0 con 0x55fa4e57e250 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcf12d64740 10 client.18975 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:37.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.870+0000 7fcf12d64740 1 -- 172.21.15.94:0/3437963224 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fa4e4e46a0 con 0x55fa4e57e250 2024-02-17T12:11:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.871+0000 7fcef17fa640 1 -- 172.21.15.94:0/3437963224 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf04028020 con 0x55fa4e57e250 2024-02-17T12:11:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.871+0000 7fcef17fa640 1 -- 172.21.15.94:0/3437963224 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcf04012d20 con 0x55fa4e57e250 2024-02-17T12:11:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.871+0000 7fcf12d64740 10 client.18975 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.871+0000 7fcf12d64740 10 client.18975 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:11:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.871+0000 7fcf12d64740 20 client.18975 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.871+0000 7fcf12d64740 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55fa4e597c10 0x55fa4e59a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.871+0000 7fcf12d64740 4 client.18975 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.871+0000 7fcf12d64740 1 -- 172.21.15.94:0/3437963224 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa4e222720 con 0x55fa4e597c10 2024-02-17T12:11:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.871+0000 7fcf0d727640 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55fa4e597c10 0x55fa4e59a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.872+0000 7fcf0d727640 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55fa4e597c10 0x55fa4e59a050 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=413051b991cad596 server_cookie=15bff18702143dfc in_seq=0 out_seq=0 2024-02-17T12:11:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.872+0000 7fcef17fa640 10 client.18975 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:11:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.873+0000 7fcef17fa640 1 -- 172.21.15.94:0/3437963224 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fa4e222720 con 0x55fa4e597c10 2024-02-17T12:11:37.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.873+0000 7fcef17fa640 10 client.18975 handle_command_reply: tid=0 2024-02-17T12:11:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.873+0000 7fcf12d64740 1 client.18975 shutdown 2024-02-17T12:11:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.873+0000 7fcf12d64740 1 -- 172.21.15.94:0/3437963224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 msgr2=0x55fa4e5847a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.873+0000 7fcf12d64740 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 0x55fa4e5847a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fcf04012730 tx=0x7fcf04012b30 comp rx=0 tx=0).stop 2024-02-17T12:11:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.873+0000 7fcf12d64740 1 -- 172.21.15.94:0/3437963224 shutdown_connections 2024-02-17T12:11:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.873+0000 7fcf12d64740 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55fa4e597c10 0x55fa4e59a050 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.873+0000 7fcf12d64740 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e587750 0x55fa4e585220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.873+0000 7fcf12d64740 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 0x55fa4e584ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.873+0000 7fcef17fa640 0 client.18975 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:11:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.873+0000 7fcf12d64740 1 --2- 172.21.15.94:0/3437963224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e57e250 0x55fa4e5847a0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.873+0000 7fcf12d64740 1 -- 172.21.15.94:0/3437963224 >> 172.21.15.94:0/3437963224 conn(0x55fa4e578980 msgr2=0x55fa4e57a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.874+0000 7fcf12d64740 1 -- 172.21.15.94:0/3437963224 shutdown_connections 2024-02-17T12:11:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.874+0000 7fcf12d64740 1 -- 172.21.15.94:0/3437963224 wait complete. 2024-02-17T12:11:37.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.874+0000 7fcf12d64740 20 client.18975 trim_cache size 0 max 16384 2024-02-17T12:11:37.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.886+0000 7fcf12d64740 1 Processor -- start 2024-02-17T12:11:37.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.886+0000 7fcf12d64740 1 -- start start 2024-02-17T12:11:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.886+0000 7fcf12d64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57e250 0x55fa4e586ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.886+0000 7fcf12d64740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57eb70 0x55fa4e585040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.886+0000 7fcf12d64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e587750 0x55fa4e581a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.886+0000 7fcf12d64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa4e46b130 con 0x55fa4e587750 2024-02-17T12:11:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.886+0000 7fcf12d64740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa4e5740b0 con 0x55fa4e57eb70 2024-02-17T12:11:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.886+0000 7fcf12d64740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa4e5742e0 con 0x55fa4e57e250 2024-02-17T12:11:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcf0d727640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57eb70 0x55fa4e585040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcf0d727640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57eb70 0x55fa4e585040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52814/0 (socket says 172.21.15.94:52814) 2024-02-17T12:11:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcf0d727640 1 -- 172.21.15.94:0/394291210 learned_addr learned my addr 172.21.15.94:0/394291210 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcf0df28640 1 --2- 172.21.15.94:0/394291210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57e250 0x55fa4e586ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcf0e729640 1 --2- 172.21.15.94:0/394291210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e587750 0x55fa4e581a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcf0d727640 1 -- 172.21.15.94:0/394291210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57e250 msgr2=0x55fa4e586ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcf0d727640 1 --2- 172.21.15.94:0/394291210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57e250 0x55fa4e586ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcf0d727640 1 -- 172.21.15.94:0/394291210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e587750 msgr2=0x55fa4e581a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcf0d727640 1 --2- 172.21.15.94:0/394291210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e587750 0x55fa4e581a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcf0d727640 1 -- 172.21.15.94:0/394291210 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa4e480c20 con 0x55fa4e57eb70 2024-02-17T12:11:37.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcf0e729640 1 --2- 172.21.15.94:0/394291210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e587750 0x55fa4e581a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:37.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcf0d727640 1 --2- 172.21.15.94:0/394291210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57eb70 0x55fa4e585040 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcef80bc8c0 tx=0x7fcef802bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66cb4bf93ea612a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcef37fe640 1 -- 172.21.15.94:0/394291210 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef80bc700 con 0x55fa4e57eb70 2024-02-17T12:11:37.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcef37fe640 1 -- 172.21.15.94:0/394291210 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcef80bc520 con 0x55fa4e57eb70 2024-02-17T12:11:37.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.887+0000 7fcef37fe640 1 -- 172.21.15.94:0/394291210 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef80bcc30 con 0x55fa4e57eb70 2024-02-17T12:11:37.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.888+0000 7fcf12d64740 1 -- 172.21.15.94:0/394291210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57eb70 msgr2=0x55fa4e585040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.888+0000 7fcf12d64740 1 --2- 172.21.15.94:0/394291210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57eb70 0x55fa4e585040 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcef80bc8c0 tx=0x7fcef802bc30 comp rx=0 tx=0).stop 2024-02-17T12:11:37.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.888+0000 7fcf12d64740 1 -- 172.21.15.94:0/394291210 shutdown_connections 2024-02-17T12:11:37.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.888+0000 7fcf12d64740 1 --2- 172.21.15.94:0/394291210 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e587750 0x55fa4e581a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.888+0000 7fcf12d64740 1 --2- 172.21.15.94:0/394291210 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57eb70 0x55fa4e585040 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.888+0000 7fcf12d64740 1 --2- 172.21.15.94:0/394291210 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57e250 0x55fa4e586ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.888+0000 7fcf12d64740 1 -- 172.21.15.94:0/394291210 >> 172.21.15.94:0/394291210 conn(0x55fa4e578980 msgr2=0x55fa4e59e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.889+0000 7fcf12d64740 1 -- 172.21.15.94:0/394291210 shutdown_connections 2024-02-17T12:11:37.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.889+0000 7fcf12d64740 1 -- 172.21.15.94:0/394291210 wait complete. 2024-02-17T12:11:37.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.889+0000 7fcf12d64740 1 -- start start 2024-02-17T12:11:37.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.890+0000 7fcf12d64740 1 Processor -- start 2024-02-17T12:11:37.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.890+0000 7fcf12d64740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57e250 0x55fa4e58d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.890+0000 7fcf12d64740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 0x55fa4e58dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.890+0000 7fcf12d64740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e587750 0x55fa4e5e8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.890+0000 7fcf0df28640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57e250 0x55fa4e58d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.890+0000 7fcf0df28640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57e250 0x55fa4e58d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52816/0 (socket says 172.21.15.94:52816) 2024-02-17T12:11:37.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.890+0000 7fcf0df28640 1 -- 172.21.15.94:0/2057785007 learned_addr learned my addr 172.21.15.94:0/2057785007 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:37.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.890+0000 7fcf12d64740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa4e5e8690 con 0x55fa4e587750 2024-02-17T12:11:37.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.890+0000 7fcf12d64740 1 -- 172.21.15.94:0/2057785007 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa4e5e8800 con 0x55fa4e57e250 2024-02-17T12:11:37.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.890+0000 7fcf12d64740 1 -- 172.21.15.94:0/2057785007 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa4e5e89c0 con 0x55fa4e57eb70 2024-02-17T12:11:37.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.890+0000 7fcf0e729640 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e587750 0x55fa4e5e8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcf12d64740 10 client.0 _ll_register_callbacks cb 0x7fcf000099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:37.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcf0d727640 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 0x55fa4e58dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcf0e729640 1 -- 172.21.15.94:0/2057785007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 msgr2=0x55fa4e58dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcf0e729640 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 0x55fa4e58dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcf0e729640 1 -- 172.21.15.94:0/2057785007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57e250 msgr2=0x55fa4e58d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:37.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcf0e729640 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57e250 0x55fa4e58d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcf0e729640 1 -- 172.21.15.94:0/2057785007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa4e4a8060 con 0x55fa4e587750 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcf0d727640 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 0x55fa4e58dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcf0df28640 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57e250 0x55fa4e58d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcf0e729640 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e587750 0x55fa4e5e8150 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fcf000039b0 tx=0x7fcf00003fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0495f1200de7338 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcef17fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcef17fa640 1 -- 172.21.15.94:0/2057785007 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0001e420 con 0x55fa4e587750 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcef17fa640 1 -- 172.21.15.94:0/2057785007 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcf00017070 con 0x55fa4e587750 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcf12d64740 1 -- 172.21.15.94:0/2057785007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf00015070 con 0x55fa4e587750 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcef17fa640 1 -- 172.21.15.94:0/2057785007 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf0001e580 con 0x55fa4e587750 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcf12d64740 10 client.18981 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.891+0000 7fcf12d64740 1 -- 172.21.15.94:0/2057785007 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fcf081aaf30 con 0x55fa4e587750 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.892+0000 7fcef17fa640 1 -- 172.21.15.94:0/2057785007 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf00041020 con 0x55fa4e587750 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.892+0000 7fcef17fa640 1 -- 172.21.15.94:0/2057785007 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fcf0001a040 con 0x55fa4e587750 2024-02-17T12:11:37.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.892+0000 7fcf12d64740 10 client.18981 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:37.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.892+0000 7fcf12d64740 10 client.18981 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:11:37.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.892+0000 7fcf12d64740 20 client.18981 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:37.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.892+0000 7fcf12d64740 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55fa4e5f0200 0x55fa4e5f2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:37.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.892+0000 7fcf12d64740 4 client.18981 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:11:37.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.892+0000 7fcf12d64740 1 -- 172.21.15.94:0/2057785007 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fa4e271890 con 0x55fa4e5f0200 2024-02-17T12:11:37.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.892+0000 7fcf0df28640 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55fa4e5f0200 0x55fa4e5f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:37.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.892+0000 7fcef17fa640 10 client.18981 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:11:37.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.892+0000 7fcf0df28640 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55fa4e5f0200 0x55fa4e5f2620 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d3c25bd5ab49992 server_cookie=6e4670c7ec14ab2c in_seq=0 out_seq=0 2024-02-17T12:11:37.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcef17fa640 1 -- 172.21.15.94:0/2057785007 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fa4e271890 con 0x55fa4e5f0200 2024-02-17T12:11:37.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcef17fa640 10 client.18981 handle_command_reply: tid=0 2024-02-17T12:11:37.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcf12d64740 1 client.18981 shutdown 2024-02-17T12:11:37.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcf12d64740 1 -- 172.21.15.94:0/2057785007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e587750 msgr2=0x55fa4e5e8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcf12d64740 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e587750 0x55fa4e5e8150 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fcf000039b0 tx=0x7fcf00003fa0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcf12d64740 1 -- 172.21.15.94:0/2057785007 shutdown_connections 2024-02-17T12:11:37.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcf12d64740 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55fa4e587750 0x55fa4e5e8150 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcf12d64740 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa4e57eb70 0x55fa4e58dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcf12d64740 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55fa4e5f0200 0x55fa4e5f2620 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcf12d64740 1 --2- 172.21.15.94:0/2057785007 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55fa4e57e250 0x55fa4e58d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcef17fa640 0 client.18981 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:11:37.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcf12d64740 1 -- 172.21.15.94:0/2057785007 >> 172.21.15.94:0/2057785007 conn(0x55fa4e578980 msgr2=0x55fa4e5772f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcf12d64740 1 -- 172.21.15.94:0/2057785007 shutdown_connections 2024-02-17T12:11:37.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcf12d64740 1 -- 172.21.15.94:0/2057785007 wait complete. 2024-02-17T12:11:37.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.893+0000 7fcf12d64740 20 client.18981 trim_cache size 0 max 16384 2024-02-17T12:11:37.901 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:11:37.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.894+0000 7fcf1018c640 1 -- 172.21.15.94:0/2687981758 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcef40a2230 msgr2=0x7fcef40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.894+0000 7fcf1018c640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcef40a2230 0x7fcef40a46f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcf0001f000 tx=0x7fcf00020000 comp rx=0 tx=0).stop 2024-02-17T12:11:37.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.894+0000 7fcf1018c640 1 -- 172.21.15.94:0/2687981758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf081064e0 msgr2=0x7fcf081a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:37.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.894+0000 7fcf1018c640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf081064e0 0x7fcf081a4090 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fcef8000c00 tx=0x7fcef8005e70 comp rx=0 tx=0).stop 2024-02-17T12:11:37.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.894+0000 7fcf1018c640 1 -- 172.21.15.94:0/2687981758 shutdown_connections 2024-02-17T12:11:37.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.894+0000 7fcf1018c640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcef40a2230 0x7fcef40a46f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.894+0000 7fcf1018c640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf0810d110 0x7fcf0810d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.894+0000 7fcf1018c640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf081064e0 0x7fcf081a4090 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.894+0000 7fcf1018c640 1 --2- 172.21.15.94:0/2687981758 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf080ff8d0 0x7fcf081a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:37.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.894+0000 7fcf1018c640 1 -- 172.21.15.94:0/2687981758 >> 172.21.15.94:0/2687981758 conn(0x7fcf080765a0 msgr2=0x7fcf081044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:37.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.894+0000 7fcf1018c640 1 -- 172.21.15.94:0/2687981758 shutdown_connections 2024-02-17T12:11:37.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:37.894+0000 7fcf1018c640 1 -- 172.21.15.94:0/2687981758 wait complete. 2024-02-17T12:11:37.908 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:11:37.908 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:11:38.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.043+0000 7f44182f4640 1 -- 172.21.15.94:0/2970648111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4410106460 msgr2=0x7f441010e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.043+0000 7f44182f4640 1 --2- 172.21.15.94:0/2970648111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4410106460 0x7f441010e030 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4408009cd0 tx=0x7f440802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.043+0000 7f44182f4640 1 -- 172.21.15.94:0/2970648111 shutdown_connections 2024-02-17T12:11:38.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.043+0000 7f44182f4640 1 --2- 172.21.15.94:0/2970648111 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4410106460 0x7f441010e030 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.043+0000 7f44182f4640 1 --2- 172.21.15.94:0/2970648111 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4410101d40 0x7f4410105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.043+0000 7f44182f4640 1 --2- 172.21.15.94:0/2970648111 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4410101390 0x7f4410101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.043+0000 7f44182f4640 1 -- 172.21.15.94:0/2970648111 >> 172.21.15.94:0/2970648111 conn(0x7f44100fcbb0 msgr2=0x7f44100fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:38.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.043+0000 7f44182f4640 1 -- 172.21.15.94:0/2970648111 shutdown_connections 2024-02-17T12:11:38.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.043+0000 7f44182f4640 1 -- 172.21.15.94:0/2970648111 wait complete. 2024-02-17T12:11:38.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f44182f4640 1 Processor -- start 2024-02-17T12:11:38.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f44182f4640 1 -- start start 2024-02-17T12:11:38.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f44182f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4410101390 0x7f441019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f44182f4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4410106460 0x7f441019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f44182f4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4410197690 0x7f4410197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.048 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f44182f4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4410198160 con 0x7f4410101390 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f44182f4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44101982d0 con 0x7f4410106460 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f44182f4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4410198490 con 0x7f4410197690 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f4416090640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4410101390 0x7f441019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f441588f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4410106460 0x7f441019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f4416090640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4410101390 0x7f441019d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54284/0 (socket says 172.21.15.94:54284) 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f4416090640 1 -- 172.21.15.94:0/1136042875 learned_addr learned my addr 172.21.15.94:0/1136042875 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f441588f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4410106460 0x7f441019dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52842/0 (socket says 172.21.15.94:52842) 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f4416891640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4410197690 0x7f4410197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f4416090640 1 -- 172.21.15.94:0/1136042875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4410197690 msgr2=0x7f4410197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f4416090640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4410197690 0x7f4410197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f4416090640 1 -- 172.21.15.94:0/1136042875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4410106460 msgr2=0x7f441019dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f4416090640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4410106460 0x7f441019dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f4416090640 1 -- 172.21.15.94:0/1136042875 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4408009900 con 0x7f4410101390 2024-02-17T12:11:38.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.044+0000 7f441588f640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4410106460 0x7f441019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:38.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.045+0000 7f4416090640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4410101390 0x7f441019d580 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f440c00d950 tx=0x7f440c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:38.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.045+0000 7f44077fe640 1 -- 172.21.15.94:0/1136042875 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c009a30 con 0x7f4410101390 2024-02-17T12:11:38.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.045+0000 7f44182f4640 1 -- 172.21.15.94:0/1136042875 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44101a43e0 con 0x7f4410101390 2024-02-17T12:11:38.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.045+0000 7f44182f4640 1 -- 172.21.15.94:0/1136042875 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44101a4910 con 0x7f4410101390 2024-02-17T12:11:38.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.046+0000 7f44077fe640 1 -- 172.21.15.94:0/1136042875 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f440c009b90 con 0x7f4410101390 2024-02-17T12:11:38.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.046+0000 7f44077fe640 1 -- 172.21.15.94:0/1136042875 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c009a30 con 0x7f4410101390 2024-02-17T12:11:38.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.049+0000 7f441afa3740 1 Processor -- start 2024-02-17T12:11:38.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.049+0000 7f441afa3740 1 -- start start 2024-02-17T12:11:38.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.049+0000 7f441afa3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 0x56120434a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.049+0000 7f441afa3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56120434ab70 0x561204353210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.049+0000 7f44077fe640 1 -- 172.21.15.94:0/1136042875 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f440c0047a0 con 0x7f4410101390 2024-02-17T12:11:38.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.049+0000 7f441afa3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 0x561204355b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.049+0000 7f441afa3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561204305960 con 0x56120434ab70 2024-02-17T12:11:38.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.049+0000 7f441afa3740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612042b9310 con 0x561204353750 2024-02-17T12:11:38.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.049+0000 7f441afa3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612042b6840 con 0x56120434a250 2024-02-17T12:11:38.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.049+0000 7f441588f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 0x561204355b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.049+0000 7f441588f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 0x561204355b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52846/0 (socket says 172.21.15.94:52846) 2024-02-17T12:11:38.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.049+0000 7f441588f640 1 -- 172.21.15.94:0/1109794778 learned_addr learned my addr 172.21.15.94:0/1109794778 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:38.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.050+0000 7f4416891640 1 --2- 172.21.15.94:0/1109794778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 0x56120434a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.050+0000 7f4416090640 1 --2- 172.21.15.94:0/1109794778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56120434ab70 0x561204353210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.050+0000 7f441588f640 1 -- 172.21.15.94:0/1109794778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 msgr2=0x56120434a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:38.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.050+0000 7f441588f640 1 --2- 172.21.15.94:0/1109794778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 0x56120434a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.050+0000 7f441588f640 1 -- 172.21.15.94:0/1109794778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56120434ab70 msgr2=0x561204353210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:38.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.050+0000 7f441588f640 1 --2- 172.21.15.94:0/1109794778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56120434ab70 0x561204353210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.050+0000 7f441588f640 1 -- 172.21.15.94:0/1109794778 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561204274060 con 0x561204353750 2024-02-17T12:11:38.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.050+0000 7f441588f640 1 --2- 172.21.15.94:0/1109794778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 0x561204355b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4400009fd0 tx=0x7f44000118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc7f8d90c252ed75 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:38.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.050+0000 7f44077fe640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43fc0a22f0 0x7f43fc0a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.050+0000 7f4404bfe640 1 -- 172.21.15.94:0/1109794778 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4400005020 con 0x561204353750 2024-02-17T12:11:38.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.050+0000 7f4404bfe640 1 -- 172.21.15.94:0/1109794778 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4400005660 con 0x561204353750 2024-02-17T12:11:38.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.050+0000 7f44077fe640 1 -- 172.21.15.94:0/1136042875 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f440c0cd650 con 0x7f4410101390 2024-02-17T12:11:38.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.050+0000 7f4416891640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43fc0a22f0 0x7f43fc0a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.051+0000 7f4404bfe640 1 -- 172.21.15.94:0/1109794778 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440000f7c0 con 0x561204353750 2024-02-17T12:11:38.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.051+0000 7f441afa3740 1 -- 172.21.15.94:0/1109794778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 msgr2=0x561204355b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.051+0000 7f441afa3740 1 --2- 172.21.15.94:0/1109794778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 0x561204355b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f4400009fd0 tx=0x7f44000118d0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.051+0000 7f4416891640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43fc0a22f0 0x7f43fc0a47b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4408005cc0 tx=0x7f4408005c50 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:38.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.051+0000 7f441afa3740 1 -- 172.21.15.94:0/1109794778 shutdown_connections 2024-02-17T12:11:38.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.051+0000 7f441afa3740 1 --2- 172.21.15.94:0/1109794778 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 0x561204355b40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.051+0000 7f441afa3740 1 --2- 172.21.15.94:0/1109794778 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56120434ab70 0x561204353210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.051+0000 7f441afa3740 1 --2- 172.21.15.94:0/1109794778 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 0x56120434a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.051+0000 7f441afa3740 1 -- 172.21.15.94:0/1109794778 >> 172.21.15.94:0/1109794778 conn(0x561204344980 msgr2=0x561204344d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:38.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.051+0000 7f441afa3740 1 -- 172.21.15.94:0/1109794778 shutdown_connections 2024-02-17T12:11:38.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.051+0000 7f441afa3740 1 -- 172.21.15.94:0/1109794778 wait complete. 2024-02-17T12:11:38.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.052+0000 7f441afa3740 1 -- start start 2024-02-17T12:11:38.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.052+0000 7f441afa3740 1 Processor -- start 2024-02-17T12:11:38.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.053+0000 7f441afa3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 0x5612043506b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.053+0000 7f441afa3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56120434ab70 0x561204350bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.053+0000 7f441afa3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 0x561204351130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.053+0000 7f441afa3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612042b2020 con 0x56120434ab70 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.053+0000 7f441afa3740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x561203f85430 con 0x561204353750 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.053+0000 7f441afa3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561203f41e00 con 0x56120434a250 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.053+0000 7f441afa3740 10 client.0 _ll_register_callbacks cb 0x561204259cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.053+0000 7f4416891640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 0x561204351130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.053+0000 7f4416891640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 0x561204351130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52860/0 (socket says 172.21.15.94:52860) 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.053+0000 7f4416891640 1 -- 172.21.15.94:0/4248122498 learned_addr learned my addr 172.21.15.94:0/4248122498 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.053+0000 7f4416090640 1 --2- 172.21.15.94:0/4248122498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 0x5612043506b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.053+0000 7f441588f640 1 --2- 172.21.15.94:0/4248122498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56120434ab70 0x561204350bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f4416891640 1 -- 172.21.15.94:0/4248122498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 msgr2=0x5612043506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f4416891640 1 --2- 172.21.15.94:0/4248122498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 0x5612043506b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f4416891640 1 -- 172.21.15.94:0/4248122498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56120434ab70 msgr2=0x561204350bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f4416891640 1 --2- 172.21.15.94:0/4248122498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56120434ab70 0x561204350bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f4416891640 1 -- 172.21.15.94:0/4248122498 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4400009c00 con 0x561204353750 2024-02-17T12:11:38.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f4416891640 1 --2- 172.21.15.94:0/4248122498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 0x561204351130 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4408034cc0 tx=0x7f44080095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cee3db0466f40182 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f43f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f43f9ffb640 1 -- 172.21.15.94:0/4248122498 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408055660 con 0x561204353750 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f43f9ffb640 1 -- 172.21.15.94:0/4248122498 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f44080557c0 con 0x561204353750 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f441afa3740 1 -- 172.21.15.94:0/4248122498 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561203fb17e0 con 0x561204353750 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f43f9ffb640 1 -- 172.21.15.94:0/4248122498 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4408034e70 con 0x561204353750 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f441afa3740 10 client.28469 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f441afa3740 1 -- 172.21.15.94:0/4248122498 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5612042b06a0 con 0x561204353750 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.054+0000 7f43f9ffb640 1 -- 172.21.15.94:0/4248122498 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x561203fb17e0 con 0x561204353750 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.055+0000 7f43f9ffb640 1 -- 172.21.15.94:0/4248122498 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f440805b070 con 0x561204353750 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.055+0000 7f441afa3740 10 client.28469 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.055+0000 7f441afa3740 10 client.28469 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.055+0000 7f441afa3740 20 client.28469 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.055+0000 7f441afa3740 1 --2- 172.21.15.94:0/4248122498 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56120436c330 0x56120436e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.055+0000 7f441afa3740 4 client.28469 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.055+0000 7f441afa3740 1 -- 172.21.15.94:0/4248122498 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561203fee720 con 0x56120436c330 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.055+0000 7f4416090640 1 --2- 172.21.15.94:0/4248122498 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56120436c330 0x56120436e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.055+0000 7f4416090640 1 --2- 172.21.15.94:0/4248122498 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56120436c330 0x56120436e750 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a20b7796b5854869 server_cookie=c6785aaa7532d0db in_seq=0 out_seq=0 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.055+0000 7f43f9ffb640 10 client.28469 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f43f9ffb640 1 -- 172.21.15.94:0/4248122498 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561203fee720 con 0x56120436c330 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f43f9ffb640 10 client.28469 handle_command_reply: tid=0 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f441afa3740 1 client.28469 shutdown 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f441afa3740 1 -- 172.21.15.94:0/4248122498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 msgr2=0x561204351130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f441afa3740 1 --2- 172.21.15.94:0/4248122498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 0x561204351130 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4408034cc0 tx=0x7f44080095c0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f441afa3740 1 -- 172.21.15.94:0/4248122498 shutdown_connections 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f441afa3740 1 --2- 172.21.15.94:0/4248122498 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56120436c330 0x56120436e750 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f441afa3740 1 --2- 172.21.15.94:0/4248122498 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x561204353750 0x561204351130 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f441afa3740 1 --2- 172.21.15.94:0/4248122498 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56120434ab70 0x561204350bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f441afa3740 1 --2- 172.21.15.94:0/4248122498 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 0x5612043506b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f43f9ffb640 0 client.28469 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f441afa3740 1 -- 172.21.15.94:0/4248122498 >> 172.21.15.94:0/4248122498 conn(0x561204344980 msgr2=0x5612043466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f441afa3740 1 -- 172.21.15.94:0/4248122498 shutdown_connections 2024-02-17T12:11:38.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f441afa3740 1 -- 172.21.15.94:0/4248122498 wait complete. 2024-02-17T12:11:38.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.056+0000 7f441afa3740 20 client.28469 trim_cache size 0 max 16384 2024-02-17T12:11:38.076 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f441afa3740 1 Processor -- start 2024-02-17T12:11:38.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f441afa3740 1 -- start start 2024-02-17T12:11:38.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f441afa3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434a250 0x561204352de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f441afa3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434ab70 0x561204350f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f441afa3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561204353750 0x56120434d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f441afa3740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x561204237130 con 0x561204353750 2024-02-17T12:11:38.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f441afa3740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612043405c0 con 0x56120434a250 2024-02-17T12:11:38.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f441afa3740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x561204349eb0 con 0x56120434ab70 2024-02-17T12:11:38.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f4416891640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561204353750 0x56120434d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f4416090640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434a250 0x561204352de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f4416891640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561204353750 0x56120434d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54312/0 (socket says 172.21.15.94:54312) 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f4416090640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434a250 0x561204352de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52876/0 (socket says 172.21.15.94:52876) 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f4416891640 1 -- 172.21.15.94:0/3727117304 learned_addr learned my addr 172.21.15.94:0/3727117304 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f441588f640 1 --2- 172.21.15.94:0/3727117304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434ab70 0x561204350f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f4416090640 1 -- 172.21.15.94:0/3727117304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434ab70 msgr2=0x561204350f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f4416090640 1 --2- 172.21.15.94:0/3727117304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434ab70 0x561204350f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f4416090640 1 -- 172.21.15.94:0/3727117304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561204353750 msgr2=0x56120434d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f4416090640 1 --2- 172.21.15.94:0/3727117304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561204353750 0x56120434d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f4416090640 1 -- 172.21.15.94:0/3727117304 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56120424cc20 con 0x56120434a250 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.070+0000 7f4416891640 1 --2- 172.21.15.94:0/3727117304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561204353750 0x56120434d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.071+0000 7f4416090640 1 --2- 172.21.15.94:0/3727117304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434a250 0x561204352de0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f440c0cad60 tx=0x7f440c0052d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e9315ed5ac9b02 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.071+0000 7f43fbfff640 1 -- 172.21.15.94:0/3727117304 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c009a30 con 0x56120434a250 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.071+0000 7f43fbfff640 1 -- 172.21.15.94:0/3727117304 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f440c099b40 con 0x56120434a250 2024-02-17T12:11:38.078 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.071+0000 7f43fbfff640 1 -- 172.21.15.94:0/3727117304 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c005610 con 0x56120434a250 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.071+0000 7f441afa3740 1 -- 172.21.15.94:0/3727117304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434a250 msgr2=0x561204352de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.071+0000 7f441afa3740 1 --2- 172.21.15.94:0/3727117304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434a250 0x561204352de0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f440c0cad60 tx=0x7f440c0052d0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.071+0000 7f441afa3740 1 -- 172.21.15.94:0/3727117304 shutdown_connections 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.071+0000 7f441afa3740 1 --2- 172.21.15.94:0/3727117304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561204353750 0x56120434d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.071+0000 7f441afa3740 1 --2- 172.21.15.94:0/3727117304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434ab70 0x561204350f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.071+0000 7f441afa3740 1 --2- 172.21.15.94:0/3727117304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434a250 0x561204352de0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.071+0000 7f441afa3740 1 -- 172.21.15.94:0/3727117304 >> 172.21.15.94:0/3727117304 conn(0x561204344980 msgr2=0x5612043730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.072+0000 7f441afa3740 1 -- 172.21.15.94:0/3727117304 shutdown_connections 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.072+0000 7f441afa3740 1 -- 172.21.15.94:0/3727117304 wait complete. 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.072+0000 7f441afa3740 1 -- start start 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.072+0000 7f441afa3740 1 Processor -- start 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.072+0000 7f441afa3740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 0x561204361b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.072+0000 7f441afa3740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434ab70 0x561204362090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.072+0000 7f441afa3740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561204353750 0x5612043625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.072+0000 7f441588f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434ab70 0x561204362090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.072+0000 7f441588f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434ab70 0x561204362090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52888/0 (socket says 172.21.15.94:52888) 2024-02-17T12:11:38.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.072+0000 7f441588f640 1 -- 172.21.15.94:0/173490131 learned_addr learned my addr 172.21.15.94:0/173490131 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:38.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.073+0000 7f441afa3740 1 -- 172.21.15.94:0/173490131 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612043c2980 con 0x561204353750 2024-02-17T12:11:38.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.073+0000 7f4416090640 1 --2- 172.21.15.94:0/173490131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 0x561204361b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.073+0000 7f441afa3740 1 -- 172.21.15.94:0/173490131 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612043c2af0 con 0x56120434ab70 2024-02-17T12:11:38.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.073+0000 7f441afa3740 1 -- 172.21.15.94:0/173490131 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612043c2cb0 con 0x56120434a250 2024-02-17T12:11:38.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.073+0000 7f4416891640 1 --2- 172.21.15.94:0/173490131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561204353750 0x5612043625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.073+0000 7f441afa3740 10 client.0 _ll_register_callbacks cb 0x7f4400009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:38.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.073+0000 7f441588f640 1 -- 172.21.15.94:0/173490131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 msgr2=0x561204361b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:38.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.073+0000 7f441588f640 1 --2- 172.21.15.94:0/173490131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 0x561204361b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.073+0000 7f441588f640 1 -- 172.21.15.94:0/173490131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561204353750 msgr2=0x5612043625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:38.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.073+0000 7f441588f640 1 --2- 172.21.15.94:0/173490131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561204353750 0x5612043625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.073+0000 7f441588f640 1 -- 172.21.15.94:0/173490131 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612042b06a0 con 0x56120434ab70 2024-02-17T12:11:38.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.073+0000 7f441588f640 1 --2- 172.21.15.94:0/173490131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434ab70 0x561204362090 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f44000058c0 tx=0x7f4400004d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c19c0644ce56907 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:38.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.073+0000 7f43f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:11:38.081 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f43f9ffb640 1 -- 172.21.15.94:0/173490131 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44000050a0 con 0x56120434ab70 2024-02-17T12:11:38.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f441afa3740 1 -- 172.21.15.94:0/173490131 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4400005660 con 0x56120434ab70 2024-02-17T12:11:38.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f43f9ffb640 1 -- 172.21.15.94:0/173490131 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f440002a070 con 0x56120434ab70 2024-02-17T12:11:38.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f43f9ffb640 1 -- 172.21.15.94:0/173490131 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4400005200 con 0x56120434ab70 2024-02-17T12:11:38.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f441afa3740 10 client.28475 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:38.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f441afa3740 1 -- 172.21.15.94:0/173490131 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f44101a4910 con 0x56120434ab70 2024-02-17T12:11:38.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f43f9ffb640 1 -- 172.21.15.94:0/173490131 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f440003a080 con 0x56120434ab70 2024-02-17T12:11:38.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f43f9ffb640 1 -- 172.21.15.94:0/173490131 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4400021040 con 0x56120434ab70 2024-02-17T12:11:38.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f441afa3740 10 client.28475 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:38.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f441afa3740 10 client.28475 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:11:38.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f441afa3740 20 client.28475 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:38.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f441afa3740 1 --2- 172.21.15.94:0/173490131 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5612043c4450 0x5612043c6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.082 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f441afa3740 4 client.28475 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:11:38.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f441afa3740 1 -- 172.21.15.94:0/173490131 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56120403d890 con 0x5612043c4450 2024-02-17T12:11:38.083 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.074+0000 7f4416090640 1 --2- 172.21.15.94:0/173490131 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5612043c4450 0x5612043c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f43f9ffb640 10 client.28475 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:11:38.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f4416090640 1 --2- 172.21.15.94:0/173490131 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5612043c4450 0x5612043c6870 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95d81d70e78bbae9 server_cookie=f5460861bc8c600e in_seq=0 out_seq=0 2024-02-17T12:11:38.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f43f9ffb640 1 -- 172.21.15.94:0/173490131 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56120403d890 con 0x5612043c4450 2024-02-17T12:11:38.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f43f9ffb640 10 client.28475 handle_command_reply: tid=0 2024-02-17T12:11:38.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f441afa3740 1 client.28475 shutdown 2024-02-17T12:11:38.084 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f441afa3740 1 -- 172.21.15.94:0/173490131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434ab70 msgr2=0x561204362090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f441afa3740 1 --2- 172.21.15.94:0/173490131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434ab70 0x561204362090 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f44000058c0 tx=0x7f4400004d80 comp rx=0 tx=0).stop 2024-02-17T12:11:38.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f441afa3740 1 -- 172.21.15.94:0/173490131 shutdown_connections 2024-02-17T12:11:38.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f441afa3740 1 --2- 172.21.15.94:0/173490131 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x5612043c4450 0x5612043c6870 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f441afa3740 1 --2- 172.21.15.94:0/173490131 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x561204353750 0x5612043625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f43f9ffb640 0 client.28475 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:11:38.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f441afa3740 1 --2- 172.21.15.94:0/173490131 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56120434ab70 0x561204362090 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f441afa3740 1 --2- 172.21.15.94:0/173490131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56120434a250 0x561204361b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f441afa3740 1 -- 172.21.15.94:0/173490131 >> 172.21.15.94:0/173490131 conn(0x561204344980 msgr2=0x5612043432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:38.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f441afa3740 1 -- 172.21.15.94:0/173490131 shutdown_connections 2024-02-17T12:11:38.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f441afa3740 1 -- 172.21.15.94:0/173490131 wait complete. 2024-02-17T12:11:38.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.075+0000 7f441afa3740 20 client.28475 trim_cache size 0 max 16384 2024-02-17T12:11:38.086 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:11:38.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.076+0000 7f44182f4640 1 -- 172.21.15.94:0/1136042875 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43fc0a22f0 msgr2=0x7f43fc0a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.076+0000 7f44182f4640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43fc0a22f0 0x7f43fc0a47b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4408005cc0 tx=0x7f4408005c50 comp rx=0 tx=0).stop 2024-02-17T12:11:38.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.076+0000 7f44182f4640 1 -- 172.21.15.94:0/1136042875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4410101390 msgr2=0x7f441019d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.076+0000 7f44182f4640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4410101390 0x7f441019d580 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f440c00d950 tx=0x7f440c00de10 comp rx=0 tx=0).stop 2024-02-17T12:11:38.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.076+0000 7f44182f4640 1 -- 172.21.15.94:0/1136042875 shutdown_connections 2024-02-17T12:11:38.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.076+0000 7f44182f4640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43fc0a22f0 0x7f43fc0a47b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.076+0000 7f44182f4640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4410197690 0x7f4410197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.076+0000 7f44182f4640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4410106460 0x7f441019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.076+0000 7f44182f4640 1 --2- 172.21.15.94:0/1136042875 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4410101390 0x7f441019d580 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.076+0000 7f44182f4640 1 -- 172.21.15.94:0/1136042875 >> 172.21.15.94:0/1136042875 conn(0x7f44100fcbb0 msgr2=0x7f44100fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:38.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.076+0000 7f44182f4640 1 -- 172.21.15.94:0/1136042875 shutdown_connections 2024-02-17T12:11:38.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.076+0000 7f44182f4640 1 -- 172.21.15.94:0/1136042875 wait complete. 2024-02-17T12:11:38.090 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:11:38.090 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:11:38.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.256+0000 7f09da4b5640 1 -- 172.21.15.94:0/2062148734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d40ff5d0 msgr2=0x7f09d40ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.256+0000 7f09da4b5640 1 --2- 172.21.15.94:0/2062148734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d40ff5d0 0x7f09d40ff9b0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009890 tx=0x7f09c802bb90 comp rx=0 tx=0).stop 2024-02-17T12:11:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.264+0000 7f09da4b5640 1 -- 172.21.15.94:0/2062148734 shutdown_connections 2024-02-17T12:11:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.264+0000 7f09da4b5640 1 --2- 172.21.15.94:0/2062148734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09d41046a0 0x7f09d410c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.264+0000 7f09da4b5640 1 --2- 172.21.15.94:0/2062148734 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09d40fff80 0x7f09d4103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.264+0000 7f09da4b5640 1 --2- 172.21.15.94:0/2062148734 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d40ff5d0 0x7f09d40ff9b0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.264+0000 7f09da4b5640 1 -- 172.21.15.94:0/2062148734 >> 172.21.15.94:0/2062148734 conn(0x7f09d40761d0 msgr2=0x7f09d40fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:38.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.268+0000 7f09da4b5640 1 -- 172.21.15.94:0/2062148734 shutdown_connections 2024-02-17T12:11:38.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.272+0000 7f09da4b5640 1 -- 172.21.15.94:0/2062148734 wait complete. 2024-02-17T12:11:38.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.272+0000 7f09da4b5640 1 Processor -- start 2024-02-17T12:11:38.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.273+0000 7f09da4b5640 1 -- start start 2024-02-17T12:11:38.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.273+0000 7f09da4b5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09d40ff5d0 0x7f09d4072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.273+0000 7f09da4b5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09d41046a0 0x7f09d4072e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.273+0000 7f09da4b5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d410b5d0 0x7f09d410ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.273+0000 7f09da4b5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d410c210 con 0x7f09d410b5d0 2024-02-17T12:11:38.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.273+0000 7f09da4b5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d410c380 con 0x7f09d41046a0 2024-02-17T12:11:38.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.273+0000 7f09da4b5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d406e2a0 con 0x7f09d40ff5d0 2024-02-17T12:11:38.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.275+0000 7f09d37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09d41046a0 0x7f09d4072e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.275+0000 7f09d37fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09d41046a0 0x7f09d4072e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52898/0 (socket says 172.21.15.94:52898) 2024-02-17T12:11:38.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.275+0000 7f09d37fe640 1 -- 172.21.15.94:0/550810934 learned_addr learned my addr 172.21.15.94:0/550810934 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:38.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.277+0000 7f09d3fff640 1 --2- 172.21.15.94:0/550810934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09d40ff5d0 0x7f09d4072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.277+0000 7f09d37fe640 1 -- 172.21.15.94:0/550810934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09d40ff5d0 msgr2=0x7f09d4072920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.277+0000 7f09d37fe640 1 --2- 172.21.15.94:0/550810934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09d40ff5d0 0x7f09d4072920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.277+0000 7f09d37fe640 1 -- 172.21.15.94:0/550810934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d410b5d0 msgr2=0x7f09d410ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.277+0000 7f09d37fe640 1 --2- 172.21.15.94:0/550810934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d410b5d0 0x7f09d410ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.277+0000 7f09d37fe640 1 -- 172.21.15.94:0/550810934 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c8009490 con 0x7f09d41046a0 2024-02-17T12:11:38.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.279+0000 7f09d37fe640 1 --2- 172.21.15.94:0/550810934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09d41046a0 0x7f09d4072e60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009800 tx=0x7f09c4009cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:38.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.280+0000 7f09d17fa640 1 -- 172.21.15.94:0/550810934 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c400e8b0 con 0x7f09d41046a0 2024-02-17T12:11:38.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.280+0000 7f09d17fa640 1 -- 172.21.15.94:0/550810934 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09c4010040 con 0x7f09d41046a0 2024-02-17T12:11:38.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.280+0000 7f09d17fa640 1 -- 172.21.15.94:0/550810934 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4014e10 con 0x7f09d41046a0 2024-02-17T12:11:38.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.281+0000 7f09da4b5640 1 -- 172.21.15.94:0/550810934 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d406e580 con 0x7f09d41046a0 2024-02-17T12:11:38.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.281+0000 7f09da4b5640 1 -- 172.21.15.94:0/550810934 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d406ea40 con 0x7f09d41046a0 2024-02-17T12:11:38.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.284+0000 7f09dd165740 1 Processor -- start 2024-02-17T12:11:38.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.284+0000 7f09dd165740 1 -- start start 2024-02-17T12:11:38.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.284+0000 7f09dd165740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fc7250 0x55db45fc7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.285+0000 7f09dd165740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7b70 0x55db45fd0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.285+0000 7f09dd165740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fd0750 0x55db45fd2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.285+0000 7f09dd165740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db45f82960 con 0x55db45fc7b70 2024-02-17T12:11:38.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.285+0000 7f09dd165740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db45f36310 con 0x55db45fc7250 2024-02-17T12:11:38.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.285+0000 7f09dd165740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db45f33840 con 0x55db45fd0750 2024-02-17T12:11:38.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.285+0000 7f09d8a52640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fc7250 0x55db45fc7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.285+0000 7f09d8a52640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fc7250 0x55db45fc7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52910/0 (socket says 172.21.15.94:52910) 2024-02-17T12:11:38.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.285+0000 7f09d8a52640 1 -- 172.21.15.94:0/1409401857 learned_addr learned my addr 172.21.15.94:0/1409401857 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:38.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.285+0000 7f09d8a52640 1 -- 172.21.15.94:0/1409401857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fd0750 msgr2=0x55db45fd2b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:11:38.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.285+0000 7f09d8a52640 1 --2- 172.21.15.94:0/1409401857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fd0750 0x55db45fd2b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.285+0000 7f09d8a52640 1 -- 172.21.15.94:0/1409401857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7b70 msgr2=0x55db45fd0210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:11:38.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.285+0000 7f09d8a52640 1 --2- 172.21.15.94:0/1409401857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7b70 0x55db45fd0210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.285+0000 7f09d8a52640 1 -- 172.21.15.94:0/1409401857 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db45ef1060 con 0x55db45fc7250 2024-02-17T12:11:38.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.286+0000 7f09d8a52640 1 --2- 172.21.15.94:0/1409401857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fc7250 0x55db45fc7630 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009d00 tx=0x7f09cc00b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac2c91eb13902cba server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:38.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.287+0000 7f09ba7fc640 1 -- 172.21.15.94:0/1409401857 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00e070 con 0x55db45fc7250 2024-02-17T12:11:38.291 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.287+0000 7f09ba7fc640 1 -- 172.21.15.94:0/1409401857 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09cc00bb30 con 0x55db45fc7250 2024-02-17T12:11:38.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.287+0000 7f09ba7fc640 1 -- 172.21.15.94:0/1409401857 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc0115f0 con 0x55db45fc7250 2024-02-17T12:11:38.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.288+0000 7f09dd165740 1 -- 172.21.15.94:0/1409401857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fc7250 msgr2=0x55db45fc7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.288+0000 7f09dd165740 1 --2- 172.21.15.94:0/1409401857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fc7250 0x55db45fc7630 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009d00 tx=0x7f09cc00b420 comp rx=0 tx=0).stop 2024-02-17T12:11:38.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.289+0000 7f09dd165740 1 -- 172.21.15.94:0/1409401857 shutdown_connections 2024-02-17T12:11:38.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.289+0000 7f09dd165740 1 --2- 172.21.15.94:0/1409401857 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fd0750 0x55db45fd2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.289+0000 7f09dd165740 1 --2- 172.21.15.94:0/1409401857 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7b70 0x55db45fd0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.289+0000 7f09dd165740 1 --2- 172.21.15.94:0/1409401857 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fc7250 0x55db45fc7630 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.289+0000 7f09dd165740 1 -- 172.21.15.94:0/1409401857 >> 172.21.15.94:0/1409401857 conn(0x55db45fc1980 msgr2=0x55db45fc1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:38.294 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.291+0000 7f09d17fa640 1 -- 172.21.15.94:0/550810934 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f09c400ea10 con 0x7f09d41046a0 2024-02-17T12:11:38.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.292+0000 7f09d17fa640 1 --2- 172.21.15.94:0/550810934 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09bc0a2440 0x7f09bc0a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.292+0000 7f09d17fa640 1 -- 172.21.15.94:0/550810934 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f09c40cd720 con 0x7f09d41046a0 2024-02-17T12:11:38.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.292+0000 7f09dd165740 1 -- 172.21.15.94:0/1409401857 shutdown_connections 2024-02-17T12:11:38.295 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.292+0000 7f09d3fff640 1 --2- 172.21.15.94:0/550810934 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09bc0a2440 0x7f09bc0a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.293+0000 7f09d3fff640 1 --2- 172.21.15.94:0/550810934 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09bc0a2440 0x7f09bc0a4900 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f09c8005b60 tx=0x7f09c8000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:38.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.293+0000 7f09dd165740 1 -- 172.21.15.94:0/1409401857 wait complete. 2024-02-17T12:11:38.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.293+0000 7f09dd165740 1 -- start start 2024-02-17T12:11:38.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.294+0000 7f09dd165740 1 Processor -- start 2024-02-17T12:11:38.296 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.294+0000 7f09dd165740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7250 0x55db45fcd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.294+0000 7f09dd165740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fc7b70 0x55db45fcdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.294+0000 7f09dd165740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fd0750 0x55db45fce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.294+0000 7f09dd165740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db45f2f020 con 0x55db45fc7250 2024-02-17T12:11:38.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.294+0000 7f09dd165740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db45c02430 con 0x55db45fc7b70 2024-02-17T12:11:38.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.294+0000 7f09dd165740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db45bbee00 con 0x55db45fd0750 2024-02-17T12:11:38.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.294+0000 7f09dd165740 10 client.0 _ll_register_callbacks cb 0x55db45ed6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:38.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.295+0000 7f09d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7250 0x55db45fcd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.295+0000 7f09d3fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7250 0x55db45fcd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54352/0 (socket says 172.21.15.94:54352) 2024-02-17T12:11:38.297 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.295+0000 7f09d3fff640 1 -- 172.21.15.94:0/1204454331 learned_addr learned my addr 172.21.15.94:0/1204454331 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:38.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.295+0000 7f09d3fff640 1 -- 172.21.15.94:0/1204454331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fd0750 msgr2=0x55db45fce0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:11:38.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.295+0000 7f09d3fff640 1 --2- 172.21.15.94:0/1204454331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fd0750 0x55db45fce0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.295+0000 7f09d3fff640 1 -- 172.21.15.94:0/1204454331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fc7b70 msgr2=0x55db45fcdbb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:11:38.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.295+0000 7f09d3fff640 1 --2- 172.21.15.94:0/1204454331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fc7b70 0x55db45fcdbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.295+0000 7f09d3fff640 1 -- 172.21.15.94:0/1204454331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc0099b0 con 0x55db45fc7250 2024-02-17T12:11:38.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.296+0000 7f09d3fff640 1 --2- 172.21.15.94:0/1204454331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7250 0x55db45fcd670 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f09c8036000 tx=0x7f09c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc910304b6a9f65 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:38.298 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.296+0000 7f09b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:38.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.296+0000 7f09b37fe640 1 -- 172.21.15.94:0/1204454331 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8045070 con 0x55db45fc7250 2024-02-17T12:11:38.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.296+0000 7f09b37fe640 1 -- 172.21.15.94:0/1204454331 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f09c802c740 con 0x55db45fc7250 2024-02-17T12:11:38.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.296+0000 7f09dd165740 1 -- 172.21.15.94:0/1204454331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db45c2e7e0 con 0x55db45fc7250 2024-02-17T12:11:38.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.297+0000 7f09b37fe640 1 -- 172.21.15.94:0/1204454331 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09c802c900 con 0x55db45fc7250 2024-02-17T12:11:38.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.297+0000 7f09dd165740 10 client.19005 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:38.299 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.297+0000 7f09dd165740 1 -- 172.21.15.94:0/1204454331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55db45f2d6a0 con 0x55db45fc7250 2024-02-17T12:11:38.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.298+0000 7f09b37fe640 1 -- 172.21.15.94:0/1204454331 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55db45c2e7e0 con 0x55db45fc7250 2024-02-17T12:11:38.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.298+0000 7f09b37fe640 1 -- 172.21.15.94:0/1204454331 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f09c8052070 con 0x55db45fc7250 2024-02-17T12:11:38.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.298+0000 7f09dd165740 10 client.19005 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:38.300 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.298+0000 7f09dd165740 10 client.19005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:38.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.298+0000 7f09dd165740 20 client.19005 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:38.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.298+0000 7f09dd165740 1 --2- 172.21.15.94:0/1204454331 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db45fe0850 0x55db45fe2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.298+0000 7f09dd165740 4 client.19005 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:38.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.298+0000 7f09dd165740 1 -- 172.21.15.94:0/1204454331 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db45c6b720 con 0x55db45fe0850 2024-02-17T12:11:38.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.298+0000 7f09d37fe640 1 --2- 172.21.15.94:0/1204454331 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db45fe0850 0x55db45fe2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.299+0000 7f09d37fe640 1 --2- 172.21.15.94:0/1204454331 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db45fe0850 0x55db45fe2cb0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed088c0be42bc76 server_cookie=ff97ef49a161e1b4 in_seq=0 out_seq=0 2024-02-17T12:11:38.301 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.299+0000 7f09b37fe640 10 client.19005 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:38.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.300+0000 7f09b37fe640 1 -- 172.21.15.94:0/1204454331 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db45c6b720 con 0x55db45fe0850 2024-02-17T12:11:38.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.300+0000 7f09b37fe640 10 client.19005 handle_command_reply: tid=0 2024-02-17T12:11:38.302 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.301+0000 7f09dd165740 1 client.19005 shutdown 2024-02-17T12:11:38.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.302+0000 7f09dd165740 1 -- 172.21.15.94:0/1204454331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7250 msgr2=0x55db45fcd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.302+0000 7f09dd165740 1 --2- 172.21.15.94:0/1204454331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7250 0x55db45fcd670 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f09c8036000 tx=0x7f09c8005f90 comp rx=0 tx=0).stop 2024-02-17T12:11:38.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.302+0000 7f09dd165740 1 -- 172.21.15.94:0/1204454331 shutdown_connections 2024-02-17T12:11:38.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.302+0000 7f09dd165740 1 --2- 172.21.15.94:0/1204454331 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db45fe0850 0x55db45fe2cb0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.302+0000 7f09dd165740 1 --2- 172.21.15.94:0/1204454331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fd0750 0x55db45fce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.302+0000 7f09dd165740 1 --2- 172.21.15.94:0/1204454331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fc7b70 0x55db45fcdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.302+0000 7f09dd165740 1 --2- 172.21.15.94:0/1204454331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7250 0x55db45fcd670 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.302+0000 7f09dd165740 1 -- 172.21.15.94:0/1204454331 >> 172.21.15.94:0/1204454331 conn(0x55db45fc1980 msgr2=0x55db45fc36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:38.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.302+0000 7f09b37fe640 0 client.19005 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:38.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.303+0000 7f09dd165740 1 -- 172.21.15.94:0/1204454331 shutdown_connections 2024-02-17T12:11:38.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.303+0000 7f09dd165740 1 -- 172.21.15.94:0/1204454331 wait complete. 2024-02-17T12:11:38.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.303+0000 7f09dd165740 20 client.19005 trim_cache size 0 max 16384 2024-02-17T12:11:38.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.319+0000 7f09dd165740 1 Processor -- start 2024-02-17T12:11:38.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09dd165740 1 -- start start 2024-02-17T12:11:38.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09dd165740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fc7250 0x55db45fcfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09dd165740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7b70 0x55db45fccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09dd165740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fd0750 0x55db45fca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09dd165740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db45eb4130 con 0x55db45fc7b70 2024-02-17T12:11:38.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09dd165740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db45fc2780 con 0x55db45fd0750 2024-02-17T12:11:38.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09dd165740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db45fc2bb0 con 0x55db45fc7250 2024-02-17T12:11:38.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09d3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fc7250 0x55db45fcfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09d37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7b70 0x55db45fccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09d3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fc7250 0x55db45fcfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:34608/0 (socket says 172.21.15.94:34608) 2024-02-17T12:11:38.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09d3fff640 1 -- 172.21.15.94:0/1144178552 learned_addr learned my addr 172.21.15.94:0/1144178552 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:38.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09d8a52640 1 --2- 172.21.15.94:0/1144178552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fd0750 0x55db45fca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09d37fe640 1 -- 172.21.15.94:0/1144178552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fc7250 msgr2=0x55db45fcfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:38.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09d37fe640 1 --2- 172.21.15.94:0/1144178552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fc7250 0x55db45fcfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09d37fe640 1 -- 172.21.15.94:0/1144178552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fd0750 msgr2=0x55db45fca8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:38.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09d37fe640 1 --2- 172.21.15.94:0/1144178552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fd0750 0x55db45fca8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09d37fe640 1 -- 172.21.15.94:0/1144178552 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db45ec9c20 con 0x55db45fc7b70 2024-02-17T12:11:38.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.320+0000 7f09d8a52640 1 --2- 172.21.15.94:0/1144178552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fd0750 0x55db45fca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:11:38.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.321+0000 7f09d37fe640 1 --2- 172.21.15.94:0/1144178552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7b70 0x55db45fccf50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f09c40cadd0 tx=0x7f09c4095d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1028bde084f1169f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:38.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.321+0000 7f09b9ffb640 1 -- 172.21.15.94:0/1144178552 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c409a880 con 0x55db45fc7b70 2024-02-17T12:11:38.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.321+0000 7f09b9ffb640 1 -- 172.21.15.94:0/1144178552 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09c40af070 con 0x55db45fc7b70 2024-02-17T12:11:38.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.321+0000 7f09b9ffb640 1 -- 172.21.15.94:0/1144178552 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c40967f0 con 0x55db45fc7b70 2024-02-17T12:11:38.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.322+0000 7f09dd165740 1 -- 172.21.15.94:0/1144178552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7b70 msgr2=0x55db45fccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.322+0000 7f09dd165740 1 --2- 172.21.15.94:0/1144178552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7b70 0x55db45fccf50 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f09c40cadd0 tx=0x7f09c4095d00 comp rx=0 tx=0).stop 2024-02-17T12:11:38.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.322+0000 7f09dd165740 1 -- 172.21.15.94:0/1144178552 shutdown_connections 2024-02-17T12:11:38.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.322+0000 7f09dd165740 1 --2- 172.21.15.94:0/1144178552 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fd0750 0x55db45fca8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.322+0000 7f09dd165740 1 --2- 172.21.15.94:0/1144178552 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fc7b70 0x55db45fccf50 secure :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f09c40cadd0 tx=0x7f09c4095d00 comp rx=0 tx=0).stop 2024-02-17T12:11:38.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.322+0000 7f09dd165740 1 --2- 172.21.15.94:0/1144178552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fc7250 0x55db45fcfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.322+0000 7f09dd165740 1 -- 172.21.15.94:0/1144178552 >> 172.21.15.94:0/1144178552 conn(0x55db45fc1980 msgr2=0x55db45fc3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:38.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.322+0000 7f09dd165740 1 -- 172.21.15.94:0/1144178552 shutdown_connections 2024-02-17T12:11:38.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.322+0000 7f09dd165740 1 -- 172.21.15.94:0/1144178552 wait complete. 2024-02-17T12:11:38.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.323+0000 7f09dd165740 1 -- start start 2024-02-17T12:11:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.323+0000 7f09dd165740 1 Processor -- start 2024-02-17T12:11:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.323+0000 7f09dd165740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fc7250 0x55db460315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.323+0000 7f09dd165740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fd0750 0x55db46031af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.323+0000 7f09dd165740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fd6ca0 0x55db45fd7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.323+0000 7f09dd165740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db45fd7660 con 0x55db45fd0750 2024-02-17T12:11:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.323+0000 7f09dd165740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db45fd77d0 con 0x55db45fd6ca0 2024-02-17T12:11:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.323+0000 7f09dd165740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db45fd7990 con 0x55db45fc7250 2024-02-17T12:11:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.323+0000 7f09dd165740 10 client.0 _ll_register_callbacks cb 0x7f09cc009510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.323+0000 7f09d37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fd0750 0x55db46031af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09d8a52640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fd6ca0 0x55db45fd7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09d3fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fc7250 0x55db460315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09d37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fd0750 0x55db46031af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54370/0 (socket says 172.21.15.94:54370) 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09d37fe640 1 -- 172.21.15.94:0/2611286878 learned_addr learned my addr 172.21.15.94:0/2611286878 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09d8a52640 1 -- 172.21.15.94:0/2611286878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fc7250 msgr2=0x55db460315b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09d8a52640 1 --2- 172.21.15.94:0/2611286878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fc7250 0x55db460315b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09d8a52640 1 -- 172.21.15.94:0/2611286878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fd0750 msgr2=0x55db46031af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09d8a52640 1 --2- 172.21.15.94:0/2611286878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fd0750 0x55db46031af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09d8a52640 1 -- 172.21.15.94:0/2611286878 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d406ea40 con 0x55db45fd6ca0 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09d3fff640 1 --2- 172.21.15.94:0/2611286878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fc7250 0x55db460315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09d37fe640 1 --2- 172.21.15.94:0/2611286878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fd0750 0x55db46031af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09d8a52640 1 --2- 172.21.15.94:0/2611286878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fd6ca0 0x55db45fd7120 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00b920 tx=0x7f09cc012e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32b9af732e754792 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09b37fe640 1 -- 172.21.15.94:0/2611286878 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc00e040 con 0x55db45fd6ca0 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09b37fe640 1 -- 172.21.15.94:0/2611286878 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f09cc003c30 con 0x55db45fd6ca0 2024-02-17T12:11:38.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09dd165740 1 -- 172.21.15.94:0/2611286878 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09cc00bb30 con 0x55db45fd6ca0 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09b37fe640 1 -- 172.21.15.94:0/2611286878 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09cc009830 con 0x55db45fd6ca0 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09dd165740 10 client.28490 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.324+0000 7f09dd165740 1 -- 172.21.15.94:0/2611286878 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55db45eef8c0 con 0x55db45fd6ca0 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09b37fe640 1 -- 172.21.15.94:0/2611286878 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f09cc030080 con 0x55db45fd6ca0 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09b37fe640 1 -- 172.21.15.94:0/2611286878 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f09cc023030 con 0x55db45fd6ca0 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09dd165740 10 client.28490 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09dd165740 10 client.28490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09dd165740 20 client.28490 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09dd165740 1 --2- 172.21.15.94:0/2611286878 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db46039810 0x55db4603bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09dd165740 4 client.28490 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09dd165740 1 -- 172.21.15.94:0/2611286878 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55db45cba890 con 0x55db46039810 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09d3fff640 1 --2- 172.21.15.94:0/2611286878 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db46039810 0x55db4603bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09b37fe640 10 client.28490 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09d3fff640 1 --2- 172.21.15.94:0/2611286878 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db46039810 0x55db4603bc30 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba563ab17cf89822 server_cookie=df887c377507c452 in_seq=0 out_seq=0 2024-02-17T12:11:38.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09b37fe640 1 -- 172.21.15.94:0/2611286878 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55db45cba890 con 0x55db46039810 2024-02-17T12:11:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09b37fe640 10 client.28490 handle_command_reply: tid=0 2024-02-17T12:11:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.325+0000 7f09dd165740 1 client.28490 shutdown 2024-02-17T12:11:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09dd165740 1 -- 172.21.15.94:0/2611286878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fd6ca0 msgr2=0x55db45fd7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09dd165740 1 --2- 172.21.15.94:0/2611286878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fd6ca0 0x55db45fd7120 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00b920 tx=0x7f09cc012e00 comp rx=0 tx=0).stop 2024-02-17T12:11:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09dd165740 1 -- 172.21.15.94:0/2611286878 shutdown_connections 2024-02-17T12:11:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09dd165740 1 --2- 172.21.15.94:0/2611286878 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db46039810 0x55db4603bc30 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09dd165740 1 --2- 172.21.15.94:0/2611286878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db45fd6ca0 0x55db45fd7120 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09b37fe640 0 client.28490 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09dd165740 1 --2- 172.21.15.94:0/2611286878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db45fd0750 0x55db46031af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09dd165740 1 --2- 172.21.15.94:0/2611286878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db45fc7250 0x55db460315b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09dd165740 1 -- 172.21.15.94:0/2611286878 >> 172.21.15.94:0/2611286878 conn(0x55db45fc1980 msgr2=0x55db45fc2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:38.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09dd165740 1 -- 172.21.15.94:0/2611286878 shutdown_connections 2024-02-17T12:11:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09dd165740 1 -- 172.21.15.94:0/2611286878 wait complete. 2024-02-17T12:11:38.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09dd165740 20 client.28490 trim_cache size 0 max 16384 2024-02-17T12:11:38.336 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:11:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09da4b5640 1 -- 172.21.15.94:0/550810934 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09bc0a2440 msgr2=0x7f09bc0a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09da4b5640 1 --2- 172.21.15.94:0/550810934 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09bc0a2440 0x7f09bc0a4900 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f09c8005b60 tx=0x7f09c8000bc0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09da4b5640 1 -- 172.21.15.94:0/550810934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09d41046a0 msgr2=0x7f09d4072e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09da4b5640 1 --2- 172.21.15.94:0/550810934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09d41046a0 0x7f09d4072e60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f09c4009800 tx=0x7f09c4009cc0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09da4b5640 1 -- 172.21.15.94:0/550810934 shutdown_connections 2024-02-17T12:11:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09da4b5640 1 --2- 172.21.15.94:0/550810934 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f09bc0a2440 0x7f09bc0a4900 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09da4b5640 1 --2- 172.21.15.94:0/550810934 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f09d410b5d0 0x7f09d410ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09da4b5640 1 --2- 172.21.15.94:0/550810934 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f09d41046a0 0x7f09d4072e60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09da4b5640 1 --2- 172.21.15.94:0/550810934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09d40ff5d0 0x7f09d4072920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:38.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.326+0000 7f09da4b5640 1 -- 172.21.15.94:0/550810934 >> 172.21.15.94:0/550810934 conn(0x7f09d40761d0 msgr2=0x7f09d4103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:38.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.327+0000 7f09da4b5640 1 -- 172.21.15.94:0/550810934 shutdown_connections 2024-02-17T12:11:38.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:38.327+0000 7f09da4b5640 1 -- 172.21.15.94:0/550810934 wait complete. 2024-02-17T12:11:38.340 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:38 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/206297585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:11:38.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:38 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/206297585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:11:38.437 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:38 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/206297585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:11:39.343 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:11:39.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:11:39.343 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:11:39.425 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:39 smithi189 ceph-mon[36588]: pgmap v423: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.8 MiB/s wr, 768 op/s 2024-02-17T12:11:39.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:39 smithi125 ceph-mon[36819]: pgmap v423: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.8 MiB/s wr, 768 op/s 2024-02-17T12:11:39.457 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:39 smithi094 ceph-mon[32499]: pgmap v423: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 8.8 MiB/s wr, 768 op/s 2024-02-17T12:11:39.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.466+0000 7f920ce17640 1 -- 172.21.15.94:0/3934334217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9208101770 msgr2=0x7f9208104d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.470 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.466+0000 7f920ce17640 1 --2- 172.21.15.94:0/3934334217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9208101770 0x7f9208104d10 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009930 tx=0x7f91f002bc30 comp rx=0 tx=0).stop 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.466+0000 7f920ce17640 1 -- 172.21.15.94:0/3934334217 shutdown_connections 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.466+0000 7f920ce17640 1 --2- 172.21.15.94:0/3934334217 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9208105440 0x7f920810cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.466+0000 7f920ce17640 1 --2- 172.21.15.94:0/3934334217 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9208101770 0x7f9208104d10 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.466+0000 7f920ce17640 1 --2- 172.21.15.94:0/3934334217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9208100dc0 0x7f92081011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.466+0000 7f920ce17640 1 -- 172.21.15.94:0/3934334217 >> 172.21.15.94:0/3934334217 conn(0x7f92080753c0 msgr2=0x7f92080757d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.466+0000 7f920ce17640 1 -- 172.21.15.94:0/3934334217 shutdown_connections 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.466+0000 7f920ce17640 1 -- 172.21.15.94:0/3934334217 wait complete. 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.467+0000 7f920ce17640 1 Processor -- start 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.467+0000 7f920ce17640 1 -- start start 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.467+0000 7f920ce17640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9208100dc0 0x7f92081963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.467+0000 7f920ce17640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9208105440 0x7f9208196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.467+0000 7f920ce17640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9208196f00 0x7f920819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.467+0000 7f920ce17640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920819bb90 con 0x7f9208100dc0 2024-02-17T12:11:39.471 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.467+0000 7f920ce17640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920819bd00 con 0x7f9208105440 2024-02-17T12:11:39.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.467+0000 7f920ce17640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920819bec0 con 0x7f9208196f00 2024-02-17T12:11:39.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.467+0000 7f920659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9208100dc0 0x7f92081963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.467+0000 7f920659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9208100dc0 0x7f92081963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54386/0 (socket says 172.21.15.94:54386) 2024-02-17T12:11:39.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.467+0000 7f920659c640 1 -- 172.21.15.94:0/794550747 learned_addr learned my addr 172.21.15.94:0/794550747 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:39.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.467+0000 7f91fffff640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9208105440 0x7f9208196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.467+0000 7f9206d9d640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9208196f00 0x7f920819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.468+0000 7f920659c640 1 -- 172.21.15.94:0/794550747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9208196f00 msgr2=0x7f920819b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.468+0000 7f920659c640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9208196f00 0x7f920819b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.468+0000 7f920659c640 1 -- 172.21.15.94:0/794550747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9208105440 msgr2=0x7f9208196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.468+0000 7f920659c640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9208105440 0x7f9208196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.468+0000 7f920659c640 1 -- 172.21.15.94:0/794550747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f0009560 con 0x7f9208100dc0 2024-02-17T12:11:39.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.468+0000 7f91fffff640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9208105440 0x7f9208196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:39.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.468+0000 7f920659c640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9208100dc0 0x7f92081963e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f91f800e9f0 tx=0x7f91f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:39.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.468+0000 7f91ff7fe640 1 -- 172.21.15.94:0/794550747 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8009700 con 0x7f9208100dc0 2024-02-17T12:11:39.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.468+0000 7f920ce17640 1 -- 172.21.15.94:0/794550747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920819c1a0 con 0x7f9208100dc0 2024-02-17T12:11:39.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.468+0000 7f91ff7fe640 1 -- 172.21.15.94:0/794550747 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91f800cd80 con 0x7f9208100dc0 2024-02-17T12:11:39.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.468+0000 7f91ff7fe640 1 -- 172.21.15.94:0/794550747 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8018920 con 0x7f9208100dc0 2024-02-17T12:11:39.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.468+0000 7f920ce17640 1 -- 172.21.15.94:0/794550747 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920819c6d0 con 0x7f9208100dc0 2024-02-17T12:11:39.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.470+0000 7f91ff7fe640 1 -- 172.21.15.94:0/794550747 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f91f8018a80 con 0x7f9208100dc0 2024-02-17T12:11:39.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.470+0000 7f920f9df740 1 Processor -- start 2024-02-17T12:11:39.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f920f9df740 1 -- start start 2024-02-17T12:11:39.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f920f9df740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2380 0x55d64a6b2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f920f9df740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2d30 0x55d64a6bb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f920f9df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a6bdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f920f9df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64a620f90 con 0x55d64a6bb8b0 2024-02-17T12:11:39.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f920f9df740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64a61e4c0 con 0x55d64a6b2d30 2024-02-17T12:11:39.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f920f9df740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64a619ca0 con 0x55d64a6b2380 2024-02-17T12:11:39.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f91fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a6bdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f9206d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2380 0x55d64a6b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f91fffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a6bdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54398/0 (socket says 172.21.15.94:54398) 2024-02-17T12:11:39.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f91fffff640 1 -- 172.21.15.94:0/1673398524 learned_addr learned my addr 172.21.15.94:0/1673398524 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:39.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f920659c640 1 --2- 172.21.15.94:0/1673398524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2d30 0x55d64a6bb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f91ff7fe640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f91f40a2230 0x7f91f40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f91fffff640 1 -- 172.21.15.94:0/1673398524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2380 msgr2=0x55d64a6b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:39.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f91fffff640 1 --2- 172.21.15.94:0/1673398524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2380 0x55d64a6b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f91fffff640 1 -- 172.21.15.94:0/1673398524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2d30 msgr2=0x55d64a6bb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:39.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f91fffff640 1 --2- 172.21.15.94:0/1673398524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2d30 0x55d64a6bb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f91fffff640 1 -- 172.21.15.94:0/1673398524 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d64a5dc0b0 con 0x55d64a6bb8b0 2024-02-17T12:11:39.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f9206d9d640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f91f40a2230 0x7f91f40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f91ff7fe640 1 -- 172.21.15.94:0/794550747 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f91f8014070 con 0x7f9208100dc0 2024-02-17T12:11:39.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.471+0000 7f91fffff640 1 --2- 172.21.15.94:0/1673398524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a6bdd00 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f91f0005ab0 tx=0x7f91f002c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333ee24b7dfac430 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:39.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.472+0000 7f91fcff9640 1 -- 172.21.15.94:0/1673398524 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f002c7e0 con 0x55d64a6bb8b0 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.472+0000 7f9206d9d640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f91f40a2230 0x7f91f40a46f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9200005c40 tx=0x7f9200004a40 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.472+0000 7f91fcff9640 1 -- 172.21.15.94:0/1673398524 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91f0004500 con 0x55d64a6bb8b0 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.472+0000 7f91fcff9640 1 -- 172.21.15.94:0/1673398524 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0005430 con 0x55d64a6bb8b0 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.472+0000 7f920f9df740 1 -- 172.21.15.94:0/1673398524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 msgr2=0x55d64a6bdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.472+0000 7f920f9df740 1 --2- 172.21.15.94:0/1673398524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a6bdd00 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f91f0005ab0 tx=0x7f91f002c660 comp rx=0 tx=0).stop 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.472+0000 7f920f9df740 1 -- 172.21.15.94:0/1673398524 shutdown_connections 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.472+0000 7f920f9df740 1 --2- 172.21.15.94:0/1673398524 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a6bdd00 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.472+0000 7f920f9df740 1 --2- 172.21.15.94:0/1673398524 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2d30 0x55d64a6bb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.472+0000 7f920f9df740 1 --2- 172.21.15.94:0/1673398524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2380 0x55d64a6b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.472+0000 7f920f9df740 1 -- 172.21.15.94:0/1673398524 >> 172.21.15.94:0/1673398524 conn(0x55d64a6ac840 msgr2=0x55d64a6acc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f920f9df740 1 -- 172.21.15.94:0/1673398524 shutdown_connections 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f920f9df740 1 -- 172.21.15.94:0/1673398524 wait complete. 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f920f9df740 1 -- start start 2024-02-17T12:11:39.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f920f9df740 1 Processor -- start 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f920f9df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6b2380 0x55d64a6b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f920f9df740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2d30 0x55d64a6b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f920f9df740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6bb8b0 0x55d64a6b92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f920f9df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64a5bef40 con 0x55d64a6b2380 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f920f9df740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64a2ed4b0 con 0x55d64a6bb8b0 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f920f9df740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64a59f670 con 0x55d64a6b2d30 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f920f9df740 10 client.0 _ll_register_callbacks cb 0x55d64a60cf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f9206d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6bb8b0 0x55d64a6b92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f9206d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6bb8b0 0x55d64a6b92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:52984/0 (socket says 172.21.15.94:52984) 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.473+0000 7f9206d9d640 1 -- 172.21.15.94:0/808532749 learned_addr learned my addr 172.21.15.94:0/808532749 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.474+0000 7f9206d9d640 1 -- 172.21.15.94:0/808532749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2d30 msgr2=0x55d64a6b8d60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.474+0000 7f9206d9d640 1 --2- 172.21.15.94:0/808532749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2d30 0x55d64a6b8d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.474+0000 7f9206d9d640 1 -- 172.21.15.94:0/808532749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6b2380 msgr2=0x55d64a6b8820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:11:39.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.474+0000 7f9206d9d640 1 --2- 172.21.15.94:0/808532749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6b2380 0x55d64a6b8820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.474+0000 7f9206d9d640 1 -- 172.21.15.94:0/808532749 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920810d790 con 0x55d64a6bb8b0 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.474+0000 7f9206d9d640 1 --2- 172.21.15.94:0/808532749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6bb8b0 0x55d64a6b92a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f920000fac0 tx=0x7f9200004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35cb754b29d644ce server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.475+0000 7f91edffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.475+0000 7f91edffb640 1 -- 172.21.15.94:0/808532749 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920001d2d0 con 0x55d64a6bb8b0 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.475+0000 7f91edffb640 1 -- 172.21.15.94:0/808532749 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f920001d5f0 con 0x55d64a6bb8b0 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.475+0000 7f920f9df740 1 -- 172.21.15.94:0/808532749 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d64a5f3f40 con 0x55d64a6bb8b0 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.475+0000 7f91edffb640 1 -- 172.21.15.94:0/808532749 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f920001d7b0 con 0x55d64a6bb8b0 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.475+0000 7f920f9df740 10 client.28502 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.475+0000 7f920f9df740 1 -- 172.21.15.94:0/808532749 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d64a618320 con 0x55d64a6bb8b0 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.476+0000 7f91edffb640 1 -- 172.21.15.94:0/808532749 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d64a5f3f40 con 0x55d64a6bb8b0 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.476+0000 7f91edffb640 1 -- 172.21.15.94:0/808532749 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9200037ba0 con 0x55d64a6bb8b0 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.476+0000 7f920f9df740 10 client.28502 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.476+0000 7f920f9df740 10 client.28502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.476+0000 7f920f9df740 20 client.28502 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:39.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.476+0000 7f920f9df740 1 --2- 172.21.15.94:0/808532749 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d64a6cbe00 0x55d64a6ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.476+0000 7f920f9df740 4 client.28502 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:39.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.476+0000 7f920f9df740 1 -- 172.21.15.94:0/808532749 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d64a610310 con 0x55d64a6cbe00 2024-02-17T12:11:39.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.476+0000 7f920659c640 1 --2- 172.21.15.94:0/808532749 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d64a6cbe00 0x55d64a6ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.476+0000 7f91edffb640 10 client.28502 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:39.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.476+0000 7f920659c640 1 --2- 172.21.15.94:0/808532749 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d64a6cbe00 0x55d64a6ce260 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=592d2defe5927972 server_cookie=9650956e2fc5232c in_seq=0 out_seq=0 2024-02-17T12:11:39.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f91edffb640 1 -- 172.21.15.94:0/808532749 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d64a610310 con 0x55d64a6cbe00 2024-02-17T12:11:39.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f91edffb640 10 client.28502 handle_command_reply: tid=0 2024-02-17T12:11:39.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f920f9df740 1 client.28502 shutdown 2024-02-17T12:11:39.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f920f9df740 1 -- 172.21.15.94:0/808532749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6bb8b0 msgr2=0x55d64a6b92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f920f9df740 1 --2- 172.21.15.94:0/808532749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6bb8b0 0x55d64a6b92a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f920000fac0 tx=0x7f9200004ab0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f920f9df740 1 -- 172.21.15.94:0/808532749 shutdown_connections 2024-02-17T12:11:39.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f920f9df740 1 --2- 172.21.15.94:0/808532749 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d64a6cbe00 0x55d64a6ce260 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f920f9df740 1 --2- 172.21.15.94:0/808532749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6bb8b0 0x55d64a6b92a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f920f9df740 1 --2- 172.21.15.94:0/808532749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2d30 0x55d64a6b8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f920f9df740 1 --2- 172.21.15.94:0/808532749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6b2380 0x55d64a6b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f91edffb640 0 client.28502 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:39.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f920f9df740 1 -- 172.21.15.94:0/808532749 >> 172.21.15.94:0/808532749 conn(0x55d64a6ac840 msgr2=0x55d64a6ae7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:39.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f920f9df740 1 -- 172.21.15.94:0/808532749 shutdown_connections 2024-02-17T12:11:39.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.477+0000 7f920f9df740 1 -- 172.21.15.94:0/808532749 wait complete. 2024-02-17T12:11:39.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.478+0000 7f920f9df740 20 client.28502 trim_cache size 0 max 16384 2024-02-17T12:11:39.516 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.514+0000 7f920f9df740 1 Processor -- start 2024-02-17T12:11:39.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.518+0000 7f920f9df740 1 -- start start 2024-02-17T12:11:39.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.518+0000 7f920f9df740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2380 0x55d64a6baf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.518+0000 7f920f9df740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2d30 0x55d64a6b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.518+0000 7f920f9df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a6b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.518+0000 7f920f9df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64a610310 con 0x55d64a6bb8b0 2024-02-17T12:11:39.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.518+0000 7f920f9df740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64a5f3f40 con 0x55d64a6b2d30 2024-02-17T12:11:39.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.518+0000 7f920f9df740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64a6a81d0 con 0x55d64a6b2380 2024-02-17T12:11:39.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.518+0000 7f920659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2380 0x55d64a6baf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.520+0000 7f9206d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a6b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.520+0000 7f9206d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a6b5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54400/0 (socket says 172.21.15.94:54400) 2024-02-17T12:11:39.522 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.520+0000 7f9206d9d640 1 -- 172.21.15.94:0/1674498224 learned_addr learned my addr 172.21.15.94:0/1674498224 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:39.525 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.523+0000 7f91fffff640 1 --2- 172.21.15.94:0/1674498224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2d30 0x55d64a6b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.527+0000 7f91fffff640 1 -- 172.21.15.94:0/1674498224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2380 msgr2=0x55d64a6baf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:39.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.527+0000 7f91fffff640 1 --2- 172.21.15.94:0/1674498224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2380 0x55d64a6baf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.527+0000 7f91fffff640 1 -- 172.21.15.94:0/1674498224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 msgr2=0x55d64a6b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:39.531 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.527+0000 7f91fffff640 1 --2- 172.21.15.94:0/1674498224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a6b5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.527+0000 7f91fffff640 1 -- 172.21.15.94:0/1674498224 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d64a5b4c00 con 0x55d64a6b2d30 2024-02-17T12:11:39.532 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.528+0000 7f91fffff640 1 --2- 172.21.15.94:0/1674498224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2d30 0x55d64a6b5a80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f91f002c170 tx=0x7f91f002bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5291aaf0e89617f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:39.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.533+0000 7f91effff640 1 -- 172.21.15.94:0/1674498224 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f002c9e0 con 0x55d64a6b2d30 2024-02-17T12:11:39.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.533+0000 7f91effff640 1 -- 172.21.15.94:0/1674498224 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91f002cb40 con 0x55d64a6b2d30 2024-02-17T12:11:39.536 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.533+0000 7f91effff640 1 -- 172.21.15.94:0/1674498224 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f003f830 con 0x55d64a6b2d30 2024-02-17T12:11:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.534+0000 7f920f9df740 1 -- 172.21.15.94:0/1674498224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2d30 msgr2=0x55d64a6b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.539 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.534+0000 7f920f9df740 1 --2- 172.21.15.94:0/1674498224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2d30 0x55d64a6b5a80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f91f002c170 tx=0x7f91f002bc30 comp rx=0 tx=0).stop 2024-02-17T12:11:39.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.538+0000 7f920f9df740 1 -- 172.21.15.94:0/1674498224 shutdown_connections 2024-02-17T12:11:39.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.538+0000 7f920f9df740 1 --2- 172.21.15.94:0/1674498224 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a6b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.538+0000 7f920f9df740 1 --2- 172.21.15.94:0/1674498224 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2d30 0x55d64a6b5a80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.543 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.538+0000 7f920f9df740 1 --2- 172.21.15.94:0/1674498224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2380 0x55d64a6baf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.544 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.538+0000 7f920f9df740 1 -- 172.21.15.94:0/1674498224 >> 172.21.15.94:0/1674498224 conn(0x55d64a6ac840 msgr2=0x55d64a6b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:39.545 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.544+0000 7f920f9df740 1 -- 172.21.15.94:0/1674498224 shutdown_connections 2024-02-17T12:11:39.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.545+0000 7f920f9df740 1 -- 172.21.15.94:0/1674498224 wait complete. 2024-02-17T12:11:39.551 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.546+0000 7f920f9df740 1 -- start start 2024-02-17T12:11:39.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.546+0000 7f920f9df740 1 Processor -- start 2024-02-17T12:11:39.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.546+0000 7f920f9df740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2380 0x55d64a6c1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.546+0000 7f920f9df740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2d30 0x55d64a6c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.546+0000 7f920f9df740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a71c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.546+0000 7f920f9df740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64a71c910 con 0x55d64a6bb8b0 2024-02-17T12:11:39.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.546+0000 7f920f9df740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64a71ca80 con 0x55d64a6b2380 2024-02-17T12:11:39.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.546+0000 7f920f9df740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d64a71cc40 con 0x55d64a6b2d30 2024-02-17T12:11:39.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.546+0000 7f920f9df740 10 client.0 _ll_register_callbacks cb 0x7f91f8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:39.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.547+0000 7f9206d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a71c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.547+0000 7f9206d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a71c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54416/0 (socket says 172.21.15.94:54416) 2024-02-17T12:11:39.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.547+0000 7f9206d9d640 1 -- 172.21.15.94:0/1131506965 learned_addr learned my addr 172.21.15.94:0/1131506965 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:39.552 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.547+0000 7f9206d9d640 1 -- 172.21.15.94:0/1131506965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2d30 msgr2=0x55d64a6c20a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:11:39.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.547+0000 7f9206d9d640 1 --2- 172.21.15.94:0/1131506965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2d30 0x55d64a6c20a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.547+0000 7f9206d9d640 1 -- 172.21.15.94:0/1131506965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2380 msgr2=0x55d64a6c1b60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:11:39.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.547+0000 7f9206d9d640 1 --2- 172.21.15.94:0/1131506965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2380 0x55d64a6c1b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.547+0000 7f9206d9d640 1 -- 172.21.15.94:0/1131506965 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d64a5dc0b0 con 0x55d64a6bb8b0 2024-02-17T12:11:39.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.547+0000 7f9206d9d640 1 --2- 172.21.15.94:0/1131506965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a71c3d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f920003a040 tx=0x7f920001b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df125718dde8952 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:39.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.547+0000 7f91edffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:39.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.548+0000 7f91edffb640 1 -- 172.21.15.94:0/1131506965 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200022070 con 0x55d64a6bb8b0 2024-02-17T12:11:39.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.548+0000 7f920f9df740 1 -- 172.21.15.94:0/1131506965 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d64a63f900 con 0x55d64a6bb8b0 2024-02-17T12:11:39.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.548+0000 7f91edffb640 1 -- 172.21.15.94:0/1131506965 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9200003b80 con 0x55d64a6bb8b0 2024-02-17T12:11:39.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.552+0000 7f91edffb640 1 -- 172.21.15.94:0/1131506965 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9200043070 con 0x55d64a6bb8b0 2024-02-17T12:11:39.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.553+0000 7f920f9df740 10 client.19026 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:39.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.553+0000 7f920f9df740 1 -- 172.21.15.94:0/1131506965 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d64a5da910 con 0x55d64a6bb8b0 2024-02-17T12:11:39.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.554+0000 7f91edffb640 1 -- 172.21.15.94:0/1131506965 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55d64a63f900 con 0x55d64a6bb8b0 2024-02-17T12:11:39.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.554+0000 7f91edffb640 1 -- 172.21.15.94:0/1131506965 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f920002d070 con 0x55d64a6bb8b0 2024-02-17T12:11:39.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.555+0000 7f920f9df740 10 client.19026 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:39.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.555+0000 7f920f9df740 10 client.19026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:39.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.555+0000 7f920f9df740 20 client.19026 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:39.557 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.555+0000 7f920f9df740 1 --2- 172.21.15.94:0/1131506965 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d64a724420 0x55d64a726880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.555+0000 7f920f9df740 4 client.19026 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:11:39.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.555+0000 7f920f9df740 1 -- 172.21.15.94:0/1131506965 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d64a319860 con 0x55d64a724420 2024-02-17T12:11:39.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.555+0000 7f920659c640 1 --2- 172.21.15.94:0/1131506965 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d64a724420 0x55d64a726880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.556+0000 7f920659c640 1 --2- 172.21.15.94:0/1131506965 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d64a724420 0x55d64a726880 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=439ea1a5389374a7 server_cookie=94c967e4926459b6 in_seq=0 out_seq=0 2024-02-17T12:11:39.558 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.556+0000 7f91edffb640 10 client.19026 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:39.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.557+0000 7f91edffb640 1 -- 172.21.15.94:0/1131506965 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d64a319860 con 0x55d64a724420 2024-02-17T12:11:39.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.557+0000 7f91edffb640 10 client.19026 handle_command_reply: tid=0 2024-02-17T12:11:39.560 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.557+0000 7f920f9df740 1 client.19026 shutdown 2024-02-17T12:11:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.557+0000 7f920f9df740 1 -- 172.21.15.94:0/1131506965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 msgr2=0x55d64a71c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.558+0000 7f920f9df740 1 --2- 172.21.15.94:0/1131506965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a71c3d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f920003a040 tx=0x7f920001b630 comp rx=0 tx=0).stop 2024-02-17T12:11:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.558+0000 7f920f9df740 1 -- 172.21.15.94:0/1131506965 shutdown_connections 2024-02-17T12:11:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.558+0000 7f920f9df740 1 --2- 172.21.15.94:0/1131506965 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d64a724420 0x55d64a726880 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.558+0000 7f920f9df740 1 --2- 172.21.15.94:0/1131506965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d64a6bb8b0 0x55d64a71c3d0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.558+0000 7f920f9df740 1 --2- 172.21.15.94:0/1131506965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d64a6b2d30 0x55d64a6c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.558+0000 7f920f9df740 1 --2- 172.21.15.94:0/1131506965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d64a6b2380 0x55d64a6c1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.558+0000 7f920f9df740 1 -- 172.21.15.94:0/1131506965 >> 172.21.15.94:0/1131506965 conn(0x55d64a6ac840 msgr2=0x55d64a6b87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.558+0000 7f91edffb640 0 client.19026 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.558+0000 7f920f9df740 1 -- 172.21.15.94:0/1131506965 shutdown_connections 2024-02-17T12:11:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.558+0000 7f920f9df740 1 -- 172.21.15.94:0/1131506965 wait complete. 2024-02-17T12:11:39.561 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.558+0000 7f920f9df740 20 client.19026 trim_cache size 0 max 16384 2024-02-17T12:11:39.562 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:11:39.562 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:11:39.562 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "b121859b-864d-4360-a54a-e5294be2afb3", 2024-02-17T12:11:39.562 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:11:39.562 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:11:39.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.559+0000 7f920ce17640 1 -- 172.21.15.94:0/794550747 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f91f40a2230 msgr2=0x7f91f40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.559+0000 7f920ce17640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f91f40a2230 0x7f91f40a46f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9200005c40 tx=0x7f9200004a40 comp rx=0 tx=0).stop 2024-02-17T12:11:39.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.559+0000 7f920ce17640 1 -- 172.21.15.94:0/794550747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9208100dc0 msgr2=0x7f92081963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.559+0000 7f920ce17640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9208100dc0 0x7f92081963e0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f91f800e9f0 tx=0x7f91f800eeb0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.559+0000 7f920ce17640 1 -- 172.21.15.94:0/794550747 shutdown_connections 2024-02-17T12:11:39.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.559+0000 7f920ce17640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f91f40a2230 0x7f91f40a46f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.559+0000 7f920ce17640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9208196f00 0x7f920819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.559+0000 7f920ce17640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9208105440 0x7f9208196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.559+0000 7f920ce17640 1 --2- 172.21.15.94:0/794550747 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9208100dc0 0x7f92081963e0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.559+0000 7f920ce17640 1 -- 172.21.15.94:0/794550747 >> 172.21.15.94:0/794550747 conn(0x7f92080753c0 msgr2=0x7f92080fdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:39.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.559+0000 7f920ce17640 1 -- 172.21.15.94:0/794550747 shutdown_connections 2024-02-17T12:11:39.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.559+0000 7f920ce17640 1 -- 172.21.15.94:0/794550747 wait complete. 2024-02-17T12:11:39.575 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:11:39.575 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:11:39.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.732+0000 7fb9dcd0c640 1 -- 172.21.15.94:0/2142853955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9d8108100 msgr2=0x7fb9d81145c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.732+0000 7fb9dcd0c640 1 --2- 172.21.15.94:0/2142853955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9d8108100 0x7fb9d81145c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009930 tx=0x7fb9c802bd10 comp rx=0 tx=0).stop 2024-02-17T12:11:39.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.736+0000 7fb9dcd0c640 1 -- 172.21.15.94:0/2142853955 shutdown_connections 2024-02-17T12:11:39.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.736+0000 7fb9dcd0c640 1 --2- 172.21.15.94:0/2142853955 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9d8108100 0x7fb9d81145c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.736+0000 7fb9dcd0c640 1 --2- 172.21.15.94:0/2142853955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9d8107760 0x7fb9d8107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.736+0000 7fb9dcd0c640 1 --2- 172.21.15.94:0/2142853955 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9d810c640 0x7fb9d810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.740 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.736+0000 7fb9dcd0c640 1 -- 172.21.15.94:0/2142853955 >> 172.21.15.94:0/2142853955 conn(0x7fb9d8076f50 msgr2=0x7fb9d8105fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:39.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.744+0000 7fb9dcd0c640 1 -- 172.21.15.94:0/2142853955 shutdown_connections 2024-02-17T12:11:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.751+0000 7fb9dcd0c640 1 -- 172.21.15.94:0/2142853955 wait complete. 2024-02-17T12:11:39.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.752+0000 7fb9dcd0c640 1 Processor -- start 2024-02-17T12:11:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.753+0000 7fb9dcd0c640 1 -- start start 2024-02-17T12:11:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.754+0000 7fb9dcd0c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9d8107760 0x7fb9d8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.754+0000 7fb9dcd0c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9d8108100 0x7fb9d8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.754+0000 7fb9dcd0c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9d810c640 0x7fb9d806b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.754+0000 7fb9dcd0c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d806c140 con 0x7fb9d8107760 2024-02-17T12:11:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.754+0000 7fb9dcd0c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d806c2b0 con 0x7fb9d8108100 2024-02-17T12:11:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.754+0000 7fb9dcd0c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d806c470 con 0x7fb9d810c640 2024-02-17T12:11:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.754+0000 7fb9d6d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9d810c640 0x7fb9d806b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.754+0000 7fb9d6d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9d810c640 0x7fb9d806b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:34672/0 (socket says 172.21.15.94:34672) 2024-02-17T12:11:39.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.754+0000 7fb9d6d9d640 1 -- 172.21.15.94:0/396831513 learned_addr learned my addr 172.21.15.94:0/396831513 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.756+0000 7fb9d6d9d640 1 -- 172.21.15.94:0/396831513 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9d8108100 msgr2=0x7fb9d8072d50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-17T12:11:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.756+0000 7fb9d6d9d640 1 --2- 172.21.15.94:0/396831513 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9d8108100 0x7fb9d8072d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.756+0000 7fb9d6d9d640 1 -- 172.21.15.94:0/396831513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9d8107760 msgr2=0x7fb9d8072810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:11:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.756+0000 7fb9d6d9d640 1 --2- 172.21.15.94:0/396831513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9d8107760 0x7fb9d8072810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.756+0000 7fb9d6d9d640 1 -- 172.21.15.94:0/396831513 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d0009840 con 0x7fb9d810c640 2024-02-17T12:11:39.763 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.760+0000 7fb9d6d9d640 1 --2- 172.21.15.94:0/396831513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9d810c640 0x7fb9d806b960 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb9c802c2a0 tx=0x7fb9c802ea20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:39.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.765+0000 7fb9c77fe640 1 -- 172.21.15.94:0/396831513 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c802eda0 con 0x7fb9d810c640 2024-02-17T12:11:39.769 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.765+0000 7fb9c77fe640 1 -- 172.21.15.94:0/396831513 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9c802e550 con 0x7fb9d810c640 2024-02-17T12:11:39.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.765+0000 7fb9c77fe640 1 -- 172.21.15.94:0/396831513 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8034680 con 0x7fb9d810c640 2024-02-17T12:11:39.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.768+0000 7fb9dcd0c640 1 -- 172.21.15.94:0/396831513 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c8009560 con 0x7fb9d810c640 2024-02-17T12:11:39.770 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.768+0000 7fb9dcd0c640 1 -- 172.21.15.94:0/396831513 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d8070510 con 0x7fb9d810c640 2024-02-17T12:11:39.775 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.773+0000 7fb9df9bf740 1 Processor -- start 2024-02-17T12:11:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.774+0000 7fb9df9bf740 1 -- start start 2024-02-17T12:11:39.776 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.774+0000 7fb9c77fe640 1 -- 172.21.15.94:0/396831513 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb9c802c940 con 0x7fb9d810c640 2024-02-17T12:11:39.777 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.776+0000 7fb9c77fe640 1 --2- 172.21.15.94:0/396831513 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb9c00a2440 0x7fb9c00a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.776+0000 7fb9df9bf740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372d6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.776+0000 7fb9df9bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 0x5607372df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.776+0000 7fb9df9bf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372e1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.776+0000 7fb9df9bf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560737245280 con 0x5607372df7b0 2024-02-17T12:11:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.776+0000 7fb9df9bf740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607372427b0 con 0x5607372d62b0 2024-02-17T12:11:39.778 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.776+0000 7fb9df9bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56073723df90 con 0x5607372d6bd0 2024-02-17T12:11:39.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.776+0000 7fb9c77fe640 1 -- 172.21.15.94:0/396831513 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb9c80ef680 con 0x7fb9d810c640 2024-02-17T12:11:39.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.776+0000 7fb9d659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372d6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.776+0000 7fb9d659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372d6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53030/0 (socket says 172.21.15.94:53030) 2024-02-17T12:11:39.779 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.776+0000 7fb9d659c640 1 -- 172.21.15.94:0/1579420258 learned_addr learned my addr 172.21.15.94:0/1579420258 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:39.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.777+0000 7fb9d6d9d640 1 --2- 172.21.15.94:0/1579420258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372e1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.777+0000 7fb9d659c640 1 -- 172.21.15.94:0/1579420258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 msgr2=0x5607372df270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:11:39.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.777+0000 7fb9d659c640 1 --2- 172.21.15.94:0/1579420258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 0x5607372df270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.777+0000 7fb9d659c640 1 -- 172.21.15.94:0/1579420258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 msgr2=0x5607372e1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:39.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.777+0000 7fb9d659c640 1 --2- 172.21.15.94:0/1579420258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372e1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.777+0000 7fb9d659c640 1 -- 172.21.15.94:0/1579420258 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607371fc570 con 0x5607372d62b0 2024-02-17T12:11:39.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.777+0000 7fb9d6d9d640 1 --2- 172.21.15.94:0/1579420258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372e1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:39.780 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.779+0000 7fb9d659c640 1 --2- 172.21.15.94:0/1579420258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372d6690 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009f80 tx=0x7fb9d000e7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eee4ca1c21928746 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:39.781 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.779+0000 7fb9d5d9b640 1 --2- 172.21.15.94:0/396831513 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb9c00a2440 0x7fb9c00a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.780+0000 7fb9c4ff9640 1 -- 172.21.15.94:0/1579420258 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0003b70 con 0x5607372d62b0 2024-02-17T12:11:39.782 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.780+0000 7fb9c4ff9640 1 -- 172.21.15.94:0/1579420258 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9d0003cd0 con 0x5607372d62b0 2024-02-17T12:11:39.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.781+0000 7fb9df9bf740 1 -- 172.21.15.94:0/1579420258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 msgr2=0x5607372d6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.781+0000 7fb9d5d9b640 1 --2- 172.21.15.94:0/396831513 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb9c00a2440 0x7fb9c00a4900 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0058c0 tx=0x7fb9cc005b20 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:39.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.781+0000 7fb9c4ff9640 1 -- 172.21.15.94:0/1579420258 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0003b70 con 0x5607372d62b0 2024-02-17T12:11:39.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.781+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/1579420258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372d6690 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009f80 tx=0x7fb9d000e7e0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.782+0000 7fb9df9bf740 1 -- 172.21.15.94:0/1579420258 shutdown_connections 2024-02-17T12:11:39.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.782+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/1579420258 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372e1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.783+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/1579420258 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 0x5607372df270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.783+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/1579420258 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372d6690 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.783+0000 7fb9df9bf740 1 -- 172.21.15.94:0/1579420258 >> 172.21.15.94:0/1579420258 conn(0x5607372d0990 msgr2=0x5607372d0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:39.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.784+0000 7fb9df9bf740 1 -- 172.21.15.94:0/1579420258 shutdown_connections 2024-02-17T12:11:39.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.785+0000 7fb9df9bf740 1 -- 172.21.15.94:0/1579420258 wait complete. 2024-02-17T12:11:39.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.785+0000 7fb9df9bf740 1 -- start start 2024-02-17T12:11:39.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.785+0000 7fb9df9bf740 1 Processor -- start 2024-02-17T12:11:39.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.786+0000 7fb9df9bf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372d62b0 0x5607372df100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.786+0000 7fb9df9bf740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d6bd0 0x5607372dc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.786+0000 7fb9df9bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372df7b0 0x5607372dc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.786+0000 7fb9df9bf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x560736f11430 con 0x5607372d62b0 2024-02-17T12:11:39.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.786+0000 7fb9df9bf740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x560736ecde00 con 0x5607372d6bd0 2024-02-17T12:11:39.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.786+0000 7fb9d6d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372df7b0 0x5607372dc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.787+0000 7fb9d6d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372df7b0 0x5607372dc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:34682/0 (socket says 172.21.15.94:34682) 2024-02-17T12:11:39.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.787+0000 7fb9d6d9d640 1 -- 172.21.15.94:0/893872592 learned_addr learned my addr 172.21.15.94:0/893872592 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:39.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.786+0000 7fb9d5d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d6bd0 0x5607372dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.789 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.787+0000 7fb9df9bf740 1 -- 172.21.15.94:0/893872592 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607371c2fd0 con 0x5607372df7b0 2024-02-17T12:11:39.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.787+0000 7fb9d6d9d640 1 -- 172.21.15.94:0/893872592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d6bd0 msgr2=0x5607372dc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:39.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.787+0000 7fb9d6d9d640 1 --2- 172.21.15.94:0/893872592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d6bd0 0x5607372dc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.787+0000 7fb9d6d9d640 1 -- 172.21.15.94:0/893872592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372d62b0 msgr2=0x5607372df100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:39.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.787+0000 7fb9d6d9d640 1 --2- 172.21.15.94:0/893872592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372d62b0 0x5607372df100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.787+0000 7fb9d6d9d640 1 -- 172.21.15.94:0/893872592 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d0009c60 con 0x5607372df7b0 2024-02-17T12:11:39.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.787+0000 7fb9df9bf740 10 client.0 _ll_register_callbacks cb 0x560737218fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:39.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.787+0000 7fb9df9bf740 1 -- 172.21.15.94:0/893872592 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560736f3d7e0 con 0x5607372df7b0 2024-02-17T12:11:39.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.788+0000 7fb9d6d9d640 1 --2- 172.21.15.94:0/893872592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372df7b0 0x5607372dc9f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80b8ab0 tx=0x7fb9c80bc8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24ce0d95948416c4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:39.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.788+0000 7fb9bdffb640 10 client.28471 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:11:39.790 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.788+0000 7fb9bdffb640 1 -- 172.21.15.94:0/893872592 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80d1070 con 0x5607372df7b0 2024-02-17T12:11:39.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.788+0000 7fb9bdffb640 1 -- 172.21.15.94:0/893872592 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb9c80c4880 con 0x5607372df7b0 2024-02-17T12:11:39.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.788+0000 7fb9bdffb640 1 -- 172.21.15.94:0/893872592 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9c80ba3e0 con 0x5607372df7b0 2024-02-17T12:11:39.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.792+0000 7fb9df9bf740 10 client.28471 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:39.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.792+0000 7fb9df9bf740 1 -- 172.21.15.94:0/893872592 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x56073723c610 con 0x5607372df7b0 2024-02-17T12:11:39.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.792+0000 7fb9bdffb640 1 -- 172.21.15.94:0/893872592 <== mon.1 v2:172.21.15.189:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb9c80d4030 con 0x5607372df7b0 2024-02-17T12:11:39.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.792+0000 7fb9df9bf740 10 client.28471 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:39.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.792+0000 7fb9df9bf740 10 client.28471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:39.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.792+0000 7fb9df9bf740 20 client.28471 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:39.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.792+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/893872592 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5607372ef340 0x5607372f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.792+0000 7fb9df9bf740 4 client.28471 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:39.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.792+0000 7fb9df9bf740 1 -- 172.21.15.94:0/893872592 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560736f7a720 con 0x5607372ef340 2024-02-17T12:11:39.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.793+0000 7fb9d659c640 1 --2- 172.21.15.94:0/893872592 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5607372ef340 0x5607372f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.794+0000 7fb9bdffb640 10 client.28471 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:39.796 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.794+0000 7fb9d659c640 1 --2- 172.21.15.94:0/893872592 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5607372ef340 0x5607372f17b0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa32bda9c9bca3eb server_cookie=a7cd826f2b293a66 in_seq=0 out_seq=0 2024-02-17T12:11:39.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.794+0000 7fb9bdffb640 1 -- 172.21.15.94:0/893872592 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560736f7a720 con 0x5607372ef340 2024-02-17T12:11:39.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.794+0000 7fb9bdffb640 10 client.28471 handle_command_reply: tid=0 2024-02-17T12:11:39.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.794+0000 7fb9df9bf740 1 client.28471 shutdown 2024-02-17T12:11:39.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.794+0000 7fb9df9bf740 1 -- 172.21.15.94:0/893872592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372df7b0 msgr2=0x5607372dc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.794+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/893872592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372df7b0 0x5607372dc9f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb9c80b8ab0 tx=0x7fb9c80bc8a0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.794+0000 7fb9df9bf740 1 -- 172.21.15.94:0/893872592 shutdown_connections 2024-02-17T12:11:39.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.794+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/893872592 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5607372ef340 0x5607372f17b0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.795+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/893872592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372df7b0 0x5607372dc9f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.795+0000 7fb9bdffb640 0 client.28471 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:39.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.795+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/893872592 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d6bd0 0x5607372dc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.795+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/893872592 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372d62b0 0x5607372df100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.795+0000 7fb9df9bf740 1 -- 172.21.15.94:0/893872592 >> 172.21.15.94:0/893872592 conn(0x5607372d0990 msgr2=0x5607372d2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:39.800 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.795+0000 7fb9df9bf740 1 -- 172.21.15.94:0/893872592 shutdown_connections 2024-02-17T12:11:39.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.795+0000 7fb9df9bf740 1 -- 172.21.15.94:0/893872592 wait complete. 2024-02-17T12:11:39.801 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.795+0000 7fb9df9bf740 20 client.28471 trim_cache size 0 max 16384 2024-02-17T12:11:39.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.817+0000 7fb9df9bf740 1 Processor -- start 2024-02-17T12:11:39.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.817+0000 7fb9df9bf740 1 -- start start 2024-02-17T12:11:39.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9df9bf740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372debc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9df9bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 0x5607372df100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9df9bf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372dad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9df9bf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607372cc0f0 con 0x5607372df7b0 2024-02-17T12:11:39.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9df9bf740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607372ec430 con 0x5607372d62b0 2024-02-17T12:11:39.821 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9df9bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607372e2480 con 0x5607372d6bd0 2024-02-17T12:11:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9d659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372debc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9d6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372dad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9d5d9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 0x5607372df100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9d659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372debc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53050/0 (socket says 172.21.15.94:53050) 2024-02-17T12:11:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9d659c640 1 -- 172.21.15.94:0/4092089882 learned_addr learned my addr 172.21.15.94:0/4092089882 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9d659c640 1 -- 172.21.15.94:0/4092089882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 msgr2=0x5607372df100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9d659c640 1 --2- 172.21.15.94:0/4092089882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 0x5607372df100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9d659c640 1 -- 172.21.15.94:0/4092089882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 msgr2=0x5607372dad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.818+0000 7fb9d659c640 1 --2- 172.21.15.94:0/4092089882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372dad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.819+0000 7fb9d659c640 1 -- 172.21.15.94:0/4092089882 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607371d8ac0 con 0x5607372d62b0 2024-02-17T12:11:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.819+0000 7fb9d6d9d640 1 --2- 172.21.15.94:0/4092089882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372dad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.819+0000 7fb9d5d9b640 1 --2- 172.21.15.94:0/4092089882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 0x5607372df100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:11:39.822 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.819+0000 7fb9d659c640 1 --2- 172.21.15.94:0/4092089882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372debc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009f50 tx=0x7fb9d0010630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81123a09a2fb7d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:39.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.819+0000 7fb9bffff640 1 -- 172.21.15.94:0/4092089882 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0003a70 con 0x5607372d62b0 2024-02-17T12:11:39.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.819+0000 7fb9bffff640 1 -- 172.21.15.94:0/4092089882 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9d00106a0 con 0x5607372d62b0 2024-02-17T12:11:39.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.819+0000 7fb9bffff640 1 -- 172.21.15.94:0/4092089882 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0004ea0 con 0x5607372d62b0 2024-02-17T12:11:39.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.820+0000 7fb9df9bf740 1 -- 172.21.15.94:0/4092089882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 msgr2=0x5607372debc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.820+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/4092089882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372debc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009f50 tx=0x7fb9d0010630 comp rx=0 tx=0).stop 2024-02-17T12:11:39.825 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.820+0000 7fb9df9bf740 1 -- 172.21.15.94:0/4092089882 shutdown_connections 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.820+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/4092089882 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372dad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.820+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/4092089882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 0x5607372df100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.820+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/4092089882 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372debc0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.820+0000 7fb9df9bf740 1 -- 172.21.15.94:0/4092089882 >> 172.21.15.94:0/4092089882 conn(0x5607372d0990 msgr2=0x5607372dc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.820+0000 7fb9df9bf740 1 -- 172.21.15.94:0/4092089882 shutdown_connections 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.820+0000 7fb9df9bf740 1 -- 172.21.15.94:0/4092089882 wait complete. 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.820+0000 7fb9df9bf740 1 -- start start 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.820+0000 7fb9df9bf740 1 Processor -- start 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9df9bf740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9df9bf740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 0x5607372e3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9df9bf740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372e4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9df9bf740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607372e4890 con 0x5607372df7b0 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9df9bf740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607372e4a00 con 0x5607372d62b0 2024-02-17T12:11:39.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9df9bf740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607372e4bc0 con 0x5607372d6bd0 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9df9bf740 10 client.0 _ll_register_callbacks cb 0x7fb9d0009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9d659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9d659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372e69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:53052/0 (socket says 172.21.15.94:53052) 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9d659c640 1 -- 172.21.15.94:0/870945638 learned_addr learned my addr 172.21.15.94:0/870945638 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9d6d9d640 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372e4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9d5d9b640 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 0x5607372e3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9d6d9d640 1 -- 172.21.15.94:0/870945638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 msgr2=0x5607372e3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9d6d9d640 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 0x5607372e3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9d6d9d640 1 -- 172.21.15.94:0/870945638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 msgr2=0x5607372e69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9d6d9d640 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372e69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9d6d9d640 1 -- 172.21.15.94:0/870945638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d0009c60 con 0x5607372df7b0 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9d659c640 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9d6d9d640 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372e4280 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8046040 tx=0x7fb9c80b8380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44480dc71be21d29 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.821+0000 7fb9bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:39.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.822+0000 7fb9bdffb640 1 -- 172.21.15.94:0/870945638 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80d1040 con 0x5607372df7b0 2024-02-17T12:11:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.822+0000 7fb9bdffb640 1 -- 172.21.15.94:0/870945638 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb9c80d9030 con 0x5607372df7b0 2024-02-17T12:11:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.822+0000 7fb9df9bf740 1 -- 172.21.15.94:0/870945638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9d0003cd0 con 0x5607372df7b0 2024-02-17T12:11:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.823+0000 7fb9bdffb640 1 -- 172.21.15.94:0/870945638 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9c80bbeb0 con 0x5607372df7b0 2024-02-17T12:11:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.823+0000 7fb9df9bf740 10 client.19038 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.823+0000 7fb9df9bf740 1 -- 172.21.15.94:0/870945638 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fb9d001c040 con 0x5607372df7b0 2024-02-17T12:11:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.824+0000 7fb9bdffb640 1 -- 172.21.15.94:0/870945638 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb9c80d8020 con 0x5607372df7b0 2024-02-17T12:11:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.824+0000 7fb9bdffb640 1 -- 172.21.15.94:0/870945638 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb9c80d4040 con 0x5607372df7b0 2024-02-17T12:11:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.824+0000 7fb9df9bf740 10 client.19038 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.824+0000 7fb9df9bf740 10 client.19038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.824+0000 7fb9df9bf740 20 client.19038 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.824+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560737347b30 0x560737349f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:39.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.824+0000 7fb9df9bf740 4 client.19038 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:11:39.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.824+0000 7fb9df9bf740 1 -- 172.21.15.94:0/870945638 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fb9d0003cd0 con 0x560737347b30 2024-02-17T12:11:39.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.824+0000 7fb9d659c640 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560737347b30 0x560737349f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:39.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.824+0000 7fb9bdffb640 10 client.19038 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:39.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.824+0000 7fb9d659c640 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560737347b30 0x560737349f20 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c0f5f887a59b4c server_cookie=afd8c3b5c0d1762d in_seq=0 out_seq=0 2024-02-17T12:11:39.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.824+0000 7fb9bdffb640 1 -- 172.21.15.94:0/870945638 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x7fb9d0003cd0 con 0x560737347b30 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.824+0000 7fb9bdffb640 10 client.19038 handle_command_reply: tid=0 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9df9bf740 1 client.19038 shutdown 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9df9bf740 1 -- 172.21.15.94:0/870945638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 msgr2=0x5607372e4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372e4280 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8046040 tx=0x7fb9c80b8380 comp rx=0 tx=0).stop 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9df9bf740 1 -- 172.21.15.94:0/870945638 shutdown_connections 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x560737347b30 0x560737349f20 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5607372df7b0 0x5607372e4280 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5607372d6bd0 0x5607372e3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9df9bf740 1 --2- 172.21.15.94:0/870945638 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5607372d62b0 0x5607372e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9df9bf740 1 -- 172.21.15.94:0/870945638 >> 172.21.15.94:0/870945638 conn(0x5607372d0990 msgr2=0x5607372d1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9bdffb640 0 client.19038 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9df9bf740 1 -- 172.21.15.94:0/870945638 shutdown_connections 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9df9bf740 1 -- 172.21.15.94:0/870945638 wait complete. 2024-02-17T12:11:39.831 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9df9bf740 20 client.19038 trim_cache size 0 max 16384 2024-02-17T12:11:39.832 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:11:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:11:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:11:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "b121859b-864d-4360-a54a-e5294be2afb3": { 2024-02-17T12:11:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:11:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "b121859b-864d-4360-a54a-e5294be2afb3", 2024-02-17T12:11:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:11:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:11:39.832 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:11:39.833 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:11:39.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9dcd0c640 1 -- 172.21.15.94:0/396831513 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb9c00a2440 msgr2=0x7fb9c00a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9dcd0c640 1 --2- 172.21.15.94:0/396831513 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb9c00a2440 0x7fb9c00a4900 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc0058c0 tx=0x7fb9cc005b20 comp rx=0 tx=0).stop 2024-02-17T12:11:39.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9dcd0c640 1 -- 172.21.15.94:0/396831513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9d810c640 msgr2=0x7fb9d806b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:39.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9dcd0c640 1 --2- 172.21.15.94:0/396831513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9d810c640 0x7fb9d806b960 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb9c802c2a0 tx=0x7fb9c802ea20 comp rx=0 tx=0).stop 2024-02-17T12:11:39.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9dcd0c640 1 -- 172.21.15.94:0/396831513 shutdown_connections 2024-02-17T12:11:39.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9dcd0c640 1 --2- 172.21.15.94:0/396831513 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb9c00a2440 0x7fb9c00a4900 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9dcd0c640 1 --2- 172.21.15.94:0/396831513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9d810c640 0x7fb9d806b960 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9dcd0c640 1 --2- 172.21.15.94:0/396831513 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb9d8108100 0x7fb9d8072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9dcd0c640 1 --2- 172.21.15.94:0/396831513 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb9d8107760 0x7fb9d8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:39.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.825+0000 7fb9dcd0c640 1 -- 172.21.15.94:0/396831513 >> 172.21.15.94:0/396831513 conn(0x7fb9d8076f50 msgr2=0x7fb9d8077570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:39.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.826+0000 7fb9dcd0c640 1 -- 172.21.15.94:0/396831513 shutdown_connections 2024-02-17T12:11:39.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:11:39.826+0000 7fb9dcd0c640 1 -- 172.21.15.94:0/396831513 wait complete. 2024-02-17T12:11:39.839 INFO:tasks.cephfs.filesystem:scrub status for tag:b121859b-864d-4360-a54a-e5294be2afb3 - {'path': '/', 'tag': 'b121859b-864d-4360-a54a-e5294be2afb3', 'options': 'recursive,force'} 2024-02-17T12:11:40.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 800.0 s, 1468.8 tps, lat 67.804 ms stddev 81.230 2024-02-17T12:11:40.905 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:40 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:11:40.905 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:40 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:11:40.905 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:40 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:11:40.905 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:40 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:11:40.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:40 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:11:40.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:40 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:11:40.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:40 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:11:40.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:40 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:11:40.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:40 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:11:40.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:40 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:11:40.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:40 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:11:40.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:40 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:11:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.111+0000 7f6c81681640 1 -- 172.21.15.94:0/1488996587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c7c107290 msgr2=0x7f6c7c107670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.111+0000 7f6c81681640 1 --2- 172.21.15.94:0/1488996587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c7c107290 0x7f6c7c107670 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009900 tx=0x7f6c6c02bc60 comp rx=0 tx=0).stop 2024-02-17T12:11:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.111+0000 7f6c81681640 1 -- 172.21.15.94:0/1488996587 shutdown_connections 2024-02-17T12:11:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.111+0000 7f6c81681640 1 --2- 172.21.15.94:0/1488996587 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c7c0691a0 0x7f6c7c10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.111+0000 7f6c81681640 1 --2- 172.21.15.94:0/1488996587 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c068800 0x7f6c7c068c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.111+0000 7f6c81681640 1 --2- 172.21.15.94:0/1488996587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c7c107290 0x7f6c7c107670 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.111+0000 7f6c81681640 1 -- 172.21.15.94:0/1488996587 >> 172.21.15.94:0/1488996587 conn(0x7f6c7c076ed0 msgr2=0x7f6c7c0772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.111+0000 7f6c81681640 1 -- 172.21.15.94:0/1488996587 shutdown_connections 2024-02-17T12:11:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.112+0000 7f6c81681640 1 -- 172.21.15.94:0/1488996587 wait complete. 2024-02-17T12:11:41.114 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.112+0000 7f6c81681640 1 Processor -- start 2024-02-17T12:11:41.115 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.112+0000 7f6c81681640 1 -- start start 2024-02-17T12:11:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.112+0000 7f6c81681640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c7c068800 0x7f6c7c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.112+0000 7f6c81681640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0691a0 0x7f6c7c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.112+0000 7f6c81681640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c7c107290 0x7f6c7c1998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.112+0000 7f6c81681640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c199f10 con 0x7f6c7c0691a0 2024-02-17T12:11:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.112+0000 7f6c81681640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c19a080 con 0x7f6c7c107290 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.112+0000 7f6c81681640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c19a240 con 0x7f6c7c068800 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c7a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0691a0 0x7f6c7c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c7affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c7c068800 0x7f6c7c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c7a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0691a0 0x7f6c7c19fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54490/0 (socket says 172.21.15.94:54490) 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c7a7fc640 1 -- 172.21.15.94:0/3389164221 learned_addr learned my addr 172.21.15.94:0/3389164221 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c7b7fe640 1 --2- 172.21.15.94:0/3389164221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c7c107290 0x7f6c7c1998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c7a7fc640 1 -- 172.21.15.94:0/3389164221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c7c068800 msgr2=0x7f6c7c19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c7a7fc640 1 --2- 172.21.15.94:0/3389164221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c7c068800 0x7f6c7c19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c7a7fc640 1 -- 172.21.15.94:0/3389164221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c7c107290 msgr2=0x7f6c7c1998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c7a7fc640 1 --2- 172.21.15.94:0/3389164221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c7c107290 0x7f6c7c1998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c7a7fc640 1 -- 172.21.15.94:0/3389164221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c009560 con 0x7f6c7c0691a0 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c7a7fc640 1 --2- 172.21.15.94:0/3389164221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0691a0 0x7f6c7c19fc50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000b4a0 tx=0x7f6c7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c63fff640 1 -- 172.21.15.94:0/3389164221 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70010610 con 0x7f6c7c0691a0 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c81681640 1 -- 172.21.15.94:0/3389164221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c19a520 con 0x7f6c7c0691a0 2024-02-17T12:11:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c63fff640 1 -- 172.21.15.94:0/3389164221 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c70010c50 con 0x7f6c7c0691a0 2024-02-17T12:11:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c63fff640 1 -- 172.21.15.94:0/3389164221 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70017e20 con 0x7f6c7c0691a0 2024-02-17T12:11:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.113+0000 7f6c81681640 1 -- 172.21.15.94:0/3389164221 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c1a67b0 con 0x7f6c7c0691a0 2024-02-17T12:11:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.116+0000 7f6c63fff640 1 -- 172.21.15.94:0/3389164221 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6c70017480 con 0x7f6c7c0691a0 2024-02-17T12:11:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.116+0000 7f6c84251740 1 Processor -- start 2024-02-17T12:11:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.116+0000 7f6c84251740 1 -- start start 2024-02-17T12:11:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.116+0000 7f6c84251740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 0x55871fcf2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.116+0000 7f6c84251740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcf2d60 0x55871fcf31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.116+0000 7f6c84251740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcfc2e0 0x55871fcfe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.116+0000 7f6c84251740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871fc619f0 con 0x55871fcfc2e0 2024-02-17T12:11:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.116+0000 7f6c84251740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871fc5ef20 con 0x55871fcf3ad0 2024-02-17T12:11:41.124 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.116+0000 7f6c84251740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871fc5a700 con 0x55871fcf2d60 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.116+0000 7f6c63fff640 1 --2- 172.21.15.94:0/3389164221 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6c640a22d0 0x7f6c640a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.116+0000 7f6c7a7fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcfc2e0 0x55871fcfe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.116+0000 7f6c7affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcf2d60 0x55871fcf31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c7affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcf2d60 0x55871fcf31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:34734/0 (socket says 172.21.15.94:34734) 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c7b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 0x55871fcf2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c7affd640 1 -- 172.21.15.94:0/669616133 learned_addr learned my addr 172.21.15.94:0/669616133 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c7b7fe640 1 --2- 172.21.15.94:0/3389164221 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6c640a22d0 0x7f6c640a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c63fff640 1 -- 172.21.15.94:0/3389164221 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6c700cd650 con 0x7f6c7c0691a0 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c7affd640 1 -- 172.21.15.94:0/669616133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 msgr2=0x55871fcf2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c7affd640 1 --2- 172.21.15.94:0/669616133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 0x55871fcf2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c7affd640 1 -- 172.21.15.94:0/669616133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcfc2e0 msgr2=0x55871fcfe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c7affd640 1 --2- 172.21.15.94:0/669616133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcfc2e0 0x55871fcfe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c7affd640 1 -- 172.21.15.94:0/669616133 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55871fbf07b0 con 0x55871fcf2d60 2024-02-17T12:11:41.125 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c7a7fc640 1 --2- 172.21.15.94:0/669616133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcfc2e0 0x55871fcfe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c7affd640 1 --2- 172.21.15.94:0/669616133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcf2d60 0x55871fcf31e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009900 tx=0x7f6c6c004800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0ba0150a4744b6b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c7b7fe640 1 --2- 172.21.15.94:0/3389164221 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6c640a22d0 0x7f6c640a4790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009fd0 tx=0x7f6c74007040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c617fa640 1 -- 172.21.15.94:0/669616133 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0049f0 con 0x55871fcf2d60 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c617fa640 1 -- 172.21.15.94:0/669616133 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c004030 con 0x55871fcf2d60 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.117+0000 7f6c617fa640 1 -- 172.21.15.94:0/669616133 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c005210 con 0x55871fcf2d60 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.118+0000 7f6c84251740 1 -- 172.21.15.94:0/669616133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcf2d60 msgr2=0x55871fcf31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.118+0000 7f6c84251740 1 --2- 172.21.15.94:0/669616133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcf2d60 0x55871fcf31e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009900 tx=0x7f6c6c004800 comp rx=0 tx=0).stop 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.118+0000 7f6c84251740 1 -- 172.21.15.94:0/669616133 shutdown_connections 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.118+0000 7f6c84251740 1 --2- 172.21.15.94:0/669616133 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcfc2e0 0x55871fcfe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.118+0000 7f6c84251740 1 --2- 172.21.15.94:0/669616133 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcf2d60 0x55871fcf31e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.118+0000 7f6c84251740 1 --2- 172.21.15.94:0/669616133 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 0x55871fcf2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.118+0000 7f6c84251740 1 -- 172.21.15.94:0/669616133 >> 172.21.15.94:0/669616133 conn(0x55871fcba550 msgr2=0x55871fceda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.118+0000 7f6c84251740 1 -- 172.21.15.94:0/669616133 shutdown_connections 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.118+0000 7f6c84251740 1 -- 172.21.15.94:0/669616133 wait complete. 2024-02-17T12:11:41.126 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.118+0000 7f6c84251740 1 -- start start 2024-02-17T12:11:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.118+0000 7f6c84251740 1 Processor -- start 2024-02-17T12:11:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c84251740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 0x55871fcf9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c84251740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 0x55871fcf97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c84251740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcfc2e0 0x55871fcf9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c84251740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871fbffbf0 con 0x55871fcf2d60 2024-02-17T12:11:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c84251740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871f92e250 con 0x55871fcf3ad0 2024-02-17T12:11:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c84251740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871fbe0200 con 0x55871fcfc2e0 2024-02-17T12:11:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c84251740 10 client.0 _ll_register_callbacks cb 0x55871fc4ed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c7affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 0x55871fcf9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c7affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 0x55871fcf9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54522/0 (socket says 172.21.15.94:54522) 2024-02-17T12:11:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c7affd640 1 -- 172.21.15.94:0/2505795382 learned_addr learned my addr 172.21.15.94:0/2505795382 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c7b7fe640 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcfc2e0 0x55871fcf9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.127 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c7affd640 1 -- 172.21.15.94:0/2505795382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcfc2e0 msgr2=0x55871fcf9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c7affd640 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcfc2e0 0x55871fcf9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c7affd640 1 -- 172.21.15.94:0/2505795382 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 msgr2=0x55871fcf97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c7a7fc640 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 0x55871fcf97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c7affd640 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 0x55871fcf97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c7affd640 1 -- 172.21.15.94:0/2505795382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c100490 con 0x55871fcf2d60 2024-02-17T12:11:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c7b7fe640 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcfc2e0 0x55871fcf9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:11:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c7affd640 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 0x55871fcf9290 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c02c400 tx=0x7f6c6c02c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ad00485f2a9c2d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c5eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c5eff5640 1 -- 172.21.15.94:0/2505795382 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c004f70 con 0x55871fcf2d60 2024-02-17T12:11:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c5eff5640 1 -- 172.21.15.94:0/2505795382 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6c6c0050d0 con 0x55871fcf2d60 2024-02-17T12:11:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.119+0000 7f6c84251740 1 -- 172.21.15.94:0/2505795382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55871f95a600 con 0x55871fcf2d60 2024-02-17T12:11:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.120+0000 7f6c5eff5640 1 -- 172.21.15.94:0/2505795382 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c6c005290 con 0x55871fcf2d60 2024-02-17T12:11:41.128 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.120+0000 7f6c84251740 10 client.19047 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.120+0000 7f6c84251740 1 -- 172.21.15.94:0/2505795382 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55871fc58d80 con 0x55871fcf2d60 2024-02-17T12:11:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.120+0000 7f6c5eff5640 1 -- 172.21.15.94:0/2505795382 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55871f95a600 con 0x55871fcf2d60 2024-02-17T12:11:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.120+0000 7f6c5eff5640 1 -- 172.21.15.94:0/2505795382 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6c6c03c050 con 0x55871fcf2d60 2024-02-17T12:11:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.120+0000 7f6c84251740 10 client.19047 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.120+0000 7f6c84251740 10 client.19047 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.120+0000 7f6c84251740 20 client.19047 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.120+0000 7f6c84251740 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55871fd0c1f0 0x55871fd0e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.120+0000 7f6c84251740 4 client.19047 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.120+0000 7f6c84251740 1 -- 172.21.15.94:0/2505795382 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55871f9ab1b0 con 0x55871fd0c1f0 2024-02-17T12:11:41.129 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.120+0000 7f6c7a7fc640 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55871fd0c1f0 0x55871fd0e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.122+0000 7f6c5eff5640 10 client.19047 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:41.130 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.122+0000 7f6c7a7fc640 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55871fd0c1f0 0x55871fd0e5e0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccec5034a95d6f5d server_cookie=bc46cf7f28a907ac in_seq=0 out_seq=0 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c5eff5640 1 -- 172.21.15.94:0/2505795382 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55871f9ab1b0 con 0x55871fd0c1f0 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c5eff5640 10 client.19047 handle_command_reply: tid=0 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c84251740 1 client.19047 shutdown 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c84251740 1 -- 172.21.15.94:0/2505795382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 msgr2=0x55871fcf9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c84251740 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 0x55871fcf9290 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c02c400 tx=0x7f6c6c02c810 comp rx=0 tx=0).stop 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c84251740 1 -- 172.21.15.94:0/2505795382 shutdown_connections 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c84251740 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55871fd0c1f0 0x55871fd0e5e0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c84251740 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcfc2e0 0x55871fcf9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c84251740 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 0x55871fcf97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c5eff5640 0 client.19047 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c84251740 1 --2- 172.21.15.94:0/2505795382 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 0x55871fcf9290 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c84251740 1 -- 172.21.15.94:0/2505795382 >> 172.21.15.94:0/2505795382 conn(0x55871fcba550 msgr2=0x55871fcef7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c84251740 1 -- 172.21.15.94:0/2505795382 shutdown_connections 2024-02-17T12:11:41.131 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c84251740 1 -- 172.21.15.94:0/2505795382 wait complete. 2024-02-17T12:11:41.132 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.123+0000 7f6c84251740 20 client.19047 trim_cache size 0 max 16384 2024-02-17T12:11:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.138+0000 7f6c84251740 1 Processor -- start 2024-02-17T12:11:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.138+0000 7f6c84251740 1 -- start start 2024-02-17T12:11:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.138+0000 7f6c84251740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 0x55871fcfb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.138+0000 7f6c84251740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 0x55871fcf97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.138+0000 7f6c84251740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcfc2e0 0x55871fcf9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.138+0000 7f6c84251740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871fd137d0 con 0x55871fcf2d60 2024-02-17T12:11:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.138+0000 7f6c84251740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871fcf2030 con 0x55871fcf3ad0 2024-02-17T12:11:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.138+0000 7f6c84251740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871fcee7c0 con 0x55871fcfc2e0 2024-02-17T12:11:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.138+0000 7f6c7affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 0x55871fcfb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.145 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.138+0000 7f6c7affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 0x55871fcfb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54526/0 (socket says 172.21.15.94:54526) 2024-02-17T12:11:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.138+0000 7f6c7affd640 1 -- 172.21.15.94:0/3995691603 learned_addr learned my addr 172.21.15.94:0/3995691603 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.139+0000 7f6c7b7fe640 1 --2- 172.21.15.94:0/3995691603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcfc2e0 0x55871fcf9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.139+0000 7f6c7a7fc640 1 --2- 172.21.15.94:0/3995691603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 0x55871fcf97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.139+0000 7f6c7affd640 1 -- 172.21.15.94:0/3995691603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcfc2e0 msgr2=0x55871fcf9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.139+0000 7f6c7affd640 1 --2- 172.21.15.94:0/3995691603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcfc2e0 0x55871fcf9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.139+0000 7f6c7affd640 1 -- 172.21.15.94:0/3995691603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 msgr2=0x55871fcf97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.139+0000 7f6c7affd640 1 --2- 172.21.15.94:0/3995691603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 0x55871fcf97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.139+0000 7f6c7affd640 1 -- 172.21.15.94:0/3995691603 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55871fbf57b0 con 0x55871fcf2d60 2024-02-17T12:11:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.139+0000 7f6c7a7fc640 1 --2- 172.21.15.94:0/3995691603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 0x55871fcf97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:11:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.139+0000 7f6c7affd640 1 --2- 172.21.15.94:0/3995691603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 0x55871fcfb9c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c02bca0 tx=0x7f6c6c02c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c74a05b27c5ee43 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.139+0000 7f6c60ff9640 1 -- 172.21.15.94:0/3995691603 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c02c880 con 0x55871fcf2d60 2024-02-17T12:11:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.139+0000 7f6c60ff9640 1 -- 172.21.15.94:0/3995691603 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c03fd80 con 0x55871fcf2d60 2024-02-17T12:11:41.146 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.139+0000 7f6c60ff9640 1 -- 172.21.15.94:0/3995691603 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0038e0 con 0x55871fcf2d60 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 -- 172.21.15.94:0/3995691603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 msgr2=0x55871fcfb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 --2- 172.21.15.94:0/3995691603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 0x55871fcfb9c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c02bca0 tx=0x7f6c6c02c380 comp rx=0 tx=0).stop 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 -- 172.21.15.94:0/3995691603 shutdown_connections 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 --2- 172.21.15.94:0/3995691603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcfc2e0 0x55871fcf9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 --2- 172.21.15.94:0/3995691603 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf3ad0 0x55871fcf97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 --2- 172.21.15.94:0/3995691603 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcf2d60 0x55871fcfb9c0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 -- 172.21.15.94:0/3995691603 >> 172.21.15.94:0/3995691603 conn(0x55871fcba550 msgr2=0x55871fcf45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 -- 172.21.15.94:0/3995691603 shutdown_connections 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 -- 172.21.15.94:0/3995691603 wait complete. 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 -- start start 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 Processor -- start 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf2d60 0x55871fd01090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcf3ad0 0x55871fd015d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.147 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcfc2e0 0x55871fd616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871fd01e00 con 0x55871fcfc2e0 2024-02-17T12:11:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871fd01f70 con 0x55871fcf2d60 2024-02-17T12:11:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55871fd61be0 con 0x55871fcf3ad0 2024-02-17T12:11:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c84251740 10 client.0 _ll_register_callbacks cb 0x7f6c6c02c660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.140+0000 7f6c7b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcfc2e0 0x55871fd616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c7b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcfc2e0 0x55871fd616a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:54540/0 (socket says 172.21.15.94:54540) 2024-02-17T12:11:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c7b7fe640 1 -- 172.21.15.94:0/1594987567 learned_addr learned my addr 172.21.15.94:0/1594987567 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c7affd640 1 --2- 172.21.15.94:0/1594987567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf2d60 0x55871fd01090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c7a7fc640 1 --2- 172.21.15.94:0/1594987567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcf3ad0 0x55871fd015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c7b7fe640 1 -- 172.21.15.94:0/1594987567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcf3ad0 msgr2=0x55871fd015d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c7b7fe640 1 --2- 172.21.15.94:0/1594987567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcf3ad0 0x55871fd015d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.148 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c7b7fe640 1 -- 172.21.15.94:0/1594987567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf2d60 msgr2=0x55871fd01090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c7b7fe640 1 --2- 172.21.15.94:0/1594987567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf2d60 0x55871fd01090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c7b7fe640 1 -- 172.21.15.94:0/1594987567 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55871fc58d80 con 0x55871fcfc2e0 2024-02-17T12:11:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c7b7fe640 1 --2- 172.21.15.94:0/1594987567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcfc2e0 0x55871fd616a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400ab80 tx=0x7f6c74004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3943b8fab53ca798 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c5eff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c5eff5640 1 -- 172.21.15.94:0/1594987567 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74027680 con 0x55871fcfc2e0 2024-02-17T12:11:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c5eff5640 1 -- 172.21.15.94:0/1594987567 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6c740277e0 con 0x55871fcfc2e0 2024-02-17T12:11:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c84251740 1 -- 172.21.15.94:0/1594987567 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c6c004030 con 0x55871fcfc2e0 2024-02-17T12:11:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c5eff5640 1 -- 172.21.15.94:0/1594987567 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c740279a0 con 0x55871fcfc2e0 2024-02-17T12:11:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c84251740 10 client.19053 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c84251740 1 -- 172.21.15.94:0/1594987567 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55871fc1b540 con 0x55871fcfc2e0 2024-02-17T12:11:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.141+0000 7f6c5eff5640 1 -- 172.21.15.94:0/1594987567 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6c6c004030 con 0x55871fcfc2e0 2024-02-17T12:11:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.142+0000 7f6c5eff5640 1 -- 172.21.15.94:0/1594987567 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f6c74008070 con 0x55871fcfc2e0 2024-02-17T12:11:41.149 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.142+0000 7f6c84251740 10 client.19053 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.142+0000 7f6c84251740 10 client.19053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.142+0000 7f6c84251740 20 client.19053 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.142+0000 7f6c84251740 1 --2- 172.21.15.94:0/1594987567 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55871fd634a0 0x55871fd658c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.142+0000 7f6c84251740 4 client.19053 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:11:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.142+0000 7f6c84251740 1 -- 172.21.15.94:0/1594987567 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55871f997550 con 0x55871fd634a0 2024-02-17T12:11:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.142+0000 7f6c7affd640 1 --2- 172.21.15.94:0/1594987567 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55871fd634a0 0x55871fd658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.144+0000 7f6c7affd640 1 --2- 172.21.15.94:0/1594987567 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55871fd634a0 0x55871fd658c0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42365644f1dc0cd server_cookie=60b607931a8014f0 in_seq=0 out_seq=0 2024-02-17T12:11:41.150 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.144+0000 7f6c5eff5640 10 client.19053 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c5eff5640 1 -- 172.21.15.94:0/1594987567 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55871f997550 con 0x55871fd634a0 2024-02-17T12:11:41.164 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c5eff5640 10 client.19053 handle_command_reply: tid=0 2024-02-17T12:11:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c84251740 1 client.19053 shutdown 2024-02-17T12:11:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c84251740 1 -- 172.21.15.94:0/1594987567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcfc2e0 msgr2=0x55871fd616a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c84251740 1 --2- 172.21.15.94:0/1594987567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcfc2e0 0x55871fd616a0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400ab80 tx=0x7f6c74004ad0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c84251740 1 -- 172.21.15.94:0/1594987567 shutdown_connections 2024-02-17T12:11:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c84251740 1 --2- 172.21.15.94:0/1594987567 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55871fd634a0 0x55871fd658c0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c84251740 1 --2- 172.21.15.94:0/1594987567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55871fcfc2e0 0x55871fd616a0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c5eff5640 0 client.19053 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c84251740 1 --2- 172.21.15.94:0/1594987567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871fcf3ad0 0x55871fd015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c84251740 1 --2- 172.21.15.94:0/1594987567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55871fcf2d60 0x55871fd01090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c84251740 1 -- 172.21.15.94:0/1594987567 >> 172.21.15.94:0/1594987567 conn(0x55871fcba550 msgr2=0x55871fcf7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c84251740 1 -- 172.21.15.94:0/1594987567 shutdown_connections 2024-02-17T12:11:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c84251740 1 -- 172.21.15.94:0/1594987567 wait complete. 2024-02-17T12:11:41.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.162+0000 7f6c84251740 20 client.19053 trim_cache size 0 max 16384 2024-02-17T12:11:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:11:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:11:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:11:41.166 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:11:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.163+0000 7f6c81681640 1 -- 172.21.15.94:0/3389164221 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6c640a22d0 msgr2=0x7f6c640a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.163+0000 7f6c81681640 1 --2- 172.21.15.94:0/3389164221 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6c640a22d0 0x7f6c640a4790 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009fd0 tx=0x7f6c74007040 comp rx=0 tx=0).stop 2024-02-17T12:11:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.163+0000 7f6c81681640 1 -- 172.21.15.94:0/3389164221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0691a0 msgr2=0x7f6c7c19fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.163+0000 7f6c81681640 1 --2- 172.21.15.94:0/3389164221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0691a0 0x7f6c7c19fc50 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000b4a0 tx=0x7f6c7000b960 comp rx=0 tx=0).stop 2024-02-17T12:11:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.163+0000 7f6c81681640 1 -- 172.21.15.94:0/3389164221 shutdown_connections 2024-02-17T12:11:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.163+0000 7f6c81681640 1 --2- 172.21.15.94:0/3389164221 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6c640a22d0 0x7f6c640a4790 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.167 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.163+0000 7f6c81681640 1 --2- 172.21.15.94:0/3389164221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6c7c107290 0x7f6c7c1998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.163+0000 7f6c81681640 1 --2- 172.21.15.94:0/3389164221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6c7c0691a0 0x7f6c7c19fc50 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.163+0000 7f6c81681640 1 --2- 172.21.15.94:0/3389164221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6c7c068800 0x7f6c7c19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.163+0000 7f6c81681640 1 -- 172.21.15.94:0/3389164221 >> 172.21.15.94:0/3389164221 conn(0x7f6c7c076ed0 msgr2=0x7f6c7c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.163+0000 7f6c81681640 1 -- 172.21.15.94:0/3389164221 shutdown_connections 2024-02-17T12:11:41.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:41.163+0000 7f6c81681640 1 -- 172.21.15.94:0/3389164221 wait complete. 2024-02-17T12:11:41.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:41 smithi189 ceph-mon[36588]: pgmap v424: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.2 MiB/s wr, 679 op/s 2024-02-17T12:11:41.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:41 smithi125 ceph-mon[36819]: pgmap v424: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.2 MiB/s wr, 679 op/s 2024-02-17T12:11:41.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:41 smithi094 ceph-mon[32499]: pgmap v424: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 8.2 MiB/s wr, 679 op/s 2024-02-17T12:11:42.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:42 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:11:42.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:42 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:11:42.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:42 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:11:43.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:43 smithi189 ceph-mon[36588]: pgmap v425: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 596 op/s 2024-02-17T12:11:43.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:43 smithi125 ceph-mon[36819]: pgmap v425: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 596 op/s 2024-02-17T12:11:43.957 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:43 smithi094 ceph-mon[32499]: pgmap v425: 97 pgs: 97 active+clean; 5.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.3 MiB/s wr, 596 op/s 2024-02-17T12:11:45.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 805.0 s, 1319.8 tps, lat 75.813 ms stddev 82.923 2024-02-17T12:11:45.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:45 smithi189 ceph-mon[36588]: pgmap v426: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.7 MiB/s wr, 855 op/s 2024-02-17T12:11:45.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:45 smithi125 ceph-mon[36819]: pgmap v426: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.7 MiB/s wr, 855 op/s 2024-02-17T12:11:45.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:45 smithi094 ceph-mon[32499]: pgmap v426: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 8.7 MiB/s wr, 855 op/s 2024-02-17T12:11:47.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:47 smithi189 ceph-mon[36588]: pgmap v427: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.4 MiB/s wr, 677 op/s 2024-02-17T12:11:47.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:47 smithi125 ceph-mon[36819]: pgmap v427: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.4 MiB/s wr, 677 op/s 2024-02-17T12:11:47.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:47 smithi094 ceph-mon[32499]: pgmap v427: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 5.4 MiB/s wr, 677 op/s 2024-02-17T12:11:49.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:49 smithi189 ceph-mon[36588]: pgmap v428: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.3 MiB/s wr, 774 op/s 2024-02-17T12:11:49.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:49 smithi125 ceph-mon[36819]: pgmap v428: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.3 MiB/s wr, 774 op/s 2024-02-17T12:11:49.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:49 smithi094 ceph-mon[32499]: pgmap v428: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 6.3 MiB/s wr, 774 op/s 2024-02-17T12:11:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 810.0 s, 1240.2 tps, lat 80.911 ms stddev 98.536 2024-02-17T12:11:51.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:51 smithi189 ceph-mon[36588]: pgmap v429: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 5.1 MiB/s wr, 642 op/s 2024-02-17T12:11:51.439 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:51 smithi125 ceph-mon[36819]: pgmap v429: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 5.1 MiB/s wr, 642 op/s 2024-02-17T12:11:51.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:51 smithi094 ceph-mon[32499]: pgmap v429: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 5.1 MiB/s wr, 642 op/s 2024-02-17T12:11:53.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:53 smithi189 ceph-mon[36588]: pgmap v430: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.3 MiB/s wr, 556 op/s 2024-02-17T12:11:53.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:53 smithi125 ceph-mon[36819]: pgmap v430: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.3 MiB/s wr, 556 op/s 2024-02-17T12:11:53.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:53 smithi094 ceph-mon[32499]: pgmap v430: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 4.3 MiB/s wr, 556 op/s 2024-02-17T12:11:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.295+0000 7f3590e81640 1 -- 172.21.15.94:0/3820415838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f358c103b90 msgr2=0x7f358c103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.295+0000 7f3590e81640 1 --2- 172.21.15.94:0/3820415838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f358c103b90 0x7f358c103ff0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3584009cd0 tx=0x7f358402c190 comp rx=0 tx=0).stop 2024-02-17T12:11:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.295+0000 7f3590e81640 1 -- 172.21.15.94:0/3820415838 shutdown_connections 2024-02-17T12:11:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.295+0000 7f3590e81640 1 --2- 172.21.15.94:0/3820415838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f358c103b90 0x7f358c103ff0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.295+0000 7f3590e81640 1 --2- 172.21.15.94:0/3820415838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c068d40 0x7f358c069180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.295+0000 7f3590e81640 1 --2- 172.21.15.94:0/3820415838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f358c106140 0x7f358c068800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.295+0000 7f3590e81640 1 -- 172.21.15.94:0/3820415838 >> 172.21.15.94:0/3820415838 conn(0x7f358c076ed0 msgr2=0x7f358c0772e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.296+0000 7f3590e81640 1 -- 172.21.15.94:0/3820415838 shutdown_connections 2024-02-17T12:11:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.296+0000 7f3590e81640 1 -- 172.21.15.94:0/3820415838 wait complete. 2024-02-17T12:11:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.297+0000 7f3590e81640 1 Processor -- start 2024-02-17T12:11:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.297+0000 7f3590e81640 1 -- start start 2024-02-17T12:11:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.297+0000 7f3590e81640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f358c068d40 0x7f358c0719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.297+0000 7f3590e81640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f358c103b90 0x7f358c071f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.297+0000 7f3590e81640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c106140 0x7f358c10a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.298+0000 7f3590e81640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c10afa0 con 0x7f358c106140 2024-02-17T12:11:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.298+0000 7f3590e81640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c10b110 con 0x7f358c068d40 2024-02-17T12:11:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.298+0000 7f3590e81640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c10b2d0 con 0x7f358c103b90 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.298+0000 7f358ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c106140 0x7f358c10a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.298+0000 7f358ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c106140 0x7f358c10a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56828/0 (socket says 172.21.15.94:56828) 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.298+0000 7f358ad9d640 1 -- 172.21.15.94:0/217567391 learned_addr learned my addr 172.21.15.94:0/217567391 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.298+0000 7f358a59c640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f358c068d40 0x7f358c0719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.298+0000 7f358ad9d640 1 -- 172.21.15.94:0/217567391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f358c103b90 msgr2=0x7f358c071f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.298+0000 7f358ad9d640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f358c103b90 0x7f358c071f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.298+0000 7f358ad9d640 1 -- 172.21.15.94:0/217567391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f358c068d40 msgr2=0x7f358c0719c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.298+0000 7f358ad9d640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f358c068d40 0x7f358c0719c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.298+0000 7f358ad9d640 1 -- 172.21.15.94:0/217567391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3584009900 con 0x7f358c106140 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.298+0000 7f358a59c640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f358c068d40 0x7f358c0719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.299+0000 7f358ad9d640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c106140 0x7f358c10a9a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3584037040 tx=0x7f3584034650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.299+0000 7f357b7fe640 1 -- 172.21.15.94:0/217567391 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584003b70 con 0x7f358c106140 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.299+0000 7f3590e81640 1 -- 172.21.15.94:0/217567391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c10b550 con 0x7f358c106140 2024-02-17T12:11:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.299+0000 7f3590e81640 1 -- 172.21.15.94:0/217567391 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c111c50 con 0x7f358c106140 2024-02-17T12:11:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.299+0000 7f357b7fe640 1 -- 172.21.15.94:0/217567391 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3584003cd0 con 0x7f358c106140 2024-02-17T12:11:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.300+0000 7f357b7fe640 1 -- 172.21.15.94:0/217567391 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584003b70 con 0x7f358c106140 2024-02-17T12:11:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.303+0000 7f357b7fe640 1 -- 172.21.15.94:0/217567391 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f358403b050 con 0x7f358c106140 2024-02-17T12:11:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.304+0000 7f3593a4d740 1 Processor -- start 2024-02-17T12:11:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.304+0000 7f3593a4d740 1 -- start start 2024-02-17T12:11:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.304+0000 7f3593a4d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 0x55f5c9926820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.304+0000 7f3593a4d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c99271e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.304+0000 7f3593a4d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c99302e0 0x55f5c99326d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.304+0000 7f3593a4d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c98959f0 con 0x55f5c9926d60 2024-02-17T12:11:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.304+0000 7f3593a4d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c9892f20 con 0x55f5c99302e0 2024-02-17T12:11:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.304+0000 7f3593a4d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c988e700 con 0x55f5c9927ad0 2024-02-17T12:11:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f357b7fe640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35740a2380 0x7f35740a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f3589d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c99302e0 0x55f5c99326d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f357b7fe640 1 -- 172.21.15.94:0/217567391 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f35840f00e0 con 0x7f358c106140 2024-02-17T12:11:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f3589d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c99302e0 0x55f5c99326d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42798/0 (socket says 172.21.15.94:42798) 2024-02-17T12:11:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f3589d9b640 1 -- 172.21.15.94:0/1775221872 learned_addr learned my addr 172.21.15.94:0/1775221872 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f358a59c640 1 --2- 172.21.15.94:0/1775221872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c99271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f3589d9b640 1 --2- 172.21.15.94:0/1775221872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 0x55f5c9926820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f358ad9d640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35740a2380 0x7f35740a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f358a59c640 1 -- 172.21.15.94:0/1775221872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 msgr2=0x55f5c9926820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f358a59c640 1 --2- 172.21.15.94:0/1775221872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 0x55f5c9926820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f358a59c640 1 -- 172.21.15.94:0/1775221872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c99302e0 msgr2=0x55f5c99326d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f358a59c640 1 --2- 172.21.15.94:0/1775221872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c99302e0 0x55f5c99326d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f358a59c640 1 -- 172.21.15.94:0/1775221872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5c98247b0 con 0x55f5c9926d60 2024-02-17T12:11:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.305+0000 7f3589d9b640 1 --2- 172.21.15.94:0/1775221872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c99302e0 0x55f5c99326d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.306+0000 7f358a59c640 1 --2- 172.21.15.94:0/1775221872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c99271e0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f358000b7d0 tx=0x7f358000bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885a080ffc28dc08 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.306+0000 7f358ad9d640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35740a2380 0x7f35740a4840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f35840b9570 tx=0x7f3584034a10 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.306+0000 7f3578ff9640 1 -- 172.21.15.94:0/1775221872 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358000d950 con 0x55f5c9926d60 2024-02-17T12:11:54.316 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.306+0000 7f3578ff9640 1 -- 172.21.15.94:0/1775221872 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f358000b440 con 0x55f5c9926d60 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.306+0000 7f3578ff9640 1 -- 172.21.15.94:0/1775221872 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580012640 con 0x55f5c9926d60 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.307+0000 7f3593a4d740 1 -- 172.21.15.94:0/1775221872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 msgr2=0x55f5c99271e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.307+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1775221872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c99271e0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f358000b7d0 tx=0x7f358000bc90 comp rx=0 tx=0).stop 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.307+0000 7f3593a4d740 1 -- 172.21.15.94:0/1775221872 shutdown_connections 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.307+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1775221872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c99302e0 0x55f5c99326d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.307+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1775221872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c99271e0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.307+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1775221872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 0x55f5c9926820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.307+0000 7f3593a4d740 1 -- 172.21.15.94:0/1775221872 >> 172.21.15.94:0/1775221872 conn(0x55f5c98ee550 msgr2=0x55f5c9921a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.307+0000 7f3593a4d740 1 -- 172.21.15.94:0/1775221872 shutdown_connections 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.307+0000 7f3593a4d740 1 -- 172.21.15.94:0/1775221872 wait complete. 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.307+0000 7f3593a4d740 1 -- start start 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.308+0000 7f3593a4d740 1 Processor -- start 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.308+0000 7f3593a4d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c992d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.317 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.308+0000 7f3593a4d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 0x55f5c992d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.308+0000 7f3593a4d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c99302e0 0x55f5c992dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.308+0000 7f3593a4d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c9833bf0 con 0x55f5c9926d60 2024-02-17T12:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.308+0000 7f3593a4d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c9562250 con 0x55f5c99302e0 2024-02-17T12:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.308+0000 7f3593a4d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c9814200 con 0x55f5c9927ad0 2024-02-17T12:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.308+0000 7f3593a4d740 10 client.0 _ll_register_callbacks cb 0x55f5c9882d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.308+0000 7f3589d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c992d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.308+0000 7f3589d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c992d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56852/0 (socket says 172.21.15.94:56852) 2024-02-17T12:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.308+0000 7f358a59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 0x55f5c992d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.308+0000 7f3589d9b640 1 -- 172.21.15.94:0/1631619715 learned_addr learned my addr 172.21.15.94:0/1631619715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.309+0000 7f3589d9b640 1 --2- 172.21.15.94:0/1631619715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c99302e0 0x55f5c992dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.309+0000 7f3589d9b640 1 -- 172.21.15.94:0/1631619715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 msgr2=0x55f5c992d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.309+0000 7f3589d9b640 1 --2- 172.21.15.94:0/1631619715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 0x55f5c992d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.318 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.309+0000 7f3589d9b640 1 -- 172.21.15.94:0/1631619715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c99302e0 msgr2=0x55f5c992dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.309+0000 7f3589d9b640 1 --2- 172.21.15.94:0/1631619715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c99302e0 0x55f5c992dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.309+0000 7f3589d9b640 1 -- 172.21.15.94:0/1631619715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358000e040 con 0x55f5c9926d60 2024-02-17T12:11:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.309+0000 7f358a59c640 1 --2- 172.21.15.94:0/1631619715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 0x55f5c992d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:11:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.309+0000 7f3571ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:11:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.309+0000 7f3589d9b640 1 --2- 172.21.15.94:0/1631619715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c992d270 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f357c007740 tx=0x7f357c000f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3b39ef0bc6bb95 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.309+0000 7f3571ffb640 1 -- 172.21.15.94:0/1631619715 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c019d00 con 0x55f5c9926d60 2024-02-17T12:11:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.309+0000 7f3571ffb640 1 -- 172.21.15.94:0/1631619715 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f357c019e60 con 0x55f5c9926d60 2024-02-17T12:11:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.310+0000 7f3593a4d740 1 -- 172.21.15.94:0/1631619715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5c95df1b0 con 0x55f5c9926d60 2024-02-17T12:11:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.310+0000 7f3571ffb640 1 -- 172.21.15.94:0/1631619715 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f357c026450 con 0x55f5c9926d60 2024-02-17T12:11:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.310+0000 7f3593a4d740 10 client.19062 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.310+0000 7f3593a4d740 1 -- 172.21.15.94:0/1631619715 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f5c988cd80 con 0x55f5c9926d60 2024-02-17T12:11:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.310+0000 7f3571ffb640 1 -- 172.21.15.94:0/1631619715 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f5c95df1b0 con 0x55f5c9926d60 2024-02-17T12:11:54.319 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.310+0000 7f3571ffb640 1 -- 172.21.15.94:0/1631619715 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f357c01a3e0 con 0x55f5c9926d60 2024-02-17T12:11:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.310+0000 7f3593a4d740 10 client.19062 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.310+0000 7f3593a4d740 10 client.19062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.310+0000 7f3593a4d740 20 client.19062 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.311+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1631619715 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5c99406f0 0x55f5c9942b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.311+0000 7f3593a4d740 4 client.19062 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:11:54.320 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.311+0000 7f3593a4d740 1 -- 172.21.15.94:0/1631619715 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5c95cb550 con 0x55f5c99406f0 2024-02-17T12:11:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.312+0000 7f358a59c640 1 --2- 172.21.15.94:0/1631619715 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5c99406f0 0x55f5c9942b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.312+0000 7f3571ffb640 10 client.19062 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.312+0000 7f358a59c640 1 --2- 172.21.15.94:0/1631619715 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5c99406f0 0x55f5c9942b10 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a186e9423a5ee5f4 server_cookie=c56ac40eb335c19 in_seq=0 out_seq=0 2024-02-17T12:11:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.313+0000 7f3571ffb640 1 -- 172.21.15.94:0/1631619715 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f5c95cb550 con 0x55f5c99406f0 2024-02-17T12:11:54.321 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.313+0000 7f3571ffb640 10 client.19062 handle_command_reply: tid=0 2024-02-17T12:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.313+0000 7f3593a4d740 1 client.19062 shutdown 2024-02-17T12:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.313+0000 7f3593a4d740 1 -- 172.21.15.94:0/1631619715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 msgr2=0x55f5c992d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.313+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1631619715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c992d270 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f357c007740 tx=0x7f357c000f60 comp rx=0 tx=0).stop 2024-02-17T12:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.313+0000 7f3593a4d740 1 -- 172.21.15.94:0/1631619715 shutdown_connections 2024-02-17T12:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.313+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1631619715 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5c99406f0 0x55f5c9942b10 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.313+0000 7f3571ffb640 0 client.19062 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.313+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1631619715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c99302e0 0x55f5c992dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.313+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1631619715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 0x55f5c992d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.313+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1631619715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c992d270 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.313+0000 7f3593a4d740 1 -- 172.21.15.94:0/1631619715 >> 172.21.15.94:0/1631619715 conn(0x55f5c98ee550 msgr2=0x55f5c99237a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.313+0000 7f3593a4d740 1 -- 172.21.15.94:0/1631619715 shutdown_connections 2024-02-17T12:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.314+0000 7f3593a4d740 1 -- 172.21.15.94:0/1631619715 wait complete. 2024-02-17T12:11:54.322 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.314+0000 7f3593a4d740 20 client.19062 trim_cache size 0 max 16384 2024-02-17T12:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.330+0000 7f3593a4d740 1 Processor -- start 2024-02-17T12:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.330+0000 7f3593a4d740 1 -- start start 2024-02-17T12:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.330+0000 7f3593a4d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c9926d60 0x55f5c992f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.330+0000 7f3593a4d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 0x55f5c992db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.330+0000 7f3593a4d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c99302e0 0x55f5c99371e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.330+0000 7f3593a4d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c9926030 con 0x55f5c99302e0 2024-02-17T12:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.330+0000 7f3593a4d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c9928e50 con 0x55f5c9926d60 2024-02-17T12:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.330+0000 7f3593a4d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c9921c10 con 0x55f5c9927ad0 2024-02-17T12:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.330+0000 7f3589d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c99302e0 0x55f5c99371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.330+0000 7f3589d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c99302e0 0x55f5c99371e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56864/0 (socket says 172.21.15.94:56864) 2024-02-17T12:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.330+0000 7f3589d9b640 1 -- 172.21.15.94:0/1448084048 learned_addr learned my addr 172.21.15.94:0/1448084048 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.330+0000 7f3589d9b640 1 --2- 172.21.15.94:0/1448084048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c9926d60 0x55f5c992f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.330+0000 7f358a59c640 1 --2- 172.21.15.94:0/1448084048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 0x55f5c992db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.331+0000 7f3589d9b640 1 -- 172.21.15.94:0/1448084048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 msgr2=0x55f5c992db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.331+0000 7f3589d9b640 1 --2- 172.21.15.94:0/1448084048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 0x55f5c992db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.331+0000 7f3589d9b640 1 -- 172.21.15.94:0/1448084048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c99302e0 msgr2=0x55f5c99371e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.331+0000 7f3589d9b640 1 --2- 172.21.15.94:0/1448084048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c99302e0 0x55f5c99371e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.331+0000 7f3589d9b640 1 -- 172.21.15.94:0/1448084048 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5c98297b0 con 0x55f5c9926d60 2024-02-17T12:11:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.331+0000 7f3589d9b640 1 --2- 172.21.15.94:0/1448084048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c9926d60 0x55f5c992f9a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f357c009b80 tx=0x7f357c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8234b4109b971fb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.332+0000 7f3573fff640 1 -- 172.21.15.94:0/1448084048 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c001280 con 0x55f5c9926d60 2024-02-17T12:11:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.332+0000 7f3573fff640 1 -- 172.21.15.94:0/1448084048 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f357c01ad70 con 0x55f5c9926d60 2024-02-17T12:11:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.332+0000 7f3573fff640 1 -- 172.21.15.94:0/1448084048 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c008be0 con 0x55f5c9926d60 2024-02-17T12:11:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.332+0000 7f3593a4d740 1 -- 172.21.15.94:0/1448084048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c9926d60 msgr2=0x55f5c992f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.333+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1448084048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c9926d60 0x55f5c992f9a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f357c009b80 tx=0x7f357c000bc0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.333+0000 7f3593a4d740 1 -- 172.21.15.94:0/1448084048 shutdown_connections 2024-02-17T12:11:54.339 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.333+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1448084048 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c99302e0 0x55f5c99371e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.333+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1448084048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c9927ad0 0x55f5c992db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.333+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1448084048 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c9926d60 0x55f5c992f9a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.333+0000 7f3593a4d740 1 -- 172.21.15.94:0/1448084048 >> 172.21.15.94:0/1448084048 conn(0x55f5c98ee550 msgr2=0x55f5c99474b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.333+0000 7f3593a4d740 1 -- 172.21.15.94:0/1448084048 shutdown_connections 2024-02-17T12:11:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.333+0000 7f3593a4d740 1 -- 172.21.15.94:0/1448084048 wait complete. 2024-02-17T12:11:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.333+0000 7f3593a4d740 1 -- start start 2024-02-17T12:11:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.333+0000 7f3593a4d740 1 Processor -- start 2024-02-17T12:11:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f3593a4d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c9937270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f3593a4d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c9927ad0 0x55f5c99337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f3593a4d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c99302e0 0x55f5c9933d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f3589d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c9937270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.340 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f3589d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c9937270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56876/0 (socket says 172.21.15.94:56876) 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f3589d9b640 1 -- 172.21.15.94:0/1000552743 learned_addr learned my addr 172.21.15.94:0/1000552743 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f3593a4d740 1 -- 172.21.15.94:0/1000552743 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c9937b20 con 0x55f5c9926d60 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f3593a4d740 1 -- 172.21.15.94:0/1000552743 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c9934260 con 0x55f5c9927ad0 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f3593a4d740 1 -- 172.21.15.94:0/1000552743 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5c9934420 con 0x55f5c99302e0 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f358a59c640 1 --2- 172.21.15.94:0/1000552743 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c9927ad0 0x55f5c99337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f3593a4d740 10 client.0 _ll_register_callbacks cb 0x7f358000b220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f3589d9b640 1 --2- 172.21.15.94:0/1000552743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c99302e0 0x55f5c9933d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f358a59c640 1 -- 172.21.15.94:0/1000552743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c99302e0 msgr2=0x55f5c9933d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f358a59c640 1 --2- 172.21.15.94:0/1000552743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c99302e0 0x55f5c9933d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f358a59c640 1 -- 172.21.15.94:0/1000552743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 msgr2=0x55f5c9937270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f358a59c640 1 --2- 172.21.15.94:0/1000552743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c9937270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.334+0000 7f358a59c640 1 -- 172.21.15.94:0/1000552743 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5c988cd80 con 0x55f5c9927ad0 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f358a59c640 1 --2- 172.21.15.94:0/1000552743 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c9927ad0 0x55f5c99337e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f358000b720 tx=0x7f3580013c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=170e041fd629ea8d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:11:54.341 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3571ffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:11:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3571ffb640 1 -- 172.21.15.94:0/1000552743 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580024050 con 0x55f5c9927ad0 2024-02-17T12:11:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3571ffb640 1 -- 172.21.15.94:0/1000552743 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3580027b80 con 0x55f5c9927ad0 2024-02-17T12:11:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3593a4d740 1 -- 172.21.15.94:0/1000552743 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5c958e600 con 0x55f5c9927ad0 2024-02-17T12:11:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3571ffb640 1 -- 172.21.15.94:0/1000552743 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f358000bee0 con 0x55f5c9927ad0 2024-02-17T12:11:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3593a4d740 10 client.28547 fetch_fsmap learned FSMap version 27 2024-02-17T12:11:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3593a4d740 1 -- 172.21.15.94:0/1000552743 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f5c984f540 con 0x55f5c9927ad0 2024-02-17T12:11:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3571ffb640 1 -- 172.21.15.94:0/1000552743 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55f5c958e600 con 0x55f5c9927ad0 2024-02-17T12:11:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3571ffb640 1 -- 172.21.15.94:0/1000552743 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3580036070 con 0x55f5c9927ad0 2024-02-17T12:11:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3593a4d740 10 client.28547 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:11:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3593a4d740 10 client.28547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:11:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3593a4d740 20 client.28547 populate_metadata read hostname 'smithi094' 2024-02-17T12:11:54.342 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1000552743 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5c9997f10 0x55f5c999a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:11:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3593a4d740 4 client.28547 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:11:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.335+0000 7f3593a4d740 1 -- 172.21.15.94:0/1000552743 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f5c953a520 con 0x55f5c9997f10 2024-02-17T12:11:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.336+0000 7f3589d9b640 1 --2- 172.21.15.94:0/1000552743 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5c9997f10 0x55f5c999a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:11:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.336+0000 7f3589d9b640 1 --2- 172.21.15.94:0/1000552743 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5c9997f10 0x55f5c999a330 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9652e6b060056642 server_cookie=6d593b4745222fee in_seq=0 out_seq=0 2024-02-17T12:11:54.343 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.336+0000 7f3571ffb640 10 client.28547 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3571ffb640 1 -- 172.21.15.94:0/1000552743 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f5c953a520 con 0x55f5c9997f10 2024-02-17T12:11:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3571ffb640 10 client.28547 handle_command_reply: tid=0 2024-02-17T12:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3593a4d740 1 client.28547 shutdown 2024-02-17T12:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3593a4d740 1 -- 172.21.15.94:0/1000552743 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c9927ad0 msgr2=0x55f5c99337e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1000552743 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c9927ad0 0x55f5c99337e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f358000b720 tx=0x7f3580013c40 comp rx=0 tx=0).stop 2024-02-17T12:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3593a4d740 1 -- 172.21.15.94:0/1000552743 shutdown_connections 2024-02-17T12:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1000552743 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55f5c9997f10 0x55f5c999a330 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1000552743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f5c99302e0 0x55f5c9933d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1000552743 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55f5c9927ad0 0x55f5c99337e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3593a4d740 1 --2- 172.21.15.94:0/1000552743 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55f5c9926d60 0x55f5c9937270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3571ffb640 0 client.28547 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3593a4d740 1 -- 172.21.15.94:0/1000552743 >> 172.21.15.94:0/1000552743 conn(0x55f5c98ee550 msgr2=0x55f5c992ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3593a4d740 1 -- 172.21.15.94:0/1000552743 shutdown_connections 2024-02-17T12:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3593a4d740 1 -- 172.21.15.94:0/1000552743 wait complete. 2024-02-17T12:11:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.362+0000 7f3593a4d740 20 client.28547 trim_cache size 0 max 16384 2024-02-17T12:11:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:11:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:11:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:11:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:11:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.363+0000 7f3590e81640 1 -- 172.21.15.94:0/217567391 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35740a2380 msgr2=0x7f35740a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.363+0000 7f3590e81640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35740a2380 0x7f35740a4840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f35840b9570 tx=0x7f3584034a10 comp rx=0 tx=0).stop 2024-02-17T12:11:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.363+0000 7f3590e81640 1 -- 172.21.15.94:0/217567391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c106140 msgr2=0x7f358c10a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:11:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.363+0000 7f3590e81640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c106140 0x7f358c10a9a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3584037040 tx=0x7f3584034650 comp rx=0 tx=0).stop 2024-02-17T12:11:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.363+0000 7f3590e81640 1 -- 172.21.15.94:0/217567391 shutdown_connections 2024-02-17T12:11:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.363+0000 7f3590e81640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35740a2380 0x7f35740a4840 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.363+0000 7f3590e81640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f358c106140 0x7f358c10a9a0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.363+0000 7f3590e81640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f358c103b90 0x7f358c071f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.363+0000 7f3590e81640 1 --2- 172.21.15.94:0/217567391 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f358c068d40 0x7f358c0719c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:11:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.363+0000 7f3590e81640 1 -- 172.21.15.94:0/217567391 >> 172.21.15.94:0/217567391 conn(0x7f358c076ed0 msgr2=0x7f358c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:11:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.364+0000 7f3590e81640 1 -- 172.21.15.94:0/217567391 shutdown_connections 2024-02-17T12:11:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:11:54.364+0000 7f3590e81640 1 -- 172.21.15.94:0/217567391 wait complete. 2024-02-17T12:11:55.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 815.0 s, 1603.6 tps, lat 62.858 ms stddev 74.559 2024-02-17T12:11:55.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:55 smithi189 ceph-mon[36588]: pgmap v431: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 6.2 MiB/s wr, 768 op/s 2024-02-17T12:11:55.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:55 smithi125 ceph-mon[36819]: pgmap v431: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 6.2 MiB/s wr, 768 op/s 2024-02-17T12:11:55.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:55 smithi094 ceph-mon[32499]: pgmap v431: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 6.2 MiB/s wr, 768 op/s 2024-02-17T12:11:57.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:57 smithi189 ceph-mon[36588]: pgmap v432: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 MiB/s wr, 539 op/s 2024-02-17T12:11:57.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:57 smithi125 ceph-mon[36819]: pgmap v432: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 MiB/s wr, 539 op/s 2024-02-17T12:11:57.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:57 smithi094 ceph-mon[32499]: pgmap v432: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.1 MiB/s wr, 539 op/s 2024-02-17T12:11:59.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:11:59 smithi189 ceph-mon[36588]: pgmap v433: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 658 op/s 2024-02-17T12:11:59.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:11:59 smithi125 ceph-mon[36819]: pgmap v433: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 658 op/s 2024-02-17T12:11:59.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:11:59 smithi094 ceph-mon[32499]: pgmap v433: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 MiB/s wr, 658 op/s 2024-02-17T12:12:00.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 820.0 s, 1846.8 tps, lat 53.799 ms stddev 57.164 2024-02-17T12:12:01.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:01 smithi189 ceph-mon[36588]: pgmap v434: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.5 MiB/s wr, 603 op/s 2024-02-17T12:12:01.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:01 smithi125 ceph-mon[36819]: pgmap v434: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.5 MiB/s wr, 603 op/s 2024-02-17T12:12:01.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:01 smithi094 ceph-mon[32499]: pgmap v434: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.5 MiB/s wr, 603 op/s 2024-02-17T12:12:03.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:03 smithi189 ceph-mon[36588]: pgmap v435: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 MiB/s wr, 504 op/s 2024-02-17T12:12:03.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:03 smithi125 ceph-mon[36819]: pgmap v435: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 MiB/s wr, 504 op/s 2024-02-17T12:12:03.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:03 smithi094 ceph-mon[32499]: pgmap v435: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.6 MiB/s wr, 504 op/s 2024-02-17T12:12:05.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 825.0 s, 231.2 tps, lat 386.744 ms stddev 250.262 2024-02-17T12:12:05.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:05 smithi189 ceph-mon[36588]: pgmap v436: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.7 MiB/s wr, 660 op/s 2024-02-17T12:12:05.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:05 smithi125 ceph-mon[36819]: pgmap v436: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.7 MiB/s wr, 660 op/s 2024-02-17T12:12:05.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:05 smithi094 ceph-mon[32499]: pgmap v436: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.7 MiB/s wr, 660 op/s 2024-02-17T12:12:07.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:07 smithi189 ceph-mon[36588]: pgmap v437: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.4 MiB/s wr, 554 op/s 2024-02-17T12:12:07.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:07 smithi125 ceph-mon[36819]: pgmap v437: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.4 MiB/s wr, 554 op/s 2024-02-17T12:12:07.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:07 smithi094 ceph-mon[32499]: pgmap v437: 97 pgs: 97 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 5.4 MiB/s wr, 554 op/s 2024-02-17T12:12:07.510 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.506+0000 7f5e83593640 1 -- 172.21.15.94:0/835835036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c10f8c0 msgr2=0x7f5e7c10d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:07.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.506+0000 7f5e83593640 1 --2- 172.21.15.94:0/835835036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c10f8c0 0x7f5e7c10d4e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5e7801a910 tx=0x7f5e7803cc60 comp rx=0 tx=0).stop 2024-02-17T12:12:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.514+0000 7f5e83593640 1 -- 172.21.15.94:0/835835036 shutdown_connections 2024-02-17T12:12:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.514+0000 7f5e83593640 1 --2- 172.21.15.94:0/835835036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e7c11a960 0x7f5e7c11cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.514+0000 7f5e83593640 1 --2- 172.21.15.94:0/835835036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e7c10fd30 0x7f5e7c10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.514+0000 7f5e83593640 1 --2- 172.21.15.94:0/835835036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c10f8c0 0x7f5e7c10d4e0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.514+0000 7f5e83593640 1 -- 172.21.15.94:0/835835036 >> 172.21.15.94:0/835835036 conn(0x7f5e7c06c220 msgr2=0x7f5e7c06c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.515+0000 7f5e83593640 1 -- 172.21.15.94:0/835835036 shutdown_connections 2024-02-17T12:12:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.516+0000 7f5e83593640 1 -- 172.21.15.94:0/835835036 wait complete. 2024-02-17T12:12:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.517+0000 7f5e83593640 1 Processor -- start 2024-02-17T12:12:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.517+0000 7f5e83593640 1 -- start start 2024-02-17T12:12:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.517+0000 7f5e83593640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e7c10f8c0 0x7f5e7c11a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.517+0000 7f5e83593640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e7c10fd30 0x7f5e7c116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.517+0000 7f5e83593640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c11a960 0x7f5e7c1165d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.517+0000 7f5e83593640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c116bd0 con 0x7f5e7c11a960 2024-02-17T12:12:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.517+0000 7f5e83593640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c116d40 con 0x7f5e7c10f8c0 2024-02-17T12:12:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.517+0000 7f5e83593640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c116f00 con 0x7f5e7c10fd30 2024-02-17T12:12:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.517+0000 7f5e80b2e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e7c10fd30 0x7f5e7c116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.518+0000 7f5e80b2e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e7c10fd30 0x7f5e7c116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44652/0 (socket says 172.21.15.94:44652) 2024-02-17T12:12:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.518+0000 7f5e80b2e640 1 -- 172.21.15.94:0/4033466715 learned_addr learned my addr 172.21.15.94:0/4033466715 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.519+0000 7f5e8132f640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e7c10f8c0 0x7f5e7c11a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.520+0000 7f5e81b30640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c11a960 0x7f5e7c1165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.520+0000 7f5e80b2e640 1 -- 172.21.15.94:0/4033466715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e7c10f8c0 msgr2=0x7f5e7c11a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.520+0000 7f5e80b2e640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e7c10f8c0 0x7f5e7c11a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.520+0000 7f5e80b2e640 1 -- 172.21.15.94:0/4033466715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c11a960 msgr2=0x7f5e7c1165d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.520+0000 7f5e80b2e640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c11a960 0x7f5e7c1165d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.520+0000 7f5e80b2e640 1 -- 172.21.15.94:0/4033466715 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7801a570 con 0x7f5e7c10fd30 2024-02-17T12:12:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.520+0000 7f5e8132f640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e7c10f8c0 0x7f5e7c11a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:12:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.520+0000 7f5e80b2e640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e7c10fd30 0x7f5e7c116090 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000b450 tx=0x7f5e7000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.523+0000 7f5e6e7fc640 1 -- 172.21.15.94:0/4033466715 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e70011890 con 0x7f5e7c10fd30 2024-02-17T12:12:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.523+0000 7f5e6e7fc640 1 -- 172.21.15.94:0/4033466715 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e700103e0 con 0x7f5e7c10fd30 2024-02-17T12:12:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.523+0000 7f5e6e7fc640 1 -- 172.21.15.94:0/4033466715 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000f580 con 0x7f5e7c10fd30 2024-02-17T12:12:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.524+0000 7f5e83593640 1 -- 172.21.15.94:0/4033466715 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7c1aa650 con 0x7f5e7c10fd30 2024-02-17T12:12:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.525+0000 7f5e83593640 1 -- 172.21.15.94:0/4033466715 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c1aaa20 con 0x7f5e7c10fd30 2024-02-17T12:12:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.529+0000 7f5e8616d740 1 Processor -- start 2024-02-17T12:12:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.529+0000 7f5e8616d740 1 -- start start 2024-02-17T12:12:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.530+0000 7f5e8616d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d0ad0 0x55d11c0cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.531+0000 7f5e6e7fc640 1 -- 172.21.15.94:0/4033466715 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5e700108e0 con 0x7f5e7c10fd30 2024-02-17T12:12:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.531+0000 7f5e6e7fc640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5e680a2500 0x7f5e680a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.532+0000 7f5e6e7fc640 1 -- 172.21.15.94:0/4033466715 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5e700cde60 con 0x7f5e7c10fd30 2024-02-17T12:12:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.532+0000 7f5e8132f640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5e680a2500 0x7f5e680a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.533+0000 7f5e81b30640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d0ad0 0x55d11c0cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.533+0000 7f5e8616d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0cfd60 0x55d11c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.533+0000 7f5e8616d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0d92e0 0x55d11c0db6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.533+0000 7f5e8616d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c03e9f0 con 0x55d11c0d0ad0 2024-02-17T12:12:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.533+0000 7f5e8616d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c03bf20 con 0x55d11c0cfd60 2024-02-17T12:12:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.533+0000 7f5e8616d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c037700 con 0x55d11c0d92e0 2024-02-17T12:12:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.533+0000 7f5e8132f640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5e680a2500 0x7f5e680a49c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5e7801a910 tx=0x7f5e78005fd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.534+0000 7f5e81b30640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d0ad0 0x55d11c0cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33216/0 (socket says 172.21.15.94:33216) 2024-02-17T12:12:07.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.534+0000 7f5e81b30640 1 -- 172.21.15.94:0/1379536647 learned_addr learned my addr 172.21.15.94:0/1379536647 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:07.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.534+0000 7f5e81b30640 1 --2- 172.21.15.94:0/1379536647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0d92e0 0x55d11c0db6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.536+0000 7f5e81b30640 1 -- 172.21.15.94:0/1379536647 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0cfd60 msgr2=0x55d11c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:07.538 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.536+0000 7f5e80b2e640 1 --2- 172.21.15.94:0/1379536647 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0cfd60 0x55d11c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.537+0000 7f5e81b30640 1 --2- 172.21.15.94:0/1379536647 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0cfd60 0x55d11c0d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.537+0000 7f5e81b30640 1 -- 172.21.15.94:0/1379536647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d0ad0 msgr2=0x55d11c0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.537+0000 7f5e81b30640 1 --2- 172.21.15.94:0/1379536647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d0ad0 0x55d11c0cf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.537+0000 7f5e81b30640 1 -- 172.21.15.94:0/1379536647 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d11bfcd7b0 con 0x55d11c0d92e0 2024-02-17T12:12:07.539 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.537+0000 7f5e81b30640 1 --2- 172.21.15.94:0/1379536647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0d92e0 0x55d11c0db6d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400ead0 tx=0x7f5e74004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f49f455236a2c07 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:07.544 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.542+0000 7f5e677fe640 1 -- 172.21.15.94:0/1379536647 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400e0f0 con 0x55d11c0d92e0 2024-02-17T12:12:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.542+0000 7f5e677fe640 1 -- 172.21.15.94:0/1379536647 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e74007d10 con 0x55d11c0d92e0 2024-02-17T12:12:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.542+0000 7f5e677fe640 1 -- 172.21.15.94:0/1379536647 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400aa60 con 0x55d11c0d92e0 2024-02-17T12:12:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.543+0000 7f5e8616d740 1 -- 172.21.15.94:0/1379536647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0d92e0 msgr2=0x55d11c0db6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:07.549 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.543+0000 7f5e8616d740 1 --2- 172.21.15.94:0/1379536647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0d92e0 0x55d11c0db6d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400ead0 tx=0x7f5e74004a40 comp rx=0 tx=0).stop 2024-02-17T12:12:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.548+0000 7f5e8616d740 1 -- 172.21.15.94:0/1379536647 shutdown_connections 2024-02-17T12:12:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.548+0000 7f5e8616d740 1 --2- 172.21.15.94:0/1379536647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0d92e0 0x55d11c0db6d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.548+0000 7f5e8616d740 1 --2- 172.21.15.94:0/1379536647 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0cfd60 0x55d11c0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.548+0000 7f5e8616d740 1 --2- 172.21.15.94:0/1379536647 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d0ad0 0x55d11c0cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.550 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.548+0000 7f5e8616d740 1 -- 172.21.15.94:0/1379536647 >> 172.21.15.94:0/1379536647 conn(0x55d11c097550 msgr2=0x55d11c0caa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:07.551 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.549+0000 7f5e8616d740 1 -- 172.21.15.94:0/1379536647 shutdown_connections 2024-02-17T12:12:07.552 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.551+0000 7f5e8616d740 1 -- 172.21.15.94:0/1379536647 wait complete. 2024-02-17T12:12:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.551+0000 7f5e8616d740 1 -- start start 2024-02-17T12:12:07.553 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.551+0000 7f5e8616d740 1 Processor -- start 2024-02-17T12:12:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e8616d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0cfd60 0x55d11c0d6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e8616d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d0ad0 0x55d11c0d68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e8616d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d92e0 0x55d11c0d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.554 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e8616d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11bfdcbf0 con 0x55d11c0d0ad0 2024-02-17T12:12:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e8616d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11bd0b250 con 0x55d11c0d92e0 2024-02-17T12:12:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e8616d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11bfbd200 con 0x55d11c0cfd60 2024-02-17T12:12:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e80b2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d0ad0 0x55d11c0d68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e80b2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d0ad0 0x55d11c0d68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33228/0 (socket says 172.21.15.94:33228) 2024-02-17T12:12:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e80b2e640 1 -- 172.21.15.94:0/220458291 learned_addr learned my addr 172.21.15.94:0/220458291 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e8132f640 1 --2- 172.21.15.94:0/220458291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0cfd60 0x55d11c0d6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e81b30640 1 --2- 172.21.15.94:0/220458291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d92e0 0x55d11c0d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e8616d740 10 client.0 _ll_register_callbacks cb 0x55d11c02bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e81b30640 1 -- 172.21.15.94:0/220458291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0cfd60 msgr2=0x55d11c0d6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e81b30640 1 --2- 172.21.15.94:0/220458291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0cfd60 0x55d11c0d6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e81b30640 1 -- 172.21.15.94:0/220458291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d0ad0 msgr2=0x55d11c0d68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e81b30640 1 --2- 172.21.15.94:0/220458291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d0ad0 0x55d11c0d68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.552+0000 7f5e81b30640 1 -- 172.21.15.94:0/220458291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e74016040 con 0x55d11c0d92e0 2024-02-17T12:12:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.553+0000 7f5e81b30640 1 --2- 172.21.15.94:0/220458291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d92e0 0x55d11c0d6e10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5e740097d0 tx=0x7f5e74007a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c8d3348824a469 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.553+0000 7f5e64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:12:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e64ff9640 1 -- 172.21.15.94:0/220458291 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400a400 con 0x55d11c0d92e0 2024-02-17T12:12:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e64ff9640 1 -- 172.21.15.94:0/220458291 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5e7400a9f0 con 0x55d11c0d92e0 2024-02-17T12:12:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e8616d740 1 -- 172.21.15.94:0/220458291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d11bd37600 con 0x55d11c0d92e0 2024-02-17T12:12:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e64ff9640 1 -- 172.21.15.94:0/220458291 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e7400e7b0 con 0x55d11c0d92e0 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e8616d740 10 client.28550 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e8616d740 1 -- 172.21.15.94:0/220458291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d11c035d80 con 0x55d11c0d92e0 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e64ff9640 1 -- 172.21.15.94:0/220458291 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e7400abb0 con 0x55d11c0d92e0 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e64ff9640 1 -- 172.21.15.94:0/220458291 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5e7402b070 con 0x55d11c0d92e0 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e8616d740 10 client.28550 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e8616d740 10 client.28550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e8616d740 20 client.28550 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e8616d740 1 --2- 172.21.15.94:0/220458291 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d11c0e9570 0x55d11c0eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e8616d740 4 client.28550 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.554+0000 7f5e8616d740 1 -- 172.21.15.94:0/220458291 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d11bd37600 con 0x55d11c0e9570 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.555+0000 7f5e8132f640 1 --2- 172.21.15.94:0/220458291 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d11c0e9570 0x55d11c0eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.555+0000 7f5e64ff9640 10 client.28550 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.555+0000 7f5e8132f640 1 --2- 172.21.15.94:0/220458291 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d11c0e9570 0x55d11c0eb9d0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7ae62dfd517bd5 server_cookie=a8a2f8fc2487a703 in_seq=0 out_seq=0 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e64ff9640 1 -- 172.21.15.94:0/220458291 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d11bd37600 con 0x55d11c0e9570 2024-02-17T12:12:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e64ff9640 10 client.28550 handle_command_reply: tid=0 2024-02-17T12:12:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e8616d740 1 client.28550 shutdown 2024-02-17T12:12:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e8616d740 1 -- 172.21.15.94:0/220458291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d92e0 msgr2=0x55d11c0d6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e8616d740 1 --2- 172.21.15.94:0/220458291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d92e0 0x55d11c0d6e10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5e740097d0 tx=0x7f5e74007a40 comp rx=0 tx=0).stop 2024-02-17T12:12:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e8616d740 1 -- 172.21.15.94:0/220458291 shutdown_connections 2024-02-17T12:12:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e8616d740 1 --2- 172.21.15.94:0/220458291 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d11c0e9570 0x55d11c0eb9d0 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e64ff9640 0 client.28550 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e8616d740 1 --2- 172.21.15.94:0/220458291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d92e0 0x55d11c0d6e10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e8616d740 1 --2- 172.21.15.94:0/220458291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d0ad0 0x55d11c0d68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e8616d740 1 --2- 172.21.15.94:0/220458291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0cfd60 0x55d11c0d6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e8616d740 1 -- 172.21.15.94:0/220458291 >> 172.21.15.94:0/220458291 conn(0x55d11c097550 msgr2=0x55d11c0cc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e8616d740 1 -- 172.21.15.94:0/220458291 shutdown_connections 2024-02-17T12:12:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e8616d740 1 -- 172.21.15.94:0/220458291 wait complete. 2024-02-17T12:12:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.556+0000 7f5e8616d740 20 client.28550 trim_cache size 0 max 16384 2024-02-17T12:12:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.571+0000 7f5e8616d740 1 Processor -- start 2024-02-17T12:12:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.571+0000 7f5e8616d740 1 -- start start 2024-02-17T12:12:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.571+0000 7f5e8616d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0cfd60 0x55d11c0d68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.571+0000 7f5e8616d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d0ad0 0x55d11c0d6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.571+0000 7f5e8616d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0d92e0 0x55d11c0d4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.571+0000 7f5e8616d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c0e9340 con 0x55d11c0cfd60 2024-02-17T12:12:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.571+0000 7f5e8616d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c0cf030 con 0x55d11c0d0ad0 2024-02-17T12:12:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.571+0000 7f5e8616d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c0c6310 con 0x55d11c0d92e0 2024-02-17T12:12:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.572+0000 7f5e8132f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0cfd60 0x55d11c0d68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.572+0000 7f5e8132f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0cfd60 0x55d11c0d68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33234/0 (socket says 172.21.15.94:33234) 2024-02-17T12:12:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.572+0000 7f5e8132f640 1 -- 172.21.15.94:0/3056184662 learned_addr learned my addr 172.21.15.94:0/3056184662 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.572+0000 7f5e80b2e640 1 --2- 172.21.15.94:0/3056184662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d0ad0 0x55d11c0d6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.572+0000 7f5e81b30640 1 --2- 172.21.15.94:0/3056184662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0d92e0 0x55d11c0d4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.572+0000 7f5e8132f640 1 -- 172.21.15.94:0/3056184662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0d92e0 msgr2=0x55d11c0d4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.572+0000 7f5e8132f640 1 --2- 172.21.15.94:0/3056184662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0d92e0 0x55d11c0d4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.572+0000 7f5e8132f640 1 -- 172.21.15.94:0/3056184662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d0ad0 msgr2=0x55d11c0d6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.572+0000 7f5e8132f640 1 --2- 172.21.15.94:0/3056184662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d0ad0 0x55d11c0d6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.572+0000 7f5e8132f640 1 -- 172.21.15.94:0/3056184662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d11bfd27b0 con 0x55d11c0cfd60 2024-02-17T12:12:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.572+0000 7f5e8132f640 1 --2- 172.21.15.94:0/3056184662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0cfd60 0x55d11c0d68e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5e78004db0 tx=0x7f5e7804d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1299b8b089f20940 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.572+0000 7f5e66ffd640 1 -- 172.21.15.94:0/3056184662 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78061070 con 0x55d11c0cfd60 2024-02-17T12:12:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.572+0000 7f5e66ffd640 1 -- 172.21.15.94:0/3056184662 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e780053e0 con 0x55d11c0cfd60 2024-02-17T12:12:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.573+0000 7f5e66ffd640 1 -- 172.21.15.94:0/3056184662 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78003740 con 0x55d11c0cfd60 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.573+0000 7f5e8616d740 1 -- 172.21.15.94:0/3056184662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0cfd60 msgr2=0x55d11c0d68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.573+0000 7f5e8616d740 1 --2- 172.21.15.94:0/3056184662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0cfd60 0x55d11c0d68e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5e78004db0 tx=0x7f5e7804d000 comp rx=0 tx=0).stop 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.573+0000 7f5e8616d740 1 -- 172.21.15.94:0/3056184662 shutdown_connections 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.573+0000 7f5e8616d740 1 --2- 172.21.15.94:0/3056184662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0d92e0 0x55d11c0d4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.573+0000 7f5e8616d740 1 --2- 172.21.15.94:0/3056184662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d0ad0 0x55d11c0d6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.573+0000 7f5e8616d740 1 --2- 172.21.15.94:0/3056184662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0cfd60 0x55d11c0d68e0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.573+0000 7f5e8616d740 1 -- 172.21.15.94:0/3056184662 >> 172.21.15.94:0/3056184662 conn(0x55d11c097550 msgr2=0x55d11c0ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.573+0000 7f5e8616d740 1 -- 172.21.15.94:0/3056184662 shutdown_connections 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.573+0000 7f5e8616d740 1 -- 172.21.15.94:0/3056184662 wait complete. 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.574+0000 7f5e8616d740 1 -- start start 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.574+0000 7f5e8616d740 1 Processor -- start 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.574+0000 7f5e8616d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0cfd60 0x55d11c0e0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.574+0000 7f5e8616d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d0ad0 0x55d11c0de7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.574+0000 7f5e8616d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d92e0 0x55d11c0ded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.574+0000 7f5e8616d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c0df400 con 0x55d11c0d92e0 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.574+0000 7f5e8616d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c0df570 con 0x55d11c0d0ad0 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.574+0000 7f5e8616d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11c13fac0 con 0x55d11c0cfd60 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.574+0000 7f5e8616d740 10 client.0 _ll_register_callbacks cb 0x7f5e74009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.574+0000 7f5e8132f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0cfd60 0x55d11c0e0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.574+0000 7f5e8132f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0cfd60 0x55d11c0e0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44684/0 (socket says 172.21.15.94:44684) 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.574+0000 7f5e8132f640 1 -- 172.21.15.94:0/2042713843 learned_addr learned my addr 172.21.15.94:0/2042713843 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.575+0000 7f5e8132f640 1 -- 172.21.15.94:0/2042713843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d0ad0 msgr2=0x55d11c0de7f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.575+0000 7f5e8132f640 1 --2- 172.21.15.94:0/2042713843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d0ad0 0x55d11c0de7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.575+0000 7f5e8132f640 1 -- 172.21.15.94:0/2042713843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d92e0 msgr2=0x55d11c0ded30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.575+0000 7f5e8132f640 1 --2- 172.21.15.94:0/2042713843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d92e0 0x55d11c0ded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.575+0000 7f5e8132f640 1 -- 172.21.15.94:0/2042713843 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c11d640 con 0x55d11c0cfd60 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.575+0000 7f5e8132f640 1 --2- 172.21.15.94:0/2042713843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0cfd60 0x55d11c0e0400 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5e7801a860 tx=0x7f5e78005df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8e6107f0cf582be server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:07.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.575+0000 7f5e64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:12:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.577+0000 7f5e64ff9640 1 -- 172.21.15.94:0/2042713843 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78061070 con 0x55d11c0cfd60 2024-02-17T12:12:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.577+0000 7f5e64ff9640 1 -- 172.21.15.94:0/2042713843 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5e7803da20 con 0x55d11c0cfd60 2024-02-17T12:12:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.577+0000 7f5e8616d740 1 -- 172.21.15.94:0/2042713843 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e74007d10 con 0x55d11c0cfd60 2024-02-17T12:12:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.577+0000 7f5e64ff9640 1 -- 172.21.15.94:0/2042713843 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e7803dbe0 con 0x55d11c0cfd60 2024-02-17T12:12:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.577+0000 7f5e8616d740 10 client.28525 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.577+0000 7f5e8616d740 1 -- 172.21.15.94:0/2042713843 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d11bff8540 con 0x55d11c0cfd60 2024-02-17T12:12:07.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.577+0000 7f5e64ff9640 1 -- 172.21.15.94:0/2042713843 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e78077080 con 0x55d11c0cfd60 2024-02-17T12:12:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.578+0000 7f5e64ff9640 1 -- 172.21.15.94:0/2042713843 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5e78069020 con 0x55d11c0cfd60 2024-02-17T12:12:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.578+0000 7f5e8616d740 10 client.28525 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.578+0000 7f5e8616d740 10 client.28525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.578+0000 7f5e8616d740 20 client.28525 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.578+0000 7f5e8616d740 1 --2- 172.21.15.94:0/2042713843 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d11c0e9570 0x55d11c142ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.578+0000 7f5e8616d740 4 client.28525 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:12:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.578+0000 7f5e8616d740 1 -- 172.21.15.94:0/2042713843 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f5e74007d10 con 0x55d11c0e9570 2024-02-17T12:12:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.578+0000 7f5e80b2e640 1 --2- 172.21.15.94:0/2042713843 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d11c0e9570 0x55d11c142ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.578+0000 7f5e64ff9640 10 client.28525 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:07.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.578+0000 7f5e80b2e640 1 --2- 172.21.15.94:0/2042713843 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d11c0e9570 0x55d11c142ff0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f8617c33aed5198 server_cookie=efdd4872920d50c6 in_seq=0 out_seq=0 2024-02-17T12:12:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.604+0000 7f5e64ff9640 1 -- 172.21.15.94:0/2042713843 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f5e74007d10 con 0x55d11c0e9570 2024-02-17T12:12:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.604+0000 7f5e64ff9640 10 client.28525 handle_command_reply: tid=0 2024-02-17T12:12:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.604+0000 7f5e8616d740 1 client.28525 shutdown 2024-02-17T12:12:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.604+0000 7f5e8616d740 1 -- 172.21.15.94:0/2042713843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0cfd60 msgr2=0x55d11c0e0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.604+0000 7f5e8616d740 1 --2- 172.21.15.94:0/2042713843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0cfd60 0x55d11c0e0400 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5e7801a860 tx=0x7f5e78005df0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.604+0000 7f5e8616d740 1 -- 172.21.15.94:0/2042713843 shutdown_connections 2024-02-17T12:12:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.604+0000 7f5e8616d740 1 --2- 172.21.15.94:0/2042713843 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55d11c0e9570 0x55d11c142ff0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.604+0000 7f5e8616d740 1 --2- 172.21.15.94:0/2042713843 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55d11c0d92e0 0x55d11c0ded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.604+0000 7f5e8616d740 1 --2- 172.21.15.94:0/2042713843 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55d11c0d0ad0 0x55d11c0de7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.604+0000 7f5e8616d740 1 --2- 172.21.15.94:0/2042713843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d11c0cfd60 0x55d11c0e0400 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.604+0000 7f5e8616d740 1 -- 172.21.15.94:0/2042713843 >> 172.21.15.94:0/2042713843 conn(0x55d11c097550 msgr2=0x55d11c0d4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.604+0000 7f5e64ff9640 0 client.28525 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.605+0000 7f5e8616d740 1 -- 172.21.15.94:0/2042713843 shutdown_connections 2024-02-17T12:12:07.607 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.605+0000 7f5e8616d740 1 -- 172.21.15.94:0/2042713843 wait complete. 2024-02-17T12:12:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.605+0000 7f5e8616d740 20 client.28525 trim_cache size 0 max 16384 2024-02-17T12:12:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:12:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:12:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:12:07.608 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:12:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.605+0000 7f5e83593640 1 -- 172.21.15.94:0/4033466715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5e680a2500 msgr2=0x7f5e680a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.605+0000 7f5e83593640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5e680a2500 0x7f5e680a49c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5e7801a910 tx=0x7f5e78005fd0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.605+0000 7f5e83593640 1 -- 172.21.15.94:0/4033466715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e7c10fd30 msgr2=0x7f5e7c116090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.605+0000 7f5e83593640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e7c10fd30 0x7f5e7c116090 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000b450 tx=0x7f5e7000b910 comp rx=0 tx=0).stop 2024-02-17T12:12:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.605+0000 7f5e83593640 1 -- 172.21.15.94:0/4033466715 shutdown_connections 2024-02-17T12:12:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.605+0000 7f5e83593640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5e680a2500 0x7f5e680a49c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.605+0000 7f5e83593640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5e7c11a960 0x7f5e7c1165d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.605+0000 7f5e83593640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e7c10fd30 0x7f5e7c116090 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.605+0000 7f5e83593640 1 --2- 172.21.15.94:0/4033466715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5e7c10f8c0 0x7f5e7c11a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.605+0000 7f5e83593640 1 -- 172.21.15.94:0/4033466715 >> 172.21.15.94:0/4033466715 conn(0x7f5e7c06c220 msgr2=0x7f5e7c11b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.606+0000 7f5e83593640 1 -- 172.21.15.94:0/4033466715 shutdown_connections 2024-02-17T12:12:07.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:07.606+0000 7f5e83593640 1 -- 172.21.15.94:0/4033466715 wait complete. 2024-02-17T12:12:09.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:09 smithi189 ceph-mon[36588]: osdmap e103: 12 total, 12 up, 12 in 2024-02-17T12:12:09.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:09 smithi189 ceph-mon[36588]: pgmap v439: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 20 MiB/s wr, 1.43k op/s 2024-02-17T12:12:09.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:09 smithi125 ceph-mon[36819]: osdmap e103: 12 total, 12 up, 12 in 2024-02-17T12:12:09.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:09 smithi125 ceph-mon[36819]: pgmap v439: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 20 MiB/s wr, 1.43k op/s 2024-02-17T12:12:09.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:09 smithi094 ceph-mon[32499]: osdmap e103: 12 total, 12 up, 12 in 2024-02-17T12:12:09.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:09 smithi094 ceph-mon[32499]: pgmap v439: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 5.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 20 MiB/s wr, 1.43k op/s 2024-02-17T12:12:09.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:12:09.839 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:12:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.964+0000 7ff607381640 1 -- 172.21.15.94:0/3036485434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6001009a0 msgr2=0x7ff600100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.964+0000 7ff607381640 1 --2- 172.21.15.94:0/3036485434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6001009a0 0x7ff600100d80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009930 tx=0x7ff5fc02bc30 comp rx=0 tx=0).stop 2024-02-17T12:12:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.964+0000 7ff607381640 1 -- 172.21.15.94:0/3036485434 shutdown_connections 2024-02-17T12:12:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.964+0000 7ff607381640 1 --2- 172.21.15.94:0/3036485434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff600105550 0x7ff60010d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.964+0000 7ff607381640 1 --2- 172.21.15.94:0/3036485434 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff600101350 0x7ff600104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.966 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.964+0000 7ff607381640 1 --2- 172.21.15.94:0/3036485434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6001009a0 0x7ff600100d80 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.964+0000 7ff607381640 1 -- 172.21.15.94:0/3036485434 >> 172.21.15.94:0/3036485434 conn(0x7ff6000765b0 msgr2=0x7ff6000fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.965+0000 7ff607381640 1 -- 172.21.15.94:0/3036485434 shutdown_connections 2024-02-17T12:12:09.967 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.965+0000 7ff607381640 1 -- 172.21.15.94:0/3036485434 wait complete. 2024-02-17T12:12:09.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.966+0000 7ff607381640 1 Processor -- start 2024-02-17T12:12:09.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.966+0000 7ff607381640 1 -- start start 2024-02-17T12:12:09.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.966+0000 7ff607381640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6001009a0 0x7ff60019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:09.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.966+0000 7ff607381640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff600105550 0x7ff60019dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:09.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.966+0000 7ff607381640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff600197810 0x7ff600197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:09.969 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.966+0000 7ff607381640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001982c0 con 0x7ff6001009a0 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.966+0000 7ff607381640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff600198430 con 0x7ff600105550 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.966+0000 7ff607381640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001985a0 con 0x7ff600197810 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.966+0000 7ff60511d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6001009a0 0x7ff60019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.966+0000 7ff60511d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6001009a0 0x7ff60019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33268/0 (socket says 172.21.15.94:33268) 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.966+0000 7ff60511d640 1 -- 172.21.15.94:0/3073648974 learned_addr learned my addr 172.21.15.94:0/3073648974 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.966+0000 7ff60491c640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff600105550 0x7ff60019dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.966+0000 7ff60591e640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff600197810 0x7ff600197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.967+0000 7ff60511d640 1 -- 172.21.15.94:0/3073648974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff600197810 msgr2=0x7ff600197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.967+0000 7ff60511d640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff600197810 0x7ff600197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.967+0000 7ff60511d640 1 -- 172.21.15.94:0/3073648974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff600105550 msgr2=0x7ff60019dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.967+0000 7ff60511d640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff600105550 0x7ff60019dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.967+0000 7ff60511d640 1 -- 172.21.15.94:0/3073648974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc009560 con 0x7ff6001009a0 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.967+0000 7ff60511d640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6001009a0 0x7ff60019d6a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009230 tx=0x7ff5fc02c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:09.970 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.967+0000 7ff5f67fc640 1 -- 172.21.15.94:0/3073648974 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc004350 con 0x7ff6001009a0 2024-02-17T12:12:09.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.967+0000 7ff607381640 1 -- 172.21.15.94:0/3073648974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6001a44f0 con 0x7ff6001009a0 2024-02-17T12:12:09.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.967+0000 7ff607381640 1 -- 172.21.15.94:0/3073648974 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6001a4a00 con 0x7ff6001009a0 2024-02-17T12:12:09.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.968+0000 7ff5f67fc640 1 -- 172.21.15.94:0/3073648974 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc0044b0 con 0x7ff6001009a0 2024-02-17T12:12:09.971 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.968+0000 7ff5f67fc640 1 -- 172.21.15.94:0/3073648974 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc003c20 con 0x7ff6001009a0 2024-02-17T12:12:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60a032740 1 Processor -- start 2024-02-17T12:12:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60a032740 1 -- start start 2024-02-17T12:12:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff5f67fc640 1 -- 172.21.15.94:0/3073648974 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff5fc004df0 con 0x7ff6001009a0 2024-02-17T12:12:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60a032740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2a2b0 0x56496ea2a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60a032740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2abd0 0x56496ea33270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60a032740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea337b0 0x56496ea35ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60a032740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496e999280 con 0x56496ea2a2b0 2024-02-17T12:12:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60a032740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496e9967b0 con 0x56496ea2abd0 2024-02-17T12:12:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60a032740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496e991f90 con 0x56496ea337b0 2024-02-17T12:12:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60591e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2a2b0 0x56496ea2a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60491c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea337b0 0x56496ea35ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:09.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60511d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2abd0 0x56496ea33270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60591e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2a2b0 0x56496ea2a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33274/0 (socket says 172.21.15.94:33274) 2024-02-17T12:12:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60491c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea337b0 0x56496ea35ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44724/0 (socket says 172.21.15.94:44724) 2024-02-17T12:12:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.970+0000 7ff60591e640 1 -- 172.21.15.94:0/3254830805 learned_addr learned my addr 172.21.15.94:0/3254830805 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:09.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.971+0000 7ff5f67fc640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff5ec0a22b0 0x7ff5ec0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.971+0000 7ff60591e640 1 -- 172.21.15.94:0/3254830805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea337b0 msgr2=0x56496ea35ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.971+0000 7ff60591e640 1 --2- 172.21.15.94:0/3254830805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea337b0 0x56496ea35ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.971+0000 7ff60591e640 1 -- 172.21.15.94:0/3254830805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2abd0 msgr2=0x56496ea33270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.971+0000 7ff60591e640 1 --2- 172.21.15.94:0/3254830805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2abd0 0x56496ea33270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.971+0000 7ff60591e640 1 -- 172.21.15.94:0/3254830805 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56496e950570 con 0x56496ea2a2b0 2024-02-17T12:12:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.971+0000 7ff5f67fc640 1 -- 172.21.15.94:0/3073648974 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff5fc0efed0 con 0x7ff6001009a0 2024-02-17T12:12:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.971+0000 7ff60591e640 1 --2- 172.21.15.94:0/3254830805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2a2b0 0x56496ea2a690 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009f50 tx=0x7ff5f80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd83a8e4678bc5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.971+0000 7ff60591e640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff5ec0a22b0 0x7ff5ec0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.972+0000 7ff5eb7fe640 1 -- 172.21.15.94:0/3254830805 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8026070 con 0x56496ea2a2b0 2024-02-17T12:12:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.972+0000 7ff5eb7fe640 1 -- 172.21.15.94:0/3254830805 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5f80169f0 con 0x56496ea2a2b0 2024-02-17T12:12:09.975 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.972+0000 7ff5eb7fe640 1 -- 172.21.15.94:0/3254830805 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80217f0 con 0x56496ea2a2b0 2024-02-17T12:12:09.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.973+0000 7ff60a032740 1 -- 172.21.15.94:0/3254830805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2a2b0 msgr2=0x56496ea2a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:09.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.973+0000 7ff60a032740 1 --2- 172.21.15.94:0/3254830805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2a2b0 0x56496ea2a690 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009f50 tx=0x7ff5f80118d0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.973+0000 7ff60591e640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff5ec0a22b0 0x7ff5ec0a4770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8025000 tx=0x7ff5f8026000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:09.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.973+0000 7ff60a032740 1 -- 172.21.15.94:0/3254830805 shutdown_connections 2024-02-17T12:12:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.973+0000 7ff60a032740 1 --2- 172.21.15.94:0/3254830805 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea337b0 0x56496ea35ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.973+0000 7ff60a032740 1 --2- 172.21.15.94:0/3254830805 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2abd0 0x56496ea33270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.973+0000 7ff60a032740 1 --2- 172.21.15.94:0/3254830805 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2a2b0 0x56496ea2a690 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.973+0000 7ff60a032740 1 -- 172.21.15.94:0/3254830805 >> 172.21.15.94:0/3254830805 conn(0x56496ea24990 msgr2=0x56496ea24d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.973+0000 7ff60a032740 1 -- 172.21.15.94:0/3254830805 shutdown_connections 2024-02-17T12:12:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.973+0000 7ff60a032740 1 -- 172.21.15.94:0/3254830805 wait complete. 2024-02-17T12:12:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60a032740 1 -- start start 2024-02-17T12:12:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60a032740 1 Processor -- start 2024-02-17T12:12:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60a032740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea2a2b0 0x56496ea331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60a032740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2abd0 0x56496ea30530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60a032740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea337b0 0x56496ea30b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60a032740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496e665430 con 0x56496ea337b0 2024-02-17T12:12:09.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60a032740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496e621e00 con 0x56496ea2abd0 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60a032740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496e916fd0 con 0x56496ea2a2b0 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60a032740 10 client.0 _ll_register_callbacks cb 0x56496e96cfd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60491c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2abd0 0x56496ea30530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60591e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea337b0 0x56496ea30b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60491c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2abd0 0x56496ea30530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36794/0 (socket says 172.21.15.94:36794) 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60491c640 1 -- 172.21.15.94:0/3797319171 learned_addr learned my addr 172.21.15.94:0/3797319171 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.974+0000 7ff60511d640 1 --2- 172.21.15.94:0/3797319171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea2a2b0 0x56496ea331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff60491c640 1 -- 172.21.15.94:0/3797319171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea2a2b0 msgr2=0x56496ea331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff60491c640 1 --2- 172.21.15.94:0/3797319171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea2a2b0 0x56496ea331a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff60491c640 1 -- 172.21.15.94:0/3797319171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea337b0 msgr2=0x56496ea30b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff60491c640 1 --2- 172.21.15.94:0/3797319171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea337b0 0x56496ea30b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff60491c640 1 -- 172.21.15.94:0/3797319171 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f8009c00 con 0x56496ea2abd0 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff60491c640 1 --2- 172.21.15.94:0/3797319171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2abd0 0x56496ea30530 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0012700 tx=0x7ff5f0012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9af7427752b9c2b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:09.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff5e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff5e8ff9640 1 -- 172.21.15.94:0/3797319171 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0017070 con 0x56496ea2abd0 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff5e8ff9640 1 -- 172.21.15.94:0/3797319171 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff5f000f8c0 con 0x56496ea2abd0 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff60a032740 1 -- 172.21.15.94:0/3797319171 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56496e6917e0 con 0x56496ea2abd0 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff5e8ff9640 1 -- 172.21.15.94:0/3797319171 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5f000fa80 con 0x56496ea2abd0 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff60a032740 10 client.28565 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff60a032740 1 -- 172.21.15.94:0/3797319171 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56496e990610 con 0x56496ea2abd0 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff5e8ff9640 1 -- 172.21.15.94:0/3797319171 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56496e6917e0 con 0x56496ea2abd0 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.975+0000 7ff5e8ff9640 1 -- 172.21.15.94:0/3797319171 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff5f001c420 con 0x56496ea2abd0 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.976+0000 7ff60a032740 10 client.28565 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.976+0000 7ff60a032740 10 client.28565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.976+0000 7ff60a032740 20 client.28565 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.976+0000 7ff60a032740 1 --2- 172.21.15.94:0/3797319171 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56496ea43ae0 0x56496ea45f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.976+0000 7ff60a032740 4 client.28565 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.976+0000 7ff60a032740 1 -- 172.21.15.94:0/3797319171 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56496e6ce720 con 0x56496ea43ae0 2024-02-17T12:12:09.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.976+0000 7ff60511d640 1 --2- 172.21.15.94:0/3797319171 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56496ea43ae0 0x56496ea45f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:09.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.976+0000 7ff60511d640 1 --2- 172.21.15.94:0/3797319171 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56496ea43ae0 0x56496ea45f00 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f2d570d16df096 server_cookie=824936549ff9431c in_seq=0 out_seq=0 2024-02-17T12:12:09.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.976+0000 7ff5e8ff9640 10 client.28565 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.977+0000 7ff5e8ff9640 1 -- 172.21.15.94:0/3797319171 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56496e6ce720 con 0x56496ea43ae0 2024-02-17T12:12:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.977+0000 7ff5e8ff9640 10 client.28565 handle_command_reply: tid=0 2024-02-17T12:12:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.977+0000 7ff60a032740 1 client.28565 shutdown 2024-02-17T12:12:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.977+0000 7ff60a032740 1 -- 172.21.15.94:0/3797319171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2abd0 msgr2=0x56496ea30530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.977+0000 7ff60a032740 1 --2- 172.21.15.94:0/3797319171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2abd0 0x56496ea30530 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0012700 tx=0x7ff5f0012bc0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.977+0000 7ff60a032740 1 -- 172.21.15.94:0/3797319171 shutdown_connections 2024-02-17T12:12:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.977+0000 7ff60a032740 1 --2- 172.21.15.94:0/3797319171 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56496ea43ae0 0x56496ea45f00 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.977+0000 7ff60a032740 1 --2- 172.21.15.94:0/3797319171 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea337b0 0x56496ea30b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.977+0000 7ff60a032740 1 --2- 172.21.15.94:0/3797319171 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2abd0 0x56496ea30530 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.977+0000 7ff60a032740 1 --2- 172.21.15.94:0/3797319171 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea2a2b0 0x56496ea331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.977+0000 7ff60a032740 1 -- 172.21.15.94:0/3797319171 >> 172.21.15.94:0/3797319171 conn(0x56496ea24990 msgr2=0x56496ea26730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:09.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.977+0000 7ff5e8ff9640 0 client.28565 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:09.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.978+0000 7ff60a032740 1 -- 172.21.15.94:0/3797319171 shutdown_connections 2024-02-17T12:12:09.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.978+0000 7ff60a032740 1 -- 172.21.15.94:0/3797319171 wait complete. 2024-02-17T12:12:09.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.978+0000 7ff60a032740 20 client.28565 trim_cache size 0 max 16384 2024-02-17T12:12:10.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:09.998+0000 7ff60a032740 1 Processor -- start 2024-02-17T12:12:10.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.001+0000 7ff60a032740 1 -- start start 2024-02-17T12:12:10.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.001+0000 7ff60a032740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2a2b0 0x56496ea3f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.001+0000 7ff60a032740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea2abd0 0x56496ea32c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.001+0000 7ff60a032740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea337b0 0x56496ea331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.002+0000 7ff60a032740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496ea293d0 con 0x56496ea2a2b0 2024-02-17T12:12:10.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.002+0000 7ff60a032740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496ea24820 con 0x56496ea337b0 2024-02-17T12:12:10.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.002+0000 7ff60a032740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496ea25a30 con 0x56496ea2abd0 2024-02-17T12:12:10.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.002+0000 7ff60591e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea337b0 0x56496ea331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.002+0000 7ff60591e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea337b0 0x56496ea331a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36808/0 (socket says 172.21.15.94:36808) 2024-02-17T12:12:10.004 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.002+0000 7ff60591e640 1 -- 172.21.15.94:0/3036575195 learned_addr learned my addr 172.21.15.94:0/3036575195 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:10.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.003+0000 7ff60511d640 1 --2- 172.21.15.94:0/3036575195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2a2b0 0x56496ea3f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.003+0000 7ff60511d640 1 -- 172.21.15.94:0/3036575195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea2abd0 msgr2=0x56496ea32c60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:12:10.005 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.003+0000 7ff60511d640 1 --2- 172.21.15.94:0/3036575195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea2abd0 0x56496ea32c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.003+0000 7ff60511d640 1 -- 172.21.15.94:0/3036575195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea337b0 msgr2=0x56496ea331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.003+0000 7ff60511d640 1 --2- 172.21.15.94:0/3036575195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea337b0 0x56496ea331a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.003+0000 7ff60511d640 1 -- 172.21.15.94:0/3036575195 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56496e92cac0 con 0x56496ea2a2b0 2024-02-17T12:12:10.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.004+0000 7ff60511d640 1 --2- 172.21.15.94:0/3036575195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2a2b0 0x56496ea3f170 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0ed6c0 tx=0x7ff5fc0f39e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee1c638b5b54877 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.005+0000 7ff5eaffd640 1 -- 172.21.15.94:0/3036575195 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0a5ea0 con 0x56496ea2a2b0 2024-02-17T12:12:10.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.005+0000 7ff5eaffd640 1 -- 172.21.15.94:0/3036575195 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc0bc730 con 0x56496ea2a2b0 2024-02-17T12:12:10.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.005+0000 7ff5eaffd640 1 -- 172.21.15.94:0/3036575195 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0bc960 con 0x56496ea2a2b0 2024-02-17T12:12:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.006+0000 7ff60a032740 1 -- 172.21.15.94:0/3036575195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2a2b0 msgr2=0x56496ea3f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.010 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.006+0000 7ff60a032740 1 --2- 172.21.15.94:0/3036575195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2a2b0 0x56496ea3f170 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0ed6c0 tx=0x7ff5fc0f39e0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.017+0000 7ff60a032740 1 -- 172.21.15.94:0/3036575195 shutdown_connections 2024-02-17T12:12:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.017+0000 7ff60a032740 1 --2- 172.21.15.94:0/3036575195 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea337b0 0x56496ea331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.017+0000 7ff60a032740 1 --2- 172.21.15.94:0/3036575195 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea2abd0 0x56496ea32c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.017+0000 7ff60a032740 1 --2- 172.21.15.94:0/3036575195 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2a2b0 0x56496ea3f170 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.017+0000 7ff60a032740 1 -- 172.21.15.94:0/3036575195 >> 172.21.15.94:0/3036575195 conn(0x56496ea24990 msgr2=0x56496ea2b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.026 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.025+0000 7ff60a032740 1 -- 172.21.15.94:0/3036575195 shutdown_connections 2024-02-17T12:12:10.030 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.029+0000 7ff60a032740 1 -- 172.21.15.94:0/3036575195 wait complete. 2024-02-17T12:12:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.029+0000 7ff60a032740 1 -- start start 2024-02-17T12:12:10.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.029+0000 7ff60a032740 1 Processor -- start 2024-02-17T12:12:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.030+0000 7ff60a032740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2a2b0 0x56496ea3a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.030+0000 7ff60a032740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2abd0 0x56496ea3ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.030+0000 7ff60a032740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea337b0 0x56496ea386a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.030+0000 7ff60a032740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496ea38be0 con 0x56496ea2abd0 2024-02-17T12:12:10.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.031+0000 7ff60a032740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496ea38d20 con 0x56496ea2a2b0 2024-02-17T12:12:10.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.031+0000 7ff60a032740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56496ea38ee0 con 0x56496ea337b0 2024-02-17T12:12:10.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.031+0000 7ff60a032740 10 client.0 _ll_register_callbacks cb 0x7ff5f8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:10.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.032+0000 7ff60511d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2a2b0 0x56496ea3a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.032+0000 7ff60511d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2a2b0 0x56496ea3a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36822/0 (socket says 172.21.15.94:36822) 2024-02-17T12:12:10.034 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.032+0000 7ff60511d640 1 -- 172.21.15.94:0/2039109020 learned_addr learned my addr 172.21.15.94:0/2039109020 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.033+0000 7ff60511d640 1 -- 172.21.15.94:0/2039109020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea337b0 msgr2=0x56496ea386a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.033+0000 7ff60511d640 1 --2- 172.21.15.94:0/2039109020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea337b0 0x56496ea386a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.033+0000 7ff60511d640 1 -- 172.21.15.94:0/2039109020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2abd0 msgr2=0x56496ea3ac00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:12:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.033+0000 7ff60511d640 1 --2- 172.21.15.94:0/2039109020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2abd0 0x56496ea3ac00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.033+0000 7ff60511d640 1 -- 172.21.15.94:0/2039109020 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6001a4a00 con 0x56496ea2a2b0 2024-02-17T12:12:10.036 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.034+0000 7ff60511d640 1 --2- 172.21.15.94:0/2039109020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2a2b0 0x56496ea3a6c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0f34c0 tx=0x7ff5fc0bc510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d8012535067cf9d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.040 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.037+0000 7ff5e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:12:10.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.044+0000 7ff5e8ff9640 1 -- 172.21.15.94:0/2039109020 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc0a5ea0 con 0x56496ea2a2b0 2024-02-17T12:12:10.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.044+0000 7ff5e8ff9640 1 -- 172.21.15.94:0/2039109020 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff5fc0c0ab0 con 0x56496ea2a2b0 2024-02-17T12:12:10.047 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.044+0000 7ff60a032740 1 -- 172.21.15.94:0/2039109020 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5f80169f0 con 0x56496ea2a2b0 2024-02-17T12:12:10.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.048+0000 7ff5e8ff9640 1 -- 172.21.15.94:0/2039109020 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5fc0c5100 con 0x56496ea2a2b0 2024-02-17T12:12:10.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.048+0000 7ff60a032740 10 client.28571 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:10.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.048+0000 7ff60a032740 1 -- 172.21.15.94:0/2039109020 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56496e9526a0 con 0x56496ea2a2b0 2024-02-17T12:12:10.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.049+0000 7ff5e8ff9640 1 -- 172.21.15.94:0/2039109020 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff5fc0da020 con 0x56496ea2a2b0 2024-02-17T12:12:10.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.049+0000 7ff5e8ff9640 1 -- 172.21.15.94:0/2039109020 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff5fc03c070 con 0x56496ea2a2b0 2024-02-17T12:12:10.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.053+0000 7ff60a032740 10 client.28571 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:10.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.053+0000 7ff60a032740 10 client.28571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:10.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.053+0000 7ff60a032740 20 client.28571 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:10.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.053+0000 7ff60a032740 1 --2- 172.21.15.94:0/2039109020 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56496eaa4350 0x56496eaa6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.053+0000 7ff60a032740 4 client.28571 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:12:10.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.053+0000 7ff60a032740 1 -- 172.21.15.94:0/2039109020 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56496e71d890 con 0x56496eaa4350 2024-02-17T12:12:10.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.054+0000 7ff60491c640 1 --2- 172.21.15.94:0/2039109020 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56496eaa4350 0x56496eaa6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.055+0000 7ff60491c640 1 --2- 172.21.15.94:0/2039109020 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56496eaa4350 0x56496eaa6740 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b09c97462eaf4e95 server_cookie=cb4800589b00c416 in_seq=0 out_seq=0 2024-02-17T12:12:10.057 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.056+0000 7ff5e8ff9640 10 client.28571 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:10.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.057+0000 7ff5e8ff9640 1 -- 172.21.15.94:0/2039109020 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56496e71d890 con 0x56496eaa4350 2024-02-17T12:12:10.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.058+0000 7ff5e8ff9640 10 client.28571 handle_command_reply: tid=0 2024-02-17T12:12:10.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.058+0000 7ff60a032740 1 client.28571 shutdown 2024-02-17T12:12:10.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.058+0000 7ff60a032740 1 -- 172.21.15.94:0/2039109020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2a2b0 msgr2=0x56496ea3a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.058+0000 7ff60a032740 1 --2- 172.21.15.94:0/2039109020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2a2b0 0x56496ea3a6c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0f34c0 tx=0x7ff5fc0bc510 comp rx=0 tx=0).stop 2024-02-17T12:12:10.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.058+0000 7ff60a032740 1 -- 172.21.15.94:0/2039109020 shutdown_connections 2024-02-17T12:12:10.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.058+0000 7ff60a032740 1 --2- 172.21.15.94:0/2039109020 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56496eaa4350 0x56496eaa6740 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.058+0000 7ff60a032740 1 --2- 172.21.15.94:0/2039109020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56496ea337b0 0x56496ea386a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.058+0000 7ff60a032740 1 --2- 172.21.15.94:0/2039109020 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56496ea2abd0 0x56496ea3ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.062 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.058+0000 7ff60a032740 1 --2- 172.21.15.94:0/2039109020 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56496ea2a2b0 0x56496ea3a6c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.058+0000 7ff60a032740 1 -- 172.21.15.94:0/2039109020 >> 172.21.15.94:0/2039109020 conn(0x56496ea24990 msgr2=0x56496ea2ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.058+0000 7ff5e8ff9640 0 client.28571 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:10.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.059+0000 7ff60a032740 1 -- 172.21.15.94:0/2039109020 shutdown_connections 2024-02-17T12:12:10.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.059+0000 7ff60a032740 1 -- 172.21.15.94:0/2039109020 wait complete. 2024-02-17T12:12:10.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.059+0000 7ff60a032740 20 client.28571 trim_cache size 0 max 16384 2024-02-17T12:12:10.063 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:12:10.063 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:12:10.063 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:12:10.063 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:12:10.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.060+0000 7ff607381640 1 -- 172.21.15.94:0/3073648974 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff5ec0a22b0 msgr2=0x7ff5ec0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.060+0000 7ff607381640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff5ec0a22b0 0x7ff5ec0a4770 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8025000 tx=0x7ff5f8026000 comp rx=0 tx=0).stop 2024-02-17T12:12:10.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.060+0000 7ff607381640 1 -- 172.21.15.94:0/3073648974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6001009a0 msgr2=0x7ff60019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.060+0000 7ff607381640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6001009a0 0x7ff60019d6a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc009230 tx=0x7ff5fc02c650 comp rx=0 tx=0).stop 2024-02-17T12:12:10.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.060+0000 7ff607381640 1 -- 172.21.15.94:0/3073648974 shutdown_connections 2024-02-17T12:12:10.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.060+0000 7ff607381640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff5ec0a22b0 0x7ff5ec0a4770 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.060+0000 7ff607381640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff600197810 0x7ff600197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.061+0000 7ff607381640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff600105550 0x7ff60019dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.061+0000 7ff607381640 1 --2- 172.21.15.94:0/3073648974 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff6001009a0 0x7ff60019d6a0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.061+0000 7ff607381640 1 -- 172.21.15.94:0/3073648974 >> 172.21.15.94:0/3073648974 conn(0x7ff6000765b0 msgr2=0x7ff6000ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.061+0000 7ff607381640 1 -- 172.21.15.94:0/3073648974 shutdown_connections 2024-02-17T12:12:10.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.061+0000 7ff607381640 1 -- 172.21.15.94:0/3073648974 wait complete. 2024-02-17T12:12:10.079 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:12:10.080 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:12:10.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.229+0000 7f91d64c0640 1 -- 172.21.15.94:0/2089741767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d010d260 msgr2=0x7f91d010d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.229+0000 7f91d64c0640 1 --2- 172.21.15.94:0/2089741767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d010d260 0x7f91d010d6c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f91c401a910 tx=0x7f91c403cca0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.230+0000 7f91d64c0640 1 -- 172.21.15.94:0/2089741767 shutdown_connections 2024-02-17T12:12:10.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.230+0000 7f91d64c0640 1 --2- 172.21.15.94:0/2089741767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91d010dce0 0x7f91d011c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.230+0000 7f91d64c0640 1 --2- 172.21.15.94:0/2089741767 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d010d260 0x7f91d010d6c0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.230+0000 7f91d64c0640 1 --2- 172.21.15.94:0/2089741767 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f91d010f6e0 0x7f91d010fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.230+0000 7f91d64c0640 1 -- 172.21.15.94:0/2089741767 >> 172.21.15.94:0/2089741767 conn(0x7f91d006c080 msgr2=0x7f91d006c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.234 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.232+0000 7f91d64c0640 1 -- 172.21.15.94:0/2089741767 shutdown_connections 2024-02-17T12:12:10.236 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.234+0000 7f91d64c0640 1 -- 172.21.15.94:0/2089741767 wait complete. 2024-02-17T12:12:10.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.235+0000 7f91d64c0640 1 Processor -- start 2024-02-17T12:12:10.237 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.235+0000 7f91d64c0640 1 -- start start 2024-02-17T12:12:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.235+0000 7f91d64c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d010d260 0x7f91d011c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.235+0000 7f91d64c0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91d010dce0 0x7f91d01185c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91d64c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f91d010f6e0 0x7f91d0118b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91d64c0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0119040 con 0x7f91d010d260 2024-02-17T12:12:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91d64c0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d01191b0 con 0x7f91d010f6e0 2024-02-17T12:12:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91d64c0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d0119370 con 0x7f91d010dce0 2024-02-17T12:12:10.239 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d010d260 0x7f91d011c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91d010dce0 0x7f91d01185c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91cffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d010d260 0x7f91d011c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33314/0 (socket says 172.21.15.94:33314) 2024-02-17T12:12:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91cffff640 1 -- 172.21.15.94:0/1678779547 learned_addr learned my addr 172.21.15.94:0/1678779547 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91d4a5d640 1 --2- 172.21.15.94:0/1678779547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f91d010f6e0 0x7f91d0118b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91cffff640 1 -- 172.21.15.94:0/1678779547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91d010dce0 msgr2=0x7f91d01185c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91cffff640 1 --2- 172.21.15.94:0/1678779547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91d010dce0 0x7f91d01185c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91cffff640 1 -- 172.21.15.94:0/1678779547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f91d010f6e0 msgr2=0x7f91d0118b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91cffff640 1 --2- 172.21.15.94:0/1678779547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f91d010f6e0 0x7f91d0118b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.240 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91cffff640 1 -- 172.21.15.94:0/1678779547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c401a570 con 0x7f91d010d260 2024-02-17T12:12:10.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.236+0000 7f91cffff640 1 --2- 172.21.15.94:0/1678779547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d010d260 0x7f91d011c530 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f91c800b450 tx=0x7f91c800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.237+0000 7f91cd7fa640 1 -- 172.21.15.94:0/1678779547 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8011890 con 0x7f91d010d260 2024-02-17T12:12:10.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.237+0000 7f91cd7fa640 1 -- 172.21.15.94:0/1678779547 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91c80103e0 con 0x7f91d010d260 2024-02-17T12:12:10.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.237+0000 7f91d64c0640 1 -- 172.21.15.94:0/1678779547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d01aa430 con 0x7f91d010d260 2024-02-17T12:12:10.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.237+0000 7f91cd7fa640 1 -- 172.21.15.94:0/1678779547 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800f5a0 con 0x7f91d010d260 2024-02-17T12:12:10.241 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.237+0000 7f91d64c0640 1 -- 172.21.15.94:0/1678779547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d01aa940 con 0x7f91d010d260 2024-02-17T12:12:10.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.238+0000 7f91d64c0640 1 -- 172.21.15.94:0/1678779547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d006d320 con 0x7f91d010d260 2024-02-17T12:12:10.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.239+0000 7f91cd7fa640 1 -- 172.21.15.94:0/1678779547 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f91c8010550 con 0x7f91d010d260 2024-02-17T12:12:10.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.240+0000 7f91cd7fa640 1 --2- 172.21.15.94:0/1678779547 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f91b80a22b0 0x7f91b80a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.240+0000 7f91cf7fe640 1 --2- 172.21.15.94:0/1678779547 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f91b80a22b0 0x7f91b80a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.242 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.240+0000 7f91cd7fa640 1 -- 172.21.15.94:0/1678779547 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f91c80cdd50 con 0x7f91d010d260 2024-02-17T12:12:10.243 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.240+0000 7f91cf7fe640 1 --2- 172.21.15.94:0/1678779547 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f91b80a22b0 0x7f91b80a4770 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f91c403d230 tx=0x7f91c403d1c0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 830.0 s, 1141.6 tps, lat 96.875 ms stddev 202.889 2024-02-17T12:12:10.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.245+0000 7f91cd7fa640 1 -- 172.21.15.94:0/1678779547 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f91c80d1030 con 0x7f91d010d260 2024-02-17T12:12:10.563 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.561+0000 7f91d64c0640 1 -- 172.21.15.94:0/1678779547 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91d010c8b0 con 0x7f91d010d260 2024-02-17T12:12:10.564 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.562+0000 7f91cd7fa640 1 -- 172.21.15.94:0/1678779547 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7f91c8096a80 con 0x7f91d010d260 2024-02-17T12:12:10.564 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:12:10.564 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:12:10.564 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:12:10.566 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.564+0000 7f91d64c0640 1 -- 172.21.15.94:0/1678779547 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f91b80a22b0 msgr2=0x7f91b80a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.564+0000 7f91d64c0640 1 --2- 172.21.15.94:0/1678779547 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f91b80a22b0 0x7f91b80a4770 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f91c403d230 tx=0x7f91c403d1c0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.564+0000 7f91d64c0640 1 -- 172.21.15.94:0/1678779547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d010d260 msgr2=0x7f91d011c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.564+0000 7f91d64c0640 1 --2- 172.21.15.94:0/1678779547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d010d260 0x7f91d011c530 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f91c800b450 tx=0x7f91c800b910 comp rx=0 tx=0).stop 2024-02-17T12:12:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.564+0000 7f91d64c0640 1 -- 172.21.15.94:0/1678779547 shutdown_connections 2024-02-17T12:12:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.564+0000 7f91d64c0640 1 --2- 172.21.15.94:0/1678779547 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f91b80a22b0 0x7f91b80a4770 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.564+0000 7f91d64c0640 1 --2- 172.21.15.94:0/1678779547 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f91d010f6e0 0x7f91d0118b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.564+0000 7f91d64c0640 1 --2- 172.21.15.94:0/1678779547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91d010dce0 0x7f91d01185c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.564+0000 7f91d64c0640 1 --2- 172.21.15.94:0/1678779547 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f91d010d260 0x7f91d011c530 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.564+0000 7f91d64c0640 1 -- 172.21.15.94:0/1678779547 >> 172.21.15.94:0/1678779547 conn(0x7f91d006c080 msgr2=0x7f91d00706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.564+0000 7f91d64c0640 1 -- 172.21.15.94:0/1678779547 shutdown_connections 2024-02-17T12:12:10.567 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.564+0000 7f91d64c0640 1 -- 172.21.15.94:0/1678779547 wait complete. 2024-02-17T12:12:10.580 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:12:10.580 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:12:10.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.715+0000 7fa52fe2e640 1 -- 172.21.15.94:0/1443273978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5281009e0 msgr2=0x7fa52810cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.715+0000 7fa52fe2e640 1 --2- 172.21.15.94:0/1443273978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5281009e0 0x7fa52810cea0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa520009be0 tx=0x7fa52002bf80 comp rx=0 tx=0).stop 2024-02-17T12:12:10.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.715+0000 7fa52fe2e640 1 -- 172.21.15.94:0/1443273978 shutdown_connections 2024-02-17T12:12:10.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.715+0000 7fa52fe2e640 1 --2- 172.21.15.94:0/1443273978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5281009e0 0x7fa52810cea0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.715+0000 7fa52fe2e640 1 --2- 172.21.15.94:0/1443273978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa528100040 0x7fa5281004a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.715+0000 7fa52fe2e640 1 --2- 172.21.15.94:0/1443273978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa528106100 0x7fa5281064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.715+0000 7fa52fe2e640 1 -- 172.21.15.94:0/1443273978 >> 172.21.15.94:0/1443273978 conn(0x7fa528077110 msgr2=0x7fa528077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.715+0000 7fa52fe2e640 1 -- 172.21.15.94:0/1443273978 shutdown_connections 2024-02-17T12:12:10.718 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.715+0000 7fa52fe2e640 1 -- 172.21.15.94:0/1443273978 wait complete. 2024-02-17T12:12:10.719 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.716+0000 7fa52fe2e640 1 Processor -- start 2024-02-17T12:12:10.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.717+0000 7fa52fe2e640 1 -- start start 2024-02-17T12:12:10.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52fe2e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa528100040 0x7fa52810f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52fe2e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5281009e0 0x7fa52810d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52fe2e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa528106100 0x7fa52810dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52fe2e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52810e3c0 con 0x7fa5281009e0 2024-02-17T12:12:10.721 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52fe2e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5281aa070 con 0x7fa528106100 2024-02-17T12:12:10.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52fe2e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5281aa1b0 con 0x7fa528100040 2024-02-17T12:12:10.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52d3c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5281009e0 0x7fa52810d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52d3c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5281009e0 0x7fa52810d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33332/0 (socket says 172.21.15.94:33332) 2024-02-17T12:12:10.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52d3c9640 1 -- 172.21.15.94:0/3780128554 learned_addr learned my addr 172.21.15.94:0/3780128554 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:10.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52dbca640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa528100040 0x7fa52810f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52e3cb640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa528106100 0x7fa52810dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52d3c9640 1 -- 172.21.15.94:0/3780128554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa528100040 msgr2=0x7fa52810f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa528100040 0x7fa52810f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52d3c9640 1 -- 172.21.15.94:0/3780128554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa528106100 msgr2=0x7fa52810dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa528106100 0x7fa52810dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52d3c9640 1 -- 172.21.15.94:0/3780128554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa518009560 con 0x7fa5281009e0 2024-02-17T12:12:10.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.718+0000 7fa52e3cb640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa528106100 0x7fa52810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:10.722 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.719+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5281009e0 0x7fa52810d5f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa51800e9c0 tx=0x7fa51800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.719+0000 7fa51effd640 1 -- 172.21.15.94:0/3780128554 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51800cde0 con 0x7fa5281009e0 2024-02-17T12:12:10.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.719+0000 7fa51effd640 1 -- 172.21.15.94:0/3780128554 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5180035f0 con 0x7fa5281009e0 2024-02-17T12:12:10.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.719+0000 7fa52fe2e640 1 -- 172.21.15.94:0/3780128554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa520009840 con 0x7fa5281009e0 2024-02-17T12:12:10.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.719+0000 7fa51effd640 1 -- 172.21.15.94:0/3780128554 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518010670 con 0x7fa5281009e0 2024-02-17T12:12:10.723 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.719+0000 7fa52fe2e640 1 -- 172.21.15.94:0/3780128554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5281aa7c0 con 0x7fa5281009e0 2024-02-17T12:12:10.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.721+0000 7fa51effd640 1 -- 172.21.15.94:0/3780128554 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa5180107d0 con 0x7fa5281009e0 2024-02-17T12:12:10.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.721+0000 7fa532a02740 1 Processor -- start 2024-02-17T12:12:10.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa532a02740 1 -- start start 2024-02-17T12:12:10.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa532a02740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d3d250 0x55adf4d3d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa532a02740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d46210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.725 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa532a02740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d46750 0x55adf4d48b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa532a02740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf4cf8960 con 0x55adf4d46750 2024-02-17T12:12:10.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa532a02740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf4cac310 con 0x55adf4d3db70 2024-02-17T12:12:10.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa532a02740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf4ca9840 con 0x55adf4d3d250 2024-02-17T12:12:10.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa52d3c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d46750 0x55adf4d48b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa52dbca640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d46210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa52dbca640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d46210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36844/0 (socket says 172.21.15.94:36844) 2024-02-17T12:12:10.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa52dbca640 1 -- 172.21.15.94:0/3039514010 learned_addr learned my addr 172.21.15.94:0/3039514010 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:10.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa52e3cb640 1 --2- 172.21.15.94:0/3039514010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d3d250 0x55adf4d3d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa51effd640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5140a2230 0x7fa5140a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.722+0000 7fa51effd640 1 -- 172.21.15.94:0/3780128554 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa518014070 con 0x7fa5281009e0 2024-02-17T12:12:10.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.723+0000 7fa52d3c9640 1 -- 172.21.15.94:0/3039514010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d3d250 msgr2=0x55adf4d3d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.723+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/3039514010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d3d250 0x55adf4d3d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.723+0000 7fa52d3c9640 1 -- 172.21.15.94:0/3039514010 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 msgr2=0x55adf4d46210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.723+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/3039514010 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d46210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.723+0000 7fa52d3c9640 1 -- 172.21.15.94:0/3039514010 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adf4c67060 con 0x55adf4d46750 2024-02-17T12:12:10.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.723+0000 7fa52e3cb640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5140a2230 0x7fa5140a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.723+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/3039514010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d46750 0x55adf4d48b40 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa5180cbc80 tx=0x7fa518003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7197ca7d82807d8a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.723+0000 7fa513fff640 1 -- 172.21.15.94:0/3039514010 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5180a2180 con 0x55adf4d46750 2024-02-17T12:12:10.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.723+0000 7fa513fff640 1 -- 172.21.15.94:0/3039514010 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa51809b620 con 0x55adf4d46750 2024-02-17T12:12:10.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.723+0000 7fa513fff640 1 -- 172.21.15.94:0/3039514010 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5180992f0 con 0x55adf4d46750 2024-02-17T12:12:10.727 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.723+0000 7fa52e3cb640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5140a2230 0x7fa5140a46f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa520006010 tx=0x7fa52002bf80 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.724+0000 7fa532a02740 1 -- 172.21.15.94:0/3039514010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d46750 msgr2=0x55adf4d48b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.724+0000 7fa532a02740 1 --2- 172.21.15.94:0/3039514010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d46750 0x55adf4d48b40 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa5180cbc80 tx=0x7fa518003ed0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.724+0000 7fa532a02740 1 -- 172.21.15.94:0/3039514010 shutdown_connections 2024-02-17T12:12:10.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.724+0000 7fa532a02740 1 --2- 172.21.15.94:0/3039514010 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d46750 0x55adf4d48b40 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.731 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.724+0000 7fa532a02740 1 --2- 172.21.15.94:0/3039514010 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d46210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.724+0000 7fa532a02740 1 --2- 172.21.15.94:0/3039514010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d3d250 0x55adf4d3d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.724+0000 7fa532a02740 1 -- 172.21.15.94:0/3039514010 >> 172.21.15.94:0/3039514010 conn(0x55adf4d37980 msgr2=0x55adf4d37d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.724+0000 7fa532a02740 1 -- 172.21.15.94:0/3039514010 shutdown_connections 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.724+0000 7fa532a02740 1 -- 172.21.15.94:0/3039514010 wait complete. 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.724+0000 7fa532a02740 1 -- start start 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa532a02740 1 Processor -- start 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa532a02740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3d250 0x55adf4d43750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa532a02740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d43c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa532a02740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d46750 0x55adf4d441d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa532a02740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf4ca5020 con 0x55adf4d3d250 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa532a02740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf4978430 con 0x55adf4d3db70 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa532a02740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf4934e00 con 0x55adf4d46750 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa52dbca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3d250 0x55adf4d43750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa52dbca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3d250 0x55adf4d43750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33352/0 (socket says 172.21.15.94:33352) 2024-02-17T12:12:10.732 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa52dbca640 1 -- 172.21.15.94:0/663004480 learned_addr learned my addr 172.21.15.94:0/663004480 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa52e3cb640 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d46750 0x55adf4d441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d43c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa532a02740 10 client.0 _ll_register_callbacks cb 0x55adf4c4ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa52dbca640 1 -- 172.21.15.94:0/663004480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d46750 msgr2=0x55adf4d441d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa52dbca640 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d46750 0x55adf4d441d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa52dbca640 1 -- 172.21.15.94:0/663004480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 msgr2=0x55adf4d43c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa52dbca640 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d43c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa52dbca640 1 -- 172.21.15.94:0/663004480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5281aa7c0 con 0x55adf4d3d250 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d43c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa52dbca640 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3d250 0x55adf4d43750 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa52400ba30 tx=0x7fa52400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7779402694e2016c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.725+0000 7fa5117fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa5117fa640 1 -- 172.21.15.94:0/663004480 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524016070 con 0x55adf4d3d250 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa5117fa640 1 -- 172.21.15.94:0/663004480 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa524005020 con 0x55adf4d3d250 2024-02-17T12:12:10.733 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa532a02740 1 -- 172.21.15.94:0/663004480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55adf49a47e0 con 0x55adf4d3d250 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa5117fa640 1 -- 172.21.15.94:0/663004480 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5240051e0 con 0x55adf4d3d250 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa532a02740 10 client.19116 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa532a02740 1 -- 172.21.15.94:0/663004480 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55adf4ca36a0 con 0x55adf4d3d250 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa5117fa640 1 -- 172.21.15.94:0/663004480 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa524027080 con 0x55adf4d3d250 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa5117fa640 1 -- 172.21.15.94:0/663004480 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa52401b470 con 0x55adf4d3d250 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa532a02740 10 client.19116 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa532a02740 10 client.19116 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa532a02740 20 client.19116 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa532a02740 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55adf4d56ae0 0x55adf4d58f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa532a02740 4 client.19116 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa532a02740 1 -- 172.21.15.94:0/663004480 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adf49e1720 con 0x55adf4d56ae0 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.726+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55adf4d56ae0 0x55adf4d58f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa5117fa640 10 client.19116 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55adf4d56ae0 0x55adf4d58f40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd0d7ec1cdc75d50 server_cookie=31923bdca8e7d962 in_seq=0 out_seq=0 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa5117fa640 1 -- 172.21.15.94:0/663004480 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55adf49e1720 con 0x55adf4d56ae0 2024-02-17T12:12:10.734 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa5117fa640 10 client.19116 handle_command_reply: tid=0 2024-02-17T12:12:10.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa532a02740 1 client.19116 shutdown 2024-02-17T12:12:10.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa532a02740 1 -- 172.21.15.94:0/663004480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3d250 msgr2=0x55adf4d43750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa532a02740 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3d250 0x55adf4d43750 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa52400ba30 tx=0x7fa52400bef0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa532a02740 1 -- 172.21.15.94:0/663004480 shutdown_connections 2024-02-17T12:12:10.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa532a02740 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d46750 0x55adf4d441d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa532a02740 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55adf4d56ae0 0x55adf4d58f40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa532a02740 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d43c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa532a02740 1 --2- 172.21.15.94:0/663004480 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3d250 0x55adf4d43750 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa5117fa640 0 client.19116 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:12:10.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa532a02740 1 -- 172.21.15.94:0/663004480 >> 172.21.15.94:0/663004480 conn(0x55adf4d37980 msgr2=0x55adf4d396d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa532a02740 1 -- 172.21.15.94:0/663004480 shutdown_connections 2024-02-17T12:12:10.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.727+0000 7fa532a02740 1 -- 172.21.15.94:0/663004480 wait complete. 2024-02-17T12:12:10.735 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.728+0000 7fa532a02740 20 client.19116 trim_cache size 0 max 16384 2024-02-17T12:12:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa532a02740 1 Processor -- start 2024-02-17T12:12:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa532a02740 1 -- start start 2024-02-17T12:12:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa532a02740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d3d250 0x55adf4d45e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa532a02740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d43ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa532a02740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d46750 0x55adf4d409b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.746 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa532a02740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf4c2a130 con 0x55adf4d46750 2024-02-17T12:12:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa532a02740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf4d56890 con 0x55adf4d3db70 2024-02-17T12:12:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa532a02740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf4d332b0 con 0x55adf4d3d250 2024-02-17T12:12:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa52e3cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d46750 0x55adf4d409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa52e3cb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d46750 0x55adf4d409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33356/0 (socket says 172.21.15.94:33356) 2024-02-17T12:12:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa52e3cb640 1 -- 172.21.15.94:0/2466762835 learned_addr learned my addr 172.21.15.94:0/2466762835 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa52dbca640 1 --2- 172.21.15.94:0/2466762835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d3d250 0x55adf4d45e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/2466762835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d43ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa52e3cb640 1 -- 172.21.15.94:0/2466762835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d3d250 msgr2=0x55adf4d45e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa52e3cb640 1 --2- 172.21.15.94:0/2466762835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d3d250 0x55adf4d45e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa52e3cb640 1 -- 172.21.15.94:0/2466762835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 msgr2=0x55adf4d43ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa52e3cb640 1 --2- 172.21.15.94:0/2466762835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d43ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.743+0000 7fa52e3cb640 1 -- 172.21.15.94:0/2466762835 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adf4c3fc20 con 0x55adf4d46750 2024-02-17T12:12:10.747 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.744+0000 7fa52e3cb640 1 --2- 172.21.15.94:0/2466762835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d46750 0x55adf4d409b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa520035000 tx=0x7fa520005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec322605f03dd7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.744+0000 7fa5137fe640 1 -- 172.21.15.94:0/2466762835 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa520004880 con 0x55adf4d46750 2024-02-17T12:12:10.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.744+0000 7fa5137fe640 1 -- 172.21.15.94:0/2466762835 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa520034430 con 0x55adf4d46750 2024-02-17T12:12:10.748 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.744+0000 7fa5137fe640 1 -- 172.21.15.94:0/2466762835 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52004f450 con 0x55adf4d46750 2024-02-17T12:12:10.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.745+0000 7fa532a02740 1 -- 172.21.15.94:0/2466762835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d46750 msgr2=0x55adf4d409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.745+0000 7fa532a02740 1 --2- 172.21.15.94:0/2466762835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d46750 0x55adf4d409b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa520035000 tx=0x7fa520005e30 comp rx=0 tx=0).stop 2024-02-17T12:12:10.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.746+0000 7fa532a02740 1 -- 172.21.15.94:0/2466762835 shutdown_connections 2024-02-17T12:12:10.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.746+0000 7fa532a02740 1 --2- 172.21.15.94:0/2466762835 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d46750 0x55adf4d409b0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.753 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.746+0000 7fa532a02740 1 --2- 172.21.15.94:0/2466762835 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3db70 0x55adf4d43ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.746+0000 7fa532a02740 1 --2- 172.21.15.94:0/2466762835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d3d250 0x55adf4d45e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.746+0000 7fa532a02740 1 -- 172.21.15.94:0/2466762835 >> 172.21.15.94:0/2466762835 conn(0x55adf4d37980 msgr2=0x55adf4d5d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.746+0000 7fa532a02740 1 -- 172.21.15.94:0/2466762835 shutdown_connections 2024-02-17T12:12:10.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.746+0000 7fa532a02740 1 -- 172.21.15.94:0/2466762835 wait complete. 2024-02-17T12:12:10.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.746+0000 7fa532a02740 1 -- start start 2024-02-17T12:12:10.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.746+0000 7fa532a02740 1 Processor -- start 2024-02-17T12:12:10.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa532a02740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3d250 0x55adf4da63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa532a02740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3db70 0x55adf4da6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa532a02740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d46750 0x55adf4d4c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa532a02740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf4d4cea0 con 0x55adf4d3db70 2024-02-17T12:12:10.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa532a02740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf4d4d010 con 0x55adf4d3d250 2024-02-17T12:12:10.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa532a02740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adf4d4d1d0 con 0x55adf4d46750 2024-02-17T12:12:10.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa532a02740 10 client.0 _ll_register_callbacks cb 0x7fa5180971c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:10.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa52d3c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3db70 0x55adf4da6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa52d3c9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3db70 0x55adf4da6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33366/0 (socket says 172.21.15.94:33366) 2024-02-17T12:12:10.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa52d3c9640 1 -- 172.21.15.94:0/3022046465 learned_addr learned my addr 172.21.15.94:0/3022046465 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:10.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa52e3cb640 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d46750 0x55adf4d4c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa52dbca640 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3d250 0x55adf4da63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa52d3c9640 1 -- 172.21.15.94:0/3022046465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d46750 msgr2=0x55adf4d4c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d46750 0x55adf4d4c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa52d3c9640 1 -- 172.21.15.94:0/3022046465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3d250 msgr2=0x55adf4da63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3d250 0x55adf4da63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa52d3c9640 1 -- 172.21.15.94:0/3022046465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa520048040 con 0x55adf4d3db70 2024-02-17T12:12:10.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.747+0000 7fa52e3cb640 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d46750 0x55adf4d4c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:10.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.748+0000 7fa52d3c9640 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3db70 0x55adf4da6920 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa51809aeb0 tx=0x7fa5180092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5ecf11575f86f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.748+0000 7fa5117fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:10.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.748+0000 7fa5117fa640 1 -- 172.21.15.94:0/3022046465 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51809af20 con 0x55adf4d3db70 2024-02-17T12:12:10.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.748+0000 7fa5117fa640 1 -- 172.21.15.94:0/3022046465 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7fa5180b3040 con 0x55adf4d3db70 2024-02-17T12:12:10.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.748+0000 7fa532a02740 1 -- 172.21.15.94:0/3022046465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa51809b620 con 0x55adf4d3db70 2024-02-17T12:12:10.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.748+0000 7fa5117fa640 1 -- 172.21.15.94:0/3022046465 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa518099220 con 0x55adf4d3db70 2024-02-17T12:12:10.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.748+0000 7fa532a02740 10 client.19122 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:10.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.748+0000 7fa532a02740 1 -- 172.21.15.94:0/3022046465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55adf4c658c0 con 0x55adf4d3db70 2024-02-17T12:12:10.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.749+0000 7fa5117fa640 1 -- 172.21.15.94:0/3022046465 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa518003ad0 con 0x55adf4d3db70 2024-02-17T12:12:10.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.749+0000 7fa5117fa640 1 -- 172.21.15.94:0/3022046465 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa51801f060 con 0x55adf4d3db70 2024-02-17T12:12:10.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.749+0000 7fa532a02740 10 client.19122 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:10.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.749+0000 7fa532a02740 10 client.19122 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:12:10.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.749+0000 7fa532a02740 20 client.19122 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:10.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.749+0000 7fa532a02740 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55adf4daee30 0x55adf4db1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.749+0000 7fa532a02740 4 client.19122 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:12:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.749+0000 7fa532a02740 1 -- 172.21.15.94:0/3022046465 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fa51809b620 con 0x55adf4daee30 2024-02-17T12:12:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.749+0000 7fa52dbca640 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55adf4daee30 0x55adf4db1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.749+0000 7fa5117fa640 10 client.19122 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:12:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.749+0000 7fa52dbca640 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55adf4daee30 0x55adf4db1250 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f86df22592b5004d server_cookie=5adc2dd26f195a25 in_seq=0 out_seq=0 2024-02-17T12:12:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa5117fa640 1 -- 172.21.15.94:0/3022046465 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fa51809b620 con 0x55adf4daee30 2024-02-17T12:12:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa5117fa640 10 client.19122 handle_command_reply: tid=0 2024-02-17T12:12:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa532a02740 1 client.19122 shutdown 2024-02-17T12:12:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa532a02740 1 -- 172.21.15.94:0/3022046465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3db70 msgr2=0x55adf4da6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa532a02740 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3db70 0x55adf4da6920 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa51809aeb0 tx=0x7fa5180092f0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa532a02740 1 -- 172.21.15.94:0/3022046465 shutdown_connections 2024-02-17T12:12:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa532a02740 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55adf4d46750 0x55adf4d4c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.757 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa532a02740 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55adf4d3db70 0x55adf4da6920 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa532a02740 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55adf4daee30 0x55adf4db1250 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa532a02740 1 --2- 172.21.15.94:0/3022046465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55adf4d3d250 0x55adf4da63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa532a02740 1 -- 172.21.15.94:0/3022046465 >> 172.21.15.94:0/3022046465 conn(0x55adf4d37980 msgr2=0x55adf4d362f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa5117fa640 0 client.19122 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:12:10.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa532a02740 1 -- 172.21.15.94:0/3022046465 shutdown_connections 2024-02-17T12:12:10.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa532a02740 1 -- 172.21.15.94:0/3022046465 wait complete. 2024-02-17T12:12:10.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.750+0000 7fa532a02740 20 client.19122 trim_cache size 0 max 16384 2024-02-17T12:12:10.758 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:12:10.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.751+0000 7fa52fe2e640 1 -- 172.21.15.94:0/3780128554 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5140a2230 msgr2=0x7fa5140a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.751+0000 7fa52fe2e640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5140a2230 0x7fa5140a46f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa520006010 tx=0x7fa52002bf80 comp rx=0 tx=0).stop 2024-02-17T12:12:10.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.751+0000 7fa52fe2e640 1 -- 172.21.15.94:0/3780128554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5281009e0 msgr2=0x7fa52810d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.751+0000 7fa52fe2e640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5281009e0 0x7fa52810d5f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa51800e9c0 tx=0x7fa51800ee80 comp rx=0 tx=0).stop 2024-02-17T12:12:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.751+0000 7fa52fe2e640 1 -- 172.21.15.94:0/3780128554 shutdown_connections 2024-02-17T12:12:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.751+0000 7fa52fe2e640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5140a2230 0x7fa5140a46f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.751+0000 7fa52fe2e640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa528106100 0x7fa52810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.751+0000 7fa52fe2e640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5281009e0 0x7fa52810d5f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.751+0000 7fa52fe2e640 1 --2- 172.21.15.94:0/3780128554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa528100040 0x7fa52810f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.751+0000 7fa52fe2e640 1 -- 172.21.15.94:0/3780128554 >> 172.21.15.94:0/3780128554 conn(0x7fa528077110 msgr2=0x7fa5280fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.751+0000 7fa52fe2e640 1 -- 172.21.15.94:0/3780128554 shutdown_connections 2024-02-17T12:12:10.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.751+0000 7fa52fe2e640 1 -- 172.21.15.94:0/3780128554 wait complete. 2024-02-17T12:12:10.766 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:12:10.766 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:12:10.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:10 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1678779547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:12:10.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.922+0000 7f750f26f640 1 -- 172.21.15.94:0/2488534869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75080676e0 msgr2=0x7f750810c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.924 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.922+0000 7f750f26f640 1 --2- 172.21.15.94:0/2488534869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75080676e0 0x7f750810c270 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f750000f0a0 tx=0x7f750002ff40 comp rx=0 tx=0).stop 2024-02-17T12:12:10.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.925+0000 7f750f26f640 1 -- 172.21.15.94:0/2488534869 shutdown_connections 2024-02-17T12:12:10.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.925+0000 7f750f26f640 1 --2- 172.21.15.94:0/2488534869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75080676e0 0x7f750810c270 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.925+0000 7f750f26f640 1 --2- 172.21.15.94:0/2488534869 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f75080fff80 0x7f7508103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.925+0000 7f750f26f640 1 --2- 172.21.15.94:0/2488534869 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75080ff5d0 0x7f75080ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.927 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.925+0000 7f750f26f640 1 -- 172.21.15.94:0/2488534869 >> 172.21.15.94:0/2488534869 conn(0x7f75080761d0 msgr2=0x7f75080fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.926+0000 7f750f26f640 1 -- 172.21.15.94:0/2488534869 shutdown_connections 2024-02-17T12:12:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.929+0000 7f750f26f640 1 -- 172.21.15.94:0/2488534869 wait complete. 2024-02-17T12:12:10.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.930+0000 7f750f26f640 1 Processor -- start 2024-02-17T12:12:10.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.930+0000 7f750f26f640 1 -- start start 2024-02-17T12:12:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.930+0000 7f750f26f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75080676e0 0x7f7508072b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750f26f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75080ff5d0 0x7f750806cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750f26f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f750806d100 0x7f750806d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750f26f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75081b2d30 con 0x7f750806d100 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750f26f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75081b2e70 con 0x7f75080676e0 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750d00b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75080676e0 0x7f7508072b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750f26f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75081b2fb0 con 0x7f75080ff5d0 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750d00b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75080676e0 0x7f7508072b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36872/0 (socket says 172.21.15.94:36872) 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750d00b640 1 -- 172.21.15.94:0/2733521838 learned_addr learned my addr 172.21.15.94:0/2733521838 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750c80a640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75080ff5d0 0x7f750806cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750d80c640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f750806d100 0x7f750806d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750d00b640 1 -- 172.21.15.94:0/2733521838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75080ff5d0 msgr2=0x7f750806cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750d00b640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75080ff5d0 0x7f750806cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750d00b640 1 -- 172.21.15.94:0/2733521838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f750806d100 msgr2=0x7f750806d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750d00b640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f750806d100 0x7f750806d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750d00b640 1 -- 172.21.15.94:0/2733521838 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750000dd10 con 0x7f75080676e0 2024-02-17T12:12:10.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750d80c640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f750806d100 0x7f750806d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750c80a640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75080ff5d0 0x7f750806cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.931+0000 7f750d00b640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75080676e0 0x7f7508072b00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f750400b480 tx=0x7f750400b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.933+0000 7f74fe7fc640 1 -- 172.21.15.94:0/2733521838 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504010870 con 0x7f75080676e0 2024-02-17T12:12:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.933+0000 7f750f26f640 1 -- 172.21.15.94:0/2733521838 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75081b3260 con 0x7f75080676e0 2024-02-17T12:12:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.933+0000 7f750f26f640 1 -- 172.21.15.94:0/2733521838 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75081b3770 con 0x7f75080676e0 2024-02-17T12:12:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.934+0000 7f74fe7fc640 1 -- 172.21.15.94:0/2733521838 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f750400f3c0 con 0x7f75080676e0 2024-02-17T12:12:10.936 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.934+0000 7f74fe7fc640 1 -- 172.21.15.94:0/2733521838 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75040175a0 con 0x7f75080676e0 2024-02-17T12:12:10.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:10 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1678779547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:12:10.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.937+0000 7f74fe7fc640 1 -- 172.21.15.94:0/2733521838 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f750400f530 con 0x7f75080676e0 2024-02-17T12:12:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.938+0000 7f7511e41740 1 Processor -- start 2024-02-17T12:12:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.938+0000 7f74fe7fc640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f74f40a2370 0x7f74f40a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.938+0000 7f7511e41740 1 -- start start 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.938+0000 7f750d80c640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f74f40a2370 0x7f74f40a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.938+0000 7f74fe7fc640 1 -- 172.21.15.94:0/2733521838 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f75040cd8c0 con 0x7f75080676e0 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.938+0000 7f7511e41740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f548250 0x56390f548630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f7511e41740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548b70 0x56390f551210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f7511e41740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f551750 0x56390f553b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f7511e41740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56390f503960 con 0x56390f548b70 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f7511e41740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56390f4b7310 con 0x56390f548250 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f7511e41740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56390f4b4840 con 0x56390f551750 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f750d80c640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f74f40a2370 0x7f74f40a4830 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f75000304d0 tx=0x7f7500032040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f750c80a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548b70 0x56390f551210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f750c80a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548b70 0x56390f551210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33378/0 (socket says 172.21.15.94:33378) 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f750c80a640 1 -- 172.21.15.94:0/932301608 learned_addr learned my addr 172.21.15.94:0/932301608 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f750d00b640 1 --2- 172.21.15.94:0/932301608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f548250 0x56390f548630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f750d80c640 1 --2- 172.21.15.94:0/932301608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f551750 0x56390f553b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f750c80a640 1 -- 172.21.15.94:0/932301608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f551750 msgr2=0x56390f553b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f750c80a640 1 --2- 172.21.15.94:0/932301608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f551750 0x56390f553b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f750c80a640 1 -- 172.21.15.94:0/932301608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f548250 msgr2=0x56390f548630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f750c80a640 1 --2- 172.21.15.94:0/932301608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f548250 0x56390f548630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f750c80a640 1 -- 172.21.15.94:0/932301608 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56390f472060 con 0x56390f548b70 2024-02-17T12:12:10.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f750c80a640 1 --2- 172.21.15.94:0/932301608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548b70 0x56390f551210 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f74f8010400 tx=0x7f74f80108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ae12e3d5fd93e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.939+0000 7f74f37fe640 1 -- 172.21.15.94:0/932301608 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f8011820 con 0x56390f548b70 2024-02-17T12:12:10.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.940+0000 7f74f37fe640 1 -- 172.21.15.94:0/932301608 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f74f8015070 con 0x56390f548b70 2024-02-17T12:12:10.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.940+0000 7f74f37fe640 1 -- 172.21.15.94:0/932301608 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74f801ce30 con 0x56390f548b70 2024-02-17T12:12:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.940+0000 7f7511e41740 1 -- 172.21.15.94:0/932301608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548b70 msgr2=0x56390f551210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.940+0000 7f7511e41740 1 --2- 172.21.15.94:0/932301608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548b70 0x56390f551210 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f74f8010400 tx=0x7f74f80108c0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.941+0000 7f7511e41740 1 -- 172.21.15.94:0/932301608 shutdown_connections 2024-02-17T12:12:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.941+0000 7f7511e41740 1 --2- 172.21.15.94:0/932301608 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f551750 0x56390f553b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.941+0000 7f7511e41740 1 --2- 172.21.15.94:0/932301608 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548b70 0x56390f551210 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.945 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.941+0000 7f7511e41740 1 --2- 172.21.15.94:0/932301608 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f548250 0x56390f548630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.941+0000 7f7511e41740 1 -- 172.21.15.94:0/932301608 >> 172.21.15.94:0/932301608 conn(0x56390f542980 msgr2=0x56390f542d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.941+0000 7f7511e41740 1 -- 172.21.15.94:0/932301608 shutdown_connections 2024-02-17T12:12:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.941+0000 7f7511e41740 1 -- 172.21.15.94:0/932301608 wait complete. 2024-02-17T12:12:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.941+0000 7f7511e41740 1 -- start start 2024-02-17T12:12:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.941+0000 7f7511e41740 1 Processor -- start 2024-02-17T12:12:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.942+0000 7f7511e41740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548250 0x56390f54e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.942+0000 7f7511e41740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548b70 0x56390f54ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.942+0000 7f7511e41740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 0x56390f54f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.942+0000 7f750d00b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548250 0x56390f54e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.942+0000 7f750d00b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548250 0x56390f54e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44836/0 (socket says 172.21.15.94:44836) 2024-02-17T12:12:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.942+0000 7f7511e41740 1 -- 172.21.15.94:0/2877057610 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56390f4b0020 con 0x56390f548b70 2024-02-17T12:12:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.942+0000 7f7511e41740 1 -- 172.21.15.94:0/2877057610 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56390f183430 con 0x56390f551750 2024-02-17T12:12:10.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.942+0000 7f7511e41740 1 -- 172.21.15.94:0/2877057610 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56390f13fe00 con 0x56390f548250 2024-02-17T12:12:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.942+0000 7f750d00b640 1 -- 172.21.15.94:0/2877057610 learned_addr learned my addr 172.21.15.94:0/2877057610 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.943+0000 7f750d80c640 1 --2- 172.21.15.94:0/2877057610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 0x56390f54f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.943+0000 7f750c80a640 1 --2- 172.21.15.94:0/2877057610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548b70 0x56390f54ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.943+0000 7f7511e41740 10 client.0 _ll_register_callbacks cb 0x56390f457cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.943+0000 7f750d80c640 1 -- 172.21.15.94:0/2877057610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548250 msgr2=0x56390f54e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.943+0000 7f750d80c640 1 --2- 172.21.15.94:0/2877057610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548250 0x56390f54e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.943+0000 7f750d80c640 1 -- 172.21.15.94:0/2877057610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548b70 msgr2=0x56390f54ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.943+0000 7f750d80c640 1 --2- 172.21.15.94:0/2877057610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548b70 0x56390f54ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.943+0000 7f750d80c640 1 -- 172.21.15.94:0/2877057610 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f800bc50 con 0x56390f551750 2024-02-17T12:12:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.944+0000 7f750d80c640 1 --2- 172.21.15.94:0/2877057610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 0x56390f54f180 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7500006010 tx=0x7f750000d9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18fee335a7f2aae7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.944+0000 7f74f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:12:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.945+0000 7f74f0ff9640 1 -- 172.21.15.94:0/2877057610 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500004040 con 0x56390f551750 2024-02-17T12:12:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.945+0000 7f74f0ff9640 1 -- 172.21.15.94:0/2877057610 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f75000041a0 con 0x56390f551750 2024-02-17T12:12:10.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.945+0000 7f7511e41740 1 -- 172.21.15.94:0/2877057610 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56390f1af7e0 con 0x56390f551750 2024-02-17T12:12:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.946+0000 7f74f0ff9640 1 -- 172.21.15.94:0/2877057610 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7500004360 con 0x56390f551750 2024-02-17T12:12:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.946+0000 7f7511e41740 10 client.28601 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.946+0000 7f7511e41740 1 -- 172.21.15.94:0/2877057610 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56390f4ae6a0 con 0x56390f551750 2024-02-17T12:12:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.946+0000 7f74f0ff9640 1 -- 172.21.15.94:0/2877057610 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56390f1af7e0 con 0x56390f551750 2024-02-17T12:12:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.946+0000 7f74f0ff9640 1 -- 172.21.15.94:0/2877057610 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f750000b040 con 0x56390f551750 2024-02-17T12:12:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.947+0000 7f7511e41740 10 client.28601 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.947+0000 7f7511e41740 10 client.28601 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:12:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.947+0000 7f7511e41740 20 client.28601 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.947+0000 7f7511e41740 1 --2- 172.21.15.94:0/2877057610 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56390f5619c0 0x56390f563de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.947+0000 7f7511e41740 4 client.28601 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.947+0000 7f7511e41740 1 -- 172.21.15.94:0/2877057610 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56390f1ec720 con 0x56390f5619c0 2024-02-17T12:12:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.947+0000 7f750d00b640 1 --2- 172.21.15.94:0/2877057610 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56390f5619c0 0x56390f563de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.947+0000 7f750d00b640 1 --2- 172.21.15.94:0/2877057610 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56390f5619c0 0x56390f563de0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c57b72aac15fbfd7 server_cookie=8153a55c2fce30cb in_seq=0 out_seq=0 2024-02-17T12:12:10.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.947+0000 7f74f0ff9640 10 client.28601 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:12:10.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.948+0000 7f74f0ff9640 1 -- 172.21.15.94:0/2877057610 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56390f1ec720 con 0x56390f5619c0 2024-02-17T12:12:10.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.948+0000 7f74f0ff9640 10 client.28601 handle_command_reply: tid=0 2024-02-17T12:12:10.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.948+0000 7f7511e41740 1 client.28601 shutdown 2024-02-17T12:12:10.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.949+0000 7f7511e41740 1 -- 172.21.15.94:0/2877057610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 msgr2=0x56390f54f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.949+0000 7f7511e41740 1 --2- 172.21.15.94:0/2877057610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 0x56390f54f180 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7500006010 tx=0x7f750000d9d0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.949+0000 7f7511e41740 1 -- 172.21.15.94:0/2877057610 shutdown_connections 2024-02-17T12:12:10.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.949+0000 7f7511e41740 1 --2- 172.21.15.94:0/2877057610 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56390f5619c0 0x56390f563de0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.952 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.949+0000 7f7511e41740 1 --2- 172.21.15.94:0/2877057610 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 0x56390f54f180 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.949+0000 7f7511e41740 1 --2- 172.21.15.94:0/2877057610 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548b70 0x56390f54ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.949+0000 7f7511e41740 1 --2- 172.21.15.94:0/2877057610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548250 0x56390f54e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.949+0000 7f7511e41740 1 -- 172.21.15.94:0/2877057610 >> 172.21.15.94:0/2877057610 conn(0x56390f542980 msgr2=0x56390f5446d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.949+0000 7f74f0ff9640 0 client.28601 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:12:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.949+0000 7f7511e41740 1 -- 172.21.15.94:0/2877057610 shutdown_connections 2024-02-17T12:12:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.949+0000 7f7511e41740 1 -- 172.21.15.94:0/2877057610 wait complete. 2024-02-17T12:12:10.953 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.949+0000 7f7511e41740 20 client.28601 trim_cache size 0 max 16384 2024-02-17T12:12:10.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:10 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1678779547' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:12:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.969+0000 7f7511e41740 1 Processor -- start 2024-02-17T12:12:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.969+0000 7f7511e41740 1 -- start start 2024-02-17T12:12:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.969+0000 7f7511e41740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 0x56390f550e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.969+0000 7f7511e41740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548b70 0x56390f54efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.969+0000 7f7511e41740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 0x56390f541a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.969+0000 7f7511e41740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56390f435130 con 0x56390f548250 2024-02-17T12:12:10.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.969+0000 7f7511e41740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56390f53e2b0 con 0x56390f551750 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.969+0000 7f7511e41740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56390f547eb0 con 0x56390f548b70 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.969+0000 7f750d00b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 0x56390f550e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.969+0000 7f750d00b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 0x56390f550e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33394/0 (socket says 172.21.15.94:33394) 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.969+0000 7f750d00b640 1 -- 172.21.15.94:0/2814187961 learned_addr learned my addr 172.21.15.94:0/2814187961 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.969+0000 7f750c80a640 1 --2- 172.21.15.94:0/2814187961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548b70 0x56390f54efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.969+0000 7f750d80c640 1 --2- 172.21.15.94:0/2814187961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 0x56390f541a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.970+0000 7f750d00b640 1 -- 172.21.15.94:0/2814187961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548b70 msgr2=0x56390f54efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.970+0000 7f750d00b640 1 --2- 172.21.15.94:0/2814187961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548b70 0x56390f54efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.970+0000 7f750d00b640 1 -- 172.21.15.94:0/2814187961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 msgr2=0x56390f541a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.970+0000 7f750d00b640 1 --2- 172.21.15.94:0/2814187961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 0x56390f541a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.970+0000 7f750d00b640 1 -- 172.21.15.94:0/2814187961 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56390f44ac20 con 0x56390f548250 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.970+0000 7f750d00b640 1 --2- 172.21.15.94:0/2814187961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 0x56390f550e30 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f750409a5f0 tx=0x7f750409a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3575a085a5257139 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.970+0000 7f74f2ffd640 1 -- 172.21.15.94:0/2814187961 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504098730 con 0x56390f548250 2024-02-17T12:12:10.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.970+0000 7f74f2ffd640 1 -- 172.21.15.94:0/2814187961 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f750409e170 con 0x56390f548250 2024-02-17T12:12:10.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.970+0000 7f74f2ffd640 1 -- 172.21.15.94:0/2814187961 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504096c40 con 0x56390f548250 2024-02-17T12:12:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.971+0000 7f7511e41740 1 -- 172.21.15.94:0/2814187961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 msgr2=0x56390f550e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.971+0000 7f7511e41740 1 --2- 172.21.15.94:0/2814187961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 0x56390f550e30 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f750409a5f0 tx=0x7f750409a2b0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.971+0000 7f7511e41740 1 -- 172.21.15.94:0/2814187961 shutdown_connections 2024-02-17T12:12:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.971+0000 7f7511e41740 1 --2- 172.21.15.94:0/2814187961 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 0x56390f541a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.971+0000 7f7511e41740 1 --2- 172.21.15.94:0/2814187961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548b70 0x56390f54efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.971+0000 7f7511e41740 1 --2- 172.21.15.94:0/2814187961 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 0x56390f550e30 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.971+0000 7f7511e41740 1 -- 172.21.15.94:0/2814187961 >> 172.21.15.94:0/2814187961 conn(0x56390f542980 msgr2=0x56390f5412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.971+0000 7f7511e41740 1 -- 172.21.15.94:0/2814187961 shutdown_connections 2024-02-17T12:12:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.971+0000 7f7511e41740 1 -- 172.21.15.94:0/2814187961 wait complete. 2024-02-17T12:12:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.972+0000 7f7511e41740 1 -- start start 2024-02-17T12:12:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.972+0000 7f7511e41740 1 Processor -- start 2024-02-17T12:12:10.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f7511e41740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 0x56390f566fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f7511e41740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548b70 0x56390f567510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f7511e41740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 0x56390f567a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f7511e41740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56390f568f70 con 0x56390f548250 2024-02-17T12:12:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f7511e41740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56390f5c8e10 con 0x56390f551750 2024-02-17T12:12:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f7511e41740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56390f5c8fd0 con 0x56390f548b70 2024-02-17T12:12:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f7511e41740 10 client.0 _ll_register_callbacks cb 0x7f74f800bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f750d00b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 0x56390f566fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f750d00b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 0x56390f566fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33406/0 (socket says 172.21.15.94:33406) 2024-02-17T12:12:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f750d00b640 1 -- 172.21.15.94:0/1460537582 learned_addr learned my addr 172.21.15.94:0/1460537582 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f750c80a640 1 --2- 172.21.15.94:0/1460537582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548b70 0x56390f567510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f750d80c640 1 --2- 172.21.15.94:0/1460537582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 0x56390f567a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.978 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f750d00b640 1 -- 172.21.15.94:0/1460537582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548b70 msgr2=0x56390f567510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f750d00b640 1 --2- 172.21.15.94:0/1460537582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548b70 0x56390f567510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f750d00b640 1 -- 172.21.15.94:0/1460537582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 msgr2=0x56390f567a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f750d00b640 1 --2- 172.21.15.94:0/1460537582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 0x56390f567a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f750d00b640 1 -- 172.21.15.94:0/1460537582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75081b3770 con 0x56390f548250 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f750d00b640 1 --2- 172.21.15.94:0/1460537582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 0x56390f566fd0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f750409a130 tx=0x7f750409a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33738a137f0e9c36 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f74f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.973+0000 7f74f0ff9640 1 -- 172.21.15.94:0/1460537582 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75040a2a90 con 0x56390f548250 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f74f0ff9640 1 -- 172.21.15.94:0/1460537582 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f75040cb1c0 con 0x56390f548250 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f7511e41740 1 -- 172.21.15.94:0/1460537582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74f8015070 con 0x56390f548250 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f74f0ff9640 1 -- 172.21.15.94:0/1460537582 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75040cb380 con 0x56390f548250 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f7511e41740 10 client.19140 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f7511e41740 1 -- 172.21.15.94:0/1460537582 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56390f4708c0 con 0x56390f548250 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f74f0ff9640 1 -- 172.21.15.94:0/1460537582 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7504099880 con 0x56390f548250 2024-02-17T12:12:10.979 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f74f0ff9640 1 -- 172.21.15.94:0/1460537582 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f75040b8a60 con 0x56390f548250 2024-02-17T12:12:10.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f7511e41740 10 client.19140 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:10.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f7511e41740 10 client.19140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:12:10.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f7511e41740 20 client.19140 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:10.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f7511e41740 1 --2- 172.21.15.94:0/1460537582 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56390f5ca310 0x56390f5cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:10.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f7511e41740 4 client.19140 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:12:10.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f7511e41740 1 -- 172.21.15.94:0/1460537582 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f74f8015070 con 0x56390f5ca310 2024-02-17T12:12:10.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.974+0000 7f750c80a640 1 --2- 172.21.15.94:0/1460537582 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56390f5ca310 0x56390f5cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.975+0000 7f750c80a640 1 --2- 172.21.15.94:0/1460537582 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56390f5ca310 0x56390f5cc700 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2205a046b6dcb3d server_cookie=c1df6a425df7d46e in_seq=0 out_seq=0 2024-02-17T12:12:10.981 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.976+0000 7f74f0ff9640 10 client.19140 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:12:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.976+0000 7f74f0ff9640 1 -- 172.21.15.94:0/1460537582 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f74f8015070 con 0x56390f5ca310 2024-02-17T12:12:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.976+0000 7f74f0ff9640 10 client.19140 handle_command_reply: tid=0 2024-02-17T12:12:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.976+0000 7f7511e41740 1 client.19140 shutdown 2024-02-17T12:12:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.976+0000 7f7511e41740 1 -- 172.21.15.94:0/1460537582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 msgr2=0x56390f566fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.976+0000 7f7511e41740 1 --2- 172.21.15.94:0/1460537582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 0x56390f566fd0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f750409a130 tx=0x7f750409a320 comp rx=0 tx=0).stop 2024-02-17T12:12:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.976+0000 7f7511e41740 1 -- 172.21.15.94:0/1460537582 shutdown_connections 2024-02-17T12:12:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.976+0000 7f7511e41740 1 --2- 172.21.15.94:0/1460537582 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x56390f5ca310 0x56390f5cc700 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.976+0000 7f7511e41740 1 --2- 172.21.15.94:0/1460537582 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56390f551750 0x56390f567a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.976+0000 7f7511e41740 1 --2- 172.21.15.94:0/1460537582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56390f548b70 0x56390f567510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.976+0000 7f7511e41740 1 --2- 172.21.15.94:0/1460537582 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56390f548250 0x56390f566fd0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.976+0000 7f7511e41740 1 -- 172.21.15.94:0/1460537582 >> 172.21.15.94:0/1460537582 conn(0x56390f542980 msgr2=0x56390f544640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.976+0000 7f74f0ff9640 0 client.19140 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:12:10.982 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f7511e41740 1 -- 172.21.15.94:0/1460537582 shutdown_connections 2024-02-17T12:12:10.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f7511e41740 1 -- 172.21.15.94:0/1460537582 wait complete. 2024-02-17T12:12:10.983 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f7511e41740 20 client.19140 trim_cache size 0 max 16384 2024-02-17T12:12:10.983 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:12:10.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f750f26f640 1 -- 172.21.15.94:0/2733521838 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f74f40a2370 msgr2=0x7f74f40a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f750f26f640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f74f40a2370 0x7f74f40a4830 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f75000304d0 tx=0x7f7500032040 comp rx=0 tx=0).stop 2024-02-17T12:12:10.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f750f26f640 1 -- 172.21.15.94:0/2733521838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75080676e0 msgr2=0x7f7508072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:10.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f750f26f640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75080676e0 0x7f7508072b00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f750400b480 tx=0x7f750400b940 comp rx=0 tx=0).stop 2024-02-17T12:12:10.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f750f26f640 1 -- 172.21.15.94:0/2733521838 shutdown_connections 2024-02-17T12:12:10.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f750f26f640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f74f40a2370 0x7f74f40a4830 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f750f26f640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f750806d100 0x7f750806d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f750f26f640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75080ff5d0 0x7f750806cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f750f26f640 1 --2- 172.21.15.94:0/2733521838 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f75080676e0 0x7f7508072b00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:10.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f750f26f640 1 -- 172.21.15.94:0/2733521838 >> 172.21.15.94:0/2733521838 conn(0x7f75080761d0 msgr2=0x7f7508102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:10.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f750f26f640 1 -- 172.21.15.94:0/2733521838 shutdown_connections 2024-02-17T12:12:10.985 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:10.977+0000 7f750f26f640 1 -- 172.21.15.94:0/2733521838 wait complete. 2024-02-17T12:12:10.992 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:12:10.993 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:12:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.138+0000 7ff751064640 1 -- 172.21.15.94:0/416119221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74c106120 msgr2=0x7ff74c106500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:11.140 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.138+0000 7ff751064640 1 --2- 172.21.15.94:0/416119221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74c106120 0x7ff74c106500 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff734009900 tx=0x7ff73402bc80 comp rx=0 tx=0).stop 2024-02-17T12:12:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.138+0000 7ff751064640 1 -- 172.21.15.94:0/416119221 shutdown_connections 2024-02-17T12:12:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.138+0000 7ff751064640 1 --2- 172.21.15.94:0/416119221 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff74c069ea0 0x7ff74c10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.138+0000 7ff751064640 1 --2- 172.21.15.94:0/416119221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74c069500 0x7ff74c069960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.138+0000 7ff751064640 1 --2- 172.21.15.94:0/416119221 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74c106120 0x7ff74c106500 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.138+0000 7ff751064640 1 -- 172.21.15.94:0/416119221 >> 172.21.15.94:0/416119221 conn(0x7ff74c077110 msgr2=0x7ff74c077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.138+0000 7ff751064640 1 -- 172.21.15.94:0/416119221 shutdown_connections 2024-02-17T12:12:11.141 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.138+0000 7ff751064640 1 -- 172.21.15.94:0/416119221 wait complete. 2024-02-17T12:12:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.139+0000 7ff751064640 1 Processor -- start 2024-02-17T12:12:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.139+0000 7ff751064640 1 -- start start 2024-02-17T12:12:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.139+0000 7ff751064640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff74c069500 0x7ff74c19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.139+0000 7ff751064640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74c069ea0 0x7ff74c19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.140+0000 7ff74ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff74c069500 0x7ff74c19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.140+0000 7ff74ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff74c069500 0x7ff74c19d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36910/0 (socket says 172.21.15.94:36910) 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.140+0000 7ff74ad9d640 1 -- 172.21.15.94:0/2694669660 learned_addr learned my addr 172.21.15.94:0/2694669660 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.140+0000 7ff74a59c640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74c069ea0 0x7ff74c19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.140+0000 7ff751064640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74c106120 0x7ff74c1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.140+0000 7ff751064640 1 -- 172.21.15.94:0/2694669660 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c197c30 con 0x7ff74c106120 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.140+0000 7ff751064640 1 -- 172.21.15.94:0/2694669660 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c197da0 con 0x7ff74c069500 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.140+0000 7ff751064640 1 -- 172.21.15.94:0/2694669660 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c197f60 con 0x7ff74c069ea0 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.140+0000 7ff74b59e640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74c106120 0x7ff74c1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.141+0000 7ff74ad9d640 1 -- 172.21.15.94:0/2694669660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74c069ea0 msgr2=0x7ff74c19d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.141+0000 7ff74ad9d640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74c069ea0 0x7ff74c19d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.141+0000 7ff74ad9d640 1 -- 172.21.15.94:0/2694669660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74c106120 msgr2=0x7ff74c1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.141+0000 7ff74ad9d640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74c106120 0x7ff74c1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.141+0000 7ff74ad9d640 1 -- 172.21.15.94:0/2694669660 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff734009560 con 0x7ff74c069500 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.141+0000 7ff74b59e640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74c106120 0x7ff74c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:12:11.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.141+0000 7ff74a59c640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74c069ea0 0x7ff74c19d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.141+0000 7ff74ad9d640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff74c069500 0x7ff74c19d3e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff734005940 tx=0x7ff73402c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.142+0000 7ff743fff640 1 -- 172.21.15.94:0/2694669660 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7340043d0 con 0x7ff74c069500 2024-02-17T12:12:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.142+0000 7ff743fff640 1 -- 172.21.15.94:0/2694669660 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7340052d0 con 0x7ff74c069500 2024-02-17T12:12:11.145 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.142+0000 7ff743fff640 1 -- 172.21.15.94:0/2694669660 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734003cc0 con 0x7ff74c069500 2024-02-17T12:12:11.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.142+0000 7ff751064640 1 -- 172.21.15.94:0/2694669660 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74c1981e0 con 0x7ff74c069500 2024-02-17T12:12:11.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.142+0000 7ff751064640 1 -- 172.21.15.94:0/2694669660 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c1a4480 con 0x7ff74c069500 2024-02-17T12:12:11.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.144+0000 7ff743fff640 1 -- 172.21.15.94:0/2694669660 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff734005440 con 0x7ff74c069500 2024-02-17T12:12:11.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.145+0000 7ff743fff640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7380a2380 0x7ff7380a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.145+0000 7ff743fff640 1 -- 172.21.15.94:0/2694669660 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff7340effc0 con 0x7ff74c069500 2024-02-17T12:12:11.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.145+0000 7ff753c37740 1 Processor -- start 2024-02-17T12:12:11.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.145+0000 7ff74a59c640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7380a2380 0x7ff7380a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.145+0000 7ff753c37740 1 -- start start 2024-02-17T12:12:11.148 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff753c37740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329060 0x5614bf329440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff753c37740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329a10 0x5614bf336620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff753c37740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf336b60 0x5614bf338f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff753c37740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bf2e4960 con 0x5614bf329060 2024-02-17T12:12:11.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff753c37740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bf298310 con 0x5614bf336b60 2024-02-17T12:12:11.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff753c37740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bf295840 con 0x5614bf329a10 2024-02-17T12:12:11.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff74a59c640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7380a2380 0x7ff7380a4840 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff74c077680 tx=0x7ff73c00b410 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:11.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff74ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329060 0x5614bf329440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff74ad9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329060 0x5614bf329440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33438/0 (socket says 172.21.15.94:33438) 2024-02-17T12:12:11.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff74ad9d640 1 -- 172.21.15.94:0/553185442 learned_addr learned my addr 172.21.15.94:0/553185442 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:11.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff74b59e640 1 --2- 172.21.15.94:0/553185442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf336b60 0x5614bf338f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff74a59c640 1 --2- 172.21.15.94:0/553185442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329a10 0x5614bf336620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff74ad9d640 1 -- 172.21.15.94:0/553185442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329a10 msgr2=0x5614bf336620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:11.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff74ad9d640 1 --2- 172.21.15.94:0/553185442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329a10 0x5614bf336620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff74ad9d640 1 -- 172.21.15.94:0/553185442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf336b60 msgr2=0x5614bf338f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:11.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff74ad9d640 1 --2- 172.21.15.94:0/553185442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf336b60 0x5614bf338f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff74ad9d640 1 -- 172.21.15.94:0/553185442 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614bf253060 con 0x5614bf329060 2024-02-17T12:12:11.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff74b59e640 1 --2- 172.21.15.94:0/553185442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf336b60 0x5614bf338f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:12:11.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff74ad9d640 1 --2- 172.21.15.94:0/553185442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329060 0x5614bf329440 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff7340b9420 tx=0x7ff734003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d9276383eeadf8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff7417fa640 1 -- 172.21.15.94:0/553185442 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734003e20 con 0x5614bf329060 2024-02-17T12:12:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.146+0000 7ff7417fa640 1 -- 172.21.15.94:0/553185442 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7340bca50 con 0x5614bf329060 2024-02-17T12:12:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.147+0000 7ff7417fa640 1 -- 172.21.15.94:0/553185442 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7340b90c0 con 0x5614bf329060 2024-02-17T12:12:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.147+0000 7ff753c37740 1 -- 172.21.15.94:0/553185442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329060 msgr2=0x5614bf329440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.147+0000 7ff753c37740 1 --2- 172.21.15.94:0/553185442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329060 0x5614bf329440 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff7340b9420 tx=0x7ff734003760 comp rx=0 tx=0).stop 2024-02-17T12:12:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.148+0000 7ff753c37740 1 -- 172.21.15.94:0/553185442 shutdown_connections 2024-02-17T12:12:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.148+0000 7ff753c37740 1 --2- 172.21.15.94:0/553185442 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf336b60 0x5614bf338f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.148+0000 7ff753c37740 1 --2- 172.21.15.94:0/553185442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329a10 0x5614bf336620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.148+0000 7ff753c37740 1 --2- 172.21.15.94:0/553185442 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329060 0x5614bf329440 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.148+0000 7ff753c37740 1 -- 172.21.15.94:0/553185442 >> 172.21.15.94:0/553185442 conn(0x5614bf323980 msgr2=0x5614bf323d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.148+0000 7ff753c37740 1 -- 172.21.15.94:0/553185442 shutdown_connections 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.148+0000 7ff753c37740 1 -- 172.21.15.94:0/553185442 wait complete. 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.148+0000 7ff753c37740 1 -- start start 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.148+0000 7ff753c37740 1 Processor -- start 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff753c37740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329060 0x5614bf333a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff753c37740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329a10 0x5614bf333fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff753c37740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf336b60 0x5614bf334510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff753c37740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bf291020 con 0x5614bf329a10 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff753c37740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bef64430 con 0x5614bf329060 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff753c37740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bef20e00 con 0x5614bf336b60 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff74a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329a10 0x5614bf333fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff74ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329060 0x5614bf333a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff74a59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329a10 0x5614bf333fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33440/0 (socket says 172.21.15.94:33440) 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff74ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329060 0x5614bf333a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36930/0 (socket says 172.21.15.94:36930) 2024-02-17T12:12:11.157 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff74a59c640 1 -- 172.21.15.94:0/3694190807 learned_addr learned my addr 172.21.15.94:0/3694190807 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff74b59e640 1 --2- 172.21.15.94:0/3694190807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf336b60 0x5614bf334510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff753c37740 10 client.0 _ll_register_callbacks cb 0x5614bf238cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff74a59c640 1 -- 172.21.15.94:0/3694190807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf336b60 msgr2=0x5614bf334510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff74a59c640 1 --2- 172.21.15.94:0/3694190807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf336b60 0x5614bf334510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff74a59c640 1 -- 172.21.15.94:0/3694190807 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329060 msgr2=0x5614bf333a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff74a59c640 1 --2- 172.21.15.94:0/3694190807 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329060 0x5614bf333a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff74a59c640 1 -- 172.21.15.94:0/3694190807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c1a4480 con 0x5614bf329a10 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff74a59c640 1 --2- 172.21.15.94:0/3694190807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329a10 0x5614bf333fd0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff73c011ae0 tx=0x7ff73c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0089803c87f5e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff7327fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.149+0000 7ff7327fc640 1 -- 172.21.15.94:0/3694190807 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c01ec40 con 0x5614bf329a10 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff753c37740 1 -- 172.21.15.94:0/3694190807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614bef907e0 con 0x5614bf329a10 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff7327fc640 1 -- 172.21.15.94:0/3694190807 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff73c017070 con 0x5614bf329a10 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff7327fc640 1 -- 172.21.15.94:0/3694190807 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff73c01eda0 con 0x5614bf329a10 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff753c37740 10 client.19152 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:11.158 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff753c37740 1 -- 172.21.15.94:0/3694190807 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5614bf28f6a0 con 0x5614bf329a10 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff7327fc640 1 -- 172.21.15.94:0/3694190807 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5614bef907e0 con 0x5614bf329a10 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff7327fc640 1 -- 172.21.15.94:0/3694190807 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff73c01a040 con 0x5614bf329a10 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff753c37740 10 client.19152 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff753c37740 10 client.19152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff753c37740 20 client.19152 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff753c37740 1 --2- 172.21.15.94:0/3694190807 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614bf346d40 0x5614bf3491a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff753c37740 4 client.19152 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff753c37740 1 -- 172.21.15.94:0/3694190807 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614befcd720 con 0x5614bf346d40 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.150+0000 7ff74ad9d640 1 --2- 172.21.15.94:0/3694190807 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614bf346d40 0x5614bf3491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.151+0000 7ff7327fc640 10 client.19152 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.151+0000 7ff74ad9d640 1 --2- 172.21.15.94:0/3694190807 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614bf346d40 0x5614bf3491a0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e90e353da370978 server_cookie=dcb754f39d7add6c in_seq=0 out_seq=0 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.151+0000 7ff7327fc640 1 -- 172.21.15.94:0/3694190807 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5614befcd720 con 0x5614bf346d40 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.151+0000 7ff7327fc640 10 client.19152 handle_command_reply: tid=0 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.151+0000 7ff753c37740 1 client.19152 shutdown 2024-02-17T12:12:11.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.152+0000 7ff753c37740 1 -- 172.21.15.94:0/3694190807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329a10 msgr2=0x5614bf333fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.152+0000 7ff753c37740 1 --2- 172.21.15.94:0/3694190807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329a10 0x5614bf333fd0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7ff73c011ae0 tx=0x7ff73c004730 comp rx=0 tx=0).stop 2024-02-17T12:12:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.152+0000 7ff753c37740 1 -- 172.21.15.94:0/3694190807 shutdown_connections 2024-02-17T12:12:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.152+0000 7ff753c37740 1 --2- 172.21.15.94:0/3694190807 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614bf346d40 0x5614bf3491a0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.152+0000 7ff753c37740 1 --2- 172.21.15.94:0/3694190807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf336b60 0x5614bf334510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.152+0000 7ff753c37740 1 --2- 172.21.15.94:0/3694190807 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf329a10 0x5614bf333fd0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.152+0000 7ff753c37740 1 --2- 172.21.15.94:0/3694190807 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329060 0x5614bf333a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.152+0000 7ff753c37740 1 -- 172.21.15.94:0/3694190807 >> 172.21.15.94:0/3694190807 conn(0x5614bf323980 msgr2=0x5614bf3256d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.152+0000 7ff7327fc640 0 client.19152 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.152+0000 7ff753c37740 1 -- 172.21.15.94:0/3694190807 shutdown_connections 2024-02-17T12:12:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.152+0000 7ff753c37740 1 -- 172.21.15.94:0/3694190807 wait complete. 2024-02-17T12:12:11.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.152+0000 7ff753c37740 20 client.19152 trim_cache size 0 max 16384 2024-02-17T12:12:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.167+0000 7ff753c37740 1 Processor -- start 2024-02-17T12:12:11.169 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.167+0000 7ff753c37740 1 -- start start 2024-02-17T12:12:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.168+0000 7ff753c37740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329060 0x5614bf334330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.168+0000 7ff753c37740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329a10 0x5614bf332a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.168+0000 7ff753c37740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf336b60 0x5614bf330cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.168+0000 7ff753c37740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bf216130 con 0x5614bf336b60 2024-02-17T12:12:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.168+0000 7ff753c37740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bf315a40 con 0x5614bf329a10 2024-02-17T12:12:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.168+0000 7ff753c37740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bf3285b0 con 0x5614bf329060 2024-02-17T12:12:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.168+0000 7ff74a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329a10 0x5614bf332a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.168+0000 7ff74a59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329a10 0x5614bf332a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36938/0 (socket says 172.21.15.94:36938) 2024-02-17T12:12:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.168+0000 7ff74a59c640 1 -- 172.21.15.94:0/1945407470 learned_addr learned my addr 172.21.15.94:0/1945407470 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:11.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.169+0000 7ff74b59e640 1 --2- 172.21.15.94:0/1945407470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf336b60 0x5614bf330cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.169+0000 7ff74b59e640 1 -- 172.21.15.94:0/1945407470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329060 msgr2=0x5614bf334330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:12:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.169+0000 7ff74b59e640 1 --2- 172.21.15.94:0/1945407470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329060 0x5614bf334330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.169+0000 7ff74b59e640 1 -- 172.21.15.94:0/1945407470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329a10 msgr2=0x5614bf332a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.169+0000 7ff74b59e640 1 --2- 172.21.15.94:0/1945407470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329a10 0x5614bf332a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.172 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.169+0000 7ff74b59e640 1 -- 172.21.15.94:0/1945407470 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614bf22bc20 con 0x5614bf336b60 2024-02-17T12:12:11.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.169+0000 7ff74a59c640 1 --2- 172.21.15.94:0/1945407470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329a10 0x5614bf332a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:11.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.170+0000 7ff74b59e640 1 --2- 172.21.15.94:0/1945407470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf336b60 0x5614bf330cf0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff744005fd0 tx=0x7ff7440129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fd4482eb48a44d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:11.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.170+0000 7ff740ff9640 1 -- 172.21.15.94:0/1945407470 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744017070 con 0x5614bf336b60 2024-02-17T12:12:11.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.170+0000 7ff740ff9640 1 -- 172.21.15.94:0/1945407470 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7440105b0 con 0x5614bf336b60 2024-02-17T12:12:11.173 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.170+0000 7ff740ff9640 1 -- 172.21.15.94:0/1945407470 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744005260 con 0x5614bf336b60 2024-02-17T12:12:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.171+0000 7ff753c37740 1 -- 172.21.15.94:0/1945407470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf336b60 msgr2=0x5614bf330cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.171+0000 7ff753c37740 1 --2- 172.21.15.94:0/1945407470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf336b60 0x5614bf330cf0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff744005fd0 tx=0x7ff7440129e0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.171+0000 7ff753c37740 1 -- 172.21.15.94:0/1945407470 shutdown_connections 2024-02-17T12:12:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.171+0000 7ff753c37740 1 --2- 172.21.15.94:0/1945407470 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf336b60 0x5614bf330cf0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.171+0000 7ff753c37740 1 --2- 172.21.15.94:0/1945407470 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329a10 0x5614bf332a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.177 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.171+0000 7ff753c37740 1 --2- 172.21.15.94:0/1945407470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329060 0x5614bf334330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.171+0000 7ff753c37740 1 -- 172.21.15.94:0/1945407470 >> 172.21.15.94:0/1945407470 conn(0x5614bf323980 msgr2=0x5614bf332360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.171+0000 7ff753c37740 1 -- 172.21.15.94:0/1945407470 shutdown_connections 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.171+0000 7ff753c37740 1 -- 172.21.15.94:0/1945407470 wait complete. 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.171+0000 7ff753c37740 1 -- start start 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.171+0000 7ff753c37740 1 Processor -- start 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.171+0000 7ff753c37740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329060 0x5614bf33d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff753c37740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329a10 0x5614bf33dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff753c37740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf336b60 0x5614bf33bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff753c37740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bf33c410 con 0x5614bf336b60 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff753c37740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bf33c580 con 0x5614bf329060 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff753c37740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614bf33c740 con 0x5614bf329a10 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff74ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329060 0x5614bf33d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff74ad9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329060 0x5614bf33d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36952/0 (socket says 172.21.15.94:36952) 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff74ad9d640 1 -- 172.21.15.94:0/3465869428 learned_addr learned my addr 172.21.15.94:0/3465869428 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:11.178 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff753c37740 10 client.0 _ll_register_callbacks cb 0x7ff734004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff74b59e640 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf336b60 0x5614bf33bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff74a59c640 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329a10 0x5614bf33dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff74ad9d640 1 -- 172.21.15.94:0/3465869428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329a10 msgr2=0x5614bf33dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff74ad9d640 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329a10 0x5614bf33dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff74ad9d640 1 -- 172.21.15.94:0/3465869428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf336b60 msgr2=0x5614bf33bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff74ad9d640 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf336b60 0x5614bf33bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff74ad9d640 1 -- 172.21.15.94:0/3465869428 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614bf28f6a0 con 0x5614bf329060 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff74b59e640 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf336b60 0x5614bf33bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff74ad9d640 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329060 0x5614bf33d7f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff7340056f0 tx=0x7ff73402c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=810383669743bf02 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff7327fc640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff7327fc640 1 -- 172.21.15.94:0/3465869428 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734003e20 con 0x5614bf329060 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff7327fc640 1 -- 172.21.15.94:0/3465869428 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7ff7340c51f0 con 0x5614bf329060 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff753c37740 1 -- 172.21.15.94:0/3465869428 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7340bca50 con 0x5614bf329060 2024-02-17T12:12:11.179 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff7327fc640 1 -- 172.21.15.94:0/3465869428 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7340c0ba0 con 0x5614bf329060 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff753c37740 10 client.28625 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.172+0000 7ff753c37740 1 -- 172.21.15.94:0/3465869428 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7ff744009d60 con 0x5614bf329060 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.173+0000 7ff7327fc640 1 -- 172.21.15.94:0/3465869428 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff7340bad50 con 0x5614bf329060 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.173+0000 7ff7327fc640 1 -- 172.21.15.94:0/3465869428 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7ff73403c070 con 0x5614bf329060 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.173+0000 7ff753c37740 10 client.28625 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.173+0000 7ff753c37740 10 client.28625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.173+0000 7ff753c37740 20 client.28625 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.173+0000 7ff753c37740 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614bf39e220 0x5614bf3a0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.173+0000 7ff753c37740 4 client.28625 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.173+0000 7ff753c37740 1 -- 172.21.15.94:0/3465869428 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7ff7340bca50 con 0x5614bf39e220 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.173+0000 7ff74a59c640 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614bf39e220 0x5614bf3a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff7327fc640 10 client.28625 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff74a59c640 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614bf39e220 0x5614bf3a0610 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdc5b32b25d82efd server_cookie=9438af4bda19354 in_seq=0 out_seq=0 2024-02-17T12:12:11.180 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff7327fc640 1 -- 172.21.15.94:0/3465869428 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7ff7340bca50 con 0x5614bf39e220 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff7327fc640 10 client.28625 handle_command_reply: tid=0 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff753c37740 1 client.28625 shutdown 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff753c37740 1 -- 172.21.15.94:0/3465869428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329060 msgr2=0x5614bf33d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff753c37740 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329060 0x5614bf33d7f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7ff7340056f0 tx=0x7ff73402c760 comp rx=0 tx=0).stop 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff753c37740 1 -- 172.21.15.94:0/3465869428 shutdown_connections 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff753c37740 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5614bf39e220 0x5614bf3a0610 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff753c37740 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5614bf336b60 0x5614bf33bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff7327fc640 0 client.28625 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff753c37740 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5614bf329a10 0x5614bf33dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff753c37740 1 --2- 172.21.15.94:0/3465869428 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5614bf329060 0x5614bf33d7f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff753c37740 1 -- 172.21.15.94:0/3465869428 >> 172.21.15.94:0/3465869428 conn(0x5614bf323980 msgr2=0x5614bf322b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff753c37740 1 -- 172.21.15.94:0/3465869428 shutdown_connections 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff753c37740 1 -- 172.21.15.94:0/3465869428 wait complete. 2024-02-17T12:12:11.181 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.174+0000 7ff753c37740 20 client.28625 trim_cache size 0 max 16384 2024-02-17T12:12:11.182 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:12:11.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.175+0000 7ff751064640 1 -- 172.21.15.94:0/2694669660 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7380a2380 msgr2=0x7ff7380a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:11.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.175+0000 7ff751064640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7380a2380 0x7ff7380a4840 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff74c077680 tx=0x7ff73c00b410 comp rx=0 tx=0).stop 2024-02-17T12:12:11.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.175+0000 7ff751064640 1 -- 172.21.15.94:0/2694669660 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff74c069500 msgr2=0x7ff74c19d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:11.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.175+0000 7ff751064640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff74c069500 0x7ff74c19d3e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff734005940 tx=0x7ff73402c6f0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.175+0000 7ff751064640 1 -- 172.21.15.94:0/2694669660 shutdown_connections 2024-02-17T12:12:11.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.175+0000 7ff751064640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff7380a2380 0x7ff7380a4840 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.175+0000 7ff751064640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff74c106120 0x7ff74c1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.183 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.175+0000 7ff751064640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff74c069ea0 0x7ff74c19d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.175+0000 7ff751064640 1 --2- 172.21.15.94:0/2694669660 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff74c069500 0x7ff74c19d3e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:11.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.175+0000 7ff751064640 1 -- 172.21.15.94:0/2694669660 >> 172.21.15.94:0/2694669660 conn(0x7ff74c077110 msgr2=0x7ff74c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:11.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.175+0000 7ff751064640 1 -- 172.21.15.94:0/2694669660 shutdown_connections 2024-02-17T12:12:11.184 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:11.175+0000 7ff751064640 1 -- 172.21.15.94:0/2694669660 wait complete. 2024-02-17T12:12:11.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:11 smithi189 ceph-mon[36588]: pgmap v440: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 27 MiB/s wr, 1.84k op/s 2024-02-17T12:12:11.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:11 smithi125 ceph-mon[36819]: pgmap v440: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 27 MiB/s wr, 1.84k op/s 2024-02-17T12:12:11.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:11 smithi094 ceph-mon[32499]: pgmap v440: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 27 MiB/s wr, 1.84k op/s 2024-02-17T12:12:12.192 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:12:12.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:12:12.192 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:12:12.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.319+0000 7f04496df640 1 -- 172.21.15.94:0/2770654510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0444100c10 msgr2=0x7f0444104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.319+0000 7f04496df640 1 --2- 172.21.15.94:0/2770654510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0444100c10 0x7f0444104c50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f0438009930 tx=0x7f043802bc30 comp rx=0 tx=0).stop 2024-02-17T12:12:12.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.319+0000 7f04496df640 1 -- 172.21.15.94:0/2770654510 shutdown_connections 2024-02-17T12:12:12.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.319+0000 7f04496df640 1 --2- 172.21.15.94:0/2770654510 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0444105330 0x7f044410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.319+0000 7f04496df640 1 --2- 172.21.15.94:0/2770654510 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0444100c10 0x7f0444104c50 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.319+0000 7f04496df640 1 --2- 172.21.15.94:0/2770654510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0444100260 0x7f0444100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.319+0000 7f04496df640 1 -- 172.21.15.94:0/2770654510 >> 172.21.15.94:0/2770654510 conn(0x7f0444077180 msgr2=0x7f0444077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:12.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.320+0000 7f04496df640 1 -- 172.21.15.94:0/2770654510 shutdown_connections 2024-02-17T12:12:12.322 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.320+0000 7f04496df640 1 -- 172.21.15.94:0/2770654510 wait complete. 2024-02-17T12:12:12.323 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.320+0000 7f04496df640 1 Processor -- start 2024-02-17T12:12:12.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.320+0000 7f04496df640 1 -- start start 2024-02-17T12:12:12.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04496df640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0444100260 0x7f0444196720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04496df640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0444105330 0x7f0444196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04496df640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f044419afe0 0x7f044419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04496df640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044419b9d0 con 0x7f0444105330 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04496df640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044419bb40 con 0x7f0444100260 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04496df640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044419bcb0 con 0x7f044419afe0 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04427fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0444105330 0x7f0444196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04427fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0444105330 0x7f0444196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33482/0 (socket says 172.21.15.94:33482) 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04427fc640 1 -- 172.21.15.94:0/2595150813 learned_addr learned my addr 172.21.15.94:0/2595150813 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f0442ffd640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0444100260 0x7f0444196720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04437fe640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f044419afe0 0x7f044419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04427fc640 1 -- 172.21.15.94:0/2595150813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f044419afe0 msgr2=0x7f044419b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04427fc640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f044419afe0 0x7f044419b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04427fc640 1 -- 172.21.15.94:0/2595150813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0444100260 msgr2=0x7f0444196720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04427fc640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0444100260 0x7f0444196720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04427fc640 1 -- 172.21.15.94:0/2595150813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0438009560 con 0x7f0444105330 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04437fe640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f044419afe0 0x7f044419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:12:12.325 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f04427fc640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0444105330 0x7f0444196c60 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0438005ab0 tx=0x7f043802cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:12.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.321+0000 7f042bfff640 1 -- 172.21.15.94:0/2595150813 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438004350 con 0x7f0444105330 2024-02-17T12:12:12.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.322+0000 7f042bfff640 1 -- 172.21.15.94:0/2595150813 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f04380044b0 con 0x7f0444105330 2024-02-17T12:12:12.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.322+0000 7f04496df640 1 -- 172.21.15.94:0/2595150813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04441a3290 con 0x7f0444105330 2024-02-17T12:12:12.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.322+0000 7f042bfff640 1 -- 172.21.15.94:0/2595150813 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0438003cc0 con 0x7f0444105330 2024-02-17T12:12:12.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.322+0000 7f04496df640 1 -- 172.21.15.94:0/2595150813 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04441a3710 con 0x7f0444105330 2024-02-17T12:12:12.326 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.325+0000 7f042bfff640 1 -- 172.21.15.94:0/2595150813 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0438004df0 con 0x7f0444105330 2024-02-17T12:12:12.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.325+0000 7f044c38d740 1 Processor -- start 2024-02-17T12:12:12.327 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.325+0000 7f044c38d740 1 -- start start 2024-02-17T12:12:12.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f042bfff640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f042c0a2330 0x7f042c0a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f044c38d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7ee380 0x55b97a7ee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f044c38d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7eed30 0x55b97a7f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f044c38d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7f78b0 0x55b97a7f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f044c38d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97a75cf90 con 0x55b97a7ee380 2024-02-17T12:12:12.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f044c38d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97a75a4c0 con 0x55b97a7f78b0 2024-02-17T12:12:12.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f044c38d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97a755ca0 con 0x55b97a7eed30 2024-02-17T12:12:12.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f04437fe640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f042c0a2330 0x7f042c0a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7ee380 0x55b97a7ee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f042bfff640 1 -- 172.21.15.94:0/2595150813 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f04380f0780 con 0x7f0444105330 2024-02-17T12:12:12.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f04427fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7eed30 0x55b97a7f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f04437fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7f78b0 0x55b97a7f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7ee380 0x55b97a7ee760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33484/0 (socket says 172.21.15.94:33484) 2024-02-17T12:12:12.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f0442ffd640 1 -- 172.21.15.94:0/2134203554 learned_addr learned my addr 172.21.15.94:0/2134203554 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:12.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f0442ffd640 1 -- 172.21.15.94:0/2134203554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7eed30 msgr2=0x55b97a7f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:12.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f0442ffd640 1 --2- 172.21.15.94:0/2134203554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7eed30 0x55b97a7f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f0442ffd640 1 -- 172.21.15.94:0/2134203554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7f78b0 msgr2=0x55b97a7f9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:12.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f0442ffd640 1 --2- 172.21.15.94:0/2134203554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7f78b0 0x55b97a7f9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f0442ffd640 1 -- 172.21.15.94:0/2134203554 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b97a7180b0 con 0x55b97a7ee380 2024-02-17T12:12:12.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f04427fc640 1 --2- 172.21.15.94:0/2134203554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7eed30 0x55b97a7f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:12.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f04437fe640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f042c0a2330 0x7f042c0a47f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f043c005fd0 tx=0x7f043c008c10 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:12.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.326+0000 7f0442ffd640 1 --2- 172.21.15.94:0/2134203554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7ee380 0x55b97a7ee760 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0434009d20 tx=0x7f0434011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18a77760d2a9d00 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:12.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.327+0000 7f04297fa640 1 -- 172.21.15.94:0/2134203554 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0434019630 con 0x55b97a7ee380 2024-02-17T12:12:12.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.327+0000 7f04297fa640 1 -- 172.21.15.94:0/2134203554 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0434009e10 con 0x55b97a7ee380 2024-02-17T12:12:12.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.327+0000 7f04297fa640 1 -- 172.21.15.94:0/2134203554 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043400eb60 con 0x55b97a7ee380 2024-02-17T12:12:12.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.327+0000 7f044c38d740 1 -- 172.21.15.94:0/2134203554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7ee380 msgr2=0x55b97a7ee760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.327+0000 7f044c38d740 1 --2- 172.21.15.94:0/2134203554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7ee380 0x55b97a7ee760 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f0434009d20 tx=0x7f0434011810 comp rx=0 tx=0).stop 2024-02-17T12:12:12.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.327+0000 7f044c38d740 1 -- 172.21.15.94:0/2134203554 shutdown_connections 2024-02-17T12:12:12.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.327+0000 7f044c38d740 1 --2- 172.21.15.94:0/2134203554 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7f78b0 0x55b97a7f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.327+0000 7f044c38d740 1 --2- 172.21.15.94:0/2134203554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7eed30 0x55b97a7f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.327+0000 7f044c38d740 1 --2- 172.21.15.94:0/2134203554 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7ee380 0x55b97a7ee760 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.327+0000 7f044c38d740 1 -- 172.21.15.94:0/2134203554 >> 172.21.15.94:0/2134203554 conn(0x55b97a7e8840 msgr2=0x55b97a7e8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.327+0000 7f044c38d740 1 -- 172.21.15.94:0/2134203554 shutdown_connections 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.327+0000 7f044c38d740 1 -- 172.21.15.94:0/2134203554 wait complete. 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f044c38d740 1 -- start start 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f044c38d740 1 Processor -- start 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f044c38d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7ee380 0x55b97a7f4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f044c38d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7eed30 0x55b97a7f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f044c38d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7f78b0 0x55b97a7f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f044c38d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97a6faf40 con 0x55b97a7f78b0 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f044c38d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97a4294b0 con 0x55b97a7eed30 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f044c38d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97a6db670 con 0x55b97a7ee380 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f044c38d740 10 client.0 _ll_register_callbacks cb 0x55b97a748f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:12.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f04437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7f78b0 0x55b97a7f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f04437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7f78b0 0x55b97a7f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33496/0 (socket says 172.21.15.94:33496) 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f04437fe640 1 -- 172.21.15.94:0/3732963205 learned_addr learned my addr 172.21.15.94:0/3732963205 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7ee380 0x55b97a7f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f04427fc640 1 --2- 172.21.15.94:0/3732963205 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7eed30 0x55b97a7f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f04437fe640 1 -- 172.21.15.94:0/3732963205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7ee380 msgr2=0x55b97a7f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f04437fe640 1 --2- 172.21.15.94:0/3732963205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7ee380 0x55b97a7f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f04437fe640 1 -- 172.21.15.94:0/3732963205 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7eed30 msgr2=0x55b97a7f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f04437fe640 1 --2- 172.21.15.94:0/3732963205 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7eed30 0x55b97a7f4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f04437fe640 1 -- 172.21.15.94:0/3732963205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0434009950 con 0x55b97a7f78b0 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.328+0000 7f04427fc640 1 --2- 172.21.15.94:0/3732963205 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7eed30 0x55b97a7f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.329+0000 7f04437fe640 1 --2- 172.21.15.94:0/3732963205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7f78b0 0x55b97a7f52c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f043c010630 tx=0x7f043c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af342abcd6f5d28a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.329+0000 7f0426ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.329+0000 7f0426ff5640 1 -- 172.21.15.94:0/3732963205 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c0089a0 con 0x55b97a7f78b0 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.329+0000 7f0426ff5640 1 -- 172.21.15.94:0/3732963205 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f043c023070 con 0x55b97a7f78b0 2024-02-17T12:12:12.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.329+0000 7f044c38d740 1 -- 172.21.15.94:0/3732963205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b97a72ff40 con 0x55b97a7f78b0 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.330+0000 7f0426ff5640 1 -- 172.21.15.94:0/3732963205 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f043c01f860 con 0x55b97a7f78b0 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.330+0000 7f044c38d740 10 client.19170 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.330+0000 7f044c38d740 1 -- 172.21.15.94:0/3732963205 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b97a754320 con 0x55b97a7f78b0 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.330+0000 7f0426ff5640 1 -- 172.21.15.94:0/3732963205 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f043c034080 con 0x55b97a7f78b0 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.330+0000 7f0426ff5640 1 -- 172.21.15.94:0/3732963205 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f043c00b050 con 0x55b97a7f78b0 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.330+0000 7f044c38d740 10 client.19170 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.330+0000 7f044c38d740 10 client.19170 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.330+0000 7f044c38d740 20 client.19170 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.330+0000 7f044c38d740 1 --2- 172.21.15.94:0/3732963205 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b97a807e60 0x55b97a80a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.330+0000 7f044c38d740 4 client.19170 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.330+0000 7f044c38d740 1 -- 172.21.15.94:0/3732963205 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b97a74c310 con 0x55b97a807e60 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.331+0000 7f0442ffd640 1 --2- 172.21.15.94:0/3732963205 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b97a807e60 0x55b97a80a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.331+0000 7f0426ff5640 10 client.19170 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.331+0000 7f0442ffd640 1 --2- 172.21.15.94:0/3732963205 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b97a807e60 0x55b97a80a280 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6262c078f934849 server_cookie=37c058f68b20973d in_seq=0 out_seq=0 2024-02-17T12:12:12.337 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f0426ff5640 1 -- 172.21.15.94:0/3732963205 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b97a74c310 con 0x55b97a807e60 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f0426ff5640 10 client.19170 handle_command_reply: tid=0 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f044c38d740 1 client.19170 shutdown 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f044c38d740 1 -- 172.21.15.94:0/3732963205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7f78b0 msgr2=0x55b97a7f52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f044c38d740 1 --2- 172.21.15.94:0/3732963205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7f78b0 0x55b97a7f52c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f043c010630 tx=0x7f043c004ab0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f044c38d740 1 -- 172.21.15.94:0/3732963205 shutdown_connections 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f044c38d740 1 --2- 172.21.15.94:0/3732963205 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b97a807e60 0x55b97a80a280 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f044c38d740 1 --2- 172.21.15.94:0/3732963205 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7f78b0 0x55b97a7f52c0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f0426ff5640 0 client.19170 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f044c38d740 1 --2- 172.21.15.94:0/3732963205 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7eed30 0x55b97a7f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f044c38d740 1 --2- 172.21.15.94:0/3732963205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7ee380 0x55b97a7f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f044c38d740 1 -- 172.21.15.94:0/3732963205 >> 172.21.15.94:0/3732963205 conn(0x55b97a7e8840 msgr2=0x55b97a7ea7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f044c38d740 1 -- 172.21.15.94:0/3732963205 shutdown_connections 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f044c38d740 1 -- 172.21.15.94:0/3732963205 wait complete. 2024-02-17T12:12:12.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.332+0000 7f044c38d740 20 client.19170 trim_cache size 0 max 16384 2024-02-17T12:12:12.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.377+0000 7f044c38d740 1 Processor -- start 2024-02-17T12:12:12.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.378+0000 7f044c38d740 1 -- start start 2024-02-17T12:12:12.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.378+0000 7f044c38d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7e8840 0x55b97a7f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.378+0000 7f044c38d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7ee380 0x55b97a7fe8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.378+0000 7f044c38d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7eed30 0x55b97a7fee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.378+0000 7f044c38d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97a80f050 con 0x55b97a7eed30 2024-02-17T12:12:12.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.378+0000 7f044c38d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97a7f5650 con 0x55b97a7e8840 2024-02-17T12:12:12.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.378+0000 7f044c38d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97a7f5810 con 0x55b97a7ee380 2024-02-17T12:12:12.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.379+0000 7f04427fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7ee380 0x55b97a7fe8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.379+0000 7f04437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7eed30 0x55b97a7fee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.379+0000 7f04437fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7eed30 0x55b97a7fee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33504/0 (socket says 172.21.15.94:33504) 2024-02-17T12:12:12.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.379+0000 7f04437fe640 1 -- 172.21.15.94:0/502097969 learned_addr learned my addr 172.21.15.94:0/502097969 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:12.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.381+0000 7f04437fe640 1 -- 172.21.15.94:0/502097969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7ee380 msgr2=0x55b97a7fe8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:12.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.381+0000 7f04437fe640 1 --2- 172.21.15.94:0/502097969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7ee380 0x55b97a7fe8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.381+0000 7f04437fe640 1 -- 172.21.15.94:0/502097969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7e8840 msgr2=0x55b97a7f50e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:12:12.383 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.381+0000 7f04437fe640 1 --2- 172.21.15.94:0/502097969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7e8840 0x55b97a7f50e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.384 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.381+0000 7f04437fe640 1 -- 172.21.15.94:0/502097969 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b97a6f0c00 con 0x55b97a7eed30 2024-02-17T12:12:12.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.386+0000 7f04437fe640 1 --2- 172.21.15.94:0/502097969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7eed30 0x55b97a7fee20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f043c028c70 tx=0x7f043c0085a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500c8c088f4a2caa server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:12.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.387+0000 7f0428ff9640 1 -- 172.21.15.94:0/502097969 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c008ee0 con 0x55b97a7eed30 2024-02-17T12:12:12.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.388+0000 7f0428ff9640 1 -- 172.21.15.94:0/502097969 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b97a72ff40 con 0x55b97a7eed30 2024-02-17T12:12:12.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.388+0000 7f0428ff9640 1 -- 172.21.15.94:0/502097969 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043c030ab0 con 0x55b97a7eed30 2024-02-17T12:12:12.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.391+0000 7f044c38d740 1 -- 172.21.15.94:0/502097969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7eed30 msgr2=0x55b97a7fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.396 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.391+0000 7f044c38d740 1 --2- 172.21.15.94:0/502097969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7eed30 0x55b97a7fee20 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f043c028c70 tx=0x7f043c0085a0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.395+0000 7f044c38d740 1 -- 172.21.15.94:0/502097969 shutdown_connections 2024-02-17T12:12:12.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.395+0000 7f044c38d740 1 --2- 172.21.15.94:0/502097969 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7eed30 0x55b97a7fee20 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.395+0000 7f044c38d740 1 --2- 172.21.15.94:0/502097969 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7ee380 0x55b97a7fe8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.395+0000 7f044c38d740 1 --2- 172.21.15.94:0/502097969 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7e8840 0x55b97a7f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.399 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.395+0000 7f044c38d740 1 -- 172.21.15.94:0/502097969 >> 172.21.15.94:0/502097969 conn(0x55b97a7ef480 msgr2=0x55b97a7e7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:12.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.401+0000 7f044c38d740 1 -- 172.21.15.94:0/502097969 shutdown_connections 2024-02-17T12:12:12.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.402+0000 7f044c38d740 1 -- 172.21.15.94:0/502097969 wait complete. 2024-02-17T12:12:12.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.402+0000 7f044c38d740 1 -- start start 2024-02-17T12:12:12.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.402+0000 7f044c38d740 1 Processor -- start 2024-02-17T12:12:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.404+0000 7f044c38d740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7e8840 0x55b97a861b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.407 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.404+0000 7f044c38d740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7ee380 0x55b97a8620c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.404+0000 7f044c38d740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7eed30 0x55b97a85cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.404+0000 7f044c38d740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97a862800 con 0x55b97a7ee380 2024-02-17T12:12:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.404+0000 7f044c38d740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97a862970 con 0x55b97a7e8840 2024-02-17T12:12:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.404+0000 7f044c38d740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b97a85d070 con 0x55b97a7eed30 2024-02-17T12:12:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.404+0000 7f044c38d740 10 client.0 _ll_register_callbacks cb 0x7f0434009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.405+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7e8840 0x55b97a861b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.405+0000 7f0442ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7e8840 0x55b97a861b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:36990/0 (socket says 172.21.15.94:36990) 2024-02-17T12:12:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.405+0000 7f0442ffd640 1 -- 172.21.15.94:0/179666186 learned_addr learned my addr 172.21.15.94:0/179666186 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.405+0000 7f0442ffd640 1 -- 172.21.15.94:0/179666186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7eed30 msgr2=0x55b97a85cb30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:12:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.405+0000 7f0442ffd640 1 --2- 172.21.15.94:0/179666186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7eed30 0x55b97a85cb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.405+0000 7f0442ffd640 1 -- 172.21.15.94:0/179666186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7ee380 msgr2=0x55b97a8620c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:12:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.405+0000 7f0442ffd640 1 --2- 172.21.15.94:0/179666186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7ee380 0x55b97a8620c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.405+0000 7f0442ffd640 1 -- 172.21.15.94:0/179666186 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b97a754320 con 0x55b97a7e8840 2024-02-17T12:12:12.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.407+0000 7f0442ffd640 1 --2- 172.21.15.94:0/179666186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7e8840 0x55b97a861b80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f043401c040 tx=0x7f0434005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25808dc39fb4aec7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:12.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.408+0000 7f0426ff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:12:12.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.408+0000 7f0426ff5640 1 -- 172.21.15.94:0/179666186 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043400ee50 con 0x55b97a7e8840 2024-02-17T12:12:12.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.408+0000 7f0426ff5640 1 -- 172.21.15.94:0/179666186 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f0434009e10 con 0x55b97a7e8840 2024-02-17T12:12:12.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.408+0000 7f044c38d740 1 -- 172.21.15.94:0/179666186 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b97a77b900 con 0x55b97a7e8840 2024-02-17T12:12:12.415 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.413+0000 7f0426ff5640 1 -- 172.21.15.94:0/179666186 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f043400e590 con 0x55b97a7e8840 2024-02-17T12:12:12.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.414+0000 7f044c38d740 10 client.28637 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:12.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.414+0000 7f044c38d740 1 -- 172.21.15.94:0/179666186 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b97a716910 con 0x55b97a7e8840 2024-02-17T12:12:12.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.414+0000 7f0426ff5640 1 -- 172.21.15.94:0/179666186 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b97a77b900 con 0x55b97a7e8840 2024-02-17T12:12:12.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.414+0000 7f0426ff5640 1 -- 172.21.15.94:0/179666186 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0434003a00 con 0x55b97a7e8840 2024-02-17T12:12:12.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.415+0000 7f044c38d740 10 client.28637 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:12.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.415+0000 7f044c38d740 10 client.28637 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:12.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.415+0000 7f044c38d740 20 client.28637 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:12.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.415+0000 7f044c38d740 1 --2- 172.21.15.94:0/179666186 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b97a7f78b0 0x55b97a857010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.415+0000 7f044c38d740 4 client.28637 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:12:12.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.415+0000 7f044c38d740 1 -- 172.21.15.94:0/179666186 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b97a455860 con 0x55b97a7f78b0 2024-02-17T12:12:12.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.415+0000 7f04427fc640 1 --2- 172.21.15.94:0/179666186 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b97a7f78b0 0x55b97a857010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.419 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.417+0000 7f04427fc640 1 --2- 172.21.15.94:0/179666186 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b97a7f78b0 0x55b97a857010 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=205e85b4b9bea82d server_cookie=d0de76c737199142 in_seq=0 out_seq=0 2024-02-17T12:12:12.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.418+0000 7f0426ff5640 10 client.28637 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:12.420 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.418+0000 7f0426ff5640 1 -- 172.21.15.94:0/179666186 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b97a455860 con 0x55b97a7f78b0 2024-02-17T12:12:12.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.418+0000 7f0426ff5640 10 client.28637 handle_command_reply: tid=0 2024-02-17T12:12:12.421 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.418+0000 7f044c38d740 1 client.28637 shutdown 2024-02-17T12:12:12.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.419+0000 7f044c38d740 1 -- 172.21.15.94:0/179666186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7e8840 msgr2=0x55b97a861b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.419+0000 7f044c38d740 1 --2- 172.21.15.94:0/179666186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7e8840 0x55b97a861b80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f043401c040 tx=0x7f0434005d30 comp rx=0 tx=0).stop 2024-02-17T12:12:12.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.419+0000 7f044c38d740 1 -- 172.21.15.94:0/179666186 shutdown_connections 2024-02-17T12:12:12.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.419+0000 7f044c38d740 1 --2- 172.21.15.94:0/179666186 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b97a7f78b0 0x55b97a857010 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.419+0000 7f044c38d740 1 --2- 172.21.15.94:0/179666186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b97a7eed30 0x55b97a85cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.419+0000 7f044c38d740 1 --2- 172.21.15.94:0/179666186 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b97a7ee380 0x55b97a8620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.419+0000 7f044c38d740 1 --2- 172.21.15.94:0/179666186 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b97a7e8840 0x55b97a861b80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.419+0000 7f044c38d740 1 -- 172.21.15.94:0/179666186 >> 172.21.15.94:0/179666186 conn(0x55b97a7ef480 msgr2=0x55b97a7e78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:12.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.419+0000 7f0426ff5640 0 client.28637 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:12.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.420+0000 7f044c38d740 1 -- 172.21.15.94:0/179666186 shutdown_connections 2024-02-17T12:12:12.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.420+0000 7f044c38d740 1 -- 172.21.15.94:0/179666186 wait complete. 2024-02-17T12:12:12.422 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.420+0000 7f044c38d740 20 client.28637 trim_cache size 0 max 16384 2024-02-17T12:12:12.423 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:12:12.423 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:12:12.423 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "27cde575-219c-4113-968f-fc1d81a29a20", 2024-02-17T12:12:12.423 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:12:12.423 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:12:12.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.421+0000 7f04496df640 1 -- 172.21.15.94:0/2595150813 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f042c0a2330 msgr2=0x7f042c0a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.421+0000 7f04496df640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f042c0a2330 0x7f042c0a47f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f043c005fd0 tx=0x7f043c008c10 comp rx=0 tx=0).stop 2024-02-17T12:12:12.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.421+0000 7f04496df640 1 -- 172.21.15.94:0/2595150813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0444105330 msgr2=0x7f0444196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.423 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.421+0000 7f04496df640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0444105330 0x7f0444196c60 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f0438005ab0 tx=0x7f043802cd80 comp rx=0 tx=0).stop 2024-02-17T12:12:12.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.422+0000 7f04496df640 1 -- 172.21.15.94:0/2595150813 shutdown_connections 2024-02-17T12:12:12.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.422+0000 7f04496df640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f042c0a2330 0x7f042c0a47f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.422+0000 7f04496df640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f044419afe0 0x7f044419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.424 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.422+0000 7f04496df640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0444105330 0x7f0444196c60 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.422+0000 7f04496df640 1 --2- 172.21.15.94:0/2595150813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0444100260 0x7f0444196720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.422+0000 7f04496df640 1 -- 172.21.15.94:0/2595150813 >> 172.21.15.94:0/2595150813 conn(0x7f0444077180 msgr2=0x7f04440fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:12.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.422+0000 7f04496df640 1 -- 172.21.15.94:0/2595150813 shutdown_connections 2024-02-17T12:12:12.425 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.422+0000 7f04496df640 1 -- 172.21.15.94:0/2595150813 wait complete. 2024-02-17T12:12:12.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:12:12.439 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:12:12.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.600+0000 7f4d6400f640 1 -- 172.21.15.94:0/1517934180 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d5c108da0 msgr2=0x7f4d5c10c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.600+0000 7f4d6400f640 1 --2- 172.21.15.94:0/1517934180 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d5c108da0 0x7f4d5c10c7d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009930 tx=0x7f4d5402cd70 comp rx=0 tx=0).stop 2024-02-17T12:12:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.602+0000 7f4d6400f640 1 -- 172.21.15.94:0/1517934180 shutdown_connections 2024-02-17T12:12:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.602+0000 7f4d6400f640 1 --2- 172.21.15.94:0/1517934180 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d5c10ceb0 0x7f4d5c114ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.602+0000 7f4d6400f640 1 --2- 172.21.15.94:0/1517934180 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d5c108da0 0x7f4d5c10c7d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.602+0000 7f4d6400f640 1 --2- 172.21.15.94:0/1517934180 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d5c1083f0 0x7f4d5c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.602+0000 7f4d6400f640 1 -- 172.21.15.94:0/1517934180 >> 172.21.15.94:0/1517934180 conn(0x7f4d5c06c6d0 msgr2=0x7f4d5c06cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.602+0000 7f4d6400f640 1 -- 172.21.15.94:0/1517934180 shutdown_connections 2024-02-17T12:12:12.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.602+0000 7f4d6400f640 1 -- 172.21.15.94:0/1517934180 wait complete. 2024-02-17T12:12:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.603+0000 7f4d6400f640 1 Processor -- start 2024-02-17T12:12:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.603+0000 7f4d6400f640 1 -- start start 2024-02-17T12:12:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.603+0000 7f4d6400f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d5c1083f0 0x7f4d5c111bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.603+0000 7f4d6400f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d5c10ceb0 0x7f4d5c1120f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.603+0000 7f4d6400f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d5c112630 0x7f4d5c1b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.603+0000 7f4d61dab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d5c1083f0 0x7f4d5c111bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.603+0000 7f4d61dab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d5c1083f0 0x7f4d5c111bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:44986/0 (socket says 172.21.15.94:44986) 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.603+0000 7f4d61dab640 1 -- 172.21.15.94:0/4043577710 learned_addr learned my addr 172.21.15.94:0/4043577710 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.603+0000 7f4d615aa640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d5c10ceb0 0x7f4d5c1120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.603+0000 7f4d6400f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c114d80 con 0x7f4d5c112630 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.603+0000 7f4d6400f640 1 -- 172.21.15.94:0/4043577710 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c1b2ed0 con 0x7f4d5c10ceb0 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.603+0000 7f4d6400f640 1 -- 172.21.15.94:0/4043577710 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c1b3010 con 0x7f4d5c1083f0 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.603+0000 7f4d625ac640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d5c112630 0x7f4d5c1b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.604+0000 7f4d61dab640 1 -- 172.21.15.94:0/4043577710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d5c10ceb0 msgr2=0x7f4d5c1120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.604+0000 7f4d61dab640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d5c10ceb0 0x7f4d5c1120f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.604+0000 7f4d61dab640 1 -- 172.21.15.94:0/4043577710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d5c112630 msgr2=0x7f4d5c1b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.604+0000 7f4d61dab640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d5c112630 0x7f4d5c1b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.604+0000 7f4d61dab640 1 -- 172.21.15.94:0/4043577710 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54009560 con 0x7f4d5c1083f0 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.604+0000 7f4d625ac640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d5c112630 0x7f4d5c1b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:12:12.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.604+0000 7f4d61dab640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d5c1083f0 0x7f4d5c111bb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000c890 tx=0x7f4d5000cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:12.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.605+0000 7f4d5affd640 1 -- 172.21.15.94:0/4043577710 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50004220 con 0x7f4d5c1083f0 2024-02-17T12:12:12.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.605+0000 7f4d5affd640 1 -- 172.21.15.94:0/4043577710 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d50007e90 con 0x7f4d5c1083f0 2024-02-17T12:12:12.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.605+0000 7f4d5affd640 1 -- 172.21.15.94:0/4043577710 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50004a00 con 0x7f4d5c1083f0 2024-02-17T12:12:12.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.605+0000 7f4d6400f640 1 -- 172.21.15.94:0/4043577710 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5c1b3270 con 0x7f4d5c1083f0 2024-02-17T12:12:12.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.605+0000 7f4d6400f640 1 -- 172.21.15.94:0/4043577710 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5c1b3700 con 0x7f4d5c1083f0 2024-02-17T12:12:12.609 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.607+0000 7f4d5affd640 1 -- 172.21.15.94:0/4043577710 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4d500079b0 con 0x7f4d5c1083f0 2024-02-17T12:12:12.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.608+0000 7f4d5affd640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4d440a2440 0x7f4d440a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.608+0000 7f4d5affd640 1 -- 172.21.15.94:0/4043577710 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f4d500ce0f0 con 0x7f4d5c1083f0 2024-02-17T12:12:12.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.608+0000 7f4d66bda740 1 Processor -- start 2024-02-17T12:12:12.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.608+0000 7f4d615aa640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4d440a2440 0x7f4d440a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.609+0000 7f4d66bda740 1 -- start start 2024-02-17T12:12:12.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.609+0000 7f4d66bda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297edd6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.609+0000 7f4d66bda740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd6bd0 0x56297eddf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.609+0000 7f4d66bda740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297ede1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.609+0000 7f4d66bda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297ed45280 con 0x56297edd62b0 2024-02-17T12:12:12.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.609+0000 7f4d66bda740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297ed427b0 con 0x56297eddf7b0 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.609+0000 7f4d66bda740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297ed3df90 con 0x56297edd6bd0 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.609+0000 7f4d625ac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297ede1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.609+0000 7f4d625ac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297ede1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:37022/0 (socket says 172.21.15.94:37022) 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.609+0000 7f4d625ac640 1 -- 172.21.15.94:0/1022041334 learned_addr learned my addr 172.21.15.94:0/1022041334 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.609+0000 7f4d615aa640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4d440a2440 0x7f4d440a4900 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4d5402d380 tx=0x7f4d54005fd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.609+0000 7f4d615aa640 1 --2- 172.21.15.94:0/1022041334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd6bd0 0x56297eddf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.609+0000 7f4d61dab640 1 --2- 172.21.15.94:0/1022041334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297edd6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.610+0000 7f4d625ac640 1 -- 172.21.15.94:0/1022041334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd6bd0 msgr2=0x56297eddf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.610+0000 7f4d625ac640 1 --2- 172.21.15.94:0/1022041334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd6bd0 0x56297eddf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.610+0000 7f4d625ac640 1 -- 172.21.15.94:0/1022041334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 msgr2=0x56297edd6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.610+0000 7f4d625ac640 1 --2- 172.21.15.94:0/1022041334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297edd6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.610+0000 7f4d625ac640 1 -- 172.21.15.94:0/1022041334 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56297ecfc570 con 0x56297eddf7b0 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.610+0000 7f4d625ac640 1 --2- 172.21.15.94:0/1022041334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297ede1ba0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009f30 tx=0x7f4d4c010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4ca1967ad6f5d4b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:12.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.610+0000 7f4d43fff640 1 -- 172.21.15.94:0/1022041334 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c01b660 con 0x56297eddf7b0 2024-02-17T12:12:12.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.610+0000 7f4d43fff640 1 -- 172.21.15.94:0/1022041334 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d4c01bca0 con 0x56297eddf7b0 2024-02-17T12:12:12.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.610+0000 7f4d43fff640 1 -- 172.21.15.94:0/1022041334 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c012e20 con 0x56297eddf7b0 2024-02-17T12:12:12.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.611+0000 7f4d66bda740 1 -- 172.21.15.94:0/1022041334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 msgr2=0x56297ede1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.611+0000 7f4d66bda740 1 --2- 172.21.15.94:0/1022041334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297ede1ba0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009f30 tx=0x7f4d4c010800 comp rx=0 tx=0).stop 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.611+0000 7f4d66bda740 1 -- 172.21.15.94:0/1022041334 shutdown_connections 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.611+0000 7f4d66bda740 1 --2- 172.21.15.94:0/1022041334 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297ede1ba0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.611+0000 7f4d66bda740 1 --2- 172.21.15.94:0/1022041334 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd6bd0 0x56297eddf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.611+0000 7f4d66bda740 1 --2- 172.21.15.94:0/1022041334 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297edd6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.611+0000 7f4d66bda740 1 -- 172.21.15.94:0/1022041334 >> 172.21.15.94:0/1022041334 conn(0x56297edd0990 msgr2=0x56297edd0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.611+0000 7f4d66bda740 1 -- 172.21.15.94:0/1022041334 shutdown_connections 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.611+0000 7f4d66bda740 1 -- 172.21.15.94:0/1022041334 wait complete. 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.612+0000 7f4d66bda740 1 -- start start 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.612+0000 7f4d66bda740 1 Processor -- start 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.612+0000 7f4d66bda740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd62b0 0x56297eddf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.612+0000 7f4d66bda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd6bd0 0x56297eddc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.612+0000 7f4d66bda740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297eddcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.612+0000 7f4d66bda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297ea11430 con 0x56297edd6bd0 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.612+0000 7f4d66bda740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297e9cde00 con 0x56297eddf7b0 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.612+0000 7f4d66bda740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297ecc2fd0 con 0x56297edd62b0 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.612+0000 7f4d66bda740 10 client.0 _ll_register_callbacks cb 0x56297ed18fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.612+0000 7f4d625ac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297eddcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.613+0000 7f4d625ac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297eddcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:37026/0 (socket says 172.21.15.94:37026) 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.613+0000 7f4d625ac640 1 -- 172.21.15.94:0/1542121426 learned_addr learned my addr 172.21.15.94:0/1542121426 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.613+0000 7f4d61dab640 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd62b0 0x56297eddf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.613+0000 7f4d615aa640 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd6bd0 0x56297eddc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.613+0000 7f4d625ac640 1 -- 172.21.15.94:0/1542121426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd62b0 msgr2=0x56297eddf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.613+0000 7f4d625ac640 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd62b0 0x56297eddf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.613+0000 7f4d625ac640 1 -- 172.21.15.94:0/1542121426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd6bd0 msgr2=0x56297eddc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.613+0000 7f4d625ac640 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd6bd0 0x56297eddc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.613+0000 7f4d625ac640 1 -- 172.21.15.94:0/1542121426 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4c009b60 con 0x56297eddf7b0 2024-02-17T12:12:12.618 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.613+0000 7f4d615aa640 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd6bd0 0x56297eddc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:12:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.613+0000 7f4d625ac640 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297eddcd00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c012a40 tx=0x7f4d4c012720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=298b61e85564b2f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.613+0000 7f4d417fa640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:12:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.614+0000 7f4d417fa640 1 -- 172.21.15.94:0/1542121426 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c01f070 con 0x56297eddf7b0 2024-02-17T12:12:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.614+0000 7f4d417fa640 1 -- 172.21.15.94:0/1542121426 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f4d4c004a40 con 0x56297eddf7b0 2024-02-17T12:12:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.614+0000 7f4d66bda740 1 -- 172.21.15.94:0/1542121426 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56297ea3d7e0 con 0x56297eddf7b0 2024-02-17T12:12:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.614+0000 7f4d417fa640 1 -- 172.21.15.94:0/1542121426 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d4c004c20 con 0x56297eddf7b0 2024-02-17T12:12:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.614+0000 7f4d66bda740 10 client.28649 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.614+0000 7f4d66bda740 1 -- 172.21.15.94:0/1542121426 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56297ed3c610 con 0x56297eddf7b0 2024-02-17T12:12:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.614+0000 7f4d417fa640 1 -- 172.21.15.94:0/1542121426 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56297ea3d7e0 con 0x56297eddf7b0 2024-02-17T12:12:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.614+0000 7f4d417fa640 1 -- 172.21.15.94:0/1542121426 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4d4c005340 con 0x56297eddf7b0 2024-02-17T12:12:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.614+0000 7f4d66bda740 10 client.28649 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.615+0000 7f4d66bda740 10 client.28649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:12.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.615+0000 7f4d66bda740 20 client.28649 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:12.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.615+0000 7f4d66bda740 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56297edef8f0 0x56297edf1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.615+0000 7f4d66bda740 4 client.28649 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:12.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.615+0000 7f4d66bda740 1 -- 172.21.15.94:0/1542121426 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56297ea7a720 con 0x56297edef8f0 2024-02-17T12:12:12.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.615+0000 7f4d61dab640 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56297edef8f0 0x56297edf1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.615+0000 7f4d417fa640 10 client.28649 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.615+0000 7f4d61dab640 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56297edef8f0 0x56297edf1ce0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c7d2da82aa82b1 server_cookie=ce0a541a580942ff in_seq=0 out_seq=0 2024-02-17T12:12:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.616+0000 7f4d417fa640 1 -- 172.21.15.94:0/1542121426 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56297ea7a720 con 0x56297edef8f0 2024-02-17T12:12:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.616+0000 7f4d417fa640 10 client.28649 handle_command_reply: tid=0 2024-02-17T12:12:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.616+0000 7f4d66bda740 1 client.28649 shutdown 2024-02-17T12:12:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.617+0000 7f4d66bda740 1 -- 172.21.15.94:0/1542121426 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 msgr2=0x56297eddcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.617+0000 7f4d66bda740 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297eddcd00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c012a40 tx=0x7f4d4c012720 comp rx=0 tx=0).stop 2024-02-17T12:12:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.617+0000 7f4d66bda740 1 -- 172.21.15.94:0/1542121426 shutdown_connections 2024-02-17T12:12:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.617+0000 7f4d66bda740 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56297edef8f0 0x56297edf1ce0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.617+0000 7f4d66bda740 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297eddcd00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.617+0000 7f4d66bda740 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd6bd0 0x56297eddc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.617+0000 7f4d417fa640 0 client.28649 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.617+0000 7f4d66bda740 1 --2- 172.21.15.94:0/1542121426 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd62b0 0x56297eddf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.622 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.617+0000 7f4d66bda740 1 -- 172.21.15.94:0/1542121426 >> 172.21.15.94:0/1542121426 conn(0x56297edd0990 msgr2=0x56297edd2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:12.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.618+0000 7f4d66bda740 1 -- 172.21.15.94:0/1542121426 shutdown_connections 2024-02-17T12:12:12.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.618+0000 7f4d66bda740 1 -- 172.21.15.94:0/1542121426 wait complete. 2024-02-17T12:12:12.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.618+0000 7f4d66bda740 20 client.28649 trim_cache size 0 max 16384 2024-02-17T12:12:12.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.637+0000 7f4d66bda740 1 Processor -- start 2024-02-17T12:12:12.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.637+0000 7f4d66bda740 1 -- start start 2024-02-17T12:12:12.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.637+0000 7f4d66bda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297eddecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.637+0000 7f4d66bda740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd6bd0 0x56297eddf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d66bda740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297ede6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d61dab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297eddecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d61dab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297eddecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33532/0 (socket says 172.21.15.94:33532) 2024-02-17T12:12:12.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d61dab640 1 -- 172.21.15.94:0/3580621540 learned_addr learned my addr 172.21.15.94:0/3580621540 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:12.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d66bda740 1 -- 172.21.15.94:0/3580621540 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297edd81a0 con 0x56297edd62b0 2024-02-17T12:12:12.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d66bda740 1 -- 172.21.15.94:0/3580621540 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297edebdd0 con 0x56297eddf7b0 2024-02-17T12:12:12.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d66bda740 1 -- 172.21.15.94:0/3580621540 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297edcc2b0 con 0x56297edd6bd0 2024-02-17T12:12:12.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d615aa640 1 --2- 172.21.15.94:0/3580621540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd6bd0 0x56297eddf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d625ac640 1 --2- 172.21.15.94:0/3580621540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297ede6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d615aa640 1 -- 172.21.15.94:0/3580621540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 msgr2=0x56297ede6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:12.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d615aa640 1 --2- 172.21.15.94:0/3580621540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297ede6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d615aa640 1 -- 172.21.15.94:0/3580621540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 msgr2=0x56297eddecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d615aa640 1 --2- 172.21.15.94:0/3580621540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297eddecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d615aa640 1 -- 172.21.15.94:0/3580621540 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56297ecd8ac0 con 0x56297edd6bd0 2024-02-17T12:12:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.638+0000 7f4d615aa640 1 --2- 172.21.15.94:0/3580621540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd6bd0 0x56297eddf230 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4d54037040 tx=0x7f4d5403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43052d30059feb38 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.639+0000 7f4d437fe640 1 -- 172.21.15.94:0/3580621540 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d540044c0 con 0x56297edd6bd0 2024-02-17T12:12:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.639+0000 7f4d437fe640 1 -- 172.21.15.94:0/3580621540 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d540052d0 con 0x56297edd6bd0 2024-02-17T12:12:12.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.639+0000 7f4d437fe640 1 -- 172.21.15.94:0/3580621540 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d54003cd0 con 0x56297edd6bd0 2024-02-17T12:12:12.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.640+0000 7f4d66bda740 1 -- 172.21.15.94:0/3580621540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd6bd0 msgr2=0x56297eddf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.640+0000 7f4d66bda740 1 --2- 172.21.15.94:0/3580621540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd6bd0 0x56297eddf230 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4d54037040 tx=0x7f4d5403f000 comp rx=0 tx=0).stop 2024-02-17T12:12:12.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.640+0000 7f4d66bda740 1 -- 172.21.15.94:0/3580621540 shutdown_connections 2024-02-17T12:12:12.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.640+0000 7f4d66bda740 1 --2- 172.21.15.94:0/3580621540 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297eddf7b0 0x56297ede6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.640+0000 7f4d66bda740 1 --2- 172.21.15.94:0/3580621540 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297edd6bd0 0x56297eddf230 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.640+0000 7f4d66bda740 1 --2- 172.21.15.94:0/3580621540 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297eddecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.640+0000 7f4d66bda740 1 -- 172.21.15.94:0/3580621540 >> 172.21.15.94:0/3580621540 conn(0x56297edd0990 msgr2=0x56297edf6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:12.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.640+0000 7f4d66bda740 1 -- 172.21.15.94:0/3580621540 shutdown_connections 2024-02-17T12:12:12.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.640+0000 7f4d66bda740 1 -- 172.21.15.94:0/3580621540 wait complete. 2024-02-17T12:12:12.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.640+0000 7f4d66bda740 1 -- start start 2024-02-17T12:12:12.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d66bda740 1 Processor -- start 2024-02-17T12:12:12.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d66bda740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297ede5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d66bda740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297edd6bd0 0x56297ede5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d66bda740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297eddf7b0 0x56297ee3fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d66bda740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297ee40320 con 0x56297edd62b0 2024-02-17T12:12:12.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d66bda740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297ee40490 con 0x56297edd6bd0 2024-02-17T12:12:12.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d66bda740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56297ee40650 con 0x56297eddf7b0 2024-02-17T12:12:12.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d66bda740 10 client.0 _ll_register_callbacks cb 0x7f4d4c0096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:12.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d61dab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297ede5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d615aa640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297edd6bd0 0x56297ede5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d61dab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297ede5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:33544/0 (socket says 172.21.15.94:33544) 2024-02-17T12:12:12.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d61dab640 1 -- 172.21.15.94:0/2412397420 learned_addr learned my addr 172.21.15.94:0/2412397420 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:12.651 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d625ac640 1 --2- 172.21.15.94:0/2412397420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297eddf7b0 0x56297ee3fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d61dab640 1 -- 172.21.15.94:0/2412397420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297eddf7b0 msgr2=0x56297ee3fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d61dab640 1 --2- 172.21.15.94:0/2412397420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297eddf7b0 0x56297ee3fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d61dab640 1 -- 172.21.15.94:0/2412397420 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297edd6bd0 msgr2=0x56297ede5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d61dab640 1 --2- 172.21.15.94:0/2412397420 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297edd6bd0 0x56297ede5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d61dab640 1 -- 172.21.15.94:0/2412397420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5c1152d0 con 0x56297edd62b0 2024-02-17T12:12:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.641+0000 7f4d615aa640 1 --2- 172.21.15.94:0/2412397420 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297edd6bd0 0x56297ede5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d61dab640 1 --2- 172.21.15.94:0/2412397420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297ede5970 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f4d5009d010 tx=0x7f4d50005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fb7261243a84a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d417fa640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d417fa640 1 -- 172.21.15.94:0/2412397420 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50004380 con 0x56297edd62b0 2024-02-17T12:12:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d417fa640 1 -- 172.21.15.94:0/2412397420 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8629+0+0 (secure 0 0 0) 0x7f4d50097080 con 0x56297edd62b0 2024-02-17T12:12:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d66bda740 1 -- 172.21.15.94:0/2412397420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d4c01bca0 con 0x56297edd62b0 2024-02-17T12:12:12.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d417fa640 1 -- 172.21.15.94:0/2412397420 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d50097240 con 0x56297edd62b0 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d66bda740 10 client.19191 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d66bda740 1 -- 172.21.15.94:0/2412397420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56297ecfe6a0 con 0x56297edd62b0 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d417fa640 1 -- 172.21.15.94:0/2412397420 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d4c01bca0 con 0x56297edd62b0 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d417fa640 1 -- 172.21.15.94:0/2412397420 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4d50020070 con 0x56297edd62b0 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d66bda740 10 client.19191 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d66bda740 10 client.19191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d66bda740 20 client.19191 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.642+0000 7f4d66bda740 1 --2- 172.21.15.94:0/2412397420 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56297edef8f0 0x56297ee49fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.643+0000 7f4d66bda740 4 client.19191 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.643+0000 7f4d66bda740 1 -- 172.21.15.94:0/2412397420 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56297eac9890 con 0x56297edef8f0 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.643+0000 7f4d615aa640 1 --2- 172.21.15.94:0/2412397420 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56297edef8f0 0x56297ee49fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.643+0000 7f4d417fa640 10 client.19191 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.643+0000 7f4d615aa640 1 --2- 172.21.15.94:0/2412397420 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56297edef8f0 0x56297ee49fe0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adf010ec201067d3 server_cookie=292e40879264ab35 in_seq=0 out_seq=0 2024-02-17T12:12:12.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.643+0000 7f4d417fa640 1 -- 172.21.15.94:0/2412397420 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56297eac9890 con 0x56297edef8f0 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.643+0000 7f4d417fa640 10 client.19191 handle_command_reply: tid=0 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.643+0000 7f4d66bda740 1 client.19191 shutdown 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.643+0000 7f4d66bda740 1 -- 172.21.15.94:0/2412397420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 msgr2=0x56297ede5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.643+0000 7f4d66bda740 1 --2- 172.21.15.94:0/2412397420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297ede5970 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f4d5009d010 tx=0x7f4d50005800 comp rx=0 tx=0).stop 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.643+0000 7f4d66bda740 1 -- 172.21.15.94:0/2412397420 shutdown_connections 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.643+0000 7f4d66bda740 1 --2- 172.21.15.94:0/2412397420 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56297edef8f0 0x56297ee49fe0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d417fa640 0 client.19191 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d66bda740 1 --2- 172.21.15.94:0/2412397420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56297eddf7b0 0x56297ee3fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d66bda740 1 --2- 172.21.15.94:0/2412397420 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56297edd6bd0 0x56297ede5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d66bda740 1 --2- 172.21.15.94:0/2412397420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56297edd62b0 0x56297ede5970 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d66bda740 1 -- 172.21.15.94:0/2412397420 >> 172.21.15.94:0/2412397420 conn(0x56297edd0990 msgr2=0x56297eddae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d66bda740 1 -- 172.21.15.94:0/2412397420 shutdown_connections 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d66bda740 1 -- 172.21.15.94:0/2412397420 wait complete. 2024-02-17T12:12:12.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d66bda740 20 client.19191 trim_cache size 0 max 16384 2024-02-17T12:12:12.655 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:12:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-17T12:12:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:12:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "27cde575-219c-4113-968f-fc1d81a29a20": { 2024-02-17T12:12:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:12:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "27cde575-219c-4113-968f-fc1d81a29a20", 2024-02-17T12:12:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:12:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:12:12.655 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:12:12.655 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:12:12.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d6400f640 1 -- 172.21.15.94:0/4043577710 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4d440a2440 msgr2=0x7f4d440a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d6400f640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4d440a2440 0x7f4d440a4900 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4d5402d380 tx=0x7f4d54005fd0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d6400f640 1 -- 172.21.15.94:0/4043577710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d5c1083f0 msgr2=0x7f4d5c111bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:12.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d6400f640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d5c1083f0 0x7f4d5c111bb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4d5000c890 tx=0x7f4d5000cd50 comp rx=0 tx=0).stop 2024-02-17T12:12:12.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d6400f640 1 -- 172.21.15.94:0/4043577710 shutdown_connections 2024-02-17T12:12:12.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d6400f640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4d440a2440 0x7f4d440a4900 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d6400f640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4d5c112630 0x7f4d5c1b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d6400f640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4d5c10ceb0 0x7f4d5c1120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d6400f640 1 --2- 172.21.15.94:0/4043577710 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d5c1083f0 0x7f4d5c111bb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:12.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.644+0000 7f4d6400f640 1 -- 172.21.15.94:0/4043577710 >> 172.21.15.94:0/4043577710 conn(0x7f4d5c06c6d0 msgr2=0x7f4d5c070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:12.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.645+0000 7f4d6400f640 1 -- 172.21.15.94:0/4043577710 shutdown_connections 2024-02-17T12:12:12.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:12.645+0000 7f4d6400f640 1 -- 172.21.15.94:0/4043577710 wait complete. 2024-02-17T12:12:12.659 INFO:tasks.cephfs.filesystem:scrub status for tag:27cde575-219c-4113-968f-fc1d81a29a20 - {'path': '/', 'tag': '27cde575-219c-4113-968f-fc1d81a29a20', 'options': 'recursive,force'} 2024-02-17T12:12:12.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:12 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:12:12.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:12 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:12:12.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:12 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:12:12.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:12 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:12:12.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:12 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:12:12.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:12 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:12:12.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:12 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:12:12.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:12 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:12:12.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:12 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:12:13.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:13 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:12:13.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:13 smithi189 ceph-mon[36588]: pgmap v441: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 27 MiB/s wr, 1.84k op/s 2024-02-17T12:12:13.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:13 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:12:13.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:13 smithi125 ceph-mon[36819]: pgmap v441: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 27 MiB/s wr, 1.84k op/s 2024-02-17T12:12:13.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:13 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:12:13.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:13 smithi094 ceph-mon[32499]: pgmap v441: 97 pgs: 8 active+clean+snaptrim, 18 active+clean+snaptrim_wait, 71 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 27 MiB/s wr, 1.84k op/s 2024-02-17T12:12:15.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 835.0 s, 1658.2 tps, lat 60.053 ms stddev 67.857 2024-02-17T12:12:15.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:15 smithi189 ceph-mon[36588]: pgmap v442: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 28 MiB/s wr, 2.02k op/s 2024-02-17T12:12:15.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:15 smithi125 ceph-mon[36819]: pgmap v442: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 28 MiB/s wr, 2.02k op/s 2024-02-17T12:12:15.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:15 smithi094 ceph-mon[32499]: pgmap v442: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 28 MiB/s wr, 2.02k op/s 2024-02-17T12:12:16.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:16 smithi189 ceph-mon[36588]: osdmap e104: 12 total, 12 up, 12 in 2024-02-17T12:12:16.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:16 smithi125 ceph-mon[36819]: osdmap e104: 12 total, 12 up, 12 in 2024-02-17T12:12:16.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:16 smithi094 ceph-mon[32499]: osdmap e104: 12 total, 12 up, 12 in 2024-02-17T12:12:17.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:17 smithi189 ceph-mon[36588]: pgmap v444: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 29 MiB/s wr, 2.17k op/s 2024-02-17T12:12:17.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:17 smithi125 ceph-mon[36819]: pgmap v444: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 29 MiB/s wr, 2.17k op/s 2024-02-17T12:12:17.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:17 smithi094 ceph-mon[32499]: pgmap v444: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 29 MiB/s wr, 2.17k op/s 2024-02-17T12:12:18.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:18 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:12:18.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:18 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:12:18.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:18 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:12:19.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:19 smithi189 ceph-mon[36588]: pgmap v445: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-02-17T12:12:19.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:19 smithi125 ceph-mon[36819]: pgmap v445: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-02-17T12:12:19.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:19 smithi094 ceph-mon[32499]: pgmap v445: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-02-17T12:12:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 840.0 s, 1515.0 tps, lat 66.019 ms stddev 71.076 2024-02-17T12:12:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.736+0000 7f4194a23640 1 -- 172.21.15.94:0/1246626738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4190105390 msgr2=0x7f419010cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.736+0000 7f4194a23640 1 --2- 172.21.15.94:0/1246626738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4190105390 0x7f419010cf60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f4180009930 tx=0x7f418002bc20 comp rx=0 tx=0).stop 2024-02-17T12:12:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.736+0000 7f4194a23640 1 -- 172.21.15.94:0/1246626738 shutdown_connections 2024-02-17T12:12:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.736+0000 7f4194a23640 1 --2- 172.21.15.94:0/1246626738 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4190105390 0x7f419010cf60 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.736+0000 7f4194a23640 1 --2- 172.21.15.94:0/1246626738 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4190100c70 0x7f4190104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.736+0000 7f4194a23640 1 --2- 172.21.15.94:0/1246626738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41901002c0 0x7f41901006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.736+0000 7f4194a23640 1 -- 172.21.15.94:0/1246626738 >> 172.21.15.94:0/1246626738 conn(0x7f41900771b0 msgr2=0x7f41900775c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.736+0000 7f4194a23640 1 -- 172.21.15.94:0/1246626738 shutdown_connections 2024-02-17T12:12:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.736+0000 7f4194a23640 1 -- 172.21.15.94:0/1246626738 wait complete. 2024-02-17T12:12:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f4194a23640 1 Processor -- start 2024-02-17T12:12:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f4194a23640 1 -- start start 2024-02-17T12:12:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f4194a23640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41901002c0 0x7f4190196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f4194a23640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4190105390 0x7f4190196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f4194a23640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4190196de0 0x7f419019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f4194a23640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419019ba30 con 0x7f4190105390 2024-02-17T12:12:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f4194a23640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419019bba0 con 0x7f41901002c0 2024-02-17T12:12:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f4194a23640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419019bd60 con 0x7f4190196de0 2024-02-17T12:12:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f418dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4190105390 0x7f4190196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f418e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41901002c0 0x7f4190196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f418dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4190105390 0x7f4190196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48798/0 (socket says 172.21.15.94:48798) 2024-02-17T12:12:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f418e59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41901002c0 0x7f4190196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:54252/0 (socket says 172.21.15.94:54252) 2024-02-17T12:12:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f418dd9b640 1 -- 172.21.15.94:0/2674490446 learned_addr learned my addr 172.21.15.94:0/2674490446 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.737+0000 7f418ed9d640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4190196de0 0x7f419019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.738+0000 7f418dd9b640 1 -- 172.21.15.94:0/2674490446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4190196de0 msgr2=0x7f419019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.738+0000 7f418dd9b640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4190196de0 0x7f419019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.738+0000 7f418dd9b640 1 -- 172.21.15.94:0/2674490446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41901002c0 msgr2=0x7f4190196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.738+0000 7f418dd9b640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41901002c0 0x7f4190196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.738+0000 7f418dd9b640 1 -- 172.21.15.94:0/2674490446 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4180009560 con 0x7f4190105390 2024-02-17T12:12:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.738+0000 7f418e59c640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41901002c0 0x7f4190196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.738+0000 7f418dd9b640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4190105390 0x7f4190196850 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f418800b9c0 tx=0x7f418800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.738+0000 7f417f7fe640 1 -- 172.21.15.94:0/2674490446 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418800cac0 con 0x7f4190105390 2024-02-17T12:12:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.738+0000 7f417f7fe640 1 -- 172.21.15.94:0/2674490446 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f418800cc20 con 0x7f4190105390 2024-02-17T12:12:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.738+0000 7f4194a23640 1 -- 172.21.15.94:0/2674490446 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419019c040 con 0x7f4190105390 2024-02-17T12:12:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.738+0000 7f417f7fe640 1 -- 172.21.15.94:0/2674490446 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41880126b0 con 0x7f4190105390 2024-02-17T12:12:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.738+0000 7f4194a23640 1 -- 172.21.15.94:0/2674490446 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419019c5a0 con 0x7f4190105390 2024-02-17T12:12:20.756 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.741+0000 7f417f7fe640 1 -- 172.21.15.94:0/2674490446 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4188014440 con 0x7f4190105390 2024-02-17T12:12:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.741+0000 7f41975fc740 1 Processor -- start 2024-02-17T12:12:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.742+0000 7f41975fc740 1 -- start start 2024-02-17T12:12:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.742+0000 7f417f7fe640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f41780a2350 0x7f41780a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.742+0000 7f41975fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c0ad0 0x55db4a4bf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.742+0000 7f417f7fe640 1 -- 172.21.15.94:0/2674490446 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f41880cd730 con 0x7f4190105390 2024-02-17T12:12:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.743+0000 7f41975fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4bfd60 0x55db4a4c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.743+0000 7f418ed9d640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f41780a2350 0x7f41780a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.743+0000 7f418e59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c0ad0 0x55db4a4bf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.743+0000 7f418e59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c0ad0 0x55db4a4bf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:32776/0 (socket says 172.21.15.94:32776) 2024-02-17T12:12:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.743+0000 7f41975fc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c92e0 0x55db4a4cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.743+0000 7f41975fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4a42e9f0 con 0x55db4a4bfd60 2024-02-17T12:12:20.757 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.743+0000 7f41975fc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4a42bf20 con 0x55db4a4c92e0 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.743+0000 7f41975fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4a427700 con 0x55db4a4c0ad0 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.743+0000 7f418e59c640 1 -- 172.21.15.94:0/2157817201 learned_addr learned my addr 172.21.15.94:0/2157817201 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.744+0000 7f418dd9b640 1 --2- 172.21.15.94:0/2157817201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4bfd60 0x55db4a4c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.744+0000 7f418ed9d640 1 --2- 172.21.15.94:0/2157817201 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c92e0 0x55db4a4cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.744+0000 7f418ed9d640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f41780a2350 0x7f41780a4810 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4180036040 tx=0x7f4180005a20 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.744+0000 7f418dd9b640 1 -- 172.21.15.94:0/2157817201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c0ad0 msgr2=0x55db4a4bf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.744+0000 7f418dd9b640 1 --2- 172.21.15.94:0/2157817201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c0ad0 0x55db4a4bf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.744+0000 7f418dd9b640 1 -- 172.21.15.94:0/2157817201 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c92e0 msgr2=0x55db4a4cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.744+0000 7f418dd9b640 1 --2- 172.21.15.94:0/2157817201 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c92e0 0x55db4a4cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.744+0000 7f418dd9b640 1 -- 172.21.15.94:0/2157817201 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db4a3bd7b0 con 0x55db4a4bfd60 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.744+0000 7f418dd9b640 1 --2- 172.21.15.94:0/2157817201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4bfd60 0x55db4a4c01e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f41880caf50 tx=0x7f418809a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ab1e46bdf6856f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.745+0000 7f417cff9640 1 -- 172.21.15.94:0/2157817201 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418809e120 con 0x55db4a4bfd60 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.745+0000 7f417cff9640 1 -- 172.21.15.94:0/2157817201 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41880985a0 con 0x55db4a4bfd60 2024-02-17T12:12:20.758 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.745+0000 7f417cff9640 1 -- 172.21.15.94:0/2157817201 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41880146f0 con 0x55db4a4bfd60 2024-02-17T12:12:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.745+0000 7f41975fc740 1 -- 172.21.15.94:0/2157817201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4bfd60 msgr2=0x55db4a4c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.745+0000 7f41975fc740 1 --2- 172.21.15.94:0/2157817201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4bfd60 0x55db4a4c01e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f41880caf50 tx=0x7f418809a2a0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.745+0000 7f41975fc740 1 -- 172.21.15.94:0/2157817201 shutdown_connections 2024-02-17T12:12:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.745+0000 7f41975fc740 1 --2- 172.21.15.94:0/2157817201 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c92e0 0x55db4a4cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.745+0000 7f41975fc740 1 --2- 172.21.15.94:0/2157817201 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4bfd60 0x55db4a4c01e0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.745+0000 7f41975fc740 1 --2- 172.21.15.94:0/2157817201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c0ad0 0x55db4a4bf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.745+0000 7f41975fc740 1 -- 172.21.15.94:0/2157817201 >> 172.21.15.94:0/2157817201 conn(0x55db4a487550 msgr2=0x55db4a4baa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.746+0000 7f41975fc740 1 -- 172.21.15.94:0/2157817201 shutdown_connections 2024-02-17T12:12:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.746+0000 7f41975fc740 1 -- 172.21.15.94:0/2157817201 wait complete. 2024-02-17T12:12:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.746+0000 7f41975fc740 1 -- start start 2024-02-17T12:12:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.746+0000 7f41975fc740 1 Processor -- start 2024-02-17T12:12:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.746+0000 7f41975fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4bfd60 0x55db4a4c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.759 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f418e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4bfd60 0x55db4a4c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f418e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4bfd60 0x55db4a4c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48824/0 (socket says 172.21.15.94:48824) 2024-02-17T12:12:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f418e59c640 1 -- 172.21.15.94:0/2336314534 learned_addr learned my addr 172.21.15.94:0/2336314534 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f41975fc740 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c0ad0 0x55db4a4c6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f41975fc740 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c92e0 0x55db4a4c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f41975fc740 1 -- 172.21.15.94:0/2336314534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4a3ccbf0 con 0x55db4a4bfd60 2024-02-17T12:12:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f41975fc740 1 -- 172.21.15.94:0/2336314534 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4a0fb250 con 0x55db4a4c0ad0 2024-02-17T12:12:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f41975fc740 1 -- 172.21.15.94:0/2336314534 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4a3ad200 con 0x55db4a4c92e0 2024-02-17T12:12:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f418dd9b640 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c0ad0 0x55db4a4c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f41975fc740 10 client.0 _ll_register_callbacks cb 0x55db4a41bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f418ed9d640 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c92e0 0x55db4a4c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f418e59c640 1 -- 172.21.15.94:0/2336314534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c92e0 msgr2=0x55db4a4c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:20.760 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f418e59c640 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c92e0 0x55db4a4c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f418e59c640 1 -- 172.21.15.94:0/2336314534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c0ad0 msgr2=0x55db4a4c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f418e59c640 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c0ad0 0x55db4a4c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f418e59c640 1 -- 172.21.15.94:0/2336314534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419019c5a0 con 0x55db4a4bfd60 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f418ed9d640 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c92e0 0x55db4a4c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f418e59c640 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4bfd60 0x55db4a4c63e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f4184012f30 tx=0x7f41840107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e394493f2a5c24c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.747+0000 7f4175ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.748+0000 7f4175ffb640 1 -- 172.21.15.94:0/2336314534 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418400abd0 con 0x55db4a4bfd60 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.748+0000 7f4175ffb640 1 -- 172.21.15.94:0/2336314534 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f418401a410 con 0x55db4a4bfd60 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.748+0000 7f41975fc740 1 -- 172.21.15.94:0/2336314534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db4a127600 con 0x55db4a4bfd60 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.748+0000 7f4175ffb640 1 -- 172.21.15.94:0/2336314534 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f418401a5d0 con 0x55db4a4bfd60 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.748+0000 7f41975fc740 10 client.19203 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.748+0000 7f41975fc740 1 -- 172.21.15.94:0/2336314534 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55db4a425d80 con 0x55db4a4bfd60 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.749+0000 7f4175ffb640 1 -- 172.21.15.94:0/2336314534 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f418401ac10 con 0x55db4a4bfd60 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.749+0000 7f4175ffb640 1 -- 172.21.15.94:0/2336314534 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4184022070 con 0x55db4a4bfd60 2024-02-17T12:12:20.761 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.749+0000 7f41975fc740 10 client.19203 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.749+0000 7f41975fc740 10 client.19203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.749+0000 7f41975fc740 20 client.19203 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.749+0000 7f41975fc740 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db4a4d9640 0x55db4a4dbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.749+0000 7f41975fc740 4 client.19203 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.749+0000 7f41975fc740 1 -- 172.21.15.94:0/2336314534 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db4a127600 con 0x55db4a4d9640 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.749+0000 7f418dd9b640 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db4a4d9640 0x55db4a4dbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.750+0000 7f418dd9b640 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db4a4d9640 0x55db4a4dbae0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5aac123e43407f1 server_cookie=6baf69d4e3de4d46 in_seq=0 out_seq=0 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.750+0000 7f4175ffb640 10 client.19203 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.752+0000 7f4175ffb640 1 -- 172.21.15.94:0/2336314534 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db4a127600 con 0x55db4a4d9640 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.752+0000 7f4175ffb640 10 client.19203 handle_command_reply: tid=0 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.752+0000 7f41975fc740 1 client.19203 shutdown 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.752+0000 7f41975fc740 1 -- 172.21.15.94:0/2336314534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4bfd60 msgr2=0x55db4a4c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.752+0000 7f41975fc740 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4bfd60 0x55db4a4c63e0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f4184012f30 tx=0x7f41840107a0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.762 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.752+0000 7f41975fc740 1 -- 172.21.15.94:0/2336314534 shutdown_connections 2024-02-17T12:12:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.752+0000 7f41975fc740 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db4a4d9640 0x55db4a4dbae0 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.752+0000 7f41975fc740 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c92e0 0x55db4a4c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.752+0000 7f41975fc740 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c0ad0 0x55db4a4c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.752+0000 7f4175ffb640 0 client.19203 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.752+0000 7f41975fc740 1 --2- 172.21.15.94:0/2336314534 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4bfd60 0x55db4a4c63e0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.752+0000 7f41975fc740 1 -- 172.21.15.94:0/2336314534 >> 172.21.15.94:0/2336314534 conn(0x55db4a487550 msgr2=0x55db4a4bc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.753+0000 7f41975fc740 1 -- 172.21.15.94:0/2336314534 shutdown_connections 2024-02-17T12:12:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.753+0000 7f41975fc740 1 -- 172.21.15.94:0/2336314534 wait complete. 2024-02-17T12:12:20.763 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.753+0000 7f41975fc740 20 client.19203 trim_cache size 0 max 16384 2024-02-17T12:12:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.770+0000 7f41975fc740 1 Processor -- start 2024-02-17T12:12:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.770+0000 7f41975fc740 1 -- start start 2024-02-17T12:12:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f41975fc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4bfd60 0x55db4a4c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f41975fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c0ad0 0x55db4a4d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f41975fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4c92e0 0x55db4a4d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f41975fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4a4c7730 con 0x55db4a4c92e0 2024-02-17T12:12:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f41975fc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4a4d9440 con 0x55db4a4bfd60 2024-02-17T12:12:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f41975fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4a4c61d0 con 0x55db4a4c0ad0 2024-02-17T12:12:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f418ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4c92e0 0x55db4a4d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f418ed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4c92e0 0x55db4a4d05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:48838/0 (socket says 172.21.15.94:48838) 2024-02-17T12:12:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f418ed9d640 1 -- 172.21.15.94:0/1595454941 learned_addr learned my addr 172.21.15.94:0/1595454941 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f418dd9b640 1 --2- 172.21.15.94:0/1595454941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c0ad0 0x55db4a4d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.774 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f418e59c640 1 --2- 172.21.15.94:0/1595454941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4bfd60 0x55db4a4c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f418ed9d640 1 -- 172.21.15.94:0/1595454941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c0ad0 msgr2=0x55db4a4d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f418ed9d640 1 --2- 172.21.15.94:0/1595454941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c0ad0 0x55db4a4d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f418ed9d640 1 -- 172.21.15.94:0/1595454941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4bfd60 msgr2=0x55db4a4c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f418ed9d640 1 --2- 172.21.15.94:0/1595454941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4bfd60 0x55db4a4c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f418ed9d640 1 -- 172.21.15.94:0/1595454941 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db4a3c27b0 con 0x55db4a4c92e0 2024-02-17T12:12:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f418e59c640 1 --2- 172.21.15.94:0/1595454941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4bfd60 0x55db4a4c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:12:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.771+0000 7f418ed9d640 1 --2- 172.21.15.94:0/1595454941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4c92e0 0x55db4a4d05e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4180035000 tx=0x7f418003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f561ed5e0be5ea server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.772+0000 7f4177fff640 1 -- 172.21.15.94:0/1595454941 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418002dea0 con 0x55db4a4c92e0 2024-02-17T12:12:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.772+0000 7f4177fff640 1 -- 172.21.15.94:0/1595454941 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4180004df0 con 0x55db4a4c92e0 2024-02-17T12:12:20.775 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.772+0000 7f4177fff640 1 -- 172.21.15.94:0/1595454941 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4180003920 con 0x55db4a4c92e0 2024-02-17T12:12:20.778 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.772+0000 7f41975fc740 1 -- 172.21.15.94:0/1595454941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4c92e0 msgr2=0x55db4a4d05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.772+0000 7f41975fc740 1 --2- 172.21.15.94:0/1595454941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4c92e0 0x55db4a4d05e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4180035000 tx=0x7f418003f000 comp rx=0 tx=0).stop 2024-02-17T12:12:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.772+0000 7f41975fc740 1 -- 172.21.15.94:0/1595454941 shutdown_connections 2024-02-17T12:12:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.772+0000 7f41975fc740 1 --2- 172.21.15.94:0/1595454941 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4c92e0 0x55db4a4d05e0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.772+0000 7f41975fc740 1 --2- 172.21.15.94:0/1595454941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4c0ad0 0x55db4a4d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.772+0000 7f41975fc740 1 --2- 172.21.15.94:0/1595454941 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4bfd60 0x55db4a4c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.772+0000 7f41975fc740 1 -- 172.21.15.94:0/1595454941 >> 172.21.15.94:0/1595454941 conn(0x55db4a487550 msgr2=0x55db4a4ba580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.773+0000 7f41975fc740 1 -- 172.21.15.94:0/1595454941 shutdown_connections 2024-02-17T12:12:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.773+0000 7f41975fc740 1 -- 172.21.15.94:0/1595454941 wait complete. 2024-02-17T12:12:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.773+0000 7f41975fc740 1 -- start start 2024-02-17T12:12:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.773+0000 7f41975fc740 1 Processor -- start 2024-02-17T12:12:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.773+0000 7f41975fc740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4bfd60 0x55db4a4d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.773+0000 7f41975fc740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c0ad0 0x55db4a4ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.779 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.773+0000 7f41975fc740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4c92e0 0x55db4a4ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.773+0000 7f41975fc740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4a4d05f0 con 0x55db4a4c92e0 2024-02-17T12:12:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.773+0000 7f41975fc740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4a4d0730 con 0x55db4a4c0ad0 2024-02-17T12:12:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.773+0000 7f41975fc740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db4a4d08f0 con 0x55db4a4bfd60 2024-02-17T12:12:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f418dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c0ad0 0x55db4a4ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f418dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c0ad0 0x55db4a4ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:54306/0 (socket says 172.21.15.94:54306) 2024-02-17T12:12:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f418dd9b640 1 -- 172.21.15.94:0/447386977 learned_addr learned my addr 172.21.15.94:0/447386977 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f418e59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4bfd60 0x55db4a4d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f418ed9d640 1 --2- 172.21.15.94:0/447386977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4c92e0 0x55db4a4ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f41975fc740 10 client.0 _ll_register_callbacks cb 0x7f4188014d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f418e59c640 1 -- 172.21.15.94:0/447386977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c0ad0 msgr2=0x55db4a4ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:20.780 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f418e59c640 1 --2- 172.21.15.94:0/447386977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c0ad0 0x55db4a4ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f418e59c640 1 -- 172.21.15.94:0/447386977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4c92e0 msgr2=0x55db4a4ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f418e59c640 1 --2- 172.21.15.94:0/447386977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4c92e0 0x55db4a4ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f418e59c640 1 -- 172.21.15.94:0/447386977 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419010d770 con 0x55db4a4bfd60 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f418ed9d640 1 --2- 172.21.15.94:0/447386977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4c92e0 0x55db4a4ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f418e59c640 1 --2- 172.21.15.94:0/447386977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4bfd60 0x55db4a4d00b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4184001eb0 tx=0x7f41840128a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab7974bc37daaf6c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f4175ffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f4175ffb640 1 -- 172.21.15.94:0/447386977 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4184019880 con 0x55db4a4bfd60 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f41975fc740 1 -- 172.21.15.94:0/447386977 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41880985a0 con 0x55db4a4bfd60 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.774+0000 7f4175ffb640 1 -- 172.21.15.94:0/447386977 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4184000e40 con 0x55db4a4bfd60 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.775+0000 7f4175ffb640 1 -- 172.21.15.94:0/447386977 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41840199e0 con 0x55db4a4bfd60 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.775+0000 7f41975fc740 10 client.28630 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.775+0000 7f41975fc740 1 -- 172.21.15.94:0/447386977 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55db4a3e8540 con 0x55db4a4bfd60 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.775+0000 7f4175ffb640 1 -- 172.21.15.94:0/447386977 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f41880985a0 con 0x55db4a4bfd60 2024-02-17T12:12:20.781 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.775+0000 7f4175ffb640 1 -- 172.21.15.94:0/447386977 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4184022040 con 0x55db4a4bfd60 2024-02-17T12:12:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.775+0000 7f41975fc740 10 client.28630 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.775+0000 7f41975fc740 10 client.28630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.775+0000 7f41975fc740 20 client.28630 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.775+0000 7f41975fc740 1 --2- 172.21.15.94:0/447386977 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db4a4d9640 0x55db4a533bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.775+0000 7f41975fc740 4 client.28630 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:12:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.775+0000 7f41975fc740 1 -- 172.21.15.94:0/447386977 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55db4a1781b0 con 0x55db4a4d9640 2024-02-17T12:12:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.775+0000 7f418dd9b640 1 --2- 172.21.15.94:0/447386977 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db4a4d9640 0x55db4a533bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.776+0000 7f418dd9b640 1 --2- 172.21.15.94:0/447386977 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db4a4d9640 0x55db4a533bd0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=249a0c03e8adec5a server_cookie=ad38173c461d2a36 in_seq=0 out_seq=0 2024-02-17T12:12:20.782 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.776+0000 7f4175ffb640 10 client.28630 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.813+0000 7f4175ffb640 1 -- 172.21.15.94:0/447386977 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55db4a1781b0 con 0x55db4a4d9640 2024-02-17T12:12:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.813+0000 7f4175ffb640 10 client.28630 handle_command_reply: tid=0 2024-02-17T12:12:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.813+0000 7f41975fc740 1 client.28630 shutdown 2024-02-17T12:12:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.813+0000 7f41975fc740 1 -- 172.21.15.94:0/447386977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4bfd60 msgr2=0x55db4a4d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:20.817 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.813+0000 7f41975fc740 1 --2- 172.21.15.94:0/447386977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4bfd60 0x55db4a4d00b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4184001eb0 tx=0x7f41840128a0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.813+0000 7f41975fc740 1 -- 172.21.15.94:0/447386977 shutdown_connections 2024-02-17T12:12:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.813+0000 7f41975fc740 1 --2- 172.21.15.94:0/447386977 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55db4a4d9640 0x55db4a533bd0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.813+0000 7f41975fc740 1 --2- 172.21.15.94:0/447386977 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55db4a4c92e0 0x55db4a4ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.813+0000 7f4175ffb640 0 client.28630 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.813+0000 7f41975fc740 1 --2- 172.21.15.94:0/447386977 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55db4a4c0ad0 0x55db4a4ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.818 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.813+0000 7f41975fc740 1 --2- 172.21.15.94:0/447386977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55db4a4bfd60 0x55db4a4d00b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.813+0000 7f41975fc740 1 -- 172.21.15.94:0/447386977 >> 172.21.15.94:0/447386977 conn(0x55db4a487550 msgr2=0x55db4a4c4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f41975fc740 1 -- 172.21.15.94:0/447386977 shutdown_connections 2024-02-17T12:12:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f41975fc740 1 -- 172.21.15.94:0/447386977 wait complete. 2024-02-17T12:12:20.819 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f41975fc740 20 client.28630 trim_cache size 0 max 16384 2024-02-17T12:12:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:12:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:12:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:12:20.820 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:12:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f4194a23640 1 -- 172.21.15.94:0/2674490446 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f41780a2350 msgr2=0x7f41780a4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f4194a23640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f41780a2350 0x7f41780a4810 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4180036040 tx=0x7f4180005a20 comp rx=0 tx=0).stop 2024-02-17T12:12:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f4194a23640 1 -- 172.21.15.94:0/2674490446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4190105390 msgr2=0x7f4190196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f4194a23640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4190105390 0x7f4190196850 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f418800b9c0 tx=0x7f418800bdc0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.822 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f4194a23640 1 -- 172.21.15.94:0/2674490446 shutdown_connections 2024-02-17T12:12:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f4194a23640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f41780a2350 0x7f41780a4810 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f4194a23640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4190196de0 0x7f419019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f4194a23640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4190105390 0x7f4190196850 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f4194a23640 1 --2- 172.21.15.94:0/2674490446 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f41901002c0 0x7f4190196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f4194a23640 1 -- 172.21.15.94:0/2674490446 >> 172.21.15.94:0/2674490446 conn(0x7f41900771b0 msgr2=0x7f41900fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:20.823 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f4194a23640 1 -- 172.21.15.94:0/2674490446 shutdown_connections 2024-02-17T12:12:20.824 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:20.814+0000 7f4194a23640 1 -- 172.21.15.94:0/2674490446 wait complete. 2024-02-17T12:12:21.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:21 smithi189 ceph-mon[36588]: pgmap v446: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.6 MiB/s wr, 712 op/s 2024-02-17T12:12:21.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:21 smithi125 ceph-mon[36819]: pgmap v446: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.6 MiB/s wr, 712 op/s 2024-02-17T12:12:21.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:21 smithi094 ceph-mon[32499]: pgmap v446: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.6 MiB/s wr, 712 op/s 2024-02-17T12:12:23.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:23 smithi189 ceph-mon[36588]: pgmap v447: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.6 MiB/s wr, 712 op/s 2024-02-17T12:12:23.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:23 smithi125 ceph-mon[36819]: pgmap v447: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.6 MiB/s wr, 712 op/s 2024-02-17T12:12:23.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:23 smithi094 ceph-mon[32499]: pgmap v447: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 5.6 MiB/s wr, 712 op/s 2024-02-17T12:12:25.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 845.0 s, 1480.6 tps, lat 67.426 ms stddev 79.457 2024-02-17T12:12:25.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:25 smithi189 ceph-mon[36588]: pgmap v448: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.0 MiB/s wr, 672 op/s 2024-02-17T12:12:25.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:25 smithi125 ceph-mon[36819]: pgmap v448: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.0 MiB/s wr, 672 op/s 2024-02-17T12:12:25.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:25 smithi094 ceph-mon[32499]: pgmap v448: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.0 MiB/s wr, 672 op/s 2024-02-17T12:12:27.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:27 smithi189 ceph-mon[36588]: pgmap v449: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.3 MiB/s wr, 599 op/s 2024-02-17T12:12:27.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:27 smithi125 ceph-mon[36819]: pgmap v449: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.3 MiB/s wr, 599 op/s 2024-02-17T12:12:27.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:27 smithi094 ceph-mon[32499]: pgmap v449: 97 pgs: 97 active+clean; 5.2 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.3 MiB/s wr, 599 op/s 2024-02-17T12:12:29.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:29 smithi189 ceph-mon[36588]: pgmap v450: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.3 MiB/s wr, 696 op/s 2024-02-17T12:12:29.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:29 smithi125 ceph-mon[36819]: pgmap v450: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.3 MiB/s wr, 696 op/s 2024-02-17T12:12:29.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:29 smithi094 ceph-mon[32499]: pgmap v450: 97 pgs: 97 active+clean; 5.3 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 6.3 MiB/s wr, 696 op/s 2024-02-17T12:12:30.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 850.0 s, 1371.6 tps, lat 73.214 ms stddev 79.604 2024-02-17T12:12:31.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:31 smithi189 ceph-mon[36588]: pgmap v451: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.5 MiB/s wr, 622 op/s 2024-02-17T12:12:31.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:31 smithi125 ceph-mon[36819]: pgmap v451: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.5 MiB/s wr, 622 op/s 2024-02-17T12:12:31.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:31 smithi094 ceph-mon[32499]: pgmap v451: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.5 MiB/s wr, 622 op/s 2024-02-17T12:12:32.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:32 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:12:32.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:32 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:12:32.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:32 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:12:33.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:33 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:12:33.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:33 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:12:33.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:33 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:12:33.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:33 smithi189 ceph-mon[36588]: pgmap v452: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.9 MiB/s wr, 546 op/s 2024-02-17T12:12:33.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:33 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:12:33.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:33 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:12:33.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:33 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:12:33.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:33 smithi125 ceph-mon[36819]: pgmap v452: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.9 MiB/s wr, 546 op/s 2024-02-17T12:12:33.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:33 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:12:33.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:33 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:12:33.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:33 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:12:33.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:33 smithi094 ceph-mon[32499]: pgmap v452: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.9 MiB/s wr, 546 op/s 2024-02-17T12:12:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.964+0000 7fdfd3325640 1 -- 172.21.15.94:0/1772175715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfcc105390 msgr2=0x7fdfcc10cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.964+0000 7fdfd3325640 1 --2- 172.21.15.94:0/1772175715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfcc105390 0x7fdfcc10cf60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4009be0 tx=0x7fdfc402bef0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.964+0000 7fdfd3325640 1 -- 172.21.15.94:0/1772175715 shutdown_connections 2024-02-17T12:12:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.964+0000 7fdfd3325640 1 --2- 172.21.15.94:0/1772175715 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfcc105390 0x7fdfcc10cf60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.964+0000 7fdfd3325640 1 --2- 172.21.15.94:0/1772175715 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfcc100c70 0x7fdfcc104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.964+0000 7fdfd3325640 1 --2- 172.21.15.94:0/1772175715 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfcc1002c0 0x7fdfcc1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.964+0000 7fdfd3325640 1 -- 172.21.15.94:0/1772175715 >> 172.21.15.94:0/1772175715 conn(0x7fdfcc0fbb40 msgr2=0x7fdfcc0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.966+0000 7fdfd3325640 1 -- 172.21.15.94:0/1772175715 shutdown_connections 2024-02-17T12:12:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.966+0000 7fdfd3325640 1 -- 172.21.15.94:0/1772175715 wait complete. 2024-02-17T12:12:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.968+0000 7fdfd3325640 1 Processor -- start 2024-02-17T12:12:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.968+0000 7fdfd3325640 1 -- start start 2024-02-17T12:12:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.968+0000 7fdfd3325640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfcc1002c0 0x7fdfcc1952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.968+0000 7fdfd3325640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfcc105390 0x7fdfcc195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.969+0000 7fdfd3325640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfcc199b80 0x7fdfcc19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.969+0000 7fdfd08c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfcc105390 0x7fdfcc195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.969+0000 7fdfd3325640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc19a570 con 0x7fdfcc105390 2024-02-17T12:12:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.969+0000 7fdfd08c0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfcc105390 0x7fdfcc195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50308/0 (socket says 172.21.15.94:50308) 2024-02-17T12:12:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.969+0000 7fdfd3325640 1 -- 172.21.15.94:0/1256613911 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc19a6e0 con 0x7fdfcc199b80 2024-02-17T12:12:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.969+0000 7fdfd08c0640 1 -- 172.21.15.94:0/1256613911 learned_addr learned my addr 172.21.15.94:0/1256613911 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.969+0000 7fdfd3325640 1 -- 172.21.15.94:0/1256613911 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc19a850 con 0x7fdfcc1002c0 2024-02-17T12:12:33.972 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.969+0000 7fdfd18c2640 1 --2- 172.21.15.94:0/1256613911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfcc199b80 0x7fdfcc19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.969+0000 7fdfd18c2640 1 -- 172.21.15.94:0/1256613911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfcc1002c0 msgr2=0x7fdfcc1952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.969+0000 7fdfd18c2640 1 --2- 172.21.15.94:0/1256613911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfcc1002c0 0x7fdfcc1952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.969+0000 7fdfd18c2640 1 -- 172.21.15.94:0/1256613911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfcc105390 msgr2=0x7fdfcc195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.969+0000 7fdfd18c2640 1 --2- 172.21.15.94:0/1256613911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfcc105390 0x7fdfcc195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.969+0000 7fdfd18c2640 1 -- 172.21.15.94:0/1256613911 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc4009840 con 0x7fdfcc199b80 2024-02-17T12:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.970+0000 7fdfd18c2640 1 --2- 172.21.15.94:0/1256613911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfcc199b80 0x7fdfcc19a030 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4036040 tx=0x7fdfc4033610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.970+0000 7fdfc27fc640 1 -- 172.21.15.94:0/1256613911 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4003b70 con 0x7fdfcc199b80 2024-02-17T12:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.970+0000 7fdfc27fc640 1 -- 172.21.15.94:0/1256613911 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfc4003cd0 con 0x7fdfcc199b80 2024-02-17T12:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.970+0000 7fdfc27fc640 1 -- 172.21.15.94:0/1256613911 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40348d0 con 0x7fdfcc199b80 2024-02-17T12:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.971+0000 7fdfd3325640 1 -- 172.21.15.94:0/1256613911 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfcc1a1e30 con 0x7fdfcc199b80 2024-02-17T12:12:33.973 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.971+0000 7fdfd3325640 1 -- 172.21.15.94:0/1256613911 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfcc1a21a0 con 0x7fdfcc199b80 2024-02-17T12:12:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.973+0000 7fdfc27fc640 1 -- 172.21.15.94:0/1256613911 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fdfc4034a30 con 0x7fdfcc199b80 2024-02-17T12:12:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfc27fc640 1 --2- 172.21.15.94:0/1256613911 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdfb80a6840 0x7fdfb80a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfd5fd1740 1 Processor -- start 2024-02-17T12:12:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfd5fd1740 1 -- start start 2024-02-17T12:12:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfc27fc640 1 -- 172.21.15.94:0/1256613911 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdfc40f0a70 con 0x7fdfcc199b80 2024-02-17T12:12:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfd5fd1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c14ad0 0x55b852c13820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfd5fd1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c13d60 0x55b852c141e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfd5fd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c306e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfd5fd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b852b829f0 con 0x55b852c2e2f0 2024-02-17T12:12:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfd5fd1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b852b7ff20 con 0x55b852c14ad0 2024-02-17T12:12:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfd5fd1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b852b7b700 con 0x55b852c13d60 2024-02-17T12:12:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfd10c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c14ad0 0x55b852c13820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfd10c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c14ad0 0x55b852c13820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:35676/0 (socket says 172.21.15.94:35676) 2024-02-17T12:12:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfd10c1640 1 -- 172.21.15.94:0/2074740415 learned_addr learned my addr 172.21.15.94:0/2074740415 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfd10c1640 1 --2- 172.21.15.94:0/1256613911 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdfb80a6840 0x7fdfb80a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.975+0000 7fdfd08c0640 1 --2- 172.21.15.94:0/2074740415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c13d60 0x55b852c141e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.976+0000 7fdfd18c2640 1 --2- 172.21.15.94:0/2074740415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c306e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.976+0000 7fdfd10c1640 1 -- 172.21.15.94:0/2074740415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c13d60 msgr2=0x55b852c141e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.976+0000 7fdfd10c1640 1 --2- 172.21.15.94:0/2074740415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c13d60 0x55b852c141e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.976+0000 7fdfd10c1640 1 -- 172.21.15.94:0/2074740415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 msgr2=0x55b852c306e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:33.982 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.976+0000 7fdfd10c1640 1 --2- 172.21.15.94:0/2074740415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c306e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.976+0000 7fdfd10c1640 1 -- 172.21.15.94:0/2074740415 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b852b117b0 con 0x55b852c14ad0 2024-02-17T12:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.976+0000 7fdfd10c1640 1 --2- 172.21.15.94:0/2074740415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c14ad0 0x55b852c13820 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc013b20 tx=0x7fdfbc0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a09f81280d378cc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.976+0000 7fdfd10c1640 1 --2- 172.21.15.94:0/1256613911 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdfb80a6840 0x7fdfb80a8d00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0108d0 tx=0x7fdfbc00b000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.976+0000 7fdfb77fe640 1 -- 172.21.15.94:0/2074740415 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc004300 con 0x55b852c14ad0 2024-02-17T12:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.976+0000 7fdfb77fe640 1 -- 172.21.15.94:0/2074740415 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfbc019040 con 0x55b852c14ad0 2024-02-17T12:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.977+0000 7fdfb77fe640 1 -- 172.21.15.94:0/2074740415 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc00eab0 con 0x55b852c14ad0 2024-02-17T12:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.977+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/2074740415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c14ad0 msgr2=0x55b852c13820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.977+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/2074740415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c14ad0 0x55b852c13820 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc013b20 tx=0x7fdfbc0103d0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.977+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/2074740415 shutdown_connections 2024-02-17T12:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.977+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/2074740415 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c306e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.977+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/2074740415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c13d60 0x55b852c141e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.977+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/2074740415 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c14ad0 0x55b852c13820 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.983 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.977+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/2074740415 >> 172.21.15.94:0/2074740415 conn(0x55b852bdb550 msgr2=0x55b852c0ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.977+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/2074740415 shutdown_connections 2024-02-17T12:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.977+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/2074740415 wait complete. 2024-02-17T12:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.977+0000 7fdfd5fd1740 1 -- start start 2024-02-17T12:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd5fd1740 1 Processor -- start 2024-02-17T12:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd5fd1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c13d60 0x55b852c1a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd5fd1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c14ad0 0x55b852c1a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd5fd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c1adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd5fd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b852b20bf0 con 0x55b852c2e2f0 2024-02-17T12:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd5fd1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85284f250 con 0x55b852c13d60 2024-02-17T12:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd5fd1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b852b01200 con 0x55b852c14ad0 2024-02-17T12:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd5fd1740 10 client.0 _ll_register_callbacks cb 0x55b852b6fd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd18c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c1adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd18c2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c1adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50326/0 (socket says 172.21.15.94:50326) 2024-02-17T12:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd18c2640 1 -- 172.21.15.94:0/2255553780 learned_addr learned my addr 172.21.15.94:0/2255553780 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd10c1640 1 --2- 172.21.15.94:0/2255553780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c13d60 0x55b852c1a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd08c0640 1 --2- 172.21.15.94:0/2255553780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c14ad0 0x55b852c1a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd18c2640 1 -- 172.21.15.94:0/2255553780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c14ad0 msgr2=0x55b852c1a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd18c2640 1 --2- 172.21.15.94:0/2255553780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c14ad0 0x55b852c1a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd18c2640 1 -- 172.21.15.94:0/2255553780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c13d60 msgr2=0x55b852c1a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd18c2640 1 --2- 172.21.15.94:0/2255553780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c13d60 0x55b852c1a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.978+0000 7fdfd18c2640 1 -- 172.21.15.94:0/2255553780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfbc013700 con 0x55b852c2e2f0 2024-02-17T12:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.979+0000 7fdfd18c2640 1 --2- 172.21.15.94:0/2255553780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c1adf0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40b9930 tx=0x7fdfc40bce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ca7aaf1d52c37b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.979+0000 7fdfb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:33.985 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.979+0000 7fdfb4ff9640 1 -- 172.21.15.94:0/2255553780 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40ee5a0 con 0x55b852c2e2f0 2024-02-17T12:12:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.979+0000 7fdfb4ff9640 1 -- 172.21.15.94:0/2255553780 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdfc40bd4d0 con 0x55b852c2e2f0 2024-02-17T12:12:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.979+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/2255553780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b85287b600 con 0x55b852c2e2f0 2024-02-17T12:12:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.979+0000 7fdfb4ff9640 1 -- 172.21.15.94:0/2255553780 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfc40c1620 con 0x55b852c2e2f0 2024-02-17T12:12:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.979+0000 7fdfd5fd1740 10 client.19221 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:33.986 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.979+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/2255553780 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b852b79d80 con 0x55b852c2e2f0 2024-02-17T12:12:33.987 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.981+0000 7fdfb4ff9640 1 -- 172.21.15.94:0/2255553780 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b85287b600 con 0x55b852c2e2f0 2024-02-17T12:12:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.982+0000 7fdfb4ff9640 1 -- 172.21.15.94:0/2255553780 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdfc404e070 con 0x55b852c2e2f0 2024-02-17T12:12:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.982+0000 7fdfd5fd1740 10 client.19221 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.982+0000 7fdfd5fd1740 10 client.19221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.982+0000 7fdfd5fd1740 20 client.19221 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.982+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/2255553780 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b852c3e3d0 0x55b852c407f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.982+0000 7fdfd5fd1740 4 client.19221 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.982+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/2255553780 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8528cc1b0 con 0x55b852c3e3d0 2024-02-17T12:12:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.983+0000 7fdfd10c1640 1 --2- 172.21.15.94:0/2255553780 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b852c3e3d0 0x55b852c407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.983+0000 7fdfb4ff9640 10 client.19221 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.983+0000 7fdfd10c1640 1 --2- 172.21.15.94:0/2255553780 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b852c3e3d0 0x55b852c407f0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3baeb4baa6fcaf93 server_cookie=65cd56c800c51e7b in_seq=0 out_seq=0 2024-02-17T12:12:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfb4ff9640 1 -- 172.21.15.94:0/2255553780 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b8528cc1b0 con 0x55b852c3e3d0 2024-02-17T12:12:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfb4ff9640 10 client.19221 handle_command_reply: tid=0 2024-02-17T12:12:33.988 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfd5fd1740 1 client.19221 shutdown 2024-02-17T12:12:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/2255553780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 msgr2=0x55b852c1adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/2255553780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c1adf0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fdfc40b9930 tx=0x7fdfc40bce30 comp rx=0 tx=0).stop 2024-02-17T12:12:33.989 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/2255553780 shutdown_connections 2024-02-17T12:12:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/2255553780 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b852c3e3d0 0x55b852c407f0 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/2255553780 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c1adf0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/2255553780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c14ad0 0x55b852c1a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/2255553780 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c13d60 0x55b852c1a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/2255553780 >> 172.21.15.94:0/2255553780 conn(0x55b852bdb550 msgr2=0x55b852c107a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfb4ff9640 0 client.19221 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/2255553780 shutdown_connections 2024-02-17T12:12:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.984+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/2255553780 wait complete. 2024-02-17T12:12:33.991 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:33.985+0000 7fdfd5fd1740 20 client.19221 trim_cache size 0 max 16384 2024-02-17T12:12:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.001+0000 7fdfd5fd1740 1 Processor -- start 2024-02-17T12:12:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.001+0000 7fdfd5fd1740 1 -- start start 2024-02-17T12:12:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.001+0000 7fdfd5fd1740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c13d60 0x55b852c1caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.001+0000 7fdfd5fd1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c14ad0 0x55b852c18c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.001+0000 7fdfd5fd1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c2e2f0 0x55b852c19180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.001+0000 7fdfd5fd1740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b852c15d50 con 0x55b852c13d60 2024-02-17T12:12:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.001+0000 7fdfd5fd1740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8528cc1b0 con 0x55b852c14ad0 2024-02-17T12:12:34.005 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.001+0000 7fdfd5fd1740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b85287b600 con 0x55b852c2e2f0 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.001+0000 7fdfd10c1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c13d60 0x55b852c1caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.001+0000 7fdfd08c0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c14ad0 0x55b852c18c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.001+0000 7fdfd10c1640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c13d60 0x55b852c1caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:50330/0 (socket says 172.21.15.94:50330) 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.001+0000 7fdfd10c1640 1 -- 172.21.15.94:0/710914284 learned_addr learned my addr 172.21.15.94:0/710914284 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.001+0000 7fdfd18c2640 1 --2- 172.21.15.94:0/710914284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c2e2f0 0x55b852c19180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.002+0000 7fdfd08c0640 1 -- 172.21.15.94:0/710914284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c2e2f0 msgr2=0x55b852c19180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.002+0000 7fdfd08c0640 1 --2- 172.21.15.94:0/710914284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c2e2f0 0x55b852c19180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.002+0000 7fdfd08c0640 1 -- 172.21.15.94:0/710914284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c13d60 msgr2=0x55b852c1caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.002+0000 7fdfd08c0640 1 --2- 172.21.15.94:0/710914284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c13d60 0x55b852c1caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.002+0000 7fdfd08c0640 1 -- 172.21.15.94:0/710914284 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b852b167b0 con 0x55b852c14ad0 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.002+0000 7fdfd18c2640 1 --2- 172.21.15.94:0/710914284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c2e2f0 0x55b852c19180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.002+0000 7fdfd10c1640 1 --2- 172.21.15.94:0/710914284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c13d60 0x55b852c1caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.002+0000 7fdfd08c0640 1 --2- 172.21.15.94:0/710914284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c14ad0 0x55b852c18c40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55b852c19bd0 tx=0x7fdfc8010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f36ef19023d46ac server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:34.006 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.002+0000 7fdfb6ffd640 1 -- 172.21.15.94:0/710914284 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8016070 con 0x55b852c14ad0 2024-02-17T12:12:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.002+0000 7fdfb6ffd640 1 -- 172.21.15.94:0/710914284 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfc8017040 con 0x55b852c14ad0 2024-02-17T12:12:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.002+0000 7fdfb6ffd640 1 -- 172.21.15.94:0/710914284 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc801b6a0 con 0x55b852c14ad0 2024-02-17T12:12:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.003+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/710914284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c14ad0 msgr2=0x55b852c18c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.003+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/710914284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c14ad0 0x55b852c18c40 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55b852c19bd0 tx=0x7fdfc8010800 comp rx=0 tx=0).stop 2024-02-17T12:12:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.003+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/710914284 shutdown_connections 2024-02-17T12:12:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.003+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/710914284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c2e2f0 0x55b852c19180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.003+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/710914284 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c14ad0 0x55b852c18c40 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.003+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/710914284 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c13d60 0x55b852c1caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.003+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/710914284 >> 172.21.15.94:0/710914284 conn(0x55b852bdb550 msgr2=0x55b852c0e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.003+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/710914284 shutdown_connections 2024-02-17T12:12:34.007 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.003+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/710914284 wait complete. 2024-02-17T12:12:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.004+0000 7fdfd5fd1740 1 -- start start 2024-02-17T12:12:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.004+0000 7fdfd5fd1740 1 Processor -- start 2024-02-17T12:12:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.005+0000 7fdfd5fd1740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c13d60 0x55b852c35300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.005+0000 7fdfd5fd1740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c14ad0 0x55b852c337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.005+0000 7fdfd10c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c13d60 0x55b852c35300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.005+0000 7fdfd10c1640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c13d60 0x55b852c35300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:35698/0 (socket says 172.21.15.94:35698) 2024-02-17T12:12:34.009 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.005+0000 7fdfd10c1640 1 -- 172.21.15.94:0/222609687 learned_addr learned my addr 172.21.15.94:0/222609687 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.006+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c33ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.006+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/222609687 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b852c34220 con 0x55b852c2e2f0 2024-02-17T12:12:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.006+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/222609687 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b852c34390 con 0x55b852c13d60 2024-02-17T12:12:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.006+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/222609687 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b852c34550 con 0x55b852c14ad0 2024-02-17T12:12:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.006+0000 7fdfd5fd1740 10 client.0 _ll_register_callbacks cb 0x7fdfbc009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.006+0000 7fdfd08c0640 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c14ad0 0x55b852c337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.006+0000 7fdfd18c2640 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c33ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.006+0000 7fdfd08c0640 1 -- 172.21.15.94:0/222609687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c13d60 msgr2=0x55b852c35300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.006+0000 7fdfd08c0640 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c13d60 0x55b852c35300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.006+0000 7fdfd08c0640 1 -- 172.21.15.94:0/222609687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 msgr2=0x55b852c33ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.006+0000 7fdfd08c0640 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c33ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.006+0000 7fdfd08c0640 1 -- 172.21.15.94:0/222609687 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b852b117b0 con 0x55b852c14ad0 2024-02-17T12:12:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.006+0000 7fdfd18c2640 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c33ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:12:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.007+0000 7fdfd08c0640 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c14ad0 0x55b852c337a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8005c20 tx=0x7fdfc8005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=828fc2eace4ea8c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.007+0000 7fdfb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:12:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.007+0000 7fdfb4ff9640 1 -- 172.21.15.94:0/222609687 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8016070 con 0x55b852c14ad0 2024-02-17T12:12:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.007+0000 7fdfb4ff9640 1 -- 172.21.15.94:0/222609687 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fdfc80035f0 con 0x55b852c14ad0 2024-02-17T12:12:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.007+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/222609687 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfbc019040 con 0x55b852c14ad0 2024-02-17T12:12:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.007+0000 7fdfb4ff9640 1 -- 172.21.15.94:0/222609687 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfc8005640 con 0x55b852c14ad0 2024-02-17T12:12:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.007+0000 7fdfd5fd1740 10 client.28645 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.007+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/222609687 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fdfc8009ae0 con 0x55b852c14ad0 2024-02-17T12:12:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.008+0000 7fdfb4ff9640 1 -- 172.21.15.94:0/222609687 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdfbc019040 con 0x55b852c14ad0 2024-02-17T12:12:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.008+0000 7fdfb4ff9640 1 -- 172.21.15.94:0/222609687 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fdfc8025410 con 0x55b852c14ad0 2024-02-17T12:12:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.008+0000 7fdfd5fd1740 10 client.28645 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.008+0000 7fdfd5fd1740 10 client.28645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.008+0000 7fdfd5fd1740 20 client.28645 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.008+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b852c95db0 0x55b852c981d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.008+0000 7fdfd5fd1740 4 client.28645 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:12:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.008+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/222609687 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b8528b8550 con 0x55b852c95db0 2024-02-17T12:12:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.008+0000 7fdfd10c1640 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b852c95db0 0x55b852c981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.008+0000 7fdfb4ff9640 10 client.28645 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.008+0000 7fdfd10c1640 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b852c95db0 0x55b852c981d0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50e6965461960b82 server_cookie=890c35c4dd0db211 in_seq=0 out_seq=0 2024-02-17T12:12:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfb4ff9640 1 -- 172.21.15.94:0/222609687 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b8528b8550 con 0x55b852c95db0 2024-02-17T12:12:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfb4ff9640 10 client.28645 handle_command_reply: tid=0 2024-02-17T12:12:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfd5fd1740 1 client.28645 shutdown 2024-02-17T12:12:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/222609687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c14ad0 msgr2=0x55b852c337a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c14ad0 0x55b852c337a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdfc8005c20 tx=0x7fdfc8005040 comp rx=0 tx=0).stop 2024-02-17T12:12:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/222609687 shutdown_connections 2024-02-17T12:12:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b852c95db0 0x55b852c981d0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b852c2e2f0 0x55b852c33ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b852c14ad0 0x55b852c337a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfb4ff9640 0 client.28645 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfd5fd1740 1 --2- 172.21.15.94:0/222609687 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b852c13d60 0x55b852c35300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/222609687 >> 172.21.15.94:0/222609687 conn(0x55b852bdb550 msgr2=0x55b852c18bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/222609687 shutdown_connections 2024-02-17T12:12:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfd5fd1740 1 -- 172.21.15.94:0/222609687 wait complete. 2024-02-17T12:12:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.023+0000 7fdfd5fd1740 20 client.28645 trim_cache size 0 max 16384 2024-02-17T12:12:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:12:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:12:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:12:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:12:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.024+0000 7fdfd3325640 1 -- 172.21.15.94:0/1256613911 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdfb80a6840 msgr2=0x7fdfb80a8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.024+0000 7fdfd3325640 1 --2- 172.21.15.94:0/1256613911 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdfb80a6840 0x7fdfb80a8d00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0108d0 tx=0x7fdfbc00b000 comp rx=0 tx=0).stop 2024-02-17T12:12:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.024+0000 7fdfd3325640 1 -- 172.21.15.94:0/1256613911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfcc199b80 msgr2=0x7fdfcc19a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.024+0000 7fdfd3325640 1 --2- 172.21.15.94:0/1256613911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfcc199b80 0x7fdfcc19a030 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4036040 tx=0x7fdfc4033610 comp rx=0 tx=0).stop 2024-02-17T12:12:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.024+0000 7fdfd3325640 1 -- 172.21.15.94:0/1256613911 shutdown_connections 2024-02-17T12:12:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.024+0000 7fdfd3325640 1 --2- 172.21.15.94:0/1256613911 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fdfb80a6840 0x7fdfb80a8d00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.024+0000 7fdfd3325640 1 --2- 172.21.15.94:0/1256613911 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fdfcc199b80 0x7fdfcc19a030 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.024+0000 7fdfd3325640 1 --2- 172.21.15.94:0/1256613911 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fdfcc105390 0x7fdfcc195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.024+0000 7fdfd3325640 1 --2- 172.21.15.94:0/1256613911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdfcc1002c0 0x7fdfcc1952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.024+0000 7fdfd3325640 1 -- 172.21.15.94:0/1256613911 >> 172.21.15.94:0/1256613911 conn(0x7fdfcc0fbb40 msgr2=0x7fdfcc0fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.024+0000 7fdfd3325640 1 -- 172.21.15.94:0/1256613911 shutdown_connections 2024-02-17T12:12:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:34.024+0000 7fdfd3325640 1 -- 172.21.15.94:0/1256613911 wait complete. 2024-02-17T12:12:35.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 855.0 s, 1372.8 tps, lat 73.104 ms stddev 77.047 2024-02-17T12:12:35.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:35 smithi189 ceph-mon[36588]: pgmap v453: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.9 MiB/s wr, 766 op/s 2024-02-17T12:12:35.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:35 smithi125 ceph-mon[36819]: pgmap v453: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.9 MiB/s wr, 766 op/s 2024-02-17T12:12:35.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:35 smithi094 ceph-mon[32499]: pgmap v453: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.9 MiB/s wr, 766 op/s 2024-02-17T12:12:37.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:37 smithi189 ceph-mon[36588]: pgmap v454: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-02-17T12:12:37.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:37 smithi125 ceph-mon[36819]: pgmap v454: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-02-17T12:12:37.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:37 smithi094 ceph-mon[32499]: pgmap v454: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.7 MiB/s wr, 518 op/s 2024-02-17T12:12:39.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:39 smithi189 ceph-mon[36588]: pgmap v455: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 MiB/s wr, 652 op/s 2024-02-17T12:12:39.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:39 smithi125 ceph-mon[36819]: pgmap v455: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 MiB/s wr, 652 op/s 2024-02-17T12:12:39.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:39 smithi094 ceph-mon[32499]: pgmap v455: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 6.2 MiB/s wr, 652 op/s 2024-02-17T12:12:40.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 860.0 s, 1369.6 tps, lat 72.266 ms stddev 79.949 2024-02-17T12:12:41.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:41 smithi189 ceph-mon[36588]: pgmap v456: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.1 MiB/s wr, 626 op/s 2024-02-17T12:12:41.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:41 smithi125 ceph-mon[36819]: pgmap v456: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.1 MiB/s wr, 626 op/s 2024-02-17T12:12:41.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:41 smithi094 ceph-mon[32499]: pgmap v456: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 6.1 MiB/s wr, 626 op/s 2024-02-17T12:12:42.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:12:42.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:12:42.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.895+0000 7f701cdac640 1 -- 172.21.15.94:0/961717849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70181060a0 msgr2=0x7f7018106480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:42.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.895+0000 7f701cdac640 1 --2- 172.21.15.94:0/961717849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70181060a0 0x7f7018106480 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f7008009900 tx=0x7f700802bc30 comp rx=0 tx=0).stop 2024-02-17T12:12:42.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.895+0000 7f701cdac640 1 -- 172.21.15.94:0/961717849 shutdown_connections 2024-02-17T12:12:42.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.895+0000 7f701cdac640 1 --2- 172.21.15.94:0/961717849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7018100c70 0x7f701810ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.895+0000 7f701cdac640 1 --2- 172.21.15.94:0/961717849 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70181002d0 0x7f7018100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.895+0000 7f701cdac640 1 --2- 172.21.15.94:0/961717849 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f70181060a0 0x7f7018106480 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.895+0000 7f701cdac640 1 -- 172.21.15.94:0/961717849 >> 172.21.15.94:0/961717849 conn(0x7f7018076e10 msgr2=0x7f7018077220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:42.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.895+0000 7f701cdac640 1 -- 172.21.15.94:0/961717849 shutdown_connections 2024-02-17T12:12:42.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.895+0000 7f701cdac640 1 -- 172.21.15.94:0/961717849 wait complete. 2024-02-17T12:12:42.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.896+0000 7f701cdac640 1 Processor -- start 2024-02-17T12:12:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.896+0000 7f701cdac640 1 -- start start 2024-02-17T12:12:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.897+0000 7f701cdac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70181002d0 0x7f70181976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.897+0000 7f701cdac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7018100c70 0x7f7018197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.900 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.897+0000 7f701cdac640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70181060a0 0x7f701819bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.897+0000 7f701cdac640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701819c5b0 con 0x7f7018100c70 2024-02-17T12:12:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.897+0000 7f700dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7018100c70 0x7f7018197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.897+0000 7f701cdac640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701819c6f0 con 0x7f70181002d0 2024-02-17T12:12:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.897+0000 7f701cdac640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701819c8b0 con 0x7f70181060a0 2024-02-17T12:12:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.897+0000 7f700dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7018100c70 0x7f7018197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34718/0 (socket says 172.21.15.94:34718) 2024-02-17T12:12:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.897+0000 7f700dd9b640 1 -- 172.21.15.94:0/2907502637 learned_addr learned my addr 172.21.15.94:0/2907502637 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.897+0000 7f7016d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70181060a0 0x7f701819bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.898+0000 7f700dd9b640 1 -- 172.21.15.94:0/2907502637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70181060a0 msgr2=0x7f701819bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.898+0000 7f700dd9b640 1 --2- 172.21.15.94:0/2907502637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70181060a0 0x7f701819bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.898+0000 7f700dd9b640 1 -- 172.21.15.94:0/2907502637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70181002d0 msgr2=0x7f70181976f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-17T12:12:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.898+0000 7f700dd9b640 1 --2- 172.21.15.94:0/2907502637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70181002d0 0x7f70181976f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.898+0000 7f700dd9b640 1 -- 172.21.15.94:0/2907502637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7008009560 con 0x7f7018100c70 2024-02-17T12:12:42.901 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.898+0000 7f700dd9b640 1 --2- 172.21.15.94:0/2907502637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7018100c70 0x7f7018197c30 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7000009f20 tx=0x7f700000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.899+0000 7f700ffff640 1 -- 172.21.15.94:0/2907502637 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700000cd80 con 0x7f7018100c70 2024-02-17T12:12:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.899+0000 7f700ffff640 1 -- 172.21.15.94:0/2907502637 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7000032920 con 0x7f7018100c70 2024-02-17T12:12:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.899+0000 7f701cdac640 1 -- 172.21.15.94:0/2907502637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f701819cb30 con 0x7f7018100c70 2024-02-17T12:12:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.899+0000 7f700ffff640 1 -- 172.21.15.94:0/2907502637 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700003a920 con 0x7f7018100c70 2024-02-17T12:12:42.902 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.899+0000 7f701cdac640 1 -- 172.21.15.94:0/2907502637 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701806ad80 con 0x7f7018100c70 2024-02-17T12:12:42.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.901+0000 7f700ffff640 1 -- 172.21.15.94:0/2907502637 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7000032a90 con 0x7f7018100c70 2024-02-17T12:12:42.903 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f701f980740 1 Processor -- start 2024-02-17T12:12:42.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f701f980740 1 -- start start 2024-02-17T12:12:42.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f700ffff640 1 --2- 172.21.15.94:0/2907502637 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f70040a2230 0x7f70040a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f701f980740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 0x55c92595e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f7016d9d640 1 --2- 172.21.15.94:0/2907502637 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f70040a2230 0x7f70040a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f701f980740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c925967270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f700ffff640 1 -- 172.21.15.94:0/2907502637 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f7000036070 con 0x7f7018100c70 2024-02-17T12:12:42.906 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f701f980740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9259677b0 0x55c925969ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f701f980740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9258cd280 con 0x55c9259677b0 2024-02-17T12:12:42.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f701f980740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9258ca7b0 con 0x55c92595ebd0 2024-02-17T12:12:42.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f701f980740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9258c5f90 con 0x55c92595e2b0 2024-02-17T12:12:42.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f7016d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9259677b0 0x55c925969ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f701659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 0x55c92595e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f7016d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9259677b0 0x55c925969ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34732/0 (socket says 172.21.15.94:34732) 2024-02-17T12:12:42.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f7016d9d640 1 -- 172.21.15.94:0/408465515 learned_addr learned my addr 172.21.15.94:0/408465515 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:42.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.902+0000 7f700dd9b640 1 --2- 172.21.15.94:0/408465515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c925967270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.903+0000 7f701659c640 1 -- 172.21.15.94:0/408465515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 msgr2=0x55c925967270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:42.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.903+0000 7f701659c640 1 --2- 172.21.15.94:0/408465515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c925967270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.903+0000 7f701659c640 1 -- 172.21.15.94:0/408465515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9259677b0 msgr2=0x55c925969ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:42.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.903+0000 7f701659c640 1 --2- 172.21.15.94:0/408465515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9259677b0 0x55c925969ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.907 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.903+0000 7f7016d9d640 1 --2- 172.21.15.94:0/2907502637 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f70040a2230 0x7f70040a46f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7010009fd0 tx=0x7f7010013b80 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.903+0000 7f701659c640 1 -- 172.21.15.94:0/408465515 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c925884570 con 0x55c92595e2b0 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.903+0000 7f701659c640 1 --2- 172.21.15.94:0/408465515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 0x55c92595e690 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7008000c00 tx=0x7f7008005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a37f8fc5897dd18 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.903+0000 7f700cd99640 1 -- 172.21.15.94:0/408465515 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080046c0 con 0x55c92595e2b0 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.903+0000 7f700cd99640 1 -- 172.21.15.94:0/408465515 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7008004820 con 0x55c92595e2b0 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.903+0000 7f700cd99640 1 -- 172.21.15.94:0/408465515 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008004df0 con 0x55c92595e2b0 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.904+0000 7f701f980740 1 -- 172.21.15.94:0/408465515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 msgr2=0x55c92595e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.904+0000 7f701f980740 1 --2- 172.21.15.94:0/408465515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 0x55c92595e690 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7008000c00 tx=0x7f7008005e50 comp rx=0 tx=0).stop 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.904+0000 7f701f980740 1 -- 172.21.15.94:0/408465515 shutdown_connections 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.904+0000 7f701f980740 1 --2- 172.21.15.94:0/408465515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9259677b0 0x55c925969ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.904+0000 7f701f980740 1 --2- 172.21.15.94:0/408465515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c925967270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.904+0000 7f701f980740 1 --2- 172.21.15.94:0/408465515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 0x55c92595e690 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.904+0000 7f701f980740 1 -- 172.21.15.94:0/408465515 >> 172.21.15.94:0/408465515 conn(0x55c925958990 msgr2=0x55c925958d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.904+0000 7f701f980740 1 -- 172.21.15.94:0/408465515 shutdown_connections 2024-02-17T12:12:42.908 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.904+0000 7f701f980740 1 -- 172.21.15.94:0/408465515 wait complete. 2024-02-17T12:12:42.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.904+0000 7f701f980740 1 -- start start 2024-02-17T12:12:42.909 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.904+0000 7f701f980740 1 Processor -- start 2024-02-17T12:12:42.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f701f980740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 0x55c925967200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f701f980740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595ebd0 0x55c925964590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f701f980740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9259677b0 0x55c925964cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f701f980740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c925599430 con 0x55c92595ebd0 2024-02-17T12:12:42.912 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f701f980740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c925555e00 con 0x55c9259677b0 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f701f980740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92584afd0 con 0x55c92595e2b0 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f701f980740 10 client.0 _ll_register_callbacks cb 0x55c9258a0fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f700dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595ebd0 0x55c925964590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f700dd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595ebd0 0x55c925964590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34746/0 (socket says 172.21.15.94:34746) 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f700dd9b640 1 -- 172.21.15.94:0/2672040878 learned_addr learned my addr 172.21.15.94:0/2672040878 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f7016d9d640 1 --2- 172.21.15.94:0/2672040878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9259677b0 0x55c925964cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f701659c640 1 --2- 172.21.15.94:0/2672040878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 0x55c925967200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f700dd9b640 1 -- 172.21.15.94:0/2672040878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 msgr2=0x55c925967200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f700dd9b640 1 --2- 172.21.15.94:0/2672040878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 0x55c925967200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f700dd9b640 1 -- 172.21.15.94:0/2672040878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9259677b0 msgr2=0x55c925964cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f700dd9b640 1 --2- 172.21.15.94:0/2672040878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9259677b0 0x55c925964cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f700dd9b640 1 -- 172.21.15.94:0/2672040878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f701810d740 con 0x55c92595ebd0 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f700dd9b640 1 --2- 172.21.15.94:0/2672040878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595ebd0 0x55c925964590 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7000009fd0 tx=0x7f70000bc8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135a096f2adadc96 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:42.913 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f6ffdffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f6ffdffb640 1 -- 172.21.15.94:0/2672040878 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70000bc550 con 0x55c92595ebd0 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f6ffdffb640 1 -- 172.21.15.94:0/2672040878 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f70000c1050 con 0x55c92595ebd0 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f701f980740 1 -- 172.21.15.94:0/2672040878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9255c57e0 con 0x55c92595ebd0 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f6ffdffb640 1 -- 172.21.15.94:0/2672040878 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70000bc6b0 con 0x55c92595ebd0 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f701f980740 10 client.19236 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.905+0000 7f701f980740 1 -- 172.21.15.94:0/2672040878 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c9258c4610 con 0x55c92595ebd0 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.906+0000 7f6ffdffb640 1 -- 172.21.15.94:0/2672040878 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c9255c57e0 con 0x55c92595ebd0 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.906+0000 7f6ffdffb640 1 -- 172.21.15.94:0/2672040878 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f700009bd20 con 0x55c92595ebd0 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.906+0000 7f701f980740 10 client.19236 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.906+0000 7f701f980740 10 client.19236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.906+0000 7f701f980740 20 client.19236 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.906+0000 7f701f980740 1 --2- 172.21.15.94:0/2672040878 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c925977990 0x55c925979d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.906+0000 7f701f980740 4 client.19236 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:42.914 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.906+0000 7f701f980740 1 -- 172.21.15.94:0/2672040878 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c925602720 con 0x55c925977990 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.907+0000 7f701659c640 1 --2- 172.21.15.94:0/2672040878 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c925977990 0x55c925979d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.907+0000 7f701659c640 1 --2- 172.21.15.94:0/2672040878 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c925977990 0x55c925979d80 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d970a1ed80dc92 server_cookie=695d9dcddcd1bbdf in_seq=0 out_seq=0 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.908+0000 7f6ffdffb640 10 client.19236 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.908+0000 7f6ffdffb640 1 -- 172.21.15.94:0/2672040878 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c925602720 con 0x55c925977990 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.908+0000 7f6ffdffb640 10 client.19236 handle_command_reply: tid=0 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.908+0000 7f701f980740 1 client.19236 shutdown 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.909+0000 7f701f980740 1 -- 172.21.15.94:0/2672040878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595ebd0 msgr2=0x55c925964590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.909+0000 7f701f980740 1 --2- 172.21.15.94:0/2672040878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595ebd0 0x55c925964590 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7000009fd0 tx=0x7f70000bc8e0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.909+0000 7f701f980740 1 -- 172.21.15.94:0/2672040878 shutdown_connections 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.909+0000 7f701f980740 1 --2- 172.21.15.94:0/2672040878 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c925977990 0x55c925979d80 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.909+0000 7f701f980740 1 --2- 172.21.15.94:0/2672040878 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c9259677b0 0x55c925964cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.909+0000 7f701f980740 1 --2- 172.21.15.94:0/2672040878 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595ebd0 0x55c925964590 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.909+0000 7f701f980740 1 --2- 172.21.15.94:0/2672040878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 0x55c925967200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.915 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.909+0000 7f701f980740 1 -- 172.21.15.94:0/2672040878 >> 172.21.15.94:0/2672040878 conn(0x55c925958990 msgr2=0x55c92595a730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:42.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.909+0000 7f6ffdffb640 0 client.19236 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:42.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.909+0000 7f701f980740 1 -- 172.21.15.94:0/2672040878 shutdown_connections 2024-02-17T12:12:42.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.909+0000 7f701f980740 1 -- 172.21.15.94:0/2672040878 wait complete. 2024-02-17T12:12:42.916 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.909+0000 7f701f980740 20 client.19236 trim_cache size 0 max 16384 2024-02-17T12:12:42.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.928+0000 7f701f980740 1 Processor -- start 2024-02-17T12:12:42.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.928+0000 7f701f980740 1 -- start start 2024-02-17T12:12:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.928+0000 7f701f980740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 0x55c925966cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.928+0000 7f701f980740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c925967200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.928+0000 7f701f980740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9259677b0 0x55c92595f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.928+0000 7f701f980740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92595d5a0 con 0x55c9259677b0 2024-02-17T12:12:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.928+0000 7f701f980740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9259599e0 con 0x55c92595ebd0 2024-02-17T12:12:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.928+0000 7f701f980740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9259540d0 con 0x55c92595e2b0 2024-02-17T12:12:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.928+0000 7f7016d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9259677b0 0x55c92595f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.928+0000 7f7016d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9259677b0 0x55c92595f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34758/0 (socket says 172.21.15.94:34758) 2024-02-17T12:12:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.928+0000 7f7016d9d640 1 -- 172.21.15.94:0/4211247563 learned_addr learned my addr 172.21.15.94:0/4211247563 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.928+0000 7f701659c640 1 --2- 172.21.15.94:0/4211247563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 0x55c925966cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.928+0000 7f700dd9b640 1 --2- 172.21.15.94:0/4211247563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c925967200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.929+0000 7f700dd9b640 1 -- 172.21.15.94:0/4211247563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 msgr2=0x55c925966cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:42.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.929+0000 7f700dd9b640 1 --2- 172.21.15.94:0/4211247563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 0x55c925966cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.929+0000 7f700dd9b640 1 -- 172.21.15.94:0/4211247563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9259677b0 msgr2=0x55c92595f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:42.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.929+0000 7f700dd9b640 1 --2- 172.21.15.94:0/4211247563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9259677b0 0x55c92595f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.929+0000 7f700dd9b640 1 -- 172.21.15.94:0/4211247563 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c925860ac0 con 0x55c92595ebd0 2024-02-17T12:12:42.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.929+0000 7f7016d9d640 1 --2- 172.21.15.94:0/4211247563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9259677b0 0x55c92595f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:42.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.929+0000 7f700dd9b640 1 --2- 172.21.15.94:0/4211247563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c925967200 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7000009620 tx=0x7f700000efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=508ae70cdc63c9f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:42.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.929+0000 7f6ffffff640 1 -- 172.21.15.94:0/4211247563 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70000c1050 con 0x55c92595ebd0 2024-02-17T12:12:42.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.929+0000 7f6ffffff640 1 -- 172.21.15.94:0/4211247563 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f70000c56a0 con 0x55c92595ebd0 2024-02-17T12:12:42.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.929+0000 7f6ffffff640 1 -- 172.21.15.94:0/4211247563 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70000bc350 con 0x55c92595ebd0 2024-02-17T12:12:42.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.930+0000 7f701f980740 1 -- 172.21.15.94:0/4211247563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 msgr2=0x55c925967200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.930+0000 7f701f980740 1 --2- 172.21.15.94:0/4211247563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c925967200 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7000009620 tx=0x7f700000efb0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.930+0000 7f701f980740 1 -- 172.21.15.94:0/4211247563 shutdown_connections 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.930+0000 7f701f980740 1 --2- 172.21.15.94:0/4211247563 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c9259677b0 0x55c92595f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.930+0000 7f701f980740 1 --2- 172.21.15.94:0/4211247563 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c925967200 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.930+0000 7f701f980740 1 --2- 172.21.15.94:0/4211247563 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c92595e2b0 0x55c925966cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.930+0000 7f701f980740 1 -- 172.21.15.94:0/4211247563 >> 172.21.15.94:0/4211247563 conn(0x55c925958990 msgr2=0x55c92597e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.930+0000 7f701f980740 1 -- 172.21.15.94:0/4211247563 shutdown_connections 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.930+0000 7f701f980740 1 -- 172.21.15.94:0/4211247563 wait complete. 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.930+0000 7f701f980740 1 -- start start 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.930+0000 7f701f980740 1 Processor -- start 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f701f980740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595e2b0 0x55c92596a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f701f980740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c92596ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f701f980740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9259677b0 0x55c9259cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f701f980740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92596b3e0 con 0x55c92595e2b0 2024-02-17T12:12:42.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f701f980740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9259cd390 con 0x55c92595ebd0 2024-02-17T12:12:42.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f701f980740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9259cd4d0 con 0x55c9259677b0 2024-02-17T12:12:42.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f701f980740 10 client.0 _ll_register_callbacks cb 0x7f7000005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:42.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f701659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595e2b0 0x55c92596a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f700dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c92596ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f700dd9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c92596ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:43182/0 (socket says 172.21.15.94:43182) 2024-02-17T12:12:42.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f700dd9b640 1 -- 172.21.15.94:0/895173251 learned_addr learned my addr 172.21.15.94:0/895173251 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:42.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f701659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595e2b0 0x55c92596a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34770/0 (socket says 172.21.15.94:34770) 2024-02-17T12:12:42.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f7016d9d640 1 --2- 172.21.15.94:0/895173251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9259677b0 0x55c9259cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f700dd9b640 1 -- 172.21.15.94:0/895173251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9259677b0 msgr2=0x55c9259cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:42.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f700dd9b640 1 --2- 172.21.15.94:0/895173251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9259677b0 0x55c9259cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f700dd9b640 1 -- 172.21.15.94:0/895173251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595e2b0 msgr2=0x55c92596a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:42.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f700dd9b640 1 --2- 172.21.15.94:0/895173251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595e2b0 0x55c92596a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f700dd9b640 1 -- 172.21.15.94:0/895173251 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7010012140 con 0x55c92595ebd0 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f701659c640 1 --2- 172.21.15.94:0/895173251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595e2b0 0x55c92596a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f700dd9b640 1 --2- 172.21.15.94:0/895173251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c92596ab10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f70000c5170 tx=0x7f70000bc8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5543fc6defd49fdb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f6ffdffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f6ffdffb640 1 -- 172.21.15.94:0/895173251 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70000bc6c0 con 0x55c92595ebd0 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f701f980740 1 -- 172.21.15.94:0/895173251 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7008004820 con 0x55c92595ebd0 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.931+0000 7f6ffdffb640 1 -- 172.21.15.94:0/895173251 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f70000c1050 con 0x55c92595ebd0 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f6ffdffb640 1 -- 172.21.15.94:0/895173251 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70000f0430 con 0x55c92595ebd0 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f701f980740 10 client.28700 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f701f980740 1 -- 172.21.15.94:0/895173251 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c9258c4610 con 0x55c92595ebd0 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f6ffdffb640 1 -- 172.21.15.94:0/895173251 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7008004820 con 0x55c92595ebd0 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f6ffdffb640 1 -- 172.21.15.94:0/895173251 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f700009bd20 con 0x55c92595ebd0 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f701f980740 10 client.28700 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:42.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f701f980740 10 client.28700 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f701f980740 20 client.28700 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f701f980740 1 --2- 172.21.15.94:0/895173251 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c9259ce790 0x55c9259d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f701f980740 4 client.28700 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f701f980740 1 -- 172.21.15.94:0/895173251 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c925651890 con 0x55c9259ce790 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f701659c640 1 --2- 172.21.15.94:0/895173251 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c9259ce790 0x55c9259d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f6ffdffb640 10 client.28700 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.932+0000 7f701659c640 1 --2- 172.21.15.94:0/895173251 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c9259ce790 0x55c9259d0b80 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ea8ecac6df16286 server_cookie=c60bc43344d02b60 in_seq=0 out_seq=0 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f6ffdffb640 1 -- 172.21.15.94:0/895173251 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c925651890 con 0x55c9259ce790 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f6ffdffb640 10 client.28700 handle_command_reply: tid=0 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701f980740 1 client.28700 shutdown 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701f980740 1 -- 172.21.15.94:0/895173251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 msgr2=0x55c92596ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701f980740 1 --2- 172.21.15.94:0/895173251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c92596ab10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f70000c5170 tx=0x7f70000bc8e0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701f980740 1 -- 172.21.15.94:0/895173251 shutdown_connections 2024-02-17T12:12:42.941 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701f980740 1 --2- 172.21.15.94:0/895173251 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c9259ce790 0x55c9259d0b80 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701f980740 1 --2- 172.21.15.94:0/895173251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c9259677b0 0x55c9259cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701f980740 1 --2- 172.21.15.94:0/895173251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c92595ebd0 0x55c92596ab10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f6ffdffb640 0 client.28700 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701f980740 1 --2- 172.21.15.94:0/895173251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c92595e2b0 0x55c92596a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701f980740 1 -- 172.21.15.94:0/895173251 >> 172.21.15.94:0/895173251 conn(0x55c925958990 msgr2=0x55c92595f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701f980740 1 -- 172.21.15.94:0/895173251 shutdown_connections 2024-02-17T12:12:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701f980740 1 -- 172.21.15.94:0/895173251 wait complete. 2024-02-17T12:12:42.942 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701f980740 20 client.28700 trim_cache size 0 max 16384 2024-02-17T12:12:42.942 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:12:42.942 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:12:42.942 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:12:42.942 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:12:42.943 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701cdac640 1 -- 172.21.15.94:0/2907502637 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f70040a2230 msgr2=0x7f70040a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701cdac640 1 --2- 172.21.15.94:0/2907502637 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f70040a2230 0x7f70040a46f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7010009fd0 tx=0x7f7010013b80 comp rx=0 tx=0).stop 2024-02-17T12:12:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701cdac640 1 -- 172.21.15.94:0/2907502637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7018100c70 msgr2=0x7f7018197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701cdac640 1 --2- 172.21.15.94:0/2907502637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7018100c70 0x7f7018197c30 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f7000009f20 tx=0x7f700000ee80 comp rx=0 tx=0).stop 2024-02-17T12:12:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701cdac640 1 -- 172.21.15.94:0/2907502637 shutdown_connections 2024-02-17T12:12:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701cdac640 1 --2- 172.21.15.94:0/2907502637 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f70040a2230 0x7f70040a46f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701cdac640 1 --2- 172.21.15.94:0/2907502637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70181060a0 0x7f701819bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701cdac640 1 --2- 172.21.15.94:0/2907502637 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7018100c70 0x7f7018197c30 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701cdac640 1 --2- 172.21.15.94:0/2907502637 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f70181002d0 0x7f70181976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701cdac640 1 -- 172.21.15.94:0/2907502637 >> 172.21.15.94:0/2907502637 conn(0x7f7018076e10 msgr2=0x7f7018104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701cdac640 1 -- 172.21.15.94:0/2907502637 shutdown_connections 2024-02-17T12:12:42.944 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:42.933+0000 7f701cdac640 1 -- 172.21.15.94:0/2907502637 wait complete. 2024-02-17T12:12:42.950 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:12:42.950 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:12:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.124+0000 7f6cb6a02640 1 -- 172.21.15.94:0/116506813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cb0106450 msgr2=0x7f6cb010e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.124+0000 7f6cb6a02640 1 --2- 172.21.15.94:0/116506813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cb0106450 0x7f6cb010e020 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009ca0 tx=0x7f6ca802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.124+0000 7f6cb6a02640 1 -- 172.21.15.94:0/116506813 shutdown_connections 2024-02-17T12:12:43.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.124+0000 7f6cb6a02640 1 --2- 172.21.15.94:0/116506813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cb0106450 0x7f6cb010e020 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.124+0000 7f6cb6a02640 1 --2- 172.21.15.94:0/116506813 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0101d30 0x7f6cb0105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.124+0000 7f6cb6a02640 1 --2- 172.21.15.94:0/116506813 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cb0101380 0x7f6cb0101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.124+0000 7f6cb6a02640 1 -- 172.21.15.94:0/116506813 >> 172.21.15.94:0/116506813 conn(0x7f6cb0078580 msgr2=0x7f6cb0078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.124+0000 7f6cb6a02640 1 -- 172.21.15.94:0/116506813 shutdown_connections 2024-02-17T12:12:43.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.124+0000 7f6cb6a02640 1 -- 172.21.15.94:0/116506813 wait complete. 2024-02-17T12:12:43.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.124+0000 7f6cb6a02640 1 Processor -- start 2024-02-17T12:12:43.127 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.124+0000 7f6cb6a02640 1 -- start start 2024-02-17T12:12:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6cb6a02640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0101380 0x7f6cb019d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6cb6a02640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cb0106450 0x7f6cb019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6cb6a02640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cb01975a0 0x7f6cb0197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6cb6a02640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb0197f90 con 0x7f6cb0101380 2024-02-17T12:12:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6cb6a02640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb0198100 con 0x7f6cb01975a0 2024-02-17T12:12:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6cb6a02640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb01982c0 con 0x7f6cb0106450 2024-02-17T12:12:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6caffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0101380 0x7f6cb019d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6caffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0101380 0x7f6cb019d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34798/0 (socket says 172.21.15.94:34798) 2024-02-17T12:12:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6caffff640 1 -- 172.21.15.94:0/2618160885 learned_addr learned my addr 172.21.15.94:0/2618160885 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6caf7fe640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cb0106450 0x7f6cb019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6cb4f9f640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cb01975a0 0x7f6cb0197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6caffff640 1 -- 172.21.15.94:0/2618160885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cb0106450 msgr2=0x7f6cb019d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6caffff640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cb0106450 0x7f6cb019d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6caffff640 1 -- 172.21.15.94:0/2618160885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cb01975a0 msgr2=0x7f6cb0197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6caffff640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cb01975a0 0x7f6cb0197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6caffff640 1 -- 172.21.15.94:0/2618160885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca4009490 con 0x7f6cb0101380 2024-02-17T12:12:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6caf7fe640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cb0106450 0x7f6cb019d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:43.130 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.125+0000 7f6caffff640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0101380 0x7f6cb019d3f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000d770 tx=0x7f6ca000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.129+0000 7f6cad7fa640 1 -- 172.21.15.94:0/2618160885 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0009c30 con 0x7f6cb0101380 2024-02-17T12:12:43.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.129+0000 7f6cad7fa640 1 -- 172.21.15.94:0/2618160885 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ca000de80 con 0x7f6cb0101380 2024-02-17T12:12:43.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.129+0000 7f6cb6a02640 1 -- 172.21.15.94:0/2618160885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca8009900 con 0x7f6cb0101380 2024-02-17T12:12:43.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.129+0000 7f6cad7fa640 1 -- 172.21.15.94:0/2618160885 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0004e80 con 0x7f6cb0101380 2024-02-17T12:12:43.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.129+0000 7f6cb6a02640 1 -- 172.21.15.94:0/2618160885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb01a4630 con 0x7f6cb0101380 2024-02-17T12:12:43.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.130+0000 7f6cb6a02640 1 -- 172.21.15.94:0/2618160885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb00676e0 con 0x7f6cb0101380 2024-02-17T12:12:43.133 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.131+0000 7f6cad7fa640 1 -- 172.21.15.94:0/2618160885 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6ca0005020 con 0x7f6cb0101380 2024-02-17T12:12:43.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.132+0000 7f6cad7fa640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6c980a22b0 0x7f6c980a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.132+0000 7f6cad7fa640 1 -- 172.21.15.94:0/2618160885 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6ca00cda50 con 0x7f6cb0101380 2024-02-17T12:12:43.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.132+0000 7f6caf7fe640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6c980a22b0 0x7f6c980a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.134 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.132+0000 7f6caf7fe640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6c980a22b0 0x7f6c980a4770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00fe920 tx=0x7f6ca4005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.138 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.136+0000 7f6cad7fa640 1 -- 172.21.15.94:0/2618160885 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6ca0096750 con 0x7f6cb0101380 2024-02-17T12:12:43.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:43 smithi189 ceph-mon[36588]: pgmap v457: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 528 op/s 2024-02-17T12:12:43.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:43 smithi125 ceph-mon[36819]: pgmap v457: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 528 op/s 2024-02-17T12:12:43.444 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.442+0000 7f6cb6a02640 1 -- 172.21.15.94:0/2618160885 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cb0198ad0 con 0x7f6cb0101380 2024-02-17T12:12:43.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.444+0000 7f6cad7fa640 1 -- 172.21.15.94:0/2618160885 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7f6ca0017090 con 0x7f6cb0101380 2024-02-17T12:12:43.447 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:12:43.447 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:12:43.447 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:12:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.450+0000 7f6cb6a02640 1 -- 172.21.15.94:0/2618160885 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6c980a22b0 msgr2=0x7f6c980a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.450+0000 7f6cb6a02640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6c980a22b0 0x7f6c980a4770 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6cb00fe920 tx=0x7f6ca4005f90 comp rx=0 tx=0).stop 2024-02-17T12:12:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.450+0000 7f6cb6a02640 1 -- 172.21.15.94:0/2618160885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0101380 msgr2=0x7f6cb019d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.450+0000 7f6cb6a02640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0101380 0x7f6cb019d3f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000d770 tx=0x7f6ca000dc30 comp rx=0 tx=0).stop 2024-02-17T12:12:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.450+0000 7f6cb6a02640 1 -- 172.21.15.94:0/2618160885 shutdown_connections 2024-02-17T12:12:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.450+0000 7f6cb6a02640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6c980a22b0 0x7f6c980a4770 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.450+0000 7f6cb6a02640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6cb01975a0 0x7f6cb0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.450+0000 7f6cb6a02640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6cb0106450 0x7f6cb019d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.450+0000 7f6cb6a02640 1 --2- 172.21.15.94:0/2618160885 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6cb0101380 0x7f6cb019d3f0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.450+0000 7f6cb6a02640 1 -- 172.21.15.94:0/2618160885 >> 172.21.15.94:0/2618160885 conn(0x7f6cb0078580 msgr2=0x7f6cb00ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.450+0000 7f6cb6a02640 1 -- 172.21.15.94:0/2618160885 shutdown_connections 2024-02-17T12:12:43.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.450+0000 7f6cb6a02640 1 -- 172.21.15.94:0/2618160885 wait complete. 2024-02-17T12:12:43.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:43 smithi094 ceph-mon[32499]: pgmap v457: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 5.2 MiB/s wr, 528 op/s 2024-02-17T12:12:43.473 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:12:43.473 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:12:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.593+0000 7f87591ac640 1 -- 172.21.15.94:0/3493881222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87541060d0 msgr2=0x7f8754068380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.593+0000 7f87591ac640 1 --2- 172.21.15.94:0/3493881222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87541060d0 0x7f8754068380 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f8744009890 tx=0x7f874402bb70 comp rx=0 tx=0).stop 2024-02-17T12:12:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.593+0000 7f87591ac640 1 -- 172.21.15.94:0/3493881222 shutdown_connections 2024-02-17T12:12:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.593+0000 7f87591ac640 1 --2- 172.21.15.94:0/3493881222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8754103c40 0x7f87541040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.593+0000 7f87591ac640 1 --2- 172.21.15.94:0/3493881222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f87540688c0 0x7f8754068d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.593+0000 7f87591ac640 1 --2- 172.21.15.94:0/3493881222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87541060d0 0x7f8754068380 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.595 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.593+0000 7f87591ac640 1 -- 172.21.15.94:0/3493881222 >> 172.21.15.94:0/3493881222 conn(0x7f8754077180 msgr2=0x7f8754077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.593+0000 7f87591ac640 1 -- 172.21.15.94:0/3493881222 shutdown_connections 2024-02-17T12:12:43.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.593+0000 7f87591ac640 1 -- 172.21.15.94:0/3493881222 wait complete. 2024-02-17T12:12:43.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.593+0000 7f87591ac640 1 Processor -- start 2024-02-17T12:12:43.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f87591ac640 1 -- start start 2024-02-17T12:12:43.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f87591ac640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87540688c0 0x7f875419dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f87591ac640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8754103c40 0x7f875419e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f87591ac640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87541060d0 0x7f87541a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f87591ac640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87541a2c40 con 0x7f87541060d0 2024-02-17T12:12:43.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f87591ac640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87541a2db0 con 0x7f8754103c40 2024-02-17T12:12:43.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f87591ac640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87541a2f70 con 0x7f87540688c0 2024-02-17T12:12:43.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f8752d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87540688c0 0x7f875419dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f8752d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87540688c0 0x7f875419dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39856/0 (socket says 172.21.15.94:39856) 2024-02-17T12:12:43.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f8752d9d640 1 -- 172.21.15.94:0/319605431 learned_addr learned my addr 172.21.15.94:0/319605431 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:43.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f875359e640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87541060d0 0x7f87541a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f875259c640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8754103c40 0x7f875419e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f8752d9d640 1 -- 172.21.15.94:0/319605431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8754103c40 msgr2=0x7f875419e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.598 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f8752d9d640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8754103c40 0x7f875419e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f8752d9d640 1 -- 172.21.15.94:0/319605431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87541060d0 msgr2=0x7f87541a2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f8752d9d640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87541060d0 0x7f87541a2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f8752d9d640 1 -- 172.21.15.94:0/319605431 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8744009490 con 0x7f87540688c0 2024-02-17T12:12:43.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.594+0000 7f875259c640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8754103c40 0x7f875419e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:12:43.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.595+0000 7f8752d9d640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87540688c0 0x7f875419dd80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8744000c00 tx=0x7f8744004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.595+0000 7f873bfff640 1 -- 172.21.15.94:0/319605431 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744034950 con 0x7f87540688c0 2024-02-17T12:12:43.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.595+0000 7f87591ac640 1 -- 172.21.15.94:0/319605431 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87541a31f0 con 0x7f87540688c0 2024-02-17T12:12:43.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.595+0000 7f873bfff640 1 -- 172.21.15.94:0/319605431 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f874402c9f0 con 0x7f87540688c0 2024-02-17T12:12:43.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.595+0000 7f873bfff640 1 -- 172.21.15.94:0/319605431 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8744032600 con 0x7f87540688c0 2024-02-17T12:12:43.599 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.595+0000 7f87591ac640 1 -- 172.21.15.94:0/319605431 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87541aaad0 con 0x7f87540688c0 2024-02-17T12:12:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.597+0000 7f875bd7e740 1 Processor -- start 2024-02-17T12:12:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.597+0000 7f875bd7e740 1 -- start start 2024-02-17T12:12:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.597+0000 7f875bd7e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469250 0x55dd56469630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.597+0000 7f873bfff640 1 -- 172.21.15.94:0/319605431 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f874403b040 con 0x7f87540688c0 2024-02-17T12:12:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.597+0000 7f875bd7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd56472210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.597+0000 7f875bd7e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 0x55dd56474b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f875bd7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd56424960 con 0x55dd56469b70 2024-02-17T12:12:43.602 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f875bd7e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd563d8310 con 0x55dd56472750 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f875bd7e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd563d5840 con 0x55dd56469250 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f8752d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd56472210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f8752d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd56472210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34832/0 (socket says 172.21.15.94:34832) 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f8752d9d640 1 -- 172.21.15.94:0/1638250222 learned_addr learned my addr 172.21.15.94:0/1638250222 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f875359e640 1 --2- 172.21.15.94:0/1638250222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469250 0x55dd56469630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f875259c640 1 --2- 172.21.15.94:0/1638250222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 0x55dd56474b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f8752d9d640 1 -- 172.21.15.94:0/1638250222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469250 msgr2=0x55dd56469630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f8752d9d640 1 --2- 172.21.15.94:0/1638250222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469250 0x55dd56469630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f8752d9d640 1 -- 172.21.15.94:0/1638250222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 msgr2=0x55dd56474b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f8752d9d640 1 --2- 172.21.15.94:0/1638250222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 0x55dd56474b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f8752d9d640 1 -- 172.21.15.94:0/1638250222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd56393060 con 0x55dd56469b70 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f873bfff640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f873c0a2230 0x7f873c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f8752d9d640 1 --2- 172.21.15.94:0/1638250222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd56472210 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f87440ed270 tx=0x7f87440f3be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af1457ea04f2589 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f875359e640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f873c0a2230 0x7f873c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f87397fa640 1 -- 172.21.15.94:0/1638250222 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87440b8840 con 0x55dd56469b70 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f873bfff640 1 -- 172.21.15.94:0/319605431 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f87440efae0 con 0x7f87540688c0 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f87397fa640 1 -- 172.21.15.94:0/1638250222 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87440bc370 con 0x55dd56469b70 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.598+0000 7f87397fa640 1 -- 172.21.15.94:0/1638250222 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87440333a0 con 0x55dd56469b70 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.599+0000 7f875bd7e740 1 -- 172.21.15.94:0/1638250222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 msgr2=0x55dd56472210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.599+0000 7f875359e640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f873c0a2230 0x7f873c0a46f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f874c005fd0 tx=0x7f874c009500 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.599+0000 7f875bd7e740 1 --2- 172.21.15.94:0/1638250222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd56472210 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f87440ed270 tx=0x7f87440f3be0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.599+0000 7f875bd7e740 1 -- 172.21.15.94:0/1638250222 shutdown_connections 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.599+0000 7f875bd7e740 1 --2- 172.21.15.94:0/1638250222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 0x55dd56474b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.599+0000 7f875bd7e740 1 --2- 172.21.15.94:0/1638250222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd56472210 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.599+0000 7f875bd7e740 1 --2- 172.21.15.94:0/1638250222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469250 0x55dd56469630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.599+0000 7f875bd7e740 1 -- 172.21.15.94:0/1638250222 >> 172.21.15.94:0/1638250222 conn(0x55dd56463980 msgr2=0x55dd56463d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.599+0000 7f875bd7e740 1 -- 172.21.15.94:0/1638250222 shutdown_connections 2024-02-17T12:12:43.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.599+0000 7f875bd7e740 1 -- 172.21.15.94:0/1638250222 wait complete. 2024-02-17T12:12:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.599+0000 7f875bd7e740 1 -- start start 2024-02-17T12:12:43.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.600+0000 7f875bd7e740 1 Processor -- start 2024-02-17T12:12:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.601+0000 7f875bd7e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56469250 0x55dd5646f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.601+0000 7f875bd7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd5646fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875bd7e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56472750 0x55dd56470080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875bd7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd563d1020 con 0x55dd56469b70 2024-02-17T12:12:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875bd7e740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd560a4430 con 0x55dd56469250 2024-02-17T12:12:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875bd7e740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd56060e00 con 0x55dd56472750 2024-02-17T12:12:43.606 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875bd7e740 10 client.0 _ll_register_callbacks cb 0x55dd56378cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd5646fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875359e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56472750 0x55dd56470080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875359e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56472750 0x55dd56470080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39886/0 (socket says 172.21.15.94:39886) 2024-02-17T12:12:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875359e640 1 -- 172.21.15.94:0/3797201015 learned_addr learned my addr 172.21.15.94:0/3797201015 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd5646fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34844/0 (socket says 172.21.15.94:34844) 2024-02-17T12:12:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f8752d9d640 1 --2- 172.21.15.94:0/3797201015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56469250 0x55dd5646f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875359e640 1 -- 172.21.15.94:0/3797201015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56469250 msgr2=0x55dd5646f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875359e640 1 --2- 172.21.15.94:0/3797201015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56469250 0x55dd5646f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875359e640 1 -- 172.21.15.94:0/3797201015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 msgr2=0x55dd5646fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875359e640 1 --2- 172.21.15.94:0/3797201015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd5646fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.602+0000 7f875359e640 1 -- 172.21.15.94:0/3797201015 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87541aaad0 con 0x55dd56472750 2024-02-17T12:12:43.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.613+0000 7f875359e640 1 --2- 172.21.15.94:0/3797201015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56472750 0x55dd56470080 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f874c0099c0 tx=0x7f874c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9960006ff5fbd93e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.617+0000 7f8736ff5640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:12:43.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.623+0000 7f8736ff5640 1 -- 172.21.15.94:0/3797201015 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c016070 con 0x55dd56472750 2024-02-17T12:12:43.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.623+0000 7f8736ff5640 1 -- 172.21.15.94:0/3797201015 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f874c004b20 con 0x55dd56472750 2024-02-17T12:12:43.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.624+0000 7f875bd7e740 1 -- 172.21.15.94:0/3797201015 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd560d07e0 con 0x55dd56472750 2024-02-17T12:12:43.633 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.630+0000 7f8736ff5640 1 -- 172.21.15.94:0/3797201015 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f874c004ce0 con 0x55dd56472750 2024-02-17T12:12:43.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.632+0000 7f875bd7e740 10 client.28681 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:43.634 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.632+0000 7f875bd7e740 1 -- 172.21.15.94:0/3797201015 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dd563cf6a0 con 0x55dd56472750 2024-02-17T12:12:43.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.634+0000 7f8736ff5640 1 -- 172.21.15.94:0/3797201015 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55dd560d07e0 con 0x55dd56472750 2024-02-17T12:12:43.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.634+0000 7f8736ff5640 1 -- 172.21.15.94:0/3797201015 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f874c0055a0 con 0x55dd56472750 2024-02-17T12:12:43.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.635+0000 7f875bd7e740 10 client.28681 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:43.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.635+0000 7f875bd7e740 10 client.28681 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:12:43.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.635+0000 7f875bd7e740 20 client.28681 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:43.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.635+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3797201015 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55dd56482730 0x55dd56484b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.635+0000 7f875bd7e740 4 client.28681 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:43.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.635+0000 7f875bd7e740 1 -- 172.21.15.94:0/3797201015 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd5610d720 con 0x55dd56482730 2024-02-17T12:12:43.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.640+0000 7f8752d9d640 1 --2- 172.21.15.94:0/3797201015 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55dd56482730 0x55dd56484b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.650+0000 7f8752d9d640 1 --2- 172.21.15.94:0/3797201015 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55dd56482730 0x55dd56484b50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=811983fb5febe785 server_cookie=6afca2f8b46dfb51 in_seq=0 out_seq=0 2024-02-17T12:12:43.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.650+0000 7f8736ff5640 10 client.28681 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:12:43.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.651+0000 7f8736ff5640 1 -- 172.21.15.94:0/3797201015 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd5610d720 con 0x55dd56482730 2024-02-17T12:12:43.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.651+0000 7f8736ff5640 10 client.28681 handle_command_reply: tid=0 2024-02-17T12:12:43.654 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.651+0000 7f875bd7e740 1 client.28681 shutdown 2024-02-17T12:12:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.653+0000 7f875bd7e740 1 -- 172.21.15.94:0/3797201015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56472750 msgr2=0x55dd56470080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.657 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.653+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3797201015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56472750 0x55dd56470080 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f874c0099c0 tx=0x7f874c004ab0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.657+0000 7f875bd7e740 1 -- 172.21.15.94:0/3797201015 shutdown_connections 2024-02-17T12:12:43.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.657+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3797201015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56472750 0x55dd56470080 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.657+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3797201015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd5646fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.657+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3797201015 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55dd56482730 0x55dd56484b50 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.657+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3797201015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56469250 0x55dd5646f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.659 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.657+0000 7f875bd7e740 1 -- 172.21.15.94:0/3797201015 >> 172.21.15.94:0/3797201015 conn(0x55dd56463980 msgr2=0x55dd564656d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.657+0000 7f8736ff5640 0 client.28681 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:12:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.658+0000 7f875bd7e740 1 -- 172.21.15.94:0/3797201015 shutdown_connections 2024-02-17T12:12:43.660 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.658+0000 7f875bd7e740 1 -- 172.21.15.94:0/3797201015 wait complete. 2024-02-17T12:12:43.664 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.663+0000 7f875bd7e740 20 client.28681 trim_cache size 0 max 16384 2024-02-17T12:12:43.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.683+0000 7f875bd7e740 1 Processor -- start 2024-02-17T12:12:43.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.683+0000 7f875bd7e740 1 -- start start 2024-02-17T12:12:43.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f875bd7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469250 0x55dd56471d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f875bd7e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469b70 0x55dd5646e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f875bd7e740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 0x55dd5646fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f875bd7e740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd56356130 con 0x55dd56469250 2024-02-17T12:12:43.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f8752d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469250 0x55dd56471d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f8752d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469250 0x55dd56471d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34858/0 (socket says 172.21.15.94:34858) 2024-02-17T12:12:43.686 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f8752d9d640 1 -- 172.21.15.94:0/3583562621 learned_addr learned my addr 172.21.15.94:0/3583562621 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f875259c640 1 --2- 172.21.15.94:0/3583562621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469b70 0x55dd5646e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f875bd7e740 1 -- 172.21.15.94:0/3583562621 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd5645f350 con 0x55dd56472750 2024-02-17T12:12:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f875bd7e740 1 -- 172.21.15.94:0/3583562621 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd564725b0 con 0x55dd56469b70 2024-02-17T12:12:43.687 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f875359e640 1 --2- 172.21.15.94:0/3583562621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 0x55dd5646fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f875259c640 1 -- 172.21.15.94:0/3583562621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 msgr2=0x55dd5646fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f875259c640 1 --2- 172.21.15.94:0/3583562621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 0x55dd5646fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f875259c640 1 -- 172.21.15.94:0/3583562621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469250 msgr2=0x55dd56471d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f875259c640 1 --2- 172.21.15.94:0/3583562621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469250 0x55dd56471d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.684+0000 7f875259c640 1 -- 172.21.15.94:0/3583562621 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd5636bc20 con 0x55dd56469b70 2024-02-17T12:12:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.685+0000 7f875259c640 1 --2- 172.21.15.94:0/3583562621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469b70 0x55dd5646e5e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55dd5646ee60 tx=0x7f8748009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d01094ff210cf49f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.685+0000 7f8738ff9640 1 -- 172.21.15.94:0/3583562621 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748017070 con 0x55dd56469b70 2024-02-17T12:12:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.685+0000 7f8738ff9640 1 -- 172.21.15.94:0/3583562621 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8748009d80 con 0x55dd56469b70 2024-02-17T12:12:43.688 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.685+0000 7f8738ff9640 1 -- 172.21.15.94:0/3583562621 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748012e80 con 0x55dd56469b70 2024-02-17T12:12:43.689 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.686+0000 7f875bd7e740 1 -- 172.21.15.94:0/3583562621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469b70 msgr2=0x55dd5646e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.686+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3583562621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469b70 0x55dd5646e5e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x55dd5646ee60 tx=0x7f8748009b90 comp rx=0 tx=0).stop 2024-02-17T12:12:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.686+0000 7f875bd7e740 1 -- 172.21.15.94:0/3583562621 shutdown_connections 2024-02-17T12:12:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.686+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3583562621 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 0x55dd5646fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.687+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3583562621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469b70 0x55dd5646e5e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.687+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3583562621 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469250 0x55dd56471d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.687+0000 7f875bd7e740 1 -- 172.21.15.94:0/3583562621 >> 172.21.15.94:0/3583562621 conn(0x55dd56463980 msgr2=0x55dd5646ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.687+0000 7f875bd7e740 1 -- 172.21.15.94:0/3583562621 shutdown_connections 2024-02-17T12:12:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.687+0000 7f875bd7e740 1 -- 172.21.15.94:0/3583562621 wait complete. 2024-02-17T12:12:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.687+0000 7f875bd7e740 1 -- start start 2024-02-17T12:12:43.690 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.688+0000 7f875bd7e740 1 Processor -- start 2024-02-17T12:12:43.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.688+0000 7f875bd7e740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469250 0x55dd56478f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.688+0000 7f875bd7e740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd564794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.688+0000 7f8752d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469250 0x55dd56478f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.688+0000 7f8752d9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469250 0x55dd56478f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39898/0 (socket says 172.21.15.94:39898) 2024-02-17T12:12:43.693 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.688+0000 7f875259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd564794c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.688+0000 7f875259c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd564794c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34872/0 (socket says 172.21.15.94:34872) 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.688+0000 7f8752d9d640 1 -- 172.21.15.94:0/3773338342 learned_addr learned my addr 172.21.15.94:0/3773338342 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.689+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 0x55dd56477420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.689+0000 7f875bd7e740 1 -- 172.21.15.94:0/3773338342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd564779b0 con 0x55dd56469b70 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.689+0000 7f875bd7e740 1 -- 172.21.15.94:0/3773338342 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd56477b20 con 0x55dd56472750 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.689+0000 7f875359e640 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 0x55dd56477420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.689+0000 7f875bd7e740 1 -- 172.21.15.94:0/3773338342 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd56477ce0 con 0x55dd56469250 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.689+0000 7f875bd7e740 10 client.0 _ll_register_callbacks cb 0x7f87440f34c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.689+0000 7f875259c640 1 -- 172.21.15.94:0/3773338342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469250 msgr2=0x55dd56478f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.689+0000 7f875259c640 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469250 0x55dd56478f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.689+0000 7f875259c640 1 -- 172.21.15.94:0/3773338342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 msgr2=0x55dd56477420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.689+0000 7f875259c640 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 0x55dd56477420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.689+0000 7f875259c640 1 -- 172.21.15.94:0/3773338342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87480043d0 con 0x55dd56469b70 2024-02-17T12:12:43.694 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.689+0000 7f875359e640 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 0x55dd56477420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.690+0000 7f875259c640 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd564794c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x55dd5610d860 tx=0x7f8748012f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7809820f90525e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.690+0000 7f8736ff5640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.690+0000 7f8736ff5640 1 -- 172.21.15.94:0/3773338342 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748017030 con 0x55dd56469b70 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.690+0000 7f8736ff5640 1 -- 172.21.15.94:0/3773338342 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8748012760 con 0x55dd56469b70 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.690+0000 7f875bd7e740 1 -- 172.21.15.94:0/3773338342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87440bc370 con 0x55dd56469b70 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.690+0000 7f8736ff5640 1 -- 172.21.15.94:0/3773338342 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8748012960 con 0x55dd56469b70 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.690+0000 7f875bd7e740 10 client.19266 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.690+0000 7f875bd7e740 1 -- 172.21.15.94:0/3773338342 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55dd563918c0 con 0x55dd56469b70 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.690+0000 7f8736ff5640 1 -- 172.21.15.94:0/3773338342 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f87440bc370 con 0x55dd56469b70 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.691+0000 7f8736ff5640 1 -- 172.21.15.94:0/3773338342 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f874801c420 con 0x55dd56469b70 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.691+0000 7f875bd7e740 10 client.19266 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.691+0000 7f875bd7e740 10 client.19266 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.691+0000 7f875bd7e740 20 client.19266 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.691+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55dd564d9e20 0x55dd564dc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.691+0000 7f875bd7e740 4 client.19266 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:12:43.695 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.691+0000 7f875bd7e740 1 -- 172.21.15.94:0/3773338342 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dd5615c890 con 0x55dd564d9e20 2024-02-17T12:12:43.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.691+0000 7f8752d9d640 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55dd564d9e20 0x55dd564dc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.691+0000 7f8736ff5640 10 client.19266 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:12:43.696 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.691+0000 7f8752d9d640 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55dd564d9e20 0x55dd564dc240 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a2f2fe1b726e707 server_cookie=dfeacaa2df93724a in_seq=0 out_seq=0 2024-02-17T12:12:43.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f8736ff5640 1 -- 172.21.15.94:0/3773338342 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dd5615c890 con 0x55dd564d9e20 2024-02-17T12:12:43.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f8736ff5640 10 client.19266 handle_command_reply: tid=0 2024-02-17T12:12:43.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f875bd7e740 1 client.19266 shutdown 2024-02-17T12:12:43.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f875bd7e740 1 -- 172.21.15.94:0/3773338342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 msgr2=0x55dd564794c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd564794c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x55dd5610d860 tx=0x7f8748012f00 comp rx=0 tx=0).stop 2024-02-17T12:12:43.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f875bd7e740 1 -- 172.21.15.94:0/3773338342 shutdown_connections 2024-02-17T12:12:43.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55dd564d9e20 0x55dd564dc240 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55dd56472750 0x55dd56477420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55dd56469b70 0x55dd564794c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f875bd7e740 1 --2- 172.21.15.94:0/3773338342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dd56469250 0x55dd56478f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f8736ff5640 0 client.19266 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:12:43.697 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f875bd7e740 1 -- 172.21.15.94:0/3773338342 >> 172.21.15.94:0/3773338342 conn(0x55dd56463980 msgr2=0x55dd564622f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f875bd7e740 1 -- 172.21.15.94:0/3773338342 shutdown_connections 2024-02-17T12:12:43.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f875bd7e740 1 -- 172.21.15.94:0/3773338342 wait complete. 2024-02-17T12:12:43.698 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f875bd7e740 20 client.19266 trim_cache size 0 max 16384 2024-02-17T12:12:43.698 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:12:43.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f87591ac640 1 -- 172.21.15.94:0/319605431 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f873c0a2230 msgr2=0x7f873c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f87591ac640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f873c0a2230 0x7f873c0a46f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f874c005fd0 tx=0x7f874c009500 comp rx=0 tx=0).stop 2024-02-17T12:12:43.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f87591ac640 1 -- 172.21.15.94:0/319605431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87540688c0 msgr2=0x7f875419dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f87591ac640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87540688c0 0x7f875419dd80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8744000c00 tx=0x7f8744004530 comp rx=0 tx=0).stop 2024-02-17T12:12:43.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f87591ac640 1 -- 172.21.15.94:0/319605431 shutdown_connections 2024-02-17T12:12:43.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.692+0000 7f87591ac640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f873c0a2230 0x7f873c0a46f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.693+0000 7f87591ac640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f87541060d0 0x7f87541a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.693+0000 7f87591ac640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f8754103c40 0x7f875419e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.693+0000 7f87591ac640 1 --2- 172.21.15.94:0/319605431 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87540688c0 0x7f875419dd80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.693+0000 7f87591ac640 1 -- 172.21.15.94:0/319605431 >> 172.21.15.94:0/319605431 conn(0x7f8754077180 msgr2=0x7f8754101070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.693+0000 7f87591ac640 1 -- 172.21.15.94:0/319605431 shutdown_connections 2024-02-17T12:12:43.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.693+0000 7f87591ac640 1 -- 172.21.15.94:0/319605431 wait complete. 2024-02-17T12:12:43.707 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:12:43.707 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:12:43.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.850+0000 7fada55ec640 1 -- 172.21.15.94:0/2114240636 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fada0108da0 msgr2=0x7fada010c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.850+0000 7fada55ec640 1 --2- 172.21.15.94:0/2114240636 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fada0108da0 0x7fada010c9f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fad9800aa40 tx=0x7fad9802cf50 comp rx=0 tx=0).stop 2024-02-17T12:12:43.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.850+0000 7fada55ec640 1 -- 172.21.15.94:0/2114240636 shutdown_connections 2024-02-17T12:12:43.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.850+0000 7fada55ec640 1 --2- 172.21.15.94:0/2114240636 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada010d0d0 0x7fada0114ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.850+0000 7fada55ec640 1 --2- 172.21.15.94:0/2114240636 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fada0108da0 0x7fada010c9f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.850+0000 7fada55ec640 1 --2- 172.21.15.94:0/2114240636 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada01083f0 0x7fada01087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.850+0000 7fada55ec640 1 -- 172.21.15.94:0/2114240636 >> 172.21.15.94:0/2114240636 conn(0x7fada006c4b0 msgr2=0x7fada006c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.850+0000 7fada55ec640 1 -- 172.21.15.94:0/2114240636 shutdown_connections 2024-02-17T12:12:43.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.850+0000 7fada55ec640 1 -- 172.21.15.94:0/2114240636 wait complete. 2024-02-17T12:12:43.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.851+0000 7fada55ec640 1 Processor -- start 2024-02-17T12:12:43.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.851+0000 7fada55ec640 1 -- start start 2024-02-17T12:12:43.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fada55ec640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada01083f0 0x7fada019fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fada55ec640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada010d0d0 0x7fada01a03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fada55ec640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fada01a4740 0x7fada01a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fada55ec640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada01a5130 con 0x7fada010d0d0 2024-02-17T12:12:43.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fada55ec640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada01a52a0 con 0x7fada01a4740 2024-02-17T12:12:43.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fada55ec640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada01a5410 con 0x7fada01083f0 2024-02-17T12:12:43.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fada4deb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fada01a4740 0x7fada01a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada01083f0 0x7fada019fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fad9f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada010d0d0 0x7fada01a03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fada4deb640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fada01a4740 0x7fada01a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:43268/0 (socket says 172.21.15.94:43268) 2024-02-17T12:12:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fad9f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada010d0d0 0x7fada01a03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34900/0 (socket says 172.21.15.94:34900) 2024-02-17T12:12:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fada4deb640 1 -- 172.21.15.94:0/3593544837 learned_addr learned my addr 172.21.15.94:0/3593544837 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fad9f7fe640 1 -- 172.21.15.94:0/3593544837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada01083f0 msgr2=0x7fada019fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fad9f7fe640 1 --2- 172.21.15.94:0/3593544837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada01083f0 0x7fada019fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fad9f7fe640 1 -- 172.21.15.94:0/3593544837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fada01a4740 msgr2=0x7fada01a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fad9f7fe640 1 --2- 172.21.15.94:0/3593544837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fada01a4740 0x7fada01a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.852+0000 7fad9f7fe640 1 -- 172.21.15.94:0/3593544837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad94009490 con 0x7fada010d0d0 2024-02-17T12:12:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.853+0000 7fad9f7fe640 1 --2- 172.21.15.94:0/3593544837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada010d0d0 0x7fada01a03c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fad9802dfd0 tx=0x7fad98005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.853+0000 7fad9d7fa640 1 -- 172.21.15.94:0/3593544837 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad980048c0 con 0x7fada010d0d0 2024-02-17T12:12:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.853+0000 7fad9d7fa640 1 -- 172.21.15.94:0/3593544837 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad980035f0 con 0x7fada010d0d0 2024-02-17T12:12:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.853+0000 7fada55ec640 1 -- 172.21.15.94:0/3593544837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9800a670 con 0x7fada010d0d0 2024-02-17T12:12:43.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.853+0000 7fad9d7fa640 1 -- 172.21.15.94:0/3593544837 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98035760 con 0x7fada010d0d0 2024-02-17T12:12:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.853+0000 7fada55ec640 1 -- 172.21.15.94:0/3593544837 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada01acc80 con 0x7fada010d0d0 2024-02-17T12:12:43.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.855+0000 7fad9d7fa640 1 -- 172.21.15.94:0/3593544837 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fad980039f0 con 0x7fada010d0d0 2024-02-17T12:12:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.855+0000 7fada9427740 1 Processor -- start 2024-02-17T12:12:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fada9427740 1 -- start start 2024-02-17T12:12:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fada9427740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18190 0x55687df18570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fada9427740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18ab0 0x55687df25830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fada9427740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df25d70 0x55687df28160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fada9427740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687ded3960 con 0x55687df25d70 2024-02-17T12:12:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fada9427740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687de87310 con 0x55687df18190 2024-02-17T12:12:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fada9427740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687de84840 con 0x55687df18ab0 2024-02-17T12:12:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fad9d7fa640 1 --2- 172.21.15.94:0/3593544837 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fad880a2230 0x7fad880a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fad9f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df25d70 0x55687df28160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fad9f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df25d70 0x55687df28160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34904/0 (socket says 172.21.15.94:34904) 2024-02-17T12:12:43.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fad9f7fe640 1 -- 172.21.15.94:0/928310207 learned_addr learned my addr 172.21.15.94:0/928310207 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fada4deb640 1 --2- 172.21.15.94:0/928310207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18190 0x55687df18570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fada4deb640 1 --2- 172.21.15.94:0/3593544837 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fad880a2230 0x7fad880a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fad9ffff640 1 --2- 172.21.15.94:0/928310207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18ab0 0x55687df25830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fad9d7fa640 1 -- 172.21.15.94:0/3593544837 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fad980f08b0 con 0x7fada010d0d0 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fad9f7fe640 1 -- 172.21.15.94:0/928310207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18ab0 msgr2=0x55687df25830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fad9f7fe640 1 --2- 172.21.15.94:0/928310207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18ab0 0x55687df25830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fad9f7fe640 1 -- 172.21.15.94:0/928310207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18190 msgr2=0x55687df18570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fad9f7fe640 1 --2- 172.21.15.94:0/928310207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18190 0x55687df18570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fad9f7fe640 1 -- 172.21.15.94:0/928310207 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55687de42060 con 0x55687df25d70 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fada4deb640 1 --2- 172.21.15.94:0/928310207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18190 0x55687df18570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fad9f7fe640 1 --2- 172.21.15.94:0/928310207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df25d70 0x55687df28160 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fad980b9d40 tx=0x7fad98003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7b9cd37afd4a3c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.856+0000 7fada4deb640 1 --2- 172.21.15.94:0/3593544837 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fad880a2230 0x7fad880a46f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fad900099c0 tx=0x7fad90009710 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.857+0000 7fad867fc640 1 -- 172.21.15.94:0/928310207 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98003760 con 0x55687df25d70 2024-02-17T12:12:43.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.857+0000 7fad867fc640 1 -- 172.21.15.94:0/928310207 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad980bd370 con 0x55687df25d70 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.857+0000 7fad867fc640 1 -- 172.21.15.94:0/928310207 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad980ee3d0 con 0x55687df25d70 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.857+0000 7fada9427740 1 -- 172.21.15.94:0/928310207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df25d70 msgr2=0x55687df28160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.857+0000 7fada9427740 1 --2- 172.21.15.94:0/928310207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df25d70 0x55687df28160 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fad980b9d40 tx=0x7fad98003d90 comp rx=0 tx=0).stop 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.857+0000 7fada9427740 1 -- 172.21.15.94:0/928310207 shutdown_connections 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.857+0000 7fada9427740 1 --2- 172.21.15.94:0/928310207 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df25d70 0x55687df28160 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.857+0000 7fada9427740 1 --2- 172.21.15.94:0/928310207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18ab0 0x55687df25830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.857+0000 7fada9427740 1 --2- 172.21.15.94:0/928310207 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18190 0x55687df18570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.857+0000 7fada9427740 1 -- 172.21.15.94:0/928310207 >> 172.21.15.94:0/928310207 conn(0x55687df12b90 msgr2=0x55687df12f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.857+0000 7fada9427740 1 -- 172.21.15.94:0/928310207 shutdown_connections 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.857+0000 7fada9427740 1 -- 172.21.15.94:0/928310207 wait complete. 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fada9427740 1 -- start start 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fada9427740 1 Processor -- start 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fada9427740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18190 0x55687df22d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fada9427740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18ab0 0x55687df23280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fada9427740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df25d70 0x55687df237c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fada9427740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687de80020 con 0x55687df18190 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fada9427740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687db53430 con 0x55687df18ab0 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fada9427740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687db0fe00 con 0x55687df25d70 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fada9427740 10 client.0 _ll_register_callbacks cb 0x55687de27cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18190 0x55687df22d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18190 0x55687df22d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34906/0 (socket says 172.21.15.94:34906) 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fad9ffff640 1 -- 172.21.15.94:0/2746945937 learned_addr learned my addr 172.21.15.94:0/2746945937 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fad9f7fe640 1 --2- 172.21.15.94:0/2746945937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18ab0 0x55687df23280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.858+0000 7fada4deb640 1 --2- 172.21.15.94:0/2746945937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df25d70 0x55687df237c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fad9ffff640 1 -- 172.21.15.94:0/2746945937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df25d70 msgr2=0x55687df237c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fad9ffff640 1 --2- 172.21.15.94:0/2746945937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df25d70 0x55687df237c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fad9ffff640 1 -- 172.21.15.94:0/2746945937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18ab0 msgr2=0x55687df23280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fad9ffff640 1 --2- 172.21.15.94:0/2746945937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18ab0 0x55687df23280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fad9ffff640 1 -- 172.21.15.94:0/2746945937 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada01acc80 con 0x55687df18190 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fad9ffff640 1 --2- 172.21.15.94:0/2746945937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18190 0x55687df22d40 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fad94012770 tx=0x7fad94012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12d4a3fe41a0006 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fad7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fad7f7fe640 1 -- 172.21.15.94:0/2746945937 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94017070 con 0x55687df18190 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fad7f7fe640 1 -- 172.21.15.94:0/2746945937 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fad94011bb0 con 0x55687df18190 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fada9427740 1 -- 172.21.15.94:0/2746945937 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55687db7f7e0 con 0x55687df18190 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fad7f7fe640 1 -- 172.21.15.94:0/2746945937 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad94011d70 con 0x55687df18190 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fada9427740 10 client.19278 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fada9427740 1 -- 172.21.15.94:0/2746945937 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55687de7e6a0 con 0x55687df18190 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fad7f7fe640 1 -- 172.21.15.94:0/2746945937 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55687db7f7e0 con 0x55687df18190 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.859+0000 7fad7f7fe640 1 -- 172.21.15.94:0/2746945937 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fad9401c5e0 con 0x55687df18190 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.860+0000 7fada9427740 10 client.19278 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.860+0000 7fada9427740 10 client.19278 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:12:43.867 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.860+0000 7fada9427740 20 client.19278 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.860+0000 7fada9427740 1 --2- 172.21.15.94:0/2746945937 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55687df36030 0x55687df38440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.860+0000 7fada9427740 4 client.19278 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.860+0000 7fada9427740 1 -- 172.21.15.94:0/2746945937 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55687dbbc720 con 0x55687df36030 2024-02-17T12:12:43.868 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.860+0000 7fad9f7fe640 1 --2- 172.21.15.94:0/2746945937 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55687df36030 0x55687df38440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.861+0000 7fad9f7fe640 1 --2- 172.21.15.94:0/2746945937 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55687df36030 0x55687df38440 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d23d971d4a252f2c server_cookie=1d078d1b41643be1 in_seq=0 out_seq=0 2024-02-17T12:12:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.861+0000 7fad7f7fe640 10 client.19278 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:12:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fad7f7fe640 1 -- 172.21.15.94:0/2746945937 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55687dbbc720 con 0x55687df36030 2024-02-17T12:12:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fad7f7fe640 10 client.19278 handle_command_reply: tid=0 2024-02-17T12:12:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fada9427740 1 client.19278 shutdown 2024-02-17T12:12:43.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fada9427740 1 -- 172.21.15.94:0/2746945937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18190 msgr2=0x55687df22d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fada9427740 1 --2- 172.21.15.94:0/2746945937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18190 0x55687df22d40 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fad94012770 tx=0x7fad94012c30 comp rx=0 tx=0).stop 2024-02-17T12:12:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fada9427740 1 -- 172.21.15.94:0/2746945937 shutdown_connections 2024-02-17T12:12:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fada9427740 1 --2- 172.21.15.94:0/2746945937 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55687df36030 0x55687df38440 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fada9427740 1 --2- 172.21.15.94:0/2746945937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df25d70 0x55687df237c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fada9427740 1 --2- 172.21.15.94:0/2746945937 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18ab0 0x55687df23280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fad7f7fe640 0 client.19278 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:12:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fada9427740 1 --2- 172.21.15.94:0/2746945937 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18190 0x55687df22d40 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fada9427740 1 -- 172.21.15.94:0/2746945937 >> 172.21.15.94:0/2746945937 conn(0x55687df12b90 msgr2=0x55687df148e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fada9427740 1 -- 172.21.15.94:0/2746945937 shutdown_connections 2024-02-17T12:12:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fada9427740 1 -- 172.21.15.94:0/2746945937 wait complete. 2024-02-17T12:12:43.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.866+0000 7fada9427740 20 client.19278 trim_cache size 0 max 16384 2024-02-17T12:12:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.881+0000 7fada9427740 1 Processor -- start 2024-02-17T12:12:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.881+0000 7fada9427740 1 -- start start 2024-02-17T12:12:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.882+0000 7fada9427740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18190 0x55687df146d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.885 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.882+0000 7fada9427740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18ab0 0x55687df2c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.882+0000 7fada9427740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df25d70 0x55687df2cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.882+0000 7fada9427740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687de05130 con 0x55687df25d70 2024-02-17T12:12:43.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.882+0000 7fada9427740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687df12450 con 0x55687df18ab0 2024-02-17T12:12:43.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.882+0000 7fada9427740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687df13760 con 0x55687df18190 2024-02-17T12:12:43.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.882+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18190 0x55687df146d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.882+0000 7fad9ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18190 0x55687df146d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:39940/0 (socket says 172.21.15.94:39940) 2024-02-17T12:12:43.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.882+0000 7fad9ffff640 1 -- 172.21.15.94:0/3106768836 learned_addr learned my addr 172.21.15.94:0/3106768836 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:43.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.882+0000 7fada4deb640 1 --2- 172.21.15.94:0/3106768836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df25d70 0x55687df2cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.882+0000 7fad9f7fe640 1 --2- 172.21.15.94:0/3106768836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18ab0 0x55687df2c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.883+0000 7fad9ffff640 1 -- 172.21.15.94:0/3106768836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18ab0 msgr2=0x55687df2c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.883+0000 7fad9ffff640 1 --2- 172.21.15.94:0/3106768836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18ab0 0x55687df2c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.886 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.883+0000 7fad9ffff640 1 -- 172.21.15.94:0/3106768836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df25d70 msgr2=0x55687df2cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.883+0000 7fad9ffff640 1 --2- 172.21.15.94:0/3106768836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df25d70 0x55687df2cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.883+0000 7fad9ffff640 1 -- 172.21.15.94:0/3106768836 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55687de1ac20 con 0x55687df18190 2024-02-17T12:12:43.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.883+0000 7fad9ffff640 1 --2- 172.21.15.94:0/3106768836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18190 0x55687df146d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fad94005c20 tx=0x7fad94011890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91107e4d5f68080 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.883+0000 7fad85ffb640 1 -- 172.21.15.94:0/3106768836 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94017070 con 0x55687df18190 2024-02-17T12:12:43.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.883+0000 7fad85ffb640 1 -- 172.21.15.94:0/3106768836 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad9401cb80 con 0x55687df18190 2024-02-17T12:12:43.887 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.883+0000 7fad85ffb640 1 -- 172.21.15.94:0/3106768836 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9400f940 con 0x55687df18190 2024-02-17T12:12:43.891 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.884+0000 7fada9427740 1 -- 172.21.15.94:0/3106768836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18190 msgr2=0x55687df146d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.884+0000 7fada9427740 1 --2- 172.21.15.94:0/3106768836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18190 0x55687df146d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fad94005c20 tx=0x7fad94011890 comp rx=0 tx=0).stop 2024-02-17T12:12:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.884+0000 7fada9427740 1 -- 172.21.15.94:0/3106768836 shutdown_connections 2024-02-17T12:12:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.884+0000 7fada9427740 1 --2- 172.21.15.94:0/3106768836 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df25d70 0x55687df2cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.884+0000 7fada9427740 1 --2- 172.21.15.94:0/3106768836 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df18ab0 0x55687df2c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.884+0000 7fada9427740 1 --2- 172.21.15.94:0/3106768836 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18190 0x55687df146d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.884+0000 7fada9427740 1 -- 172.21.15.94:0/3106768836 >> 172.21.15.94:0/3106768836 conn(0x55687df12b90 msgr2=0x55687df13fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.884+0000 7fada9427740 1 -- 172.21.15.94:0/3106768836 shutdown_connections 2024-02-17T12:12:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.884+0000 7fada9427740 1 -- 172.21.15.94:0/3106768836 wait complete. 2024-02-17T12:12:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.884+0000 7fada9427740 1 -- start start 2024-02-17T12:12:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.884+0000 7fada9427740 1 Processor -- start 2024-02-17T12:12:43.892 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fada9427740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18190 0x55687df2b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fada9427740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18ab0 0x55687df2ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fada9427740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df25d70 0x55687df2bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fada9427740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687df8c590 con 0x55687df18ab0 2024-02-17T12:12:43.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fada9427740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687df8c700 con 0x55687df25d70 2024-02-17T12:12:43.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fada9427740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55687df8c8c0 con 0x55687df18190 2024-02-17T12:12:43.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fada9427740 10 client.0 _ll_register_callbacks cb 0x7fad980bd140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:43.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fad9f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18ab0 0x55687df2ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fad9f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18ab0 0x55687df2ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34928/0 (socket says 172.21.15.94:34928) 2024-02-17T12:12:43.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fad9f7fe640 1 -- 172.21.15.94:0/2194795057 learned_addr learned my addr 172.21.15.94:0/2194795057 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:43.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fada4deb640 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df25d70 0x55687df2bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fad9ffff640 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18190 0x55687df2b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fad9f7fe640 1 -- 172.21.15.94:0/2194795057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18190 msgr2=0x55687df2b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fad9f7fe640 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18190 0x55687df2b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fad9f7fe640 1 -- 172.21.15.94:0/2194795057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df25d70 msgr2=0x55687df2bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:43.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fad9f7fe640 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df25d70 0x55687df2bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fad9f7fe640 1 -- 172.21.15.94:0/2194795057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55687de7e6a0 con 0x55687df18ab0 2024-02-17T12:12:43.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fada4deb640 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df25d70 0x55687df2bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:43.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fad9f7fe640 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18ab0 0x55687df2ba70 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fad980b9d40 tx=0x7fad980057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e65e98dcffa8410 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:43.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fad7f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:43.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fad7f7fe640 1 -- 172.21.15.94:0/2194795057 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad980f0b70 con 0x55687df18ab0 2024-02-17T12:12:43.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fad7f7fe640 1 -- 172.21.15.94:0/2194795057 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fad980c14c0 con 0x55687df18ab0 2024-02-17T12:12:43.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.885+0000 7fada9427740 1 -- 172.21.15.94:0/2194795057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad980bd370 con 0x55687df18ab0 2024-02-17T12:12:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.886+0000 7fad7f7fe640 1 -- 172.21.15.94:0/2194795057 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad98003760 con 0x55687df18ab0 2024-02-17T12:12:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.886+0000 7fada9427740 10 client.19284 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.886+0000 7fada9427740 1 -- 172.21.15.94:0/2194795057 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55687de408c0 con 0x55687df18ab0 2024-02-17T12:12:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.886+0000 7fad7f7fe640 1 -- 172.21.15.94:0/2194795057 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad980d8020 con 0x55687df18ab0 2024-02-17T12:12:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.887+0000 7fad7f7fe640 1 -- 172.21.15.94:0/2194795057 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fad980bd5d0 con 0x55687df18ab0 2024-02-17T12:12:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.887+0000 7fada9427740 10 client.19284 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.887+0000 7fada9427740 10 client.19284 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:12:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.887+0000 7fada9427740 20 client.19284 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.887+0000 7fada9427740 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55687df8dcc0 0x55687df900e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.887+0000 7fada9427740 4 client.19284 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:12:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.887+0000 7fada9427740 1 -- 172.21.15.94:0/2194795057 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55687dc0b890 con 0x55687df8dcc0 2024-02-17T12:12:43.895 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.887+0000 7fad9ffff640 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55687df8dcc0 0x55687df900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.887+0000 7fad7f7fe640 10 client.19284 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:12:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.887+0000 7fad9ffff640 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55687df8dcc0 0x55687df900e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=936162e8efc63512 server_cookie=2328be8f034ae9b3 in_seq=0 out_seq=0 2024-02-17T12:12:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fad7f7fe640 1 -- 172.21.15.94:0/2194795057 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55687dc0b890 con 0x55687df8dcc0 2024-02-17T12:12:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fad7f7fe640 10 client.19284 handle_command_reply: tid=0 2024-02-17T12:12:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada9427740 1 client.19284 shutdown 2024-02-17T12:12:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada9427740 1 -- 172.21.15.94:0/2194795057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18ab0 msgr2=0x55687df2ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada9427740 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18ab0 0x55687df2ba70 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fad980b9d40 tx=0x7fad980057b0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada9427740 1 -- 172.21.15.94:0/2194795057 shutdown_connections 2024-02-17T12:12:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada9427740 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55687df8dcc0 0x55687df900e0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada9427740 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55687df25d70 0x55687df2bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.896 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fad7f7fe640 0 client.19284 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:12:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada9427740 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55687df18ab0 0x55687df2ba70 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada9427740 1 --2- 172.21.15.94:0/2194795057 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55687df18190 0x55687df2b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada9427740 1 -- 172.21.15.94:0/2194795057 >> 172.21.15.94:0/2194795057 conn(0x55687df12b90 msgr2=0x55687df14850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada9427740 1 -- 172.21.15.94:0/2194795057 shutdown_connections 2024-02-17T12:12:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada9427740 1 -- 172.21.15.94:0/2194795057 wait complete. 2024-02-17T12:12:43.897 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada9427740 20 client.19284 trim_cache size 0 max 16384 2024-02-17T12:12:43.897 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:12:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada55ec640 1 -- 172.21.15.94:0/3593544837 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fad880a2230 msgr2=0x7fad880a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada55ec640 1 --2- 172.21.15.94:0/3593544837 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fad880a2230 0x7fad880a46f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fad900099c0 tx=0x7fad90009710 comp rx=0 tx=0).stop 2024-02-17T12:12:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada55ec640 1 -- 172.21.15.94:0/3593544837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada010d0d0 msgr2=0x7fada01a03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.888+0000 7fada55ec640 1 --2- 172.21.15.94:0/3593544837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada010d0d0 0x7fada01a03c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fad9802dfd0 tx=0x7fad98005240 comp rx=0 tx=0).stop 2024-02-17T12:12:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.889+0000 7fada55ec640 1 -- 172.21.15.94:0/3593544837 shutdown_connections 2024-02-17T12:12:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.889+0000 7fada55ec640 1 --2- 172.21.15.94:0/3593544837 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fad880a2230 0x7fad880a46f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.889+0000 7fada55ec640 1 --2- 172.21.15.94:0/3593544837 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fada01a4740 0x7fada01a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.898 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.889+0000 7fada55ec640 1 --2- 172.21.15.94:0/3593544837 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fada010d0d0 0x7fada01a03c0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.889+0000 7fada55ec640 1 --2- 172.21.15.94:0/3593544837 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fada01083f0 0x7fada019fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.889+0000 7fada55ec640 1 -- 172.21.15.94:0/3593544837 >> 172.21.15.94:0/3593544837 conn(0x7fada006c4b0 msgr2=0x7fada0070dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.889+0000 7fada55ec640 1 -- 172.21.15.94:0/3593544837 shutdown_connections 2024-02-17T12:12:43.899 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:43.889+0000 7fada55ec640 1 -- 172.21.15.94:0/3593544837 wait complete. 2024-02-17T12:12:43.905 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:12:43.906 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:12:44.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.053+0000 7fbeb929d640 1 -- 172.21.15.94:0/169397285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb410d9d0 msgr2=0x7fbeb410de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:44.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.053+0000 7fbeb929d640 1 --2- 172.21.15.94:0/169397285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb410d9d0 0x7fbeb410de50 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009ca0 tx=0x7fbeac02c140 comp rx=0 tx=0).stop 2024-02-17T12:12:44.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.058+0000 7fbeb929d640 1 -- 172.21.15.94:0/169397285 shutdown_connections 2024-02-17T12:12:44.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.058+0000 7fbeb929d640 1 --2- 172.21.15.94:0/169397285 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb411a960 0x7fbeb411cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.058+0000 7fbeb929d640 1 --2- 172.21.15.94:0/169397285 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb410d9d0 0x7fbeb410de50 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.058+0000 7fbeb929d640 1 --2- 172.21.15.94:0/169397285 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbeb410f9a0 0x7fbeb410d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.058+0000 7fbeb929d640 1 -- 172.21.15.94:0/169397285 >> 172.21.15.94:0/169397285 conn(0x7fbeb406c260 msgr2=0x7fbeb406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:44.068 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.067+0000 7fbeb929d640 1 -- 172.21.15.94:0/169397285 shutdown_connections 2024-02-17T12:12:44.072 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.069+0000 7fbeb929d640 1 -- 172.21.15.94:0/169397285 wait complete. 2024-02-17T12:12:44.073 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.070+0000 7fbeb929d640 1 Processor -- start 2024-02-17T12:12:44.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.074+0000 7fbeb929d640 1 -- start start 2024-02-17T12:12:44.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.074+0000 7fbeb929d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb410d9d0 0x7fbeb4119fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.074+0000 7fbeb929d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb410f9a0 0x7fbeb41140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.074+0000 7fbeb929d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbeb411a960 0x7fbeb41145e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.074+0000 7fbeb929d640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb4114be0 con 0x7fbeb410d9d0 2024-02-17T12:12:44.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.074+0000 7fbeb929d640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb4114d50 con 0x7fbeb410f9a0 2024-02-17T12:12:44.077 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.074+0000 7fbeb929d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb4114f10 con 0x7fbeb411a960 2024-02-17T12:12:44.079 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.076+0000 7fbeb27fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb410f9a0 0x7fbeb41140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.077+0000 7fbeb27fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb410f9a0 0x7fbeb41140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:43292/0 (socket says 172.21.15.94:43292) 2024-02-17T12:12:44.080 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.077+0000 7fbeb27fc640 1 -- 172.21.15.94:0/3291908520 learned_addr learned my addr 172.21.15.94:0/3291908520 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:44.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.080+0000 7fbeb2ffd640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb410d9d0 0x7fbeb4119fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.085 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.083+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbeb411a960 0x7fbeb41145e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.083+0000 7fbeb2ffd640 1 -- 172.21.15.94:0/3291908520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbeb411a960 msgr2=0x7fbeb41145e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:44.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.083+0000 7fbeb2ffd640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbeb411a960 0x7fbeb41145e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.083+0000 7fbeb2ffd640 1 -- 172.21.15.94:0/3291908520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb410f9a0 msgr2=0x7fbeb41140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:44.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.083+0000 7fbeb2ffd640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb410f9a0 0x7fbeb41140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.083+0000 7fbeb2ffd640 1 -- 172.21.15.94:0/3291908520 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac009900 con 0x7fbeb410d9d0 2024-02-17T12:12:44.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.084+0000 7fbeb2ffd640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb410d9d0 0x7fbeb4119fe0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009fd0 tx=0x7fbea800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:44.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.086+0000 7fbe9bfff640 1 -- 172.21.15.94:0/3291908520 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea8011830 con 0x7fbeb410d9d0 2024-02-17T12:12:44.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.086+0000 7fbe9bfff640 1 -- 172.21.15.94:0/3291908520 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbea8011e70 con 0x7fbeb410d9d0 2024-02-17T12:12:44.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.086+0000 7fbe9bfff640 1 -- 172.21.15.94:0/3291908520 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800f2e0 con 0x7fbeb410d9d0 2024-02-17T12:12:44.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.088+0000 7fbeb929d640 1 -- 172.21.15.94:0/3291908520 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb41bb6d0 con 0x7fbeb410d9d0 2024-02-17T12:12:44.091 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.088+0000 7fbeb929d640 1 -- 172.21.15.94:0/3291908520 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb41bbb60 con 0x7fbeb410d9d0 2024-02-17T12:12:44.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.091+0000 7fbebbe76740 1 Processor -- start 2024-02-17T12:12:44.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.094+0000 7fbebbe76740 1 -- start start 2024-02-17T12:12:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.104+0000 7fbebbe76740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4a250 0x557c0ba4a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.104+0000 7fbebbe76740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba4ab70 0x557c0ba53210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.104+0000 7fbebbe76740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba53750 0x557c0ba55b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.104+0000 7fbebbe76740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0ba05960 con 0x557c0ba4a250 2024-02-17T12:12:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.104+0000 7fbebbe76740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0b9b9310 con 0x557c0ba4ab70 2024-02-17T12:12:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.104+0000 7fbebbe76740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0b9b6840 con 0x557c0ba53750 2024-02-17T12:12:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.106+0000 7fbeb37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4a250 0x557c0ba4a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.106+0000 7fbeb37fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4a250 0x557c0ba4a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34946/0 (socket says 172.21.15.94:34946) 2024-02-17T12:12:44.113 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.106+0000 7fbeb37fe640 1 -- 172.21.15.94:0/2209179824 learned_addr learned my addr 172.21.15.94:0/2209179824 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:44.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.107+0000 7fbeb27fc640 1 --2- 172.21.15.94:0/2209179824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba53750 0x557c0ba55b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.114 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.111+0000 7fbeb37fe640 1 -- 172.21.15.94:0/2209179824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba53750 msgr2=0x557c0ba55b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:44.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.111+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/2209179824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba53750 0x557c0ba55b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.111+0000 7fbeb37fe640 1 -- 172.21.15.94:0/2209179824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba4ab70 msgr2=0x557c0ba53210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:12:44.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.111+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/2209179824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba4ab70 0x557c0ba53210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.111+0000 7fbeb37fe640 1 -- 172.21.15.94:0/2209179824 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c0b974060 con 0x557c0ba4a250 2024-02-17T12:12:44.115 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.112+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/2209179824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4a250 0x557c0ba4a630 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fbea40099c0 tx=0x7fbea4009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc854076aeb8230 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.114+0000 7fbe997fa640 1 -- 172.21.15.94:0/2209179824 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40094c0 con 0x557c0ba4a250 2024-02-17T12:12:44.116 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.114+0000 7fbe997fa640 1 -- 172.21.15.94:0/2209179824 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbea4013470 con 0x557c0ba4a250 2024-02-17T12:12:44.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.115+0000 7fbe9bfff640 1 -- 172.21.15.94:0/3291908520 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fbea8010380 con 0x7fbeb410d9d0 2024-02-17T12:12:44.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.115+0000 7fbe9bfff640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbe9c0a2440 0x7fbe9c0a4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.117 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.115+0000 7fbebbe76740 1 -- 172.21.15.94:0/2209179824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4a250 msgr2=0x557c0ba4a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:44.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.115+0000 7fbebbe76740 1 --2- 172.21.15.94:0/2209179824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4a250 0x557c0ba4a630 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fbea40099c0 tx=0x7fbea4009e80 comp rx=0 tx=0).stop 2024-02-17T12:12:44.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.116+0000 7fbe9bfff640 1 -- 172.21.15.94:0/3291908520 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbea80cd9f0 con 0x7fbeb410d9d0 2024-02-17T12:12:44.118 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.116+0000 7fbebbe76740 1 -- 172.21.15.94:0/2209179824 shutdown_connections 2024-02-17T12:12:44.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.116+0000 7fbebbe76740 1 --2- 172.21.15.94:0/2209179824 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba53750 0x557c0ba55b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.116+0000 7fbebbe76740 1 --2- 172.21.15.94:0/2209179824 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba4ab70 0x557c0ba53210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.116+0000 7fbebbe76740 1 --2- 172.21.15.94:0/2209179824 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4a250 0x557c0ba4a630 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.116+0000 7fbebbe76740 1 -- 172.21.15.94:0/2209179824 >> 172.21.15.94:0/2209179824 conn(0x557c0ba44980 msgr2=0x557c0ba44d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:44.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.116+0000 7fbeb2ffd640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbe9c0a2440 0x7fbe9c0a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.116+0000 7fbebbe76740 1 -- 172.21.15.94:0/2209179824 shutdown_connections 2024-02-17T12:12:44.119 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.116+0000 7fbebbe76740 1 -- 172.21.15.94:0/2209179824 wait complete. 2024-02-17T12:12:44.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.117+0000 7fbebbe76740 1 -- start start 2024-02-17T12:12:44.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.117+0000 7fbebbe76740 1 Processor -- start 2024-02-17T12:12:44.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.117+0000 7fbebbe76740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4a250 0x557c0ba50670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.117+0000 7fbebbe76740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba4ab70 0x557c0ba50bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.117+0000 7fbebbe76740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba53750 0x557c0ba510f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.117+0000 7fbebbe76740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0b9b2020 con 0x557c0ba4a250 2024-02-17T12:12:44.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.117+0000 7fbebbe76740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0b685430 con 0x557c0ba4ab70 2024-02-17T12:12:44.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.117+0000 7fbebbe76740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0b641e00 con 0x557c0ba53750 2024-02-17T12:12:44.120 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.117+0000 7fbebbe76740 10 client.0 _ll_register_callbacks cb 0x557c0b959cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:44.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.118+0000 7fbeb27fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba4ab70 0x557c0ba50bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.121 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.118+0000 7fbeb27fc640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba4ab70 0x557c0ba50bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:43296/0 (socket says 172.21.15.94:43296) 2024-02-17T12:12:44.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.118+0000 7fbeb27fc640 1 -- 172.21.15.94:0/3838999943 learned_addr learned my addr 172.21.15.94:0/3838999943 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:44.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.118+0000 7fbeb2ffd640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbe9c0a2440 0x7fbe9c0a4900 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x557c0ba49d20 tx=0x7fbea8096040 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:44.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.119+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/3838999943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4a250 0x557c0ba50670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.119+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/3838999943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba53750 0x557c0ba510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.119+0000 7fbeb27fc640 1 -- 172.21.15.94:0/3838999943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba53750 msgr2=0x557c0ba510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:44.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.119+0000 7fbeb27fc640 1 --2- 172.21.15.94:0/3838999943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba53750 0x557c0ba510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.119+0000 7fbeb27fc640 1 -- 172.21.15.94:0/3838999943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4a250 msgr2=0x557c0ba50670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:44.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.119+0000 7fbeb27fc640 1 --2- 172.21.15.94:0/3838999943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4a250 0x557c0ba50670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.119+0000 7fbeb27fc640 1 -- 172.21.15.94:0/3838999943 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea4009620 con 0x557c0ba4ab70 2024-02-17T12:12:44.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.119+0000 7fbeb27fc640 1 --2- 172.21.15.94:0/3838999943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba4ab70 0x557c0ba50bb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbeac006010 tx=0x7fbeac034f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41118366454e76d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:44.122 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.119+0000 7fbe96ff5640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:12:44.123 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.120+0000 7fbe96ff5640 1 -- 172.21.15.94:0/3838999943 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac032ae0 con 0x557c0ba4ab70 2024-02-17T12:12:44.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.120+0000 7fbe96ff5640 1 -- 172.21.15.94:0/3838999943 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbeac034440 con 0x557c0ba4ab70 2024-02-17T12:12:44.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.120+0000 7fbebbe76740 1 -- 172.21.15.94:0/3838999943 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c0b6b17e0 con 0x557c0ba4ab70 2024-02-17T12:12:44.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.120+0000 7fbe96ff5640 1 -- 172.21.15.94:0/3838999943 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbeac034600 con 0x557c0ba4ab70 2024-02-17T12:12:44.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.120+0000 7fbebbe76740 10 client.28745 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:44.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.120+0000 7fbebbe76740 1 -- 172.21.15.94:0/3838999943 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557c0b9b06a0 con 0x557c0ba4ab70 2024-02-17T12:12:44.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.121+0000 7fbe96ff5640 1 -- 172.21.15.94:0/3838999943 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x557c0b6b17e0 con 0x557c0ba4ab70 2024-02-17T12:12:44.124 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.122+0000 7fbe96ff5640 1 -- 172.21.15.94:0/3838999943 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbeac004450 con 0x557c0ba4ab70 2024-02-17T12:12:44.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.122+0000 7fbebbe76740 10 client.28745 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:44.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.122+0000 7fbebbe76740 10 client.28745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:44.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.122+0000 7fbebbe76740 20 client.28745 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:44.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.122+0000 7fbebbe76740 1 --2- 172.21.15.94:0/3838999943 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557c0ba63ba0 0x557c0ba65fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.122+0000 7fbebbe76740 4 client.28745 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:44.125 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.122+0000 7fbebbe76740 1 -- 172.21.15.94:0/3838999943 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c0b6ee720 con 0x557c0ba63ba0 2024-02-17T12:12:44.126 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.124+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/3838999943 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557c0ba63ba0 0x557c0ba65fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.128 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.126+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/3838999943 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557c0ba63ba0 0x557c0ba65fc0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe0ad9d4fd020c server_cookie=5378d91af808ef06 in_seq=0 out_seq=0 2024-02-17T12:12:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.126+0000 7fbe96ff5640 10 client.28745 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.127+0000 7fbe96ff5640 1 -- 172.21.15.94:0/3838999943 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c0b6ee720 con 0x557c0ba63ba0 2024-02-17T12:12:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.127+0000 7fbe96ff5640 10 client.28745 handle_command_reply: tid=0 2024-02-17T12:12:44.129 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.127+0000 7fbebbe76740 1 client.28745 shutdown 2024-02-17T12:12:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.129+0000 7fbebbe76740 1 -- 172.21.15.94:0/3838999943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba4ab70 msgr2=0x557c0ba50bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.129+0000 7fbebbe76740 1 --2- 172.21.15.94:0/3838999943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba4ab70 0x557c0ba50bb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbeac006010 tx=0x7fbeac034f00 comp rx=0 tx=0).stop 2024-02-17T12:12:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.129+0000 7fbebbe76740 1 -- 172.21.15.94:0/3838999943 shutdown_connections 2024-02-17T12:12:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.129+0000 7fbebbe76740 1 --2- 172.21.15.94:0/3838999943 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557c0ba63ba0 0x557c0ba65fc0 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.129+0000 7fbebbe76740 1 --2- 172.21.15.94:0/3838999943 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba53750 0x557c0ba510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.129+0000 7fbebbe76740 1 --2- 172.21.15.94:0/3838999943 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba4ab70 0x557c0ba50bb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.129+0000 7fbebbe76740 1 --2- 172.21.15.94:0/3838999943 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4a250 0x557c0ba50670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.129+0000 7fbebbe76740 1 -- 172.21.15.94:0/3838999943 >> 172.21.15.94:0/3838999943 conn(0x557c0ba44980 msgr2=0x557c0ba466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:44.131 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.129+0000 7fbe96ff5640 0 client.28745 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:44.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.129+0000 7fbebbe76740 1 -- 172.21.15.94:0/3838999943 shutdown_connections 2024-02-17T12:12:44.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.129+0000 7fbebbe76740 1 -- 172.21.15.94:0/3838999943 wait complete. 2024-02-17T12:12:44.132 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.129+0000 7fbebbe76740 20 client.28745 trim_cache size 0 max 16384 2024-02-17T12:12:44.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.147+0000 7fbebbe76740 1 Processor -- start 2024-02-17T12:12:44.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.147+0000 7fbebbe76740 1 -- start start 2024-02-17T12:12:44.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.147+0000 7fbebbe76740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba4a250 0x557c0ba52da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.147+0000 7fbebbe76740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4ab70 0x557c0ba50f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.147+0000 7fbebbe76740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba53750 0x557c0ba4d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.147+0000 7fbebbe76740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0b937130 con 0x557c0ba4ab70 2024-02-17T12:12:44.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.147+0000 7fbebbe76740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0ba45780 con 0x557c0ba53750 2024-02-17T12:12:44.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.147+0000 7fbebbe76740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0ba405c0 con 0x557c0ba4a250 2024-02-17T12:12:44.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.147+0000 7fbeb27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4ab70 0x557c0ba50f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.147+0000 7fbeb27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4ab70 0x557c0ba50f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34964/0 (socket says 172.21.15.94:34964) 2024-02-17T12:12:44.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.147+0000 7fbeb27fc640 1 -- 172.21.15.94:0/1176803729 learned_addr learned my addr 172.21.15.94:0/1176803729 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:44.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.147+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/1176803729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba4a250 0x557c0ba52da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.153 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.148+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/1176803729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba53750 0x557c0ba4d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.148+0000 7fbeb27fc640 1 -- 172.21.15.94:0/1176803729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba4a250 msgr2=0x557c0ba52da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:44.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.148+0000 7fbeb27fc640 1 --2- 172.21.15.94:0/1176803729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba4a250 0x557c0ba52da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.148+0000 7fbeb27fc640 1 -- 172.21.15.94:0/1176803729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba53750 msgr2=0x557c0ba4d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:44.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.148+0000 7fbeb27fc640 1 --2- 172.21.15.94:0/1176803729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba53750 0x557c0ba4d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.148+0000 7fbeb27fc640 1 -- 172.21.15.94:0/1176803729 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c0b94cc20 con 0x557c0ba4ab70 2024-02-17T12:12:44.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.148+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/1176803729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba53750 0x557c0ba4d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:12:44.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.148+0000 7fbeb27fc640 1 --2- 172.21.15.94:0/1176803729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4ab70 0x557c0ba50f10 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0326b0 tx=0x7fbeac032dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95333bded874d00c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:44.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.148+0000 7fbe98ff9640 1 -- 172.21.15.94:0/1176803729 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac02c840 con 0x557c0ba4ab70 2024-02-17T12:12:44.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.148+0000 7fbe98ff9640 1 -- 172.21.15.94:0/1176803729 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbeac032ae0 con 0x557c0ba4ab70 2024-02-17T12:12:44.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.148+0000 7fbe98ff9640 1 -- 172.21.15.94:0/1176803729 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac03fae0 con 0x557c0ba4ab70 2024-02-17T12:12:44.154 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.149+0000 7fbebbe76740 1 -- 172.21.15.94:0/1176803729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4ab70 msgr2=0x557c0ba50f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.149+0000 7fbebbe76740 1 --2- 172.21.15.94:0/1176803729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4ab70 0x557c0ba50f10 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0326b0 tx=0x7fbeac032dd0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.149+0000 7fbebbe76740 1 -- 172.21.15.94:0/1176803729 shutdown_connections 2024-02-17T12:12:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.149+0000 7fbebbe76740 1 --2- 172.21.15.94:0/1176803729 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba53750 0x557c0ba4d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.149+0000 7fbebbe76740 1 --2- 172.21.15.94:0/1176803729 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4ab70 0x557c0ba50f10 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.149+0000 7fbebbe76740 1 --2- 172.21.15.94:0/1176803729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba4a250 0x557c0ba52da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.149+0000 7fbebbe76740 1 -- 172.21.15.94:0/1176803729 >> 172.21.15.94:0/1176803729 conn(0x557c0ba44980 msgr2=0x557c0ba4f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.149+0000 7fbebbe76740 1 -- 172.21.15.94:0/1176803729 shutdown_connections 2024-02-17T12:12:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.149+0000 7fbebbe76740 1 -- 172.21.15.94:0/1176803729 wait complete. 2024-02-17T12:12:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.149+0000 7fbebbe76740 1 -- start start 2024-02-17T12:12:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.149+0000 7fbebbe76740 1 Processor -- start 2024-02-17T12:12:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.150+0000 7fbebbe76740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba4a250 0x557c0ba592f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.150+0000 7fbebbe76740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4ab70 0x557c0ba59830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.155 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.150+0000 7fbebbe76740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba53750 0x557c0ba59d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.150+0000 7fbebbe76740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0baba110 con 0x557c0ba4ab70 2024-02-17T12:12:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.150+0000 7fbebbe76740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0baba250 con 0x557c0ba53750 2024-02-17T12:12:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.150+0000 7fbebbe76740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c0baba410 con 0x557c0ba4a250 2024-02-17T12:12:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.150+0000 7fbebbe76740 10 client.0 _ll_register_callbacks cb 0x7fbea40093f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.150+0000 7fbeb27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4ab70 0x557c0ba59830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.150+0000 7fbeb37fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba4a250 0x557c0ba592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.150+0000 7fbeb27fc640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4ab70 0x557c0ba59830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:34966/0 (socket says 172.21.15.94:34966) 2024-02-17T12:12:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.150+0000 7fbeb27fc640 1 -- 172.21.15.94:0/644569204 learned_addr learned my addr 172.21.15.94:0/644569204 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:44.156 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.150+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/644569204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba53750 0x557c0ba59d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.151+0000 7fbeb37fe640 1 -- 172.21.15.94:0/644569204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba53750 msgr2=0x557c0ba59d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.151+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/644569204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba53750 0x557c0ba59d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.151+0000 7fbeb37fe640 1 -- 172.21.15.94:0/644569204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4ab70 msgr2=0x557c0ba59830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:44.159 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.151+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/644569204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4ab70 0x557c0ba59830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.151+0000 7fbeb37fe640 1 -- 172.21.15.94:0/644569204 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c0b9b06a0 con 0x557c0ba4a250 2024-02-17T12:12:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.151+0000 7fbeb37fe640 1 --2- 172.21.15.94:0/644569204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba4a250 0x557c0ba592f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbea4013d90 tx=0x7fbeac032f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4130d57ad5ae7d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.151+0000 7fbe96ff5640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:12:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.152+0000 7fbe96ff5640 1 -- 172.21.15.94:0/644569204 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4021040 con 0x557c0ba4a250 2024-02-17T12:12:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.152+0000 7fbe96ff5640 1 -- 172.21.15.94:0/644569204 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fbea4034070 con 0x557c0ba4a250 2024-02-17T12:12:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.152+0000 7fbebbe76740 1 -- 172.21.15.94:0/644569204 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbea4013470 con 0x557c0ba4a250 2024-02-17T12:12:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.152+0000 7fbe96ff5640 1 -- 172.21.15.94:0/644569204 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbea4012750 con 0x557c0ba4a250 2024-02-17T12:12:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.152+0000 7fbebbe76740 10 client.28717 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.152+0000 7fbebbe76740 1 -- 172.21.15.94:0/644569204 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557c0b9728c0 con 0x557c0ba4a250 2024-02-17T12:12:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.152+0000 7fbe96ff5640 1 -- 172.21.15.94:0/644569204 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbea4012d90 con 0x557c0ba4a250 2024-02-17T12:12:44.160 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.153+0000 7fbe96ff5640 1 -- 172.21.15.94:0/644569204 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fbea4027040 con 0x557c0ba4a250 2024-02-17T12:12:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.153+0000 7fbebbe76740 10 client.28717 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.153+0000 7fbebbe76740 10 client.28717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.153+0000 7fbebbe76740 20 client.28717 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.153+0000 7fbebbe76740 1 --2- 172.21.15.94:0/644569204 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557c0babb9f0 0x557c0babdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.153+0000 7fbebbe76740 4 client.28717 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:12:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.153+0000 7fbebbe76740 1 -- 172.21.15.94:0/644569204 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fbea4013470 con 0x557c0babb9f0 2024-02-17T12:12:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.153+0000 7fbeb27fc640 1 --2- 172.21.15.94:0/644569204 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557c0babb9f0 0x557c0babdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.153+0000 7fbe96ff5640 10 client.28717 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.153+0000 7fbeb27fc640 1 --2- 172.21.15.94:0/644569204 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557c0babb9f0 0x557c0babdde0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7effdc7523c7a4fb server_cookie=461cefe71321dc4a in_seq=0 out_seq=0 2024-02-17T12:12:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.153+0000 7fbe96ff5640 1 -- 172.21.15.94:0/644569204 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fbea4013470 con 0x557c0babb9f0 2024-02-17T12:12:44.161 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.153+0000 7fbe96ff5640 10 client.28717 handle_command_reply: tid=0 2024-02-17T12:12:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.153+0000 7fbebbe76740 1 client.28717 shutdown 2024-02-17T12:12:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.154+0000 7fbebbe76740 1 -- 172.21.15.94:0/644569204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba4a250 msgr2=0x557c0ba592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.154+0000 7fbebbe76740 1 --2- 172.21.15.94:0/644569204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba4a250 0x557c0ba592f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbea4013d90 tx=0x7fbeac032f40 comp rx=0 tx=0).stop 2024-02-17T12:12:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbebbe76740 1 -- 172.21.15.94:0/644569204 shutdown_connections 2024-02-17T12:12:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbebbe76740 1 --2- 172.21.15.94:0/644569204 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x557c0babb9f0 0x557c0babdde0 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbebbe76740 1 --2- 172.21.15.94:0/644569204 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x557c0ba53750 0x557c0ba59d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbe96ff5640 0 client.28717 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbebbe76740 1 --2- 172.21.15.94:0/644569204 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x557c0ba4ab70 0x557c0ba59830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbebbe76740 1 --2- 172.21.15.94:0/644569204 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c0ba4a250 0x557c0ba592f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbebbe76740 1 -- 172.21.15.94:0/644569204 >> 172.21.15.94:0/644569204 conn(0x557c0ba44980 msgr2=0x557c0ba432f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbebbe76740 1 -- 172.21.15.94:0/644569204 shutdown_connections 2024-02-17T12:12:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbebbe76740 1 -- 172.21.15.94:0/644569204 wait complete. 2024-02-17T12:12:44.162 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbebbe76740 20 client.28717 trim_cache size 0 max 16384 2024-02-17T12:12:44.163 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:12:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbeb929d640 1 -- 172.21.15.94:0/3291908520 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbe9c0a2440 msgr2=0x7fbe9c0a4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbeb929d640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbe9c0a2440 0x7fbe9c0a4900 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x557c0ba49d20 tx=0x7fbea8096040 comp rx=0 tx=0).stop 2024-02-17T12:12:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbeb929d640 1 -- 172.21.15.94:0/3291908520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb410d9d0 msgr2=0x7fbeb4119fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.155+0000 7fbeb929d640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb410d9d0 0x7fbeb4119fe0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009fd0 tx=0x7fbea800b870 comp rx=0 tx=0).stop 2024-02-17T12:12:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.156+0000 7fbeb929d640 1 -- 172.21.15.94:0/3291908520 shutdown_connections 2024-02-17T12:12:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.156+0000 7fbeb929d640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fbe9c0a2440 0x7fbe9c0a4900 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.156+0000 7fbeb929d640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbeb411a960 0x7fbeb41145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.156+0000 7fbeb929d640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fbeb410f9a0 0x7fbeb41140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.164 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.156+0000 7fbeb929d640 1 --2- 172.21.15.94:0/3291908520 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fbeb410d9d0 0x7fbeb4119fe0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:44.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.156+0000 7fbeb929d640 1 -- 172.21.15.94:0/3291908520 >> 172.21.15.94:0/3291908520 conn(0x7fbeb406c260 msgr2=0x7fbeb411c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:44.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.156+0000 7fbeb929d640 1 -- 172.21.15.94:0/3291908520 shutdown_connections 2024-02-17T12:12:44.165 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:44.156+0000 7fbeb929d640 1 -- 172.21.15.94:0/3291908520 wait complete. 2024-02-17T12:12:44.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:44 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2618160885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:12:44.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:44 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2618160885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:12:44.460 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:44 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2618160885' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:12:45.173 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:12:45.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:12:45.174 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:12:45.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 865.0 s, 1482.2 tps, lat 67.864 ms stddev 75.652 2024-02-17T12:12:45.305 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.303+0000 7ff006b3e640 1 -- 172.21.15.94:0/2700279649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff000109c30 msgr2=0x7ff0001156c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.303+0000 7ff006b3e640 1 --2- 172.21.15.94:0/2700279649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff000109c30 0x7ff0001156c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7feff8009cd0 tx=0x7feff802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.303+0000 7ff006b3e640 1 -- 172.21.15.94:0/2700279649 shutdown_connections 2024-02-17T12:12:45.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.303+0000 7ff006b3e640 1 --2- 172.21.15.94:0/2700279649 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff000109c30 0x7ff0001156c0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.303+0000 7ff006b3e640 1 --2- 172.21.15.94:0/2700279649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff000109290 0x7ff0001096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.303+0000 7ff006b3e640 1 --2- 172.21.15.94:0/2700279649 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff00010e920 0x7ff00010ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.303+0000 7ff006b3e640 1 -- 172.21.15.94:0/2700279649 >> 172.21.15.94:0/2700279649 conn(0x7ff000072030 msgr2=0x7ff000072440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:45.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.303+0000 7ff006b3e640 1 -- 172.21.15.94:0/2700279649 shutdown_connections 2024-02-17T12:12:45.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.303+0000 7ff006b3e640 1 -- 172.21.15.94:0/2700279649 wait complete. 2024-02-17T12:12:45.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7ff006b3e640 1 Processor -- start 2024-02-17T12:12:45.306 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7ff006b3e640 1 -- start start 2024-02-17T12:12:45.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7ff006b3e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff000109290 0x7ff00010a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7ff006b3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff000109c30 0x7ff00010aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.308 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7ff006b3e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff00010e920 0x7ff00010afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7ff006b3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00010c4b0 con 0x7ff000109c30 2024-02-17T12:12:45.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7ff006b3e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001b2900 con 0x7ff00010e920 2024-02-17T12:12:45.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7ff006b3e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001b2a40 con 0x7ff000109290 2024-02-17T12:12:45.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7feff7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff000109c30 0x7ff00010aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7ff0048da640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff000109290 0x7ff00010a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7ff0048da640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff000109290 0x7ff00010a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60704/0 (socket says 172.21.15.94:60704) 2024-02-17T12:12:45.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7ff0048da640 1 -- 172.21.15.94:0/3528923407 learned_addr learned my addr 172.21.15.94:0/3528923407 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:45.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7feff7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff000109c30 0x7ff00010aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41736/0 (socket says 172.21.15.94:41736) 2024-02-17T12:12:45.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7ff0050db640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff00010e920 0x7ff00010afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.304+0000 7ff0048da640 1 -- 172.21.15.94:0/3528923407 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff00010e920 msgr2=0x7ff00010afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.305+0000 7ff0048da640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff00010e920 0x7ff00010afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.305+0000 7ff0048da640 1 -- 172.21.15.94:0/3528923407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff000109c30 msgr2=0x7ff00010aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.309 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.305+0000 7ff0048da640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff000109c30 0x7ff00010aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.305+0000 7ff0048da640 1 -- 172.21.15.94:0/3528923407 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff8009900 con 0x7ff000109290 2024-02-17T12:12:45.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.305+0000 7ff0050db640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff00010e920 0x7ff00010afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:12:45.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.305+0000 7feff7fff640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff000109c30 0x7ff00010aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:45.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.305+0000 7ff0048da640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff000109290 0x7ff00010a560 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7feffc00d950 tx=0x7feffc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:45.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.305+0000 7feff5ffb640 1 -- 172.21.15.94:0/3528923407 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc009c30 con 0x7ff000109290 2024-02-17T12:12:45.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.305+0000 7ff006b3e640 1 -- 172.21.15.94:0/3528923407 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0001b2d20 con 0x7ff000109290 2024-02-17T12:12:45.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.305+0000 7feff5ffb640 1 -- 172.21.15.94:0/3528923407 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feffc009d90 con 0x7ff000109290 2024-02-17T12:12:45.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.305+0000 7feff5ffb640 1 -- 172.21.15.94:0/3528923407 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc00f7e0 con 0x7ff000109290 2024-02-17T12:12:45.310 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.305+0000 7ff006b3e640 1 -- 172.21.15.94:0/3528923407 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0001b3220 con 0x7ff000109290 2024-02-17T12:12:45.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.307+0000 7feff5ffb640 1 -- 172.21.15.94:0/3528923407 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7feffc010ad0 con 0x7ff000109290 2024-02-17T12:12:45.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.308+0000 7feff5ffb640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fefe80a2230 0x7fefe80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.308+0000 7feff7fff640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fefe80a2230 0x7fefe80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.308+0000 7feff5ffb640 1 -- 172.21.15.94:0/3528923407 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7feffc0cdd30 con 0x7ff000109290 2024-02-17T12:12:45.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.308+0000 7ff0097ee740 1 Processor -- start 2024-02-17T12:12:45.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.308+0000 7feff7fff640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fefe80a2230 0x7fefe80a46f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff00010bd20 tx=0x7feff0005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:45.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.308+0000 7ff0097ee740 1 -- start start 2024-02-17T12:12:45.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0097ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 0x55c11f6f0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0097ee740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0d30 0x55c11f6f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0097ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f98b0 0x55c11f6fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0097ee740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11f65ef90 con 0x55c11f6f98b0 2024-02-17T12:12:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0097ee740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11f65c4c0 con 0x55c11f6f0d30 2024-02-17T12:12:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0097ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11f657ca0 con 0x55c11f6f0380 2024-02-17T12:12:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0048da640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 0x55c11f6f0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0048da640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 0x55c11f6f0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60716/0 (socket says 172.21.15.94:60716) 2024-02-17T12:12:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0048da640 1 -- 172.21.15.94:0/2835664429 learned_addr learned my addr 172.21.15.94:0/2835664429 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0050db640 1 --2- 172.21.15.94:0/2835664429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f98b0 0x55c11f6fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7feff7fff640 1 --2- 172.21.15.94:0/2835664429 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0d30 0x55c11f6f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0048da640 1 -- 172.21.15.94:0/2835664429 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0d30 msgr2=0x55c11f6f92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0048da640 1 --2- 172.21.15.94:0/2835664429 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0d30 0x55c11f6f92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0048da640 1 -- 172.21.15.94:0/2835664429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f98b0 msgr2=0x55c11f6fbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0048da640 1 --2- 172.21.15.94:0/2835664429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f98b0 0x55c11f6fbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0048da640 1 -- 172.21.15.94:0/2835664429 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c11f61a0b0 con 0x55c11f6f0380 2024-02-17T12:12:45.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0050db640 1 --2- 172.21.15.94:0/2835664429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f98b0 0x55c11f6fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:45.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.309+0000 7ff0048da640 1 --2- 172.21.15.94:0/2835664429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 0x55c11f6f0760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7feffc0a2950 tx=0x7feffc096ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4122e960b18520ce server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:45.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.311+0000 7fefe6ffd640 1 -- 172.21.15.94:0/2835664429 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc09a6e0 con 0x55c11f6f0380 2024-02-17T12:12:45.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.311+0000 7fefe6ffd640 1 -- 172.21.15.94:0/2835664429 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feffc09e830 con 0x55c11f6f0380 2024-02-17T12:12:45.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.311+0000 7fefe6ffd640 1 -- 172.21.15.94:0/2835664429 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc096f20 con 0x55c11f6f0380 2024-02-17T12:12:45.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.312+0000 7ff0097ee740 1 -- 172.21.15.94:0/2835664429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 msgr2=0x55c11f6f0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.312+0000 7ff0097ee740 1 --2- 172.21.15.94:0/2835664429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 0x55c11f6f0760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7feffc0a2950 tx=0x7feffc096ab0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.313+0000 7ff0097ee740 1 -- 172.21.15.94:0/2835664429 shutdown_connections 2024-02-17T12:12:45.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.313+0000 7ff0097ee740 1 --2- 172.21.15.94:0/2835664429 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f98b0 0x55c11f6fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.313+0000 7ff0097ee740 1 --2- 172.21.15.94:0/2835664429 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0d30 0x55c11f6f92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.313+0000 7ff0097ee740 1 --2- 172.21.15.94:0/2835664429 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 0x55c11f6f0760 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.313+0000 7ff0097ee740 1 -- 172.21.15.94:0/2835664429 >> 172.21.15.94:0/2835664429 conn(0x55c11f6ea840 msgr2=0x55c11f6eac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:45.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.313+0000 7ff0097ee740 1 -- 172.21.15.94:0/2835664429 shutdown_connections 2024-02-17T12:12:45.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.313+0000 7ff0097ee740 1 -- 172.21.15.94:0/2835664429 wait complete. 2024-02-17T12:12:45.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.313+0000 7ff0097ee740 1 -- start start 2024-02-17T12:12:45.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.313+0000 7ff0097ee740 1 Processor -- start 2024-02-17T12:12:45.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0097ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 0x55c11f6f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0097ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 0x55c11f6f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0097ee740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f98b0 0x55c11f6f7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0097ee740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11f5fcf40 con 0x55c11f6f0d30 2024-02-17T12:12:45.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0097ee740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11f32b4b0 con 0x55c11f6f98b0 2024-02-17T12:12:45.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0097ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11f5dd670 con 0x55c11f6f0380 2024-02-17T12:12:45.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0048da640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 0x55c11f6f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7feff7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 0x55c11f6f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0048da640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 0x55c11f6f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60718/0 (socket says 172.21.15.94:60718) 2024-02-17T12:12:45.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0048da640 1 -- 172.21.15.94:0/1002224918 learned_addr learned my addr 172.21.15.94:0/1002224918 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:45.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0097ee740 10 client.0 _ll_register_callbacks cb 0x55c11f64af70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:45.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0050db640 1 --2- 172.21.15.94:0/1002224918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f98b0 0x55c11f6f7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0048da640 1 -- 172.21.15.94:0/1002224918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f98b0 msgr2=0x55c11f6f7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0048da640 1 --2- 172.21.15.94:0/1002224918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f98b0 0x55c11f6f7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0048da640 1 -- 172.21.15.94:0/1002224918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 msgr2=0x55c11f6f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.319 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0048da640 1 --2- 172.21.15.94:0/1002224918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 0x55c11f6f6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0048da640 1 -- 172.21.15.94:0/1002224918 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0001b3220 con 0x55c11f6f0380 2024-02-17T12:12:45.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7feff7fff640 1 --2- 172.21.15.94:0/1002224918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 0x55c11f6f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:45.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7ff0048da640 1 --2- 172.21.15.94:0/1002224918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 0x55c11f6f68a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7feffc0970b0 tx=0x7feffc0cb8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a9655f8bd73b250 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:45.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7fefdffff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:12:45.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.314+0000 7fefdffff640 1 -- 172.21.15.94:0/1002224918 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc09e830 con 0x55c11f6f0380 2024-02-17T12:12:45.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.315+0000 7fefdffff640 1 -- 172.21.15.94:0/1002224918 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7feffc096ec0 con 0x55c11f6f0380 2024-02-17T12:12:45.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.315+0000 7ff0097ee740 1 -- 172.21.15.94:0/1002224918 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c11f631f40 con 0x55c11f6f0380 2024-02-17T12:12:45.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.315+0000 7fefdffff640 1 -- 172.21.15.94:0/1002224918 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feffc0a2e80 con 0x55c11f6f0380 2024-02-17T12:12:45.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.315+0000 7ff0097ee740 10 client.28729 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:45.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.315+0000 7ff0097ee740 1 -- 172.21.15.94:0/1002224918 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c11f656320 con 0x55c11f6f0380 2024-02-17T12:12:45.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.315+0000 7fefdffff640 1 -- 172.21.15.94:0/1002224918 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c11f631f40 con 0x55c11f6f0380 2024-02-17T12:12:45.320 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.315+0000 7fefdffff640 1 -- 172.21.15.94:0/1002224918 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feffc099b30 con 0x55c11f6f0380 2024-02-17T12:12:45.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.315+0000 7ff0097ee740 10 client.28729 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:45.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.315+0000 7ff0097ee740 10 client.28729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:45.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.315+0000 7ff0097ee740 20 client.28729 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:45.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.315+0000 7ff0097ee740 1 --2- 172.21.15.94:0/1002224918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c11f7098d0 0x55c11f70bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.315+0000 7ff0097ee740 4 client.28729 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:45.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.315+0000 7ff0097ee740 1 -- 172.21.15.94:0/1002224918 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c11f64e310 con 0x55c11f7098d0 2024-02-17T12:12:45.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.316+0000 7feff7fff640 1 --2- 172.21.15.94:0/1002224918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c11f7098d0 0x55c11f70bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.316+0000 7fefdffff640 10 client.28729 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:45.321 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.316+0000 7feff7fff640 1 --2- 172.21.15.94:0/1002224918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c11f7098d0 0x55c11f70bcc0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=184d6986a360f09b server_cookie=bf24fe58aedbe1fc in_seq=0 out_seq=0 2024-02-17T12:12:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.322+0000 7fefdffff640 1 -- 172.21.15.94:0/1002224918 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c11f64e310 con 0x55c11f7098d0 2024-02-17T12:12:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.322+0000 7fefdffff640 10 client.28729 handle_command_reply: tid=0 2024-02-17T12:12:45.324 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.323+0000 7ff0097ee740 1 client.28729 shutdown 2024-02-17T12:12:45.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.329+0000 7ff0097ee740 1 -- 172.21.15.94:0/1002224918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 msgr2=0x55c11f6f68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.329+0000 7ff0097ee740 1 --2- 172.21.15.94:0/1002224918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 0x55c11f6f68a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7feffc0970b0 tx=0x7feffc0cb8f0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.333+0000 7ff0097ee740 1 -- 172.21.15.94:0/1002224918 shutdown_connections 2024-02-17T12:12:45.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.333+0000 7ff0097ee740 1 --2- 172.21.15.94:0/1002224918 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c11f7098d0 0x55c11f70bcc0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.333+0000 7ff0097ee740 1 --2- 172.21.15.94:0/1002224918 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f98b0 0x55c11f6f7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.333+0000 7ff0097ee740 1 --2- 172.21.15.94:0/1002224918 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 0x55c11f6f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.333+0000 7ff0097ee740 1 --2- 172.21.15.94:0/1002224918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f0380 0x55c11f6f68a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.333+0000 7ff0097ee740 1 -- 172.21.15.94:0/1002224918 >> 172.21.15.94:0/1002224918 conn(0x55c11f6ea840 msgr2=0x55c11f6ec7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:45.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.335+0000 7fefdffff640 0 client.28729 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:45.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.341+0000 7ff0097ee740 1 -- 172.21.15.94:0/1002224918 shutdown_connections 2024-02-17T12:12:45.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.347+0000 7ff0097ee740 1 -- 172.21.15.94:0/1002224918 wait complete. 2024-02-17T12:12:45.351 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.348+0000 7ff0097ee740 20 client.28729 trim_cache size 0 max 16384 2024-02-17T12:12:45.380 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.377+0000 7ff0097ee740 1 Processor -- start 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.377+0000 7ff0097ee740 1 -- start start 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.377+0000 7ff0097ee740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0380 0x55c11f6f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.377+0000 7ff0097ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 0x55c11f6f7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.377+0000 7ff0097ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f98b0 0x55c11f6f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.377+0000 7ff0048da640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0380 0x55c11f6f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.377+0000 7ff0048da640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0380 0x55c11f6f8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42552/0 (socket says 172.21.15.94:42552) 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.377+0000 7ff0048da640 1 -- 172.21.15.94:0/690792740 learned_addr learned my addr 172.21.15.94:0/690792740 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.377+0000 7ff0097ee740 1 -- 172.21.15.94:0/690792740 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11f6df350 con 0x55c11f6f0d30 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.377+0000 7ff0097ee740 1 -- 172.21.15.94:0/690792740 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11f6eb290 con 0x55c11f6f0380 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.378+0000 7ff0097ee740 1 -- 172.21.15.94:0/690792740 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11f64e310 con 0x55c11f6f98b0 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.378+0000 7ff0050db640 1 --2- 172.21.15.94:0/690792740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f98b0 0x55c11f6f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.378+0000 7ff0048da640 1 -- 172.21.15.94:0/690792740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f98b0 msgr2=0x55c11f6f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.378+0000 7ff0048da640 1 --2- 172.21.15.94:0/690792740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f98b0 0x55c11f6f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.381 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.378+0000 7ff0048da640 1 -- 172.21.15.94:0/690792740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 msgr2=0x55c11f6f7140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:12:45.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.378+0000 7ff0048da640 1 --2- 172.21.15.94:0/690792740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 0x55c11f6f7140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.378+0000 7ff0048da640 1 -- 172.21.15.94:0/690792740 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c11f5f2c00 con 0x55c11f6f0380 2024-02-17T12:12:45.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.378+0000 7ff0048da640 1 --2- 172.21.15.94:0/690792740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0380 0x55c11f6f8f60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7feffc00dfe0 tx=0x7feffc0cb960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1d5cc8a2f7b77ad server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:45.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.378+0000 7fefe67fc640 1 -- 172.21.15.94:0/690792740 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc09e830 con 0x55c11f6f0380 2024-02-17T12:12:45.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.379+0000 7fefe67fc640 1 -- 172.21.15.94:0/690792740 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feffc0b3200 con 0x55c11f6f0380 2024-02-17T12:12:45.382 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.379+0000 7fefe67fc640 1 -- 172.21.15.94:0/690792740 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc096d50 con 0x55c11f6f0380 2024-02-17T12:12:45.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.379+0000 7ff0097ee740 1 -- 172.21.15.94:0/690792740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0380 msgr2=0x55c11f6f8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.379+0000 7ff0097ee740 1 --2- 172.21.15.94:0/690792740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0380 0x55c11f6f8f60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7feffc00dfe0 tx=0x7feffc0cb960 comp rx=0 tx=0).stop 2024-02-17T12:12:45.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.379+0000 7ff0097ee740 1 -- 172.21.15.94:0/690792740 shutdown_connections 2024-02-17T12:12:45.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.379+0000 7ff0097ee740 1 --2- 172.21.15.94:0/690792740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f98b0 0x55c11f6f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.379+0000 7ff0097ee740 1 --2- 172.21.15.94:0/690792740 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 0x55c11f6f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.379+0000 7ff0097ee740 1 --2- 172.21.15.94:0/690792740 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0380 0x55c11f6f8f60 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.379+0000 7ff0097ee740 1 -- 172.21.15.94:0/690792740 >> 172.21.15.94:0/690792740 conn(0x55c11f6ea840 msgr2=0x55c11f6f1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:45.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.380+0000 7ff0097ee740 1 -- 172.21.15.94:0/690792740 shutdown_connections 2024-02-17T12:12:45.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.380+0000 7ff0097ee740 1 -- 172.21.15.94:0/690792740 wait complete. 2024-02-17T12:12:45.385 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.380+0000 7ff0097ee740 1 -- start start 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.380+0000 7ff0097ee740 1 Processor -- start 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.380+0000 7ff0097ee740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0380 0x55c11f6ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.380+0000 7ff0097ee740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 0x55c11f6fff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.380+0000 7ff0097ee740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f98b0 0x55c11f75fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.380+0000 7ff0097ee740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11f760250 con 0x55c11f6f0d30 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.380+0000 7ff0097ee740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11f7603c0 con 0x55c11f6f0380 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.380+0000 7feff7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 0x55c11f6fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7ff0097ee740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c11f760580 con 0x55c11f6f98b0 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7feff7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 0x55c11f6fff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41760/0 (socket says 172.21.15.94:41760) 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7feff7fff640 1 -- 172.21.15.94:0/2886342342 learned_addr learned my addr 172.21.15.94:0/2886342342 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7ff0048da640 1 --2- 172.21.15.94:0/2886342342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0380 0x55c11f6ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7ff0050db640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f98b0 0x55c11f75fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7ff0097ee740 10 client.0 _ll_register_callbacks cb 0x7feffc009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7ff0050db640 1 -- 172.21.15.94:0/2886342342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0380 msgr2=0x55c11f6ffa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7ff0050db640 1 --2- 172.21.15.94:0/2886342342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0380 0x55c11f6ffa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.386 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7ff0050db640 1 -- 172.21.15.94:0/2886342342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 msgr2=0x55c11f6fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7ff0050db640 1 --2- 172.21.15.94:0/2886342342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 0x55c11f6fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7ff0050db640 1 -- 172.21.15.94:0/2886342342 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c11f656320 con 0x55c11f6f98b0 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7ff0050db640 1 --2- 172.21.15.94:0/2886342342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f98b0 0x55c11f75fd10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7feff802c630 tx=0x7feff80346e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6316ee4c609373da server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7fefdffff640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.381+0000 7fefdffff640 1 -- 172.21.15.94:0/2886342342 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8044070 con 0x55c11f6f98b0 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.382+0000 7fefdffff640 1 -- 172.21.15.94:0/2886342342 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7feff80326b0 con 0x55c11f6f98b0 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.382+0000 7ff0097ee740 1 -- 172.21.15.94:0/2886342342 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c11f67d900 con 0x55c11f6f98b0 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.382+0000 7fefdffff640 1 -- 172.21.15.94:0/2886342342 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7feff8032870 con 0x55c11f6f98b0 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.382+0000 7ff0097ee740 10 client.28735 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.382+0000 7ff0097ee740 1 -- 172.21.15.94:0/2886342342 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c11f618910 con 0x55c11f6f98b0 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.382+0000 7fefdffff640 1 -- 172.21.15.94:0/2886342342 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55c11f67d900 con 0x55c11f6f98b0 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.382+0000 7fefdffff640 1 -- 172.21.15.94:0/2886342342 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7feff80043e0 con 0x55c11f6f98b0 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.382+0000 7ff0097ee740 10 client.28735 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.382+0000 7ff0097ee740 10 client.28735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:45.387 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.382+0000 7ff0097ee740 20 client.28735 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:45.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.382+0000 7ff0097ee740 1 --2- 172.21.15.94:0/2886342342 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c11f761db0 0x55c11f7641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.382+0000 7ff0097ee740 4 client.28735 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:12:45.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.382+0000 7ff0097ee740 1 -- 172.21.15.94:0/2886342342 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c11f357860 con 0x55c11f761db0 2024-02-17T12:12:45.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.383+0000 7ff0048da640 1 --2- 172.21.15.94:0/2886342342 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c11f761db0 0x55c11f7641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.383+0000 7fefdffff640 10 client.28735 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:45.388 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.383+0000 7ff0048da640 1 --2- 172.21.15.94:0/2886342342 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c11f761db0 0x55c11f7641d0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88e3b9ac99d94952 server_cookie=a62b20981a27eb93 in_seq=0 out_seq=0 2024-02-17T12:12:45.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7fefdffff640 1 -- 172.21.15.94:0/2886342342 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c11f357860 con 0x55c11f761db0 2024-02-17T12:12:45.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7fefdffff640 10 client.28735 handle_command_reply: tid=0 2024-02-17T12:12:45.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7ff0097ee740 1 client.28735 shutdown 2024-02-17T12:12:45.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7ff0097ee740 1 -- 172.21.15.94:0/2886342342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f98b0 msgr2=0x55c11f75fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7ff0097ee740 1 --2- 172.21.15.94:0/2886342342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f98b0 0x55c11f75fd10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7feff802c630 tx=0x7feff80346e0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7ff0097ee740 1 -- 172.21.15.94:0/2886342342 shutdown_connections 2024-02-17T12:12:45.389 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7ff0097ee740 1 --2- 172.21.15.94:0/2886342342 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55c11f761db0 0x55c11f7641d0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7fefdffff640 0 client.28735 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:45.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7ff0097ee740 1 --2- 172.21.15.94:0/2886342342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c11f6f98b0 0x55c11f75fd10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7ff0097ee740 1 --2- 172.21.15.94:0/2886342342 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55c11f6f0d30 0x55c11f6fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7ff0097ee740 1 --2- 172.21.15.94:0/2886342342 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55c11f6f0380 0x55c11f6ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7ff0097ee740 1 -- 172.21.15.94:0/2886342342 >> 172.21.15.94:0/2886342342 conn(0x55c11f6ea840 msgr2=0x55c11f6ebe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:45.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7ff0097ee740 1 -- 172.21.15.94:0/2886342342 shutdown_connections 2024-02-17T12:12:45.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7ff0097ee740 1 -- 172.21.15.94:0/2886342342 wait complete. 2024-02-17T12:12:45.390 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.384+0000 7ff0097ee740 20 client.28735 trim_cache size 0 max 16384 2024-02-17T12:12:45.390 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:12:45.390 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:12:45.390 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "f66d1b62-3131-45f7-a6e5-43d519d671e8", 2024-02-17T12:12:45.391 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:12:45.391 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:12:45.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.385+0000 7ff006b3e640 1 -- 172.21.15.94:0/3528923407 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fefe80a2230 msgr2=0x7fefe80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.385+0000 7ff006b3e640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fefe80a2230 0x7fefe80a46f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff00010bd20 tx=0x7feff0005f90 comp rx=0 tx=0).stop 2024-02-17T12:12:45.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.385+0000 7ff006b3e640 1 -- 172.21.15.94:0/3528923407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff000109290 msgr2=0x7ff00010a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.391 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.385+0000 7ff006b3e640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff000109290 0x7ff00010a560 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7feffc00d950 tx=0x7feffc00de10 comp rx=0 tx=0).stop 2024-02-17T12:12:45.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.385+0000 7ff006b3e640 1 -- 172.21.15.94:0/3528923407 shutdown_connections 2024-02-17T12:12:45.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.385+0000 7ff006b3e640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fefe80a2230 0x7fefe80a46f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.385+0000 7ff006b3e640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff00010e920 0x7ff00010afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.385+0000 7ff006b3e640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff000109c30 0x7ff00010aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.385+0000 7ff006b3e640 1 --2- 172.21.15.94:0/3528923407 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff000109290 0x7ff00010a560 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.385+0000 7ff006b3e640 1 -- 172.21.15.94:0/3528923407 >> 172.21.15.94:0/3528923407 conn(0x7ff000072030 msgr2=0x7ff000108350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:45.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.385+0000 7ff006b3e640 1 -- 172.21.15.94:0/3528923407 shutdown_connections 2024-02-17T12:12:45.392 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.385+0000 7ff006b3e640 1 -- 172.21.15.94:0/3528923407 wait complete. 2024-02-17T12:12:45.400 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:45 smithi094 ceph-mon[32499]: pgmap v458: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 781 op/s 2024-02-17T12:12:45.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:12:45.402 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:12:45.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:45 smithi189 ceph-mon[36588]: pgmap v458: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 781 op/s 2024-02-17T12:12:45.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:45 smithi125 ceph-mon[36819]: pgmap v458: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 781 op/s 2024-02-17T12:12:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.569+0000 7fb48f93a640 1 -- 172.21.15.94:0/1354873311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb488106430 msgr2=0x7fb48810e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.572 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.569+0000 7fb48f93a640 1 --2- 172.21.15.94:0/1354873311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb488106430 0x7fb48810e000 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb480009c10 tx=0x7fb48002bf30 comp rx=0 tx=0).stop 2024-02-17T12:12:45.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.572+0000 7fb48f93a640 1 -- 172.21.15.94:0/1354873311 shutdown_connections 2024-02-17T12:12:45.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.572+0000 7fb48f93a640 1 --2- 172.21.15.94:0/1354873311 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb488106430 0x7fb48810e000 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.572+0000 7fb48f93a640 1 --2- 172.21.15.94:0/1354873311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb488101d10 0x7fb488105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.572+0000 7fb48f93a640 1 --2- 172.21.15.94:0/1354873311 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb488101360 0x7fb488101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.575 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.572+0000 7fb48f93a640 1 -- 172.21.15.94:0/1354873311 >> 172.21.15.94:0/1354873311 conn(0x7fb4880fcc20 msgr2=0x7fb4880ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:45.578 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.575+0000 7fb48f93a640 1 -- 172.21.15.94:0/1354873311 shutdown_connections 2024-02-17T12:12:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.579+0000 7fb48f93a640 1 -- 172.21.15.94:0/1354873311 wait complete. 2024-02-17T12:12:45.581 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.579+0000 7fb48f93a640 1 Processor -- start 2024-02-17T12:12:45.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.582+0000 7fb48f93a640 1 -- start start 2024-02-17T12:12:45.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.582+0000 7fb48f93a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb488101360 0x7fb4880727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.582+0000 7fb48f93a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb488106430 0x7fb488072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.582+0000 7fb48f93a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb48806b830 0x7fb48806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.582+0000 7fb48f93a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48806c470 con 0x7fb488106430 2024-02-17T12:12:45.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.582+0000 7fb48f93a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48806c5e0 con 0x7fb488101360 2024-02-17T12:12:45.586 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.582+0000 7fb48f93a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4880702d0 con 0x7fb48806b830 2024-02-17T12:12:45.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.587+0000 7fb48ded7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb48806b830 0x7fb48806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.587+0000 7fb48ded7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb48806b830 0x7fb48806bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60748/0 (socket says 172.21.15.94:60748) 2024-02-17T12:12:45.589 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.587+0000 7fb48ded7640 1 -- 172.21.15.94:0/2390432656 learned_addr learned my addr 172.21.15.94:0/2390432656 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:45.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.587+0000 7fb48d6d6640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb488101360 0x7fb4880727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.588+0000 7fb48ced5640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb488106430 0x7fb488072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.591 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.589+0000 7fb48ded7640 1 -- 172.21.15.94:0/2390432656 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb488101360 msgr2=0x7fb4880727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.590+0000 7fb48ded7640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb488101360 0x7fb4880727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.590+0000 7fb48ded7640 1 -- 172.21.15.94:0/2390432656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb488106430 msgr2=0x7fb488072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.590+0000 7fb48ded7640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb488106430 0x7fb488072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.592 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.590+0000 7fb48ded7640 1 -- 172.21.15.94:0/2390432656 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb480009840 con 0x7fb48806b830 2024-02-17T12:12:45.593 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.591+0000 7fb48ded7640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb48806b830 0x7fb48806bce0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb480036040 tx=0x7fb480034700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:45.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.591+0000 7fb48d6d6640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb488101360 0x7fb4880727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:45.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.596+0000 7fb47a7fc640 1 -- 172.21.15.94:0/2390432656 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48002cd80 con 0x7fb48806b830 2024-02-17T12:12:45.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.596+0000 7fb47a7fc640 1 -- 172.21.15.94:0/2390432656 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb480034b00 con 0x7fb48806b830 2024-02-17T12:12:45.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.596+0000 7fb47a7fc640 1 -- 172.21.15.94:0/2390432656 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48003f660 con 0x7fb48806b830 2024-02-17T12:12:45.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.597+0000 7fb48f93a640 1 -- 172.21.15.94:0/2390432656 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb488070550 con 0x7fb48806b830 2024-02-17T12:12:45.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.597+0000 7fb48f93a640 1 -- 172.21.15.94:0/2390432656 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb488070a10 con 0x7fb48806b830 2024-02-17T12:12:45.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.601+0000 7fb4925e7740 1 Processor -- start 2024-02-17T12:12:45.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.601+0000 7fb4925e7740 1 -- start start 2024-02-17T12:12:45.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.601+0000 7fb4925e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d923402b0 0x558d92340690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.603 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.601+0000 7fb4925e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d92340bd0 0x558d92349270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.601+0000 7fb4925e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923497b0 0x558d9234bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.601+0000 7fb4925e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d922af280 con 0x558d92340bd0 2024-02-17T12:12:45.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.601+0000 7fb4925e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d922ac7b0 con 0x558d923402b0 2024-02-17T12:12:45.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.601+0000 7fb4925e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d922a7f90 con 0x558d923497b0 2024-02-17T12:12:45.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.602+0000 7fb48ced5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d923402b0 0x558d92340690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.602+0000 7fb48ced5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d923402b0 0x558d92340690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42592/0 (socket says 172.21.15.94:42592) 2024-02-17T12:12:45.604 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.602+0000 7fb48ced5640 1 -- 172.21.15.94:0/2046024345 learned_addr learned my addr 172.21.15.94:0/2046024345 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:45.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.602+0000 7fb48ced5640 1 --2- 172.21.15.94:0/2046024345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923497b0 0x558d9234bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.602+0000 7fb48ced5640 1 -- 172.21.15.94:0/2046024345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923497b0 msgr2=0x558d9234bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.602+0000 7fb48ced5640 1 --2- 172.21.15.94:0/2046024345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923497b0 0x558d9234bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.602+0000 7fb48ced5640 1 -- 172.21.15.94:0/2046024345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d92340bd0 msgr2=0x558d92349270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:12:45.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.602+0000 7fb48ced5640 1 --2- 172.21.15.94:0/2046024345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d92340bd0 0x558d92349270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.602+0000 7fb48ced5640 1 -- 172.21.15.94:0/2046024345 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d92266570 con 0x558d923402b0 2024-02-17T12:12:45.605 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.603+0000 7fb48ced5640 1 --2- 172.21.15.94:0/2046024345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d923402b0 0x558d92340690 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb47c014960 tx=0x7fb47c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f15a86c11b128a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:45.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.605+0000 7fb4737fe640 1 -- 172.21.15.94:0/2046024345 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c0142d0 con 0x558d923402b0 2024-02-17T12:12:45.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.605+0000 7fb4737fe640 1 -- 172.21.15.94:0/2046024345 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb47c01a020 con 0x558d923402b0 2024-02-17T12:12:45.607 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.605+0000 7fb4737fe640 1 -- 172.21.15.94:0/2046024345 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c012a40 con 0x558d923402b0 2024-02-17T12:12:45.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.606+0000 7fb4925e7740 1 -- 172.21.15.94:0/2046024345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d923402b0 msgr2=0x558d92340690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.606+0000 7fb4925e7740 1 --2- 172.21.15.94:0/2046024345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d923402b0 0x558d92340690 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb47c014960 tx=0x7fb47c00c490 comp rx=0 tx=0).stop 2024-02-17T12:12:45.608 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.606+0000 7fb47a7fc640 1 -- 172.21.15.94:0/2390432656 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb480033510 con 0x7fb48806b830 2024-02-17T12:12:45.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.607+0000 7fb47a7fc640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4740a2430 0x7fb4740a48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.607+0000 7fb47a7fc640 1 -- 172.21.15.94:0/2390432656 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb4800f0670 con 0x7fb48806b830 2024-02-17T12:12:45.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.608+0000 7fb48ced5640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4740a2430 0x7fb4740a48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.610 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.609+0000 7fb48ced5640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4740a2430 0x7fb4740a48f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0058c0 tx=0x7fb47c005850 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.609+0000 7fb4925e7740 1 -- 172.21.15.94:0/2046024345 shutdown_connections 2024-02-17T12:12:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.609+0000 7fb4925e7740 1 --2- 172.21.15.94:0/2046024345 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923497b0 0x558d9234bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.609+0000 7fb4925e7740 1 --2- 172.21.15.94:0/2046024345 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d92340bd0 0x558d92349270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.609+0000 7fb4925e7740 1 --2- 172.21.15.94:0/2046024345 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d923402b0 0x558d92340690 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.611 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.609+0000 7fb4925e7740 1 -- 172.21.15.94:0/2046024345 >> 172.21.15.94:0/2046024345 conn(0x558d9233a990 msgr2=0x558d9233ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:45.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.610+0000 7fb4925e7740 1 -- 172.21.15.94:0/2046024345 shutdown_connections 2024-02-17T12:12:45.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.611+0000 7fb4925e7740 1 -- 172.21.15.94:0/2046024345 wait complete. 2024-02-17T12:12:45.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.611+0000 7fb4925e7740 1 -- start start 2024-02-17T12:12:45.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.611+0000 7fb4925e7740 1 Processor -- start 2024-02-17T12:12:45.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.612+0000 7fb4925e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923402b0 0x558d92349250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.612+0000 7fb4925e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d92340bd0 0x558d923465e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.612+0000 7fb4925e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923497b0 0x558d92346cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.612+0000 7fb4925e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d91f7b430 con 0x558d923402b0 2024-02-17T12:12:45.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.612+0000 7fb4925e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d91f37e00 con 0x558d92340bd0 2024-02-17T12:12:45.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.612+0000 7fb4925e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d9222cfd0 con 0x558d923497b0 2024-02-17T12:12:45.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.612+0000 7fb4925e7740 10 client.0 _ll_register_callbacks cb 0x558d92282fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:45.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.612+0000 7fb48ded7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923497b0 0x558d92346cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.612+0000 7fb48ded7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923497b0 0x558d92346cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60760/0 (socket says 172.21.15.94:60760) 2024-02-17T12:12:45.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.612+0000 7fb48ded7640 1 -- 172.21.15.94:0/190110260 learned_addr learned my addr 172.21.15.94:0/190110260 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:45.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.613+0000 7fb48ced5640 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d92340bd0 0x558d923465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.613+0000 7fb48d6d6640 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923402b0 0x558d92349250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.613+0000 7fb48ced5640 1 -- 172.21.15.94:0/190110260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923497b0 msgr2=0x558d92346cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.613+0000 7fb48ced5640 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923497b0 0x558d92346cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.613+0000 7fb48ced5640 1 -- 172.21.15.94:0/190110260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923402b0 msgr2=0x558d92349250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.616 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.613+0000 7fb48ced5640 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923402b0 0x558d92349250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.613+0000 7fb48ced5640 1 -- 172.21.15.94:0/190110260 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47c014640 con 0x558d92340bd0 2024-02-17T12:12:45.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.613+0000 7fb48d6d6640 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923402b0 0x558d92349250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:12:45.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.613+0000 7fb48ced5640 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d92340bd0 0x558d923465e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0125c0 tx=0x7fb47c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfaf91133fffbbf3 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:45.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.613+0000 7fb470ff9640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:12:45.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.614+0000 7fb470ff9640 1 -- 172.21.15.94:0/190110260 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c012ea0 con 0x558d92340bd0 2024-02-17T12:12:45.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.614+0000 7fb470ff9640 1 -- 172.21.15.94:0/190110260 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb47c01b030 con 0x558d92340bd0 2024-02-17T12:12:45.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.614+0000 7fb4925e7740 1 -- 172.21.15.94:0/190110260 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d91fa77e0 con 0x558d92340bd0 2024-02-17T12:12:45.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.614+0000 7fb470ff9640 1 -- 172.21.15.94:0/190110260 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb47c026070 con 0x558d92340bd0 2024-02-17T12:12:45.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.614+0000 7fb4925e7740 10 client.28775 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:45.617 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.614+0000 7fb4925e7740 1 -- 172.21.15.94:0/190110260 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558d922a6610 con 0x558d92340bd0 2024-02-17T12:12:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.614+0000 7fb470ff9640 1 -- 172.21.15.94:0/190110260 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x558d91fa77e0 con 0x558d92340bd0 2024-02-17T12:12:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.615+0000 7fb470ff9640 1 -- 172.21.15.94:0/190110260 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb47c0140d0 con 0x558d92340bd0 2024-02-17T12:12:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.615+0000 7fb4925e7740 10 client.28775 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.615+0000 7fb4925e7740 10 client.28775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.615+0000 7fb4925e7740 20 client.28775 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:45.619 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.615+0000 7fb4925e7740 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558d92359880 0x558d9235bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.615+0000 7fb4925e7740 4 client.28775 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.615+0000 7fb4925e7740 1 -- 172.21.15.94:0/190110260 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d91fe4720 con 0x558d92359880 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.615+0000 7fb48d6d6640 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558d92359880 0x558d9235bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.615+0000 7fb48d6d6640 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558d92359880 0x558d9235bcc0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aafb5f3a0e6d18f server_cookie=218a3ebd70117f55 in_seq=0 out_seq=0 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.615+0000 7fb470ff9640 10 client.28775 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb470ff9640 1 -- 172.21.15.94:0/190110260 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558d91fe4720 con 0x558d92359880 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb470ff9640 10 client.28775 handle_command_reply: tid=0 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb4925e7740 1 client.28775 shutdown 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb4925e7740 1 -- 172.21.15.94:0/190110260 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d92340bd0 msgr2=0x558d923465e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb4925e7740 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d92340bd0 0x558d923465e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0125c0 tx=0x7fb47c00c490 comp rx=0 tx=0).stop 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb4925e7740 1 -- 172.21.15.94:0/190110260 shutdown_connections 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb4925e7740 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558d92359880 0x558d9235bcc0 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb4925e7740 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923497b0 0x558d92346cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.620 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb4925e7740 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d92340bd0 0x558d923465e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb4925e7740 1 --2- 172.21.15.94:0/190110260 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923402b0 0x558d92349250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb4925e7740 1 -- 172.21.15.94:0/190110260 >> 172.21.15.94:0/190110260 conn(0x558d9233a990 msgr2=0x558d9233c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:45.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb470ff9640 0 client.28775 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:45.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb4925e7740 1 -- 172.21.15.94:0/190110260 shutdown_connections 2024-02-17T12:12:45.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb4925e7740 1 -- 172.21.15.94:0/190110260 wait complete. 2024-02-17T12:12:45.621 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.616+0000 7fb4925e7740 20 client.28775 trim_cache size 0 max 16384 2024-02-17T12:12:45.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.635+0000 7fb4925e7740 1 Processor -- start 2024-02-17T12:12:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.635+0000 7fb4925e7740 1 -- start start 2024-02-17T12:12:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.636+0000 7fb4925e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923402b0 0x558d92348d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.636+0000 7fb4925e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d92340bd0 0x558d92349250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.636+0000 7fb4925e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923497b0 0x558d92350690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.636+0000 7fb4925e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d9233f3a0 con 0x558d923497b0 2024-02-17T12:12:45.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.636+0000 7fb4925e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d9233f580 con 0x558d92340bd0 2024-02-17T12:12:45.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.636+0000 7fb4925e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d92359680 con 0x558d923402b0 2024-02-17T12:12:45.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.636+0000 7fb48ded7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923497b0 0x558d92350690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.636+0000 7fb48ded7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923497b0 0x558d92350690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41778/0 (socket says 172.21.15.94:41778) 2024-02-17T12:12:45.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.636+0000 7fb48ded7640 1 -- 172.21.15.94:0/3230408585 learned_addr learned my addr 172.21.15.94:0/3230408585 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:45.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.636+0000 7fb48ced5640 1 --2- 172.21.15.94:0/3230408585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d92340bd0 0x558d92349250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.637+0000 7fb48d6d6640 1 --2- 172.21.15.94:0/3230408585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923402b0 0x558d92348d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.637+0000 7fb48ded7640 1 -- 172.21.15.94:0/3230408585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923402b0 msgr2=0x558d92348d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.637+0000 7fb48ded7640 1 --2- 172.21.15.94:0/3230408585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923402b0 0x558d92348d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.637+0000 7fb48ded7640 1 -- 172.21.15.94:0/3230408585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d92340bd0 msgr2=0x558d92349250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.637+0000 7fb48ded7640 1 --2- 172.21.15.94:0/3230408585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d92340bd0 0x558d92349250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.637+0000 7fb48ded7640 1 -- 172.21.15.94:0/3230408585 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d92242ac0 con 0x558d923497b0 2024-02-17T12:12:45.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.637+0000 7fb48ced5640 1 --2- 172.21.15.94:0/3230408585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d92340bd0 0x558d92349250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:45.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.637+0000 7fb48ded7640 1 --2- 172.21.15.94:0/3230408585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923497b0 0x558d92350690 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb480006010 tx=0x7fb48002bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a511cfe2d489b57b server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:45.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.637+0000 7fb472ffd640 1 -- 172.21.15.94:0/3230408585 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4800bd150 con 0x558d923497b0 2024-02-17T12:12:45.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.638+0000 7fb472ffd640 1 -- 172.21.15.94:0/3230408585 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4800c12a0 con 0x558d923497b0 2024-02-17T12:12:45.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.638+0000 7fb472ffd640 1 -- 172.21.15.94:0/3230408585 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4800edfa0 con 0x558d923497b0 2024-02-17T12:12:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.638+0000 7fb4925e7740 1 -- 172.21.15.94:0/3230408585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923497b0 msgr2=0x558d92350690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.638+0000 7fb4925e7740 1 --2- 172.21.15.94:0/3230408585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923497b0 0x558d92350690 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb480006010 tx=0x7fb48002bf30 comp rx=0 tx=0).stop 2024-02-17T12:12:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.639+0000 7fb4925e7740 1 -- 172.21.15.94:0/3230408585 shutdown_connections 2024-02-17T12:12:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.639+0000 7fb4925e7740 1 --2- 172.21.15.94:0/3230408585 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923497b0 0x558d92350690 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.639+0000 7fb4925e7740 1 --2- 172.21.15.94:0/3230408585 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d92340bd0 0x558d92349250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.639+0000 7fb4925e7740 1 --2- 172.21.15.94:0/3230408585 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d923402b0 0x558d92348d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.639+0000 7fb4925e7740 1 -- 172.21.15.94:0/3230408585 >> 172.21.15.94:0/3230408585 conn(0x558d9233a990 msgr2=0x558d92360660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.639+0000 7fb4925e7740 1 -- 172.21.15.94:0/3230408585 shutdown_connections 2024-02-17T12:12:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.639+0000 7fb4925e7740 1 -- 172.21.15.94:0/3230408585 wait complete. 2024-02-17T12:12:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.639+0000 7fb4925e7740 1 -- start start 2024-02-17T12:12:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.639+0000 7fb4925e7740 1 Processor -- start 2024-02-17T12:12:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb4925e7740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923402b0 0x558d92350ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb4925e7740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d92340bd0 0x558d9234eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb4925e7740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d923497b0 0x558d9234f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb4925e7740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d9234fb60 con 0x558d923402b0 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb4925e7740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d9234fcd0 con 0x558d923497b0 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb4925e7740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d923afdf0 con 0x558d92340bd0 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb4925e7740 10 client.0 _ll_register_callbacks cb 0x7fb47c00ca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb48d6d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923402b0 0x558d92350ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb48d6d6640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923402b0 0x558d92350ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41788/0 (socket says 172.21.15.94:41788) 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb48d6d6640 1 -- 172.21.15.94:0/321688302 learned_addr learned my addr 172.21.15.94:0/321688302 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb48ced5640 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d92340bd0 0x558d9234eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb48ded7640 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d923497b0 0x558d9234f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb48d6d6640 1 -- 172.21.15.94:0/321688302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d92340bd0 msgr2=0x558d9234eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb48d6d6640 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d92340bd0 0x558d9234eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb48d6d6640 1 -- 172.21.15.94:0/321688302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d923497b0 msgr2=0x558d9234f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb48d6d6640 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d923497b0 0x558d9234f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.647 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb48d6d6640 1 -- 172.21.15.94:0/321688302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558d922a6610 con 0x558d923402b0 2024-02-17T12:12:45.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.640+0000 7fb48ded7640 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d923497b0 0x558d9234f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:45.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.641+0000 7fb48d6d6640 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923402b0 0x558d92350ad0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb484005c20 tx=0x7fb484006410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20906fc752f3c939 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:45.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.641+0000 7fb470ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:45.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.641+0000 7fb470ff9640 1 -- 172.21.15.94:0/321688302 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4840192d0 con 0x558d923402b0 2024-02-17T12:12:45.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.641+0000 7fb470ff9640 1 -- 172.21.15.94:0/321688302 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb484019430 con 0x558d923402b0 2024-02-17T12:12:45.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.641+0000 7fb4925e7740 1 -- 172.21.15.94:0/321688302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb47c01a020 con 0x558d923402b0 2024-02-17T12:12:45.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.641+0000 7fb470ff9640 1 -- 172.21.15.94:0/321688302 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4840195f0 con 0x558d923402b0 2024-02-17T12:12:45.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.641+0000 7fb4925e7740 10 client.19329 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:45.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.641+0000 7fb4925e7740 1 -- 172.21.15.94:0/321688302 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fb488070a10 con 0x558d923402b0 2024-02-17T12:12:45.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.641+0000 7fb470ff9640 1 -- 172.21.15.94:0/321688302 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb48402e080 con 0x558d923402b0 2024-02-17T12:12:45.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.641+0000 7fb470ff9640 1 -- 172.21.15.94:0/321688302 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb484017440 con 0x558d923402b0 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.642+0000 7fb4925e7740 10 client.19329 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.642+0000 7fb4925e7740 10 client.19329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.642+0000 7fb4925e7740 20 client.19329 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.642+0000 7fb4925e7740 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558d923b1600 0x558d923b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.642+0000 7fb4925e7740 4 client.19329 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.642+0000 7fb4925e7740 1 -- 172.21.15.94:0/321688302 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558d92033890 con 0x558d923b1600 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.642+0000 7fb48ced5640 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558d923b1600 0x558d923b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.642+0000 7fb470ff9640 10 client.19329 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.642+0000 7fb48ced5640 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558d923b1600 0x558d923b3a20 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471aed35552bffda server_cookie=e1d5e8b03deb0ca9 in_seq=0 out_seq=0 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb470ff9640 1 -- 172.21.15.94:0/321688302 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x558d92033890 con 0x558d923b1600 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb470ff9640 10 client.19329 handle_command_reply: tid=0 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb4925e7740 1 client.19329 shutdown 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb4925e7740 1 -- 172.21.15.94:0/321688302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923402b0 msgr2=0x558d92350ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.649 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb4925e7740 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923402b0 0x558d92350ad0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb484005c20 tx=0x7fb484006410 comp rx=0 tx=0).stop 2024-02-17T12:12:45.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb4925e7740 1 -- 172.21.15.94:0/321688302 shutdown_connections 2024-02-17T12:12:45.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb4925e7740 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x558d923b1600 0x558d923b3a20 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb4925e7740 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x558d923497b0 0x558d9234f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb470ff9640 0 client.19329 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:45.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb4925e7740 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558d92340bd0 0x558d9234eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb4925e7740 1 --2- 172.21.15.94:0/321688302 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x558d923402b0 0x558d92350ad0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb4925e7740 1 -- 172.21.15.94:0/321688302 >> 172.21.15.94:0/321688302 conn(0x558d9233a990 msgr2=0x558d92341910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:45.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb4925e7740 1 -- 172.21.15.94:0/321688302 shutdown_connections 2024-02-17T12:12:45.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb4925e7740 1 -- 172.21.15.94:0/321688302 wait complete. 2024-02-17T12:12:45.650 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.643+0000 7fb4925e7740 20 client.19329 trim_cache size 0 max 16384 2024-02-17T12:12:45.650 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:12:45.651 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (24 inodes in the stack)", 2024-02-17T12:12:45.651 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:12:45.651 INFO:teuthology.orchestra.run.smithi094.stdout: "f66d1b62-3131-45f7-a6e5-43d519d671e8": { 2024-02-17T12:12:45.651 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:12:45.651 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "f66d1b62-3131-45f7-a6e5-43d519d671e8", 2024-02-17T12:12:45.651 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:12:45.651 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:12:45.651 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:12:45.651 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:12:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.644+0000 7fb48f93a640 1 -- 172.21.15.94:0/2390432656 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4740a2430 msgr2=0x7fb4740a48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.644+0000 7fb48f93a640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4740a2430 0x7fb4740a48f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb47c0058c0 tx=0x7fb47c005850 comp rx=0 tx=0).stop 2024-02-17T12:12:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.644+0000 7fb48f93a640 1 -- 172.21.15.94:0/2390432656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb48806b830 msgr2=0x7fb48806bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.644+0000 7fb48f93a640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb48806b830 0x7fb48806bce0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb480036040 tx=0x7fb480034700 comp rx=0 tx=0).stop 2024-02-17T12:12:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.644+0000 7fb48f93a640 1 -- 172.21.15.94:0/2390432656 shutdown_connections 2024-02-17T12:12:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.644+0000 7fb48f93a640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb4740a2430 0x7fb4740a48f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.652 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.644+0000 7fb48f93a640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb48806b830 0x7fb48806bce0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.644+0000 7fb48f93a640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb488106430 0x7fb488072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.644+0000 7fb48f93a640 1 --2- 172.21.15.94:0/2390432656 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb488101360 0x7fb4880727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.644+0000 7fb48f93a640 1 -- 172.21.15.94:0/2390432656 >> 172.21.15.94:0/2390432656 conn(0x7fb4880fcc20 msgr2=0x7fb4880fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.644+0000 7fb48f93a640 1 -- 172.21.15.94:0/2390432656 shutdown_connections 2024-02-17T12:12:45.653 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:12:45.644+0000 7fb48f93a640 1 -- 172.21.15.94:0/2390432656 wait complete. 2024-02-17T12:12:45.660 INFO:tasks.cephfs.filesystem:scrub status for tag:f66d1b62-3131-45f7-a6e5-43d519d671e8 - {'path': '/', 'tag': 'f66d1b62-3131-45f7-a6e5-43d519d671e8', 'options': 'recursive,force'} 2024-02-17T12:12:46.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:46 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:12:46.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:46 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:12:46.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:46 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:12:46.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:46 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:12:46.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:46 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:12:46.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:46 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:12:46.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:46 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:12:46.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:46 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:12:46.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:46 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:12:47.207 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.205+0000 7f83a1639640 1 -- 172.21.15.94:0/882088657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f839c1013a0 msgr2=0x7f839c101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.205+0000 7f83a1639640 1 --2- 172.21.15.94:0/882088657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f839c1013a0 0x7f839c101780 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f838c009930 tx=0x7f838c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:12:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.205+0000 7f83a1639640 1 -- 172.21.15.94:0/882088657 shutdown_connections 2024-02-17T12:12:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.205+0000 7f83a1639640 1 --2- 172.21.15.94:0/882088657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f839c106470 0x7f839c10e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.205+0000 7f83a1639640 1 --2- 172.21.15.94:0/882088657 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839c101d50 0x7f839c105d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.205+0000 7f83a1639640 1 --2- 172.21.15.94:0/882088657 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f839c1013a0 0x7f839c101780 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.205+0000 7f83a1639640 1 -- 172.21.15.94:0/882088657 >> 172.21.15.94:0/882088657 conn(0x7f839c0fcc40 msgr2=0x7f839c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.205+0000 7f83a1639640 1 -- 172.21.15.94:0/882088657 shutdown_connections 2024-02-17T12:12:47.208 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.205+0000 7f83a1639640 1 -- 172.21.15.94:0/882088657 wait complete. 2024-02-17T12:12:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.206+0000 7f83a1639640 1 Processor -- start 2024-02-17T12:12:47.210 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.206+0000 7f83a1639640 1 -- start start 2024-02-17T12:12:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.206+0000 7f83a1639640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839c1013a0 0x7f839c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.206+0000 7f83a1639640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f839c106470 0x7f839c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.206+0000 7f83a1639640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f839c19c020 0x7f839c19c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.206+0000 7f83a1639640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c19ca10 con 0x7f839c106470 2024-02-17T12:12:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.206+0000 7f83a1639640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c19cb80 con 0x7f839c1013a0 2024-02-17T12:12:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.206+0000 7f83a1639640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c19ccf0 con 0x7f839c19c020 2024-02-17T12:12:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f8393fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f839c106470 0x7f839c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f839b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f839c19c020 0x7f839c19c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f839b7fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f839c19c020 0x7f839c19c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60790/0 (socket says 172.21.15.94:60790) 2024-02-17T12:12:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f839b7fe640 1 -- 172.21.15.94:0/1166060683 learned_addr learned my addr 172.21.15.94:0/1166060683 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f8393fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f839c106470 0x7f839c197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41808/0 (socket says 172.21.15.94:41808) 2024-02-17T12:12:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f839affd640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839c1013a0 0x7f839c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.211 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f839b7fe640 1 -- 172.21.15.94:0/1166060683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839c1013a0 msgr2=0x7f839c197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f839b7fe640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839c1013a0 0x7f839c197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f839b7fe640 1 -- 172.21.15.94:0/1166060683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f839c106470 msgr2=0x7f839c197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f839b7fe640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f839c106470 0x7f839c197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f839b7fe640 1 -- 172.21.15.94:0/1166060683 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c009560 con 0x7f839c19c020 2024-02-17T12:12:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f8393fff640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f839c106470 0x7f839c197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:12:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f839affd640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839c1013a0 0x7f839c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:12:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.207+0000 7f839b7fe640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f839c19c020 0x7f839c19c4d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f839400eb10 tx=0x7f839400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.208+0000 7f8398ff9640 1 -- 172.21.15.94:0/1166060683 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8394004b20 con 0x7f839c19c020 2024-02-17T12:12:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.208+0000 7f8398ff9640 1 -- 172.21.15.94:0/1166060683 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8394004c80 con 0x7f839c19c020 2024-02-17T12:12:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.208+0000 7f83a1639640 1 -- 172.21.15.94:0/1166060683 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839c0733b0 con 0x7f839c19c020 2024-02-17T12:12:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.208+0000 7f8398ff9640 1 -- 172.21.15.94:0/1166060683 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8394010650 con 0x7f839c19c020 2024-02-17T12:12:47.212 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.208+0000 7f83a1639640 1 -- 172.21.15.94:0/1166060683 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839c073860 con 0x7f839c19c020 2024-02-17T12:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.210+0000 7f8398ff9640 1 -- 172.21.15.94:0/1166060683 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f8394005020 con 0x7f839c19c020 2024-02-17T12:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f8398ff9640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83880a22d0 0x7f83880a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f83a4206740 1 Processor -- start 2024-02-17T12:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f83a4206740 1 -- start start 2024-02-17T12:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f839affd640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83880a22d0 0x7f83880a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f8398ff9640 1 -- 172.21.15.94:0/1166060683 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f8394014070 con 0x7f839c19c020 2024-02-17T12:12:47.214 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f83a4206740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290dead0 0x556e290dd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f83a4206740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290ddd60 0x556e290de1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f83a4206740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f83a4206740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e2904c9f0 con 0x556e290dead0 2024-02-17T12:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f83a4206740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e29049f20 con 0x556e290ddd60 2024-02-17T12:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f83a4206740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e29045700 con 0x556e290e72e0 2024-02-17T12:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f839affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290dead0 0x556e290dd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f839affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290dead0 0x556e290dd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41822/0 (socket says 172.21.15.94:41822) 2024-02-17T12:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f839affd640 1 -- 172.21.15.94:0/1918296557 learned_addr learned my addr 172.21.15.94:0/1918296557 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f839b7fe640 1 --2- 172.21.15.94:0/1918296557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.215 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.211+0000 7f8393fff640 1 --2- 172.21.15.94:0/1918296557 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290ddd60 0x556e290de1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.212+0000 7f839b7fe640 1 -- 172.21.15.94:0/1918296557 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290ddd60 msgr2=0x556e290de1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.212+0000 7f839b7fe640 1 --2- 172.21.15.94:0/1918296557 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290ddd60 0x556e290de1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.212+0000 7f839b7fe640 1 -- 172.21.15.94:0/1918296557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290dead0 msgr2=0x556e290dd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.212+0000 7f839b7fe640 1 --2- 172.21.15.94:0/1918296557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290dead0 0x556e290dd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.212+0000 7f839b7fe640 1 -- 172.21.15.94:0/1918296557 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e28fdb7b0 con 0x556e290e72e0 2024-02-17T12:12:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.212+0000 7f839b7fe640 1 --2- 172.21.15.94:0/1918296557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e96d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f83940cbe70 tx=0x7f839400cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca6a735afbe6da8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:47.216 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.213+0000 7f83917fa640 1 -- 172.21.15.94:0/1918296557 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839409f260 con 0x556e290e72e0 2024-02-17T12:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.213+0000 7f83917fa640 1 -- 172.21.15.94:0/1918296557 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83940a38b0 con 0x556e290e72e0 2024-02-17T12:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.213+0000 7f83917fa640 1 -- 172.21.15.94:0/1918296557 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83940a3420 con 0x556e290e72e0 2024-02-17T12:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.214+0000 7f83a4206740 1 -- 172.21.15.94:0/1918296557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 msgr2=0x556e290e96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.214+0000 7f83a4206740 1 --2- 172.21.15.94:0/1918296557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e96d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f83940cbe70 tx=0x7f839400cee0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.214+0000 7f839affd640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83880a22d0 0x7f83880a4790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f838c000c00 tx=0x7f838c004b30 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.214+0000 7f83a4206740 1 -- 172.21.15.94:0/1918296557 shutdown_connections 2024-02-17T12:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.214+0000 7f83a4206740 1 --2- 172.21.15.94:0/1918296557 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e96d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.214+0000 7f83a4206740 1 --2- 172.21.15.94:0/1918296557 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290ddd60 0x556e290de1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.214+0000 7f83a4206740 1 --2- 172.21.15.94:0/1918296557 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290dead0 0x556e290dd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.214+0000 7f83a4206740 1 -- 172.21.15.94:0/1918296557 >> 172.21.15.94:0/1918296557 conn(0x556e290a5550 msgr2=0x556e290d8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.214+0000 7f83a4206740 1 -- 172.21.15.94:0/1918296557 shutdown_connections 2024-02-17T12:12:47.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.214+0000 7f83a4206740 1 -- 172.21.15.94:0/1918296557 wait complete. 2024-02-17T12:12:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.214+0000 7f83a4206740 1 -- start start 2024-02-17T12:12:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f83a4206740 1 Processor -- start 2024-02-17T12:12:47.221 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f83a4206740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290ddd60 0x556e290e4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f83a4206740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290dead0 0x556e290e4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f83a4206740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f8393fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290dead0 0x556e290e4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f839affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290ddd60 0x556e290e4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f8393fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290dead0 0x556e290e4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41828/0 (socket says 172.21.15.94:41828) 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f839affd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290ddd60 0x556e290e4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:42632/0 (socket says 172.21.15.94:42632) 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f8393fff640 1 -- 172.21.15.94:0/3711647753 learned_addr learned my addr 172.21.15.94:0/3711647753 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f83a4206740 1 -- 172.21.15.94:0/3711647753 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e28feabf0 con 0x556e290dead0 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f83a4206740 1 -- 172.21.15.94:0/3711647753 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e28d19250 con 0x556e290ddd60 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f83a4206740 1 -- 172.21.15.94:0/3711647753 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e28fcb200 con 0x556e290e72e0 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f83a4206740 10 client.0 _ll_register_callbacks cb 0x556e29039d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.215+0000 7f839b7fe640 1 --2- 172.21.15.94:0/3711647753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f8393fff640 1 -- 172.21.15.94:0/3711647753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 msgr2=0x556e290e4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:47.222 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f8393fff640 1 --2- 172.21.15.94:0/3711647753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f8393fff640 1 -- 172.21.15.94:0/3711647753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290ddd60 msgr2=0x556e290e4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f8393fff640 1 --2- 172.21.15.94:0/3711647753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290ddd60 0x556e290e4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f8393fff640 1 -- 172.21.15.94:0/3711647753 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839c073860 con 0x556e290dead0 2024-02-17T12:12:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f8393fff640 1 --2- 172.21.15.94:0/3711647753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290dead0 0x556e290e4770 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f838400bf30 tx=0x7f83840128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ec699f3e6534bf server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f83827fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f83827fc640 1 -- 172.21.15.94:0/3711647753 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384004500 con 0x556e290dead0 2024-02-17T12:12:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f83827fc640 1 -- 172.21.15.94:0/3711647753 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f83840035f0 con 0x556e290dead0 2024-02-17T12:12:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f83a4206740 1 -- 172.21.15.94:0/3711647753 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e28d45600 con 0x556e290dead0 2024-02-17T12:12:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f83827fc640 1 -- 172.21.15.94:0/3711647753 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83840037b0 con 0x556e290dead0 2024-02-17T12:12:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f83a4206740 10 client.19341 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f83a4206740 1 -- 172.21.15.94:0/3711647753 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556e29043d80 con 0x556e290dead0 2024-02-17T12:12:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.216+0000 7f83827fc640 1 -- 172.21.15.94:0/3711647753 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x556e28d45600 con 0x556e290dead0 2024-02-17T12:12:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.217+0000 7f83827fc640 1 -- 172.21.15.94:0/3711647753 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f8384010c20 con 0x556e290dead0 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.217+0000 7f83a4206740 10 client.19341 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.217+0000 7f83a4206740 10 client.19341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.217+0000 7f83a4206740 20 client.19341 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.217+0000 7f83a4206740 1 --2- 172.21.15.94:0/3711647753 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556e290f7630 0x556e290f9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.217+0000 7f83a4206740 4 client.19341 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.217+0000 7f83a4206740 1 -- 172.21.15.94:0/3711647753 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e28d961b0 con 0x556e290f7630 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.217+0000 7f839affd640 1 --2- 172.21.15.94:0/3711647753 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556e290f7630 0x556e290f9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.218+0000 7f839affd640 1 --2- 172.21.15.94:0/3711647753 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556e290f7630 0x556e290f9a50 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f314e301f45884c server_cookie=7b0f3328599449ae in_seq=0 out_seq=0 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.218+0000 7f83827fc640 10 client.19341 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.218+0000 7f83827fc640 1 -- 172.21.15.94:0/3711647753 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556e28d961b0 con 0x556e290f7630 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.218+0000 7f83827fc640 10 client.19341 handle_command_reply: tid=0 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.218+0000 7f83a4206740 1 client.19341 shutdown 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.218+0000 7f83a4206740 1 -- 172.21.15.94:0/3711647753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290dead0 msgr2=0x556e290e4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.218+0000 7f83a4206740 1 --2- 172.21.15.94:0/3711647753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290dead0 0x556e290e4770 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f838400bf30 tx=0x7f83840128e0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.219+0000 7f83a4206740 1 -- 172.21.15.94:0/3711647753 shutdown_connections 2024-02-17T12:12:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.219+0000 7f83a4206740 1 --2- 172.21.15.94:0/3711647753 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556e290f7630 0x556e290f9a50 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.219+0000 7f83a4206740 1 --2- 172.21.15.94:0/3711647753 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.219+0000 7f83827fc640 0 client.19341 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.219+0000 7f83a4206740 1 --2- 172.21.15.94:0/3711647753 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290dead0 0x556e290e4770 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.219+0000 7f83a4206740 1 --2- 172.21.15.94:0/3711647753 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290ddd60 0x556e290e4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.219+0000 7f83a4206740 1 -- 172.21.15.94:0/3711647753 >> 172.21.15.94:0/3711647753 conn(0x556e290a5550 msgr2=0x556e290da7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.219+0000 7f83a4206740 1 -- 172.21.15.94:0/3711647753 shutdown_connections 2024-02-17T12:12:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.219+0000 7f83a4206740 1 -- 172.21.15.94:0/3711647753 wait complete. 2024-02-17T12:12:47.225 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.219+0000 7f83a4206740 20 client.19341 trim_cache size 0 max 16384 2024-02-17T12:12:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.238+0000 7f83a4206740 1 Processor -- start 2024-02-17T12:12:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.238+0000 7f83a4206740 1 -- start start 2024-02-17T12:12:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.238+0000 7f83a4206740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290ddd60 0x556e290e6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.238+0000 7f83a4206740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290dead0 0x556e290e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.238+0000 7f83a4206740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.238+0000 7f83a4206740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e290e9ef0 con 0x556e290ddd60 2024-02-17T12:12:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.238+0000 7f83a4206740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e290feca0 con 0x556e290dead0 2024-02-17T12:12:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.238+0000 7f83a4206740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e290e27a0 con 0x556e290e72e0 2024-02-17T12:12:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.238+0000 7f839affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290ddd60 0x556e290e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.238+0000 7f839affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290ddd60 0x556e290e6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41834/0 (socket says 172.21.15.94:41834) 2024-02-17T12:12:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.238+0000 7f839affd640 1 -- 172.21.15.94:0/1554741413 learned_addr learned my addr 172.21.15.94:0/1554741413 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.238+0000 7f839b7fe640 1 --2- 172.21.15.94:0/1554741413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.238+0000 7f8393fff640 1 --2- 172.21.15.94:0/1554741413 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290dead0 0x556e290e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.239+0000 7f839b7fe640 1 -- 172.21.15.94:0/1554741413 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290dead0 msgr2=0x556e290e4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.239+0000 7f839b7fe640 1 --2- 172.21.15.94:0/1554741413 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290dead0 0x556e290e4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.239+0000 7f839b7fe640 1 -- 172.21.15.94:0/1554741413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290ddd60 msgr2=0x556e290e6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.239+0000 7f839b7fe640 1 --2- 172.21.15.94:0/1554741413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290ddd60 0x556e290e6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.239+0000 7f839b7fe640 1 -- 172.21.15.94:0/1554741413 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e28fe07b0 con 0x556e290e72e0 2024-02-17T12:12:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.239+0000 7f839b7fe640 1 --2- 172.21.15.94:0/1554741413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e14d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8394097570 tx=0x7f839409f160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c47707ebb38e6f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.239+0000 7f8390ff9640 1 -- 172.21.15.94:0/1554741413 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839409f260 con 0x556e290e72e0 2024-02-17T12:12:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.239+0000 7f8390ff9640 1 -- 172.21.15.94:0/1554741413 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83940bf080 con 0x556e290e72e0 2024-02-17T12:12:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.239+0000 7f8390ff9640 1 -- 172.21.15.94:0/1554741413 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83940a3680 con 0x556e290e72e0 2024-02-17T12:12:47.246 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.240+0000 7f83a4206740 1 -- 172.21.15.94:0/1554741413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 msgr2=0x556e290e14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.240+0000 7f83a4206740 1 --2- 172.21.15.94:0/1554741413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e14d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8394097570 tx=0x7f839409f160 comp rx=0 tx=0).stop 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.240+0000 7f83a4206740 1 -- 172.21.15.94:0/1554741413 shutdown_connections 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.240+0000 7f83a4206740 1 --2- 172.21.15.94:0/1554741413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290e72e0 0x556e290e14d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.240+0000 7f83a4206740 1 --2- 172.21.15.94:0/1554741413 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290dead0 0x556e290e4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.240+0000 7f83a4206740 1 --2- 172.21.15.94:0/1554741413 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290ddd60 0x556e290e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.240+0000 7f83a4206740 1 -- 172.21.15.94:0/1554741413 >> 172.21.15.94:0/1554741413 conn(0x556e290a5550 msgr2=0x556e290df580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.240+0000 7f83a4206740 1 -- 172.21.15.94:0/1554741413 shutdown_connections 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.240+0000 7f83a4206740 1 -- 172.21.15.94:0/1554741413 wait complete. 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f83a4206740 1 -- start start 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f83a4206740 1 Processor -- start 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f83a4206740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290ddd60 0x556e290ee0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f83a4206740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290dead0 0x556e290ee620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f83a4206740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290e72e0 0x556e290ec6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.247 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f83a4206740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e290ecdb0 con 0x556e290e72e0 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f83a4206740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e290ecf20 con 0x556e290dead0 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f83a4206740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e290ed0e0 con 0x556e290ddd60 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f83a4206740 10 client.0 _ll_register_callbacks cb 0x7f8394004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f839b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290e72e0 0x556e290ec6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f839affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290ddd60 0x556e290ee0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f839affd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290ddd60 0x556e290ee0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:60822/0 (socket says 172.21.15.94:60822) 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f839affd640 1 -- 172.21.15.94:0/3744724011 learned_addr learned my addr 172.21.15.94:0/3744724011 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f839b7fe640 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290e72e0 0x556e290ec6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:41836/0 (socket says 172.21.15.94:41836) 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f8393fff640 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290dead0 0x556e290ee620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.241+0000 7f839b7fe640 1 -- 172.21.15.94:0/3744724011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290ddd60 msgr2=0x556e290ee0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f839b7fe640 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290ddd60 0x556e290ee0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f839b7fe640 1 -- 172.21.15.94:0/3744724011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290dead0 msgr2=0x556e290ee620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f839b7fe640 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290dead0 0x556e290ee620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.248 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f839b7fe640 1 -- 172.21.15.94:0/3744724011 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556e28fdb7b0 con 0x556e290e72e0 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f839affd640 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290ddd60 0x556e290ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f839b7fe640 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290e72e0 0x556e290ec6e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f839409ae10 tx=0x7f8394009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf238401ecd3ffd4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83827fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83827fc640 1 -- 172.21.15.94:0/3744724011 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839409f260 con 0x556e290e72e0 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83827fc640 1 -- 172.21.15.94:0/3744724011 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f83940b3e40 con 0x556e290e72e0 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83a4206740 1 -- 172.21.15.94:0/3744724011 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83940a38b0 con 0x556e290e72e0 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83827fc640 1 -- 172.21.15.94:0/3744724011 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83940a3420 con 0x556e290e72e0 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83a4206740 10 client.19347 fetch_fsmap learned FSMap version 27 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83a4206740 1 -- 172.21.15.94:0/3744724011 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556e29006540 con 0x556e290e72e0 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83827fc640 1 -- 172.21.15.94:0/3744724011 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f83940a38b0 con 0x556e290e72e0 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83827fc640 1 -- 172.21.15.94:0/3744724011 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f839401f060 con 0x556e290e72e0 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83a4206740 10 client.19347 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83a4206740 10 client.19347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:12:47.249 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83a4206740 20 client.19347 populate_metadata read hostname 'smithi094' 2024-02-17T12:12:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83a4206740 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556e290f7630 0x556e29150a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:12:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83a4206740 4 client.19347 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:12:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.242+0000 7f83a4206740 1 -- 172.21.15.94:0/3744724011 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556e28d82550 con 0x556e290f7630 2024-02-17T12:12:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.243+0000 7f839affd640 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556e290f7630 0x556e29150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:12:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.243+0000 7f83827fc640 10 client.19347 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:47.250 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.243+0000 7f839affd640 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556e290f7630 0x556e29150a10 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f06a98f80b1ae86e server_cookie=56fc3fb93987b7f2 in_seq=0 out_seq=0 2024-02-17T12:12:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83827fc640 1 -- 172.21.15.94:0/3744724011 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556e28d82550 con 0x556e290f7630 2024-02-17T12:12:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83827fc640 10 client.19347 handle_command_reply: tid=0 2024-02-17T12:12:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83a4206740 1 client.19347 shutdown 2024-02-17T12:12:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83a4206740 1 -- 172.21.15.94:0/3744724011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290e72e0 msgr2=0x556e290ec6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83a4206740 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290e72e0 0x556e290ec6e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f839409ae10 tx=0x7f8394009a70 comp rx=0 tx=0).stop 2024-02-17T12:12:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83a4206740 1 -- 172.21.15.94:0/3744724011 shutdown_connections 2024-02-17T12:12:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83a4206740 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x556e290f7630 0x556e29150a10 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83827fc640 0 client.19347 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:12:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83a4206740 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x556e290e72e0 0x556e290ec6e0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83a4206740 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x556e290dead0 0x556e290ee620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83a4206740 1 --2- 172.21.15.94:0/3744724011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556e290ddd60 0x556e290ee0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83a4206740 1 -- 172.21.15.94:0/3744724011 >> 172.21.15.94:0/3744724011 conn(0x556e290a5550 msgr2=0x556e290e2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83a4206740 1 -- 172.21.15.94:0/3744724011 shutdown_connections 2024-02-17T12:12:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83a4206740 1 -- 172.21.15.94:0/3744724011 wait complete. 2024-02-17T12:12:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.303+0000 7f83a4206740 20 client.19347 trim_cache size 0 max 16384 2024-02-17T12:12:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:12:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:12:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:12:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:12:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.304+0000 7f83a1639640 1 -- 172.21.15.94:0/1166060683 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83880a22d0 msgr2=0x7f83880a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.304+0000 7f83a1639640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83880a22d0 0x7f83880a4790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f838c000c00 tx=0x7f838c004b30 comp rx=0 tx=0).stop 2024-02-17T12:12:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.304+0000 7f83a1639640 1 -- 172.21.15.94:0/1166060683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f839c19c020 msgr2=0x7f839c19c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:12:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.304+0000 7f83a1639640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f839c19c020 0x7f839c19c4d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f839400eb10 tx=0x7f839400c2d0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.304+0000 7f83a1639640 1 -- 172.21.15.94:0/1166060683 shutdown_connections 2024-02-17T12:12:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.304+0000 7f83a1639640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f83880a22d0 0x7f83880a4790 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.304+0000 7f83a1639640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f839c19c020 0x7f839c19c4d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.304+0000 7f83a1639640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f839c106470 0x7f839c197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.304+0000 7f83a1639640 1 --2- 172.21.15.94:0/1166060683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f839c1013a0 0x7f839c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:12:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.304+0000 7f83a1639640 1 -- 172.21.15.94:0/1166060683 >> 172.21.15.94:0/1166060683 conn(0x7f839c0fcc40 msgr2=0x7f839c0feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:12:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.306+0000 7f83a1639640 1 -- 172.21.15.94:0/1166060683 shutdown_connections 2024-02-17T12:12:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:12:47.306+0000 7f83a1639640 1 -- 172.21.15.94:0/1166060683 wait complete. 2024-02-17T12:12:47.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:47 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:12:47.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:47 smithi189 ceph-mon[36588]: pgmap v459: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 598 op/s 2024-02-17T12:12:47.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:47 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:12:47.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:47 smithi125 ceph-mon[36819]: pgmap v459: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 598 op/s 2024-02-17T12:12:47.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:47 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:12:47.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:47 smithi094 ceph-mon[32499]: pgmap v459: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.7 MiB/s wr, 598 op/s 2024-02-17T12:12:48.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:48 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:12:48.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:48 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:12:48.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:48 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:12:49.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:49 smithi125 ceph-mon[36819]: pgmap v460: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.1 MiB/s wr, 801 op/s 2024-02-17T12:12:49.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:49 smithi094 ceph-mon[32499]: pgmap v460: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.1 MiB/s wr, 801 op/s 2024-02-17T12:12:49.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:49 smithi189 ceph-mon[36588]: pgmap v460: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 7.1 MiB/s wr, 801 op/s 2024-02-17T12:12:50.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 870.0 s, 1585.4 tps, lat 63.152 ms stddev 70.760 2024-02-17T12:12:51.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:51 smithi189 ceph-mon[36588]: pgmap v461: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.0 MiB/s wr, 741 op/s 2024-02-17T12:12:51.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:51 smithi125 ceph-mon[36819]: pgmap v461: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.0 MiB/s wr, 741 op/s 2024-02-17T12:12:51.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:51 smithi094 ceph-mon[32499]: pgmap v461: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.0 MiB/s wr, 741 op/s 2024-02-17T12:12:53.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:53 smithi189 ceph-mon[36588]: pgmap v462: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.8 MiB/s wr, 619 op/s 2024-02-17T12:12:53.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:53 smithi125 ceph-mon[36819]: pgmap v462: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.8 MiB/s wr, 619 op/s 2024-02-17T12:12:53.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:53 smithi094 ceph-mon[32499]: pgmap v462: 97 pgs: 97 active+clean; 5.3 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 4.8 MiB/s wr, 619 op/s 2024-02-17T12:12:55.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 875.0 s, 1400.4 tps, lat 70.118 ms stddev 73.562 2024-02-17T12:12:55.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:55 smithi189 ceph-mon[36588]: pgmap v463: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.8 MiB/s wr, 843 op/s 2024-02-17T12:12:55.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:55 smithi125 ceph-mon[36819]: pgmap v463: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.8 MiB/s wr, 843 op/s 2024-02-17T12:12:55.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:55 smithi094 ceph-mon[32499]: pgmap v463: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 6.8 MiB/s wr, 843 op/s 2024-02-17T12:12:57.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:57 smithi189 ceph-mon[36588]: pgmap v464: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 MiB/s wr, 616 op/s 2024-02-17T12:12:57.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:57 smithi125 ceph-mon[36819]: pgmap v464: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 MiB/s wr, 616 op/s 2024-02-17T12:12:57.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:57 smithi094 ceph-mon[32499]: pgmap v464: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.7 MiB/s wr, 616 op/s 2024-02-17T12:12:59.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:12:59 smithi189 ceph-mon[36588]: pgmap v465: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.0 MiB/s wr, 741 op/s 2024-02-17T12:12:59.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:12:59 smithi125 ceph-mon[36819]: pgmap v465: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.0 MiB/s wr, 741 op/s 2024-02-17T12:12:59.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:12:59 smithi094 ceph-mon[32499]: pgmap v465: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.0 MiB/s wr, 741 op/s 2024-02-17T12:13:00.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 880.0 s, 1253.6 tps, lat 81.154 ms stddev 87.177 2024-02-17T12:13:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.484+0000 7f9eed6f4640 1 -- 172.21.15.94:0/2803183640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee81050c0 msgr2=0x7f9ee81054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.484+0000 7f9eed6f4640 1 --2- 172.21.15.94:0/2803183640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee81050c0 0x7f9ee81054a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009900 tx=0x7f9ed802bc80 comp rx=0 tx=0).stop 2024-02-17T12:13:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.484+0000 7f9eed6f4640 1 -- 172.21.15.94:0/2803183640 shutdown_connections 2024-02-17T12:13:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.484+0000 7f9eed6f4640 1 --2- 172.21.15.94:0/2803183640 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ee8100d00 0x7f9ee810cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.484+0000 7f9eed6f4640 1 --2- 172.21.15.94:0/2803183640 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee8100360 0x7f9ee81007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.484+0000 7f9eed6f4640 1 --2- 172.21.15.94:0/2803183640 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee81050c0 0x7f9ee81054a0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.484+0000 7f9eed6f4640 1 -- 172.21.15.94:0/2803183640 >> 172.21.15.94:0/2803183640 conn(0x7f9ee8076e60 msgr2=0x7f9ee8077270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.484+0000 7f9eed6f4640 1 -- 172.21.15.94:0/2803183640 shutdown_connections 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.484+0000 7f9eed6f4640 1 -- 172.21.15.94:0/2803183640 wait complete. 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9eed6f4640 1 Processor -- start 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9eed6f4640 1 -- start start 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9eed6f4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ee8100360 0x7f9ee80fdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9eed6f4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee8100d00 0x7f9ee80fe190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9eed6f4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee81050c0 0x7f9ee80fe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9eed6f4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee81a5b30 con 0x7f9ee81050c0 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9eed6f4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee81a5ca0 con 0x7f9ee8100d00 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9eed6f4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee81a5e60 con 0x7f9ee8100360 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee81050c0 0x7f9ee80fe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9ee77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee81050c0 0x7f9ee80fe6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45002/0 (socket says 172.21.15.94:45002) 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9ee77fe640 1 -- 172.21.15.94:0/4070708851 learned_addr learned my addr 172.21.15.94:0/4070708851 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9ee6ffd640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ee8100360 0x7f9ee80fdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9edffff640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee8100d00 0x7f9ee80fe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9ee77fe640 1 -- 172.21.15.94:0/4070708851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ee8100360 msgr2=0x7f9ee80fdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9ee77fe640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ee8100360 0x7f9ee80fdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9ee77fe640 1 -- 172.21.15.94:0/4070708851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee8100d00 msgr2=0x7f9ee80fe190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9ee77fe640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee8100d00 0x7f9ee80fe190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.485+0000 7f9ee77fe640 1 -- 172.21.15.94:0/4070708851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed8009560 con 0x7f9ee81050c0 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.486+0000 7f9ee77fe640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee81050c0 0x7f9ee80fe6d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000e960 tx=0x7f9ee000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.486+0000 7f9ee4ff9640 1 -- 172.21.15.94:0/4070708851 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0004d20 con 0x7f9ee81050c0 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.486+0000 7f9eed6f4640 1 -- 172.21.15.94:0/4070708851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee81a6140 con 0x7f9ee81050c0 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.486+0000 7f9ee4ff9640 1 -- 172.21.15.94:0/4070708851 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ee0004e80 con 0x7f9ee81050c0 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.486+0000 7f9ee4ff9640 1 -- 172.21.15.94:0/4070708851 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00116d0 con 0x7f9ee81050c0 2024-02-17T12:13:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.486+0000 7f9eed6f4640 1 -- 172.21.15.94:0/4070708851 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee81a6650 con 0x7f9ee81050c0 2024-02-17T12:13:00.497 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.488+0000 7f9ee4ff9640 1 -- 172.21.15.94:0/4070708851 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9ee0005340 con 0x7f9ee81050c0 2024-02-17T12:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.488+0000 7f9ef03a6740 1 Processor -- start 2024-02-17T12:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.488+0000 7f9ef03a6740 1 -- start start 2024-02-17T12:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ee4ff9640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9ed40a2300 0x7f9ed40a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ef03a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009fad0 0x55b62009e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ef03a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009ed60 0x55b62009f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ef03a6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 0x55b6200aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ef03a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62000d9f0 con 0x55b62009ed60 2024-02-17T12:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ef03a6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62000af20 con 0x55b6200a82e0 2024-02-17T12:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ef03a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b620006700 con 0x55b62009fad0 2024-02-17T12:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9edffff640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9ed40a2300 0x7f9ed40a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9edffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009ed60 0x55b62009f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ee4ff9640 1 -- 172.21.15.94:0/4070708851 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9ee00ce160 con 0x7f9ee81050c0 2024-02-17T12:13:00.498 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9edffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009ed60 0x55b62009f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45014/0 (socket says 172.21.15.94:45014) 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9edffff640 1 -- 172.21.15.94:0/3375811377 learned_addr learned my addr 172.21.15.94:0/3375811377 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ee77fe640 1 --2- 172.21.15.94:0/3375811377 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 0x55b6200aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ee6ffd640 1 --2- 172.21.15.94:0/3375811377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009fad0 0x55b62009e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ee6ffd640 1 -- 172.21.15.94:0/3375811377 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 msgr2=0x55b6200aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ee6ffd640 1 --2- 172.21.15.94:0/3375811377 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 0x55b6200aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ee6ffd640 1 -- 172.21.15.94:0/3375811377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009ed60 msgr2=0x55b62009f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ee6ffd640 1 --2- 172.21.15.94:0/3375811377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009ed60 0x55b62009f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ee6ffd640 1 -- 172.21.15.94:0/3375811377 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b61ff9c7b0 con 0x55b62009fad0 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9edffff640 1 --2- 172.21.15.94:0/3375811377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009ed60 0x55b62009f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9ee77fe640 1 --2- 172.21.15.94:0/3375811377 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 0x55b6200aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.489+0000 7f9edffff640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9ed40a2300 0x7f9ed40a47c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0012960 tx=0x7f9ed000bb70 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.490+0000 7f9ee6ffd640 1 --2- 172.21.15.94:0/3375811377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009fad0 0x55b62009e820 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8000c00 tx=0x7f9ed802c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cfc8b251c36d5ea server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.490+0000 7f9edd7fa640 1 -- 172.21.15.94:0/3375811377 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed802cab0 con 0x55b62009fad0 2024-02-17T12:13:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.490+0000 7f9edd7fa640 1 -- 172.21.15.94:0/3375811377 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ed802cc10 con 0x55b62009fad0 2024-02-17T12:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.490+0000 7f9edd7fa640 1 -- 172.21.15.94:0/3375811377 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8005150 con 0x55b62009fad0 2024-02-17T12:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 -- 172.21.15.94:0/3375811377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009fad0 msgr2=0x55b62009e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/3375811377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009fad0 0x55b62009e820 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8000c00 tx=0x7f9ed802c800 comp rx=0 tx=0).stop 2024-02-17T12:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 -- 172.21.15.94:0/3375811377 shutdown_connections 2024-02-17T12:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/3375811377 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 0x55b6200aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/3375811377 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009ed60 0x55b62009f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/3375811377 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009fad0 0x55b62009e820 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 -- 172.21.15.94:0/3375811377 >> 172.21.15.94:0/3375811377 conn(0x55b620066550 msgr2=0x55b620099a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 -- 172.21.15.94:0/3375811377 shutdown_connections 2024-02-17T12:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 -- 172.21.15.94:0/3375811377 wait complete. 2024-02-17T12:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 -- start start 2024-02-17T12:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 Processor -- start 2024-02-17T12:13:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009ed60 0x55b6200a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200a5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 0x55b6200a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61ffabbf0 con 0x55b62009fad0 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61fcda250 con 0x55b6200a82e0 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61ff8c200 con 0x55b62009ed60 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9ef03a6740 10 client.0 _ll_register_callbacks cb 0x55b61fffad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9edffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.491+0000 7f9edffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200a5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45016/0 (socket says 172.21.15.94:45016) 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ee6ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009ed60 0x55b6200a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9edffff640 1 -- 172.21.15.94:0/306878012 learned_addr learned my addr 172.21.15.94:0/306878012 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ee77fe640 1 --2- 172.21.15.94:0/306878012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 0x55b6200a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ee6ffd640 1 -- 172.21.15.94:0/306878012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 msgr2=0x55b6200a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ee6ffd640 1 --2- 172.21.15.94:0/306878012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 0x55b6200a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ee6ffd640 1 -- 172.21.15.94:0/306878012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 msgr2=0x55b6200a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ee6ffd640 1 --2- 172.21.15.94:0/306878012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200a5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ee6ffd640 1 -- 172.21.15.94:0/306878012 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee810d7f0 con 0x55b62009ed60 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9edffff640 1 --2- 172.21.15.94:0/306878012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ee6ffd640 1 --2- 172.21.15.94:0/306878012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009ed60 0x55b6200a52d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9ed802c1c0 tx=0x7f9ed8004970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80b85449886c4953 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ece7fc640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ece7fc640 1 -- 172.21.15.94:0/306878012 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed802cab0 con 0x55b62009ed60 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ece7fc640 1 -- 172.21.15.94:0/306878012 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9ed80043f0 con 0x55b62009ed60 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ef03a6740 1 -- 172.21.15.94:0/306878012 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b61fd06600 con 0x55b62009ed60 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ece7fc640 1 -- 172.21.15.94:0/306878012 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ed8005100 con 0x55b62009ed60 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ef03a6740 10 client.28780 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ef03a6740 1 -- 172.21.15.94:0/306878012 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b620004d80 con 0x55b62009ed60 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.492+0000 7f9ece7fc640 1 -- 172.21.15.94:0/306878012 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55b61fd06600 con 0x55b62009ed60 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.493+0000 7f9ece7fc640 1 -- 172.21.15.94:0/306878012 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9ed803c050 con 0x55b62009ed60 2024-02-17T12:13:00.502 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.493+0000 7f9ef03a6740 10 client.28780 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.493+0000 7f9ef03a6740 10 client.28780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.493+0000 7f9ef03a6740 20 client.28780 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.493+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/306878012 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b6200b8250 0x55b6200ba640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.493+0000 7f9ef03a6740 4 client.28780 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.493+0000 7f9ef03a6740 1 -- 172.21.15.94:0/306878012 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b61fd571b0 con 0x55b6200b8250 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.493+0000 7f9edffff640 1 --2- 172.21.15.94:0/306878012 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b6200b8250 0x55b6200ba640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.493+0000 7f9ece7fc640 10 client.28780 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.493+0000 7f9edffff640 1 --2- 172.21.15.94:0/306878012 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b6200b8250 0x55b6200ba640 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de87202af888c22d server_cookie=576681d3381221e3 in_seq=0 out_seq=0 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ece7fc640 1 -- 172.21.15.94:0/306878012 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b61fd571b0 con 0x55b6200b8250 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ece7fc640 10 client.28780 handle_command_reply: tid=0 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ef03a6740 1 client.28780 shutdown 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ef03a6740 1 -- 172.21.15.94:0/306878012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009ed60 msgr2=0x55b6200a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/306878012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009ed60 0x55b6200a52d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9ed802c1c0 tx=0x7f9ed8004970 comp rx=0 tx=0).stop 2024-02-17T12:13:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ef03a6740 1 -- 172.21.15.94:0/306878012 shutdown_connections 2024-02-17T12:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/306878012 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b6200b8250 0x55b6200ba640 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/306878012 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 0x55b6200a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/306878012 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200a5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/306878012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009ed60 0x55b6200a52d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ece7fc640 0 client.28780 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ef03a6740 1 -- 172.21.15.94:0/306878012 >> 172.21.15.94:0/306878012 conn(0x55b620066550 msgr2=0x55b62009b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ef03a6740 1 -- 172.21.15.94:0/306878012 shutdown_connections 2024-02-17T12:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ef03a6740 1 -- 172.21.15.94:0/306878012 wait complete. 2024-02-17T12:13:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.494+0000 7f9ef03a6740 20 client.28780 trim_cache size 0 max 16384 2024-02-17T12:13:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9ef03a6740 1 Processor -- start 2024-02-17T12:13:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9ef03a6740 1 -- start start 2024-02-17T12:13:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9ef03a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009ed60 0x55b6200a7a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9ef03a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9ef03a6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 0x55b6200a3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9ef03a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61fd571b0 con 0x55b62009fad0 2024-02-17T12:13:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9ef03a6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61fd06600 con 0x55b6200a82e0 2024-02-17T12:13:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9ef03a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b62009eb10 con 0x55b62009ed60 2024-02-17T12:13:00.511 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9edffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9edffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200a5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45018/0 (socket says 172.21.15.94:45018) 2024-02-17T12:13:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9edffff640 1 -- 172.21.15.94:0/1188324612 learned_addr learned my addr 172.21.15.94:0/1188324612 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9ee6ffd640 1 --2- 172.21.15.94:0/1188324612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009ed60 0x55b6200a7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9edffff640 1 -- 172.21.15.94:0/1188324612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009ed60 msgr2=0x55b6200a7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9edffff640 1 --2- 172.21.15.94:0/1188324612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009ed60 0x55b6200a7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9ee77fe640 1 --2- 172.21.15.94:0/1188324612 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 0x55b6200a3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9edffff640 1 -- 172.21.15.94:0/1188324612 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 msgr2=0x55b6200a3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9edffff640 1 --2- 172.21.15.94:0/1188324612 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 0x55b6200a3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.508+0000 7f9edffff640 1 -- 172.21.15.94:0/1188324612 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b61ffa17b0 con 0x55b62009fad0 2024-02-17T12:13:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.509+0000 7f9edffff640 1 --2- 172.21.15.94:0/1188324612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200a5b70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x55b6200a4b30 tx=0x7f9ed001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34985f712d3847 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.509+0000 7f9edcff9640 1 -- 172.21.15.94:0/1188324612 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0027040 con 0x55b62009fad0 2024-02-17T12:13:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.509+0000 7f9edcff9640 1 -- 172.21.15.94:0/1188324612 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ed002a820 con 0x55b62009fad0 2024-02-17T12:13:00.512 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.509+0000 7f9edcff9640 1 -- 172.21.15.94:0/1188324612 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed001e7e0 con 0x55b62009fad0 2024-02-17T12:13:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.509+0000 7f9ef03a6740 1 -- 172.21.15.94:0/1188324612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 msgr2=0x55b6200a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.509+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/1188324612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200a5b70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x55b6200a4b30 tx=0x7f9ed001f000 comp rx=0 tx=0).stop 2024-02-17T12:13:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 -- 172.21.15.94:0/1188324612 shutdown_connections 2024-02-17T12:13:00.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/1188324612 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b6200a82e0 0x55b6200a3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/1188324612 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200a5b70 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/1188324612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b62009ed60 0x55b6200a7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 -- 172.21.15.94:0/1188324612 >> 172.21.15.94:0/1188324612 conn(0x55b620066550 msgr2=0x55b6200befe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 -- 172.21.15.94:0/1188324612 shutdown_connections 2024-02-17T12:13:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 -- 172.21.15.94:0/1188324612 wait complete. 2024-02-17T12:13:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 -- start start 2024-02-17T12:13:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 Processor -- start 2024-02-17T12:13:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b62009ed60 0x55b6200af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6200a82e0 0x55b6200adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6200af990 con 0x55b62009fad0 2024-02-17T12:13:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6200ae130 con 0x55b62009ed60 2024-02-17T12:13:00.517 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6200ae2f0 con 0x55b6200a82e0 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ef03a6740 10 client.0 _ll_register_callbacks cb 0x7f9ed8005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9edffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9edffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200ad6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:45030/0 (socket says 172.21.15.94:45030) 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9edffff640 1 -- 172.21.15.94:0/484013942 learned_addr learned my addr 172.21.15.94:0/484013942 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ee6ffd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b62009ed60 0x55b6200af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.510+0000 7f9ee77fe640 1 --2- 172.21.15.94:0/484013942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6200a82e0 0x55b6200adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9edffff640 1 -- 172.21.15.94:0/484013942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6200a82e0 msgr2=0x55b6200adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9edffff640 1 --2- 172.21.15.94:0/484013942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6200a82e0 0x55b6200adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9edffff640 1 -- 172.21.15.94:0/484013942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b62009ed60 msgr2=0x55b6200af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9edffff640 1 --2- 172.21.15.94:0/484013942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b62009ed60 0x55b6200af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9edffff640 1 -- 172.21.15.94:0/484013942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed000b7d0 con 0x55b62009fad0 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ee77fe640 1 --2- 172.21.15.94:0/484013942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6200a82e0 0x55b6200adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9edffff640 1 --2- 172.21.15.94:0/484013942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200ad6b0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0010740 tx=0x7f9ed002aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bde5550d40c83 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:00.518 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ece7fc640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ece7fc640 1 -- 172.21.15.94:0/484013942 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0027040 con 0x55b62009fad0 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ece7fc640 1 -- 172.21.15.94:0/484013942 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9ed002a540 con 0x55b62009fad0 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ef03a6740 1 -- 172.21.15.94:0/484013942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ed802cc10 con 0x55b62009fad0 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ece7fc640 1 -- 172.21.15.94:0/484013942 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ed001e520 con 0x55b62009fad0 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ef03a6740 10 client.19365 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ef03a6740 1 -- 172.21.15.94:0/484013942 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b61ffc7540 con 0x55b62009fad0 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ece7fc640 1 -- 172.21.15.94:0/484013942 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9ed0041020 con 0x55b62009fad0 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ece7fc640 1 -- 172.21.15.94:0/484013942 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f9ed00337d0 con 0x55b62009fad0 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ef03a6740 10 client.19365 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ef03a6740 10 client.19365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ef03a6740 20 client.19365 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/484013942 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b62010fb30 0x55b620111f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ef03a6740 4 client.19365 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:13:00.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.511+0000 7f9ef03a6740 1 -- 172.21.15.94:0/484013942 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b61fd43550 con 0x55b62010fb30 2024-02-17T12:13:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.512+0000 7f9ee6ffd640 1 --2- 172.21.15.94:0/484013942 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b62010fb30 0x55b620111f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.512+0000 7f9ece7fc640 10 client.19365 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:00.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.512+0000 7f9ee6ffd640 1 --2- 172.21.15.94:0/484013942 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b62010fb30 0x55b620111f50 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c7b0f1b944fcc17 server_cookie=d6c84be714c09293 in_seq=0 out_seq=0 2024-02-17T12:13:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.531+0000 7f9ece7fc640 1 -- 172.21.15.94:0/484013942 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b61fd43550 con 0x55b62010fb30 2024-02-17T12:13:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.531+0000 7f9ece7fc640 10 client.19365 handle_command_reply: tid=0 2024-02-17T12:13:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.531+0000 7f9ef03a6740 1 client.19365 shutdown 2024-02-17T12:13:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9ef03a6740 1 -- 172.21.15.94:0/484013942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 msgr2=0x55b6200ad6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/484013942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200ad6b0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0010740 tx=0x7f9ed002aa10 comp rx=0 tx=0).stop 2024-02-17T12:13:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9ef03a6740 1 -- 172.21.15.94:0/484013942 shutdown_connections 2024-02-17T12:13:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/484013942 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55b62010fb30 0x55b620111f50 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/484013942 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b6200a82e0 0x55b6200adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/484013942 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55b62009fad0 0x55b6200ad6b0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9ece7fc640 0 client.19365 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9ef03a6740 1 --2- 172.21.15.94:0/484013942 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55b62009ed60 0x55b6200af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9ef03a6740 1 -- 172.21.15.94:0/484013942 >> 172.21.15.94:0/484013942 conn(0x55b620066550 msgr2=0x55b6200a3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9ef03a6740 1 -- 172.21.15.94:0/484013942 shutdown_connections 2024-02-17T12:13:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9ef03a6740 1 -- 172.21.15.94:0/484013942 wait complete. 2024-02-17T12:13:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9ef03a6740 20 client.19365 trim_cache size 0 max 16384 2024-02-17T12:13:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:13:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:13:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:13:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:13:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9eed6f4640 1 -- 172.21.15.94:0/4070708851 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9ed40a2300 msgr2=0x7f9ed40a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9eed6f4640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9ed40a2300 0x7f9ed40a47c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0012960 tx=0x7f9ed000bb70 comp rx=0 tx=0).stop 2024-02-17T12:13:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9eed6f4640 1 -- 172.21.15.94:0/4070708851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee81050c0 msgr2=0x7f9ee80fe6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.532+0000 7f9eed6f4640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee81050c0 0x7f9ee80fe6d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000e960 tx=0x7f9ee000ee20 comp rx=0 tx=0).stop 2024-02-17T12:13:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.533+0000 7f9eed6f4640 1 -- 172.21.15.94:0/4070708851 shutdown_connections 2024-02-17T12:13:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.533+0000 7f9eed6f4640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9ed40a2300 0x7f9ed40a47c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.533+0000 7f9eed6f4640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9ee81050c0 0x7f9ee80fe6d0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.533+0000 7f9eed6f4640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9ee8100d00 0x7f9ee80fe190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.533+0000 7f9eed6f4640 1 --2- 172.21.15.94:0/4070708851 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ee8100360 0x7f9ee80fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.533+0000 7f9eed6f4640 1 -- 172.21.15.94:0/4070708851 >> 172.21.15.94:0/4070708851 conn(0x7f9ee8076e60 msgr2=0x7f9ee8069c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.533+0000 7f9eed6f4640 1 -- 172.21.15.94:0/4070708851 shutdown_connections 2024-02-17T12:13:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:00.533+0000 7f9eed6f4640 1 -- 172.21.15.94:0/4070708851 wait complete. 2024-02-17T12:13:01.403 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:01 smithi189 ceph-mon[36588]: pgmap v466: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.8 MiB/s wr, 560 op/s 2024-02-17T12:13:01.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:01 smithi125 ceph-mon[36819]: pgmap v466: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.8 MiB/s wr, 560 op/s 2024-02-17T12:13:01.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:01 smithi094 ceph-mon[32499]: pgmap v466: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 4.8 MiB/s wr, 560 op/s 2024-02-17T12:13:03.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:03 smithi094 ceph-mon[32499]: osdmap e105: 12 total, 12 up, 12 in 2024-02-17T12:13:03.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:03 smithi094 ceph-mon[32499]: pgmap v468: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 655 op/s 2024-02-17T12:13:03.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:03 smithi189 ceph-mon[36588]: osdmap e105: 12 total, 12 up, 12 in 2024-02-17T12:13:03.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:03 smithi189 ceph-mon[36588]: pgmap v468: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 655 op/s 2024-02-17T12:13:03.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:03 smithi125 ceph-mon[36819]: osdmap e105: 12 total, 12 up, 12 in 2024-02-17T12:13:03.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:03 smithi125 ceph-mon[36819]: pgmap v468: 97 pgs: 97 active+clean; 5.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 6.3 MiB/s wr, 655 op/s 2024-02-17T12:13:05.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 885.0 s, 671.8 tps, lat 146.504 ms stddev 300.657 2024-02-17T12:13:05.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:05 smithi189 ceph-mon[36588]: pgmap v469: 97 pgs: 97 active+clean; 5.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 23 MiB/s wr, 1.81k op/s 2024-02-17T12:13:05.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:05 smithi125 ceph-mon[36819]: pgmap v469: 97 pgs: 97 active+clean; 5.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 23 MiB/s wr, 1.81k op/s 2024-02-17T12:13:05.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:05 smithi094 ceph-mon[32499]: pgmap v469: 97 pgs: 97 active+clean; 5.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 23 MiB/s wr, 1.81k op/s 2024-02-17T12:13:07.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:07 smithi189 ceph-mon[36588]: pgmap v470: 97 pgs: 97 active+clean; 5.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 25 MiB/s wr, 1.93k op/s 2024-02-17T12:13:07.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:07 smithi125 ceph-mon[36819]: pgmap v470: 97 pgs: 97 active+clean; 5.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 25 MiB/s wr, 1.93k op/s 2024-02-17T12:13:07.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:07 smithi094 ceph-mon[32499]: pgmap v470: 97 pgs: 97 active+clean; 5.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 25 MiB/s wr, 1.93k op/s 2024-02-17T12:13:09.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:09 smithi189 ceph-mon[36588]: pgmap v471: 97 pgs: 97 active+clean; 5.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 26 MiB/s wr, 1.99k op/s 2024-02-17T12:13:09.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:09 smithi125 ceph-mon[36819]: pgmap v471: 97 pgs: 97 active+clean; 5.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 26 MiB/s wr, 1.99k op/s 2024-02-17T12:13:09.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:09 smithi094 ceph-mon[32499]: pgmap v471: 97 pgs: 97 active+clean; 5.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 26 MiB/s wr, 1.99k op/s 2024-02-17T12:13:10.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 890.0 s, 1162.0 tps, lat 87.053 ms stddev 100.101 2024-02-17T12:13:11.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:11 smithi189 ceph-mon[36588]: osdmap e106: 12 total, 12 up, 12 in 2024-02-17T12:13:11.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:11 smithi189 ceph-mon[36588]: pgmap v473: 97 pgs: 97 active+clean; 5.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 29 MiB/s wr, 2.20k op/s 2024-02-17T12:13:11.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:11 smithi125 ceph-mon[36819]: osdmap e106: 12 total, 12 up, 12 in 2024-02-17T12:13:11.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:11 smithi125 ceph-mon[36819]: pgmap v473: 97 pgs: 97 active+clean; 5.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 29 MiB/s wr, 2.20k op/s 2024-02-17T12:13:11.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:11 smithi094 ceph-mon[32499]: osdmap e106: 12 total, 12 up, 12 in 2024-02-17T12:13:11.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:11 smithi094 ceph-mon[32499]: pgmap v473: 97 pgs: 97 active+clean; 5.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 29 MiB/s wr, 2.20k op/s 2024-02-17T12:13:13.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:13 smithi189 ceph-mon[36588]: pgmap v474: 97 pgs: 97 active+clean; 5.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 30 MiB/s wr, 2.15k op/s 2024-02-17T12:13:13.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:13 smithi125 ceph-mon[36819]: pgmap v474: 97 pgs: 97 active+clean; 5.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 30 MiB/s wr, 2.15k op/s 2024-02-17T12:13:13.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:13 smithi094 ceph-mon[32499]: pgmap v474: 97 pgs: 97 active+clean; 5.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 30 MiB/s wr, 2.15k op/s 2024-02-17T12:13:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.085+0000 7f4ff52ca640 1 -- 172.21.15.94:0/2537966070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fe8004c80 msgr2=0x7f4fe8005100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.085+0000 7f4ff52ca640 1 --2- 172.21.15.94:0/2537966070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fe8004c80 0x7f4fe8005100 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009860 tx=0x7f4fdc02bbb0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.088+0000 7f4ff52ca640 1 -- 172.21.15.94:0/2537966070 shutdown_connections 2024-02-17T12:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.088+0000 7f4ff52ca640 1 --2- 172.21.15.94:0/2537966070 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fe80986e0 0x7f4fe8098b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.088+0000 7f4ff52ca640 1 --2- 172.21.15.94:0/2537966070 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fe8004c80 0x7f4fe8005100 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.088+0000 7f4ff52ca640 1 --2- 172.21.15.94:0/2537966070 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fe809acf0 0x7f4fe8004740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.088+0000 7f4ff52ca640 1 -- 172.21.15.94:0/2537966070 >> 172.21.15.94:0/2537966070 conn(0x7f4fe800a740 msgr2=0x7f4fe800ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.090+0000 7f4ff52ca640 1 -- 172.21.15.94:0/2537966070 shutdown_connections 2024-02-17T12:13:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.091+0000 7f4ff52ca640 1 -- 172.21.15.94:0/2537966070 wait complete. 2024-02-17T12:13:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.093+0000 7f4ff52ca640 1 Processor -- start 2024-02-17T12:13:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.093+0000 7f4ff52ca640 1 -- start start 2024-02-17T12:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.093+0000 7f4ff52ca640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fe8004c80 0x7f4fe80a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.094+0000 7f4ff52ca640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fe80986e0 0x7f4fe80a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.094+0000 7f4ff52ca640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fe809acf0 0x7f4fe80a2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.094+0000 7f4ff52ca640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80a2a10 con 0x7f4fe809acf0 2024-02-17T12:13:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.094+0000 7f4ff52ca640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80a2b80 con 0x7f4fe80986e0 2024-02-17T12:13:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.094+0000 7f4ff52ca640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80a2d40 con 0x7f4fe8004c80 2024-02-17T12:13:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.095+0000 7f4feffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fe8004c80 0x7f4fe80a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.095+0000 7f4feffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fe8004c80 0x7f4fe80a9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:57272/0 (socket says 172.21.15.94:57272) 2024-02-17T12:13:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.095+0000 7f4feffff640 1 -- 172.21.15.94:0/3356767486 learned_addr learned my addr 172.21.15.94:0/3356767486 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.095+0000 7f4fef7fe640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fe80986e0 0x7f4fe80a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.096+0000 7f4ff4ac9640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fe809acf0 0x7f4fe80a2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.096+0000 7f4ff4ac9640 1 -- 172.21.15.94:0/3356767486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fe8004c80 msgr2=0x7f4fe80a9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.096+0000 7f4ff4ac9640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fe8004c80 0x7f4fe80a9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.096+0000 7f4ff4ac9640 1 -- 172.21.15.94:0/3356767486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fe80986e0 msgr2=0x7f4fe80a9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.096+0000 7f4ff4ac9640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fe80986e0 0x7f4fe80a9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.096+0000 7f4ff4ac9640 1 -- 172.21.15.94:0/3356767486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe4009560 con 0x7f4fe809acf0 2024-02-17T12:13:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.097+0000 7f4ff4ac9640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fe809acf0 0x7f4fe80a2280 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4ff004f1b0 tx=0x7f4ff006d4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.097+0000 7f4fed7fa640 1 -- 172.21.15.94:0/3356767486 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff007b070 con 0x7f4fe809acf0 2024-02-17T12:13:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.097+0000 7f4fed7fa640 1 -- 172.21.15.94:0/3356767486 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4ff006db90 con 0x7f4fe809acf0 2024-02-17T12:13:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.097+0000 7f4fed7fa640 1 -- 172.21.15.94:0/3356767486 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00725e0 con 0x7f4fe809acf0 2024-02-17T12:13:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.097+0000 7f4ff52ca640 1 -- 172.21.15.94:0/3356767486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fdc009490 con 0x7f4fe809acf0 2024-02-17T12:13:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.097+0000 7f4ff52ca640 1 -- 172.21.15.94:0/3356767486 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe80a6e00 con 0x7f4fe809acf0 2024-02-17T12:13:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.101+0000 7f4fed7fa640 1 -- 172.21.15.94:0/3356767486 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f4ff006dd00 con 0x7f4fe809acf0 2024-02-17T12:13:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.102+0000 7f4fed7fa640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4fe00a22b0 0x7f4fe00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.102+0000 7f4fed7fa640 1 -- 172.21.15.94:0/3356767486 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4ff0130430 con 0x7f4fe809acf0 2024-02-17T12:13:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.102+0000 7f4feffff640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4fe00a22b0 0x7f4fe00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.103+0000 7f4feffff640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4fe00a22b0 0x7f4fe00a4770 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4005c80 tx=0x7f4fe4009260 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:14.154 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.150+0000 7f4ff91da740 1 Processor -- start 2024-02-17T12:13:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.150+0000 7f4ff91da740 1 -- start start 2024-02-17T12:13:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.150+0000 7f4ff91da740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0dc60 0x55578de0c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.150+0000 7f4ff91da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0cef0 0x55578de0d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.150+0000 7f4ff91da740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de16470 0x55578de18860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.150+0000 7f4ff91da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578dd7b9f0 con 0x55578de0cef0 2024-02-17T12:13:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.150+0000 7f4ff91da740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578dd78f20 con 0x55578de0dc60 2024-02-17T12:13:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.150+0000 7f4ff91da740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578dd74700 con 0x55578de16470 2024-02-17T12:13:14.155 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.150+0000 7f4fef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0cef0 0x55578de0d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.150+0000 7f4fef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0cef0 0x55578de0d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51520/0 (socket says 172.21.15.94:51520) 2024-02-17T12:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.150+0000 7f4fef7fe640 1 -- 172.21.15.94:0/1289030725 learned_addr learned my addr 172.21.15.94:0/1289030725 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.150+0000 7f4feffff640 1 --2- 172.21.15.94:0/1289030725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0dc60 0x55578de0c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.151+0000 7f4ff4ac9640 1 --2- 172.21.15.94:0/1289030725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de16470 0x55578de18860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.151+0000 7f4fef7fe640 1 -- 172.21.15.94:0/1289030725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de16470 msgr2=0x55578de18860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.151+0000 7f4fef7fe640 1 --2- 172.21.15.94:0/1289030725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de16470 0x55578de18860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.151+0000 7f4fef7fe640 1 -- 172.21.15.94:0/1289030725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0dc60 msgr2=0x55578de0c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.151+0000 7f4fef7fe640 1 --2- 172.21.15.94:0/1289030725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0dc60 0x55578de0c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.151+0000 7f4fef7fe640 1 -- 172.21.15.94:0/1289030725 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55578dd0a7b0 con 0x55578de0cef0 2024-02-17T12:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.151+0000 7f4fef7fe640 1 --2- 172.21.15.94:0/1289030725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0cef0 0x55578de0d370 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009830 tx=0x7f4fdc032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db78c2223c49a05 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:14.156 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.151+0000 7f4fda7fc640 1 -- 172.21.15.94:0/1289030725 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc004400 con 0x55578de0cef0 2024-02-17T12:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.151+0000 7f4fda7fc640 1 -- 172.21.15.94:0/1289030725 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fdc005510 con 0x55578de0cef0 2024-02-17T12:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.151+0000 7f4fda7fc640 1 -- 172.21.15.94:0/1289030725 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc02c7b0 con 0x55578de0cef0 2024-02-17T12:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.152+0000 7f4ff91da740 1 -- 172.21.15.94:0/1289030725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0cef0 msgr2=0x55578de0d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.152+0000 7f4ff91da740 1 --2- 172.21.15.94:0/1289030725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0cef0 0x55578de0d370 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009830 tx=0x7f4fdc032430 comp rx=0 tx=0).stop 2024-02-17T12:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.152+0000 7f4ff91da740 1 -- 172.21.15.94:0/1289030725 shutdown_connections 2024-02-17T12:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.152+0000 7f4ff91da740 1 --2- 172.21.15.94:0/1289030725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de16470 0x55578de18860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.152+0000 7f4ff91da740 1 --2- 172.21.15.94:0/1289030725 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0cef0 0x55578de0d370 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.152+0000 7f4ff91da740 1 --2- 172.21.15.94:0/1289030725 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0dc60 0x55578de0c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.152+0000 7f4ff91da740 1 -- 172.21.15.94:0/1289030725 >> 172.21.15.94:0/1289030725 conn(0x55578ddd4550 msgr2=0x55578de07be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.152+0000 7f4ff91da740 1 -- 172.21.15.94:0/1289030725 shutdown_connections 2024-02-17T12:13:14.157 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.152+0000 7f4ff91da740 1 -- 172.21.15.94:0/1289030725 wait complete. 2024-02-17T12:13:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.163+0000 7f4ff91da740 1 -- start start 2024-02-17T12:13:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.164+0000 7f4ff91da740 1 Processor -- start 2024-02-17T12:13:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.164+0000 7f4ff91da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0cef0 0x55578de133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.168 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.164+0000 7f4ff91da740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0dc60 0x55578de13920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.164+0000 7f4ff91da740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de16470 0x55578de13e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.164+0000 7f4ff91da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578dd19bf0 con 0x55578de0cef0 2024-02-17T12:13:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.164+0000 7f4ff91da740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578da48250 con 0x55578de0dc60 2024-02-17T12:13:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.164+0000 7f4ff91da740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578dcfa200 con 0x55578de16470 2024-02-17T12:13:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.164+0000 7f4feffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0cef0 0x55578de133e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.164+0000 7f4feffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0cef0 0x55578de133e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51536/0 (socket says 172.21.15.94:51536) 2024-02-17T12:13:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.164+0000 7f4fef7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0dc60 0x55578de13920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.164+0000 7f4feffff640 1 -- 172.21.15.94:0/2085772980 learned_addr learned my addr 172.21.15.94:0/2085772980 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.164+0000 7f4ff4ac9640 1 --2- 172.21.15.94:0/2085772980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de16470 0x55578de13e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.165+0000 7f4ff91da740 10 client.0 _ll_register_callbacks cb 0x55578dd68d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.165+0000 7f4fef7fe640 1 -- 172.21.15.94:0/2085772980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de16470 msgr2=0x55578de13e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.165+0000 7f4fef7fe640 1 --2- 172.21.15.94:0/2085772980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de16470 0x55578de13e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.165+0000 7f4fef7fe640 1 -- 172.21.15.94:0/2085772980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0cef0 msgr2=0x55578de133e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.165+0000 7f4fef7fe640 1 --2- 172.21.15.94:0/2085772980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0cef0 0x55578de133e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.165+0000 7f4fef7fe640 1 -- 172.21.15.94:0/2085772980 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe80a9cb0 con 0x55578de0dc60 2024-02-17T12:13:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.165+0000 7f4fef7fe640 1 --2- 172.21.15.94:0/2085772980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0dc60 0x55578de13920 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc02c1c0 tx=0x7f4fdc034980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=548ceb820d40f6e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.165+0000 7f4fcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:13:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.165+0000 7f4fcf7fe640 1 -- 172.21.15.94:0/2085772980 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc04d070 con 0x55578de0dc60 2024-02-17T12:13:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.165+0000 7f4fcf7fe640 1 -- 172.21.15.94:0/2085772980 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4fdc02c760 con 0x55578de0dc60 2024-02-17T12:13:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.165+0000 7f4ff91da740 1 -- 172.21.15.94:0/2085772980 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55578da74600 con 0x55578de0dc60 2024-02-17T12:13:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.166+0000 7f4fcf7fe640 1 -- 172.21.15.94:0/2085772980 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fdc0347f0 con 0x55578de0dc60 2024-02-17T12:13:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.166+0000 7f4ff91da740 10 client.28826 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:14.170 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.166+0000 7f4ff91da740 1 -- 172.21.15.94:0/2085772980 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({fsmap=27}) v3 -- 0x55578dd72d80 con 0x55578de0dc60 2024-02-17T12:13:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.168+0000 7f4fcf7fe640 1 -- 172.21.15.94:0/2085772980 <== mon.2 v2:172.21.15.125:3300/0 4 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4fdc02c960 con 0x55578de0dc60 2024-02-17T12:13:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.168+0000 7f4ff91da740 10 client.28826 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.168+0000 7f4ff91da740 10 client.28826 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.168+0000 7f4ff91da740 20 client.28826 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.168+0000 7f4ff91da740 1 --2- 172.21.15.94:0/2085772980 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55578de26180 0x55578de285b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.168+0000 7f4ff91da740 4 client.28826 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:13:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.168+0000 7f4ff91da740 1 -- 172.21.15.94:0/2085772980 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55578dac51b0 con 0x55578de26180 2024-02-17T12:13:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.168+0000 7f4feffff640 1 --2- 172.21.15.94:0/2085772980 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55578de26180 0x55578de285b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.169+0000 7f4feffff640 1 --2- 172.21.15.94:0/2085772980 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55578de26180 0x55578de285b0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfd89a174b796345 server_cookie=cac9585da66ec406 in_seq=0 out_seq=0 2024-02-17T12:13:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.169+0000 7f4fcf7fe640 10 client.28826 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.169+0000 7f4fcf7fe640 1 -- 172.21.15.94:0/2085772980 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55578dac51b0 con 0x55578de26180 2024-02-17T12:13:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.169+0000 7f4fcf7fe640 10 client.28826 handle_command_reply: tid=0 2024-02-17T12:13:14.174 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.169+0000 7f4ff91da740 1 client.28826 shutdown 2024-02-17T12:13:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.170+0000 7f4ff91da740 1 -- 172.21.15.94:0/2085772980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0dc60 msgr2=0x55578de13920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.170+0000 7f4ff91da740 1 --2- 172.21.15.94:0/2085772980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0dc60 0x55578de13920 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc02c1c0 tx=0x7f4fdc034980 comp rx=0 tx=0).stop 2024-02-17T12:13:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.170+0000 7f4ff91da740 1 -- 172.21.15.94:0/2085772980 shutdown_connections 2024-02-17T12:13:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.170+0000 7f4ff91da740 1 --2- 172.21.15.94:0/2085772980 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55578de26180 0x55578de285b0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.170+0000 7f4ff91da740 1 --2- 172.21.15.94:0/2085772980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de16470 0x55578de13e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.170+0000 7f4ff91da740 1 --2- 172.21.15.94:0/2085772980 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0dc60 0x55578de13920 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.170+0000 7f4ff91da740 1 --2- 172.21.15.94:0/2085772980 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0cef0 0x55578de133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.170+0000 7f4ff91da740 1 -- 172.21.15.94:0/2085772980 >> 172.21.15.94:0/2085772980 conn(0x55578ddd4550 msgr2=0x55578de09930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.170+0000 7f4fcf7fe640 0 client.28826 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.170+0000 7f4ff91da740 1 -- 172.21.15.94:0/2085772980 shutdown_connections 2024-02-17T12:13:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.170+0000 7f4ff91da740 1 -- 172.21.15.94:0/2085772980 wait complete. 2024-02-17T12:13:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.170+0000 7f4ff91da740 20 client.28826 trim_cache size 0 max 16384 2024-02-17T12:13:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.190+0000 7f4ff91da740 1 Processor -- start 2024-02-17T12:13:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.190+0000 7f4ff91da740 1 -- start start 2024-02-17T12:13:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.190+0000 7f4ff91da740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0cef0 0x55578de15b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.190+0000 7f4ff91da740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de0dc60 0x55578de13c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4ff91da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de16470 0x55578de1d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4ff91da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578de19080 con 0x55578de16470 2024-02-17T12:13:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4ff91da740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578de23220 con 0x55578de0cef0 2024-02-17T12:13:14.194 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4ff91da740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578de221e0 con 0x55578de0dc60 2024-02-17T12:13:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4feffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0cef0 0x55578de15b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4ff4ac9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de16470 0x55578de1d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4feffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0cef0 0x55578de15b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:48534/0 (socket says 172.21.15.94:48534) 2024-02-17T12:13:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4feffff640 1 -- 172.21.15.94:0/621390158 learned_addr learned my addr 172.21.15.94:0/621390158 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4fef7fe640 1 --2- 172.21.15.94:0/621390158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de0dc60 0x55578de13c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4feffff640 1 -- 172.21.15.94:0/621390158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de0dc60 msgr2=0x55578de13c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4feffff640 1 --2- 172.21.15.94:0/621390158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de0dc60 0x55578de13c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4feffff640 1 -- 172.21.15.94:0/621390158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de16470 msgr2=0x55578de1d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4feffff640 1 --2- 172.21.15.94:0/621390158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de16470 0x55578de1d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.195 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4feffff640 1 -- 172.21.15.94:0/621390158 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55578de18d10 con 0x55578de0cef0 2024-02-17T12:13:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4ff4ac9640 1 --2- 172.21.15.94:0/621390158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de16470 0x55578de1d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4fef7fe640 1 --2- 172.21.15.94:0/621390158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de0dc60 0x55578de13c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.191+0000 7f4feffff640 1 --2- 172.21.15.94:0/621390158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0cef0 0x55578de15b10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40094d0 tx=0x7f4fe4013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfdfb836f1f4c77 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.192+0000 7f4fd9ffb640 1 -- 172.21.15.94:0/621390158 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4024040 con 0x55578de0cef0 2024-02-17T12:13:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.192+0000 7f4fd9ffb640 1 -- 172.21.15.94:0/621390158 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fe4015070 con 0x55578de0cef0 2024-02-17T12:13:14.196 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.192+0000 7f4fd9ffb640 1 -- 172.21.15.94:0/621390158 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe401e420 con 0x55578de0cef0 2024-02-17T12:13:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.192+0000 7f4ff91da740 1 -- 172.21.15.94:0/621390158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0cef0 msgr2=0x55578de15b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.192+0000 7f4ff91da740 1 --2- 172.21.15.94:0/621390158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0cef0 0x55578de15b10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40094d0 tx=0x7f4fe4013000 comp rx=0 tx=0).stop 2024-02-17T12:13:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.192+0000 7f4ff91da740 1 -- 172.21.15.94:0/621390158 shutdown_connections 2024-02-17T12:13:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.192+0000 7f4ff91da740 1 --2- 172.21.15.94:0/621390158 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de16470 0x55578de1d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.192+0000 7f4ff91da740 1 --2- 172.21.15.94:0/621390158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de0dc60 0x55578de13c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.192+0000 7f4ff91da740 1 --2- 172.21.15.94:0/621390158 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de0cef0 0x55578de15b10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.192+0000 7f4ff91da740 1 -- 172.21.15.94:0/621390158 >> 172.21.15.94:0/621390158 conn(0x55578ddd4550 msgr2=0x55578de0c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4ff91da740 1 -- 172.21.15.94:0/621390158 shutdown_connections 2024-02-17T12:13:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4ff91da740 1 -- 172.21.15.94:0/621390158 wait complete. 2024-02-17T12:13:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4ff91da740 1 -- start start 2024-02-17T12:13:14.200 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4ff91da740 1 Processor -- start 2024-02-17T12:13:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4ff91da740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de0cef0 0x55578de1d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4ff91da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0dc60 0x55578de0fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4ff91da740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de16470 0x55578de10320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4ff91da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578de10a20 con 0x55578de0dc60 2024-02-17T12:13:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4ff91da740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578de10b90 con 0x55578de16470 2024-02-17T12:13:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4ff91da740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55578de7c6d0 con 0x55578de0cef0 2024-02-17T12:13:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4ff91da740 10 client.0 _ll_register_callbacks cb 0x7f4fe4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4fef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0dc60 0x55578de0fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4ff4ac9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de16470 0x55578de10320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.201 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4fef7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0dc60 0x55578de0fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:51562/0 (socket says 172.21.15.94:51562) 2024-02-17T12:13:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4fef7fe640 1 -- 172.21.15.94:0/2218784326 learned_addr learned my addr 172.21.15.94:0/2218784326 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.193+0000 7f4feffff640 1 --2- 172.21.15.94:0/2218784326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de0cef0 0x55578de1d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4ff4ac9640 1 -- 172.21.15.94:0/2218784326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de0cef0 msgr2=0x55578de1d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4ff4ac9640 1 --2- 172.21.15.94:0/2218784326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de0cef0 0x55578de1d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4ff4ac9640 1 -- 172.21.15.94:0/2218784326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0dc60 msgr2=0x55578de0fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4ff4ac9640 1 --2- 172.21.15.94:0/2218784326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0dc60 0x55578de0fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4ff4ac9640 1 -- 172.21.15.94:0/2218784326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe80a6e00 con 0x55578de16470 2024-02-17T12:13:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4fef7fe640 1 --2- 172.21.15.94:0/2218784326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0dc60 0x55578de0fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4ff4ac9640 1 --2- 172.21.15.94:0/2218784326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de16470 0x55578de10320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4ff012dc50 tx=0x7f4ff00fc8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93e4f22d681dace6 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4fcf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:13:14.202 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4fcf7fe640 1 -- 172.21.15.94:0/2218784326 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff01010d0 con 0x55578de16470 2024-02-17T12:13:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4fcf7fe640 1 -- 172.21.15.94:0/2218784326 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f4ff0105690 con 0x55578de16470 2024-02-17T12:13:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4ff91da740 1 -- 172.21.15.94:0/2218784326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4fdc005510 con 0x55578de16470 2024-02-17T12:13:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4fcf7fe640 1 -- 172.21.15.94:0/2218784326 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ff011b260 con 0x55578de16470 2024-02-17T12:13:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4ff91da740 10 client.28832 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4ff91da740 1 -- 172.21.15.94:0/2218784326 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f4fe40115c0 con 0x55578de16470 2024-02-17T12:13:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.194+0000 7f4fcf7fe640 1 -- 172.21.15.94:0/2218784326 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4fdc005510 con 0x55578de16470 2024-02-17T12:13:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.195+0000 7f4fcf7fe640 1 -- 172.21.15.94:0/2218784326 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f4ff00ff040 con 0x55578de16470 2024-02-17T12:13:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.195+0000 7f4ff91da740 10 client.28832 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.195+0000 7f4ff91da740 10 client.28832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.195+0000 7f4ff91da740 20 client.28832 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.195+0000 7f4ff91da740 1 --2- 172.21.15.94:0/2218784326 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55578de7de20 0x55578de80210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.195+0000 7f4ff91da740 4 client.28832 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:13:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.195+0000 7f4ff91da740 1 -- 172.21.15.94:0/2218784326 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55578dab1550 con 0x55578de7de20 2024-02-17T12:13:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.195+0000 7f4feffff640 1 --2- 172.21.15.94:0/2218784326 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55578de7de20 0x55578de80210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.195+0000 7f4fcf7fe640 10 client.28832 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.195+0000 7f4feffff640 1 --2- 172.21.15.94:0/2218784326 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55578de7de20 0x55578de80210 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be531a860a9b4ecc server_cookie=e3b1997532e0f683 in_seq=0 out_seq=0 2024-02-17T12:13:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4fcf7fe640 1 -- 172.21.15.94:0/2218784326 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55578dab1550 con 0x55578de7de20 2024-02-17T12:13:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4fcf7fe640 10 client.28832 handle_command_reply: tid=0 2024-02-17T12:13:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4ff91da740 1 client.28832 shutdown 2024-02-17T12:13:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4ff91da740 1 -- 172.21.15.94:0/2218784326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de16470 msgr2=0x55578de10320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4ff91da740 1 --2- 172.21.15.94:0/2218784326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de16470 0x55578de10320 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4ff012dc50 tx=0x7f4ff00fc8e0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4ff91da740 1 -- 172.21.15.94:0/2218784326 shutdown_connections 2024-02-17T12:13:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4ff91da740 1 --2- 172.21.15.94:0/2218784326 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55578de7de20 0x55578de80210 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4ff91da740 1 --2- 172.21.15.94:0/2218784326 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55578de16470 0x55578de10320 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4fcf7fe640 0 client.28832 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4ff91da740 1 --2- 172.21.15.94:0/2218784326 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55578de0dc60 0x55578de0fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4ff91da740 1 --2- 172.21.15.94:0/2218784326 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55578de0cef0 0x55578de1d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4ff91da740 1 -- 172.21.15.94:0/2218784326 >> 172.21.15.94:0/2218784326 conn(0x55578ddd4550 msgr2=0x55578de11b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4ff91da740 1 -- 172.21.15.94:0/2218784326 shutdown_connections 2024-02-17T12:13:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4ff91da740 1 -- 172.21.15.94:0/2218784326 wait complete. 2024-02-17T12:13:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.214+0000 7f4ff91da740 20 client.28832 trim_cache size 0 max 16384 2024-02-17T12:13:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:13:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:13:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:13:14.218 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:13:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.215+0000 7f4ff52ca640 1 -- 172.21.15.94:0/3356767486 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4fe00a22b0 msgr2=0x7f4fe00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.215+0000 7f4ff52ca640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4fe00a22b0 0x7f4fe00a4770 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4005c80 tx=0x7f4fe4009260 comp rx=0 tx=0).stop 2024-02-17T12:13:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.215+0000 7f4ff52ca640 1 -- 172.21.15.94:0/3356767486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fe809acf0 msgr2=0x7f4fe80a2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.215+0000 7f4ff52ca640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fe809acf0 0x7f4fe80a2280 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4ff004f1b0 tx=0x7f4ff006d4b0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.215+0000 7f4ff52ca640 1 -- 172.21.15.94:0/3356767486 shutdown_connections 2024-02-17T12:13:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.215+0000 7f4ff52ca640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f4fe00a22b0 0x7f4fe00a4770 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.215+0000 7f4ff52ca640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f4fe809acf0 0x7f4fe80a2280 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.215+0000 7f4ff52ca640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f4fe80986e0 0x7f4fe80a9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.215+0000 7f4ff52ca640 1 --2- 172.21.15.94:0/3356767486 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4fe8004c80 0x7f4fe80a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.215+0000 7f4ff52ca640 1 -- 172.21.15.94:0/3356767486 >> 172.21.15.94:0/3356767486 conn(0x7f4fe800a740 msgr2=0x7f4fe8092320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.215+0000 7f4ff52ca640 1 -- 172.21.15.94:0/3356767486 shutdown_connections 2024-02-17T12:13:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:14.215+0000 7f4ff52ca640 1 -- 172.21.15.94:0/3356767486 wait complete. 2024-02-17T12:13:15.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 895.0 s, 1218.0 tps, lat 82.351 ms stddev 89.009 2024-02-17T12:13:15.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:15 smithi125 ceph-mon[36819]: pgmap v475: 97 pgs: 97 active+clean; 5.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 881 op/s 2024-02-17T12:13:15.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:15 smithi094 ceph-mon[32499]: pgmap v475: 97 pgs: 97 active+clean; 5.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 881 op/s 2024-02-17T12:13:15.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:15 smithi189 ceph-mon[36588]: pgmap v475: 97 pgs: 97 active+clean; 5.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 14 MiB/s wr, 881 op/s 2024-02-17T12:13:15.660 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:13:15.661 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:13:15.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.927+0000 7f0ae660e640 1 -- 172.21.15.94:0/3421821388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae00694e0 msgr2=0x7f0ae0069940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:15.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.927+0000 7f0ae660e640 1 --2- 172.21.15.94:0/3421821388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae00694e0 0x7f0ae0069940 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009930 tx=0x7f0ad402bc30 comp rx=0 tx=0).stop 2024-02-17T12:13:15.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.927+0000 7f0ae660e640 1 -- 172.21.15.94:0/3421821388 shutdown_connections 2024-02-17T12:13:15.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.927+0000 7f0ae660e640 1 --2- 172.21.15.94:0/3421821388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ae0069e80 0x7f0ae010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.927+0000 7f0ae660e640 1 --2- 172.21.15.94:0/3421821388 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae00694e0 0x7f0ae0069940 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.927+0000 7f0ae660e640 1 --2- 172.21.15.94:0/3421821388 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ae01071b0 0x7f0ae0107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.927+0000 7f0ae660e640 1 -- 172.21.15.94:0/3421821388 >> 172.21.15.94:0/3421821388 conn(0x7f0ae0076ee0 msgr2=0x7f0ae00772f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.927+0000 7f0ae660e640 1 -- 172.21.15.94:0/3421821388 shutdown_connections 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.927+0000 7f0ae660e640 1 -- 172.21.15.94:0/3421821388 wait complete. 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.928+0000 7f0ae660e640 1 Processor -- start 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.928+0000 7f0ae660e640 1 -- start start 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.928+0000 7f0ae660e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ae00694e0 0x7f0ae019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.928+0000 7f0ae660e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0069e80 0x7f0ae019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.928+0000 7f0ae660e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ae01071b0 0x7f0ae01975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.928+0000 7f0ae660e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0197c20 con 0x7f0ae0069e80 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.928+0000 7f0ae660e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0197d90 con 0x7f0ae01071b0 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.928+0000 7f0ae660e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae0197f50 con 0x7f0ae00694e0 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0adf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0069e80 0x7f0ae019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:15.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0adffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ae00694e0 0x7f0ae019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:15.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0adffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ae00694e0 0x7f0ae019d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:45796/0 (socket says 172.21.15.94:45796) 2024-02-17T12:13:15.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0ae4bab640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ae01071b0 0x7f0ae01975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:15.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0adffff640 1 -- 172.21.15.94:0/1743426811 learned_addr learned my addr 172.21.15.94:0/1743426811 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:15.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0adf7fe640 1 -- 172.21.15.94:0/1743426811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ae00694e0 msgr2=0x7f0ae019d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:15.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0adf7fe640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ae00694e0 0x7f0ae019d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0adf7fe640 1 -- 172.21.15.94:0/1743426811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ae01071b0 msgr2=0x7f0ae01975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:15.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0adf7fe640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ae01071b0 0x7f0ae01975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0adf7fe640 1 -- 172.21.15.94:0/1743426811 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad4009560 con 0x7f0ae0069e80 2024-02-17T12:13:15.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0ae4bab640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ae01071b0 0x7f0ae01975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:15.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0adffff640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ae00694e0 0x7f0ae019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:15.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0adf7fe640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0069e80 0x7f0ae019d960 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009230 tx=0x7f0ad402c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:15.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.929+0000 7f0add7fa640 1 -- 172.21.15.94:0/1743426811 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40042d0 con 0x7f0ae0069e80 2024-02-17T12:13:15.934 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.930+0000 7f0add7fa640 1 -- 172.21.15.94:0/1743426811 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ad4004430 con 0x7f0ae0069e80 2024-02-17T12:13:15.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.930+0000 7f0add7fa640 1 -- 172.21.15.94:0/1743426811 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4003bd0 con 0x7f0ae0069e80 2024-02-17T12:13:15.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.930+0000 7f0ae660e640 1 -- 172.21.15.94:0/1743426811 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae01981d0 con 0x7f0ae0069e80 2024-02-17T12:13:15.935 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.930+0000 7f0ae660e640 1 -- 172.21.15.94:0/1743426811 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae01004b0 con 0x7f0ae0069e80 2024-02-17T12:13:15.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.933+0000 7f0ae91e9740 1 Processor -- start 2024-02-17T12:13:15.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.933+0000 7f0ae91e9740 1 -- start start 2024-02-17T12:13:15.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.933+0000 7f0ae91e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb53b0 0x56205ffb5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0ae91e9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 0x56205ffbe560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0ae91e9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffbeaa0 0x56205ffc0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0ae91e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205ff24280 con 0x56205ffb53b0 2024-02-17T12:13:15.937 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0ae91e9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205ff217b0 con 0x56205ffb5cd0 2024-02-17T12:13:15.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0ae91e9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205ff1cf90 con 0x56205ffbeaa0 2024-02-17T12:13:15.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0add7fa640 1 -- 172.21.15.94:0/1743426811 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f0ad4003d30 con 0x7f0ae0069e80 2024-02-17T12:13:15.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0ae4bab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb53b0 0x56205ffb5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:15.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0adffff640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 0x56205ffbe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:15.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0ae4bab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb53b0 0x56205ffb5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56110/0 (socket says 172.21.15.94:56110) 2024-02-17T12:13:15.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0ae4bab640 1 -- 172.21.15.94:0/3196148251 learned_addr learned my addr 172.21.15.94:0/3196148251 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:15.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0adf7fe640 1 --2- 172.21.15.94:0/3196148251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffbeaa0 0x56205ffc0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:15.938 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0adffff640 1 -- 172.21.15.94:0/3196148251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffbeaa0 msgr2=0x56205ffc0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:15.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0adffff640 1 --2- 172.21.15.94:0/3196148251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffbeaa0 0x56205ffc0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0adffff640 1 -- 172.21.15.94:0/3196148251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb53b0 msgr2=0x56205ffb5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:15.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0adffff640 1 --2- 172.21.15.94:0/3196148251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb53b0 0x56205ffb5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0adffff640 1 -- 172.21.15.94:0/3196148251 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56205fedb570 con 0x56205ffb5cd0 2024-02-17T12:13:15.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0ae4bab640 1 --2- 172.21.15.94:0/3196148251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb53b0 0x56205ffb5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:15.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0adffff640 1 --2- 172.21.15.94:0/3196148251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 0x56205ffbe560 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009670 tx=0x7f0ad0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e592e7e48938308 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:15.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0add7fa640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0ac80a2230 0x7f0ac80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.939 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.934+0000 7f0ac5ffb640 1 -- 172.21.15.94:0/3196148251 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0015050 con 0x56205ffb5cd0 2024-02-17T12:13:15.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.935+0000 7f0ac5ffb640 1 -- 172.21.15.94:0/3196148251 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ad0016040 con 0x56205ffb5cd0 2024-02-17T12:13:15.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.935+0000 7f0ae4bab640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0ac80a2230 0x7f0ac80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:15.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.935+0000 7f0ac5ffb640 1 -- 172.21.15.94:0/3196148251 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0003900 con 0x56205ffb5cd0 2024-02-17T12:13:15.940 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.935+0000 7f0add7fa640 1 -- 172.21.15.94:0/1743426811 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0ad40eff70 con 0x7f0ae0069e80 2024-02-17T12:13:15.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.935+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3196148251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 msgr2=0x56205ffbe560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:15.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.935+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3196148251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 0x56205ffbe560 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0009670 tx=0x7f0ad0005f90 comp rx=0 tx=0).stop 2024-02-17T12:13:15.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.936+0000 7f0ae4bab640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0ac80a2230 0x7f0ac80a46f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x56205ffb4ad0 tx=0x7f0ad8009500 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:15.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.936+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3196148251 shutdown_connections 2024-02-17T12:13:15.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.936+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3196148251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffbeaa0 0x56205ffc0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.936+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3196148251 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 0x56205ffbe560 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.936+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3196148251 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb53b0 0x56205ffb5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.936+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3196148251 >> 172.21.15.94:0/3196148251 conn(0x56205ffafcd0 msgr2=0x56205ffb00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:15.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.936+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3196148251 shutdown_connections 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.936+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3196148251 wait complete. 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0ae91e9740 1 -- start start 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0ae91e9740 1 Processor -- start 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0ae91e9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffb53b0 0x56205ffbe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0ae91e9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 0x56205ffbb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0ae91e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffbeaa0 0x56205ffbbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0ae91e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205fbf0430 con 0x56205ffbeaa0 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0ae91e9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205fbace00 con 0x56205ffb5cd0 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0ae91e9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205fea1fd0 con 0x56205ffb53b0 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0ae91e9740 10 client.0 _ll_register_callbacks cb 0x56205fef7fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0adffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffb53b0 0x56205ffbe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0ae4bab640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffbeaa0 0x56205ffbbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0adffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffb53b0 0x56205ffbe400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:45810/0 (socket says 172.21.15.94:45810) 2024-02-17T12:13:15.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0adffff640 1 -- 172.21.15.94:0/3057788463 learned_addr learned my addr 172.21.15.94:0/3057788463 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.937+0000 7f0adf7fe640 1 --2- 172.21.15.94:0/3057788463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 0x56205ffbb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0ae4bab640 1 -- 172.21.15.94:0/3057788463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffb53b0 msgr2=0x56205ffbe400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0ae4bab640 1 --2- 172.21.15.94:0/3057788463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffb53b0 0x56205ffbe400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0ae4bab640 1 -- 172.21.15.94:0/3057788463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 msgr2=0x56205ffbb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0ae4bab640 1 --2- 172.21.15.94:0/3057788463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 0x56205ffbb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0ae4bab640 1 -- 172.21.15.94:0/3057788463 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad0010e20 con 0x56205ffbeaa0 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0adf7fe640 1 --2- 172.21.15.94:0/3057788463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 0x56205ffbb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0ae4bab640 1 --2- 172.21.15.94:0/3057788463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffbeaa0 0x56205ffbbcf0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009fd0 tx=0x7f0ad8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e253b663c28b5d8d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0abf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0abf7fe640 1 -- 172.21.15.94:0/3057788463 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8022070 con 0x56205ffbeaa0 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3057788463 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56205fc1c7e0 con 0x56205ffbeaa0 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0abf7fe640 1 -- 172.21.15.94:0/3057788463 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0ad8004770 con 0x56205ffbeaa0 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0abf7fe640 1 -- 172.21.15.94:0/3057788463 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ad8016070 con 0x56205ffbeaa0 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0ae91e9740 10 client.19395 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:15.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3057788463 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56205ff1b610 con 0x56205ffbeaa0 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.938+0000 7f0abf7fe640 1 -- 172.21.15.94:0/3057788463 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ad8034020 con 0x56205ffbeaa0 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.939+0000 7f0abf7fe640 1 -- 172.21.15.94:0/3057788463 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ad8010430 con 0x56205ffbeaa0 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.939+0000 7f0ae91e9740 10 client.19395 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.939+0000 7f0ae91e9740 10 client.19395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.939+0000 7f0ae91e9740 20 client.19395 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.939+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3057788463 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56205ffce930 0x56205ffd0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.939+0000 7f0ae91e9740 4 client.19395 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.939+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3057788463 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56205fc59720 con 0x56205ffce930 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.939+0000 7f0adffff640 1 --2- 172.21.15.94:0/3057788463 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56205ffce930 0x56205ffd0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.939+0000 7f0adffff640 1 --2- 172.21.15.94:0/3057788463 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56205ffce930 0x56205ffd0d50 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=184144e7e920fd98 server_cookie=e746d54e99a4938f in_seq=0 out_seq=0 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.939+0000 7f0abf7fe640 10 client.19395 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0abf7fe640 1 -- 172.21.15.94:0/3057788463 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56205fc59720 con 0x56205ffce930 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0abf7fe640 10 client.19395 handle_command_reply: tid=0 2024-02-17T12:13:15.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0ae91e9740 1 client.19395 shutdown 2024-02-17T12:13:15.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3057788463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffbeaa0 msgr2=0x56205ffbbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:15.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3057788463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffbeaa0 0x56205ffbbcf0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009fd0 tx=0x7f0ad8004ab0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3057788463 shutdown_connections 2024-02-17T12:13:15.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3057788463 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56205ffce930 0x56205ffd0d50 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3057788463 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffbeaa0 0x56205ffbbcf0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0abf7fe640 0 client.19395 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:15.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3057788463 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 0x56205ffbb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3057788463 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffb53b0 0x56205ffbe400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3057788463 >> 172.21.15.94:0/3057788463 conn(0x56205ffafcd0 msgr2=0x56205ffb1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:15.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3057788463 shutdown_connections 2024-02-17T12:13:15.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.940+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3057788463 wait complete. 2024-02-17T12:13:15.950 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.941+0000 7f0ae91e9740 20 client.19395 trim_cache size 0 max 16384 2024-02-17T12:13:15.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.969+0000 7f0ae91e9740 1 Processor -- start 2024-02-17T12:13:15.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.969+0000 7f0ae91e9740 1 -- start start 2024-02-17T12:13:15.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.969+0000 7f0ae91e9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb53b0 0x56205ffb6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.969+0000 7f0ae91e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb5cd0 0x56205ffaebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.969+0000 7f0ae91e9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffbeaa0 0x56205ffbdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:15.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.970+0000 7f0ae91e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205ffb5230 con 0x56205ffb5cd0 2024-02-17T12:13:15.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.970+0000 7f0ae91e9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205ffcb3e0 con 0x56205ffb53b0 2024-02-17T12:13:15.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.970+0000 7f0ae91e9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205ffb6fe0 con 0x56205ffbeaa0 2024-02-17T12:13:15.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.970+0000 7f0adf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb5cd0 0x56205ffaebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:15.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.970+0000 7f0adf7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb5cd0 0x56205ffaebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56114/0 (socket says 172.21.15.94:56114) 2024-02-17T12:13:15.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.970+0000 7f0adf7fe640 1 -- 172.21.15.94:0/2648528567 learned_addr learned my addr 172.21.15.94:0/2648528567 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:15.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.970+0000 7f0ae4bab640 1 --2- 172.21.15.94:0/2648528567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffbeaa0 0x56205ffbdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:15.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.970+0000 7f0adf7fe640 1 -- 172.21.15.94:0/2648528567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffbeaa0 msgr2=0x56205ffbdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:15.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.970+0000 7f0adf7fe640 1 --2- 172.21.15.94:0/2648528567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffbeaa0 0x56205ffbdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.973 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.970+0000 7f0adf7fe640 1 -- 172.21.15.94:0/2648528567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb53b0 msgr2=0x56205ffb6aa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-17T12:13:15.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.970+0000 7f0adf7fe640 1 --2- 172.21.15.94:0/2648528567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb53b0 0x56205ffb6aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.970+0000 7f0adf7fe640 1 -- 172.21.15.94:0/2648528567 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56205feb7ac0 con 0x56205ffb5cd0 2024-02-17T12:13:15.980 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.978+0000 7f0adf7fe640 1 --2- 172.21.15.94:0/2648528567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb5cd0 0x56205ffaebb0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40b8f70 tx=0x7f0ad40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7513015894b488 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.982+0000 7f0ac57fa640 1 -- 172.21.15.94:0/2648528567 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40bc470 con 0x56205ffb5cd0 2024-02-17T12:13:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.982+0000 7f0ac57fa640 1 -- 172.21.15.94:0/2648528567 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ad4005440 con 0x56205ffb5cd0 2024-02-17T12:13:15.984 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.982+0000 7f0ac57fa640 1 -- 172.21.15.94:0/2648528567 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40c4e20 con 0x56205ffb5cd0 2024-02-17T12:13:15.987 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.983+0000 7f0ae91e9740 1 -- 172.21.15.94:0/2648528567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb5cd0 msgr2=0x56205ffaebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:15.988 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.983+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/2648528567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb5cd0 0x56205ffaebb0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40b8f70 tx=0x7f0ad40035f0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.990+0000 7f0ae91e9740 1 -- 172.21.15.94:0/2648528567 shutdown_connections 2024-02-17T12:13:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.990+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/2648528567 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffbeaa0 0x56205ffbdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.990+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/2648528567 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffb5cd0 0x56205ffaebb0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.990+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/2648528567 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb53b0 0x56205ffb6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:15.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.990+0000 7f0ae91e9740 1 -- 172.21.15.94:0/2648528567 >> 172.21.15.94:0/2648528567 conn(0x56205ffafcd0 msgr2=0x56205ffae4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:15.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:15.996+0000 7f0ae91e9740 1 -- 172.21.15.94:0/2648528567 shutdown_connections 2024-02-17T12:13:16.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.000+0000 7f0ae91e9740 1 -- 172.21.15.94:0/2648528567 wait complete. 2024-02-17T12:13:16.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.000+0000 7f0ae91e9740 1 -- start start 2024-02-17T12:13:16.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.000+0000 7f0ae91e9740 1 Processor -- start 2024-02-17T12:13:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.004+0000 7f0ae91e9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffb53b0 0x56205ffc2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.004+0000 7f0ae91e9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 0x56205ffc2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.004+0000 7f0ae91e9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffbeaa0 0x562060024e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.004+0000 7f0ae91e9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205ffc3210 con 0x56205ffbeaa0 2024-02-17T12:13:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.004+0000 7f0ae91e9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56205ffc3380 con 0x56205ffb5cd0 2024-02-17T12:13:16.006 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.004+0000 7f0ae91e9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562060036620 con 0x56205ffb53b0 2024-02-17T12:13:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.004+0000 7f0ae91e9740 10 client.0 _ll_register_callbacks cb 0x7f0ad00106e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.005+0000 7f0adffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffb53b0 0x56205ffc2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.005+0000 7f0adffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffb53b0 0x56205ffc2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:45816/0 (socket says 172.21.15.94:45816) 2024-02-17T12:13:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.005+0000 7f0adffff640 1 -- 172.21.15.94:0/3505359330 learned_addr learned my addr 172.21.15.94:0/3505359330 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:16.007 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.005+0000 7f0adffff640 1 -- 172.21.15.94:0/3505359330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 msgr2=0x56205ffc2cd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:13:16.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.005+0000 7f0adffff640 1 --2- 172.21.15.94:0/3505359330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 0x56205ffc2cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.005+0000 7f0adffff640 1 -- 172.21.15.94:0/3505359330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffbeaa0 msgr2=0x562060024e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:13:16.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.005+0000 7f0adffff640 1 --2- 172.21.15.94:0/3505359330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffbeaa0 0x562060024e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.005+0000 7f0adffff640 1 -- 172.21.15.94:0/3505359330 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae01004b0 con 0x56205ffb53b0 2024-02-17T12:13:16.008 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.007+0000 7f0adffff640 1 --2- 172.21.15.94:0/3505359330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffb53b0 0x56205ffc2790 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0010df0 tx=0x7f0ad000b5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfd31e68bb99fb6e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:16.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.010+0000 7f0abeffd640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:13:16.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.010+0000 7f0abeffd640 1 -- 172.21.15.94:0/3505359330 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0015070 con 0x56205ffb53b0 2024-02-17T12:13:16.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.010+0000 7f0abeffd640 1 -- 172.21.15.94:0/3505359330 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f0ad002dd70 con 0x56205ffb53b0 2024-02-17T12:13:16.012 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.010+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3505359330 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ad0016040 con 0x56205ffb53b0 2024-02-17T12:13:16.015 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.013+0000 7f0abeffd640 1 -- 172.21.15.94:0/3505359330 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ad000bb80 con 0x56205ffb53b0 2024-02-17T12:13:16.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.018+0000 7f0ae91e9740 10 client.28822 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:16.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.018+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3505359330 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56205fedd6a0 con 0x56205ffb53b0 2024-02-17T12:13:16.021 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.020+0000 7f0abeffd640 1 -- 172.21.15.94:0/3505359330 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ad0016040 con 0x56205ffb53b0 2024-02-17T12:13:16.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.021+0000 7f0abeffd640 1 -- 172.21.15.94:0/3505359330 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f0ad0021070 con 0x56205ffb53b0 2024-02-17T12:13:16.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.021+0000 7f0ae91e9740 10 client.28822 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:16.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.021+0000 7f0ae91e9740 10 client.28822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:16.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.021+0000 7f0ae91e9740 20 client.28822 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:16.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.021+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3505359330 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56205ffce930 0x562060039d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.021+0000 7f0ae91e9740 4 client.28822 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:13:16.023 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.021+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3505359330 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56205fca8890 con 0x56205ffce930 2024-02-17T12:13:16.025 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.022+0000 7f0adf7fe640 1 --2- 172.21.15.94:0/3505359330 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56205ffce930 0x562060039d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.027 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.025+0000 7f0adf7fe640 1 --2- 172.21.15.94:0/3505359330 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56205ffce930 0x562060039d40 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e7db68224bbea27 server_cookie=34c2d6abf42ccb23 in_seq=0 out_seq=0 2024-02-17T12:13:16.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.027+0000 7f0abeffd640 10 client.28822 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:16.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.027+0000 7f0abeffd640 1 -- 172.21.15.94:0/3505359330 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56205fca8890 con 0x56205ffce930 2024-02-17T12:13:16.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.027+0000 7f0abeffd640 10 client.28822 handle_command_reply: tid=0 2024-02-17T12:13:16.031 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.027+0000 7f0ae91e9740 1 client.28822 shutdown 2024-02-17T12:13:16.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.030+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3505359330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffb53b0 msgr2=0x56205ffc2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.030+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3505359330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffb53b0 0x56205ffc2790 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0010df0 tx=0x7f0ad000b5c0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.030+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3505359330 shutdown_connections 2024-02-17T12:13:16.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.030+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3505359330 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56205ffce930 0x562060039d40 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.030+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3505359330 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56205ffbeaa0 0x562060024e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.032 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.030+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3505359330 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56205ffb5cd0 0x56205ffc2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.030+0000 7f0ae91e9740 1 --2- 172.21.15.94:0/3505359330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56205ffb53b0 0x56205ffc2790 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.030+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3505359330 >> 172.21.15.94:0/3505359330 conn(0x56205ffafcd0 msgr2=0x56205ffbb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:16.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.031+0000 7f0abeffd640 0 client.28822 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:16.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.031+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3505359330 shutdown_connections 2024-02-17T12:13:16.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.031+0000 7f0ae91e9740 1 -- 172.21.15.94:0/3505359330 wait complete. 2024-02-17T12:13:16.033 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.031+0000 7f0ae91e9740 20 client.28822 trim_cache size 0 max 16384 2024-02-17T12:13:16.034 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:13:16.034 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:13:16.034 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:13:16.034 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:13:16.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.032+0000 7f0ac6ffd640 1 -- 172.21.15.94:0/1743426811 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0ac80a2230 msgr2=0x7f0ac80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.032+0000 7f0ac6ffd640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0ac80a2230 0x7f0ac80a46f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x56205ffb4ad0 tx=0x7f0ad8009500 comp rx=0 tx=0).stop 2024-02-17T12:13:16.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.032+0000 7f0ac6ffd640 1 -- 172.21.15.94:0/1743426811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0069e80 msgr2=0x7f0ae019d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.032+0000 7f0ac6ffd640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0069e80 0x7f0ae019d960 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009230 tx=0x7f0ad402c5d0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.032+0000 7f0ac6ffd640 1 -- 172.21.15.94:0/1743426811 shutdown_connections 2024-02-17T12:13:16.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.032+0000 7f0ac6ffd640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f0ac80a2230 0x7f0ac80a46f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.032+0000 7f0ac6ffd640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f0ae01071b0 0x7f0ae01975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.032+0000 7f0ac6ffd640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f0ae0069e80 0x7f0ae019d960 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.032+0000 7f0ac6ffd640 1 --2- 172.21.15.94:0/1743426811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ae00694e0 0x7f0ae019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.032+0000 7f0ac6ffd640 1 -- 172.21.15.94:0/1743426811 >> 172.21.15.94:0/1743426811 conn(0x7f0ae0076ee0 msgr2=0x7f0ae00ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:16.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.032+0000 7f0ac6ffd640 1 -- 172.21.15.94:0/1743426811 shutdown_connections 2024-02-17T12:13:16.035 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.032+0000 7f0ac6ffd640 1 -- 172.21.15.94:0/1743426811 wait complete. 2024-02-17T12:13:16.050 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:13:16.050 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:13:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.206+0000 7fd9b309a640 1 -- 172.21.15.94:0/3968599304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9ac11a960 msgr2=0x7fd9ac11cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.206+0000 7fd9b309a640 1 --2- 172.21.15.94:0/3968599304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9ac11a960 0x7fd9ac11cd50 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd99c01a940 tx=0x7fd99c03cca0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.208+0000 7fd9b309a640 1 -- 172.21.15.94:0/3968599304 shutdown_connections 2024-02-17T12:13:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.208+0000 7fd9b309a640 1 --2- 172.21.15.94:0/3968599304 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9ac11a960 0x7fd9ac11cd50 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.208+0000 7fd9b309a640 1 --2- 172.21.15.94:0/3968599304 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac10d9e0 0x7fd9ac10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.208+0000 7fd9b309a640 1 --2- 172.21.15.94:0/3968599304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9ac10f9b0 0x7fd9ac10d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.212 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.208+0000 7fd9b309a640 1 -- 172.21.15.94:0/3968599304 >> 172.21.15.94:0/3968599304 conn(0x7fd9ac06c270 msgr2=0x7fd9ac06c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:16.213 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.211+0000 7fd9b309a640 1 -- 172.21.15.94:0/3968599304 shutdown_connections 2024-02-17T12:13:16.214 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.212+0000 7fd9b309a640 1 -- 172.21.15.94:0/3968599304 wait complete. 2024-02-17T12:13:16.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.213+0000 7fd9b309a640 1 Processor -- start 2024-02-17T12:13:16.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.214+0000 7fd9b309a640 1 -- start start 2024-02-17T12:13:16.216 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.214+0000 7fd9b309a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9ac10d9e0 0x7fd9ac119f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.214+0000 7fd9b309a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9ac10f9b0 0x7fd9ac116060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.217 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.214+0000 7fd9b309a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac11a960 0x7fd9ac1165a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.214+0000 7fd9b309a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac116ae0 con 0x7fd9ac10f9b0 2024-02-17T12:13:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.214+0000 7fd9b309a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac116c50 con 0x7fd9ac11a960 2024-02-17T12:13:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.214+0000 7fd9b309a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac116e10 con 0x7fd9ac10d9e0 2024-02-17T12:13:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.215+0000 7fd9b1637640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac11a960 0x7fd9ac1165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.215+0000 7fd9b1637640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac11a960 0x7fd9ac1165a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55040/0 (socket says 172.21.15.94:55040) 2024-02-17T12:13:16.218 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.215+0000 7fd9b1637640 1 -- 172.21.15.94:0/1539879015 learned_addr learned my addr 172.21.15.94:0/1539879015 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:16.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.215+0000 7fd9abfff640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9ac10f9b0 0x7fd9ac116060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.215+0000 7fd9b0e36640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9ac10d9e0 0x7fd9ac119f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.215+0000 7fd9b1637640 1 -- 172.21.15.94:0/1539879015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9ac10d9e0 msgr2=0x7fd9ac119f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.215+0000 7fd9b1637640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9ac10d9e0 0x7fd9ac119f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.215+0000 7fd9b1637640 1 -- 172.21.15.94:0/1539879015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9ac10f9b0 msgr2=0x7fd9ac116060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.215+0000 7fd9b1637640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9ac10f9b0 0x7fd9ac116060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.215+0000 7fd9b1637640 1 -- 172.21.15.94:0/1539879015 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c01a570 con 0x7fd9ac11a960 2024-02-17T12:13:16.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.216+0000 7fd9b1637640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac11a960 0x7fd9ac1165a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd99c03d230 tx=0x7fd99c004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:16.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.216+0000 7fd9a9ffb640 1 -- 172.21.15.94:0/1539879015 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0049d0 con 0x7fd9ac11a960 2024-02-17T12:13:16.219 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.216+0000 7fd9a9ffb640 1 -- 172.21.15.94:0/1539879015 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd99c004b30 con 0x7fd9ac11a960 2024-02-17T12:13:16.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.216+0000 7fd9a9ffb640 1 -- 172.21.15.94:0/1539879015 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c003a00 con 0x7fd9ac11a960 2024-02-17T12:13:16.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.217+0000 7fd9b309a640 1 -- 172.21.15.94:0/1539879015 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ac1aa5f0 con 0x7fd9ac11a960 2024-02-17T12:13:16.220 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.217+0000 7fd9b309a640 1 -- 172.21.15.94:0/1539879015 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ac1aaaa0 con 0x7fd9ac11a960 2024-02-17T12:13:16.222 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.220+0000 7fd9a9ffb640 1 -- 172.21.15.94:0/1539879015 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd99c003b60 con 0x7fd9ac11a960 2024-02-17T12:13:16.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.221+0000 7fd9a9ffb640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd9940a23c0 0x7fd9940a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.223 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.221+0000 7fd9a9ffb640 1 -- 172.21.15.94:0/1539879015 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fd99c100b40 con 0x7fd9ac11a960 2024-02-17T12:13:16.225 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.223+0000 7fd9b0e36640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd9940a23c0 0x7fd9940a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.226 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.219+0000 7fd9b309a640 1 -- 172.21.15.94:0/1539879015 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ac06d510 con 0x7fd9ac11a960 2024-02-17T12:13:16.227 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.225+0000 7fd9b0e36640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd9940a23c0 0x7fd9940a4880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a00090e0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:16.233 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.231+0000 7fd9a9ffb640 1 -- 172.21.15.94:0/1539879015 <== mon.2 v2:172.21.15.125:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd99c0c9030 con 0x7fd9ac11a960 2024-02-17T12:13:16.587 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.585+0000 7fd9b309a640 1 -- 172.21.15.94:0/1539879015 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9ac117540 con 0x7fd9ac11a960 2024-02-17T12:13:16.590 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.588+0000 7fd9a9ffb640 1 -- 172.21.15.94:0/1539879015 <== mon.2 v2:172.21.15.125:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7fd99c0c9020 con 0x7fd9ac11a960 2024-02-17T12:13:16.590 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:16.590 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:13:16.591 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:13:16.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.591+0000 7fd9b309a640 1 -- 172.21.15.94:0/1539879015 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd9940a23c0 msgr2=0x7fd9940a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.591+0000 7fd9b309a640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd9940a23c0 0x7fd9940a4880 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a00090e0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.596 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.591+0000 7fd9b309a640 1 -- 172.21.15.94:0/1539879015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac11a960 msgr2=0x7fd9ac1165a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.591+0000 7fd9b309a640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac11a960 0x7fd9ac1165a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd99c03d230 tx=0x7fd99c004600 comp rx=0 tx=0).stop 2024-02-17T12:13:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.592+0000 7fd9b309a640 1 -- 172.21.15.94:0/1539879015 shutdown_connections 2024-02-17T12:13:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.592+0000 7fd9b309a640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd9940a23c0 0x7fd9940a4880 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.593+0000 7fd9b309a640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd9ac11a960 0x7fd9ac1165a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.593+0000 7fd9b309a640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd9ac10f9b0 0x7fd9ac116060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.593+0000 7fd9b309a640 1 --2- 172.21.15.94:0/1539879015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd9ac10d9e0 0x7fd9ac119f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.593+0000 7fd9b309a640 1 -- 172.21.15.94:0/1539879015 >> 172.21.15.94:0/1539879015 conn(0x7fd9ac06c270 msgr2=0x7fd9ac11c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.594+0000 7fd9b309a640 1 -- 172.21.15.94:0/1539879015 shutdown_connections 2024-02-17T12:13:16.597 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.595+0000 7fd9b309a640 1 -- 172.21.15.94:0/1539879015 wait complete. 2024-02-17T12:13:16.615 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:13:16.615 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:13:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.746+0000 7fb93cac0640 1 -- 172.21.15.94:0/3588519933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb938069ec0 msgr2=0x7fb93810df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.746+0000 7fb93cac0640 1 --2- 172.21.15.94:0/3588519933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb938069ec0 0x7fb93810df90 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fb930009ca0 tx=0x7fb93002c140 comp rx=0 tx=0).stop 2024-02-17T12:13:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.747+0000 7fb93cac0640 1 -- 172.21.15.94:0/3588519933 shutdown_connections 2024-02-17T12:13:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.747+0000 7fb93cac0640 1 --2- 172.21.15.94:0/3588519933 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb938069ec0 0x7fb93810df90 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.747+0000 7fb93cac0640 1 --2- 172.21.15.94:0/3588519933 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb938069520 0x7fb938069980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.747+0000 7fb93cac0640 1 --2- 172.21.15.94:0/3588519933 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb938106120 0x7fb938106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.747+0000 7fb93cac0640 1 -- 172.21.15.94:0/3588519933 >> 172.21.15.94:0/3588519933 conn(0x7fb938077110 msgr2=0x7fb938077520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:16.749 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.747+0000 7fb93cac0640 1 -- 172.21.15.94:0/3588519933 shutdown_connections 2024-02-17T12:13:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.747+0000 7fb93cac0640 1 -- 172.21.15.94:0/3588519933 wait complete. 2024-02-17T12:13:16.752 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.747+0000 7fb93cac0640 1 Processor -- start 2024-02-17T12:13:16.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.748+0000 7fb93cac0640 1 -- start start 2024-02-17T12:13:16.754 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.748+0000 7fb93cac0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb938069520 0x7fb9381000b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.748+0000 7fb93cac0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb938069ec0 0x7fb9381005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.748+0000 7fb93cac0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb938106120 0x7fb938100b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.748+0000 7fb93659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb938069520 0x7fb9381000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.748+0000 7fb93659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb938069520 0x7fb9381000b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55054/0 (socket says 172.21.15.94:55054) 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.748+0000 7fb93659c640 1 -- 172.21.15.94:0/1383263200 learned_addr learned my addr 172.21.15.94:0/1383263200 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.748+0000 7fb935d9b640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb938069ec0 0x7fb9381005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.748+0000 7fb936d9d640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb938106120 0x7fb938100b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.748+0000 7fb93cac0640 1 -- 172.21.15.94:0/1383263200 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb938102000 con 0x7fb938106120 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.749+0000 7fb93cac0640 1 -- 172.21.15.94:0/1383263200 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9381aa140 con 0x7fb938069520 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.749+0000 7fb93cac0640 1 -- 172.21.15.94:0/1383263200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9381aa280 con 0x7fb938069ec0 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.749+0000 7fb935d9b640 1 -- 172.21.15.94:0/1383263200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb938069520 msgr2=0x7fb9381000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.749+0000 7fb935d9b640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb938069520 0x7fb9381000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.749+0000 7fb935d9b640 1 -- 172.21.15.94:0/1383263200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb938106120 msgr2=0x7fb938100b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.755 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.749+0000 7fb935d9b640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb938106120 0x7fb938100b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.749+0000 7fb935d9b640 1 -- 172.21.15.94:0/1383263200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930009900 con 0x7fb938069ec0 2024-02-17T12:13:16.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.750+0000 7fb935d9b640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb938069ec0 0x7fb9381005f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00e9f0 tx=0x7fb92c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:16.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.750+0000 7fb9277fe640 1 -- 172.21.15.94:0/1383263200 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c00ccc0 con 0x7fb938069ec0 2024-02-17T12:13:16.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.750+0000 7fb9277fe640 1 -- 172.21.15.94:0/1383263200 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb92c00ce20 con 0x7fb938069ec0 2024-02-17T12:13:16.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.750+0000 7fb93cac0640 1 -- 172.21.15.94:0/1383263200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9381aa560 con 0x7fb938069ec0 2024-02-17T12:13:16.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.750+0000 7fb93cac0640 1 -- 172.21.15.94:0/1383263200 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9381aaa70 con 0x7fb938069ec0 2024-02-17T12:13:16.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.750+0000 7fb9277fe640 1 -- 172.21.15.94:0/1383263200 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c004a00 con 0x7fb938069ec0 2024-02-17T12:13:16.756 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.752+0000 7fb93f76b740 1 Processor -- start 2024-02-17T12:13:16.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.753+0000 7fb93f76b740 1 -- start start 2024-02-17T12:13:16.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.753+0000 7fb93f76b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562932265250 0x562932265630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.753+0000 7fb93f76b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265b70 0x56293226e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.758 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.753+0000 7fb93f76b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56293226e750 0x562932270b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.753+0000 7fb93f76b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562932220960 con 0x56293226e750 2024-02-17T12:13:16.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.753+0000 7fb93f76b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629321d4310 con 0x562932265250 2024-02-17T12:13:16.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.753+0000 7fb93f76b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629321d1840 con 0x562932265b70 2024-02-17T12:13:16.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.753+0000 7fb9277fe640 1 -- 172.21.15.94:0/1383263200 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fb92c004c00 con 0x7fb938069ec0 2024-02-17T12:13:16.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.753+0000 7fb935d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56293226e750 0x562932270b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.753+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562932265250 0x562932265630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.753+0000 7fb935d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56293226e750 0x562932270b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56184/0 (socket says 172.21.15.94:56184) 2024-02-17T12:13:16.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.753+0000 7fb935d9b640 1 -- 172.21.15.94:0/3761941921 learned_addr learned my addr 172.21.15.94:0/3761941921 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:16.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.753+0000 7fb93659c640 1 --2- 172.21.15.94:0/3761941921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265b70 0x56293226e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb93659c640 1 -- 172.21.15.94:0/3761941921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562932265250 msgr2=0x562932265630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:16.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb93659c640 1 --2- 172.21.15.94:0/3761941921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562932265250 0x562932265630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb93659c640 1 -- 172.21.15.94:0/3761941921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56293226e750 msgr2=0x562932270b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:16.759 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb93659c640 1 --2- 172.21.15.94:0/3761941921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56293226e750 0x562932270b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb93659c640 1 -- 172.21.15.94:0/3761941921 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56293218f060 con 0x562932265b70 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb936d9d640 1 --2- 172.21.15.94:0/3761941921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562932265250 0x562932265630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb935d9b640 1 --2- 172.21.15.94:0/3761941921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56293226e750 0x562932270b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb93659c640 1 --2- 172.21.15.94:0/3761941921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265b70 0x56293226e210 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb928009c30 tx=0x7fb928011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b3fb8b87c18207 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb9277fe640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb9200a22b0 0x7fb9200a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb936d9d640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb9200a22b0 0x7fb9200a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb9277fe640 1 -- 172.21.15.94:0/1383263200 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb92c014070 con 0x7fb938069ec0 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb924ff9640 1 -- 172.21.15.94:0/3761941921 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928009ea0 con 0x562932265b70 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb924ff9640 1 -- 172.21.15.94:0/3761941921 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb928005500 con 0x562932265b70 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.754+0000 7fb924ff9640 1 -- 172.21.15.94:0/3761941921 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928004cf0 con 0x562932265b70 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.755+0000 7fb93f76b740 1 -- 172.21.15.94:0/3761941921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265b70 msgr2=0x56293226e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.755+0000 7fb93f76b740 1 --2- 172.21.15.94:0/3761941921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265b70 0x56293226e210 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb928009c30 tx=0x7fb928011810 comp rx=0 tx=0).stop 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.755+0000 7fb936d9d640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb9200a22b0 0x7fb9200a4770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb930004af0 tx=0x7fb930005c50 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:16.760 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.755+0000 7fb93f76b740 1 -- 172.21.15.94:0/3761941921 shutdown_connections 2024-02-17T12:13:16.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.755+0000 7fb93f76b740 1 --2- 172.21.15.94:0/3761941921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56293226e750 0x562932270b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.755+0000 7fb93f76b740 1 --2- 172.21.15.94:0/3761941921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265b70 0x56293226e210 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.755+0000 7fb93f76b740 1 --2- 172.21.15.94:0/3761941921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562932265250 0x562932265630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.755+0000 7fb93f76b740 1 -- 172.21.15.94:0/3761941921 >> 172.21.15.94:0/3761941921 conn(0x56293225f980 msgr2=0x56293225fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:16.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.755+0000 7fb93f76b740 1 -- 172.21.15.94:0/3761941921 shutdown_connections 2024-02-17T12:13:16.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.755+0000 7fb93f76b740 1 -- 172.21.15.94:0/3761941921 wait complete. 2024-02-17T12:13:16.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.756+0000 7fb93f76b740 1 -- start start 2024-02-17T12:13:16.761 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.756+0000 7fb93f76b740 1 Processor -- start 2024-02-17T12:13:16.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.756+0000 7fb93f76b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265250 0x56293226b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.756+0000 7fb93f76b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562932265b70 0x56293226bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.756+0000 7fb93f76b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56293226e750 0x56293226c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.756+0000 7fb93f76b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629321cd020 con 0x56293226e750 2024-02-17T12:13:16.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.756+0000 7fb93f76b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562931ea0430 con 0x562932265b70 2024-02-17T12:13:16.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.756+0000 7fb93f76b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562931e5ce00 con 0x562932265250 2024-02-17T12:13:16.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.756+0000 7fb93f76b740 10 client.0 _ll_register_callbacks cb 0x562932174cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:16.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.757+0000 7fb935d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562932265b70 0x56293226bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.757+0000 7fb93659c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265250 0x56293226b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.757+0000 7fb935d9b640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562932265b70 0x56293226bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55080/0 (socket says 172.21.15.94:55080) 2024-02-17T12:13:16.765 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.757+0000 7fb935d9b640 1 -- 172.21.15.94:0/2210600923 learned_addr learned my addr 172.21.15.94:0/2210600923 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.757+0000 7fb936d9d640 1 --2- 172.21.15.94:0/2210600923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56293226e750 0x56293226c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.757+0000 7fb93659c640 1 -- 172.21.15.94:0/2210600923 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562932265b70 msgr2=0x56293226bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.757+0000 7fb93659c640 1 --2- 172.21.15.94:0/2210600923 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562932265b70 0x56293226bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.757+0000 7fb93659c640 1 -- 172.21.15.94:0/2210600923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56293226e750 msgr2=0x56293226c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.757+0000 7fb93659c640 1 --2- 172.21.15.94:0/2210600923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56293226e750 0x56293226c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.757+0000 7fb93659c640 1 -- 172.21.15.94:0/2210600923 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb928009860 con 0x562932265250 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.757+0000 7fb935d9b640 1 --2- 172.21.15.94:0/2210600923 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562932265b70 0x56293226bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.757+0000 7fb93659c640 1 --2- 172.21.15.94:0/2210600923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265250 0x56293226b700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb928004990 tx=0x7fb928009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3d5e7df917fbe79 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.757+0000 7fb91dffb640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb91dffb640 1 -- 172.21.15.94:0/2210600923 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9280037f0 con 0x562932265250 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb91dffb640 1 -- 172.21.15.94:0/2210600923 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb928003950 con 0x562932265250 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb93f76b740 1 -- 172.21.15.94:0/2210600923 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562931ecc7e0 con 0x562932265250 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb91dffb640 1 -- 172.21.15.94:0/2210600923 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb928003b10 con 0x562932265250 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb93f76b740 10 client.28837 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:16.766 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb93f76b740 1 -- 172.21.15.94:0/2210600923 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5629321cb6a0 con 0x562932265250 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb91dffb640 1 -- 172.21.15.94:0/2210600923 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x562931ecc7e0 con 0x562932265250 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb91dffb640 1 -- 172.21.15.94:0/2210600923 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb928023410 con 0x562932265250 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb93f76b740 10 client.28837 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb93f76b740 10 client.28837 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb93f76b740 20 client.28837 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb93f76b740 1 --2- 172.21.15.94:0/2210600923 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x56293227eba0 0x562932280fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb93f76b740 4 client.28837 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.758+0000 7fb93f76b740 1 -- 172.21.15.94:0/2210600923 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562931f09720 con 0x56293227eba0 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.759+0000 7fb935d9b640 1 --2- 172.21.15.94:0/2210600923 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x56293227eba0 0x562932280fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.759+0000 7fb935d9b640 1 --2- 172.21.15.94:0/2210600923 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x56293227eba0 0x562932280fc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba1ca9fc77482495 server_cookie=15781073a520338f in_seq=0 out_seq=0 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.759+0000 7fb91dffb640 10 client.28837 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.759+0000 7fb91dffb640 1 -- 172.21.15.94:0/2210600923 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562931f09720 con 0x56293227eba0 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.759+0000 7fb91dffb640 10 client.28837 handle_command_reply: tid=0 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.759+0000 7fb93f76b740 1 client.28837 shutdown 2024-02-17T12:13:16.767 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.760+0000 7fb93f76b740 1 -- 172.21.15.94:0/2210600923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265250 msgr2=0x56293226b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.760+0000 7fb93f76b740 1 --2- 172.21.15.94:0/2210600923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265250 0x56293226b700 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb928004990 tx=0x7fb928009f00 comp rx=0 tx=0).stop 2024-02-17T12:13:16.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.760+0000 7fb93f76b740 1 -- 172.21.15.94:0/2210600923 shutdown_connections 2024-02-17T12:13:16.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.760+0000 7fb93f76b740 1 --2- 172.21.15.94:0/2210600923 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56293226e750 0x56293226c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.760+0000 7fb93f76b740 1 --2- 172.21.15.94:0/2210600923 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x56293227eba0 0x562932280fc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.760+0000 7fb93f76b740 1 --2- 172.21.15.94:0/2210600923 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562932265b70 0x56293226bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.760+0000 7fb91dffb640 0 client.28837 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:13:16.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.760+0000 7fb93f76b740 1 --2- 172.21.15.94:0/2210600923 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265250 0x56293226b700 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.760+0000 7fb93f76b740 1 -- 172.21.15.94:0/2210600923 >> 172.21.15.94:0/2210600923 conn(0x56293225f980 msgr2=0x5629322616d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:16.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.760+0000 7fb93f76b740 1 -- 172.21.15.94:0/2210600923 shutdown_connections 2024-02-17T12:13:16.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.760+0000 7fb93f76b740 1 -- 172.21.15.94:0/2210600923 wait complete. 2024-02-17T12:13:16.768 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.760+0000 7fb93f76b740 20 client.28837 trim_cache size 0 max 16384 2024-02-17T12:13:16.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.791+0000 7fb93f76b740 1 Processor -- start 2024-02-17T12:13:16.793 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.791+0000 7fb93f76b740 1 -- start start 2024-02-17T12:13:16.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.791+0000 7fb93f76b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562932265250 0x56293226de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.791+0000 7fb93f76b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265b70 0x56293226bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.791+0000 7fb93f76b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56293226e750 0x562932268960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.792+0000 7fb93f76b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562932152130 con 0x562932265250 2024-02-17T12:13:16.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.792+0000 7fb93f76b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562932270b40 con 0x56293226e750 2024-02-17T12:13:16.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.792+0000 7fb93f76b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56293225aa10 con 0x562932265b70 2024-02-17T12:13:16.794 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.792+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56293226e750 0x562932268960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.792+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56293226e750 0x562932268960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55086/0 (socket says 172.21.15.94:55086) 2024-02-17T12:13:16.795 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.792+0000 7fb936d9d640 1 -- 172.21.15.94:0/813500280 learned_addr learned my addr 172.21.15.94:0/813500280 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:16.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.794+0000 7fb935d9b640 1 --2- 172.21.15.94:0/813500280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265b70 0x56293226bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.797 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.795+0000 7fb93659c640 1 --2- 172.21.15.94:0/813500280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562932265250 0x56293226de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.796+0000 7fb93659c640 1 -- 172.21.15.94:0/813500280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265b70 msgr2=0x56293226bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:16.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.796+0000 7fb93659c640 1 --2- 172.21.15.94:0/813500280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265b70 0x56293226bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.798 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.796+0000 7fb93659c640 1 -- 172.21.15.94:0/813500280 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56293226e750 msgr2=0x562932268960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:16.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.796+0000 7fb93659c640 1 --2- 172.21.15.94:0/813500280 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56293226e750 0x562932268960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.796+0000 7fb93659c640 1 -- 172.21.15.94:0/813500280 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562932167c20 con 0x562932265250 2024-02-17T12:13:16.799 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.797+0000 7fb93659c640 1 --2- 172.21.15.94:0/813500280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562932265250 0x56293226de30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb92800d950 tx=0x7fb928011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed206fba24ac32d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:16.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.802+0000 7fb91ffff640 1 -- 172.21.15.94:0/813500280 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb928011dc0 con 0x562932265250 2024-02-17T12:13:16.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.802+0000 7fb91ffff640 1 -- 172.21.15.94:0/813500280 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb928015070 con 0x562932265250 2024-02-17T12:13:16.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.803+0000 7fb93f76b740 1 -- 172.21.15.94:0/813500280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562932265250 msgr2=0x56293226de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.803+0000 7fb93f76b740 1 --2- 172.21.15.94:0/813500280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562932265250 0x56293226de30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb92800d950 tx=0x7fb928011810 comp rx=0 tx=0).stop 2024-02-17T12:13:16.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.805+0000 7fb93f76b740 1 -- 172.21.15.94:0/813500280 shutdown_connections 2024-02-17T12:13:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.805+0000 7fb93f76b740 1 --2- 172.21.15.94:0/813500280 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56293226e750 0x562932268960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.805+0000 7fb93f76b740 1 --2- 172.21.15.94:0/813500280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562932265b70 0x56293226bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.805+0000 7fb93f76b740 1 --2- 172.21.15.94:0/813500280 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562932265250 0x56293226de30 secure :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fb92800d950 tx=0x7fb928011810 comp rx=0 tx=0).stop 2024-02-17T12:13:16.808 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.805+0000 7fb93f76b740 1 -- 172.21.15.94:0/813500280 >> 172.21.15.94:0/813500280 conn(0x56293225f980 msgr2=0x5629322669f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:16.812 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.810+0000 7fb93f76b740 1 -- 172.21.15.94:0/813500280 shutdown_connections 2024-02-17T12:13:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.811+0000 7fb93f76b740 1 -- 172.21.15.94:0/813500280 wait complete. 2024-02-17T12:13:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.811+0000 7fb93f76b740 1 -- start start 2024-02-17T12:13:16.813 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.811+0000 7fb93f76b740 1 Processor -- start 2024-02-17T12:13:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.812+0000 7fb93f76b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562932265b70 0x5629322757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.812+0000 7fb93f76b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293226e750 0x562932273b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.812+0000 7fb93f76b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56293227eba0 0x562932274040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.812+0000 7fb93f76b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562932274580 con 0x562932265b70 2024-02-17T12:13:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.812+0000 7fb93f76b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629322746f0 con 0x56293227eba0 2024-02-17T12:13:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.812+0000 7fb93f76b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562932274830 con 0x56293226e750 2024-02-17T12:13:16.814 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.812+0000 7fb93f76b740 10 client.0 _ll_register_callbacks cb 0x7fb928009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:16.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.812+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56293227eba0 0x562932274040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.812+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56293227eba0 0x562932274040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55090/0 (socket says 172.21.15.94:55090) 2024-02-17T12:13:16.815 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.812+0000 7fb936d9d640 1 -- 172.21.15.94:0/4001816887 learned_addr learned my addr 172.21.15.94:0/4001816887 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:16.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.814+0000 7fb936d9d640 1 -- 172.21.15.94:0/4001816887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293226e750 msgr2=0x562932273b00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:13:16.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.814+0000 7fb936d9d640 1 --2- 172.21.15.94:0/4001816887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293226e750 0x562932273b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.814+0000 7fb936d9d640 1 -- 172.21.15.94:0/4001816887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562932265b70 msgr2=0x5629322757b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:13:16.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.814+0000 7fb936d9d640 1 --2- 172.21.15.94:0/4001816887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562932265b70 0x5629322757b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.816 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.814+0000 7fb936d9d640 1 -- 172.21.15.94:0/4001816887 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629321cb6a0 con 0x56293227eba0 2024-02-17T12:13:16.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.820+0000 7fb936d9d640 1 --2- 172.21.15.94:0/4001816887 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56293227eba0 0x562932274040 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb930005cc0 tx=0x7fb9300095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6a01756963bc134 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:16.823 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.821+0000 7fb91dffb640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:13:16.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.825+0000 7fb91dffb640 1 -- 172.21.15.94:0/4001816887 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb930050070 con 0x56293227eba0 2024-02-17T12:13:16.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.825+0000 7fb91dffb640 1 -- 172.21.15.94:0/4001816887 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fb930032af0 con 0x56293227eba0 2024-02-17T12:13:16.828 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.826+0000 7fb93f76b740 1 -- 172.21.15.94:0/4001816887 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb928005500 con 0x56293227eba0 2024-02-17T12:13:16.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.828+0000 7fb91dffb640 1 -- 172.21.15.94:0/4001816887 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb930003970 con 0x56293227eba0 2024-02-17T12:13:16.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.831+0000 7fb93f76b740 10 client.28865 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:16.833 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.831+0000 7fb93f76b740 1 -- 172.21.15.94:0/4001816887 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56293218d8c0 con 0x56293227eba0 2024-02-17T12:13:16.834 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.832+0000 7fb91dffb640 1 -- 172.21.15.94:0/4001816887 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb928005500 con 0x56293227eba0 2024-02-17T12:13:16.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.833+0000 7fb91dffb640 1 -- 172.21.15.94:0/4001816887 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fb930055460 con 0x56293227eba0 2024-02-17T12:13:16.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.834+0000 7fb93f76b740 10 client.28865 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:16.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.834+0000 7fb93f76b740 10 client.28865 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:13:16.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.834+0000 7fb93f76b740 20 client.28865 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:16.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.834+0000 7fb93f76b740 1 --2- 172.21.15.94:0/4001816887 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5629322e75c0 0x5629322e9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:16.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.834+0000 7fb93f76b740 4 client.28865 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:13:16.836 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.834+0000 7fb93f76b740 1 -- 172.21.15.94:0/4001816887 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562931f58890 con 0x5629322e75c0 2024-02-17T12:13:16.849 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.838+0000 7fb93659c640 1 --2- 172.21.15.94:0/4001816887 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5629322e75c0 0x5629322e9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:16.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.851+0000 7fb93659c640 1 --2- 172.21.15.94:0/4001816887 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5629322e75c0 0x5629322e9a20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c7bec8045d88ded server_cookie=f0f8bd48f0c6db6a in_seq=0 out_seq=0 2024-02-17T12:13:16.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.852+0000 7fb91dffb640 10 client.28865 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:13:16.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.852+0000 7fb91dffb640 1 -- 172.21.15.94:0/4001816887 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562931f58890 con 0x5629322e75c0 2024-02-17T12:13:16.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.852+0000 7fb91dffb640 10 client.28865 handle_command_reply: tid=0 2024-02-17T12:13:16.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.852+0000 7fb93f76b740 1 client.28865 shutdown 2024-02-17T12:13:16.862 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.860+0000 7fb93f76b740 1 -- 172.21.15.94:0/4001816887 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56293227eba0 msgr2=0x562932274040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.860+0000 7fb93f76b740 1 --2- 172.21.15.94:0/4001816887 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56293227eba0 0x562932274040 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb930005cc0 tx=0x7fb9300095c0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.861+0000 7fb93f76b740 1 -- 172.21.15.94:0/4001816887 shutdown_connections 2024-02-17T12:13:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.861+0000 7fb93f76b740 1 --2- 172.21.15.94:0/4001816887 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5629322e75c0 0x5629322e9a20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.861+0000 7fb93f76b740 1 --2- 172.21.15.94:0/4001816887 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56293227eba0 0x562932274040 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.863 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.861+0000 7fb93f76b740 1 --2- 172.21.15.94:0/4001816887 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56293226e750 0x562932273b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.861+0000 7fb93f76b740 1 --2- 172.21.15.94:0/4001816887 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562932265b70 0x5629322757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.861+0000 7fb93f76b740 1 -- 172.21.15.94:0/4001816887 >> 172.21.15.94:0/4001816887 conn(0x56293225f980 msgr2=0x562932260db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:16.864 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.862+0000 7fb91dffb640 0 client.28865 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:13:16.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.863+0000 7fb93f76b740 1 -- 172.21.15.94:0/4001816887 shutdown_connections 2024-02-17T12:13:16.865 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.863+0000 7fb93f76b740 1 -- 172.21.15.94:0/4001816887 wait complete. 2024-02-17T12:13:16.866 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.864+0000 7fb93f76b740 20 client.28865 trim_cache size 0 max 16384 2024-02-17T12:13:16.866 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:13:16.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.866+0000 7fb93cac0640 1 -- 172.21.15.94:0/1383263200 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb9200a22b0 msgr2=0x7fb9200a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.866+0000 7fb93cac0640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb9200a22b0 0x7fb9200a4770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb930004af0 tx=0x7fb930005c50 comp rx=0 tx=0).stop 2024-02-17T12:13:16.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.866+0000 7fb93cac0640 1 -- 172.21.15.94:0/1383263200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb938069ec0 msgr2=0x7fb9381005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:16.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.866+0000 7fb93cac0640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb938069ec0 0x7fb9381005f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00e9f0 tx=0x7fb92c00eeb0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.866+0000 7fb93cac0640 1 -- 172.21.15.94:0/1383263200 shutdown_connections 2024-02-17T12:13:16.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.866+0000 7fb93cac0640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fb9200a22b0 0x7fb9200a4770 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.866+0000 7fb93cac0640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fb938106120 0x7fb938100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.866+0000 7fb93cac0640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb938069ec0 0x7fb9381005f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.866+0000 7fb93cac0640 1 --2- 172.21.15.94:0/1383263200 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fb938069520 0x7fb9381000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:16.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.866+0000 7fb93cac0640 1 -- 172.21.15.94:0/1383263200 >> 172.21.15.94:0/1383263200 conn(0x7fb938077110 msgr2=0x7fb9380fdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:16.869 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.867+0000 7fb93cac0640 1 -- 172.21.15.94:0/1383263200 shutdown_connections 2024-02-17T12:13:16.870 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:16.867+0000 7fb93cac0640 1 -- 172.21.15.94:0/1383263200 wait complete. 2024-02-17T12:13:16.883 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:16 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1539879015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:13:16.884 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:13:16.884 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:13:16.903 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:16 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1539879015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:13:16.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:16 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1539879015' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:13:17.049 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.046+0000 7f1fbde76640 1 -- 172.21.15.94:0/1206538854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb810d9d0 msgr2=0x7f1fb810de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.050 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.046+0000 7f1fbde76640 1 --2- 172.21.15.94:0/1206538854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb810d9d0 0x7f1fb810de50 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009900 tx=0x7f1fac02bc80 comp rx=0 tx=0).stop 2024-02-17T12:13:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.048+0000 7f1fbde76640 1 -- 172.21.15.94:0/1206538854 shutdown_connections 2024-02-17T12:13:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.048+0000 7f1fbde76640 1 --2- 172.21.15.94:0/1206538854 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fb811a960 0x7f1fb811cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.048+0000 7f1fbde76640 1 --2- 172.21.15.94:0/1206538854 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb810d9d0 0x7f1fb810de50 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.048+0000 7f1fbde76640 1 --2- 172.21.15.94:0/1206538854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb810f9a0 0x7f1fb810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.048+0000 7f1fbde76640 1 -- 172.21.15.94:0/1206538854 >> 172.21.15.94:0/1206538854 conn(0x7f1fb806c260 msgr2=0x7f1fb806c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.049+0000 7f1fbde76640 1 -- 172.21.15.94:0/1206538854 shutdown_connections 2024-02-17T12:13:17.051 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.049+0000 7f1fbde76640 1 -- 172.21.15.94:0/1206538854 wait complete. 2024-02-17T12:13:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.050+0000 7f1fbde76640 1 Processor -- start 2024-02-17T12:13:17.052 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.050+0000 7f1fbde76640 1 -- start start 2024-02-17T12:13:17.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.050+0000 7f1fbde76640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fb810d9d0 0x7f1fb81a6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.050+0000 7f1fbde76640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb810f9a0 0x7f1fb81a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.051+0000 7f1fbde76640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb811a960 0x7f1fb81aae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.051+0000 7f1fbde76640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb81ab490 con 0x7f1fb810f9a0 2024-02-17T12:13:17.053 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.051+0000 7f1fbde76640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb81ab600 con 0x7f1fb810d9d0 2024-02-17T12:13:17.054 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.051+0000 7f1fbde76640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb81ab7c0 con 0x7f1fb811a960 2024-02-17T12:13:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.051+0000 7f1fb7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb811a960 0x7f1fb81aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.051+0000 7f1fb7fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb811a960 0x7f1fb81aae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:45892/0 (socket says 172.21.15.94:45892) 2024-02-17T12:13:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.051+0000 7f1fb7fff640 1 -- 172.21.15.94:0/2064682986 learned_addr learned my addr 172.21.15.94:0/2064682986 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.051+0000 7f1fb6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb810f9a0 0x7f1fb81a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.051+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fb810d9d0 0x7f1fb81a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.052+0000 7f1fb7fff640 1 -- 172.21.15.94:0/2064682986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fb810d9d0 msgr2=0x7f1fb81a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.052+0000 7f1fb7fff640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fb810d9d0 0x7f1fb81a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.052+0000 7f1fb7fff640 1 -- 172.21.15.94:0/2064682986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb810f9a0 msgr2=0x7f1fb81a6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.052+0000 7f1fb7fff640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb810f9a0 0x7f1fb81a6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.052+0000 7f1fb7fff640 1 -- 172.21.15.94:0/2064682986 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fac009560 con 0x7f1fb811a960 2024-02-17T12:13:17.055 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.052+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb810f9a0 0x7f1fb81a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:17.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.052+0000 7f1fb7fff640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb811a960 0x7f1fb81aae90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800c970 tx=0x7f1fa800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:17.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.052+0000 7f1fb4ff9640 1 -- 172.21.15.94:0/2064682986 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8007bb0 con 0x7f1fb811a960 2024-02-17T12:13:17.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.052+0000 7f1fb4ff9640 1 -- 172.21.15.94:0/2064682986 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fa8007d10 con 0x7f1fb811a960 2024-02-17T12:13:17.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.052+0000 7f1fb4ff9640 1 -- 172.21.15.94:0/2064682986 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8018770 con 0x7f1fb811a960 2024-02-17T12:13:17.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.053+0000 7f1fbde76640 1 -- 172.21.15.94:0/2064682986 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb81abaa0 con 0x7f1fb811a960 2024-02-17T12:13:17.056 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.053+0000 7f1fbde76640 1 -- 172.21.15.94:0/2064682986 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb81b3320 con 0x7f1fb811a960 2024-02-17T12:13:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.057+0000 7f1fc0a47740 1 Processor -- start 2024-02-17T12:13:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.057+0000 7f1fb4ff9640 1 -- 172.21.15.94:0/2064682986 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1fa800f450 con 0x7f1fb811a960 2024-02-17T12:13:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.057+0000 7f1fc0a47740 1 -- start start 2024-02-17T12:13:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.057+0000 7f1fc0a47740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 0x555fbb496630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.059 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.057+0000 7f1fc0a47740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496b70 0x555fbb49f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.057+0000 7f1fc0a47740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 0x555fbb4a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.057+0000 7f1fc0a47740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb451960 con 0x555fbb496250 2024-02-17T12:13:17.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.057+0000 7f1fc0a47740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb405310 con 0x555fbb49f750 2024-02-17T12:13:17.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.057+0000 7f1fc0a47740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb402840 con 0x555fbb496b70 2024-02-17T12:13:17.060 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.058+0000 7f1fb6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 0x555fbb496630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.058+0000 7f1fb4ff9640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1fa00a22f0 0x7f1fa00a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.058+0000 7f1fb6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 0x555fbb496630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56234/0 (socket says 172.21.15.94:56234) 2024-02-17T12:13:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.058+0000 7f1fb6ffd640 1 -- 172.21.15.94:0/3959022662 learned_addr learned my addr 172.21.15.94:0/3959022662 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.058+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/3959022662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 0x555fbb4a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.058+0000 7f1fb4ff9640 1 -- 172.21.15.94:0/2064682986 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1fa80cdee0 con 0x7f1fb811a960 2024-02-17T12:13:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.058+0000 7f1fb6ffd640 1 -- 172.21.15.94:0/3959022662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496b70 msgr2=0x555fbb49f210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:13:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.058+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/3959022662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496b70 0x555fbb49f210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.058+0000 7f1fb6ffd640 1 -- 172.21.15.94:0/3959022662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 msgr2=0x555fbb4a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.058+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/3959022662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 0x555fbb4a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.058+0000 7f1fb6ffd640 1 -- 172.21.15.94:0/3959022662 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fbb3c0060 con 0x555fbb496250 2024-02-17T12:13:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.059+0000 7f1fb7fff640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1fa00a22f0 0x7f1fa00a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.061 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.059+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/3959022662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 0x555fbb496630 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009230 tx=0x7f1fac005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb7bdb5b1326164 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:17.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.059+0000 7f1f9dffb640 1 -- 172.21.15.94:0/3959022662 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac004a70 con 0x555fbb496250 2024-02-17T12:13:17.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.059+0000 7f1f9dffb640 1 -- 172.21.15.94:0/3959022662 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fac0041f0 con 0x555fbb496250 2024-02-17T12:13:17.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.059+0000 7f1f9dffb640 1 -- 172.21.15.94:0/3959022662 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0038e0 con 0x555fbb496250 2024-02-17T12:13:17.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.060+0000 7f1fc0a47740 1 -- 172.21.15.94:0/3959022662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 msgr2=0x555fbb496630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.060+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/3959022662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 0x555fbb496630 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009230 tx=0x7f1fac005b90 comp rx=0 tx=0).stop 2024-02-17T12:13:17.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.060+0000 7f1fb7fff640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1fa00a22f0 0x7f1fa00a47b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8097200 tx=0x7f1fa8004730 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:17.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.060+0000 7f1fc0a47740 1 -- 172.21.15.94:0/3959022662 shutdown_connections 2024-02-17T12:13:17.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.060+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/3959022662 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 0x555fbb4a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.060+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/3959022662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496b70 0x555fbb49f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.060+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/3959022662 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 0x555fbb496630 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.060+0000 7f1fc0a47740 1 -- 172.21.15.94:0/3959022662 >> 172.21.15.94:0/3959022662 conn(0x555fbb490980 msgr2=0x555fbb490d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:17.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.060+0000 7f1fc0a47740 1 -- 172.21.15.94:0/3959022662 shutdown_connections 2024-02-17T12:13:17.063 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.060+0000 7f1fc0a47740 1 -- 172.21.15.94:0/3959022662 wait complete. 2024-02-17T12:13:17.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.060+0000 7f1fc0a47740 1 -- start start 2024-02-17T12:13:17.064 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.061+0000 7f1fc0a47740 1 Processor -- start 2024-02-17T12:13:17.065 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.061+0000 7f1fc0a47740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496250 0x555fbb49c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.061+0000 7f1fc0a47740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496b70 0x555fbb49cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.061+0000 7f1fc0a47740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 0x555fbb49d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.061+0000 7f1fc0a47740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb3fe020 con 0x555fbb496b70 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.061+0000 7f1fc0a47740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb0d1430 con 0x555fbb49f750 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.061+0000 7f1fc0a47740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb08de00 con 0x555fbb496250 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.061+0000 7f1fc0a47740 10 client.0 _ll_register_callbacks cb 0x555fbb3a5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.061+0000 7f1fb6ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496250 0x555fbb49c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.061+0000 7f1fb6ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496250 0x555fbb49c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:45894/0 (socket says 172.21.15.94:45894) 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.061+0000 7f1fb6ffd640 1 -- 172.21.15.94:0/4212582052 learned_addr learned my addr 172.21.15.94:0/4212582052 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.061+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/4212582052 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 0x555fbb49d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.061+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/4212582052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496b70 0x555fbb49cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.062+0000 7f1fb77fe640 1 -- 172.21.15.94:0/4212582052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496250 msgr2=0x555fbb49c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.062+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/4212582052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496250 0x555fbb49c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.066 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.062+0000 7f1fb77fe640 1 -- 172.21.15.94:0/4212582052 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 msgr2=0x555fbb49d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.062+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/4212582052 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 0x555fbb49d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.062+0000 7f1fb77fe640 1 -- 172.21.15.94:0/4212582052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb811d680 con 0x555fbb496b70 2024-02-17T12:13:17.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.062+0000 7f1f96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:17.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.062+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/4212582052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496b70 0x555fbb49cd10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000bad0 tx=0x7f1fb000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7528a9f9ace696d server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:17.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.062+0000 7f1f96ffd640 1 -- 172.21.15.94:0/4212582052 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0010040 con 0x555fbb496b70 2024-02-17T12:13:17.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.062+0000 7f1f96ffd640 1 -- 172.21.15.94:0/4212582052 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1fb0014420 con 0x555fbb496b70 2024-02-17T12:13:17.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.062+0000 7f1fc0a47740 1 -- 172.21.15.94:0/4212582052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fbb0fd7e0 con 0x555fbb496b70 2024-02-17T12:13:17.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.063+0000 7f1f96ffd640 1 -- 172.21.15.94:0/4212582052 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fb00145e0 con 0x555fbb496b70 2024-02-17T12:13:17.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.063+0000 7f1fc0a47740 10 client.19431 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:17.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.063+0000 7f1fc0a47740 1 -- 172.21.15.94:0/4212582052 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555fbb3fc6a0 con 0x555fbb496b70 2024-02-17T12:13:17.067 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.063+0000 7f1f96ffd640 1 -- 172.21.15.94:0/4212582052 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x555fbb0fd7e0 con 0x555fbb496b70 2024-02-17T12:13:17.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.064+0000 7f1f96ffd640 1 -- 172.21.15.94:0/4212582052 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1fb0013850 con 0x555fbb496b70 2024-02-17T12:13:17.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.064+0000 7f1fc0a47740 10 client.19431 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:17.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.064+0000 7f1fc0a47740 10 client.19431 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:13:17.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.064+0000 7f1fc0a47740 20 client.19431 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:17.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.064+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/4212582052 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x555fbb4afcc0 0x555fbb4b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.064+0000 7f1fc0a47740 4 client.19431 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:13:17.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.064+0000 7f1fc0a47740 1 -- 172.21.15.94:0/4212582052 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fbb13a720 con 0x555fbb4afcc0 2024-02-17T12:13:17.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.064+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/4212582052 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x555fbb4afcc0 0x555fbb4b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.065+0000 7f1f96ffd640 10 client.19431 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:13:17.069 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.065+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/4212582052 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x555fbb4afcc0 0x555fbb4b20e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5843218e5d94ffd3 server_cookie=81134ab2aecf3f0a in_seq=0 out_seq=0 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1f96ffd640 1 -- 172.21.15.94:0/4212582052 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f1fac05aa90 con 0x555fbb4afcc0 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1f96ffd640 10 client.19431 handle_command_reply: tid=0 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1fc0a47740 1 client.19431 shutdown 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1fc0a47740 1 -- 172.21.15.94:0/4212582052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496b70 msgr2=0x555fbb49cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/4212582052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496b70 0x555fbb49cd10 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000bad0 tx=0x7f1fb000bf90 comp rx=0 tx=0).stop 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1fc0a47740 1 -- 172.21.15.94:0/4212582052 shutdown_connections 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/4212582052 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x555fbb4afcc0 0x555fbb4b20e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/4212582052 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 0x555fbb49d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/4212582052 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496b70 0x555fbb49cd10 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/4212582052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496250 0x555fbb49c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1fc0a47740 1 -- 172.21.15.94:0/4212582052 >> 172.21.15.94:0/4212582052 conn(0x555fbb490980 msgr2=0x555fbb4926d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1f96ffd640 0 client.19431 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1fc0a47740 1 -- 172.21.15.94:0/4212582052 shutdown_connections 2024-02-17T12:13:17.070 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.066+0000 7f1fc0a47740 1 -- 172.21.15.94:0/4212582052 wait complete. 2024-02-17T12:13:17.071 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.067+0000 7f1fc0a47740 20 client.19431 trim_cache size 0 max 16384 2024-02-17T12:13:17.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fc0a47740 1 Processor -- start 2024-02-17T12:13:17.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fc0a47740 1 -- start start 2024-02-17T12:13:17.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fc0a47740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb496250 0x555fbb49ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fc0a47740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496b70 0x555fbb49d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fc0a47740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb49f750 0x555fbb4b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fc0a47740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb383130 con 0x555fbb496b70 2024-02-17T12:13:17.086 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fc0a47740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb48c2b0 con 0x555fbb496250 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fc0a47740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb49ad20 con 0x555fbb49f750 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fb77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496b70 0x555fbb49d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fb77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496b70 0x555fbb49d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56260/0 (socket says 172.21.15.94:56260) 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fb77fe640 1 -- 172.21.15.94:0/182660749 learned_addr learned my addr 172.21.15.94:0/182660749 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/182660749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb496250 0x555fbb49ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/182660749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb49f750 0x555fbb4b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fb77fe640 1 -- 172.21.15.94:0/182660749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb49f750 msgr2=0x555fbb4b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/182660749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb49f750 0x555fbb4b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fb77fe640 1 -- 172.21.15.94:0/182660749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb496250 msgr2=0x555fbb49ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/182660749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb496250 0x555fbb49ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fb77fe640 1 -- 172.21.15.94:0/182660749 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fbb398c20 con 0x555fbb496b70 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.082+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/182660749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb49f750 0x555fbb4b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.083+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/182660749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496b70 0x555fbb49d070 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00022f0 tx=0x7f1fb0009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9038eebed2a5f1e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.083+0000 7f1f9d7fa640 1 -- 172.21.15.94:0/182660749 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0010070 con 0x555fbb496b70 2024-02-17T12:13:17.087 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.083+0000 7f1f9d7fa640 1 -- 172.21.15.94:0/182660749 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fb0007000 con 0x555fbb496b70 2024-02-17T12:13:17.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.083+0000 7f1f9d7fa640 1 -- 172.21.15.94:0/182660749 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb00145e0 con 0x555fbb496b70 2024-02-17T12:13:17.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.083+0000 7f1fc0a47740 1 -- 172.21.15.94:0/182660749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496b70 msgr2=0x555fbb49d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.083+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/182660749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496b70 0x555fbb49d070 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00022f0 tx=0x7f1fb0009370 comp rx=0 tx=0).stop 2024-02-17T12:13:17.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 -- 172.21.15.94:0/182660749 shutdown_connections 2024-02-17T12:13:17.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/182660749 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb49f750 0x555fbb4b6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/182660749 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496b70 0x555fbb49d070 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/182660749 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb496250 0x555fbb49ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 -- 172.21.15.94:0/182660749 >> 172.21.15.94:0/182660749 conn(0x555fbb490980 msgr2=0x555fbb496f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:17.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 -- 172.21.15.94:0/182660749 shutdown_connections 2024-02-17T12:13:17.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 -- 172.21.15.94:0/182660749 wait complete. 2024-02-17T12:13:17.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 -- start start 2024-02-17T12:13:17.088 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 Processor -- start 2024-02-17T12:13:17.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 0x555fbb4a6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496b70 0x555fbb4a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 0x555fbb4a5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb4a5930 con 0x555fbb496250 2024-02-17T12:13:17.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb4a5aa0 con 0x555fbb49f750 2024-02-17T12:13:17.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fbb5061d0 con 0x555fbb496b70 2024-02-17T12:13:17.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fc0a47740 10 client.0 _ll_register_callbacks cb 0x7f1fac02c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:17.092 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fb6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 0x555fbb4a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fb6ffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 0x555fbb4a6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56262/0 (socket says 172.21.15.94:56262) 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.084+0000 7f1fb6ffd640 1 -- 172.21.15.94:0/2791907404 learned_addr learned my addr 172.21.15.94:0/2791907404 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/2791907404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496b70 0x555fbb4a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/2791907404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 0x555fbb4a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1fb6ffd640 1 -- 172.21.15.94:0/2791907404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496b70 msgr2=0x555fbb4a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/2791907404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496b70 0x555fbb4a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1fb6ffd640 1 -- 172.21.15.94:0/2791907404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 msgr2=0x555fbb4a5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/2791907404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 0x555fbb4a5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1fb6ffd640 1 -- 172.21.15.94:0/2791907404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fbb3fc6a0 con 0x555fbb496250 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1fb6ffd640 1 --2- 172.21.15.94:0/2791907404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 0x555fbb4a6900 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009230 tx=0x7f1fac02c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708e0164b3c4903f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1f96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1f96ffd640 1 -- 172.21.15.94:0/2791907404 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0053c0 con 0x555fbb496250 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1f96ffd640 1 -- 172.21.15.94:0/2791907404 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1fac000e40 con 0x555fbb496250 2024-02-17T12:13:17.093 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1fc0a47740 1 -- 172.21.15.94:0/2791907404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1fac0041f0 con 0x555fbb496250 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1f96ffd640 1 -- 172.21.15.94:0/2791907404 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1fac005520 con 0x555fbb496250 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1fc0a47740 10 client.19437 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1fc0a47740 1 -- 172.21.15.94:0/2791907404 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555fbb3be8c0 con 0x555fbb496250 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.085+0000 7f1f96ffd640 1 -- 172.21.15.94:0/2791907404 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1fac062080 con 0x555fbb496250 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.086+0000 7f1f96ffd640 1 -- 172.21.15.94:0/2791907404 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f1fac04b070 con 0x555fbb496250 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.086+0000 7f1fc0a47740 10 client.19437 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.086+0000 7f1fc0a47740 10 client.19437 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.086+0000 7f1fc0a47740 20 client.19437 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.086+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/2791907404 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x555fbb507610 0x555fbb509a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.086+0000 7f1fc0a47740 4 client.19437 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.086+0000 7f1fc0a47740 1 -- 172.21.15.94:0/2791907404 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555fbb189890 con 0x555fbb507610 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.086+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/2791907404 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x555fbb507610 0x555fbb509a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.086+0000 7f1f96ffd640 10 client.19437 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.086+0000 7f1fb77fe640 1 --2- 172.21.15.94:0/2791907404 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x555fbb507610 0x555fbb509a30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69ea26e8ef9a2f3d server_cookie=292e4c2f985f01c5 in_seq=0 out_seq=0 2024-02-17T12:13:17.094 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.086+0000 7f1f96ffd640 1 -- 172.21.15.94:0/2791907404 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555fbb189890 con 0x555fbb507610 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.086+0000 7f1f96ffd640 10 client.19437 handle_command_reply: tid=0 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.086+0000 7f1fc0a47740 1 client.19437 shutdown 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fc0a47740 1 -- 172.21.15.94:0/2791907404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 msgr2=0x555fbb4a6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/2791907404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 0x555fbb4a6900 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009230 tx=0x7f1fac02c300 comp rx=0 tx=0).stop 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fc0a47740 1 -- 172.21.15.94:0/2791907404 shutdown_connections 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/2791907404 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x555fbb507610 0x555fbb509a30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/2791907404 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x555fbb49f750 0x555fbb4a5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/2791907404 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555fbb496b70 0x555fbb4a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1f96ffd640 0 client.19437 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fc0a47740 1 --2- 172.21.15.94:0/2791907404 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x555fbb496250 0x555fbb4a6900 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fc0a47740 1 -- 172.21.15.94:0/2791907404 >> 172.21.15.94:0/2791907404 conn(0x555fbb490980 msgr2=0x555fbb4b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fc0a47740 1 -- 172.21.15.94:0/2791907404 shutdown_connections 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fc0a47740 1 -- 172.21.15.94:0/2791907404 wait complete. 2024-02-17T12:13:17.095 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fc0a47740 20 client.19437 trim_cache size 0 max 16384 2024-02-17T12:13:17.096 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:13:17.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fbde76640 1 -- 172.21.15.94:0/2064682986 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1fa00a22f0 msgr2=0x7f1fa00a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fbde76640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1fa00a22f0 0x7f1fa00a47b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8097200 tx=0x7f1fa8004730 comp rx=0 tx=0).stop 2024-02-17T12:13:17.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fbde76640 1 -- 172.21.15.94:0/2064682986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb811a960 msgr2=0x7f1fb81aae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fbde76640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb811a960 0x7f1fb81aae90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800c970 tx=0x7f1fa800ce30 comp rx=0 tx=0).stop 2024-02-17T12:13:17.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fbde76640 1 -- 172.21.15.94:0/2064682986 shutdown_connections 2024-02-17T12:13:17.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fbde76640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f1fa00a22f0 0x7f1fa00a47b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fbde76640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fb811a960 0x7f1fb81aae90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fbde76640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f1fb810f9a0 0x7f1fb81a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fbde76640 1 --2- 172.21.15.94:0/2064682986 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f1fb810d9d0 0x7f1fb81a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fbde76640 1 -- 172.21.15.94:0/2064682986 >> 172.21.15.94:0/2064682986 conn(0x7f1fb806c260 msgr2=0x7f1fb811c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:17.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fbde76640 1 -- 172.21.15.94:0/2064682986 shutdown_connections 2024-02-17T12:13:17.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.087+0000 7f1fbde76640 1 -- 172.21.15.94:0/2064682986 wait complete. 2024-02-17T12:13:17.101 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:13:17.101 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:13:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.242+0000 7f86fcc9a640 1 -- 172.21.15.94:0/3399764383 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f810d730 msgr2=0x7f86f810db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.242+0000 7f86fcc9a640 1 --2- 172.21.15.94:0/3399764383 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f810d730 0x7f86f810db10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009900 tx=0x7f86e002bc40 comp rx=0 tx=0).stop 2024-02-17T12:13:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.242+0000 7f86fcc9a640 1 -- 172.21.15.94:0/3399764383 shutdown_connections 2024-02-17T12:13:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.242+0000 7f86fcc9a640 1 --2- 172.21.15.94:0/3399764383 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f81080e0 0x7f86f81145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.242+0000 7f86fcc9a640 1 --2- 172.21.15.94:0/3399764383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86f8107720 0x7f86f8107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.242+0000 7f86fcc9a640 1 --2- 172.21.15.94:0/3399764383 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f810d730 0x7f86f810db10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.242+0000 7f86fcc9a640 1 -- 172.21.15.94:0/3399764383 >> 172.21.15.94:0/3399764383 conn(0x7f86f80756c0 msgr2=0x7f86f8077ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.242+0000 7f86fcc9a640 1 -- 172.21.15.94:0/3399764383 shutdown_connections 2024-02-17T12:13:17.245 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.242+0000 7f86fcc9a640 1 -- 172.21.15.94:0/3399764383 wait complete. 2024-02-17T12:13:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.243+0000 7f86fcc9a640 1 Processor -- start 2024-02-17T12:13:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.243+0000 7f86fcc9a640 1 -- start start 2024-02-17T12:13:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86fcc9a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f8107720 0x7f86f819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86fcc9a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86f81080e0 0x7f86f819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86fcc9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f810d730 0x7f86f81a2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.247 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86fcc9a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f81a2d40 con 0x7f86f810d730 2024-02-17T12:13:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86fcc9a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f81a2eb0 con 0x7f86f8107720 2024-02-17T12:13:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86fcc9a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f81a3070 con 0x7f86f81080e0 2024-02-17T12:13:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86f659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f8107720 0x7f86f819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86effff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86f81080e0 0x7f86f819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86f659c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f8107720 0x7f86f819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55154/0 (socket says 172.21.15.94:55154) 2024-02-17T12:13:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86f659c640 1 -- 172.21.15.94:0/1582904578 learned_addr learned my addr 172.21.15.94:0/1582904578 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/1582904578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f810d730 0x7f86f81a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86f659c640 1 -- 172.21.15.94:0/1582904578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86f81080e0 msgr2=0x7f86f819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86f659c640 1 --2- 172.21.15.94:0/1582904578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86f81080e0 0x7f86f819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86f659c640 1 -- 172.21.15.94:0/1582904578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f810d730 msgr2=0x7f86f81a2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.248 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86f659c640 1 --2- 172.21.15.94:0/1582904578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f810d730 0x7f86f81a2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.244+0000 7f86f659c640 1 -- 172.21.15.94:0/1582904578 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0009560 con 0x7f86f8107720 2024-02-17T12:13:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.245+0000 7f86f659c640 1 --2- 172.21.15.94:0/1582904578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f8107720 0x7f86f819ded0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f86e0000c00 tx=0x7f86e0004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.245+0000 7f86ef7fe640 1 -- 172.21.15.94:0/1582904578 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00043c0 con 0x7f86f8107720 2024-02-17T12:13:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.245+0000 7f86ef7fe640 1 -- 172.21.15.94:0/1582904578 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86e00052d0 con 0x7f86f8107720 2024-02-17T12:13:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.245+0000 7f86ef7fe640 1 -- 172.21.15.94:0/1582904578 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0003c70 con 0x7f86f8107720 2024-02-17T12:13:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.246+0000 7f86fcc9a640 1 -- 172.21.15.94:0/1582904578 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f81a32f0 con 0x7f86f8107720 2024-02-17T12:13:17.249 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.246+0000 7f86fcc9a640 1 -- 172.21.15.94:0/1582904578 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f81aabd0 con 0x7f86f8107720 2024-02-17T12:13:17.250 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.248+0000 7f86ff86b740 1 Processor -- start 2024-02-17T12:13:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.248+0000 7f86ff86b740 1 -- start start 2024-02-17T12:13:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.248+0000 7f86ff86b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899250 0x562687899630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.248+0000 7f86ff86b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562687899b70 0x5626878a2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.248+0000 7f86ff86b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5626878a2750 0x5626878a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.248+0000 7f86ff86b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562687854960 con 0x562687899250 2024-02-17T12:13:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.248+0000 7f86ff86b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562687808310 con 0x5626878a2750 2024-02-17T12:13:17.258 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.248+0000 7f86ff86b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562687805840 con 0x562687899b70 2024-02-17T12:13:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.249+0000 7f86f6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899250 0x562687899630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.249+0000 7f86f6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899250 0x562687899630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56290/0 (socket says 172.21.15.94:56290) 2024-02-17T12:13:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.249+0000 7f86f6d9d640 1 -- 172.21.15.94:0/2752989306 learned_addr learned my addr 172.21.15.94:0/2752989306 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.249+0000 7f86effff640 1 --2- 172.21.15.94:0/2752989306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5626878a2750 0x5626878a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.249+0000 7f86f6d9d640 1 -- 172.21.15.94:0/2752989306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562687899b70 msgr2=0x5626878a2210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-17T12:13:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.249+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/2752989306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562687899b70 0x5626878a2210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.249+0000 7f86f6d9d640 1 -- 172.21.15.94:0/2752989306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5626878a2750 msgr2=0x5626878a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.249+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/2752989306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5626878a2750 0x5626878a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.249+0000 7f86f6d9d640 1 -- 172.21.15.94:0/2752989306 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626877c3060 con 0x562687899250 2024-02-17T12:13:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.249+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/2752989306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899250 0x562687899630 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009fd0 tx=0x7f86f00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4094076f6a993bec server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.249+0000 7f86ecff9640 1 -- 172.21.15.94:0/2752989306 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0004d20 con 0x562687899250 2024-02-17T12:13:17.259 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.249+0000 7f86ef7fe640 1 -- 172.21.15.94:0/1582904578 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f86e0005440 con 0x7f86f8107720 2024-02-17T12:13:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.249+0000 7f86ecff9640 1 -- 172.21.15.94:0/2752989306 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86f0004e80 con 0x562687899250 2024-02-17T12:13:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.250+0000 7f86ef7fe640 1 --2- 172.21.15.94:0/1582904578 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f86e40a22f0 0x7f86e40a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.250+0000 7f86ecff9640 1 -- 172.21.15.94:0/2752989306 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0005280 con 0x562687899250 2024-02-17T12:13:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.250+0000 7f86ef7fe640 1 -- 172.21.15.94:0/1582904578 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f86e00f07f0 con 0x7f86f8107720 2024-02-17T12:13:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.250+0000 7f86f659c640 1 --2- 172.21.15.94:0/1582904578 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f86e40a22f0 0x7f86e40a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.250+0000 7f86ff86b740 1 -- 172.21.15.94:0/2752989306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899250 msgr2=0x562687899630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.250+0000 7f86ff86b740 1 --2- 172.21.15.94:0/2752989306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899250 0x562687899630 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009fd0 tx=0x7f86f00118d0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.251+0000 7f86ff86b740 1 -- 172.21.15.94:0/2752989306 shutdown_connections 2024-02-17T12:13:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.251+0000 7f86ff86b740 1 --2- 172.21.15.94:0/2752989306 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5626878a2750 0x5626878a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.251+0000 7f86ff86b740 1 --2- 172.21.15.94:0/2752989306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562687899b70 0x5626878a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.251+0000 7f86ff86b740 1 --2- 172.21.15.94:0/2752989306 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899250 0x562687899630 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.260 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.251+0000 7f86ff86b740 1 -- 172.21.15.94:0/2752989306 >> 172.21.15.94:0/2752989306 conn(0x562687893980 msgr2=0x562687893d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.251+0000 7f86ff86b740 1 -- 172.21.15.94:0/2752989306 shutdown_connections 2024-02-17T12:13:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.251+0000 7f86f659c640 1 --2- 172.21.15.94:0/1582904578 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f86e40a22f0 0x7f86e40a47b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x562687898d20 tx=0x7f86e002bc40 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.251+0000 7f86ff86b740 1 -- 172.21.15.94:0/2752989306 wait complete. 2024-02-17T12:13:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.251+0000 7f86ff86b740 1 -- start start 2024-02-17T12:13:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.251+0000 7f86ff86b740 1 Processor -- start 2024-02-17T12:13:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86ff86b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x56268789f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.261 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86ff86b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 0x56268789fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86ff86b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626878a2750 0x5626878a0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86ff86b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562687801020 con 0x562687899b70 2024-02-17T12:13:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86ff86b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626874d4430 con 0x562687899250 2024-02-17T12:13:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86ff86b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562687490e00 con 0x5626878a2750 2024-02-17T12:13:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86ff86b740 10 client.0 _ll_register_callbacks cb 0x5626877a8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 0x56268789fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 0x56268789fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56292/0 (socket says 172.21.15.94:56292) 2024-02-17T12:13:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86effff640 1 -- 172.21.15.94:0/2558418427 learned_addr learned my addr 172.21.15.94:0/2558418427 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x56268789f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626878a2750 0x5626878a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.262 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86effff640 1 -- 172.21.15.94:0/2558418427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626878a2750 msgr2=0x5626878a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86effff640 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626878a2750 0x5626878a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86effff640 1 -- 172.21.15.94:0/2558418427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 msgr2=0x56268789f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86effff640 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x56268789f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86effff640 1 -- 172.21.15.94:0/2558418427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f0009c80 con 0x562687899b70 2024-02-17T12:13:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626878a2750 0x5626878a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:13:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86effff640 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 0x56268789fbd0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f86e8011570 tx=0x7f86e8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3781a725675036 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:17.263 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86ddffb640 1 -- 172.21.15.94:0/2558418427 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8013670 con 0x562687899b70 2024-02-17T12:13:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86ddffb640 1 -- 172.21.15.94:0/2558418427 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f86e8018070 con 0x562687899b70 2024-02-17T12:13:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.252+0000 7f86ff86b740 1 -- 172.21.15.94:0/2558418427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56268753d720 con 0x562687899b70 2024-02-17T12:13:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.253+0000 7f86ddffb640 1 -- 172.21.15.94:0/2558418427 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86e80137d0 con 0x562687899b70 2024-02-17T12:13:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.253+0000 7f86ff86b740 10 client.19449 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.253+0000 7f86ff86b740 1 -- 172.21.15.94:0/2558418427 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5626877ff6a0 con 0x562687899b70 2024-02-17T12:13:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.253+0000 7f86ddffb640 1 -- 172.21.15.94:0/2558418427 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86e8029080 con 0x562687899b70 2024-02-17T12:13:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.253+0000 7f86ddffb640 1 -- 172.21.15.94:0/2558418427 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f86e801dc00 con 0x562687899b70 2024-02-17T12:13:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.253+0000 7f86ff86b740 10 client.19449 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.253+0000 7f86ff86b740 10 client.19449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.253+0000 7f86ff86b740 20 client.19449 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:17.264 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.253+0000 7f86ff86b740 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5626878b2c40 0x5626878b5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.253+0000 7f86ff86b740 4 client.19449 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:13:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.253+0000 7f86ff86b740 1 -- 172.21.15.94:0/2558418427 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56268758c890 con 0x5626878b2c40 2024-02-17T12:13:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.253+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5626878b2c40 0x5626878b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.254+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5626878b2c40 0x5626878b5060 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=950ceb30321831f4 server_cookie=9eed0b703d624e95 in_seq=0 out_seq=0 2024-02-17T12:13:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.254+0000 7f86ddffb640 10 client.19449 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.254+0000 7f86ddffb640 1 -- 172.21.15.94:0/2558418427 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56268758c890 con 0x5626878b2c40 2024-02-17T12:13:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.254+0000 7f86ddffb640 10 client.19449 handle_command_reply: tid=0 2024-02-17T12:13:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.255+0000 7f86ff86b740 1 client.19449 shutdown 2024-02-17T12:13:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.255+0000 7f86ff86b740 1 -- 172.21.15.94:0/2558418427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 msgr2=0x56268789fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.255+0000 7f86ff86b740 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 0x56268789fbd0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f86e8011570 tx=0x7f86e8009f90 comp rx=0 tx=0).stop 2024-02-17T12:13:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.255+0000 7f86ff86b740 1 -- 172.21.15.94:0/2558418427 shutdown_connections 2024-02-17T12:13:17.265 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.255+0000 7f86ff86b740 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x5626878b2c40 0x5626878b5060 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.255+0000 7f86ff86b740 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626878a2750 0x5626878a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.255+0000 7f86ff86b740 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 0x56268789fbd0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.255+0000 7f86ff86b740 1 --2- 172.21.15.94:0/2558418427 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x56268789f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.255+0000 7f86ddffb640 0 client.19449 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.255+0000 7f86ff86b740 1 -- 172.21.15.94:0/2558418427 >> 172.21.15.94:0/2558418427 conn(0x562687893980 msgr2=0x5626878956d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.255+0000 7f86ff86b740 1 -- 172.21.15.94:0/2558418427 shutdown_connections 2024-02-17T12:13:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.255+0000 7f86ff86b740 1 -- 172.21.15.94:0/2558418427 wait complete. 2024-02-17T12:13:17.266 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.255+0000 7f86ff86b740 20 client.19449 trim_cache size 0 max 16384 2024-02-17T12:13:17.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86ff86b740 1 Processor -- start 2024-02-17T12:13:17.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86ff86b740 1 -- start start 2024-02-17T12:13:17.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86ff86b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x5626878a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86ff86b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562687899b70 0x56268789ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86ff86b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626878a2750 0x56268789e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86ff86b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x562687786130 con 0x5626878a2750 2024-02-17T12:13:17.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86ff86b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626878949e0 con 0x562687899250 2024-02-17T12:13:17.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86ff86b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626878a4f40 con 0x562687899b70 2024-02-17T12:13:17.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86f6d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x5626878a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86f6d9d640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x5626878a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55182/0 (socket says 172.21.15.94:55182) 2024-02-17T12:13:17.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86f6d9d640 1 -- 172.21.15.94:0/3384926291 learned_addr learned my addr 172.21.15.94:0/3384926291 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:17.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/3384926291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626878a2750 0x56268789e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86effff640 1 --2- 172.21.15.94:0/3384926291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562687899b70 0x56268789ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.279 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86f6d9d640 1 -- 172.21.15.94:0/3384926291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562687899b70 msgr2=0x56268789ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/3384926291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562687899b70 0x56268789ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86f6d9d640 1 -- 172.21.15.94:0/3384926291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626878a2750 msgr2=0x56268789e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/3384926291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626878a2750 0x56268789e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.269+0000 7f86f6d9d640 1 -- 172.21.15.94:0/3384926291 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56268779bc20 con 0x562687899250 2024-02-17T12:13:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.270+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/3384926291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x5626878a1dc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f86f00046c0 tx=0x7f86f0027040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=812bf62ed5b144fb server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.270+0000 7f86dffff640 1 -- 172.21.15.94:0/3384926291 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0032070 con 0x562687899250 2024-02-17T12:13:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.270+0000 7f86dffff640 1 -- 172.21.15.94:0/3384926291 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86f002b040 con 0x562687899250 2024-02-17T12:13:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.270+0000 7f86dffff640 1 -- 172.21.15.94:0/3384926291 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f002a040 con 0x562687899250 2024-02-17T12:13:17.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.270+0000 7f86ff86b740 1 -- 172.21.15.94:0/3384926291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 msgr2=0x5626878a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 --2- 172.21.15.94:0/3384926291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x5626878a1dc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f86f00046c0 tx=0x7f86f0027040 comp rx=0 tx=0).stop 2024-02-17T12:13:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 -- 172.21.15.94:0/3384926291 shutdown_connections 2024-02-17T12:13:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 --2- 172.21.15.94:0/3384926291 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5626878a2750 0x56268789e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 --2- 172.21.15.94:0/3384926291 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562687899b70 0x56268789ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 --2- 172.21.15.94:0/3384926291 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x5626878a1dc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 -- 172.21.15.94:0/3384926291 >> 172.21.15.94:0/3384926291 conn(0x562687893980 msgr2=0x56268789df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 -- 172.21.15.94:0/3384926291 shutdown_connections 2024-02-17T12:13:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 -- 172.21.15.94:0/3384926291 wait complete. 2024-02-17T12:13:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 -- start start 2024-02-17T12:13:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 Processor -- start 2024-02-17T12:13:17.281 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x5626878a8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 0x5626878a85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626878a2750 0x5626878a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626878aa000 con 0x562687899b70 2024-02-17T12:13:17.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x562687909170 con 0x562687899250 2024-02-17T12:13:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x562687909300 con 0x5626878a2750 2024-02-17T12:13:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86ff86b740 10 client.0 _ll_register_callbacks cb 0x7f86f0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 0x5626878a85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86effff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 0x5626878a85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56312/0 (socket says 172.21.15.94:56312) 2024-02-17T12:13:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86effff640 1 -- 172.21.15.94:0/871524589 learned_addr learned my addr 172.21.15.94:0/871524589 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:17.283 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x5626878a8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626878a2750 0x5626878a8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86effff640 1 -- 172.21.15.94:0/871524589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626878a2750 msgr2=0x5626878a8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86effff640 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626878a2750 0x5626878a8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86effff640 1 -- 172.21.15.94:0/871524589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 msgr2=0x5626878a8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:17.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.271+0000 7f86effff640 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x5626878a8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86effff640 1 -- 172.21.15.94:0/871524589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626877c3060 con 0x562687899b70 2024-02-17T12:13:17.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x5626878a8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:13:17.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86effff640 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 0x5626878a85a0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x5626878a0c60 tx=0x7f86e8014df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a4e588e121cc4f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:17.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:17.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ddffb640 1 -- 172.21.15.94:0/871524589 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e8009e30 con 0x562687899b70 2024-02-17T12:13:17.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ddffb640 1 -- 172.21.15.94:0/871524589 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f86e8018040 con 0x562687899b70 2024-02-17T12:13:17.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ff86b740 1 -- 172.21.15.94:0/871524589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626875007e0 con 0x562687899b70 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ddffb640 1 -- 172.21.15.94:0/871524589 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86e8009470 con 0x562687899b70 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ff86b740 10 client.19455 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ff86b740 1 -- 172.21.15.94:0/871524589 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5626877c18c0 con 0x562687899b70 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ddffb640 1 -- 172.21.15.94:0/871524589 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5626875007e0 con 0x562687899b70 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ddffb640 1 -- 172.21.15.94:0/871524589 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f86e801da20 con 0x562687899b70 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ff86b740 10 client.19455 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ff86b740 10 client.19455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ff86b740 20 client.19455 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ff86b740 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56268790ab50 0x56268790cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ff86b740 4 client.19455 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.272+0000 7f86ff86b740 1 -- 172.21.15.94:0/871524589 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5626874ac700 con 0x56268790ab50 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56268790ab50 0x56268790cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ddffb640 10 client.19455 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:17.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86f6d9d640 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56268790ab50 0x56268790cf70 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67059c4d269c472d server_cookie=7d6f6591432e9978 in_seq=0 out_seq=0 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ddffb640 1 -- 172.21.15.94:0/871524589 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5626874ac700 con 0x56268790ab50 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ddffb640 10 client.19455 handle_command_reply: tid=0 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ff86b740 1 client.19455 shutdown 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ff86b740 1 -- 172.21.15.94:0/871524589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 msgr2=0x5626878a85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ff86b740 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 0x5626878a85a0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x5626878a0c60 tx=0x7f86e8014df0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ff86b740 1 -- 172.21.15.94:0/871524589 shutdown_connections 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ff86b740 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56268790ab50 0x56268790cf70 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ff86b740 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626878a2750 0x5626878a8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ddffb640 0 client.19455 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ff86b740 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x562687899b70 0x5626878a85a0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ff86b740 1 --2- 172.21.15.94:0/871524589 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x562687899250 0x5626878a8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ff86b740 1 -- 172.21.15.94:0/871524589 >> 172.21.15.94:0/871524589 conn(0x562687893980 msgr2=0x5626878922f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ff86b740 1 -- 172.21.15.94:0/871524589 shutdown_connections 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ff86b740 1 -- 172.21.15.94:0/871524589 wait complete. 2024-02-17T12:13:17.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.273+0000 7f86ff86b740 20 client.19455 trim_cache size 0 max 16384 2024-02-17T12:13:17.287 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:13:17.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.274+0000 7f86fcc9a640 1 -- 172.21.15.94:0/1582904578 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f86e40a22f0 msgr2=0x7f86e40a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.274+0000 7f86fcc9a640 1 --2- 172.21.15.94:0/1582904578 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f86e40a22f0 0x7f86e40a47b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x562687898d20 tx=0x7f86e002bc40 comp rx=0 tx=0).stop 2024-02-17T12:13:17.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.274+0000 7f86fcc9a640 1 -- 172.21.15.94:0/1582904578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f8107720 msgr2=0x7f86f819ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:17.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.274+0000 7f86fcc9a640 1 --2- 172.21.15.94:0/1582904578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f8107720 0x7f86f819ded0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f86e0000c00 tx=0x7f86e0004040 comp rx=0 tx=0).stop 2024-02-17T12:13:17.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.274+0000 7f86fcc9a640 1 -- 172.21.15.94:0/1582904578 shutdown_connections 2024-02-17T12:13:17.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.274+0000 7f86fcc9a640 1 --2- 172.21.15.94:0/1582904578 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f86e40a22f0 0x7f86e40a47b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.274+0000 7f86fcc9a640 1 --2- 172.21.15.94:0/1582904578 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f86f810d730 0x7f86f81a2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.274+0000 7f86fcc9a640 1 --2- 172.21.15.94:0/1582904578 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86f81080e0 0x7f86f819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.274+0000 7f86fcc9a640 1 --2- 172.21.15.94:0/1582904578 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f86f8107720 0x7f86f819ded0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:17.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.274+0000 7f86fcc9a640 1 -- 172.21.15.94:0/1582904578 >> 172.21.15.94:0/1582904578 conn(0x7f86f80756c0 msgr2=0x7f86f8075cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:17.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.274+0000 7f86fcc9a640 1 -- 172.21.15.94:0/1582904578 shutdown_connections 2024-02-17T12:13:17.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:17.274+0000 7f86fcc9a640 1 -- 172.21.15.94:0/1582904578 wait complete. 2024-02-17T12:13:17.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:17 smithi125 ceph-mon[36819]: pgmap v476: 97 pgs: 97 active+clean; 5.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 772 op/s 2024-02-17T12:13:17.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:17 smithi094 ceph-mon[32499]: pgmap v476: 97 pgs: 97 active+clean; 5.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 772 op/s 2024-02-17T12:13:18.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:17 smithi189 ceph-mon[36588]: pgmap v476: 97 pgs: 97 active+clean; 5.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 13 MiB/s wr, 772 op/s 2024-02-17T12:13:18.290 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-17T12:13:18.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:13:18.290 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-17T12:13:18.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.443+0000 7f37f4f3e640 1 -- 172.21.15.94:0/2949780106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37f0101380 msgr2=0x7f37f0101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.443+0000 7f37f4f3e640 1 --2- 172.21.15.94:0/2949780106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37f0101380 0x7f37f0101760 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009930 tx=0x7f37e002bc30 comp rx=0 tx=0).stop 2024-02-17T12:13:18.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.443+0000 7f37f4f3e640 1 -- 172.21.15.94:0/2949780106 shutdown_connections 2024-02-17T12:13:18.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.443+0000 7f37f4f3e640 1 --2- 172.21.15.94:0/2949780106 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37f0106450 0x7f37f010e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.443+0000 7f37f4f3e640 1 --2- 172.21.15.94:0/2949780106 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37f0101d30 0x7f37f0105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.443+0000 7f37f4f3e640 1 --2- 172.21.15.94:0/2949780106 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37f0101380 0x7f37f0101760 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.443+0000 7f37f4f3e640 1 -- 172.21.15.94:0/2949780106 >> 172.21.15.94:0/2949780106 conn(0x7f37f0078580 msgr2=0x7f37f0078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:18.446 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.443+0000 7f37f4f3e640 1 -- 172.21.15.94:0/2949780106 shutdown_connections 2024-02-17T12:13:18.447 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.443+0000 7f37f4f3e640 1 -- 172.21.15.94:0/2949780106 wait complete. 2024-02-17T12:13:18.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.444+0000 7f37f4f3e640 1 Processor -- start 2024-02-17T12:13:18.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.445+0000 7f37f4f3e640 1 -- start start 2024-02-17T12:13:18.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.445+0000 7f37f4f3e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37f0101380 0x7f37f0073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.445+0000 7f37f4f3e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37f0106450 0x7f37f00739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.445+0000 7f37f4f3e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37f0075120 0x7f37f0073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.445+0000 7f37f4f3e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01aa3d0 con 0x7f37f0075120 2024-02-17T12:13:18.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.445+0000 7f37f4f3e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01aa540 con 0x7f37f0106450 2024-02-17T12:13:18.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.445+0000 7f37f4f3e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f01aa700 con 0x7f37f0101380 2024-02-17T12:13:18.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.445+0000 7f37ee59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37f0101380 0x7f37f0073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.445+0000 7f37eed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37f0075120 0x7f37f0073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.445+0000 7f37ee59c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37f0101380 0x7f37f0073480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:45974/0 (socket says 172.21.15.94:45974) 2024-02-17T12:13:18.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.445+0000 7f37ee59c640 1 -- 172.21.15.94:0/138618865 learned_addr learned my addr 172.21.15.94:0/138618865 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:18.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.446+0000 7f37ee59c640 1 -- 172.21.15.94:0/138618865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37f0106450 msgr2=0x7f37f00739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.446+0000 7f37ee59c640 1 --2- 172.21.15.94:0/138618865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37f0106450 0x7f37f00739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.446+0000 7f37ee59c640 1 -- 172.21.15.94:0/138618865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37f0075120 msgr2=0x7f37f0073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.446+0000 7f37ee59c640 1 --2- 172.21.15.94:0/138618865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37f0075120 0x7f37f0073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.446+0000 7f37ee59c640 1 -- 172.21.15.94:0/138618865 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e0009560 con 0x7f37f0101380 2024-02-17T12:13:18.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.446+0000 7f37ee59c640 1 --2- 172.21.15.94:0/138618865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37f0101380 0x7f37f0073480 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009230 tx=0x7f37e002c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:18.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.446+0000 7f37df7fe640 1 -- 172.21.15.94:0/138618865 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e0004350 con 0x7f37f0101380 2024-02-17T12:13:18.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.446+0000 7f37f4f3e640 1 -- 172.21.15.94:0/138618865 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f01aa980 con 0x7f37f0101380 2024-02-17T12:13:18.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.446+0000 7f37df7fe640 1 -- 172.21.15.94:0/138618865 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37e00044b0 con 0x7f37f0101380 2024-02-17T12:13:18.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.446+0000 7f37df7fe640 1 -- 172.21.15.94:0/138618865 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e0003d00 con 0x7f37f0101380 2024-02-17T12:13:18.450 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.446+0000 7f37f4f3e640 1 -- 172.21.15.94:0/138618865 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f01aae90 con 0x7f37f0101380 2024-02-17T12:13:18.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.448+0000 7f37df7fe640 1 -- 172.21.15.94:0/138618865 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f37e0004df0 con 0x7f37f0101380 2024-02-17T12:13:18.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.448+0000 7f37f7bed740 1 Processor -- start 2024-02-17T12:13:18.451 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.449+0000 7f37f7bed740 1 -- start start 2024-02-17T12:13:18.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.449+0000 7f37f7bed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323380 0x559552323760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.449+0000 7f37f7bed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323d30 0x55955232c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.449+0000 7f37f7bed740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 0x55955232ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.449+0000 7f37f7bed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559552291f90 con 0x559552323d30 2024-02-17T12:13:18.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.449+0000 7f37f7bed740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955228f4c0 con 0x55955232c8b0 2024-02-17T12:13:18.452 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.449+0000 7f37f7bed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955228aca0 con 0x559552323380 2024-02-17T12:13:18.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.449+0000 7f37eed9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323380 0x559552323760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.449+0000 7f37df7fe640 1 --2- 172.21.15.94:0/138618865 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f37d80a2230 0x7f37d80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.449+0000 7f37eed9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323380 0x559552323760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:45986/0 (socket says 172.21.15.94:45986) 2024-02-17T12:13:18.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.449+0000 7f37eed9d640 1 -- 172.21.15.94:0/604737278 learned_addr learned my addr 172.21.15.94:0/604737278 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:18.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.449+0000 7f37ee59c640 1 --2- 172.21.15.94:0/604737278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323d30 0x55955232c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.450+0000 7f37edd9b640 1 --2- 172.21.15.94:0/604737278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 0x55955232ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.450+0000 7f37eed9d640 1 --2- 172.21.15.94:0/138618865 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f37d80a2230 0x7f37d80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.450+0000 7f37df7fe640 1 -- 172.21.15.94:0/138618865 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f37e00f00c0 con 0x7f37f0101380 2024-02-17T12:13:18.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.450+0000 7f37eed9d640 1 -- 172.21.15.94:0/604737278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 msgr2=0x55955232ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.450+0000 7f37eed9d640 1 --2- 172.21.15.94:0/604737278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 0x55955232ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.450+0000 7f37eed9d640 1 -- 172.21.15.94:0/604737278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323d30 msgr2=0x55955232c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.450+0000 7f37eed9d640 1 --2- 172.21.15.94:0/604737278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323d30 0x55955232c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.453 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.450+0000 7f37eed9d640 1 -- 172.21.15.94:0/604737278 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55955224d0b0 con 0x559552323380 2024-02-17T12:13:18.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.450+0000 7f37ee59c640 1 --2- 172.21.15.94:0/604737278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323d30 0x55955232c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:18.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.451+0000 7f37eed9d640 1 --2- 172.21.15.94:0/604737278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323380 0x559552323760 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f37e8013d90 tx=0x7f37e800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff9185f957a33ee8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:18.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.451+0000 7f37eed9d640 1 --2- 172.21.15.94:0/138618865 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f37d80a2230 0x7f37d80a46f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f37e800fc90 tx=0x7f37e801e000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:18.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.451+0000 7f37dcff9640 1 -- 172.21.15.94:0/604737278 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8013580 con 0x559552323380 2024-02-17T12:13:18.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.451+0000 7f37dcff9640 1 -- 172.21.15.94:0/604737278 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37e8013870 con 0x559552323380 2024-02-17T12:13:18.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.451+0000 7f37dcff9640 1 -- 172.21.15.94:0/604737278 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8005220 con 0x559552323380 2024-02-17T12:13:18.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 -- 172.21.15.94:0/604737278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323380 msgr2=0x559552323760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 --2- 172.21.15.94:0/604737278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323380 0x559552323760 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f37e8013d90 tx=0x7f37e800f790 comp rx=0 tx=0).stop 2024-02-17T12:13:18.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 -- 172.21.15.94:0/604737278 shutdown_connections 2024-02-17T12:13:18.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 --2- 172.21.15.94:0/604737278 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 0x55955232ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 --2- 172.21.15.94:0/604737278 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323d30 0x55955232c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 --2- 172.21.15.94:0/604737278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323380 0x559552323760 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 -- 172.21.15.94:0/604737278 >> 172.21.15.94:0/604737278 conn(0x55955231d840 msgr2=0x55955231dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 -- 172.21.15.94:0/604737278 shutdown_connections 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 -- 172.21.15.94:0/604737278 wait complete. 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 -- start start 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 Processor -- start 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 0x5595523297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 0x559552329d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 0x55955232a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955222ff40 con 0x559552323380 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559551f5e4b0 con 0x55955232c8b0 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559552210670 con 0x559552323d30 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.452+0000 7f37f7bed740 10 client.0 _ll_register_callbacks cb 0x55955227df70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.453+0000 7f37edd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 0x559552329d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.453+0000 7f37ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 0x5595523297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.453+0000 7f37edd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 0x559552329d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:45996/0 (socket says 172.21.15.94:45996) 2024-02-17T12:13:18.457 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.453+0000 7f37ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 0x5595523297d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56336/0 (socket says 172.21.15.94:56336) 2024-02-17T12:13:18.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.453+0000 7f37edd9b640 1 -- 172.21.15.94:0/591259673 learned_addr learned my addr 172.21.15.94:0/591259673 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:18.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.453+0000 7f37eed9d640 1 --2- 172.21.15.94:0/591259673 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 0x55955232a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.453+0000 7f37ee59c640 1 -- 172.21.15.94:0/591259673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 msgr2=0x559552329d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.453+0000 7f37ee59c640 1 --2- 172.21.15.94:0/591259673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 0x559552329d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.453+0000 7f37ee59c640 1 -- 172.21.15.94:0/591259673 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 msgr2=0x55955232a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.453+0000 7f37ee59c640 1 --2- 172.21.15.94:0/591259673 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 0x55955232a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.453+0000 7f37ee59c640 1 -- 172.21.15.94:0/591259673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e801b040 con 0x559552323380 2024-02-17T12:13:18.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.454+0000 7f37ee59c640 1 --2- 172.21.15.94:0/591259673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 0x5595523297d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f37e00ed880 tx=0x7f37e002c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b2d3447dcd762f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:18.458 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.454+0000 7f37d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.454+0000 7f37d5ffb640 1 -- 172.21.15.94:0/591259673 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e00bd220 con 0x559552323380 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.454+0000 7f37d5ffb640 1 -- 172.21.15.94:0/591259673 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f37e00bcb20 con 0x559552323380 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.454+0000 7f37f7bed740 1 -- 172.21.15.94:0/591259673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559552264f40 con 0x559552323380 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.454+0000 7f37d5ffb640 1 -- 172.21.15.94:0/591259673 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37e00c0c70 con 0x559552323380 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.455+0000 7f37f7bed740 10 client.19467 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.455+0000 7f37f7bed740 1 -- 172.21.15.94:0/591259673 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559552289320 con 0x559552323380 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.455+0000 7f37d5ffb640 1 -- 172.21.15.94:0/591259673 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x559552264f40 con 0x559552323380 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.455+0000 7f37d5ffb640 1 -- 172.21.15.94:0/591259673 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f37e003c070 con 0x559552323380 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.455+0000 7f37f7bed740 10 client.19467 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.455+0000 7f37f7bed740 10 client.19467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.455+0000 7f37f7bed740 20 client.19467 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.455+0000 7f37f7bed740 1 --2- 172.21.15.94:0/591259673 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55955233cc50 0x55955233f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.456+0000 7f37f7bed740 4 client.19467 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:13:18.461 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.456+0000 7f37f7bed740 1 -- 172.21.15.94:0/591259673 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559552281310 con 0x55955233cc50 2024-02-17T12:13:18.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.456+0000 7f37edd9b640 1 --2- 172.21.15.94:0/591259673 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55955233cc50 0x55955233f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.456+0000 7f37edd9b640 1 --2- 172.21.15.94:0/591259673 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55955233cc50 0x55955233f070 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b996ce0f510c4a8a server_cookie=67401546236aaa63 in_seq=0 out_seq=0 2024-02-17T12:13:18.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.456+0000 7f37d5ffb640 10 client.19467 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:18.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37d5ffb640 1 -- 172.21.15.94:0/591259673 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559552281310 con 0x55955233cc50 2024-02-17T12:13:18.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37d5ffb640 10 client.19467 handle_command_reply: tid=0 2024-02-17T12:13:18.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37f7bed740 1 client.19467 shutdown 2024-02-17T12:13:18.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37f7bed740 1 -- 172.21.15.94:0/591259673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 msgr2=0x5595523297d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37f7bed740 1 --2- 172.21.15.94:0/591259673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 0x5595523297d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f37e00ed880 tx=0x7f37e002c740 comp rx=0 tx=0).stop 2024-02-17T12:13:18.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37f7bed740 1 -- 172.21.15.94:0/591259673 shutdown_connections 2024-02-17T12:13:18.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37f7bed740 1 --2- 172.21.15.94:0/591259673 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55955233cc50 0x55955233f070 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37f7bed740 1 --2- 172.21.15.94:0/591259673 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 0x55955232a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37f7bed740 1 --2- 172.21.15.94:0/591259673 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 0x559552329d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.462 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37f7bed740 1 --2- 172.21.15.94:0/591259673 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 0x5595523297d0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37d5ffb640 0 client.19467 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:18.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37f7bed740 1 -- 172.21.15.94:0/591259673 >> 172.21.15.94:0/591259673 conn(0x55955231d840 msgr2=0x55955231f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:18.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37f7bed740 1 -- 172.21.15.94:0/591259673 shutdown_connections 2024-02-17T12:13:18.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37f7bed740 1 -- 172.21.15.94:0/591259673 wait complete. 2024-02-17T12:13:18.463 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.457+0000 7f37f7bed740 20 client.19467 trim_cache size 0 max 16384 2024-02-17T12:13:18.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.476+0000 7f37f7bed740 1 Processor -- start 2024-02-17T12:13:18.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.476+0000 7f37f7bed740 1 -- start start 2024-02-17T12:13:18.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.476+0000 7f37f7bed740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559552323380 0x55955231c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.480 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.476+0000 7f37f7bed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 0x55955232a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.476+0000 7f37f7bed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55955232c8b0 0x559552333880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.476+0000 7f37f7bed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55955233ca20 con 0x55955232c8b0 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.476+0000 7f37f7bed740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x559552322cd0 con 0x559552323380 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.476+0000 7f37f7bed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x559552264f40 con 0x559552323d30 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.476+0000 7f37eed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55955232c8b0 0x559552333880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.476+0000 7f37edd9b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 0x55955232a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.476+0000 7f37ee59c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559552323380 0x55955231c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.476+0000 7f37eed9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55955232c8b0 0x559552333880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56350/0 (socket says 172.21.15.94:56350) 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.476+0000 7f37eed9d640 1 -- 172.21.15.94:0/3839287494 learned_addr learned my addr 172.21.15.94:0/3839287494 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.477+0000 7f37eed9d640 1 -- 172.21.15.94:0/3839287494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 msgr2=0x55955232a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.477+0000 7f37eed9d640 1 --2- 172.21.15.94:0/3839287494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 0x55955232a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.477+0000 7f37eed9d640 1 -- 172.21.15.94:0/3839287494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559552323380 msgr2=0x55955231c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.477+0000 7f37eed9d640 1 --2- 172.21.15.94:0/3839287494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559552323380 0x55955231c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.481 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.477+0000 7f37eed9d640 1 -- 172.21.15.94:0/3839287494 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559552225c00 con 0x55955232c8b0 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.477+0000 7f37eed9d640 1 --2- 172.21.15.94:0/3839287494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55955232c8b0 0x559552333880 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f37e8013d90 tx=0x7f37e8009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97028d2b74fc7784 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.477+0000 7f37d7fff640 1 -- 172.21.15.94:0/3839287494 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e802c070 con 0x55955232c8b0 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.477+0000 7f37d7fff640 1 -- 172.21.15.94:0/3839287494 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f37e8013580 con 0x55955232c8b0 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.477+0000 7f37d7fff640 1 -- 172.21.15.94:0/3839287494 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e8004a00 con 0x55955232c8b0 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.478+0000 7f37f7bed740 1 -- 172.21.15.94:0/3839287494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55955232c8b0 msgr2=0x559552333880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.478+0000 7f37f7bed740 1 --2- 172.21.15.94:0/3839287494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55955232c8b0 0x559552333880 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f37e8013d90 tx=0x7f37e8009f40 comp rx=0 tx=0).stop 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.478+0000 7f37f7bed740 1 -- 172.21.15.94:0/3839287494 shutdown_connections 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.478+0000 7f37f7bed740 1 --2- 172.21.15.94:0/3839287494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55955232c8b0 0x559552333880 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.478+0000 7f37f7bed740 1 --2- 172.21.15.94:0/3839287494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 0x55955232a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.478+0000 7f37f7bed740 1 --2- 172.21.15.94:0/3839287494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x559552323380 0x55955231c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.478+0000 7f37f7bed740 1 -- 172.21.15.94:0/3839287494 >> 172.21.15.94:0/3839287494 conn(0x55955231d840 msgr2=0x55955231c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.478+0000 7f37f7bed740 1 -- 172.21.15.94:0/3839287494 shutdown_connections 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.478+0000 7f37f7bed740 1 -- 172.21.15.94:0/3839287494 wait complete. 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.478+0000 7f37f7bed740 1 -- start start 2024-02-17T12:13:18.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.478+0000 7f37f7bed740 1 Processor -- start 2024-02-17T12:13:18.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.478+0000 7f37f7bed740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 0x559552332560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37f7bed740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 0x559552332aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37f7bed740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 0x5595523931a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37f7bed740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x559552333260 con 0x559552323380 2024-02-17T12:13:18.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37f7bed740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595523333d0 con 0x55955232c8b0 2024-02-17T12:13:18.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37f7bed740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595523936e0 con 0x559552323d30 2024-02-17T12:13:18.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37f7bed740 10 client.0 _ll_register_callbacks cb 0x7f37e0009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:18.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 0x559552332560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37ee59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 0x559552332560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56354/0 (socket says 172.21.15.94:56354) 2024-02-17T12:13:18.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37ee59c640 1 -- 172.21.15.94:0/1843376601 learned_addr learned my addr 172.21.15.94:0/1843376601 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:18.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37eed9d640 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 0x5595523931a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37edd9b640 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 0x559552332aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37ee59c640 1 -- 172.21.15.94:0/1843376601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 msgr2=0x559552332aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37ee59c640 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 0x559552332aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37ee59c640 1 -- 172.21.15.94:0/1843376601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 msgr2=0x5595523931a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37ee59c640 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 0x5595523931a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37ee59c640 1 -- 172.21.15.94:0/1843376601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55955224d0b0 con 0x559552323380 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.479+0000 7f37eed9d640 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 0x5595523931a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37ee59c640 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 0x559552332560 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f37e00b94a0 tx=0x7f37e0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6256f226eb10f6e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37d5ffb640 1 -- 172.21.15.94:0/1843376601 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e00edbf0 con 0x559552323380 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37d5ffb640 1 -- 172.21.15.94:0/1843376601 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f37e00bd220 con 0x559552323380 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37f7bed740 1 -- 172.21.15.94:0/1843376601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595522b0900 con 0x559552323380 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37d5ffb640 1 -- 172.21.15.94:0/1843376601 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37e00bc660 con 0x559552323380 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37f7bed740 10 client.19473 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37f7bed740 1 -- 172.21.15.94:0/1843376601 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55955224b910 con 0x559552323380 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37d5ffb640 1 -- 172.21.15.94:0/1843376601 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5595522b0900 con 0x559552323380 2024-02-17T12:13:18.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37d5ffb640 1 -- 172.21.15.94:0/1843376601 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f37e003c040 con 0x559552323380 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37f7bed740 10 client.19473 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37f7bed740 10 client.19473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37f7bed740 20 client.19473 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37f7bed740 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559552394a10 0x559552396e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37f7bed740 4 client.19473 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37f7bed740 1 -- 172.21.15.94:0/1843376601 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559551f8a860 con 0x559552394a10 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.480+0000 7f37edd9b640 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559552394a10 0x559552396e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.481+0000 7f37edd9b640 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559552394a10 0x559552396e30 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e34366c22f14b554 server_cookie=4876797492670a1a in_seq=0 out_seq=0 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.481+0000 7f37d5ffb640 10 client.19473 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37d5ffb640 1 -- 172.21.15.94:0/1843376601 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559551f8a860 con 0x559552394a10 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37d5ffb640 10 client.19473 handle_command_reply: tid=0 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37f7bed740 1 client.19473 shutdown 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37f7bed740 1 -- 172.21.15.94:0/1843376601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 msgr2=0x559552332560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37f7bed740 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 0x559552332560 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f37e00b94a0 tx=0x7f37e0005fd0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37f7bed740 1 -- 172.21.15.94:0/1843376601 shutdown_connections 2024-02-17T12:13:18.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37f7bed740 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x559552394a10 0x559552396e30 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37f7bed740 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55955232c8b0 0x5595523931a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37f7bed740 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559552323d30 0x559552332aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37d5ffb640 0 client.19473 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:18.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37f7bed740 1 --2- 172.21.15.94:0/1843376601 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x559552323380 0x559552332560 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37f7bed740 1 -- 172.21.15.94:0/1843376601 >> 172.21.15.94:0/1843376601 conn(0x55955231d840 msgr2=0x55955231f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:18.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37f7bed740 1 -- 172.21.15.94:0/1843376601 shutdown_connections 2024-02-17T12:13:18.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37f7bed740 1 -- 172.21.15.94:0/1843376601 wait complete. 2024-02-17T12:13:18.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.482+0000 7f37f7bed740 20 client.19473 trim_cache size 0 max 16384 2024-02-17T12:13:18.489 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:13:18.489 INFO:teuthology.orchestra.run.smithi094.stdout: "return_code": 0, 2024-02-17T12:13:18.490 INFO:teuthology.orchestra.run.smithi094.stdout: "scrub_tag": "1039f0aa-8164-4e96-97fb-22754200622d", 2024-02-17T12:13:18.490 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": "asynchronous" 2024-02-17T12:13:18.490 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:13:18.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.483+0000 7f37f4f3e640 1 -- 172.21.15.94:0/138618865 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f37d80a2230 msgr2=0x7f37d80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.483+0000 7f37f4f3e640 1 --2- 172.21.15.94:0/138618865 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f37d80a2230 0x7f37d80a46f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f37e800fc90 tx=0x7f37e801e000 comp rx=0 tx=0).stop 2024-02-17T12:13:18.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.483+0000 7f37f4f3e640 1 -- 172.21.15.94:0/138618865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37f0101380 msgr2=0x7f37f0073480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.483+0000 7f37f4f3e640 1 --2- 172.21.15.94:0/138618865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37f0101380 0x7f37f0073480 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009230 tx=0x7f37e002c650 comp rx=0 tx=0).stop 2024-02-17T12:13:18.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.483+0000 7f37f4f3e640 1 -- 172.21.15.94:0/138618865 shutdown_connections 2024-02-17T12:13:18.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.483+0000 7f37f4f3e640 1 --2- 172.21.15.94:0/138618865 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f37d80a2230 0x7f37d80a46f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.483+0000 7f37f4f3e640 1 --2- 172.21.15.94:0/138618865 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f37f0075120 0x7f37f0073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.483+0000 7f37f4f3e640 1 --2- 172.21.15.94:0/138618865 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f37f0106450 0x7f37f00739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.483+0000 7f37f4f3e640 1 --2- 172.21.15.94:0/138618865 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37f0101380 0x7f37f0073480 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.483+0000 7f37f4f3e640 1 -- 172.21.15.94:0/138618865 >> 172.21.15.94:0/138618865 conn(0x7f37f0078580 msgr2=0x7f37f00ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:18.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.483+0000 7f37f4f3e640 1 -- 172.21.15.94:0/138618865 shutdown_connections 2024-02-17T12:13:18.491 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.483+0000 7f37f4f3e640 1 -- 172.21.15.94:0/138618865 wait complete. 2024-02-17T12:13:18.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:13:18.498 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:13:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.663+0000 7f363f64e640 1 -- 172.21.15.94:0/4260872466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36380ffb60 msgr2=0x7f36380fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.665 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.663+0000 7f363f64e640 1 --2- 172.21.15.94:0/4260872466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36380ffb60 0x7f36380fffc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3628009930 tx=0x7f362802bc30 comp rx=0 tx=0).stop 2024-02-17T12:13:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.663+0000 7f363f64e640 1 -- 172.21.15.94:0/4260872466 shutdown_connections 2024-02-17T12:13:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.663+0000 7f363f64e640 1 --2- 172.21.15.94:0/4260872466 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f363810dda0 0x7f3638110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.663+0000 7f363f64e640 1 --2- 172.21.15.94:0/4260872466 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36380ffb60 0x7f36380fffc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.663+0000 7f363f64e640 1 --2- 172.21.15.94:0/4260872466 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f3638102220 0x7f36380ff620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.663+0000 7f363f64e640 1 -- 172.21.15.94:0/4260872466 >> 172.21.15.94:0/4260872466 conn(0x7f36380765b0 msgr2=0x7f36380fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.663+0000 7f363f64e640 1 -- 172.21.15.94:0/4260872466 shutdown_connections 2024-02-17T12:13:18.666 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.663+0000 7f363f64e640 1 -- 172.21.15.94:0/4260872466 wait complete. 2024-02-17T12:13:18.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.664+0000 7f363f64e640 1 Processor -- start 2024-02-17T12:13:18.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.664+0000 7f363f64e640 1 -- start start 2024-02-17T12:13:18.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.664+0000 7f363f64e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36380ffb60 0x7f363819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.664+0000 7f363f64e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3638102220 0x7f363819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363f64e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f363810dda0 0x7f3638199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363f64e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3638199fc0 con 0x7f3638102220 2024-02-17T12:13:18.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363f64e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363819a130 con 0x7f36380ffb60 2024-02-17T12:13:18.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363f64e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363819a2f0 con 0x7f363810dda0 2024-02-17T12:13:18.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363d3ea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36380ffb60 0x7f363819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.669 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363d3ea640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36380ffb60 0x7f363819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:55274/0 (socket says 172.21.15.94:55274) 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363d3ea640 1 -- 172.21.15.94:0/2608416331 learned_addr learned my addr 172.21.15.94:0/2608416331 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363dbeb640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f363810dda0 0x7f3638199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363cbe9640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3638102220 0x7f363819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363cbe9640 1 -- 172.21.15.94:0/2608416331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f363810dda0 msgr2=0x7f3638199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363cbe9640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f363810dda0 0x7f3638199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363cbe9640 1 -- 172.21.15.94:0/2608416331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36380ffb60 msgr2=0x7f363819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363cbe9640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36380ffb60 0x7f363819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.665+0000 7f363cbe9640 1 -- 172.21.15.94:0/2608416331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3628009560 con 0x7f3638102220 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.666+0000 7f363cbe9640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3638102220 0x7f363819fd00 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3628009230 tx=0x7f362802c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.666+0000 7f362e7fc640 1 -- 172.21.15.94:0/2608416331 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36280042d0 con 0x7f3638102220 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.666+0000 7f363f64e640 1 -- 172.21.15.94:0/2608416331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363819a570 con 0x7f3638102220 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.666+0000 7f363f64e640 1 -- 172.21.15.94:0/2608416331 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36381a6860 con 0x7f3638102220 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.666+0000 7f362e7fc640 1 -- 172.21.15.94:0/2608416331 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3628004430 con 0x7f3638102220 2024-02-17T12:13:18.670 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.666+0000 7f362e7fc640 1 -- 172.21.15.94:0/2608416331 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628003b30 con 0x7f3638102220 2024-02-17T12:13:18.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.668+0000 7f362e7fc640 1 -- 172.21.15.94:0/2608416331 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f3628003c90 con 0x7f3638102220 2024-02-17T12:13:18.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.669+0000 7f3642229740 1 Processor -- start 2024-02-17T12:13:18.671 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.669+0000 7f3642229740 1 -- start start 2024-02-17T12:13:18.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.669+0000 7f3642229740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3c82b0 0x55714f3c8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.669+0000 7f3642229740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 0x55714f3d1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.672 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.669+0000 7f3642229740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3d17b0 0x55714f3d3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.669+0000 7f3642229740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714f337280 con 0x55714f3d17b0 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.669+0000 7f3642229740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714f3347b0 con 0x55714f3c8bd0 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.669+0000 7f3642229740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714f32ff90 con 0x55714f3c82b0 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.669+0000 7f363dbeb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3c82b0 0x55714f3c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.669+0000 7f363dbeb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3c82b0 0x55714f3c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:46030/0 (socket says 172.21.15.94:46030) 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.669+0000 7f363dbeb640 1 -- 172.21.15.94:0/117003360 learned_addr learned my addr 172.21.15.94:0/117003360 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.669+0000 7f363d3ea640 1 --2- 172.21.15.94:0/117003360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 0x55714f3d1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.670+0000 7f362e7fc640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f36240a22b0 0x7f36240a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.670+0000 7f363dbeb640 1 -- 172.21.15.94:0/117003360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 msgr2=0x55714f3d1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.670+0000 7f363dbeb640 1 --2- 172.21.15.94:0/117003360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 0x55714f3d1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.670+0000 7f363dbeb640 1 -- 172.21.15.94:0/117003360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3d17b0 msgr2=0x55714f3d3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.670+0000 7f363dbeb640 1 --2- 172.21.15.94:0/117003360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3d17b0 0x55714f3d3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.670+0000 7f363dbeb640 1 -- 172.21.15.94:0/117003360 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55714f2ee570 con 0x55714f3c82b0 2024-02-17T12:13:18.673 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.670+0000 7f362e7fc640 1 -- 172.21.15.94:0/2608416331 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f36280efe30 con 0x7f3638102220 2024-02-17T12:13:18.674 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.670+0000 7f363dbeb640 1 --2- 172.21.15.94:0/117003360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3c82b0 0x55714f3c8690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3630009fa0 tx=0x7f3630010980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=835345fa92e3bc29 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:18.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.671+0000 7f36237fe640 1 -- 172.21.15.94:0/117003360 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3630003b70 con 0x55714f3c82b0 2024-02-17T12:13:18.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.671+0000 7f36237fe640 1 -- 172.21.15.94:0/117003360 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3630028070 con 0x55714f3c82b0 2024-02-17T12:13:18.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.671+0000 7f36237fe640 1 -- 172.21.15.94:0/117003360 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3630024a60 con 0x55714f3c82b0 2024-02-17T12:13:18.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.671+0000 7f363dbeb640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f36240a22b0 0x7f36240a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.672+0000 7f363dbeb640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f36240a22b0 0x7f36240a4770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f363002e000 tx=0x7f3630024d10 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:18.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.672+0000 7f3642229740 1 -- 172.21.15.94:0/117003360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3c82b0 msgr2=0x55714f3c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.678 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.672+0000 7f3642229740 1 --2- 172.21.15.94:0/117003360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3c82b0 0x55714f3c8690 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3630009fa0 tx=0x7f3630010980 comp rx=0 tx=0).stop 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.672+0000 7f3642229740 1 -- 172.21.15.94:0/117003360 shutdown_connections 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.672+0000 7f3642229740 1 --2- 172.21.15.94:0/117003360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3d17b0 0x55714f3d3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.672+0000 7f3642229740 1 --2- 172.21.15.94:0/117003360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 0x55714f3d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.672+0000 7f3642229740 1 --2- 172.21.15.94:0/117003360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3c82b0 0x55714f3c8690 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.672+0000 7f3642229740 1 -- 172.21.15.94:0/117003360 >> 172.21.15.94:0/117003360 conn(0x55714f3c2990 msgr2=0x55714f3c2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.672+0000 7f3642229740 1 -- 172.21.15.94:0/117003360 shutdown_connections 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.672+0000 7f3642229740 1 -- 172.21.15.94:0/117003360 wait complete. 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.673+0000 7f3642229740 1 -- start start 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.673+0000 7f3642229740 1 Processor -- start 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.673+0000 7f3642229740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c82b0 0x55714f3d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.673+0000 7f3642229740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3c8bd0 0x55714f3ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.673+0000 7f3642229740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3d17b0 0x55714f3cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.673+0000 7f3642229740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714f003430 con 0x55714f3d17b0 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.673+0000 7f363dbeb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3d17b0 0x55714f3cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.679 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f363cbe9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3c8bd0 0x55714f3ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f363dbeb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3d17b0 0x55714f3cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56378/0 (socket says 172.21.15.94:56378) 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f3642229740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714efbfe00 con 0x55714f3c82b0 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f363dbeb640 1 -- 172.21.15.94:0/3667541892 learned_addr learned my addr 172.21.15.94:0/3667541892 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f363d3ea640 1 --2- 172.21.15.94:0/3667541892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c82b0 0x55714f3d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f3642229740 1 -- 172.21.15.94:0/3667541892 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714f2b4fd0 con 0x55714f3c8bd0 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f3642229740 10 client.0 _ll_register_callbacks cb 0x55714f30afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f363dbeb640 1 -- 172.21.15.94:0/3667541892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3c8bd0 msgr2=0x55714f3ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f363dbeb640 1 --2- 172.21.15.94:0/3667541892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3c8bd0 0x55714f3ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f363dbeb640 1 -- 172.21.15.94:0/3667541892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c82b0 msgr2=0x55714f3d11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f363dbeb640 1 --2- 172.21.15.94:0/3667541892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c82b0 0x55714f3d11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f363dbeb640 1 -- 172.21.15.94:0/3667541892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3630009c40 con 0x55714f3d17b0 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f363dbeb640 1 --2- 172.21.15.94:0/3667541892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3d17b0 0x55714f3cecb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3630013430 tx=0x7f3630035000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5e3a1980fa325f server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f3620ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f3620ff9640 1 -- 172.21.15.94:0/3667541892 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3630004810 con 0x55714f3d17b0 2024-02-17T12:13:18.680 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f3620ff9640 1 -- 172.21.15.94:0/3667541892 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f363002e400 con 0x55714f3d17b0 2024-02-17T12:13:18.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.674+0000 7f3642229740 1 -- 172.21.15.94:0/3667541892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55714f02f7e0 con 0x55714f3d17b0 2024-02-17T12:13:18.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.675+0000 7f3620ff9640 1 -- 172.21.15.94:0/3667541892 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36300249b0 con 0x55714f3d17b0 2024-02-17T12:13:18.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.675+0000 7f3642229740 10 client.19482 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:18.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.675+0000 7f3642229740 1 -- 172.21.15.94:0/3667541892 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55714f32e610 con 0x55714f3d17b0 2024-02-17T12:13:18.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.675+0000 7f3620ff9640 1 -- 172.21.15.94:0/3667541892 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55714f02f7e0 con 0x55714f3d17b0 2024-02-17T12:13:18.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.675+0000 7f3620ff9640 1 -- 172.21.15.94:0/3667541892 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3630035070 con 0x55714f3d17b0 2024-02-17T12:13:18.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.675+0000 7f3642229740 10 client.19482 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:18.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.675+0000 7f3642229740 10 client.19482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:18.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.675+0000 7f3642229740 20 client.19482 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:18.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.675+0000 7f3642229740 1 --2- 172.21.15.94:0/3667541892 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55714f3e18f0 0x55714f3e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.675+0000 7f3642229740 4 client.19482 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:13:18.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.675+0000 7f3642229740 1 -- 172.21.15.94:0/3667541892 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55714f06c720 con 0x55714f3e18f0 2024-02-17T12:13:18.681 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.675+0000 7f363d3ea640 1 --2- 172.21.15.94:0/3667541892 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55714f3e18f0 0x55714f3e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.682 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.677+0000 7f363d3ea640 1 --2- 172.21.15.94:0/3667541892 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55714f3e18f0 0x55714f3e3d10 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12b17ad1392709e3 server_cookie=7ee9c71650aaaf in_seq=0 out_seq=0 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.677+0000 7f3620ff9640 10 client.19482 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.677+0000 7f3620ff9640 1 -- 172.21.15.94:0/3667541892 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55714f06c720 con 0x55714f3e18f0 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3620ff9640 10 client.19482 handle_command_reply: tid=0 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3642229740 1 client.19482 shutdown 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3642229740 1 -- 172.21.15.94:0/3667541892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3d17b0 msgr2=0x55714f3cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3642229740 1 --2- 172.21.15.94:0/3667541892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3d17b0 0x55714f3cecb0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f3630013430 tx=0x7f3630035000 comp rx=0 tx=0).stop 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3642229740 1 -- 172.21.15.94:0/3667541892 shutdown_connections 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3642229740 1 --2- 172.21.15.94:0/3667541892 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55714f3e18f0 0x55714f3e3d10 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3642229740 1 --2- 172.21.15.94:0/3667541892 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3d17b0 0x55714f3cecb0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3642229740 1 --2- 172.21.15.94:0/3667541892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3c8bd0 0x55714f3ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3642229740 1 --2- 172.21.15.94:0/3667541892 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c82b0 0x55714f3d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3642229740 1 -- 172.21.15.94:0/3667541892 >> 172.21.15.94:0/3667541892 conn(0x55714f3c2990 msgr2=0x55714f3c4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3620ff9640 0 client.19482 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:18.683 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3642229740 1 -- 172.21.15.94:0/3667541892 shutdown_connections 2024-02-17T12:13:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3642229740 1 -- 172.21.15.94:0/3667541892 wait complete. 2024-02-17T12:13:18.684 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.678+0000 7f3642229740 20 client.19482 trim_cache size 0 max 16384 2024-02-17T12:13:18.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f3642229740 1 Processor -- start 2024-02-17T12:13:18.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f3642229740 1 -- start start 2024-02-17T12:13:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f3642229740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 0x55714f3d86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f3642229740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 0x55714f3d8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f3642229740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3d17b0 0x55714f3d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f3642229740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714f3c2210 con 0x55714f3c82b0 2024-02-17T12:13:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f3642229740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714f3ca1a0 con 0x55714f3c8bd0 2024-02-17T12:13:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f3642229740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714f3d4480 con 0x55714f3d17b0 2024-02-17T12:13:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f363d3ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 0x55714f3d86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f363d3ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 0x55714f3d86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56394/0 (socket says 172.21.15.94:56394) 2024-02-17T12:13:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f363d3ea640 1 -- 172.21.15.94:0/3557914864 learned_addr learned my addr 172.21.15.94:0/3557914864 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f363cbe9640 1 --2- 172.21.15.94:0/3557914864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 0x55714f3d8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f363dbeb640 1 --2- 172.21.15.94:0/3557914864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3d17b0 0x55714f3d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f363d3ea640 1 -- 172.21.15.94:0/3557914864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3d17b0 msgr2=0x55714f3d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f363d3ea640 1 --2- 172.21.15.94:0/3557914864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3d17b0 0x55714f3d0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f363d3ea640 1 -- 172.21.15.94:0/3557914864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 msgr2=0x55714f3d8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f363d3ea640 1 --2- 172.21.15.94:0/3557914864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 0x55714f3d8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.697+0000 7f363d3ea640 1 -- 172.21.15.94:0/3557914864 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55714f2caac0 con 0x55714f3c82b0 2024-02-17T12:13:18.701 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.698+0000 7f363d3ea640 1 --2- 172.21.15.94:0/3557914864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 0x55714f3d86f0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f3634005d00 tx=0x7f3634005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da429b2c2cac10bc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:18.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.698+0000 7f3622ffd640 1 -- 172.21.15.94:0/3557914864 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634017030 con 0x55714f3c82b0 2024-02-17T12:13:18.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.698+0000 7f3622ffd640 1 -- 172.21.15.94:0/3557914864 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3634018040 con 0x55714f3c82b0 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.698+0000 7f3622ffd640 1 -- 172.21.15.94:0/3557914864 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634010aa0 con 0x55714f3c82b0 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.699+0000 7f3642229740 1 -- 172.21.15.94:0/3557914864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 msgr2=0x55714f3d86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.699+0000 7f3642229740 1 --2- 172.21.15.94:0/3557914864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 0x55714f3d86f0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f3634005d00 tx=0x7f3634005f90 comp rx=0 tx=0).stop 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.699+0000 7f3642229740 1 -- 172.21.15.94:0/3557914864 shutdown_connections 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.699+0000 7f3642229740 1 --2- 172.21.15.94:0/3557914864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3d17b0 0x55714f3d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.699+0000 7f3642229740 1 --2- 172.21.15.94:0/3557914864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 0x55714f3d8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.699+0000 7f3642229740 1 --2- 172.21.15.94:0/3557914864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 0x55714f3d86f0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.699+0000 7f3642229740 1 -- 172.21.15.94:0/3557914864 >> 172.21.15.94:0/3557914864 conn(0x55714f3c2990 msgr2=0x55714f3e8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.699+0000 7f3642229740 1 -- 172.21.15.94:0/3557914864 shutdown_connections 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.699+0000 7f3642229740 1 -- 172.21.15.94:0/3557914864 wait complete. 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.699+0000 7f3642229740 1 -- start start 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.699+0000 7f3642229740 1 Processor -- start 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f3642229740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 0x55714f3d8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f3642229740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 0x55714f3d6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.706 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f3642229740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3d17b0 0x55714f3d75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f3642229740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714f3d7c70 con 0x55714f3c82b0 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f363d3ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 0x55714f3d8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f3642229740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714f3d7de0 con 0x55714f3c8bd0 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f363d3ea640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 0x55714f3d8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56406/0 (socket says 172.21.15.94:56406) 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f363d3ea640 1 -- 172.21.15.94:0/2336363339 learned_addr learned my addr 172.21.15.94:0/2336363339 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f3642229740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55714f440690 con 0x55714f3d17b0 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f363cbe9640 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 0x55714f3d6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f363dbeb640 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3d17b0 0x55714f3d75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f3642229740 10 client.0 _ll_register_callbacks cb 0x7f36300099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f363d3ea640 1 -- 172.21.15.94:0/2336363339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3d17b0 msgr2=0x55714f3d75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f363d3ea640 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3d17b0 0x55714f3d75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f363d3ea640 1 -- 172.21.15.94:0/2336363339 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 msgr2=0x55714f3d6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f363d3ea640 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 0x55714f3d6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f363d3ea640 1 -- 172.21.15.94:0/2336363339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3634009bc0 con 0x55714f3c82b0 2024-02-17T12:13:18.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f363cbe9640 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 0x55714f3d6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f363d3ea640 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 0x55714f3d8b80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3634010eb0 tx=0x7f3634010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfd851386632b34 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.700+0000 7f3620ff9640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.701+0000 7f3620ff9640 1 -- 172.21.15.94:0/2336363339 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363402c070 con 0x55714f3c82b0 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.701+0000 7f3620ff9640 1 -- 172.21.15.94:0/2336363339 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f3634003cd0 con 0x55714f3c82b0 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.701+0000 7f3642229740 1 -- 172.21.15.94:0/2336363339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3630028070 con 0x55714f3c82b0 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.701+0000 7f3620ff9640 1 -- 172.21.15.94:0/2336363339 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3634017040 con 0x55714f3c82b0 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.701+0000 7f3642229740 10 client.19488 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.701+0000 7f3642229740 1 -- 172.21.15.94:0/2336363339 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55714f2f06a0 con 0x55714f3c82b0 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.701+0000 7f3620ff9640 1 -- 172.21.15.94:0/2336363339 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3630028070 con 0x55714f3c82b0 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.701+0000 7f3620ff9640 1 -- 172.21.15.94:0/2336363339 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f3634031660 con 0x55714f3c82b0 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.701+0000 7f3642229740 10 client.19488 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.702+0000 7f3642229740 10 client.19488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.702+0000 7f3642229740 20 client.19488 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:18.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.702+0000 7f3642229740 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55714f441e10 0x55714f444230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.702+0000 7f3642229740 4 client.19488 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.702+0000 7f3642229740 1 -- 172.21.15.94:0/2336363339 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55714f0bb890 con 0x55714f441e10 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.702+0000 7f363cbe9640 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55714f441e10 0x55714f444230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.702+0000 7f363cbe9640 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55714f441e10 0x55714f444230 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b5ac1875945abf9 server_cookie=9ea299898f946e28 in_seq=0 out_seq=0 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.702+0000 7f3620ff9640 10 client.19488 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.702+0000 7f3620ff9640 1 -- 172.21.15.94:0/2336363339 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+267 (crc 0 0 0) 0x55714f0bb890 con 0x55714f441e10 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.702+0000 7f3620ff9640 10 client.19488 handle_command_reply: tid=0 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.702+0000 7f3642229740 1 client.19488 shutdown 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f3642229740 1 -- 172.21.15.94:0/2336363339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 msgr2=0x55714f3d8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f3642229740 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 0x55714f3d8b80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f3634010eb0 tx=0x7f3634010ee0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f3642229740 1 -- 172.21.15.94:0/2336363339 shutdown_connections 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f3642229740 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55714f441e10 0x55714f444230 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f3642229740 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55714f3d17b0 0x55714f3d75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f3642229740 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55714f3c8bd0 0x55714f3d6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f3642229740 1 --2- 172.21.15.94:0/2336363339 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55714f3c82b0 0x55714f3d8b80 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f3642229740 1 -- 172.21.15.94:0/2336363339 >> 172.21.15.94:0/2336363339 conn(0x55714f3c2990 msgr2=0x55714f3cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f3620ff9640 0 client.19488 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f3642229740 1 -- 172.21.15.94:0/2336363339 shutdown_connections 2024-02-17T12:13:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f3642229740 1 -- 172.21.15.94:0/2336363339 wait complete. 2024-02-17T12:13:18.710 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f3642229740 20 client.19488 trim_cache size 0 max 16384 2024-02-17T12:13:18.710 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:13:18.710 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "scrub active (404 inodes in the stack)", 2024-02-17T12:13:18.710 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": { 2024-02-17T12:13:18.710 INFO:teuthology.orchestra.run.smithi094.stdout: "1039f0aa-8164-4e96-97fb-22754200622d": { 2024-02-17T12:13:18.711 INFO:teuthology.orchestra.run.smithi094.stdout: "path": "/", 2024-02-17T12:13:18.711 INFO:teuthology.orchestra.run.smithi094.stdout: "tag": "1039f0aa-8164-4e96-97fb-22754200622d", 2024-02-17T12:13:18.711 INFO:teuthology.orchestra.run.smithi094.stdout: "options": "recursive,force" 2024-02-17T12:13:18.711 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:18.711 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:18.711 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:13:18.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f363f64e640 1 -- 172.21.15.94:0/2608416331 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f36240a22b0 msgr2=0x7f36240a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f363f64e640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f36240a22b0 0x7f36240a4770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f363002e000 tx=0x7f3630024d10 comp rx=0 tx=0).stop 2024-02-17T12:13:18.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f363f64e640 1 -- 172.21.15.94:0/2608416331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3638102220 msgr2=0x7f363819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:18.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.703+0000 7f363f64e640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3638102220 0x7f363819fd00 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3628009230 tx=0x7f362802c5d0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.704+0000 7f363f64e640 1 -- 172.21.15.94:0/2608416331 shutdown_connections 2024-02-17T12:13:18.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.704+0000 7f363f64e640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f36240a22b0 0x7f36240a4770 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.704+0000 7f363f64e640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f363810dda0 0x7f3638199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.704+0000 7f363f64e640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f3638102220 0x7f363819fd00 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.704+0000 7f363f64e640 1 --2- 172.21.15.94:0/2608416331 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f36380ffb60 0x7f363819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:18.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.704+0000 7f363f64e640 1 -- 172.21.15.94:0/2608416331 >> 172.21.15.94:0/2608416331 conn(0x7f36380765b0 msgr2=0x7f363810f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:18.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.704+0000 7f363f64e640 1 -- 172.21.15.94:0/2608416331 shutdown_connections 2024-02-17T12:13:18.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:18.704+0000 7f363f64e640 1 -- 172.21.15.94:0/2608416331 wait complete. 2024-02-17T12:13:18.720 INFO:tasks.cephfs.filesystem:scrub status for tag:1039f0aa-8164-4e96-97fb-22754200622d - {'path': '/', 'tag': '1039f0aa-8164-4e96-97fb-22754200622d', 'options': 'recursive,force'} 2024-02-17T12:13:18.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:18 smithi125 ceph-mon[36819]: scrub queued for path: / 2024-02-17T12:13:18.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:18 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:13:18.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:18 smithi125 ceph-mon[36819]: scrub summary: active paths [/] 2024-02-17T12:13:18.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:18 smithi094 ceph-mon[32499]: scrub queued for path: / 2024-02-17T12:13:18.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:18 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:13:18.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:18 smithi094 ceph-mon[32499]: scrub summary: active paths [/] 2024-02-17T12:13:19.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:18 smithi189 ceph-mon[36588]: scrub queued for path: / 2024-02-17T12:13:19.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:18 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:13:19.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:18 smithi189 ceph-mon[36588]: scrub summary: active paths [/] 2024-02-17T12:13:19.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:19 smithi125 ceph-mon[36819]: scrub summary: idle+waiting paths [/] 2024-02-17T12:13:19.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:19 smithi125 ceph-mon[36819]: pgmap v477: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 1.22k op/s 2024-02-17T12:13:19.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:19 smithi094 ceph-mon[32499]: scrub summary: idle+waiting paths [/] 2024-02-17T12:13:19.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:19 smithi094 ceph-mon[32499]: pgmap v477: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 1.22k op/s 2024-02-17T12:13:20.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:19 smithi189 ceph-mon[36588]: scrub summary: idle+waiting paths [/] 2024-02-17T12:13:20.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:19 smithi189 ceph-mon[36588]: pgmap v477: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 20 MiB/s wr, 1.22k op/s 2024-02-17T12:13:20.245 INFO:teuthology.orchestra.run.smithi094.stderr:progress: 900.0 s, 1200.2 tps, lat 82.391 ms stddev 129.914 2024-02-17T12:13:20.349 INFO:teuthology.orchestra.run.smithi094.stdout:transaction type: 2024-02-17T12:13:20.349 INFO:teuthology.orchestra.run.smithi094.stdout:scaling factor: 32 2024-02-17T12:13:20.349 INFO:teuthology.orchestra.run.smithi094.stdout:query mode: simple 2024-02-17T12:13:20.349 INFO:teuthology.orchestra.run.smithi094.stdout:number of clients: 100 2024-02-17T12:13:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:number of threads: 4 2024-02-17T12:13:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:duration: 900 s 2024-02-17T12:13:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:number of transactions actually processed: 2068705 2024-02-17T12:13:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:latency average = 43.501 ms 2024-02-17T12:13:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:latency stddev = 70.576 ms 2024-02-17T12:13:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:tps = 2298.295711 (including connections establishing) 2024-02-17T12:13:20.350 INFO:teuthology.orchestra.run.smithi094.stdout:tps = 2298.301084 (excluding connections establishing) 2024-02-17T12:13:20.352 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-02-17T12:13:21.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:21 smithi125 ceph-mon[36819]: pgmap v478: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 20 MiB/s wr, 1.24k op/s 2024-02-17T12:13:21.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:21 smithi094 ceph-mon[32499]: pgmap v478: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 20 MiB/s wr, 1.24k op/s 2024-02-17T12:13:21.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:21 smithi189 ceph-mon[36588]: pgmap v478: 97 pgs: 97 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 20 MiB/s wr, 1.24k op/s 2024-02-17T12:13:22.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:22 smithi189 ceph-mon[36588]: scrub summary: idle 2024-02-17T12:13:22.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:22 smithi125 ceph-mon[36819]: scrub summary: idle 2024-02-17T12:13:22.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:22 smithi094 ceph-mon[32499]: scrub summary: idle 2024-02-17T12:13:23.905 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:23 smithi189 ceph-mon[36588]: pgmap v479: 97 pgs: 97 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 23 MiB/s wr, 1.47k op/s 2024-02-17T12:13:23.937 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:23 smithi125 ceph-mon[36819]: pgmap v479: 97 pgs: 97 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 23 MiB/s wr, 1.47k op/s 2024-02-17T12:13:23.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:23 smithi094 ceph-mon[32499]: pgmap v479: 97 pgs: 97 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 23 MiB/s wr, 1.47k op/s 2024-02-17T12:13:25.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:25 smithi125 ceph-mon[36819]: pgmap v480: 97 pgs: 97 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2024-02-17T12:13:25.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:25 smithi094 ceph-mon[32499]: pgmap v480: 97 pgs: 97 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2024-02-17T12:13:25.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:25 smithi189 ceph-mon[36588]: pgmap v480: 97 pgs: 97 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 26 MiB/s wr, 1.77k op/s 2024-02-17T12:13:27.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.399+0000 7fa9591e2640 1 -- 172.21.15.94:0/1116151698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9541055e0 msgr2=0x7fa954105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:27.401 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.399+0000 7fa9591e2640 1 --2- 172.21.15.94:0/1116151698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9541055e0 0x7fa954105a90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa944009930 tx=0x7fa94402bcd0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.400+0000 7fa9591e2640 1 -- 172.21.15.94:0/1116151698 shutdown_connections 2024-02-17T12:13:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.400+0000 7fa9591e2640 1 --2- 172.21.15.94:0/1116151698 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa9541055e0 0x7fa954105a90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.400+0000 7fa9591e2640 1 --2- 172.21.15.94:0/1116151698 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa954100b00 0x7fa954104fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.400+0000 7fa9591e2640 1 --2- 172.21.15.94:0/1116151698 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9541001e0 0x7fa9541005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.400+0000 7fa9591e2640 1 -- 172.21.15.94:0/1116151698 >> 172.21.15.94:0/1116151698 conn(0x7fa9540771a0 msgr2=0x7fa9540775b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.407+0000 7fa9591e2640 1 -- 172.21.15.94:0/1116151698 shutdown_connections 2024-02-17T12:13:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.410+0000 7fa9591e2640 1 -- 172.21.15.94:0/1116151698 wait complete. 2024-02-17T12:13:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.411+0000 7fa9591e2640 1 Processor -- start 2024-02-17T12:13:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.415+0000 7fa9591e2640 1 -- start start 2024-02-17T12:13:27.417 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.415+0000 7fa9591e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9541001e0 0x7fa954114140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.415+0000 7fa9591e2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa954100b00 0x7fa954114680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.415+0000 7fa9591e2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9541055e0 0x7fa95410d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.415+0000 7fa9591e2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95410d920 con 0x7fa9541001e0 2024-02-17T12:13:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.415+0000 7fa9591e2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95410da90 con 0x7fa954100b00 2024-02-17T12:13:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.415+0000 7fa9591e2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95410dc50 con 0x7fa9541055e0 2024-02-17T12:13:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.416+0000 7fa95259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa954100b00 0x7fa954114680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.416+0000 7fa95259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa954100b00 0x7fa954114680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34558/0 (socket says 172.21.15.94:34558) 2024-02-17T12:13:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.416+0000 7fa95259c640 1 -- 172.21.15.94:0/1878216644 learned_addr learned my addr 172.21.15.94:0/1878216644 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.419+0000 7fa95359e640 1 --2- 172.21.15.94:0/1878216644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9541055e0 0x7fa95410d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.421+0000 7fa95259c640 1 -- 172.21.15.94:0/1878216644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9541055e0 msgr2=0x7fa95410d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.421+0000 7fa95259c640 1 --2- 172.21.15.94:0/1878216644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9541055e0 0x7fa95410d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.421+0000 7fa95259c640 1 -- 172.21.15.94:0/1878216644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9541001e0 msgr2=0x7fa954114140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.421+0000 7fa95259c640 1 --2- 172.21.15.94:0/1878216644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9541001e0 0x7fa954114140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.423 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.421+0000 7fa95259c640 1 -- 172.21.15.94:0/1878216644 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa944009560 con 0x7fa954100b00 2024-02-17T12:13:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.422+0000 7fa95259c640 1 --2- 172.21.15.94:0/1878216644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa954100b00 0x7fa954114680 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00d950 tx=0x7fa93c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:27.428 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.425+0000 7fa94bfff640 1 -- 172.21.15.94:0/1878216644 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c009c30 con 0x7fa954100b00 2024-02-17T12:13:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.425+0000 7fa94bfff640 1 -- 172.21.15.94:0/1878216644 <== mon.2 v2:172.21.15.125:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa93c009d90 con 0x7fa954100b00 2024-02-17T12:13:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.425+0000 7fa94bfff640 1 -- 172.21.15.94:0/1878216644 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00f740 con 0x7fa954100b00 2024-02-17T12:13:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.426+0000 7fa9591e2640 1 -- 172.21.15.94:0/1878216644 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95410df30 con 0x7fa954100b00 2024-02-17T12:13:27.429 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.426+0000 7fa9591e2640 1 -- 172.21.15.94:0/1878216644 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa954111ea0 con 0x7fa954100b00 2024-02-17T12:13:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.432+0000 7fa95bdb9740 1 Processor -- start 2024-02-17T12:13:27.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:27 smithi125 ceph-mon[36819]: pgmap v481: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 23 MiB/s wr, 1.73k op/s 2024-02-17T12:13:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.436+0000 7fa94bfff640 1 -- 172.21.15.94:0/1878216644 <== mon.2 v2:172.21.15.125:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa93c010ad0 con 0x7fa954100b00 2024-02-17T12:13:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.438+0000 7fa95bdb9740 1 -- start start 2024-02-17T12:13:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.438+0000 7fa95bdb9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230ea9a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.438+0000 7fa95bdb9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230ea9ad60 0x56230ea9b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.438+0000 7fa95bdb9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230eaa42e0 0x56230eaa66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.438+0000 7fa95bdb9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230ea099f0 con 0x56230ea9ad60 2024-02-17T12:13:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.438+0000 7fa95bdb9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230ea06f20 con 0x56230eaa42e0 2024-02-17T12:13:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.438+0000 7fa95bdb9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230ea02700 con 0x56230ea9bad0 2024-02-17T12:13:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.439+0000 7fa94bfff640 1 --2- 172.21.15.94:0/1878216644 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa9400a2500 0x7fa9400a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.440+0000 7fa94bfff640 1 -- 172.21.15.94:0/1878216644 <== mon.2 v2:172.21.15.125:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa93c0cdd20 con 0x7fa954100b00 2024-02-17T12:13:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.440+0000 7fa95259c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230eaa42e0 0x56230eaa66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.443+0000 7fa95359e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230ea9a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.443+0000 7fa95359e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230ea9a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56584/0 (socket says 172.21.15.94:56584) 2024-02-17T12:13:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.443+0000 7fa95359e640 1 -- 172.21.15.94:0/1287497075 learned_addr learned my addr 172.21.15.94:0/1287497075 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:27.448 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.444+0000 7fa95359e640 1 --2- 172.21.15.94:0/1878216644 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa9400a2500 0x7fa9400a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.444+0000 7fa95359e640 1 -- 172.21.15.94:0/1287497075 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230eaa42e0 msgr2=0x56230eaa66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.444+0000 7fa95359e640 1 --2- 172.21.15.94:0/1287497075 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230eaa42e0 0x56230eaa66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.444+0000 7fa95359e640 1 -- 172.21.15.94:0/1287497075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230ea9ad60 msgr2=0x56230ea9b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.444+0000 7fa95359e640 1 --2- 172.21.15.94:0/1287497075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230ea9ad60 0x56230ea9b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.444+0000 7fa95359e640 1 -- 172.21.15.94:0/1287497075 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56230e9987b0 con 0x56230ea9bad0 2024-02-17T12:13:27.449 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.446+0000 7fa95359e640 1 --2- 172.21.15.94:0/1287497075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230ea9a820 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa94402c820 tx=0x7fa944004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d75d5f3b08bc2574 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.450+0000 7fa95359e640 1 --2- 172.21.15.94:0/1878216644 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa9400a2500 0x7fa9400a49c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa944039000 tx=0x7fa944048000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.451+0000 7fa9497fa640 1 -- 172.21.15.94:0/1287497075 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944004df0 con 0x56230ea9bad0 2024-02-17T12:13:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.452+0000 7fa9497fa640 1 -- 172.21.15.94:0/1287497075 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa944005430 con 0x56230ea9bad0 2024-02-17T12:13:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.452+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1287497075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 msgr2=0x56230ea9a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.452+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1287497075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230ea9a820 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa94402c820 tx=0x7fa944004410 comp rx=0 tx=0).stop 2024-02-17T12:13:27.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:27 smithi094 ceph-mon[32499]: pgmap v481: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 23 MiB/s wr, 1.73k op/s 2024-02-17T12:13:27.461 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.456+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1287497075 shutdown_connections 2024-02-17T12:13:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.456+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1287497075 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230eaa42e0 0x56230eaa66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.456+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1287497075 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230ea9ad60 0x56230ea9b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.456+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1287497075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230ea9a820 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.456+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1287497075 >> 172.21.15.94:0/1287497075 conn(0x56230ea62550 msgr2=0x56230ea95a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.464+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1287497075 shutdown_connections 2024-02-17T12:13:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.469+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1287497075 wait complete. 2024-02-17T12:13:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.469+0000 7fa95bdb9740 1 -- start start 2024-02-17T12:13:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.469+0000 7fa95bdb9740 1 Processor -- start 2024-02-17T12:13:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.469+0000 7fa95bdb9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230ea9ad60 0x56230eaa1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.469+0000 7fa95bdb9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230eaa17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.469+0000 7fa95bdb9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230eaa42e0 0x56230eaa1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.469+0000 7fa95bdb9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230e9a7bf0 con 0x56230ea9ad60 2024-02-17T12:13:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.469+0000 7fa95bdb9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230e6d6250 con 0x56230eaa42e0 2024-02-17T12:13:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.469+0000 7fa95bdb9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230e988200 con 0x56230ea9bad0 2024-02-17T12:13:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.470+0000 7fa95bdb9740 10 client.0 _ll_register_callbacks cb 0x56230e9f6d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.474+0000 7fa95259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230eaa17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.474+0000 7fa95359e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230eaa42e0 0x56230eaa1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.474+0000 7fa95359e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230eaa42e0 0x56230eaa1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.125:3300/0 says I am v2:172.21.15.94:34574/0 (socket says 172.21.15.94:34574) 2024-02-17T12:13:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.474+0000 7fa95359e640 1 -- 172.21.15.94:0/3536703560 learned_addr learned my addr 172.21.15.94:0/3536703560 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:27.479 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.476+0000 7fa95359e640 1 -- 172.21.15.94:0/3536703560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 msgr2=0x56230eaa17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.476+0000 7fa95359e640 1 --2- 172.21.15.94:0/3536703560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230eaa17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.476+0000 7fa95359e640 1 -- 172.21.15.94:0/3536703560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230ea9ad60 msgr2=0x56230eaa1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.476+0000 7fa95359e640 1 --2- 172.21.15.94:0/3536703560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230ea9ad60 0x56230eaa1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.476+0000 7fa95359e640 1 -- 172.21.15.94:0/3536703560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa954114f10 con 0x56230eaa42e0 2024-02-17T12:13:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.477+0000 7fa95359e640 1 --2- 172.21.15.94:0/3536703560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230eaa42e0 0x56230eaa1d10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa944005d80 tx=0x7fa94402bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eb6df511523b3a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.479+0000 7fa93a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.125:3300/0 2024-02-17T12:13:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.479+0000 7fa93a7fc640 1 -- 172.21.15.94:0/3536703560 <== mon.2 v2:172.21.15.125:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440035f0 con 0x56230eaa42e0 2024-02-17T12:13:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.479+0000 7fa95bdb9740 1 -- 172.21.15.94:0/3536703560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56230e702600 con 0x56230eaa42e0 2024-02-17T12:13:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.479+0000 7fa93a7fc640 1 -- 172.21.15.94:0/3536703560 <== mon.2 v2:172.21.15.125:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa944004df0 con 0x56230eaa42e0 2024-02-17T12:13:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.485+0000 7fa93a7fc640 1 -- 172.21.15.94:0/3536703560 <== mon.2 v2:172.21.15.125:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa944004fb0 con 0x56230eaa42e0 2024-02-17T12:13:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.497+0000 7fa95bdb9740 10 client.28943 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.497+0000 7fa95bdb9740 1 -- 172.21.15.94:0/3536703560 --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56230ea00d80 con 0x56230eaa42e0 2024-02-17T12:13:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.501+0000 7fa93a7fc640 1 -- 172.21.15.94:0/3536703560 <== mon.2 v2:172.21.15.125:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56230e702600 con 0x56230eaa42e0 2024-02-17T12:13:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.502+0000 7fa93a7fc640 1 -- 172.21.15.94:0/3536703560 <== mon.2 v2:172.21.15.125:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa944034b50 con 0x56230eaa42e0 2024-02-17T12:13:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.502+0000 7fa95bdb9740 10 client.28943 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.502+0000 7fa95bdb9740 10 client.28943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.502+0000 7fa95bdb9740 20 client.28943 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.502+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/3536703560 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56230ead64d0 0x56230ead88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.502+0000 7fa95bdb9740 4 client.28943 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:13:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.502+0000 7fa95bdb9740 1 -- 172.21.15.94:0/3536703560 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56230e7531b0 con 0x56230ead64d0 2024-02-17T12:13:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.506+0000 7fa952d9d640 1 --2- 172.21.15.94:0/3536703560 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56230ead64d0 0x56230ead88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.514+0000 7fa952d9d640 1 --2- 172.21.15.94:0/3536703560 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56230ead64d0 0x56230ead88c0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a1dc34da78c6f86 server_cookie=58ef9f9853f08d68 in_seq=0 out_seq=0 2024-02-17T12:13:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.515+0000 7fa93a7fc640 10 client.28943 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.518+0000 7fa93a7fc640 1 -- 172.21.15.94:0/3536703560 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56230e7531b0 con 0x56230ead64d0 2024-02-17T12:13:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.518+0000 7fa93a7fc640 10 client.28943 handle_command_reply: tid=0 2024-02-17T12:13:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.518+0000 7fa95bdb9740 1 client.28943 shutdown 2024-02-17T12:13:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.528+0000 7fa95bdb9740 1 -- 172.21.15.94:0/3536703560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230eaa42e0 msgr2=0x56230eaa1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.528+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/3536703560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230eaa42e0 0x56230eaa1d10 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa944005d80 tx=0x7fa94402bcd0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.533+0000 7fa95bdb9740 1 -- 172.21.15.94:0/3536703560 shutdown_connections 2024-02-17T12:13:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.533+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/3536703560 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56230ead64d0 0x56230ead88c0 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.533+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/3536703560 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230eaa42e0 0x56230eaa1d10 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.533+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/3536703560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230eaa17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.533+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/3536703560 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230ea9ad60 0x56230eaa1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.533+0000 7fa95bdb9740 1 -- 172.21.15.94:0/3536703560 >> 172.21.15.94:0/3536703560 conn(0x56230ea62550 msgr2=0x56230ea977a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.533+0000 7fa93a7fc640 0 client.28943 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.537+0000 7fa95bdb9740 1 -- 172.21.15.94:0/3536703560 shutdown_connections 2024-02-17T12:13:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.540+0000 7fa95bdb9740 1 -- 172.21.15.94:0/3536703560 wait complete. 2024-02-17T12:13:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.544+0000 7fa95bdb9740 20 client.28943 trim_cache size 0 max 16384 2024-02-17T12:13:27.579 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.577+0000 7fa95bdb9740 1 Processor -- start 2024-02-17T12:13:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.579+0000 7fa95bdb9740 1 -- start start 2024-02-17T12:13:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.580+0000 7fa95bdb9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230ea9ad60 0x56230eaa39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.580+0000 7fa95bdb9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230eaa17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.580+0000 7fa95bdb9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230eaa42e0 0x56230eaa1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.580+0000 7fa95bdb9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230ea96850 con 0x56230eaa42e0 2024-02-17T12:13:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.580+0000 7fa95bdb9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230ea95310 con 0x56230ea9ad60 2024-02-17T12:13:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.580+0000 7fa95259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230eaa17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.580+0000 7fa95259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230eaa17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56604/0 (socket says 172.21.15.94:56604) 2024-02-17T12:13:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.580+0000 7fa95259c640 1 -- 172.21.15.94:0/1348708213 learned_addr learned my addr 172.21.15.94:0/1348708213 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:27.583 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.580+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1348708213 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230eaa2260 con 0x56230ea9bad0 2024-02-17T12:13:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.583+0000 7fa952d9d640 1 --2- 172.21.15.94:0/1348708213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230ea9ad60 0x56230eaa39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.583+0000 7fa95359e640 1 --2- 172.21.15.94:0/1348708213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230eaa42e0 0x56230eaa1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.585+0000 7fa95259c640 1 -- 172.21.15.94:0/1348708213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230ea9ad60 msgr2=0x56230eaa39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.585+0000 7fa95259c640 1 --2- 172.21.15.94:0/1348708213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230ea9ad60 0x56230eaa39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.585+0000 7fa95259c640 1 -- 172.21.15.94:0/1348708213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230eaa42e0 msgr2=0x56230eaa1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.585+0000 7fa95259c640 1 --2- 172.21.15.94:0/1348708213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230eaa42e0 0x56230eaa1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.585+0000 7fa95259c640 1 -- 172.21.15.94:0/1348708213 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56230e99d7b0 con 0x56230ea9bad0 2024-02-17T12:13:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.585+0000 7fa952d9d640 1 --2- 172.21.15.94:0/1348708213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230ea9ad60 0x56230eaa39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:27.592 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.588+0000 7fa95259c640 1 --2- 172.21.15.94:0/1348708213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230eaa17e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009610 tx=0x7fa93c099fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9b9f6e7b91c90b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.590+0000 7fa948ff9640 1 -- 172.21.15.94:0/1348708213 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c014020 con 0x56230ea9bad0 2024-02-17T12:13:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.590+0000 7fa948ff9640 1 -- 172.21.15.94:0/1348708213 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa93c09e7c0 con 0x56230ea9bad0 2024-02-17T12:13:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.590+0000 7fa948ff9640 1 -- 172.21.15.94:0/1348708213 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0ba0b0 con 0x56230ea9bad0 2024-02-17T12:13:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.593+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1348708213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 msgr2=0x56230eaa17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.593+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1348708213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230eaa17e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009610 tx=0x7fa93c099fd0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.594+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1348708213 shutdown_connections 2024-02-17T12:13:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.594+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1348708213 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230eaa42e0 0x56230eaa1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.594+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1348708213 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230ea9bad0 0x56230eaa17e0 secure :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009610 tx=0x7fa93c099fd0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.594+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1348708213 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230ea9ad60 0x56230eaa39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.594+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1348708213 >> 172.21.15.94:0/1348708213 conn(0x56230ea62550 msgr2=0x56230ea9c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.600+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1348708213 shutdown_connections 2024-02-17T12:13:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.601+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1348708213 wait complete. 2024-02-17T12:13:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.601+0000 7fa95bdb9740 1 -- start start 2024-02-17T12:13:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.601+0000 7fa95bdb9740 1 Processor -- start 2024-02-17T12:13:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.602+0000 7fa95bdb9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230ea9ad60 0x56230eaab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.602+0000 7fa95bdb9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230eaa42e0 0x56230eaa9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.602+0000 7fa95bdb9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230eaa9a80 0x56230eaa9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.602+0000 7fa95bdb9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230eaab850 con 0x56230eaa42e0 2024-02-17T12:13:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.602+0000 7fa95bdb9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230eaab9c0 con 0x56230ea9ad60 2024-02-17T12:13:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.602+0000 7fa95bdb9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x56230eb2c9c0 con 0x56230eaa9a80 2024-02-17T12:13:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.602+0000 7fa95bdb9740 10 client.0 _ll_register_callbacks cb 0x7fa944009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.602+0000 7fa95359e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230eaa9a80 0x56230eaa9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.603+0000 7fa95359e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230eaa9a80 0x56230eaa9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.94:56606/0 (socket says 172.21.15.94:56606) 2024-02-17T12:13:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.603+0000 7fa95359e640 1 -- 172.21.15.94:0/1067025944 learned_addr learned my addr 172.21.15.94:0/1067025944 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.607+0000 7fa952d9d640 1 --2- 172.21.15.94:0/1067025944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230ea9ad60 0x56230eaab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.607+0000 7fa95259c640 1 --2- 172.21.15.94:0/1067025944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230eaa42e0 0x56230eaa9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.608+0000 7fa95359e640 1 -- 172.21.15.94:0/1067025944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230ea9ad60 msgr2=0x56230eaab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.608+0000 7fa95359e640 1 --2- 172.21.15.94:0/1067025944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230ea9ad60 0x56230eaab1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.608+0000 7fa95359e640 1 -- 172.21.15.94:0/1067025944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230eaa42e0 msgr2=0x56230eaa9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:27.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.608+0000 7fa95359e640 1 --2- 172.21.15.94:0/1067025944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230eaa42e0 0x56230eaa9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:27.611 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.608+0000 7fa95359e640 1 -- 172.21.15.94:0/1067025944 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c009a00 con 0x56230eaa9a80 2024-02-17T12:13:27.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.610+0000 7fa95359e640 1 --2- 172.21.15.94:0/1067025944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230eaa9a80 0x56230eaa9f00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa944005d80 tx=0x7fa9440058f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b75bea33096c9503 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:27.613 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.610+0000 7fa93a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.189:3300/0 2024-02-17T12:13:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.614+0000 7fa93a7fc640 1 -- 172.21.15.94:0/1067025944 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944004210 con 0x56230eaa9a80 2024-02-17T12:13:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.614+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1067025944 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa944005430 con 0x56230eaa9a80 2024-02-17T12:13:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.614+0000 7fa93a7fc640 1 -- 172.21.15.94:0/1067025944 <== mon.1 v2:172.21.15.189:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa944000e40 con 0x56230eaa9a80 2024-02-17T12:13:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.617+0000 7fa93a7fc640 1 -- 172.21.15.94:0/1067025944 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa944004df0 con 0x56230eaa9a80 2024-02-17T12:13:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.618+0000 7fa95bdb9740 10 client.28912 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.618+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1067025944 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fa954111ea0 con 0x56230eaa9a80 2024-02-17T12:13:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.623+0000 7fa93a7fc640 1 -- 172.21.15.94:0/1067025944 <== mon.1 v2:172.21.15.189:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa944005430 con 0x56230eaa9a80 2024-02-17T12:13:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.623+0000 7fa93a7fc640 1 -- 172.21.15.94:0/1067025944 <== mon.1 v2:172.21.15.189:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa944034440 con 0x56230eaa9a80 2024-02-17T12:13:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.625+0000 7fa95bdb9740 10 client.28912 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.625+0000 7fa95bdb9740 10 client.28912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.625+0000 7fa95bdb9740 20 client.28912 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.625+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1067025944 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56230eb2dc70 0x56230eb30060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.625+0000 7fa95bdb9740 4 client.28912 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-02-17T12:13:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.625+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1067025944 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56230e73f550 con 0x56230eb2dc70 2024-02-17T12:13:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.627+0000 7fa952d9d640 1 --2- 172.21.15.94:0/1067025944 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56230eb2dc70 0x56230eb30060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.638+0000 7fa952d9d640 1 --2- 172.21.15.94:0/1067025944 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56230eb2dc70 0x56230eb30060 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e875055f93972caf server_cookie=acfeee5475a8e091 in_seq=0 out_seq=0 2024-02-17T12:13:27.640 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:27.639+0000 7fa93a7fc640 10 client.28912 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:27.653 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:27 smithi189 ceph-mon[36588]: pgmap v481: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 23 MiB/s wr, 1.73k op/s 2024-02-17T12:13:29.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:29 smithi125 ceph-mon[36819]: pgmap v482: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 26 MiB/s wr, 2.09k op/s 2024-02-17T12:13:29.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:29 smithi094 ceph-mon[32499]: pgmap v482: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 26 MiB/s wr, 2.09k op/s 2024-02-17T12:13:29.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:29 smithi189 ceph-mon[36588]: pgmap v482: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 26 MiB/s wr, 2.09k op/s 2024-02-17T12:13:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.393+0000 7fa93a7fc640 1 -- 172.21.15.94:0/1067025944 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56230e73f550 con 0x56230eb2dc70 2024-02-17T12:13:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.393+0000 7fa93a7fc640 10 client.28912 handle_command_reply: tid=0 2024-02-17T12:13:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.393+0000 7fa95bdb9740 1 client.28912 shutdown 2024-02-17T12:13:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.393+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1067025944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230eaa9a80 msgr2=0x56230eaa9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.393+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1067025944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230eaa9a80 0x56230eaa9f00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa944005d80 tx=0x7fa9440058f0 comp rx=0 tx=0).stop 2024-02-17T12:13:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.393+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1067025944 shutdown_connections 2024-02-17T12:13:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.393+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1067025944 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x56230eb2dc70 0x56230eb30060 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.393+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1067025944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56230eaa9a80 0x56230eaa9f00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.393+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1067025944 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x56230eaa42e0 0x56230eaa9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.393+0000 7fa95bdb9740 1 --2- 172.21.15.94:0/1067025944 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x56230ea9ad60 0x56230eaab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.393+0000 7fa93a7fc640 0 client.28912 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.393+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1067025944 >> 172.21.15.94:0/1067025944 conn(0x56230ea62550 msgr2=0x56230ea9fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1067025944 shutdown_connections 2024-02-17T12:13:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa95bdb9740 1 -- 172.21.15.94:0/1067025944 wait complete. 2024-02-17T12:13:30.397 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa95bdb9740 20 client.28912 trim_cache size 0 max 16384 2024-02-17T12:13:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:{ 2024-02-17T12:13:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "message": "", 2024-02-17T12:13:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout: "return_code": 0 2024-02-17T12:13:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stdout:} 2024-02-17T12:13:30.398 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa9591e2640 1 -- 172.21.15.94:0/1878216644 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa9400a2500 msgr2=0x7fa9400a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa9591e2640 1 --2- 172.21.15.94:0/1878216644 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa9400a2500 0x7fa9400a49c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa944039000 tx=0x7fa944048000 comp rx=0 tx=0).stop 2024-02-17T12:13:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa9591e2640 1 -- 172.21.15.94:0/1878216644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa954100b00 msgr2=0x7fa954114680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa9591e2640 1 --2- 172.21.15.94:0/1878216644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa954100b00 0x7fa954114680 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00d950 tx=0x7fa93c00de10 comp rx=0 tx=0).stop 2024-02-17T12:13:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa9591e2640 1 -- 172.21.15.94:0/1878216644 shutdown_connections 2024-02-17T12:13:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa9591e2640 1 --2- 172.21.15.94:0/1878216644 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa9400a2500 0x7fa9400a49c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa9591e2640 1 --2- 172.21.15.94:0/1878216644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa9541055e0 0x7fa95410d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa9591e2640 1 --2- 172.21.15.94:0/1878216644 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa954100b00 0x7fa954114680 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa9591e2640 1 --2- 172.21.15.94:0/1878216644 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa9541001e0 0x7fa954114140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa9591e2640 1 -- 172.21.15.94:0/1878216644 >> 172.21.15.94:0/1878216644 conn(0x7fa9540771a0 msgr2=0x7fa954102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.394+0000 7fa9591e2640 1 -- 172.21.15.94:0/1878216644 shutdown_connections 2024-02-17T12:13:30.399 INFO:teuthology.task.background_exec.ubuntu@smithi094.front.sepia.ceph.com.smithi094.stderr:2024-02-17T12:13:30.395+0000 7fa9591e2640 1 -- 172.21.15.94:0/1878216644 wait complete. 2024-02-17T12:13:30.846 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-02-17T12:13:30.936 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/: 2024-02-17T12:13:30.936 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:30.936 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 . 2024-02-17T12:13:30.936 INFO:teuthology.orchestra.run.smithi094.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 .. 2024-02-17T12:13:30.936 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 data 2024-02-17T12:13:30.936 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:30.936 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data: 2024-02-17T12:13:30.936 INFO:teuthology.orchestra.run.smithi094.stdout:total 36 2024-02-17T12:13:30.936 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 . 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 .. 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Feb 17 11:58 base 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Feb 17 11:58 current_logfiles 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Feb 17 11:58 global 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 log 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_commit_ts 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_dynshmem 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Feb 17 11:58 pg_hba.conf 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Feb 17 11:58 pg_ident.conf 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 12:13 pg_logical 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 17 11:58 pg_multixact 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_notify 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_replslot 2024-02-17T12:13:30.937 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_serial 2024-02-17T12:13:30.938 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_snapshots 2024-02-17T12:13:30.938 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 12:13 pg_stat 2024-02-17T12:13:30.938 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 12:13 pg_stat_tmp 2024-02-17T12:13:30.938 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 12:13 pg_subtrans 2024-02-17T12:13:30.938 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_tblspc 2024-02-17T12:13:30.938 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pg_twophase 2024-02-17T12:13:30.938 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 PG_VERSION 2024-02-17T12:13:30.938 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Feb 17 12:13 pg_wal 2024-02-17T12:13:30.938 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 17 12:03 pg_xact 2024-02-17T12:13:30.938 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Feb 17 11:58 postgresql.auto.conf 2024-02-17T12:13:30.938 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28082 Feb 17 11:58 postgresql.conf 2024-02-17T12:13:30.938 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Feb 17 11:58 postmaster.opts 2024-02-17T12:13:30.938 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base: 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Feb 17 11:58 . 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 17 11:58 1 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 17 11:58 13493 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Feb 17 12:01 13494 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 pgsql_tmp 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/1: 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:total 7777 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 17 11:58 . 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Feb 17 11:58 .. 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 112 2024-02-17T12:13:30.939 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 113 2024-02-17T12:13:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 17 11:58 1247 2024-02-17T12:13:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1247_fsm 2024-02-17T12:13:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1247_vm 2024-02-17T12:13:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Feb 17 11:58 1249 2024-02-17T12:13:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1249_fsm 2024-02-17T12:13:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1249_vm 2024-02-17T12:13:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Feb 17 11:58 1255 2024-02-17T12:13:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1255_fsm 2024-02-17T12:13:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1255_vm 2024-02-17T12:13:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Feb 17 11:58 1259 2024-02-17T12:13:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1259_fsm 2024-02-17T12:13:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1259_vm 2024-02-17T12:13:30.940 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Feb 17 11:58 13344 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13344_fsm 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13344_vm 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13346 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13348 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13349 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13349_fsm 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13349_vm 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13351 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13353 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13354 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13354_fsm 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13354_vm 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13356 2024-02-17T12:13:30.941 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13358 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13359 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13359_fsm 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13359_vm 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13361 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13363 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 1417 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 1418 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 174 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 175 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2187 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2224 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2328 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2336 2024-02-17T12:13:30.942 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2337 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2579 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2600 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2600_fsm 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2600_vm 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2601 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2601_fsm 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2601_vm 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2602 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2602_fsm 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2602_vm 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2603 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2603_fsm 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2603_vm 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2604 2024-02-17T12:13:30.943 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2605 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2605_fsm 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2605_vm 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2606 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2606_fsm 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2606_vm 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2607 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2607_fsm 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2607_vm 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Feb 17 11:58 2608 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2608_fsm 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2608_vm 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Feb 17 11:58 2609 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2609_fsm 2024-02-17T12:13:30.944 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2609_vm 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2610 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2610_fsm 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2610_vm 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2611 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2612 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2612_fsm 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2612_vm 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2613 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2615 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2615_fsm 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2615_vm 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2616 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2616_fsm 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2616_vm 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Feb 17 11:58 2617 2024-02-17T12:13:30.945 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2617_fsm 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2617_vm 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Feb 17 11:58 2618 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2618_fsm 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2618_vm 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Feb 17 11:58 2619 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2619_fsm 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2619_vm 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2620 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2650 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2651 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2652 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2653 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2654 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2655 2024-02-17T12:13:30.946 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2656 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2657 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Feb 17 11:58 2658 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 17 11:58 2659 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2660 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2661 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2662 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2663 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2664 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2665 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2666 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2667 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2668 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2669 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2670 2024-02-17T12:13:30.947 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Feb 17 11:58 2673 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Feb 17 11:58 2674 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Feb 17 11:58 2675 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2678 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2679 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2680 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2681 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2682 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2683 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2684 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2685 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2686 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2687 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2688 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2689 2024-02-17T12:13:30.948 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Feb 17 11:58 2690 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Feb 17 11:58 2691 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2692 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2693 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2696 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2699 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2701 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2702 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2703 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2704 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2753 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2753_fsm 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2753_vm 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2754 2024-02-17T12:13:30.949 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2755 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2756 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2757 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2830 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2831 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2832 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2833 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2834 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2835 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2836 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2837 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Feb 17 11:58 2838 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2838_fsm 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2838_vm 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2839 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2840 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2840_fsm 2024-02-17T12:13:30.950 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2840_vm 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2841 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2995 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2996 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3079 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3079_fsm 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3079_vm 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3080 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3081 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 3085 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3118 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3119 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 17 11:58 3164 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3256 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3257 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3258 2024-02-17T12:13:30.951 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3350 2024-02-17T12:13:30.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3351 2024-02-17T12:13:30.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3379 2024-02-17T12:13:30.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3380 2024-02-17T12:13:30.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3381 2024-02-17T12:13:30.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3394 2024-02-17T12:13:30.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3394_fsm 2024-02-17T12:13:30.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3394_vm 2024-02-17T12:13:30.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3395 2024-02-17T12:13:30.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3429 2024-02-17T12:13:30.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3430 2024-02-17T12:13:30.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3431 2024-02-17T12:13:30.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3433 2024-02-17T12:13:30.952 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3439 2024-02-17T12:13:30.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3440 2024-02-17T12:13:30.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 3455 2024-02-17T12:13:30.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Feb 17 11:58 3456 2024-02-17T12:13:30.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3456_fsm 2024-02-17T12:13:30.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3456_vm 2024-02-17T12:13:30.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3466 2024-02-17T12:13:30.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3467 2024-02-17T12:13:30.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3468 2024-02-17T12:13:30.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3501 2024-02-17T12:13:30.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3502 2024-02-17T12:13:30.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3503 2024-02-17T12:13:30.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3534 2024-02-17T12:13:30.953 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3541 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3541_fsm 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3541_vm 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3542 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3574 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3575 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3576 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3596 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3597 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3598 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3599 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3600 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3600_fsm 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3600_vm 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3601 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3601_fsm 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3601_vm 2024-02-17T12:13:30.954 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3602 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3602_fsm 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3602_vm 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3603 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3603_fsm 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3603_vm 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3604 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3605 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3606 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3607 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3608 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 3609 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3712 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3764 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3764_fsm 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3764_vm 2024-02-17T12:13:30.955 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3766 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3767 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3997 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4143 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4144 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4145 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4146 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4147 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4148 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4149 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4150 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4151 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4152 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4153 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4154 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4155 2024-02-17T12:13:30.956 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4156 2024-02-17T12:13:30.957 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4157 2024-02-17T12:13:30.957 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4158 2024-02-17T12:13:30.957 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4159 2024-02-17T12:13:30.957 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4160 2024-02-17T12:13:30.957 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4161 2024-02-17T12:13:30.957 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4162 2024-02-17T12:13:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4163 2024-02-17T12:13:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4164 2024-02-17T12:13:30.959 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4165 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4166 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4167 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4168 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4169 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4170 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4171 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4172 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4173 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4174 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 5002 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 548 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 549 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6102 2024-02-17T12:13:30.960 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6104 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6106 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6110 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6111 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6112 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6113 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6117 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 826 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 827 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 828 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 17 11:58 pg_filenode.map 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 PG_VERSION 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:total 7777 2024-02-17T12:13:30.961 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Feb 17 11:58 . 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Feb 17 11:58 .. 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 112 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 113 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 17 11:58 1247 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1247_fsm 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1247_vm 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Feb 17 11:58 1249 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1249_fsm 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1249_vm 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Feb 17 11:58 1255 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1255_fsm 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1255_vm 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Feb 17 11:58 1259 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1259_fsm 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1259_vm 2024-02-17T12:13:30.962 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Feb 17 11:58 13344 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13344_fsm 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13344_vm 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13346 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13348 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13349 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13349_fsm 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13349_vm 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13351 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13353 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13354 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13354_fsm 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13354_vm 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13356 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13358 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13359 2024-02-17T12:13:30.963 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13359_fsm 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13359_vm 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13361 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13363 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 1417 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 1418 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 174 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 175 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2187 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2224 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2328 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2336 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2337 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2579 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2600 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2600_fsm 2024-02-17T12:13:30.964 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2600_vm 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2601 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2601_fsm 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2601_vm 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2602 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2602_fsm 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2602_vm 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2603 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2603_fsm 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2603_vm 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2604 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2605 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2605_fsm 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2605_vm 2024-02-17T12:13:30.965 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2606 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2606_fsm 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2606_vm 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2607 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2607_fsm 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2607_vm 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Feb 17 11:58 2608 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2608_fsm 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2608_vm 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Feb 17 11:58 2609 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2609_fsm 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2609_vm 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2610 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2610_fsm 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2610_vm 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2611 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2612 2024-02-17T12:13:30.966 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2612_fsm 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2612_vm 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2613 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2615 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2615_fsm 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2615_vm 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2616 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2616_fsm 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2616_vm 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Feb 17 11:58 2617 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2617_fsm 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2617_vm 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Feb 17 11:58 2618 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2618_fsm 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2618_vm 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Feb 17 11:58 2619 2024-02-17T12:13:30.967 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2619_fsm 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2619_vm 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2620 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2650 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2651 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2652 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2653 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2654 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2655 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2656 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2657 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Feb 17 11:58 2658 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 17 11:58 2659 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2660 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2661 2024-02-17T12:13:30.968 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2662 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2663 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2664 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2665 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2666 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2667 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2668 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2669 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2670 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Feb 17 11:58 2673 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Feb 17 11:58 2674 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Feb 17 11:58 2675 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2678 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2679 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2680 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2681 2024-02-17T12:13:30.969 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2682 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2683 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2684 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2685 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2686 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2687 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2688 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2689 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Feb 17 11:58 2690 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Feb 17 11:58 2691 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2692 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2693 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2696 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2699 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2701 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2702 2024-02-17T12:13:30.970 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2703 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2704 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2753 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2753_fsm 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2753_vm 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2754 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2755 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2756 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2757 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2830 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2831 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2832 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2833 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2834 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2835 2024-02-17T12:13:30.971 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2836 2024-02-17T12:13:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2837 2024-02-17T12:13:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Feb 17 11:58 2838 2024-02-17T12:13:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2838_fsm 2024-02-17T12:13:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2838_vm 2024-02-17T12:13:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2839 2024-02-17T12:13:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2840 2024-02-17T12:13:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2840_fsm 2024-02-17T12:13:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2840_vm 2024-02-17T12:13:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2841 2024-02-17T12:13:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2995 2024-02-17T12:13:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2996 2024-02-17T12:13:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3079 2024-02-17T12:13:30.972 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3079_fsm 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3079_vm 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3080 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3081 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 3085 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3118 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3119 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 17 11:58 3164 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3256 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3257 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3258 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3350 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3351 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3379 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3380 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3381 2024-02-17T12:13:30.973 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3394 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3394_fsm 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3394_vm 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3395 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3429 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3430 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3431 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3433 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3439 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3440 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 3455 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Feb 17 11:58 3456 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3456_fsm 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3456_vm 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3466 2024-02-17T12:13:30.974 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3467 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3468 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3501 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3502 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3503 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3534 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3541 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3541_fsm 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3541_vm 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3542 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3574 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3575 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3576 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3596 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3597 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3598 2024-02-17T12:13:30.975 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3599 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3600 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3600_fsm 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3600_vm 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3601 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3601_fsm 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3601_vm 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3602 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3602_fsm 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3602_vm 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3603 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3603_fsm 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3603_vm 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3604 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3605 2024-02-17T12:13:30.976 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3606 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3607 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3608 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 3609 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3712 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3764 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3764_fsm 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3764_vm 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3766 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3767 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3997 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4143 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4144 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4145 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4146 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4147 2024-02-17T12:13:30.977 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4148 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4149 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4150 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4151 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4152 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4153 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4154 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4155 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4156 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4157 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4158 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4159 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4160 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4161 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4162 2024-02-17T12:13:30.978 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4163 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4164 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4165 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4166 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4167 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4168 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4169 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4170 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4171 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4172 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4173 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4174 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 5002 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 548 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 549 2024-02-17T12:13:30.979 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6102 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6104 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6106 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6110 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6111 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6112 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6113 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6117 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 826 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 827 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 828 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 17 11:58 pg_filenode.map 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 PG_VERSION 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:30.980 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:total 626286 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Feb 17 12:01 . 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Feb 17 11:58 .. 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 112 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 113 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 17 11:58 1247 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1247_fsm 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1247_vm 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Feb 17 11:58 1249 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1249_fsm 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1249_vm 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Feb 17 11:58 1255 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1255_fsm 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1255_vm 2024-02-17T12:13:30.981 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Feb 17 12:12 1259 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1259_fsm 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1259_vm 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Feb 17 11:58 13344 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13344_fsm 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13344_vm 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13346 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13348 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13349 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13349_fsm 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13349_vm 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13351 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13353 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13354 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13354_fsm 2024-02-17T12:13:30.982 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13354_vm 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13356 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13358 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13359 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 13359_fsm 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13359_vm 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 13361 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 13363 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 1417 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 1418 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 445497344 Feb 17 12:13 16396 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Feb 17 12:13 16396_fsm 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 12:01 16396_vm 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1499136 Feb 17 12:13 16397 2024-02-17T12:13:30.983 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 12:13 16397_fsm 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 12:13 16397_vm 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2031616 Feb 17 12:13 16399 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 12:12 16399_fsm 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 12:12 16399_vm 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 12:13 16400 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 17 12:13 16402 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Feb 17 12:13 16404 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 111894528 Feb 17 12:13 16406 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 12:13 16406_fsm 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 12:11 16406_vm 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 174 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 175 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2187 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2224 2024-02-17T12:13:30.984 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2328 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2336 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2337 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2579 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2600 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2600_fsm 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2600_vm 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2601 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2601_fsm 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2601_vm 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2602 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2602_fsm 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2602_vm 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2603 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2603_fsm 2024-02-17T12:13:30.985 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2603_vm 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2604 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2605 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2605_fsm 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2605_vm 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2606 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2606_fsm 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2606_vm 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2607 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2607_fsm 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2607_vm 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Feb 17 11:58 2608 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2608_fsm 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2608_vm 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Feb 17 11:58 2609 2024-02-17T12:13:30.986 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2609_fsm 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2609_vm 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2610 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2610_fsm 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2610_vm 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2611 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2612 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2612_fsm 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2612_vm 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2613 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2615 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2615_fsm 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2615_vm 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2616 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2616_fsm 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2616_vm 2024-02-17T12:13:30.987 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Feb 17 11:58 2617 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2617_fsm 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2617_vm 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Feb 17 11:58 2618 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2618_fsm 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2618_vm 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Feb 17 12:12 2619 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 12:00 2619_fsm 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2619_vm 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2620 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2650 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2651 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2652 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2653 2024-02-17T12:13:30.988 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2654 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2655 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2656 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2657 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Feb 17 11:58 2658 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Feb 17 11:58 2659 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2660 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2661 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2662 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2663 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2664 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2665 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2666 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2667 2024-02-17T12:13:30.989 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2668 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2669 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2670 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Feb 17 11:58 2673 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Feb 17 11:58 2674 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Feb 17 11:58 2675 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2678 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2679 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2680 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2681 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2682 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2683 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2684 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2685 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2686 2024-02-17T12:13:30.990 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2687 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2688 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Feb 17 11:58 2689 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Feb 17 11:58 2690 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Feb 17 11:58 2691 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2692 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2693 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 12:12 2696 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2699 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2701 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2702 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2703 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 2704 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2753 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2753_fsm 2024-02-17T12:13:30.991 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2753_vm 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2754 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2755 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2756 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 2757 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2830 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2831 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2832 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2833 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2834 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2835 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2836 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2837 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Feb 17 11:58 2838 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2838_fsm 2024-02-17T12:13:30.992 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2838_vm 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2839 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2840 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2840_fsm 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2840_vm 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2841 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2995 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2996 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3079 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3079_fsm 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3079_vm 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3080 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3081 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 3085 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3118 2024-02-17T12:13:30.993 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3119 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Feb 17 11:58 3164 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3256 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3257 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3258 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3350 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3351 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3379 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3380 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3381 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3394 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3394_fsm 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3394_vm 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3395 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3429 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3430 2024-02-17T12:13:30.994 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3431 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3433 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3439 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3440 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Feb 17 11:58 3455 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Feb 17 11:58 3456 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3456_fsm 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3456_vm 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3466 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3467 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3468 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3501 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3502 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3503 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3534 2024-02-17T12:13:30.995 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3541 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3541_fsm 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3541_vm 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3542 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3574 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3575 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3576 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3596 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3597 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3598 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3599 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3600 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3600_fsm 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3600_vm 2024-02-17T12:13:30.996 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3601 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3601_fsm 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3601_vm 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3602 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3602_fsm 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3602_vm 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3603 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3603_fsm 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3603_vm 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3604 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3605 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3606 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3607 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3608 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Feb 17 11:58 3609 2024-02-17T12:13:30.997 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3712 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3764 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 3764_fsm 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3764_vm 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3766 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 3767 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3997 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4143 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4144 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4145 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4146 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4147 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4148 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4149 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4150 2024-02-17T12:13:30.998 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4151 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4152 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4153 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4154 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4155 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4156 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4157 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4158 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4159 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4160 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4161 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4162 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4163 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4164 2024-02-17T12:13:30.999 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4165 2024-02-17T12:13:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4166 2024-02-17T12:13:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4167 2024-02-17T12:13:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4168 2024-02-17T12:13:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4169 2024-02-17T12:13:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4170 2024-02-17T12:13:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4171 2024-02-17T12:13:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4172 2024-02-17T12:13:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4173 2024-02-17T12:13:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4174 2024-02-17T12:13:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 5002 2024-02-17T12:13:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 548 2024-02-17T12:13:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 549 2024-02-17T12:13:31.000 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6102 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6104 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6106 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6110 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6111 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6112 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6113 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6117 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 826 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 827 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 828 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 17 11:58 pg_filenode.map 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Feb 17 11:58 pg_internal.init 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 11:58 PG_VERSION 2024-02-17T12:13:31.001 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Feb 17 11:58 .. 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/global: 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout:total 560 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Feb 17 11:58 . 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1213 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1213_fsm 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1213_vm 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1214 2024-02-17T12:13:31.002 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1214_fsm 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1214_vm 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 1232 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 1233 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1260 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1260_fsm 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1260_vm 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1261 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1261_fsm 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1261_vm 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1262 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 1262_fsm 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 1262_vm 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2396 2024-02-17T12:13:31.003 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Feb 17 11:58 2396_fsm 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2396_vm 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2397 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2671 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2672 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2676 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2677 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2694 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2695 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2697 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Feb 17 11:58 2698 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2846 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2847 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2964 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2965 2024-02-17T12:13:31.004 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 2966 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 2967 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 3592 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 3593 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4060 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4061 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4175 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4176 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4177 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4178 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4181 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4182 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4183 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4184 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 4185 2024-02-17T12:13:31.005 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 4186 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6000 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6001 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6002 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 6100 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6114 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 6115 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 12:13 pg_control 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Feb 17 11:58 pg_filenode.map 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Feb 17 11:58 pg_internal.init 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/log: 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:total 2 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 . 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.006 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1223 Feb 17 12:13 postgresql-Sat.log 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout:total 1 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 12:13 . 2024-02-17T12:13:31.007 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 mappings 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Feb 17 12:13 replorigin_checkpoint 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 snapshots 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 12:13 .. 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 12:13 .. 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.008 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 17 11:58 . 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 members 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 offsets 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout:total 8 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 . 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 17 11:58 .. 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 11:58 0000 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-02-17T12:13:31.009 INFO:teuthology.orchestra.run.smithi094.stdout:total 8 2024-02-17T12:13:31.010 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 11:58 . 2024-02-17T12:13:31.010 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 17 11:58 .. 2024-02-17T12:13:31.010 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Feb 17 12:01 0000 2024-02-17T12:13:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.010 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-02-17T12:13:31.010 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.010 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T12:13:31.010 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.010 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.010 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-02-17T12:13:31.010 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.010 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T12:13:31.011 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.012 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-02-17T12:13:31.012 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.012 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T12:13:31.012 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.012 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-02-17T12:13:31.012 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.012 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T12:13:31.012 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.012 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.012 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout:total 15 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Feb 17 12:13 . 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Feb 17 12:13 db_0.stat 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Feb 17 12:13 db_13494.stat 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Feb 17 12:13 global.stat 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 12:13 . 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-02-17T12:13:31.013 INFO:teuthology.orchestra.run.smithi094.stdout:total 152 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Feb 17 12:13 . 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 155648 Feb 17 12:13 001F 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.014 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:total 1048576 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Feb 17 12:13 . 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:13 0000000100000000000000AF 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:03 0000000100000000000000B0 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:04 0000000100000000000000B1 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:04 0000000100000000000000B2 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:03 0000000100000000000000B3 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:04 0000000100000000000000B4 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:04 0000000100000000000000B5 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:04 0000000100000000000000B6 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:04 0000000100000000000000B7 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:04 0000000100000000000000B8 2024-02-17T12:13:31.015 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:06 0000000100000000000000B9 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:05 0000000100000000000000BA 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:07 0000000100000000000000BB 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:06 0000000100000000000000BC 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:05 0000000100000000000000BD 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:04 0000000100000000000000BE 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000BF 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000C0 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000C1 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:12 0000000100000000000000C2 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:10 0000000100000000000000C3 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000C4 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:09 0000000100000000000000C5 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:10 0000000100000000000000C6 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:09 0000000100000000000000C7 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000C8 2024-02-17T12:13:31.016 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000C9 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000CA 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:11 0000000100000000000000CB 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000CC 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000CD 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000CE 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:09 0000000100000000000000CF 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:11 0000000100000000000000D0 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:12 0000000100000000000000D1 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000D2 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:09 0000000100000000000000D3 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000D4 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:09 0000000100000000000000D5 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000D6 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:09 0000000100000000000000D7 2024-02-17T12:13:31.017 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000D8 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:11 0000000100000000000000D9 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000DA 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000DB 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000DC 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000DD 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000DE 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:09 0000000100000000000000DF 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000E0 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:09 0000000100000000000000E1 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:10 0000000100000000000000E2 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000E3 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000E4 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000E5 2024-02-17T12:13:31.018 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:08 0000000100000000000000E6 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:12 0000000100000000000000E7 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:13 0000000100000000000000E8 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:13 0000000100000000000000E9 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:13 0000000100000000000000EA 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:13 0000000100000000000000EB 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:13 0000000100000000000000EC 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:13 0000000100000000000000ED 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Feb 17 12:13 0000000100000000000000EE 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 archive_status 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:total 0 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Feb 17 11:58 . 2024-02-17T12:13:31.019 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 65 Feb 17 12:13 .. 2024-02-17T12:13:31.020 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:31.020 INFO:teuthology.orchestra.run.smithi094.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-02-17T12:13:31.020 INFO:teuthology.orchestra.run.smithi094.stdout:total 512 2024-02-17T12:13:31.020 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Feb 17 12:03 . 2024-02-17T12:13:31.020 INFO:teuthology.orchestra.run.smithi094.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Feb 17 12:13 .. 2024-02-17T12:13:31.020 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Feb 17 12:03 0000 2024-02-17T12:13:31.020 INFO:teuthology.orchestra.run.smithi094.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 262144 Feb 17 12:13 0001 2024-02-17T12:13:31.020 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-02-17T12:13:31.032 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi094.front.sepia.ceph.com 2024-02-17T12:13:31.041 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-02-17T12:13:31.050 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-02-17T12:13:31.050 INFO:teuthology.task.exec:Executing custom commands... 2024-02-17T12:13:31.051 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi094.front.sepia.ceph.com 2024-02-17T12:13:31.051 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-02-17T12:13:31.115 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-02-17T12:13:31.206 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-17T12:13:31.312 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.308+0000 7f43bd29a640 1 -- 172.21.15.94:0/473357932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43b8105000 msgr2=0x7f43b81053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.308+0000 7f43bd29a640 1 --2- 172.21.15.94:0/473357932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43b8105000 0x7f43b81053e0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f43a8009900 tx=0x7f43a802bc30 comp rx=0 tx=0).stop 2024-02-17T12:13:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.308+0000 7f43bd29a640 1 -- 172.21.15.94:0/473357932 shutdown_connections 2024-02-17T12:13:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.308+0000 7f43bd29a640 1 --2- 172.21.15.94:0/473357932 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43b8100c70 0x7f43b810ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.308+0000 7f43bd29a640 1 --2- 172.21.15.94:0/473357932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43b81002d0 0x7f43b8100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.308+0000 7f43bd29a640 1 --2- 172.21.15.94:0/473357932 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43b8105000 0x7f43b81053e0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.308+0000 7f43bd29a640 1 -- 172.21.15.94:0/473357932 >> 172.21.15.94:0/473357932 conn(0x7f43b8076e50 msgr2=0x7f43b8077260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.308+0000 7f43bd29a640 1 -- 172.21.15.94:0/473357932 shutdown_connections 2024-02-17T12:13:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.308+0000 7f43bd29a640 1 -- 172.21.15.94:0/473357932 wait complete. 2024-02-17T12:13:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43bd29a640 1 Processor -- start 2024-02-17T12:13:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43bd29a640 1 -- start start 2024-02-17T12:13:31.313 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43bd29a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43b81002d0 0x7f43b8196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43bd29a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43b8100c70 0x7f43b8196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43bd29a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43b8105000 0x7f43b819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43bd29a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b819b580 con 0x7f43b8105000 2024-02-17T12:13:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43bd29a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b819b6f0 con 0x7f43b8100c70 2024-02-17T12:13:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43bd29a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b819b8b0 con 0x7f43b81002d0 2024-02-17T12:13:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43b8105000 0x7f43b819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43b77fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43b8105000 0x7f43b819af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40824/0 (socket says 172.21.15.94:40824) 2024-02-17T12:13:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43b77fe640 1 -- 172.21.15.94:0/1931802084 learned_addr learned my addr 172.21.15.94:0/1931802084 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43affff640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43b8100c70 0x7f43b8196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43b6ffd640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43b81002d0 0x7f43b8196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:31.314 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43b77fe640 1 -- 172.21.15.94:0/1931802084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43b81002d0 msgr2=0x7f43b8196710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:31.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43b77fe640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43b81002d0 0x7f43b8196710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43b77fe640 1 -- 172.21.15.94:0/1931802084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43b8100c70 msgr2=0x7f43b8196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:31.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43b77fe640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43b8100c70 0x7f43b8196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43b77fe640 1 -- 172.21.15.94:0/1931802084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a8009560 con 0x7f43b8105000 2024-02-17T12:13:31.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43b6ffd640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43b81002d0 0x7f43b8196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:13:31.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.309+0000 7f43b77fe640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43b8105000 0x7f43b819af80 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f43b000b970 tx=0x7f43b000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:31.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.310+0000 7f43b4ff9640 1 -- 172.21.15.94:0/1931802084 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b000cb40 con 0x7f43b8105000 2024-02-17T12:13:31.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.310+0000 7f43b4ff9640 1 -- 172.21.15.94:0/1931802084 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f43b000cca0 con 0x7f43b8105000 2024-02-17T12:13:31.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.310+0000 7f43bd29a640 1 -- 172.21.15.94:0/1931802084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b819bb90 con 0x7f43b8105000 2024-02-17T12:13:31.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.310+0000 7f43b4ff9640 1 -- 172.21.15.94:0/1931802084 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b00077d0 con 0x7f43b8105000 2024-02-17T12:13:31.315 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.310+0000 7f43bd29a640 1 -- 172.21.15.94:0/1931802084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b81a3410 con 0x7f43b8105000 2024-02-17T12:13:31.316 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.311+0000 7f43bd29a640 1 -- 172.21.15.94:0/1931802084 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b8101790 con 0x7f43b8105000 2024-02-17T12:13:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.312+0000 7f43b4ff9640 1 -- 172.21.15.94:0/1931802084 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f43b0014440 con 0x7f43b8105000 2024-02-17T12:13:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.312+0000 7f43b4ff9640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43a40a22d0 0x7f43a40a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.312+0000 7f43b6ffd640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43a40a22d0 0x7f43a40a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.312+0000 7f43b4ff9640 1 -- 172.21.15.94:0/1931802084 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f43b00cdd70 con 0x7f43b8105000 2024-02-17T12:13:31.317 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.313+0000 7f43b6ffd640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43a40a22d0 0x7f43a40a4790 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f43a8005ec0 tx=0x7f43a8000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:31.318 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.316+0000 7f43b4ff9640 1 -- 172.21.15.94:0/1931802084 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43b0096b80 con 0x7f43b8105000 2024-02-17T12:13:31.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:31 smithi125 ceph-mon[36819]: pgmap v483: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 18 MiB/s wr, 1.58k op/s 2024-02-17T12:13:31.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:31 smithi094 ceph-mon[32499]: pgmap v483: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 18 MiB/s wr, 1.58k op/s 2024-02-17T12:13:31.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.513+0000 7f43bd29a640 1 -- 172.21.15.94:0/1931802084 --> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f43b8110340 con 0x7f43a40a22d0 2024-02-17T12:13:31.523 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.521+0000 7f43b4ff9640 1 -- 172.21.15.94:0/1931802084 <== mgr.15435 v2:172.21.15.94:6800/2106228828 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+350 (secure 0 0 0) 0x7f43b8110340 con 0x7f43a40a22d0 2024-02-17T12:13:31.523 INFO:teuthology.orchestra.run.smithi094.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-17T00:00:00", "created": "2024-02-17T11:58:07", "first": "2024-02-17T11:59:00", "last": "2024-02-17T12:13:00", "last_pruned": "2024-02-17T12:13:00", "created_count": 15, "pruned_count": 8, "active": true} 2024-02-17T12:13:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.523+0000 7f43bd29a640 1 -- 172.21.15.94:0/1931802084 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43a40a22d0 msgr2=0x7f43a40a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.523+0000 7f43bd29a640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43a40a22d0 0x7f43a40a4790 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f43a8005ec0 tx=0x7f43a8000bc0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.523+0000 7f43bd29a640 1 -- 172.21.15.94:0/1931802084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43b8105000 msgr2=0x7f43b819af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.523+0000 7f43bd29a640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43b8105000 0x7f43b819af80 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f43b000b970 tx=0x7f43b000be30 comp rx=0 tx=0).stop 2024-02-17T12:13:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.523+0000 7f43bd29a640 1 -- 172.21.15.94:0/1931802084 shutdown_connections 2024-02-17T12:13:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.523+0000 7f43bd29a640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f43a40a22d0 0x7f43a40a4790 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.523+0000 7f43bd29a640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f43b8105000 0x7f43b819af80 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.523+0000 7f43bd29a640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f43b8100c70 0x7f43b8196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.523+0000 7f43bd29a640 1 --2- 172.21.15.94:0/1931802084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f43b81002d0 0x7f43b8196710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.523+0000 7f43bd29a640 1 -- 172.21.15.94:0/1931802084 >> 172.21.15.94:0/1931802084 conn(0x7f43b8076e50 msgr2=0x7f43b80fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.523+0000 7f43bd29a640 1 -- 172.21.15.94:0/1931802084 shutdown_connections 2024-02-17T12:13:31.526 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.523+0000 7f43bd29a640 1 -- 172.21.15.94:0/1931802084 wait complete. 2024-02-17T12:13:31.536 DEBUG:teuthology.orchestra.run.smithi094:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-17T12:13:31.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.637+0000 7f6bc48dd640 1 -- 172.21.15.94:0/3917024026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc00ff680 msgr2=0x7f6bc00ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.637+0000 7f6bc48dd640 1 --2- 172.21.15.94:0/3917024026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc00ff680 0x7f6bc00ffb00 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8009930 tx=0x7f6ba802bc30 comp rx=0 tx=0).stop 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.637+0000 7f6bc48dd640 1 -- 172.21.15.94:0/3917024026 shutdown_connections 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.637+0000 7f6bc48dd640 1 --2- 172.21.15.94:0/3917024026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bc0100040 0x7f6bc010e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.637+0000 7f6bc48dd640 1 --2- 172.21.15.94:0/3917024026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc00ff680 0x7f6bc00ffb00 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.637+0000 7f6bc48dd640 1 --2- 172.21.15.94:0/3917024026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bc0106160 0x7f6bc0106540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.637+0000 7f6bc48dd640 1 -- 172.21.15.94:0/3917024026 >> 172.21.15.94:0/3917024026 conn(0x7f6bc0076610 msgr2=0x7f6bc00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.637+0000 7f6bc48dd640 1 -- 172.21.15.94:0/3917024026 shutdown_connections 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.637+0000 7f6bc48dd640 1 -- 172.21.15.94:0/3917024026 wait complete. 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bc48dd640 1 Processor -- start 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bc48dd640 1 -- start start 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bc48dd640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bc00ff680 0x7f6bc019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bc48dd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc0100040 0x7f6bc019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bc48dd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bc0106160 0x7f6bc01975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:31.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bc48dd640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0197c30 con 0x7f6bc0100040 2024-02-17T12:13:31.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bc48dd640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0197da0 con 0x7f6bc00ff680 2024-02-17T12:13:31.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bc48dd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0197f60 con 0x7f6bc0106160 2024-02-17T12:13:31.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bbdd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc0100040 0x7f6bc019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:31.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bbdd9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc0100040 0x7f6bc019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:40844/0 (socket says 172.21.15.94:40844) 2024-02-17T12:13:31.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bbdd9b640 1 -- 172.21.15.94:0/4127847876 learned_addr learned my addr 172.21.15.94:0/4127847876 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:31.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bbed9d640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bc0106160 0x7f6bc01975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:31.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bbe59c640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bc00ff680 0x7f6bc019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:31.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bbdd9b640 1 -- 172.21.15.94:0/4127847876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bc0106160 msgr2=0x7f6bc01975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:31.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bbdd9b640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bc0106160 0x7f6bc01975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bbdd9b640 1 -- 172.21.15.94:0/4127847876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bc00ff680 msgr2=0x7f6bc019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:31.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bbdd9b640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bc00ff680 0x7f6bc019d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bbdd9b640 1 -- 172.21.15.94:0/4127847876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba8009560 con 0x7f6bc0100040 2024-02-17T12:13:31.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.638+0000 7f6bbed9d640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bc0106160 0x7f6bc01975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:13:31.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.639+0000 7f6bbdd9b640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc0100040 0x7f6bc019d970 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8005940 tx=0x7f6ba80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:31.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.639+0000 7f6bb77fe640 1 -- 172.21.15.94:0/4127847876 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba80044e0 con 0x7f6bc0100040 2024-02-17T12:13:31.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.639+0000 7f6bc48dd640 1 -- 172.21.15.94:0/4127847876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc01981e0 con 0x7f6bc0100040 2024-02-17T12:13:31.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.639+0000 7f6bb77fe640 1 -- 172.21.15.94:0/4127847876 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6ba802cbc0 con 0x7f6bc0100040 2024-02-17T12:13:31.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.639+0000 7f6bb77fe640 1 -- 172.21.15.94:0/4127847876 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8003e20 con 0x7f6bc0100040 2024-02-17T12:13:31.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.639+0000 7f6bc48dd640 1 -- 172.21.15.94:0/4127847876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc01a44d0 con 0x7f6bc0100040 2024-02-17T12:13:31.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.640+0000 7f6bc48dd640 1 -- 172.21.15.94:0/4127847876 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc01029a0 con 0x7f6bc0100040 2024-02-17T12:13:31.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.641+0000 7f6bb77fe640 1 -- 172.21.15.94:0/4127847876 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f6ba80035f0 con 0x7f6bc0100040 2024-02-17T12:13:31.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.641+0000 7f6bb77fe640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6bac0a22d0 0x7f6bac0a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:31.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.641+0000 7f6bbe59c640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6bac0a22d0 0x7f6bac0a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:31.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.641+0000 7f6bb77fe640 1 -- 172.21.15.94:0/4127847876 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f6ba80f03b0 con 0x7f6bc0100040 2024-02-17T12:13:31.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.642+0000 7f6bbe59c640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6bac0a22d0 0x7f6bac0a4790 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80058c0 tx=0x7f6bb8005c60 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:31.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.646+0000 7f6bb77fe640 1 -- 172.21.15.94:0/4127847876 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6ba80b9110 con 0x7f6bc0100040 2024-02-17T12:13:31.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:31 smithi189 ceph-mon[36588]: pgmap v483: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 18 MiB/s wr, 1.58k op/s 2024-02-17T12:13:31.841 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.838+0000 7f6bc48dd640 1 -- 172.21.15.94:0/4127847876 --> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6bc01113a0 con 0x7f6bac0a22d0 2024-02-17T12:13:31.848 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.846+0000 7f6bb77fe640 1 -- 172.21.15.94:0/4127847876 <== mgr.15435 v2:172.21.15.94:6800/2106228828 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f6bc01113a0 con 0x7f6bac0a22d0 2024-02-17T12:13:31.848 INFO:teuthology.orchestra.run.smithi094.stdout:/ 1m 6m3h 2024-02-17T12:13:31.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.848+0000 7f6bc48dd640 1 -- 172.21.15.94:0/4127847876 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6bac0a22d0 msgr2=0x7f6bac0a4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:31.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.848+0000 7f6bc48dd640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6bac0a22d0 0x7f6bac0a4790 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80058c0 tx=0x7f6bb8005c60 comp rx=0 tx=0).stop 2024-02-17T12:13:31.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.848+0000 7f6bc48dd640 1 -- 172.21.15.94:0/4127847876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc0100040 msgr2=0x7f6bc019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:31.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.848+0000 7f6bc48dd640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc0100040 0x7f6bc019d970 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8005940 tx=0x7f6ba80042f0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.848+0000 7f6bc48dd640 1 -- 172.21.15.94:0/4127847876 shutdown_connections 2024-02-17T12:13:31.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.848+0000 7f6bc48dd640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f6bac0a22d0 0x7f6bac0a4790 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.848+0000 7f6bc48dd640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bc0106160 0x7f6bc01975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.848+0000 7f6bc48dd640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f6bc0100040 0x7f6bc019d970 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.851 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.848+0000 7f6bc48dd640 1 --2- 172.21.15.94:0/4127847876 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f6bc00ff680 0x7f6bc019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:31.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.848+0000 7f6bc48dd640 1 -- 172.21.15.94:0/4127847876 >> 172.21.15.94:0/4127847876 conn(0x7f6bc0076610 msgr2=0x7f6bc0104190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:31.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.848+0000 7f6bc48dd640 1 -- 172.21.15.94:0/4127847876 shutdown_connections 2024-02-17T12:13:31.852 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:31.848+0000 7f6bc48dd640 1 -- 172.21.15.94:0/4127847876 wait complete. 2024-02-17T12:13:31.862 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-02-17T12:13:31.873 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-02-17T12:13:32.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:32 smithi125 ceph-mon[36819]: from='client.19503 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:13:32.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:32 smithi125 ceph-mon[36819]: from='client.19509 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:13:32.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:32 smithi094 ceph-mon[32499]: from='client.19503 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:13:32.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:32 smithi094 ceph-mon[32499]: from='client.19509 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:13:32.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:32 smithi189 ceph-mon[36588]: from='client.19503 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:13:32.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:32 smithi189 ceph-mon[36588]: from='client.19509 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-17T12:13:33.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:33 smithi125 ceph-mon[36819]: pgmap v484: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 17 MiB/s wr, 1.50k op/s 2024-02-17T12:13:33.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:33 smithi094 ceph-mon[32499]: pgmap v484: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 17 MiB/s wr, 1.50k op/s 2024-02-17T12:13:33.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:33 smithi189 ceph-mon[36588]: pgmap v484: 97 pgs: 97 active+clean; 5.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 17 MiB/s wr, 1.50k op/s 2024-02-17T12:13:34.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:34 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:13:34.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:34 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:13:34.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:34 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:13:35.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:35 smithi125 ceph-mon[36819]: pgmap v485: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-02-17T12:13:35.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:35 smithi094 ceph-mon[32499]: pgmap v485: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-02-17T12:13:35.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:35 smithi189 ceph-mon[36588]: pgmap v485: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-02-17T12:13:36.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:36 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:13:36.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:36 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:13:36.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:36 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:13:36.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:36 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:13:36.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:36 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:13:36.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:36 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:13:36.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:36 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:13:36.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:36 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:13:36.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:36 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:13:37.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:37 smithi125 ceph-mon[36819]: pgmap v486: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.2 MiB/s wr, 670 op/s 2024-02-17T12:13:37.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:37 smithi094 ceph-mon[32499]: pgmap v486: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.2 MiB/s wr, 670 op/s 2024-02-17T12:13:37.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:37 smithi189 ceph-mon[36588]: pgmap v486: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 6.2 MiB/s wr, 670 op/s 2024-02-17T12:13:39.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:39 smithi125 ceph-mon[36819]: pgmap v487: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.3 MiB/s wr, 586 op/s 2024-02-17T12:13:39.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:39 smithi094 ceph-mon[32499]: pgmap v487: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.3 MiB/s wr, 586 op/s 2024-02-17T12:13:39.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:39 smithi189 ceph-mon[36588]: pgmap v487: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.3 MiB/s wr, 586 op/s 2024-02-17T12:13:41.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:41 smithi125 ceph-mon[36819]: pgmap v488: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-02-17T12:13:41.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:41 smithi094 ceph-mon[32499]: pgmap v488: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-02-17T12:13:41.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:41 smithi189 ceph-mon[36588]: pgmap v488: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.7 MiB/s wr, 190 op/s 2024-02-17T12:13:43.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:43 smithi125 ceph-mon[36819]: pgmap v489: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 932 KiB/s wr, 106 op/s 2024-02-17T12:13:43.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:43 smithi094 ceph-mon[32499]: pgmap v489: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 932 KiB/s wr, 106 op/s 2024-02-17T12:13:43.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:43 smithi189 ceph-mon[36588]: pgmap v489: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 932 KiB/s wr, 106 op/s 2024-02-17T12:13:45.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:45 smithi125 ceph-mon[36819]: pgmap v490: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 412 KiB/s wr, 48 op/s 2024-02-17T12:13:45.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:45 smithi094 ceph-mon[32499]: pgmap v490: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 412 KiB/s wr, 48 op/s 2024-02-17T12:13:45.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:45 smithi189 ceph-mon[36588]: pgmap v490: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 412 KiB/s wr, 48 op/s 2024-02-17T12:13:47.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:47 smithi125 ceph-mon[36819]: pgmap v491: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 KiB/s wr, 6 op/s 2024-02-17T12:13:47.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:47 smithi094 ceph-mon[32499]: pgmap v491: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 KiB/s wr, 6 op/s 2024-02-17T12:13:47.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:47 smithi189 ceph-mon[36588]: pgmap v491: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 KiB/s wr, 6 op/s 2024-02-17T12:13:48.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-02-17T12:13:48.722 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-17T12:13:48.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.840+0000 7fa5bf148640 1 -- 172.21.15.94:0/1649149421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b8105340 msgr2=0x7fa5b810cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:48.844 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.840+0000 7fa5bf148640 1 --2- 172.21.15.94:0/1649149421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b8105340 0x7fa5b810cf10 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0009cd0 tx=0x7fa5b002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.840+0000 7fa5bf148640 1 -- 172.21.15.94:0/1649149421 shutdown_connections 2024-02-17T12:13:48.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.840+0000 7fa5bf148640 1 --2- 172.21.15.94:0/1649149421 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b8105340 0x7fa5b810cf10 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.840+0000 7fa5bf148640 1 --2- 172.21.15.94:0/1649149421 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5b8100c20 0x7fa5b8104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.840+0000 7fa5bf148640 1 --2- 172.21.15.94:0/1649149421 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5b8100270 0x7fa5b8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.840+0000 7fa5bf148640 1 -- 172.21.15.94:0/1649149421 >> 172.21.15.94:0/1649149421 conn(0x7fa5b8077230 msgr2=0x7fa5b8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:48.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.840+0000 7fa5bf148640 1 -- 172.21.15.94:0/1649149421 shutdown_connections 2024-02-17T12:13:48.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.840+0000 7fa5bf148640 1 -- 172.21.15.94:0/1649149421 wait complete. 2024-02-17T12:13:48.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bf148640 1 Processor -- start 2024-02-17T12:13:48.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bf148640 1 -- start start 2024-02-17T12:13:48.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bf148640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5b8100270 0x7fa5b81976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bf148640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5b8105340 0x7fa5b8197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bf148640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b819bf50 0x7fa5b819c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.845 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bf148640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b819c940 con 0x7fa5b819bf50 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bf148640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b819cab0 con 0x7fa5b8105340 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bf148640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b819cc20 con 0x7fa5b8100270 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bd6e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b819bf50 0x7fa5b819c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bd6e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b819bf50 0x7fa5b819c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57112/0 (socket says 172.21.15.94:57112) 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bd6e5640 1 -- 172.21.15.94:0/2295285424 learned_addr learned my addr 172.21.15.94:0/2295285424 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bcee4640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5b8100270 0x7fa5b81976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5b7fff640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5b8105340 0x7fa5b8197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bd6e5640 1 -- 172.21.15.94:0/2295285424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5b8100270 msgr2=0x7fa5b81976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bd6e5640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5b8100270 0x7fa5b81976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bd6e5640 1 -- 172.21.15.94:0/2295285424 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5b8105340 msgr2=0x7fa5b8197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bd6e5640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5b8105340 0x7fa5b8197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bd6e5640 1 -- 172.21.15.94:0/2295285424 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b0009900 con 0x7fa5b819bf50 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.841+0000 7fa5bd6e5640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b819bf50 0x7fa5b819c400 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00348f0 tx=0x7fa5b0034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:48.846 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.842+0000 7fa5b5ffb640 1 -- 172.21.15.94:0/2295285424 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00328c0 con 0x7fa5b819bf50 2024-02-17T12:13:48.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.842+0000 7fa5b5ffb640 1 -- 172.21.15.94:0/2295285424 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0003e50 con 0x7fa5b819bf50 2024-02-17T12:13:48.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.842+0000 7fa5bf148640 1 -- 172.21.15.94:0/2295285424 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b81a4200 con 0x7fa5b819bf50 2024-02-17T12:13:48.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.842+0000 7fa5b5ffb640 1 -- 172.21.15.94:0/2295285424 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0040450 con 0x7fa5b819bf50 2024-02-17T12:13:48.847 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.842+0000 7fa5bf148640 1 -- 172.21.15.94:0/2295285424 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b81a4680 con 0x7fa5b819bf50 2024-02-17T12:13:48.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.844+0000 7fa5b5ffb640 1 -- 172.21.15.94:0/2295285424 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fa5b000a040 con 0x7fa5b819bf50 2024-02-17T12:13:48.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5c1df9740 1 Processor -- start 2024-02-17T12:13:48.853 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5c1df9740 1 -- start start 2024-02-17T12:13:48.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5b5ffb640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5a00a22b0 0x7fa5a00a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5c1df9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b8452b0 0x55556b845690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5c1df9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b845bd0 0x55556b84e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5c1df9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b84e7b0 0x55556b850ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5c1df9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556b7b4280 con 0x55556b84e7b0 2024-02-17T12:13:48.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5c1df9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556b7b17b0 con 0x55556b8452b0 2024-02-17T12:13:48.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5c1df9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556b7acf90 con 0x55556b845bd0 2024-02-17T12:13:48.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5b5ffb640 1 -- 172.21.15.94:0/2295285424 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa5b00f0d80 con 0x7fa5b819bf50 2024-02-17T12:13:48.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5b7fff640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5a00a22b0 0x7fa5a00a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5bd6e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b84e7b0 0x55556b850ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5bd6e5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b84e7b0 0x55556b850ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57122/0 (socket says 172.21.15.94:57122) 2024-02-17T12:13:48.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5bd6e5640 1 -- 172.21.15.94:0/2700293360 learned_addr learned my addr 172.21.15.94:0/2700293360 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:48.854 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5bcee4640 1 --2- 172.21.15.94:0/2700293360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b8452b0 0x55556b845690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5b7fff640 1 --2- 172.21.15.94:0/2700293360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b845bd0 0x55556b84e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5bd6e5640 1 -- 172.21.15.94:0/2700293360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b845bd0 msgr2=0x55556b84e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:48.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5bd6e5640 1 --2- 172.21.15.94:0/2700293360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b845bd0 0x55556b84e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5bd6e5640 1 -- 172.21.15.94:0/2700293360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b8452b0 msgr2=0x55556b845690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:48.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5bd6e5640 1 --2- 172.21.15.94:0/2700293360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b8452b0 0x55556b845690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5bd6e5640 1 -- 172.21.15.94:0/2700293360 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55556b76b570 con 0x55556b84e7b0 2024-02-17T12:13:48.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.845+0000 7fa5b7fff640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5a00a22b0 0x7fa5a00a4770 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac005e10 tx=0x7fa5ac0127c0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:48.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.846+0000 7fa5bd6e5640 1 --2- 172.21.15.94:0/2700293360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b84e7b0 0x55556b850ba0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00ba210 tx=0x7fa5b00bd6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4745ae42425174c8 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:48.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.846+0000 7fa59effd640 1 -- 172.21.15.94:0/2700293360 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0091c70 con 0x55556b84e7b0 2024-02-17T12:13:48.855 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.846+0000 7fa59effd640 1 -- 172.21.15.94:0/2700293360 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0089030 con 0x55556b84e7b0 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.846+0000 7fa59effd640 1 -- 172.21.15.94:0/2700293360 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00a8110 con 0x55556b84e7b0 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.846+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2700293360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b84e7b0 msgr2=0x55556b850ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.846+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2700293360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b84e7b0 0x55556b850ba0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00ba210 tx=0x7fa5b00bd6d0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.846+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2700293360 shutdown_connections 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.846+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2700293360 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b84e7b0 0x55556b850ba0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.846+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2700293360 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b845bd0 0x55556b84e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.846+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2700293360 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b8452b0 0x55556b845690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.846+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2700293360 >> 172.21.15.94:0/2700293360 conn(0x55556b83f990 msgr2=0x55556b83fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.846+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2700293360 shutdown_connections 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.846+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2700293360 wait complete. 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.847+0000 7fa5c1df9740 1 -- start start 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.847+0000 7fa5c1df9740 1 Processor -- start 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.847+0000 7fa5c1df9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b8452b0 0x55556b84b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.856 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.847+0000 7fa5c1df9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 0x55556b84bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.847+0000 7fa5c1df9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b84e7b0 0x55556b84c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.847+0000 7fa5c1df9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556b480430 con 0x55556b845bd0 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.847+0000 7fa5c1df9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556b43ce00 con 0x55556b84e7b0 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.847+0000 7fa5c1df9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556b731fd0 con 0x55556b8452b0 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.847+0000 7fa5c1df9740 10 client.0 _ll_register_callbacks cb 0x55556b787fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.847+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 0x55556b84bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.847+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 0x55556b84bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57130/0 (socket says 172.21.15.94:57130) 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.847+0000 7fa5b7fff640 1 -- 172.21.15.94:0/79656359 learned_addr learned my addr 172.21.15.94:0/79656359 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa5bcee4640 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b8452b0 0x55556b84b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa5bd6e5640 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b84e7b0 0x55556b84c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa5b7fff640 1 -- 172.21.15.94:0/79656359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b8452b0 msgr2=0x55556b84b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa5b7fff640 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b8452b0 0x55556b84b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa5b7fff640 1 -- 172.21.15.94:0/79656359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b84e7b0 msgr2=0x55556b84c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:48.857 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa5b7fff640 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b84e7b0 0x55556b84c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa5b7fff640 1 -- 172.21.15.94:0/79656359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b81a4680 con 0x55556b845bd0 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa5bd6e5640 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b84e7b0 0x55556b84c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa5b7fff640 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 0x55556b84bb00 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac010b10 tx=0x7fa5ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da24f041f5a2a954 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa597fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa597fff640 1 -- 172.21.15.94:0/79656359 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00da50 con 0x55556b845bd0 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa597fff640 1 -- 172.21.15.94:0/79656359 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa5ac00dbb0 con 0x55556b845bd0 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa5c1df9740 1 -- 172.21.15.94:0/79656359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55556b4ac7e0 con 0x55556b845bd0 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa597fff640 1 -- 172.21.15.94:0/79656359 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5ac00a8b0 con 0x55556b845bd0 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa5c1df9740 10 client.19521 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa5c1df9740 1 -- 172.21.15.94:0/79656359 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55556b7ab610 con 0x55556b845bd0 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.848+0000 7fa597fff640 1 -- 172.21.15.94:0/79656359 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55556b4ac7e0 con 0x55556b845bd0 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.849+0000 7fa597fff640 1 -- 172.21.15.94:0/79656359 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa5ac02e070 con 0x55556b845bd0 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.849+0000 7fa5c1df9740 10 client.19521 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:48.858 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.849+0000 7fa5c1df9740 10 client.19521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.849+0000 7fa5c1df9740 20 client.19521 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.849+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55556b85e7c0 0x55556b860be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.849+0000 7fa5c1df9740 4 client.19521 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.849+0000 7fa5c1df9740 1 -- 172.21.15.94:0/79656359 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55556b4e9720 con 0x55556b85e7c0 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.849+0000 7fa5bcee4640 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55556b85e7c0 0x55556b860be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.849+0000 7fa597fff640 10 client.19521 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.849+0000 7fa5bcee4640 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55556b85e7c0 0x55556b860be0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f64c10721e911b6 server_cookie=e22c27d4f9293cb4 in_seq=0 out_seq=0 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa597fff640 1 -- 172.21.15.94:0/79656359 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55556b4e9720 con 0x55556b85e7c0 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa597fff640 10 client.19521 handle_command_reply: tid=0 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa5c1df9740 1 client.19521 shutdown 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa5c1df9740 1 -- 172.21.15.94:0/79656359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 msgr2=0x55556b84bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 0x55556b84bb00 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac010b10 tx=0x7fa5ac004730 comp rx=0 tx=0).stop 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa5c1df9740 1 -- 172.21.15.94:0/79656359 shutdown_connections 2024-02-17T12:13:48.859 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55556b85e7c0 0x55556b860be0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b84e7b0 0x55556b84c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa597fff640 0 client.19521 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:48.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 0x55556b84bb00 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/79656359 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b8452b0 0x55556b84b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa5c1df9740 1 -- 172.21.15.94:0/79656359 >> 172.21.15.94:0/79656359 conn(0x55556b83f990 msgr2=0x55556b841730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:48.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa5c1df9740 1 -- 172.21.15.94:0/79656359 shutdown_connections 2024-02-17T12:13:48.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa5c1df9740 1 -- 172.21.15.94:0/79656359 wait complete. 2024-02-17T12:13:48.860 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.850+0000 7fa5c1df9740 20 client.19521 trim_cache size 0 max 16384 2024-02-17T12:13:48.873 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.865+0000 7fa5c1df9740 1 Processor -- start 2024-02-17T12:13:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.865+0000 7fa5c1df9740 1 -- start start 2024-02-17T12:13:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.865+0000 7fa5c1df9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b8452b0 0x55556b84a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.865+0000 7fa5c1df9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 0x55556b84be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.865+0000 7fa5c1df9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b84e7b0 0x55556b848820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.865+0000 7fa5c1df9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556b83b0f0 con 0x55556b845bd0 2024-02-17T12:13:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.865+0000 7fa5c1df9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556b8409e0 con 0x55556b8452b0 2024-02-17T12:13:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.865+0000 7fa5c1df9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556b844cf0 con 0x55556b84e7b0 2024-02-17T12:13:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 0x55556b84be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa5b7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 0x55556b84be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57136/0 (socket says 172.21.15.94:57136) 2024-02-17T12:13:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa5b7fff640 1 -- 172.21.15.94:0/2782854474 learned_addr learned my addr 172.21.15.94:0/2782854474 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa5b7fff640 1 -- 172.21.15.94:0/2782854474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b84e7b0 msgr2=0x55556b848820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa5b7fff640 1 --2- 172.21.15.94:0/2782854474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b84e7b0 0x55556b848820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.874 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa5bcee4640 1 --2- 172.21.15.94:0/2782854474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b8452b0 0x55556b84a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa5b7fff640 1 -- 172.21.15.94:0/2782854474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b8452b0 msgr2=0x55556b84a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa5b7fff640 1 --2- 172.21.15.94:0/2782854474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b8452b0 0x55556b84a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa5b7fff640 1 -- 172.21.15.94:0/2782854474 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55556b747ac0 con 0x55556b845bd0 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa5bcee4640 1 --2- 172.21.15.94:0/2782854474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b8452b0 0x55556b84a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa5b7fff640 1 --2- 172.21.15.94:0/2782854474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 0x55556b84be60 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac02f3a0 tx=0x7fa5ac00d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab30df957ec2771 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa59e7fc640 1 -- 172.21.15.94:0/2782854474 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00a770 con 0x55556b845bd0 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa59e7fc640 1 -- 172.21.15.94:0/2782854474 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5ac018070 con 0x55556b845bd0 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.866+0000 7fa59e7fc640 1 -- 172.21.15.94:0/2782854474 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac01fa10 con 0x55556b845bd0 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2782854474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 msgr2=0x55556b84be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2782854474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 0x55556b84be60 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac02f3a0 tx=0x7fa5ac00d7d0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2782854474 shutdown_connections 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2782854474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b84e7b0 0x55556b848820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2782854474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b845bd0 0x55556b84be60 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.875 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2782854474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b8452b0 0x55556b84a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2782854474 >> 172.21.15.94:0/2782854474 conn(0x55556b83f990 msgr2=0x55556b849e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2782854474 shutdown_connections 2024-02-17T12:13:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2782854474 wait complete. 2024-02-17T12:13:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 -- start start 2024-02-17T12:13:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 Processor -- start 2024-02-17T12:13:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b8452b0 0x55556b855890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b845bd0 0x55556b853c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b84e7b0 0x55556b8542b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556b8549b0 con 0x55556b8452b0 2024-02-17T12:13:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556b8b4cf0 con 0x55556b84e7b0 2024-02-17T12:13:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55556b8b4e30 con 0x55556b845bd0 2024-02-17T12:13:48.876 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.867+0000 7fa5c1df9740 10 client.0 _ll_register_callbacks cb 0x7fa5b0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:48.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa5bcee4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b8452b0 0x55556b855890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa5bcee4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b8452b0 0x55556b855890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57144/0 (socket says 172.21.15.94:57144) 2024-02-17T12:13:48.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa5bcee4640 1 -- 172.21.15.94:0/2666244274 learned_addr learned my addr 172.21.15.94:0/2666244274 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:48.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa5bcee4640 1 -- 172.21.15.94:0/2666244274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b845bd0 msgr2=0x55556b853c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:48.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa5bcee4640 1 --2- 172.21.15.94:0/2666244274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b845bd0 0x55556b853c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa5bcee4640 1 -- 172.21.15.94:0/2666244274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b84e7b0 msgr2=0x55556b8542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:48.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa5bcee4640 1 --2- 172.21.15.94:0/2666244274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b84e7b0 0x55556b8542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa5bcee4640 1 -- 172.21.15.94:0/2666244274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55556b7ab610 con 0x55556b8452b0 2024-02-17T12:13:48.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa5bcee4640 1 --2- 172.21.15.94:0/2666244274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b8452b0 0x55556b855890 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x55556b8614e0 tx=0x7fa5a80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f042c1f51c903867 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:48.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa597fff640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:48.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa597fff640 1 -- 172.21.15.94:0/2666244274 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a8016040 con 0x55556b8452b0 2024-02-17T12:13:48.877 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa597fff640 1 -- 172.21.15.94:0/2666244274 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fa5a8004220 con 0x55556b8452b0 2024-02-17T12:13:48.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2666244274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa5b0089030 con 0x55556b8452b0 2024-02-17T12:13:48.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa597fff640 1 -- 172.21.15.94:0/2666244274 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5a80043e0 con 0x55556b8452b0 2024-02-17T12:13:48.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa5c1df9740 10 client.19527 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:48.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2666244274 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55556b76d6a0 con 0x55556b8452b0 2024-02-17T12:13:48.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.868+0000 7fa597fff640 1 -- 172.21.15.94:0/2666244274 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0089030 con 0x55556b8452b0 2024-02-17T12:13:48.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.869+0000 7fa597fff640 1 -- 172.21.15.94:0/2666244274 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7fa5a8003c50 con 0x55556b8452b0 2024-02-17T12:13:48.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.869+0000 7fa5c1df9740 10 client.19527 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:48.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.869+0000 7fa5c1df9740 10 client.19527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:48.878 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.869+0000 7fa5c1df9740 20 client.19527 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.869+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2666244274 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55556b8b6540 0x55556b8b89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.869+0000 7fa5c1df9740 4 client.19527 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-02-17T12:13:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.869+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2666244274 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55556b538890 con 0x55556b8b6540 2024-02-17T12:13:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.869+0000 7fa5b7fff640 1 --2- 172.21.15.94:0/2666244274 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55556b8b6540 0x55556b8b89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.869+0000 7fa5b7fff640 1 --2- 172.21.15.94:0/2666244274 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55556b8b6540 0x55556b8b89a0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c55b7885db208204 server_cookie=3a4bc0a59d7b6115 in_seq=0 out_seq=0 2024-02-17T12:13:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.869+0000 7fa597fff640 10 client.19527 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.869+0000 7fa597fff640 1 -- 172.21.15.94:0/2666244274 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55556b538890 con 0x55556b8b6540 2024-02-17T12:13:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.869+0000 7fa597fff640 10 client.19527 handle_command_reply: tid=0 2024-02-17T12:13:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5c1df9740 1 client.19527 shutdown 2024-02-17T12:13:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2666244274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b8452b0 msgr2=0x55556b855890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:48.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2666244274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b8452b0 0x55556b855890 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x55556b8614e0 tx=0x7fa5a80118d0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2666244274 shutdown_connections 2024-02-17T12:13:48.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2666244274 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55556b8b6540 0x55556b8b89a0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2666244274 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55556b84e7b0 0x55556b8542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2666244274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55556b845bd0 0x55556b853c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa597fff640 0 client.19527 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:48.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5c1df9740 1 --2- 172.21.15.94:0/2666244274 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55556b8452b0 0x55556b855890 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2666244274 >> 172.21.15.94:0/2666244274 conn(0x55556b83f990 msgr2=0x55556b841430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:48.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2666244274 shutdown_connections 2024-02-17T12:13:48.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5c1df9740 1 -- 172.21.15.94:0/2666244274 wait complete. 2024-02-17T12:13:48.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5c1df9740 20 client.19527 trim_cache size 0 max 16384 2024-02-17T12:13:48.880 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:13:48.881 INFO:teuthology.orchestra.run.smithi094.stdout: "status": "no active scrubs running", 2024-02-17T12:13:48.881 INFO:teuthology.orchestra.run.smithi094.stdout: "scrubs": {} 2024-02-17T12:13:48.881 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:13:48.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5bf148640 1 -- 172.21.15.94:0/2295285424 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5a00a22b0 msgr2=0x7fa5a00a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:48.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5bf148640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5a00a22b0 0x7fa5a00a4770 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac005e10 tx=0x7fa5ac0127c0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5bf148640 1 -- 172.21.15.94:0/2295285424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b819bf50 msgr2=0x7fa5b819c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:48.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5bf148640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b819bf50 0x7fa5b819c400 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fa5b00348f0 tx=0x7fa5b0034cc0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5bf148640 1 -- 172.21.15.94:0/2295285424 shutdown_connections 2024-02-17T12:13:48.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5bf148640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fa5a00a22b0 0x7fa5a00a4770 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5bf148640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fa5b819bf50 0x7fa5b819c400 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5bf148640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fa5b8105340 0x7fa5b8197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5bf148640 1 --2- 172.21.15.94:0/2295285424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5b8100270 0x7fa5b81976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.870+0000 7fa5bf148640 1 -- 172.21.15.94:0/2295285424 >> 172.21.15.94:0/2295285424 conn(0x7fa5b8077230 msgr2=0x7fa5b8068560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:48.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.871+0000 7fa5bf148640 1 -- 172.21.15.94:0/2295285424 shutdown_connections 2024-02-17T12:13:48.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.871+0000 7fa5bf148640 1 -- 172.21.15.94:0/2295285424 wait complete. 2024-02-17T12:13:48.884 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-17T12:13:48.884 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:13:48.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.984+0000 7f738b09a640 1 -- 172.21.15.94:0/1926987508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7384100c60 msgr2=0x7f7384104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:48.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.984+0000 7f738b09a640 1 --2- 172.21.15.94:0/1926987508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7384100c60 0x7f7384104ca0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f7374009cd0 tx=0x7f737402c0f0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.984+0000 7f738b09a640 1 -- 172.21.15.94:0/1926987508 shutdown_connections 2024-02-17T12:13:48.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.984+0000 7f738b09a640 1 --2- 172.21.15.94:0/1926987508 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7384105380 0x7f738410cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.984+0000 7f738b09a640 1 --2- 172.21.15.94:0/1926987508 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7384100c60 0x7f7384104ca0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.984+0000 7f738b09a640 1 --2- 172.21.15.94:0/1926987508 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73841002b0 0x7f7384100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.989 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.984+0000 7f738b09a640 1 -- 172.21.15.94:0/1926987508 >> 172.21.15.94:0/1926987508 conn(0x7f7384077120 msgr2=0x7f7384077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:48.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.984+0000 7f738b09a640 1 -- 172.21.15.94:0/1926987508 shutdown_connections 2024-02-17T12:13:48.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.984+0000 7f738b09a640 1 -- 172.21.15.94:0/1926987508 wait complete. 2024-02-17T12:13:48.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f738b09a640 1 Processor -- start 2024-02-17T12:13:48.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f738b09a640 1 -- start start 2024-02-17T12:13:48.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f738b09a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73841002b0 0x7f73841985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f738b09a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7384105380 0x7f7384198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f738b09a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73841990b0 0x7f73841a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f738b09a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73841a3d00 con 0x7f73841990b0 2024-02-17T12:13:48.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f738b09a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73841a3e70 con 0x7f73841002b0 2024-02-17T12:13:48.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f738b09a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73841a4030 con 0x7f7384105380 2024-02-17T12:13:48.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f7389637640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73841990b0 0x7f73841a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f7389637640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73841990b0 0x7f73841a3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57166/0 (socket says 172.21.15.94:57166) 2024-02-17T12:13:48.990 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f7389637640 1 -- 172.21.15.94:0/3299577754 learned_addr learned my addr 172.21.15.94:0/3299577754 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f7388e36640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73841002b0 0x7f73841985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f7383fff640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7384105380 0x7f7384198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f7389637640 1 -- 172.21.15.94:0/3299577754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7384105380 msgr2=0x7f7384198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f7389637640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7384105380 0x7f7384198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f7389637640 1 -- 172.21.15.94:0/3299577754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73841002b0 msgr2=0x7f73841985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f7389637640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73841002b0 0x7f73841985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.985+0000 7f7389637640 1 -- 172.21.15.94:0/3299577754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7374009900 con 0x7f73841990b0 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.986+0000 7f7389637640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73841990b0 0x7f73841a3660 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f736c00c970 tx=0x7f736c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.986+0000 7f7381ffb640 1 -- 172.21.15.94:0/3299577754 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c007bb0 con 0x7f73841990b0 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.986+0000 7f7381ffb640 1 -- 172.21.15.94:0/3299577754 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f736c007d10 con 0x7f73841990b0 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.986+0000 7f738b09a640 1 -- 172.21.15.94:0/3299577754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73841a4310 con 0x7f73841990b0 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.986+0000 7f7381ffb640 1 -- 172.21.15.94:0/3299577754 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f736c018770 con 0x7f73841990b0 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.986+0000 7f738b09a640 1 -- 172.21.15.94:0/3299577754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73841a47a0 con 0x7f73841990b0 2024-02-17T12:13:48.991 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.987+0000 7f738b09a640 1 -- 172.21.15.94:0/3299577754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73840683c0 con 0x7f73841990b0 2024-02-17T12:13:48.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.988+0000 7f7381ffb640 1 -- 172.21.15.94:0/3299577754 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f736c00f450 con 0x7f73841990b0 2024-02-17T12:13:48.992 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.988+0000 7f7381ffb640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f73700a2230 0x7f73700a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:48.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.988+0000 7f7381ffb640 1 -- 172.21.15.94:0/3299577754 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f736c0ce180 con 0x7f73841990b0 2024-02-17T12:13:48.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.988+0000 7f7388e36640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f73700a2230 0x7f73700a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:48.993 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.989+0000 7f7388e36640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f73700a2230 0x7f73700a46f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f737c00bbe0 tx=0x7f737c00b390 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:48.995 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:48.993+0000 7f7381ffb640 1 -- 172.21.15.94:0/3299577754 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f736c096dd0 con 0x7f73841990b0 2024-02-17T12:13:49.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.204+0000 7f738b09a640 1 -- 172.21.15.94:0/3299577754 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7384069de0 con 0x7f73841990b0 2024-02-17T12:13:49.207 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.205+0000 7f7381ffb640 1 -- 172.21.15.94:0/3299577754 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7f736c09a490 con 0x7f73841990b0 2024-02-17T12:13:49.207 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:49.208 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:13:49.208 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:13:49.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.207+0000 7f738b09a640 1 -- 172.21.15.94:0/3299577754 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f73700a2230 msgr2=0x7f73700a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.207+0000 7f738b09a640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f73700a2230 0x7f73700a46f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f737c00bbe0 tx=0x7f737c00b390 comp rx=0 tx=0).stop 2024-02-17T12:13:49.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.207+0000 7f738b09a640 1 -- 172.21.15.94:0/3299577754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73841990b0 msgr2=0x7f73841a3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.207+0000 7f738b09a640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73841990b0 0x7f73841a3660 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f736c00c970 tx=0x7f736c00ce30 comp rx=0 tx=0).stop 2024-02-17T12:13:49.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.207+0000 7f738b09a640 1 -- 172.21.15.94:0/3299577754 shutdown_connections 2024-02-17T12:13:49.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.207+0000 7f738b09a640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f73700a2230 0x7f73700a46f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.207+0000 7f738b09a640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f73841990b0 0x7f73841a3660 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.207+0000 7f738b09a640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7384105380 0x7f7384198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.207+0000 7f738b09a640 1 --2- 172.21.15.94:0/3299577754 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f73841002b0 0x7f73841985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.207+0000 7f738b09a640 1 -- 172.21.15.94:0/3299577754 >> 172.21.15.94:0/3299577754 conn(0x7f7384077120 msgr2=0x7f73840fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.207+0000 7f738b09a640 1 -- 172.21.15.94:0/3299577754 shutdown_connections 2024-02-17T12:13:49.210 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.207+0000 7f738b09a640 1 -- 172.21.15.94:0/3299577754 wait complete. 2024-02-17T12:13:49.221 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:13:49.221 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-17T12:13:49.328 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.323+0000 7f1191d2c640 1 -- 172.21.15.94:0/1314482301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118c06a450 msgr2=0x7f118c105e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.323+0000 7f1191d2c640 1 --2- 172.21.15.94:0/1314482301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118c06a450 0x7f118c105e00 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f1180009930 tx=0x7f118002bc30 comp rx=0 tx=0).stop 2024-02-17T12:13:49.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.323+0000 7f1191d2c640 1 -- 172.21.15.94:0/1314482301 shutdown_connections 2024-02-17T12:13:49.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.323+0000 7f1191d2c640 1 --2- 172.21.15.94:0/1314482301 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f118c1064e0 0x7f118c10e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.323+0000 7f1191d2c640 1 --2- 172.21.15.94:0/1314482301 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118c06a450 0x7f118c105e00 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.323+0000 7f1191d2c640 1 --2- 172.21.15.94:0/1314482301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f118c069aa0 0x7f118c069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.323+0000 7f1191d2c640 1 -- 172.21.15.94:0/1314482301 >> 172.21.15.94:0/1314482301 conn(0x7f118c077150 msgr2=0x7f118c077560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.329 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.323+0000 7f1191d2c640 1 -- 172.21.15.94:0/1314482301 shutdown_connections 2024-02-17T12:13:49.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.323+0000 7f1191d2c640 1 -- 172.21.15.94:0/1314482301 wait complete. 2024-02-17T12:13:49.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.323+0000 7f1191d2c640 1 Processor -- start 2024-02-17T12:13:49.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f1191d2c640 1 -- start start 2024-02-17T12:13:49.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f1191d2c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f118c069aa0 0x7f118c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f1191d2c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f118c1064e0 0x7f118c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f1191d2c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118c19c000 0x7f118c19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f1191d2c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c19c9f0 con 0x7f118c19c000 2024-02-17T12:13:49.330 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f1191d2c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c19cb60 con 0x7f118c069aa0 2024-02-17T12:13:49.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f1191d2c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f118c19ccd0 con 0x7f118c1064e0 2024-02-17T12:13:49.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f118bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118c19c000 0x7f118c19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f118bfff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118c19c000 0x7f118c19c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57184/0 (socket says 172.21.15.94:57184) 2024-02-17T12:13:49.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f118bfff640 1 -- 172.21.15.94:0/1693821368 learned_addr learned my addr 172.21.15.94:0/1693821368 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f118affd640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f118c1064e0 0x7f118c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f118b7fe640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f118c069aa0 0x7f118c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f118bfff640 1 -- 172.21.15.94:0/1693821368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f118c1064e0 msgr2=0x7f118c197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f118bfff640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f118c1064e0 0x7f118c197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.331 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f118bfff640 1 -- 172.21.15.94:0/1693821368 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f118c069aa0 msgr2=0x7f118c197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f118bfff640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f118c069aa0 0x7f118c197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f118bfff640 1 -- 172.21.15.94:0/1693821368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1180009560 con 0x7f118c19c000 2024-02-17T12:13:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f118b7fe640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f118c069aa0 0x7f118c197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:13:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.324+0000 7f118bfff640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118c19c000 0x7f118c19c4b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f118400cad0 tx=0x7f118400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.325+0000 7f1188ff9640 1 -- 172.21.15.94:0/1693821368 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118400ae20 con 0x7f118c19c000 2024-02-17T12:13:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.325+0000 7f1188ff9640 1 -- 172.21.15.94:0/1693821368 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1184005020 con 0x7f118c19c000 2024-02-17T12:13:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.325+0000 7f1191d2c640 1 -- 172.21.15.94:0/1693821368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f118c1a42b0 con 0x7f118c19c000 2024-02-17T12:13:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.325+0000 7f1188ff9640 1 -- 172.21.15.94:0/1693821368 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118400f6b0 con 0x7f118c19c000 2024-02-17T12:13:49.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.325+0000 7f1191d2c640 1 -- 172.21.15.94:0/1693821368 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c1a46e0 con 0x7f118c19c000 2024-02-17T12:13:49.342 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f1188ff9640 1 -- 172.21.15.94:0/1693821368 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f1184005520 con 0x7f118c19c000 2024-02-17T12:13:49.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f11949da740 1 Processor -- start 2024-02-17T12:13:49.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f11949da740 1 -- start start 2024-02-17T12:13:49.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f11949da740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 0x5557410b9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f11949da740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 0x5557410c2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f11949da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 0x5557410c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f11949da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555741074960 con 0x5557410c2750 2024-02-17T12:13:49.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f11949da740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555741028310 con 0x5557410b9b70 2024-02-17T12:13:49.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f11949da740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555741025840 con 0x5557410b9250 2024-02-17T12:13:49.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f1188ff9640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f11740a2230 0x7f11740a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f118affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 0x5557410c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f118affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 0x5557410c4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57198/0 (socket says 172.21.15.94:57198) 2024-02-17T12:13:49.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f118b7fe640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 0x5557410c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.343 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f118affd640 1 -- 172.21.15.94:0/3984658616 learned_addr learned my addr 172.21.15.94:0/3984658616 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.327+0000 7f118affd640 1 --2- 172.21.15.94:0/3984658616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 0x5557410b9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.328+0000 7f118bfff640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f11740a2230 0x7f11740a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.328+0000 7f1188ff9640 1 -- 172.21.15.94:0/1693821368 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f11840ce120 con 0x7f118c19c000 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.328+0000 7f118affd640 1 -- 172.21.15.94:0/3984658616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 msgr2=0x5557410b9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.328+0000 7f118affd640 1 --2- 172.21.15.94:0/3984658616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 0x5557410b9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.328+0000 7f118affd640 1 -- 172.21.15.94:0/3984658616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 msgr2=0x5557410c2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.328+0000 7f118affd640 1 --2- 172.21.15.94:0/3984658616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 0x5557410c2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.328+0000 7f118affd640 1 -- 172.21.15.94:0/3984658616 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555740fe3060 con 0x5557410c2750 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.328+0000 7f118b7fe640 1 --2- 172.21.15.94:0/3984658616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 0x5557410c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.328+0000 7f118affd640 1 --2- 172.21.15.94:0/3984658616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 0x5557410c4b40 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f118c1022b0 tx=0x7f1180005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c770ec6e721118 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.328+0000 7f118bfff640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f11740a2230 0x7f11740a46f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f11840cb8e0 tx=0x7f118406ca90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.328+0000 7f1171ffb640 1 -- 172.21.15.94:0/3984658616 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118002cad0 con 0x5557410c2750 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.328+0000 7f1171ffb640 1 -- 172.21.15.94:0/3984658616 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1180004510 con 0x5557410c2750 2024-02-17T12:13:49.344 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.328+0000 7f1171ffb640 1 -- 172.21.15.94:0/3984658616 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180004df0 con 0x5557410c2750 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.329+0000 7f11949da740 1 -- 172.21.15.94:0/3984658616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 msgr2=0x5557410c4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.329+0000 7f11949da740 1 --2- 172.21.15.94:0/3984658616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 0x5557410c4b40 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f118c1022b0 tx=0x7f1180005e50 comp rx=0 tx=0).stop 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.329+0000 7f11949da740 1 -- 172.21.15.94:0/3984658616 shutdown_connections 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.329+0000 7f11949da740 1 --2- 172.21.15.94:0/3984658616 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 0x5557410c4b40 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.329+0000 7f11949da740 1 --2- 172.21.15.94:0/3984658616 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 0x5557410c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.329+0000 7f11949da740 1 --2- 172.21.15.94:0/3984658616 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 0x5557410b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.329+0000 7f11949da740 1 -- 172.21.15.94:0/3984658616 >> 172.21.15.94:0/3984658616 conn(0x5557410b3980 msgr2=0x5557410b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.329+0000 7f11949da740 1 -- 172.21.15.94:0/3984658616 shutdown_connections 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.329+0000 7f11949da740 1 -- 172.21.15.94:0/3984658616 wait complete. 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.329+0000 7f11949da740 1 -- start start 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.329+0000 7f11949da740 1 Processor -- start 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f11949da740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 0x5557410bf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f11949da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9b70 0x5557410bfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.345 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f11949da740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410c2750 0x5557410c0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f11949da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555741021020 con 0x5557410b9b70 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f11949da740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555740cf4430 con 0x5557410c2750 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f11949da740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555740cb0e00 con 0x5557410b9250 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f11949da740 10 client.0 _ll_register_callbacks cb 0x555740fc8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f118b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9b70 0x5557410bfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f118b7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9b70 0x5557410bfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57214/0 (socket says 172.21.15.94:57214) 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f118b7fe640 1 -- 172.21.15.94:0/2745806781 learned_addr learned my addr 172.21.15.94:0/2745806781 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f118affd640 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 0x5557410bf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f118affd640 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410c2750 0x5557410c0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f118b7fe640 1 -- 172.21.15.94:0/2745806781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 msgr2=0x5557410bf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f118b7fe640 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 0x5557410bf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f118b7fe640 1 -- 172.21.15.94:0/2745806781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410c2750 msgr2=0x5557410c0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f118b7fe640 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410c2750 0x5557410c0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f118b7fe640 1 -- 172.21.15.94:0/2745806781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f118c10e8a0 con 0x5557410b9b70 2024-02-17T12:13:49.346 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f118affd640 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 0x5557410bf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f118b7fe640 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9b70 0x5557410bfcc0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f117c011a90 tx=0x7f117c011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52679486d32e5a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f116affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f116affd640 1 -- 172.21.15.94:0/2745806781 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c00fe70 con 0x5557410b9b70 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f116affd640 1 -- 172.21.15.94:0/2745806781 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f117c005020 con 0x5557410b9b70 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f11949da740 1 -- 172.21.15.94:0/2745806781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555740d207e0 con 0x5557410b9b70 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f116affd640 1 -- 172.21.15.94:0/2745806781 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f117c005220 con 0x5557410b9b70 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f11949da740 10 client.19545 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.330+0000 7f11949da740 1 -- 172.21.15.94:0/2745806781 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55574101f6a0 con 0x5557410b9b70 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.331+0000 7f116affd640 1 -- 172.21.15.94:0/2745806781 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f117c025020 con 0x5557410b9b70 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.331+0000 7f116affd640 1 -- 172.21.15.94:0/2745806781 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f117c019480 con 0x5557410b9b70 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.331+0000 7f11949da740 10 client.19545 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.331+0000 7f11949da740 10 client.19545 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.331+0000 7f11949da740 20 client.19545 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:49.347 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.331+0000 7f11949da740 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5557410d2c40 0x5557410d5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.331+0000 7f11949da740 4 client.19545 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:13:49.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.331+0000 7f11949da740 1 -- 172.21.15.94:0/2745806781 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555740d5d720 con 0x5557410d2c40 2024-02-17T12:13:49.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.331+0000 7f118affd640 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5557410d2c40 0x5557410d5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.332+0000 7f116affd640 10 client.19545 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:13:49.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.332+0000 7f118affd640 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5557410d2c40 0x5557410d5060 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=562f5c3b25e4390e server_cookie=40b7ca00c182c0d in_seq=0 out_seq=0 2024-02-17T12:13:49.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.332+0000 7f116affd640 1 -- 172.21.15.94:0/2745806781 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555740d5d720 con 0x5557410d2c40 2024-02-17T12:13:49.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.332+0000 7f116affd640 10 client.19545 handle_command_reply: tid=0 2024-02-17T12:13:49.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.332+0000 7f11949da740 1 client.19545 shutdown 2024-02-17T12:13:49.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.333+0000 7f11949da740 1 -- 172.21.15.94:0/2745806781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9b70 msgr2=0x5557410bfcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.333+0000 7f11949da740 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9b70 0x5557410bfcc0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f117c011a90 tx=0x7f117c011f50 comp rx=0 tx=0).stop 2024-02-17T12:13:49.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.333+0000 7f11949da740 1 -- 172.21.15.94:0/2745806781 shutdown_connections 2024-02-17T12:13:49.348 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.333+0000 7f11949da740 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x5557410d2c40 0x5557410d5060 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.333+0000 7f11949da740 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410c2750 0x5557410c0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.333+0000 7f11949da740 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9b70 0x5557410bfcc0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.333+0000 7f116affd640 0 client.19545 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:13:49.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.333+0000 7f11949da740 1 --2- 172.21.15.94:0/2745806781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 0x5557410bf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.333+0000 7f11949da740 1 -- 172.21.15.94:0/2745806781 >> 172.21.15.94:0/2745806781 conn(0x5557410b3980 msgr2=0x5557410b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.333+0000 7f11949da740 1 -- 172.21.15.94:0/2745806781 shutdown_connections 2024-02-17T12:13:49.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.333+0000 7f11949da740 1 -- 172.21.15.94:0/2745806781 wait complete. 2024-02-17T12:13:49.349 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.333+0000 7f11949da740 20 client.19545 trim_cache size 0 max 16384 2024-02-17T12:13:49.354 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.345+0000 7f11949da740 1 Processor -- start 2024-02-17T12:13:49.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.345+0000 7f11949da740 1 -- start start 2024-02-17T12:13:49.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f11949da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9250 0x5557410c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f11949da740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 0x5557410c0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f11949da740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410c2750 0x5557410baa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f11949da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x555740fa6130 con 0x5557410b9250 2024-02-17T12:13:49.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f11949da740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557410b8c90 con 0x5557410b9b70 2024-02-17T12:13:49.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f11949da740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557410da2a0 con 0x5557410c2750 2024-02-17T12:13:49.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f118affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9250 0x5557410c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f118affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9250 0x5557410c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57222/0 (socket says 172.21.15.94:57222) 2024-02-17T12:13:49.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f118affd640 1 -- 172.21.15.94:0/2835819861 learned_addr learned my addr 172.21.15.94:0/2835819861 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f118affd640 1 -- 172.21.15.94:0/2835819861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410c2750 msgr2=0x5557410baa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.355 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f118affd640 1 --2- 172.21.15.94:0/2835819861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410c2750 0x5557410baa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f118affd640 1 -- 172.21.15.94:0/2835819861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 msgr2=0x5557410c0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f118affd640 1 --2- 172.21.15.94:0/2835819861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 0x5557410c0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f118affd640 1 -- 172.21.15.94:0/2835819861 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555740fbbc20 con 0x5557410b9250 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f118affd640 1 --2- 172.21.15.94:0/2835819861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9250 0x5557410c1eb0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f118002cd50 tx=0x7f1180000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d8fb56b159523e server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f11717fa640 1 -- 172.21.15.94:0/2835819861 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180057d00 con 0x5557410b9250 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f11717fa640 1 -- 172.21.15.94:0/2835819861 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1180057e60 con 0x5557410b9250 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.346+0000 7f11717fa640 1 -- 172.21.15.94:0/2835819861 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180005280 con 0x5557410b9250 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.347+0000 7f11949da740 1 -- 172.21.15.94:0/2835819861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9250 msgr2=0x5557410c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.347+0000 7f11949da740 1 --2- 172.21.15.94:0/2835819861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9250 0x5557410c1eb0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f118002cd50 tx=0x7f1180000bc0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.347+0000 7f11949da740 1 -- 172.21.15.94:0/2835819861 shutdown_connections 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.347+0000 7f11949da740 1 --2- 172.21.15.94:0/2835819861 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410c2750 0x5557410baa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.347+0000 7f11949da740 1 --2- 172.21.15.94:0/2835819861 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 0x5557410c0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.347+0000 7f11949da740 1 --2- 172.21.15.94:0/2835819861 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410b9250 0x5557410c1eb0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.356 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.347+0000 7f11949da740 1 -- 172.21.15.94:0/2835819861 >> 172.21.15.94:0/2835819861 conn(0x5557410b3980 msgr2=0x5557410aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.347+0000 7f11949da740 1 -- 172.21.15.94:0/2835819861 shutdown_connections 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.347+0000 7f11949da740 1 -- 172.21.15.94:0/2835819861 wait complete. 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.347+0000 7f11949da740 1 -- start start 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.347+0000 7f11949da740 1 Processor -- start 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f11949da740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 0x5557410c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f11949da740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 0x5557410c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f11949da740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 0x5557410c8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f11949da740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557410c8c30 con 0x5557410c2750 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f11949da740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x555741129150 con 0x5557410b9b70 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f11949da740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x555741129290 con 0x5557410b9250 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f11949da740 10 client.0 _ll_register_callbacks cb 0x7f1180004780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f118affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 0x5557410c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f118affd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 0x5557410c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57236/0 (socket says 172.21.15.94:57236) 2024-02-17T12:13:49.357 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f118affd640 1 -- 172.21.15.94:0/3202248978 learned_addr learned my addr 172.21.15.94:0/3202248978 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f118affd640 1 --2- 172.21.15.94:0/3202248978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 0x5557410c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f118affd640 1 -- 172.21.15.94:0/3202248978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 msgr2=0x5557410c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f118b7fe640 1 --2- 172.21.15.94:0/3202248978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 0x5557410c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f118affd640 1 --2- 172.21.15.94:0/3202248978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 0x5557410c9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f118affd640 1 -- 172.21.15.94:0/3202248978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 msgr2=0x5557410c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f118affd640 1 --2- 172.21.15.94:0/3202248978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 0x5557410c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f118affd640 1 -- 172.21.15.94:0/3202248978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555740fe3060 con 0x5557410c2750 2024-02-17T12:13:49.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f118affd640 1 --2- 172.21.15.94:0/3202248978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 0x5557410c8560 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f118002c170 tx=0x7f1180005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7fe50d5b34406 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f116affd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:49.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f116affd640 1 -- 172.21.15.94:0/3202248978 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118006c070 con 0x5557410c2750 2024-02-17T12:13:49.358 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f11949da740 1 -- 172.21.15.94:0/3202248978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1180004510 con 0x5557410c2750 2024-02-17T12:13:49.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f116affd640 1 -- 172.21.15.94:0/3202248978 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f1180005560 con 0x5557410c2750 2024-02-17T12:13:49.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f116affd640 1 -- 172.21.15.94:0/3202248978 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1180005720 con 0x5557410c2750 2024-02-17T12:13:49.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f11949da740 10 client.19551 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:49.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.348+0000 7f11949da740 1 -- 172.21.15.94:0/3202248978 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555740fe18c0 con 0x5557410c2750 2024-02-17T12:13:49.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.349+0000 7f116affd640 1 -- 172.21.15.94:0/3202248978 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1180004510 con 0x5557410c2750 2024-02-17T12:13:49.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.349+0000 7f116affd640 1 -- 172.21.15.94:0/3202248978 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f118006f020 con 0x5557410c2750 2024-02-17T12:13:49.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.349+0000 7f11949da740 10 client.19551 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:49.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.349+0000 7f11949da740 10 client.19551 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-02-17T12:13:49.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.349+0000 7f11949da740 20 client.19551 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:49.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.349+0000 7f11949da740 1 --2- 172.21.15.94:0/3202248978 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55574112a560 0x55574112c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.359 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.349+0000 7f11949da740 4 client.19551 mds_command: new command op to 24469 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:13:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.349+0000 7f11949da740 1 -- 172.21.15.94:0/3202248978 --> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555740dac890 con 0x55574112a560 2024-02-17T12:13:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.349+0000 7f118b7fe640 1 --2- 172.21.15.94:0/3202248978 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55574112a560 0x55574112c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.349+0000 7f118b7fe640 1 --2- 172.21.15.94:0/3202248978 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55574112a560 0x55574112c950 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b04e888126a95be server_cookie=af4265622315354a in_seq=0 out_seq=0 2024-02-17T12:13:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.349+0000 7f116affd640 10 client.19551 ms_handle_connect on v2:172.21.15.189:6836/860722582 2024-02-17T12:13:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f116affd640 1 -- 172.21.15.94:0/3202248978 <== mds.2 v2:172.21.15.189:6836/860722582 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555740dac890 con 0x55574112a560 2024-02-17T12:13:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f116affd640 10 client.19551 handle_command_reply: tid=0 2024-02-17T12:13:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f11949da740 1 client.19551 shutdown 2024-02-17T12:13:49.360 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f11949da740 1 -- 172.21.15.94:0/3202248978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 msgr2=0x5557410c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f11949da740 1 --2- 172.21.15.94:0/3202248978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 0x5557410c8560 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f118002c170 tx=0x7f1180005fd0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f11949da740 1 -- 172.21.15.94:0/3202248978 shutdown_connections 2024-02-17T12:13:49.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f11949da740 1 --2- 172.21.15.94:0/3202248978 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x5557410c2750 0x5557410c8560 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f11949da740 1 --2- 172.21.15.94:0/3202248978 >> [v2:172.21.15.189:6836/860722582,v1:172.21.15.189:6837/860722582] conn(0x55574112a560 0x55574112c950 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f11949da740 1 --2- 172.21.15.94:0/3202248978 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x5557410b9b70 0x5557410c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f116affd640 0 client.19551 ms_handle_reset on v2:172.21.15.189:6836/860722582 2024-02-17T12:13:49.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f11949da740 1 --2- 172.21.15.94:0/3202248978 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5557410b9250 0x5557410c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f11949da740 1 -- 172.21.15.94:0/3202248978 >> 172.21.15.94:0/3202248978 conn(0x5557410b3980 msgr2=0x5557410baa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f11949da740 1 -- 172.21.15.94:0/3202248978 shutdown_connections 2024-02-17T12:13:49.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f11949da740 1 -- 172.21.15.94:0/3202248978 wait complete. 2024-02-17T12:13:49.361 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f11949da740 20 client.19551 trim_cache size 0 max 16384 2024-02-17T12:13:49.362 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:13:49.362 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f1191d2c640 1 -- 172.21.15.94:0/1693821368 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f11740a2230 msgr2=0x7f11740a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f1191d2c640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f11740a2230 0x7f11740a46f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f11840cb8e0 tx=0x7f118406ca90 comp rx=0 tx=0).stop 2024-02-17T12:13:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f1191d2c640 1 -- 172.21.15.94:0/1693821368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118c19c000 msgr2=0x7f118c19c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f1191d2c640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118c19c000 0x7f118c19c4b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f118400cad0 tx=0x7f118400cf90 comp rx=0 tx=0).stop 2024-02-17T12:13:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f1191d2c640 1 -- 172.21.15.94:0/1693821368 shutdown_connections 2024-02-17T12:13:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f1191d2c640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f11740a2230 0x7f11740a46f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f1191d2c640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f118c19c000 0x7f118c19c4b0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f1191d2c640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f118c1064e0 0x7f118c197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f1191d2c640 1 --2- 172.21.15.94:0/1693821368 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f118c069aa0 0x7f118c197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.350+0000 7f1191d2c640 1 -- 172.21.15.94:0/1693821368 >> 172.21.15.94:0/1693821368 conn(0x7f118c077150 msgr2=0x7f118c0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.351+0000 7f1191d2c640 1 -- 172.21.15.94:0/1693821368 shutdown_connections 2024-02-17T12:13:49.363 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.351+0000 7f1191d2c640 1 -- 172.21.15.94:0/1693821368 wait complete. 2024-02-17T12:13:49.364 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:13:49.365 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-17T12:13:49.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:49 smithi125 ceph-mon[36819]: pgmap v492: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-02-17T12:13:49.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:49 smithi094 ceph-mon[32499]: pgmap v492: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-02-17T12:13:49.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.466+0000 7f68b5a55640 1 -- 172.21.15.94:0/1927496033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0100c50 msgr2=0x7f68b0104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.466+0000 7f68b5a55640 1 --2- 172.21.15.94:0/1927496033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0100c50 0x7f68b0104c90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6898009930 tx=0x7f689802bc30 comp rx=0 tx=0).stop 2024-02-17T12:13:49.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.466+0000 7f68b5a55640 1 -- 172.21.15.94:0/1927496033 shutdown_connections 2024-02-17T12:13:49.472 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.466+0000 7f68b5a55640 1 --2- 172.21.15.94:0/1927496033 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68b0105370 0x7f68b010cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.466+0000 7f68b5a55640 1 --2- 172.21.15.94:0/1927496033 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0100c50 0x7f68b0104c90 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.466+0000 7f68b5a55640 1 --2- 172.21.15.94:0/1927496033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b01002a0 0x7f68b0100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.466+0000 7f68b5a55640 1 -- 172.21.15.94:0/1927496033 >> 172.21.15.94:0/1927496033 conn(0x7f68b0076400 msgr2=0x7f68b0076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.466+0000 7f68b5a55640 1 -- 172.21.15.94:0/1927496033 shutdown_connections 2024-02-17T12:13:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.466+0000 7f68b5a55640 1 -- 172.21.15.94:0/1927496033 wait complete. 2024-02-17T12:13:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.467+0000 7f68b5a55640 1 Processor -- start 2024-02-17T12:13:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.467+0000 7f68b5a55640 1 -- start start 2024-02-17T12:13:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.467+0000 7f68b5a55640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68b01002a0 0x7f68b019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.467+0000 7f68b5a55640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0105370 0x7f68b019db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.467+0000 7f68b5a55640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b01976b0 0x7f68b0197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.467+0000 7f68b5a55640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b0198180 con 0x7f68b0105370 2024-02-17T12:13:49.473 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.467+0000 7f68b5a55640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b01982f0 con 0x7f68b01002a0 2024-02-17T12:13:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.467+0000 7f68b5a55640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b01984b0 con 0x7f68b01976b0 2024-02-17T12:13:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.467+0000 7f68a7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0105370 0x7f68b019db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.467+0000 7f68a7fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0105370 0x7f68b019db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57250/0 (socket says 172.21.15.94:57250) 2024-02-17T12:13:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.467+0000 7f68a7fff640 1 -- 172.21.15.94:0/905723228 learned_addr learned my addr 172.21.15.94:0/905723228 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.467+0000 7f68aeffd640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68b01002a0 0x7f68b019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.468+0000 7f68af7fe640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b01976b0 0x7f68b0197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.468+0000 7f68a7fff640 1 -- 172.21.15.94:0/905723228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b01976b0 msgr2=0x7f68b0197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.468+0000 7f68a7fff640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b01976b0 0x7f68b0197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.468+0000 7f68a7fff640 1 -- 172.21.15.94:0/905723228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68b01002a0 msgr2=0x7f68b019d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.468+0000 7f68a7fff640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68b01002a0 0x7f68b019d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.468+0000 7f68a7fff640 1 -- 172.21.15.94:0/905723228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6898009560 con 0x7f68b0105370 2024-02-17T12:13:49.474 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.468+0000 7f68af7fe640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b01976b0 0x7f68b0197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:13:49.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.468+0000 7f68a7fff640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0105370 0x7f68b019db30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f6898009230 tx=0x7f6898004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.468+0000 7f68b4a53640 1 -- 172.21.15.94:0/905723228 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689802cbc0 con 0x7f68b0105370 2024-02-17T12:13:49.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.468+0000 7f68b4a53640 1 -- 172.21.15.94:0/905723228 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68980052d0 con 0x7f68b0105370 2024-02-17T12:13:49.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.468+0000 7f68b4a53640 1 -- 172.21.15.94:0/905723228 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898003dc0 con 0x7f68b0105370 2024-02-17T12:13:49.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.468+0000 7f68b5a55640 1 -- 172.21.15.94:0/905723228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b01a43f0 con 0x7f68b0105370 2024-02-17T12:13:49.475 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.468+0000 7f68b5a55640 1 -- 172.21.15.94:0/905723228 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b01a4880 con 0x7f68b0105370 2024-02-17T12:13:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68b4a53640 1 -- 172.21.15.94:0/905723228 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f68980035f0 con 0x7f68b0105370 2024-02-17T12:13:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68b862b740 1 Processor -- start 2024-02-17T12:13:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68b862b740 1 -- start start 2024-02-17T12:13:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68b862b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 0x55aaa0a4a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68b862b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a53210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68b862b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a53750 0x55aaa0a55b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68b862b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa0a05960 con 0x55aaa0a4a250 2024-02-17T12:13:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68b862b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa09b9310 con 0x55aaa0a53750 2024-02-17T12:13:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68b862b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa09b6840 con 0x55aaa0a4ab70 2024-02-17T12:13:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 0x55aaa0a4a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.482 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 0x55aaa0a4a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57252/0 (socket says 172.21.15.94:57252) 2024-02-17T12:13:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68af7fe640 1 -- 172.21.15.94:0/2359736465 learned_addr learned my addr 172.21.15.94:0/2359736465 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68a7fff640 1 --2- 172.21.15.94:0/2359736465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a53750 0x55aaa0a55b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68b4a53640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f689c0a2230 0x7f689c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.471+0000 7f68aeffd640 1 --2- 172.21.15.94:0/2359736465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a53210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.472+0000 7f68af7fe640 1 -- 172.21.15.94:0/2359736465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4ab70 msgr2=0x55aaa0a53210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.472+0000 7f68af7fe640 1 --2- 172.21.15.94:0/2359736465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a53210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.472+0000 7f68af7fe640 1 -- 172.21.15.94:0/2359736465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a53750 msgr2=0x55aaa0a55b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.472+0000 7f68af7fe640 1 --2- 172.21.15.94:0/2359736465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a53750 0x55aaa0a55b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.472+0000 7f68af7fe640 1 -- 172.21.15.94:0/2359736465 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaa0974060 con 0x55aaa0a4a250 2024-02-17T12:13:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.472+0000 7f68a7fff640 1 --2- 172.21.15.94:0/2359736465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a53750 0x55aaa0a55b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:13:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.472+0000 7f68af7fe640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f689c0a2230 0x7f689c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.483 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.472+0000 7f68b4a53640 1 -- 172.21.15.94:0/905723228 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f68980f0410 con 0x7f68b0105370 2024-02-17T12:13:49.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.472+0000 7f68af7fe640 1 --2- 172.21.15.94:0/2359736465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 0x55aaa0a4a630 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f68a800e3a0 tx=0x7f68a800e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d9f05d4effb7e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.472+0000 7f68a5ffb640 1 -- 172.21.15.94:0/2359736465 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a80129f0 con 0x55aaa0a4a250 2024-02-17T12:13:49.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.472+0000 7f68a5ffb640 1 -- 172.21.15.94:0/2359736465 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68a80099c0 con 0x55aaa0a4a250 2024-02-17T12:13:49.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.472+0000 7f68a5ffb640 1 -- 172.21.15.94:0/2359736465 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8019dd0 con 0x55aaa0a4a250 2024-02-17T12:13:49.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68b862b740 1 -- 172.21.15.94:0/2359736465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 msgr2=0x55aaa0a4a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68af7fe640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f689c0a2230 0x7f689c0a46f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f68a801d000 tx=0x7f68a801e000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68b862b740 1 --2- 172.21.15.94:0/2359736465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 0x55aaa0a4a630 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f68a800e3a0 tx=0x7f68a800e860 comp rx=0 tx=0).stop 2024-02-17T12:13:49.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68b862b740 1 -- 172.21.15.94:0/2359736465 shutdown_connections 2024-02-17T12:13:49.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68b862b740 1 --2- 172.21.15.94:0/2359736465 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a53750 0x55aaa0a55b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68b862b740 1 --2- 172.21.15.94:0/2359736465 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a53210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68b862b740 1 --2- 172.21.15.94:0/2359736465 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 0x55aaa0a4a630 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.484 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68b862b740 1 -- 172.21.15.94:0/2359736465 >> 172.21.15.94:0/2359736465 conn(0x55aaa0a44980 msgr2=0x55aaa0a44d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68b862b740 1 -- 172.21.15.94:0/2359736465 shutdown_connections 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68b862b740 1 -- 172.21.15.94:0/2359736465 wait complete. 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68b862b740 1 -- start start 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68b862b740 1 Processor -- start 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68b862b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 0x55aaa0a50780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.473+0000 7f68b862b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a50cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68b862b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a53750 0x55aaa0a51200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68b862b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa09b2020 con 0x55aaa0a4a250 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68b862b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa0685430 con 0x55aaa0a53750 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68b862b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa0641e00 con 0x55aaa0a4ab70 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68b862b740 10 client.0 _ll_register_callbacks cb 0x55aaa0959cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68aeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 0x55aaa0a50780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68aeffd640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 0x55aaa0a50780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57268/0 (socket says 172.21.15.94:57268) 2024-02-17T12:13:49.485 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68aeffd640 1 -- 172.21.15.94:0/3050464238 learned_addr learned my addr 172.21.15.94:0/3050464238 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68a7fff640 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a50cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68aeffd640 1 -- 172.21.15.94:0/3050464238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4ab70 msgr2=0x55aaa0a50cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68aeffd640 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a50cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68af7fe640 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a53750 0x55aaa0a51200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68aeffd640 1 -- 172.21.15.94:0/3050464238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a53750 msgr2=0x55aaa0a51200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68aeffd640 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a53750 0x55aaa0a51200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68aeffd640 1 -- 172.21.15.94:0/3050464238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a8009b90 con 0x55aaa0a4a250 2024-02-17T12:13:49.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68a7fff640 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a50cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:13:49.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68aeffd640 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 0x55aaa0a50780 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009f10 tx=0x7f68a0011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ade16cd49df550 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f688effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:49.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f688effd640 1 -- 172.21.15.94:0/3050464238 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a0015870 con 0x55aaa0a4a250 2024-02-17T12:13:49.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f688effd640 1 -- 172.21.15.94:0/3050464238 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f68a0011e60 con 0x55aaa0a4a250 2024-02-17T12:13:49.486 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68b862b740 1 -- 172.21.15.94:0/3050464238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aaa06ee720 con 0x55aaa0a4a250 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f688effd640 1 -- 172.21.15.94:0/3050464238 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68a00159d0 con 0x55aaa0a4a250 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68b862b740 10 client.19563 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f68b862b740 1 -- 172.21.15.94:0/3050464238 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aaa09b06a0 con 0x55aaa0a4a250 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.474+0000 7f688effd640 1 -- 172.21.15.94:0/3050464238 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aaa06ee720 con 0x55aaa0a4a250 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.475+0000 7f688effd640 1 -- 172.21.15.94:0/3050464238 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f68a001d3b0 con 0x55aaa0a4a250 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.475+0000 7f68b862b740 10 client.19563 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.475+0000 7f68b862b740 10 client.19563 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.475+0000 7f68b862b740 20 client.19563 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.475+0000 7f68b862b740 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55aaa0a63ba0 0x55aaa0a65fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.475+0000 7f68b862b740 4 client.19563 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.475+0000 7f68b862b740 1 -- 172.21.15.94:0/3050464238 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aaa073d890 con 0x55aaa0a63ba0 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.475+0000 7f68a7fff640 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55aaa0a63ba0 0x55aaa0a65fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.475+0000 7f688effd640 10 client.19563 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:13:49.487 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.475+0000 7f68a7fff640 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55aaa0a63ba0 0x55aaa0a65fe0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4cac6b3c64b84a3c server_cookie=8f327359b59e82cc in_seq=0 out_seq=0 2024-02-17T12:13:49.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f688effd640 1 -- 172.21.15.94:0/3050464238 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aaa073d890 con 0x55aaa0a63ba0 2024-02-17T12:13:49.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f688effd640 10 client.19563 handle_command_reply: tid=0 2024-02-17T12:13:49.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f68b862b740 1 client.19563 shutdown 2024-02-17T12:13:49.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f68b862b740 1 -- 172.21.15.94:0/3050464238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 msgr2=0x55aaa0a50780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f68b862b740 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 0x55aaa0a50780 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009f10 tx=0x7f68a0011630 comp rx=0 tx=0).stop 2024-02-17T12:13:49.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f68b862b740 1 -- 172.21.15.94:0/3050464238 shutdown_connections 2024-02-17T12:13:49.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f68b862b740 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55aaa0a63ba0 0x55aaa0a65fe0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f68b862b740 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a53750 0x55aaa0a51200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f688effd640 0 client.19563 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:13:49.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f68b862b740 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a50cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f68b862b740 1 --2- 172.21.15.94:0/3050464238 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a4a250 0x55aaa0a50780 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.488 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f68b862b740 1 -- 172.21.15.94:0/3050464238 >> 172.21.15.94:0/3050464238 conn(0x55aaa0a44980 msgr2=0x55aaa0a466d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f68b862b740 1 -- 172.21.15.94:0/3050464238 shutdown_connections 2024-02-17T12:13:49.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f68b862b740 1 -- 172.21.15.94:0/3050464238 wait complete. 2024-02-17T12:13:49.489 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.476+0000 7f68b862b740 20 client.19563 trim_cache size 0 max 16384 2024-02-17T12:13:49.496 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68b862b740 1 Processor -- start 2024-02-17T12:13:49.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68b862b740 1 -- start start 2024-02-17T12:13:49.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68b862b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4a250 0x55aaa0a52eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68b862b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a51020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68b862b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 0x55aaa0a4d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68b862b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa0937130 con 0x55aaa0a53750 2024-02-17T12:13:49.497 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68b862b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa0a36a40 con 0x55aaa0a4ab70 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68b862b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa0a49ce0 con 0x55aaa0a4a250 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 0x55aaa0a4d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 0x55aaa0a4d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57280/0 (socket says 172.21.15.94:57280) 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68af7fe640 1 -- 172.21.15.94:0/4052743762 learned_addr learned my addr 172.21.15.94:0/4052743762 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68aeffd640 1 --2- 172.21.15.94:0/4052743762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4a250 0x55aaa0a52eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68a7fff640 1 --2- 172.21.15.94:0/4052743762 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a51020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68af7fe640 1 -- 172.21.15.94:0/4052743762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4a250 msgr2=0x55aaa0a52eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68af7fe640 1 --2- 172.21.15.94:0/4052743762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4a250 0x55aaa0a52eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68af7fe640 1 -- 172.21.15.94:0/4052743762 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a4ab70 msgr2=0x55aaa0a51020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68af7fe640 1 --2- 172.21.15.94:0/4052743762 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a51020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.488+0000 7f68af7fe640 1 -- 172.21.15.94:0/4052743762 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaa094cc20 con 0x55aaa0a53750 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.489+0000 7f68af7fe640 1 --2- 172.21.15.94:0/4052743762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 0x55aaa0a4d9e0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f68a8013fd0 tx=0x7f68a8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9152164b03fd61 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.489+0000 7f68a57fa640 1 -- 172.21.15.94:0/4052743762 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a80129f0 con 0x55aaa0a53750 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.489+0000 7f68a57fa640 1 -- 172.21.15.94:0/4052743762 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f68a802a040 con 0x55aaa0a53750 2024-02-17T12:13:49.498 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.489+0000 7f68a57fa640 1 -- 172.21.15.94:0/4052743762 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8024e00 con 0x55aaa0a53750 2024-02-17T12:13:49.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.489+0000 7f68b862b740 1 -- 172.21.15.94:0/4052743762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 msgr2=0x55aaa0a4d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.489+0000 7f68b862b740 1 --2- 172.21.15.94:0/4052743762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 0x55aaa0a4d9e0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f68a8013fd0 tx=0x7f68a8011000 comp rx=0 tx=0).stop 2024-02-17T12:13:49.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.489+0000 7f68b862b740 1 -- 172.21.15.94:0/4052743762 shutdown_connections 2024-02-17T12:13:49.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.489+0000 7f68b862b740 1 --2- 172.21.15.94:0/4052743762 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 0x55aaa0a4d9e0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.489+0000 7f68b862b740 1 --2- 172.21.15.94:0/4052743762 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a51020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.489+0000 7f68b862b740 1 --2- 172.21.15.94:0/4052743762 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4a250 0x55aaa0a52eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.489+0000 7f68b862b740 1 -- 172.21.15.94:0/4052743762 >> 172.21.15.94:0/4052743762 conn(0x55aaa0a44980 msgr2=0x55aaa0a4f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.489+0000 7f68b862b740 1 -- 172.21.15.94:0/4052743762 shutdown_connections 2024-02-17T12:13:49.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.489+0000 7f68b862b740 1 -- 172.21.15.94:0/4052743762 wait complete. 2024-02-17T12:13:49.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68b862b740 1 -- start start 2024-02-17T12:13:49.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68b862b740 1 Processor -- start 2024-02-17T12:13:49.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68b862b740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4a250 0x55aaa0a593b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.499 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68b862b740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68b862b740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 0x55aaa0aba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68b862b740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa0a5a1c0 con 0x55aaa0a53750 2024-02-17T12:13:49.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68b862b740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa0aba650 con 0x55aaa0a4ab70 2024-02-17T12:13:49.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68b862b740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaa0aba790 con 0x55aaa0a4a250 2024-02-17T12:13:49.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68b862b740 10 client.0 _ll_register_callbacks cb 0x7f68a80096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:49.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 0x55aaa0aba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 0x55aaa0aba110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57286/0 (socket says 172.21.15.94:57286) 2024-02-17T12:13:49.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68af7fe640 1 -- 172.21.15.94:0/80530156 learned_addr learned my addr 172.21.15.94:0/80530156 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68aeffd640 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4a250 0x55aaa0a593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68a7fff640 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68af7fe640 1 -- 172.21.15.94:0/80530156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4a250 msgr2=0x55aaa0a593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.500 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68af7fe640 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4a250 0x55aaa0a593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68af7fe640 1 -- 172.21.15.94:0/80530156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a4ab70 msgr2=0x55aaa0a598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68af7fe640 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68af7fe640 1 -- 172.21.15.94:0/80530156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aaa0974060 con 0x55aaa0a53750 2024-02-17T12:13:49.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.490+0000 7f68aeffd640 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4a250 0x55aaa0a593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:13:49.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f68af7fe640 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 0x55aaa0aba110 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f68a8006010 tx=0x7f68a8024e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd01235ad7a068a server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f688effd640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:49.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f688effd640 1 -- 172.21.15.94:0/80530156 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8009860 con 0x55aaa0a53750 2024-02-17T12:13:49.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f688effd640 1 -- 172.21.15.94:0/80530156 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f68a8016030 con 0x55aaa0a53750 2024-02-17T12:13:49.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f68b862b740 1 -- 172.21.15.94:0/80530156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aaa06b17e0 con 0x55aaa0a53750 2024-02-17T12:13:49.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f688effd640 1 -- 172.21.15.94:0/80530156 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68a8005b70 con 0x55aaa0a53750 2024-02-17T12:13:49.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f68b862b740 10 client.19569 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:49.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f68b862b740 1 -- 172.21.15.94:0/80530156 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55aaa09728c0 con 0x55aaa0a53750 2024-02-17T12:13:49.501 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f688effd640 1 -- 172.21.15.94:0/80530156 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55aaa06b17e0 con 0x55aaa0a53750 2024-02-17T12:13:49.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f688effd640 1 -- 172.21.15.94:0/80530156 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f68a8035070 con 0x55aaa0a53750 2024-02-17T12:13:49.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f68b862b740 10 client.19569 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:49.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f68b862b740 10 client.19569 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-17T12:13:49.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f68b862b740 20 client.19569 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:49.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f68b862b740 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55aaa0abbb70 0x55aaa0abdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f68b862b740 4 client.19569 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:13:49.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.491+0000 7f68b862b740 1 -- 172.21.15.94:0/80530156 --> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aaa065d700 con 0x55aaa0abbb70 2024-02-17T12:13:49.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f68aeffd640 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55aaa0abbb70 0x55aaa0abdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f688effd640 10 client.19569 ms_handle_connect on v2:172.21.15.189:6838/308580523 2024-02-17T12:13:49.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f68aeffd640 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55aaa0abbb70 0x55aaa0abdf60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f63389c78a726a5 server_cookie=4af497d854690191 in_seq=0 out_seq=0 2024-02-17T12:13:49.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f688effd640 1 -- 172.21.15.94:0/80530156 <== mds.1 v2:172.21.15.189:6838/308580523 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aaa065d700 con 0x55aaa0abbb70 2024-02-17T12:13:49.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f688effd640 10 client.19569 handle_command_reply: tid=0 2024-02-17T12:13:49.502 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f68b862b740 1 client.19569 shutdown 2024-02-17T12:13:49.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f68b862b740 1 -- 172.21.15.94:0/80530156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 msgr2=0x55aaa0aba110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f68b862b740 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 0x55aaa0aba110 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f68a8006010 tx=0x7f68a8024e10 comp rx=0 tx=0).stop 2024-02-17T12:13:49.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f68b862b740 1 -- 172.21.15.94:0/80530156 shutdown_connections 2024-02-17T12:13:49.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f68b862b740 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.189:6838/308580523,v1:172.21.15.189:6839/308580523] conn(0x55aaa0abbb70 0x55aaa0abdf60 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f68b862b740 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55aaa0a53750 0x55aaa0aba110 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f68b862b740 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55aaa0a4ab70 0x55aaa0a598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f68b862b740 1 --2- 172.21.15.94:0/80530156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aaa0a4a250 0x55aaa0a593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f688effd640 0 client.19569 ms_handle_reset on v2:172.21.15.189:6838/308580523 2024-02-17T12:13:49.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f68b862b740 1 -- 172.21.15.94:0/80530156 >> 172.21.15.94:0/80530156 conn(0x55aaa0a44980 msgr2=0x55aaa0a43b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.492+0000 7f68b862b740 1 -- 172.21.15.94:0/80530156 shutdown_connections 2024-02-17T12:13:49.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b862b740 1 -- 172.21.15.94:0/80530156 wait complete. 2024-02-17T12:13:49.503 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b862b740 20 client.19569 trim_cache size 0 max 16384 2024-02-17T12:13:49.504 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:13:49.504 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b5a55640 1 -- 172.21.15.94:0/905723228 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f689c0a2230 msgr2=0x7f689c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b5a55640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f689c0a2230 0x7f689c0a46f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f68a801d000 tx=0x7f68a801e000 comp rx=0 tx=0).stop 2024-02-17T12:13:49.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b5a55640 1 -- 172.21.15.94:0/905723228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0105370 msgr2=0x7f68b019db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b5a55640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0105370 0x7f68b019db30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f6898009230 tx=0x7f6898004270 comp rx=0 tx=0).stop 2024-02-17T12:13:49.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b5a55640 1 -- 172.21.15.94:0/905723228 shutdown_connections 2024-02-17T12:13:49.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b5a55640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f689c0a2230 0x7f689c0a46f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b5a55640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b01976b0 0x7f68b0197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b5a55640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f68b0105370 0x7f68b019db30 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b5a55640 1 --2- 172.21.15.94:0/905723228 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f68b01002a0 0x7f68b019d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b5a55640 1 -- 172.21.15.94:0/905723228 >> 172.21.15.94:0/905723228 conn(0x7f68b0076400 msgr2=0x7f68b00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b5a55640 1 -- 172.21.15.94:0/905723228 shutdown_connections 2024-02-17T12:13:49.505 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.493+0000 7f68b5a55640 1 -- 172.21.15.94:0/905723228 wait complete. 2024-02-17T12:13:49.506 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-02-17T12:13:49.506 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-17T12:13:49.612 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.607+0000 7f5cbcc83640 1 -- 172.21.15.94:0/2958158084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb81023c0 msgr2=0x7f5cb81027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.607+0000 7f5cbcc83640 1 --2- 172.21.15.94:0/2958158084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb81023c0 0x7f5cb81027a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009900 tx=0x7f5ca802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.607+0000 7f5cbcc83640 1 -- 172.21.15.94:0/2958158084 shutdown_connections 2024-02-17T12:13:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.607+0000 7f5cbcc83640 1 --2- 172.21.15.94:0/2958158084 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5cb810b790 0x7f5cb810dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.607+0000 7f5cbcc83640 1 --2- 172.21.15.94:0/2958158084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cb8102ce0 0x7f5cb810b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.607+0000 7f5cbcc83640 1 --2- 172.21.15.94:0/2958158084 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb81023c0 0x7f5cb81027a0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.607+0000 7f5cbcc83640 1 -- 172.21.15.94:0/2958158084 >> 172.21.15.94:0/2958158084 conn(0x7f5cb8076400 msgr2=0x7f5cb8076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.607+0000 7f5cbcc83640 1 -- 172.21.15.94:0/2958158084 shutdown_connections 2024-02-17T12:13:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.607+0000 7f5cbcc83640 1 -- 172.21.15.94:0/2958158084 wait complete. 2024-02-17T12:13:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.608+0000 7f5cbcc83640 1 Processor -- start 2024-02-17T12:13:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.608+0000 7f5cbcc83640 1 -- start start 2024-02-17T12:13:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.608+0000 7f5cbcc83640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5cb81023c0 0x7f5cb819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.608+0000 7f5cbcc83640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb8102ce0 0x7f5cb819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.613 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.608+0000 7f5cbcc83640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cb810b790 0x7f5cb8197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.608+0000 7f5cbcc83640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb8197c70 con 0x7f5cb8102ce0 2024-02-17T12:13:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.608+0000 7f5cbcc83640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb8197de0 con 0x7f5cb81023c0 2024-02-17T12:13:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.608+0000 7f5cbcc83640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb8197fa0 con 0x7f5cb810b790 2024-02-17T12:13:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.608+0000 7f5cb5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb8102ce0 0x7f5cb819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5cb5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb8102ce0 0x7f5cb819d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57292/0 (socket says 172.21.15.94:57292) 2024-02-17T12:13:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5cb5d9b640 1 -- 172.21.15.94:0/1831811335 learned_addr learned my addr 172.21.15.94:0/1831811335 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5cb659c640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5cb81023c0 0x7f5cb819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5cb6d9d640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cb810b790 0x7f5cb8197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5cb5d9b640 1 -- 172.21.15.94:0/1831811335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cb810b790 msgr2=0x7f5cb8197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cb810b790 0x7f5cb8197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5cb5d9b640 1 -- 172.21.15.94:0/1831811335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5cb81023c0 msgr2=0x7f5cb819d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5cb81023c0 0x7f5cb819d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.614 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5cb5d9b640 1 -- 172.21.15.94:0/1831811335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca8009560 con 0x7f5cb8102ce0 2024-02-17T12:13:49.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5cb6d9d640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cb810b790 0x7f5cb8197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:13:49.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb8102ce0 0x7f5cb819d9b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00b4a0 tx=0x7f5cac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5ca77fe640 1 -- 172.21.15.94:0/1831811335 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac010610 con 0x7f5cb8102ce0 2024-02-17T12:13:49.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5cbcc83640 1 -- 172.21.15.94:0/1831811335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb8198280 con 0x7f5cb8102ce0 2024-02-17T12:13:49.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5ca77fe640 1 -- 172.21.15.94:0/1831811335 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cac010c50 con 0x7f5cb8102ce0 2024-02-17T12:13:49.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5ca77fe640 1 -- 172.21.15.94:0/1831811335 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac017e20 con 0x7f5cb8102ce0 2024-02-17T12:13:49.615 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.609+0000 7f5cbcc83640 1 -- 172.21.15.94:0/1831811335 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb81a4510 con 0x7f5cb8102ce0 2024-02-17T12:13:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.611+0000 7f5ca77fe640 1 -- 172.21.15.94:0/1831811335 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f5cac017480 con 0x7f5cb8102ce0 2024-02-17T12:13:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cbf84c740 1 Processor -- start 2024-02-17T12:13:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cbf84c740 1 -- start start 2024-02-17T12:13:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cbf84c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932edc4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cbf84c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edc4b70 0x55932edcd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.623 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cbf84c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edcd750 0x55932edcfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cbf84c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932ed7f960 con 0x55932edc4250 2024-02-17T12:13:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cbf84c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932ed33310 con 0x55932edcd750 2024-02-17T12:13:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cbf84c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932ed30840 con 0x55932edc4b70 2024-02-17T12:13:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5ca77fe640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5ca00a2230 0x7f5ca00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cb6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932edc4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cb6d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932edc4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57308/0 (socket says 172.21.15.94:57308) 2024-02-17T12:13:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cb6d9d640 1 -- 172.21.15.94:0/817016266 learned_addr learned my addr 172.21.15.94:0/817016266 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cb6d9d640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5ca00a2230 0x7f5ca00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cb659c640 1 --2- 172.21.15.94:0/817016266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edc4b70 0x55932edcd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5ca77fe640 1 -- 172.21.15.94:0/1831811335 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5cac0cd650 con 0x7f5cb8102ce0 2024-02-17T12:13:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.612+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/817016266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edcd750 0x55932edcfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.624 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.613+0000 7f5cb6d9d640 1 -- 172.21.15.94:0/817016266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edc4b70 msgr2=0x55932edcd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.613+0000 7f5cb6d9d640 1 --2- 172.21.15.94:0/817016266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edc4b70 0x55932edcd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.613+0000 7f5cb6d9d640 1 -- 172.21.15.94:0/817016266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edcd750 msgr2=0x55932edcfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.613+0000 7f5cb6d9d640 1 --2- 172.21.15.94:0/817016266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edcd750 0x55932edcfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.613+0000 7f5cb6d9d640 1 -- 172.21.15.94:0/817016266 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55932ecee060 con 0x55932edc4250 2024-02-17T12:13:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.613+0000 7f5cb6d9d640 1 --2- 172.21.15.94:0/817016266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932edc4630 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0015f30 tx=0x7f5cb0012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5cb4a2f73d69fa server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.613+0000 7f5cb6d9d640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5ca00a2230 0x7f5ca00a46f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0012d20 tx=0x7f5cb001d000 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.613+0000 7f5ca4ff9640 1 -- 172.21.15.94:0/817016266 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb000cce0 con 0x55932edc4250 2024-02-17T12:13:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.613+0000 7f5ca4ff9640 1 -- 172.21.15.94:0/817016266 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cb001b040 con 0x55932edc4250 2024-02-17T12:13:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.613+0000 7f5ca4ff9640 1 -- 172.21.15.94:0/817016266 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0010660 con 0x55932edc4250 2024-02-17T12:13:49.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 -- 172.21.15.94:0/817016266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 msgr2=0x55932edc4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/817016266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932edc4630 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0015f30 tx=0x7f5cb0012820 comp rx=0 tx=0).stop 2024-02-17T12:13:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 -- 172.21.15.94:0/817016266 shutdown_connections 2024-02-17T12:13:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/817016266 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edcd750 0x55932edcfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/817016266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edc4b70 0x55932edcd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/817016266 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932edc4630 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 -- 172.21.15.94:0/817016266 >> 172.21.15.94:0/817016266 conn(0x55932edbe980 msgr2=0x55932edbed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 -- 172.21.15.94:0/817016266 shutdown_connections 2024-02-17T12:13:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 -- 172.21.15.94:0/817016266 wait complete. 2024-02-17T12:13:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 -- start start 2024-02-17T12:13:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 Processor -- start 2024-02-17T12:13:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932edca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4b70 0x55932edcad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 0x55932edcb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932ed2c020 con 0x55932edc4250 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e9ff430 con 0x55932edc4b70 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932e9bbe00 con 0x55932edcd750 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.614+0000 7f5cbf84c740 10 client.0 _ll_register_callbacks cb 0x55932ecd3cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cb659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932edca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cb659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932edca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57310/0 (socket says 172.21.15.94:57310) 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cb659c640 1 -- 172.21.15.94:0/2477997816 learned_addr learned my addr 172.21.15.94:0/2477997816 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/2477997816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4b70 0x55932edcad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cb659c640 1 -- 172.21.15.94:0/2477997816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 msgr2=0x55932edcb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cb659c640 1 --2- 172.21.15.94:0/2477997816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 0x55932edcb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cb659c640 1 -- 172.21.15.94:0/2477997816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4b70 msgr2=0x55932edcad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cb659c640 1 --2- 172.21.15.94:0/2477997816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4b70 0x55932edcad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cb659c640 1 -- 172.21.15.94:0/2477997816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb0015b10 con 0x55932edc4250 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/2477997816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4b70 0x55932edcad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cb659c640 1 --2- 172.21.15.94:0/2477997816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932edca7d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009900 tx=0x7f5ca802cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a156ef661fb47c server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5c9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:49.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5c9dffb640 1 -- 172.21.15.94:0/2477997816 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8004310 con 0x55932edc4250 2024-02-17T12:13:49.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5c9dffb640 1 -- 172.21.15.94:0/2477997816 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5ca8046070 con 0x55932edc4250 2024-02-17T12:13:49.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cbf84c740 1 -- 172.21.15.94:0/2477997816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55932ea2b7e0 con 0x55932edc4250 2024-02-17T12:13:49.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5c9dffb640 1 -- 172.21.15.94:0/2477997816 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ca8004470 con 0x55932edc4250 2024-02-17T12:13:49.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cbf84c740 10 client.19581 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:49.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5cbf84c740 1 -- 172.21.15.94:0/2477997816 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55932ed2a6a0 con 0x55932edc4250 2024-02-17T12:13:49.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.615+0000 7f5c9dffb640 1 -- 172.21.15.94:0/2477997816 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x55932ea2b7e0 con 0x55932edc4250 2024-02-17T12:13:49.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.616+0000 7f5c9dffb640 1 -- 172.21.15.94:0/2477997816 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5ca8051070 con 0x55932edc4250 2024-02-17T12:13:49.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.616+0000 7f5cbf84c740 10 client.19581 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:49.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.616+0000 7f5cbf84c740 10 client.19581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:49.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.616+0000 7f5cbf84c740 20 client.19581 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:49.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.616+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/2477997816 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55932edddb90 0x55932eddffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.616+0000 7f5cbf84c740 4 client.19581 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.616+0000 7f5cbf84c740 1 -- 172.21.15.94:0/2477997816 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55932ea68720 con 0x55932edddb90 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.616+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/2477997816 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55932edddb90 0x55932eddffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.616+0000 7f5c9dffb640 10 client.19581 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.616+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/2477997816 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55932edddb90 0x55932eddffb0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33d3796c32459fa server_cookie=5313f90fbc3a8a71 in_seq=0 out_seq=0 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5c9dffb640 1 -- 172.21.15.94:0/2477997816 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55932ea68720 con 0x55932edddb90 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5c9dffb640 10 client.19581 handle_command_reply: tid=0 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5cbf84c740 1 client.19581 shutdown 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5cbf84c740 1 -- 172.21.15.94:0/2477997816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 msgr2=0x55932edca7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/2477997816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932edca7d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009900 tx=0x7f5ca802cc80 comp rx=0 tx=0).stop 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5cbf84c740 1 -- 172.21.15.94:0/2477997816 shutdown_connections 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/2477997816 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55932edddb90 0x55932eddffb0 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/2477997816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 0x55932edcb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.629 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/2477997816 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4b70 0x55932edcad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/2477997816 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932edca7d0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5c9dffb640 0 client.19581 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:49.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5cbf84c740 1 -- 172.21.15.94:0/2477997816 >> 172.21.15.94:0/2477997816 conn(0x55932edbe980 msgr2=0x55932edc06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5cbf84c740 1 -- 172.21.15.94:0/2477997816 shutdown_connections 2024-02-17T12:13:49.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5cbf84c740 1 -- 172.21.15.94:0/2477997816 wait complete. 2024-02-17T12:13:49.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.617+0000 7f5cbf84c740 20 client.19581 trim_cache size 0 max 16384 2024-02-17T12:13:49.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cbf84c740 1 Processor -- start 2024-02-17T12:13:49.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cbf84c740 1 -- start start 2024-02-17T12:13:49.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cbf84c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4250 0x55932edccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cbf84c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4b70 0x55932edd4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cbf84c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 0x55932edcb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cbf84c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932ecb1130 con 0x55932edc4b70 2024-02-17T12:13:49.637 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cbf84c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932ea2b7e0 con 0x55932edc4250 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cbf84c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932edc49d0 con 0x55932edcd750 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cb5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4b70 0x55932edd4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cb5d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4b70 0x55932edd4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57312/0 (socket says 172.21.15.94:57312) 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cb5d9b640 1 -- 172.21.15.94:0/3438583983 learned_addr learned my addr 172.21.15.94:0/3438583983 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cb659c640 1 --2- 172.21.15.94:0/3438583983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4250 0x55932edccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cb6d9d640 1 --2- 172.21.15.94:0/3438583983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 0x55932edcb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cb5d9b640 1 -- 172.21.15.94:0/3438583983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 msgr2=0x55932edcb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/3438583983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 0x55932edcb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cb5d9b640 1 -- 172.21.15.94:0/3438583983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4250 msgr2=0x55932edccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/3438583983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4250 0x55932edccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cb5d9b640 1 -- 172.21.15.94:0/3438583983 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55932ecc6c20 con 0x55932edc4b70 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.629+0000 7f5cb6d9d640 1 --2- 172.21.15.94:0/3438583983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 0x55932edcb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/3438583983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4b70 0x55932edd4980 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5cac099e10 tx=0x7f5cac0963f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb699481338705f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5c9ffff640 1 -- 172.21.15.94:0/3438583983 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0981b0 con 0x55932edc4b70 2024-02-17T12:13:49.638 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5c9ffff640 1 -- 172.21.15.94:0/3438583983 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cac098410 con 0x55932edc4b70 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5c9ffff640 1 -- 172.21.15.94:0/3438583983 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0997c0 con 0x55932edc4b70 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5cbf84c740 1 -- 172.21.15.94:0/3438583983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4b70 msgr2=0x55932edd4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/3438583983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4b70 0x55932edd4980 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f5cac099e10 tx=0x7f5cac0963f0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5cbf84c740 1 -- 172.21.15.94:0/3438583983 shutdown_connections 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/3438583983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 0x55932edcb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/3438583983 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4b70 0x55932edd4980 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/3438583983 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4250 0x55932edccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5cbf84c740 1 -- 172.21.15.94:0/3438583983 >> 172.21.15.94:0/3438583983 conn(0x55932edbe980 msgr2=0x55932ede4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5cbf84c740 1 -- 172.21.15.94:0/3438583983 shutdown_connections 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5cbf84c740 1 -- 172.21.15.94:0/3438583983 wait complete. 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.630+0000 7f5cbf84c740 1 -- start start 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cbf84c740 1 Processor -- start 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cbf84c740 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932ee32290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cbf84c740 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4b70 0x55932ee327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.639 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cbf84c740 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 0x55932ee2c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cbf84c740 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932ee2cc60 con 0x55932edc4250 2024-02-17T12:13:49.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cbf84c740 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932ee2cdd0 con 0x55932edc4b70 2024-02-17T12:13:49.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cbf84c740 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x55932ee2cf90 con 0x55932edcd750 2024-02-17T12:13:49.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cbf84c740 10 client.0 _ll_register_callbacks cb 0x7f5cb0015420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-17T12:13:49.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cb659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932ee32290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cb659c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932ee32290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57328/0 (socket says 172.21.15.94:57328) 2024-02-17T12:13:49.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cb659c640 1 -- 172.21.15.94:0/696972438 learned_addr learned my addr 172.21.15.94:0/696972438 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:49.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4b70 0x55932ee327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.640 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cb6d9d640 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 0x55932ee2c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cb659c640 1 -- 172.21.15.94:0/696972438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 msgr2=0x55932ee2c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cb659c640 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 0x55932ee2c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cb659c640 1 -- 172.21.15.94:0/696972438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4b70 msgr2=0x55932ee327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-17T12:13:49.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cb659c640 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4b70 0x55932ee327d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cb659c640 1 -- 172.21.15.94:0/696972438 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb81a4510 con 0x55932edc4250 2024-02-17T12:13:49.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cb6d9d640 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 0x55932ee2c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:13:49.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4b70 0x55932ee327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:13:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cb659c640 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932ee32290 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8003820 tx=0x7f5ca8003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3931fe399d8be8bc server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5c9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.94:3300/0 2024-02-17T12:13:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5c9dffb640 1 -- 172.21.15.94:0/696972438 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca802ccf0 con 0x55932edc4250 2024-02-17T12:13:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5c9dffb640 1 -- 172.21.15.94:0/696972438 <== mon.0 v2:172.21.15.94:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f5ca8046040 con 0x55932edc4250 2024-02-17T12:13:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.631+0000 7f5cbf84c740 1 -- 172.21.15.94:0/696972438 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5cb001b040 con 0x55932edc4250 2024-02-17T12:13:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.632+0000 7f5c9dffb640 1 -- 172.21.15.94:0/696972438 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ca802c880 con 0x55932edc4250 2024-02-17T12:13:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.632+0000 7f5cbf84c740 10 client.19587 fetch_fsmap learned FSMap version 27 2024-02-17T12:13:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.632+0000 7f5cbf84c740 1 -- 172.21.15.94:0/696972438 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55932ecec8c0 con 0x55932edc4250 2024-02-17T12:13:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.632+0000 7f5c9dffb640 1 -- 172.21.15.94:0/696972438 <== mon.0 v2:172.21.15.94:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5cb001b040 con 0x55932edc4250 2024-02-17T12:13:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.632+0000 7f5c9dffb640 1 -- 172.21.15.94:0/696972438 <== mon.0 v2:172.21.15.94:3300/0 5 ==== fsmap(e 27) v1 ==== 7882+0+0 (secure 0 0 0) 0x7f5ca8051040 con 0x55932edc4250 2024-02-17T12:13:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.632+0000 7f5cbf84c740 10 client.19587 fetch_fsmap finished waiting for FSMap version 27 2024-02-17T12:13:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.632+0000 7f5cbf84c740 10 client.19587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-17T12:13:49.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.632+0000 7f5cbf84c740 20 client.19587 populate_metadata read hostname 'smithi094' 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.632+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55932ee39d90 0x55932ee3c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.632+0000 7f5cbf84c740 4 client.19587 mds_command: new command op to 24488 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.632+0000 7f5cbf84c740 1 -- 172.21.15.94:0/696972438 --> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55932eab7890 con 0x55932ee39d90 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.632+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55932ee39d90 0x55932ee3c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5c9dffb640 10 client.19587 ms_handle_connect on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cb5d9b640 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55932ee39d90 0x55932ee3c1b0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c28e45a30c102e server_cookie=a1ea69c8c1e6e0bb in_seq=0 out_seq=0 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5c9dffb640 1 -- 172.21.15.94:0/696972438 <== mds.0 v2:172.21.15.125:6838/1158324088 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55932eab7890 con 0x55932ee39d90 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5c9dffb640 10 client.19587 handle_command_reply: tid=0 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbf84c740 1 client.19587 shutdown 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbf84c740 1 -- 172.21.15.94:0/696972438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 msgr2=0x55932ee32290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932ee32290 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8003820 tx=0x7f5ca8003850 comp rx=0 tx=0).stop 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbf84c740 1 -- 172.21.15.94:0/696972438 shutdown_connections 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.125:6838/1158324088,v1:172.21.15.125:6839/1158324088] conn(0x55932ee39d90 0x55932ee3c1b0 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55932edcd750 0x55932ee2c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x55932edc4b70 0x55932ee327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5c9dffb640 0 client.19587 ms_handle_reset on v2:172.21.15.125:6838/1158324088 2024-02-17T12:13:49.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbf84c740 1 --2- 172.21.15.94:0/696972438 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x55932edc4250 0x55932ee32290 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbf84c740 1 -- 172.21.15.94:0/696972438 >> 172.21.15.94:0/696972438 conn(0x55932edbe980 msgr2=0x55932edc9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbf84c740 1 -- 172.21.15.94:0/696972438 shutdown_connections 2024-02-17T12:13:49.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbf84c740 1 -- 172.21.15.94:0/696972438 wait complete. 2024-02-17T12:13:49.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbf84c740 20 client.19587 trim_cache size 0 max 16384 2024-02-17T12:13:49.644 INFO:teuthology.orchestra.run.smithi094.stdout:[] 2024-02-17T12:13:49.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbcc83640 1 -- 172.21.15.94:0/1831811335 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5ca00a2230 msgr2=0x7f5ca00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbcc83640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5ca00a2230 0x7f5ca00a46f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0012d20 tx=0x7f5cb001d000 comp rx=0 tx=0).stop 2024-02-17T12:13:49.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbcc83640 1 -- 172.21.15.94:0/1831811335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb8102ce0 msgr2=0x7f5cb819d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:49.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.633+0000 7f5cbcc83640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb8102ce0 0x7f5cb819d9b0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00b4a0 tx=0x7f5cac00b960 comp rx=0 tx=0).stop 2024-02-17T12:13:49.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.634+0000 7f5cbcc83640 1 -- 172.21.15.94:0/1831811335 shutdown_connections 2024-02-17T12:13:49.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.634+0000 7f5cbcc83640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f5ca00a2230 0x7f5ca00a46f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.634+0000 7f5cbcc83640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cb810b790 0x7f5cb8197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.634+0000 7f5cbcc83640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f5cb8102ce0 0x7f5cb819d9b0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.634+0000 7f5cbcc83640 1 --2- 172.21.15.94:0/1831811335 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f5cb81023c0 0x7f5cb819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.634+0000 7f5cbcc83640 1 -- 172.21.15.94:0/1831811335 >> 172.21.15.94:0/1831811335 conn(0x7f5cb8076400 msgr2=0x7f5cb80fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.634+0000 7f5cbcc83640 1 -- 172.21.15.94:0/1831811335 shutdown_connections 2024-02-17T12:13:49.646 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:49.634+0000 7f5cbcc83640 1 -- 172.21.15.94:0/1831811335 wait complete. 2024-02-17T12:13:49.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:49 smithi189 ceph-mon[36588]: pgmap v492: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-02-17T12:13:50.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:50 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3299577754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:13:50.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:50 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3299577754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:13:50.647 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-02-17T12:13:50.648 INFO:tasks.fwd_scrub:done joining 2024-02-17T12:13:50.648 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-02-17T12:13:50.659 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:13:50.660 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:50 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3299577754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:13:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.780+0000 7f9b63a0e640 1 -- 172.21.15.94:0/2304661401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c101d30 msgr2=0x7f9b5c105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.780+0000 7f9b63a0e640 1 --2- 172.21.15.94:0/2304661401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c101d30 0x7f9b5c105d70 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009930 tx=0x7f9b4c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:13:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.780+0000 7f9b63a0e640 1 -- 172.21.15.94:0/2304661401 shutdown_connections 2024-02-17T12:13:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.780+0000 7f9b63a0e640 1 --2- 172.21.15.94:0/2304661401 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b5c106450 0x7f9b5c10e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.780+0000 7f9b63a0e640 1 --2- 172.21.15.94:0/2304661401 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c101d30 0x7f9b5c105d70 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.780+0000 7f9b63a0e640 1 --2- 172.21.15.94:0/2304661401 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9b5c101380 0x7f9b5c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.780+0000 7f9b63a0e640 1 -- 172.21.15.94:0/2304661401 >> 172.21.15.94:0/2304661401 conn(0x7f9b5c0fcc20 msgr2=0x7f9b5c0ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:50.783 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.780+0000 7f9b63a0e640 1 -- 172.21.15.94:0/2304661401 shutdown_connections 2024-02-17T12:13:50.784 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.780+0000 7f9b63a0e640 1 -- 172.21.15.94:0/2304661401 wait complete. 2024-02-17T12:13:50.785 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.781+0000 7f9b63a0e640 1 Processor -- start 2024-02-17T12:13:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.781+0000 7f9b63a0e640 1 -- start start 2024-02-17T12:13:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.781+0000 7f9b63a0e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c101380 0x7f9b5c19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.781+0000 7f9b63a0e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b5c106450 0x7f9b5c19db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.781+0000 7f9b63a0e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9b5c1976f0 0x7f9b5c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.781+0000 7f9b63a0e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c1981c0 con 0x7f9b5c101380 2024-02-17T12:13:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.781+0000 7f9b63a0e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c198330 con 0x7f9b5c1976f0 2024-02-17T12:13:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.781+0000 7f9b63a0e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c1984f0 con 0x7f9b5c106450 2024-02-17T12:13:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.781+0000 7f9b617aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c101380 0x7f9b5c19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.781+0000 7f9b617aa640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c101380 0x7f9b5c19d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57348/0 (socket says 172.21.15.94:57348) 2024-02-17T12:13:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.781+0000 7f9b617aa640 1 -- 172.21.15.94:0/351810263 learned_addr learned my addr 172.21.15.94:0/351810263 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.781+0000 7f9b61fab640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9b5c1976f0 0x7f9b5c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.781+0000 7f9b60fa9640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b5c106450 0x7f9b5c19db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:50.786 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.782+0000 7f9b617aa640 1 -- 172.21.15.94:0/351810263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b5c106450 msgr2=0x7f9b5c19db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.782+0000 7f9b617aa640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b5c106450 0x7f9b5c19db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.782+0000 7f9b617aa640 1 -- 172.21.15.94:0/351810263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9b5c1976f0 msgr2=0x7f9b5c197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.782+0000 7f9b617aa640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9b5c1976f0 0x7f9b5c197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.782+0000 7f9b617aa640 1 -- 172.21.15.94:0/351810263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b58009490 con 0x7f9b5c101380 2024-02-17T12:13:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.782+0000 7f9b617aa640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c101380 0x7f9b5c19d5e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800ea60 tx=0x7f9b5800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.782+0000 7f9b527fc640 1 -- 172.21.15.94:0/351810263 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5800cde0 con 0x7f9b5c101380 2024-02-17T12:13:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.782+0000 7f9b527fc640 1 -- 172.21.15.94:0/351810263 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9b580035f0 con 0x7f9b5c101380 2024-02-17T12:13:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.782+0000 7f9b63a0e640 1 -- 172.21.15.94:0/351810263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4c009560 con 0x7f9b5c101380 2024-02-17T12:13:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.782+0000 7f9b527fc640 1 -- 172.21.15.94:0/351810263 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b58010690 con 0x7f9b5c101380 2024-02-17T12:13:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.782+0000 7f9b63a0e640 1 -- 172.21.15.94:0/351810263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c1a4790 con 0x7f9b5c101380 2024-02-17T12:13:50.787 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.783+0000 7f9b63a0e640 1 -- 172.21.15.94:0/351810263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5c102850 con 0x7f9b5c101380 2024-02-17T12:13:50.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.784+0000 7f9b527fc640 1 -- 172.21.15.94:0/351810263 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9b58003af0 con 0x7f9b5c101380 2024-02-17T12:13:50.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.785+0000 7f9b527fc640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9b480a2230 0x7f9b480a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:50.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.785+0000 7f9b60fa9640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9b480a2230 0x7f9b480a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:50.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.785+0000 7f9b527fc640 1 -- 172.21.15.94:0/351810263 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f9b58014070 con 0x7f9b5c101380 2024-02-17T12:13:50.788 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.786+0000 7f9b60fa9640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9b480a2230 0x7f9b480a46f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c036040 tx=0x7f9b4c005fd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:50.791 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.789+0000 7f9b527fc640 1 -- 172.21.15.94:0/351810263 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b580973e0 con 0x7f9b5c101380 2024-02-17T12:13:50.972 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.970+0000 7f9b63a0e640 1 -- 172.21.15.94:0/351810263 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b5c109680 con 0x7f9b5c101380 2024-02-17T12:13:50.974 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.972+0000 7f9b527fc640 1 -- 172.21.15.94:0/351810263 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+23218 (secure 0 0 0) 0x7f9b5809aaa0 con 0x7f9b5c101380 2024-02-17T12:13:50.974 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:50.974 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":106,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T12:13:10.355878+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":24,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2639307451":"2024-02-18T11:58:06.744891+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/875984108":"2024-02-18T11:58:06.744891+0000","172.21.15.94:0/2865658350":"2024-02-18T11:58:06.744891+0000","172.21.15.94:0/142548766":"2024-02-18T11:58:06.744891+0000","172.21.15.94:0/1603490109":"2024-02-18T11:58:06.744891+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6801/1808883811":"2024-02-18T11:58:06.744891+0000","172.21.15.94:6800/1808883811":"2024-02-18T11:58:06.744891+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":11,"length":1},{"begin":24,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:13:50.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.974+0000 7f9b63a0e640 1 -- 172.21.15.94:0/351810263 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9b480a2230 msgr2=0x7f9b480a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:50.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.974+0000 7f9b63a0e640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9b480a2230 0x7f9b480a46f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c036040 tx=0x7f9b4c005fd0 comp rx=0 tx=0).stop 2024-02-17T12:13:50.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.974+0000 7f9b63a0e640 1 -- 172.21.15.94:0/351810263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c101380 msgr2=0x7f9b5c19d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:50.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.974+0000 7f9b63a0e640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c101380 0x7f9b5c19d5e0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800ea60 tx=0x7f9b5800ef20 comp rx=0 tx=0).stop 2024-02-17T12:13:50.976 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.974+0000 7f9b63a0e640 1 -- 172.21.15.94:0/351810263 shutdown_connections 2024-02-17T12:13:50.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.974+0000 7f9b63a0e640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f9b480a2230 0x7f9b480a46f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:50.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.974+0000 7f9b63a0e640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9b5c1976f0 0x7f9b5c197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:50.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.974+0000 7f9b63a0e640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b5c106450 0x7f9b5c19db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:50.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.974+0000 7f9b63a0e640 1 --2- 172.21.15.94:0/351810263 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9b5c101380 0x7f9b5c19d5e0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:50.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.974+0000 7f9b63a0e640 1 -- 172.21.15.94:0/351810263 >> 172.21.15.94:0/351810263 conn(0x7f9b5c0fcc20 msgr2=0x7f9b5c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:50.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.974+0000 7f9b63a0e640 1 -- 172.21.15.94:0/351810263 shutdown_connections 2024-02-17T12:13:50.977 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:50.974+0000 7f9b63a0e640 1 -- 172.21.15.94:0/351810263 wait complete. 2024-02-17T12:13:50.987 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T11:51:20.894601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T11:55:56.349040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T11:55:57.234916+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 24, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T12:13:50.988 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T12:13:51.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.091+0000 7f7176823640 1 -- 172.21.15.94:0/1852553169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7170100260 msgr2=0x7f7170100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.091+0000 7f7176823640 1 --2- 172.21.15.94:0/1852553169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7170100260 0x7f7170100640 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f7160009930 tx=0x7f716002bc30 comp rx=0 tx=0).stop 2024-02-17T12:13:51.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.091+0000 7f7176823640 1 -- 172.21.15.94:0/1852553169 shutdown_connections 2024-02-17T12:13:51.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.091+0000 7f7176823640 1 --2- 172.21.15.94:0/1852553169 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f7170105330 0x7f717010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.091+0000 7f7176823640 1 --2- 172.21.15.94:0/1852553169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7170100c10 0x7f7170104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.091+0000 7f7176823640 1 --2- 172.21.15.94:0/1852553169 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7170100260 0x7f7170100640 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.091+0000 7f7176823640 1 -- 172.21.15.94:0/1852553169 >> 172.21.15.94:0/1852553169 conn(0x7f7170077180 msgr2=0x7f7170077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:51.096 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.091+0000 7f7176823640 1 -- 172.21.15.94:0/1852553169 shutdown_connections 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.091+0000 7f7176823640 1 -- 172.21.15.94:0/1852553169 wait complete. 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.092+0000 7f7176823640 1 Processor -- start 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.092+0000 7f7176823640 1 -- start start 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.092+0000 7f7176823640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7170100260 0x7f71701966d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.092+0000 7f7176823640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7170105330 0x7f7170196c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.092+0000 7f7176823640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f717019af90 0x7f717019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.092+0000 7f7176823640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717019b980 con 0x7f7170105330 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.092+0000 7f7176823640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717019baf0 con 0x7f717019af90 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.092+0000 7f7176823640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717019bc60 con 0x7f7170100260 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.092+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7170105330 0x7f7170196c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f716f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7170105330 0x7f7170196c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57360/0 (socket says 172.21.15.94:57360) 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f716f7fe640 1 -- 172.21.15.94:0/1839185720 learned_addr learned my addr 172.21.15.94:0/1839185720 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f7174dc0640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f717019af90 0x7f717019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.097 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f716ffff640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7170100260 0x7f71701966d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f716f7fe640 1 -- 172.21.15.94:0/1839185720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7170100260 msgr2=0x7f71701966d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f716f7fe640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7170100260 0x7f71701966d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f716f7fe640 1 -- 172.21.15.94:0/1839185720 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f717019af90 msgr2=0x7f717019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f716f7fe640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f717019af90 0x7f717019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f716f7fe640 1 -- 172.21.15.94:0/1839185720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7160009560 con 0x7f7170105330 2024-02-17T12:13:51.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f716f7fe640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7170105330 0x7f7170196c10 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f716400c970 tx=0x7f716400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:51.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f716d7fa640 1 -- 172.21.15.94:0/1839185720 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164007bb0 con 0x7f7170105330 2024-02-17T12:13:51.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f716d7fa640 1 -- 172.21.15.94:0/1839185720 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7164007d10 con 0x7f7170105330 2024-02-17T12:13:51.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f7176823640 1 -- 172.21.15.94:0/1839185720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71701a3240 con 0x7f7170105330 2024-02-17T12:13:51.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f716d7fa640 1 -- 172.21.15.94:0/1839185720 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7164018730 con 0x7f7170105330 2024-02-17T12:13:51.098 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.093+0000 7f7176823640 1 -- 172.21.15.94:0/1839185720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71701a36a0 con 0x7f7170105330 2024-02-17T12:13:51.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.094+0000 7f7176823640 1 -- 172.21.15.94:0/1839185720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7170067f70 con 0x7f7170105330 2024-02-17T12:13:51.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.095+0000 7f716d7fa640 1 -- 172.21.15.94:0/1839185720 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f716400f450 con 0x7f7170105330 2024-02-17T12:13:51.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.096+0000 7f716d7fa640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f71580a2230 0x7f71580a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.096+0000 7f716ffff640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f71580a2230 0x7f71580a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.099 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.096+0000 7f716d7fa640 1 -- 172.21.15.94:0/1839185720 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f71640ce2a0 con 0x7f7170105330 2024-02-17T12:13:51.100 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.096+0000 7f716ffff640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f71580a2230 0x7f71580a46f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7160036040 tx=0x7f7160005fd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:51.102 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.100+0000 7f716d7fa640 1 -- 172.21.15.94:0/1839185720 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7164096fa0 con 0x7f7170105330 2024-02-17T12:13:51.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.279+0000 7f7176823640 1 -- 172.21.15.94:0/1839185720 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7170100c10 con 0x7f7170105330 2024-02-17T12:13:51.282 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.279+0000 7f716d7fa640 1 -- 172.21.15.94:0/1839185720 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f716409a660 con 0x7f7170105330 2024-02-17T12:13:51.282 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-02-17T12:13:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.281+0000 7f7176823640 1 -- 172.21.15.94:0/1839185720 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f71580a2230 msgr2=0x7f71580a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.281+0000 7f7176823640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f71580a2230 0x7f71580a46f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7160036040 tx=0x7f7160005fd0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.281+0000 7f7176823640 1 -- 172.21.15.94:0/1839185720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7170105330 msgr2=0x7f7170196c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.281+0000 7f7176823640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7170105330 0x7f7170196c10 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f716400c970 tx=0x7f716400ce30 comp rx=0 tx=0).stop 2024-02-17T12:13:51.284 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.281+0000 7f7176823640 1 -- 172.21.15.94:0/1839185720 shutdown_connections 2024-02-17T12:13:51.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.281+0000 7f7176823640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f71580a2230 0x7f71580a46f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.281+0000 7f7176823640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f717019af90 0x7f717019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.281+0000 7f7176823640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7170105330 0x7f7170196c10 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.281+0000 7f7176823640 1 --2- 172.21.15.94:0/1839185720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7170100260 0x7f71701966d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.281+0000 7f7176823640 1 -- 172.21.15.94:0/1839185720 >> 172.21.15.94:0/1839185720 conn(0x7f7170077180 msgr2=0x7f71700fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:51.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.282+0000 7f7176823640 1 -- 172.21.15.94:0/1839185720 shutdown_connections 2024-02-17T12:13:51.285 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.282+0000 7f7176823640 1 -- 172.21.15.94:0/1839185720 wait complete. 2024-02-17T12:13:51.294 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:51 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/351810263' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:13:51.294 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:51 smithi094 ceph-mon[32499]: pgmap v493: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-02-17T12:13:51.295 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T12:13:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.398+0000 7ff864862640 1 -- 172.21.15.94:0/4029428434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff860105330 msgr2=0x7ff86010cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.398+0000 7ff864862640 1 --2- 172.21.15.94:0/4029428434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff860105330 0x7ff86010cf00 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7ff850009930 tx=0x7ff85002bc30 comp rx=0 tx=0).stop 2024-02-17T12:13:51.402 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.398+0000 7ff864862640 1 -- 172.21.15.94:0/4029428434 shutdown_connections 2024-02-17T12:13:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.398+0000 7ff864862640 1 --2- 172.21.15.94:0/4029428434 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff860105330 0x7ff86010cf00 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.398+0000 7ff864862640 1 --2- 172.21.15.94:0/4029428434 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff860100c10 0x7ff860104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.398+0000 7ff864862640 1 --2- 172.21.15.94:0/4029428434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff860100260 0x7ff860100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.398+0000 7ff864862640 1 -- 172.21.15.94:0/4029428434 >> 172.21.15.94:0/4029428434 conn(0x7ff860077180 msgr2=0x7ff860077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.398+0000 7ff864862640 1 -- 172.21.15.94:0/4029428434 shutdown_connections 2024-02-17T12:13:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.398+0000 7ff864862640 1 -- 172.21.15.94:0/4029428434 wait complete. 2024-02-17T12:13:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.398+0000 7ff864862640 1 Processor -- start 2024-02-17T12:13:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff864862640 1 -- start start 2024-02-17T12:13:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff864862640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff860100260 0x7ff860196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff864862640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff860105330 0x7ff860196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff864862640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff860196ec0 0x7ff8601a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff864862640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8601a6000 con 0x7ff860100260 2024-02-17T12:13:51.403 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff864862640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8601a6140 con 0x7ff860196ec0 2024-02-17T12:13:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff864862640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8601a6280 con 0x7ff860105330 2024-02-17T12:13:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff860100260 0x7ff860196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff85e59c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff860100260 0x7ff860196440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57370/0 (socket says 172.21.15.94:57370) 2024-02-17T12:13:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff85e59c640 1 -- 172.21.15.94:0/2339802852 learned_addr learned my addr 172.21.15.94:0/2339802852 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff85dd9b640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff860105330 0x7ff860196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff85ed9d640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff860196ec0 0x7ff8601a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff85e59c640 1 -- 172.21.15.94:0/2339802852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff860105330 msgr2=0x7ff860196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff85e59c640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff860105330 0x7ff860196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff85e59c640 1 -- 172.21.15.94:0/2339802852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff860196ec0 msgr2=0x7ff8601a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff85e59c640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff860196ec0 0x7ff8601a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff85e59c640 1 -- 172.21.15.94:0/2339802852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff850009560 con 0x7ff860100260 2024-02-17T12:13:51.404 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff85dd9b640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff860105330 0x7ff860196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:13:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff85e59c640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff860100260 0x7ff860196440 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ff85800ba80 tx=0x7ff85800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff8577fe640 1 -- 172.21.15.94:0/2339802852 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85800cc00 con 0x7ff860100260 2024-02-17T12:13:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff8577fe640 1 -- 172.21.15.94:0/2339802852 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff85800cd60 con 0x7ff860100260 2024-02-17T12:13:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff864862640 1 -- 172.21.15.94:0/2339802852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8601a6560 con 0x7ff860100260 2024-02-17T12:13:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.399+0000 7ff8577fe640 1 -- 172.21.15.94:0/2339802852 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8580077d0 con 0x7ff860100260 2024-02-17T12:13:51.405 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.400+0000 7ff864862640 1 -- 172.21.15.94:0/2339802852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8601a6a70 con 0x7ff860100260 2024-02-17T12:13:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.401+0000 7ff864862640 1 -- 172.21.15.94:0/2339802852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff860067f70 con 0x7ff860100260 2024-02-17T12:13:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.402+0000 7ff8577fe640 1 -- 172.21.15.94:0/2339802852 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff858014a90 con 0x7ff860100260 2024-02-17T12:13:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.402+0000 7ff8577fe640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff84c0a22b0 0x7ff84c0a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.402+0000 7ff85dd9b640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff84c0a22b0 0x7ff84c0a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.402+0000 7ff8577fe640 1 -- 172.21.15.94:0/2339802852 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7ff8580cd8a0 con 0x7ff860100260 2024-02-17T12:13:51.406 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.403+0000 7ff85dd9b640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff84c0a22b0 0x7ff84c0a4770 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff8601985a0 tx=0x7ff84800b5f0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:51.408 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.406+0000 7ff8577fe640 1 -- 172.21.15.94:0/2339802852 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff85804fd20 con 0x7ff860100260 2024-02-17T12:13:51.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:51 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/351810263' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:13:51.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:51 smithi125 ceph-mon[36819]: pgmap v493: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-02-17T12:13:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.579+0000 7ff864862640 1 -- 172.21.15.94:0/2339802852 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff860100c10 con 0x7ff860100260 2024-02-17T12:13:51.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.580+0000 7ff8577fe640 1 -- 172.21.15.94:0/2339802852 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff8580966b0 con 0x7ff860100260 2024-02-17T12:13:51.582 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-02-17T12:13:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.582+0000 7ff864862640 1 -- 172.21.15.94:0/2339802852 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff84c0a22b0 msgr2=0x7ff84c0a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.584 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.582+0000 7ff864862640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff84c0a22b0 0x7ff84c0a4770 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff8601985a0 tx=0x7ff84800b5f0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.582+0000 7ff864862640 1 -- 172.21.15.94:0/2339802852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff860100260 msgr2=0x7ff860196440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.582+0000 7ff864862640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff860100260 0x7ff860196440 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ff85800ba80 tx=0x7ff85800bf40 comp rx=0 tx=0).stop 2024-02-17T12:13:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.582+0000 7ff864862640 1 -- 172.21.15.94:0/2339802852 shutdown_connections 2024-02-17T12:13:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.582+0000 7ff864862640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff84c0a22b0 0x7ff84c0a4770 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.582+0000 7ff864862640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff860196ec0 0x7ff8601a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.582+0000 7ff864862640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff860105330 0x7ff860196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.582+0000 7ff864862640 1 --2- 172.21.15.94:0/2339802852 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff860100260 0x7ff860196440 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.582+0000 7ff864862640 1 -- 172.21.15.94:0/2339802852 >> 172.21.15.94:0/2339802852 conn(0x7ff860077180 msgr2=0x7ff8600fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.582+0000 7ff864862640 1 -- 172.21.15.94:0/2339802852 shutdown_connections 2024-02-17T12:13:51.585 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.582+0000 7ff864862640 1 -- 172.21.15.94:0/2339802852 wait complete. 2024-02-17T12:13:51.595 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T12:13:51.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:51 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/351810263' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:13:51.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:51 smithi189 ceph-mon[36588]: pgmap v493: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 8 op/s 2024-02-17T12:13:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.696+0000 7f7908f35640 1 -- 172.21.15.94:0/762535683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79040ff7b0 msgr2=0x7f79040ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.696+0000 7f7908f35640 1 --2- 172.21.15.94:0/762535683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79040ff7b0 0x7f79040ffc30 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f78ec009900 tx=0x7f78ec02bc30 comp rx=0 tx=0).stop 2024-02-17T12:13:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.697+0000 7f7908f35640 1 -- 172.21.15.94:0/762535683 shutdown_connections 2024-02-17T12:13:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.697+0000 7f7908f35640 1 --2- 172.21.15.94:0/762535683 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7904100170 0x7f790410c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.697+0000 7f7908f35640 1 --2- 172.21.15.94:0/762535683 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79040ff7b0 0x7f79040ffc30 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.697+0000 7f7908f35640 1 --2- 172.21.15.94:0/762535683 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79041057c0 0x7f7904105ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.699 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.697+0000 7f7908f35640 1 -- 172.21.15.94:0/762535683 >> 172.21.15.94:0/762535683 conn(0x7f7904076640 msgr2=0x7f79040fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:51.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.697+0000 7f7908f35640 1 -- 172.21.15.94:0/762535683 shutdown_connections 2024-02-17T12:13:51.700 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.697+0000 7f7908f35640 1 -- 172.21.15.94:0/762535683 wait complete. 2024-02-17T12:13:51.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.697+0000 7f7908f35640 1 Processor -- start 2024-02-17T12:13:51.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.697+0000 7f7908f35640 1 -- start start 2024-02-17T12:13:51.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7908f35640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79040ff7b0 0x7f790419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7908f35640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7904100170 0x7f790419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7908f35640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79041057c0 0x7f79041975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7908f35640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7904197c30 con 0x7f79041057c0 2024-02-17T12:13:51.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7908f35640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7904197da0 con 0x7f79040ff7b0 2024-02-17T12:13:51.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7908f35640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7904197f60 con 0x7f7904100170 2024-02-17T12:13:51.702 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7902d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79041057c0 0x7f79041975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7902d9d640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79041057c0 0x7f79041975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57390/0 (socket says 172.21.15.94:57390) 2024-02-17T12:13:51.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7902d9d640 1 -- 172.21.15.94:0/4042362016 learned_addr learned my addr 172.21.15.94:0/4042362016 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:51.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f790259c640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79040ff7b0 0x7f790419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7901d9b640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7904100170 0x7f790419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7902d9d640 1 -- 172.21.15.94:0/4042362016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7904100170 msgr2=0x7f790419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7902d9d640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7904100170 0x7f790419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7902d9d640 1 -- 172.21.15.94:0/4042362016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79040ff7b0 msgr2=0x7f790419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7902d9d640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79040ff7b0 0x7f790419d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7902d9d640 1 -- 172.21.15.94:0/4042362016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec009560 con 0x7f79041057c0 2024-02-17T12:13:51.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7902d9d640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79041057c0 0x7f79041975e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00eaa0 tx=0x7f78fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:51.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f78fb7fe640 1 -- 172.21.15.94:0/4042362016 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc00cdb0 con 0x7f79041057c0 2024-02-17T12:13:51.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f78fb7fe640 1 -- 172.21.15.94:0/4042362016 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f78fc005020 con 0x7f79041057c0 2024-02-17T12:13:51.703 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7908f35640 1 -- 172.21.15.94:0/4042362016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7904198240 con 0x7f79041057c0 2024-02-17T12:13:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f78fb7fe640 1 -- 172.21.15.94:0/4042362016 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78fc0116b0 con 0x7f79041057c0 2024-02-17T12:13:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.698+0000 7f7908f35640 1 -- 172.21.15.94:0/4042362016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79041a44d0 con 0x7f79041057c0 2024-02-17T12:13:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.699+0000 7f7908f35640 1 -- 172.21.15.94:0/4042362016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79040683c0 con 0x7f79041057c0 2024-02-17T12:13:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.700+0000 7f78fb7fe640 1 -- 172.21.15.94:0/4042362016 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f78fc005520 con 0x7f79041057c0 2024-02-17T12:13:51.704 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.701+0000 7f78fb7fe640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f78f00a2230 0x7f78f00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.701+0000 7f78fb7fe640 1 -- 172.21.15.94:0/4042362016 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f78fc0ce2a0 con 0x7f79041057c0 2024-02-17T12:13:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.701+0000 7f790259c640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f78f00a2230 0x7f78f00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.705 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.701+0000 7f790259c640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f78f00a2230 0x7f78f00a46f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f78f40058c0 tx=0x7f78f400a2d0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:51.707 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.705+0000 7f78fb7fe640 1 -- 172.21.15.94:0/4042362016 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f78fc096fa0 con 0x7f79041057c0 2024-02-17T12:13:51.879 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.877+0000 7f7908f35640 1 -- 172.21.15.94:0/4042362016 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7904198b10 con 0x7f79041057c0 2024-02-17T12:13:51.880 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.877+0000 7f78fb7fe640 1 -- 172.21.15.94:0/4042362016 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7904198b10 con 0x7f79041057c0 2024-02-17T12:13:51.880 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-02-17T12:13:51.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.879+0000 7f7908f35640 1 -- 172.21.15.94:0/4042362016 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f78f00a2230 msgr2=0x7f78f00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.879+0000 7f7908f35640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f78f00a2230 0x7f78f00a46f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f78f40058c0 tx=0x7f78f400a2d0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.879+0000 7f7908f35640 1 -- 172.21.15.94:0/4042362016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79041057c0 msgr2=0x7f79041975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.879+0000 7f7908f35640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79041057c0 0x7f79041975e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f78fc00eaa0 tx=0x7f78fc00ef60 comp rx=0 tx=0).stop 2024-02-17T12:13:51.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.879+0000 7f7908f35640 1 -- 172.21.15.94:0/4042362016 shutdown_connections 2024-02-17T12:13:51.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.879+0000 7f7908f35640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f78f00a2230 0x7f78f00a46f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.879+0000 7f7908f35640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f79041057c0 0x7f79041975e0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.882 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.879+0000 7f7908f35640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7904100170 0x7f790419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.879+0000 7f7908f35640 1 --2- 172.21.15.94:0/4042362016 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f79040ff7b0 0x7f790419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.879+0000 7f7908f35640 1 -- 172.21.15.94:0/4042362016 >> 172.21.15.94:0/4042362016 conn(0x7f7904076640 msgr2=0x7f7904102580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:51.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.879+0000 7f7908f35640 1 -- 172.21.15.94:0/4042362016 shutdown_connections 2024-02-17T12:13:51.883 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.880+0000 7f7908f35640 1 -- 172.21.15.94:0/4042362016 wait complete. 2024-02-17T12:13:51.893 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-02-17T12:13:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.992+0000 7f35eba5c640 1 -- 172.21.15.94:0/2505833253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4101d60 msgr2=0x7f35e4105da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.992+0000 7f35eba5c640 1 --2- 172.21.15.94:0/2505833253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4101d60 0x7f35e4105da0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009930 tx=0x7f35e002bc30 comp rx=0 tx=0).stop 2024-02-17T12:13:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.992+0000 7f35eba5c640 1 -- 172.21.15.94:0/2505833253 shutdown_connections 2024-02-17T12:13:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.992+0000 7f35eba5c640 1 --2- 172.21.15.94:0/2505833253 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35e4106480 0x7f35e410e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.992+0000 7f35eba5c640 1 --2- 172.21.15.94:0/2505833253 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4101d60 0x7f35e4105da0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.997 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.992+0000 7f35eba5c640 1 --2- 172.21.15.94:0/2505833253 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35e41013b0 0x7f35e4101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.992+0000 7f35eba5c640 1 -- 172.21.15.94:0/2505833253 >> 172.21.15.94:0/2505833253 conn(0x7f35e40fcc30 msgr2=0x7f35e40ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:51.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.992+0000 7f35eba5c640 1 -- 172.21.15.94:0/2505833253 shutdown_connections 2024-02-17T12:13:51.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.992+0000 7f35eba5c640 1 -- 172.21.15.94:0/2505833253 wait complete. 2024-02-17T12:13:51.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.992+0000 7f35eba5c640 1 Processor -- start 2024-02-17T12:13:51.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35eba5c640 1 -- start start 2024-02-17T12:13:51.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35eba5c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35e41013b0 0x7f35e419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35eba5c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4106480 0x7f35e419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35eba5c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35e41976e0 0x7f35e4197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:51.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35eba5c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e41981b0 con 0x7f35e4106480 2024-02-17T12:13:51.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35eba5c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4198320 con 0x7f35e41013b0 2024-02-17T12:13:51.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35eba5c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e41984e0 con 0x7f35e41976e0 2024-02-17T12:13:51.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35e8ff7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4106480 0x7f35e419db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.998 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35e8ff7640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4106480 0x7f35e419db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:57394/0 (socket says 172.21.15.94:57394) 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35e8ff7640 1 -- 172.21.15.94:0/2552144017 learned_addr learned my addr 172.21.15.94:0/2552144017 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35e9ff9640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35e41976e0 0x7f35e4197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35e97f8640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35e41013b0 0x7f35e419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35e8ff7640 1 -- 172.21.15.94:0/2552144017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35e41976e0 msgr2=0x7f35e4197b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35e8ff7640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35e41976e0 0x7f35e4197b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35e8ff7640 1 -- 172.21.15.94:0/2552144017 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35e41013b0 msgr2=0x7f35e419d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35e8ff7640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35e41013b0 0x7f35e419d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35e8ff7640 1 -- 172.21.15.94:0/2552144017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e0009560 con 0x7f35e4106480 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35e97f8640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35e41013b0 0x7f35e419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35e8ff7640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4106480 0x7f35e419db10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009230 tx=0x7f35e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.993+0000 7f35da7fc640 1 -- 172.21.15.94:0/2552144017 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e002cbc0 con 0x7f35e4106480 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.994+0000 7f35eba5c640 1 -- 172.21.15.94:0/2552144017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e41a43d0 con 0x7f35e4106480 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.994+0000 7f35da7fc640 1 -- 172.21.15.94:0/2552144017 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f35e00052d0 con 0x7f35e4106480 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.994+0000 7f35da7fc640 1 -- 172.21.15.94:0/2552144017 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0003e50 con 0x7f35e4106480 2024-02-17T12:13:51.999 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.994+0000 7f35eba5c640 1 -- 172.21.15.94:0/2552144017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e41a48b0 con 0x7f35e4106480 2024-02-17T12:13:52.000 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.995+0000 7f35eba5c640 1 -- 172.21.15.94:0/2552144017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e4102880 con 0x7f35e4106480 2024-02-17T12:13:52.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.996+0000 7f35da7fc640 1 -- 172.21.15.94:0/2552144017 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f35e00035f0 con 0x7f35e4106480 2024-02-17T12:13:52.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.996+0000 7f35da7fc640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35d00a2230 0x7f35d00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:13:52.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.996+0000 7f35e97f8640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35d00a2230 0x7f35d00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:13:52.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.997+0000 7f35da7fc640 1 -- 172.21.15.94:0/2552144017 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f35e00f0b80 con 0x7f35e4106480 2024-02-17T12:13:52.001 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:51.997+0000 7f35e97f8640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35d00a2230 0x7f35d00a46f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009b40 tx=0x7f35d4005e50 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:13:52.002 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.000+0000 7f35da7fc640 1 -- 172.21.15.94:0/2552144017 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f35e00b98e0 con 0x7f35e4106480 2024-02-17T12:13:52.224 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.222+0000 7f35eba5c640 1 -- 172.21.15.94:0/2552144017 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f35e41096b0 con 0x7f35e4106480 2024-02-17T12:13:52.232 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.230+0000 7f35da7fc640 1 -- 172.21.15.94:0/2552144017 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) v1 ==== 81+0+736943 (secure 0 0 0) 0x7f35e0005440 con 0x7f35e4106480 2024-02-17T12:13:52.239 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:13:52.239 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: "epoch": 27, 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: "active_gid": 15435, 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: "active_name": "x", 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: "active_addrs": { 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6800", 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2106228828 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v1", 2024-02-17T12:13:52.240 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:6801", 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2106228828 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: "active_addr": "172.21.15.94:6801/2106228828", 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: "active_change": "2024-02-17T11:58:06.745111+0000", 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: "active_mgr_features": 4540701547738038271, 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: "available": true, 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: "standbys": [ 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 24929, 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "y", 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_features": 4540701547738038271, 2024-02-17T12:13:52.241 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.242 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.243 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.244 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.245 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.246 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.247 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.248 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.250 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-02-17T12:13:52.251 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-02-17T12:13:52.252 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.252 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.252 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-02-17T12:13:52.252 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.252 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.252 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.252 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-02-17T12:13:52.252 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.254 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.256 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.257 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:52.258 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-02-17T12:13:52.259 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.260 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-02-17T12:13:52.261 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.262 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-02-17T12:13:52.263 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.264 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.265 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.267 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.268 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.270 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.271 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-17T12:13:52.272 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-17T12:13:52.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.274 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-02-17T12:13:52.275 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-02-17T12:13:52.276 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.277 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.278 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.279 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.280 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.281 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.282 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.283 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.284 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.286 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-02-17T12:13:52.287 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-02-17T12:13:52.288 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-02-17T12:13:52.289 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.291 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.292 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-02-17T12:13:52.292 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-02-17T12:13:52.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.292 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.292 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.292 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-17T12:13:52.292 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.292 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.292 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.292 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.293 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.294 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-02-17T12:13:52.295 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.296 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.297 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.299 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.300 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.301 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-02-17T12:13:52.302 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-02-17T12:13:52.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-02-17T12:13:52.304 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.305 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.306 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.307 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.308 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.309 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-02-17T12:13:52.310 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.311 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.312 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.313 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.314 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.315 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.316 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.318 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-02-17T12:13:52.319 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.320 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-02-17T12:13:52.323 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-02-17T12:13:52.324 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-02-17T12:13:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-17T12:13:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-02-17T12:13:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-02-17T12:13:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-17T12:13:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.453 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.454 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.455 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.456 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.457 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.458 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-02-17T12:13:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-02-17T12:13:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-17T12:13:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.460 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.461 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.462 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.464 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.465 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.466 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.467 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.468 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.469 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-02-17T12:13:52.470 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.471 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.472 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.473 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.474 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.475 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-17T12:13:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-17T12:13:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.476 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.477 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.478 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.479 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.480 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.481 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.482 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.483 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.484 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.485 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.486 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.487 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.488 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.489 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.490 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.492 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.494 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.495 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.496 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-17T12:13:52.497 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-17T12:13:52.498 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-02-17T12:13:52.499 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-17T12:13:52.500 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-17T12:13:52.501 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-17T12:13:52.502 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.503 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.504 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.505 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.506 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.507 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:52.508 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.509 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.510 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.511 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.512 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.513 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.514 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.515 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.516 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.517 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.518 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.519 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.521 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-02-17T12:13:52.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.523 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.524 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.525 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-02-17T12:13:52.527 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-02-17T12:13:52.528 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.529 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.530 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.531 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.532 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.533 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.534 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.535 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-02-17T12:13:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-02-17T12:13:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-02-17T12:13:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-17T12:13:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.536 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.537 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.538 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.542 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.543 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.545 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.546 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.547 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.549 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.550 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-17T12:13:52.551 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.552 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.553 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.554 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.555 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.556 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.557 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.558 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.559 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-02-17T12:13:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-02-17T12:13:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-02-17T12:13:52.560 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-02-17T12:13:52.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.562 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.563 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.564 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.565 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.566 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.567 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.568 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.568 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.568 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.568 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.568 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.568 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.568 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.568 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.568 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.568 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.571 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.572 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.573 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.574 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.575 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.576 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.577 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.578 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.580 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.580 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.580 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.580 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.580 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.580 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.580 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.580 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.580 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.580 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.580 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.580 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.581 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.582 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.583 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.585 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.586 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.587 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.588 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.589 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.589 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.589 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.589 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.589 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.589 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.589 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.589 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.589 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.589 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.589 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.589 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.591 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.592 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.593 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.595 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.596 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.597 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.598 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.599 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.600 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.601 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.603 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.604 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.606 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.607 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.608 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.609 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.610 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.610 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.610 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.611 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.611 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.611 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.611 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.611 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.611 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.612 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.613 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.614 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.615 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.616 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.617 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-02-17T12:13:52.618 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.619 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.620 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.621 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.622 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.623 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.624 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.625 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.626 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.627 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.628 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.628 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.628 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.628 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.629 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.630 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.631 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.632 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.633 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.634 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.635 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-02-17T12:13:52.636 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.637 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.638 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.639 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.640 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.641 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.642 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.643 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.644 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.644 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.644 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.644 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.644 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.644 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.644 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.644 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.644 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.644 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.644 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.644 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.645 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.646 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.647 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.648 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.649 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.650 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.651 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.652 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.653 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.654 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.655 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.656 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-02-17T12:13:52.657 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-02-17T12:13:52.658 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.659 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-02-17T12:13:52.660 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.661 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.662 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.663 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.664 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.665 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.666 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.667 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.668 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.669 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.670 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.671 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.672 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.673 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.674 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.674 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.674 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.674 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.674 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.674 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.674 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.674 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.674 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.674 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.674 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.675 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.676 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.677 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.678 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.679 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.680 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.681 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.682 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.683 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: "drive_group_interval": { 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "drive_group_interval", 2024-02-17T12:13:52.684 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300.0", 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.685 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.686 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.687 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.688 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.689 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.690 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.691 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.692 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.693 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.694 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.695 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.696 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.697 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.698 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.699 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.700 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.701 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.702 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.703 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.704 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.705 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.706 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.707 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.708 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.709 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.710 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.711 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.712 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.713 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.714 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.715 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.716 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.716 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.716 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.716 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.716 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.716 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.716 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.716 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.716 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.716 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.716 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.716 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.716 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.717 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.718 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.719 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-02-17T12:13:52.720 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.721 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.722 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.723 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.724 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.725 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.726 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.727 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.728 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.729 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.730 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.731 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.732 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-02-17T12:13:52.733 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.734 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.735 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.736 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.737 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.738 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.739 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.740 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.740 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.740 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.740 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.740 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-02-17T12:13:52.740 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-02-17T12:13:52.740 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.740 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.740 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.740 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-17T12:13:52.740 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.740 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.740 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.741 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.742 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.743 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.744 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.745 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.746 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.747 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.748 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.749 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-02-17T12:13:52.750 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-02-17T12:13:52.751 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-02-17T12:13:52.752 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.753 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.754 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.755 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.756 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.757 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.758 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.759 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.760 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.761 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.762 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.763 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.764 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.765 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.766 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-02-17T12:13:52.767 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.768 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.769 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.770 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.771 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.772 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.773 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.774 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.775 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.776 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.777 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: "gid": 24967, 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "z", 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: "mgr_features": 4540701547738038271, 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:52.778 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.779 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.780 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.781 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.782 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.783 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.784 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.785 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.786 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-02-17T12:13:52.787 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-02-17T12:13:52.788 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.789 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.790 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.791 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.792 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.793 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-02-17T12:13:52.794 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.795 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.796 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-02-17T12:13:52.797 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-02-17T12:13:52.798 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.799 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.800 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.801 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.802 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.803 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-02-17T12:13:52.804 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.805 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.806 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.807 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.808 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.808 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:52.808 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:52.808 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.808 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.808 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.808 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:52.808 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.808 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.808 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.808 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-17T12:13:52.808 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.808 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.809 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.810 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-02-17T12:13:52.811 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-02-17T12:13:52.812 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.813 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.813 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.813 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-17T12:13:52.813 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.815 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.815 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.815 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.815 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-02-17T12:13:52.816 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-02-17T12:13:52.817 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:52.818 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.819 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.820 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.821 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.822 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.823 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.824 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-02-17T12:13:52.825 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-02-17T12:13:52.826 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-02-17T12:13:52.827 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.828 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.829 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.830 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.831 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.832 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-17T12:13:52.833 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-17T12:13:52.834 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-17T12:13:52.835 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-17T12:13:52.836 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.837 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.838 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.839 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.840 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.841 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.842 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-17T12:13:52.843 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.844 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.844 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.844 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.844 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-02-17T12:13:52.844 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-02-17T12:13:52.844 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.844 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.844 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.844 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:52.844 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.844 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.844 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.844 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.845 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.846 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.847 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.848 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.849 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.850 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.851 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.852 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-17T12:13:52.853 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.854 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.855 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.856 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.857 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.858 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-02-17T12:13:52.859 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-02-17T12:13:52.860 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-02-17T12:13:52.861 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-02-17T12:13:52.862 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.863 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.864 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.865 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.866 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.867 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.868 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.868 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.868 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-17T12:13:52.868 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.868 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.868 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.868 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.868 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-02-17T12:13:52.868 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-02-17T12:13:52.868 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.868 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.868 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.868 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.869 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.870 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.871 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.872 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.873 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.874 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.875 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.876 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.877 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-02-17T12:13:52.878 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-02-17T12:13:52.879 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.880 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.881 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.882 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.883 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.884 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.885 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.886 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.887 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.888 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.889 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.890 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-17T12:13:52.891 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-17T12:13:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.892 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-17T12:13:52.893 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-02-17T12:13:52.894 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-17T12:13:52.895 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-02-17T12:13:52.896 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-17T12:13:52.897 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.898 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.899 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.900 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.901 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.902 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.903 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.904 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.905 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.906 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.907 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.908 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.909 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.910 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.911 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.912 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.913 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.914 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-02-17T12:13:52.915 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-17T12:13:52.916 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-02-17T12:13:52.917 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-17T12:13:52.918 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.919 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.920 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.921 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.922 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.923 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.924 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-02-17T12:13:52.925 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.926 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.927 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.928 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.929 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.930 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.930 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.930 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.930 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.930 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.930 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.930 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.930 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.930 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.930 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.930 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.930 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.930 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.931 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.932 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.933 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.934 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.935 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.936 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.937 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-02-17T12:13:52.938 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.939 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.940 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.941 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.942 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.943 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.944 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.945 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.946 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.947 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.948 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.949 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-17T12:13:52.950 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.951 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.952 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.953 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.954 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.955 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.956 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.957 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.958 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.959 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-02-17T12:13:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-02-17T12:13:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-02-17T12:13:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.960 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.961 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.962 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.963 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-02-17T12:13:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-17T12:13:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-02-17T12:13:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-02-17T12:13:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.964 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.965 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.966 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.967 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.968 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.969 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.970 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.971 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.972 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.973 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-17T12:13:52.974 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-02-17T12:13:52.975 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.976 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.977 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.978 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.979 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.980 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.981 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.982 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.983 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.984 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.985 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.986 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.987 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.988 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.989 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.990 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.991 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.992 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:52.993 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.994 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-02-17T12:13:52.995 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.996 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:52.997 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:52.998 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:52.999 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.000 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-17T12:13:53.001 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.002 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.003 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.004 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.005 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.006 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.009 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.010 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.011 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.012 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.013 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.014 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.015 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-02-17T12:13:53.016 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.017 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.019 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.020 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.021 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.022 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.023 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.024 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.025 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.026 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.027 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.028 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.029 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-02-17T12:13:53.030 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.031 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.032 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.033 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.034 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.035 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.036 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.037 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.038 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.039 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.040 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.040 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.040 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.040 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.040 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.040 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.040 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.040 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.040 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.040 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.040 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.040 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.040 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.041 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.042 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.043 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.044 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.045 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.046 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.047 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.048 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.049 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.050 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.051 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.052 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.053 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.054 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.055 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.056 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.057 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.058 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.059 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.060 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.060 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-17T12:13:53.060 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-17T12:13:53.060 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.060 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.060 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.060 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.060 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.060 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.060 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.060 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.060 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.060 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.061 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.062 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.063 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.064 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.065 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.066 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.067 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.068 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.068 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.068 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.068 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.068 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.068 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-02-17T12:13:53.068 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-02-17T12:13:53.068 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.068 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.068 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.068 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-02-17T12:13:53.068 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.068 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.069 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.070 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-02-17T12:13:53.071 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-02-17T12:13:53.072 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.073 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.074 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.075 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.076 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.077 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.078 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.079 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.080 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.081 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.081 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.081 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.081 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.081 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.081 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.081 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.081 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-02-17T12:13:53.081 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.081 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.081 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.081 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-02-17T12:13:53.082 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.083 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.084 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.085 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.086 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.087 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.088 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.089 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:52 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1839185720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:13:53.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:52 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2339802852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:13:53.090 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:52 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/4042362016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:13:53.090 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:52 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1839185720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:13:53.090 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:52 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2339802852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:13:53.091 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:52 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/4042362016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:13:53.092 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.092 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.092 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.092 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.092 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.092 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.092 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.092 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.092 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-02-17T12:13:53.092 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.093 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.094 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.095 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.096 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.096 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.096 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.096 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.096 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.096 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.096 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.096 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.096 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.096 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.096 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.096 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.096 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.097 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.098 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: "drive_group_interval": { 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "drive_group_interval", 2024-02-17T12:13:53.099 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300.0", 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.100 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.101 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.102 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.103 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.103 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.103 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.103 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.103 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.103 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.103 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.103 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.103 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.103 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.103 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.103 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.103 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.104 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.105 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.105 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.105 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.105 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.105 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.105 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.105 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.105 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.105 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.105 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.105 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-02-17T12:13:53.105 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-02-17T12:13:53.105 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.106 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.107 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.108 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.109 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.110 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.111 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.112 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.113 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.114 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.115 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.116 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.117 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.118 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.119 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-02-17T12:13:53.119 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-02-17T12:13:53.119 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.119 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.119 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.119 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.119 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.119 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.119 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.119 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.119 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.119 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.119 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.120 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.121 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.121 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-02-17T12:13:53.121 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-02-17T12:13:53.121 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.121 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.121 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.121 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.121 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.121 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.121 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.121 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.121 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.121 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.122 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.123 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.124 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.124 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.124 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.124 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.124 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-02-17T12:13:53.124 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.124 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.124 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.124 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-02-17T12:13:53.124 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-02-17T12:13:53.124 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.124 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.124 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.125 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.126 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.127 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.128 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.129 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.130 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.131 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.132 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.133 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.134 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.135 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.136 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.137 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.138 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.139 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.140 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.141 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-02-17T12:13:53.142 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.143 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.144 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.145 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.146 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.147 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.148 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-02-17T12:13:53.149 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.150 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.151 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.152 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.153 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.154 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.155 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.156 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-02-17T12:13:53.157 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:53.158 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-02-17T12:13:53.159 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-02-17T12:13:53.160 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.161 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.162 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.163 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.164 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.164 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.164 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.164 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.164 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.164 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.164 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.164 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.164 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.164 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.164 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.164 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.164 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.165 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.166 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.167 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.168 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-17T12:13:53.169 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.170 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.171 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.172 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.173 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.174 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.175 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.176 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.177 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.178 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.179 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-02-17T12:13:53.180 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-02-17T12:13:53.181 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.182 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-02-17T12:13:53.183 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.184 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.185 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.186 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.187 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.188 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.189 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.190 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.191 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-02-17T12:13:53.192 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.193 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "modules": [ 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "dashboard", 2024-02-17T12:13:53.194 INFO:teuthology.orchestra.run.smithi094.stdout: "iostat", 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "nfs", 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "restful", 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "snap_schedule" 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "available_modules": [ 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "alerts", 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.195 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How frequently to reexamine health status", 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.196 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.197 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.198 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.199 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.200 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_destination": { 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_destination", 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email address to send alerts to", 2024-02-17T12:13:53.201 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_from_name": { 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_from_name", 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "Ceph", 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Email From: name", 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.202 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_host": { 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_host", 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP server", 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.203 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_password": { 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_password", 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Password to authenticate with", 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.204 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_port": { 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_port", 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "465", 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP port", 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.205 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_sender": { 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_sender", 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SMTP envelope sender", 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_ssl": { 2024-02-17T12:13:53.206 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_ssl", 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "smtp_user": { 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "smtp_user", 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.207 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "User to authenticate as", 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.208 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "balancer", 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "active": { 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "active", 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically balance PGs across cluster", 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.209 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_time": { 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_time", 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0000", 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "beginning time of day to automatically balance", 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.210 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "begin_weekday": { 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "begin_weekday", 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.211 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_max_iterations": { 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_max_iterations", 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "25", 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "250", 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.212 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_metrics": { 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_metrics", 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "pgs,objects,bytes", 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "crush_compat_step": { 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crush_compat_step", 2024-02-17T12:13:53.213 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.5", 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0.001", 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "0.999", 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "aggressiveness of optimization", 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "end_time": { 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_time", 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.214 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2359", 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ending time of day to automatically balance", 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "end_weekday": { 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "end_weekday", 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.215 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "0", 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "6", 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.216 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.217 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.218 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.219 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.220 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "min_score": { 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_score", 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.221 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.222 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-17T12:13:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-02-17T12:13:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-02-17T12:13:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.223 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "upmap", 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "crush-compat", 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "none", 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "read", 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap", 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap-read" 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Balancer mode", 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.224 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_ids": { 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_ids", 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.225 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.226 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_deviation": { 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_deviation", 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "upmap_max_optimizations": { 2024-02-17T12:13:53.227 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "upmap_max_optimizations", 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.228 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm", 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_down_multiplier": { 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_down_multiplier", 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.229 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_refresh_rate": { 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_refresh_rate", 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "20", 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.230 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "agent_starting_port": { 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "agent_starting_port", 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4721", 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.231 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_ptrace": { 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_ptrace", 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-17T12:13:53.232 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_interval": { 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_interval", 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-17T12:13:53.233 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "autotune_memory_target_ratio": { 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "autotune_memory_target_ratio", 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0.7", 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.234 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm_log_destination": { 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cephadm_log_destination", 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "file", 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "file,syslog", 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: "syslog" 2024-02-17T12:13:53.235 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "cgroups_split": { 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cgroups_split", 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.236 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "config_checks_enabled": { 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_checks_enabled", 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.237 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "config_dashboard": { 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_dashboard", 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-17T12:13:53.238 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_alertmanager": { 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_alertmanager", 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.239 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_base": { 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_base", 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Container image name, without the tag", 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.240 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_elasticsearch": { 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_elasticsearch", 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "elasticsearch container image", 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.241 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_grafana": { 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_grafana", 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.12", 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.242 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_haproxy": { 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_haproxy", 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "HAproxy container image", 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.243 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_agent": { 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_agent", 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger agent container image", 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_collector": { 2024-02-17T12:13:53.244 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_collector", 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger collector container image", 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_jaeger_query": { 2024-02-17T12:13:53.245 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_jaeger_query", 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Jaeger query container image", 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_keepalived": { 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_keepalived", 2024-02-17T12:13:53.246 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Keepalived container image", 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_loki": { 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_loki", 2024-02-17T12:13:53.247 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Loki container image", 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_node_exporter": { 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_node_exporter", 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.248 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_nvmeof": { 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_nvmeof", 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.249 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Nvme-of container image", 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_prometheus": { 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_prometheus", 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.250 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Prometheus container image", 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_promtail": { 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_promtail", 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.251 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Promtail container image", 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "container_image_snmp_gateway": { 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_image_snmp_gateway", 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.252 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "SNMP Gateway container image", 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "container_init": { 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "container_init", 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.253 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Run podman/docker with `--init`", 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "daemon_cache_timeout": { 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "daemon_cache_timeout", 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:53.254 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "default_cephadm_command_timeout": { 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_cephadm_command_timeout", 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "900", 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.255 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "default_registry": { 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "default_registry", 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "docker.io", 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.256 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "device_cache_timeout": { 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_cache_timeout", 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1800", 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.257 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache device inventory", 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "device_enhanced_scan": { 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_enhanced_scan", 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.258 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "facts_cache_timeout": { 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "facts_cache_timeout", 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.259 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "seconds to cache host facts data", 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "host_check_interval": { 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "host_check_interval", 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to perform a host check", 2024-02-17T12:13:53.260 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: "hw_monitoring": { 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hw_monitoring", 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-02-17T12:13:53.261 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "inventory_list_all": { 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "inventory_list_all", 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.262 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.263 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: "log_refresh_metadata": { 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_refresh_metadata", 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.264 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.265 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.266 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.267 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-17T12:13:53.268 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-17T12:13:53.269 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "*", 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "max_count_per_host": { 2024-02-17T12:13:53.270 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_count_per_host", 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of daemons per service per host", 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "max_osd_draining_count": { 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_osd_draining_count", 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.271 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: "migration_current": { 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "migration_current", 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.272 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "internal - do not modify", 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: "mode": { 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mode", 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.273 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "root", 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm-package", 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: "root" 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "mode for remote execution of cephadm", 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: "oob_default_addr": { 2024-02-17T12:13:53.274 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "oob_default_addr", 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "169.254.1.1", 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Default address for RedFish API (oob management).", 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: "prometheus_alerts_path": { 2024-02-17T12:13:53.275 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus_alerts_path", 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_insecure": { 2024-02-17T12:13:53.276 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_insecure", 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_password": { 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_password", 2024-02-17T12:13:53.277 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_url": { 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_url", 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.278 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: "registry_username": { 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "registry_username", 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.279 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: "secure_monitoring_stack": { 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "secure_monitoring_stack", 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.280 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "service_discovery_port": { 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "service_discovery_port", 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.281 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8765", 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "cephadm service discovery port", 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "ssh_config_file": { 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssh_config_file", 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.282 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "use_agent": { 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_agent", 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.283 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "use_repo_digest": { 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "use_repo_digest", 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.284 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_failed_host_check": { 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_failed_host_check", 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.285 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if the host check fails", 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_daemons": { 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_daemons", 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.286 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_on_stray_hosts": { 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_on_stray_hosts", 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.287 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crash", 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.288 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.289 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.290 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.291 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.292 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.292 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.292 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.292 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.292 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.292 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.292 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.292 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.292 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.292 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.293 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.293 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.293 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.293 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.293 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.293 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.293 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.293 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.293 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.294 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.294 INFO:teuthology.orchestra.run.smithi094.stdout: "retain_interval": { 2024-02-17T12:13:53.294 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retain_interval", 2024-02-17T12:13:53.294 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.294 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.294 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.294 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "31536000", 2024-02-17T12:13:53.294 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.294 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.294 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.294 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-17T12:13:53.294 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.294 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.295 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.295 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.295 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_recent_interval": { 2024-02-17T12:13:53.295 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_recent_interval", 2024-02-17T12:13:53.295 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.295 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.295 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.295 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1209600", 2024-02-17T12:13:53.295 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.295 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.295 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.295 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-17T12:13:53.295 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dashboard", 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.296 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_HOST": { 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.297 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.298 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_ENABLED": { 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_ENABLED", 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.299 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.300 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: "ENABLE_BROWSABLE_API": { 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.301 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.302 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.303 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.304 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.304 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.304 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.304 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.304 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-17T12:13:53.304 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-17T12:13:53.304 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.304 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.304 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.304 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.304 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.304 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.304 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.305 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.306 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.307 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.308 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-17T12:13:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-17T12:13:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.309 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_PASSWORD": { 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.310 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.311 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_URL": { 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_URL", 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.312 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_API_USERNAME": { 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.313 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.314 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.315 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.316 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.317 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_HOST": { 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.318 INFO:teuthology.orchestra.run.smithi094.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.319 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.320 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-17T12:13:53.321 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-17T12:13:53.322 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-17T12:13:53.323 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-17T12:13:53.324 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-17T12:13:53.325 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_ENABLED": { 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.326 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.327 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.328 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.329 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8", 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.330 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "45", 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ACCESS_KEY": { 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.331 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.332 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "admin", 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SECRET_KEY": { 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.333 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: "RGW_API_SSL_VERIFY": { 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.334 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.335 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.336 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.337 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: "cross_origin_url": { 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cross_origin_url", 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.338 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "crt_file": { 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "crt_file", 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.339 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "debug": { 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "debug", 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Enable/disable debug options", 2024-02-17T12:13:53.340 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: "jwt_token_ttl": { 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "jwt_token_ttl", 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "28800", 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.341 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.342 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.343 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.344 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.345 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.346 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "motd": { 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "motd", 2024-02-17T12:13:53.347 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "The message of the day", 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect_resolve_ip_addr": { 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "redirect_resolve_ip_addr", 2024-02-17T12:13:53.348 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.349 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.350 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8080", 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.351 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl_server_port": { 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl_server_port", 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.352 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8443", 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.353 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "redirect", 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: "redirect" 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-02-17T12:13:53.354 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: "url_prefix": { 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url_prefix", 2024-02-17T12:13:53.355 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.356 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_monitoring": { 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_monitoring", 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.357 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "monitor device health metrics", 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.358 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.359 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.360 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.361 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.362 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: "mark_out_threshold": { 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mark_out_threshold", 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "2419200", 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.363 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: "pool_name": { 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pool_name", 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "device_health_metrics", 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.364 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "retention_period": { 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "retention_period", 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15552000", 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long to retain device health metrics", 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.365 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_frequency": { 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_frequency", 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.366 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: "self_heal": { 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "self_heal", 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.367 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how frequently to wake up and check device health", 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.368 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: "warn_threshold": { 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "warn_threshold", 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7257600", 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.369 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "diskprediction_local", 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.370 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.371 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.372 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.373 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.374 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "predict_interval": { 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predict_interval", 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "86400", 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.375 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: "predictor_model": { 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "predictor_model", 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "prophetstor", 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.376 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "600", 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.377 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "influx", 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": false, 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "influxdb python module not found", 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: "batch_size": { 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "batch_size", 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5000", 2024-02-17T12:13:53.378 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "database": { 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "database", 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "ceph", 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.379 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "hostname": { 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "hostname", 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.380 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server hostname", 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "30", 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "5", 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.381 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-17T12:13:53.382 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.382 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.382 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.382 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.382 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.382 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.382 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.382 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.382 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.382 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.382 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.382 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.383 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.384 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.385 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.386 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.386 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.386 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.386 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.386 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.386 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.386 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.386 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.386 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.386 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.386 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.386 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.386 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "password": { 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "password", 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.387 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "password of InfluxDB server user", 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "port": { 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "port", 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "8086", 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.388 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "InfluxDB server port", 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "ssl": { 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ssl", 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "false", 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-17T12:13:53.389 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: "threads": { 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threads", 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "32", 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-17T12:13:53.390 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "username": { 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "username", 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "username of InfluxDB server user", 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.391 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.392 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.392 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.392 INFO:teuthology.orchestra.run.smithi094.stdout: "verify_ssl": { 2024-02-17T12:13:53.392 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "verify_ssl", 2024-02-17T12:13:53.392 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.392 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.392 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.392 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "true", 2024-02-17T12:13:53.392 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.392 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.392 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.392 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-17T12:13:53.392 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "insights", 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.393 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.394 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.395 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.396 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.397 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "iostat", 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.398 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.399 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.399 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.399 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.399 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.399 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.399 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.399 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.399 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.399 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.399 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.399 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.399 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.399 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.400 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.401 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.402 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.402 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.402 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.402 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.402 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.402 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.402 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.402 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.402 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.402 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.402 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.402 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.402 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.403 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "k8sevents", 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: "ceph_event_retention_days": { 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "ceph_event_retention_days", 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "7", 2024-02-17T12:13:53.404 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "config_check_secs": { 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "config_check_secs", 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "10", 2024-02-17T12:13:53.405 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.406 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.407 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.408 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.409 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.410 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "localpool", 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: "failure_domain": { 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "failure_domain", 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.411 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "host", 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "failure domain for any created local pool", 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.412 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.413 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.414 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.415 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.416 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "min_size": { 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "min_size", 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.417 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default min_size for any created local pool", 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "num_rep": { 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "num_rep", 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3", 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.418 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default replica count for any created local pool", 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_num": { 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_num", 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "128", 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "default pg_num for any created local pool", 2024-02-17T12:13:53.419 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: "prefix": { 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prefix", 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "name prefix for any created local pool", 2024-02-17T12:13:53.420 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "subtree": { 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "subtree", 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "rack", 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.421 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mds_autoscaler", 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.422 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.423 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.424 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.425 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.426 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirroring", 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.427 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.428 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.428 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.428 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.428 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.428 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.428 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.428 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.428 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.428 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.428 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.428 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.428 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.428 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.429 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.430 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.431 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.432 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "nfs", 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.433 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.434 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.435 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.436 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.437 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: "fail_fs": { 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "fail_fs", 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.438 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.439 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.440 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.441 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.442 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.443 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator": { 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "orchestrator", 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.444 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: "cephadm", 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: "rook", 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: "test_orchestrator" 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Orchestrator backend", 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_perf_query", 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.445 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.446 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.447 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.448 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.448 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.448 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.448 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.448 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.448 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.448 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.235+0000 7f35eba5c640 1 -- 172.21.15.94:0/2552144017 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35d00a2230 msgr2=0x7f35d00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:53.449 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.235+0000 7f35eba5c640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35d00a2230 0x7f35d00a46f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009b40 tx=0x7f35d4005e50 comp rx=0 tx=0).stop 2024-02-17T12:13:53.449 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.449 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.449 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.449 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.449 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.449 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.449 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.449 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.449 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.450 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.451 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "osd_support", 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.452 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.453 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.454 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.455 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.456 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "pg_autoscaler", 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.457 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.458 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.459 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.460 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.461 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.462 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "threshold": { 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "threshold", 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "3.0", 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1.0", 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "scaling threshold", 2024-02-17T12:13:53.463 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "progress", 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_pg_recovery_event": { 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_pg_recovery_event", 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.464 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.465 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.466 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.467 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.468 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.468 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.468 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.468 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.468 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.468 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.468 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.468 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.468 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.468 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.468 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.468 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.468 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.469 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.470 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "max_completed_events": { 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_completed_events", 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "50", 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.471 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "number of past completed events to remember", 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: "sleep_interval": { 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "sleep_interval", 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "5", 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.472 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "how long the module is going to sleep", 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "prometheus", 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: "cache": { 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "cache", 2024-02-17T12:13:53.473 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "exclude_perf_counters": { 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "exclude_perf_counters", 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.474 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.475 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.476 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.477 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.478 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.479 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools": { 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools", 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.480 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300", 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.481 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "scrape_interval": { 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "scrape_interval", 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15.0", 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.482 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "::", 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.483 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "9283", 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.484 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: "stale_cache_strategy": { 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stale_cache_strategy", 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "log", 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.485 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_behaviour": { 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_behaviour", 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "default", 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: "default", 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: "error" 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.486 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "standby_error_status_code": { 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "standby_error_status_code", 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "500", 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "400", 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "599", 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.487 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.488 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.488 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.488 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.488 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.488 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.488 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.488 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-02-17T12:13:53.488 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.488 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.488 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.488 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.488 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.489 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.489 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.489 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.489 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.489 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.489 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.489 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.489 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.489 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.490 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.491 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.492 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.493 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_snap_create": { 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_snap_create", 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10", 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.494 INFO:teuthology.orchestra.run.smithi094.stdout: "mirror_snapshot_schedule": { 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "mirror_snapshot_schedule", 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "trash_purge_schedule": { 2024-02-17T12:13:53.495 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "trash_purge_schedule", 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.496 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "restful", 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "enable_auth": { 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enable_auth", 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.497 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: "key_file": { 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "key_file", 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.498 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.499 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.500 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.501 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.502 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.503 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "server_addr": { 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_addr", 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.504 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: "server_port": { 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "server_port", 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.505 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rgw", 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.506 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.507 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.508 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.509 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.510 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rook", 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: "drive_group_interval": { 2024-02-17T12:13:53.511 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "drive_group_interval", 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "300.0", 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.512 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.513 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.514 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.514 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.514 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.514 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.514 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.235+0000 7f35eba5c640 1 -- 172.21.15.94:0/2552144017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4106480 msgr2=0x7f35e419db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:13:53.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.235+0000 7f35eba5c640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4106480 0x7f35e419db10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009230 tx=0x7f35e0004270 comp rx=0 tx=0).stop 2024-02-17T12:13:53.514 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.235+0000 7f35eba5c640 1 -- 172.21.15.94:0/2552144017 shutdown_connections 2024-02-17T12:13:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.235+0000 7f35eba5c640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f35d00a2230 0x7f35d00a46f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.235+0000 7f35eba5c640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f35e41976e0 0x7f35e4197b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.235+0000 7f35eba5c640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f35e4106480 0x7f35e419db10 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.235+0000 7f35eba5c640 1 --2- 172.21.15.94:0/2552144017 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f35e41013b0 0x7f35e419d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:13:53.515 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.235+0000 7f35eba5c640 1 -- 172.21.15.94:0/2552144017 >> 172.21.15.94:0/2552144017 conn(0x7f35e40fcc30 msgr2=0x7f35e40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:13:53.515 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.515 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.515 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.515 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.515 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.515 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.516 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.516 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.516 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.516 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.516 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.516 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.516 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.516 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.516 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.516 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.516 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.516 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.516 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.517 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.518 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: "storage_class": { 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "storage_class", 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "local", 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.519 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "selftest", 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.520 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.521 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.522 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.523 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.524 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: "roption1": { 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption1", 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.525 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "roption2": { 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "roption2", 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "xyz", 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.526 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption1": { 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption1", 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.527 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption2": { 2024-02-17T12:13:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption2", 2024-02-17T12:13:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.528 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.529 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.529 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.529 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.529 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption3": { 2024-02-17T12:13:53.529 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption3", 2024-02-17T12:13:53.529 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "float", 2024-02-17T12:13:53.529 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.529 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.529 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.529 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.529 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.529 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.529 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption4": { 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption4", 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.530 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption5": { 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption5", 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.531 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption6": { 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption6", 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.532 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: "rwoption7": { 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rwoption7", 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "1", 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "42", 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.533 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "testkey": { 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testkey", 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.534 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: "testlkey": { 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testlkey", 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.535 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "testnewline": { 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "testnewline", 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.536 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: "allow_m_granularity": { 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "allow_m_granularity", 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.537 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "allow minute scheduled snapshots", 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "dump_on_update": { 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "dump_on_update", 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.538 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "dump database to debug log on update", 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.539 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.540 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.541 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.542 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.543 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "stats", 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.544 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.545 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.546 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.547 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.548 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "status", 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.549 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.550 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.550 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.550 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.550 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.550 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.550 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.550 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.550 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.550 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.550 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.550 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.550 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.550 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.551 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.552 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.553 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.553 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.553 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.553 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.553 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.553 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.553 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.553 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.553 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.553 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.553 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.553 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.553 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.554 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telegraf", 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: "address": { 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "address", 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.555 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.556 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "15", 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.557 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.558 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.559 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.560 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.561 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "telemetry", 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_basic": { 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_basic", 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.562 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_crash": { 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_crash", 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.563 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_device": { 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_device", 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.564 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "True", 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_ident": { 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_ident", 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.565 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: "channel_perf": { 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "channel_perf", 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.566 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: "contact": { 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "contact", 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.567 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: "description": { 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "description", 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.568 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: "device_url": { 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "device_url", 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.569 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: "enabled": { 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "enabled", 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.570 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.571 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "24", 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "8", 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: "last_opt_revision": { 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "last_opt_revision", 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.572 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "1", 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard": { 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard", 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.573 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "leaderboard_description": { 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "leaderboard_description", 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.574 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.575 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.576 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.576 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.576 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.576 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.576 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.576 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.576 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.576 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.576 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.576 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.576 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.576 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.576 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.577 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.577 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.577 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.577 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.577 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.577 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.577 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.577 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.577 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.577 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.577 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.577 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.577 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.578 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.579 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: "organization": { 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "organization", 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.580 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.581 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.581 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.581 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.581 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.581 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.581 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.581 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.581 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.581 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.581 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.581 INFO:teuthology.orchestra.run.smithi094.stdout: "proxy": { 2024-02-17T12:13:53.581 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "proxy", 2024-02-17T12:13:53.581 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.582 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.582 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.582 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.582 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.582 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.235+0000 7f35eba5c640 1 -- 172.21.15.94:0/2552144017 shutdown_connections 2024-02-17T12:13:53.582 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:13:52.235+0000 7f35eba5c640 1 -- 172.21.15.94:0/2552144017 wait complete. 2024-02-17T12:13:53.583 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.583 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.583 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.583 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.584 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.584 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.584 INFO:teuthology.orchestra.run.smithi094.stdout: "url": { 2024-02-17T12:13:53.584 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "url", 2024-02-17T12:13:53.584 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.584 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.584 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.584 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-17T12:13:53.584 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.584 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.584 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.584 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.584 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "test_orchestrator", 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.585 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.586 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.587 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.588 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.589 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.590 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.591 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.592 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.593 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.594 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "max_concurrent_clones": { 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "max_concurrent_clones", 2024-02-17T12:13:53.595 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "4", 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: "periodic_async_work": { 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "periodic_async_work", 2024-02-17T12:13:53.596 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Periodically check for async work", 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "snapshot_clone_delay": { 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snapshot_clone_delay", 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.597 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "0", 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix", 2024-02-17T12:13:53.598 INFO:teuthology.orchestra.run.smithi094.stdout: "can_run": true, 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "error_string": "", 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "module_options": { 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "discovery_interval": { 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "discovery_interval", 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "uint", 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "100", 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.599 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: "identifier": { 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "identifier", 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.600 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "interval": { 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "interval", 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "secs", 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "60", 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.601 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "log_level": { 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_level", 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.602 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster": { 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster", 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.603 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_cluster_level": { 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_cluster_level", 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "info", 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.604 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [ 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: "", 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: "critical", 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: "debug", 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: "error", 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: "info", 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: "warning" 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: "log_to_file": { 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "log_to_file", 2024-02-17T12:13:53.605 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "bool", 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 1, 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "False", 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_host": { 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_host", 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.606 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "", 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_port": { 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_port", 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "int", 2024-02-17T12:13:53.607 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "10051", 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: "zabbix_sender": { 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "zabbix_sender", 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "str", 2024-02-17T12:13:53.608 INFO:teuthology.orchestra.run.smithi094.stdout: "level": "advanced", 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: "flags": 0, 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: "min": "", 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: "max": "", 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: "enum_allowed": [], 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: "desc": "", 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: "long_desc": "", 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: "tags": [], 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: "see_also": [] 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: "services": { 2024-02-17T12:13:53.609 INFO:teuthology.orchestra.run.smithi094.stdout: "dashboard": "https://172.21.15.94:8443/" 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: "always_on_modules": { 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: "octopus": [ 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.610 INFO:teuthology.orchestra.run.smithi094.stdout: "pacific": [ 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "quincy": [ 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-02-17T12:13:53.611 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: "reef": [ 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-02-17T12:13:53.612 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: ], 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "squid": [ 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "balancer", 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "crash", 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "devicehealth", 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "orchestrator", 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "pg_autoscaler", 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "progress", 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "rbd_support", 2024-02-17T12:13:53.613 INFO:teuthology.orchestra.run.smithi094.stdout: "status", 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: "telemetry", 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: "volumes" 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: "last_failure_osd_epoch": 88, 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: "active_clients": [ 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "devicehealth", 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 658852438 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.614 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "libcephsqlite", 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1021672798 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "rbd_support", 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-02-17T12:13:53.615 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 1230488244 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 3227515104 2024-02-17T12:13:53.616 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "snap_schedule", 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 294633386 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: "name": "volumes", 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: "addrvec": [ 2024-02-17T12:13:53.617 INFO:teuthology.orchestra.run.smithi094.stdout: { 2024-02-17T12:13:53.618 INFO:teuthology.orchestra.run.smithi094.stdout: "type": "v2", 2024-02-17T12:13:53.618 INFO:teuthology.orchestra.run.smithi094.stdout: "addr": "172.21.15.94:0", 2024-02-17T12:13:53.618 INFO:teuthology.orchestra.run.smithi094.stdout: "nonce": 2638374457 2024-02-17T12:13:53.618 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.618 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-02-17T12:13:53.618 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:53.618 INFO:teuthology.orchestra.run.smithi094.stdout: ] 2024-02-17T12:13:53.618 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:13:53.653 DEBUG:tasks.check_counter:Getting stats from a 2024-02-17T12:13:53.653 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph daemon mds.a perf dump 2024-02-17T12:13:53.654 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:52 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1839185720' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:13:53.654 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:52 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2339802852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:13:53.654 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:52 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/4042362016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:13:53.654 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:53 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2552144017' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-17T12:13:53.654 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:53 smithi094 ceph-mon[32499]: pgmap v494: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-02-17T12:13:53.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:53 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2552144017' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-17T12:13:53.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:53 smithi189 ceph-mon[36588]: pgmap v494: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-02-17T12:13:53.654 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:53 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2552144017' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-17T12:13:53.655 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:53 smithi125 ceph-mon[36819]: pgmap v494: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 9 op/s 2024-02-17T12:13:53.934 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:13:55.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:55 smithi125 ceph-mon[36819]: pgmap v495: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-02-17T12:13:55.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:55 smithi094 ceph-mon[32499]: pgmap v495: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-02-17T12:13:55.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:55 smithi189 ceph-mon[36588]: pgmap v495: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 28 KiB/s wr, 9 op/s 2024-02-17T12:13:55.922 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:13:55.922 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 4, 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 4, 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 3024, 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 448, 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.001641036, 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000037413, 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000625144, 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 3, 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000763093, 2024-02-17T12:13:55.923 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000254364 2024-02-17T12:13:55.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:55.924 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:13:55.924 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:13:55.924 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:13:55.924 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:13:55.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:55.924 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-17T12:13:55.924 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 448 2024-02-17T12:13:55.924 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:55.924 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T12:13:55.924 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 3, 2024-02-17T12:13:55.924 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 222, 2024-02-17T12:13:55.924 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 480, 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 271536, 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.015655989, 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.010785831, 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.002418754, 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 219, 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.016259331, 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000074243 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:13:55.925 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 480, 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 271536 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 303, 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 291, 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1729328, 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 174752, 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.054070700, 2024-02-17T12:13:55.926 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.015986450, 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.042468842, 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.006126132, 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 290, 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.013677116, 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000047162 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1729328, 2024-02-17T12:13:55.927 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 174752 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T12:13:55.928 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T12:13:55.929 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-02-17T12:13:55.930 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148714, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 30, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-02-17T12:13:55.931 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-02-17T12:13:55.932 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-02-17T12:13:55.932 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-02-17T12:13:55.932 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-02-17T12:13:55.932 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-02-17T12:13:55.932 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-02-17T12:13:55.932 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-02-17T12:13:55.932 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-02-17T12:13:55.932 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:55.932 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T12:13:55.932 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 306, 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1686587, 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 306, 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 306, 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1686587, 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:55.933 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:55.934 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:13:56.715 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-02-17T12:13:56.715 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-02-17T12:13:56.715 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-02-17T12:13:56.715 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-02-17T12:13:56.716 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-02-17T12:13:56.716 DEBUG:tasks.check_counter:Getting stats from d 2024-02-17T12:13:56.716 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph daemon mds.d perf dump 2024-02-17T12:13:56.973 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:13:57.390 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:57 smithi094 ceph-mon[32499]: pgmap v496: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-02-17T12:13:57.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:57 smithi125 ceph-mon[36819]: pgmap v496: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-02-17T12:13:57.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:57 smithi189 ceph-mon[36588]: pgmap v496: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 12 KiB/s wr, 8 op/s 2024-02-17T12:13:58.893 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:13:58.893 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T12:13:58.893 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 309, 2024-02-17T12:13:58.893 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 297, 2024-02-17T12:13:58.893 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1730960, 2024-02-17T12:13:58.893 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 178496, 2024-02-17T12:13:58.893 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-02-17T12:13:58.893 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.055548474, 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.016777081, 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.041668365, 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.006166629, 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 296, 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.013359779, 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000045134 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:58.894 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1730960, 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 178496 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 7, 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 230, 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 3504, 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 273040, 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.015591299, 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.010731173, 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.002311046, 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T12:13:58.895 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 228, 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.022381002, 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000098162 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 273040 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T12:13:58.896 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 0, 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2, 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 0, 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 0, 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.000256744, 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000000000, 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000190407, 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:58.897 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T12:13:58.898 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-02-17T12:13:58.899 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-02-17T12:13:58.900 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T12:13:58.901 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-02-17T12:13:58.901 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T12:13:58.901 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-02-17T12:13:58.901 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148692, 2024-02-17T12:13:58.901 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 30, 2024-02-17T12:13:58.901 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-02-17T12:13:58.901 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-02-17T12:13:58.901 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-02-17T12:13:58.901 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-02-17T12:13:58.901 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-02-17T12:13:58.901 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-02-17T12:13:58.901 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-02-17T12:13:58.901 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-02-17T12:13:58.902 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 312, 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1687367, 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 312, 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 312, 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1687367, 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:13:58.903 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:13:59.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:13:59 smithi125 ceph-mon[36819]: pgmap v497: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-02-17T12:13:59.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:13:59 smithi094 ceph-mon[32499]: pgmap v497: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-02-17T12:13:59.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:13:59 smithi189 ceph-mon[36588]: pgmap v497: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 10 op/s 2024-02-17T12:13:59.841 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-02-17T12:13:59.841 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-02-17T12:13:59.841 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-02-17T12:13:59.841 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-02-17T12:13:59.841 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-02-17T12:13:59.841 DEBUG:tasks.check_counter:Getting stats from g 2024-02-17T12:13:59.842 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph daemon mds.g perf dump 2024-02-17T12:14:00.092 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:14:01.431 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:01 smithi094 ceph-mon[32499]: pgmap v498: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-02-17T12:14:01.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:01 smithi125 ceph-mon[36819]: pgmap v498: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-02-17T12:14:01.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:01 smithi189 ceph-mon[36588]: pgmap v498: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-02-17T12:14:02.004 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:14:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T12:14:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 309, 2024-02-17T12:14:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 295, 2024-02-17T12:14:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1732896, 2024-02-17T12:14:02.004 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 176448, 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.052817266, 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.016239153, 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.039925608, 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.006211383, 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 295, 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.012490722, 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000042341 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:14:02.005 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1732896, 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 176448 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 3, 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 224, 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 480, 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 271888, 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.016875571, 2024-02-17T12:14:02.006 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.011779568, 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.002236335, 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 221, 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.013195874, 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000059709 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 480, 2024-02-17T12:14:02.007 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 271888 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 0, 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2, 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 0, 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 0, 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.000458677, 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000000000, 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000371258, 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:02.008 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-02-17T12:14:02.009 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T12:14:02.010 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-02-17T12:14:02.011 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T12:14:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T12:14:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-02-17T12:14:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-02-17T12:14:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-02-17T12:14:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-02-17T12:14:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T12:14:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-02-17T12:14:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T12:14:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-02-17T12:14:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148714, 2024-02-17T12:14:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 30, 2024-02-17T12:14:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-02-17T12:14:02.012 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-02-17T12:14:02.013 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 308, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1686847, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 308, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 308, 2024-02-17T12:14:02.014 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1686847, 2024-02-17T12:14:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-02-17T12:14:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:14:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:14:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:14:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:14:02.015 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:14:02.015 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:14:02.704 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-02-17T12:14:02.704 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-02-17T12:14:02.704 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-02-17T12:14:02.704 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-02-17T12:14:02.705 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-02-17T12:14:02.705 DEBUG:tasks.check_counter:Getting stats from j 2024-02-17T12:14:02.705 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph daemon mds.j perf dump 2024-02-17T12:14:02.950 INFO:teuthology.orchestra.run.smithi094.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.a/config 2024-02-17T12:14:03.356 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:03 smithi094 ceph-mon[32499]: pgmap v499: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-02-17T12:14:03.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:03 smithi125 ceph-mon[36819]: pgmap v499: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-02-17T12:14:03.656 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:03 smithi189 ceph-mon[36588]: pgmap v499: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 26 KiB/s wr, 21 op/s 2024-02-17T12:14:04.732 INFO:teuthology.orchestra.run.smithi094.stdout:{ 2024-02-17T12:14:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T12:14:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 0, 2024-02-17T12:14:04.732 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 2, 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 0, 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 0, 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 4, 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 0, 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.001047627, 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.000000000, 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.000170914, 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:04.733 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 7, 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 231, 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 3504, 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 273216, 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 6, 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 3, 2024-02-17T12:14:04.734 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.016226046, 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.011403550, 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.002272255, 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 229, 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.016956042, 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000074043 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:04.735 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-02-17T12:14:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 273216 2024-02-17T12:14:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T12:14:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_messages": 311, 2024-02-17T12:14:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages": 299, 2024-02-17T12:14:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_bytes": 1731504, 2024-02-17T12:14:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_bytes": 179744, 2024-02-17T12:14:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_created_connections": 2, 2024-02-17T12:14:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_active_connections": 1, 2024-02-17T12:14:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_total_time": 0.057294563, 2024-02-17T12:14:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_send_time": 0.018332386, 2024-02-17T12:14:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_recv_time": 0.042087756, 2024-02-17T12:14:04.736 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_running_fast_dispatch_time": 0.006248811, 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 298, 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.013604114, 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000045651 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_recv_encrypted_bytes": 1731504, 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "msgr_send_encrypted_bytes": 179744 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:04.737 INFO:teuthology.orchestra.run.smithi094.stdout: "cct": { 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "total_workers": 0, 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "unhealthy_workers": 0 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "mempool": { 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_bytes": 0, 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "bloom_filter_items": 0, 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_alloc_items": 0, 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_data_items": 0, 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T12:14:04.738 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_other_items": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_bytes": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_extent_items": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_bytes": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_blob_items": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_fsck_items": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_bytes": 0, 2024-02-17T12:14:04.739 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_txc_items": 0, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_bytes": 0, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "bluestore_writing_items": 0, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_bytes": 0, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_items": 0, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_reader_items": 0, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "bluefs_file_writer_items": 0, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_bytes": 148670, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_anon_items": 30, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_bytes": 0, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "buffer_meta_items": 0, 2024-02-17T12:14:04.740 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_bytes": 0, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_items": 0, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_bytes": 0, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_mapbl_items": 0, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_bytes": 0, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osd_pglog_items": 0, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_bytes": 0, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_items": 0, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "osdmap_mapping_items": 0, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_bytes": 0, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "pgmap_items": 0, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_bytes": 928, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "mds_co_items": 2, 2024-02-17T12:14:04.741 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_bytes": 0, 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_1_items": 0, 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_bytes": 0, 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "unittest_2_items": 0 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: }, 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "val": 0, 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "max": 104857600, 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "get_started": 0, 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "get": 314, 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "get_sum": 1687627, 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "get_or_fail_success": 314, 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "take": 0, 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "take_sum": 0, 2024-02-17T12:14:04.742 INFO:teuthology.orchestra.run.smithi094.stdout: "put": 314, 2024-02-17T12:14:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "put_sum": 1687627, 2024-02-17T12:14:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "wait": { 2024-02-17T12:14:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "avgcount": 0, 2024-02-17T12:14:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "sum": 0.000000000, 2024-02-17T12:14:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: "avgtime": 0.000000000 2024-02-17T12:14:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:14:04.743 INFO:teuthology.orchestra.run.smithi094.stdout: } 2024-02-17T12:14:04.743 INFO:teuthology.orchestra.run.smithi094.stdout:} 2024-02-17T12:14:05.373 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:05 smithi094 ceph-mon[32499]: pgmap v500: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 KiB/s wr, 24 op/s 2024-02-17T12:14:05.402 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-02-17T12:14:05.402 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-02-17T12:14:05.402 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-02-17T12:14:05.403 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-02-17T12:14:05.403 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-02-17T12:14:05.403 DEBUG:tasks.check_counter:Getting stats from b 2024-02-17T12:14:05.403 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph daemon mds.b perf dump 2024-02-17T12:14:05.433 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:05 smithi125 ceph-mon[36819]: pgmap v500: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 KiB/s wr, 24 op/s 2024-02-17T12:14:05.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:05 smithi189 ceph-mon[36588]: pgmap v500: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 KiB/s wr, 24 op/s 2024-02-17T12:14:05.661 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T12:14:07.438 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:07 smithi125 ceph-mon[36819]: pgmap v501: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-02-17T12:14:07.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:07 smithi094 ceph-mon[32499]: pgmap v501: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-02-17T12:14:07.618 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2024-02-17T12:14:07.618 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T12:14:07.618 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 16213, 2024-02-17T12:14:07.618 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 15982, 2024-02-17T12:14:07.618 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 8065814, 2024-02-17T12:14:07.618 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 38905638, 2024-02-17T12:14:07.618 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 363, 2024-02-17T12:14:07.618 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 5, 2024-02-17T12:14:07.618 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 1.024619989, 2024-02-17T12:14:07.618 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.345843803, 2024-02-17T12:14:07.619 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.307860132, 2024-02-17T12:14:07.619 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.285574890, 2024-02-17T12:14:07.619 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:07.619 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 15981, 2024-02-17T12:14:07.619 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 1.083905646, 2024-02-17T12:14:07.619 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000067824 2024-02-17T12:14:07.619 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.619 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:07.619 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 1769, 2024-02-17T12:14:07.619 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000302004, 2024-02-17T12:14:07.619 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000170 2024-02-17T12:14:07.619 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.619 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 50048, 2024-02-17T12:14:07.620 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 4640 2024-02-17T12:14:07.620 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.620 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T12:14:07.620 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 23980, 2024-02-17T12:14:07.620 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 26251, 2024-02-17T12:14:07.620 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 10870747, 2024-02-17T12:14:07.620 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 30763722, 2024-02-17T12:14:07.620 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 19, 2024-02-17T12:14:07.620 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 6, 2024-02-17T12:14:07.620 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 1.264418161, 2024-02-17T12:14:07.620 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.469411834, 2024-02-17T12:14:07.621 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 1.653408626, 2024-02-17T12:14:07.621 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.274642129, 2024-02-17T12:14:07.621 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:07.621 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 26248, 2024-02-17T12:14:07.621 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 1.643556447, 2024-02-17T12:14:07.621 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000062616 2024-02-17T12:14:07.621 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.621 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:07.621 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 21132, 2024-02-17T12:14:07.621 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.002610531, 2024-02-17T12:14:07.621 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000123 2024-02-17T12:14:07.621 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.621 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 3673248, 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 7803520 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 12255, 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 11638, 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 7752379, 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 9508637, 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 24, 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 7, 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.755853367, 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.240981016, 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.250211030, 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.228228678, 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:07.622 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 11637, 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.646194463, 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000055529 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 3422, 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000593838, 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000173 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 2294368, 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 385552 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: "cct": { 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: "total_workers": 1, 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: "unhealthy_workers": 0 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.623 INFO:teuthology.orchestra.run.smithi125.stdout: "finisher-MDSRank": { 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: "queue_len": 0, 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: "complete_latency": { 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 29348, 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 8.394644666, 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000286038 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: "finisher-PurgeQueue": { 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: "queue_len": 0, 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: "complete_latency": { 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 896, 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.032110389, 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000035837 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:07.624 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": { 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: "request": 4254, 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: "reply": 4254, 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: "reply_latency": { 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 4254, 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 63.804284336, 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.014998656 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: "slow_reply": 0, 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: "forward": 0, 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_fetch_complete": 0, 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_fetch_keys": 0, 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_commit": 1490, 2024-02-17T12:14:07.625 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_split": 4, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_merge": 0, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes": 1663, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_top": 383, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_bottom": 1280, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_pin_tail": 0, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_pinned": 1116, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_expired": 0, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_with_caps": 1054, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": 1055, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "subtrees": 2, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse": 7913, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_hit": 6380, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_forward": 0, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_discover": 0, 2024-02-17T12:14:07.626 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_dir_fetch": 0, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_remote_ino": 0, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_lock": 461, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "load_cent": 0, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "q": 0, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "exported": 0, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "exported_inodes": 0, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "imported": 0, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "imported_inodes": 0, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_dir_fetch": 0, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_backtrace_fetch": 29709, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_peer_discover": 0, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rfiles": 1026, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rbytes": 1732289339, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rsnaps": 7, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_backtrace_fetch": 29709, 2024-02-17T12:14:07.627 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_set_tag": 0, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_inotable_repaired": 0, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dir_inodes": 1015, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dir_base_inodes": 29, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dirfrag_rstats": 986, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_file_inodes": 28694, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_inode_file_caps": 0, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_revoke": 991, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_grant": 3043, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_flushsnap_ack": 1, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_caps": 4652, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_caps_dirty": 2331, 2024-02-17T12:14:07.628 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_cap_release": 183, 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "process_request_cap_release": 1247 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_cache": { 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_update": 0, 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_update_receipt": 0, 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_try_discover": 0, 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_send_discover": 0, 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_handle_discover": 4, 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays": 0, 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays_delayed": 0, 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays_enqueuing": 0, 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_created": 514, 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_enqueued": 276, 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_reintegrated": 238, 2024-02-17T12:14:07.629 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_migrated": 0, 2024-02-17T12:14:07.630 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_processing": 0, 2024-02-17T12:14:07.630 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_enqueued": 0, 2024-02-17T12:14:07.630 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_prioritized": 0, 2024-02-17T12:14:07.630 INFO:teuthology.orchestra.run.smithi125.stdout: "recovery_started": 0, 2024-02-17T12:14:07.630 INFO:teuthology.orchestra.run.smithi125.stdout: "recovery_completed": 0, 2024-02-17T12:14:07.630 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_enqueue_scrub": 29, 2024-02-17T12:14:07.630 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_exportdir": 0, 2024-02-17T12:14:07.630 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_flush": 0, 2024-02-17T12:14:07.630 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_fragmentdir": 4, 2024-02-17T12:14:07.630 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_fragstats": 0, 2024-02-17T12:14:07.630 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_inodestats": 0 2024-02-17T12:14:07.630 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.630 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_log": { 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "evlrg": 0, 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "evadd": 8314, 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "evex": 8160, 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "evtrm": 8160, 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "ev": 154, 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "evexg": 0, 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "evexd": 305, 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "segadd": 83, 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "segex": 82, 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "segtrm": 82, 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "seg": 1, 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "segmjr": 1, 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "segexg": 0, 2024-02-17T12:14:07.631 INFO:teuthology.orchestra.run.smithi125.stdout: "segexd": 2, 2024-02-17T12:14:07.632 INFO:teuthology.orchestra.run.smithi125.stdout: "expos": 51226223, 2024-02-17T12:14:07.632 INFO:teuthology.orchestra.run.smithi125.stdout: "wrpos": 52443690, 2024-02-17T12:14:07.632 INFO:teuthology.orchestra.run.smithi125.stdout: "rdpos": 0, 2024-02-17T12:14:07.632 INFO:teuthology.orchestra.run.smithi125.stdout: "jlat": { 2024-02-17T12:14:07.632 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 3658, 2024-02-17T12:14:07.632 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 43.892644835, 2024-02-17T12:14:07.632 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.011999082 2024-02-17T12:14:07.632 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.632 INFO:teuthology.orchestra.run.smithi125.stdout: "replayed": 0 2024-02-17T12:14:07.632 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.632 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_mem": { 2024-02-17T12:14:07.632 INFO:teuthology.orchestra.run.smithi125.stdout: "ino": 1522, 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: "ino+": 1827, 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: "ino-": 305, 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: "dir": 75, 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: "dir+": 81, 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: "dir-": 6, 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: "dn": 1663, 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: "dn+": 2688, 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: "dn-": 1025, 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: "cap": 1055, 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: "cap+": 1527, 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: "cap-": 472, 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: "rss": 75328, 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: "heap": 215420 2024-02-17T12:14:07.633 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_server": { 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: "dispatch_client_request": 5843, 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: "dispatch_server_request": 1, 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_request": 4254, 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_session": 525, 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_peer_request": 2, 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: "req_create_latency": { 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 1300, 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 52.461280014, 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.040354830 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getattr_latency": { 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 200, 2024-02-17T12:14:07.634 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 5.389395496, 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.026946977 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getfilelock_latency": { 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: "req_link_latency": { 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 238, 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.476164252, 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.002000690 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookup_latency": { 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 1040, 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 1.038815920, 2024-02-17T12:14:07.635 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000998861 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookuphash_latency": { 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupino_latency": { 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 2, 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000190449, 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000095224 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupname_latency": { 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.636 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupparent_latency": { 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupsnap_latency": { 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lssnap_latency": { 2024-02-17T12:14:07.637 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 16, 2024-02-17T12:14:07.638 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.002813259, 2024-02-17T12:14:07.638 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000175828 2024-02-17T12:14:07.638 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.638 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mkdir_latency": { 2024-02-17T12:14:07.638 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 37, 2024-02-17T12:14:07.638 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.081210783, 2024-02-17T12:14:07.638 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.002194886 2024-02-17T12:14:07.638 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.638 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mknod_latency": { 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mksnap_latency": { 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 16, 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.115788678, 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.007236792 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: "req_open_latency": { 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 512, 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.431311839, 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000842405 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.639 INFO:teuthology.orchestra.run.smithi125.stdout: "req_readdir_latency": { 2024-02-17T12:14:07.640 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 109, 2024-02-17T12:14:07.640 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.179665661, 2024-02-17T12:14:07.640 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.001648308 2024-02-17T12:14:07.640 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.640 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rename_latency": { 2024-02-17T12:14:07.640 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 476, 2024-02-17T12:14:07.640 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 3.071957178, 2024-02-17T12:14:07.640 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.006453691 2024-02-17T12:14:07.640 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.640 INFO:teuthology.orchestra.run.smithi125.stdout: "req_renamesnap_latency": { 2024-02-17T12:14:07.640 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.640 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.640 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: "req_snapdiff_latency": { 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmdir_latency": { 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 2, 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.008617901, 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.004308950 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmsnap_latency": { 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 9, 2024-02-17T12:14:07.641 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.071985948, 2024-02-17T12:14:07.642 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.007998438 2024-02-17T12:14:07.642 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.642 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmxattr_latency": { 2024-02-17T12:14:07.642 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.642 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.642 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.642 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.642 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setattr_latency": { 2024-02-17T12:14:07.642 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 1, 2024-02-17T12:14:07.642 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.001568732, 2024-02-17T12:14:07.642 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.001568732 2024-02-17T12:14:07.642 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.642 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setdirlayout_latency": { 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setfilelock_latency": { 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setlayout_latency": { 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.643 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setxattr_latency": { 2024-02-17T12:14:07.644 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 8, 2024-02-17T12:14:07.644 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.022097942, 2024-02-17T12:14:07.644 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.002762242 2024-02-17T12:14:07.644 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.644 INFO:teuthology.orchestra.run.smithi125.stdout: "req_symlink_latency": { 2024-02-17T12:14:07.644 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.644 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.644 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.644 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.644 INFO:teuthology.orchestra.run.smithi125.stdout: "req_unlink_latency": { 2024-02-17T12:14:07.644 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 288, 2024-02-17T12:14:07.644 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.451420284, 2024-02-17T12:14:07.644 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.001567431 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: "cap_revoke_eviction": 0, 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: "cap_acquisition_throttle": 0, 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getvxattr_latency": { 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_sessions": { 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: "session_count": 2, 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: "session_add": 6, 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: "session_remove": 4, 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: "sessions_open": 2, 2024-02-17T12:14:07.645 INFO:teuthology.orchestra.run.smithi125.stdout: "sessions_stale": 0, 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "total_load": 421, 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "average_load": 210, 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "avg_session_uptime": 941, 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "mdthresh_evicted": 0 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "mempool": { 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_bytes": 0, 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_items": 0, 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_items": 0, 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_items": 0, 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T12:14:07.646 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_items": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_bytes": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_items": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_bytes": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_items": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T12:14:07.647 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_items": 0, 2024-02-17T12:14:07.648 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_bytes": 0, 2024-02-17T12:14:07.648 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_items": 0, 2024-02-17T12:14:07.648 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T12:14:07.648 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T12:14:07.648 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_bytes": 0, 2024-02-17T12:14:07.648 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_items": 0, 2024-02-17T12:14:07.648 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_bytes": 0, 2024-02-17T12:14:07.648 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_items": 0, 2024-02-17T12:14:07.648 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T12:14:07.648 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_items": 0, 2024-02-17T12:14:07.648 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T12:14:07.648 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_items": 0, 2024-02-17T12:14:07.648 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_bytes": 902893, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_items": 1409, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_bytes": 1056, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_items": 12, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_bytes": 0, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_items": 0, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_bytes": 0, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_items": 0, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_bytes": 0, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_items": 0, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_bytes": 8352, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_items": 190, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_items": 0, 2024-02-17T12:14:07.649 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_bytes": 0, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_items": 0, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_bytes": 5429113, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_items": 103351, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_bytes": 0, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_items": 0, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_bytes": 0, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_items": 0 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "objecter": { 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "op_active": 0, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "op_laggy": 0, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "op_send": 38414, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "op_send_bytes": 56115255, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "op_resend": 37, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "op_reply": 38414, 2024-02-17T12:14:07.650 INFO:teuthology.orchestra.run.smithi125.stdout: "op_latency": { 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 38414, 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 243.267872888, 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.006332792 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: "op_inflight": 0, 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: "oplen_avg": { 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 38414, 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 45014 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: "op": 38414, 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: "op_r": 29873, 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: "op_w": 8541, 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: "op_rmw": 0, 2024-02-17T12:14:07.651 INFO:teuthology.orchestra.run.smithi125.stdout: "op_pg": 0, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_stat": 1411, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_create": 1591, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_read": 135, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_write": 3764, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_writefull": 390, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_writesame": 0, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_append": 0, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_zero": 0, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_truncate": 4, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_delete": 824, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_mapext": 0, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_sparse_read": 0, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_clonerange": 0, 2024-02-17T12:14:07.652 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_getxattr": 29738, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_setxattr": 3182, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_cmpxattr": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_rmxattr": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_resetxattrs": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_call": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_watch": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_notify": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_pgls": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_pgls_filter": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_other": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_active": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_send": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_resend": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_ping": 0, 2024-02-17T12:14:07.653 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_active": 0, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_send": 0, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_resend": 0, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_active": 0, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_send": 0, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_resend": 0, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_active": 0, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_send": 0, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_resend": 0, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "command_active": 0, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "command_send": 0, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "command_resend": 0, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "map_epoch": 106, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "map_full": 0, 2024-02-17T12:14:07.654 INFO:teuthology.orchestra.run.smithi125.stdout: "map_inc": 22, 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_sessions": 12, 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_session_open": 12, 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_session_close": 0, 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_laggy": 0, 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_wr": 3644, 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_rd": 0, 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_del": 331 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: "oft": { 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_objs": 1, 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_kv_pairs": 0, 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_updates": 0, 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_removes": 0 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.655 INFO:teuthology.orchestra.run.smithi125.stdout: "osd-slow-ops": { 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: "slow_ops_count": 0 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: "purge_queue": { 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_ops": 0, 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_ops_high_water": 17, 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing": 0, 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_high_water": 10, 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executed_ops": 290, 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executed": 276, 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_item_in_journal": 0 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2024-02-17T12:14:07.656 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 52444, 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 21413564, 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 52444, 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 52444, 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 21413564, 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-objecter_bytes": { 2024-02-17T12:14:07.657 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 38414, 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 56144267, 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 38414, 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 56144267, 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:07.658 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-objecter_ops": { 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 1024, 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 38414, 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 38414, 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 38414, 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 38414, 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.659 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-write_buf_throttle": { 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 3758096384, 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 276, 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 29012, 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 276, 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2024-02-17T12:14:07.660 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 106, 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 29012, 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-write_buf_throttle-0x5619051706e0": { 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 3758096384, 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 8314, 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 48249386, 2024-02-17T12:14:07.661 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:07.662 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 8314, 2024-02-17T12:14:07.662 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2024-02-17T12:14:07.662 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2024-02-17T12:14:07.662 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 3658, 2024-02-17T12:14:07.662 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 48249386, 2024-02-17T12:14:07.662 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:07.662 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:07.662 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:07.662 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:07.662 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:07.662 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:07.662 INFO:teuthology.orchestra.run.smithi125.stdout:} 2024-02-17T12:14:07.663 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:07 smithi189 ceph-mon[36588]: pgmap v501: 97 pgs: 97 active+clean; 5.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-02-17T12:14:08.491 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-02-17T12:14:08.491 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-02-17T12:14:08.492 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-02-17T12:14:08.492 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-02-17T12:14:08.492 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=9 2024-02-17T12:14:08.492 DEBUG:tasks.check_counter:Getting stats from e 2024-02-17T12:14:08.492 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph daemon mds.e perf dump 2024-02-17T12:14:08.878 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T12:14:09.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:09 smithi125 ceph-mon[36819]: osdmap e107: 12 total, 12 up, 12 in 2024-02-17T12:14:09.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:09 smithi125 ceph-mon[36819]: pgmap v503: 97 pgs: 5 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 84 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-02-17T12:14:09.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:09 smithi094 ceph-mon[32499]: osdmap e107: 12 total, 12 up, 12 in 2024-02-17T12:14:09.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:09 smithi094 ceph-mon[32499]: pgmap v503: 97 pgs: 5 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 84 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-02-17T12:14:09.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:09 smithi189 ceph-mon[36588]: osdmap e107: 12 total, 12 up, 12 in 2024-02-17T12:14:09.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:09 smithi189 ceph-mon[36588]: pgmap v503: 97 pgs: 5 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 84 active+clean; 5.4 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-02-17T12:14:10.593 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2024-02-17T12:14:10.594 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T12:14:10.594 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 1093, 2024-02-17T12:14:10.594 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 1093, 2024-02-17T12:14:10.594 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 291220, 2024-02-17T12:14:10.594 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 339191, 2024-02-17T12:14:10.594 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 7, 2024-02-17T12:14:10.594 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 2, 2024-02-17T12:14:10.594 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.082320267, 2024-02-17T12:14:10.594 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.030604360, 2024-02-17T12:14:10.594 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.019886131, 2024-02-17T12:14:10.594 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.023302162, 2024-02-17T12:14:10.594 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:10.595 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 1092, 2024-02-17T12:14:10.595 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.046038553, 2024-02-17T12:14:10.595 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000042159 2024-02-17T12:14:10.595 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.595 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:10.595 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.595 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.595 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.595 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.595 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-02-17T12:14:10.595 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 448 2024-02-17T12:14:10.595 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.595 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 1338, 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 1320, 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 2026418, 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 493587, 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 4, 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 3, 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.126694687, 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.035044309, 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.058720249, 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.037229967, 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 1319, 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.041985917, 2024-02-17T12:14:10.596 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000031831 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 1781872, 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 184448 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 1081, 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 1307, 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 362864, 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 895905, 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 6, 2024-02-17T12:14:10.597 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 4, 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.112851416, 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.037447503, 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.028069495, 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.035564207, 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 1304, 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.080205605, 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000061507 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.598 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 560336 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: "cct": { 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: "total_workers": 1, 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: "unhealthy_workers": 0 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: "finisher-MDSRank": { 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: "queue_len": 0, 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: "complete_latency": { 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 5249, 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.131439112, 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000025040 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.599 INFO:teuthology.orchestra.run.smithi125.stdout: "finisher-PurgeQueue": { 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: "queue_len": 0, 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: "complete_latency": { 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": { 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: "request": 0, 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: "reply": 0, 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: "reply_latency": { 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.600 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "slow_reply": 0, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "forward": 0, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_fetch_complete": 0, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_fetch_keys": 0, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_commit": 0, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_split": 0, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_merge": 0, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes": 0, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_top": 0, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_bottom": 0, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_pin_tail": 0, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_pinned": 0, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_expired": 0, 2024-02-17T12:14:10.601 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_with_caps": 0, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": 0, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "subtrees": 1, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse": 0, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_hit": 0, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_forward": 0, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_discover": 0, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_dir_fetch": 0, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_remote_ino": 0, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_lock": 0, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "load_cent": 0, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "q": 0, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "exported": 0, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "exported_inodes": 0, 2024-02-17T12:14:10.602 INFO:teuthology.orchestra.run.smithi125.stdout: "imported": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "imported_inodes": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_dir_fetch": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_backtrace_fetch": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_peer_discover": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rfiles": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rbytes": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rsnaps": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_backtrace_fetch": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_set_tag": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_inotable_repaired": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dir_inodes": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dirfrag_rstats": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_file_inodes": 0, 2024-02-17T12:14:10.603 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_inode_file_caps": 0, 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_revoke": 0, 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_grant": 0, 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_caps": 0, 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_caps_dirty": 0, 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_cap_release": 0, 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: "process_request_cap_release": 0 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_cache": { 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_update": 0, 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_update_receipt": 0, 2024-02-17T12:14:10.604 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_try_discover": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_send_discover": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_handle_discover": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays_delayed": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays_enqueuing": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_created": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_enqueued": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_reintegrated": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_migrated": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_processing": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_enqueued": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_prioritized": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "recovery_started": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "recovery_completed": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T12:14:10.605 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_exportdir": 0, 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_flush": 0, 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_fragmentdir": 0, 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_fragstats": 0, 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_inodestats": 0 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_log": { 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: "evlrg": 0, 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: "evadd": 0, 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: "evex": 0, 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: "evtrm": 0, 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: "ev": 8, 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: "evexg": 0, 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: "evexd": 0, 2024-02-17T12:14:10.606 INFO:teuthology.orchestra.run.smithi125.stdout: "segadd": 0, 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "segex": 0, 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "segtrm": 0, 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "seg": 2, 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "segmjr": 2, 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "segexg": 0, 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "segexd": 0, 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "expos": 4194304, 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "wrpos": 4196655, 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "rdpos": 4196264, 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "jlat": { 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: "replayed": 8 2024-02-17T12:14:10.607 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_mem": { 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "ino": 3, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "ino+": 3, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "ino-": 0, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "dir": 1, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "dir+": 1, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "dir-": 0, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "dn": 0, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "dn+": 0, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "dn-": 0, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "cap": 0, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "cap+": 0, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "cap-": 0, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "rss": 46900, 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: "heap": 215420 2024-02-17T12:14:10.608 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_server": { 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "dispatch_client_request": 0, 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "dispatch_server_request": 0, 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_request": 0, 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_session": 0, 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_peer_request": 0, 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "req_create_latency": { 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getattr_latency": { 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.609 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getfilelock_latency": { 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: "req_link_latency": { 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookup_latency": { 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.610 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookuphash_latency": { 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupino_latency": { 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupname_latency": { 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.611 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupparent_latency": { 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupsnap_latency": { 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lssnap_latency": { 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mkdir_latency": { 2024-02-17T12:14:10.612 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.613 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.613 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.613 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.613 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mknod_latency": { 2024-02-17T12:14:10.613 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.613 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.613 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.613 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.613 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mksnap_latency": { 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: "req_open_latency": { 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: "req_readdir_latency": { 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.614 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rename_latency": { 2024-02-17T12:14:10.615 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.615 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.615 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.615 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.615 INFO:teuthology.orchestra.run.smithi125.stdout: "req_renamesnap_latency": { 2024-02-17T12:14:10.615 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.615 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.615 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.615 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.615 INFO:teuthology.orchestra.run.smithi125.stdout: "req_snapdiff_latency": { 2024-02-17T12:14:10.615 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.615 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.616 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.616 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.616 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmdir_latency": { 2024-02-17T12:14:10.616 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.616 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.616 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.616 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.617 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmsnap_latency": { 2024-02-17T12:14:10.617 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.617 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.617 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.617 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.617 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmxattr_latency": { 2024-02-17T12:14:10.617 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.617 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.618 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.618 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.618 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setattr_latency": { 2024-02-17T12:14:10.618 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.618 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.618 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.618 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.618 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setdirlayout_latency": { 2024-02-17T12:14:10.618 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.618 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.618 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.618 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.618 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setfilelock_latency": { 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setlayout_latency": { 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setxattr_latency": { 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.619 INFO:teuthology.orchestra.run.smithi125.stdout: "req_symlink_latency": { 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: "req_unlink_latency": { 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: "cap_revoke_eviction": 0, 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: "cap_acquisition_throttle": 0, 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getvxattr_latency": { 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.620 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_sessions": { 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: "session_count": 2, 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: "session_add": 4, 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: "session_remove": 2, 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: "sessions_open": 2, 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: "sessions_stale": 0, 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: "total_load": 0, 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: "average_load": 0, 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: "avg_session_uptime": 2288, 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: "mdthresh_evicted": 0 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: "mempool": { 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_bytes": 0, 2024-02-17T12:14:10.621 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_items": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_items": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_items": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_items": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_bytes": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_items": 0, 2024-02-17T12:14:10.622 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_bytes": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_items": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_items": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_bytes": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_items": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_bytes": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_items": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_bytes": 0, 2024-02-17T12:14:10.623 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_items": 0, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_items": 0, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_items": 0, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_bytes": 190177, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_items": 53, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_bytes": 0, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_items": 0, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_bytes": 0, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_items": 0, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_bytes": 0, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_items": 0, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_bytes": 0, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_items": 0, 2024-02-17T12:14:10.624 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_bytes": 8352, 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_items": 190, 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_items": 0, 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_bytes": 0, 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_items": 0, 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_bytes": 8320, 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_items": 13, 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_bytes": 0, 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_items": 0, 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_bytes": 0, 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_items": 0 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "objecter": { 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "op_active": 0, 2024-02-17T12:14:10.625 INFO:teuthology.orchestra.run.smithi125.stdout: "op_laggy": 0, 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: "op_send": 3162, 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: "op_send_bytes": 21, 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: "op_resend": 3, 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: "op_reply": 3162, 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: "op_latency": { 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 3162, 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 10.668820913, 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.003374073 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: "op_inflight": 0, 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: "oplen_avg": { 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 3162, 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 3163 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.626 INFO:teuthology.orchestra.run.smithi125.stdout: "op": 3162, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "op_r": 3162, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "op_w": 0, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "op_rmw": 0, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "op_pg": 0, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_stat": 2071, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_create": 0, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_read": 1089, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_write": 0, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_writefull": 0, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_writesame": 0, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_append": 0, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_zero": 0, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_truncate": 0, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_delete": 0, 2024-02-17T12:14:10.627 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_mapext": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_sparse_read": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_clonerange": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_getxattr": 1, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_setxattr": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_cmpxattr": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_rmxattr": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_resetxattrs": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_call": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_watch": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_notify": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_pgls": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_pgls_filter": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_other": 0, 2024-02-17T12:14:10.628 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_active": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_send": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_resend": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_ping": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_active": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_send": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_resend": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_active": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_send": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_resend": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_active": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_send": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_resend": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "command_active": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "command_send": 0, 2024-02-17T12:14:10.629 INFO:teuthology.orchestra.run.smithi125.stdout: "command_resend": 0, 2024-02-17T12:14:10.630 INFO:teuthology.orchestra.run.smithi125.stdout: "map_epoch": 107, 2024-02-17T12:14:10.630 INFO:teuthology.orchestra.run.smithi125.stdout: "map_full": 0, 2024-02-17T12:14:10.630 INFO:teuthology.orchestra.run.smithi125.stdout: "map_inc": 22, 2024-02-17T12:14:10.630 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_sessions": 5, 2024-02-17T12:14:10.630 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_session_open": 6, 2024-02-17T12:14:10.630 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_session_close": 1, 2024-02-17T12:14:10.631 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_laggy": 0, 2024-02-17T12:14:10.631 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_wr": 0, 2024-02-17T12:14:10.631 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_rd": 2, 2024-02-17T12:14:10.631 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_del": 0 2024-02-17T12:14:10.631 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.631 INFO:teuthology.orchestra.run.smithi125.stdout: "oft": { 2024-02-17T12:14:10.632 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_objs": 0, 2024-02-17T12:14:10.632 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_kv_pairs": 0, 2024-02-17T12:14:10.632 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_updates": 0, 2024-02-17T12:14:10.632 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_removes": 0 2024-02-17T12:14:10.632 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.632 INFO:teuthology.orchestra.run.smithi125.stdout: "osd-slow-ops": { 2024-02-17T12:14:10.632 INFO:teuthology.orchestra.run.smithi125.stdout: "slow_ops_count": 0 2024-02-17T12:14:10.632 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.632 INFO:teuthology.orchestra.run.smithi125.stdout: "purge_queue": { 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_ops": 0, 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_ops_high_water": 0, 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing": 0, 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_high_water": 0, 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executed_ops": 0, 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executed": 0, 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_item_in_journal": 0 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 3508, 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 2344209, 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:10.633 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 3508, 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 3508, 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 2344209, 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-objecter_bytes": { 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:10.634 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 3162, 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 2356, 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 3162, 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 2356, 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.635 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-objecter_ops": { 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 1024, 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 3162, 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 3162, 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 3162, 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 3162, 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.636 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-write_buf_throttle": { 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 3758096384, 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 0, 2024-02-17T12:14:10.637 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 0, 2024-02-17T12:14:10.638 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:10.638 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.638 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.638 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.638 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:10.638 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:10.638 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-write_buf_throttle-0x55986d270960": { 2024-02-17T12:14:10.638 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:10.638 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 3758096384, 2024-02-17T12:14:10.638 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:10.638 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2024-02-17T12:14:10.638 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2024-02-17T12:14:10.638 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:10.639 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:10.639 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2024-02-17T12:14:10.639 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2024-02-17T12:14:10.639 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 0, 2024-02-17T12:14:10.639 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 0, 2024-02-17T12:14:10.639 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:10.639 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:10.639 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:10.639 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:10.639 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:10.639 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:10.639 INFO:teuthology.orchestra.run.smithi125.stdout:} 2024-02-17T12:14:11.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:11 smithi125 ceph-mon[36819]: pgmap v504: 97 pgs: 5 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 84 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-02-17T12:14:11.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:11 smithi094 ceph-mon[32499]: pgmap v504: 97 pgs: 5 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 84 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-02-17T12:14:11.531 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-02-17T12:14:11.531 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-02-17T12:14:11.532 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-02-17T12:14:11.532 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-02-17T12:14:11.532 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T12:14:11.532 DEBUG:tasks.check_counter:Getting stats from h 2024-02-17T12:14:11.532 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph daemon mds.h perf dump 2024-02-17T12:14:11.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:11 smithi189 ceph-mon[36588]: pgmap v504: 97 pgs: 5 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 84 active+clean; 5.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 69 KiB/s wr, 27 op/s 2024-02-17T12:14:11.774 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T12:14:13.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:13 smithi125 ceph-mon[36819]: pgmap v505: 97 pgs: 5 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 84 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-02-17T12:14:13.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:13 smithi094 ceph-mon[32499]: pgmap v505: 97 pgs: 5 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 84 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-02-17T12:14:13.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:13 smithi189 ceph-mon[36588]: pgmap v505: 97 pgs: 5 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 84 active+clean; 5.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 43 KiB/s wr, 10 op/s 2024-02-17T12:14:13.992 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2024-02-17T12:14:13.992 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T12:14:13.992 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 3179, 2024-02-17T12:14:13.992 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 3181, 2024-02-17T12:14:13.992 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 900489, 2024-02-17T12:14:13.992 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 988676, 2024-02-17T12:14:13.992 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 6, 2024-02-17T12:14:13.992 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 2, 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.199107250, 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.059827156, 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.049362176, 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.074902837, 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 3179, 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.098069660, 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000030849 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:13.993 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 9, 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 232, 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 4060, 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 648236, 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 7, 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 4, 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.012665889, 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.007210727, 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.003299090, 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.000052155, 2024-02-17T12:14:13.994 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 230, 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.012902216, 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000056096 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 647552 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 344, 2024-02-17T12:14:13.995 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 326, 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 1781868, 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 184135, 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 4, 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 2, 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.052863580, 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.015912071, 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.040296598, 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.006523802, 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 325, 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.029923521, 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000092072 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:13.996 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 1781600, 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 183824 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: "cct": { 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: "total_workers": 1, 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: "unhealthy_workers": 0 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: "finisher-MDSRank": { 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: "queue_len": 0, 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: "complete_latency": { 2024-02-17T12:14:13.997 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 5282, 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.109441001, 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000020719 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: "finisher-PurgeQueue": { 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: "queue_len": 0, 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: "complete_latency": { 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: "mds": { 2024-02-17T12:14:13.998 INFO:teuthology.orchestra.run.smithi125.stdout: "request": 0, 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: "reply": 0, 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: "reply_latency": { 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: "slow_reply": 0, 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: "forward": 0, 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_fetch_complete": 0, 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_fetch_keys": 0, 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_commit": 0, 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_split": 0, 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_merge": 0, 2024-02-17T12:14:13.999 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes": 0, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_top": 0, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_bottom": 0, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_pin_tail": 0, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_pinned": 0, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_expired": 0, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "inodes_with_caps": 0, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "caps": 0, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "subtrees": 1, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse": 0, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_hit": 0, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_forward": 0, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_discover": 0, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_dir_fetch": 0, 2024-02-17T12:14:14.000 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_remote_ino": 0, 2024-02-17T12:14:14.001 INFO:teuthology.orchestra.run.smithi125.stdout: "traverse_lock": 0, 2024-02-17T12:14:14.001 INFO:teuthology.orchestra.run.smithi125.stdout: "load_cent": 0, 2024-02-17T12:14:14.001 INFO:teuthology.orchestra.run.smithi125.stdout: "q": 0, 2024-02-17T12:14:14.001 INFO:teuthology.orchestra.run.smithi125.stdout: "exported": 0, 2024-02-17T12:14:14.001 INFO:teuthology.orchestra.run.smithi125.stdout: "exported_inodes": 0, 2024-02-17T12:14:14.001 INFO:teuthology.orchestra.run.smithi125.stdout: "imported": 0, 2024-02-17T12:14:14.001 INFO:teuthology.orchestra.run.smithi125.stdout: "imported_inodes": 0, 2024-02-17T12:14:14.001 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_dir_fetch": 0, 2024-02-17T12:14:14.001 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_backtrace_fetch": 0, 2024-02-17T12:14:14.001 INFO:teuthology.orchestra.run.smithi125.stdout: "openino_peer_discover": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rfiles": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rbytes": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "root_rsnaps": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_backtrace_fetch": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_set_tag": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_inotable_repaired": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dir_inodes": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_dirfrag_rstats": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "scrub_file_inodes": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_inode_file_caps": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_revoke": 0, 2024-02-17T12:14:14.002 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_grant": 0, 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_caps": 0, 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_caps_dirty": 0, 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_cap_release": 0, 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "process_request_cap_release": 0 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_cache": { 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_update": 0, 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_update_receipt": 0, 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_try_discover": 0, 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_send_discover": 0, 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "dir_handle_discover": 0, 2024-02-17T12:14:14.003 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays_delayed": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "num_strays_enqueuing": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_created": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_enqueued": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_reintegrated": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "strays_migrated": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_processing": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_enqueued": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "num_recovering_prioritized": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "recovery_started": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "recovery_completed": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_exportdir": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_flush": 0, 2024-02-17T12:14:14.004 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_fragmentdir": 0, 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_fragstats": 0, 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: "ireq_inodestats": 0 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_log": { 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: "evlrg": 0, 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: "evadd": 0, 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: "evex": 0, 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: "evtrm": 0, 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: "ev": 10, 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: "evexg": 0, 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: "evexd": 0, 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: "segadd": 0, 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: "segex": 0, 2024-02-17T12:14:14.005 INFO:teuthology.orchestra.run.smithi125.stdout: "segtrm": 0, 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "seg": 2, 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "segmjr": 2, 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "segexg": 0, 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "segexd": 0, 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "expos": 4194304, 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "wrpos": 4197020, 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "rdpos": 4196879, 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "jlat": { 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "replayed": 10 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_mem": { 2024-02-17T12:14:14.006 INFO:teuthology.orchestra.run.smithi125.stdout: "ino": 3, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "ino+": 3, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "ino-": 0, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "dir": 1, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "dir+": 1, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "dir-": 0, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "dn": 0, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "dn+": 0, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "dn-": 0, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "cap": 0, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "cap+": 0, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "cap-": 0, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "rss": 50612, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "heap": 215420 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.007 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_server": { 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: "dispatch_client_request": 0, 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: "dispatch_server_request": 0, 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_request": 0, 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_client_session": 0, 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: "handle_peer_request": 0, 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: "req_create_latency": { 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getattr_latency": { 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.008 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getfilelock_latency": { 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: "req_link_latency": { 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookup_latency": { 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.009 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookuphash_latency": { 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupino_latency": { 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupname_latency": { 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.010 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupparent_latency": { 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lookupsnap_latency": { 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: "req_lssnap_latency": { 2024-02-17T12:14:14.011 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.012 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.012 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.012 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.012 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mkdir_latency": { 2024-02-17T12:14:14.012 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.012 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.012 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.012 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.013 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mknod_latency": { 2024-02-17T12:14:14.013 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.013 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.013 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.013 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.013 INFO:teuthology.orchestra.run.smithi125.stdout: "req_mksnap_latency": { 2024-02-17T12:14:14.013 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.013 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.013 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: "req_open_latency": { 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: "req_readdir_latency": { 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rename_latency": { 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.014 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: "req_renamesnap_latency": { 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: "req_snapdiff_latency": { 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmdir_latency": { 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.015 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmsnap_latency": { 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: "req_rmxattr_latency": { 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setattr_latency": { 2024-02-17T12:14:14.016 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setdirlayout_latency": { 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setfilelock_latency": { 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.017 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setlayout_latency": { 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: "req_setxattr_latency": { 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: "req_symlink_latency": { 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.018 INFO:teuthology.orchestra.run.smithi125.stdout: "req_unlink_latency": { 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: "cap_revoke_eviction": 0, 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: "cap_acquisition_throttle": 0, 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: "req_getvxattr_latency": { 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_sessions": { 2024-02-17T12:14:14.019 INFO:teuthology.orchestra.run.smithi125.stdout: "session_count": 0, 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: "session_add": 4, 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: "session_remove": 4, 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: "sessions_open": 2, 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: "sessions_stale": 0, 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: "total_load": 0, 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: "average_load": 0, 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: "avg_session_uptime": 1417, 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: "mdthresh_evicted": 0 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: "mempool": { 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_bytes": 0, 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_items": 0, 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T12:14:14.020 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_items": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_items": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_items": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_bytes": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_items": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_bytes": 0, 2024-02-17T12:14:14.021 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_items": 0, 2024-02-17T12:14:14.022 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T12:14:14.022 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T12:14:14.022 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T12:14:14.022 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T12:14:14.022 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T12:14:14.022 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_items": 0, 2024-02-17T12:14:14.022 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_bytes": 0, 2024-02-17T12:14:14.022 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_items": 0, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_bytes": 0, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_items": 0, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_bytes": 0, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_items": 0, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_items": 0, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_items": 0, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_bytes": 182097, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_items": 49, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_bytes": 0, 2024-02-17T12:14:14.023 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_items": 0, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_bytes": 0, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_items": 0, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_bytes": 0, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_items": 0, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_bytes": 0, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_items": 0, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_bytes": 8472, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_items": 193, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_items": 0, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_bytes": 0, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_items": 0, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_bytes": 8320, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_items": 13, 2024-02-17T12:14:14.024 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_bytes": 0, 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_items": 0, 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_bytes": 0, 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_items": 0 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "objecter": { 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "op_active": 0, 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "op_laggy": 0, 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "op_send": 3182, 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "op_send_bytes": 21, 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "op_resend": 3, 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "op_reply": 3182, 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "op_latency": { 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 3182, 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 11.345809937, 2024-02-17T12:14:14.025 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.003565622 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: "op_inflight": 0, 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: "oplen_avg": { 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 3182, 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 3183 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: "op": 3182, 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: "op_r": 3182, 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: "op_w": 0, 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: "op_rmw": 0, 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: "op_pg": 0, 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_stat": 2085, 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_create": 0, 2024-02-17T12:14:14.026 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_read": 1095, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_write": 0, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_writefull": 0, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_writesame": 0, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_append": 0, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_zero": 0, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_truncate": 0, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_delete": 0, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_mapext": 0, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_sparse_read": 0, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_clonerange": 0, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_getxattr": 1, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_setxattr": 0, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_cmpxattr": 0, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_rmxattr": 0, 2024-02-17T12:14:14.027 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_resetxattrs": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_call": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_watch": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_notify": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_pgls": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_pgls_filter": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "osdop_other": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_active": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_send": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_resend": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "linger_ping": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_active": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_send": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "poolop_resend": 0, 2024-02-17T12:14:14.028 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_active": 0, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_send": 0, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "poolstat_resend": 0, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_active": 0, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_send": 0, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "statfs_resend": 0, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "command_active": 0, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "command_send": 0, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "command_resend": 0, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "map_epoch": 107, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "map_full": 0, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "map_inc": 22, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_sessions": 4, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_session_open": 5, 2024-02-17T12:14:14.029 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_session_close": 1, 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_laggy": 0, 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_wr": 0, 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_rd": 2, 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_del": 0 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: "oft": { 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_objs": 0, 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_kv_pairs": 0, 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_updates": 0, 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: "omap_total_removes": 0 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: "osd-slow-ops": { 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: "slow_ops_count": 0 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.030 INFO:teuthology.orchestra.run.smithi125.stdout: "purge_queue": { 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_ops": 0, 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_ops_high_water": 0, 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing": 0, 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executing_high_water": 0, 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executed_ops": 0, 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_executed": 0, 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "pq_item_in_journal": 0 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 3528, 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 2348333, 2024-02-17T12:14:14.031 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 3528, 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 3528, 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 2348333, 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-objecter_bytes": { 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2024-02-17T12:14:14.032 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 3182, 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 2721, 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 3182, 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 2721, 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.033 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-objecter_ops": { 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 1024, 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 3182, 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 3182, 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 3182, 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 3182, 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.034 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-write_buf_throttle": { 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 3758096384, 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 0, 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 0, 2024-02-17T12:14:14.035 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-write_buf_throttle-0x55c71e50a960": { 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 3758096384, 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 0, 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 0, 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:14.036 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2024-02-17T12:14:14.037 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2024-02-17T12:14:14.037 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 0, 2024-02-17T12:14:14.037 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 0, 2024-02-17T12:14:14.037 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:14.037 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:14.037 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:14.037 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:14.037 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:14.037 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:14.037 INFO:teuthology.orchestra.run.smithi125.stdout:} 2024-02-17T12:14:14.842 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-02-17T12:14:14.842 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-02-17T12:14:14.843 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-02-17T12:14:14.843 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-02-17T12:14:14.843 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T12:14:14.843 DEBUG:tasks.check_counter:Getting stats from k 2024-02-17T12:14:14.843 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph daemon mds.k perf dump 2024-02-17T12:14:15.086 INFO:teuthology.orchestra.run.smithi125.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.b/config 2024-02-17T12:14:15.438 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:15 smithi125 ceph-mon[36819]: pgmap v506: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2024-02-17T12:14:15.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:15 smithi094 ceph-mon[32499]: pgmap v506: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2024-02-17T12:14:15.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:15 smithi189 ceph-mon[36588]: pgmap v506: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.9 KiB/s wr, 6 op/s 2024-02-17T12:14:16.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:16 smithi189 ceph-mon[36588]: osdmap e108: 12 total, 12 up, 12 in 2024-02-17T12:14:16.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:16 smithi125 ceph-mon[36819]: osdmap e108: 12 total, 12 up, 12 in 2024-02-17T12:14:16.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:16 smithi094 ceph-mon[32499]: osdmap e108: 12 total, 12 up, 12 in 2024-02-17T12:14:17.244 INFO:teuthology.orchestra.run.smithi125.stdout:{ 2024-02-17T12:14:17.245 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T12:14:17.245 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 0, 2024-02-17T12:14:17.245 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 2, 2024-02-17T12:14:17.245 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 0, 2024-02-17T12:14:17.245 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 0, 2024-02-17T12:14:17.245 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 4, 2024-02-17T12:14:17.245 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 0, 2024-02-17T12:14:17.245 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.000916284, 2024-02-17T12:14:17.245 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.000000000, 2024-02-17T12:14:17.245 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.000116712, 2024-02-17T12:14:17.245 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T12:14:17.245 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T12:14:17.246 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 315, 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 303, 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 1732592, 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 182240, 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 2, 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 1, 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.051251436, 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.015234350, 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.038744925, 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.006550737, 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 302, 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.019314131, 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000063954 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:17.247 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:17.248 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:17.248 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:17.248 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:17.248 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 1732592, 2024-02-17T12:14:17.248 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 182240 2024-02-17T12:14:17.248 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:17.248 INFO:teuthology.orchestra.run.smithi125.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T12:14:17.248 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_messages": 7, 2024-02-17T12:14:17.248 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages": 234, 2024-02-17T12:14:17.248 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_bytes": 3504, 2024-02-17T12:14:17.248 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_bytes": 273744, 2024-02-17T12:14:17.248 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_created_connections": 6, 2024-02-17T12:14:17.248 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_active_connections": 3, 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_total_time": 0.013043974, 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_send_time": 0.008312077, 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_recv_time": 0.002287638, 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 232, 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.014705721, 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000063386 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:17.249 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_recv_encrypted_bytes": 3504, 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: "msgr_send_encrypted_bytes": 273744 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: "cct": { 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: "total_workers": 0, 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: "unhealthy_workers": 0 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: "mempool": { 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_bytes": 0, 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: "bloom_filter_items": 0, 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_alloc_items": 0, 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T12:14:17.250 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_data_items": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_other_items": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_bytes": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_extent_items": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_bytes": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_blob_items": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T12:14:17.251 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_fsck_items": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_bytes": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_txc_items": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_bytes": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluestore_writing_items": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_bytes": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_items": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_reader_items": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T12:14:17.252 INFO:teuthology.orchestra.run.smithi125.stdout: "bluefs_file_writer_items": 0, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_bytes": 148709, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_anon_items": 31, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_bytes": 0, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "buffer_meta_items": 0, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_bytes": 0, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_items": 0, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_bytes": 0, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_mapbl_items": 0, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_bytes": 0, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "osd_pglog_items": 0, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_bytes": 0, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_items": 0, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T12:14:17.253 INFO:teuthology.orchestra.run.smithi125.stdout: "osdmap_mapping_items": 0, 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_bytes": 0, 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "pgmap_items": 0, 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_bytes": 928, 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "mds_co_items": 2, 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_bytes": 0, 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_1_items": 0, 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_bytes": 0, 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "unittest_2_items": 0 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: }, 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "val": 0, 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "max": 104857600, 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "get_started": 0, 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "get": 318, 2024-02-17T12:14:17.254 INFO:teuthology.orchestra.run.smithi125.stdout: "get_sum": 1688147, 2024-02-17T12:14:17.255 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:17.255 INFO:teuthology.orchestra.run.smithi125.stdout: "get_or_fail_success": 318, 2024-02-17T12:14:17.255 INFO:teuthology.orchestra.run.smithi125.stdout: "take": 0, 2024-02-17T12:14:17.255 INFO:teuthology.orchestra.run.smithi125.stdout: "take_sum": 0, 2024-02-17T12:14:17.255 INFO:teuthology.orchestra.run.smithi125.stdout: "put": 318, 2024-02-17T12:14:17.255 INFO:teuthology.orchestra.run.smithi125.stdout: "put_sum": 1688147, 2024-02-17T12:14:17.255 INFO:teuthology.orchestra.run.smithi125.stdout: "wait": { 2024-02-17T12:14:17.255 INFO:teuthology.orchestra.run.smithi125.stdout: "avgcount": 0, 2024-02-17T12:14:17.255 INFO:teuthology.orchestra.run.smithi125.stdout: "sum": 0.000000000, 2024-02-17T12:14:17.255 INFO:teuthology.orchestra.run.smithi125.stdout: "avgtime": 0.000000000 2024-02-17T12:14:17.255 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:17.255 INFO:teuthology.orchestra.run.smithi125.stdout: } 2024-02-17T12:14:17.255 INFO:teuthology.orchestra.run.smithi125.stdout:} 2024-02-17T12:14:17.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:17 smithi189 ceph-mon[36588]: pgmap v508: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 5 op/s 2024-02-17T12:14:17.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:17 smithi125 ceph-mon[36819]: pgmap v508: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 5 op/s 2024-02-17T12:14:17.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:17 smithi094 ceph-mon[32499]: pgmap v508: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 5 op/s 2024-02-17T12:14:17.958 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-02-17T12:14:17.958 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-02-17T12:14:17.958 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-02-17T12:14:17.958 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-02-17T12:14:17.958 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-02-17T12:14:17.958 DEBUG:tasks.check_counter:Getting stats from c 2024-02-17T12:14:17.959 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph daemon mds.c perf dump 2024-02-17T12:14:18.331 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:14:19.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:19 smithi125 ceph-mon[36819]: pgmap v509: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-17T12:14:19.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:19 smithi094 ceph-mon[32499]: pgmap v509: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-17T12:14:19.508 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:19 smithi189 ceph-mon[36588]: pgmap v509: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-17T12:14:20.414 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-02-17T12:14:20.414 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T12:14:20.414 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 49, 2024-02-17T12:14:20.414 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 76, 2024-02-17T12:14:20.414 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 11346, 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 42832, 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 12, 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 3, 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.020716930, 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.003660034, 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.010959380, 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000685164, 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 74, 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.013797636, 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000186454 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 67, 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000010513, 2024-02-17T12:14:20.415 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000156 2024-02-17T12:14:20.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.416 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 4240, 2024-02-17T12:14:20.416 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 10032 2024-02-17T12:14:20.416 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.416 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T12:14:20.416 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 417, 2024-02-17T12:14:20.416 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 445, 2024-02-17T12:14:20.416 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 1803469, 2024-02-17T12:14:20.416 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 259386, 2024-02-17T12:14:20.416 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 11, 2024-02-17T12:14:20.416 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 5, 2024-02-17T12:14:20.416 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.059735938, 2024-02-17T12:14:20.416 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.018596494, 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.044856795, 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.006939135, 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 445, 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.026995836, 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000060664 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 83, 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000012964, 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000156 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1799104, 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 208512 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T12:14:20.417 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 658, 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1028, 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 179909, 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 1549582, 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 66, 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 6, 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.101526431, 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.035738335, 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.056218069, 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.005799798, 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1025, 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.058014752, 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000056599 2024-02-17T12:14:20.418 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1384, 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000284925, 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000205 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 563712 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-02-17T12:14:20.419 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-02-17T12:14:20.420 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-02-17T12:14:20.420 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 60, 2024-02-17T12:14:20.420 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.001549629, 2024-02-17T12:14:20.420 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000025827 2024-02-17T12:14:20.420 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:20.420 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.420 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-02-17T12:14:20.420 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-02-17T12:14:20.420 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-02-17T12:14:20.420 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-02-17T12:14:20.420 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000007454, 2024-02-17T12:14:20.420 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000003727 2024-02-17T12:14:20.420 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 6, 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 3, 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3, 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000583524, 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000194508 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 3, 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-02-17T12:14:20.421 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 11, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 10, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 0, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 10, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 10, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 1, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 1, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 1, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 7, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 6, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-02-17T12:14:20.422 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 1, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 1026, 2024-02-17T12:14:20.423 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 1732132033, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 7, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T12:14:20.424 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 1, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-02-17T12:14:20.425 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.426 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 8, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 8, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 2, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 2, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 2, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-02-17T12:14:20.427 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-02-17T12:14:20.428 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 4196655, 2024-02-17T12:14:20.428 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 0, 2024-02-17T12:14:20.428 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-02-17T12:14:20.428 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 7, 2024-02-17T12:14:20.428 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.025746290, 2024-02-17T12:14:20.428 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.003678041 2024-02-17T12:14:20.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.428 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 0 2024-02-17T12:14:20.428 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.428 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-02-17T12:14:20.428 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 13, 2024-02-17T12:14:20.428 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 13, 2024-02-17T12:14:20.428 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 0, 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 11, 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 12, 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 1, 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 10, 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 10, 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 0, 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 1, 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 1, 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 49224, 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 207228 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 6, 2024-02-17T12:14:20.429 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 6, 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 32, 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3, 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000583524, 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000194508 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-02-17T12:14:20.430 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.431 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.432 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.433 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.434 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.434 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-02-17T12:14:20.434 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.434 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.434 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.434 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-02-17T12:14:20.434 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.434 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.434 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.434 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.435 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.436 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.437 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-02-17T12:14:20.438 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-02-17T12:14:20.439 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 2, 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 4, 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 2, 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 2, 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 2, 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 1, 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 943, 2024-02-17T12:14:20.440 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T12:14:20.441 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-02-17T12:14:20.442 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 240359, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 88, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 88, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 1, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8352, 2024-02-17T12:14:20.443 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 190, 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 50416, 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 201, 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-02-17T12:14:20.444 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 45, 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 12833, 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 33, 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 45, 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 45, 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.234995168, 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.005222114 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 45, 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 67 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 45, 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 0, 2024-02-17T12:14:20.445 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 45, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 0, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 10, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 0, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 7, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 9, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 6, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 0, 2024-02-17T12:14:20.446 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 20, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-02-17T12:14:20.447 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 108, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 23, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 8, 2024-02-17T12:14:20.448 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 11, 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 3, 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 14, 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 0, 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 1 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 1, 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-02-17T12:14:20.449 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-02-17T12:14:20.450 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 1120, 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 1870146, 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 1120, 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 1120, 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 1870146, 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-02-17T12:14:20.451 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 45, 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 12833, 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 45, 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 12833, 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.452 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 45, 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 45, 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 45, 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 45, 2024-02-17T12:14:20.453 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-02-17T12:14:20.454 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-02-17T12:14:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-02-17T12:14:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-02-17T12:14:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x5616f8b8e6e0": { 2024-02-17T12:14:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-02-17T12:14:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:20.455 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 8, 2024-02-17T12:14:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 2351, 2024-02-17T12:14:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 8, 2024-02-17T12:14:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-02-17T12:14:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-02-17T12:14:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 7, 2024-02-17T12:14:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2351, 2024-02-17T12:14:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:20.456 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:20.457 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:20.457 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-02-17T12:14:21.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:21 smithi189 ceph-mon[36588]: pgmap v510: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-17T12:14:21.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:21 smithi125 ceph-mon[36819]: pgmap v510: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-17T12:14:21.460 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:21 smithi094 ceph-mon[32499]: pgmap v510: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-17T12:14:22.774 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-02-17T12:14:22.774 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-02-17T12:14:22.774 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-02-17T12:14:22.774 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-02-17T12:14:22.774 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T12:14:22.774 DEBUG:tasks.check_counter:Getting stats from f 2024-02-17T12:14:22.774 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph daemon mds.f perf dump 2024-02-17T12:14:23.026 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:14:23.355 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:23 smithi189 ceph-mon[36588]: pgmap v511: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-17T12:14:23.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:23 smithi125 ceph-mon[36819]: pgmap v511: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-17T12:14:23.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:23 smithi094 ceph-mon[32499]: pgmap v511: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-02-17T12:14:25.067 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-02-17T12:14:25.068 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T12:14:25.068 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 0, 2024-02-17T12:14:25.068 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 2, 2024-02-17T12:14:25.068 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 0, 2024-02-17T12:14:25.068 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 0, 2024-02-17T12:14:25.068 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 4, 2024-02-17T12:14:25.068 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 0, 2024-02-17T12:14:25.068 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.002128709, 2024-02-17T12:14:25.068 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.000000000, 2024-02-17T12:14:25.068 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.000882748, 2024-02-17T12:14:25.068 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T12:14:25.068 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 217, 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 214, 2024-02-17T12:14:25.069 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 203184, 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 127728, 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 2, 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 1, 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.027167659, 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.008546556, 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.010810514, 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.005504023, 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 214, 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.014878025, 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000069523 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:25.070 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:25.071 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:25.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:25.071 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 203184, 2024-02-17T12:14:25.071 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 127728 2024-02-17T12:14:25.071 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:25.071 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T12:14:25.071 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1, 2024-02-17T12:14:25.071 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 161, 2024-02-17T12:14:25.071 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 160, 2024-02-17T12:14:25.071 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 105648, 2024-02-17T12:14:25.071 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 3, 2024-02-17T12:14:25.072 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 2, 2024-02-17T12:14:25.072 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.008065632, 2024-02-17T12:14:25.072 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.005646359, 2024-02-17T12:14:25.072 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.001118888, 2024-02-17T12:14:25.072 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-17T12:14:25.072 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:25.072 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 159, 2024-02-17T12:14:25.072 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.013051305, 2024-02-17T12:14:25.072 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000082083 2024-02-17T12:14:25.072 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:25.072 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:25.072 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:25.072 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 160, 2024-02-17T12:14:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 105648 2024-02-17T12:14:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-02-17T12:14:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 0, 2024-02-17T12:14:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-02-17T12:14:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-02-17T12:14:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-02-17T12:14:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-02-17T12:14:25.073 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-02-17T12:14:25.074 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T12:14:25.075 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 150015, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 31, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 0, 2024-02-17T12:14:25.076 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 0, 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 928, 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 2, 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-02-17T12:14:25.077 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 214, 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 170046, 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 214, 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 214, 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 170046, 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:25.078 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-02-17T12:14:25.404 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:25 smithi189 ceph-mon[36588]: pgmap v512: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-02-17T12:14:25.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:25 smithi125 ceph-mon[36819]: pgmap v512: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-02-17T12:14:25.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:25 smithi094 ceph-mon[32499]: pgmap v512: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-02-17T12:14:25.871 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-02-17T12:14:25.871 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-02-17T12:14:25.872 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-02-17T12:14:25.872 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-02-17T12:14:25.872 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-02-17T12:14:25.872 DEBUG:tasks.check_counter:Getting stats from i 2024-02-17T12:14:25.872 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph daemon mds.i perf dump 2024-02-17T12:14:26.128 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:14:27.285 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:27 smithi189 ceph-mon[36588]: pgmap v513: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-02-17T12:14:27.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:27 smithi125 ceph-mon[36819]: pgmap v513: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-02-17T12:14:27.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:27 smithi094 ceph-mon[32499]: pgmap v513: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-02-17T12:14:27.838 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-02-17T12:14:27.838 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T12:14:27.838 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 50, 2024-02-17T12:14:27.839 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 252, 2024-02-17T12:14:27.839 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 12717, 2024-02-17T12:14:27.839 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 563392, 2024-02-17T12:14:27.839 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 15, 2024-02-17T12:14:27.839 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-02-17T12:14:27.839 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.020595844, 2024-02-17T12:14:27.839 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.007904435, 2024-02-17T12:14:27.839 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.006041052, 2024-02-17T12:14:27.839 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000622282, 2024-02-17T12:14:27.839 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:27.839 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 248, 2024-02-17T12:14:27.839 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.042869258, 2024-02-17T12:14:27.840 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000172859 2024-02-17T12:14:27.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.840 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:27.840 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 37, 2024-02-17T12:14:27.840 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000004357, 2024-02-17T12:14:27.840 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000117 2024-02-17T12:14:27.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.840 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 5200, 2024-02-17T12:14:27.840 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 529488 2024-02-17T12:14:27.840 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.840 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T12:14:27.840 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1597, 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1321, 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 2118243, 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 470595, 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 15, 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.150417965, 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.049788525, 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.110455650, 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.014200001, 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1323, 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.085933898, 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000064953 2024-02-17T12:14:27.841 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.842 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:27.842 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2128, 2024-02-17T12:14:27.842 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000431847, 2024-02-17T12:14:27.842 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000202 2024-02-17T12:14:27.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.842 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1799344, 2024-02-17T12:14:27.842 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 191120 2024-02-17T12:14:27.842 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.842 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T12:14:27.842 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 69, 2024-02-17T12:14:27.842 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 101, 2024-02-17T12:14:27.842 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 11699, 2024-02-17T12:14:27.842 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 661855, 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 64, 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.026054843, 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.003026563, 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.023532596, 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000411782, 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 98, 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.011834949, 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000120764 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 62, 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000008420, 2024-02-17T12:14:27.843 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000135 2024-02-17T12:14:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-02-17T12:14:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 131248 2024-02-17T12:14:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-02-17T12:14:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-02-17T12:14:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-02-17T12:14:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-02-17T12:14:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-02-17T12:14:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-02-17T12:14:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 49, 2024-02-17T12:14:27.844 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.002388283, 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000048740 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000010903, 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000005451 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 7, 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 4, 2024-02-17T12:14:27.845 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 4, 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.002381757, 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000595439 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 3, 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 11, 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 12, 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 0, 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 12, 2024-02-17T12:14:27.846 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 11, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 2, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 12, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 8, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 2, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-02-17T12:14:27.847 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 1026, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 1732132033, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 7, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-02-17T12:14:27.848 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 2 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-02-17T12:14:27.849 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 3, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-02-17T12:14:27.850 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 10, 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 10, 2024-02-17T12:14:27.851 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 2, 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 2, 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 2, 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 4194304, 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 4197020, 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 0, 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 9, 2024-02-17T12:14:27.852 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.027972399, 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.003108044 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 0 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 15, 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 15, 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 0, 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 13, 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 14, 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 1, 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 12, 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 12, 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 0, 2024-02-17T12:14:27.853 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 4, 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 4, 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 52332, 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 207228 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 12, 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 7, 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 15, 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 1, 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.854 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.001699887, 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000849943 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.855 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 2, 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000681870, 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000340935 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.856 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.857 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.858 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.858 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-02-17T12:14:27.858 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.858 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.858 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.858 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.858 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-02-17T12:14:27.858 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.858 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.859 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.859 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-02-17T12:14:27.859 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.859 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.859 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.859 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.860 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.861 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-02-17T12:14:27.862 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-02-17T12:14:27.863 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.864 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.865 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 0, 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 4, 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 4, 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 0, 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 3, 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 3, 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 86, 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.866 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T12:14:27.867 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T12:14:27.868 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 256359, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 89, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 88, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 1, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-02-17T12:14:27.869 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8352, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 190, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 58400, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 237, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.870 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 48, 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 13333, 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 33, 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 48, 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 48, 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.406991644, 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.008478992 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 48, 2024-02-17T12:14:27.871 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 72 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 48, 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 0, 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 48, 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 0, 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 10, 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 0, 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 9, 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 8, 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-02-17T12:14:27.872 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 6, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 0, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 20, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T12:14:27.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-02-17T12:14:27.874 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 108, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 23, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 10, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 14, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 4, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 17, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 0, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 2 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.875 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 1, 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-02-17T12:14:27.876 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 1712, 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 1965284, 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 1712, 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 1712, 2024-02-17T12:14:27.877 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 1965284, 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 48, 2024-02-17T12:14:27.878 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 13333, 2024-02-17T12:14:27.879 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 48, 2024-02-17T12:14:27.879 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 13333, 2024-02-17T12:14:27.879 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:27.879 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.879 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.879 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.879 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:27.879 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.879 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-02-17T12:14:27.879 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:27.879 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-02-17T12:14:27.879 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:27.879 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 48, 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 48, 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 48, 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 48, 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.880 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.881 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.882 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:27.882 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:27.882 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x5588f00be6e0": { 2024-02-17T12:14:27.882 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:27.882 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-02-17T12:14:27.882 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:27.882 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 10, 2024-02-17T12:14:27.882 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 2716, 2024-02-17T12:14:27.882 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:27.882 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 10, 2024-02-17T12:14:27.882 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-02-17T12:14:27.882 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-02-17T12:14:27.882 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 9, 2024-02-17T12:14:27.883 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 2716, 2024-02-17T12:14:27.883 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:27.883 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:27.883 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:27.883 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:27.883 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:27.883 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:27.883 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-02-17T12:14:28.638 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-02-17T12:14:28.639 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-02-17T12:14:28.639 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-02-17T12:14:28.639 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-02-17T12:14:28.639 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T12:14:28.639 DEBUG:tasks.check_counter:Getting stats from l 2024-02-17T12:14:28.639 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d shell --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 -- ceph daemon mds.l perf dump 2024-02-17T12:14:28.876 INFO:teuthology.orchestra.run.smithi189.stderr:Inferring config /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/mon.c/config 2024-02-17T12:14:29.249 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:29 smithi189 ceph-mon[36588]: pgmap v514: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-17T12:14:29.437 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:29 smithi125 ceph-mon[36819]: pgmap v514: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-17T12:14:29.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:29 smithi094 ceph-mon[32499]: pgmap v514: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-02-17T12:14:30.605 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-02-17T12:14:30.605 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-0": { 2024-02-17T12:14:30.605 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 1397, 2024-02-17T12:14:30.605 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 1399, 2024-02-17T12:14:30.605 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 15097945, 2024-02-17T12:14:30.605 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 434477, 2024-02-17T12:14:30.605 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 7, 2024-02-17T12:14:30.605 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 3, 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.123472043, 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.031388916, 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.040073677, 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.043018222, 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1397, 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.083137582, 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000059511 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.606 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-17T12:14:30.607 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 0 2024-02-17T12:14:30.607 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.607 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-1": { 2024-02-17T12:14:30.607 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 653, 2024-02-17T12:14:30.607 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 652, 2024-02-17T12:14:30.607 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 11003791, 2024-02-17T12:14:30.607 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 242845, 2024-02-17T12:14:30.607 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 4, 2024-02-17T12:14:30.607 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 3, 2024-02-17T12:14:30.607 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.058363230, 2024-02-17T12:14:30.607 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.017613859, 2024-02-17T12:14:30.607 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.023517223, 2024-02-17T12:14:30.607 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.015517122, 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 651, 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.041772932, 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000064167 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 208512, 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 94752 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: "AsyncMessenger::Worker-2": { 2024-02-17T12:14:30.608 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_messages": 8, 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages": 120, 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_bytes": 5160, 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_bytes": 287247, 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_created_connections": 5, 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_active_connections": 4, 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_total_time": 0.007192250, 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_send_time": 0.004544240, 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_recv_time": 0.001748839, 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_running_fast_dispatch_time": 0.000065392, 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_messages_queue_lat": { 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 119, 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.012776758, 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000107367 2024-02-17T12:14:30.609 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_handle_ack_lat": { 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: "msgr_send_encrypted_bytes": 286304 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: "cct": { 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: "total_workers": 1, 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: "unhealthy_workers": 0 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-MDSRank": { 2024-02-17T12:14:30.610 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 3145, 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.089820749, 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000028559 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: "finisher-PurgeQueue": { 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: "queue_len": 0, 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: "complete_latency": { 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:30.611 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "mds": { 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "request": 0, 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "reply": 0, 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "reply_latency": { 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_reply": 0, 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "forward": 0, 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_complete": 0, 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_fetch_keys": 0, 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_commit": 0, 2024-02-17T12:14:30.612 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_split": 0, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_merge": 0, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes": 743, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_top": 446, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_bottom": 297, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pin_tail": 0, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_pinned": 106, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_expired": 0, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "inodes_with_caps": 0, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "caps": 0, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "subtrees": 2, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse": 0, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_hit": 0, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_forward": 0, 2024-02-17T12:14:30.613 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_discover": 0, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_dir_fetch": 0, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_remote_ino": 0, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "traverse_lock": 0, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "load_cent": 0, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "q": 0, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "exported": 0, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "exported_inodes": 0, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "imported": 0, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "imported_inodes": 0, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_dir_fetch": 0, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_backtrace_fetch": 0, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "openino_peer_discover": 0, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rfiles": 1026, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rbytes": 1732302270, 2024-02-17T12:14:30.614 INFO:teuthology.orchestra.run.smithi189.stdout: "root_rsnaps": 7, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_fetch": 0, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_set_tag": 0, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_backtrace_repaired": 0, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_inotable_repaired": 0, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_inodes": 0, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dir_base_inodes": 0, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_dirfrag_rstats": 0, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "scrub_file_inodes": 0, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_inode_file_caps": 0, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_revoke": 0, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_grant": 0, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_trunc": 0, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-17T12:14:30.615 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps": 0, 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_caps_dirty": 0, 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_cap_release": 0, 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: "process_request_cap_release": 0 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_cache": { 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update": 0, 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_update_receipt": 0, 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_try_discover": 0, 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_send_discover": 0, 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: "dir_handle_discover": 0, 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays": 0, 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_delayed": 0, 2024-02-17T12:14:30.616 INFO:teuthology.orchestra.run.smithi189.stdout: "num_strays_enqueuing": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_created": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_enqueued": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_reintegrated": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "strays_migrated": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_processing": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_enqueued": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "num_recovering_prioritized": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_started": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "recovery_completed": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_enqueue_scrub": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_exportdir": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_flush": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragmentdir": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_fragstats": 0, 2024-02-17T12:14:30.617 INFO:teuthology.orchestra.run.smithi189.stdout: "ireq_inodestats": 0 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_log": { 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "evlrg": 0, 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "evadd": 0, 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "evex": 0, 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "evtrm": 0, 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "ev": 2563, 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "evexg": 0, 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "evexd": 0, 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "segadd": 0, 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "segex": 0, 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "segtrm": 0, 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "seg": 3, 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "segmjr": 39, 2024-02-17T12:14:30.618 INFO:teuthology.orchestra.run.smithi189.stdout: "segexg": 0, 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: "segexd": 0, 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: "expos": 51226223, 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: "wrpos": 52443802, 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: "rdpos": 52443690, 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: "jlat": { 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: "replayed": 2563 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_mem": { 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: "ino": 313, 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: "ino+": 451, 2024-02-17T12:14:30.619 INFO:teuthology.orchestra.run.smithi189.stdout: "ino-": 138, 2024-02-17T12:14:30.620 INFO:teuthology.orchestra.run.smithi189.stdout: "dir": 41, 2024-02-17T12:14:30.620 INFO:teuthology.orchestra.run.smithi189.stdout: "dir+": 41, 2024-02-17T12:14:30.620 INFO:teuthology.orchestra.run.smithi189.stdout: "dir-": 0, 2024-02-17T12:14:30.620 INFO:teuthology.orchestra.run.smithi189.stdout: "dn": 743, 2024-02-17T12:14:30.620 INFO:teuthology.orchestra.run.smithi189.stdout: "dn+": 743, 2024-02-17T12:14:30.620 INFO:teuthology.orchestra.run.smithi189.stdout: "dn-": 0, 2024-02-17T12:14:30.620 INFO:teuthology.orchestra.run.smithi189.stdout: "cap": 0, 2024-02-17T12:14:30.620 INFO:teuthology.orchestra.run.smithi189.stdout: "cap+": 0, 2024-02-17T12:14:30.620 INFO:teuthology.orchestra.run.smithi189.stdout: "cap-": 0, 2024-02-17T12:14:30.620 INFO:teuthology.orchestra.run.smithi189.stdout: "rss": 53252, 2024-02-17T12:14:30.620 INFO:teuthology.orchestra.run.smithi189.stdout: "heap": 215420 2024-02-17T12:14:30.620 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.620 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_server": { 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_client_request": 0, 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "dispatch_server_request": 0, 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_request": 0, 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_client_session": 0, 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "handle_peer_request": 0, 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "req_create_latency": { 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getattr_latency": { 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.621 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getfilelock_latency": { 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: "req_link_latency": { 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookup_latency": { 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookuphash_latency": { 2024-02-17T12:14:30.622 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupino_latency": { 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupname_latency": { 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupparent_latency": { 2024-02-17T12:14:30.623 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lookupsnap_latency": { 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: "req_lssnap_latency": { 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.624 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mkdir_latency": { 2024-02-17T12:14:30.625 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.625 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.625 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.625 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.626 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mknod_latency": { 2024-02-17T12:14:30.626 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.626 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.626 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.626 INFO:teuthology.orchestra.run.smithi189.stdout: "req_mksnap_latency": { 2024-02-17T12:14:30.626 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.626 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.626 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.626 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.626 INFO:teuthology.orchestra.run.smithi189.stdout: "req_open_latency": { 2024-02-17T12:14:30.626 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.626 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.627 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.627 INFO:teuthology.orchestra.run.smithi189.stdout: "req_readdir_latency": { 2024-02-17T12:14:30.627 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.627 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.627 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.627 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rename_latency": { 2024-02-17T12:14:30.627 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.627 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.627 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.627 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.627 INFO:teuthology.orchestra.run.smithi189.stdout: "req_renamesnap_latency": { 2024-02-17T12:14:30.628 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.628 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.628 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.628 INFO:teuthology.orchestra.run.smithi189.stdout: "req_snapdiff_latency": { 2024-02-17T12:14:30.628 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.628 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.628 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.628 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.628 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmdir_latency": { 2024-02-17T12:14:30.628 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.628 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.628 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmsnap_latency": { 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: "req_rmxattr_latency": { 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setattr_latency": { 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.629 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setdirlayout_latency": { 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setfilelock_latency": { 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setlayout_latency": { 2024-02-17T12:14:30.630 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "req_setxattr_latency": { 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "req_symlink_latency": { 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.631 INFO:teuthology.orchestra.run.smithi189.stdout: "req_unlink_latency": { 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_revoke_eviction": 0, 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "cap_acquisition_throttle": 0, 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "req_getvxattr_latency": { 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_sessions": { 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "session_count": 2, 2024-02-17T12:14:30.632 INFO:teuthology.orchestra.run.smithi189.stdout: "session_add": 2, 2024-02-17T12:14:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "session_remove": 0, 2024-02-17T12:14:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_open": 0, 2024-02-17T12:14:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "sessions_stale": 0, 2024-02-17T12:14:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "total_load": 0, 2024-02-17T12:14:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "average_load": 0, 2024-02-17T12:14:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "avg_session_uptime": 2298, 2024-02-17T12:14:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "mdthresh_evicted": 0 2024-02-17T12:14:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "mempool": { 2024-02-17T12:14:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_bytes": 0, 2024-02-17T12:14:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bloom_filter_items": 0, 2024-02-17T12:14:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_bytes": 0, 2024-02-17T12:14:30.633 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_alloc_items": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_bytes": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_data_items": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_onode_items": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_meta_items": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_bytes": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_other_items": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_cache_buffer_items": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_bytes": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_extent_items": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_bytes": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_blob_items": 0, 2024-02-17T12:14:30.634 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_shared_blob_items": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_inline_bl_items": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_bytes": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_fsck_items": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_bytes": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_txc_items": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_deferred_items": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_bytes": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluestore_writing_items": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_bytes": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_items": 0, 2024-02-17T12:14:30.635 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_bytes": 0, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_reader_items": 0, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_bytes": 0, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "bluefs_file_writer_items": 0, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_bytes": 229551, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_anon_items": 680, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_bytes": 0, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "buffer_meta_items": 0, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_bytes": 0, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_items": 0, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_bytes": 0, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_mapbl_items": 0, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_bytes": 0, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_pglog_items": 0, 2024-02-17T12:14:30.636 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_bytes": 8352, 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_items": 190, 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_bytes": 0, 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "osdmap_mapping_items": 0, 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_bytes": 0, 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "pgmap_items": 0, 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_bytes": 1651974, 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_co_items": 54781, 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_bytes": 0, 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_1_items": 0, 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_bytes": 0, 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "unittest_2_items": 0 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "objecter": { 2024-02-17T12:14:30.637 INFO:teuthology.orchestra.run.smithi189.stdout: "op_active": 0, 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "op_laggy": 0, 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send": 1876, 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "op_send_bytes": 26, 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "op_resend": 4, 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "op_reply": 1876, 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "op_latency": { 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1876, 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 6.948860219, 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.003704083 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "op_inflight": 0, 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "oplen_avg": { 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 1876, 2024-02-17T12:14:30.638 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 1877 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "op": 1876, 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "op_r": 1876, 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "op_w": 0, 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "op_rmw": 0, 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "op_pg": 0, 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_stat": 1100, 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_create": 0, 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_read": 773, 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_write": 0, 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writefull": 0, 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_writesame": 0, 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_append": 0, 2024-02-17T12:14:30.639 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_zero": 0, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_truncate": 0, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_delete": 0, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_mapext": 0, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_sparse_read": 0, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_clonerange": 0, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_getxattr": 2, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_setxattr": 0, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_cmpxattr": 0, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_rmxattr": 0, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_resetxattrs": 0, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_call": 0, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_watch": 0, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_notify": 0, 2024-02-17T12:14:30.640 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_src_cmpxattr": 0, 2024-02-17T12:14:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls": 0, 2024-02-17T12:14:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_pgls_filter": 0, 2024-02-17T12:14:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "osdop_other": 0, 2024-02-17T12:14:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_active": 0, 2024-02-17T12:14:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_send": 0, 2024-02-17T12:14:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_resend": 0, 2024-02-17T12:14:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "linger_ping": 0, 2024-02-17T12:14:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_active": 0, 2024-02-17T12:14:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_send": 0, 2024-02-17T12:14:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "poolop_resend": 0, 2024-02-17T12:14:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_active": 0, 2024-02-17T12:14:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_send": 0, 2024-02-17T12:14:30.641 INFO:teuthology.orchestra.run.smithi189.stdout: "poolstat_resend": 0, 2024-02-17T12:14:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_active": 0, 2024-02-17T12:14:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_send": 0, 2024-02-17T12:14:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "statfs_resend": 0, 2024-02-17T12:14:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "command_active": 0, 2024-02-17T12:14:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "command_send": 0, 2024-02-17T12:14:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "command_resend": 0, 2024-02-17T12:14:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "map_epoch": 108, 2024-02-17T12:14:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "map_full": 0, 2024-02-17T12:14:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "map_inc": 18, 2024-02-17T12:14:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_sessions": 7, 2024-02-17T12:14:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_open": 7, 2024-02-17T12:14:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_session_close": 0, 2024-02-17T12:14:30.642 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_laggy": 0, 2024-02-17T12:14:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_wr": 0, 2024-02-17T12:14:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_rd": 2, 2024-02-17T12:14:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_del": 0 2024-02-17T12:14:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "oft": { 2024-02-17T12:14:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_objs": 0, 2024-02-17T12:14:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_kv_pairs": 0, 2024-02-17T12:14:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_updates": 0, 2024-02-17T12:14:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "omap_total_removes": 0 2024-02-17T12:14:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "osd-slow-ops": { 2024-02-17T12:14:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: "slow_ops_count": 0 2024-02-17T12:14:30.643 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "purge_queue": { 2024-02-17T12:14:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops": 0, 2024-02-17T12:14:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_ops_high_water": 0, 2024-02-17T12:14:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing": 0, 2024-02-17T12:14:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executing_high_water": 0, 2024-02-17T12:14:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed_ops": 0, 2024-02-17T12:14:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_executed": 0, 2024-02-17T12:14:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "pq_item_in_journal": 0 2024-02-17T12:14:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-17T12:14:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-02-17T12:14:30.644 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 2054, 2024-02-17T12:14:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 25910000, 2024-02-17T12:14:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 2054, 2024-02-17T12:14:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-02-17T12:14:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-02-17T12:14:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 2054, 2024-02-17T12:14:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 25910000, 2024-02-17T12:14:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.645 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_bytes": { 2024-02-17T12:14:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 104857600, 2024-02-17T12:14:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-02-17T12:14:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-02-17T12:14:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 1876, 2024-02-17T12:14:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 25368630, 2024-02-17T12:14:30.646 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 1876, 2024-02-17T12:14:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 25368630, 2024-02-17T12:14:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-objecter_ops": { 2024-02-17T12:14:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 1024, 2024-02-17T12:14:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:30.647 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-02-17T12:14:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-02-17T12:14:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 1876, 2024-02-17T12:14:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 1876, 2024-02-17T12:14:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 1876, 2024-02-17T12:14:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 1876, 2024-02-17T12:14:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.648 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle": { 2024-02-17T12:14:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-02-17T12:14:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-02-17T12:14:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-02-17T12:14:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-02-17T12:14:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-02-17T12:14:30.649 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-02-17T12:14:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-02-17T12:14:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-02-17T12:14:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "throttle-write_buf_throttle-0x55a7be388960": { 2024-02-17T12:14:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "val": 0, 2024-02-17T12:14:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "max": 3758096384, 2024-02-17T12:14:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "get_started": 0, 2024-02-17T12:14:30.650 INFO:teuthology.orchestra.run.smithi189.stdout: "get": 0, 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "get_sum": 0, 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_fail": 0, 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "get_or_fail_success": 0, 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "take": 0, 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "take_sum": 0, 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "put": 0, 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "put_sum": 0, 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "wait": { 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "avgcount": 0, 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "sum": 0.000000000, 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: "avgtime": 0.000000000 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-02-17T12:14:30.651 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-02-17T12:14:31.416 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-02-17T12:14:31.417 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-02-17T12:14:31.417 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=7 2024-02-17T12:14:31.417 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-02-17T12:14:31.417 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-02-17T12:14:31.417 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/github.com_ceph_ceph-c_6822c7b54e949006be7a0802650d01723d834d1d/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-02-17T12:14:31.418 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-17T12:14:31.430 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-17T12:14:31.430 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T12:14:31.430 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-17T12:14:31.465 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T12:14:31.465 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-17T12:14:31.526 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:14:31.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:31 smithi189 ceph-mon[36588]: pgmap v515: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T12:14:31.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:31 smithi125 ceph-mon[36819]: pgmap v515: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T12:14:31.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.704+0000 7f77165e2640 1 -- 172.21.15.94:0/2996335093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77101064a0 msgr2=0x7f771010e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:31.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.704+0000 7f77165e2640 1 --2- 172.21.15.94:0/2996335093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77101064a0 0x7f771010e070 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f7708009cd0 tx=0x7f770802c0f0 comp rx=0 tx=0).stop 2024-02-17T12:14:31.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.704+0000 7f77165e2640 1 -- 172.21.15.94:0/2996335093 shutdown_connections 2024-02-17T12:14:31.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.704+0000 7f77165e2640 1 --2- 172.21.15.94:0/2996335093 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f77101064a0 0x7f771010e070 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:31.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.704+0000 7f77165e2640 1 --2- 172.21.15.94:0/2996335093 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f771006a450 0x7f7710105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:31.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.704+0000 7f77165e2640 1 --2- 172.21.15.94:0/2996335093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7710069aa0 0x7f7710069e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:31.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.704+0000 7f77165e2640 1 -- 172.21.15.94:0/2996335093 >> 172.21.15.94:0/2996335093 conn(0x7f7710077120 msgr2=0x7f7710077530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:31.708 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.705+0000 7f77165e2640 1 -- 172.21.15.94:0/2996335093 shutdown_connections 2024-02-17T12:14:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.705+0000 7f77165e2640 1 -- 172.21.15.94:0/2996335093 wait complete. 2024-02-17T12:14:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.705+0000 7f77165e2640 1 Processor -- start 2024-02-17T12:14:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.705+0000 7f77165e2640 1 -- start start 2024-02-17T12:14:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f77165e2640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710069aa0 0x7f771019d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f77165e2640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77101064a0 0x7f771019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f77165e2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7710197640 0x7f7710197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f77165e2640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7710198110 con 0x7f7710069aa0 2024-02-17T12:14:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f77165e2640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7710198280 con 0x7f77101064a0 2024-02-17T12:14:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f77165e2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7710198440 con 0x7f7710197640 2024-02-17T12:14:31.709 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f770ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710069aa0 0x7f771019d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:31.710 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:31 smithi094 ceph-mon[32499]: pgmap v515: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-02-17T12:14:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f770ffff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710069aa0 0x7f771019d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56770/0 (socket says 172.21.15.94:56770) 2024-02-17T12:14:31.711 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f770ffff640 1 -- 172.21.15.94:0/83518455 learned_addr learned my addr 172.21.15.94:0/83518455 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:14:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f7714b7f640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7710197640 0x7f7710197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f770f7fe640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77101064a0 0x7f771019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f770ffff640 1 -- 172.21.15.94:0/83518455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7710197640 msgr2=0x7f7710197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f770ffff640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7710197640 0x7f7710197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f770ffff640 1 -- 172.21.15.94:0/83518455 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77101064a0 msgr2=0x7f771019dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f770ffff640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77101064a0 0x7f771019dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f770ffff640 1 -- 172.21.15.94:0/83518455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7708009900 con 0x7f7710069aa0 2024-02-17T12:14:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f770ffff640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710069aa0 0x7f771019d580 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f770000d950 tx=0x7f770000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.706+0000 7f770d7fa640 1 -- 172.21.15.94:0/83518455 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7700009a30 con 0x7f7710069aa0 2024-02-17T12:14:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.707+0000 7f770d7fa640 1 -- 172.21.15.94:0/83518455 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7700009b90 con 0x7f7710069aa0 2024-02-17T12:14:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.707+0000 7f77165e2640 1 -- 172.21.15.94:0/83518455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77101a43e0 con 0x7f7710069aa0 2024-02-17T12:14:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.707+0000 7f770d7fa640 1 -- 172.21.15.94:0/83518455 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7700004a00 con 0x7f7710069aa0 2024-02-17T12:14:31.712 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.707+0000 7f77165e2640 1 -- 172.21.15.94:0/83518455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77101a48c0 con 0x7f7710069aa0 2024-02-17T12:14:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.708+0000 7f77165e2640 1 -- 172.21.15.94:0/83518455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77101028a0 con 0x7f7710069aa0 2024-02-17T12:14:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.709+0000 7f770d7fa640 1 -- 172.21.15.94:0/83518455 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f7700009d00 con 0x7f7710069aa0 2024-02-17T12:14:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.709+0000 7f770d7fa640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f76f80a2230 0x7f76f80a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.709+0000 7f770f7fe640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f76f80a2230 0x7f76f80a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.709+0000 7f770d7fa640 1 -- 172.21.15.94:0/83518455 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f77000cda60 con 0x7f7710069aa0 2024-02-17T12:14:31.713 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.710+0000 7f770f7fe640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f76f80a2230 0x7f76f80a46f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7710198e90 tx=0x7f770400b390 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:31.716 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.713+0000 7f770d7fa640 1 -- 172.21.15.94:0/83518455 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f77000966b0 con 0x7f7710069aa0 2024-02-17T12:14:31.889 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.887+0000 7f77165e2640 1 -- 172.21.15.94:0/83518455 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77101096c0 con 0x7f7710069aa0 2024-02-17T12:14:31.890 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.888+0000 7f770d7fa640 1 -- 172.21.15.94:0/83518455 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+23218 (secure 0 0 0) 0x7f7700099d70 con 0x7f7710069aa0 2024-02-17T12:14:31.891 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:14:31.891 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":108,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T12:14:15.368567+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":26,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2639307451":"2024-02-18T11:58:06.744891+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/875984108":"2024-02-18T11:58:06.744891+0000","172.21.15.94:0/2865658350":"2024-02-18T11:58:06.744891+0000","172.21.15.94:0/142548766":"2024-02-18T11:58:06.744891+0000","172.21.15.94:0/1603490109":"2024-02-18T11:58:06.744891+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6801/1808883811":"2024-02-18T11:58:06.744891+0000","172.21.15.94:6800/1808883811":"2024-02-18T11:58:06.744891+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":13,"length":1},{"begin":26,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:14:31.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.890+0000 7f77165e2640 1 -- 172.21.15.94:0/83518455 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f76f80a2230 msgr2=0x7f76f80a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:31.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.890+0000 7f77165e2640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f76f80a2230 0x7f76f80a46f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7710198e90 tx=0x7f770400b390 comp rx=0 tx=0).stop 2024-02-17T12:14:31.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.890+0000 7f77165e2640 1 -- 172.21.15.94:0/83518455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710069aa0 msgr2=0x7f771019d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:31.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.890+0000 7f77165e2640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710069aa0 0x7f771019d580 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f770000d950 tx=0x7f770000de10 comp rx=0 tx=0).stop 2024-02-17T12:14:31.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.891+0000 7f77165e2640 1 -- 172.21.15.94:0/83518455 shutdown_connections 2024-02-17T12:14:31.893 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.891+0000 7f77165e2640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f76f80a2230 0x7f76f80a46f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:31.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.891+0000 7f77165e2640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7710197640 0x7f7710197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:31.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.891+0000 7f77165e2640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f77101064a0 0x7f771019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:31.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.891+0000 7f77165e2640 1 --2- 172.21.15.94:0/83518455 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f7710069aa0 0x7f771019d580 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:31.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.891+0000 7f77165e2640 1 -- 172.21.15.94:0/83518455 >> 172.21.15.94:0/83518455 conn(0x7f7710077120 msgr2=0x7f77100fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:31.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.891+0000 7f77165e2640 1 -- 172.21.15.94:0/83518455 shutdown_connections 2024-02-17T12:14:31.894 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:31.891+0000 7f77165e2640 1 -- 172.21.15.94:0/83518455 wait complete. 2024-02-17T12:14:31.904 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-17T11:51:20.894601+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-17T11:55:56.349040+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-17T11:55:57.234916+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 26, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-17T12:14:31.905 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-17T12:14:32.016 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.012+0000 7fe8c89d0640 1 -- 172.21.15.94:0/3793602965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c4100c10 msgr2=0x7fe8c4104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.012+0000 7fe8c89d0640 1 --2- 172.21.15.94:0/3793602965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c4100c10 0x7fe8c4104c50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009930 tx=0x7fe8ac02bc30 comp rx=0 tx=0).stop 2024-02-17T12:14:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.012+0000 7fe8c89d0640 1 -- 172.21.15.94:0/3793602965 shutdown_connections 2024-02-17T12:14:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.012+0000 7fe8c89d0640 1 --2- 172.21.15.94:0/3793602965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8c4105330 0x7fe8c410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.012+0000 7fe8c89d0640 1 --2- 172.21.15.94:0/3793602965 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c4100c10 0x7fe8c4104c50 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.012+0000 7fe8c89d0640 1 --2- 172.21.15.94:0/3793602965 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c4100260 0x7fe8c4100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.012+0000 7fe8c89d0640 1 -- 172.21.15.94:0/3793602965 >> 172.21.15.94:0/3793602965 conn(0x7fe8c4077180 msgr2=0x7fe8c4077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.012+0000 7fe8c89d0640 1 -- 172.21.15.94:0/3793602965 shutdown_connections 2024-02-17T12:14:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.012+0000 7fe8c89d0640 1 -- 172.21.15.94:0/3793602965 wait complete. 2024-02-17T12:14:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c89d0640 1 Processor -- start 2024-02-17T12:14:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c89d0640 1 -- start start 2024-02-17T12:14:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c89d0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8c4100260 0x7fe8c4198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c89d0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c4105330 0x7fe8c4198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.017 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c89d0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c41990e0 0x7fe8c41a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c89d0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c41a3d30 con 0x7fe8c4105330 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c89d0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c41a3ea0 con 0x7fe8c41990e0 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c89d0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c41a4060 con 0x7fe8c4100260 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c4105330 0x7fe8c4198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c1d9b640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c4105330 0x7fe8c4198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56798/0 (socket says 172.21.15.94:56798) 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c1d9b640 1 -- 172.21.15.94:0/295596229 learned_addr learned my addr 172.21.15.94:0/295596229 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c259c640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8c4100260 0x7fe8c4198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c2d9d640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c41990e0 0x7fe8c41a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c1d9b640 1 -- 172.21.15.94:0/295596229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8c4100260 msgr2=0x7fe8c4198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c1d9b640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8c4100260 0x7fe8c4198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c1d9b640 1 -- 172.21.15.94:0/295596229 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c41990e0 msgr2=0x7fe8c41a3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c1d9b640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c41990e0 0x7fe8c41a3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c1d9b640 1 -- 172.21.15.94:0/295596229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac009560 con 0x7fe8c4105330 2024-02-17T12:14:32.018 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.013+0000 7fe8c259c640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8c4100260 0x7fe8c4198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:14:32.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.014+0000 7fe8c1d9b640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c4105330 0x7fe8c4198b50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009230 tx=0x7fe8ac02c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:32.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.014+0000 7fe8bb7fe640 1 -- 172.21.15.94:0/295596229 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac004350 con 0x7fe8c4105330 2024-02-17T12:14:32.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.014+0000 7fe8c89d0640 1 -- 172.21.15.94:0/295596229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c41a42e0 con 0x7fe8c4105330 2024-02-17T12:14:32.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.014+0000 7fe8bb7fe640 1 -- 172.21.15.94:0/295596229 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe8ac0044b0 con 0x7fe8c4105330 2024-02-17T12:14:32.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.014+0000 7fe8bb7fe640 1 -- 172.21.15.94:0/295596229 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac003da0 con 0x7fe8c4105330 2024-02-17T12:14:32.019 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.014+0000 7fe8c89d0640 1 -- 172.21.15.94:0/295596229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c41a4840 con 0x7fe8c4105330 2024-02-17T12:14:32.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.015+0000 7fe8c89d0640 1 -- 172.21.15.94:0/295596229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c4067f70 con 0x7fe8c4105330 2024-02-17T12:14:32.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.017+0000 7fe8bb7fe640 1 -- 172.21.15.94:0/295596229 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fe8ac004df0 con 0x7fe8c4105330 2024-02-17T12:14:32.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.017+0000 7fe8bb7fe640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe8b00a2230 0x7fe8b00a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.017+0000 7fe8c259c640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe8b00a2230 0x7fe8b00a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.017+0000 7fe8bb7fe640 1 -- 172.21.15.94:0/295596229 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fe8ac0f0080 con 0x7fe8c4105330 2024-02-17T12:14:32.020 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.018+0000 7fe8c259c640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe8b00a2230 0x7fe8b00a46f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009fc0 tx=0x7fe8bc005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:32.024 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.021+0000 7fe8bb7fe640 1 -- 172.21.15.94:0/295596229 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe8ac0b8de0 con 0x7fe8c4105330 2024-02-17T12:14:32.205 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.203+0000 7fe8c89d0640 1 -- 172.21.15.94:0/295596229 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8c41a4470 con 0x7fe8c4105330 2024-02-17T12:14:32.206 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.204+0000 7fe8bb7fe640 1 -- 172.21.15.94:0/295596229 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8c41a4470 con 0x7fe8c4105330 2024-02-17T12:14:32.206 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 1 2024-02-17T12:14:32.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.206+0000 7fe8c89d0640 1 -- 172.21.15.94:0/295596229 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe8b00a2230 msgr2=0x7fe8b00a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.206+0000 7fe8c89d0640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe8b00a2230 0x7fe8b00a46f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009fc0 tx=0x7fe8bc005f90 comp rx=0 tx=0).stop 2024-02-17T12:14:32.208 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.206+0000 7fe8c89d0640 1 -- 172.21.15.94:0/295596229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c4105330 msgr2=0x7fe8c4198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.206+0000 7fe8c89d0640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c4105330 0x7fe8c4198b50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009230 tx=0x7fe8ac02c680 comp rx=0 tx=0).stop 2024-02-17T12:14:32.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.206+0000 7fe8c89d0640 1 -- 172.21.15.94:0/295596229 shutdown_connections 2024-02-17T12:14:32.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.206+0000 7fe8c89d0640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fe8b00a2230 0x7fe8b00a46f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.206+0000 7fe8c89d0640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fe8c41990e0 0x7fe8c41a3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.206+0000 7fe8c89d0640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fe8c4105330 0x7fe8c4198b50 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.206+0000 7fe8c89d0640 1 --2- 172.21.15.94:0/295596229 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8c4100260 0x7fe8c4198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.206+0000 7fe8c89d0640 1 -- 172.21.15.94:0/295596229 >> 172.21.15.94:0/295596229 conn(0x7fe8c4077180 msgr2=0x7fe8c40fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:32.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.206+0000 7fe8c89d0640 1 -- 172.21.15.94:0/295596229 shutdown_connections 2024-02-17T12:14:32.209 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.206+0000 7fe8c89d0640 1 -- 172.21.15.94:0/295596229 wait complete. 2024-02-17T12:14:32.219 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-17T12:14:32.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.328+0000 7fcf83eb5640 1 -- 172.21.15.94:0/204503890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf7c0685d0 msgr2=0x7fcf7c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.328+0000 7fcf83eb5640 1 --2- 172.21.15.94:0/204503890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf7c0685d0 0x7fcf7c0689b0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009930 tx=0x7fcf6c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:14:32.332 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.328+0000 7fcf83eb5640 1 -- 172.21.15.94:0/204503890 shutdown_connections 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.328+0000 7fcf83eb5640 1 --2- 172.21.15.94:0/204503890 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf7c106430 0x7fcf7c10e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.328+0000 7fcf83eb5640 1 --2- 172.21.15.94:0/204503890 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf7c068f80 0x7fcf7c105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.328+0000 7fcf83eb5640 1 --2- 172.21.15.94:0/204503890 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf7c0685d0 0x7fcf7c0689b0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.328+0000 7fcf83eb5640 1 -- 172.21.15.94:0/204503890 >> 172.21.15.94:0/204503890 conn(0x7fcf7c077180 msgr2=0x7fcf7c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.328+0000 7fcf83eb5640 1 -- 172.21.15.94:0/204503890 shutdown_connections 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.328+0000 7fcf83eb5640 1 -- 172.21.15.94:0/204503890 wait complete. 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.328+0000 7fcf83eb5640 1 Processor -- start 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.328+0000 7fcf83eb5640 1 -- start start 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf83eb5640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf7c0685d0 0x7fcf7c19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf83eb5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf7c106430 0x7fcf7c19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf83eb5640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf7c197590 0x7fcf7c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf83eb5640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c197f80 con 0x7fcf7c197590 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf83eb5640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1980f0 con 0x7fcf7c0685d0 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf83eb5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c1982b0 con 0x7fcf7c106430 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf82452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf7c197590 0x7fcf7c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.333 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf82452640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf7c197590 0x7fcf7c197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56808/0 (socket says 172.21.15.94:56808) 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf82452640 1 -- 172.21.15.94:0/1583021420 learned_addr learned my addr 172.21.15.94:0/1583021420 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf81c51640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf7c0685d0 0x7fcf7c19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf81450640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf7c106430 0x7fcf7c19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf82452640 1 -- 172.21.15.94:0/1583021420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf7c106430 msgr2=0x7fcf7c19d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf82452640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf7c106430 0x7fcf7c19d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf82452640 1 -- 172.21.15.94:0/1583021420 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf7c0685d0 msgr2=0x7fcf7c19d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf82452640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf7c0685d0 0x7fcf7c19d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf82452640 1 -- 172.21.15.94:0/1583021420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c009560 con 0x7fcf7c197590 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf82452640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf7c197590 0x7fcf7c197a40 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400cad0 tx=0x7fcf7400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.329+0000 7fcf72ffd640 1 -- 172.21.15.94:0/1583021420 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400ad60 con 0x7fcf7c197590 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.330+0000 7fcf72ffd640 1 -- 172.21.15.94:0/1583021420 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf74005020 con 0x7fcf7c197590 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.330+0000 7fcf83eb5640 1 -- 172.21.15.94:0/1583021420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7c1a42a0 con 0x7fcf7c197590 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.330+0000 7fcf72ffd640 1 -- 172.21.15.94:0/1583021420 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400f6b0 con 0x7fcf7c197590 2024-02-17T12:14:32.334 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.330+0000 7fcf83eb5640 1 -- 172.21.15.94:0/1583021420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7c1a4760 con 0x7fcf7c197590 2024-02-17T12:14:32.335 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.331+0000 7fcf83eb5640 1 -- 172.21.15.94:0/1583021420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7c067f70 con 0x7fcf7c197590 2024-02-17T12:14:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.332+0000 7fcf72ffd640 1 -- 172.21.15.94:0/1583021420 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fcf74005520 con 0x7fcf7c197590 2024-02-17T12:14:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.332+0000 7fcf72ffd640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcf680a2230 0x7fcf680a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.333+0000 7fcf81c51640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcf680a2230 0x7fcf680a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.333+0000 7fcf72ffd640 1 -- 172.21.15.94:0/1583021420 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fcf740ce430 con 0x7fcf7c197590 2024-02-17T12:14:32.336 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.333+0000 7fcf81c51640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcf680a2230 0x7fcf680a46f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c005ab0 tx=0x7fcf6c005fd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:32.338 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.336+0000 7fcf72ffd640 1 -- 172.21.15.94:0/1583021420 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf74097080 con 0x7fcf7c197590 2024-02-17T12:14:32.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:32 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/83518455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:14:32.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:32 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/295596229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:14:32.517 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.515+0000 7fcf83eb5640 1 -- 172.21.15.94:0/1583021420 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcf7c1096c0 con 0x7fcf7c197590 2024-02-17T12:14:32.518 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.515+0000 7fcf72ffd640 1 -- 172.21.15.94:0/1583021420 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcf7401d0a0 con 0x7fcf7c197590 2024-02-17T12:14:32.518 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 32 2024-02-17T12:14:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.517+0000 7fcf83eb5640 1 -- 172.21.15.94:0/1583021420 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcf680a2230 msgr2=0x7fcf680a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.517+0000 7fcf83eb5640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcf680a2230 0x7fcf680a46f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c005ab0 tx=0x7fcf6c005fd0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.517+0000 7fcf83eb5640 1 -- 172.21.15.94:0/1583021420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf7c197590 msgr2=0x7fcf7c197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.520 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.517+0000 7fcf83eb5640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf7c197590 0x7fcf7c197a40 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400cad0 tx=0x7fcf7400ced0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.517+0000 7fcf83eb5640 1 -- 172.21.15.94:0/1583021420 shutdown_connections 2024-02-17T12:14:32.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.517+0000 7fcf83eb5640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fcf680a2230 0x7fcf680a46f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.517+0000 7fcf83eb5640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fcf7c197590 0x7fcf7c197a40 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.517+0000 7fcf83eb5640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf7c106430 0x7fcf7c19d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.517+0000 7fcf83eb5640 1 --2- 172.21.15.94:0/1583021420 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fcf7c0685d0 0x7fcf7c19d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.517+0000 7fcf83eb5640 1 -- 172.21.15.94:0/1583021420 >> 172.21.15.94:0/1583021420 conn(0x7fcf7c077180 msgr2=0x7fcf7c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:32.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.517+0000 7fcf83eb5640 1 -- 172.21.15.94:0/1583021420 shutdown_connections 2024-02-17T12:14:32.521 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.517+0000 7fcf83eb5640 1 -- 172.21.15.94:0/1583021420 wait complete. 2024-02-17T12:14:32.531 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-17T12:14:32.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.636+0000 7f0122fe9640 1 -- 172.21.15.94:0/336846376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f011c105330 msgr2=0x7f011c10cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.636+0000 7f0122fe9640 1 --2- 172.21.15.94:0/336846376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f011c105330 0x7f011c10cf00 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f010c009930 tx=0x7f010c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:14:32.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.636+0000 7f0122fe9640 1 -- 172.21.15.94:0/336846376 shutdown_connections 2024-02-17T12:14:32.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.636+0000 7f0122fe9640 1 --2- 172.21.15.94:0/336846376 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f011c105330 0x7f011c10cf00 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.641 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.636+0000 7f0122fe9640 1 --2- 172.21.15.94:0/336846376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f011c100c10 0x7f011c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.636+0000 7f0122fe9640 1 --2- 172.21.15.94:0/336846376 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f011c100260 0x7f011c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.636+0000 7f0122fe9640 1 -- 172.21.15.94:0/336846376 >> 172.21.15.94:0/336846376 conn(0x7f011c077180 msgr2=0x7f011c077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.636+0000 7f0122fe9640 1 -- 172.21.15.94:0/336846376 shutdown_connections 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.636+0000 7f0122fe9640 1 -- 172.21.15.94:0/336846376 wait complete. 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0122fe9640 1 Processor -- start 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0122fe9640 1 -- start start 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0122fe9640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f011c100260 0x7f011c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0122fe9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f011c105330 0x7f011c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0122fe9640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f011c199830 0x7f011c199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0122fe9640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c19a2e0 con 0x7f011c199830 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0122fe9640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c19a450 con 0x7f011c100260 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0122fe9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c19a5c0 con 0x7f011c105330 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0121586640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f011c199830 0x7f011c199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0121586640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f011c199830 0x7f011c199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56828/0 (socket says 172.21.15.94:56828) 2024-02-17T12:14:32.642 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0121586640 1 -- 172.21.15.94:0/1432932976 learned_addr learned my addr 172.21.15.94:0/1432932976 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:14:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f011bfff640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f011c105330 0x7f011c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0120d85640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f011c100260 0x7f011c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0121586640 1 -- 172.21.15.94:0/1432932976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f011c105330 msgr2=0x7f011c19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0121586640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f011c105330 0x7f011c19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0121586640 1 -- 172.21.15.94:0/1432932976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f011c100260 msgr2=0x7f011c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0121586640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f011c100260 0x7f011c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0121586640 1 -- 172.21.15.94:0/1432932976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c009560 con 0x7f011c199830 2024-02-17T12:14:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f011bfff640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f011c105330 0x7f011c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:14:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0120d85640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f011c100260 0x7f011c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-17T12:14:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.637+0000 7f0121586640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f011c199830 0x7f011c199ce0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f010c005940 tx=0x7f010c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.638+0000 7f0119ffb640 1 -- 172.21.15.94:0/1432932976 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c0044c0 con 0x7f011c199830 2024-02-17T12:14:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.638+0000 7f0122fe9640 1 -- 172.21.15.94:0/1432932976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011c1a6510 con 0x7f011c199830 2024-02-17T12:14:32.643 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.638+0000 7f0119ffb640 1 -- 172.21.15.94:0/1432932976 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f010c0052d0 con 0x7f011c199830 2024-02-17T12:14:32.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.638+0000 7f0119ffb640 1 -- 172.21.15.94:0/1432932976 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c003e60 con 0x7f011c199830 2024-02-17T12:14:32.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.638+0000 7f0122fe9640 1 -- 172.21.15.94:0/1432932976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011c1a6a20 con 0x7f011c199830 2024-02-17T12:14:32.644 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.639+0000 7f0122fe9640 1 -- 172.21.15.94:0/1432932976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011c101730 con 0x7f011c199830 2024-02-17T12:14:32.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.640+0000 7f0119ffb640 1 -- 172.21.15.94:0/1432932976 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f010c005440 con 0x7f011c199830 2024-02-17T12:14:32.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.641+0000 7f0119ffb640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f01080a2230 0x7f01080a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.641+0000 7f0120d85640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f01080a2230 0x7f01080a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.641+0000 7f0119ffb640 1 -- 172.21.15.94:0/1432932976 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f010c0f0390 con 0x7f011c199830 2024-02-17T12:14:32.645 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.642+0000 7f0120d85640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f01080a2230 0x7f01080a46f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0114005c40 tx=0x7f01140082f0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:32.648 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.646+0000 7f0119ffb640 1 -- 172.21.15.94:0/1432932976 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f010c0b90f0 con 0x7f011c199830 2024-02-17T12:14:32.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:32 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/83518455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:14:32.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:32 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/295596229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:14:32.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:32 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/83518455' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:14:32.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:32 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/295596229' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-17T12:14:32.826 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.824+0000 7f0122fe9640 1 -- 172.21.15.94:0/1432932976 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f011c103330 con 0x7f011c199830 2024-02-17T12:14:32.827 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.824+0000 7f0119ffb640 1 -- 172.21.15.94:0/1432932976 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+11 (secure 0 0 0) 0x7f010c0035f0 con 0x7f011c199830 2024-02-17T12:14:32.827 INFO:teuthology.orchestra.run.smithi094.stdout:pg_num: 64 2024-02-17T12:14:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.826+0000 7f0122fe9640 1 -- 172.21.15.94:0/1432932976 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f01080a2230 msgr2=0x7f01080a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.826+0000 7f0122fe9640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f01080a2230 0x7f01080a46f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0114005c40 tx=0x7f01140082f0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.826+0000 7f0122fe9640 1 -- 172.21.15.94:0/1432932976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f011c199830 msgr2=0x7f011c199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.826+0000 7f0122fe9640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f011c199830 0x7f011c199ce0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f010c005940 tx=0x7f010c0040c0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.826+0000 7f0122fe9640 1 -- 172.21.15.94:0/1432932976 shutdown_connections 2024-02-17T12:14:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.826+0000 7f0122fe9640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f01080a2230 0x7f01080a46f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.826+0000 7f0122fe9640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f011c199830 0x7f011c199ce0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.826+0000 7f0122fe9640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f011c105330 0x7f011c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.826+0000 7f0122fe9640 1 --2- 172.21.15.94:0/1432932976 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f011c100260 0x7f011c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.829 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.826+0000 7f0122fe9640 1 -- 172.21.15.94:0/1432932976 >> 172.21.15.94:0/1432932976 conn(0x7f011c077180 msgr2=0x7f011c068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:32.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.826+0000 7f0122fe9640 1 -- 172.21.15.94:0/1432932976 shutdown_connections 2024-02-17T12:14:32.830 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.826+0000 7f0122fe9640 1 -- 172.21.15.94:0/1432932976 wait complete. 2024-02-17T12:14:32.840 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-17T12:14:32.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.940+0000 7f2b6858c640 1 -- 172.21.15.94:0/2316574533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b60100c10 msgr2=0x7f2b60104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.940+0000 7f2b6858c640 1 --2- 172.21.15.94:0/2316574533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b60100c10 0x7f2b60104c50 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f2b50009930 tx=0x7f2b5002bc30 comp rx=0 tx=0).stop 2024-02-17T12:14:32.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 -- 172.21.15.94:0/2316574533 shutdown_connections 2024-02-17T12:14:32.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 --2- 172.21.15.94:0/2316574533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b60105330 0x7f2b6010cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 --2- 172.21.15.94:0/2316574533 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b60100c10 0x7f2b60104c50 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 --2- 172.21.15.94:0/2316574533 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b60100260 0x7f2b60100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 -- 172.21.15.94:0/2316574533 >> 172.21.15.94:0/2316574533 conn(0x7f2b60077180 msgr2=0x7f2b60077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:32.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 -- 172.21.15.94:0/2316574533 shutdown_connections 2024-02-17T12:14:32.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 -- 172.21.15.94:0/2316574533 wait complete. 2024-02-17T12:14:32.946 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 Processor -- start 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 -- start start 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b60100260 0x7f2b6019d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b60105330 0x7f2b6019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b60197650 0x7f2b60197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60198120 con 0x7f2b60100260 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60198290 con 0x7f2b60197650 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b6858c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60198450 con 0x7f2b60105330 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b66328640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b60100260 0x7f2b6019d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b66328640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b60100260 0x7f2b6019d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56846/0 (socket says 172.21.15.94:56846) 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.941+0000 7f2b66328640 1 -- 172.21.15.94:0/229858872 learned_addr learned my addr 172.21.15.94:0/229858872 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.942+0000 7f2b66b29640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b60197650 0x7f2b60197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.942+0000 7f2b66328640 1 -- 172.21.15.94:0/229858872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b60105330 msgr2=0x7f2b6019dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.942+0000 7f2b66328640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b60105330 0x7f2b6019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.942+0000 7f2b66328640 1 -- 172.21.15.94:0/229858872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b60197650 msgr2=0x7f2b60197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:32.947 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.942+0000 7f2b66328640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b60197650 0x7f2b60197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:32.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.942+0000 7f2b66328640 1 -- 172.21.15.94:0/229858872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b50009560 con 0x7f2b60100260 2024-02-17T12:14:32.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.942+0000 7f2b66b29640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b60197650 0x7f2b60197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:14:32.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.942+0000 7f2b66328640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b60100260 0x7f2b6019d590 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00d950 tx=0x7f2b5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:32.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.942+0000 7f2b577fe640 1 -- 172.21.15.94:0/229858872 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c009c30 con 0x7f2b60100260 2024-02-17T12:14:32.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.942+0000 7f2b577fe640 1 -- 172.21.15.94:0/229858872 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b5c009d90 con 0x7f2b60100260 2024-02-17T12:14:32.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.942+0000 7f2b6858c640 1 -- 172.21.15.94:0/229858872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b601a43f0 con 0x7f2b60100260 2024-02-17T12:14:32.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.942+0000 7f2b577fe640 1 -- 172.21.15.94:0/229858872 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00f7e0 con 0x7f2b60100260 2024-02-17T12:14:32.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.942+0000 7f2b6858c640 1 -- 172.21.15.94:0/229858872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b601a48d0 con 0x7f2b60100260 2024-02-17T12:14:32.948 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.943+0000 7f2b6858c640 1 -- 172.21.15.94:0/229858872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b60101730 con 0x7f2b60100260 2024-02-17T12:14:32.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.944+0000 7f2b577fe640 1 -- 172.21.15.94:0/229858872 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f2b5c010ad0 con 0x7f2b60100260 2024-02-17T12:14:32.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.945+0000 7f2b577fe640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2b4c0a2230 0x7f2b4c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:32.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.945+0000 7f2b65b27640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2b4c0a2230 0x7f2b4c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:32.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.945+0000 7f2b577fe640 1 -- 172.21.15.94:0/229858872 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f2b5c0cde00 con 0x7f2b60100260 2024-02-17T12:14:32.949 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.945+0000 7f2b65b27640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2b4c0a2230 0x7f2b4c0a46f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2b50036040 tx=0x7f2b50000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:32.951 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:32.949+0000 7f2b577fe640 1 -- 172.21.15.94:0/229858872 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b5c096a50 con 0x7f2b60100260 2024-02-17T12:14:33.143 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.140+0000 7f2b6858c640 1 -- 172.21.15.94:0/229858872 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2b60109590 con 0x7f2b60100260 2024-02-17T12:14:33.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.140+0000 7f2b577fe640 1 -- 172.21.15.94:0/229858872 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 17 entries v108) v1 ==== 81+0+1000 (secure 0 0 0) 0x7f2b5c09a110 con 0x7f2b60100260 2024-02-17T12:14:33.146 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3895514831 2024-02-18T11:48:31.902937+0000 2024-02-17T12:14:33.146 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/281060900 2024-02-18T11:48:31.902937+0000 2024-02-17T12:14:33.146 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2416598774 2024-02-18T11:48:58.554916+0000 2024-02-17T12:14:33.146 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1681584721 2024-02-18T11:48:31.902937+0000 2024-02-17T12:14:33.146 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/3511647110 2024-02-18T11:48:58.554916+0000 2024-02-17T12:14:33.146 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/2384585889 2024-02-18T11:48:58.554916+0000 2024-02-17T12:14:33.146 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2639307451 2024-02-18T11:58:06.744891+0000 2024-02-17T12:14:33.146 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/281060900 2024-02-18T11:48:31.902937+0000 2024-02-17T12:14:33.147 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/2384585889 2024-02-18T11:48:58.554916+0000 2024-02-17T12:14:33.147 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1237669817 2024-02-18T11:48:58.554916+0000 2024-02-17T12:14:33.147 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/875984108 2024-02-18T11:58:06.744891+0000 2024-02-17T12:14:33.147 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2865658350 2024-02-18T11:58:06.744891+0000 2024-02-17T12:14:33.147 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/142548766 2024-02-18T11:58:06.744891+0000 2024-02-17T12:14:33.147 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/1603490109 2024-02-18T11:58:06.744891+0000 2024-02-17T12:14:33.147 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:0/2293278094 2024-02-18T11:48:31.902937+0000 2024-02-17T12:14:33.147 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6801/1808883811 2024-02-18T11:58:06.744891+0000 2024-02-17T12:14:33.147 INFO:teuthology.orchestra.run.smithi094.stdout:172.21.15.94:6800/1808883811 2024-02-18T11:58:06.744891+0000 2024-02-17T12:14:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:listed 17 entries 2024-02-17T12:14:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.142+0000 7f2b6858c640 1 -- 172.21.15.94:0/229858872 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2b4c0a2230 msgr2=0x7f2b4c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.142+0000 7f2b6858c640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2b4c0a2230 0x7f2b4c0a46f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2b50036040 tx=0x7f2b50000bc0 comp rx=0 tx=0).stop 2024-02-17T12:14:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.142+0000 7f2b6858c640 1 -- 172.21.15.94:0/229858872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b60100260 msgr2=0x7f2b6019d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.142+0000 7f2b6858c640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b60100260 0x7f2b6019d590 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00d950 tx=0x7f2b5c00de10 comp rx=0 tx=0).stop 2024-02-17T12:14:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.142+0000 7f2b6858c640 1 -- 172.21.15.94:0/229858872 shutdown_connections 2024-02-17T12:14:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.142+0000 7f2b6858c640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f2b4c0a2230 0x7f2b4c0a46f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.142+0000 7f2b6858c640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f2b60197650 0x7f2b60197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:33.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.142+0000 7f2b6858c640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b60105330 0x7f2b6019dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:33.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.142+0000 7f2b6858c640 1 --2- 172.21.15.94:0/229858872 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f2b60100260 0x7f2b6019d590 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:33.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.142+0000 7f2b6858c640 1 -- 172.21.15.94:0/229858872 >> 172.21.15.94:0/229858872 conn(0x7f2b60077180 msgr2=0x7f2b60068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:33.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.143+0000 7f2b6858c640 1 -- 172.21.15.94:0/229858872 shutdown_connections 2024-02-17T12:14:33.152 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.143+0000 7f2b6858c640 1 -- 172.21.15.94:0/229858872 wait complete. 2024-02-17T12:14:33.156 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-17T12:14:33.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:14:33.156 DEBUG:teuthology.orchestra.run.smithi094:> sudo umount /tmp/cephfs 2024-02-17T12:14:33.296 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi094.front.sepia.ceph.com 2024-02-17T12:14:33.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:14:33.297 DEBUG:teuthology.orchestra.run.smithi094:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-02-17T12:14:33.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:14:33.356 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-02-17T12:14:33.432 INFO:teuthology.orchestra.run.smithi094.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-02-17T12:14:33.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:14:33.433 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-02-17T12:14:33.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:14:33.463 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-02-17T12:14:33.571 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-17T12:14:33.571 DEBUG:teuthology.orchestra.run.smithi094:> set -ex 2024-02-17T12:14:33.571 DEBUG:teuthology.orchestra.run.smithi094:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-17T12:14:33.587 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056361,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=218 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/bd693d0e97b2c196d576d9c1d5110e6613737036ec7e20db23b8f6a8bf60599e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/X6DVHCDAWRRRN3UY624DE3627G:/var/lib/containers/storage/overlay/l/N7RGVRHR7YQOOAMXABA3KCFHH7,upperdir=/var/lib/containers/storage/overlay/bd693d0e97b2c196d576d9c1d5110e6613737036ec7e20db23b8f6a8bf60599e/diff,workdir=/var/lib/containers/storage/overlay/bd693d0e97b2c196d576d9c1d5110e6613737036ec7e20db23b8f6a8bf60599e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f5d49329dfb8bfda6e7a02449d48146d6130914fefb8b9a858b5c5b4d188902e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/X6DVHCDAWRRRN3UY624DE3627G:/var/lib/containers/storage/overlay/l/N7RGVRHR7YQOOAMXABA3KCFHH7,upperdir=/var/lib/containers/storage/overlay/f5d49329dfb8bfda6e7a02449d48146d6130914fefb8b9a858b5c5b4d188902e/diff,workdir=/var/lib/containers/storage/overlay/f5d49329dfb8bfda6e7a02449d48146d6130914fefb8b9a858b5c5b4d188902e/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248836k,nr_inodes=812209,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/80c9492e06db5176f88899473e951ba4473f079bb2511763bed10cfe6dea1b27/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/X6DVHCDAWRRRN3UY624DE3627G:/var/lib/containers/storage/overlay/l/N7RGVRHR7YQOOAMXABA3KCFHH7,upperdir=/var/lib/containers/storage/overlay/80c9492e06db5176f88899473e951ba4473f079bb2511763bed10cfe6dea1b27/diff,workdir=/var/lib/containers/storage/overlay/80c9492e06db5176f88899473e951ba4473f079bb2511763bed10cfe6dea1b27/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/b2f41a124309a6b4b3d14014f1bda27079be604fb077e72e0e64b830c20d9481/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/X6DVHCDAWRRRN3UY624DE3627G:/var/lib/containers/storage/overlay/l/N7RGVRHR7YQOOAMXABA3KCFHH7,upperdir=/var/lib/containers/storage/overlay/b2f41a124309a6b4b3d14014f1bda27079be604fb077e72e0e64b830c20d9481/diff,workdir=/var/lib/containers/storage/overlay/b2f41a124309a6b4b3d14014f1bda27079be604fb077e72e0e64b830c20d9481/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/8d4ff0f659acee1d5158542e9b0e014f64f4e72858b819bdcf1675314b1b4446/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/X6DVHCDAWRRRN3UY624DE3627G:/var/lib/containers/storage/overlay/l/N7RGVRHR7YQOOAMXABA3KCFHH7,upperdir=/var/lib/containers/storage/overlay/8d4ff0f659acee1d5158542e9b0e014f64f4e72858b819bdcf1675314b1b4446/diff,workdir=/var/lib/containers/storage/overlay/8d4ff0f659acee1d5158542e9b0e014f64f4e72858b819bdcf1675314b1b4446/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0a8d8d0e92e0180d0cde2c4b3dd7c2eb188e18eff7e33dcedb567d170ebbbaa2/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/X6DVHCDAWRRRN3UY624DE3627G:/var/lib/containers/storage/overlay/l/N7RGVRHR7YQOOAMXABA3KCFHH7,upperdir=/var/lib/containers/storage/overlay/0a8d8d0e92e0180d0cde2c4b3dd7c2eb188e18eff7e33dcedb567d170ebbbaa2/diff,workdir=/var/lib/containers/storage/overlay/0a8d8d0e92e0180d0cde2c4b3dd7c2eb188e18eff7e33dcedb567d170ebbbaa2/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/acec4110a485938f6161befb5aff24d3e365aeeb20e0872b3c644b9caefd1a8c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/X6DVHCDAWRRRN3UY624DE3627G:/var/lib/containers/storage/overlay/l/N7RGVRHR7YQOOAMXABA3KCFHH7,upperdir=/var/lib/containers/storage/overlay/acec4110a485938f6161befb5aff24d3e365aeeb20e0872b3c644b9caefd1a8c/diff,workdir=/var/lib/containers/storage/overlay/acec4110a485938f6161befb5aff24d3e365aeeb20e0872b3c644b9caefd1a8c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d9e51c0bf99663b20bf5b8e8ce7d8cdfa018789d1adf4db418d9ca05b13ace30/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/X6DVHCDAWRRRN3UY624DE3627G:/var/lib/containers/storage/overlay/l/N7RGVRHR7YQOOAMXABA3KCFHH7,upperdir=/var/lib/containers/storage/overlay/d9e51c0bf99663b20bf5b8e8ce7d8cdfa018789d1adf4db418d9ca05b13ace30/diff,workdir=/var/lib/containers/storage/overlay/d9e51c0bf99663b20bf5b8e8ce7d8cdfa018789d1adf4db418d9ca05b13ace30/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/73ed89f2e586398eb146266ffa9f2f4f2663619935fc0e5d959c94d901f9536c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/X6DVHCDAWRRRN3UY624DE3627G:/var/lib/containers/storage/overlay/l/N7RGVRHR7YQOOAMXABA3KCFHH7,upperdir=/var/lib/containers/storage/overlay/73ed89f2e586398eb146266ffa9f2f4f2663619935fc0e5d959c94d901f9536c/diff,workdir=/var/lib/containers/storage/overlay/73ed89f2e586398eb146266ffa9f2f4f2663619935fc0e5d959c94d901f9536c/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0d7a2e8c6d9c6404c43dd4b104e816d0c1e6c622e337ae1ebf3ac94bcc7dc51e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/X6DVHCDAWRRRN3UY624DE3627G:/var/lib/containers/storage/overlay/l/N7RGVRHR7YQOOAMXABA3KCFHH7,upperdir=/var/lib/containers/storage/overlay/0d7a2e8c6d9c6404c43dd4b104e816d0c1e6c622e337ae1ebf3ac94bcc7dc51e/diff,workdir=/var/lib/containers/storage/overlay/0d7a2e8c6d9c6404c43dd4b104e816d0c1e6c622e337ae1ebf3ac94bcc7dc51e/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497672k,nr_inodes=819200,mode=755,inode64 0 0 2024-02-17T12:14:33.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:14:33.587 DEBUG:teuthology.orchestra.run.smithi094:> ip netns list 2024-02-17T12:14:33.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-17T12:14:33.642 DEBUG:teuthology.orchestra.run.smithi094:> sudo ip link delete ceph-brx 2024-02-17T12:14:33.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:33 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1583021420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:14:33.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:33 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1432932976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:14:33.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:33 smithi189 ceph-mon[36588]: pgmap v516: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-17T12:14:33.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:33 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/229858872' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-17T12:14:33.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:33 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1583021420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:14:33.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:33 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1432932976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:14:33.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:33 smithi125 ceph-mon[36819]: pgmap v516: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-17T12:14:33.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:33 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/229858872' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-17T12:14:33.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:33 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1583021420' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-17T12:14:33.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:33 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1432932976' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-17T12:14:33.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:33 smithi094 ceph-mon[32499]: pgmap v516: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 170 B/s wr, 6 op/s 2024-02-17T12:14:33.706 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:33 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/229858872' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-17T12:14:33.806 INFO:teuthology.orchestra.run.smithi094.stderr:Cannot find device "ceph-brx" 2024-02-17T12:14:33.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:14:33.808 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-17T12:14:33.821 INFO:tasks.cephadm:Teardown begin 2024-02-17T12:14:33.821 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-17T12:14:33.821 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:14:33.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.949+0000 7feaeefa0640 1 -- 172.21.15.94:0/923300891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae8100210 msgr2=0x7feae810ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:33.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.949+0000 7feaeefa0640 1 --2- 172.21.15.94:0/923300891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae8100210 0x7feae810ddf0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7feae0009ca0 tx=0x7feae002c140 comp rx=0 tx=0).stop 2024-02-17T12:14:33.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.949+0000 7feaeefa0640 1 -- 172.21.15.94:0/923300891 shutdown_connections 2024-02-17T12:14:33.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 --2- 172.21.15.94:0/923300891 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae8100210 0x7feae810ddf0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:33.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 --2- 172.21.15.94:0/923300891 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae80ff850 0x7feae80ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:33.954 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 --2- 172.21.15.94:0/923300891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae81070e0 0x7feae81074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 -- 172.21.15.94:0/923300891 >> 172.21.15.94:0/923300891 conn(0x7feae8076390 msgr2=0x7feae80767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 -- 172.21.15.94:0/923300891 shutdown_connections 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 -- 172.21.15.94:0/923300891 wait complete. 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 Processor -- start 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 -- start start 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae80ff850 0x7feae81975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae8100210 0x7feae8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae81070e0 0x7feae819be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae819c430 con 0x7feae80ff850 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae819c5a0 con 0x7feae8100210 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaeefa0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae819c760 con 0x7feae81070e0 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaedf9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae80ff850 0x7feae81975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaedf9e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae80ff850 0x7feae81975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56866/0 (socket says 172.21.15.94:56866) 2024-02-17T12:14:33.955 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.950+0000 7feaedf9e640 1 -- 172.21.15.94:0/2724928515 learned_addr learned my addr 172.21.15.94:0/2724928515 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.951+0000 7feaee79f640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae81070e0 0x7feae819be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.951+0000 7feaed79d640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae8100210 0x7feae8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.951+0000 7feaedf9e640 1 -- 172.21.15.94:0/2724928515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae81070e0 msgr2=0x7feae819be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.951+0000 7feaedf9e640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae81070e0 0x7feae819be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.951+0000 7feaedf9e640 1 -- 172.21.15.94:0/2724928515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae8100210 msgr2=0x7feae8197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.951+0000 7feaedf9e640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae8100210 0x7feae8197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.951+0000 7feaedf9e640 1 -- 172.21.15.94:0/2724928515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae0009900 con 0x7feae80ff850 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.951+0000 7feaedf9e640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae80ff850 0x7feae81975c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7feae400d950 tx=0x7feae400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.951+0000 7feadeffd640 1 -- 172.21.15.94:0/2724928515 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4009c30 con 0x7feae80ff850 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.951+0000 7feadeffd640 1 -- 172.21.15.94:0/2724928515 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7feae4009d90 con 0x7feae80ff850 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.951+0000 7feaeefa0640 1 -- 172.21.15.94:0/2724928515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae819ca40 con 0x7feae80ff850 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.951+0000 7feadeffd640 1 -- 172.21.15.94:0/2724928515 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae400f720 con 0x7feae80ff850 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.951+0000 7feaeefa0640 1 -- 172.21.15.94:0/2724928515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae81a42c0 con 0x7feae80ff850 2024-02-17T12:14:33.956 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.952+0000 7feaeefa0640 1 -- 172.21.15.94:0/2724928515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae80682e0 con 0x7feae80ff850 2024-02-17T12:14:33.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.953+0000 7feadeffd640 1 -- 172.21.15.94:0/2724928515 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7feae4010a60 con 0x7feae80ff850 2024-02-17T12:14:33.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.954+0000 7feadeffd640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fead40a2230 0x7fead40a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:33.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.954+0000 7feaed79d640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fead40a2230 0x7fead40a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:33.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.954+0000 7feadeffd640 1 -- 172.21.15.94:0/2724928515 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8585+0+0 (secure 0 0 0) 0x7feae40cdd60 con 0x7feae80ff850 2024-02-17T12:14:33.959 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.954+0000 7feaed79d640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fead40a2230 0x7fead40a46f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feae80fd9c0 tx=0x7fead8005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:33.960 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:33.957+0000 7feadeffd640 1 -- 172.21.15.94:0/2724928515 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feae4096a60 con 0x7feae80ff850 2024-02-17T12:14:34.167 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.165+0000 7feaeefa0640 1 -- 172.21.15.94:0/2724928515 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feae81094b0 con 0x7feae80ff850 2024-02-17T12:14:34.168 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.166+0000 7feadeffd640 1 -- 172.21.15.94:0/2724928515 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7feae4017090 con 0x7feae80ff850 2024-02-17T12:14:34.168 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:14:34.169 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:14:34.169 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:14:34.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.168+0000 7feaeefa0640 1 -- 172.21.15.94:0/2724928515 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fead40a2230 msgr2=0x7fead40a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.168+0000 7feaeefa0640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fead40a2230 0x7fead40a46f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feae80fd9c0 tx=0x7fead8005f90 comp rx=0 tx=0).stop 2024-02-17T12:14:34.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.168+0000 7feaeefa0640 1 -- 172.21.15.94:0/2724928515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae80ff850 msgr2=0x7feae81975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.170 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.168+0000 7feaeefa0640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae80ff850 0x7feae81975c0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7feae400d950 tx=0x7feae400de10 comp rx=0 tx=0).stop 2024-02-17T12:14:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.168+0000 7feaeefa0640 1 -- 172.21.15.94:0/2724928515 shutdown_connections 2024-02-17T12:14:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.168+0000 7feaeefa0640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fead40a2230 0x7fead40a46f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.168+0000 7feaeefa0640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feae81070e0 0x7feae819be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.168+0000 7feaeefa0640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7feae8100210 0x7feae8197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.168+0000 7feaeefa0640 1 --2- 172.21.15.94:0/2724928515 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7feae80ff850 0x7feae81975c0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.168+0000 7feaeefa0640 1 -- 172.21.15.94:0/2724928515 >> 172.21.15.94:0/2724928515 conn(0x7feae8076390 msgr2=0x7feae80fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.168+0000 7feaeefa0640 1 -- 172.21.15.94:0/2724928515 shutdown_connections 2024-02-17T12:14:34.171 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.168+0000 7feaeefa0640 1 -- 172.21.15.94:0/2724928515 wait complete. 2024-02-17T12:14:34.182 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-17T12:14:34.286 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.282+0000 7f5173ff4640 1 -- 172.21.15.94:0/1218865177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c105360 msgr2=0x7f516c10cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.282+0000 7f5173ff4640 1 --2- 172.21.15.94:0/1218865177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c105360 0x7f516c10cf30 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f5160009cd0 tx=0x7f516002c0f0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.282+0000 7f5173ff4640 1 -- 172.21.15.94:0/1218865177 shutdown_connections 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.282+0000 7f5173ff4640 1 --2- 172.21.15.94:0/1218865177 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c105360 0x7f516c10cf30 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.282+0000 7f5173ff4640 1 --2- 172.21.15.94:0/1218865177 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f516c100c40 0x7f516c104c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.282+0000 7f5173ff4640 1 --2- 172.21.15.94:0/1218865177 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f516c100290 0x7f516c100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.282+0000 7f5173ff4640 1 -- 172.21.15.94:0/1218865177 >> 172.21.15.94:0/1218865177 conn(0x7f516c076400 msgr2=0x7f516c076810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.282+0000 7f5173ff4640 1 -- 172.21.15.94:0/1218865177 shutdown_connections 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.282+0000 7f5173ff4640 1 -- 172.21.15.94:0/1218865177 wait complete. 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.282+0000 7f5173ff4640 1 Processor -- start 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5173ff4640 1 -- start start 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5173ff4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f516c100290 0x7f516c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5173ff4640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f516c105360 0x7f516c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5173ff4640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c199080 0x7f516c1a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:34.287 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5173ff4640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c1a3cd0 con 0x7f516c199080 2024-02-17T12:14:34.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5173ff4640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c1a3e40 con 0x7f516c105360 2024-02-17T12:14:34.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5173ff4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c1a4000 con 0x7f516c100290 2024-02-17T12:14:34.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5172591640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c199080 0x7f516c1a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:34.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5172591640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c199080 0x7f516c1a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:56884/0 (socket says 172.21.15.94:56884) 2024-02-17T12:14:34.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5172591640 1 -- 172.21.15.94:0/2723397026 learned_addr learned my addr 172.21.15.94:0/2723397026 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:14:34.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f517158f640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f516c105360 0x7f516c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:34.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5171d90640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f516c100290 0x7f516c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:34.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5172591640 1 -- 172.21.15.94:0/2723397026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f516c100290 msgr2=0x7f516c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5172591640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f516c100290 0x7f516c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5172591640 1 -- 172.21.15.94:0/2723397026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f516c105360 msgr2=0x7f516c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5172591640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f516c105360 0x7f516c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5172591640 1 -- 172.21.15.94:0/2723397026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160009900 con 0x7f516c199080 2024-02-17T12:14:34.288 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f517158f640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f516c105360 0x7f516c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:14:34.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5172591640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c199080 0x7f516c1a3630 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f5160004af0 tx=0x7f5160034cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:34.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f515effd640 1 -- 172.21.15.94:0/2723397026 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51600328c0 con 0x7f516c199080 2024-02-17T12:14:34.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f5173ff4640 1 -- 172.21.15.94:0/2723397026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516c1a4280 con 0x7f516c199080 2024-02-17T12:14:34.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f515effd640 1 -- 172.21.15.94:0/2723397026 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5160003e50 con 0x7f516c199080 2024-02-17T12:14:34.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.283+0000 7f515effd640 1 -- 172.21.15.94:0/2723397026 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160004860 con 0x7f516c199080 2024-02-17T12:14:34.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.284+0000 7f5173ff4640 1 -- 172.21.15.94:0/2723397026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c1a4730 con 0x7f516c199080 2024-02-17T12:14:34.289 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.284+0000 7f5173ff4640 1 -- 172.21.15.94:0/2723397026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516c101760 con 0x7f516c199080 2024-02-17T12:14:34.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.286+0000 7f515effd640 1 -- 172.21.15.94:0/2723397026 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f516000a040 con 0x7f516c199080 2024-02-17T12:14:34.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.286+0000 7f515effd640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f51580a2230 0x7f51580a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:34.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.286+0000 7f5171d90640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f51580a2230 0x7f51580a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:34.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.286+0000 7f515effd640 1 -- 172.21.15.94:0/2723397026 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f51600f1160 con 0x7f516c199080 2024-02-17T12:14:34.290 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.287+0000 7f5171d90640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f51580a2230 0x7f51580a46f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f51640058c0 tx=0x7f516400a4c0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:34.292 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.290+0000 7f515effd640 1 -- 172.21.15.94:0/2723397026 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f51600b9f00 con 0x7f516c199080 2024-02-17T12:14:34.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:34 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2724928515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:14:34.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.503+0000 7f5173ff4640 1 -- 172.21.15.94:0/2723397026 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f516c108630 con 0x7f516c199080 2024-02-17T12:14:34.506 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.504+0000 7f515effd640 1 -- 172.21.15.94:0/2723397026 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11344 (secure 0 0 0) 0x7f51600bd5c0 con 0x7f516c199080 2024-02-17T12:14:34.507 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:14:34.507 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6835/2052147813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6834","nonce":2052147813},{"type":"v1","addr":"172.21.15.94:6835","nonce":2052147813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6837/2565649123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6836","nonce":2565649123},{"type":"v1","addr":"172.21.15.94:6837","nonce":2565649123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6839/2246951573","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6838","nonce":2246951573},{"type":"v1","addr":"172.21.15.94:6839","nonce":2246951573}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.94:6841/755410555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6840","nonce":755410555},{"type":"v1","addr":"172.21.15.94:6841","nonce":755410555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.125:6833/1464839618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6832","nonce":1464839618},{"type":"v1","addr":"172.21.15.125:6833","nonce":1464839618}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":25802,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/667623913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":667623913},{"type":"v1","addr":"172.21.15.189:6835","nonce":667623913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-17T11:55:58.000681+0000","modified":"2024-02-17T12:05:14.959584+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24488,"mds_1":24487,"mds_2":24469},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6835/1238777175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6834","nonce":1238777175},{"type":"v1","addr":"172.21.15.125:6835","nonce":1238777175}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.125:6837/1075914834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6836","nonce":1075914834},{"type":"v1","addr":"172.21.15.125:6837","nonce":1075914834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6837/860722582","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":860722582},{"type":"v1","addr":"172.21.15.189:6837","nonce":860722582}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.189:6839/308580523","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6838","nonce":308580523},{"type":"v1","addr":"172.21.15.189:6839","nonce":308580523}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24488":{"gid":24488,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.125:6839/1158324088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6838","nonce":1158324088},{"type":"v1","addr":"172.21.15.125:6839","nonce":1158324088}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_26896":{"gid":26896,"name":"l","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.189:6833/1336939575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":1336939575},{"type":"v1","addr":"172.21.15.189:6833","nonce":1336939575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-17T12:14:34.507 INFO:teuthology.orchestra.run.smithi094.stderr:dumped fsmap epoch 27 2024-02-17T12:14:34.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.506+0000 7f5173ff4640 1 -- 172.21.15.94:0/2723397026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f51580a2230 msgr2=0x7f51580a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.506+0000 7f5173ff4640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f51580a2230 0x7f51580a46f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f51640058c0 tx=0x7f516400a4c0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.506+0000 7f5173ff4640 1 -- 172.21.15.94:0/2723397026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c199080 msgr2=0x7f516c1a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.506+0000 7f5173ff4640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c199080 0x7f516c1a3630 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f5160004af0 tx=0x7f5160034cc0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.506+0000 7f5173ff4640 1 -- 172.21.15.94:0/2723397026 shutdown_connections 2024-02-17T12:14:34.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.506+0000 7f5173ff4640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f51580a2230 0x7f51580a46f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.506+0000 7f5173ff4640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f516c199080 0x7f516c1a3630 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.506+0000 7f5173ff4640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f516c105360 0x7f516c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.509 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.506+0000 7f5173ff4640 1 --2- 172.21.15.94:0/2723397026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f516c100290 0x7f516c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.506+0000 7f5173ff4640 1 -- 172.21.15.94:0/2723397026 >> 172.21.15.94:0/2723397026 conn(0x7f516c076400 msgr2=0x7f516c0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:34.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.506+0000 7f5173ff4640 1 -- 172.21.15.94:0/2723397026 shutdown_connections 2024-02-17T12:14:34.510 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.506+0000 7f5173ff4640 1 -- 172.21.15.94:0/2723397026 wait complete. 2024-02-17T12:14:34.520 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-17T12:14:34.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.619+0000 7f9475b4f640 1 -- 172.21.15.94:0/4027270474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94701002f0 msgr2=0x7f94701006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.619+0000 7f9475b4f640 1 --2- 172.21.15.94:0/4027270474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94701002f0 0x7f94701006d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f9460009930 tx=0x7f946002bc30 comp rx=0 tx=0).stop 2024-02-17T12:14:34.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 -- 172.21.15.94:0/4027270474 shutdown_connections 2024-02-17T12:14:34.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 --2- 172.21.15.94:0/4027270474 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94701053c0 0x7f947010cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 --2- 172.21.15.94:0/4027270474 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9470100ca0 0x7f9470104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 --2- 172.21.15.94:0/4027270474 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94701002f0 0x7f94701006d0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 -- 172.21.15.94:0/4027270474 >> 172.21.15.94:0/4027270474 conn(0x7f9470077180 msgr2=0x7f9470077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:34.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 -- 172.21.15.94:0/4027270474 shutdown_connections 2024-02-17T12:14:34.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 -- 172.21.15.94:0/4027270474 wait complete. 2024-02-17T12:14:34.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 Processor -- start 2024-02-17T12:14:34.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 -- start start 2024-02-17T12:14:34.625 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94701002f0 0x7f94701985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94701053c0 0x7f9470198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9470199060 0x7f94701a3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94701a3cb0 con 0x7f94701002f0 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.620+0000 7f9475b4f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94701a3e20 con 0x7f9470199060 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f9475b4f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94701a3fe0 con 0x7f94701053c0 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94701002f0 0x7f94701985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946f7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94701002f0 0x7f94701985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47442/0 (socket says 172.21.15.94:47442) 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946f7fe640 1 -- 172.21.15.94:0/3783126693 learned_addr learned my addr 172.21.15.94:0/3783126693 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f9466ffd640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94701053c0 0x7f9470198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946ffff640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9470199060 0x7f94701a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946f7fe640 1 -- 172.21.15.94:0/3783126693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94701053c0 msgr2=0x7f9470198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946f7fe640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94701053c0 0x7f9470198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946f7fe640 1 -- 172.21.15.94:0/3783126693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9470199060 msgr2=0x7f94701a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.626 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946f7fe640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9470199060 0x7f94701a3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946f7fe640 1 -- 172.21.15.94:0/3783126693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9460009560 con 0x7f94701002f0 2024-02-17T12:14:34.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946ffff640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9470199060 0x7f94701a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:14:34.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f9466ffd640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94701053c0 0x7f9470198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:14:34.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946f7fe640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94701002f0 0x7f94701985e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f9460005940 tx=0x7f94600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:34.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946d7fa640 1 -- 172.21.15.94:0/3783126693 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94600044e0 con 0x7f94701002f0 2024-02-17T12:14:34.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946d7fa640 1 -- 172.21.15.94:0/3783126693 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f946002cbc0 con 0x7f94701002f0 2024-02-17T12:14:34.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f9475b4f640 1 -- 172.21.15.94:0/3783126693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94701a4260 con 0x7f94701002f0 2024-02-17T12:14:34.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f946d7fa640 1 -- 172.21.15.94:0/3783126693 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460003e20 con 0x7f94701002f0 2024-02-17T12:14:34.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.621+0000 7f9475b4f640 1 -- 172.21.15.94:0/3783126693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94701a4770 con 0x7f94701002f0 2024-02-17T12:14:34.627 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.622+0000 7f9475b4f640 1 -- 172.21.15.94:0/3783126693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9470067f70 con 0x7f94701002f0 2024-02-17T12:14:34.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.623+0000 7f946d7fa640 1 -- 172.21.15.94:0/3783126693 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f94600035f0 con 0x7f94701002f0 2024-02-17T12:14:34.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.624+0000 7f946d7fa640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f945c0a2230 0x7f945c0a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:34.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.624+0000 7f9466ffd640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f945c0a2230 0x7f945c0a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:34.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.624+0000 7f946d7fa640 1 -- 172.21.15.94:0/3783126693 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8585+0+0 (secure 0 0 0) 0x7f94600f03e0 con 0x7f94701002f0 2024-02-17T12:14:34.628 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.624+0000 7f9466ffd640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f945c0a2230 0x7f945c0a46f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9470068de0 tx=0x7f945800a5c0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:34.630 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.628+0000 7f946d7fa640 1 -- 172.21.15.94:0/3783126693 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f94600b9140 con 0x7f94701002f0 2024-02-17T12:14:34.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:34 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2724928515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:14:34.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:34 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2724928515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:14:34.806 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.803+0000 7f9475b4f640 1 -- 172.21.15.94:0/3783126693 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94701085f0 con 0x7f94701002f0 2024-02-17T12:14:34.807 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.805+0000 7f946d7fa640 1 -- 172.21.15.94:0/3783126693 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+23218 (secure 0 0 0) 0x7f94600bc800 con 0x7f94701002f0 2024-02-17T12:14:34.807 INFO:teuthology.orchestra.run.smithi094.stdout: 2024-02-17T12:14:34.808 INFO:teuthology.orchestra.run.smithi094.stdout:{"epoch":108,"fsid":"3bc5cfc8-cd8a-11ee-95ba-87774f69a715","created":"2024-02-17T11:48:04.980164+0000","modified":"2024-02-17T12:14:15.368567+0000","last_up_change":"2024-02-17T11:55:08.340026+0000","last_in_change":"2024-02-17T11:54:48.533109+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-17T11:51:20.894601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-17T11:55:56.349040+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-17T11:55:57.234916+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":26,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e7ed7cb6-3fc6-4264-8da2-fb1c340fe308","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6802","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6803","nonce":4275068800}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6804","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6805","nonce":4275068800}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6808","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6809","nonce":4275068800}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6806","nonce":4275068800},{"type":"v1","addr":"172.21.15.94:6807","nonce":4275068800}]},"public_addr":"172.21.15.94:6803/4275068800","cluster_addr":"172.21.15.94:6805/4275068800","heartbeat_back_addr":"172.21.15.94:6809/4275068800","heartbeat_front_addr":"172.21.15.94:6807/4275068800","state":["exists","up"]},{"osd":1,"uuid":"bf2a8735-cd15-4041-9013-178ba85852c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6800","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6801","nonce":142575339}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6802","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6803","nonce":142575339}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6806","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6807","nonce":142575339}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6804","nonce":142575339},{"type":"v1","addr":"172.21.15.125:6805","nonce":142575339}]},"public_addr":"172.21.15.125:6801/142575339","cluster_addr":"172.21.15.125:6803/142575339","heartbeat_back_addr":"172.21.15.125:6807/142575339","heartbeat_front_addr":"172.21.15.125:6805/142575339","state":["exists","up"]},{"osd":2,"uuid":"65b51d55-af5c-411c-988a-05331aad4a52","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6801","nonce":1204918236}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6803","nonce":1204918236}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6807","nonce":1204918236}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":1204918236},{"type":"v1","addr":"172.21.15.189:6805","nonce":1204918236}]},"public_addr":"172.21.15.189:6801/1204918236","cluster_addr":"172.21.15.189:6803/1204918236","heartbeat_back_addr":"172.21.15.189:6807/1204918236","heartbeat_front_addr":"172.21.15.189:6805/1204918236","state":["exists","up"]},{"osd":3,"uuid":"36e54a46-a0de-4a53-8bb4-6a49a457c91e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6810","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6811","nonce":3136219621}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6812","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6813","nonce":3136219621}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6816","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6817","nonce":3136219621}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6814","nonce":3136219621},{"type":"v1","addr":"172.21.15.94:6815","nonce":3136219621}]},"public_addr":"172.21.15.94:6811/3136219621","cluster_addr":"172.21.15.94:6813/3136219621","heartbeat_back_addr":"172.21.15.94:6817/3136219621","heartbeat_front_addr":"172.21.15.94:6815/3136219621","state":["exists","up"]},{"osd":4,"uuid":"dbe91382-c694-4eb0-8e3a-1f9cdbd45356","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6808","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6809","nonce":256065927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6810","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6811","nonce":256065927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6814","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6815","nonce":256065927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6812","nonce":256065927},{"type":"v1","addr":"172.21.15.125:6813","nonce":256065927}]},"public_addr":"172.21.15.125:6809/256065927","cluster_addr":"172.21.15.125:6811/256065927","heartbeat_back_addr":"172.21.15.125:6815/256065927","heartbeat_front_addr":"172.21.15.125:6813/256065927","state":["exists","up"]},{"osd":5,"uuid":"aa318c31-a0a3-4375-9980-5786a855c103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6809","nonce":1566223244}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6811","nonce":1566223244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6815","nonce":1566223244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":1566223244},{"type":"v1","addr":"172.21.15.189:6813","nonce":1566223244}]},"public_addr":"172.21.15.189:6809/1566223244","cluster_addr":"172.21.15.189:6811/1566223244","heartbeat_back_addr":"172.21.15.189:6815/1566223244","heartbeat_front_addr":"172.21.15.189:6813/1566223244","state":["exists","up"]},{"osd":6,"uuid":"3fbef085-0c9b-4eec-9516-74e139980355","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6818","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6819","nonce":422477558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6820","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6821","nonce":422477558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6824","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6825","nonce":422477558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6822","nonce":422477558},{"type":"v1","addr":"172.21.15.94:6823","nonce":422477558}]},"public_addr":"172.21.15.94:6819/422477558","cluster_addr":"172.21.15.94:6821/422477558","heartbeat_back_addr":"172.21.15.94:6825/422477558","heartbeat_front_addr":"172.21.15.94:6823/422477558","state":["exists","up"]},{"osd":7,"uuid":"3c76060b-420b-47f4-b746-75064957a855","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6816","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6817","nonce":2400671517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6818","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6819","nonce":2400671517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6822","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6823","nonce":2400671517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6820","nonce":2400671517},{"type":"v1","addr":"172.21.15.125:6821","nonce":2400671517}]},"public_addr":"172.21.15.125:6817/2400671517","cluster_addr":"172.21.15.125:6819/2400671517","heartbeat_back_addr":"172.21.15.125:6823/2400671517","heartbeat_front_addr":"172.21.15.125:6821/2400671517","state":["exists","up"]},{"osd":8,"uuid":"9f6c6b96-ee0a-48c7-abfd-e73733791afc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6817","nonce":3010520253}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6819","nonce":3010520253}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6823","nonce":3010520253}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":3010520253},{"type":"v1","addr":"172.21.15.189:6821","nonce":3010520253}]},"public_addr":"172.21.15.189:6817/3010520253","cluster_addr":"172.21.15.189:6819/3010520253","heartbeat_back_addr":"172.21.15.189:6823/3010520253","heartbeat_front_addr":"172.21.15.189:6821/3010520253","state":["exists","up"]},{"osd":9,"uuid":"bad201d9-439f-4236-bbf7-e736d0b12a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6826","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6827","nonce":3841273455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6828","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6829","nonce":3841273455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6832","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6833","nonce":3841273455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.94:6830","nonce":3841273455},{"type":"v1","addr":"172.21.15.94:6831","nonce":3841273455}]},"public_addr":"172.21.15.94:6827/3841273455","cluster_addr":"172.21.15.94:6829/3841273455","heartbeat_back_addr":"172.21.15.94:6833/3841273455","heartbeat_front_addr":"172.21.15.94:6831/3841273455","state":["exists","up"]},{"osd":10,"uuid":"61683c46-c206-4556-b68d-d69457f6fd61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6824","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6825","nonce":2398610115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6826","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6827","nonce":2398610115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6830","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6831","nonce":2398610115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.125:6828","nonce":2398610115},{"type":"v1","addr":"172.21.15.125:6829","nonce":2398610115}]},"public_addr":"172.21.15.125:6825/2398610115","cluster_addr":"172.21.15.125:6827/2398610115","heartbeat_back_addr":"172.21.15.125:6831/2398610115","heartbeat_front_addr":"172.21.15.125:6829/2398610115","state":["exists","up"]},{"osd":11,"uuid":"52468b82-565a-4bc9-a784-43acd747d719","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6825","nonce":656579186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6827","nonce":656579186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6831","nonce":656579186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":656579186},{"type":"v1","addr":"172.21.15.189:6829","nonce":656579186}]},"public_addr":"172.21.15.189:6825/656579186","cluster_addr":"172.21.15.189:6827/656579186","heartbeat_back_addr":"172.21.15.189:6831/656579186","heartbeat_front_addr":"172.21.15.189:6829/656579186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:32.044929+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:50:55.592620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:18.870352+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:51:43.911765+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:08.615555+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:33.101504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:52:58.435225+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:23.990663+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:53:48.376817+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:14.759736+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:54:40.787852+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-17T11:55:06.655134+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.94:0/3895514831":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6800/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/2416598774":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1681584721":"2024-02-18T11:48:31.902937+0000","172.21.15.94:0/3511647110":"2024-02-18T11:48:58.554916+0000","172.21.15.94:6800/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/2639307451":"2024-02-18T11:58:06.744891+0000","172.21.15.94:6801/281060900":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6801/2384585889":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/1237669817":"2024-02-18T11:48:58.554916+0000","172.21.15.94:0/875984108":"2024-02-18T11:58:06.744891+0000","172.21.15.94:0/2865658350":"2024-02-18T11:58:06.744891+0000","172.21.15.94:0/142548766":"2024-02-18T11:58:06.744891+0000","172.21.15.94:0/1603490109":"2024-02-18T11:58:06.744891+0000","172.21.15.94:0/2293278094":"2024-02-18T11:48:31.902937+0000","172.21.15.94:6801/1808883811":"2024-02-18T11:58:06.744891+0000","172.21.15.94:6800/1808883811":"2024-02-18T11:58:06.744891+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":13,"length":1},{"begin":26,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-17T12:14:34.809 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.807+0000 7f9475b4f640 1 -- 172.21.15.94:0/3783126693 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f945c0a2230 msgr2=0x7f945c0a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.807+0000 7f9475b4f640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f945c0a2230 0x7f945c0a46f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9470068de0 tx=0x7f945800a5c0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.807+0000 7f9475b4f640 1 -- 172.21.15.94:0/3783126693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94701002f0 msgr2=0x7f94701985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.807+0000 7f9475b4f640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94701002f0 0x7f94701985e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f9460005940 tx=0x7f94600042f0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.807+0000 7f9475b4f640 1 -- 172.21.15.94:0/3783126693 shutdown_connections 2024-02-17T12:14:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.807+0000 7f9475b4f640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f945c0a2230 0x7f945c0a46f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.807+0000 7f9475b4f640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9470199060 0x7f94701a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.807+0000 7f9475b4f640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94701053c0 0x7f9470198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.807+0000 7f9475b4f640 1 --2- 172.21.15.94:0/3783126693 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f94701002f0 0x7f94701985e0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.807+0000 7f9475b4f640 1 -- 172.21.15.94:0/3783126693 >> 172.21.15.94:0/3783126693 conn(0x7f9470077180 msgr2=0x7f94700fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.807+0000 7f9475b4f640 1 -- 172.21.15.94:0/3783126693 shutdown_connections 2024-02-17T12:14:34.810 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.807+0000 7f9475b4f640 1 -- 172.21.15.94:0/3783126693 wait complete. 2024-02-17T12:14:34.821 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-17T12:14:34.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.923+0000 7fad9e44e640 1 -- 172.21.15.94:0/772289864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad981061e0 msgr2=0x7fad981065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.923+0000 7fad9e44e640 1 --2- 172.21.15.94:0/772289864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad981061e0 0x7fad981065c0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fad8c009900 tx=0x7fad8c02bc30 comp rx=0 tx=0).stop 2024-02-17T12:14:34.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.923+0000 7fad9e44e640 1 -- 172.21.15.94:0/772289864 shutdown_connections 2024-02-17T12:14:34.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.923+0000 7fad9e44e640 1 --2- 172.21.15.94:0/772289864 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad98100d00 0x7fad9810ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.923+0000 7fad9e44e640 1 --2- 172.21.15.94:0/772289864 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad98100360 0x7fad981007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.923+0000 7fad9e44e640 1 --2- 172.21.15.94:0/772289864 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad981061e0 0x7fad981065c0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.923+0000 7fad9e44e640 1 -- 172.21.15.94:0/772289864 >> 172.21.15.94:0/772289864 conn(0x7fad98076e40 msgr2=0x7fad98077250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:34.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.923+0000 7fad9e44e640 1 -- 172.21.15.94:0/772289864 shutdown_connections 2024-02-17T12:14:34.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.923+0000 7fad9e44e640 1 -- 172.21.15.94:0/772289864 wait complete. 2024-02-17T12:14:34.928 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.923+0000 7fad9e44e640 1 Processor -- start 2024-02-17T12:14:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.923+0000 7fad9e44e640 1 -- start start 2024-02-17T12:14:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9e44e640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad98100360 0x7fad981966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9e44e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad98100d00 0x7fad98196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9e44e640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad981061e0 0x7fad9819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9e44e640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9819b520 con 0x7fad981061e0 2024-02-17T12:14:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9e44e640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9819b690 con 0x7fad98100360 2024-02-17T12:14:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9e44e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9819b850 con 0x7fad98100d00 2024-02-17T12:14:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9c9eb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad981061e0 0x7fad9819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9c9eb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad981061e0 0x7fad9819af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47454/0 (socket says 172.21.15.94:47454) 2024-02-17T12:14:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9c9eb640 1 -- 172.21.15.94:0/1859631840 learned_addr learned my addr 172.21.15.94:0/1859631840 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:14:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad97fff640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad98100360 0x7fad981966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:34.929 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad977fe640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad98100d00 0x7fad98196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9c9eb640 1 -- 172.21.15.94:0/1859631840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad98100d00 msgr2=0x7fad98196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9c9eb640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad98100d00 0x7fad98196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9c9eb640 1 -- 172.21.15.94:0/1859631840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad98100360 msgr2=0x7fad981966b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9c9eb640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad98100360 0x7fad981966b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9c9eb640 1 -- 172.21.15.94:0/1859631840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c009560 con 0x7fad981061e0 2024-02-17T12:14:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad977fe640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad98100d00 0x7fad98196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:14:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9c9eb640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad981061e0 0x7fad9819af20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fad9000eb10 tx=0x7fad9000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad957fa640 1 -- 172.21.15.94:0/1859631840 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90004b20 con 0x7fad981061e0 2024-02-17T12:14:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad957fa640 1 -- 172.21.15.94:0/1859631840 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad90004c80 con 0x7fad981061e0 2024-02-17T12:14:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9e44e640 1 -- 172.21.15.94:0/1859631840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad9819bb30 con 0x7fad981061e0 2024-02-17T12:14:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad957fa640 1 -- 172.21.15.94:0/1859631840 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90010650 con 0x7fad981061e0 2024-02-17T12:14:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.924+0000 7fad9e44e640 1 -- 172.21.15.94:0/1859631840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad981a33b0 con 0x7fad981061e0 2024-02-17T12:14:34.930 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.925+0000 7fad9e44e640 1 -- 172.21.15.94:0/1859631840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad98101750 con 0x7fad981061e0 2024-02-17T12:14:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.926+0000 7fad957fa640 1 -- 172.21.15.94:0/1859631840 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fad90005020 con 0x7fad981061e0 2024-02-17T12:14:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.927+0000 7fad957fa640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fad800a22b0 0x7fad800a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:34.931 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.927+0000 7fad97fff640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fad800a22b0 0x7fad800a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.927+0000 7fad957fa640 1 -- 172.21.15.94:0/1859631840 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8585+0+0 (secure 0 0 0) 0x7fad90014070 con 0x7fad981061e0 2024-02-17T12:14:34.932 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.927+0000 7fad97fff640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fad800a22b0 0x7fad800a4770 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad8c006010 tx=0x7fad8c005e40 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:34.933 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:34.931+0000 7fad957fa640 1 -- 172.21.15.94:0/1859631840 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fad900972d0 con 0x7fad981061e0 2024-02-17T12:14:35.144 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.142+0000 7fad9e44e640 1 -- 172.21.15.94:0/1859631840 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fad981979b0 con 0x7fad981061e0 2024-02-17T12:14:35.146 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.144+0000 7fad957fa640 1 -- 172.21.15.94:0/1859631840 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7fad9009a990 con 0x7fad981061e0 2024-02-17T12:14:35.146 INFO:teuthology.orchestra.run.smithi094.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-17T12:14:35.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.147+0000 7fad9e44e640 1 -- 172.21.15.94:0/1859631840 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fad800a22b0 msgr2=0x7fad800a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:35.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.147+0000 7fad9e44e640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fad800a22b0 0x7fad800a4770 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad8c006010 tx=0x7fad8c005e40 comp rx=0 tx=0).stop 2024-02-17T12:14:35.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.147+0000 7fad9e44e640 1 -- 172.21.15.94:0/1859631840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad981061e0 msgr2=0x7fad9819af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:35.149 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.147+0000 7fad9e44e640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad981061e0 0x7fad9819af20 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fad9000eb10 tx=0x7fad9000c2d0 comp rx=0 tx=0).stop 2024-02-17T12:14:35.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.147+0000 7fad9e44e640 1 -- 172.21.15.94:0/1859631840 shutdown_connections 2024-02-17T12:14:35.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.147+0000 7fad9e44e640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fad800a22b0 0x7fad800a4770 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:35.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.147+0000 7fad9e44e640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fad981061e0 0x7fad9819af20 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:35.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.147+0000 7fad9e44e640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad98100d00 0x7fad98196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:35.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.147+0000 7fad9e44e640 1 --2- 172.21.15.94:0/1859631840 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fad98100360 0x7fad981966b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:35.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.147+0000 7fad9e44e640 1 -- 172.21.15.94:0/1859631840 >> 172.21.15.94:0/1859631840 conn(0x7fad98076e40 msgr2=0x7fad980ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:35.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.147+0000 7fad9e44e640 1 -- 172.21.15.94:0/1859631840 shutdown_connections 2024-02-17T12:14:35.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.147+0000 7fad9e44e640 1 -- 172.21.15.94:0/1859631840 wait complete. 2024-02-17T12:14:35.161 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-17T12:14:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.264+0000 7fd4b5c88640 1 -- 172.21.15.94:0/1979746921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b0100cc0 msgr2=0x7fd4b0104d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.264+0000 7fd4b5c88640 1 --2- 172.21.15.94:0/1979746921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b0100cc0 0x7fd4b0104d00 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009930 tx=0x7fd4a402bc30 comp rx=0 tx=0).stop 2024-02-17T12:14:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.264+0000 7fd4b5c88640 1 -- 172.21.15.94:0/1979746921 shutdown_connections 2024-02-17T12:14:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.264+0000 7fd4b5c88640 1 --2- 172.21.15.94:0/1979746921 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4b01053e0 0x7fd4b010cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.264+0000 7fd4b5c88640 1 --2- 172.21.15.94:0/1979746921 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b0100cc0 0x7fd4b0104d00 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.264+0000 7fd4b5c88640 1 --2- 172.21.15.94:0/1979746921 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4b0100310 0x7fd4b01006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.264+0000 7fd4b5c88640 1 -- 172.21.15.94:0/1979746921 >> 172.21.15.94:0/1979746921 conn(0x7fd4b00771e0 msgr2=0x7fd4b00775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.264+0000 7fd4b5c88640 1 -- 172.21.15.94:0/1979746921 shutdown_connections 2024-02-17T12:14:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.264+0000 7fd4b5c88640 1 -- 172.21.15.94:0/1979746921 wait complete. 2024-02-17T12:14:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.264+0000 7fd4b5c88640 1 Processor -- start 2024-02-17T12:14:35.267 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4b5c88640 1 -- start start 2024-02-17T12:14:35.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4b5c88640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b0100310 0x7fd4b019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:35.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4b5c88640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4b01053e0 0x7fd4b019daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:35.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4b5c88640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4b01976c0 0x7fd4b0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:35.270 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4b5c88640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0198190 con 0x7fd4b0100310 2024-02-17T12:14:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4b5c88640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b0198300 con 0x7fd4b01976c0 2024-02-17T12:14:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4b5c88640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b01984c0 con 0x7fd4b01053e0 2024-02-17T12:14:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b0100310 0x7fd4b019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4af7fe640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b0100310 0x7fd4b019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47474/0 (socket says 172.21.15.94:47474) 2024-02-17T12:14:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4af7fe640 1 -- 172.21.15.94:0/3951400036 learned_addr learned my addr 172.21.15.94:0/3951400036 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:14:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4affff640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4b01976c0 0x7fd4b0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4aeffd640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4b01053e0 0x7fd4b019daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4af7fe640 1 -- 172.21.15.94:0/3951400036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4b01053e0 msgr2=0x7fd4b019daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4af7fe640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4b01053e0 0x7fd4b019daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4af7fe640 1 -- 172.21.15.94:0/3951400036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4b01976c0 msgr2=0x7fd4b0197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4af7fe640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4b01976c0 0x7fd4b0197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4af7fe640 1 -- 172.21.15.94:0/3951400036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a4009560 con 0x7fd4b0100310 2024-02-17T12:14:35.271 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4affff640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4b01976c0 0x7fd4b0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-17T12:14:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4af7fe640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b0100310 0x7fd4b019d5b0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000d950 tx=0x7fd4a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4acff9640 1 -- 172.21.15.94:0/3951400036 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a0009c30 con 0x7fd4b0100310 2024-02-17T12:14:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4b5c88640 1 -- 172.21.15.94:0/3951400036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b01a4410 con 0x7fd4b0100310 2024-02-17T12:14:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.265+0000 7fd4acff9640 1 -- 172.21.15.94:0/3951400036 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd4a0009d90 con 0x7fd4b0100310 2024-02-17T12:14:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.266+0000 7fd4acff9640 1 -- 172.21.15.94:0/3951400036 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a000f7e0 con 0x7fd4b0100310 2024-02-17T12:14:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.266+0000 7fd4b5c88640 1 -- 172.21.15.94:0/3951400036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b01a4940 con 0x7fd4b0100310 2024-02-17T12:14:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.266+0000 7fd4b5c88640 1 -- 172.21.15.94:0/3951400036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b01017e0 con 0x7fd4b0100310 2024-02-17T12:14:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.267+0000 7fd4acff9640 1 -- 172.21.15.94:0/3951400036 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7fd4a0010ad0 con 0x7fd4b0100310 2024-02-17T12:14:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.268+0000 7fd4acff9640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd4980a2230 0x7fd4980a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.268+0000 7fd4aeffd640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd4980a2230 0x7fd4980a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:35.272 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.268+0000 7fd4acff9640 1 -- 172.21.15.94:0/3951400036 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8799+0+0 (secure 0 0 0) 0x7fd4a00cdf00 con 0x7fd4b0100310 2024-02-17T12:14:35.273 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.269+0000 7fd4aeffd640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd4980a2230 0x7fd4980a46f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009900 tx=0x7fd4a4005fd0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:35.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.272+0000 7fd4acff9640 1 -- 172.21.15.94:0/3951400036 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd4a0096a80 con 0x7fd4b0100310 2024-02-17T12:14:35.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:35 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a[32495]: 2024-02-17T12:14:35.142+0000 7ff47df9e700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-17T12:14:35.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:35 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/2723397026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:14:35.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:35 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3783126693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:14:35.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:35 smithi094 ceph-mon[32499]: pgmap v517: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 64 KiB/s wr, 7 op/s 2024-02-17T12:14:35.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:35 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1859631840' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-17T12:14:35.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:35 smithi094 ceph-mon[32499]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-17T12:14:35.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:35 smithi094 ceph-mon[32499]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-17T12:14:35.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:35 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/1859631840' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-17T12:14:35.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:35 smithi094 ceph-mon[32499]: osdmap e109: 12 total, 12 up, 12 in 2024-02-17T12:14:35.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:35 smithi094 ceph-mon[32499]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-17T12:14:35.492 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:35.490+0000 7fd4b5c88640 1 -- 172.21.15.94:0/3951400036 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd4b0108670 con 0x7fd4b0100310 2024-02-17T12:14:35.654 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-i[68005]: ignoring --setuser ceph since I am not root 2024-02-17T12:14:35.654 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-i[68005]: ignoring --setgroup ceph since I am not root 2024-02-17T12:14:35.654 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-mds[68009]: -- 172.21.15.189:0/1429593238 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55895fce21a0 con 0x55895fcb9000 2024-02-17T12:14:35.654 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-i[68005]: starting mds.i at 2024-02-17T12:14:35.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/2723397026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:14:35.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3783126693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:14:35.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-mon[36588]: pgmap v517: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 64 KiB/s wr, 7 op/s 2024-02-17T12:14:35.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1859631840' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-17T12:14:35.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-mon[36588]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-17T12:14:35.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-mon[36588]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-17T12:14:35.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/1859631840' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-17T12:14:35.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-mon[36588]: osdmap e109: 12 total, 12 up, 12 in 2024-02-17T12:14:35.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-mon[36588]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-17T12:14:35.655 INFO:journalctl@ceph.mds.c.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-c[67686]: ignoring --setuser ceph since I am not root 2024-02-17T12:14:35.655 INFO:journalctl@ceph.mds.c.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-c[67686]: ignoring --setgroup ceph since I am not root 2024-02-17T12:14:35.656 INFO:journalctl@ceph.mds.c.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-mds[67690]: -- 172.21.15.189:0/926295664 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558520e0a1a0 con 0x558520de1000 2024-02-17T12:14:35.656 INFO:journalctl@ceph.mds.c.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-c[67686]: starting mds.c at 2024-02-17T12:14:35.656 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[72363]: ignoring --setuser ceph since I am not root 2024-02-17T12:14:35.656 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[72363]: ignoring --setgroup ceph since I am not root 2024-02-17T12:14:35.656 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-mds[72367]: -- 172.21.15.189:0/3333042309 <== mon.1 v2:172.21.15.189:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5561d54141a0 con 0x5561d53eac00 2024-02-17T12:14:35.656 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:14:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[72363]: starting mds.l at 2024-02-17T12:14:35.688 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b[67430]: ignoring --setuser ceph since I am not root 2024-02-17T12:14:35.688 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b[67430]: ignoring --setgroup ceph since I am not root 2024-02-17T12:14:35.688 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-mds[67434]: -- 172.21.15.125:0/2772222473 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563b3be561a0 con 0x563b3be2d000 2024-02-17T12:14:35.688 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b[67430]: starting mds.b at 2024-02-17T12:14:35.688 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-h[67107]: ignoring --setuser ceph since I am not root 2024-02-17T12:14:35.689 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-h[67107]: ignoring --setgroup ceph since I am not root 2024-02-17T12:14:35.689 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-mds[67111]: -- 172.21.15.125:0/2616512341 <== mon.2 v2:172.21.15.125:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55aebe3201a0 con 0x55aebe2f6c00 2024-02-17T12:14:35.689 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-h[67107]: starting mds.h at 2024-02-17T12:14:35.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/2723397026' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-17T12:14:35.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3783126693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-17T12:14:35.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-mon[36819]: pgmap v517: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 64 KiB/s wr, 7 op/s 2024-02-17T12:14:35.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1859631840' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-17T12:14:35.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-mon[36819]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-17T12:14:35.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-mon[36819]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-17T12:14:35.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/1859631840' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-17T12:14:35.690 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-mon[36819]: osdmap e109: 12 total, 12 up, 12 in 2024-02-17T12:14:35.690 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-mon[36819]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-02-17T12:14:35.690 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-e[66801]: ignoring --setuser ceph since I am not root 2024-02-17T12:14:35.690 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-e[66801]: ignoring --setgroup ceph since I am not root 2024-02-17T12:14:35.690 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-mds[66805]: -- 172.21.15.125:0/520547951 <== mon.0 v2:172.21.15.94:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5612bd9781a0 con 0x5612bd94e800 2024-02-17T12:14:35.690 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 12:14:35 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-e[66801]: starting mds.e at 2024-02-17T12:14:36.147 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.145+0000 7fd4acff9640 1 -- 172.21.15.94:0/3951400036 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd4a009a140 con 0x7fd4b0100310 2024-02-17T12:14:36.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.147+0000 7fd4b5c88640 1 -- 172.21.15.94:0/3951400036 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd4980a2230 msgr2=0x7fd4980a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:36.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.147+0000 7fd4b5c88640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd4980a2230 0x7fd4980a46f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009900 tx=0x7fd4a4005fd0 comp rx=0 tx=0).stop 2024-02-17T12:14:36.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.147+0000 7fd4b5c88640 1 -- 172.21.15.94:0/3951400036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b0100310 msgr2=0x7fd4b019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:36.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.147+0000 7fd4b5c88640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b0100310 0x7fd4b019d5b0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fd4a000d950 tx=0x7fd4a000de10 comp rx=0 tx=0).stop 2024-02-17T12:14:36.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.147+0000 7fd4b5c88640 1 -- 172.21.15.94:0/3951400036 shutdown_connections 2024-02-17T12:14:36.150 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.147+0000 7fd4b5c88640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7fd4980a2230 0x7fd4980a46f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:36.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.147+0000 7fd4b5c88640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7fd4b01976c0 0x7fd4b0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:36.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.147+0000 7fd4b5c88640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4b01053e0 0x7fd4b019daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:36.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.147+0000 7fd4b5c88640 1 --2- 172.21.15.94:0/3951400036 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7fd4b0100310 0x7fd4b019d5b0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:36.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.147+0000 7fd4b5c88640 1 -- 172.21.15.94:0/3951400036 >> 172.21.15.94:0/3951400036 conn(0x7fd4b00771e0 msgr2=0x7fd4b00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:36.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.148+0000 7fd4b5c88640 1 -- 172.21.15.94:0/3951400036 shutdown_connections 2024-02-17T12:14:36.151 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.148+0000 7fd4b5c88640 1 -- 172.21.15.94:0/3951400036 wait complete. 2024-02-17T12:14:36.164 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-17T12:14:36.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.269+0000 7f997eb9a640 1 -- 172.21.15.94:0/2998392494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978100c10 msgr2=0x7f9978104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:36.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.269+0000 7f997eb9a640 1 --2- 172.21.15.94:0/2998392494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978100c10 0x7f9978104c50 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f9968009930 tx=0x7f996802bc30 comp rx=0 tx=0).stop 2024-02-17T12:14:36.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 -- 172.21.15.94:0/2998392494 shutdown_connections 2024-02-17T12:14:36.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 --2- 172.21.15.94:0/2998392494 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9978105330 0x7f997810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:36.274 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 --2- 172.21.15.94:0/2998392494 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978100c10 0x7f9978104c50 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 --2- 172.21.15.94:0/2998392494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9978100260 0x7f9978100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 -- 172.21.15.94:0/2998392494 >> 172.21.15.94:0/2998392494 conn(0x7f9978077180 msgr2=0x7f9978077590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 -- 172.21.15.94:0/2998392494 shutdown_connections 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 -- 172.21.15.94:0/2998392494 wait complete. 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 Processor -- start 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 -- start start 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9978100260 0x7f9978101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978105330 0x7f9978102230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9978102770 0x7f99781a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781a5f60 con 0x7f9978105330 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781a60d0 con 0x7f9978100260 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997eb9a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781a6290 con 0x7f9978102770 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f9977fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978105330 0x7f9978102230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:36.275 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f9977fff640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978105330 0x7f9978102230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47498/0 (socket says 172.21.15.94:47498) 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f9977fff640 1 -- 172.21.15.94:0/492253222 learned_addr learned my addr 172.21.15.94:0/492253222 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.270+0000 7f997c936640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9978100260 0x7f9978101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.271+0000 7f997d137640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9978102770 0x7f99781a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.271+0000 7f9977fff640 1 -- 172.21.15.94:0/492253222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9978102770 msgr2=0x7f99781a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.271+0000 7f9977fff640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9978102770 0x7f99781a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.271+0000 7f9977fff640 1 -- 172.21.15.94:0/492253222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9978100260 msgr2=0x7f9978101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.271+0000 7f9977fff640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9978100260 0x7f9978101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.271+0000 7f9977fff640 1 -- 172.21.15.94:0/492253222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9968009560 con 0x7f9978105330 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.271+0000 7f997c936640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9978100260 0x7f9978101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.271+0000 7f9977fff640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978105330 0x7f9978102230 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f9968009230 tx=0x7f996802c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.271+0000 7f9975ffb640 1 -- 172.21.15.94:0/492253222 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968004350 con 0x7f9978105330 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.271+0000 7f9975ffb640 1 -- 172.21.15.94:0/492253222 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f99680044b0 con 0x7f9978105330 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.271+0000 7f9975ffb640 1 -- 172.21.15.94:0/492253222 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9968003da0 con 0x7f9978105330 2024-02-17T12:14:36.276 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.271+0000 7f997eb9a640 1 -- 172.21.15.94:0/492253222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99781a6510 con 0x7f9978105330 2024-02-17T12:14:36.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.271+0000 7f997eb9a640 1 -- 172.21.15.94:0/492253222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99781a6a20 con 0x7f9978105330 2024-02-17T12:14:36.277 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.272+0000 7f997eb9a640 1 -- 172.21.15.94:0/492253222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9978067f70 con 0x7f9978105330 2024-02-17T12:14:36.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.274+0000 7f9975ffb640 1 -- 172.21.15.94:0/492253222 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7f9968004df0 con 0x7f9978105330 2024-02-17T12:14:36.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.274+0000 7f9975ffb640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f99640a2260 0x7f99640a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:36.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.274+0000 7f997c936640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f99640a2260 0x7f99640a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:36.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.274+0000 7f9975ffb640 1 -- 172.21.15.94:0/492253222 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8799+0+0 (secure 0 0 0) 0x7f99680f0840 con 0x7f9978105330 2024-02-17T12:14:36.278 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.275+0000 7f997c936640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f99640a2260 0x7f99640a4720 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9960009fc0 tx=0x7f9960005f90 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:36.280 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.278+0000 7f9975ffb640 1 -- 172.21.15.94:0/492253222 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99680b94d0 con 0x7f9978105330 2024-02-17T12:14:36.456 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:36.453+0000 7f997eb9a640 1 -- 172.21.15.94:0/492253222 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9978109590 con 0x7f9978105330 2024-02-17T12:14:36.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:14:36.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3951400036' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-17T12:14:36.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-17T12:14:36.458 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: Cluster is now healthy 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/3951400036' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.125:6834/2441532374,v1:172.21.15.125:6837/2441532374] up:boot 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.125:6835/3601901637,v1:172.21.15.125:6836/3601901637] up:boot 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.125:6838/706527052,v1:172.21.15.125:6839/706527052] up:boot 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.189:6837/1485924130,v1:172.21.15.189:6839/1485924130] up:boot 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.189:6832/4163209173,v1:172.21.15.189:6833/4163209173] up:boot 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: mds.? [v2:172.21.15.189:6836/2885864304,v1:172.21.15.189:6838/2885864304] up:boot 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: fsmap 12 up:standby 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T12:14:36.459 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:14:36.460 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:14:36.460 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:14:36.460 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:36 smithi094 ceph-mon[32499]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:14:36.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:14:36.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3951400036' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-17T12:14:36.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-17T12:14:36.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-17T12:14:36.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: Cluster is now healthy 2024-02-17T12:14:36.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/3951400036' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-17T12:14:36.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.125:6834/2441532374,v1:172.21.15.125:6837/2441532374] up:boot 2024-02-17T12:14:36.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.125:6835/3601901637,v1:172.21.15.125:6836/3601901637] up:boot 2024-02-17T12:14:36.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.125:6838/706527052,v1:172.21.15.125:6839/706527052] up:boot 2024-02-17T12:14:36.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.189:6837/1485924130,v1:172.21.15.189:6839/1485924130] up:boot 2024-02-17T12:14:36.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.189:6832/4163209173,v1:172.21.15.189:6833/4163209173] up:boot 2024-02-17T12:14:36.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: mds.? [v2:172.21.15.189:6836/2885864304,v1:172.21.15.189:6838/2885864304] up:boot 2024-02-17T12:14:36.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: fsmap 12 up:standby 2024-02-17T12:14:36.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T12:14:36.655 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T12:14:36.656 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T12:14:36.656 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T12:14:36.656 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T12:14:36.656 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:14:36.656 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:14:36.656 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:14:36.656 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:36 smithi189 ceph-mon[36588]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:14:36.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-17T12:14:36.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3951400036' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-17T12:14:36.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-17T12:14:36.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-17T12:14:36.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: Cluster is now healthy 2024-02-17T12:14:36.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/3951400036' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-17T12:14:36.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.125:6834/2441532374,v1:172.21.15.125:6837/2441532374] up:boot 2024-02-17T12:14:36.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.125:6835/3601901637,v1:172.21.15.125:6836/3601901637] up:boot 2024-02-17T12:14:36.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.125:6838/706527052,v1:172.21.15.125:6839/706527052] up:boot 2024-02-17T12:14:36.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.189:6837/1485924130,v1:172.21.15.189:6839/1485924130] up:boot 2024-02-17T12:14:36.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.189:6832/4163209173,v1:172.21.15.189:6833/4163209173] up:boot 2024-02-17T12:14:36.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: mds.? [v2:172.21.15.189:6836/2885864304,v1:172.21.15.189:6838/2885864304] up:boot 2024-02-17T12:14:36.688 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: fsmap 12 up:standby 2024-02-17T12:14:36.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-17T12:14:36.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-17T12:14:36.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-17T12:14:36.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-17T12:14:36.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-17T12:14:36.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-17T12:14:36.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-17T12:14:36.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-17T12:14:36.689 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:36 smithi125 ceph-mon[36819]: from='mgr.15435 172.21.15.94:0/3821364887' entity='mgr.x' 2024-02-17T12:14:37.410 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.407+0000 7f9975ffb640 1 -- 172.21.15.94:0/492253222 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) v1 ==== 181+0+0 (secure 0 0 0) 0x7f99680bcb90 con 0x7f9978105330 2024-02-17T12:14:37.410 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_metadata' removed 2024-02-17T12:14:37.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.410+0000 7f997eb9a640 1 -- 172.21.15.94:0/492253222 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f99640a2260 msgr2=0x7f99640a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:37.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.410+0000 7f997eb9a640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f99640a2260 0x7f99640a4720 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9960009fc0 tx=0x7f9960005f90 comp rx=0 tx=0).stop 2024-02-17T12:14:37.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.410+0000 7f997eb9a640 1 -- 172.21.15.94:0/492253222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978105330 msgr2=0x7f9978102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:37.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.410+0000 7f997eb9a640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978105330 0x7f9978102230 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f9968009230 tx=0x7f996802c680 comp rx=0 tx=0).stop 2024-02-17T12:14:37.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.411+0000 7f997eb9a640 1 -- 172.21.15.94:0/492253222 shutdown_connections 2024-02-17T12:14:37.413 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.411+0000 7f997eb9a640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7f99640a2260 0x7f99640a4720 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:37.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.411+0000 7f997eb9a640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9978102770 0x7f99781a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:37.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.411+0000 7f997eb9a640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7f9978105330 0x7f9978102230 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:37.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.411+0000 7f997eb9a640 1 --2- 172.21.15.94:0/492253222 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7f9978100260 0x7f9978101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:37.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.411+0000 7f997eb9a640 1 -- 172.21.15.94:0/492253222 >> 172.21.15.94:0/492253222 conn(0x7f9978077180 msgr2=0x7f9978068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:37.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.411+0000 7f997eb9a640 1 -- 172.21.15.94:0/492253222 shutdown_connections 2024-02-17T12:14:37.414 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.411+0000 7f997eb9a640 1 -- 172.21.15.94:0/492253222 wait complete. 2024-02-17T12:14:37.433 DEBUG:teuthology.orchestra.run.smithi094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-17T12:14:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.542+0000 7ff96fd4f640 1 -- 172.21.15.94:0/3480252165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff968102d10 msgr2=0x7ff96810b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.542+0000 7ff96fd4f640 1 --2- 172.21.15.94:0/3480252165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff968102d10 0x7ff96810b1f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff958009900 tx=0x7ff95802bcd0 comp rx=0 tx=0).stop 2024-02-17T12:14:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.542+0000 7ff96fd4f640 1 -- 172.21.15.94:0/3480252165 shutdown_connections 2024-02-17T12:14:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.542+0000 7ff96fd4f640 1 --2- 172.21.15.94:0/3480252165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff96810b7c0 0x7ff96810dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.542+0000 7ff96fd4f640 1 --2- 172.21.15.94:0/3480252165 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff968102d10 0x7ff96810b1f0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:37.547 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.542+0000 7ff96fd4f640 1 --2- 172.21.15.94:0/3480252165 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff9681023f0 0x7ff9681027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.542+0000 7ff96fd4f640 1 -- 172.21.15.94:0/3480252165 >> 172.21.15.94:0/3480252165 conn(0x7ff9680771e0 msgr2=0x7ff9680775f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.543+0000 7ff96fd4f640 1 -- 172.21.15.94:0/3480252165 shutdown_connections 2024-02-17T12:14:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.543+0000 7ff96fd4f640 1 -- 172.21.15.94:0/3480252165 wait complete. 2024-02-17T12:14:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.543+0000 7ff96fd4f640 1 Processor -- start 2024-02-17T12:14:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.543+0000 7ff96fd4f640 1 -- start start 2024-02-17T12:14:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.543+0000 7ff96fd4f640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9681023f0 0x7ff96819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.543+0000 7ff96fd4f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff968102d10 0x7ff96819dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.543+0000 7ff96fd4f640 1 --2- >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff96810b7c0 0x7ff9681976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.543+0000 7ff96fd4f640 1 -- --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff968197e40 con 0x7ff9681023f0 2024-02-17T12:14:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.543+0000 7ff96fd4f640 1 -- --> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff968197fb0 con 0x7ff96810b7c0 2024-02-17T12:14:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.543+0000 7ff96fd4f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff968198170 con 0x7ff968102d10 2024-02-17T12:14:37.548 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.543+0000 7ff96daeb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9681023f0 0x7ff96819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.543+0000 7ff96daeb640 1 --2- >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9681023f0 0x7ff96819d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.94:3300/0 says I am v2:172.21.15.94:47504/0 (socket says 172.21.15.94:47504) 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.543+0000 7ff96daeb640 1 -- 172.21.15.94:0/880348827 learned_addr learned my addr 172.21.15.94:0/880348827 (peer_addr_for_me v2:172.21.15.94:0/0) 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.544+0000 7ff96daeb640 1 -- 172.21.15.94:0/880348827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff968102d10 msgr2=0x7ff96819dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.544+0000 7ff96daeb640 1 --2- 172.21.15.94:0/880348827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff968102d10 0x7ff96819dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.544+0000 7ff96daeb640 1 -- 172.21.15.94:0/880348827 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff96810b7c0 msgr2=0x7ff9681976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.544+0000 7ff96daeb640 1 --2- 172.21.15.94:0/880348827 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff96810b7c0 0x7ff9681976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.544+0000 7ff96daeb640 1 -- 172.21.15.94:0/880348827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff958009560 con 0x7ff9681023f0 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.544+0000 7ff96daeb640 1 --2- 172.21.15.94:0/880348827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9681023f0 0x7ff96819d5a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff96400c920 tx=0x7ff96400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.544+0000 7ff95effd640 1 -- 172.21.15.94:0/880348827 <== mon.0 v2:172.21.15.94:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964007c70 con 0x7ff9681023f0 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.544+0000 7ff95effd640 1 -- 172.21.15.94:0/880348827 <== mon.0 v2:172.21.15.94:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff964007dd0 con 0x7ff9681023f0 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.544+0000 7ff95effd640 1 -- 172.21.15.94:0/880348827 <== mon.0 v2:172.21.15.94:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964018860 con 0x7ff9681023f0 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.544+0000 7ff96fd4f640 1 -- 172.21.15.94:0/880348827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff968198450 con 0x7ff9681023f0 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.544+0000 7ff96fd4f640 1 -- 172.21.15.94:0/880348827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9681a4520 con 0x7ff9681023f0 2024-02-17T12:14:37.549 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.545+0000 7ff96fd4f640 1 -- 172.21.15.94:0/880348827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff968069fb0 con 0x7ff9681023f0 2024-02-17T12:14:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.546+0000 7ff95effd640 1 -- 172.21.15.94:0/880348827 <== mon.0 v2:172.21.15.94:3300/0 4 ==== mgrmap(e 27) v1 ==== 136899+0+0 (secure 0 0 0) 0x7ff96400fac0 con 0x7ff9681023f0 2024-02-17T12:14:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.546+0000 7ff95effd640 1 --2- 172.21.15.94:0/880348827 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff9540a2230 0x7ff9540a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-17T12:14:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.547+0000 7ff96d2ea640 1 --2- 172.21.15.94:0/880348827 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff9540a2230 0x7ff9540a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-17T12:14:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.547+0000 7ff95effd640 1 -- 172.21.15.94:0/880348827 <== mon.0 v2:172.21.15.94:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8245+0+0 (secure 0 0 0) 0x7ff9640ce170 con 0x7ff9681023f0 2024-02-17T12:14:37.550 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.547+0000 7ff96d2ea640 1 --2- 172.21.15.94:0/880348827 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff9540a2230 0x7ff9540a46f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff958036040 tx=0x7ff958000bc0 comp rx=0 tx=0).ready entity=mgr.15435 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-17T12:14:37.553 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.551+0000 7ff95effd640 1 -- 172.21.15.94:0/880348827 <== mon.0 v2:172.21.15.94:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff964096f10 con 0x7ff9681023f0 2024-02-17T12:14:37.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:37 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/492253222' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-17T12:14:37.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:37 smithi189 ceph-mon[36588]: pgmap v519: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 6 op/s 2024-02-17T12:14:37.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:37 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/492253222' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-17T12:14:37.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:37 smithi125 ceph-mon[36819]: pgmap v519: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 6 op/s 2024-02-17T12:14:37.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:37 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/492253222' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-17T12:14:37.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:37 smithi094 ceph-mon[32499]: pgmap v519: 97 pgs: 97 active+clean; 4.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 76 KiB/s wr, 6 op/s 2024-02-17T12:14:37.726 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:37.724+0000 7ff96fd4f640 1 -- 172.21.15.94:0/880348827 --> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff968106940 con 0x7ff9681023f0 2024-02-17T12:14:38.411 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:38.409+0000 7ff95effd640 1 -- 172.21.15.94:0/880348827 <== mon.0 v2:172.21.15.94:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff968106940 con 0x7ff9681023f0 2024-02-17T12:14:38.411 INFO:teuthology.orchestra.run.smithi094.stderr:pool 'cephfs_data' removed 2024-02-17T12:14:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:38.413+0000 7ff96fd4f640 1 -- 172.21.15.94:0/880348827 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff9540a2230 msgr2=0x7ff9540a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:38.413+0000 7ff96fd4f640 1 --2- 172.21.15.94:0/880348827 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff9540a2230 0x7ff9540a46f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff958036040 tx=0x7ff958000bc0 comp rx=0 tx=0).stop 2024-02-17T12:14:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:38.413+0000 7ff96fd4f640 1 -- 172.21.15.94:0/880348827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9681023f0 msgr2=0x7ff96819d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-17T12:14:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:38.413+0000 7ff96fd4f640 1 --2- 172.21.15.94:0/880348827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9681023f0 0x7ff96819d5a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff96400c920 tx=0x7ff96400cde0 comp rx=0 tx=0).stop 2024-02-17T12:14:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:38.414+0000 7ff96fd4f640 1 -- 172.21.15.94:0/880348827 shutdown_connections 2024-02-17T12:14:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:38.414+0000 7ff96fd4f640 1 --2- 172.21.15.94:0/880348827 >> [v2:172.21.15.94:6800/2106228828,v1:172.21.15.94:6801/2106228828] conn(0x7ff9540a2230 0x7ff9540a46f0 secure :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff958036040 tx=0x7ff958000bc0 comp rx=0 tx=0).stop 2024-02-17T12:14:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:38.414+0000 7ff96fd4f640 1 --2- 172.21.15.94:0/880348827 >> [v2:172.21.15.125:3300/0,v1:172.21.15.125:6789/0] conn(0x7ff96810b7c0 0x7ff9681976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:38.414+0000 7ff96fd4f640 1 --2- 172.21.15.94:0/880348827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff968102d10 0x7ff96819dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-17T12:14:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:38.414+0000 7ff96fd4f640 1 --2- 172.21.15.94:0/880348827 >> [v2:172.21.15.94:3300/0,v1:172.21.15.94:6789/0] conn(0x7ff9681023f0 0x7ff96819d5a0 secure :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7ff96400c920 tx=0x7ff96400cde0 comp rx=0 tx=0).stop 2024-02-17T12:14:38.416 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:38.414+0000 7ff96fd4f640 1 -- 172.21.15.94:0/880348827 >> 172.21.15.94:0/880348827 conn(0x7ff9680771e0 msgr2=0x7ff9680fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-17T12:14:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:38.414+0000 7ff96fd4f640 1 -- 172.21.15.94:0/880348827 shutdown_connections 2024-02-17T12:14:38.417 INFO:teuthology.orchestra.run.smithi094.stderr:2024-02-17T12:14:38.414+0000 7ff96fd4f640 1 -- 172.21.15.94:0/880348827 wait complete. 2024-02-17T12:14:38.440 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-17T12:14:38.501 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-17T12:14:38.536 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-17T12:14:38.569 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-17T12:14:38.570 DEBUG:teuthology.orchestra.run.smithi094:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-17T12:14:38.603 DEBUG:teuthology.orchestra.run.smithi125:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-17T12:14:38.619 DEBUG:teuthology.orchestra.run.smithi189:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-17T12:14:38.635 INFO:tasks.cephadm:Stopping all daemons... 2024-02-17T12:14:38.635 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-17T12:14:38.635 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.a 2024-02-17T12:14:38.671 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:38 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/492253222' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-17T12:14:38.672 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:38 smithi094 ceph-mon[32499]: osdmap e110: 12 total, 12 up, 12 in 2024-02-17T12:14:38.672 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:38 smithi094 ceph-mon[32499]: from='client.? 172.21.15.94:0/880348827' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-17T12:14:38.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:38 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/492253222' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-17T12:14:38.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:38 smithi125 ceph-mon[36819]: osdmap e110: 12 total, 12 up, 12 in 2024-02-17T12:14:38.687 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:38 smithi125 ceph-mon[36819]: from='client.? 172.21.15.94:0/880348827' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-17T12:14:38.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:38 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/492253222' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-17T12:14:38.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:38 smithi189 ceph-mon[36588]: osdmap e110: 12 total, 12 up, 12 in 2024-02-17T12:14:38.904 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:38 smithi189 ceph-mon[36588]: from='client.? 172.21.15.94:0/880348827' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-17T12:14:38.958 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:38 smithi094 systemd[1]: Stopping Ceph mon.a for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:14:39.338 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:39 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a[32495]: 2024-02-17T12:14:39.012+0000 7ff4837a9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:14:39.338 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:39 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a[32495]: 2024-02-17T12:14:39.012+0000 7ff4837a9700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-17T12:14:39.338 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:39 smithi094 podman[108320]: 2024-02-17 12:14:39.054370852 +0000 UTC m=+0.165186952 container died fb2eb803b897b54c521527681de37aa41d30c103a5357cf24c5a3722d0440580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, ceph=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-17T12:14:39.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:39 smithi094 podman[108320]: 2024-02-17 12:14:39.439498626 +0000 UTC m=+0.550314727 container cleanup fb2eb803b897b54c521527681de37aa41d30c103a5357cf24c5a3722d0440580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-17T12:14:39.708 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:39 smithi094 bash[108320]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a 2024-02-17T12:14:40.013 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.a.service' 2024-02-17T12:14:40.092 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:39 smithi094 podman[108334]: 2024-02-17 12:14:39.794623991 +0000 UTC m=+0.739493410 container remove fb2eb803b897b54c521527681de37aa41d30c103a5357cf24c5a3722d0440580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD) 2024-02-17T12:14:40.093 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:40 smithi094 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-17T12:14:40.093 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:40 smithi094 systemd[1]: Stopped Ceph mon.a for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T12:14:40.093 INFO:journalctl@ceph.mon.a.smithi094.stdout:Feb 17 12:14:40 smithi094 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.a.service: Consumed 24.267s CPU time. 2024-02-17T12:14:40.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:14:40.131 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-17T12:14:40.132 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-17T12:14:40.132 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.b 2024-02-17T12:14:40.438 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:40 smithi125 systemd[1]: Stopping Ceph mon.b for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:14:40.887 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:40 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-b[36815]: 2024-02-17T12:14:40.498+0000 7fb5c7351700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:14:40.887 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:40 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-b[36815]: 2024-02-17T12:14:40.498+0000 7fb5c7351700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-17T12:14:40.887 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:40 smithi125 podman[73638]: 2024-02-17 12:14:40.529847878 +0000 UTC m=+0.178521855 container died 232c18f9347d9b84a1560773cfc20747b17a1bf6c7f8a9f0302f9728ebecfc14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-b, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-17T12:14:41.902 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:41 smithi125 podman[73638]: 2024-02-17 12:14:41.400775853 +0000 UTC m=+1.049449840 container cleanup 232c18f9347d9b84a1560773cfc20747b17a1bf6c7f8a9f0302f9728ebecfc14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:14:41.903 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:41 smithi125 bash[73638]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-b 2024-02-17T12:14:44.871 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.b.service' 2024-02-17T12:14:44.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:44 smithi125 podman[73651]: 2024-02-17 12:14:44.661996887 +0000 UTC m=+4.135415982 container remove 232c18f9347d9b84a1560773cfc20747b17a1bf6c7f8a9f0302f9728ebecfc14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True) 2024-02-17T12:14:44.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:44 smithi125 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.b.service: Deactivated successfully. 2024-02-17T12:14:44.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:44 smithi125 systemd[1]: Stopped Ceph mon.b for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T12:14:44.938 INFO:journalctl@ceph.mon.b.smithi125.stdout:Feb 17 12:14:44 smithi125 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.b.service: Consumed 14.864s CPU time. 2024-02-17T12:14:45.025 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:14:45.025 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-17T12:14:45.025 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-17T12:14:45.025 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.c 2024-02-17T12:14:45.356 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:45 smithi189 systemd[1]: Stopping Ceph mon.c for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:14:45.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:45 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-c[36571]: 2024-02-17T12:14:45.353+0000 7fe81d985700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:14:45.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:45 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-c[36571]: 2024-02-17T12:14:45.353+0000 7fe81d985700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-17T12:14:45.654 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:45 smithi189 podman[75068]: 2024-02-17 12:14:45.497388032 +0000 UTC m=+0.249937138 container died a90006e43b15a690a4cb86daee7382196cda083bfd32953829ee745da343d565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-17T12:14:46.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:45 smithi189 podman[75068]: 2024-02-17 12:14:45.900800872 +0000 UTC m=+0.653349976 container cleanup a90006e43b15a690a4cb86daee7382196cda083bfd32953829ee745da343d565 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-c, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-17T12:14:46.154 INFO:journalctl@ceph.mon.c.smithi189.stdout:Feb 17 12:14:45 smithi189 bash[75068]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mon-c 2024-02-17T12:14:46.264 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mon.c.service' 2024-02-17T12:14:46.309 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:14:46.310 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-17T12:14:46.310 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-17T12:14:46.310 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mgr.x 2024-02-17T12:14:46.639 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 12:14:46 smithi094 systemd[1]: Stopping Ceph mgr.x for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:14:46.941 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 12:14:46 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x[32729]: 2024-02-17T12:14:46.636+0000 7f2ccf753700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:14:46.941 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 12:14:46 smithi094 podman[108440]: 2024-02-17 12:14:46.667515158 +0000 UTC m=+0.142432140 container died 452c06f58c50e0d24beb2dfd45a3b94108942deeeaf99056fe52494bdff76a3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-17T12:14:47.208 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 12:14:47 smithi094 podman[108440]: 2024-02-17 12:14:47.038496166 +0000 UTC m=+0.513413148 container cleanup 452c06f58c50e0d24beb2dfd45a3b94108942deeeaf99056fe52494bdff76a3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:6822c7b54e949006be7a0802650d01723d834d1d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-17T12:14:47.208 INFO:journalctl@ceph.mgr.x.smithi094.stdout:Feb 17 12:14:47 smithi094 bash[108440]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-x 2024-02-17T12:14:47.465 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mgr.x.service' 2024-02-17T12:14:47.502 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:14:47.502 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-17T12:14:47.502 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-17T12:14:47.502 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mgr.y 2024-02-17T12:14:47.919 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 12:14:47 smithi125 systemd[1]: Stopping Ceph mgr.y for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:14:48.437 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 12:14:48 smithi125 podman[73771]: 2024-02-17 12:14:48.184431775 +0000 UTC m=+0.472644006 container died bb76ce3979008a37b54d1708008b9d54bb6c82e4eb375293ccf834bd80d28bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-17T12:14:49.687 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 12:14:49 smithi125 podman[73771]: 2024-02-17 12:14:49.5492527 +0000 UTC m=+1.837464930 container cleanup bb76ce3979008a37b54d1708008b9d54bb6c82e4eb375293ccf834bd80d28bf0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-17T12:14:49.688 INFO:journalctl@ceph.mgr.y.smithi125.stdout:Feb 17 12:14:49 smithi125 bash[73771]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-y 2024-02-17T12:14:50.037 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mgr.y.service' 2024-02-17T12:14:50.075 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:14:50.075 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-17T12:14:50.075 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-17T12:14:50.075 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mgr.z 2024-02-17T12:14:50.404 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 12:14:50 smithi189 systemd[1]: Stopping Ceph mgr.z for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:14:51.654 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 12:14:51 smithi189 podman[75191]: 2024-02-17 12:14:51.322412968 +0000 UTC m=+1.052018622 container died 907011b69dcd9fd9bad578f3a60863c27376ebb6526de702df265092aac13b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T12:14:52.053 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 12:14:51 smithi189 podman[75191]: 2024-02-17 12:14:51.783158503 +0000 UTC m=+1.512764143 container cleanup 907011b69dcd9fd9bad578f3a60863c27376ebb6526de702df265092aac13b32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-17T12:14:52.053 INFO:journalctl@ceph.mgr.z.smithi189.stdout:Feb 17 12:14:51 smithi189 bash[75191]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mgr-z 2024-02-17T12:14:52.274 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mgr.z.service' 2024-02-17T12:14:52.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:14:52.312 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-17T12:14:52.312 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-17T12:14:52.312 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.0 2024-02-17T12:14:52.630 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:52 smithi094 systemd[1]: Stopping Ceph osd.0 for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:14:52.958 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:52 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0[47699]: 2024-02-17T12:14:52.627+0000 7f680216d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:14:52.958 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:52 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0[47699]: 2024-02-17T12:14:52.628+0000 7f680216d700 -1 osd.0 111 *** Got signal Terminated *** 2024-02-17T12:14:52.959 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:52 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0[47699]: 2024-02-17T12:14:52.628+0000 7f680216d700 -1 osd.0 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T12:14:58.627 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:58 smithi094 podman[108561]: 2024-02-17 12:14:58.339282051 +0000 UTC m=+5.835589456 container died 4e27d1ad4c35ec305c91703babef102318b91e74d5d7d907c789333bf3dcec69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-17T12:14:58.958 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:58 smithi094 podman[108561]: 2024-02-17 12:14:58.721094922 +0000 UTC m=+6.217402336 container cleanup 4e27d1ad4c35ec305c91703babef102318b91e74d5d7d907c789333bf3dcec69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux ) 2024-02-17T12:14:58.958 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:58 smithi094 bash[108561]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0 2024-02-17T12:14:59.318 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:59 smithi094 podman[108581]: 2024-02-17 12:14:59.006246725 +0000 UTC m=+0.662074127 container remove 4e27d1ad4c35ec305c91703babef102318b91e74d5d7d907c789333bf3dcec69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-17T12:14:59.587 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:59 smithi094 podman[108667]: 2024-02-17 12:14:59.219884743 +0000 UTC m=+0.017817497 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:14:59.587 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:59 smithi094 podman[108667]: 2024-02-17 12:14:59.349202049 +0000 UTC m=+0.147134835 container create 27de35284eb64aae63c4f00a4dc7637168a89020bbfb182fec906afac03f36cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-02-17T12:14:59.931 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:59 smithi094 podman[108667]: 2024-02-17 12:14:59.585586855 +0000 UTC m=+0.383519599 container init 27de35284eb64aae63c4f00a4dc7637168a89020bbfb182fec906afac03f36cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:14:59.931 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:59 smithi094 podman[108667]: 2024-02-17 12:14:59.58926507 +0000 UTC m=+0.387197820 container start 27de35284eb64aae63c4f00a4dc7637168a89020bbfb182fec906afac03f36cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-02-17T12:14:59.931 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:59 smithi094 podman[108667]: 2024-02-17 12:14:59.69589247 +0000 UTC m=+0.493825229 container attach 27de35284eb64aae63c4f00a4dc7637168a89020bbfb182fec906afac03f36cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T12:15:00.208 INFO:journalctl@ceph.osd.0.smithi094.stdout:Feb 17 12:14:59 smithi094 podman[108667]: 2024-02-17 12:14:59.929166912 +0000 UTC m=+0.727099654 container died 27de35284eb64aae63c4f00a4dc7637168a89020bbfb182fec906afac03f36cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-17T12:15:00.645 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.0.service' 2024-02-17T12:15:00.688 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:15:00.688 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-17T12:15:00.688 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-17T12:15:00.688 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.1 2024-02-17T12:15:01.022 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:00 smithi125 systemd[1]: Stopping Ceph osd.1 for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:15:01.437 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1[41431]: 2024-02-17T12:15:01.019+0000 7fa2c859b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:15:01.438 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1[41431]: 2024-02-17T12:15:01.019+0000 7fa2c859b700 -1 osd.1 111 *** Got signal Terminated *** 2024-02-17T12:15:01.438 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1[41431]: 2024-02-17T12:15:01.019+0000 7fa2c859b700 -1 osd.1 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T12:15:07.031 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:06 smithi125 podman[73907]: 2024-02-17 12:15:06.737386015 +0000 UTC m=+5.850592547 container died ae4c64f2631d3967a2428684a450225261d00b01891c392b2c77f41eac223c84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1, RELEASE=main-6822c7b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-02-17T12:15:07.390 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:07 smithi125 podman[73907]: 2024-02-17 12:15:07.128427712 +0000 UTC m=+6.241634245 container cleanup ae4c64f2631d3967a2428684a450225261d00b01891c392b2c77f41eac223c84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-02-17T12:15:07.390 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:07 smithi125 bash[73907]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1 2024-02-17T12:15:07.686 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:07 smithi125 podman[73924]: 2024-02-17 12:15:07.388339305 +0000 UTC m=+0.643427597 container remove ae4c64f2631d3967a2428684a450225261d00b01891c392b2c77f41eac223c84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, ceph=True, RELEASE=main-6822c7b) 2024-02-17T12:15:07.938 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:07 smithi125 podman[74013]: 2024-02-17 12:15:07.587454942 +0000 UTC m=+0.017697926 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:15:07.938 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:07 smithi125 podman[74013]: 2024-02-17 12:15:07.726650906 +0000 UTC m=+0.156893877 container create 7ca29745c9b653dbe045f7ff9bd367e5ae5fa3a568ef9e7c74a518e61a523a83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1-deactivate, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-17T12:15:08.328 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:07 smithi125 podman[74013]: 2024-02-17 12:15:07.952390503 +0000 UTC m=+0.382633493 container init 7ca29745c9b653dbe045f7ff9bd367e5ae5fa3a568ef9e7c74a518e61a523a83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=main-6822c7b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-17T12:15:08.328 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:07 smithi125 podman[74013]: 2024-02-17 12:15:07.955685337 +0000 UTC m=+0.385928311 container start 7ca29745c9b653dbe045f7ff9bd367e5ae5fa3a568ef9e7c74a518e61a523a83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=main-6822c7b) 2024-02-17T12:15:08.328 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:08 smithi125 podman[74013]: 2024-02-17 12:15:08.067105422 +0000 UTC m=+0.497348412 container attach 7ca29745c9b653dbe045f7ff9bd367e5ae5fa3a568ef9e7c74a518e61a523a83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-17T12:15:08.597 INFO:journalctl@ceph.osd.1.smithi125.stdout:Feb 17 12:15:08 smithi125 podman[74013]: 2024-02-17 12:15:08.326718222 +0000 UTC m=+0.756961223 container died 7ca29745c9b653dbe045f7ff9bd367e5ae5fa3a568ef9e7c74a518e61a523a83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-1-deactivate, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T12:15:09.024 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.1.service' 2024-02-17T12:15:09.065 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:15:09.065 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-17T12:15:09.066 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-17T12:15:09.066 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.2 2024-02-17T12:15:09.381 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:09 smithi189 systemd[1]: Stopping Ceph osd.2 for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:15:09.654 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:09 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2[41765]: 2024-02-17T12:15:09.378+0000 7fd52fce0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:15:09.654 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:09 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2[41765]: 2024-02-17T12:15:09.378+0000 7fd52fce0700 -1 osd.2 111 *** Got signal Terminated *** 2024-02-17T12:15:09.654 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:09 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2[41765]: 2024-02-17T12:15:09.378+0000 7fd52fce0700 -1 osd.2 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T12:15:15.256 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:14 smithi189 podman[75328]: 2024-02-17 12:15:14.979304846 +0000 UTC m=+5.715693559 container died 3483965cd0cd5c3f500a0f05ba1b224347fa9c8f418ba4447233e74657616e97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-17T12:15:15.649 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:15 smithi189 podman[75328]: 2024-02-17 12:15:15.345851028 +0000 UTC m=+6.082239740 container cleanup 3483965cd0cd5c3f500a0f05ba1b224347fa9c8f418ba4447233e74657616e97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=) 2024-02-17T12:15:15.649 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:15 smithi189 bash[75328]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2 2024-02-17T12:15:15.904 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:15 smithi189 podman[75345]: 2024-02-17 12:15:15.64728231 +0000 UTC m=+0.659359445 container remove 3483965cd0cd5c3f500a0f05ba1b224347fa9c8f418ba4447233e74657616e97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-17T12:15:16.219 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:15 smithi189 podman[75432]: 2024-02-17 12:15:15.862477817 +0000 UTC m=+0.017534117 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:15:16.220 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:15 smithi189 podman[75432]: 2024-02-17 12:15:15.97342173 +0000 UTC m=+0.128478070 container create 00f95bd8ae7b744194323a186b4aa59db879a3f98bdfde92a79bb1dcf1601af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, ceph=True) 2024-02-17T12:15:16.624 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:16 smithi189 podman[75432]: 2024-02-17 12:15:16.218129138 +0000 UTC m=+0.373185456 container init 00f95bd8ae7b744194323a186b4aa59db879a3f98bdfde92a79bb1dcf1601af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-17T12:15:16.624 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:16 smithi189 podman[75432]: 2024-02-17 12:15:16.221586248 +0000 UTC m=+0.376642541 container start 00f95bd8ae7b744194323a186b4aa59db879a3f98bdfde92a79bb1dcf1601af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-17T12:15:16.624 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:16 smithi189 podman[75432]: 2024-02-17 12:15:16.309563387 +0000 UTC m=+0.464619690 container attach 00f95bd8ae7b744194323a186b4aa59db879a3f98bdfde92a79bb1dcf1601af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T12:15:16.904 INFO:journalctl@ceph.osd.2.smithi189.stdout:Feb 17 12:15:16 smithi189 podman[75432]: 2024-02-17 12:15:16.622447237 +0000 UTC m=+0.777503544 container died 00f95bd8ae7b744194323a186b4aa59db879a3f98bdfde92a79bb1dcf1601af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-02-17T12:15:17.205 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.2.service' 2024-02-17T12:15:17.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:15:17.250 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-17T12:15:17.250 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-17T12:15:17.250 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.3 2024-02-17T12:15:17.580 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:17 smithi094 systemd[1]: Stopping Ceph osd.3 for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:15:17.843 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:17 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3[54664]: 2024-02-17T12:15:17.577+0000 7fab7c572700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:15:17.843 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:17 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3[54664]: 2024-02-17T12:15:17.577+0000 7fab7c572700 -1 osd.3 111 *** Got signal Terminated *** 2024-02-17T12:15:17.843 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:17 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3[54664]: 2024-02-17T12:15:17.577+0000 7fab7c572700 -1 osd.3 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T12:15:19.706 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:19 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:19.286+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:19.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:19 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:19.704+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:20.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:20 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:20.278+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:20.708 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:20 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:20.288+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:21.208 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:20 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:20.744+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:21.522 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:21 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:21.213+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:21.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:21 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:21.249+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:21.708 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:21 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:21.283+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:21.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:21 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:21.519+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:22.208 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:21 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:21.729+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:22.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:22 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:22.217+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:22.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:22 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:22.142+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:22.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:22 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:22.291+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:22.708 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:22 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:22.325+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:22.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:22 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:22.517+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:23.208 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:22 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:22.713+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:23.488 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:23 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:23.168+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:23.488 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:23 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:23.251+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:23.673 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:23 smithi094 podman[108774]: 2024-02-17 12:15:23.316759715 +0000 UTC m=+5.879207336 container died 0058133c74461d2ac53926a7814770873bdae3d16a5fabf823c792ee91791cc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-17T12:15:23.673 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:23 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:23.320+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:23.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:23 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:23.257+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:23.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:23 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:23.485+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:23.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:23 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:23.681+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:23.959 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:23 smithi094 podman[108774]: 2024-02-17 12:15:23.770329754 +0000 UTC m=+6.332777386 container cleanup 0058133c74461d2ac53926a7814770873bdae3d16a5fabf823c792ee91791cc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:15:23.959 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:23 smithi094 bash[108774]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3 2024-02-17T12:15:24.370 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:24 smithi094 podman[108803]: 2024-02-17 12:15:24.088889715 +0000 UTC m=+0.766274957 container remove 0058133c74461d2ac53926a7814770873bdae3d16a5fabf823c792ee91791cc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=main-6822c7b, GIT_CLEAN=True) 2024-02-17T12:15:24.515 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:24 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:24.185+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:24.516 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:24 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:24.234+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:24.661 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:24 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:24.367+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:24.661 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:24 smithi094 podman[108891]: 2024-02-17 12:15:24.30030755 +0000 UTC m=+0.017491503 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:15:24.662 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:24 smithi094 podman[108891]: 2024-02-17 12:15:24.423329774 +0000 UTC m=+0.140513768 container create 8d737bd4076314750d878c1f9c4e4c1a5b3fefb4a1846b4e60456536a7be702d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-17T12:15:24.662 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:24 smithi094 podman[108891]: 2024-02-17 12:15:24.659673766 +0000 UTC m=+0.376857716 container init 8d737bd4076314750d878c1f9c4e4c1a5b3fefb4a1846b4e60456536a7be702d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3-deactivate, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214) 2024-02-17T12:15:24.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:24 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:24.298+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:24.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:24 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:24.513+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:24.952 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:24 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:24.687+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:24.952 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:24 smithi094 podman[108891]: 2024-02-17 12:15:24.663306896 +0000 UTC m=+0.380490849 container start 8d737bd4076314750d878c1f9c4e4c1a5b3fefb4a1846b4e60456536a7be702d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-17T12:15:24.952 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:24 smithi094 podman[108891]: 2024-02-17 12:15:24.785241326 +0000 UTC m=+0.502425278 container attach 8d737bd4076314750d878c1f9c4e4c1a5b3fefb4a1846b4e60456536a7be702d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3-deactivate, ceph=True, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-17T12:15:25.208 INFO:journalctl@ceph.osd.3.smithi094.stdout:Feb 17 12:15:24 smithi094 podman[108891]: 2024-02-17 12:15:24.950709035 +0000 UTC m=+0.667892984 container died 8d737bd4076314750d878c1f9c4e4c1a5b3fefb4a1846b4e60456536a7be702d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-02-17T12:15:25.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:25 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:25.147+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:25.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:25 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:25.233+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:25.558 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:25 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:25.359+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:25.577 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.3.service' 2024-02-17T12:15:25.621 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:15:25.621 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-17T12:15:25.621 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-17T12:15:25.621 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.4 2024-02-17T12:15:25.650 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:25 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:25.262+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:25.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:25 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:25.498+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:25.908 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:25 smithi125 systemd[1]: Stopping Ceph osd.4 for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:15:25.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:25 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:25.646+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:26.187 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:25 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4[48098]: 2024-02-17T12:15:25.905+0000 7fc8db618700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:15:26.188 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:25 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4[48098]: 2024-02-17T12:15:25.905+0000 7fc8db618700 -1 osd.4 111 *** Got signal Terminated *** 2024-02-17T12:15:26.188 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:25 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4[48098]: 2024-02-17T12:15:25.905+0000 7fc8db618700 -1 osd.4 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T12:15:26.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:26 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:26.098+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:26.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:26 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:26.271+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:26.685 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:26 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:26.395+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:26.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:26 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:26.307+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:26.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:26 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:26.490+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:26.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:26 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:26.683+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:27.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:27 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:27.120+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:27.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:27 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:27.259+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:27.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:27 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:27.303+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:27.708 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:27 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:27.376+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:27.894 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:27 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:27.448+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:28.154 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:28 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:28.140+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:28.208 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:27 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:27.733+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:28.501 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:28 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:28.239+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:28.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:28 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:28.339+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:28.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:28 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:28.339+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:28.708 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:28 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:28.399+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:28.708 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:28 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:28.399+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:28.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:28 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:28.498+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:29.208 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:28 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:28.707+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:29.545 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:29 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:29.161+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:29.545 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:29 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:29.212+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:29.687 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:29 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4[48098]: 2024-02-17T12:15:29.238+0000 7fc8d7c0c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.492717+0000 front 2024-02-17T12:15:05.492763+0000 (oldest deadline 2024-02-17T12:15:28.992869+0000) 2024-02-17T12:15:29.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:29 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:29.339+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:29.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:29 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:29.339+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:29.708 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:29 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:29.681+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:29.708 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:29 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:29.432+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:29.709 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:29 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:29.432+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:29.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:29 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:29.543+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:29.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:29 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:29.543+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:03.853529+0000 front 2024-02-17T12:15:03.853542+0000 (oldest deadline 2024-02-17T12:15:28.553510+0000) 2024-02-17T12:15:30.575 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:30 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:30.205+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:30.575 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:30 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:30.209+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:30.687 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:30 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4[48098]: 2024-02-17T12:15:30.251+0000 7fc8d7c0c700 -1 osd.4 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.492717+0000 front 2024-02-17T12:15:05.492763+0000 (oldest deadline 2024-02-17T12:15:28.992869+0000) 2024-02-17T12:15:30.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:30 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:30.379+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:30.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:30 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:30.379+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:30.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:30 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:30.573+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:30.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:30 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:30.573+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:03.853529+0000 front 2024-02-17T12:15:03.853542+0000 (oldest deadline 2024-02-17T12:15:28.553510+0000) 2024-02-17T12:15:30.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:30 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:30.642+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:30.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:30 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:30.470+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:30.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:30 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:30.470+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:31.553 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:31 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:31.203+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:31.554 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:31 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:31.237+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:31.554 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:31 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:31.237+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:31.687 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:31 smithi125 podman[74134]: 2024-02-17 12:15:31.533958984 +0000 UTC m=+5.722147380 container died 3818f7c8493b52e2fe9ec0894898f3f1b8052641be5a8cf52ec2b1ba0ec20e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-17T12:15:31.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:31 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:31.360+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:31.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:31 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:31.360+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:31.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:31 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:31.550+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:31.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:31 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:31.550+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:03.853529+0000 front 2024-02-17T12:15:03.853542+0000 (oldest deadline 2024-02-17T12:15:28.553510+0000) 2024-02-17T12:15:31.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:31 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:31.513+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:31.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:31 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:31.513+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:31.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:31 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:31.672+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:31.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:31 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:31.672+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:32.187 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:31 smithi125 podman[74134]: 2024-02-17 12:15:31.923093004 +0000 UTC m=+6.111281398 container cleanup 3818f7c8493b52e2fe9ec0894898f3f1b8052641be5a8cf52ec2b1ba0ec20e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-02-17T12:15:32.188 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:31 smithi125 bash[74134]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4 2024-02-17T12:15:32.539 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:32 smithi125 podman[74153]: 2024-02-17 12:15:32.232775903 +0000 UTC m=+0.677666559 container remove 3818f7c8493b52e2fe9ec0894898f3f1b8052641be5a8cf52ec2b1ba0ec20e13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, GIT_CLEAN=True) 2024-02-17T12:15:32.540 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:32 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:32.366+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:32.540 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:32 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:32.366+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:32.557 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:32 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:32.239+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:32.557 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:32 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:32.188+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:32.557 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:32 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:32.188+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:32.821 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:32 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:32.748+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:32.821 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:32 smithi125 podman[74244]: 2024-02-17 12:15:32.440800744 +0000 UTC m=+0.017322452 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:15:32.821 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:32 smithi125 podman[74244]: 2024-02-17 12:15:32.566907715 +0000 UTC m=+0.143429414 container create 6f702f6475416e67bd6ee38b45cf324edd77b4f0c074f865c6bcd02d797f855d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214) 2024-02-17T12:15:32.821 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:32 smithi125 podman[74244]: 2024-02-17 12:15:32.819552392 +0000 UTC m=+0.396074091 container init 6f702f6475416e67bd6ee38b45cf324edd77b4f0c074f865c6bcd02d797f855d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-17T12:15:32.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:32 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:32.554+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:32.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:32 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:32.554+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:03.853529+0000 front 2024-02-17T12:15:03.853542+0000 (oldest deadline 2024-02-17T12:15:28.553510+0000) 2024-02-17T12:15:32.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:32 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:32.562+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:32.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:32 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:32.562+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:32.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:32 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:32.702+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:32.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:32 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:32.702+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:33.109 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:32 smithi125 podman[74244]: 2024-02-17 12:15:32.822848702 +0000 UTC m=+0.399370395 container start 6f702f6475416e67bd6ee38b45cf324edd77b4f0c074f865c6bcd02d797f855d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-17T12:15:33.109 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:32 smithi125 podman[74244]: 2024-02-17 12:15:32.927783378 +0000 UTC m=+0.504305086 container attach 6f702f6475416e67bd6ee38b45cf324edd77b4f0c074f865c6bcd02d797f855d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-17T12:15:33.397 INFO:journalctl@ceph.osd.4.smithi125.stdout:Feb 17 12:15:33 smithi125 podman[74244]: 2024-02-17 12:15:33.10762959 +0000 UTC m=+0.684151295 container died 6f702f6475416e67bd6ee38b45cf324edd77b4f0c074f865c6bcd02d797f855d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-17T12:15:33.520 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:33 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:33.189+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:33.520 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:33 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:33.186+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:33.520 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:33 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:33.186+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:33.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:33 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:33.395+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:33.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:33 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:33.395+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:33.825 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.4.service' 2024-02-17T12:15:33.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:33 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:33.518+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:33.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:33 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:33.518+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:03.853529+0000 front 2024-02-17T12:15:03.853542+0000 (oldest deadline 2024-02-17T12:15:28.553510+0000) 2024-02-17T12:15:33.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:15:33.910 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-17T12:15:33.910 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-17T12:15:33.911 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.5 2024-02-17T12:15:33.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:33 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:33.593+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:33.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:33 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:33.593+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:33.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:33 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:33.749+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:33.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:33 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:33.749+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:34.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:33 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:33.705+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:34.227 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:33 smithi189 systemd[1]: Stopping Ceph osd.5 for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:15:34.227 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:34 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:34.159+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:34.227 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:34 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:34.159+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:34.498 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:34 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:34.237+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:34.498 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:34 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:34.224+0000 7fc4d0360700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:15:34.498 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:34 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:34.224+0000 7fc4d0360700 -1 osd.5 111 *** Got signal Terminated *** 2024-02-17T12:15:34.499 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:34 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:34.224+0000 7fc4d0360700 -1 osd.5 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T12:15:34.499 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:34 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:34.495+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:34.499 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:34 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:34.495+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:03.853529+0000 front 2024-02-17T12:15:03.853542+0000 (oldest deadline 2024-02-17T12:15:28.553510+0000) 2024-02-17T12:15:34.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:34 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:34.417+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:34.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:34 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:34.417+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:34.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:34 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:34.579+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:34.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:34 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:34.579+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:34.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:34 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:34.733+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:34.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:34 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:34.733+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:35.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:34 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:34.703+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:35.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:35.203+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:35.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:35.110+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:35.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:35.110+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:35.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:35 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:35.402+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:35.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:35 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:35.402+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:35.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:35.446+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:35.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:35 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:35.446+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:03.853529+0000 front 2024-02-17T12:15:03.853542+0000 (oldest deadline 2024-02-17T12:15:28.553510+0000) 2024-02-17T12:15:35.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:35 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:35.736+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:35.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:35 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:35.736+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:35.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:35 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:35.551+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:35.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:35 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:35.551+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:36.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:35 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:35.710+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:36.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:36 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:36.246+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:36.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:36 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:36.246+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:36.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:36 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:36.138+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:36.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:36 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:36.138+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:36.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:36 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:36.138+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:36.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:36 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:36.384+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:36.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:36 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:36.384+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:36.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:36 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:36.459+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:36.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:36 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:36.459+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:03.853529+0000 front 2024-02-17T12:15:03.853542+0000 (oldest deadline 2024-02-17T12:15:28.553510+0000) 2024-02-17T12:15:36.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:36.574+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:36.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:36.574+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:36.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:36.773+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:36.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:36 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:36.773+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:37.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:36 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:36.717+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:37.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:36 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:36.717+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:37.429 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:37 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:37.169+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:37.429 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:37 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:37.169+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:37.429 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:37 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:37.169+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:37.430 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:37 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:37.292+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:37.430 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:37 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:37.292+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:37.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:37 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:37.341+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:37.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:37 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:37.341+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:37.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:37 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:37.341+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:37.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:37 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:37.426+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:37.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:37 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:37.426+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:03.853529+0000 front 2024-02-17T12:15:03.853542+0000 (oldest deadline 2024-02-17T12:15:28.553510+0000) 2024-02-17T12:15:37.904 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:37 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:37.426+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.654588+0000 front 2024-02-17T12:15:12.654612+0000 (oldest deadline 2024-02-17T12:15:36.754813+0000) 2024-02-17T12:15:37.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:37 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:37.814+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:37.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:37 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:37.814+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:37.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:37 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:37.565+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:37.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:37 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:37.565+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:38.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:37 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:37.758+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:38.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:37 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:37.758+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:38.404 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:38 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:38.381+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.151545+0000 front 2024-02-17T12:14:55.151444+0000 (oldest deadline 2024-02-17T12:15:20.952145+0000) 2024-02-17T12:15:38.404 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:38 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:38.381+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:03.853529+0000 front 2024-02-17T12:15:03.853542+0000 (oldest deadline 2024-02-17T12:15:28.553510+0000) 2024-02-17T12:15:38.404 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:38 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5[48723]: 2024-02-17T12:15:38.381+0000 7fc4c894c700 -1 osd.5 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.654588+0000 front 2024-02-17T12:15:12.654612+0000 (oldest deadline 2024-02-17T12:15:36.754813+0000) 2024-02-17T12:15:38.405 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:38 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:38.336+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:38.405 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:38 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:38.336+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:38.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:38 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:38.136+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:38.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:38 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:38.136+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:38.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:38 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:38.136+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:38.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:38 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:38.377+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:38.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:38 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:38.377+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:38.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:38 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:38.377+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:38.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:38 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:38.527+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:38.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:38 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:38.527+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:38.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:38 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:38.769+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:38.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:38 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:38.769+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:38.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:38 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:38.769+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:39.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:38 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:38.760+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:39.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:38 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:38.761+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:39.354 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:39 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:39.099+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:39.354 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:39 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:39.099+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:39.354 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:39 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:39.099+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:39.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:39 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:39.352+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:39.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:39 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:39.352+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:39.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:39 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:39.420+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:39.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:39 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:39.420+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:39.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:39 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:39.420+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:39.779 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:39 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:39.512+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:39.779 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:39 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:39.512+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:40.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:39 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:39.747+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:40.230 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:39 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:39.747+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:40.231 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:39 smithi189 podman[75552]: 2024-02-17 12:15:39.922998449 +0000 UTC m=+5.820810906 container died 085ff22ec9d9d44111bbf7feddde5c5af6993e86be8e5b0f06d4f9c5a239b9bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-02-17T12:15:40.231 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:39 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:39.776+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:40.231 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:39 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:39.776+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:40.232 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:39 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:39.776+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:40.232 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:40 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:40.088+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:40.232 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:40 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:40.088+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:40.232 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:40 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:40.088+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:40.570 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:40 smithi189 podman[75552]: 2024-02-17 12:15:40.316696766 +0000 UTC m=+6.214509222 container cleanup 085ff22ec9d9d44111bbf7feddde5c5af6993e86be8e5b0f06d4f9c5a239b9bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240214) 2024-02-17T12:15:40.570 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:40 smithi189 bash[75552]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5 2024-02-17T12:15:40.570 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:40 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:40.352+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:40.570 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:40 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:40.352+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:40.751 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:40 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:40.458+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:40.751 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:40 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:40.458+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:40.751 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:40 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:40.458+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:40.864 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:40 smithi189 podman[75569]: 2024-02-17 12:15:40.568266069 +0000 UTC m=+0.639862905 container remove 085ff22ec9d9d44111bbf7feddde5c5af6993e86be8e5b0f06d4f9c5a239b9bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, GIT_BRANCH=HEAD) 2024-02-17T12:15:40.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:40 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:40.517+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:40.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:40 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:40.517+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:40.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:40 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:40.517+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.621510+0000 front 2024-02-17T12:15:13.621533+0000 (oldest deadline 2024-02-17T12:15:39.521608+0000) 2024-02-17T12:15:40.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:40 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:40.745+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:40.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:40 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:40.745+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:40.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:40 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:40.745+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:41.155 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:40 smithi189 podman[75659]: 2024-02-17 12:15:40.765585384 +0000 UTC m=+0.017819679 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:15:41.155 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:40 smithi189 podman[75659]: 2024-02-17 12:15:40.877531015 +0000 UTC m=+0.129765291 container create ebece399e08a75beb1d003f4ad129401b60726cd796727832453d21b4fac9261 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-17T12:15:41.155 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:41 smithi189 podman[75659]: 2024-02-17 12:15:41.063657948 +0000 UTC m=+0.315892239 container init ebece399e08a75beb1d003f4ad129401b60726cd796727832453d21b4fac9261 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-17T12:15:41.155 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:41 smithi189 podman[75659]: 2024-02-17 12:15:41.067104719 +0000 UTC m=+0.319339011 container start ebece399e08a75beb1d003f4ad129401b60726cd796727832453d21b4fac9261 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=main-6822c7b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-17T12:15:41.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:41 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:41.103+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:41.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:41 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:41.103+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:41.156 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:41 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:41.103+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:41.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:40 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:40.748+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:41.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:40 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:40.748+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:41.652 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:41 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:41.358+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:41.652 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:41 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:41.358+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:41.653 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:41 smithi189 podman[75659]: 2024-02-17 12:15:41.155187547 +0000 UTC m=+0.407421871 container attach ebece399e08a75beb1d003f4ad129401b60726cd796727832453d21b4fac9261 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5-deactivate, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-17T12:15:41.653 INFO:journalctl@ceph.osd.5.smithi189.stdout:Feb 17 12:15:41 smithi189 podman[75659]: 2024-02-17 12:15:41.362067498 +0000 UTC m=+0.614301783 container died ebece399e08a75beb1d003f4ad129401b60726cd796727832453d21b4fac9261 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214) 2024-02-17T12:15:41.937 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:41 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:41.505+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:41.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:41 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:41.505+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:41.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:41 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:41.505+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:41.938 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:41 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:41.749+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:41.938 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:41 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:41.749+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:41.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:41 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:41.507+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:41.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:41 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:41.507+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:41.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:41 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:41.507+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.621510+0000 front 2024-02-17T12:15:13.621533+0000 (oldest deadline 2024-02-17T12:15:39.521608+0000) 2024-02-17T12:15:41.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:41 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:41.696+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:41.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:41 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:41.696+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:41.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:41 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:41.696+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:41.978 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.5.service' 2024-02-17T12:15:42.021 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:15:42.021 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-17T12:15:42.021 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-17T12:15:42.022 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.6 2024-02-17T12:15:42.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:42 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:42.114+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:42.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:42 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:42.114+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:42.334 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:42 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:42.114+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:42.437 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:42 smithi094 systemd[1]: Stopping Ceph osd.6 for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:15:42.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:42 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:42.332+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:42.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:42 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:42.332+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:42.709 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:42 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:42.435+0000 7f084f9ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:15:42.709 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:42 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:42.435+0000 7f084f9ab700 -1 osd.6 111 *** Got signal Terminated *** 2024-02-17T12:15:42.709 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:42 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:42.435+0000 7f084f9ab700 -1 osd.6 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T12:15:42.709 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:42 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:42.534+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:42.709 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:42 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:42.534+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:42.709 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:42 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:42.534+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.621510+0000 front 2024-02-17T12:15:13.621533+0000 (oldest deadline 2024-02-17T12:15:39.521608+0000) 2024-02-17T12:15:42.714 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:42 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:42.457+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:42.715 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:42 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:42.457+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:42.715 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:42 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:42.457+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:43.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:42 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:42.712+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:43.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:42 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:42.712+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:43.208 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:42 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:42.722+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:43.208 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:42 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:42.722+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:43.209 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:42 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:42.722+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:43.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:43 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:43.319+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:43.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:43 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:43.319+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:43.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:43 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:43.111+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:43.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:43 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:43.111+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:43.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:43 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:43.111+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:43.730 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:43 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:43.471+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:43.730 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:43 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:43.471+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:43.731 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:43 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:43.471+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:43.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:43 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:43.579+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:43.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:43 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:43.579+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:43.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:43 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:43.579+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.621510+0000 front 2024-02-17T12:15:13.621533+0000 (oldest deadline 2024-02-17T12:15:39.521608+0000) 2024-02-17T12:15:43.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:43 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:43.725+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:43.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:43 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:43.725+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:43.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:43 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:43.725+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:44.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:43 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:43.727+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:44.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:43 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:43.727+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:44.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:44 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:44.275+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:44.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:44 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:44.275+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:44.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:44 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:44.114+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:44.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:44 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:44.114+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:44.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:44 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:44.114+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:44.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:44 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:44.514+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:44.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:44 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:44.514+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:44.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:44 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:44.514+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:44.938 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:44 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:44.728+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:44.938 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:44 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:44.728+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:44.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:44 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:44.585+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:44.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:44 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:44.585+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:44.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:44 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:44.585+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.621510+0000 front 2024-02-17T12:15:13.621533+0000 (oldest deadline 2024-02-17T12:15:39.521608+0000) 2024-02-17T12:15:44.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:44 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:44.676+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:44.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:44 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:44.676+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:44.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:44 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:44.676+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:45.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:45 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:45.269+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:45.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:45 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:45.269+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:45.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:45 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:45.136+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:45.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:45 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:45.136+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:45.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:45 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:45.136+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:45.938 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:45 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:45.753+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:45.938 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:45 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:45.753+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:45.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:45 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:45.560+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:45.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:45 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:45.560+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:45.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:45 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:45.560+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:45.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:45 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:45.663+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:45.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:45 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:45.663+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:45.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:45 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:45.663+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:45.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:45 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:45.626+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:45.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:45 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:45.626+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:45.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:45 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:45.626+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.621510+0000 front 2024-02-17T12:15:13.621533+0000 (oldest deadline 2024-02-17T12:15:39.521608+0000) 2024-02-17T12:15:46.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:46 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:46.310+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:46.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:46 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:46.310+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:46.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:46 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:46.310+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:46.655 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:46 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:46.174+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:46.655 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:46 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:46.174+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:46.655 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:46 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:46.174+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:46.655 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:46 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:46.174+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:46.937 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:46 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:46.773+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:46.938 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:46 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:46.773+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:46.938 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:46 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:46.773+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:18.797441+0000 front 2024-02-17T12:15:18.797496+0000 (oldest deadline 2024-02-17T12:15:45.798002+0000) 2024-02-17T12:15:46.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:46 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:46.524+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:46.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:46 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:46.524+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:46.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:46 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:46.524+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:46.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:46 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:46.615+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:55.018695+0000 front 2024-02-17T12:14:55.018644+0000 (oldest deadline 2024-02-17T12:15:18.518682+0000) 2024-02-17T12:15:46.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:46 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:46.615+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.420700+0000 front 2024-02-17T12:15:05.420781+0000 (oldest deadline 2024-02-17T12:15:27.720445+0000) 2024-02-17T12:15:46.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:46 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:46.615+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.621510+0000 front 2024-02-17T12:15:13.621533+0000 (oldest deadline 2024-02-17T12:15:39.521608+0000) 2024-02-17T12:15:46.959 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:46 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6[61660]: 2024-02-17T12:15:46.615+0000 7f084bf9f700 -1 osd.6 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.422714+0000 front 2024-02-17T12:15:22.422737+0000 (oldest deadline 2024-02-17T12:15:46.522915+0000) 2024-02-17T12:15:46.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:46 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:46.630+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:46.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:46 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:46.630+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:46.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:46 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:46.630+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:46.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:46 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:46.630+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:47.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:47 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:47.357+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:47.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:47 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:47.357+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:47.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:47 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:47.357+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:47.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:47 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:47.133+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:47.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:47 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:47.133+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:47.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:47 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:47.133+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:47.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:47 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:47.133+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:47.824 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:47 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:47.486+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:47.824 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:47 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:47.486+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:47.825 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:47 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:47.486+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:47.860 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:47 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:47.597+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:47.860 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:47 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:47.597+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:47.860 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:47 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:47.597+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:47.860 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:47 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:47.597+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:48.140 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:47 smithi094 podman[108999]: 2024-02-17 12:15:47.968831511 +0000 UTC m=+5.758098469 container died 3fd1c36b839e34a95d8041dd08c420af58aadac1386c086309ff9f4265a221c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T12:15:48.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:47 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:47.822+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:48.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:47 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:47.822+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:48.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:47 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:47.822+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:18.797441+0000 front 2024-02-17T12:15:18.797496+0000 (oldest deadline 2024-02-17T12:15:45.798002+0000) 2024-02-17T12:15:48.390 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:48 smithi094 podman[108999]: 2024-02-17 12:15:48.386279593 +0000 UTC m=+6.175546565 container cleanup 3fd1c36b839e34a95d8041dd08c420af58aadac1386c086309ff9f4265a221c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T12:15:48.390 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:48 smithi094 bash[108999]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6 2024-02-17T12:15:48.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:48 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:48.312+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:48.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:48 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:48.312+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:48.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:48 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:48.312+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:48.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:48 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:48.117+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:48.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:48 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:48.117+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:48.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:48 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:48.117+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:48.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:48 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:48.117+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:48.842 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:48 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:48.436+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:48.842 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:48 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:48.437+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:48.842 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:48 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:48.437+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:48.842 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:48 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:48.437+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:15:48.958 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:48 smithi094 podman[109017]: 2024-02-17 12:15:48.646165754 +0000 UTC m=+0.676072387 container remove 3fd1c36b839e34a95d8041dd08c420af58aadac1386c086309ff9f4265a221c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T12:15:48.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:48 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:48.572+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:48.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:48 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:48.573+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:48.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:48 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:48.573+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:48.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:48 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:48.573+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:49.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:48 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:48.839+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:49.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:48 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:48.839+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:49.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:48 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:48.839+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:18.797441+0000 front 2024-02-17T12:15:18.797496+0000 (oldest deadline 2024-02-17T12:15:45.798002+0000) 2024-02-17T12:15:49.347 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:49 smithi094 podman[109113]: 2024-02-17 12:15:48.995523191 +0000 UTC m=+0.017927071 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:15:49.347 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:49 smithi094 podman[109113]: 2024-02-17 12:15:49.122506182 +0000 UTC m=+0.144910061 container create 338745304b97cdbe06be91e02aab22c1127c423db213b084daf9a8fc1793eabc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6-deactivate, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-17T12:15:49.347 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:49 smithi094 podman[109113]: 2024-02-17 12:15:49.34205497 +0000 UTC m=+0.364458848 container init 338745304b97cdbe06be91e02aab22c1127c423db213b084daf9a8fc1793eabc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-02-17T12:15:49.347 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:49 smithi094 podman[109113]: 2024-02-17 12:15:49.345357548 +0000 UTC m=+0.367761436 container start 338745304b97cdbe06be91e02aab22c1127c423db213b084daf9a8fc1793eabc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-17T12:15:49.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:49 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:49.328+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:49.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:49 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:49.328+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:49.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:49 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:49.328+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:49.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:49 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:49.103+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:49.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:49 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:49.103+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:49.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:49 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:49.103+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:49.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:49 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:49.103+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:49.634 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:49 smithi094 podman[109113]: 2024-02-17 12:15:49.450703291 +0000 UTC m=+0.473107176 container attach 338745304b97cdbe06be91e02aab22c1127c423db213b084daf9a8fc1793eabc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-17T12:15:49.634 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:49 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:49.553+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:49.634 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:49 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:49.553+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:49.634 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:49 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:49.553+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:49.634 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:49 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:49.553+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:49.810 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:49 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:49.441+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:49.810 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:49 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:49.442+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:49.810 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:49 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:49.442+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:49.810 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:49 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:49.442+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:15:49.922 INFO:journalctl@ceph.osd.6.smithi094.stdout:Feb 17 12:15:49 smithi094 podman[109113]: 2024-02-17 12:15:49.631880966 +0000 UTC m=+0.654284844 container died 338745304b97cdbe06be91e02aab22c1127c423db213b084daf9a8fc1793eabc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, org.label-schema.build-date=20240214) 2024-02-17T12:15:50.089 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:49 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:49.807+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:50.089 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:49 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:49.807+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:50.089 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:49 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:49.807+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:18.797441+0000 front 2024-02-17T12:15:18.797496+0000 (oldest deadline 2024-02-17T12:15:45.798002+0000) 2024-02-17T12:15:50.277 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.6.service' 2024-02-17T12:15:50.317 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:15:50.317 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-17T12:15:50.317 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-17T12:15:50.317 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.7 2024-02-17T12:15:50.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:50 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:50.343+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:50.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:50 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:50.343+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:50.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:50 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:50.343+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:50.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:50 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:50.099+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:50.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:50 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:50.099+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:50.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:50 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:50.099+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:50.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:50 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:50.099+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:50.627 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:50 smithi125 systemd[1]: Stopping Ceph osd.7 for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:15:50.627 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:50 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:50.445+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:50.628 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:50 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:50.445+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:50.628 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:50 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:50.445+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:50.628 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:50 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:50.445+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:15:50.937 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:50 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:50.624+0000 7f378491f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:15:50.938 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:50 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:50.624+0000 7f378491f700 -1 osd.7 111 *** Got signal Terminated *** 2024-02-17T12:15:50.938 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:50 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:50.624+0000 7f378491f700 -1 osd.7 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T12:15:50.938 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:50 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:50.835+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:50.938 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:50 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:50.835+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:50.938 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:50 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:50.835+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:18.797441+0000 front 2024-02-17T12:15:18.797496+0000 (oldest deadline 2024-02-17T12:15:45.798002+0000) 2024-02-17T12:15:50.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:50 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:50.572+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:50.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:50 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:50.572+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:50.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:50 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:50.572+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:50.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:50 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:50.572+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:51.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:51 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:51.347+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:51.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:51 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:51.347+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:51.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:51 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:51.347+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:51.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:51 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:51.137+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:51.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:51 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:51.137+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:51.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:51 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:51.137+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:51.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:51 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:51.137+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:51.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:51 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:51.407+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:51.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:51 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:51.407+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:51.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:51 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:51.407+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:51.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:51 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:51.407+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:15:51.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:51 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:51.595+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:51.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:51 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:51.595+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:51.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:51 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:51.595+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:51.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:51 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:51.595+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:52.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:51 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:51.831+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:52.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:51 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:51.831+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:52.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:51 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:51.831+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:18.797441+0000 front 2024-02-17T12:15:18.797496+0000 (oldest deadline 2024-02-17T12:15:45.798002+0000) 2024-02-17T12:15:52.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:52 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:52.378+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:52.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:52 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:52.378+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:52.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:52 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:52.378+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:52.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:52 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:52.140+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:52.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:52 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:52.140+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:52.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:52 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:52.140+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:52.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:52 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:52.140+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:52.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:52 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:52.140+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:15:52.822 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:52 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:52.454+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:52.822 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:52 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:52.454+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:52.822 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:52 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:52.454+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:52.822 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:52 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:52.454+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:15:52.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:52 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:52.613+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:52.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:52 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:52.613+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:52.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:52 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:52.613+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:52.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:52 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:52.613+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:53.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:52 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:52.819+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:53.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:52 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:52.819+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:53.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:52 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:52.819+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:18.797441+0000 front 2024-02-17T12:15:18.797496+0000 (oldest deadline 2024-02-17T12:15:45.798002+0000) 2024-02-17T12:15:53.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:52 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:52.819+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.898868+0000 front 2024-02-17T12:15:29.898889+0000 (oldest deadline 2024-02-17T12:15:52.198924+0000) 2024-02-17T12:15:53.364 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:53 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:53.104+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:53.365 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:53 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:53.104+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:53.365 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:53 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:53.105+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:53.365 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:53 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:53.105+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:53.365 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:53 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:53.105+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:15:53.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:53 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:53.362+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:53.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:53 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:53.362+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:53.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:53 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:53.362+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:53.783 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:53 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:53.438+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:53.783 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:53 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:53.438+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:53.783 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:53 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:53.438+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:53.783 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:53 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:53.438+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:15:53.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:53.650+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:53.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:53.650+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:53.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:53.650+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:53.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:53 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:53.650+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:54.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:53 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:53.780+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:54.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:53 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:53.780+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:54.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:53 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:53.780+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:18.797441+0000 front 2024-02-17T12:15:18.797496+0000 (oldest deadline 2024-02-17T12:15:45.798002+0000) 2024-02-17T12:15:54.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:53 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:53.780+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.898868+0000 front 2024-02-17T12:15:29.898889+0000 (oldest deadline 2024-02-17T12:15:52.198924+0000) 2024-02-17T12:15:54.388 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:54 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:54.063+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:54.388 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:54 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:54.063+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:54.388 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:54 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:54.063+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:54.388 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:54 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:54.063+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:54.388 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:54 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:54.063+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:15:54.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:54 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:54.385+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:54.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:54 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:54.385+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:54.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:54 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:54.385+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:54.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:54 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:54.428+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:54.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:54 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:54.428+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:54.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:54 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:54.428+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:54.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:54 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:54.428+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:15:54.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:54 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:54.622+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:54.960 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:54 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:54.622+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:54.960 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:54 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:54.622+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:54.960 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:54 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:54.622+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:55.187 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:54 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:54.781+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.995012+0000 front 2024-02-17T12:15:05.995048+0000 (oldest deadline 2024-02-17T12:15:31.795917+0000) 2024-02-17T12:15:55.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:54 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:54.781+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.796423+0000 front 2024-02-17T12:15:11.796499+0000 (oldest deadline 2024-02-17T12:15:35.896510+0000) 2024-02-17T12:15:55.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:54 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:54.781+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:18.797441+0000 front 2024-02-17T12:15:18.797496+0000 (oldest deadline 2024-02-17T12:15:45.798002+0000) 2024-02-17T12:15:55.188 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:54 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7[54962]: 2024-02-17T12:15:54.781+0000 7f3780f13700 -1 osd.7 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.898868+0000 front 2024-02-17T12:15:29.898889+0000 (oldest deadline 2024-02-17T12:15:52.198924+0000) 2024-02-17T12:15:55.399 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:55 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:55.109+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:55.399 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:55 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:55.109+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:55.399 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:55 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:55.109+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:55.400 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:55 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:55.109+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:55.400 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:55 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:55.109+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:15:55.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:55 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:55.396+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:55.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:55 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:55.396+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:55.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:55 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:55.396+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:55.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:55 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:55.396+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.074579+0000 front 2024-02-17T12:15:30.074549+0000 (oldest deadline 2024-02-17T12:15:54.774368+0000) 2024-02-17T12:15:55.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:55 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:55.462+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:55.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:55 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:55.462+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:55.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:55 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:55.462+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:55.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:55 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:55.462+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:15:55.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:55 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:55.462+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:15:55.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:55 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:55.605+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:55.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:55 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:55.605+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:55.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:55 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:55.605+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:55.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:55 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:55.605+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:56.385 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:56 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:56.070+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:56.385 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:56 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:56.070+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:56.385 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:56 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:56.070+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:56.385 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:56 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:56.070+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:56.385 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:56 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:56.070+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:15:56.394 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:56 smithi125 podman[74359]: 2024-02-17 12:15:56.103363888 +0000 UTC m=+5.602197336 container died 54c335f255a294bd3f3dbb3dd5902bf1b7fd53322b15518a60cccd3985c09ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-02-17T12:15:56.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:56 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:56.382+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:56.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:56 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:56.382+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:56.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:56 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:56.382+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:56.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:56 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:56.382+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.074579+0000 front 2024-02-17T12:15:30.074549+0000 (oldest deadline 2024-02-17T12:15:54.774368+0000) 2024-02-17T12:15:56.687 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:56 smithi125 podman[74359]: 2024-02-17 12:15:56.516331218 +0000 UTC m=+6.015164651 container cleanup 54c335f255a294bd3f3dbb3dd5902bf1b7fd53322b15518a60cccd3985c09ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-02-17T12:15:56.687 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:56 smithi125 bash[74359]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7 2024-02-17T12:15:56.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:56 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:56.497+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:56.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:56 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:56.497+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:56.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:56 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:56.497+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:56.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:56 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:56.497+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:15:56.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:56 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:56.497+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:15:56.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:56 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:56.648+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:56.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:56 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:56.648+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:56.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:56 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:56.648+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:56.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:56 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:56.648+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:57.036 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:56 smithi125 podman[74378]: 2024-02-17 12:15:56.734568554 +0000 UTC m=+0.628923037 container remove 54c335f255a294bd3f3dbb3dd5902bf1b7fd53322b15518a60cccd3985c09ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-17T12:15:57.298 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:57 smithi125 podman[74473]: 2024-02-17 12:15:57.035189259 +0000 UTC m=+0.111251775 container create 788bd69e98edf60d9ab98e1e2928725a168a002a89580b5fa4ff624e7cabfe75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7-deactivate, RELEASE=main-6822c7b, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=) 2024-02-17T12:15:57.298 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:57 smithi125 podman[74473]: 2024-02-17 12:15:56.941723847 +0000 UTC m=+0.017786377 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:15:57.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:57 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:57.023+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:57.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:57 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:57.023+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:57.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:57 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:57.023+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:57.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:57 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:57.023+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:57.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:57 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:57.023+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:15:57.589 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:57 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:57.464+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:57.589 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:57 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:57.464+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:57.590 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:57 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:57.464+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:57.590 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:57 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:57.464+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:15:57.590 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:57 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:57.464+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:15:57.590 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:57 smithi125 podman[74473]: 2024-02-17 12:15:57.296508511 +0000 UTC m=+0.372571035 container init 788bd69e98edf60d9ab98e1e2928725a168a002a89580b5fa4ff624e7cabfe75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-02-17T12:15:57.590 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:57 smithi125 podman[74473]: 2024-02-17 12:15:57.300387035 +0000 UTC m=+0.376449553 container start 788bd69e98edf60d9ab98e1e2928725a168a002a89580b5fa4ff624e7cabfe75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-17T12:15:57.590 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:57 smithi125 podman[74473]: 2024-02-17 12:15:57.387980018 +0000 UTC m=+0.464042539 container attach 788bd69e98edf60d9ab98e1e2928725a168a002a89580b5fa4ff624e7cabfe75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=main-6822c7b) 2024-02-17T12:15:57.590 INFO:journalctl@ceph.osd.7.smithi125.stdout:Feb 17 12:15:57 smithi125 podman[74473]: 2024-02-17 12:15:57.587987746 +0000 UTC m=+0.664050282 container died 788bd69e98edf60d9ab98e1e2928725a168a002a89580b5fa4ff624e7cabfe75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-17T12:15:57.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:57 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:57.409+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:57.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:57 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:57.409+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:57.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:57 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:57.409+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:57.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:57 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:57.409+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.074579+0000 front 2024-02-17T12:15:30.074549+0000 (oldest deadline 2024-02-17T12:15:54.774368+0000) 2024-02-17T12:15:57.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:57 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:57.679+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:57.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:57 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:57.679+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:57.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:57 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:57.679+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:57.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:57 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:57.679+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:57.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:57 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:57.679+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:15:58.271 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.7.service' 2024-02-17T12:15:58.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:15:58.320 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-17T12:15:58.320 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-02-17T12:15:58.320 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.8 2024-02-17T12:15:58.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:58 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:58.039+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:58.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:58 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:58.039+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:58.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:58 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:58.039+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:58.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:58 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:58.039+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:58.333 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:58 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:58.039+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:15:58.605 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:58 smithi189 systemd[1]: Stopping Ceph osd.8 for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:15:58.605 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:58 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:58.392+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:58.606 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:58 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:58.392+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:58.606 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:58 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:58.392+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:58.606 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:58 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:58.392+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.074579+0000 front 2024-02-17T12:15:30.074549+0000 (oldest deadline 2024-02-17T12:15:54.774368+0000) 2024-02-17T12:15:58.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:58 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:58.490+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:58.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:58 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:58.490+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:58.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:58 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:58.490+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:58.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:58 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:58.490+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:15:58.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:58 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:58.490+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:15:58.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:58 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:58.603+0000 7fe8f64ff700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:15:58.905 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:58 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:58.603+0000 7fe8f64ff700 -1 osd.8 111 *** Got signal Terminated *** 2024-02-17T12:15:58.905 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:58 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:58.603+0000 7fe8f64ff700 -1 osd.8 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T12:15:58.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:58 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:58.699+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:58.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:58 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:58.699+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:58.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:58 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:58.699+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:58.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:58 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:58.699+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:58.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:58 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:58.699+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:15:59.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:59 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:59.020+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:15:59.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:59 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:59.020+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:15:59.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:59 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:59.020+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:15:59.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:59 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:59.020+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:15:59.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:15:59 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:15:59.020+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:15:59.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:59 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:59.440+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:15:59.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:59 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:59.440+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:15:59.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:59 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:59.440+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:15:59.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:15:59 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:15:59.440+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.074579+0000 front 2024-02-17T12:15:30.074549+0000 (oldest deadline 2024-02-17T12:15:54.774368+0000) 2024-02-17T12:15:59.937 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:59 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:59.529+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:15:59.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:59 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:59.529+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:15:59.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:59 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:59.529+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:15:59.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:59 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:59.529+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:15:59.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:15:59 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:15:59.529+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:15:59.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:59 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:59.654+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:15:59.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:59 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:59.654+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:15:59.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:59 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:59.654+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:15:59.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:59 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:59.654+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:15:59.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:15:59 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:15:59.654+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:16:00.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:00 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:00.023+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:00.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:00 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:00.023+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:00.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:00 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:00.023+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:00.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:00 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:00.023+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:00.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:00 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:00.023+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:00.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:00 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:00.461+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:16:00.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:00 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:00.461+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:16:00.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:00 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:00.461+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:16:00.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:00 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:00.461+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.074579+0000 front 2024-02-17T12:15:30.074549+0000 (oldest deadline 2024-02-17T12:15:54.774368+0000) 2024-02-17T12:16:00.937 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:00 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:00.504+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:00.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:00 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:00.504+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:00.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:00 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:00.504+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:00.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:00 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:00.504+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:00.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:00 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:00.504+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:00.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:00 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:00.504+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:00.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:00 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:00.608+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:16:00.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:00 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:00.608+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:16:00.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:00 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:00.608+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:16:00.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:00 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:00.608+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:16:00.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:00 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:00.608+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:16:01.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:01.000+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:01.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:01.000+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:01.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:01.000+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:01.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:01.000+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:01.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:01.000+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:01.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:01.425+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:16:01.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:01.425+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:16:01.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:01.425+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:16:01.905 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:01.425+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.074579+0000 front 2024-02-17T12:15:30.074549+0000 (oldest deadline 2024-02-17T12:15:54.774368+0000) 2024-02-17T12:16:01.905 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:01 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:01.425+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.774638+0000 front 2024-02-17T12:15:34.774681+0000 (oldest deadline 2024-02-17T12:16:00.674702+0000) 2024-02-17T12:16:01.937 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:01.466+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:01.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:01.466+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:01.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:01.466+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:01.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:01.466+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:01.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:01.466+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:01.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:01 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:01.466+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:01.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:01 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:01.616+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:16:01.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:01 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:01.616+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:16:01.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:01 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:01.616+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:16:01.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:01 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:01.616+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:16:01.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:01 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:01.616+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:16:01.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:01 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:01.616+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:36.748308+0000 front 2024-02-17T12:15:36.748317+0000 (oldest deadline 2024-02-17T12:16:01.448285+0000) 2024-02-17T12:16:02.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:02.005+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:02.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:02.005+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:02.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:02.005+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:02.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:02.005+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:02.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:02.005+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:02.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:02.005+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:02.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:02.455+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:16:02.905 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:02.455+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:16:02.905 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:02.455+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:16:02.905 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:02.455+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.074579+0000 front 2024-02-17T12:15:30.074549+0000 (oldest deadline 2024-02-17T12:15:54.774368+0000) 2024-02-17T12:16:02.905 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:02 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:02.455+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.774638+0000 front 2024-02-17T12:15:34.774681+0000 (oldest deadline 2024-02-17T12:16:00.674702+0000) 2024-02-17T12:16:02.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:02 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:02.440+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:02.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:02 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:02.440+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:02.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:02 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:02.440+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:02.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:02 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:02.440+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:02.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:02 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:02.440+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:02.938 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:02 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:02.440+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:02.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:02.624+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:16:02.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:02.624+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:16:02.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:02.624+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:16:02.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:02.624+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:16:02.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:02.624+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:16:02.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:02 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:02.624+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:36.748308+0000 front 2024-02-17T12:15:36.748317+0000 (oldest deadline 2024-02-17T12:16:01.448285+0000) 2024-02-17T12:16:03.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:03.008+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:03.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:03.008+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:03.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:03.008+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:03.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:03.008+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:03.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:03.008+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:03.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:03.008+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:03.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:03 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:03.421+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:03.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:03 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:03.421+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:03.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:03 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:03.421+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:03.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:03 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:03.421+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:03.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:03 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:03.421+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:03.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:03 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:03.421+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:03.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:03.443+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.169492+0000 front 2024-02-17T12:14:56.169559+0000 (oldest deadline 2024-02-17T12:15:22.069533+0000) 2024-02-17T12:16:03.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:03.443+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:12.670995+0000 front 2024-02-17T12:15:12.671163+0000 (oldest deadline 2024-02-17T12:15:35.571139+0000) 2024-02-17T12:16:03.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:03.443+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.473451+0000 front 2024-02-17T12:15:22.473405+0000 (oldest deadline 2024-02-17T12:15:45.373269+0000) 2024-02-17T12:16:03.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:03.443+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.074579+0000 front 2024-02-17T12:15:30.074549+0000 (oldest deadline 2024-02-17T12:15:54.774368+0000) 2024-02-17T12:16:03.904 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8[55633]: 2024-02-17T12:16:03.443+0000 7fe8eeaeb700 -1 osd.8 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.774638+0000 front 2024-02-17T12:15:34.774681+0000 (oldest deadline 2024-02-17T12:16:00.674702+0000) 2024-02-17T12:16:03.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:03 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:03.575+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:16:03.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:03 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:03.575+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:16:03.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:03 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:03.575+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:16:03.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:03 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:03.575+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:16:03.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:03 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:03.575+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:16:03.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:03 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:03.575+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:36.748308+0000 front 2024-02-17T12:15:36.748317+0000 (oldest deadline 2024-02-17T12:16:01.448285+0000) 2024-02-17T12:16:04.404 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:04 smithi189 podman[75764]: 2024-02-17 12:16:04.152967685 +0000 UTC m=+5.647673964 container died 44dce335494ff05fa341878ee473e756a540f2d90e9cf1f82370aa5cbc2953fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-17T12:16:04.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:03.987+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:04.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:03.987+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:04.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:03.987+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:04.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:03.987+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:04.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:03.987+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:04.405 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:03 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:03.987+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:04.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:04 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:04.410+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:04.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:04 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:04.410+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:04.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:04 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:04.410+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:04.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:04 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:04.410+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:04.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:04 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:04.410+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:04.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:04 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:04.410+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:04.715 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:04 smithi189 podman[75764]: 2024-02-17 12:16:04.520124452 +0000 UTC m=+6.014830723 container cleanup 44dce335494ff05fa341878ee473e756a540f2d90e9cf1f82370aa5cbc2953fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-17T12:16:04.715 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:04 smithi189 bash[75764]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8 2024-02-17T12:16:04.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:04 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:04.543+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:16:04.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:04 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:04.543+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:16:04.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:04 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:04.543+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:16:04.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:04 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:04.543+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:16:04.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:04 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:04.543+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:16:04.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:04 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:04.543+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:36.748308+0000 front 2024-02-17T12:15:36.748317+0000 (oldest deadline 2024-02-17T12:16:01.448285+0000) 2024-02-17T12:16:04.982 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:04 smithi189 podman[75792]: 2024-02-17 12:16:04.713331444 +0000 UTC m=+0.563037223 container remove 44dce335494ff05fa341878ee473e756a540f2d90e9cf1f82370aa5cbc2953fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:16:05.248 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:05 smithi189 podman[75884]: 2024-02-17 12:16:05.029132355 +0000 UTC m=+0.017733426 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:16:05.248 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:05 smithi189 podman[75884]: 2024-02-17 12:16:05.130951441 +0000 UTC m=+0.119552506 container create 9d9dae6bcb4f8fc9c738c1a916627d4acda7334a9dc851da933943cb4ab4b2f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-17T12:16:05.248 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:04 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:04.979+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:05.249 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:04 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:04.979+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:05.249 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:04 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:04.979+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:05.249 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:04 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:04.979+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:05.249 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:04 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:04.979+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:05.249 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:04 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:04.979+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:05.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:05 smithi189 podman[75884]: 2024-02-17 12:16:05.366887258 +0000 UTC m=+0.355488335 container init 9d9dae6bcb4f8fc9c738c1a916627d4acda7334a9dc851da933943cb4ab4b2f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:16:05.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:05 smithi189 podman[75884]: 2024-02-17 12:16:05.37032894 +0000 UTC m=+0.358930009 container start 9d9dae6bcb4f8fc9c738c1a916627d4acda7334a9dc851da933943cb4ab4b2f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:16:05.654 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:05 smithi189 podman[75884]: 2024-02-17 12:16:05.458372843 +0000 UTC m=+0.446973921 container attach 9d9dae6bcb4f8fc9c738c1a916627d4acda7334a9dc851da933943cb4ab4b2f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-17T12:16:05.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:05 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:05.371+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:05.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:05 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:05.371+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:05.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:05 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:05.371+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:05.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:05 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:05.371+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:05.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:05 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:05.371+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:05.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:05 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:05.371+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:05.935 INFO:journalctl@ceph.osd.8.smithi189.stdout:Feb 17 12:16:05 smithi189 podman[75884]: 2024-02-17 12:16:05.656078451 +0000 UTC m=+0.644679530 container died 9d9dae6bcb4f8fc9c738c1a916627d4acda7334a9dc851da933943cb4ab4b2f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=main-6822c7b, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-17T12:16:05.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:05 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:05.533+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:16:05.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:05 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:05.533+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:16:05.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:05 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:05.533+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:16:05.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:05 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:05.533+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:16:05.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:05 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:05.533+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:16:05.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:05 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:05.533+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:36.748308+0000 front 2024-02-17T12:15:36.748317+0000 (oldest deadline 2024-02-17T12:16:01.448285+0000) 2024-02-17T12:16:06.297 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:05 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:05.932+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:06.297 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:05 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:05.932+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:06.297 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:05 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:05.932+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:06.297 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:05 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:05.932+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:06.298 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:05 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:05.932+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:06.298 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:05 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:05.932+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:06.319 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.8.service' 2024-02-17T12:16:06.401 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:16:06.401 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-02-17T12:16:06.401 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-02-17T12:16:06.401 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.9 2024-02-17T12:16:06.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:06 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:06.341+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:06.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:06 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:06.341+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:06.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:06 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:06.341+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:06.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:06 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:06.341+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:06.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:06 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:06.341+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:06.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:06 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:06.341+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:06.708 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:06 smithi094 systemd[1]: Stopping Ceph osd.9 for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:16:06.708 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:06.485+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:16:06.708 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:06.485+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:16:06.709 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:06.485+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:16:06.709 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:06.485+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:16:06.709 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:06.485+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:16:06.709 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:06.485+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:36.748308+0000 front 2024-02-17T12:15:36.748317+0000 (oldest deadline 2024-02-17T12:16:01.448285+0000) 2024-02-17T12:16:07.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:06 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:06.884+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:07.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:06 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:06.884+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:07.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:06 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:06.884+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:07.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:06 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:06.884+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:07.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:06 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:06.884+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:07.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:06 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:06.884+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:07.208 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:06.729+0000 7f4f6736a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:16:07.209 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:06.729+0000 7f4f6736a700 -1 osd.9 111 *** Got signal Terminated *** 2024-02-17T12:16:07.209 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:06 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:06.729+0000 7f4f6736a700 -1 osd.9 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T12:16:07.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:07 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:07.344+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:07.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:07 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:07.344+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:07.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:07 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:07.344+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:07.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:07 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:07.344+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:07.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:07 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:07.344+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:07.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:07 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:07.344+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:07.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:07 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:07.506+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:16:07.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:07 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:07.506+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:16:07.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:07 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:07.506+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:16:07.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:07 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:07.506+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:16:07.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:07 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:07.506+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:16:07.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:07 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:07.506+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:36.748308+0000 front 2024-02-17T12:15:36.748317+0000 (oldest deadline 2024-02-17T12:16:01.448285+0000) 2024-02-17T12:16:08.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:07 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:07.881+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:08.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:07 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:07.881+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:08.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:07 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:07.881+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:08.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:07 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:07.881+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:08.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:07 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:07.881+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:08.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:07 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:07.881+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:08.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:08 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:08.383+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:08.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:08 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:08.383+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:08.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:08 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:08.383+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:08.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:08 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:08.383+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:08.689 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:08 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:08.383+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:08.689 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:08 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:08.383+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:08.689 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:08 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:08.383+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-02-17T12:15:45.305008+0000 front 2024-02-17T12:15:45.304917+0000 (oldest deadline 2024-02-17T12:16:08.204723+0000) 2024-02-17T12:16:08.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:08 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:08.464+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:16:08.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:08 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:08.464+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:16:08.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:08 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:08.464+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:16:08.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:08 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:08.464+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:16:08.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:08 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:08.464+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:16:08.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:08 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:08.464+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:36.748308+0000 front 2024-02-17T12:15:36.748317+0000 (oldest deadline 2024-02-17T12:16:01.448285+0000) 2024-02-17T12:16:09.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:08 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:08.841+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:09.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:08 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:08.842+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:09.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:08 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:08.842+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:09.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:08 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:08.842+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:09.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:08 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:08.842+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:09.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:08 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:08.842+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:09.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:09 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:09.408+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:09.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:09 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:09.408+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:09.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:09 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:09.408+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:09.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:09 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:09.408+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:09.689 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:09 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:09.408+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:09.689 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:09 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:09.408+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:09.689 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:09 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:09.408+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-02-17T12:15:45.305008+0000 front 2024-02-17T12:15:45.304917+0000 (oldest deadline 2024-02-17T12:16:08.204723+0000) 2024-02-17T12:16:09.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:09 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:09.508+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:16:09.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:09 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:09.508+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:16:09.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:09 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:09.508+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:16:09.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:09 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:09.508+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:16:09.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:09 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:09.508+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:16:09.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:09 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:09.508+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:36.748308+0000 front 2024-02-17T12:15:36.748317+0000 (oldest deadline 2024-02-17T12:16:01.448285+0000) 2024-02-17T12:16:10.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:09 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:09.863+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:10.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:09 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:09.863+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:10.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:09 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:09.863+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:10.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:09 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:09.863+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:10.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:09 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:09.863+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:10.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:09 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:09.863+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:10.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:10 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:10.406+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:10.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:10 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:10.406+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:10.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:10 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:10.406+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:10.689 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:10 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:10.406+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:10.689 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:10 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:10.406+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:10.689 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:10 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:10.406+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:10.689 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:10 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:10.406+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-02-17T12:15:45.305008+0000 front 2024-02-17T12:15:45.304917+0000 (oldest deadline 2024-02-17T12:16:08.204723+0000) 2024-02-17T12:16:10.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:10 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:10.508+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:16:10.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:10 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:10.508+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:16:10.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:10 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:10.508+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:16:10.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:10 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:10.508+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:16:10.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:10 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:10.508+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:16:10.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:10 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:10.508+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:36.748308+0000 front 2024-02-17T12:15:36.748317+0000 (oldest deadline 2024-02-17T12:16:01.448285+0000) 2024-02-17T12:16:11.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:10 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:10.845+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:11.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:10 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:10.845+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:11.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:10 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:10.845+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:11.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:10 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:10.845+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:11.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:10 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:10.845+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:11.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:10 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:10.845+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:11.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:11 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:11.365+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:11.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:11 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:11.365+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:11.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:11 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:11.365+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:11.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:11 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:11.365+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:11.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:11 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:11.365+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:11.689 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:11 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:11.365+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:11.689 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:11 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:11.365+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-02-17T12:15:45.305008+0000 front 2024-02-17T12:15:45.304917+0000 (oldest deadline 2024-02-17T12:16:08.204723+0000) 2024-02-17T12:16:11.958 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:11 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:11.558+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.640443+0000 front 2024-02-17T12:14:56.640405+0000 (oldest deadline 2024-02-17T12:15:19.540602+0000) 2024-02-17T12:16:11.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:11 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:11.558+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.243444+0000 front 2024-02-17T12:15:05.243327+0000 (oldest deadline 2024-02-17T12:15:31.143343+0000) 2024-02-17T12:16:11.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:11 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:11.558+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:13.444741+0000 front 2024-02-17T12:15:13.444784+0000 (oldest deadline 2024-02-17T12:15:38.744766+0000) 2024-02-17T12:16:11.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:11 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:11.558+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:22.146133+0000 front 2024-02-17T12:15:22.146172+0000 (oldest deadline 2024-02-17T12:15:46.246317+0000) 2024-02-17T12:16:11.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:11 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:11.558+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:30.847895+0000 front 2024-02-17T12:15:30.848016+0000 (oldest deadline 2024-02-17T12:15:56.747794+0000) 2024-02-17T12:16:11.959 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:11 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9[68258]: 2024-02-17T12:16:11.558+0000 7f4f6395e700 -1 osd.9 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:36.748308+0000 front 2024-02-17T12:15:36.748317+0000 (oldest deadline 2024-02-17T12:16:01.448285+0000) 2024-02-17T12:16:12.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:11 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:11.801+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:12.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:11 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:11.801+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:12.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:11 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:11.801+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:12.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:11 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:11.801+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:12.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:11 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:11.801+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:12.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:11 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:11.801+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:12.444 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:12 smithi094 podman[109220]: 2024-02-17 12:16:12.187132461 +0000 UTC m=+5.589920966 container died e205b66edec5192e6b573b8ea879af59e0659e0560570db8a1339975f678f046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-17T12:16:12.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:12.353+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:12.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:12.353+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:12.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:12.353+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:12.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:12.353+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:12.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:12.353+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:12.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:12.353+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:12.689 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:12 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:12.353+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-02-17T12:15:45.305008+0000 front 2024-02-17T12:15:45.304917+0000 (oldest deadline 2024-02-17T12:16:08.204723+0000) 2024-02-17T12:16:12.708 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:12 smithi094 podman[109220]: 2024-02-17 12:16:12.525645459 +0000 UTC m=+5.928433972 container cleanup e205b66edec5192e6b573b8ea879af59e0659e0560570db8a1339975f678f046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T12:16:12.708 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:12 smithi094 bash[109220]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9 2024-02-17T12:16:13.133 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:12 smithi094 podman[109237]: 2024-02-17 12:16:12.735183115 +0000 UTC m=+0.538913200 container remove e205b66edec5192e6b573b8ea879af59e0659e0560570db8a1339975f678f046 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:16:13.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:12 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:12.791+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:13.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:12 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:12.791+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:13.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:12 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:12.791+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:13.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:12 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:12.791+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:13.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:12 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:12.791+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:13.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:12 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:12.791+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:13.457 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:13 smithi094 podman[109329]: 2024-02-17 12:16:13.035002981 +0000 UTC m=+0.018314928 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:16:13.458 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:13 smithi094 podman[109329]: 2024-02-17 12:16:13.136326492 +0000 UTC m=+0.119638478 container create 91b9485c38f72ed0621a19af8ddc9e69c93509460d37bf0869076de31f71d48c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9-deactivate, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b) 2024-02-17T12:16:13.458 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:13 smithi094 podman[109329]: 2024-02-17 12:16:13.372542586 +0000 UTC m=+0.355854531 container init 91b9485c38f72ed0621a19af8ddc9e69c93509460d37bf0869076de31f71d48c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b) 2024-02-17T12:16:13.458 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:13 smithi094 podman[109329]: 2024-02-17 12:16:13.375595954 +0000 UTC m=+0.358907896 container start 91b9485c38f72ed0621a19af8ddc9e69c93509460d37bf0869076de31f71d48c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-17T12:16:13.458 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:13 smithi094 podman[109329]: 2024-02-17 12:16:13.455638896 +0000 UTC m=+0.438950836 container attach 91b9485c38f72ed0621a19af8ddc9e69c93509460d37bf0869076de31f71d48c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-02-17T12:16:13.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:13 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:13.380+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:13.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:13 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:13.380+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:13.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:13 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:13.380+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:13.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:13 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:13.380+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:13.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:13 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:13.380+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:13.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:13 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:13.380+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:13.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:13 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:13.380+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-02-17T12:15:45.305008+0000 front 2024-02-17T12:15:45.304917+0000 (oldest deadline 2024-02-17T12:16:08.204723+0000) 2024-02-17T12:16:13.708 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:13 smithi094 podman[109329]: 2024-02-17 12:16:13.667436105 +0000 UTC m=+0.650748046 container died 91b9485c38f72ed0621a19af8ddc9e69c93509460d37bf0869076de31f71d48c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-17T12:16:14.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:13 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:13.764+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:14.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:13 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:13.764+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:14.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:13 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:13.764+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:14.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:13 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:13.764+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:14.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:13 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:13.764+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:14.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:13 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:13.764+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:14.203 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.9.service' 2024-02-17T12:16:14.204 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:14 smithi094 podman[109355]: 2024-02-17 12:16:14.184486267 +0000 UTC m=+0.512135612 container remove 91b9485c38f72ed0621a19af8ddc9e69c93509460d37bf0869076de31f71d48c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=main-6822c7b) 2024-02-17T12:16:14.204 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:14 smithi094 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.9.service: Deactivated successfully. 2024-02-17T12:16:14.204 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:14 smithi094 systemd[1]: Stopped Ceph osd.9 for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715. 2024-02-17T12:16:14.204 INFO:journalctl@ceph.osd.9.smithi094.stdout:Feb 17 12:16:14 smithi094 systemd[1]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.9.service: Consumed 7min 48.719s CPU time. 2024-02-17T12:16:14.244 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:16:14.245 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-02-17T12:16:14.245 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-02-17T12:16:14.245 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.10 2024-02-17T12:16:14.547 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:14 smithi125 systemd[1]: Stopping Ceph osd.10 for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:16:14.547 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:14.359+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:14.547 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:14.359+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:14.547 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:14.359+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:14.548 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:14.359+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:14.548 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:14.359+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:14.548 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:14.359+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:14.548 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:14.359+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-02-17T12:15:45.305008+0000 front 2024-02-17T12:15:45.304917+0000 (oldest deadline 2024-02-17T12:16:08.204723+0000) 2024-02-17T12:16:14.548 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:14.544+0000 7ff725de0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:16:14.548 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:14.544+0000 7ff725de0700 -1 osd.10 111 *** Got signal Terminated *** 2024-02-17T12:16:14.937 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:14 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:14.544+0000 7ff725de0700 -1 osd.10 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T12:16:15.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:14 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:14.785+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:15.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:14 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:14.785+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:15.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:14 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:14.785+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:15.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:14 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:14.785+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:15.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:14 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:14.785+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:15.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:14 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:14.785+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:15.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:15 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:15.313+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:15.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:15 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:15.313+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:15.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:15 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:15.313+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:15.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:15 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:15.313+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:15.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:15 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:15.313+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:15.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:15 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:15.313+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:15.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:15 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:15.313+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-02-17T12:15:45.305008+0000 front 2024-02-17T12:15:45.304917+0000 (oldest deadline 2024-02-17T12:16:08.204723+0000) 2024-02-17T12:16:16.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:15 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:15.819+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:16.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:15 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:15.819+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:16.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:15 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:15.820+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:16.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:15 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:15.820+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:16.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:15 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:15.820+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:16.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:15 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:15.820+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:16.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:16 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:16.306+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:16.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:16 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:16.306+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:16.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:16 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:16.306+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:16.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:16 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:16.306+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:16.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:16 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:16.306+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:16.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:16 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:16.306+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:16.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:16 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:16.306+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-02-17T12:15:45.305008+0000 front 2024-02-17T12:15:45.304917+0000 (oldest deadline 2024-02-17T12:16:08.204723+0000) 2024-02-17T12:16:17.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:16 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:16.817+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:17.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:16 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:16.817+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:17.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:16 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:16.817+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:17.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:16 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:16.817+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:17.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:16 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:16.817+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:17.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:16 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:16.817+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:17.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:17 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:17.272+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:17.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:17 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:17.272+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:17.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:17 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:17.272+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:17.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:17 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:17.272+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:17.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:17 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:17.272+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:17.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:17 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:17.272+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:17.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:17 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:17.272+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-02-17T12:15:45.305008+0000 front 2024-02-17T12:15:45.304917+0000 (oldest deadline 2024-02-17T12:16:08.204723+0000) 2024-02-17T12:16:18.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:17 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:17.773+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:18.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:17 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:17.773+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:18.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:17 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:17.773+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:18.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:17 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:17.773+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:18.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:17 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:17.773+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:18.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:17 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:17.773+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:18.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:18 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:18.272+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:18.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:18 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:18.272+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:18.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:18 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:18.272+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:18.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:18 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:18.273+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:18.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:18 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:18.273+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:18.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:18 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:18.273+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:18.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:18 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:18.273+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-02-17T12:15:45.305008+0000 front 2024-02-17T12:15:45.304917+0000 (oldest deadline 2024-02-17T12:16:08.204723+0000) 2024-02-17T12:16:19.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:18 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:18.788+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:19.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:18 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:18.788+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:19.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:18 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:18.788+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:19.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:18 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:18.788+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:19.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:18 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:18.788+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:19.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:18 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:18.788+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:19.589 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:19 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:19.251+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.997109+0000 front 2024-02-17T12:14:56.996799+0000 (oldest deadline 2024-02-17T12:15:19.896745+0000) 2024-02-17T12:16:19.589 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:19 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:19.251+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:01.498376+0000 front 2024-02-17T12:15:01.498358+0000 (oldest deadline 2024-02-17T12:15:27.398519+0000) 2024-02-17T12:16:19.589 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:19 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:19.251+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:11.499772+0000 front 2024-02-17T12:15:11.499672+0000 (oldest deadline 2024-02-17T12:15:36.799646+0000) 2024-02-17T12:16:19.589 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:19 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:19.251+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.902070+0000 front 2024-02-17T12:15:21.902117+0000 (oldest deadline 2024-02-17T12:15:47.801815+0000) 2024-02-17T12:16:19.589 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:19 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:19.251+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:29.502959+0000 front 2024-02-17T12:15:29.502946+0000 (oldest deadline 2024-02-17T12:15:54.803081+0000) 2024-02-17T12:16:19.589 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:19 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:19.251+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:34.803580+0000 front 2024-02-17T12:15:34.803602+0000 (oldest deadline 2024-02-17T12:16:00.103518+0000) 2024-02-17T12:16:19.590 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:19 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10[61884]: 2024-02-17T12:16:19.251+0000 7ff71e3cc700 -1 osd.10 111 heartbeat_check: no reply from 172.21.15.94:6822 osd.6 since back 2024-02-17T12:15:45.305008+0000 front 2024-02-17T12:15:45.304917+0000 (oldest deadline 2024-02-17T12:16:08.204723+0000) 2024-02-17T12:16:20.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:19 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:19.739+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:20.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:19 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:19.739+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:20.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:19 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:19.739+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:20.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:19 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:19.739+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:20.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:19 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:19.739+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:20.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:19 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:19.739+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:20.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:19 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:19.739+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6820 osd.7 since back 2024-02-17T12:15:53.372952+0000 front 2024-02-17T12:15:53.372898+0000 (oldest deadline 2024-02-17T12:16:19.272729+0000) 2024-02-17T12:16:20.421 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:20 smithi125 podman[74578]: 2024-02-17 12:16:20.093300888 +0000 UTC m=+5.657926046 container died bf1642c5b51cfefde3453b6970218dd2d34c9297bffb7c65f71a1b6ac0ea834a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-17T12:16:20.687 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:20 smithi125 podman[74578]: 2024-02-17 12:16:20.420146096 +0000 UTC m=+5.984771298 container cleanup bf1642c5b51cfefde3453b6970218dd2d34c9297bffb7c65f71a1b6ac0ea834a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-17T12:16:20.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:20 smithi125 bash[74578]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10 2024-02-17T12:16:20.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:20 smithi125 podman[74594]: 2024-02-17 12:16:20.587732421 +0000 UTC m=+0.497063973 container remove bf1642c5b51cfefde3453b6970218dd2d34c9297bffb7c65f71a1b6ac0ea834a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-17T12:16:21.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:20 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:20.715+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:21.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:20 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:20.715+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:21.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:20 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:20.715+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:21.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:20 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:20.715+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:21.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:20 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:20.715+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:21.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:20 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:20.715+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:21.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:20 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:20.715+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6820 osd.7 since back 2024-02-17T12:15:53.372952+0000 front 2024-02-17T12:15:53.372898+0000 (oldest deadline 2024-02-17T12:16:19.272729+0000) 2024-02-17T12:16:21.317 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:21 smithi125 podman[74684]: 2024-02-17 12:16:20.926838966 +0000 UTC m=+0.017688002 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:16:21.317 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:21 smithi125 podman[74684]: 2024-02-17 12:16:21.030403133 +0000 UTC m=+0.121252159 container create f4c081c7ce9c49e0eadced7f7030699b972e26877770b8969922786465fb7fc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10-deactivate, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=main-6822c7b, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-02-17T12:16:21.317 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:21 smithi125 podman[74684]: 2024-02-17 12:16:21.232572266 +0000 UTC m=+0.323421303 container init f4c081c7ce9c49e0eadced7f7030699b972e26877770b8969922786465fb7fc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10-deactivate, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-17T12:16:21.318 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:21 smithi125 podman[74684]: 2024-02-17 12:16:21.235667752 +0000 UTC m=+0.326516782 container start f4c081c7ce9c49e0eadced7f7030699b972e26877770b8969922786465fb7fc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-17T12:16:21.318 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:21 smithi125 podman[74684]: 2024-02-17 12:16:21.315808994 +0000 UTC m=+0.406658027 container attach f4c081c7ce9c49e0eadced7f7030699b972e26877770b8969922786465fb7fc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-17T12:16:21.688 INFO:journalctl@ceph.osd.10.smithi125.stdout:Feb 17 12:16:21 smithi125 podman[74684]: 2024-02-17 12:16:21.530064972 +0000 UTC m=+0.620914008 container died f4c081c7ce9c49e0eadced7f7030699b972e26877770b8969922786465fb7fc3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-17T12:16:22.129 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.10.service' 2024-02-17T12:16:22.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:21 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:21.668+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:22.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:21 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:21.668+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:22.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:21 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:21.668+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:22.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:21 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:21.668+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:22.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:21 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:21.668+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:22.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:21 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:21.668+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:22.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:21 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:21.668+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6820 osd.7 since back 2024-02-17T12:15:53.372952+0000 front 2024-02-17T12:15:53.372898+0000 (oldest deadline 2024-02-17T12:16:19.272729+0000) 2024-02-17T12:16:22.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:16:22.172 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-02-17T12:16:22.172 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-02-17T12:16:22.172 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.11 2024-02-17T12:16:22.492 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:22 smithi189 systemd[1]: Stopping Ceph osd.11 for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:16:22.904 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:22 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:22.490+0000 7fd31d459700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:16:22.904 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:22 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:22.490+0000 7fd31d459700 -1 osd.11 111 *** Got signal Terminated *** 2024-02-17T12:16:22.905 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:22 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:22.490+0000 7fd31d459700 -1 osd.11 111 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-17T12:16:22.905 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:22 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:22.631+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:22.905 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:22 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:22.631+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:22.905 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:22 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:22.631+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:22.905 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:22 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:22.631+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:22.905 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:22 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:22.631+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:22.905 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:22 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:22.631+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:22.905 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:22 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:22.631+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6820 osd.7 since back 2024-02-17T12:15:53.372952+0000 front 2024-02-17T12:15:53.372898+0000 (oldest deadline 2024-02-17T12:16:19.272729+0000) 2024-02-17T12:16:23.904 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:23 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:23.626+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:23.904 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:23 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:23.626+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:23.904 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:23 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:23.626+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:23.905 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:23 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:23.626+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:23.905 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:23 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:23.626+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:23.905 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:23 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:23.626+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:23.905 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:23 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:23.626+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6820 osd.7 since back 2024-02-17T12:15:53.372952+0000 front 2024-02-17T12:15:53.372898+0000 (oldest deadline 2024-02-17T12:16:19.272729+0000) 2024-02-17T12:16:25.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:24 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:24.663+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:25.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:24 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:24.663+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:25.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:24 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:24.663+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:25.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:24 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:24.663+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:25.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:24 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:24.663+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:25.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:24 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:24.663+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:25.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:24 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:24.663+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6820 osd.7 since back 2024-02-17T12:15:53.372952+0000 front 2024-02-17T12:15:53.372898+0000 (oldest deadline 2024-02-17T12:16:19.272729+0000) 2024-02-17T12:16:26.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:25 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:25.708+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:26.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:25 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:25.708+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:26.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:25 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:25.708+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:26.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:25 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:25.708+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:26.154 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:25 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:25.708+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:26.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:25 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:25.708+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:26.155 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:25 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:25.708+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6820 osd.7 since back 2024-02-17T12:15:53.372952+0000 front 2024-02-17T12:15:53.372898+0000 (oldest deadline 2024-02-17T12:16:19.272729+0000) 2024-02-17T12:16:27.056 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:26 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:26.716+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6806 osd.0 since back 2024-02-17T12:14:56.165110+0000 front 2024-02-17T12:14:56.165015+0000 (oldest deadline 2024-02-17T12:15:20.865063+0000) 2024-02-17T12:16:27.056 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:26 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:26.716+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6804 osd.1 since back 2024-02-17T12:15:05.466592+0000 front 2024-02-17T12:15:05.466864+0000 (oldest deadline 2024-02-17T12:15:30.766600+0000) 2024-02-17T12:16:27.056 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:26 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:26.716+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6804 osd.2 since back 2024-02-17T12:15:14.267518+0000 front 2024-02-17T12:15:14.267610+0000 (oldest deadline 2024-02-17T12:15:35.268006+0000) 2024-02-17T12:16:27.056 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:26 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:26.716+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.94:6814 osd.3 since back 2024-02-17T12:15:21.069601+0000 front 2024-02-17T12:15:21.069688+0000 (oldest deadline 2024-02-17T12:15:45.769601+0000) 2024-02-17T12:16:27.056 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:26 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:26.716+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6812 osd.4 since back 2024-02-17T12:15:25.770288+0000 front 2024-02-17T12:15:25.770368+0000 (oldest deadline 2024-02-17T12:15:51.670147+0000) 2024-02-17T12:16:27.056 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:26 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:26.716+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.189:6812 osd.5 since back 2024-02-17T12:15:35.770856+0000 front 2024-02-17T12:15:35.770892+0000 (oldest deadline 2024-02-17T12:16:01.070896+0000) 2024-02-17T12:16:27.056 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:26 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11[62253]: 2024-02-17T12:16:26.716+0000 7fd315a45700 -1 osd.11 111 heartbeat_check: no reply from 172.21.15.125:6820 osd.7 since back 2024-02-17T12:15:53.372952+0000 front 2024-02-17T12:15:53.372898+0000 (oldest deadline 2024-02-17T12:16:19.272729+0000) 2024-02-17T12:16:28.404 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:28 smithi189 podman[75992]: 2024-02-17 12:16:28.124487596 +0000 UTC m=+5.754962753 container died 14c9be00b6c3ea6162e6245713f6109226e7f3cef4a3273eacd9306551ebf670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-17T12:16:28.759 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:28 smithi189 podman[75992]: 2024-02-17 12:16:28.49038025 +0000 UTC m=+6.120855404 container cleanup 14c9be00b6c3ea6162e6245713f6109226e7f3cef4a3273eacd9306551ebf670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-02-17T12:16:28.760 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:28 smithi189 bash[75992]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11 2024-02-17T12:16:29.052 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:28 smithi189 podman[76019]: 2024-02-17 12:16:28.75794403 +0000 UTC m=+0.626629388 container remove 14c9be00b6c3ea6162e6245713f6109226e7f3cef4a3273eacd9306551ebf670 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11, RELEASE=main-6822c7b, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-17T12:16:29.354 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:29 smithi189 podman[76107]: 2024-02-17 12:16:29.050317717 +0000 UTC m=+0.111752378 container create ed826807328ed423f3cb8cf41f297c39574b40384d17e83d2f0c49250397df5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-17T12:16:29.354 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:29 smithi189 podman[76107]: 2024-02-17 12:16:28.957072591 +0000 UTC m=+0.018507259 image pull 32fa8af9a90292f8d3f58c0165592a03314a1d4bd036f3a7a316b32879cd3acd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d 2024-02-17T12:16:29.355 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:29 smithi189 podman[76107]: 2024-02-17 12:16:29.27744255 +0000 UTC m=+0.338877209 container init ed826807328ed423f3cb8cf41f297c39574b40384d17e83d2f0c49250397df5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11-deactivate, RELEASE=main-6822c7b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-17T12:16:29.355 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:29 smithi189 podman[76107]: 2024-02-17 12:16:29.280530585 +0000 UTC m=+0.341965240 container start ed826807328ed423f3cb8cf41f297c39574b40384d17e83d2f0c49250397df5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-17T12:16:29.655 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:29 smithi189 podman[76107]: 2024-02-17 12:16:29.352362994 +0000 UTC m=+0.413797641 container attach ed826807328ed423f3cb8cf41f297c39574b40384d17e83d2f0c49250397df5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-17T12:16:29.655 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:29 smithi189 podman[76107]: 2024-02-17 12:16:29.582278648 +0000 UTC m=+0.643713303 container died ed826807328ed423f3cb8cf41f297c39574b40384d17e83d2f0c49250397df5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240214, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-17T12:16:30.133 INFO:journalctl@ceph.osd.11.smithi189.stdout:Feb 17 12:16:30 smithi189 podman[76133]: 2024-02-17 12:16:30.113767356 +0000 UTC m=+0.522465306 container remove ed826807328ed423f3cb8cf41f297c39574b40384d17e83d2f0c49250397df5a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-osd-11-deactivate, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-17T12:16:30.138 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@osd.11.service' 2024-02-17T12:16:30.179 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:16:30.179 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-02-17T12:16:30.179 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-02-17T12:16:30.179 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.a 2024-02-17T12:16:30.498 INFO:journalctl@ceph.mds.a.smithi094.stdout:Feb 17 12:16:30 smithi094 systemd[1]: Stopping Ceph mds.a for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:16:30.958 INFO:journalctl@ceph.mds.a.smithi094.stdout:Feb 17 12:16:30 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-a[74646]: 2024-02-17T12:16:30.495+0000 7f24d0911700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:16:30.959 INFO:journalctl@ceph.mds.a.smithi094.stdout:Feb 17 12:16:30 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-a[74646]: 2024-02-17T12:16:30.495+0000 7f24d0911700 -1 mds.a *** got signal Terminated *** 2024-02-17T12:16:33.940 INFO:journalctl@ceph.mds.a.smithi094.stdout:Feb 17 12:16:33 smithi094 podman[109451]: 2024-02-17 12:16:33.679586069 +0000 UTC m=+3.295811740 container died 8d200e9967fa2f7ddc37cafda2522614bad9bb46c8e627aec5aeee08fb7f8af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T12:16:34.209 INFO:journalctl@ceph.mds.a.smithi094.stdout:Feb 17 12:16:34 smithi094 podman[109451]: 2024-02-17 12:16:34.029179407 +0000 UTC m=+3.645405076 container cleanup 8d200e9967fa2f7ddc37cafda2522614bad9bb46c8e627aec5aeee08fb7f8af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-a, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=main-6822c7b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-17T12:16:34.209 INFO:journalctl@ceph.mds.a.smithi094.stdout:Feb 17 12:16:34 smithi094 bash[109451]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-a 2024-02-17T12:16:34.209 INFO:journalctl@ceph.mds.a.smithi094.stdout:Feb 17 12:16:34 smithi094 podman[109464]: 2024-02-17 12:16:34.18867002 +0000 UTC m=+0.504178725 container remove 8d200e9967fa2f7ddc37cafda2522614bad9bb46c8e627aec5aeee08fb7f8af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=main-6822c7b, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True) 2024-02-17T12:16:34.396 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.a.service' 2024-02-17T12:16:34.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:16:34.434 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-02-17T12:16:34.434 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-02-17T12:16:34.434 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.d 2024-02-17T12:16:34.709 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 12:16:34 smithi094 systemd[1]: Stopping Ceph mds.d for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:16:35.208 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 12:16:34 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-d[73689]: 2024-02-17T12:16:34.749+0000 7f572381f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:16:35.208 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 12:16:34 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-d[73689]: 2024-02-17T12:16:34.749+0000 7f572381f700 -1 mds.d *** got signal Terminated *** 2024-02-17T12:16:38.208 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 12:16:37 smithi094 podman[109571]: 2024-02-17 12:16:37.750486731 +0000 UTC m=+3.097403578 container died 005a4583f339358fa83ac416104cbbc5b0368f6845000180e7a572c80186da23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-d, RELEASE=main-6822c7b, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-17T12:16:38.588 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 12:16:38 smithi094 podman[109571]: 2024-02-17 12:16:38.308261362 +0000 UTC m=+3.655178208 container cleanup 005a4583f339358fa83ac416104cbbc5b0368f6845000180e7a572c80186da23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-d, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=main-6822c7b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-17T12:16:38.588 INFO:journalctl@ceph.mds.d.smithi094.stdout:Feb 17 12:16:38 smithi094 bash[109571]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-d 2024-02-17T12:16:38.727 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.d.service' 2024-02-17T12:16:38.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:16:38.763 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-02-17T12:16:38.763 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-02-17T12:16:38.763 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.g 2024-02-17T12:16:39.208 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 12:16:38 smithi094 systemd[1]: Stopping Ceph mds.g for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:16:39.208 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 12:16:39 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-g[74285]: 2024-02-17T12:16:39.071+0000 7fbb59b8d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:16:39.209 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 12:16:39 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-g[74285]: 2024-02-17T12:16:39.071+0000 7fbb59b8d700 -1 mds.g *** got signal Terminated *** 2024-02-17T12:16:43.158 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 12:16:42 smithi094 podman[109691]: 2024-02-17 12:16:42.862256895 +0000 UTC m=+3.882546118 container died 2f5bb2bd9bd504f631c3f86312bfb50fab185a93591cb5a19ea3566a4e6662d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-g, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=main-6822c7b) 2024-02-17T12:16:43.458 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 12:16:43 smithi094 podman[109691]: 2024-02-17 12:16:43.222154808 +0000 UTC m=+4.242444028 container cleanup 2f5bb2bd9bd504f631c3f86312bfb50fab185a93591cb5a19ea3566a4e6662d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-g, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-17T12:16:43.458 INFO:journalctl@ceph.mds.g.smithi094.stdout:Feb 17 12:16:43 smithi094 bash[109691]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-g 2024-02-17T12:16:43.623 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.g.service' 2024-02-17T12:16:43.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:16:43.657 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-02-17T12:16:43.658 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-02-17T12:16:43.658 DEBUG:teuthology.orchestra.run.smithi094:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.j 2024-02-17T12:16:44.208 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 12:16:43 smithi094 systemd[1]: Stopping Ceph mds.j for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:16:44.209 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 12:16:43 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-j[73338]: 2024-02-17T12:16:43.983+0000 7f9bbcfa3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:16:44.209 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 12:16:43 smithi094 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-j[73338]: 2024-02-17T12:16:43.983+0000 7f9bbcfa3700 -1 mds.j *** got signal Terminated *** 2024-02-17T12:16:47.111 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 12:16:46 smithi094 podman[109809]: 2024-02-17 12:16:46.782436745 +0000 UTC m=+2.901722978 container died 8085e8836ad5a9b7c8dfe8ae8808bba3dd7f074aa9623826d8375d44143683d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=main-6822c7b, io.buildah.version=1.33.5) 2024-02-17T12:16:47.428 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 12:16:47 smithi094 podman[109809]: 2024-02-17 12:16:47.200499508 +0000 UTC m=+3.319785740 container cleanup 8085e8836ad5a9b7c8dfe8ae8808bba3dd7f074aa9623826d8375d44143683d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=main-6822c7b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-02-17T12:16:47.428 INFO:journalctl@ceph.mds.j.smithi094.stdout:Feb 17 12:16:47 smithi094 bash[109809]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-j 2024-02-17T12:16:47.627 DEBUG:teuthology.orchestra.run.smithi094:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.j.service' 2024-02-17T12:16:47.661 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:16:47.662 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-02-17T12:16:47.662 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-02-17T12:16:47.662 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.b 2024-02-17T12:16:47.971 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 12:16:47 smithi125 systemd[1]: Stopping Ceph mds.b for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:16:48.437 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 12:16:47 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b[67430]: 2024-02-17T12:16:47.968+0000 7fe3b02c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:16:48.437 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 12:16:47 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b[67430]: 2024-02-17T12:16:47.968+0000 7fe3b02c5700 -1 mds.b *** got signal Terminated *** 2024-02-17T12:16:51.471 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 12:16:51 smithi125 podman[74792]: 2024-02-17 12:16:51.207107196 +0000 UTC m=+3.335308739 container died f403f8ef7044d1cdfe5cbdb3a47628955951dfa9517f2244f736886793f07fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, ceph=True) 2024-02-17T12:16:51.929 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 12:16:51 smithi125 podman[74792]: 2024-02-17 12:16:51.568395949 +0000 UTC m=+3.696597490 container cleanup f403f8ef7044d1cdfe5cbdb3a47628955951dfa9517f2244f736886793f07fa8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5) 2024-02-17T12:16:51.929 INFO:journalctl@ceph.mds.b.smithi125.stdout:Feb 17 12:16:51 smithi125 bash[74792]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-b 2024-02-17T12:16:52.144 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.b.service' 2024-02-17T12:16:52.181 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:16:52.181 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-02-17T12:16:52.181 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-02-17T12:16:52.182 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.e 2024-02-17T12:16:52.437 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 12:16:52 smithi125 systemd[1]: Stopping Ceph mds.e for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:16:52.937 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 12:16:52 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-e[66801]: 2024-02-17T12:16:52.505+0000 7ff447363700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:16:52.938 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 12:16:52 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-e[66801]: 2024-02-17T12:16:52.505+0000 7ff447363700 -1 mds.e *** got signal Terminated *** 2024-02-17T12:16:55.490 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 12:16:55 smithi125 podman[74915]: 2024-02-17 12:16:55.21027962 +0000 UTC m=+2.814665939 container died 0eaf5a395ba020c55f4c2b8aaffc650450f2d9804ac3c5efefe3562f09e12847 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=main-6822c7b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-17T12:16:55.741 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 12:16:55 smithi125 podman[74915]: 2024-02-17 12:16:55.489075131 +0000 UTC m=+3.093461429 container cleanup 0eaf5a395ba020c55f4c2b8aaffc650450f2d9804ac3c5efefe3562f09e12847 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-e, org.label-schema.build-date=20240214, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-6822c7b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-02-17T12:16:55.742 INFO:journalctl@ceph.mds.e.smithi125.stdout:Feb 17 12:16:55 smithi125 bash[74915]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-e 2024-02-17T12:16:55.904 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.e.service' 2024-02-17T12:16:55.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:16:55.941 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-02-17T12:16:55.941 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-02-17T12:16:55.941 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.h 2024-02-17T12:16:56.262 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 12:16:56 smithi125 systemd[1]: Stopping Ceph mds.h for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:16:56.687 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 12:16:56 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-h[67107]: 2024-02-17T12:16:56.259+0000 7f94ca385700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:16:56.688 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 12:16:56 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-h[67107]: 2024-02-17T12:16:56.259+0000 7f94ca385700 -1 mds.h *** got signal Terminated *** 2024-02-17T12:16:59.499 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 12:16:59 smithi125 podman[75051]: 2024-02-17 12:16:59.211509993 +0000 UTC m=+3.055828148 container died 0cfe22aa45829914a188a30cadff04a5f0ec492a515ae9ad008d94ade7339afd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=main-6822c7b, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, ceph=True) 2024-02-17T12:16:59.845 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 12:16:59 smithi125 podman[75051]: 2024-02-17 12:16:59.584558443 +0000 UTC m=+3.428876591 container cleanup 0cfe22aa45829914a188a30cadff04a5f0ec492a515ae9ad008d94ade7339afd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-h, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-02-17T12:16:59.846 INFO:journalctl@ceph.mds.h.smithi125.stdout:Feb 17 12:16:59 smithi125 bash[75051]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-h 2024-02-17T12:17:00.006 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.h.service' 2024-02-17T12:17:00.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:17:00.043 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-02-17T12:17:00.043 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-02-17T12:17:00.043 DEBUG:teuthology.orchestra.run.smithi125:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.k 2024-02-17T12:17:00.365 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 12:17:00 smithi125 systemd[1]: Stopping Ceph mds.k for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:17:00.687 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 12:17:00 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-k[66481]: 2024-02-17T12:17:00.363+0000 7fa8be4a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:17:00.688 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 12:17:00 smithi125 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-k[66481]: 2024-02-17T12:17:00.363+0000 7fa8be4a8700 -1 mds.k *** got signal Terminated *** 2024-02-17T12:17:04.196 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 12:17:03 smithi125 podman[75172]: 2024-02-17 12:17:03.960345149 +0000 UTC m=+3.707246874 container died db4b881c9c9bebca26bb5e27ab82eeac4e6e8c1c7bedf3098a231527b46adab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-k, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, io.buildah.version=1.33.5) 2024-02-17T12:17:04.507 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 12:17:04 smithi125 podman[75172]: 2024-02-17 12:17:04.263261503 +0000 UTC m=+4.010163220 container cleanup db4b881c9c9bebca26bb5e27ab82eeac4e6e8c1c7bedf3098a231527b46adab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-k, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS) 2024-02-17T12:17:04.507 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 12:17:04 smithi125 bash[75172]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-k 2024-02-17T12:17:04.507 INFO:journalctl@ceph.mds.k.smithi125.stdout:Feb 17 12:17:04 smithi125 podman[75186]: 2024-02-17 12:17:04.430847528 +0000 UTC m=+0.463138147 container remove db4b881c9c9bebca26bb5e27ab82eeac4e6e8c1c7bedf3098a231527b46adab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-6822c7b) 2024-02-17T12:17:04.564 DEBUG:teuthology.orchestra.run.smithi125:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.k.service' 2024-02-17T12:17:04.599 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:17:04.599 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-02-17T12:17:04.600 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-02-17T12:17:04.600 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.c 2024-02-17T12:17:04.890 INFO:journalctl@ceph.mds.c.smithi189.stdout:Feb 17 12:17:04 smithi189 systemd[1]: Stopping Ceph mds.c for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:17:05.154 INFO:journalctl@ceph.mds.c.smithi189.stdout:Feb 17 12:17:04 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-c[67686]: 2024-02-17T12:17:04.887+0000 7f91f768a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:17:05.154 INFO:journalctl@ceph.mds.c.smithi189.stdout:Feb 17 12:17:04 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-c[67686]: 2024-02-17T12:17:04.887+0000 7f91f768a700 -1 mds.c *** got signal Terminated *** 2024-02-17T12:17:07.591 INFO:journalctl@ceph.mds.c.smithi189.stdout:Feb 17 12:17:07 smithi189 podman[76213]: 2024-02-17 12:17:07.288045581 +0000 UTC m=+2.488442632 container died 1241d9f934b6a3a1615865c99c28e00ec7fc3a0c0865219d6c4e90b0cc6399fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-c, GIT_CLEAN=True, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-17T12:17:07.905 INFO:journalctl@ceph.mds.c.smithi189.stdout:Feb 17 12:17:07 smithi189 podman[76213]: 2024-02-17 12:17:07.662352874 +0000 UTC m=+2.862749931 container cleanup 1241d9f934b6a3a1615865c99c28e00ec7fc3a0c0865219d6c4e90b0cc6399fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-c, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-17T12:17:07.905 INFO:journalctl@ceph.mds.c.smithi189.stdout:Feb 17 12:17:07 smithi189 bash[76213]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-c 2024-02-17T12:17:08.041 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.c.service' 2024-02-17T12:17:08.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:17:08.080 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-02-17T12:17:08.080 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-02-17T12:17:08.080 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.f 2024-02-17T12:17:08.385 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:17:08 smithi189 systemd[1]: Stopping Ceph mds.f for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:17:08.654 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:17:08 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[71429]: 2024-02-17T12:17:08.382+0000 7f121f3da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:17:08.654 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:17:08 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f[71429]: 2024-02-17T12:17:08.382+0000 7f121f3da700 -1 mds.f *** got signal Terminated *** 2024-02-17T12:17:11.561 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:17:11 smithi189 podman[76343]: 2024-02-17 12:17:11.283383248 +0000 UTC m=+2.990994286 container died 7a5d228de56b83698faddf5d61a8a4da8df53ad4d628d7250ca23898b8177af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f, RELEASE=main-6822c7b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-17T12:17:11.835 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:17:11 smithi189 podman[76343]: 2024-02-17 12:17:11.632769745 +0000 UTC m=+3.340380787 container cleanup 7a5d228de56b83698faddf5d61a8a4da8df53ad4d628d7250ca23898b8177af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, RELEASE=main-6822c7b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:17:11.836 INFO:journalctl@ceph.mds.f.smithi189.stdout:Feb 17 12:17:11 smithi189 bash[76343]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-f 2024-02-17T12:17:12.056 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.f.service' 2024-02-17T12:17:12.091 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:17:12.091 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-02-17T12:17:12.091 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-02-17T12:17:12.092 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.i 2024-02-17T12:17:12.404 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 12:17:12 smithi189 systemd[1]: Stopping Ceph mds.i for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:17:12.904 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 12:17:12 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-i[68005]: 2024-02-17T12:17:12.402+0000 7f61317bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:17:12.904 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 12:17:12 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-i[68005]: 2024-02-17T12:17:12.402+0000 7f61317bb700 -1 mds.i *** got signal Terminated *** 2024-02-17T12:17:15.477 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 12:17:15 smithi189 podman[76467]: 2024-02-17 12:17:15.21828913 +0000 UTC m=+2.916399517 container died d80d879d3207bcdf38e0018843a9278c62ae8705394b0bbd032b78ed53b1f9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=main-6822c7b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-02-17T12:17:15.904 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 12:17:15 smithi189 podman[76467]: 2024-02-17 12:17:15.677771707 +0000 UTC m=+3.375882085 container cleanup d80d879d3207bcdf38e0018843a9278c62ae8705394b0bbd032b78ed53b1f9ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-i, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=main-6822c7b, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-17T12:17:15.904 INFO:journalctl@ceph.mds.i.smithi189.stdout:Feb 17 12:17:15 smithi189 bash[76467]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-i 2024-02-17T12:17:16.050 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.i.service' 2024-02-17T12:17:16.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:17:16.085 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-02-17T12:17:16.085 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-02-17T12:17:16.085 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.l 2024-02-17T12:17:16.404 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:17:16 smithi189 systemd[1]: Stopping Ceph mds.l for 3bc5cfc8-cd8a-11ee-95ba-87774f69a715... 2024-02-17T12:17:16.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:17:16 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[72363]: 2024-02-17T12:17:16.405+0000 7f6a3b1f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-17T12:17:16.904 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:17:16 smithi189 ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l[72363]: 2024-02-17T12:17:16.405+0000 7f6a3b1f9700 -1 mds.l *** got signal Terminated *** 2024-02-17T12:17:19.592 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:17:19 smithi189 podman[76589]: 2024-02-17 12:17:19.223784597 +0000 UTC m=+2.925750261 container died 5dc8e5360759ff9943e845b7629f27b84a5611da9dc02ea1e33e195b0fb5780c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, GIT_CLEAN=True, RELEASE=main-6822c7b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214) 2024-02-17T12:17:19.877 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:17:19 smithi189 podman[76589]: 2024-02-17 12:17:19.689747077 +0000 UTC m=+3.391712710 container cleanup 5dc8e5360759ff9943e845b7629f27b84a5611da9dc02ea1e33e195b0fb5780c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:0ea7aacdb4e186a4241327d9a31fe2d56ad03bdb5c914caaf5a828d6cec3b51d, name=ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l, RELEASE=main-6822c7b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-02-17T12:17:19.877 INFO:journalctl@ceph.mds.l.smithi189.stdout:Feb 17 12:17:19 smithi189 bash[76589]: ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715-mds-l 2024-02-17T12:17:20.072 DEBUG:teuthology.orchestra.run.smithi189:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3bc5cfc8-cd8a-11ee-95ba-87774f69a715@mds.l.service' 2024-02-17T12:17:20.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-17T12:17:20.106 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-02-17T12:17:20.106 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-17T12:17:20.296 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T12:17:23.139 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-17T12:17:23.336 INFO:teuthology.orchestra.run.smithi125.stdout:Deleting cluster with fsid: 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T12:17:25.853 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-17T12:17:26.044 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T12:17:28.726 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-17T12:17:28.761 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-17T12:17:28.796 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-17T12:17:28.825 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-17T12:17:28.827 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122/remote/smithi094/crash 2024-02-17T12:17:28.828 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/crash -- . 2024-02-17T12:17:28.867 INFO:teuthology.orchestra.run.smithi094.stderr:tar: /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-17T12:17:28.868 INFO:teuthology.orchestra.run.smithi094.stderr:tar: Error is not recoverable: exiting now 2024-02-17T12:17:28.869 DEBUG:teuthology.misc:Transferring archived files from smithi125:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122/remote/smithi125/crash 2024-02-17T12:17:28.870 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/crash -- . 2024-02-17T12:17:28.914 INFO:teuthology.orchestra.run.smithi125.stderr:tar: /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-17T12:17:28.915 INFO:teuthology.orchestra.run.smithi125.stderr:tar: Error is not recoverable: exiting now 2024-02-17T12:17:28.916 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122/remote/smithi189/crash 2024-02-17T12:17:28.917 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/crash -- . 2024-02-17T12:17:28.964 INFO:teuthology.orchestra.run.smithi189.stderr:tar: /var/lib/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-17T12:17:28.964 INFO:teuthology.orchestra.run.smithi189.stderr:tar: Error is not recoverable: exiting now 2024-02-17T12:17:28.965 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-17T12:17:28.965 DEBUG:teuthology.orchestra.run.smithi094:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-02-17T12:17:29.033 INFO:teuthology.orchestra.run.smithi094.stdout:2024-02-17T11:49:56.825946+0000 mon.a (mon.0) 189 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-17T12:17:29.035 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-17T12:17:29.035 INFO:tasks.cephadm:Compressing logs... 2024-02-17T12:17:29.035 DEBUG:teuthology.orchestra.run.smithi094:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-17T12:17:29.037 DEBUG:teuthology.orchestra.run.smithi125:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-17T12:17:29.040 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-17T12:17:29.072 INFO:teuthology.orchestra.run.smithi094.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-17T12:17:29.076 INFO:teuthology.orchestra.run.smithi125.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-17T12:17:29.077 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-17T12:17:29.077 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.10.log 2024-02-17T12:17:29.078 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.audit.log/var/log/ceph/cephadm.log: 2024-02-17T12:17:29.078 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.b.log 2024-02-17T12:17:29.078 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.4.log 2024-02-17T12:17:29.078 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.1.log 2024-02-17T12:17:29.080 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.k.log 2024-02-17T12:17:29.081 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.h.log 2024-02-17T12:17:29.081 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.7.log 2024-02-17T12:17:29.081 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.e.log 2024-02-17T12:17:29.081 INFO:teuthology.orchestra.run.smithi125.stderr:gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mgr.y.log 2024-02-17T12:17:29.081 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.7.log: /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-17T12:17:29.081 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.log 2024-02-17T12:17:29.081 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-17T12:17:29.081 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mon.b.log 2024-02-17T12:17:29.081 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-17T12:17:29.083 INFO:teuthology.orchestra.run.smithi189.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-17T12:17:29.083 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-17T12:17:29.083 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.l.log 2024-02-17T12:17:29.083 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-17T12:17:29.083 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.i.log 2024-02-17T12:17:29.084 INFO:teuthology.orchestra.run.smithi125.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-volume.log: /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mon.b.log: 94.4% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.k.log.gz 2024-02-17T12:17:29.089 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.11.log 2024-02-17T12:17:29.089 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.audit.log: /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mgr.z.log 2024-02-17T12:17:29.089 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.8.log 2024-02-17T12:17:29.089 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.5.log 2024-02-17T12:17:29.089 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mgr.z.log: /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.f.log 2024-02-17T12:17:29.089 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-17T12:17:29.090 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.log 2024-02-17T12:17:29.090 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.2.log 2024-02-17T12:17:29.090 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.c.log 2024-02-17T12:17:29.090 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-17T12:17:29.090 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mon.c.log 2024-02-17T12:17:29.090 INFO:teuthology.orchestra.run.smithi125.stderr: 92.2% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mgr.y.log.gz 2024-02-17T12:17:29.091 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.6.log 2024-02-17T12:17:29.091 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mon.a.log 2024-02-17T12:17:29.091 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-17T12:17:29.091 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.g.log 2024-02-17T12:17:29.091 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.j.log 2024-02-17T12:17:29.091 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.3.log 2024-02-17T12:17:29.092 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mgr.x.log 2024-02-17T12:17:29.092 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.0.log 2024-02-17T12:17:29.092 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-17T12:17:29.092 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mgr.x.log: /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.log 2024-02-17T12:17:29.092 INFO:teuthology.orchestra.run.smithi094.stderr:gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-17T12:17:29.092 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.cephadm.log: /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.9.log 2024-02-17T12:17:29.092 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.d.log 2024-02-17T12:17:29.092 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.a.log 2024-02-17T12:17:29.092 INFO:teuthology.orchestra.run.smithi094.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.d.log: /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.a.log: 86.4% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-17T12:17:29.092 INFO:teuthology.orchestra.run.smithi094.stderr: 94.6% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.j.log.gz 2024-02-17T12:17:29.092 INFO:teuthology.orchestra.run.smithi094.stderr: 94.7% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.g.log.gz 2024-02-17T12:17:29.093 INFO:teuthology.orchestra.run.smithi094.stderr: 94.6% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.a.log.gz 2024-02-17T12:17:29.093 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-volume.log: /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mon.c.log: 92.2% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mgr.z.log.gz 2024-02-17T12:17:29.126 INFO:teuthology.orchestra.run.smithi125.stderr: 83.8% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-17T12:17:29.127 INFO:teuthology.orchestra.run.smithi189.stderr: 83.7% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-17T12:17:29.128 INFO:teuthology.orchestra.run.smithi125.stderr: 90.2% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-17T12:17:29.128 INFO:teuthology.orchestra.run.smithi125.stderr: 86.4% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-17T12:17:29.133 INFO:teuthology.orchestra.run.smithi094.stderr: 90.1% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-17T12:17:29.142 INFO:teuthology.orchestra.run.smithi125.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-17T12:17:29.150 INFO:teuthology.orchestra.run.smithi125.stderr: 94.7% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.h.log.gz 2024-02-17T12:17:29.166 INFO:teuthology.orchestra.run.smithi189.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-17T12:17:29.166 INFO:teuthology.orchestra.run.smithi094.stderr: 93.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-17T12:17:29.168 INFO:teuthology.orchestra.run.smithi189.stderr: 90.2% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-17T12:17:29.172 INFO:teuthology.orchestra.run.smithi094.stderr: 84.1% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-17T12:17:29.191 INFO:teuthology.orchestra.run.smithi125.stderr: 94.6% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.e.log.gz 2024-02-17T12:17:29.199 INFO:teuthology.orchestra.run.smithi189.stderr: 86.4% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-17T12:17:29.220 INFO:teuthology.orchestra.run.smithi125.stderr: 91.0% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-17T12:17:29.252 INFO:teuthology.orchestra.run.smithi094.stderr: 91.1% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-17T12:17:29.316 INFO:teuthology.orchestra.run.smithi189.stderr: 93.1% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.c.log.gz 2024-02-17T12:17:29.355 INFO:teuthology.orchestra.run.smithi189.stderr: 93.3% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.i.log.gz 2024-02-17T12:17:29.503 INFO:teuthology.orchestra.run.smithi094.stderr: 94.6% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.d.log.gz 2024-02-17T12:17:29.524 INFO:teuthology.orchestra.run.smithi189.stderr: 91.0% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-17T12:17:29.554 INFO:teuthology.orchestra.run.smithi094.stderr: 90.0% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mgr.x.log.gz 2024-02-17T12:17:29.624 INFO:teuthology.orchestra.run.smithi125.stderr: 91.9% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mon.b.log.gz 2024-02-17T12:17:29.789 INFO:teuthology.orchestra.run.smithi189.stderr: 94.6% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.f.log.gz 2024-02-17T12:17:30.583 INFO:teuthology.orchestra.run.smithi094.stderr: 90.3% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mon.a.log.gz 2024-02-17T12:17:30.763 INFO:teuthology.orchestra.run.smithi189.stderr: 91.8% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mon.c.log.gz 2024-02-17T12:17:37.030 INFO:teuthology.orchestra.run.smithi125.stderr: 93.9% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.b.log.gz 2024-02-17T12:17:41.420 INFO:teuthology.orchestra.run.smithi189.stderr: 93.6% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-mds.l.log.gz 2024-02-17T12:18:20.862 INFO:teuthology.orchestra.run.smithi125.stderr: 91.7% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.4.log.gz 2024-02-17T12:18:23.947 INFO:teuthology.orchestra.run.smithi125.stderr: 91.1% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.10.log.gz 2024-02-17T12:18:26.267 INFO:teuthology.orchestra.run.smithi094.stderr: 91.2% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.9.log.gz 2024-02-17T12:18:27.088 INFO:teuthology.orchestra.run.smithi125.stderr: 91.1% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.1.log.gz 2024-02-17T12:18:32.240 INFO:teuthology.orchestra.run.smithi125.stderr: 91.0% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.7.log.gz 2024-02-17T12:18:32.242 INFO:teuthology.orchestra.run.smithi125.stderr: 2024-02-17T12:18:32.243 INFO:teuthology.orchestra.run.smithi125.stderr:real 1m3.188s 2024-02-17T12:18:32.243 INFO:teuthology.orchestra.run.smithi125.stderr:user 3m36.444s 2024-02-17T12:18:32.243 INFO:teuthology.orchestra.run.smithi125.stderr:sys 0m6.637s 2024-02-17T12:18:44.216 INFO:teuthology.orchestra.run.smithi094.stderr: 91.0% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.6.log.gz 2024-02-17T12:18:47.100 INFO:teuthology.orchestra.run.smithi094.stderr: 91.0% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.0.log.gz 2024-02-17T12:18:50.732 INFO:teuthology.orchestra.run.smithi094.stderr: 91.0% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.3.log.gz 2024-02-17T12:18:50.734 INFO:teuthology.orchestra.run.smithi094.stderr: 2024-02-17T12:18:50.734 INFO:teuthology.orchestra.run.smithi094.stderr:real 1m21.683s 2024-02-17T12:18:50.734 INFO:teuthology.orchestra.run.smithi094.stderr:user 3m45.151s 2024-02-17T12:18:50.734 INFO:teuthology.orchestra.run.smithi094.stderr:sys 0m7.077s 2024-02-17T12:19:26.977 INFO:teuthology.orchestra.run.smithi189.stderr: 91.1% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.8.log.gz 2024-02-17T12:19:53.264 INFO:teuthology.orchestra.run.smithi189.stderr: 91.2% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.2.log.gz 2024-02-17T12:20:01.185 INFO:teuthology.orchestra.run.smithi189.stderr: 91.2% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.5.log.gz 2024-02-17T12:20:13.332 INFO:teuthology.orchestra.run.smithi189.stderr: 91.0% -- replaced with /var/log/ceph/3bc5cfc8-cd8a-11ee-95ba-87774f69a715/ceph-osd.11.log.gz 2024-02-17T12:20:13.431 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-02-17T12:20:13.431 INFO:teuthology.orchestra.run.smithi189.stderr:real 2m44.279s 2024-02-17T12:20:13.431 INFO:teuthology.orchestra.run.smithi189.stderr:user 4m15.402s 2024-02-17T12:20:13.431 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m8.593s 2024-02-17T12:20:13.432 INFO:tasks.cephadm:Archiving logs... 2024-02-17T12:20:13.433 DEBUG:teuthology.misc:Transferring archived files from smithi094:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122/remote/smithi094/log 2024-02-17T12:20:13.434 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-17T12:20:54.631 DEBUG:teuthology.misc:Transferring archived files from smithi125:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122/remote/smithi125/log 2024-02-17T12:20:54.632 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-17T12:21:34.449 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122/remote/smithi189/log 2024-02-17T12:21:34.450 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-17T12:22:21.986 INFO:tasks.cephadm:Removing cluster... 2024-02-17T12:22:21.987 DEBUG:teuthology.orchestra.run.smithi094:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 --force 2024-02-17T12:22:22.180 INFO:teuthology.orchestra.run.smithi094.stdout:Deleting cluster with fsid: 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T12:22:22.739 DEBUG:teuthology.orchestra.run.smithi125:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 --force 2024-02-17T12:22:22.929 INFO:teuthology.orchestra.run.smithi125.stdout:Deleting cluster with fsid: 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T12:22:23.464 DEBUG:teuthology.orchestra.run.smithi189:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 --force 2024-02-17T12:22:23.692 INFO:teuthology.orchestra.run.smithi189.stdout:Deleting cluster with fsid: 3bc5cfc8-cd8a-11ee-95ba-87774f69a715 2024-02-17T12:22:24.294 INFO:tasks.cephadm:Removing cephadm ... 2024-02-17T12:22:24.294 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-17T12:22:24.313 DEBUG:teuthology.orchestra.run.smithi125:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-17T12:22:24.331 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-17T12:22:24.346 INFO:tasks.cephadm:Teardown complete 2024-02-17T12:22:24.346 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-17T12:22:24.358 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-17T12:22:24.358 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-17T12:22:24.361 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-17T12:22:24.373 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-17T12:22:24.498 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-17T12:22:24.511 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-17T12:22:24.511 DEBUG:teuthology.orchestra.run.smithi094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-17T12:22:24.514 DEBUG:teuthology.orchestra.run.smithi125:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-17T12:22:24.516 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-17T12:22:24.529 INFO:teuthology.orchestra.run.smithi094.stderr:bash: line 1: ntpq: command not found 2024-02-17T12:22:24.529 INFO:teuthology.orchestra.run.smithi125.stderr:bash: line 1: ntpq: command not found 2024-02-17T12:22:24.530 INFO:teuthology.orchestra.run.smithi189.stderr:bash: line 1: ntpq: command not found 2024-02-17T12:22:24.534 INFO:teuthology.orchestra.run.smithi094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-17T12:22:24.534 INFO:teuthology.orchestra.run.smithi094.stdout:=============================================================================== 2024-02-17T12:22:24.534 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 38 +3116us[+3116us] +/- 111ms 2024-02-17T12:22:24.534 INFO:teuthology.orchestra.run.smithi094.stdout:^* hv02.front.sepia.ceph.com 3 6 377 38 -1017us[-1009us] +/- 56ms 2024-02-17T12:22:24.534 INFO:teuthology.orchestra.run.smithi094.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 -1009us[-1009us] +/- 91ms 2024-02-17T12:22:24.534 INFO:teuthology.orchestra.run.smithi094.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:22:24.573 INFO:teuthology.orchestra.run.smithi125.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-17T12:22:24.573 INFO:teuthology.orchestra.run.smithi125.stdout:=============================================================================== 2024-02-17T12:22:24.573 INFO:teuthology.orchestra.run.smithi125.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 +3148us[+3148us] +/- 111ms 2024-02-17T12:22:24.574 INFO:teuthology.orchestra.run.smithi125.stdout:^* hv02.front.sepia.ceph.com 3 6 377 41 -996us[ -992us] +/- 56ms 2024-02-17T12:22:24.574 INFO:teuthology.orchestra.run.smithi125.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -986us[ -986us] +/- 90ms 2024-02-17T12:22:24.574 INFO:teuthology.orchestra.run.smithi125.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:22:24.587 INFO:teuthology.orchestra.run.smithi189.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-17T12:22:24.587 INFO:teuthology.orchestra.run.smithi189.stdout:=============================================================================== 2024-02-17T12:22:24.587 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 +3143us[+3143us] +/- 111ms 2024-02-17T12:22:24.588 INFO:teuthology.orchestra.run.smithi189.stdout:^* hv02.front.sepia.ceph.com 3 6 377 44 -1018us[-1013us] +/- 56ms 2024-02-17T12:22:24.588 INFO:teuthology.orchestra.run.smithi189.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -976us[ -976us] +/- 91ms 2024-02-17T12:22:24.588 INFO:teuthology.orchestra.run.smithi189.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-17T12:22:24.599 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-17T12:22:24.610 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-17T12:22:24.611 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-17T12:22:24.621 DEBUG:teuthology.orchestra.run.smithi094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-17T12:22:24.767 DEBUG:teuthology.orchestra.run.smithi125:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-17T12:22:24.850 DEBUG:teuthology.orchestra.run.smithi189:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-17T12:22:24.939 DEBUG:teuthology.orchestra.run.smithi094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-17T12:22:24.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:22:24.973 DEBUG:teuthology.orchestra.run.smithi125:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-17T12:22:25.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:22:25.006 DEBUG:teuthology.orchestra.run.smithi189:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-17T12:22:25.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:22:25.036 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-17T12:22:25.049 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-17T12:22:25.098 INFO:teuthology.task.internal:Duration was 2533.072731 seconds 2024-02-17T12:22:25.098 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-17T12:22:25.160 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-17T12:22:25.160 DEBUG:teuthology.orchestra.run.smithi094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-17T12:22:25.163 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-17T12:22:25.166 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-17T12:22:25.242 INFO:teuthology.orchestra.run.smithi094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-17T12:22:25.255 INFO:teuthology.orchestra.run.smithi189.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-17T12:22:25.270 INFO:teuthology.orchestra.run.smithi125.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-17T12:22:25.744 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-17T12:22:25.744 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi094.front.sepia.ceph.com 2024-02-17T12:22:25.744 DEBUG:teuthology.orchestra.run.smithi094:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-17T12:22:25.767 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi125.front.sepia.ceph.com 2024-02-17T12:22:25.767 DEBUG:teuthology.orchestra.run.smithi125:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-17T12:22:25.790 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-02-17T12:22:25.790 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-17T12:22:25.818 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-17T12:22:25.819 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-17T12:22:25.821 DEBUG:teuthology.orchestra.run.smithi125:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-17T12:22:25.833 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-17T12:22:25.887 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-17T12:22:25.888 DEBUG:teuthology.orchestra.run.smithi094:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-17T12:22:25.892 DEBUG:teuthology.orchestra.run.smithi125:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-17T12:22:25.903 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-17T12:22:26.290 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-17T12:22:26.302 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-17T12:22:26.302 DEBUG:teuthology.orchestra.run.smithi094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-17T12:22:26.349 DEBUG:teuthology.orchestra.run.smithi125:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-17T12:22:26.389 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-17T12:22:26.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-17T12:22:26.440 DEBUG:teuthology.orchestra.run.smithi094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-17T12:22:26.443 DEBUG:teuthology.orchestra.run.smithi125:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-17T12:22:26.445 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-17T12:22:26.469 INFO:teuthology.orchestra.run.smithi094.stdout:kernel.core_pattern = core 2024-02-17T12:22:26.478 INFO:teuthology.orchestra.run.smithi125.stdout:kernel.core_pattern = core 2024-02-17T12:22:26.536 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-02-17T12:22:26.562 DEBUG:teuthology.orchestra.run.smithi094:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-17T12:22:26.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:22:26.577 DEBUG:teuthology.orchestra.run.smithi125:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-17T12:22:26.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:22:26.592 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-17T12:22:26.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-17T12:22:26.606 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-17T12:22:26.619 INFO:teuthology.task.internal:Transferring archived files... 2024-02-17T12:22:26.620 DEBUG:teuthology.misc:Transferring archived files from smithi094:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122/remote/smithi094 2024-02-17T12:22:26.620 DEBUG:teuthology.orchestra.run.smithi094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-17T12:22:26.666 DEBUG:teuthology.misc:Transferring archived files from smithi125:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122/remote/smithi125 2024-02-17T12:22:26.666 DEBUG:teuthology.orchestra.run.smithi125:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-17T12:22:26.706 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122/remote/smithi189 2024-02-17T12:22:26.707 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-17T12:22:26.746 INFO:teuthology.task.internal:Removing archive directory... 2024-02-17T12:22:26.746 DEBUG:teuthology.orchestra.run.smithi094:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-17T12:22:26.748 DEBUG:teuthology.orchestra.run.smithi125:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-17T12:22:26.750 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-17T12:22:26.796 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-17T12:22:26.812 INFO:teuthology.task.internal:Not uploading archives. 2024-02-17T12:22:26.812 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-17T12:22:26.843 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-17T12:22:26.843 DEBUG:teuthology.orchestra.run.smithi094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-17T12:22:26.845 DEBUG:teuthology.orchestra.run.smithi125:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-17T12:22:26.848 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-17T12:22:26.857 INFO:teuthology.orchestra.run.smithi094.stdout: 265915 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 12:22 /home/ubuntu/cephtest 2024-02-17T12:22:26.860 INFO:teuthology.orchestra.run.smithi125.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 12:22 /home/ubuntu/cephtest 2024-02-17T12:22:26.862 INFO:teuthology.orchestra.run.smithi189.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 12:22 /home/ubuntu/cephtest 2024-02-17T12:22:26.863 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-17T12:22:26.876 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-17T12:22:26.903 INFO:teuthology.nuke:Checking targets against current locks 2024-02-17T12:22:26.931 DEBUG:teuthology.nuke:shortname: smithi094 2024-02-17T12:22:26.931 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-17T12:22:26.956 DEBUG:teuthology.nuke:shortname: smithi125 2024-02-17T12:22:26.956 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-17T12:22:26.961 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi094.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 11:31:48.489394', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG6smY1OgeolRnBZxzg0cPkWaGtYzRTvBjeOAvk+Tg7hEfmIaXh6F9yZ0rlGIOCxRA/TDXBEJdoarl7ytQsTcKU='} 2024-02-17T12:22:26.981 DEBUG:teuthology.nuke:shortname: smithi189 2024-02-17T12:22:26.981 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-17T12:22:26.986 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi125.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 11:31:48.487684', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDlXkWaPWNtFn8/H+ch7nJ/biNFnxA6pCB6Cb7U4zEV6p7vu9ABQzXPtZbFiyNcU8jchLsjEuYjG+5N3aMp6Jzc='} 2024-02-17T12:22:27.012 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-16_20:13:25-fs-main-distro-default-smithi/7563122', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-17 11:31:48.490480', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNYrbB3c5Fp9foewg7kcNImMuQ/hpiCvDnGIYCbqCjSVXftEuoy3ssLVmWZr7yzeMAZllzl5M9UupWehnnHwTSA='} 2024-02-17T12:22:27.019 INFO:teuthology.orchestra.console.smithi094:Power off 2024-02-17T12:22:27.019 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-17T12:22:27.045 DEBUG:teuthology.orchestra.console.smithi094:power off output: Chassis Power Control: Down/Off 2024-02-17T12:22:27.045 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T12:22:27.059 INFO:teuthology.orchestra.console.smithi125:Power off 2024-02-17T12:22:27.060 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-17T12:22:27.074 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-02-17T12:22:27.084 DEBUG:teuthology.orchestra.console.smithi125:power off output: Chassis Power Control: Down/Off 2024-02-17T12:22:27.084 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T12:22:27.101 INFO:teuthology.orchestra.console.smithi189:Power off 2024-02-17T12:22:27.101 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-17T12:22:27.110 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2024-02-17T12:22:27.121 DEBUG:teuthology.orchestra.console.smithi189:power off output: Chassis Power Control: Down/Off 2024-02-17T12:22:27.121 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T12:22:27.147 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-02-17T12:22:31.075 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T12:22:31.111 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T12:22:31.148 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T12:22:31.186 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is on 2024-02-17T12:22:31.221 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is on 2024-02-17T12:22:31.258 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-02-17T12:22:35.191 DEBUG:teuthology.orchestra.console.smithi094:pexpect command: ipmitool -H smithi094.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T12:22:35.222 DEBUG:teuthology.orchestra.console.smithi125:pexpect command: ipmitool -H smithi125.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T12:22:35.259 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T12:22:35.302 DEBUG:teuthology.orchestra.console.smithi094:check power output: Chassis Power is off 2024-02-17T12:22:35.331 DEBUG:teuthology.orchestra.console.smithi125:check power output: Chassis Power is off 2024-02-17T12:22:35.369 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is on 2024-02-17T12:22:35.403 INFO:teuthology.orchestra.console.smithi094:Power off completed 2024-02-17T12:22:35.431 INFO:teuthology.orchestra.console.smithi125:Power off completed 2024-02-17T12:22:39.373 DEBUG:teuthology.orchestra.console.smithi189:pexpect command: ipmitool -H smithi189.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-17T12:22:39.483 DEBUG:teuthology.orchestra.console.smithi189:check power output: Chassis Power is off 2024-02-17T12:22:39.583 INFO:teuthology.orchestra.console.smithi189:Power off completed 2024-02-17T12:22:39.688 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} duration: 2533.0727305412292 failure_reason: 'The following counters failed to be set on mds daemons: {''mds.imported'', ''mds.exported''}' flavor: default owner: scheduled_yuriw@teuthology status: fail success: false 2024-02-17T12:22:39.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-17T12:22:39.765 INFO:teuthology.run:FAIL